Nerdy internals of an Apple text editor 👨🏻‍🔧

In this article, we’ll dive into the details of the way Paper functions as a TextView-based text editor for Apple platforms.

The first article was just a warm-up — here is where we get to truly geek out! 🤓

Before we start, I’ll add that for the time being Paper is built on the older TextKit 1 framework, so the article is relative to TextKit 1. That said, all of the concepts, abstractions, and principles discussed here still exist in TextKit 2, either unchanged or under a better API.

Text view

To understand how text editing works in native Apple text editors, we first need to discuss the centerpiece of the whole system — the TextView class. Technically, NSTextView and UITextView have their differences, but the API is similar enough that we can treat them as a single TextView class. I will highlight the differences where necessary.

TextView is a massive component that only grows in complexity with each release of respective operating systems. The TextEdit app consists almost entirely of a single TextView. When a single class can be used to build an entire app — you know it’s a beast.

Luckily, TextView is not just one huge pile of code. Apple tried to subdivide it into a bunch of layers — each represented by a flagship class. The layers build on top of each other to create a text editing experience.

Attributes

With the general structure of TextView out of the way, let’s zoom in on NSTextStorage, or rather its parent class NSAttributedString, as it is the foundation of rich text editing in Apple’s frameworks.

NSAttributedString consists of two parts:

1. A regular text string.
2. String-value pairs of attributes attached to ranges of text within the string.

Attributes are used mostly for styling purposes, but nothing restricts you from assigning custom string-value pairs for your own needs.

To get started, let’s make an NSAttributedString via the API:

Attributes.m

NSMutableAttributedString *string = [NSMutableAttributedString.alloc
  initWithString:@"The quick brown fox jumps over the lazy dog."];

NSMutableParagraphStyle *style = NSMutableParagraphStyle.new;
style.firstLineHeadIndent = 30.0;

[string addAttribute:NSParagraphStyleAttributeName
               value:style
               range:NSMakeRange(0, string.length)];
[string addAttribute:NSFontAttributeName
               value:[NSFont systemFontOfSize:25.0]
               range:NSMakeRange(0, string.length)];

[string addAttribute:NSForegroundColorAttributeName
               value:NSColor.brownColor
               range:NSMakeRange(10, 5)];

[string addAttribute:NSFontAttributeName
               value:[NSFont boldSystemFontOfSize:25.0]
               range:NSMakeRange(20, 5)];

[string addAttribute:NSBackgroundColorAttributeName
               value:NSColor.lightGrayColor
               range:NSMakeRange(26, 4)];

[string addAttribute:NSUnderlineStyleAttributeName
               value:@(NSUnderlineStyleSingle)
               range:NSMakeRange(35, 4)];
[string addAttribute:NSFontAttributeName
               value:[NSFontManager.sharedFontManager
                      convertFont:[NSFont boldSystemFontOfSize:25.0]
                      toHaveTrait:NSFontItalicTrait]
               range:NSMakeRange(35, 4)];

NSRange is a structure consisting of a location and a length. NSMakeRange(10,5) means a range of 5 characters starting from position 10, or in other words, an inclusive range between positions 10 and 14. In case different ranges define the same attribute under the same position then the last applied range takes precedence. In the example above, the bold and italic fonts overwrite the default font that is applied to the whole string.

This code can be easily visualized in TextEdit as it is pretty much an NSTextView with some buttons.

The second big part of the API is dedicated to checking what attributes are applied to what ranges. The API itself is quite peculiar. A lot of thought has gone into making it fast and efficient, but as a result, the usage can be a bit of a pain.

For instance, if you want to check whether a certain attribute exists at a certain position you would use this method:

Attributes.m

id value = [string attribute:NSFontAttributeName
                     atIndex:6
              effectiveRange:nil];

If the value is nil, then it does not exist. Otherwise, it is the value of the attribute which in this case is a NSFont/UIFont object. So this method can be used both to query the value and to check the existence of the attribute.

But it gets better. You can pass a pointer to the NSRange structure as the last argument (the good old C technique to return multiple values from a single function call):

Attributes.m

NSRange effectiveRange;
id value = [string attribute:NSFontAttributeName
                     atIndex:6
              effectiveRange:&effectiveRange];

And it will return either:

• The range of the continuous span of the same attribute with the same value.
• Or the range of the gap where the attribute is absent.

Though not exactly… You see the effectiveRange here is not what you think it is. Quoting the documentation:

The range isn’t necessarily the maximum range covered by the attribute, and its extent is implementation-dependent.

In other words, it could be the correct maximum range… but it also might not be.

“Ahh — I just love having a bit of non-determinism in my code!”

To get the guaranteed maximum range you need to use a different method.

Attributes.m

NSRange effectiveRange;
id value = [string attribute:NSFontAttributeName
                     atIndex:6
       longestEffectiveRange:&effectiveRange
                     inRange:NSMakeRange(0, string.length)];

I suppose, this separation is done to make the checking of the attribute existence faster with the former method as the latter one probably needs to do some range merging to figure out the longest range when multiple ranges overlap. Still — how the effectiveRange in the former method is even useful? 🤷🏼‍♂️

The same pair of methods exist to query an NSDictionary of all the attributes at a position and the effectiveRange for which this unique combination of attributes spans.

Attributes.m

NSRange effectiveRange;
NSDictionary<NSAttributedStringKey, id> attributes =

  [string attributesAtIndex:6
      longestEffectiveRange:&effectiveRange
                    inRange:NSMakeRange(0, string.length)];

Finally, there is a convenience method to iterate over attributes within a range. With the longest constant name that ever existed for specifying which mode of attribute range inspection you prefer.

Attributes.m

[string enumerateAttribute:NSFontAttributeName
                   inRange:NSMakeRange(0, string.length)
  options:NSAttributedStringEnumerationLongestEffectiveRangeNotRequired
  usingBlock:^(id value, NSRange range, BOOL *stop) {
  // do something
}];

Styling

With the foundational knowledge behind us, it’s time to discuss how the syntax highlighting and text styling work in Paper.

As mentioned before, styling means applying special framework-defined attributes to ranges of text. In addition to them, Paper also uses custom attributes to identify the structure of the text before styling it. Here’s the breakdown:

1. Meta attributes

   • Defined by the Markdown parser to identify individual parts of the Markdown syntax.
   • These are custom string-value pairs used purely for semantics.
   • They do not influence the visual look of the text.
   • You can think of them as a simplified AST for Markdown.

2. Styling attributes

   • The visual attributes applied on top of the parts marked by meta attributes.
   • These are built-in string-value pairs defined by AppKit and UIKit.

The attributes are kept in sync with:

• The Markdown text in NSTextStorage that changes due to user input.
• The text-affecting settings that change as the user adjusts them from various menu items, sliders, and gestures.

Technically, we can identify three types of events that trigger this attribute update process:

1. Document opened — full update of meta attributes and styling attributes.
2. Text changed — partial update of meta attributes and styling attributes in the affected part. Most of the time only in the edited text. Sometimes in the whole paragraph. More on that in the next chapter.
3. Setting changed — full update of styling attributes but not meta attributes.

In every update there is a well-defined sequence of steps:

1. Start the text editing transaction

   • Without a transaction, every attribute change would trigger an expensive layout recalc by the NSLayoutManager. Instead, we want to batch all the changes and re-layout only once in step [4.].

2. Parse the Markdown structure

   • This is where the Markdown string is broken down into pieces denoted by the meta attributes.
   • This step is skipped for setting change since the Markdown structure does not change in this case.

3. Update layout-affecting attributes

   • The first batch of styling attributes.
   • This is every visual attribute that can influence the position or size of the glyphs in the text view.
4. End the text editing transaction

5. Update decorative attributes

   • The second batch of styling attributes.
   • The decorative attributes (or rendering attributes in Apple’s terminology) are applied outside the transaction. The reason is simple — they don’t affect the layout, so updating them is not expensive. And they are not even aware of the transaction since they live in the NSLayoutManager itself, not in NSTextStorage.

The most important attribute of the layout-affecting ones is NSParagraphStyle. It defines the bulk of the values that influence the layout of the lines and paragraphs.

The last chunk of attributes that participate in the styling process are the typing attributes. They are tied to the attributes at the position preceding the caret (for empty selection) or to the one at the start of the selection (for non-empty selection). Once you type a character, the typing attributes are assigned to the newly inserted text automatically. In a Markdown editor, they are not that important as the styling is derived entirely from the Markdown syntax, but they are crucial for rich text editors where the styles stick to the caret until you turn them off or move the caret to a new location. Despite being a Markdown editor, Paper does have a rich text editing experience called the Preview Mode. In this mode, the editor behaves just like a rich text editor with toggleable typing attributes being highlighted, for example, on the toolbar in the iOS app.

Performance

The separation of meta, layout, and decorative attributes plays nicely into keeping certain editor changes fast. For instance, toggling between light and dark modes requires updating only decorative attributes which is very fast as it does not trigger the layout. Setting changes such as text size adjustments, though require a re-layout of the whole document, is still reasonably fast compared to doing that plus a full re-parse of the Markdown structure.

That said, the most crucial performance piece of any text editor is undoubtedly the typing speed. The bad news is that due to how Markdown works, any text change has the potential to affect the styling of the whole paragraph.

Thus the logical thing to do is to re-parse and re-style the whole paragraph on every keystroke. The problem with that is while this is technically the most correct approach, it can slow down the editing for longer paragraphs. At the same time, if you’re simply typing out a long sentence, the Markdown structure does not change. There is really no need to re-style everything all the time for those simple typing scenarios.

So to make typing snappier, I’ve built an algorithm that looks at the next character being typed as well as what characters are around it. The gist of the logic is that if you’re typing a special Markdown symbol, or the location of the edit is surrounded by one, then you should update the whole paragraph, otherwise you can simply rely on the typing attributes. It’s a simple algorithm that does marvels for the speed of the editor in the majority of typing situations.

The only nasty exception to the above is when you have code blocks in the document. Code blocks are the only multi-paragraph Markdown constructs in Paper. A keystroke has the potential to re-style the whole document.

For now, I decided to ignore code blocks in documents beyond a certain character limit. It keeps the editor fast for the majority of users who don’t care about code, at the same time making Paper more useful for dev-adjacent audiences.

The final technique that I use to speed things up is to cache every complex value object in the string-value attribute pair.

• NSFont/UIFont
• NSColor/UIColor
• NSParagraphStyle

They are being re-assigned on every keystroke and never change unless a text-affecting setting is changed, so it makes sense to reuse them instead of creating new instances every time.

Meta attributes

Besides the highlighting logic, meta attributes play a crucial role in various features that need to know about the structure of the text.

Formatting shortcuts

• Toggling styles on a selected piece of Markdown text requires detailed information about the existing Markdown styles inside the selection.
• If the selection completely encloses the same style, then the style is removed.
• If the selection does not contain the same style, then the style is added.
• If the selection partially encloses the same style, then the style is moved to the selection.
• You also need to be careful not to mix the styles that cannot be mixed. The conflicting styles need to be removed first, before a new style can be added. For example, styles that define the type of the paragraph such as heading and blockquote cannot be mixed.

Jumping between chapters

• Paper has a feature that allows you to jump to the previous or the next edge of the chapter.
• Meta attributes help to locate the headings relative to the position of the caret.

Outline

• The outline feature relies on being able to traverse every heading.
• Pressing on the item in the outline moves the caret to that chapter.

Rearranging chapters

• Paper also has a feature that allows rearranging chapters in the outline.

Converting formats

• Converting the Markdown content to RTF, HTML, and DOCX relies on knowing the structure of the text.
• Since Paper does not include any external libraries, having a pre-parsed model of the text allows me to traverse the structure, building the respective output format in the process.

TextFormats.m

- (NSString *)toHtml:(NSMutableAttributedString *)string {
  [self encloseInHtmlTags:string
                         :MdStrongAttributeName
                         :@{
    MdStrong: @[ @"<strong>", @"</strong>" ]
  }];
  [self encloseInHtmlTags:string
                         :MdEmphasisAttributeName
                         :@{
    MdEmphasis: @[ @"<em>", @"</em>" ]
  }];
  [self encloseInHtmlTags:string
                         :MdUnderlineAttributeName
                         :@{
    MdUnderline: @[ @"<u>", @"</u>" ]
  }];
  [self encloseInHtmlTags:string
                         :MdStrikethroughAttributeName
                         :@{
    MdStrikethrough: @[ @"<s>", @"</s>" ]
  }];
  [self encloseInHtmlTags:string
                         :MdHighlightAttributeName
                         :@{
    MdHighlight: @[ @"<mark>", @"</mark>" ]
  }];
  [self encloseInHtmlTags:string
                         :MdCodeAttributeName
                         :@{
    MdCode: @[ @"<code>", @"</code>" ]
  }];
  [self encloseInHtmlTags:string
                         :MdHeadingAttributeName
                         :@{
    MdHeading1: @[ @"<h1>", @"</h1>" ],
    MdHeading2: @[ @"<h2>", @"</h2>" ],
    MdHeading3: @[ @"<h3>", @"</h3>" ],
    MdHeading4: @[ @"<h4>", @"</h4>" ],
    MdHeading5: @[ @"<h5>", @"</h5>" ],
    MdHeading6: @[ @"<h6>", @"</h6>" ]
  }];
  [self encloseInHtmlTags:string
                         :ParagraphAttributeName
                         :@{
    Paragraph: @[ @"<p>", @"</p>" ]
  }];

  [self encloseInBlockquoteHtmlTags:string];
  [self encloseInListHtmlTags:string];
  [self transformFootnotesForHtml:string];
  [self deleteCharactersWithAttributes:string :MetaAttributes.tags];
  [self insertHtmlBreaksOnEmptyLines:string];

  return string;
}

Text container math

The most important rule for the text container is to maintain the preferred line length, dividing the remaining space between side insets.

There are however trickier cases where you need to fake the symmetry. Like when the heading tags are placed outside of the regular flow of text. The text container is shifted to the left and the paragraphs are indented with NSParagraphStyle.

While there is enough space, it tries to keep the margins visually symmetrical. If there is no extra space left, then it breaks the symmetry in favor of keeping the specified line length. But only while there is padding remaining on the right side. When there is no padding left, the minimum margins take precedence over keeping the line length to its preferred width.

You can achieve this gradual collapsing with a combination of min and max functions. It takes a second or two to get your head around the math, but once you do, it feels quite elegant in my opinion. I love this kind of simple mathy code that leads to beautiful visual results.

MaTvTextContainerModule.m

- (CGFloat)leftInset {
  return (self.availableInsetWidth - fmin(
    self.availableInsetWidth - self.totalMinInset,
    self.leftPadding
  )) / 2.0;
}

- (CGFloat)rightInset {
  return self.availableInsetWidth - self.leftInset;
}

- (CGFloat)availableInsetWidth {
  return self.availableWidth - self.textContainerWidth;
}

- (CGFloat)textContainerWidth {
  return fmin(
    self.maxContentWidth,
    self.availableWidth - self.totalMinInset
  );
}

- (CGFloat)maxContentWidth {
  return self.lineLength * self.characterWidth + self.leftPadding;
}

- (CGFloat)availableWidth {
  return CGRectGetWidth(self.clipView.bounds);
}

- (CGFloat)totalMinInset {
  return self.minInset * 2.0;
}

- (CGFloat)minInset {
  return
    CGRectGetMinX(self.window.titlebarButtonGroupBoundingRect_) +
    CGRectGetMaxX(self.window.titlebarButtonGroupBoundingRect_);
}

- (CGFloat)leftPadding {
  return [@"### " sizeWithAttributes:@{
    NSFontAttributeName: Font.body
  }].width;
}


Selection anchoring

Text selection always has an anchor point. It’s something we are so used to that we never stop to think about.

On the Mac, we click and drag to select the text and we instinctively know that the selection will increase when dragging to the right and decrease when dragging to the left. But only until we hit the point of the click. Then the opposite happens.

On iOS the selection is a bit more interactive. We can drag one edge and then the other one becomes the anchor, and vice versa.

The same logic applies when we extend the selection with the keyboard. Hold the Option key plus a left or a right arrow and you can jump between the edges of the words. Do the same while holding the Shift key, in addition to the Option key, and you can select with word increments. And again — it remembers where you started.

It even works naturally when you first click and drag and then continue extending or shrinking the selection with the keyboard. The initial point of the click remains the anchor.

Selection affinity

Another fascinating concept of text editing that you most probably don’t know about is selection affinity. Quoting Apple’s documentation:

Selection affinity determines whether, for instance, the insertion point appears after the last character on a line or before the first character on the following line in cases where text wraps across line boundaries.

My guess is you still have no clue what it means, so let’s see it in action.

Pay attention to the screencast below. When I move the caret with the arrow keys, it simply switches the lines when moving around the wrapping point denoted by the space character. However, if I move the caret to the end of the line with the shortcut, it attaches itself to the right side of the wrapping space while staying on the same line.

There are also other instances where the TextView decides to play this trick. It’s a tiny detail and sort of makes sense when you think about it, but quite hard to actually notice.

Undo coalescing

Yet another important mechanism of text editors that often becomes second nature to us is undo coalescing.

It’s an algorithm that decides when is the right time to group the typed text into an undoable action. Different apps have different strategies, and depending on your favorite text editor, you might find the strategies of other editors annoying or unnatural.

TextView seems to use a surprisingly trivial algorithm. It groups together everything that is typed without being interrupted by other actions. You can take a break, stare at the blinking caret for a bit, and then continue typing — it will still fold everything before and after the mental break into a single undo. Even line breaks don’t break the coalescing.

Other editors can have different ideas about the optimal way of turning what you have typed into an undo stack. Sublime, for instance, prefers to group separate words into undoes as you type. In case you’re typing out a longer word, it will break the long word into smaller undoes. My hunch is that with every new word it starts a timer. If the word ends before the timer runs out, it groups the whole word into a single undo. If the timer runs out, it groups the typed chunk and starts a new timer.

There’s really no right or wrong way of doing this, and that’s why it’s so fun to compare the various approaches.

Uniform Type Identifiers

The last chapter will focus on cross-app data exchange, but first, we need to discuss the system that underpins it — the UTIs. It’s a hierarchical system where data types conform to (inherit from) parent data types.

• public.* types are defined by Apple. They identify the widely accepted formats such as public.html and public.jpeg.
• Developers can create their own identifiers using the reverse domain naming scheme to avoid collisions.

The benefit of the hierarchical system is that, for example, if your app can view any text format then you don’t need to list all of them — you can just say that it works with public.text. And indeed, Paper declares that it can open any text file, and although you won’t get any highlighting, you can still open .html, .rtf, or any other text format.

When exchanging data via a programmatic interface such as the clipboard, UTIs can be used directly. Files however are a bit trickier. File is a cross-platform concept and de-facto identifiers for files in the cross-platform realm are file extensions. Even if Apple would redo their systems to rely on some file-level UTI metadata field instead of the file extension (and it appears they have), other systems would not know anything about it. So to stay compatible, every UTI can define one or more file extensions that are associated with it.

Now, most of the time you work with either public UTIs or private ones that you’ve created specifically for your app. Things are relatively straightforward in these scenarios. The harder case is when you have a format that’s widely accepted, but not defined by Apple. This is exactly the case with Markdown. I will explain some of the annoying edge cases with these semi-public UTIs in the next chapter.

Pasteboard

UTIs transition nicely into the topic of cross-app exchange driven primarily by the clipboard, or in Apple’s technical terms — the pasteboard.

The pasteboard is nothing more than a dictionary where UTIs are mapped to serialized data — in either textual or binary format. In fact, using the Clipboard Viewer from Additional Tools for Xcode you can inspect the contents of the pasteboard in real time.

As you can see, a single copy action writes multiple representations of the same data at once (for backward compatibility some apps also write legacy non-UTI identifiers such as NeXT Rich Text Format v1.0 pasteboard type). That’s how, for instance, if you copy from Pages and paste it into MarkEdit — you get just the text, but if you paste it into TextEdit — you get the whole shebang.

As a general rule, editors pick whatever is the richest format they can handle. Some apps provide ways to force a specific format to be used. For example, a common menu item in the Edit menu of rich text editors is Paste and Match Style or Paste as Plain Text. It tells the app to use the plain text format from the pasteboard. The styles applied to the pasted text are usually taken from the typing attributes.

A fun fact is that drag and drop is also powered by the pasteboard, but a different one. The standard one is called the general pasteboard and it’s used for copy-paste. You can even create custom ones for bespoke cross-app interactions.

Another fun fact is that RTF is basically the serialized form of NSAttributedString. Or vice versa, NSAttributedString is the programmatic interface for RTF.

Rtf.m

NSAttributedString *string = [NSAttributedString.alloc initWithString:
  @"The quick brown fox jumps over the lazy dog."];
NSData *data = [string dataFromRange:NSMakeRange(0, string.length)
                     documentAttributes:@{
  NSDocumentTypeDocumentOption: NSRTFTextDocumentType
} error:nil];

// {\rtf1\ansi\ansicpg1252\cocoartf2759…
NSLog(@"%@", [NSString.alloc initWithData:data]);

This means that TextView is out-of-the-box compatible with the pasteboard since it works on top of NSTextStorage — the child class of NSAttributedString. No extra coding is needed to copy the contents to the pasteboard.

Now, as I mentioned in the last chapter, this is all great for public UTIs. But what about semi-public ones like Markdown? From my experience, the cross-app exchange is a mixed bag…

Imagine you want to copy from one Markdown editor and paste it into another one. Let’s say both have implemented the standard protocol to export formats with various levels of richness and to import the richest format given. Copying from the first editor exports Markdown as public.text and the rich text representation as public.rtf. When pasting to the second editor, it will pick public.rtf instead of the native Markdown format since there is no indication that the text is indeed Markdown. You end up with this weird double conversion that leads to all sorts of small formatting issues, such as extra newlines due to slight variations in the way Markdown↔RTF translation works in both apps, as well as just fundamental styling differences between Markdown and RTF. For the user it is obvious — “I copy Markdown from here and paste it here — it should just copy 1:1”, but under the hood there is a lot of needless conversion.

For this to work nicely, both apps should magically agree to export the net.daringfireball.markdown UTI and prefer it over public.rtf. If only one of the apps does it — it won’t make a difference. Paper tried to be a good citizen by exporting the Markdown UTI, but none of the other apps seem to prefer it over rich text. In addition to that, Pages has a weird behavior where it does prefer net.daringfireball.markdown over public.rtf, but in doing so it just inserts the raw Markdown string as is without converting it to rich text (why-y-y??? 😫). For this reason, I had to drop the Markdown UTI.

“But why export RTF at all? Markdown is all about plain text — drop RTF and problem solved” — you might think. Well, that’s true, but I want to provide a seamless copy-paste experience from Paper to rich text editors. And being a good OS citizen, you should provide many formats that represent the copied data, so that the receiving application could pick the richest one it can handle. In Paper, you can copy the Markdown text from the editor and paste it into the Mail app, and it would paste as nicely formatted rich text, not as some variant of Markdown. This is a great experience in my opinion. The only problem is that it often leads to less-than-ideal UX in other cases.

Another feature closely related to the pasteboard is sharing on iOS. It’s quite similar to copy-paste, only with a bit of UI on top. Your app exports data in various formats and the receiving app decides what format it wants to grab. Strangely enough, UTIs are not used to identify the data (well actually they kind of are through some bizarre scripting language in a config file 😱). Rather, classes such as NSAttributedString, NSURL, and UIImage are directly used to represent the type. Unlike the pasteboard that applies to all apps automatically, the sharing feature on iOS requires apps to explicitly opt-in to be present in that top row of apps by providing a share extension with a custom UI.

That’s it for now

Check out the first article if you haven’t already. It has a lot more tidbits about the app and the development process.

And if you want to read even more from me, I’ve written a small (unrelated) article about the native Apple writing app ecosystem.