Getting Regional with help links

Today’s progress makes help file loading more streamlined going forward and also includes some post processing support for loaded help, which provides the visual enhancements needed to get the help looking the way we want it to. This includes marking text with regions as well as actually modifying the help buffer.

That last part took longer than it should have before I figured out the weirdness that was going on (which is not actually weird at all and actually what you should expect). Everything is all sorted out now though, so it’s back to smooth sailing.

The first change of note is that at the end of yesterday, the test command was manually loading and displaying help to verify that the routines for finding and modifying the contents of a help view were working as expected. That code has now been pulled out into it’s own method, with some extra intelligence.

There is a lot of state that hyperhelp is going to need to keep for any particular help view so that it can track what’s going on. Although there are various ways to try and handle this situation, saving data directly as a view setting is the method that I’ve chosen here. Such settings are always specific to the view that they have been applied to, and Sublime will persist them between sessions. That gives us everything we want without any extra work on our part.

If you looked at the code at the end of yesterday, you may have noticed that the help view had a couple of settings applied to it when help was displayed in it, _hh_pkg and _hh_file, which track the package that contains the help being displayed and the name of the help file, respectively. That was a bit of ground work for the code added today.

The new code for displaying help tries to be as intelligent as it can be about displaying the help, so as to do as little work as possible. This includes it doing it’s own check to see if there is already an existing help view or not. If there isn’t, it can just go ahead and display help like it did in the test code yesterday.

The extra logic comes into play when a help view already exists. In this case the loading code checks to see if the package and help file to be displayed are the same file that is already loaded, and if it is it can just focus the view and leave without doing anything further.

Also included is the idea of post processing on a loaded help buffer in order to make some changes to the help text as it originally appeared. This only has to be done when a new help file is loaded, and while not highly resource intensive it still takes non-zero time, so it’s a good idea to not do any extra work if it doesn’t need to be done.

A simple example of post processing is to find all of the link text that appears in the help view and apply an underlined style to it to help underscore (pun mildly intended) where all of the possible jumping off locations exist. This is currently always enabled, although it will probably become something that is controlled via a setting in a future code change so that people that don’t like that sort of thing can turn it off.

Doings this is extremely simple thanks to the Sublime API and is literally a single line of code (although I implemented it in two lines for better readability). Sublime has an API method called find_by_selector() that finds all of the sections of text in the buffer that match a particular scope. Since our syntax applies the scope to all link text, a single call is enough to find every link in the document. A call to add_regions() is all that’s needed to visually mark all of the links with underscores.

The more complicated post processing has to do with the handling of hidden anchor points. If you recall from yesterday, an anchor point is text wrapped in asterisks, such as *anchor* and will ultimately represent a navigation point in a help file; a link can jump to an anchor point in a file, and the user will also be able to move through the document by skipping between anchor points.

There may be cases in which you want a navigation point to appear in the document without it visually appearing as one, such as in a header or  at the start of a paragraph. For this reason we also support a hidden anchor, which works like a regular anchor but which contains pipe characters, such as *|hidden_anchor|*. The idea behind these kinds of anchors is that they work the same but are not visually distinct.

In order to achieve hiding these, hyperhelp needs to find all such anchors and modify the text in the buffer so that they no longer contain the *| prefix and |* suffix, which causes Sublime to see the anchors as regular text. Along the way we also need to store the text of the anchor and it’s position for later use, since once we modify the buffer there will be no way to tell where these anchors actually existed.

This starts off similar to the link code, by using find_by_selector() to find all of the hidden anchors. The list is always presented in document order and we’re modifying the text (specifically making it shorter), so it’s important to work from the bottom up so that the found positions still track with the contents of the buffer.

For each link we replace the entire construct with just the text in the middle, then save the anchor text and location into a list which is eventually applied as a setting on the view. Thus going forward even though we can’t tell where the anchors existed, we can look in the settings to find them.

In order to facilitate testing, the code is temporarily marking the hidden anchors with a colored region so that it’s easy to visualize where they are. After modifying the sample help file a little bit, the result of all of today’s code work is something that looks like the following screenshot.

Help region enhancements

Help region enhancements

As we can see here, all of the links are now underlined. The inversed blocks of text are the hidden anchors, which are colored with a filled region in a color identical to the color used to render regular visible anchors.

It was this code that gave me a little bit of a headache until I realized the very simple fact that I was overlooking. As seen in this screen shot, there are five hidden anchors. When they’re processed, we reverse the list and start from the bottom, removing the characters that mark each one as a hidden anchor. Then the location and the anchor text is saved and we go on to the next one.

This goes from the bottom up instead of from the top down because the locations that are found are only valid while the text remains in the same state. For example if you search for all of the hidden anchors and then delete the first word from the buffer, all of the positions you have are now wrong by four characters because the text in front of them was modified.

What I failed to take into account is that when storing the final locations of the anchors, the positions that I was saving were going to change as I moved upward through the document. Specifically, since we remove four characters for every hidden link we process, the first link in the document remains at the same position, but the second one is moved backwards by four characters, the third one by eight characters, and so on.

Conceptually I knew this was a problem but simply overlooked the fact that just because I’m processing the anchors from the bottom up doesn’t mean that the positions remain the same after the modifications happen.

One last thing to note here is that Sublime doesn’t persist assigned regions between sessions, so if you were to open a help file, then quit Sublime and start it again, the links are no longer underlined and the hidden anchors are once again hidden (which is a bummer for testing).

In order to resolve this situation, the help code detects when it’s being loaded and scans all of the windows to see if they have a help view, and applies a variant of the regular post processing to any help view it finds. All links become underlined, but when it comes to the hidden anchors they need to be added as regions based on the saved setting instead of by looking in the buffer. As an added benefit, this is a great independent test that the positions being stored are actually correct.

That brings today’s Devember coding session to a close. Now that there are links in the buffer and anchor points (of two kinds) tomorrow can proceed with the logic for actually following a link to somewhere inside the document as well as navigation between anchors and links in general.