These aren’t the resources you’re looking for

What started as the simple notion to finish off the last immediately planned help authoring function (adding a stub help system to an existing package) ended up being slightly more epic than I originally intended it to be. That’s not to say that the code for doing this was overly complex, but I bumped into something I’d heard mentioned in passing and spent some time trying to verify how best to proceed.

In the end I was victorious, though. Perhaps doubly so, if you count that if I had finished this as easily as I originally intended to, I would have had to sit and write some actual help finally. So, that was a pretty close call.

The premise behind this command is really quit simple in the broadest strokes; given a package, create a sample index.txt help file, a hyperhelp.json that has the minimal required keys and references the help file, then reload the help index information to activate the change, optionally also opening up the related files for immediate editing.

There are a couple of minimal snags in doing this. The first is that you don’t want to offer the command for a package that already has help, so this has to be careful not to clobber any existing files. From a UX perspective that also means that if there is a prompt for the package to add help for, it shouldn’t show you packages that already have help.

Another is that although the help system defines that the help index file always be named hyperhelp.json and that there be at a minimum a help file named index.txt, it is left to the package author what the document root should be inside the package, so before the stub files can be created, the user has to be prompted for that as well.

Apart from these, the operation is straight forward in that it needs to create the folder structure if it’s not there, then add the files. The real problem comes in the least likely of places, attempting to reload the help indexes.

Although in some cases you can access a file from a package by directly opening it from the package in the Packages folder, that is not always possible and is in fact not good in the general case. The reason for this is that it’s possible for a package to be installed as a zipped sublime-package file, and if that’s the case there is no file to open.

Sublime has some API calls to get around this, which include sublime.load_resource() to get the contents of a resource file and sublime.find_resources() to find them in the first place. The data for these API calls is discovered as Sublime starts, then kept up to date by having watchers check the contents of the file system to catch changes. This results in these calls being incredibly fast, and transparently able to work with files regardless of how the package is installed.

hyperhelp uses the find_resources() call to locate all of the available help index files, which is much faster and easier than first detecting every package, then looking through it’s entire contents to see if it has an index. It also uses load_resource() to load the index files and the help files themselves.

An issue arises in the rare case that a command does something that creates a new resource and then immediately tries to access it. Observation shows that the code in Sublime that handles the notifications that new resources have been added doesn’t trigger while a command is running. I’m not sure if there is something like a mutex locking it, or the notifications are handled by the main GUI thread, or what.

Whatever is going on, it adds up to trouble for this authoring command because the newly created help index file is not immediately visible to the command. This makes it much less straight forward to be able to immediately activate the new index, which is required for being able to use the existing authoring commands to open the appropriate files.

If you assume that once the current command ends and control returns to the main thread, the notifications are handled (or deferred changes become visible to that thread), then you could use sublime.set_timeout() to defer access to the resources until after the current command ends. I played briefly with this but it became clear quickly that the shortest possible timeout is not enough, which means that this takes an indeterminate amount of time.

For the time being I’m using the approach of a one second delay before refreshing, to give a bit of a time delay. This is coupled with asking the user if they want to open the new files or not (which is arguably a good idea anyway, and should at a minimum be a setting) and then imposing another delay.

This works well enough for now, although a better position would probably be to modify the code in the hyperhelp API to try and fall back to a local file if it can’t find the associated resource (and having the help index loader be able to take a local index file name instead of just a resource), so that for the purposes of authoring help things still work in the short term until the new resources get noticed by Sublime.

A less invasive method would be to spawn a short lived background thread that periodically queries to see if the resource is available and then kicks off the reload. This seems less helpful in that there will be some delay in which commands using the new help won’t work.

In any event, that’s a thought for some other day and not right now. Possibly that will have to wait until after Devember is over and the code gets split back into the separate repositories, as there are more interesting things to work on in the interim.