I’m going to use the word “abstraction” in two different ways, but bear with me for a moment and consider this:

  1. Every startup is based on an insightful abstraction of a complex idea.
  2. Every software abstraction is leaky in some critical way.

Uber is an abstraction of taxis. How do we abstract “what a taxi is” using the higher-level technologies we have now? Amazon is an abstraction of mail-order sales, which have been around since Sears pioneered them in 1897. Google is an abstraction of card catalogs and yellow pages. And so forth.

In programming, an abstraction is a higher-level description of a process or mechanism that is designed to hide some level of complexity behind a simple set of controls. The menu on a word processor hides the complexity of the software behind it. The API we use to “log in with Facebook” or “log in with Google” hides the complexity of securing your authorization across multiple websites and applications while revealing your identity to advertisers.

We have a saying in software, “All non-trivial abstractions are leaky.” What this means is that all important abstractions can’t really hide what they do; if an API or programming library does something significant, then the programmer who uses them has to know at least some of the underlying details in order to use it well.

Many years ago when I worked for CompuServe, my boss used to joke that I had a superpower: I could look at almost any system and, with a minimum amount of exposure, explain in exceptional detail the underlying data structures and basic algorithms in use. In abstraction terms, I have X-ray vision for leaks; I could see where the pieces were fit together.

Like a lot of people nowadays, I’m besotted with Roam, a note-taking application. I have an Emacs Org-mode folder with, literally, thousands of notes, and I’ve been tinkering with my own note-management app for, like, ever. I’ve downloaded a couple of “Roam clones” from Github, and none of them really worked for me.

And then I watched the How to Use Roam videos on You-tube, and I spotted the leaky abstraction. I’m gonna show it to you right now. Here:

Do you see it? This is a screenshot from the “how to embed a note in another note” section, and the leak is in the upper right hand corner. It’s an obvious abstraction, once you see it, but it takes a second or two to recognize what you’re looking at. It’s that symbol, Qm2N2Uapn. That’s the leak: Every note is its own entry in the database. Once I saw it, all of the algorithm came apart in my head almost instantly. Most clones think in terms like Evernote: “Pages have notes.” But no, that’s not the abstraction at all. The abstraction is simpler than that: “Notes exist, have unique identities, and may have content which includes links to other notes. Notes may have child notes. A page is a note with a title that can be linked to.” That’s the whole abstraction on the server. Clients impose semantic meaning on pages by giving them “page” views, and queries to the API will almost always start with “a page,” but that’s almost irrelevant to the core functionality of “big notes have small notes upon their backs to bite ’em, and small notes have even smaller notes, and so on ad infinitum.”

Implementation is a slightly harder beast, but it’s really no more than getting the CRUD (Create, Retrieve, Update, and Delete) right on top of a few simple relationships: Pages, Notes, and Note->Note relationships. Notes have types, you need a very good autosuggest toolkit for the Page table, and you’ll need a custom editor for your front-end, but that’s a different engineering challenge from getting the back-end correct.

In fact, it occurs to me that this is a fantastic place to use RDFa as a way to make clear what operation you’re performing. Hmm…