As I've been working on my own little Zettelkasten engine, I've realized that there are exactly two operations that matter more than any other. The client will handle many of the details of showing notes and note collections (called "boxes", since Zettelkasten literally means "note box"), but the server needs to do exactly two things exactly right every time:
- ingest a note
- return a box when requested
Ingest a Note
Notes are completely free-form, at least from the user's point of view. They're just markdown documents which the user creates and then drops into the system. The user may specify which box to drop the note into, and the user may specify which notes to place this note between, or the user may specify that this note goes under another note. Obviously, when the user creates the note we know (a) what time it is, (b) who the user is, and (c) what tool the user used.
Notes that don't have a specified destination get shoved into the top level of the diary: a box labeled with the date the note was made.
If the client does specify a relationship with other notes, we'd better check that those notes exist first.
Here's the thing: in the content, the user is going to be creating one of several things: urls to external resources (which should periodically be checked, like bookmarks, for various HTTP 400 states), links to local resources (which I would like, in some future world, to include in the scan for text and metadata), and references to other boxes in the system.
When you add a note, it has to be parsed and the references have to be unpacked. Once you have all the references unpacked, you must make sure that every single one of those boxes exists-- and you have to generate those boxes if they don't.
So ingesting a note is hard! There's validation of the request-to-place, then there's making room in the box, which using SQL means shenanigans to describe the location of the note in relation to other notes. And then there's validation and generation of the boxes the note refers to.
In advanced systems like Roam Research, there's a third step: unpacking every possible window of legitimate terms in the content and corresponding those terms with every single box label, in order to help the user find notes that may be related to other boxes.
Opening a Box
Opening a box is the other complicated process. When the user opens a box, they're going to take out the cards. In computer terms, this means that a box and all of its notes are delivered to the client in one piece. But more than just that goes with the box.
First, every reference to the requested box is included in the payload, so that the user can see which notes they've added that think about, talk about, or refer to the contents of this box. This means that, again is SQL terms, we have to look up the box-to-note relationship described, find that note in its current box, and create a list of links all the way to the current box label for that note, so that the context of the note referring to the requested box is available to the client. Secondly, the alternative collection of unmarked references must also be provided, unless the user has marked a suspected unmarked reference as erroneous.
This doesn't seem quite so complicated, but I'll let you in on
something: in SQL, the function to collect every note associated with a
box, in the tree-like order a Zettlekasten
permits, is
over 80 lines long. That's a single SELECT statement.
Relationships
Inside a box, notes relate to other notes mostly by position. You can open or close subnote collections, arrange them in an outline, move them, copy them, and of course delete them. A good client will permit drag-and-drop, which will of course be a major goal of the project, as well as excellent in-line editing. But it was these two functions above that took the most work, at least server-side.
The client, well... that'll be a whole 'nother story.