Yep, I've definitely gone down the wrong path with my current research project, which means I've probably wasted at least a day or two on it, which I find deeply frustrating.

As I've been working my way through Lisp In Small Pieces, I've been trying to push the boundaries of what the interpreter does in order to make it seem more, well, "modern" to me. The target language is Javascript, both in the interpreter and in the transpiler; I ultimately want to end up with something similar to Gazelle or Pixie, but with even more facility. The laundry list is very long.

The problem I have right now is Sourcemaps. Sourcemaps are a tool supported by all major browsers and Node/Iojs, that translate error conditions that occur in the running code into near-equivalent locations in the original source files. This was meant to support minimizing features like uglify or Google Closure, but it turns out to work just fine for transpilers.

The problem is three-fold: (1) the original Lisp syntax doesn't support the carriage of metadata along with processing the the source, (2) Lisp macros mangle the source in unhealthy ways, (3) including the metadata necessary to support Sourcemaps makes the reader unsuitable to use as a general-purpose list-processing (i.e. "LISP") utility for users.

I thought I had a solution using Javascript's non-primitives for String and Number; that would have let me carry the values forward successfully. Unfortunately, the Javascript non-primitives are not automatically self-evaluating; (new String("foo")) != (new String("foo")), so comparisons and utility at run-time are severely impacted. Expectations are not being met. I know why this is the case, I'm just not happy with it.

An alternative is to encode the reader to pass back the sourcemap as a second AST, and have the reader traverse that as needed to report on state of the compilation.

The problem is that this is a cross-cutting concern, an aspect-oriented problem. Tracking which branch of the AST you just went down, or how far along the linked list that represents that branch, is not easily instrumented in Javascript. I want the evaluate core to operate as McCarthy described in 1959, but if run in a different context (oh, look, a monad!) also returns me optional sourcemap details about the evaluation pass.

I'm going to spend some of today reading through other folks' solutions. The non-Lispy javascripts are no help; they have their own paradigms for ASTs that don't involve the self-modifying capacities of Lisp.