Cairn

Design

Nine settled decisions.

Each one is downstream of a single fact: the agent doing the work can now write the record. When expressiveness and capture reliability conflict, capture wins.

The nine #

1 · Separate repo

One graph spans all projects; nodes carry project:. This is what makes the laptop/HPC split tractable — and it's why cross-project edges exist at all.

2 · Reference commits, never branches

Branches get merged and deleted, and the most valuable nodes here are exactly the dead ends git is designed to garbage-collect. A SHA is forever.

3 · One file per node, append-only

Never delete a node; change its status. Files never move, id is permanent. This is also what makes two writers non-colliding.

4 · Tiny schema

Every optional field is a field that stops getting filled in. Adding one is a real decision — it goes in lib.TYPES and the schema doc, or it doesn't happen.

5 · Status is an event log

history: gives the time axis for free, which is what makes the replay slider and the weekly digest work with nothing extra to maintain.

6 · Nothing is committed unreviewed

Draft the node, show it, then commit. An auto-populated graph nobody has read is the failure this project exists to avoid. Unprompted capture goes to inbox/ instead.

7 · Capture ≠ structure

inbox/ accepts unstructured junk from anywhere; a separate triage pass turns junk into nodes. Never require filing at capture time — that's how capture dies.

8 · Artifacts stay where they are

Nodes reference big outputs by host:/path. The repo stays small forever, so it syncs instantly and diffs stay readable.

9 · One graph, many writers, synced through git

No server, no auth, no real-time editing — but multi-writer was never the thing being refused. One file per node, append-only and never renamed, is what makes two people non-colliding, and git log --diff-filter=A already answers “whose node is this?” without a field for it. Add collaborators to the repo; don't shard the graph per project, or you lose the cross-project edges that are the hardest part to reconstruct.

Two of these have been tested by being wrong first. Rule 4 was broken once, on purpose, to add relates — and it took a written argument. Rule 6 is why sync --push commits only build/, even though committing nodes would be more convenient.

What this deliberately isn't #

  • Not an experiment tracker. W&B and MLflow own metrics and curves; the graph links out to them.
  • Not a replacement for project repos. Nothing heavy ever enters: code by repo + commit, big outputs by host:/path.
  • Not a general task manager — only research commitments.
  • No server, no auth, no real-time editing, no discussion threads. A static HTML file in a git repo.

And the next step is deliberately not more features. The failure mode for a project like this is building the whole system and then not using it.

Why this can exist now #

Every prior attempt died on the same rock: capture cost.

Renku does compute provenance but has no concept of a hypothesis or a dead end. W&B, MLflow and DVC track runs and artifacts, not ideas. The HCI work — Kery's Variolite and Verdant, Burrito, Hindsight — attacked exactly this and needed a discipline nobody sustains, because a graph at 60% coverage is worse than none: you can't trust a missing node to mean “not tried.”

The agent doing the work can now write the record. Every design decision here serves that one fact.

What about Graphify?

Graphify turns a codebase into a queryable knowledge graph — tree-sitter AST parsing, typed edges (calls, imports, inherits), Leiden community detection, an interactive force-directed graph.html, and a generated report naming the most-connected “god nodes”. It is good, it is fast, and it is orthogonal to this.

The distinction is the one its own commenters kept making: Graphify maps the current state of a codebase; Cairn maps the history of research decisions. Graphify does read WHY: and HACK: comments and ADRs as first-class nodes, so it captures rationale that exists in the repo — but a dead end leaves no comment, because the code was deleted. “We tried fitting published final Tg values as Tgp and the literature refutes it” is not in any AST, will never be in one, and is exactly the node that stops the idea being had twice.

Practically: run both. Graphify in a project repo to orient in the code, Cairn across projects to know what has been tried. Three of its ideas were worth taking, and two are in here:

  • The generated report as an entry point. build/report.md. Its “start here” section is Graphify's god-nodes idea; its “connections across projects” section is the surprising-connections idea, except the definition can be exact here rather than heuristic.
  • EXTRACTED vs INFERRED on every edge, so you always know what was read versus guessed. The analogue that matters here is contemporaneity — was this node written the day the work happened, or reconstructed six weeks later? — and it was already derivable from git, so it cost no field.
  • A git merge driver that union-merges the generated graph on parallel commits. Cairn's version of that problem is build/, and sync resolves it by regenerating rather than merging.

Not taken: force-directed layout (research history has a real time axis, and a force layout throws it away), and a vector store or MCP server for search — a few hundred nodes of text parse in well under a second.

Does it pay for itself? #

Cairn tracking Cairn. Measured, not estimated: per-turn usage from 40 agent transcripts — 29,797 assistant turns — over the same five weeks that produced the first 61 nodes.

0.45% of five weeks' token spend
to write the first 61 nodes
2.5 MB whole repo,
excluding git history
5.1k lines of python
+ one html template

Which means the question was posed wrong. Node-writing cost is a rounding error. The interesting quantity is where in the context distribution the work happens: turns above 300k context are 31% of turns but 52% of cost.

A memory file is paid per turn, not per node, because it sits in the context re-read on every one — a 6,000-word status doc costs about 1.8%, four times what writing those 61 nodes cost. The two differ in what survives, not in what they cost.

What would make me abandon this #

All three are nodes in the graph, and none is settled.

Does the convention hold without steering?

Two projects are now writing nodes as the work happens, rather than reconstructing them afterwards.

If the schema only survives when its author is in the room, it doesn't survive.

Does it work for someone who isn't a computational scientist?

The model is domain-neutral — a bench experiment validates today with no repo, no commit and no host.

The interface isn't neutral, and artifacts: host:/path is the first thing that will break.

Is it better than keeping a memory file current?

Not on cost — the two are the same order of magnitude.

The difference is that a memory file gets overwritten, and current-state representations garbage-collect dead ends. That needs a real test, not an argument.

What isn't in question is whether the code is valuable: a few thousand lines, rebuildable in a week by anyone who has read the schema. If something durable is here, it's the empirical claim about capture cost.

Three things the first backfill found #

The backfill was the first real test of the tooling, and using the result was the second. Between them:

  1. The pre-commit hook probed for the wrong thing. It looked for “a python3 that can import yaml”, which on the cluster is satisfied by a 3.6.8 too old to parse lib.py — so every commit was blocked with a SyntaxError reported as schema drift. Hence scripts/find_python.sh.
  2. Frontmatter did not round-trip. An unquoted comma inside a flow-style history note split the value and turned its tail into a key: note: past 1,000 systems silently became note: past 1 plus a junk key 000 systems. 17 of the 55 backfilled nodes were affected. The writer now quotes flow scalars properly, and the validator rejects unexpected keys in a history entry so the same class of corruption can't return quietly.
  3. Untyped edges lost information. Three findings could only be connected in prose. Hence relates — the one schema addition.

Two node types that wanted extra fields — milestones reaching for repo and artifacts — were made to use their bodies instead.