Skip to content

Store and state

The store is the source of truth. Linear, FINDINGS.md, the daemon's JSON and the drawer are views of it; the loop, the supervisor and the serve daemon's action endpoints write it, all through the store API. It is one SQLite file per project in WAL mode, at ~/.holophyte/<slug>/store.db, with a versioned schema (PRAGMA user_version, currently 56) and forward-only migrations. A build that opens a store stamped newer than it understands refuses and exits. Every connection, writable or read-only, waits store.schema.BUSY_TIMEOUT_S (30 s) for another writer's lock before raising database is locked, so the loop and its supervisor contend on one file without either dying.

KO-522 adds evidenceStates to the JSON in runs.ticketSnapshot, parsed from tickets.body at claim time. No SQL schema migration is needed. Older snapshots without the key compare as an empty list; adding evidence after a claim is contract drift.

Conventions

A new table or column follows the ones already there:

  • camelCase table and column names (reviewRounds, runEvents, activeRunId), one casing across the whole file.
  • List fields are JSON text. SQLite has no array type, so a list of strings or objects is stored as a JSON document defaulting to '[]'.
  • Optional fields are nullable; everything else is NOT NULL.
  • Union types are CHECK constraints, so an unknown status, phase or verdict is refused by the database, not by a caller who remembered to look. store/enums.py holds each vocabulary and renders its CHECK.
  • Rows are keyed by a synthetic id INTEGER PRIMARY KEY, which other tables reference by that id.

Tables

Table One row per Written by Notes
projects project loop activeRunId is a legacy column, neither asserted nor written since the lease moved to tickets
tickets Linear issue the loop has mirrored loop activeRunId is the lease: held by one run at a time, renewed by that run's heartbeat; status machine below; blockedQuestion when parked for a human; the contract snapshot the merge gate compares against
ticketCriteria version of a ticket criterion ticket mirror, migration backfill, steer amendment append-only; newest id per (ticketId, criterion) whose nullable steerNoteId does not name a withdrawn note is current; criterion renders as C1, C2, … and is never reassigned; position is 1-based or NULL when retired; text excludes a leading bold id; witnessKind ∈ test, verify, reviewer_read, nullable witnessTarget; status ∈ open, retired; source ∈ filed, backfill, amendment; at records the version time; migration backfills only open tickets
runs attempt at a ticket loop, supervisor (end only) phase machine below; lastHeartbeat, timeBoxMs, outcome, outcomeReason, outcomeClass (work or infra), resumePhase, host
reviewRounds review or adjudication round loop verdict, structured findings, their fingerprint, the verify result shown to the reviewer, the agent route
runEvents narrative event loop, supervisor phase changes, warnings, sweeps; the story FINDINGS.md does not tell. level ∈ narrative, detail; a detail row of kind crash carries the traceback of a run that crashed in its payload, its summary the one-line reason
sweepStrikes supervisor sighting supervisor consecutive silent sightings per run
supervisorHeartbeats supervisor process; one row for the host sweep supervisor pid, start, last beat, passes, host; the host sweep keeps one sentinel row per store, pid 0, which readers print as "host sweep"
loopRestarts self-merge re-exec loop sha; the supervisor checks the loop came back
linearDeliveries push to Linear loop what was projected, when
interventions operator or supervisor decision on a run operator commands, supervisor action ∈ redirect, kill, extend_time_box, resume, close_out, requeue, approve, repoint plus the babysit action store.babysit() writes (a fixed set, currently 9 values); actor names who acted, maintainer, operator or supervisor, null on a row from before schema 48; the record-before-acting rule lives here
ledger entry in a run's narrative loop, operator commands kind ∈ merge, failure, round, adjudication, intervention, note, source ∈ loop, operator; written before the Linear comment that projects it; served at /ledger and /runs/N/ledger
followUps FOLLOW_UP line of a run's fix commit loop kind ∈ feature, guardrail; captured pending at the fix turn and settled when the run merges: a feature as a Backlog draft (filedAs) or a duplicate of an open one (duplicateOf), a guardrail as a findings-ledger entry; error when the draft could not be filed
storyProposals a story child's feature follow-up proposed as a new child of the story loop state ∈ proposed, accepted, rejected, superseded; written when the adjudicator calls the follow-up in_story; outside the approved plan, so it changes no child, witness, generation or frontier and cannot be claimed; raisedBy is the child whose run raised it
steerNotes holo steer note on a ticket operator commands, loop kind ∈ amendment, hint; interventionId the steer row written before it, or the operator_note row and eventId the event of a parked run's send-back; an amendment joins every later run's ticket text, a hint the next implement turn's opening alone; runId a live run's when steered while it ran, the note then carried by that run's next implementer turn; consumedBy and consumedAt the run that first carried it, withdrawnBy the steer row of a holo steer --withdraw that drops an amendment from every later run, the note never rewritten; interrupt set by holo steer --now, stopping that run's running implementer turn to resume it with the note

A story is a parent ticket whose children serve one outcome; the five tables stories, storyWitnesses, storyChildren, witnessResults and storyDecisions hold its plan, its witnesses, its children's roles, each witness run on main and the questions it put to the operator. A story's nullable witnessPaths holds the JSON list of its ## Witness paths, NULL when it names none; a witnessResults row's nullable carriedFrom names the commit whose verdict a loop pass carried to this one without running the witness set, NULL for a row that ran.

The last pull request read is stored on runs in prSeenAt, prSeenChecks, prSeenReview, prSeenThreads, prSeenOpenThreads, prSeenOpenThreadsFloor, prSeenTitle, prSeenState, prSeenAdditions, prSeenDeletions, prSeenChangedFiles, prSeenPreviewUrl, prSeenScreenshots and prSeenVideos. The nullable prSeenState is draft, ready, closed or merged, and is served as pr.state on an attention pr_open item. prSeenPreviewUrl is the newest successful deployment's URL on the head commit, and prSeenScreenshots and prSeenVideos JSON lists of {"caption", "url"} objects, the Evidence section's images and video links.

The two state machines

Both live as data (TICKET_TRANSITIONS in store/tickets.py, RUN_PHASE_TRANSITIONS in store/operate.py), store.tickets.transition() refuses any edge not in them, and a test fails if the rendered diagram in The loop drifts from the tables.

Ticket status answers "can this be claimed?": ready is claimable; in_flight has or had a run; merged and abandoned are terminal; needs_spec failed the template validator; blocked_on_deps waits on a Linear blocks relation; blocked_on_operator waits on a human and carries a blockedQuestion. A failed run leaves the ticket in_flight with no active run, which pickable() refuses; --requeue walks it back to ready along the legal edges and records why.

Run phase answers "what is this attempt doing?" and is written only by runs.set_phase(), which also moves the heartbeat and appends an event in the same transaction. failed, killed and done are terminal; set_phase() on an ended run raises RunEnded so a loop that lost its run to the supervisor stops instead of advancing it.

Reads

store/read.py holds the typed read views: one query, one frozen dataclass, no SQL anywhere else. open_readonly() opens the file with mode=ro so a reader can never take the write lock. Ticket, BlockedTicket, OpenTicket, RunSnapshot, LiveRun, ApprovedCandidate, EndedRun, MergedRun, FailedAttempt, RecentFailedRun, ReviewRound, EndedRound, RunDetail, RunRound, NarrativeEvent, LedgerEntry, LedgerWindowEntry, Strike, SupervisorBeat and the functions that return them are pinned by an allow-list in tests/test_store_surface.py, so a new read is a deliberate addition. The serve daemon, --report, the sweep and the FINDINGS renderer all read through them, which is why the terminal, the JSON and the file never disagree.

Leases and locks

  • projects.activeRunId is the claim lease, taken and released under BEGIN IMMEDIATE. A loop that dies holding it blocks every later claim until the supervisor sweeps the run, which is the supervisor's reason to exist.
  • supervisor.lock in the state directory holds a project supervisor's pid; a second supervisor for the same project exits naming it. A dead pid is reclaimed under an flock on a sidecar. The host sweep takes the same kind of lock in the host's home, HOLOPHYTE_HOME/supervisor.lock, for each run, and skips a project whose own lock names a live pid.
  • Beside it, four host files no store owns: host.toml, the registry; sweep.json, what the host sweep remembers between runs, rewritten whole after every project; host-actions.jsonl, the host daemon's append-only ledger of run-sweep requests, written before it acts; and console-build.json, the host daemon's last failed startup console build, removed by the next build that succeeds.
  • Review scratch directories under ~/.cache/holophyte/reviews/ are temporary; a review container whose directory is gone is a stray.

Interventions

Every out-of-band change to a run has a row here before the change, with an action from the fixed set and a narrative event carrying the note. The operator commands (--requeue, --approve, --babysit, --repoint) write it; the REPL rung of the escalation ladder calls store.record_intervention() directly. Backdating or mislabelling a row is worse than no row; the runbook says why.

Ledger

The narrative of a run -- each review round's verdict and the implementer's answer, the terminal adjudication, the merge line, a failure's why, a note such as a run parked for merge approval, and every intervention -- is a row in ledger (design note 9). projection.ledger() writes the row on the run's own connection and only then posts the Linear comment, so the comment is a projection of the row the way a ticket's Linear state is a projection of tickets.status: a board that is down or absent loses its copy and nothing else. record_intervention() writes the intervention entry in the same transaction as the interventions row, so --requeue and --approve land in the narrative beside the loop's own entries. store.read.ledger(conn, run_id) returns a run's entries oldest first and store.read.ledger_since() a window across runs, newest first. The daemon serves them as /runs/N/ledger and /ledger?since=MS (HTTP endpoints). A project with [report] findings = "repo" also has FINDINGS.md rendered from runs and reviewRounds; by default ("none") nothing is rendered.

FINDINGS

FINDINGS.md in a project repository that opts in with [report] findings = "repo" (the default "none" renders nothing: the store is the record) is a rendered window over the store: above a <!-- store-rendered below --> marker, frozen pre-store history that is never rewritten; below it, the newest twenty-five run and round entries rendered from runs and reviewRounds at every close-out, with everything older counted in one archive line. Each merged run's entry carries the byte-stable timing line (actual · estimate · rounds) that --report computes from the same columns. It is committed by the loop as part of the merge, so a public repository's evidence is exactly what the store holds and nothing an implementer typed.