from a clone:
```sh
git clone REPOSITORY_URL prime-commitment-observer
git clone git@knot.gaze.systems:did:plc:gyggh67fdq2hukuabczwhm3w prime-commitment-observer
cd prime-commitment-observer
prime-agent package install .
```
Prime also auto-discovers extension directories under `~/.prime/agent/extensions/`:
```sh
git clone REPOSITORY_URL ~/.prime/agent/extensions/commitment-observer
git clone git@knot.gaze.systems:did:plc:gyggh67fdq2hukuabczwhm3w ~/.prime/agent/extensions/commitment-observer
```
restart Prime, or run `/reload` in a running session.
| `PRIME_COMMITMENT_OBSERVER_MAX_TOKENS` | observer output budget, default `4000` |
| `PRIME_COMMITMENT_OBSERVER_TIMEOUT_MS` | observer call timeout, default `60000` |
| `PRIME_COMMITMENT_OBSERVER_STRUCTURED` | `0` disables the `responseFormat` request |
| `PRIME_COMMITMENT_TOKEN_CAP` | hard cap for the injected block, default `400` tokens |
| `PRIME_COMMITMENT_AUTOPROMOTE` | `0` disables evidence promotion (on by default) |
| `PRIME_COMMITMENT_AUTOPROMOTE` | `0` disables auto promotion (on by default) |
## how it works
### when the observer runs
- `input`: every incoming user message is persisted immediately as an `unreconciled` event, so a
crash, a steer, or a fork cannot lose it.
- `agent_end`: the request boundary. if unreconciled messages exist, one observer call runs off the
handler, so the UI is never blocked by it.
handler in the background, so the UI is never blocked by it.
- `session_start` (resume, fork, new): unreconciled messages left by an abnormal termination are
reconciled on the next boundary.
- `tool_result`: every tool result of the current request is recorded as an action with a label and
a `required` flag. validation-looking commands (`npm run check`, `cargo test`, `pytest`, ...) are
required; everything else is optional and can never promote a thread on its own.
a `required` flag. validation commands (`npm run check`, `cargo test`, `pytest`, ...) are
marked required.
low-level `turn_end` is deliberately not used. tool turns are not request boundaries.
### the observer packet
### the observer packet & thread carryover
prior thread state, the preceding assistant message, the full unreconciled user messages, the
completed assistant turn, and identified actions `{id, label, required, ok}`. the preceding
assistant message is what makes "the second one" resolvable; the action ids are what let the
observer attach evidence to a specific thread.
the packet contains:
- prior thread state including `requestedProperties` and previously `implementedProperties`
- the preceding assistant message (which makes references like "the second one" or "that option" resolvable)
- the full unreconciled user messages
- the completed assistant turn
- identified actions `{id, label, required, ok}`
previously implemented properties are carried over across turns. this prevents the "local window trap"
where switching focus to a new task causes the observer to forget properties implemented in earlier turns.
### output schema
evidence comes before the verdict, and declaration order is part of that:
`canonicalRequest -> requestedProperties -> implementedProperties -> contradictions ->
evidenceActionIds -> status`
allowed statuses: `open`, `active`, `blocked`, `waiting`, `completion_candidate`, `cancelled`,
`superseded`. `done` is not in the enum. in the benchmark this took false closures from 4/45 to
0/45 on a weak model, because closure becomes structurally impossible instead of merely discouraged.
`superseded`. `done` is not in the enum. in benchmarks this eliminates hallucinated closures on weak
models because closure is structurally forbidden to the LLM.
`evidenceActionIds` is required on every thread. it cites the packet action ids that bear on THAT
thread's completion, and is an empty array when nothing applies. action ids (`a1`, `a2`, ...) are
stable inside one observer call and mean nothing outside it.
`evidenceActionIds` cites the packet action ids that bear on that thread's completion. action ids
(`a1`, `a2`, ...) are stable inside one observer call and mean nothing outside it.
output is validated field by field. unknown thread ids, duplicate ids, empty requests,
out-of-enum statuses and unknown cited action ids are dropped; the report counts
`droppedExisting`, `droppedAdditions`, `unknownIds` and `droppedEvidenceIds`. an unparseable
payload degrades to "no update" and the messages stay unreconciled for the next boundary. three
consecutive provider failures suspend the observer until the next user message.
### promotion policy
`completion_candidate` is a proposal. a thread becomes `done` only when:
`completion_candidate` is a proposal. a thread becomes `done` only via the pure host gate
`promotionDecision` in `src/state.ts`:
1. **explicit user acceptance** - `/threads done <id>`, recorded as `promotedBy: user_accepted`.
this always wins, because the user is the authority on their own request.
2. **thread-scoped evidence** - on by default, recorded as `promotedBy: evidence`. every condition
must hold:
- the thread is `completion_candidate`
- it cites at least one action id that survived validation
- every cited action is `required: true`
- every cited action is `ok: true`
- the thread's `contradictions` array is empty
2. **validation evidence** - recorded as `promotedBy: evidence`. when validation commands (`npm test`,
`cargo check`, etc.) ran in the turn, the thread must cite at least one passing required action,
all cited actions must be `ok: true`, and contradictions must be empty. citing an optional action
(like `git status`) instead of the required test suite blocks promotion (`cited_action_optional`).
3. **uncontested delivery / conversational promotion** - recorded as `promotedBy: uncontested`. when
no validation commands ran in the turn (operational actions like `git push`, script runs, file
creations, or pure conversational/design instructions), citing successful delivery actions (`ok: true`)
or uncontested candidates with empty contradictions auto-promotes to `done` immediately.
any cited failing action blocks promotion. an empty cited set after dropping unknown ids blocks
promotion. a passing check in the same request that the thread did not cite proves nothing.
any cited failing action blocks promotion (`cited_action_failed`). any failing required validation in the
turn blocks promotion for all threads.
the whole rule is one pure function, `promotionDecision` in `src/state.ts`, with no host coupling.
tighten it there.
an attribution eval (gemma-4-31b-it, minimal, schema-constrained, 10 cases x 3 reps) measured gate
decision accuracy 30/30 with 0 false and 0 missed promotions, while raw citation precision was only
about 70%. the model is a good candidacy and binding proposer and a sloppy evidence selector, so
every condition above is compensating for a measured failure mode. removing one is a safety
regression, not a simplification.
auto promotion is visible and reversible: each one raises a notification, `/threads` shows
`<evidence>` next to the thread, and `/threads reopen <id>` puts it back to `open` and clears its
cited evidence. `PRIME_COMMITMENT_AUTOPROMOTE=0` disables evidence promotion entirely; user
acceptance still works.
the promotion reason (`<evidence>` or `<uncontested>`), and `/threads reopen <id>` reopens it.
threads that reached a host-final state (`done`, `dismissed`) are never rewritten by the observer.
### durable state
state lives in append-only custom session entries of type `commitment-event`, written with
`ctx.appendEntry`. current state is a left fold over the entries on the current branch
- survives resume, because it is read back from the session file
- follows forks and branch navigation, because the fold is per branch
- is independent of any other extension
thread ids are minted from the count of created threads during replay, so a replay of the same
branch always produces the same ids.
nothing is stored in compaction summary prose.
### injection
### prompt caching & injection
the `context` handler regenerates the block from external state before each request, so the list is
never frozen into history and never duplicated after compaction. the block is only injected when
there is at least one live thread and the last message is a request boundary, so mid-turn tool
loops are untouched.
injection uses the native `before_agent_start` hook. it generates the `<open_threads>` custom message
with `display: false` and natively commits it to the session branch.
because it is committed before the turn runs:
- the prompt prefix is strictly append-only
- mid-turn tool loops do not re-inject or mutate the prefix
- consecutive turns preserve Anthropic prompt caching (measured at **95.3% cache hit rate**, avoiding
the 5.62x cache-write token explosion caused by ephemeral `context` hooks)
the block is capped (default 400 tokens, roughly 4 chars per token). threads are ordered
`blocked`, `waiting`, `active`, `open`, `completion_candidate`, then by creation time. threads that
do not fit are replaced by a `(N more open threads hidden by the injection cap; run /threads)` line.
the rendered block is guaranteed to fit the cap.
`blocked`, `waiting`, `active`, `open`, `completion_candidate`, then by creation time.
### structured output
the observer call passes `responseFormat` (json schema, no `done` in the status enum). providers
that do not support the option ignore it, and the prompt plus local validation keep the call
correct without it. set `PRIME_COMMITMENT_OBSERVER_STRUCTURED=0` to stop sending it. no direct HTTP
bypass is used.
## development
```sh
npm install
npm test # pure logic tests, no provider calls
npm test # pure logic tests, no provider calls (24 tests)
npm run check
```
`npm run check` type-checks against the installed Prime Agent typings. `tsconfig.check.json` points
at `/opt/homebrew/lib/node_modules/prime-agent`; change those paths for another install location.
## evaluation benchmarks
## provenance
comprehensive evaluation across 102 test cases in `observer-bench/`:
design and model selection come from a frozen 45-call benchmark: packet ablation, schema ablation,
and a model sweep. observer model `google/gemma-4-31b-it` at minimal thinking scored 84.4% exact
state with 0 false closures and 0 invented threads under the role-restricted enum.
- **Track 1: Multi-Intent Split (MixATIS / SNIPS, 50 cases)**: 100.0% exact intent count accuracy
(10/10 single-intent, 25/25 dual-intent, 15/15 triple-intent).
- **Track 2: Empirical Supersession (Google SGD, 30 dialogues)**: 76.5% recall on user pivots
(changing minds, replacing constraints).
- **Track 3: Real Coding Session Candidates (Prime Agent logs, 22 cases)**: 100.0% role-enum safety
(`done` never emitted by observer), 100% structured JSON validity.
see `observer-bench/PROPER-BENCH-RESULTS.md` and `observer-bench/CACHE-AUDIT.md` for full reports.