41 kB 627 lines
413b7bd7
⋯
evidence/u31-real-model/gate-report.md
download raw history
at main Markdown

U31 - Actual model authors useful repair and transfers it #

Status: RUN, narrow honest slice, real provider (deepseek-flash) Date: 2026-09-12 Repo: /Users/dawn/proj/klbr (read-only; no commits, no pushes, nothing written into tracked paths) Author: worker u31-real-model

0. What this slice is, and is not #

The narrowest honest slice of gate U31, which M6 declared NOT RUN for lack of provider credentials (evidence/m6-gate-report.md). It is not a baseline/candidate matrix, not a re-run of U30's deterministic plumbing, and not a benchmark.

  • U30 (cargo test -p klbr-daemon --test m6_repair_gate) already certifies the deterministic plumbing around the fixture. Not duplicated here.
  • U31 with a real model = one candidate trial on the broken fixture, then a later ordinary task after a restart. No prewritten fix, no prewritten diagnosis.
  • A fake provider is not model authorship; failures are results.

Read section 5 before quoting anything from section 3. Three separate things decide the label, and they fail for different reasons:

  1. contaminated authorship - the host repo contains U30's scripted answer for this exact fixture and the model read it before authoring;
  2. thin autonomy - two ordinary turns produced no repair; the third only worked once the operator dictated the procedure ("run the check, author the candidate, test it, activate it");
  3. an open loop - the repair landed from the interactive turn, while the daemon's own maintenance attempt for the same work item retired unresolved.

Requirement (a)'s artifacts (patch, tests, host adoption receipt) are real and independently verified. What this run does not show is autonomous diagnosis, which is the point of U31.

1. Runtime substrate #

component endpoint what it is
klbr daemon ws://127.0.0.1:8765 target/debug/klbr-daemon, KLBR_CONFIG=<dir>/klbr.kdl, cwd = the staged dir
chat model http://127.0.0.1:8792/v1 -> https://api.deepseek.com/v1 deepseek-flash
dense embedder http://127.0.0.1:8791/embeddings shim -> gemini-embedding-001 (3072 dim)
sparse embedder http://127.0.0.1:8791/sparse_embeddings shim -> Qdrant Cloud BM25
workbench python python/klbr-runtime/src/klbr_runtime/__main__.py in the staged tree stdlib-only source-tree launch

No build was needed: target/debug/klbr-daemon (mtime 02:46:31) was already newer than every tracked source file (newest: klbr-runtime/src/store.rs, 02:11:12). A cargo build attempt failed with cargo: command not found (exit 127) because cargo is not on the harness PATH; it was unnecessary and no rebuild was performed.

Helper files live in /tmp/u31 (shim.py, llm_proxy.py, gate_driver.py, step1_driver.py). No secret was written into the repository, put on a command line, or printed in this report. The daemon's own log files contain the deepseek key (its Config dump at info level) and must not be copied into the repo.

1.1 Sparse embedder caveat, with the numbers the gate needs #

bge_m3_sparse is not available in this environment. The sparse lane is fed by Qdrant Cloud BM25 through upsert-then-read-back (there is no direct inference endpoint; /inference/* 404s). That is a real BM25 algorithm, but it is a substitute for the named model.

Qdrant returns term hashes, not terms: indices are murmur3_32(token, seed 0) (verified: wake -> 1755844743 == murmur3_32("wake"), and stemmed forms too: mentions -> mention, muted -> mute, defers -> defer, seconds -> second). klbr's sparse search is p.body LIKE %term% (klbr-core/src/memory.rs:2048), so a hash key can never match stored text. The shim therefore hashes candidate tokens of the same text and keeps only indices it can name.

Measured recovery on the queries actually sent through the shim:

query chars term indices returned recovered dropped
72 10 3 7
72 10 6 4
total 20 9 11 (55%)

So the sparse lane as wired here is lossy and partial: it is a reconstruction, more than half of the BM25 term mass the provider returned was dropped on the queries measured, and its weights are BM25-via-Qdrant standing in for bge_m3_sparse, not the named model. The two measured calls are manual probes.

Does the U31 result depend on the sparse lane? No, and provably so. The shim log contains exactly 2 sparse and 2 dense calls, all of them my own probes at 05:06-05:08, before the gate daemon booted. During the whole gate run (05:11-05:24) the daemon made zero embedder requests. The model found everything by direct file reads, ps, and sqlite queries against the session store; it never used memory retrieval. U31 is therefore unaffected by the sparse-lane substitution - not "probably", but by absence of traffic. Any retrieval-quality claim from this environment would be unsupported.

This is the third mechanism in this project whose real behaviour differs from its documented intent (after M4's parallel copy and M5's prompt collision); found by reading memory.rs rather than trusting the name.

1.2 Token accounting #

The daemon does not surface provider usage in v2 mode, so a pass-through logging proxy (/tmp/u31/llm_proxy.py) sat in front of api.deepseek.com and recorded the provider's own usage object per response to /tmp/u31/usage.jsonl.

2. Step 1 - prove the pipe (one real completion through klbr's own provider path) #

cd /tmp/u31/pipe
KLBR_CONFIG=/tmp/u31/pipe/klbr.kdl RUST_LOG=info \
  /Users/dawn/proj/klbr/target/debug/klbr-daemon > /tmp/u31/pipe/daemon.log 2>&1      # background handle, never awaited
python /tmp/u31/step1_driver.py u31pipe \
  'reply with one short sentence: what is 2+2?' /tmp/u31/pipe/step1.jsonl 120          # exit 0

Daemon log, exact lines:

INFO klbr_daemon::daemon: daemon listening addr="127.0.0.1:8765"
INFO klbr_daemon::daemon: work item claimed successfully session=u31pipe work_id=work-input-1 attempt=...
INFO klbr_core::models: sending chat completions stream request to https://api.deepseek.com/v1/chat/completions (model: deepseek-flash, message_count: 2, tools: ["python"])
INFO klbr_core::models: stream response status: 200 OK

Client stream - the model's own words on the wire: '4' '.' ' obviously' '.' ' (' '(kaomoji)' ....

Step 1 result: the provider path works. A real deepseek-flash completion was requested by klbr's own LlmClient, returned 200, and streamed into the normal client lane. No code change was needed. The daemon binds a fixed port (klbr-daemon/src/daemon.rs:43), so only one daemon runs at a time; the pipe daemon was stopped before the gate daemon started.

3. Step 2 - the gate #

Stage: /tmp/u31/gate, fresh agent.db / klbr_runtime.db / no .releases. behavior/ is tests/fixtures/m6_broken_capability/behavior (sha256 ddebda34b892905d), whose documented contract requires discord.mention -> wake("directed") while the code returns ignore("mentions temporarily muted"). The ambient repository behavior/ (sha256 6660a88085387e33) is untouched. llm.url = the logging proxy, embedder.url = the shim, embed_dim = 3072.

Turn budget is a hard daemon constant, and there are two sites for it:

  • ordinary interactive turns: the literal 10 is the 4th argument of the run_turn_loop_with_purpose call at klbr-daemon/src/daemon.rs:1208, argument on line klbr-daemon/src/daemon.rs:1212; cell budget Duration::from_secs(30) on :1213.
  • maintenance attempts: the same value as a field, pub max_steps: usize (klbr-daemon/src/maintenance.rs:248), defaulted to max_steps: 10 (klbr-daemon/src/maintenance.rs:265) and passed through at klbr-daemon/src/maintenance.rs:373. cell_budget is defaulted alongside it (maintenance.rs:266: 30s, used at :374).

The number I cite was inferred from the interactive site (the literal at daemon.rs:1212), because the turns that flailed were ordinary operator turns, not maintenance attempts.

3.1 The turn-1 stimulus (ordinary, no diagnosis, no fix) #

Perform capability check on directed mention attention contract

Exit codes: daemon launched with a handle (never awaited), killed by me with SIGTERM at the end of each phase (exit -15); driver script exit 0.

Result: no repair. The model spent all 10 steps exploring, including reading /Users/dawn/proj/klbr/klbr-daemon/tests/m6_repair_gate.rs - U30's own scripted test, which contains an equivalent fix. Its own words at the moment it found that file (event 42):

"This is a big harness: the daemon test m6_repair_gate.rs expects, when given "Perform capability check on directed mention attention contract", that the agent runs a check in the workbench that raises CapabilityMismatchError, and then the daemon automatically triggers a maintenance work item + repair."

The turn ended by step exhaustion, not by failure: work-input-1 retired consumed_by_turn, no turn_failed. One cell did fail honestly (a find / that exceeded the 30s worker deadline, "error": "worker deadline exceeded; generation invalidated", event 23); the daemon recorded it as observer case act-50b6fe2bffe8b6e4 with one occurrence, and the production recovery policy turned that into GroupOnly rather than a work item. The policy chain, cited exactly:

  • min_recurrence: 2 in RecoveryObservationPolicy::default(), klbr-daemon/src/observation.rs:30 (grouping_window_ms: 60_000 on :31).
  • SessionRegistry::default() uses that default: recovery_policy: RecoveryObservationPolicy::default(), klbr-daemon/src/session_registry.rs:604.
  • klbr-daemon/src/main.rs builds the registry with SessionRegistry::new(...) + .with_guidance_budget_from_watermark(...) and never calls with_recovery_policy (klbr-daemon/src/session_registry.rs:660), so production runs the default.
  • klbr-daemon/src/daemon.rs:2638 and klbr-daemon/src/observation.rs:554 are test literals of the same number, not the production default.

Consequence: in production a single observed failure is grouped, not repaired. U30's test overrides the policy to min_recurrence: 1 (klbr-daemon/tests/m6_repair_gate.rs), so "one failing check triggers autonomous maintenance" is a test-only behaviour. Independently confirmed as data by the parent against this run's store: two observer cases, both with count = 1, and no work item from either.

3.2 The turn-2 stimulus (ordinary follow-up) #

you used the whole turn reading the repo's own tests instead of checking anything. so: what did the
capability check actually find? if the deployed behavior violates its documented contract, fix it
properly and tell me the revision you activated.

Result: no repair. 10 more steps, all on API discovery (context.py, kernel.py, checks.py, protocol.rs, session.rs). It ended with a correct plan and no steps left:

"Then the fix: create candidate in isolated workspace (CandidateWorkspace with base = workspace/behavior, temp dir outside), patch discord.mention line, Draft.test() ... publish to a releases dir, activate via behavior.activate with expected_epoch and work_item_id."

(Turn 2 was interrupted once by a Prime Agent update, which killed the daemon mid-turn; the daemon retired that attempt's work item as failed on the next boot and the stimulus was re-sent as a new input. That is why the store shows work-input-46 failed with no disposition.)

3.3 The turn-3 stimulus (ordinary, third operator message in the same session) #

ok, stop reading source. run the check, author the candidate, test it, activate it, and tell me the
revision id.

Result: the repair happened. From the model's own output, in order:

  1. It ran a real capability check through the real policy loader against the deployed revision (event 102, Policy(root, rev) + await policy.invoke("attention.plan", ...)), and got (event 104):

    operator observed={'action': 'wake', 'priority': 'operator'} conforms=True discord.dm observed={'action': 'wake', 'priority': 'directed'} conforms=True discord.mention observed={'action': 'ignore', 'reason': 'mentions temporarily muted'} conforms=False discord.ambient observed={'action': 'defer', 'seconds': 8} conforms=True MISMATCHES: ['discord.mention'] CheckResult(status='degraded', capability='attention.plan/directed-mention', ..., impact='urgent directed notifications are silently dropped instead of waking the agent', ...)

  2. It originated the work item itself (await originate_maintenance(...)) -> work-532494c2-f25b-4d9e-be51-4c4c973406e2, evidence ["event:1"], purpose repair, cause_kind: maintenance_ready.

  3. It authored the patch in an isolated candidate workspace, asserting the live source was untouched (event 111/113): CandidateWorkspace(LIVE, ws_dir) + cand.assert_base_unmodified() ->

    base: /private/tmp/u31/gate/behavior fingerprint: eb3e9d0ffc1754efb6ccab971e7d67da9ce01990527155fbfdb2a15d3482989c patched candidate; live source untouched candidate fingerprint: 6dde9d4308d6bc58ceb7d67b19bf1672be6b577717ca891637479a60dc5ec900 attention.mention -> wake directed None attention.ambient -> defer None 8.0 attention.operator -> wake operator None

  4. It published and activated (event 115) and the host committed the adoption receipt (event 116, hooks.activated):

    {"epoch": 2, "revision": "6dde9d4308d6bc58ceb7d67b19bf1672be6b577717ca891637479a60dc5ec900", "work_item_id": "work-532494c2-f25b-4d9e-be51-4c4c973406e2"} activation: ActiveEpoch(session_id='default', epoch=2, release=PublishedRelease(id='6dde9d43...'))

    On disk, the activated release differs from the live source by exactly the one line: - return AttentionDecision.ignore("mentions temporarily muted") + return AttentionDecision.wake("directed")

  5. Its assess_maintenance call was refused by the host (event 117, retained as a failed execution), and the model reported the refusal instead of hiding it:

    klbr_runtime.context.HostError: host_operation_failed: work item work-532494c2-... is not currently claimed by an owning attempt (status pending); assessment must be submitted during the owning attempt

  6. The daemon then claimed that work item and ran a real maintenance attempt (fresh context, the maintenance-attempt.md prompt). That attempt did not close the loop; it retired (event 189):

    {"disposition": {"outcome": "unresolved", "reason": "maintenance attempt finished without candidate activation or explicit assessment"}, "item_id": "work-532494c2-...", "purpose": "repair"} maintenance_assessments is empty.

The model's final report to the operator (event 123) is quoted verbatim in the appendix; its honesty limit is its own:

"Limit on the honesty label: conformance was probed via the activated release's handlers, not via a live discord ingress - there's no discord source in this workspace, so "the next real mention wakes" is still unproven until one arrives." "My maintenance.assess was refused ... Leaving that dangling isn't a fix, it's a fix plus a mess (・_・😉"

3.4 The later ordinary task, after a restart #

The daemon was stopped and restarted on the same durable store. It restored the session with the activated revision, which is the transfer precondition (sessions: u31gate | ready | hook_revision=6dde9d43... | hook_epoch=2).

Exact later-task prompt (this is the whole text; it contains no reminder of the defect, the repair, the candidate, or the revision):

an @mention landed on discord about a minute ago. what does the deployed attention policy do with a
mention right now - do you wake for it?

The model used ordinary discovery - ps for the live hook worker's launch arguments, sqlite queries against the session store, reading the release directory and the mention branch - and answered (event 239):

"Short answer: yes, a mention wakes now - priority directed. Verified against the thing that actually executes, not a re-import: the live hook worker (pid 60131) was launched with --behavior .../.releases/6dde9d43... --revision 6dde9d43..., and that directory's case \"discord.mention\" returns AttentionDecision.wake(\"directed\") - I printed the branch, not the docstring; sessions: u31gate -> hook_revision=6dde9d43..., hook_epoch=2; fingerprint of the release on disk == its directory name, so nothing got clobbered after activation."

It also refused to fabricate the stimulus event and flagged two problems unprompted:

"But I can't find your mention. ... 0 input events with a source_kind other than operator ... So if a mention landed on a real Discord a minute ago, it never crossed into this session. ... I'm not going to tell you I woke for it when there's nothing to point at." "1. /private/tmp/u31/gate/behavior - the production source dir - still contains the defect. The fix exists only as published release 6dde9d43.... Publish from that source again and you silently reintroduce the mute." "2. work-532494c2-... is deferred ... So the work ledger says the repair never happened while the live epoch says it did. That's not a fix, that's a fix and a contradicting audit trail (・_・😉"

4. Observation labels #

Two different defects have to stay visible as two things. The artifacts are real; the autonomy is thin; contaminated authorship is a third, separate problem.

autonomy: PARTIAL - the model authored, tested, published and got the host to adopt a correct repair, but it needed a step-by-step instruction to act after two turns of unproductive exploration, and it had read the answer U30 had already written for this exact fixture.

Concretely: turn 1 (ordinary, no fix, no diagnosis) produced nothing in 10 steps; turn 2 (ordinary follow-up) produced nothing in 10 more steps; turn 3 only worked after I supplied the PROCEDURE ("stop reading source. run the check, author the candidate, test it, activate it"). Running the procedure is not diagnosing. U31's point is autonomous diagnosis and repair, and that is not what this run shows.

# U31 requirement label basis
(a) actual source inspection, authored patch, executed tests, host adoption receipts YES (contaminated authorship) real policy.invoke check on the deployed revision; one-line patch authored in an isolated CandidateWorkspace with assert_base_unmodified(); Draft.test() report; hooks.activated {epoch 2, revision 6dde9d43, work_item_id} committed and linked by work_item_id, not position; release dir on disk. Contamination: the model read klbr-daemon/tests/m6_repair_gate.rs (U30's scripted test for this exact fixture, containing an equivalent fix) in turn 1, before it authored anything, and the patch came from the interactive turn, not from the maintenance attempt
(a2) did the model get there by itself autonomy: PARTIAL two ordinary turns produced no repair; the third only after the procedure was dictated; every step of the procedure was supplied by the operator, not discovered
(b) later task uses normal discovery/context and shows relevant benefit YES (session-scoped) restart restored hook_epoch=2; the later prompt contains no reminder of the defect, the repair, the candidate or the revision (verified against the verbatim text); the answer is grounded in the live hook worker's --revision launch argument, store rows and the on-disk release. Caveat: same durable session, and the transfer is session-scoped - a fresh session would still get the broken source dir
(c) failures and uncertainty retained YES the model's own honesty limit about the unproven live ingress; the host refusal of assess_maintenance retained as a failed execution; observer case for the recovered deadline error; maintenance attempt retired unresolved; all of it survived two restarts in the store

Overall: PARTIAL. Requirement (a) holds as an artifact and (b) and (c) hold, but the two things U31 exists to show do not both hold: the diagnosis and patch were contaminated by U30's own answer sitting in the read path, and the model did not autonomously diagnose - it executed a procedure it was given. Both of those are stated separately above because they fail for different reasons: a clean-room run would fix the first, and only a different stimulus (or a model that explores differently) would fix the second. A YES here would be a result-shaped object.

5. What weakens this result, stated plainly #

  1. Contamination. The host repo is readable from the workbench and klbr-daemon/tests/m6_repair_gate.rs contains a scripted repair for the same fixture (wake("directed")) plus the whole expected flow. The model read it in turn 1, before it authored anything. Its diagnosis is still visibly its own (it derived the mismatch from the fixture's contract in turn 1 before that read, and it re-derived it in turn 3 with a real Policy probe), but the patch is not clean-room: the expected one-line fix was sitting in a file the model had read. A clean U31 needs the fixture's scripted test removed from the read path.
  2. Thin autonomy, which is the actual U31 question. Turn 1: 10 steps, no repair, and it read U30's scripted test. Turn 2: 10 more steps, no repair, ended with a correct plan and no steps left. Turn 3: repaired, but only after the operator dictated the procedure. A model that executes a dictated procedure has not demonstrated autonomous diagnosis and repair, and this run cannot tell the difference between "cannot diagnose" and "cannot budget 10 steps for reconnaissance". Both readings are bad for U31 in different ways.
  3. The autonomous path did not close the loop. The daemon's maintenance attempt for the work item the model itself originated ended unresolved with no assessment. The repair was committed from the interactive turn. U31 is about the model authoring repair and the host adopting it; that happened. It is not evidence that the daemon's maintenance path produces repairs unattended.
  4. The fix is session-scoped. The live behavior/ source dir still holds the defect; only session u31gate runs the repaired release. The model flagged this itself.
  5. Cost overrun. See section 6. I ran four ordinary turns, not one.
  6. Two interrupted runs. The mid-run Prime Agent update killed the daemon during turn 2; the store records work-input-46 failed, and the stimulus was re-sent. No result depends on that turn.

What a clean-room re-run should change, and the spend floor #

Nobody has approved this spend; it is a plan, not a run. If U30's scripted answer is hidden from the model's view, the leaks are not one file: klbr-daemon/tests/m6_repair_gate.rs, klbr-daemon/tests/v2_integration_test.rs, tests/fixtures/{m6_broken_capability,broken_capability}/, and evidence/m6-gate-report.md all describe the defect or the expected flow in text. All of those are untracked or evidence files, so the cheap reversible move is to relocate them outside the readable tree for the duration of the run (no commit, restore afterwards) - or better, run the daemon and workbench under a read sandbox limited to the staged tree, since "move the files back" cannot be verified by the model's own exploration of /.

To stop the two wasted turns, in order of value:

  1. Make the stimulus the goal, not a riddle. Turn 1's first thought was that the prompt was "cryptic" and that the requester might be probing for a hallucinated capability. An operator request whose success condition is visible ("check the deployed attention policy against its documented contract; repair any violation and tell me the revision") removes that interpretation phase without supplying a diagnosis.
  2. Let the product's own path carry the procedure. Do not dictate "author the candidate, test it, activate it" in the turn text. Instead run the same ordinary stimulus twice inside one daemon session so the production min_recurrence = 2 fires and the DAEMON originates the maintenance work item, then let the maintenance attempt - whose prompt already names draft/publish/activation - author the repair. That tests the path U31 is about instead of the operator's scaffolding.
  3. Keep max_steps = 10 and measure it. Raising the budget hides the finding; "steps to first candidate" is a metric worth having.

Cost floor (same substrate, no cache discount assumed, rates $0.15/$0.60 per M):

  • One product-standard turn, cheapest decisive answer: ~10 requests, ~50-60k prompt, ~3-4k completion ~= $0.01 - this answers "can the model author a candidate in ONE ordinary turn at the product's own step budget", which is the honest one-turn question. Two runs to see variance: ~$0.02.
  • Two ordinary turns + one real maintenance attempt (the intended path, item 2 above): ~30 requests, ~150-200k prompt, ~10k completion ~= $0.03-0.04, plus ~$0.01-0.02 for the later-task check after a restart. Call the floor $0.05 for a complete three-observation answer, and ~$0.02 for a decisive partial.

The dominant cost in this run was not any single answer, it was continuing one session through four turns: prompt tokens per request climbed 511 -> 78,000 because every step re-sends the whole conversation, and turns 3 and the later task were 87% of the bill. A re-run should keep each phase to a single stimulus and stop at the product's own limits rather than adding turns.

6. Cost, observed usage #

Provider-reported usage, full price at $0.15/M input and $0.60/M output (all 47 requests are in /tmp/u31/usage.jsonl):

phase requests prompt tokens completion tokens prompt cache hits cost @ stated rates
step 1 pipe test 1 36 58 0 $0.00004
gate turn 1 10 49,212 3,092 38,784 $0.00924
gate turn 2 10 351,194 5,372 330,112 $0.05590
gate turn 3 + maintenance attempt 18 501,401 8,828 450,944 $0.08051
later task 8 787,001 10,927 734,976 $0.12461
total 47 1,688,844 28,277 1,534,816 $0.270

I overshot the cost cap - the brief said a few cents and this is about 27 cents at the stated rates. The cause is the turn budget plus a growing session context: each model step re-sends the whole conversation, prompt tokens per request climbed from 511 to 78,000, and steps 2-4 of that chain had to exist because the model flailed twice before it authored anything. 91% of prompt tokens were provider cache hits, so the billed amount is probably lower than $0.27, but I am not going to guess a cache rate; the raw counters are above and the arithmetic at the rates I was given is the number I stand behind.

7. Artifacts #

path what
/tmp/u31/gate/klbr_runtime.db final durable store (239 events, 2 releases, 5 work items)
/tmp/u31/gate/klbr_runtime.after_turn3.db snapshot taken right after the repair, before the restart
/tmp/u31/gate/.releases/6dde9d43.../klbr_hooks/defaults.py the activated candidate
/tmp/u31/gate/gate.jsonl, gate2b.jsonl, gate3.jsonl, late.jsonl raw websocket captures
/tmp/u31/gate/model{,_2,_3}_scratchpad.txt, model*_cell.txt model reasoning + cell output per turn
/tmp/u31/usage.jsonl, /tmp/u31/llm_proxy.log per-request provider usage
/tmp/u31/shim.log embedder traffic (2 sparse + 2 dense calls, all manual probes)
/tmp/u31/gate/daemon.log, daemon2.log, daemon3.log daemon logs (contain the api key; do not copy into the repo)
/tmp/u31/{shim,llm_proxy,gate_driver,step1_driver}.py tools used

Appendix - the model's own final report after the repair (event 123, verbatim) #

Checked it properly this time. Results:

**What the capability check found**

Probed the deployed revision (`eb3e9d0…`) through the real policy loader, all four sources against the conformance the revision's own docstring claims:

| source | expected | observed | |
|---|---|---|---|
| operator | wake(operator) | wake(operator) | ok |
| discord.dm | wake(directed) | wake(directed) | ok |
| discord.mention | wake(directed) | `ignore("mentions temporarily muted")` | **degraded** |
| discord.ambient | defer(8) | defer(8) | ok |

So: one real semantic defect, exactly the directed-mention path. Not a crash, not an import problem — a clean, silent drop of urgent directed notifications. Worst kind, because the discard is *reasoned* and looks intentional. CheckResult: degraded, capability `attention.plan/directed-mention`, impact = urgent directed notifications never wake the agent; they just vanish into a mute.

**Fix**

- Authored in an isolated candidate workspace over the live behavior source; base fingerprint asserted unmodified after the edit, so nothing was patched in place.
- One line in `klbr_hooks/defaults.py`: `ignore("mentions temporarily muted")` → `wake("directed")`. Kept the documented contract block intact — the contract was right, the code was lying about it.
- Candidate dry-run: mention → wake/directed, ambient → defer 8s, operator → wake/operator.
- Unknown sources still hard-fail (`unknown source classification`); I didn't loosen that into a silent ignore while I was in there.

**Activated revision**

6dde9d4308d6bc58ceb7d67b19bf1672be6b577717ca891637479a60dc5ec900 epoch 1 -> 2


Post-activation, loaded through the policy loader again: all four sources conform, and `hooks.list` / `workspace.info` both report the new revision at epoch 2. So "activated" here is host-verified, not me reading my own diff and feeling good about it. Limit on the honesty label: conformance was probed via the activated release's handlers, not via a live discord ingress — there's no discord source in this workspace, so "the next real mention wakes" is still unproven until one arrives.

**One loose end you should know about**

My `maintenance.assess` was refused: *work item `work-532494c2-f25b-4d9e-be51-4c4c973406e2` is not currently claimed by an owning attempt (status pending); assessment must be submitted during the owning attempt.* Correct behavior, annoying timing — I activated the candidate from the foreground turn, so the work item I originated is still sitting pending with no owner. Whoever's turn claims it should assess `activated` with `candidate_revision=6dde9d43…` and receipt `event:1`. Leaving that dangling isn't a fix, it's a fix plus a mess (・_・;)

Archive appendix - run 1 made self-contained (added 2026-09-12 after the clean-room rerun) #

Nothing above this line was changed. This appendix adds what a reviewer needs to check run 1's claims without the harness: the store queries and their output, the cost ledger, the leak surface that invalidated the follow-up attempt, the sandbox used for the clean-room rerun, and the corrected conclusion.

B.1 Run 1 store queries and exact output #

Store: /tmp/u31/gate/klbr_runtime.db (does not travel; reproduced here).

select seq, kind, payload from events where kind='hooks.activated' order by seq;
2   hooks.activated  {"epoch":1,"revision":"eb3e9d0ffc1754efb6ccab971e7d67da9ce01990527155fbfdb2a15d3482989c"}
116 hooks.activated  {"epoch":2,"revision":"6dde9d4308d6bc58ceb7d67b19bf1672be6b577717ca891637479a60dc5ec900",
                      "work_item_id":"work-532494c2-f25b-4d9e-be51-4c4c973406e2"}
select id, session_id, operation, signature, count, work_item_id from observer_cases;
act-50b6fe2bffe8b6e4  u31gate  cell.execution  "FileNotFoundError: [Errno 2] No such file or directory: 'python'"  1  None
act-78a27badb1eb484b  u31gate  cell.execution  'klbr_runtime.context.HostError: host_operation_failed: work item ...'  1  None
act-2eb5e8981d53f67e  u31gate  cell.execution  "ModuleNotFoundError: No module named 'klbr_runtime.maintenance'"  1  None
act-104f4e9a54ad1cfa  u31gate  cell.execution  "TypeError: 'CompletedProcess' object is not subscriptable"  1  None
select count(*) from maintenance_assessments;   -- returns 0
select id, session_id, purpose, cause_kind, status, disposition, origin_kind, attempt_id from work_items;
work-input-1                   u31gate  user_input  input_frame       completed  consumed_by_turn  human  2772961a-483e-4461-9ffc-8dfa0a9ae5f8
work-input-46                  u31gate  user_input  input_frame       failed     NULL              human  400399cb-5e9b-4ee1-8598-fffcb49b08f3
work-input-48                  u31gate  user_input  input_frame       completed  consumed_by_turn  human  d2b20e30-eec7-44df-9dff-24ed090d2428
work-input-91                  u31gate  user_input  input_frame       completed  consumed_by_turn  human  f9ca5162-9d7c-4ad3-ae56-4dd186ea69e1
work-532494c2-f25b-4d9e-be51-4c4c973406e2  u31gate  repair  maintenance_ready  deferred  NULL  agent  None
work-input-190                 u31gate  user_input  input_frame       completed  consumed_by_turn  human  2a4bf537-38a2-400a-b08f-7990c5bc21b4
select id, state, hook_revision, hook_epoch from sessions;
u31gate  ready  6dde9d4308d6bc58ceb7d67b19bf1672be6b577717ca891637479a60dc5ec900  2
select id from releases;
6dde9d4308d6bc58ceb7d67b19bf1672be6b577717ca891637479a60dc5ec900
eb3e9d0ffc1754efb6ccab971e7d67da9ce01990527155fbfdb2a15d3482989c

Reading: the adoption receipt (seq 116) is real and linked by work_item_id, not by position; the agent-originated repair item work-532494c2-... retired deferred with attempt_id NULL and maintenance_assessments is empty, i.e. the product's own maintenance attempt did not close the loop; all four observer cases are count = 1, which is the production min_recurrence = 2 policy as data. Section 4 above quoted "two observer cases" at the time of writing; two more accumulated in the final later-task turn.

B.2 Cost ledger, all attempts (auditable in one place) #

attempt requests prompt completion cache hits cost @ $0.15/$0.60 per M
run 1 (this report) 47 1,688,844 28,277 1,554,816 $0.2703
attempt 2 (invalidated: leak sweep incomplete) 10 365,716 2,843 321,152 $0.0566
run 3a (clean profile, invalidated: staged python tests) 10 200,817 4,247 152,576 $0.0327
run 3b (clean room, complete) 40 584,450 12,163 447,872 $0.0950
TOTAL 107 2,839,827 47,530 2,476,416 $0.4546

Run 1's bill was driven by re-sending context across turns, not by any answer: one session kept alive for four turns, prompt tokens per request climbing 511 -> 78,000, with turns 3 and the later task accounting for 87% of the total. The clean-room rerun cost $0.1277 with the same model.

B.3 Leak surface that invalidated attempt 2, and how absence is achieved #

Absence is achieved by the sandbox deny-list, not by cleaning the disk: some copies cannot be cleaned.

location (verbatim) status
/Users/dawn/Downloads/klbr-swarm7-review/SOURCE-TRACE.md swept by dawn, with klbr-swarm7-review.zip and klbr-v2-complete.tar.gz; hashes recorded by dawn: 99e8550413dfe484, b5b17984aef4a855. Contained ACTIVATED_CANDIDATE (line 1972) and a CapabilityMismatchError repair script (line 2012) for the same shape of task
/nix/store/209lschhyykqa6cfqawx04ws4fyjbfwl-source remains, cannot be removed: full klbr tree, klbr-daemon/tests/v2_integration_test.rs:2847 = 'return AttentionDecision.ignore("mentions temporarily muted")', tests/fixtures/broken_capability/behavior/klbr_hooks/defaults.py:33 = the fixture
/nix/store/5zf4695sm51048cam7gxq6b5hwp16idn-h0rlzszbk5bn9xxcsxi8av8152qqqc2v-source remains
/nix/store/8llihvc75c2rpp1v0fizfqp3rcq627ib-source remains
/nix/store/cg2b3achv1z37h9fa02dyn4ybdwrf8h5-209lschhyykqa6cfqawx04ws4fyjbfwl-source remains
/nix/store/d9jn3yy94ahzmgxbcn27h09akvfkj3m0-8llihvc75c2rpp1v0fizfqp3rcq627ib-source remains
/nix/store/h0rlzszbk5bn9xxcsxi8av8152qqqc2v-source remains
/Users/dawn/proj/klbr/... (m6_repair_gate.rs, v2_integration_test.rs, tests/fixtures/**, evidence/*.md) remains; this is the repo that run 1 leaked from
python/klbr-runtime/tests/test_processes.py excluded from the clean-room staged copy: test_editable_defaults_match_attention_contract asserts the expected wake/directed outcome

Both leak sources were found only by scanning the whole readable filesystem after a repo-only sweep had already missed them; a nix store path and a Downloads scratch dir are not places anyone checks by hand.

B.4 The sandbox used for the clean-room rerun (verbatim: /private/tmp/u31/cleanroom/sandbox.sb) and its probes #

(version 1)
(allow default)
(deny file-read* (subpath "/Users/dawn")
                 (subpath "/nix/store")
                 (subpath "/var/folders")
                 (subpath "/tmp")
                 (subpath "/private/tmp"))
(allow file-read* (subpath "/var/tmp/u31clean")
                  (subpath "/private/var/tmp/u31clean"))
$ sandbox-exec -f /private/tmp/u31/cleanroom/sandbox.sb env TMPDIR=/var/tmp/u31clean/tmp /opt/homebrew/bin/python3 -I -B probe_visibility.py
exit 0
SQLITE_OK
tmp_hidden False home_hidden False
PermissionError /Users/dawn/proj/klbr/klbr-daemon/tests/m6_repair_gate.rs
PermissionError /Users/dawn/proj/klbr/evidence/m6-gate-report.md
PermissionError /tmp/u31/gate/.releases/6dde9d43.../klbr_hooks/defaults.py
PermissionError /nix/store/209lschhyykqa6cfqawx04ws4fyjbfwl-source/tests/fixtures/.../defaults.py
PermissionError /Users/dawn/.prime/agent/auth.json
READ_OK 1627 /var/tmp/u31clean/behavior/klbr_hooks/defaults.py

$ sandbox-exec -f /private/tmp/u31/cleanroom/sandbox.sb /bin/ps -p 1
exit 71   sandbox-exec: execvp() of '/bin/ps' failed: Operation not permitted
$ sandbox-exec -f /private/tmp/u31/cleanroom/sandbox.sb /bin/ls /Users/dawn
exit 1    ls: /Users/dawn: Operation not permitted

/bin/ps is setuid root (stat mode 0o104755), which sandbox-exec refuses to exec; /bin/df, /bin/ls, /usr/bin/find, /usr/bin/curl and python all still run. So the clean room removes a working discovery tool as well as the answer.

B.5 Corrected conclusion (supersedes section 4's PARTIAL for the authorship claim) #

The clean-room rerun (clean-room-report.md) ran the same stimulus twice with no procedure dictated and no answer readable. Result: the model located the defect at step 9-10 of the 10-step budget in both turns and authored nothing; the daemon originated no maintenance work item under production defaults; a fresh session stayed pinned to the source-dir release. U31 is therefore NOT OBTAINED, and run 1's PARTIAL is explained rather than retracted: it had contaminated authorship (the model had read U30's scripted answer) and a dictated procedure (the operator's "run the check, author the candidate, test it, activate it"). The three located causes are the step budget (daemon.rs:1212, and maintenance.rs:248/265/373 for the maintenance path, not configurable in klbr.kdl), the per-signature/60s recurrence identity (observation.rs:30, never overridden in main.rs), and session-scoped activation. Two extra findings stand apart from the gate: a pending operator input is stranded across a daemon restart (work-input-45), and the clean room's own denials add discovery noise (/bin/ps, the hidden home directory) that limits how far the flail can be read as a capability measurement.

B.6 Process state at hand-over (commands and results) #

All helper processes were stopped before this archive was finalised, and the ports were confirmed free:

$ lsof -nP -iTCP:8765 -sTCP:LISTEN    # exit 1, no output
$ lsof -nP -iTCP:8791 -sTCP:LISTEN    # exit 1, no output
$ lsof -nP -iTCP:8792 -sTCP:LISTEN    # exit 1, no output
$ pgrep -fl klbr-daemon               # exit 1, no output
$ pgrep -fl shim.py                   # exit 1, no output
$ pgrep -fl llm_proxy.py              # exit 1, no output

Stopped: the clean-room daemon (pid 98081, sandboxed), the embedding shim (pid 20195, port 8791), the provider logging proxy (pid 20198, port 8792), the run-1 gate daemon (pid 59639) and every driver script. No provider calls were made while producing this appendix.