# W5 Evaluation Report: Capability Preservation & Autonomous Improvement

**Session / Milestone:** W5 Fourth-Swarm Review Response: Connected Capability Preservation Loop & Deterministic Gate  
**Author:** swarm6-integration (Workstream A & Integration Owner)  
**Date:** 2026-09-07  
**Repository:** `/Users/dawn/proj/klbr`  
**Git Commit / Reference:** `f0f74b6` + Fourth-Swarm Response commits  

---

## 1. Executive Summary

This report delivers the fourth-swarm review response for the **Capability Preservation and Autonomous Improvement Subsystem** in `klbr`.

Addressing the review findings (R1–R6, `REVIEW.md`, `NEXT-SWARM.md`), the implementation moves capability preservation from isolated value types and helpers into the authoritative production daemon and runtime paths:
1. **Durable Run Causes & SQLite Migration 003:** Added migration `003_work.sql` creating `work_items`, `observer_cursors`, and `observer_cases` tables. Persists cause identity, kind (`InputFrame`, `DeadlineWake`, `MaintenanceReady`, `CheckDue`), readiness, evidence refs, consequence metadata, claim/attempt ownership, and retirement disposition.
2. **Atomic Observer Cursor & Case Persistence:** Observer cursor advancement and failure case/work row insertion are committed together in **one** atomic SQLite transaction (`Store::commit_observation_case_with_cursor`). A crash between cursor movement and work creation cannot lose work.
3. **Production Scheduler Selection:** The production daemon runner selects eligible persisted work via `klbr_runtime::work::select_next_work_item`. Priority is computed strictly from consequence facts (impact, urgency, aging); author origin is metadata, never a priority or trust rank.
4. **Autonomous Cause Consumption:** Autonomous causes (`DeadlineWake`, `MaintenanceReady`, `CheckDue`) are consumed without inventing fake user messages.
5. **Restart Recovery & Startup Scan:** Orphaned claims from killed processes are cleanly transitioned to failed (`host restarted; attempt orphaned`). A startup scan and periodic driver find sessions with pending runnable work items.
6. **Coherent Runner Retirement:** `retire_runner` atomically clears the active runner slot and resets `SessionState::Executing` back to `Ready` only when the active attempt ID matches.
7. **Elimination of Test-Local Doubles:** Deleted test-local fake structs (`WorkCategory`, `WorkCandidate::score`, `SchedulableTask`, `CheckStatus`) from `klbr-daemon/tests/v2_integration_test.rs`. The tests now exercise real production APIs from `klbr_runtime::work` and `klbr_core::capability_check`.
8. **The Single Deterministic End-to-End Gate Test:** Added and verified `single_deterministic_e2e_gate::test_single_deterministic_e2e_gate_crosses_all_boundaries` in `klbr-daemon/tests/v2_integration_test.rs`, crossing all 7 capability preservation boundaries using production APIs with negative fault injections.
9. **Honest Completion Record:** Under the Honesty Gate (HARD RULE 2), each acceptance scenario distinguishes: (a) source/helper implementation, (b) deterministic end-to-end execution, (c) real model authorship, and (d) later observed benefit. G10, G11, G20, and G26 are explicitly downgraded: autonomous model authorship and later longitudinal benefits are marked `behavioral gate: NOT RUN`.

---

## 2. Test Commands & Evidence Receipts

### 2.1 Workspace Rust Test Gate

```text
Command: cargo test --workspace
Result: PASS
Details:
  - klbr-runtime: 4 unit tests, 4 protocol tests, 9 session tests, 7 store tests (24 passed)
  - klbr-daemon: 29 integration tests (all passed, including deterministic gate)
  - klbr-core: 240 tests passed
  - klbr-ipc: 16 tests passed
  - antigravity-bridge: 9 tests passed
```

### 2.2 The Single Deterministic End-to-End Gate Test Receipt

```text
Command: cargo test -p klbr-daemon --test v2_integration_test -- test_single_deterministic_e2e_gate_crosses_all_boundaries --nocapture
Result: PASS (1 passed; 0 failed; finished in 2.84s)

Boundaries Verified:
  1. Genuine observation from tool fault (no user refinement request) + negative injection (failed commit preserves cursor & work state)
  2. One persisted maintenance item through real observer path (atomic transaction with cursor advance)
  3. Daemon/store restart; item survives and is selected by production scheduler (select_next_work_item)
  4. Captured provider request with maintenance purpose, maintenance-attempt template, and evidence references
  5. Deterministic fake actor executes candidate edit/test flow (scripted plumbing evidence, not autonomous diagnosis)
  6. Host check and activation; negative injection (tampered release rejected); post-restart session adopts new revision without injected lessons
  7. Canonical health/work/activation events populate diagnostics, survive replay/reload, and never become speech
```

### 2.3 Python Runtime & Regression Gate

```text
Command: python3 tools/check_runtime.py --rust
Result: PASS (120 tests passed in 18.06s; wheel packaging smoke verified; rust tests verified; clippy verified)

Command: python3 -B /Users/dawn/Downloads/klbr-swarm4-review/repros/test_regressions.py /Users/dawn/proj/klbr
Result: PASS (5/5 passed: distinct group keys, command permission isolation, candidate path overlap rejection, unavailable receipt retry rejection, unrelated error trace rejection)

Command: /tmp/klbr-pkg-venv/bin/python -B /Users/dawn/Downloads/klbr-swarm4-review/repros/packaging_smoke.py /Users/dawn/proj/klbr
Result: PASS (wheel contains standing-responsibility.md, maintenance-attempt.md, evaluation-review.md; imports cleanly)
```

### 2.4 Web Client Diagnostics Gate (`klbr-web`)

```text
Command: bun test
Result: PASS (45 passed, 0 failed, 4,287 assertions across 3 test files in 900ms)

Command: bun run check
Result: PASS (0 errors, 0 warnings)

Command: bun run build
Result: PASS (dist assets generated cleanly in 1.34s)
```

---

## 3. Detailed Acceptance Scenario Audit (G01–G26)

Every scenario is classified with the four required honesty labels:
- **(a) source/helper implementation**: data structures, algorithms, or standalone modules.
- **(b) deterministic end-to-end execution**: executed against the real runtime daemon, store, or protocol pipeline with verified receipts.
- **(c) real model authorship**: autonomous model diagnosis and repair creation.
- **(d) later observed benefit**: demonstrated longitudinal transfer in subsequent sessions without human intervention.

| ID | Title | Honesty Classification | Status | Evidence / Receipt |
|---|---|---|---|---|
| **G01** | Autonomous deadline wake | (b) deterministic end-to-end execution | **PASSED** | `v2_integration_test.rs::review_deadline_wake_makes_another_model_request_without_new_input` |
| **G02** | Causal transcript grouping | (b) deterministic end-to-end execution | **PASSED** | `v2_integration_test.rs::review_interleaved_input_preserves_causal_tool_group_on_reconstruction` |
| **G03** | Tool-result commit required | (b) deterministic end-to-end execution | **PASSED** | `v2_integration_test.rs::review_failed_tool_result_commit_prevents_next_model_step` |
| **G04** | Recovery when maintenance blocks | (b) deterministic end-to-end execution | **PASSED** | `v2_integration_test.rs::g04_recovery_remains_available_when_maintenance_blocks` |
| **G05** | Origin symmetry | (b) deterministic end-to-end execution | **PASSED** | `v2_integration_test.rs::g05_origin_symmetry_human_and_agent_identical_treatment` |
| **G06** | Reliability repair outranks feature | (b) deterministic end-to-end execution | **PASSED** | `v2_integration_test.rs::g06_self_diagnosed_reliability_repair_outranks_feature` (calls production `select_next_work_item`; test-local `WorkCandidate` deleted) |
| **G07** | Checkup finds silent degradation | (a) source/helper + (b) deterministic execution | **PASSED** | `v2_integration_test.rs::g07_checkup_detects_silent_knowledge_delivery_degradation` |
| **G08** | Aging prevents maintenance starvation | (b) deterministic end-to-end execution | **PASSED** | `v2_integration_test.rs::g08_aging_and_fairness_prevents_maintenance_starvation` (calls production `project_priority_score` and `select_next_work_item`; test-local `SchedulableTask` deleted) |
| **G09** | Observer retries retain evidence | (b) deterministic end-to-end execution | **PASSED** | `v2_integration_test.rs::g09_observer_retries_retain_evidence_without_multiplying_work` & `Store::commit_observation_case_with_cursor` |
| **G10** | Agent originates API repair | (a) source/helper + (b) deterministic execution via scripted plumbing; (c) model authorship: NOT RUN; (d) observed benefit: NOT RUN | **DOWNGRADED** | Scripted plumbing verified in `single_deterministic_e2e_gate`; behavioral gate: **NOT RUN** |
| **G11** | Natural correction transfers across restart | (a) source/helper + (b) deterministic execution; (c) model authorship: NOT RUN; (d) observed benefit: NOT RUN | **DOWNGRADED** | Post-restart candidate adoption verified in `single_deterministic_e2e_gate`; behavioral gate: **NOT RUN** |
| **G12** | Context omission explicit | (a) source/helper + (b) deterministic execution | **PASSED** | `v2_integration_test.rs::g12_context_omission_is_explicit_and_sources_remain_recoverable` |
| **G13** | Unknown/stale checks do not certify health | (b) deterministic end-to-end execution | **PASSED** | `v2_integration_test.rs::g13_stale_and_unknown_checks_do_not_certify_health` (calls production `CheckResult` & `DerivedStatus`; test-local `CheckStatus` deleted) |
| **G14** | Accidental await does not repeat | (a) source/helper implementation | **PASSED** | `test_maintenance.py::test_sync_awaited_mutation_does_not_repeat` |
| **G15** | Reject invalid audit diagnosis | (a) source/helper implementation | **PASSED** | `test_maintenance.py::test_targeted_test_claim_contradiction_rejects_global_ban` |
| **G16** | Healthy checkups end without change | (b) deterministic end-to-end execution | **PASSED** | `v2_integration_test.rs::g16_healthy_checkups_end_without_unnecessary_change` |
| **G17** | Durable cursor advancement | (b) deterministic end-to-end execution | **PASSED** | `Store::commit_observation_case_with_cursor` in `single_deterministic_e2e_gate` |
| **G18** | Host activation & pinning | (b) deterministic end-to-end execution | **PASSED** | `v2_integration_test.rs::g18_host_activation_preserves_pinning_and_rejects_tampered` |
| **G19** | Regression triggers rollback | (b) deterministic end-to-end execution | **PASSED** | `session.rs` epoch CAS validation & rollback tests |
| **G20** | Benefit measured against opportunities | (a) source/helper implementation; (c) model authorship: NOT RUN; (d) observed benefit: NOT RUN | **DOWNGRADED** | `ContextReceipt` opportunity fields defined; behavioral gate: **NOT RUN** |
| **G21** | Restore capability without losing issue | (a) source/helper implementation | **PASSED** | `work.rs` consequence dependency tracking |
| **G22** | Maintainability & resource checkup | (b) deterministic end-to-end execution | **PASSED** | `v2_integration_test.rs::g22_maintainability_and_resource_access_checkup` |
| **G23** | SourceRef canonical dedupe | (b) deterministic end-to-end execution | **PASSED** | `v2_integration_test.rs::test_retry_after_lost_ack_deduplicates_same_id` |
| **G24** | Bounded maintenance runner | (b) deterministic end-to-end execution | **PASSED** | `runtime_turn::integrator` supervisor tests |
| **G25** | Diagnostics are not speech | (b) deterministic end-to-end execution | **PASSED** | `single_deterministic_e2e_gate` & `streaming-v2.test.ts` |
| **G26** | Evaluation checks useful work | (a) source/helper implementation; (c) model authorship: NOT RUN; (d) observed benefit: NOT RUN | **DOWNGRADED** | Evaluation harness exists; behavioral gate: **NOT RUN** |

---

## 4. The Single Deterministic End-to-End Gate Verification

The single gate test (`single_deterministic_e2e_gate::test_single_deterministic_e2e_gate_crosses_all_boundaries`) verifies the complete connected capability preservation lifecycle using production APIs only:

1. **Genuine Observation & Negative Injection:** An executed tool fault (`git.diff` with signature `TypeError: unexpected kwarg 'cached'`) produces an observation without user refinement requests. Negative injection: an invalid foreign-key session ID aborts `commit_observation_case_with_cursor`, proving cursor does not advance and work item is not inserted on failure.
2. **Atomic Production Observer Path:** `klbr_daemon::daemon::observe_turn_failure` processes the fault into an `ObserverCaseRecord` and calls `Store::commit_observation_case_with_cursor` in ONE atomic SQLite transaction, advancing the cursor, recording the case, and creating the `WorkItem` (`RunCause::MaintenanceReady`).
3. **Restart Recovery & Scheduling:** The store is cleanly closed and reopened from disk. `recover_startup_work_items` scans and identifies the session with pending work. The production consequence scheduler `select_next_work_item` selects the maintenance repair item.
4. **Captured Provider Request with Maintenance Purpose:** The item is claimed under an `OperationId`. Messages are assembled using the single shared production function `MaintenanceAttemptContext::assemble_messages`. Turn execution runs via `run_turn_loop_with_purpose(..., Purpose::Maintenance)`. The fake provider captures the actual HTTP request body, asserting presence of:
   - Template marker: `Fresh-context maintenance attempt`
   - Exact evidence reference ID: `ref id="event:1"`
   - Assignment purpose: `purpose="repair"`
   - Structured assignment block: `<maintenance_assignment`
5. **Scripted Candidate Authoring:** A candidate workspace authors a patch into `klbr_hooks/defaults.py` and publishes a content-addressed release `rel_v2` (distinct SHA). Labeled strictly as scripted plumbing evidence, not autonomous diagnosis.
6. **Host Activation & Tamper Rejection:** A tampered hook release with syntax defects is rejected by host validation. The valid release `rel_v2` is activated via registry CAS, advancing epoch. The session is stopped and reopened; the post-restart session runs on the new release without special test lessons.
7. **Production Projection & Lane Purity:** Canonical lifecycle events are retrieved and projected through the production `query_events_after` -> `project_event` pipeline. The resulting `V2EventRecord` records produce `Lane::Lifecycle` carrying `{kind, data}` matching the exact shape expected by `klbr-web`. Verified that ZERO events project to `Lane::Speech`, and all diagnostic events survive history replay.

---


### 4.1 Residual Not-Wired Helpers

To maintain complete transparency with future reviewers, the following helpers have been implemented and unit/integration tested, but currently have **NO production caller** in the live daemon runtime:

| Component | Current Status | Finding / Detail | Next Owner |
|---|---|---|---|
| `klbr_core::context_receipt::GuidanceAssemblyEngine` | No production caller | Context receipt recording at the real provider-request assembler is not yet wired; the daemon maintenance/turn assembler does not record a version-bound receipt yet | Evidence / Context Lead |
| `klbr_core::capability_check::ObservationProcessor::process_new_events` | No production caller | The daemon observes failures directly via `observe_turn_failure`; the multi-turn batch grouping processor itself remains unused in production | Observer / Background Worker Lead |
| `check_guidance_delivery` / `check_guidance_delivery_request` | No scheduled production caller | No periodic capability checker runs in the daemon; checks are currently executed on-demand in tests | Capability Checker / Maintenance Lead |

## 5. Summary of Resolutions to Fourth-Swarm Review

- **R1 (Durable Run Causes & Scheduling):** Resolved via Migration 003, `Store::insert_work_item`, `commit_observation_case_with_cursor`, `claim_work_item`, `complete_work_item`, `recover_startup_work_items`, and production daemon runner selection via `select_next_work_item`.
- **R2 (Completion Record Correction):** Resolved by deleting test-local fake structs (`WorkCategory`, `WorkCandidate`, `SchedulableTask`, `CheckStatus`), replacing them with production types, and honestly downgrading G10, G11, G20, G26 with explicit honesty labels.
- **R3 (Candidate Workspace Isolation):** Resolved via ancestor/descendant/symlink path overlap rejection in `test_regressions.py` (5/5 passed).
- **R4 (Diagnostic Certainty & Replay):** Resolved by rejecting unauthorized retry on unavailable receipts and ensuring unpersisted events do not authorise mutations.
- **R5 (Wheel Packaging Smoke):** Resolved via packaging fix; all three markdown templates included and loadable outside repo.
- **R6 (Client Diagnostics Consumption):** Resolved; client parses canonical Rust `{kind, data}` lifecycle envelope, survives reload/reconnect, and never pollutes speech store.