# 03 — phases, measured outcomes, risks & open questions

## 7. phases

| phase | where | what | exit criterion |
|---|---|---|---|
| 0 | laptop, desktop | baseline runs + hardware constants (§6.2–6.3) | numbers for SSD GB/s, compute ceiling, thermal sag, per-turn time |
| 1 | Colab | traces, replayer, prune variants, quality evals, MoCE closed loop, SCR staleness (§6.1) | quality vs K table, misses vs (K, λ, policy, cache size) table |
| 2 | local | latency model = phase 1 misses × phase 0 constants. pick K, λ, policies. validate top 2–3 on the Air | chosen config hits the turn-time bar |
| 3 | Colab or rented GPU | data gen (persona + abstention), healing + persona LoRA on dense/router/shared with experts frozen, merge, quantize dense only, transplant into the pruned Swift GGUF | evals ≥ the untuned pruned model on agent tasks; fabrication down; persona score up |
| 4 | same | slop mining on tuned outputs vs dawn's text → FTPO | slop rate down, no eval regression |
| 5 | local, ongoing | runtime squeeze (§2.5 table), each technique A/B'd with its exactness class written down | — |

### 7.1 measured outcomes & finalist decision (phases 0, 1a, 1b, 1c, 2)

1. **phase 0 (13" M5 Air 24 GB / 1 TB SSD hardware measurements, `bench/README.md`):**
   - `iostat -Id` verified 100% physical NVMe reads (`disk0` delta == 100% of requested MiB on 1 GiB virgin slices of a 12 GiB file): **2.41 GB/s** (1-thread standard GGUF), **3.90 GB/s** (1-thread `.experts`), **6.80 GB/s** (2–4 thread `pread` fan-out).
   - M5 10-core GPU sustained FP16 GEMM: **14.07 TFLOP/s** (`356 tok/s` compute ceiling) with **0.86% thermal sag** over 60s continuous load (`14.00 -> 13.88 TFLOP/s`).
2. **phase 1a (131,072-token multi-domain REAP calibration + 8,192-token OOS verification, `results/reap_saliency_v2.json`, `results/oos_prune_retention.json`):**
   - 99.97% expert coverage (`1,310,720` picks/layer, `0` dead `0`-hit experts kept across all 9 keep-maps after the zero-prior damped LCB fix).
   - REAP L2 retention orders `reap_l2_sum > reap_hybrid > reap_shrink` at every `K ∈ {320, 256, 224}` both **in-sample** (`91.76% / 85.28% / 81.11%` for `reap_l2_sum`) and **out-of-sample** on 8,192 strictly held-out tokens (`87.18% / 79.71% / 75.27%` for `reap_l2_sum`, `86.54% / 78.16% / 72.98%` for `reap_hybrid`).
3. **phase 1b (240-config in-sample + OOS closed-loop MoCE & cache replay, `results/phase1b_heldout_cache_moce_sweep.json`):**
   - OOS baseline cache hit rate matches in-sample within 0.05% (`82.32%` OOS vs `82.37%` in-sample on `K512` LRU).
   - on the M5 Air (`cache_cap = 113` experts/layer = `7.5 GB` in `Q2_0`), `lru` + closed-loop MoCE (`λ = 0.09`, `protect_top_j = 3`) beats every other tested online eviction policy at `λ = 0.09` (`near_pick_ema`, `ema`, `inverse_router_ema`, `lfu`; `belady_min` is the offline future-oracle ceiling), cutting OOS misses/token from `84.84` (`K512` LRU `λ = 0`) to **`53.95` (`K320_reap_hybrid`, `88.76%` hit rate, `32.1–43.5 tok/s` decode)**, **`56.29` (`K320_reap_l2_sum`, `88.27%` hit rate, `31.3–42.7 tok/s` decode)**, and **`44.65` (`K256_reap_l2_sum`, `90.70%` hit rate, `35.9–47.4 tok/s` decode — lower than unpruned no-MoCE Belady's MIN oracle `46.13` by combining pruning + closed-loop routing bias)**. (Note: Tier-1 KLD/top-1 and Tier-2 below were evaluated at `λ = 0` standard routing; the quality impact of `λ = 0.09` MoCE routing bias is evaluated during Phase 5 runtime integration.)
4. **phase 1c (Tier-1 KLD + Top-1 screen & Tier-2 20-task behavioral smoke suite, `results/phase1c_kld_results.json`, `results/phase1c_tier2_results.json`, `audits/opus_audit_phase1c.md`, `audits/opus_audit_phase1c_v2.md`):**
   - **at `K = 320` (`13.73 GB` non-resident `Q2_0`, `302.9 tok/s` 4K cold prefill):** `K320_reap_hybrid` (`0.3941` macro KLD, `83.20%` top-1 agreement) and `K320_reap_l2_sum` (`0.4178` macro KLD, `82.35%` top-1 agreement) are statistically tied on `ops_tools` (`0.600` vs `0.627`, `z = -0.77` ns) and `chat_json` (`0.305` vs `0.304`, `z = +0.07` ns), with `K320_reap_hybrid` marginally ahead on `ja` (`0.277` vs `0.323`, `z = -2.63`).
   - **at `K = 256` (`9.49 GB` non-resident `Q2_0`, `317.5 tok/s` 4K cold prefill):** `K256_reap_l2_sum` (`0.5642` macro KLD, `79.76%` top-1) decisively beats `K256_reap_hybrid` (`0.7485` macro KLD; `ops_tools` `z = -8.67`, `ja` `z = -3.48`). against `K256_reap_shrink` (`0.6069` macro KLD), `K256_reap_shrink` has lower `ops_tools` KLD (`0.796` vs `0.895`, `z = +2.27`) while `K256_reap_l2_sum` wins `ja` (`0.350` vs `0.520`, `z = -8.03`), `chat_json` (`0.447` vs `0.504`, `z = -2.17`), and `ops_tools` top-1 (`76.64%` vs `76.37%`).
   - **at `K = 224` (`7.36 GB` non-resident `Q2_0`):** `K224_reap_l2_sum` (`0.6667` macro KLD, `77.62%` top-1) degrades significantly vs `K256_reap_l2_sum` on both `ops_tools` (`z = -3.94`) and `ja` (`z = -3.92`).
   - **Tier-2 20-task smoke suite (`v1_run_rescored_ja_trunc_only`, `n = 1` seed at `max_tokens = 2048`):** all pruned variants score **`18–19/20 (90–95%)`** vs `20/20` unpruned (pairwise Fisher exact `p >= 0.23`, so Tier-2 is a non-regression smoke check with no pairwise ranking power; `4` of the `7` pruned failures are 2048-token cap truncations).
   - **finalist checkpoints uploaded to `90-008/Swift-1.5-Qwen3.8-Flash-Next-REAP-Pruned-GGUF`:**
     1. **`K320_reap_hybrid`** (and `K320_reap_l2_sum` keep-map): primary quality finalist (`0.394` macro KLD, `83.2%` top-1, `2.74s` 256-tok cold prefill floor, `32.1–43.5 tok/s` warm decode w/ `lru+MoCE` on M5 Air).
     2. **`K256_reap_l2_sum`**: primary speed/50%-pruned finalist (`0.564` macro KLD, `79.8%` top-1, `2.11s` 256-tok cold prefill floor, `35.9–47.4 tok/s` warm decode w/ `lru+MoCE` on M5 Air, fits in a single 80 GB A100 in int8 for Phase 3 LoRA healing without expert streaming).
     3. **`K224_reap_l2_sum`**: lower-bound size reference (`7.36 GB` non-resident, `39.6–51.0 tok/s` warm decode).
5. **phase 0 + phase 2 local validation on 13" M5 Air 24 GB (`results/phase0_phase2_m5_validation.json`, `scripts/patch_strata_metal.py`, `scripts/prepare_strata_k256.py`, `audits/opus_audit_phase0_2_m5.md`, `audits/opus_audit_phase0_2_m5_v2.md`, `audits/opus_audit_phase0_2_m5_v3.md`, `audits/opus_audit_phase0_2_m5_v4.md`):**
   - all 22 archived runs (`5` Strata speed + `9` Strata teacher-forced KLD + `4` patched `llama.cpp` + `4` unpatched `llama.cpp`) guarded by a 0.25s `vm_stat` + `iostat -Id` watchdog (`min_avail_gib = 2.2 GiB`, `max_swapout_delta_mib = 64.0 MiB`, **`0.0 MiB` swapouts across all 22 runs**):
   - **upstream `llama.cpp` (`llama-cli` chat mode, `-ngl 99 -cmoe -nr -lzm on -n 64`) on `K256_reap_l2_sum`:** `1.40 tok/s` decode (`105,851.73 MB` whole-run `disk0` read at `-ub 1024` and `85,725.73 MB` at `-ub 2048` across load + prompt + `-n 64` generation), `14.30 tok/s` prefill at `-ub 1024` and `21.70 tok/s` at `-ub 2048` on `data/m5_val_prompts/llama_prompt_1264tok.txt` (`5,600` bytes).
   - **patched `llama.cpp` (`llama-completion -no-cnv -t 8`, `scripts/patch_m5_lazy_moe.py`):** `3.22 tok/s` decode on the `22`-token prompt (`33,428.34 MB` whole-run `disk0` read across load + 22 prompt + 63 decode tokens), `38.11 tok/s` prefill at `-ub 1024` (`46,808.58 MB` whole-run `disk0`, hitting EOS after prefill with `0` decode tokens) and `63.27 tok/s` at `-ub 2048` (`28,272.94 MB` whole-run `disk0`, prefill only) on `data/m5_val_prompts/llama_prompt_1264tok.txt` (not a like-for-like `disk0` / decode comparison against unpatched `llama-cli` chat mode), while `--moe-cache-mib 1536` degrades to `0.28 tok/s`.
   - **optimized `Strata` Metal engine (`vendor/strata/build-metal/strata_window_cli`, uniform `3.40 GB` arena across all 14 Strata runs, `K256-q2_0-metal.gguf` + `K256-expert-profile.bin` + `PleTable` Direct I/O):**
     - **startup + arena warm-fill:** **`424.45–504.58 ms` across the `MOCE=0.85` speed runs (`424.45–506.90 ms` across all 14 Strata runs in `results/phase0_phase2_m5_validation.json`; `339.50–356.06 ms` in `audits/opus_audit_phase0_2_m5_v3.md`)** window init + **`0.6–1.3s` (`2.69–5.92 GB/s` across the 10 `Q2_0` runs, `3.98–5.04 GB/s` across the 4 `IQ3_S` runs)** `STRP` profile fill (`2,441` resident expert slots in 1 unified `Q2_0` size class vs `1,550` slots / `1,440` filled in 3 classes on `IQ3_S` in the same `3.40 GB` arena; `2.90 GB` active GPU dense weights `mlock`ed, `0.34 GB` CPU `token_embd` + `1.28 GB` skipped BF16 `hc_*` unpinned in the `4.53 GB` file map, `0.0 MiB` swapouts).
     - **cold prompt prefill (Metal 4 `matmul2d` neural accelerators + `QSA flash` + router GEMM over `n_exp_ = 256`):** **`320.58 tok/s` (`3,194.19 ms`)** at 1,024 tokens and **`353.49 tok/s` (`5,793.65 ms`)** at 2,048 tokens in the earlier `v3` pass (`audits/opus_audit_phase0_2_m5_v3.md`), and **`229.99 tok/s` (`4,452.38 ms`)** at 1,024 tokens and **`268.23 tok/s` (`7,635.20 ms`)** at 2,048 tokens in `results/phase0_phase2_m5_validation.json` (`[hypothesis: passive SoC thermal throttling + concurrent background CPU load]`), with `1.36 GB` transient prompt scratch `mmap`ped without `memset` and `cx_->free`d + `munmap`ped immediately after prefill (`phys_footprint 4.73 -> 4.68 GB`, `resident RSS 7.41 -> 7.36 GB`).
     - **steady-state decode progression on the 43-token SQLite WAL prompt (`n_gen = 64`, `63` decode steps, uniform `3.40 GB` arena):**
       - `K256-metal.gguf` (`IQ3_S`/`IQ4_NL` mixed, 3 size classes, `1,550` slots, `MOCE=0.0`): **`4.88 tok/s` (`205.02 ms/tok`)**, `77.32%` combined hit rate (`52.62%` arena + `24.70%` read-ahead, `108.9 misses/tok`, `585.2 MB/tok` SSD read).
       - `K256-q2_0-metal.gguf` (uniform `Q2_0`, 1 size class, `2,441` slots, `MOCE=0.0`): **`6.32 tok/s` (`158.22 ms/tok`; `9.51 tok/s` in `audits/opus_audit_phase0_2_m5_v3.md`)**, `82.44%` combined hit rate (`63.86%` arena + `18.58%` read-ahead, `84.3 misses/tok`, `304.5 MB/tok` SSD read).
       - `K256-q2_0-metal.gguf` (uniform `Q2_0`, 1 size class, `2,441` slots, `MOCE=0.85`): **`8.04 tok/s` (`124.31 ms/tok`: `34.15 ms` wait + `88.35 ms` GPU; `13.52 tok/s` with `16.11 ms` wait + `56.29 ms` GPU in `audits/opus_audit_phase0_2_m5_v3.md`)**, **`91.48%` combined hit rate** (`82.34%` arena + `9.14%` read-ahead, **`40.9 misses/tok`** = **−51.5% vs `Q2_0 MOCE=0.0`** and **−62.4% vs `IQ3_S MOCE=0.0`**, `127.2 MB/tok` SSD read).
     - **steady-state decode after long prompts (`K256-q2_0-metal.gguf`, `MOCE=0.85`, `3.40 GB` arena, `results/phase0_phase2_m5_validation.json`):** **`9.33 tok/s` (`107.20 ms/tok`: `30.78 ms` wait + `73.87 ms` GPU)** after 1,024-token prefill (`92.63%` combined hit rate: `84.38%` arena + `8.25%` read-ahead, `35.4 misses/tok`, `115.3 MB/tok` SSD read) and **`8.37 tok/s` (`119.51 ms/tok`: `24.38 ms` wait + `92.44 ms` GPU)** after 2,048-token prefill (`93.84%` combined hit rate: `86.70%` arena + `7.14%` read-ahead, `29.6 misses/tok`, `97.2 MB/tok` SSD read).
     - **3-domain teacher-forced full-vocabulary (`248,320`) FP64 KLD, PPL & top-1 quality decomposition on complete untruncated turns (`n = 64` final assistant tokens × 1 complete turn/domain = `192` evaluated assistant tokens after `n_prompt = 244 / 159 / 456` prompt tokens in `data/m5_val_prompts/eval_tf_{ops_tools,chat_json,ja}.txt`, scoring the final 64 visible JSON tokens on `ops_tools` and final `<think>` reasoning + `</think>` + complete visible JSON/Japanese output on `chat_json` and `ja`, with `0` `<think>` template tokens in any scored span, relative to the pruned `K256_reap_l2_sum` `IQ3_S`/`IQ4_NL` reference):**
       - **isolated `MOCE=0.85` reranking (`Q2_0 MOCE=0.85` vs `Q2_0 MOCE=0.0`):** **`0.0588` macro mean KLD** (`0.1477` `ops_tools`, `0.0052` `chat_json`, `0.0234` `ja`) and **`96.88%` macro top-1 agreement** (`92.19%` `ops_tools`, `100.00%` `chat_json`, `98.44%` `ja`; PPL `3.6716 -> 4.1013` `ops_tools`, `1.1808 -> 1.1993` `chat_json`, `1.9031 -> 1.9188` `ja`).
       - **unhealed `IQ3_S/IQ4_NL -> Q2_0` expert conversion (`Q2_0 MOCE=0.0` vs `IQ3_S MOCE=0.0`):** **`0.2032` macro mean KLD** (`0.4621` `ops_tools`, `0.0395` `chat_json`, `0.1081` `ja`) and **`90.63%` macro top-1 agreement** (`79.69%` / `100.00%` / `92.19%`; PPL `2.6114 -> 3.6716` `ops_tools`, `1.1725 -> 1.1808` `chat_json`, `1.6488 -> 1.9031` `ja`), motivating Phase 3's LoRA healing against the deployed `Q2_0` expert weights.
       - **combined `Q2_0 + MOCE=0.85` vs `IQ3_S + MOCE=0.0`:** **`0.2386` macro mean KLD** (`0.5640` `ops_tools`, `0.0406` `chat_json`, `0.1112` `ja`) and **`89.58%` macro top-1 agreement** (`76.56%` / `100.00%` / `92.19%`; PPL `2.6114 -> 4.1013` `ops_tools`, `1.1725 -> 1.1993` `chat_json`, `1.6488 -> 1.9188` `ja`).
6. **phase 3 local healing, byte-level GGUF transplant & multi-prompt generalization on 13" M5 Air 24 GB (`results/phase3_local_healing_validation.json`, `results/phase3_22win_kld_audit_v1.json`, `results/phase3_22win_kld_audit_v2.json`, `results/phase3_9prompt_speed_audit_v1.json`, `results/phase3_9prompt_speed_audit_v2.json`, `scripts/heal_k256_q2_0.cpp`, `scripts/run_phase3_local_healing.py`, `audits/opus_audit_phase3_local_v1.md`, `audits/opus_audit_phase3_local_v2.md`):**
   - **Signed-d importance-weighted Lloyd-Max `Q2_0` expert quantization + real-trace per-layer `alpha_l` calibration (`T = 1,022`, `gamma = 0.00`) + shared-expert ridge solver (`blk.0..47.ffn_down_shexp.weight`):**
     - Unlocks the IEEE `fp16` sign bit of `d` in `GGML_TYPE_Q2_0` (type `42`), weighting `W_gate`/`W_up` by tempered real activation RMS `(E[mixed_d^2])^0.5` (`beta = 0.50`) and `W_down` by per-expert analytical fourth-root importance `(sigma_g * sigma_u)^0.5 ~ (E[h_c^2])^0.25` (`beta = 0.25` on `W_down` vs `beta = 0.50` on `W_gate`/`W_up`) over the 1,022-token multi-domain calibration trace (`data/m5_val_prompts/cal_disjoint_1024tok.txt`; `0` 16-gram overlap on scored spans across all 22 unseen windows and 3 canary windows; 18/22 unseen windows have `0` full-window context overlap, while 4 windows share prompt context: `<tools>` boilerplate in `cj20_tool_5turn_t3`/`cj20_t3_answer_tool`/`cj03_toolcall` and another turn of `persona` conversation 3 in `persona3_t4`).
     - Calibrates exact real-trace MSE scales `alpha_l = <y_ref, y_q2> / ||y_q2||_2^2` (`L0 = 1.308`, `L24 = 1.022`, `L47 = 1.121` vs unseen held-out oracle `L0 = 1.337`, `L24 = 1.021`, `L47 = 1.134`; only `2/48` layers worse than `alpha = 1` on unseen data vs `22/48` for Gaussian `v1`, reducing unseen routed residual from `24.13%` to `23.37%`), solves ridge regression (`46/48` layers full-rank `640/640`; `L0` rank `430`, `L2` rank `617`; `eta_shd = 0.15`, `ridge_rel = 6.0`) on `blk.0..47.ffn_down_shexp.weight` (in-sample `MoE MSE 19,061.40 -> 18,665.36 -> 18,460.89`, `-3.15%` total: `-2.08%` from `alpha_l` + `-1.10%` from ridge after `IQ4_NL` requant; `-0.9%` LOBO CV, `-0.46%` on unseen dump), and transplants the `48` modified `ffn_down_shexp` tensors into `K256-q2_0-healed-metal.gguf` (`1175 / 1223` other tensors bit-identical).
   - **Full 22-window (`2686`-token) 4-stratum unseen KLD audit on `v2` & `v1` (`results/phase3_22win_kld_audit_v1.json`, `results/phase3_22win_kld_audit_v2.json`, `audits/opus_audit_phase3_local_v2.md`):**
     - Across all 22 unseen windows (`17` unused docs, `2686` scored tokens across `ops`, `chat_json/tool`, `ja`, `persona`), **`v2` healed `Q2_0` beats unhealed `Q2_0` in `20/22` windows at `MOCE=0.0`** (**`-28.55%` [95% CI `-33.5%`, `-23.2%`] pooled KLD**, `0.2029 -> 0.1450`, **`+2.27 pt` top-1**, `85.22% -> 87.49%`) and **`21/22` windows at `MOCE=0.85`** (**`-26.98%` [`-32.0%`, `-22.1%`] pooled KLD**, `0.2123 -> 0.1550`, **`+1.53 pt` top-1**, `85.55% -> 87.08%`).
     - Comparing `v2` vs `v1` across the 22 windows: **`-5.08%` [`-10.8%`, `-0.1%`] at `MOCE=0.0`** (`0.1527 -> 0.1450`), `-3.26%` [`-7.6%`, `+0.4%`] at `MOCE=0.85` (`0.1602 -> 0.1550`), `-3.51%` [`-8.8%`, `+1.5%`] (n.s.) on the 18 context-clean windows (`0.1522 -> 0.1469`), and **`-16.53%` [`-28.3%`, `-4.4%`] on isolated MoCE KLD** (`0.0389 -> 0.0325`, `16/22` windows better). Per stratum (`MOCE=0.0`, `unhealed -> v1 -> v2`): `ops` `0.1650 -> 0.1199 -> 0.1077`, `chat_json` `0.1554 -> 0.1243 -> 0.1130`, `persona` `0.3576 -> 0.2637 -> 0.2527` (`72.1% -> 78.4% -> 77.9%` top-1), and `ja` `0.2374 -> 0.1723 -> 0.1796` (`80.2% -> 85.0% -> 83.7%` top-1, regressed slightly vs `v1` because `cal_disjoint_1024tok.txt` had only ~160 `ja` tokens). On `cj20_tool_5turn_t3` (which shares `<tools>` boilerplate in context), `v2` is tied with unhealed at `MOCE=0.0` (`0.1444` vs `0.1440`, vs `0.1783` in `v1`) and `0.1417` vs `0.1534` at `MOCE=0.85`.
     - On the 3-domain 192-token canary set (`eval_tf_{ops_tools,chat_json,ja}.txt`), `v2` macro KLD drops by **`-23.08%`** (`0.2032 -> 0.1563`, vs `v1`'s `0.1320`, worse than `v1` on the 192-token canary) at `MOCE=0.0`, **`-28.57%`** (`0.2182 -> 0.1559`, vs `v1`'s `0.1365`; `ops_tools` `-29.74%`, `ja` `-36.84%` with `93.75%` top-1, `chat_json` `98.44%` top-1) at `MOCE=0.85`, and **`-57.3%`** on isolated `MOCE=0.85` (`0.0616 -> 0.0263`, vs `v1`'s `0.0287`).
   - **9-prompt diverse speed & hit-rate audit on `v2` (`results/phase3_9prompt_speed_audit_v2.json`) + `results/phase3_local_healing_validation.json` speed suite:**
     - Across the 9 diverse prompts (`coldshift_ja2rust`, `ja_pedal_398`, `json_extract_163`, `math_43`, `ops_csharp_713`, `persona_458`, `prompt_1024tok_ops`, `prompt_43tok_wal`, `tool_multiturn_588`), `v2` at `MOCE=0.85` achieves **`92.96%` mean combined hit rate** (`90.88%` on multi-turn tool use to `95.60%` on JSON extraction, `33.8` mean misses/tok vs `85.12%` at `MOCE=0.0`; `v1` was `93.27%`, with `persona_458` moving `93.04% -> 91.26%`, `33.4 -> 41.9` misses/tok) and **`9/9` bit-exact determinism across reps**.
     - On `results/phase3_local_healing_validation.json` (`3.40 GB` arena, `0.0 MiB` swapouts across all 27 runs), `K256-q2_0-healed-metal.gguf` (`MOCE=0.85`) runs at **`284.31 / 324.59 tok/s` prefill** (`1K / 2K`) and **`8.4 / 9.03 / 10.74 tok/s` decode** (`90.93% / 92.13% / 92.55%` combined hit rate, `43.6 / 37.8 / 35.7 misses/tok` vs `84.7` at `MOCE=0.0`).

7. **phase 4 (`#97`) WFLD Layer-47 anti-slop + phase 5 (`#98`) Strata Metal `v3` runtime optimization on 13" M5 Air 24 GB (`results/phase4_colab_validation.json`, `results/phase4_healed_persona_ftpo_writers.bin`, `scripts/solve_phase4_wfld_l47.cpp`, `scripts/run_phase4_colab_ftpo.py`, `scripts/strata_metal_complete.diff`, `audits/opus_strata_metal_opts_v1.md`, `audits/opus_strata_metal_opts_v2.md`, `audits/opus_strata_metal_opts_v3.md`):**
   - **Phase 4 `v3.2` Special-Token-Orthogonal Per-Word-Null WFLD at Layer 47 (`blk.47.ffn_down_shexp.weight`, `47/48` tensors bit-identical to Phase 3, `||dW||/||W|| = 0.0978`) + Prefix-Conditioned Sampler Penalty (`STRATA_ANTI_SLOP_PENALTY=6.0`):**
     - **Bars (a)–(c) Slop Reduction (`audits/opus_strata_metal_opts_v3.md` §6.2 & §6.3):** `-84.14%` on FTPO Train (`n = 28`, `0.6326 -> 0.1003`), `-69.98%` on Same-Doc Held-Out (`n = 14`, `0.8733 -> 0.2622`), `-69.67%` on Dev Separate-Doc Held-Out (`n = 21`, `0.4703 -> 0.1426`, `20/21` new argmax a natural word), and on the **one-shot untouched 1,100-token Blind Multi-Turn Chat Test (`p4_blind_test_1100`, `n = 21`)**: `-54.44%` from weights alone (`0.5543 -> 0.2525`) and **`-98.98%` (`0.5543 -> 0.0056`, `21/21` argmax clean) when stacked with the 0-overhead prefix-conditioned logit penalty in `Window::finish`**.
     - **Bar (d) Per-Word Held-Out Collateral (`p4_collateral_1640`, `n = 142`, `0` 8-gram overlap, bar `>= -0.05` per word & `<= 10.0%` below `-0.10`):** `" end"` `-0.0140` (`0 < -0.1`), `" than"` `-0.0343` (`3 < -0.1`), `" what"` `+0.1598` (`1 < -0.1`), `" version"` `+0.0071` (`3 < -0.1`), `" away"` `+0.0066` (`4 < -0.1`), `" actually"` `-0.0419` (`2 < -0.1`); total `< -0.10` is **`13 / 142 = 9.15% <= 10.0%`** (overall mean `+0.0211` nats).
     - **Bars (f) & (g) Multi-Turn Chat Special-Token & Ordinary-Token Stability (`§6.2` Dev & `§6.3` Blind):** **`0` assistant-side argmax flips to `<|im_end|>`/`<|im_start|>`/`<think>`/`</think>`/`<|endoftext|>`**, `56/56` (`dev`) and `48/48` (`blind`, `+0.002` mean $\Delta\log p$) real turn ends preserved, `p(<|im_end|>)` at slop positions `0.0521 -> 0.0511` (`decreases`), dev `non-slop KLD = 0.00472 <= 0.005` (`ordinary Δlog p = +0.0208 >= -0.01`), and blind `non-slop KLD = 0.0030 <= 0.005` (`ordinary Δlog p = +0.012 >= -0.01`).
     - **Bar (e) Real 48-Layer Strata Metal Engine 3-Domain KLD vs Phase 3 (`<= 0.01`):** `ops_tools` **`0.005273`** (`PPL 3.0165 -> 3.0059`, `top-1 = 96.88%`), `chat_json` **`0.000765`** (`PPL 1.2023 -> 1.1980`, `top-1 = 98.44%`), `ja` **`0.000970`** (`PPL 1.7785 -> 1.7730`, `top-1 = 98.44%`), **3-Domain Macro KLD = `0.002336` (`97.92%` macro top-1)**.
   - **Phase 5 (`#98`) Strata Metal `v3` Runtime Speed & Hit-Rate Progression on 24 GB M5 Air (`ops_1024`, `7.0 GB` Arena, `MLOCK=1, WIRE=1`, `0.0 MiB` swapouts, `audits/opus_strata_metal_opts_v3.md` §1–§4):**
     - **Stock 7.0 GB Arena (`unwired, PF=8`):** `17.03 tok/s` (`58.72 ms/tok`), `96.32%` hit rate (`17.7 misses/tok`, `7.26 ms` SSD wait).
     - **Strata Metal `v3` Defaults (`EXPERT_SPLIT=1, SHEXP_OVERLAP=1, SPLIT_TURN=1, PRED_Q8=1, ROUTER_BF16=1, MOE_DN_TILE=8,2,4, PREWARM=1, PF=6, UNPERT=1`):** **`22.08 tok/s` (`45.28 ms/tok`, `+29.7%` vs `7.0 GB` stock, `+105%` vs Phase 3 `3.4 GB` arena's `10.74 tok/s`, and `15.8x` faster than unpatched `llama.cpp`'s `1.40 tok/s`)**, **`97.59%` combined hit rate** (`92.81%` arena + `4.78%` read-ahead, `11.6 misses/tok`, `5.08 ms` SSD wait, `38.86 ms` pure GPU kernel sum; `22.25 tok/s` / `44.95 ms/tok` and `97.82%` hit rate / `10.5 misses/tok` / `4.61 ms` SSD wait with `PF2=2, IO=8`).

      - **Phase 5 Disk Session Save/Load + Suffix Cache Reuse (`SCR`) with 36-Layer GDN Staleness (`results/phase5_scr_and_session_validation.json`, `audits/opus_phase5_scr_and_persona_v1..v2.md`):** `--save-session` / `--load-session` (`1,983` cells = `139.46 MiB`) saves in `21.70 ms` and loads in **`28.93 ms` cold NVMe (`5.18 GB/s`, `281x` faster than `8,138 ms` cold prefill) / `5.00 ms` warm (`1,627x` faster)** with **`0.000000` KLD (`100.00%` top-1)** at `MOCE=0, EXPERT_SPLIT=0`. On `[A B C] -> [A B' C]` folds (`A=512, B=1024 -> B'=60`), `Window::scr_relocate` relocates + RoPE-rerotates `C` across all 12 QSA layers in **`0.2–0.5 ms`**, and `keep` (`S_ABC`, keeping the post-edit 36-layer GDN state) yields **`+0.0038` nats/tok** (`C=511`, PPL `7.49 -> 7.20`, `92.19%` top-1) to **`+0.038` nats/tok** (`C=1535`, **`3.48x–3.79x` prefill speedup**, `3.20%` repeated 4-grams over 128 greedy tokens) of isolated 36-layer GDN staleness (`lossy J=|C|` sanity rows = `0.000000` KLD), plus `0 / 190` empty turns at `step0` on the 190-pair Real-VN (`Steins;Gate` + `VA-11 Hall-A`) + Persona dataset.

training compute layout for phase 3 [est]:
- **training happens after pruning**, so the model is smaller. at 224 experts: routed experts at int8 ≈ 52.8 GB + dense (~3.6B params) bf16 ≈ 7.3 GB, which fits on **one A100 80 GB with no expert streaming**, leaving ~15–20 GB for activations (chunked loss, short-ish sequences, activation checkpointing).
- **train against the deployed expert weights.** the experts are frozen anyway, so use the exact dequantized GGUF expert values in the training forward. the persona/healing LoRA then also compensates for the deployment quant's error, which is free QAT. the only train/deploy mismatch left is the export quant of the dense tensors. needs a torch kernel for the chosen expert format; Q2_0 is simple scalar (no lookup tables), IQ formats are harder. another point for Q2_0.
- **multiple Colabs:**
  - data generation is embarrassingly parallel and probably most of the compute: persona traces, teacher top-k logits, abstention sampling (several samples per question), slop mining, eval runs, prune-variant evals. one Colab per shard, results to HF.
  - data-parallel training over WAN is feasible only because LoRA grads are small (r=16 on dense projections ≈ 50M params ≈ 100 MB bf16 per sync). use DiLoCo-style local steps + infrequent sync, elastic so a disconnect doesn't kill the run. Colab can't accept inbound connections, so sync goes through a relay/object store. concurrent-session limits per Colab plan: unverified.
- RL is not in the plan (SFT/KD + FTPO). if it ever is, e.g. for agent behavior, Modal's mixed-precision, mixed-accelerator sampler result applies directly: quantized GGUF samplers on Colab/local, BF16 trainer elsewhere. it needed masked importance sampling + score centering + top-p with mask replay, and token-in-token-out. uncorrected runs collapsed, and the collapse sign was the agent **stopping its tool calls**. routing mismatch includes MoE top-k **and the sparse-attention indexer**; MoCE at sampling time would add more [src: modal.com/blog/toward-rl-on-mixed-precision-mixed-accelerator-samplers, 2026-10-13].

other phase 3 notes [est]:
- if we ever train at 320 experts in int8 (~75 GB experts + 7.3 GB dense > 80 GB VRAM) without a Q2_0 dequant kernel, fallback is streaming frozen experts per layer from Colab's ~167 GB host RAM over PCIe with large token batches
- vocab-sized logits at 16K seq are 16 GB per sequence unless the loss is chunked (use chunked cross-entropy / KL)
- rented 8×H100 is the escape hatch if single-A100 + multi-Colab data gen feels slow
- tinker / together / fireworks don't fine-tune Flash-Next [web]
- an unverified "LoRA over GGUF in ~40 GiB" claim exists [web]

---

## 8. risks
- 224 experts may fail JA or agent evals even with good calibration. then 256/320 + more streaming.
- SCR staleness on 36 GDN layers could be much worse than ds4's +0.015–0.040 nats/token.
- MoCE + pruning compounding quality loss. λ has to be retuned per K.
- Air thermals make sustained cold prefill slow. reuse has to carry it.
- desktop 16 GB RAM may make the desktop slower than the laptop.
- Strata-Metal at 24 GB is untested ("other chips and memory sizes are untested" [src: MAC.md]).
- abstention tuning over-refusing.
- trace-based saliency drifts once persona training changes the router. re-check routing after phase 3.

---

## 9. `scripts/mine_llm_isms_from_transcripts.py`: required fixes
tested 2026-10-14:
- parses **0 turns** from the 16 Claude Code jsonl files in `~/.claude/projects`. CC rows are `{"type":"assistant","message":{"content":[…]}}`; the script expects `messages`.
- albedo transcripts are erlang-term binaries in `~/.albedo/albedo.sqlite`. read them through the session API (`agents.sessions()` / `h.messages()`).
- albedo-era assistant text is already persona-styled, and providers have different slop profiles, so tag by provider.
- make `--human-ref-dir` required (without it the log-odds collapse to z ≈ δ/10). dawn's user turns are the baseline.
- use Monroe's informative prior α_w = α₀ · p_background(w). the current `0.01·p·α₀ + 0.01` is nearly flat.
- don't strip punctuation before n-gramming (n-grams cross sentence boundaries).
- token-variant mapping: add newline/quote/after-punctuation forms, keep phrases > 4 tokens, don't `.capitalize()` whole phrases.
- drop the 1.5% / 3.6% "safety cap" (letter-lipogram paper, no basis).

---

## 10. open questions (dawn)
1. is the desktop's 16 GB RAM fixed?
2. ~~Air: 13" or 15"? SSD size?~~ answered locally: **13" M5 Air (`Mac17,3`), 1 TB SSD (~303 GB free)**.
3. does the laptop have to run the same model, or only the same persona? currently assuming the same model.
4. how much do vision and JA matter vs scripting? sets the calibration mix and how far pruning can go.
5. persona inside `<think>` too, or only visible replies?
6. which harness drives it: Connectome itself, an albedo-style python kernel, or something custom? the eval runs through the same one (§2.9).
7. is Gadsby-4B yours? (used as evidence in §2.2 either way)

---