opus audit — Phase 3 local healing v2 verification #
scope: read-only verification of the v2 changes (scripts/heal_k256_q2_0.cpp, scripts/run_phase3_local_healing.py, scripts/patch_strata_metal.py, results/phase3_local_healing_validation.json, PLAN.md §2.7.A #31 and §7.1 item 6) against audits/opus_audit_phase3_local_v1.md. i re-ran the full 22-window / 2,686-token unseen KLD suite on the v2 model (the v2 report only ran 4 of the 22 windows), regenerated the v1 held-out 512-token MoE dump (bit-identical, ‖y_ref‖² = 36288.5374 both times) to compare v1 and v2 routed residuals, re-ran the ridge cross-validation probe on the new 1,022-token trace, and re-ran the 9-prompt speed suite on v2. no repo files edited; artifacts are in /tmp/p3audit/ (kld_results_v2.json, v2_eval_cal1024.tsv(.cv), v2_eval_heldout.tsv, speed_results_v2.json).
0. tl;dr #
- v2 is a real but small improvement over v1, and it's good vs unhealed. v2 vs unhealed on all 22 windows: −28.6% pooled KLD [−33.5, −23.2] at MoCE 0 (wins 20/22), −27.0% [−32.0, −22.1] at MoCE 0.85 (wins 21/22), top-1 +2.3 / +1.5 pt. v2 vs v1: −5.1% [−10.8, −0.1] at MoCE 0 (11/22 windows), −3.3% [−7.6, +0.4] at MoCE 0.85, and top-1 −0.3 / −0.5 pt (n.s.). on the 18 windows whose context shares no 16-gram with the new calibration trace, v2 vs v1 is −3.5% [−8.8, +1.5] → not distinguishable from v1.
- the routed-MoE local metrics improved cleanly and transfer: on the regenerated unseen 512-token dump, routed residual drops 24.13% (v1 shipped) → 23.37% (v2 shipped), with v2's real-trace α within 0.01 pt of the oracle and only 2/48 layers where α is worse than α=1 (was 22/48).
- japanese got worse vs v1 (ja KLD 0.172 → 0.180 at MoCE 0, top-1 85.0 → 83.7; 3 of 5 ja windows regress), while ops / chat-json improved. the canary 192-token set says v2 is worse than v1 (−23% vs −35%), PLAN.md reports v2's canary numbers without saying so.
- the "fixed tool-calling regression" is partly circular: the new calibration trace contains the same
<tools>system block as the windowscj20_tool_5turn_t3(117 shared 16-grams in its context),cj20_t3_answer_tool(117) andcj03_toolcall(43), and a different turn of the same persona conversation aspersona3_t4(85). the scored spans are clean (the self-test is right about that), but fixing cj20 by calibrating on cj20's boilerplate is in-distribution, not generalization. and the 4-window v2 benchmark in the results JSON includes exactly that window. - code: the α, cache-fingerprint and I/O changes are correct. two real defects: (1) the yr_q2 fingerprint doesn't cover the gate/up experts, so a
gain_gusweep still silently reuses a stale cache; (2) "full-rank 640/640" is false for L0 (rank 430) and L2 (617). plus a doc/code mismatch on the W_down importance exponent. thestream.cppguard is correct; the multi-chunk dump is untested (the 1,023-token dump is single-chunk) and holds whole files in RAM. - stream.cpp / speed on v2: see §4.
1. code verification #
1.1 --quantize #
- W_gate/W_up importance: now
w_d = sqrt(E[mixed_d²])from the 1,022-token dump, used as the per-element weight on squared error in the LS/score. correct plumbing. note that the diag-Hessian-optimal weight on squared weight error isE[x_d²]itself (β=1); β=0.5 is a tempering choice, so call it that, not "AWQ". it's untested against β=1 or β=0. fine as a default. - W_down importance:
s_down[c] = (vg·vu)^(1/4),vg = Σ_d g_cd² E[x_d²], i.e.sqrt(σ_g σ_u). the header comment and PLAN say "SwiGLU RMSsqrt(E[h_c²])", butE[h²]^(1/2) ≈ σ_g σ_u, so the code uses β=0.25 on W_down vs β=0.5 on gate/up: an undocumented inconsistency. it also assumes diagonal, uncorrelated, zero-mean inputs and ignores the SiLU. the exact per-expertE[h_c²]is computable from the routed tokens in the same dump (≈20 tokens per expert on average, which is thin but unbiased; you could blend toward the analytic estimate). low priority, but fix the docs either way. - the hc_ffn_norm fallback writes
max(ms2·0.25, 1e-6)(a variance) intolayer_s_in, while the real path writes an RMS. so the two code paths aren't on the same scale. harmless since the runner always passescal_pfx, but the fallback should besqrt(...)or deleted. - for layers whose gate/up are already Q2_0 in the source, gate/up are still dequantized for the down importance only when
td != Q2_0. that's correct. - verified that stage 1 actually re-ran with the new binary:
d_bak.bin21:03 > heal binary 20:56 >.mixed20:23.
1.2 --calibrate-and-heal #
-
α (Part A): γ=0 →
α = cos/ratio = ⟨y_ref,y_q⟩/‖y_q‖², computed over all tokens and all routed experts (via the routedyr_q2). my probe reproduces the shipped α to within 5.8e-5 of the in-sample oracle. thecos ≥ 0.1clamp is inert. idempotent (rewrites fromd_bak). ✓ -
cache fingerprint: hashes
d_bak(down scales) + layer-0.mixed+t_cal. two holes:yr_q2depends on the gate/up Q2 weights too, and those aren't hashed. a--quantizere-run that changes onlygain_gu(or the gate/up importance β) leavesd_bakbyte-identical (down importance is computed from the source gate/up, not the Q2 ones), so the stale cache is accepted. this is exactly the sweep scenario the fingerprint was meant to catch. fix: hash the healed.expertsgate/up slices (or just its mtime+size), plusgain_gu.- layer-0
mixedis a function of the embeddings/dense weights only, so a dump from a different expert quantization (e.g. re-dumping from Q2 instead of IQ3_S) hashes identical. hash a late layer too (e.g. L47.mixedor.yr).
-
ridge (Part B): math unchanged. my probe on the 1,022-token trace: rank(Z) = 640 for 46/48 layers, but L0 = 430 and L2 = 617, so "full-rank (640/640)" in PLAN.md is wrong for 2 layers (λ>0 keeps it well-posed anyway). effective dof at ρ=6 is 5–70 of 640. cross-validation (float ΔW):
split ρ=0.25 ρ=1 ρ=6 (shipped, η=.15) ρ=24 in-sample −10.7% −6.7% −2.9% −1.4% leave-one-block-out (4 contiguous) −1.6% −1.4% −0.9% −0.6% even/odd −3.9% −3.1% −1.7% −1.0% doubling T didn't change the picture: in-sample is still ~3× LOBO, and the realized gain after IQ4_NL requant is −1.10% in-sample (25 multipliers; v1 with 7 got −1.68% on its own trace) and −0.46% on the unseen dump (v1: −0.36%). still safe, still ~noise. the PLAN line "MoE MSE 19,061 → 18,665 → 18,461 (−3.15%)" is in-sample and mostly α (−2.08%), the ridge is the last −1.10%.
-
writers file now holds only the 48 changed
ffn_down_shexptensors, transplant count 48 / 1175 untouched. ✓ error-checked I/O ✓.t_calinferred from file size with divisibility check ✓ (but.ids/.wts/.yrsizes aren't cross-checked against it).
1.3 patch_strata_metal.py #
stream.cppguardif (moce_bonus_ > 0.0f) { wait + evict }: correct and matches the v1 recommendation. at MoCE 0 routing can't seesmap_, so skipping is safe. source and binary are consistent (stream.cpp 20:27:45, CLI 20:28:12).- multi-chunk
dump_moe_layer: logic reads the previous whole file into a staticstd::map<ext, vector>at l=0 and rewrites[old rows | new rows]per layer. as far as i can trace it's correct for in-order layers, with these caveats: (a) it's never exercised, since the 1,023-token dump withPROMPT_CHUNK=1024is a single chunk (T=1022); (b) per chunk it holds a full previous copy of.mixed,.yrand.hhin RAM (≈0.5+0.5+0.03 GB at T=1022, growing linearly) inside the inference process, and rewrites everything per chunk (O(chunks²) I/O); (c)first_chunkkeys onlast_T_ == 0 && s_pfx != pfxwith static state, so a second dump in the same process with the same prefix appends instead of resetting (fine for the one-shot CLI, wrong for a server); (d)fseekwithlongis fine on macOS LP64 only. add a 2-chunk test (e.g.PROMPT_CHUNK=512on the 1,024 file, compare against the single-chunk dump byte-for-byte) before relying on it.
1.4 run_phase3_local_healing.py #
- stage 1 is still skipped if
.experts+d_bakexist, with no fingerprint, so changing gains or the cal trace requires manual deletion. same class of hazard as §1.2. verify_disjoint_calibration_promptschecks 16-grams of scored spans only. it passes, correctly. but it doesn't check contexts or doc identity, and the new trace shares context with 4 windows (§2.2). extend it to report context overlap and doc-level reuse, and fail on doc reuse.- the results JSON's
unseen_22window_2686tok_auditblock is v1 numbers (copied from my audit file), not v2. that's labeledhealed_v1_*in the keys, so it's honest, but PLAN.md mixes the v1 22-window and v2 4-window numbers in one bullet.
2. falsifier: full 22-window unseen KLD on the v2 model #
same 22 windows, same flags as v1 (§3.1 of the v1 audit). the IQ3_S reference was re-run and reproduced every window's PPL exactly (deterministic). v1/unhealed numbers are from the v1 audit runs, same binary family (the MoCE-0 path now skips the eviction wait, which can't change logits at MoCE 0).
2.1 pooled (95% CI = window bootstrap) #
| comparison | KLD A → B | Δ KLD | windows B better | Δ top-1 |
|---|---|---|---|---|
| unhealed → v2, MoCE 0 | 0.2029 → 0.145 | -28.55% (-33.5, -23.2) | 20/22 | +2.27 (1.29, 3.4) |
| unhealed → v2, MoCE .85 | 0.2123 → 0.155 | -26.98% (-32.0, -22.1) | 21/22 | +1.53 (0.41, 2.7) |
| v1 → v2, MoCE 0 | 0.1527 → 0.145 | -5.08% (-10.8, -0.1) | 11/22 | -0.26 (-1.15, 0.51) |
| v1 → v2, MoCE .85 | 0.1602 → 0.155 | -3.26% (-7.6, 0.4) | 13/22 | -0.52 (-1.39, 0.31) |
| v1 → v2, MoCE 0, 18 clean windows | 0.1522 → 0.1469 | -3.51% (-8.8, 1.5) | 8/18 | -0.37 (-1.35, 0.49) |
| v1 → v2, MoCE .85, 18 clean | 0.1591 → 0.1568 | -1.45% (-5.5, 2.0) | 10/18 | -0.51 (-1.56, 0.54) |
| isolated MoCE (m.85 vs own m0), v1 → v2 | 0.0389 → 0.0325 | -16.53% (-28.3, -4.4) | 16/22 | +1.34 |
"clean" = the 18 windows whose full token file (context + scored) shares no 16-gram with cal_disjoint_1024tok.txt.
reading: v2 clearly beats unhealed (−28.6%, a bit more than v1's −24.7%) and the falsifier isn't triggered. v2 vs v1 is a small, marginal gain at MoCE 0 that loses significance once the 4 context-overlapping windows are dropped, and top-1 doesn't move. the clearest v2 win is isolated MoCE robustness (−16.5% vs v1).
2.2 per stratum, KLD / top-1 (token-pooled) #
| model | ops | chat_json | ja | persona | pooled |
|---|---|---|---|---|---|
| unhealed, MoCE 0 | 0.1650 / 89.7 | 0.1554 / 89.5 | 0.2374 / 80.2 | 0.3576 / 72.1 | 0.2029 / 85.22 |
| v1, MoCE 0 | 0.1199 / 90.9 | 0.1243 / 90.4 | 0.1723 / 85.0 | 0.2637 / 78.4 | 0.1527 / 87.75 |
| v2, MoCE 0 | 0.1077 / 90.6 | 0.1130 / 90.8 | 0.1796 / 83.7 | 0.2527 / 77.9 | 0.1450 / 87.49 |
| unhealed, MoCE .85 | 0.1589 / 91.4 | 0.1702 / 89.5 | 0.2513 / 79.6 | 0.3702 / 72.7 | 0.2123 / 85.55 |
| v1, MoCE .85 | 0.1175 / 90.0 | 0.1341 / 90.2 | 0.1887 / 84.1 | 0.2683 / 80.7 | 0.1602 / 87.60 |
| v2, MoCE .85 | 0.1070 / 90.6 | 0.1267 / 90.1 | 0.1929 / 82.2 | 0.2652 / 79.2 | 0.1550 / 87.08 |
ops and chat-json improve vs v1. ja regresses vs v1 (MoCE 0: 0.172 → 0.180, top-1 85.0 → 83.7; ja05, ja12, ja30 each 4–8% worse). persona is flat (−4% KLD, −0.5 pt top-1). so the new calibration mix traded some japanese for tool/ops. its only ja content is one doc (ja[2], ~100 tokens), versus a big block of tool calls.
2.3 the 4 context-overlap windows (†) #
the scored spans are clean, but the calibration trace contains the identical <tools> system block, and for persona3 a different turn of the same conversation:
| window | shared 16-grams with cal (whole window) | v1 → v2 KLD (MoCE 0) |
|---|---|---|
| cj20_tool_5turn_t3 | 117 / 701 | 0.1783 → 0.1444 (−19%) |
| cj20_t3_answer_tool | 117 / 742 | 0.0927 → 0.0430 (−54%) |
| cj03_toolcall | 43 / 1064 | 0.0631 → 0.0621 |
| persona3_t4 | 85 / 1974 (same doc, calibration_corpus conversation 3) |
0.2856 → 0.2983 (+4%) |
the cj20 "regression fix" is real but in-distribution. note that cj20 at MoCE 0 is now tied with unhealed (0.1444 vs 0.1440), not better. the results JSON's 4-window benchmark (ops12, cj20, ja22, persona0) was chosen after seeing v1's per-window losses and includes cj20, so its −29.7% headline is optimistic; the full-suite figure is −28.6%.
2.4 per window #
† = context shares 16-grams with the calibration trace.
| window | unhealed m0 | v1 m0 | v2 m0 | v2 vs v1 | unhealed m.85 | v1 m.85 | v2 m.85 | top1 m0 unh/v1/v2 |
|---|---|---|---|---|---|---|---|---|
| ops14_csharp_bug | 0.0797 | 0.0767 | 0.0775 | +1.1% | 0.0776 | 0.0834 | 0.0791 | 89.8/92.2/90.6 |
| ops17_term_multiturn | 0.0627 | 0.0331 | 0.0344 | +3.9% | 0.0595 | 0.0437 | 0.0374 | 95.3/95.3/95.3 |
| ops19_term | 0.0581 | 0.0513 | 0.0583 | +13.7% | 0.0603 | 0.0332 | 0.0456 | 93.0/91.4/93.0 |
| ops12_java_turn1 | 0.3805 | 0.2778 | 0.2489 | -10.4% | 0.3384 | 0.2758 | 0.2567 | 79.7/82.8/82.0 |
| cj03_toolcall † | 0.0978 | 0.0631 | 0.0621 | -1.6% | 0.1349 | 0.0780 | 0.0731 | 96.1/98.4/99.2 |
| cj20_tool_5turn_t3 † | 0.1440 | 0.1783 | 0.1444 | -19.0% | 0.1534 | 0.1363 | 0.1417 | 91.4/89.8/91.4 |
| cj15_php_review | 0.0947 | 0.0703 | 0.0654 | -6.9% | 0.0959 | 0.0908 | 0.0838 | 85.9/89.1/87.5 |
| cj19_italian | 0.1779 | 0.1493 | 0.1497 | +0.3% | 0.1982 | 0.1839 | 0.1735 | 82.8/81.2/84.4 |
| cj06_dostoevsky | 0.4949 | 0.3406 | 0.3320 | -2.5% | 0.5289 | 0.3677 | 0.3618 | 78.9/82.0/79.7 |
| cj07_json_extract_near | 0.0420 | 0.0348 | 0.0349 | +0.0% | 0.0442 | 0.0322 | 0.0410 | 95.3/95.3/95.3 |
| cj17_math | 0.2147 | 0.1663 | 0.1645 | -1.0% | 0.2379 | 0.1839 | 0.1805 | 82.0/83.6/83.6 |
| ja05 | 0.2284 | 0.1627 | 0.1691 | +3.9% | 0.2378 | 0.1666 | 0.1744 | 79.7/82.8/82.8 |
| ja12_table | 0.2305 | 0.1847 | 0.1982 | +7.3% | 0.2354 | 0.2060 | 0.2181 | 77.3/85.9/79.7 |
| ja22_pedal | 0.2965 | 0.1856 | 0.1845 | -0.6% | 0.3175 | 0.2114 | 0.2162 | 80.5/85.2/85.2 |
| ja30_climate | 0.2173 | 0.1750 | 0.1859 | +6.3% | 0.2397 | 0.1880 | 0.1833 | 82.0/85.2/86.7 |
| persona0_t2 | 0.4100 | 0.2858 | 0.2878 | +0.7% | 0.4163 | 0.2945 | 0.3020 | 68.8/76.6/77.3 |
| persona3_t4 † | 0.3872 | 0.2856 | 0.2983 | +4.4% | 0.4340 | 0.3357 | 0.3125 | 73.4/76.6/73.4 |
| cj07_answer_json | 0.0006 | 0.0001 | 0.0001 | -1.1% | 0.0019 | 0.0001 | 0.0002 | 100.0/100.0/100.0 |
| cj20_t3_answer_tool † | 0.1030 | 0.0927 | 0.0430 | -53.6% | 0.1053 | 0.1090 | 0.0615 | 95.3/96.1/97.7 |
| ja05_answer | 0.1030 | 0.0625 | 0.0665 | +6.5% | 0.1058 | 0.0885 | 0.0753 | 86.4/90.9/86.4 |
| persona0_t2_answer | 0.2755 | 0.2197 | 0.1721 | -21.7% | 0.2603 | 0.1746 | 0.1811 | 74.2/82.0/82.8 |
| ops19_answer_json | 0.2441 | 0.1606 | 0.1194 | -25.6% | 0.2587 | 0.1515 | 0.1160 | 90.6/93.0/92.2 |
2.5 the 192-token canary #
v2 canary macro KLD 0.1563 vs v1 0.1320 (MoCE 0), i.e. v2 is 18% worse than v1 on the canary (−23.1% vs −35.1% vs unhealed). given §2.1, that's mostly the canary's 192-token noise, the same thing that made v1 look better than it was. but PLAN.md reports the v2 canary numbers as gains without the v1 comparison, which reads as if nothing got worse.
3. local MoE metrics on the regenerated unseen dump (512 tok, bit-identical to v1's) #
| α=1 (AWQ only) | shipped α | oracle α (on this data) | shipped α + ridge | layers α worse than 1 | |
|---|---|---|---|---|---|
| v1 | 24.31% | 24.21% | 23.81% | 24.13% | 22/48 |
| v2 | 23.92% | 23.48% | 23.47% | 23.37% | 2/48 |
so all three v2 local changes transfer: the importance-weighted quantizer is −1.6% relative (α=1 column), the real-trace α now captures essentially all of the oracle gain (vs 20% for v1's Gaussian α), and the ridge adds −0.46%. caveat: this dump contains ~100 tokens of calibration_corpus conversation 1, which the new trace also samples (different region, 0 shared 16-grams, 4 shared 8-grams), plus the cj20 tool boilerplate.
4. speed / hit rate on v2 (9-prompt suite, MoCE .85 × 2 reps + MoCE 0) #
| prompt | v1 hit (m.85) | v2 hit (m.85) | v2 misses/tok | v2 decode tok/s r0/r1 | v2 det. r0==r1 | v1 hit (m0) | v2 hit (m0) |
|---|---|---|---|---|---|---|---|
| ja_pedal_398 | 0.9276 | 0.9253 | 35.9 | 9.5 / 11.19 | yes | 0.8422 | 0.8471 |
| json_extract_163 | 0.9560 | 0.9560 | 21.1 | 10.68 / 10.07 | yes | 0.8663 | 0.8699 |
| tool_multiturn_588 | 0.9098 | 0.9088 | 43.8 | 13.17 / 11.71 | yes | 0.8290 | 0.8390 |
| persona_458 | 0.9304 | 0.9126 | 41.9 | 11.14 / 11.04 | yes | 0.8453 | 0.8469 |
| math_43 | 0.9443 | 0.9455 | 26.2 | 9.48 / 10.46 | yes | 0.8454 | 0.8469 |
| ops_csharp_713 | 0.9249 | 0.9272 | 34.9 | 8.95 / 11.1 | yes | 0.8409 | 0.8422 |
| coldshift_ja2rust | 0.9394 | 0.9364 | 30.5 | 10.7 / 10.43 | yes | 0.8690 | 0.8661 |
| prompt_43tok_wal | 0.9313 | 0.9295 | 33.8 | 10.83 / 9.45 | yes | 0.8494 | 0.8551 |
| prompt_1024tok_ops | 0.9309 | 0.9248 | 36.1 | 11.71 / 9.14 | yes | 0.8448 | 0.8473 |
- mean combined hit at MoCE .85: v1 0.9327 → v2 0.9296 (−0.3 pt); MoCE 0: 0.8480 → 0.8512. persona_458 drops 1.8 pt (misses/tok 33.4 → 41.9) and ops1024 drops 0.6 pt (matching the results JSON's own 0.931 → 0.921). free-decode trajectories differ between models, so a single prompt is a single sample, but persona is the target workload: re-check it with more persona prompts before calling it noise.
- determinism: 9/9 bit-identical across reps at MoCE .85 on v2 ✓. MoCE 0 path (guard skips the wait) ran clean on all 9.
- PLAN.md §7.1 item 6 quotes the 9-prompt hit rates (93.27%, 90.98–95.60%) and 9/9 determinism as if they were v2. they are v1-model numbers. update them to v2: mean 92.96%, min 90.88% (tool), max 95.60% (json).
5. PLAN.md corrections #
- "full-rank (
640/640)" → 46/48 layers full rank; L0 rank 430, L2 rank 617. - "W_down by per-expert analytical SwiGLU RMS
(σ_g σ_u)^(1/2)": that's E[h²]^(1/4), not the RMS. state the exponent honestly (β=0.25 vs 0.5 on gate/up). - the "4-stratum unseen benchmark −29.67%" headline → replace with the full 22-window v2 result: −28.6% [−33.5, −23.2] at MoCE 0, −27.0% at MoCE .85, +2.3 / +1.5 pt top-1, v2 vs v1 −5.1% [−10.8, −0.1] (n.s. on clean windows), ja regressed vs v1.
- "eliminating the cj20 regression" → note the tool boilerplate in the calibration trace (in-distribution), and that cj20 is tied with unhealed, not better.
- canary: add v1 → v2 (0.1320 → 0.1563, worse) next to the v2 numbers.
- 9-prompt speed/determinism → v2 numbers (§4), including the persona hit-rate drop.
- "0 16-gram overlap against all 22 unseen windows": true for scored spans only. add "4 windows share context (tools block / same persona conversation)".
6. recommendations #
- ship v2 (it's at least as good as v1 and clearly better than unhealed), but quote −28.6% / +2.3 pt on 22 windows, not the 4-window number.
- rebalance the calibration trace: more japanese (ja regressed), and swap persona conversation 3 for one that no eval window uses. then the tool/persona gains can be claimed as out-of-sample. the clean move is to freeze a doc-level split file (cal docs vs eval docs) and make the self-test enforce it.
- fix the fingerprint (gate/up + late-layer hash) before any
gain_gu/ β sweep. - test the multi-chunk dump against a single-chunk dump byte-for-byte before using T > 1,022.
- ridge: still ~0.5% out of sample. keep or drop; don't spend more on it.
- next real lever: per-expert exact
E[h_c²]for W_down and a β ∈ (0.5, 1) ablation for gate/up, both evaluated on the 22 windows plus a held-out ja/persona expansion. persona remains the worst stratum (KLD 0.25, top-1 78%).
v3 verification addendum #
re-checked PLAN.md (§2.7.A #31, §7.1 item 6), scripts/heal_k256_q2_0.cpp (identical to vendor/strata/metal/src/heal_k256_q2_0.cpp, binary rebuilt after the source), scripts/run_phase3_local_healing.py (--self-test PASS, re-run by me) and results/phase3_local_healing_validation.json. nothing re-measured; all numbers cross-checked against my v2 runs.
closed:
-
PLAN.md: all 7 §5 corrections are present in both #31 and item 6:
- rank 46/48 full, with L0 = 430 and L2 = 617;
- the β = 0.25 vs 0.5 exponents;
- the 22-window v2 numbers (−28.55% / −26.98%, +2.27 / +1.53 pt), plus v2 vs v1 −5.08% and the clean-18 −3.51% not significant;
- the japanese regression;
- cj20 being in-distribution and tied with unhealed;
- the canary v1 → v2 got worse;
- the v2 speed numbers (92.96%, persona 93.04 → 91.26%);
- the 4 context-overlapping windows.
no stale headline is left (no −29.67%, no "eliminating the cj20 regression", 93.27% only as the v1 reference).
-
results JSON: the v2 22-window block and the v1/v2 9-prompt block match
kld_results_v2.json/speed_results_v2.jsonexactly. -
yr_q2 fingerprint: now 4 × uint64: d_bak, L0 + L47
.mixed, gate/up of expert 0 at L0/L24/L47 from the healed experts, and t_cal. a globalgain_gu/ β change rewrites expert 0's gate/up, so it's now caught, and the L47.mixedcatches a teacher swap. -
cross-checks:
.yr/.ids/.wts/.hhsizes are now checked against t_cal. -
fallback: the hc fallback now writes an RMS.
-
self-test: it now asserts 0 scored-span overlap plus exactly 18/22 context-clean windows, and names the 4 overlapping ones.
still open (minor, none affects the shipped model):
-
run_phase3_local_healing.pyskips Stage 1 if.quant.metamerely exists; it doesn't compare the stored (gain_gu, gain_d, cal_pfx, t_cal) against the values it's about to request. change the"0.50", "0.30"args and Stage 1 is still skipped. read the 3 uint64s and compare them. -
.quant.metahashes the cal_pfx path string, not the dump contents. the dump step likewise only checks that.mixed/.yrexist, so a re-madecal_disjoint_1024tok.txtunder the same prefix wouldn't invalidate anything. hash the token file into the meta. -
provenance:
.quant.meta(22:43:19) andcal_teacher_yr_q2_1024.bin.meta(22:43:20) are newer than the data they vouch for:d_bakis 21:03 and theyr_q2bin is 21:04. so they were written without re-running--quantizeor recomputingyr_q2.- the d_bak + .mixed hashes would match regardless.
- the gate/up sample is the one input nothing re-read:
.expertswas rewritten at 22:44 (α re-applied fromd_bak, deterministic), but no--quantizeran. - stored quant.meta values: gain_gu 0.5, gain_d 0.3, t_cal 1022, which is consistent.
fine for now. just don't treat these metas as having validated the cache. the next real run will recompute or confirm them.
-
the gate/up hash samples 1 expert × 3 layers. that's adequate for global knob changes, but it would miss a per-layer or per-expert-only edit. acceptable; document it.
verdict: v2 corrections verified. the remaining items are about cache/skip hygiene for future sweeps, not about the current K256-q2_0-healed artifact or its reported numbers.