# opus: Phase 5 SCR (GPU readback) + anti-overfitting persona solve — v2

**short version**
- **Phase 5:** your GPU numbers pass. two readings to keep straight:
  - in your runs SCR lands about as far from a fresh `[A B' C]` as the unedited `[A B C]` does (KL 0.10–0.14 for both). so the
    kept KV of C, not the GDN state, carries most of B.
  - "load 5 ms" was almost certainly a warm page cache: 140 MiB in 5 ms is 28 GB/s, above any SSD. measure it cold.
- **step0 with `</think>\n\n`:** the special-token argmaxes are gone (**0/190**, was 14/70). there is **no uppercase gap** in the persona
  context: capitalized starters hold 0.004 of the mass at step0 vs 0.235 for lowercase, and 0.001 vs 0.119 at sentence starts inside
  replies.
- **the new trace still has two packing bugs that cap what any style method can show:**
  - none of the 190 prompts contains its user turn;
  - replies are cut to ~22 tokens (173/190 `chosen_ids` have exactly 22), so most end-of-reply kaomoji and all closers are missing.
- **approach 2 (zero-parameter shaping):** zero parameters doesn't mean a 1.00 ratio. the measured ratio is 0.12, because the persona
  data is already lowercase, so the gain is ~0. on the capitalized blind chat document it costs **KL 0.084**.
  - the slop-penalty-only variant passes blind KL (0.0042) but removes nothing (base slop mass 0.001).
- **approach 1 (ultra-low-DoF):** a **rank-1 direction over 16 kaomoji/paren rows of `output.weight`, early-stopped at 30 Adam steps**
  passes both bars:
  - generalization ratio 0.72 overall (0.76 on style tokens);
  - blind chat KL **0.00002**, ftpo_heldout 0.00002;
  - held-out kaomoji log p +0.147 nats/token.
  - **caveat:** i picked 30 steps after seeing held-out numbers, so the 57 held-out pairs are now a dev set. needs a 3-way split before it counts.
- **how it overfits:** every longer or wider config drops below the 0.70 ratio, and the split-swap test shows it's the fit growing with
  steps, not the split (details in §5).

## 1. Phase 5 GPU readback (parent's M5 Air runs of `window_session_scr.patch`)
| test | result |
|---|---|
| session round trip, 1,983 cells (140.25 MiB) | save 23.43 ms, **load 5.00 ms** vs 8,138 ms cold prefill |
| same, `MOCE_BONUS=0`, `EXPERT_SPLIT=0` | **mean/p90 KLD 0.000000, top-1 100%, PPL 20.4210 = 20.4210** |
| same, `EXPERT_SPLIT=1` | KLD 0.002791 (residency-dependent partition of the 10 experts → ~3e-7 f32 order differences per layer) |
| SCR short tail (A 512, B 1024 → B' 60, C 511, eval 64) | orig 0.1089 / keep J0 0.1034 / keep J8 0.1147 (PPL 7.20) / gdnex J8 0.1109 / lossy J8 **1.818** / sanity lossy J=511 **0.000000** |
| SCR long tail (C 1535) | keep J0 0.1384 / orig 0.1377 / gdnex 0.1003 (→ GDN staleness **+0.038**); load 6.0, relocate 0.5, splice prefill 1,186.7 ms vs 4,131.9 ms → **3.48×** (3.79× at J8) |
| loop check, keep J0, 128 greedy tokens | 4/125 repeated 4-grams (3.2%): no collapse |

what i read from it:
- **the sanity rows are exactly 0**: relocation, restore order and junction handling are right end to end.
- **lossy (1.82) vs keep (0.10–0.14)**: keeping `S_ABC` is essential. a GDN that never read C is far worse than one that read B.
- **staleness is mostly not GDN.** keep − gdnex = +0.004 (short tail) / +0.038 (long tail). that's inside ds4's +0.015…0.040 range,
  even with 36 recurrent layers. but gdnex itself is still 0.10–0.11 from fresh, about as far as `orig`.
  - so C's **KV**, computed while B was in context, carries B's content. that's the CLM premise ("faithful to its computational past"),
    not a bug.
  - KL to a fresh `[A B' C]` is a distance from a counterfactual, not an error. B' is a lossy summary, so fresh isn't ground truth
    either.
  - the metric that decides is downstream quality. **keep J8 has the lowest PPL of all rows (7.20 vs orig 7.49, gdnex 8.07)**. add the
    fresh row's PPL next to these in the table.
- **the speedup is capped by the short-prompt rate, not by SCR.** 60 + J tokens took ~1.1–1.2 s, i.e. ~20 ms/token, the SSD-bound
  short-turn regime of PLAN §2.7. relocate + load are < 7 ms.
  - so SCR's win grows with |C|: 3.5× at 1.5K, roughly linear beyond.
  - your `mn` 64 → 16 change for PromptGemm is the right lever for B'.
- **load 5 ms = 28 GB/s, i.e. the page cache** (the file was just written). for the cold number: `sudo purge`, then load.
  expect ~30–50 ms at this size (A.1 table in v1).
- **EXPERT_SPLIT=1 is history-dependent in general**, not only across sessions. any change in arena residency (a different earlier
  prompt) reorders the f32 sums. 0.0028 KLD is that noise floor. keep `EXPERT_SPLIT=0` for every exactness test.

## 2. step0 and reply starts on the new dump (7,536 rows, base model, exact head)
- **special-token argmax at step0: 0 / 190** (v1: 14 / 70 with `</think>\n`). the template fix was the fix.
- step0 top-1 over 190: `the` 79, `i` 52, `“` 22, `<tool_call>` 12, `{"` 9, `` ` `` 3, `その` 3, `The` **1**, others 1 each.
  - top-5 mass: `the` .102, `“` .074, `<tool_call>` .044, `i` .041, `you` .024, `that` .022, `it` .011 … `The` .003.
- **capitalized vs lowercase starters (the 12 words × both spacings):**
  - step0: **0.004 vs 0.235** (train) / 0.002 vs 0.238 (held-out);
  - sentence starts inside replies: 0.001 vs 0.119.
  - **the model already writes lowercase in this context.** there's no "The → the" gap to close here.
- the `“` (22 top-1s) is the base framing VN lines as quoted dialogue. that's a real style tendency, and a sampler or LoRA question,
  not a casing one.
- opener slop at step0 is still a non-event: p(rejected opener) 0.0002, p(slop set) 0.001–0.002.
- where CE goes in the replies: **83% on content words** (~4 nats/token: the user line is missing, see §3), 11–12% punctuation,
  ~2% kaomoji (~1.4–1.6 nats/token), < 1% parens.
  - reply CE is therefore the wrong yardstick for style. §4–5 report style-token CE separately.

## 3. the trace has two packing bugs (fix before any further persona work)
the jsonl prompts are right (`<|im_start|>user\n…<|im_end|>\n<|im_start|>assistant\n<think>…</think>\n\n`). the packed trace isn't:
1. **no user turn in any of the 190 pairs.** each pair's segment is only the tail
   `…\n<|im_start|>assistant\n<think>\n{cot}\n</think>\n\n`.
   - the model answers a line it never saw, so VN content is unpredictable (that's the 4 nats/word).
   - every VN pair has the same cot, so "voice" can only key on that one string.
2. **replies truncated to ~22 tokens** (173/190 `chosen_ids` = 22; 122 replies end without `<|im_end|>`, straight into the next
   `<|im_start|>`).
   - `load_real_vn_specs` attaches kaomoji at the **end** of even-index VN replies, so they're almost all cut: 28 kaomoji tokens over
     120 VN pairs, vs 300 over 70 original pairs. closers are gone entirely.
- re-pack each pair whole (full prompt + full reply + `<|im_end|>\n`), ~150–250 tokens each → ~30–45K tokens, dumped in chunks.
  the packer isn't in `scripts/` (it was inline), so please commit it.

## 4. why v1 overfit
- dof vs data: v1 had **|S| = 1,129–2,917 rows × 2,560 = 2.9M–7.5M parameters against 2,300 target tokens**. S included the
  replies' own content targets plus top-k, so the cheapest solution is a lookup table keyed on z: train CE 2.83 → 0.04.
- the KL anchors only constrain the anchor *directions*. 256 null directions held 87% of the anchor energy, and chat z's are
  high-rank, so the leftover leaked into held-out chat (0.017–0.075).
- what a style edit should look like instead: ~10²–10³ parameters, rows that are style tokens only, early stopping chosen on a split
  nobody else uses, and a check that the held-out gain shows up per token class.

## 5. approach 1: ultra-low-DoF style-only (`persona_head_solver_v2.cpp`, `editset`, `rank=r`)
all runs: chosen-NLL + opener UL + KL(w=100) on cal/collateral/ftpo_sft/prompt rows + LSE guard; null r = 256 on the direction;
scored after the Q6_K round trip. ratio = held-out gain / train gain, in nats/token of reply CE ("style" = kaomoji + paren targets).
E1b = 16 tokens: `) ; _ ` ( ) ` ; ・ ´ (` `) ;) ω `- ω` with leading-space variants (no `<`, `>`, `</`: code). E1 also had those.
E2 = 12-word case pairs + 26 slop starters (74).

| run | params | steps | ratio all / style | held-out style gain | blind KL | ftpo_heldout KL |
|---|---|---|---|---|---|---|
| E1 full rows, lr 1e-3 | 48.6K | 200 | diverged (CE +0.93) | −0.91 | 0.292 | 0.239 |
| E2 full rows, lr 1e-3 | 189K | 200 | diverged (CE +4.86) | −3.38 | 2.57 | 1.89 |
| E1b full rows, lr 1e-4 | 41K | 200 | 0.18 / 0.28 | +0.397 | 0.0031 | 0.0002 |
| E12 rank 2 | 5.3K | 200 | −0.06 / 0.31 | +0.332 | 0.0092 | 0.0055 |
| E1 rank 1 | 2.6K | 200 | 0.22 / 0.40 | +0.431 | 0.0022 | 0.0004 |
| E1b rank 1, ridge 1e-2 | 2.6K | 200 | 0.22 / 0.37 | +0.337 | 0.0008 | 0.0003 |
| E1b rank 1 | 2.6K | 100 | 0.32 / 0.50 | +0.418 | 0.0004 | 0.0002 |
| **E1b rank 1** | **2.6K** | **30** | **0.72 / 0.76** | **+0.131** (kaomoji +0.147) | **0.00002** | **0.00002** |
| E1b rank 1, splits swapped | 2.6K | 100 | 0.10 / 0.33 | +0.315 | 0.0008 | 0.0001 |

- **rank 1 + 30 steps is the only config that clears both bars.** its gain is small and real: held-out kaomoji tokens +0.147
  nats/token (p ×1.16), ordinary words −0.0006, no flips to a special token anywhere.
- **the ratio falls with steps at fixed rank**, and the swapped-split run does the same (0.33 style ratio at 100 steps). so the
  low ratio isn't the split's fault. even 2.6K parameters, run long enough, fit the training contexts. **early stopping is the
  regularizer that matters**, more than rank, ridge or null space.
- **the 30 was picked after seeing these held-out numbers.** before it counts:
  - split the pairs 3 ways (train / dev for the stopping step / test, by source and character);
  - choose the step on dev and report test once;
  - fix the packing first (§3). today's data has almost no end-of-reply kaomoji, which is exactly where this direction would act.
- **stacking on v3.2:** the dumps' R_post47 already includes the deployed `blk.47.ffn_down_shexp` (v3.2). a head-row edit composes
  on top of it exactly (z is downstream of R), so the numbers above are stacked.

## 6. approach 2: zero-parameter context-conditioned shaping (scored exactly with `shape=1`)
- **triggers:**
  - in assistant replies only: the reply start (`</think>` + `\n\n` or `\n`), and after a token ending in `. ! ?` or a newline;
  - never inside `<think>`, `<tool_call>`, ``` fences, or a reply that starts with `{`, `[`, `` ` `` or `<tool_call>`.
- **rule:** each of 105 capital→lowercase pairs (no-space and Ġ variants of 53 words) moves its mass,
  `ℓ'_low = logaddexp(ℓ_low, ℓ_up)`, `ℓ'_up = ℓ_up − 8`; 26 slop starters get −8.

| variant | ratio all / style | train / held-out reply gain | blind KL (KL at its 53 triggers) | ftpo_heldout KL | ftpo_sft KL | think / tool / JSON |
|---|---|---|---|---|---|---|
| case pairs + slop starters | 0.12 / 0.99 | +0.0026 / +0.0003 | **0.084** (1.74) | 0.088 | 0.063 | 0 by construction |
| slop starters only | 0.87 / 0.99 | −0.0006 / −0.0005 | 0.0042 (0.085) | 0.0033 | 0.0037 | 0 |

- "zero parameters → ratio 1.00" doesn't hold. with nothing fitted there's nothing to overfit, but the gain still depends on the data.
  on persona data the base is already lowercase, so the gain is ~0 on both splits, and the ratio of two near-zero numbers means nothing.
- **the KL bar and this rule contradict each other** on the blind document. its replies are written Capitalized ("Enabling WAL mode…",
  "When a peer sends RST…"), so a lowercase rule *has to* move ~1.7 nats at each sentence start there. any always-on style rule will
  fail "blind chat KL ≤ 0.005" on a capitalized document.
  - two ways out: gate it on the persona being active (a flag the server sets from the system prompt or request), or define the bar
    on non-trigger rows, where it's 0 by construction.
- the slop-starter penalty alone is cheap (0.004) but has nothing to remove in these contexts. keep it as a sampler-side guard for
  *other* prompts, where the base may still produce "Certainly!", e.g. without the persona system prompt. mine those first, as in v1.
- in Strata this belongs where the window has the token history: `Window::finish` before argmax, or the shared sampler. the trigger
  state is ~5 booleans per sequence, updated per committed token.

## 7. what i'd do next, in order
1. re-pack the trace whole (§3) and re-dump. it costs minutes, and every persona number so far is limited by it.
2. 3-way split; rerun `E1b rank=1` with the step chosen on dev; report test once. bar: test/train ≥ 0.70 on style tokens,
   blind + ftpo_heldout KL ≤ 0.005, 0 special flips.
3. shaping: ship only the persona-gated version, and only if dawn wants lowercase forced even where the base would capitalize. in the
   persona context it does nothing today.
4. the voice itself (VN banter content, abstention, lexical choices) is still Phase 3 LoRA work with the full forward. the head-side
   lever is worth ~0.1–0.15 nats on kaomoji tokens, which is nice and small.
5. Phase 5: add the fresh row's PPL to the SCR table, run a cold-cache load, and run the multi-fold drift test (N folds, KL vs fresh
   vs N) before choosing a re-ground policy.

## files
- `/tmp/opus_scr_probes/persona_head_solver_v2.cpp` (+ binary): v1's exact solver plus `editset`, `rank=r` (dW = A Bᵀ, B in the null
  space), `shape=1` (rule scoring, triggers as §6), `dump_top=1`, and per-trigger KL and generalization lines.
- `/tmp/opus_scr_probes/persona/v2/`: `spec*.txt`, `vn_roles.txt` (spans fixed for the truncated replies), one directory per run in §5/§6
  (`rows.tsv`, `output_weight_rows.q6k`), `shape/top5.tsv` (top-5 at every reply and trigger row).
- CPU only. nothing ran on the GPU.


## 8. v3 verification: whole-pair 17,352-row dump + 3-way split (train 113 / dev 40 / test 37)
(`persona_head_solver_v3.cpp`: role 4 = test, plus dev-tracked early stopping; runs in `/tmp/opus_scr_probes/persona/v3/`. CPU only.)

**the cold session numbers close A.1:** load 28.93 ms cold (5.18 GB/s) / 5.00 ms warm, save 21.70 ms, KLD 0.000000. the fresh
`[A B' C]` reference PPL is 6.548 (short tail) / 9.187 (long tail).

### 8.1 the data is right now
- **190/190** pairs carry the full user turn, and **190/190** replies end in `<|im_end|>` (`chosen_ids` include `[248046, 198]`).
  90/190 have a kaomoji/paren token in the last 8 positions before the end.
- **style tokens** (kaomoji + paren targets): train 421, dev 173, test 146. orig pairs carry them inline, VN pairs at the reply end.
- **step0** top-1 over 190: `the` 46, `i` 41, `“` 21, `<tool_call>` 15 (all in silent_tool pairs, where it's correct), `that` 10,
  `no` 7, `{"` 5, `you` 5, **`<|im_end|>` 4**, `so` 4, `The` 2.
  - the 4 `<|im_end|>`s are 3 VN pairs (kurisu ×2, rintaro) at p 0.08–0.14, and one ja rtuning pair at p 0.20: the model would end the
    turn without answering. these are the only special-token top-1s left at step0.
- capitalized vs lowercase starters at step0: 0.010 / 0.003 / 0.019 vs 0.244 / 0.279 / 0.220 (train/dev/test). still no casing gap.
- reply CE is now 2.45–2.61 (was 3.4–3.7 without the user turn), and base p on style tokens averages 0.77 (orig) / 0.59 (VN).

### 8.2 pre-registered protocol and result
- **rule (fixed before any run):** train on `train`. every 5 steps up to 300, compute the style-token gain (mean Δlog p over rows
  whose target is a style token) on train and on dev. **keep the last step with dev/train ≥ 0.75** (a margin over the 0.70 bar) and dev
  gain > 0. score test once at that step. the edit set comes only from the train split's 16 most frequent style tokens (E1c:
  `( ω ・ ´ ) ω ` ` `) ) (` _ ; ;) ; `-`, excluding `<` `>` `</`).

| variant (dev-tracked trajectory) | best dev/train ratio (step) | max dev style gain (step, ratio) | rule fires? |
|---|---|---|---|
| **E1c rank 1, lr 1e-3** (primary) | 0.594 (35) | +0.235 (150, 0.45) | **no → no edit** |
| E1c rank 1, lr 3e-4 | 0.594 (85) | +0.219 (300, 0.52) | no |
| E1c rank 2 | 0.464 (5) | +0.155 (70, 0.29) | no |
| E1c + 12-word case pairs, rank 1 | 0.697 (35) | +0.155 (125, 0.48) | no |

**so, against the bar: no style-only head edit reaches dev/train ≥ 0.70 at any step, so the pre-registered rule ships no edit.**
the v2 "0.72 at 30 steps" was a dev-selected number on a truncated trace, and it doesn't reproduce.

**secondary selection, declared before test was read:** the plain early-stopping rule, i.e. the step with the max dev style gain,
primary variant only (step 150). test scored once:

| | train | dev | test |
|---|---|---|---|
| style-token gain, nats/token | +0.521 | +0.230 | **+0.196** (95% CI over test pairs 0.057–0.38; p × 1.22) |
| test/train ratio | | | **0.38 (fails ≥ 0.70)** |
| other reply tokens | −0.005 | −0.014 | −0.010 |
| p(`<|im_end|>`) at real turn ends | 0.971 → 0.942 | 0.995 → 0.953 | 0.994 → 0.959 |

- KL: blind chat **0.0033**, ftpo_heldout **0.0003**, cal 0.0001, collateral 0.0001 (all ≤ 0.005); argmax flips to a special token **0**
  on every trace; edit = 16 rows, |ΔW|/|W| 0.40 on them.
- so it passes the KL and flip bars and fails the ratio. it also costs ~3.5 points of `<|im_end|>` at real turn ends: the direction
  raises " (" exactly where a reply would end. that's a kaomoji *before* the end, mostly harmless, but it's the v3.1 mechanism
  in reverse and has to be watched.

### 8.3 why the ratio fails: data structure, not parameter count
- **the ratio is already low before anything can be memorized.** at step 5 (rank 1, 2.6K params, |ΔW| ≈ 0) the first-order gains are
  `g_train·g_train` vs `g_dev·g_train`, so the ratio is the train/dev gradient alignment: 0.23 at step 5, never above 0.59–0.70.
  that's split heterogeneity, not memorization.
- **where the gain is, by source (step 150):**

  | | orig pairs (inline, hand-written kaomoji) | VN pairs (kaomoji attached at the reply end by the builder) |
  |---|---|---|
  | train | +0.116 (302 tokens) | **+1.551** (119) |
  | dev | +0.029 (105) | +0.538 (68) |
  | test | **−0.004** (90) | +0.518 (56) |

  - the whole generalizing gain is the VN end-of-reply kaomoji. `load_real_vn_specs` picks it as `kaomojis[idx % 5]` on even indices,
    so *which* one appears has nothing to do with the reply's content.
  - train fits the specific choice per reply (+1.55); held-out can only learn "a kaomoji from these 5 comes at the end" (+0.52).
  - a 1/3 ratio is the ceiling of that design, not a fitting failure. the inline, content-placed kaomoji of the orig pairs don't
    transfer at all.
- per-token CE on style tokens is the wrong unit for a style target. base p is already 0.6–0.8 on them, since once a kaomoji starts
  its tail is predictable, so all the signal sits on the **initiation** token, and its *identity* is arbitrary.

### 8.4 recommendation
1. **don't ship a head-row style edit from this data.** the pre-registered rule says no edit. the max-dev-gain edit passes KL/flips but
   fails the ratio, and its gain is a builder artifact.
2. **measure style as a rate, not as CE.** free-generate the dev/test prompts (greedy + T=0.7, 4 samples), count replies with a
   kaomoji, lowercase-start rate, slop-starter rate and `<|im_end|>`-at-step0 rate, and compare to base. that metric matches what dawn
   sees and doesn't reward guessing which of 5 kaomoji the builder picked.
3. **if kaomoji placement is wanted, teach the decision, not the glyph.**
   - a position-conditioned sampler nudge (+b on `" ("` at a reply's last sentence end) is zero-parameter, persona-gated and exactly
     scorable with `shape=1`;
   - or the same rank-1 edit restricted to the single initiation token `" ("`, scored on initiation positions only (one row, no
     glyph identities).
4. the 4 step0 `<|im_end|>`s (an empty turn) are worth a sampler guard: at step0 of an assistant reply, `<|im_end|>` −B. that's zero
   collateral elsewhere, the same reasoning as §6.
5. voice proper (VN banter, abstention, ja) stays Phase 3 LoRA with the full forward, and the dev/test split + rate metrics above as its
   bars.