# opus audit v2: phase 3 prep fix verification

read-only · auditor: opus-verify-phase3-v2 · scope: the 6 fix groups in `audits/opus_audit_phase3_prep.md` §4.
upstream reference: llama.cpp `master` fetched 2026-10-15 (`ggml/src/ggml-quants.c`, `ggml/src/ggml-common.h`, `ggml/include/ggml.h`, `src/models/qwen4exp.cpp`, `src/llama-arch.cpp`, `src/llama-model.cpp`). probe scripts and fetched sources are in `/tmp/v2audit/` (`q2probe.py`, `clsprobe.py`, `surgprobe.py`, `repprobe.py`, `repprobe2.py`, `up_q2.c`). the sqlite DB was opened `mode=ro`. the only file written in the repo is this report.

**verdict: NOT PASS. one blocker left, in the surgery formatting path.** the codec, the transplant guard and the miner/PLAN work all check out. surgery still changes the contents of code fences on real data:

- the final double-space collapse (`re.sub(r"(?<=\S)[ \t]{2,}(?=\S)", " ", ...)`, `prepare_phase3_sft_data.py` L351) runs over the **whole reply, fences included**. on the live gemini reference set it rewrites fence contents in **34 of 140 fenced turns (24%)**, and all 34 are explained by that one regex alone. aligned assignments, `print(a,  b)`, padded markdown tables, 4-space indented code blocks and string literals with double spaces all get changed. this is the B2 falsifier ("broken markdown"), just a smaller version of it.

everything else is major or minor (below).

self-tests: `python3 -B mine_llm_isms_from_transcripts.py --self-test`, `python3 -B scripts/prepare_phase3_sft_data.py --self-test` and `python3 -B scripts/phase3_lora_and_transplant.py --self-test` all **PASS** (exit 0, 0.07 s / 0.07 s / 0.72 s). there's no `__pycache__` or `*.pyc` anywhere in the repo before or after.

---

## 1. Q2_0 codec (B1): PASS

- the upstream `quantize_row_q2_0_ref` and `dequantize_row_q2_0` are byte-for-byte what the script's docstring says: `d = amax`, `id = 1.0f/d` computed from the *unrounded* float `d`, `q = clamp((int)roundf(w*id)+1, 0, 3)`, `qs[j/4] |= q << ((j%4)*2)`, and `y = ((int)q-1)*d` with `d = FP16_TO_FP32`. `block_q2_0` = `ggml_half d` + `uint8_t qs[16]` (18 B, `QK2_0 = 64`), and `GGML_TYPE_Q2_0 = 42`.
- **independent golden:** I pulled the two functions **verbatim out of upstream `ggml-quants.c`** (not the script's re-typed copy), compiled them with an fp16 shim (`/tmp/v2audit/up_q2.c`), and compared:

| case | quant bytes == C | dequant == C |
|---|---|---|
| N(0,1) ×16,384 | yes | yes |
| heavy-tailed (x³·50) ×4,096 | yes | yes |
| exact halfway (±0.5·amax, 0.4999999, 0.5000001, scales 2⁻²⁰…2¹⁰) | yes | yes |
| all-zero blocks | yes | yes |
| fp16-subnormal scales (1e-6) | yes | yes |
| random bytes, all 4 codes, incl. inf/NaN fp16 `d` (512 blocks) | n/a | yes (NaN==NaN) |

  pure-Python path (system `python3`, no numpy) and the numpy path (`uv run --with numpy`) both match **bit-exactly**. the dequant return type is `array('f')` and `np.ndarray` (float32) respectively, so it's contiguous. no boxed lists.
- the double-precision emulation is sound. float/float division and float×float products done in double and then rounded to float give exactly the IEEE float result, and `c_roundf` reproduces round-half-away.
- **minor:**
  - (a) the self-test's "compiled C ggml-quants.c" is a hand-transcribed copy, not the upstream file. it's correct (my verbatim extraction agrees), but PLAN #28 L271 overstates it.
  - (b) the self-test hard-codes `/usr/bin/clang` and `_Float16`, so it will fail on a Colab/Linux box without clang. fall back to `cc`/`gcc`, or skip with a warning.
  - (c) `quantize_q2_0_f32` raises `OverflowError` for amax > 65504 where C stores `inf`. that's irrelevant for real weights.
  - (d) the quantizer is pure-Python only. that's fine, since experts are never requantized.

## 2. tensor classification + transplant (B4): PASS (minor nits)

- I built every tensor name `qwen4exp.cpp::load_arch_tensors` can create (including the `create_tensor_qkv` / `create_tensor_gate_up_exps` alternatives from `llama-model.cpp`, and the `nextn.*` names in `llama-arch.cpp`) for `blk.0..48` and classified them:
  - **trainable (1,779):** every trunk name in the PLAN §2.2 L61 list. that's `output_hc_{norm,down,up}`, `attn_{qkv,q,k,v,output,gate,q_norm,k_norm}`, dotted `indexer.{q_proj,k_proj,q_norm,k_norm}`, `ssm_{conv1d,beta,alpha,norm,out}`, `ssm_dt.bias`, `ple_{key,value,norm_key,norm_query,norm_conv,conv1d}`, `hc_{attn,ffn}_{norm,down,up,inject}`, `ffn_gate_inp`, `ffn_gate_inp_shexp` and `ffn_{gate,up,down}_shexp`.
  - **no trainable name contains `_exps`, `embd`, or `blk.48.`.**
  - `frozen_routed_experts`: `ffn_{gate,up,down,gate_up}_exps` (+ `.bias/.scale`). fused `gate_up_exps` is now caught.
  - `frozen_ple_and_embd`: `token_embd`, `output`, `per_layer_token_embd`, `ssm_a` (bare name, deliberate) and all of `blk.48.*`.
  - `other_frozen`: only the optional `attn_{q,k,v,qkv}.bias` (absent in this arch's GGUFs, and frozen is safe).
- `transplant_gguf_tensors` refuses anything with `cat != "trainable_dense_router_shared"` *before* copying. the self-test exercises the fused expert, an unknown `other_frozen` and a wrong-`ggml_type` tuple, and all three raise `ValueError`. the header and expert bytes stay identical after a legitimate write. `.tmp` gets unlinked on error, and there's a free-space check.
- `swift_writers_plus_router` scope: only `ssm_out`, `attn_output`/`attn_out`, `ffn_down_shexp` and the two router gates in `blk.0..47`.
- **minor:**
  - (a) bare `bytes` replacements are still accepted with no type check. the audit's "at minimum take `(ggml_type, bytes)`" is only optional. a same-size wrong-type buffer passed as bare bytes still goes through. make the tuple mandatory.
  - (b) the regexes use `.match` with `$`, so `"blk.4.ssm_out.weight\n"` classifies as trainable. GGUF names can't contain newlines, so it's cosmetic, but `fullmatch` is cleaner.
  - (c) there's no directory fsync after `os.replace`.
  - (d) `attn_out` isn't a qwen4exp name (`LLM_TENSOR_ATTN_OUT` → `attn_output`). it's harmless in the regex, but PLAN §2.2 L60 calls the family `attn_out.weight`.
  - (e) **PLAN count:** in qwen4exp each layer is *either* full-attention (`attn_output`) *or* GDN (`ssm_out`), via `if (!hparams.is_recr(il))`. so the writer families give 48 + 48 = **96** tensors on a 48-layer trunk, not "144" (L60). re-check against the SC117 card before quoting 144/240.
  - (f) the trunk range `0..47` is hard-coded, not read from `block_count`. it fails safe (anything else is frozen).

## 3. surgery formatting + kaomoji (B2, B3): PARTIAL, blocker remains

**fixed:**
- span deletion is now in place. newlines, bullet/numbered/nested lists, tables and fences survive opener, pre-tool and closing removal.
- a trailing fence is never touched, and a solicitation after a fence is removed cleanly (`"patch:\n```diff…```"`).
- the seed JSONL (6 rows) equals both a fresh `build_synthetic_phase3_seed_examples()` and the report preview.

**kaomoji:**
- `✧ ♡ ♥ ✿ ☆ ♪ ★ ♨ ❀`, `(◕‿◕✿)`, `٩(◕‿◕｡)۶`, `(´• ω •`)` and `(｡•̀ᴗ-)✧` all survive.
- on the live DB only `🐾 ✅ ❌ ⚠ U+FE0F ✨` get stripped (6 chars, 0 kaomoji). the report's count of 1 differs only because the DB grew by one argon turn (656 → 657 text turns, 27,042 → 27,046 silent tool calls).

**still broken:**
- **B2' (blocker): fence/table/indent whitespace collapse** (L351, described above). measured: 34/140 fenced real turns changed. probes:
  - `x      = 1` → `x = 1`, `print(a,  b)` → `print(a, b)` inside a ```python fence
  - `| name   | kld    |` → `| name | kld |`
  - `    make  -j8   all` → `    make -j8 all`

  fix: run the collapse only on non-fence, non-inline-code segments, and ideally only around the positions where emoji were removed.
- **B3' (major, low frequency): the emoji-adjacent punctuation cleanup** `re.sub(r"[ \t]+([.!?,;:])", r"\1", part)` (L325) runs on the whole non-fence part whenever it contains any emoji:
  - kaomoji: `( ; ω ; )` → `(; ω; )`
  - inline code: `` `a ; b` `` → `` `a; b` ``

  only the space right where the emoji was should be removed. it also leaves a trailing space (`"fixed it 🚀\n"` → `"fixed it \n"`).
- minor: BMP Emoji_Presentation characters outside U+2600–27BF survive (`⭐ U+2B50`, `⏰ U+23F0`, `⌛`).

## 4. surgery false positives (M1–M4, M7): MOSTLY FIXED

**fixed:**
- the 4 original M1 probes, and "If you want lower latency, use the smaller model." / "If you want the details, the root cause is …", are all kept.
- M3:
  - "You're right, but…", "Makes sense, except…", "Exactly right, though…" and "Great question — however…" are kept.
  - "Absolutely not, …" and "You're right that …" are kept.
  - "Good catch! …" is stripped.
- M2:
  - "Let me check X, since Y" → "if it's unset the slice can balloon to 14 GB."
  - "Let me check the unit file. The OOM is …" → "the OOM is …".
  - a trailing pre-tool sentence after an explanation is stripped.
  - `了解です。まずはログを確認します。` → "" and `原因はスワップです。ログを確認します。` → `原因はスワップです。`.
- M4: `README.md`, `Makefile`, `HOME`, `CUDA_VISIBLE_DEVICES`, `K=320`, `Qwen3.8-Flash-Next`, `MemoryHigh`, `GitHub`, `PyTorch`, `Llama.cpp`, `ggml_type`, `x86_64`, `OOMPolicy=kill`, `WireGuard`, `JSON` and URLs/POSIX paths are all preserved.
- M7: the self-test iterates all 25 `phase3_speech_act_ngrams` (15 pre-tool, 9 closing, 1 opener) and asserts the right counter is 1 for each, so coverage is 100%.

**remaining (major):**
- **M1' informational / genuine-question tails still stripped** (with a tool call absent):
  - "If you want determinism, I can confirm the seed is fixed at the top of main." (`i can` matches any statement)
  - "If you want it faster, I can't help much; the bottleneck is the disk." (`\bi\s+can\b` matches `I can't`)
  - "Should I write the parser in Rust or Go?" (genuine design fork; `GENUINE_TECH_TRADEOFF_RE` only knows project jargon)
  - "Would you like to keep the old schema or migrate everything now?" (an either/or decision the user owns)

  fixes:
  - require `can(?!'t|not)`
  - require the offer verb to act *for the user* (`i can (apply|run|push|do|fix|write|…)`)
  - exempt `X or Y?` alternatives

  `TECHNICAL_TOKEN_IN_SENT_RE` (L132) is defined but **never used**. wiring it into the closing guard was part of the M1 fix.
- **M2' hypotheses still lost when there's no comma + since/because/as:**
  - "Let me check the unit because the OOM killer usually targets the biggest cgroup." → ""
  - "I'll test this hypothesis: the fd leaks in the retry loop." → ""
  - "I'll run the tests first, then we can talk about the race." → ""
  - "Let me see if I understand: you want the cache cleared on boot?" → "" (a clarification question, deleted when a tool call follows)

  `PRE_TOOL_NARRATION_SENT_RE` ends in `[^\n]*`, so any sentence that *starts* with a pre-tool verb is deleted whole. fix: allow `,?\s*(since|because|as|so|:)`, and refuse to delete sentences containing `?` or `:` followed by a clause.

**minor:**
- `PPPoE`, `iOS`, `macOS` get lowercased (`pppoe`, `ios`, `macos`). the seed row 2 target contains `pppoe`.
- Windows paths get lowercased.
- the coverage test silently skips if the JSON is missing, has no waiver list, and has no `else: fail` for an unknown category.
- module docstring L18–20 of the miner is unrelated. see §6.

## 5. R-Tuning (M5, M6): FIXED, two issues

**fixed:**
- word-boundary matching: `42`≠`1042`, `rope`≠`europe`, `3.14`≠`3.141`, `8080`≠`18080`.
- the RFC any-of group gives `known_keep_teacher`.
- a template pool of 6.
- a silent `web_search` (`target_response == ""`, `target_tool_call` set).
- `kd_logit_extraction_order = "extract_teacher_topk_logits_after_surgery_on_target_response"`, and PLAN §2.4 L118 now mandates surgery → teacher-forced top-k.
- "pulling it up first" is gone from the code and the targets.

**remaining:**
- **(major) `ABSTENTION_TEMPLATES[0]` ends in "want me to look it up or run a check?".** `want me to` is literally a `phase3_speech_act_ngrams` closing-solicitation phrase. every lost fact whose prompt hashes to slot 0 (≈1/6) trains the exact servile closing that surgery removes everywhere else. surgery doesn't catch it, because there's no sentence boundary before it. drop the question, or use the silent-search target instead.
- **(minor) default-mode alias semantics depend on spaces.** a flat list of single-token aliases is treated as **all-of**: `["42", "forty-two"]` with every answer "42" → `lost_abstain_honest`. a flat list with any multi-word string is treated as any-of. that's a silent trap for whoever writes probe sets. make flat lists always any-of (or require explicit groups), and document it.
- **(nit)** the helper is literally named `_word_boundary_おそらく` (L389/L404, "probably"), a pointless wrapper around `_word_boundary_match`. rename or inline it.

## 6. miner + JSON + PLAN: PASS

- **prior:** `p_bg = 0.85·p_indep + 0.15·p_unif` (L783), per-order normalized, with no pooled term in `alpha_w` (`pooled_*` fields are reference-only). `summary.prior_type = "0.85_per_order_unigram_plus_0.15_per_order_uniform"`.
- **recompute:** all 32 rows' `log_odds_delta` / `z_score` recompute from stored `alpha_w` with N_llm = 97,737, N_hum = 13,455, α₀ = 5000, and **0 mismatches**.
- **c_hum = 0 rows:**
  - 23 rows, **23 distinct deltas**, range 0.626–2.814. this matches PLAN #26 L264 exactly, and there's no ceiling cluster anymore.
  - Spearman(z, c_llm) on those rows = 0.82, down from 0.996 in v1, so the ordering is no longer pure frequency.
- **family floor:** `max(5, int(0.05·c_llm))` with ≥ 2 families meeting it (L765–768). all 32 rows satisfy it.
- **FTPO:** `think that's`, `so far` and `as expected` are absent from every list (and from the miner source). FTPO = `rather than, right away, end to end, exactly what, here's what, what's actually, short version` → 126 tuples, consistent with PLAN §2.6 L172, #27 L269 and `summary`.
- **self-test 5b** now really uses `alpha_0=5000.0` and asserts a > 0.2 delta gap plus a numeric δ/z recompute.
- **PLAN:**
  - §2.6 L174 and #26 say selection is allowlist-driven.
  - §2.2 records both trainable scopes.
  - §2.4 has the any-of matcher, template pool, silent search and logit order.
  - #26–#28 are present.
- **minor:**
  - the miner module docstring L18–20 still says "50/50 mixture of empirical pooled frequency and independent unigram-product". stale, fix it to 0.85/0.15.
  - the L781 comment calls the prior "external/leave-one-out", but the unigram table is built from the same corpus (each n-gram contributes to its own unigrams). it's weak, but it isn't leave-one-out.
  - the `calibration_mix_v2.txt` sha256 is still unrecorded (it waits on a Colab run, as expected).
  - `rather than` / `exactly what` in FTPO are still a judgment call for dawn (common English).

---

## fix list (ordered)
1. **(blocker)** L351: limit the whitespace collapse to non-fence, non-inline-code segments, ideally only where emoji were removed. add a self-test with an aligned fence plus a padded table.
2. L325: only remove the space that came from a deleted emoji. don't run the punctuation regex on kaomoji or inline code. add `( ; ω ; )` and `` `a ; b` `` + emoji to the self-test.
3. remove "want me to …?" from `ABSTENTION_TEMPLATES[0]`. add a self-test asserting no template contains a `phase3_speech_act_ngrams` phrase.
4. closing guard:
   - `can(?!'t|not)`
   - an action-verb offer requirement
   - exempt `X or Y?` forks
   - wire up or delete `TECHNICAL_TOKEN_IN_SENT_RE`
5. pre-tool: keep `because` / `:`-clauses and questions.
6. R-Tuning: make flat alias lists any-of. rename `_word_boundary_おそらく`.
7. transplant: make `(ggml_type, bytes)` mandatory, use `fullmatch`, fsync the directory, and fix the "144" count in PLAN §2.2 L60.
8. Q2_0 self-test: compiler fallback (`cc`/`gcc`) for Linux/Colab. word PLAN #28 as "transcribed from" ggml-quants.c.
9. miner docstring L18–20.