# opus audit v3: phase 3 prep fix verification

read-only · auditor: opus-verify-phase3-v3 · scope: the 9 items in `audits/opus_audit_phase3_prep_v2.md` §fix list.
the DB was opened `mode=ro`, probes ran in-kernel with `sys.dont_write_bytecode=True`, and no `__pycache__`/`*.pyc` exists anywhere in the repo afterwards. the only file written is this report.

**verdict: NOT PASS. one new major false positive in the M2' hypothesis path. items 1–4 and 6–9 pass.**

the new causal-clause regex `PRE_TOOL_HYPOTHESIS_CLAUSE_RE` (`prepare_phase3_sft_data.py` L83) has no word boundary before `since|because|as|so that`, and it treats bare `as` as causal. so any pre-tool sentence that contains the letters `as␠` (`was`, `has`, `alias`, `as root`, `as well`) gets cut mid-word, and only the garbage tail is kept as the "hypothesis". this hits exactly the falsifier ("any false positive on M2' probes").

## self-tests (python3 -B)

| script | exit | time |
|---|---|---|
| `mine_llm_isms_from_transcripts.py --self-test` | 0, PASS | 0.07 s |
| `scripts/prepare_phase3_sft_data.py --self-test` | 0, PASS | 0.08 s |
| `scripts/phase3_lora_and_transplant.py --self-test` | 0, PASS | 0.71 s |

## 1 & 2. emoji strip / layout (B2', B3'): PASS

- the global `(?<=\S)[ \t]{2,}(?=\S)` collapse and the `[ \t]+([.!?,;:])` punctuation regex are both gone. `_strip_emoji_preserving_layout` (L181) only touches the `[ \t]*` captured right next to each emoji match, and fences are split out and passed through untouched (L361–368).
- **live data (fresh run, 658 text turns):** `code_fences_altered = 0 / 140` fenced turns. that's from both the script's own check and my own `findall` comparison. there are 0 unclosed-fence turns and 0 `~~~` fences.
- **stricter check:** on turns with no emoji and nothing stripped, the whitespace skeleton is identical in 100% of cases. the only indented lines that changed (2 turns) are nested bullet prose being lowercased by design, not code.
- the committed `results/phase3_speech_act_surgery_report.json` says `code_fences_altered: 0 / 140` (657 turns, since the DB has grown by one since). the seed JSONL equals a fresh `build_synthetic_phase3_seed_examples()`.
- **probes, all with an emoji stripped in the same reply:**
  - an aligned fence (`x      = 1`, `print(a,  b)`), a padded table (`| name   | kld    |`) and `    make  -j8   all` all come through byte-identical.
  - `( ; ω ; )` and `` `a ; b` `` survive, while `🚀 ✅ 🇯🇵 ⭐` are stripped.
  - `"fixed it 🚀\n"` → `"fixed it\n"`, so there's no trailing space anymore.
- **nits (not blocking):**
  - (a) an emoji *inside* inline code is stripped: `` `echo 🚀` `` → `` `echo ` ``. it happens in 1 of 658 live turns.
  - (b) a table cell padded around an emoji loses its padding: `| name 🚀  |` → `| name |`. that's only at the emoji position, which the v2 spec allowed.
  - (c) the final `.strip()` (L385) eats the indentation of a reply whose *first* line is 4-space code.
  - (d) `~~~` fences aren't protected from emoji stripping or lowercasing. there are 0 in the live data.
  - (e) `hello🚀world` → `helloworld`.

## 3. ABSTENTION_TEMPLATES: PASS

- none of the 6 templates contains anything from `banned_ngrams` (32), `phase3_speech_act_ngrams` (25), `phase4_lexical_ftpo_ngrams` (7) or the 576 `surface_variants`. I checked both word-boundary and raw substring matches.
- there's no `?` and no `want me` anywhere, and every template passes through surgery unchanged.
- the self-test asserts against `banned_ngrams`, which is a superset of phase3 ∪ phase4 (I checked), so the docstring's claim holds.

## 4. closing guard (M1'): PASS (false negatives noted)

- `can(?!'t|\u2019t|not)` plus `OFFER_ACTION_VERB_PAT` are in place (L108–125). `CHOICE_FORK_QUESTION_RE` (L142) with `(?!or\b)` handles the leading `Or`.
- `TECHNICAL_TOKEN_IN_SENT_RE` was deleted, which is acceptable ("wire up or delete").
- **kept (0 false positives in 14 probes):**
  - `I can confirm the seed…`, `I can't help much…`, `I cannot…`, `I can not…`
  - `Should I write the parser in Rust or Go?`, `Would you like to keep the old schema or migrate everything now?`
  - `Would you like Postgres or SQLite…?`, `Do you want the cache in Redis or in-process?`, `Want the tables sharded by user or by tenant?`
  - `If you want lower latency, use…`, `If you want the details, the root cause…`, `If you want to reproduce it, run…`
  - `If you'd rather avoid swap, I can see why…`, `Should I be worried about…?`
- **stripped as intended:**
  - `Do you want me to apply…?`, `Or do you want me to revert it instead?` (sentence-initial `Or`), `If you want, I can apply the fix now.`
  - `If you want, I'll push it.`, `Want me to run the tests?`, `Let me know if…`, `Shall I commit this?`, `Should I go ahead and commit this?`, `Happy to help with anything else!`
- live data: the fork exemption keeps 3 final sentences. 2 are real forks (branch vs `main`, extend-login vs trace-wire-format). 1 is borderline (`…or were you just checking that permissions work?`).
- **false negatives (minor, not the falsifier):** the fork regex matches any ` or `, so binary permission asks survive:
  - `Do you want me to push this or not?`
  - `Should I commit this, or not?`
  - `Would you like me to fix this or anything else?`
  - `Would you like me to apply the patch, or would you rather review it first?` (the classic menu-offer)

  suggestion: don't count `or (not|anything else|something else|would you rather|should i|do you want)` as a fork. separately, `If you want, I can apply it to main.py.` survives because `[^.!?\n]` stops at the `.` in `main.py`.

## 5. pre-tool hypothesis / questions (M2'): FAIL (new false positive)

- the v2 probes now pass:
  - `…because the OOM killer usually targets the biggest cgroup.` → `the OOM killer usually targets the biggest cgroup.`
  - `I'll test this hypothesis: the fd leaks…` → `the fd leaks…`
  - `…first, then we can talk about the race.` → `we can talk about the race.`
  - `Let me see if I understand: you want the cache cleared on boot?` is kept whole (the `?` guard is at L256 and L311).
  - `Let me check the config? It might be stale.` is kept.
- **new false positives.** each is a single pre-tool sentence with `has_following_tool_call=True`:

| input | output |
|---|---|
| `Let me check whether this was the cause.` | `the cause.` |
| `Let me check what the service has to say in the journal.` | `to say in the journal.` |
| `Let me look at the alias table.` | `table.` |
| `Let me verify the alias works.` | `works.` |
| `Let me run the tests as root.` | `root.` |
| `Let me check the logs as well.` | `well.` |
| `Let me check the logs so that we know which worker died.` | `we know which worker died.` (purpose clause, not a hypothesis) |

- cause: in L83 `(?:,?\s*(?:since|because|as|so\s+that)\s+|:\s+|,\s*then\s+)`, the `\s*` can match zero characters, so `w·as␠`, `h·as␠` and `ali·as␠` match. `as` is also ambiguous even as a whole word.
- these sentences should either be deleted whole (pure narration) or kept verbatim. a mid-word fragment becomes a broken SFT target.
- live impact today is ~0: only 1 pre-tool strip on 4 text+tool turns, and that one is correct (`"no, i haven't. let me look."` → `"no, i haven't."`). but this path will run over all teacher-generated phase 3 data.
- **fix:**
  - require `(?:,\s*|\s+)\b(?:since|because)\b\s+` (or `,\s*as\s+` only after a comma), and drop `so that`.
  - add the 7 probes above to self-test §5, asserting the output is either `""` or the unchanged input.

## 6. R-Tuning aliases + naming: PASS

- a flat list is always any-of (L428–431). `["42","forty-two"]` → `known_keep_teacher` at 1.0 (self-tested), and the teacher answer `"forty-two"` alone also passes.
- nested groups require every group: `[["42","forty-two"],["seconds","sec"]]` gives 0.75 when one sample says `42 s`, which is correct.
- `match_mode="all"` is explicit opt-in.
- `_word_boundary_おそらく` is gone everywhere. only `_word_boundary_match` remains.

## 7. transplant: PASS (nits)

- the `(ggml_type:int, bytes)` tuple is mandatory (L273), a bare-bytes call is self-tested to raise `ValueError` (L505), and the type is checked against the header (L278).
- all 4 regexes are used via `.fullmatch` (L113–128).
- the directory is fsync'd after `os.replace` (L312–316).
- PLAN §2.2 L60 now reads 12 `attn_output` + 36 `ssm_out` + 48 `ffn_down_shexp` = 96 writers, plus 96 routers = 192, with the SC117 144 explained.
- **nits:**
  - (a) the L93 comment still says "144 writers + 96 router gates = 240 tensors". it's stale, make it 96 + 96 = 192.
  - (b) `isinstance(item[0], int)` accepts `bool`, so `(True, buf)` passes as type 1 = F16. harmless in practice, but `type(item[0]) is int` is stricter.

## 8. Q2_0 compiler fallback + PLAN wording: PASS

- `shutil.which("clang") or which("gcc") or which("cc")` (L436), with a clear `RuntimeError` if none is found.
- PLAN #28 L271 now says "compiled C reference transcribed from `ggml-quants.c`".
- **nit:** `_Float16` needs GCC ≥ 12 on x86-64. older Colab images would fail to compile instead of skipping.

## 9. miner docstring: PASS

L18–21 now describe the 85/15 mixture (`0.85 * p_indep + 0.15 * p_unif`, no self-pooled term).

---

## remaining fix list
1. **(major)** `PRE_TOOL_HYPOTHESIS_CLAUSE_RE` L83: add `\b` and require a real separator before `since|because`, drop bare `as` (or require `, as`) and `so that`. add the 7 probes from §5 to the self-test.
2. (minor) `CHOICE_FORK_QUESTION_RE`: don't exempt `or not` / `or anything else` / `or would you rather` permission asks.
3. (nit) L93 comment 144/240 → 96/96/192; `type(...) is int`; skip emoji stripping inside inline code; protect `~~~` fences.