# opus_r29_prompt — Round 29 Track A: `gemm_swiglu_gu` shared-expert gate+up fusion + `route_topk` unrolls

patch: `/tmp/opus_r29_prompt/opus_r29_prompt.patch` (`git -C vendor/strata apply --check` OK). It touches
`metal/kernels/prompt.metal` (route_topk, +3/−1 lines) and `metal/tests/comb_fold_parity.cpp` (new `gu` + `topk` modes).
**`window.cpp` is unchanged.** The fusion doesn't win robustly (details below), so I didn't wire it in.
Scratch tree + build: `/tmp/opus_r29_prompt/strata`, `/tmp/opus_r29_prompt/build` (run with `STRATA_METAL_KERNELS=/tmp/opus_r29_prompt/strata/metal/kernels`).

## TL;DR
- **gemm_swiglu_gu (two accumulators, one A tile) is bitwise identical but not faster.** On average it ties
  `gate gemm_f16 + gemm_swiglu_up` (±2–3%, which is the noise floor here). It wins −3..−5% at T = 192/256, loses
  +10..+14% at T = 224/288, and loses +1..+3% at 384..1536 in the most recent repeated runs. The kernel is compute-bound
  on the matrix units, and the gsh round trip (the "503 MB") overlaps with that compute. Two TM×TN accumulators halve
  occupancy, which eats the saving. This matches Round 11's finding (`audits/opus_r11_dense.md`: 2-acc fused 1016 µs vs
  976 µs unfused). Not shipped.
- **Interleaved single-accumulator variant `gemm_swiglu_il`** (one GEMM over B = gate/up rows interleaved, row 2j = gate
  j, 2j+1 = up j; each M5 lane's 4 columns = (g,u) of two outputs, half2 store) is also **bitwise identical**. It is the
  only shape with a repeatable win: **−3.0..−4.4% at T = 1024** (≈ −1.0 ms/chunk, 6 of 6 runs, tiles il128_64_4 /
  il128_128_8 / il64_256_8). It ties at 2048 (−0.8..+1.2%) and is +1..+3% at 512. Shipping it needs `dq(Wshg)`/`dq(Wshu)`
  writing one interleaved `[1280, 2560]` buffer (`dq_f16`'s `perm = 2` over the pair; window.cpp lines 3674-3676 +
  dense_mm.metal). That is outside my boundaries, so it stays in the test as a measured follow-up candidate.
- **route_topk:** the `#pragma clang loop unroll(full)` additions are **no-ops**. With only the pragmas (lines 35, 41,
  104, 106, 125) the A/B ratio is 1.000–1.002, so the compiler already fully unrolls these constant-trip loops. The real
  cost was `v[ex / 32] = -INFINITY`, a *runtime* index into `float v[16]`. Replacing it with an unrolled select
  `v[i] = i == (ex >> 5) ? -INFINITY : v[i]` gives **−0.7..−1.2%** (ratio 0.988–0.993 in 3 of 3 quiet runs, every T and
  MoCE setting). It is bitwise identical (ids + w, 512 and 256 experts, MoCE 0 / 320335 / −16.3 / −0.05). The absolute
  gain is tiny: route_topk is ~2.3 ms per 48 at T=2048, so ≈ 20 µs/chunk. That is the only shipping change. I left the
  pragmas out of the diff because they measured nothing.

## Parity (all `memcmp == 0`, outputs pre-filled 0xCD)
- `comb_fold_parity 0 gu`: T ∈ {1, 7, 161, 256, 512, 1024, 1983, 2048} × 7 gu tiles + window path → 65 PASS
- `comb_fold_parity 0 gu check il64_128_4,il128_64_4,64_64_4` → 33 PASS (interleaved)
- During exploration, also bitwise: threadgroup-parked silu(g) (one accumulator, 8–32 KB TG) and K-stepped TK=32
  `multiply_accumulate` (two accumulators) at 3 tiles each, at every T.
- `comb_fold_parity 0 topk base_prompt.metal` → 25 PASS; `route_parity` PASS (K=512 and K=256+MoCE)
- regressions: `comb_fold_parity 1024` 21 PASS, `0 epi` 113 PASS, `256 hcup` 16 PASS, `0 ab base_prompt.metal` 22 PASS
- window.cpp variant (gu in the mm_hist wave next to Wsg, `swiglu_h` only when `!swu`) compiled clean in the scratch
  engine build before I reverted it. The full model was never run (rules).

## Timings: 48 shared experts, ms per chunk, median of 9 alternating; Δ vs `gate + gemm_swiglu_up` (the window at T > 160)
GPU shared with the live :8080 server, so absolute numbers drift up to 2×. Runs whose baseline inflated are discarded or marked.

| T | gate+swiglu_up | best 2-acc gu (Δ, repeated runs) | best interleaved il (Δ) |
|---|---|---|---|
| 192 | 4.6–5.0 | 64_64_4: −3.1, −3.3, −4.8, −4.8 | il64_128_4: −8.7, −7.3 (one run each way) |
| 224 | — | 32_64_2: +1.3 median; 64_64_4: +9.9 | — |
| 256 | 5.9–6.4 | 64_64_4: −3.3, −5.5, −2.9, −5.0 | il128_64_4: −8.3 median (noisy: −18.7..−2.5) |
| 288 | — | 64_64_4: +13.9 median | — |
| 320 | — | 64_64_4: +2.8 median | il32_64_2: +2.2 |
| 384 | 8.7–10.0 | 64_128_4: −1.4, −1.0, then +3.0, +3.0, +1.5, +3.4 | — |
| 512 | 11.6–12.9 | 64_128_4: −2.1, −1.6, then +2.5, +4.4, +2.6, +3.7 | il128_64_4: +1.2, +1.8, +2.2 |
| 1024 | 24.2–27 | 64_128_4: ±1.5 (8 runs, median ~0) | **il128_64_4: −3.8, −4.4, −3.7, −5.2, −3.0, −3.0** |
| 1536 | 37–41 | 128_128_8 / 64_128_4: −1..+2 | — |
| 2048 | 50–58 | 64_128_4: −2.1, −1.7, −0.4, −2.4, +1.4, +1.1 | il128_64_4: +0.5, +0.3, −0.8 |

Other shapes swept (all slower or tied): 32_64_2 / 32_128_4 / 64_64_4 at T ≥ 512 (+14..+97%), 128_128_4 (+20..+120%),
64_128_8 (+19..+48% at ≥ 512); TG-parked silu(g) (`64_128_4` +0.7..+9%, smaller tiles same as 2-acc); K-stepped
TK=32 (+9..+13% at 1024/2048), TK=64 at 64_128_4 (+20..+26%).
At T ≤ 160 the overlapped `gate ∥ up + swiglu_h` beats every fused form (+11..+48%), so `swu`'s 160 threshold stands.

## Why the "503 MB" doesn't show up
gsh is T·640·4 B written and read once per layer. At T=1024 that is 5.2 MB ≈ 40 µs of DRAM per layer, but it runs
concurrently with a ~500 µs compute-bound GEMM. Removing a round trip only pays when the extra accumulator costs
nothing. The single-accumulator interleave keeps occupancy, which is why only `il` shows a (small) T=1024 win. The
"`mixh` read twice from DRAM" claim doesn't hold either: each A row-block is already re-read by NFF/TN = 5–10
N-tiles, so A is cache-resident either way.

## Test usage
- `comb_fold_parity T gu [bench [tile,...]]`: tiles `TM_TN_SG` (two-accumulator `gemm_swiglu_gu`) or `ilTM_TN_SG`
  (interleaved). T 0 = the parity sweep. bench rows: `gate ∥ up + swiglu_h`, `gate + gemm_swiglu_up`, then each tile.
  Without bench, pass any word in place of `bench` to give a tile list (e.g. `0 gu check il128_64_4`).
- `comb_fold_parity T topk baseline-prompt.metal`: route_topk bitwise vs a baseline copy (512 and 256 experts, 4 MoCE
  settings), plus paired ×48 times at 512 experts. T 0 = 512, 1024, 2048.
- Both fused kernels live in the test (`kSwigluFused`), not in prompt.metal: zero runtime-library cost, no dead code in
  the engine's kernels.

## Follow-up (needs files outside this track)
Interleaved shared gate/up: dq both into one `[2·NFF, N]` f16 buffer with rows interleaved (dense_mm.metal `dq_f16`
needs a row offset, or the two GGUF tensors dequantised as one `perm = 2` pair), then dispatch `gemm_swiglu_il`
(il128_64_4) at T ≥ ~768 in the mm_hist wave. Expected ≈ −1 ms/chunk at T=1024, ~0 at 2048. Worth it only if the
1024-row streaming chunk is the common case.