# opus r35 track A: router dq reuse + gemm_f16 tiles (TN=96, M%128 padding), SLC swizzle (rejected)

patch: `/tmp/opus_r35_dense/opus_r35_dense.patch` (`git -C vendor/strata apply --check` = OK)
scratch tree: `/tmp/opus_r35_dense/strata`; test binary `/tmp/opus_r35_dense/bin/dense_mm_parity` (linked against
`vendor/strata/build-metal/libmtl_rt.a`, run with `STRATA_METAL_KERNELS=/tmp/opus_r35_dense/strata/metal/kernels`).
Files touched: `metal/kernels/dense_mm.metal` (+1 instantiation), `window.cpp` (gemm selector 3458-3470, router 3668-3687
only), `metal/tests/dense_mm_parity.cpp`. No full-model runs, :8080 untouched.

## 1. router dequant reuse (window.cpp prompt_chunk)
Under `stream_ && pred_on && !p.ids3.ptr` the router of layer l+1 that layer l dequantizes for the read-ahead is reused as
layer l+1's own router: `w_r` / `w_r2` ping-pong between `p.Wr` and `p.Wr2` by layer parity, `dq(tr, w_r)` only at l == 0
(or when reuse is off). PRED2 (`p.ids3`) overwrites Wr2 with l+2, so it keeps the old path. Same tensor, same `dq`
call, same perm → bitwise-identical weights. Hazards: the dq into `w_r2` at layer l targets the buffer layer l-1's
router GEMM read (several barriers back, in hc_read); `w_r` / `w_r2` are always distinct within a layer. `run_graph_seg`
runs the segments in order with no replay, so layer l+1 always sees layer l's dq.
Saves 47 `dq_f16` dispatches (512x2560) per streamed chunk; measured dq cost per call (`--dq-ab`, 2560x512): 17-20 us
(quantized types), 30 us BF16, 57 us F32 → 0.8-2.7 ms of dq work per chunk (they overlap other dqs, so wall saving is
lower). Compiles (`clang++ -fsyntax-only` with the build's flags); not measured end-to-end (no full-model runs allowed).
**Needs a prompt_parity / streamed-prompt logits check before merging** (I couldn't run the model).

## 2. gemm_f16 tiles (dense_mm_parity --tiles / --ab, all memcmp == 0 vs 128_64_4)
Methodology: the GPU was shared (other agents/server: Device Utilization swinging 0-84%), so medians of 5 rounds were
useless (2-4x swings). `--tiles` now reports the min over interleaved rounds (`DENSE_ROUNDS`, default 5/9) and can be
restricted with `DENSE_TILES=a,b`; repeated focused runs agreed within ~1-3%.

### Walbe (N=96, K=2560, 36 calls): new `gemm_f16_16_96_2`, used at M >= 640
| M | 16_64_2 | 64_32_2 | 16_96_2 | 32_96_2 | 64_96_4 |
|---|---|---|---|---|---|
| 512 | 29.6 | 31.5 | 30.2 | 39.7 | 40.1 |
| 640 | 33.0 | 34.3 | **30.9** | 41.0 | 40.5 |
| 768 | 39.8 | 37.4 (cur) | **33.0** | 41.4 | 46.4 |
| 1024 | 55.4 | 42.9 (cur) | **41.5** | 47.5 | 48.0 |
| 1983 | 116.8 | 103.8 (cur) | **91.6** | 92.0 | 94.1 |
| 2048 | 122.8 | 98.3 (cur) | **94.3** | 96.0 | 95.2 |
64_96_2 and 128_96_4 are 2-3x slower (not shipped). -3..-12% on the call, but only ~0.3-0.4 ms/chunk total.

### big-N GEMMs (Wqkv, Wz, Wout, Wq, Wpv, Wshd; the `else` branch), M < 1536
128_64_4 loses whenever M % 128 is 1..64 (a 128-row tail tile that is up to half padding); 64_128_4 ties it elsewhere.
New rule: `M < 1536 && (M % 128 == 0 || M % 128 > 64) ? 128_64_4 : 64_128_4`. Focused runs (us/call, 128_64_4 → 64_128_4):
- M=192: Wqkv 920 → 787, Wz 708 → 552, Wout 683 → 563, Wq 1135 → 942, Wshd 64 → 51
- M=306: Wqkv 1686 → 1412, Wz 970 → 847, Wout 1013 → 888, Wq 2044 → 1710, Wshd 92 → 80
- M=448: Wqkv 2250 → 1990, Wout 1353 → 1233, Wq 3240 → 2862; M=576/700: -7..-9%; M=1300/1450: -4..-6%
- ties (rule keeps 128_64_4): M=256, 384, 640, 896, 1000, 1100
`--ab` chunk totals (sum of isolated GEMMs x calls, r34 → new): M=150 1.17x, 192 1.14x, 306 1.13-1.14x, 448 1.09x,
520 1.07x, 700/800 1.05x, 1050 1.03-1.05x, 1200 1.04x; 200/330/640/1000/1400/1600/1983/2048 1.00x (unchanged tiles bar
Walbe). So the win is on the tail / short chunks (a 306-token prompt: ~22 ms of dense GEMM per chunk), not 2K chunks.

### W512 (N=512, 95 calls), Wshgu (640), Wk (1152/1664): no change
The current picks (32_64_2 <384, 32_128_2 above; Nn<=1664 rule) are within ~1-3% of the best everywhere, e.g. W512
M=1983: 32_128_2 402-404, 64_128_4 403-404, 64_64_2 405, 128_64_4 416; M=1024: 32_128_2 197, 64_64_2 197, 64_128_4 210.
Only W512 M=768 (128_64_4 -4%) and Wshgu 64_64_2 at M~1024 (-2.5%) showed anything, inside run-to-run noise. Not shipped.

### SLC swizzle (rejected, not in the patch)
Tried `gemm_f16_sw<TM,TN,SG,G>`: threadgroup ids remapped so G column tiles of B run down all row tiles of A (same tiles,
same bits, memcmp 0). 128_64_4_sw{2,4,8} and 64_128_4_sw{2,4,8} vs plain on Wqkv/Wz/Wout/Wq at M = 256..2048: never a
reproducible win; 128_64_4_sw2 ≈ plain (±1%), sw4/sw8 up to +10-60% slower, 64_128_4_sw* often 2-3x slower (e.g. Wqkv
M=1983: 7827 plain vs 28523 sw2 / 23062 sw4). The default dispatch order already keeps B cached well enough; the swizzle
kills the row-tile reuse of A between concurrently running threadgroups.

## verification
- `dense_mm_parity` (parity): ALL PASS
- `--tiles M` (all 12 tiles incl. 16_96_2 x 13 shapes) at M = 100, 256, 306, 640, 1000, 1983, 2048: ALL BITWISE
- `--ab M` at 150..2048: ALL BITWISE
- window.cpp: compiles (-fsyntax-only, build flags). Router reuse not run end-to-end (forbidden here).