# opus r36 track A: decode predL_/predL2_ token-0 fast path + gemv micro-opts

patch: `/tmp/opus_r36_decode/opus_r36_decode.patch` (`git -C vendor/strata apply --check` clean against the current working tree)
scratch: `/tmp/opus_r36_decode/strata` (build in `build-metal`, METAL_CPP_DIR set; `window.cpp` + `strata_window_cli` build clean, no new warnings)

## what changed
- `window.cpp` `Window::layer` pred block: when `STRATA_METAL_PREFETCH_DRAFT <= 0` (read the same way as stream.cpp `env_int`, default 1)
  the `predL_` / `predL2_` gemvs run at `ntok = 1` on the window's tile (`tile_ntok = ntok`). otherwise unchanged.
  verified readers: `ExpertStream::ahead` is the only consumer of the float pred rows (via `at_layer` / `after_layer`), and with
  `k_draft_max <= 0` it skips every `t > 0`. prompt chains pass `pred = nullptr` (cut_ids_), so they're unaffected. live
  server (pid 78503) runs with `STRATA_METAL_PREFETCH_DRAFT=0`, so this path is on in production.
  rows t >= 1 of predL_/predL2_ now hold stale data; nothing reads them in this mode.
- `Window::gemv`: new trailing `int tile_ntok = 0` (0 = use ntok). **NOTE: this touches the declaration at window.cpp:320**
  (outside the listed line ranges, but the default parameter can't be added any other way).
- prewarm: no change needed, `prewarm_pipelines` runs `layer()` for t = 1..max_t_, so it makes `gemv_8_16_1_4_1#1`.
- `gemv_parity --pred TYPE NIN NOUT NTOK L,R,S`: new mode. token 0 of 2x gemv#NTOK vs 2x gemv#1 vs 1x gemvc#1, memcmp, then
  3-way rotated paired timings (GEMV_AB_COLD=1 for DRAM weights).
- `gemv.metal`: **no change shipped** (see below).

## measurements (M5, live server running, so a few runs were noisy outliers; the 6x repeats below were stable)
parity: token 0 bitwise for nt = 2, 3, 4 (all three variants). `gemv_parity` default: ALL PASS (7.9 s).

Q8_0 2560 -> 512+512, tile 16,1,4,1, median of 15 rotated rounds, 6 invocations each:
| | 2 gemv #4 (before) | 2 gemv #1 (shipped) | gemvc #1 |
|-|-|-|-|
| hot | 21.64-21.66 us | 18.09-18.14 us (-16.3%) | 18.47-18.54 us (-14.5%) |
| cold | 24.07-24.32 us | 22.72-23.13 us (-5.6%) | 22.82-23.15 us (-5.2%) |

nt=3: hot -3.1%, cold ~0. nt=2: hot +1.3%, cold -0.8% (#2 already takes the raw-x direct path, so it's a wash).
gemv_cat at #1 loses to the concurrent pair in every config, so it's NOT wired.
expected saving per 4-token verify window: ~46 x (1.4 .. 3.5) us = ~65-160 us of GPU time, before overlap with the
router GEMV they sit beside. small but free; not measured end-to-end (no full-model runs per rules).

## gemv.metal experiments (rejected, all bitwise, `--ab` vs pristine kernels)
- `gemv_direct_x` widened to NTOK <= 4 at LPR 16 NR 1: nt=4 +1.7% hot / +4.3% cold (2560->512), +1.3..1.4% (2560->10240);
  nt=3 ~0. matches the existing "wider raw-dot windows did not improve" note. reverted.
- `[[max_total_threads_per_threadgroup(SG * 32)]]` on gemv: 0.0 +- 0.9% across nt 1/4 hot/cold. reverted.
- for reference, single gemv #1 2560->512 Q8_0: 16,1,4,1 5.9 us hot / 12.0 cold, 32,1,4,4 7.1 / 11.7, 32,1,4,1 6.4 / 11.6,
  so a non-bitwise tile swap for the pred path wouldn't buy anything either.