# round 27 track b — qsa prep and merge

## decision

ship parallel aligned `float2 (m,l)` loads, threadgroup-cached `l`, and lane-strided `simd_max` in `qsa_merge2` and `qsa_merge`. keep the serial `L` and output FMA chains, all three merge barriers, and buffer layouts unchanged. `qsa_merge2` is the production decode path.

do **not** ship raw-row single-barrier `norm_rope` fusion or reduction unroll: bitwise-correct, but confirmation found no useful speedup. `qsa_score` already fully unrolls its four-value dot products; `qsa_attn2` already uses `char4` key loads and fully unrolls the eight-value and six-head inner loops. neither changed without evidence.

## artifacts and provenance

- patch: `/tmp/sol_r27_qsa/sol_r27_qsa.patch`
- isolated candidate: `/tmp/sol_r27_qsa/strata/`
- immutable baseline kernels: `/tmp/sol_r27_qsa/baseline/`
- rejected fusion source: `/tmp/sol_r27_qsa/qsa-with-prep-fusion.metal`
- exact source hashes: `/tmp/sol_r27_qsa/manifest.json`
- vendor head: `296b59d2f73c8f04a805df29c2fa97655adace78` (baseline kernel hashes, not head alone, identify measured working-tree inputs)
- all raw logs: `/tmp/sol_r27_qsa/*.log`

only the three owned files occur in the patch. no tracked vendor file edited, no server calls or full-model benchmarks. `git diff --check` and `git -C vendor/strata apply --check /tmp/sol_r27_qsa/sol_r27_qsa.patch` both exit 0.

## evaluation

apple m5; runtime library `vendor/strata/build-metal/libmtl_rt.a`; clang++ c++20 -O2. kernel runtime defaults retained. alternating A/B timing order, median nine paired rounds, batches of 32 existing bitwise dispatches or 16 new full-window dispatches. large prep uses one dispatch per batch. gpu clock/load clearly varied across invocations; use within-pair ratios, not absolute times across runs. concurrent live-server load was not controlled.

build:

```
clang++ -std=c++20 -O2 -Ivendor/strata/metal/src   /tmp/sol_r27_qsa/strata/metal/tests/qsa_parity.cpp   vendor/strata/build-metal/libmtl_rt.a -framework Metal -framework Foundation   -o /tmp/sol_r27_qsa/qsa_parity
```

set `STRATA_METAL_KERNELS=/tmp/sol_r27_qsa/strata/metal/kernels` for every invocation. existing `--bitwise /tmp/sol_r27_qsa/qsa.metal.baseline` tests v1; add `QSA_ATTN=2` and use `qsa2.metal.baseline` for v2. new `--paired /tmp/sol_r27_qsa/baseline P --tokens N` compares every buffer byte and times prep/score/topk/attn/merge separately. `--prep-only` avoids allocating enormous attention partial buffers for prefill; full-window tokens capped at 128, prep-only at 2048.

## correctness

- final v1: 40/40 random-KV `(position, ntok)` cases, full `part` and `xp` memcmp equal; positions 0,1,63,64,512,2047,2048,2050,8192,30000; tokens 1,2,4,8.
- final v2: identical 40/40 matrix, full `part` and `xp` memcmp equal.
- final full-window seeded-input tests: `(P,N,attn,mrope)` = `(0,1,1,0)`, `(3,3,1,0)`, `(2047,8,1,0)`, `(2050,8,2,0)`, `(3000,8,2,1)`. all 13 complete buffers pass: `qcur qidx kq vq ksc vsc tail dead pooled scores ids part xp`. identical nonzero seeded history for tail/dead/pooled, other outputs initialized identically. prior KV history is zero in this mode; existing bitwise mode supplies random KV codes/scales/selection instead.
- prep-only T=2048 full-buffer comparison passes; the untouched attention buffers are trivially equal in this mode.
- all final commands exit 0; final bitwise invocations 1.70s/1.63s, final full-window invocations 0.13–0.61s, final prefill 0.25s.

bitwise claims cover tested finite production-style inputs and these specialization modes, not all possible NaNs or all function constants. no summation reassociation: `L` remains serial `fma(l[k], ek[k], L)` in ascending k; output chain unchanged. `(m,l)` alignment follows even `QD=256`, head stride 258 and chunk stride 6192. maxima are parallelized, sums are not.

## measured merge wins

final existing random-KV evaluator, gpu microseconds baseline / candidate:

| kernel | P | tokens | baseline | candidate | latency reduction |
|---|---:|---:|---:|---:|---:|
| merge2 | 512 | 1 | 18.69 | 15.94 | 14.7% |
| merge2 | 512 | 4 | 9.45 | 8.12 | 14.1% |
| merge2 | 2048 | 1 | 12.28 | 9.87 | 19.6% |
| merge2 | 2048 | 4 | 50.52 | 46.22 | 8.5% |
| merge2 | 8192 | 1 | 12.25 | 9.82 | 19.8% |
| merge2 | 8192 | 4 | 50.53 | 46.58 | 7.8% |
| merge | 512 | 1 | 13.45 | 11.66 | 13.3% |
| merge | 512 | 4 | 13.90 | 11.70 | 15.8% |
| merge | 2048 | 1 | 7.73 | 5.89 | 23.8% |
| merge | 2048 | 4 | 24.88 | 23.41 | 5.9% |
| merge | 8192 | 1 | 7.57 | 5.87 | 22.5% |
| merge | 8192 | 4 | 24.91 | 23.34 | 6.3% |

initial independent random-KV runs showed the same direction. three additional merge2 full-window confirmations at P=2050,N=8 gave 101.081/90.672, 101.065/90.281, 100.320/90.599 us (~10%). small-context v1 tradeoff: P=3,N=3 final 10.703/10.948 us (~2.3% slower), P=0,N=1 7.023/7.125 (~1.5% slower). these are below the predeclared 3% effect threshold, not a universal speedup claim. score and attention are unchanged; apparent differences there are load/clock noise.

## rejected prep fusion

initial full-window T=2048 timing looked ~33% faster, but this did not survive isolated confirmation. three prep-only confirmations plain: 2919.667/2918.042, 3041.292/3052.083, 3045.125/3048.708 us. mrope: 1591.625/1597.875, 3055.542/3099.292, 2983.917/2993.292. neutral or slightly worse.

parent-requested decode confirmation at P=3000, two independent runs per token count:

| tokens | run 1 baseline/candidate us | run 2 baseline/candidate us |
|---:|---:|---:|
| 1 | 6.880/6.893 | 6.875/6.922 |
| 2 | 8.448/8.508 | 8.477/8.523 |
| 4 | 12.190/12.143 | 12.164/12.156 |
| 8 | 20.018/19.872 | 20.018/19.901 |

no >3% benefit; baseline helper restored. the compiler may already eliminate some barrier/reduction overhead, or added raw-row/gamma loads may offset it; no assembly claim made.

## budget incident / limits

one early full-window T=2048 invocation took 6.11s, exceeding the requested 5s. the old partial allocation scales to multiple gigabytes at that size. parent was notified immediately; added prep-only mode and a full-window token cap. all subsequent invocations were below 5s. no full-model or :8080 use. no full-model end-to-end throughput claim.