# round 26 track b: qsa prompt mm

## contract (before measurement)

owner: `metal/kernels/qsa_prompt.metal`; only attention's cooperative operand address preparation, softmax loop hints/output copy, and score loop hints change. no dispatch/layout/interface changes; fallback cooperative layout must remain valid. byte offsets are uint32, matching the existing cache addressing bound.

baseline: working-tree snapshot of the owned kernel at vendor commit `296b59d2f73c8f04a805df29c2fa97655adace78`, saved at `/tmp/sol_r26_qsa/baseline_qsa_prompt.metal` (the working tree already contains prior-round changes).

question: do 32-bit operand offsets, fixed-loop unroll hints and vector threadgroup output loads reduce m5 qsa mm latency without changing any output byte?

evaluator: existing `qsa_prompt_parity --mm-compare`, deterministic rng seed 5, alternating paired dispatch order, full output buffer memcmp (including poisoned unwritten regions). primary matrix: t=96,1984 and pos0=0,2048,8192,30000. screen using 5 pairs, confirm using 9 pairs; reject bitwise changes or repeatable regressions. require roughly >=2% repeatable long-prompt improvement to ship a kernel change; otherwise preserve a negative/inconclusive result. ablate vector load and score hints if combined result loses. no full-model runs; no server requests; each test timeout 20s for cold compile, target <10s elapsed.

artifacts and immutable variant snapshots live in `/tmp/sol_r26_qsa/`; gpu tests run sequentially to avoid self-contention. budget: at most 60 short invocations plus 2 compile repairs.

## decision

ship attention offsets + fixed-loop hints + vector threadgroup loads. do **not** ship score unroll hints: bitwise pass, but repeated paired results did not establish a reliable gain. `qsa_score_mm` is unchanged in the patch. `qsa_flash` and all other kernels are unchanged.

- patch: `/tmp/sol_r26_qsa/sol_r26_qsa.patch` (two owned files only).
- `git -C vendor/strata apply --check /tmp/sol_r26_qsa/sol_r26_qsa.patch`: exit 0.
- no tracked vendor files edited; no requests to :8080; no full-model runs.
- existing `--mm-compare` reused; harness now checks post-timing partials and merged/prepared activation buffers bitwise, in addition to the pre-timing whole partial-buffer memcmp.

## measured confirmation

apple m5; macos 27.0 build 26a428; cooperative layout probe true; safe math (no reassociation). timings below are medians of 9 paired alternating-order single dispatches, milliseconds. full partials, post-timing partials, and all merged/prepared output bytes passed `memcmp == 0` for every row. tg memory unchanged at 7424 bytes.

| t | pos0 | baseline ms | candidate ms | reduction |
|---:|---:|---:|---:|---:|
| 96 | 0 | 0.362500 | 0.291625 | 19.55% |
| 96 | 2048 | 6.023500 | 4.092000 | 32.07% |
| 96 | 8192 | 5.318875 | 3.293667 | 38.08% |
| 96 | 30000 | 1.918875 | 1.493208 | 22.18% |
| 1984 | 0 | 14.569125 | 11.676958 | 19.85% |
| 1984 | 2048 | 30.884125 | 23.683833 | 23.31% |
| 1984 | 8192 | 31.790833 | 22.275375 | 29.93% |
| 1984 | 30000 | 37.322500 | 28.089667 | 24.74% |

`final_attn_*.log` contains all samples. a final review rerun at t1984,pos8192 gave **31.815083 → 22.381875 ms (-29.65%)**, all three byte checks passed (`final_review_attn_1984_8192.log`). at 12 qsa layers this isolated-kernel delta is roughly 113 ms/chunk; this is not an end-to-end server claim.

short-case absolute clocks varied considerably across processes (the live server was left alone). long-case paired gains were stable across screening, ablation and confirmation. neither clocks nor competing external activity were controlled; use the paired within-process deltas rather than comparing unrelated absolute samples.

## ablations / negatives

- offsets + all attention loop hints but scalar copy: long t1984,pos2048/8192/30000 medians 30.879→24.543 / 33.366→25.781 / 36.744→28.541 ms. (`ablate_offsets_softmax_*.log`)
- vector load incremental against that variant, measured directly paired: 24.394500→23.880250 ms at pos2048 (-2.11%), 23.688458→22.506375 at pos8192 (-4.99%), 28.195500→27.637125 at pos30000 (-1.98%). t96,pos8192 2.514417→2.353750 (-6.39%). all bitwise checks passed. (`vector_increment_*.log`)
- score hints: screen t1984,pos30000 initially looked faster (10.396→9.670 ms), but 15-pair confirmation gave 5.177625→5.169417 (-0.16%). pos2048 0.481500→0.478792 (-0.56%), pos8192 1.450417→1.449083 (-0.09%). t96 pos8192 regressed 0.077625→0.081000 (+4.35%). no robust benefit: rejected. (`confirm_score_*.log`; candidate source `combined.metal` / `score.metal` retained)
- the literal proposed `#pragma clang loop unroll(4)` failed compilation: metal clang accepts `unroll_count(4)`, used here. one compile repair; failed attempt preserved in `screen_combined_attn_1984_8192.log`.

## parity coverage / limits

- primary 2×4 t/pos matrix above: exact pre/post partials and merged/prepared equality, including causal tail chunks, crossing the 2051-cell budget at pos2048, invalid implicit ids poisoned to -1, and untouched buffer regions poisoned 0xa5.
- force generic cooperative decode (`COOP_K4=0`) at t96,pos0/8192/30000: all partial and merge/prepared byte checks pass on m5. this is not physical m4 validation.
- final score+topk pipeline checks at t96,pos2048 and t1984,pos8192/30000: scores, ids and post-timing equality pass. stage code is unchanged in final candidate.
- all successful benchmark invocations completed under 5s, within requested <10s limit; timeout 20s reserved for cold compilation. gpu runs were sequential; no helper agents spawned.
- rng seed 5; no exhaustive adversarial float corpus or physical non-m5 testing. safe-math compilation and unaltered arithmetic sequencing preserved the tested bit patterns.

## reproduce / preserved sources

scratch tree: `/tmp/sol_r26_qsa/strata`; immutable source snapshots: `baseline_qsa_prompt.metal`, `baseline_qsa.metal`, `offsets_softmax.metal`, `vector.metal`, `score.metal`, `combined.metal` in the experiment directory. final kernel in scratch differs from `vector.metal` only by the alignment comment. baseline kernel sha256: `c768c25c3186e0715a77ad2d9a58277871b4aab50d51fca0d4dd3fa42962ac6a`.

build command (existing runtime library is unmodified):

```text
clang++ -std=c++20 -O2 -I/tmp/sol_r26_qsa/strata/metal/src -I<workspace>/vendor/strata/third_party/ggml /tmp/sol_r26_qsa/strata/metal/tests/qsa_prompt_parity.cpp <workspace>/vendor/strata/build-metal/libmtl_rt.a -framework Metal -framework Foundation -o /tmp/sol_r26_qsa/qsa_prompt_parity
```

run each matrix row sequentially:

```text
STRATA_METAL_KERNELS=/tmp/sol_r26_qsa/strata/metal/kernels /tmp/sol_r26_qsa/qsa_prompt_parity --mm-compare attn 1984 8192 /tmp/sol_r26_qsa/baseline_qsa_prompt.metal 9
```

replace attn with score to isolate score; `--score-compare t pos baseline_prompt baseline_qsa pairs` checks score/topk pipeline. final source sha256 values:
- `/tmp/sol_r26_qsa/strata/metal/kernels/qsa_prompt.metal`: `4602c462fa5096f3ee9e65b03460c5b441db3f4666c573fe0b30a5ffbf0f2971`
- `/tmp/sol_r26_qsa/strata/metal/tests/qsa_prompt_parity.cpp`: `5bc787fecab37b0d7f8bb6695cf4957a89694650084ed509ceb81678f6f37d23`
- `/tmp/sol_r26_qsa/sol_r26_qsa.patch`: `c7e42096f883395eb76aa8022711f1a48514d077d17a34c78f9c38b212ea726a`