# round 14 track b: qsa metadata hoist

## decision

- candidate patch: `/tmp/sol_r14_qsa/sol_r14_qsa.patch`.
- changes only `metal/kernels/qsa_prompt.metal`; no tracked vendor files edited.
- keep `qsa_flash`'s existing `(2,T)` grid. transposition did not establish a useful improvement.
- hoist byte offsets and invalid-cell zero scales once per cell into threadgroup metadata. this removes repeated clamp/row arithmetic and validity selects from both cooperative key/value fills, without changing their mathematical operations.
- parent-requested extension: port the same hoist to `qsa_attn_mm`, cache key scales alongside value scales, and stop its pv loop at `nc` rather than `QCH`.
- correctness passes. flash has a promising 7–9% improvement in the initial paired measurements. **final quiet-gpu timing confirmation is still required**, especially for the later mm extension. parent explicitly owns that confirmation after all three tracks finish. concurrent sibling gpu work contaminated the broad timing sweep; do not treat those regressions or wins as confirmed performance effects.

## source and reproduction

- vendor commit: `296b59d2f73c8f04a805df29c2fa97655adace78` plus the current tree snapshot; baseline kernel sha256 `19f364f8de572ade4f56bafb188a1a87bc28febde23fbb99d34ad3e0bade8c6b`.
- final kernel sha256 `f24eabc2fbcf046a1b3e2d9df010e4a52b8a24819e9211652c49e1b8b7ffba83`.
- unchanged evaluator sha256 `e65e20c5467a78b64a07715d6d806c2ddfde6ee1e0c24df8c9a64447bec650a2`.
- exact baseline: `/tmp/sol_r14_qsa/baseline_qsa_prompt.metal`.
- isolated source/build: `/tmp/sol_r14_qsa/strata`, `/tmp/sol_r14_qsa/build`.
- machine: apple m5, existing live model server left alone; no requests to :8080 and no full-model benchmark.
- configure: `cmake -S /tmp/sol_r14_qsa/strata/metal -B /tmp/sol_r14_qsa/build -G Ninja -DCMAKE_BUILD_TYPE=Release`.
- build: `cmake --build /tmp/sol_r14_qsa/build --target qsa_prompt_parity coop_layout -j 2`.
- comparison: `/tmp/sol_r14_qsa/build/qsa_prompt_parity --flash-compare T pos0 /tmp/sol_r14_qsa/baseline_qsa_prompt.metal 1 7`.
- mm comparison: same executable, `--mm-compare attn T pos0 /tmp/sol_r14_qsa/baseline_qsa_prompt.metal 3`. the current evaluator has no `--compare` flag.
- samples alternate baseline/candidate order; deterministic rng seed 5; one dispatch per sample. every standalone gpu invocation had a 4.8-second deadline and completed below it.
- build repair: initial scratch copy omitted `third_party/ggml/ggml-common.h`; copied that existing directory, then build passed. no source changes were needed.

## correctness

all comparisons returned exit 0 with `memcmp PASS`:

- final flash: `T={96,512,1024,1984}` × `pos0={0,2048,8192,30000}` (16 cases), after the mm extension.
- flash signed-zero/tiny-cancellation: `QSA_FLASH_EDGE=1` and `=2`, each at `(T,pos0)=(1,0),(2,0),(31,0),(32,0),(33,0),(64,0),(96,1984),(96,2050),(1984,30000)` (18 cases). the flash body was unchanged by the later mm extension.
- flash ordinary boundary/fallback: nine random boundary cases, plus `COOP_K4=0` at `(96,0),(96,2048),(1984,30000)` (12 cases).
- final mm: the same 16-case grid, nine random boundary cases `(1,0),(2,0),(31,0),(32,0),(33,0),(64,0),(65,0),(96,1984),(96,2050)`, and those three `COOP_K4=0` cases.
- full `qsa_prompt_parity`: exit 0, `ALL PASS`; attention/score numerical references pass, and fused-prep qcurh/flash byte parity passes.
- invalid ids remain ids in the generic `!coop_k4` path. the optimized path stores row-0 offsets with positive-zero scales for invalid cells, matching the old unconditional-load operands. valid signed-zero scales are loaded unchanged.
- `git diff --check` and `git -C vendor/strata apply --check /tmp/sol_r14_qsa/sol_r14_qsa.patch`: pass.

raw evidence: `final_flash_*.log`, `edge*.log`, `extra.json`, `mm_*.log`, `mm.json`, `full_parity_final.log` under `/tmp/sol_r14_qsa/`.

## flash timing lead

initial seven-pair measurements, before the broad sweep's contention:

| t | pos0 | baseline ms | hoist ms | change |
|---:|---:|---:|---:|---:|
| 96 | 0 | 0.267 | 0.235 | -12.0% |
| 1984 | 0 | 10.330 | 9.433 | -8.7% |
| 1984 | 2048 | 21.837 | 19.821 | -9.2% |
| 1984 | 8192 | 26.913 | 24.938 | -7.3% |
| 1984 | 30000 | 32.759 | 30.391 | -7.2% |

logs: `offset_*.log`. static flash threadgroup memory remains 7616 bytes. mm cooperative threadgroup memory increases 6912 → 7424 bytes; the forced generic path remains 6912 bytes. mm's exploratory timings were often faster but included severe contention outliers (e.g. 45.474 → 150.673 ms at t1984/pos2048), so no mm speed claim is justified yet.

### broad hoist sweep (contaminated; correctness evidence only)

| t | pos0 | baseline ms | hoist ms | change |
|---:|---:|---:|---:|---:|
| 96 | 0 | 0.115 | 0.105 | -8.7% |
| 96 | 2048 | 3.444 | 3.002 | -12.8% |
| 96 | 8192 | 5.044 | 5.594 | +10.9% |
| 96 | 30000 | 6.020 | 6.047 | +0.4% |
| 512 | 0 | 5.774 | 5.675 | -1.7% |
| 512 | 2048 | 15.258 | 14.675 | -3.8% |
| 512 | 8192 | 17.334 | 15.724 | -9.3% |
| 512 | 30000 | 22.947 | 22.178 | -3.4% |
| 1024 | 0 | 3.333 | 3.022 | -9.3% |
| 1024 | 2048 | 11.335 | 10.451 | -7.8% |
| 1024 | 8192 | 14.041 | 13.690 | -2.5% |
| 1024 | 30000 | 17.680 | 16.620 | -6.0% |
| 1984 | 0 | 10.498 | 9.708 | -7.5% |
| 1984 | 2048 | 22.096 | 20.140 | -8.9% |
| 1984 | 8192 | 27.163 | 25.111 | -7.6% |
| 1984 | 30000 | 34.254 | 34.756 | +1.5% |

logs: `confirm_*.log`, `confirmation.json`. a sibling `moe_mm_parity --gu-bench` process was observed alongside the live server. do not claim an isolated no-regression result from this sweep.

## rejected variants and layout findings

### grid transposition

explicit `(T,2)` candidate mapping plus all owned callers updated; all 16 comparisons bitwise pass. no production mapping/caller change retained. three paired samples/case:

| t | pos0 | `(2,t)` ms | `(t,2)` ms | change |
|---:|---:|---:|---:|---:|
| 96 | 0 | 0.270 | 0.273 | +1.1% |
| 96 | 2048 | 2.545 | 2.553 | +0.3% |
| 96 | 8192 | 2.678 | 2.653 | -0.9% |
| 96 | 30000 | 3.709 | 3.084 | -16.9% |
| 512 | 0 | 2.349 | 2.375 | +1.1% |
| 512 | 2048 | 5.590 | 5.589 | -0.0% |
| 512 | 8192 | 6.816 | 6.829 | +0.2% |
| 512 | 30000 | 8.396 | 8.443 | +0.6% |
| 1024 | 0 | 2.902 | 2.903 | +0.0% |
| 1024 | 2048 | 11.149 | 11.135 | -0.1% |
| 1024 | 8192 | 13.862 | 13.766 | -0.7% |
| 1024 | 30000 | 16.829 | 16.806 | -0.1% |
| 1984 | 0 | 10.381 | 10.343 | -0.4% |
| 1984 | 2048 | 21.864 | 21.872 | +0.0% |
| 1984 | 8192 | 26.925 | 26.995 | +0.3% |
| 1984 | 30000 | 33.219 | 33.720 | +1.5% |

logs: `grid_*.log`, `grid_records.json`; candidate snapshots `grid_qsa_prompt.metal`, `grid_qsa_prompt_parity.cpp`. the t96/pos30000 lead was not consistent at larger t. the grid order is not a reliable model of physical gpu scheduling; no demonstrated cache-locality win here.

### packed rv + shuffle

`coop_layout 16_64_32` confirms lane 0's rv dimensions are `0,8,16,24,32,40,48,56`, repeated for its second four cells. neighboring dimensions belong to different lanes, not adjacent elements of one lane's row. simple per-lane char2/char4 conversion is therefore not valid. measured an explicit cross-lane char4-load/shuffle redistribution: bitwise pass on four exploratory cases, but t1984 timings regressed 10.363 → 16.134 ms at pos0=0, 21.870 → 33.475 at2048, 32.720 → 39.544 at30000. rejected immediately; no further confirmation needed. snapshot/logs: `packed_qsa_prompt.metal`, `packed_*.log`, `layout.log`.

### opS tail

for rk, `j=0..3` address cells below 32 and `j=4..7` address cells 32..63 (lane-dependent base within each eight-cell region). **both k0 iterations are dimension ranges, not cell halves**: skipping the second k0 iteration loses valid dot-product dimensions even when `nc<=32`. a narrower experiment zeroing the invalid rk half instead of loading it was bitwise clean in five cases but regressed, e.g. t1984/pos0 10.327 → 11.337 ms. rejected. snapshot/logs: `tail_qsa_prompt.metal`, `tail_*.log`.

## next gate

parent should run the final patch against the saved baseline with all other experimental gpu jobs paused, checking all flash timing cases and mm timings before accepting either hunk. if mm regresses, drop only the `qsa_attn_mm` hunks; flash is independent. do not apply any rejected grid/packed/tail snapshot.