round 24 track b: qsa prompt offsets and unrolling #

decision #

ship flash-only 32-bit ko[8] / vo[8] offsets, fixed eight/four-trip loop unrolls, and qbase hoisting. reject the attention-mm changes (repeatable long-T regressions), unroll-only changes (noise), and score-mm changes (no robust >2% gain).

patch: /tmp/sol_r24_qsa/sol_r24_qsa.patch; git -C vendor/strata apply --check /tmp/sol_r24_qsa/sol_r24_qsa.patch passed. only metal/kernels/qsa_prompt.metal, inside qsa_flash, changes. no tracked vendor files were edited. no test changes were necessary: the existing test already provides paired A/B and full-buffer memcmp.

tradeoff: this is not a universal speedup. T=1984 first-chunk flash is ~2.35% slower, pos2048 ~0.9% slower, while pos8192 is ~11% faster and pos30000 ~1.2% faster. the equal-case summed per-dispatch median is 3.30% / 3.09% faster in two confirmation passes. this supports a mixed-position net win, not an end-to-end model claim; reject if first-chunk-only latency is the objective.

contract / provenance #

• machine: Apple M5 (cooperative k4 path), local existing vendor/strata/build-metal/qsa_prompt_parity executable.
• baseline git HEAD: 296b59d2f73c8f04a805df29c2fa97655adace78; actual working-tree kernel frozen as /tmp/sol_r24_qsa/baseline.metal, sha256 a36eb1e45221a10a3f41a5528218e561c319067b9787572e5262695031af117f.
• source snapshots and immutable per-run logs: /tmp/sol_r24_qsa/{unroll,offset,offset_only,score,flash}.metal, /tmp/sol_r24_qsa/records.json, and named .log files. scratch checkout /tmp/sol_r24_qsa/strata.
• plan written before runs: /tmp/sol_r24_qsa/plan.md. acceptance: all memcmp passes, reproducible >2% summed-median gain and no repeatable >3% case regression. low-T/pos0 timing is inconclusive, not counted as a reliable improvement.
• 113 invocations including baseline self-check; every exit status 0; maximum wall duration 2.094 seconds (below 5-second cap). GPU tests serialized in this session. live server left alone; no full-model runs, no :8080 access. other system/GPU activity was not controlled; substantial clock-state noise appeared in short kernels.
• environment: STRATA_METAL_KERNELS=/tmp/sol_r24_qsa/strata/metal/kernels, QSA_TOPK=1. compare modes supply deterministic selected ids directly; setting QSA_TOPK does not make them exercise runtime top-k selection.

final flash confirmation #

paired alternating order, seven pairs; each timing is 12 consecutive dispatches for T=96 and three for T=1984. table reports milliseconds per dispatch, normalized from those command-buffer medians. two separate passes:

T96/pos0 swings with clocks; one pass improves 3.5%, another regresses 9.3% on only ~5 microseconds per dispatch. that is not a repeatable case regression. the large-T results are stable across screening and both confirmations. no shape-specific branches were added to fit these measurements.

reproduce:

STRATA_METAL_KERNELS=/tmp/sol_r24_qsa/strata/metal/kernels QSA_TOPK=1 \
 vendor/strata/build-metal/qsa_prompt_parity --flash-compare 1984 8192 \
 /tmp/sol_r24_qsa/baseline.metal 3 7

use layers 12 at T96; repeat positions 0/2048/8192/30000. long_flash_* and repeat_flash_* preserve every pair, not just selected medians.

ablations #

• unroll + qbase, without offsets: all 16 attention/flash memcmp checks passed; T1984 flash differences -0.2% to +0.5%, attention-mm 0% to +1.1%. not a useful independent win.
• offsets + unroll: all 16 checks passed. T1984 flash pos8192 24.790 -> 22.173 ms; attention-mm T1984 regressed ~0.7–3.0%, so attention edits were removed.
• offsets only: all 16 checks passed; T1984 flash pos8192 24.795 -> 22.206 ms; pos0 regressed 2.9%. offsets explain nearly all the flash gain; no claim is made that unroll changes register allocation. no ISA/register-count measurements were taken.
• score unroll + pointer hoist: all eight cases passed twice. confirmation T1984 positions 0/2048/8192/30000: 0.188375 -> 0.187250, 0.474000 -> 0.472500, 1.356208 -> 1.339667, 4.479375 -> 4.406250 ms. gains 0.3–1.6%, below the threshold and noisy in screening; rejected.

correctness and scope #

all 112 recorded variant invocations plus the baseline check passed memcmp == 0. final candidate tested flash output at both T values and all four positions in both confirmations; final unchanged attention partial buffer and score buffer also checked at the full eight-case matrix. signed-zero and tiny-cancellation flash edge modes each passed at T96 and all four positions. full-buffer comparison includes poisoned unwritten attention/score storage, not just numerically active cells. tg memory unchanged (flash 7616 bytes).

only address representation / fixed-trip expansion changes; floating-point expression and reduction order preserved. existing uint(cid[c]) offsets originate in the same 32-bit cache indexing already used by the kernel; adding dimensions before pointer formation remains valid for its supported context/cache extents. M4/non-k4 fallback code unchanged, but not run on M4 hardware. no full model or end-to-end timing was performed.

final review: inspected the entire patch with view_diff and surrounding softmax/load code with view_code; no shared helper warranted for local address expressions. patch apply check passed.