opus_r10_qsa — Round 10 Track A: qsa_flash operand-fill hoisting + vec4 epilogue #

patch: /tmp/opus_r10_qsa/opus_r10_qsa.patch (git -C vendor/strata apply --check clean against the current dirty tree). touches only metal/kernels/qsa_prompt.metal (qsa_flash only, coop_k4 paths only; the !coop_k4 / M4 code is byte-for-byte the old code). qsa_prompt_parity.cpp unchanged (its --flash-compare from r9 was enough). scratch: /tmp/opus_r10_qsa/strata, build /tmp/opus_r10_qsa/build, baseline (= current r9 kernel) /tmp/opus_r10_qsa/baseline_qsa_prompt.metal, every variant tried in /tmp/opus_r10_qsa/v_*/.

result (Apple M5, paired alternating --flash-compare, all memcmp PASS) #

(two repeats for pos0 0, "a / b".) so pg:qsa_flash x12 at T1984 should go ~272 -> ~126 ms in kprof (not run: no full-model benches while :8080 is live).

bitwise: memcmp PASS for T = 96, 512, 1024, 1984 x pos0 = 0, 30000, plus (96,1984), (96,2050), (1,0), (64,1987), (200,1900), (1,2050), (1,2051), (3,2049), (33,30000). COOP_K4=0 (the untouched generic path) PASS at (512,0), (1984,30000). full default qsa_prompt_parity (rel-L2 vs qsa_attn+merge, fused qsa_prep memcmp): ALL PASS. every process < 4 s.

what changed #

1. V operand (rv) fill, coop_k4: per K=32 run, a lane's 64 elements are 8 cells (runs of 4 cells, second half +16) x 8 dims (coop_layout 16_64_32 dump). the 8 cells' row pointers and scales are taken once (8 cid + 8 vs_tg reads instead of 64
   • 64), and the 64 byte loads are unconditional: an empty cell (id < 0) points at row 0 and gets scale 0.
2. K operand (rk) fill, coop_k4: same trick. the lane's 8 cells come from get_multidimensional_index(4j)[1] and the scale column is sg (k >> 6 == sg inside the simdgroup's 64 dims), so pointers + scales are hoisted out of the k0 loop (once per chunk), char4 loads unconditional.
3. epilogue, coop_k4 (target 1): step 4, lsum[h] once per run, float4 gate load, half4 store. ~-2.5% on its own.

why ±0 is still bitwise: an empty cell now feeds q * 0.0f = ±0 instead of +0. the accumulators start at +0 and IEEE RN gives x + (-0) = x for every x incl. +0, so S and P·V are unchanged; additionally P = 0 exactly on cells c >= nc. the scale is selected to 0 before the multiply, so stale vs_tg/ks_tg entries (only written for id >= 0) can never inject NaN/Inf.

ablation (T1984 pos0 0, x12, ms after vs ~272 before) #

• rv: unconditional loads only (no hoist): 253. hoist + select (ok ? load*s : 0): 249. vec4 cid/int4 + float4 math: 299 (worse). hoist and unconditional: 157. both are needed: with any per-element predicate the compiler keeps 64 predicated, non-immediate-offset loads; with neither it can issue 64 base+imm byte loads off 8 pointers.
• • rk same treatment: 146 (unhoisted, uncond) -> 125 (hoisted out of k0).
• • epilogue vec4: 128.5 -> 125.4.
• target 2a (fac once per 4 in the tot rescale): 0 effect (125.3 vs 125.4) -> dropped.
• rv hoisted over both c0 runs (16 pointers): 130, worse than per-run (register pressure) -> dropped.
• target 2c (rescale tot in place and opO.run(tPh, rv, tot), no acc): memcmp FAIL on M5 (different summation order than fma(tot, f, acc)) -> not kept, as required.
• tot.store epilogue into dead sp: not built; the vec4 epilogue is already a 3% item now, a two-round store + barriers could at best shave part of that.

caveats / follow-ups #

• the rv hoist assumes every first-half run of a lane has the same 4 cells (true in the M5 dump; coop_probe only checks runs of 4 and the +16 second half). a layout change would show up as a rel-L2 FAIL in the default qsa_prompt_parity run.
• the M4 (!coop_k4) paths still use branchy per-element loads; the same unconditional+hoist trick will likely pay there too but needs an M4 to measure.
• the same "branchy dequant load per element" pattern may exist in qsa_attn_mm / mm_gu3 / mm_down3 right-input fills — worth a look by whoever owns those.