7.8 kB 73 lines
2eb7dc0e
Round 15 blitz: prompt_chunk router+plan/shexp wave zipping (-190 barriers), fused Wal+Wbe N=96 GEMM (-36 dispatches), QSA PSO pre-warm (-202 ms Chunk-1 stall), qsa_score_mm causal-future skip (-19.4%), qsa_prep 320B TG + 2 barriers
dawn
⋯
audits/sol_r15_qsa.md
sourcerendered
download raw history
at main Markdown

round 15 track b: qsa score and top-k #

decision #

keep the causal-future tile exit, score epilogue/dead-load hoists, top-k cached-load bound, and redundant radix barrier removal. keep the original 64×64 score tile: the two-pass 32×64 experiment was byte-identical but slower.

  • clean patch: /tmp/sol_r15_qsa/sol_r15_qsa.patch
  • scratch source: /tmp/sol_r15_qsa/strata
  • evidence, baseline snapshots, rejected slice32 source, logs, and hashes: /tmp/sol_r15_qsa/
  • git -C vendor/strata apply --check /tmp/sol_r15_qsa/sol_r15_qsa.patch: exit 0.
  • only the three assigned files change, and kernel edits stay within the assigned definitions. tracked vendor files were not edited. no server requests, port 8080 interaction, or full-model benchmarks.

contract and correctness #

owner: score/top-k kernels and standalone parity harness. buffer layout, causal tail ownership, reduction arithmetic, tie order, and all output bytes must stay unchanged. fixed rng seed 5; alternate baseline/candidate order. use three-pair exploration and two five-pair confirmation rounds. reject the occupancy variant unless exact parity and repeatable >2% aggregate benefit without substantial regressions. every test process had a 5-second deadline; observed tests finished below one second.

baseline was the dirty working-tree snapshot at task start, not upstream head. repository head was 296b59d2f73c8f04a805df29c2fa97655adace78; exact per-file baseline/candidate hashes are in manifest.json. the harness links the existing vendor/strata/build-metal/libmtl_rt.a; kernels are selected exclusively through STRATA_METAL_KERNELS pointing into scratch.

why the changes preserve behavior:

  • early exit is uniform for the entire threadgroup and occurs before matmul/barrier. if b0 >= max_bid, every possible full block in that tile fails the original epilogue predicate. column zero is excluded because it independently writes every token's tail score, including cases with zero full blocks.
  • j and b are invariant per thread; tt = tid >> 6; tt += 2 visits exactly the original indices. once t >= n_tok, later rows are also outside the prompt. dead[0..127] is invariant across the four tokens assigned to a simdgroup.
  • cached keys are initialized for exactly the block indices later read. the same predicate b < nb bounds every consumer; uncached contexts do not read keys.
  • after reading sel, the next pass cannot overwrite sel until both the histogram-clear barrier and histogram-complete barrier. the final pass no longer writes sel at all. the removed end-of-pass barrier protects no live cross-threadgroup state.

checks, all exit 0:

  • all nine requested (T, pos0) combinations: scores and qids full-buffer memcmp == 0, including poisoned unused/padding entries, both before and after timing; both confirmation rounds passed.
  • boundaries (T,pos0): (1,0), (2,0), (3,0), (17,0), (31,1), (17,2048), (31,4092), (17,16368), (17,16384), (31,30000) all byte-identical. these cover tiny tails, partial 16-token tiles, the 1024/4096-block cache thresholds, and both implicit/all-cell and selected paths.
  • existing --topk-compare with QSA_TOPK_EDGE=1, T=96, pos0=2048/4092/16368/16384/30000: all byte-identical, including ties and signed zeros. this is correctness evidence, not a claim of universal edge-workload speedup.
  • standalone qsa_prompt_parity: ALL PASS; repeated after final-source rebuild. existing scalar-vs-matmul score check remains tolerance-based, while the new baseline-vs-candidate comparison is exact.
  • standalone qsa_parity with QSA_TOPK=1: ALL PASS; zero of 6153 ids differ against its cpu reference at default context 3000.

measured performance #

apple m5, safe math. gpu command-buffer times; five alternating pairs per stage, one extra stage warmup, median reported. score and top-k measured separately and as a serial dependency chain with explicit buffer barriers. no model throughput claim. absolute times vary substantially between processes (especially small-T cold clocks); use within-process paired comparisons, not times from different rounds. the live server was left untouched, so these are short local measurements rather than an exclusive-machine throughput result.

confirmation round 2, milliseconds, baseline → candidate:

T pos0 score top-k combined combined reduction
512 2048 0.112625 → 0.100750 0.121417 → 0.108750 0.230625 → 0.213000 7.64%
512 8192 0.328458 → 0.316000 0.241750 → 0.236042 0.606208 → 0.586792 3.20%
512 30000 1.158875 → 1.137333 0.518375 → 0.511000 1.645667 → 1.633125 0.76%
1024 2048 0.255500 → 0.210417 0.237708 → 0.217333 0.484625 → 0.428167 11.65%
1024 8192 0.693750 → 0.660542 0.437583 → 0.431625 1.132292 → 1.081792 4.46%
1024 30000 2.280583 → 2.229125 1.018208 → 1.005417 3.318250 → 3.238167 2.41%
1984 2048 0.587875 → 0.473583 0.472375 → 0.439625 1.054500 → 0.906125 14.07%
1984 8192 1.450042 → 1.317000 0.861500 → 0.846000 2.319000 → 2.172250 6.33%
1984 30000 4.529917 → 4.374208 1.975167 → 1.950250 6.511333 → 6.351500 2.45%

across both confirmation rounds (18 cases), score median reduction was 1.69–20.64%, top-k 1.02–10.43%, combined 0.76–14.67%. geometric mean of paired case-median ratios gives 8.23%, 3.66%, and 6.25% reductions respectively; these are descriptive aggregates, not confidence intervals. both rounds improve every tested ordinary case, but sub-percent high-context combined gains should be treated as small/noisy. see confirm{1,2}_*.log for every sample. final-source smoke checks repeat the default parity and both T=1984 endpoint contexts successfully.

rejected occupancy experiment #

source snapshot: /tmp/sol_r15_qsa/slice32/; evidence: slice32_*.log. the variant executes two 32-row matmuls with an 8192-byte shared output tile and a barrier before reuse. all nine cases preserve exact scores and ids. five-pair score comparisons against the optimized 16384-byte 64-row candidate lose 1.1–8.5% across all nine cases, so it is not in the patch. preliminary occupancy combined-stage timings used an earlier harness without explicit dispatch barriers; those combined timings were discarded. its isolated score timings are unaffected.

reproduction #

clang++ -std=c++20 -O2 -I/tmp/sol_r15_qsa/strata/metal/src \
  /tmp/sol_r15_qsa/strata/metal/tests/qsa_prompt_parity.cpp \
  vendor/strata/build-metal/libmtl_rt.a -framework Metal -framework Foundation \
  -o /tmp/sol_r15_qsa/qsa_prompt_parity

STRATA_METAL_KERNELS=/tmp/sol_r15_qsa/strata/metal/kernels \
  /tmp/sol_r15_qsa/qsa_prompt_parity --score-compare 1984 2048 \
  /tmp/sol_r15_qsa/baseline/qsa_prompt.metal /tmp/sol_r15_qsa/baseline/qsa.metal 5

repeat the latter invocation individually for T=512/1024/1984 and pos0=2048/8192/30000, with a 5-second process deadline. run no-argument parity with the same environment; run vendor/strata/build-metal/qsa_parity with that environment and QSA_TOPK=1.

setup repairs: the first baseline-only harness invocation exited 2 because the snapshot lacked mm_bench.metal, which the existing cooperative-matmul probe needs; copied the remaining scratch kernel helpers and reran successfully (baseline_repaired.log). the first 32-row compile exited 3 because half is a reserved metal type; renamed the loop variable to tile and reran. neither failed run supplied performance evidence. the final harness adds explicit chain barriers, stage warmups, and a post-timing full-buffer parity check; all confirmation results use that evaluator.