7.7 kB 71 lines
70aef903
Round 11 blitz: gemm_f16 tile selector (-18% / -234 ms), qsa_attn_mm hoisted rv/rk (-44.5%), branchless W8<T_IQ3_S> (-50.8% / 2.03x) + W8<T_IQ2_XS> k_hoist (-57 ms), 3,397.5 ms GPU kprof (583.7 tok/s GPU) & 2048 prompt chunk on :8080
dawn
⋯
audits/sol_r11_qsa.md
sourcerendered
download raw history
at main Markdown

round 11 track b: qsa attention and score #

decision #

keep the qsa_attn_mm cooperative k/v hoist only. full partial-buffer memcmp passes all 16 requested cases. two nine-pair confirmations at T=1984, pos0=30000 give 70.488 → 39.162 ms and 70.649 → 39.163 ms, about 44.5% less kernel time / 1.80x throughput on apple m5.

reject the score changes: 8192-byte threadgroup storage and bitwise equality are demonstrated, but a repeatable speedup is not. the retained patch leaves qsa_score_mm, qsa_flash, dispatch interfaces, and the non-coop_k4 path's arithmetic unchanged. the test gains --mm-compare attn|score T pos0 baseline.metal pairs.

boundaries and provenance #

  • scratch tree: /tmp/sol_r11_qsa/strata; no tracked vendor files edited, no server requests, no full-model runs, no child agents.
  • clean patch: /tmp/sol_r11_qsa/sol_r11_qsa.patch; git -C vendor/strata apply --check passes.
  • base submodule head: 296b59d2f73c8f04a805df29c2fa97655adace78, plus the existing dirty workspace, not head alone. kernel baseline sha256: 2cdfe150dc1b97c8502299bd34fb06a9f2253d12525ab3dca7f990988ac825a5.
  • exact copied baseline: /tmp/sol_r11_qsa/baseline.metal; baseline tester: /tmp/sol_r11_qsa/baseline_qsa_prompt_parity.cpp.
  • snapshots, binary hashes, runtime-source hash: /tmp/sol_r11_qsa/manifest.json. plan fixed before candidate runs: /tmp/sol_r11_qsa/plan.md.
  • machine: apple m5, coop_k4=1, macos 27.0 build 26A428. safe metal math; no fast-math override. deterministic seed 5; random signed int8 caches, finite positive fp16 scales, fp16 queries. explicit selected ids are evenly spaced unique causal cells; implicit rows poison ids with -1. score inputs use deterministic random fp32 values.

implementation and evaluator #

k row pointers and all four scale groups are resolved once before the eight k steps. the cooperative layout repeats its eight cell identities 16 k later, so the pointer index must be (i >> 2) & 7. v pointers/scales are resolved once per 32-cell run and dimension slice. empty cells read valid row-zero addresses with zero scales; parity includes incomplete 64-cell chunks and the final three-cell selection chunk.

the comparator compiles baseline and candidate in the same process, dispatches them to separate identically sentinel-filled outputs, and compares the entire byte buffer, including inactive/unwritten regions. attention compares part before merge, not a tolerance or rounded output. score compares all scores including the scalar dead-key tail and untouched beyond-causal cells. each side receives one untimed correctness dispatch before measurements. timings alternate baseline/candidate order, one dispatch per sample; final tester prints every pair.

all requested attention cases passed: T={96,512,1024,1984} × pos0={0,1984,2050,30000}. logs: /tmp/sol_r11_qsa/attn-T-pos0.log. static threadgroup memory stays 6912 → 6912 bytes. no m4 hardware test; its fallback arithmetic is retained.

confirmation case baseline median ms candidate median ms result
1984 / 30000, nine pairs, repeat 1 70.487958 39.162417 pass, -44.44%
1984 / 30000, nine pairs, repeat 2 70.648958 39.162750 pass, -44.57%
96 / 1984, nine pairs, repeat 1 2.845500 1.518750 pass, -46.63%
96 / 1984, nine pairs, repeat 2 2.854875 1.517750 pass, -46.84%
1984 / 0, nine pairs 29.593292 15.733958 pass, -46.83%

confirmation logs: /tmp/sol_r11_qsa/attn-confirm-{0..4}.log (also copied into audits/sol_r11_qsa_evidence/ alongside the matrix logs and manifest). all 18 long-context confirmation pairs favor the candidate; repeat-1 baseline samples span 70.160–70.800 ms and candidate samples 39.132–39.196 ms. these stable confirmations support retention under the predeclared >3% improvement rule.

score: valid negative result #

three variants were tested against the original 64×64 score gemm:

  1. two 32×64 gemms into an 8192-byte buffer, float4 head reduction, packed-float4 device output for arbitrary row-stride alignment. parity passes, no consistent speed gain.
  2. one 64×64 gemm into a cooperative accumulator, manual two-window scatter into an 8192-byte buffer. parity passes, roughly 1–2% slower on the large case.
  3. same full gemm, native accumulator stores through tO.slice(0, -row * IH) into the clipped 32-row view, barriers between reuse, float4 reduction. this avoids the manual scatter and passes all 20 requested cases: T={96,512,1024,1984} × pos0={0,1984,2050,2052,30000}.

variant 3 confirms actual pipeline static allocation 16384 → 8192 bytes, but long-context 15-pair medians were 4.532750 → 4.540500 ms and 4.542000 → 4.546917 ms; another three-pair sweep gave 4.529333 → 4.624542 ms. low-position cases often regressed too. reduced allocation does not establish an occupancy or performance win. not included in the patch.

rejected snapshots: /tmp/sol_r11_qsa/score-two-gemm.metal, score-window.metal; logs score-staged-*.log, score-window-*.log, and score-matrix-T-pos0.log. the score-mode evaluator remains useful for later work.

run quality, repairs, and limits #

  • first attention candidate failed parity because the repeated k-layout pointer index omitted & 7; repaired before the matrix/confirmations. the failed snapshot is preserved as candidate-a.metal. no failing kernel code is retained.
  • score baseline-vs-baseline evaluator control passed before modification. final same-source attention control passes; nine-pair medians 2.038333 → 2.048625 ms (~0.5% difference), with early warm-up/load outliers visible in the raw pairs.
  • this is a shared live-server machine, not an isolated performance environment. exploratory samples varied heavily; one early T96/1984 median regressed and was explicitly retested twice. no speed claim relies on that noisy sample or on a single run.
  • all completed invocations were below five seconds. each was launched with a five-second deadline. one redundant final large-case check hit that deadline and was terminated (5.026 seconds including termination overhead); it produced no measurement and is an execution failure, not a parity failure. /tmp/sol_r11_qsa/final-attn-1984-30000-9.log preserves it.
  • the smaller post-review baseline comparison passes (final-attn-small.log); its heavily contended 124.001 → 82.592 ms medians are not used for the primary claim. final score-mode smoke passes with the unchanged retained score kernel; invalid target returns status 2. final build exits 0.
  • the largest attention test allocates about 3.24 gb for its two partial buffers; it is a test-only cost. no model benchmark or end-to-end server speedup was measured.

reproduction #

from the project root, build the standalone evaluator (the existing runtime library is read, not rebuilt):

clang++ -std=c++20 -O2 -I/tmp/sol_r11_qsa/strata/metal/src /tmp/sol_r11_qsa/strata/metal/tests/qsa_prompt_parity.cpp vendor/strata/build-metal/libmtl_rt.a -framework Metal -framework Foundation -o /tmp/sol_r11_qsa/qsa_prompt_parity

set STRATA_METAL_KERNELS=/tmp/sol_r11_qsa/strata/metal/kernels, then run with an external five-second deadline:

/tmp/sol_r11_qsa/qsa_prompt_parity --mm-compare attn 1984 30000 /tmp/sol_r11_qsa/baseline.metal 9

for the rejected score candidate, copy the archived score-window.metal into the scratch kernel path only, and use --mm-compare score ...; restore the retained kernel afterward. do not run these concurrently with other gpu benchmarks. source review completed via full patch images and surrounding definitions; patch touches only the owned attention kernel and test file.