6.9 kB 71 lines
393a2a86
round 39: port serving fixes and PLE reuse; verify parity, record perf uncertainty
dawn
⋯
audits/astra_r39_qsa.md
sourcerendered
download raw history
at main Markdown

round 39 track a — qsa upstream fixes and local merge experiment #

decision #

  • keep 831ba47a's clamped-row/unconditional-load/value-select fix in the four non-cooperative prompt fills. preserve our staged scales and cooperative paths.
  • keep ad734531's TK = 64 default and explicit v2 || v3 gather/HS/binding conditions. this tree already sets v2 = v3 = true for --v3, so the binding change is documentary/defensive here, not a newly repaired v3 failure. existing half-input gather and --gu-bench, --dn-bench, --ab, --dtile remain intact.
  • keep the local four-partial-load batching experiment in qsa_merge2. this is inspired by 731528be, not a literal upstream transplant. active-chunk bounds already existed and were not changed.
  • reject the exact requested six-shuffle qsa_score butterfly: scores FAIL, exit 1, on its first --ab case at p=3000. no speed claim for a numerically invalid candidate. cuda's reduction-order argument does not establish equivalence to metal simd_sum; the mismatch is measured, its compiler-level cause is not proven.
  • reject four-partial batching in qsa_merge: bitwise correct, but nt=3 regressed repeatedly by 4.8–6.6% across all three contexts. qsa.metal is entirely unchanged in the delivered patch.

artifacts and reproducibility #

  • patch: /tmp/astra_r39_qsa/astra_r39_qsa.patch
  • durable copy, source snapshots, evaluators, metadata, commands, exit statuses and full logs: results/round39/astra_r39_qsa/
  • exact baseline: the working-tree snapshot, not merely git head; metadata.json records its head and four owned-file sha256 values. baseline-kernels/ and baseline-moe_mm_parity.cpp preserve inputs. rejected-score.metal and rejected-merge1.metal preserve rejected experiments; final/ preserves the delivered sources.
  • all editing/building ran in /tmp/astra_r39_qsa/strata; no tracked vendor files, live service or full-model tests were touched.
  • device: apple m5; system version recorded in metadata.json. apple clang 21.0.0, release -O2. build used cmake -S metal -B build-metal -G Ninja followed by cmake --build build-metal --target qsa_parity qsa_prompt_parity moe_mm_parity -j 3.
  • tests were serialized on the gpu. initial moe runs exceeded the original 15s/20s limits and were terminated; after parent explicitly extended CPU-reference allowance to 180s, both full tests completed. all unsuccessful attempts remain in the logs.
  • git -C vendor/strata apply --check /tmp/astra_r39_qsa/astra_r39_qsa.patch: exit 0. scratch git diff --check: exit 0. final visual diff/context review completed.

correctness and test results #

all commands below run from the scratch tree; BASE=/tmp/astra_r39_qsa/baseline.

test result
baseline qsa_parity --ab $BASE 3000 exit 0, all memcmps pass
baseline moe_mm_parity exit 3, missing mm_gu_42_16_64_16_2
final moe_mm_parity (no args) exit 0, 23.961s, PASS; fp16-reference rel l2 2.03e-06, exact-reference 2.37e-04
final moe_mm_parity --v3 --gu IQ2_S --dn Q2_0 --tile 16,32,32,4 --dtile 64,32,2 exit 0, 20.011s, PASS; fp16-reference rel l2 1.60e-05, exact-reference 4.38e-04
same v3 command plus --ab $BASE exit 0, HS/YE/Y memcmp pass, t=256
default and specified v3 plus --T 1 exit 0, CPU-reference parity pass
v3 plus --T 1 --gu-bench --ab $BASE, and --dn-bench both exit 0; HS/YE/Y memcmp pass; flags preserved
qsa_prompt_parity exit 0, ALL PASS, default cooperative layout
COOP_K4=0 qsa_prompt_parity exit 0, ALL PASS, edited non-cooperative layout
qsa_prompt_parity --mm-compare attn 96 P $BASE/qsa_prompt.metal 3 p=0 and 30000, both COOP_K4=0 and default: exit 0; output, post-timing and merge+prepared memcmp pass
qsa_prompt_parity --flash-compare 96 P $BASE/qsa_prompt.metal 1 3 same four combinations: exit 0, memcmp pass
QSA_ATTN=2 qsa_parity --ab $BASE P p=3000,10180,30000: all full-buffer memcmps pass; nt=1,2,3,4,8 plus prompt prep nt=1984,2048
QSA_ATTN=2 qsa_parity --bitwise $BASE/qsa2.metal exit 0; zero of 40 cases differ, including short contexts/remainders

important distinction: moe's no-argument and specified-v3 CPU checks are tolerance checks and print PASS, not ALL PASS or bitwise equality to CPU. the separate v3 A/B command provides bitwise equality to the baseline kernels. prompt's default suite likewise includes tolerance checks; the comparison modes provide bitwise evidence for both edited kernels. no unsupported claim of IEEE equality to a differently accumulated CPU reference is made.

merge2 paired timings #

three sweeps: initial, confirmation, then final with rejected merge1 restored. all pass. the final sweep below uses only retained changes. measurements are median GPU µs; paired ratio is median(candidate/baseline), not necessarily the quotient of medians. inspected evaluator code uses five paired rounds, 16 dispatch repetitions for decode (the file header's 12-repetition description is stale). each stage is warmed before interleaved A/B timing.

p nt baseline µs candidate µs paired candidate/baseline
3000 1 15.143 9.273 0.6122
3000 2 21.924 21.516 0.9812
3000 3 33.688 32.969 0.9791
3000 4 45.458 43.227 0.9522
3000 8 91.102 86.547 0.9501
10180 1 10.961 6.844 0.6240
10180 2 21.977 21.589 0.9834
10180 3 33.732 32.961 0.9772
10180 4 45.344 43.313 0.9546
10180 8 91.393 86.607 0.9493
30000 1 17.180 10.807 0.6295
30000 2 22.125 21.547 0.9739
30000 3 33.500 33.471 0.9955
30000 4 45.469 43.544 0.9577
30000 8 91.594 88.021 0.9556

nt=1 improves 35–41% across the earlier sweeps and 37–39% in the final sweep; other cases are neutral to about 5% faster. one initial p=30000, nt=3 ratio was 1.0008 (0.08% noise); subsequent ratios were 0.9902 and 0.9955. no repeatable regression. absolute clocks/times drift (even unchanged stages), so use paired ratios, not cross-run raw times. this is a standalone-kernel result, not evidence of whole-model throughput improvement.

for the rejected merge1, confirmation nt=3 ratios were 1.0551, 1.0660, 1.0563 at p=3000,10180,30000. despite wins elsewhere, these violate the requested neutral-to-faster-across-suite rule.

prompt safety fix timing observation #

with COOP_K4=0, p=30000/t=96 non-cooperative attn median fell from 4.954 to 1.537ms, flash from 5.685 to 1.828ms, with memcmp pass. p=0 also improved. these three-pair observations are secondary to the safety fix, not broad performance claims. cooperative mode remains bitwise identical. actual m1 hardware was not available; the m1 fault rationale comes from upstream 831ba47a, while our runtime evidence is m5 with both constant settings.