7.8 kB 65 lines
aeb4f92d
Round 10 blitz: qsa_flash 2.2x (-51.9% / -146.1 ms in kprof via unconditional hoisted rv/rk + vec4 epilogue), gdn_pre float4 conv, uint4 mm_gather_h + s_id-free mm_place
dawn
⋯
audits/sol_r10_gdn.md
sourcerendered
download raw history
at main Markdown

round 10 track b: gdn vectorization and scan #

contract / preflight #

owner: gdn_prompt.metal; only its parity test may change alongside it. isolated source: /tmp/sol_r10_gdn/strata; tracked vendor files stay untouched. no server or full-model runs.

baseline is the exact working-copy kernel captured at /tmp/sol_r10_gdn/baseline.metal. maintain arithmetic order, layouts, entry names and both packed/legacy launch shapes. require memcmp on all six pre outputs, scan output/state and post fp16/fp32 (including untouched fp32 when disabled). fixed rng seed 21; t=1,3,37,300,512,1024,2048, f32=0/1. compare separate baseline/candidate libraries. evaluate paired 36-dispatch gpu timings at t=512/1024/2048; each invocation under 5 seconds, no parallel gpu benchmarking. accept only repeatable >3% improvements without regressions; otherwise retain baseline.

important inspection correction: post currently owns lane + k*32, not lane*4+k. naive contiguous float4 loads change the reduction tree and threaten bitwise identity. pre uses the same strided mapping. first test weight vector loads / conv boundary split and scan pointer hoisting; independently probe post vectorization with explicit reduction-layout preservation. max 3 candidate variants, short exploratory then confirmation runs.

source and execution #

  • device: apple m5; macos 27.0 (26a428), apple clang 21.0.0 (clang-2100.3.27.1). runtime safe math, default STRATA_METAL_MATH unset; production hk=16, hv=48, channels=10240, eps=1e-6.
  • vendor head: 296b59d2f73c8f04a805df29c2fa97655adace78; the baseline is the dirty working-copy snapshot, not that commit's pristine tree. baseline kernel sha256: 8f8205112850c4470b08a3b9430ea8c857fb8c9890a3c7a43815532bdf3c379a.
  • fixed source snapshots: /tmp/sol_r10_gdn/baseline.metal, candidate1.metal, candidate2.metal, candidate3.metal. final kernel differs from candidate3 only by moving a pragma to column zero.
  • isolated final source: /tmp/sol_r10_gdn/strata/metal/{kernels/gdn_prompt.metal,tests/gdn_prompt_parity.cpp}.
  • linked against existing vendor/strata/build-metal/libmtl_rt.a, built with clang++ -std=c++20 -O2 -Ivendor/strata/metal/src /tmp/sol_r10_gdn/strata/metal/tests/gdn_prompt_parity.cpp vendor/strata/build-metal/libmtl_rt.a -framework Metal -framework Foundation -o /tmp/sol_r10_gdn/gdn_prompt_parity.
  • run pattern: STRATA_METAL_KERNELS=/tmp/sol_r10_gdn/strata/metal/kernels /tmp/sol_r10_gdn/gdn_prompt_parity --ab /tmp/sol_r10_gdn/baseline.metal 2048 0 bench. omit bench for parity only. one process per shape/mode; gpu tests run sequentially. job deadlines were 4.9 seconds.

variants #

  1. vector-load four conv weights, split mature-token conv from history boundary; invariant scan pointers plus unroll2; post retains scalar reduction loads and vectorizes elementwise loads/stores. all bitwise checks passed, but pre/scan gains were around 0.5–1%, and post was slightly slower.
  2. same pre; scan unroll4; post loads one contiguous float4 and shuffles back into the original strided reduction mapping before the unchanged four simd_sums. this preserves the reduction tree, unlike naive contiguous reduction. all checks passed; scan around 0.4–1% faster, post roughly flat to 1.8% faster with some regressions/noisy samples. not included in final patch.
  3. value-head-only float4 conv for packed pre and t>=3. key-head arithmetic and reductions are untouched; t<3 retains scalar history handling; legacy launch stays scalar. loads vectorized qkv taps and weights, transposes the 4x4 tap weights in registers, keeps exactly four ordered fmas per element and the original silu arithmetic. stores vc as float4; last-token history still writes its original channel-major format. scan and post are unchanged.

correctness #

all three variants passed the 7 token lengths × 2 f32 modes × 10 outputs matrix: 420/420 individual memcmp comparisons. each variant individually passed 140/140. final selected source was rerun after formatting: 140/140 passed, exit 0 for all 14 invocations. outputs are qn, kn, vc, gb, kq, history, o, state, yh, y. f32=0 checks that y retains its sentinel bytes rather than silently skipping the output check.

reference and candidate libraries compile separately from immutable source files. pre outputs are snapshotted and restored before scan; state starts from the same input for both scans; reference scan output is restored before post. each stage therefore gets identical inputs even if an earlier candidate stage fails.

a deliberate fp16 output corruption returned exit 1 with yh FAIL; invalid --ab arguments returned exit 2. baseline-vs-baseline control returned exit 0. final git -C vendor/strata apply --check /tmp/sol_r10_gdn/sol_r10_gdn.patch passed. vendor owned files remained byte-identical to the captured starting files; no live server/full-model command ran.

raw parity: /tmp/sol_r10_gdn/parity_v*_t*_f*.log, parity_matrix.json, and final_t*_f*.log. controls: negative_control.log, invalid_args.log, final_baseline_control.log.

paired 36-layer confirmation #

confirmation warms each stage for 50 ms, runs one untimed 36-dispatch batch of each library, then measures three alternating-order A/B pairs. reported b/c are separate medians; percentage is from median paired candidate/baseline ratio. repeated dispatches use the same buffers, not 36 distinct model layers. no full-model speedup is claimed.

final candidate3 pre, milliseconds for 36 dispatches:

t f32 baseline candidate paired change
512 0 12.035 11.694 -2.84%
1024 0 23.905 23.425 -2.01%
2048 0 47.747 46.635 -2.34%
512 1 12.038 11.713 -2.89%
1024 1 23.924 23.269 -2.81%
2048 1 47.097 46.207 -1.92%

scan/post in candidate3 are unchanged and serve as noise controls. at t=2048 f32=0, scan was 201.771/202.670 ms and post 34.037/34.135 ms. estimated pre saving is only about 1.1 ms across 36 layers at t=2048.

external gpu contention occurred in two groups of samples (e.g. t512 pre suddenly ~125 ms instead of ~12 ms). those samples are not used to support a win. one candidate2 t2048 f32=0 confirmation was terminated at the 4.9-second deadline (observed 4.926 s); no complete timing from it is claimed. all completed standalone invocations stayed below 5 seconds (largest confirmation 3.712 s). exploratory t2048 candidate2 f32=0 completed earlier: pre 46.964/46.805 ms, scan 195.890/194.583 ms, post 33.925/33.663 ms; that earlier evaluator had only a 15 ms warmup and no discarded 36-dispatch pair.

raw performance: /tmp/sol_r10_gdn/candidate*_t*.log, confirm_v*_t*_f*.log, confirmation.json. the json records exit status and duration, including the deadline stop. initial exploratory results were preserved, not replaced by confirmation logs.

decision and deliverables #

the predeclared >3% rule was not met: no production acceptance claim. parent explicitly requested the value-only float4 candidate anyway for a later uncontended full-model kprof check. therefore the requested clean main patch includes candidate3 plus the reusable A/B evaluator, while scan and post stay unchanged. parent should make the final accept/reject call from that check; do not describe this as an established full-model win.

  • main patch: /tmp/sol_r10_gdn/sol_r10_gdn.patch (only the two owned files; apply check passed).
  • optional rejected exploratory patches: /tmp/sol_r10_gdn/candidate1.patch, candidate2.patch, candidate3.patch (kernel-only; candidate3's pragma has original snapshot whitespace).
  • report: audits/sol_r10_gdn.md.

visual diff/context review completed for final test code and selected kernel; no tracked vendor file was edited.