opus_r25_gdn — round 25 track A: gdn_scan_r / gdn_pre / gdn_post #
patch: /tmp/opus_r25_gdn/opus_r25_gdn.patch (git -C vendor/strata apply --check passes). Only metal/kernels/gdn_prompt.metal (gdn_post) changes; the test file is unchanged.
scratch: /tmp/opus_r25_gdn/strata/metal (kernels + src + tests), test binary /tmp/opus_r25_gdn/gpp built with clang++ against the scratch mtl_rt; baseline is /tmp/opus_r25_gdn/base_gdn_prompt.metal.
harness: gdn_prompt_parity --ab base T f32 bench (36-layer runs, 3 paired reps, median ratio). Device: Apple M5. The :8080 server was live, so I repeated every run 3-6x, interleaved with a null run (base vs base) to get the noise floor (null stage ratios 0.99-1.005, stage 2 null shows about +0.1-0.3% order bias).
shipped #
gdn_post: compute the sigmoid gate and load gamma before the 4 simd_sums (same ops in the same order, so bitwise identical).
- bitwise: 10/10 buffers PASS at T = 512/1024/2048 × f32 = 0/1; the default
gdn_prompt_paritymode passes ALL PASS too. - stage 2 (gdn_post) median ratios, interleaved with null:
- T2048 f32=1: null 1.0013 [1.0007..1.0028] vs cand 0.9946 [0.9901..0.9974] (6 reps, no overlap); final version: null [0.988..1.010] vs 0.9935-0.9960 (4 reps)
- T2048 f32=0: null 0.9997 vs cand 0.9920
- T1024 f32=1: null 1.0049 vs cand 0.9963
- net: about -0.7% of gdn_post (~0.25 ms of the 34.8 ms kprof). Small, but it showed up every time.
- only preloading z/gamma, without moving the sigmoid up ("postA"): no clear change (0.993-1.003), not shipped.
measured, not shipped (all bitwise PASS, none a win) #
gdn_scan_r (gdn_scanr_4), stage 1 median ratios:
| variant | T512 | T1024 |
|---|---|---|
| null | 0.998 | 1.000 |
.y reduce only on sg==0 (split_y) |
0.999 | 1.003 (neutral) |
| kq load after barrier in sg==0 (kq_late) | 1.011 | 1.019 (loss) |
| g_b/v loaded for t after barrier, no g_b_n/v_n (gbv_late) | 1.015 | 1.025 (loss) |
| kn_l/qn_l loaded after barrier (knq_late) | 1.051 | 1.040 (loss) |
| split_y + kq_late together (the brief's 1+2+3) | 1.013 | — (loss) |
| gbv_late + kq_late | 1.088 | 1.099 (noisy, loss) |
| all four | 1.19 | 1.09 (loss) |
why: the per-token loop is bound by load latency through the barrier, not by registers. The t+1 loads put in front of threadgroup_barrier hide their latency behind the barrier wait and the other simdgroups. Moving them after the barrier puts that latency back on the critical path, and fewer live registers buys nothing here: no occupancy step shows up, since tg = 128 and st[32] dominates the register count. The sg==0-only .y sum saves 3 threadgroup loads and adds on the 3 other sgs, but sg 0 is the slow one anyway (it does the o store), so it nets zero. If anything, the better direction is deeper prefetch (t+2), which I didn't try.
gdn_pre: hoisting cw_q/cw_k to constant-offset [m*32], moving t == T-1 out of the m loop and adding unroll pragmas to the hist k loops: stage 0 0.9985-1.010, neutral (the compiler already folds these offsets, and the hist path runs for 1 of T tokens). Not shipped. The k unroll pragmas at 97/137/148/166 are only on t < 3 / t == T-1 paths, so by construction they can't move the timing.