Round 33 blitz: gdn_step_t kl split + pre-barrier3 state commit (-1.1..-4.1%), gr_norm hoist (-6..-11%), GrdUnit<T_Q8_0,2> packed char4 (-10..-16%) -> 628.51 tok/s GPU prefill, mtp_ms 124.20/193.49/323.71ms
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596# Round 33 Track C Audit Report: Decode GDN Recurrence (`gdn.metal`) & `glue.metal`
- **Agent:** `argon-r33-gdn-and-glue-opt` (`gemini-4-argon`)- **Target Device:** Apple M5 (10-core GPU, 24 GB unified memory)- **Patch Path:** `/tmp/argon_r33_gdn/argon_r33_gdn.patch` (`git -C vendor/strata apply --check` clean)- **Scratch Repo:** `/tmp/argon_r33_gdn/strata`- **Baseline Kernels:** `/tmp/argon_r33_gdn/base_kernels`
---
## 1. Optimizations Implemented & Tested
### Target 1: `gdn_step_t` (`metal/kernels/gdn.metal`, lines 99–162)1. **Critical-Path `kl` Split Across Barrier 1 and Barrier 2:** - Immediately after Barrier 1 (`line 125`), only `tot_k`, `rsk`, and `kl = (rsk * ks[qk_idx]) * scale` are computed before launching the 32-iteration `kv` FMA loop (`for (int r = 0; r < RPT; ++r) kv = fma(st[r], simd_broadcast(kl, ushort(r)), kv);`) and storing `red[ib][j] = kv`. - `tot_q`, `rsq`, `ql = (rsq * qs[qk_idx]) * scale`, `g = gb[0]`, `be = gb[1]`, and `vj = vs[j]` are issued after `red[ib][j] = kv` right before Barrier 2 (`line 140`), shortening the live range of `ql`, `g`, `be`, and `vj` across the 32-step `kv` loop and overlapping their latency with the `red[ib][j]` store and Barrier 2.2. **Early Conv-History Commit (`hist_out`) & Pre-Barrier-3 State Commit (`state`):** - **Conv History (`hist_out`):** `hst[0..2]` is fully updated at the end of the `conv + SiLU` block (`line 101`, before Barrier 1). Emitting the `hist_out[ch * 3 + k] = hst[k]` stores immediately inside `if (ch >= 0)` when `t + 1 == commit_at && (tid >= 256 || h < int(a.h_k))` makes `hst[0..2]` dead before Barrier 1 on single-token decode (`GDN_NTOK == 1`) and on the final token of any multi-token window. - **Recurrent State (`state`):** `st[0..31]` is finalized at the end of the `st`/`o` update loop (`line 150`, before Barrier 3). Previously, the 32 global state stores per thread were deferred until after Barrier 3, the `ib == 0` readout reduction, Barrier 4, and the `ib == 0` `y`/`plane` stores — leaving simdgroups `ib = 1..3` (75% of the threadgroup) idle across two barriers before issuing 64 KB of global stores per head. Issuing `st_base[r * st_stride] = st[r]` immediately before Barrier 3 overlaps the global state writeback with `ib == 0`'s readout norm and output plane stores across Barriers 3 and 4. - **Rejected Variant (`1b` — deferring `tot_q`/`rsq`/`ql`/`g`/`be` until *after* Barrier 2):** Tested and rejected (+1.5% to +3.7% regression) because issuing 7 threadgroup loads (`part[0..3]`, `gb[0..1]`, `qs[qk_idx]`) and `rsqrt` after Barrier 2 delays the `red[0..3][j]` loads and the `st`/`o` loop. - **Rejected Variant (`zz` load moved after `st`/`o` loop):** Tested and rejected (+1% to +5% regression) because prefetching `zz` at the top of the token loop hides its global memory latency behind the 4 barriers of token `t`.
### Target 2: `swiglu_prep` & `moe_combine_sh` (`metal/kernels/glue.metal`, lines 18–56)- In `moe_combine_sh`: hoisted `const int row_base = int(t * a.topk);`, `device const int* te = plan + P_TOKENT + row_base;`, and `device const float* yep = ye + k;` outside the 10-way unrolled expert accumulation loop so `yep[te[j] * int(a.n_embd)]` eliminates the `+ k` address addition across all 10 loads.- In `swiglu_prep`: hoisted `const uint idx = t * a.n + d;` across `g[idx]`, `u[idx]`, and `xp + idx` while preserving the uniform `t * (a.n / 16) + tg.x` address for the `j == 0` plane-4 sum write.
### Test Harness (`metal/tests/gdn_parity.cpp`)- Extended `gdn_parity --ab <ref.metal|ref_dir>` to accept a directory (`/tmp/argon_r33_gdn/base_kernels`) or file path, run all 88/88 GDN bitwise checks (`ntok = 1..8`, `plane = 0 / 49152`, `commit = 0 / 1 / 0x101..0x107`), run paired A/B timings across all 32 `(plane, ntok, commit)` configurations (36 layers rotated, 113 MB state), and when `glue.metal` is present in `ref_dir`, run 24/24 bitwise checks (`swiglu_prep` and `moe_combine_sh` for `ntok = 1..8`, `topk = 10, 8`) plus paired A/B timings.
---
## 2. Verification & Bitwise Parity Results
- **`gdn_parity --ab /tmp/argon_r33_gdn/base_kernels`** (`gdn_step`, `IB = 4`, 512 threads): - Pipeline occupancy: `max 512 threads per threadgroup (the kernel needs 512)` for all `GDN_NTOK = 1..8`. - GDN A/B bitwise checks: **88/88 `PASS` (`0 fail(s)`)** across `y`, `state`, and `hist_out`. - Glue A/B bitwise checks: **24/24 `PASS` (`0 fail(s)`)** across `swiglu_prep` (`xp` planes 0..4) and `moe_combine_sh` (`bo`, `topk = 10, 8`).- **`GDN_IB8=1 gdn_parity --ab /tmp/argon_r33_gdn/base_kernels --no-bench`** (`gdn_step8`, `IB = 8`, 1024 threads): - Pipeline occupancy: `max 1024 threads per threadgroup (the kernel needs 1024)` for all `GDN_NTOK = 1..8`. - GDN A/B bitwise checks: **88/88 `PASS` (`0 fail(s)`)**.- **`gdn_parity`** (double-precision CPU reference + window invariants + `plane = 49152` verification): **`ALL PASS`**.- **`fuse_parity`** (`gemv_cat`, `router_route`, `moe_combine_sh + gr_norm == gr_norm_comb / gr_norm_comb_4s` across `ntok = 1..8`): **`ALL PASS`**.
---
## 3. Paired A/B Benchmark Results on Apple M5
### 3.1 `gdn_step` Paired A/B (36 layers rotated, 7 interleaved pairs x 72 dispatches)
| `plane` | `ntok` | Mode | Base (us/layer) | Cand (us/layer) | Paired Ratio (`cand/base`) | 36-Layer Total (Base -> Cand) ||---:|---:|---|---:|---:|---:|---:|| 0 | 1 | commit | 55.81 | 55.08 | **0.9877** (-1.23%) | 2.009 -> 1.983 ms || 0 | 1 | verify | 27.77 | 27.25 | **0.9895** (-1.05%) | 1.000 -> 0.981 ms || 0 | 2 | commit | 60.26 | 58.54 | **0.9775** (-2.25%) | 2.169 -> 2.107 ms || 0 | 2 | verify | 32.34 | 31.75 | **0.9813** (-1.87%) | 1.164 -> 1.143 ms || 0 | 3 | commit | 61.59 | 60.82 | **0.9842** (-1.58%) | 2.217 -> 2.190 ms || 0 | 3 | verify | 37.68 | 36.96 | **0.9844** (-1.56%) | 1.356 -> 1.330 ms || 0 | 4 | commit | 63.45 | 62.54 | **0.9891** (-1.09%) | 2.284 -> 2.251 ms || 0 | 4 | verify | 42.85 | 41.80 | **0.9768** (-2.32%) | 1.543 -> 1.505 ms || 0 | 5 | commit | 66.20 | 64.15 | **0.9717** (-2.83%) | 2.383 -> 2.309 ms || 0 | 5 | verify | 48.23 | 46.85 | **0.9702** (-2.98%) | 1.736 -> 1.687 ms || 0 | 6 | commit | 71.00 | 68.55 | **0.9638** (-3.62%) | 2.556 -> 2.468 ms || 0 | 6 | verify | 53.81 | 52.12 | **0.9692** (-3.08%) | 1.937 -> 1.876 ms || 0 | 7 | commit | 75.97 | 73.37 | **0.9658** (-3.42%) | 2.735 -> 2.641 ms || 0 | 7 | verify | 59.01 | 57.23 | **0.9698** (-3.02%) | 2.124 -> 2.060 ms || 0 | 8 | commit | 81.66 | 78.20 | **0.9589** (-4.11%) | 2.940 -> 2.815 ms || 0 | 8 | verify | 64.65 | 62.72 | **0.9683** (-3.17%) | 2.327 -> 2.258 ms || 49152 | 1 | commit | 56.16 | 56.03 | **0.9943** (-0.57%) | 2.022 -> 2.017 ms || 49152 | 1 | verify | 27.37 | 27.23 | **0.9959** (-0.41%) | 0.985 -> 0.980 ms || 49152 | 2 | commit | 64.22 | 62.80 | **0.9808** (-1.92%) | 2.312 -> 2.261 ms || 49152 | 2 | verify | 33.07 | 32.66 | **0.9868** (-1.32%) | 1.190 -> 1.176 ms || 49152 | 3 | commit | 62.00 | 60.73 | **0.9797** (-2.03%) | 2.232 -> 2.186 ms || 49152 | 3 | verify | 38.34 | 38.05 | **0.9934** (-0.66%) | 1.380 -> 1.370 ms || 49152 | 4 | commit | 64.05 | 63.00 | **0.9833** (-1.67%) | 2.306 -> 2.268 ms || 49152 | 4 | verify | 43.43 | 43.18 | **0.9942** (-0.58%) | 1.564 -> 1.555 ms || 49152 | 5 | commit | 66.96 | 65.55 | **0.9794** (-2.06%) | 2.411 -> 2.360 ms || 49152 | 5 | verify | 49.08 | 48.56 | **0.9917** (-0.83%) | 1.767 -> 1.748 ms || 49152 | 6 | commit | 71.80 | 70.17 | **0.9766** (-2.34%) | 2.585 -> 2.526 ms || 49152 | 6 | verify | 54.65 | 54.10 | **0.9919** (-0.81%) | 1.967 -> 1.948 ms || 49152 | 7 | commit | 77.41 | 75.36 | **0.9748** (-2.52%) | 2.787 -> 2.713 ms || 49152 | 7 | verify | 60.30 | 59.62 | **0.9865** (-1.35%) | 2.171 -> 2.146 ms || 49152 | 8 | commit | 82.69 | 80.06 | **0.9717** (-2.83%) | 2.977 -> 2.882 ms || 49152 | 8 | verify | 65.62 | 64.97 | **0.9899** (-1.01%) | 2.362 -> 2.339 ms |
### 3.2 `glue.metal` Paired A/B (`swiglu_prep` & `moe_combine_sh`, 256 dispatches/cb with buffer barrier)
| Kernel | `ntok` | Base (us) | Cand (us) | Paired Ratio (`cand/base`) ||---|---:|---:|---:|---:|| `swiglu_prep` | 1 | 1.01 | 1.02 | 1.0118 || `swiglu_prep` | 2 | 1.10 | 1.10 | 1.0003 || `swiglu_prep` | 4 | 1.25 | 1.25 | **0.9982** || `swiglu_prep` | 8 | 1.57 | 1.55 | **0.9941** || `moe_combine_sh` | 1 | 1.28 | 1.30 | 1.0178 || `moe_combine_sh` | 2 | 1.50 | 1.49 | **0.9973** || `moe_combine_sh` | 4 | 1.97 | 1.97 | **0.9937** || `moe_combine_sh` | 8 | 2.78 | 2.77 | **0.9971** |123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116# opus_r33_gr — Round 33 Track A: hyper-connection read (`gr_norm`, `gr_norm_comb`, `gr_down_f`, `gr_up_f`)
Patch: `/tmp/opus_r33_gr/opus_r33_gr.patch` (`git -C vendor/strata apply --check` OK; touches only `metal/kernels/gr.metal`and `metal/tests/fuse_parity.cpp`). Scratch tree: `/tmp/opus_r33_gr/strata`; baseline kernels: `/tmp/opus_r33_gr/base_kernels`;raw final logs: `/tmp/opus_r33_gr/final_bench.txt`. Device: Apple M5 (10-core GPU), GPU quiet for the final run.
## Shipped (all bitwise PASS, ntok 1..8, every gr.metal/gr2.metal entry point incl. BF16/Q4/Q5/F32-inject; default `fuse_parity` ALL PASS)
1. **`gr_norm`**: `apply` branch hoisted out of the 10-step unrolled loop, pointers pre-offset (`rp = r + tid`, `bp = bo + t*2560 + tid`, `o += tid; w += tid`) so every access is an immediate `[i * 256]`. **-6..-11% at ntok 1..4, -2..-3% at 7..8** (~0.2 µs per dispatch; gr_norm is ~2.3-4.3 µs, mostly launch-bound).2. **`grd_acc` Q8_0, NR = 2 (`gr_down_q8_*_r2`, `gr_down_q8f_*_r2`)**: a `GrdUnit<F, NR>` holder with a `GrdUnit<T_Q8_0, 2>` specialization that keeps the codes packed as `char4[8]` per row (8 regs instead of 32 floats) and widens with `float4(char4)` per token. Same fmas in the same order, so bitwise identical. The generic path is unchanged codegen (all other formats/NR = 1: ±0.5%, noise). **ntok 7: -13..-16%, ntok 8: -10..-15%**; ntok 5-6 neutral; ntok ≤ 4 never uses `_r2` in the engine.3. **`fuse_parity --gr-unroll ... bench`**: 16 pairs × 128 iters, reports the median pair ratio (A/A noise went from ±15% to ±0.3% for gr_down/gr_up, ~±2% for the tiny gr_norm_comb); gr_norm/gr_norm_comb now timed; added same-entry A/B benches for the engine's real entry points (`gr_down_q8_4_4`, `_4_4_r2`, `q8f_4_4(_r2)`, `q8_8_8(_r2)`).
## Final paired timings (median Δ, quiet GPU)
| kernel | ntok | base µs | new µs | median Δ ||---|---|---|---|---|| gr_down_q8_4_4 | 1 | 23.213 | 23.210 | +0.0% || gr_down_q8_4_4 | 2 | 23.836 | 23.831 | +0.0% || gr_down_q8_4_4 | 3 | 31.574 | 31.606 | +0.1% || gr_down_q8_4_4 | 4 | 40.602 | 40.610 | +0.0% || gr_down_q8_4_4 | 5 | 55.545 | 55.580 | +0.0% || gr_down_q8_4_4 | 6 | 75.455 | 75.441 | +0.0% || gr_down_q8_4_4 | 7 | 97.495 | 97.445 | -0.0% || gr_down_q8_4_4 | 8 | 119.752 | 119.762 | -0.0% || gr_down_q8_4_4_r2 | 1 | 23.348 | 23.443 | +0.4% || gr_down_q8_4_4_r2 | 2 | 23.814 | 23.786 | -0.2% || gr_down_q8_4_4_r2 | 3 | 32.917 | 32.977 | +0.2% || gr_down_q8_4_4_r2 | 4 | 41.266 | 41.331 | +0.1% || gr_down_q8_4_4_r2 | 5 | 49.493 | 49.428 | -0.1% || gr_down_q8_4_4_r2 | 6 | 58.338 | 58.370 | +0.1% || gr_down_q8_4_4_r2 | 7 | 79.517 | 68.066 | -14.4% || gr_down_q8_4_4_r2 | 8 | 90.285 | 79.752 | -11.7% || gr_down_q8_8_8 | 1 | 23.584 | 23.729 | +0.6% || gr_down_q8_8_8 | 2 | 24.839 | 24.843 | -0.0% || gr_down_q8_8_8 | 3 | 34.813 | 34.802 | -0.1% || gr_down_q8_8_8 | 4 | 45.466 | 45.464 | -0.0% || gr_down_q8_8_8 | 5 | 62.634 | 62.615 | -0.0% || gr_down_q8_8_8 | 6 | 83.515 | 83.568 | +0.1% || gr_down_q8_8_8 | 7 | 115.168 | 115.243 | +0.0% || gr_down_q8_8_8 | 8 | 140.286 | 140.303 | +0.1% || gr_down_q8_8_8_r2 | 1 | 23.953 | 24.036 | +0.3% || gr_down_q8_8_8_r2 | 2 | 25.738 | 25.852 | +0.5% || gr_down_q8_8_8_r2 | 3 | 34.776 | 34.844 | +0.2% || gr_down_q8_8_8_r2 | 4 | 46.604 | 46.623 | +0.0% || gr_down_q8_8_8_r2 | 5 | 58.045 | 58.184 | +0.3% || gr_down_q8_8_8_r2 | 6 | 70.499 | 70.514 | -0.1% || gr_down_q8_8_8_r2 | 7 | 100.167 | 83.844 | -16.3% || gr_down_q8_8_8_r2 | 8 | 114.260 | 97.570 | -14.6% || gr_down_q8f_4_4_r2 | 1 | 24.422 | 24.454 | +0.1% || gr_down_q8f_4_4_r2 | 2 | 24.889 | 24.863 | -0.4% || gr_down_q8f_4_4_r2 | 3 | 33.414 | 33.478 | +0.2% || gr_down_q8f_4_4_r2 | 4 | 41.238 | 41.304 | +0.2% || gr_down_q8f_4_4_r2 | 5 | 49.481 | 49.448 | -0.1% || gr_down_q8f_4_4_r2 | 6 | 58.534 | 58.637 | +0.2% || gr_down_q8f_4_4_r2 | 7 | 80.516 | 70.128 | -12.9% || gr_down_q8f_4_4_r2 | 8 | 91.064 | 81.766 | -10.2% || gr_norm apply 0 | 1 | 3.241 | 2.882 | -11.1% || gr_norm apply 0 | 2 | 2.478 | 2.227 | -10.4% || gr_norm apply 0 | 3 | 2.506 | 2.269 | -9.2% || gr_norm apply 0 | 4 | 2.578 | 2.400 | -7.1% || gr_norm apply 0 | 5 | 2.668 | 2.523 | -5.5% || gr_norm apply 0 | 6 | 3.039 | 2.849 | -6.4% || gr_norm apply 0 | 7 | 3.044 | 2.958 | -2.8% || gr_norm apply 0 | 8 | 3.185 | 3.084 | -3.3% || gr_norm apply 1 | 1 | 3.293 | 3.088 | -6.1% || gr_norm apply 1 | 2 | 3.347 | 3.096 | -7.7% || gr_norm apply 1 | 3 | 3.361 | 3.160 | -6.2% || gr_norm apply 1 | 4 | 3.468 | 3.256 | -5.9% || gr_norm apply 1 | 5 | 3.620 | 3.428 | -5.4% || gr_norm apply 1 | 6 | 3.834 | 3.763 | -1.8% || gr_norm apply 1 | 7 | 4.069 | 3.980 | -2.2% || gr_norm apply 1 | 8 | 4.346 | 4.255 | -2.2% || gr_norm_comb | 1 | 5.135 | 5.047 | -1.4% || gr_norm_comb | 2 | 5.335 | 5.363 | +0.7% || gr_norm_comb | 3 | 6.176 | 6.248 | +1.3% || gr_norm_comb | 4 | 6.754 | 6.678 | -1.0% || gr_norm_comb | 5 | 6.985 | 6.874 | -1.5% || gr_norm_comb | 6 | 8.941 | 8.889 | -0.7% || gr_norm_comb | 7 | 9.091 | 9.133 | +0.5% || gr_norm_comb | 8 | 10.624 | 10.563 | -0.7% || gr_up_q8 | 1 | 23.770 | 23.762 | -0.0% || gr_up_q8 | 2 | 24.547 | 24.535 | -0.1% || gr_up_q8 | 3 | 26.172 | 26.186 | +0.1% || gr_up_q8 | 4 | 29.213 | 29.184 | -0.1% || gr_up_q8 | 5 | 32.855 | 32.889 | +0.1% || gr_up_q8 | 6 | 37.063 | 37.025 | -0.1% || gr_up_q8 | 7 | 42.019 | 41.957 | -0.1% || gr_up_q8 | 8 | 47.111 | 47.046 | -0.1% |
## Tried and rejected (measured, bitwise-safe but not wins)
- `deq<T_Q8_0>` `#pragma unroll(full)`: 0.0% (the compiler already unrolls it). Not shipped, quant.metal untouched.- Packed Q8 for **NR = 1** gr_down: +1..3% at ntok 2 (`q8_8_8` +3.2%), neutral elsewhere → restricted to NR = 2.- Packed Q8 + always-unrolled token loop at ntok 7-8: +12..45% (registers). Rolled-from-ntok-5 for NR = 2: `4_4_r2` -3% at ntok 5 but `8_8_r2` +0.4..1%; kept the existing ≤ 6 threshold.- `unroll_count(3)` on the gr_down `u` loop: +2..42% everywhere. Software-pipelined prefetch of the next Q8 block: +3..53%.- Aligned 9×`uint` Q8 block load + funnel shift instead of 16 `ushort` loads: +25..66% (much worse).- `gr_up_q8`: staging `lo` in `threadgroup float[8*320]`: +1..6% (lo reads are already L1 broadcast hits); packed Q8 dot: +2..8% at ntok 4-8; `unroll_count(3)`: +4..30%.- `gr_norm_comb` apply-hoist + `gs` guard: -0.3..-0.8% vs repeated A/A (within the ~1-2% order bias of that bench) → not shipped.
## Why the rest is stuck (diagnosis)
- ntok 1: gr_down_q8 (~23.2 µs for 3.48 MB) and gr_up_q8 (~23.8 µs, 3.48 MB) are both at ~150 GB/s = M5 DRAM roofline.- ntok 4 gr_down_q8_4_4 (40.6 µs): non-bitwise probes — removing the xn loads → 25.8 µs, removing the weight loads → ~26-31 µs, a fully coalesced (wrong-math) xn pattern → no change. It is the sum of DRAM weights + L2 xn re-reads (1 row/threadgroup × ntok × 40 KB), and `_r2` (half the xn traffic) is no faster at ntok 4 → latency/occupancy bound. Real gains need a different row→threadgroup mapping / grid, i.e. host changes in `window.cpp` (out of my file scope).123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566# round 33 track b: decode qsa attention and top-k
## decision: no ship
none of six candidates meets the predeclared acceptance rule: bitwise parity plus a reproducible >=3% attention improvement without material shape regressions. production files were never edited. `qsa_score` was unchanged; this experiment evaluated the proposed attention and top-k changes only.
`/tmp/sol_r33_qsa/sol_r33_qsa.patch` is intentionally empty. it passes `git -C vendor/strata apply --check --allow-empty /tmp/sol_r33_qsa/sol_r33_qsa.patch`; ordinary `apply --check` requires a nonempty patch and is not applicable to a no-change outcome. do not apply the separately labelled rejected patches.
## setup and contract
- local apple m5, as reported by qsa_parity. live server and :8080 untouched; no model benchmarks.- baseline vendor head: `296b59d2f73c8f04a805df29c2fa97655adace78`, plus current working-tree kernel snapshots captured before edits.- immutable baseline: `/tmp/sol_r33_qsa/reference/`.- qsa.metal sha256: `d2d3f311030eae507ab83148afe267b7746a15ac2231aa50aa0a22cc4403db4c`- qsa2.metal sha256: `1e1937f95e2224ce09944c8cb549af82edc02a0ab38a1cceeb203f4f7b58d59b`- existing `vendor/strata/build-metal/qsa_parity` executable; no evaluator edits or rebuild needed.- kernels selected through `STRATA_METAL_KERNELS=/tmp/sol_r33_qsa/strata/metal/kernels`; attention selected with `QSA_ATTN=1` or `2`.- each benchmark invocation ran serially, normally <0.4 s; broad bitwise invocations each ~1.3 s. no invocation exceeded 10 s.- timing method: existing paired benchmark, alternating order, median of nine rounds, 16 repeated dispatches; random bitwise mode uses 32 repeated dispatches.- source snapshots and logs for every explored candidate retained under `/tmp/sol_r33_qsa/`. scratch kernels restored to baseline after experiments.
## candidates
all candidates include the proposed final top-k pass cache (`k_local[4]`, fallback for `per > 4`):
1. `allscale`: precomputed row index, half4 key/value scales preloaded before existing barrier; v1 value pointer/scale offset hoisted.2. `vsonly`: as above but preload only value scales.3. `rowonly`: precompute row index, no scale staging.4. `byteoffset`: store row byte offset rather than row index, no scale staging.5. `ksonly`: precomputed row index plus key scale staging only.6. `alias` (parent-requested follow-up): value scales alias dead `qs` storage after the post-q.k barrier, populated before the existing post-softmax barrier. no extra threadgroup storage or barriers. passes paired pos0=3000 ntok=1/2/4/8 on both versions, but attn1 still regresses (n=4: 162.943 / 165.719 us; n=8: 320.544 / 326.474 us), and attn2 regresses at n=8 (379.219 / 381.766 us). this rules out added threadgroup storage as the sole explanation of this regression; no profiler evidence establishes a precise cause.
## correctness
- `allscale`: all 24 paired cases passed full-buffer memcmp (13 buffers), covering both attention versions, pos0=37/2048/3000, ntok=1/2/4/8.- `allscale`: both random-input bitwise suites passed: **0/40 differences for each version, 80/80 total**. these cover pos0=0,1,63,64,512,2047,2048,2050,8192,30000 and ntok=1/2/4/8; both partial attention and merged output are compared by memcmp.- other candidates: paired pos0=3000 at ntok=1/4/8 passed for both versions. no equivalence claim beyond these tested shapes.- final-cache `per > 4` path was not independently exercised for topk; no topk change ships.
## measured results
representative confirmation at pos0=3000; microseconds, reference / candidate:
| candidate | kernel | ntok=1 | ntok=4 | ntok=8 ||---|---|---:|---:|---:|| allscale | attn1 | 46.659 / 47.404 | 163.211 / 166.000 | 321.250 / 326.747 || allscale | attn2 | 50.935 / 50.180 | 193.630 / 191.810 | 380.570 / 380.727 || byteoffset | attn1 | inconclusive clock transition | 162.992 / 162.130 | 322.190 / 320.750 || byteoffset | attn2 | 50.802 / 50.852 | 193.391 / 192.997 | 379.760 / 378.534 |
allscale consistently makes attn1 ~1.6–1.8% slower; attn2 gains ~1–2% at small windows but disappear at ntok=8. value-only staging behaves similarly. key-only staging does not establish an improvement. row-only and byte-offset changes yield roughly 0–0.6% gains on stable longer-window measurements, below the planned selection threshold.
top-k differences are small and inconsistent, with occasional much larger apparent clock/thermal transitions affecting both reference and unchanged stages. for example allscale confirmation attn1 topk at n=1/4/8 was 8.250/8.156, 13.677/13.687, 8.289/8.086 us; initial same-shape results did not establish a repeatable win. do not infer a top-k speedup from isolated favorable samples. the baseline same-versus-same smoke was effectively equal (attn1 162.992 / 162.904 us at n=4).
## reproduction and evidence
```STRATA_METAL_KERNELS=/tmp/sol_r33_qsa/strata/metal/kernels QSA_ATTN=1 \ vendor/strata/build-metal/qsa_parity --paired /tmp/sol_r33_qsa/reference 3000 --tokens 4STRATA_METAL_KERNELS=/tmp/sol_r33_qsa/strata/metal/kernels QSA_ATTN=2 \ vendor/strata/build-metal/qsa_parity --bitwise /tmp/sol_r33_qsa/reference/qsa2.metal```
copy the two kernels from the named candidate snapshot (`allscale/`, `byteoffset/`, etc.) into scratch `strata/metal/kernels/` first; the final scratch tree deliberately equals baseline. logs: `allscale_av*_p*_n*.log`, `bitwise_av*.log`, `confirm_*.log`, and candidate-specific logs. rejected diffs: `*.rejected.patch`. experiment contract: `plan.md`.
this is a valid negative performance result, not a failed implementation: the suggested transformations preserve tested outputs, but do not justify a production change on this m5.