# opus_r33_gr — Round 33 Track A: hyper-connection read (`gr_norm`, `gr_norm_comb`, `gr_down_f`, `gr_up_f`)

Patch: `/tmp/opus_r33_gr/opus_r33_gr.patch` (`git -C vendor/strata apply --check` OK; touches only `metal/kernels/gr.metal`
and `metal/tests/fuse_parity.cpp`). Scratch tree: `/tmp/opus_r33_gr/strata`; baseline kernels: `/tmp/opus_r33_gr/base_kernels`;
raw final logs: `/tmp/opus_r33_gr/final_bench.txt`. Device: Apple M5 (10-core GPU), GPU quiet for the final run.

## Shipped (all bitwise PASS, ntok 1..8, every gr.metal/gr2.metal entry point incl. BF16/Q4/Q5/F32-inject; default `fuse_parity` ALL PASS)

1. **`gr_norm`**: `apply` branch hoisted out of the 10-step unrolled loop, pointers pre-offset (`rp = r + tid`,
   `bp = bo + t*2560 + tid`, `o += tid; w += tid`) so every access is an immediate `[i * 256]`.
   **-6..-11% at ntok 1..4, -2..-3% at 7..8** (~0.2 µs per dispatch; gr_norm is ~2.3-4.3 µs, mostly launch-bound).
2. **`grd_acc` Q8_0, NR = 2 (`gr_down_q8_*_r2`, `gr_down_q8f_*_r2`)**: a `GrdUnit<F, NR>` holder with a
   `GrdUnit<T_Q8_0, 2>` specialization that keeps the codes packed as `char4[8]` per row (8 regs instead of 32 floats)
   and widens with `float4(char4)` per token. Same fmas in the same order, so bitwise identical. The generic path is
   unchanged codegen (all other formats/NR = 1: ±0.5%, noise).
   **ntok 7: -13..-16%, ntok 8: -10..-15%**; ntok 5-6 neutral; ntok ≤ 4 never uses `_r2` in the engine.
3. **`fuse_parity --gr-unroll ... bench`**: 16 pairs × 128 iters, reports the median pair ratio (A/A noise went from
   ±15% to ±0.3% for gr_down/gr_up, ~±2% for the tiny gr_norm_comb); gr_norm/gr_norm_comb now timed; added same-entry
   A/B benches for the engine's real entry points (`gr_down_q8_4_4`, `_4_4_r2`, `q8f_4_4(_r2)`, `q8_8_8(_r2)`).

## Final paired timings (median Δ, quiet GPU)

| kernel | ntok | base µs | new µs | median Δ |
|---|---|---|---|---|
| gr_down_q8_4_4 | 1 | 23.213 | 23.210 | +0.0% |
| gr_down_q8_4_4 | 2 | 23.836 | 23.831 | +0.0% |
| gr_down_q8_4_4 | 3 | 31.574 | 31.606 | +0.1% |
| gr_down_q8_4_4 | 4 | 40.602 | 40.610 | +0.0% |
| gr_down_q8_4_4 | 5 | 55.545 | 55.580 | +0.0% |
| gr_down_q8_4_4 | 6 | 75.455 | 75.441 | +0.0% |
| gr_down_q8_4_4 | 7 | 97.495 | 97.445 | -0.0% |
| gr_down_q8_4_4 | 8 | 119.752 | 119.762 | -0.0% |
| gr_down_q8_4_4_r2 | 1 | 23.348 | 23.443 | +0.4% |
| gr_down_q8_4_4_r2 | 2 | 23.814 | 23.786 | -0.2% |
| gr_down_q8_4_4_r2 | 3 | 32.917 | 32.977 | +0.2% |
| gr_down_q8_4_4_r2 | 4 | 41.266 | 41.331 | +0.1% |
| gr_down_q8_4_4_r2 | 5 | 49.493 | 49.428 | -0.1% |
| gr_down_q8_4_4_r2 | 6 | 58.338 | 58.370 | +0.1% |
| gr_down_q8_4_4_r2 | 7 | 79.517 | 68.066 | -14.4% |
| gr_down_q8_4_4_r2 | 8 | 90.285 | 79.752 | -11.7% |
| gr_down_q8_8_8 | 1 | 23.584 | 23.729 | +0.6% |
| gr_down_q8_8_8 | 2 | 24.839 | 24.843 | -0.0% |
| gr_down_q8_8_8 | 3 | 34.813 | 34.802 | -0.1% |
| gr_down_q8_8_8 | 4 | 45.466 | 45.464 | -0.0% |
| gr_down_q8_8_8 | 5 | 62.634 | 62.615 | -0.0% |
| gr_down_q8_8_8 | 6 | 83.515 | 83.568 | +0.1% |
| gr_down_q8_8_8 | 7 | 115.168 | 115.243 | +0.0% |
| gr_down_q8_8_8 | 8 | 140.286 | 140.303 | +0.1% |
| gr_down_q8_8_8_r2 | 1 | 23.953 | 24.036 | +0.3% |
| gr_down_q8_8_8_r2 | 2 | 25.738 | 25.852 | +0.5% |
| gr_down_q8_8_8_r2 | 3 | 34.776 | 34.844 | +0.2% |
| gr_down_q8_8_8_r2 | 4 | 46.604 | 46.623 | +0.0% |
| gr_down_q8_8_8_r2 | 5 | 58.045 | 58.184 | +0.3% |
| gr_down_q8_8_8_r2 | 6 | 70.499 | 70.514 | -0.1% |
| gr_down_q8_8_8_r2 | 7 | 100.167 | 83.844 | -16.3% |
| gr_down_q8_8_8_r2 | 8 | 114.260 | 97.570 | -14.6% |
| gr_down_q8f_4_4_r2 | 1 | 24.422 | 24.454 | +0.1% |
| gr_down_q8f_4_4_r2 | 2 | 24.889 | 24.863 | -0.4% |
| gr_down_q8f_4_4_r2 | 3 | 33.414 | 33.478 | +0.2% |
| gr_down_q8f_4_4_r2 | 4 | 41.238 | 41.304 | +0.2% |
| gr_down_q8f_4_4_r2 | 5 | 49.481 | 49.448 | -0.1% |
| gr_down_q8f_4_4_r2 | 6 | 58.534 | 58.637 | +0.2% |
| gr_down_q8f_4_4_r2 | 7 | 80.516 | 70.128 | -12.9% |
| gr_down_q8f_4_4_r2 | 8 | 91.064 | 81.766 | -10.2% |
| gr_norm apply 0 | 1 | 3.241 | 2.882 | -11.1% |
| gr_norm apply 0 | 2 | 2.478 | 2.227 | -10.4% |
| gr_norm apply 0 | 3 | 2.506 | 2.269 | -9.2% |
| gr_norm apply 0 | 4 | 2.578 | 2.400 | -7.1% |
| gr_norm apply 0 | 5 | 2.668 | 2.523 | -5.5% |
| gr_norm apply 0 | 6 | 3.039 | 2.849 | -6.4% |
| gr_norm apply 0 | 7 | 3.044 | 2.958 | -2.8% |
| gr_norm apply 0 | 8 | 3.185 | 3.084 | -3.3% |
| gr_norm apply 1 | 1 | 3.293 | 3.088 | -6.1% |
| gr_norm apply 1 | 2 | 3.347 | 3.096 | -7.7% |
| gr_norm apply 1 | 3 | 3.361 | 3.160 | -6.2% |
| gr_norm apply 1 | 4 | 3.468 | 3.256 | -5.9% |
| gr_norm apply 1 | 5 | 3.620 | 3.428 | -5.4% |
| gr_norm apply 1 | 6 | 3.834 | 3.763 | -1.8% |
| gr_norm apply 1 | 7 | 4.069 | 3.980 | -2.2% |
| gr_norm apply 1 | 8 | 4.346 | 4.255 | -2.2% |
| gr_norm_comb | 1 | 5.135 | 5.047 | -1.4% |
| gr_norm_comb | 2 | 5.335 | 5.363 | +0.7% |
| gr_norm_comb | 3 | 6.176 | 6.248 | +1.3% |
| gr_norm_comb | 4 | 6.754 | 6.678 | -1.0% |
| gr_norm_comb | 5 | 6.985 | 6.874 | -1.5% |
| gr_norm_comb | 6 | 8.941 | 8.889 | -0.7% |
| gr_norm_comb | 7 | 9.091 | 9.133 | +0.5% |
| gr_norm_comb | 8 | 10.624 | 10.563 | -0.7% |
| gr_up_q8 | 1 | 23.770 | 23.762 | -0.0% |
| gr_up_q8 | 2 | 24.547 | 24.535 | -0.1% |
| gr_up_q8 | 3 | 26.172 | 26.186 | +0.1% |
| gr_up_q8 | 4 | 29.213 | 29.184 | -0.1% |
| gr_up_q8 | 5 | 32.855 | 32.889 | +0.1% |
| gr_up_q8 | 6 | 37.063 | 37.025 | -0.1% |
| gr_up_q8 | 7 | 42.019 | 41.957 | -0.1% |
| gr_up_q8 | 8 | 47.111 | 47.046 | -0.1% |

## Tried and rejected (measured, bitwise-safe but not wins)

- `deq<T_Q8_0>` `#pragma unroll(full)`: 0.0% (the compiler already unrolls it). Not shipped, quant.metal untouched.
- Packed Q8 for **NR = 1** gr_down: +1..3% at ntok 2 (`q8_8_8` +3.2%), neutral elsewhere → restricted to NR = 2.
- Packed Q8 + always-unrolled token loop at ntok 7-8: +12..45% (registers). Rolled-from-ntok-5 for NR = 2:
  `4_4_r2` -3% at ntok 5 but `8_8_r2` +0.4..1%; kept the existing ≤ 6 threshold.
- `unroll_count(3)` on the gr_down `u` loop: +2..42% everywhere. Software-pipelined prefetch of the next Q8 block: +3..53%.
- Aligned 9×`uint` Q8 block load + funnel shift instead of 16 `ushort` loads: +25..66% (much worse).
- `gr_up_q8`: staging `lo` in `threadgroup float[8*320]`: +1..6% (lo reads are already L1 broadcast hits);
  packed Q8 dot: +2..8% at ntok 4-8; `unroll_count(3)`: +4..30%.
- `gr_norm_comb` apply-hoist + `gs` guard: -0.3..-0.8% vs repeated A/A (within the ~1-2% order bias of that bench) → not shipped.

## Why the rest is stuck (diagnosis)

- ntok 1: gr_down_q8 (~23.2 µs for 3.48 MB) and gr_up_q8 (~23.8 µs, 3.48 MB) are both at ~150 GB/s = M5 DRAM roofline.
- ntok 4 gr_down_q8_4_4 (40.6 µs): non-bitwise probes — removing the xn loads → 25.8 µs, removing the weight loads →
  ~26-31 µs, a fully coalesced (wrong-math) xn pattern → no change. It is the sum of DRAM weights + L2 xn re-reads
  (1 row/threadgroup × ntok × 40 KB), and `_r2` (half the xn traffic) is no faster at ntok 4 → latency/occupancy bound.
  Real gains need a different row→threadgroup mapping / grid, i.e. host changes in `window.cpp` (out of my file scope).