# round 12 track c: `gemm_f16` 2d l2/slc tile swizzling & `gemm_hc_up_reg` tile + epilogue optimization

- **patch:** `/tmp/argon_r12_gemm/argon_r12_gemm.patch` (`git -C vendor/strata apply --check` exits 0; touches `metal/kernels/prompt.metal` and `metal/src/engine/window.cpp` only)
- **scratch checkout & standalone benchmarks:** `/tmp/argon_r12_gemm/strata` (`build-metal`), `/tmp/argon_r12_gemm/bench_r12.cpp`, `/tmp/argon_r12_gemm/bench_swz2.cpp`, `/tmp/argon_r12_gemm/bench_swz_overhead.cpp`, `/tmp/argon_r12_gemm/bench_epi_ab.cpp`, `/tmp/argon_r12_gemm/window.o`

---

## 1. executive summary & decision

1. **target 1 (`gemm_f16` 2D L2/SLC tile swizzling in `metal/kernels/dense_mm.metal`): REJECTED (0 changes to `dense_mm.metal`).**
   - All tested 2D threadgroup swizzle mappings (`GROUP_M = 2, 4, 8, 16`, `GROUP_N = 2, 4, 8`, and pure grid transpose `my = tg.x, nx = tg.y`) are **100% bitwise identical (`memcmp == 0`)** across all 12 production `(N, K)` shapes and all `M = 256, 512, 1024, 1983, 2048`.
   - However, on Apple M5 (`64_128_4`, `128_64_4`, `32_128_2` selected in Round 11), the hardware threadgroup walker already schedules 2D `(tg.x, tg.y)` grids across the 10 GPU cores with strong L2/SLC locality. Any software swizzle either ties within `< 0.2%` noise (`GN4`, `GN8`, `GM2` at `M = 2048`) or **severely degrades M5's hardware L2/SLC prefetching** on `64_128_4` at `M = 1983, 2048` (e.g., pure transpose slows the 12-shape chunk from `978.2 ms` to `1,966.1 ms`, `GM16` slows it to `1,302.6 ms`, `GM8` slows it to `1,013.5 ms`).
   - Even when compiled as zero-argument compile-time template constants (`bench_swz_overhead`), no swizzle variant delivers a repeatable speedup across `M = 256..2048`. Following the falsifier rule ("Keep ONLY what delivers a clean, repeatable speedup across the production shapes"), `dense_mm.metal` is left untouched.

2. **target 2 (`gemm_hc_up_reg` tile shape & epilogue optimization in `metal/kernels/prompt.metal` + `metal/src/engine/window.cpp`): SHIPPED (`+6.9%` to `+7.8%` faster, saving `7.3–7.9 ms/chunk` at `T = 1983..2048`).**
   - **Epilogue Unroll & Pointer Hoisting (`prompt.metal`):** Added `#pragma clang loop unroll(full)` to the outer cooperative-tensor loop (`i += 4`) and inner 4-stream reduction loop (`c = 0..3`), hoisted `device const half* xnh_t = xnh + ulong(t) * 4 * n + d;` and `const ulong out_idx = ulong(t) * n + d;`, and replaced `s = s / 4.0f` with `s *= 0.25f`. Isolated paired A/B testing (`11` interleaved rounds) shows the epilogue optimization alone yields a consistent **1.008x–1.019x (`~1.0–1.9%`)** speedup across every tile (`64_64_4`, `64_128_4`, `32_128_4`) and every `T`, while remaining **100% bitwise identical (`memcmp == 0`)** on both `mixh` (`fp16`) and `mixed` (`fp32`).
   - **Tile Shape Sweep (`32_128_4` across all `T` on M5 `k4`):** Sweeping 10 register tile instantiations (`64_64_4`, `64_128_4`, `128_64_4`, `64_32_4`, `32_128_2`, `32_128_4`, `64_128_2`, `32_256_4`, `64_256_4`, `32_64_2`, `64_64_2`, `16_128_4`, `16_128_2`, `32_64_4`, `16_64_2`) revealed that **`gemm_hc_up_reg_32_128_4`** (`TM = 32, TN = 128, SG = 4`, `0` bytes threadgroup memory) is the fastest tile across **every single `T` from `1` to `2048`**:
     - Why `32_128_4` wins: Halving `TM` from `64` to `32` while keeping `TN = 128, SG = 4` cuts each thread's cooperative-tensor register footprint in half (`acc.get_capacity()` drops from `64` floats to `32` floats per lane = `8` outputs x `4` streams). Lower register pressure boosts M5 occupancy while keeping 4 simdgroups cooperating across the `K = 320` reduction and 32 coalesced output columns per threadgroup. By contrast, `SG = 2` variants (`32_128_2`, `64_128_2`) increase per-lane accumulator capacity or halve K-reduction parallelism.
     - Furthermore, at `T = 1983`, `TM = 64` pads to `31 * 64 = 1984` (`1` wasted row), whereas at common non-multiple-of-64 lengths `TM = 32` also reduces tail padding.
   - Combined (`gemm_hc_up_reg_32_128_4` + unrolled/hoisted epilogue), the 96-call per-chunk time drops from **`112.39 ms -> 105.10 ms` (`1.069x`, `-7.29 ms/chunk`)** at `T = 1983` and **`116.52 ms -> 108.76 ms` (`1.071x`, `-7.76 ms/chunk`)** at `T = 2048`.

---

## 2. target 1 evidence: `gemm_f16` 2d l2/slc tile swizzling

### 2.1 tested swizzle mappings (`bench_swz_overhead.cpp` & `bench_swz2.cpp`)
All variants were compiled as zero-overhead template specializations (`orig`, `GM4`, `GM8`, `GM16`, `GN4`, `GN8`, `Transp`) and verified against `orig` (`memcmp == 0` on `0xFF`-poisoned output buffer `Y` across all 12 production shapes at `M = 256, 512, 1024, 1983, 2048`, `fails = 0`).

### 2.2 paired 12-shape chunk totals on quiet M5 (`7` interleaved rounds per shape)

| `M` | `orig` (ms/chunk) | `GM4` (ms/chunk) | `GM8` (ms/chunk) | `GM16` (ms/chunk) | `GN4` (ms/chunk) | `GN8` (ms/chunk) | `Transp` (`M`-major, ms/chunk) | Bitwise `memcmp == 0` |
|---:|---:|---:|---:|---:|---:|---:|---:|:---:|
| `256` | **192.05** | 192.40 | 192.63 | 192.90 | 191.97 | 192.76 | 192.13 | PASS (`0` fails) |
| `512` | **251.60** | 250.60 | 253.35 | 250.84 | 249.88 | 251.14 | 252.23 | PASS (`0` fails) |
| `1024` | **512.02** | 515.03 | 523.92 | 511.98 | 510.58 | 512.69 | 512.60 | PASS (`0` fails) |
| `1983` | **978.22** | 981.79 | 1013.48 | 1302.59 | 976.02 | 976.60 | 1966.07 | PASS (`0` fails) |
| `2048` | **1011.13** | 1004.93 | 1024.29 | 1652.16 | 1011.77 | 1012.36 | 1856.75 | PASS (`0` fails) |

### 2.3 per-shape breakdown at `M = 1983` (us/call, median of 7 interleaved rounds)

| Shape | `N x K` | Calls | Tile | `orig` (us) | `GM4` (us) | `GM8` (us) | `GM16` (us) | `GN4` (us) | `GN8` (us) | `Transp` (us) |
|---|---|---:|---|---:|---:|---:|---:|---:|---:|---:|
| `hc_read` | `324 x 10240` | 96 | `32_128_2` | 1164.0 | 1160.6 | 1172.6 | 1170.2 | 1161.6 | 1164.5 | 1161.9 |
| `Wqkv` | `10240 x 2560` | 37 | `64_128_4` | **7200.0** | 7223.1 | 7446.8 | 8167.7 | 7191.7 | 7198.1 | 17324.6 |
| `Wz` | `6144 x 2560` | 36 | `64_128_4` | **4329.4** | 4345.2 | 4444.9 | 4874.5 | 4329.7 | 4325.7 | 9248.7 |
| `Wout` | `2560 x 6144` | 48 | `64_128_4` | **4450.2** | 4439.6 | 4753.0 | 9708.2 | 4408.6 | 4414.2 | 10608.4 |
| `Wq` | `12288 x 2560` | 12 | `64_128_4` | **8627.3** | 8682.5 | 8913.1 | 9759.4 | 8638.0 | 8635.9 | 20167.9 |
| `Wpv` | `2560 x 2560` | 1 | `64_128_4` | **1806.3** | 1821.4 | 1897.5 | 2640.2 | 1807.1 | 1817.4 | 2226.2 |
| `Wshgu` | `640 x 2560` | 96 | `64_128_4` | **468.2** | 478.8 | 487.8 | 479.6 | 467.2 | 467.6 | 474.5 |
| `Wshd` | `2560 x 640` | 48 | `64_128_4` | **458.6** | 460.8 | 462.4 | 460.9 | 458.6 | 460.0 | 460.7 |
| `W512` | `512 x 2560` | 119 | `32_128_2` | **401.7** | 410.8 | 409.6 | 404.3 | 403.7 | 402.7 | 410.5 |
| `Wik` | `128 x 2560` | 12 | `32_64_2` | **105.0** | 106.0 | 106.3 | 104.3 | 105.5 | 104.2 | 105.7 |
| `Walbe` | `48 x 2560` | 72 | `64_32_2` | **80.9** | 82.0 | 81.8 | 81.4 | 81.4 | 81.3 | 82.3 |
| `Wsg` | `1 x 2560` | 48 | `64_32_2` | **71.4** | 71.5 | 71.6 | 71.5 | 71.4 | 71.4 | 71.3 |

**Analysis:** On M5, `gemm_f16_64_128_4` reads `64` rows of `A` (`320 KB` at `K = 2560`) and `128` rows of `B` (`640 KB` at `K = 2560`) per threadgroup. When Metal dispatches `(tg.x, tg.y)` in its native order, simultaneously active threadgroups across the 10 GPU cores share the `320 KB` `mA` slice in L2 while streaming consecutive `mB` slices sequentially. Remapping `tg` into `M`-major strips (`GM >= 8` or `Transp`) forces concurrent threadgroups to read disjoint `mA` rows (`31 * 320 KB = 9.9 MB`) while hammering the same `mB` slice, thrashing L2 and causing up to a `2.4x` slowdown on `Wqkv`/`Wout`/`Wq`.

---

## 3. target 2 evidence: `gemm_hc_up_reg` tile sweep & epilogue optimization

### 3.1 full tile + swizzle sweep (`bench_r12.cpp` & `bench_hcup2.cpp`, 96 calls/chunk, unbound `mixed`)

All candidates verified `memcmp == 0` on both `mixh` (`fp16`) and `mixed` (`fp32`, when bound) across `T = 1, 4, 7, 8, 16, 24, 32, 48, 64, 128, 256, 384, 512, 768, 1024, 1536, 1983, 2048` (`0` failures).

| Candidate (`N = 10240, K = 320`, 96 calls) | `T = 256` (ms) | `T = 512` (ms) | `T = 1024` (ms) | `T = 1983` (ms) | `T = 2048` (ms) |
|---|---:|---:|---:|---:|---:|
| `gemm_hc_up_reg_64_64_4` (old `T < 1024`) | 15.55 | 30.22 | 60.84 | 116.95 | 125.29 |
| `gemm_hc_up_reg_64_128_4` (old `T >= 1024`) | 16.39 | 30.61 | 58.92 | 113.47 | 123.54 |
| `gemm_hc_up_reg_32_128_2` (opt epilogue) | 17.12 | 31.78 | 61.36 | 117.49 | 126.48 |
| **`gemm_hc_up_reg_32_128_4` (opt epilogue, SHIPPED)** | **14.82** | **28.41** | **55.22** | **106.65** | **115.31** |
| `gemm_hc_up_reg_64_128_2` (opt epilogue) | 32.80 | 67.52 | 134.08 | 258.16 | 267.07 |
| `gemm_hc_up_reg_32_256_4` (opt epilogue) | 17.09 | 31.05 | 59.58 | 114.06 | 123.90 |
| `gemm_hc_up_reg_64_256_4` (opt epilogue) | 35.40 | 71.75 | 140.89 | 266.12 | 277.68 |
| `gemm_hc_up_reg_32_64_2` (opt epilogue) | 15.93 | 31.29 | 62.00 | 120.51 | 125.26 |
| `gemm_hc_up_reg_64_64_2` (opt epilogue) | 19.27 | 36.55 | 70.58 | 135.13 | 149.28 |
| `gemm_hc_up_reg_16_128_4` (opt epilogue) | 16.95 | 33.08 | 62.82 | 118.92 | 122.81 |
| `gemm_hc_up_reg_32_64_4` (opt epilogue) | 17.48 | 34.72 | 71.65 | 138.51 | 143.11 |

### 3.2 isolated paired A/B: baseline epilogue vs unrolled/hoisted epilogue (`bench_epi_ab.cpp`, 11 interleaved rounds, 96 calls/chunk)

| `T` | `mixed` | `64_64_4` base -> opt (ms) | `64_128_4` base -> opt (ms) | `32_128_4` base -> opt (ms) | Old Shipped -> New Shipped (`opt_32_128_4`) | Saved (ms/chunk) |
|---:|---|---:|---:|---:|---:|---:|
| `16` | unbound | `4.67 -> 4.64` (`1.006x`) | `5.39 -> 5.42` (`0.994x`) | `4.29 -> 4.29` (`1.000x`) | `4.67 -> 4.29 ms` (**1.089x**) | `-0.38 ms` |
| `64` | unbound | `5.18 -> 5.16` (`1.004x`) | `6.19 -> 6.16` (`1.005x`) | `5.01 -> 4.99` (`1.004x`) | `5.18 -> 4.99 ms` (**1.039x**) | `-0.19 ms` |
| `256` | unbound | `15.40 -> 15.28` (`1.008x`) | `16.11 -> 15.91` (`1.013x`) | `14.60 -> 14.46` (`1.009x`) | `15.40 -> 14.46 ms` (**1.065x**) | `-0.94 ms` |
| `256` | bound | `16.17 -> 16.02` (`1.009x`) | `16.71 -> 16.59` (`1.007x`) | `15.24 -> 15.09` (`1.010x`) | `16.17 -> 15.09 ms` (**1.072x**) | `-1.08 ms` |
| `512` | unbound | `30.09 -> 29.84` (`1.008x`) | `30.34 -> 30.03` (`1.011x`) | `28.44 -> 28.15` (`1.010x`) | `30.09 -> 28.15 ms` (**1.069x**) | `-1.94 ms` |
| `512` | bound | `31.45 -> 31.15` (`1.010x`) | `31.27 -> 30.98` (`1.009x`) | `29.52 -> 29.18` (`1.011x`) | `31.45 -> 29.18 ms` (**1.078x**) | `-2.27 ms` |
| `1024` | unbound | `60.22 -> 59.46` (`1.013x`) | `58.59 -> 57.94` (`1.011x`) | `55.36 -> 54.79` (`1.010x`) | `58.59 -> 54.79 ms` (**1.069x**) | `-3.80 ms` |
| `1024` | bound | `62.31 -> 61.73` (`1.009x`) | `60.11 -> 59.46` (`1.011x`) | `56.97 -> 56.41` (`1.010x`) | `60.11 -> 56.41 ms` (**1.066x**) | `-3.70 ms` |
| `1983` | unbound | `117.70 -> 115.56` (`1.019x`) | `112.39 -> 111.05` (`1.012x`) | `106.18 -> 105.10` (`1.010x`) | `112.39 -> 105.10 ms` (**1.069x**) | **`-7.29 ms`** |
| `1983` | bound | `120.43 -> 118.95` (`1.012x`) | `115.50 -> 114.34` (`1.010x`) | `109.08 -> 108.00` (`1.010x`) | `115.50 -> 108.00 ms` (**1.069x**) | **`-7.50 ms`** |
| `2048` | unbound | `120.95 -> 119.88` (`1.009x`) | `116.52 -> 115.16` (`1.012x`) | `109.82 -> 108.76` (`1.010x`) | `116.52 -> 108.76 ms` (**1.071x**) | **`-7.76 ms`** |
| `2048` | bound | `125.55 -> 124.51` (`1.008x`) | `119.80 -> 118.50` (`1.011x`) | `112.86 -> 111.80` (`1.009x`) | `119.80 -> 111.80 ms` (**1.072x**) | **`-8.00 ms`** |

---

## 4. verification & parity results

1. **`dense_mm_parity`:**
   - `/tmp/argon_r12_gemm/strata/build-metal/dense_mm_parity`: **ALL PASS** (`3.05s`, all 13 quant formats x 6 shapes, `memcmp(wh)=0, memcmp(xh)=0`).
   - `/tmp/argon_r12_gemm/strata/build-metal/dense_mm_parity --tiles M` (`M = 256, 512, 1024`) and `--ab M` (`M = 1983, 2048`): **ALL BITWISE** (`< 3s` per invocation).
2. **`comb_fold_parity`:**
   - `/tmp/argon_r12_gemm/strata/build-metal/comb_fold_parity 64`: **ALL PASS** (`4.67s`, `T = 1, 7, 64`).
   - `/tmp/argon_r12_gemm/strata/build-metal/comb_fold_parity T hcup confirm` (`T = 64, 512, 1024, 1983, 2048`): **ALL PASS** (`< 2.1s` per invocation, `tgmem = 0` on all `reg` kernels).
3. **Engine compilation & patch check:**
   - `window.cpp` compiled to `/tmp/argon_r12_gemm/window.o` using `strata_metal_kernels` release flags: **exit 0** (`0` warnings/errors).
   - `git -C vendor/strata apply --check /tmp/argon_r12_gemm/argon_r12_gemm.patch`: **exit 0**.