opus r11 dense: gemm_f16 tile table + fused shared-expert swiglu #

patch: /tmp/opus_r11_dense/opus_r11_dense.patch (git -C vendor/strata apply --check clean, 188 lines; touches metal/kernels/dense_mm.metal, metal/tests/dense_mm_parity.cpp, metal/src/engine/window.cpp 3301-3326 only). experimental/rejected: /tmp/opus_r11_dense/experimental_swiglu_and_26_tiles.patch (fused kernel + all 26 tile instantiations). raw logs: /tmp/opus_r11_dense/tiles/ (sweep M_shape*.txt, A/B final_ab_*, quiet_ab_*). scratch build: /tmp/opus_r11_dense/strata/build-metal (dense_mm_parity + strata_metal_kernels incl. window.cpp compile clean).

headline #

• every gemm_f16 tile is bitwise identical on M5. 26 tiles (TM 16..128, TN 32..256, SG 1..8) x 12 production (N, K) shapes x M = 256/512/1024/1983/2048: memcmp == 0 against 128_64_4 in all cases (0xFF-poisoned Y, so unwritten elements would also fail). MPP's K reduction order doesn't depend on the tile. So tile choice is free, speed only.
• new selector: 4 new instantiations (16_64_2, 32_64_2, 32_128_2, 64_32_2), selector rewritten for M > 128 (M <= 128 bit-for-bit the old choice):
  • N <= 64 (Wal/Wbe/Wsg): 16_64_2 for M < 1024, else 64_32_2 (old 64_32_4: 1.3-1.7x on Walbe)
  • N <= 128 (Wik): 16_64_2 for M < 768, else 32_64_2 (1.1-1.3x)
  • HC read (324 x 10240): old tile for M < 1536, 32_128_2 above (1.05-1.11x at 1983)
  • N <= 512 (Wr/Wr2/Wk/Wv/Wiq) and Wshg/Wshu below M 384: 32_64_2 for M < 384, 32_128_2 above (1.02-1.2x)
  • everything else (Wqkv, Wz, Wout, Wq, Wpv, Wshd, Wshg/u): 128_64_4 for M < 1536, 64_128_4 above. That's the old "bug #1" (64_128 only at M >= 2048). At M = 1983 128_64_4 falls off a cliff on Wout (K = 6144): 6.3-8.2 ms vs 4.4-4.7 ms in quiet runs (1.5-1.8x), and is 6-10% slower on the other big shapes. Below ~1792 the two tie (±1%), so the threshold is conservative at 1536.
• fused gemm_swiglu_f16: built as asked (two get_destination_cooperative_tensor accumulators, dynamic_length_v K, no threadgroup memory, no k0 loop, swiglu_h's exact expression in registers, direct half stores). Bitwise identical (memcmp == 0 on h) for all 10 tiles tried, but NOT faster: best fused 1016 us vs best unfused 976 us (2x gemm 64_64_2
  • swiglu_h) at M 1983. Two TMxTN accumulators double register pressure (fused 64_64_4: 1862 us). swiglu_h + the fp32 g/u traffic is only ~12-20 us/layer (<1 ms/chunk), so even a perfect fusion is capped at ~1 ms/chunk. Rejected, not in the patch. (One untested idea: dq Wshg/Wshu interleaved with perm=2 into one N=1280 B, then one op.run with the gemm_hc_up REG epilogue pairing adjacent columns. One accumulator, but the same ~1 ms ceiling. Not worth it.)
• Nn = 1 (Wsg): no dedicated row-dot. A hand dot can't match MPP's reduction bits, and at large M it's memory-bound on reading A (~85 us = A at ~120 GB/s). Tile changes give 1.3-1.45x at M <= 512, ~1.03x at 1983.

caveat: GPU contention #

A sibling agent was running moe_mm_parity (and the :8080 server was live) through most of the measurements, so absolute times drift by up to 40% between runs. Within a process the rounds interleave old and new, so the paired ratio is the reliable number. The first sweep (mostly quiet) gives the clearest absolute picture.

quiet-ish sweep, ms per 48-layer chunk (min of 1-2 runs/shape, 12 shapes x calls) #

final paired A/B, shipped selector (dense_mm_parity --ab M, 5 runs x 9 rounds, contended) #

(At 1024 the shipped selector is identical to old for all big shapes; the -0.9% there comes from the earlier 1024-threshold version and from noise. Rerun after the switch to 1536: 1024 1.01x, 1536 1.01x, 1983 1.05x.) The contended runs understate the 1983 win. In quieter runs the 1983 per-chunk saving is ~140-230 ms (1.13-1.19x), mostly Wout (1.5-1.8x) plus Wqkv/Wz/Wq/hc_read (1.06-1.11x).

per-shape paired A/B at 1024 and 1983 (pre-1536-threshold run; at 1024 the big shapes are now old=new) #

full sweep: best 3 tiles per shape (us/call, min over runs), all bitwise #

verification #

• dense_mm_parity (existing parity): ALL PASS with the patched kernels.
• dense_mm_parity --tiles M [shape] / --ab M [shape]: every run exits 0 with "ALL BITWISE" (memcmp vs 128_64_4). Each invocation runs < 5 s (max seen 4.2 s under contention, 1.5-2.7 s typically).
• window.cpp compile-checked via ninja strata_metal_kernels in the scratch tree. No full-model run (per the rules), so the end-to-end kprof gain still needs to be measured by the parent.
• Instantiation cost: 26 extra tiles added ~0.8-1 s to the runtime MSL compile, which is why only the 4 the selector uses ship.
• new_tile in the test mirrors window.cpp by hand. If one changes, change the other.