# round 29 track b: paired moe gate/up decode

## result

- patch: `/tmp/sol_r29_moemm/sol_r29_moemm.patch` (sha256 `0504a66525ab48e1a297ec9d4b7c39018de9f43c5fe4ccee566f1fa224c09645`). `git -C vendor/strata apply --check` exits 0.
- isolated source: `/tmp/sol_r29_moemm/strata`; baseline kernels: `/tmp/sol_r29_moemm/baseline`. vendor head at snapshot: `296b59d2f73c8f04a805df29c2fa97655adace78`; snapshot includes its then-current working files, not just head.
- all four requested formats at all three requested token counts passed **HS_memcmp=PASS YE_memcmp=PASS Y_memcmp=PASS** on final source, exit 0.
- all invocations were individually under 10 seconds (final maximum 9.768 seconds). no server calls or full-model benchmarks.
- iq2_s scale caching has a reproducible positive signal (~2–5% gate/up latency reduction). iq2_xxs direct byte loads have a ~2–3% preliminary signal. iq3_s fma and table trim alone did not establish a speed gain.
- later runs became heavily contended/noisy (unchanged iq2_xs also varied dramatically). do not interpret final noisy non-iq2_s medians as clean regressions or wins; confirm in a quiet gpu window before promoting.

## implementation

only owned lines/files changed:

1. iq2_s table: 2080 -> 2048 uints. loader writes exactly 2048, and index max is 2047. source-level static allocation drops from 16512 to 16384 bytes for the selected tile; this alone did not measurably change time.
2. iq2_s paired decoder now has an in-block helper accepting the already-loaded scale. mm_gu3 caches gate/up row scales once per 256-weight block and reuses them across eight k32 matmuls, preserving accumulation order. cache is `TN * TK / 256` floats per operand/lane (four floats each for the target tile), no new threadgroup storage. existing wrapper retains full-row semantics for other callers. non-iq2_s keeps a single-iteration inner loop.
3. iq2_xxs: replace 32-bit grid-index read plus variable byte shifts with two byte reads. no changes to sign or scale math.
4. iq3_s: integer odd-scale construction -> equivalent fma.
5. standalone test: shared bitwise comparator used in both parity and isolated a/b bench paths; supports the requested `-T` alias and four-number down tile, retaining existing three-number form. mismatched down TM fails pre-gpu with exit 2 (tested). a/b timing remains the original alternating 19 paired samples and original filtering.

## evaluator / command

apple m5, existing `libmtl_rt.a`, clang++ c++20 -O2. standalone executable `/tmp/sol_r29_moemm/moe_mm_parity` built from scratch test source linked to `vendor/strata/build-metal/libmtl_rt.a` and metal/foundation frameworks. runtime uses the scratch kernels via `STRATA_METAL_KERNELS`:

```
STRATA_METAL_KERNELS=/tmp/sol_r29_moemm/strata/metal/kernels \
/tmp/sol_r29_moemm/moe_mm_parity --v3 --gu FORMAT --dn Q2_0 \
  --tile 16,32,32,4 --dtile 16,64,32,2 -T TOKENS \
  --ab /tmp/sol_r29_moemm/baseline --gu-bench
```

formats: IQ2_S / IQ2_XXS / IQ3_S / IQ2_XS; tokens: 512 / 1024 / 2048. seeded existing data generator; 512 experts. existing runtime fast compilation supplies MM_V3_ONLY. a = candidate, b = baseline. positive printed diff is `(b-a)/a`, **not** latency reduction `(b-a)/b`.

## earlier low-noise evidence

`logs/v1_*`: table trim + iq2_xxs byte loads + iq3_s fma, before scale caching:

| format | t512 a / b us | t1024 a / b us | t2048 a / b us |
|---|---:|---:|---:|
| IQ2_S | 10067 / 10062 | 16811 / 16860 | 28917 / 28492 |
| IQ2_XXS | 7813 / 8097 | 13109 / 13417 | 22609 / 23034 |
| IQ3_S | 10478 / 10463 | 16844 / 16807 | 28485 / 28364 |

`logs/v2_IQ2_S_*`: cached-scale source with a redundant min-bounded loop, subsequently simplified:

| t | candidate us | baseline us | latency reduction |
|---|---:|---:|---:|
| 512 | 9886.7 | 10105.5 | 2.17% |
| 1024 | 15990.6 | 16924.1 | 5.52% |
| 2048 | 27140.5 | 28495.9 | 4.76% |

an independent scale-cache 1024 run (`v2fix_IQ2_S_1024.txt`) gave 15992.5 / 16870.9 us (5.20% reduction). the min-bounded loop unexpectedly regressed iq3_s (~8–25%); replacing it with a fixed compile-time trip count removed that in `v3_IQ3_S_1024.txt`: 16860.9 / 16775.4 us (-0.51%, noise). this correction is in the patch.

## final exact-source matrix

all final logs: `/tmp/sol_r29_moemm/logs/final_FORMAT_T.txt`. all 12 have 3/3 memcmp passes. timing below is raw, including severe contention later in the matrix:

| format | t | candidate median us | baseline median us |
|---|---:|---:|---:|
| IQ2_S | 512 | 10022.3 | 10291.1 |
| IQ2_S | 1024 | 16276.1 | 16926.2 |
| IQ2_S | 2048 | 28069.6 | 28867.4 |
| IQ2_XXS | 512 | 11142.1 | 11032.1 |
| IQ2_XXS | 1024 | 23791.9 | 24683.8 |
| IQ2_XXS | 2048 | 49890.8 | 39101.6 |
| IQ3_S | 512 | 45668.4 | 41311.6 |
| IQ3_S | 1024 | 46026.5 | 44646.8 |
| IQ3_S | 2048 | 115838.9 | 89895.8 |
| IQ2_XS | 512 | 14022.5 | 13563.5 |
| IQ2_XS | 1024 | 22634.9 | 22979.8 |
| IQ2_XS | 2048 | 29303.8 | 28569.8 |

final iq2_s relative reductions: 2.61%, 3.84%, 2.76%. later samples show multi-fold slowdowns, including the unchanged iq2_xs control. raw medians must not be cherry-picked into claims.

## limitations / next check

bitwise correctness is established for requested formats/tile/token counts against frozen baseline, not an exhaustive format/tile proof. no full-model throughput estimate is claimed. quiet confirmation is needed for the final non-iq2_s timing matrix. source reviews used view_diff and view_code; no tracked vendor files were modified.