round 30 track a: mm_down3<Q2_0> decode + mm_gather_h sizing #

result #

• patch: /tmp/opus_r30_moemm/opus_r30_moemm.patch (sha256 befa6043a59aa31b7eff35d4d90bb1d0a85312dab3ef33d7a860b15a263d7d79), git -C vendor/strata apply --check exits 0. touches only W4<T_Q2_0> (moe_mm.metal 519-533) and the gather sweep in moe_mm_parity.cpp. window.cpp unchanged.
• ship: fp16 exponent-trick Q2_0 decode. down GEMM mm_down3<Q2_0,16,64,32,2> latency -10.5% at T=2048, -8.3% at T=1024, -2.3% at T=512; bonus: Q2_0 gate/up (--gu Q2_0 --gu-bench, T=2048) -8.1%.
• bitwise: HS/YE/Y memcmp PASS vs frozen baseline at T=512/1024/2048; plus an exhaustive GPU check of the new decode against the old fp32 one over all 65536 fp16 d (subnormals, ±0, inf, NaN included) × 256 code bytes: 0 / 16,777,216 mismatches (/tmp/opus_r30_moemm/q2chk/). CPU-reference parity (-T 16, gu Q2_0 and IQ2_S) PASS.
• the task's scale-hoisting idea (variants A/B/C) is not a win: the compiler already CSEs the repeated d load; hoisting across op.run loses to register pressure.
• mm_gather_h: no change. it's write-bandwidth bound; all threadgroup sizes are within ~1% of each other.

the change #

static inline half4 dec(uint q, half d) {
    const uint2 c = (uint2(q | (q << 14), (q >> 4) | (q << 10)) & 0x00030003u) | 0x64006400u;
    return (as_type<half4>(c) - 1025.0h) * d;
}

the four 2-bit codes are spread into four ushort lanes and OR-ed into the fp16 pattern of 1024 + c, so - 1025 gives c - 1 exactly, and (c - 1) ∈ {-1,0,1,2} times an fp16 d is exact in fp16 (or overflows to the same inf the fp32 → fp16 path gives). that drops 4 int→float + 1 float4→half4 conversion per half4 and does the math in 16-bit. the old comment "fp16-math decode measured slower" was about a different fp16 form; this one wins. runtime compiles with safe math (mtl_rt), and the exhaustive check used that same compile path.

measurements (apple m5, moe_mm_parity --v3 --gu IQ2_S --dn Q2_0 --tile 16,32,32,4 --dtile 16,64,32,2 -T N --ab <baseline> --dn-bench, A = candidate, B = baseline, µs median of interleaved pairs) #

final-source numbers repeated twice with the same result (2048: 7120/7967, 7113/7950; 1024: 4498/4903, 4487/4891). one early D run hit heavy gpu contention (server traffic): absolute numbers inflated, but the interleaved ratio still pointed the same way. kprof scaling (312.4 ms × ~0.9-0.1) guesses ~25-33 ms of prefill GPU time, not measured end to end (no full-model runs per the rules).

mm_gather_h (target 2) #

--gather-bench T (sweep now 32/64/80/128/160/256, 7 interleaved samples), candidate vs baseline kernel, µs per dispatch:

XS memcmp PASS everywhere. 24576 rows × 5120 B = 126 MB written in ~1.19 ms ≈ 105 GB/s plus reads, which is the write roofline here; the "idle 75% of the second wave" doesn't matter because the threads are waiting on memory either way. tg 256 is as good as anything, so window.cpp keeps it. the real win would be to skip the copy (gate/up reading xh through srt_tok), which is outside this track.

not done / next #

• other Q2_0 decoders outside my lines (e.g. any W8<T_Q2_0> or the moe.metal / gemv Q2_0 paths) may take the same trick; worth a follow-up with the same exhaustive check.
• each invocation stayed under 10 s (the max was 7.4 s), except one CPU-reference parity run at -T 256 that I killed at 30 s; I reran it at -T 16 (1.8 s). no :8080 or strata_window_cli.