hoisted l, IQ3_S super-block loop #

short version: one tiny change ships, the half4 coalesced HS store turned on for T_IQ2_S, for about -0.3% on the IQ2_S gate/up at T=1024/2048 (34 of 48 layers). the uniform-ib / hoisted-l rewrite and moving IQ3_S into the super-block loop are both 100% bitwise identical but +1.3..4.4% slower, so they're dropped (´・ω・`)

patch: /tmp/opus_r32_moemm/opus_r32_moemm.patch (1 line in metal/kernels/moe_mm.metal, git -C vendor/strata apply --check OK)
scratch: /tmp/opus_r32_moemm/strata, baseline kernels /tmp/opus_r32_moemm/base_kernels, every variant in /tmp/opus_r32_moemm/v_*/
moe_mm_parity.cpp unchanged (the existing --ab / --gu-bench mode was enough)

method #

existing build-metal/moe_mm_parity with STRATA_METAL_KERNELS=<dir> vs --ab <dir>, --v3 --dn Q2_0 --tile 16,32,32,4 --dtile 64,32,2 --gu-bench. each cell is 4 or 6 runs that alternate which kernel is A, and the number is the mean of the candidate-vs-base median diffs (negative = faster). every run also memcmps HS/YE/Y. base vs base on IQ2_S@1024 came out -0.02% [-0.07..+0.04]. the identical-code cells (IQ2_XXS/IQ2_XS under H_all) spread about ±0.4% (one +1.3% outlier), with the :8080 server resident, so I only treat a move as real if every run agrees on the sign.

results (all bitwise PASS: HS_memcmp / YE_memcmp / Y_memcmp) #

variant	change	fmt	T=512	T=1024	T=2048
H_all / F	half4 HS store, IQ2_S	IQ2_S	-0.10 (4), -0.17 (6)	-0.38 [all 4 <0], -0.35 [all 6 <0]	-0.26 [all 4 <0], -0.06 (6, one +2.0 outlier; median -0.22)
H_all / F	half4 HS store, IQ3_S	IQ3_S	-1.25 (very noisy), +0.32 [-0.38..+1.27]	-0.33 [all <0], -0.30 [all <0]	-0.10, -0.60 (noisy)
U	`block(bl, ib = ki>>5, l = kl>>3 hoisted, ...)` for IQ2_S/XXS/XS	IQ2_S		+4.29 [3.6..4.7]	+1.29
U	same	IQ2_XXS		+2.72
U3	U + IQ3_S `block` (ib/l uniform) inside the 256 super-block `sb` loop	IQ3_S		+3.61
S3	IQ3_S in the `sb` loop, still indexed by `j_in` (no ib change)	IQ3_S		+4.05	+4.43

shipped #

T_IQ2_S added to the half4 store gate. extra checks: default parity PASS (rel L2 1.60e-05), --ab at T=512 bitwise PASS with tiles 16,32,32,4 and 16,64,32,2.

not shipped:

IQ3_S half4: -0.3% at T=1024 and 2048, but T=512 has no clear sign (+0.32 mean, runs from -0.38 to +1.27), and r14 measured +3..5% for IQ3_S at mid T with this store. it's only 2 layers, so it's not proven enough to ship. flipping it on later is just adding || F == T_IQ3_S.
IQ4_XS / K-quants / Q8_0 (removing the F gate entirely): not benched. r14 saw Q6_K +1.1% at T=512, so they keep the scalar store.
target 2 (uniform ib / hoisted l): a reliable regression. this matches r14 variant D, where the compiler already got the lane-constant part from j_in. turning ib into a loop-carried scalar (ki >> 5 of the non-unrolled 8-step ki loop) seems to make the codegen worse, not better.
IQ3_S in the sb loop: +4% whether or not ib changes. the 8-step ki loop plus the dg/du arrays cost more than the one rdh a 32-step that it saves.

take-away #

the mm_gu3 decode address math is still not the bottleneck. 3 rounds now (r14 D, r32 U, U3, S3) have shown that restructuring the indexing loses. the epilogue store is good for a few tenths of a percent at most.