round 31 track b experiment plan #
owner: q2 decode combine helpers and dn_body in isolated /tmp/sol_r31_moe/strata. no interfaces, summation ordering, format layout, other kernels or live server changes.
hypotheses: half-mantissa and float-mantissa decode remove integer conversion overhead; packed loads/pointer stepping reduce address overhead. each is compared against immutable base kernel snapshot, with expert_parity --ab exact memcmp over NE=1..4,8/random and paired dn timings; gemv uses its built-in ab check. baseline self-ab first. accept only reproducible >=2% improvement without material regressions (>2%) across required cases; repeat apparent winners three times. all gpu invocations serial, reps=4/layers=1 initially to stay <10s (20s timeout for cold compile), no full-model work. reject failed parity, stop after bounded variants if gains not established. live server noise is uncontrolled; report rather than overclaim.
result: ship half-mantissa combine only #
patch: /tmp/sol_r31_moe/sol_r31_moe.patch (git apply --check against vendor/strata: exit 0). changes only q2_combine<T_Q2_0R> in moe.metal and gemv_q2_raw_combine in gemv.metal. no test code or dn loop changes survive. neither tracked vendor source nor :8080 touched.
machine: apple m5. baseline vendor head: 296b59d2f73c8f04a805df29c2fa97655adace78, dirty working tree, so commit alone is not the measured source. exact baseline kernels preserved at /tmp/sol_r31_moe/base/; exact candidate at /tmp/sol_r31_moe/half/ and /tmp/sol_r31_moe/strata/metal/kernels/. source hashes in /tmp/sol_r31_moe/source_hashes.json. prebuilt parity binaries from vendor/strata/build-metal; no evaluator edits. default safe math.
correctness #
all expert invocations exited 0 and printed 0 fail(s) ALL PASS; the harness compares full plan, h, ye, y via memcmp for same-expert ne=1,2,3,4,8 and random routing at ntok=1,2,3,4,8. candidate was checked repeatedly against the immutable baseline. host exhaustive check of all 65,536 packed ushort words, all eight code positions, also exactly reproduces code-1 via half-bit decode. summation/fma order is unchanged.
gemv: bitwise parity at 640→2560, ntok=1,2,4 for both Q2_0 and Q2_0R (142). Q2_0R is essential: Q2_0 uses the prepared-input path and does not exercise the changed helper. the raw helper is used only for ntok=1; raw ntok=2,4 retain the decoded path. logs: /tmp/sol_r31_moe/gem_q2_nt{1,2,4}.log, /tmp/sol_r31_moe/gem_half_confirm*_nt{1,2,4}.log.
paired expert timings #
command (same for baseline and all candidates):
STRATA_METAL_KERNELS=/tmp/sol_r31_moe/strata/metal/kernels \
vendor/strata/build-metal/expert_parity --gu IQ2_S --dn Q2_0 \
--ab /tmp/sol_r31_moe/base --bench --layers 1 --reps 4
baseline self-ab ratios 0.9868–1.0116 (baseline.log). first clean half run 0.8299–0.9218 (half.log). three confirmation runs, seven alternating pairs per case:
| case | confirm 1 | confirm 2 | confirm 3 | median ratio |
|---|---|---|---|---|
| ne=1 | 0.9145 | 0.9007 | 0.9086 | 0.9086 |
| ne=2 | 0.8600 | 0.8506 | 0.8539 | 0.8539 |
| ne=3 | 0.8417 | 0.8250 | 0.8261 | 0.8261 |
| ne=4 | 0.8221 | 0.8271 | 0.8258 | 0.8258 |
| ntok=1 | 0.9059 | 0.9036 | 0.9155 | 0.9059 |
| ntok=2 | 0.9246 | 0.9154 | 0.9173 | 0.9173 |
| ntok=4 | 0.9277 | 0.9194 | 0.9236 | 0.9236 |
| ntok=8 | 0.8900 | 0.8865 | 0.8936 | 0.8900 |
this is 7.6–17.4% lower dn latency by median case ratio, not an end-to-end throughput claim. example confirmation 3: ne=1 45.4→40.8 us; ne=4 98.4→81.2 us; mixed ntok=8 266.7→238.2 us. unchanged gu is a useful control (~1.00). logs half_confirm1/2/3.log. all gpu invocations serial in this session; other tracks and live server may overlap, so parent will do exclusive confirmation. reps/layers were intentionally kept small to honor <10s invocations.
gemv timings #
STRATA_METAL_KERNELS=/tmp/sol_r31_moe/strata/metal/kernels \
vendor/strata/build-metal/gemv_parity --ab /tmp/sol_r31_moe/base Q2_0R 640 2560 1 gemv
repeat with ntok 2/4 and with Q2_0. raw ntok=1 median paired ratios on confirmation repeats 2/3: 0.8754 / 0.8930 (12.5% / 10.7% lower). raw ntok=2/4 ratios ~0.995–0.999, as expected for untouched paths; Q2_0 ratios 0.9961–0.9999. first raw runs showed frequency ramping, recorded rather than discarded, and were followed by stable repeats. no material regressions observed.
rejected alternatives / evidence #
- packed_ushort2 down loads: bitwise pass, ratio 0.9970–1.0173, no established gain (
packed.log). - pointer stepping + packed load: bitwise pass, ne=3 +2.6% regression, mixed cases -2.1..-4.2% (
pointer.log). combined with winning half against half-only: ratio 0.9824–1.0124, no >=2% incremental improvement (half_pointer_vs_half.log). excluded. - unroll_count(5): bitwise pass, ne=2 +7.9%, ne=3 +6.0%, mixed ntok=8 +8.0% (
unroll.log). excluded. - fp32 mantissa: bitwise pass but severe external gpu interference (unchanged gu and dn times ballooned) makes its timing inconclusive, not a valid performance rejection (
float.log). do not ship because half has reproducible wins while fp32 does not have clean confirmation.
all standalone processes finished below 10s (slowest float run 7.84s); timeout=20 allowed cold compilation. no full model benchmark. experiments stopped after confirmed half winner; quiet confirmation belongs to parent.
review #
reviewed complete scratch git diff visually, searched existing half-mantissa idioms (matches moe_mm), checked field ordering x/y/z/w and exact fma sequence. the two small helpers stay local to their kernel translation units; moving them to units.metal would violate ownership and broaden the patch. no unrelated edits, added debug output, or hard-coded shape assumptions.