round 28 track b: fused router #

decision #

primary patch: /tmp/sol_r28_router/sol_r28_router.patch. it adds the short paired --router-ab evaluator and a q8_0 router entry point. route_body and router_route behavior are unchanged, as requested by parent after the initial negative results. moe.metal is untouched. no tracked vendor files were edited.

an optional alternative (not cumulative) patch is /tmp/sol_r28_router/sol_r28_router_f32_nt7_optional.patch. it additionally specializes f32, ntok=7 using direct four-chain fmas. this narrow case repeatedly improved ~5%, but does not accelerate this model's bf16 router. parent can choose whether to audit/promote it.

contract and setup #

• question: do fixed inner-loop unrolls, removing the lookup-table barrier, or direct register combines speed the real fused router without changing any output bits?
• baseline: working-tree kernel snapshot, vendor head 296b59d2f73c8f04a805df29c2fa97655adace78; copied into /tmp/sol_r28_router/base. q8 entry point appended to baseline solely to expose the requested evaluator case. source snapshots and raw run records remain under /tmp/sol_r28_router/.
• owner/path: router kernel's per-layer fused gemv/routing dispatch. preserve ABI, counter reset and fences, fma/reduction order, split plans and residency rules. no host changes.
• hardware: local apple m5, safe math (STRATA_METAL_MATH=safe). shared machine/live server was left running; absolute clocks visibly fluctuate, so use alternating paired ratios, not ratio of separately aggregated medians.
• dimensions: n_in=2560; n_expert=512 or 256. types 30/bf16, 8/q8_0, 0/f32; ntok 1..8; split 0/1.
• decision rule written before sweep: aggregate improvement >=2%, no stable >3% regression; then independent confirmation. no claim of whole-model speedup.
• workload budget: no :8080 requests, no full-model runs, serial short standalone tests only. cached matrix invocations ~0.1–0.5s; first successful cold compile 13.4s. an initial attempt including the entire gemv kernel library hit its 20s compile deadline and was killed; evaluator repaired to supply only the gemv argument struct and NTOK constant.

parity and timing #

four deterministic parity trials per case cover random matrices, n_expert=256, exact ties (identical rows), residency maps, moce on/off and the draft-moCE flag. memcmp compares the entire allocated logits, ids, weights and plan buffers, including sentinel-filled inactive output space; both counters must reset to zero. 48 cases x4 = 192 comparisons per complete sweep, all pass for each ablation and final primary/optional matrices.

timing: 15ms warmup, seven alternating a/b pairs, 16 dispatch repetitions each; report median paired candidate/base ratio. early ablation sweeps timed the final parity trial's draft-moCE settings. final harness explicitly resets moce to zero for timing. seed 928; random weight seeds 928..931.

conclusion: no reproducible benefit from unrolling alone; the barrier guard generally regressed slightly; direct bf16 worsened ntok5 by ~11% and ntok6/7 by ~7–9%. these proposed production-bf16 changes are rejected, not shipped. source-identical controls gave ratios 0.99823 / 0.99009 / 1.00559; small differences near 1% are not wins.

the isolated optional f32 ntok7 path was a repeatable exception. moce-off confirmation, three invocations per split:

representative steady-clock gpu times: 66.711 -> 63.505us (split0), 67.125 -> 63.633us (split1). this is narrowly a standalone f32 seven-token result, not a bf16/live-server result. four fma chains preserve unit_combine<16>'s arithmetic order. full optional matrix: 48/48 cases passed. optional confirmation source differs from primary only by the scoped direct path.

legacy cpu-reference route test: 400 windows, zero id/plan mismatches, max relative weight errors 2.16e-7 (k512) and 3.06e-7 (k256/moce), exit0, 2.20s. primary patch's behavior-identical matrix also passes 48/48.

reproduction and artifacts #

build command (from project root):

clang++ -std=c++20 -O2 /tmp/sol_r28_router/strata/metal/tests/route_parity.cpp -Ivendor/strata/metal/src vendor/strata/build-metal/libmtl_rt.a -framework Metal -framework Foundation -o /tmp/sol_r28_router/route_parity

single case (numeric type ids 30/8/0; ntok1..8; split0/1):

STRATA_METAL_MATH=safe STRATA_METAL_KERNELS=/tmp/sol_r28_router/strata/metal/kernels /tmp/sol_r28_router/route_parity --router-ab /tmp/sol_r28_router/base 30 1 0

baseline directories must contain the q8 router instantiation when testing type8. comparator compiles only quant, units, moe and router, avoiding unrelated gemv specialization compile cost.

raw records: direct-results.json, guard-results.json, unroll-results.json, direct-only-results.json, confirmation.json, narrow-results.json, production-confirmation.json, final-results.json, primary-results.json, legacy-parity.txt, control.txt under /tmp/sol_r28_router/. candidate snapshots: unroll_guard.metal, guard.metal, direct.metal, direct-only.metal. the primary scratch tree now contains only the harness and q8 entry, not optional specialization.

review: visual diff and surrounding code reviewed; helper search found gemv_raw_combine, but it requires decoded thread-local q arrays and thus does not implement the optional direct weight-load path. git diff --check passes. git -C vendor/strata apply --check /tmp/sol_r28_router/sol_r28_router.patch passes. no subagents, server interaction or full-model benchmarks.