# round 28 track b: fused router

## decision

primary patch: `/tmp/sol_r28_router/sol_r28_router.patch`.
it adds the short paired `--router-ab` evaluator and a q8_0 router entry point. `route_body` and `router_route` behavior are unchanged, as requested by parent after the initial negative results. `moe.metal` is untouched. no tracked vendor files were edited.

an optional alternative (not cumulative) patch is `/tmp/sol_r28_router/sol_r28_router_f32_nt7_optional.patch`. it additionally specializes f32, ntok=7 using direct four-chain fmas. this narrow case repeatedly improved ~5%, but does not accelerate this model's bf16 router. parent can choose whether to audit/promote it.

## contract and setup

- question: do fixed inner-loop unrolls, removing the lookup-table barrier, or direct register combines speed the real fused router without changing any output bits?
- baseline: working-tree kernel snapshot, vendor head `296b59d2f73c8f04a805df29c2fa97655adace78`; copied into `/tmp/sol_r28_router/base`. q8 entry point appended to baseline solely to expose the requested evaluator case. source snapshots and raw run records remain under `/tmp/sol_r28_router/`.
- owner/path: router kernel's per-layer fused gemv/routing dispatch. preserve ABI, counter reset and fences, fma/reduction order, split plans and residency rules. no host changes.
- hardware: local apple m5, safe math (`STRATA_METAL_MATH=safe`). shared machine/live server was left running; absolute clocks visibly fluctuate, so use alternating paired ratios, not ratio of separately aggregated medians.
- dimensions: n_in=2560; n_expert=512 or 256. types 30/bf16, 8/q8_0, 0/f32; ntok 1..8; split 0/1.
- decision rule written before sweep: aggregate improvement >=2%, no stable >3% regression; then independent confirmation. no claim of whole-model speedup.
- workload budget: no :8080 requests, no full-model runs, serial short standalone tests only. cached matrix invocations ~0.1–0.5s; first successful cold compile 13.4s. an initial attempt including the entire gemv kernel library hit its 20s compile deadline and was killed; evaluator repaired to supply only the gemv argument struct and NTOK constant.

## parity and timing

four deterministic parity trials per case cover random matrices, n_expert=256, exact ties (identical rows), residency maps, moce on/off and the draft-moCE flag. `memcmp` compares the entire allocated logits, ids, weights and plan buffers, including sentinel-filled inactive output space; both counters must reset to zero. 48 cases x4 = 192 comparisons per complete sweep, all pass for each ablation and final primary/optional matrices.

timing: 15ms warmup, seven alternating a/b pairs, 16 dispatch repetitions each; report median paired candidate/base ratio. early ablation sweeps timed the final parity trial's draft-moCE settings. final harness explicitly resets moce to zero for timing. seed 928; random weight seeds 928..931.

| early ablation | type id | median ratio over ntok/split | min–max |
|---|---:|---:|---:|
| unroll | 0 | 1.00373 | 0.99935–1.01229 |
| unroll | 8 | 0.99866 | 0.98896–1.01130 |
| unroll | 30 | 0.99910 | 0.99032–1.01450 |
| guard | 0 | 1.01801 | 0.99831–1.04143 |
| guard | 8 | 1.00790 | 0.99006–1.02247 |
| guard | 30 | 1.01531 | 1.00424–1.04164 |
| unroll+guard+direct-bf16 | 0 | 1.02106 | 1.00761–1.04995 |
| unroll+guard+direct-bf16 | 8 | 1.00716 | 0.99807–1.02363 |
| unroll+guard+direct-bf16 | 30 | 1.04608 | 1.01363–1.11216 |
| direct-only | 0 | 1.02086 | 0.94422–1.08185 |
| direct-only | 30 | 1.04110 | 1.01015–1.11666 |

conclusion: no reproducible benefit from unrolling alone; the barrier guard generally regressed slightly; direct bf16 worsened ntok5 by ~11% and ntok6/7 by ~7–9%. these proposed production-bf16 changes are rejected, not shipped. source-identical controls gave ratios 0.99823 / 0.99009 / 1.00559; small differences near 1% are not wins.

the isolated optional f32 ntok7 path was a repeatable exception. moce-off confirmation, three invocations per split:

| repetition | split0 ratio | split1 ratio |
|---|---:|---:|
| 1 | 0.93889 | 0.94086 |
| 2 | 0.94834 | 0.95160 |
| 3 | 0.95195 | 0.94308 |

representative steady-clock gpu times: 66.711 -> 63.505us (split0), 67.125 -> 63.633us (split1). this is narrowly a standalone f32 seven-token result, not a bf16/live-server result. four fma chains preserve `unit_combine<16>`'s arithmetic order. full optional matrix: 48/48 cases passed. optional confirmation source differs from primary only by the scoped direct path.

legacy cpu-reference route test: 400 windows, zero id/plan mismatches, max relative weight errors 2.16e-7 (k512) and 3.06e-7 (k256/moce), exit0, 2.20s. primary patch's behavior-identical matrix also passes 48/48.

## reproduction and artifacts

build command (from project root):

```text
clang++ -std=c++20 -O2 /tmp/sol_r28_router/strata/metal/tests/route_parity.cpp -Ivendor/strata/metal/src vendor/strata/build-metal/libmtl_rt.a -framework Metal -framework Foundation -o /tmp/sol_r28_router/route_parity
```

single case (numeric type ids 30/8/0; ntok1..8; split0/1):

```text
STRATA_METAL_MATH=safe STRATA_METAL_KERNELS=/tmp/sol_r28_router/strata/metal/kernels /tmp/sol_r28_router/route_parity --router-ab /tmp/sol_r28_router/base 30 1 0
```

baseline directories must contain the q8 router instantiation when testing type8. comparator compiles only quant, units, moe and router, avoiding unrelated gemv specialization compile cost.

raw records: `direct-results.json`, `guard-results.json`, `unroll-results.json`, `direct-only-results.json`, `confirmation.json`, `narrow-results.json`, `production-confirmation.json`, `final-results.json`, `primary-results.json`, `legacy-parity.txt`, `control.txt` under `/tmp/sol_r28_router/`. candidate snapshots: `unroll_guard.metal`, `guard.metal`, `direct.metal`, `direct-only.metal`. the primary scratch tree now contains only the harness and q8 entry, not optional specialization.

review: visual diff and surrounding code reviewed; helper search found gemv_raw_combine, but it requires decoded thread-local q arrays and thus does not implement the optional direct weight-load path. `git diff --check` passes. `git -C vendor/strata apply --check /tmp/sol_r28_router/sol_r28_router.patch` passes. no subagents, server interaction or full-model benchmarks.