round 18 track b #
contract / preflight #
owner: metal grouped-residual kernels and Window::gr_norm dispatch selection. preserve operation order, generic shape fallback, apply=false semantics and buffer bindings. no live server or full-model runs. question: do fixed-shape pointer hoisting / unrolling and stream staging improve latency without changing a single output bit? compare baseline 4s, optimized 4s, per-stream grid and staged 4s; compare baseline/optimized gr_down_q8_8_4. evaluator: deterministic fuse_parity inputs, memcmp R/xn and down/inject for ntok 1..8; ordered alternating gpu timings, 3 exploratory and 5 confirmation pairs, 64 dispatch repetitions each. improvement must exceed 3% median, with bitwise parity mandatory. each invocation timeout 4.9s, no concurrent gpu tests; compile jobs exempt from test invocation limit. live server may introduce noise; report it. source snapshots: /tmp/sol_r18_grcomb/baseline and candidate scratch tree; no tracked vendor edits. stop if compilation cannot be repaired in two attempts or tests exceed the invocation budget.
result / decision #
ship the hoisted/unrolled gr_norm_comb with grid (4, ntok). keep optimized gr_norm_comb_4s available, but do not dispatch it from Window::gr_norm. shared-bo staging was slower. do not ship the gr_down_f change: it passed bitwise parity but did not establish a repeatable improvement.
main patch: /tmp/sol_r18_grcomb/sol_r18_grcomb.patch (three files; no gr.metal edits). rejected/neutral down experiment: /tmp/sol_r18_grcomb/neutral_down.patch. neither patch has been applied to tracked vendor sources.
execution / evidence #
- machine: Apple M5 (the local 10-core GPU laptop). no server access, full-model benchmarks, or spawned agents.
- vendor input HEAD:
296b59d2f73c8f04a805df29c2fa97655adace78; input working-tree snapshots are in/tmp/sol_r18_grcomb/baseline/, since HEAD alone does not describe this already-dirty development tree. - scratch baseline commit:
faeb472; all executed shader fixtures, evaluator source, binary, and input/candidate hashes:/tmp/sol_r18_grcomb/sha256.json. - all narrow GPU invocations completed below 1.6 seconds. first existing full
fuse_parityinvocation was terminated at 4.9s during cold shader compilation. parent then explicitly permitted a 20s timeout for the full test's compile overhead. full suite subsequently passed twice: 5.45s and 13.73s (including compilation), exit 0; final log/tmp/sol_r18_grcomb/final_full_parity.log. - standalone compile/link succeeded; scratch
window.cppcompiled successfully with the existing engine compiler flags, output/tmp/sol_r18_grcomb/window.o. - final
git -C vendor/strata apply --check /tmp/sol_r18_grcomb/sol_r18_grcomb.patch: exit 0. scratchgit diff --check: exit 0. --gr-ab ... 0 normrejects invalid token counts with exit 2 before creating a GPU context.
correctness #
memcmp == 0 for both R and xn across ntok 1..8, apply false/true, topk 1/10, for optimized 4s, per-stream grid and staged-bo variants versus the original 4s. 96 final A/B parity cases passed. original full suite also checks both production combine kernels versus moe_combine_sh + gr_norm, and passed all 32 cases (plus gemv/router tests). down Q8_0 lo and inject are bitwise equal for ntok 1..8. generic n != 2560 behavior is retained but was not newly exercised.
timing contract adjustment #
initial 80ms/30ms warm-up left obvious clock ramps. before confirmation, increased warm-up to 300ms baseline + 100ms candidate, with five alternating-order pairs of 64 dispatches each. preserved both exploratory and confirmation logs rather than replacing the original evidence. the live machine can change clocks or have other GPU work; absolute timings are not stable, so a direct optimized-4s-vs-grid confirmation was also run. this is a kernel microbenchmark, not an end-to-end speedup estimate.
original 4s vs selected per-stream kernel #
medians of five paired batches, microseconds/dispatch:
| ntok | original 4s | selected grid4 | reduction |
|---|---|---|---|
| 1 | 22.673 | 4.458 | 80.3% |
| 2 | 22.820 | 4.680 | 79.5% |
| 3 | 22.882 | 5.902 | 74.2% |
| 4 | 22.878 | 6.453 | 71.8% |
| 5 | 54.246 | 16.863 | 68.9% |
| 6 | 22.977 | 8.649 | 62.4% |
| 7 | 43.543 | 13.086 | 69.9% |
| 8 | 22.951 | 10.456 | 54.4% |
optimized 4s reduced baseline time by 46–49% in this confirmation; staged shared bo reduced it by 43–46%. the selected grid kernel reduced it by 54–80%.
direct optimized 4s vs selected grid4 #
these runs happened at much slower clocks for most token counts, but both candidates were measured back-to-back in alternating order:
| ntok | optimized 4s | selected grid4 | reduction |
|---|---|---|---|
| 1 | 51.937 | 20.042 | 61.4% |
| 2 | 51.971 | 20.734 | 60.1% |
| 3 | 52.184 | 26.059 | 50.1% |
| 4 | 52.386 | 28.171 | 46.2% |
| 5 | 52.582 | 31.563 | 40.0% |
| 6 | 11.881 | 8.551 | 28.0% |
| 7 | 54.546 | 30.451 | 44.2% |
| 8 | 52.787 | 46.443 | 12.0% |
all eight direct comparisons favor grid4 (12–61%), exceeding the 3% decision threshold.
down experiment (not shipped) #
implemented exactly the requested xnp pointer hoist, UNIT/16 load unroll, and KS reduction unroll. ntok 1..8 all bitwise PASS. confirmation median reductions were 1.30%, 4.40%, -0.14%, 0.13%, -0.18%, -0.03%, 0.11%, 0.12%. ntok2 was clock-noisy, not a broad repeatable win. preserve this as a measured neutral result, not a performance claim.
reproduce #
artifacts live under /tmp/sol_r18_grcomb/; norm.metal contains original 4s plus all three candidates, down.metal original/candidate down, direct.metal uses optimized 4s as the reference. the focused evaluator is the new --gr-ab <absolute shader path> <ntok> <norm|down> mode in metal/tests/fuse_parity.cpp.
from the project root (replace $PWD with the checkout path if launched elsewhere):
clang++ -std=c++20 -O2 -Ivendor/strata/metal/src /tmp/sol_r18_grcomb/strata/metal/tests/fuse_parity.cpp vendor/strata/build-metal/libmtl_rt.a -framework Metal -framework Foundation -o /tmp/sol_r18_grcomb/fuse_parity
STRATA_METAL_KERNELS=/tmp/sol_r18_grcomb/strata/metal/kernels /tmp/sol_r18_grcomb/fuse_parity --gr-ab /tmp/sol_r18_grcomb/norm.metal 1 norm
STRATA_METAL_KERNELS=/tmp/sol_r18_grcomb/strata/metal/kernels /tmp/sol_r18_grcomb/fuse_parity --gr-ab /tmp/sol_r18_grcomb/direct.metal 1 norm
STRATA_METAL_KERNELS=/tmp/sol_r18_grcomb/strata/metal/kernels /tmp/sol_r18_grcomb/fuse_parity --gr-ab /tmp/sol_r18_grcomb/down.metal 1 down
repeat ntok 1..8, sequentially. paired logs: confirm_norm_*.log, confirm_down_*.log, direct_*.log; additional topk1 regression logs final_norm_*.log, post-review endpoint checks postreview_norm_1.log / postreview_norm_8.log. machine-readable medians: measurements.json.
review #
reviewed every patch hunk with view_diff, helper/Window context with view_code, then re-reviewed formatting fixes. no arithmetic reassociation or reduction-order changes; all new pointer accesses stay inside the existing fixed-shape specialization and topk guard, generic fallback remains. no live server settings changed.