round 36 track b: bounded prompt row-tail trimming #
decision #
ship /tmp/sol_r36_prompt/sol_r36_prompt.patch: only metal/kernels/prompt.metal and metal/tests/comb_fold_parity.cpp change. dense_mm.metal and window.cpp remain byte-identical. the patch passes git -C vendor/strata apply --check.
- trim 64-row parents to 16/32 rows in hc-down, both the ordinary column tile and its 96-column tail. factor the row selection into
hc_down_rows. - factor swiglu's existing accumulator epilogue into
swiglu_up_tile; trim 64-row parents to 16/32 rows, retaining original grid offsets and SG count. - deliberately leave 32-row parents untrimmed. the initial broad variant's small wins did not survive confirmation; hc-down SG2 showed a repeatable ~0.5–1.5% full-tile penalty. limiting the descriptors to parents larger than 32 avoids it. small-T 289/297 are therefore not claimed wins.
- keep swiglu's current 64_128_4 selection. test-only 32_128_2 was slower in every requested gate+up chain sample.
- reject dense row trimming: correctness passed, but the same-tile source comparison exposed a large Wout tail regression that the existing tile-policy
dense_mm_parity --abcannot detect.
setup and contract #
apple M5, local standalone Metal tests only. upstream git HEAD at snapshot: 296b59d2f73c8f04a805df29c2fa97655adace78; working-tree sources, not merely HEAD, are snapshotted in /tmp/sol_r36_prompt/baseline-{prompt,dense_mm}.metal. final source snapshot: /tmp/sol_r36_prompt/final-source/; hashes: sha256.json. all implementation/build activity stayed under /tmp/sol_r36_prompt/strata and /tmp/sol_r36_prompt/build. no tracked strata file, live server, port 8080, or full model was touched; no subagents spawned.
initial question/decision rule: /tmp/sol_r36_prompt/plan.md. require exact memcmp parity, repeatable >1% improvement to select a new tile policy, and no >1% repeatable full-tile regression. existing source A/B uses 12 paired samples × 8 dispatches, alternating order and rotating output-buffer placement. brackets in logs are the sorted sample ratios at indices 2 and 9, not confidence intervals. GPU tests were serialized; every invocation completed under 10 seconds (largest observed requested tile-chain sweep: 6.60 s). builds run separately. clocks/temperature were not externally locked; noisy exploratory measurements are retained rather than presented as precise end-to-end predictions.
build:
cmake -S /tmp/sol_r36_prompt/strata/metal -B /tmp/sol_r36_prompt/build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build /tmp/sol_r36_prompt/build --target comb_fold_parity dense_mm_parity -j 4
final prompt measurements #
ratio = candidate / baseline; lower is faster. final-comb-*.log is the final-source run; bounded-{1,2}-*.log confirms the same bounded-descriptor variant before the final cosmetic regrouping.
| T | hc-down ratio | swiglu ratio | interpretation |
|---|---|---|---|
| 289 | 0.9994 | 1.0039 | TM32 unchanged; noise |
| 297 | 0.9963 | 1.0004 | TM32 unchanged; noise |
| 306 | 0.9986 | 1.0039 | no eligible tail |
| 512 | 1.0004 | 1.0002 | full tiles |
| 1024 | 1.0013 | 0.9989 | full tiles |
| 1025 | 0.9536 | 0.9861 | 1-row tail; confirmed |
| 1041 | 0.9669 | 0.9894 | 17-row tail; confirmed |
| 1983 | 0.9859 | 0.9995 | hc-down TM32 unchanged; do not attribute its fluctuation to trimming |
| 1987 | 0.9909 | 0.9937 | swiglu 3-row tail; hc-down TM32 unchanged |
| 2048 | 0.9932 | 1.0011 | full tiles |
at 1025/1041, the two earlier confirmation runs gave hc-down 0.9542/0.9539 and 0.9661/0.9651; swiglu 0.9887/0.9832 and 0.9875/0.9881. the smaller swiglu 1987 benefit was usually ~0.5–1.0%, but one late noisy confirmation was flat; do not sell that as a large speedup. no full-model latency estimate is claimed.
swiglu tile policy experiment #
comb_fold_parity T epi bench 32_128_2,64_128_4, requested T values; tiles-T.log. values are ms for 48 gate+fused-up chains, not the up kernel alone. both candidates use the same gate dispatch in each process.
| T | 32_128_2 | 64_128_4 | SG2 / SG4 |
|---|---|---|---|
| 384 | 8.784 | 8.721 | 1.0072 |
| 512 | 11.623 | 11.576 | 1.0041 |
| 1024 | 24.332 | 24.288 | 1.0018 |
| 1983 | 48.320 | 47.927 | 1.0082 |
| 1987 | 50.424 | 50.145 | 1.0056 |
| 2048 | 61.928 | 60.397 | 1.0253 |
no selection change is justified. these exploratory runs used the broad row-trimming candidate, which preserves the same TM64 behavior as final; their TM32 candidate had the extra tail descriptor. full-T 384/512/1024/2048 still reject SG2 without depending on a partial row tile.
dense negative result, and why the new harness exists #
dense_mm_parity --ab compares r34-vs-current tile policies using one shader revision. it is not source A/B. both policies can share a regression and still print “1.00x; ALL BITWISE”. the new comb_fold_parity T denseab baseline-dense_mm.metal [shape] compiles both source revisions and compares exactly the same tile, inputs, and rotating output buffers across revisions. it uses the corrected window tile policy, supports all 13 dense shapes, and fails for an unknown requested shape or T outside 1..2048.
broad dense candidate: factor gemm_f16_tile, retain SG, add TM16/TM32 parent-tail dispatch. 13 shapes × eight requested M values all passed memcmp; logs dense-source-ab-M.log. but Wout (N=2560, K=6144, tile64_128_4) at M1987 gave ratio 1.6845 in the exploratory sweep, then 1.7014 and 1.7785 on isolated confirmation, while candidate-self source A/B was 1.0003. even TM32-only trimming regressed 1.6347. see dense-Wout-confirm-1987.log, dense-Wout-confirm2.log, dense-Wout-self.log, dense-Wout-32only-1987.log.
this is consistent with a descriptor/resource or tail-path performance cliff, but register spilling itself was not measured. a late reverse-direction check (original as current, rejected candidate as baseline) showed only 0.9209, so the precise magnitude depends on run conditions; it still did not rescue the candidate. artifact candidate-dense-v1.metal retains the rejected implementation. dense is entirely excluded from the deliverable patch, as parent requested.
verification #
final-source tests:
comb_fold_parity 0 epi: ALL PASS, 18 token counts × seven tiles × both epilogues = 252 bitwise output checks. includes 16/17/32/33 boundary rows, 289/297/306, 1025/1041 and 1987.final-comb-0.log.comb_fold_parity T ab /tmp/sol_r36_prompt/baseline-prompt.metal gemm, T=289,297,306,512,1024,1025,1041,1983,1987,2048: ALL PASS; includes hc-up as an unchanged control.final-comb-T.log.dense_mm_parity: ALL PASS (float-reference numerical tolerance for GEMM, exact dequant/operand conversion).final-dense-default.log. do not mislabel its float-reference check as bitwise GEMM parity.DENSE_ROUNDS=3 dense_mm_parity --ab M, M=289,297,306,512,1024,1983,1987,2048: ALL BITWISE across all 13 shapes.final-dense-policy-ab-M.log. these final runs use the unchanged dense shader.- source A/B harness rejected unknown shape with exit 1 (
final-dense-unknown-shape.log); original/candidate reverse check remained bitwise correct (final-dense-rejected-reverse.log). - final visual diff/context review completed,
git diff --checkpasses, and patch application check passes against the actual vendor checkout.
all raw evidence, baseline and rejected source snapshots, build products and the patch are in /tmp/sol_r36_prompt/. only this report is written into the main checkout.