6.7 kB 72 lines
a1f70833
Round 25 blitz: mm_down3 Q2_0 64-elem block step + run_q<2/6/10/14> (-11.8% in kprof, 361.9->319.2 ms), gdn_post gate/gamma preload (-0.7%), mm_gu3 half4 HS store pointer hoist
dawn
⋯
audits/sol_r25_ple.md
sourcerendered
download raw history
at main Markdown

round 25 track b — ple loop inversion / scaling / history #

decision #

no production kernel changes recommended. the proposed k-outer inversion preserves tested outputs but is slower on the m5 at large prompt sizes. accumulating directly in bo[] does not fix it. hc==4 scaling and history unrolling show no repeatable net win. the supplied patch is optional test-harness improvements only, not an optimization claim.

  • scratch: /tmp/sol_r25_ple/strata
  • clean patch: /tmp/sol_r25_ple/sol_r25_ple.patch
  • sources/logs: /tmp/sol_r25_ple/; hashes: sha256.json
  • baseline: vendor strata head 296b59d2f73c8f04a805df29c2fa97655adace78 plus pre-existing dirty working-tree metal sources copied at task start. baseline.metal is the exact captured ple source, not git head.
  • no tracked vendor files changed; no server requests, full-model runs, or subagents.

experiment contract and execution #

owner is ple.metal's per-dispatch arithmetic; no buffers, public ABI, persistent state, or synchronization ordering should change. preserve each output's expert FMA order. reject any bitwise mismatch; require repeatable paired speed improvement, emphasizing T>=512 without clear smaller-T regressions. plan recorded before implementation in /tmp/sol_r25_ple/plan.md.

machine: Apple M5, local apple clang 21.0.0.21000327, safe metal math (runtime default, fast math disabled). fixed evaluator N=2560, HC=4, TOPK=10, seed 11+T, shuffled expert rows with occasional dropped slots. A/B compares residual and normalized buffers via memcmp; added history comparison uses separate identical initial history buffers. five paired timing samples alternate execution order, reporting median candidate/baseline ratios. timings below are GPU milliseconds, not wall time. use the paired-ratio median, which need not equal the ratio of independently reported median times.

build:

cmake -S /tmp/sol_r25_ple/strata/metal -B /tmp/sol_r25_ple/build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build /tmp/sol_r25_ple/build --target ple_moe_parity
/tmp/sol_r25_ple/build/ple_moe_parity --ab /tmp/sol_r25_ple/baseline.metal T

candidate source is loaded from the scratch tree at runtime. exact snapshots are all.metal, inversion.metal, direct.metal, unrolled.metal, scaling.metal, history.metal. each log prefix names the candidate used. final scratch ple.metal is restored byte-for-byte to baseline.

measured results #

full proposed set (all-T*.log) #

bitwise pass at all requested T = 1,4,8,64,512,1024,2048 for moe norm, combined moe norm, query norm, and history. ple_comb_query_gate_norm is not covered by this evaluator; scaling there is therefore not validated and is not shipped. these are finite randomized parity checks, not exhaustive IEEE edge-value proofs.

T combined moe norm standalone ratio with conv_res ratio
1 0.5901 0.6676
4 0.5961 0.6393
8 0.5902 0.7654
64 1.0527 1.1091
512 1.1082 1.0802
1024 1.0826 1.0677
2048 1.0799 1.0622

ablations #

  • inversion only, preserving original division/history: standalone ratios at T512/1024/2048 = 1.0761 / 1.1009 / 1.0887; with conv = 1.0574 / 1.0724 / 1.0599. all pass.
  • direct bo accumulation (parent follow-up): first run standalone ratios = 1.1440 / 1.1173 / 1.0564, with conv = 1.0491 / 1.0584 / 1.0519. confirmation T512/1024 = 1.0844 / 1.0798, with conv 1.0658 / 1.0639; T2048 confirmation became noisy. all completed checks pass. naming one fewer array does not establish lower actual register pressure: the compiler may already coalesce disjoint s/bo live ranges. no ISA/register counters were collected, so the physical cause is unproven.
  • cnt==10 i-outer k-inner full unroll: no consistent standalone win; confirmation T512/1024/2048 ratios 1.0052 / 1.0233 / 1.0144. combined-chain apparent gains occurred during severe drift and do not establish a win. all completed checks pass.
  • hc==4 scaling only, with original division fallback for other hc: completed all seven requested T with parity passes for covered kernels; timings inconsistent. no confirmed benefit. do not replace division by a computed reciprocal for arbitrary hc if bitwise equivalence is required.
  • history-only unrolling: all seven requested T pass; 128-repetition history timing ratios span 0.9780–1.1563, with no consistent benefit. full unrolling does not itself prove removal of a spill; no generated-code inspection was done.

baseline-vs-baseline calibration at T512/1024/2048: standalone combined moe norm ratios 0.9994 / 0.9942 / 1.0015, combined with conv 1.0059 / 1.0054 / 1.0086 (calibration-T*.log). some other runs show severe timing drift, including unchanged controls: those isolated apparent gains are inconclusive rather than winners. the repeated inversion slowdown in comparatively stable runs is enough to reject shipping it; it is not enough to diagnose hardware counters.

optional patch and final verification #

only metal/tests/ple_moe_parity.cpp changes:

  • A/B history memcmp and timing with independent histories;
  • A/B timing for ple_moe_gate_norm as well as combined norm;
  • allow timing at small T;
  • short A/B warmup (3 ms each) and two repetitions per timed sample, 128 for the tiny history kernel, to stay within the short-invocation budget. five alternating paired samples remain. this is screening, not high-confidence thermal/performance isolation.

final build passes. final baseline-vs-baseline A/B checks at all seven T exit 0 / ALL PASS (final-test-T*.log), wall times 0.167–1.970s. non-A/B unfused-chain and CPU history reference checks (ple_moe_parity 2048) exit 0 / ALL PASS in 0.454s (final-nonab.log). note the non-A/B default test skips the TB fusion case without bench, while its separate history reference loop includes 2048.

git -C vendor/strata apply --check /tmp/sol_r25_ple/sol_r25_ple.patch exits 0. final visual diff/context review completed. optional tests do not alter production behavior.

one cold compile for unrolled-T1.log hit the configured 5s timeout and was terminated (wall 5.025s, exit -15; no result). the candidate source was preserved and later separate T invocations completed. no longer timeout or full-model benchmark was used. initial full-set cold compile passed in 4.981s. no measurement is claimed for that interrupted unrolled T1 run.

next step #

keep production ple.metal unchanged. use the optional short A/B test patch if useful; otherwise retain only the negative-result report. an exclusive GPU/thermal window and generated-code inspection would be needed before attributing remaining small effects to instruction count or register allocation.