reconstruction: recovered from the final report delivered in parent transcript row223028; original file bytes and referenced /tmp raw evidence are lost. historical validation statements below describe checks made at measurement time, not checks repeated during recovery.
verdict #
recommend reverting chunk-0 affinity to0 pending task-level validation: incumbent buys real -15.32% prefill latency with -18.12% prompt bytes, but changes ops distributions by meanKL0.055018..0.076038 and4–7 argmax decisions per64 forced positions. direction is unestablished; at least one change moves toward the corpus target. this is unvalidated behaviour change, not demonstrated harm. hard also fails preservation. this report recommends the revert; it does not modify the repo.
measurement is incumbent300.85 vs off0 (primary), hard320335 vs off0 and tail-soft320300.85 vs off0 (secondary). incumbent was already shipped, not a proposed new default. hypothesis: resident-biased routing reduces expert SSD reads on chunk0. each speed block alternates off/candidate for5 pairs after discarded warmups. all paired runs exit0; no service started, no ports used, no source/binary rebuilds.
paired prefill (persona_458.txt, 458 tokens, single 2048 chunk) #
all prompt-read accounting below uses STRATA_METAL_TRACE=1. cli sends 457 rows as one pos0=0 chunk, then the final prompt token through the head window. prefill_ms is end-to-end prefill, excluding init; chunk-only ms comes from the trace. no temperatures/sampling flags exist here: non-serve CLI generation is greedy. eight generated ids per speed run.
incumbent vs off #
| pair | arm | end-to-end prefill excluding init ms | tok/s | candidate/off end-to-end ratio | chunk-only ms | candidate/off chunk-only ratio |
|---|---|---|---|---|---|---|
| 1 | 0 | 4668.05 | 98.11 | — | 4466 | — |
| 1 | 300.85 | 3411.79 | 134.24 | 0.730881 | 3210 | 0.718764 |
| 2 | 0 | 3948.37 | 116.00 | — | 3758 | — |
| 2 | 300.85 | 3448.57 | 132.81 | 0.873416 | 3252 | 0.865354 |
| 3 | 0 | 3913.79 | 117.02 | — | 3704 | — |
| 3 | 300.85 | 3432.47 | 133.43 | 0.877019 | 3251 | 0.877700 |
| 4 | 0 | 4031.42 | 113.61 | — | 3828 | — |
| 4 | 300.85 | 3413.94 | 134.16 | 0.846833 | 3234 | 0.844828 |
| 5 | 0 | 4036.32 | 113.47 | — | 3830 | — |
| 5 | 300.85 | 3390.54 | 135.08 | 0.840008 | 3201 | 0.835770 |
median paired end-to-end prefill excluding init ratio = 0.846833 (-15.32% latency); observed range 0.730881..0.877019. median paired chunk-only ratio = 0.844828; observed range 0.718764..0.877700. all 5 pairs faster. ranges are descriptive min..max, not a 95% confidence interval; n=5 does not support a strong CI claim.
hard vs off #
| pair | arm | end-to-end prefill excluding init ms | tok/s | candidate/off end-to-end ratio | chunk-only ms | candidate/off chunk-only ratio |
|---|---|---|---|---|---|---|
| 1 | 0 | 4294.02 | 106.66 | — | 4047 | — |
| 1 | 320335 | 3044.54 | 150.43 | 0.709019 | 2854 | 0.705214 |
| 2 | 0 | 3985.86 | 114.91 | — | 3785 | — |
| 2 | 320335 | 2939.44 | 155.81 | 0.737467 | 2732 | 0.721797 |
| 3 | 0 | 4078.10 | 112.31 | — | 3849 | — |
| 3 | 320335 | 3020.01 | 151.65 | 0.740543 | 2813 | 0.730839 |
| 4 | 0 | 3921.55 | 116.79 | — | 3708 | — |
| 4 | 320335 | 2863.14 | 159.96 | 0.730104 | 2699 | 0.727886 |
| 5 | 0 | 4125.33 | 111.02 | — | 3883 | — |
| 5 | 320335 | 3132.49 | 146.21 | 0.759331 | 2927 | 0.753799 |
median paired end-to-end prefill excluding init ratio = 0.737467 (-26.25% latency); observed range 0.709019..0.759331. median paired chunk-only ratio = 0.727886; observed range 0.705214..0.753799. all 5 pairs faster. ranges are descriptive min..max, not a 95% confidence interval; n=5 does not support a strong CI claim.
tail-soft vs off #
| pair | arm | end-to-end prefill excluding init ms | tok/s | candidate/off end-to-end ratio | chunk-only ms | candidate/off chunk-only ratio |
|---|---|---|---|---|---|---|
| 1 | 0 | 4080.13 | 112.25 | — | 3865 | — |
| 1 | 320300.85 | 4000.26 | 114.49 | 0.980425 | 3756 | 0.971798 |
| 2 | 0 | 4178.27 | 109.61 | — | 3938 | — |
| 2 | 320300.85 | 3523.48 | 129.98 | 0.843287 | 3322 | 0.843575 |
| 3 | 0 | 4214.84 | 108.66 | — | 4017 | — |
| 3 | 320300.85 | 3738.81 | 122.50 | 0.887059 | 3535 | 0.880010 |
| 4 | 0 | 4038.87 | 113.40 | — | 3822 | — |
| 4 | 320300.85 | 3723.68 | 123.00 | 0.921961 | 3488 | 0.912611 |
| 5 | 0 | 4134.25 | 110.78 | — | 3911 | — |
| 5 | 320300.85 | 3614.63 | 126.71 | 0.874313 | 3388 | 0.866275 |
median paired end-to-end prefill excluding init ratio = 0.887059 (-11.29% latency); observed range 0.843287..0.980425. median paired chunk-only ratio = 0.880010; observed range 0.843575..0.971798. all 5 pairs faster. ranges are descriptive min..max, not a 95% confidence interval; n=5 does not support a strong CI claim.
free-generation behavioural preservation #
hard320335 changes greedy free persona generation in4/5 pairs: first difference at generated position3 in pairs1,2,5 and position4 in pair3; only pair4 matches all8 ids. incumbent300.85 and tail-soft320300.85 each match off for all8 ids in5/5 pairs. this is a hard-specific distinction, measured on free generation with default spec1, not teacher-forced prefixes. it is direct short-continuation behavioural divergence, not proof of better or worse output; do not promote hard to another default without task-level validation.
teacher-forced KLD #
reference: fresh arm0 logit dump per corpus; order on each corpus: reference0, then repeat0/incumbent/hard/tail-soft, repeated for cycle2. all main runs exit0. teacher-forced targets are fixed corpus ids, not generated greedy continuations.
eval_tf_chat_json.txt (159 prompt +64 eval) #
| arm | cycle | ref_ppl | cand_ppl | mean_kld | median_kld | top1 | per-arm control comparison |
|---|---|---|---|---|---|---|---|
| 0 | 1 | 1.2153 | 1.2153 | 0.000000 | 0.000000 | 64/64 | delta 0.000000 vs repeat0 floor 0.000000: equal; 0 token difference |
| 0 | 2 | 1.2153 | 1.2153 | 0.000000 | 0.000000 | 64/64 | delta 0.000000 vs repeat0 floor 0.000000: equal; 0 token difference |
| 300.85 | 1 | 1.2153 | 1.2222 | 0.004057 | 0.000008 | 64/64 | delta 0.004057 vs repeat0 floor 0.000000: above; 0 token difference |
| 300.85 | 2 | 1.2153 | 1.1997 | 0.004782 | 0.000014 | 64/64 | delta 0.004782 vs repeat0 floor 0.000000: above; 0 token difference |
| 320335 | 1 | 1.2153 | 1.2534 | 0.011501 | 0.000025 | 64/64 | delta 0.011501 vs repeat0 floor 0.000000: above; 0 token difference |
| 320335 | 2 | 1.2153 | 1.2415 | 0.009995 | 0.000020 | 64/64 | delta 0.009995 vs repeat0 floor 0.000000: above; 0 token difference |
| 320300.85 | 1 | 1.2153 | 1.2176 | 0.005447 | 0.000009 | 64/64 | delta 0.005447 vs repeat0 floor 0.000000: above; 0 token difference |
| 320300.85 | 2 | 1.2153 | 1.1977 | 0.004598 | 0.000011 | 64/64 | delta 0.004598 vs repeat0 floor 0.000000: above; 0 token difference |
boosted-arm run-to-run aggregate KLD spread on this corpus is 0.000725 (300.85), 0.001506 (320335), 0.000849 (320300.85). this is |meanKLD(c1)-meanKLD(c2)| versus the same ref, NOT direct run-vs-run KLD and NOT a confidence interval.
eval_tf_ops_tools.txt (244 prompt +64 eval) #
| arm | cycle | ref_ppl | cand_ppl | mean_kld | median_kld | top1 | per-arm control comparison |
|---|---|---|---|---|---|---|---|
| 0 | 1 | 2.5027 | 2.5027 | 0.000000 | 0.000000 | 64/64 | delta 0.000000 vs repeat0 floor 0.000000: equal; 0 token difference |
| 0 | 2 | 2.5027 | 2.5027 | 0.000000 | 0.000000 | 64/64 | delta 0.000000 vs repeat0 floor 0.000000: equal; 0 token difference |
| 300.85 | 1 | 2.5027 | 2.4035 | 0.055902 | 0.010415 | 60/64 | delta 0.055902 vs repeat0 floor 0.000000: above; 4 token difference |
| 300.85 | 2 | 2.5027 | 2.4070 | 0.055018 | 0.022781 | 59/64 | delta 0.055018 vs repeat0 floor 0.000000: above; 5 token difference |
| 320335 | 1 | 2.5027 | 2.8730 | 0.105676 | 0.024062 | 59/64 | delta 0.105676 vs repeat0 floor 0.000000: above; 5 token difference |
| 320335 | 2 | 2.5027 | 2.8732 | 0.139598 | 0.050984 | 58/64 | delta 0.139598 vs repeat0 floor 0.000000: above; 6 token difference |
| 320300.85 | 1 | 2.5027 | 2.4665 | 0.071406 | 0.019307 | 61/64 | delta 0.071406 vs repeat0 floor 0.000000: above; 3 token difference |
| 320300.85 | 2 | 2.5027 | 2.3810 | 0.045119 | 0.014085 | 59/64 | delta 0.045119 vs repeat0 floor 0.000000: above; 5 token difference |
boosted-arm run-to-run aggregate KLD spread on this corpus is 0.000884 (300.85), 0.033922 (320335), 0.026287 (320300.85). this is |meanKLD(c1)-meanKLD(c2)| versus the same ref, NOT direct run-vs-run KLD and NOT a confidence interval.
what the KLD tables establish #
chat: incumbent mean across two runs0.0044195, tail-soft0.0050225, hard0.010748. incumbent-vs-tail-soft ordering reverses across cycles and is not separable at this repetition count; hard has clearly higher divergence in these draws, not demonstrated lower task quality. all chat arms64/64: top-1 is saturated on chat, not evidence of zero distribution change; on ops it detects reference disagreements, not their correctness.
ops main cycles: incumbent0.055018..0.055902, tail-soft0.045119..0.071406, hard0.105676..0.139598. including the extra incumbent position diagnostic expands its observed range to 0.055018..0.076038, top1=60/64,59/64,57/64. these are three observations, not a reproducibility bound or confidence interval; the two close main-cycle means did not establish stability. incumbent-vs-tail-soft again reverses; hard consistently deviates more in these draws, but only two repeats do not establish a sampling interval. ops incumbent's4-token and5-token disagreements are larger than the one-token caution in trap44. tail-soft3/5 disagreements, hard5/6. a difference of one token between cycles (60/64 vs59/64) is one token, not a trustworthy ranking; neither cycle is near control64/64.
control routing is residency-independent: with field0, bonus0 makes prompt.metal:51 false, so the affinity branch never reads smap:55. boosted arms do read current residency and therefore can change routing with I/O timing. measured repeat0 is0.000000 at six-decimal reporting precision in both cycles on both corpora, but this does not bound treatment variability, nor prove all decode paths universally deterministic. control-vs-control exactness is a property of this configuration, not a universal noise floor. the historical0.0012 figure came from a different configuration and is non-applicable as a floor or acceptance threshold here. agreeing draws at n=2 are not evidence of reproducibility; the third incumbent ops run expands its range by0.021020. no fixed noise floor is estimated from these few draws. the old decode MoCE anchor0.00315 was not reproduced here; no uncertainty interval or precise quality-cost ratio to it is justified.
PPL uses the same fixed corpus target at each position for reference and candidate (cli:1777-1786); KLD averages across the complete reference vocabulary distribution (:1789-1793). lower candidate target-PPL and higher KL are not contradictory: the corpus target is not the reference distribution. no sharpening inference, no invented Monte Carlo standard error, and no “quality improved” claim follows from lower PPL. top1 disagreement is disagreement with reference, not proof that candidate is less correct on every changed position.
n_gen=1 first-token diagnostic #
reuses row0 of the64-row ref. prefill-invariance to n_gen verified by arm0 self-check on both corpora: KL0.000000, top1=1/1; saved row0 is byte-identical to corresponding64-row reference (993280 bytes each). diagnostics are separate runs, not cycle2.
| corpus | off selfcheck | incumbent first KL | hard first KL | tail-soft first KL |
|---|---|---|---|---|
| chat | 0.000000,1/1 | 0.000046 | 0.000017 | 0.000039 |
| ops | 0.000000,1/1 | 0.001761 | 0.006843 | 0.001156 |
all diagnostic argmax comparisons1/1. these first-token values are tiny compared with aggregate KL, not a first-decision concentration. do not subtract a diagnostic first-token value from another run's aggregate to claim an exact body mean: boosted routing varies across runs.
ops inc-64 position diagnostic (extra run, not a cycle) #
extra run KL0.076038, median0.015371, top1=57/64. its first-position KL0.001431 and same-run remaining63 meanKL0.077222 prove this draw's change is in the body, not the first token. this extra draw also shows that the two main-cycle incumbent means being close did not bound its variability.
sorted top4 positions37,34,60,55 contribute49.65% of total KL; top8 contribute72.47%. high-KL and argmax flip are different events. largest values:37 targetThe KL0.873594 (same argmaxThe);34 plan0.674194 (same);60 _complete0.524931 (same);55 [],0.343284 (same argmax [");28 false0.307667 (reference true, candidate false);53 commands0.295313 (both argmaxtask);48 complete0.277288 (reference complete, candidate done);38 fix0.230256 (same).
seven flips: positions8 the→ to;14 calling→ dere;19 pointer→ reference;24 backtick→();28 true→ false;47 complete→ as;48 complete→ done. this includes punctuation and a boolean semantic choice as well as ordinary prose. high-KL structural field pieces also appear without an argmax flip. not all heavy-tail positions are structural, and this is not a measured malformed-JSON/tool-execution failure. in particular boolean position28 agrees with corpus target in the candidate, so “corrupts this true label” would be false. distributional drift and reference disagreements are the demonstrated preservation failure, not demonstrated task harm.
all64 positions, exact target/reference/candidate token text and per-position KL: /tmp/moce_chunk0/OPS_POSITIONS.md and ops_position_kld.json. chat candidate full logits were not retained, so exact chat-vs-ops high-KL positional coincidence is unmeasured; first-token diagnostics only establish that neither aggregate is concentrated at position1. numeric position coincidence across different texts would also need semantic alignment.
SSD reads #
mechanism cross-check: incumbent/off median paired prompt bytes ratio 0.818789 (-18.12%) accompanies prefill latency ratio0.846833 (-15.32%). this is not an inert knob or an isolated timing fluctuation. all figures are prompt counters captured using STRATA_METAL_TRACE=1, not the final decode-only report.
| arm | n | prompt MB median [range] | p_reads_ median [range] | distinct layer-expert lookups median | arena fraction median | ahead fraction median | miss fraction median | p_streamed_ submissions median |
|---|---|---|---|---|---|---|---|---|
| 0 | 15 | 19509.5 [19474.5..19641.7] | 13363 [13340..13454] | 15453 | 0.1485 | 0.7483 | 0.1032 | 13290 |
| 300.85 | 5 | 15970.8 [15965.9..16021.8] | 10988 [10987..11024] | 13354 | 0.1957 | 0.6883 | 0.1159 | 10429 |
| 320335 | 5 | 13095 [13068.9..13137.2] | 8987 [8970..9016] | 11274 | 0.2424 | 0.6251 | 0.1322 | 8525 |
| 320300.85 | 5 | 16475.6 [16441.4..16528.9] | 11326 [11304..11362] | 13642 | 0.1896 | 0.6968 | 0.1137 | 10880 |
fractions: p_hits_/p_lookups_, p_pre_/p_lookups_, p_misses_/p_lookups_, printed by stream.cpp:1189-1192. each lookup is a distinct wanted layer-expert at a cut (:834-856), not one of4580 token-wise choices per layer. hits means existing non-prefetch slot; ahead includes queued/in-flight prefetch slot; misses means absent slot. their fractions sum to1 subject to printed rounding. rising arena fraction and rising missed fraction coexist because ahead fraction falls more. p_streamed_ counts submissions (:893,931,1117), not bytes. bytes are p_bytes_read_; reported MB is decimal and rounded to0.1MB, not exact raw bytes. read counts include asynchronous/predicted reads and are not the same as distinct demand lookups.
incumbent narrows distinct demand from15453 to median13354 (-13.58%) and raises arena-hit share from.1485 to median.1957. representative rounded counts: off ~2295 arena hits +11563 ahead +1595 misses; incumbent ~2613 arena hits +9192 ahead +1549 misses. these are approximate reconstructed counts because fractions print four decimal places. so this is mixed routing change and increased residency selection, not free cache reuse. miss fraction rises while absolute missed lookups slightly fall.
literal representative STRATA_METAL_TRACE=1 prompt reports (complete logs hold every pair):
arm=0: expert stream prompts 1 chunks: 15453 distinct lookups (321.9 per layer), in the arena 0.1485, streamed ahead 0.7483, missed 0.1032; 13290 streamed in (19641.7 MB/chunk, 13454 reads); host 2381.2 ms per chunk, of it waiting 2291.5 ms
arm=300.85: expert stream prompts 1 chunks: 13327 distinct lookups (277.6 per layer), in the arena 0.1961, streamed ahead 0.6897, missed 0.1142; 10429 streamed in (15969.9 MB/chunk, 10988 reads); host 1495.2 ms per chunk, of it waiting 1428.6 ms
arm=320335: expert stream prompts 1 chunks: 11266 distinct lookups (234.7 per layer), in the arena 0.2429, streamed ahead 0.6251, missed 0.1321; 8543 streamed in (13068.9 MB/chunk, 8970 reads); host 1041.6 ms per chunk, of it waiting 980.7 ms
arm=320300.85: expert stream prompts 1 chunks: 13635 distinct lookups (284.1 per layer), in the arena 0.1897, streamed ahead 0.6970, missed 0.1132; 10880 streamed in (16516.7 MB/chunk, 11355 reads); host 1588.9 ms per chunk, of it waiting 1479.7 ms
warmups: first exploratory off run and first TRACE run of each arm discarded, then pairs begin. startup arena reads4129 expert slots (6.50GB) from the same profile each run. literal fill-speed movement:
warm_0: expert arena 6.50 GB, 4129 slots in 2 size classes, 4129 experts read from the profile's top (24576 ranked) in 1.6 s (3.89 GB/s); 8 I/O threads, read-ahead top 6; 1 buffers
warm_trace_0: expert arena 6.50 GB, 4129 slots in 2 size classes, 4129 experts read from the profile's top (24576 ranked) in 2.2 s (2.86 GB/s); 8 I/O threads, read-ahead top 6; 1 buffers
warm_trace_300.85: expert arena 6.50 GB, 4129 slots in 2 size classes, 4129 experts read from the profile's top (24576 ranked) in 2.0 s (3.10 GB/s); 8 I/O threads, read-ahead top 6; 1 buffers
warm_trace_320335: expert arena 6.50 GB, 4129 slots in 2 size classes, 4129 experts read from the profile's top (24576 ranked) in 2.0 s (3.10 GB/s); 8 I/O threads, read-ahead top 6; 1 buffers
warm_trace_320300.85: expert arena 6.50 GB, 4129 slots in 2 size classes, 4129 experts read from the profile's top (24576 ranked) in 1.9 s (3.32 GB/s); 8 I/O threads, read-ahead top 6; 1 buffers
speed_300.85_p1_0: expert arena 6.50 GB, 4129 slots in 2 size classes, 4129 experts read from the profile's top (24576 ranked) in 2.1 s (2.97 GB/s); 8 I/O threads, read-ahead top 6; 1 buffers
speed_300.85_p5_300.85: expert arena 6.50 GB, 4129 slots in 2 size classes, 4129 experts read from the profile's top (24576 ranked) in 2.1 s (2.92 GB/s); 8 I/O threads, read-ahead top 6; 1 buffers
the fill rate moves non-monotonically; interleaving is essential. startup fill time is in init_ms, not prefill_ms, but still diagnoses SSD/thermal state.
counter-reset caveat: strata_window_cli.cpp:1827 -> window.cpp:4137 -> window.cpp:453 -> stream.cpp:1175-1180 erases prompt statistics after prefill and first head. TRACE/PKPROF cause a report before reset; without them final MB/window numbers are decode-only. timer starts cli:1701 after init ends:1664, includes head:1746-1751. no source fix made.
honest conclusion #
recommend setting chunk0 back to0 and retaining these measurements as an optimization rejected pending validation: speedup is real, read-count mechanism is real, and distributional deviation is strongly corpus-specific and exceeds the agreed preservation bar. no task-quality degradation or improvement is established. the conservative deployment rationale is that the existing weight/adapter validation does not cover this perturbation (training-path context supplied by the parent, not independently audited in this measurement), and15% prefill saving does not justify an unvalidated behaviour change under that policy. criterion: exact reproduction of the validated reference behaviour, which chunk-0 affinity does not provide, at an observed magnitude whose future range we cannot bound and a task-quality direction we cannot sign. incumbent is faster than off in5/5 paired runs, but has4/64,5/64,7/64 teacher-forced reference-argmax disagreements on ops versus control0/64. this criterion is a conservative preservation policy, not a measured task-quality verdict. switching from incumbent to off forfeits15.32% latency saving measured relative to off (equivalently off/incumbent median ratio1.180870, about18.09% longer latency relative to incumbent). do not describe that inversion as a15.3% slowdown.
greedy persona GEN_IDS: all5 incumbent/off pairs identical for all8 ids, and all5 tail-soft/off pairs identical. hard/off equality only pair4; first differing generated position (1-based) is3 for pairs1,2,5 and4 for pair3. no claim of full-continuation parity beyond8 ids. teacher-forced GEN_IDS represent argmax predictions at forced prefixes, not a free-generated trajectory.
separate speed blocks support candidate-vs-off claims only. comparing incumbent0.846833 to tail-soft0.887059 is descriptive, not a directly paired incumbent-vs-tail-soft estimate. trace mechanism includes narrower routing as well as increased resident selection; it is not a free cache optimization. no universal quality claim can come from these two64-token corpora. the observed incumbent ops KL range0.055018..0.076038 is roughly12–17 times the two-cycle chat mean0.0044195, a corpus-specific divergence scale, not a relative degradation score.
next #
- build an ops task-level evaluator before changing routing defaults. score candidates against expected corpus/task outcomes: well-formed calls, correct booleans, tool names, argument ids and task completion. KL(ref||candidate) here is exact vocabulary-distribution deviation at forced corpus prefixes; it does not determine whether a changed answer is better or worse. at position28 the candidate boolean agrees with the corpus and the reference does not. neither a reference argmax nor a high KL is an error oracle. retain KL as a preservation/distribution diagnostic, not a replacement for task correctness. until that evaluator exists, leave further MoCE/bonus/residency changes unpromoted; another KL-only sweep cannot settle task quality. no new runs requested here.
- quality-test shipped pos0>0 hard320335 on multi-chunk prompts. do not infer its quality from chunk0 measurements or apply this rejection as a later-chunk result.
window.cpp:3875makes it the later-chunk default, but global env overrides all chunks and cannot isolate a chosen chunk. design per-chunk arms first; no new benchmark or source change done here. - repo evidence located:
audits/opus_prefill_opt.md:82-89gives1024 prompt+64 TF quality checks with prompt affinity disabled and decode-side MoCE comparisons; this does not isolate pos0>0 hard affinity.:112-115reports2048-vs1024 chunk read savings and byte-identical dumps, not hard-affinity quality.audits/argon_r15_prep.md:72-74tests320335 kernel parity/performance, not downstream quality. trap44:217-221records a2487-token five-arm multi-chunk sweep and confounding. therefore there is a documented historical multi-chunk sweep, not “never tested at all,” but I did not locate its raw five-arm quality table0.00508/0.00404 in repository markdown or results logs. parent supplied those older values; they are not independently verified here and do not isolate last-chunk effect. exact later-chunk quality remains unresolved. do not compare spreads across corpora/configurations as a statistical bound on that old table. - fix prompt-read observability only after archiving this run; CLI currently destroys counters without TRACE/PKPROF. parent owns that fix and the default revert.
reproducibility / evidence #
all files under /tmp/moce_chunk0/; provenance.json records exact common argv/env and binarySHA256; each run JSON records argv/arm/exit/duration, each matching log is raw CLI output. root commit86e8e5aade3c7d773f4cff7c90de5bf78bdc771a; vendor commit296b59d2f73c8f04a805df29c2fa97655adace78 with pre-existing dirty changes. vendor_status.txt, measured_sources.diff, source_hashes.json capture relevant state; no edits to repo. fixed6.5GB arena, profile, PLE+model paths from production config; prompt chunk2048 via env, ctx4096, supplied MTP path with CLI default spec1. ignored/nonexistent flags (--temp,--prompt-chunk,--experts) were not used. .experts discovered adjacent to GGUF, confirmed raw logs. all measurements serialized; no live server touched. PACKED_MOCE.md gives encoding: incumbent body3+bonus~0.85; hard body3+bonus35+last32 token rows exact; tail-soft body3+bonus~0.85+tail32. tail is positional, not recently arrived experts.
validation: 62/62 CLI executions exit0; binary hash unchanged; measured source hashes unchanged; 30 retained speed executions; 16 compared main teacher-forced runs plus2 references;8 first-token diagnostic executions;1 extra ops64 position run;5 discarded warmups. numpy rowwise KL agrees with extra CLI within displayed rounding,7/64 argmax disagreements verified. first-row control dumps byte-identical to full64 reference row0 on both corpora.