round 40: add apple silicon README, setup script, and complete strata metal diff
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980`swift-1.5-qwen3.8-flash-next` (`qwen4_exp`, 512 experts, 36 gated deltanet + 12 sparse attention layers + 51.2b n-gram ple table + neural mtp draft head) running on apple silicon via a patched `strata` metal engine.
`./scripts/setup_apple_silicon.sh` clones the pinned `strata` and `llama.cpp` trees into `vendor/`, applies `scripts/strata_metal_complete.diff`, downloads apple's `metal-cpp` headers, compiles the metal engine (`strata_window_cli`, `convert_k256_q2_0`, `heal_k256_q2_0`), and streams + repacks the unpruned `k=512` `iq2_xs` model (`3.90 gb` dense `.gguf` + `36.23 gb` `.experts` + `28.80 gb` `ple-iq4_nl.gguf` + `889 mb` `mtp-q2_0.gguf`) into `models/K512_iq2_xs/`.
`./scripts/serve_k512_local.sh` starts the local server on `http://127.0.0.1:8080` (`/v1/chat/completions` and `/v1/messages`, 32k context, `6.5 gb` wired expert arena, neural mtp + suffix speculative decoding).
for the `k=256` reap-pruned checkpoint (`https://huggingface.co/90-008/Swift-1.5-Qwen3.8-Flash-Next-REAP-Pruned-GGUF`, which also hosts `k=320` and `k=224`), run `python3 scripts/download_k256_local.py`, `uv run --with regex python3 scripts/prepare_strata_k256.py` (optionally `python3 scripts/run_phase3_local_healing.py` for the healed `q2_0` build), and `./scripts/serve_k256_local.sh`.
research notes, ablations, and benchmark logs live in `PLAN.md`, `blueprint_review.md`, `audits/`, and `results/`.
## requirements
macos 26+ on apple silicon (m5 required for the default `coop_k4` `matmul2d` cooperative-tensor path; tested on macos 27 on a 24 gb m5 macbook air), xcode command line tools, `cmake`, `ninja`, `python3`, and `uv`.
## installation
```sh./scripts/setup_apple_silicon.sh./scripts/serve_k512_local.sh```
if you only want to build the engine without downloading the ~70 gb of model files:
```shSKIP_MODEL_DOWNLOAD=1 ./scripts/setup_apple_silicon.sh```
## optimizations
### model, pruning & quantization- unpruned `k=512` `ukisai` gsq-rco `iq2_xs` metal bundle (`3.90 gb` 16 kib-aligned dense `.gguf` + `36.23 gb` contiguous `[gate | up | down]` `.experts`, `blk.8/13/37` `iq1_m` replaced with `iq3_s`/`iq2_xxs`, duplicate `28.80 gb` ple table omitted)- 131k-token multi-domain reap + zero-prior damped lcb expert pruning (`k=320` `reap_hybrid` and `k=256` / `k=224` `reap_l2_sum`)- signed-`d` (`fp16` sign-bit polarity switch) activation-weighted lloyd-max `q2_0` expert quantizer (`convert_k256_q2_0` / `heal_k256_q2_0`, shrinking `k=256` experts from `23.42 gib` across 3 size classes to `15.82 gib` in 1 unified 85-page slot class; `k=256` path)- real-trace per-layer `alpha_l` mse scale calibration + 48-layer `ffn_down_shexp` ridge regression healing with byte-level gguf transplant (`K256-q2_0-healed-metal.gguf`; `k=256` path)- native 49th-layer neural mtp draft pack (`mtp-q2_0.gguf`, `889 mb`, runtime `q8_0` dense quantization + `madvise(MADV_DONTNEED)` on `181 mb` bf16 source matrices + `707.8 mb` `mlock`ed `q2_0` routed experts + `40,525`-token `draft_vocab_en.bin`)- `reasoning_effort|default('low')` chat template default
### ssd expert streaming, unified memory arena & moce routing- zero-copy file-backed `mmap` of `<model>-metal.gguf` with selective `mlock` on active gpu dense tensors and `MADV_DONTNEED` on cpu-only `token_embd` and skipped bf16 `hc_*` originals- wired metal expert arena (`STRATA_METAL_WIRE_ARENA=1`, `STRATA_METAL_MLOCK_DENSE=1`, fixed `make_arena` class-sizing loop) + separate on-demand `MAP_ANON` `PromptGemm` scratch buffer (`munmap`ped + `cx_->free`d immediately after prefill)- dynamic `n_expert` (`k = 224 / 256 / 320 / 512`) across host and metal router/moe shaders- closed-loop moce cache-prior routing (`STRATA_METAL_MOCE_BONUS=0.85`, top-3 unperturbed protection, race-free single-read `smap` bitmask in `moe_route` / `router_route` / `route_topk`, deterministic `smap_` publication at layer/cut boundaries, and opt-in rank-1 bonus unscaling fix)- `acceptmoe` speculative draft routing (`STRATA_METAL_DRAFT_MOCE=1`, `r_prot = 1` and squared `moce_scale` on draft rows `sg > 0`) + root-token-only decode prefetch (`STRATA_METAL_PREFETCH_DRAFT=0`, `STRATA_METAL_PREFETCH_UNPERT=1`)- `q8_0` cross-layer lookahead router (`STRATA_METAL_PRED_Q8=1`) and lossless `bf16` main router (`STRATA_METAL_ROUTER_BF16=1`)- token-0-only router-guess gemv fast path (`tile_ntok` in `Window::gemv`) across all 93 `predL_` / `predL2_` dispatches when `STRATA_METAL_PREFETCH_DRAFT <= 0`- resident-first 2-phase routed moe execution (`STRATA_METAL_EXPERT_SPLIT=1`, `STRATA_METAL_SHEXP_OVERLAP=1`, `STRATA_METAL_SPLIT_TURN=1`) with `MTLSharedEvent` ssd handshake and `P_GSLOT` plan slot pinning- prefill next-layer (`l + 1`) router lookahead before layer `l`'s ssd cut, most-tokens-first demand queueing, `layer_rank_` hottest-first background prefetch, `in_prompt_` single-piece `pread`, `STRATA_METAL_PROMPT_CHUNK=2048`, and tail-protected `STRATA_METAL_PROMPT_MOCE`- multi-chunk (`n > C`) hot-expert lru and `!hot` scratch-pool preservation across prompt chunks + `allow_hot = false` on initial layer-0 `hold_region` prefetch + separate prompt/decode ssd read counters- asynchronous background ple ssd `gather_batch` thread during prefill + direct-mode ple `prefetch_rows` duplicate-slot reuse and error-drain in `PleTable::gather_batch`
### speculative decoding, kv sessions & suffix cache reuse (`scr`)- single-command-buffer gpu neural mtp draft chain (`mtp_chain.metal`, `STRATA_METAL_MTP_CHAIN=1`) with `SuffixDrafter::propose_after` hybrid continuation- 64-cell attention sink (`STRATA_METAL_MTP_SINK=64`) + 256-cell local sliding window (`STRATA_METAL_MTP_WIN=256`, synced with `mtp_prefill` `tail_cap`) + `mtp_wrote` valid-run kv tracking + dead mtp indexer elimination (`QSA_ROW0=24` k/v-only `qsa_prep`)- 24 kib `tail_bak_` qsa raw-key snapshot/restore in `Window::run` / `Window::commit` for unrestricted `T <= 4` speculative rollback across 4-cell qsa block boundaries- opt-in online cost-model `SpecGate` (`STRATA_SPEC_GATE=suffix|1`), step-aware `STRATA_METAL_SPEC_MIN_P_SCHEDULE` / `STRATA_METAL_SPEC_MIN_CUM_P`, and exact/typical `temp > 0` speculative verification (`STRATA_SPEC_SAMPLE=exact|typical`)- 4-thread `F_NOCACHE` nvme session save/load (`--save-session` / `--load-session`, `Window::session_save` / `session_load`) serializing all 36 gdn + 12 qsa + ple states directly from/into `MTLBuffer`s- suffix cache reuse (`scr`, `Window::scr_relocate` + `strata::core::scr_plan` + `scr_carry` `0..3` newline pad insertion for `Δ ≡ 0 (mod 4)` + `<path>.pads` + 2-slot mru `ck_ring` in `--serve`) relocating qsa group-0 neox rope keys/pooled keys in place while retaining post-edit 36-layer gdn state, including automatic `[A B C] -> [A C D]` stripped-`<think>` multi-turn reuse
### apple silicon metal decode kernels (`ntok = 1..8`)- `gemv.metal`: m5 per-format/per-shape/per-`ntok` tile selector in `Window::gemv_tile` & `mtp_encode::tile` (`{32,2,4,1}` for `2560->512`, `{8,2,4,1}` for `q2_0 640->2560` at `ntok >= 4`), `#pragma clang loop unroll(full)` across `NTOK`/`NR`/`KS` loops, `fmt<F>::TAB > 1` barrier guards, `gemv_cat` (`ssm_alpha + ssm_beta`, mtp `k + v` / `gate + up`) & `q8_0` `gemv_catk`, `gemv_direct_x` (`q8_0`), `gemv_q2_raw_combine` with ieee-754 fp16 `0x64006400u` mantissa-or decode (`q2_0r`), and 5-register `packed_uint4` `gemv_iq4_combine` (`iq4_xs`)- `moe.metal` & `quant.metal`: single-buffer `q[UNIT]` gate/up register reuse (`NE > 1`), `NE == 1` pointer/activation hoisting in `gu_body`/`dn_body`, `UNIT == 32` fused `qg`/`qu` decode (`q6_k`), `q2_combine<F>` (`q2_0`/`q2_0r`/`iq2_xs`), 6-vgpr `deq_iq_packed` + `iq_combine`, ieee-754 fp16 `0x64006400u` mantissa-or decode in `q2_combine<T_Q2_0R>`, 64-bit `uint2` grid loads in `deq<T_IQ2_XXS/IQ2_XS/IQ2_S>`, `ushort` `q2`/`s2` + pre-shifted `qh` in `deq<T_IQ2_S>`, `sign4_iq2xs` + dead sign-table (`ksg`/`load_signs`) removal + shrunk `TAB` (`1024`/`512`/`2048`), `QUANT_CTAB` constant-memory tables (`iq3_s` pinned to threadgroup `tab`), m5 `gl=16` / `dl=8,dr=2` tiles, `plan_body` `topk10` `((i * 205) >> 11)` fast division + `unroll_count(10)` + `sg == 0` `simd_prefix_exclusive_sum`, and `moe_route` `e512` + preloaded 16-bit `res_mask`- `window.cpp` decode graph: wave-zipped resident routed experts (`phase = 1`) with shared-expert gemvs and `swiglu_prep`, deferred moe combine (`comb_defer`) folded into 4-tg-per-token `gr_norm_comb` (`gr2.metal`) and layer-0 `ple_comb_query_gate_norm`, and load-time `window_init` pso prewarming- `gr.metal`, `glue.metal`, & `units.metal`: 128-thread `gr_down_*_4_4` (`ntok <= 4`) and 2-row-per-simdgroup `gr_down_*_4_4_r2` (`ntok > 4`) with direct `float4` fma, packed `char4` `GrdUnit<T_Q8_0, 2>` with `ROLL = true` at `KS = 4`, `GR_NTOK` loop unrolling, program-scope `constant` `gr_q2s`/`gr_p4s` & `glue_q2s`/`glue_p4s`, barrier-free `simd_shuffle_down` in `gr_up_f` and `swiglu_prep`, `n == 2560` unroll + `apply` hoist + pre-offset pointers in `gr_norm`, `float4` vectorized `xprep`, and `topk == 10` unrolled + pre-offset `moe_combine_sh` (`STRATA_HC_Q8=1` default, opt-in `STRATA_HC_Q4`/`STRATA_HC_Q5`)- `gdn.metal`: `gdn_step_t` `simd_broadcast`, `ib == 0` readout pruning, 32-bit `st_off` state offsets, `float4` `conv_w`, program-scope `constant` `gdn_q2s`/`gdn_p4s`, `5 -> 4` barriers/token with token-invariant `dt`/`ssm_a` hoist + shrunk `part[8]`/`red2[IB-1][S]`, and critical-path `kl` split across barrier 2 + pre-barrier-1 `hist_out` writeback + pre-barrier-3 global `state` store overlap- `qsa.metal` & `qsa2.metal`: parallelized `qsa_prep` (`320 b` threadgroup memory, `4 -> 2` barriers), hoisted/unrolled `qsa_score`, register-cached `keys[4]` + `20 -> 13` radix barriers in `qsa_topk2`, `char4` `K` loads + register-cached softmax `s0, s1` + unrolled `V` x4 in `qsa_attn`, and barrier-free `simd_shuffle_down` + shared `ek[]`/`L_sh` + `float2 (m, l)` loads + `simd_max` in `qsa_merge`- `ple.metal`, `mtp.metal`, `mtp_chain.metal`, & `sample.metal`: fused `ple_comb_query_gate_norm` and `ple_query_gate_norm`, register-cached `ple_gnorm`, `ple_conv_res` (eliminating `gated` buffer), ping-pong `part[16]` barrier halving across all `ple.metal` norms, `[t][c]` multi-row stream batching for `mtp.fc_hidden`, fused `mtp_rms_xprep` & `mtp_combine_gnorm`, `float4`/unrolled `mtp_xprep`/`mtp_rms_xprep`/`mtp_combine_gnorm`/`mtp_embed`/`mtp_pick`/`mtp_slop_pen`, and unrolled `tg_argmax` in `sample.metal`
### apple silicon metal prefill (`prompt_chunk`) kernels- `window.cpp` prefill graph & scratch: `-1,247` redundant barriers removed + router/plan/shared-expert wave zipping (`-190` barriers) + `sigs` right after `route_topk` so moe plan and shared-expert gemms overlap host/ssd `at_cut(l)` + `Wr`/`Wr2` parity ping-pong (`-47` router `dq_f16` dispatches/chunk) + fused gdn `ssm_alpha + ssm_beta` (`N = 96`) gemm (`-36` dispatches) + fused qsa `Wk + Wv + Wik (+ Wiq)` (`N = 1152 / 1664`) gemm (`-24..36` dispatches) + `qsa_score_mm`/`qsa_topk` pso prewarming + multi-stage `PromptGemm` scratch aliasing (`2.35 gb -> 1.12 gb` at `MT = 2048`)- `dense_mm.metal` & `mma.metal`: m5 shape-aware `gemm_f16` tile selector (`M > 1536` threshold, `16_64_2`, `32_64_2`, `32_128_2`, `64_32_2`, `16_96_2` for `Walbe`, and `64_128_4` when `M < 1536 && M % 128 in 1..64`), hardware 2d-grid `dq_f16` with `half4` stores + `TAB == 1` barrier elimination + `T_Q5_1` + register table init & `perm == 4` contiguous writes in `dq_f16<T_IQ4_XS/IQ4_NL>`, unrolled `DW<T_IQ4_XS/IQ4_NL/Q6_K>`, and `half4` `act_f16`- `prompt.metal`: folded `mm_combine` + `moe_out` into per-token 4-stream `gr_norm_h_moec` and `gr_apply_moec` (plus `gr_norm_h4`) with register-cached `rv`, stream-by-stream reduction, compacted `moe_comb_slots` `roff`, and `k`-outer loop inversion; cooperative-tensor register epilogues in `gemm_hc_up_reg_32_128_4` (unbound dead `p.mixed`, unrolled epilogue, `hc_up_tile<16/32>` row-tail trimming), `gemm_hc_down` (fusing `gr_epi_down`, `* 0.25f`, `TN = 96` tail tile, guarded `TM > 32` row-tail trimming), and `gemm_swiglu_up` (fusing `swiglu_h`, guarded `TM > 32` row-tail trimming); `n_exp == 512` unrolled `route_topk`- `moe_mm.metal` & `ple.metal`: `mm_clear` elimination (pad-clear folded into `mm_scan`), 32-subblock nibble-counted `s_id`-free `mm_place`, 128-bit `uint4` fp16 `mm_gather_h`, single-`rw` `mm_gu3` / `mm_down3` with `TAB > 1` barrier guards, hoisted row pointers (`g0_l`, `u0_l`, `w0_l`) and `boff`/`sboff`, hoisted `tab + hh`, branchless `s >> (4 * hh)` in `W8<T_IQ3_S/IQ2_S>` + `W8<T_IQ3_S>::block`, 256-wide super-block scale caching `dg[]`/`du[]` (`T_IQ2_S/IQ2_XXS/IQ2_XS`), `W8<T_IQ2_XXS>` direct byte loads, `sign4_iq2xs` + constant-offset pointer folding, scalar `uint` or in `W8<T_Q6_K>` / `W4<T_Q6_K>`, `half4` coalesced `HS` store epilogue (`T_IQ2_XS/IQ2_XXS/IQ3_XXS/Q2_0/IQ2_S/IQ3_S`), 64-element block stepping (`k0 += 64, w0_l += 18`) + ieee-754 fp16 `0x64006400u` mantissa-or decode in `W4<T_Q2_0>::run_q` (`mm_down3`), and 1-tg-per-token `ple_moe_gate_norm_c` with simd slot compaction + `ple_conv_res` (`half4` weights, branchless `t >= 9`) + `ple_hist_adv`- `qsa_prompt.metal` & `gdn_prompt.metal`: `qsa_prep#qf16` fused fp16 query output, `qsa_score_mm` causal-future tile skip (`b0 >= max_bid`) + `dead[0..3]` hoist, `qsa_topk` cached `keys[4]` + simdgroup bucket scan + 2-barrier fused `(equal, greater)` emit (and skipped when `p.flash && pos0 + T <= 2051`), `qsa_flash` 4-simdgroup online softmax with `sp[4][HPG*QCH]` (`13,760 b` threadgroup memory, 1 barrier/tile) + unconditional hoisted `rv`/`rk` fills + byte-offset/zero-scale hoist + 32-bit `ko[8]`/`vo[8]` offsets with pre-added `d0 + kl` + `c0 < nc` pv bound + `vec4` epilogue, 1-simdgroup-per-head barrier-free `gdn_pre` (`float4` value-head conv, vector `simd_sum(float4)`) & `gdn_post` (hoisted pointers, preloaded gate/gamma, vector `simd_sum(float4)`), and single-pass `TB = 0` `simd_broadcast` `gdn_scanr_4` with 32-bit base offsets + `hk`-grouped head scheduling + final-token dead-lookahead elimination
### anti-slop, persona data & serving- `slop4` (`slop4.h`) per-position stateful shaper (`12 b/pos` in `window.cpp` + gpu `mtp_slop_pen` in `mtp_chain.metal` with `STRATA_METAL_MTP_SLOP=1`), gated strictly on `PROSE && !TOOL && !FENCE`, with clause-entry branch-point penalties, `slop4::hold()` `<|im_end|>` mass preservation, `</think>\n\n` `step0` empty-turn guard, `slop_scan`, and `scr_relocate` support- phase 4 `v3.2` layer-47 special-token-orthogonal per-word-null wfld edit (`blk.47.ffn_down_shexp.weight` + prefix-conditioned penalty in `Window::finish`; `k=256` path)- monroe prior log-odds llm-ism miner (`mine_llm_isms_from_transcripts.py`, pure per-order unigram+uniform prior with zero self-pooled weight) partitioning structural speech acts (`phase3_sentence_surgery` in `prepare_phase3_sft_data.py`, in-place span deletion preserving markdown/fences/kaomoji + r-tuning word-boundary abstention) vs surface lexical cliches (`phase4_lexical_ftpo`)- strata `v0.1.45` `serve/` & `tools/strata_tokenizer.py` ports + local hardening: incremental `PromptEncoder` prefix/plain-span reuse capped at `PROMPT_ENCODER_KEEP_MAX = 16` with rejected-prompt forgetting, bounded tool-call parser scan cursor, content-keyed json lru logit-bias cache (stale `_last` identity shortcut removed), macos iokit/metal telemetry (`MTLCreateSystemDefaultDevice` cached once and `objc_release`d), `STRATA_OMIT_IMAGES_WHEN_NO_VISION=1` text-only image omission, and anthropic hard thinking budgets + `answer_room` answer reservation- standalone `llama.cpp` `-cmoe` apple silicon apfs / mmap patch (`scripts/patch_m5_lazy_moe.py`)12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576# round 40 opus review: repo organization, README, fresh-clone path
read-only. i inspected the files and replayed the diff into `/tmp/r40_apply`; nothing was built and no models were downloaded.
## blockers
1. **setup crashes at the end of the ~70 GB download, on a stock python3.** `scripts/setup_apple_silicon.sh` runs `python3 scripts/build_k512_iq2_xs_metal.py` and `python3 scripts/build_mtp_q2_0.py` with the bare interpreter. - the k512 builder ends with `ST.extract(...)`. `vendor/strata/tools/strata_tokenizer.py` imports `regex` at module top, so a fresh mac python3 dies after the dense, expert and PLE downloads, before it writes `tokenizer/` and `strata-k512-iq2xs.json`. - `build_mtp_q2_0.py` runs `vendor/strata/tools/mtp_pack.py`, which imports `numpy` and `gguf` (llama.cpp's gguf-py, itself numpy-based). - fix: run both through `uv`, which the README already requires, as the serve scripts do: `"$UV_BIN" run --with regex --with numpy python3 …`. `mtp_pack.py` is started via `sys.executable`, so it inherits the uv env. `prepare_strata_k256.py` has the same `regex` dependency when `tokenizer/` is missing; the README command should be `uv run --with regex python3 scripts/prepare_strata_k256.py`.2. **the README promises k=320 / k=224 but the scripts only handle k=256.** `download_k256_local.py` and `prepare_strata_k256.py` hardcode `K256_reap_l2_sum` and `NE = 256`. either say "k=256" only, or add a `--k` argument.3. **the macOS requirement is probably wrong.** the README says "macos 15+". `dense_mm.metal`, `moe_mm.metal`, `qsa_prompt.metal` and `mm_bench.metal` include `<MetalPerformancePrimitives/MetalPerformancePrimitives.h>`, which ships with macOS 26. `Window::m5_layout()` → `coop_k4()` compiles `mm_bench.metal` unconditionally on the prompt path. `Context::supports_metal4()` exists but nothing in the engine calls it. upstream `docs/MAC.md` says 15+, but our tested system is macOS 27 on the M5. unless someone has run it on 15, write "macos 26+ (tested: macOS 27, M5)".
## (1) hardcoded user paths
- zero `/Users/dawn` (or any `/Users/`) in `scripts/` (setup, serve, build, prepare, download, mtp) and zero in `scripts/strata_metal_complete.diff`. the serve scripts fall back to `$HOME/.local/bin/uv` only when `uv` is not on PATH.- the generated configs (`strata-k512-iq2xs.json`, `strata-k256.json`) contain absolute paths computed from `ROOT` at build time, under the git-ignored `models/`. that is fine. the comment "portable repo-relative" in `build_k512_iq2_xs_metal.py:348` is inaccurate (they are absolute, per machine); reword it.- `download_k256_local.py` uses the cwd-relative `Path("models/K256_reap_l2_sum")`. run from anywhere but the repo root, it downloads 2 shards into the wrong place. use a `ROOT` from `__file__` like the other scripts.
## (2) README opts: test-only or rejected items listed as live?
checked against the engine dispatch sites (`metal/src/**`) and env defaults:- the round 39 rejections are absent: no cuda butterfly, no `qsa_merge` 4-partial batching, no `qsa_merge2` batching. the `qsa_merge` items listed are the earlier live ones (dispatched at `window.cpp:1285/2737/3662`, `prompt_chain.inc:262`). `qsa_topk2` is live (dispatched under the `qsa_topk` key).- defaults match the README wording: `STRATA_METAL_MTP_CHAIN` (on unless "0"), `MTP_SINK` 64, `MTP_WIN` 256, `PREFETCH_UNPERT` 1, `STRATA_HC_Q8` on, `MTP_SLOP`/`DRAFT_MOCE`/`PUBLISH_PREFETCH` on. the opt-ins are labeled as such (`STRATA_SPEC_GATE`, `STRATA_SPEC_SAMPLE`, `STRATA_HC_Q4/Q5`, rank-1 unscaling). `--save-session` and `--load-session` exist in `strata_window_cli`. the helpers `gemv_q2_raw_combine`, `gemv_iq4_combine`, `gemv_direct_x` and `tg_argmax` are inlined into dispatched kernels.- wording to check: "`qsa_flash` … unconditional hoisted `rv`/`rk` fills". if that refers to round 39's clamped-row / value-select port, it only runs on the non-cooperative (M1/M4 or `COOP_K4=0`) path, not on the M5 default. say so, or drop it. if it is the earlier cooperative-path hoist, it is fine.- k=256-only artifacts are listed without that scope: the healed `K256-q2_0-healed-metal.gguf`, the `q2_0` quantizer, `alpha_l`/ridge healing and the layer-47 wfld edit. the default `k=512` service uses none of them, and nothing in the README or scripts builds the healed gguf (`prepare_strata_k256.py` silently falls back to the unhealed IQ3_S). either tag those bullets "(k=256 path)" and give the convert/heal commands, or mark them as research artifacts.- "m4 / m5 recommended for matmul2d cooperative tensors": only the M5 gets the `coop_k4` matrix-unit layout; the M4 takes the older prompt kernels (`prompt_chain.inc:10-11`). "m5 (tuned); m4 runs the older prompt kernels" is more accurate.
## (3) fresh-clone completeness
- `scripts/strata_metal_complete.diff` reproduces the patched tree exactly: applied to `git archive 296b59d2` of the 58 pre-existing paths, plus the 20 new files, it gives 78/78 files byte-identical to the current `vendor/strata` working tree. the working tree has no changes outside the diff (excluding `third_party/metal-cpp`, `build-metal`, `__pycache__`). the diff has no binary hunks. `git apply` exits 0 with one whitespace warning (the known blank line at the `qsa.metal` EOF).- everything the build and configs need is either upstream at the pinned commit (`metal/tools/metal_repack.py`, `tools/mtp_fetch.py`, `tools/mtp_pack.py`, `data/expert-profile.bin`, `data/draft_vocab_en.bin`) or in the diff (`strata_window_cli.cpp`, `convert_k256_q2_0.cpp`, `heal_k256_q2_0.cpp`, `slop4.h`, `scr_state.hpp`, `spec_gate.hpp`, …). the CMake variables `METAL_CPP_DIR` and `STRATA_GGML_DIR` match the setup invocation.- repo-tracked inputs exist: `results/reap_saliency_v2.json`, `results/keep_maps/keep_reap_l2_sum_k256.json`, `mine_llm_isms_from_transcripts.py`, `scripts/prepare_phase3_sft_data.py`, `scripts/patch_m5_lazy_moe.py`. `README.md` and `scripts/setup_apple_silicon.sh` exist only in the jj working-copy change (git shows them as `A`, not in the last commit), so make sure they land in the round 40 commit.- minor: setup's `git -C vendor/strata clean -fd` wipes untracked non-ignored files on every re-run (metal-cpp gets re-downloaded, sha-checked). harmless on a fresh clone, surprising in a dev tree.- minor: the k512 builder caches the source GGUF headers in `/tmp/iq2xs_s1.hdr` etc. and only checks their size, so a stale header from another upload would be reused silently. low risk while the URLs are pinned.286 unmodified linesExpand all2872882892902912922932942952962972982993003013023033043053063073083093103113123133143153163173183193203213223233243253263273283293303313323333343353363373383393403413423433443453463473483493503513523533543553563573583593603613623633643653663673683693703713723733743753763773783793803813823833843853863873883893903913923933943953963973983994004014024034044054064074084094104116 unmodified lines286 unmodified linesExpand all os.fsync(fd_e) finally: os.close(fd_e)
# Optional standalone 1-tensor PLE GGUF (per_layer_token_embd.weight, 28.80 GB IQ4_NL) ple_k256 = ROOT / "models" / "K256_reap_l2_sum" / "Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S-K256_reap_l2_sum-00001-of-00002.gguf" ple_out = out_dir / "ple-iq4_nl.gguf" ple_path = ple_k256 if ple_k256.exists() else ple_out if ("--with-ple" in sys.argv or not ple_path.exists()): ple_t = next(t for t in r1.tensors if t["name"] == MR.PLE) ple_hdr = bytearray(struct.pack("<4sIQQ", b"GGUF", 3, 1, 0)) ple_hdr += pack_str(ple_t["name"]) ple_hdr += struct.pack(f"<I{len(ple_t['dims'])}QIQ", len(ple_t["dims"]), *ple_t["dims"], ple_t["type"], 0) ple_data0 = (len(ple_hdr) + 31) // 32 * 32 ple_hdr += b"\0" * (ple_data0 - len(ple_hdr)) ple_final_size = ple_data0 + ple_t["bytes"] if not ple_out.exists() or ple_out.stat().st_size != ple_final_size: print(f"Downloading standalone PLE table ({ple_t['bytes']/1e9:.2f} GB) to {ple_out.name}...", flush=True) tmp_ple = ple_out.with_name(ple_out.name + ".part") fd_p = os.open(str(tmp_ple), os.O_CREAT | os.O_RDWR | os.O_TRUNC, 0o644) set_nocache(fd_p) try: os.pwrite(fd_p, bytes(ple_hdr), 0) os.ftruncate(fd_p, ple_final_size) win = 64 * 1024 * 1024 offs_p = list(range(0, ple_t["bytes"], win)) q_p = [] nxt_p = 0 while nxt_p < min(8, len(offs_p)): off_p = offs_p[nxt_p] ln_p = min(win, ple_t["bytes"] - off_p) q_p.append((off_p, start_blob(ex, U_IQ2_S1, ple_t["abs"] + off_p, ln_p, chunk_mb=16))) nxt_p += 1 while q_p: off_p, job_p = q_p.pop(0) if nxt_p < len(offs_p): off2 = offs_p[nxt_p] ln2 = min(win, ple_t["bytes"] - off2) q_p.append((off2, start_blob(ex, U_IQ2_S1, ple_t["abs"] + off2, ln2, chunk_mb=16))) nxt_p += 1 blob_p = collect_blob(job_p) os.pwrite(fd_p, blob_p, ple_data0 + off_p) del blob_p os.fsync(fd_p) finally: os.close(fd_p) os.replace(tmp_ple, ple_out) ple_path = ple_out
# Extract tokenizer from GGUF metadata and set default reasoning_effort to 'low' tok_dir = out_dir / "tokenizer" if not (tok_dir / "vocab.json").exists(): sys.path.insert(0, str(ROOT / "vendor" / "strata" / "tools")) import strata_tokenizer as ST ST.extract(str(gguf_out), str(out_dir)) jinja_p = tok_dir / "chat_template.jinja" if jinja_p.exists(): jtxt = jinja_p.read_text(encoding="utf-8") if "reasoning_effort|default('xhigh')" in jtxt: jinja_p.write_text(jtxt.replace("reasoning_effort|default('xhigh')", "reasoning_effort|default('low')"), encoding="utf-8")
# Write strata-k512-iq2xs.json using checkout ROOT cfg_path = out_dir / "strata-k512-iq2xs.json" cfg = { "exe": str(ROOT / "vendor" / "strata" / "build-metal" / "strata_window_cli"), "args": [ "--metal-gguf", str(gguf_out), "--ple-gguf", str(ple_path), "--mtp-gguf", str(out_dir / "mtp-q2_0.gguf"), "--n-expert", "512", "--max-context", "32768", "--spec", "3", "--spec-min-p", "0.35", "--suffix-draft", "3", ], "cwd": str(ROOT / "vendor" / "strata"), "tokenizer": str(tok_dir), "model_name": "swift-1.5-k512-iq2xs", "log": "/tmp/strata-k512-serve.log", "lib_dirs": [], "port": 8080, "backend": "metal", "fit_max_tokens": True, "sampling": { "temperature": 0.6, "top_p": 0.95, "top_k": 40, "min_p": 0.05, "presence_penalty": 0.4, "repeat_penalty": 1.08, }, "env": { "STRATA_METAL_ARENA_GB": "6.5", "STRATA_METAL_MLOCK_DENSE": "1", "STRATA_METAL_WIRE_ARENA": "1", "STRATA_METAL_EXPERT_PROFILE": str(ROOT / "vendor" / "strata" / "data" / "expert-profile.bin"), "STRATA_METAL_MOCE_BONUS": "0.85", "STRATA_METAL_EXPERT_SPLIT": "1", "STRATA_METAL_SHEXP_OVERLAP": "1", "STRATA_METAL_SPLIT_TURN": "1", "STRATA_METAL_PRED_Q8": "1", "STRATA_METAL_ROUTER_BF16": "1", "STRATA_METAL_MOE_DN_TILE": "8,2,4", "STRATA_METAL_PREFETCH": "6", "STRATA_METAL_PREFETCH2": "2", "STRATA_METAL_IO_THREADS": "8", "STRATA_METAL_IO_URGENT": "2", "STRATA_ANTI_SLOP_PENALTY": "9.0", "STRATA_ANTI_SLOP_HONEST": "6.0", "STRATA_ANTI_SLOP_MD": "4.0", "STRATA_ANTI_SLOP_COLON": "3.0", "STRATA_METAL_PROMPT_CHUNK": "2048", "STRATA_METAL_PREFETCH_DRAFT": "0", "STRATA_METAL_PUBLISH_PREFETCH": "1", "STRATA_METAL_MTP_SLOP": "1", "STRATA_METAL_DRAFT_MOCE": "1", }, "reasoning_effort": "low", } cfg_path.write_text(json.dumps(cfg, indent=2) + "\n", encoding="utf-8")
print( f"COMPLETE in {time.monotonic()-t0:.1f}s! " f"GGUF={gguf_out.stat().st_size/1e9:.3f} GB, Experts={exp_out.stat().st_size/1e9:.3f} GB", flush=True,6 unmodified lines12345567891044 unmodified linesimport os, time, urllib.requestfrom pathlib import Path
repo = "90-008/Swift-1.5-Qwen3.8-Flash-Next-REAP-Pruned-GGUF"out_dir = Path("models/K256_reap_l2_sum")ROOT = Path(__file__).resolve().parent.parentout_dir = ROOT / "models" / "K256_reap_l2_sum"out_dir.mkdir(parents=True, exist_ok=True)tok_path = Path.home() / ".cache/huggingface/token"tok = tok_path.read_text().strip() if tok_path.exists() else ""
44 unmodified lines193 unmodified linesExpand all1941951961971981992002012022032042052062072082092102112122132142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562572582592601 unmodified line193 unmodified linesExpand all root / "results" / "keep_maps" / "keep_reap_l2_sum_k256.json", out_prof, ) build_metal_gguf_and_experts(shard1, out_gguf, out_exp)
tok_dir = model_dir / "tokenizer" if not (tok_dir / "vocab.json").exists(): sys.path.insert(0, str(root / "vendor" / "strata" / "tools")) import strata_tokenizer as ST ST.extract(str(out_gguf), str(model_dir)) jinja_p = tok_dir / "chat_template.jinja" if jinja_p.exists(): jtxt = jinja_p.read_text(encoding="utf-8") if "reasoning_effort|default('xhigh')" in jtxt: jinja_p.write_text(jtxt.replace("reasoning_effort|default('xhigh')", "reasoning_effort|default('low')"), encoding="utf-8")
healed_gguf = model_dir / "K256-q2_0-healed-metal.gguf" active_gguf = healed_gguf if healed_gguf.exists() else out_gguf cfg = { "exe": str(root / "vendor" / "strata" / "build-metal" / "strata_window_cli"), "args": [ "--metal-gguf", str(active_gguf), "--ple-gguf", str(shard1), "--n-expert", "256", "--max-context", "32768", ], "cwd": str(root / "vendor" / "strata"), "tokenizer": str(tok_dir), "model_name": "swift-1.5-k256-healed" if healed_gguf.exists() else "swift-1.5-k256", "log": "/tmp/strata-k256-serve.log", "lib_dirs": [], "port": 8080, "backend": "metal", "fit_max_tokens": True, "env": { "STRATA_METAL_ARENA_GB": "6.0", "STRATA_METAL_MLOCK_DENSE": "1", "STRATA_METAL_WIRE_ARENA": "1", "STRATA_METAL_EXPERT_PROFILE": str(out_prof), "STRATA_METAL_MOCE_BONUS": "0.85", "STRATA_METAL_EXPERT_SPLIT": "1", "STRATA_METAL_SHEXP_OVERLAP": "1", "STRATA_METAL_SPLIT_TURN": "1", "STRATA_METAL_PRED_Q8": "1", "STRATA_METAL_ROUTER_BF16": "1", "STRATA_METAL_MOE_DN_TILE": "8,2,4", "STRATA_METAL_PREFETCH": "6", "STRATA_METAL_PREFETCH2": "2", "STRATA_METAL_IO_THREADS": "8", "STRATA_METAL_IO_URGENT": "2", "STRATA_ANTI_SLOP_PENALTY": "6.0", }, "sampling": { "temperature": 0.6, "top_p": 0.95, "top_k": 40, "min_p": 0.05, "presence_penalty": 0.4, "repeat_penalty": 1.08, }, "reasoning_effort": "low", } (model_dir / "strata-k256.json").write_text(json.dumps(cfg, indent=2) + "\n", encoding="utf-8") return 0
if __name__ == "__main__":1 unmodified line123455678910#!/usr/bin/env bashset -euo pipefailexport STRATA_OMIT_IMAGES_WHEN_NO_VISION=1ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"UV_BIN="${UV:-$(command -v uv || echo "$HOME/.local/bin/uv")}"exec /Users/dawn/.local/bin/uv run --with regex --with jinja2 python3 \exec "$UV_BIN" run --with regex --with jinja2 python3 \ "$ROOT/vendor/strata/serve/server.py" \ --engine strata \ --config "$ROOT/models/K256_reap_l2_sum/strata-k256.json" \ --port "${PORT:-8080}" "$@"1234556789910#!/usr/bin/env bashset -euo pipefailexport STRATA_OMIT_IMAGES_WHEN_NO_VISION=1ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"UV_BIN="${UV:-$(command -v uv || echo "$HOME/.local/bin/uv")}"exec /Users/dawn/.local/bin/uv run --with regex --with jinja2 python3 \exec "$UV_BIN" run --with regex --with jinja2 python3 \ "$ROOT/vendor/strata/serve/server.py" \ --engine strata \ --config "$ROOT/models/K512_iq2_xs/strata-k512-iq2xs.json" \ --port "${PORT:-8080}" --port "${PORT:-8080}" "$@"12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849#!/usr/bin/env bashset -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"STRATA_COMMIT="296b59d2f73c8f04a805df29c2fa97655adace78"LLAMA_COMMIT="a0e6dc7384edb8f5d927155e1d26c10b4a310e6d"METAL_CPP_URL="https://developer.apple.com/metal/cpp/files/metal-cpp_26.zip"METAL_CPP_SHA256="4df3c078b9aadcb516212e9cb03004cbc5ce9a3e9c068fa3144d021db585a3a4"
mkdir -p "$ROOT/vendor"
if [ ! -d "$ROOT/vendor/llama.cpp/.git" ]; then git clone https://github.com/ggml-org/llama.cpp.git "$ROOT/vendor/llama.cpp"figit -C "$ROOT/vendor/llama.cpp" fetch --all --tagsgit -C "$ROOT/vendor/llama.cpp" checkout "$LLAMA_COMMIT"
if [ ! -d "$ROOT/vendor/strata/.git" ]; then git clone https://github.com/Niko1221/Strata.git "$ROOT/vendor/strata"figit -C "$ROOT/vendor/strata" fetch --all --tagsgit -C "$ROOT/vendor/strata" checkout -f "$STRATA_COMMIT"git -C "$ROOT/vendor/strata" clean -fdgit -C "$ROOT/vendor/strata" apply "$ROOT/scripts/strata_metal_complete.diff"
if [ ! -f "$ROOT/vendor/strata/third_party/metal-cpp/Metal/Metal.hpp" ]; then TMP_ZIP="$(mktemp -t metal-cpp.XXXXXX.zip)" TMP_DIR="$(mktemp -d -t metal-cpp.XXXXXX)" curl -fL "$METAL_CPP_URL" -o "$TMP_ZIP" echo "$METAL_CPP_SHA256 $TMP_ZIP" | shasum -a 256 -c - unzip -q "$TMP_ZIP" -d "$TMP_DIR" mkdir -p "$ROOT/vendor/strata/third_party" rm -rf "$ROOT/vendor/strata/third_party/metal-cpp" mv "$TMP_DIR/metal-cpp" "$ROOT/vendor/strata/third_party/metal-cpp" rm -rf "$TMP_ZIP" "$TMP_DIR"fi
cmake -S "$ROOT/vendor/strata/metal" -B "$ROOT/vendor/strata/build-metal" -G Ninja \ -DCMAKE_BUILD_TYPE=Release \ -DMETAL_CPP_DIR="$ROOT/vendor/strata/third_party/metal-cpp" \ -DSTRATA_GGML_DIR="$ROOT/vendor/llama.cpp"
ninja -C "$ROOT/vendor/strata/build-metal" strata_window_cli convert_k256_q2_0 heal_k256_q2_0
UV_BIN="${UV:-$(command -v uv || echo "$HOME/.local/bin/uv")}"if [ "${SKIP_MODEL_DOWNLOAD:-0}" != "1" ]; then "$UV_BIN" run --with regex python3 "$ROOT/scripts/build_k512_iq2_xs_metal.py" "$UV_BIN" run --with numpy python3 "$ROOT/scripts/build_mtp_q2_0.py"fi6 unmodified lines78910111213141536 unmodified lines525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101649 unmodified linesExpand all7517527537547127557567577587597607617627637207217227237247257267277287297307317327337647657667677687697707717727737747757767777787797804 unmodified lines7857867877887897907917927938 unmodified lines8028038048058068078088098108118128138148158168178181 unmodified line820821822823824825826827828313 unmodified linesExpand all1142114311441145114611471148114911501151111211131114111511161117111811191120112111221123112411251126115211531154115511561157115811591160116111621163116411651166116723 unmodified lines1191119211931194116411951196119711981199120012011202120312041205120656 unmodified lines6 unmodified lines#include "strata/core/layer.hpp"#include "strata/core/session.hpp"#include "strata/kernels/ngram.hpp"#include "strata/spec/suffix_drafter.hpp"#include "strata/spec/spec_gate.hpp"#include "engine/scr_state.hpp"#include "engine/suffix_window.hpp"#include "strata/core/scr.hpp"
36 unmodified linesstatic double now_ms() { return std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now().time_since_epoch()).count();}
// One policy instance per decode: a new prompt does not inherit another workload's timings.struct SpecControl { strata::spec::SpecGate gate; bool enabled = true, full = false; int mtp_used = 0;
SpecControl() { const char* mode = std::getenv("STRATA_SPEC_GATE"); enabled = mode && std::strcmp(mode, "0") != 0; full = mode && std::strcmp(mode, "1") == 0; }
int propose(strata::spec::SuffixDrafter& suffix, int32_t* input, const int32_t* mtp, const float* probs, int n_mtp, int suffix_max, int rem, int64_t pos, int64_t cells) { int nd = strata::spec::suffix_limit(n_mtp, rem, pos, cells); if (full) nd = gate.spec_gate_pick(0, nd, probs); mtp_used = nd; for (int j = 0; j < nd; ++j) input[1 + j] = mtp[j]; int limit = strata::spec::suffix_limit(std::max(suffix_max, nd), rem, pos, cells) - nd; if (suffix_max <= 0 || limit <= 0) return nd; if (enabled) limit = gate.spec_gate_pick(1, limit, nullptr, nd, probs); if (limit <= 0) return nd; const double begin = now_ms(); const int sk = nd > 0 ? suffix.propose_after(input + 1, nd, limit, input + 1 + nd) : suffix.propose(limit, input + 1); if (sk > 0) gate.spec_gate_draft_cost(1, (now_ms() - begin) / (1000 * sk)); return nd + sk; }
void result(int nd, int keep) { const int accepted = keep - 1; gate.spec_gate_result(0, std::min(mtp_used, keep), std::min(mtp_used, accepted)); if (accepted >= mtp_used) gate.spec_gate_result(1, std::min(nd - mtp_used, keep - mtp_used), accepted - mtp_used); }
void describe() const { if (enabled) std::fprintf(stderr, "%s\n", gate.spec_gate_describe().c_str()); }};
static double logsumexp_f64(const float* lg, int64_t n, int32_t* argmax_out) { float mx = lg[0]; int32_t am = 0; for (int64_t i = 1; i < n; ++i) {649 unmodified linesExpand all
if ((use_mtp || (suffix_draft > 0 && temp == 0.0f))) { strata::spec::SuffixDrafter suffix(4, 32, (size_t) max_cells); for (int64_t id : ids) suffix.append((int32_t) id); int32_t mtp_drafts[8] = {}, sbuf[8] = {}; int32_t mtp_drafts[8] = {}; float mtp_probs[8] = {}; int n_mtp_drafts = 0; bool first_win = true; SpecControl spec_control; while (gen < max_new) { int32_t input[8] = {cur_tok}, output[8] = {}; int nd = 0; const int rem = max_new - gen; if (!first_win && use_mtp && n_mtp_drafts > 0) { const int max_d = strata::spec::suffix_limit(n_mtp_drafts, rem, pos, max_cells); for (int i = 0; i < max_d; ++i) input[1 + i] = mtp_drafts[i]; nd = max_d; const int s_lim = strata::spec::suffix_limit( std::max(suffix_draft, nd), rem, pos, max_cells); if (suffix_draft > 0 && s_lim > nd) { const int sk = suffix.propose_after(input + 1, nd, s_lim - nd, sbuf); for (int i = 0; i < sk; ++i) input[1 + nd + i] = sbuf[i]; nd += sk; } } else if (!first_win && suffix_draft > 0) { const int limit = strata::spec::suffix_limit(suffix_draft, rem, pos, max_cells); nd = limit > 0 ? suffix.propose(limit, input + 1) : 0; } nd = spec_control.propose(suffix, input, mtp_drafts, mtp_probs, !first_win && use_mtp ? n_mtp_drafts : 0, !first_win ? suffix_draft : 0, rem, pos, max_cells); first_win = false; const double verify_begin = now_ms(); if (!me::window_run(w, 1 + nd, input, pos, true, output, err)) { std::printf("ERR %s\n", err.c_str()); std::fflush(stdout); cached_ids.clear(); finish = "error"; break; } spec_control.gate.spec_gate_forward(1 + nd, (now_ms() - verify_begin) / 1000); // the penalties see the window's tokens (cached_ids + input[0..keep-1]), as the one-token path's int keep; if (temp > 0.0f && spec_exact) { keep = spec_sample(w, input, output, nd, n_vocab, temp, top_k, top_p, min_p, pen_rep, pen_pres, cached_ids, lg_buf, rng);4 unmodified lines me::window_copy_logits(w, keep - 1, lg_buf.data()); output[keep - 1] = sample_logits(lg_buf.data(), n_vocab, output[keep - 1], temp, top_k, top_p, min_p, pen_rep, pen_pres, cached_ids, rng); } } spec_control.result(nd, keep); if (!me::window_commit(w, keep, err)) { std::printf("ERR %s\n", err.c_str()); std::fflush(stdout); cached_ids.clear();8 unmodified lines ++gen; if (strata::spec::suffix_stop(cur_tok)) break; } if (!strata::spec::suffix_stop(cur_tok) && gen < max_new && use_mtp) { const double draft_begin = now_ms(); if (!me::mtp_draft(1 + nd, output, pos, keep - 1, spec - 1, mtp_drafts, mtp_probs, spec_min_p, &n_mtp_drafts, err)) { std::printf("ERR %s\n", err.c_str()); std::fflush(stdout); cached_ids.clear(); finish = "error"; break; } spec_control.gate.spec_gate_draft_cost(0, (now_ms() - draft_begin) / (1000 * std::max(1, spec - 1))); } else if (use_mtp) { me::mtp_prefill(output, keep, pos, err); } pos += keep;1 unmodified line finish = "stop"; break; } } spec_control.describe(); } else { for (int step = 0; step < max_new; ++step) { int32_t greedy_tok = 0; if (!me::window_run(w, 1, &cur_tok, pos, true, &greedy_tok, err) ||313 unmodified linesExpand all strata::metal_engine::window_reset_stats(w); const int decode_steps = std::max(0, n_gen - 1); const double t_dec0 = now_ms(); int produced = 0, spec_rounds = 0, spec_drafted = 0, spec_accepted = 0; SpecControl spec_control; while (produced < decode_steps) { int32_t win_in[8] = {cur_tok}; int32_t win_out[8] = {}; int nd = 0; const int rem = decode_steps - produced; if (!tf_mode && use_mtp && n_mtp_drafts > 0) { const int max_d = strata::spec::suffix_limit(n_mtp_drafts, rem, pos, max_cells); for (int i = 0; i < max_d; ++i) win_in[1 + i] = mtp_drafts[i]; nd = max_d; const int s_lim = strata::spec::suffix_limit(std::max(suffix_draft, nd), rem, pos, max_cells); if (suffix_draft > 0 && s_lim > nd) { int32_t sbuf[8] = {}; const int sk = suffix.propose_after(win_in + 1, nd, s_lim - nd, sbuf); for (int i = 0; i < sk; ++i) win_in[1 + nd + i] = sbuf[i]; nd += sk; } } else if (!tf_mode && suffix_draft > 0) { const int max_d = strata::spec::suffix_limit(suffix_draft, rem, pos, max_cells); if (max_d > 0) nd = suffix.propose(max_d, win_in + 1); } nd = spec_control.propose(suffix, win_in, mtp_drafts, mtp_probs, !tf_mode && use_mtp ? n_mtp_drafts : 0, !tf_mode ? suffix_draft : 0, rem, pos, max_cells); const int T = 1 + nd; const double verify_begin = now_ms(); if (!strata::metal_engine::window_run(w, T, win_in, pos, true, win_out, err)) { std::fprintf(stderr, "strata_window_cli: decode step failed: %s\n", err.c_str()); return 1; } spec_control.gate.spec_gate_forward(T, (now_ms() - verify_begin) / 1000); const int n_keep = strata::spec::suffix_keep(win_in, win_out, nd, false); spec_control.result(nd, n_keep); if (!strata::metal_engine::window_commit(w, n_keep, err)) { std::fprintf(stderr, "strata_window_cli: window_commit failed: %s\n", err.c_str()); return 1; }23 unmodified lines if (!strata::metal_engine::mtp_draft(T, win_out, pos, n_keep - 1, spec - 1, mtp_drafts, mtp_probs, spec_min_p, &n_mtp_drafts, err)) { std::fprintf(stderr, "strata_window_cli: mtp_draft failed: %s\n", err.c_str()); return 1; } mtp_ms += now_ms() - tm0; const double draft_ms = now_ms() - tm0; mtp_ms += draft_ms; spec_control.gate.spec_gate_draft_cost(0, draft_ms / (1000 * std::max(1, spec - 1))); } pos += n_keep; } } if (!tf_mode && (use_mtp || suffix_draft > 0)) spec_control.describe(); const double decode_ms = now_ms() - t_dec0; const double decode_tok_s = decode_steps > 0 ? (decode_steps * 1000.0 / decode_ms) : 0.0; const double decode_ms_tok = decode_steps > 0 ? (decode_ms / decode_steps) : 0.0; const double prefill_tok_s = prompt_ids.size() * 1000.0 / prefill_ms;56 unmodified lines