| `persona_chat` (`289` prompt + `219` gen) | `124.6–128.1 tok/s` (`2.26–2.32 s`) | `33.01–33.25 tok/s` (`30.1–30.3 ms/tok`) | lowercase + kaomoji |
| component (`k=512` `iq2_xs`) | `32k` context | `128k` context (default) | notes |
| :--- | :--- | :--- | :--- |
| active gpu dense tensors (`k512-iq2_xs-metal.gguf`) | `2.27 gb` | `2.27 gb` | `mlock`ed zero-copy file map (`0.34 gb` cpu `token_embd` + `1.28 gb` bf16 `hc_*` unpinned) |
| active gpu dense tensors (`k512-iq2_xs-metal.gguf`) | `2.27 gb` | `2.27 gb` | `146` coalesced `mlock` spans (`145` page-aligned lazy ranges for `0.34 gb` cpu `token_embd` + `1.28 gb` bf16 `hc_*`) |
| hyper-connection `q8_0` matrices (`290` matrices) | `0.68 gb` | `0.68 gb` | quantized at load from bf16 `hc_*` |
| neural mtp draft head (`mtp-q2_0.gguf`) | `0.85 gb` | `0.85 gb` | `0.71 gb` `mlock`ed `q2_0` experts + `0.09 gb` `q8_0` dense + `0.05 gb` `40,525`-tok draft vocab (`0.18 gb` bf16 released) |
| context state (`32,896` / `131,200` cells) | `0.58 gb` | `2.12 gb` | `36` gdn recurrent states (`112 mb`, `0 b/tok` growth) + `12` qsa `int8` kv & pooled indexers (`14.2 kib/cell`) |
| context state (`32,896` / `131,200` cells) | `0.70 gb` | `2.23 gb` | `36` dual-slot `gst_[2]` (`226 mb`) & 3-slot `ghist_[3]` (`0 b/tok` growth) + `12` qsa `int8` kv & pooled indexers (`14.2 kib/cell`) |
| wired moe expert arena (`4,129` expert slots) | `6.50 gb` | `6.50 gb` | `mlock`ed (`33.6%` of all `24,576` experts; remaining `29.7 gb` of `36.23 gb` `.experts` streamed from ssd) |
| ple n-gram table page cache reserve | `0.50 gb` | `0.50 gb` | `28.80 gb` `ple-iq4_nl.gguf` (`51.2b` params) stays on ssd (`16` row reads/token) |
| elastic vision encoder (`strata-vision` + `mmproj` bf16) | `0.00 gb` | `0.00 gb` | `0.92 gb` active only during image turns (`vram_elastic`, `60s` idle unload; `.sve` cached by sha-256) |
| **steady-state decode total** | **`8.79 gb` (`9.5–9.8 gb` rss)** | **`10.30 gb` (`10.8 gb` rss)** | **`9.60` / `11.10 gb` planned against `18.80 gb` cap (`>13 gb` left free for macos)** |
| transient prefill scratch (`PromptGemm`, `MT=2048`) | `+1.12 gb` | `+1.32 gb` | separate `MAP_ANON` buffer during prefill only, `munmap`ped immediately after prefill |
| **steady-state decode total** | **`8.91 gb` (`9.6–9.9 gb` rss)** | **`10.42 gb` (`10.9 gb` rss)** | **`9.60` / `11.10 gb` planned against `18.80 gb` cap (`>13 gb` left free for macos)** |
| prefill scratch (`PromptGemm`, `MT=2048`, `STRATA_METAL_PHASE_TRADE=1`) | **`+0.00 gb`** | **`+0.00 gb`** | phase-switched inside the `6.50 gb` expert arena (`1.12` / `1.32 gb`, `~720` slots borrowed during prefill and returned to decode lru after prefill) |
## optimizations
### model & quantization
- native 49th-layer neural mtp draft pack (`mtp-q2_0.gguf`, `889 mb`, runtime `q8_0` dense quantization + `madvise(MADV_DONTNEED)` on `181 mb` bf16 source matrices + `707.8 mb` `mlock`ed `q2_0` routed experts + `40,525`-token `draft_vocab_en.bin`)
- `reasoning_effort|default('low')` chat template default
### ssd expert streaming, unified memory arena & moce routing
- zero-copy file-backed `mmap` of `<model>-metal.gguf` with selective `mlock` on active gpu dense tensors and `MADV_DONTNEED` on cpu-only `token_embd` and skipped bf16 `hc_*` originals
- wired metal expert arena (`STRATA_METAL_WIRE_ARENA=1`, `STRATA_METAL_MLOCK_DENSE=1`, fixed `make_arena` class-sizing loop) + separate on-demand `MAP_ANON` `PromptGemm` scratch buffer (`munmap`ped + `cx_->free`d immediately after prefill)
- zero-copy file-backed `mmap` of `<model>-metal.gguf` with coalesced inward page-aligned `MADV_DONTNEED` ranges (`145` lazy spans for cpu-only `token_embd` and skipped bf16 `hc_*`) and `146` coalesced `mlock` complement spans on active gpu dense tensors (never dropping shared 16 kib boundary pages)
- wired metal expert arena (`STRATA_METAL_WIRE_ARENA=1`, `STRATA_METAL_MLOCK_DENSE=1`, fixed `make_arena` class-sizing loop) + phase-switched `PromptGemm` prefill scratch (`STRATA_METAL_PHASE_TRADE=1`, borrowing `1.12–1.32 gb` of tail slots from the `6.50 gb` arena during prefill and returning them to the decode lru after prefill at `+0.00 gb` extra memory)
- dynamic `n_expert` (`k = 224 / 256 / 320 / 512`) across host and metal router/moe shaders
- closed-loop moce cache-prior routing (`STRATA_METAL_MOCE_BONUS=0.85`, top-3 unperturbed protection, race-free single-read `smap` bitmask in `moe_route` / `router_route` / `route_topk`, deterministic `smap_` publication at layer/cut boundaries, and opt-in rank-1 bonus unscaling fix)
- `acceptmoe` speculative draft routing (`STRATA_METAL_DRAFT_MOCE=1`, `r_prot = 1` and squared `moce_scale` on draft rows `sg > 0`) + root-token-only decode prefetch (`STRATA_METAL_PREFETCH_DRAFT=0`, `STRATA_METAL_PREFETCH_UNPERT=1`)
- `q8_0` cross-layer lookahead router (`STRATA_METAL_PRED_Q8=1`) and lossless `bf16` main router (`STRATA_METAL_ROUTER_BF16=1`)
- multi-chunk (`n > C`) hot-expert lru and `!hot` scratch-pool preservation across prompt chunks + `allow_hot = false` on initial layer-0 `hold_region` prefetch + separate prompt/decode ssd read counters
- asynchronous background ple ssd `gather_batch` thread during prefill + direct-mode ple `prefetch_rows` duplicate-slot reuse and error-drain in `PleTable::gather_batch`
### speculative decoding, kv sessions & suffix cache reuse (`scr`)
- single-command-buffer gpu neural mtp draft chain (`mtp_chain.metal`, `STRATA_METAL_MTP_CHAIN=1`) with `SuffixDrafter::propose_after` hybrid continuation
- single-command-buffer gpu neural mtp draft chain (`mtp_chain.metal`, `STRATA_METAL_MTP_CHAIN=1`) with combined `(a + 1)`-row catch-up + draft step-0 batch (`STRATA_METAL_MTP_DEFER_CATCHUP=1`, `QSA_LAST_Q` function constant 29, `-20% to -26.8%` per draft round) and `SuffixDrafter::propose_after` hybrid continuation
- 64-cell attention sink (`STRATA_METAL_MTP_SINK=64`) + 256-cell local sliding window (`STRATA_METAL_MTP_WIN=256`, synced with `mtp_prefill` `tail_cap`) + `mtp_wrote` valid-run kv tracking + dead mtp indexer elimination (`QSA_ROW0=24` k/v-only `qsa_prep`)
- 24 kib `tail_bak_` qsa raw-key snapshot/restore in `Window::run` / `Window::commit` for unrestricted `T <= 4` speculative rollback across 4-cell qsa block boundaries
- zero-copy dual-slot gdn recurrent state (`gst_[2]`, `ghist_[3]`, `STRATA_METAL_GDN_RING=1`, `a.commit == 2u` `dual2` / `3u` `spec1` in `gdn.metal`, eliminating `Window::fold()` 100% of the time on speculative verification) + 24 kib `tail_bak_` qsa raw-key snapshot/restore in `Window::run` / `Window::commit`
- opt-in online cost-model `SpecGate` (`STRATA_SPEC_GATE=suffix|1`), step-aware `STRATA_METAL_SPEC_MIN_P_SCHEDULE` / `STRATA_METAL_SPEC_MIN_CUM_P`, and exact/typical `temp > 0` speculative verification (`STRATA_SPEC_SAMPLE=exact|typical`)
- elastic metal vision (`strata-vision` + `mmproj-Swift-Qwen3.8-Flash-Next-BF16.gguf` + 2d `(t, h, w)` m-rope `qsa_prep#mrope` + `GENI` `.sve` embedding injection + multi-turn image kv reuse)
- 4-thread `F_NOCACHE` nvme session save/load (`--save-session` / `--load-session`, `Window::session_save` / `session_load`) serializing all 36 gdn + 12 qsa + ple + mtp sliding-window states (`mtp_cell0_`, `mtp_cell1_`, `mtp_sink_`) directly from/into `MTLBuffer`s at `5.2 gb/s`
- bonsai-style automatic nvme prompt & conversation `.kvc` cache (`--cache-disk` / `--cache-disk-size 16384` in `--serve`): startup `.kvc` header indexing (`< 1 ms`), post-`DONE` zero-latency auto-save with obsolete-prefix pruning, 2-prompt shared-prefix learning & `.pin` checkpointing (`prompts-seen.bin`), zero-copy `prefix-share` delta `.kvc` storage against pinned prefixes (`.parent` + `SessionHeader` `base_cells`, storing only `[c0, n_cells)` positional cells while reading `[0, c0)` directly from the parent `.kvc`), and automatic cross-conversation / post-restart nvme restore + `scr` splice over disk
- `window.cpp` prefill graph & scratch: `-1,247` redundant barriers removed + router/plan/shared-expert wave zipping (`-190` barriers) + `sigs` right after `route_topk` so moe plan and shared-expert gemms overlap host/ssd `at_cut(l)` + `Wr`/`Wr2` parity ping-pong (`-47` router `dq_f16` dispatches/chunk) + fused gdn `ssm_alpha + ssm_beta` (`N = 96`) gemm (`-36` dispatches) + fused qsa `Wk + Wv + Wik (+ Wiq)` (`N = 1152 / 1664`) gemm (`-24..36` dispatches) + `qsa_score_mm`/`qsa_topk` pso prewarming + multi-stage `PromptGemm` scratch aliasing (`2.35 gb -> 1.12 gb` at `MT = 2048`)
- `dense_mm.metal` & `mma.metal`: m5 shape-aware `gemm_f16` tile selector (`M > 1536` threshold, `16_64_2`, `32_64_2`, `32_128_2`, `64_32_2`, `16_96_2` for `Walbe`, and `64_128_4` when `M < 1536 && M % 128 in 1..64`), hardware 2d-grid `dq_f16` with `half4` stores + `TAB == 1` barrier elimination + `T_Q5_1` + register table init & `perm == 4` contiguous writes in `dq_f16<T_IQ4_XS/IQ4_NL>`, unrolled `DW<T_IQ4_XS/IQ4_NL/Q6_K>`, and `half4` `act_f16`
- `prompt.metal`: folded `mm_combine` + `moe_out` into per-token 4-stream `gr_norm_h_moec` and `gr_apply_moec` (plus `gr_norm_h4`) with register-cached `rv`, stream-by-stream reduction, compacted `moe_comb_slots` `roff`, and `k`-outer loop inversion; cooperative-tensor register epilogues in `gemm_hc_up_reg_32_128_4` (unbound dead `p.mixed`, unrolled epilogue, `hc_up_tile<16/32>` row-tail trimming), `gemm_hc_down` (fusing `gr_epi_down`, `* 0.25f`, `TN = 96` tail tile, guarded `TM > 32` row-tail trimming), and `gemm_swiglu_up` (fusing `swiglu_h`, guarded `TM > 32` row-tail trimming); `n_exp == 512` unrolled `route_topk`
- `moe_mm.metal` & `ple.metal`: `mm_clear` elimination (pad-clear folded into `mm_scan`), 32-subblock nibble-counted `s_id`-free `mm_place`, 128-bit `uint4` fp16 `mm_gather_h`, single-`rw` `mm_gu3` / `mm_down3` with `TAB > 1` barrier guards, hoisted row pointers (`g0_l`, `u0_l`, `w0_l`) and `boff`/`sboff`, hoisted `tab + hh`, branchless `s >> (4 * hh)` in `W8<T_IQ3_S/IQ2_S>` + `W8<T_IQ3_S>::block`, 256-wide super-block scale caching `dg[]`/`du[]` (`T_IQ2_S/IQ2_XXS/IQ2_XS`), `W8<T_IQ2_XXS>` direct byte loads, `sign4_iq2xs` + constant-offset pointer folding, scalar `uint` or in `W8<T_Q6_K>` / `W4<T_Q6_K>`, `half4` coalesced `HS` store epilogue (`T_IQ2_XS/IQ2_XXS/IQ3_XXS/Q2_0/IQ2_S/IQ3_S`), 64-element block stepping (`k0 += 64, w0_l += 18`) + ieee-754 fp16 `0x64006400u` mantissa-or decode in `W4<T_Q2_0>::run_q` (`mm_down3`), and 1-tg-per-token `ple_moe_gate_norm_c` with simd slot compaction + `ple_conv_res` (`half4` weights, branchless `t >= 9`) + `ple_hist_adv`
- `qsa_prompt.metal` & `gdn_prompt.metal`: `qsa_prep#qf16` fused fp16 query output, `qsa_score_mm` causal-future tile skip (`b0 >= max_bid`) + `dead[0..3]` hoist, `qsa_topk` cached `keys[4]` + simdgroup bucket scan + 2-barrier fused `(equal, greater)` emit (and skipped when `p.flash && pos0 + T <= 2051`), `qsa_flash` 4-simdgroup online softmax with `sp[4][HPG*QCH]` (`13,760 b` threadgroup memory, 1 barrier/tile) + unconditional hoisted `rv`/`rk` fills + byte-offset/zero-scale hoist + 32-bit `ko[8]`/`vo[8]` offsets with pre-added `d0 + kl` + `c0 < nc` pv bound + `vec4` epilogue, 1-simdgroup-per-head barrier-free `gdn_pre` (`float4` value-head conv, vector `simd_sum(float4)`) & `gdn_post` (hoisted pointers, preloaded gate/gamma, vector `simd_sum(float4)`), and single-pass `TB = 0` `simd_broadcast` `gdn_scanr_4` with 32-bit base offsets + `hk`-grouped head scheduling + final-token dead-lookahead elimination
- `qsa_prompt.metal` & `gdn_prompt.metal`: 0-barrier 1-simdgroup-per-row `qsa_prep_sg` (`tg = 32` for `ntok >= 3`, `-8% to -33%` faster, `100%` bitwise) + `qsa_prep#qf16` fused fp16 query output, `qsa_score_mm` causal-future tile skip (`b0 >= max_bid`) + `dead[0..3]` hoist, `qsa_topk` cached `keys[4]` + simdgroup bucket scan + 2-barrier fused `(equal, greater)` emit (and skipped when `p.flash && pos0 + T <= 2051`), `qsa_flash` 4-simdgroup online softmax with `sp[4][HPG*QCH]` (`13,760 b` threadgroup memory, 1 barrier/tile) + unconditional hoisted `rv`/`rk` fills + byte-offset/zero-scale hoist + 32-bit `ko[8]`/`vo[8]` offsets with pre-added `d0 + kl` + `c0 < nc` pv bound + `vec4` epilogue, 1-simdgroup-per-head barrier-free `gdn_pre` (`float4` value-head conv, vector `simd_sum(float4)`) & `gdn_post` (hoisted pointers, preloaded gate/gamma, vector `simd_sum(float4)`), chunked WY `matmul2d` `gdn_wy_prep` + `gdn_wy_scan` (`16`-token chunks, `(I + A)^-1` via 6 fp32 `16x16` `matmul2d`s, 3-mma fp16 `(hi, lo)` split, `46 mb` scratch aliased inside `p.qkv`, `-26.6% to -28.3%` scan time / `-53.6 ms` per 2k chunk), and logit-domain delayed-softmax top-10 routing in `route_topk` (`-4.5% to -8.0%`) & `moe_route`
### anti-slop, persona data & serving
- `slop4` (`slop4.h`) per-position stateful shaper (`12 b/pos` in `window.cpp` + gpu `mtp_slop_pen` in `mtp_chain.metal` with `STRATA_METAL_MTP_SLOP=1`), gated strictly on `PROSE && !TOOL && !FENCE`, with clause-entry branch-point penalties, `slop4::hold()` `<|im_end|>` mass preservation, `</think>\n\n` `step0` empty-turn guard, `slop_scan`, and `scr_relocate` support
- monroe prior log-odds llm-ism miner (`mine_llm_isms_from_transcripts.py`, pure per-order unigram+uniform prior with zero self-pooled weight) partitioning structural speech acts (`phase3_sentence_surgery` in `prepare_phase3_sft_data.py`, in-place span deletion preserving markdown/fences/kaomoji + r-tuning word-boundary abstention) vs surface lexical cliches (`phase4_lexical_ftpo`)