`regolith` runs `swift-1.5-qwen3.8-flash-next` (`qwen4_exp`, 512 experts, 36 gated deltanet + 12 sparse attention layers + 51.2b n-gram ple table + neural mtp draft head) on apple silicon via a patched `strata` metal engine.
`./scripts/setup_apple_silicon.sh` clones the pinned `strata` and `llama.cpp` trees into `vendor/`, applies `scripts/strata_metal_complete.diff`, downloads apple's `metal-cpp` headers, compiles `strata_window_cli`, and streams + repacks the unpruned `k=512` `iq2_xs` model (`3.90 gb` dense `.gguf` + `36.23 gb` `.experts` + `28.80 gb` `ple-iq4_nl.gguf` + `889 mb` `mtp-q2_0.gguf`) into `models/K512_iq2_xs/`.
`./scripts/serve_k512_local.sh` starts the local server on `http://127.0.0.1:8080` (`/v1/chat/completions` and `/v1/messages`, 32k context, `6.5 gb` wired expert arena, neural mtp + suffix speculative decoding).
`./scripts/serve_k512_local.sh` starts the local server on `http://127.0.0.1:8080` (`/v1/chat/completions` and `/v1/messages`, 128k context, `6.5 gb` wired expert arena, neural mtp + suffix speculative decoding).
research notes, ablations, and benchmark logs live in `PLAN.md`, `blueprint_review.md`, `audits/`, and `results/`.
## requirements
| `coding` (`306` prompt + `219` gen) | `120.6–126.8 tok/s` (`2.41–2.54 s`) | `35.46–37.10 tok/s` (`27.0–28.2 ms/tok`) | c++20 lock-free ring buffer |
| `reasoning` (`297` prompt + `219` gen) | `121.6–122.9 tok/s` (`2.42–2.46 s`) | `34.92–37.20 tok/s` (`26.9–28.6 ms/tok`) | gdn/qsa derivation |
| `persona_chat` (`289` prompt + `219` gen) | `124.6–128.1 tok/s` (`2.26–2.32 s`) | `33.01–33.25 tok/s` (`30.1–30.3 ms/tok`) | lowercase + kaomoji |
| component (`k=512` `iq2_xs`, `32k` context) | resident ram | notes |
| :--- | :--- | :--- |
| active gpu dense tensors (`k512-iq2_xs-metal.gguf`) | `2.27 gb` | `mlock`ed zero-copy file map (`0.34 gb` cpu `token_embd` + `1.28 gb` bf16 `hc_*` unpinned) |
| hyper-connection `q8_0` matrices (`290` matrices) | `0.68 gb` | quantized at load from bf16 `hc_*` |
| neural mtp draft head (`mtp-q2_0.gguf`) | `0.85 gb` | `0.71 gb` `mlock`ed `q2_0` experts + `0.09 gb` `q8_0` dense + `0.05 gb` `40,525`-tok draft vocab (`0.18 gb` bf16 released) |
| `32k` context state (`32,896` cells) | `0.58 gb` | `36` gdn recurrent states (`112 mb`, `0 b/tok` growth) + `12` qsa `int8` kv & pooled indexers (`14.2 kib/cell`) |
| wired moe expert arena (`4,129` expert slots) | `6.50 gb` | `mlock`ed (`33.6%` of all `24,576` experts; remaining `29.7 gb` of `36.23 gb` `.experts` streamed from ssd) |
| ple n-gram table page cache reserve | `0.50 gb` | `28.80 gb` `ple-iq4_nl.gguf` (`51.2b` params) stays on ssd (`16` row reads/token) |
| **steady-state decode total** | **`8.79 gb` `phys_footprint` (`9.52–9.80 gb` rss)** | **`9.60 gb` planned against `18.80 gb` cap (`>14 gb` left free for macos)** |
| transient prefill scratch (`PromptGemm`, `MT=2048`) | `+1.12 gb` (during prefill only) | separate `MAP_ANON` buffer, `munmap`ped immediately after prefill (`10.72 gb` peak during prefill) |
| component (`k=512` `iq2_xs`) | `32k` context | `128k` context (default) | notes |
| :--- | :--- | :--- | :--- |
| active gpu dense tensors (`k512-iq2_xs-metal.gguf`) | `2.27 gb` | `2.27 gb` | `mlock`ed zero-copy file map (`0.34 gb` cpu `token_embd` + `1.28 gb` bf16 `hc_*` unpinned) |
| hyper-connection `q8_0` matrices (`290` matrices) | `0.68 gb` | `0.68 gb` | quantized at load from bf16 `hc_*` |
| neural mtp draft head (`mtp-q2_0.gguf`) | `0.85 gb` | `0.85 gb` | `0.71 gb` `mlock`ed `q2_0` experts + `0.09 gb` `q8_0` dense + `0.05 gb` `40,525`-tok draft vocab (`0.18 gb` bf16 released) |
| context state (`32,896` / `131,200` cells) | `0.58 gb` | `2.12 gb` | `36` gdn recurrent states (`112 mb`, `0 b/tok` growth) + `12` qsa `int8` kv & pooled indexers (`14.2 kib/cell`) |
| wired moe expert arena (`4,129` expert slots) | `6.50 gb` | `6.50 gb` | `mlock`ed (`33.6%` of all `24,576` experts; remaining `29.7 gb` of `36.23 gb` `.experts` streamed from ssd) |
| ple n-gram table page cache reserve | `0.50 gb` | `0.50 gb` | `28.80 gb` `ple-iq4_nl.gguf` (`51.2b` params) stays on ssd (`16` row reads/token) |
| **steady-state decode total** | **`8.79 gb` (`9.5–9.8 gb` rss)** | **`10.30 gb` (`10.8 gb` rss)** | **`9.60` / `11.10 gb` planned against `18.80 gb` cap (`>13 gb` left free for macos)** |
| transient prefill scratch (`PromptGemm`, `MT=2048`) | `+1.12 gb` | `+1.32 gb` | separate `MAP_ANON` buffer during prefill only, `munmap`ped immediately after prefill |
## optimizations
### model & quantization