`swift-1.5-qwen3.8-flash-next` (`qwen4_exp`, 512 experts, 36 gated deltanet + 12 sparse attention layers + 51.2b n-gram ple table + neural mtp draft head) running on apple silicon via a patched `strata` metal engine.
`regolith` runs `swift-1.5-qwen3.8-flash-next` (`qwen4_exp`, 512 experts, 36 gated deltanet + 12 sparse attention layers + 51.2b n-gram ple table + neural mtp draft head) on apple silicon via a patched `strata` metal engine.
`./scripts/setup_apple_silicon.sh` clones the pinned `strata` and `llama.cpp` trees into `vendor/`, applies `scripts/strata_metal_complete.diff`, downloads apple's `metal-cpp` headers, compiles the metal engine (`strata_window_cli`, `convert_k256_q2_0`, `heal_k256_q2_0`), and streams + repacks the unpruned `k=512` `iq2_xs` model (`3.90 gb` dense `.gguf` + `36.23 gb` `.experts` + `28.80 gb` `ple-iq4_nl.gguf` + `889 mb` `mtp-q2_0.gguf`) into `models/K512_iq2_xs/`.
`./scripts/serve_k512_local.sh` starts the local server on `http://127.0.0.1:8080` (`/v1/chat/completions` and `/v1/messages`, 32k context, `6.5 gb` wired expert arena, neural mtp + suffix speculative decoding).
```sh
SKIP_MODEL_DOWNLOAD=1 ./scripts/setup_apple_silicon.sh
```
## m5 24 gb performance & memory
measured on a fanless 13" m5 macbook air (`mac17,3`, 10-core gpu, 24 gb unified memory, 1 tb nvme ssd) running unpruned `k=512` `iq2_xs` at `32k` context (`--spec 3 --spec-min-p 0.35 --suffix-draft 3`, `STRATA_METAL_MOCE_BONUS=0.85`):
| workload | prompt prefill | decode | notes |
| :--- | :--- | :--- | :--- |
| pure gpu prefill (`kprof`, `1,983` tok) | `633.28 tok/s` (`3.13 s`) | — | `2,047` metal dispatches/chunk |
| `prompt_2048tok_ops` (`1,984` prompt + `128` gen) | `421.7–432.8 tok/s` (`4.58–4.70 s`) | `42.26–42.57 tok/s` (`23.5–23.7 ms/tok`) | `88.9%` draft accept (`3.26 tok/round`) |
| `prompt_albedo_11k` (`10,180` prompt + `32` gen) | `434.8–473.8 tok/s` (`21.49–23.42 s`) | `34.13–35.47 tok/s` (`28.2–29.3 ms/tok`) | 5 chunks of `2,048`, `78.3%` accept |
| `coding` (`306` prompt + `219` gen) | `120.6–126.8 tok/s` (`2.41–2.54 s`) | `35.46–37.10 tok/s` (`27.0–28.2 ms/tok`) | c++20 lock-free ring buffer |
| `reasoning` (`297` prompt + `219` gen) | `121.6–122.9 tok/s` (`2.42–2.46 s`) | `34.92–37.20 tok/s` (`26.9–28.6 ms/tok`) | gdn/qsa derivation |
| `persona_chat` (`289` prompt + `219` gen) | `124.6–128.1 tok/s` (`2.26–2.32 s`) | `33.01–33.25 tok/s` (`30.1–30.3 ms/tok`) | lowercase + kaomoji |
| component (`k=512` `iq2_xs`, `32k` context) | resident ram | notes |
| :--- | :--- | :--- |
| active gpu dense tensors (`k512-iq2_xs-metal.gguf`) | `2.27 gb` | `mlock`ed zero-copy file map (`0.34 gb` cpu `token_embd` + `1.28 gb` bf16 `hc_*` unpinned) |
| hyper-connection `q8_0` matrices (`290` matrices) | `0.68 gb` | quantized at load from bf16 `hc_*` |
| neural mtp draft head (`mtp-q2_0.gguf`) | `0.85 gb` | `0.71 gb` `mlock`ed `q2_0` experts + `0.09 gb` `q8_0` dense + `0.05 gb` `40,525`-tok draft vocab (`0.18 gb` bf16 released) |
| `32k` context state (`32,896` cells) | `0.58 gb` | `36` gdn recurrent states (`112 mb`, `0 b/tok` growth) + `12` qsa `int8` kv & pooled indexers (`14.2 kib/cell`) |
| wired moe expert arena (`4,129` expert slots) | `6.50 gb` | `mlock`ed (`33.6%` of all `24,576` experts; remaining `29.7 gb` of `36.23 gb` `.experts` streamed from ssd) |
| ple n-gram table page cache reserve | `0.50 gb` | `28.80 gb` `ple-iq4_nl.gguf` (`51.2b` params) stays on ssd (`16` row reads/token) |
| **steady-state decode total** | **`8.79 gb` `phys_footprint` (`9.52–9.80 gb` rss)** | **`9.60 gb` planned against `18.80 gb` cap (`>14 gb` left free for macos)** |
| transient prefill scratch (`PromptGemm`, `MT=2048`) | `+1.12 gb` (during prefill only) | separate `MAP_ANON` buffer, `munmap`ped immediately after prefill (`10.72 gb` peak during prefill) |
## optimizations
### model, pruning & quantization
- unpruned `k=512` `ukisai` gsq-rco `iq2_xs` metal bundle (`3.90 gb` 16 kib-aligned dense `.gguf` + `36.23 gb` contiguous `[gate | up | down]` `.experts`, `blk.8/13/37` `iq1_m` replaced with `iq3_s`/`iq2_xxs`, duplicate `28.80 gb` ple table omitted)