# opus colibri scout: what `JustVugg/colibri` has that `vendor/strata/metal/` doesn't

clone: `https://github.com/JustVugg/colibri` @ depth 1 in `/tmp/colibri` (340k lines; the Apple GPU code is `c/backend_metal.mm`,
2,367 lines, kernels inline as one MSL string at lines 27–691). our side: `vendor/strata/metal/` at the working tree as of this run.
probes (4 small Metal/ObjC++ programs, ~1 min of GPU in total, run while no strata process was up; nothing in `vendor/` edited,
`:8080` not touched): `/tmp/colibri_probe/{bench,benchA,benchA2,benchC,benchD}.mm`, raw numbers in `/tmp/colibri_probe/results.txt`.

## 0. tl;dr

colibri is a **CPU engine with an opt-in Metal offload layer**, built around models far bigger than RAM (GLM-5.2 744B, Kimi K3,
DeepSeek V4.x). its best Apple number is 2.24 tok/s on an M5 Max (`docs/METAL-M5MAX-PERF-REPORT.md`). for **Qwen3.8 Flash Next
(our model) colibri has no Metal path at all**: `c/qwen38.c` only wires the Vulkan tier (`qwen38.c:2220-2409`, `q38_vk_*`), and
its docs measure 1.1–4.0 tok/s on x86 CPUs (`docs/qwen38.md`, "Memory and speed" and the int4 table). strata does ~21–22 tok/s
on the 24 GB M5 Air with all 512 experts. so the honest answer is: **at the kernel, runtime and SSD-streaming level there is
nothing in colibri that's ahead of us**, and a few of the things the brief flagged as suspects (hazard tracking, residency sets,
passive waits, entropy coding) i measured and they don't pay here.

what *is* worth taking is host-side policy, ranked:

| # | steal | colibri source | strata today | expected on the M5 Air | effort |
|---|---|---|---|---|---|
| 1 | **online cost-model draft depth** (`SpecGate`: pick k maximizing E(k)/cost(k) from measured per-position acceptance and measured T(n), k=0 allowed) | `c/spec_draft.h:14-160` | fixed `--spec 3 --spec-min-p 0.35 --suffix-draft 3` (`strata_window_cli.cpp:709-780, 852-853`; config `models/K512_iq2_xs/strata-k512-iq2xs.json`) | workload-dependent: 0 on high-acceptance text, **~5–10 % on stretches where drafts mostly fail** (a T=3/4 window that keeps 1 token is a pure loss under streaming); also the right place to stop drafting during prompt-heavy / tool turns | ~120 lines host-only, no kernel, exactness unchanged |
| 2 | **max-rank window on MoCE substitution** (`CACHE_ROUTE`'s `ROUTE_M`: a resident may only replace a slot if it is inside the true top-M) | `docs/CACHE_ROUTE.md`, `ROUTE_J/ROUTE_M` | `router.metal:43-111` scales every resident's prob by λ from rank `r_prot` on, no rank cap | quality guard, ~0 ms: bounds the worst-case substitution (a rank-40 resident can win today if within the 0.85-logit bonus). measure routed relMSE vs misses with the existing replay (`/tmp/opus_metal_probes`) at M = 16/24/32 | ~20 lines in `router_route` |
| 3 | **learned expert-usage history that survives restarts** (`.coli_usage`, decayed counts appended per turn; startup pin = history; optional between-turn re-pin) | `c/route_trace.h:1-60`, `c/colibri.c:13216-13255` (PIN=auto), `colibri.c:9729-9739` (REPIN) | static calibration profile `data/expert-profile.bin` (`stream.cpp:440-466`), fill 4,129 slots from its top in 1.6–1.9 s (serve log) | first-turn hit rate after a restart + prompt-hot protection (`hold_region`) aligned to the persona's real traffic. steady state is LRU-driven and won't move | ~80 lines in `ExpertStream` (dump LRU order + counts at exit/idle, blend at `make_arena`) |
| 4 | **PLE rows issued early, collected later** (the row ids are a pure function of the tokens, so the reads can start before anything else and be collected at the layer that needs them) | `c/qwen38_core.h:2060-2110` (`q38_ple_prefetch`) | decode window gathers synchronously before the CB (`window.cpp:1480-1491`), and the PLE GEMVs sit before layer 0 (`window.cpp:1546-1551`). the async API exists (`ngram.cpp:398-407` `prefetch_rows`, `507-538` fast path) and the CUDA/core path uses it (`src/core/mtp.cpp:1581`, `verify.cpp:2264`), the metal window doesn't | small: "host before" is 0.225 ms/window in total (audits/opus_strata_metal_opts_v3.md:111), so ≤ ~0.2 ms (≤ 0.5 %). worth it only bundled with #1 (prefetch the rows of the MTP drafts right when `mtp_draft` returns) | ~30 lines |
| 5 | cheap diagnostics colibri ships and we don't: a grid-size guard (`backend_metal.mm:1653-1680`, they saw silent corruption past ~5.4e7 threadgroups) and a GPU keep-alive experiment (`backend_metal.mm:1060-1085`, `COLI_METAL_SPIN`) for the first window after idle | see left | `require_fit` checks threads per TG only (`mtl_rt.mm:159-169`); `warm()` exists (`mtl_rt.mm:336-339`) but nothing runs between requests | 0 ms steady state; the keep-alive could matter for TTFT after the server sat idle (DVFS ramp), unmeasured | trivial |

everything else colibri does on Metal we either already do better or i measured to be worthless here (§3, §4).

## 1. architecture side by side

| | colibri (`c/backend_metal.mm`) | strata (`vendor/strata/metal/`) |
|---|---|---|
| what runs on the GPU | per-call offload: `coli_metal_matmul` (1118-1167), `coli_metal_gemm` (1637-1687), fused MLA layer for S≤4 (`coli_metal_layer_decode`, 1532-1632), batched routed-expert SwiGLU (`moe_submit_impl`, 1738-1840). everything else (routing bookkeeping, scatter-add, GDN/KDA prep, sampling) on the CPU with OpenMP | the whole decode window — 48 layers, router, MoE, GDN, QSA, HC/PLE, head, argmax — one command buffer (`window.cpp:1460-1608`, cached per window shape 1506-1532) |
| host↔GPU data | `newBufferWithBytes`/`memcpy` of activations in and out of every call (1151-1164, 1511, 1564, 1623-1629, 1788); scatter-add of expert outputs on the CPU (`moe_finish` 1854-1856) | activations never leave the GPU inside a window; expert outputs combined on the GPU (`moe_mm.metal:269` `mm_combine`, fused variants in `ple.metal:174-183`) |
| command buffers | one CB per layer/op, `commit` + `waitUntilCompleted` each (e.g. 1163, 1517, 1619, 1682); docs call Metal's submit "~5 ms" (`docs/metal.md`) | one CB per window; inside it 48 segments with a `MTLSharedEvent` handshake per layer (`mtl_rt.mm:273-316`), shared-expert work placed between signal and wait (`sigs`, window.cpp:1612) |
| encoder | default serial `computeCommandEncoder`; explicit `memoryBarrierWithScope:Buffers` inside the attention chain (1413, 1447, 1571, 1813-1834) | `MTLDispatchTypeConcurrent` + explicit barriers only where a dispatch depends on the previous (`mtl_rt.mm:252-255, 283-288`), `commandBufferWithUnretainedReferences` (230, 251, 279) |
| buffers | `MTLResourceStorageModeShared`; `COLI_METAL_UNTRACKED=1` adds `HazardTrackingModeUntracked` (753, 928-929), **default off, never measured in-tree** | Shared, default tracking (`mtl_rt.mm:171-198`) |
| residency | `useResource:` per slab per CB (1380, 1513-1514, 1567-1570, 1801); `COLI_METAL_RESSET=1` puts slabs in a queue-attached `MTLResidencySet` (776-862, 995-1017), **default off, its own doc says "unverified without a run"** (`docs/experiments/e5-metal-residency-set.md`, table row "Known regression") | not needed: weights are 1 zero-copy wrap of the file map (`stream.cpp:206-238`) + an arena of ≤ 4 buffers (`stream.h:58-59`; serve log: "1 buffers"), bound with `setBuffer` per dispatch, so Metal makes them resident implicitly |
| expert addressing | per-dispatch tables of `gpuAddress` per expert built on the CPU and uploaded with `newBufferWithBytes` (1771-1787), kernel dereferences `waddr[e]` (267-279) | `smap` (48 × 512 uint32: chunk << 28 \| 16 KiB page) written by the I/O threads, read by the kernels (`moe.metal:52-56`, function constant 20); resident-first split (`moe.metal:66-123`, constant 26) |
| SSD reads | `pread` (+ `O_DIRECT`/`F_NOCACHE` twins, `st.h:276`, `compat.h:55-61`), `posix_fadvise(WILLNEED)` → `F_RDADVISE` on macOS (`compat.h:30-48`) as a read-ahead *hint* (`qwen38_core.h:1523-1549`); OpenMP loader pool (`K3_PIPE`, `PIPE_WORKERS`); io_uring on Linux (`uring.h`) | `F_NOCACHE` + `F_RDAHEAD 0` (`stream.cpp:60-64, 117-119`), one request per expert from the contiguous `.experts` copy (`stream.cpp:79-128, 341-360`), 8 I/O threads with 2 reserved for urgent misses (`stream.cpp:156-158, 374-402`), pieces of one miss on several threads (`read_part`, 338-372) |
| cache | per-layer LRU + learned pin from `.coli_usage`, live `REPIN`, `CACHE_ROUTE` | size-classed LRU arena (`stream.cpp:129-155, 282-336`), profile fill, MoCE (λ=0.85, protect 3), next-layer read-ahead from `W(l+1)·XF(l)` (`stream.cpp:656-716`), deterministic limbo retirement (`stream.h:105-109`) |
| quant decode | fmt 1/2/3/4/5/6/8: int8, int4 (per-row / grouped), int2, bf16, E8/IQ3 lattice with FWHT-rotated activations (`moe_fwht` 345-364), fp8 e4m3 by bit tricks (126-144); fp32 activations, scalar `dot(float4)` | the ggml/IQ families + Q2_0 + GSQ-RCO packs with templated decoders (`quant.metal:24-120`), constant-memory vs threadgroup grid tables per format (`QUANT_CTAB`, window.cpp:1354-1355), matmul2d/MPP tiles for prompts (`mm_bench.metal:2-18`, serve log "matmul2d layout K4") |
| attention / recurrent | MLA absorption with materialized scores (`a_score`/`a_smax`/`a_clat`, 421-445); KDA recurrence one thread per (h,i) re-reading S from device twice (652-690) | QSA flash/top-k/merge (`qsa.metal:93-399`), GDN step kernels (`gdn.metal:36`), all fused into the window |
| speculative | MTP up to 3 + prompt lookup up to 5, **depth chosen online by `SpecGate`** (`spec_draft.h`); state snapshot per verify row for GDN/PLE rewind (`docs/speculative.md`) | MTP (one CB per draft round, `window.cpp:2805-2808`) + suffix drafter (`strata/spec/suffix_drafter.hpp`), typical / exact sampled acceptance (`strata_window_cli.cpp:383-415`), **fixed depth** |

## 2. what i measured (Apple M5, 10-core GPU, macOS 27.0, no strata running)

### 2.1 hazard tracking: tracked vs `MTLHazardTrackingModeUntracked` — no difference
`benchA.mm`/`benchA2.mm`: 1,600 dispatches per CB, 7 buffer bindings + 16 B of `setBytes` each (≈ a decode window's shape),
`commandBufferWithUnretainedReferences`, 24 × 4 MiB buffers rotated so every dispatch reads the previous one's output. warm, median of 25:

| dispatch | encoder | barrier | tracked: encode / GPU | untracked: encode / GPU |
|---|---|---|---|---|
| 1 TG, trivial | concurrent | every dispatch | 0.601 / 1.548 ms | 0.596 / 1.575 ms |
| 1 TG, trivial | concurrent | none | 0.320 / 0.110 | 0.321 / 0.109 |
| 1 TG, trivial | serial | (implicit) | 0.309 / 1.412 | 0.307 / 1.411 |
| 64 TGs, trivial | concurrent | every dispatch | 0.609 / 2.147 | 0.605 / 2.137 |
| 64 TGs, ~20 µs ALU | concurrent | every dispatch | 0.595 / 99.40 | 0.595 / 99.42 |

- **tracked = untracked within noise in every arm.** the first run of `bench.mm` showed tracked 3.6 ms vs untracked 1.7 ms GPU, but
  that was tracked-first on a cold GPU; with a warm-up and the order swapped the gap is gone (`benchA2`). don't flip
  `MTLResourceHazardTrackingModeUntracked` on the strength of that kind of number.
- with untracked buffers the shared-expert encoder that runs between signal and wait (`mtl_rt.mm:292-295`) would also need an
  `MTLFence`, since the event wait doesn't order prior encoders. so it's risk for ~0 gain. colibri never measured its own switch either.
- the number that is real: **a buffer barrier costs ~0.9 µs of GPU per dispatch** (1.55 ms with vs 0.11 ms without for 1,600 tiny
  dispatches). that's the existing barrier-pruning line of work (audits/argon_prefill_barriers.md did it for prefill), not a colibri idea.
- encode is 0.6 ms for 1,600 dispatches with barriers, matching the ~0.96 ms "encode/submit/wait" per window in
  opus_strata_metal_opts_v3.md:111. replaying a cached window as an ICB (`mtl_rt.mm:341-388` already has `run_icb` for benchmarks)
  could save ~0.4 ms/window. adjacent, not from colibri.

### 2.2 the layer handshake: colibri's "passive wait" lesson does not transfer
`bench.mm` B: 48 segments of a 48 MB streaming kernel, each followed by signal + wait like `run_graph_seg`, host releases immediately. median of 20:

| host wait | window wall | CB GPU | waiter CPU |
|---|---|---|---|
| no handshake (one encoder) | 18.44 ms | 18.15 | 0.06 |
| **spin on `signaledValue` (strata, `mtl_rt.mm:301`)** | 21.24 | 21.03 | 21.2 |
| spin + `__builtin_arm_yield` | 21.30 | 21.11 | 21.3 |
| spin + `std::this_thread::yield` | 20.88 | 20.67 | 20.5 |
| `waitUntilSignaledValue:timeoutMS:` | 27.52 | 27.24 | 0.59 |
| `notifyListener` + condvar | 27.20 | 26.96 | 0.59 |

- spin costs **~60 µs per handshake** (same as opus_strata_metal_opts_v1 §2.3's 57–73 µs); either passive wait costs **~190 µs**, i.e.
  **+6.2 ms per token** on 48 layers. keep the spin. (the `this_thread::yield` variant is ~0.4 ms/window faster in two runs; within
  noise, but free to try on the real engine.)
- the colibri report's trap (OMP hot-team active spin on 18 cores tripled attention kernel time on the M5 Max,
  `docs/METAL-M5MAX-PERF-REPORT.md`, rows A→B) is about *many* spinning cores sharing the SoC power budget. `benchC.mm`: GPU streaming
  bandwidth with N CPU threads spinning, 160 windows each, twice: 0 → 132.4/131.3 GB/s, **1 → 132.4/131.2**, 4 → 132.0/130.9,
  10 → 129.7/129.6. one spinning core is free on the Air (in 6 s bursts; a 10-minute fanless thermal test is still open), 10 cost 2 %.
  the takeaway for us: **nothing else may spin while a window runs**. today that holds: I/O threads block in `pread`/condvars
  (`stream.cpp:381`), and `gpu/runtime.cpp:182` yields.
- a sleep-until-predicted-then-spin hybrid (`benchD.mm`) gave nonsense timings and i ran out of budget to debug it; not a result either way.

### 2.3 entropy coding the experts (colibri's `int4-rans256-g0`): doesn't apply to IQ2_XS
colibri entropy-codes int4 expert nibbles losslessly, ratio ~0.76 (`docs/int4-rans256-g0.md`), because a symmetric int4 quantizer's
codes are far from uniform. on our `K512-iq2_xs-metal.gguf.experts`, 24 random 1 MiB pages: **byte order-0 entropy 7.912 / 8 bits
(0.989), u16 entropy 15.77 / 16 (0.986), zlib-9 0.973, lzma-6 0.975.** IQ2_XS grid indices + sign bits are already ~uniform, so the
most a static rANS could save is 1–2.7 % of the ~5 ms SSD wait (≤ 0.1 ms) at a real CPU decode cost. dead end.

## 3. the stealable items in detail

### 3.1 `SpecGate`: online cost-model draft depth (rank 1)
**colibri** (`c/spec_draft.h`): per draft source it keeps a_j = P(draft j accepted | drafts before it accepted) as an EMA with memory
16 verifies (`spec_gate_result`, 107-117), the measured wall time T(n) of an n-row forward as an EMA with memory 8 (`spec_gate_forward`,
85-96), a fitted ρ for row counts not seen yet, and the per-extra-draft cost d of the drafter (`spec_gate_draft_cost`, 99-102). each
verify picks k = argmax over 0..maxk of

    value(k) = E(k) / cost(k),  E(k) = 1 + a1 + a1·a2 + … + a1…ak,  cost(k) = (T(k+1) + (k-1)·d) / T(1)

and drafts only if value > 1 (`spec_gate_value`/`spec_gate_pick`, 129-160). it probes one deeper every 16 picks (every 4 while that
row count has < 3 timing samples) so estimates can recover, and forces a plain step after 64 drafted forwards so T(1) stays measured.
tokens never change: it only decides how many rows the verify carries.

**strata**: the window size is fixed by flags. `mtp_draft` stops a draft chain when the drafter's prob < `spec_min_p` (0.35 in the
served config), suffix drafts fill up to `max(suffix_draft, nd)` (`strata_window_cli.cpp:720-733`), and nothing looks at what a
T=2/3/4 window actually cost. under SSD streaming that cost is not 1 + small: each extra row brings its own top-10 per layer, ~+60 %
experts moved for T=2 (opus_strata_metal_opts_v1 §0 #7), and sol_spec_mtp_decode.md:116-118 works out that a T=4 window only breaks
even if it finishes under ~2.5× T(1) at 0.7 acceptance. the true ratio depends on the arena state and the text, so it should be
measured online, which is exactly what the gate does.

**worked example** (illustrative numbers, not measured): T(1) = 46 ms, T(3) = 70 ms (1.52×), MTP draft 2 ms/step.
- code/repetitive text, a1 = 0.8, a2 = 0.7: E(2) = 2.36, cost 1.57 → value 1.50 (draft 2, +50 %).
- persona prose at temp 0.6, a1 = 0.35, a2 = 0.3: E(2) = 1.455, cost 1.57 → **0.93 < 1**; E(1) = 1.35 vs cost(1) = T(2)/T(1) ≈ 1.3
  → 1.04, basically break-even. the fixed `--spec 3` keeps paying T(3) or T(4) here, ~7 % slower than plain decoding.

**sketch for strata** (host only, `strata_window_cli.cpp` + a small header next to `suffix_window.hpp`):
1. `struct SpecGate { double acc[2][8], acc_w[2][8], t[10], t_w[10], rho, rho_w, d[2], d_w[2]; int picks[2], since_plain; }` with
   colibri's EMA update; source 0 = MTP, 1 = suffix.
2. after each `window_run(w, 1 + nd, …)` (line 735): `spec_gate_forward(g, 1 + nd, wall_ms)`. use the window's own wall time (it
   already exists: `win_ms_[t]`, window.cpp:456-457), so SSD waits and the larger expert union are inside the cost.
3. after `keep` is known (745-747): `spec_gate_result(g, src, judged = min(nd, keep), accepted = keep - 1)`, split by source when MTP
   and suffix drafts are concatenated (input[1..n_mtp] vs the rest).
4. before building `input` (720-733): `k = spec_gate_pick(g, MTP, n_mtp_drafts)`, then `nd = min(k, max_d)`. strata can do better
   than colibri here: **use the drafter's own `mtp_probs[i]` as a1..ak for this verify** (calibrated per token, already returned by
   `mtp_draft`), and keep the EMA only as a calibration factor between the drafter's prob and real acceptance (`acc ≈ c_j ·
   mtp_probs[j]`). then `spec_min_p` becomes a derived quantity instead of a hand-set 0.35.
5. pass `spec - 1` drafts to `mtp_draft` as today; the gate trims. `STRATA_SPEC_GATE=0` restores the fixed behaviour.
6. report line like colibri's `spec_gate_describe` (162-180): T(1), T(n)/T(1), acceptance by position. it's free telemetry we lack.

exactness: unchanged by construction (it only shortens the drafted tail). falsifier for the A/B: tok/s on the three domains
(ops_tools, chat_json, persona prose) at temp 0.6 with `--spec 3`; gate on must be ≥ fixed on all three, and the prose domain is
where it should win.

### 3.2 max-rank window on MoCE (rank 2)
colibri's cache-aware routing (`docs/CACHE_ROUTE.md`, after arXiv:2412.00099) keeps the true top-J and fills the rest preferring
residents **that still rank inside the true top-M** (`ROUTE_M=12` default), with agreement/KL telemetry (`route_agree`, `route_kl`).
strata's MoCE (`router.metal:43-111`) multiplies every resident's prob by `moce_scale` from rank `r_prot` on, with no rank bound. the
v1 audit found the bonus is large next to the router's spread (median z1 − z3 = 0.35 logits vs bonus 0.85), so deep-ranked residents
do get pulled in. a cap is cheap: at `rank == r_prot`, before scaling, find the M-th largest unscaled prob (M extra
`simd_shuffle_xor` argmax rounds over the 16 values per lane, or one threshold pass), and scale only residents with `v[i] >= thr`.
expected effect: lower routed relMSE at a small miss increase. it's a quality knob, so it needs the replay harness numbers (misses/tok
and relMSE at M = 16/24/32 vs no cap) before anyone ships it; `ahead()`'s `eff_lg` (stream.cpp:666-668) would get the same cap so
the read-ahead stays consistent with the router.

### 3.3 a usage history that learns across sessions (rank 3)
colibri appends per-turn expert counts to `<model>/.coli_usage` (`route_trace.h:19-60` format), uses it as the startup pin set when
`PIN=auto` (`colibri.c:13216-13229`), and can re-pin between turns from a decaying map (`colibri.c:9729-9739`). strata's arena fill
reads only the static calibration ranking (`stream.cpp:440-466`) and marks the top of it prompt-hot (`stream.cpp:601-612`). sketch:
- at shutdown and every N windows while idle, write `<gguf>.experts.usage`: per (layer, expert) a decayed hit count (bump in
  `at_layer` where `where_` is consulted, stream.cpp:798-826), plus the LRU order.
- `make_arena`: rank = sort by `α·usage + (1−α)·profile_rank_score` (α≈0.7 once the history has ≥ 1k tokens), same stepped fill.
- keep the exact-answer property: residency doesn't change math without MoCE, and with MoCE it only changes which resident wins, as
  today's LRU already does. but it does change routing under MoCE, so the A/B should be after a restart, first 512 tokens, hit rate
  and tok/s.

### 3.4 PLE rows early (rank 4)
colibri computes all PLE row ids at forward start and issues the reads in parallel, collecting at the PLE layer
(`qwen38_core.h:2060-2110`; its comment: "Riordino puro: stessi byte, letti prima"). strata reads them synchronously in
`Window::run` before the CB starts (`window.cpp:1480-1491`), the GEMVs on `emb_` sit at the top of the window (1546-1551), and the
metal path never calls the async `PleTable::prefetch_rows` the core path uses (`src/core/mtp.cpp:1581`). the cheapest version: when
`mtp_draft` returns drafts (strata_window_cli.cpp:770), compute the next window's row ids for `[cur_tok, drafts…]` and call
`prefetch_rows`. `gather_batch`'s exact-match fast path (`ngram.cpp:513-537`) then collects the in-flight tickets without
re-reading. bounded by the 0.225 ms "host before", so it's a small win. pair it with 3.1.

### 3.5 two cheap guards (rank 5)
- **grid-size check.** colibri found that M-series silently skipped threadgroups past a grid of ~5.4e7 (clean at 3.1e7), leaving stale
  output, and now chunks every GEMM to ≤ 2^25 elements (`backend_metal.mm:1653-1680`). their kernel indexes rows with int, so it may
  partly be their own overflow, but strata's `require_fit` (`mtl_rt.mm:159-169`) only checks threads per threadgroup, the other
  silent-skip mode. a one-line `grid[0]*grid[1]*grid[2] < 2^25` warning in `encode()` (mtl_rt.mm:208-226) costs nothing and covers
  2048-token prompt chunks if someone raises the chunk size.
- **GPU keep-alive after idle.** `COLI_METAL_SPIN` (`backend_metal.mm:1060-1085`) keeps a trivial CB in flight on a separate queue so
  the GPU doesn't clock down between bursts. strata's steady decode has no gaps worth it, but the first windows of a request after the
  server idled may be slow while DVFS ramps up. measure TTFT and the first 8 windows' ms after 60 s idle. if they're slow, run
  `Context::warm` for ~50 ms when a request arrives, before prefill.

## 4. looked at and not worth taking

| colibri mechanism | where | why not |
|---|---|---|
| `MTLHazardTrackingModeUntracked` | 753, 928-929 | measured no difference (§2.1); would need fences around the overlap encoder |
| `MTLResidencySet` replacing `useResource:` | 776-862, 995-1017, 1791-1802 | colibri needs it because it re-declares thousands of slabs per CB; strata binds ≤ 6 big buffers with `setBuffer`. nothing to amortize. (colibri's own E4 heap variant: −85 % GPU stall but +12 s disk-load regression; E5 never measured.) |
| passive OMP / host waits | METAL-M5MAX-PERF-REPORT.md | §2.2: passive event waits cost +130 µs per handshake; one spinning core is free |
| `F_RDADVISE` / `posix_fadvise(WILLNEED)` read-ahead hints | `compat.h:30-48`, `qwen38_core.h:1523-1549` | hints go into the unified buffer cache; strata does real async `F_NOCACHE` preads into the arena, deterministically retired (stream.cpp:656-716, `limbo_`) |
| F_NOCACHE storage probe with cold-range steering | `colibri.c:11975-12164` | useful only for picking a cache default by SSD speed; strata sizes the arena by memory (arena_budget.h) and the Air has one SSD |
| multi-SSD mirrors / bandwidth-weighted striping | `docs/multidisk.md`, `deepseek_v4.c:336-515` | one internal SSD on the laptop |
| `int4-rans256-g0` entropy-coded experts | `docs/int4-rans256-g0.md`, `rans.h` | §2.3: IQ2_XS bytes are ~99 % entropy |
| int4-g64 expert sidecar | `docs/qwen38.md` "Routed experts as int4-g64" | strata already streams 2-bit-class packs (1.38–1.75 MB/expert vs their 2.76 MB) |
| async MoE begin/end ("resident work submitted before misses are read") | 1903-1936, `docs/metal.md` | strata's resident-first split inside one CB (`moe.metal:66-123`, `STRATA_METAL_EXPERT_SPLIT=1`) is the same idea without a CPU round trip |
| `r_top8_par` simdgroup top-k | 502-558 | strata's `router_route` does the softmax, top-10, MoCE and plan in one dispatch (`router.metal:118`) |
| fp8 e4m3 bit-trick decode, E8 lattice + FWHT | 126-144, 289-309, 345-364 | not our formats; the FWHT rotation trick is relevant only to rotated quantizers (QuIP#/E8), which GSQ-RCO isn't |
| offline residency simulator | `docs/experiments/cnre-offline-simulator.md`, `tools/residency_sim.py` | we already have closed-loop MoCE-aware replays (audits/opus_strata_metal_opts_v1.md §1, audits/sol_moe_cache_policy.md), which colibri's tool explicitly doesn't model (no prefetch, no routing substitution) |
| prompt lookup drafts | `spec_draft.h:182-198` | = strata's suffix drafter (already wired, `strata_window_cli.cpp:710-733`) |
| verify-row state copies for GDN/PLE rewind | `docs/speculative.md` | strata folds the kept prefix at commit (`window.cpp:380, 716, 1120, 2211-2212`), no per-row copies needed |
| continuous batching / multiplexed rows (`mux_rows`) | `qwen38_core.h:2097-2102`, `decode_batch.h` | throughput across users; single-user latency is our target |

## 5. where strata is ahead (don't regress these)

1. **one CB per decode window** with per-layer event handshakes and the shared expert hidden between signal and wait
   (`mtl_rt.mm:273-316`, `window.cpp:1610-1615`), vs colibri's CB + `waitUntilCompleted` + memcpy per layer or per op.
2. **zero activation traffic to the host** inside a window; colibri copies x in and out and scatter-adds expert outputs on the CPU.
3. **arena + `smap` indirection** (`moe.metal:52-56`): no per-dispatch address tables, no `useResource` lists, residency changes by
   writing one uint32. colibri rebuilds and uploads `gpuAddress` tables for every MoE block (1771-1787).
4. **resident-first expert split and MoCE in the router kernel**, closed-loop tested; colibri's CACHE_ROUTE is CPU-side, opt-in and
   labelled experimental.
5. **I/O scheduling**: urgent vs read-ahead queues with reserved urgent threads, one request per expert from the contiguous copy, a
   miss split over threads, deterministic limbo retirement so SSD timing can't leak into routing (`stream.cpp:338-439`, `stream.h:105-110`).
6. **kernel efficiency**: strata runs at 122–145 GB/s of a ~153 GB/s part (audits/opus_strata_metal_opts_v3.md); colibri's best
   `moe_gemv` claim is 358–389 GB/s (`backend_metal.mm:265-266`) on Max-class parts with 400–600 GB/s. relatively, ours is ahead, and
   it has fp32-free paths, matmul2d prompt tiles, constant-vs-threadgroup grid tables per format.
7. **QSA flash / GDN kernels fused into the window**; colibri's MLA path materializes S×H×T scores and caps out at 2^30 threads
   (`backend_metal.mm:1509`), its KDA step re-reads the whole state from device memory twice per token (652-690).
8. **safe math by default** for window/one-token bitwise identity (`mtl_rt.mm:80-88`), plus `require_fit` (159-169) against silent
   threadgroup-size skips. colibri runs default math and documents prefill top-1 flips on near ties (`docs/metal.md`, #622).
9. **the spin handshake**: keep it (§2.2). if someone ports colibri's "passive wait" advice to `run_graph_seg`, that costs ~6 ms/token.

## 6. colibri lessons worth writing down (not code)
- they lost 39 % on the M5 Max to busy-waiting CPU threads stealing the SoC power budget. for us: keep exactly one spinner (§2.2).
- E4 (MTLHeap sub-allocation of expert slabs) cut GPU residency stalls 85 % but added +12–13 s of expert-load time, which they put
  down to first-touch/lock contention on heap pages written by the CPU (`docs/experiments/e5-metal-residency-set.md`). strata's arena
  is one wired anonymous mmap wrapped once (`stream.cpp:530, 626-647`). don't move it into an `MTLHeap`.
- their DSv4.1 I/O study (`docs/experiments/dsv41-expert-io-2026-09-14.md`) shows a 208-thread OpenMP team hid a 63 % I/O win behind
  libgomp spinning: check that the CPU side isn't the bottleneck before measuring an I/O change. our version is the
  `STRATA_METAL_PROFILE=1` host-before / run_graph / GPU split.