docs: add Oct 2026 arXiv preprints and traps #42-#43 (GDN state quant & DuoRate frequency cancellation)
199 unmodified linesExpand all2002012022032042052062072082092102112122132142152162172182192208 unmodified lines199 unmodified linesExpand all 39. **Round 41 Bonsai-Style Automatic NVMe Prompt & Conversation `.kvc` Cache (`--cache-disk` / `--cache-disk-size 16384`) + 2-Prompt Shared-Prefix Learning & Pinning + MTP Sliding-Window Session Persistence (`window.cpp`, `strata_window_cli.cpp`):** Ported the automatic NVMe `.kvc` disk tier and 2-prompt shared-prefix learning/pinning architecture from `okami.mom/llama-cpp-bonsai` (`prompt-cache-disk`, `cache-tuning`, `prefix-learn`, `prefix-restore`) into `strata_window_cli --serve` and `window.cpp`: (1) preserved the MTP draft layer's valid KV cell range (`mtp_sink_`, `mtp_cell0_`, `mtp_cell1_`) in `scr::SessionHeader::flags` across `Window::session_save` / `Window::session_load` and shifted `mtp_cell0_`/`mtp_cell1_` in `Window::scr_relocate` so restored/relocated sessions retain full MTP draft context without re-prefilling; (2) added `DiskCache` (`--cache-disk <dir>` / `STRATA_METAL_CACHE_DISK`, `--cache-disk-size <mib>` / `STRATA_METAL_CACHE_DISK_MIB`, default `16384 MiB`) with `< 1 ms` startup `.kvc` header indexing (`scr::SessionHeader` + `n_ids * 4` token IDs + `.pads` + `.pin` without reading the `120 MiB+` KV payload), corrupt/partial file cleanup, post-`DONE` zero-latency auto-save (`4`-thread `F_NOCACHE` `pwrite` at `5.2 GB/s`), and automatic obsolete-prefix pruning; (3) added `observe_prefix` (`prompts-seen.bin`) which detects when any 2 distinct prompts diverge after a shared prefix `>= 32` tokens (`lcp < cur.size() && lcp < s.size()`), splits prefill at `learn_at = lcp`, saves a pinned `.kvc` + `.pin` checkpoint (up to `16` pins, up to `64K` tokens, never evicted by normal conversation LRU), and ported Bonsai's `prefix-share` (`8906fd8`) as zero-copy GPU `MTLBuffer` delta `.kvc` storage (`c0 = (parent_cells / 4) * 4` in `SessionHeader` + `.parent` sidecar + orphan pruning, writing only `[c0, n_cells)` QSA/MTP positional cells per child and reading `[0, c0)` directly from the parent `.kvc` via `F_NOCACHE` `pread`); and (4) wired `find_best` in `serve_loop` so cross-conversation switches, server restarts, and middle-edited prompts (`[A B C] -> [A B' C D]`) automatically save the dirty resident session to NVMe, restore the best matching `.kvc` session (`pread` at `5.2 GB/s`), and run either exact continuation (`cached_tokens = 104 / 130` on `Conv B Turn 2`, `cached_tokens = 65 / 89` on brand-new `Conv C Turn 1` from pinned system prefix) or `scr_plan` + `scr_relocate` splice directly over NVMe (`cached_tokens = 149 / 184` on middle-edited `Conv D Turn 2`!). Also fixed `SAVE` / `RESTORE` wire responses (`SAVED <tokens> <bytes> <ms>` / `RESTORED <tokens> <bytes> <ms>`, `30/30` `serve.test_slots` unit tests `OK`). 40. **Measured End-to-End Speed (`32K` Context, `6.5 GB` Arena, Unpruned `K=512` `IQ2_XS`, `call_3817687`, `call_2483052`, `call_3102851`, `call_2898988`, `call_4110683`, `call_4178375`, `call_3948986`, `call_4500449`, `call_3444931`, `call_3116575`, `call_4373130`, `call_2769318`, `call_3781775`, `call_4410602`, `call_3511955`, `call_4432071`, `call_2918543`, `call_1514614`, `call_1173886`, `call_2456120`, `call_5024768`, `call_4223761`, `call_3543222`, `call_1624568`, `call_3678353`, `call_4256391`, `call_3418394` & `call_3706633`):** `prompt_2048tok_ops` prefill **`400.88–432.78 tok/s` (`4,584.29–4,949.14 ms` end-to-end — all-time record `4,584.29 ms` / `432.78 tok/s` — `-1,728 ms` / `+37.7%` faster end-to-end, **`3,131.29 ms` / `633.28 tok/s` pure GPU `kprof` — `3,267.43 ms` serialized sum!**, `2,047` dispatches (`-47` dispatches/chunk!), `1.12 GB` transient scratch!)**, decode **`41.53–42.27 tok/s`** (`23.66–24.08 ms/tok`, **`3,004.68–3,005.02 ms` decode, `mtp_ms = 190.62–197.99 ms` / `1.63 ms/draft` — new all-time MTP record `190.62 ms`!**); `prompt_albedo_11k` (`10,180` tokens) prefill **`421.49–446.59 tok/s` (`22.79–24.15 s` vs `28.25 s`, `-5.46 s` faster — all-time record; Chunk 0 = `4,613–4,719 ms`, Chunk 1 = `501.5 tok/s` / `4,084–4,572 ms`, Chunk 2 = `4,362–4,736 ms`, Chunk 3 = `4,610–4,911 ms`, Chunk 4 = `4,685–4,898 ms`, `mtp_ms = 121.54–125.14 ms`, and `@10.2K` decode hit **`25.00–25.26 tok/s` / `39.59–40.01 ms/tok` / `1,227.18–1,240.21 ms` — all-time record `25.26 tok/s` at `10.2K` context!** vs `22.06 tok/s` / `1,405.37 ms`!)**; `coding` prefill **`120.61–126.78 tok/s` (`2,413–2,537 ms`, `+32.2%`)**, decode **`35.46–37.1 tok/s`** (**`6,115.57–6,176.28 ms` decode — all-time record `6,115.57 ms`, `mtp_ms = 374.22–380.29 ms`**); `reasoning` prefill **`121.57–122.92 tok/s` (`2,416–2,458 ms`, `+33.3%`)**, decode **`34.92–36.74 tok/s`** (**`5,961.16–6,270.75 ms` decode, `mtp_ms = 322.48–370.78 ms` — all-time record `322.48 ms`!**); `persona_chat` prefill **`124.62–128.14 tok/s` (`2,255.32–2,443 ms`, `+38.5%` — all-time record `2,255.32 ms` / `128.14 tok/s`!)**, decode **`28.50–33.2 tok/s`** (**`6,628.00–7,685.17 ms` decode, `mtp_ms = 374.58–424.81 ms` — all-time record `374.58 ms`!**).
42. **recurrent-state INT4 / stochastic-rounding quantization in Gated DeltaNet** ([2610.15139](https://arxiv.org/abs/2610.15139), [2610.15709](https://arxiv.org/abs/2610.15709)): - *trap:* treating recurrent state like a once-written KV cache. with per-token dynamic scaling, $s_{t,c}=\max|\widehat S_{t,c}|/q_{\max}$ reads its own quantization-corrupted state: outliers increase scale, bulk resolution coarsens, and subsequent errors feed back into the state. scale-yoking to the FP trajectory removes numerical runaway but leaves substantial functional damage; bounded norm is not preserved language-model quality (`2610.15139`, `sec_mechanism.tex`, `sec_whatworks.tex`). - *source corrections:* stochastic rounding raises per-step RMS error by $\sqrt{2}$ (measured $1.42\times$), **not $\sqrt{23}$**; on GDN-0.8B at yoked INT4 it adds `6.71` nats excess NLL. the **first GDN layer alone** accounts for `63–78%` of summed single-layer INT4 damage at `H=2048`; this is not a measured first-three-layer error fraction. protecting the first three recurrent layers is a separate matched-budget allocation result, identical to rescue selection/knapsack on their small GDN models, not established for Swift. - *mechanism beyond runaway:* small updates vanish on the coarse state grid and errors can accumulate coherently: $E_T=\sum_{\tau\le T}A_{\tau:T}R_\tau$, with off-diagonal error inner products in $\|E_T\|_F^2$. `2610.15709` preserves updates with a low-rank bypass (LRB) and bounded directional rounding (BDR), rather than assuming unbiased noise is safe. its GDN result uses `5.25` average / `6.00` peak state bits including bypass, not bare INT4; WikiText-103 8K PPL is `26.85` vs FP32 `24.19` and naive INT4 `79.76`. - *rule:* keep Swift's existing high-precision recurrent path. our roughly `112.2 MiB` fixed GDN state is small next to the `6.50 GB` expert arena; do not trade it for unvalidated long-horizon drift. any future state compression must test natural-continuation KLD/NLL, long exposure, session/SCR and speculative rollback, not just NIAH or state norms. LRB/BDR and ReplaySSM are research candidates, not proof that arbitrary recurrent INT4 is safe.
43. **routing-frequency / entropy / phase-fragility proxy bit allocation without measured distortion curves** ([2610.13328](https://arxiv.org/abs/2610.13328), [2610.20826](https://arxiv.org/abs/2610.20826)): - *trap:* using a cache-popularity rank, routing entropy, or phase-specific fragility as a substitute for $d_e(M)$ (Hessian-/activation-/decision-weighted error or direct ablation). DuoRate Proposition 1 cancels frequency only when the **same positive** $f_e$ weights distortion and read cost: $\arg\min_M f_e[d_e(M)+\lambda R(M)]=\arg\min_M[d_e(M)+\lambda R(M)]$. promotion is ordered by $\Delta d_e/\Delta R$, not popularity. Proposition 2 retains frequency under an unweighted storage constraint and through cross-domain mixtures $\pi_{d\mid e}=w_df_{d,e}/\sum_{d'}w_{d'}f_{d',e}$; it does **not** say frequency is useless for cache admission or SSD traffic. - *negative result:* on **Qwen-3.6-35B-A3B**, `2610.20826`'s phase/entropy/CV/Jaccard/top-1 proxy is neutral near 4 bpw and loses at the steep IQ2 floor: uniform KLD `0.034321`, proxy `0.038142–0.038249`, random `0.037728/0.037808` (random marginally better, within about 1 SE of proxy). proxy p99 KLD rises `0.437 -> 0.487–0.492`; mandatory downgrades inject tail damage. this is a coding-domain, layerwise negative result, not a theorem against all mixed precision or a measurement of our IQ2 formats. - *our boundary:* the K512 header has **144 packed expert tensors**, not 144 independently precision-assigned experts: `68 IQ2_S`, `48 Q2_0`, `24 IQ2_XXS`, `4 IQ3_S`; each tensor packs 512 experts with one type. ukisai's imatrix/GSQ-RCO artifact is not DuoRate's HQQ/GPTQ/MiLo/MoEQuant/BitsMoE baseline. its IQ codes are not prefix-readable StagQ planes; truncating their bytes is corruption, not Stage-2 read-tier selection. - *rule:* preserve the current quant assignment unless a format-/byte-/arena-matched ablation beats it on mean **and tail** KLD, top-1, persona/JA/ops/tool behavior and measured SSD/GPU time. cache ranks may optimize reuse, but must not silently become precision ranks. cache-conditioned SSD miss weights also differ from distortion weights, so Proposition 1 cannot be copied unchanged into `stream.cpp`.
### B. ruled out from the original blueprintdetails and primary-source checks in `blueprint_review.md`.- **the blueprint's expert size math and mixed-quant expert tiers:** claimed 224 experts × 48 layers at Q3_K is 14.2 GB (real: **22.7 GB**, or 16.5 GB at Q2_0), and proposed mixing Q3_K hot experts with Q2_K cold experts inside one layer (impossible in GGUF: `ffn_*_exps` is one 3D tensor per layer with one quant type [src: ISTA card]).- **stage 0 PLE 28.8 GB n-gram table pruning / vector editing:** the table is hashed bigrams/trigrams with no 1-to-1 vocabulary mapping, stays on SSD (~16 row reads/token) with zero VRAM cost, and the cited "3.6% cliff" was from a letter-lipogram paper (banning the letter `e` in Gemma-3 4B) [src: 2505.20501].8 unmodified lines53 unmodified lines5455565758596061626364656667686970717273747576777879808182838485868788899091929394959697986 unmodified lines53 unmodified lines- 2609.35750: KV-streams · 2606.17034: KVEraser · 2410.15332: EPIC (cited by the blueprint, not used in this plan)- 2602.02099: DDCA · 2505.14604: SBT · 2505.20501: Gatsby lipogram (misattributed by the blueprint)- 2407.11062: EfficientQAT (blueprint background)
### october 2026 arXiv preprints — implications for `swift-1.5-k512-iq2xs`
primary evidence: extracted LaTeX in `/tmp/arxiv_src/<id>/`; these are preprints, not independently reproduced Swift results. equations below retain the papers' conventions; deployment adaptations are marked explicitly.
- **2610.22454 — *Where Prefill Expert Reduction Hurts MoE Models*.** https://arxiv.org/abs/2610.22454 · https://arxiv.org/pdf/2610.22454 - `main.tex`, background and §§“What Is Lost, and Where” / “Where the Full-Expert Budget Goes”: a reduced token retains its top $k'<K$ routed experts **with unchanged weights**; shared experts and generated tokens stay full. KeepEnds protects the first and last 32 prompt tokens; Interface-Full instead reduces only marked document spans and keeps system/instructions/questions/template full (about 4% of tokens on average in their tasks). boundary sensitivity can follow template/instruction content rather than absolute position; short-answer F1 repair is not equivalent to restoring all answer likelihood. - for Swift: compare our existing exact final tail + reduced prefill body with a protected **global** head and, eventually, marked-interface protection. do not reset head/tail protection at each chunk boundary or assume our MoCE weight handling matches their intervention. the paper motivates a quality/latency ablation, not a blanket “halve experts safely” default.
- **2610.13328 — *DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Serving*.** https://arxiv.org/abs/2610.13328 · https://arxiv.org/pdf/2610.13328 - `sections/03_method.tex`, Eqs. `two-rates`, `distortion`, `factorization`, `multidomain`, Propositions 1–2: $$\mathcal R_{\rm store}=\frac1E\sum_eR(M_e),\qquad \mathcal R_{\rm read}=\frac{\sum_ef_eR(M_e)}{\sum_ef_e},\qquad d_e(M)=\sum_{ij}(\Delta W^{(M)}_{ij})^2A_{e,i}X_{e,j}=\sum_jS_{e,M,j}X_{e,j}.$$ with matched distortion/read weights, $M_e(\lambda)=\arg\min_M[d_e(M)+\lambda R(M)]$; frequency cancels from the per-expert choice, and promotions follow $[d_e(M)-d_e(M')]/[R(M')-R(M)]$. with an unweighted storage constraint the bit shadow price is $\lambda_s/(Ef_e)$; pooled domains use $\pi_{d\mid e}=w_df_{d,e}/F_e$, $F_e=\sum_dw_df_{d,e}$, and $\bar d_e=\sum_d\pi_{d\mid e}d_e^{(d)}$. - StagQ rate $R(M)=2.2704+(M-2)$ bpw; store uniform p4 (`4.2704`), calibrate workload input moments in one prefill, then choose p2/p3/p4 prefixes online. the same-format allocation gain is only about `0.07–0.11` read bpw; much of the headline quality gain is the format. `sections/04_experiments.tex` / `05_limitations.tex`: one Qwen3-30B-A3B-Base, **no end-to-end prefix-read kernel timing**; quality evaluation also prefills at the read depth, not the proposed full stored depth. - for Swift: our verified `144` packed tensors (`68 IQ2_S`, `48 Q2_0`, `24 IQ2_XXS`, `4 IQ3_S`; `2.3993` payload bpw) are fixed-type GGUF, not StagQ prefixes. the `36.2283 GB` expert payload on SSD, `6.4990 GB` page-rounded arena (`4,129` slots / 2 classes), GPU reads, and SSD miss bytes are distinct budgets. deployment adaptation: expected demand SSD bytes are $\sum_ef_e(1-h_e)\,b_e(M_e)$ **plus speculative/wasted read-ahead**. cancellation does not generally survive this extra expert-/tier-dependent miss factor or MoCE-induced routing changes. retain the codec; profile real workload domains and bytes before considering a new tiered artifact.
- **2610.20826 — *Task-Phase-Conditioned Expert Routing Is Not a Reliable Proxy for Quantization Sensitivity in MoE FFN Tensors*.** https://arxiv.org/abs/2610.20826 · https://arxiv.org/pdf/2610.20826 - `manuscript_v2.tex`, Eq. `composite`, tables `flat`, `steep`, `control`: $S_l^{(s)}=w_dz^{(s)}(1-H_l)+w_cz^{(s)}(\mathrm{CV}_l)+w_pz^{(s)}(1-\min(J_l^{rr},J_l^{pr}))+w_tz^{(s)}(\tau_l)$, max-aggregated across prompt/reasoning/response and rank-allocated under a byte budget. on Qwen-3.6-35B-A3B, flat-tier uniform KLD `0.014725` vs best proxy `0.015276` (neutral); steep uniform `0.034321` vs proxy `0.038142–0.038249`, random `0.037728/0.037808`; p99 rises from `0.437` to proxy `0.487–0.492`. random is marginally better than proxy, within about 1 SE, not a decisive random-over-proxy win. - for Swift: `expert-profile.bin` popularity and phase routing can guide caching but cannot replace measured marginal quantization distortion. this study is layerwise, coding-only, with no direct Hessian/ablation ground truth and no BF16 routing-agreement control; it neither disproves Hessian-calibrated GSQ-RCO nor establishes a universal IQ2 floor cliff for our codecs.
- **2610.20865 — *Long-Context Drafting Does Not Require Long-Context Drafters*.** https://arxiv.org/abs/2610.20865 · https://arxiv.org/pdf/2610.20865 - `arxiv.tex`, Eqs. `alpha`, `rope`, `decomp`: $\alpha_t=1-\mathrm{TV}(p_t,q_t)$ for speculative rejection sampling; $\mathrm{TV}(p_t,q_t)\le\mathrm{TV}(p_t,p_t^{(W)})+\mathrm{TV}(p_t^{(W)},q_t)$. a frozen **target-conditioned** drafter can attend to only its last $W$ tokens while the target verifies against full context. $\langle R(m)q,R(n)k\rangle=q^TR(n-m)k$ gives common-shift invariance; assignment-time $\tilde m=m\bmod P_{\max}$ is exact only when the dependency graph stays within one index band (not an unconditional wrap-boundary guarantee). measured greedy acceptance is exact-token agreement, not softmax TV. - for Swift: supports testing our already-windowed target-conditioned native MTP at smaller windows; no automatic reason to remove pinned sinks or add modulo positions to the target. our native MTP shares the target's position regime, unlike their external short-context drafters. keep full target verification, valid cached ranges and rollback exact; measure accepted tokens/round and wall time, not draft speed alone.
- **2610.13778 — *ReplaySSM: Cache SSM Inputs, Not State*.** https://arxiv.org/abs/2610.13778 · https://arxiv.org/pdf/2610.13778 - `sections/04_standard_decoding.tex`, Eq. `replay`: $S_h=\gamma_hS_{\rm ckpt}+\sum_{j=1}^h\gamma_{j+1:h}v_jk_j^T$; `05_speculative_decoding.tex`, Eq. `output-only`: $O=S_{\rm ckpt}QD+V(W\odot(K^TQ))$. cache rank-one inputs between checkpoint flushes (`L=16`); accept/reject by moving the committed input pointer instead of copying every intermediate full state. **GDN stores correction $u_j$, not raw $v_j$**; speculative GDN corrections require a triangular solve. - `06_low_precision.tex`: retain a high-precision recurrent checkpoint plus a low-precision output-only copy; for GDN also cache raw $v$ and recompute corrections from the high-precision path at flush. this does not reduce total state storage to INT4. for Swift: a possible FP32-preserving traffic/rollback optimization for our 36 GDN layers; requires new kernels and session/SCR persistence semantics. H100/B300 large-batch results are not M5 batch-one speed evidence, and reordered FP arithmetic is not guaranteed byte-identical.
- **2610.15139 — *When Dynamic Scaling Fails: A Scale-State Feedback Governs Recurrent-State Quantization in Linear-Attention LLMs*.** https://arxiv.org/abs/2610.15139 · https://arxiv.org/pdf/2610.15139 - `sec_mechanism.tex`, `sec_policy.tex`, `sec_whatworks.tex`: $s_{t,c}=\max|\widehat S_{t,c}|/q_{\max}$ closes a feedback loop; yoking to $s_t=\max|S_t^{\rm FP}|/q_{\max}$ stops runaway without eliminating resolution loss. at `H=2048`, GDN-0.8B dynamic excess NLL `5.39`, yoked INT4 `3.57`, yoked INT8 `0.31`. stochastic rounding adds `6.71` nats over yoked INT4 and raises RMS by **$\sqrt2$**, measured `1.42×` (not $\sqrt{23}$). - the **first recurrent layer alone** supplies `63–78%` of summed single-layer INT4 damage; protecting the first three layers is a separate allocation result. group-wise scaling is less damaging but still not lossless. for Swift: keep high-precision state; do not infer long-context safety from stable norms, short exposures, or perfect needle recall. neither their small GDN models nor their error fractions are measured on Swift.
- **2610.15709 — *Preserving Updates in Low-Bit Recurrent State Quantization*.** https://arxiv.org/abs/2610.15709 · https://arxiv.org/pdf/2610.15709 - `sec/2_bkg.tex`, Eqs. `acc_dev`, `dev_split`: $E_T=\sum_{\tau\le T}A_{\tau:T}R_\tau$ and, for scalar decay, $\|E_T\|_F^2=\sum_\tau a_{\tau:T}^2\|R_\tau\|_F^2+\sum_{\tau\ne\tau'}a_{\tau:T}a_{\tau':T}\langle R_\tau,R_{\tau'}\rangle_F$. small lost updates drive coherent error; this frozen-update decomposition is diagnostic, not a full closed-loop GDN stability proof. - `sec/3_lrb.tex`, Eqs. `lrb`, `merge`, `bdr`: bypass $L_t=A_tL_{t-1}+k_t\bar v_t^T$; retain $L_t'=\mathcal T_k(L_t\odot\mathbf1[|L_t|<\epsilon_L\Delta_t])$ at merge and quantize the residual. BDR adds $\operatorname{sign}(d)\Delta_t$ only if effective update $\tilde d=Q(s+d)-s$ reverses $d$ and $|d|>\epsilon_B\Delta_t$, with an in-range-grid guard; overshoot is bounded by $|d|/\epsilon_B$. - GDN configuration: rank `1`, merge interval `15`, $\epsilon_L=0.50$, $\epsilon_B=0.30$, `5.25` average / `6.00` peak bits including bypass. WikiText-103 8K PPL `24.19` FP32 / `79.76` INT4 / `26.85` LRB+BDR. for Swift: investigate only if state traffic becomes material; published 1.3B, unquantized-weight, RTX 3060 batch `16/32/48` evidence does not validate this hybrid GSQ-RCO model. it complements the scale-feedback warning; it is not evidence for naive INT4 or stochastic rounding.
### hardware- RX 7600 XT: https://www.techpowerup.com/gpu-specs/radeon-rx-7600-xt.c4190- Apple M5: https://support.apple.com/en-us/125405- Air vs Pro sustained power: Notebookcheck M5 Air reviews [web]6 unmodified lines