# 04 — sources & references

## 11. sources

### models / repos (Hugging Face)
- Qwen3.8-Flash-Next card + `config.json`: https://huggingface.co/Qwen/Qwen3.8-Flash-Next
- Swift 1.5 BF16: https://huggingface.co/ukisai/Swift1.5-Qwen3.8-Flash-Next
- Swift 1.5 GSQ-RCO GGUF (base for this plan, tensor-allocation files): https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF
- SC117 abliterated transplant (the tensor-diff table + transplant method): https://huggingface.co/SC117/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF
- ISTA GSQ-RCO GGUFs (sizes, memory requirements, Q2_0 3.4× prefill vs IQ2_XS): https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF
- ISTA GSQ-RCO Coder (256-expert prune, per-layer budgets, calibration lessons): https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF
- REAP-320 multi-domain (reliability knee, fabrication probe, routing-mass warning, MTP +32%): https://huggingface.co/AnonimousA/Qwen3.8-Flash-Next-REAP-320-GGUF
- REAP-256-duo (344K calibration, CPU prefill +1.58× vs decode 1.02× tie, `--kv-unified` / `reasoning_effort` / empty-`content` stall notes): https://huggingface.co/AnonimousA/Qwen3.8-Flash-Next-REAP-256-duo-GGUF
- Qwen3.8-Flash-Next expert routing traces (3.84M prefill + 2.55M decode tokens, top-32 logits + bf16 router input states + fp32 gate matrices): https://huggingface.co/datasets/aswinkumar99/qwen3.8-flash-next-expert-traces
- Ukisai multi-turn agent SFT (15.2K Terminus-2 traces, `nl2bash` + terminal): https://huggingface.co/datasets/ukisai/Qwen3.8-27B-multi-turn-agent-sft
- Terminal-Bench 2.1 Qwen3.8 traces (89 ops tasks): https://huggingface.co/datasets/Lottolabs/terminal-bench-2.1-qwen3.8-27b-traces
- Whittle distillation set (25.1K reasoning + tool + JSON traces + top-20 logprobs): https://huggingface.co/datasets/logic65/whittle-distill-qwen3.8-27b
- Japanese imatrix calibration (~200K tokens JA+EN): https://huggingface.co/datasets/ChiTako/japanese-imatrix-calibration
- Japanese QA + reasoning 100K (`<think>` chains): https://huggingface.co/datasets/hotchpotch/japanese-qa-reasoning-100k
- Whittle (n-gram transfer, margin values, preserve-reasoning lesson): https://huggingface.co/logic65/Whittle-Qwen-3.8-35B-A3B , https://huggingface.co/logic65/Whittle-Next-27B-A3B
- Gadsby-4B: https://huggingface.co/errrinn/Gadsby-4B
- Qwen3.8-27B (for architecture comparison): https://huggingface.co/Qwen/Qwen3.8-27B

### runtimes
- Strata: https://github.com/Niko1221/Strata (`README.md`, `docs/MAC.md`, `docs/MODELS.md`, `docs/DETAILS.md`, `docs/AMD_HIP.md`; issues #192, #438, #938, #942)
- Slotstream: https://github.com/carloslfu/slotstream
- ds4-recipe: https://tangled.org/astrra.space/ds4-recipe (`README.md`, `00-start-here-environment.md`, `06-expert-cache-and-routing.md`, `07-prefill-pipeline.md`, `13-suffix-cache-reuse.md`)
- llama.cpp PRs: https://github.com/ggml-org/llama.cpp/pull/27742 (merged), /27861 (open), /28136 (closed), /28243 (closed), /22929 (merged)
- SCR reference code: https://github.com/facebookresearch/context-language-models/tree/main/suffix_cache_reuse
- RCO: https://github.com/IST-DASLab/RCO · GSQ: https://github.com/IST-DASLab/GSQ
- Antislop: https://github.com/sam-paech/auto-antislop
- jacobian-lens: https://github.com/anthropics/jacobian-lens
- Modal, mixed-precision/mixed-accelerator RL samplers: https://modal.com/blog/toward-rl-on-mixed-precision-mixed-accelerator-samplers
- Connectome host: https://github.com/anima-research/connectome-host (`README.md`, `docs/AGENT-MEMORY-GUIDE.md`, `docs/AGENT-ONBOARDING.md` §5b)
- Connectome KV/RoPE/rolling thread: https://animalabs.ai/field-notes/kv_perturbation_thread/ (experiments: github.com/mlegls/kv-diff-experiment)
- connectome-min: https://github.com/Phylliida/connectome-min

### papers (arXiv)
- 2608.30320: Qwen3.8-Next architecture (tech report)
- 2402.10962: instruction (in)stability / drift in dialogs
- 2507.11538: IFScale, adherence vs instruction density
- 2412.00099: MoCE, Mixture of Cache-Conditional Experts (Qualcomm)
- 2609.37725: Context Language Models; SCR in §4.3 / App. B
- 2608.26070: Prefix Sliding
- 2605.00649: RCO, Riemannian Constrained Optimization
- 2604.18556: GSQ, Gumbel-Softmax Quantization
- 2510.15061: Antislop / FTPO
- 2311.09677: R-Tuning
- 2405.05904: Gekhman et al., fine-tuning on new knowledge → hallucination
- 2403.05612: Kang et al., unfamiliar finetuning examples control hallucination
- 2509.04664: Kalai et al., why language models hallucinate
- 2312.17238: Eliseev & Mazur, MoE offloading (next-layer gate prefetch)
- 2601.07372: Engram (input-side n-gram memory)
- 2609.35750: KV-streams · 2606.17034: KVEraser · 2410.15332: EPIC (cited by the blueprint, not used in this plan)
- 2602.02099: DDCA · 2505.14604: SBT · 2505.20501: Gatsby lipogram (misattributed by the blueprint)
- 2407.11062: EfficientQAT (blueprint background)


### october 2026 arXiv preprints — implications for `swift-1.5-k512-iq2xs`

primary evidence: extracted LaTeX in `/tmp/arxiv_src/<id>/`; these are preprints, not independently reproduced Swift results. equations below retain the papers' conventions; deployment adaptations are marked explicitly.

- **2610.22454 — *Where Prefill Expert Reduction Hurts MoE Models*.** https://arxiv.org/abs/2610.22454 · https://arxiv.org/pdf/2610.22454
  - `main.tex`, background and §§“What Is Lost, and Where” / “Where the Full-Expert Budget Goes”: a reduced token retains its top $k'<K$ routed experts **with unchanged weights**; shared experts and generated tokens stay full. KeepEnds protects the first and last 32 prompt tokens; Interface-Full instead reduces only marked document spans and keeps system/instructions/questions/template full (about 4% of tokens on average in their tasks). boundary sensitivity can follow template/instruction content rather than absolute position; short-answer F1 repair is not equivalent to restoring all answer likelihood.
  - for Swift: compare our existing exact final tail + reduced prefill body with a protected **global** head and, eventually, marked-interface protection. do not reset head/tail protection at each chunk boundary or assume our MoCE weight handling matches their intervention. the paper motivates a quality/latency ablation, not a blanket “halve experts safely” default.

- **2610.13328 — *DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Serving*.** https://arxiv.org/abs/2610.13328 · https://arxiv.org/pdf/2610.13328
  - `sections/03_method.tex`, Eqs. `two-rates`, `distortion`, `factorization`, `multidomain`, Propositions 1–2:
    $$\mathcal R_{\rm store}=\frac1E\sum_eR(M_e),\qquad \mathcal R_{\rm read}=\frac{\sum_ef_eR(M_e)}{\sum_ef_e},\qquad d_e(M)=\sum_{ij}(\Delta W^{(M)}_{ij})^2A_{e,i}X_{e,j}=\sum_jS_{e,M,j}X_{e,j}.$$
    with matched distortion/read weights, $M_e(\lambda)=\arg\min_M[d_e(M)+\lambda R(M)]$; frequency cancels from the per-expert choice, and promotions follow $[d_e(M)-d_e(M')]/[R(M')-R(M)]$. with an unweighted storage constraint the bit shadow price is $\lambda_s/(Ef_e)$; pooled domains use $\pi_{d\mid e}=w_df_{d,e}/F_e$, $F_e=\sum_dw_df_{d,e}$, and $\bar d_e=\sum_d\pi_{d\mid e}d_e^{(d)}$.
  - StagQ rate $R(M)=2.2704+(M-2)$ bpw; store uniform p4 (`4.2704`), calibrate workload input moments in one prefill, then choose p2/p3/p4 prefixes online. the same-format allocation gain is only about `0.07–0.11` read bpw; much of the headline quality gain is the format. `sections/04_experiments.tex` / `05_limitations.tex`: one Qwen3-30B-A3B-Base, **no end-to-end prefix-read kernel timing**; quality evaluation also prefills at the read depth, not the proposed full stored depth.
  - for Swift: our verified `144` packed tensors (`68 IQ2_S`, `48 Q2_0`, `24 IQ2_XXS`, `4 IQ3_S`; `2.3993` payload bpw) are fixed-type GGUF, not StagQ prefixes. the `36.2283 GB` expert payload on SSD, `6.4990 GB` page-rounded arena (`4,129` slots / 2 classes), GPU reads, and SSD miss bytes are distinct budgets. deployment adaptation: expected demand SSD bytes are $\sum_ef_e(1-h_e)\,b_e(M_e)$ **plus speculative/wasted read-ahead**. cancellation does not generally survive this extra expert-/tier-dependent miss factor or MoCE-induced routing changes. retain the codec; profile real workload domains and bytes before considering a new tiered artifact.

- **2610.20826 — *Task-Phase-Conditioned Expert Routing Is Not a Reliable Proxy for Quantization Sensitivity in MoE FFN Tensors*.** https://arxiv.org/abs/2610.20826 · https://arxiv.org/pdf/2610.20826
  - `manuscript_v2.tex`, Eq. `composite`, tables `flat`, `steep`, `control`: $S_l^{(s)}=w_dz^{(s)}(1-H_l)+w_cz^{(s)}(\mathrm{CV}_l)+w_pz^{(s)}(1-\min(J_l^{rr},J_l^{pr}))+w_tz^{(s)}(\tau_l)$, max-aggregated across prompt/reasoning/response and rank-allocated under a byte budget. on Qwen-3.6-35B-A3B, flat-tier uniform KLD `0.014725` vs best proxy `0.015276` (neutral); steep uniform `0.034321` vs proxy `0.038142–0.038249`, random `0.037728/0.037808`; p99 rises from `0.437` to proxy `0.487–0.492`. random is marginally better than proxy, within about 1 SE, not a decisive random-over-proxy win.
  - for Swift: `expert-profile.bin` popularity and phase routing can guide caching but cannot replace measured marginal quantization distortion. this study is layerwise, coding-only, with no direct Hessian/ablation ground truth and no BF16 routing-agreement control; it neither disproves Hessian-calibrated GSQ-RCO nor establishes a universal IQ2 floor cliff for our codecs.

- **2610.20865 — *Long-Context Drafting Does Not Require Long-Context Drafters*.** https://arxiv.org/abs/2610.20865 · https://arxiv.org/pdf/2610.20865
  - `arxiv.tex`, Eqs. `alpha`, `rope`, `decomp`: $\alpha_t=1-\mathrm{TV}(p_t,q_t)$ for speculative rejection sampling; $\mathrm{TV}(p_t,q_t)\le\mathrm{TV}(p_t,p_t^{(W)})+\mathrm{TV}(p_t^{(W)},q_t)$. a frozen **target-conditioned** drafter can attend to only its last $W$ tokens while the target verifies against full context. $\langle R(m)q,R(n)k\rangle=q^TR(n-m)k$ gives common-shift invariance; assignment-time $\tilde m=m\bmod P_{\max}$ is exact only when the dependency graph stays within one index band (not an unconditional wrap-boundary guarantee). measured greedy acceptance is exact-token agreement, not softmax TV.
  - for Swift: supports testing our already-windowed target-conditioned native MTP at smaller windows; no automatic reason to remove pinned sinks or add modulo positions to the target. our native MTP shares the target's position regime, unlike their external short-context drafters. keep full target verification, valid cached ranges and rollback exact; measure accepted tokens/round and wall time, not draft speed alone.

- **2610.13778 — *ReplaySSM: Cache SSM Inputs, Not State*.** https://arxiv.org/abs/2610.13778 · https://arxiv.org/pdf/2610.13778
  - `sections/04_standard_decoding.tex`, Eq. `replay`: $S_h=\gamma_hS_{\rm ckpt}+\sum_{j=1}^h\gamma_{j+1:h}v_jk_j^T$; `05_speculative_decoding.tex`, Eq. `output-only`: $O=S_{\rm ckpt}QD+V(W\odot(K^TQ))$. cache rank-one inputs between checkpoint flushes (`L=16`); accept/reject by moving the committed input pointer instead of copying every intermediate full state. **GDN stores correction $u_j$, not raw $v_j$**; speculative GDN corrections require a triangular solve.
  - `06_low_precision.tex`: retain a high-precision recurrent checkpoint plus a low-precision output-only copy; for GDN also cache raw $v$ and recompute corrections from the high-precision path at flush. this does not reduce total state storage to INT4. for Swift: a possible FP32-preserving traffic/rollback optimization for our 36 GDN layers; requires new kernels and session/SCR persistence semantics. H100/B300 large-batch results are not M5 batch-one speed evidence, and reordered FP arithmetic is not guaranteed byte-identical.

- **2610.15139 — *When Dynamic Scaling Fails: A Scale-State Feedback Governs Recurrent-State Quantization in Linear-Attention LLMs*.** https://arxiv.org/abs/2610.15139 · https://arxiv.org/pdf/2610.15139
  - `sec_mechanism.tex`, `sec_policy.tex`, `sec_whatworks.tex`: $s_{t,c}=\max|\widehat S_{t,c}|/q_{\max}$ closes a feedback loop; yoking to $s_t=\max|S_t^{\rm FP}|/q_{\max}$ stops runaway without eliminating resolution loss. at `H=2048`, GDN-0.8B dynamic excess NLL `5.39`, yoked INT4 `3.57`, yoked INT8 `0.31`. stochastic rounding adds `6.71` nats over yoked INT4 and raises RMS by **$\sqrt2$**, measured `1.42×` (not $\sqrt{23}$).
  - the **first recurrent layer alone** supplies `63–78%` of summed single-layer INT4 damage; protecting the first three layers is a separate allocation result. group-wise scaling is less damaging but still not lossless. for Swift: keep high-precision state; do not infer long-context safety from stable norms, short exposures, or perfect needle recall. neither their small GDN models nor their error fractions are measured on Swift.

- **2610.15709 — *Preserving Updates in Low-Bit Recurrent State Quantization*.** https://arxiv.org/abs/2610.15709 · https://arxiv.org/pdf/2610.15709
  - `sec/2_bkg.tex`, Eqs. `acc_dev`, `dev_split`: $E_T=\sum_{\tau\le T}A_{\tau:T}R_\tau$ and, for scalar decay, $\|E_T\|_F^2=\sum_\tau a_{\tau:T}^2\|R_\tau\|_F^2+\sum_{\tau\ne\tau'}a_{\tau:T}a_{\tau':T}\langle R_\tau,R_{\tau'}\rangle_F$. small lost updates drive coherent error; this frozen-update decomposition is diagnostic, not a full closed-loop GDN stability proof.
  - `sec/3_lrb.tex`, Eqs. `lrb`, `merge`, `bdr`: bypass $L_t=A_tL_{t-1}+k_t\bar v_t^T$; retain $L_t'=\mathcal T_k(L_t\odot\mathbf1[|L_t|<\epsilon_L\Delta_t])$ at merge and quantize the residual. BDR adds $\operatorname{sign}(d)\Delta_t$ only if effective update $\tilde d=Q(s+d)-s$ reverses $d$ and $|d|>\epsilon_B\Delta_t$, with an in-range-grid guard; overshoot is bounded by $|d|/\epsilon_B$.
  - GDN configuration: rank `1`, merge interval `15`, $\epsilon_L=0.50$, $\epsilon_B=0.30$, `5.25` average / `6.00` peak bits including bypass. WikiText-103 8K PPL `24.19` FP32 / `79.76` INT4 / `26.85` LRB+BDR. for Swift: investigate only if state traffic becomes material; published 1.3B, unquantized-weight, RTX 3060 batch `16/32/48` evidence does not validate this hybrid GSQ-RCO model. it complements the scale-feedback warning; it is not evidence for naive INT4 or stochastic rounding.

### hardware
- RX 7600 XT: https://www.techpowerup.com/gpu-specs/radeon-rx-7600-xt.c4190
- Apple M5: https://support.apple.com/en-us/125405
- Air vs Pro sustained power: Notebookcheck M5 Air reviews [web]
- Colab A100 80 GB RAM/disk: [web], verify on first launch with `free -g; df -h`

### local artifacts
- `scripts/mine_llm_isms_from_transcripts.py`: slop miner (all §9 + `audits/opus_audit_slop_miner_and_1c_v3.md` fixes applied; `--self-test` PASS; output in `results/banned_llm_isms.json`)
- `scripts/run_colab_a100_agent_persona.py`: Phase 6 Colab A100 48-layer exact GGUF-backed 96-writer LoRA SFT + FTPO/DPO trainer & GGUF tensor transplanter
- `audits/r40_rocm14_metal_optimizations.md` & `scripts/strata_rocm14_metal_opts.patch`: Round 40–43 Metal kernel & engine optimizations