# 00 — overview & architecture decisions: swift 1.5 / qwen3.8-flash-next persona
each number across `00-`..`04-` is tagged:
- **[src]**: read from a primary source (model card, config, paper, repo doc), linked in `04-sources-and-references.md`
- **[web]**: from a web-search summary and not checked against a primary page. treat it as a lead.
- **[est]**: our own arithmetic, assumptions stated next to it
docs index:
- `00-overview-and-decisions.md` — goal & core architecture decisions (§1, §2.1–2.6, §2.8–2.9)
- `01-wrong-turns-and-traps.md` — mistakes caught, wrong turns, and ruled-out ideas (§2.7.A #1–#35, §2.7.B)
- `02-hardware-and-streaming-model.md` — sizes, streaming speed evidence, latency model & measurement plan (§3–§6)
- `03-phases-and-results.md` — phases, measured outcomes (phases 0–3, phase 6 a100 lora, round 40–43 metal opts), risks & open questions (§7–§10)
- `04-sources-and-references.md` — models, runtimes, papers, hardware & local artifacts (§11)
---
## 1. goal
a **persistent agent** with dawn's persona (lowercase, kaomoji, not officecore): long-lived, always-on, with memory across sessions. it's **not a coding agent**. its only "coding" is tool-adjacent: a python tool the way albedo uses it, and linux ops (logs, services, files, cron, packages). it should run usably on:
| machine | gpu / chip | memory | notes |
|---|---|---|---|
| desktop | AMD RX 7600 XT, 16 GB VRAM, gfx1102, 288 GB/s, PCIe 4.0 x8 [src] | **16 GB system RAM** | headroom target from blueprint: ~5 GB VRAM + ~6.5 GB RAM left free |
| laptop | **13" MacBook Air, M5 (`Mac17,3`), 10-core CPU (4S+6E), 10-core GPU, Metal 4** (fanless) [src: local `system_profiler`] | 24 GB unified, 153 GB/s, **1 TB SSD (~303 GB free)** [src: local] | headroom target: ~9 GB left free |
| measurement | Google Colab A100 **80 GB** | ~167 GB RAM, ~236 GB disk [web] | ephemeral runtime |
| training | Colab, or rent a GPU box when Colab is annoying | | |
"usable" means:
- **prefill ≥ ~200 tok/s on cold prompts** is the stated bar ("somewhat usable", 450 not required)
- **seconds per agent turn** is the real UX number (see §5.1)
---
## 2. decisions so far
### 2.1 base model
- **ukisai Swift 1.5** (Qwen3.8-Flash-Next fine-tune, `qwen4_exp` architecture), as **ukisai's GSQ-RCO GGUFs**. no re-quantization pipeline of our own.
- why Swift:
- median thinking tokens −63.4% on their coding evals (mean 17,683 → 7,823) [src: swift card]
- its whole post-training changed only the dense residual writers. relative change vs base [src: SC117 card]:
- `ssm_out` ~0.30%
- `attn_output` ~0.25%
- `ffn_down_shexp` ~1.6%
- **`ffn_down_exps` byte-identical to the Qwen base on all 48 layers**
- known Swift costs [src: swift card]:
- IFBench 73.2 → 70.1 (watch this, since a persona relies on instruction following)
- medium/low effort settings lose ~2.5 points
- architecture facts we rely on [src: `config.json`, tech report 2608.30320]:
- hidden 2560, vocab 248,320, 48 layers
- layer pattern: 36 Gated DeltaNet + 12 attention layers, every 4th layer full attention, replaced by **QSA** (sparse attention) after continued pretraining
- QSA indexer: 4 heads, head dim 128, 1 KV head, compress ratio 4, budget 2048 tokens / 512 micro-blocks
- attention KV: 2,048 B per token per attention layer (24 KiB/token over 12 layers)
- MoE: 512 routed experts per layer, **top-10** + 1 shared, expert intermediate 640, so one expert = 3 × 2560 × 640 = 4.915M params
- **n-gram / PLE table**: 51.2B params, 16 heads × 160 dims, ~20M rows per head, hashed bigram/trigram, injected at layer 2. 28.8 GB at IQ4_NL; can stay on SSD (~16 row reads per token)
- **Gated Residual (GR)**: 4 residual branches read through an elementwise gate
- MTP head (~4B) for speculative decoding
- template **preserves thinking by default**
- BF16 checkpoint is ~360 GB [src: HF listing]. we never need it whole.
### 2.2 persona = dense-only tune + tensor transplant
- train **only trunk (`blk.0..47`) dense/router/shared params** (`scripts/phase3_lora_and_transplant.py` supports two explicit whitelist scopes):
- `swift_writers_plus_router` (default minimal-drift scope): the 3 residual-writer families (`12` `attn_output.weight` on QSA layers + `36` `ssm_out.weight` on GDN layers + `48` `ffn_down_shexp.weight` = `96` trunk writer tensors; note SC117's 144 count also included 48 `attn_gate`/`ssm_alpha`) + MoE router gates (`48` `ffn_gate_inp.weight` + `48` `ffn_gate_inp_shexp.weight` = `96` router tensors -> `192` trunk tensors total), keeping hyper-connection stream mixers (`hc_*`), PLE projections, and the MTP head (`blk.48.*`) untouched so MTP acceptance doesn't degrade.
- `all_trunk_dense_router_shared`: all trunk `0..47` non-expert/non-PLE-table tensors (`attn_{q,k,v,qkv,gate,out,q_norm,k_norm}`, `indexer.{q_proj,k_proj,q_norm,k_norm}`, `ssm_{conv1d,dt,beta,alpha,norm,out}`, `ffn_{gate,up,down}_shexp`, `ffn_gate_inp*`, `ple_{key,value,norm_key,norm_query,norm_conv,conv1d}`, `hc_{attn,ffn}_{norm,down,up,inject}`, `output_hc_{norm,down,up}`).
- **routed experts (`blk.*.ffn_{gate,up,down,gate_up}_exps.*`), PLE n-gram table, embeddings, `ssm_a`, and `blk.48.*` stay frozen.** precedent: Swift did exactly this.
- export:
1. merge LoRA into the dense tensors
2. quantize **only those tensors** (GSQ/requant at the exact `ggml_type` the Swift GGUF already uses)
3. **whitelist-only byte-level transplant** (`transplant_gguf_tensors`, requiring mandatory `(ggml_type, raw_bytes)` tuples and verifying both `ggml_type` and byte size + directory `fsync`) into the Swift GSQ-RCO GGUF, the way SC117 transplanted 144 tensors and left every GSQ value/scale bit-identical [src: SC117 card]
- persona data:
- persistent-agent traces: multi-turn conversations, python/shell tool loops, ops tasks, and memory recall after compaction. for voice/style references from albedo transcripts, filter **specifically to `gemini 3.8 flash` and `gemini 4 argon` turns** (which are close to the target voice) plus `APPEND_SYSTEM.md` — do not scrape blanket transcripts from other models.
- store all untuned pruned checkpoints + saliency/routing traces on HF (`90-008/...`) so the phase 3 dense/router tune can be re-run or retuned anytime without redoing phase 1–2.
- `<think>` and tool calls kept intact
- multi-turn traces with **preserved reasoning**
- **voice-override examples:** context or memory demands a format or register (JSON/tool args, a file in a given style, a message to someone else). the target follows it, with voice only where it fits. this guards against a persona tune that ranks voice above instructions.
- rationale for baking the voice in: it takes persona instructions out of the prompt (shorter prompt, cheaper prefill), avoids system-prompt drift (instruction drift within ~8 rounds [src: 2402.10962]), and leaves fewer simultaneous instructions (adherence drops as instruction density rises [src: 2507.11538, IFScale]).
- **correctness-filtered**. lesson from Gadsby-4B [src]: a blanket bigram ban gave great voice and wrong answers ("17×23 = sixteen forty nine"). the persona must come from the data, not from a mask.
- open: persona inside `<think>` too, or only in visible replies? (§10)
### 2.3 expert pruning: yes
- candidates: **224 / 256 / 320 experts per layer**. choose by behavioral evals, not by router statistics.
- AnonimousA found that "retained routing mass" predicted the *opposite* of their behavioral probes when ranking two prunes [src: REAP-320 card]
- method: per-layer budgets (GGUF stores one expert count per model, so every layer keeps the same K [src: ISTA Coder card]).
- selection either by **RCO** (Riemannian Constrained Optimization, exact budgets, optimizes full-model KL) [src: 2605.00649, IST-DASLab/RCO]
- or REAP-style saliency from our own routing traces (§6.1)
- **calibration mix is the product** [src: REAP-320 card (525K tokens), REAP-256-duo card (344K tokens), ISTA Coder card]. target **~400K–525K tokens** assembled from real open HF datasets (never a small hand-written file, and never personal transcripts):
- **terminal / linux ops / multi-turn tool use (~30%):**
- `ukisai/Qwen3.8-27B-multi-turn-agent-sft` [src: HF, Apache-2.0, 15.2K Terminus-2 traces incl. `nl2bash` — Ukisai's own Swift dataset]
- `Lottolabs/terminal-bench-2.1-qwen3.8-27b-traces` [src: HF, MIT, full prompts + trajectories for all 89 Terminal-Bench 2.1 ops tasks]
- **reasoning + open chat + strict JSON extraction (voice-override) + sandbox-verified tool use (~25%):**
- `logic65/whittle-distill-qwen3.8-27b` [src: HF, 25.1K complete Qwen3.8 traces with `<think>` + top-20 logprobs]
- **Japanese (JA) chat, reasoning, roleplay & tool use (~25%):**
- `ChiTako/japanese-imatrix-calibration` [src: HF, MIT, ~200K tokens built for `llama-imatrix`: `chat_ja`, `sft_ja`, `cot_ja`, `character_roleplay`, `character_behavior`, `tool_agent`, `wikipedia_ja`, `fineweb_ja`]
- `hotchpotch/japanese-qa-reasoning-100k` [src: HF, ODC-By, 105K Japanese `<think>` + QA examples from `fineweb-2-edu-japanese`]
- *(no hand-written synthetic filler in phase 1 calibration; `ChiTako` + `logic65` + `ukisai` + `Lottolabs` + `hotchpotch` already cover chat, roleplay, strict JSON/format adherence, JA, reasoning, and terminal/tool loops)*
- **images** (added before final export; ISTA's first code-only search killed vision, adding image data fixed it)
- SWE-style repo coding is expendable, which helps the pruning budget
- ISTA's first code-only search killed vision. adding image data fixed it.
- ISTA Coder broke on Chinese (Strata #438) because its calibration had no CJK. that only shows calibration decides what survives; it says nothing against a JA-calibrated prune.
- REAP-320: a multi-domain prune beat a code-only prune **even at code** (HumanEval 96.3 vs 95.7), reproduced at K=256.
- known risk: the community "reliability knee" is at ~K=288–320 (rare tokens, clean termination) [src: REAP-320 card]. it was measured with calibrations that had no JA. 224 is below it, so it has to be proven by eval.
- mechanics: **slice the GGUF directly**.
- drop expert rows from each layer's `ffn_{gate,up,down}_exps` 3D tensors and the matching router rows
- update `expert_count`
- remaining experts stay bit-identical. no requant, no BF16 needed.
- top-k 10 → 8 (blueprint idea): test at runtime with an `expert_used_count` override first. don't bake it in.
### 2.4 healing after pruning: abstention-aware, combined with the persona run
- pruned facts don't come back. world-knowledge loss is **accepted** (dawn), mitigated by:
- **abstention training** in the healing data (`scripts/prepare_phase3_sft_data.py` `classify_rtuning_abstention`):
1. run the pruned model on factual probes, sampled several times, and match answers using **word-boundary any-of alias groups** (`(?<![a-z0-9_])...(?![a-z0-9_])`, so `"42"` never matches `"1042"`)
2. where it still matches the teacher (`>= 75%` agreement), target = the speech-act-cleaned teacher answer
3. where it's wrong or unstable, target = a diverse pool of persona-voice abstention templates (`ABSTENTION_TEMPLATES`: `"i don't know / i'm not sure / i haven't checked"` in dawn's voice, never officecore `"let me check"`) or a **silent** `web_search` tool call (`target_response = ""` + `target_tool_call`, no `"pulling it up first"` pre-tool narration)
- basis: R-Tuning [src: 2311.09677]. Kalai et al. 2025 [src: 2509.04664]: training rewards guessing over saying idk.
- **don't** naively distill the teacher on facts the student lost. that teaches confident fabrication [src: Gekhman et al. 2405.05904; Kang et al. 2403.05612].
- the agent has a search tool, so knowing when it doesn't know matters more than recall.
- **teacher = unpruned Swift GSQ-RCO IQ3_S GGUF.**
- resident shard 41.77 GB [src: SC117 card size table, same tensor layout] fits in the A100's 80 GB, with the PLE shard on disk
- ISTA reports IQ3_S matches/exceeds base on their tasks [src: ISTA card]
- this replaces the blueprint's "KD against BF16 Swift", which has no feasible teacher on one GPU
- teacher outputs: **top-k logits generated offline teacher-forced on the post-surgery target text** with llama.cpp on the A100 (surgery MUST run before teacher-forced logit extraction so token positions align 1-to-1), plus the known/unknown labels.
- trainable params: router + dense/shared (same set as §2.2), so **healing and persona are one training run**.
- eval both directions: "made something up" **and** "said idk when it knew" (abstention tuning tends to over-refuse).
### 2.5 runtime: start from Strata, squeeze everything that measures positive
squeezing rule (dawn: "we should be doing anything we can"; agreed with one condition): **every technique goes in if it measures positive after its cost**, however small. no technique goes in on faith. this is ds4-recipe's discipline [src: ds4 README/01]: exactness tags, measured gain, "tried without paying" sections.
base runtime:
- **Strata** (Niko1221) [src]:
- CUDA/HIP/Metal
- VRAM/RAM/SSD expert tiering
- positioned reads for the PLE table
- 8-bit KV above 4K
- MTP
- gfx1102 accepted: one RX 7600 XT with 64 GB RAM ran Swift IQ3_XXS at ~43 tok/s decode, 250–300 tok/s prefill (#942)
- Strata-Metal streams experts from SSD on 32–48 GB Macs
- mainline llama.cpp as fallback/oracle. state [src: GitHub API, checked 2026-10-14]:
- qwen4exp model support merged (#27742)
- expert LRU cache still an open PR (#27861)
- lazy-PLE direct reads closed unmerged (#28136)
- **MTP graph closed unmerged (#28243)**, so MTP needs Strata or unsloth's fork
- hybrid context checkpoints exist (`--ctx-checkpoints`, `--cache-ram`, `--spec-use-checkpoints`, fixed in #22929)
techniques, in expected order of payoff:
| # | technique | status | evidence |
|---|---|---|---|
| 1 | **pruning** (§2.3) | do | shrinks the non-resident bytes that set the per-turn floor (§5.1) |
| 2 | **MoCE cache-aware routing**: `z' = z + λ·Δavg·mask`, mask = resident experts ∪ protected original top-J, training-free | do, λ sweep | [src: MoCE 2412.00099] LRU expert lifetime ~22 tokens → 5–9× longer. learned prior did **not** beat training-free. top-1 pick critical; fine-grained MoEs tolerate swaps from rank 3 down. [src: ds4 06] λ 0.1 / top-2 protected: misses −52…−73%, decode +12–19%; warm 150-token turn 7.1 → 4.5 s, 1,081-token turn 14.3 → 10.0 s. production λ 0.09 / top-3 |
| 3 | **disk KV cache incl. GDN state** (~110 MiB per checkpoint [est]) | do | ds4: 98.75–99.5% of prompt tokens served from KV in production [src: ds4 README] |
| 4 | **SCR** (suffix cache reuse) for context edits/compaction, done the CLM way: relocate KV + re-rotate RoPE in attention layers, **keep the post-edit GDN state** | do, measure staleness | [src: CLM 2609.37725 App. B] (K=6 spans). [src: ds4 13] a 459K-relocated / 559-fresh turn: 28.6 s vs ~1,111 s cold. staleness +0.015…0.040 nats/token on a model with **no** recurrent layers; with 36 GDN layers it's unmeasured |
| 5 | **bigger prefill chunks** (4096–8192 instead of Strata's 1024 on small Macs) | measure | [src: Strata MAC.md] streamed UD-Q5 on M5 Max: 700 → 920 → 1,100 tok/s at 1024/2048/4096. [src: ds4 07] 32K vs 8K chunk +7.5–25%. cost: buffer memory comes out of the expert cache |
| 6 | **eviction: popularity EMA + near-pick credit** (credit the router's unbiased top-N, not just the chosen 10). dawn's **inverse-router eviction** idea is the same signal from the other side; test it as a variant | do, measure | [src: ds4 06] 12 production tapes: best online score −4.0% misses vs production; pure LRU/LFU +5…+46% worse; Belady optimum −56.3%; perfect 8-token foresight only −9.6%. expect a few % |
| 7 | **non-evicting staged prefetch**: run layer L+1's router on layer L's hidden state (on Flash-Next: apply L+1's GR read gate first), stage confident (margin ≥ ~0.15) misses into a separate pinned shelf, adopt only on a real miss | do, measure | [src: ds4 06 technique 10] 66% recall at 1.27× reads. **evicting prefetch lost 8.6% live**. same trick as Mixtral-offloading (Eliseev & Mazur 2023). on the Air, one SSD serves demand and speculative reads, so contention is worse |
| 8 | experts' copy (each expert contiguous, 1 read instead of 3) + **2–4 thread `pread` fan-out** | do | [src: local `bench/README.md`, `iostat -Id` verified] on 13" M5 Air 1 TB SSD: 1-thread std 512 KiB = **2.41 GB/s** → 1-thread contiguous 1.5 MiB = **3.93 GB/s** (+63%) → 2–4 threads = **6.60–7.03 GB/s** (saturates PCIe Gen4 x4). also batch layer-2 PLE 4 KiB reads across 8 threads (**0.04 → 0.33 GB/s**, 1.6 ms/tok → 0.2 ms/tok) |
| 9 | kernel work (GEMM plans, MoE kernels) | later | [src: ds4 07] hipBLASLt prepared GEMMs were the biggest prefill lever (+43–45%). raises the big-cold-prompt ceiling, barely moves the per-turn floor |
| 10 | **prefix sliding** as an **untrained runtime toggle, default off**: restrict the QSA indexer to prompt + last W reasoning tokens | low priority, A/B | inside `<think>` it saves ~0.3% of decode bytes at Swift's ~7.8K-token mean thinking [est]. whole-context ≤7–10% at 128K but cuts long-range retrieval. paper: tested ≤7B, full attention, reasoning traces; agentic tool output is its named failure case; LiveCodeBench needs W ≥ 16K [src: 2608.26070]. **not baked into training** |
| 11 | MTP | measure | [src: Strata MAC.md] 1.3–1.4× when resident; **slower when streaming** on the 32 GB M4 (its 1.2 GB do more as expert cache) |
| 12 | top-k 10 → 8 override | measure (quality + speed) | metadata override only |
caveats carried with these:
- with MoCE (λ > 0), **output depends on cache state**, so it's nondeterministic across sessions. ds4 had to put a routing-policy key on saved KV checkpoints [src: ds4 06]. MoCE also stacks with pruning (both shrink the effective pool), so the right λ is probably lower on a pruned model.
- any prefetch that changes residency changes outputs under MoCE [src: ds4 06].
- persistent-agent workload: many small turns (user messages, tool outputs), regular compaction, long idle gaps, rare cold starts. that raises the value of the disk KV cache, SCR (compaction is exactly the edit SCR handles; the CLM paper is about context-managing agents) and MoCE in small-turn prefill. it lowers prefix sliding further, since recall of earlier conversation is the job.
- keep `preserve_thinking` on. Whittle measured post-success rewrite loops when reasoning was not sent back [src: Whittle card].
### 2.6 anti-slop: sentence-level speech-act surgery in Phase 3 + lexical FTPO in Phase 4
- **two distinct mechanisms for two distinct kinds of LLM habits (dawn's catch, §2.7.A #27):**
1. **structural speech-act habits (`phase3_sentence_surgery`, NEVER single-token FTPO):**
- *what they are:* pre-tool announcements (`"let me check..."`, `"i'll look at..."`, `"now let me..."`), servile end-of-turn permission-solicitations (`"do you want me to..."`, `"want me to..."`, `"if you want, i can..."`), and sycophantic openers (`"you're right"`, `"good catch"`).
- *why token-level FTPO fails on them:* if the model's hidden state already intends to emit a pre-tool announcement or ask a servile closing question, penalizing the final token of `"do you want"` or `"let me check"` just causes **whack-a-mole synonym substitution** (`"would you prefer"`, `"shall i proceed"`, `"allow me to inspect"`) while damaging normal uses of `"want"` and `"let"`.
- *how we fix them:* **sentence-level surgery on Phase 3 SFT targets** (`scripts/prepare_phase3_sft_data.py`) — strip pre-tool announcement sentences before `<tool_call>` (so the model just calls the tool), strip servile closing solicitation sentences at the end of turns (unless a genuine architectural fork exists), and strip sycophantic openers so turns start directly with the thought.
2. **surface lexical / rhetorical filler cliches (`phase4_lexical_ftpo`, handled by Antislop / FTPO):**
- *what they are:* phrase-level rhetorical filler (`"rather than"`, `"right away"`, `"end to end"`, `"exactly what"`, `"here's what"`, `"what's actually"`, `"short version"`, `"two things"`, `"instead of guessing"`, `"one catch"` — explicitly excluding epistemic hedging like `"think that's"` and honest progress qualifiers like `"so far"` / `"as expected"`).
- *how we fix them:* **Antislop / FTPO** (final-token preference optimization) [src: 2510.15061, github.com/sam-paech/auto-antislop] on the `ftpo_qwen_token_id_tuples` subset of `results/banned_llm_isms.json` (`7` cross-family phrases -> `126` Qwen token tuples) + phrases mined from the tuned model's own outputs.
- no runtime n-gram mask. backtracking on a hybrid means restoring ~110 MiB recurrent snapshots [est], so rollback isn't free the way KV truncation is.
- `scripts/mine_llm_isms_from_transcripts.py` serves as a **statistical confirmation & cross-family filter over curated rhetorical/signpost candidate lists** (`RHETORICAL_ALLOWLIST` + `SIGNPOST_PREFIXES` minus `EXCLUDED_WORK_AND_GENERIC_WORDS`), requiring `>= 2` model families each with `>= max(5, 5% of c_llm)` hits and partitioning into `phase3_sentence_surgery` (`25` phrases, verified by coverage test in `scripts/prepare_phase3_sft_data.py`) vs `phase4_lexical_ftpo` (`7` phrases, `126` tuples in `results/banned_llm_isms.json`).
### 2.8 optional side quest
- J-lens [src: anthropics/jacobian-lens]:
1. fit a corpus-averaged Jacobian lens on a smaller supported model first
2. expose top-k readouts at a few layers as a tool result / appended message
- never in the prefix. on Flash-Next it needs a custom wrapper for the 4-branch GR residual.
### 2.9 harness fit: Connectome (anima-research/connectome-host)
harness undecided (dawn: "it depends", her harness might implement Connectome-style folds). so the runtime stays harness-agnostic: SCR handles any mid-context span replacement (Connectome folds, LCM-style summaries, plain compaction), and Connectome and LCM have the same shape (old spans fold into summaries, summaries condense upward, a recent tail stays verbatim), so both are the same SCR case. what changes the local KV cost is config: tail size (how far folds land from the live edge), layout stability (how often already-presented summaries get rewritten; Connectome's `kv-stable` minimizes this), and fold frequency. those are the eval parameters, and in any harness: long tail + stable layout = cheap. training data mixes memory formats instead of overfitting one harness's prompts. Connectome is not the optimization target, but it's the closest thing to what this agent is, so the runtime and training data should fit it. read 2026-10-14 [src: connectome-host README, docs/AGENT-MEMORY-GUIDE.md, docs/AGENT-ONBOARDING.md, animalabs.ai/field-notes/kv_perturbation_thread, Phylliida/connectome-min].
**what it does with context:**
- built for long-lived "residents" with autobiographical memory
- context = **head** (pinned verbatim, default 4K) + **middle** (raw, or folded into L1…L8 recollections) + **tail** (verbatim, default 30K; production 300–450K)
- folding happens only when the budget forces it, deep behind the live edge, chosen by a solver that disturbs as little computed context as possible:
- `kv-stable` (default): keeps the layout stable between compiles
- `kv-unified`: a cost-aware solver priced in cache read/write $/Mtok
- **memories are written by the agent's own model, first person, in its own voice**, from an *as-of* reconstruction:
1. head
2. earlier memories replayed as the agent's own `[Recall L1-3]` answers
3. an in-band "you will soon form a new memory" marker
4. the raw chunk with tool calls
5. "preserve concrete details: paths, values, decisions, open asks"
- thinking blocks are hidden from the compression pass
- merges (6 → 1 level up) re-read one level deeper, so they're not summaries of summaries
- chronicle keeps everything; `history--search` tools are optional
- big tool results (>24K chars) go to workspace files with an inline preview; only the 6 most recent images stay live, within 30K tokens
- talks to models through membrane (Anthropic format; `ANTHROPIC_BASE_URL` gateway). Strata serves `/v1/messages`, so it can probably point straight at Strata (untested).
**their KV finding (Qwen2.5 7B/14B, mlegls' experiments):**
- rolling the window is mostly a RoPE relabel. un-rotated keys are nearly identical, attention maps are unchanged, and the disturbance fades within ~10 tokens of the new front.
- repligate proposed suffix caching from this.
- **the catch they hit:** trimmed cached values with re-rotated keys → output collapses into loops ("worked worked worked"). recomputing values while keeping the shifted keys stays coherent.
- our reading (hypothesis): it matters which tokens get recomputed. CLM recomputes the last 16 tokens of each span; ds4 recomputes junction rows; Connectome pins the head, so attention sinks survive. SCR must recompute junctions and never drop the head/sink. the SCR staleness test (§6.1.7) should include their looping failure as a check.
- their framing agrees with keeping stale GDN state: reused state is "more faithful to the model's computational past" than a re-prefill.
**what it means for us:**
1. **a fold is exactly SCR's edit** `[A B C] → [A B' C]`: middle span → recollection, tail relocated. on Flash-Next:
- attention layers: relocate + re-rotate
- GDN layers: keep the post-edit state, which still "remembers" the raw stretch
- long-run drift is uncharted (months of folds, GDN never re-grounded). measure KL vs fresh prefill after N folds; decide a re-ground policy (e.g. on restart, or when drift crosses a threshold).
2. **context sizes must shrink locally.** production budgets are 300–600K. Flash-Next is native 262K; 262K of q8 KV ≈ 3.2 GiB, which comes straight out of the laptop's expert cache. start around budget ~100–128K / tail ~64–100K, and tune with the latency model. bigger tail = fewer folds = fewer expensive events.
3. **`kv-unified` can be given local prices**: cache write = our prefill cost per token, read ≈ 0. that makes its solver optimize for the laptop. caveat: it assumes prefix-cache semantics, while with SCR a fold costs only the new recollection plus junctions. may need a small solver change.
4. **compression passes are separate inferences** with a different context from the live one, so the live prefix cache doesn't cover them. but consecutive passes share head + earlier memories as a prefix (verify order in context-manager). so keep a dedicated **compression session** in the disk KV cache: each L1 pass then prefills only the new memory + marker + chunk (~3–8K tokens). merges are rarer and bigger. memories are written ahead of need (`maxSpeculativeL1s`), so schedule passes in idle gaps (good for Air thermals). the raw chunk already exists in the live cache, so it's a candidate for multi-source SCR (ds4 pulls rows from up to 4 sources).
5. **training data** should contain Connectome-shaped turns:
- memory formation in voice (first person, as-of, concrete details kept)
- `[Recall …]` exchanges
- merges
- prose-routing conventions (`skip_reply`, `>>#channel`)
- MCPL tool use and heartbeat wakes
- the habit of **writing exact details to workspace files**, since memories are lossy by design
6. **eval:** run the persona inside Connectome (or connectome-min, a faithful 4-file model with 371 tests) with a deliberately small budget so folds happen often. test recall at L1/L2 depth, persona consistency across folds, and SCR on vs off.
---