# round 20 track b — gather and plan

## decision

per the parent’s follow-up, the final proposed patch includes the verified `mm_hist` / `mm_place` changes, width2560 float/half gather unrolling, and the short parameterized evaluator. `window.cpp` remains unchanged at tg256; `mm_scan` and `meta[8]` remain unchanged. the parent identified the late timing spikes as concurrent track a/c parity runs.

performance caveat: the copy did not meet this experiment’s original repeatable >5% threshold. tg256 remained as fast as or faster than 32/64/160 in the relatively stable sweeps. plan improvements were 3.5–7.8% in selected quiet measurements (including 12.224 -> 11.276 us at t2048), but other repetitions regressed. this is a correctness-verified candidate with small/noisy performance changes, not a robust large speedup claim.

- proposed patch: `/tmp/sol_r20_gather/sol_r20_gather.patch`
- kernel-only experimental patch (now also included in the main patch): `/tmp/sol_r20_gather/experimental-not-recommended.patch`
- isolated source/build: `/tmp/sol_r20_gather/strata/`
- measured candidate: `/tmp/sol_r20_gather/variants/all-fastpaths.metal`
- baseline snapshot: `/tmp/sol_r20_gather/baseline/`

both patches passed `git -C vendor/strata apply --check`. no tracked vendor files were edited. no :8080 requests, full-model runs, or child agents.

## contract and setup

owner: the existing plan kernels own deterministic sorting/offsets; gather owns activation conversion/copy; the engine only selects threadgroup size. preserve `meta[8] = all_rows`, padding, buffer layouts, entry order, and generic-width/expert-count fallbacks. plan: `/tmp/sol_r20_gather/plan.md`.

apple m5. baseline was the dirty vendor working-tree snapshot, not pristine upstream HEAD; vendor HEAD was `296b59d2f73c8f04a805df29c2fa97655adace78`. source hashes and artifacts are in `/tmp/sol_r20_gather/manifest.json`. the live server was left alone. other concurrent GPU activity was not controlled; later measurements became severely unstable. the parent subsequently confirmed track a/c standalone benchmarks overlapped those runs, so those samples cannot support a speedup claim.

candidate tested all requested specializations together:
- histogram single conditional initialization/readback for <=1024 experts;
- place two explicit initialization stores at 512 experts, full unroll of the 32-lane broadcast scan;
- float gather ten unrolled iterations at width2560;
- half gather unrolled 10/5/2 stores at tg32/64/160, explicit 256+64 copy at tg256.

threadgroup sweeps ran through the same `mtl::Dispatch` interface in the standalone evaluator; the engine's tg256 was not changed merely to benchmark it. `mm_scan` and indirect row count were unchanged.

## measurements

three paired samples per cell, eight dispatch repetitions per sample, median GPU us/layer; 20ms warmup per gather pair in confirmation sweeps. baseline/candidate gather order alternates. seed7, topk10, 512 experts, tile16. measured snapshots were left untouched during runs. logs retain every attempted run, including contaminated runs.

a relatively stable repeated sweep (`sweep3-T*.log`):

| tokens | padded rows | baseline plan | candidate plan | baseline gather tg256 | candidate gather tg256 |
|---:|---:|---:|---:|---:|---:|
|256|8096|7.625|8.016|254.609|253.953|
|512|8432|8.188|8.021|270.062|269.766|
|1024|14352|8.953|8.297|489.187|490.578|
|2048|24576|11.422|12.057|1161.984|1159.594|

t2048 candidate tg32/64/160/256 in that sweep: 1185.151 / 1166.484 / 1163.359 / 1159.594 us. output-only effective bandwidth at tg256 is about 108.5 gb/s (125,829,120 output bytes); this is not a measurement of physical dram read+write traffic. apparent unused thread slots do not imply the copy is execution-bound.

`sweep2` and `sweep4` also failed the predeclared repeatable >5% copy win. plan deltas changed sign. final combined-path runs (`final-candidate-T*.log`) all passed parity, but timings became heavily contaminated (baseline tg256 t2048 1900us, and a later identical-source comparison reached 6408us). do not cherry-pick these as wins/losses. final combined candidate plan+gather was 889.875 / 360.042 / 661.167 / 1739.927 us for t256/512/1024/2048, respectively; these are explicitly **not comparable clean performance estimates**.

## correctness and build evidence

built `moe_mm_parity` and `strata_window_cli` successfully in scratch `build-metal`, release/ninja; `final-build.log` records the final successful build. cli was built, never executed. each standalone invocation had a five-second deadline; successful invocations were about 0.01–1.39 seconds.

candidate and baseline were compiled in the same process. byte comparisons passed for all allocated bytes of `bcnt`, `boff`, `eoff`, `tiles`, `meta`, `srt_tok`, `srt_w`, and `pos`. `XS` passed full bitwise `memcmp` against baseline float gather for all active/padded rows at each threadgroup, direct and indirect dispatch, and after repeated combined plan+gather. candidate float gather also matched baseline float gather. existing token/position uniqueness/padding checks passed.

coverage:
- t256/512/1024/2048, experts512, width2560;
- t1/experts64, t7/experts512, t64/experts1024, t256/experts64;
- generic width768, t7/experts64.

these kernels do not accept quantized weights or a quantization-format parameter, so this coverage is format-independent. no claim is made that all quantized gemm formats were separately executed.

negative control: a scratch baseline deliberately writing `srt_w[row] = w[i] + 1` produced `srt_w memcmp failed`, exit1 (`negative-comparator.log`). invalid t0 exited2 (`invalid.log`). final production-baseline self-comparison passed (`final-baseline.log`).

## reproduce

```text
cmake -S /tmp/sol_r20_gather/strata/metal -B /tmp/sol_r20_gather/strata/build-metal -G Ninja -DCMAKE_BUILD_TYPE=Release -DMETAL_CPP_DIR=<workspace>/vendor/strata/third_party/metal-cpp -DSTRATA_GGML_DIR=<workspace>/vendor/llama.cpp
cmake --build /tmp/sol_r20_gather/strata/build-metal --target moe_mm_parity strata_window_cli -j 4
MM_BASELINE_KERNEL=/tmp/sol_r20_gather/baseline/metal/kernels/moe_mm.metal STRATA_METAL_KERNELS=/tmp/sol_r20_gather/variants /tmp/sol_r20_gather/strata/build-metal/moe_mm_parity --gather-bench 2048 512 2560
```

repeat one t per invocation for 256/512/1024/2048. omit `STRATA_METAL_KERNELS` to test the final scratch candidate (identical to the variants snapshot). to test unchanged baseline, set `STRATA_METAL_KERNELS=/tmp/sol_r20_gather/baseline/metal/kernels`. final review used `view_diff` and contextual `view_code`; the main patch now includes the kernel candidate at the parent’s explicit request, with the measurement caveats above.