Round 37: multi-chunk prompt hot-expert LRU & scratch pool preservation (stream.cpp, window.cpp) -> 34.01-34.74 tok/s @10.2K decode (+34.6-37.5%) & 458-473 tok/s @10.2K prefill, 42.49 tok/s @2K decode

c005fa30 parent f73b01b4 +61 −41
browse files