{"ref":"main","files":[{"name":".devops","kind":"directory","size":0,"lastCommitHash":"9d77fa17254e1dee4b9e92504c91611a60b1359f","lastCommitWhen":"2026-08-18T12:02:22+02:00","lastCommitMessage":"ci : Update OpenVINO to 2026.3, skip nemotron-h rollback test (#27292)\n\n* update to ov-2026.3, update device drivers\n\n* ci: skip nemotron-h rollback test on OpenVINO\n\nThe OpenVINO backend does not support SSM_SCAN, so the Nemotron-H recurrent state rollback graph is split and cannot preserve the recurrent cache output shape. Keep the test enabled for other backends and retain the qwen35 OpenVINO rollback coverage.\n\n---------\n\nCo-authored-by: ravi9 "},{"name":".gemini","kind":"directory","size":0,"lastCommitHash":"3595ae5963f1583f53beecf9725c919d309e15da","lastCommitWhen":"2025-12-29T16:01:32+01:00","lastCommitMessage":"contributing: tighten AI usage policy (#18388)\n\n* contributing: tighten AI usage policy\n\n* refactor AGENTS.md\n\n* proofreading\n\n* update contributing\n\n* add claude.md\n\n* add trailing newline\n\n* add note about dishonest practices\n\n* rm point about dishonest\n\n* rm requirement watermarking\n\n* add .gemini/settings.json\n\n* allow initially AI-generated content\n\n* revise\n\n* Update CONTRIBUTING.md\n\nCo-authored-by: Johannes Gäßler \n\n* improve\n\n* trailing space\n\n* Apply suggestions from code review\n\nCo-authored-by: Johannes Gäßler \n\n* update\n\n---------\n\nCo-authored-by: Johannes Gäßler "},{"name":".github","kind":"directory","size":0,"lastCommitHash":"9a4dd89d0a6b609e21cd6f43cc4c057f05b356cc","lastCommitWhen":"2026-09-25T10:39:48-07:00","lastCommitMessage":"release: ship CPU backend variants in the CUDA and HIP bundles (#241) (#271)\n\nThe Windows CUDA x64, Linux CUDA and Windows HIP release jobs build with\nGGML_NATIVE=OFF but without GGML_BACKEND_DL/GGML_CPU_ALL_VARIANTS. With\nnative detection off and no explicit ISA flags, ggml enables AVX/AVX2/FMA/F16C\nby default, so each bundle ships a single AVX2-only ggml-cpu library. On a CPU\nwithout AVX2 the process dies at startup with no message: even\n`llama-server --version` exits silently (#241). The CPU and Vulkan bundles\nalready build every variant and pick one at runtime (Vulkan since #198).\n\nBuild those three bundles with GGML_BACKEND_DL=ON and GGML_CPU_ALL_VARIANTS=ON\nas well. The arm64 CUDA leg already used BACKEND_DL with GGML_CPU=OFF and is\nunchanged.\n\n\nClaude-Session: https://claude.ai/code/session_012Q6ecxozUXFFFLE3N2SMiR\n\nCo-authored-by: Claude Opus 5.5 (1M context) "},{"name":".pi","kind":"directory","size":0,"lastCommitHash":"3a653fea932e6a026e14cc0fb6507a986679f688","lastCommitWhen":"2026-08-22T11:31:30+03:00","lastCommitMessage":"ci : add older, min and dry-run options to ccache-clear (#27504)\n\n* ci : add older, min and dry-run options to ccache-clear\n\nAssisted-by: pi:llama.cpp/Qwen3.8-27B\n\n* pi : add note about not wrapping lines in PR descriptions\n\n[no ci]\n\nAssisted-by: pi:llama.cpp/Qwen3.8-27B"},{"name":"app","kind":"directory","size":0,"lastCommitHash":"680a9ae63d60d35c21a0dcd7d3fabdb9c6bfc963","lastCommitWhen":"2026-08-12T14:15:03+02:00","lastCommitMessage":"cmake : introduce semantic versioning (#26839)\n\n* cmake : introduce semantic versioning (wip)\n\nThis commit introduces semantic versioning to llama.cpp.\n\n* squash! cmake : introduce semantic versioning (wip)\n\n* cmake : update test-cmake README notes [no ci]\n\n* include libmtmd in output so show its semversioned\n\n* ci : add make-release workflow\n\n* ci : fix build number check in build-cmake-pkg.yml\n\n* examples : remove trailing whitespace\n\n* ci : abort if upstream ggml version does not exist\n\n* ci : extract step contents into scripts\n\n* ci : add GGML_NATIVE=OFF to ubuntu job\n\n* examples : remove CI build information from test-cmake [no ci]\n\nThis commit removes the nightly/release information that I added\npreviously to keep this focused only on using building and installing\nllama.cpp with cmake and being able to quickly verify changes or\ntroubleshoot issues.\n\n* ci : merge scripts into single script\n\n* remove -dev-build_number support\n\nThis commit removes the incremental build number (versioning) support\nthat I added. This was incorrect and we should only use the semver for\nthe version. Releases will be tag a nightly build and package\nmaintainers/managers that build from source can use the tag and it is\ntherefor important that the correct version is reported. So a\nnightly-build will report the semver without the build number. The build\nnumber and commit as availble via cmake and test-cmake has been updated\nto include an example of using them:\n```console\n$ ./build.sh\n[test-cmake] version: 0.1.0, build: 10360 (08c69e381)\n...\n```\n\nRefs: https://github.com/ggml-org/llama.cpp/pull/26839#discussion_r3755836969\n\n* docs: add initial release.md documentation\n\n* cmake : clean-up and add LLAMA_BUILD_IS_DEV option\n\n* ci : remove version input from make-release job\n\n* ci : add LLAMA_BUILD_IS_DEV=OFF to build-cmake-pkg.yml\n\nRefs: https://github.com/danbev/llama.cpp/actions/runs/31576801921/job/94050639145\n\n* docs : update release notes with LLAMA_BUILD_IS_DEV info [no ci]\n\n* ci : add TODO to winget workflow [no ci]\n\n---------\n\nCo-authored-by: Georgi Gerganov "},{"name":"benches","kind":"directory","size":0,"lastCommitHash":"9b342d0a9f2f4892daec065491583ec2be129685","lastCommitWhen":"2026-03-16T21:50:43+02:00","lastCommitMessage":"benches : add Nemotron 3 Nano on DGX Spark (#20652)\n\n[no ci]"},{"name":"ci","kind":"directory","size":0,"lastCommitHash":"95b8e33e16bb9a60de780a70930ebf729db6a90a","lastCommitWhen":"2026-08-23T15:57:07+03:00","lastCommitMessage":"ci : add test-llama-archs tensor split for Metal (#27598)\n\nRun test-llama-archs with 1 to 4 GGML_METAL_DEVICES, mirroring the\nexisting CUDA runs, and dispatch the job unconditionally since the\nper-backend guards now decide what to run.\n\nAssisted-by: llama.cpp:DeepSeek-v4-Flash-0731"},{"name":"cmake","kind":"directory","size":0,"lastCommitHash":"bf0040e15fd5b716262658f4d652c9cee959cf91","lastCommitWhen":"2026-08-20T15:42:26+02:00","lastCommitMessage":"CI: Use LLVM's OpenMP over MSVC_DEBUG_non_redist on Windows (#26678)\n\n* CI: Use LLVM's OpenMP over MSFT_DEBUG_non_redist on Windows\n\nCurrently, we ship the non-redist debug version of microsoft's libomp.\nThis PR changes this to official LLVM's release, also packaging\nthe license as needed.\n\n* Remove LLVM SHA from job name to increase legibility\n\n* Add temp validations to CI\n\n* Revert \"Add temp validations to CI\"\n\nThis reverts commit eef97c88b5bac280803ebb3c3b7bb09f89b0fd88.\n\n* Build OpenMP in CI\n\n* Make OpenMP fetch self-contained in cmake and cache in CI\n\n* Robustify Licens-packaging\n\n1. Ship OpenMP license, not LLVM's.\n2. Invalidate cache also on checksum of the license\n\n* Remove stale reference in docs/build.md\n\n* No longer package base license in release\n\nThis was scope-creep\n\n* Add explanatory comment to OpenMP license\n\n* Remove arm64 smoke\n\nForgot this during conflict resolution during rebase of\nc54c0e9cf6030a5a54ce8bdd81b3e146d9787d42\n\n* Remove GGML_OPENMP_FETCH_CACHE_DIR as requested by @CISC\n\n* whitespace changes"},{"name":"common","kind":"directory","size":0,"lastCommitHash":"a33d3e21afd4bd4952a0b56fcd6dfee92e3f9ad3","lastCommitWhen":"2026-10-14T02:26:30-04:00","lastCommitMessage":"scr: retain up to 64 matching spans for fragmented transcripts\n"},{"name":"conversion","kind":"directory","size":0,"lastCommitHash":"279df66449f597f9b91d1976865486ba33e2b984","lastCommitWhen":"2026-09-25T11:22:35-07:00","lastCommitMessage":"spec : add DFlash2 support (local convolution + candidate selector) (#27342) (#27816) (#261)\n\n* spec : add DFlash2 support (local convolution + candidate selector) (#27342)\n\n* support DFlash2\n\n* Add p_min in DFlash2\n\nAssisted-by: Claude Opus 5\n\n* Revert unnecessary changes\n\nAssisted-by: Claude Opus 5\n\n* Revert draft sampling in rejection sampling\n\nAssisted-by: Claude Opus 5\n\n* Refactor code structure\n\nAssisted-by: Claude Opus 5\n\n* Delete embedding scaling\n\nAssisted-by: Claude Opus 5\n\n* Gate output transforms on DFlash2\n\nAssisted-by: Claude Opus 5\n\n* Optimize Dflash 2 cost\n\nAssisted-by: Claude Opus 5\n\n* Avoid using atoi\n\nAssisted-by: Claude Opus 5\n\n* Modify comments\n\nAssisted-by: Claude Opus 5\n\n* Move llama_model_dflash_selector_top_k to llama-ext.h\n\nAssisted-by: Claude Opus 5\n\n* Formatting\n\nAssisted-by: Claude Opus 5\n\n* Apply patch to fix the mrope bug\n\nAssisted-by: Claude Opus 5\n\n* fix ci\n\nAssisted-by: Claude Opus 5\n\n* Fix graph number calculation\n\nAssisted-by: Claude Opus 5\n\n* rename hid and unary\n\nAssisted-by: Claude Opus 5\n\n---------\n\n\n\n\n* revert top-k.cu changes\n\n---------\n\n\n\n(cherry picked from commit b10f9ca58c89ccfc3653ac01e979dd085d582b76)\n\nCo-authored-by: Xuan-Son Nguyen \nCo-authored-by: Zihan Zhang \nCo-authored-by: Jian Chen "},{"name":"docs","kind":"directory","size":0,"lastCommitHash":"adfffbe41b2cabcd51fff326ab045662265062bb","lastCommitWhen":"2026-09-25T11:42:36-07:00","lastCommitMessage":"x86: SSE2/SSSE3 vec_dot for PTQ1_0 and PQ2_0 (#248)\n\n* x86: add SSE2/SSSE3 vec_dot for PTQ1_0 and PQ2_0\n\nPTQ1_0 had no x86 SIMD path at all - arch-fallback.h aliased it to the\ngeneric C implementation, so every x86 CPU decoded base-3 trits one\nweight at a time. PQ2_0 had an x86 kernel, but it is gated on VNNI\n(dpbusd), so everything below Ice Lake / Zen 4 fell through to the\nscalar loop.\n\nAdd a 128-bit path for both, in two tiers:\n\n SSE2 - trit decode by multiply-and-shift, sign-extended _mm_madd_epi16\n for the dot. Works on any x86-64.\n SSSE3 - _mm_maddubs_epi16 for the unsigned x signed multiply-add, which\n halves the ALU work in the inner loop.\n\nThe ternary offset (codes are 0..2, values -1..1) is folded into the\naccumulator as a separate maddubs against ones rather than subtracted\nper element, which avoids negating -128 activations.\n\nAlso add GGML_SSSE3 as a build variant so the dispatcher can score it,\nmatching how the other x86 feature tiers are handled.\n\nMeasured on an i5-12600K with AVX disabled at compile time, 8 threads.\nIsolated A/B: same binary, only libggml-cpu.so swapped, runs alternated\nto cancel background load, two pairs per configuration.\n\n Ternary Bonsai 1.7B PQ2_0 pp128 tg32\n scalar 15.33/15.39 9.49/9.62\n SSE2 42.91/43.23 25.20/25.35\n SSSE3 59.49/58.98 32.07/31.42\n\n Ternary Bonsai 2 27B PTQ1_0 pp64 tg16\n scalar 0.71/0.72 0.60/0.62\n SSSE3 2.66/2.66 2.11/2.10\n\n3.3x tg on PQ2_0 and 3.5x on PTQ1_0 over scalar.\n\nAlso verified on a dual Xeon E5645 (Westmere, SSE4.2 ceiling, no AVX):\nbuilds clean and runs both formats, which is the hardware tier this is\nmainly for.\n\n* test-quantize-fns: exact vec_dot check for PTQ1_0 and PQ2_0\n\nThe existing test_vec_dot_q() compares against a tolerance, which is the\nright call for lossy formats but too loose to catch an unpack bug in a\nternary kernel: a wrong trit ordering still lands inside the error bound.\n\nAdd a check that is exact instead. Weights and activations are driven\nover 256 bit patterns per format, and every scale is a power of two, so\ndequantize-then-dot and the packed vec_dot must agree bit for bit. Any\nmismatch in trit order, plane split, or the -1 offset shows up as a hard\nfailure rather than a slightly larger error.\n\nCovers both block counts (1 and 3) so the multi-block loop is exercised.\n\nPasses on SSE2, SSSE3 and AVX2 builds, and on a dual Xeon E5645 where\nneither AVX path is available.\n\n* x86: follow PQ2_0 to the Q8_K activation path\n\nPQ2_0 moved to Q8_K activations, so ggml_vec_dot_pq2_0_q8_0 is no longer\nwhat the traits table dispatches and the SSE tier added earlier in this\nbranch was dead code for normal inference. Add the same 128-bit tier to\nggml_vec_dot_pq2_0_q8_K instead, below the AVX2 path.\n\nStructure mirrors the AVX2 kernel at half the width: the four sub-block\ndots of a 128-weight block accumulate in int32 and the block scale is\napplied once, rather than per sub-block. The existing unpack already\nemits byte b's bit-pair j at element 4b+j, which is the order the Q8_K\nactivations are in, so no repermutation is needed.\n\nCPUs with AVX2 keep the upstream kernel; this only changes what runs\nbelow it, where PQ2_0 was falling back to the scalar loop.\n\nAlso fix the ternary test to build the right activation type per format\n(Q8_K for PQ2_0, Q8_0 for PTQ1_0) and expand Q8_K by hand, since it is\nan activation-only type with no to_float. Verified the exact comparison\nstill catches errors: injecting an off-by-one in the accumulator fails\nall 512 PQ2_0 cases.\n\ni5-12600K, AVX disabled at compile time, 8 threads, isolated A/B\n(same binary, only libggml-cpu.so swapped, alternated, two pairs):\n\n Ternary Bonsai 1.7B PQ2_0 pp128 tg32\n scalar 15.04/15.12 9.97/9.83\n SSSE3 83.81/82.98 44.91/45.21\n\n4.5x tg. Higher than the 3.3x the Q8_0 path gave, because one activation\nscale per 256 removes the per-sub-block float chain.\n\ntest-quantize-fns passes on SSE2, SSSE3 and AVX2 builds."},{"name":"examples","kind":"directory","size":0,"lastCommitHash":"36761482b026e0a4de4f6cabe08cf756b786227c","lastCommitWhen":"2026-09-16T14:36:22-07:00","lastCommitMessage":"speculative: stop plain drafter+target falling back to KV checkpoints (#173)\n\n* speculative: stop plain drafter+target falling back to KV checkpoints\n\nOn hybrid (recurrent + attention) models, plain drafter+target speculative\ndecoding was classified as unable to do partial sequence removal, and fell\nback to a whole-KV checkpoint save/restore every round. On a 27B qwen35 that\nwas 74-77% of each round -- roughly 0.5 s per drafted token, ~15x the\ndrafter's own compute -- which made speculation about 2x slower than plain\ndecoding regardless of draft quality.\n\nTwo causes, both needed:\n\n1. need_n_rs_seq() listed DRAFT_MTP, DRAFT_EAGLE3, DRAFT_DFLASH and\n DRAFT_DSPARK but not DRAFT_SIMPLE, so the ordinary drafter+target case got\n n_rs_seq = 0. common_context_can_seq_rm() then failed its\n llama_n_rs_seq() > 0 test, llama_memory_seq_rm() failed on the hybrid\n memory, and the context was classified COMMON_CONTEXT_SEQ_RM_TYPE_FULL.\n LLM_ARCH_QWEN35 is already in llm_arch_supports_rs_rollback(), so the\n architecture was never the blocker.\n\n2. The draft context explicitly zeroed cparams.n_rs_seq. It has to match the\n target: on partial acceptance both contexts drop the rejected suffix, and a\n draft context at n_rs_seq = 0 silently fails its seq_rm and keeps stale\n positions, which then trips the M-RoPE monotonic-position check on the next\n batch (seq_pos_max == batch seq_pos_min). Traced: tgt_ok=1 dft_ok=0 with\n tgt_max=10 against dft_max=12.\n\nNo change is needed in speculative-simple.cpp -- it already removes the\nrejected draft tokens unconditionally after the use_ckpt_tgt block, and that\npath is always reached when use_ckpt_tgt is false. An earlier version of this\nfix added a second removal there; ablating it gave identical results, so it\nwas dropped.\n\nMeasured on an Arc B390, 2B-PTQ1_0 drafting 27B-PTQ1_0, greedy, against a\n4.43 t/s llama-bench decode-only baseline:\n\n k=1 1.137x / 1.356x (two prompts)\n k=2 0.482x -> 1.085x\n k=4 0.535x -> 0.727x\n\nGenerated text is byte-identical to a k=1 reference over the common prefix.\n\nAlso note the reported acceptance rate changes meaning for hybrid models that\npreviously took the checkpoint path: that path continues before n_drafted and\nn_accept are incremented, so partial-acceptance rounds were never counted and\nthe rate read 100%. With partial rounds counted it is 14-87% depending on k\nand workload.\n\n* speculative: keep the checkpoint path when the rollback window does not fit the ubatch\n\nRecurrent and hybrid memory keep the last n_rs_seq + 1 tokens of a sequence inside one ubatch and assert on it. Decide the fit once in common_context_params_to_llama, warn, and fall back to n_rs_seq = 0 so small -ub configurations keep working. The draft context inherits that value instead of overriding it.\n\nspeculative-simple now checkpoints per round when a draft is longer than llama_n_rs_seq(), as the server does, so an n-gram speculator ahead of draft-simple no longer relies on a seq_rm that cannot succeed.\n\n* speculative: base the rollback fit on the effective micro-batch\n\nllama_context clamps the micro-batch to min(n_batch, n_ubatch) and n_batch to n_ctx, so checking the requested -ub let --batch-size 8 --ubatch-size 512 past the guard and into the same assertion. Check the effective size instead.\n\nThe memory fitter no longer forces the draft context's n_rs_seq to zero: recurrent state allocates mem_size * (1 + n_rs_seq) rows, so a fit that measured a different value underestimated the draft context.\n\nIn speculative-simple the pre-draft reset now loads a checkpoint only for a FULL context. use_ckpt_dft carries the previous round's RS-overflow decision, and no checkpoint is taken for an RS context at that point, so the load could roll the draft context back to a stale position."},{"name":"ggml","kind":"directory","size":0,"lastCommitHash":"0163094fa579c4547035f7700b1f6ca009fc8331","lastCommitWhen":"2026-10-13T19:13:57-04:00","lastCommitMessage":"bonsai: q8_0 decode/verify flash attention raw K/V lead staging (fattn-verify-lead)\n\nq8_0 decode/verify flash attention (the 4x8 tile) stages raw K/V with cp.async and starts the next tile's K and V a phase early. Bit-identical; decode +2.9% at 64K depth.\n"},{"name":"gguf-py","kind":"directory","size":0,"lastCommitHash":"279df66449f597f9b91d1976865486ba33e2b984","lastCommitWhen":"2026-09-25T11:22:35-07:00","lastCommitMessage":"spec : add DFlash2 support (local convolution + candidate selector) (#27342) (#27816) (#261)\n\n* spec : add DFlash2 support (local convolution + candidate selector) (#27342)\n\n* support DFlash2\n\n* Add p_min in DFlash2\n\nAssisted-by: Claude Opus 5\n\n* Revert unnecessary changes\n\nAssisted-by: Claude Opus 5\n\n* Revert draft sampling in rejection sampling\n\nAssisted-by: Claude Opus 5\n\n* Refactor code structure\n\nAssisted-by: Claude Opus 5\n\n* Delete embedding scaling\n\nAssisted-by: Claude Opus 5\n\n* Gate output transforms on DFlash2\n\nAssisted-by: Claude Opus 5\n\n* Optimize Dflash 2 cost\n\nAssisted-by: Claude Opus 5\n\n* Avoid using atoi\n\nAssisted-by: Claude Opus 5\n\n* Modify comments\n\nAssisted-by: Claude Opus 5\n\n* Move llama_model_dflash_selector_top_k to llama-ext.h\n\nAssisted-by: Claude Opus 5\n\n* Formatting\n\nAssisted-by: Claude Opus 5\n\n* Apply patch to fix the mrope bug\n\nAssisted-by: Claude Opus 5\n\n* fix ci\n\nAssisted-by: Claude Opus 5\n\n* Fix graph number calculation\n\nAssisted-by: Claude Opus 5\n\n* rename hid and unary\n\nAssisted-by: Claude Opus 5\n\n---------\n\n\n\n\n* revert top-k.cu changes\n\n---------\n\n\n\n(cherry picked from commit b10f9ca58c89ccfc3653ac01e979dd085d582b76)\n\nCo-authored-by: Xuan-Son Nguyen \nCo-authored-by: Zihan Zhang \nCo-authored-by: Jian Chen "},{"name":"grammars","kind":"directory","size":0,"lastCommitHash":"8edaca9034bd6dbe729895315b922bdd90d9766b","lastCommitWhen":"2026-06-15T01:33:38+08:00","lastCommitMessage":"docs : fix typos in CUDA-FEDORA.md and grammars/README.md (#24459)"},{"name":"include","kind":"directory","size":0,"lastCommitHash":"6067dbea986ed1359db98c42ab06eb49426bbf16","lastCommitWhen":"2026-09-17T21:25:55-07:00","lastCommitMessage":"dspark: add corrected draft runtime and shared packed target head\n"},{"name":"licenses","kind":"directory","size":0,"lastCommitHash":"516a4ca9b5f2fa72c2a71f412929a67cf76a6213","lastCommitWhen":"2026-01-14T18:02:47+01:00","lastCommitMessage":"refactor : remove libcurl, use OpenSSL when available (#18828)\n\n"},{"name":"media","kind":"directory","size":0,"lastCommitHash":"2cfef4d117d67ab1dec002915b48a15d11ee1973","lastCommitWhen":"2025-09-10T14:51:28+03:00","lastCommitMessage":"media : add transparent icon svg and png [no ci] (#15891)\n\n"},{"name":"models","kind":"directory","size":0,"lastCommitHash":"ad1de39e0708e3ced9c71bb3c82d93a2c046a73f","lastCommitWhen":"2026-08-15T17:11:05+02:00","lastCommitMessage":"model: add Kimi-K3 text model (#26185)\n\n* model: add Kimi-K3 text model\n\nHybrid KDA (linear) + MLA (full) attention as in Kimi-Linear-48B, plus five\nthings that architecture does not have:\n\n 1. cross-layer residual attention (attn_res_block_size)\n 2. latent MoE (routed experts run at n_expert_latent)\n 3. situ activation (replaces SwiGLU everywhere)\n 4. MLA output gate (sigmoid gate before o_proj)\n 5. full-rank KDA gate (single ssm_g instead of ssm_g_a/ssm_g_b)\n\nK3's text_config reports KimiLinearForCausalLM - the older 48B architecture -\nso get_model_architecture routes on the top-level name instead.\n\nThe KDA decay gate has two forms, selected by linear_attn_config's\ngate_lower_bound. It is not a clamp: when set it swaps the activation entirely\n(fla/ops/kda/gate.py), from -exp(A_log)*softplus(x) to\nlower_bound*sigmoid(exp(A_log)*x). K3 sets it to -5.0; kimi-linear leaves it\nunset, so that path is unchanged.\n\nCross-layer residuals reuse ggml_dsv4_hc_pre for the weighted sum. That op is\nCPU + CUDA only, so Metal/Vulkan will fall back per-node until those kernels\nexist.\n\nThe routed experts ship as compressed-tensors \"mxfp4-pack-quantized\". That is\nbit-compatible with ggml's MXFP4 - same E2M1 code assignment, same E8M0 scale\nbyte, only the nibble positions within a block differ - so they are repacked\nrather than dequantized, losslessly and without a ~5.5 TB bf16 round-trip.\nThe repack is built lazily because gguf_writer holds every added tensor until\nthe final write. DeepSeek-V4 was already doing the identical bit-shuffling, so\nit now shares the helper.\n\nVerified against Moonshot's own code path (transformers + fla's Triton KDA\nkernels) on a tiny model exercising every K3-specific feature. Final-position\nlogits vs the fp32 reference: 6.7e-05 rel / corr 1.00000000 for both the\nchunked and the recurrent delta-net path. MXFP4 blocks dequantize to the source\nweights with 0.0e+00 error.\n\nAssisted-By: Claude Opus 5 (1M context) \n\n* model: fix ty errors in the Kimi-K3 converter\n\n- `_res_parts` buffers (kind, tensor) pairs, not bare tensors\n- `get_tensors` must return an Iterator, matching ModelBase\n- LazyBase's `func` takes one argument, so pass the expert loaders through\n `args` instead of the closure\n- borrowing KimiLinearModel.set_vocab from an unrelated TextModel is\n deliberate and safe, but not expressible in the signature\n\nNo behaviour change: the MXFP4 repack still dequantizes to the source weights\nwith 0.0e+00 error and end-to-end logits are unchanged (8.386e-03 rel,\ncorr 0.99996630).\n\nAssisted-By: Claude Opus 5 (1M context) \n\n* Update conversion/kimi_k3.py\n\nCo-authored-by: Boris Dvorkin \n\n* Increase LLAMA_MAX_EXPERTS from 512 to 1024\n\n* tests : support for Kimi K3 in archs test\n\n* chat : add Kimi K3 chat format (reasoning, content, typed tool calls)\n\nK3's assistant output is an XTML-ish tagged format built by the template's\nopen_tag/close_tag macros. Two properties break generic parsing:\n\n1. The generation prompt ends with open_tag('think'), so the completion\n starts inside the think section with no opening marker in the output\n (thinking_forced_open).\n2. Only <|open|>/<|close|>/<|sep|>/<|end_of_msg|> are special tokens; tag\n names (\"think\", \"response\", \"message\") are ordinary text tokens.\n\nAdds common_chat_params_init_kimi_k3 (PEG_NATIVE) with detection on the\nmarker trio, reasoning extraction, response unwrapping, and tool-call\nparsing of the tools/call/argument tag structure with argument types\ntaken from the tool schema. Includes the K3 chat template fixture and 9\ntest-chat cases derived from real generations of the full 2.8T model.\n\nVerified end-to-end against Kimi-K3-Q2_K (GrEarl/Kimi-K3-GGUF) on 8x B200:\ncontent, reasoning_content, streaming deltas, and tool_calls all correct;\nfinish_reason stop/tool_calls as appropriate.\n\nCo-Authored-By: Claude Fable 5 \n\n* chat : add message_delimiters for Kimi K3\n\nPer-role message-start markers for token-level span splitting. User and\nassistant messages carry only the role attribute, so their full opener\n(through <|sep|>) is used; system and tool messages continue with more\nattributes (type=/tool=/index=), so those delimiters stop after the\nrole's closing quote. Verified against the K3 tiktoken vocabulary that\nthe closing quote is always a standalone token across all attribute\nvariants, so the token-level prefix match stays exact.\n\nCo-Authored-By: Claude Fable 5 \n\n* fix: apply nits from @ngxson and text fixes from @danielhanchen\n\n* tests : added missing hyperparameters and tensors for Kimi K3 in test-llama-archs\n\n* chore : move overly verbose header file comments to Kimi K3 source file\n\n* tests : re-enabled KIMI_K3 in test-llama-archs for WebGPU backend\n\n* model-saver : emit kda_gate_lower_bound for Kimi K3\n\nQuick fix. The Kimi K3 loader reads kda_gate_lower_bound and gates a graph branch on it (it scales the KDA gate when the bound is above -INFINITY), but the model\nsaver never wrote the key, so a save->load roundtrip silently dropped it back to the -INFINITY default and changed the model's output. The real K3 config sets gate_lower_bound = -5.0.\n\nI propose to emit it from the saver, and set it to -5.0 in the test-llama-archs K3 case so the roundtrip check exercises it (the roundtrip fails without the saver line).\n\n* Refactor conditional for model architecture check\n\n* tests : re-enabled (again) KIMI_K3 and MINIMAX_M3 in test-llama-archs for WebGPU backend\n\n* fix code comments\n\n* add template on conversion\n\n* move repack_mxfp4_blocks to model base\n\n* nits\n\n* add_value_length\n\n* optimize res_stack construction\n\n* nits\n\n---------\n\nCo-authored-by: Boris Dvorkin \nCo-authored-by: Stanisław Szymczyk \nCo-authored-by: Deepankar Singh \nCo-authored-by: Claude Fable 5 \nCo-authored-by: Caleb DeLeeuw \nCo-authored-by: Xuan Son Nguyen "},{"name":"patches","kind":"directory","size":0,"lastCommitHash":"8906fd8c9fd9d2dcbffad469432b8d2edf0925fd","lastCommitWhen":"2026-10-18T08:02:45-04:00","lastCommitMessage":"cache: conversations are stored against their learned prefix\n"},{"name":"pocs","kind":"directory","size":0,"lastCommitHash":"6990e2f1f7581d332a6a1f34d6c567be70138184","lastCommitWhen":"2026-04-17T11:11:46+03:00","lastCommitMessage":"libs : rename libcommon -> libllama-common (#21936)\n\n* cmake : allow libcommon to be shared\n\n* cmake : rename libcommon to libllama-common\n\n* cont : set -fPIC for httplib\n\n* cont : export all symbols\n\n* cont : fix build_info exports\n\n* libs : add libllama-common-base\n\n* log : add common_log_get_verbosity_thold()"},{"name":"requirements","kind":"directory","size":0,"lastCommitHash":"2468576f241235452013308597e6de1b78866996","lastCommitWhen":"2026-08-11T21:58:53+08:00","lastCommitMessage":"requirements: use stable torch packages on s390x (#26864)"},{"name":"scripts","kind":"directory","size":0,"lastCommitHash":"985b14912bc9a3d8f27c1ac569e46f4092123722","lastCommitWhen":"2026-08-24T10:49:20+03:00","lastCommitMessage":"ci : apply ccache-clear with older/min/dry-run to all ccache jobs (#27602)\n\n* ci : apply ccache-clear with older/min/dry-run to all ccache jobs\n\nAssisted-by: llama.cpp:DeepSeek-v4-Flash-0731\n\n* ci : install gh in ccache-clear if missing (container jobs)\n\nThe ccache-clear action relies on the gh CLI, which is not present in\ncontainer-based jobs. Install it on demand so those jobs can clear caches.\n\nAssisted-by: llama.cpp:DeepSeek-v4-Flash-0731\n\n* ci : install gh via apt repo in ccache-clear\n\nThe install.sh script used previously is no longer served (404). Switch to\nthe official GitHub CLI apt repository, which is still available.\n\nAssisted-by: llama.cpp:DeepSeek-v4-Flash-0731\n\n* ci : pass --repo to gh cache commands in ccache-clear\n\nIn container jobs gh cannot auto-detect the repository from git, so\ngh cache list/delete fail with 'failed to run git: not a git repository'.\nPass the repository explicitly via --repo using GITHUB_REPOSITORY.\n\nAssisted-by: llama.cpp:DeepSeek-v4-Flash-0731\n\n* ci : drop -new suffix from vulkan ccache key\n\nThe -new suffix was only needed to force a fresh cache. With\nccache-clear now evicting stale caches, the original key can be used\nagain. The old ccache-vulkan-ubuntu-24.04-arm-new entries still match\nthe ccache-clear key prefix and are cleaned up automatically.\n\nAssisted-by: llama.cpp:DeepSeek-v4-Flash-0731\n\n* ci : fix ccache-clear date parsing on macOS (BSD date)\n\nmacOS ships BSD date, which has no -d option. The older cutoff check\nwas silently disabled there: 'date: illegal option -- d' errors in the\nlog and the loop was only stopped by the min limit, risking deletion\nof caches not older than the cutoff (e.g. saved by a concurrent job).\n\nParse the ISO-8601 timestamps with GNU date when available and fall\nback to BSD date otherwise (TZ=UTC, fractional seconds dropped).\n\nAssisted-by: llama.cpp:DeepSeek-v4-Flash-0731\n\n* ci : extract ccache-clear logic into scripts/ccache-clear.sh\n\nThe composite action now consists of a dedicated step that installs the\nGitHub CLI when missing (e.g. in container jobs) and a thin step that\ncalls the new script. The script follows the make-release-checks.sh\nconventions (usage/env header, set -euo pipefail, CLI flags) and only\nchecks that gh is available. The action inputs are unchanged, so the\nworkflow steps are untouched.\n\nAssisted-by: llama.cpp:DeepSeek-v4-Flash-0731\n\n* ci : remove unused apple ccaches"},{"name":"skills","kind":"directory","size":0,"lastCommitHash":"4a08fa29705b8177e332b134306566c2c4b95902","lastCommitWhen":"2026-08-23T18:38:51+02:00","lastCommitMessage":"test: move tools/parser to tests (#27548)"},{"name":"src","kind":"directory","size":0,"lastCommitHash":"6d1917060d72cfffc6b54391ef8006b7f51361b9","lastCommitWhen":"2026-10-13T19:13:57-04:00","lastCommitMessage":"bonsai: fused CUDA kernel for DeltaNet conv path (gdn-conv)\n\nOne CUDA kernel for the DeltaNet conv path (concat -> ssm_conv -> silu -> conv-state writes): the concat is never materialized; same arithmetic. Bit-identical; MTP decode +1.5%. GGML_CUDA_SSM_CONV_CONCAT_FUSION=0 turns it off.\n"},{"name":"tests","kind":"directory","size":0,"lastCommitHash":"8906fd8c9fd9d2dcbffad469432b8d2edf0925fd","lastCommitWhen":"2026-10-18T08:02:45-04:00","lastCommitMessage":"cache: conversations are stored against their learned prefix\n"},{"name":"tools","kind":"directory","size":0,"lastCommitHash":"8906fd8c9fd9d2dcbffad469432b8d2edf0925fd","lastCommitWhen":"2026-10-18T08:02:45-04:00","lastCommitMessage":"cache: conversations are stored against their learned prefix\n"},{"name":"vendor","kind":"directory","size":0,"lastCommitHash":"6657ded4faa3b8450221119fc6b4d002e35104a2","lastCommitWhen":"2026-08-23T10:38:29+03:00","lastCommitMessage":"vendor : update subprocess.h (#27409)"},{"name":".clang-format","kind":"file","size":4961,"lastCommitHash":"f1fbffb5c0b34b2a68febb7da3fd0f8333f1ed4c","lastCommitWhen":"2025-09-16T08:59:19+02:00","lastCommitMessage":"fix: apply clang-format to CUDA macros (#16017)\n\nclang-format previously broke long CUDA macros (e.g. __launch_bounds__) into\nunreadable line breaks inside template declarations, such as:\n\n template\n __launch_bounds__(nwarps*ggml_cuda_get_physical_warp_size(), 1)\n\nThis change adjusts formatting rules so that CUDA macros remain consistent\nand aligned with the surrounding template syntax."},{"name":".clang-tidy","kind":"file","size":931,"lastCommitHash":"351f3da39c85f59d581fc184f09283da7f099a3b","lastCommitWhen":"2025-09-22T19:57:46+02:00","lastCommitMessage":"clang-tidy : disable warning about performance enum size (#16127)\n\nDisable 'performance-enum-size' checking:\n\nEnum 'llama_token_type' uses a larger base type ('unsigned int', size: 4 bytes)\nthan necessary for its value set, consider using 'std::uint8_t' (1 byte) as the\nbase type to reduce its size."},{"name":".dockerignore","kind":"file","size":261,"lastCommitHash":"67e9fd3b74b7fab3a153161f1942cc2121aa90a3","lastCommitWhen":"2026-06-20T05:54:42-05:00","lastCommitMessage":"docker : prebuild web UI for s390x build [no release] (#24829)"},{"name":".ecrc","kind":"file","size":97,"lastCommitHash":"ad76569f8e78ab6ca921bda25cef25a157361719","lastCommitWhen":"2024-08-27T08:58:50+03:00","lastCommitMessage":"common : Update stb_image.h to latest version (#9161)\n\n* Update stb_image.h to latest version\r\n\r\nFixes https://github.com/ggerganov/llama.cpp/issues/7431\r\n\r\n* Update .ecrc"},{"name":".editorconfig","kind":"file","size":1217,"lastCommitHash":"59778f0196a82db32580bb649d5d839355d6d7bf","lastCommitWhen":"2026-05-16T02:02:40+02:00","lastCommitMessage":"ui: Restructure repo to use `tools/ui` folder and `ui` / `UI` / `llama-ui` / `LLAMA_UI` naming (#23064)\n\n* webui: Move static build output from `tools/server/public` to `build/ui` directory\n\n* refactor: Move to `tools/ui`\n\n* refactor: rename CMake variables and preprocessor defines\n\n- Rename LLAMA_BUILD_WEBUI -> LLAMA_BUILD_UI (old kept as deprecated)\n- Rename LLAMA_USE_PREBUILT_WEBUI -> LLAMA_USE_PREBUILT_UI (old kept as deprecated)\n- Backward compat: old vars auto-forward to new ones with DEPRECATION warning\n- Rename internal vars: WEBUI_SOURCE -> UI_SOURCE, WEBUI_SOURCE_DIR -> UI_SOURCE_DIR, etc.\n- Rename HF bucket: LLAMA_WEBUI_HF_BUCKET -> LLAMA_UI_HF_BUCKET\n- Emit both LLAMA_BUILD_WEBUI and LLAMA_BUILD_UI preprocessor defines\n- Emit both LLAMA_WEBUI_DEFAULT_ENABLED and LLAMA_UI_DEFAULT_ENABLED\n\n* refactor: rename CLI flags (--webui -> --ui) with backward compat\n\n- Add --ui/--no-ui (old --webui/--no-webui kept as deprecated aliases)\n- Add --ui-config (old --webui-config kept as deprecated alias)\n- Add --ui-config-file (old --webui-config-file kept as deprecated alias)\n- Add --ui-mcp-proxy/--no-ui-mcp-proxy (old --webui-mcp-proxy kept as deprecated)\n- Add new env vars: LLAMA_ARG_UI, LLAMA_ARG_UI_CONFIG, LLAMA_ARG_UI_CONFIG_FILE, LLAMA_ARG_UI_MCP_PROXY\n- C++ struct fields: params.ui, params.ui_config_json, params.ui_mcp_proxy added alongside old fields\n- Backward compat: old fields synced to new ones in g_params_to_internals\n\n* refactor: update C++ server internals with backward compat\n\n- Rename json_webui_settings -> json_ui_settings (both kept in server_context_meta)\n- Rename params.webui usage -> params.ui (both synced, old still works)\n- JSON API emits both \"ui\"/\"ui_settings\" and \"webui\"/\"webui_settings\" keys\n- Server routes use params.ui_mcp_proxy || params.webui_mcp_proxy\n- Preprocessor guards use #if defined(LLAMA_BUILD_UI) || defined(LLAMA_BUILD_WEBUI)\n\n* refactor: rename CI/CD workflows, artifacts, and build script\n\n- Rename webui-build.yml -> ui-build.yml; artifact webui-build -> ui-build\n- Rename webui-publish.yml -> ui-publish.yml; var HF_BUCKET_WEBUI_STATIC_OUTPUT -> HF_BUCKET_UI_STATIC_OUTPUT\n- Rename server-webui.yml -> server-ui.yml; job webui-build/checks -> ui-build/checks\n- Update server.yml: job/artifact refs webui-build -> ui-build\n- Update release.yml: all webui-build/publish refs -> ui-build/publish; HF_TOKEN_WEBUI_STATIC_OUTPUT -> HF_TOKEN_UI_STATIC_OUTPUT\n- Update server-self-hosted.yml: webui-build -> ui-build\n- Update build-self-hosted.yml: HF_WEBUI_VERSION -> HF_UI_VERSION\n- Rename webui-download.cmake -> ui-download.cmake (internal refs updated)\n- Update labeler.yml: server/webui -> server/ui path label\n\n* docs: update CODEOWNERS and server README docs\n\n- Update CODEOWNERS: team ggml-org/llama-webui -> ggml-org/llama-ui, path /tools/server/webui/ -> /tools/ui/\n- Update server README.md: CLI tables show --ui flags with deprecated --webui aliases\n- Update server README-dev.md: \"WebUI\" -> \"UI\", paths updated to tools/ui/\n\n* fix: Small fixes for UI build\n\n* fix: CMake.txt syntax\n\n* chore: Formatting\n\n* fix: `.editorconfig` for llama-ui\n\n* chore: Formatting\n\n* refactor: Use `APP_NAME` in Error route\n\n* refactor: Cleanup\n\n* refactor: Single migration service\n\n* make llama-ui a linkable target\n\n* fix: UI Build output\n\n* fix: Missing change\n\n* fix: separate llama-ui npm build output into build/tools/ui/dist subfolder + use cmake npm build instead of downloading ui-build.yml artifacts in CI\n\n* refactor: UI workflows cleanup\n\n---------\n\nCo-authored-by: Xuan Son Nguyen "},{"name":".flake8","kind":"file","size":565,"lastCommitHash":"1d36b3670b285e69e58b9d687c770a2a0a192194","lastCommitWhen":"2025-05-02T20:27:13+02:00","lastCommitMessage":"llama : move end-user examples to tools directory (#13249)\n\n* llama : move end-user examples to tools directory\n\n---------\n\nCo-authored-by: Xuan Son Nguyen "},{"name":".gitignore","kind":"file","size":1713,"lastCommitHash":"f7ca93d12c523491e9123ee7ce6a881447c20e52","lastCommitWhen":"2026-06-12T15:53:26+02:00","lastCommitMessage":"ui: PWA support (#23871)\n\n* feat: Add basic PWA support and service worker for offline caching\n\n* feat: Vite PWA implementation WIP\n\n* feat: Improve PWA icons generation\n\n* feat: Add PWA workbox to server routes\n\n* feat: Include `version.json` in static assets\n\n* feat: Add HTTP cache headers for PWA static assets\n\n* feat: Update app name for `apple-mobile-web-app-title`\n\n* feat: Implement PWA versioning and automatic update detection\n\n* chore: Update `.gitignore` files\n\n* feat: Splash Screens\n\n* feat: Add dark mode favicon support\n\n* refactor: Cleanup\n\n* fix: Use dark logo for dark splash screens\n\n* refactor: Simplify favicons SVG code\n\n* fix: Adjust caching and polling for reliable service worker updates\n\n* fix: Add missing favicon entry\n\n* fix: Align PWA service worker configuration with SvelteKit build structure\n\n* fix: Replace hashed bundle paths with versioned static paths\n\n* test: Add PWA tests\n\n* ci: Add build output for unit tests\n\n* refactor: Cleanup\n\n* fix: Server build & release versioning\n\n* chore: Update package-lock.json\n\n* chore: Increase PWA cache size\n\n* chore: Update packages\n\n* feat: Update favicons\n\n* refactor: Post-merge fix\n\n* feat: support explicit build version for PWA cache busting\n\n* fix: CI\n\n* feat: Improve PWA Refresh Alert UI\n\n* feat: Add toggleable build version display\n\n* refactor: Cleanup\n\n* feat: Add version mismatch detection and manual app reload\n\n* refactor: replace dynamic imports with static\n\n* refactor: Cleanup\n\n* feat: Add safe space for `pwa-.png` rendered icons\n\n* fix: use relative paths for PWA assets to support base path deployment\n\n* feat: add PWA mode detection via URL query parameter\n\n* feat: Use ?cache=true for SW-cached PWA assets\n\n* refactor: Build process cleanup\n\n* refactor: Decouple PWA versioning and remove ?cache=true workaround\n\n* chore: Update README logo\n\n* feat: Include PWA Assets generation in build script\n\n* refactor: `usePwa` hook for core layout\n\n* fix: Relativize base vite plugin\n\n* fix: remove unnecessary backslash escapes in test regexes\n\n* test: update static asset paths for API Key test\n\n* refactor: Move SvelteKit PWA Options config to constants\n\n* ui: fix update notification never appearing\n\nKeep the PWA hook object intact instead of destructuring needRefreshByStorage,\nwhich freezes the reactive getter. Also exclude loading.html from PWA\nprecache to prevent 404 errors and broken SW installation."},{"name":".gitmodules","kind":"file","size":0,"lastCommitHash":"d4cdd9c1c3cebdafca735958597de4ff7b7c0f54","lastCommitWhen":"2025-07-03T07:48:32+03:00","lastCommitMessage":"ggml : remove kompute backend (#14501)\n\nggml-ci"},{"name":".pre-commit-config.yaml","kind":"file","size":447,"lastCommitHash":"a2ac89d6efb41b535778bfeaecaae8fe295b6ed3","lastCommitWhen":"2024-05-03T22:36:41+03:00","lastCommitMessage":"convert.py : add python logging instead of print() (#6511)\n\n* convert.py: add python logging instead of print()\r\n\r\n* convert.py: verbose flag takes priority over dump flag log suppression\r\n\r\n* convert.py: named instance logging\r\n\r\n* convert.py: use explicit logger id string\r\n\r\n* convert.py: convert extra print() to named logger\r\n\r\n* convert.py: sys.stderr.write --> logger.error\r\n\r\n* *.py: Convert all python scripts to use logging module\r\n\r\n* requirements.txt: remove extra line\r\n\r\n* flake8: update flake8 ignore and exclude to match ci settings\r\n\r\n* gh-actions: add flake8-no-print to flake8 lint step\r\n\r\n* pre-commit: add flake8-no-print to flake8 and also update pre-commit version\r\n\r\n* convert-hf-to-gguf.py: print() to logger conversion\r\n\r\n* *.py: logging basiconfig refactor to use conditional expression\r\n\r\n* *.py: removed commented out logging\r\n\r\n* fixup! *.py: logging basiconfig refactor to use conditional expression\r\n\r\n* constant.py: logger.error then exit should be a raise exception instead\r\n\r\n* *.py: Convert logger error and sys.exit() into a raise exception (for atypical error)\r\n\r\n* gguf-convert-endian.py: refactor convert_byteorder() to use tqdm progressbar\r\n\r\n* verify-checksum-model.py: This is the result of the program, it should be printed to stdout.\r\n\r\n* compare-llama-bench.py: add blank line for readability during missing repo response\r\n\r\n* reader.py: read_gguf_file() use print() over logging\r\n\r\n* convert.py: warning goes to stderr and won't hurt the dump output\r\n\r\n* gguf-dump.py: dump_metadata() should print to stdout\r\n\r\n* convert-hf-to-gguf.py: print --> logger.debug or ValueError()\r\n\r\n* verify-checksum-models.py: use print() for printing table\r\n\r\n* *.py: refactor logging.basicConfig()\r\n\r\n* gguf-py/gguf/*.py: use __name__ as logger name\r\n\r\nSince they will be imported and not run directly.\r\n\r\n* python-lint.yml: use .flake8 file instead\r\n\r\n* constants.py: logger no longer required\r\n\r\n* convert-hf-to-gguf.py: add additional logging\r\n\r\n* convert-hf-to-gguf.py: print() --> logger\r\n\r\n* *.py: fix flake8 warnings\r\n\r\n* revert changes to convert-hf-to-gguf.py for get_name()\r\n\r\n* convert-hf-to-gguf-update.py: use triple quoted f-string instead\r\n\r\n* *.py: accidentally corrected the wrong line\r\n\r\n* *.py: add compilade warning suggestions and style fixes"},{"name":"AGENTS.md","kind":"file","size":10491,"lastCommitHash":"c06b847984ae718517c5cb3d2fc11abb6c157f8e","lastCommitWhen":"2026-08-29T22:39:43-07:00","lastCommitMessage":"Update AGENTS.md"},{"name":"AUTHORS","kind":"file","size":86579,"lastCommitHash":"169e4a7ff201e666c2325bcd35afb64573e39d09","lastCommitWhen":"2026-08-18T14:35:04+03:00","lastCommitMessage":"readme : update status badges + regen AUTHORS (#27317)\n\n* readme : update status badges\n\n* authors : regen"},{"name":"build-xcframework.sh","kind":"file","size":24300,"lastCommitHash":"27e345b574dd8c8838e2c06e47699a3135f16ec9","lastCommitWhen":"2026-08-18T11:16:51+03:00","lastCommitMessage":"build : fix xcframework + cmake clean-up (#27304)\n\n* xcframework : fix build\n\n* mtmd : remove unused include path\n\n* vendor : use vendor::hash alias target in cmake\n\nCMake reserves \"::\" in target names for imported/alias targets, so the real\ntarget keeps the name vendor-hash and a vendor::hash ALIAS target is added.\nConsumers (mtmd, llama-gguf-hash) now link against the namespaced alias.\n\nAssisted-by: pi:llama.cpp/Qwen3.8-27B\n\n* vendor : add cmake targets for all vendored libs with vendor:: aliases\n\nAdd INTERFACE targets for the header-only vendor libs (miniaudio, nlohmann,\nsheredom, stb) and ALIAS targets named vendor:: for all of them,\nincluding cpp-httplib and hash. Each exposes the vendor/ root so includes\nare namespaced, e.g. .\n\nConsolidate the per-lib add_subdirectory calls into a single\nadd_subdirectory(vendor), keeping the cpp-httplib gate on LLAMA_BUILD_COMMON.\nConsumers (llama-common, mtmd) now link the aliases instead of relying on\nraw vendor/ include paths.\n\nhash: consumers now include via \"hash/hash.h\"; the vendor/hash dir is kept\nas a PRIVATE include so the synced upstream sources compile unmodified.\n\nAssisted-by: pi:llama.cpp/Qwen3.8-27B\n\n* readme : use foo/bar names in acknowledgements\n\nAssisted-by: pi:llama.cpp/Qwen3.8-27B\n\n* ocd : fix valign"},{"name":"CLAUDE.md","kind":"file","size":106,"lastCommitHash":"3595ae5963f1583f53beecf9725c919d309e15da","lastCommitWhen":"2025-12-29T16:01:32+01:00","lastCommitMessage":"contributing: tighten AI usage policy (#18388)\n\n* contributing: tighten AI usage policy\n\n* refactor AGENTS.md\n\n* proofreading\n\n* update contributing\n\n* add claude.md\n\n* add trailing newline\n\n* add note about dishonest practices\n\n* rm point about dishonest\n\n* rm requirement watermarking\n\n* add .gemini/settings.json\n\n* allow initially AI-generated content\n\n* revise\n\n* Update CONTRIBUTING.md\n\nCo-authored-by: Johannes Gäßler \n\n* improve\n\n* trailing space\n\n* Apply suggestions from code review\n\nCo-authored-by: Johannes Gäßler \n\n* update\n\n---------\n\nCo-authored-by: Johannes Gäßler "},{"name":"CMakeLists.txt","kind":"file","size":10847,"lastCommitHash":"6067dbea986ed1359db98c42ab06eb49426bbf16","lastCommitWhen":"2026-09-17T21:25:55-07:00","lastCommitMessage":"dspark: add corrected draft runtime and shared packed target head\n"},{"name":"CMakePresets.json","kind":"file","size":4570,"lastCommitHash":"84b396e0510855a95d591afdf1f21c562cb3712a","lastCommitWhen":"2025-07-13T08:12:36+03:00","lastCommitMessage":"cmake : Add CMake presets for Linux and GCC (#14656)\n\n"},{"name":"CODEOWNERS","kind":"file","size":6491,"lastCommitHash":"fa72aeccb23947074c12b5fec25f5b6ced28cbfe","lastCommitWhen":"2026-07-24T17:53:54+02:00","lastCommitMessage":"HIP: remove rocWMMA FlashAttention (#26046)"},{"name":"CONTRIBUTING.md","kind":"file","size":12093,"lastCommitHash":"f40a7bf69ba1ac1c07608fd675606fa0e090efc3","lastCommitWhen":"2026-08-29T23:28:05-07:00","lastCommitMessage":"Update CONTRIBUTING.md"},{"name":"convert_hf_to_gguf_update.py","kind":"file","size":28225,"lastCommitHash":"1f66c3ce1c26c95db3fadb734086c7d9fba23bb9","lastCommitWhen":"2026-07-22T09:54:08+08:00","lastCommitMessage":"Add support for Laguna XS.2 & M.1 (#25165)"},{"name":"convert_hf_to_gguf.py","kind":"file","size":12798,"lastCommitHash":"3581ba0cf591b3f772fbb002de0f70e294bc0396","lastCommitWhen":"2026-08-02T23:16:31+08:00","lastCommitMessage":"convert: add option to create separate dspark GGUF (#26452)\n\n* convert: add option to create separate dspark GGUF\n\n* add --no-nextn\n\n* fix convert bug"},{"name":"convert_llama_ggml_to_gguf.py","kind":"file","size":19112,"lastCommitHash":"29b28a98242692e352df38254c9a2bb9fa79c884","lastCommitWhen":"2026-03-21T08:54:34+01:00","lastCommitMessage":"ci : switch from pyright to ty (#20826)\n\n* type fixes\n\n* switch to ty\n\n* tweak rules\n\n* tweak more rules\n\n* more tweaks\n\n* final tweak\n\n* use common import-not-found rule"},{"name":"convert_lora_to_gguf.py","kind":"file","size":23200,"lastCommitHash":"dd4623a74f0c85e6b1dd9ee99a92b9c67cac3708","lastCommitWhen":"2026-06-15T00:55:26+02:00","lastCommitMessage":"convert : fix lora base model arch retrieval (#24621)"},{"name":"flake.nix","kind":"file","size":7243,"lastCommitHash":"1adc9812bd33dc85489bf093528d61c22917d54f","lastCommitWhen":"2025-08-13T11:21:31-07:00","lastCommitMessage":"fix(nix): remove non-functional llama-cpp cachix cache from flake.nix (#15295)\n\nThe flake.nix included references to llama-cpp.cachix.org cache with a comment\nclaiming it's 'Populated by the CI in ggml-org/llama.cpp', but:\n\n1. No visible CI workflow populates this cache\n2. The cache is empty for recent builds (tested b6150, etc.)\n3. This misleads users into expecting pre-built binaries that don't exist\n\nThis change removes the non-functional cache references entirely, leaving only\nthe working cuda-maintainers cache that actually provides CUDA dependencies.\n\nUsers can still manually add the llama-cpp cache if it becomes functional in the future."},{"name":"LICENSE","kind":"file","size":1078,"lastCommitHash":"7a4ca3cbd905907ac4d12bc14b878fdfbe4fd1d6","lastCommitWhen":"2026-02-02T08:38:55+02:00","lastCommitMessage":"docs : Minor cleanups (#19252)\n\n* Update old URLs to github.com/ggml-org/\n\n* Bump copyrights"},{"name":"Makefile","kind":"file","size":257,"lastCommitHash":"37f10f955f70e0158d50343d0b9a3f92d194daae","lastCommitWhen":"2025-08-20T13:31:16+03:00","lastCommitMessage":"make : remove make in favor of CMake (#15449)\n\nThis commit removes the content from the Makefile and updates the\ncurrent deprecation message to information that `make` has been\nreplaced by CMake instead.\n\nThe message when `make` is invoked will now be the following:\n```console\n$ make\nMakefile:6: *** Build system changed:\n The Makefile build has been replaced by CMake.\n\n For build instructions see:\n https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md\n\n. Stop.\n```\n\nThe motivation for this is that many, if not all targets fail to build\nnow, after changes to the system, and `make` has also been deprected for\nsome time now."},{"name":"mypy.ini","kind":"file","size":163},{"name":"pyproject.toml","kind":"file","size":1908,"lastCommitHash":"4fb16eccce5e451b40014355f97374d692480a4d","lastCommitWhen":"2026-06-02T22:11:12+03:00","lastCommitMessage":"model: add Mellum architecture (#23966)\n\n* model: support for Mellum architecture\n\n* model: improve mellum.py formatting\n\n* model: improve mellum.py formatting once again\n\n* deps: downgrade transformers to 4.57.6 (to fix CI)\n\n* deps: remove huggingface_hub dependency\n\n* deps: remove huggingface_hub from test requirements\n\n---------\n\nCo-authored-by: Sigbjørn Skjæret "},{"name":"pyrightconfig.json","kind":"file","size":696,"lastCommitHash":"29b28a98242692e352df38254c9a2bb9fa79c884","lastCommitWhen":"2026-03-21T08:54:34+01:00","lastCommitMessage":"ci : switch from pyright to ty (#20826)\n\n* type fixes\n\n* switch to ty\n\n* tweak rules\n\n* tweak more rules\n\n* more tweaks\n\n* final tweak\n\n* use common import-not-found rule"},{"name":"README.md","kind":"file","size":8699,"lastCommitHash":"8906fd8c9fd9d2dcbffad469432b8d2edf0925fd","lastCommitWhen":"2026-10-18T08:02:45-04:00","lastCommitMessage":"cache: conversations are stored against their learned prefix\n"},{"name":"requirements.txt","kind":"file","size":551,"lastCommitHash":"669912d9a5bf927312c553332ff997f0a99da8fb","lastCommitWhen":"2025-03-05T13:05:13Z","lastCommitMessage":"`tool-call`: fix Qwen 2.5 Coder support, add micro benchmarks, support trigger patterns for lazy grammars (#12034)\n\n* sampler: turn lazy grammar trigger words to regexes\n\n* add scripts/tool_bench.sh & .py\n\n* constrain llama json output regardless of function name if matches at beginning\n\n* update relaxed newline space rule in grammar tests\n\n* support add_generation_prompt query parameter (useful for /apply_template)\n\n* Update src/llama-grammar.cpp\n\nCo-authored-by: Georgi Gerganov \n\n---------\n\nCo-authored-by: Georgi Gerganov "},{"name":"SECURITY.md","kind":"file","size":7505,"lastCommitHash":"9303cdd8d30ae0d37a517366771e0e162d525f95","lastCommitWhen":"2026-08-05T13:27:06+02:00","lastCommitMessage":"security : clarify about AI-generated reports (#26579)\n\n* security : clarify about AI-generated reports\n\n* nits\n\n* nits 2"},{"name":"ty.toml","kind":"file","size":643,"lastCommitHash":"a8681a0ed2e3f2a7452e642639671bcec20b865c","lastCommitWhen":"2026-05-20T17:37:10+02:00","lastCommitMessage":"mtmd : DeepSeek-OCR image processing fixes, img_tool::resize padding refactor (#23345)\n\n* mtmd : deepseek-ocr fixes, improvements and refactoring\n\n- image processing changes to achieve full parity with Pillow (reference impl)\n- SAM mask casting only when flash-attn is on\n- SAM refactor (build_sam() extracted so deepseek-ocr-2 can reuse it)\n- llama-chat changes to fix server/WebUI issue (new media_markers_first())\n- adapted test-chat-template and added test cases for deepseek-ocr\n- changed regression test for deepseek-ocr to use CER+chrF scores for ground-truth comparison; removed embedding-model\n- ty.toml ignore unresolved-import for tools/mtmd/tests/**\n\n* image-text reordering fix removed\n\n* refactor bool add_padding + pad_rounding enum into a single pad_style enum"}],"branches":[{"name":"main","hash":"8906fd8c9fd9d2dcbffad469432b8d2edf0925fd","when":"2026-10-18T08:02:45-04:00","isDefault":true,"message":"cache: conversations are stored against their learned prefix\n"},{"name":"bolt","hash":"87e1b429607ae85055c50d7dc94af0a05082d060","when":"2026-10-16T01:34:35-04:00","isDefault":false,"message":"moe-cache: LLAMA_MOE_CACHE_MAP/LLAMA_MOE_CACHE_SPLIT place a layer cache on another GPU; moe-trace tools for routing traces\n"}],"totalBranches":2,"tags":[],"totalTags":0,"languages":[{"name":"C++","size":20524263,"share":62.89357546865747,"percentage":63},{"name":"C","size":4610427,"share":14.127973241584169,"percentage":14},{"name":"Python","size":2698388,"share":8.268811860465815,"percentage":8},{"name":"TypeScript","size":1558034,"share":4.774365294468029,"percentage":5},{"name":"GLSL","size":871302,"share":2.669976412453504,"percentage":3},{"name":"Svelte","size":821416,"share":2.5171081264726896,"percentage":3},{"name":"HTML","size":775814,"share":2.377367526358487,"percentage":2},{"name":"Shell","size":288737,"share":0.8847919313884133,"percentage":1},{"name":"CMake","size":278065,"share":0.8520891621147244,"percentage":1},{"name":"Wolfram","size":113401,"share":0.34750063140982096,"percentage":0},{"name":"Nix","size":36108,"share":0.11064763801858725,"percentage":0},{"name":"CSS","size":23371,"share":0.07161698094971759,"percentage":0},{"name":"JavaScript","size":23163,"share":0.07097959564153475,"percentage":0},{"name":"PowerShell","size":9359,"share":0.028679274515784815,"percentage":0},{"name":"Batch File","size":802,"share":0.0024576106594357753,"percentage":0},{"name":"Sass","size":415,"share":0.0012717062639225024,"percentage":0},{"name":"Makefile","size":257,"share":0.0007875385778989953,"percentage":0}],"failed":{"files":false,"refs":false,"languages":false},"compact":{".pi":"gg","tools/ui/static":"recommended-mcp","examples/llama.android/app/src":"main","examples/llama.swiftui/llama.swiftui/Resources":"models","examples/llama.android/lib/src/androidTest":"java","examples/llama.android/lib/src/test":"java","examples/llama.android/app/src/main/java":"com","examples/llama.android/lib/src/androidTest/java":"android","examples/llama.android/lib/src/main/java":"com","examples/llama.android/lib/src/test/java":"android","tools/ui/src/routes/(chat)/chat":"[id]","examples/llama.android/app/src/main/java/com":"example","examples/llama.android/lib/src/androidTest/java/android":"llama","examples/llama.android/lib/src/main/java/com":"arm","examples/llama.android/lib/src/test/java/android":"llama","examples/llama.android/app/src/main/java/com/example":"llama","examples/llama.android/lib/src/androidTest/java/android/llama":"cpp","examples/llama.android/lib/src/main/java/com/arm":"aichat","examples/llama.android/lib/src/test/java/android/llama":"cpp"}}