[{"name":"jinja","kind":"directory","size":0,"lastCommitHash":"d9f918d2d06079b4336688e819eee821c8a9cd9e","lastCommitWhen":"2026-08-22T16:28:28+02:00","lastCommitMessage":"common: add json.h abstraction (#27511)\n\n* add common/json\n\n* migrate common\n\n* adapt jinja\n\n* migrate server\n\n* big wip\n\n* migrate tests\n\n* wip\n\n* revert some excessive changes\n\n* wip\n\n* wip 2\n\n* revert redundant changes\n\n* fix server crash\n\n* various fixes\n\n* fix ci\n\n* harden a bit\n\n* clean up\n\n* rm json-shim\n\n* add some comments\n\n* rm redundant decl"},{"name":"arg.cpp","kind":"file","size":224066,"lastCommitHash":"ad2d86b8dc2153be655bbf209e8f85afa3b2908c","lastCommitWhen":"2026-10-13T19:13:57-04:00","lastCommitMessage":"bonsai: two-slot owed-time fairness scheduler (fairness)\n\nTwo-slot fairness: while a prompt is pending beside another active request, each server step is either a decode step or one prompt chunk (cut at the solo ubatch boundaries), picked by owed GPU time, so chat keeps ~22 t/s during a long prefill instead of ~1. No contention runs the old path unchanged. --fair-chunk (default n_ubatch, 0 = off), --fair-decode-weight (default 1).\n"},{"name":"arg.h","kind":"file","size":6316,"lastCommitHash":"ee4c505a4fb37be8ea37a78af272e74dad2835c1","lastCommitWhen":"2026-08-19T11:04:26+02:00","lastCommitMessage":"server: add dedup-cache-models preset option (#27346)"},{"name":"base64.hpp","kind":"file","size":12878},{"name":"build-info.cpp.in","kind":"file","size":944,"lastCommitHash":"680a9ae63d60d35c21a0dcd7d3fabdb9c6bfc963","lastCommitWhen":"2026-08-12T14:15:03+02:00","lastCommitMessage":"cmake : introduce semantic versioning (#26839)\n\n* cmake : introduce semantic versioning (wip)\n\nThis commit introduces semantic versioning to llama.cpp.\n\n* squash! cmake : introduce semantic versioning (wip)\n\n* cmake : update test-cmake README notes [no ci]\n\n* include libmtmd in output so show its semversioned\n\n* ci : add make-release workflow\n\n* ci : fix build number check in build-cmake-pkg.yml\n\n* examples : remove trailing whitespace\n\n* ci : abort if upstream ggml version does not exist\n\n* ci : extract step contents into scripts\n\n* ci : add GGML_NATIVE=OFF to ubuntu job\n\n* examples : remove CI build information from test-cmake [no ci]\n\nThis commit removes the nightly/release information that I added\npreviously to keep this focused only on using building and installing\nllama.cpp with cmake and being able to quickly verify changes or\ntroubleshoot issues.\n\n* ci : merge scripts into single script\n\n* remove -dev-build_number support\n\nThis commit removes the incremental build number (versioning) support\nthat I added. This was incorrect and we should only use the semver for\nthe version. Releases will be tag a nightly build and package\nmaintainers/managers that build from source can use the tag and it is\ntherefor important that the correct version is reported. So a\nnightly-build will report the semver without the build number. The build\nnumber and commit as availble via cmake and test-cmake has been updated\nto include an example of using them:\n```console\n$ ./build.sh\n[test-cmake] version: 0.1.0, build: 10360 (08c69e381)\n...\n```\n\nRefs: https://github.com/ggml-org/llama.cpp/pull/26839#discussion_r3755836969\n\n* docs: add initial release.md documentation\n\n* cmake : clean-up and add LLAMA_BUILD_IS_DEV option\n\n* ci : remove version input from make-release job\n\n* ci : add LLAMA_BUILD_IS_DEV=OFF to build-cmake-pkg.yml\n\nRefs: https://github.com/danbev/llama.cpp/actions/runs/31576801921/job/94050639145\n\n* docs : update release notes with LLAMA_BUILD_IS_DEV info [no ci]\n\n* ci : add TODO to winget workflow [no ci]\n\n---------\n\nCo-authored-by: Georgi Gerganov "},{"name":"build-info.h","kind":"file","size":234,"lastCommitHash":"680a9ae63d60d35c21a0dcd7d3fabdb9c6bfc963","lastCommitWhen":"2026-08-12T14:15:03+02:00","lastCommitMessage":"cmake : introduce semantic versioning (#26839)\n\n* cmake : introduce semantic versioning (wip)\n\nThis commit introduces semantic versioning to llama.cpp.\n\n* squash! cmake : introduce semantic versioning (wip)\n\n* cmake : update test-cmake README notes [no ci]\n\n* include libmtmd in output so show its semversioned\n\n* ci : add make-release workflow\n\n* ci : fix build number check in build-cmake-pkg.yml\n\n* examples : remove trailing whitespace\n\n* ci : abort if upstream ggml version does not exist\n\n* ci : extract step contents into scripts\n\n* ci : add GGML_NATIVE=OFF to ubuntu job\n\n* examples : remove CI build information from test-cmake [no ci]\n\nThis commit removes the nightly/release information that I added\npreviously to keep this focused only on using building and installing\nllama.cpp with cmake and being able to quickly verify changes or\ntroubleshoot issues.\n\n* ci : merge scripts into single script\n\n* remove -dev-build_number support\n\nThis commit removes the incremental build number (versioning) support\nthat I added. This was incorrect and we should only use the semver for\nthe version. Releases will be tag a nightly build and package\nmaintainers/managers that build from source can use the tag and it is\ntherefor important that the correct version is reported. So a\nnightly-build will report the semver without the build number. The build\nnumber and commit as availble via cmake and test-cmake has been updated\nto include an example of using them:\n```console\n$ ./build.sh\n[test-cmake] version: 0.1.0, build: 10360 (08c69e381)\n...\n```\n\nRefs: https://github.com/ggml-org/llama.cpp/pull/26839#discussion_r3755836969\n\n* docs: add initial release.md documentation\n\n* cmake : clean-up and add LLAMA_BUILD_IS_DEV option\n\n* ci : remove version input from make-release job\n\n* ci : add LLAMA_BUILD_IS_DEV=OFF to build-cmake-pkg.yml\n\nRefs: https://github.com/danbev/llama.cpp/actions/runs/31576801921/job/94050639145\n\n* docs : update release notes with LLAMA_BUILD_IS_DEV info [no ci]\n\n* ci : add TODO to winget workflow [no ci]\n\n---------\n\nCo-authored-by: Georgi Gerganov "},{"name":"chat-auto-parser-generator.cpp","kind":"file","size":23962,"lastCommitHash":"d9f918d2d06079b4336688e819eee821c8a9cd9e","lastCommitWhen":"2026-08-22T16:28:28+02:00","lastCommitMessage":"common: add json.h abstraction (#27511)\n\n* add common/json\n\n* migrate common\n\n* adapt jinja\n\n* migrate server\n\n* big wip\n\n* migrate tests\n\n* wip\n\n* revert some excessive changes\n\n* wip\n\n* wip 2\n\n* revert redundant changes\n\n* fix server crash\n\n* various fixes\n\n* fix ci\n\n* harden a bit\n\n* clean up\n\n* rm json-shim\n\n* add some comments\n\n* rm redundant decl"},{"name":"chat-auto-parser-helpers.cpp","kind":"file","size":12939,"lastCommitHash":"d9f918d2d06079b4336688e819eee821c8a9cd9e","lastCommitWhen":"2026-08-22T16:28:28+02:00","lastCommitMessage":"common: add json.h abstraction (#27511)\n\n* add common/json\n\n* migrate common\n\n* adapt jinja\n\n* migrate server\n\n* big wip\n\n* migrate tests\n\n* wip\n\n* revert some excessive changes\n\n* wip\n\n* wip 2\n\n* revert redundant changes\n\n* fix server crash\n\n* various fixes\n\n* fix ci\n\n* harden a bit\n\n* clean up\n\n* rm json-shim\n\n* add some comments\n\n* rm redundant decl"},{"name":"chat-auto-parser-helpers.h","kind":"file","size":4518,"lastCommitHash":"57ace0d612a11133ac86edcc7af1b323bf05f12f","lastCommitWhen":"2026-04-03T09:07:59+03:00","lastCommitMessage":"chat : avoid including json in chat.h (#21306)"},{"name":"chat-auto-parser.h","kind":"file","size":18435,"lastCommitHash":"d9f918d2d06079b4336688e819eee821c8a9cd9e","lastCommitWhen":"2026-08-22T16:28:28+02:00","lastCommitMessage":"common: add json.h abstraction (#27511)\n\n* add common/json\n\n* migrate common\n\n* adapt jinja\n\n* migrate server\n\n* big wip\n\n* migrate tests\n\n* wip\n\n* revert some excessive changes\n\n* wip\n\n* wip 2\n\n* revert redundant changes\n\n* fix server crash\n\n* various fixes\n\n* fix ci\n\n* harden a bit\n\n* clean up\n\n* rm json-shim\n\n* add some comments\n\n* rm redundant decl"},{"name":"chat-diff-analyzer.cpp","kind":"file","size":76321,"lastCommitHash":"d9f918d2d06079b4336688e819eee821c8a9cd9e","lastCommitWhen":"2026-08-22T16:28:28+02:00","lastCommitMessage":"common: add json.h abstraction (#27511)\n\n* add common/json\n\n* migrate common\n\n* adapt jinja\n\n* migrate server\n\n* big wip\n\n* migrate tests\n\n* wip\n\n* revert some excessive changes\n\n* wip\n\n* wip 2\n\n* revert redundant changes\n\n* fix server crash\n\n* various fixes\n\n* fix ci\n\n* harden a bit\n\n* clean up\n\n* rm json-shim\n\n* add some comments\n\n* rm redundant decl"},{"name":"chat-peg-parser.cpp","kind":"file","size":48089,"lastCommitHash":"d9f918d2d06079b4336688e819eee821c8a9cd9e","lastCommitWhen":"2026-08-22T16:28:28+02:00","lastCommitMessage":"common: add json.h abstraction (#27511)\n\n* add common/json\n\n* migrate common\n\n* adapt jinja\n\n* migrate server\n\n* big wip\n\n* migrate tests\n\n* wip\n\n* revert some excessive changes\n\n* wip\n\n* wip 2\n\n* revert redundant changes\n\n* fix server crash\n\n* various fixes\n\n* fix ci\n\n* harden a bit\n\n* clean up\n\n* rm json-shim\n\n* add some comments\n\n* rm redundant decl"},{"name":"chat-peg-parser.h","kind":"file","size":11904,"lastCommitHash":"d9f918d2d06079b4336688e819eee821c8a9cd9e","lastCommitWhen":"2026-08-22T16:28:28+02:00","lastCommitMessage":"common: add json.h abstraction (#27511)\n\n* add common/json\n\n* migrate common\n\n* adapt jinja\n\n* migrate server\n\n* big wip\n\n* migrate tests\n\n* wip\n\n* revert some excessive changes\n\n* wip\n\n* wip 2\n\n* revert redundant changes\n\n* fix server crash\n\n* various fixes\n\n* fix ci\n\n* harden a bit\n\n* clean up\n\n* rm json-shim\n\n* add some comments\n\n* rm redundant decl"},{"name":"chat.cpp","kind":"file","size":174373,"lastCommitHash":"d9f918d2d06079b4336688e819eee821c8a9cd9e","lastCommitWhen":"2026-08-22T16:28:28+02:00","lastCommitMessage":"common: add json.h abstraction (#27511)\n\n* add common/json\n\n* migrate common\n\n* adapt jinja\n\n* migrate server\n\n* big wip\n\n* migrate tests\n\n* wip\n\n* revert some excessive changes\n\n* wip\n\n* wip 2\n\n* revert redundant changes\n\n* fix server crash\n\n* various fixes\n\n* fix ci\n\n* harden a bit\n\n* clean up\n\n* rm json-shim\n\n* add some comments\n\n* rm redundant decl"},{"name":"chat.h","kind":"file","size":15550,"lastCommitHash":"d9f918d2d06079b4336688e819eee821c8a9cd9e","lastCommitWhen":"2026-08-22T16:28:28+02:00","lastCommitMessage":"common: add json.h abstraction (#27511)\n\n* add common/json\n\n* migrate common\n\n* adapt jinja\n\n* migrate server\n\n* big wip\n\n* migrate tests\n\n* wip\n\n* revert some excessive changes\n\n* wip\n\n* wip 2\n\n* revert redundant changes\n\n* fix server crash\n\n* various fixes\n\n* fix ci\n\n* harden a bit\n\n* clean up\n\n* rm json-shim\n\n* add some comments\n\n* rm redundant decl"},{"name":"CMakeLists.txt","kind":"file","size":7140,"lastCommitHash":"904cb4ae461799612c62c826737bb23ddc3ec7fb","lastCommitWhen":"2026-10-13T19:13:57-04:00","lastCommitMessage":"bonsai: suffix cache reuse (SCR) for linear attention hybrids (scr)\n\nSuffix cache reuse: after an edit in the middle of a cached prompt (compaction, a trimmed tool output, a system prompt change), the attention KV of the text after the edit is moved to its new positions (q8_0 keys re-rotated) and the DeltaNet layers continue from the pre-edit state, so only the new text is prefilled. Approximate (KL ~0.017 nats/token after a compaction); per request \"scr\", server default --scr.\n"},{"name":"common.cpp","kind":"file","size":82167,"lastCommitHash":"03c09b184854049a99ef034054b19c225ea3196b","lastCommitWhen":"2026-10-13T19:13:55-04:00","lastCommitMessage":"bonsai: optimize time-to-first-token on cached prompts (ttft)\n\nTime to first token on cached prompts: no scheduler reserve per request with backend sampling, no rewrite of a just-restored checkpoint, recycled checkpoint buffers, no tiny batch split for a nearby last user message.\n"},{"name":"common.h","kind":"file","size":52032,"lastCommitHash":"ad2d86b8dc2153be655bbf209e8f85afa3b2908c","lastCommitWhen":"2026-10-13T19:13:57-04:00","lastCommitMessage":"bonsai: two-slot owed-time fairness scheduler (fairness)\n\nTwo-slot fairness: while a prompt is pending beside another active request, each server step is either a decode step or one prompt chunk (cut at the solo ubatch boundaries), picked by owed GPU time, so chat keeps ~22 t/s during a long prefill instead of ~1. No contention runs the old path unchanged. --fair-chunk (default n_ubatch, 0 = off), --fair-decode-weight (default 1).\n"},{"name":"console.cpp","kind":"file","size":41627,"lastCommitHash":"506200cf8b5c8419ce97d16dc8c50f4634e21ebe","lastCommitWhen":"2026-04-06T20:54:06+02:00","lastCommitMessage":"cli: fix stripping of \\n in multiline input (#21485)\n\n* llama-cli: fix stripping of \\n in multiline input\n\n* Change & string to string_view\n\n* Apply suggestions from code review\n\nCo-authored-by: Sigbjørn Skjæret \n\n* Fix EditorConfig linter error\n\n---------\n\nCo-authored-by: Sigbjørn Skjæret "},{"name":"console.h","kind":"file","size":1170,"lastCommitHash":"b5ed0e058c98645c1cab752d012fdc4be22bceef","lastCommitWhen":"2026-03-05T10:47:28+01:00","lastCommitMessage":"cli : add command and file auto-completion (#19985)\n\n"},{"name":"debug.cpp","kind":"file","size":6538,"lastCommitHash":"5594d132244aeb1bae54dd431e7efbc908f5e3b8","lastCommitWhen":"2026-04-27T08:06:39+03:00","lastCommitMessage":"common: fix missing exports in llama-common (#22340)\n\n* common: refactor common/debug to move abort_on_nan into base_callback_data\n\nPassing bool abort_on_nan as template parameter for common_debug_cb_eval is unnecessary and creates an issue with LTO.\nIt should just be a member of the base_callback_data instead.\n\n* cont : cleanup\n\n* common : use pimpl in debug.h to reduce header dependencies\n\nMove common_debug_cb_user_data's data members (std::regex,\nstd::vector) into a private impl struct in debug.cpp.\n\nThis removes the includes of common.h and from debug.h,\nreducing transitive dependencies for any translation unit that\nincludes the header.\n\nAssisted-by: llama.cpp:local pi\n\n---------\n\nCo-authored-by: Georgi Gerganov "},{"name":"debug.h","kind":"file","size":1275,"lastCommitHash":"5594d132244aeb1bae54dd431e7efbc908f5e3b8","lastCommitWhen":"2026-04-27T08:06:39+03:00","lastCommitMessage":"common: fix missing exports in llama-common (#22340)\n\n* common: refactor common/debug to move abort_on_nan into base_callback_data\n\nPassing bool abort_on_nan as template parameter for common_debug_cb_eval is unnecessary and creates an issue with LTO.\nIt should just be a member of the base_callback_data instead.\n\n* cont : cleanup\n\n* common : use pimpl in debug.h to reduce header dependencies\n\nMove common_debug_cb_user_data's data members (std::regex,\nstd::vector) into a private impl struct in debug.cpp.\n\nThis removes the includes of common.h and from debug.h,\nreducing transitive dependencies for any translation unit that\nincludes the header.\n\nAssisted-by: llama.cpp:local pi\n\n---------\n\nCo-authored-by: Georgi Gerganov "},{"name":"download.cpp","kind":"file","size":36561,"lastCommitHash":"d9f918d2d06079b4336688e819eee821c8a9cd9e","lastCommitWhen":"2026-08-22T16:28:28+02:00","lastCommitMessage":"common: add json.h abstraction (#27511)\n\n* add common/json\n\n* migrate common\n\n* adapt jinja\n\n* migrate server\n\n* big wip\n\n* migrate tests\n\n* wip\n\n* revert some excessive changes\n\n* wip\n\n* wip 2\n\n* revert redundant changes\n\n* fix server crash\n\n* various fixes\n\n* fix ci\n\n* harden a bit\n\n* clean up\n\n* rm json-shim\n\n* add some comments\n\n* rm redundant decl"},{"name":"download.h","kind":"file","size":4447,"lastCommitHash":"ee4c505a4fb37be8ea37a78af272e74dad2835c1","lastCommitWhen":"2026-08-19T11:04:26+02:00","lastCommitMessage":"server: add dedup-cache-models preset option (#27346)"},{"name":"dspark-markov-metal.h","kind":"file","size":1051,"lastCommitHash":"6067dbea986ed1359db98c42ab06eb49426bbf16","lastCommitWhen":"2026-09-17T21:25:55-07:00","lastCommitMessage":"dspark: add corrected draft runtime and shared packed target head\n"},{"name":"dspark-markov-metal.mm","kind":"file","size":11279,"lastCommitHash":"6067dbea986ed1359db98c42ab06eb49426bbf16","lastCommitWhen":"2026-09-17T21:25:55-07:00","lastCommitMessage":"dspark: add corrected draft runtime and shared packed target head\n"},{"name":"dspark-markov.cu","kind":"file","size":18270,"lastCommitHash":"08ac1fd17ef176abaab6287981b515fbc5582210","lastCommitWhen":"2026-09-17T22:59:12-07:00","lastCommitMessage":"dspark: validate mask selection and graph reuse contracts\n"},{"name":"dspark-markov.h","kind":"file","size":3232,"lastCommitHash":"08ac1fd17ef176abaab6287981b515fbc5582210","lastCommitWhen":"2026-09-17T22:59:12-07:00","lastCommitMessage":"dspark: validate mask selection and graph reuse contracts\n"},{"name":"fit.cpp","kind":"file","size":50850,"lastCommitHash":"2fb989b9e79bf4da8159855e24892c8f4c20300f","lastCommitWhen":"2026-08-22T16:16:06+02:00","lastCommitMessage":"fit: also take into account n_streams (#27496)\n\n* fit: also take into account n_streams\n\n* server: make the draft context follow the target context\n\nWith a non-unified KV cache the target context now holds n_ctx_train\ntokens per sequence, while the draft context was still created with\nn_ctx = 0 and fell back to n_ctx_train / n_streams per sequence. A slot\nfilled beyond that point makes the draft batch fail to decode, and the\nserver answers 500 on the request.\n\nThe draft context now takes its size from the target context, so both\nhold the same number of tokens per sequence. Contexts that share their\ncells with the target no longer need the kv_size override.\n\nThe memory reserved for the draft model before fitting is measured at\nthe largest context the target can take, since the draft context grows\nwith the target and a fixed byte margin cannot express that.\n\n* fit: take an optional second model into account\n\nIllustrates the alternative discussed on the draft context fix. The\nmemory of a draft or MTP context is currently handed to the fit as a\nfixed byte margin, which cannot express a memory that grows with the\ncontext the fit is still deciding on.\n\ncommon_fit_params now takes an optional second model that shares the\ndevices of the main one. Its context follows the main context and its\nmemory is measured again whenever that context changes, so the reduce\npath stays exact instead of conservative. A model that cannot be\nmeasured on its own, such as a shared cell MTP context, is skipped with\na warning and the main model is fitted alone.\n\nThis drops the reservation block in the server, which no longer has to\nprobe the trained context size of the target to guess an upper bound.\n\n---------\n\nCo-authored-by: Pascal "},{"name":"fit.h","kind":"file","size":3339,"lastCommitHash":"2fb989b9e79bf4da8159855e24892c8f4c20300f","lastCommitWhen":"2026-08-22T16:16:06+02:00","lastCommitMessage":"fit: also take into account n_streams (#27496)\n\n* fit: also take into account n_streams\n\n* server: make the draft context follow the target context\n\nWith a non-unified KV cache the target context now holds n_ctx_train\ntokens per sequence, while the draft context was still created with\nn_ctx = 0 and fell back to n_ctx_train / n_streams per sequence. A slot\nfilled beyond that point makes the draft batch fail to decode, and the\nserver answers 500 on the request.\n\nThe draft context now takes its size from the target context, so both\nhold the same number of tokens per sequence. Contexts that share their\ncells with the target no longer need the kv_size override.\n\nThe memory reserved for the draft model before fitting is measured at\nthe largest context the target can take, since the draft context grows\nwith the target and a fixed byte margin cannot express that.\n\n* fit: take an optional second model into account\n\nIllustrates the alternative discussed on the draft context fix. The\nmemory of a draft or MTP context is currently handed to the fit as a\nfixed byte margin, which cannot express a memory that grows with the\ncontext the fit is still deciding on.\n\ncommon_fit_params now takes an optional second model that shares the\ndevices of the main one. Its context follows the main context and its\nmemory is measured again whenever that context changes, so the reduce\npath stays exact instead of conservative. A model that cannot be\nmeasured on its own, such as a shared cell MTP context, is skipped with\na warning and the main model is fitted alone.\n\nThis drops the reservation block in the server, which no longer has to\nprobe the trained context size of the target to guess an upper bound.\n\n---------\n\nCo-authored-by: Pascal "},{"name":"hf-cache.cpp","kind":"file","size":15487,"lastCommitHash":"d9f918d2d06079b4336688e819eee821c8a9cd9e","lastCommitWhen":"2026-08-22T16:28:28+02:00","lastCommitMessage":"common: add json.h abstraction (#27511)\n\n* add common/json\n\n* migrate common\n\n* adapt jinja\n\n* migrate server\n\n* big wip\n\n* migrate tests\n\n* wip\n\n* revert some excessive changes\n\n* wip\n\n* wip 2\n\n* revert redundant changes\n\n* fix server crash\n\n* various fixes\n\n* fix ci\n\n* harden a bit\n\n* clean up\n\n* rm json-shim\n\n* add some comments\n\n* rm redundant decl"},{"name":"hf-cache.h","kind":"file","size":775,"lastCommitHash":"4b4d13ae721e5eb79b749ca2c6feefd157f90ed7","lastCommitWhen":"2026-06-17T18:04:58+02:00","lastCommitMessage":"server: (router) add model management API (#23976)\n\n* wip\n\n* server: (router) add SSE realtime updates API\n\n* nits\n\n* wip\n\n* add download API\n\n* add download api\n\n* update docs\n\n* add delete endpoint\n\n* fix std::terminate\n\n* fix crash\n\n* fix 2\n\n* add tests\n\n* nits"},{"name":"http.h","kind":"file","size":5347,"lastCommitHash":"c264f65ff9d8f592a590e3221f712a8883b7dd81","lastCommitWhen":"2026-07-08T14:52:43+02:00","lastCommitMessage":"cli : move to HTTP-based implementation (#24948)\n\n* cli: move to HTTP-based implementation\n\n* wip\n\n* working\n\n* remote server ok\n\n* cli support router mode\n\nCo-authored-by: Piotr Wilkin \n\n* case: router with only one model\n\n* Apply suggestions from code review\n\nCo-authored-by: Piotr Wilkin (ilintar) \n\n* remove outdated comment\n\n* use destructor instead\n\n* add ftype\n\n* cli-view --> cli-ui\n\n* pimpl\n\n* no more json in header\n\n* nits fixes\n\n* also show model aliases\n\n---------\n\nCo-authored-by: Piotr Wilkin \nCo-authored-by: Piotr Wilkin (ilintar) "},{"name":"imatrix-loader.cpp","kind":"file","size":5966,"lastCommitHash":"27df9199d16c393868d20b3ed28b4f72e30de6bd","lastCommitWhen":"2026-08-15T11:45:30+02:00","lastCommitMessage":"fix: check gguf array type before reading (#27075)\n\n* fix: check gguf array type before reading\n\n* update skill"},{"name":"imatrix-loader.h","kind":"file","size":765,"lastCommitHash":"e7bcf1c3a8513429c913a619156eb4c905914e87","lastCommitWhen":"2026-06-04T17:45:40+02:00","lastCommitMessage":"Move duplicated imatrix code into single common imatrix-loader.cpp (#22445)\n\n* Deduplicate imatrix loading code\n\n* Add back LLAMA_TRACE, early exit on quantize missing metadata"},{"name":"json-schema-to-grammar.cpp","kind":"file","size":52894,"lastCommitHash":"d9f918d2d06079b4336688e819eee821c8a9cd9e","lastCommitWhen":"2026-08-22T16:28:28+02:00","lastCommitMessage":"common: add json.h abstraction (#27511)\n\n* add common/json\n\n* migrate common\n\n* adapt jinja\n\n* migrate server\n\n* big wip\n\n* migrate tests\n\n* wip\n\n* revert some excessive changes\n\n* wip\n\n* wip 2\n\n* revert redundant changes\n\n* fix server crash\n\n* various fixes\n\n* fix ci\n\n* harden a bit\n\n* clean up\n\n* rm json-shim\n\n* add some comments\n\n* rm redundant decl"},{"name":"json-schema-to-grammar.h","kind":"file","size":1364,"lastCommitHash":"d9f918d2d06079b4336688e819eee821c8a9cd9e","lastCommitWhen":"2026-08-22T16:28:28+02:00","lastCommitMessage":"common: add json.h abstraction (#27511)\n\n* add common/json\n\n* migrate common\n\n* adapt jinja\n\n* migrate server\n\n* big wip\n\n* migrate tests\n\n* wip\n\n* revert some excessive changes\n\n* wip\n\n* wip 2\n\n* revert redundant changes\n\n* fix server crash\n\n* various fixes\n\n* fix ci\n\n* harden a bit\n\n* clean up\n\n* rm json-shim\n\n* add some comments\n\n* rm redundant decl"},{"name":"json.cpp","kind":"file","size":14613,"lastCommitHash":"70adb1b4cea5ee39f867792c78dc59320921eda7","lastCommitWhen":"2026-08-23T01:11:10+02:00","lastCommitMessage":"common: json.h: fix clang lto (#27575)"},{"name":"json.h","kind":"file","size":13257,"lastCommitHash":"70adb1b4cea5ee39f867792c78dc59320921eda7","lastCommitWhen":"2026-08-23T01:11:10+02:00","lastCommitMessage":"common: json.h: fix clang lto (#27575)"},{"name":"kv-mean-center.cpp","kind":"file","size":2045,"lastCommitHash":"36d1deef278725a43aedcdd8d68a9be7815cbbe6","lastCommitWhen":"2026-08-25T05:55:11Z","lastCommitMessage":"kv-cache: mean-centering on hybrid models + calibration-basis guard (fork #53)\n"},{"name":"kv-mean-center.h","kind":"file","size":1302,"lastCommitHash":"36d1deef278725a43aedcdd8d68a9be7815cbbe6","lastCommitWhen":"2026-08-25T05:55:11Z","lastCommitMessage":"kv-cache: mean-centering on hybrid models + calibration-basis guard (fork #53)\n"},{"name":"llguidance.cpp","kind":"file","size":8776,"lastCommitHash":"dd1ea524333b1e697489067d7a4c39c60d32beee","lastCommitWhen":"2026-08-10T16:58:56+03:00","lastCommitMessage":"llama : support multi-output backend sampling (#25532)\n\n* Enable backend sampling with token speculation\n\n* Clamp the mask sum before converting it into the sampled index\n\n* Add a numeric context parameter declaring the maximum outputs one sequence\n\n* More fixes\n\n* Don't reuse memory for output views.\n\n* Match dist between CPU and GPU\n\n* Fix CPU and backend sampling mismatches\n\n* Simpify some of the changes\n\n* Fix tests on Vulkan\n\n* More test fixes\n\n* Rebase changes\n\n* Rebase and address review comments\n\n* Address review comments\n\n* Address review comments\n\n* Update src/llama-sampler.cpp\n\nCo-authored-by: Georgi Gerganov \n\n---------\n\nCo-authored-by: Georgi Gerganov "},{"name":"log.cpp","kind":"file","size":12345,"lastCommitHash":"cda63856b804af3aae3e35c1da3eacaeb4bfeb0a","lastCommitWhen":"2026-06-17T09:19:11+03:00","lastCommitMessage":"common: update logging to enforce max_capacity and optimize queue resizing (#24490)\n\n* common: update logging to enforce max_capacity and optimize queue resizing logic\n\n* common/log: remove queue expansion logic"},{"name":"log.h","kind":"file","size":5583,"lastCommitHash":"67b2b7f2f2d6dac7962b219168a4c7a20c7359b7","lastCommitWhen":"2026-05-14T13:05:52+03:00","lastCommitMessage":"logs : reduce (#23021)\n\n* logs : reduce\n\n* args : fix envs\n\n* server : fix build\n\n* common : print verbosity level at start\n\n* server : clean-up logs\n\n* server : print prompt processing timings + sampling params\n\n* minor : whitespaces"},{"name":"ngram-cache.cpp","kind":"file","size":11652,"lastCommitHash":"72d3b1898a9c81152710cc37dd1dfd26764055d9","lastCommitWhen":"2026-01-28T19:42:42+02:00","lastCommitMessage":"spec : add self‑speculative decoding (no draft model required) + refactor (#18471)\n\n* server: introduce self-speculative decoding\n\n* server: moved self-call into speculative.cpp\n\n* can_speculate() includes self-speculation\n\nCo-authored-by: Georgi Gerganov \n\n* server: can_speculate() tests self-spec\n\n* server: replace can_speculate() with slot.can_speculate()\n\nCo-authored-by: Sigbjørn Skjæret \n\n* common: use %zu format specifier for size_t in logging\n\nCo-authored-by: Sigbjørn Skjæret \n\n* server: can_speculate() requires a task instance\n\n* common: ngram map, config self-speculative decoding\n\n* common: add enum common_speculative_type\n\n* common: add vector of speculative states\n\n* common: add option --spec-draftless\n\n* server: cleanup (remove slot.batch_spec, rename)\n\n* common: moved self-spec impl to ngram-map\n\n* common: cleanup (use common_speculative_state_draft)\n\n* spec : refactor\n\n* cont : naming\n\n* spec: remove --spec-config\n\n* doc: (draftless) speculative decoding\n\n* common: print performance in spec decoding\n\n* minor : cleanup\n\n* common : better names\n\n* minor : cleanup + fix build\n\n* minor: comments\n\n* CODEOWNERS: add common/ngram-map.* (#18471)\n\n* common : rename speculative.draftless_type -> speculative.type\n\n* ngram-map : fix uninitialized values\n\n* ngram-map : take into account the input can become shorter\n\n* ngram-map : revert len check for now\n\n* arg : change `--spec-draftless` -> `--spec-type`\n\n* spec : add common_speculative_state::accept()\n\n* spec : refactor + add common_speculative_begin()\n\n* spec : fix begin() call with mtmd\n\n* spec : additional refactor + remove common_speculative_params\n\n---------\n\nCo-authored-by: Georgi Gerganov \nCo-authored-by: Sigbjørn Skjæret "},{"name":"ngram-cache.h","kind":"file","size":4147,"lastCommitHash":"72d3b1898a9c81152710cc37dd1dfd26764055d9","lastCommitWhen":"2026-01-28T19:42:42+02:00","lastCommitMessage":"spec : add self‑speculative decoding (no draft model required) + refactor (#18471)\n\n* server: introduce self-speculative decoding\n\n* server: moved self-call into speculative.cpp\n\n* can_speculate() includes self-speculation\n\nCo-authored-by: Georgi Gerganov \n\n* server: can_speculate() tests self-spec\n\n* server: replace can_speculate() with slot.can_speculate()\n\nCo-authored-by: Sigbjørn Skjæret \n\n* common: use %zu format specifier for size_t in logging\n\nCo-authored-by: Sigbjørn Skjæret \n\n* server: can_speculate() requires a task instance\n\n* common: ngram map, config self-speculative decoding\n\n* common: add enum common_speculative_type\n\n* common: add vector of speculative states\n\n* common: add option --spec-draftless\n\n* server: cleanup (remove slot.batch_spec, rename)\n\n* common: moved self-spec impl to ngram-map\n\n* common: cleanup (use common_speculative_state_draft)\n\n* spec : refactor\n\n* cont : naming\n\n* spec: remove --spec-config\n\n* doc: (draftless) speculative decoding\n\n* common: print performance in spec decoding\n\n* minor : cleanup\n\n* common : better names\n\n* minor : cleanup + fix build\n\n* minor: comments\n\n* CODEOWNERS: add common/ngram-map.* (#18471)\n\n* common : rename speculative.draftless_type -> speculative.type\n\n* ngram-map : fix uninitialized values\n\n* ngram-map : take into account the input can become shorter\n\n* ngram-map : revert len check for now\n\n* arg : change `--spec-draftless` -> `--spec-type`\n\n* spec : add common_speculative_state::accept()\n\n* spec : refactor + add common_speculative_begin()\n\n* spec : fix begin() call with mtmd\n\n* spec : additional refactor + remove common_speculative_params\n\n---------\n\nCo-authored-by: Georgi Gerganov \nCo-authored-by: Sigbjørn Skjæret "},{"name":"ngram-map.cpp","kind":"file","size":20115,"lastCommitHash":"defa95c306598bff66640c64dc8788adf90b72ea","lastCommitWhen":"2026-07-07T10:25:04+03:00","lastCommitMessage":"speculative : fix out-of-bounds read in ngram-map on prompt shrink (#23936)\n\n* speculative : fix out-of-bounds read in ngram-map on prompt shrink\n\n* speculative : fix ngram-map cleanup cutoff after prompt shrink"},{"name":"ngram-map.h","kind":"file","size":4869,"lastCommitHash":"0b6ff47996019d559862d279ccb66dd9390934cd","lastCommitWhen":"2026-03-31T13:50:51+02:00","lastCommitMessage":"fix: correct misspellings in code comments (#21217)\n\n- emdeddings → embeddings (gemma3.cpp, gemma3n-iswa.cpp,\ngemma-embedding.cpp)\n- imlpemented → implemented (llama-adapter.cpp)\n- interere → interfere (llama-graph.cpp)\n- overridde → overridden (chat.cpp)\n- stastistics → statistics (ngram-map.h)\n- layed → laid (llama-kv-cache.h)\n- worster → worst (llama-context.cpp)\n- sequantial → sequential (llama-batch.h)"},{"name":"ngram-mod.cpp","kind":"file","size":1132,"lastCommitHash":"b000431a0bec02acc4173f5937313ab72fc476c5","lastCommitWhen":"2026-05-29T09:21:37+03:00","lastCommitMessage":"ngram-mod : Add missing include (#23857)\n\n[no release]\n\nSigned-off-by: Omid Azizi "},{"name":"ngram-mod.h","kind":"file","size":728,"lastCommitHash":"49277958109f738eb5377242e46734788ba21be8","lastCommitWhen":"2026-01-30T21:27:27+02:00","lastCommitMessage":"ngram-mod : fix build [no ci] (#19216)\n\n"},{"name":"peg-parser.cpp","kind":"file","size":83454,"lastCommitHash":"d9f918d2d06079b4336688e819eee821c8a9cd9e","lastCommitWhen":"2026-08-22T16:28:28+02:00","lastCommitMessage":"common: add json.h abstraction (#27511)\n\n* add common/json\n\n* migrate common\n\n* adapt jinja\n\n* migrate server\n\n* big wip\n\n* migrate tests\n\n* wip\n\n* revert some excessive changes\n\n* wip\n\n* wip 2\n\n* revert redundant changes\n\n* fix server crash\n\n* various fixes\n\n* fix ci\n\n* harden a bit\n\n* clean up\n\n* rm json-shim\n\n* add some comments\n\n* rm redundant decl"},{"name":"peg-parser.h","kind":"file","size":20148,"lastCommitHash":"d9f918d2d06079b4336688e819eee821c8a9cd9e","lastCommitWhen":"2026-08-22T16:28:28+02:00","lastCommitMessage":"common: add json.h abstraction (#27511)\n\n* add common/json\n\n* migrate common\n\n* adapt jinja\n\n* migrate server\n\n* big wip\n\n* migrate tests\n\n* wip\n\n* revert some excessive changes\n\n* wip\n\n* wip 2\n\n* revert redundant changes\n\n* fix server crash\n\n* various fixes\n\n* fix ci\n\n* harden a bit\n\n* clean up\n\n* rm json-shim\n\n* add some comments\n\n* rm redundant decl"},{"name":"preset.cpp","kind":"file","size":17411,"lastCommitHash":"5f754ea0e2fd21e1213db7ebebfd65d938d9d69c","lastCommitWhen":"2026-08-15T13:17:35+02:00","lastCommitMessage":"common: support --models-dir loading MTP assistant models (#24431)\n\n* common: support --models-dir loading MTP assistant models\n\n* common: preset: check for MTP models with strict prefix\n\n* common: preset: Take advantage of PR #27005\n\n* handle other draft types\n\n* drop eagle3\n\n* clean up\n\n---------\n\nCo-authored-by: Xuan Son Nguyen "},{"name":"preset.h","kind":"file","size":3223,"lastCommitHash":"8e7f22b67ef4667b4ddd50230771287f328cfb3f","lastCommitWhen":"2026-08-13T00:02:27+02:00","lastCommitMessage":"common: add system-level config file (#26118)\n\n* common: Add CLI > ENV > models-presets > INI precedence\n\n1. CLI flags have the highest precedence\n2. ENV vars have the second-highest precedence\n3. System and User configs have the lowest precedence\n - Linux/BSD/Mac\n - /etc/llama.cpp/config.ini < ${XDG_CONFIG_HOME:-~/.config}/llama.cpp/config.ini\n - Windows\n - %PROGRAMDATA%\\llama.cpp\\config.ini < %APPDATA%\\llama.cpp\\config.ini\n\n* fix UB\n\n* use common_get_env\n\n* ignore_unknown_keys\n\n* nits\n\n* add docs\n\n---------\n\nCo-authored-by: Xuan Son Nguyen "},{"name":"reasoning-budget.cpp","kind":"file","size":11078,"lastCommitHash":"dd1ea524333b1e697489067d7a4c39c60d32beee","lastCommitWhen":"2026-08-10T16:58:56+03:00","lastCommitMessage":"llama : support multi-output backend sampling (#25532)\n\n* Enable backend sampling with token speculation\n\n* Clamp the mask sum before converting it into the sampled index\n\n* Add a numeric context parameter declaring the maximum outputs one sequence\n\n* More fixes\n\n* Don't reuse memory for output views.\n\n* Match dist between CPU and GPU\n\n* Fix CPU and backend sampling mismatches\n\n* Simpify some of the changes\n\n* Fix tests on Vulkan\n\n* More test fixes\n\n* Rebase changes\n\n* Rebase and address review comments\n\n* Address review comments\n\n* Address review comments\n\n* Update src/llama-sampler.cpp\n\nCo-authored-by: Georgi Gerganov \n\n---------\n\nCo-authored-by: Georgi Gerganov "},{"name":"reasoning-budget.h","kind":"file","size":2379,"lastCommitHash":"910196f6b3dfc6aca88fa732e2b02f270ff9b56b","lastCommitWhen":"2026-07-25T11:58:09+02:00","lastCommitMessage":"common : add support for multiple end sequences in the reasoning budget sampler (#25544)\n\n* common : extract trie/ac to a separate file\n\n* common : support multiple token sequences in the reasoning budget sampler\n\n* common/trie : return matched word index\n\n* common/trie : rename \"word\" to \"pattern\"\n\n* common/reasoning-budget : expose matched end sequence\n\n* common/sampling : replay end sequence when reasoning budget is done\n\n* cont : update to use multiple end sequences\n\n* cont : clean up"},{"name":"sampling.cpp","kind":"file","size":34772,"lastCommitHash":"dd1ea524333b1e697489067d7a4c39c60d32beee","lastCommitWhen":"2026-08-10T16:58:56+03:00","lastCommitMessage":"llama : support multi-output backend sampling (#25532)\n\n* Enable backend sampling with token speculation\n\n* Clamp the mask sum before converting it into the sampled index\n\n* Add a numeric context parameter declaring the maximum outputs one sequence\n\n* More fixes\n\n* Don't reuse memory for output views.\n\n* Match dist between CPU and GPU\n\n* Fix CPU and backend sampling mismatches\n\n* Simpify some of the changes\n\n* Fix tests on Vulkan\n\n* More test fixes\n\n* Rebase changes\n\n* Rebase and address review comments\n\n* Address review comments\n\n* Address review comments\n\n* Update src/llama-sampler.cpp\n\nCo-authored-by: Georgi Gerganov \n\n---------\n\nCo-authored-by: Georgi Gerganov "},{"name":"sampling.h","kind":"file","size":5573,"lastCommitHash":"dd1ea524333b1e697489067d7a4c39c60d32beee","lastCommitWhen":"2026-08-10T16:58:56+03:00","lastCommitMessage":"llama : support multi-output backend sampling (#25532)\n\n* Enable backend sampling with token speculation\n\n* Clamp the mask sum before converting it into the sampled index\n\n* Add a numeric context parameter declaring the maximum outputs one sequence\n\n* More fixes\n\n* Don't reuse memory for output views.\n\n* Match dist between CPU and GPU\n\n* Fix CPU and backend sampling mismatches\n\n* Simpify some of the changes\n\n* Fix tests on Vulkan\n\n* More test fixes\n\n* Rebase changes\n\n* Rebase and address review comments\n\n* Address review comments\n\n* Address review comments\n\n* Update src/llama-sampler.cpp\n\nCo-authored-by: Georgi Gerganov \n\n---------\n\nCo-authored-by: Georgi Gerganov "},{"name":"scr.cpp","kind":"file","size":6447,"lastCommitHash":"904cb4ae461799612c62c826737bb23ddc3ec7fb","lastCommitWhen":"2026-10-13T19:13:57-04:00","lastCommitMessage":"bonsai: suffix cache reuse (SCR) for linear attention hybrids (scr)\n\nSuffix cache reuse: after an edit in the middle of a cached prompt (compaction, a trimmed tool output, a system prompt change), the attention KV of the text after the edit is moved to its new positions (q8_0 keys re-rotated) and the DeltaNet layers continue from the pre-edit state, so only the new text is prefilled. Approximate (KL ~0.017 nats/token after a compaction); per request \"scr\", server default --scr.\n"},{"name":"scr.h","kind":"file","size":2575,"lastCommitHash":"a33d3e21afd4bd4952a0b56fcd6dfee92e3f9ad3","lastCommitWhen":"2026-10-14T02:26:30-04:00","lastCommitMessage":"scr: retain up to 64 matching spans for fragmented transcripts\n"},{"name":"speculative.cpp","kind":"file","size":157899,"lastCommitHash":"86b38e68e23315e40289066c45d39e2fdd1bee17","lastCommitWhen":"2026-10-13T19:13:58-04:00","lastCommitMessage":"bonsai: drop rejected MTP rows on fair step to prevent draft desync (mtp-fair-deferred)\n\nMTP drops the rejected rows of a verification from its deferred queue at accept. Without it a fair prompt step of the other slot flushed them into the draft context, and every later draft of the decoding slot failed (draft positions ahead of the target).\n"},{"name":"speculative.h","kind":"file","size":6181,"lastCommitHash":"81a5c121f5ddfda40c037d117b5b0c685a7fa924","lastCommitWhen":"2026-10-13T19:13:56-04:00","lastCommitMessage":"bonsai: MTP catch-up rows ride in next draft batch (mtp-decode)\n\nDecode: MTP catch-up rows of a verify batch ride in the next draft batch (one MTP pass per round instead of two, rows computed identically), and the conv-state concat takes the transposing kernel. Greedy and sampled output identical; ~+6% decode on the 3060.\n"},{"name":"subproc.cpp","kind":"file","size":2901,"lastCommitHash":"d2a818231effb12b7b20b80b3b8c7756a9a33a04","lastCommitWhen":"2026-07-26T20:54:25+02:00","lastCommitMessage":"common: add `subproc.h` wrapper, disabled on android/ios (#26102)\n\n* add common/subproc.h|cpp\n\n* add compile flag LLAMA_SUBPROCESS\n\n* disabled by default on android and ios\n\n* test-jinja: use common subproc\n\n* mtmd: disable video if subproc is not set\n\n* disable subproc on wasm\n\n* make is_created atomic\n\n* migrate server-mcp"},{"name":"subproc.h","kind":"file","size":1867,"lastCommitHash":"d2a818231effb12b7b20b80b3b8c7756a9a33a04","lastCommitWhen":"2026-07-26T20:54:25+02:00","lastCommitMessage":"common: add `subproc.h` wrapper, disabled on android/ios (#26102)\n\n* add common/subproc.h|cpp\n\n* add compile flag LLAMA_SUBPROCESS\n\n* disabled by default on android and ios\n\n* test-jinja: use common subproc\n\n* mtmd: disable video if subproc is not set\n\n* disable subproc on wasm\n\n* make is_created atomic\n\n* migrate server-mcp"},{"name":"trie.cpp","kind":"file","size":3838,"lastCommitHash":"910196f6b3dfc6aca88fa732e2b02f270ff9b56b","lastCommitWhen":"2026-07-25T11:58:09+02:00","lastCommitMessage":"common : add support for multiple end sequences in the reasoning budget sampler (#25544)\n\n* common : extract trie/ac to a separate file\n\n* common : support multiple token sequences in the reasoning budget sampler\n\n* common/trie : return matched word index\n\n* common/trie : rename \"word\" to \"pattern\"\n\n* common/reasoning-budget : expose matched end sequence\n\n* common/sampling : replay end sequence when reasoning budget is done\n\n* cont : update to use multiple end sequences\n\n* cont : clean up"},{"name":"trie.h","kind":"file","size":2337,"lastCommitHash":"910196f6b3dfc6aca88fa732e2b02f270ff9b56b","lastCommitWhen":"2026-07-25T11:58:09+02:00","lastCommitMessage":"common : add support for multiple end sequences in the reasoning budget sampler (#25544)\n\n* common : extract trie/ac to a separate file\n\n* common : support multiple token sequences in the reasoning budget sampler\n\n* common/trie : return matched word index\n\n* common/trie : rename \"word\" to \"pattern\"\n\n* common/reasoning-budget : expose matched end sequence\n\n* common/sampling : replay end sequence when reasoning budget is done\n\n* cont : update to use multiple end sequences\n\n* cont : clean up"},{"name":"unicode.cpp","kind":"file","size":4160,"lastCommitHash":"acb7c790698fa28a0fbfc0468804926815b94de3","lastCommitWhen":"2026-03-11T10:26:12+01:00","lastCommitMessage":"common/parser: handle reasoning budget (#20297)\n\n* v1\n\n* Finished!\n\n* Handlie cli\n\n* Reasoning sampler\n\n* Apply suggestions from code review\n\nCo-authored-by: Sigbjørn Skjæret \n\n* Less explosive terminology :)\n\n* Add utf-8 case and tests\n\n* common : migrate reasoning budget sampler to common\n\n* cont : clean up\n\n* cont : expose state and allow passing as initial state\n\n* cont : remove unused imports\n\n* cont : update state machine doc string\n\n---------\n\nCo-authored-by: Sigbjørn Skjæret \nCo-authored-by: Alde Rojas "},{"name":"unicode.h","kind":"file","size":1083,"lastCommitHash":"acb7c790698fa28a0fbfc0468804926815b94de3","lastCommitWhen":"2026-03-11T10:26:12+01:00","lastCommitMessage":"common/parser: handle reasoning budget (#20297)\n\n* v1\n\n* Finished!\n\n* Handlie cli\n\n* Reasoning sampler\n\n* Apply suggestions from code review\n\nCo-authored-by: Sigbjørn Skjæret \n\n* Less explosive terminology :)\n\n* Add utf-8 case and tests\n\n* common : migrate reasoning budget sampler to common\n\n* cont : clean up\n\n* cont : expose state and allow passing as initial state\n\n* cont : remove unused imports\n\n* cont : update state machine doc string\n\n---------\n\nCo-authored-by: Sigbjørn Skjæret \nCo-authored-by: Alde Rojas "}]