[{"name":"batched","kind":"directory","size":0,"lastCommitHash":"6990e2f1f7581d332a6a1f34d6c567be70138184","lastCommitWhen":"2026-04-17T11:11:46+03:00","lastCommitMessage":"libs : rename libcommon -> libllama-common (#21936)\n\n* cmake : allow libcommon to be shared\n\n* cmake : rename libcommon to libllama-common\n\n* cont : set -fPIC for httplib\n\n* cont : export all symbols\n\n* cont : fix build_info exports\n\n* libs : add libllama-common-base\n\n* log : add common_log_get_verbosity_thold()"},{"name":"batched.swift","kind":"directory","size":0,"lastCommitHash":"29f538ac630d6544406a0702476e36808a6bd1b3","lastCommitWhen":"2025-08-21T06:12:28+02:00","lastCommitMessage":"examples : remove references to `make` in examples [no ci] (#15457)\n\nThis commit removes references to `make` in the examples, as the build\nsystem has been updated to use CMake directly and using `make` will now\ngenerate an error since Commit 37f10f955f70e0158d50343d0b9a3f92d194daae\n(\"make : remove make in favor of CMake (#15449)\")."},{"name":"convert-llama2c-to-ggml","kind":"directory","size":0,"lastCommitHash":"27df9199d16c393868d20b3ed28b4f72e30de6bd","lastCommitWhen":"2026-08-15T11:45:30+02:00","lastCommitMessage":"fix: check gguf array type before reading (#27075)\n\n* fix: check gguf array type before reading\n\n* update skill"},{"name":"debug","kind":"directory","size":0,"lastCommitHash":"5594d132244aeb1bae54dd431e7efbc908f5e3b8","lastCommitWhen":"2026-04-27T08:06:39+03:00","lastCommitMessage":"common: fix missing exports in llama-common (#22340)\n\n* common: refactor common/debug to move abort_on_nan into base_callback_data\n\nPassing bool abort_on_nan as template parameter for common_debug_cb_eval is unnecessary and creates an issue with LTO.\nIt should just be a member of the base_callback_data instead.\n\n* cont : cleanup\n\n* common : use pimpl in debug.h to reduce header dependencies\n\nMove common_debug_cb_user_data's data members (std::regex,\nstd::vector) into a private impl struct in debug.cpp.\n\nThis removes the includes of common.h and from debug.h,\nreducing transitive dependencies for any translation unit that\nincludes the header.\n\nAssisted-by: llama.cpp:local pi\n\n---------\n\nCo-authored-by: Georgi Gerganov "},{"name":"deprecation-warning","kind":"directory","size":0,"lastCommitHash":"cb8f4fa3f8e359e61ed570adb8a3c2ce74be5179","lastCommitWhen":"2026-03-04T09:30:40+01:00","lastCommitMessage":"Fix locale-dependent float printing in GGUF metadata (#17331)\n\n* Set C locale for consistent float formatting across all binaries.\n\n* Add C locale setting to all tools binaries\n\nAdd std::setlocale(LC_NUMERIC, \"C\") to all 16 binaries in the tools/\ndirectory to ensure consistent floating-point formatting.\n\n* Apply suggestion from @JohannesGaessler\n\n---------\n\nCo-authored-by: Johannes Gäßler "},{"name":"diffusion","kind":"directory","size":0,"lastCommitHash":"e6dd0e29a6751d4859abaa8899959f5ddf756f4e","lastCommitWhen":"2026-07-23T20:32:56+08:00","lastCommitMessage":"args: refactor mlock/mmap/directio into load-mode (#20834)\n\n* args: overhaul mmap/mlock/dio into single arg\n\nSigned-off-by: Aaron Teo \n\n* docs: update docs with llama-gen-docs\n\nSigned-off-by: Aaron Teo \n\n* chore: satisfy code quality\n\nSigned-off-by: Aaron Teo \n\n* args: make the `+` sign an actual modifier now\n\nSigned-off-by: Aaron Teo \n\n* chore: general code clean up + comments\n\nSigned-off-by: Aaron Teo \n\n* arg: fix deprecated flags support\n\nSigned-off-by: Aaron Teo \n\n* arg: quick sanity check\n\nSigned-off-by: Aaron Teo \n\n* bench: sync llama-bench argument parsing\n\nSigned-off-by: Aaron Teo \n\n* fix: bugfix variable behaviour + llama-bench lm column size\n\nSigned-off-by: Aaron Teo \n\n* arg: inverse commands should do the opposite instead of doing nothing\n\nSigned-off-by: Aaron Teo \n\n* bench: fix incorrect dash\n\nSigned-off-by: Aaron Teo \n\n* bench: fix missing modifiers for deprecated flags\n\nSigned-off-by: Aaron Teo \n\n* llama: switch back to thread_local\n\nSigned-off-by: Aaron Teo \n\n* arg: switch back to single enum\n\nSigned-off-by: Aaron Teo \n\n* docs: update arg docs\n\nSigned-off-by: Aaron Teo \n\n* chore: fix missing `mlock` from llama_load_mode_from_str + cleanup llama-bench\n\nSigned-off-by: Aaron Teo \n\n* llama: fix mlock not activating\n\nSigned-off-by: Aaron Teo \n\n* arg: add deprecation warning when old and new flags are combined\n\nSigned-off-by: Aaron Teo \n\n* arg: cont add comment for todo in the future\n\nSigned-off-by: Aaron Teo \n\n* docs: sync with upstream\n\nSigned-off-by: Aaron Teo \n\n* docs: re-sync with upstream again\n\nSigned-off-by: Aaron Teo \n\n---------\n\nSigned-off-by: Aaron Teo "},{"name":"embedding","kind":"directory","size":0,"lastCommitHash":"6990e2f1f7581d332a6a1f34d6c567be70138184","lastCommitWhen":"2026-04-17T11:11:46+03:00","lastCommitMessage":"libs : rename libcommon -> libllama-common (#21936)\n\n* cmake : allow libcommon to be shared\n\n* cmake : rename libcommon to libllama-common\n\n* cont : set -fPIC for httplib\n\n* cont : export all symbols\n\n* cont : fix build_info exports\n\n* libs : add libllama-common-base\n\n* log : add common_log_get_verbosity_thold()"},{"name":"eval-callback","kind":"directory","size":0,"lastCommitHash":"5594d132244aeb1bae54dd431e7efbc908f5e3b8","lastCommitWhen":"2026-04-27T08:06:39+03:00","lastCommitMessage":"common: fix missing exports in llama-common (#22340)\n\n* common: refactor common/debug to move abort_on_nan into base_callback_data\n\nPassing bool abort_on_nan as template parameter for common_debug_cb_eval is unnecessary and creates an issue with LTO.\nIt should just be a member of the base_callback_data instead.\n\n* cont : cleanup\n\n* common : use pimpl in debug.h to reduce header dependencies\n\nMove common_debug_cb_user_data's data members (std::regex,\nstd::vector) into a private impl struct in debug.cpp.\n\nThis removes the includes of common.h and from debug.h,\nreducing transitive dependencies for any translation unit that\nincludes the header.\n\nAssisted-by: llama.cpp:local pi\n\n---------\n\nCo-authored-by: Georgi Gerganov "},{"name":"gen-docs","kind":"directory","size":0,"lastCommitHash":"99111b19ce482f081e92ec6c6cdbe6a4c815c515","lastCommitWhen":"2026-08-03T18:51:02+02:00","lastCommitMessage":"server: add get_info tool (#26522)\n\n* server: add get_info tool\n\n* fix --rpc in docs\n\n* server: harden get_info probe result handling\n\nReport the OS as unknown when the probe process fails to spawn or times\nout, so the diagnostic text from run() is never returned as an OS name.\nStrip the probe output on both ends, which also drops the blank line\nthat ver prints before the version on Windows. Name the output and\ntimeout limits, and report an unreadable working directory as unknown\ninstead of an empty string.\n\n* server: simplify get_info result handling\n\nDrop the named limits and the working directory error branch, keeping\nthe probe result handling to a single expression.\n\n---------\n\nCo-authored-by: Pascal "},{"name":"gguf","kind":"directory","size":0,"lastCommitHash":"cb8f4fa3f8e359e61ed570adb8a3c2ce74be5179","lastCommitWhen":"2026-03-04T09:30:40+01:00","lastCommitMessage":"Fix locale-dependent float printing in GGUF metadata (#17331)\n\n* Set C locale for consistent float formatting across all binaries.\n\n* Add C locale setting to all tools binaries\n\nAdd std::setlocale(LC_NUMERIC, \"C\") to all 16 binaries in the tools/\ndirectory to ensure consistent floating-point formatting.\n\n* Apply suggestion from @JohannesGaessler\n\n---------\n\nCo-authored-by: Johannes Gäßler "},{"name":"gguf-hash","kind":"directory","size":0,"lastCommitHash":"27e345b574dd8c8838e2c06e47699a3135f16ec9","lastCommitWhen":"2026-08-18T11:16:51+03:00","lastCommitMessage":"build : fix xcframework + cmake clean-up (#27304)\n\n* xcframework : fix build\n\n* mtmd : remove unused include path\n\n* vendor : use vendor::hash alias target in cmake\n\nCMake reserves \"::\" in target names for imported/alias targets, so the real\ntarget keeps the name vendor-hash and a vendor::hash ALIAS target is added.\nConsumers (mtmd, llama-gguf-hash) now link against the namespaced alias.\n\nAssisted-by: pi:llama.cpp/Qwen3.8-27B\n\n* vendor : add cmake targets for all vendored libs with vendor:: aliases\n\nAdd INTERFACE targets for the header-only vendor libs (miniaudio, nlohmann,\nsheredom, stb) and ALIAS targets named vendor:: for all of them,\nincluding cpp-httplib and hash. Each exposes the vendor/ root so includes\nare namespaced, e.g. .\n\nConsolidate the per-lib add_subdirectory calls into a single\nadd_subdirectory(vendor), keeping the cpp-httplib gate on LLAMA_BUILD_COMMON.\nConsumers (llama-common, mtmd) now link the aliases instead of relying on\nraw vendor/ include paths.\n\nhash: consumers now include via \"hash/hash.h\"; the vendor/hash dir is kept\nas a PRIVATE include so the synced upstream sources compile unmodified.\n\nAssisted-by: pi:llama.cpp/Qwen3.8-27B\n\n* readme : use foo/bar names in acknowledgements\n\nAssisted-by: pi:llama.cpp/Qwen3.8-27B\n\n* ocd : fix valign"},{"name":"idle","kind":"directory","size":0,"lastCommitHash":"6990e2f1f7581d332a6a1f34d6c567be70138184","lastCommitWhen":"2026-04-17T11:11:46+03:00","lastCommitMessage":"libs : rename libcommon -> libllama-common (#21936)\n\n* cmake : allow libcommon to be shared\n\n* cmake : rename libcommon to libllama-common\n\n* cont : set -fPIC for httplib\n\n* cont : export all symbols\n\n* cont : fix build_info exports\n\n* libs : add libllama-common-base\n\n* log : add common_log_get_verbosity_thold()"},{"name":"llama-eval","kind":"directory","size":0,"lastCommitHash":"da46e59cbf899282a70eb48a3502f7811e9ede29","lastCommitWhen":"2026-07-08T10:00:03+03:00","lastCommitMessage":"llama-eval : fix crash when answer is None in HTML dump (#25435)\n\ndict.get(\"key\", default) returns None (not default) when the key\nexists but its value is explicitly None. This caused an AttributeError\nin _escape_html() when a task errored before grading and answer was\nset to None.\n\nAssisted-by: pi:llama.cpp/Qwen3.6-27B"},{"name":"llama.android","kind":"directory","size":0,"lastCommitHash":"bbce619adb409880fb6db850a1c5a5f36a4dc7b1","lastCommitWhen":"2026-05-22T11:46:26+03:00","lastCommitMessage":"cmake : add install() for impl libraries + fix apple builds (#23511)\n\n* pi : update\n\n* ci : fix ios build\n\n* ci : fix andoroid\n\n* ci : fix apple builds\n\n* cmake : add install() for impl libraries\n\nAdd install(TARGETS LIBRARY) for all -impl libraries that were\nchanged from STATIC to shared (controlled by BUILD_SHARED_LIBS) in\ncommit bb28c1fe2. Without this, cmake --install fails to copy the shared\nlibraries, causing runtime errors like:\n\n llama-server: error while loading shared libraries: libllama-server-impl.so\n\nRef: https://github.com/ggml-org/llama.cpp/issues/23494#issuecomment-4512912515\n\nAssisted-by: llama.cpp:local pi\n\n* ci : fix xcframework build"},{"name":"llama.swiftui","kind":"directory","size":0,"lastCommitHash":"745aa5319b9930068aff5e87cf5e9eef7227339b","lastCommitWhen":"2025-06-06T14:11:15+03:00","lastCommitMessage":"llama : deprecate llama_kv_self_ API (#14030)\n\n* llama : deprecate llama_kv_self_ API\n\nggml-ci\n\n* llama : allow llama_memory_(nullptr)\n\nggml-ci\n\n* memory : add flag for optional data clear in llama_memory_clear\n\nggml-ci"},{"name":"lookahead","kind":"directory","size":0,"lastCommitHash":"6990e2f1f7581d332a6a1f34d6c567be70138184","lastCommitWhen":"2026-04-17T11:11:46+03:00","lastCommitMessage":"libs : rename libcommon -> libllama-common (#21936)\n\n* cmake : allow libcommon to be shared\n\n* cmake : rename libcommon to libllama-common\n\n* cont : set -fPIC for httplib\n\n* cont : export all symbols\n\n* cont : fix build_info exports\n\n* libs : add libllama-common-base\n\n* log : add common_log_get_verbosity_thold()"},{"name":"lookup","kind":"directory","size":0,"lastCommitHash":"dd1ea524333b1e697489067d7a4c39c60d32beee","lastCommitWhen":"2026-08-10T16:58:56+03:00","lastCommitMessage":"llama : support multi-output backend sampling (#25532)\n\n* Enable backend sampling with token speculation\n\n* Clamp the mask sum before converting it into the sampled index\n\n* Add a numeric context parameter declaring the maximum outputs one sequence\n\n* More fixes\n\n* Don't reuse memory for output views.\n\n* Match dist between CPU and GPU\n\n* Fix CPU and backend sampling mismatches\n\n* Simpify some of the changes\n\n* Fix tests on Vulkan\n\n* More test fixes\n\n* Rebase changes\n\n* Rebase and address review comments\n\n* Address review comments\n\n* Address review comments\n\n* Update src/llama-sampler.cpp\n\nCo-authored-by: Georgi Gerganov \n\n---------\n\nCo-authored-by: Georgi Gerganov "},{"name":"model-conversion","kind":"directory","size":0,"lastCommitHash":"2468576f241235452013308597e6de1b78866996","lastCommitWhen":"2026-08-11T21:58:53+08:00","lastCommitMessage":"requirements: use stable torch packages on s390x (#26864)"},{"name":"parallel","kind":"directory","size":0,"lastCommitHash":"6990e2f1f7581d332a6a1f34d6c567be70138184","lastCommitWhen":"2026-04-17T11:11:46+03:00","lastCommitMessage":"libs : rename libcommon -> libllama-common (#21936)\n\n* cmake : allow libcommon to be shared\n\n* cmake : rename libcommon to libllama-common\n\n* cont : set -fPIC for httplib\n\n* cont : export all symbols\n\n* cont : fix build_info exports\n\n* libs : add libllama-common-base\n\n* log : add common_log_get_verbosity_thold()"},{"name":"passkey","kind":"directory","size":0,"lastCommitHash":"6990e2f1f7581d332a6a1f34d6c567be70138184","lastCommitWhen":"2026-04-17T11:11:46+03:00","lastCommitMessage":"libs : rename libcommon -> libllama-common (#21936)\n\n* cmake : allow libcommon to be shared\n\n* cmake : rename libcommon to libllama-common\n\n* cont : set -fPIC for httplib\n\n* cont : export all symbols\n\n* cont : fix build_info exports\n\n* libs : add libllama-common-base\n\n* log : add common_log_get_verbosity_thold()"},{"name":"retrieval","kind":"directory","size":0,"lastCommitHash":"6990e2f1f7581d332a6a1f34d6c567be70138184","lastCommitWhen":"2026-04-17T11:11:46+03:00","lastCommitMessage":"libs : rename libcommon -> libllama-common (#21936)\n\n* cmake : allow libcommon to be shared\n\n* cmake : rename libcommon to libllama-common\n\n* cont : set -fPIC for httplib\n\n* cont : export all symbols\n\n* cont : fix build_info exports\n\n* libs : add libllama-common-base\n\n* log : add common_log_get_verbosity_thold()"},{"name":"rs-rollback","kind":"directory","size":0,"lastCommitHash":"e8b6452b1fceaa0ed8aa960d459e64f823d15e2d","lastCommitWhen":"2026-08-26T21:21:42-07:00","lastCommitMessage":"metal: fix the im2col kernel/dispatch mismatch and add a wide-row set_rows (#118)\n\nRebased onto prism-v7. The previous branch was based on prism, and a base\nswitch without a rebase pulled the old prism history into the diff.\n\nim2col: the kernel choice and the dispatch geometry disagreed. The normal\nkernel is dispatched with an (N, KH, KW) threadgroup and the ext kernel with\nthe CHW-folded grid, but the two decisions were made independently, one\nagainst a fixed 1024 and one against the pipeline's own threadgroup limit. On\na device whose limit is below 1024 that pairs the normal kernel with the ext\ndispatch. The caller now derives the branch once from the pipeline's own limit\nand passes it in.\n\nset_rows: wide f32 rows, such as recurrent-state snapshots, gave one\nthreadgroup per row, which cannot saturate bandwidth for rows of 100k+\nelements. Adds kernel_set_rows_f32_wide, which tiles each row across\nthreadgroups and copies with float4. Guarded on alignment and contiguity.\n\nAlso adds two rollback correctness harnesses under examples/rs-rollback. They\ncount rollbacks and fail when none happened, so a run too short to roll back\ncannot report a vacuous pass.\n\nPorting notes for this branch:\n- the Metal shaders moved to per-op files, so kernel_set_rows_f32_wide was\n relanded in kernels/quantize.metal rather than cherry-picked\n- get_pipeline_set_rows now takes the op, so the wide variant matches it\n- test_set_rows gained a src_type parameter, so the new wide cases use it\n\nThe delta-net conv-state change from the old branch is NOT included. It needs\nthe rotating snapshot ring in llama_memory_recurrent, which this branch does\nnot have. See the PR discussion.\n\ntest-backend-ops on M5: SET_ROWS 175/175, IM2COL 93/93, 3/3 backends. The wide\nkernel is confirmed to compile and run on the 4096-wide cases."},{"name":"simple","kind":"directory","size":0,"lastCommitHash":"cb8f4fa3f8e359e61ed570adb8a3c2ce74be5179","lastCommitWhen":"2026-03-04T09:30:40+01:00","lastCommitMessage":"Fix locale-dependent float printing in GGUF metadata (#17331)\n\n* Set C locale for consistent float formatting across all binaries.\n\n* Add C locale setting to all tools binaries\n\nAdd std::setlocale(LC_NUMERIC, \"C\") to all 16 binaries in the tools/\ndirectory to ensure consistent floating-point formatting.\n\n* Apply suggestion from @JohannesGaessler\n\n---------\n\nCo-authored-by: Johannes Gäßler "},{"name":"simple-chat","kind":"directory","size":0,"lastCommitHash":"cb8f4fa3f8e359e61ed570adb8a3c2ce74be5179","lastCommitWhen":"2026-03-04T09:30:40+01:00","lastCommitMessage":"Fix locale-dependent float printing in GGUF metadata (#17331)\n\n* Set C locale for consistent float formatting across all binaries.\n\n* Add C locale setting to all tools binaries\n\nAdd std::setlocale(LC_NUMERIC, \"C\") to all 16 binaries in the tools/\ndirectory to ensure consistent floating-point formatting.\n\n* Apply suggestion from @JohannesGaessler\n\n---------\n\nCo-authored-by: Johannes Gäßler "},{"name":"simple-cmake-pkg","kind":"directory","size":0,"lastCommitHash":"817d743cc17cf644dab8408eb0f1e6eac89562c1","lastCommitWhen":"2025-12-04T14:17:30+01:00","lastCommitMessage":"examples : add missing code block end marker [no ci] (#17756)\n\nThis commit adds the missing code block end marker in simple-cmake-pkg\nto correct the formatting."},{"name":"speculative","kind":"directory","size":0,"lastCommitHash":"dd1ea524333b1e697489067d7a4c39c60d32beee","lastCommitWhen":"2026-08-10T16:58:56+03:00","lastCommitMessage":"llama : support multi-output backend sampling (#25532)\n\n* Enable backend sampling with token speculation\n\n* Clamp the mask sum before converting it into the sampled index\n\n* Add a numeric context parameter declaring the maximum outputs one sequence\n\n* More fixes\n\n* Don't reuse memory for output views.\n\n* Match dist between CPU and GPU\n\n* Fix CPU and backend sampling mismatches\n\n* Simpify some of the changes\n\n* Fix tests on Vulkan\n\n* More test fixes\n\n* Rebase changes\n\n* Rebase and address review comments\n\n* Address review comments\n\n* Address review comments\n\n* Update src/llama-sampler.cpp\n\nCo-authored-by: Georgi Gerganov \n\n---------\n\nCo-authored-by: Georgi Gerganov "},{"name":"speculative-simple","kind":"directory","size":0,"lastCommitHash":"36761482b026e0a4de4f6cabe08cf756b786227c","lastCommitWhen":"2026-09-16T14:36:22-07:00","lastCommitMessage":"speculative: stop plain drafter+target falling back to KV checkpoints (#173)\n\n* speculative: stop plain drafter+target falling back to KV checkpoints\n\nOn hybrid (recurrent + attention) models, plain drafter+target speculative\ndecoding was classified as unable to do partial sequence removal, and fell\nback to a whole-KV checkpoint save/restore every round. On a 27B qwen35 that\nwas 74-77% of each round -- roughly 0.5 s per drafted token, ~15x the\ndrafter's own compute -- which made speculation about 2x slower than plain\ndecoding regardless of draft quality.\n\nTwo causes, both needed:\n\n1. need_n_rs_seq() listed DRAFT_MTP, DRAFT_EAGLE3, DRAFT_DFLASH and\n DRAFT_DSPARK but not DRAFT_SIMPLE, so the ordinary drafter+target case got\n n_rs_seq = 0. common_context_can_seq_rm() then failed its\n llama_n_rs_seq() > 0 test, llama_memory_seq_rm() failed on the hybrid\n memory, and the context was classified COMMON_CONTEXT_SEQ_RM_TYPE_FULL.\n LLM_ARCH_QWEN35 is already in llm_arch_supports_rs_rollback(), so the\n architecture was never the blocker.\n\n2. The draft context explicitly zeroed cparams.n_rs_seq. It has to match the\n target: on partial acceptance both contexts drop the rejected suffix, and a\n draft context at n_rs_seq = 0 silently fails its seq_rm and keeps stale\n positions, which then trips the M-RoPE monotonic-position check on the next\n batch (seq_pos_max == batch seq_pos_min). Traced: tgt_ok=1 dft_ok=0 with\n tgt_max=10 against dft_max=12.\n\nNo change is needed in speculative-simple.cpp -- it already removes the\nrejected draft tokens unconditionally after the use_ckpt_tgt block, and that\npath is always reached when use_ckpt_tgt is false. An earlier version of this\nfix added a second removal there; ablating it gave identical results, so it\nwas dropped.\n\nMeasured on an Arc B390, 2B-PTQ1_0 drafting 27B-PTQ1_0, greedy, against a\n4.43 t/s llama-bench decode-only baseline:\n\n k=1 1.137x / 1.356x (two prompts)\n k=2 0.482x -> 1.085x\n k=4 0.535x -> 0.727x\n\nGenerated text is byte-identical to a k=1 reference over the common prefix.\n\nAlso note the reported acceptance rate changes meaning for hybrid models that\npreviously took the checkpoint path: that path continues before n_drafted and\nn_accept are incremented, so partial-acceptance rounds were never counted and\nthe rate read 100%. With partial rounds counted it is 14-87% depending on k\nand workload.\n\n* speculative: keep the checkpoint path when the rollback window does not fit the ubatch\n\nRecurrent and hybrid memory keep the last n_rs_seq + 1 tokens of a sequence inside one ubatch and assert on it. Decide the fit once in common_context_params_to_llama, warn, and fall back to n_rs_seq = 0 so small -ub configurations keep working. The draft context inherits that value instead of overriding it.\n\nspeculative-simple now checkpoints per round when a draft is longer than llama_n_rs_seq(), as the server does, so an n-gram speculator ahead of draft-simple no longer relies on a seq_rm that cannot succeed.\n\n* speculative: base the rollback fit on the effective micro-batch\n\nllama_context clamps the micro-batch to min(n_batch, n_ubatch) and n_batch to n_ctx, so checking the requested -ub let --batch-size 8 --ubatch-size 512 past the guard and into the same assertion. Check the effective size instead.\n\nThe memory fitter no longer forces the draft context's n_rs_seq to zero: recurrent state allocates mem_size * (1 + n_rs_seq) rows, so a fit that measured a different value underestimated the draft context.\n\nIn speculative-simple the pre-draft reset now loads a checkpoint only for a FULL context. use_ckpt_dft carries the previous round's RS-overflow decision, and no checkpoint is taken for an RS context at that point, so the load could roll the draft context back to a stale position."},{"name":"sycl","kind":"directory","size":0,"lastCommitHash":"37a215c9e909822fb8ebb72b5bd37623b0b9611f","lastCommitWhen":"2026-08-17T07:31:29+03:00","lastCommitMessage":"[SYCL] support OP OPT_STEP_ADAMW, OPT_STEP_SGD (#25268)\n\n* fix conflict\n\n* fix conflict of ops.md\n\n* fix conflict of ops.md\n\n* update the ops.md\n\n---------\n\nCo-authored-by: Neo Zhang Jianyu "},{"name":"test-cmake","kind":"directory","size":0,"lastCommitHash":"885c5bbe8e04dc78db25beb911a2715312ad7b54","lastCommitWhen":"2026-08-14T10:32:59+02:00","lastCommitMessage":"tests : replace personal home directory paths with generic placeholders (#27043)\n\nScrub developer-specific /home// paths from example docs and test\nfixtures so they don't leak into the tree.\n\n- examples/test-cmake/README.md: /home/danbev/... -> /path/to/llama.cpp/...\n- tests/test-chat.cpp: /home/jarvis/... -> /home/user/... (input and\n expected string kept identical so the parser test still passes)\n\nCo-authored-by: Jim Wu "},{"name":"training","kind":"directory","size":0,"lastCommitHash":"e6dd0e29a6751d4859abaa8899959f5ddf756f4e","lastCommitWhen":"2026-07-23T20:32:56+08:00","lastCommitMessage":"args: refactor mlock/mmap/directio into load-mode (#20834)\n\n* args: overhaul mmap/mlock/dio into single arg\n\nSigned-off-by: Aaron Teo \n\n* docs: update docs with llama-gen-docs\n\nSigned-off-by: Aaron Teo \n\n* chore: satisfy code quality\n\nSigned-off-by: Aaron Teo \n\n* args: make the `+` sign an actual modifier now\n\nSigned-off-by: Aaron Teo \n\n* chore: general code clean up + comments\n\nSigned-off-by: Aaron Teo \n\n* arg: fix deprecated flags support\n\nSigned-off-by: Aaron Teo \n\n* arg: quick sanity check\n\nSigned-off-by: Aaron Teo \n\n* bench: sync llama-bench argument parsing\n\nSigned-off-by: Aaron Teo \n\n* fix: bugfix variable behaviour + llama-bench lm column size\n\nSigned-off-by: Aaron Teo \n\n* arg: inverse commands should do the opposite instead of doing nothing\n\nSigned-off-by: Aaron Teo \n\n* bench: fix incorrect dash\n\nSigned-off-by: Aaron Teo \n\n* bench: fix missing modifiers for deprecated flags\n\nSigned-off-by: Aaron Teo \n\n* llama: switch back to thread_local\n\nSigned-off-by: Aaron Teo \n\n* arg: switch back to single enum\n\nSigned-off-by: Aaron Teo \n\n* docs: update arg docs\n\nSigned-off-by: Aaron Teo \n\n* chore: fix missing `mlock` from llama_load_mode_from_str + cleanup llama-bench\n\nSigned-off-by: Aaron Teo \n\n* llama: fix mlock not activating\n\nSigned-off-by: Aaron Teo \n\n* arg: add deprecation warning when old and new flags are combined\n\nSigned-off-by: Aaron Teo \n\n* arg: cont add comment for todo in the future\n\nSigned-off-by: Aaron Teo \n\n* docs: sync with upstream\n\nSigned-off-by: Aaron Teo \n\n* docs: re-sync with upstream again\n\nSigned-off-by: Aaron Teo \n\n---------\n\nSigned-off-by: Aaron Teo "},{"name":"CMakeLists.txt","kind":"file","size":1039,"lastCommitHash":"e8b6452b1fceaa0ed8aa960d459e64f823d15e2d","lastCommitWhen":"2026-08-26T21:21:42-07:00","lastCommitMessage":"metal: fix the im2col kernel/dispatch mismatch and add a wide-row set_rows (#118)\n\nRebased onto prism-v7. The previous branch was based on prism, and a base\nswitch without a rebase pulled the old prism history into the diff.\n\nim2col: the kernel choice and the dispatch geometry disagreed. The normal\nkernel is dispatched with an (N, KH, KW) threadgroup and the ext kernel with\nthe CHW-folded grid, but the two decisions were made independently, one\nagainst a fixed 1024 and one against the pipeline's own threadgroup limit. On\na device whose limit is below 1024 that pairs the normal kernel with the ext\ndispatch. The caller now derives the branch once from the pipeline's own limit\nand passes it in.\n\nset_rows: wide f32 rows, such as recurrent-state snapshots, gave one\nthreadgroup per row, which cannot saturate bandwidth for rows of 100k+\nelements. Adds kernel_set_rows_f32_wide, which tiles each row across\nthreadgroups and copies with float4. Guarded on alignment and contiguity.\n\nAlso adds two rollback correctness harnesses under examples/rs-rollback. They\ncount rollbacks and fail when none happened, so a run too short to roll back\ncannot report a vacuous pass.\n\nPorting notes for this branch:\n- the Metal shaders moved to per-op files, so kernel_set_rows_f32_wide was\n relanded in kernels/quantize.metal rather than cherry-picked\n- get_pipeline_set_rows now takes the op, so the wide variant matches it\n- test_set_rows gained a src_type parameter, so the new wide cases use it\n\nThe delta-net conv-state change from the old branch is NOT included. It needs\nthe rotating snapshot ring in llama_memory_recurrent, which this branch does\nnot have. See the PR discussion.\n\ntest-backend-ops on M5: SET_ROWS 175/175, IM2COL 93/93, 3/3 backends. The wide\nkernel is confirmed to compile and run on the 4096-wide cases."},{"name":"convert_legacy_llama.py","kind":"file","size":61041,"lastCommitHash":"5d246a792ddf839e87fb777681dd2a9ac7f7eb5e","lastCommitWhen":"2026-05-24T09:51:31+02:00","lastCommitMessage":"convert : minor fixes for numpy 2.x (#23571)"},{"name":"json_schema_pydantic_example.py","kind":"file","size":3214,"lastCommitHash":"3fd62a6b1c9ca7b7c0093e984cc9c133c6f2726d","lastCommitWhen":"2024-07-07T15:04:39-04:00","lastCommitMessage":"py : type-check all Python scripts with Pyright (#8341)\n\n* py : type-check all Python scripts with Pyright\r\n\r\n* server-tests : use trailing slash in openai base_url\r\n\r\n* server-tests : add more type annotations\r\n\r\n* server-tests : strip \"chat\" from base_url in oai_chat_completions\r\n\r\n* server-tests : model metadata is a dict\r\n\r\n* ci : disable pip cache in type-check workflow\r\n\r\nThe cache is not shared between branches, and it's 250MB in size,\r\nso it would become quite a big part of the 10GB cache limit of the repo.\r\n\r\n* py : fix new type errors from master branch\r\n\r\n* tests : fix test-tokenizer-random.py\r\n\r\nApparently, gcc applies optimisations even when pre-processing,\r\nwhich confuses pycparser.\r\n\r\n* ci : only show warnings and errors in python type-check\r\n\r\nThe \"information\" level otherwise has entries\r\nfrom 'examples/pydantic_models_to_grammar.py',\r\nwhich could be confusing for someone trying to figure out what failed,\r\nconsidering that these messages can safely be ignored\r\neven though they look like errors."},{"name":"json_schema_to_grammar.py","kind":"file","size":35187,"lastCommitHash":"c57607016a1ebdd08d269e3378eee5546fc3bf3a","lastCommitWhen":"2026-06-20T17:43:04-05:00","lastCommitMessage":"common/json-schema-to-grammar : align spacing rules with parsers (#24835)"},{"name":"llama.vim","kind":"file","size":26263,"lastCommitHash":"92f7da00b49ad814b95832dd6610a825bbdd3033","lastCommitWhen":"2026-03-05T08:50:21+01:00","lastCommitMessage":"chore : correct typos [no ci] (#20041)\n\n* fix(docs): correct typos found during code review\n\nNon-functional changes only:\n- Fixed minor spelling mistakes in comments\n- Corrected typos in user-facing strings\n- No variables, logic, or functional code was modified.\n\nSigned-off-by: Marcel Petrick \n\n* Update docs/backend/CANN.md\n\nCo-authored-by: Aaron Teo \n\n* Revert \"Auxiliary commit to revert individual files from 846d1c301281178efbc6ce6060ad34c1ebe45af8\"\n\nThis reverts commit 02fcf0c7db661d5ff3eff96b2b2db9fdb7213256.\n\n* Update tests/test-backend-ops.cpp\n\nCo-authored-by: Sigbjørn Skjæret \n\n* Update tests/test-backend-ops.cpp\n\nCo-authored-by: Sigbjørn Skjæret \n\n---------\n\nSigned-off-by: Marcel Petrick \nCo-authored-by: Aaron Teo \nCo-authored-by: Sigbjørn Skjæret "},{"name":"pydantic_models_to_grammar_examples.py","kind":"file","size":13719,"lastCommitHash":"1d36b3670b285e69e58b9d687c770a2a0a192194","lastCommitWhen":"2025-05-02T20:27:13+02:00","lastCommitMessage":"llama : move end-user examples to tools directory (#13249)\n\n* llama : move end-user examples to tools directory\n\n---------\n\nCo-authored-by: Xuan Son Nguyen "},{"name":"pydantic_models_to_grammar.py","kind":"file","size":56266,"lastCommitHash":"29b28a98242692e352df38254c9a2bb9fa79c884","lastCommitWhen":"2026-03-21T08:54:34+01:00","lastCommitMessage":"ci : switch from pyright to ty (#20826)\n\n* type fixes\n\n* switch to ty\n\n* tweak rules\n\n* tweak more rules\n\n* more tweaks\n\n* final tweak\n\n* use common import-not-found rule"},{"name":"reason-act.sh","kind":"file","size":363,"lastCommitHash":"e9b6350e61d592634263a14b3d77ecbf6c1fb096","lastCommitWhen":"2025-06-30T10:17:18+02:00","lastCommitMessage":"scripts : make the shell scripts cross-platform (#14341)\n\n"},{"name":"regex_to_grammar.py","kind":"file","size":431,"lastCommitHash":"e235b267a2539d043734ff340eff74107722eb57","lastCommitWhen":"2024-07-05T07:53:33+03:00","lastCommitMessage":"py : switch to snake_case (#8305)\n\n* py : switch to snake_case\r\n\r\nggml-ci\r\n\r\n* cont\r\n\r\nggml-ci\r\n\r\n* cont\r\n\r\nggml-ci\r\n\r\n* cont : fix link\r\n\r\n* gguf-py : use snake_case in scripts entrypoint export\r\n\r\n* py : rename requirements for convert_legacy_llama.py\r\n\r\nNeeded for scripts/check-requirements.sh\r\n\r\n---------\r\n\r\nCo-authored-by: Francis Couture-Harpin "},{"name":"server_embd.py","kind":"file","size":969,"lastCommitHash":"a19b5cef16d885c44c635da4a5c97113c1577de8","lastCommitWhen":"2025-04-08T19:54:51+03:00","lastCommitMessage":"llama : fix FA when KV cache is not used (i.e. embeddings) (#12825)\n\n* ggml : FA supports F32 V\n\n* graph : cast KV to F16 when the KV cache is not used\n\nggml-ci\n\n* server : add test that exercises embeddings with FA enabled\n\nggml-ci"},{"name":"server-llama2-13B.sh","kind":"file","size":798,"lastCommitHash":"e9b6350e61d592634263a14b3d77ecbf6c1fb096","lastCommitWhen":"2025-06-30T10:17:18+02:00","lastCommitMessage":"scripts : make the shell scripts cross-platform (#14341)\n\n"},{"name":"ts-type-to-grammar.sh","kind":"file","size":928,"lastCommitHash":"e9b6350e61d592634263a14b3d77ecbf6c1fb096","lastCommitWhen":"2025-06-30T10:17:18+02:00","lastCommitMessage":"scripts : make the shell scripts cross-platform (#14341)\n\n"}]