{"entries":[{"name":"__init__.py","mode":"0100644","size":29,"lastCommit":{"hash":"672a6f101839a150180fc28891ebed379c8f2353","oid":"672a6f101839a150180fc28891ebed379c8f2353","parents":["3807c3de04cde853418033c95e96642876545f3e"],"author":{"name":"Brian","email":"mofosyne@gmail.com","when":"2024-07-18T20:40:15+10:00"},"committer":{"name":"GitHub","email":"noreply@github.com","when":"2024-07-18T20:40:15+10:00"},"message":"convert-*.py: GGUF Naming Convention Refactor and Metadata Override Refactor (#7499)\n\nMain thing is that the default output filename will take this form\r\n\r\n{name}{parameters}{finetune}{version}{encoding}{kind}\r\n\r\nIn addition this add and remove some entries in the KV store and adds a metadata class with automatic heuristics capability to derive some values based on model card content\r\n\r\n* No Change:\r\n - Internal GGUF Spec\r\n - `general.architecture`\r\n - `general.quantization_version`\r\n - `general.alignment`\r\n - `general.file_type`\r\n - General Model Details\r\n - `general.name`\r\n - `general.author`\r\n - `general.version`\r\n - `general.description`\r\n - Licensing details\r\n - `general.license`\r\n - Typically represents the converted GGUF repo (Unless made from scratch)\r\n - `general.url`\r\n - Model Source during conversion\r\n - `general.source.url`\r\n\r\n* Removed:\r\n - Model Source during conversion\r\n - `general.source.huggingface.repository`\r\n\r\n* Added:\r\n - General Model Details\r\n - `general.organization`\r\n - `general.finetune`\r\n - `general.basename`\r\n - `general.quantized_by`\r\n - `general.size_label`\r\n - Licensing details\r\n - `general.license.name`\r\n - `general.license.link`\r\n - Typically represents the converted GGUF repo (Unless made from scratch)\r\n - `general.doi`\r\n - `general.uuid`\r\n - `general.repo_url`\r\n - Model Source during conversion\r\n - `general.source.doi`\r\n - `general.source.uuid`\r\n - `general.source.repo_url`\r\n - Base Model Source\r\n - `general.base_model.count`\r\n - `general.base_model.{id}.name`\r\n - `general.base_model.{id}.author`\r\n - `general.base_model.{id}.version`\r\n - `general.base_model.{id}.organization`\r\n - `general.base_model.{id}.url` (Model Website/Paper)\r\n - `general.base_model.{id}.doi`\r\n - `general.base_model.{id}.uuid`\r\n - `general.base_model.{id}.repo_url` (Model Source Repository (git/svn/etc...))\r\n - Array based KV stores\r\n - `general.tags`\r\n - `general.languages`\r\n - `general.datasets`\r\n\r\n---------\r\n\r\nCo-authored-by: compilade \r\nCo-authored-by: Xuan Son Nguyen ","tree":"0fe1866907aae3695e938839f383826b6f7f36ba"}},{"name":"test_gguf_reader_validation.py","mode":"0100644","size":1271,"lastCommit":{"hash":"5788b510a1e3394fcc2d6b13ba2d9d8fc4a5c139","oid":"5788b510a1e3394fcc2d6b13ba2d9d8fc4a5c139","parents":["2e17f69ef4e7fcb9ee950362b3530d8e4e2ea316"],"author":{"name":"hcl","email":"chenglunhu@gmail.com","when":"2026-08-04T17:12:48+08:00"},"committer":{"name":"GitHub","email":"noreply@github.com","when":"2026-08-04T12:12:48+03:00"},"message":"gguf-py: validate n_dims and guard against uint64 overflow in reader (#25401)\n\nThe Python GGUF reader lacked two guards the C++ loader has:\n- n_dims read as uint32 with no GGML_MAX_DIMS bound -> crafted file with\n huge n_dims triggers oversized memmap read / OOM.\n- np.prod(dims) on uint64 wraps silently -> a crafted dims triple can\n overflow to a tiny element count, passing an undersized read through.\n\nAdd a GGML_MAX_DIMS check and compute the element count with Python ints.\n\nFixes #25378","tree":"154141302cb3b2baf68f0b5cc9fa56e0cc31452e"}},{"name":"test_metadata.py","mode":"0100755","size":15628,"lastCommit":{"hash":"92f7da00b49ad814b95832dd6610a825bbdd3033","oid":"92f7da00b49ad814b95832dd6610a825bbdd3033","parents":["7a99dc85e2d26a9b9c540cad887322eae8924f03"],"author":{"name":"Marcel Petrick","email":"mail@marcelpetrick.it","when":"2026-03-05T08:50:21+01:00"},"committer":{"name":"GitHub","email":"noreply@github.com","when":"2026-03-05T08:50:21+01:00"},"message":"chore : correct typos [no ci] (#20041)\n\n* fix(docs): correct typos found during code review\n\nNon-functional changes only:\n- Fixed minor spelling mistakes in comments\n- Corrected typos in user-facing strings\n- No variables, logic, or functional code was modified.\n\nSigned-off-by: Marcel Petrick \n\n* Update docs/backend/CANN.md\n\nCo-authored-by: Aaron Teo \n\n* Revert \"Auxiliary commit to revert individual files from 846d1c301281178efbc6ce6060ad34c1ebe45af8\"\n\nThis reverts commit 02fcf0c7db661d5ff3eff96b2b2db9fdb7213256.\n\n* Update tests/test-backend-ops.cpp\n\nCo-authored-by: Sigbjørn Skjæret \n\n* Update tests/test-backend-ops.cpp\n\nCo-authored-by: Sigbjørn Skjæret \n\n---------\n\nSigned-off-by: Marcel Petrick \nCo-authored-by: Aaron Teo \nCo-authored-by: Sigbjørn Skjæret ","tree":"50c55acfdf117e309c1875e72aced1a2fb30618f"}},{"name":"test_quants.py","mode":"0100755","size":10526,"lastCommit":{"hash":"5eae9cb1d9ecf0bbe031352da61b8b22a3e10bbb","oid":"5eae9cb1d9ecf0bbe031352da61b8b22a3e10bbb","parents":["3ca19b0e9f3f4f444d22c9f509805d037a611847"],"author":{"name":"Richard Davison","email":"richard.davison1@gmail.com","when":"2026-03-11T21:02:54+01:00"},"committer":{"name":"GitHub","email":"noreply@github.com","when":"2026-03-11T21:02:54+01:00"},"message":"ggml : add NVFP4 quantization type support (#19769)\n\n* WIP: add NVFP4 quantization support\n\n* tests\n\n* improve NVFP4 dot product implementation performance and fix bad super call\n\n* typo\n\n* Use nvfp4 kvalues\n\n* vulkan : fix NVFP4 shader compilation by including kvalues_mxfp4 lookup table\n\n* vulcal and perf fixes\n\n* wip\n\n* Fix metal\n\n* fix vulcan\n\n* Rename threshold & fix wrong scale\n\n* Fix MOE\n\n* Shelf backend implementations (CUDA, Metal, Vulkan, arch-specific SIMD)\n\nRemove NVFP4 support from GPU backends and architecture-specific\noptimized dot products. These should be added in separate PRs so\nbackend specialists can review them independently.\n\nReverted files:\n- ggml-cuda: common.cuh, convert.cu, mmq.cu/cuh, mmvq.cu, vecdotq.cuh,\n quantize.cu/cuh, mma.cuh, ggml-cuda.cu, fattn-tile.cuh\n- ggml-metal: ggml-metal.metal, ggml-metal-device.cpp, ggml-metal-impl.h,\n ggml-metal-ops.cpp\n- ggml-vulkan: ggml-vulkan.cpp, all vulkan-shaders/*\n- ggml-cpu arch: arm/quants.c, x86/quants.c, powerpc/quants.c, s390/quants.c\n\nCore NVFP4 support (type definition, CPU fallback dot product,\nquantization, dequantization, conversion) is retained.\n\n* Fix arch-fallback.h: add NVFP4 generic fallback for all platforms\n\nAfter shelving backend-specific SIMD implementations, the generic\nCPU dot product needs to be aliased on ARM, x86, PowerPC, and s390\nplatforms that previously relied on arch-specific versions.\n\n* quantize: add NVFP4 as a quantization type option\n\n* Fix ggml_fp32_to_ue4m3: handle subnormal values\n\nPreviously, values with ue4m3_exp <= 0 were clamped to 0, causing\nall small scales to underflow. This made NVFP4 quantization via\nllama-quantize produce garbage (PPL = 5.8M) since typical transformer\nweights have amax/6.0 in the range 0.001-0.01, which falls in the\nUE4M3 subnormal range.\n\nNow subnormals are properly encoded as man * 2^-9 (exp=0, man=1..7),\nmatching the decode path in ggml_ue4m3_to_fp32.\n\nResult: NVFP4 requantization now produces PPL = 15.25 (vs F16 = 14.33),\ncomparable to Q4_1 (PPL = 15.81) at slightly lower BPW (4.70 vs 5.15).\n\n* Restore ARM NEON NVFP4 dot product implementation\n\nRestores the optimized ggml_vec_dot_nvfp4_q8_0 for ARM NEON using\nvqtbl1q_s8 lookup and ggml_vdotq_s32 dot products.\n\ntg128 performance: 4.37 t/s (generic) -> 13.66 t/s (NEON) = 3.1x speedup\n\n* Optimize ARM NEON NVFP4 dot product: LUT + vpaddq + vfmaq\n\n- Add ue4m3_scale_lut[128] to ggml-common.h replacing branch-heavy\n ggml_ue4m3_to_fp32() in the hot loop\n- Use vpaddq_s32 for pairwise int32 reduction instead of vaddvq_s32\n- Accumulate with vfmaq_f32 into float32x4_t vector accumulators\n\ntg128: 8.1 -> 31.0 t/s (3.8x speedup, 77% of Q4_1 speed)\n\n* ARM NEON NVFP4: rearrange q8 to match nibble layout\n\nAlternative approach: rearrange q8 data to match the NVFP4 lo/hi\nnibble layout instead of rearranging the looked-up NVFP4 values.\nEliminates vcombine_s8(vget_low, vget_low) shuffles.\n\nPerformance is equivalent (~18.5 t/s) - the bottleneck is the 2x\nblock overhead from QK=16 vs QK=32, not the shuffle instructions.\n\n* CPU only backend 64 super-block layout\n\n* cleanup\n\n* Remove unused LUT\n\n* int\n\n* exclude NVFP4 from unsupported ops in metal build\n\n* remove quantization for now\n\n* store scales as native UE4M3, preserve original model bits when possible\n\n* Update convert_hf_to_gguf.py\n\nCo-authored-by: Sigbjørn Skjæret \n\n* correct comment\n\n* format\n\n* reduce duplication and cleanup\n\n* Address comments\n\n* move detection to prepare_tensors\n\n* Use math instead of const\n\n* Move\n\n* fix comment\n\n* Shelf quantize tests\n\n* Rebase and move check\n\n* cleanup\n\n* lint\n\n* Update gguf-py/gguf/scripts/gguf_convert_endian.py\n\nCo-authored-by: Sigbjørn Skjæret \n\n* Use fallback quant config\n\n* Simplify\n\nCo-authored-by: Sigbjørn Skjæret \n\n* organize\n\n* Refactor\n\n* Update convert_hf_to_gguf.py\n\nCo-authored-by: Sigbjørn Skjæret \n\n* Update convert_hf_to_gguf.py\n\nCo-authored-by: Sigbjørn Skjæret \n\n* Update convert_hf_to_gguf.py\n\nCo-authored-by: Sigbjørn Skjæret \n\n* add quantize_nvfp4 (required for test_quants.py)\n\n* add quantize_nvfp4 (required for test_quants.py)\n\n* add quantize_nvfp4 (required for test_quants.py)\n\n* fix return type\n\n---------\n\nCo-authored-by: Sigbjørn Skjæret ","tree":"8b9a28d5fa6c8f84bff57959960dac7178b73e3c"}}],"lastCommit":{"hash":"5788b510a1e3394fcc2d6b13ba2d9d8fc4a5c139","oid":"5788b510a1e3394fcc2d6b13ba2d9d8fc4a5c139","parents":["2e17f69ef4e7fcb9ee950362b3530d8e4e2ea316"],"author":{"name":"hcl","email":"chenglunhu@gmail.com","when":"2026-08-04T17:12:48+08:00"},"committer":{"name":"GitHub","email":"noreply@github.com","when":"2026-08-04T12:12:48+03:00"},"message":"gguf-py: validate n_dims and guard against uint64 overflow in reader (#25401)\n\nThe Python GGUF reader lacked two guards the C++ loader has:\n- n_dims read as uint32 with no GGML_MAX_DIMS bound -> crafted file with\n huge n_dims triggers oversized memmap read / OOM.\n- np.prod(dims) on uint64 wraps silently -> a crafted dims triple can\n overflow to a tiny element count, passing an undersized read through.\n\nAdd a GGML_MAX_DIMS check and compute the element count with Python ints.\n\nFixes #25378","tree":"154141302cb3b2baf68f0b5cc9fa56e0cc31452e"}}