Skip to content

Releases: ggml-org/llama.cpp

b10893

b10893 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Sep 12:40
c32d1da

b10892

b10892 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Sep 11:54
e5a8d43

b10891

b10891 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Sep 11:26
3ff67eb

vulkan: fall back to shared-memory reduction for dmmv on PowerVR (#28341)

The Imagination proprietary Vulkan compiler returns VK_ERROR_UNKNOWN from
vkCreateComputePipelines for every dequant mul_mat_vec shader built with the
subgroup-only reduction that requires a subgroup size >= 16. That covers the
k-quants, the i-quants, TQ2_0, MXFP4 and NVFP4. ggml rethrows, so the first
generated token of any such model kills the process.

Reproduced on a Pixel 11 Pro (PowerVR C-Series CXTP-48-1536 MC1, driver
1.662.3024, subgroup size 128, min 32, max 128). The failure is independent of
subgroup size: 32, 64 and 128 all fail, as does dropping the full-subgroups
flag and the required-subgroup-size pNext. The legacy quants, which use the
plain subgroup reduction, compile and run fine.

The shared-memory reduction variant compiles and matches the CPU reference for
q2_K, q3_K, q4_K, q5_K and q6_K. The hybrid variant also compiles but costs
27% of token throughput (3.78 vs 5.20 t/s on Qwen3.5-2B-Q4_K_M).

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b10889

b10889 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Sep 10:04
311d421

b10888

b10888 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Sep 09:39
72797e8

vulkan : add command-buffer debug labels for GPU profilers (#28101)

  • vulkan : add command-buffer debug labels for GPU profilers

Co-authored-by: gabby-zy z2262718160@gmail.com
Assisted-by: Claude Code

  • vulkan : close the queue debug label with the label struct

Co-authored-by: gabby-zy z2262718160@gmail.com

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b10887

b10887 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Sep 09:07
4ea6d1b

b10886

b10886 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Sep 08:32
f1b6fbf

ggml-cpu(s390x): add Q1_0 vector intrinsic support (#28606)

  • ggml-cpu: add ggml_vec_dot_q1_0_q8_0 support

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • ggml-cpu: clean up variable naming for understanding

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • docs: update support for Q1_0

Signed-off-by: Aaron Teo aaron.teo1@ibm.com


Signed-off-by: Aaron Teo aaron.teo1@ibm.com

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b10885

b10885 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Sep 07:51
d7e8643

model: fix all granite family parameter counts (#28643)

  • model: fix all granite family parameter counts

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • model: fix additional include, add missing A prefix for active experts

Signed-off-by: Aaron Teo aaron.teo1@ibm.com

  • model: fix code alignment, rm unused 40 block case

Signed-off-by: Aaron Teo aaron.teo1@ibm.com


Signed-off-by: Aaron Teo aaron.teo1@ibm.com

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b10883

b10883 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 09 Sep 17:29
91f6a6c

vulkan: use spec constant for matrix matrix multiplication A-type (#25773)

  • vulkan: use spec constant for mul mat type_a

vulkan: use map for mul_mm shapes

cleanup

fix indentation

fix cm2 and shmem init

fix cm2 spec constants

fix cm2 bindings

consolidate shmem tables and reduce size by type spec constant

fix compiler warning

fix missing Q2_0 type

fix unused warning when integer dot glslc support is missing

use minimal shmem size 8 instead of 1 to workaround cm2 compiler bug

fix missing Q2_0 type in cm2 matmul

fix types

  • remove LUT quants from unified shader

  • clean up

  • restore coopmat2 q4_k/q5_k optimization

  • split out q4_k/q5_k cm2 shader to fix Ampere regression

  • revert iq shmem table renames

  • simplify cm2 code with single uint8_t buffer

  • fix fp4 extension use switch being overwritten by generic shader

  • clean up

  • adapt TQ1_0 changes

  • adapt #27471 f16 Intel tuning changes

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b10881

b10881 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 09 Sep 15:32
22397c3

vulkan: Convert FILL to distribute workgroups in 2D to avoid exceeding maxComputeWorkGroupCount (#28592)

  • divide workload to 2D

This is to workaround FILL exceeding maxComputeWorkGroupCount for Intel GPUs on Qwen 3.8 flash next

  • minor change

  • Fixed comment

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI: