Repository navigation
Feature matrix
Eve edited this page Nov 8, 2026
·
14 revisions
| CPU (AVX/AVX2) | CPU (ARM NEON) | Metal | CUDA | ROCm | SYCL | Vulkan | |
|---|---|---|---|---|---|---|---|
| K-quants | β | β | β | β | β | β | β β΅ |
| I-quants | β π’β΄ | β π’β΄ | β π’β΄ | β | β | PartialΒΉ | β π’β΄ |
| Parallel Multi-GPU | N/A | N/A | N/A | β | β | Sequential only | β |
| K cache quants | β | β | β | β | β | β | β |
| MoE architecture | β | β | β | β | β | β | β |
| Flash Attention | β | β | β | β | β | β | β |
- β : feature works
- π«: feature does not work
- β: unknown, please contribute if you can test it yourself
- π’: feature is slow
- ΒΉ: IQ3_S and IQ1_S, see #5886
- β΄: Slower than K-quants of comparable size
- β΅: Generally the CUDA or ROCM backends are faster than Vulkan in most cases. See #10879 for benchmarks.
Useful information for users that doesn't fit into Readme.
- Home
- Feature Matrix
- GGML Tips & Tricks
- Chat Templating
- Metadata Override
- HuggingFace Model Card Metadata Interoperability Consideration
These are information useful for Maintainers and Developers which does not fit into code comments
Click on a badge to jump to workflow. This is here as a useful general view of all the actions so that we may notice quicker if main branch automation is broken and where.