Instructions to use apertixai/cerno-4b-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use apertixai/cerno-4b-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf apertixai/cerno-4b-GGUF:Q4_0 # Run inference directly in the terminal: llama cli -hf apertixai/cerno-4b-GGUF:Q4_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf apertixai/cerno-4b-GGUF:Q4_0 # Run inference directly in the terminal: llama cli -hf apertixai/cerno-4b-GGUF:Q4_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf apertixai/cerno-4b-GGUF:Q4_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf apertixai/cerno-4b-GGUF:Q4_0
Use Docker
docker model run hf.co/apertixai/cerno-4b-GGUF:Q4_0
- Llama new
- LM Studio
- Jan
- Ollama
How to use apertixai/cerno-4b-GGUF with Ollama:
ollama run hf.co/apertixai/cerno-4b-GGUF:Q4_0
- Unsloth Desktop
- Pi
How to use apertixai/cerno-4b-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf apertixai/cerno-4b-GGUF:Q4_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "apertixai/cerno-4b-GGUF:Q4_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use apertixai/cerno-4b-GGUF with Docker Model Runner:
docker model run hf.co/apertixai/cerno-4b-GGUF:Q4_0
- Lemonade
How to use apertixai/cerno-4b-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull apertixai/cerno-4b-GGUF:Q4_0
Run and chat with the model
lemonade run user.cerno-4b-GGUF-Q4_0
List all available models
lemonade list
- Hermes Agent
How to use apertixai/cerno-4b-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf apertixai/cerno-4b-GGUF:Q4_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default apertixai/cerno-4b-GGUF:Q4_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use apertixai/cerno-4b-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf apertixai/cerno-4b-GGUF:Q4_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "apertixai/cerno-4b-GGUF:Q4_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Cerno 4B Q4_0 (GGUF)
The on-device build of Cerno 4B (v1.1 weights) Β· 4-bit Β· llama.cpp Β· one forward pass Β· calibrated probabilities
Cerno 27B Β· Cerno 9B Β· Cerno 4B Β· 9B GGUF Β· 4B GGUF Β· Inference code
What it is
The 4-bit llama.cpp build of Cerno 4B (made from the v1.1 weights) for phones, laptops and anything llama.cpp runs on. Same interface as the FP8 model: a state and a set of typed questions, a probability for every option of every question in one forward pass, no text generation. Made with quantisation-aware training to llama.cpp's Q4_0 grid, so it keeps the quality of the full-precision model:
| Cerno 4B v1.1, FP8 (vLLM) | this file, Q4_0 (llama.cpp) | |
|---|---|---|
| Decision Index 0.3 public index, complete run | 57.60 | 57.23 |
| requests answered | 140,176 of 140,178 | 140,178 of 140,178 |
| answers identical to the FP8 model on the kit's 400-request sample | 98.5% |
The run through llama.cpp used the maintainers' kit at commit 9eb2dbe, llama-server on one H200 with the public engine
(evals/decision_index/engine_llama.py, greedy read-out), every request answered, nothing truncated. Run-to-run noise on the public
index is about 0.1-0.2 points.
Format
- Body: every linear weight in Q4_0 (block-32 symmetric 4-bit, the format the Snapdragon NPU, the Adreno GPU and the ARM CPU paths of llama.cpp all accelerate). Head and embeddings: Q8_0. Exactly the forward the model was trained for; do not re-quantise it to another mix.
- File:
cerno-4b-v1.1-q4_0.gguf(3.37 GB), SHA-256 inSHA256SUMS. - Temperature: the shipped calibration temperature (1.534) lives in
decider_config.jsonnext to the FP8 weights (apertixai/cerno-4b); the engine reads it from there.
Run it
# 1. serve the file (any llama.cpp build; on Android see the notes below)
llama-server -m cerno-4b-v1.1-q4_0.gguf --host 127.0.0.1 --port 8080 -c 32768 -b 8192 -ub 2048 -fa on -ngl 99
# 2. score the Decision Index sample or the whole suite with the public engine
git clone https://github.com/apertixsk/cerno-inference && cd cerno-inference
pip install -r requirements.txt && export PYTHONPATH=$PWD:$PWD/third_party/decider
python -m decision_index pipeline --edition 0.3 --engine evals.decision_index.engine_llama:LlamaGGUFEngine \
--option model=apertixai/cerno-4b --option gguf=cerno-4b-v1.1-q4_0.gguf --option base_url=http://127.0.0.1:8080 \
--out runs/cerno-4b-q4_0
model= points at the FP8 repo only for the tokenizer and decider_config.json; no FP8 weights are downloaded. Programmatic use
is the same as for the FP8 model (engine(state, questions)), see the inference repository.
Android / Snapdragon: build llama.cpp with its Snapdragon preset (Hexagon NPU + Adreno OpenCL + CPU in one binary, see
docs/backend/snapdragon/README.md upstream), push the file, start llama-server on the phone with --devices HTP0:0 (NPU),
--devices GPUOpenCL (GPU) or -ngl 0 (CPU), forward the port with adb forward tcp:8080 tcp:8080, and run the same command
from the laptop. The workload is one forward pass per question, so prompt-processing throughput is the whole latency.
Family
| Model | Weights | Public index | On-device build |
|---|---|---|---|
| Cerno 27B | cerno-27b (FP8) |
65.19 | |
| Cerno 9B v1.1 | cerno-9b (FP8) |
58.59 | cerno-9b-GGUF (57.96) |
| Cerno 4B v1.1 | cerno-4b (FP8) |
57.60 | cerno-4b-GGUF (57.23) |
Licence and credits
Apache-2.0, like the FP8 weights it was made from (base model Qwen3.5-4B-Base, Apache-2.0). Quantisation format by the llama.cpp project. Jev and System One are TypeSafe AI's names; this project is not affiliated with TypeSafe AI or with the Decision Index maintainers.
- Downloads last month
- 176
4-bit
Model tree for apertixai/cerno-4b-GGUF
Evaluation results
- public index, complete run through llama.cpp on Decision Index 0.3 public suiteself-reported57.230