Skip to main content
This guide shows how to run an agent harness fully locally with an LFM. The pattern is the same for every harness: they all talk to an OpenAI-compatible endpoint, so you serve the model once and then point your agent harness of choice, such as Hermes Agent, OpenClaw, and Pi, at it.

Serve the model locally

Any server that exposes an OpenAI-compatible /v1 endpoint works. Install one backend and start it with tool calling enabled. Each backend serves on its own default port, so note the local URL yours prints. You point your harness at that URL.
Each backend uses its own default port, so your endpoint depends on which one you run. llama.cpp and MLX use 8080, vLLM and zml/llmd use 8000, SGLang uses 30000, LM Studio uses 1234, and Atomic Chat uses 1337. The examples in this guide use http://localhost:8080/v1. When you configure a harness, replace the port with your server’s.

Model configuration

LFM3-1.7B is a 1.7B-parameter text model post-trained for agent harnesses such as Hermes Agent, OpenClaw, and Pi. It runs fast on consumer hardware and supports native tool calling, which makes it a good fit for agentic workloads. The two settings worth choosing up front are the quantization and the context length. Both trade memory for quality or capacity, so pick them to fit your hardware. Quantization. Because LFM3-1.7B is small, you have room to trade size for quality. For the GGUF quants, which cover llama.cpp and LM Studio, we recommend starting with Q4_K_M and stepping up to Q8_0 or BF16 depending on your available memory. MLX uses its own quantization. Pick the 4-bit or bf16 build. vLLM and SGLang run the full-precision weights on GPU, and zml/llmd runs the FP8 weights. Need more capability? LFM2.5-24B-A2B is a 24B-parameter reasoning model with only 2B active parameters per token, post-trained inside Hermes Agent, OpenClaw, Pi, and OpenCode. It handles longer, multi-step agent tasks and agentic coding more reliably, at the cost of more memory. The backend examples below, except zml/llmd, work with it if you swap the model ID. Context length. Agents consume context quickly. If you hit truncation or context-overflow errors mid-run, raise the served context or trim the agent’s history. LFM3-1.7B supports up to 256K tokens. The examples serve the full window, but if you’re memory-constrained, serve a smaller window such as 32K tokens, which is usually plenty for a single agent task.

Start a server

Install one backend and start it with tool calling enabled.
Install:
LFM3 requires llama.cpp b11960 or newer. For Linux and build-from-source options, see the llama.cpp guide.Run:The -hf flag auto-downloads the GGUF.
Check that the model is loaded and reachable (replace 8080 with your server’s port):

Connect your agent harness

Every harness connects the same way: install it, point it at your local server, then run. The examples below use http://localhost:8080/v1 and model ID LFM3-1.7B. Replace the port with the one your server prints and the model ID with the one returned by /v1/models. For zml/llmd, use http://localhost:8000/v1 and LFM3-1.7B-FP8. Only the exact commands differ per harness.
Docs: Custom / self-hosted providers.Install:
Configure:Use the interactive wizard:
Or set it directly, then enable tool-use enforcement (without it, the model tends to describe actions instead of calling tools):
Run:
[!Note] If web_search is missing from the model’s available tools, it may be due to search or browser being listed in agent.disabled_toolsets. Remove both entries in hermes config edit and restart Hermes.
Now, you have your agent harness running fully locally on your machine.

References