Skip to content

About

An open, programmable decision layer for models and compute.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

6.5k stars

Watchers

61 watching

Forks

Latest commit

 

History

3,505 Commits

Folders and files

vLLM Semantic Router

An open, programmable decision layer for models and compute.

Documentation | System Duo | Blog | Publications | Hugging Face | Slack

vllm-project%2Fsemantic-router | Trendshift Decision 3.0 — #1 on Hugging Face Trending Collections, October 14, 2026

Main GitHub Release Go Ask DeepWiki


About

Intelligence beyond any one model.

Give your agent harness one API for many models. vLLM Semantic Router selects or combines models for each call, guided by your policy.

Your harness keeps the agent loop, tools, and task state. The Router chooses among configured backends across local, private, and cloud compute.

Dimension Fragmented today With vLLM SR
Models Different models excel at different tasks. Select or combine models.
Compute Hardware varies in speed and capacity. Choose among configured backends.
Location Edge, private, and cloud. Keep calls within approved locations.
Preference Priorities change by task. Set quality, latency, and cost priorities.

Explore how it works →

Getting Started

Install

curl -fsSL https://vllm-sr.ai/install.sh | bash -s -- --channel stable

For pip, uv, or agent-driven installation, see the Installation Guide.

Try it online

Try System Duo: pick an example and it runs live in your browser, with a free key on first run.

Latest News

Earlier announcements

More announcements are available on the Blog and Publications pages.

Community

For questions, feedback, or to contribute, please join the #semantic-router channel in vLLM Slack. Track contributors, workgroups, and weekly activity at community.vllm-sr.ai.

Community Meetings

We host two monthly community meetings across APAC and the Americas:

  • APAC-friendly meeting — second Wednesday of the month: 9:00-10:00 AM Singapore time (UTC+8; the same local time in Beijing)
  • Americas-friendly meeting — fourth Wednesday of the month: 8:00-9:00 PM Eastern Time (America/New_York) / 5:00-6:00 PM Pacific Time

Contributing

If you want to contribute, start with CONTRIBUTING.md.

For repository-native development workflow and validation commands, use AGENTS.md as the entrypoint and tools/agent/docs/README.md as the canonical index.

Citation

If you find Semantic Router helpful in your research or projects, please consider citing it:

@misc{semanticrouter2025,
  title={vLLM Semantic Router},
  author={vLLM Semantic Router Team},
  year={2025},
  howpublished={\url{https://github.com/vllm-project/semantic-router}},
}

Ecosystem & partnerships

An open ecosystem spanning research, infrastructure, and enterprise adoption.

About

An open, programmable decision layer for models and compute.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

6.5k stars

Watchers

61 watching

Forks

Releases

Packages

Used by

Contributors

Languages