vLLM

Pinned Loading

1. vllm vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 96.5k 24.5k

2. vllm-omni vllm-omni Public

   A framework for efficient model inference with omni-modality models

   Python 7.6k 2.1k

3. recipes recipes Public

   Common recipes to run vLLM

   JavaScript 1.1k 489

4. llm-compressor llm-compressor Public

   State-of-the-art LLM compression, built for production inference with vLLM

   Python 4k 729

5. speculators speculators Public

   A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM

   Python 942 285

6. semantic-router semantic-router Public

   An open, programmable decision layer for models and compute.

   Go 6.4k 1.1k