SGLang is a high-performance serving framework for large language models and multimodal models.
Python 38.6k 10.2k
SGLang's TorchTPU Plugin
Python 74 38
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
Python 1.3k 386
High Performance Serving Runtime For Interactive Super Intelligence System
Python 1.7k 702
A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.
Python 5.5k 968
JAX backend for SGL
Python 374 162
SGLang kernel library for NPU
FlashMLA: Efficient Multi-head Latent Attention Kernels
DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling
SGLang Kernel Wheel Index
