OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)

This is the documentation repository for SGLang. It is auto-generated fromhttps://github.com/sgl-project/sglang/tree/main/docs.

SGLang is a fast serving framework for large language models and vision language models.

Train speculative decoding models effortlessly and port them smoothly to SGLang serving.

A workload for deploying LLM inference services on Kubernetes

JAX backend for SGL

SGLang kernel library for NPU

Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serving systems.

SGLang kernel library for Intel XPU

Kernel Library Wheel for SGLang