TPU inference for vLLM, with unified JAX and PyTorch support.

A high-throughput and memory-efficient inference and serving engine for LLMs

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Community maintained hardware plugin for vLLM on Intel Gaudi

System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge

Performance dashboard for vLLM

Community maintained hardware plugin for vLLM on Apple Silicon

Cost-efficient and pluggable Infrastructure components for GenAI inference

vLLM Daily Summarization of Merged PRs

vLLM Model plugin for the encoder-decoder BART model