This is a repository containing the tools for testing vLLM correctness and perf regression

vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs

ROCm support of Ultra-Fast and Cheaper Long-Context LLM Inference

A high-throughput and memory-efficient inference and serving engine for LLMs

NVIDIA Inference Xfer Library (NIXL)

AI Tensor Engine for ROCm

The collaborative spreadsheet for AI. Chain cells into powerful pipelines, experiment with prompts and models, and evaluate LLM responses in real-time. Work together seamlessly to build and iterate on AI applications.

This is a repository to monitor the fast changing ROCm/aiter repository to alert user that AITER function of interests e.g. in vLLM, in SGLang has been updated at certain commit.

vLLM Workshop Content