Installation#

Setup using Python#

Prerequisites#

  • OS: Linux

  • Python: 3.10 – 3.12

  • GPU: NVIDIA compute capability 7.0+ (e.g., V100, T4, RTX20xx, A100, L4, H100, etc.)

  • CUDA 12.8+

Note

LMCache does not support Windows natively. To run LMCache on Windows, you can use the Windows Subsystem for Linux (WSL) with a compatible Linux distribution, or use some community-maintained forks.

Install Stable LMCache from PyPI#

The simplest way to install the latest stable release of LMCache is through PyPI:

pipinstalllmcache

Install Latest LMCache from TestPyPI#

These wheels are continually built from the latest LMCache source code (not officially stable release).

pipinstall--index-urlhttps://pypi.org/simple--extra-index-urlhttps://test.pypi.org/simplelmcache==0.2.2.dev57

See the latest pre-release of LMCache:latest LMCache pre-releases and replace0.2.2.dev57 with the latest pre-release version.

This will install all dependencies from the real PyPI and only LMCache itself from TestPyPI.

Confirm that you have the latest pre-release:

python>>>importlmcache>>>fromimportlib.metadataimportversion>>>print(version("lmcache"))0.2.2.dev57# should be the latest pre-release version you installed

Install Latest LMCache from Source#

To install from source, clone the repository and install in editable mode:

gitclonehttps://github.com/LMCache/LMCache.gitcdLMCachepipinstall-e.

Install LMCache with uv#

We recommend developers to useuv for a better package management:

gitclonehttps://github.com/LMCache/LMCache.gitcdLMCacheuvvenv--python3.12source.venv/bin/activateuvpipinstall-e.

LMCache with vLLM v1#

LMCache is integrated with the latest vLLM (vLLM v1). To use it, install the latest vLLM package:

pipinstallvllm

Test whether LMCache works with vLLM v1 by running:

python3-c"import vllm.distributed.kv_transfer.kv_connector.v1.lmcache_connector"

LMCache with vLLM v0#

Note

LMCache is also integrated with vLLM v0. Refer tothe example in vLLM.See theexamples README to understand how to run the script for vLLM v0.

Setup using Docker#

Prerequisites#

  • Docker Engine 27.0+

Pre-built LMCache integrated with vLLM Images#

We provide pre-built container images of LMCache integrated with vLLM.

You can get the latest stable image as follows:

dockerpulllmcache/vllm-openai

You can get the nightly build of latest code of LMcache and vLLM as follows:

dockerpulllmcache/vllm-openai:latest-nightly