Pre-built release container images on NGC#
Pre-built TensorRT LLM releases are made available as container imageson NGC. This is likely the simplest way to obtain TensorRT LLM. Please refer to thedocumentation in NGC for usage instructions.
Container image tags
In the example shell commands,x.y.z corresponds to the TensorRT-LLM containerversion to use. If omitted,IMAGE_TAG will default totensorrt_llm.__version__(e.g., this documentation was generated from the1.2.0rc4 source tree).If this does not work, e.g., because a container for the version you arecurrently working with has not been released yet, you can try using acontainer published for a previousGitHub pre-release or release(see alsoNGC Catalog).
Containers can also be built locally, seeNVIDIA/TensorRT-LLMfor all related options.