Repository files navigation

TensorRT 11.0 is coming soon in 2026 Q2 with powerful new capabilities designed to accelerate your AI inference workflows. With this major version bump, TensorRT's API will be streamlined and a few legacy features will be removed.

We recommend migrating early for the following features:

• Weakly-typed networks and related APIs will be removed, replaced byStrongly Typed Networks.
• Implicit quantization and related APIs will be removed, replaced byExplicit Quantization
• IPluginV2 and related APIs will be removed, replaced byIPluginV3
• TREX tool will be removed, replaced byNsight Deep Learning Designer
• Python bindings for Python 3.9 and older versions will be removed starting TensorRT 10.16. RPM packages for RHEL/Rocky Linux 8 and RHEL/Rocky Linux 9 now depend on Python 3.12.

This repository contains the Open Source Software (OSS) components of NVIDIA TensorRT. It includes the sources for TensorRT plugins and ONNX parser, as well as sample applications demonstrating usage and capabilities of the TensorRT platform. These open source software components are a subset of the TensorRT General Availability (GA) release with some extensions and bug-fixes.

• For code contributions to TensorRT-OSS, please see ourContribution Guide andCoding Guidelines.
• For a summary of new additions and updates shipped with TensorRT-OSS releases, please refer to theChangelog.
• For business inquiries, please contactresearchinquiries@nvidia.com
• For press and other inquiries, please contact Hector Marinez athmarinez@nvidia.com

Need enterprise support? NVIDIA global support is available for TensorRT with theNVIDIA AI Enterprise software suite. Check outNVIDIA LaunchPad for free access to a set of hands-on labs with TensorRT hosted on NVIDIA infrastructure.

Join theTensorRT and Triton community and stay current on the latest product updates, bug fixes, content, best practices, and more.

We provide the TensorRT Python package for an easy installation.
To install:

pip install tensorrt

You can skip theBuild section to enjoy TensorRT with Python.

To build the TensorRT-OSS components, you will first need the following software packages.

TensorRT GA build

• TensorRT v10.15.1.29
  • Available from direct download links listed below

System Packages

• CUDA
  • Recommended versions:
  • cuda-13.1.0
  • cuda-12.9.0
• CUDNN (optional)
  • cuDNN 8.9
• GNU make >= v4.1
• cmake >= v3.31
• python >= v3.10, <= v3.13.x
• pip >= v19.0
• Essential utilities
  • git,pkg-config,wget

Optional Packages

• Containerized build

  • Docker >= 19.03
  • NVIDIA Container Toolkit

• PyPI packages (for demo applications/tests)

  • onnx
  • onnxruntime
  • tensorflow-gpu >= 2.5.1
  • Pillow >= 9.0.1
  • pycuda < 2021.1
  • numpy
  • pytest

• Code formatting tools (for contributors)

  • Clang-format
  • Git-clang-format

  NOTE:onnx-tensorrt,cub, andprotobuf packages are downloaded along with TensorRT OSS, and not required to be installed.

1. Download TensorRT OSS

   git clone -b main https://github.com/nvidia/TensorRT TensorRTcd TensorRTgit submodule update --init --recursive

2. (Optional - if not using TensorRT container) Specify the TensorRT GA release build path

   If using the TensorRT OSS build container, TensorRT libraries are preinstalled under/usr/lib/x86_64-linux-gnu and you may skip this step.

   Else download and extract the TensorRT GA build fromNVIDIA Developer Zone with the direct links below:

   • TensorRT 10.15.1.29 for CUDA 13.1, Linux x86_64
   • TensorRT 10.15.1.29 for CUDA 12.9, Linux x86_64
   • TensorRT 10.15.1.29 for CUDA 13.1, Windows x86_64
   • TensorRT 10.15.1.29 for CUDA 12.9, Windows x86_64

   Example: Ubuntu 22.04 on x86-64 with cuda-13.1

   cd~/Downloadstar -xvzf TensorRT-10.15.1.29.Linux.x86_64-gnu.cuda-13.1.tar.gzexport TRT_LIBPATH=`pwd`/TensorRT-10.15.1.29/lib

   Example: Windows on x86-64 with cuda-12.9

   Expand-Archive-Path TensorRT-10.15.1.29.Windows.win10.cuda-12.9.zip$env:TRT_LIBPATH="$pwd\TensorRT-10.15.1.29\lib"

For Linux platforms, we recommend that you generate a docker container for building TensorRT OSS as described below. For native builds, please install theprerequisiteSystem Packages.

1. Generate the TensorRT-OSS build container.

   Example: Ubuntu 24.04 on x86-64 with cuda-13.1 (default)

   ./docker/build.sh --file docker/ubuntu-24.04.Dockerfile --tag tensorrt-ubuntu24.04-cuda13.1

   Example: Rockylinux8 on x86-64 with cuda-13.1

   ./docker/build.sh --file docker/rockylinux8.Dockerfile --tag tensorrt-rockylinux8-cuda13.1

   Example: Ubuntu 24.04 cross-compile for Jetson (aarch64) with cuda-13.1 (JetPack SDK)

   ./docker/build.sh --file docker/ubuntu-cross-aarch64.Dockerfile --tag tensorrt-jetpack-cuda13.1

   Example: Ubuntu 24.04 on aarch64 with cuda-13.1

   ./docker/build.sh --file docker/ubuntu-24.04-aarch64.Dockerfile --tag tensorrt-aarch64-ubuntu24.04-cuda13.1

2. Launch the TensorRT-OSS build container.

   Example: Ubuntu 24.04 build container

   ./docker/launch.sh --tag tensorrt-ubuntu24.04-cuda13.1 --gpus all

   NOTE:
   1. Use the--tag corresponding to build container generated in Step 1.
   2.NVIDIA Container Toolkit is required for GPU access (running TensorRT applications) inside the build container.
   3.sudo password for Ubuntu build containers is 'nvidia'.
   4. Specify port number using--jupyter <port> for launching Jupyter notebooks.
   5. Write permission to this folder is required as this folder will be mounted inside the docker container for uid:gid of 1000:1000.

• Generate Makefiles and build

  Example: Linux (x86-64) build with default cuda-13.1

  cd$TRT_OSSPATHmkdir -p build&&cd buildcmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/outmake -j$(nproc)

  Example: Linux (aarch64) build with default cuda-13.1

  cd$TRT_OSSPATHmkdir -p build&&cd buildcmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64-native.toolchainmake -j$(nproc)

  Example: Native build on Jetson Thor (aarch64) with cuda-13.1

  cd$TRT_OSSPATHmkdir -p build&&cd buildcmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DTRT_PLATFORM_ID=aarch64CC=/usr/bin/gcc make -j$(nproc)

  NOTE: C compiler must be explicitly specified via CC= for native aarch64 builds of protobuf.

  Example: Ubuntu 24.04 Cross-Compile for Jetson Thor (aarch64) with cuda-13.1 (JetPack)

  cd$TRT_OSSPATHmkdir -p build&&cd buildcmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_cross.toolchainmake -j$(nproc)

  Example: Ubuntu 24.04 Cross-Compile for DriveOS (aarch64) with cuda-13.1

  cd$TRT_OSSPATHmkdir -p build&&cd buildcmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_dos_cross.toolchainmake -j$(nproc)

  Example: Native builds on Windows (x86) with cuda-13.1

  cd$TRT_OSSPATHNew-Item -ItemType Directory -Path buildcd buildcmake .. -DTRT_LIB_DIR="$env:TRT_LIBPATH" -DTRT_OUT_DIR="$pwd\\out"msbuild TensorRT.sln /property:Configuration=Release -m:$env:NUMBER_OF_PROCESSORS

  NOTE: The default CUDA version used by CMake is 13.1. To override this, for example to 12.9, append-DCUDA_VERSION=12.9 to the cmake command.

• Required CMake build arguments are:

  • TRT_LIB_DIR: Path to the TensorRT installation directory containing libraries.
  • TRT_OUT_DIR: Output directory where generated build artifacts will be copied.

• Optional CMake build arguments:

  • CMAKE_BUILD_TYPE: Specify if binaries generated are for release or debug (contain debug symbols). Values consists of [Release] |Debug
  • CUDA_VERSION: The version of CUDA to target, for example [12.9.9].
  • CUDNN_VERSION: The version of cuDNN to target, for example [8.9].
  • PROTOBUF_VERSION: The version of Protobuf to use, for example [3.20.1]. Note: Changing this will not configure CMake to use a system version of Protobuf, it will configure CMake to download and try building that version.
  • CMAKE_TOOLCHAIN_FILE: The path to a toolchain file for cross compilation.
  • BUILD_PARSERS: Specify if the parsers should be built, for example [ON] |OFF. If turned OFF, CMake will try to find precompiled versions of the parser libraries to use in compiling samples. First in${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
  • BUILD_PLUGINS: Specify if the plugins should be built, for example [ON] |OFF. If turned OFF, CMake will try to find a precompiled version of the plugin library to use in compiling samples. First in${TRT_LIB_DIR}, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.
  • BUILD_SAMPLES: Specify if the samples should be built, for example [ON] |OFF.
  • BUILD_SAFE_SAMPLES: Specify if safety samples should be built, for example [ON] |OFF.
  • TRT_SAFETY_INFERENCE_ONLY: Specify if only build the safety inference components, for example [ON] |OFF. If turned ON, all other components will be turned OFF exceptBUILD_SAFE_SAMPLES.
  • GPU_ARCHS: GPU (SM) architectures to target. By default we generate CUDA code for all major SMs. Specific SM versions can be specified here as a quoted space-separated list to reduce compilation time and binary size. Table of compute capabilities of NVIDIA GPUs can be foundhere. Examples: - NVidia A100:-DGPU_ARCHS="80" - RTX 50 series:-DGPU_ARCHS="120" - Multiple SMs:-DGPU_ARCHS="80 120"
  • TRT_PLATFORM_ID: Bare-metal build (unlike containerized cross-compilation). Currently supported options:x86_64 (default).

• Generate Makefiles and build

  Example: Cross-Compile for DOS7 Linux (aarch64)

  cd$TRT_OSSPATHmkdir -p build&&cd buildcmake .. -DBUILD_SAMPLES=ON -DBUILD_PLUGINS=OFF -DBUILD_PARSERS=OFF -DTRT_OUT_DIR=`pwd`/bin_dynamic_cross -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_dos_cross.toolchainmake -j$(nproc)

  Example: Cross-Compile for DOS6.5 Linux (aarch64)

  cd$TRT_OSSPATHmkdir -p build&&cd buildcmake .. -DBUILD_SAMPLES=ON -DBUILD_PLUGINS=OFF -DBUILD_PARSERS=OFF -DTRT_OUT_DIR=`pwd`/bin_dynamic_cross -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64_dos_cross.toolchain -DCUDA_VERSION=11.4 -DGPU_ARCHS=87make -j$(nproc)

  Example: Native build for DOS6.5 and DOS7 Linux (aarch64)

  cd$TRT_OSSPATHmkdir -p build&&cd buildcmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64-native.toolchain -DBUILD_SAMPLES=ON -DBUILD_PLUGINS=OFF -DBUILD_PARSERS=OFFmake -j$(nproc)

  Example: Cross-Compile for DOS6.5 QNX (aarch64)

  cd$TRT_OSSPATHmkdir -p build&&cd buildexport CUDA_VERSION=11.4export CUDA=cuda-$CUDA_VERSIONexport CUDA_ROOT=/usr/local/cuda-safe-$CUDA_VERSIONexport QNX_BASE=/drive/toolchains/qnx_toolchain# Set to your QNX toolchain installation pathexport QNX_HOST=$QNX_BASE/host/linux/x86_64/export QNX_TARGET=$QNX_BASE/target/qnx7/export PATH=$PATH:$QNX_HOST/usr/bincmake .. -DBUILD_SAMPLES=ON -DBUILD_PLUGINS=OFF -DBUILD_PARSERS=OFF -DBUILD_SAFE_SAMPLES=OFF -DCMAKE_CUDA_COMPILER=$CUDA_ROOT/bin/nvcc -DTRT_OUT_DIR=`pwd`/bin_dynamic_cross -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_qnx.toolchain -DCUDA_VERSION=$CUDA_VERSION -DGPU_ARCHS=87make -j$(nproc)

  NOTE: SetQNX_BASE to your QNX toolchain installation path.If your CUDA version is not the same as in the example, setCUDA_VERSION (for examples that use it in multiple places) or add-DCUDA_VERSION=<version> to the cmake command.

  Example: Cross-Compile for DOS6.5 QNX Safety (aarch64)

  cd$TRT_OSSPATHmkdir -p build&&cd buildexport CUDA_VERSION=11.4export QNX_BASE=/drive/toolchains/qnx_toolchain# Set to your QNX toolchain installation pathexport QNX_HOST=$QNX_BASE/host/linux/x86_64/export QNX_TARGET=$QNX_BASE/target/qnx7/export PATH=$PATH:$QNX_HOST/usr/binexport CUDA=cuda-$CUDA_VERSIONexport CUDA_ROOT=/usr/local/cuda-safe-$CUDA_VERSIONcmake .. -DBUILD_SAMPLES=OFF -DBUILD_SAFE_SAMPLES=ON -DBUILD_PLUGINS=OFF -DBUILD_PARSERS=OFF -DTRT_SAFETY_INFERENCE_ONLY=ON -DTRT_OUT_DIR=`pwd`/bin_dynamic_cross -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_qnx_safe.toolchain -DCUDA_VERSION=$CUDA_VERSION -DCMAKE_CUDA_COMPILER=$CUDA_ROOT/bin/nvcc -DGPU_ARCHS=87make -j$(nproc)

  NOTE: SetQNX_BASE to your QNX toolchain installation path.If your CUDA version is not the same as in the example, setCUDA_VERSION (for examples that use it in multiple places) or add-DCUDA_VERSION=<version> to the cmake command.

  Example: Cross-Compile for DOS7 QNX (aarch64)

  cd$TRT_OSSPATHmkdir -p build&&cd buildexport CUDA_VERSION=13.1export CUDA=cuda-$CUDA_VERSIONexport CUDA_ROOT=/usr/local/cuda-safe-$CUDA_VERSIONexport QNX_BASE=/drive/toolchains/qnx_toolchain# Set to your QNX toolchain installation pathexport QNX_HOST=$QNX_BASE/host/linux/x86_64/export QNX_TARGET=$QNX_BASE/target/qnx/export PATH=$PATH:$QNX_HOST/usr/bincmake .. -DBUILD_SAMPLES=ON -DBUILD_PLUGINS=OFF -DBUILD_PARSERS=OFF -DBUILD_SAFE_SAMPLES=OFF -DCMAKE_CUDA_COMPILER=$CUDA_ROOT/bin/nvcc -DTRT_OUT_DIR=`pwd`/bin_dynamic_cross -DTRT_LIB_DIR=$TRT_LIBPATH -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_qnx.toolchain -DCUDA_VERSION=$CUDA_VERSION -DGPU_ARCHS=110make -j$(nproc)

  NOTE: SetQNX_BASE to your QNX toolchain installation path.If your CUDA version is not the same as in the example, setCUDA_VERSION (for examples that use it in multiple places) or add-DCUDA_VERSION=<version> to the cmake command.

• TensorRT Developer Home
• TensorRT QuickStart Guide
• TensorRT Developer Guide
• TensorRT Sample Support Guide
• TensorRT ONNX Tools
• TensorRT Discussion Forums
• TensorRT Release Notes

• Please refer toTensorRT Release Notes