Using Dev Containers#

The TensorRT-LLM repository contains aDev Containersconfiguration in.devcontainer. These files are intended foruse withVisual Studio Code.

Due to the various container options supported by TensorRT-LLM (seeBuilding from Source Code on Linux andNVIDIA/TensorRT-LLM), the DevContainer configuration also offers some degree of customization.

Generally, theinitializeCommand indevcontainer.json will runmake_env.py to generate an.env file fordocker-compose.Most importantly, thedocker-compose.yml uses${DEV_CONTAINER_IMAGE}as base image.The generated.devcontainer/.env is not tracked by Git and combinesdata from the following sources:

  • jenkins/current_image_tags.properties which contains the image tagscurrently used by CI.

  • .devcontainer/devcontainer.env which contains common configurationsettings and is tracked by Git.

  • .devcontainer/devcontainer.env.user (optional) which is ignored byGit and can be edited to customize the Dev Container behavior.

The source files are processed usingsh, in the order in which theyare listed above. Thus, features like command substitution are supported.

The following sections provide more detail on particular Dev Containerconfiguration parameters which can be customized.

Note

After editing any of the configuration files, it may be necessaryto execute the “Dev Containers: Reopen Folder in SSH” (if applicable) and“Dev Containers: Rebuild and Reopen in Container” Visual Studio Codecommands.

Container image selection#

By default,make_env.py will attempt to auto-select a suitable containerimage as follows:

  1. Reuse the development container image used by CI. This requires accessto the NVIDIA internal artifact repository.

  2. Use the most recentNGC Development container imageassociated with a Git tag which is reachable from the currently checkedout commit.

  3. Build a development image locally.

SetDEV_CONTAINER_IMAGE=<some_uri> to bypass the aforementioned discoverymechanism if desired.

By settingLOCAL_BUILD=0, the local image build can be disabled. In thiscase, execution fails if no suitable pre-built image is found.

SettingLOCAL_BUILD=1 forces building of a local image, even if a pre-builtimage is available.

Volume Mounts#

Docker volume mounts can be customized by editingdocker-compose.yml, which allows using any variables defined in.env.

By default, the Dev Container configuration mounts the VS Code workspace into/workspaces/tensorrt_llm and~/.cache/huggingface into/huggingface.The source paths can be overridden by settingSOURCE_DIR andHOME_DIRin.devcontainer/devcontainer.env.user, respectively. This is ofparticular relevance when usingDocker Rootless Mode,which requires configuring UID/GID translation using a tool likebindfs.The Dev Container scripts contain heuristics to detect Docker RootlessMode and will issue an error if these variables are not set.An analogous logic is applied toHF_HOME.

Overriding Docker Compose configuration#

When starting the container,.devcontainer/docker-compose.ymlismerged with.devcontainer/docker-compose.override.yml. The latter file is nottracked by Git and will be created bymake_env.py if it does not exist.

This mechanism can be used, e.g., to add custom volume mounts:

# Example .devcontainer/docker-compose.override.ymlversion:"3.9"services:tensorrt_llm-dev:volumes:# Uncomment the following lines to enable# # Mount TRTLLM data volume:# - /home/scratch.trt_llm_data/:/home/scratch.trt_llm_data/:ro

It is possible to conditionally mount volumes by combining, e.g.,[this method] (https://stackoverflow.com/a/61954812) and shell commandsubstitution in.devcontainer/devcontainer.env.user.

If no.devcontainer/docker-compose.override.yml file is found, the Dev Containerinitialization script will create one with the contents listed above.