Description

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

Versioning

TensorRT LLM images use the following tag formats:

• General Availability (GA) releases: 1.3.0
• Main-branch release candidates: 1.4.0rc2
• Main-branch nightly builds: 1.4.0rc2.dev202610270008
• Side-branch builds: feat-qwen3.5-opt-1.4.0a0.dev202610281318, with a normalized branch-name prefix. Older tags may omit this prefix, such as 1.4.0a0.dev202610230958.

Tag components:

• rcN denotes the release candidate number; for example, rc2 means release candidate 2. This component appears in both main-branch release candidate and nightly tags.
• a0 is the fixed alpha-version marker used for side-branch builds.
• .devYYYYMMDDHHMM contains the automatically generated release date and time in UTC, formatted as year, month, day, hour, and minute. For example, .dev202610270008 represents October 27, 2026 at 00:08 UTC.
• The branch-name prefix identifies the source branch in normalized form. For example, the feat/qwen3.5-opt branch in the NVIDIA/TensorRT-LLM repository becomes the prefix feat-qwen3.5-opt.

Main-branch release candidates, main-branch nightly builds, and side-branch builds are classified as pre-releases.

Overview

TensorRT LLM Release Container

The TensorRT LLM Release container provides a pre-built environment for running TensorRT LLM.

Visit the official GitHub repository for more details.

Running TensorRT LLM Using Docker

A typical command to launch the container is:

docker run --rm -it --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 --gpus=all \
    		nvcr.io/nvidia/tensorrt-llm/release:x.y.z

where x.y.z is the version of the TensorRT LLM container to use (cf. release history on GitHub and tags in NGC Catalog). To sanity check, run the following command:

python3 -c "import tensorrt_llm"

This command will print the TensorRT LLM version if everything is working correctly. After verification, you can explore and try the example scripts included in /app/tensorrt_llm/examples.

Alternatively, if you have already cloned the TensorRT LLM repository, you can use the following convenient command to run the container:

make -C docker ngc-release_run LOCAL_USER=1 DOCKER_PULL=1 IMAGE_TAG=x.y.z

This command pulls the specified container from the NVIDIA NGC registry, sets up the local user's account within the container, and launches it with full GPU support.

For comprehensive information about TensorRT LLM, including documentation, source code, examples, and installation guidelines, visit the following official resources:

• TensorRT LLM GitHub Repository
• TensorRT LLM Online Documentation

Security CVEs

To review known CVEs on this image, refer to the Security Scanning tab on this page.

License

By pulling and using the container, you accept the terms and conditions of this End User License Agreement and Product-Specific Terms.