Overview

The Dynamo TensorRT-LLM runtime container is a pre-built, Docker-based environment designed to run NVIDIA Dynamo with the TensorRT-LLM backend for maximum inference performance on NVIDIA GPUs.

tensorrtllm-runtime-enterprise is part of the NVIDIA Dynamo Enterprise collection. It is published separately from the open-source release; the -enterprise suffix marks what NVIDIA supports commercially. At the same version, tensorrtllm-runtime-enterprise:<version> contains the same binaries as tensorrtllm-runtime:<version>.

Quick Links: Key Components | Access and Support | Release Info | Getting Started | Support

Key Components

• TensorRT-LLM Backend: Open-source library for optimizing Large Language Model (LLM) inference with state-of-the-art optimizations for maximum performance on NVIDIA GPUs.
• Disaggregated Serving (P/D): Separates prefill and decode phases across specialized workers for improved throughput and latency optimization.
• Planner: SLA-aware autoscaler that adjusts prefill and decode worker replica counts to meet TTFT and ITL targets without overscaling GPUs.
• KV Router: Intelligent request routing with prefix-aware caching to maximize KV cache reuse across workers.
• NIXL (NVIDIA Inference Xfer Library): KV cache transport that selects the best available path (GPU Direct RDMA when prerequisites are met, non-RDMA fallbacks otherwise).
• OpenAI-Compatible Frontend: HTTP API server compatible with OpenAI's chat completions and completions endpoints.
• Kubernetes-Native Infrastructure: Service discovery via EndpointSlices and transport-agnostic request plane (TCP default) enable deployment without external dependencies. etcd and NATS remain available as optional alternatives for non-Kubernetes environments. For more information about Dynamo features, please refer to the GitHub repository and documentation.

Access and Support

• Access and use: Available without an NVIDIA AI Enterprise subscription and at no charge for access and use in development or production.
• Commercial support: An active NVIDIA AI Enterprise subscription is required to open commercial support cases and receive NVIDIA Enterprise Support.
• Scope: Only the exact -enterprise artifacts and versions listed in the Dynamo Enterprise collection are covered. An artifact being identical to a supported one does not put it in support scope. Recipes and EFA-optimized tags are not in scope.

See Dynamo Enterprise for scope, supported artifacts, and coverage terms.

Release Info

For the complete release history including TensorRT-LLM versions, CUDA support, and architecture details, see the Release Artifacts page. Pre-built containers are available for both x86_64 (AMD64) and ARM64 architectures.

Getting Started

1. Select the Tags tab and locate the container image release that you want to run.
2. In the Pull Tag column, click the icon to copy the docker pull command.
3. Open a command prompt and paste the pull command. Ensure the pull completes successfully.
4. Run the container:

docker run --gpus all -it nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime-enterprise:<version>

For next steps, including deployment options and examples, please refer to the Dynamo README.

Support Matrix

Please refer to the support matrix for detailed hardware, architecture, and model support information.

Related Containers

• vLLM Runtime - Broadest model and feature coverage
• SGLang Runtime - High-throughput optimized backend
• Dynamo Frontend - Standalone frontend with EndpointPicker (EPP)
• Kubernetes Operator - Kubernetes deployment automation

Security Vulnerabilities in Open Source Packages

Please review the Security Scanning tab to view the latest security scan results.

For certain open-source vulnerabilities listed in the scan results, NVIDIA provides a response in the form of a Vulnerability Exploitability eXchange (VEX) document. The VEX information can be reviewed and downloaded from the Security Scanning tab.

License

NVIDIA Dynamo is released under the Apache-2.0 open-source license, making it freely available for development, research, and deployment.

Enterprise Support

An active NVIDIA AI Enterprise subscription adds the support channel. Security patches ship in the releases themselves, in the same artifacts for all users.

• Security Patching: NVIDIA prioritizes patching vulnerabilities categorized as Critical and High severity, based on CVSS and NVIDIA's scanning logic, and ships fixes roll-forward in subsequent Dynamo releases. Medium and Low severity findings are addressed best-effort.
• Bug Intake: Subscribers file issues through NVIDIA Enterprise Support. Fixes land on main and ship roll-forward in the next Dynamo release; there is no on-demand or backport release for bugs.
• Release Coverage: Support covers Dynamo major and minor releases, and the patch releases (1.X.1) that ship security fixes between them.
• Hardware Scope: Support covers the hardware, operating systems, and architectures listed in the published Compatibility matrix and is not scoped to individual models.

Get Help

• NVIDIA AI Enterprise customers: File a support case through NVIDIA Enterprise Support.
• Community and out-of-scope questions: Refer to the Dynamo Documentation or file on Dynamo GitHub Issues.
• Release Notes: GitHub Releases