Overview

The Dynamo vLLM runtime container is a pre-built, Docker-based environment designed to run NVIDIA Dynamo with the vLLM backend for high-performance, distributed large language model (LLM) inference.

vllm-runtime-enterprise is part of the NVIDIA Dynamo Enterprise collection. It is published separately from the open-source release; the -enterprise suffix marks what NVIDIA supports commercially. At the same version, vllm-runtime-enterprise:<version> contains the same binaries as vllm-runtime:<version>.

Quick Links: Key Components | Access and Support | Release Info | Getting Started | Support

Key Components

• vLLM Backend: High-performance LLM inference engine with optimized attention mechanisms, continuous batching, and efficient KV cache management.
• Disaggregated Serving (P/D): Separates prefill and decode phases across specialized workers for improved throughput and latency optimization.
• Planner: SLA-aware autoscaler that adjusts prefill and decode worker replica counts to meet TTFT and ITL targets without overscaling GPUs.
• KV Router: Intelligent request routing with prefix-aware caching to maximize KV cache reuse across workers.
• NIXL (NVIDIA Inference Xfer Library): KV cache transport that selects the best available path (GPU Direct RDMA when prerequisites are met, non-RDMA fallbacks otherwise).
• OpenAI-Compatible Frontend: HTTP API server compatible with OpenAI's chat completions and completions endpoints.
• Kubernetes-Native Infrastructure: Service discovery via EndpointSlices and transport-agnostic request plane (TCP default) enable deployment without external dependencies. etcd and NATS remain available as optional alternatives for non-Kubernetes environments. For more information about Dynamo features, please refer to the GitHub repository and documentation.

Access and Support

• Access and use: Available without an NVIDIA AI Enterprise subscription and at no charge for access and use in development or production.
• Commercial support: An active NVIDIA AI Enterprise subscription is required to open commercial support cases and receive NVIDIA Enterprise Support.
• Scope: Only the exact -enterprise artifacts and versions listed in the Dynamo Enterprise collection are covered. An artifact being identical to a supported one does not put it in support scope. Recipes and EFA-optimized tags are not in scope.

See Dynamo Enterprise for scope, supported artifacts, and coverage terms.

Release Info

For the complete release history including vLLM versions, CUDA support, and architecture details, see the Release Artifacts page. Pre-built containers are available for both x86_64 (AMD64) and ARM64 architectures.

Getting Started

1. Select the Tags tab and locate the container image release that you want to run.
2. In the Pull Tag column, click the icon to copy the docker pull command.
3. Open a command prompt and paste the pull command. Ensure the pull completes successfully.
4. Run the container:

docker run --gpus all -it nvcr.io/nvidia/ai-dynamo/vllm-runtime-enterprise:<version> bash

For next steps, including deployment options and examples, please refer to the Dynamo README.

Support Matrix

Please refer to the support matrix for detailed hardware, architecture, and model support information.

Related Containers

• SGLang Runtime - High-throughput optimized backend
• TensorRT-LLM Runtime - Maximum inference performance
• Dynamo Frontend - Standalone frontend with EndpointPicker (EPP)
• Kubernetes Operator - Kubernetes deployment automation

Security Vulnerabilities in Open Source Packages

Please review the Security Scanning tab to view the latest security scan results.

For certain open-source vulnerabilities listed in the scan results, NVIDIA provides a response in the form of a Vulnerability Exploitability eXchange (VEX) document. The VEX information can be reviewed and downloaded from the Security Scanning tab.

License

NVIDIA Dynamo is released under the Apache-2.0 open-source license, making it freely available for development, research, and deployment.

Enterprise Support

An active NVIDIA AI Enterprise subscription adds the support channel. Security patches ship in the releases themselves, in the same artifacts for all users.

• Security Patching: NVIDIA prioritizes patching vulnerabilities categorized as Critical and High severity, based on CVSS and NVIDIA's scanning logic, and ships fixes roll-forward in subsequent Dynamo releases. Medium and Low severity findings are addressed best-effort.
• Bug Intake: Subscribers file issues through NVIDIA Enterprise Support. Fixes land on main and ship roll-forward in the next Dynamo release; there is no on-demand or backport release for bugs.
• Release Coverage: Support covers Dynamo major and minor releases, and the patch releases (1.X.1) that ship security fixes between them.
• Hardware Scope: Support covers the hardware, operating systems, and architectures listed in the published Compatibility matrix and is not scoped to individual models.

Get Help

• NVIDIA AI Enterprise customers: File a support case through NVIDIA Enterprise Support.
• Community and out-of-scope questions: Refer to the Dynamo Documentation or file on Dynamo GitHub Issues.
• Release Notes: GitHub Releases