Overview

The Dynamo Frontend container is a lightweight, framework-less image designed to deploy and run CPU-bound frontend components without requiring CUDA or backend engine dependencies (vLLM, SGLang, TensorRT-LLM). It enables flexible deployment topologies by separating the frontend from inference backends, and integrates with the Gateway API Inference Extension (GAIE) for Kubernetes-native request routing.

dynamo-frontend-enterprise is part of the NVIDIA Dynamo Enterprise collection. It is published separately from the open-source release; the -enterprise suffix marks what NVIDIA supports commercially. At the same version, dynamo-frontend-enterprise:<version> contains the same binaries as dynamo-frontend:<version>.

Quick Links: Key Components | Access and Support | Release Info | Getting Started | Support

Key Components

• OpenAI-Compatible Frontend: HTTP API server compatible with OpenAI's chat completions and completions endpoints, handling request preprocessing, validation, and response formatting.
• Endpoint Picker Plugin (EPP): Handles routing, flow control, and request management for intelligent backend selection. Integrates with Gateway API Inference Extension (GAIE) for Kubernetes-native load balancing.
• Request Router: Routes requests to appropriate backend workers based on prefix matching, load, and KV cache state.
• Mock Workers: Test Dynamo components without GPU backends for development, CI/CD, and validation workflows.
• Kubernetes-Native Infrastructure: Service discovery via EndpointSlices and transport-agnostic request plane (TCP default) enable deployment without etcd or NATS dependencies. For more information about Dynamo frontend and GAIE, please refer to the GitHub repository and GAIE documentation.

Access and Support

• Access and use: Available without an NVIDIA AI Enterprise subscription and at no charge for access and use in development or production.
• Commercial support: An active NVIDIA AI Enterprise subscription is required to open commercial support cases and receive NVIDIA Enterprise Support.
• Scope: Only the exact -enterprise artifacts and versions listed in the Dynamo Enterprise collection are covered. An artifact being identical to a supported one does not put it in support scope. Recipes are not in scope.

See Dynamo Enterprise for scope, supported artifacts, and coverage terms.

Release Info

For the complete release history including architecture details, see the Release Artifacts page. Pre-built containers are available for both x86_64 (AMD64) and ARM64 architectures.

Getting Started

1. Select the Tags tab and locate the container image release that you want to run.
2. In the Pull Tag column, click the icon to copy the docker pull command.
3. Open a command prompt and paste the pull command. Ensure the pull completes successfully.
4. Run the container:

docker run -it --entrypoint /bin/bash nvcr.io/nvidia/ai-dynamo/dynamo-frontend-enterprise:<version>

For next steps, including deployment options and examples, please refer to the Dynamo README.

Use Cases

• Separated Frontend Deployment: Run frontend on CPU nodes while backends run on GPU nodes
• Gateway API Integration: Use with GAIE for Kubernetes-native inference routing
• Development & Testing: Test Dynamo pipelines with mock workers without GPU resources
• CI/CD Validation: Validate configurations and routing logic in automated pipelines

Support Matrix

Please refer to the support matrix for detailed hardware and architecture support.

Related Containers

• vLLM Runtime - vLLM inference backend
• SGLang Runtime - SGLang inference backend
• TensorRT-LLM Runtime - TensorRT-LLM inference backend
• Kubernetes Operator - Kubernetes deployment automation

Security Vulnerabilities in Open Source Packages

Please review the Security Scanning tab to view the latest security scan results.

For certain open-source vulnerabilities listed in the scan results, NVIDIA provides a response in the form of a Vulnerability Exploitability eXchange (VEX) document. The VEX information can be reviewed and downloaded from the Security Scanning tab.

License

NVIDIA Dynamo is released under the Apache-2.0 open-source license, making it freely available for development, research, and deployment.

Enterprise Support

An active NVIDIA AI Enterprise subscription adds the support channel. Security patches ship in the releases themselves, in the same artifacts for all users.

• Security Patching: NVIDIA prioritizes patching vulnerabilities categorized as Critical and High severity, based on CVSS and NVIDIA's scanning logic, and ships fixes roll-forward in subsequent Dynamo releases. Medium and Low severity findings are addressed best-effort.
• Bug Intake: Subscribers file issues through NVIDIA Enterprise Support. Fixes land on main and ship roll-forward in the next Dynamo release; there is no on-demand or backport release for bugs.
• Release Coverage: Support covers Dynamo major and minor releases, and the patch releases (1.X.1) that ship security fixes between them.
• Hardware Scope: Support covers the hardware, operating systems, and architectures listed in the published Compatibility matrix and is not scoped to individual models.

Get Help

• NVIDIA AI Enterprise customers: File a support case through NVIDIA Enterprise Support.
• Community and out-of-scope questions: Refer to the Dynamo Documentation or file on Dynamo GitHub Issues.
• Release Notes: GitHub Releases