kubernetes-operator is a container that runs as part of the Dynamo cloud platform. Dynamo cloud is a kubernetes platform for deploying and managing inference services. This container manages the lifecycle of Dynamo inference deployments in kubernetes.
Overview
The Dynamo Kubernetes Operator simplifies the deployment, configuration, and lifecycle management of Dynamo inference graphs in Kubernetes environments. It automates the reconciliation of custom resources (CRDs) to ensure your desired state is always achieved, managing health checks, scaling, and component coordination directly through Kubernetes primitives.
Quick Links: Key Components | Release Info | Getting Started | Support
Key Components
- DynamoGraph CRD: Defines the complete inference graph topology including frontend, router, planner, and worker configurations.
- DynamoGroupDeployment (DGD): Manages groups of worker replicas with coordinated scaling and health monitoring.
- DynamoGroupDeploymentReplica (DGDR): Individual worker instance management with optional PVC mounting for profiling.
- Validation Webhooks: Validates CRD configurations at submission time, preventing invalid deployments before they start.
- Kubernetes-Native Service Discovery: Uses EndpointSlices for service discovery, eliminating etcd dependency in Kubernetes deployments.
- Scaling Adapter: Integrates with Planner for SLA-aware autoscaling based on real-time load and latency metrics. For more information about Dynamo Kubernetes deployment, please refer to the GitHub repository and deployment documentation.
Release Info
For the complete release history including architecture details, see the Release Artifacts page. Pre-built containers are available for both x86_64 (AMD64) and ARM64 architectures.
Getting Started
- Select the Tags tab and locate the container image release that you want to run.
- In the Pull Tag column, click the icon to copy the docker pull command.
- Open a command prompt and paste the pull command. Ensure the pull completes successfully.
- Install using Helm (recommended):
helm install dynamo-crds oci://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-crds --version <version>
helm install dynamo-platform oci://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-platform --version <version>
For next steps, including deployment options and examples, please refer to the Dynamo README.
Support Matrix
Please refer to the support matrix for detailed hardware, architecture, and Kubernetes version support.
Related Containers
- vLLM Runtime - vLLM inference backend
- SGLang Runtime - SGLang inference backend
- TensorRT-LLM Runtime - TensorRT-LLM inference backend
- Dynamo Frontend - Standalone frontend with EndpointPicker (EPP)
Helm Charts
- dynamo-crds: Custom Resource Definitions for DynamoGraph, DGD, and DGDR
- dynamo-platform: Operator deployment with optional etcd and NATS
Available from:
helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/
License
NVIDIA Dynamo is released under the Apache-2.0 open-source license, making it freely available for development, research, and deployment.
Technical Support
- Documentation: Dynamo Documentation
- GitHub Issues: Dynamo GitHub Issues
- Release Notes: GitHub Releases