Overview

The Dynamo Planner runtime container is a pre-packaged, Docker-based environment for running NVIDIA Dynamo's Planner, Profiler, and Global Planner — the SLA-aware control plane that sizes prefill and decode workers for distributed LLM inference. It packages the control loop together with its forecasting, profiling, and Kubernetes integration stack in a slim, distroless runtime image, isolated from the engine runtimes so a single image can drive scaling decisions for SGLang, TensorRT-LLM, and vLLM deployments.

Quick Links: Key Components | Release Info | Getting Started | Support

Key Components

• Planner Control Loop: Continuously observes traffic and engine telemetry and adjusts prefill and decode worker counts to meet TTFT and ITL targets without overscaling GPUs.
• Throughput-Based Scaling: Long-interval scaling (default 180s) using pre-deployment engine performance data and traffic prediction with selectable load predictors — ARIMA, Prophet, Kalman, or constant.
• Load-Based Scaling: Short-interval scaling (default 5s) using ForwardPassMetrics from the Dynamo event plane and an online linear regression to react to bursts.
• Optimization Targets: throughput (default queue and KV-utilization thresholds, zero-config), latency (aggressive low-latency), and sla (regression-based TTFT and ITL targeting with profiling).
• Profiler and Global Planner: Bundled dynamo.profiler for pre-deployment SLA sweeps and dynamo.global_planner for multi-DGD coordination and shared GPU budgets.
• Backend and Mode Coverage: SGLang, TensorRT-LLM, and vLLM across agg, disagg, prefill, and decode modes. KV Router compatible.
• Kubernetes-Native Connectors: KubernetesConnector for in-cluster scaling and VirtualConnector for out-of-cluster control.
• Observability: Prometheus metrics under the dynamo_planner_* namespace, Grafana dashboard, periodic HTML diagnostics, and an optional live dashboard.

For more information about the Planner, see the Planner Guide and Planner Design docs.

Release Info

For the complete release history, see the Release Artifacts page.

Pre-built containers are available for both x86_64 (AMD64) and ARM64 architectures. The Planner image is shared across all backend runtimes — one image scales SGLang, TensorRT-LLM, and vLLM workers. The runtime stage is distroless and the Planner does not require a GPU.

Getting Started

The Planner is typically deployed by the Dynamo Kubernetes Operator as part of a DynamoGraphDeployment. The container can also be run standalone for development and inspection.

1. Select the Tags tab and locate the container image release that you want to run.
2. In the Pull Tag column, click the icon to copy the docker pull command.
3. Open a command prompt and paste the pull command. Ensure the pull completes successfully.
4. Run the container:

docker run -it nvcr.io/nvidia/ai-dynamo/dynamo-planner:<version> \
  python -m dynamo.planner --help

For deployment recipes, including SLA-based scaling with auto-profiling via DynamoGraphDeploymentRequest, see the Planner Examples page.

Support Matrix

Please refer to the support matrix for detailed hardware, architecture, and model support information. Note: load-based scaling currently requires vLLM with InstrumentedScheduler; throughput-based scaling is available across all supported backends.

Related Containers

• vLLM Runtime - Broadest model and feature coverage
• SGLang Runtime - High-throughput modular serving
• TensorRT-LLM Runtime - Maximum inference performance
• Dynamo Frontend - Standalone frontend with EndpointPicker (EPP)
• Kubernetes Operator - K8s deployment automation

License

NVIDIA Dynamo is released under the Apache-2.0 open-source license, making it freely available for development, research, and deployment.

Technical Support

• Documentation: Dynamo Documentation
• GitHub Issues: Dynamo GitHub Issues
• Release Notes: GitHub Releases