Overview

The Dynamo Sidecar container runs beside a stock inference engine and connects it to Dynamo over the engine's native gRPC API. The engine keeps its own serve command and argument surface, and Dynamo runs in a separate process that handles worker registration, discovery, and request handling. One CPU-only multi-arch image carries the sidecar executables for SGLang, TensorRT-LLM, and vLLM.

Experimental in v1.5.0. Sidecar packaging, launchers, and API coverage are still evolving, and the sidecar path does not yet match every feature of the in-process backends. Interfaces, defaults, and image layout may change without a deprecation cycle.

Quick Links: Key Components | Release Info | Getting Started | Support

Key Components

• dynamo-sglang-sidecar: Connects a SGLang engine to Dynamo over its gRPC API.
• dynamo-trtllm-sidecar: Connects a TensorRT-LLM engine to Dynamo over its gRPC API.
• dynamo-vllm-sidecar: Connects a vLLM engine to Dynamo over its gRPC API.
• dynamo-sidecar entrypoint: Convenience wrapper mapping the short names sglang, trtllm, and vllm onto the executables above, so ad-hoc docker run needs only the engine name. Kubernetes deployments set the executable directly as the container command.

For the design and the per-engine guides, see Sidecar Backends and the sidecar source.

Release Info

For the complete release history including architecture details, see the Release Artifacts page. Pre-built containers are available for both x86_64 (AMD64) and ARM64 architectures.

Getting Started

1. Select the Tags tab and locate the container image release that you want to run.
2. In the Pull Tag column, click the icon to copy the docker pull command.
3. Open a command prompt and paste the pull command. Ensure the pull completes successfully.
4. Run the sidecar for your engine:

docker run --rm nvcr.io/nvidia/ai-dynamo/dynamo-sidecar:<version> vllm --help
docker run --rm nvcr.io/nvidia/ai-dynamo/dynamo-sidecar:<version> sglang --help
docker run --rm nvcr.io/nvidia/ai-dynamo/dynamo-sidecar:<version> trtllm --help

For Kubernetes deployments, see Sidecar Backends.

Use Cases

• Engine Isolation: Keep the upstream engine's native serve path while Dynamo runs in a separate process
• Dependency Separation: Isolate Dynamo and engine dependencies instead of importing private engine APIs
• Failure Attribution: Separate engine-specific and Dynamo-specific logs and health checks

Support Matrix

Please refer to the support matrix for detailed hardware and architecture support.

Related Containers

• vLLM Runtime - vLLM inference backend
• SGLang Runtime - SGLang inference backend
• TensorRT-LLM Runtime - TensorRT-LLM inference backend
• Dynamo Frontend - Framework-less frontend and EPP
• Kubernetes Operator - K8s deployment automation

License

NVIDIA Dynamo is released under the Apache-2.0 open-source license, making it freely available for development, research, and deployment.

Technical Support

• Documentation: Dynamo Documentation
• Sidecar Guide: Sidecar Backends
• GitHub Issues: Dynamo GitHub Issues
• Release Notes: GitHub Releases