Dynamo vLLM Runtime | NVIDIA NGC

NVIDIA

Dynamo vLLM Runtime

Container

NVIDIA

Dynamo vLLM Runtime

The Dynamo vLLM runtime image is a containerized build of Dynamo + vLLM which serves as the base runtime environment for vLLM based inference with Dynamo's distributed inference framework.

Layer	Label		Created
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4	CMD	`—`	07/02/2025 6:45 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4	ENTRYPOINT	`/opt/nvidia/nvidia_entrypoint.sh`	07/02/2025 6:45 PM UTC
sha256:fe58af229f620dff5714630f16a1e54eb7c79336b4fccbfdfcc3c41434d4aace	RUN	`ARCH_ALT=x86_64 sed '/^#\s/d' /workspace/launch_message.txt > ~/.launch_screen && echo "cat ~/.launch_screen" >> ~/.bashrc`	07/02/2025 6:45 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4	ENV	`VLLM_KV_CAPI_PATH=/opt/dynamo/bindings/lib/libdynamo_llm_capi.so`	07/02/2025 6:45 PM UTC
sha256:49eabc9d0d62d4369c7881de3702cf874cf4e35f09c0649fd7af7718aa57d9b7	RUN	`ARCH_ALT=x86_64 uv pip install ai-dynamo[vllm] --find-links wheelhouse && uv pip install nixl --find-links wheelhouse && ln -sf $VIRTUAL_ENV/bin/* /usr/local/bin/`	07/02/2025 6:45 PM UTC
sha256:407e093fcc17bed98957f93d1f55085d99ac8b65bf383891c1959f6eae252e25	COPY	`/workspace/dist/*.whl wheelhouse/`	07/02/2025 6:44 PM UTC
sha256:16deb5125098651f55bfa9c379ac80e1ea344e14f14a2428c1f878108a71c3f5	COPY	`/workspace/wheels/nixl/*.whl wheelhouse/`	07/02/2025 6:44 PM UTC
sha256:3494e01cdae1fe69a7724727966d8ed4400a00e63fe7c25089f90927e082a1f9	RUN	`ARCH_ALT=x86_64 uv pip install /workspace/benchmarks`	07/02/2025 6:44 PM UTC
sha256:6ecb47aefe62d3e15307c35d98f860a7b714e9dfacda40eb562ae0ecd47ef426	COPY	`. /workspace`	07/02/2025 6:44 PM UTC
sha256:867334e101298072df71eaf735cf13a0c0b9667add314c23970cab45ec08ba01	RUN	`ARCH_ALT=x86_64 uv pip install --requirement /tmp/requirements.txt`	07/02/2025 6:44 PM UTC