Dynamo vLLM Runtime | NVIDIA NGC

NVIDIA

Dynamo vLLM Runtime

Container

NVIDIA

Dynamo vLLM Runtime

The Dynamo vLLM runtime image is a containerized build of Dynamo + vLLM which serves as the base runtime environment for vLLM based inference with Dynamo's distributed inference framework.

Layer	Label		Created
9aa48464819f478fbae0a43235969c8bcad42ad08d73b4b0e1dcea9a61a8fe36	CONFIG	`Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /workspace`	07/16/2025 9:27 PM UTC
a5cc367c025e74f23ec8a688df63ee32320218df6a5545fea704b0fcccf0fbb5	ENTRYPOINT	`/opt/nvidia/nvidia_entrypoint.sh`	07/16/2025 9:27 PM UTC
bf770e3f5c2dfa784cefe18cd2f9f386da6a858cb1b99aaf6911d7225fd9434f	RUN	`ARCH_ALT=x86_64 sed '/^#\s/d' /workspace/launch_message.txt > ~/.launch_screen && echo "cat ~/.launch_screen" >> ~/.bashrc`	07/16/2025 9:27 PM UTC
693dc68ec4a6a9e089d1895651b6986e88916a572ce9987cc45dc34291458868	ENV	`VLLM_KV_CAPI_PATH=/opt/dynamo/bindings/lib/libdynamo_llm_capi.so`	07/16/2025 9:27 PM UTC
52b8e45bfcf1bbd42240b6b26d9edb663d52258e1d60cedc4ccee7f7e5790746	RUN	`ARCH_ALT=x86_64 uv pip install ai-dynamo[vllm] --find-links wheelhouse && uv pip install nixl --find-links wheelhouse && ln -sf $VIRTUAL_ENV/bin/* /usr/local/bin/`	07/16/2025 9:27 PM UTC
40f46183f9527b06c10796bd4bd9573dc8fb1b42dfe39eadec6ee3097d974bb2	COPY	`/workspace/dist/*.whl wheelhouse/`	07/16/2025 9:26 PM UTC
485ae9cb62263c30d2bec436c4f014571cfb1794ce16df4d5b8a4ea3cf43ff37	COPY	`/workspace/wheels/nixl/*.whl wheelhouse/`	07/16/2025 9:26 PM UTC
b847ae1fc22251dce410a05da7196684a27b38af225b127bdc3150449cc6ad1e	RUN	`ARCH_ALT=x86_64 uv pip install /workspace/benchmarks`	07/16/2025 9:26 PM UTC
37b6fdcf490740b6051fea17c9500361c3bbd6ca2bab4f55b0be80f117006e0e	COPY	`. /workspace`	07/16/2025 9:26 PM UTC
68415560c11b7b6f3c5192423d12e39dd03bd26bab61bacca3ff0d6975ba14d4	RUN	`ARCH_ALT=x86_64 uv pip install --requirement /tmp/requirements.txt`	07/16/2025 9:26 PM UTC