Dynamo vLLM Runtime | NVIDIA NGC

NVIDIA

Dynamo vLLM Runtime

Container

NVIDIA

Dynamo vLLM Runtime

The Dynamo vLLM runtime image is a containerized build of Dynamo + vLLM which serves as the base runtime environment for vLLM based inference with Dynamo's distributed inference framework.

Layer	Label		Created
a59b9b4eeb18720b62916db4d402cb333e991e94f7020bcd4a735bd532b01087	CONFIG	`Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /workspace`	07/02/2025 6:45 PM UTC
1757b8c1d8c8ba14a6f7e74771a537368c56682d6e1200a998eb84427368ea50	ENTRYPOINT	`/opt/nvidia/nvidia_entrypoint.sh`	07/02/2025 6:45 PM UTC
6018562ee50f6f18cc9a313acc0f22e73acf5505fc9b864933a812b1417a5188	RUN	`ARCH_ALT=x86_64 sed '/^#\s/d' /workspace/launch_message.txt > ~/.launch_screen && echo "cat ~/.launch_screen" >> ~/.bashrc`	07/02/2025 6:45 PM UTC
789b02304f11b1701905af8d0392041a2d5150b5474e06843857bab4bce7e32c	ENV	`VLLM_KV_CAPI_PATH=/opt/dynamo/bindings/lib/libdynamo_llm_capi.so`	07/02/2025 6:45 PM UTC
4f5d36e6f540ca8e0b92b6d2c314957705d899ac6dcf7538ec5f023ae08547da	RUN	`ARCH_ALT=x86_64 uv pip install ai-dynamo[vllm] --find-links wheelhouse && uv pip install nixl --find-links wheelhouse && ln -sf $VIRTUAL_ENV/bin/* /usr/local/bin/`	07/02/2025 6:45 PM UTC
6144c432850bc7492196f0e0efefe0191563eaae87b36a58af37fc130cec5a34	COPY	`/workspace/dist/*.whl wheelhouse/`	07/02/2025 6:44 PM UTC
e5aaeaf254ad27fbf44bd65d4fffbec27f54bdbb6119b0b2b060963f54e1bf2e	COPY	`/workspace/wheels/nixl/*.whl wheelhouse/`	07/02/2025 6:44 PM UTC
84dae2d6d93c0f15024c94de492b280fdbbddc5c7b464567b5df86e890d124bb	RUN	`ARCH_ALT=x86_64 uv pip install /workspace/benchmarks`	07/02/2025 6:44 PM UTC
dab7497e835b71a67c030d885d608f34a48d75b0bc69963856a63de2f0108607	COPY	`. /workspace`	07/02/2025 6:44 PM UTC
4112c201855e387d0b3cb74b6fd3c2179c92bdf6b9f6ea33ec7680930e4c58ad	RUN	`ARCH_ALT=x86_64 uv pip install --requirement /tmp/requirements.txt`	07/02/2025 6:44 PM UTC