NVIDIA
NVIDIA
Dynamo vLLM Runtime
Container
NVIDIA
NVIDIA
Dynamo vLLM Runtime

The Dynamo vLLM runtime image is a containerized build of Dynamo + vLLM which serves as the base runtime environment for vLLM based inference with Dynamo's distributed inference framework.

LayerLabelCreated
a59b9b4eeb18720b62916db4d402cb333e991e94f7020bcd4a735bd532b01087CONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /workspace
07/02/2025 6:45 PM UTC
1757b8c1d8c8ba14a6f7e74771a537368c56682d6e1200a998eb84427368ea50ENTRYPOINT
/opt/nvidia/nvidia_entrypoint.sh
07/02/2025 6:45 PM UTC
6018562ee50f6f18cc9a313acc0f22e73acf5505fc9b864933a812b1417a5188RUN
ARCH_ALT=x86_64 sed '/^#\s/d' /workspace/launch_message.txt > ~/.launch_screen &&
  echo "cat ~/.launch_screen" >> ~/.bashrc
07/02/2025 6:45 PM UTC
789b02304f11b1701905af8d0392041a2d5150b5474e06843857bab4bce7e32cENV
VLLM_KV_CAPI_PATH=/opt/dynamo/bindings/lib/libdynamo_llm_capi.so
07/02/2025 6:45 PM UTC
4f5d36e6f540ca8e0b92b6d2c314957705d899ac6dcf7538ec5f023ae08547daRUN
ARCH_ALT=x86_64 uv pip install ai-dynamo[vllm] --find-links wheelhouse &&
  uv pip install nixl --find-links wheelhouse &&
  ln -sf $VIRTUAL_ENV/bin/* /usr/local/bin/
07/02/2025 6:45 PM UTC
6144c432850bc7492196f0e0efefe0191563eaae87b36a58af37fc130cec5a34COPY
/workspace/dist/*.whl wheelhouse/
07/02/2025 6:44 PM UTC
e5aaeaf254ad27fbf44bd65d4fffbec27f54bdbb6119b0b2b060963f54e1bf2eCOPY
/workspace/wheels/nixl/*.whl wheelhouse/
07/02/2025 6:44 PM UTC
84dae2d6d93c0f15024c94de492b280fdbbddc5c7b464567b5df86e890d124bbRUN
ARCH_ALT=x86_64 uv pip install /workspace/benchmarks
07/02/2025 6:44 PM UTC
dab7497e835b71a67c030d885d608f34a48d75b0bc69963856a63de2f0108607COPY
. /workspace
07/02/2025 6:44 PM UTC
4112c201855e387d0b3cb74b6fd3c2179c92bdf6b9f6ea33ec7680930e4c58adRUN
ARCH_ALT=x86_64 uv pip install --requirement /tmp/requirements.txt
07/02/2025 6:44 PM UTC

NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking "Accept All", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on "Manage Settings." By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.