Skip to main content
ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
  • NVIDIA
    Dynamo vLLM Runtime
    Container
    NVIDIA
    Dynamo vLLM Runtime

    The Dynamo vLLM runtime image is a containerized build of Dynamo + vLLM which serves as the base runtime environment for vLLM based inference with Dynamo's distributed inference framework.

  • LayerLabelCreated
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4CMD
    —
    		07/02/2025 6:45 PM UTC
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENTRYPOINT
    /opt/nvidia/nvidia_entrypoint.sh
    		07/02/2025 6:45 PM UTC
    sha256:fe58af229f620dff5714630f16a1e54eb7c79336b4fccbfdfcc3c41434d4aaceRUN
    ARCH_ALT=x86_64 sed '/^#\s/d' /workspace/launch_message.txt > ~/.launch_screen &&
  echo "cat ~/.launch_screen" >> ~/.bashrc
    		07/02/2025 6:45 PM UTC
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
    VLLM_KV_CAPI_PATH=/opt/dynamo/bindings/lib/libdynamo_llm_capi.so
    		07/02/2025 6:45 PM UTC
    sha256:49eabc9d0d62d4369c7881de3702cf874cf4e35f09c0649fd7af7718aa57d9b7RUN
    ARCH_ALT=x86_64 uv pip install ai-dynamo[vllm] --find-links wheelhouse &&
  uv pip install nixl --find-links wheelhouse &&
  ln -sf $VIRTUAL_ENV/bin/* /usr/local/bin/
    		07/02/2025 6:45 PM UTC
    sha256:407e093fcc17bed98957f93d1f55085d99ac8b65bf383891c1959f6eae252e25COPY
    /workspace/dist/*.whl wheelhouse/
    		07/02/2025 6:44 PM UTC
    sha256:16deb5125098651f55bfa9c379ac80e1ea344e14f14a2428c1f878108a71c3f5COPY
    /workspace/wheels/nixl/*.whl wheelhouse/
    		07/02/2025 6:44 PM UTC
    sha256:3494e01cdae1fe69a7724727966d8ed4400a00e63fe7c25089f90927e082a1f9RUN
    ARCH_ALT=x86_64 uv pip install /workspace/benchmarks
    		07/02/2025 6:44 PM UTC
    sha256:6ecb47aefe62d3e15307c35d98f860a7b714e9dfacda40eb562ae0ecd47ef426COPY
    . /workspace
    		07/02/2025 6:44 PM UTC
    sha256:867334e101298072df71eaf735cf13a0c0b9667add314c23970cab45ec08ba01RUN
    ARCH_ALT=x86_64 uv pip install --requirement /tmp/requirements.txt
    		07/02/2025 6:44 PM UTC