ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
Dynamo vLLM Runtime
Container
NVIDIA
NVIDIA
Dynamo vLLM Runtime

The Dynamo vLLM runtime image is a containerized build of Dynamo + vLLM which serves as the base runtime environment for vLLM based inference with Dynamo's distributed inference framework.

LayerLabelCreated
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4CMD
—
04/21/2026 4:50 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENTRYPOINT
/opt/nvidia/nvidia_entrypoint.sh
04/21/2026 4:50 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
VLLM_USE_FLASHINFER_SAMPLER=1
04/21/2026 4:50 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
DYNAMO_COMMIT_SHA=5dc542097ab165ff000fd1b6059a7b2288cc5bc7
04/21/2026 4:50 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
DYNAMO_COMMIT_SHA=5dc542097ab165ff000fd1b6059a7b2288cc5bc7
04/21/2026 4:50 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4USER
dynamo
04/21/2026 4:50 PM UTC
sha256:dbe1f4f3d0c6d519b1f9f1a91fbfd0f16e66dc223cf293456d290e80844c80dcRUN
ARCH_ALT=x86_64 PYTHON_VERSION=3.12 SITE_PACKAGES=/opt/dynamo/venv/lib/python3.12/site-packages ENABLE_KVBM=true ENABLE_GPU_MEMORY_SERVICE=true ENABLE_MODELEXPRESS_P2P=false MODELEXPRESS_REF=3d73992ce6c10e52ddc54f7f12af35d27e173f15 /bin/bash -l -o pipefail -c cd /usr/local/lib &&
  if [ -f libaws-c-common.so.1.0.0 ] &&
  [ ! -L libaws-c-common.so.1 ]; then rm -f libaws-c-common.so.1 libaws-c-common.so &&
  ln -s libaws-c-common.so.1.0.0 libaws-c-common.so.1 &&
  ln -s libaws-c-common.so.1 libaws-c-common.so; fi &&
  if [ -f libaws-c-s3.so.1.0.0 ] &&
  [ ! -L libaws-c-s3.so.0unstable ]; then rm -f libaws-c-s3.so.0unstable libaws-c-s3.so &&
  ln -s libaws-c-s3.so.1.0.0 libaws-c-s3.so.0unstable &&
  ln -s libaws-c-s3.so.0unstable libaws-c-s3.so; fi &&
  if [ -f libs2n.so.1.0.0 ] &&
  [ ! -L libs2n.so.1 ]; then rm -f libs2n.so.1 libs2n.so &&
  ln -s libs2n.so.1.0.0 libs2n.so.1 &&
  ln -s libs2n.so.1 libs2n.so; fi &&
  for lib in libcrypto libssl; do versioned=$(ls -1 ${lib}.so.1.1.* 2>/dev/null | head -1); if [ -n "$versioned" ] &&
  [ ! -L "${lib}.so.1.1" ]; then rm -f "${lib}.so.1.1" &&
  ln -s "$(basename "$versioned")" "${lib}.so.1.1"; fi; done &&
  ldconfig
04/21/2026 4:50 PM UTC
sha256:5e455272378ceacde02984024de20326d0c648f93714b915b608a824876c95fcRUN
ARCH_ALT=x86_64 PYTHON_VERSION=3.12 SITE_PACKAGES=/opt/dynamo/venv/lib/python3.12/site-packages ENABLE_KVBM=true ENABLE_GPU_MEMORY_SERVICE=true ENABLE_MODELEXPRESS_P2P=false MODELEXPRESS_REF=3d73992ce6c10e52ddc54f7f12af35d27e173f15 /bin/bash -l -o pipefail -c chmod g+w /workspace /workspace/* /opt/dynamo /opt/dynamo/* ${VIRTUAL_ENV} &&
  chmod 755 /opt/dynamo/.launch_screen &&
  echo 'source /opt/dynamo/venv/bin/activate' >> /etc/bash.bashrc &&
  echo 'cat /opt/dynamo/.launch_screen' >> /etc/bash.bashrc
04/21/2026 4:50 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4USER
root
04/21/2026 4:50 PM UTC
sha256:d7b4b56c12892f376b8054c747087f3c8362b06a760944fe892f7672d92f6229RUN
ARCH_ALT=x86_64 PYTHON_VERSION=3.12 SITE_PACKAGES=/opt/dynamo/venv/lib/python3.12/site-packages ENABLE_KVBM=true ENABLE_GPU_MEMORY_SERVICE=true ENABLE_MODELEXPRESS_P2P=false MODELEXPRESS_REF=3d73992ce6c10e52ddc54f7f12af35d27e173f15 /bin/bash -l -o pipefail -c sed '/^#\s/d' /opt/dynamo/launch_message.txt > /opt/dynamo/.launch_screen
04/21/2026 4:50 PM UTC
...