ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
Dynamo Tensorrt-LLM Runtime
Container
NVIDIA
NVIDIA
Dynamo Tensorrt-LLM Runtime

The Dynamo TensorRT-LLM runtime image is a containerized build of Dynamo + TensorRT-LLM which serves as the base runtime environment for tensorrt-llm based inference with Dynamo's distributed inference framework.

LayerLabelCreated
c689ba100b85b2129937cab5adaaee3624a1a945e7fd15e0f85e725208dd093cCONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /workspace; ExposedPorts 6006/tcp, 8888/tcp
09/10/2025 7:29 PM UTC
e2f2586af021cec51859541f2a3e4117a2f38ea8647b5c2f81a3613c28bb6f81COPY
ATTRIBUTION* LICENSE /workspace/
09/10/2025 7:29 PM UTC
dc27f90946fc9e529479f2447b02595b6f0a7cb4acd426b330f2a26d70f5ce7fRUN
ARCH=amd64 ARCH_ALT=x86_64 RUSTARCH=x86_64-unknown-linux-gnu /bin/bash -c pip install /workspace/dist/ai_dynamo_runtime*cp312*.whl &&
  pip install /workspace/dist/ai_dynamo*any.whl
09/10/2025 7:29 PM UTC
1c52208573f6c95b9a11243d27cad83405309703fd0c01da321482d4b4ab98baRUN
ARCH=amd64 ARCH_ALT=x86_64 RUSTARCH=x86_64-unknown-linux-gnu /bin/bash -c mkdir -p /opt/dynamo/bindings/wheels &&
  mkdir /opt/dynamo/bindings/lib &&
  cp dist/ai_dynamo*cp312*.whl /opt/dynamo/bindings/wheels/
09/10/2025 7:29 PM UTC
7925c6925e17ddbd5cf74ae256cf30f8c2b5d20ff2f7e82f65d2c6c5800fced1RUN
ARCH=amd64 ARCH_ALT=x86_64 RUSTARCH=x86_64-unknown-linux-gnu /bin/bash -c uv build --wheel --out-dir /workspace/dist &&
  cd /workspace/lib/bindings/python &&
  uv build --wheel --out-dir /workspace/dist --python 3.12
09/10/2025 7:29 PM UTC
21797785f73caa2964dd671e600a5fa4b2bc01e624e5c64e99493ce7bc451b38COPY
/uv /uvx /bin/
09/10/2025 7:28 PM UTC
0d8a1e6062dfabdc040f1a7a630f71d970c0f67f7e4362bbae54f495ad26c2a7RUN
ARCH=amd64 ARCH_ALT=x86_64 RUSTARCH=x86_64-unknown-linux-gnu /bin/bash -c cargo build --release --locked --workspace
09/10/2025 7:28 PM UTC
4a7f446a1756f7ec5f5f4c685d2ff8f646c0ae18d5dce1c3c328c1ae0834fbd4RUN
ARCH=amd64 ARCH_ALT=x86_64 RUSTARCH=x86_64-unknown-linux-gnu /bin/bash -c wget --tries=3 --waitretry=5 "https://static.rust-lang.org/rustup/archive/1.28.1/${RUSTARCH}/rustup-init" &&
  chmod +x rustup-init &&
  ./rustup-init -y --no-modify-path --profile default --default-toolchain $RUST_VERSION --default-host ${RUSTARCH} &&
  rm rustup-init &&
  chmod -R a+w $RUSTUP_HOME $CARGO_HOME
09/10/2025 7:22 PM UTC
fb9c88f767aa61be7add10c27ccdc83e37d6391a62187d6c2894e1bfbc249a35ENV
RUSTUP_HOME=/usr/local/rustup CARGO_HOME=/usr/local/cargo PATH=/usr/local/cargo/bin:/usr/local/bin/etcd/:/usr/local/lib/python3.12/dist-packages/torch_tensorrt/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/mpi/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/ucx/bin:/opt/amazon/efa/bin:/opt/tensorrt/bin RUST_VERSION=1.89.0
09/10/2025 7:22 PM UTC
2ab6172ab72aeeebba920f12ef111df4c486a76045311f3d70601a041808d303ARG
RUSTARCH=x86_64-unknown-linux-gnu
09/10/2025 7:22 PM UTC
...