NVIDIA
NVIDIA
Dynamo Tensorrt-LLM Runtime
Container
NVIDIA
NVIDIA
Dynamo Tensorrt-LLM Runtime

The Dynamo TensorRT-LLM runtime image is a containerized build of Dynamo + TensorRT-LLM which serves as the base runtime environment for tensorrt-llm based inference with Dynamo's distributed inference framework.

LayerLabelCreated
0ba95b2747069a45fa207f553380c256f29fed43bad4e3acea08efddb3f3aedeCONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /workspace; ExposedPorts 6006/tcp, 8888/tcp
09/18/2025 12:44 AM UTC
f0bfa09518f5d2c81a399faef402a8e5e248cd9339d6e5e6339c54e9ab54b3b4COPY
ATTRIBUTION* LICENSE /workspace/
09/18/2025 12:44 AM UTC
8db20d0e8b7227cc77ae3c8b9a33464cb7e9e5458415c006745f2fda1ed31490RUN
ARCH=arm64 ARCH_ALT=aarch64 RUSTARCH=aarch64-unknown-linux-gnu /bin/bash -c pip install /workspace/dist/ai_dynamo_runtime*cp312*.whl &&
  pip install /workspace/dist/ai_dynamo*any.whl
09/18/2025 12:44 AM UTC
63f70ab9a59222f2e6352052a8c3dbd9c4156b9eb9d0ff5313e4b0db3034ad53RUN
ARCH=arm64 ARCH_ALT=aarch64 RUSTARCH=aarch64-unknown-linux-gnu /bin/bash -c mkdir -p /opt/dynamo/bindings/wheels &&
  mkdir /opt/dynamo/bindings/lib &&
  cp dist/ai_dynamo*cp312*.whl /opt/dynamo/bindings/wheels/
09/18/2025 12:44 AM UTC
2ab9d0a12bf12540dc22b1dbef8811e1e94b2cca4006367a3aac2502f8ad6240RUN
ARCH=arm64 ARCH_ALT=aarch64 RUSTARCH=aarch64-unknown-linux-gnu /bin/bash -c uv build --wheel --out-dir /workspace/dist &&
  cd /workspace/lib/bindings/python &&
  uv build --wheel --out-dir /workspace/dist --python 3.12
09/18/2025 12:44 AM UTC
e82357370886999456c17e5b9676e2459dd88e21c827868c8582ab3fca8901d7COPY
/uv /uvx /bin/
09/18/2025 12:42 AM UTC
5fbe3973bc117cb02024af41d23bf4c44cdeb20460520d954c68414d21c923e2RUN
ARCH=arm64 ARCH_ALT=aarch64 RUSTARCH=aarch64-unknown-linux-gnu /bin/bash -c cargo build --release --locked --workspace
09/18/2025 12:42 AM UTC
1bd841e1367fb3531b1b1b31f0a1300fa4b8595642e050630d8ed4ab7a5abc72RUN
ARCH=arm64 ARCH_ALT=aarch64 RUSTARCH=aarch64-unknown-linux-gnu /bin/bash -c wget --tries=3 --waitretry=5 "https://static.rust-lang.org/rustup/archive/1.28.1/${RUSTARCH}/rustup-init" &&
  chmod +x rustup-init &&
  ./rustup-init -y --no-modify-path --profile default --default-toolchain $RUST_VERSION --default-host ${RUSTARCH} &&
  rm rustup-init &&
  chmod -R a+w $RUSTUP_HOME $CARGO_HOME
09/18/2025 12:31 AM UTC
d06576aff095fa80f2aea98f4dcbbb47645b1abf3eb7fe9d5a076dc7353165ceENV
RUSTUP_HOME=/usr/local/rustup CARGO_HOME=/usr/local/cargo PATH=/usr/local/cargo/bin:/usr/local/bin/etcd/:/usr/local/lib/python3.12/dist-packages/torch_tensorrt/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/mpi/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/ucx/bin:/opt/amazon/efa/bin:/opt/tensorrt/bin RUST_VERSION=1.89.0
09/18/2025 12:31 AM UTC
224cf867dfcd37d54703124910fb55bd44d55f25aaa7ec1dbda211efb38896a1ARG
RUSTARCH=aarch64-unknown-linux-gnu
09/18/2025 12:31 AM UTC
...

NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking "Accept All", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on "Manage Settings." By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.