NVIDIA
NVIDIA
Dynamo Tensorrt-LLM Runtime
Container
NVIDIA
NVIDIA
Dynamo Tensorrt-LLM Runtime

The Dynamo TensorRT-LLM runtime image is a containerized build of Dynamo + TensorRT-LLM which serves as the base runtime environment for tensorrt-llm based inference with Dynamo's distributed inference framework.

  • LayerLabelCreated
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENTRYPOINT
    /opt/nvidia/nvidia_entrypoint.sh
    09/18/2025 12:44 AM UTC
    sha256:bfa5e07f3adf81289eab758266abc2c34d5eb6a89518df0f0871f9a17008153cCOPY
    ATTRIBUTION* LICENSE /workspace/
    09/18/2025 12:44 AM UTC
    sha256:3eb95d95050267bf40ace17f35faf67fff1e8ddf750a03bbe8fe05b0d8db6504RUN
    ARCH=arm64 ARCH_ALT=aarch64 RUSTARCH=aarch64-unknown-linux-gnu /bin/bash -c pip install /workspace/dist/ai_dynamo_runtime*cp312*.whl &&
      pip install /workspace/dist/ai_dynamo*any.whl
    09/18/2025 12:44 AM UTC
    sha256:94db7925324626a6fad74aa64ad411a59983f768cf872f54f25d68c18056267dRUN
    ARCH=arm64 ARCH_ALT=aarch64 RUSTARCH=aarch64-unknown-linux-gnu /bin/bash -c mkdir -p /opt/dynamo/bindings/wheels &&
      mkdir /opt/dynamo/bindings/lib &&
      cp dist/ai_dynamo*cp312*.whl /opt/dynamo/bindings/wheels/
    09/18/2025 12:44 AM UTC
    sha256:455f680648c066c40868869707907dbd59a0143fe98baaaa4d4451d864a7f0d5RUN
    ARCH=arm64 ARCH_ALT=aarch64 RUSTARCH=aarch64-unknown-linux-gnu /bin/bash -c uv build --wheel --out-dir /workspace/dist &&
      cd /workspace/lib/bindings/python &&
      uv build --wheel --out-dir /workspace/dist --python 3.12
    09/18/2025 12:44 AM UTC
    sha256:640637a448fdc8dd5247143c5dae40478297979f8f59bf8052a5dd9aa0ff9ad7COPY
    /uv /uvx /bin/
    09/18/2025 12:42 AM UTC
    sha256:e1fd605321351ea6fab386f19661b149416f4d8823e42685f22b414986dfb54fRUN
    ARCH=arm64 ARCH_ALT=aarch64 RUSTARCH=aarch64-unknown-linux-gnu /bin/bash -c cargo build --release --locked --workspace
    09/18/2025 12:42 AM UTC
    sha256:f104d5acd92c3524a9889217a7e725ece4434029ff4adc2c3b4f0223d790054aRUN
    ARCH=arm64 ARCH_ALT=aarch64 RUSTARCH=aarch64-unknown-linux-gnu /bin/bash -c wget --tries=3 --waitretry=5 "https://static.rust-lang.org/rustup/archive/1.28.1/${RUSTARCH}/rustup-init" &&
      chmod +x rustup-init &&
      ./rustup-init -y --no-modify-path --profile default --default-toolchain $RUST_VERSION --default-host ${RUSTARCH} &&
      rm rustup-init &&
      chmod -R a+w $RUSTUP_HOME $CARGO_HOME
    09/18/2025 12:31 AM UTC
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
    RUSTUP_HOME=/usr/local/rustup CARGO_HOME=/usr/local/cargo PATH=/usr/local/cargo/bin:/usr/local/bin/etcd/:/usr/local/lib/python3.12/dist-packages/torch_tensorrt/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/mpi/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/ucx/bin:/opt/amazon/efa/bin:/opt/tensorrt/bin RUST_VERSION=1.89.0
    09/18/2025 12:31 AM UTC
    sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
    RUSTARCH=aarch64-unknown-linux-gnu
    09/18/2025 12:31 AM UTC
    ...