Dynamo Tensorrt-LLM Runtime

NVIDIA

Container

NVIDIA

Dynamo Tensorrt-LLM Runtime

The Dynamo TensorRT-LLM runtime image is a containerized build of Dynamo + TensorRT-LLM which serves as the base runtime environment for tensorrt-llm based inference with Dynamo's distributed inference framework.

Layer	Label		Created
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4	CMD	`—`	08/08/2025 11:45 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4	ENTRYPOINT	`/opt/nvidia/nvidia_entrypoint.sh`	08/08/2025 11:45 PM UTC
sha256:8c1d32752f23679768528572d68273da1f482ea1131262d14989cf3000737af1	RUN	`ARCH_ALT=x86_64 TORCH_VER=2.8.0a0+5228986c39.nv25.5 TORCHVISION_VER=0.22.0a0 SETUPTOOLS_VER=78.1.1 PYTORCH_TRITON_VER=3.3.0+git96316ce52.nvinternal JINJA2_VER=3.1.6 NETWORKX_VER=3.4.2 SYMPY_VER=1.14.0 PACKAGING_VER=23.2 FLASH_ATTN_VER=2.7.3 MPMATH_VER=1.3.0 HAS_TRTLLM_CONTEXT=0 TENSORRTLLM_PIP_WHEEL=tensorrt_llm==1.0.0rc4 TENSORRTLLM_INDEX_URL=https://pypi.python.org/simple sed '/^#\s/d' /workspace/launch_message.txt > ~/.launch_screen && echo "cat ~/.launch_screen" >> ~/.bashrc`	08/08/2025 11:45 PM UTC
sha256:f0966edc022fca0a44cf7f9d6a95b3748a9e115ab3b0222832fa3e9834ffba5d	COPY	`ATTRIBUTION* LICENSE /workspace/`	08/08/2025 11:45 PM UTC
sha256:4986864206e3ff9441203d04d6a61fd7f9a4612bc6c988fd28f9bf824797909b	RUN	`ARCH_ALT=x86_64 TORCH_VER=2.8.0a0+5228986c39.nv25.5 TORCHVISION_VER=0.22.0a0 SETUPTOOLS_VER=78.1.1 PYTORCH_TRITON_VER=3.3.0+git96316ce52.nvinternal JINJA2_VER=3.1.6 NETWORKX_VER=3.4.2 SYMPY_VER=1.14.0 PACKAGING_VER=23.2 FLASH_ATTN_VER=2.7.3 MPMATH_VER=1.3.0 HAS_TRTLLM_CONTEXT=0 TENSORRTLLM_PIP_WHEEL=tensorrt_llm==1.0.0rc4 TENSORRTLLM_INDEX_URL=https://pypi.python.org/simple uv pip install /workspace/benchmarks`	08/08/2025 11:45 PM UTC
sha256:286e2a0cbbd566dbc79499044b4fa146383948b5de5cae950b9f131af1332a1a	COPY	`components/backends/trtllm /workspace/components/backends/trtllm`	08/08/2025 11:45 PM UTC
sha256:cf337d819751112fea215b345fd233d59cc4875ceaeaf3726335d46bbd5aecb2	COPY	`benchmarks /workspace/benchmarks`	08/08/2025 11:45 PM UTC
sha256:43d495fcfca7c2fe6ed22252d8120e6ac8f797a6837d71bcb3bd2de7ef9fe35e	COPY	`tests /workspace/tests`	08/08/2025 11:45 PM UTC
sha256:e9267f209a9efb3ac8fb861914530df5d546af3b49e8b112fcaf79da453f9b49	RUN	`ARCH_ALT=x86_64 TORCH_VER=2.8.0a0+5228986c39.nv25.5 TORCHVISION_VER=0.22.0a0 SETUPTOOLS_VER=78.1.1 PYTORCH_TRITON_VER=3.3.0+git96316ce52.nvinternal JINJA2_VER=3.1.6 NETWORKX_VER=3.4.2 SYMPY_VER=1.14.0 PACKAGING_VER=23.2 FLASH_ATTN_VER=2.7.3 MPMATH_VER=1.3.0 HAS_TRTLLM_CONTEXT=0 TENSORRTLLM_PIP_WHEEL=tensorrt_llm==1.0.0rc4 TENSORRTLLM_INDEX_URL=https://pypi.python.org/simple echo 'source /usr/local/bin/set_trtllm_env.sh' >> /root/.bashrc`	08/08/2025 11:45 PM UTC
sha256:9d9766aaa4e027a0f4322f25f5f0c9c3dd64470b6b768394db49650e35e0efd6	COPY	`/usr/local/bin/set_trtllm_env.sh /usr/local/bin/set_trtllm_env.sh`	08/08/2025 11:45 PM UTC