ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
Dynamo Tensorrt-LLM Runtime
Container
NVIDIA
NVIDIA
Dynamo Tensorrt-LLM Runtime

The Dynamo TensorRT-LLM runtime image is a containerized build of Dynamo + TensorRT-LLM which serves as the base runtime environment for tensorrt-llm based inference with Dynamo's distributed inference framework.

LayerLabelCreated
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4CMD
—
08/08/2025 11:45 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENTRYPOINT
/opt/nvidia/nvidia_entrypoint.sh
08/08/2025 11:45 PM UTC
sha256:8c1d32752f23679768528572d68273da1f482ea1131262d14989cf3000737af1RUN
ARCH_ALT=x86_64 TORCH_VER=2.8.0a0+5228986c39.nv25.5 TORCHVISION_VER=0.22.0a0 SETUPTOOLS_VER=78.1.1 PYTORCH_TRITON_VER=3.3.0+git96316ce52.nvinternal JINJA2_VER=3.1.6 NETWORKX_VER=3.4.2 SYMPY_VER=1.14.0 PACKAGING_VER=23.2 FLASH_ATTN_VER=2.7.3 MPMATH_VER=1.3.0 HAS_TRTLLM_CONTEXT=0 TENSORRTLLM_PIP_WHEEL=tensorrt_llm==1.0.0rc4 TENSORRTLLM_INDEX_URL=https://pypi.python.org/simple sed '/^#\s/d' /workspace/launch_message.txt > ~/.launch_screen &&
  echo "cat ~/.launch_screen" >> ~/.bashrc
08/08/2025 11:45 PM UTC
sha256:f0966edc022fca0a44cf7f9d6a95b3748a9e115ab3b0222832fa3e9834ffba5dCOPY
ATTRIBUTION* LICENSE /workspace/
08/08/2025 11:45 PM UTC
sha256:4986864206e3ff9441203d04d6a61fd7f9a4612bc6c988fd28f9bf824797909bRUN
ARCH_ALT=x86_64 TORCH_VER=2.8.0a0+5228986c39.nv25.5 TORCHVISION_VER=0.22.0a0 SETUPTOOLS_VER=78.1.1 PYTORCH_TRITON_VER=3.3.0+git96316ce52.nvinternal JINJA2_VER=3.1.6 NETWORKX_VER=3.4.2 SYMPY_VER=1.14.0 PACKAGING_VER=23.2 FLASH_ATTN_VER=2.7.3 MPMATH_VER=1.3.0 HAS_TRTLLM_CONTEXT=0 TENSORRTLLM_PIP_WHEEL=tensorrt_llm==1.0.0rc4 TENSORRTLLM_INDEX_URL=https://pypi.python.org/simple uv pip install /workspace/benchmarks
08/08/2025 11:45 PM UTC
sha256:286e2a0cbbd566dbc79499044b4fa146383948b5de5cae950b9f131af1332a1aCOPY
components/backends/trtllm /workspace/components/backends/trtllm
08/08/2025 11:45 PM UTC
sha256:cf337d819751112fea215b345fd233d59cc4875ceaeaf3726335d46bbd5aecb2COPY
benchmarks /workspace/benchmarks
08/08/2025 11:45 PM UTC
sha256:43d495fcfca7c2fe6ed22252d8120e6ac8f797a6837d71bcb3bd2de7ef9fe35eCOPY
tests /workspace/tests
08/08/2025 11:45 PM UTC
sha256:e9267f209a9efb3ac8fb861914530df5d546af3b49e8b112fcaf79da453f9b49RUN
ARCH_ALT=x86_64 TORCH_VER=2.8.0a0+5228986c39.nv25.5 TORCHVISION_VER=0.22.0a0 SETUPTOOLS_VER=78.1.1 PYTORCH_TRITON_VER=3.3.0+git96316ce52.nvinternal JINJA2_VER=3.1.6 NETWORKX_VER=3.4.2 SYMPY_VER=1.14.0 PACKAGING_VER=23.2 FLASH_ATTN_VER=2.7.3 MPMATH_VER=1.3.0 HAS_TRTLLM_CONTEXT=0 TENSORRTLLM_PIP_WHEEL=tensorrt_llm==1.0.0rc4 TENSORRTLLM_INDEX_URL=https://pypi.python.org/simple echo 'source /usr/local/bin/set_trtllm_env.sh' >> /root/.bashrc
08/08/2025 11:45 PM UTC
sha256:9d9766aaa4e027a0f4322f25f5f0c9c3dd64470b6b768394db49650e35e0efd6COPY
/usr/local/bin/set_trtllm_env.sh /usr/local/bin/set_trtllm_env.sh
08/08/2025 11:45 PM UTC
...