Skip to main content
ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
Dynamo Tensorrt-LLM Runtime
Container
NVIDIA
Dynamo Tensorrt-LLM Runtime

The Dynamo TensorRT-LLM runtime image is a containerized build of Dynamo + TensorRT-LLM which serves as the base runtime environment for tensorrt-llm based inference with Dynamo's distributed inference framework.

LayerLabelCreated
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4CMD
—
08/27/2025 3:33 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENTRYPOINT
/opt/nvidia/nvidia_entrypoint.sh
08/27/2025 3:33 AM UTC
sha256:30a08c1922f5f7f84af1e84d59658c693712201d06af68de817519a51f94d355RUN
ARCH_ALT=x86_64 TORCH_VER=2.8.0a0+5228986c39.nv25.6 TORCHVISION_VER=0.22.0a0+95f10a4e SETUPTOOLS_VER=78.1.1 PYTORCH_TRITON_VER=3.3.0+git96316ce52.nvinternal JINJA2_VER=3.1.6 NETWORKX_VER=3.5 SYMPY_VER=1.14.0 PACKAGING_VER=23.2 FLASH_ATTN_VER=2.7.4.post1 MPMATH_VER=1.3.0 HAS_TRTLLM_CONTEXT=0 TENSORRTLLM_PIP_WHEEL=tensorrt_llm==1.0.0rc6 TENSORRTLLM_INDEX_URL=https://pypi.python.org/simple sed '/^#\s/d' /workspace/launch_message.txt > ~/.launch_screen &&
  echo "cat ~/.launch_screen" >> ~/.bashrc
08/27/2025 3:33 AM UTC
sha256:76a1f1ee7abd9456567f277b46aff4c28a9c46637dc468e4151219f897219377COPY
ATTRIBUTION* LICENSE /workspace/
08/27/2025 3:33 AM UTC
sha256:f199e1392571d9a10cc58350dae799174190d197a3ad0d6c764b21dbca1e9e55RUN
ARCH_ALT=x86_64 TORCH_VER=2.8.0a0+5228986c39.nv25.6 TORCHVISION_VER=0.22.0a0+95f10a4e SETUPTOOLS_VER=78.1.1 PYTORCH_TRITON_VER=3.3.0+git96316ce52.nvinternal JINJA2_VER=3.1.6 NETWORKX_VER=3.5 SYMPY_VER=1.14.0 PACKAGING_VER=23.2 FLASH_ATTN_VER=2.7.4.post1 MPMATH_VER=1.3.0 HAS_TRTLLM_CONTEXT=0 TENSORRTLLM_PIP_WHEEL=tensorrt_llm==1.0.0rc6 TENSORRTLLM_INDEX_URL=https://pypi.python.org/simple python3 -m pip install --no-cache-dir --break-system-packages /workspace/benchmarks
08/27/2025 3:33 AM UTC
sha256:418d20f9d58798817d99cd5a9dd4186efd35e738fe53d287dd99ea2cd84f5a62COPY
components/backends/trtllm /workspace/components/backends/trtllm
08/27/2025 3:33 AM UTC
sha256:82993c6cae2956688cdcd8b4217831eff80edafafe20d1ec04de623351c9f9f1COPY
benchmarks /workspace/benchmarks
08/27/2025 3:33 AM UTC
sha256:4ab302f8c48a0aecbf00494c8bce02a652813512ffd7fb2cb74d55f4d19afee2COPY
tests /workspace/tests
08/27/2025 3:33 AM UTC
sha256:4ef78a8661a0282d6510efaa4fbc838d37badc7ba92b77bc78922654e3871e3dCOPY
/usr/local/lib/python3.12/dist-packages/pytorch_triton-3.3.0+git96316ce52.nvinternal.dist-info /usr/local/lib/python3.12/dist-packages/pytorch_triton-3.3.0+git96316ce52.nvinternal.dist-info
08/27/2025 3:33 AM UTC
sha256:15f2e3d8a473f72bdc10815cfe51f71d9f0369bddedf86ec90220ead407813b2COPY
/usr/local/lib/python3.12/dist-packages/triton /usr/local/lib/python3.12/dist-packages/triton
08/27/2025 3:33 AM UTC
1-10 of 112 items

Modal Content

NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking "Accept All", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on "Manage Settings." By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.

Manage Settings