ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
cfa1325d86f6831b8b59da76971e73976afa573f580a70c9d3c78cdecd935e74CONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
06/01/2026 8:55 AM UTC
5079c5c20899bf8b54047eb4a7b7a7cb6e2e69955074194cff11ef0973010017COPY
scripts/generate_container_oss_attribution.sh /tmp/generate_container_oss_attribution.sh
06/01/2026 8:54 AM UTC
cc71c343fc5935f148edbf1df2c4713bd8c5022a5429c95b2a593ea932c6acaaENV
TRT_LLM_GIT_COMMIT=a422420db98c2d71a7011c9f12d3a79c194161a4 TRT_LLM_VERSION=1.3.0rc17
06/01/2026 8:54 AM UTC
bd078c9c39c26a0db92b4c6e4c8d21c3bb86b16dabd1181a79c232728c95bf60ARG
TARGETARCH=amd64
06/01/2026 8:54 AM UTC
b9df2153e5c0b8e5a17f75da8c71494482708320fe9b43f5f0e95d744921f6b9ARG
TRT_LLM_VER=1.3.0rc17
06/01/2026 8:54 AM UTC
2b16c70fb331705479d804042823dda00099c90adc975618a00d96799d7b73d5ARG
GIT_COMMIT=a422420db98c2d71a7011c9f12d3a79c194161a4
06/01/2026 8:54 AM UTC
488d6571fcd79f36a275712cd202a3f3f7e282c2eadd5837956239b4345f4b0bRUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/uv/archive-v0 &&
  rm -rf /usr/local/lib/python3.12/dist-packages/setuptools/_vendor/jaraco.context-5.3.0.dist-info &&
  rm -rf /usr/local/lib/python3.12/dist-packages/setuptools/_vendor/wheel-0.45.1.dist-info
06/01/2026 8:54 AM UTC
92dad8da5f015c0954bf409f1d268be87d88fc540dc91428c1371ff6a899b0a5COPY
examples examples
06/01/2026 8:54 AM UTC
d81a3e4910b365d6f7992e7e5acc9f983251da8365e6bb37ad1b11a689d91207COPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
06/01/2026 8:54 AM UTC
6ab75605a448538ca2d16fb92949d5ccfe1fbb461df2465d3a54560f736c7409ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
06/01/2026 8:54 AM UTC
...

NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking "Accept All", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on "Manage Settings." By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.