ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
1cee14ec2e5ed57b5fdd31c806c54b880a2176a8c1bc1f2a55bfc706c3249a78CONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
02/08/2026 3:53 PM UTC
b7970174b5d6f70bd70727f6055547f021c62b5c5b9d6bf2ba92e092435d2caeCOPY
scripts/generate_container_oss_attribution.sh /tmp/generate_container_oss_attribution.sh
02/08/2026 3:53 PM UTC
c7455655f3327ca8e3a905b098ca7e7ad33332ebce8a056813524c467e14e6baENV
TRT_LLM_GIT_COMMIT=b464c750567e0b1b35712084fda1e575d85fb97c TRT_LLM_VERSION=1.3.0rc3
02/08/2026 3:53 PM UTC
4fcea1b871ae6431b20ffae2e213348a739ce2d977773713aa4326bda2ff9ebfARG
TARGETARCH=amd64
02/08/2026 3:53 PM UTC
fae9e38044b994d2a0a6b70317c01a594fe6de04a100494a56d087cc59fec9b0ARG
TRT_LLM_VER=1.3.0rc3
02/08/2026 3:53 PM UTC
a766bbf4a9d14a76240ffda409778eab3181b7c368203590177b83ec7acf5478ARG
GIT_COMMIT=b464c750567e0b1b35712084fda1e575d85fb97c
02/08/2026 3:53 PM UTC
1295f6a4875791e80938d75517a6877e34cb7528027affba72f81422ddafee0dRUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip &&
  rm -rf /root/.cache/uv/archive-v0 &&
  rm -rf /usr/local/lib/python3.12/dist-packages/setuptools/_vendor/jaraco.context-5.3.0.dist-info &&
  rm -rf /usr/local/lib/python3.12/dist-packages/setuptools/_vendor/wheel-0.45.1.dist-info
02/08/2026 3:53 PM UTC
598c30248e44065b0d181a575bcd94a867afb211abdab611f2ff8e22935d7dc7COPY
examples examples
02/08/2026 3:53 PM UTC
979d785577c1f77aab6be881f428e58b7298771c5414ee966d57d94a3da09b4cCOPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
02/08/2026 3:53 PM UTC
8d3f9448d74b2e7188305b434bf393765ede7f0de75d88b26d79c8f6cb8f9e27ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
02/08/2026 3:53 PM UTC
...

NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking "Accept All", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on "Manage Settings." By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.