ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
98f47806d7ebac08717ae5115fa88501be04170bfbce55d98299b2e04b173034CONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
05/25/2026 3:25 PM UTC
73f930d9fe2cbf4071bf3722ae89e975ae1dedcf5e7febf824268699b07838baCOPY
scripts/generate_container_oss_attribution.sh /tmp/generate_container_oss_attribution.sh
05/25/2026 3:25 PM UTC
4c5abdd3bbcdf1f3161872867a75533fde4bd3efa6b10702b7a2db94d77c7882ENV
TRT_LLM_GIT_COMMIT=4517988cb39689f5462000f9251c5255a0492825 TRT_LLM_VERSION=1.3.0rc16
05/25/2026 3:25 PM UTC
d0920c0ce7fcab7b0b3ce7679e76fb61bda6d541509369cb2650e8dbac08b060ARG
TARGETARCH=amd64
05/25/2026 3:25 PM UTC
a6ac6d4e312846f16a8f68822b31c6e73c207f4b8d73b3cc874134e98575e3a3ARG
TRT_LLM_VER=1.3.0rc16
05/25/2026 3:25 PM UTC
5d7a7575a99029d413110f140ea899c66ca59f4d7c420cc96fa16dac18dfe248ARG
GIT_COMMIT=4517988cb39689f5462000f9251c5255a0492825
05/25/2026 3:25 PM UTC
e85d5496866d4a30ffe3cd027369674336d9745071933c70d6b4489ec35be9b6RUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/uv/archive-v0 &&
  rm -rf /usr/local/lib/python3.12/dist-packages/setuptools/_vendor/jaraco.context-5.3.0.dist-info &&
  rm -rf /usr/local/lib/python3.12/dist-packages/setuptools/_vendor/wheel-0.45.1.dist-info
05/25/2026 3:25 PM UTC
95267fdb7f489c4406a7b39559fffdda0fae7b8c26e0e11dc9a5d1c22b6dbf67COPY
examples examples
05/25/2026 3:25 PM UTC
96d850928a2f8d97d03899c3a6666a15c61b5cd65868b5279f13b5ec1ed614c1COPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
05/25/2026 3:25 PM UTC
a0185b4f6c2a78218d5bd97295cf768777841e7d3f247b2314128e3a1edbfe68ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
05/25/2026 3:25 PM UTC
...

NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking "Accept All", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on "Manage Settings." By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.