ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
02da80c3b3094a5ca5192ae5d25ac7242ef4499cf84a9bd8133595875703ce04CONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
05/15/2026 5:12 AM UTC
07fd7cc9e052fc2d9e7d3517b413679284c01622ddd622c55609dbbf0c132720COPY
scripts/generate_container_oss_attribution.sh /tmp/generate_container_oss_attribution.sh
05/15/2026 5:12 AM UTC
0986cbc0eeb5493b0302d5293adc764441c26ca5084b4df7c015831b70ee41bfENV
TRT_LLM_GIT_COMMIT=11e16cd71e5898aae29798a0bfeb86109e796cea TRT_LLM_VERSION=1.3.0rc12.post1
05/15/2026 5:12 AM UTC
457792c2d66f019fbc9e5c3f10d6005a5ebfd698f3467fd6e18e8381d8a04324ARG
TARGETARCH=amd64
05/15/2026 5:12 AM UTC
3983c0985e9ee1d37c7860f58e4543e52e48090460db3a6920ce9f933f70f619ARG
TRT_LLM_VER=1.3.0rc12.post1
05/15/2026 5:12 AM UTC
8ac4d7f48830b1d514c35dac9f88757603f6e0d1092facfa9ab47ccef49a2992ARG
GIT_COMMIT=11e16cd71e5898aae29798a0bfeb86109e796cea
05/15/2026 5:12 AM UTC
870d9df37cd804d9e250a4df995924c4689a0d8a169ac2d64f0dabe36cf7c17bRUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/uv/archive-v0 &&
  rm -rf /usr/local/lib/python3.12/dist-packages/setuptools/_vendor/jaraco.context-5.3.0.dist-info &&
  rm -rf /usr/local/lib/python3.12/dist-packages/setuptools/_vendor/wheel-0.45.1.dist-info
05/15/2026 5:12 AM UTC
f2b3057091b36d79debb14ac181f5894168ed936675d586c9a8ea14f7acebde0COPY
examples examples
05/15/2026 5:12 AM UTC
1554276aecbf79ca5e54e3120d83a7104e757f364902d6526342cf91dd7ea9f0COPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
05/15/2026 5:12 AM UTC
e2994a57adf2172e838bb2adb73d57813b6444925805984b15f4cedb2a045590ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
05/15/2026 5:12 AM UTC
...

NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking "Accept All", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on "Manage Settings." By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.