ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
a1fd25f50907bbe3d036aea0c2c04c0305b5d884365a9aa5f4b1701e0ea876fdCONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
05/20/2026 8:09 AM UTC
ad2a292c85ce97434502b0d04f4a0e5498d879c23510c2ed3187bfee18ccbe3cCOPY
scripts/generate_container_oss_attribution.sh /tmp/generate_container_oss_attribution.sh
05/20/2026 8:09 AM UTC
aafc86790750c171bdd6647c1afb555b810808b52d872aa4b37a55019a0bfc02ENV
TRT_LLM_GIT_COMMIT=c72d43d8968211635e1eca2aff88ebb080cd3b3e TRT_LLM_VERSION=1.3.0rc15
05/20/2026 8:09 AM UTC
12035639d48b33467355dc44f317ab8ead515488937c2c222f67ccbd6f16760dARG
TARGETARCH=amd64
05/20/2026 8:09 AM UTC
14320fe0f4c703f00ec97983ac2f507a5b2d7de92a78bb5423d051d5865ea01eARG
TRT_LLM_VER=1.3.0rc15
05/20/2026 8:09 AM UTC
86da4763e8e6789242c3492f7e43c405b42204e489087ae738833a5eb28bf8f3ARG
GIT_COMMIT=c72d43d8968211635e1eca2aff88ebb080cd3b3e
05/20/2026 8:09 AM UTC
dab7cd8eb1cd49d94df9d7c122bee60be4471d78f35e0497134bb55dacc46359RUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/uv/archive-v0 &&
  rm -rf /usr/local/lib/python3.12/dist-packages/setuptools/_vendor/jaraco.context-5.3.0.dist-info &&
  rm -rf /usr/local/lib/python3.12/dist-packages/setuptools/_vendor/wheel-0.45.1.dist-info
05/20/2026 8:09 AM UTC
5e9085f6e9fc6068f504a2fe92c5288038b7668117716727db4869416a259f62COPY
examples examples
05/20/2026 8:09 AM UTC
d4ac42c1e94430e637ec74f31abe463ac296b2015fde0f3c0045827ed38a221dCOPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
05/20/2026 8:09 AM UTC
b4cb0bf1fdcab7d76cef19a8d96bc6c6b9e603b2acfeea321f8eea595be25a06ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
05/20/2026 8:09 AM UTC
...

NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking "Accept All", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on "Manage Settings." By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.