ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
5a2a102aaadc8f2534e4a583952ceecec633c56da4258861da78189ff5725dfcCONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
03/05/2026 7:12 PM UTC
0424b1ebba298360eac33c39bdede21128c677056da507b535b2af62a1c59008COPY
scripts/generate_container_oss_attribution.sh /tmp/generate_container_oss_attribution.sh
03/05/2026 7:12 PM UTC
f00c084bc17e4431f3fd6d1aaf1825d22566bc3161fc47f381cedb117774b572ENV
TRT_LLM_GIT_COMMIT=fdeaaa9e2e00082921b5f701c7d7822a85dcc243 TRT_LLM_VERSION=1.3.0rc5.post1
03/05/2026 7:12 PM UTC
d08740e636c7c93a3485e5b5c71a6fad62eab2a0d78883e6ae3de3b0a4f34816ARG
TARGETARCH=amd64
03/05/2026 7:12 PM UTC
d6ea2a5cae35869025d0d51f5c38b9a3826138e39aa441abe0b6d46fd382f962ARG
TRT_LLM_VER=1.3.0rc5.post1
03/05/2026 7:12 PM UTC
17d093837da3bd9db1c06266f440ac09a75cbd5639705199d811f3daa86ce6eeARG
GIT_COMMIT=fdeaaa9e2e00082921b5f701c7d7822a85dcc243
03/05/2026 7:12 PM UTC
a4fed94aad2fdb3e8cab0acc7dd48fda768347a956fc903692056d7ac1f04412RUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip &&
  rm -rf /root/.cache/uv/archive-v0 &&
  rm -rf /usr/local/lib/python3.12/dist-packages/setuptools/_vendor/jaraco.context-5.3.0.dist-info &&
  rm -rf /usr/local/lib/python3.12/dist-packages/setuptools/_vendor/wheel-0.45.1.dist-info
03/05/2026 7:12 PM UTC
2a284a0008ff84043d47580df85ce7ca675fef9d8ad9b26c9c29bf90ed60cdf9COPY
examples examples
03/05/2026 7:12 PM UTC
0e1fc6c8477cd14d558264c3f6ddfe20baf5a35b5b38c5f49dd1f42a7b13d584COPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
03/05/2026 7:12 PM UTC
8624a115905a8febda28ad1d6e0d16583f3c01eadfc950d5839a38a4172a9582ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
03/05/2026 7:12 PM UTC
...

NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking "Accept All", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on "Manage Settings." By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.