ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
c456dc9e1ccacfac81eef26387d5097c9a8688cb42aba545a62c50fe80654441CONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
03/30/2026 9:04 AM UTC
e2a8c2f3ceba5d511f2b3209799c19d8864b7255a118edd37902aea945048f18COPY
scripts/generate_container_oss_attribution.sh /tmp/generate_container_oss_attribution.sh
03/30/2026 9:04 AM UTC
aa80544951292ca3105f95e14527d01708ae01b6a9cf242a51256a2d67fb658bENV
TRT_LLM_GIT_COMMIT=628bb566050d693894ddf22de03581dd101747c3 TRT_LLM_VERSION=1.3.0rc10
03/30/2026 9:04 AM UTC
089250414d2a37b5fd973beb71dbf2329987a15332621a86f9f8c7ba1c139120ARG
TARGETARCH=amd64
03/30/2026 9:04 AM UTC
3ab0f7bef64b2987f50bafc675b5cf7832f396271e957efb8474d4fb2d7693d8ARG
TRT_LLM_VER=1.3.0rc10
03/30/2026 9:04 AM UTC
5e55c6ae6d691c566b2addbc8811889c43826cad21f823aefe84d32f71fbc3e0ARG
GIT_COMMIT=628bb566050d693894ddf22de03581dd101747c3
03/30/2026 9:04 AM UTC
e12fcfffcfd8132a1940fd177c900e90e7cfd925360c6d76e129e1699ac00894RUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/uv/archive-v0 &&
  rm -rf /usr/local/lib/python3.12/dist-packages/setuptools/_vendor/jaraco.context-5.3.0.dist-info &&
  rm -rf /usr/local/lib/python3.12/dist-packages/setuptools/_vendor/wheel-0.45.1.dist-info
03/30/2026 9:04 AM UTC
864499fb68eb59d83b91943b8142ba36744ac599dc83a738e848e668195fc575COPY
examples examples
03/30/2026 9:04 AM UTC
5dbc9acc8cab9ae7878fab12e4ee6975d69f338764124fb850ea03726bdf014cCOPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
03/30/2026 9:04 AM UTC
0f2ca22fa7ccd10c7ebc057173f83d56d8ce4bc56f5c99d2e174865741a8c27cARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
03/30/2026 9:04 AM UTC
...

NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking "Accept All", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on "Manage Settings." By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.