ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
ce054c34f8f8d169a4fce78025fd31ea2e6b1b398b1f392e6e37b599465e18c3CONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
10/13/2025 7:50 AM UTC
7b46f6c7024acadff445e1c9f7119a3c2e9faa8e82b3f7617ae8dce5971e6775ARG
TRT_LLM_VER=1.2.0rc0.post1
10/13/2025 7:50 AM UTC
f8dfd869a9cff13cf7394402e57c47c2e8c710ed9d81cd4a62b72f66af464ae1ARG
GIT_COMMIT=6632b4051ed2c67482fada3ecdd7aed08bbbce9e
10/13/2025 7:50 AM UTC
ff45ff612312911c79c0bd36ef94902f8f5c4384d7d8a04f0e8cc88d7644f8e2RUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip
10/13/2025 7:50 AM UTC
60ac3c6abc3afecf48642eced9f55d70ab8b185934bb9f54f67862d662f62f3dCOPY
examples examples
10/13/2025 7:50 AM UTC
a040c60280e6273c179c6fd5311b63ab1e15d6e00246df0ed9026fefc0db40c3COPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
10/13/2025 7:50 AM UTC
8687b75c22b468f9f6a57c02d7a2d06a22db3c75185f0dbf0158e35084307a82ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
10/13/2025 7:50 AM UTC
630e2d8647e7196faffb1c49101d221b84aa697e41f28dc6b962c33ca0d4c2efCOPY
/src/tensorrt_llm/benchmarks benchmarks
10/13/2025 7:50 AM UTC
e7211cb8ca9b359ca6d10946bd368b42672470e054129da5cb33d3b51db664fdARG
SRC_DIR=/src/tensorrt_llm
10/13/2025 7:50 AM UTC
b7f4c6b69a0e4cfcbe54b76c58bab942b1096162e74e108c5fedef351a24d767RUN
/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin &&
  test -f bin/executorWorker &&
  ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig &&
  ! ( ldd -v bin/executorWorker | grep tensorrt_llm | grep -q "not found" )
10/13/2025 7:50 AM UTC
...