ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
5e50173d825ca3cf2cf93e446b5d72ed780549566d12b8d97390553109852aabCONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
07/19/2025 8:23 PM UTC
0308036c6f79aa65366e09d5e17255571a5f5c0da41509fb3077c20ab87bd7fbARG
TRT_LLM_VER=1.0.0rc4
07/19/2025 8:23 PM UTC
96019bd51e3a0af92b80013faa95c23ae6f3efa7e1f7a0a655ff0002ee010e8fARG
GIT_COMMIT=69e9f6d48944b2ae0124ff57aa59340aa4dfae15
07/19/2025 8:23 PM UTC
d109224c664b7624c437afc83dadc0443be908c94339d0212582a6b610d34ee2RUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip
07/19/2025 8:23 PM UTC
bbe74a09b55fb19eb366a8f0c98878343a9c2b9236acc53f4e6739c7886f5334COPY
examples examples
07/19/2025 8:23 PM UTC
53a3da386fd2557d15406e4e911ee5f22eebc59a8aff8a3d15606202c5a2bd23COPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
07/19/2025 8:23 PM UTC
02801f75ba4872e5fb33728b5c4a31d7518748c9c008d7a5981b68dc232aaa70ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
07/19/2025 8:23 PM UTC
82d4abc17825efd4ebf119a981270a06fcbb05a2c5dd80bb8762e6b25b35fae2COPY
/src/tensorrt_llm/benchmarks benchmarks
07/19/2025 8:23 PM UTC
0394c2e859066c58c915ad9a56a427b1a669e7de6ea258b1640d4078279efcadARG
SRC_DIR=/src/tensorrt_llm
07/19/2025 8:23 PM UTC
b764798997ff4048d811ca3be6702af81cf7b3abe1c7405a981fba42bb325962RUN
/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin &&
  test -f bin/executorWorker &&
  ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig &&
  ! ( ldd -v bin/executorWorker | grep tensorrt_llm | grep -q "not found" )
07/19/2025 8:23 PM UTC
...