ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
ea2b5e6b6559ed7f61acca4568263ab80aa9b2b2658c860411be1fb06642bf2bCONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
06/21/2025 11:57 AM UTC
b154abdf94dc53d90486d436bbbdc55954d7c5f4254ca93dc2d4e525563a8157ARG
TRT_LLM_VER=1.0.0rc0
06/21/2025 11:57 AM UTC
13a5a1bfc23a44f2674c870cdda2299eb7773f6f2aa28b5e9e8b30518f569464ARG
GIT_COMMIT=ebadc130865e828c568bb305fff5d279a5168566
06/21/2025 11:57 AM UTC
2ce9c365c28b21e3b0ccba027acd4d77080f1e20701bf1f4010a2fff5bdfd53fRUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip
06/21/2025 11:57 AM UTC
296c3b47110d1c7949463fbb08fd21c8342c825ad4977e953542c4eb099588c4COPY
examples examples
06/21/2025 11:57 AM UTC
1aa7f14aa76f3a0f9cb81a7da8acb8af7d714d884142c685de75b17841674f30COPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
06/21/2025 11:57 AM UTC
db85b0f16d34be5e0dfb5d74adb130a6c3e8da86ba3f73edf925389344c5dab8ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
06/21/2025 11:57 AM UTC
9d3e71a891d91ef2b5470f0fe35f1656a94c1712204b7d4224c6a81149adaa5cCOPY
/src/tensorrt_llm/benchmarks benchmarks
06/21/2025 11:57 AM UTC
f0e698ccb64cb8eef37fee6c9f9da85714bd12c36a3ac341279c606d62877e32ARG
SRC_DIR=/src/tensorrt_llm
06/21/2025 11:57 AM UTC
85084e5dc8a02a7c2c3b77dc805c8ba907fed985a51b9620793480d3deb437feRUN
/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin &&
  test -f bin/executorWorker &&
  ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig &&
  ! ( ldd -v bin/executorWorker | grep tensorrt_llm | grep -q "not found" )
06/21/2025 11:57 AM UTC
...