ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
9cdca3ec0a0febe03fbb28e9ecde6a79dac380ffce390ab9d1ee2d0e558a767eCONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
11/20/2025 3:42 AM UTC
661dc922360d1543c8d57757069f0427c70d530d0c6267b2df9ea7828103f839ARG
TRT_LLM_VER=1.2.0rc3
11/20/2025 3:42 AM UTC
d26696519fcdb41e7dc27718a9e6a556a9f58b9ee9db2ad46655af55bc4a748cARG
GIT_COMMIT=2128f73d58508a1a0b37119bd851edb19ab88635
11/20/2025 3:42 AM UTC
a4d3b3485d0c453da59d56c52ab777564414a5a0038e86e74893fe915460607bRUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip
11/20/2025 3:42 AM UTC
696e41c6cbf8054cd1b705e86527e33b572ef434858439aab31adcba066b50b2COPY
examples examples
11/20/2025 3:42 AM UTC
1be6ee2d450ae7883852a9f654d17767b8dfde25e26f3ea2131b12c69436ad42COPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
11/20/2025 3:42 AM UTC
cfbde8103be8bfc3ae6d914be6a2da6720e491d920e4f1944e01cb90f803c56dARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
11/20/2025 3:42 AM UTC
539a7bad8459543a0de25f825640cf5619649c02f4c08d26138837ccca3f74f2COPY
/src/tensorrt_llm/benchmarks benchmarks
11/20/2025 3:42 AM UTC
8923569b7377fea11f1ddac9442f3b95737e2a0f8ddceb90ab36c713ed9203e4ARG
SRC_DIR=/src/tensorrt_llm
11/20/2025 3:42 AM UTC
e82757d8e91ea74169ec8cdff85382af091390be232130ad25b033f72b892e2eRUN
/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin &&
  test -f bin/executorWorker &&
  ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig &&
  ! ( ldd -v bin/executorWorker | grep tensorrt_llm | grep -q "not found" )
11/20/2025 3:42 AM UTC
...