ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
86d312cd037b18e0da9d2238152eea3d2f3276741b0ce5dae8e84e2e47083d25CONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
08/29/2025 3:13 PM UTC
f7711b2e60176159ffeaef500f54a4f8fee971520e0d9ac25a8db7315344257fARG
TRT_LLM_VER=1.1.0rc2
08/29/2025 3:13 PM UTC
904b590a71195c4e2000501b118d3b31f5f63a6a6b6c04419fbe92797c68e1f2ARG
GIT_COMMIT=15ec2b855d2a829939cda4c6fecea50af294a52c
08/29/2025 3:13 PM UTC
2fc0a5571ae11e94c33e3b5ccaddfcd7adbe0a140d51e606434fd1bb85161371RUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip
08/29/2025 3:13 PM UTC
346b827ecb96e6eaeefa2010945eb426d1b929ca7c7fe5873b3cb575b3f6156aCOPY
examples examples
08/29/2025 3:13 PM UTC
2c94e284a5c8743397ca363c80868ec7707ad3ec53ce90ac2a5c3fac54953386COPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
08/29/2025 3:13 PM UTC
07277d12d690bcb8f15a61e665824a018454ce7406d785f07acfd16aab2f7f05ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
08/29/2025 3:13 PM UTC
18822609265e0358f70cda24dd4ba1c58b62d95d2fcca39af8db9c82336bfc37COPY
/src/tensorrt_llm/benchmarks benchmarks
08/29/2025 3:13 PM UTC
d76bb4f7f9b9ad55ca4c83f64baf6d2349a236249b4316cf5792dc9d4f8255bfARG
SRC_DIR=/src/tensorrt_llm
08/29/2025 3:13 PM UTC
0344711022b420e73a227254a229139fd43685484b21405598a058a3aafc11dbRUN
/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin &&
  test -f bin/executorWorker &&
  ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig &&
  ! ( ldd -v bin/executorWorker | grep tensorrt_llm | grep -q "not found" )
08/29/2025 3:13 PM UTC
...