ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
ec6e8b9d68f6fdee506daa70d273b1d07cd9c51291f2575f3e6c59a2dc03fbcdCONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
02/04/2026 5:21 AM UTC
1b1c92a0c9f93f12cc4cbe61fc1f221156abbf20081f8d742c88d20d2a151fb9ARG
TRT_LLM_VER=1.2.0rc2.post2
02/04/2026 5:21 AM UTC
ea389ada3d6e5337ed149962bbc706595c65d6cee87560385aa947ea03e3ea25ARG
GIT_COMMIT=910c070e88366f7d9c7f8474fe080ada90eb4df3
02/04/2026 5:21 AM UTC
c68779fe7776e3155eca184a60d96a2437845fbdad5b1f47e2b8bb1126028aa8RUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip
02/04/2026 5:21 AM UTC
bf73c1e72ecee676973d5ed6ec7642450fae5599cf810614bb2921149617a8edCOPY
examples examples
02/04/2026 5:20 AM UTC
36ccc7c0b0b6d2bbe5df513d4e476dba0c91bbceb6490c901462d88cfb6e678bCOPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
02/04/2026 5:20 AM UTC
b08a925b7ae68f89e5bd9365bb357eb1a023f748175dce64ddb825ce598eec52ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
02/04/2026 5:20 AM UTC
9bb43d19556056a317109b5b11fd68e75574f944fea1a105b492117be2ea08b2COPY
/src/tensorrt_llm/benchmarks benchmarks
02/04/2026 5:20 AM UTC
847f4abe9df3b9cfe6d8bf3d177a22f368ac3dc73c482389cb3d8b1b1d09bb75ARG
SRC_DIR=/src/tensorrt_llm
02/04/2026 5:20 AM UTC
4a6c1088e469354a79b84540a2586915d5b002319c2a3c988f906c8e4e4aeda8RUN
/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin &&
  test -f bin/executorWorker &&
  ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig &&
  ! ( ldd -v bin/executorWorker | grep tensorrt_llm | grep -q "not found" )
02/04/2026 5:20 AM UTC
...