ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
41528c9ef868f743461b733ef45db346d37346f910b00484cef9a2859ef3b825CONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
09/23/2025 4:20 PM UTC
35a37652ec8b3b1b23ea20ebf5921b5fd008b75bc5292dcee1b08f7cc1f3529cARG
TRT_LLM_VER=1.0.0
09/23/2025 4:20 PM UTC
a22beae8e0b81e7091a96f35f409eeafcb68437b1fa3aa47e53f550b2f5cda72ARG
GIT_COMMIT=ae8270b713446948246f16fadf4e2a32e35d0f62
09/23/2025 4:20 PM UTC
96df974a2702a456fdf2e3b3966723548296db3203df3ae6ea6b117827cf92dfRUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip
09/23/2025 4:20 PM UTC
a68f7a6b41fa0d9865c8d992bccb63cf6ca300ad1534b72ba9f3175485a899d6COPY
examples examples
09/23/2025 4:20 PM UTC
4fe8c36a8c09b29fcc7c708f20d81d7267b08d96e5197e342763c06fdceac652COPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
09/23/2025 4:20 PM UTC
962809c3cb621cd7341ec293dbdbfbf28e4e9fdcadfa72e94e97b57a044989b7ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
09/23/2025 4:20 PM UTC
94758340955e946649ea7bf4f12ec20f365320f569fe1ef4dc14df48663ee08fCOPY
/src/tensorrt_llm/benchmarks benchmarks
09/23/2025 4:20 PM UTC
13824a4b8fca1ef8ea6f565a692868abdb2636399fba7b56dd65f2853b8278adARG
SRC_DIR=/src/tensorrt_llm
09/23/2025 4:20 PM UTC
a724db38bab0276854b316a6ad4f7ee2d9ef3e1572490320d0eb965c88e25f9fRUN
/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin &&
  test -f bin/executorWorker &&
  ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig &&
  ! ( ldd -v bin/executorWorker | grep tensorrt_llm | grep -q "not found" )
09/23/2025 4:20 PM UTC
...