ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
cfd46ffdee8bc9bc5a90a212298c4a1b24f2cc1aef54a88f8c16c5c4ebc61206CONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
09/05/2025 5:43 AM UTC
22bc36389fa8a7f569cf78df96fc2d46240841702919c8f10372ab89bf4a9c8aARG
TRT_LLM_VER=1.1.0rc2.post1
09/05/2025 5:43 AM UTC
8d11ced5fad4aea87d0f48cde03e35cc3fa2e2a6a729b6f09ce389306aa79cf8ARG
GIT_COMMIT=9d6e87aed37b6f0b3b2be097c5fafe1497190a71
09/05/2025 5:43 AM UTC
4994cd5f137b1c6bf56b6f720a41978dbf89f06ac40cb76713e6e56f8d7160e9RUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip
09/05/2025 5:43 AM UTC
f8bebd2ea5b5eb6410b57faedab796d5a364968f177663384cbdb84875b994cfCOPY
examples examples
09/05/2025 5:43 AM UTC
29142d30c3fc9233890015e8f0d22a80d4c4465520e2a5129c91b37e7330ffefCOPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
09/05/2025 5:43 AM UTC
b3a0bd6f862b04da7b56c2fc29bd10a2256229fd685cd29d0838af08d4cc54d1ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
09/05/2025 5:43 AM UTC
c15a5952d4be20a5132d83be4982de3a52d9d8091cc8c8aef846bfd699d308c9COPY
/src/tensorrt_llm/benchmarks benchmarks
09/05/2025 5:43 AM UTC
8cb9c6e4f50da691db3261cd519e62570f7b4a3557f28be60c6de86c23e98080ARG
SRC_DIR=/src/tensorrt_llm
09/05/2025 5:43 AM UTC
b1cb500d3ca4c3e3932ce29c2baa67fce621e97ec9edf1e52fbd8c601a31da20RUN
/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin &&
  test -f bin/executorWorker &&
  ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig &&
  ! ( ldd -v bin/executorWorker | grep tensorrt_llm | grep -q "not found" )
09/05/2025 5:43 AM UTC
...