ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
1b84c0188d3604b8c00130bf061ffea1f760bffe55fdbceb208b47ab0a1eff23CONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
08/03/2025 6:35 PM UTC
838eaf918e5b97ed9383975dbd7620ea25aa614abfeec4d6f2f711797056ee53ARG
TRT_LLM_VER=0.21.0
08/03/2025 6:35 PM UTC
91794220b90f82f7b77b7b292bfe407c32bd13627dd4c32998ab8207ea9bdee6ARG
GIT_COMMIT=751d5f175ca485f26227f69756179d209d8ebbda
08/03/2025 6:35 PM UTC
639d93960ddbc3fc8ab66f3c60a6f331c7b300486125ec68feb12a12ffbeefceRUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip
08/03/2025 6:35 PM UTC
26faad6e459353670dcbf12ccd2c0267de3f8f64314111f6f23422d059fd3d3eCOPY
examples examples
08/03/2025 6:35 PM UTC
bc1a745060de211d0e7398c8e6c2efc80254eb9e871cabd29e80435adef98888COPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
08/03/2025 6:35 PM UTC
84714c55a7b2822bbc55d92a9c7fd3447fc00adc013c24b30594e43d6f8d57acARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
08/03/2025 6:35 PM UTC
9ec4b2965d1cb9dc59059a50fda75a54df62fa429bf1baf7406b9fe80338abbdCOPY
/src/tensorrt_llm/benchmarks benchmarks
08/03/2025 6:35 PM UTC
4f5ad902dea7527527ee79d9d1b24b58f804430529a2e02e90ed84aa00ef4fc0ARG
SRC_DIR=/src/tensorrt_llm
08/03/2025 6:35 PM UTC
cf1d57ff3a6ef5ef43a4ab6b3dd51ef4f26706261fdc041fb88f280566bc4bcdRUN
/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin &&
  test -f bin/executorWorker &&
  ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig &&
  ! ( ldd -v bin/executorWorker | grep tensorrt_llm | grep -q "not found" )
08/03/2025 6:35 PM UTC
...