ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
c4d45490232983c50ae5055758fde14d4866e9cf4fc339d164503dec1e363474CONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
09/29/2025 11:03 AM UTC
17f7db721b815b8760f3852cf5c39608d6fea4ed325314f70e86bf7c3704c6dbARG
TRT_LLM_VER=1.2.0rc0
09/29/2025 11:03 AM UTC
856535e87fcde98ee8a807272a01158b9aec0779812f44742b7bb6aa686dd794ARG
GIT_COMMIT=560ded5450b79efde720162fc397d7efa59aae6d
09/29/2025 11:03 AM UTC
60a4a5e72bdaac29fb05e2c42fbc17c17ad4b6b134a7edd6aba5072ff3d0ce70RUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip
09/29/2025 11:03 AM UTC
50caf9d9967f96fdf4404de5d2b74924eb4e099ccb00cb4d2a47f18b2fa8be55COPY
examples examples
09/29/2025 11:03 AM UTC
8942964aebb91830693a5772886209e755ddf4f34b4fa86824f5d3f7313dba75COPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
09/29/2025 11:03 AM UTC
a895485a1d4fe938107e1b864ffca55ab6165b6d4ddef1b5349a67e0152d5412ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
09/29/2025 11:03 AM UTC
a30258e7f8292d88771afe66f48be356a6035ced8df809e66744a67f59f0a28aCOPY
/src/tensorrt_llm/benchmarks benchmarks
09/29/2025 11:03 AM UTC
bc304119aed905f827dfd99b5415d3702e6ed4dd39c9d4555a717e248ec2518bARG
SRC_DIR=/src/tensorrt_llm
09/29/2025 11:03 AM UTC
d067b9033fa57d5147de9255fe40367a0ba01563e3c030f0daa64336c8c7b9a6RUN
/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin &&
  test -f bin/executorWorker &&
  ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig &&
  ! ( ldd -v bin/executorWorker | grep tensorrt_llm | grep -q "not found" )
09/29/2025 11:03 AM UTC
...