ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
f2a021169dcd844215ca74b1664ac13efdefad4eb32d2fa416285b14b35d7dd3CONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
08/15/2025 8:59 AM UTC
6d8142916fa471e4c8bd7b048990417f9ce074a6b903e1686290f892a121ec6eARG
TRT_LLM_VER=1.1.0rc0
08/15/2025 8:59 AM UTC
c3ce25a1c9d3ba869983419a85bbbc8fee9ec3a61085a32fbc1bb7fae2959f4bARG
GIT_COMMIT=26f413ad9004bf4611b12c7aa15aed7897b720fb
08/15/2025 8:59 AM UTC
cb66b3b429598e823c6a73476c9df2cde2050a1b2bc4aadf40e742a42a2abe47RUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip
08/15/2025 8:59 AM UTC
49b324f97ad36ed01e12edd34cb432aa3d84b61ad8601245041f53b3905aa681COPY
examples examples
08/15/2025 8:59 AM UTC
8ca2089d338e43e5312d21775352369dae2320633e9d664dff78a21cb9471308COPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
08/15/2025 8:59 AM UTC
d889fd5bb63d84d28f5afa93ee6e15645d6d19e3cd52ac1726c3c4ae7cbd1424ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
08/15/2025 8:59 AM UTC
c29bae8422f2ec50301f5459fe065e57642c5970bdfb6dc90ee619cd8e7e00c3COPY
/src/tensorrt_llm/benchmarks benchmarks
08/15/2025 8:59 AM UTC
d8a9b2278e08e8c263cf00094edbec7874e0aa4b28a7201b8a091bcae17ac3e0ARG
SRC_DIR=/src/tensorrt_llm
08/15/2025 8:59 AM UTC
52257f65a69aca4b6e45fef0757dfc6953061b846b120a2e63503ef3d81a370cRUN
/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin &&
  test -f bin/executorWorker &&
  ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig &&
  ! ( ldd -v bin/executorWorker | grep tensorrt_llm | grep -q "not found" )
08/15/2025 8:59 AM UTC
...