ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
c1347e71299c2075ea8ac9193db0f121ce8b7904a4755baa73e53adf5f39c7b2CONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
09/02/2025 1:10 AM UTC
d6568fef05670439e8033e26a6454ebf4e104c15f09d022d6f10f73967caa156ARG
TRT_LLM_VER=1.1.0rc3
09/02/2025 1:10 AM UTC
ef12ab8cf1f67f7deb0db429dc100b456e781a8dd3b24e40afb939b168f0839fARG
GIT_COMMIT=e81c50dbd2811ec858eccc2c71b5e7a330ff7e24
09/02/2025 1:10 AM UTC
641af7614bb11fa0e86d9b11e41517cddaa8b46668c0d0ea7a0595299843dfccRUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip
09/02/2025 1:10 AM UTC
6427c58bfc3365c065c4799194a4ff5de8ec4f7e66c57d20b3e9d59720ede5b1COPY
examples examples
09/02/2025 1:10 AM UTC
84d0833655c456c16d4d7ce252abc232bd19b11613fc75c95d04182442750990COPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
09/02/2025 1:10 AM UTC
c1b05a7b5ae58484233a69a0c427264bf654ebc094d3e312865ceb047fd0a7f4ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
09/02/2025 1:10 AM UTC
4689694c6e3ba66524670a48e95f599be1046dd286699f3293dcbaa4bbfcfccdCOPY
/src/tensorrt_llm/benchmarks benchmarks
09/02/2025 1:10 AM UTC
183c2a4fe3648379a532e54b14fba5feb4387a81bdf79af9680c4ec19dde6051ARG
SRC_DIR=/src/tensorrt_llm
09/02/2025 1:10 AM UTC
d3528dc8f890742dd4d5f64423179e1afe0172d21392bd5216255ad4b2e5bbefRUN
/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin &&
  test -f bin/executorWorker &&
  ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig &&
  ! ( ldd -v bin/executorWorker | grep tensorrt_llm | grep -q "not found" )
09/02/2025 1:10 AM UTC
...