ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
TRT_LLM_GIT_COMMIT=ebadc130865e828c568bb305fff5d279a5168566 TRT_LLM_VERSION=1.0.0rc0
06/21/2025 11:57 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
TRT_LLM_VER=1.0.0rc0
06/21/2025 11:57 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
GIT_COMMIT=ebadc130865e828c568bb305fff5d279a5168566
06/21/2025 11:57 AM UTC
sha256:a3ecc34384c7babe61eeacb868daccbc524092554c1f0ea437f89d4daac2e702RUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip
06/21/2025 11:57 AM UTC
sha256:0616548d463763b3cd8eec38b15c803dd373e55c20530ccaf9b35708f165e5abCOPY
examples examples
06/21/2025 11:57 AM UTC
sha256:701c7c634da7ccccf75659a9ed59610e1dfec0dda4adb42ac1cb5231ecfd04fdCOPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
06/21/2025 11:57 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
06/21/2025 11:57 AM UTC
sha256:3349da948e2ed53eb64fd2e8413bb26c94757951e3058eccf4b57aba40f7c7e8COPY
/src/tensorrt_llm/benchmarks benchmarks
06/21/2025 11:57 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
SRC_DIR=/src/tensorrt_llm
06/21/2025 11:57 AM UTC
sha256:ff46064dca0af97f9b82112ebd2f3c21f5e83dc91c39c8a514774d0c520cd67aRUN
/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin &&
  test -f bin/executorWorker &&
  ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig &&
  ! ( ldd -v bin/executorWorker | grep tensorrt_llm | grep -q "not found" )
06/21/2025 11:57 AM UTC
...