ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
a12c9e59ba3466fcb02fbfdd6daef652bea0995c3bd8112333df4b3a0c25e65bCONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
09/11/2025 2:33 PM UTC
489347b4486804f70d46e15323d23e9bd3104bb99c1c968b4012e757b01cd7a1ARG
TRT_LLM_VER=1.1.0rc2.post2
09/11/2025 2:33 PM UTC
6727d2b9e00ec96f0f646381057b1dc15de74602314a80930a6f70a744064babARG
GIT_COMMIT=ef0d06df5812b510f9d3a03b3cbb6fbf6a06406f
09/11/2025 2:33 PM UTC
3c437a178599158cac17ae46d93a9f1cb6e3fa3f9106228b5c3bd9affb7a0e6eRUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip
09/11/2025 2:33 PM UTC
50ed82506ab2e01569ea4a9400b2f3242d79fd14bd97cc585f3d22398d239ebeCOPY
examples examples
09/11/2025 2:33 PM UTC
f932e01103da2b460ab789ac4ebefb95753b6649ba79e4348db074fd30ccebaaCOPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
09/11/2025 2:33 PM UTC
a06d5e66393bb0289416a81f245e5677414efbecc1b737e1ad41d70c16777194ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
09/11/2025 2:33 PM UTC
1343d11a23623fae7322fee21b8d861ff8509e0319582d984c42941bd83a1827COPY
/src/tensorrt_llm/benchmarks benchmarks
09/11/2025 2:33 PM UTC
87c263eb420e24f560f8fcd505193d91b754ea85dfe0e714f089a80ad5b98940ARG
SRC_DIR=/src/tensorrt_llm
09/11/2025 2:33 PM UTC
5147c73d6083f6a8334f26836d383f768d3c92f391f0aac055a8df0a9cec39f1RUN
/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin &&
  test -f bin/executorWorker &&
  ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig &&
  ! ( ldd -v bin/executorWorker | grep tensorrt_llm | grep -q "not found" )
09/11/2025 2:33 PM UTC
...