ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
c15ad2f95b845144731aadb999cfc07ba16140a583497a9a83f86d585872cf2bCONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
07/07/2025 12:16 AM UTC
aee7e73599d3b257b30257ebbbb92f39a80d909992596bc2f222de459cb60326ARG
TRT_LLM_VER=1.0.0rc2
07/07/2025 12:16 AM UTC
49383eb37e9392f81ef1b174d01aa7adbff2e849b908eade30c8dc93b634735fARG
GIT_COMMIT=66f299a205100154ede4ab1b45a5ad16aa2f02b7
07/07/2025 12:16 AM UTC
b68284f293921f487f6ba29e7a98944da45c971084d89dde99b7eb8fc58c64faRUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip
07/07/2025 12:16 AM UTC
d3e060d9a93efeffb679aba60bc9cb3246c17dc05957755bb2b2c14c17b4be65COPY
examples examples
07/07/2025 12:16 AM UTC
4d72c7cc8c98fff8fe66525422864f78cf1a1d8c2079842b664d5dd8bc62f748COPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
07/07/2025 12:16 AM UTC
2327229e4c3fa9904ae3437c8ba055d95d1105c9f02130b8403a96db3fea879dARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
07/07/2025 12:16 AM UTC
5194448a0727b3df37bf8d316b8303b2ef5dec7d74f348d3dd08fee2b70eb22aCOPY
/src/tensorrt_llm/benchmarks benchmarks
07/07/2025 12:16 AM UTC
9f6fdf384282910bd17ceb0b8348ab9bca84ec7eb51221a378cd923a86d2beafARG
SRC_DIR=/src/tensorrt_llm
07/07/2025 12:16 AM UTC
872bc71314e5070289cc3e96fc2826449831cf5117a8bca71351675f3703e58bRUN
/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin &&
  test -f bin/executorWorker &&
  ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig &&
  ! ( ldd -v bin/executorWorker | grep tensorrt_llm | grep -q "not found" )
07/07/2025 12:16 AM UTC
...