ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIAKit SDK - Windows (PB25h1)NVIDIAKit SDK - Linux (PB25h1)NVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT LLM Release
Container
NVIDIA
NVIDIA
TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

LayerLabelCreated
f646e7e2e4f5a3e3adbe2687aad447ac93778fdf502e708fdc2fc4fc029ee29aCONFIG
Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp
06/09/2025 7:59 PM UTC
aa594a2fbb86b269c279eb912f6bba8bd7a74e92e143d3eccf9c9426c3860d87ARG
TRT_LLM_VER=0.21.0rc1
06/09/2025 7:59 PM UTC
e38805be1736d53f97dae9b2ccbd1d3e0f68adf63be63b5b29a02de930c6916dARG
GIT_COMMIT=9c012d5bf8936b040b3822b35c823460e8f63a75
06/09/2025 7:59 PM UTC
cb74d9f620b43740732fc5403acb67aa6247f908b4c906775bd90dbc27e6073cRUN
SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples &&
  rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt &&
  rm -rf /root/.cache/pip
06/09/2025 7:59 PM UTC
90febe22cca70b95e691631c4da5c2af1a6dd13c7d126192cd49fa941b788d36COPY
examples examples
06/09/2025 7:59 PM UTC
e32a591e88cbc1c9a63555a5f20a2ff00ae11a47e94ddbdfb02b666e8e15fc9eCOPY
/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/
06/09/2025 7:59 PM UTC
bbe7fbc5dbb5a896b235186dac769f16632915c8556564fe3695ec8fb99e15e0ARG
CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build
06/09/2025 7:59 PM UTC
880582507cbb779e08b767b97bbd4a45dfffba1c4a22c34956fe743453441563COPY
/src/tensorrt_llm/benchmarks benchmarks
06/09/2025 7:59 PM UTC
54c8918a39331411f5a419673fc781fd3afa7698b8ec31cc4b49e6f2f4eab2fbARG
SRC_DIR=/src/tensorrt_llm
06/09/2025 7:59 PM UTC
b7a0f8aa7975d2f7c2945b853312bb35b8cb606f0e836a5d47df82a773cc2294RUN
/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin &&
  test -f bin/executorWorker &&
  ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib &&
  test -f lib/libnvinfer_plugin_tensorrt_llm.so &&
  echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf &&
  ldconfig &&
  ! ( ldd -v bin/executorWorker | grep tensorrt_llm | grep -q "not found" )
06/09/2025 7:59 PM UTC
...