TensorRT LLM Release

NVIDIA

Container

NVIDIA

TensorRT LLM Release

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

Layer	Label		Created
79b71f72d9b74765f1005b5ec6f22acec02c92e1c78de06f15cb7a85ed5db32f	CONFIG	`Entrypoint /opt/nvidia/nvidia_entrypoint.sh; WorkingDir /app/tensorrt_llm; ExposedPorts 6006/tcp, 8888/tcp`	09/15/2025 12:37 PM UTC
36bae3fbe0abe7a243904b025e0b5746aaf1f5e28eb0842d09e1a917284398b5	ARG	`TRT_LLM_VER=1.1.0rc5`	09/15/2025 12:37 PM UTC
25ade3f41ad2aa7ad29a005aede066df5362ed8cdba032d373332a2198979a22	ARG	`GIT_COMMIT=7657d83553ad27ed6e686d6d5d94889b51b8f71e`	09/15/2025 12:37 PM UTC
1708189494e50e24c9ef89ad90c5ef5489ccc663b53c1dfa313d7f2b6bc2ee9c	RUN	`SRC_DIR=/src/tensorrt_llm CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build /bin/bash -c chmod -R a+w examples && rm -v benchmarks/cpp/bertBenchmark.cpp benchmarks/cpp/gptManagerBenchmark.cpp benchmarks/cpp/disaggServerBenchmark.cpp benchmarks/cpp/CMakeLists.txt && rm -rf /root/.cache/pip`	09/15/2025 12:37 PM UTC
8dbee75c0d33e474f771516215ca4068caa5a89b566b60b91e559e08e3858023	COPY	`examples examples`	09/15/2025 12:37 PM UTC
699c026c7ac926b016c8a8667e2c0e816d9188554ea2684208dab0d60f3610dc	COPY	`/src/tensorrt_llm/cpp/build/benchmarks/bertBenchmark /src/tensorrt_llm/cpp/build/benchmarks/gptManagerBenchmark /src/tensorrt_llm/cpp/build/benchmarks/disaggServerBenchmark benchmarks/cpp/`	09/15/2025 12:37 PM UTC
13ebc051b23b662f9903f1923df80a87a880d403f4ff3c31d2ab35919b704a02	ARG	`CPP_BUILD_DIR=/src/tensorrt_llm/cpp/build`	09/15/2025 12:37 PM UTC
c86d2ed27925d7c7ac8dbe4cfbeb6ff5e88e9d00191eac997f811929a9fada7c	COPY	`/src/tensorrt_llm/benchmarks benchmarks`	09/15/2025 12:37 PM UTC
f367c46dbebb401eb8a8fb37a75ca883f59749d691ebd732f9f27914a710259f	ARG	`SRC_DIR=/src/tensorrt_llm`	09/15/2025 12:37 PM UTC
77bd5b47bb03c4293f975b02932a9beba0ab47d0350fe3439d6473211c1cfef8	RUN	`/bin/bash -c ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/bin")') bin && test -f bin/executorWorker && ln -sv $(python3 -c 'import site; print(f"{site.getsitepackages()[0]}/tensorrt_llm/libs")') lib && test -f lib/libnvinfer_plugin_tensorrt_llm.so && echo "/app/tensorrt_llm/lib" > /etc/ld.so.conf.d/tensorrt_llm.conf && ldconfig && ! ( ldd -v bin/executorWorker \| grep tensorrt_llm \| grep -q "not found" )`	09/15/2025 12:37 PM UTC