ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT Inference Server
Container
NVIDIA
NVIDIA
TensorRT Inference Server

TensorRT Inference Server provides a data center inference solution optimized for NVIDIA GPUs. It maximizes inference utilization and performance on GPUs via an HTTP or gRPC endpoint, allowing remote clients to request inference for any model that is being managed by the server, as well as providing real-time metrics on latency and requests.

LayerLabelCreated
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4LABEL
com.nvidia.build.ref=770a232eb96f066c1479b047c001b9224220058e
04/15/2019 6:31 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
NVIDIA_BUILD_REF
04/15/2019 6:31 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4LABEL
com.nvidia.build.id=6146088
04/15/2019 6:31 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
NVIDIA_BUILD_ID=6146088
04/15/2019 6:31 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
NVIDIA_BUILD_ID
04/15/2019 6:31 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENTRYPOINT
/opt/tensorrtserver/nvidia_entrypoint.sh
04/15/2019 6:31 PM UTC
sha256:f01a41b5b94f380e305e5fa2575a80b9598ee9a2df8f2e0a383ada23ff43cb95COPY
file:f5e3ef2755e4e55dcfbc877a31e36e1fa97bd7b2ffeac168efdee2e41dd3e396 in /opt/tensorrtserver
04/15/2019 6:31 PM UTC
sha256:f9d63f46ef85a5c9c5b5a8d054cdb0625354b7d00601d74a5ba3af7495d87aa2RUN
TRTIS_CONTAINER_VERSION=19.04 TRTIS_VERSION=1.1.0 ln -sf ${_CUDA_COMPAT_PATH}/lib.real ${_CUDA_COMPAT_PATH}/lib &&
  echo ${_CUDA_COMPAT_PATH}/lib > /etc/ld.so.conf.d/00-cuda-compat.conf &&
  ldconfig &&
  rm -f ${_CUDA_COMPAT_PATH}/lib
04/15/2019 6:31 PM UTC
sha256:2e015049247f15bcc4d7e2b598558a0450c9fd2b82a4c3644a538fb091d5b581COPY
dir:21603a4e58fb994c8af9dd09dbffe2a6119a311a9817f56704dd0bd09114bac7 in lib
04/15/2019 6:31 PM UTC
sha256:e3efb8dd1e3c6389d2d3a7622771f75694b12beb407fb53e5be5ef47aa1c8dbaCOPY
file:e7c1142ad066d1552a4a773a8b91ac5e577c2f4618a6f487510ffde88e6d848a in bin/
04/15/2019 6:31 PM UTC
...