ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT Inference Server
Container
NVIDIA
NVIDIA
TensorRT Inference Server

TensorRT Inference Server provides a data center inference solution optimized for NVIDIA GPUs. It maximizes inference utilization and performance on GPUs via an HTTP or gRPC endpoint, allowing remote clients to request inference for any model that is being managed by the server, as well as providing real-time metrics on latency and requests.

LayerLabelCreated
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
LD_LIBRARY_PATH=/workspace/install/lib:
10/03/2019 9:55 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
PATH=//workspace/install/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
10/03/2019 9:55 PM UTC
sha256:44f3d6e03ff8f407b2a75446397f105d2e187ef76be6242af680b041335692d4RUN
python -m pip install --user --upgrade pip &&
  python -m pip install --upgrade install/python/tensorrtserver-*.whl numpy pillow
10/03/2019 9:55 PM UTC
sha256:a43725b6291fae825b0482990738fe9f2ba84ed087dae9b7902f3415661a67a5COPY
file:c8e3b9bf6aba2778f2250e95144b73aef129b32b070cb7eaf6148b53e4b19abe in images/mug.jpg
10/03/2019 9:55 PM UTC
sha256:5c49350a2b4ecb5fa8a29b35d0a71d89493437a8b64b97ee86a58b1dfaa5aa19COPY
file:640bfa0c87b517999faf20bf20ba15fcb560877bc7304061c88bd4744b44f3bc in /tmp/test.sh
10/03/2019 9:54 PM UTC
sha256:05df472df363b124b5d65d128783e857c6c61e26aff9a5d351e709bf07fb69f9RUN
cd install &&
  export VERSION=`cat /workspace/VERSION` &&
  tar zcf /workspace/v$VERSION.clients.tar.gz *
10/03/2019 9:54 PM UTC
sha256:1c647a741e7d0fc022309cafa1cbed82de3c8dcda4a407123d84fb74a73d6d63RUN
cd build &&
  cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/workspace/install &&
  make -j16 trtis-clients
10/03/2019 9:54 PM UTC
sha256:25c58c5dc464c5421720fc0549af85c0278f403dd92f7b0f9c90267c5988298bCOPY
dir:fb4fabe73e503700e798611a524c40b83a2e56eb978d99539f882ac396b1b5d1 in src/core
10/03/2019 9:50 PM UTC
sha256:e39b199f8328da0775c5a5a0c1e12438c1e47ba15f374a64228ad63f781104c1COPY
dir:21d21dd35512d280d240f8c8d06cc5de5b2b0cb4357dadf254833c9b62678c4a in src/clients
10/03/2019 9:50 PM UTC
sha256:0ccd4b1b22da6a47b828896d98a706a49df3d9b69468940f443218545fc4e48dCOPY
dir:6ecde89f528c19a78655438b76fda43fe8c3b86d189608a02dd6f7bb6a5e90ce in build
10/03/2019 9:50 PM UTC