ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT Inference Server
Container
NVIDIA
NVIDIA
TensorRT Inference Server

TensorRT Inference Server provides a data center inference solution optimized for NVIDIA GPUs. It maximizes inference utilization and performance on GPUs via an HTTP or gRPC endpoint, allowing remote clients to request inference for any model that is being managed by the server, as well as providing real-time metrics on latency and requests.

LayerLabelCreated
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
LD_LIBRARY_PATH=/workspace/install/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64
12/06/2019 1:41 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
PATH=//workspace/install/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
12/06/2019 1:41 AM UTC
sha256:69704a0e57fe70124fe2daedc9ef743df467fe36dbd8cc87c93f2d7bca0849cbRUN
python -m pip install --user --upgrade pip &&
  python -m pip install --upgrade install/python/tensorrtserver-*.whl numpy pillow
12/06/2019 1:41 AM UTC
sha256:0edd6fa48901ca5c97858e1a835590c2c252965a0e7cec8d8b69db57c52a3a14COPY
file:c8e3b9bf6aba2778f2250e95144b73aef129b32b070cb7eaf6148b53e4b19abe in images/mug.jpg
12/06/2019 1:41 AM UTC
sha256:44dd2433359630e6143cf49ccf75c0a0b0f74a28afc8595b875dbc2fa1d8976fCOPY
file:739b0ef33071ef274b805fc5b94589d10f3f0428805e32a8401e5b2d355eb58e in /tmp/test.sh
12/06/2019 1:41 AM UTC
sha256:c4ac356ce382cf8cadcc7b0fd1125e63b6fe73016e9e0d4a0df2ef2fa0f4378cRUN
cd install &&
  export VERSION=`cat /workspace/VERSION` &&
  tar zcf /workspace/v$VERSION.clients.tar.gz *
12/06/2019 1:41 AM UTC
sha256:273d7ff1b270b40858b7d36925d874cf5b14a948742aaf8df6beddedb1103404RUN
cd build &&
  cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/workspace/install &&
  make -j16 trtis-clients
12/06/2019 1:41 AM UTC
sha256:d20722e77c5ef9b604ad23dae3b6ab6d8c90c9a4ac28875908344234f7d0608aCOPY
dir:0f7fb2f047530bf2069ead7ef29968479143ee9a9642a748f7e856f2a9a79775 in src/core
12/06/2019 1:36 AM UTC
sha256:d3962457174b46da7f4e84bc323060e2354e66464bbd18bee28651518e97c67cCOPY
dir:ffa6684d26fc84241121cbd5026cf273fa7c0fbdde1518412519bfd467fb4e10 in src/clients
12/06/2019 1:36 AM UTC
sha256:77142de50cd1027473da0be1b63f0684f01b240bc456ef99299f557d61d12b3cCOPY
dir:577b950fabf0175a59a5fe76abf40257f25d68ea72c93fcfec2a1b7a3a4870a2 in build
12/06/2019 1:36 AM UTC

NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking "Accept All", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on "Manage Settings." By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.