ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
TensorRT Inference Server
Container
NVIDIA
NVIDIA
TensorRT Inference Server

TensorRT Inference Server provides a data center inference solution optimized for NVIDIA GPUs. It maximizes inference utilization and performance on GPUs via an HTTP or gRPC endpoint, allowing remote clients to request inference for any model that is being managed by the server, as well as providing real-time metrics on latency and requests.

LayerLabelCreated
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4LABEL
com.nvidia.build.ref=63a054c5d5f2ac1e301d4104ddec8b244a809222
03/11/2019 8:04 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
NVIDIA_BUILD_REF
03/11/2019 8:04 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4LABEL
com.nvidia.build.id=5810010
03/11/2019 8:04 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
NVIDIA_BUILD_ID=5810010
03/11/2019 8:04 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ARG
NVIDIA_BUILD_ID
03/11/2019 8:04 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENTRYPOINT
/opt/tensorrtserver/nvidia_entrypoint.sh
03/11/2019 8:04 AM UTC
sha256:de16cdf0b3792a6823a5794c4c18c9bf155ac16a787d32140e913219b50b0d98COPY
file:25a5fa6ab7e6511c4bb6fe43eca2139d9e0622ac8b4230cb068a1a6620e01f85 in /opt/tensorrtserver
03/11/2019 8:04 AM UTC
sha256:e557f6820f995c48a210e84dd28bc06666450da7996b736aa78d947c93d19ed0COPY
dir:671d615bcc7f1420907a0fd568da620fb34779bc6df4b7bc0a8e22f2a1acaadb in lib
03/11/2019 8:04 AM UTC
sha256:a479a56d1923b7512d343cd566df53191b1fa65f8f8f8518cd392759ecda99a6COPY
file:25e001e1c7b70ed4495b3fca252e93645f8dfc425a1be10f4221942a24d019f2 in bin/
03/11/2019 8:03 AM UTC
sha256:8506f66e36387b62b156eb0c2c43fad88ce39dc3a4327c1505616d579ea0d8b7COPY
file:5e666f796155692be498d908c47ae8d144319cd37ff7987c2f8d8fb640ef2cd7 in LICENSE.pytorch
03/11/2019 8:03 AM UTC
...

NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking "Accept All", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on "Manage Settings." By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.