TensorRT Inference Server

NVIDIA

Container

NVIDIA

TensorRT Inference Server

TensorRT Inference Server provides a data center inference solution optimized for NVIDIA GPUs. It maximizes inference utilization and performance on GPUs via an HTTP or gRPC endpoint, allowing remote clients to request inference for any model that is being managed by the server, as well as providing real-time metrics on latency and requests.

Layer	Label		Created
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4	LABEL	`com.nvidia.build.ref=f08fa83d2e02f17fd9f8095d0ab6346a17458b69`	09/14/2018 2:17 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4	ARG	`NVIDIA_BUILD_REF`	09/14/2018 2:17 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4	LABEL	`com.nvidia.build.id=688039`	09/14/2018 2:17 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4	ENV	`NVIDIA_BUILD_ID=688039`	09/14/2018 2:17 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4	ARG	`NVIDIA_BUILD_ID`	09/14/2018 2:17 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4	ENTRYPOINT	`/opt/tensorrtserver/nvidia_entrypoint.sh`	09/14/2018 2:17 AM UTC
sha256:b84ca821428b267dd24208bac85c02031671ec935c3aa53095c6949a907a092f	COPY	`file:a21a526470474f5f0473877772fa87c942e5c79aca46d1786ca6fbde4c1a1b86 in /opt/tensorrtserver`	09/14/2018 2:17 AM UTC
sha256:40bee3ea6360b919421cfda3b549896a9015590c8994e92c6ac762eb4bfcaf51	COPY	`dir:5c4387df12cb000f5582a15f35e705090b5d055597b90e82ad3c28aff7891f17 in lib`	09/14/2018 2:17 AM UTC
sha256:9264e01df7dfcc8bc147b32da4383d12fd046d469bddb7f01af4be4a1b43b88f	COPY	`dir:d22bac5144df325b0b9cf9e6a271c9003b9d435671d0ec43ba31f01900737006 in bin`	09/14/2018 2:17 AM UTC
sha256:2ff75334c16b2eeb20d3256bdefdff8d9c6b3d1611c4751a62108ff2123bc17f	COPY	`file:cc54bebe042e0ba56830fe9d1064cdb5322205b9ac896fb4a24af2c040576381 in .`	09/14/2018 2:17 AM UTC