Resources
Traffic Cam Analyzer on A100 MIG

Traffic Cam Analyzer on A100 MIG

Logo for Traffic Cam Analyzer on A100 MIG
Description
Build an example Traffic Cam Footage Analyzer Using the NVIDIA DeepStream SDK and Triton Inference Server to identify and classify vehicles (sedan, truck, SUV, etc.) from live traffic camera streams
Publisher
NVIDIA
Latest Version
1
Modified
April 4, 2023
Compressed Size
37.1 KB

Scaling DeepStream-Triton on NVIDIA A100 MIG instances through Kubernetes

This repository contains configuration files to run a reference end-to-end video analytics application using NVIDIA DeepStream. The application is an example Traffic Cam Analyzer that locates 4 different objects on the road (car, pedestrian, roadsign and bicycle) and then classifies the cars into 6 different classes - sedan, minivan, truck etc.

We use TrafficCamNet for object detection and VehicleTypeNet for classification, both of which are pre-trained models available in the models section on NGC.

The models are configured using NVIDIA DeepStream and served using Triton Inference Server. A variant of the DeepStream container on NGC comes enabled with Triton Inference Server built-in and can be obtained by locating the appropriate tag.

The deployment is first done on a single 2g.10gb MIG instance of a NVIDIA A100 GPU then scaled all the way up to 8 x A100's, all configured with the same MIG profile to showcase how Multi Instance GPU (MIG) can be leveraged to serve IVA use-cases in parallel - either scaling a single use-case, or deploying different use-cases on a single GPU. The 2g.10gb MIG profile is optimal for this specific use-case, but you may feel free to configure your MIG slices appropriately for your use-case.

Watch a live demonstration of this example at GTC 2021!

Pre-requisite:

  1. A server with 1 or more (preferably 8) NVIDIA A100's, either on cloud (AWS p4dn.24xlarge) or on-prem.
  2. GPUs sliced with 2g.10gb MIG profile.
  3. NVIDIA Driver 460+
  4. Docker image - nvcr.io/nvidia/deepstream:5.1-21.02-triton
  5. Kubernetes setup on master and worker node (for scaling across MIG instances)

Instructions to run:

  1. Launch the docker container

docker run -it --rm --gpus device=<MIG-instance-UUID> nvcr.io/nvidia/deepstream:5.1-21.02-triton

  1. Download this Resource from NGC - Use the WGET Command Up Above or the NGC CLI command

  2. Execute the automate script

cd ds_triton && bash automate_script.sh

Expected output:

On a 2g.10gb MIG instance, this application would run at 30 frames per second for 35 full HD video streams.