NGC | Catalog
CatalogCollectionsDeepStream SDK

DeepStream SDK

For contents of this collection and more information, please view on a desktop device.

Description

NVIDIA DeepStream SDK enables developers to build accelerated pipelines for a wide range of use-cases such as IVA, retail, industrial inspection, and many more with minimal development effort.

Curator

-

Modified

August 8, 2023
Containers
Helm Charts
Models
Resources

NVIDIA DeepStream SDK

NVIDIA’s DeepStream SDK is a multi-sensor AI-based streaming analytics toolkit for video, image, and audio understanding. DeepStream is an integral part of NVIDIA Metropolis, the platform for building end-to-end services and solutions that transform real-time pixels and sensor data into actionable insights. DeepStream SDK features hardware-accelerated building blocks, called plugins, which bring deep neural networks and other complex processing tasks into a processing pipeline. The DeepStream SDK allows you to focus on building optimized Vision AI applications without having to design complete solutions from scratch.

The DeepStream SDK uses AI to perceive pixels and generate metadata while offering integration from the edge-to-the-cloud. The DeepStream SDK can be used to build applications across various use cases including retail analytics, patient monitoring in healthcare facilities, parking management, optical inspection (AOI), supply chain management (logistics), operations and industrial inspection.

What is in the NVIDIA DeepStream SDK Collection

NVIDIA NGC offers collections that easily group relevant information together. We have created the NVIDIA DeepStream SDK collection, so it is easy to find all information in a single place.

Please note that we have consolidated the DeepStream containers starting with DeepStream 6.3 release as follows:

Type

Architecture

Container Name

Target

Triton

Multi-Arch

deepstream:6.3-triton-multiarch

Deployment

DeepStream Samples

x86 only 
Jetson only

deepstream:6.3-samples 
deepstream-l4t:6.3-samples

Deployment

Development and Graph Composer

x86

deepstream:6.3-gc-triton-devel

x86 development. Includes Graph Composer GUI

DeepStream SDK 6.3 Release

For a full list of new features and changes, and known limitations please refer to the DeepStream 6.3 Release Notes.

Key New Features and Enhancements available on DeepStream 6.3:

 

Category

Details

New Features

• New multi-arch containers that support both x86 and Jetson platforms

• Additional plugins in source-code format:

      - Analytics plugin

      - Multi-Object Tracker plugin

      - New muxer plugin

      - Demuxer plugin

• Support for MQTT protocol

REST APIs to control DeepStream pipeline on-the-fly

• CUDA accelerated support for JPEG encoder x86

Optical flow support on NVIDIA Jetson AGX Orin

deepstream-nmos app now supports AMWA BCP-002-02 spec

• New Jupyter Notebook based on deepstream-test3 app

Enhancements

• DeepStream tracker Re-ID embeddings are now available as user metadata and available for downstream consumption

Re-ID accuracy improvements for DeepStream Object Trackers

• Support for TAO and ONNX models for Re-ID on DeepStream Object Trackers

Video decoder improvements

      - 10 and 12 bit yuv420 decoding support

      - 8 bit yuv444 decoding support

      - 10 and 12 bit YUV444 decoding support for x86 platforms

      - 10bit YUV 420 and YUV444 h265 encoding support for x86 platforms

Preprocessing plugin now supports Triton inference plugin

DeepStream sample apps adds Triton inference plugin support

• Support dynamic creation/destruction of NMOS Senders and Receivers for deepstream-nmos app

UDP-RTP plugin optimizations for supporting Mellanox NIC as Receiver

      - SMPTE 2110 compliance

New Muxer plugin enhancements

• Multiple Performance optimizations



Key New Features and Enhancements available on GXFand Graph Composer 3.0:

Category

Details

New Features

• New Python APIs in addition to existing C/C++ APIs

• Event triggered data-out support

New GXF distributed scheduling support

Enhancements

Registry and Graph Composer are now independent entities  

• GXF scheduler optimizations

GXF now supports Bayer, RAW16 and 3D RGBD formats

• GXF error reporting and logging improvements

• Graph Composer sub-graph support improvements

• Graph Composer applications add support for multiple clock sources

 

Documentation

Item

Documentation

Documentation

DeepStream SDK Documentation
DeepStream SDK APIs
DeepStream Python APIs

Getting Started

Quick Start Guide
Understanding DeepStream Plugins
Getting Started with Python
Getting Started with Graph Composer

Developing with C/C++

DeepStream Reference Application (deepstream-app)
Additional DeepStream reference apps

Developing with Python

Python Application GitHub Repository
Compile and Install Python Bindings
Python Jupyter Notebooks

Developing with Graph Composer

Graph Composer Reference Apps
Graph Composer Apps (GitHub Repo)

DeepStream and TAO Toolkit Integration

TAO Supported Models
DeepStream-TAO Apps (GitHub Repo)
DeepStream Reference Apps (GitHub Repo)

Deep Dives with DeepStream Ninjas

DeepStream Multi-Object Trackers
DeepStream Inference Options with Triton and TensorRT

Additional Examples

Additional  GitHub Repos

Learn More

New To DeepStream? Start here
Additional resources on DevZone

 

Supported Hardware and Software Dependencies

 


Jetson Series

Modules

JetPack

DeepStream Supported

Orin

Jetson AGX Orin, Jetson Orin NX, Jetson Orin Nano

5.1.2

6.3 (latest)

Xavier

Jetson AGX Xavier, Jetson Xavier NX

5.1.2

6.3 (latest)

TX2

Jetson TX2 NX Jetson TX2

4.6.x

6.0 (legacy)

Nano

Jetson Nano

4.6.x

6.0 (legacy)


Enterprise GPU Architecture

GPUs

Ada Lovelace

L4, L40

Ampere

A2, A10, A16, A30, A40, A100, RTX A6000

Hopper

H100

Turing

T4

Volta

V100

 

Note:

 

If you are looking for older versions of DeepStream, please refer to the x86 or Jetson archive. Archived documentation is available here.

 


Software Dependencies

x86

Jetson (via JetPack)

Operating System

Ubuntu LTS 20.04

Ubuntu LTS 20.04

GStreamer

1.16

1.16

Rivermax

1.20

1.20

DLFW (Triton)

23.03

23.01

TensorRT

8.5.3

8.5.2.2

CUDA

12.1

11.4.19 RC5

cuDNN

8.8.x

8.6

GPU Driver (RM)

525.125.06. (525 TRD4)

N/A

 

Technical Support

DeepStream Support is available via:

Method

Available to

Forums

All developers

Direct Support

NVIDIA AI Enterprise License holders
(Note: DeepStream 6.2 is the latest version of DeepStream available on NVAIE)

 

License

The following licenses apply to the DeepStream SDK assets:

Asset

Applicable EULA

Notes

SDK

DeepStream SDK EULA

A copy of the license is available on the following folder of the SDK:
/opt/nvidia/deepstream/deepstream-6.3/LicenseAgreement.pdf

Containers

DeepStream NGC License

License grants redistribution rights allowing developers

 to build applications on top of the DeepStream containers

Development Containers

DeepStream NGC Development License

A development-only license. Does not allow redistribution

of the container

NOTE: By pulling, downloading, or using the DeepStream SDK, you accept the terms and conditions of the EULA licenses listed above:

 

Ethical AI

NVIDIA’s platforms and application frameworks enable developers to build a wide array of AI applications. Consider potential algorithmic bias when choosing or creating the models being deployed. Work with the model’s developer to ensure that it meets the requirements for the relevant industry and use case; that the necessary instruction and documentation are provided to understand error rates, confidence intervals, and results; and that the model is being used under the conditions and in the manner intended.