DeepStream SDK | NVIDIA NGC

NGC Catalog

Welcome Guest

For contents of this collection and more information, please view on a desktop device.

Description

NVIDIA DeepStream SDK enables developers to build accelerated pipelines for a wide range of use-cases such as IVA, retail, industrial inspection, and many more with minimal development effort.

Curator

Modified

May 6, 2024

Containers

Helm Charts

Models

Resources

NVIDIA DeepStream SDK

NVIDIA’s DeepStream SDK is a multi-sensor AI-based streaming analytics toolkit for video, image, and audio understanding. DeepStream is an integral part of NVIDIA Metropolis, the platform for building end-to-end services and solutions that transform real-time pixels and sensor data into actionable insights. DeepStream SDK features hardware-accelerated building blocks, called plugins, which bring deep neural networks and other complex processing tasks into a processing pipeline. The DeepStream SDK allows you to focus on building optimized Vision AI applications without having to design complete solutions from scratch.

The DeepStream SDK uses AI to perceive pixels and generate metadata while offering integration from the edge-to-the-cloud. The DeepStream SDK can be used to build applications across various use cases including retail analytics, patient monitoring in healthcare facilities, parking management, optical inspection (AOI), supply chain management (logistics), operations and manufacturing.

What is in the NVIDIA DeepStream SDK Collection

NVIDIA NGC offers collections that easily group relevant information together. We have created the NVIDIA DeepStream SDK collection, so it is easy to find all the information in a single place.

DeepStream x86 Containers: Containers for Enterprise Grade GPUs.
DeepStream Jetson containers: Containers for Orin based systems.
DeepStream Debian Installers: For both x86 and Jetson (ARM) based platforms. These assets are now available under the resource page of NGC.
GXF Runtime and Graph Composer Installers: GXF runtime is available for both x86 and Jetson. While Graph Composer can run on x86 systems running Ubuntu 22.04 and Windows 10.
TAO Pretrained Models Supported: All models supported by DeepStream out-of-the-box are now available in the DeepStream collection. Details on how to run these models in DeepStream are available in DeepStream-TAO GitHub repo or the DeepStream documentation.

Please note that we have consolidated the DeepStream containers as follows:

Type	Architecture	Container Name	Target
Triton	Multi-Arch x86 + Jetson	deepstream:7.0-triton-multiarch	Deployment
DeepStream Samples	Multi-Arch x86 + Jetson	deepstream:7.0-samples-multiarch	Deployment
DeepStream-ARM-SBSA	ARM SBSA	depstream:7.0-triton-arm-sbsa	Deployment
Development and Graph Composer	x86	deepstream:7.0-gc-triton-devel	Development. Includes Graph Composer GUI

NOTE: All DeepStream dockerfiles are available on GitHub for easy customization.

DeepStream SDK 7.0 Release

For a full list of new features and changes, and known limitations please refer to the DeepStream 7.0 Release Notes.

Key New Features and Enhancements available in DeepStream 7.0:

Category

Details

New Features

• New DeepStream Libraries. Unleash the power of DeepStream without GStreamer dependency with easy-to-use Python APIs.

• New DeepStream Service Maker. A new abstraction layer to build vision AI applications in minutes.

• DeepStream now supports WSL2 as a development environment.

• DeepStream 3D Framework adds support for LIDAR and RADAR with BEVFusion and V2X models.

• Introducing PipeTuner a new tool to automatically optimize DeepStream application parameters.

• New Single-View 3D tracking capability available on NvTracker.

• Support for ARM SBSA based servers.

Key New Features and Enhancements available on GXF and Graph Composer 4.0

Category

Details

New Features

• C++ and Python Application APIs

• Event-based scheduler

• New logger

• Complex valued parameter types

• Multi-arch support in container builder

• Runtime only package for x86

Enhancements

• Improved Python APIs with distributed execution support

• Asynchronous UCX transmitter/receiver

• More log control options

• Performance enhancements in core to reduce entity creation latency and remove lock contentions

• DLPack and CUDAArray support

Documentation

Item	Documentation
Documentation	DeepStream SDK Documentation DeepStream SDK APIs DeepStream Python APIs
Getting Started	Quick Start Guide Understanding DeepStream Plugins Getting Started with Python Getting Started with Graph Composer
Developing with C/C++	DeepStream Reference Application (deepstream-app) Additional DeepStream reference apps
Developing with Python	Python Application GitHub Repository Compile and Install Python Bindings Python Jupyter Notebooks DeepStream Libraries
Developing with Graph Composer	Graph Composer Reference Apps Graph Composer Apps (GitHub Repo)
DeepStream and TAO Toolkit Integration	TAO Supported Models DeepStream-TAO Apps (GitHub Repo) DeepStream Reference Apps (GitHub Repo)
Deep Dives with DeepStream Ninjas	DeepStream Multi-Object Trackers DeepStream Inference Options with Triton and TensorRT
Additional Examples	Additional GitHub Repos
Learn More	New To DeepStream? Start here Additional resources on DevZone

Supported Hardware and Software Dependencies

Jetson Series	Modules	JetPack	Latest DeepStream Supported
Orin	Jetson AGX Orin, Jetson Orin NX, Jetson Orin Nano IGX (including dGPU mode)	6.0	7.0
Xavier	Jetson AGX Xavier, Jetson Xavier NX	5.1.3	6.3 (legacy)
TX2	Jetson TX2 NX Jetson TX2	4.6.x	6.0 (legacy)
Nano	Jetson Nano	4.6.x	6.0 (legacy)

Enterprise GPU Architecture	GPUs
Ada Lovelace	L4, L40
Ampere	A2, A10, A16, A30, A40, A100, RTX A6000
Hopper	H100
Turing	T4

Software Dependencies	x86 / ARM SBSA	Jetson (via JetPack)
Operating System	Ubuntu LTS 22.04	Ubuntu LTS 22.04
GStreamer	1.20.3	1.20.3
Rivermax	1.40	1.40
DLFW (Triton)	23.10	24.03
TensorRT	8.6.1.6	8.6.2.3
CUDA	12.2 U2	12.2
cuDNN	8.9.6	8.9.4.25
GPU Driver (RM)	535 (535 TRD8)	N/A
CV-CUDA (DeepStream Libraries)	0.5	N/A
NvImageCodec (DeepStream Libraries)	0.2	N/A
PyNvVideoCodec (DeepStream Libraries)	1.0	N/A

Note:

x86 or Jetsonhere.

Technical Support

DeepStream Support is available via:

Method	Available to
Forums	All developers
Direct Support	NVIDIA AI Enterprise License holders

Known Limitations

There are known bugs and limitations in the SDK. To learn more about those, refer to the release notes.

License

The following licenses apply to the DeepStream SDK assets:

Asset	Applicable EULA	Notes
SDK	DeepStream SDK EULA	A copy of the license is available on the following folder of the SDK: `/opt/nvidia/deepstream/deepstream-7.0/LicenseAgreement.pdf`
Containers	DeepStream NGC License	License grants redistribution rights allowing developers to build applications on top of the DeepStream containers
Development Containers	DeepStream NGC Development License	A development-only license. Does not allow redistribution of the container
TAO Models	NVIDIA AI Product License	All TAO pre-trained models included in the DeepStream SDK are covered by the NVIDIA AI Product License.

NOTE: By pulling, downloading, or using the DeepStream SDK, you accept the terms and conditions of the EULA licenses listed above:

Ethical AI

NVIDIA’s platforms and application frameworks enable developers to build a wide array of AI applications. Consider potential algorithmic bias when choosing or creating the models being deployed. Work with the model’s developer to ensure that it meets the requirements for the relevant industry and use case; that the necessary instruction and documentation are provided to understand error rates, confidence intervals, and results; and that the model is being used under the conditions and in the manner intended.