NVIDIA DeepStream SDK
NVIDIA’s DeepStream SDK is a multi-sensor AI-based streaming analytics toolkit for video, image, and audio understanding. DeepStream is an integral part of NVIDIA Metropolis, the platform for building end-to-end services and solutions that transform real-time pixels and sensor data into actionable insights. DeepStream SDK features hardware-accelerated building blocks, called plugins, which bring deep neural networks and other complex processing tasks into a processing pipeline. The DeepStream SDK allows you to focus on building optimized Vision AI applications without having to design complete solutions from scratch.
The DeepStream SDK uses AI to perceive pixels and generate metadata while offering integration from the edge-to-the-cloud. The DeepStream SDK can be used to build applications across various use cases including retail analytics, patient monitoring in healthcare facilities, parking management, optical inspection (AOI), supply chain management (logistics), operations and industrial inspection.
What is in the NVIDIA DeepStream SDK Collection
NVIDIA NGC offers collections that easily group relevant information together. We have created the NVIDIA DeepStream SDK collection, so it is easy to find all information in a single place.
Please note that we have consolidated the DeepStream containers starting with DeepStream 6.3 release as follows:
Type |
Architecture |
Container Name |
Target |
|---|---|---|---|
Triton |
Multi-Arch |
deepstream:6.3-triton-multiarch |
Deployment |
DeepStream Samples |
x86 only |
deepstream:6.3-samples |
Deployment |
Development and Graph Composer |
x86 |
deepstream:6.3-gc-triton-devel |
x86 development. Includes Graph Composer GUI |
For a full list of new features and changes, and known limitations please refer to the DeepStream 6.3 Release Notes.
Key New Features and Enhancements available on DeepStream 6.3:
Category |
Details |
New Features |
• New multi-arch containers that support both x86 and Jetson platforms • Additional plugins in source-code format: • Support for MQTT protocol • REST APIs to control DeepStream pipeline on-the-fly • CUDA accelerated support for JPEG encoder x86 • Optical flow support on NVIDIA Jetson AGX Orin • deepstream-nmos app now supports AMWA BCP-002-02 spec • New Jupyter Notebook based on deepstream-test3 app |
Enhancements |
• DeepStream tracker Re-ID embeddings are now available as user metadata and available for downstream consumption • Re-ID accuracy improvements for DeepStream Object Trackers • Support for TAO and ONNX models for Re-ID on DeepStream Object Trackers • Video decoder improvements - 10 and 12 bit yuv420 decoding support - 8 bit yuv444 decoding support - 10 and 12 bit YUV444 decoding support for x86 platforms - 10bit YUV 420 and YUV444 h265 encoding support for x86 platforms • Preprocessing plugin now supports Triton inference plugin • DeepStream sample apps adds Triton inference plugin support • Support dynamic creation/destruction of NMOS Senders and Receivers for deepstream-nmos app • UDP-RTP plugin optimizations for supporting Mellanox NIC as Receiver - SMPTE 2110 compliance • New Muxer plugin enhancements • Multiple Performance optimizations |
Key New Features and Enhancements available on GXFand Graph Composer 3.0:
Category |
Details |
New Features |
• New Python APIs in addition to existing C/C++ APIs • Event triggered data-out support |
Enhancements |
• Registry and Graph Composer are now independent entities • GXF scheduler optimizations • GXF now supports Bayer, RAW16 and 3D RGBD formats • GXF error reporting and logging improvements • Graph Composer sub-graph support improvements • Graph Composer applications add support for multiple clock sources |
Item |
Documentation |
|---|---|
Documentation |
DeepStream SDK Documentation |
Getting Started |
Quick Start Guide |
Developing with C/C++ |
DeepStream Reference Application (deepstream-app) |
Developing with Python |
Python Application GitHub Repository |
Developing with Graph Composer |
Graph Composer Reference Apps |
DeepStream and TAO Toolkit Integration |
TAO Supported Models |
Deep Dives with DeepStream Ninjas |
DeepStream Multi-Object Trackers |
Additional Examples |
|
Learn More |
New To DeepStream? Start here |
Jetson Series |
Modules |
JetPack |
DeepStream Supported |
|---|---|---|---|
Orin |
Jetson AGX Orin, Jetson Orin NX, Jetson Orin Nano |
6.3 (latest) |
|
Xavier |
Jetson AGX Xavier, Jetson Xavier NX |
6.3 (latest) |
|
TX2 |
Jetson TX2 NX Jetson TX2 |
6.0 (legacy) |
|
Nano |
Jetson Nano |
6.0 (legacy) |
Enterprise GPU Architecture |
GPUs |
|---|---|
Ada Lovelace |
L4, L40 |
Ampere |
A2, A10, A16, A30, A40, A100, RTX A6000 |
Hopper |
H100 |
Turing |
T4 |
Volta |
V100 |
Note:
If you are looking for older versions of DeepStream, please refer to the x86 or Jetson archive. Archived documentation is available here.
Software Dependencies |
x86 |
Jetson (via JetPack) |
|---|---|---|
Operating System |
Ubuntu LTS 20.04 |
Ubuntu LTS 20.04 |
GStreamer |
1.16 |
1.16 |
Rivermax |
1.20 |
1.20 |
DLFW (Triton) |
23.03 |
23.01 |
TensorRT |
8.5.3 |
8.5.2.2 |
CUDA |
12.1 |
11.4.19 RC5 |
cuDNN |
8.8.x |
8.6 |
GPU Driver (RM) |
525.125.06. (525 TRD4) |
N/A |
DeepStream Support is available via:
Method |
Available to |
|---|---|
Forums |
|
Direct Support |
NVIDIA AI Enterprise License holders |
The following licenses apply to the DeepStream SDK assets:
Asset |
Applicable EULA |
Notes |
|---|---|---|
SDK |
A copy of the license is available on the following folder of the SDK: |
|
Containers |
License grants redistribution rights allowing developers to build applications on top of the DeepStream containers |
|
Development Containers |
A development-only license. Does not allow redistribution of the container |
NOTE: By pulling, downloading, or using the DeepStream SDK, you accept the terms and conditions of the EULA licenses listed above:
NVIDIA’s platforms and application frameworks enable developers to build a wide array of AI applications. Consider potential algorithmic bias when choosing or creating the models being deployed. Work with the model’s developer to ensure that it meets the requirements for the relevant industry and use case; that the necessary instruction and documentation are provided to understand error rates, confidence intervals, and results; and that the model is being used under the conditions and in the manner intended.