Active Speaker Detection

The Active Speaker Detection collection contains the required components to run the NVIDIA AR SDK Active Speaker Detection (ASD) effect, which processes video and multiple audio track inputs to detect, identify and track speaker identities across video frames.

This collection is part of the NVIDIA Augmented Reality SDK.

This feature is available for commercial/non-commercial use.

Technical Details

For detailed model architecture, specifications, training information, and ethical considerations, please visit the Active Speaker Detection Model Page.

Installation

To install the Active Speaker Detection feature:

Download the Augmented Reality SDK Core

To download the SDK Core package:

1. Navigate to the Augmented Reality SDK Core Resource
2. Choose the appropriate version for your platform, either <version>_windows or <version>_linux
3. Click Download to download the package

Install the Augmented Reality SDK Core and Features

To install the SDK Core and features, navigate to the AR SDK Documentation
and follow the installation instructions for Windows
or Linux.

To install the Active Speaker Detection feature using the install_feature.ps1 or install_feature.sh script, use the feature name nvaractivespeakerdetection.

Try It Out

To try out the Active Speaker Detection feature, after downloading the SDK and feature, head over to the NVIDIA AR SDK Sample Apps GitHub repository to download and compile the sample applications that demonstrate the Active Speaker Detection functionality. The ActiveSpeakerDetectionApp demonstrates this on Windows and Linux, with an additional ActiveSpeakerDetectionTritonClientApp available on Linux to demonstrate the Triton support for this feature.

Documentation

User Guide: AR SDK Documentation

Terms of Use

The use of NVIDIA Active Speaker Detection is governed by the NVIDIA SOFTWARE LICENSE AGREEMENT and Product-Specific Terms for NVIDIA AI Products. Use of the models is governed by the NVIDIA Open Model License.