It is a real-time AI service that analyzes video and multiple audio streams to detect which on-screen speakers are actively talking.
NVIDIA AI for Media Active Speaker Detection ST 2110 NIM
The NVIDIA AI for Media Active Speaker Detection ST 2110 NIM brings multi-speaker detection and identification to live broadcast workflows over IP SMPTE ST 2110 video.
This collection is part of NVIDIA AI for Media.
This NIM is available for commercial use subject to NVIDIA terms. To try out this feature, you must have access to the AI for Media NIM Collection.
Technical Details
For detailed model architecture, specifications, training information, and ethical considerations, please visit the Active Speaker Detection Model Page.
The Active Speaker Detection ST 2110 NIM is designed for live IP video workflows that require real-time speaker intelligence. The feature analyzes video and pre-diarized audio to identify and track active speakers across frames. It returns per-frame speaker data, including bounding boxes, speaker identifiers, active speaking state, and confidence scores. For ST 2110 deployments, this NIM is intended to run with NVIDIA Holoscan for Media.
Installation
To deploy the Active Speaker Detection ST 2110 NIM:
Get Access and Authenticate to NGC
To access the NIM container:
- Confirm that your NGC account has access to the Active Speaker Detection ST 2110 NIM container images: NIM, sample sender, sample receiver, and Helm charts.
- Generate an NGC API key with access to the NGC Catalog. The same key authenticates to both the NVIDIA Container Registry and the NGC Helm Registry.
- Accept the governing terms on each container image and Helm chart page in NGC.
- Authenticate to the NVIDIA Container Registry (nvcr.io) and the NGC Helm Registry (https://helm.ngc.nvidia.com/nim/nvidia/).
Deploy With NVIDIA Holoscan for Media
To deploy the Active Speaker Detection ST 2110 NIM, configure an ST 2110-enabled NVIDIA Holoscan for Media environment and then follow the deployment instructions provided for this NIM.
For initial evaluation, the sample chart provides a complete end-to-end pipeline. For production deployments, use the NIM service chart for direct deployment or the operator chart for Kubernetes-native lifecycle management.
Documentation
- Active Speaker Detection ST 2110 NIM Documentation
- AI for Media Documentation
- NVIDIA Holoscan for Media Documentation
Terms of Use
The use of Active Speaker Detection ST 2110 NIM is governed by the following agreements:
