NVIDIA Maxine™ is a GPU-accelerated SDK with state-of-the-art AI features for developers to build virtual collaboration and content creation applications such as video conferencing and live streaming.
Maxine’s AI SDKs - Video Effects (VFX), Audio Effects (AFX), and Augmented Reality (AR) - are highly optimized and include modular features that can be chained into end-to-end pipelines to deliver the highest GPU performance, both on PCs and in data centers. Maxine can also be used with NVIDIA Riva, an SDK for building conversational AI applications, to offer world-class language-based capabilities such as transcription and translation.
Developers can add Maxine AI effects into their existing applications or develop new pipelines from scratch using NVIDIA DeepStream, an SDK for building intelligent video analytics, and NVIDIA Video Codec, an SDK for accelerated encode, decode, and transcode.
This collection houses Maxine's SDK containers and the resources needed to use them. These resources contain the SDK Programming Guides detailing the SDK-specific APIs, general feature-level description, and bare-metal installation.
Maxine has three SDKs. This collection makes the Video Effects SDK and the Audio Effects SDK publicly available. To access the Augmented Reality SDK, please apply for the Early Access Program.
Video Effects SDK - The VFX SDK provides five features that can be organized into three categories. Super Resolution and Upscaler enhance the video resolution quality and look and feel of the video streams. Artifact Reduction and Video Noise Removal remove video noise from the streams. Artifact Reduction reduces video compression artifacts, and Video Noise Removal reduces the video degradation caused by low light conditions or a lower quality camera. Lastly, AI Green Screen gives semantic segmentation tailored to typical video conferencing environments. Interested? Check out the container.
Audio Effects SDK - The AFX SDK provides features to reduce unwanted audio noise. It comes with two features, Noise Reduction (NR) and Room Echo Removal (RER). To provide a pleasant experience, NR removes several common background noises using state-of-the-art AI models while preserving the speaker’s natural voice. RER removes reverberations from audio using state-of-the-art AI models, restoring clarity of a speaker’s voice. There is also a model that combines both features into one. Check out the container and the AFX SDK documentation for more! We also prodovide Audio Super Resolution and Acoustic Echo Cancelation to our Early Access Partners. Interested? Please apply to our Early Access Program for more information.
Augmented Reality SDK - Augmented Reality (AR) SDK provides five features. Face tracking, as the name suggests, tracks faces. Facial Landmark Detection which predicts and tracks the pixel locations of human facial landmark points and head poses in images or videos. Face Meshes reconstructs and tracks a 3D human face and its head pose from the provided facial landmarks. Body Pose Detection which predicts and tracks the 3D human pose from images or videos. It predicts 34 keypoints of body pose in 2D and 3D. Lastly, Eye Contact redirects the gaze of the user to center with the camera enabling the user to have an Eye Contact with their audience even while they are looking away. You can enroll in the Early Access Program to access the AR SDK container and check specifics and examples.
There are a plethora of resources you can access like our Programming Guide, Getting Started Guide, Devloper Blogs and API Videos. All these resoureces are accessible via our Getting Started Page
The sample applications provide an example for using Maxine APIs. Some familiarity with CUDA is assumed. To brush up on CUDA basics, check out the tutorials, the documentation and the best practices guide.
There are three main prerequisites for Maxine's Containers:
After installing the prerequisites, follow the container cards for specifics on using each of the SDKs. SDK-specific programming guides are available inside Audio Effects SDK, Video Effects SDK, and Augmented Reality SDK containers. You can also find them in the documentation which is available here.
Please refer to the Container Pages with links available above for SDK-specific licenses.
NVIDIA’s platforms and application frameworks enable developers to build a wide array of AI applications. Please consider potential algorithmic bias when choosing or creating the models being deployed. Work with the model’s developer to ensure: