NVIDIA Maxine™

NVIDIA Maxine is a suite of GPU-accelerated SDKs and cloud-native microservices for deploying AI features that enhance audio, video, and augmented reality effects for real-time communications services and platforms. Maxine’s state-of-the-art models create high-quality effects that can be achieved with standard microphones and webcams.

Three Maxine SDKs are available: Video Effects, Audio Effects, and Augmented Reality (AR).

In addition to what is available on NGC publicly, our Early Access Program contains additional SDK features and microservices available under the terms and conditions of that program. Apply today.

What is included

This collection houses publicly available Maxine SDK resources and the information needed to use them. These resources contain the SDK Programming Guides detailing the SDK-specific APIs, general feature-level description, and bare-metal installation. If you are interested in Maxine’s early access features, apply through our Early Access Program.

Resources

Audio Effects SDK

(Windows/Linux: 1.3.0) - Improve audio quality from incoming and/or outgoing audio feeds, making conversations easier to understand.

• Key Features
  • Significant performance and quality improvements Audio Super Resolution
  • Acoustic Echo Cancellation
  • Noise Removal
  • Room Echo Removal
  • Noise Removal and Room Echo Removal: Combines both effects to remove noise and suppress audio reverberations
  • Speaker Focus
• Latest Release
  • Significant performance and quality improvements for Speaker Focus (early access)
  • Quality enhancement for Audio Super Resolution
  • Noise Removal and Speaker Focus (early access): Combines both effects to remove noise and isolates the prominent speaker from all other background speakers
• Operating Systems
  • Window 10, Windows 11 64-bit, Ubuntu 18.04, CentOS 7, CentOS 8, RHEL7, RHEL8, Debian 10+
• Dependencies
  • Windows SDK: NVIDIA Display Driver 520.46+, CMake 3.9+
  • Server SDKs (Linux): NVIDIA Display Driver 520.61.05 or later, CUDA 11.8, CMake 3.12+, (TRT 8.5.1.7 and CuDNN 8.6.0 are packaged)
• Supported Hardware
  • Windows SDK: NVIDIA GeForce RTX 20XX and 30XX Series, Quadro RTX 3000, TITAN RTX, or higher (any NVIDIA GPUs with Tensor Cores)
  • Server SDK (Linux): V100, T4, A2, A10, A16, A30, A40, A100 (with MIG support)
  • Support for Ada-generation GPUs for Windows SDKs

Windows Audio Effects SDK and Linux Audio Effects SDK

Video Effects SDK

(Windows/Linux: 0.7.2) - Apply incredible AI effects for video quality enhancement. The Video Effects SDK is free to use within any application on both client-side or server/data center.

• Key Features
  • Virtual Background
  • Super Resolution (up to 4X scaling factor)
  • Upscaler (up to 4X scaling factor)
  • Artifact Reduction
  • Video Noise Removal
• Latest Release
  • Latency improvements for Virtual Background
• Operating Systems
  • Windows 10, Windows 11 64-bit, Ubuntu 18.04, Ubuntu 20.04, CentOS 7
• Dependencies
  • Windows SDK: NVIDIA Display Driver 511.65 or later, CMake 3.12+, Microsoft Visual Studio 2017 or later
  • Server SDKs (Linux): CUDA 11.8.0, TRT 8.5.1.7, cuDNN 8.6.0.163, CMAKE 3.10, Display Driver 520.61 or later
• Supported Hardware
  • Windows SDK: NVIDIA® GeForce RTX® 20XX and 30XX Series, Quadro RTX™ 3000, TITAN RTX™, or higher (any NVIDIA GPUs with Tensor Cores)
  • Server SDK: V100, T4, A10, A30, A100 (with MIG support)
  • Support for Ada-generation GPUs for Windows SDKs

Windows Video Effects SDK and Linux Video Effects SDK

Augmented Reality SDK

(Windows/Linux: 0.8.2)- Enable real-time 3D tracking of a person’s face using a standard web camera. Create unique AR effects such as overlaying 3D content on a face, driving 3D characters and virtual interactions in real time. Note: The Linux version of the Augmented Reality SDK is currently only available in the Early Access Program.

• Key Features
  • Face Tracking
  • Face Landmark tracking
  • Face Mesh
  • Body Pose Estimation
  • Eye Contact
  • Face Expression Estimation
• Latest Release
  • Face Expression Estimation
    • 6DOF head pose now available
    • Expression estimation model updated
    • New face model for visualization with updated blendshapes, and face area partitioning
  • Eye Contact
    • Performance improvements via CUDA graph functionality
• Operating Systems
  • Windows 10, Windows 11 64-bit, Ubuntu 18.04, Ubuntu 20.04, CentOS 7
• Supported Hardware
  • Windows SDK: NVIDIA GeForce RTX 20XX and 30XX Series, Quadro RTX 3000, TITAN RTX, or higher (any NVIDIA GPUs with Tensor Cores)
  • Server SDK: V100, T4, A10, A30, A100 (with MIG support)
  • Support for Ada-generation GPUs for Windows SDKs
• Software Dependencies
  • Windows SDK: NVIDIA Display Driver 511.65+ or more recent, CMake 3.12+
  • Server SDKs (Linux): CUDA 11.8.0, TRT 8.5.1.7, CuDNN 8.6.0.163, CMake 3.12+, NVIDIA Display Driver 520.61 or later

Windows AR SDK and Linux AR SDK (early access program)

Getting started with Maxine

Procedure

Follow the resource cards for specifics on using each of the SDKs. SDK-specific programming guides are available inside Audio Effects SDK, Video Effects SDK, and Augmented Reality SDK Program Guides. You can also find them in the documentation which is available here.

License

The NVIDIA Maxine license agreement is contained in the SDK download packages. Please refer to the SDK packages for the SDK-specific licenses.

Ethical AI

NVIDIA’s platforms and application frameworks enable developers to build a wide array of AI applications. Please consider potential algorithmic bias when choosing or creating the models being deployed. Work with the model’s developer to ensure:

• The model meets the requirements for the relevant industry and use case
• The necessary instruction and documentation are provided to understand error rates, confidence intervals, and results
• The model is being used under the conditions and in the manner intended