Resources
Maxine Windows Augmented Reality SDK

Maxine Windows Augmented Reality SDK

Logo for Maxine Windows Augmented Reality SDK
Features
Description
Maxine Augmented Reality SDK enables real-time modeling and tracking of human faces from video.
Publisher
NVIDIA
Latest Version
0.8.6.1_GA
Modified
December 24, 2024
Compressed Size
1.09 GB

NVIDIA Maxine Windows Augmented Reality SDK

The NVIDIA Maxine AR SDK offers AI-based, real-time 3D face tracking and modeling, as well as body pose estimation based on a standard web camera feed. Developers can create unique AR effects such as overlaying 3D content on a face and driving 3D characters and virtual interactions in real time. The SDK is powered by NVIDIA graphics processing units (GPUs) with Tensor Cores, and as a result, the algorithm throughput is greatly accelerated and latency is reduced.

The SDK has the following AI-powered features:
  1. Face Tracking: detects, localizes, and tracks human faces in images or videos by using bounding boxes.

  2. Face Landmark Tracking: predicts and tracks the pixel locations of human facial landmark points using 68 or 126 landmark points. The 68 detected facial landmarks follow the Multi-PIE 68 point mark-ups information in facial point annotations. The 126 landmark points detector can predict additional points on the cheeks, the eyes, and on laugh lines. Additionally, it tracks head pose and facial deformation due to head movement and expression in three degrees of freedom in real time.

  3. Face Mesh: reconstructs and tracks a human face via a 3D mesh, as well as its head pose, from the provided facial landmarks.

  4. Body Pose Estimation: predicts and tracks 34 key points of the human body, with joint angles, in 2D and 3D. It also supports multi-person tracking.

  5. Eye Contact: simulates eye contact by estimating and aligning gaze with the camera to enhance engagement in video communication.

  6. Face Expression Estimation: estimates face expression (blendshape) coefficients from the provided facial landmarks.

  7. Video Live Portrait: animates a person's portrait photo (a.k.a. source image) using a driving video by matching the head movement and facial expressions in it.

  8. Speech Live Portrait: animates a person's portrait photo (a.k.a. source image) using an audio input by animating the lip motion to match that of the audio.

    9. For more detailed information, please refer to the documentation guide in the SDK packages.