Sign in to access this content
Model
NVIDIA Audio2Face is a microservice for animating 3D character's facial characteristics to match any audio track, whether for a game, film, or real-time digital assistant.
Sign in to access this content
| Field | Response |
|---|---|
| Intended Task/Domain: | Customer Service, Media & Entertainment |
| Model Type: | Transformer, CNN |
| Intended Users: | Interactive avatar developers, Digital content creators |
| Output: | Facial pose |
| Describe how the model works: | Audio input is encoded and concatenated with emotion label, then passed into CNN-based pose decoder to output facial pose. |
| Name the adversely impacted groups this has been tested to deliver comparable outcomes regardless of: | Not Applicable |
| Technical Limitations & Mitigation: | This model may not work well with poor audio input. |
| Verified to have met prescribed NVIDIA quality standards: | Yes |
| Performance Metrics: | Lipsync accuracy, Latency, Throughput |
| Potential Known Risks: | This model may generate inaccurate lip poses given low-quality audio input. |
| Licensing: | Use of this model is governed by the NVIDIA Open Model License |