Nvidia has announced the open-sourcing of Audio2Face, an AI-driven tool that creates realistic facial animations for 3D avatars based on audio input. This initiative allows developers to access the tool and its framework to design realistic 3D characters for games and applications.
Audio2Face operates by analyzing the acoustic features of a voice, which enables it to generate animation data that corresponds to a 3D avatar’s facial expressions and lip movements. According to Nvidia, developers can utilize this tool for crafting 3D characters for both pre-scripted content and live-stream applications.
The tool has already been implemented in various gaming projects, including works by Farm51, the creators of Chernobylite 2: Exclusion Zone, as well as the game Alien: Rogue Incursion Evolved Edition.
Alongside the open-source release of the Audio2Face models and software development kits, Nvidia is also providing access to the tool’s training framework. This feature allows users to customize and adjust the models to suit various use cases.
Source: https://www.theverge.com/news/785981/nvidia-audio2face-ai-voice-animation-open-source