Microsoft has released a research study detailing VASA-1, an AI model designed to animate portrait photos by synchronizing them with audio files, enabling the images to "talk and sing" in a manner that appears realistic. The primary application of VASA-1 is aimed at the creation of virtual characters. The model excels in generating lip movements that align precisely with the accompanying audio. Moreover, it can depict a range of subtle facial expressions and natural head movements, enhancing the authenticity and liveliness of the animated portraits.
Microsoft has also demonstrated the capabilities of VASA-1 through several videos, including an animated rendition of the Mona Lisa rapping. The model allows users to adjust features such as head movements and gaze direction. In its offline mode, VASA-1 produces videos at a resolution of 512x512 pixels and 45 frames per second, while the online mode supports video generation at up to 40 frames per second. Despite its innovative features, Microsoft has stated that it does not intend to commercialize VASA-1 due to concerns about the potential misuse of the technology in creating deepfake content.
