Research

3 posts

Audio-Visual Face Reenactment

An identity-aware talking head video method that combines a dense motion field from learnable keypoints with audio conditioning on the mouth region, and ships state-of-the-art results across unseen faces, languages, and voices.

6 Oct 2022 · 1 min read

research

A lip sync expert is all you need

Wav2Lip uses a pretrained, frozen lip-sync discriminator to push speech-to-lip generation into unconstrained, in-the-wild video, and introduces the LSE-D and LSE-C benchmarks the field still uses today.

23 Aug 2020 · 1 min read

research

Speaking styles for lip-to-speech synthesis

We release Lip2Wav, a 120-hour single-speaker dataset, and a new architecture that learns individual speaking styles to generate natural speech from lip movements, four times more intelligible than prior work.

17 May 2020 · 1 min read

Back to Blog