rudrabha

co-founder & CTO @ sync

Audio-Visual Face Reenactment

An identity-aware talking head video method that combines a dense motion field from learnable keypoints with audio conditioning on the mouth region, and ships state-of-the-art results across unseen faces, languages, and voices.

research

A lip sync expert is all you need

Wav2Lip uses a pretrained, frozen lip-sync discriminator to push speech-to-lip generation into unconstrained, in-the-wild video, and introduces the LSE-D and LSE-C benchmarks the field still uses today.

research

Speaking styles for lip-to-speech synthesis

We release Lip2Wav, a 120-hour single-speaker dataset, and a new architecture that learns individual speaking styles to generate natural speech from lip movements, four times more intelligible than prior work.

Back to Blog