lipsync-1.8.0-beta in research preview
lipsync-1.8.0-beta is a new zero-shot lip sync model from sync. that holds up on real, in-the-wild video, with sharper teeth, better skin tone matching, and the start of a whole new model family.
We bought a lot of GPUs. Then we baked a new zero-shot lip sync ai model that actually holds up on real, in-the-wild video.
It’s our best model yet by a wide margin, and it’s the first version of something bigger.
A lot of you kept asking for better teeth. Fair ask: bad teeth are usually the first thing that breaks the illusion. Our previous architecture couldn’t get there, so we threw it out and rebuilt around newer techniques that needed more compute.
1.8.x is the start of a new class of models entirely, not a tweak on the previous one.
Technical updates
- better w/ details, matching skin tones, and consistent lip color
- generate high quality, natural looking teeth
- smooth, temporally consistent lipsync
- generates ~1.6 frames per sec rn, will optimize soon
Available now in the playground.