What is Wav2Lip? The open-source lip sync model
What Wav2Lip is, how its frozen lip-sync expert works, why it looks dated in 2026, and how the team behind it went on to build sync. labs and sync-3.
Wav2Lip has 13.1k stars and 2.8k forks on GitHub, and the paper behind it has been cited more than 1,445 times, which makes it the most influential open-source lip sync model ever released. Wav2Lip is a 2020 deep learning model that takes a video of a face and a separate audio track and re-syncs the mouth to match the new audio, even for a person the model has never seen. The team behind Wav2Lip went on to build sync. labs, and the same official repo now points people to sync for the HD commercial model.
If you are researching Wav2Lip today, you are usually deciding one of two things: whether to run the open-source model yourself, or whether a modern model does the job better. This page explains what Wav2Lip is, how it works, where it shows its age in 2026, and what the same research lineage looks like now.
Wav2Lip is a speech-to-lip model that works on unseen faces
Wav2Lip generates lip motion for an arbitrary talking-face video from any audio track, without training on that specific person first. You give it a video and an audio clip, and it edits the mouth region of the video so the lips match the words in the audio. It was published as “A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild” at ACM Multimedia 2020.
The “in the wild” part was the breakthrough. Earlier models could re-sync lips on still images or on speakers they had seen during training, but they collapsed on a real, unconstrained video of a stranger, with sections drifting out of sync entirely. Wav2Lip was the first to hold sync on arbitrary identities in real footage, which is why it became the default open-source starting point for a generation of lip sync projects.
How Wav2Lip works: a frozen lip-sync expert
Wav2Lip’s core idea is a pretrained, frozen lip-sync discriminator that grades the generator’s output. Most prior systems trained the discriminator alongside the generator, so it was always a step behind and never strong enough to punish bad lip motion. Wav2Lip instead pretrained a strong lip-sync expert, a SyncNet-style network, on a large corpus of real synced video, then froze it. The generator had to satisfy a teacher that already knew what good sync looked like and could not be gamed by blurrier, easier outputs. We covered the architecture in more depth in the original paper writeup.
The paper’s second contribution was measurement. It introduced LSE-D (Lip-Sync Error Distance) and LSE-C (Lip-Sync Error Confidence), two metrics computed from that frozen expert, plus the ReSyncED benchmark of real synced video. Those metrics are still used to score lip sync models today, which is a large part of why the paper keeps getting cited.
The team behind Wav2Lip built sync. labs
The researchers who created Wav2Lip founded sync. labs to take the same problem past what open-source could do. The connection is not marketing: the official Rudrabha/Wav2Lip repository description tells visitors, “For HD commercial model, please try out Sync Labs.” Wav2Lip was the research; sync-3 and lipsync-2 are what that research became once it was rebuilt for production resolution, speed, and languages.
That lineage matters if you are choosing between running Wav2Lip yourself and using a hosted model. The people who know the model’s limits best are the ones who spent the next several years engineering around them.
Where Wav2Lip falls short in 2026
Wav2Lip renders the mouth at a low resolution, so results look soft or blurry on modern footage. The original model generates the mouth crop at 96x96 pixels, per the model code, which was fine for 2020 research demos but visibly dated against HD and 4K video today. A few limits show up quickly in real use:
- Low resolution. The 96x96 mouth region is upscaled back into the frame, which softens the lower face. There is no native HD or 4K output.
- Audio only, no dubbing pipeline. Wav2Lip syncs to an audio file you already have. It does not translate, generate speech, or clone a voice, so building a dubbing workflow means stitching several tools together.
- Struggles on hard shots. Side profiles, occlusions, extreme poses, and multi-speaker frames are where the mouth region degrades most.
- You run and maintain it. It is a research codebase with model weights and Python dependencies, not a hosted product, so you handle GPUs, setup, and scaling yourself.
None of this makes Wav2Lip a bad choice for a hobby project or a research baseline. It just means the bar for production video has moved a long way since 2020.
Wav2Lip vs modern lip sync models
The gap between the open-source model and current hosted models is mostly resolution, robustness, and what happens around the sync step. Specs below are from each project’s own published sources.
| Model | Resolution | Works on unseen faces | Voice cloning + dubbing | How you run it |
|---|---|---|---|---|
| Wav2Lip | 96x96 mouth region, upscaled | Yes | No | Self-hosted open-source code |
| sync-3 | Up to 4K at 60fps | Yes, zero-shot, preserves how the person speaks | Yes, one-pass across 95+ languages | Hosted: web, API, MCP, plugins |
| lipsync-2 | HD, zero-shot | Yes, preserves how the person speaks | Pairs with one-pass dubbing | Hosted |
Sources: Wav2Lip, sync-3, lipsync-2 as of July 2026.
sync-3 is the current flagship and does everything the earlier models do, only better. It reads the whole scene before adjusting the mouth, where the face sits, the lighting, and who is speaking, which is what holds it together on the side profiles, close-ups, and multi-speaker frames where Wav2Lip degrades, up to 4K at 60fps. It works zero-shot on any video, live-action, animation, or AI-generated, and preserves how a specific person speaks with no training or fine-tuning, the same problem Wav2Lip was reaching for. It can also build a talking video from a single photo with image-to-video. That zero-shot, speaker-preserving approach first shipped in lipsync-2, which is still available, and sync-3 carries it further. And unlike Wav2Lip, sync. labs handles translation, voice cloning, and lip sync in one pass across 95+ languages, so you are not assembling a pipeline by hand.
How to choose between Wav2Lip and a hosted model
Start from what you are trying to ship. If you want to experiment, learn how lip sync works, or need a free research baseline you fully control, Wav2Lip is still a reasonable open-source starting point. If you are producing video anyone will watch, the resolution and robustness gap is the deciding factor.
| Your job | Best fit | Why |
|---|---|---|
| Learn or prototype lip sync for free | Wav2Lip | Open-source, well-documented, the field’s default baseline |
| Ship HD or 4K video that looks current | sync-3 | Up to 4K at 60fps and holds on hard shots |
| Dub a video into another language | sync. labs | One-pass translation, voice cloning, and lip sync across 95+ languages |
| Keep how a specific person speaks | sync-3 | Zero-shot, preserves the speaker’s own style with no fine-tuning |
| Avoid running GPUs and code yourself | sync-3 | Hosted in the browser, API, MCP clients, and plugins |
For a wider view of the open-source options around Wav2Lip, see the best free and open-source lip sync tools.
Frequently asked questions
What is Wav2Lip?
Wav2Lip is an open-source deep learning model, published at ACM Multimedia 2020, that re-syncs the lips in a talking-face video to a separate audio track, even for a person the model has never seen. It was the first model to hold lip sync on arbitrary identities in real, unconstrained video.
Is Wav2Lip free to use?
Yes. Wav2Lip is open-source on GitHub, so you can run the model and its weights yourself for free. You provide the GPUs, setup, and maintenance. For an HD, hosted version, the same repository points people to sync. labs.
How does Wav2Lip work?
Wav2Lip trains its generator against a pretrained, frozen lip-sync expert, a SyncNet-style network trained on real synced video. Because the expert is already strong and cannot be gamed by blurry output, the generator is pushed toward accurate mouth motion. The same expert produces the LSE-D and LSE-C metrics the field still uses.
What is the difference between Wav2Lip and sync-3?
Wav2Lip renders the mouth at a low 96x96 resolution and only syncs to audio you already have. sync-3, built by the same research lineage, runs up to 4K at 60fps, holds on hard shots, works zero-shot on any video while preserving how a specific person speaks, and handles translation, voice cloning, and lip sync in one pass across 95+ languages.
Why does Wav2Lip look blurry?
Wav2Lip generates the mouth region at 96x96 pixels and upscales it back into the frame, which softens the lower face on HD footage. It was designed in 2020 before high-resolution lip sync was practical. Modern models like sync-3 render natively at much higher resolution.
Try sync-3 in the browser and see what the Wav2Lip research lineage looks like at up to 4K, with dubbing across 95+ languages built in.