Video Dubbing API Guide

Video dubbing combines translated audio with lip-synced video so dubbed content looks natural in the target language. You can either use Sync Labs’ built-in dubbing flow with dubParams, or provide translated audio yourself and use Sync Labs for the lipsync step.

Prerequisites

• A Sync Labs API key
• A source video (URL or uploaded asset)
• For built-in dubbing: source audio in the video file
• For manual dubbing: translated audio in the target language (from a TTS service or human voice actor)

Install the SDK for your language:

$
# Python
$
pip install syncsdk
$
$
# TypeScript
$
npm i @sync.so/sdk

Set your API key:

$
export SYNC_API_KEY="your-api-key"

Built-in API Dubbing with dubParams

Use dubParams when you want Sync Labs to extract the source audio from the video, translate and dub it through ElevenLabs, then run lipsync on the dubbed result. This is the simplest path when your input video already has source audio.

When dubParams is present:

• provide a single video input with audio
• set dubParams.targetLang to the target language code, such as "es", "fr", or "hi"
• optionally set dubParams.sourceLang; omit it or use "auto" for automatic source-language detection
• optionally set dubParams.numSpeakers; 0 enables automatic speaker detection
• do not include a separate audio input for the translated track; audio inputs are ignored while dubbing is enabled

1
import { SyncClient } from "@sync.so/sdk";
2
3
const sync = new SyncClient();
4
5
const response = await sync.generations.create({
6
    input: [
7
        {
8
            type: "video",
9
            url: "https://your-cdn.com/original-video-with-audio.mp4",
10
        },
11
    ],
12
    model: "lipsync-2",
13
    dubParams: {
14
        providerName: "elevenlabs",
15
        targetLang: "es",
16
        sourceLang: "auto",
17
        numSpeakers: 0,
18
    },
19
});
20
21
console.log(`Dubbing job submitted: ${response.id}`);

Manual Dubbing Pipeline

1
import { SyncClient } from "@sync.so/sdk";
2
3
const sync = new SyncClient();
4
5
// Source video with original language
6
const videoUrl = "https://your-cdn.com/original-video.mp4";
7
// Translated audio in target language
8
const dubbedAudioUrl = "https://your-cdn.com/translated-audio-spanish.wav";
9
10
const response = await sync.generations.create({
11
    input: [
12
        { type: "video", url: videoUrl },
13
        { type: "audio", url: dubbedAudioUrl },
14
    ],
15
    model: "lipsync-2",
16
    options: { sync_mode: "cut_off" },
17
});
18
19
const jobId = response.id;
20
console.log(`Dubbing job submitted: ${jobId}`);

1
let generation = await sync.generations.get(jobId);
2
while (!["COMPLETED", "FAILED", "REJECTED"].includes(generation.status)) {
3
    console.log(`Status: ${generation.status}`);
4
    await new Promise((r) => setTimeout(r, 10000));
5
    generation = await sync.generations.get(jobId);
6
}
7
8
if (generation.status === "COMPLETED") {
9
    console.log(`Dubbed video ready: ${generation.outputUrl}`);
10
} else {
11
    console.log(`Dubbing failed for job ${jobId}`);
12
}

Using the ElevenLabs Integration

Sync Labs has a built-in ElevenLabs integration that handles text-to-speech and lipsync in a single API call. Instead of generating audio separately, you pass the translated text directly.

1
import { SyncClient } from "@sync.so/sdk";
2
3
const sync = new SyncClient();
4
5
const response = await sync.generations.create({
6
    input: [
7
        {
8
            type: "video",
9
            url: "https://your-cdn.com/original-video.mp4",
10
        },
11
        {
12
            type: "text",
13
            provider: {
14
                name: "elevenlabs",
15
                voiceId: "EXAVITQu4vr4xnSDxMaL",
16
                script: "Hola, bienvenidos a nuestra plataforma. Hoy les mostraremos las nuevas funciones.",
17
                stability: 0.5,
18
                similarityBoost: 0.75,
19
            },
20
        },
21
    ],
22
    model: "lipsync-2",
23
    options: { sync_mode: "cut_off" },
24
});
25
26
console.log(`Job ID: ${response.id}`);

Supported Languages

Sync Labs’ lipsync models are language-agnostic. They work with audio in any language — the models analyze mouth shapes from the audio waveform, not the language itself. If your translated audio is clear and well-produced, the lipsync output will match.

For a complete translation pipeline walkthrough (transcription, translation, TTS, and lipsync), see the Video Translation API Guide.

Multi-Speaker Dubbing

For videos with multiple speakers, use the segments API to assign different audio tracks to different time ranges. Each segment can reference a separate audio input with a distinct voice.

1
from sync import Sync
2
from sync.common import Audio, Video
3
4
sync = Sync()
5
6
response = sync.generations.create(
7
    input=[
8
        Video(url="https://your-cdn.com/interview.mp4"),
9
        Audio(url="https://your-cdn.com/speaker-a-spanish.wav", ref_id="speaker_a"),
10
        Audio(url="https://your-cdn.com/speaker-b-spanish.wav", ref_id="speaker_b"),
11
    ],
12
    segments=[
13
        {"startTime": 0, "endTime": 15, "audioInput": {"refId": "speaker_a"}},
14
        {"startTime": 15, "endTime": 30, "audioInput": {"refId": "speaker_b"}},
15
    ],
16
    model="lipsync-2",
17
)

See the Segments Guide for full documentation and more examples.

Performance Tips

Next Steps

• Video Translation API Guide — Build a full transcription-to-lipsync pipeline
• Batch API — Process hundreds of dubbing jobs at once
• Segments Guide — Handle multi-speaker videos
• Text-to-Speech Lip Sync Guide — Combine TTS with lipsync