You generated a HeyGen / D-ID / Synthesia talking head, or fed an existing clip to sync. (formerly SyncLabs) or Wav2Lip, and the mouth is visibly out of sync with the audio. Sometimes lips lead by a few frames, sometimes they trail, sometimes they form the wrong shape entirely. Viewers clock it in under two seconds and trust drops immediately.
Fastest fix: isolate a clean vocals-only track (no music bed, no room noise), re-encode it to 44.1 kHz WAV, and re-render. Bad source audio causes most lip-sync drift, and every modern engine maps phonemes off the audio waveform. If the audio is already clean and the language is non-English, switch the legacy model for a language-agnostic one (sync. lipsync-2, HeyGen Avatar IV, or Synthesia) rather than fighting the offset in post.
Which bucket are you in?
| Symptom | Likely cause | Go to |
|---|---|---|
| Mouth freezes or syncs to background sounds | Noisy / music-bedded source audio | Step 1 |
| English syncs, native language drifts | English-biased legacy model | Step 2 |
| Aligned at start, worse by the end | Sample-rate or fps mismatch (growing drift) | Step 3 |
| Constant offset, same all clip | fps metadata wrong, or re-mux shift | Steps 3 + 6 |
| Wrong mouth shapes, jitter | Face crop too tight/loose, or still input frame | Steps 4 + 7 |
| Fine in tool preview, off in delivery | Audio shifted during final re-encode | Step 6 |
Common causes, ordered by hit rate
1. Source audio has noise, music, or long silences
Every lip-sync engine locks onto phonemes in the waveform. Heavy room noise, a music bed, or silences over ~1 second confuse the model: it freezes the mouth or syncs to noise transients. sync.’s own docs say to isolate and upload the vocals track because instrumental sounds interfere with quality.
How to spot it: open the audio in Audacity. A visible noise floor above roughly -40 dB, or silences over 1 second, mean the model is guessing.
2. A legacy model is fed non-English audio
Wav2Lip, SadTalker, and older HeyGen avatars were trained mostly on English phonemes, so Mandarin, Japanese, or Hindi mouth shapes land on the closest English shape. As of June 2026 this is a tool choice, not a hard limit: sync. lipsync-2/lipsync-2-pro adapt to any language without language-specific training, HeyGen Avatar IV covers 175+ languages, and Synthesia (160+ languages and accents) has the strongest non-English lip-sync of the avatar tools.
How to spot it: English audio syncs cleanly; the same speaker in their native language drifts.
3. Audio sample rate or video fps mismatch (growing drift)
If a tool ingests at one rate and the pipeline assumes another, timing stretches and drift grows over the clip. Note the nuance: Wav2Lip internally resamples audio to 16 kHz (FFT window 800, hop 200, 80 mel bands), so feeding it 22 kHz is fine because it down-samples itself; the danger is a wrapper that mishandles the rate. Cloud tools (HeyGen, sync.) want a clean 44.1 kHz WAV/MP3. A 24 fps clip tagged as 30 fps gets re-timed the same way.
How to spot it: lips align at clip start and drift further by the end. Check the audio sample rate and the declared fps.
4. Face crop too tight or too loose
If the face box clips the chin or swallows the neck, the mouth detector mis-locks and animates the wrong region.
How to spot it: inspect the face-detection overlay if your tool shows one. The box should run forehead to mid-chin.
5. Still or near-still input frame (cloud models)
sync. lipsync-2/lipsync-2-pro require natural speaking motion in the input video. A locked-off photo or a clip where the mouth never moves prevents proper sync regardless of how clean the audio is.
How to spot it: drift only on static-portrait inputs; live footage of the same person syncs fine. (For a true still image, use HeyGen Avatar IV / Talking Photo, which is built for single photos.)
6. Two-pass workflows with audio re-encoded
Render a clip with built-in lip-sync, then re-encode for delivery in a different container or codec, and the audio stream can shift by 1 to 3 frames during the re-mux.
How to spot it: sync was fine in the tool preview but off in the final delivery file.
Before you start
- Save the original audio and source video separately, untouched.
- Decide whether drift is constant (offset bug) or growing (sample-rate / fps bug). This picks your fix.
- Note the tool, its model version, and the audio language.
- Set a tolerance: under 2 frames at 24 fps is usually invisible, 3-5 frames is noticeable, over 5 is unacceptable.
- Back up the project before re-rendering.
Information worth collecting: audio format / sample rate / bitrate / duration / noise floor; video codec / fps (declared and actual) / resolution; tool, model version, language setting; the timestamps where drift is worst; and whether drift varies by phoneme (model issue) or by time (timing issue).
Step-by-step fix
Step 1: Get a clean vocals-only track
Before feeding any lip-sync tool, isolate the voice and remove noise. This is the single biggest lever.
# Audacity (free)
- Noise Reduction: capture a noise profile from ~1s of silence, apply at 12 dB
- Normalize to -3 dBFS
- Trim leading and trailing silence to under 200 ms
- Export 16-bit PCM WAV at 44.1 kHz
# Adobe Podcast Enhance / similar (faster, AI denoise + de-reverb)
- Upload, enhance, download, then re-encode to 44.1 kHz WAVIf the audio has music under it, strip the instrumental first (any stem-splitter, or sync.’s vocal isolation). Cloud engines compare the whole waveform, and the music throws them off.
Step 2: Match the model to the language
# English
- sync. lipsync-2, HeyGen, D-ID, Synthesia all work; Wav2Lip works for quick/local jobs
# Mandarin / Cantonese / Japanese / Korean / Hindi (any non-English)
- sync. lipsync-2 or lipsync-2-pro: language-agnostic, no fine-tune needed
- HeyGen Avatar IV: 175+ languages, diffusion audio-to-expression engine
- Synthesia: strongest non-English avatar lip-sync (160+ languages and accents)
- Avoid plain Wav2Lip / SadTalker without a language-matched fine-tuneFor a second-pass clean-up on existing video, lipsync-2-pro adds reasoning_enabled (extra frame analysis for artifacts and edge cases).
Step 3: Lock audio and video to matching specs
# Re-encode audio to a clean 44.1 kHz WAV (good for HeyGen, sync., Synthesia)
ffmpeg -i input.mp3 -ar 44100 -ac 1 -c:a pcm_s16le clean.wav
# Inspect the declared video framerate
ffprobe -v error -select_streams v:0 -show_entries stream=r_frame_rate input.mp4
# Force video to the correct fps if metadata is wrong
ffmpeg -i input.mp4 -r 24 -c:v libx264 -crf 18 -c:a copy fixed.mp4Use -ac 1 (mono) for talking-head voice; it is what the engines expect and removes any stereo-phase weirdness. Local Wav2Lip will down-sample to 16 kHz on its own, so do not pre-resample for it.
Step 4: Tighten the face crop
For tools that expose face-crop settings (HeyGen advanced, SadTalker):
- Bounding box: forehead to mid-chin, with ~10 percent padding
- Center horizontally on the nose tip
- Avoid full-body or wide shots; tighter helps mouth detection
- HeyGen single photo: use Photo to Video with Avatar IV, front-facing, one subject in frameStep 5: Align offset in post
If a constant offset survives the re-render:
# DaVinci Resolve / Premiere Pro
- Video on V1, audio on A1
- Slide audio 1-5 frames earlier or later until lips land
- For drift that GROWS over time, use a speed/time-stretch ramp:
- clip start: 100 percent speed
- clip end: 100.5 percent (or 99.5 percent) speed
- Time-stretch the shorter stream to match the longer oneConstant offset = slide. Growing offset = time-stretch. Do not time-stretch a constant offset, or you will introduce drift that was not there.
Step 6: Re-render with a consistent codec
# Final delivery without a re-mux audio shift
ffmpeg -i synced.mp4 -c:v libx264 -crf 18 -c:a aac -b:a 192k \
-movflags +faststart -avoid_negative_ts make_zero final.mp4-avoid_negative_ts make_zero prevents the 1-3 frame audio shift on container re-mux. Keep one final-render config for the whole project so delivery never reintroduces drift.
Step 7: Use a second-pass lip-sync if quality is critical
For ad-tier deliverables, re-sync on top of the generated clip:
- First pass: generate the talking head (HeyGen Avatar IV / D-ID / Synthesia), export with audio
- Second pass: feed that clip to sync. lipsync-2-pro
- Set temperature ~0.5 (lower 0.3 for subtle delivery, higher 0.8 for expressive)
- Enable active_speaker_detection for multi-person frames
- This compounds quality, especially for non-English audiotemperature on lipsync-2/lipsync-2-pro ranges 0 to 1, default 0.5, and controls how expressive the lip movement is.
How to confirm it’s fixed
- Watch the first 5 seconds at 100 percent speed; lips should land on phonemes.
- Slow to 25 percent and watch hard consonants (
b,p,m); the mouth must fully close on these. This is the fastest manual sync test. - Check the worst earlier timestamp and the final second; if both land, growing drift is gone.
- Export and watch on phone, laptop, and a large screen. Small screens hide drift; a TV reveals it.
Long-term prevention
- Standardize source audio: vocals-only, 44.1 kHz WAV, mono,
-3 dBFS, under 200 ms leading silence. - Pick a language-agnostic model (sync. lipsync-2, HeyGen Avatar IV, Synthesia) from the start for non-English work.
- Lock the project to a single fps standard (24 or 30); never mix.
- Tighten the face crop in pre-flight, not after the first render.
- Use one final-render codec config across the project.
Common pitfalls
- Treating drift as a tool bug when it is almost always source-audio quality.
- Feeding noisy lavalier / Zoom audio, or audio with a music bed, straight into a lip-sync tool.
- Pre-resampling to a “magic” rate. Cloud tools want clean 44.1 kHz; Wav2Lip resamples to 16 kHz internally, so let it.
- Mixing 24 fps and 30 fps content in one project.
- Time-stretching a constant offset (or sliding a growing one).
- Re-encoding for delivery in a different codec without rechecking offset.
FAQ
Why does lip-sync work in English but drift in my language? Legacy models (Wav2Lip, SadTalker, older avatars) are English-phoneme dominant. As of June 2026, switch to a language-agnostic engine: sync. lipsync-2, HeyGen Avatar IV, or Synthesia. They adapt to any language without a fine-tune.
Is 2-frame drift noticeable? Under 2 frames at 24 fps is invisible to most viewers. 3-5 frames is noticeable, and over 5 frames is unacceptable for delivery.
Constant offset vs growing drift, how do I tell? Compare the first second to the last. If the gap is the same, it is a constant offset (slide the audio). If it widens, it is a timing mismatch (fix sample rate / fps, then time-stretch).
My still photo won’t sync in sync. lipsync-2 needs natural speaking motion in the input video, so a locked-off still won’t work. For a single photo use HeyGen Avatar IV / Talking Photo, which is built for that.
Do I need 16 kHz or 44.1 kHz audio? Hand cloud tools (HeyGen, sync., Synthesia) a clean 44.1 kHz WAV. Wav2Lip down-samples to 16 kHz itself, so don’t pre-resample for it.
Can I fix drift in post on a live-action talking head? Yes. A frame-level audio slide in Resolve fixes constant offset; growing drift needs a time-stretch ramp on the shorter stream.