AI Voice Swap and Lip Sync using Wav2Lip-HQ-Updated

Channel:
Canada NanoNomad
Subscribers:
2,810
Published on ● Video Link: https://www.youtube.com/watch?v=fJOeaAKo09o


Category:
Vlog
Duration: 10:38
5,874 views
90

Using Flowframes to smooth a 30 second video loop made from 3 seconds of video, then taking some synthesized speech and using Wav2Lip-HQ-Updated ESRGAN to lip sync the video to the audio.

Wav2Lip-HQ Updated: https://github.com/GucciFlipFlops1917/wav2lip-hq-updated-ESRGAN

Flowframes: https://nmkd.itch.io/flowframes

AI Voice Cloning GUI: https://www.youtube.com/watch?v=snz-VzgGgmA

Video on Using Facebook's FILM Interpolation model: https://www.youtube.com/watch?v=aTYTTRxD1Hw

Model files mirrored here, as original sources appear offline:
https://huggingface.co/AOLCDROM/WAV2LIP-HQ-Updated-MIRROR/tree/main


Installation:
conda create -n env_name python=3.8 git pip
git clone https://github.com/GucciFlipFlops1917/wav2lip-hq-updated-ESRGAN.git
cd wav2lip-hq-updated-ESRGAN
pip3 install -r requirements.txt


rename to s3fd.pth:
https://www.adrianbulat.com/downloads/python-fan/s3fd-619a316812.pth
wav2lip_gan.pth
https://drive.google.com/uc?id=10Iu05Modfti3pDbxCFPnofmfVlbkvrCm
face_segmentation.pth
From: https://drive.google.com/uc?id=154JgKpzCPW82qINcVieuPH3fZ2e0P812
pretrained.state
From: https://drive.google.com/uc?id=1_MGeOLdARWHylC1PCU2p5_FQztD4Bo7B



Other Videos By NanoNomad


2023-06-04Revisiting YourTTS - Details about Training, Datasets, and experiences Voice Cloning with Coqui TTS
2023-06-03DEMO: YourTTS Multi-speaker VCTK Irish-accented Dataset after 275k Steps trained using Coqui TTS
2023-05-22Tortoise TTS Fine Tuning Wrap-Up
2023-05-16Tortoise TTS DEMO: G-Man performs Gilbert and Sullivan's 'The Major-General's Song'
2023-05-15Train Tortoise TTS in English, Spanish, French, Italian, Portuguese, German, and more? Maybe?
2023-05-10DEMO: Testing French-Speaking Tortoise TTS
2023-05-10DEMO: Testing German-Speaking Tortoise TTS
2023-05-08DEMO: Testing Spanish Speaking Tortoise TTS
2023-05-07DEMO: Testing Tortoise TTS Speaking in Portuguese
2023-05-04Make Using Tortoise TTS Faster with Fine-Tuned Models
2023-05-01AI Voice Swap and Lip Sync using Wav2Lip-HQ-Updated
2023-04-22Voice Cloning with Tortoise TTS and Model Training Using the AI Voice Cloning WebUI
2023-04-07Locally Hosted Chatbots with RWKV through ChatRWKV and the Text-Generation-WebUI | 14B Model on 3GB!
2023-03-29Create Datasets for Voice Model Training on Google Colab | Updated Tools for Coqui TTS Training
2023-03-22Train a VITS Speech Model using Coqui TTS | Updated Script and Audio Processing Tools
2023-03-15Training or Fine Tuning a Hindi Language VITS TTS Voice Model with Coqui TTS on Google Colab
2023-03-05Install and Configure Retroarch for PS Vita with Thumbnails, Overlays and Shaders
2023-03-03Fallout 1 on the PS Vita is the Best Way to Play
2023-02-24Train or Fine Tune VITS on (theoretically) Any Language | Train Multi-Speaker Model | Train YourTTS
2023-02-12Even more Voice Cloning | Train a Multi-Speaker VITS model using Google Colab and a Custom Dataset
2023-02-04Updated | Near-Automated Voice Cloning | Whisper STT + Coqui TTS | Fine Tune a VITS Model on Colab


Tags:
AI
machine learning
lip sync
wav2lip