✔ DeepFake ANYONE with ONLY ONE PHOTO! 👉 Using ROOP + Retrieval Voice Conversion (RVC)
21🔨 Chris Hemsworth (THOR) having trouble using a basic electric razor. 🤣
Here is a quick deepfake test I created using open-source software as listed below. Originally I posted it on Reddit and socials a few months ago, and now it's on YouTube since I plan to make some tutorials showing my process for anyone who is interested. Since first creating this video several months ago I've managed to make several advancements to the overall look and quality of the output.
=========LINKS:=========
🚨▶RVC - https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI
🚨▶ROOP - https://github.com/s0md3v/roop
1️⃣▶SUBSCRIBE FOR MORE IN-DEPTH TUTORIALS COMING SOON!
🔗▶ Original Reddit Post - https://www.reddit.com/r/StableDiffusion/comments/13znvyx/chris_hemsworth_is_weird/?utm_source=share&utm_medium=web2x&context=3
🔗▶ Original Instagram Post - https://www.instagram.com/p/CtCqWiPRRLw/
🔗▶ Original FB Post - https://www.facebook.com/1429283064/videos/1431054767752725/
=========CONTACT/SUPPORT=========
☕Buy Me a Coffee! - https://bmc.link/calebcriste
💘Help Support Me On Patreon! - https://www.patreon.com/RealityCheckVR
📆Book a One-on-One Consultation! - https://bit.ly/3Okad8v
-
⚫Twitter: https://twitter.com/RealityCheckVR
🔵FB Page: https://www.facebook.com/RealityCheckVR
🟠Instagram: https://instagram.com/RealityCheckVR
🟡 LinkedIn! - https://www.linkedin.com/in/calebcriste/
#chrishemsworth #deepfake #voiceclone
--------LONG DESCRIPTION-------- (From original Reddit Post)
This was my first attempt at any sort of DeepFake video creation and there is certainly a lot of room for improvement. I spent about a day testing out different workflows and came up with one that works well for someone running an old 1080TI GPU. If you guys are looking for a more in-depth video tutorial, let me know what questions you have and I will make one ASAP!
For the one photo face swap I used ROOP - https://github.com/s0md3v/roop
This uses InsightFace to swap the faces frame by frame
(I’ve only got this working using my CPU, a 3-minute video at 30fps takes me 1-2 hours to swap.)
For the voice cloning, I used RVC - https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI
They recommend 15-50 minutes of isolated speaking or singing to train a voice. All famous people generally have a lot of interviews and or audiobooks that can be used to clone voices very quickly. With my GPU it took just under 3 hours to train 20 minutes of Chris’s voice at 200 epochs.
When the video comes out it generally has some fuzz around the face because the quality of the model that is currently public is only 128x128 face resolution, because of this I created a softer feather by placing the original footage below the face-swapped footage and feathered back in the original pixels around the face, moving the mask as the face moved frame by frame. (this step can be made faster by auto-tracking the mask to my face in the future)
To clear up the face I tested out Topaz Video Enhancer which did a pretty good job using the ‘Artemis Low Quality’ Model’ but oversharpened a lot of facial hair.
I then tried HitPaw Video Enhancer but it was too cartoony, still when blended at 15-20% it added to the overall realism. Still, both of these programs have free trials but are not a good solution for open-source workflows.
*To clear up the face using Stable Diffusion I used the ControlNet Tile processor and played around with the denoise sensitivity until I had what I felt was a better-looking image. At that point, I just batch-upscaled the video and blended it with the original at about 40%.
As far as the actual video goes idk what was going on in my head. I didn’t plan anything or script anything. I just had the idea to do the test so I set up my camera and started saying whatever random stuff I could. I definitely want to do some more of these with some actual planning.
Keep your hands and hair out of your face, quick movements aren't great either, keep your face away from the edges of the screen, and don’t look sideways too much or the effect begins to break.
Most importantly, BE RESPONSIBLE! This tech is for everyone and we ALL have the ability to do some really cool things. At the same time, this tech is moving faster than what people are ready for and we need to educate others on what is possible. Please let me know how you think I could improve my workflow or if I can help out with anything!