Training SDXL to Generate Text Using IA3 LoRA | It's like Kai's Power Tools, I Guess?

Channel:
Canada NanoNomad
Subscribers:
2,810
Published on ● Video Link: https://www.youtube.com/watch?v=aJuUDFirex0


Duration: 15:25
470 views
23

Can you reliably generate text with Stable Diffusion XL?
Maybe? Let's explore this "new" method. I try training a SDXL IA3 LoRA using Prodigy to guide text generation while trying to minimize the number of parameters disrupted by training.

[00:00] Intro/topics
[00:28] Image generation flaws/difficulties
[00:36] Other text generation methods
[01:05] LoRA types VS Dreambooth, and my limitations
[01:40] SD1.5 VS SDXL
[02:00] Briefly discussing the project
[03:20] Going over the dataset
[05:50] Python script for generating PNGs from TTF files
[08:16] Dataset pruning
]09:11] Test run 1
[11:13] Test run 2
[12:23] Test run 3
[12:30] Summary
[14:00] Using the IA3 LoRA in the Auto1111 WebUI with some before/after comparisons

Test IA3 Text-Image Adapters:
https://huggingface.co/AOLCDROM/iA3-adapters

More about the Prodigy optimizer:
https://github.com/konstmish/prodigy

More about the IA3 PEFT adapter:
https://huggingface.co/docs/peft/conceptual_guides/ia3

T-shirts dataset:
https://www.kaggle.com/datasets/sunnykusawa/tshirts

More on Pangrams:
https://www.prdaily.com/16-clever-pangrams-for-word-lovers/

Font to Image Python Script:
http://nanonomad.com/2024/05/07/training-sdxl-to-generate-text-using-ia3-lora/



Other Videos By NanoNomad


2024-06-26Fine Tuning XTTS v2 with forked Coqui | Coqui AI is dead; Long live Coqui!
2024-06-202x Faster LLM Training on Windows | LLaMA-Factory with Unsloth and Flash Attention 2
2024-06-1564kb Scene Demo/Intro/Cracktro Multimedia Mix #1 (90 min) | Flash/Photo-sensitivity Warning
2024-06-10Stable Audio Open 1.0 | Open Source* Generative Audio and Fine Tuning*
2024-06-04Troubleshooting Sega Saturn Emulation with Retroarch for iOS/Apple
2024-05-29Play Windows 98 and MS-DOS Games on iPad/iOS/iPhone with DOSBox-Pure and Retroarch for FREE
2024-05-25The Lost Art of Optical Disc Repair | Fixing and Testing a PlayStation Disc
2024-05-22Retroarch iOS Updates | Improved Performance, MS-DOS Core, Doom and Touch Input
2024-05-17RetroArch for iPad and iPhone now on the App Store | Installation, Setup, Quick Performance Overview
2024-05-13Micca Speck 4K Media Player | Unboxing, Firmware Update, Setup, Demos, and Opinions
2024-05-06Training SDXL to Generate Text Using IA3 LoRA | It's like Kai's Power Tools, I Guess?
2024-04-17Replacing Faulty Asus Phoenix RTX 3060 GPU Cooler - It's Easy
2024-03-21Bark TTS, Seamless Translation, RVC, Music Generation and More with the TTS Generation WebUI
2024-02-14Train Better Stable Diffusion Models | Prep Datasets Using this Free "Magic" Image Tool
2024-02-12Emulate a Sound Blaster in real MS-DOS on Modern Hardware | Retro Gaming on "Current" PCs
2024-01-28How to Play Hundreds of Point-and-Click Adventures on iOS for FREE with ScummVM with NO SIDELOADING
2024-01-18Training LoRAs and GLoRAs for Stable Diffusion 1.5 and XL Using the New Prodigy Optimizer
2024-01-03Nick Rekieta - Role Model (Voice Parody. It's silly. It's a joke.)
2023-11-19Automated Image Captioning with LLMs - Recognize Anything, BLIP-2, and Kosmos-2
2023-10-27Fine-Tuning Mistral 7B using QLoRA and PEFT on Unstructured Scraped Text Data | Making it Evil?
2023-09-20Exploring XTTS v1 and Tools to make Better Audio Datasets (the lazy way)


Tags:
AI
Stable Diffusion
SDXL
LoRA
Text to Image