Locally Hosted Chatbots with RWKV through ChatRWKV and the Text-Generation-WebUI | 14B Model on 3GB!

Channel:
Canada NanoNomad
Subscribers:
2,340
Published on ● Video Link: https://www.youtube.com/watch?v=UeAD1qWNb1U


Duration: 12:53
1,805 views
44

A quick look at installing RWKV, ChatRWKV the Text-Generation-WebUI and the Long Term Memory extension for chatbots. This is a great LLM based around an RNN rather than a GPT model. Very impressive output on many of the released models. The WebUI has developed fast, and now supports features like TavernAI cards, and with the long term memory plugin, should make for some interesting chat bots.

Configure swap space for low RAM (less than 32gb) users: https://youtu.be/UeAD1qWNb1U?t=170

Install ChatRWKV:
https://youtu.be/UeAD1qWNb1U?t=229

Models:
https://youtu.be/UeAD1qWNb1U?t=408

Model loading strategies:
https://youtu.be/UeAD1qWNb1U?t=453

Editing the ChatRWKV script chat.py:
https://youtu.be/UeAD1qWNb1U?t=481

Installing Text-Generation-WebUI:
https://youtu.be/UeAD1qWNb1U?t=596

Installing Long Term Memory extension:
https://youtu.be/UeAD1qWNb1U?t=630

Models:
https://huggingface.co/BlinkDL/

RWKV pip package: https://pypi.org/project/rwkv/ (please always check for latest version and upgrade). Strategies found here.

ChatRWKV:
conda create -n rwkv
conda activate rwkv
conda install -c conda-forge cudatoolkit=11.7 cudnn
git clone https://github.com/BlinkDL/ChatRWKV.git
cd ChatRWKV
pip install rwkv
pip install ninja
pip install -r requirements.txt
pip install torch==2.0.0+cu117 torchvision==0.15.1+cu117 --extra-index-url https://download.pytorch.org/whl/cu117
conda install -c conda-forge cudatoolkit=11.7 cudnn
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

Text-Generation-WebUI:
git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
pip install -r requirements.txt

Long Term Memory extension:
cd text-generation-webui
git clone https://github.com/wawawario2/long_term_memory extensions/long_term_memory
pip install -r extensions/long_term_memory/requirements.txt
python -m pytest -v extensions/long_term_memory/

Example loading Text-Generation-WebUI:
python server.py --rwkv-strategy "cuda fp16i8 -[replace this with an angled bracket - you cant use them in youtube descriptions] cpu fp32 *12" --rwkv-cuda-on --extensions long_term_memory



Other Videos By NanoNomad


2023-05-22Tortoise TTS Fine Tuning Wrap-Up
2023-05-16Tortoise TTS DEMO: G-Man performs Gilbert and Sullivan's 'The Major-General's Song'
2023-05-15Train Tortoise TTS in English, Spanish, French, Italian, Portuguese, German, and more? Maybe?
2023-05-10DEMO: Testing French-Speaking Tortoise TTS
2023-05-10DEMO: Testing German-Speaking Tortoise TTS
2023-05-08DEMO: Testing Spanish Speaking Tortoise TTS
2023-05-07DEMO: Testing Tortoise TTS Speaking in Portuguese
2023-05-04Make Using Tortoise TTS Faster with Fine-Tuned Models
2023-05-01AI Voice Swap and Lip Sync using Wav2Lip-HQ-Updated
2023-04-22Voice Cloning with Tortoise TTS and Model Training Using the AI Voice Cloning WebUI
2023-04-07Locally Hosted Chatbots with RWKV through ChatRWKV and the Text-Generation-WebUI | 14B Model on 3GB!
2023-03-29Create Datasets for Voice Model Training on Google Colab | Updated Tools for Coqui TTS Training
2023-03-22Train a VITS Speech Model using Coqui TTS | Updated Script and Audio Processing Tools
2023-03-15Training or Fine Tuning a Hindi Language VITS TTS Voice Model with Coqui TTS on Google Colab
2023-03-05Install and Configure Retroarch for PS Vita with Thumbnails, Overlays and Shaders
2023-03-03Fallout 1 on the PS Vita is the Best Way to Play
2023-02-24Train or Fine Tune VITS on (theoretically) Any Language | Train Multi-Speaker Model | Train YourTTS
2023-02-12Even more Voice Cloning | Train a Multi-Speaker VITS model using Google Colab and a Custom Dataset
2023-02-04Updated | Near-Automated Voice Cloning | Whisper STT + Coqui TTS | Fine Tune a VITS Model on Colab
2023-01-30YourTTS Training Discussion | Experiences, Multistage Training, Demos, Prior Training Preservation
2023-01-27Updated | Fine-Tuning YourTTS with Automated STT Datasets on Google Colab for AI Voice Cloning


Tags:
LLM
AI
RWKV
chatbot
chatgpt
chatrwkv