How To Install One Click, Pre-configured Hugging Face (HUGS) AI Models On DigitalOcean GPU Droplets
9In this video you will learn how to install a Hugging Face image on a DigitalOcean GPU droplet with one of the available open-source models. DigitalOceans GPU droplets allows for the deployment of Hugging Face Generative AI Services also known as HUGS which has an extensive library of pre-trained models and their Text Generation Inference (TGI) solution. This enables seamless integration of machine learning (ML) Large Language Models (LLMs) with DigitalOcean GPU droplets. You will also learn how to create an SSH public and private key pair using PuTTYgen (PuTTY Key Generator) to log into your GPU droplet using PuTTY which is a SSH client. Once logged in via SSH you will immediately be greeted with the Bearer Token unique to you, which will allow you to run a TGI. Finally, we will run a TGI POST request query using the Messages API of the model you chose when making your GPU droplet. This will give you a text response with an answer to your question in the Ubuntu Linux command line terminal window.
🟢Free $200 DigitalOcean GPU cloud credits using my referral link: hhttps://digitalocean.pxf.io/c/1245219/1373759/15890
How To Get $200 In Free DigitalOcean Cloud Credits • How To Get $200 In Free DigitalOcean ...
How To Install PuTTY on Windows 11 And Windows 10 • How To Install PuTTY On Windows 10 Or...
AI 1-Click Models hhttps://digitalocean.pxf.io/ai-modelsDownload PuTTY hhttps://www.putty.org/
Steps To Install Hugs Ai Models On DigitalOcean GPU Droplet:
1. Go to hhttps://digitalocean.pxf.io/c/1245219/1373759/15890and create a free DigitalOcean account. The above link is my referral link granting you $200 in free cloud credit for 60 days as a new user
2. Once you have created your DigitalOcean account, you will be taken to your DigitalOcean project dashboard. Click Create.
3. Click GPU Droplets
4. Customise your GPU droplet to your liking ensuring to select 1-click models and select one of the Hugging Face images from the following list :
🔸 meta-llama/Llama
🔸 Qwen/Qwen
🔸 mistralai/Mixtral
🔸 mistralai/Mistral
🔸 NousResearch/Hermes
🔸 NousResearch/Nous
🔸 google/gemma
Note: The above Ai model list is what's currently available for selection as of the time of publishing of this video.
5. Generate your SSH public and private key pair and add your public key to DigitalOcean and select it for use with your droplet
6. Find your chosen AI models TGI command from hhttps://digitalocean.pxf.io/ai-modelsand ensure to select the one for either Single GPU or Multi GPU. Paste this command into a text document
7. Login to your GPU droplet with your private SSH key and copy and paste the bearer token displayed into the necessary location in the command in step 6 above. Make the other necessary specific edits to the TGI command such as your GPU droplets IP, model, and query. Copy the command to your clipboard
8. Check your NVIDIA GPU specs before proceeding with the TGI command by typing in the below command to list your GPU(s) and minimal spec info:
nvidia-smi -L
9. Finally, run the inference query below all your required edits:
curl -v -X 'POST' \
'http://your_droplet_ip/v1/chat/completions' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer 'your_token_here'' \
-d '{
"messages": [{"role":"user", "content":"When was the first iPhone released?"}],
"max_tokens": 64,
"stream": false
}'
Congratulations, you have successfully created a DigitalOcean GPU Droplet with a pre-configured Hugging Face AI model and ran your first text based inference via the Ubuntu Linux command line terminal.
Timestamps:
0:00 - Intro & Context
1:12 - Create DigitalOcean GPU Droplet (Cloud Server)
2:06 - Choose A Hugging Face 1 Click Model Image
3:12 - Generate SSH Private And Public Key Pair For Authentication
7:41 - Grab Text Inference Command For Hugs AI 1-Click Model
9:45 - Log Into Your GPU Droplet Using Your Private Key
11:18 - Check GPU Specs - nvidia-smi -L
11:53 - Run A Hugs Text Generation Inference (TGI) Command Query
13:03 - Closing & Outro
Grow your YouTube Channel with vidIQ hhttps://vidiq.com/70s
More DigitalOcean Server Guides (Playlist) • DigitalOcean Cloud Computing Guides
Ubuntu Linux Tutorials (Playlist) • Ubuntu Linux Tutorials
Artificial Intelligence (AI), Machine Learning (ML), And Large Language Models (LLMs) Tutorials (Playlist) • Artificial Intelligence (AI), Machine...
NVIDIA Graphics Card (GPU) Guides (Playlist) • NVIDIA Graphics Card (GPU) Guides
Social Media
SUBSCRIBE / @websplaining
Instagram (Thumbnail Collection) https://www.instagram.com/websplainingX (For Newest uploads) hhttps://x.com/websplainingReddit (Discussions) https://www.reddit.com/r/WebsplainingPersonal Website hhttps://www.websplaining.com/
#HuggingFace #DigitalOcean #GPU
DONT FORGET TO LIKE, COMMENT, AND SUBSCRIBE!