I Ran a Local LLM on the ESP32 – Here's What Happened

Channel:
United States Dave Bennett
Subscribers:
201,000
Published on ● Video Link: https://www.youtube.com/watch?v=E6E_KrfyWFQ


Duration: 0:00
16,413 views
698

In this video, I take on the challenge of running a Large Language Model (LLM) on the ESP32 microcontroller.

Using a model based on the tiny stories dataset, with just 260K parameters, I managed to get the LLM running locally on the ESP32-S3.

Github link:
https://github.com/DaveBben/esp32-llm


🔗 Other Useful Links:

ESP32 Board I used (affiliate link): https://amzn.to/3z1hvL1
Tinyllamas repo: https://huggingface.co/karpathy/tinyllamas
llama2.c repository by Andrej Karpathy: https://github.com/karpathy/llama2.c/tree/master



Other Videos By Dave Bennett


2025-04-29Using AI Agents to Analyze Your Medical Data with n8n + PostgreSQL + Ollama + Apple Health
2025-04-05I Built an AI That Waters House Plants
2025-02-09What happens when DeepSeek R1 (671B) runs on the Raspberry Pi 5?
2024-09-03I Ran a Local LLM on the ESP32 – Here's What Happened
2024-04-07You're missing out on building Rust Desktop Apps!
2024-03-23Building a gadget from the Huberman lab podcast
2023-11-22Finding AirTags using Python
2023-02-27Let's Build a ChatGPT Smart Speaker
2022-08-22Easy Monitor Mod
2022-08-11Stand Up Reminder - Easy Arduino Project
2022-08-04Force Sensitive Resistors - Pressure Sensing for your Arduino Projects
2022-04-19TikTok Smartwatch - Advanced Arduino Project
2022-01-02HTC EVO 4G (Revisited): When 4G was Cool
2021-09-14Install Minecraft on Chromebook - 2021
2021-09-14Install Discord on Chromebook
2021-09-14Gaming Emulation on Chromebook
2021-09-14Install Linux Apps on Chromebook
2021-06-18Run Windows 11 on Chromebook
2021-05-13Play Genshin Impact on Chromebook
2021-05-01Run Windows 10 on Chromebook - 2021
2021-03-20Play Portal on Android