GitHub – slvDev/esp32-ai · GitHub
This is a 28.9 million parameter language model that generates text on an ESP32-S3 microcontroller. It runs on the chip itself, with nothing sent to a server, and it displays generated text at 9.88 tokens per second on a small screen wired to the chip. It fits because most of the model lives in flash instead of RAM, using Per-Layer Embeddings, an idea from Google’s Gemma 3n.
Thanks to Michael Stauffer