A $25 ESP32 Trains Its Own Tiny LLM While an 8B MoE Model Squeezes Under 1GB of RAM

A From-Scratch LLM Lives Inside a $25 ESP32

A hobbyist blogging as NSCT trained a small language model from scratch and squeezed the whole thing — weights, tokenizer, inference loop — onto a $25 ESP32 board with 16 MB of flash. There is no cloud call anywhere in the stack: unplug the board from the wall, run it on battery, and it keeps generating short stories at roughly 2.5 words per second, according to the builder's write-up.

The honest caveat: 2.5 words per second is slow, and a model this small produces simple, repetitive prose rather than anything resembling a capable assistant. This is a demonstration of the full pipeline fitting on a microcontroller, not a usable chatbot.

What makes it worth noting is the "trained from scratch" part — most tiny-LLM-on-ESP32 projects quantize an existing checkpoint. Here the model, training data, and deployment target were designed together for a 16 MB flash budget, the kind of ground-up constraint-first approach TinyML keeps pushing toward.

An 8B MoE Model Tries to Fit in a Phone's Memory

Edge0, posted on Hugging Face, packages an 8B-parameter sparse mixture-of-experts model — built on inclusionAI's Ling 3.0 tiny base, 128 experts with 8 active per token — into native engines for iOS, macOS, Android and Windows. The trick is streaming unused experts off SSD and keeping only the active set in RAM, which the team says holds peak memory under 1 GiB while decoding at 23.9–25.3 tokens per second.

Those numbers are self-reported and were measured on a Mac mini M4 Pro, not yet on an actual phone, so real phone-class performance is unverified. The team also flags this as a preview: agentic tool use is weak, and quality after 4-bit quantization trails the full-precision base by about 2.8 points on average.

Still, the expert-offload-plus-prerouter design — predicting which experts to load one step ahead so loading overlaps computation — is a genuinely interesting answer to how a model with billions of parameters stays inside a phone's memory budget.

A Paper Proposes Rebuilding Phone Memory Itself for 200B-Class Agents

A new arXiv paper posted on October 29, 2026 proposes restructuring phone memory rather than just shrinking models further. "NORmal" architects 3D ferroelectric NOR flash, co-fabricated in the back-end-of-line, as main memory — aiming to let smartphones host 200B-parameter-class multi-agent LLM teams, where a large planner coordinates several smaller workers across a session.

This is architecture research, not shipped silicon. The paper models the approach rather than demonstrating it in a commercial phone, and getting ferroelectric NOR into production memory stacks is a multi-year proposition at best.

It matters because the memory wall, not raw compute, is usually the hard limit on how large a model can live on a phone. Proposals like this hint that the next jump in on-device model size may come from memory architecture rather than quantization tricks alone.

Toit Ships Its First Major Version in Four Years

Toit, the open-source programming language and runtime built for microcontrollers — primarily the ESP32 — shipped version 2.0.0 on October 31, 2026, its first major version bump in more than four years, according to one of the project's own maintainers.

The maintainer's post doesn't itemize every change, so treat the "major" framing as the team's own characterization rather than an independently verified feature list. What's concrete is the version number and the four-year cadence to reach it.

Toit occupies the same territory as MicroPython and CircuitPython — fast iteration on resource-constrained boards — and a stable 2.x line matters for anyone building the on-device inference and sensor pipelines this beat covers daily.

A 10.1-Inch Touchscreen Turns the Arduino UNO Q Into a Self-Contained AI Box

Waveshare's new UNO-Q-DSI-TOUCH-10.1A, reported on October 31, 2026, is a 10.1-inch touchscreen that plugs into the Arduino UNO Q's MIPI DSI port and bundles integrated speakers, audio jacks and camera connectors into one module.

It's an accessory, not new silicon, and the report relays Waveshare's own product listing rather than independent hands-on testing — treat the spec sheet as vendor-stated until reviews land.

The Arduino UNO Q already runs small local vision-language pipelines for tasks like anomaly detection; a display with this much built-in I/O makes it easier to build a self-contained on-device AI appliance without wiring up a separate screen, mic and camera.

Five small stories today: a from-scratch LLM on a $25 microcontroller, an 8B mixture-of-experts model learning to live in under a gigabyte, a memory-architecture proposal aimed at the phone's RAM wall, and two bits of tooling — a new major version of Toit and a touchscreen for the Arduino UNO Q. The pattern holds: the edge keeps getting more capable without getting bigger.