Edge Dispatch

ESP32 Special: Small LLMs Learn to Chat, Listen and Keep the Fish Alive

A full day inside the ESP32 world: chatty microcontroller LLMs, a $5-chip speech model, and two new boards from Espressif's own community.

Sep 30, 202624

Wearable AI Chips Land as an ESP32 Board Learns to Run a Full Offline Voice Loop

From a Qualcomm earbud chip to an ESP32-S3 that hears, thinks and speaks with no cloud, edge AI keeps shrinking into pockets and ears.

Sep 29, 202645

A $300 GPU Streams a 177B AI Model From an SSD While llama.cpp Learns to Skip Ahead

Community builders push token throughput further this week — via SSD-streamed MoE experts, prompt-lookup drafting, and a wrapper for Apple's built-in on-device LLM.

Sep 28, 202656

A Classifier That Only Picks, a Model That Skips Tokens, and an AI Firm Bets 2.6B Parameters Can Do Agent Work

Liquid AI ships a 2.6B agentic model for edge hardware, a 144M classifier skips text generation entirely, and a new paper shows byte-level LLMs can beat tokenized ones with sc

Sep 27, 202662

Needle Threads a Raspberry Pi, an NPU Learns to Move a Robot Arm Fast, and a Biped Joins the LLM Toolkit

A tool-calling model flips switches on a Pi 5, a Qualcomm NPU speeds up a robot arm sevenfold, and a $2,500 biped joins LeRobot.

Sep 26, 2026118

NPUs Learn to Fuse: AMD Opens Its XDNA Compiler, Qualcomm Previews Linux, and Two New On-Device AI Models Land

AMD open-sources fused FlashAttention kernels for XDNA NPUs, Qualcomm ships a Linux preview for Snapdragon X2, and fresh research pushes tiny and trillion-scale models toward

Sep 25, 202689

A Diffusion Model and a 28.9M LLM Both Now Run on Bare Microcontrollers

An RP2350 draws faces from noise, an ESP32-S3 writes stories, and AI smart glasses squeeze in a 1-bit model — all without a server.

Sep 24, 2026169

The ESP32 Beat Skips the LLM Today: A Desk Companion, Drumsticks, and a JavaScript OS

Three ESP32 builds this week show the chip's range — a reactive desk companion, motion-sensing drumsticks, and a scripting OS — none of them running a language model.

Sep 23, 202690

Tiny Audio AI Models Learn to Whisper Without Ever Calling Home

Two new speech stacks and a tool-calling model push voice AI fully on-device, while Nordic and Zephyr quietly harden the hardware underneath.

Sep 22, 202665

llama.cpp's Nightly Grind Teaches Phone Chips New AI Tricks While NVIDIA Ships a Rival Edge Runtime

Small llama.cpp builds keep adding Hexagon DSP ops, NVIDIA's TensorRT-Edge-LLM adds Day-0 model support, and a new leaderboard measures tokens per joule.

Sep 21, 202659

A 27B AI Model Shrinks to 5.9GB as Ternary Quantization Keeps Pushing the Floor Down

PrismML compresses a 27B model to 5.9GB, Intel's BITCOS beats the 1.585-bit ternary limit, and a 44M-parameter model claims exact arithmetic on a laptop CPU.

Sep 20, 202675

A 4B AI Model Drives a Robot Arm on Jetson Thor, No Datacenter in the Loop

NVIDIA post-trains Cosmos 3 Edge for on-device manipulation, a PKU lab ships a llama.cpp engine for VLA policies, and Jetson's next Orin Nano gets a ship date.

Sep 19, 202668

AI PCs Go Big: A 300B-Parameter Desktop, an 80-TOPS Mini PC, and Edge NPUs Redraw the Local-Inference Map

GMKtec, ASUS and Radxa all shipped NPU hardware this week while OpenVINO and a Qualcomm robotics runtime pushed what those chips can actually run.

Sep 18, 202690

Jetson Thor Sprints Past llama.cpp While a 15M-Parameter LLM Still Fits an $8 Chip

NVIDIA posts a 6.4x MLPerf edge win on Jetson AGX Thor, a dense TinyStories model skips the flash trick, and a paper splits VLA robots between cloud and a tiny local model.

Sep 17, 2026127

ESP32 Special: An $8 Chip Runs a 29-Million-Parameter LLM, and Vendors Rethink the Board Around It

A one-chip LLM, a Wi-Fi upgrade to Seeed's tiny displays, and Tuya's push to make ESP32 an AI-agent target, not just a Wi-Fi one.

Sep 16, 202679

A Wristband Reads Muscles, a Ring Wants Your Ideas: Edge AI Moves Onto the Body

New wearable and phone releases push transcription, gesture control and silent speech fully on-device, while ESP32 and Jetson tooling keeps pace.

Sep 15, 202675

Runtimes on the Move: llama.cpp, ExecuTorch and LiteRT All Update as Edge AI's Software Layer Speeds Up

llama.cpp shipped two builds in two days, ExecuTorch hit 1.0 with new NPU backends, and LiteRT tuned fp16 kernels for mobile CPUs.

Sep 14, 202684

Ternary Weights and Tiny Tanks: Small-Model Quantization Gets Concrete for Edge AI

A ternary 8B model, a BitNet toy for TinyStories, a fish tank run by a 14M LLM, and a Rust retrieval encoder push quantization research toward real hardware.

Sep 13, 202661

Split-Brain Robots: Jetson Handles the Vision, a $49 MCU Kit Handles the Milliseconds — Edge AI Keeps Dividing the Job

A student-built quadruped, a walking-robot policy on a Rockchip SBC, a tiny STM32N6 vision camera, and a $49 TinyML kit all landed this week.

Sep 12, 202669

Qualcomm Chases 30B AI Models on a Phone as Edge Silicon Keeps Multiplying

A phone NPU claims 30B MoE inference, a 35B model streams from storage on a Mac, and an XDNA1 NPU gets a Linux bring-up.

Sep 11, 2026107

A $1 Microcontroller Learns to Dream Faces as On-Device AI Keeps Shrinking

An RP2350 chip runs a diffusion model, an ESP32-S3 speaks Japanese, and researchers tackle mobile power and tiny-drone control at the edge.

Sep 10, 202665

A 90-Million-Parameter LLM Talks From a 2004 PSP, While On-Device AI Keeps Finding Smaller Homes

A modded Sony PSP runs a tiny LLM, a $3 chip learns to speak, and a 2B open model claims agentic skills for phones.

Sep 9, 202674

One Toolkit, One Chip, One Watch: On-Device AI Keeps Colonizing the Phone Layer

A GitHub toolkit, a 100MB cloning TTS, a Pi-powered dashcam agent, and a phone-to-watch AI rollout — all inference staying on the device.

Sep 8, 202677

Runtimes Race Ahead: llama.cpp 0.4.0 and a 30B On-Device AI Agent Test the Edge's Limits

A new llama.cpp release, an ExecuTorch-powered 30B agent model, a cheap RK3576 vision board, and a DIY Jetson robot dog mark a busy week for edge toolchains.

Sep 7, 202681

Quantization Papers Pile Up as Researchers Argue Over Where Small AI Should Spend Its Bits

Fresh arXiv work rethinks quantization strategy and sustainability, an Apple-adjacent paper shrinks the dictation encoder, and new silicon and Jetson guidance round out the ed

Sep 6, 2026109

Robot Policies Start Running Like Local AI Chatbots — Straight on the Device

A llama.cpp-style engine ports VLA robot policies to Jetson, NVIDIA doubles entry-level robotics compute, and a $10 chip proves the floor of local AI.

Sep 5, 202688

AMD, Qualcomm and NVIDIA All Chase the Same Local-AI Bottleneck: Memory

Three chipmakers and a mini-PC builder all attack the same problem this week: getting AI inference closer to memory, not just closer to silicon.

Sep 4, 2026128

A Microcontroller Learns to Retrain Itself as On-Device AI Keeps Multiplying Tasks

Fresh arXiv work tackles MCU vision drift and phone LLM memory pressure, while a solar bird feeder and a desktop WALL-E show the hobbyist edge staying busy.

Sep 3, 202665

A Solo Coder Brings Vision to a Local LLM as Edge Silicon and TinyML Builders Push Back the Frontier

DeepSeek V4 Flash gains on-device vision on a Mac, a $1 chip draws pictures, and Qualcomm ships new edge silicon ahead of IFA.

Sep 2, 202694

A Million-Token LLM Tries to Fit On Your Phone as Tiny Voice Models Multiply

An iFLYTEK spin-off open-sources a 1.7B model claiming native million-token context on-device, while a 14MB tool-caller and an open voice-agent LLM push the small-model race f

Sep 1, 2026202

llama.cpp Teaches an NPU to Share the Load as Small AI Engines Keep Multiplying

llama.cpp adds multi-NPU Hexagon support, ONNX Runtime brings quantized KV caches to the browser, and a solo Rust engine beats llama.cpp on tiny models.

Aug 31, 2026101

A 27B LLM Shrinks to Phone Size as Edge Quantization Keeps Rewriting the Rules

A 1-bit Qwen derivative fits an iPhone, a healing trick beats its own teacher at 4-bit, and a dense LLM limps along on an $8 chip.

Aug 30, 202668

A Duck Robot Runs Its Balance Loop On-Chip as Local AI Quantization Claims Get Audited

A Rockchip-powered duck robot, two offline Raspberry Pi builds, and audits exposing quantization's blind spots and mislabeled GGUF files.

Aug 29, 202693

Edge Dispatch: Intel's Wildcat Lake Brings a Modest NPU to AI PCs, While Edge AI Claims It's Going Mainstream in IoT

Intel details a 17 TOPS NPU chip built for chiplets, and an industry trend piece argues edge inference is leaving the pilot stage — both light on independent proof so far.

Aug 28, 202699

Edge Dispatch: IBM Ships an Edge-First Granite LLM as RISC-V AI Boards Keep Fragmenting the Toolchain

IBM's Granite 4.2 targets edge devices with a 3B open model, while a new RISC-V AI pocket computer ships locked to its own OS fork.

Aug 27, 202688

Edge Dispatch: Jetson Orin Nano 2 Doubles Edge Robotics AI, While Local LLM Agents Get a Bigger Home on the Desktop

NVIDIA doubles its entry robotics brain, Perplexity moves agents onto local GPUs, and Liquid AI ships a speedup and a benchmark suite for on-device models.

Aug 26, 202699

Edge Dispatch: Hearing Aids Get Their Own AI Chips as On-Device Voice and Vision Push Past the Phone

Hearing aids ship dedicated on-device AI chips, new AI glasses land, and real-phone benchmarks show why raw specs don't tell the whole story.

Aug 25, 2026108

Edge Dispatch: Meta's Muse Glimmer Bets Big on On-Device Agentic AI as the Runtime Wars Keep Multiplying

Meta ships an on-device agentic model, an MoE engine claims 753B on one GPU, and researchers find 10 CVEs in a local inference engine.

Aug 24, 2026245

Edge Dispatch: Liquid AI Trains Its Way Around the Q4_0 Quality Tax, While New Papers Chip Away at What Quantization Quietly Breaks

Liquid AI distills instead of just rounding for 4-bit LFM2.5 checkpoints, as fresh research flags what low-bit quantization costs in memory and multilingual accuracy.

Aug 23, 2026111

Edge Dispatch: Edge Dispatch: NVIDIA Squeezes a 4B World Model Onto the Robot Itself as Liquid AI and AMD Chase the Same Edge

A Jetson Thor robot policy, a GGUF port for VLA models, a 2.6B tool-calling LLM, and a new AMD robotics module — all inference, no data center.

Aug 22, 2026123

Edge Dispatch: AMD's Ryzen AI Halo Jumps to 192GB While Qualcomm Pushes On-Device AI Agents

AMD bumps its NPU mini PC to 192GB of unified memory, Qualcomm ships five agentic apps for Snapdragon X, and Korea rethinks its NPU strategy.

Aug 21, 2026158

Edge Dispatch: A Robot Arm Joins the NPU Party: FastFlowLM Adds a Vision-Language-Action Model to Ryzen AI

FastFlowLM's first stable release puts a robotics policy on Ryzen AI's NPU, while Korea ships a boxed NPU appliance and a Raspberry Pi learns to narrate what it sees.

Aug 20, 2026100

Edge Dispatch: Korea's KT Ships a Boxed NPU LLM Station While the ESP32 Crowd Trims Memory Further

A Korean telecom sells an all-in-one on-prem LLM box built on a domestic NPU, while ESP32 tinkerers keep shrinking what a model needs to run.

Aug 19, 2026116

Edge Dispatch: Ryzen AI's NPU Runtime Goes Official While a Raspberry Pi Learns to See and Speak with a Tiny LLM

AMD folds a hobbyist NPU runtime into ROCm, Google shows Gemma driving a robot from a Raspberry Pi 5, and a 45M-parameter model books tool calls on a phone.

Aug 18, 2026108

Edge Dispatch: Raspberry Pi's GPU Joins the AI Party While a 14MB Model Learns to Call Tools

A Raspberry Pi 5 runs Gemma and vision models split across CPU and GPU, and a 45M-parameter model fits tool-calling into 28MB of RAM.

Aug 17, 202692

Edge Dispatch: The $8 AI Chip Grows a Coffee Habit: ESP32 Tiny-LLM Trick Gets a Second Model

A new 'Barista' model and a closer look at Google's Per-Layer Embeddings trick show what running an LLM on a microcontroller can and can't do.

Aug 17, 2026100