
A 367K-Parameter LLM Fits on a Floppy as Nordic Puts an NPU in Wi-Fi Chips
A DOS-era language model on a 486, an Arduino SBC pitched for agent memory, a Versal VLM paper, and Nordic's nRF71 NPU plus u-blox modules.
Category
LLMs and AI on microcontrollers and tiny hardware — news, bench logs, and reference guides
97 posts

A DOS-era language model on a 486, an Arduino SBC pitched for agent memory, a Versal VLM paper, and Nordic's nRF71 NPU plus u-blox modules.

Liquid AI's LFM3 puts a 1.7B model in 1.9 GB on an iPhone 17 Pro, a new paper targets phone NPUs, and a $7 ESP32 drives an iPhone.

llama.cpp merged an x86 VNNI Q2_0 kernel on November 15, a Jetson Thor box PC appeared, and an ESP32-S3 controller launched at $69.

Tether's 0.8B tool-calling model, Huawei's W4A4 quantizer, a 4.5M-parameter open TTS and a lattice-quantization experiment, all with the caveats left in.

A tiny LLM on an ESP32-S3, a vendor-claimed 3x YOLO speedup on a 12 g drone board, llama.cpp v1.0, and a $62 vision module that beats a CM4 on NPU work.

Edge0 open-sources a 145B model it says fits in 29GB on a laptop, Kora reports 25 tokens/sec on a Snapdragon NPU, and DeepX and Intel sign an OpenVINO deal.

A new MLIR compiler emits dependency-free C for a tiny LLM on a Cortex-M7, and a Cortex-M33 board runs an AI fan-fault detector on its own microphones.

An independent pruning pipeline shrinks a 26B mixture-of-experts model to 4.75GB while two new papers chase lossless low-bit inference.

LeRobot's new release lets you talk to a running robot while NVIDIA's VFlash research cuts VLA planning latency 80% on Jetson Thor.

A viral ESP32 bridge lets AI agents control a real iPhone, AMD's NPU now hot-swaps LoRA adapters, and an NPU briefly tries to render Windows.

An open-vocabulary speech model gets near real-time on an ESP32-S3, a new benchmark shows why MAC counts lie, and ST overhauls its STM32 AI toolchain.

A 3.8MB voice model, an early ChatML LLM, and a $21 board all land on the ESP32 this week.

A watch beta, a microcontroller voice SDK, a lightweight robot-vision model and a new ExecuTorch release all push inference further onto the device itself.

A new llama.cpp build spreads LFM3 over several Qualcomm NPUs, a sub-30MB model targets tiny automation, and Raspberry Pi prices climb again.

From a from-scratch LLM on a $25 microcontroller to an 8B MoE model living in under a gigabyte, the edge keeps shrinking its memory budget.

An open-source Android AI agent tops 2,100 GitHub stars in two days as a neuromorphic kit teaches a Raspberry Pi 5 to learn sounds on the fly.

A tiny world model plays games on a Pico 2 W, AMD ships an early NPU/GPU SDK, and a measurement study checks edge-NPU VLM hype.

A cheap ESP32 becomes an AI agent's eyes and hands on a real iPhone, while a tiny distilled LLM runs an $8 virtual fish tank offline.

An open-vocabulary speech engine, a full offline voice assistant, and a new ESP32-P4 compute module show how much now runs on one microcontroller.
Arm's SME2 doubles streaming-ASR speed on a Pixel 11, a battery-sipping voice stack lands on Android, and two cheap microcontrollers go local.

ML Drift goes open source, ExecuTorch grows up on Cortex-M, a 1.58-bit MoE lands on a 12GB GPU, and two new boards hit the bench.

A from-scratch tiny LLM, a sub-1-bit quantization trick, a MoE model on a phone NPU, and two new chips round out today's edge sweep.

NVIDIA's Nemotron Bolt talks to a Jetson-powered robot, Embedl speeds Jetson deployment, and two new boards push NPUs onto cheaper SBCs.

Intel ships speculative decoding for NPUs and a Google bridge to them, AMD and Intel merge CPU math libraries, and two new boards push edge compute further.

A $1 chip pushes an LLM toward 4B parameters, Google opens its Coral NPU compiler, and an ESP32 learns to hear offline in seven languages.

A 28.9M LLM, two phone/voice agents, a new AI-ready dev board and a drone video hack all land on the same $7 chip family.

Meta's 30B on-device agent, a 254M offline speech model, and a $10 ESP32 that hears without the cloud lead today's edge sweep.

Ollama, llama.cpp and PyTorch all shipped fresh runtime builds this week, and a 16-core SoC says it can match desktop AI boxes for less.

Apple's Neural Engine runs a 27B model at low bit-width, a distilled 14M LLM animates a $8 microcontroller pet, and new research questions KV-cache quant on Raspberry Pi.

A single-camera drone outflies champion pilots, a Jetson runtime slashes VLA latency, and Reachy Mini learns to remember people on a Pi.

AMD ships an official NPU SDK, hobbyists wake up an older Ryzen chip's idle silicon, and new hardware arrives already quoting on-device LLM speeds.
A microcontroller runs open-vocabulary speech recognition with no neural accelerator, while NPU boards and an $8 chip push tiny-LLM throughput further.

Two fresh ESP32 voice-AI repos go fully offline, a mystery ESP32-S31 chip shows up on a display board, and an ESP32 radio hack learns to transmit.

OpenBMB squeezes a 35B MoE vision model onto an iPhone, Google shrinks multimodal search to 191MB, and a $7 ESP32 learns to tap a real iPhone's screen.

llama.cpp opens a path to Qualcomm's Hexagon NPU, a 2-bit automation model surges on GitHub, and a lighter world model targets robot arms.

Samsung's sub-1-bit quantization paper gets a reality check, plus fresh research on tiny LLMs and ESP32 voice silicon.

A dual-core Arduino runs robot policies on-device, a paper argues for skill-sized models over one big brain, and two ESP32 boards ship.

Microsoft wires llama.cpp into Windows ML's NPU stack while an agent-built FPGA accelerator and fresh AI PC benchmarks show where edge inference actually stands.

A distilled LLM runs a fish tank on an $8 ESP32-S3, four Raspberry Pi 5s share a 30B model, and a 2B decision model lands for edge agents.

Two ESP32 voice projects lean on cloud AI while a maker's e-ink reader builds the hardware the next on-device model will need.

Meta ships ESP32 and Linux SDKs for its Muse AI, Qualcomm's next phones get 2nm NPUs, and a 68KB brain gives a cheap robot arm memory.

PyTorch declares its on-device AI runtime production-ready while llama.cpp adds a decision-model endpoint and two new NPU modules land for edge boards.

A BitNet cluster splits an LLM across seven ESP32-S3 boards, a 2-bit tool-calling model lands on GitHub, and a quantization paper gets llama.cpp 15x faster on an M4 Pro.

NVIDIA's Cosmos 3 Edge drives a robot arm on-device, a $5-class chip gets a voice it hasn't spoken yet, and two new boards widen the SBC lineup.

A 17MB runtime puts LLMs entirely on AMD's XDNA2 NPU, an old terminal gets an offline brain, and Intel's NPU gets unlocked by reverse engineering.

A tiny GPIO-control model and a Japanese TTS join the ESP32 pile-up while Raspberry Pi raises prices and a Jetson robot chases bubbles.

A full day inside the ESP32 world: chatty microcontroller LLMs, a $5-chip speech model, and two new boards from Espressif's own community.

From a Qualcomm earbud chip to an ESP32-S3 that hears, thinks and speaks with no cloud, edge AI keeps shrinking into pockets and ears.

Community builders push token throughput further this week — via SSD-streamed MoE experts, prompt-lookup drafting, and a wrapper for Apple's built-in on-device LLM.

Liquid AI ships a 2.6B agentic model for edge hardware, a 144M classifier skips text generation entirely, and a new paper shows byte-level LLMs can beat tokenized ones with sc

A tool-calling model flips switches on a Pi 5, a Qualcomm NPU speeds up a robot arm sevenfold, and a $2,500 biped joins LeRobot.

AMD open-sources fused FlashAttention kernels for XDNA NPUs, Qualcomm ships a Linux preview for Snapdragon X2, and fresh research pushes tiny and trillion-scale models toward

An RP2350 draws faces from noise, an ESP32-S3 writes stories, and AI smart glasses squeeze in a 1-bit model — all without a server.

Three ESP32 builds this week show the chip's range — a reactive desk companion, motion-sensing drumsticks, and a scripting OS — none of them running a language model.

Two new speech stacks and a tool-calling model push voice AI fully on-device, while Nordic and Zephyr quietly harden the hardware underneath.

Small llama.cpp builds keep adding Hexagon DSP ops, NVIDIA's TensorRT-Edge-LLM adds Day-0 model support, and a new leaderboard measures tokens per joule.

PrismML compresses a 27B model to 5.9GB, Intel's BITCOS beats the 1.585-bit ternary limit, and a 44M-parameter model claims exact arithmetic on a laptop CPU.

NVIDIA post-trains Cosmos 3 Edge for on-device manipulation, a PKU lab ships a llama.cpp engine for VLA policies, and Jetson's next Orin Nano gets a ship date.

GMKtec, ASUS and Radxa all shipped NPU hardware this week while OpenVINO and a Qualcomm robotics runtime pushed what those chips can actually run.

NVIDIA posts a 6.4x MLPerf edge win on Jetson AGX Thor, a dense TinyStories model skips the flash trick, and a paper splits VLA robots between cloud and a tiny local model.

A one-chip LLM, a Wi-Fi upgrade to Seeed's tiny displays, and Tuya's push to make ESP32 an AI-agent target, not just a Wi-Fi one.

New wearable and phone releases push transcription, gesture control and silent speech fully on-device, while ESP32 and Jetson tooling keeps pace.

llama.cpp shipped two builds in two days, ExecuTorch hit 1.0 with new NPU backends, and LiteRT tuned fp16 kernels for mobile CPUs.

A ternary 8B model, a BitNet toy for TinyStories, a fish tank run by a 14M LLM, and a Rust retrieval encoder push quantization research toward real hardware.

A student-built quadruped, a walking-robot policy on a Rockchip SBC, a tiny STM32N6 vision camera, and a $49 TinyML kit all landed this week.
A phone NPU claims 30B MoE inference, a 35B model streams from storage on a Mac, and an XDNA1 NPU gets a Linux bring-up.

An RP2350 chip runs a diffusion model, an ESP32-S3 speaks Japanese, and researchers tackle mobile power and tiny-drone control at the edge.

A modded Sony PSP runs a tiny LLM, a $3 chip learns to speak, and a 2B open model claims agentic skills for phones.

A GitHub toolkit, a 100MB cloning TTS, a Pi-powered dashcam agent, and a phone-to-watch AI rollout — all inference staying on the device.

A new llama.cpp release, an ExecuTorch-powered 30B agent model, a cheap RK3576 vision board, and a DIY Jetson robot dog mark a busy week for edge toolchains.

Fresh arXiv work rethinks quantization strategy and sustainability, an Apple-adjacent paper shrinks the dictation encoder, and new silicon and Jetson guidance round out the ed

A llama.cpp-style engine ports VLA robot policies to Jetson, NVIDIA doubles entry-level robotics compute, and a $10 chip proves the floor of local AI.

Three chipmakers and a mini-PC builder all attack the same problem this week: getting AI inference closer to memory, not just closer to silicon.

Fresh arXiv work tackles MCU vision drift and phone LLM memory pressure, while a solar bird feeder and a desktop WALL-E show the hobbyist edge staying busy.
DeepSeek V4 Flash gains on-device vision on a Mac, a $1 chip draws pictures, and Qualcomm ships new edge silicon ahead of IFA.

An iFLYTEK spin-off open-sources a 1.7B model claiming native million-token context on-device, while a 14MB tool-caller and an open voice-agent LLM push the small-model race f

llama.cpp adds multi-NPU Hexagon support, ONNX Runtime brings quantized KV caches to the browser, and a solo Rust engine beats llama.cpp on tiny models.

A 1-bit Qwen derivative fits an iPhone, a healing trick beats its own teacher at 4-bit, and a dense LLM limps along on an $8 chip.

A Rockchip-powered duck robot, two offline Raspberry Pi builds, and audits exposing quantization's blind spots and mislabeled GGUF files.

Intel details a 17 TOPS NPU chip built for chiplets, and an industry trend piece argues edge inference is leaving the pilot stage — both light on independent proof so far.

IBM's Granite 4.2 targets edge devices with a 3B open model, while a new RISC-V AI pocket computer ships locked to its own OS fork.

NVIDIA doubles its entry robotics brain, Perplexity moves agents onto local GPUs, and Liquid AI ships a speedup and a benchmark suite for on-device models.

Hearing aids ship dedicated on-device AI chips, new AI glasses land, and real-phone benchmarks show why raw specs don't tell the whole story.

Meta ships an on-device agentic model, an MoE engine claims 753B on one GPU, and researchers find 10 CVEs in a local inference engine.

Liquid AI distills instead of just rounding for 4-bit LFM2.5 checkpoints, as fresh research flags what low-bit quantization costs in memory and multilingual accuracy.

Flipper Devices' pocket Linux box promises an LLM that runs offline and knows the device inside out. Rockchip's own numbers say what a 6 TOPS RK3576 really does — and the NPU driver isn't in the kernel Flipper chose.

A Jetson Thor robot policy, a GGUF port for VLA models, a 2.6B tool-calling LLM, and a new AMD robotics module — all inference, no data center.

AMD bumps its NPU mini PC to 192GB of unified memory, Qualcomm ships five agentic apps for Snapdragon X, and Korea rethinks its NPU strategy.

FastFlowLM's first stable release puts a robotics policy on Ryzen AI's NPU, while Korea ships a boxed NPU appliance and a Raspberry Pi learns to narrate what it sees.

A Korean telecom sells an all-in-one on-prem LLM box built on a domestic NPU, while ESP32 tinkerers keep shrinking what a model needs to run.

AMD folds a hobbyist NPU runtime into ROCm, Google shows Gemma driving a robot from a Raspberry Pi 5, and a 45M-parameter model books tool calls on a phone.

A Raspberry Pi 5 runs Gemma and vision models split across CPU and GPU, and a 45M-parameter model fits tool-calling into 28MB of RAM.

A new 'Barista' model and a closer look at Google's Per-Layer Embeddings trick show what running an LLM on a microcontroller can and can't do.

A 17MB runtime that runs LLMs on AMD's Ryzen AI NPUs — built by three academics, acquired by AMD on July 17, 2026, folded into ROCm in August. The NPU rung of the edge ladder just got real.

A $5 Pico 2 writes TinyStories. A $15 Pi Zero 2 W runs SmolLM2-135M. A $299 RISC-V board claims 30B. What AI really fits at every rung of the edge hardware ladder.

On August 5, 2026 a 180.9M-parameter mixture-of-experts LLM ran on a $6-10 ESP32-P4 — and got two Hacker News points. The undercovered microcontroller AI story of the year.

In July 2026 a 28.9-million-parameter LLM ran fully on-device on an $8 ESP32-S3 at almost 10 tokens per second. Here's how the trick works, who built it, what's hype, and what you can actually build with a microcontroller LLM today.