Skip to content

Tag

Edge Dispatch

92 posts

A 367K-Parameter LLM Fits on a Floppy as Nordic Puts an NPU in Wi-Fi Chips
Edge AI3 min read

A 367K-Parameter LLM Fits on a Floppy as Nordic Puts an NPU in Wi-Fi Chips

A DOS-era language model on a 486, an Arduino SBC pitched for agent memory, a Versal VLM paper, and Nordic's nRF71 NPU plus u-blox modules.

20 views
Read
A 1.7B Non-Transformer LLM Targets the iPhone as Mobile NPUs Get a GUI Agent Paper
Edge AI4 min read

A 1.7B Non-Transformer LLM Targets the iPhone as Mobile NPUs Get a GUI Agent Paper

Liquid AI's LFM3 puts a 1.7B model in 1.9 GB on an iPhone 17 Pro, a new paper targets phone NPUs, and a $7 ESP32 drives an iPhone.

106 views
Read
A 2-Bit LLM Kernel Triples CPU Speed in llama.cpp as Jetson Thor Gets an Industrial Box
Edge AI3 min read

A 2-Bit LLM Kernel Triples CPU Speed in llama.cpp as Jetson Thor Gets an Industrial Box

llama.cpp merged an x86 VNNI Q2_0 kernel on November 15, a Jetson Thor box PC appeared, and an ESP32-S3 controller launched at $69.

91 views
Read
A 0.8B Agent LLM, a 4.5M-Parameter Voice and Two 4-Bit Quantizers: Tiny AI Gets Specific
Edge AI4 min read

A 0.8B Agent LLM, a 4.5M-Parameter Voice and Two 4-Bit Quantizers: Tiny AI Gets Specific

Tether's 0.8B tool-calling model, Huawei's W4A4 quantizer, a 4.5M-parameter open TTS and a lattice-quantization experiment, all with the caveats left in.

123 views
Read
A 14M-Parameter LLM Tends a Fish Tank on an $8 ESP32 as Drone AI Claims a 3x Speedup
Edge AI4 min read

A 14M-Parameter LLM Tends a Fish Tank on an $8 ESP32 as Drone AI Claims a 3x Speedup

A tiny LLM on an ESP32-S3, a vendor-claimed 3x YOLO speedup on a 12 g drone board, llama.cpp v1.0, and a $62 vision module that beats a CM4 on NPU work.

112 views
Read
A 145B Open LLM Claims a Laptop Seat as Snapdragon's NPU Runs Gemma 4 26B
Edge AI4 min read

A 145B Open LLM Claims a Laptop Seat as Snapdragon's NPU Runs Gemma 4 26B

Edge0 open-sources a 145B model it says fits in 29GB on a laptop, Kora reports 25 tokens/sec on a Snapdragon NPU, and DeepX and Intel sign an OpenVINO deal.

87 views
Read
An 8M-Parameter LLM Gets Compiled to C for a Cortex-M7 as Tiny AI Listens to Fans
Edge AI4 min read

An 8M-Parameter LLM Gets Compiled to C for a Cortex-M7 as Tiny AI Listens to Fans

A new MLIR compiler emits dependency-free C for a tiny LLM on a Cortex-M7, and a Cortex-M33 board runs an AI fan-fault detector on its own microphones.

160 views
Read
A 26B MoE Model Squeezes Onto a 6GB GPU as Edge AI Quantization Races Ahead
Edge AI4 min read

A 26B MoE Model Squeezes Onto a 6GB GPU as Edge AI Quantization Races Ahead

An independent pruning pipeline shrinks a 26B mixture-of-experts model to 4.75GB while two new papers chase lossless low-bit inference.

559 views
Read
AI Robots Get a Voice Mid-Task as Nvidia's VFlash Cuts Planning Time by 80%
Edge AI4 min read

AI Robots Get a Voice Mid-Task as Nvidia's VFlash Cuts Planning Time by 80%

LeRobot's new release lets you talk to a running robot while NVIDIA's VFlash research cuts VLA planning latency 80% on Jetson Thor.

316 views
Read
A $7 ESP32 Gives AI Agents a Real iPhone While Ryzen AI's NPU Learns to Swap Brains
Edge AI4 min read

A $7 ESP32 Gives AI Agents a Real iPhone While Ryzen AI's NPU Learns to Swap Brains

A viral ESP32 bridge lets AI agents control a real iPhone, AMD's NPU now hot-swaps LoRA adapters, and an NPU briefly tries to render Windows.

116 views
Read
A $5 Microcontroller Finally Hears Full Sentences While AI Chips Hit a Memory Wall
Edge AI4 min read

A $5 Microcontroller Finally Hears Full Sentences While AI Chips Hit a Memory Wall

An open-vocabulary speech model gets near real-time on an ESP32-S3, a new benchmark shows why MAC counts lie, and ST overhauls its STM32 AI toolchain.

208 views
Read
An ESP32 Speaks, Chats, and Gets a New $21 Brain in Today's All-Microcontroller AI Special
Edge AI3 min read

An ESP32 Speaks, Chats, and Gets a New $21 Brain in Today's All-Microcontroller AI Special

A 3.8MB voice model, an early ChatML LLM, and a $21 board all land on the ESP32 this week.

2.6K views
Read
Apple Watch's On-Device AI Learns to Eavesdrop While a Tiny Korean VLM Teaches Robots to See
Edge AI5 min read

Apple Watch's On-Device AI Learns to Eavesdrop While a Tiny Korean VLM Teaches Robots to See

A watch beta, a microcontroller voice SDK, a lightweight robot-vision model and a new ExecuTorch release all push inference further onto the device itself.

289 views
Read
llama.cpp Splits Liquid AI's LFM3 Across Multiple NPUs While a 29MB Automation LLM Chases Microcontrollers
Edge AI4 min read

llama.cpp Splits Liquid AI's LFM3 Across Multiple NPUs While a 29MB Automation LLM Chases Microcontrollers

A new llama.cpp build spreads LFM3 over several Qualcomm NPUs, a sub-30MB model targets tiny automation, and Raspberry Pi prices climb again.

241 views
Read
A $25 ESP32 Trains Its Own Tiny LLM While an 8B MoE Model Squeezes Under 1GB of RAM
Edge AI4 min read

A $25 ESP32 Trains Its Own Tiny LLM While an 8B MoE Model Squeezes Under 1GB of RAM

From a from-scratch LLM on a $25 microcontroller to an 8B MoE model living in under a gigabyte, the edge keeps shrinking its memory budget.

361 views
Read
A Viral Android AI Agent Taps Its Own Phone While a Neuromorphic Chip Teaches a Raspberry Pi to Learn Live
Edge AI4 min read

A Viral Android AI Agent Taps Its Own Phone While a Neuromorphic Chip Teaches a Raspberry Pi to Learn Live

An open-source Android AI agent tops 2,100 GitHub stars in two days as a neuromorphic kit teaches a Raspberry Pi 5 to learn sounds on the fly.

257 views
Read
A Game Runs Entirely on a $6 Chip While AMD Tries to Skip ROCm for Local AI
Edge AI4 min read

A Game Runs Entirely on a $6 Chip While AMD Tries to Skip ROCm for Local AI

A tiny world model plays games on a Pico 2 W, AMD ships an early NPU/GPU SDK, and a measurement study checks edge-NPU VLM hype.

183 views
Read
A $7 ESP32 Gives an AI Agent Hands on a Real iPhone While a 14M LLM Keeps a Fish Alive
Edge AI4 min read

A $7 ESP32 Gives an AI Agent Hands on a Real iPhone While a 14M LLM Keeps a Fish Alive

A cheap ESP32 becomes an AI agent's eyes and hands on a real iPhone, while a tiny distilled LLM runs an $8 virtual fish tank offline.

329 views
Read
Three ESP32 Builds Show How Far On-Device AI Has Come in a Week
Edge AI3 min read

Three ESP32 Builds Show How Far On-Device AI Has Come in a Week

An open-vocabulary speech engine, a full offline voice assistant, and a new ESP32-P4 compute module show how much now runs on one microcontroller.

451 views
Read
A Pixel Phone Speeds Up On-Device Speech AI While a $2 Chip Learns to See
Edge AI5 min read

A Pixel Phone Speeds Up On-Device Speech AI While a $2 Chip Learns to See

Arm's SME2 doubles streaming-ASR speed on a Pixel 11, a battery-sipping voice stack lands on Android, and two cheap microcontrollers go local.

317 views
Read
Google Open-Sources Its On-Device AI Engine While a Ternary LLM Shrinks a 30B Model to 12GB
Edge AI4 min read

Google Open-Sources Its On-Device AI Engine While a Ternary LLM Shrinks a 30B Model to 12GB

ML Drift goes open source, ExecuTorch grows up on Cortex-M, a 1.58-bit MoE lands on a 12GB GPU, and two new boards hit the bench.

394 views
Read
A 384M LLM Learns to Call Tools From Scratch While Rockchip Chases 44 TOPS at the Edge
Edge AI4 min read

A 384M LLM Learns to Call Tools From Scratch While Rockchip Chases 44 TOPS at the Edge

A from-scratch tiny LLM, a sub-1-bit quantization trick, a MoE model on a phone NPU, and two new chips round out today's edge sweep.

317 views
Read
A 4GB NVIDIA Model Learns to Chat With Robots While Raspberry Pi Gets a Local-LLM NPU
Edge AI5 min read

A 4GB NVIDIA Model Learns to Chat With Robots While Raspberry Pi Gets a Local-LLM NPU

NVIDIA's Nemotron Bolt talks to a Jetson-powered robot, Embedl speeds Jetson deployment, and two new boards push NPUs onto cheaper SBCs.

181 views
Read
Intel's NPU Learns to Guess Ahead While AMD and Intel Merge Their CPU AI Playbooks
Edge AI4 min read

Intel's NPU Learns to Guess Ahead While AMD and Intel Merge Their CPU AI Playbooks

Intel ships speculative decoding for NPUs and a Google bridge to them, AMD and Intel merge CPU math libraries, and two new boards push edge compute further.

188 views
Read
A $1 Microcontroller Stretches Toward a 4B LLM While Google Opens Its NPU Blueprint
Edge AI4 min read

A $1 Microcontroller Stretches Toward a 4B LLM While Google Opens Its NPU Blueprint

A $1 chip pushes an LLM toward 4B parameters, Google opens its Coral NPU compiler, and an ESP32 learns to hear offline in seven languages.

224 views
Read
Five ESP32 Builds Push Local AI From a Dev Board to a Drone's Camera
Edge AI4 min read

Five ESP32 Builds Push Local AI From a Dev Board to a Drone's Camera

A 28.9M LLM, two phone/voice agents, a new AI-ready dev board and a drone video hack all land on the same $7 chip family.

661 views
Read
A 30B AI Agent Learns to Live on One GPU While a 254M Model Learns to Listen Offline
Edge AI5 min read

A 30B AI Agent Learns to Live on One GPU While a 254M Model Learns to Listen Offline

Meta's 30B on-device agent, a 254M offline speech model, and a $10 ESP32 that hears without the cloud lead today's edge sweep.

224 views
Read
Edge Runtimes Learn to Run LLMs at 2 Bits While a New Chip Takes Aim at DGX Spark
Edge AI4 min read

Edge Runtimes Learn to Run LLMs at 2 Bits While a New Chip Takes Aim at DGX Spark

Ollama, llama.cpp and PyTorch all shipped fresh runtime builds this week, and a 16-core SoC says it can match desktop AI boxes for less.

302 views
Read
A 27B AI Hits an iPhone's Neural Engine at 3 Bits While a 14M LLM Keeps a Fish Tank Alive on ESP32
Edge AI5 min read

A 27B AI Hits an iPhone's Neural Engine at 3 Bits While a 14M LLM Keeps a Fish Tank Alive on ESP32

Apple's Neural Engine runs a 27B model at low bit-width, a distilled 14M LLM animates a $8 microcontroller pet, and new research questions KV-cache quant on Raspberry Pi.

284 views
Read
An AI Racing Drone Beats Humans While a Jetson VLA Model Sprints 12x Faster
Edge AI4 min read

An AI Racing Drone Beats Humans While a Jetson VLA Model Sprints 12x Faster

A single-camera drone outflies champion pilots, a Jetson runtime slashes VLA latency, and Reachy Mini learns to remember people on a Pi.

253 views
Read
AMD Opens Its NPU Stack to AI Agents While Qualcomm Puts an LLM on Your Face
Edge AI4 min read

AMD Opens Its NPU Stack to AI Agents While Qualcomm Puts an LLM on Your Face

AMD ships an official NPU SDK, hobbyists wake up an older Ryzen chip's idle silicon, and new hardware arrives already quoting on-device LLM speeds.

300 views
Read
Edge AI Learns to Hear Without Any Dedicated AI Silicon At All
Edge AI4 min read

Edge AI Learns to Hear Without Any Dedicated AI Silicon At All

A microcontroller runs open-vocabulary speech recognition with no neural accelerator, while NPU boards and an $8 chip push tiny-LLM throughput further.

532 views
Read
ESP32 Special: Two Offline Voice-AI Builds Debut While a New Chip Powers a Display Board
Edge AI3 min read

ESP32 Special: Two Offline Voice-AI Builds Debut While a New Chip Powers a Display Board

Two fresh ESP32 voice-AI repos go fully offline, a mystery ESP32-S31 chip shows up on a display board, and an ESP32 radio hack learns to transmit.

542 views
Read
A 35B MoE Vision AI Learns to Fit a Phone While a $7 ESP32 Hijacks an iPhone
Edge AI4 min read

A 35B MoE Vision AI Learns to Fit a Phone While a $7 ESP32 Hijacks an iPhone

OpenBMB squeezes a 35B MoE vision model onto an iPhone, Google shrinks multimodal search to 191MB, and a $7 ESP32 learns to tap a real iPhone's screen.

224 views
Read
A Hexagon NPU Learns New AI Tricks While Edge Robots Get Lighter World Models
Edge AI4 min read

A Hexagon NPU Learns New AI Tricks While Edge Robots Get Lighter World Models

llama.cpp opens a path to Qualcomm's Hexagon NPU, a 2-bit automation model surges on GitHub, and a lighter world model targets robot arms.

405 views
Read
A 1-Bit AI Trick Shrinks a 70B Model to Fit on a $200 GPU
Edge AI4 min read

A 1-Bit AI Trick Shrinks a 70B Model to Fit on a $200 GPU

Samsung's sub-1-bit quantization paper gets a reality check, plus fresh research on tiny LLMs and ESP32 voice silicon.

277 views
Read
Small AI Models Steer Robots While ESP32 Boards Do the Grunt Work
Edge AI4 min read

Small AI Models Steer Robots While ESP32 Boards Do the Grunt Work

A dual-core Arduino runs robot policies on-device, a paper argues for skill-sized models over one big brain, and two ESP32 boards ship.

283 views
Read
Windows Learns GGUF as AI Agents Design Their Own Inference Chip
Edge AI5 min read

Windows Learns GGUF as AI Agents Design Their Own Inference Chip

Microsoft wires llama.cpp into Windows ML's NPU stack while an agent-built FPGA accelerator and fresh AI PC benchmarks show where edge inference actually stands.

343 views
Read
A 14M-Parameter LLM Keeps a Virtual Fish Tank Alive on an $8 Chip
Edge AI4 min read

A 14M-Parameter LLM Keeps a Virtual Fish Tank Alive on an $8 Chip

A distilled LLM runs a fish tank on an $8 ESP32-S3, four Raspberry Pi 5s share a 30B model, and a 2B decision model lands for edge agents.

266 views
Read
ESP32 Special: Cloud Voices and an E-Ink Reader, No New On-Device LLM Yet
Edge AI3 min read

ESP32 Special: Cloud Voices and an E-Ink Reader, No New On-Device LLM Yet

Two ESP32 voice projects lean on cloud AI while a maker's e-ink reader builds the hardware the next on-device model will need.

485 views
Read
Meta Opens Muse to ESP32 Makers as a Tiny LLM Learns to Remember a Hidden Object
Edge AI5 min read

Meta Opens Muse to ESP32 Makers as a Tiny LLM Learns to Remember a Hidden Object

Meta ships ESP32 and Linux SDKs for its Muse AI, Qualcomm's next phones get 2nm NPUs, and a 68KB brain gives a cheap robot arm memory.

292 views
Read
ExecuTorch Hits 1.0 as llama.cpp Learns to Make Instant AI Decisions
Edge AI4 min read

ExecuTorch Hits 1.0 as llama.cpp Learns to Make Instant AI Decisions

PyTorch declares its on-device AI runtime production-ready while llama.cpp adds a decision-model endpoint and two new NPU modules land for edge boards.

409 views
Read
Seven $5 Chips Learn to Share an LLM, One Bit at a Time
Edge AI5 min read

Seven $5 Chips Learn to Share an LLM, One Bit at a Time

A BitNet cluster splits an LLM across seven ESP32-S3 boards, a 2-bit tool-calling model lands on GitHub, and a quantization paper gets llama.cpp 15x faster on an M4 Pro.

462 views
Read
A 4B AI Model Learns to Run a Robot Arm Entirely on Jetson Thor
Edge AI5 min read

A 4B AI Model Learns to Run a Robot Arm Entirely on Jetson Thor

NVIDIA's Cosmos 3 Edge drives a robot arm on-device, a $5-class chip gets a voice it hasn't spoken yet, and two new boards widen the SBC lineup.

513 views
Read
An NPU-Only Runtime Lands for Ryzen AI as a Vintage Terminal Learns to Think Offline
Edge AI4 min read

An NPU-Only Runtime Lands for Ryzen AI as a Vintage Terminal Learns to Think Offline

A 17MB runtime puts LLMs entirely on AMD's XDNA2 NPU, an old terminal gets an offline brain, and Intel's NPU gets unlocked by reverse engineering.

325 views
Read
A 312K-Parameter LLM Learns to Flip Switches as Pi Prices Climb Again
Edge AI4 min read

A 312K-Parameter LLM Learns to Flip Switches as Pi Prices Climb Again

A tiny GPIO-control model and a Japanese TTS join the ESP32 pile-up while Raspberry Pi raises prices and a Jetson robot chases bubbles.

485 views
Read
ESP32 Special: Small LLMs Learn to Chat, Listen and Keep the Fish Alive
Edge AI4 min read

ESP32 Special: Small LLMs Learn to Chat, Listen and Keep the Fish Alive

A full day inside the ESP32 world: chatty microcontroller LLMs, a $5-chip speech model, and two new boards from Espressif's own community.

318 views
Read
Wearable AI Chips Land as an ESP32 Board Learns to Run a Full Offline Voice Loop
Edge AI4 min read

Wearable AI Chips Land as an ESP32 Board Learns to Run a Full Offline Voice Loop

From a Qualcomm earbud chip to an ESP32-S3 that hears, thinks and speaks with no cloud, edge AI keeps shrinking into pockets and ears.

423 views
Read
A $300 GPU Streams a 177B AI Model From an SSD While llama.cpp Learns to Skip Ahead
Edge AI4 min read

A $300 GPU Streams a 177B AI Model From an SSD While llama.cpp Learns to Skip Ahead

Community builders push token throughput further this week — via SSD-streamed MoE experts, prompt-lookup drafting, and a wrapper for Apple's built-in on-device LLM.

495 views
Read
A Classifier That Only Picks, a Model That Skips Tokens, and an AI Firm Bets 2.6B Parameters Can Do Agent Work
Edge AI4 min read

A Classifier That Only Picks, a Model That Skips Tokens, and an AI Firm Bets 2.6B Parameters Can Do Agent Work

Liquid AI ships a 2.6B agentic model for edge hardware, a 144M classifier skips text generation entirely, and a new paper shows byte-level LLMs can beat tokenized ones with sc

198 views
Read
Needle Threads a Raspberry Pi, an NPU Learns to Move a Robot Arm Fast, and a Biped Joins the LLM Toolkit
Edge AI4 min read

Needle Threads a Raspberry Pi, an NPU Learns to Move a Robot Arm Fast, and a Biped Joins the LLM Toolkit

A tool-calling model flips switches on a Pi 5, a Qualcomm NPU speeds up a robot arm sevenfold, and a $2,500 biped joins LeRobot.

343 views
Read
NPUs Learn to Fuse: AMD Opens Its XDNA Compiler, Qualcomm Previews Linux, and Two New On-Device AI Models Land
Edge AI5 min read

NPUs Learn to Fuse: AMD Opens Its XDNA Compiler, Qualcomm Previews Linux, and Two New On-Device AI Models Land

AMD open-sources fused FlashAttention kernels for XDNA NPUs, Qualcomm ships a Linux preview for Snapdragon X2, and fresh research pushes tiny and trillion-scale models toward

305 views
Read
A Diffusion Model and a 28.9M LLM Both Now Run on Bare Microcontrollers
Edge AI4 min read

A Diffusion Model and a 28.9M LLM Both Now Run on Bare Microcontrollers

An RP2350 draws faces from noise, an ESP32-S3 writes stories, and AI smart glasses squeeze in a 1-bit model — all without a server.

639 views
Read
The ESP32 Beat Skips the LLM Today: A Desk Companion, Drumsticks, and a JavaScript OS
Edge AI3 min read

The ESP32 Beat Skips the LLM Today: A Desk Companion, Drumsticks, and a JavaScript OS

Three ESP32 builds this week show the chip's range — a reactive desk companion, motion-sensing drumsticks, and a scripting OS — none of them running a language model.

291 views
Read
Tiny Audio AI Models Learn to Whisper Without Ever Calling Home
Edge AI4 min read

Tiny Audio AI Models Learn to Whisper Without Ever Calling Home

Two new speech stacks and a tool-calling model push voice AI fully on-device, while Nordic and Zephyr quietly harden the hardware underneath.

254 views
Read
llama.cpp's Nightly Grind Teaches Phone Chips New AI Tricks While NVIDIA Ships a Rival Edge Runtime
Edge AI4 min read

llama.cpp's Nightly Grind Teaches Phone Chips New AI Tricks While NVIDIA Ships a Rival Edge Runtime

Small llama.cpp builds keep adding Hexagon DSP ops, NVIDIA's TensorRT-Edge-LLM adds Day-0 model support, and a new leaderboard measures tokens per joule.

305 views
Read
A 27B AI Model Shrinks to 5.9GB as Ternary Quantization Keeps Pushing the Floor Down
Edge AI5 min read

A 27B AI Model Shrinks to 5.9GB as Ternary Quantization Keeps Pushing the Floor Down

PrismML compresses a 27B model to 5.9GB, Intel's BITCOS beats the 1.585-bit ternary limit, and a 44M-parameter model claims exact arithmetic on a laptop CPU.

306 views
Read
A 4B AI Model Drives a Robot Arm on Jetson Thor, No Datacenter in the Loop
Edge AI4 min read

A 4B AI Model Drives a Robot Arm on Jetson Thor, No Datacenter in the Loop

NVIDIA post-trains Cosmos 3 Edge for on-device manipulation, a PKU lab ships a llama.cpp engine for VLA policies, and Jetson's next Orin Nano gets a ship date.

317 views
Read
AI PCs Go Big: A 300B-Parameter Desktop, an 80-TOPS Mini PC, and Edge NPUs Redraw the Local-Inference Map
Edge AI4 min read

AI PCs Go Big: A 300B-Parameter Desktop, an 80-TOPS Mini PC, and Edge NPUs Redraw the Local-Inference Map

GMKtec, ASUS and Radxa all shipped NPU hardware this week while OpenVINO and a Qualcomm robotics runtime pushed what those chips can actually run.

356 views
Read
Jetson Thor Sprints Past llama.cpp While a 15M-Parameter LLM Still Fits an $8 Chip
Edge AI4 min read

Jetson Thor Sprints Past llama.cpp While a 15M-Parameter LLM Still Fits an $8 Chip

NVIDIA posts a 6.4x MLPerf edge win on Jetson AGX Thor, a dense TinyStories model skips the flash trick, and a paper splits VLA robots between cloud and a tiny local model.

463 views
Read
ESP32 Special: An $8 Chip Runs a 29-Million-Parameter LLM, and Vendors Rethink the Board Around It
Edge AI3 min read

ESP32 Special: An $8 Chip Runs a 29-Million-Parameter LLM, and Vendors Rethink the Board Around It

A one-chip LLM, a Wi-Fi upgrade to Seeed's tiny displays, and Tuya's push to make ESP32 an AI-agent target, not just a Wi-Fi one.

284 views
Read
A Wristband Reads Muscles, a Ring Wants Your Ideas: Edge AI Moves Onto the Body
Edge AI4 min read

A Wristband Reads Muscles, a Ring Wants Your Ideas: Edge AI Moves Onto the Body

New wearable and phone releases push transcription, gesture control and silent speech fully on-device, while ESP32 and Jetson tooling keeps pace.

251 views
Read
Runtimes on the Move: llama.cpp, ExecuTorch and LiteRT All Update as Edge AI's Software Layer Speeds Up
Edge AI4 min read

Runtimes on the Move: llama.cpp, ExecuTorch and LiteRT All Update as Edge AI's Software Layer Speeds Up

llama.cpp shipped two builds in two days, ExecuTorch hit 1.0 with new NPU backends, and LiteRT tuned fp16 kernels for mobile CPUs.

336 views
Read
Ternary Weights and Tiny Tanks: Small-Model Quantization Gets Concrete for Edge AI
Edge AI4 min read

Ternary Weights and Tiny Tanks: Small-Model Quantization Gets Concrete for Edge AI

A ternary 8B model, a BitNet toy for TinyStories, a fish tank run by a 14M LLM, and a Rust retrieval encoder push quantization research toward real hardware.

269 views
Read
Split-Brain Robots: Jetson Handles the Vision, a $49 MCU Kit Handles the Milliseconds — Edge AI Keeps Dividing the Job
Edge AI3 min read

Split-Brain Robots: Jetson Handles the Vision, a $49 MCU Kit Handles the Milliseconds — Edge AI Keeps Dividing the Job

A student-built quadruped, a walking-robot policy on a Rockchip SBC, a tiny STM32N6 vision camera, and a $49 TinyML kit all landed this week.

283 views
Read
Qualcomm Chases 30B AI Models on a Phone as Edge Silicon Keeps Multiplying
Edge AI4 min read

Qualcomm Chases 30B AI Models on a Phone as Edge Silicon Keeps Multiplying

A phone NPU claims 30B MoE inference, a 35B model streams from storage on a Mac, and an XDNA1 NPU gets a Linux bring-up.

354 views
Read
A $1 Microcontroller Learns to Dream Faces as On-Device AI Keeps Shrinking
Edge AI4 min read

A $1 Microcontroller Learns to Dream Faces as On-Device AI Keeps Shrinking

An RP2350 chip runs a diffusion model, an ESP32-S3 speaks Japanese, and researchers tackle mobile power and tiny-drone control at the edge.

188 views
Read
A 90-Million-Parameter LLM Talks From a 2004 PSP, While On-Device AI Keeps Finding Smaller Homes
Edge AI4 min read

A 90-Million-Parameter LLM Talks From a 2004 PSP, While On-Device AI Keeps Finding Smaller Homes

A modded Sony PSP runs a tiny LLM, a $3 chip learns to speak, and a 2B open model claims agentic skills for phones.

189 views
Read
One Toolkit, One Chip, One Watch: On-Device AI Keeps Colonizing the Phone Layer
Edge AI4 min read

One Toolkit, One Chip, One Watch: On-Device AI Keeps Colonizing the Phone Layer

A GitHub toolkit, a 100MB cloning TTS, a Pi-powered dashcam agent, and a phone-to-watch AI rollout — all inference staying on the device.

209 views
Read
Runtimes Race Ahead: llama.cpp 0.4.0 and a 30B On-Device AI Agent Test the Edge's Limits
Edge AI4 min read

Runtimes Race Ahead: llama.cpp 0.4.0 and a 30B On-Device AI Agent Test the Edge's Limits

A new llama.cpp release, an ExecuTorch-powered 30B agent model, a cheap RK3576 vision board, and a DIY Jetson robot dog mark a busy week for edge toolchains.

251 views
Read
Quantization Papers Pile Up as Researchers Argue Over Where Small AI Should Spend Its Bits
Edge AI4 min read

Quantization Papers Pile Up as Researchers Argue Over Where Small AI Should Spend Its Bits

Fresh arXiv work rethinks quantization strategy and sustainability, an Apple-adjacent paper shrinks the dictation encoder, and new silicon and Jetson guidance round out the ed

237 views
Read
Robot Policies Start Running Like Local AI Chatbots — Straight on the Device
Edge AI4 min read

Robot Policies Start Running Like Local AI Chatbots — Straight on the Device

A llama.cpp-style engine ports VLA robot policies to Jetson, NVIDIA doubles entry-level robotics compute, and a $10 chip proves the floor of local AI.

226 views
Read
AMD, Qualcomm and NVIDIA All Chase the Same Local-AI Bottleneck: Memory
Edge AI5 min read

AMD, Qualcomm and NVIDIA All Chase the Same Local-AI Bottleneck: Memory

Three chipmakers and a mini-PC builder all attack the same problem this week: getting AI inference closer to memory, not just closer to silicon.

335 views
Read
A Microcontroller Learns to Retrain Itself as On-Device AI Keeps Multiplying Tasks
Edge AI4 min read

A Microcontroller Learns to Retrain Itself as On-Device AI Keeps Multiplying Tasks

Fresh arXiv work tackles MCU vision drift and phone LLM memory pressure, while a solar bird feeder and a desktop WALL-E show the hobbyist edge staying busy.

173 views
Read
A Solo Coder Brings Vision to a Local LLM as Edge Silicon and TinyML Builders Push Back the Frontier
Edge AI4 min read

A Solo Coder Brings Vision to a Local LLM as Edge Silicon and TinyML Builders Push Back the Frontier

DeepSeek V4 Flash gains on-device vision on a Mac, a $1 chip draws pictures, and Qualcomm ships new edge silicon ahead of IFA.

228 views
Read
A Million-Token LLM Tries to Fit On Your Phone as Tiny Voice Models Multiply
Edge AI4 min read

A Million-Token LLM Tries to Fit On Your Phone as Tiny Voice Models Multiply

An iFLYTEK spin-off open-sources a 1.7B model claiming native million-token context on-device, while a 14MB tool-caller and an open voice-agent LLM push the small-model race f

453 views
Read
llama.cpp Teaches an NPU to Share the Load as Small AI Engines Keep Multiplying
Edge AI4 min read

llama.cpp Teaches an NPU to Share the Load as Small AI Engines Keep Multiplying

llama.cpp adds multi-NPU Hexagon support, ONNX Runtime brings quantized KV caches to the browser, and a solo Rust engine beats llama.cpp on tiny models.

280 views
Read
A 27B LLM Shrinks to Phone Size as Edge Quantization Keeps Rewriting the Rules
Edge AI4 min read

A 27B LLM Shrinks to Phone Size as Edge Quantization Keeps Rewriting the Rules

A 1-bit Qwen derivative fits an iPhone, a healing trick beats its own teacher at 4-bit, and a dense LLM limps along on an $8 chip.

192 views
Read
A Duck Robot Runs Its Balance Loop On-Chip as Local AI Quantization Claims Get Audited
Edge AI4 min read

A Duck Robot Runs Its Balance Loop On-Chip as Local AI Quantization Claims Get Audited

A Rockchip-powered duck robot, two offline Raspberry Pi builds, and audits exposing quantization's blind spots and mislabeled GGUF files.

205 views
Read
Edge Dispatch: Intel's Wildcat Lake Brings a Modest NPU to AI PCs, While Edge AI Claims It's Going Mainstream in IoT
Edge AI3 min read

Edge Dispatch: Intel's Wildcat Lake Brings a Modest NPU to AI PCs, While Edge AI Claims It's Going Mainstream in IoT

Intel details a 17 TOPS NPU chip built for chiplets, and an industry trend piece argues edge inference is leaving the pilot stage — both light on independent proof so far.

201 views
Read
Edge Dispatch: IBM Ships an Edge-First Granite LLM as RISC-V AI Boards Keep Fragmenting the Toolchain
Edge AI3 min read

Edge Dispatch: IBM Ships an Edge-First Granite LLM as RISC-V AI Boards Keep Fragmenting the Toolchain

IBM's Granite 4.2 targets edge devices with a 3B open model, while a new RISC-V AI pocket computer ships locked to its own OS fork.

237 views
Read
Edge Dispatch: Jetson Orin Nano 2 Doubles Edge Robotics AI, While Local LLM Agents Get a Bigger Home on the Desktop
Edge AI4 min read

Edge Dispatch: Jetson Orin Nano 2 Doubles Edge Robotics AI, While Local LLM Agents Get a Bigger Home on the Desktop

NVIDIA doubles its entry robotics brain, Perplexity moves agents onto local GPUs, and Liquid AI ships a speedup and a benchmark suite for on-device models.

284 views
Read
Edge Dispatch: Hearing Aids Get Their Own AI Chips as On-Device Voice and Vision Push Past the Phone
Edge AI4 min read

Edge Dispatch: Hearing Aids Get Their Own AI Chips as On-Device Voice and Vision Push Past the Phone

Hearing aids ship dedicated on-device AI chips, new AI glasses land, and real-phone benchmarks show why raw specs don't tell the whole story.

262 views
Read
Edge Dispatch: Meta's Muse Glimmer Bets Big on On-Device Agentic AI as the Runtime Wars Keep Multiplying
Edge AI4 min read

Edge Dispatch: Meta's Muse Glimmer Bets Big on On-Device Agentic AI as the Runtime Wars Keep Multiplying

Meta ships an on-device agentic model, an MoE engine claims 753B on one GPU, and researchers find 10 CVEs in a local inference engine.

463 views
Read
Edge Dispatch: Liquid AI Trains Its Way Around the Q4_0 Quality Tax, While New Papers Chip Away at What Quantization Quietly Breaks
Edge AI4 min read

Edge Dispatch: Liquid AI Trains Its Way Around the Q4_0 Quality Tax, While New Papers Chip Away at What Quantization Quietly Breaks

Liquid AI distills instead of just rounding for 4-bit LFM2.5 checkpoints, as fresh research flags what low-bit quantization costs in memory and multilingual accuracy.

254 views
Read
Edge Dispatch: Edge Dispatch: NVIDIA Squeezes a 4B World Model Onto the Robot Itself as Liquid AI and AMD Chase the Same Edge
Edge AI3 min read

Edge Dispatch: Edge Dispatch: NVIDIA Squeezes a 4B World Model Onto the Robot Itself as Liquid AI and AMD Chase the Same Edge

A Jetson Thor robot policy, a GGUF port for VLA models, a 2.6B tool-calling LLM, and a new AMD robotics module — all inference, no data center.

349 views
Read
Edge Dispatch: AMD's Ryzen AI Halo Jumps to 192GB While Qualcomm Pushes On-Device AI Agents
Edge AI5 min read

Edge Dispatch: AMD's Ryzen AI Halo Jumps to 192GB While Qualcomm Pushes On-Device AI Agents

AMD bumps its NPU mini PC to 192GB of unified memory, Qualcomm ships five agentic apps for Snapdragon X, and Korea rethinks its NPU strategy.

356 views
Read
Edge Dispatch: A Robot Arm Joins the NPU Party: FastFlowLM Adds a Vision-Language-Action Model to Ryzen AI
Edge AI5 min read

Edge Dispatch: A Robot Arm Joins the NPU Party: FastFlowLM Adds a Vision-Language-Action Model to Ryzen AI

FastFlowLM's first stable release puts a robotics policy on Ryzen AI's NPU, while Korea ships a boxed NPU appliance and a Raspberry Pi learns to narrate what it sees.

288 views
Read
Edge Dispatch: Korea's KT Ships a Boxed NPU LLM Station While the ESP32 Crowd Trims Memory Further
Edge AI3 min read

Edge Dispatch: Korea's KT Ships a Boxed NPU LLM Station While the ESP32 Crowd Trims Memory Further

A Korean telecom sells an all-in-one on-prem LLM box built on a domestic NPU, while ESP32 tinkerers keep shrinking what a model needs to run.

280 views
Read
Edge Dispatch: Ryzen AI's NPU Runtime Goes Official While a Raspberry Pi Learns to See and Speak with a Tiny LLM
Edge AI4 min read

Edge Dispatch: Ryzen AI's NPU Runtime Goes Official While a Raspberry Pi Learns to See and Speak with a Tiny LLM

AMD folds a hobbyist NPU runtime into ROCm, Google shows Gemma driving a robot from a Raspberry Pi 5, and a 45M-parameter model books tool calls on a phone.

251 views
Read
Edge Dispatch: Raspberry Pi's GPU Joins the AI Party While a 14MB Model Learns to Call Tools
Edge AI4 min read

Edge Dispatch: Raspberry Pi's GPU Joins the AI Party While a 14MB Model Learns to Call Tools

A Raspberry Pi 5 runs Gemma and vision models split across CPU and GPU, and a 45M-parameter model fits tool-calling into 28MB of RAM.

249 views
Read
Edge Dispatch: The $8 AI Chip Grows a Coffee Habit: ESP32 Tiny-LLM Trick Gets a Second Model
Edge AI4 min read

Edge Dispatch: The $8 AI Chip Grows a Coffee Habit: ESP32 Tiny-LLM Trick Gets a Second Model

A new 'Barista' model and a closer look at Google's Per-Layer Embeddings trick show what running an LLM on a microcontroller can and can't do.

265 views
Read