Skip to content

Tag

On-Device AI

22 posts

A 1.7B Non-Transformer LLM Targets the iPhone as Mobile NPUs Get a GUI Agent Paper
Edge AI4 min read

A 1.7B Non-Transformer LLM Targets the iPhone as Mobile NPUs Get a GUI Agent Paper

Liquid AI's LFM3 puts a 1.7B model in 1.9 GB on an iPhone 17 Pro, a new paper targets phone NPUs, and a $7 ESP32 drives an iPhone.

106 views
Read
A 0.8B Agent LLM, a 4.5M-Parameter Voice and Two 4-Bit Quantizers: Tiny AI Gets Specific
Edge AI4 min read

A 0.8B Agent LLM, a 4.5M-Parameter Voice and Two 4-Bit Quantizers: Tiny AI Gets Specific

Tether's 0.8B tool-calling model, Huawei's W4A4 quantizer, a 4.5M-parameter open TTS and a lattice-quantization experiment, all with the caveats left in.

123 views
Read
An ESP32 Speaks, Chats, and Gets a New $21 Brain in Today's All-Microcontroller AI Special
Edge AI3 min read

An ESP32 Speaks, Chats, and Gets a New $21 Brain in Today's All-Microcontroller AI Special

A 3.8MB voice model, an early ChatML LLM, and a $21 board all land on the ESP32 this week.

2.6K views
Read
Apple Watch's On-Device AI Learns to Eavesdrop While a Tiny Korean VLM Teaches Robots to See
Edge AI5 min read

Apple Watch's On-Device AI Learns to Eavesdrop While a Tiny Korean VLM Teaches Robots to See

A watch beta, a microcontroller voice SDK, a lightweight robot-vision model and a new ExecuTorch release all push inference further onto the device itself.

289 views
Read
A Game Runs Entirely on a $6 Chip While AMD Tries to Skip ROCm for Local AI
Edge AI4 min read

A Game Runs Entirely on a $6 Chip While AMD Tries to Skip ROCm for Local AI

A tiny world model plays games on a Pico 2 W, AMD ships an early NPU/GPU SDK, and a measurement study checks edge-NPU VLM hype.

183 views
Read
Three ESP32 Builds Show How Far On-Device AI Has Come in a Week
Edge AI3 min read

Three ESP32 Builds Show How Far On-Device AI Has Come in a Week

An open-vocabulary speech engine, a full offline voice assistant, and a new ESP32-P4 compute module show how much now runs on one microcontroller.

451 views
Read
A Pixel Phone Speeds Up On-Device Speech AI While a $2 Chip Learns to See
Edge AI5 min read

A Pixel Phone Speeds Up On-Device Speech AI While a $2 Chip Learns to See

Arm's SME2 doubles streaming-ASR speed on a Pixel 11, a battery-sipping voice stack lands on Android, and two cheap microcontrollers go local.

317 views
Read
Five ESP32 Builds Push Local AI From a Dev Board to a Drone's Camera
Edge AI4 min read

Five ESP32 Builds Push Local AI From a Dev Board to a Drone's Camera

A 28.9M LLM, two phone/voice agents, a new AI-ready dev board and a drone video hack all land on the same $7 chip family.

661 views
Read
A 30B AI Agent Learns to Live on One GPU While a 254M Model Learns to Listen Offline
Edge AI5 min read

A 30B AI Agent Learns to Live on One GPU While a 254M Model Learns to Listen Offline

Meta's 30B on-device agent, a 254M offline speech model, and a $10 ESP32 that hears without the cloud lead today's edge sweep.

224 views
Read
A 27B AI Hits an iPhone's Neural Engine at 3 Bits While a 14M LLM Keeps a Fish Tank Alive on ESP32
Edge AI5 min read

A 27B AI Hits an iPhone's Neural Engine at 3 Bits While a 14M LLM Keeps a Fish Tank Alive on ESP32

Apple's Neural Engine runs a 27B model at low bit-width, a distilled 14M LLM animates a $8 microcontroller pet, and new research questions KV-cache quant on Raspberry Pi.

284 views
Read
A 35B MoE Vision AI Learns to Fit a Phone While a $7 ESP32 Hijacks an iPhone
Edge AI4 min read

A 35B MoE Vision AI Learns to Fit a Phone While a $7 ESP32 Hijacks an iPhone

OpenBMB squeezes a 35B MoE vision model onto an iPhone, Google shrinks multimodal search to 191MB, and a $7 ESP32 learns to tap a real iPhone's screen.

224 views
Read
A 4B AI Model Learns to Run a Robot Arm Entirely on Jetson Thor
Edge AI5 min read

A 4B AI Model Learns to Run a Robot Arm Entirely on Jetson Thor

NVIDIA's Cosmos 3 Edge drives a robot arm on-device, a $5-class chip gets a voice it hasn't spoken yet, and two new boards widen the SBC lineup.

513 views
Read
ESP32 Special: Small LLMs Learn to Chat, Listen and Keep the Fish Alive
Edge AI4 min read

ESP32 Special: Small LLMs Learn to Chat, Listen and Keep the Fish Alive

A full day inside the ESP32 world: chatty microcontroller LLMs, a $5-chip speech model, and two new boards from Espressif's own community.

318 views
Read
A $300 GPU Streams a 177B AI Model From an SSD While llama.cpp Learns to Skip Ahead
Edge AI4 min read

A $300 GPU Streams a 177B AI Model From an SSD While llama.cpp Learns to Skip Ahead

Community builders push token throughput further this week — via SSD-streamed MoE experts, prompt-lookup drafting, and a wrapper for Apple's built-in on-device LLM.

495 views
Read
A Wristband Reads Muscles, a Ring Wants Your Ideas: Edge AI Moves Onto the Body
Edge AI4 min read

A Wristband Reads Muscles, a Ring Wants Your Ideas: Edge AI Moves Onto the Body

New wearable and phone releases push transcription, gesture control and silent speech fully on-device, while ESP32 and Jetson tooling keeps pace.

251 views
Read
One Toolkit, One Chip, One Watch: On-Device AI Keeps Colonizing the Phone Layer
Edge AI4 min read

One Toolkit, One Chip, One Watch: On-Device AI Keeps Colonizing the Phone Layer

A GitHub toolkit, a 100MB cloning TTS, a Pi-powered dashcam agent, and a phone-to-watch AI rollout — all inference staying on the device.

209 views
Read
A Million-Token LLM Tries to Fit On Your Phone as Tiny Voice Models Multiply
Edge AI4 min read

A Million-Token LLM Tries to Fit On Your Phone as Tiny Voice Models Multiply

An iFLYTEK spin-off open-sources a 1.7B model claiming native million-token context on-device, while a 14MB tool-caller and an open voice-agent LLM push the small-model race f

453 views
Read
Edge Dispatch: Hearing Aids Get Their Own AI Chips as On-Device Voice and Vision Push Past the Phone
Edge AI4 min read

Edge Dispatch: Hearing Aids Get Their Own AI Chips as On-Device Voice and Vision Push Past the Phone

Hearing aids ship dedicated on-device AI chips, new AI glasses land, and real-phone benchmarks show why raw specs don't tell the whole story.

262 views
Read
Edge Dispatch: Meta's Muse Glimmer Bets Big on On-Device Agentic AI as the Runtime Wars Keep Multiplying
Edge AI4 min read

Edge Dispatch: Meta's Muse Glimmer Bets Big on On-Device Agentic AI as the Runtime Wars Keep Multiplying

Meta ships an on-device agentic model, an MoE engine claims 753B on one GPU, and researchers find 10 CVEs in a local inference engine.

463 views
Read
Flipper One Wants to Be the First Hacker Tool With a Local LLM — What Its 6 TOPS NPU Can Actually Run
Edge AI7 min read

Flipper One Wants to Be the First Hacker Tool With a Local LLM — What Its 6 TOPS NPU Can Actually Run

Flipper Devices' pocket Linux box promises an LLM that runs offline and knows the device inside out. Rockchip's own numbers say what a 6 TOPS RK3576 really does — and the NPU driver isn't in the kernel Flipper chose.

444 views
Read
Edge Dispatch: Raspberry Pi's GPU Joins the AI Party While a 14MB Model Learns to Call Tools
Edge AI4 min read

Edge Dispatch: Raspberry Pi's GPU Joins the AI Party While a 14MB Model Learns to Call Tools

A Raspberry Pi 5 runs Gemma and vision models split across CPU and GPU, and a 45M-parameter model fits tool-calling into 28MB of RAM.

249 views
Read
AMD Just Bought the 'Ollama of NPUs': What FastFlowLM Means for Local LLMs
Edge AI6 min read

AMD Just Bought the 'Ollama of NPUs': What FastFlowLM Means for Local LLMs

A 17MB runtime that runs LLMs on AMD's Ryzen AI NPUs — built by three academics, acquired by AMD on July 17, 2026, folded into ROCm in August. The NPU rung of the edge ladder just got real.

731 views
Read