Tag

On-Device AI

10 posts

ESP32 Special: Small LLMs Learn to Chat, Listen and Keep the Fish Alive
Edge AI4 min read

ESP32 Special: Small LLMs Learn to Chat, Listen and Keep the Fish Alive

A full day inside the ESP32 world: chatty microcontroller LLMs, a $5-chip speech model, and two new boards from Espressif's own community.

24 views
Read
A $300 GPU Streams a 177B AI Model From an SSD While llama.cpp Learns to Skip Ahead
Edge AI4 min read

A $300 GPU Streams a 177B AI Model From an SSD While llama.cpp Learns to Skip Ahead

Community builders push token throughput further this week — via SSD-streamed MoE experts, prompt-lookup drafting, and a wrapper for Apple's built-in on-device LLM.

56 views
Read
A Wristband Reads Muscles, a Ring Wants Your Ideas: Edge AI Moves Onto the Body
Edge AI4 min read

A Wristband Reads Muscles, a Ring Wants Your Ideas: Edge AI Moves Onto the Body

New wearable and phone releases push transcription, gesture control and silent speech fully on-device, while ESP32 and Jetson tooling keeps pace.

75 views
Read
One Toolkit, One Chip, One Watch: On-Device AI Keeps Colonizing the Phone Layer
Edge AI4 min read

One Toolkit, One Chip, One Watch: On-Device AI Keeps Colonizing the Phone Layer

A GitHub toolkit, a 100MB cloning TTS, a Pi-powered dashcam agent, and a phone-to-watch AI rollout — all inference staying on the device.

77 views
Read
A Million-Token LLM Tries to Fit On Your Phone as Tiny Voice Models Multiply
Edge AI4 min read

A Million-Token LLM Tries to Fit On Your Phone as Tiny Voice Models Multiply

An iFLYTEK spin-off open-sources a 1.7B model claiming native million-token context on-device, while a 14MB tool-caller and an open voice-agent LLM push the small-model race f

202 views
Read
Edge Dispatch: Hearing Aids Get Their Own AI Chips as On-Device Voice and Vision Push Past the Phone
Edge AI4 min read

Edge Dispatch: Hearing Aids Get Their Own AI Chips as On-Device Voice and Vision Push Past the Phone

Hearing aids ship dedicated on-device AI chips, new AI glasses land, and real-phone benchmarks show why raw specs don't tell the whole story.

108 views
Read
Edge Dispatch: Meta's Muse Glimmer Bets Big on On-Device Agentic AI as the Runtime Wars Keep Multiplying
Edge AI4 min read

Edge Dispatch: Meta's Muse Glimmer Bets Big on On-Device Agentic AI as the Runtime Wars Keep Multiplying

Meta ships an on-device agentic model, an MoE engine claims 753B on one GPU, and researchers find 10 CVEs in a local inference engine.

245 views
Read
Flipper One Wants to Be the First Hacker Tool With a Local LLM — What Its 6 TOPS NPU Can Actually Run
Edge AI7 min read

Flipper One Wants to Be the First Hacker Tool With a Local LLM — What Its 6 TOPS NPU Can Actually Run

Flipper Devices' pocket Linux box promises an LLM that runs offline and knows the device inside out. Rockchip's own numbers say what a 6 TOPS RK3576 really does — and the NPU driver isn't in the kernel Flipper chose.

176 views
Read
Edge Dispatch: Raspberry Pi's GPU Joins the AI Party While a 14MB Model Learns to Call Tools
Edge AI4 min read

Edge Dispatch: Raspberry Pi's GPU Joins the AI Party While a 14MB Model Learns to Call Tools

A Raspberry Pi 5 runs Gemma and vision models split across CPU and GPU, and a 45M-parameter model fits tool-calling into 28MB of RAM.

92 views
Read
AMD Just Bought the 'Ollama of NPUs': What FastFlowLM Means for Local LLMs
Edge AI6 min read

AMD Just Bought the 'Ollama of NPUs': What FastFlowLM Means for Local LLMs

A 17MB runtime that runs LLMs on AMD's Ryzen AI NPUs — built by three academics, acquired by AMD on July 17, 2026, folded into ROCm in August. The NPU rung of the edge ladder just got real.

234 views
Read