AMD Opens Its NPU Stack to AI Agents While Qualcomm Puts an LLM on Your Face

AMD Opens Up Its Ryzen AI NPU Stack

AMD's developer blog introduced FastFlow-SDK this week, a single entry point for its Ryzen AI NPUs built from three pieces: FastFlow (the serving layer, formerly FastFlowLM, now at version 1.0.8), IRON (a close-to-metal Python API for the NPU built on an open MLIR compiler toolchain), and HRX, a lightweight runtime that loads and executes workloads on AMD silicon across Windows and Linux. The SDK targets XDNA2 NPUs only — the Ryzen AI 300, 400 and Max series.

FastFlow plugs into AMD's Lemonade serving stack, exposing an OpenAI-compatible streaming API with tool calling and reasoning-model support. AMD frames this as developer infrastructure, not a model drop: weights still need porting, as the company's own same-week MiniCPM-V-4.7-1B Day-0 release shows.

The caveat: this is AMD documenting its own stack, and it only covers the newest NPU generation. Older Ryzen AI chips are left out — exactly the gap hobbyists are patching below.

A Hobbyist Pipeline Wakes an Older Ryzen NPU — Vision Only

A write-up on the Ecne Project's Pulse blog documents a community pipeline that puts the NPU inside AMD's Ryzen AI 8845 (Phoenix/Hawk Point, XDNA1) to work running vision models under Linux — silicon that has sat idle for most owners since launch.

The pipeline runs CNN workloads in INT8 directly on the NPU. The author is upfront about the limit: the Phoenix/Hawk Point driver stack only exposes INT8 CNN support, not BF16, so large language model inference is still off the table on this generation of chip. Vision works; chat does not.

It's a useful contrast with AMD's own FastFlow-SDK, which only targets the newer XDNA2 chips. For the millions of Ryzen AI 7000/8000-series laptops already sold, community tooling — not vendor software — is currently the only way to wake the NPU up at all.

A Credit-Card Compute Module Runs Qwen3.5 Locally

DFRobot's LattePanda line added the Mu Ultra, a 69.6 mm × 60 mm x86 compute module built around Intel's Core Ultra 5 226V and Core Ultra 7 256V, rated at up to 115 TOPS of combined INT8 compute including 47 TOPS from the dedicated NPU, per the module's press release carried by Kyodo News and detailed further by TechPowerUp.

In the vendor's own testing, the module ran Qwen3.5-9B at 18 tokens/sec and Qwen3.5-2B at 55 tokens/sec using INT4 quantization through OpenVINO GenAI — on the integrated GPU, not the NPU. Worth flagging: the headline NPU TOPS figure and the demonstrated chat speed come from two different compute blocks on the same chip, and both numbers are self-reported.

Still, a sub-70mm module hitting double-digit tokens/sec on a 9B model is a useful reference point for anyone sizing a compact edge PC for local chat.

Raspberry Pi's New HAT Adds Its Own RAM for Local LLMs

Raspberry Pi launched the AI HAT+ 2, a $130 accessory pairing Hailo's 10H chip with its own 8GB of LPDDR4X memory, as detailed by hardware reviewer Jeff Geerling. The Hailo-10H draws a maximum of 3W and is built to run LLMs standalone, entirely on the HAT, without touching the Pi's own CPU cycles or system RAM.

That onboard 8GB is the real change from earlier Hailo HATs: a Raspberry Pi 5 only ships with up to 8GB total, so giving the accelerator its own separate memory pool means a model running on the HAT doesn't compete with whatever else the Pi is doing.

Caveat: models still have to be compiled through Hailo's own toolchain, not loaded straight from a GGUF file, so the ecosystem is narrower than a CPU-only llama.cpp setup. For projects that need both an LLM and a free CPU, the trade looks reasonable.

Qualcomm's New XR Chip Claims 45 Tokens/Sec for a 3B Model

Qualcomm announced the Snapdragon Reality Elite, its new flagship chipset for standalone XR headsets, claiming an NPU 160% more powerful than its predecessor, as reported by UploadVR. Qualcomm's own figures: a 3-billion-parameter LLM running at roughly 45 tokens per second entirely on the headset's NPU, and a 512×512 vision-generation model returning a result in about 1.7 seconds.

These are vendor-supplied numbers from a chip that hasn't yet shipped in a consumer headset, so treat them as a ceiling — real apps running alongside hand tracking and rendering will see less. But a 3B model is squarely in the range people already run on phones, now aimed at a device with nothing tethered to it.

If the numbers hold once headsets ship, XR becomes another form factor where the model runs entirely on the device worn on your face.

Five stories, one thread: NPUs built into chips sold years ago are only now getting software that can reach them, while brand-new silicon arrives already quoting on-device LLM numbers. The gap between what ships and what actually runs stays the real edge AI story.