Google Open-Sources Its On-Device AI Engine While a Ternary LLM Shrinks a 30B Model to 12GB

Google Open-Sources the GPU Engine Inside LiteRT

Google has open-sourced ML Drift, the cross-platform GPU compute engine that actually runs on-device inference for LiteRT, under an Apache 2.0 license. It abstracts OpenGL ES, OpenCL, Metal and WebGPU behind one API and is already powering Chrome, YouTube Shorts, Google Photos and Meet, according to InfoQ's coverage and Google's own LiteRT October release notes.

Google's own benchmark, run on Gemma models on workstation hardware, claims 12% lower memory overhead than llama.cpp and MLX LM, plus up to 40% lower frame latency in YouTube Shorts and 30% faster Adobe Lightroom photo-editing features. These are vendor-reported numbers, not independently verified, and the legacy TensorFlow Lite GPU delegate stops getting updates now that ML Drift exists.

The significance for tiny hardware: this is the actual engine under millions of phones' on-device AI features, now out in the open for other runtime builders to study, borrow from, or benchmark against directly.

ExecuTorch 1.2 Makes Cortex-M a First-Class Target, Adds Real-Time Speech

PyTorch's ExecuTorch project shipped version 1.2.0, promoting its Cortex-M backend to a first-class embedded target rather than an experimental add-on, and adding real-time speech inference through Voxtral Realtime, according to the project's own GitHub release notes. The release also claims backend improvements and reduced binary size for mobile and embedded deployments.

The release notes don't publish hard benchmark numbers for Voxtral Realtime's latency or memory use, and "first-class" status is PyTorch's own characterization rather than a third-party designation.

Still, moving Cortex-M from a side experiment to a flagship target matters: it signals that microcontroller-class inference is now a permanent line item in ExecuTorch's roadmap, not a one-off demo.

A Ternary 1.58-Bit MoE Puts a 30B Model on a 12GB GPU

A quant-maker going by boro64 published Ternary-Nemotron-3.5-Lightning-30B-A3B-GGUF on Hugging Face, converting a Nemotron 3.5 30B-A3B mixture-of-experts model into 1.58-bit ternary weights. The uploader's own numbers, discussed in a r/LocalLLaMA thread, claim the model fits a 12GB GPU, holds a 368K-token context window and decodes around 172 tokens per second.

Those figures are self-reported, not an independent eval, and quality at 1.58 bits typically trades off noticeably against full precision — there's no published benchmark comparing it to the original Nemotron checkpoint in this release.

Even so, it extends a steady 2026 pattern: sub-2-bit GGUF conversions keep shrinking MoE models that would normally need 60GB-plus of VRAM down to cards a hobbyist already owns.

Adafruit's Metro ESP32-P4 Hydra Pairs Two Chips on One Arduino-Shaped Board

Adafruit launched the Metro ESP32-P4 Hydra on October 26, 2026, an Arduino UNO-shaped board built around Espressif's ESP32-P4 paired with a second ESP32-C6 chip that handles Wi-Fi 6 and Bluetooth, since the P4 itself has no wireless radio of its own. The board ships with 16MB of PSRAM, 16MB of NOR flash and MIPI DSI/CSI interfaces for camera and display work, per CNX Software's write-up.

Pricing wasn't confirmed in available coverage, a caveat worth flagging for anyone shopping. But the two-chip pattern — a capable compute SoC with no radio, paired with a small wireless companion — is becoming a common recipe for boards that need to run vision or audio models locally while still staying connected.

BeagleBoard's Open-Source Motor Board Pairs Cortex-M33 With On-Device TinyEngine AI

BeagleBoard.org introduced the BeagleConnect QuadESC, an open-source four-channel brushless DC motor-control board for robotics and drones, built around the BeagleConnect Torque module — a Cortex-M33 controller with CAN FD and Qwiic connectors — detailed in two LinuxGizmos write-ups posted October 25, 2026. The boards run TinyEngine AI locally for motor-control inference instead of streaming every control-loop decision to a host.

Neither write-up lists pricing or a ship date, and the specific models TinyEngine AI runs aren't detailed. Still, closed-loop motor control is a class of edge inference that rarely gets attention next to chatbots and vision demos — exactly the TinyML niche this beat exists to track.

Nothing dramatic today, just the usual churn: a vendor opening up its own engine, a runtime growing up on bare metal, a quant shrinking another MoE model, and two boards aimed at builders rather than benchmarks.