llama.cpp's Nightly Grind Teaches Phone Chips New AI Tricks While NVIDIA Ships a Rival Edge Runtime

llama.cpp's Nightly Grind Adds Hexagon Ops and New Model Support

llama.cpp's release cadence didn't pause after its 0.4.1 tag on September 14, 2026, which bumped the ggml backend to v0.24.0, added a precision-control API, and replaced the separate --mmap, --mlock and --direct-io flags with a single --load-mode switch, per Freedom.Tech. That release also added architecture support for Maple 20B-A1B, Tencent Hy 4 and Spark 2.5, according to AI/TLDR's changelog.

Nightly builds kept moving. Build b11019 fixed a memory-alignment bug that silently corrupted GGUF tensors embedded inside larger container files, per 16×AI. Build b11054, tagged September 19, 2026, added TOP_K sampling on Qualcomm's Hexagon DSP alongside CUDA 13 and ROCm 10.0 driver support, per 16×AI. The following builds added a faster GEGLU_QUICK operator and 32-bit GET_ROWS support for the same Hexagon chips, and fixed a freeze affecting the newly released Ling 3.0 Flash model, per The Neural Feed's build-by-build coverage.

No single build is dramatic. Together they show the maintainers chasing Qualcomm's Hexagon DSP operator by operator, the same slow grind that made llama.cpp relevant on CPUs and GPUs before it.

NVIDIA's TensorRT-Edge-LLM Adds Day-0 Support for New Small Models

NVIDIA pushed TensorRT-Edge-LLM to version 0.10.0, adding what its release notes call Day-0 support for Qwen3.8-27B and for NVIDIA's own Nemotron-3.5 Lightning model, the latter paired with MTP and DFlash speculative-decoding techniques, according to the project's GitHub releases page. The same version adds Cosmos3-Edge, DiffusionGemma, Nemotron-3.5-ASR and DSpark speculative decoding.

TensorRT-Edge-LLM is NVIDIA's runtime for Jetson-class hardware, distinct from the datacenter-focused TensorRT-LLM. Day-0 support means the runtime ships kernels the same day a model lands rather than weeks later — useful for Jetson teams, though it's NVIDIA's own framing and independent throughput numbers haven't surfaced yet.

The mix of an ASR model, a diffusion model and a speculative-decoding scheme in one release suggests NVIDIA sees edge deployment moving beyond chat toward the full stack of speech, vision and generation running on a single board.

A New Leaderboard Measures Small LLMs in Tokens Per Joule, Not Just Speed

Developer Yuvraj Singh has published smolperfbenchmark, a public leaderboard for small open-weight models — roughly 135 million to a few billion parameters — running on real consumer and edge boards rather than datacenter GPUs. The first live device is the 8GB Jetson Orin Nano Super, with Raspberry Pi 5, a OnePlus 10R, a Xiaomi Pad 6 and a Mac Mini M4 listed as coming next.

The headline metric is tokens per joule, alongside time-to-first-token, inter-token latency, power draw and junction temperature — an attempt to rank efficiency rather than raw speed, which is what actually matters once a model runs off a battery. Per-device harness folders let results be checked script by script instead of taken on faith.

It's a one-person project, not an industry standard, but it fills a real gap: most public leaderboards rank capability, not how many useful tokens a Jetson or a Pi squeezes from a watt-hour.

Brainchip Puts Its Neuromorphic Co-Processor on $99 Cards

Brainchip has started shipping M.2 and PCIe versions of its AKD1500 edge-AI co-processor, along with a BrainBoard 1500 SPI carrier module, priced from $99, according to CNX Software. The AKD1500 is rated at up to 800 GOPS at around 3 watts, aimed at always-on sensing tasks that don't justify a full NPU.

Brainchip first detailed the AKD1500 chip in November 2025; what's new is that the cards and SPI module for embedding it in a design are now actually purchasable at a stated price rather than sitting on a roadmap.

For makers, an M.2 card means an AKD1500 can be dropped into any board with an M.2 slot the way a Hailo or Coral accelerator would — a small but concrete step toward making neuromorphic inference something a hobbyist plugs in rather than designs around.

Realtek's RTL8773J Brings Bluetooth 7 to a Dual-Core Cortex-M55/M33 Chip

Realtek has detailed the RTL8773J, which it calls the first Bluetooth 7-ready microcontroller family, pairing a 200 MHz Arm Cortex-M55 application core with a Cortex-M33 core and support for High-Data-Throughput (HDT) mode, according to CNX Software.

Bluetooth 7's HDT mode targets higher-throughput audio and sensor links, and the Cortex-M55 core carries Arm's Helium DSP extensions, relevant for on-device signal processing and small neural-network inference — though Realtek's own materials emphasize audio and connectivity rather than ML specifically.

It's a connectivity chip first, not an AI chip, but the Cortex-M55/M33 pairing puts it in the same silicon family increasingly used for wake-word detection on wearables — a reminder that the AI story at this tier often arrives on a chip bought for something else entirely.

Nothing here is a breakthrough on its own — an operator here, a purchasable card there — but it's the kind of small, dated plumbing that decides whether an edge project ships this quarter.