Edge Dispatch: Meta's Muse Glimmer Bets Big on On-Device Agentic AI as the Runtime Wars Keep Multiplying

Meta's Muse Glimmer bets on local-first, not local-eventually

Meta AI Research released open weights for Muse Glimmer, a 30-billion-parameter agentic model built specifically to run locally rather than in a datacenter. The published weights quantize to roughly 4-bit precision, shrinking the model to under 20GB, with headroom left for its KV cache, an image-perception encoder, and a speculative-decoding drafter.

The weights are downloadable now on Hugging Face and already loadable through Ollama, LM Studio, and Unsloth. Meta says dedicated integrations for llama.cpp, MLX, and ExecuTorch are coming "in the coming days" — meaning the three runtimes this beat follows most closely don't have first-class support at launch.

Thirty billion parameters, even compressed, is prosumer-GPU or high-memory-Mac territory, not phone or microcontroller territory. But a frontier lab shipping an agentic model with on-device deployment as the design constraint from the start, not a bolted-on afterthought, is itself notable on a beat built on the assumption that the biggest labs treat local inference as secondary.

FreeToken claims a 753B model on one workstation GPU

MarkTechPost covered FreeToken on August 23, 2026, an edge-native mixture-of-experts serving engine whose developers say it runs the 753-billion-parameter GLM-5.2 model (40 billion active) on a single workstation GPU. On an 8GB RTX 4060 laptop, they report an NVFP4-quantized 35B model decoding at 39.3 tokens/second; on an RTX PRO 6000, GLM-5.2 reportedly hits 14.9 tokens/second against llama.cpp's 7.3.

These are self-reported figures from the project, relayed through MarkTechPost's write-up, with no independent third-party benchmark yet. And "single workstation GPU" still means an RTX PRO 6000, a card well outside consumer budgets.

The mechanism matters more than the flagship number: serving that only activates the experts a token actually needs is what makes models this large plausible outside a datacenter rack at all, and the same principle is the one behind the laptop-GPU numbers in the same benchmark set.

Ten CVEs found in a local inference engine

Researchers at Cyera disclosed 10 vulnerabilities in a widely used local inference engine, coordinating with VulnCheck after direct disclosure with the project's maintainers stalled. VulnCheck allocated CVE-2026-43622 through CVE-2026-43632, with CVE-2026-43625 left unused.

The report doesn't spell out patch status for every finding, and severity varies by CVE — worth reading the full write-up rather than the headline count of ten.

It's a useful corrective for a beat that often frames local inference as inherently safer than sending prompts to a cloud API. The runtimes powering phones, single-board computers, and AI PCs are ordinary C++ codebases parsing untrusted model files and network input, and they carry ordinary software vulnerabilities.

LiteRT-LM gets a feature llama.cpp can't reach yet

Google shipped LiteRT-LM v0.10.1 with support for Gemma 4's multi-token-prediction heads, an architecture feature that Groundy reports llama.cpp currently has no path to use. Google's own published figures show Gemma 4 E2B reaching 3,700 prefill tokens/second and 31 decode tokens/second on Qualcomm's Dragonwing IQ8 NPU, against 133 prefill and 7.6 decode tokens/second on a Raspberry Pi 5's CPU alone.

Those numbers come from Google, not an independent lab, and the NPU-versus-CPU gap says as much about accelerator hardware as it does about the runtime.

The real signal is that model-architecture features are becoming a runtime lock-in lever: if MTP heads only pay off inside LiteRT-LM, developers choosing Gemma 4 are also implicitly choosing Google's runtime over the llama.cpp ecosystem this beat has tracked for years.

Base Compute says AI agents wrote its Metal kernels — and beat llama.cpp

Base Compute published benchmarks on Hugging Face for its BaseRT runtime, reporting 1.39–1.76x llama.cpp and 1.98–2.55x MLX on prefill, and 1.90x llama.cpp / 1.43x MLX on decode, running NVIDIA's Nemotron 3 Nano hybrid MoE model on Apple Silicon. As RuntimeWire notes, BaseRT hadn't supported that architecture before; the company says its Base Optimization Stack pipeline used AI research agents to write the Metal kernels from scratch.

This is self-reported by the vendor building the runtime, on one model family, and "AI agents write the kernels" needs scrutiny beyond a single benchmark table before anyone calls the problem solved.

If automated, per-architecture kernel tuning holds up across more models, it would shortcut the months of manual porting work that currently gates how fast new architectures reach fast on-device inference on Macs.

Five different teams, five different bets on how local inference should be built — a frontier lab's agentic model, an edge-native MoE engine, a security audit of the engines everyone already runs, a runtime-locked model feature, and an automated kernel-tuning pipeline. The sources below are worth reading past the headline numbers.