Skip to content
Back to Edge

A 0.8B Agent LLM, a 4.5M-Parameter Voice and Two 4-Bit Quantizers: Tiny AI Gets Specific

Prateek SinghNovember 15, 20264 min read123 views
A 0.8B Agent LLM, a 4.5M-Parameter Voice and Two 4-Bit Quantizers: Tiny AI Gets Specific

Tether's 0.8B tool-calling model, Huawei's W4A4 quantizer, a 4.5M-parameter open TTS and a lattice-quantization experiment, all with the caveats left in.

BotPsy: a 0.8B tool-calling model for on-device assistants

On November 11, 2026, Tether AI Research published BotPsy on the Hugging Face blog. It is a 0.8B-parameter model built on Qwen3.5-0.8B and tuned for function calling and agentic workflows. The target tasks are web search, calendar, email, file handling, commerce and device or vehicle control.

The authors say BotPsy beats every model up to 1.2B parameters that they tested on tool-use and agentic benchmarks. That is the lab's own comparison, and the field of rivals was chosen by the authors. They also say it is best for low-to-medium difficulty tasks and not for high-stakes use, because it still makes mistakes.

The post describes the model as suitable for on-device deployment, but the excerpt I read gives no measured speeds on phones or boards. Treat it as a candidate for local agents, not a proven one. A sub-1B model that can emit reliable tool calls is the kind of part a phone or SBC assistant needs, if the claims hold up.

Huawei's MACaQ pushes weights and activations to 4 bits

Huawei's Computing Systems Lab published MACaQ on GitHub on November 10, 2026, under Apache 2.0. According to On The Wire, it outputs GGUF files for llama.cpp. The method is described in arXiv 2611.10649 by Lorenz K. Muller, Philippe Bich, Chiara Boretti, Hyun-Min Chang, Jiawei Zhuang and Lukas Cavigelli.

It is an MXFP4 W4A4 post-training method, so both weights and activations sit at 4 bits. Each linear layer is handled independently, which means only one layer must be in memory while quantizing. A refinement step then nudges individual 4-bit codes to reduce error.

The cost is runtime overhead, because a per-channel scale and a block-Hadamard transform run online. The paper puts it at about 2.3% for Qwen3-32B and about 8% for the smallest model tested, at 0.6B. Small models, the ones that fit edge hardware, pay the larger relative tax. I have not seen independent accuracy or speed results on edge devices.

tiny-voca: an open 4.5M-parameter text-to-speech model

On November 14, 2026, a r/LocalLLaMA poster released tiny-voca, an English text-to-speech model with 4.5M parameters and two voices. The post says the code, weights and dataset are all open, that it runs in the browser with nothing sent to a server, and that it runs about 70 times faster than real time on a CPU.

The 70x figure is the author's own, and I did not see which CPU it was measured on. Two voices and English only is a narrow product. Still, a fully open voice model at this size is small enough to consider for a Pi-class board, and the open dataset lets others check how it was trained. The browser demo makes it easy to judge the voice quality yourself.

Leech-lattice quantization takes Qwen3 to about 2.7 bits per weight

Also on November 14, 2026, a r/LocalLLaMA post described an open-source Rust and CUDA project. It quantizes Qwen3 4B, 8B and 14B to about 2.7 bits per weight using Leech lattice quantization. The poster says it beats llama.cpp's IQ2_S on quality at the same size but loses on speed.

Those are self-reported results, and I worked from the post's summary rather than a reproduction. The speed loss is the catch for edge use, since decode speed on small devices is usually memory-bound and cares about decoding cost. It is also a CUDA implementation, so it does not yet speak to phones or microcontrollers.

It belongs in the same pile as today's other items: quantization research is trading bits, accuracy and decode cost, and no format wins on all three.

All four items are early: a model card, a preprint, and two forum posts. The numbers are the authors' own, so read the sources before trusting any of them.

References & Citations

Subscribe to new posts from theaivibe.org

No spam — just new posts. One-click unsubscribe.
Share this article

Related Posts