Edge Dispatch: Liquid AI Trains Its Way Around the Q4_0 Quality Tax, While New Papers Chip Away at What Quantization Quietly Breaks

Liquid AI trains its 4-bit checkpoints instead of just rounding them

Liquid AI released new Q4_0 GGUF checkpoints for its LFM2.5 family (230M, 350M, 1.2B-Instruct, and 2.6B) built with what it calls Quantization-Aware Distillation, or QAD: a high-precision teacher model trains a quantized student directly, rather than quantizing a finished model after the fact. The company says QAD closes 48–73 percent of the accuracy gap between BF16 and standard post-training Q4_0, depending on model size, with the resulting checkpoints retaining 96.5–97.4 percent of BF16 baseline scores across GPQA Diamond, MMLU-Pro, IFEval, and tool-use benchmarks, per Liquid AI's writeup and a parallel Hugging Face post.

The throughput numbers are self-reported and measured on real edge hardware: a MacBook Pro and a NucBox EVO-X2 (GPU inference), plus a Samsung Galaxy S26 Ultra and a Raspberry Pi 5 (Arm CPU inference). Liquid AI claims the QAD Q4_0 checkpoints match Q5_K_M quality at 4–33 percent higher decode throughput for the small models, and match Q4_K_M quality at 3–14 percent higher throughput for the larger ones, also matching Unsloth's UD-Q4_K_XL where comparable. The Agent Times flagged the release the same week.

None of this is independently verified, and it is vendor benchmarking on vendor models. But if the recovery numbers hold up outside Liquid AI's own tests, it is a real argument for training-time quantization awareness over pure post-hoc rounding on memory-starved boards like the Pi 5.

A coordinate-descent tweak closes ground on 2-bit quantization

A new preprint on discrete optimization for layer-wise quantization, SchurQuant, reports that adding closed-form row-wise scale and zero-point refitting with coordinate descent over integer codes on top of the standard GPTQ objective improves mean zero-shot accuracy on 2-bit Qwen3-4B by 11.88 percentage points, according to the arXiv paper.

This is a research result, not a released tool: no repo, no third-party benchmark, and the paper notes the gains shrink at higher bit precision where GPTQ's tighter search space leaves less room to improve. It is one more data point in a crowded field of 2-bit tweaks chasing the same problem.

Why it matters here: a 4B model at usable 2-bit accuracy is squarely in NPU-and-phone territory, and incremental gains like this are what eventually separate 2-bit quantization from a lab curiosity.

INT4 quantization can quietly make models more confused, not just less precise

A new paper, "Compress and Forget," tested bitsandbytes INT4 quantization on fine-tuned Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3, and Phi-3.5-mini-instruct on a retrieval task under proactive interference — cases where a model has to recall the right fact among several similar, competing ones. Accuracy fell sharply under high interference for every model tested, for example dropping from 81.0 percent to 68.3 percent in one case, per the arXiv paper.

The caveat: this is specifically bitsandbytes INT4, on three particular models, and it is not clear the effect generalizes to GGUF-style quantization schemes common on edge runtimes. It is also a single research group's evaluation, not a broad benchmark sweep.

The relevance for on-device assistants juggling multiple facts, contacts, or conversation threads is direct: standard perplexity and accuracy benchmarks may look fine on a quantized model even as it gets quietly worse at telling similar facts apart.

Quantization hits non-English small models harder, a new paper argues

A preprint titled "The Multilingual Quantization Tax" evaluated quantized Gemma and Qwen 3.5 architectures across eight typologically diverse languages using MMLU Pro X Lite and Global PIQA, and reports that parameter truncation exposes uneven pre-training coverage across languages — a pattern the authors call "Typological Fragility," among other effects, according to the paper.

The summary available does not spell out exact bit-widths or absolute accuracy numbers for each language, so how large the effect is in practice remains to be seen from the full paper.

Most small-model quantization coverage focuses on English benchmarks. If this holds up, it is a reminder that a 4-bit edge model marketed as multilingual may be quietly worse for some languages than its English scores suggest — a real concern as SLMs spread to phones and NPU boards worldwide.

Today's sweep skews academic rather than shipped hardware, but the theme holds: quantization keeps getting better on paper while researchers keep finding new ways it quietly breaks things in practice, and both threads matter for anything running a model on a device with no room to spare.