I Gave Quantized Checkpoints a Type, and the Type Immediately Caught Real Bugs

Here's a fact that still bothers me: a four-bit quantized model file tells you how many elements it has and what dtype tag they carry — and almost nothing else that matters. Not which rule derived the shared scales. Not the zero-point convention. Not the sub-byte packing order. Not the layout its scale plane was written in. The file works only because the producer and the consumer happen to agree on all of it, silently, in code.

In 2026 alone, six documented incidents across vLLM and SGLang turned exactly those silent agreements into silently wrong model output. Not crashes — wrong numbers. Scale tensors dropped on load. Signed scales read as unsigned — roughly four orders of magnitude of dequantization error. A swizzled producer wired to a linear consumer. Every one of them loaded cleanly.

So I spent a stretch of this year building GRIT — the Grouped Reduced-precision Interchange Type. It's my answer to a simple question: what is the smallest thing a checkpoint could carry that would have made those failures loud? The paper is on Zenodo, the spec and all five implementations are on GitHub, and everything below reproduces from a clean checkout.

What GRIT actually is

A GRIT array is a quadruple: (Grade, Placement, Planes, Shape). The Grade is the complete numeric contract — element format, scale-derivation rule, zero-point convention, rounding, sparsity, the works — with a canonical string form and a 64-bit id. The Placement is the physical layout, carried as a value rather than baked into a type parameter, so "which layout" and "which numeric contract" stop being one fused enum name like marlin_24. The Planes are up to four byte buffers — data, two scale levels, aux. The Shape comes from the caller.

All of it fits in a 64-byte plain-old-data descriptor — every field at a fixed offset, little-endian, no variable-length anything. The descriptor plus the shape determines every plane's exact byte length, so "does this descriptor match these bytes?" is decidable in O(1) at a boundary crossing. And the check is total: any 64 bytes you throw at it, including adversarial garbage, terminate with exactly one status and no reads outside the buffer.

The part I'd defend hardest isn't the descriptor, though. It's the law set attached to decode(): NaN poison, sparse-wins-over-poison, slice honesty, group-axis transpose as requantization rather than a view, fixed outermost-first evaluation order, no FMA substitution, placement invariance. Formats tell you what bytes mean; almost nobody writes down what the decoder is allowed to do. That's where the silent divergence lives — and it's exactly the edge where existing semantics bow out: StableHLO's 0 < scales constraint affirmatively excludes the hardest of these cases, so the two semantics are disjoint precisely where implementations diverge in practice.

The two guarantees the whole project stands on, both testable:

• Equal gid and equal plane bytes ⇒ bit-identical decode() on every conformant implementation;
• grit_check never has undefined behaviour, even on adversarial 64-byte descriptors.

My favorite finding: two files, identical bytes, opposite meanings

Nobody rewrites their checkpoint format because a blog post asked nicely. So the wedge is grit scan: a read-only auditor that checks a declared contract against bytes where a descriptor exists, and infers one from container evidence where it doesn't — which today means GGUF and safetensors files you already have on disk.

I pointed it at four real Hugging Face checkpoints. My favorite finding: a GPTQ file and an AWQ file whose zero-point planes are identical in byte count, shape, dtype and tensor name — and carry opposite conventions. One stores zero points as-is; the other stores them minus one. Load one as the other and every weight shifts by a full quantization step. Nothing anywhere in either file records which convention is inside.

That one genuinely surprised me. I expected the scanner to find sloppy metadata; I did not expect two of the most widely deployed quantization families to be formally indistinguishable at the byte level while meaning different numbers. It's the purest possible specimen of the whole problem: the meaning isn't in the file. It's in a comment thread somewhere, and in the source of whichever loader you happen to use.

The scanner, and an experiment with a control group

Claims about scanners are cheap, so the field study is reported with both runs, warts first. We downloaded four real, popular checkpoints — GGUF Q4_K_M, GPTQ-Int4, AWQ-Int4, compressed-tensors W4A16 — and read them byte by byte. Run 1, hand analysis plus the scanner as first shipped: real mismatches confirmed in three of the four files, but the tool auto-caught only one of them, indirectly. Honest score: not good enough. We closed exactly three inference gaps and re-ran on bit-identical bytes. Run 2: 12 findings became 349, and every mismatch class now fires automatically. The part that makes the number mean something: a false-positive control on two known-clean files stayed at zero findings, exit 0. The complete experiment log, both runs, is audit/scan-experiment.md in the repo.

Proof over promises

The project is built to be checked rather than trusted. One normative spec (2,164 lines), an executable Python reference, and five zero-dependency implementations — C11, C++20, Rust, pure-stdlib Python, strict TypeScript — that reproduce a 68-vector SHA-256-pinned conformance suite and agree bit-for-bit on 96/96 cross-language descriptor fingerprints, verified by a committed harness and a CI job that re-proves it on every push, not a one-off script. The Rust implementation is differentially fuzzed; the C++ one runs under AddressSanitizer and a strict build.

And the check is cheap enough to leave on. A level-1 structural check costs 296 ns in C (887 cycles) and 687 ns in Rust; on a synthetic 8-shard, 1.07 GB MXFP4 checkpoint, checking every tensor at load time adds 2.7 ms, 33.4 KB of headers, and 108 KiB of resident memory for 192 checks. Against a multi-second checkpoint load, the safety margin is effectively free.

What GRIT does not claim

This section exists because the project's motto has to apply to its own marketing. The spec carries a component-by-component claims table with the closest prior art for each piece, and the honest scope statement next to it. Parameterised quantized types, nested two-level scales, and scale-plus-sparsity in one format are not GRIT's inventions — see MLIR's sub-channel quantized types, compressed-tensors, and Qualcomm's LPBQ. The arithmetic-contract-as-a-value is deployed art in StableHLO and JAX's DotAlgorithm. Canonicalise-then-fingerprint is Apache Avro's discipline, step for step — GRIT changes the object being hashed. The portable POD descriptor pattern is the Khronos Data Format Specification's and DLPack's, down to the same LSB-first sub-byte packing rule. And the FNV-1a fingerprints defend against drift and mislabeling, not against an adversary — there is no collision-resistance claim.

The v1 grammar also has real holes, all named in the spec rather than papered over: AMD's FNUZ FP8 variants are inexpressible; there is no codebook/LUT element class, so NF4 and the llama.cpp IQ* families are out of scope; GPTQ act-order g_idx grouping is unsupported, because act-order can't be hidden in Placement without making the placement-invariance law false; and dense-nibble zero-point planes — GPTQ/AWQ qzeros, two values per byte — are not byte-representable under the v1 padding rule. The scanner reports all of these as inferred_inexpressible instead of pretending. And the field study is n=4: it establishes that the mismatch classes exist in the wild, not how prevalent they are — prevalence needs a stratified sweep of hundreds of checkpoints, which is future work.

How it was built

GRIT started as a dare, not a product idea: find something genuinely missing, and only build it if the gap survives an adversarial attempt to prove it already exists. Before any code, the idea was attacked with a sweep of the closest prior work — MLIR sub-channel types, StableHLO DotAlgorithm, compressed-tensors, Avro, Khronos DFS, DLPack, torchao, TOSA block-scaled types, OCP MX, IEEE P3109 — and every novelty claim that did not survive was retracted before publication. What survived was not a format but a missing contract, and that decided everything else: a checkable type, not another container.

I'll be straightforward about the method, because it's part of the story: I built GRIT in an intensive collaboration with Claude (Anthropic) — spec drafting, orchestrated implementation across the five languages, and above all adversarial verification: exact-arithmetic oracle sweeps against every encoder, a hostile three-reviewer panel run against the paper before release, a novelty sweep against everything from StableHLO to OCP MX to P3109, and a citation audit that fetched every reference against the published record. The working rules were simple: every number must regenerate from a clean checkout, and every reviewer finding is either fixed or documented as a limitation — never softened. Every number in the paper traces to a repo artifact. The motto of the whole project applies to its own construction: nothing here asks to be trusted; everything here asks to be checked.

Try it on a checkpoint you already have

# the scanner — zero adoption required
pip install grit-datatype
python3 -m grit.scan --deep --json path/to/checkpoints/

# the type, in your language of choice
cargo add grit-datatype
npm install grit-datatype

# reproduce every claim in this post from a clean checkout
bash spec/crosslang/run.sh   # 96/96 on every implementation

• GRIT on GitHub — spec (normative), paper, all five implementations, benchmarks, the field-study log
• Project site · interactive workbench — build and corrupt descriptor bytes in your browser and watch the check catch them; it runs the byte-identical npm build
• The paper — DOI 10.5281/zenodo.21817716
• PyPI · crates.io · npm — the repo tag and all three registries move in lockstep

Try the scanner on a checkpoint you already have. If it finds something I didn't predict, that's the most useful thing you could possibly tell me.

GRIT is open source under Apache-2.0, built in the open by a human–AI team — direction, constraints and the standard of evidence by the author, with Claude (Anthropic) as repository co-author. Cost figures are one-machine measurements presented as indicative; the harnesses that produced them are in the repo. The field study is n=4 and claims existence of the mismatch classes, not prevalence.