GRIT: A Grouped Reduced-precision Interchange Type with an O(1) Checkable Boundary Contract
Prateek Singh
Abstract
Block-scaled reduced-precision arrays — the MXFP4 / Q4_K / NVFP4 / sparse-INT8 family — ship in files that record element counts and dtype tags but not the contract that gives the bytes meaning: the scale-derivation rule, the zero-point convention, the sub-byte packing order, the layout of the scale planes. Producers and consumers agree on these silently, in tool source code, and in 2026 that silence produced six documented incidents of silently wrong model output across vLLM and SGLang. GRIT (Grouped Reduced-precision Interchange Type) is a descriptor language and boundary-check protocol for this family: a GRIT array is a quadruple (Grade, Placement, Planes, Shape) serializing to a fixed-layout 64-byte POD descriptor, and a consumer verifies descriptor-vs-bytes agreement in O(1) — measured at 296 ns per call in C — at every load boundary, with a total check that terminates with exactly one status on any 64 bytes. A normative specification fixes decode() semantics through an explicit law set (NaN poison, sparse-wins-over-poison, slice honesty, group-axis transpose as requantization, fixed evaluation order, no FMA substitution, placement invariance). Five zero-dependency implementations — C11, C++20, Rust, pure-stdlib Python, strict TypeScript — reproduce a 68-vector SHA-256-pinned conformance suite and agree bit-for-bit on 96/96 cross-language descriptor fingerprints, re-proven by CI on every push. A read-only scanner audits existing GGUF and safetensors checkpoints with no adoption required; a two-run field study on four real checkpoints (12 findings as first shipped, 349 after closing three inference gaps on bit-identical bytes, with a known-clean control at zero) demonstrates that the mismatch classes exist in deployed artifacts, including GPTQ and AWQ zero-point planes that are byte-identical while carrying opposite conventions.
1. Introduction
A quantized checkpoint is a contract executed without a document. The file records tensor names, shapes, element counts, and a dtype tag; it does not record which rule derived the shared scales, which zero-point convention the packed values assume, which order sub-byte elements occupy within a byte, or which layout the scale planes were written in. Producer and consumer agree on all of this silently, in code, and the file "works" exactly as long as the agreement holds. When it breaks, it breaks silently: the 2026 incident record across vLLM and SGLang — six documented incidents — includes scale tensors dropped on load, signed scales read as unsigned (roughly four orders of magnitude of dequantization error), a swizzled producer feeding a linear consumer, silent E8M0 truncation, and the GPTQ "zero point minus one" convention turning tens of thousands of zero points into NaN perplexity as late as January 2026. None of these crashed. All of them loaded cleanly and produced wrong numbers.
GRIT — the Grouped Reduced-precision Interchange Type — is designed as the smallest complete answer to one question: what is the least a checkpoint could carry that would have made those failures loud? The answer proposed here is not a new container format but a checkable type: a descriptor language for the numeric contract, a value-carried identity for the physical layout, and a total O(1) boundary check that either passes or fails with a specific status before any bytes are interpreted.
2. The Type
A GRIT array is a quadruple (Grade, Placement, Planes, Shape):
- Grade — the complete numeric contract: element format, group sizes, scale formats, scale-derivation rule, zero-point convention, sparsity, packing, and the rounding/saturation/accumulation promises — with a canonical string form and a 64-bit FNV-1a id.
- Placement — the physical layout, carried as a 64-bit value rather than a type parameter, so a new layout is a new constant. Layout-as-value is standard practice (cuBLASLt orders, XLA
LayoutProto, NVIDIA'sCUtensorMap); GRIT's contribution is that the value is hashed, registry-free, and attached to the bytes next to the numeric contract. - Planes — up to four byte buffers:
data,scale0,scale1,aux. - Shape — rank and extents, carried in the view.
The quadruple serializes to a fixed-layout 64-byte descriptor — every field at a fixed offset, little-endian, no variable-length fields:
| Offset | Field | Carries |
|---|---|---|
[0, 8) | magic · hdr · ver · levels · flags | "GRIT", header size, version, level count, flag bits (zero-point presence & convention, sparsity, bit order, interleave) |
[8, 16) | elem · scale0 · scale1 · zp | formats from a closed ladder (e2m1, e8m0, e4m3, binary16/32, u2/u4/u6/u8, s4/s8) |
[16, 22) | axis · k0 · k1 | group axis and group sizes for one- or two-level block scaling |
[22, 28) | sparse · container | n:m sparsity, metadata format/order, container packing |
[28, 32) | note | assumption provenance — excluded from every fingerprint |
[32, 56) | placement × 3 | 64-bit placement ids for the data, scale, and metadata planes |
[56, 64) | grade_id | FNV-1a-64 of the canonical grade string |
The descriptor plus the shape determines every plane's exact byte length via the specification's §7.3 size equations, so descriptor-vs-bytes agreement is decidable in O(1) at a boundary crossing. The grade, hashed frame, and shape fingerprint together into a 128-bit gid (FNV-1a-128): two tools holding the same gid hold the same contract. Two guarantees are normative and testable: (i) equal gid and equal plane bytes imply bit-identical decode() on every conformant implementation; (ii) grit_check has no undefined behaviour on any 64-byte input, adversarial inputs included — every input terminates with exactly one status and no reads outside the buffer.
3. The Contract Vocabulary
The Grade names fields that current formats leave implicit:
esr— the shared-scale derivation rule, with no default, always emitted.exp2floormaxabsvsexp2ceilmaxabsvsexp2rnemaxabsvsratiomaxabsis a real fork: a PyTorch maintainer records in pytorch#146414 that there is no consensus and that the OCP floor rule differs from a vendor's modified ceiling rule. It is also the field the nearest neighbour drops — torchao has aScaleCalculationModeand does not serialize it. GRIT's rne rule is specified to the bit —e_floor + (f·f > 2 ? 1 : 0)withf = amax·2^−e_floorexact in binary64 — and unit-tested in all five implementations.- Explicit zero-point conventions —
zpc(asisvsminus1) andzps(sharedvsl1-scaled) are declared flags, not folklore about which tool wrote the file. - Exact format identity —
e4m3ande4m3fnuzare different formats; spelling one as the other is a parse error rather than a silent alias. - Sparsity metadata order —
SPORDER_ASC_LSBis a declared, checkable precondition instead of the undefined behaviour PTX leaves it as. - Layout, separately — a placement value, so "which layout" and "which numeric contract" stop being one fused enum name like
marlin_24; the capability vllm#8913 has requested since 2024.
4. The decode() Law Set
decode() is the normative semantics: two GRIT arrays are semantically equal iff their decodes are equal, and everything else in the specification exists to make that statement checkable at an O(1) boundary. The laws constrain what a decoder is allowed to do — the location of silent divergence that format documentation does not reach: NaN poison (a poisoned scale poisons its group), sparse-wins-over-poison (a structurally-zero element is zero even under a poisoned scale), slice honesty (a slice decodes as the slice of the decode), group-axis transpose as requantization (never a view), fixed outermost-first evaluation order, no FMA substitution, and placement invariance (decode is independent of physical layout). StableHLO's 0 < scales constraint affirmatively excludes the hardest of these cases, so the two semantics are disjoint precisely on the edges where implementations diverge in practice.
5. grit scan and the Field Study
Adoption of a new descriptor cannot be assumed, so the deployable artifact is a read-only auditor over existing containers. grit scan recovers declared descriptors from a __grit__ safetensors header key or grit.desc.<tensor> GGUF KV entries, and where no descriptor exists — today, everywhere — infers one from well-known ggml block structs, llama.cpp file types, and the GPTQ / AWQ / compressed-tensors safetensors families, labeling every inferred contract as inferred, never declared. It reports descriptor-vs-bytes violations against the §7.3 size equations, declared-vs-actual disagreement (file_type_mismatch), convention ambiguity (zp_convention_ambiguous), cross-file gid disagreement with a field-by-field diff, grade-drift clusters, and honest inexpressibility (inferred_inexpressible). Exit codes are graded and CI-gateable: 0 clean, 1 structural violation, 2 disagreement, 3 warnings only, 4 unparseable.
The field study is reported with a two-run structure on four real, popular checkpoints (GGUF Q4_K_M, GPTQ-Int4, AWQ-Int4, compressed-tensors W4A16), read byte by byte. Run 1 — hand analysis plus the scanner as first shipped — confirmed real mismatches in three of four files but auto-caught only one, indirectly. Exactly three inference gaps were closed and the scan re-run on bit-identical bytes: Run 2 produced 349 findings where Run 1 produced 12, with every mismatch class firing automatically, while a false-positive control on two known-clean files remained at zero findings, exit 0. The complete experiment log, both runs, is audit/scan-experiment.md. The single most instructive finding: a GPTQ file and an AWQ file whose zero-point planes are identical in byte count, shape, dtype, and tensor name while carrying opposite conventions (zpc=minus1 vs zpc=asis) — loading one as the other shifts every weight by a full quantization step, and nothing in either file records which convention is inside. The study is n=4: it establishes existence of the mismatch classes in deployed artifacts, not prevalence.
6. Verification
One normative specification (2,164 lines) and an executable Python reference anchor five zero-dependency implementations. Every number below is CI-enforced on each push and reproducible from a clean checkout via REPRODUCING.md:
| Suite | Result |
|---|---|
| Shared conformance vectors (SHA-256-pinned) | 68 entries — 34 positive · 15 negative · 16 fingerprint · 3 supersedes |
| C (gcc, C11) | 910 checks, 0 failures |
| C++ (g++, C++20; plain + strict + ASan) | 801 checks, 0 failures |
Rust (stable, no unsafe in the hot path) | 37 tests + conformance suite |
| Python (stdlib only; numpy never imported) | 124 tests |
| TypeScript (strict, Node ≥ 18) | 91 tests |
Cross-language gid harness (spec/crosslang/run.sh) | 96/96 descriptors identical across all five + the reference; 96 distinct gids |
7. Cost
Measured on one i9-13900HK (indicative, not guaranteed; harnesses in bench/, methodology in paper §8.4): the level-1 structural check costs 296 ns per call in C (887 cycles), 687 ns in Rust, and 9.8 µs in Python; the level-2 content digest runs at 0.46 GB/s (C), 1.24 GB/s (Rust), and 0.017 GB/s (Python). Checking every tensor of a synthetic 8-shard, 1.07 GB MXFP4 checkpoint at load time adds +33.4 KB of headers, +2.7 ms of wall clock, and +108 KiB RSS for 192 checks.
8. Related Work and Non-Claims
The specification carries a component-by-component claims table with the closest prior art for each component. Parameterised quantized types, nested two-level scales, and scale-plus-sparsity in one format are prior art (MLIR UniformQuantizedSubChannelType, compressed-tensors, Qualcomm LPBQ). The arithmetic contract as a first-class value is deployed art (StableHLO / JAX DotAlgorithm), and the refinement structure is textbook graded modal typing under the same word Grade (Granule, ICFP 2019). Canonicalise-then-fingerprint for registry-free cross-language identity is Apache Avro's discipline step for step; GRIT changes the object being hashed. The portable POD descriptor pattern is the Khronos Data Format Specification's and DLPack's, including the identical LSB-first sub-byte packing rule. FNV-1a fingerprints defend against drift and mislabeling, not against an adversary — no collision resistance is claimed, and consumers must treat a gid match as a spurious-match risk bounded only under non-adversarial assumptions.
9. Limitations
The v1 grammar has named coverage holes: AMD FNUZ FP8 is inexpressible (bias_ovr must be 0 in v1); there is no codebook/LUT element class, so NF4 and the llama.cpp IQ* families are out of scope; GPTQ act-order g_idx grouping is unsupported — contiguous grouping is structural, and act-order cannot be hidden in Placement without falsifying the placement-invariance law; and dense-nibble zero-point planes (GPTQ/AWQ qzeros, two values per byte) are not byte-representable under the v1 CT_PAD8-always rule. The scanner names all of these inferred_inexpressible rather than approximating. The field study is n=4 and claims existence, not prevalence; a prevalence claim requires a stratified sweep of hundreds of checkpoints, which is future work.
10. Availability
The paper is archived on Zenodo (DOI 10.5281/zenodo.21817716). The normative specification, the executable reference, all five implementations, the conformance vectors, the benchmark harnesses, and the field-study log are at github.com/singhpratech/grit-datatype (Apache-2.0), with an interactive descriptor workbench at singhpratech.github.io/grit-datatype running the byte-identical npm build in the browser. Packages: pip install grit-datatype · cargo add grit-datatype · npm install grit-datatype — the repository tag and all three registries move in lockstep. The repository is built in the open by a human–AI team: direction, constraints, and the standard of evidence by the author, with Claude (Anthropic) as repository co-author; every number regenerates from a clean checkout, and every reviewer finding is either fixed or documented as a limitation.