Apache Arrow IPC vs JSON: The Numbers Behind the Switch

Why this comparison keeps coming up

There's a discovery moment that happens in roughly the eighteenth month of every analytical platform's life. A backend engineer profiles the slow endpoint — the bulk export, the model-features fetch, the dashboard backend — and finds the network is fine, the database is fine, the disk is fine. CPU is pegged at 100% and the flame graph is a single tall column. The column says json.dumps. The endpoint is serializing a million-row Pandas frame into a list-of-dicts and the encoder is paying twelve bytes of quote-comma-colon overhead for every typed scalar it touches.

That moment is when teams discover Apache Arrow IPC. Not because Arrow is fashionable — it has been around since 2016 — but because the part of the bill they had quietly written off as "the cost of doing JSON" turns out to be 30-80% of total request time on any columnar payload. As one frequently-cited Arrow Flight survey puts it, "more than 80% of the total time spent in accessing data is elapsed in serialization/de-serialization" with traditional formats like JSON (IJIRSET, 2023). That's the part everyone undercounts.

The interesting question in 2025 isn't whether Arrow IPC is faster — it is, by a wide margin, on the workloads it's designed for. The interesting question is when the switch is worth doing and when JSON is still the right answer.

The numbers

The cleanest public numbers come from the Arrow Flight benchmark study by Ahmad et al. (arXiv 2204.03032). On a Mellanox ConnectX-3 / Connect-IB interconnect, single-node Flight transfers reach ~6000 MB/s on DoGet and ~4800 MB/s on DoPut, using roughly 95% of available network bandwidth. On a remote-host setup over a more typical datacenter network the peak drops to ~2000 MB/s on DoGet and ~1650 MB/s on DoPut with 16 parallel streams. Those are not JSON-shaped numbers — they're wire-speed-shaped numbers.

End-to-end matters more than peak throughput, though, because most service calls are encode/decode-bound, not wire-bound. Dremio's own measurements show ~20× speedup over turbodbc and ~30× over ODBC when Arrow Flight replaces the database client protocol (Israeli Tech Radar, 2023, citing Dremio's published Flight benchmarks). The wire is identical. The only thing that changed is that rows stop being transposed into one-tuple-at-a-time encoded blobs and start moving as already-laid-out columnar batches.

Read the chart honestly. The Arrow Flight number at the bottom is a real measured wire-speed figure on a specific interconnect. The JSON serialize/deserialize bars and the in-process Arrow IPC bars are benchmark-shaped — they describe a typed columnar payload with no deeply nested objects, which is the shape Arrow is designed for and the shape JSON is worst at. On a heterogeneous payload of small nested objects, JSON's encoder is much more competitive and the gap narrows dramatically. Don't quote these numbers for a payload that doesn't look like a Pandas frame.

Where JSON still wins

The honest list of workloads where switching to Arrow IPC is a mistake:

• Small payloads. Under a few kilobytes, Arrow's per-message framing overhead (schema, record-batch metadata, dictionary buffers if any) is a meaningful fraction of total bytes. A 200-byte JSON response will round-trip faster than the equivalent Arrow IPC message because JSON has zero per-message setup.
• Heterogeneous nested data. Arrow is a columnar format. It can encode nested structs, lists, and maps, but the encoding is awkward and the wins shrink when most of your payload is shaped like a tree rather than a table. Configuration blobs, user-profile objects, mixed-shape API responses — JSON stays cleaner.
• Human debuggability. A developer can curl | jq a JSON endpoint and read the response. Arrow IPC requires a decoder and at minimum a quick PyArrow call to inspect. For internal APIs where humans regularly look at the wire, that friction has a real cost.
• Schema-on-read flexibility. JSON is forgiving. You can add a field, drop a field, change a type, and most clients keep working. Arrow batches carry a schema, version skew between producer and consumer is a real concern, and the failure mode is louder.
• Browser fetch with no WASM. Native fetch(...).json() is a one-liner in every browser. Arrow IPC in a browser requires apache-arrow (the JS package works, but it's another 100-200 KB of bundle) or a WASM decoder. For public-internet JS clients, JSON is structurally the simpler answer.

The pattern is consistent: when the payload is small, irregular, or read by humans, JSON's overhead is amortized by its flexibility. When the payload is large, typed, and read by another service, Arrow IPC's discipline pays back many times over.

What the Arrow ecosystem ate in 2024-2025

The reason this comparison feels different in 2025 than it did in 2022 is that the Arrow story stopped being "the in-memory columnar format used by Pandas and Spark." Three concrete shifts happened in the past eighteen months:

Arrow Flight and Flight SQL became real database transports. InfluxDB IOx uses Arrow as its in-memory format, Apache Parquet as its persistence format, and Arrow Flight for RPC (per the Apache Arrow Powered By page). Dremio ships an Arrow Flight SQL JDBC driver and contributed it to the Arrow community in November 2022 (Dremio blog). Hopsworks moved its Python feature service to a DuckDB-on-Arrow-Flight architecture for the same reason: the JDBC/ODBC wire was the bottleneck, not the engine.

ADBC went mainstream. Arrow Database Connectivity is the columnar successor to JDBC/ODBC, and as of 2025 it has been adopted by Snowflake (Snowflake Engineering, 2024), BigQuery, DuckDB, Databricks, and PowerBI (per Voltron Data's adoption recap). The pitch is straightforward: a database driver that returns Arrow batches instead of row-by-row tuples skips the encoding tax entirely.

The engines under your engines speak Arrow natively. Polars, DataFusion, and DuckDB all use Arrow as the canonical in-memory representation, which means moving data between them is a buffer handoff, not a copy. DuckDB's zero-copy Arrow integration has been GA since late 2021, and as of May 2025 DuckDB ships an official Arrow IPC community extension that reads and writes .arrows files (and Arrow IPC HTTP streams) directly via the lightweight nanoarrow library — no Arrow C++ dependency required.

nanoarrow is making Arrow viable in embedded contexts. The full Arrow C++ library is large (tens of MB), which historically kept it out of mobile, embedded, and lightweight-extension contexts. nanoarrow 0.6.0 (Oct 2024) is a few hundred KB of C, implements the Arrow C Data and IPC interfaces, and is what DuckDB's Arrow extension was built on. The footprint problem that kept Arrow out of edge use cases is mostly solved.

The migration story most teams skip

Switching the wire format is the easy part. Python, Node, Rust, Go, Java, and C++ all expose Arrow IPC reads and writes in five lines of code. Anybody can produce a record batch and POST it to anybody else. That's not where the work is.

The work is in the things JSON let you ignore:

• Schema definition. JSON's shape is implicit; you discover it when a field goes missing. Arrow batches carry an explicit schema, which means somebody on your team owns deciding what the schema is, where the canonical definition lives, and how it changes. Protobuf and Thrift have solved this — you just have to actually solve it now instead of pretending it doesn't exist.
• Client library coverage. Arrow has first-class libraries in Python, JS, Rust, Go, Java, C++, and R. Coverage in the long tail (Ruby, Elixir, PHP, niche languages) is thinner. If your consumer fleet includes one of those, check before you switch.
• Version skew. Arrow's schema-evolution rules are stricter than JSON's "just add a field." A producer that emits a new column will break a strict consumer. Plan for explicit versioning and a deprecation lane.
• Tooling and observability. Your existing logging and replay tools probably pretty-print JSON. They almost certainly don't render Arrow batches. A small arrow-to-jsonl tap that materializes the first N rows of any captured batch is a one-afternoon job and saves many days of confused debugging.

None of this is hard. It's the work JSON let you skip — and the team that doesn't budget for it spends three weeks rediscovering Protobuf-era lessons the hard way.

What to do on Monday

If you serve a /data/export-style endpoint to analytical consumers — internal dashboards, notebooks, ML pipelines, anything that pulls a typed columnar slice — the highest-leverage move you can make this week is to add Arrow IPC as a content-negotiated option alongside JSON. Keep JSON as the default for human/browser callers. Let machine callers opt in with Accept: application/vnd.apache.arrow.stream.

The conceptual shape in FastAPI:

from fastapi import FastAPI, Request, Response
import pyarrow as pa
import pyarrow.ipc as ipc
import io, json

app = FastAPI()

@app.get("/data/export")
def export(request: Request):
    df = build_dataframe()  # returns a pyarrow.Table
    accept = request.headers.get("accept", "")
    if "application/vnd.apache.arrow.stream" in accept:
        sink = io.BytesIO()
        with ipc.new_stream(sink, df.schema) as writer:
            writer.write_table(df)
        return Response(sink.getvalue(),
                        media_type="application/vnd.apache.arrow.stream")
    return Response(df.to_pandas().to_json(orient="records"),
                    media_type="application/json")

Same shape in Node/Express with apache-arrow's RecordBatchStreamWriter. The Arrow path adds maybe 15 lines, doesn't touch the JSON path, and lets notebook consumers cut their fetch time substantially with a one-line change to their request header. Most teams that do this find that within a month the Arrow path serves the majority of bytes — loud internal consumers switch immediately, long-tail integrations keep using JSON. That's the right outcome.

The broader lesson: JSON did not become dominant because it was a good binary format. It became dominant because it was readable, ubiquitous, and good enough when bandwidth was the bottleneck. Both of those conditions have flipped on typed-columnar workloads. Arrow IPC is what the field looks like once the encoding tax stops being free.