We Published Our 110× Loss. One Release Later, It Was Gone.

On Sunday I typed a sentence into my own release notes that most maintainers would call self-sabotage:

"Measured honestly: on v0.2.0 the native CPU pipeline wins every end-to-end query shape tested, by ~3× to ~110×."

The project is gpudb — my GPU-accelerated DuckDB extension, the one that runs SQL aggregates on NVIDIA CUDA and, uniquely, on Apple Silicon Metal. The sentence went into BENCHMARK.md next to a table where my extension loses every single row to plain DuckDB. Not by a little. The worst cell was 109× slower.

About twenty-one hours later, v0.3.0 shipped, and every one of those cells now sits within 0–20% of native. The 11.05-second GROUP BY runs in 0.109 seconds. Same queries, same machine, same methodology, identical results.

This is the story of both halves: how a "GPU database" ended up 109× slower than the CPU it was supposed to beat, and why the honest write-up of that loss turned out to be the fastest path to fixing it. It ends somewhere I did not expect when I started this project in May: with me deleting the GPU from the hot path — on purpose, and the benchmark getting two orders of magnitude faster because of it.

The question I didn't want to answer

Some context. gpudb is a DuckDB extension, not a fork — you LOAD it into the stock CLI and get gpu_sum / gpu_min / gpu_max aggregates backed by CUDA on Linux and Metal on Apple Silicon. Its operator-level numbers are genuinely good: the fused multi-aggregate kernel reads a TPC-H SF10 column once and computes SUM, MIN, MAX and COUNT in 1.13 ms on an M4 Max — about 25× faster than DuckDB's 16-thread CPU path on the same column. Those numbers are real, reproducible, and in the repo.

In May I submitted gpudb to the DuckDB community-extensions registry (PR #1898). During review, someone asked the question every GPU project quietly dreads, and it's a completely fair one:

Your scorecard measures the aggregate operator on data that's already resident. What does a user actually see when they rewrite a whole query around gpu_sum and run it through the full DuckDB pipeline — scan, filter, project, aggregate?

I could have answered with the operator numbers again. Instead I built the end-to-end suite: rewritten TPC-H Q6 and Q1, plus a high-cardinality GROUP BY, through the stock DuckDB CLI v1.5.2, 16 threads, warm cache, median of five runs, correctness gated before any timing counted (the Q6 revenue total matches the published TPC-H reference answer to the printed digit). Both sides cast to DOUBLE identically. Apples to apples, no escape hatches.

The numbers we published

Native won everything.

Query shape	native	gpu_sum (v0.2.0)	native advantage
Q6 · SF1 — filter → scalar SUM	0.002 s	0.006 s	~3×
Q6 · SF10	0.017 s	0.057 s	~3.4×
Q1 · SF1 — 4-group agg, 3 SUMs + count	0.011 s	0.33 s	~30×
Q1 · SF10	0.098 s	3.45 s	~35×
GROUP BY l_orderkey · SF1 (1.5M groups)	0.010 s	0.944 s	~86×
GROUP BY l_orderkey · SF10 (15M groups)	0.092 s	11.05 s	~109×

And then — this is the part people found surprising — we published it. Not in a private issue. In BENCHMARK.md and the v0.2.0 release notes, under the heading "honest: native CPU wins." The conclusion was stated plainly: in v0.2.0, gpu_sum is not a way to make whole queries faster.

Where 109× actually came from

Here's the detail that makes this interesting rather than merely embarrassing: the SF10 GROUP BY row that loses 109× end-to-end and the operator scorecard row where Metal wins 1.30× are the same data and the same kernel. The ~140× swing between those two measurements is entirely the extension's execution path. The kernel was never the problem.

Writing the honest benchmark section forced the diagnosis. Four things, in order of damage:

1. Buffering, twice. DuckDB drives aggregates through per-chunk update callbacks. The v0.2.0 extension appended every incoming value into a BufferPool state — a full serialized copy of the column — before the GPU ever saw a byte. Then finalize snapshotted that buffered state into another vector. The operator benchmarks skip all of this because their data is already resident. The copy itself was the cost, not the reduction.

2. A global mutex, and 15 million finalizes. Every GPU dispatch serialized on one lock, and grouped aggregation finalized per-state — one tiny reduction per group. At SF10 that's 15 million dispatch decisions to sum an average of four values each.

3. On Metal, DOUBLE never reached the GPU at all. Apple GPUs don't implement IEEE-754 doubles, so sum_f64 falls back to a host loop. Which means the Q6/Q1 numbers on this machine were: full buffering overhead, zero GPU upside.

4. Q1's four-group regime is a structural CPU win anyway. A hash table with four entries lives in L1. No accelerator fixes that; the benchmark section had already documented it.

Notice what this list is: it's an argument that the loss was structural, not a tuning problem. No amount of kernel work would have saved a path whose defining operation is copying the column twice before doing any math. That prediction — written down in the v0.2.0 notes — is what v0.3.0 tested.

The fix was deleting the GPU

v0.3.0 rewrote the SQL aggregate path from "buffer every value, reduce at finalize" to streaming running accumulators — the same algorithmic shape as a native DuckDB aggregate. Each state is a 24-byte plain-old-data struct. A chunk arrives, the accumulator advances, finalize returns the value. The BufferPool is gone. The global mutex is gone. The batched GPU finalize path is gone.

Read that again, because it's the counter-intuitive core of this whole story: the SQL aggregate path in a GPU database extension no longer dispatches to the GPU — and that's why it got 100× faster.

The reason is unified memory. On Apple Silicon, the CPU and GPU read the same LPDDR5X at the same bandwidth. There is no discrete-GPU world where you eat a transfer once and then compute from faster memory — the memory is the memory. So for a single streaming pass like a SUM, shipping the column to the GPU buys you nothing and costs you everything: the staging copy, the dispatch latency, the synchronization. The GPU earns its keep where there's real parallel work per byte — fused multi-aggregates, high-cardinality grouping at operator level, and above all joins — not in a per-value accumulate driven through a callback interface.

One release later

The v0.3.0 record run repeats the exact v0.2.0 suite — same queries, same methodology, same machine, correctness gated first:

Query shape	native	v0.3.0	ratio	was (v0.2.0)
Q6 · SF1	0.002 s	0.002 s	1.00×	~3×
Q6 · SF10	0.017 s	0.018 s	1.06×	~3.4×
Q1 · SF1	0.011 s	0.012 s	1.09×	~30×
Q1 · SF10	0.098 s	0.101 s	1.03×	~35×
GROUP BY · SF1	0.010 s	0.012 s	1.20×	~86×
GROUP BY · SF10	0.092 s	0.109 s	1.18×	~109×

Every cell within 0–20% of native; the worst is the SF1 GROUP BY at 1.20×. And the same honesty rules still apply, so three residuals are recorded rather than hidden:

• The remaining 0–20% has not been separately profiled. Plausible contributors: the extension's defensive input probes, callback indirection, and native's fused specializations. Small enough that gpu_* aggregates are no longer a footgun in real queries; not zero, and written down as such.
• The brand-new gpu_min(DOUBLE) / gpu_max(DOUBLE) overloads run ~1.5× slower than native min/max. Their value in this release is that they exist at all (no prior version had them) — correct-but-slower, recorded honestly.
• Simpler won on safety too: with the BufferPool and mutex machinery deleted, the state is a 24-byte POD, all 96 unit checks pass, and the SQL suite runs clean — the previously expected-fail window-function cases are now strict positive assertions.

So is the GPU pointless? No — it moved

The operator-level tools still drive both backends hard, and that's where the honest wins live: 25.5× on fused multi-aggregates (M4 Max), 17.9× on resident SUM (RTX 4090), 3–4× on billion-row GROUP BYs. What changed is the answer to "where does the GPU belong in the SQL path?"

The answer is joins. A join has real parallel work per byte — build, probe, high selectivity fan-out — the profile where an accelerator's arithmetic actually outruns the memory system. And here the project got lucky in the way healthy open-source projects get lucky: a contributor, @lmangani, showed up with a real Metal hash join plus on-device segment reduce (PR #43), verified at 9.9× on a 1M×10M inner join on M4 Max. Landing and extending that stack is the v0.4.0 headline.

Meanwhile community-extensions PR #1898 continues through review — once it merges, INSTALL gpudb FROM community works from the stock DuckDB CLI with no -unsigned flag.

Why you should publish your losses

Here's the accounting on that one embarrassing paragraph, roughly one day later:

It produced the fix. Writing "measured honestly: native wins every shape" forced the four-part diagnosis, and the diagnosis predicted the fix so precisely that v0.3.0 was mostly a matter of typing it in. If I had answered the reviewer with operator numbers and moved on, the 11-second GROUP BY would still be shipping today.

It made the good numbers believable. A benchmark file that contains "native wins 109×, here's why" earns the right to also say "the fused kernel wins 25×" and be believed. Benchmarks are a trust exercise; the loss column is what makes the win column credible.

It cost nothing. The feared outcome of publishing a bad number — ridicule, lost users — did not materialize. What materialized was a better architecture, a contributor bringing a 9.9× join, and this post.

The whole episode fits in one sentence, and it's the sentence I'd put on a poster for anyone building performance-critical open source: the benchmark section where my project loses 110× has done more for the project than every section where it wins.

Everything here is reproducible from the repo — BENCHMARK.md has the full methodology, both record runs, and the scripts. If you want the deeper backstory: why gpudb exists and what I learned writing the kernels.