DuckDB Eats the Cloud: Local Analytics Outperform BigQuery and Athena

A financial data platform engineer in France just dropped benchmark results that should make every cloud vendor executive sweat. On a 20GB dataset of time-series data, DuckDB running on a single server crushed both BigQuery and Athena, delivering queries 3-10x faster at essentially zero marginal cost. The cloud’s supposedly "serverless" advantage evaporated in a puff of local SSD smoke.

But here’s where it gets spicy: the benchmark’s most vocal critics aren’t wrong. They’re just furious that someone dared to expose the uncomfortable truth about how much businesses overpay for distributed infrastructure they never actually use.

The Numbers That Started the Fight

The setup was straightforward: 57 queries across ~20GB of Parquet files (ZSTD compressed), representing typical analytics workloads, scans, aggregations, joins, window functions, plus 15 wide-scan SELECT * queries. Four runs per query, measuring both cold and warm performance.

M = 8 threads, 16GB RAM | XL = 32 threads, 64GB RAM

The hardware wasn’t exotic: an AMD EPYC 9224 with 256GB RAM and a Samsung 870 EVO SATA SSD. Total cost? Roughly €500/month for two dedicated Hetzner servers running the platform. Compare that to BigQuery’s $0.0282 per query, and the break-even math gets interesting fast.

Why DuckDB Wins on Speed (And It’s Not Just the SSD)

DuckDB’s vectorized execution engine processes data in batches optimized for CPU cache locality. Combine that with zero network latency, and you get sub-second median query times that cloud platforms simply can’t match, even with their massive distributed infrastructure.

The scaling behavior tells the real story:

Doubling resources roughly halves latency. Going from 4 to 32 threads (8x) improved performance by 5x. Not perfectly linear, but predictable enough for capacity planning, something cloud "serverless" promises but rarely delivers at small scales.

Performance by query category reveals even more:

Notice how window functions absolutely destroy DuckDB + R2 performance? That’s the network penalty hitting hard, 12 seconds vs 947 ms local. But even with that handicap, DuckDB + R2 still beats Athena on everything except window functions.

The Cost Reality Check That Makes Architects Uncomfortable

Here’s where benchmarks meet business reality. One commenter ran the numbers: that €500/month Hetzner setup equals roughly $7500 in upfront hardware costs or $1500/month on EC2. To match that with cloud queries, you’d need:

• ~1,700 BigQuery queries per day
• ~13,500 narrow Athena queries per day

Most companies aren’t running that volume. The median data team runs a few hundred queries daily, making the fixed-cost local setup dramatically cheaper.

But the real cost killer isn’t query pricing, it’s egress. The benchmark highlighted Cloudflare R2’s zero egress fees versus AWS’s $0.09/GB. For a scenario of 1000 queries/day pulling 5GB each:

• S3 egress: $450/day = $13,500/month
• R2 egress: $0/month

That’s not a rounding error. That’s a business model.

The Controversy: Apples to Orangutans?

The backlash was immediate and technical. Critics correctly pointed out that comparing an embedded database to serverless warehouses is fundamentally flawed. One commenter called it "as stupid as it gets", arguing you’re paying for convenience, not just compute.

They’re right. DuckDB abstracts nothing, you manage everything. BigQuery and Athena abstract infrastructure, scaling, redundancy, security patching, and 3 AM pager alerts. The benchmark simulates 100% utilization, the absolute worst case for serverless billing. Real workloads have peaks and valleys where serverless shines.

The counterargument from the benchmark author is equally valid: "Most companies aren’t running petabyte workloads." For sub-TB data, which is faster and cheaper? The analysis shows that for warm queries, local DuckDB delivers better latency while staying under cloud per-query pricing.

This is the architectural tension that makes the debate valuable. It’s not about which tool is "better", it’s about honest cost accounting for your actual scale.

Cold Start Penalties: The Hidden User Experience Killer

The benchmark exposed a critical but often overlooked metric: cold start overhead.

That 14-20 second cold start for DuckDB + R2 is brutal for sporadic queries. The first run has to fetch Parquet metadata (file footers, schema, row group statistics) over the network. Subsequent queries are fast because metadata caches, but that initial penalty dominates user experience.

BigQuery and Athena’s minimal cold start overhead (~0-2%) is their superpower for intermittent workloads. You pay for that convenience, but it’s not marketing fluff, it’s real engineering.

Wide Scans: Where Cloud Cost Models Collapse

The benchmark added 15 SELECT * queries to simulate data exports and ML pipelines. This changed everything:

Athena’s cost advantage comes from column pruning, reading only referenced columns. When you SELECT *, there’s nothing to prune. Costs converge toward BigQuery’s level, erasing Athena’s primary selling point.

This is the kind of real-world insight that benchmarks should surface but rarely do. Your architecture decisions must account for query patterns, not just average case scenarios.

The Architectural Implications Nobody Wants to Admit

This benchmark reveals a broader pattern: the trend toward single-binary, simple backends that eliminate cloud complexity, like DuckDB does for analytics. The industry’s obsession with distributed systems is hitting a wall where operational overhead exceeds value.

For AI workloads, local compute disrupting cloud-based AI workloads, similar to how DuckDB challenges cloud analytics shows the same pattern. A $599 Mac Mini M4 running local models is eating cloud AI’s lunch for the same reason DuckDB wins, data locality and avoiding network tax.

The comparison to the hidden costs and architectural pitfalls of supposedly ‘easy’ cloud databases, reinforcing DuckDB’s cost and simplicity advantages is stark. That $21k DynamoDB bill didn’t come from heavy usage, it came from architectural assumptions that serverless means "no thinking required."

When to Use What: A Decision Framework

Based on the data, here’s a practical guide:

Use DuckDB Local when:
– Sub-second latency is non-negotiable (5-8x faster than cloud)
– You have sustained query volume (>1000/day) to amortize fixed costs
– Data fits on local SSDs (<1TB hot data)
– You can manage infrastructure and want control

Use DuckDB + R2 when:
– You need free egress (R2’s $0 egress vs S3’s $0.09/GB)
– Warm query performance matters more than cold starts
– Data exceeds local storage but queries are predictable
– You’re building a managed platform and can cache metadata

Use BigQuery when:
– You’re deep in GCP ecosystem (integration convenience)
– Sporadic cold queries dominate (minimal cold start penalty)
– You need automatic scaling beyond single-machine limits
– The convenience premium is worth the 3-10x latency hit

Use Athena when:
– You’re in AWS ecosystem with data already in S3
– Queries are highly selective (column pruning saves real money)
– Wide scans are rare (cost advantage disappears with SELECT *)
– Query volume is low enough that per-query billing makes sense

The Real Takeaway: Question Your Defaults

The most valuable insight isn’t that DuckDB is "better", it’s that questioning architectural overengineering and applying rational cost-benefit analysis, relevant when evaluating cloud warehouse spending is now mandatory. Most teams default to cloud warehouses because "that’s what you do", not because they’ve done the math for their actual scale.

The benchmark author is building a financial data platform with ~300GB on R2 and two Hetzner servers. Their early-stage optimization for latency and per-query cost makes perfect sense. At €500/month fixed cost, they’re delivering better warm query performance than BQ/Athena while staying under cloud per-query pricing.

That’s not "stupid." That’s rational engineering.

The Code You Need to Run This Yourself

The full benchmark suite is available on GitHub. Here’s how to reproduce the DuckDB local tests:

# Prerequisites
pip install duckdb pandas google-cloud-bigquery boto3

# DuckDB Local benchmark
git clone https://github.com/Insydia-Studio/benchmark-duckdb-athena-bigquery
 cd benchmark-duckdb-athena-bigquery
python scripts/run_duckdb_local_benchmark.py

The repository includes:
– 672 DuckDB local query runs
– 672 DuckDB + R2 query runs
– 168 BigQuery runs
– 168 Athena runs
– Wide scan benchmarks
– Full analysis scripts

Final Verdict: The Cloud Isn’t Dead, But Your Bill Might Be

DuckDB isn’t going to replace BigQuery for petabyte-scale analytics. That’s not the point. The point is that a huge swath of business analytics, sub-TB datasets, sustained query loads, latency-sensitive applications, are massively overpaying for distributed infrastructure they never utilize.

The benchmark proves that for this common scenario, local analytics delivers 3-10x better performance at a fraction of the cost. The controversy proves that challenging cloud orthodoxy triggers defensive reactions from people whose architectures depend on it.

Both sides are right. Use the right tool for your actual scale, not the scale in your vendor’s marketing deck. And maybe, just maybe, stop paying $0.09/GB for egress when Cloudflare will charge you zero.

The data doesn’t lie. Your architecture decisions might.