Two sets of measurements, both from Hardcharger™’s repository (bench/,
docs/benchmarks.md),
both against GizmoSQL, Hardcharger’s predecessor, with the same data. Numbers are medians. Every
run checked its results (row counts and column sums), so a fast wrong answer can’t count.
Over a real network: two AWS c8gn.8xlarge machines, TLS on both servers, median of five runs:
Workload
Hardcharger
GizmoSQL 1.42
Fetch SF10 lineitem (60M rows, 10.5 GB of Arrow)
0.97 s
15.2 s
15.7× faster
Fetch SF1 lineitem (6M rows)
0.148 s
1.74 s
11.7× faster
Fetch 100M narrow rows
0.45 s
5.43 s
12.0× faster
Fetch SF1 lineitem across regions (46 ms away)
1.96 s
12.26 s
6.3× faster
Load SF10 lineitem (bulk ingest, 8 upload parts)
3.00 s
9.44 s
3.1× faster
Load SF1 lineitem across regions (8 upload parts)
2.91 s
13.94 s
4.8× faster
The network fetches above read the result in partitions (32 on the LAN, 8 in order across
regions) and the loads send 8 parts at once; the performance model
explains why that matters.
Queries derived from TPC-H at SF10, on one machine
Same machine, same data, each server on loopback with no TLS. Each server loads Parquet files of
the TPC-H-derived data at SF10 (lineitem: 59,986,052 rows) into its own DuckDB file through init
SQL. Every query runs once to warm up, then five times; the figure is the median, measured at the
client from submit to the last row in a pyarrow.Table. Hardcharger is called over Warp SQL’s
one-round-trip path; GizmoSQL 1.41.0 through adbc-driver-gizmosql 2.0.15. Script:
bench/tpch.py.
Machine: Apple M5 Max, 18 cores, 64 GB, macOS 27. Hardcharger on DuckDB v2.0.0-alpha48624,
GizmoSQL on DuckDB v1.5.6. “DuckDB 1.5.6 in-process” is the same database file queried from the
duckdb Python package, with no server at all.
Query
Hardcharger (ms)
GizmoSQL 1.41 (ms)
Speedup
DuckDB 1.5.6 in-process (ms)
Q01
41.9
96.0
2.29×
85.3
Q02
8.2
20.1
2.45×
16.1
Q03
28.5
43.4
1.52×
36.9
Q04
27.2
40.7
1.50×
36.5
Q05
32.1
38.1
1.19×
30.9
Q06
11.2
14.8
1.32×
11.1
Q07
27.1
39.9
1.47×
27.3
Q08
29.9
39.4
1.32×
30.0
Q09
99.0
102.4
1.03×
96.5
Q10
64.1
74.6
1.16×
69.1
Q11
6.1
10.0
1.64×
5.2
Q12
26.3
34.6
1.32×
30.1
Q13
116.7
133.5
1.14×
126.2
Q14
23.8
28.9
1.21×
23.3
Q15
16.8
22.2
1.32×
17.9
Q16
29.1
33.5
1.15×
27.7
Q17
28.7
34.6
1.21×
30.0
Q18
83.4
98.9
1.19×
90.8
Q19
22.0
54.1
2.46×
48.2
Q20
29.6
33.9
1.15×
26.3
Q21
60.4
144.8
2.40×
125.8
Q22
20.4
62.8
3.08×
25.1
Total
832.5
1,201.2
1.44×
1,016.3
Bulk transfer, SELECT * FROM lineitem LIMIT 10000000 (1.73 GB of Arrow), median of 3:
Seconds
Hardcharger
0.83
Hardcharger, Warp-IPC-Compression: zstd
0.96
GizmoSQL 1.41
1.32
DuckDB 1.5.6 in-process
0.88
Every query returned the same row count on both servers (Q11 returns no rows at SF10 because the
query text carries SF1’s fraction; it does on both).
Part of the gap is the engine: Hardcharger beats even in-process DuckDB 1.5.6 (832 against
1,016 ms), so DuckDB 2.0 is faster on these queries before any protocol is involved (Q01, Q19
and Q21 most of all).
GizmoSQL adds about 185 ms over in-process DuckDB across the 22 queries (about 8 ms a query:
gRPC, Flight’s two round trips, and the server’s per-statement work). Hardcharger’s one-round-trip
path answers small queries within about a millisecond of in-process DuckDB 1.5.6 (Q11: 6.1
against 5.2 ms) while running a different engine.
bench/concurrency.py: ten clients, each with its own connection and session, each running Q01,
Q03, Q06, Q10, Q14 and Q19 six times (360 queries), same machine and data:
Throughput
Median latency
p95 latency
Hardcharger
39.0 queries/s
223 ms
471 ms
GizmoSQL 1.41
21.9 queries/s
277 ms
1,328 ms
Hardcharger finishes the 360 queries in 9.2 s against 16.5 s (1.8× the throughput), and its
slowest queries wait far less (p95 2.8× lower).
Moving large results to a client, and bulk-loading a client’s Arrow table, from a second machine.
Scripts: bench/remote/ (provisioning, server start, client harness, summarizer); raw results:
bench/remote/results/2026-10-17-throughput/.
Machines: c8gn.8xlarge (Graviton4, 32 vCPU, 64 GiB, 100 Gbit/s), Ubuntu 24.04 arm64,
Spot. LAN: server and client in one placement group in us-east-2c (RTT 0.09 ms; iperf3: 1
stream 9.5 Gbit/s, 16 and 32 streams 99.8 Gbit/s). WAN: a third machine in us-west-2c
(RTT 46 ms), with Ubuntu’s default TCP buffers, as a client usually has.
TLS on both servers, one self-signed P-256 certificate, TLS 1.3. threads=32,
memory_limit=40GB, the same database file in /dev/shm.
Hardcharger: the hardcharger-linux-arm64 CI artifact from main, default settings (122,880-row batches, max_result_partitions 32). Client:
adbc-driver-warpsql. “n partitions” rows set adbc.warp.statement.result_partitions = n and
result_order = any or preserve (“in order”). GizmoSQL 1.42.0 (DuckDB 1.5.6) through
adbc-driver-gizmosql 2.0.15.
Fetch:SELECT * FROM <table>, drained as Arrow batches. Ingest:adbc_ingest(mode="create")
of lineitem from client memory. Every fetch is checked in full (row count and three column sums
against the server’s), and every ingest afterwards.
One warm-up, then 5 measured repetitions; median, p10 and p90. “Wire MB” is what crossed the
client’s network interface, TLS included. Client and server cores are CPU time over wall time.
One TLS stream runs at the network’s per-flow limit, like plaintext. SF10 lineitem
over TLS took 8.760 s (1.18 GB/s of Arrow, 9.5 Gbit/s on the wire), against 8.747 s over
plaintext.
Partitions go past one flow. With 32 partitions SF10 took 0.971 s (10.7 GB/s of Arrow,
86 Gbit/s on the wire). Order costs nothing on the LAN: interleaved partitions were as fast as
any-order ones.
8 partitions beat 16, and 32 beat both (SF10: 1.43, 1.59 and 0.97 s). The server was never
the limit; the client’s reading was. Why 16 did worse than 8 is not understood yet.
Ingest is as fast as GizmoSQL’s or faster. SF1: 1.30 s (GizmoSQL 1.61 s);
SF10: 9.34 s (GizmoSQL 9.43 s). Plaintext ingest is no longer
slower than TLS.
zstd is for slow links. On the LAN, zstd fetch is slower than none (SF10: 13.5 s against
8.8 s: the client decompresses on 1.5 cores), and zstd ingest much slower (27.7 s: the Go driver
compresses on one core). Over the WAN, zstd on one stream is 3× faster than none (4.05 s against
12.64 s).
WAN: partitions win, one stream doesn’t. SF1 across regions with 8 partitions in order:
1.96 s (GizmoSQL 12.26 s). One stream: 12.64 s, the same as GizmoSQL, bound by the 4 MB
tcp_wmem limit at 46 ms. 32 partitions in order over the WAN was erratic (median 27.7 s,
p10 4.8 s): interleaving waits for the slowest of 32 long-distance flows. Use 8, or any order.
Where Hardcharger still lost (before partitioned uploads)
Ingest over a fast LAN: Warp SQL’s ingest was one upload on one connection, so SF10 couldn’t
beat the 9.5 Gbit/s flow limit (9.34 s); a protocol that splits one upload over many
connections loaded it about 3× faster. Partitioned uploads
closed that gap.
Bytes on the wire. Arrow IPC is not the most compact encoding: a more compact wire format
sent about 30% fewer bytes for lineitem in the same tests.
One stream across a WAN: as slow as GizmoSQL whenever the client doesn’t ask for
partitions. The drivers now ask for 8 by default.
zstd over fast links, on the client side (one core compresses or decompresses in the Go
driver).
GizmoSQL runs DuckDB 1.5.6; Hardcharger runs DuckDB 2.0. Both read the same file format.
The ADBC ingest clients sent the source batches as they were (122,880 rows, about 21 MB). The Go
Flight SQL client refuses gRPC messages over 16 MiB by default, so GizmoSQL’s client was given
adbc.flight.sql.client_option.with_max_msg_size = 1 GiB.
Only Hardcharger compresses (zstd); GizmoSQL’s driver has no compression option.
Over the WAN only SF1 was measured. Tuned TCP buffers would raise every one-stream number there.
Spot instances and one placement group, on one day. Most configurations were stable (p90 within
a few percent of the median).
Two changes since the runs above, measured on fresh VMs with the same scripts:
Partitioned ingest (Warp SQL §8.3, capability partitioned_ingest): the client opens one
ingest, sends its upload as N parts at once, each over its own connection, and commits them as
one. The server reads every part through one reader, so DuckDB inserts from all of them in
parallel, in one transaction. All or nothing; rows land in no defined order. The ADBC driver
(adbc.warp.ingest.parts) uses 8 parts for uploads of two batches or more.
Split results by default: the ADBC driver and the hardcharger CLI ask for up to 8
partitions, in order. Results of fewer than two batches stay one stream.
Same setup as above (c8gn.8xlarge, LAN in one placement group, WAN across regions, TLS,
threads=32). Hardcharger with default settings; “Warp SQL” rows set adbc.warp.ingest.parts
and result_partitions to 1. Five measured repetitions after a warm-up; every ingest is checked
(row count, sum(l_extendedprice), count(DISTINCT l_orderkey) against the source) and every
fetch’s warm-up sums three columns of every batch. All matched.
Ingest is 3× faster than one stream on the LAN (SF10: 9.14 s to 3.00 s) and nearly 5×
across regions (SF1: 14.0 s to 2.9 s).
The LAN SF10 load is now bound by DuckDB’s insert, not the network: the server used 24.8
cores for 8 parts and 25.7 for 16, and 16 parts were no faster than 8.
The split-results default pays off on any link, with no option set: LAN SF1 fetch 0.90 s
to 0.17 s, SF10 8.76 s to 1.38 s, WAN SF1 14.0 s to 2.30 s.
LAN SF10 ingest, by 5 to 12% to a protocol whose client opens many more connections:
both are bound by DuckDB’s insert on the server, and Warp SQL’s wire format is 23% larger
for lineitem.
WAN SF1 ingest, by 20 to 35% to the same protocol with 8 parts; 16 parts come closer.
SF10 fetch at the default 8 partitions is 62% slower than the fastest result measured;
32 partitions took 0.97 s in the section above. The default stays 8, which suits the WAN
better.
Ordered bulk loads stay on one stream (9.14 s for SF10 on the LAN).
bench/remote/provision.shdown# terminate and delete everything, then verify
Cost: about 1.3 hours of three Spot c8gn.8xlarge (about $0.35/h each) and about 25 GB across
regions: about $2. Every resource was tagged Purpose=hardcharger-benchmark and deleted.
