Benchmarking

Run the OLTP, engine, and ClickBench suites, and how to read what each one reports.

See the performance numbers and methodology for published results and how they were measured.

bun run oltp

packages/bench/src/oltp.ts compares lakefront against a real PostgreSQL, using pgbench’s own builtin workloads: select-only (a single point lookup by primary key), tpcb-like (pgbench’s default: three UPDATEs across three tables, one SELECT, one INSERT, all in one transaction), and a bare single-row UPDATE with no transaction wrapper on either side, to isolate raw write cost from transaction overhead. The three pgbench tables plus a history table map onto four boards in the same lake.

Network setup

pgbench reaches Postgres over TCP; Lakefront’s HTTP node also includes a network hop. The harness reports embedded results separately to show the cost of running the engine without the HTTP serving layer.

Run pgbench from the host

Run pgbench and psql from the host so both systems are reached through a Docker port mapping. Without host Postgres client tools, the harness falls back to docker exec and prints a warning.

bun run oltp                 # writePath "wal" (the default), scale 10, 10s per stage
bun run oltp -- --lake       # pin writePath "lake" instead
bun run oltp -- --scale=50 --time=30
bun run oltp -- --cluster    # also measure through a 2-node cluster + router

Durability pairing rule

Under writePath: "wal", the comparison target is Postgres with synchronous_commit=on (the default). It fsyncs its WAL per commit, the same durability class a WAL ack lives in.

The harness also prints a synchronous_commit=off Postgres line on the write workloads as a reference with weaker durability. Section headers identify the durability setting for each comparison.

bun run bench

packages/bench/src/main.ts is the engine benchmark suite: no external Postgres comparison, just this system measured against itself over time.

It seeds boards at three sizes (1,000 rows, 100,000 rows, and optionally 1,000,000), then runs the OLAP and OLTP workload sets from workloads.ts (group-bys, filters, top-N, wide reads, point lookups, range scans) against each size.

It also runs dedicated write workloads: single-row insert, batch insert, single-row update, and a full write-then-read-your-write round trip, plus a concurrency sweep at 1/8/32 in flight.

bun run bench                    # engine, node, and cluster layers, default sizes
bun run bench -- --large         # adds the 1M-row board
bun run bench -- --save          # records this run as the new baseline
bun run bench -- --layers=engine # restrict to one layer (engine, node, cluster)
bun run bench -- --single-node   # topology: "single-node" instead of "clustered"

Each layer runs the same workloads through a different entry point:

• engine: calls Engine directly
• node: goes through one serve node over HTTP
• cluster: spins up three nodes and a router and goes through that

Comparing the three layers against each other is how you see what HTTP and routing cost on top of the bare engine, on the same machine, same workload.

Pass --save after a change you want to compare future runs against. It writes bench/baseline.json, and every run without --save diffs against whatever baseline is already there.

bun run clickbench / clickbench

clickbench runs the ClickBench suite against a lakefront board: one flat ~100M-row web-analytics table and 43 analytical queries. It is a genuine fit for the OLAP half of this system (wide table, no joins, filters/group-bys/aggregations/top-N).

It is a poor fit for everything else about it: one table means one board, so multi-tenancy, routing, and sync-then-query freshness are all inert for this run. Reading a ClickBench score as a verdict on the multi-board architecture would be a mistake; it measures DuckDB’s execution plus this system’s serving overhead, nothing more.

bun run clickbench            # loads the dataset if needed, then runs the 43 queries
bun run clickbench -- --reload
bun run clickbench -- --runs=5
bun run clickbench:duckdb     # native DuckDB, same machine, same dataset, same queries

clickbench needs hits.parquet downloaded locally first (the command prints the curl line if it’s missing).

clickbench:duckdb runs the identical create/load/query sequence against a bare DuckDBInstance with no lakefront serving layer at all. Same machine, so hardware is not a confound the way it would be comparing against ClickHouse’s own published numbers on different hardware. The gap between the two runs is what the serving layer costs over the engine it embeds, and nothing else.

How to read the output

oltp and bench both report tps (or ops/sec) alongside p50/p95 latency per row, one row per system/client-count combination. Read across a client-count to see how each system holds up under concurrency, and down a system’s rows to see its own latency distribution.

bench’s read workloads additionally split lakefront’s own latency into syncMs (sync-then-query catch-up cost) and queryMs (execution once synced), which is the number that actually distinguishes a cold rebuild from a warm cache hit: a workload dominated by syncMs on repeat runs means the replica isn’t staying warm, not that queries are slow.

clickbench prints cold/warm/hot timings per query (first run, second run, last run) and a summary of hot p50/p95 and total time across all 43. The hot column is the number worth comparing across runs or against clickbench:duckdb, since cold includes whatever DuckDB’s own plan caching and buffer warmup cost on a first touch.