Architecture

How Lakefront splits writes and reads across a WAL-backed truth store and per-board replicas, and where that shape wins or loses.

Four pieces do the work: a WAL that makes writes durable immediately, DuckLake that makes them permanent, a DuckDB replica per board that answers reads, and a router that keeps each board’s traffic on one node.

• Writers append to a group-committed WAL table in the catalog Postgres.
• A flusher folds that WAL into DuckLake, a Parquet-plus-Postgres-catalog lakehouse, off the write path entirely.
• Server nodes keep a DuckDB replica file per board and bring it up to date before every query.
• A router in front picks which node handles a given board.

Truth store

DuckLake is the durable layer: Parquet files for bulk data, a Postgres catalog for metadata and, below a size threshold, the rows themselves.

DuckLake stores the authoritative data and snapshots used to rebuild replicas. It:

• turns storage into commodity object storage plus a small catalog
• gives every commit an immutable snapshot to rebuild from
• lets any node materialize any board from nothing but the lake

See the truth store for what that costs and how inlining, snapshots, and maintenance work.

Write path

A mutation appends to lakefront_wal, a plain table in the catalog Postgres, inside a group-committed transaction. The write is durable the instant that transaction commits, not when the lake eventually sees it. The lake becomes a batch layer, drained by a flusher running off the ack path.

An alternate "lake" write path commits straight into DuckLake for callers who need the simpler contract at a lower throughput ceiling. See write paths for the durability contract, group commit, and ordering guarantees.

Replicas

Each board is a single DuckDB file, attached into a node’s shared instance on demand and evicted under an LRU when the node holds more than it can keep warm. The file is a cache, never truth. It can be deleted and rebuilt from the lake at any time, which is what lets any server node answer for any board. See replicas for sync-then-query, cache outcomes, and ART indexing.

Server nodes

A server node is one process holding a DuckDB instance, a connection pool, and the replica cache for whatever boards route to it. It does not own any board permanently; state is soft, and losing a node’s local disk costs a round of rebuilds, not data. The HTTP layer in lakefront/server is a thin wrapper around the engine described above.

Router

Requests for a board need to land on the same node repeatedly, or the replica cache never warms. The router in lakefront/router picks that node per board and handles failover when it is unavailable. See routing for the hashing scheme and hedging behavior.

Read lifecycle: sync-then-query

1. The router sends the request to the node that owns this board’s cache affinity.
2. The node attaches the board’s replica file (or reuses it if already attached).
3. The replica syncs forward to the current head, applying any WAL records or lake changes it is missing.
4. The query runs against the now-current replica file, in-process, with no network hop.

Write lifecycle: append, ack, flush later

1. A mutation is appended to the WAL table as part of a group-committed Postgres transaction.
2. The write acks the instant that Postgres transaction commits: durable, but not yet in the lake.
3. A background flusher later drains the WAL into DuckLake in batches, off the write path.
4. Replicas pick up the mutation independently, by syncing from the WAL directly. They do not wait for the flush.

Lineage

This reconstructs the shape of monday.com’s mondayDB 3: CQRS, a soft-stateful per-tenant serving cache, and sync-then-query reads. The substitutions below are not an equivalent implementation.

Lakefront adopts file-per-tenant isolation, sync-then-query reads, and a replaceable serving cache from that architecture.

Trade-offs

Wins:

• Many small, independently-evolving boards (one per team, project, or customer) with a read-mostly workload and bursts of point writes.
• Analytics run against serving-fresh data without a separate warehouse.
• Per-board files make eviction, blast radius, and schema evolution board-scoped instead of table-scoped.
• The WAL write path keeps point writes cheap even though the lake commit underneath it is not.

Loses:

• A single huge table that does not decompose into boards. There is nothing here that shards one enormous table for you.
• Write-dominated OLTP with little read traffic: the WAL and the replica sync machinery exist to make reads fast, and a plain Postgres table would write faster.
• Cross-board joins at any real scale: each board is its own lake table and its own replica file, so a query spanning many boards means fanning out rather than one execution plan over shared storage.