--- title: Engine description: open/openSafe and the Engine class, covering reads, writes, transactions, declared boards, watch, and operational helpers. --- ```ts import { open, openSafe, Engine } from "lakefront"; ``` One DuckDB instance holding the lake plus every attached board replica, fronted by a connection pool and an LRU of attached files. Every fallible method returns a `Result` from `better-result`. ## Functions ### open ```ts function open(options: OpenOptions): Promise; ``` | Parameter | Type | Default | Description | | --------- | ------------- | ---------- | -------------------------------- | | `options` | `OpenOptions` | (required) | See [OpenOptions](#openoptions). | **Returns** the opened `Engine`. Attaches the lake, starts the WAL and flusher (under `writePath: "wal"`), and starts the head-cache poller. `open` throws `LakeUnavailable` on a boot failure; use `openSafe` to receive that failure as a `Result`. It creates the `cache` directory and, for a local (non-`s3://`) `data`, the data directory itself, so there is nothing to `mkdir` before calling it. ```ts await using engine = await open({ postgres: "postgres://lakefront:lakefront@127.0.0.1:5432/lakefront", data: "./data", }); ``` Two required fields. Everything else (where the cache lives, how many rows stay inlined, how the cluster is shaped) has a default. `Engine` implements `AsyncDisposable`, so `await using engine = await open(...)` closes it (connection pool, WAL, flusher) when the scope ends; `close()` is idempotent. ### openSafe ```ts function openSafe(options: OpenOptions): Promise>; ``` Returns a boot failure as a `Result` instead of throwing. ### resetLake ```ts function resetLake(options: Pick): Promise; ``` | Parameter | Type | Default | Description | | --------- | -------------------------------------------------------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------- | | `options` | `Pick` | (required) | Same shape `open()` takes, so a reset names exactly the lake the engine would open (`namespace` defaults to `"lakefront"` the same way). | Removes both halves of a lake: drops the DuckLake catalog schema and deletes the `data` directory. :::danger Destructive. This standalone function is intended for test fixtures, demos, and intentional resets. Do not call this against a deployment holding data you want to keep. ::: ### dropCatalogSchema ```ts function dropCatalogSchema(postgres: string, namespace: string): Promise; ``` | Parameter | Type | Default | Description | | ----------- | -------- | ---------- | -------------------------------------------- | | `postgres` | `string` | (required) | Catalog Postgres, URL or keyword/value form. | | `namespace` | `string` | (required) | The catalog schema to drop. | Drops the DuckLake catalog schema alone, without deleting the data directory. Safe to call when the schema does not exist. `resetLake` calls this and then removes `data` too. ### toKeywordDsn ```ts function toKeywordDsn(input: string): string; ``` Converts a standard Postgres URL or the keyword/value form into the keyword/value DSN DuckLake's `ATTACH` accepts. What `open()` does internally to `postgres` before use. ### pgOptionsFromDsn ```ts function pgOptionsFromDsn(input: string): { hostname: string; port: number; database: string; username: string; password: string; max: number; }; ``` Converts either input form into the options shape Bun's `SQL` client accepts. ### messageOf ```ts function messageOf(cause: unknown): string; ``` Normalizes a thrown value into a message without swallowing its type: `cause instanceof Error ? cause.message : String(cause)`. Also exported from `lakefront/contract`, where it is defined; re-exported here so engine code keeps one import path. ## Engine Instances come from `open`/`openSafe`; there is no public constructor. `Engine` implements `AsyncDisposable`. **Reads** ### query ```ts query( boardId: string, sqlTemplate: string, options?: QueryOptions, ): Promise> ``` | Parameter | Type | Default | Description | | ------------- | -------------- | ---------- | -------------------------------------------- | | `boardId` | `string` | (required) | The board to query. | | `sqlTemplate` | `string` | (required) | SQL using `{{table}}` for the board's table. | | `options` | `QueryOptions` | `{}` | See [QueryOptions](#queryoptions). | **Returns** `Result`. Sync-then-query: the board's replica catches up to the head (or to `minWatermark`, if given) before the SQL runs against it. `sync` runs that catch-up alone, without a query. See the [consistency guide](/concepts/consistency) for the watermark model and read-your-write semantics. ### sync ```ts sync(boardId: string): Promise> ``` Runs the sync-then-query catch-up for `boardId` without executing a query. Returns the [SyncResult](#syncresult) it produced. ### watermarkOf ```ts watermarkOf(boardId: string): Promise> ``` The board's replica watermark, without a sync. ### rowCount ```ts rowCount(boardId: string): Promise> ``` Row count on the board's attached replica. ### head ```ts head(): number ``` The cached head, with no round-trip. `headFromCatalog()` pays for the authoritative one. ### headFromCatalog ```ts headFromCatalog(): Promise> ``` The authoritative head, read from the catalog. One round-trip; `head()` is the cached answer most callers want. ### withReplica ```ts withReplica( boardId: string, fn: (ctx: { conn: DuckDBConnection; replica: LocalReplica; lake: Lake; head: HeadCache }) => Promise, ): Promise> ``` | Parameter | Type | Default | Description | | --------- | --------------------- | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | `boardId` | `string` | (required) | The board whose replica to hand to `fn`. | | `fn` | `(ctx) => Promise` | (required) | Callback given a leased connection and the attached replica. `lake` and `head` are internal types, not otherwise part of this reference surface. | Low-level access to an attached replica on a pooled connection. Attaching and busy-tracking are handled; sync is not, so the caller decides. :::warning[Bypasses sync-then-query] A caller that reads through `withReplica` without syncing first can read a stale replica. Reach for `query` unless raw connection access is specifically what is needed. ::: **Writes** All writes return `Result`, where the `number` is the watermark the write committed at: hand it back as `minWatermark` on a subsequent `query` to read your own write. ### insert ```ts insert(boardId: string, row: Row, options?: WriteOptions): Promise> ``` Inserts one row. Sugar over `insertMany`. ### insertMany ```ts insertMany( boardId: string, rows: readonly Row[], options?: WriteOptions, ): Promise> ``` | Parameter | Type | Default | Description | | --------- | ---------------- | ---------- | ---------------------------------- | | `boardId` | `string` | (required) | Board to write to. | | `rows` | `readonly Row[]` | (required) | Rows to insert. | | `options` | `WriteOptions` | `{}` | See [WriteOptions](#writeoptions). | Inserts rows as one write. ### upsert ```ts upsert(boardId: string, row: Row, options?: WriteOptions): Promise> ``` Upserts one complete row. Sugar over `upsertMany`. ### upsertMany ```ts upsertMany( boardId: string, rows: readonly Row[], options?: WriteOptions, ): Promise> ``` | Parameter | Type | Default | Description | | --------- | ---------------- | ---------- | ----------------------------------- | | `boardId` | `string` | (required) | Board to write to. | | `rows` | `readonly Row[]` | (required) | Complete row versions, not patches. | | `options` | `WriteOptions` | `{}` | See [WriteOptions](#writeoptions). | Append-based upsert: writes a new version of each row instead of issuing an UPDATE. An UPDATE on a DuckLake table costs about 2.5x an INSERT, because it has to locate the existing rows and mark them deleted first; appending skips that. Rows must be complete, and superseded versions accumulate until `compact()` reclaims them. The winner is chosen by version stamp, so across nodes it is last-writer-wins to whatever precision the clocks agree on, which is why `update`/`updateMany` remain the default and this is opt-in. ### update ```ts update( boardId: string, id: string, patch: Row, options?: WriteOptions, ): Promise> ``` | Parameter | Type | Default | Description | | --------- | -------------- | ---------- | ---------------------------------- | | `boardId` | `string` | (required) | Board to patch. | | `id` | `string` | (required) | Row id to patch. | | `patch` | `Row` | (required) | Columns to set. | | `options` | `WriteOptions` | `{}` | See [WriteOptions](#writeoptions). | Patches one row by id. ### updateMany ```ts updateMany( boardId: string, updates: ReadonlyArray<{ readonly id: string; readonly patch: Row }>, options?: WriteOptions, ): Promise> ``` | Parameter | Type | Default | Description | | --------- | ------------------------------------------- | ---------- | ---------------------------------- | | `boardId` | `string` | (required) | Board to patch. | | `updates` | `ReadonlyArray<{ id: string; patch: Row }>` | (required) | One entry per row to patch. | | `options` | `WriteOptions` | `{}` | See [WriteOptions](#writeoptions). | Applies many single-row patches as one atomic write, the batch shape OLTP loops actually produce, without paying a lake commit or a WAL append per row. Entries apply in array order, so two patches to the same id stack exactly as two `update()` calls would. ### delete ```ts delete(boardId: string, id: string, options?: WriteOptions): Promise> ``` Deletes one row by id. Sugar over `deleteMany`. ### deleteMany ```ts deleteMany( boardId: string, ids: readonly string[], options?: WriteOptions, ): Promise> ``` | Parameter | Type | Default | Description | | --------- | ------------------- | ---------- | ---------------------------------- | | `boardId` | `string` | (required) | Board to delete from. | | `ids` | `readonly string[]` | (required) | Row ids to delete. | | `options` | `WriteOptions` | `{}` | See [WriteOptions](#writeoptions). | Deletes rows by id. ### compact ```ts compact(boardId: string): Promise> ``` Collapses superseded versions left behind by `upsert`. **Transactions** ### transaction ```ts transaction( boards: readonly string[], fn: (tx: TransactionOps) => Promise, options?: WriteOptions, ): Promise> ``` | Parameter | Type | Default | Description | | --------- | ------------------------------------ | ---------- | ----------------------------------------------------------------------------------- | | `boards` | `readonly string[]` | (required) | Every board the body touches, declared up front. | | `fn` | `(tx: TransactionOps) => Promise` | (required) | The transaction body; runs against the handle in [TransactionOps](#transactionops). | | `options` | `WriteOptions` | `{}` | See [WriteOptions](#writeoptions). | Runs several writes as one atomic commit spanning every board named in `boards`. The whole block runs on one connection (or as one WAL unit, under `writePath: "wal"`), and the returned watermark is the read-your-write watermark for everything the block wrote. ```ts const r = await engine.transaction(["accounts", "history"], async (tx) => { await tx.update("accounts", id, { balance: next }); await tx.insert("history", { id: crypto.randomUUID(), delta }); }); ``` **Lifecycle** ### createBoard ```ts createBoard(boardId: string, options?: CreateBoardOptions): Promise> ``` | Parameter | Type | Default | Description | | --------- | -------------------- | ---------- | ---------------------------------------------- | | `boardId` | `string` | (required) | Board to create. | | `options` | `CreateBoardOptions` | `{}` | See [CreateBoardOptions](#createboardoptions). | Fails with `BoardExists` if the board is already there. It is the explicit counterpart to write-time auto-create, and the only place a board's columns can be declared rather than inferred: ```ts await engine.createBoard("billing", { columns: { amount: "double", currency: "text", paid: "boolean", due: "timestamp" }, strict: true, }); ``` Base columns (`id`, `updated_at`, `state`) exist on every board already and cannot be redeclared, neither can a column under the reserved `_` prefix. Declaring a name that collides with either fails with `ReservedColumnName`. ### dropBoard ```ts dropBoard(boardId: string): Promise> ``` Removes the board and every row, everywhere. Other nodes need no message: the drop advances the snapshot log, so their next sync discovers the table gone. ### truncateBoard ```ts truncateBoard(boardId: string): Promise> ``` Empties a board, keeping it and its columns. ### boards ```ts boards(): Promise> ``` Every board in the lake, whether or not this node has it cached. ### hasBoard ```ts hasBoard(boardId: string): Promise> ``` Whether `boardId` exists in the lake. ### addColumn ```ts addColumn(boardId: string, column: string, type: ColumnType): Promise> ``` | Parameter | Type | Default | Description | | --------- | ------------ | ---------- | ------------------------------ | | `boardId` | `string` | (required) | Board to add the column to. | | `column` | `string` | (required) | Column name. | | `type` | `ColumnType` | (required) | See [ColumnType](#columntype). | Declares a column on a board, without waiting for data to imply it. Required under `columns: "declared"` (engine-wide) or a board created with `strict: true`, where inference is off. Useful under inference too: it fixes a column's type up front instead of letting the first value decide. ### widenColumn ```ts widenColumn(boardId: string, column: string, to: WidenTarget): Promise> ``` | Parameter | Type | Default | Description | | --------- | ------------- | ---------- | --------------------------------------------- | | `boardId` | `string` | (required) | Board to widen a column on. | | `column` | `string` | (required) | Column name. | | `to` | `WidenTarget` | (required) | Target type, see [WidenTarget](#widentarget). | Widens one column's type, deliberately and irreversibly. Column types are inferred once, when a write first mentions the column, and fixed after that: a later value that does not fit is refused with `ColumnTypeConflict` rather than silently reshaping the board. This is the escape hatch, and it only ever widens: a request that would narrow a column is refused, so nothing already written can stop fitting. ```ts await engine.widenColumn("acme_roadmap", "priority", "VARCHAR"); ``` **Feed** ### watch ```ts watch( boardId: string, options?: { after?: number; signal?: AbortSignal; pollMs?: number }, ): AsyncGenerator ``` | Parameter | Type | Default | Description | | ---------------- | ------------- | ------------ | ------------------------------------------------------- | | `boardId` | `string` | (required) | Board to follow. | | `options.after` | `number` | current head | Watermark to start from; a past value catches up first. | | `options.signal` | `AbortSignal` | `undefined` | Ends the loop when aborted. | | `options.pollMs` | `number` | `500` | Poll fallback interval for writes on other nodes. | Follows a board's change feed as an async iterable: catch-up from `after`, then live. ```ts for await (const event of engine.watch("tasks", { after })) { if (event.resync) { // re-read current state, then continue -- the iterator already // moved its cursor to event.from } else { // event.records } } ``` Each yielded event is a non-empty `ChangesPage`. Local writes wake the loop immediately; writes landing on other nodes are picked up by the poll fallback, so the feed is complete either way, and the wakeup only sharpens latency. `resync: true` is yielded as an ordinary value and the iterator continues from the new floor; it is not an error. A read failure is thrown (the same `EngineError` `changes()` would return as a `Result`). Design, retention, and what `resync` means for a consumer are covered in the [watch guide](/guides/watch). ### changes ```ts changes(boardId: string, options: { after: number; limit?: number }): Promise> ``` One page of the change feed for a board above a watermark this engine handed out. `watch` is a loop built on top of it, and is the surface most callers want directly. ### onChange ```ts onChange(listener: (event: ChangeEvent) => void): () => void ``` Registers a local trigger, called once per touched board after each of this node's own write acks; fires on a microtask, so a slow or throwing listener can neither delay nor fail the write. It sees only this node's own writes (that local signal is what wakes `watch` between polls) and is not itself a substitute for `watch` or `changes`. Returns the unsubscribe function. **Operations** ### maintain ```ts maintain(olderThanHours?: number): Promise> ``` Runs lake maintenance now (snapshot expiry, file cleanup, small-file merge); under `writePath: "wal"` the flusher also runs it hourly. ### flush ```ts flush(boardId?: string): Promise> ``` Drains the WAL into the lake immediately: one board, or every board with pending records when `boardId` is omitted. A no-op under `writePath: "lake"`. ### evict ```ts evict(boardId: string, deleteFile?: boolean): Promise ``` Drops a replica from the cache. The file stays on disk unless `deleteFile` is set, which simulates losing the node's storage entirely. ### stats ```ts stats(): EngineStats ``` Returns the [EngineStats](#enginestats) snapshot. ### cachedBoards ```ts cachedBoards(): readonly string[] ``` Boards currently attached in this node's replica cache. ### close ```ts close(): Promise ``` Closes the connection pool, WAL, and flusher. Idempotent: a second call returns immediately rather than deadlocking on connections the first close already released. `await using engine = await open(...)` calls this at scope exit via `Symbol.asyncDispose`. ## Types ### OpenOptions What `open()` and `openSafe()` take: two required fields (where the catalog Postgres is, where the data lives) and defaults for everything else. | Option | Type | Default | What it does | | ------------------ | ------------------------------ | -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `postgres` | `string` | (required) | The catalog Postgres: a standard URL (`postgres://user:pass@host/db`) or the keyword/value form (`postgres:dbname=... host=...`). This one database holds the DuckLake catalog, the WAL, and, via inlining, recent row data itself. | | `data` | `string` | (required) | Where Parquet data files land: a local directory, or an `s3://` prefix (with `s3` set). Small mutations stay inlined in the catalog and never reach here; see `inliningRowLimit`. | | `s3` | `S3Config` | `undefined` | Object storage credentials, required when `data` is an `s3://` prefix. | | `namespace` | `string` | `"lakefront"` | Namespaces this lake's catalog tables inside Postgres, so independent lakes can share one database. Set it only to run more than one lake against the same Postgres. | | `cache` | `string` | `/.cache` (local) or `.lakefront/cache` (s3) | Directory for this process's replica cache files. Per-process soft state; two engines must not share one. | | `inliningRowLimit` | `number` | `10_000` | Rows below this stay inlined in the catalog instead of becoming Parquet. Inlined rows are the speed layer; DuckLake's own default of 10 sprays tiny files. | | `cacheCapacity` | `number` | `256` | Max attached replica files. Locally this exists so eviction is exercised rather than theoretical; a production node holds far more. | | `poolSize` | `number` | `24` | DuckDB connection pool size. | | `headRefreshMs` | `number` | `5` | How often the lake head refreshes in the background. Bounds how stale a cross-board read can be; read-your-write is unaffected. | | `topology` | `"single-node" \| "clustered"` | `"clustered"` | `"clustered"` polls the head in the background. `"single-node"` asserts this process is the only writer, so no polling. See below. | | `duckdbThreads` | `number` | `1` | DuckDB threads per query. A 1k-row group-by measured 3.2x faster on one thread than across 14: parallelizing one small query wastes cores better spent on concurrent queries. Raise it for boards near 10^6 rows, where the crossover flips. | | `replicaWalWrites` | `boolean` | `false` | Enables DuckDB's own WAL on replica files. Off by default because the file is a disposable cache and a second write-ahead log is write amplification; costs commit durability across a crash. | | `indexColumns` | `readonly string[]` | `["id"]` | Columns kept ART-indexed on every replica, for point lookups. Add only columns actually hit by point predicates: every entry slows sync apply (about 1.7x measured with two indexes) and costs memory. | | `autoCreate` | `"on-write" \| "never"` | `"on-write"` | Whether a write to an unknown board creates it. `"never"` makes `createBoard` the only way a board comes into existence; reads never create a board either way. | | `columns` | `"infer" \| "declared"` | `"infer"` | `"infer"` types a column from the first data that names it. `"declared"` refuses an unknown column with `UnknownColumn` instead, engine-wide. See below. | | `writePath` | `"wal" \| "lake"` | `"wal"` | `"wal"` acks at a group-committed WAL table in the catalog Postgres and flushes to the lake off-path. `"lake"` commits every write to DuckLake synchronously. See below. | | `flushIntervalMs` | `number` | `30_000` | How often the WAL flusher drains into the lake, under `writePath: "wal"`. `0` disables the timer, leaving `flush()` calls in charge (what tests use). | | `groupCommit` | `boolean` | `true` | Batches concurrent writes into shared DuckLake commits. A commit costs about 7.5ms almost regardless of payload and is serialized catalog-wide, so amortizing it is the only thing that raises write throughput. | A few of these carry rationale too long for a cell: - **`topology`**: `"clustered"` cross-board reads are stale by at most one interval, bounded by the head-refresh poll. `"single-node"` trades that staleness for exact cross-board reads. Running two `"single-node"` engines against one lake produces incorrect reads. - **`columns`**: `"infer"` is what makes a board schemaless. A single board can opt into declared-only columns on its own via `createBoard(id, { strict: true })`, independent of this engine-wide setting. - **`writePath`**: `"lake"` is simpler, but caps throughput at double-digit tps per node. `"wal"` trades that ceiling for the durability contract described under [write paths](/concepts/write-paths). ### S3Config | Field | Type | Default | What it does | | ---------- | --------- | ----------- | ----------------------------------- | | `endpoint` | `string` | (required) | S3-compatible endpoint. | | `keyId` | `string` | (required) | Access key id. | | `secret` | `string` | (required) | Secret key. | | `ssl` | `boolean` | `undefined` | Whether to use TLS to the endpoint. | | `region` | `string` | `undefined` | Bucket region. | ### QueryOptions | Option | Type | Default | What it does | | -------------- | ------------------------------------------ | ----------- | ---------------------------------------------------------------------------------------------------------- | | `minWatermark` | `number \| undefined` | `undefined` | Read-your-write: refuses to serve a replica older than this watermark, forcing the head forward if needed. | | `bindings` | `ReadonlyMap \| undefined` | `undefined` | Named `{{token}}` bindings beyond `{{table}}`. | | `params` | `readonly Cell[] \| undefined` | `undefined` | Positional bind values (`$1`, `$2`, ...). | ### QueryOutcome | Field | Type | What it is | | ----------- | ---------------------------------- | ----------------------------------------------------------------------------------------------------- | | `result` | `QueryResult` | `{ columns: string[], rows: Cell[][] }`: columnar, not row objects. | | `sync` | `SyncResult` | What the preceding sync did; see [SyncResult](#syncresult). | | `cache` | `"hit" \| "rebuild" \| "reattach"` | Whether the replica was already attached, rebuilt from the lake, or reattached from an existing file. | | `syncMs` | `number` | Time spent syncing, split from query time so a caller can tell freshness cost from execution cost. | | `queryMs` | `number` | Time spent executing the query itself. | | `watermark` | `number` | The replica's watermark after this query: pass back as `minWatermark` to read this exact write. | ### SyncResult | Field | Type | What it is | | ---------------- | ---------- | ---------------------------------------------------------------- | | `from` | `number` | Watermark the sync started from. | | `to` | `number` | Watermark the sync landed at. | | `skipped` | `boolean` | `true` when the replica was already at the head and nothing ran. | | `changesApplied` | `number` | Change records applied. | | `rowsInserted` | `number` | Rows inserted by this sync. | | `rowsDeleted` | `number` | Rows deleted by this sync. | | `columnsAdded` | `string[]` | Columns the sync added to the replica's schema. | ### WriteOptions | Option | Type | Default | What it does | | ---------------- | -------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `idempotencyKey` | `string` | `undefined` | Makes the write safely retryable: a second attempt with the same key applies nothing and returns the original watermark. Requires `writePath: "wal"`; keys are retained for a bounded window (expired by maintenance after 24h). See the [idempotency guide](/guides/idempotency). | | `actor` | `string` | `undefined` | Audit identity recorded on the WAL records this write produces. Recorded only under `writePath: "wal"`. | ### TransactionOps The handle `transaction`'s body runs against. Methods here throw instead of returning `Result`: a failure aborts the whole transaction rather than being handled statement by statement, and `Engine.transaction` converts the throw back into the outer `Result` at the boundary. | Method | Signature | Notes | | ------------ | -------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `insert` | `(boardId, row: Row) => Promise` | Sugar over `insertMany`. | | `insertMany` | `(boardId, rows: readonly Row[]) => Promise` | | | `upsert` | `(boardId, row: Row) => Promise` | Sugar over `upsertMany`. See `Engine.upsert`. | | `upsertMany` | `(boardId, rows: readonly Row[]) => Promise` | Append-based upsert inside the transaction; see `Engine.upsertMany`. | | `update` | `(boardId, id: string, patch: Row) => Promise` | | | `updateMany` | `(boardId, updates: ReadonlyArray<{ id: string; patch: Row }>) => Promise` | | | `delete` | `(boardId, id: string) => Promise` | Sugar over `deleteMany`. | | `deleteMany` | `(boardId, ids: readonly string[]) => Promise` | | | `scalar` | `(boardId, sql: string, params?: readonly Cell[]) => Promise` | One value from arbitrary SQL. Sees this transaction's own writes on `writePath: "lake"`; under `"wal"` it sees only committed state. Use `get` for a read that must observe the transaction's own writes. | | `get` | `(boardId, id: string) => Promise` | One row by primary key. Sees this transaction's own writes under both write paths: the `"wal"` scope answers it from an in-memory overlay of the transaction's pending writes. | ### CreateBoardOptions | Field | Type | Default | What it does | | --------- | -------------------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `columns` | `Readonly>` | `{}` | Columns declared up front, name to type. Typed before any data arrives, so the first value written never decides the type. | | `strict` | `boolean` | `false` | When true, this board refuses writes that name an undeclared column (`UnknownColumn`), on every node, regardless of the engine-wide `columns` option. Stored in the catalog and read with a short-TTL cache, so the policy travels with the board. | ### ColumnType The public column-type vocabulary uses plain lowercase names, mapped onto the DuckDB type underneath: | `ColumnType` | Underlying SQL type | Notes | | ------------- | ------------------- | ---------------------------------------------------------------------------------------------------------- | | `"boolean"` | `BOOLEAN` | | | `"tinyint"` | `TINYINT` | | | `"smallint"` | `SMALLINT` | | | `"integer"` | `INTEGER` | | | `"bigint"` | `BIGINT` | | | `"hugeint"` | `HUGEINT` | | | `"float"` | `FLOAT` | | | `"double"` | `DOUBLE` | | | `"text"` | `VARCHAR` | | | `"timestamp"` | `TIMESTAMP` | Declarable, but opaque to inference: outside the widening lattice, so it cannot be widened into or out of. | `COLUMN_TYPES` is the object above, exported as the runtime source of truth; `ColumnType` is `keyof typeof COLUMN_TYPES`. See the [schema guide](/guides/schema) for how inference and widening behave in practice. ### WidenTarget `Exclude`: every `widenColumn` destination except `"timestamp"`, since that type sits outside the lattice a column's inferred type moves up. ### EngineStats | Field | Type | What it is | | ------------------ | ------------------- | ---------------------------------------------------------------------------------------------------------------------- | | `flusherParked` | `ParkedBoardStat[]` | Boards the periodic flusher has given up on for now (`{ boardId, failures, since }`); empty under `writePath: "lake"`. | | `cached` | `number` | Replica files currently attached. | | `capacity` | `number` | `cacheCapacity` this engine opened with. | | `hits` | `number` | Replica-cache hits. | | `rebuilds` | `number` | Replicas rebuilt from the lake. | | `reattaches` | `number` | Replicas reattached from an existing file. | | `evictions` | `number` | Replicas evicted from the cache. | | `connectionsInUse` | `number` | Pooled connections currently leased. | ### ChangeEvent `{ boardId: string, watermark: number, ops: readonly WalRecordOp[] }`. Fired to `onChange` listeners after a write's ack, once per touched board. ### WalRecordOp `{ kind: WalKind, payload: WalPayload }`. A change record's operational core. ### ChangesPage Discriminated on `resync`: `{ resync: false, records: ChangeRecord[], next: number } | { resync: true, from: number }`. `resync: true` means the requested range is beyond retention; re-read current state and continue from `from`. ### FlushStats ```ts interface FlushStats { readonly boardId: string; readonly from: number; readonly to: number; readonly records: number; readonly skipped: boolean; } ``` What one `flush()` call did for one board. ### ParkedBoard ```ts interface ParkedBoard { readonly boardId: string; readonly failures: number; readonly since: number; readonly retryAt: number; readonly message: string; } ``` A board the periodic flusher has given up on for now: its WAL records are untouched (durability never depended on flushing), but they accumulate until the cause is repaired. `stats()` itself returns the narrower wire shape `ParkedBoardStat` (`{ boardId, failures, since }`); `ParkedBoard` is the richer local shape, with the retry backoff and message, that the flusher tracks internally. ### WalKind `"insert" | "upsert" | "update" | "delete" | "drop"`. ### WalOp `{ boardId: string, kind: WalKind, payload: WalPayload }`. ### WalPayload A union keyed by shape: `{ rows: Row[] }` (insert/upsert), `{ id: string, patch: Row }` (update), `{ ids: string[] }` (delete), or `{ drop: true }` (a whole-board tombstone). ### Row `Readonly>`, defined once in `lakefront/contract` and re-exported here. ### Cell `string | number | boolean | null`, defined once in `lakefront/contract` and re-exported here. ### QueryResult `{ columns: string[], rows: Cell[][] }`, defined once in `lakefront/contract` and re-exported here. ### EngineError The union of every tagged error class the engine can raise: `InvalidBoardId`, `EntityNotAllowed`, `PlaceholderUnbound`, `LakeUnavailable`, `QueryFailed`, `SyncFailed`, `ReplicaUnavailable`, `WriteFailed`, `ReservedColumnName`, `ColumnTypeConflict`, `BoardNotFound`, `BoardExists`, `UnknownColumn`, each also exported by name. Full catalog, fields, and when each is raised: the [error catalog](/reference/errors). ## Constants ### COLUMN_TYPES The runtime source of truth for the [ColumnType](#columntype) table above: `{ boolean: "BOOLEAN", tinyint: "TINYINT", ..., text: "VARCHAR", timestamp: "TIMESTAMP" }`. ### BASE_COLUMNS ```ts BASE_COLUMNS = { id: "VARCHAR", updated_at: "TIMESTAMP", state: "VARCHAR" }; ``` The fixed columns every board table has; their types are fixed by the system and never inferred from data. ### RESERVED_COLUMN_PREFIX ```ts RESERVED_COLUMN_PREFIX = "_"; ``` The namespace user columns may not write into: a column named `_v` decides upsert ordering, compaction, and version-scoped deletes. ### ENGINE_ERRORS The lookup object every derivation in the system uses: keyed by tag, values are the error classes themselves. `WIRE_TAGS`, the engine's own error-wrapping, and the reference catalog are all built from it, so a class added here is everywhere at once. See the [error catalog](/reference/errors).