Boards and schema

How a board's schema comes to be, inferred from data or declared up front, plus column widening and the explicit lifecycle calls.

A board is a dynamic, user-defined table: the multi-tenant unit.

Boards are created on first write

await engine.insert("acme_roadmap", { id: "task-1", state: "active", title: "Ship it" });

Nothing created acme_roadmap first. Under the default autoCreate: "on-write", a write to an unknown board creates it. That also means a typo’d board id silently creates a permanent board rather than failing.

autoCreate: "never" on open() turns that off: a write to a board that doesn’t exist is refused with BoardNotFound, and createBoard becomes the only way one comes into being. A read never creates a board, under either setting.

Two ways to define a schema

Under the default columns: "infer", the first write that mentions a column decides its type, and later writes are checked against it rather than re-deciding it. That’s what makes a board schemaless.

createBoard is the alternative: declare columns, with types, before any row exists.

await engine.createBoard("billing", {
  columns: { amount: "double", currency: "text", paid: "boolean", due: "timestamp" },
  strict: true,
});

columns types each column up front, so the first value to arrive never decides it. strict: true goes further: it makes writes to billing refuse any column not on that list, on every node in the cluster. See below for what that means and where the bit lives.

Column types are inferred, once

A value that doesn’t fit an existing column’s type is refused, not accommodated:

const conflict = await engine.insert("acme_roadmap", {
  id: "task-9",
  state: "active",
  priority: "N/A", // priority is already bigint
});
conflict.isErr() && conflict.error._tag; // "ColumnTypeConflict"

The recovery is explicit: widen the column, then retry the write.

await engine.widenColumn("acme_roadmap", "priority", "text");
await engine.insert("acme_roadmap", { id: "task-9", state: "active", priority: "N/A" });

Per-board strict, or engine-wide

strict: true on createBoard is a property of that one board, not of the process that set it: the bit is stored in the catalog (lakefront_boards), read with a 30-second-TTL cache, so a write hitting a different node still sees the same refusal. A write naming an undeclared column on a strict board fails with UnknownColumn:

await engine.insert("billing", { id: "b1", state: "active", amount: 5, memo: "late fee" });
// UnknownColumn: "memo" was never declared, and this board is strict

The engine-wide equivalent is columns: "declared" on open(): every board on this engine refuses undeclared columns, not just ones created with strict: true. A board’s own strict bit and the engine’s columns option both gate the same check. Either is enough to trigger it.

Column type vocabulary

ColumnType is the public vocabulary createBoard, addColumn, and widenColumn accept, lowercase:

Adding and widening columns

A column arrives implicitly the first time a write names it, or explicitly ahead of time:

await engine.addColumn("acme_roadmap", "assignee", "text");

addColumn is required on a strict board (per-board or engine-wide columns: "declared"), since inference is off and nothing else can introduce a column. Under "infer" it’s still useful, for fixing a column’s type up front instead of letting the first value passing through decide it.

widenColumn(boardId, column, to) moves a column’s type deliberately, and only ever upward. A request that would narrow a column is refused, so nothing already written can stop fitting. The type lattice, narrowest to widest:

BOOLEAN → TINYINT → SMALLINT → INTEGER → BIGINT → HUGEINT → FLOAT / DOUBLE → VARCHAR

(timestamp is not on this lattice at all. See above.) VARCHAR sits at the top: any value can be rendered as text, so it’s always a valid destination and never a valid source.

Both run atomically. A replica syncing past a widening also pays an inline index rebuild, described in replica sync.

Widening is irreversible by construction. A request that would move a column back down the lattice is refused:

const narrowed = await engine.widenColumn("acme_roadmap", "priority", "tinyint");
narrowed.isErr() && narrowed.error._tag; // "ColumnTypeConflict": that would narrow the column

Every row has BASE_COLUMNS

import { BASE_COLUMNS } from "lakefront";
// { id: "VARCHAR", updated_at: "TIMESTAMP", state: "VARCHAR" }

id, updated_at, and state exist on every board table and are the system’s contract rather than inferred. A value that disagrees with their type is reported. They cannot be redeclared through createBoard’s columns. Everything else is user-defined.

The engine sets updated_at on every insert, upsert, and update: the UTC time it accepted the write. Rows written in one call or one transaction share a single timestamp. A value you pass for updated_at is overwritten, so treat the column as read-only.

Column names starting with _ are reserved for the engine’s own columns (the version stamp upsert and compact read, for instance), and a write or declaration naming one is refused with ReservedColumnName.

await engine.insert("b", { id: "x", state: "active", _v: 1 });
// ReservedColumnName: the "_" prefix names the engine's own columns

Explicit lifecycle

await engine.createBoard("acme_archive"); // BoardExists if it's already there
await engine.hasBoard("acme_archive"); // Result<boolean, EngineError>
await engine.boards(); // every board in the lake
await engine.truncateBoard("acme_archive"); // empties it, keeps its columns
await engine.dropBoard("acme_archive"); // removes it and every row, everywhere

Use createBoard to create boards explicitly, declare column types, or enable strict schema checks.

dropBoard and truncateBoard reach every other node in the cluster with no message passing. The change rides the same WAL tombstone mechanism writes do, so a syncing replica discovers the board is gone or emptied the next time it asks for what changed. For how that propagation works, see write paths.