SQL builders for DuckLake catalogs: listing snapshots, reading the rows that changed between them, tagging commits, and expiring old snapshots.
Every commit to a DuckLake catalog creates a snapshot, one numbering for all
of its tables. Read tables at a snapshot with QuackDB.Ecto.Snapshot, or
AT (VERSION => ...) in SQL. Attach a catalog with QuackDB.DDL.attach/3.
The table function helpers return SQL fragments that work as Ecto sources
and in raw SQL, like QuackDB.Source:
alias QuackDB.DuckLake
from snapshot in DuckLake.snapshots(:lake),
where: snapshot.snapshot_time <= ^event_at,
order_by: [desc: snapshot.snapshot_id],
limit: 1,
select: snapshot.snapshot_id
QuackDB.query!(conn, ["SELECT * FROM ", DuckLake.changes(:lake, :orders, from: 2, to: 5)])set_commit_message/3 and expire_snapshots/2 build CALL statements.
Summary
Functions
Builds a ducklake_table_changes(...) fragment with the rows of table
that changed in snapshots :from through :to, both included. Both are
snapshot ids or DateTimes; DateTimes take the snapshots committed
between them. Each row has the table's columns and snapshot_id, rowid,
and change_type: insert, delete, update_preimage, or
update_postimage.
Builds a ducklake_current_snapshot(...) fragment with the catalog's latest snapshot id.
Like changes/3, but only the deleted rows, without change columns. An
update deletes the row's old version, so it appears here too.
Builds a CALL ducklake_expire_snapshots(...) statement. Expired snapshots
can no longer be read; DuckLake keeps every snapshot until they expire.
Like changes/3, but only the inserted rows, without change columns.
Builds a CALL ducklake_set_commit_message(...) statement that tags the
snapshot the current transaction commits. Run it inside the transaction,
after its writes
Builds a ducklake_snapshots(...) fragment listing the catalog's snapshots,
with snapshot_id, snapshot_time, schema_version, changes, author,
commit_message, and commit_extra_info columns.
Types
Functions
Builds a ducklake_table_changes(...) fragment with the rows of table
that changed in snapshots :from through :to, both included. Both are
snapshot ids or DateTimes; DateTimes take the snapshots committed
between them. Each row has the table's columns and snapshot_id, rowid,
and change_type: insert, delete, update_preimage, or
update_postimage.
table is a table name in the main schema, a {schema, table} pair, or
an Ecto schema. A schema's @schema_prefix may name the catalog, as
"lake" or "lake.sales", or the schema within it.
Builds a ducklake_current_snapshot(...) fragment with the catalog's latest snapshot id.
Like changes/3, but only the deleted rows, without change columns. An
update deletes the row's old version, so it appears here too.
Builds a CALL ducklake_expire_snapshots(...) statement. Expired snapshots
can no longer be read; DuckLake keeps every snapshot until they expire.
DuckLake.expire_snapshots(:lake, older_than: DateTime.add(DateTime.utc_now(), -30, :day))
DuckLake.expire_snapshots(:lake, versions: [1, 2], dry_run: true)Requires :older_than or :versions.
Like changes/3, but only the inserted rows, without change columns.
Builds a CALL ducklake_set_commit_message(...) statement that tags the
snapshot the current transaction commits. Run it inside the transaction,
after its writes:
Repo.transaction(fn ->
Repo.insert!(order)
Repo.query!(DuckLake.set_commit_message(:lake, "checkout", extra_info: %{visit: 42}))
end)Supports :author and :extra_info, a string or a map stored as JSON.
Builds a ducklake_snapshots(...) fragment listing the catalog's snapshots,
with snapshot_id, snapshot_time, schema_version, changes, author,
commit_message, and commit_extra_info columns.