DuckDB for Elixir applications: the DBConnection client.
Queries, streams, transactions, and native appends over DuckDB's Quack
protocol (version 3, served by DuckDB 2.0). QuackDB.Server supervises a
local DuckDB process, and Ecto.Adapters.QuackDB puts Ecto on top of this
client.
Connection options
:uriand:tokenlocate and authenticate the Quack server.:heartbeat_timeoutis the connection lease in seconds the client asks for (default60). The server drops a connection that sends no request for that long, so each connection sends a heartbeat every third of the lease over a separate HTTP connection. Long queries, slow streams, and long transactions keep their connection. A connection whose lease expires anyway reconnects.:client_ididentifies this client in the server's logs.:acknowledge_resultsconfirms every fully received result, so a server withquack_enable_reconnectsdrops its cached copy right away instead of waiting forquack_result_ttl.:fetch_retriesis how often a result batch is requested again after its HTTP connection broke (default1). Batches are numbered, so a retry never skips or repeats rows.:copy_binariescopies decoded strings and blobs off the HTTP response (defaultfalse); it can also be passed per query. Without it, a decoded string of 64 bytes or more is a slice of the response it arrived in, so keeping it keeps the whole response, up to one result batch, in memory. Copying about doubles decode time for string columns and costs nothing for other columns. Use it when you keep a few values from large results, for example in a cache or a long-lived process.:connect_timeout,:receive_timeout,:shutdown_timeout, and:mint_optionstune the HTTP transport.
Query options include :inline_rows, the number of rows the server may
return with the first response before the rest is fetched in batches.
Summary
Functions
Interrupts the query running on the remote connection connection_id.
Streams query results as column-oriented batches.
Runs a query and returns a QuackDB.Columns struct.
Streams query results as QuackDB.Columns batches.
Runs a query and returns its result as a column-oriented map.
Appends column-oriented values to a DuckDB table by streaming binary vectors into an INSERT.
Appends row-oriented values to a DuckDB table by streaming binary vectors into an INSERT.
Appends an enumerable of row maps/keywords in batches through native append.
Appends any Table.Reader compatible tabular data through native append.
Types
@type append_type() :: QuackDB.Type.spec()
Native append column type specs.
@type start_option() :: {:uri, String.t()} | {:token, String.t()} | {:name, GenServer.name()} | {:connect_timeout, timeout()} | {:receive_timeout, timeout()} | {:shutdown_timeout, timeout()} | {:mint_options, keyword()} | {:heartbeat_timeout, pos_integer()} | {:client_id, String.t()} | {:acknowledge_results, boolean()} | {:fetch_retries, non_neg_integer()} | {:copy_binaries, boolean()}
Functions
@spec cancel(String.t() | URI.t(), String.t(), Keyword.t()) :: :ok | {:error, Exception.t()}
Interrupts the query running on the remote connection connection_id.
Run it from another process than the one waiting on the query. The
connection id appears in every QuackDB.Result, QuackDB.Error, and
[:quackdb, :query, :start] telemetry event, including those of Ecto
queries, and a query interrupted this way fails with a :query_cancelled
error:
:telemetry.attach("report-connection", [:quackdb, :query, :start], fn _event, _measurements, metadata, parent ->
send(parent, {:running, metadata.connection_id})
end, self())
task = Task.async(fn -> MyApp.AnalyticsRepo.all(yearly_report) end)
receive do
{:running, connection_id} -> QuackDB.cancel("http://[::1]:9494", connection_id)
end
Task.await(task)
#=> ** (QuackDB.Error) Interrupted: query was cancelledPass query_uuid: uuid to cancel only that query. Transport timeouts and
disconnects already cancel their query on the server.
@spec child_spec([start_option()]) :: Supervisor.child_spec()
@spec column_batches(DBConnection.conn(), iodata(), [term()], Keyword.t()) :: Enumerable.t()
Streams query results as column-oriented batches.
Each item is a map from disambiguated column names to the values in that fetch
batch. This keeps large analytical results vector-shaped without materializing
the whole result set. Prefer columnar_batches/4 when you also need batch
metadata.
@spec columnar(DBConnection.conn(), iodata(), [term()], Keyword.t()) :: {:ok, QuackDB.Columns.t()} | {:error, Exception.t()}
Runs a query and returns a QuackDB.Columns struct.
This preserves column order, original names, row count, and result metadata in addition to the column vectors.
@spec columnar!(DBConnection.conn(), iodata(), [term()], Keyword.t()) :: QuackDB.Columns.t()
@spec columnar_batches(DBConnection.conn(), iodata(), [term()], Keyword.t()) :: Enumerable.t()
Streams query results as QuackDB.Columns batches.
This uses a columnar cursor path so large analytical results can stay vector-shaped instead of being materialized as row lists first.
@spec columns(DBConnection.conn(), iodata(), [term()], Keyword.t()) :: {:ok, %{required(String.t()) => [term()]}} | {:error, Exception.t()}
Runs a query and returns its result as a column-oriented map.
Duplicate column names are disambiguated with suffixes such as _2 and _3.
Prefer columnar/4 when you also need column order and result metadata.
@spec insert_columns( DBConnection.conn(), String.t() | atom(), [insert_column()], Keyword.t() ) :: {:ok, QuackDB.Result.t()} | {:error, Exception.t()}
Appends column-oriented values to a DuckDB table by streaming binary vectors into an INSERT.
Column values are provided as {name, values} pairs. All columns must have the
same row count. Pass :columns with type specs when values are empty, contain
only nils, or need a specific nested DuckDB type. Explicit MAP columns accept
ordinary Elixir maps and DuckDB-style key/value entries.
@spec insert_columns!( DBConnection.conn(), String.t() | atom(), [insert_column()], Keyword.t() ) :: QuackDB.Result.t()
@spec insert_rows( DBConnection.conn(), String.t() | atom(), [insert_row()], Keyword.t() ) :: {:ok, QuackDB.Result.t()} | {:error, Exception.t()}
Appends row-oriented values to a DuckDB table by streaming binary vectors into an INSERT.
Rows are maps or keywords. Without :columns, every row's keys name the
columns and every value takes part in a column's type: integers widen to
BIGINT or HUGEINT, integers and floats make DOUBLE, decimals and
integers make one DECIMAL, and maps make a STRUCT of all their keys. Any
other mix, such as integers and strings, fails with :mixed_append_types
instead of truncating or dropping values. Pass :columns with type specs when
values are empty, contain only nils, mix shapes (:variant keeps them), or
need a specific DuckDB type.
Plain Elixir maps infer as DuckDB STRUCT values. For explicit
{:map, key_type, value_type} columns, ordinary Elixir maps are encoded as
DuckDB MAP values:
QuackDB.insert_rows!(conn, "events", [[labels: %{env: "prod"}]],
columns: [labels: {:map, :varchar, :varchar}]
)DuckDB-style key/value entries are also accepted for explicit MAP columns:
QuackDB.insert_rows!(conn, "events", [[labels: [%{key: "env", value: "prod"}]]],
columns: [labels: {:map, :varchar, :varchar}]
)Duplicate MAP keys decode with the later entry winning, matching Map.put/3.
Keys and values are encoded through the declared DuckDB key/value types; for
example, atom keys in {:map, :varchar, :varchar} columns become strings.
Declare VARIANT columns as :variant, so each value keeps its own shape,
and JSON columns as :json.
Options:
:columns-[{name, type}]column types, also naming the inserted columns. Omitted table columns get their defaults.:schema- the table's schema, or a nested schema path as a list.:batch_size- rows per request; requests carry vectors of at most 2048 rows.:on_conflict-:raise(default),:nothing, or{:nothing, targets}.:returning- columns to return; the result then carries those rows.
@spec insert_rows!( DBConnection.conn(), String.t() | atom(), [insert_row()], Keyword.t() ) :: QuackDB.Result.t()
@spec insert_stream( DBConnection.conn(), String.t() | atom(), Enumerable.t(), Keyword.t() ) :: {:ok, QuackDB.Result.t()} | {:error, Exception.t()}
Appends an enumerable of row maps/keywords in batches through native append.
@spec insert_stream!( DBConnection.conn(), String.t() | atom(), Enumerable.t(), Keyword.t() ) :: QuackDB.Result.t() | nil
Appends any Table.Reader compatible tabular data through native append.
@spec maps(DBConnection.conn(), iodata(), [term()], Keyword.t()) :: Enumerable.t()
@spec ping(DBConnection.conn(), Keyword.t()) :: :ok | {:error, Exception.t()}
@spec prepare(DBConnection.conn(), iodata(), Keyword.t()) :: {:ok, QuackDB.Query.t()} | {:error, Exception.t()}
@spec prepare_execute(DBConnection.conn(), iodata(), [term()], Keyword.t()) :: {:ok, QuackDB.Query.t(), QuackDB.Result.t()} | {:error, Exception.t()}
@spec query(DBConnection.conn(), iodata(), [term()], Keyword.t()) :: {:ok, QuackDB.Result.t()} | {:error, Exception.t()}
@spec query!(DBConnection.conn(), iodata(), [term()], Keyword.t()) :: QuackDB.Result.t()
@spec rows(DBConnection.conn(), iodata(), [term()], Keyword.t()) :: Enumerable.t()
@spec start_link([start_option()]) :: GenServer.on_start()
@spec stream(DBConnection.conn(), iodata(), [term()], Keyword.t()) :: QuackDB.Stream.t()