DuckDB

Available v2.15.0 Released October 31, 2026

Overview  

DuckDB is supported as both a source and a target. As a target, Amfelt loads and historizes data into an embedded DuckDB database file — a fast, zero-administration analytical warehouse that lives entirely inside your infrastructure. As a source, Amfelt reads an existing DuckDB file, so a warehouse Amfelt built can be read back and a DuckDB file produced elsewhere can be brought in. It suits development and test environments, edge and single-node deployments, and lightweight analytical marts that do not warrant a server-based warehouse.

DuckDB is an embedded, in-process database: there is no server to connect to. The Amfelt agent opens the database file directly, so the agent and the warehouse always run on the same host, and DuckDB’s single-writer model applies (Amfelt caps the connector at one connection automatically).

Capabilities  

Prerequisites  

Before connecting DuckDB:

• A writable location on the Amfelt agent’s host for the database file (e.g. /data/warehouse.duckdb). For a target, the agent creates the file if it does not exist.
• Disk space for the warehouse: staging, history, and bridge layers all live in the one file.
• Single writer: DuckDB allows one writing process at a time. Amfelt enforces max_connections: 1 on the target; no other process may hold the file open for writing during a load.
• No server, no credentials: DuckDB is embedded — there is no network endpoint and no login role. Access control is the file system’s.
• Read-only source access: a DuckDB source opens the database read-only. The file must already exist — a path that does not is an error, not a newly created empty database — and the source can never modify, or upgrade the format of, the file it reads.
• Sources and targets cannot share a file concurrently: DuckDB’s read-write lock is exclusive, so an extract cannot run while a load holds the same database file. Schedule jobs that read and write one file apart from each other.

Unsupported Source Column Types  

A DuckDB source cannot currently extract these column types. A table containing one fails at extraction rather than returning partial data:

• STRUCT and LIST — DuckDB’s nested types are not mapped yet. Support for them is planned; until it lands, flatten nested columns in a view and extract that instead.

Targets are unaffected: both write normally. Only extraction is blocked.

INTERVAL was listed here until the platform gained a logical category for the type. Interval columns now extract, and their months, days and nanoseconds are fingerprinted as separate components — a month is not a fixed number of days, so folding them into one scalar would make INTERVAL '1 month' and INTERVAL '30 days' indistinguishable and change detection would miss a row edited from one to the other.

JSON Numeric Precision  

JSON columns extract as text, but the DuckDB driver decodes JSON numbers through a 64-bit float before Amfelt sees them. An integer larger than 2^53 inside a JSON document loses precision — {"id":9007199254740993} extracts as {"id":9007199254740992}. Store large integers as their own typed column, not inside a JSON document, if exact values matter.

Connect  

1. In Amfelt, add a DuckDB target and select the DuckDB connector.
2. Provide the database property: the path to the database file on the agent host (see Configuration reference). The agent creates the file on first use.
3. Validate the connection and deploy the warehouse layers.

A connection resolves to an ODCS server descriptor like:

servers:
  - server: duckdb-local
    type: duckdb
    database: /data/warehouse.duckdb

Adding DuckDB as a Source  

1. In Amfelt, add a DuckDB source and select the DuckDB connector.
2. Provide the database property: the path to an existing database file on the agent host. Unlike the target, the source does not create the file — it opens the database read-only, so a path that does not exist is reported as an error.
3. Choose Discover schema to read the source’s schemas, objects, and properties.
4. Select the objects to extract and deploy the job.

The source performs full extracts and incremental extracts. An incremental extract reads only the rows whose cursor column is at or after the lower bound of the run. The cursor is a timestamp (a TIMESTAMP or TIMESTAMP WITH TIME ZONE column) or a sequence (a SMALLINT, INTEGER or BIGINT column); a cursor on a column of any other type fails the run instead of reading the object in full. A TIMESTAMP cursor is compared as a UTC wall clock, whatever the time zone of the agent host. There is no change-data-capture path for DuckDB.

Authentication  

DuckDB is an embedded database file: there are no credentials and no vault entry. Protect the warehouse by restricting file-system access to the database file and the agent’s service account.

Single-Writer Scheduling  

Only one connection writes at a time (max_connections: 1 on the target, enforced by the platform). Schedule jobs that target the same DuckDB file apart from each other.

State Schemas  

A target deploy that creates the history layer also creates the state schema <prefix>_sys in the database file. It holds what each history apply commits with its rows: the watermark of each object and the ledger of the files loaded. The first quarantined file of an object creates <prefix>_qrn, when it does not exist yet, and the object’s quarantine table in it, which holds the quarantined rows until they are purged. No bridge view reads either schema. DuckDB has no roles or grants, so whoever can open the file can read these schemas, as every other schema in it.

Sync Modes  

Target History Modes  

Configuration Reference  

Type Mapping  

Each DuckDB physical type maps to an Apache Arrow type (the data-plane intermediate) and then to an ODCS logical type. See the official DuckDB data type reference  .

Troubleshooting  

Changelog  

Release history for the DuckDB connector, newest first.

Licensing  

Third-party open-source packages bundled by this connector, with their licenses. Attribution only: informational, not an evaluation of the connector.