Skip to content

Copy views and UNIQUE constraints (0.5.0) - #6

Merged
dridk merged 4 commits into
mainfrom
dev
Aug 30, 2026
Merged

Copy views and UNIQUE constraints (0.5.0)#6
dridk merged 4 commits into
mainfrom
dev

Conversation

@dridk

@dridk dridk commented Aug 30, 2026

Copy link
Copy Markdown
Owner

What this adds

duckdb's sqlite extension exposes neither views nor UNIQUE constraints on an attached database, so both are read back from the source with the sqlite3 module, the way the indexes already were.

Views come from sqlite_master with their SQL, translated out of sqlite's [bracket] quoting and replayed as CREATE VIEW. They land as genuine duckdb views: they show up in duckdb_views(), duckdb re-serialises the SQL as its own, and they recompute when the underlying table changes.

Two wrinkles handled:

  • Their SQL is sqlite's, not duckdb's. A view using something duckdb has no equivalent for (MATCH, julianday()) is skipped with a warning rather than failing the whole conversion.
  • A view sitting on another view works whatever order sqlite_master lists them in: the pass retries while it still makes progress.

UNIQUE needs a second route. A column or table level UNIQUE becomes an autoindex whose sqlite_master row carries no SQL at all, so it is invisible to the index pass. PRAGMA index_list reports it with origin = 'u' and index_info gives the columns. duckdb has no ALTER TABLE ADD CONSTRAINT, so it is replayed as a unique index, which enforces the same guarantee — the tests check that by actually trying to insert duplicates.

ConversionResult grows a views count.

Still not copied

FOREIGN KEY and CHECK. The reason first written down in 79027c8 was wrong and 61c5d00 corrects it: it is not that duckdb cannot express them. It accepts and enforces both in CREATE TABLE, the clauses inject cleanly into the DDL it already derives, and PRAGMA foreign_key_list reports them structured with no SQL to parse.

The real obstacle is loading order. duckdb checks foreign keys row by row, so a self-referencing table cannot be bulk loaded. Prototyped on chinook, nine of the thirteen tables load with their foreign keys intact; employees.ReportsTo fails and takes customers, invoices and invoice_items with it. Closing that gap needs multi pass inserts, and there is no ALTER TABLE ADD CONSTRAINT to add the keys after the data.

Verification

32 tests to 38, including a deliberately untranslatable view checking the graceful degradation, and duplicate inserts checking that UNIQUE is really enforced rather than merely present. CI green on 3.9 through 3.13.

Version 0.4.0 to 0.5.0.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XtZtVXPQqXvdHLQ98bT2gF

sacha and others added 4 commits August 30, 2026 17:12
duckdb's sqlite extension exposes neither on an attached database, so both are
read back from the source with the sqlite3 module, the way the indexes already
were.

Views come from sqlite_master with their SQL, translated out of sqlite's
[bracket] quoting. Their SQL is sqlite's, not duckdb's, so one using a construct
duckdb has no equivalent for is skipped with a warning instead of failing the
whole conversion. A view sitting on another view works whatever order
sqlite_master lists them in: the pass retries while it still makes progress.

UNIQUE needs a second route. A column or table level UNIQUE becomes an
autoindex whose sqlite_master row carries no SQL at all, so it is invisible to
the index pass; PRAGMA index_list reports it with origin 'u' and index_info
gives the columns. duckdb has no ALTER TABLE ADD CONSTRAINT, so it is replayed
as a unique index, which enforces the same guarantee.

That leaves FOREIGN KEY and CHECK, for the same missing ALTER TABLE: carrying
them over would mean generating the whole CREATE TABLE by hand, with a type
mapping of our own, rather than reusing the one duckdb derives.

ConversionResult grows a views count. Bump to 0.5.0, since 0.4.0 is released.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XtZtVXPQqXvdHLQ98bT2gF
Keep main's trimmed wording and drop the explanatory paragraphs, but bring the
"What is converted" table and the Todo up to date with the views and UNIQUE
constraints that dev now copies.

Also ignore *.db at the repository root, where converting a database for a
quick check leaves one behind. The pattern is anchored so that the example
database stays tracked.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XtZtVXPQqXvdHLQ98bT2gF
The reason given in 79027c8 was wrong. It claimed carrying them over would mean
generating the whole CREATE TABLE by hand with a type mapping of our own. It
would not: duckdb accepts and enforces both in CREATE TABLE, and the clauses
inject cleanly into the DDL it already derives for the attached database.
PRAGMA foreign_key_list even reports them structured, with no SQL to parse.

The real obstacle is the loading order. duckdb checks foreign keys row by row,
so a self-referencing table cannot be bulk loaded: on chinook, employees.ReportsTo
fails and takes customers, invoices and invoice_items down with it. Nine of the
thirteen tables load with their foreign keys intact. Getting the last four in
needs multi pass inserts, and no ALTER TABLE ADD CONSTRAINT exists to add the
keys after the data.

Correct the README and the docstring. No behaviour change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XtZtVXPQqXvdHLQ98bT2gF
What was left on it is already stated right above, in the table of what gets
converted and in the paragraph explaining why foreign keys and checks are not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XtZtVXPQqXvdHLQ98bT2gF
@dridk
dridk merged commit 23c219f into main Aug 30, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant