Conversation
The manual loop built `CREATE TABLE {table} AS SELECT * FROM __other.{table}`
without quoting the identifiers, so any table needing quotes ("my table",
"order") aborted the conversion. It also relied on `SHOW tables`, which
enumerates views too: the sqlite extension re-parses each view's SQL with the
duckdb parser, and MS Access style `[bracket]` quoting is a syntax error there.
`COPY FROM DATABASE` copies tables, data, constraints and indexes in one
statement, and listing tables through `duckdb_tables` skips views entirely.
`ATTACH` now states `(TYPE SQLITE, READ_ONLY)` rather than relying on magic
byte detection, which fails on an empty sqlite file.
Copying indexes needs duckdb >= 1.1.0 (1.0.0 raises NotImplementedException),
so the dependency floor moves up. Views are still not converted.
Fixes #2
Fixes #3
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XtZtVXPQqXvdHLQ98bT2gF
`uvx sqlite2duckdb source.db target.db` runs the tool without installing it. pip and `uv tool install` stay documented for a persistent install. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XtZtVXPQqXvdHLQ98bT2gF
Ships the COPY FROM DATABASE conversion: quoted identifiers and [bracket] quoted views now convert, constraints and indexes are preserved, and duckdb >= 1.1.0 is required. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XtZtVXPQqXvdHLQ98bT2gF
Attaching a sqlite database only exposes PRIMARY KEY and NOT NULL to duckdb; UNIQUE, FOREIGN KEY and CHECK are dropped at the ATTACH layer, before COPY FROM DATABASE runs. Views are not copied either. Claiming "relation and constraint" as done was too broad. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XtZtVXPQqXvdHLQ98bT2gF
chinook is the canonical sqlite demo database, and it quotes its DDL with [brackets], which duckdb's own parser rejects. Shipping it gives the test suite a real database that nobody wrote for the tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XtZtVXPQqXvdHLQ98bT2gF
pyproject declared `requres-python` instead of `requires-python`, so the floor was silently dropped from the wheel metadata and pip would install on any version. Fix the typo and raise it to 3.9, duckdb 1.1's own floor. Declare pytest and faker as a dev dependency group. tests/utils.py has always imported faker but nothing listed it, so a fresh clone could not run the suite. Drop requirements.txt, a stale duplicate of dependencies missing the >=1.1.0 floor that nothing consumed. Restrict the sdist to an allow list. Hatchling ships everything git does not ignore, which had put the 864K example database in it: 592K down to 13K. Add GitHub Actions running ruff and pytest on 3.9 through 3.13, and publishing on tags through PyPI trusted publishing rather than a manual twine upload. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XtZtVXPQqXvdHLQ98bT2gF
duckdb replays sqlite's index DDL verbatim through its own parser, which
rejects the [bracket] quoting that chinook.db and MS Access exports use, so
`python -m sqlite2duckdb chinook.db out.db` died with
Parser Error: syntax error at or near "["
The attached catalog is parsed on bind, so both the SCHEMA and the DATA halves
of COPY FROM DATABASE fail, and the sqlite extension exposes no setting to turn
index parsing off.
Recreate each table from the DDL duckdb derives for the attached catalog
instead. Unlike sqlite's own it always parses, and it still carries the primary
keys and the NOT NULL constraints that a CREATE TABLE AS SELECT would lose.
Then read the indexes back from sqlite_master and translate their quoting.
Along the way, make the function usable as a library: raise FileNotFoundError
and FileExistsError rather than a bare Exception, return a ConversionResult,
report progress through logging instead of printing to stdout, accept PathLike,
and delete a partially written target instead of leaving one behind that trips
the "already exists" guard on the next run.
The CLI grows --force, --quiet and --verbose, and only prompts when stdin is a
tty. It used to hang or raise EOFError under CI, in a pipe, or under
`uvx ... < /dev/null`.
Tests go from 8 to 32: value fidelity rather than row counts alone, the error
paths, the CLI end to end, the bracket quoted index, and chinook itself.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XtZtVXPQqXvdHLQ98bT2gF
The overwrite prompt, the duckdb >=1.1.0 requirement and the supported Python versions were undocumented, and views were only mentioned inside the constraints bullet despite being dropped silently. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XtZtVXPQqXvdHLQ98bT2gF
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
python -m sqlite2duckdb chinook.db out.dbcrashed withParser Error: syntax error at or near "[". Chasing that down surfaced a handful of other problems: the CLI could not run unattended, the Python floor was never applied, and there was no CI to catch any of it.The crash
duckdb replays sqlite's index DDL verbatim through its own parser, which rejects the
[bracket]quoting that chinook and MS Access exports use:The attached catalog is parsed on bind, so both the
SCHEMAand theDATAhalves ofCOPY FROM DATABASEfail, and the sqlite extension exposes no setting to turn index parsing off.duckdb_indexes()on the attached database raises the same error.So
COPY FROM DATABASEis gone. Each table is now recreated from the DDL duckdb derives for the attached catalog (duckdb_tables().sql), which always parses and still carries the primary keys and NOT NULL constraints that aCREATE TABLE AS SELECTwould lose. The indexes are then read back fromsqlite_masterand their quoting translated.This is a regression fix: the pre-
0bc556aloop never touched indexes, so chinook worked in 0.3.x.Also in here
Bugs.
requres-pythonwas a typo, so the>=3.8floor never reached the wheel metadata.importlib.metadata.version()raisedPackageNotFoundErroron an uninstalled checkout. A failed conversion left a half-written.duckdbbehind, which then tripped the "already exists" guard on the next run.Patharguments crashed.db_namecame from an unfilteredduckdb_databasesrow and was interpolated without escaping.API.
sqlite_to_duckdb(src, dst, *, overwrite=False)returns aConversionResult, raisesFileNotFoundError/FileExistsErrorinstead of bareException, acceptsPathLike, and reports throughloggingrather than printing to stdout.CLI.
--force,--quiet,--verbose. The overwrite prompt now only appears on a tty; it used to hang or raiseEOFErrorunder CI, in a pipe, or underuvx ... < /dev/null.Packaging. Everything moves to uv: dev dependency group (
fakerwas imported by the tests but declared nowhere, so a fresh clone could not run them), committeduv.lock,requirements.txtdropped. The sdist gets an allow list, taking it from 592K back to 13K.CI. ruff and pytest on 3.9 through 3.13, plus publish-on-tag through PyPI trusted publishing.
Issues
Closes #3 — that traceback died in
SHOW tableson a northwind export, the same[bracket]cause. There is now a regression test for it.Refs #1 — the reporter's exact
information_schema.table_constraintsquery returns 12 rows on a converted chinook instead of none. UNIQUE, FOREIGN KEY and CHECK are still not copied, so this may or may not be enough to close it.Refs #2 —
COPY FROM DATABASEwas adopted in0bc556aand is removed again here, for the reason above. The simplification still stands: theUSE/SHOW tablesdance the issue pointed at is long gone.Verification
8 tests to 32, including value fidelity rather than row counts alone, the error paths, the CLI end to end, and five integration tests against the real chinook now shipped in
examples/. They compare against the sqlite source rather than hardcoding: same table set, same row counts, same values, every named index present.CI is green on all five Python versions.
🤖 Generated with Claude Code
https://claude.ai/code/session_01XtZtVXPQqXvdHLQ98bT2gF