Skip to content

feat(database): add a WFL ORM and audited SQLite migrations - #16

Merged
logbie merged 33 commits into
mainfrom
codex/wfl-orm-migrations
Sep 20, 2026
Merged

logbie merged 33 commits into
mainfrom
codex/wfl-orm-migrations

Conversation

@logbie

@logbie logbie commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Scriptorium now uses a reusable WFL ORM and versioned SQLite migrations for all seven persistence tables. Fresh databases initialize through explicit migrations; supported unversioned databases are inspected and adopted while retaining records, installation state, and extension objects.

Every test scenario, fixture, helper, integration driver, and the complete runner is WFL. The converted suite includes real HTTP, migration interruption/concurrency, and database-plus-uploads recovery, with pinned Scribe coverage.

Changes

  • Add declarative models and typed records, consistent missing/NULL/default semantics, protected assignment, parameterized composed queries, ordered pagination, guarded bulk writes, conflict-targeted upserts, and bounded batched relationships.
  • Add immutable migration definitions, checksummed ledgers and append-only events; scaffold/status/plan/targeted-up/reversible-down commands; pinned native transactions, bounded locks, drift detection, safe legacy adoption, and SQLite rebuilds preserving rows, identities, sequences, constraints, indexes, triggers, and extension relationships.
  • Route users, sessions, posts, pages, settings, media, and login attempts through the ORM. Keep compatibility wrappers and document the specialized SQLite clock/year queries. Preserve the existing include tree, installer lockout, authentication/CSRF, uploads, routes, themes, hooks, data directory, and port behavior.
  • Replace the Python runner and all three Python suites with WFL equivalents, preserving the original behavior inventories. Provision official WFL on both governance platforms and run the complete suite on Blacksmith using a freshly pulled, resolved Docker image.
  • Provide an executable beginner-to-composed-query/relationship/transaction progression using explicit migrations throughout, API/operator documentation, independent reviews, and the acceptance audit.

Compatibility and risk

  • Risk class and reason: R3: schema evolution, authentication/installer persistence, uploads, and data recovery.
  • Affected contracts: All seven tables retain their established nullable/default behavior, identities, aliases, ordering, and application results. No new cascades or legacy-data cleanup are introduced. Scribe remains pinned to 93d62af5a6ed6c3ce257ef888107fc3ca1e2dc1d. Official WFL 26.9.16 at 23c1a4577da68d853fa30c49a17773427471eca4 provides the required upstream capabilities.
  • Upgrade and recovery: Stop writers; back up matching application/migration sources, runtime/configuration, database/sidecars, and uploads. Confirm the printed database target with wfl scripts/migrate.wfl status and plan, then apply up. Both shipped historical migrations are explicitly irreversible. Restore a complete matching backup or use a reviewed forward repair; recreating discarded data is not rollback. Inspect interrupted history/schema/integrity before resuming with the same immutable sources. See the migration and recovery guide.
  • Remaining risks or gaps: Schema equivalence checks deliberately reject unsupported declarations. Atomicity and locking apply per migration version, not to the entire multi-version command. Existing WFL analyzer limitations can emit nonfatal diagnostics; the independent analysis report documents them. This PR does not claim a clean static-analysis gate or production deployment verification.

Validation

  • Tested revision and environment: Final PR head fd1a65e5998d17788e22dc667f079679906d9945; GitHub merge checkout a223a9f20ad95d4f31a1051d6f1a67f851ef1b0c; Blacksmith Linux x64. Fresh official image bsbyrdwfl/wfl@sha256:1092a0c557731113f08673c764e0deb9e0b053992b4488974863f0a8d692db91, WFL 26.9.16, source 23c1a4577da68d853fa30c49a17773427471eca4; Scribe 93d62af5a6ed6c3ce257ef888107fc3ca1e2dc1d. Official publication.
  • Regression evidence: Retained behavioral Reds cover shared defaults, numeric/large identity handling, atomic installation, wrapper contracts, schema/rebuild safety, historical-index drift, and process diagnostics. The full runner's isolated deliberate WFL assertion failed the official-image CI job with 41 functional passes; removing that assertion produced the final 41-suite pass. Historical failed attempts remain identified in the verification record.
Check or exact command Result and evidence
wfl --execution-timeout 1200 scripts/run_tests.wfl on Blacksmith 41/41 suites passed, exit 0; application, ORM/migrations/recovery, real HTTP, tooling, runtime probes, executable examples, and pinned Scribe. Final CI.
Isolated complete-runner failure proof 42 suites: 41 passed, one deliberate assertion failed, exit 1; only ci-propagation.test.wfl failed. Red CI. The deliberate test is removed from this final revision. Nested negative fixtures are checked by the passing runner suite and are not additional top-level failures.
Governance on Linux and Windows Three WFL tooling suites passed: hygiene 28, migration CLI 4, runner 9 cases; repository hygiene passed with 179 paths. Governance.
Focused runtime/media gates All six jobs passed, including portable HTTP 413 with no persisted rows/files and a successful subsequent authenticated request. Capability gates.
Official Windows complete suite 41/41 suites passed, exit 0, on source 304a5fdb0c63be5118ae13dbab4c929b699d4076; final head only removes three empty EOF lines. Verified official executable SHA256 32375058e0111f6c159144a8ffa9bd5b3a3bde81f578eca015149053ea4c6cbb; local log target/full-official-26.9.16-green.log. Final full-diff whitespace check passed and the worktree is clean.
Migration, HTTP, and recovery boundaries The full gate executes file-backed adoption/rebuild/rollback, native read-only/lock errors, killed-owner restart, competing runners, ledger/data preservation, extension/theme compatibility, auth/CSRF, content/users/settings/media/throttling, and stopped database-plus-uploads restore.
Independent technical review Independent agents reviewed foundation/API alignment, SQL safety, ownership, migration adoption/rebuild/recovery, integration, and WFL test conversion; findings were fixed and affected checks rerun. Final audit found no remaining implementation blocker. CodeRabbit skipped automatic review because 118 files exceed its 100-file limit; its successful status is not review approval.
Upstream prerequisites Reviewed and separately approved WFL #737, #738, #739, and #741 are merged and included in the verified official runtime. Linux/Windows runtime gates and real 305-second boundary tests passed.

Checklist

  • The title, summary, and risk assessment match the final diff.
  • Required validation is recorded above; historical failures and remaining analysis limitations are explicit.
  • Documentation, executable examples, and upgrade/recovery guidance are updated.
  • The diff was reviewed for repository hygiene, secrets, and private site data.

Technical review and final CI are complete. Scriptorium merge and production deployment remain separate maintainer actions.

@coderabbitai

coderabbitai Bot commented Sep 20, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 118 files, which is 18 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 6d145fe7-1b2d-48f4-8cbd-71f03a70c77c

📥 Commits

Reviewing files that changed from the base of the PR and between 4ec5c88 and fd1a65e.

📒 Files selected for processing (118)
  • .github/workflows/governance.yml
  • .github/workflows/runtime-capabilities.yml
  • .github/workflows/update-scribe.yml
  • .github/workflows/wfl-tests.yml
  • .repo-hygiene.toml
  • .wflcfg
  • CLAUDE.md
  • CONTRIBUTING.md
  • README.md
  • REPOSITORY_HYGIENE.md
  • TestPrograms/.wflcfg
  • TestPrograms/db-contracts.test.wfl
  • TestPrograms/db-review-contracts.test.wfl
  • TestPrograms/migration-adoption.test.wfl
  • TestPrograms/migration-engine.test.wfl
  • TestPrograms/migration-failures.test.wfl
  • TestPrograms/migration-index-safety.test.wfl
  • TestPrograms/migration-legacy-matrix.test.wfl
  • TestPrograms/migration-rebuild.test.wfl
  • TestPrograms/migration-schema-safety.test.wfl
  • TestPrograms/orm-crud.test.wfl
  • TestPrograms/orm-models.test.wfl
  • TestPrograms/orm-predicates.test.wfl
  • TestPrograms/orm-query-order.test.wfl
  • TestPrograms/orm-records.test.wfl
  • TestPrograms/orm-relationship-limits.test.wfl
  • TestPrograms/orm-relationships.test.wfl
  • TestPrograms/orm-types.test.wfl
  • TestPrograms/orm-write-review.test.wfl
  • app/auth.wfl
  • app/db.wfl
  • app/migrations.wfl
  • app/migrations/20200101000000_initial.wfl
  • app/migrations/20200102000000_session_csrf.wfl
  • app/models.wfl
  • app/persistence-helpers.wfl
  • app/render.wfl
  • docs/ARCHITECTURE.md
  • docs/independent-migration-review.md
  • docs/migration-evidence.md
  • docs/migrations.md
  • docs/orm-acceptance-audit.md
  • docs/orm-implementation-evidence.md
  • docs/orm-independent-review.md
  • docs/orm-migrations-design.md
  • docs/orm-verification.md
  • docs/orm.md
  • docs/persistence-inventory.md
  • docs/runtime-capabilities.md
  • docs/runtime-review.md
  • docs/type-analysis-review.md
  • docs/wfl-test-inventory.md
  • docs/wfl-tooling-validation.md
  • examples/orm/.wflcfg
  • examples/orm/migrations/20260920000000_initial.wfl
  • examples/orm/migrations/20260920000100_author_index.wfl
  • examples/orm/models.wfl
  • examples/orm/progression.test.wfl
  • lib/orm/connections.wfl
  • lib/orm/migrations.wfl
  • lib/orm/models.wfl
  • lib/orm/predicates.wfl
  • lib/orm/queries.wfl
  • lib/orm/records.wfl
  • lib/orm/relationships.wfl
  • lib/orm/schema.wfl
  • lib/orm/types.wfl
  • lib/orm/writes.wfl
  • main.wfl
  • scripts/.wflcfg
  • scripts/migrate.wfl
  • scripts/resolve_runtime.wfl
  • scripts/run_tests.py
  • scripts/run_tests.wfl
  • scripts/test_support.wfl
  • testing.md
  • tests/fixtures/legacy-database.wfl
  • tests/fixtures/migration-registry.wfl
  • tests/fixtures/migration-worker.wfl
  • tests/fixtures/orm-model.wfl
  • tests/integration/.wflcfg
  • tests/integration/EVIDENCE.md
  • tests/integration/README.md
  • tests/integration/authentication.test.wfl
  • tests/integration/content-users.test.wfl
  • tests/integration/fixtures/.wflcfg
  • tests/integration/fixtures/http-helpers.wfl
  • tests/integration/fixtures/legacy-extension.wfl
  • tests/integration/fixtures/legacy-site.wfl
  • tests/integration/fixtures/request.wfl
  • tests/integration/legacy-upgrade.test.wfl
  • tests/integration/media.test.wfl
  • tests/integration/migrations-recovery.test.wfl
  • tests/integration/recovery.test.wfl
  • tests/integration/server-port.test.wfl
  • tests/integration/test_server_port.py
  • tests/integration/throttle.test.wfl
  • tests/runtime/.wflcfg
  • tests/runtime/fixtures/http-redirect.wfl
  • tests/runtime/fixtures/process-blocking.wfl
  • tests/runtime/fixtures/process-child.wfl
  • tests/runtime/fixtures/process-error.wfl
  • tests/runtime/http-redirect-capability.test.wfl
  • tests/runtime/orm-native-baseline.test.wfl
  • tests/runtime/process-capabilities.test.wfl
  • tests/runtime/rebuild-foreign-keys-capability.test.wfl
  • tests/runtime/transaction-validation-capability.test.wfl
  • tests/tooling/.wflcfg
  • tests/tooling/fixtures/delayed-write.wfl
  • tests/tooling/fixtures/runner-error.wfl
  • tests/tooling/fixtures/runner-fail.wfl
  • tests/tooling/fixtures/runner-pass.wfl
  • tests/tooling/fixtures/runner-timeout.wfl
  • tests/tooling/hygiene.test.wfl
  • tests/tooling/migrations.test.wfl
  • tests/tooling/runner.test.wfl
  • tests/tooling/test_repo_hygiene.py
  • tests/tooling/test_run_tests.py

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@blacksmith-sh

This comment has been minimized.

@logbie
logbie marked this pull request as ready for review September 20, 2026 13:52

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@logbie
logbie merged commit 96f4e91 into main Sep 20, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant