Skip to content

Extract versioned-SQL generation into sql.mk; install generated upgrade scripts (fixes #28)#32

Open
jnasbyupgrade wants to merge 1 commit into
Postgres-Extensions:masterfrom
jnasbyupgrade:fix-28-install-upgrade-scripts
Open

Extract versioned-SQL generation into sql.mk; install generated upgrade scripts (fixes #28)#32
jnasbyupgrade wants to merge 1 commit into
Postgres-Extensions:masterfrom
jnasbyupgrade:fix-28-install-upgrade-scripts

Conversation

@jnasbyupgrade

@jnasbyupgrade jnasbyupgrade commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

What this is

This PR is a pure build-system refactor: it extracts the versioned-SQL generation machinery out of the Makefile into a new, documented sql.mk, and fixes #28 so generated upgrade scripts land in DATA reliably on a clean build.

Dependency — READ THIS: The #18 pg_upgrade source reconcile that used to be bundled into this PR has been pulled out into its own PR: #38 (fix-omit-source). This refactor regenerates the current-version file in place from sql/cat_tools.sql.in; on master that source still lacks the #18 omit_column additions, so until #38 merges to master and this branch is rebased onto it, this PR's binary pg_upgrade CI legs will be RED. That is expected and correct — the fix lives in #38. Do not merge this before #38; after #38 merges, rebase this branch onto master and the pg_upgrade legs go green.

What moved into sql.mk (new)

Included after include pgxntool/base.mk, sql.mk now owns:

  • B, versioned_in/versioned_out, upgrade_scripts_out
  • historical_installs — the committed cat_tools--0.1.* install list, derived (glob the committed install-shaped .sql, subtract update scripts and everything generated from .sql.in) instead of the old hardcoded list
  • the DATA handling, including the Clean-build 'make test' fails on all PG>=12: generated upgrade scripts excluded from DATA (parse-time wildcard) #28 fix (DATA += $(upgrade_scripts_out) then DATA := $(sort $(DATA)))
  • all / installcheck / EXTRA_CLEAN
  • the SED version-marker machinery (LT95/LT93, _apply_version_seds), the $B/%.sql pattern rule, and the EXTENSION_VERSION_FILES rules

The Makefile now just include sql.mk (with a pointer comment) and keeps its non-SQL-gen content (the upgrade-test TEST_BUILD_DIR block, old_version/clean_old_version).

Why #28 happens (documented in sql.mk's header)

GNU Make is two-phase. We generate the .sql we install; the generated files are gitignored and absent on a clean tree when the makefile is parsed. Any parse-time $(wildcard) over them — including base.mk's DATA seed $(wildcard sql/*--*--*.sql) — silently comes up short, and Make's cached directory listing means it never notices them later. So on a fresh build the generated update scripts never make it into DATA, make install skips them, and a later ALTER EXTENSION cat_tools UPDATE fails with "no update path". The fix: name-derive the lists from the .sql.in sources, feed the generated names into DATA explicitly, and $(sort). Rule of thumb: glob what git tracks, name-derive what we generate.

Clean-tree #28 proof

On a truly clean tree (make clean, no generated 0.2.* .sql present):

  • Before (upstream/master): none of the generated 0.2.0--0.2.1, 0.2.0--0.2.2, 0.2.1--0.2.2 upgrade scripts appear in DATA.
  • After (this branch): all three appear in DATA, and a clean make install copies all of them into the extension directory.

make print-DATA is byte-identical on a clean tree vs. a fully-built tree (parse-stable), and historical_installs resolves to exactly the committed cat_tools--0.1.0/0.1.3/0.1.4/0.1.5.sql.

@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 143aa8c8-dd6b-485e-a29f-079c5e8a2996

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@jnasbyupgrade jnasbyupgrade force-pushed the fix-28-install-upgrade-scripts branch 2 times, most recently from 2136cb5 to cb4c207 Compare July 15, 2026 23:11
@jnasbyupgrade jnasbyupgrade changed the title Install generated upgrade scripts on a clean build (fixes #28) Extract versioned-SQL generation into sql.mk; install generated upgrade scripts (fixes #28) Jul 15, 2026
@jnasbyupgrade jnasbyupgrade force-pushed the fix-28-install-upgrade-scripts branch from cb4c207 to 077c091 Compare July 15, 2026 23:18
…de scripts (fixes Postgres-Extensions#28)

The machinery that generates our versioned SQL (the cat_tools--X.Y.Z.sql
install scripts and cat_tools--A.B.C--X.Y.Z.sql update scripts) from .sql.in
sources, and the DATA list that installs it, is moved out of the Makefile into
a new, documented sql.mk (included after pgxntool/base.mk so it can use
EXTENSION_VERSION_FILES, PG_CONFIG, MAJORVER, datadir, ...).

sql.mk documents the GNU Make two-phase (parse vs. recipe) hazard at the root
of Postgres-Extensions#28: we GENERATE the .sql we install, the generated files are gitignored and
absent on a clean tree at parse time, and any parse-time $(wildcard) over them
(including base.mk's DATA seed $(wildcard sql/*--*--*.sql)) silently comes up
short -- the cached directory listing means it never notices them later either.
The fix is to name-derive the install/update lists from the .sql.in SOURCES
(which DO exist at parse time), feed the generated names into DATA explicitly,
and $(sort) to dedup against base.mk's own wildcard. This makes the generated
upgrade scripts land in DATA reliably on a clean build, so `make install`
copies them and a later `ALTER EXTENSION cat_tools UPDATE` finds its path (Postgres-Extensions#28).

The historical cat_tools--0.1.* install list is no longer hardcoded: it is
derived by globbing the committed install-shaped .sql and subtracting the
update scripts and everything we generate from .sql.in. Globbing COMMITTED
files at parse time is safe; the rule of thumb is glob what git tracks,
name-derive what we generate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jnasbyupgrade jnasbyupgrade force-pushed the fix-28-install-upgrade-scripts branch from 077c091 to f56c8bb Compare July 16, 2026 00:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant