Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,15 +33,18 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: Test consumer POM release checks
run: python3 -m unittest discover -s scripts -p 'test_consumer_poms.py'
- name: Test release tooling
run: python3 -m unittest discover -s scripts -p 'test_*.py'

- uses: actions/setup-java@v4
with:
distribution: temurin
java-version: ${{ matrix.java }}
cache: maven

- name: Install the pinned Maven toolchain
run: bash scripts/setup-maven.sh

- name: Compile and run unit tests
run: mvn $MAVEN_ARGS -pl chdb-jdbc -am test

Expand All @@ -63,6 +66,8 @@ jobs:
distribution: temurin
java-version: "26"
cache: maven
- name: Install the pinned Maven toolchain
run: bash scripts/setup-maven.sh
- run: mvn $MAVEN_ARGS -pl chdb-jdbc -am test

# The real matrix: build the shim, stage the platform package, run the integration tests
Expand Down Expand Up @@ -119,6 +124,9 @@ jobs:
java-version: "11"
cache: maven

- name: Install the pinned Maven toolchain
run: bash scripts/setup-maven.sh

# ubuntu-22.04 rather than 24.04 on purpose: the shim's glibc and libstdc++ requirements
# are whatever the build image has, so building on the oldest supported image is what
# makes the package usable on older distributions.
Expand Down Expand Up @@ -362,6 +370,9 @@ jobs:
path: target/engine/download
key: chdb-engine-${{ matrix.platform }}-${{ hashFiles('scripts/engine.properties') }}

- name: Install the pinned Maven toolchain
run: bash scripts/setup-maven.sh

- name: Build the Java side and resolve the test classpath
run: |
mvn $MAVEN_ARGS -pl chdb-integration-tests -am test-compile dependency:build-classpath -Dmdep.outputFile="$PWD/target/it-cp.txt" -Dmdep.includeScope=test
Expand Down Expand Up @@ -443,6 +454,9 @@ jobs:
java-version: "21"
cache: maven

- name: Install the pinned Maven toolchain
run: bash scripts/setup-maven.sh

- name: Cache the pinned engine download
uses: actions/cache@v4
with:
Expand Down
9 changes: 9 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,9 @@ jobs:
java-version: "11"
cache: maven

- name: Install the pinned Maven toolchain
run: bash scripts/setup-maven.sh

- name: Resolve the version and the channel
id: resolve
env:
Expand Down Expand Up @@ -261,6 +264,9 @@ jobs:
java-version: "11"
cache: maven

- name: Install the pinned Maven toolchain
run: bash scripts/setup-maven.sh

- name: Cache the pinned engine download
uses: actions/cache@v4
with:
Expand Down Expand Up @@ -384,6 +390,9 @@ jobs:
# reads. The release profile passes --pinentry-mode loopback so it never wants a tty.
gpg-passphrase: MAVEN_GPG_PASSPHRASE

- name: Install the pinned Maven toolchain
run: bash scripts/setup-maven.sh

- name: Download every staged platform package
uses: actions/download-artifact@v4
with:
Expand Down
31 changes: 31 additions & 0 deletions docs/release-readiness.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,37 @@ restating.

---

## Snapshot validation checkpoint — 2026-11-19

PR #45 is merged at `7649ef3`. The `maven-central` environment now exists with the four
expected secrets, and the maintainer has admin access. The release owner explicitly waived
the one-week RC waiting window for 1.1.0 and deferred required environment reviewers and
disabling administrator bypass. Other release gates are not waived.

- Signed four-platform dry run `44269092646`, attempt 2, succeeded. All 24 signatures were
produced; the six POM signatures were independently verified against the published key.
- Real snapshot run `44272921635`, attempt 1, failed while uploading to Central with HTTP
502, including a JDBC signature and metadata checksums. Partial metadata is visible, but
referenced artifacts are missing; snapshot publishing and consumption are not complete.
- Attempt 2 ran on an Ubuntu ARM64 runner and stopped with `mvn: command not found`, exit
127, before Maven or another Central upload began. This is a separate tooling failure,
not evidence that the 502 is resolved or that signing credentials are wrong.

The local tooling fix explicitly bootstraps Apache Maven 3.9.16 from a pinned, SHA-512-verified
Apache distribution in build and release jobs. `setup-java` supplies Java, not Maven; runner
labels alone must not be treated as a Maven installation contract.

Local validation passed: all 12 release-tooling regression tests (including installation
without system Maven, failed downloads, checksum rejection, and the macOS checksum fallback),
ShellCheck and workflow structure checks. The actual Apache archive was downloaded and
verified with Homebrew excluded from PATH, then the installed Maven ran 178 JDBC unit tests
and one Arrow unit test successfully on Java 11. The 29 native Arrow tests were not enabled
in that local toolchain smoke test. CI validation of the tooling fix is still pending.

After the fix is reviewed and merged, start a new snapshot workflow at the new commit.
Rerunning the old workflow again would still use its old checkout without the tooling fix.
Each new push, PR creation, merge, or actual publishing attempt needs its own confirmation.

## Central preparation checkpoint — 2026-11-18

The Central path uses plan A: six JAR modules (JDBC, Arrow and four native platforms),
Expand Down
35 changes: 35 additions & 0 deletions scripts/setup-maven.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/usr/bin/env bash
# Runner labels do not guarantee Maven is installed (including ubuntu-latest ARM64).
# Download only the pinned Apache distribution and verify it before executing anything.
set -euo pipefail

MAVEN_VERSION=3.9.16
# Apache's apache-maven-3.9.16-bin.tar.gz.sha512, pinned alongside the version.
MAVEN_SHA512=831a8591fe20c8243b1dbe7d71e3244f31d1665b0804b2e825e38cbbe5ce0cafb8338851f90780735568773e0a6cd07bbec107cda0b896b008b861075358b6f6

: "${RUNNER_TEMP:?RUNNER_TEMP must identify the runner temporary directory}"
: "${GITHUB_PATH:?GITHUB_PATH must identify the Actions path file}"

MAVEN_SETUP_DIR="$(mktemp -d "${RUNNER_TEMP%/}/chdb-maven.XXXXXX")"
MAVEN_ARCHIVE="$MAVEN_SETUP_DIR/apache-maven-$MAVEN_VERSION-bin.tar.gz"
MAVEN_URL="https://archive.apache.org/dist/maven/maven-3/$MAVEN_VERSION/binaries/apache-maven-$MAVEN_VERSION-bin.tar.gz"

printf 'Installing Apache Maven %s from its pinned distribution\n' "$MAVEN_VERSION"
curl --disable --fail --location --proto '=https' --proto-redir '=https' \
--silent --show-error --retry 3 --connect-timeout 15 --max-time 180 \
--output "$MAVEN_ARCHIVE" "$MAVEN_URL"

if command -v sha512sum >/dev/null 2>&1; then
MAVEN_ACTUAL_SHA512="$(sha512sum "$MAVEN_ARCHIVE" | awk '{print $1}')"
else
MAVEN_ACTUAL_SHA512="$(shasum -a 512 "$MAVEN_ARCHIVE" | awk '{print $1}')"
fi
if [ "$MAVEN_ACTUAL_SHA512" != "$MAVEN_SHA512" ]; then
printf 'Maven archive SHA-512 mismatch; refusing to extract or execute it.\n' >&2
exit 1
fi

tar -xzf "$MAVEN_ARCHIVE" -C "$MAVEN_SETUP_DIR"
MAVEN_BIN="$MAVEN_SETUP_DIR/apache-maven-$MAVEN_VERSION/bin"
"$MAVEN_BIN/mvn" --version
printf '%s\n' "$MAVEN_BIN" >> "$GITHUB_PATH"
107 changes: 107 additions & 0 deletions scripts/test_setup_maven.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
"""Exercise the Maven bootstrap without network or a preinstalled Maven."""

import os
from pathlib import Path
import re
import shutil
import subprocess
import tempfile
import unittest

INSTALLER = Path(__file__).with_name("setup-maven.sh")
PIN = re.search(r"^MAVEN_SHA512=([0-9a-f]{128})$", INSTALLER.read_text(), re.M).group(1)


class MavenSetupChecks(unittest.TestCase):
def setUp(self):
self.work = tempfile.TemporaryDirectory(prefix="chdb-maven-test-")
self.addCleanup(self.work.cleanup)
self.root = Path(self.work.name)
self.bin = self.root / "bin"
self.bin.mkdir()
self.path_file = self.root / "github-path"
self.path_file.touch()
self.env = {
**os.environ,
"RUNNER_TEMP": str(self.root),
"GITHUB_PATH": str(self.path_file),
"PATH": f"{self.bin}:/usr/bin:/bin",
"FAKE_SHA512": PIN,
"FAKE_ROOT": str(self.root),
}
self.command("mvn", 'echo "untrusted system Maven must not run" >&2; exit 99')
self.command("curl", '''
touch "$FAKE_ROOT/download-called"
if [ "${FAKE_DOWNLOAD_FAIL:-0}" = 1 ]; then exit 22; fi
while [ "$#" -gt 0 ]; do
if [ "$1" = --output ]; then printf archive > "$2"; exit 0; fi
shift
done
exit 1
''')
self.command("sha512sum", 'printf "%s %s\\n" "$FAKE_SHA512" "$1"')
self.command("tar", '''
touch "$FAKE_ROOT/extract-called"
while [ "$#" -gt 0 ]; do
if [ "$1" = -C ]; then destination="$2"; break; fi
shift
done
mkdir -p "$destination/apache-maven-3.9.16/bin"
printf '#!/bin/bash\\nprintf "Apache Maven 3.9.16\\\\n"\\n' > "$destination/apache-maven-3.9.16/bin/mvn"
chmod +x "$destination/apache-maven-3.9.16/bin/mvn"
''')

def command(self, name, body):
command = self.bin / name
command.write_text("#!/bin/bash\nset -eu\n" + body + "\n")
command.chmod(0o755)

def run_setup(self):
return subprocess.run(["/bin/bash", str(INSTALLER)], env=self.env,
text=True, capture_output=True, timeout=15)

def test_installs_without_using_system_maven(self):
result = self.run_setup()
self.assertEqual(0, result.returncode, result.stderr)
self.assertIn("Apache Maven 3.9.16", result.stdout)
installed_bin = Path(self.path_file.read_text().strip())
self.assertTrue(installed_bin.is_relative_to(self.root))
self.assertTrue((installed_bin / "mvn").is_file())

def test_checksum_mismatch_stops_before_extraction(self):
self.env["FAKE_SHA512"] = "0" * 128
result = self.run_setup()
self.assertNotEqual(0, result.returncode)
self.assertIn("SHA-512 mismatch", result.stderr)
self.assertFalse((self.root / "extract-called").exists())
self.assertEqual("", self.path_file.read_text())

def test_shasum_fallback_without_sha512sum(self):
(self.bin / "sha512sum").unlink()
self.command("shasum", 'printf "%s %s\\n" "$FAKE_SHA512" "$3"')
for name in ("mktemp", "awk", "touch", "mkdir", "chmod"):
(self.bin / name).symlink_to(shutil.which(name))
self.env["PATH"] = str(self.bin)
result = self.run_setup()
self.assertEqual(0, result.returncode, result.stderr)
self.assertTrue(self.path_file.read_text().strip())

def test_failed_download_does_not_extract_or_update_path(self):
self.env["FAKE_DOWNLOAD_FAIL"] = "1"
self.assertNotEqual(0, self.run_setup().returncode)
self.assertFalse((self.root / "extract-called").exists())
self.assertEqual("", self.path_file.read_text())

def test_missing_runner_context_fails_before_network(self):
for variable in ("RUNNER_TEMP", "GITHUB_PATH"):
with self.subTest(variable=variable):
original = self.env.pop(variable)
try:
self.assertNotEqual(0, self.run_setup().returncode)
self.assertFalse((self.root / "download-called").exists())
finally:
self.env[variable] = original


if __name__ == "__main__":
unittest.main()
Loading