Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: Test consumer POM release checks
run: python3 -m unittest discover -s scripts -p 'test_consumer_poms.py'

- uses: actions/setup-java@v4
with:
distribution: temurin
Expand Down Expand Up @@ -279,8 +282,8 @@ jobs:

- name: Consume the driver as a repository artifact
# Issue #10: nothing had ever depended on this as a published artifact, and a reactor
# build cannot find the things a consumer hits -- the POMs come from target/, the BOM is
# never imported, and the native libraries are read from a directory rather than
# build cannot find the things a consumer hits -- reactor POMs can hide broken
# published dependencies, and native libraries are read from a directory rather than
# unpacked from a JAR in a local repository.
#
# The script deploys to a throwaway file:// repository and resolves from it in a project
Expand Down
18 changes: 11 additions & 7 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -482,15 +482,17 @@ jobs:
# chdb-integration-tests are not published, and the integration suite has already run
# on each platform's own runner, where it could actually exercise that platform.
#
# `verify`, not `package`, for the dry run: signing is bound to verify, so `package`
# would silently produce no .asc files and prove nothing.
# Exercise the deploy lifecycle even in a dry run. Plugin 0.12.0 skipUpload
# suppresses both release uploads and deferred snapshot uploads.
run: |
set -euo pipefail
GOAL=deploy
if [ "$DRY_RUN" = "true" ]; then GOAL=verify; fi
echo "mvn -Prelease $GOAL for $VERSION ($CHANNEL)"
mvn $MAVEN_ARGS -Prelease "$GOAL" \
-pl .,chdb-bom,chdb-jdbc,chdb-arrow,chdb-native-macos-aarch64,chdb-native-macos-x86_64,chdb-native-linux-x86_64-gnu,chdb-native-linux-aarch64-gnu
MODULES=chdb-jdbc,chdb-arrow,chdb-native-macos-aarch64,chdb-native-macos-x86_64,chdb-native-linux-x86_64-gnu,chdb-native-linux-aarch64-gnu
# Validate flattened POMs before anything can be uploaded. The deploy invocation
# regenerates them from the same checkout using the same release profile.
mvn $MAVEN_ARGS -Prelease process-resources -pl "$MODULES" -am
python3 scripts/check-consumer-poms.py --version "$VERSION"
mvn $MAVEN_ARGS -Prelease deploy -DskipUpload="$DRY_RUN" \
-pl ".,$MODULES"

- name: What was produced
if: always()
Expand Down Expand Up @@ -534,7 +536,9 @@ jobs:
path: |
**/target/*.asc
**/target/*.pom
**/.flattened-pom.xml
**/target/native/META-INF/chdb/native/*/*/manifest.properties
**/target/native/META-INF/chdb/native/*/*/sha256sums.txt
if-no-files-found: warn
include-hidden-files: true
retention-days: 90
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
target/
build/
*.class
.flattened-pom.xml
__pycache__/

# Native artefacts: 350 MB of engine and a built shim. Produced by
# scripts/build-native.sh into <module>/target/native, never committed.
Expand Down
2 changes: 1 addition & 1 deletion CHDB_JAVA_V1_WORK_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -518,7 +518,7 @@ Exit condition: a Java user who knows nothing about the implementation can insta

- [ ] Publish a Maven snapshot.
- [ ] Have at least two external Java projects try the snapshot.
- [ ] Verify the POM transitive dependencies and the BOM.
- [ ] Verify standalone consumer POMs and transitive dependencies without an internal parent or BOM (Central plan A).
- [ ] Verify sources, javadoc, signatures, licences and the SBOM.
- [ ] Verify that a Maven Central download checksums identically to the CI artifact.
- [ ] Verify the real download and first load on all four platforms.
Expand Down
25 changes: 11 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,35 +77,32 @@ Building for several platforms — a CI matrix, or a distribution your users ins
architecture — declare the driver plus each native package you need:

```xml
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.clickhouse.chdb</groupId>
<artifactId>chdb-bom</artifactId>
<version>1.1.0-rc.1</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<properties>
<chdb.version>1.1.0-rc.1</chdb.version>
</properties>

<dependencies>
<dependency>
<groupId>com.clickhouse.chdb</groupId>
<groupId>com.clickhouse.chdb</groupId>
<artifactId>chdb-jdbc</artifactId>
<version>${chdb.version}</version>
</dependency>
<dependency>
<groupId>com.clickhouse.chdb</groupId>
<artifactId>chdb-native-linux-x86_64-gnu</artifactId>
<version>${chdb.version}</version>
</dependency>
<dependency>
<groupId>com.clickhouse.chdb</groupId>
<artifactId>chdb-native-macos-aarch64</artifactId>
<version>${chdb.version}</version>
</dependency>
</dependencies>
```

The BOM keeps the driver and every native package on one version, which they must be.
Use the same `chdb.version` for the driver, optional Arrow module and all native packages.
The Central publishing configuration selects six library modules with standalone consumer POMs; the internal
parent and BOM are not published. Existing RC coordinates remain available from the RC repository.

**A native package is large**, because it contains the whole engine: about 110 MiB for macOS
arm64, 127 MiB for macOS x86_64, 157 MiB for Linux aarch64 and 176 MiB for Linux x86_64
Expand Down Expand Up @@ -317,7 +314,7 @@ The resulting package is stamped `engine.source=local` and must not be published
chdb-jdbc/ pure-Java driver, loader and type mapping
chdb-jni/ the JNI shim (C++17, ~1400 lines)
chdb-native-<platform>/ one Maven package per platform
chdb-bom/ keeps them all on one version
chdb-bom/ legacy RC BOM (not published to Central)
chdb-integration-tests/ tests that need a real engine
chdb-examples/ runnable examples
scripts/engine.properties the pinned engine version and its checksums
Expand Down
3 changes: 3 additions & 0 deletions chdb-arrow/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@
<version>1.1.0-SNAPSHOT</version>
</parent>
<artifactId>chdb-arrow</artifactId>
<properties>
<maven.deploy.skip>false</maven.deploy.skip>
</properties>
<name>chDB Arrow</name>
<description>Optional Apache Arrow Java integration for chDB.</description>
<dependencies>
Expand Down
3 changes: 3 additions & 0 deletions chdb-jdbc/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@
</parent>

<artifactId>chdb-jdbc</artifactId>
<properties>
<maven.deploy.skip>false</maven.deploy.skip>
</properties>
<packaging>jar</packaging>
<name>chDB JDBC Driver</name>
<description>
Expand Down
1 change: 1 addition & 0 deletions chdb-native-linux-aarch64-gnu/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
</description>

<properties>
<maven.deploy.skip>false</maven.deploy.skip>
<chdb.platform.id>linux-aarch64-gnu</chdb.platform.id>
<chdb.platform.os>linux</chdb.platform.os>
<chdb.platform.arch>aarch64</chdb.platform.arch>
Expand Down
1 change: 1 addition & 0 deletions chdb-native-linux-x86_64-gnu/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
</description>

<properties>
<maven.deploy.skip>false</maven.deploy.skip>
<chdb.platform.id>linux-x86_64-gnu</chdb.platform.id>
<chdb.platform.os>linux</chdb.platform.os>
<chdb.platform.arch>x86_64</chdb.platform.arch>
Expand Down
1 change: 1 addition & 0 deletions chdb-native-macos-aarch64/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
</description>

<properties>
<maven.deploy.skip>false</maven.deploy.skip>
<chdb.platform.id>macos-aarch64</chdb.platform.id>
<chdb.platform.os>macos</chdb.platform.os>
<chdb.platform.arch>aarch64</chdb.platform.arch>
Expand Down
1 change: 1 addition & 0 deletions chdb-native-macos-x86_64/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
</description>

<properties>
<maven.deploy.skip>false</maven.deploy.skip>
<chdb.platform.id>macos-x86_64</chdb.platform.id>
<chdb.platform.os>macos</chdb.platform.os>
<chdb.platform.arch>x86_64</chdb.platform.arch>
Expand Down
31 changes: 23 additions & 8 deletions docs/publishing.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,38 @@
# Publishing runbook

Everything needed to publish `com.clickhouse.chdb` artifacts. The first test version is
`1.0.0-rc.1`, published to GitHub Packages; stable releases go to Maven Central once the
ClickHouse Publisher Pro setup is ready.
Everything needed to publish `com.clickhouse.chdb` artifacts. The current RC is
`1.1.0-rc.1` on `maven.chdb.io`; the first stable release targets Maven Central after the
credentials, configuration and release-readiness gates are complete.

---

## Status

**Central preparation update (2026-11-18):** the Central configuration now selects only JDBC,
Arrow and the four native platform modules. `flatten-maven-plugin` generates standalone
consumer POMs during ordinary builds as well as releases, so neither the internal parent nor
BOM is published to Central. The RC workflow below retains its historical parent/BOM layout.
`scripts/check-consumer-poms.py` rejects unflattened or incomplete consumer metadata before
Central upload. `checksums=required` avoids optional SHA-256/SHA-512 sidecars; GPG signatures
remain required. The native libraries' embedded SHA-256 verification is unchanged.

The Central plugin is pinned to `0.12.0`. A workflow dry run executes `deploy` with
`-DskipUpload=true`, exercising staging without sending releases or snapshots to Central.
It does not validate the account token or replace Sonatype's server-side validation. CI
still stores native staging files and release metadata as GitHub Actions artifacts.
Actual uploads keep `autoPublish=false`; release bundles wait for manual Portal approval.
See `docs/release-readiness.md` for the current blockers and release-owner decisions.

| | |
|---|---|
| POM metadata Central requires | **done** — `<scm>`, `<developers>`, `<licenses>`, `<organization>`, `<issueManagement>` |
| Sources and javadoc jars | **done** — real ones for `chdb-jdbc` and `chdb-arrow`, empty ones for the native packages |
| GPG signing | **done and verified** — signs jar, sources, javadoc and pom |
| Publishing plugin | **done** — `central-publishing-maven-plugin`, `autoPublish=false` |
| Third-party licence inventory | **done** — generated from the engine, shipped in the package, drift-tested |
| `com.clickhouse.chdb` namespace | **pending** — stable releases use the ClickHouse Central workspace |
| `com.clickhouse.chdb` namespace | **confirmed in preceding Portal review** — Central upload not yet exercised |
| A way to stage all four platforms for one release | **done** — `.github/workflows/release.yml`, issue #15 |
| GitHub Packages RC publishing | **implemented** — `.github/workflows/rc-release.yml` |
| Public Maven RC repository | **implemented** — `.github/workflows/rc-release.yml` stages for `maven.chdb.io` |
| Proof that the artifacts work when consumed from a repository | **done, locally** — `scripts/verify-consumer.sh`, issue #10 |
| GPG key for the project | **not started** — needs a decision about whose key |
| Position on the LGPL components | **not started** — needs chdb-io |
Expand Down Expand Up @@ -113,8 +128,8 @@ and the publishing plugin talks to Sonatype. What it adds:

- `maven-gpg-plugin` at `verify`, with `--pinentry-mode loopback` so the passphrase can come
from a CI secret rather than a terminal.
- `central-publishing-maven-plugin` with `autoPublish=false`. A release here spans seven
modules plus their parent POM and hundreds of megabytes; it cannot be unpublished. A human
- `central-publishing-maven-plugin` with `autoPublish=false`. A release here spans six
library modules and hundreds of megabytes; it cannot be unpublished. A human
confirming the bundle in the portal is worth
the extra step, at least until several releases have gone out uneventfully.
- Sources and javadoc jars. `chdb-jdbc` and `chdb-arrow` produce real ones. The four native packages produce
Expand Down Expand Up @@ -376,7 +391,7 @@ Each step can invalidate the next, so:
```

That deploys to a throwaway `file://` repository and resolves from it in a project outside
this checkout, with its own empty local repository — so the POMs, the BOM import, the
this checkout, with its own empty local repository — so the standalone POMs (no parent/BOM), the
transitive `chdb-jdbc` dependency, the unpack-from-JAR path and both "wrong platform
package" error messages are all exercised against a repository rather than a reactor. What
it cannot exercise is a network transfer: a `file://` deploy is a copy. **So the only thing
Expand Down
58 changes: 48 additions & 10 deletions docs/release-readiness.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,42 @@ restating.

---

## Central preparation checkpoint — 2026-11-18

The Central path uses plan A: six JAR modules (JDBC, Arrow and four native platforms),
standalone flattened consumer POMs, and no published parent or BOM. The legacy RC publishing
workflow retains its original parent/BOM coordinates for compatibility. Use one `chdb.version`
property in consumer projects instead of importing a BOM.

Verified against `chdb-io/chdb-java` on this date: `maven-central` environment is absent,
repository-level secrets are absent, and the current maintainer has write but not admin access.
Inherited organization secrets were not verified. Namespace publishing access was confirmed
in the preceding Portal review; a successful Central upload has not yet been demonstrated.

The release owner still needs to identify the project signing key, arrange the environment's
four secrets and reviewers with an administrator, and supply evidence for the licence,
external-consumer and concurrent-soak gates below. Never place credentials in this file.
The work plan also requires at least one week of RC soak and external validation. The
`v1.1.0-rc.1` publication on 2026-11-17 does not complete that window before 2026-11-24.
Any change to that gate must be an explicit release-owner decision, not an assumption in CI.

Keep POMs at `1.1.0-SNAPSHOT` for the dry run and real snapshot validation. Only then prepare
the `1.1.0` release commit, its successful four-platform `build`, and its matching tag.
Each remote mutation requires separate user confirmation under the machine's standing rules.

Local preparation checks passed on macOS ARM64 with engine `26.11.1`:

- Seven regression tests for the consumer POM validator and all six generated release POMs.
- Java 11: 178 JDBC unit tests and all 30 Arrow tests, including 29 native integration tests.
- A fresh `file://` consumer repository: no parent/BOM, transitive JDBC, native extraction
checksum verification, correct query results, loader error paths and Arrow reading.
- Three-module signed snapshot dry run (JDBC, Arrow, macOS ARM64): twelve GPG signatures
verified with an isolated disposable test key. `skipUpload=true` stopped snapshot deployment;
publishing endpoints were also overridden to an unreachable localhost port for this test.

These checks are not a four-platform CI dry run, a production-key check, a network upload,
two independent external-project reports, legal approval, or a long-running concurrent soak.

## Running order

The release owner selected `v1.1.0` as the first stable Java version on 2026-11-17,
Expand Down Expand Up @@ -65,8 +101,9 @@ not whether to carry on.
| `CENTRAL_TOKEN_PASSWORD` | portal token password |

**Done when:** `release.yml` run with `channel=snapshot` and `dry_run=true` produces a `.asc`
for every artifact. That exercises staging on all four platforms, signing and packaging, and
uploads nothing.
for every library artifact. It exercises all four platforms, standalone POM validation, signing
and the deploy lifecycle using Central plugin `0.12.0` with `skipUpload=true`. No artifacts
are sent to Central; ordinary GitHub Actions staging and metadata artifacts are still uploaded.

### 4. Enable SNAPSHOTs for the namespace and publish a snapshot

Expand Down Expand Up @@ -256,18 +293,18 @@ scripts/verify-consumer.sh
```

It deploys to a throwaway `file://` repository, builds a consumer project outside this checkout
with its own empty local repository, imports the BOM, declares exactly one platform package with
no version, and runs a query. It asserts that the POMs resolve from a repository, that the BOM
supplies the versions, that `chdb-jdbc` arrives transitively, that the libraries are unpacked
with its own empty local repository, declares exactly one platform package with an explicit
version property, and runs a query. It asserts that standalone POMs resolve without a parent
or BOM, that `chdb-jdbc` arrives transitively, that the libraries are unpacked
from the JAR rather than read from `target/`, that their checksums match what
`scripts/build-native.sh` staged, and that the "no platform package" and "wrong platform
package" errors each name the coordinate the machine actually needs. `build` runs it on all
four platforms, so the answer cannot go stale between releases.

**What the gate found on its first run**, which is the part issue #10 asks for:

- The BOM import and the transitive `chdb-jdbc` dependency were already right, and are now
asserted rather than assumed.
- Originally this checked a BOM import. Plan A now checks resolution without any parent or
BOM and preserves the transitive `chdb-jdbc` assertion.
- **A snapshot deploy does not write the filenames a reactor build uses.** It writes
`chdb-jdbc-<version>-<timestamp>-1.jar` plus a `maven-metadata.xml` that maps `-SNAPSHOT` onto
that timestamp; without the metadata a consumer gets a 404 for a version that is demonstrably
Expand Down Expand Up @@ -444,8 +481,9 @@ them found real defects when first run.
profile, the signing, the licence inventory, the four-platform assembly, and a local proof that
the artifacts work when consumed from a repository rather than from the reactor.

What is left is a DNS record on `chdb.org`, a decision about whose key signs, a licence answer
from whoever owns the engine, two projects willing to depend on a snapshot, and a concurrent
soak. The **Running order** at the top of this file is the sequence, because each of those can
What remains is the configured Central publishing environment and production signing key,
a licence answer from whoever owns the engine, two projects willing to depend on a snapshot,
and concurrent soak plus the RC validation window. Namespace access was confirmed in the
preceding Portal review. The **Running order** at the top of this file is the sequence, because each gate can
invalidate the next; §3.1 is the part of Track 3 that blocks, and §3.2 is the part that does
not.
Loading
Loading