Skip to content

WIP: integration branch (do not merge) - #96

Draft
Christopher Obbard (obbardc) wants to merge 71 commits into
mainfrom
wip/obbardc/integration
Draft

WIP: integration branch (do not merge)#96
Christopher Obbard (obbardc) wants to merge 71 commits into
mainfrom
wip/obbardc/integration

Conversation

@obbardc

@obbardc Christopher Obbard (obbardc) commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Scratch branch where I stage work before splitting it into reviewable PRs.
Not for review or merge. Opened so the pipeline runs against the whole
stack; the individual PRs will follow and carry the discussion.

github-advanced-security[bot]
github-advanced-security Bot found potential problems Sep 3, 2026
View reviewed changes
Comment thread .github/actions/prepare-kernel-source/action.yml Fixed
Comment thread .github/actions/prepare-kernel-source/action.yml Fixed
Comment thread .github/actions/prepare-kernel-source/action.yml Fixed
Comment thread .github/actions/prepare-kernel-source/action.yml Fixed
Comment thread .github/actions/prepare-kernel-source/action.yml Fixed
Comment thread .github/workflows/build-kernel-debian.yml Fixed
Comment thread .github/workflows/build-kernel-debian.yml Fixed
Comment thread .github/workflows/build-kernel-debian.yml Fixed
Comment thread .github/actions/prepare-kernel-source/action.yml Fixed
Comment thread .github/actions/debusine-build/action.yml Fixed
@obbardc
WIP: temporarily disable all builds
707795f 
Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
WIP: temporarily allow PR to push into staging
5f6594c 
Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
wip: manual suffix
a8bafc8 
Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
Revert "WIP: temporarily allow PR to push into staging"
77a3ae2 
This reverts commit dada06c.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
ci: bring back a release workflow, building the ref it ships
cc60ffa 
Releasing has had no workflow since the promotion was folded into the nightly
build. That commit removed release.yml because its model could not work: it
promoted out of qli-staging days after the fact, which meant the pipeline had
to keep a durable staging workspace for it to read, since the workspace a build
actually ran in is named after that run and does not outlive it.

This one does not promote a previous run's build. It builds the ref it
releases, in the same workflow the nightly calls, and promotes that build while
the run still holds its workspace -- so what reaches qli is the artifact this
run produced and tested, and the awkward dependency on a workspace outliving
the run that filled it never arises.

What is released is written down rather than typed in. ci/build-matrix.yaml
gains a releases list beside builds: same shape, plus a target_workspace, and
ref_strategy restricted to pinned_ref so that two dispatches of one entry
cannot ship different kernels. Updating a release is a pull request that
changes branch_or_tag, which puts the ref that ships under review before the
run that ships it -- where an upstream-version dispatch field put it at the
mercy of whoever typed it. The refs come from main, which is where releasing
last worked: qcom-next and qcom-next-debug, trixie and forky, at
qcom-next-7.2-20260826.

target_workspace stays rejected on a builds entry. Where a nightly is published
follows from why it is running and remains the calling workflow's to decide; a
release is the one case where the destination belongs to the entry, because
putting one ref into one archive is the whole of what it is.

The two lists are validated together however few of them a caller selects, so a
broken release entry fails a nightly run rather than lying in wait for whoever
next tries to release. Releases are Debian-only, enforced rather than assumed:
promotion runs through Debusine and the Ubuntu path has no workspace to promote
into, so an Ubuntu entry there would name a destination nothing could deliver
it to.

Which environment the promote job runs in becomes the caller's, through a new
promote-environment input defaulting to Staging. The nightly keeps promoting
into staging unattended; a release passes Production, so whatever approval that
environment requires stands in front of qli. The run summary lists the refs,
revisions and destinations before that gate, so the approval is given against
what is shipping rather than against a run number.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
wip
b2f8f57 
Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
ci: say why the family guards exist and why a skipped check reads oddly
5671553 
The if: guards on the build jobs look removable, and are not: an empty matrix
is a workflow error rather than an empty job list, so a family the run selects
nothing on has to be skipped rather than started with nothing to do.

Skipping one has a visible cost that looks like a bug. GitHub never expands a
job it skips, so it cannot evaluate name: ${{ matrix.name }} and shows the
expression's source text -- a check literally called "matrix.name". A ||
fallback does not help, printing its own source text just the same, and the
only name that would read well is one never mentioning matrix, which would mean
renaming every check in every workflow to tidy up one that appears when a
family is empty. Both facts are written down so the next reader reaches that
conclusion without rediscovering it.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
scripts: clone the kernel over HTTPS by default
fecd182 
build-kernel.sh defaulted to the SSH clone URL while CI and the matrix use
HTTPS, so a local build needed a GitHub key that a read-only clone does not.
The same URL is what the changelog records as the source, so the default now
matches what ci/build-matrix.yaml says.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
ci: drop the PR-patch inputs nothing passes
a860850 
prepare-kernel-source took qcom-next-pr and kernel-topics-pr and merged or
applied them after the clone, but no workflow passes either: the build
workflows have no input to carry them, so the two steps could never run.

They could not have been wired up as they stood, either. The commit SHA and
date are read from HEAD before the patches would be applied, so a patched
build would name a commit it did not build. And a merge commit made at build
time has no stable SHA, so the version could never be reproduced from it.
An orig tarball keyed on the commit, which the source package is about to
be, rules such builds out for good.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
ci: require debian-revision in prepare and drop the matrix lookup
e5889cc 
The action looked the revision up in ci/build-matrix.yaml when its input was
empty, for a caller with no matrix entry in hand. No such caller exists: the
build workflows are workflow_call only, are called with a matrix entry, and
give the input a default of their own, so the lookup could not be reached.
The revision is now a required input, and a caller that forgets it fails at
the action boundary rather than building at a revision nobody chose.

resolve-matrix.py keeps --field, which is still a useful way to ask the
matrix a question from a shell.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
ci: require the inputs that describe the build
3559248 
Every input naming what a build is -- the entry, its suite and flavour, the
kernel repository and ref, the package names and the Debian revision -- had a
default spelling out the qcom-next entry. No caller relied on one: daily.yml,
pr-build.yml and release.yml pass all of them from the matrix. What the
defaults did was let a caller that dropped one build the default entry's
kernel under another entry's name without anything noticing. They are now
required, so an omission is a workflow error at parse time.

The inputs that keep a default are the ones for which empty is a value:
tag-pattern outside latest_tag, an empty fragment or module list, and the
overrides and destinations the caller chooses rather than the entry.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
scripts: derive every version field through derive-localversion.sh
f414b89 
Three scripts derived LOCALVERSION from a tag, each its own way. CI used
ci/scripts/derive-localversion.sh. prepare-source.sh carried a copy of the
same rule. build-kernel.sh carried an older one that produced the
pre-plus, SHA-less form (qcom-next-20260210), passed no snapshot, and so
gave a local build a Debian version with no +git component at all -- a
different version from the one CI gives the same commit.

prepare-source.sh now calls derive-localversion.sh, with the exact tag or
the branch HEAD sits on as the ref and HEAD's committer date for the
branch-tip case, which is precisely what the prepare action feeds it. It
takes a --flavour for the identity part of LOCALVERSION, defaulting to
qcom-next, and build-kernel.sh passes that through and derives nothing of
its own. A local build of a commit now produces the version CI would, which
is the precondition for a source package built locally being interchangeable
with one built in CI.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
build-kernel.sh: resolve the latest tag the way CI does
3afaa93 
--latest-tag picked the tag with the highest version sort, which puts an
older-dated release candidate of a newer kernel above the newest snapshot.
CI resolves by trailing date through ci/scripts/resolve-kernel-ref.sh, so a
local build now calls the same script and lands on the same tag.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
prepare-source.sh: record where a local build came from
6c31e20 
The changelog names the repository and ref a build was cut from, but only
CI passed them, so a local build's changelog read "Source: unknown
unknown". The origin remote and the tag or branch HEAD answers to are the
same facts, read from the checkout, so they are now the defaults for
--git-clone and --git-ref. Naming the checkout once also serves the version
derivation, which used the same tag-or-branch rule of its own.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
packaging: date debian/changelog from the kernel commit
7344811 
The changelog entry carried a date fixed in the template, so every build of
every kernel was dated 14 November 2025. The date is now a prepare input,
CHANGELOG_DATE, and prepare-source.sh fills it with HEAD's committer date:
the same instant the snapshot comes from, so the entry is dated by the
source it describes and two builds of one commit write one changelog.

That date does more than read well. dpkg-source takes the debian tarball's
mtimes from the changelog date and dpkg-buildpackage exports it as
SOURCE_DATE_EPOCH, so dating the changelog from the commit is part of what
makes the source package a function of the commit alone.

The action reads the date on the runner and passes it in, as it does the
SHA and the ref: prepare-source.sh runs in a container as root and git will
not read a checkout owned by someone else. Its git lookups now tolerate
that, and only run for a value the caller did not supply. A prepare run by
hand without one gets today's date.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
scripts: add build-source-package.sh
e865428 
Both build paths made the orig tarball with tar czf on a fresh clone, so
every run wrote new mtimes and a new gzip header and the same upstream
version got a different orig each time. Two suites sharing a source package
and upstream version -- trixie and forky, differing only in Debian revision
-- therefore could not share a pool, and a rerun could not reproduce what
it had built.

build-source-package.sh takes a tree prepare-source.sh has prepared and
writes the .orig.tar.gz, .debian.tar.xz, .dsc and .changes into an output
directory. The orig comes from git archive of the commit, whose entries
carry the commit's timestamp and root ownership, through gzip -n, which
writes no timestamp: two runs on one commit give one tarball, byte for
byte. Verified here by building trixie and forky into one directory, where
the second run rebuilt the orig and compared it with the first.

The upstream version names the commit as ~g<sha>, so the script checks
that the commit it archives is that one, and refuses a tree that differs
from the commit outside debian/ -- dpkg-source would fold the difference
into an automatic patch and the package would no longer describe the commit
its version names. --write-fields hands the result to a caller as KEY=VALUE
lines, for CI.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
build-kernel.sh: build a source package, and build from one
eb70254 
--source-package stops after build-source-package.sh has written the
.orig.tar.gz, .dsc and .changes into the output directory, and --dsc
builds binaries from an existing source package with no clone and no
prepare. Together they let a local build follow the path CI takes: source
package first, binaries from it.

Native mode unpacks the .dsc into the output directory and runs
dpkg-buildpackage -b there, so the .deb files land where every other mode
puts them. sbuild takes a .dsc as it stands. Docker mode cannot go through
docker_deb_build.py, which builds from a tree only, so it runs sbuild in
the same pkg-builder image with the same flags docker_deb_build.py uses,
building the image through docker_deb_build.py --rebuild if it is absent.
The default path from a tree is unchanged.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
docs: describe building and consuming a source package locally
1fff653 
debian/README.md now lists the two scripts beside build-kernel.sh, shows
the source-package flow (--source-package, --dsc, and the two scripts run
by hand), and says what makes the orig tarball reproducible and what the
script refuses. It also stops claiming that build-kernel.sh derives
LOCALVERSION from the tag and that debian/rules recovers it from a package
name; neither has been true for a while.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
ci: build the source package in prepare
170691f 
prepare-kernel-source now runs build-source-package.sh after
prepare-source.sh, in the same pkg-builder container, and uploads the
resulting .changes set as source-package-<flavour>-<suite>. Nothing
consumes it yet: the prepared-tree artifact is still uploaded and still
what both build paths read, so this commit changes what a run produces
and not what it builds. The next commits move each family onto it.

The version the caller is told now comes from the fields file the script
writes, read from the .dsc it built, rather than from a sed over the
changelog on the runner host. The action also reports the orig tarball's
name and SHA-256, which a caller can hold against what an archive already
has for this upstream version.

The container is told to trust the checkout, which the runner owns and
root inside the container would otherwise refuse to read; it is this job's
own clone.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
ci: hand Debusine the source package prepare built
f1a29ec 
The build job downloaded the prepared tree, unpacked it, and had
debusine-action's generate-source-package assemble an orig tarball from it
with tar czf -- a different tarball on every run. It now downloads the
.changes set the prepare job built, reads the version from the .dsc, and
gives lib/build that .dsc: the source package Debusine builds is the one
prepare-kernel-source wrote and reported, with the reproducible orig.

DEBUSINE_ASSEMBLE_ORIG in debusine-action stays for its other callers;
nothing here uses it any more.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
ci: build the Ubuntu package from the source package
c09362b 
The build job downloaded the prepared tree and ran build-kernel.sh
--local-source --skip-prepare, which had docker_deb_build.py make an orig
tarball of its own with tar czf before handing sbuild a .dsc. It now
downloads the .changes set prepare built and passes its .dsc to
build-kernel.sh --dsc, which runs sbuild on it in the suite's pkg-builder
image. The Ubuntu family therefore builds the same kind of thing Debusine
builds for the Debian family: the source package the run reported, with the
reproducible orig, and nothing is prepared twice.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
ci: stop uploading the prepared tree
371b401 
Both build paths now start from the source-package artifact, so the
tarball of the prepared tree has no reader. It was the larger of the two
uploads, and the reason the README explained why an artifact had to be a
tar.gz: the tree carried execute bits that upload-artifact's zip would
strip. A .changes set is plain files, so that note goes with it, and the
pipeline diagrams now show the source package as the handoff.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
ci: fail when the archive holds a different orig for this version
fecd9b0 
The orig tarball is now a function of the kernel commit, so the archive can
hold a different one for the same upstream version only if something about
how it was written differs -- and that is what this check is for. The
already-published step, which already reads the target workspace's source
index, now also finds every stanza naming this run's orig and compares its
SHA-256 with the one prepare reported. A mismatch fails the run with both
checksums, before anything is built, rather than surfacing as a file
conflict at promotion or, worse, not at all.

The case that matters is the packaging rebuild: 0qli1 to 0qli2 of one
snapshot, where the archive requires the two revisions to share one file.
The check reads this suite's index only, since apt indexes are per suite;
across suites the reproducibility of the tarball itself is the guarantee.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
build-kernel.sh: need docker-pkg-build only to build a missing image
83a2447 
Docker mode required docker_deb_build.py up front, because building from
a tree goes through it. Building from a .dsc runs sbuild in the image
directly and needs the script only to build the image when it is absent,
so that path now looks for it without insisting, and says what to do if
the image is missing and the script is too. A run from a .dsc also stops
printing the tree-preparation settings it has no use for.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
ci: build the Ubuntu leg in one job
58ef67d 
The prepare job and the build job ran on the same runner label, so the
only thing the artifact between them did was cross a job boundary that
had no other reason to exist: uploaded by one ephemeral runner to be
downloaded by the next. One job now clones, prepares, builds the source
package and builds the binaries from it in place. The action still
uploads the .changes set, as the record of what was built, but nothing
downloads it.

The build job's own image-fetch step goes with it. It repeated the
action's pull-or-build decision for one suite; the action makes that
decision for every suite before prepare-source.sh runs, and leaves the
docker-pkg-build checkout where build-kernel.sh looks for it when it had
to build the image.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
ci: publish the source package to S3 beside the binaries
d675ee1 
The .deb files reach S3 under one run's directory; the source package they
were built from now sits in source/ beneath it, on both paths. The
.changes records every checksum, the .orig.tar.gz is the exact source, and
the .dsc is enough to rebuild the packages anywhere. The Actions artifact
holding the same files expires after a week; this is the copy that stays
with the binaries.

The Debian publish job downloads the artifact for it, since the source
package was built on another runner; the Ubuntu job uploads it from the
workspace it was built in.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
ci: test for a published version before comparing the orig
ef3f2c4 
The orig comparison ran first, so a night on which the tracked tag had not
moved failed instead of skipping: the version was already published, and
the orig in the archive predates git archive, so the checksums differ. That
is a run with nothing to do reported as a conflict.

A run that publishes nothing uploads no files, so nothing it built can
conflict with anything. The already-published test therefore comes first
and exits, and the orig comparison guards only a version that is actually
going to be promoted. That leaves it doing the job it was added for: a
packaging rebuild publishing a second Debian revision of one upstream
version, where the archive requires both revisions to name one file.

Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
@obbardc
wenable more builds
a27bced 
Signed-off-by: Christopher Obbard <chris.obbard@oss.qualcomm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@github-advanced-security github-advanced-security[bot] github-advanced-security[bot] left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@obbardc Christopher Obbard (obbardc)

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@obbardc @github-advanced-security