Skip to content

perf(ci): fewer plumbing jobs, routed by matrix-mode (csharp-ci, scala-ci) - #42

Merged
monsieurleberre merged 2 commits into
devfrom
perf/plumbing-jobs-on-cheap-runner
Sep 29, 2026
Merged

monsieurleberre merged 2 commits into
devfrom
perf/plumbing-jobs-on-cheap-runner

Conversation

@monsieurleberre

Copy link
Copy Markdown
Contributor

Why

With CI_MATRIX_MODE=cheap the heavy legs run on ["self-hosted","hetzner"], but csharp-ci / scala-ci still run normalize matrix, coverage-output, matrix-output and matrix-comment on hard-coded ubuntu-latest. Each takes ~6 s and is billed as a full minute; they are now the bulk of the remaining billed minutes (~40/day for canton-dotnet-sdk-internal alone, its jvm-helper-ci call is scala-ci).

What

  1. Fewer jobs (csharp-ci, scala-ci): matrix-output and matrix-comment are folded into coverage-output. Plumbing jobs per run: 4 -> 2 (normalize matrix, coverage-output). The build matrix cannot be computed without a job, so normalize matrix stays.
  2. Same runner selection as the heavy legs: new optional input plumbing-runner (label or JSON array, default empty). Empty follows the existing switch: matrix-mode / CI_MATRIX_MODE == cheap and github.event.repository.private == true -> ["self-hosted","hetzner"]; anything else -> ubuntu-latest (today's behaviour). Public repos never reach self-hosted (same guarantee as cheap matrix, fail-safe when the field is absent). csharp-ci's pack job uses the same selection.
  3. scripts/render_matrix_comment.py gets an optional runner-name arg so the merged job (whose display name is no longer unique per artifact-prefix) can still find itself when csharp and scala run in one caller run. Tests added (65 pass).

Job inventory (hard-coded GitHub-hosted, before -> after)

Workflow Job Before After
csharp-ci normalize matrix ubuntu-latest plumbing-runner rule
csharp-ci coverage-output ubuntu-latest plumbing-runner rule (absorbs matrix-output + matrix-comment)
csharp-ci matrix-output ubuntu-latest removed (merged)
csharp-ci matrix-comment (prefix) ubuntu-latest removed (merged)
csharp-ci pack (opt-in) ubuntu-latest plumbing-runner rule
scala-ci normalize / coverage-output / matrix-output / matrix-comment ubuntu-latest same as csharp-ci (no pack job)

Not touched: build-and-test (matrix runner), go-ci/terraform-ci (already runs-on input), update-badges (see below), csharp-publish-public (release path, deliberately hosted), this repo's own build-and-test.yaml resolve runner (self-lint only).

Compat argument

  • Inputs: one new optional input, defaults preserve behaviour. Outputs coverage / matrix-status unchanged (now sourced from coverage-output). Secrets unchanged.
  • Job names: coverage-output kept, because peacefulstudio/canton-ledger-api-csharp-internal requires csharp-ci / coverage-output on dev. matrix-output / matrix-comment (<prefix>) disappear; I found no ruleset or branch protection requiring them (checked all consumer repos I can read; gate-style aggregators use needs on the call).
  • Behaviour deltas: coverage-output runs on !cancelled() (was success()); after a failed shard it now succeeds with empty coverage (the call still fails through build-and-test), and skipped/success both satisfy required checks. Coverage-missing error is raised after both outputs are written so matrix-status is still emitted. PR comment: same header, same conditions, now the last steps of coverage-output; if it fails it fails the job (as before it failed the call).
  • Permissions: merged job requests actions: read + pull-requests: write; callers already had to grant these (the old matrix-comment requested them unconditionally).
  • Behavioural change for opt-in callers only: private/internal repos with CI_MATRIX_MODE=cheap (or matrix-mode: cheap) now get plumbing on Hetzner automatically. Escape hatch: plumbing-runner: ubuntu-latest.

Consumers (all pin @v2 moving tag)

csharp-ci: bosun, budget, canton-dotnet-sdk-internal, canton-dotnet-sdk-mini-demo-{internal,public}, canton-ledger-api-csharp-{internal,public}, canton-localnet{,-internal,-public}, daml-codegen-csharp-{internal,public}, dotnet-extensions-internal, nuget-skills. scala-ci: canton-dotnet-sdk-internal (as jvm-helper-ci), daml-codegen-csharp-{internal,public}. (Found via local checkouts + code search; search index can lag.)

Not verified live

actionlint is clean and unit tests pass, but nothing exercises runs-on expression / the merged job on real runners yet. Per CONTRIBUTING, please integration-test via a draft PR in a consumer pointing at this branch (e.g. canton-dotnet-sdk-internal csharp-ci/scala-ci @perf/plumbing-jobs-on-cheap-runner) before releasing.

Release (owner runs; not done here)

Intended version: v2.5.0 (additive input). Floating v2 must move (consumers pin @v2):

git fetch origin && git checkout dev && git pull --ff-only
git tag -a v2.5.0 -m "v2.5.0: plumbing jobs follow cheap mode, fewer jobs" && git tag -fa v2 -m "Latest v2 (= v2.5.0)" && git push origin v2.5.0 v2 --force

(Move CHANGELOG [Unreleased] to [2.5.0] in the release commit as usual.)

@monsieurleberre
monsieurleberre marked this pull request as ready for review September 29, 2026 10:58
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-29T11:05:27.819478Z 4d14c2a Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4d14c2a364

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/csharp-ci.yaml
Comment thread scripts/render_matrix_comment.py
@monsieurleberre
monsieurleberre merged commit 88ac32c into dev Sep 29, 2026
2 checks passed
@monsieurleberre
monsieurleberre deleted the perf/plumbing-jobs-on-cheap-runner branch September 29, 2026 11:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant