Dagger modules and Go libraries behind Basis' CI/CD pipelines.
As of version 3.0 the repository is configuration-driven: a consumer project no longer writes a Dagger module of its own, it just declares its targets in ci/pipeline.toml and includes the CI template. The orchestrator module does the rest.
| Component | Role |
|---|---|
|
The generic Dagger module. Reads |
|
Git and registry utilities (change detection, bump/commit, |
|
Per-build-technology Dagger modules. Called by the orchestrator, not by the project. |
|
Go library: the configuration schema ( |
|
Go library: GitLab client for commit statuses and merge request notes. |
orchestrator is the generic Dagger module introduced in 3.0. It replaces the per-project Dagger module that existed up to 2.x: all behaviour comes from the consumer project’s declarative file, and the targets declared there become simultaneously the build strategies, the quality checks and the image map used by check-images/promote.
A project needs exactly two files.
-
ci/pipeline.toml— the repository’s targets (see Theci/pipeline.tomlschema). -
.gitlab-ci.yml— five lines including the shared template:
include:
- project: 'basis/iac/ci-templates'
ref: v1.7.0
file: 'templates/dagger-orchestrator.gitlab-ci.yml'The template defines the check, build, pre-prod and promote stages, and calls the orchestrator functions with the GitLab variables. Two variables can be overridden in the project’s own variables: block:
-
DAGGER_MODULE— defaults to the orchestrator version pinned by the template. Projects with a custom build point it at"."(a local module, see Projects with a custom build). -
PIPELINE_CONFIG— defaults toci/pipeline.toml.
The module constructor takes --source (the repository root) and --config-path:
dagger call -m github.com/BasisTI/daggerverse/orchestrator@3.6.0 \
--source . --config-path ci/pipeline.toml \
validateLoads and validates the declarative file and returns a textual report with the resolved targets (defaults applied), the derived image map and any warnings. It makes no network calls — it is the configuration lint job, triggered on merge requests that touch ci/pipeline.toml.
The warnings cover what is valid in the schema but the generic orchestrator cannot execute: type = "custom" targets.
Runs tests and static analysis on the changed targets that declare sonar = true. It takes --base-branch, --commit-sha, --sonar-host, --sonar-token, the optional --stop-on-first-fail, the optional --gitlab-host / --gitlab-token / --gitlab-project-id / --gitlab-ref for commit statuses, and the optional --merge-request-id / --merge-request-source-branch / --merge-request-target-branch.
Nothing is published, but the build still carries the project’s real version — the one declared in the version file, from the last publish. Forcing a constant there would pin SonarQube’s PREVIOUS_VERSION new-code anchor to that constant, and the whole codebase would read as new code forever.
When the merge request triple is present, the analysis becomes a Pull Request analysis: new code is the diff against the base branch instead of the project’s new-code period, and SonarQube decorates the merge request in GitLab. All three are required — with any of them missing it falls back to branch analysis, which is the right thing outside a merge request and is what the develop job does.
Fails with an explanatory message if the configuration has custom targets.
Detects the targets changed between --base-branch and the current commit, builds and publishes each image to --registry, applies the sha-<commit> tag and returns the list of published images (one per line).
If --git-remote-url is given, it bumps the version files of the published targets and commits back to --git-branch (defaults to develop). It also fails if there are custom targets.
With the GitLab parameters set, the published images are posted as a comment on the merge request the commit came from — no need to open the job and hunt for the refs in the log. See Merge request notes.
Checks that every image derived from the configuration already exists in the registry, before promotion. Runs on merge requests targeting main, using --build-branch (defaults to origin/develop) to locate the commit the images were built from.
Custom targets are not skipped here: their images live in the registry and need to be checked like any other.
Promotes the images derived from the configuration from --src-registry to --dst-registry (which, when empty, is --src-registry itself). Like check-images, it covers custom targets' images too.
Every promotion walks all of the project’s images, not only the ones that changed — that is how an old image keeps its production- tag present. But the copy only happens when it changes something: before copying, the source tag’s digest is compared with the production tag’s, and if they match the image is skipped with ⏭️ Já promovida. Services untouched for months stop generating registry traffic on every merge to main, and the log shows at a glance what actually went into the promotion.
Implemented in pipeline/config. Parsing is strict: any unknown field is an error.
| Field | Type | Description |
|---|---|---|
|
int |
Required, currently always |
|
table |
Repository metadata. |
|
table |
Per-build-type defaults. |
|
table |
One block per buildable artifact. At least one is required. |
| Field | Type | Description |
|---|---|---|
|
string |
Required. The registry group: |
| Block | Field | Type | Description |
|---|---|---|---|
|
|
string |
Base image for Maven builds. |
|
|
bool |
Binds a Docker daemon to the build (Testcontainers). |
|
|
string |
Version of |
|
|
string |
Build image (e.g. |
|
|
string |
Runtime image (e.g. |
|
|
string |
Build image (e.g. |
|
|
string |
Runtime image (e.g. |
| Field | Type | Description |
|---|---|---|
|
string |
Required. One of |
|
string |
Change-detection path. Defaults to the target name. |
|
string |
Directory mounted into the build. Defaults to the effective |
|
string |
Image name in the registry, without group and without tag. Defaults to the target name. Effective names must be unique across targets. |
|
string |
Version file bumped by |
|
bool |
Marks |
|
list of string |
Additional paths that also trigger this target’s build/check (e.g. a shared library, the root |
|
bool |
The target takes part in |
|
string |
The build system that analyses the target’s code, when it differs from how the image is built. One of |
type answers "how is the image built"; quality-type answers "which build system tests and analyses the code". For the vast majority of targets they are the same thing, and quality-type is omitted.
They diverge when a project needs its own Dockerfile for the runtime — virtual display, headless browser, native drivers — while the code is still a plain uv/Maven/npm project with tests and static analysis. Without quality-type such a target would have to choose between publishing the right image and having its quality measured; with it, publish-all builds from the Dockerfile and check-quality runs through the declared build system.
That is the case of the licitacao scrapers, which run a browser at runtime:
[targets.scrapers_mte]
type = "dockerfile" # the image comes from the scraper's Dockerfile
path = "scrapers/mte"
sonar = true
quality-type = "uv" # the code is Python: tests and Sonar through the uv moduleThe quality build system’s fields (run-subdir, uv-build-image, customizations, maven-image, …) apply normally on the target and are what check-quality consumes — and validate shows them in a separate quality (<type>) block.
Two validation rules keep the field honest:
-
quality-typewithoutsonar = trueis an error: it would be an inert field, misleading whoever reads the file; -
sonar = trueon adockerfiletarget withoutquality-typeis a validation error, with a message naming exactly the missing field — not a late failure when the check runs.
| Field | Type | Description |
|---|---|---|
|
string |
Overrides |
|
bool |
Overrides |
|
string |
Overrides |
|
bool |
Multi-module build: mounts the root and builds with |
|
string |
Path of the module inside the reactor. Required when |
|
list of string |
Extra options passed to |
| Field | Type | Description |
|---|---|---|
|
string |
Overrides |
|
string |
Overrides |
|
string |
Execution subdirectory inside the build (e.g. |
|
list of string |
uv module build customizations: |
| Field | Type | Description |
|---|---|---|
|
string |
Path to the Dockerfile, relative to the effective |
type = "npm" has no fields of its own: its images come exclusively from [defaults.npm].
Four targets — three Maven and one Angular frontend — all with a directory name matching the target name, which makes path and image unnecessary:
schema-version = 1
[project]
group = "contavinculada"
[defaults.maven]
image = "maven:3.9.11-eclipse-temurin-21"
use-docker = false
[defaults.npm]
build-image = "node:22-alpine"
run-image = "nginx:alpine"
[targets.contavinculada]
type = "maven"
sonar = true
[targets.integracaosgo]
type = "maven"
sonar = true
[targets.snf]
type = "maven"
sonar = true
[targets.frontend]
type = "npm"
sonar = trueTwo modules of the same reactor. Both mount the root (source-path = "."), share the root pom.xml as their version file (root-version-file = true) and are rebuilt when the shared contract or the root pom changes:
schema-version = 1
[project]
group = "triagem"
[defaults.maven]
image = "maven:3.9.11-eclipse-temurin-25"
use-docker = true
[targets.triagem-core]
type = "maven"
source-path = "."
reactor = true
module = "triagem-core"
version-file = "pom.xml"
root-version-file = true
extra-trigger-paths = ["triagem-contracts", "pom.xml"]
sonar = true
[targets.triagem-ingestao]
type = "maven"
source-path = "."
reactor = true
module = "triagem-ingestao"
version-file = "pom.xml"
root-version-file = true
extra-trigger-paths = ["triagem-contracts", "pom.xml"]
sonar = trueMore real-world examples live in pipeline/config/testdata/.
This is the schema’s central rule.
The image map consumed by check-images and promote is not declared: it is derived from the same targets that describe the build, through Config.ProjectImages(), which returns "<group>/<effective image>" → "<effective path>" for every target.
Before 3.0, the list of images to check and promote was maintained by hand, in parallel with the project orchestrator’s build targets. Adding a new service and forgetting the second list was enough for the promotion to silently skip an image — or to point at one that no longer existed.
With a single source of truth, divergence is impossible: the set of what gets built and the set of what gets promoted are literally the same collection of targets. type = "custom" targets enter the map normally — the generic orchestrator does not build them, but their images live in the registry and must be checked and promoted like the rest.
Some projects have builds that none of the generic types covers. They keep a Dagger module of their own, but they do not duplicate the configuration: that module loads the same ci/pipeline.toml through pipeline/config and merges the hand-written custom targets with the generic ones derived from the file.
In the schema, those targets are marked type = "custom":
[targets.beneficios]
type = "maven"
path = "apps/beneficios"
sonar = true
[targets.rh-dp]
type = "custom"
[targets.lightdash-content]
type = "custom"
extra-trigger-paths = ["rh-dp/dbtrh"]The generic orchestrator does not know how to build those targets and fails with a clear message in publish-all and check-quality, stating that the project must point DAGGER_MODULE at its local module. validate only emits a warning, and check-images/promote keep working through the generic module, including for those targets' images.
A multi-module reactor is the case where a module depends on sibling modules under the same root pom.xml. Building the module’s directory in isolation does not work: the sibling dependencies are not on disk.
With reactor = true the maven module (ReactorMode option):
-
mounts the reactor root instead of the module directory;
-
builds with
-pl <module> -am, so Maven builds the target module and everything it needs; -
injects
-Drevision=<version>into every invocation instead of runningversions:set.
versions:set is avoided on purpose: rewriting the POMs would fight flatten-maven-plugin and break resolution of the sibling dependencies -am has just built.
-
The root
pom.xmldeclares<version>${revision}</version>and a<revision>property holding a local development value:<version>${revision}</version> <properties> <revision>0.0.1-SNAPSHOT</revision> </properties>
-
flatten-maven-plugin configured, so installed/published artifacts carry the resolved version and not the literal
${revision}. -
maven-install-plugin active on
verify. This is mandatory: the Jib stage runs without-am, that is, without rebuilding the sibling modules — they must already be in the local repository, installed by the build-and-test stage.
Nobody writes that number by hand. The <revision> property value in the file is not used by the pipeline: every Maven invocation receives -Drevision=<CalVer>, and a command-line property beats the POM’s. Whatever is committed there only matters to someone running mvn on their own machine without passing -Drevision — which is why the initial value should be a development SNAPSHOT, not a CalVer.
After publishing the images, publish-all writes the CalVer back to the file and commits: when the pom’s <version> is ${revision}, orchestrator-utils edits the root pom’s <revision> property rather than the <version> element. That is why reactor targets declare version-file = "pom.xml" together with root-version-file = true.
In other words, the number you find in the repository is the trace of the last successful pipeline, not an entry someone has to maintain.
-
ParentPomandFullBuildModuleswere removed. Multi-module is nowReactorMode(reactor = truein the schema), which builds with-pl <module> -amfrom the root. See Maven reactor mode. -
The
buildImagedefault is no longer the-alpinevariant, it is nowmaven:3.9.11-eclipse-temurin-21. Alpine is musl-based and the node binary frontend-maven-plugin downloads is linked against glibc, so frontend builds died on it.
-
EntryPointInfowas removed fromBuildTarget. The execution subdirectory of Python targets became therun-subdirfield of the declarative configuration, forwarded to theuvmodule. -
BuildStrategyandQualityStrategyno longer carry opaque configuration fields: the strategies close over the resolved target.
The repository runs two tagging schemes in parallel, because it holds two kinds of artifact.
| Artifact | Tag format | Example |
|---|---|---|
Dagger modules ( |
Repository-wide tag, no prefix |
|
Go libraries ( |
Directory-prefixed tag, as Go modules require |
|
In practice: DAGGER_MODULE references …daggerverse/orchestrator@3.0.0, while go.mod files reference github.com/BasisTI/daggerverse/pipeline v0.9.1. The two numberings evolve independently — a 3.0.0 release of the modules can coexist with pipeline/v0.9.1.
publish-all bumps the version files and commits back to the branch. That commit must not trigger a new pipeline, on pain of an infinite loop.
Suppression is done exclusively through the push option, in CommitAndPush (orchestrator-utils):
git push -o ci.skip <remote> HEAD:<branch>The commit message is just Bump versão para <version>, without the [skip ci] prefix. The prefix was removed on purpose: [skip ci] in the message is honoured by GitLab on merge request pipelines too, so a merge request containing the bump commit would have its pipeline blocked — check-quality included. The push option, by contrast, only affects the push that produced it.
orchestrator-utils recognises both formats when looking for the build commit (--grep=^\[skip ci\] and --grep=^Bump versão para `, with `--invert-grep), so check-images and promote skip bump commits when locating the build branch.
The gitlabci/ directory is a standalone Go library (github.com/BasisTI/daggerverse/gitlabci).
gitlabci.Client reports each stage’s progress as a commit status, visible in the merge request/commit UI. It calls POST /api/v4/projects/:id/statuses/:sha to move each stage from running to success or failed.
client := &gitlabci.Client{
BaseURL: "https://gitlab.example.com",
Token: "glpat-...",
ProjectID: "12345",
Ref: "develop",
}
client.SetCommitStatus(sha, gitlabci.StateRunning, "my-app: Build", "")
// ... run the stage ...
client.SetCommitStatus(sha, gitlabci.StateSuccess, "my-app: Build", "")Names follow the "{project}: {stage}" pattern, for example portal-web: Install Dependencies or api-gateway: Docker Build and Push.
Ref is load-bearing. Without it GitLab attaches each status to the most recent pipeline for that SHA. If a second pipeline is born on the same commit mid-build — a push to develop that is also a merge request head, say — the terminal status lands on the new pipeline and the original one stays running forever, with nobody left to close it.
The orchestrator builds the client from --gitlab-host, --gitlab-token, --gitlab-project-id and --gitlab-ref. When any of them is missing, the modules behave exactly the same, simply without reporting statuses.
The same client posts reports as merge request comments. Each report carries an invisible HTML marker, which is how a rerun edits the existing note instead of stacking a new one — the same way SonarQube does it. One marker per subject, so reports from different sources coexist on the same merge request:
| Marker | Written by |
|---|---|
|
|
|
Quality gates that report their own verdict, for projects whose checks do not go through SonarQube. |
Client.ReportPublishedImages(commitSha, published, version) is the entry point for the first one. The publish job runs on a push to the branch, already past the merge, so there is no CI_MERGE_REQUEST_IID in the environment — the merge request is discovered from the commit through GET /repository/commits/:sha/merge_requests, which matches both the merge commit and the squash commit. A direct push with no merge request simply has nowhere to comment, and the job says so.
Failures here are reported and swallowed. The report is a convenience, and bringing down a publish that already succeeded because a comment did not go through would trade a small problem for a large one.
Generic, type-safe orchestration. It uses generics to abstract over the Dagger SDK types, letting orchestrators on different SDK versions share the same logic.
-
BuildStrategy[Dir, Secret]/QualityStrategy[Dir, Secret]— the signatures any module implements to plug into the orchestration. -
DaggerOps[Dir, Secret]— callbacks isolating the SDK-specific operations (change detection, directory access, image tagging). -
PublishAll— detects the changed projects, runs their build strategies in sequence, aggregates the published images and tags them withsha-<commit>. -
CheckQuality— runs tests and static analysis on the changed projects, with optional fail-fast.
Reading and validation of ci/pipeline.toml, and the anti-drift derivation of the image map (ProjectImages, ProjectImagesJSON). Consumed both by the generic orchestrator and by the projects' custom modules.
BuildOptions(host, token, projectKey, waitForQualityGate, extra) assembles the -Dsonar.* options in canonical order. The maven, npm and uv modules delegate to it; each keeps only its own edges (reading the *dagger.Secret and handling a nil config).
These modules are called by the orchestrator; a project does not normally invoke them directly.
Build and test (clean verify), Sonar analysis and image publishing through the Jib plugin. Supports Maven Wrapper, local repository caching, an optional Docker daemon for Testcontainers, reactor mode and commit status reporting.
Outside reactor mode the version is applied with versions:set; in reactor mode, with -Drevision. When no version is given the pom is left untouched — that is the quality-check case, which publishes nothing.
The scanner is invoked by full coordinates — org.sonarsource.scanner.maven:sonar-maven-plugin:<version>:sonar — rather than through the sonar:sonar prefix. The prefix only resolves when the project declares the plugin, since it is not in Maven’s default pluginGroups, and that implicit dependency broke projects not generated by JHipster.
The version is pinned (sonar-plugin-version, with a default in the maven module) for two reasons: a new SonarSource release must not break every pipeline overnight without a commit anywhere, and the scanner has to stay compatible with the server, which upgrades on its own schedule. As a consequence, the <version> declared in the pom no longer influences which scanner runs — whoever needs a different one declares it in ci/pipeline.toml.
Node.js/Angular projects. Runs npm ci, npm run build, optional SonarQube analysis, and publishes dist/ inside an nginx container. Supports version override by patching package.json, and commit status reporting.
Python projects using uv. Two-stage dependency sync (uv sync), optional SonarQube analysis and publishing of a slim Python image. Supports run-subdir and build customizations for dlt and dbt projects.
Git and registry utilities used by the orchestrator: change detection between branches, commit SHA lookup, image tagging with sha-<commit>, bumping and committing version files, image existence checks and promotion between registries.
All Git operations run inside an alpine/git container; image operations use crane.