Skip to content

test(lint): install the Galaxy roles in the ansible-lint test - #208

Merged
DevSecNinja merged 3 commits into
mainfrom
steward/bats-order-independent-lint
Nov 18, 2026
Merged

DevSecNinja merged 3 commits into
mainfrom
steward/bats-order-independent-lint

Conversation

@DevSecNinja

@DevSecNinja DevSecNinja commented Nov 18, 2026 •

Copy link
Copy Markdown
Owner

Description

lint-test: ansible-lint passes on roles only passed when another .bats file had already installed the Galaxy roles. run-tests.sh passes the files in find order, which differs between runners, so the suite failed on Renovate PR #202 and on steward/lint-ci with The role 'geerlingguy.docker' was not found. The ansible-lint test now installs the roles from ansible/requirements.yml itself, after its tool check and before running ansible-lint, so it no longer depends on test order.

Closes #207

Changes

  • tests/bash/lint-test.bats: the ansible-lint test runs ansible-galaxy role install -r requirements.yml before ansible-lint, and fails with the galaxy output if the install fails. An earlier setup_file version was dropped after the Copilot review: on a machine without Ansible it ran before the "can be installed" test had installed it, so it never installed the roles.

Checklist

  • Linting passes locally (mise exec -- lefthook run pre-commit): not run. repo-steward works API-only without a clone, so CI (CI - Bats Tests, Lint) is the check.
  • Commit messages follow Conventional Commits (signed commits via steward commit-files)
  • Documentation updated (if applicable): the comment above the install explains why

lint-test.bats ran ansible-lint without installing the Galaxy roles that our
roles depend on (geerlingguy.docker). It only passed when docker-test.bats or
syntax-test.bats had run first and installed them. run-tests.sh passes the
.bats files in find order, which differs between runners, so the suite
failed whenever lint-test.bats came first:

  syntax-check[specific]: The role 'geerlingguy.docker' was not found

A setup_file now installs the roles from ansible/requirements.yml once before
the file's tests, so the lint test no longer depends on test order.
Copilot AI balanced review requested due to automatic review settings November 18, 2026 16:05
@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The Bats file now has a file-level setup that attempts to install Ansible Galaxy roles when ansible-galaxy is available. Installation failures produce a warning and do not fail setup.

Changes: Bats role setup

Layer / File(s) Summary
Role installation setup
tests/bash/lint-test.bats
The file-level setup checks for ansible-galaxy and attempts to install roles from ansible/requirements.yml. If installation fails, it prints a warning. If the command is unavailable, it skips installation.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other · Severity of issue fixed: Medium

Merge Risk: Minimal · up to 0ab94

The lint test now installs its required Ansible roles itself, so it no longer fails when it runs before the files that install them. This is a test-only change with no production impact. No concrete merge-blocking risk was found, and the author's CI run is the remaining check.

Pre-merge checks: Passed 4

Check name Status Explanation
Linked Issues check Passed Issue #207 requires that lint-test: ansible-lint passes on roles pass regardless of .bats file order. The diff at the reviewed head (0ab9445) adds a setup_file to tests/bash/lint-test.bats. It…
Out of Scope Changes check Passed The whole-PR diff against merge base 7ec2323 changes only tests/bash/lint-test.bats (+14 lines). The added setup_file and its comment directly implement the objective of #207. No unrelated files o…
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the main change: a test setup that installs the Galaxy roles so the ansible-lint test passes regardless of file execution order. The raw summary and objectives confirm this: a file-lev…
Full details: Linked Issues check

Explanation

Issue #207 requires that lint-test: ansible-lint passes on roles pass regardless of .bats file order. The diff at the reviewed head (0ab9445) adds a setup_file to tests/bash/lint-test.bats. It runs ansible-galaxy role install -r ansible/requirements.yml before this file's tests. Because ansible-lint runs offline, this installs the Galaxy roles (including geerlingguy.docker) that the lint test needs, so the test no longer depends on whether docker-test.bats or syntax-test.bats ran first. The change addresses the stated root cause. The change is test-only and adds no coding requirement beyond the issue. Not verified: that the roles installed by ansible-galaxy default location are the ones ansible-lint resolves, and that CI passes. No CI results were provided. The PR author also notes local lint was not run.

Full details: Out of Scope Changes check

Explanation

The whole-PR diff against merge base 7ec2323 changes only tests/bash/lint-test.bats (+14 lines). The added setup_file and its comment directly implement the objective of #207. No unrelated files or behavior changes are present. The existing tests are unchanged.

Full details: Title check

Explanation

The title names the main change: a test setup that installs the Galaxy roles so the ansible-lint test passes regardless of file execution order. The raw summary and objectives confirm this: a file-level Bats setup runs ansible-galaxy role install from ansible/requirements.yml before the lint tests. The title is specific and concise, and the test(lint) prefix matches the test-only change.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a5bc4cae-2040-4641-8737-9b7aefce9f68
Commits

Reviewing files that changed from the base of the PR and between 7ec2323 and 0ab9445.

Files selected for processing: 1

  • tests/bash/lint-test.bats

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

On a fresh machine, role installation is skipped before Ansible is installed and never retried, causing offline lint to fail to resolve geerlingguy.docker when this test file runs first.

1 open issue
What changed in this PR

This PR addresses #207 by preparing Galaxy role dependencies within the lint test file, rather than relying on another Bats file running first.

Changes:

  • Adds a once-per-file setup hook to install roles from ansible/requirements.yml when ansible-galaxy is available.
  • Emits a TAP warning if installation fails, leaving the lint test to report missing dependencies.
File Description
tests/​bash/​lint-test.bats Adds Galaxy role installation and failure reporting before the file’s tests.

Review effort: Balanced

Comment thread tests/bash/lint-test.bats Resolved
Copilot review on #208: on a machine without Ansible, setup_file skipped the
role install, and the later "ansible-lint is available or can be installed"
test installs Ansible without rerunning it. The lint test then still couldn't
resolve geerlingguy.docker when this file ran first.

Install the roles inside the ansible-lint test instead, after its tool check
and before ansible-lint runs, and fail with the galaxy output if that fails.
Copilot AI balanced review requested due to automatic review settings November 18, 2026 16:11
@DevSecNinja DevSecNinja changed the title test(lint): install the Galaxy roles in lint-test setup_file test(lint): install the Galaxy roles in the ansible-lint test Nov 18, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The lint test installs roles twice under different configuration contexts, causing clean repository-root runs to download roles into two separate directories.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 18, 2026 16:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the change to install Galaxy roles before running the lint test.

No open issues 🎉

Review effort: Balanced

@DevSecNinja
DevSecNinja merged commit d468ddb into main Nov 18, 2026
22 checks passed
@DevSecNinja
DevSecNinja deleted the steward/bats-order-independent-lint branch November 18, 2026 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bats lint test fails when lint-test.bats runs before the role install

2 participants