Skip to content

ci(test): report bats results with the shared test-report action - #215

Open
DevSecNinja wants to merge 5 commits into
mainfrom
steward/test-report-trial
Open

DevSecNinja wants to merge 5 commits into
mainfrom
steward/test-report-trial

Conversation

@DevSecNinja

@DevSecNinja DevSecNinja commented Nov 20, 2026 •

Copy link
Copy Markdown
Owner

Trial of the shared test-report action from DevSecNinja/.github#604.

  • Replaces the third-party mikepenz/action-junit-report step with DevSecNinja/.github/actions/test-report, which writes the bats results to the job summary (counts per suite plus a table of failed tests) and adds ::error annotations on the failing lines. It uses standard-library Python only.
  • Drops checks: write, which only the old reporter needed.
  • Fixes tests/bash/run-tests.sh: under set -e, a failing bats exited before report.xml was renamed to test-results.xml, so failed runs never had a report. That affected the old reporter too. The script still exits with bats' exit code.
  • Keeps the existing test-results.xml artifact upload.

Trial runs:

The action is pinned to the reviewed head of DevSecNinja/.github#604 for this trial. Before merging, I'll repin it to the released commit SHA with a version comment.

Closes #214

Replace the third-party JUnit reporter with the shared test-report action
and drop checks: write.

Refs #214
Copilot AI balanced review requested due to automatic review settings November 20, 2026 09:44
@coderabbitai

coderabbitai Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The CI workflow removes the checks: write permission and replaces the JUnit report action with the shared test-report action. The replacement reads the Bats test results file and runs even if an earlier step fails.

Changes: CI test reporting

Layer / File(s) Summary
Workflow permissions and test reporting
.github/workflows/ci.yml
The workflow removes checks: write and uses the shared test-report action with tests/bash/test-results.xml and the title Bats tests. The action retains the if: always() condition.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Change: Feature

Suggested reviewers: copilot

Merge Risk: Minimal · up to ce924

The CI workflow now reports Bats results through the shared test-report action, using the job summary and annotations, and no longer needs checks: write. No concrete defect was found. Before merging, confirm that no required status check refers to the old Bats Test Results name, and repin the action to the released commit SHA as planned.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title "ci(test): report bats results with the shared test-report action" matches the main change. The workflow replaces mikepenz/action-junit-report with the DevSecNinja/.github test-report action…
Linked Issues check Passed Directly linked issue #214 has three coding requirements in .github/workflows/ci.yml at the reviewed head. (1) Replace the third-party reporter: the mikepenz/action-junit-report step is removed and re…
Out of Scope Changes check Passed The only changed file in the reviewed diff summary is .github/workflows/ci.yml, and the changes map to #214. The replaced reporter step, the removed checks: write permission, and the removed old repor…
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Full details: Title check

Explanation

The title "ci(test): report bats results with the shared test-report action" matches the main change. The workflow replaces mikepenz/action-junit-report with the DevSecNinja/.github test-report action to report Bats results. The title names the purpose and the new action, so it is specific and accurate.

Full details: Linked Issues check

Explanation

Directly linked issue #214 has three coding requirements in .github/workflows/ci.yml at the reviewed head. (1) Replace the third-party reporter: the mikepenz/action-junit-report step is removed and replaced by DevSecNinja/.github/actions/test-report, with paths set to tests/bash/test-results.xml and title Bats tests. (2) Drop checks: write: the top-level permissions block is now only contents: read. (3) Keep the artifact upload: the actions/upload-artifact step for test-results (path tests/bash/test-results.xml, retention-days 30, if: always()) is unchanged. The job summary and failing-line annotations come from the external shared action. Its behavior is not in this repository and was not inspected, so this verdict covers the workflow changes that the issue requests. The action is pinned to commit 7046a8ea, which the PR marks as a trial pin. The issue does not require a released pin. No automated test requirement applies.

Full details: Out of Scope Changes check

Explanation

The only changed file in the reviewed diff summary is .github/workflows/ci.yml, and the changes map to #214. The replaced reporter step, the removed checks: write permission, and the removed old reporter options (check name, detailed summary, passed-test display) all belong to the reporter swap. No unrelated steps, dependencies, or triggers changed. The Python setup, test run, and artifact upload steps are unchanged.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c44d04f4-2f4c-492f-bbc4-e32eab83a366
Commits

Reviewing files that changed from the base of the PR and between 6dda8d2 and ce92411.

Files selected for processing: 1

  • .github/workflows/ci.yml

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the CI test-reporting change.

No open issues 🎉

What changed in this PR

This PR trials the shared test-report action for Bats results in the infrastructure CI workflow, addressing #214.

Changes:

  • Replaces mikepenz/action-junit-report with the shared action for job summaries and error annotations.
  • Removes the unused checks: write permission while preserving the XML artifact upload.
  • Pins the trial action to a commit, with a released commit pin planned before merging.
File Description
.github/​workflows/​ci.yml Switches test reporters and removes the old reporter’s write permission.

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 20, 2026 09:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The automatically discovered temporary test in tests/bash/report-trial.bats contains an assertion that can never pass, causing full test runs and CI to fail.

1 open issue

Review effort: Balanced

Comment thread tests/bash/report-trial.bats Resolved
Copilot AI balanced review requested due to automatic review settings November 20, 2026 09:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The temporary tests/bash/report-trial.bats suite is automatically discovered and always fails because it expects goodbye from echo hello, causing every full test run and CI job to fail.

1 open issue

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 20, 2026 09:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are identified for the shared Bats reporting and failed-test report handling changes.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 20, 2026 09:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded, although verification was limited to static review without runtime tests or lint checks.

No open issues 🎉

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci(test): report bats results with the shared test-report action

2 participants