Skip to content

refactor(skillspector): report findings as checks only, drop the PR comment - #262

Merged
DevSecNinja merged 2 commits into
mainfrom
devsecninja-skillspector-checks-only
Nov 16, 2026
Merged

DevSecNinja merged 2 commits into
mainfrom
devsecninja-skillspector-checks-only

Conversation

@DevSecNinja

Copy link
Copy Markdown
Owner

SkillSpector findings now show up as checks only. The bot no longer posts a PR comment.

Changes

  • .github/workflows/skillspector.yml: removed the Post SkillSpector report job, including the "Not needed" path that marked an old comment as stale, and the scan job's scanned output that only that job used. The run summary ($GITHUB_STEP_SUMMARY), the SARIF upload and the skillspector-reports artifact are unchanged.
  • scripts/skillspector-report.py: removed the comment-only code (--comment-pr, --not-needed-pr, gh/upsert_comment/not_needed, the HTML marker and the pr: alert filter). The summary path is now a required argument.
  • tests/test_skillspector_report.py: removed the comment tests and kept a CLI test that fails on a missing summary.
  • CONTRIBUTING.md and the .lefthook.toml comment no longer mention the PR comment or the removed check.

Do the gate and the Code Scanning check agree?

Yes, so no change was needed. build_sarif already uploads accepted findings under a separate <rule>/accepted rule at note level with no security-severity, because GitHub ignores SARIF suppressions. Code Scanning's severity threshold therefore never fails on an accepted finding. The open SkillSpector alerts on main confirm it: every accepted rule (TM1/accepted, PE3/accepted, MP3/accepted, …) has no security severity. #245 also passed the SkillSpector check while carrying accepted findings. On #249, both checks fail because of unaccepted findings, not accepted ones.

Result

A PR that changes AI primitives gets the Scan AI primitives gate, the SkillSpector Code Scanning check with annotations and the run summary, and no comment.

Testing

  • test_skillspector*.py: 38 tests pass (1 skipped).
  • tests/validate-prompts.sh passes.
  • All pre-commit hooks pass on a signed commit.

Closes #250

…omment

Remove the comment job, its stale-comment path and the comment-only script code and tests. Findings now surface through the Scan AI primitives gate, the SkillSpector Code Scanning check and the run summary. Accepted findings already upload as note-level rules without a security severity, so they fail neither check.

Closes #250

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@DevSecNinja
DevSecNinja requested a balanced review from Copilot November 16, 2026 12:33
@coderabbitai

coderabbitai Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 55 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 5e3d4dad-c008-4931-b3bc-91b79223b959
Commits

Reviewing files that changed from the base of the PR and between eef88cf and 9dc3336.

Files selected for processing: 5

  • .github/workflows/skillspector.yml
  • .lefthook.toml
  • CONTRIBUTING.md
  • scripts/skillspector-report.py
  • tests/test_skillspector_report.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the change removing automated PR comments in favor of checks-only reporting.

No open issues 🎉

What changed in this PR

This PR simplifies SkillSpector reporting by removing automated PR comments while preserving the scan gate, Code Scanning annotations, Actions run summary, and report artifacts.

Changes:

  • Removes the comment job and stale-comment handling from the workflow.
  • Deletes comment-only reporting code and tests, and makes the summary path required.
  • Updates documentation to describe checks-only reporting; accepted-finding SARIF handling remains unchanged.
File Description
tests/​test_skillspector_report.py Removes comment tests and retains summary, SARIF, and CLI coverage.
scripts/​skillspector-report.py Removes PR-comment functionality and requires a summary input path.
CONTRIBUTING.md Documents checks-only reporting and removes references to the deleted job.
.lefthook.toml Updates only the explanatory comment to mention summary and SARIF reporting.
.github/​workflows/​skillspector.yml Removes the comment job and its unused scan output.

Review effort: Balanced

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@DevSecNinja
DevSecNinja merged commit 033e5d1 into main Nov 16, 2026
23 checks passed
@DevSecNinja
DevSecNinja deleted the devsecninja-skillspector-checks-only branch November 16, 2026 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(skillspector): report findings as checks only, drop the PR comment

2 participants