Repository navigation
refactor(skillspector): report findings as checks only, drop the PR comment - #262
Conversation
…omment Remove the comment job, its stale-comment path and the comment-only script code and tests. Findings now surface through the Scan AI primitives gate, the SkillSpector Code Scanning check and the run summary. Accepted findings already upload as note-level rules without a security severity, so they fail neither check. Closes #250 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 55 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues were identified for the change removing automated PR comments in favor of checks-only reporting.
No open issues 🎉
What changed in this PR
This PR simplifies SkillSpector reporting by removing automated PR comments while preserving the scan gate, Code Scanning annotations, Actions run summary, and report artifacts.
Changes:
- Removes the comment job and stale-comment handling from the workflow.
- Deletes comment-only reporting code and tests, and makes the summary path required.
- Updates documentation to describe checks-only reporting; accepted-finding SARIF handling remains unchanged.
| File | Description |
|---|---|
tests/test_skillspector_report.py |
Removes comment tests and retains summary, SARIF, and CLI coverage. |
scripts/skillspector-report.py |
Removes PR-comment functionality and requires a summary input path. |
CONTRIBUTING.md |
Documents checks-only reporting and removes references to the deleted job. |
.lefthook.toml |
Updates only the explanatory comment to mention summary and SARIF reporting. |
.github/workflows/skillspector.yml |
Removes the comment job and its unused scan output. |
Review effort: Balanced
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
SkillSpector findings now show up as checks only. The bot no longer posts a PR comment.
Changes
.github/workflows/skillspector.yml: removed thePost SkillSpector reportjob, including the "Not needed" path that marked an old comment as stale, and the scan job'sscannedoutput that only that job used. The run summary ($GITHUB_STEP_SUMMARY), the SARIF upload and theskillspector-reportsartifact are unchanged.scripts/skillspector-report.py: removed the comment-only code (--comment-pr,--not-needed-pr,gh/upsert_comment/not_needed, the HTML marker and thepr:alert filter). The summary path is now a required argument.tests/test_skillspector_report.py: removed the comment tests and kept a CLI test that fails on a missing summary.CONTRIBUTING.mdand the.lefthook.tomlcomment no longer mention the PR comment or the removed check.Do the gate and the Code Scanning check agree?
Yes, so no change was needed.
build_sarifalready uploads accepted findings under a separate<rule>/acceptedrule atnotelevel with nosecurity-severity, because GitHub ignores SARIF suppressions. Code Scanning's severity threshold therefore never fails on an accepted finding. The open SkillSpector alerts onmainconfirm it: every accepted rule (TM1/accepted,PE3/accepted,MP3/accepted, …) has no security severity. #245 also passed the SkillSpector check while carrying accepted findings. On #249, both checks fail because of unaccepted findings, not accepted ones.Result
A PR that changes AI primitives gets the Scan AI primitives gate, the SkillSpector Code Scanning check with annotations and the run summary, and no comment.
Testing
test_skillspector*.py: 38 tests pass (1 skipped).tests/validate-prompts.shpasses.Closes #250