Add regression-range hackbot agent (mozregression --prompt bisection)#6354
Open
msujaws wants to merge 1 commit into
Open
Add regression-range hackbot agent (mozregression --prompt bisection)#6354msujaws wants to merge 1 commit into
msujaws wants to merge 1 commit into
Conversation
Adds a new hackbot agent that automatically finds a regression range for a regression bug by bisecting with mozregression's --prompt mode (mozilla/mozregression#2197): it reads the bug, determines good/bad bounds and a natural-language reproduction directive, drives a real bisection, and reports the resulting changeset/pushlog range -- recording a Bugzilla comment and, at high confidence, proposed field updates (recorded, never auto-applied). - libs/agent-tools/agent_tools/mozregression.py: framework-neutral run_mozregression tool (async subprocess wrapper) that invokes mozregression --prompt and parses the final range. Handles both regression ("Last good"/"First bad") and --find-fix ("First good"/"Last bad") wording, takes the last pushlog URL, and falls back to the pushlog fromchange/tochange. Always returns a dict; never raises. - agents/regression-range/: the agent (mirrors frontend-triage) -- bugzilla read + searchfox + mozilla_vcs + the mozregression tool + recorded actions. System prompt: scope to deterministic-in-fresh-headless-profile repros (incl. about:/ chrome pages and browser UI: menus, tabs, sidebars, toolbars); resolve pref names via searchfox (never guess); anchor the good bound just before a suspected regressor's landing; run scroll/layout checks at a realistic window size. - services/hackbot-api: register the agent (RegressionRangeInputs + AGENT_REGISTRY). - services/hackbot-ui: label overrides for the new finding fields. - Tests for the tool's argv construction and range parsing, and the registry entry. Issue mozilla#6259
Collaborator
|
This is great, I wanted to talk to you about doing this, but you did it earlier than I reached out :) What are your thoughts on the split between this and frontend-triage? Should frontend-triage use this as a sub-agent instead of them running separately? |
Comment on lines
+20
to
+25
| # mozregression is installed from the fork carrying PR #2197 (the --prompt mode | ||
| # is not yet in a release), so it lives outside the workspace lockfile. Pin to a | ||
| # specific commit once the PR settles; the branch tracks the PR for now. | ||
| ARG MOZREGRESSION_REF="git+https://github.com/msujaws/mozregression.git@prompt-devtools-mcp-verdict" | ||
| RUN --mount=type=cache,target=/root/.cache/uv \ | ||
| uv pip install --python /opt/venv "mozregression @ ${MOZREGRESSION_REF}" |
Member
There was a problem hiding this comment.
Can we instead use [tool.uv.sources] in pyproject.toml so it gets installed with other deps without a custom step in Dockerfile?
suhaibmujahid
left a comment
suhaibmujahid
left a comment
Member
There was a problem hiding this comment.
You might want to add the new agent to the CODEOWNERS file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #6259 — a hackbot agent that automatically finds a regression range for a regression bug.
What it does
Given a Bugzilla bug id, the agent:
not_a_regression/not_automatable).mozilla_vcs) when available, anchoringgoodjust before it.mozregression --prompt(mozilla/mozregression#2197) to drive a real bisection — each candidate build is classified GOOD/BAD by an agent driving the Firefox DevTools MCP (the natural-language equivalent ofgit bisect run).cf_has_regression_range,regressed_by, clearingregressionwindow-wanted). Actions are recorded for human review — never auto-applied.Key components
libs/agent-tools/agent_tools/mozregression.py— framework-neutralrun_mozregressiontool: an async subprocess wrapper that builds the CLI invocation and parses the final range. Handles both regression (Last good/First bad) and--find-fix(First good/Last bad) wording, takes the last pushlog URL, and falls back to the pushlogfromchange/tochange. Always returns a dict; never raises.agents/regression-range/— the agent, mirroring thefrontend-triagestructure (brokered Bugzilla read + searchfox + mozilla_vcs + the mozregression tool + recorded actions). The system prompt scopes to reproductions that are deterministic in a fresh headless profile (includingabout:/chrome pages and browser UI — menus, tabs, sidebars, toolbars), resolves prefs via searchfox, anchors bounds to a suspected regressor's landing, and runs scroll/layout checks at a realistic window size.services/hackbot-api— registers the agent (RegressionRangeInputs+AGENT_REGISTRY).services/hackbot-ui— label overrides for the new finding fields.Verification (local)
run_mozregressiontool drove a real#2197 --promptbisection to a changeset/pushlog range.docker composeon real bugs: correct scoping decisions, searchfox-verified prefs, regressor-anchored bounds, and honestinconclusivereporting when a symptom wasn't observable headless (while still identifying the likely regressor).Notes
MOZREGRESSION_REFbuild arg). Pin to a specific commit once Bump python-igraph from 0.8.3 to 0.9.0 #2197 lands / is released.docker composeruns,BUGZILLA_API_URLmust include the/restsuffix.