fix(gooddata-eval): stop alert sim-user drifting trigger and filters

[myhoai]

JIRA: QA-28623
risk: nonprod

Summary by CodeRabbit

• Bug Fixes
  • Improved alert setup handling for “always” and “once” trigger schedules.
  • Clarified the difference between no filters and unspecified filters.
  • Improved support for range-based thresholds and time windows.
  • Added validation for recipients, trigger cadence, filters, thresholds, and operators before confirmation.
  • Prevented unspecified fields, filters, and time windows from being inferred automatically.
  • Improved filter value handling and alert confirmation summaries for greater clarity.
  • Preserved the original alert request during simulated interactions for more accurate responses.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 0ee15288-deaa-4da1-9cc4-ec2bec14d064

📥 Commits

Reviewing files that changed from the base of the PR and between ef5f860 and 32cc3a4.

📒 Files selected for processing (2)

• packages/gooddata-eval/src/gooddata_eval/core/agentic/alert_skill.py
• packages/gooddata-eval/tests/test_agentic_alert_skill.py

🚧 Files skipped from review as they are similar to previous changes (1)

• packages/gooddata-eval/src/gooddata_eval/core/agentic/alert_skill.py

---

📝 Walkthrough

Walkthrough

The alert skill now distinguishes unspecified and empty filters, preserves filter lists and threshold ranges, passes the original question to simulated responses, enforces ALWAYS and ONCE cadence, and validates complete alert proposals before confirmation.

Changes

Alert semantics

Layer / File(s)	Summary
Filter normalization and validation packages/gooddata-eval/src/gooddata_eval/core/agentic/alert_skill.py, packages/gooddata-eval/tests/test_agentic_alert_skill.py	Filter normalization and comparison distinguish absent, unspecified, empty, and exact filter values. Tests cover filter precedence, subset matching, and “None (All time)”.
Prompt, simulation, and confirmation rules packages/gooddata-eval/src/gooddata_eval/core/agentic/alert_skill.py, packages/gooddata-eval/tests/test_agentic_alert_skill.py	Prompts define ALWAYS and ONCE cadence, preserve BETWEEN bounds, retain the original question, reject invented fields and filters, and validate recipients, triggers, filters, thresholds, and operators before confirmation. Tests cover prompt generation and interaction handling.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant EvaluationLoop
  participant AlertSkill
  participant SimulatedUser
  EvaluationLoop->>AlertSkill: pass original request
  AlertSkill->>SimulatedUser: generate response with question and conversation history
  SimulatedUser->>AlertSkill: confirm alert fields and cadence

Loading

Suggested reviewers: hkad98, lupko, pcerny

Poem

A rabbit checks each alert with care,
“Always” and “once” are written there.
Empty filters stay empty and clear,
Two threshold bounds appear sincere.
All fields confirm before I hop—
No hidden data, no guesswork stop!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the alert simulator-user fix and names the affected trigger and filter context.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/gooddata-eval/src/gooddata_eval/core/agentic/alert_skill.py`:
- Around line 200-205: Update generate_simulated_alert_response and its call
from run_agentic_alert_skill to include the original question in the generated
system prompt, ensuring simulated users can derive required filters even when
normalized filter prose is None. Preserve the existing filter guidance and add
an interaction test where the agent requests a required filter without repeating
it.
- Around line 175-178: Use the normalized trigger key from the trigger-handling
flow when constructing the goal text, so omitted triggers are rendered as ALWAYS
rather than “not specified”; update the relevant goal-generation logic near the
trigger instructions and add a regression test using a fixture without Trigger
that rejects an ONCE proposal.
- Around line 318-331: Update _normalize_expected_filters so non-list,
non-marker Filters values return None instead of arbitrary text, matching its
documented contract and allowing _check_filters to treat them as unspecified.
Preserve any prose separately through the existing simulated-user state or
context mechanism if it is needed there, without changing marker handling or
exact list expectations.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 5b3316b3-07d6-468c-8810-579a35579094

📥 Commits

Reviewing files that changed from the base of the PR and between cbd27a1 and ef5f860.

📒 Files selected for processing (2)

• packages/gooddata-eval/src/gooddata_eval/core/agentic/alert_skill.py
• packages/gooddata-eval/tests/test_agentic_alert_skill.py

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.74%. Comparing base (2f1a82f) to head (a84b842).
⚠️ Report is 4 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1721      +/-   ##
==========================================
+ Coverage   78.59%   78.74%   +0.15%     
==========================================
  Files         271      271              
  Lines       18772    18794      +22     
==========================================
+ Hits        14754    14800      +46     
+ Misses       4018     3994      -24     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.