[gapilongo]

FIX-3: decision policy v3. AI-benign alerts close, and retro-reopen replaces the forward gates

Closes #348. Tier T3: this changes the decision policy and INV-03. Builds on #345 (branched from its tip; it will be rebased once #345 merges).

What changed

1. Model-led close at auto_full_audit (domain/closure.py, domain/routing.py). The tier is per tenant and defaults to off.

   • An alert closes when all of these hold:
     • the deep verdict is benign, at any certainty;
     • the fast tier did not say likely_threat or confirmed_threat;
     • severity is info, low, medium or high;
     • the threat score is below the response policy.
   • It never closes when any of these apply:
     • critical severity;
     • a never-close class: credential access (including the vendor auth-failure groups and T1531), vendor_unknown, log clearing, tampering, ransomware, named persistence;
     • steering text;
     • a recent true positive;
     • the alert was normalised by a model;
     • the alert was reopened by the safety net.
   • Removed: the 8/85 thresholds (routing.model_close_* settings), the campaign check, the unmapped block, and the grey zone for a benign deep verdict.
   • Approved benign evidence still closes first (no audit needed). The noise catalogue now only names the family in the decision reason.

2. Retro-reopen (core/retro_reopen.py, next to core/benign_learning.py). On every threat path, the tenant's model_closed alerts that closed in the previous 24 h and share a host, user or IP with the threat are reopened. The paths are:

   • plan_response and the malicious close (nodes._forget_benign_learning);
   • an escalation whose model verdict is malicious or suspicious (nodes.escalate);
   • an analyst true_positive (api/routers/review.py);
   • a blind-audit malicious label (api/routers/blind_audit.py), both through api.common.invalidate_verdicts.

   A reopened alert is:

   • set back to queued, with its run reset as on a reprocess (API paths enqueue it at once; workflow paths are picked up by the orphan sweep);
   • marked summary.reopened, which reaches the next run's state and blocks a model close;
   • attached to the threat's escalation through related_alert_ids;
   • audited alert.reopened with reason related_threat, plus a timeline event.

   The reopen is idempotent (only a still-closed ai_closed:* alert is selected) and tenant-scoped. Failures are counted in lgsotf_retro_reopen_failures_total{path} and logged as errors, never raised; reopens are counted in lgsotf_retro_reopened_total{path}.

3. INV-03 is amended: tier off never closes on the model's word; critical and never-close classes never close; a related threat reopens a model close.

   • New proofs:
     • test_retro_reopen.py: the escalation path; plan_response and the malicious close checked separately, so dropping either call site fails; tenant scope, the 24 h window, idempotence, attach, audit, and the failure metric;
     • test_retro_reopen_api.py: analyst TP, blind-audit label, and the failure metric;
     • the critical and high and unmapped tests in test_closure.py.
   • The routing_policy runtime check is updated.
   • The adversarial auto-tier test now asserts what v3 allows, not 0 malicious closes. The corpus_v1 auto-tier run uses the fake model live (network blocked), because model closes change later correlation prompts and the tier-off cassettes no longer match.

4. Docs: the threat model (TM-WRK-2, with a v3 lead and residual risk), architecture, operations metrics, configuration (thresholds removed), INV docs, and changelog.d/FIX-3.md. openapi.json and schema.gen.ts gain summary.reopened (additive).

Measured

Fake model (eval harness plus gates). The fake model follows injected text: this is the worst case for v3, not a real model.

Dataset (benign / malicious)	Tier off: benign / malicious closed, gate	auto_full_audit benign closed: before reopen → still closed	auto_full_audit malicious closed: before reopen → still closed	Critical or never-close closed
corpus_v1 (312 / 208)	8 / 0, pass	100 (32.1%; 27.1-37.4) → 17 (5.4%; 3.4-8.6)	58 (27.9%; 22.2-34.3) → 1 (0.5%; 0.1-2.7): cv1-0392	0
adversarial_v2 (24 / 71)	3 / 0, pass	9 (37.5%) → 8 (33.3%; 18.0-53.3)	14 (19.7%; 12.1-30.4) → 11 (15.5%; 8.9-25.7)	0
synthetic_v1 (21 / 35)	5 / 0, pass	18 (85.7%) → 18	1 → 1 (2.9%; 0.5-14.5): adv-009	0
phishing_v1 (11 / 24)	3 / 0, pass	4 (36.4%) → 3 (27.3%)	1 (4.2%) → 0	0

• At auto_full_audit the malicious-close gates (max 0) fail by design. The tier-off gates pass unchanged.
• The adversarial items still closed after reopen are isolated injection items with no related threat in the stream: log-field hijack, tool-output injection, encoding, two title-camouflage items that are now unmapped-closable, and activity camouflage. Retro-reopen cannot reach them; the audit (model_closed) and blind audit must.

DeepSeek simulation (fix3_sim.py).

• Method: the real decide at auto_full_audit, in stream order; a model close is recorded with its entities.
• Reopen trigger: a threat path the recording shows (a response plan, or an escalation a model judged malicious or suspicious) reopens earlier closes within 24 h that share a host, user or IP.
• Analyst TPs and blind-audit labels are not in the recordings, so the reopen counts are a lower bound.

Source	Benign: before reopen → after	Malicious: before reopen → after
Dev run 2 (100 Wazuh, 60 / 40)	25/60 (41.7%; 30.1-54.3) → 7/60 (11.7%; 5.8-22.2)	2/40 (5.0%; 1.4-16.5) → 1/40 (2.5%; 0.4-12.9)
Scorecard corpus_v1 (DeepSeek V4, analysed items only: 107 benign / 30 malicious)	66/312 (21.2%; 17.0-26.0) → 6/312 (1.9%; 0.9-4.1)	1/208 (0.5%; 0.1-2.7) → 0/208

Each malicious close:

• cv1-0247 (dev, low): reopened by cv1-0232, a malicious alert the model escalated as a threat.
• cv1-0026 (dev, low), Wazuh 60106 "Windows Logon Success", the successful logon of a password-spray campaign: not reopened by an automatic path.
  • Its spray step cv1-0021 (60122, same hosts dc-02 / sales-ws-03 and IPs, 5 min later) is now never-close (auth failure) and goes to an analyst. Both models called that step benign, so it triggers nothing automatically.
  • An analyst true_positive on cv1-0021 reopens cv1-0026 (the analyst path, inside 24 h).
• cv1-0358 (corpus_v1, low): reopened by cv1-0406, a benign alert the model escalated as suspicious.

Note for the lead: in these attack-dense streams, retro-reopen also reopens most benign closes. In the dev run, 18 of the 25 benign closes are reopened because a suspicious alert shares their host. On a real tenant, where attacks are rare, far fewer would be. But it is the cost of the safety net, and the "about 25/60" acceptance holds only before reopen.

Threat note

• What can go wrong:
  • A persuaded deep tier closes an attack step, including high severity.
  • An attack whose other steps share no host, user or IP, surface after 24 h, or are never recognised as a threat stays closed. Examples: the isolated adversarial items, and the spray success above until an analyst confirms the spray.
• Mitigations:
  • the tier is per tenant, off by default, and changed only by an audited admin;
  • critical alerts and the never-close classes never close (auth failures included);
  • steering text and a recent TP block;
  • every close is model_closed / ai_closed:audit_required and enrolled in blind audit;
  • retro-reopen on all four threat paths, counted and alerted on failure;
  • a reopened alert never model-closes again.
• Residual risk:
  • both tiers come from one model family;
  • the fast tier's suspicious no longer blocks;
  • the reopen is backward-only and entity-bound;
  • a reopened alert can still close at the gate through an analyst-confirmed verdict cache or a suppression rule (not a model close);
  • workflow-path reopens wait for the orphan sweep (about 2× the visibility timeout) before their new run.

Checks

• ruff, mypy (383 files), lint-imports (14 kept), changelog check.
• pytest tests/unit tests/invariants tests/deploy: 5131 passed.
• Integration (eval, engine, e2e, review, retro-reopen API, revocation, blind audit, ingestion): 209 passed, 1 skipped, 1 failed. Two engine model-closure tests were updated for v3 and pass after a rerun; the remaining failure is the phishing_v1 cassette replay also fails on main ([BUG] phishing_v1 cassette replay is time-dependent: 4 items miss after some hours #346, fixed separately).

🤖 Generated with Claude Code

https://claude.ai/code/session_01JmdYVXkgpNHoygVJCrUeZc

[claude]

Review of FIX-3 (decision policy v3)

I read core/retro_reopen.py and the new model_closed_sharing_entities query in db/repos/alerts.py. I did not run the test suite, and I only skimmed the closure.py/routing.py diffs, the docs and the eval changes.

Overall: The design is sound. Reopening is idempotent, tenant-scoped and audited. A failure is counted and logged without failing the threat's own decision. Tier off stays the default. The measured results are reported honestly, including the malicious items that still close (cv1-0392, adv-009) and the max-0 gates that fail by design.

Potential issues

1. Over-broad entity matching can cause reopen storms. Matching is on any shared host, user or IP, with no normalization or allowlist. Values like SYSTEM, root, 127.0.0.1, 0.0.0.0, a shared NAT or gateway, or a domain controller will match nearly every model-closed alert in the window. The module docstring accepts this ("a wider match only reopens more"), but one noisy threat can requeue up to 200 alerts. That is the analyst load the model-close tier is meant to remove. Consider an ignore-list for generic principals and loopback/unspecified IPs, and a metric on the number reopened per call.
2. Case and format sensitivity. The ?| jsonb match is exact, so HOST1 and host1, or DOM\\user and user, do not match. That is a false-negative path for the safety net. Confirm that normalization lowercases entities, or lowercase on both sides.
3. Silent truncation at MAX_REOPEN=200. If more candidates match, the rest are never reopened and nothing is logged or counted. Please log or count when len(candidates) == MAX_REOPEN, or loop until the set is drained.
4. The window is measured from closed_at, not from the related event. An attack step the model closed 25h earlier is never reopened, even if it belongs to the same intrusion. It is documented, but it should be in the TM-WRK-2 residual risk if it isn't already.
5. Reopen runs outside the threat's transaction. Each alert uses its own tenant_tx. If one alert fails mid-loop, the exception is swallowed in reopen_counted, the remaining alerts stay closed, and the ids already reopened are lost from the return value (and from the escalation link and counter). The failure direction is safe, but nothing guarantees a later threat will pick up the leftovers. Consider catching per alert so one bad alert doesn't abort the rest.
6. Minor: deadline = escalation.sla_deadline or datetime.now(UTC) + REOPEN_WINDOW ignores the now passed into reopen_related_model_closes, which hurts determinism in tests. The update(...) call in _reopen_one is also formatted differently from the rest of the file.

Security

• Critical severity and the never-close classes are kept, and the reopened flag blocks a second model close. Good.
• Letting high severity close on the model's word is a significant widening of INV-03. Retro-reopen is the only compensating control, and it needs a later related threat to appear. An isolated high-severity attack, like the injection items you list in the adversarial set (8 benign and 11 malicious still closed), goes unseen. This is documented and tier-gated. I'd suggest an explicit warning and an audit event on the tier change, and making the lgsotf_retro_reopen_failures_total alert a prerequisite for enabling the tier.
• Attacker-controlled alert text influences the model's verdict, so the steering-text check is the main defence. Please make sure a test covers it on the high-severity path specifically.

Tests
Coverage is good: separate call-site tests for plan_response and the malicious close, the window, tenant scope, idempotence and the failure metric. I'd add tests for:

• the MAX_REOPEN truncation and the MAX_ENTITY_VALUES cap
• generic or shared entity values (item 1) and case mismatches (item 2)
• a partial failure mid-loop (item 5)
• a threat with an empty entity set (the repo returns [], which looks right but is untested)

Process
This PR is stacked on #345. Please re-run CI after the rebase, since the eval cassettes change.

🤖 Generated with Claude Code