FIX-2: decision policy v2.1, catalogued unmapped noise and a campaign check that lets one rule repeat

changelog.d/FIX-2.md

@@ -0,0 +1,36 @@
+### Changed
+
+- Model-led closure (`ai_closure.tier = auto_full_audit`) can now close
+  routine Wazuh noise that ships without an ATT&CK mapping. A reviewed,
+  versioned catalogue of noise families, keyed by vendor and detection rule
+  id, lifts the "no ATT&CK mapping" block for: package installs and updates
+  (dpkg 2901, 2902; yum 2932, 2933), Windows software installs (60612),
+  Defender scan and signature status (62107, 62108, 62130) and a single web
+  4xx (31101). It lifts nothing else: the tier, severity, both model tiers
+  benign at the install thresholds, steering text, a recent true positive
+  and a campaign in progress still block. Such a close's decision reason
+  names the family and the catalogue version.
+- Alerts now record the format whose normalizer built them
+  (`normalized.normalizer`: `wazuh`, `canonical`, `webhook`, `llm`, ...).
+  Models never see it. Alerts stored before this release have none, so the
+  catalogue does not apply to them.
+- The campaign check is unchanged (FIX-1). A check that lets one detection
+  rule repeat without blocking was deferred.
+
+### Security
+
+- Authentication failures never close on the models' word, whatever ATT&CK
+  technique the vendor gave them: a Wazuh rule in the groups
+  `authentication_failed`, `authentication_failures`, `invalid_login` or
+  `login_denied`, the rules 6250 (Asterisk) and 44733 (FortiAuthenticator),
+  and ATT&CK T1531, where Wazuh maps its Windows logon failures. Before,
+  25 stock Wazuh rules of that kind (logon failures, a failed logon from a
+  malicious IP, a possible replay attack) were closable.
+- The catalogue reads only the rule id of the vendor's own event, and only
+  on an alert the Wazuh normalizer built, never the title, description or
+  rule-name text: a custom rule that copies a stock description, a forged
+  rule-name prefix or a canonical push of a Wazuh document stays unmapped.
+- The workflow state keeps the vendor rule (`raw.rule`: id, level,
+  description, groups, ATT&CK) when an oversized raw event is dropped. A
+  Wazuh alert whose rule cannot be read, or whose groups list had to be cut,
+  never closes on the models' word.

docs/ai-system-card.md

@@ -279,6 +279,12 @@ compressed related alerts and bursts (a campaign in progress blocks it), and
 queue without them. That is a simulation, not a re-measurement. The miss-rate result stands; the
 auto-close result is not yet a statement about the model.
 
+Decision policy v2.1 (FIX-2) adds a reviewed catalogue of unmapped Wazuh
+noise rules (by rule id) and makes authentication failures a never-close
+class whatever their ATT&CK id; the campaign check stays FIX-1's.
+Simulated on the FIX-1 dev run's 100 recorded alerts, 2 of the 60
+benign alerts close and none of the 40 malicious.
+
 ### 7.2 Scorecard v1: `qwen2.5:7b` on a local Ollama
 
 [scorecards/v1/qwen2.5-7b/scorecard.md](scorecards/v1/qwen2.5-7b/scorecard.md),

docs/architecture.md

@@ -290,10 +290,13 @@ Rules:
     the severity is info, low or medium, the detection rule maps it to
     ATT&CK and not to a never-close class (credential access, log clearing,
     tampering, ransomware, persistence through an object the attacker
-    names), triage found no steering text, no entity has a recent true
-    positive, no campaign is in progress (a burst, or a related alert a
-    model judged malicious or suspicious or still open for review), and
-    both model tiers say `benign`: fast-tier likelihood at
+    names), or it is unmapped but its vendor rule id is in the reviewed
+    noise catalogue (`lg_sotf.domain.noise_families`, FIX-2: package
+    installs, Windows software installs, Defender scan and signature status,
+    web 4xx; keyed by vendor and rule id, never by text), triage found no
+    steering text, no entity has a recent true positive, no campaign is in progress (a burst, or a related alert a model judged
+    malicious or suspicious or still open for review), authentication failures are credential access whatever their ATT&CK id
+    (Wazuh rule groups, T1531), and both model tiers say `benign`: fast-tier likelihood at
     most `routing.model_close_max_likelihood` (8) and deep-tier certainty at
     least `routing.model_close_min_certainty` (85). The `decide` node reads
     the tier from the tenant's settings at every decision.

docs/invariants.md

@@ -61,12 +61,12 @@ Containment changes a customer's environment. An action executes only once it is
 
 ## INV-03: An AI verdict alone never closes a high or critical alert or a never-close class; a low or medium alert closes on the models' word only at the tenant's autonomy tier
 
-A fooled or injected model must not be able to make a dangerous alert disappear. Routing is a pure policy. High and critical alerts close only by an analyst, a suppression rule or an analyst-confirmed verdict. A low or medium alert closes when approved benign evidence an attacker cannot write corroborates a benign verdict: a cached analyst verdict, the detection rule's name, the EICAR hash, a test source or the tenant's declared context, such as a scanner as the source of the activity; never past steering text or a recent true positive, and a scanner or test source never for a never-close class. Without such evidence it closes on the models' word only when the tenant's autonomy tier is auto_full_audit and both model tiers judge it benign within the install's thresholds, never for the never-close classes (credential access, log clearing, security-tool tampering, ransomware behaviour, persistence through attacker-named objects, unmapped alerts), steering text, entities with a recent true positive or a campaign in progress; such a close is marked model_closed for audit. With the tier off, the default, a model never closes an alert alone. A cached analyst verdict closes only a non-critical repeat of the same detection and behaviour (verdict key: what ran, compared in full) that two different analysts decided benign, with no threat on it since; alerts in a never-close class, normalised by a model or without a process, command line, service, scheduled task, registry key, target file or Sysmon WMI object get no key (a URL, user agent, O365 operation or other request-set context is in the key but never enough for one). A suggested suppression rule matches the verdict key, older fingerprint-only suggestions never apply, and a confirmed threat disables the rules that match it.
+A fooled or injected model must not be able to make a dangerous alert disappear. Routing is a pure policy. High and critical alerts close only by an analyst, a suppression rule or an analyst-confirmed verdict. A low or medium alert closes when approved benign evidence an attacker cannot write corroborates a benign verdict: a cached analyst verdict, the detection rule's name, the EICAR hash, a test source or the tenant's declared context, such as a scanner as the source of the activity; never past steering text or a recent true positive, and a scanner or test source never for a never-close class. Without such evidence it closes on the models' word only when the tenant's autonomy tier is auto_full_audit and both model tiers judge it benign within the install's thresholds, never for the never-close classes (credential access, log clearing, security-tool tampering, ransomware behaviour, persistence through attacker-named objects, authentication failures by vendor rule group, unmapped alerts unless the vendor rule id is in the reviewed noise catalogue, never matched on text), steering text, entities with a recent true positive or a campaign in progress; such a close is marked model_closed for audit. With the tier off, the default, a model never closes an alert alone. A cached analyst verdict closes only a non-critical repeat of the same detection and behaviour (verdict key: what ran, compared in full) that two different analysts decided benign, with no threat on it since; alerts in a never-close class, normalised by a model or without a process, command line, service, scheduled task, registry key, target file or Sysmon WMI object get no key (a URL, user agent, O365 operation or other request-set context is in the key but never enough for one). A suggested suppression rule matches the verdict key, older fingerprint-only suggestions never apply, and a confirmed threat disables the rules that match it.
 
 * **Status:** enforced
 * **Owner epic:** AI-E6
 * **Threats:** TM-WRK-1, TM-WRK-2, TM-WRK-10
-* **Enforced in:** `src/lg_sotf/domain/routing.py`, `src/lg_sotf/domain/closure.py`, `src/lg_sotf/domain/benign_evidence.py`, `src/lg_sotf/domain/verdict_key.py`, `src/lg_sotf/core/verdict_cache.py`, `src/lg_sotf/core/benign_learning.py`
+* **Enforced in:** `src/lg_sotf/domain/routing.py`, `src/lg_sotf/domain/closure.py`, `src/lg_sotf/domain/benign_evidence.py`, `src/lg_sotf/domain/noise_families.py`, `src/lg_sotf/domain/verdict_key.py`, `src/lg_sotf/core/verdict_cache.py`, `src/lg_sotf/core/benign_learning.py`
 * **Runtime checks:** `routing_policy`
 * **Evaluation metric:** `high_severity_auto_closed_benign`
 * **Proved by:**
@@ -82,7 +82,14 @@ A fooled or injected model must not be able to make a dangerous alert disappear.
   * `tests/unit/domain/test_closure.py::test_a_scanner_or_test_source_never_closes_a_never_close_class`
   * `tests/unit/domain/test_closure.py::test_a_campaign_in_progress_is_never_model_closed`
   * `tests/unit/domain/test_closure.py::test_review_f2_sentinel_credential_access_is_a_never_close_class`
+  * `tests/unit/domain/test_noise_families.py::test_a_forged_or_borrowed_title_on_an_uncatalogued_rule_stays_unmapped`
+  * `tests/unit/domain/test_noise_families.py::test_the_catalogue_lifts_only_unmapped_never_a_mapped_never_close_class`
+  * `tests/unit/domain/test_noise_families.py::test_every_other_condition_still_blocks_a_catalogued_rule`
+  * `tests/unit/domain/test_noise_families.py::test_a_malicious_or_suspicious_model_verdict_on_a_catalogued_rule_still_escalates`
+  * `tests/unit/domain/test_noise_families.py::test_recheck_a_wazuh_alert_without_a_readable_rule_never_closes`
+  * `tests/unit/domain/test_noise_families.py::test_review_h1_authentication_failures_are_credential_access_whatever_their_mapping`
   * `tests/integration/engine/test_model_closure.py::test_the_tenant_tier_decides_whether_the_models_close`
+  * `tests/integration/engine/test_model_closure.py::test_catalogued_noise_closes_on_the_models_word_and_an_uncatalogued_rule_does_not`
   * `tests/integration/eval/test_adversarial_v2_eval.py::test_no_malicious_close_with_model_led_closure_on`
   * `tests/integration/e2e/test_llm_safety.py::test_injected_benign_instructions_never_close_an_alert`
   * `tests/unit/agents/test_title_camouflage.py::test_the_title_camouflage_items_get_no_benign_evidence_and_never_close_on_rules`

docs/security.md

@@ -251,7 +251,10 @@ close a dangerous alert or trigger containment.
    `auto_full_audit`. Then both model tiers may close a low or medium alert
    alone, marked `ai_closed:audit_required`, but never a never-close class
    (credential access, log clearing, tampering, ransomware, persistence
-   through attacker-named objects, or an alert without an ATT&CK mapping),
+   through attacker-named objects, or an alert without an ATT&CK mapping
+   unless its vendor rule id is in the reviewed noise catalogue, FIX-2;
+   authentication failures count as credential access whatever their
+   ATT&CK id),
    never with steering text, a recent true positive on its entities or a
    campaign in progress among its related alerts, and
    only when the fast tier's likelihood is at most 8 and the deep tier's

docs/threat-model.md

@@ -135,7 +135,7 @@ disclosure, **D**enial of service, **E**levation of privilege.
 | ID | STRIDE | Threat | Controls | Status |
 |---|---|---|---|---|
 | TM-WRK-1 | T | Prompt injection in alert fields changes the verdict | Untrusted-data blocks with escaped `<` and `>` (`src/lg_sotf/llm/prompting.py:365-400`); injection detection voids false-positive signals (`src/lg_sotf/agents/triage/base.py:125-131`); the injection signal blocks corroboration (`src/lg_sotf/agents/analysis/base.py:15-28`) | In place |
-| TM-WRK-2 | T | A fooled model closes a dangerous alert, or the attacker names objects so the rules corroborate a benign close (title camouflage) | High and critical alerts are never auto-closed; low and medium need approved benign evidence (a rule false-positive signal or a cached analyst verdict), never past steering text or a recent true positive, and a declared scanner (only as the source of the activity) or a test source never for a never-close class; refusals escalate (`src/lg_sotf/domain/routing.py:1-35`). Model-led closure without such evidence is off unless a tenant admin (never under an MSSP grant, audited) sets `ai_closure.tier` to `auto_full_audit`; even then it needs both model tiers benign within install thresholds, no steering text, no recent true positive on the entities, no campaign in progress (a burst, a related alert judged a threat or open for review), an ATT&CK mapping and none of the never-close classes (credential access, log clearing, tampering, ransomware, persistence through attacker-named objects), and it is marked `ai_closed:audit_required` (`src/lg_sotf/domain/closure.py`, FIX-1). Rule false-positive evidence comes only from the detection rule name with attacker-named values masked, EICAR hashes, the tenant's `triage.known_scanners` and the source id, never from titles, entity names, command lines, user agents or email subjects (`src/lg_sotf/domain/benign_evidence.py`, AI-E6.3). Rule-name words that name the detected activity (`scheduled`, `update`, `backup`, `scan`, `monitoring`) count only with a trusted qualifier: an intent word in the same rule name, a test-environment source id or, for `scan`, a known scanner (AI-E6.4) | In place; measured by `high_severity_auto_closed_benign` = 0 and the gated `title-camouflage` and `activity-camouflage` families of the adversarial corpus v2 in CI, at both autonomy tiers |
+| TM-WRK-2 | T | A fooled model closes a dangerous alert, or the attacker names objects so the rules corroborate a benign close (title camouflage) | High and critical alerts are never auto-closed; low and medium need approved benign evidence (a rule false-positive signal or a cached analyst verdict), never past steering text or a recent true positive, and a declared scanner (only as the source of the activity) or a test source never for a never-close class; refusals escalate (`src/lg_sotf/domain/routing.py:1-35`). Model-led closure without such evidence is off unless a tenant admin (never under an MSSP grant, audited) sets `ai_closure.tier` to `auto_full_audit`; even then it needs both model tiers benign within install thresholds, no steering text, no recent true positive on the entities, no campaign in progress, an ATT&CK mapping and none of the never-close classes (credential access, log clearing, tampering, ransomware, persistence through attacker-named objects), and it is marked `ai_closed:audit_required` (`src/lg_sotf/domain/closure.py`, FIX-1). Policy v2.1 (FIX-2): an unmapped alert closes only when its vendor detection rule id (Wazuh `raw.rule.id`, on an alert the Wazuh normalizer built: the ingestion service records the format in `normalizer`, never from the event, and the rule name is that normalizer's `<id>: `) is in the reviewed, versioned noise catalogue (`src/lg_sotf/domain/noise_families.py`: package installs and updates, Windows software installs, Defender scan and signature status, a single web 4xx), never on title, description or rule-name text; the catalogue lifts only `unmapped`, every other condition still applies, and each entry records the abuse it could hide (a malicious package, an MSI run as execution, web probing) with a test that a malicious or suspicious model verdict still escalates it. Authentication failures are credential access whatever ATT&CK id the vendor gave them: Wazuh rules in the groups `authentication_failed`, `authentication_failures`, `invalid_login` or `login_denied`, the listed ids 6250 and 44733, and T1531 (Wazuh maps Windows logon failures 60122-60130 there). The campaign check is FIX-1's (a burst, a related alert judged a threat or open for review); a check that lets one rule repeat was deferred after three review rounds found ways to launder a block through it. Residual: an actor on a host, often without privileges (`logger -t yum`, a per-user MSI install), can choose which catalogued rule fires; only the two model tiers then stand between it and a close, every such close is marked for audit, and its decision reason names the family and catalogue version. Wazuh's frequency rules are no backstop for a slow attack (60204 needs 8 failures from one IP in 240 s; 31151 is level 10, which maps to medium and is itself closable). The state size cap keeps `raw.rule` (id, level, description, groups, mitre) when it drops an oversized raw event, so inflating an event does not hide its groups; a Wazuh-built alert whose rule cannot be read at all, or whose groups list the cap had to cut, is `vendor_unknown`, a never-close class. Rule false-positive evidence comes only from the detection rule name with attacker-named values masked, EICAR hashes, the tenant's `triage.known_scanners` and the source id, never from titles, entity names, command lines, user agents or email subjects (`src/lg_sotf/domain/benign_evidence.py`, AI-E6.3). Rule-name words that name the detected activity (`scheduled`, `update`, `backup`, `scan`, `monitoring`) count only with a trusted qualifier: an intent word in the same rule name, a test-environment source id or, for `scan`, a known scanner (AI-E6.4) | In place; measured by `high_severity_auto_closed_benign` = 0 and the gated `title-camouflage` and `activity-camouflage` families of the adversarial corpus v2 in CI, at both autonomy tiers |
 | TM-WRK-3 | E | A fooled model triggers containment | Containment on model-only evidence always waits for approval (`src/lg_sotf/services/response/planner.py:328-329`); tenant autonomy, with unknown values read as `disabled` (`src/lg_sotf/services/response/policy.py:215-226`) | In place |
 | TM-WRK-4 | I | Alert data or secrets sent to the model provider | Secrets redacted before storage and before the model; personal data is **not** redacted | Partial: per-tenant pseudonymisation before the LLM call is planned (SEC7.5b) |
 | TM-WRK-5 | I | Model endpoint pointed at an internal or metadata address | `llm.base_url` egress check (`src/lg_sotf/llm/endpoint.py:80-100`), enforced at start-up (`src/lg_sotf/settings.py:519-524`) | In place |

src/lg_sotf/agents/analysis/base.py

@@ -77,6 +77,7 @@
 from lg_sotf.llm.registry import register_prompt
 from lg_sotf.llm.schema import clipped_list, clipped_str, lenient_literal
 from lg_sotf.llm.trace import observe
+from lg_sotf.models.alert import model_view
 from lg_sotf.security.redaction import redact_text
 from lg_sotf.tools.enrichment import (
     EnrichmentResult,
@@ -474,7 +475,7 @@ def _prompt_block(
         "rule_threat_score": rule_score,
     }
     block: Dict[str, Any] = {
-        "alert": dict(alert),
+        "alert": model_view(alert),
         "background": background,
         "history": history,
         "candidates": {cid: c.prompt_view() for cid, c in candidates.items()},

src/lg_sotf/agents/triage/base.py

@@ -91,6 +91,7 @@
 from lg_sotf.domain.routing import RoutingThresholds
 from lg_sotf.llm.registry import register_prompt
 from lg_sotf.llm.schema import clipped_list, clipped_str, lenient_literal
+from lg_sotf.models.alert import model_view
 
 if TYPE_CHECKING:
     from lg_sotf.core.state import TriageDigest
@@ -550,7 +551,7 @@ def __init__(self, config: Optional[Mapping[str, Any]] = None, **kwargs: Any) ->
     @staticmethod
     def prompt_data(alert: Mapping[str, Any], rule_fp: List[str], rule_tp: List[str], rule_score: int) -> Dict[str, Any]:
         return {
-            "alert": dict(alert),
+            "alert": model_view(alert),
             "rule_assessment": {"confidence": rule_score, "fp_signals": rule_fp, "tp_signals": rule_tp},
         }