Skip to content

fix(deployed): cite the experiments that back the assignment and file-rule evidence - #43

Merged
fadwen merged 1 commit into
mainfrom
fix/inex-evidence
Oct 15, 2026
Merged

fadwen merged 1 commit into
mainfrom
fix/inex-evidence

Conversation

@fadwen

@fadwen fadwen commented Oct 15, 2026

Copy link
Copy Markdown
Owner

Summary

Round 9 of the validation kit (28 September) concluded that a remediation included and excluded for the same device group never runs: Graph stored only the include, VM 125 resolved the policy, and no run was seen in the 55 minutes watched. Validation/Findings.md recorded the exclusion as holding on the device, and Test-IntuneDeployedScript cites that experiment, ASSIGN-INEX, in the evidence for its "no include assignment" warning.

The conclusion does not hold. The device's probe file for the policy had 180 records by 15 October (120 detections, 60 remediations, all as SYSTEM), the first on the evening of 28 September and the last that morning. In the tenant, read-only: the policy's lastModifiedDateTime equals its createdDateTime, the audit log since 27 September holds only its create and assign events, its one stored assignment is an hourly include of ISL-Validation-Devices, and its deviceRunStates carry lastStateUpdateDateTime 13 October. The service drops the exclusion and the policy runs every hour on every device in the group.

The command's verdict was already right. Its warning fires only when a policy has no include, and this policy has one, so it says nothing, which is correct for a policy that runs. Only the evidence string was wrong: it listed ASSIGN-INEX among policies "absent from every device's resolved policies", and Findings itself recorded that VM 125 resolved it.

A second evidence id in the same command named an experiment that does not exist: W32-FILE-NOTEXIST-PRESENT for W32-FILE-NOTEXIST-FALSE.

Changes

  • Public/Test-IntuneDeployedScript.ps1: the IslAssignmentIssue evidence names ASSIGN-NONE and ASSIGN-EXCLONLY; the file doesNotExist evidence names W32-FILE-NOTEXIST-FALSE.
  • Validation/Findings.md: the round 9 row records the corrected result with the device and tenant evidence, marked as a documented behaviour that does not hold, and the "for the tool" paragraph says why the command stays silent on an include-plus-exclude policy.
  • CHANGELOG.md: Fixed entry under Unreleased.

Verification

  • Test-IntuneDeployedScript, RuleReference and Module.Contract suites: 41 passed on PowerShell 7.6.6.
  • Build/Build-RuleReference.ps1 regenerates docs/Rules.md without a content change (the policy findings are not in the rule table).
  • PSScriptAnalyzer (Error, Warning) clean on the changed file; no line over 115 characters.
  • Tenant evidence gathered with GET /beta/deviceManagement/deviceHealthScripts?$filter=displayName eq 'ISL-ASSIGN-INEX'&$expand=assignments, its deviceRunStates, and auditEvents since 27 September; nothing in the tenant was changed.

Notes

  • The lab host was not reachable from the development machine when this PR was prepared; the probe-file counts are from a read of VM 125 on 15 October through the kit's Invoke-GuestPowerShell, not re-run here.
  • The validation kit fixes found in the same pass are in a separate PR stacked on this one.

…-rule evidence

Round 9 concluded that a remediation included and excluded for the same
group never runs, and Test-IntuneDeployedScript cited that experiment,
ASSIGN-INEX, as evidence for its no-include warning. The device has run
the policy every hour since 28 September: Graph keeps only the include,
the policy is unchanged since creation, and its run state was updated on
13 October. The warning never fired on it, so the verdict stands; the
evidence names ASSIGN-NONE and ASSIGN-EXCLONLY only and Findings records
the corrected result. The file doesNotExist evidence named an experiment
that does not exist; it is W32-FILE-NOTEXIST-FALSE.

@fadwen fadwen left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notes on the lines whose reason the diff does not show.

Evidence = 'A remediation with no assignment and one with only an exclusion were absent ' +
'from every device''s resolved policies after an agent restart, while the same scripts ' +
'with an include assignment ran (ASSIGN-NONE, ASSIGN-EXCLONLY, ASSIGN-INEX)'
'with an include assignment ran (ASSIGN-NONE, ASSIGN-EXCLONLY)'

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only the evidence changes. The condition above fires on a policy with no include, and the include-plus-exclude policy has one (the service keeps only the include), so the command never warned about it; the device evidence now says that silence was the right verdict. No test asserted the evidence text, which is why this went unnoticed since round 9.

Comment thread Validation/Findings.md
| No assignment | A policy must be assigned to apply (portal) | **Never resolved**: absent from both devices' `ProcessResolvedPolicies` lists and logs; no run, no probe record (ASSIGN-NONE) | ✅ |
| Only an exclusion | Not documented | The same: never resolved on either device (ASSIGN-EXCLONLY) | ⚠️ |
| Include and exclude of the same group | Exclusion wins over inclusion (Intune assignment docs) | **Graph shows only the include**: `POST /assign` with both accepted, `GET /assignments` returns one `groupAssignmentTarget`. VM 125 resolved the policy and inspected its schedule (`inspect hourly schedule for policy <INEX>`, queued with the others) but **the runner never executed it**: the 25 other resolved remediations ran in the following cycle, this one did not in 55 minutes, no probe record. The exclusion holds on the device while the API hides it, so there is nothing to detect after the fact (ASSIGN-INEX) | ⚠️ |
| Include and exclude of the same group | Exclusion wins over inclusion (Intune assignment docs) | **The exclusion is dropped and the policy runs.** `POST /assign` with both accepted, `GET /assignments` returns one `groupAssignmentTarget`, hourly. VM 125 resolved the policy at the restart and inspected its schedule, and the round's ten-minute read and a 55-minute watch saw no run, which this entry first recorded as the exclusion holding on the device. It did not hold: the device's probe file for the policy had 180 records by 2026-10-15 (120 detections, 60 remediations, all as SYSTEM), the first on the evening of 2026-09-28 and the last that morning, and Graph's `deviceRunStates` for it carry `lastStateUpdateDateTime` 2026-10-13. The policy was not changed after the round: `lastModifiedDateTime` equals `createdDateTime` and the audit log holds only its create and assign events. The service keeps the include, the device runs the policy every hour, and nothing of the exclusion survives for a tool to find (ASSIGN-INEX, corrected 2026-10-15) | ❌ |

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Marked with the cross rather than the warning sign because the documented behaviour, exclusion wins, is what does not hold: the exclusion is not stored at all. The row keeps the round 9 observation and says what it was misread as, since the tenant audit write-up and the earlier Findings text both relied on it.

@fadwen
fadwen merged commit ea5a2b0 into main Oct 15, 2026
4 checks passed
@fadwen
fadwen deleted the fix/inex-evidence branch October 15, 2026 17:53
@fadwen fadwen mentioned this pull request Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant