Skip to content

fix(entra): skip PIM alert checks until PIM has scanned the alert - #2805

Open
julien-ly wants to merge 1 commit into
maester365:mainfrom
julien-ly:fix/pim-alerts-never-scanned
Open

julien-ly wants to merge 1 commit into
maester365:mainfrom
julien-ly:fix/pim-alerts-never-scanned

Conversation

@julien-ly

@julien-ly julien-ly commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

📑 Description

Test-MtPimAlertsExists treats an inactive alert as a pass. PIM also reports an alert it has never scanned as inactive, with lastScannedDateTime set to 0001-01-01, so MT.1029–MT.1032 passed on a tenant where PIM had not scanned anything yet (details in #2803).

  • An inactive alert that PIM has never scanned is now skipped, with a reason that points to the refresh action. The four tests already return $null (not evaluated) when the function does, so they are unchanged. An active alert is evaluated as before, and so is an alert without lastScannedDateTime.
  • The pass message said "All privileged role assignments are managed by PIM" for all four tests, but only MT.1031 is about PIM management. It now names the alert and its last scan: "Well done. No role assignment in scope is affected by the PIM alert {alert name} (last scan: {date} UTC)."
  • New-PimAlert takes a LastScannedDateTime parameter, set to a scanned date by default. New unit tests cover a never-scanned alert given as a string and as a DateTime, an active alert whatever its scan date, and an alert without a scan date.
  • The pages of MT.1029–MT.1032 say when the test is skipped.

Dismissed alerts are not changed by this PR: the skip only applies to an alert that has never been scanned.

Closes #2803

✅ Checks

  • My pull request adheres to the code style of this project.
  • My code requires changes to the documentation.
  • I have updated the documentation as required.
  • The build and unit tests pass after running /powershell/tests/pester.ps1 locally.

ℹ️ Additional Information

On a tenant where PIM has not scanned the alerts yet, MT.1029–MT.1032 now report Skipped instead of Passed. The pass message changes for all four tests.

Local run of ./powershell/tests/pester.ps1 on Windows: 8,759 of 8,760 tests pass. The only failure is the missing byte order mark in powershell/tests/functions/Invoke-MtPackGitHubRequest.Tests.ps1, already on main and fixed by #2795.

Summary by CodeRabbit

  • Bug Fixes

    • PIM alert checks now skip inactive alerts that have never been scanned, with a message explaining that the alert needs to be refreshed.
    • Results with no affected assignments now clarify that no in-scope assignment is affected and include the last scan time when available.
    • Active alerts continue to be evaluated regardless of their scan date.
  • Documentation

    • Updated test descriptions to explain when checks are skipped because alerts have not been scanned.

@julien-ly
julien-ly requested review from a team as code owners November 13, 2026 19:10
@coderabbitai

coderabbitai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The PIM alert check now skips inactive alerts whose scan timestamp is unset. No-incident results state that no in-scope assignment is affected and include the scan time when available. Tests and test descriptions cover these behaviors.

Changes: PIM alert checks

Layer / File(s) Summary
Unscanned alert handling
powershell/internal/checks/maester/entra/Test-MtPimAlertsExists.ps1, powershell/tests/functions/Test-MtPimAlertsExists.Tests.ps1, tests/maester/entra/Test.MT.1029.md–Test.MT.1032.md
The check normalizes supported scan timestamp values to UTC. It skips inactive alerts with a minimum-date timestamp and records a custom skip reason. Tests cover skipped inactive alerts and confirm that active alerts are still evaluated. The four test descriptions state that checks are skipped until PIM scans the alerts.
No-incident result message
powershell/internal/checks/maester/entra/Test-MtPimAlertsExists.ps1, powershell/tests/functions/Test-MtPimAlertsExists.Tests.ps1
The no-affected-assignment result now identifies that no in-scope assignment is affected. It includes the parsed scan time in UTC when available. The test checks the formatted time.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: merill

Merge Risk: Minimal · up to 6ef1d

This change makes the PIM alert checks report Skipped instead of Passed when PIM has never scanned an alert, and it updates the pass message and docs. No merge-blocking risk is evident from the supplied changes.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the fix and its scope: PIM alert checks in the entra area are skipped until PIM has scanned the alert. It matches the main change in Test-MtPimAlertsExists, which skips inactive alerts…
Linked Issues check Passed [#2803] Objective 1: an inactive PIM alert that has never been scanned must not pass. The reviewed head adds a check in Test-MtPimAlertsExists.ps1. When an alert is inactive and lastScannedDateTime pa…
Out of Scope Changes check Passed All changes connect to #2803. The function change adds the never-scanned skip and the scan-aware pass message. The Tests.ps1 changes add the New-PimAlert LastScannedDateTime parameter and tests for th…
Description check Passed The description covers the core change: the never-scanned inactive alert skip (lastScannedDateTime 0001-01-01), the new pass message, the New-PimAlert parameter and new unit tests, and the documentati…
Full details: Title check

Explanation

The title names the fix and its scope: PIM alert checks in the entra area are skipped until PIM has scanned the alert. It matches the main change in Test-MtPimAlertsExists, which skips inactive alerts with a never-scanned timestamp. The title is concise, follows the conventional commit style, and is specific enough for a teammate scanning history.

Full details: Linked Issues check

Explanation

[#2803] Objective 1: an inactive PIM alert that has never been scanned must not pass. The reviewed head adds a check in Test-MtPimAlertsExists.ps1. When an alert is inactive and lastScannedDateTime parses to a UTC year of 1 or earlier (0001-01-01), the function calls Add-MtTestResultDetail with SkippedBecause Custom and returns $null. The four tests already treat $null as not evaluated. Active alerts keep the prior evaluation path (AffectedRoleAssignments from alertIncidents). Objective 2: the pass message must identify the alert and its last scan. The new message reads 'Well done. No role assignment in scope is affected by the PIM alert {alertName} (last scan: {yyyy-MM-dd HH:mm} UTC).' and is built from $Alert.alertName and $lastScanUtc. Objective 3: the skip reason points to the refresh action (POST /beta/identityGovernance/roleManagementAlerts/alerts/refresh). Test coverage for the never-scanned, active, and missing-date cases is reported in Test-MtPimAlertsExists.Tests.ps1 (not executed in this review). Residual point: an inactive alert with a null lastScannedDateTime is still evaluated. The issue offers 'unset' as a possible direction, not a firm requirement, and the reported evidence is the 0001-01-01 value, which the PR handles. The PR states this choice explicitly.

Full details: Out of Scope Changes check

Explanation

All changes connect to #2803. The function change adds the never-scanned skip and the scan-aware pass message. The Tests.ps1 changes add the New-PimAlert LastScannedDateTime parameter and tests for the new behavior. The Test.MT.1029–1032 documentation pages state the skip condition. Dismissed-alert handling is unchanged, as the PR states. No unrelated file changes were identified in the reviewed summary.

Full details: Description check

Explanation

The description covers the core change: the never-scanned inactive alert skip (lastScannedDateTime 0001-01-01), the new pass message, the New-PimAlert parameter and new unit tests, and the documentation updates. It links the issue (#2803) and explains that dismissed alerts are unchanged. The template's checklist is ticked, and the test run is reported as 8,759 of 8,760 passing, with one unrelated BOM failure. The template's Code freeze notice and contributing links are boilerplate and do not need content. The description is mostly complete. Minor gap: the checklist says the build and unit tests pass, but the run had one failure, so that box is slightly inconsistent with the reported result. The objectives say the description reports that pester.ps1 was not run locally, which conflicts with the description's own run report; this is a discrepancy to clarify.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 2d659e8 and 6ef1d84.

Files selected for processing: 6

  • powershell/internal/checks/maester/entra/Test-MtPimAlertsExists.ps1
  • powershell/tests/functions/Test-MtPimAlertsExists.Tests.ps1
  • tests/maester/entra/Test.MT.1029.md
  • tests/maester/entra/Test.MT.1030.md
  • tests/maester/entra/Test.MT.1031.md
  • tests/maester/entra/Test.MT.1032.md

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@earbona23 earbona23 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read this against main (2d659e8). The route you took is the one that works, and it closes the concern I raised in #2803 rather than working around it.

One file is enough, and that is because of return $null. In #2803 I noted that the four tests encode isActive -eq $false as a pass themselves, so a change to the shared function could not move a verdict — unless the function returns $null. That is the exception, and it is the one you used. All four tests guard it first, before the line that would re-encode the pass:

# Test.MT.1029.ps1:31-33, identical in 1030, 1031 and 1032
if ($null -eq $check) { return $null }
# Passed when the PIM alert is not active or has no affected items after filtering.
return ($check.isActive -eq $false -or $check.numberOfAffectedItems -eq '0')

So the four .ps1 files genuinely do not need to change. Nobody should re-raise that.

The fixture risk is closed. New-PimAlert now takes LastScannedDateTime with a scanned default, which is what keeps the existing cases from all turning into "never scanned" the moment the function starts reading that field.

The new pass message breaks nothing else. Your checklist leaves "unit tests pass locally" unticked, so here is that part measured on main. The old string All privileged role assignments are managed by PIM. Well done! occurs in 6 files:

file occurrences what it is
powershell/internal/checks/maester/entra/Test-MtPimAlertsExists.ps1 1 the source line this PR changes
powershell/tests/functions/Test-MtPimAlertsExists.Tests.ps1 1 the only assertion — already updated here
report/src/lib/testResults.ts 2 committed sample report data
report/src/lib/testResultsLegacy.ts 2 committed sample report data
build/aitools/test-metadata/test-results.json 1 committed sample report data
website/public/demo/report.html 1 demo report

The two golden fixtures that a grep for managed by PIM turns up — powershell/tests/fixtures/golden/shipped-2x/maester-config.json and powershell/tests/fixtures/golden/tags-and-blocks.json — contain 0 occurrences of the pass message; their match is MT.1031's own title ("Privileged role on Control Plane are managed by PIM only"). And Compare-MtTestResult.Tests.ps1, the only suite that reads testResultsLegacy.ts, diffs two committed files that this PR does not touch. So no assertion outside the one you edited depends on that string.

Two observations, neither a request to widen this PR.

A stale scan still passes. The guard is $lastScanUtc.Year -le 1, so a tenant whose last PIM scan was eight months ago keeps reading as a pass. What changes is that the date is now printed in the result, which turns a silent state into a visible one — that is what would make a follow-up measurable, and it is a better place to stop than guessing at a staleness threshold here.

The skip returns before Add-MtTestResultDetail -Description $testDescription, so a skipped result carries the custom reason without the Security Impact / Mitigation steps block. That is the same shape as the existing catch two dozen lines down, so it is consistent with the function as it stands.

On the red check: of the 15 checks here the only failure is build-validation (ubuntu-latest), which is the one leg that runs with PESTER_COVERAGE and therefore the only one running the OS-independent tests. It is failing the same way on all eight of my open PRs; #2795 and #2797 are what clear it. Not yours.

@julien-ly

Copy link
Copy Markdown
Contributor Author

Thanks for the review. Ran the four tests with this PR on a lab tenant (head 6ef1d84, main 2d659e8, read-only), where PIM last scanned its alerts at 01:20 UTC today:

Test main This PR
MT.1029 Failed Failed (same text, two stale Global Administrator accounts)
MT.1030 Passed Passed
MT.1031 Passed Passed
MT.1032 Passed Passed

The three passes now read, for example, "Well done. No role assignment in scope is affected by the PIM alert Roles are being assigned outside of Privileged Identity Management (last scan: 2026-11-13 01:20 UTC)." instead of the shared "All privileged role assignments are managed by PIM. Well done!". The never-scanned path can no longer be reproduced on this tenant, since its alerts have been scanned; the new unit tests cover it with the date as a string and as a DateTime.

Agreed on both observations. A staleness threshold would be a separate decision, and the printed date now gives a basis for it.

@earbona23

Copy link
Copy Markdown
Contributor

Thank you for running it. The blast radius in your table is exactly the expected one: MT.1029 unchanged in verdict and text, and the change confined to the three pass messages.

One thing worth stating for whoever reviews after me: your tenant has scanned its alerts, so the branch this PR adds — inactive and never scanned — cannot be exercised there, and you say so plainly. That leaves the four new unit cases as the only evidence for the path being introduced, which is where review attention should go rather than at the tenant run. They cover the string form, the DateTime form, an active alert with a minimum-value date, and an alert with no date at all, which is the full set of shapes the parse in the function can meet. I read them against the function and found no shape missing.

@earbona23 earbona23 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. I reviewed this on 13 November and closed with "two observations, neither a request to widen this PR"; a comment was all I could leave then. Re-verified against today's main first, since it has moved since.

  • Merges cleanly with main — merge-tree reports no conflict.
  • 12/12 on Test-MtPimAlertsExists.Tests.ps1, Pester 5.7.1 (CI's pinned version; I print it now after a Pester 6 default caught me out on another PR today).
  • The control discriminates. Reverting only Test-MtPimAlertsExists.ps1 to main and keeping your tests: 8 passed, 4 failed:
[-] returns zero affected items for an inactive alert
[-] skips an inactive alert that PIM has never scanned
[-] skips an inactive alert whose scan date is the minimum DateTime value
[-] evaluates an inactive alert without a scan date as before

The fourth is the one I want to point at: it pins that an inactive alert without a scan date still behaves as it did before. A new guard like this usually comes with cases for the new path only, and then nothing notices when the old path moves too. Asserting the unchanged half is what makes this safe to merge rather than merely plausible.

I had to run that control twice — my first reading reported four fewer tests executed rather than four failing, which was an artefact of how I was collecting the counts, not of your suite. The numbers above are from a clean run reproduced twice, counting per test.

My two earlier observations stand as observations:

  • A stale scan still passes, because the guard is $lastScanUtc.Year -le 1. What this PR changes is that the date is now in the result, so the state is visible instead of silent — which is the right place to stop rather than inventing a staleness threshold here.
  • The skip returns before Add-MtTestResultDetail -Description $testDescription, so a skipped row carries the reason without the Security Impact block. Same shape as the existing catch further down, so it is consistent with the function as it stands.

On the red check: build-validation (ubuntu-latest) is failing on main itself and every open PR inherits it — #2795 and #2797 are what clear it. Not caused by this change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PIM alert checks (MT.1029–MT.1032) pass when the alert has never been scanned

2 participants