Skip to content

fix: one unhealthy drive no longer fails its controller's inventory - #103

Open
g-carre wants to merge 4 commits into
test/ARTESCA-17960-megaraid-disk-failure-scenariofrom
fix/ARTESCA-17960-megaraid-offline-volume
Open

g-carre wants to merge 4 commits into
test/ARTESCA-17960-megaraid-disk-failure-scenariofrom
fix/ARTESCA-17960-megaraid-offline-volume

Conversation

@g-carre

@g-carre g-carre commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Commits, one per adapter fix: megaraid, ssacli, storcli2 drives, storcli2 volumes.

Fixes ARTESCA-17960 (RD-2154). Stacked on #102: it targets #102's branch, so the diff only shows the fix. Retarget it to main once #102 is merged. It supersedes #91.

Problem

One unhealthy drive could fail the inventory of its whole controller, so every healthy drive on that controller lost its device paths. On the RD-2154 node this blanked all disks in the UI.

Fixes

  • megaraid (the RD-2154 case). When the drive of a single-drive RAID0 volume fails, the volume goes offline (OfLn, Exposed to OS: No) and is no longer a block device. getPaths still tried to resolve its paths, failed, and failed LogicalVolumes for the whole controller.
    • A volume reported Exposed to OS: No now gets empty paths without error. Only this explicit value skips resolution, since older storcli versions may not report the field.
    • OfLn is now reported Failed instead of Unknown.
  • ssacli (from fix: a single failed drive no longer blanks a controller's device paths #91). A failed or pulled drive whose /dev node is gone made the lsblk lookup fail, which aborted the whole inventory. A lookup failure now marks the device's usage as unknown instead. A free drive whose usage is unknown is reported Unknown, never UnassignedGood, so a drive that may carry data is never offered for a new volume.
  • storcli2 drives (from fix: a single failed drive no longer blanks a controller's device paths #91). A Used JBOD drive whose by-id link cannot be resolved is kept with empty paths instead of failing the whole inventory.
  • storcli2 volumes. The storcli2 scenario of test: describe hardware scenarios in YAML, play them on the adapters #102 found that the offline volume of a failed drive was still reported with a permanent path built from its NAA Id, although it has no block device. A volume reported Exposed to OS: No now has no paths: neither the permanent path built from its NAA Id, nor a device path from a possibly stale OS drive name.
  • Volume creation. It still waits for the new volume: a volume listed before it is exposed to the OS has no paths yet, so it is treated as not settled, and creation keeps rescanning and retrying.
  • DESIGN.md. Documents the contract that getter read paths degrade gracefully. The megaraid create/settle path is deliberately stricter.

#91's change for multi-drive megaraid volumes is not carried over, since ARTESCA does not use them.

Tests

  • Scenarios. The failed phases of megaraid-disk-failure and storcli2-disk-failure now report their expected tables, so both knownBug blocks are removed. Until they were removed, the tests failed with "the bug looks fixed". All 8 phases pass, for example:
    --- PASS: TestScenarios/megaraid-disk-failure/healthy
    --- PASS: TestScenarios/megaraid-disk-failure/failed
    --- PASS: TestScenarios/megaraid-disk-failure/replaced
    --- PASS: TestScenarios/megaraid-disk-failure/restored
    
  • megaraid unit tests, table-driven:
    • getPaths with Exposed to OS set to No (empty paths, no error), Yes, or empty (paths still resolved);
    • the volume status mapping, including OfLn → Failed.
  • Volume creation: a new volume first listed as not exposed must be awaited, not returned without paths.
  • ssacli: a failing lsblk lookup does not abort the inventory, and a free drive whose device could not be inspected is reported Unknown.
  • storcli2, from fix: a single failed drive no longer blanks a controller's device paths #91: an unresolvable JBOD drive, and a degraded storcli2 RAID volume. New: an offline volume not exposed to the OS gets no paths, and permanentPath covers Exposed to OS set to No, Yes and missing.

Delivery

disk-management-agent picks this up once raidmgmt is released and the agent bumps it. scality/disk-management-agent#74 marks its failed phase as a known bug, so it will fail and ask for the marker to be removed at that point.

@g-carre
g-carre requested a review from a team as a code owner October 9, 2026 18:13
Comment thread pkg/implementation/raidcontroller/megaraid/logicalvolume.go Outdated
Comment thread pkg/implementation/raidcontroller/megaraid/logicalvolume_internal_test.go Outdated
@g-carre g-carre changed the title fix(megaraid): do not resolve paths of a volume not exposed to the OS fix: one unhealthy drive no longer fails its controller's inventory Oct 9, 2026
Comment thread pkg/implementation/raidcontroller/megaraid/logicalvolume.go
Comment thread pkg/implementation/physicaldrivegetter/ssacli.go Outdated
Comment thread pkg/implementation/physicaldrivegetter/ssacli.go
@g-carre
g-carre force-pushed the fix/ARTESCA-17960-megaraid-offline-volume branch 2 times, most recently from 822e10d to 8f95fdf Compare October 9, 2026 19:08
@g-carre
g-carre force-pushed the test/ARTESCA-17960-megaraid-disk-failure-scenario branch from 14ede3a to 1f7e333 Compare October 9, 2026 19:15
@g-carre
g-carre force-pushed the fix/ARTESCA-17960-megaraid-offline-volume branch from 8f95fdf to b95587e Compare October 9, 2026 19:15
@g-carre
g-carre force-pushed the test/ARTESCA-17960-megaraid-disk-failure-scenario branch from 1f7e333 to 2e737a1 Compare October 9, 2026 19:21
@g-carre
g-carre force-pushed the fix/ARTESCA-17960-megaraid-offline-volume branch 2 times, most recently from a20e355 to 009eba2 Compare October 9, 2026 21:36
@g-carre
g-carre force-pushed the test/ARTESCA-17960-megaraid-disk-failure-scenario branch from e7d17c7 to ce633e6 Compare October 9, 2026 21:41
@g-carre
g-carre force-pushed the fix/ARTESCA-17960-megaraid-offline-volume branch from 009eba2 to 578d3d4 Compare October 9, 2026 21:41
Comment thread pkg/implementation/logicalvolumegetter/storcli2.go Outdated
When the drive of a single-drive RAID0 volume fails, the volume goes offline
and is no longer a block device of the host: it has no by-id link, and the
paths of its failed drive cannot be computed. getPaths returned an error,
which failed LogicalVolumes for the whole controller, so the healthy drives
of that controller lost their paths too (RD-2154).

- A volume whose "Exposed to OS" is "No" gets empty paths without error.
  Only this explicit value skips the resolution, as older storcli versions
  may not report the field.
- An offline volume (OfLn) is reported Failed instead of Unknown.
- Volume creation still waits for the new volume: a volume listed before it
  is exposed to the OS has no paths yet, so it is not settled and creation
  keeps rescanning and retrying.

DESIGN.md states that getter read paths degrade gracefully while the create
path stays strict. The failed phase of the megaraid-disk-failure scenario
now reports its expected table: its knownBug block is removed.

Fixes: ARTESCA-17960
A failed or pulled drive whose device node has disappeared made the lsblk
lookup of its Disk Name fail, which aborted the physical drive inventory of
the whole controller. The lookup only refines the status, so its failure
no longer aborts the inventory. Instead:

- the usage of the device is unknown, and a free drive whose usage is
  unknown is reported Unknown, never UnassignedGood, so a drive that may
  carry data is never offered for a new volume;
- the device path is dropped, as the kernel can reuse a vanished /dev/sdX
  name for another disk.

Array members and failed drives keep the status ssacli reports.

Refs: ARTESCA-17960
… resolved

A JBOD drive whose by-id link cannot be resolved, for instance because udev
has not settled yet, failed the physical drive inventory of the whole
controller. Keep it with empty paths instead. Also cover a degraded RAID
volume, whose paths come from the controller output only.

Refs: ARTESCA-17960
@g-carre
g-carre force-pushed the fix/ARTESCA-17960-megaraid-offline-volume branch from 578d3d4 to 7619662 Compare October 9, 2026 21:51
Comment thread pkg/implementation/logicalvolumegetter/storcli2.go
The storcli2 volume getter read the device path of a volume from its OS
drive name and built its permanent path from its NAA Id. The offline RAID0
volume of a failed drive keeps its NAA Id, and may keep a stale OS drive
name, but it is not exposed to the OS: it has no block device and no by-id
link, so both paths named nothing, or another disk. A volume whose "Exposed
to OS" is "No" now has no paths; only this explicit value drops them, as
older storcli2 versions may not report the field.

The failed phase of the storcli2-disk-failure scenario now reports its
expected table: its knownBug block is removed.

Fixes: ARTESCA-17960
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant