A degraded RAID array is a race against the clock: one disk has already failed, and until the array rebuilds, a second failure means total data loss. The array keeps serving data — which is exactly why a degraded array can go unnoticed for weeks if nobody's watching.
Confirm which disk failed
cat /proc/mdstat # md0 : active raid1 sda1[0] sdb1[1](F) <- (F) = failed # [2/1] [U_] <- U=up, _=missing mdadm --detail /dev/md0
Add a replacement and rebuild
mdadm --manage /dev/md0 --remove /dev/sdb1 # drop the failed member mdadm --manage /dev/md0 --add /dev/sdc1 # add the replacement watch cat /proc/mdstat # recovery = X% [===>.....]
The rebuild runs in the background; the array is still vulnerable until it reaches [UU]. To avoid the manual scramble next time, configure a hot spare in /etc/mdadm/mdadm.conf so mdadm rebuilds automatically the moment a disk drops.
Practise it
Degraded arrays are part of the simulator's storage incidents. Drill the replace-and-rebuild loop in Linux troubleshooting practice.