Pages

mdadm tips on Linux software RAID

mdadm is a tool for managing, creating and reporting on Linux software RAID arrays.

I will describe some tips which I found useful at the moment.

Improve RAID1 re-sync time with write-intent bitmap

The RAID driver writes out periodically bitmap information recording which areas of the RAID component have been modified since the RAID array was last in sync.

If, for example one of two members of a RAID1 array fails and is removed from the array, md (the multiple disk software RAID drive) will record bits to the bitmap relating to the changes the active member is undertaking since the two members were last in sync. If the same failed/removed drive is re-added to the RAID1 array, md will notice and will recover only the portions indicated by the bitmap. In this way a lengthy re-sync is avoided (a full re-sync is normally needed if the drives are not in sync when the array starts up).
Two types of bitmaps can be specified:
- internal which is stored with the metadata on the array
- external which is stored as a file on a ext2 or ext3 filesystem

To enable the write-intent bitmap a few requirements are imposed:
- the bitmap file should not pre-exist when creating it
- if an external bitmap is used, the absolute path must be specified
- if an external bitmap is used, it must be on another filesystem than the RAID array describes
- if an external bitmap is used, it works only with ext2, ext3 filesystems
- for an already existing array, before adding the bitmap, the array must be in sync and have a persistent superblock. This can be verified by:
mdadm --detail /dev/mdX
And make sure of the output:

State : active
Persistence : Superblock is persistent

Creating a RAID1 (consisting of /dev/sda1, /dev/sdb1) array with external bitmap is done by:
mdadm -C /dev/md0 -a yes -l1 -n2 /dev/sda1 /dev/sdb1 -b /tmp/bitmap.file
Creating a RAID1 (consisting of /dev/sda1, /dev/sdb1) array with internal bitmap is done by:
mdadm -C /dev/md0 -a yes -l1 -n2 /dev/sda1 /dev/sdb1 -b internal

To add write-intent bitmapping to an existing array:
External:
mdadm /dev/md0 -G -b /tmp/bitmap.file
Internal:
mdadm /dev/md0 -G -b internal
To turn off the write-intent bitmapping:
mdadm /dev/md0 -G -b none

RAID data consistency check

RAID detects bad blocks passively. If an error occurs when attempting to read a block (soft error) an try is made to rewrite the data using another valid copy in the array. If the rewrite fails (hard error), the faulty device (holding the bad block) is evicted from the active array.
This can be critical for example for a RAID5 array (3 drives out of which 1 failed) trying to rebuild the failed drive, resulting in data loss.

In kernel versions 2.6.16 or greater the following command will initiate data consistency and bad blocks check attempting to fix and bad blocks:
echo check >> /sys/block/mdX/md/sync_action
The progress can be seen in /proc/mdstat.

Renaming RAID arrays

In order to rename an array from /dev/md0 (made of /dev/sda1 and /dev/sdb1) to /dev/md6 you need to:
Fist stop the RAID device:
mdadm --stop /dev/md0
Reassemble the RAID device as /dev/md6
mdadm --assemble /dev/md6 --super-minor=0 --update=super-minor /dev/sda1 /dev/sdb1
This command will update the array's superblocks to the new number (6 in the /dev/md6)

Renaming of arrays may be useful when moving an already existing array to a system which has already created arrays with the same name.

No comments: