Array configurations

An array is an ordered configuration, or group, of physical devices (drives) that is used to define logical volumes or devices. An array is a type of MDisk that is made up of disk drives; these drives are members of the array. A Redundant Array of Independent Disks (RAID) is a method of configuring member drives to create high availability and high performance systems.

The system supports nondistributed and distributed array configurations. In nondistributed arrays, entire drives are defined as "hot-spares". Hot-spare drives are idle and do not process I/O for the system until a drive failure occurs. When a member drive fails, the system automatically replaces the failed drive with a hot-spare drive. The system then resynchronizes the array to restore its redundancy.However, all member drives within a distributed array have a rebuild area that is reserved for drive failures. All the drives in an array can process I/O data and provide faster rebuild times when a drive fails.The RAID level provides different degrees of redundancy and performance; it also determines the number of members in the array.

If encryption is enabled on your system, you can create encrypted arrays and distributed arrays. For more information, see Configuring encryption.

Array objects shows the relationships of the array components on the system.

Figure 1. Array objects
This image shows an overview of RAID objects.

Arrays

A nondistributed array can contain 2 - 16 drives; several arrays create the capacity for a pool. For redundancy, spare drives ("hot-spares") are allocated to assume read or write operations if any of the other drives fail. The rest of the time, the spare drives are idle and do not process requests for the system. When a member drive fails in the array, the data can only be recovered onto the spare as fast as that drive can write the data. Because of this bottleneck, rebuilding the data can take many hours as the system tries to balance host and rebuild workload. As a result, the load on the remaining member drives can significantly increase. The latency of I/O to the rebuilding array is affected for this entire time. Because volume data is striped across MDisks, all volumes are affected during the time it takes to rebuild the drive.

Distributed arrays

Distributed array configurations create large-scale internal MDisks. These arrays, which can contain 4 - 128 drives, also contain rebuild areas that are used to maintain redundancy after a drive fails. If not enough drives are available on the system (for example, in configurations with fewer than four flash drives), you cannot configure a distributed array. Distributed RAID arrays solve rebuild bottlenecks in nondistributed array configurations because rebuild areas are distributed across all the drives in the array. The rebuild write workload is spread across all the drives rather just the single spare drive which results in faster rebuilds on an array. Distributed arrays remove the need for separate drives that are idle until a failure occurs. Instead of allocating one or more drives as spares, the spare capacity is distributed over specific rebuild areas across all the member drives. Data can be copied faster to the rebuild area and redundancy is restored much more rapidly. Additionally, as the rebuild progresses, the performance of the pool is more uniform because all of the available drives are used for every volume extent. After the failed drive is replaced, data is copied back to the drive from the distributed spare capacity. Unlike "hot spares" drives, read/write requests are processed on other parts of the drive that are not being used as rebuild areas. The number of rebuild areas is based on the width of the array. The size of the rebuild area determines how many times the distributed array can recover failed drives without risking becoming degraded. For example, a distributed array that uses RAID 6 drives can handle two concurrent failures. After the failed drives have been rebuilt, the array can tolerate another two drive failures. If all of the rebuild areas are used to recover data, the array becomes degraded on the next drive failure. Verify that your model supports distributed arrays before completing array configuration.

Array configuration guidelines

The management GUI automatically defaults to the recommended number of rebuild areas that is based on the current width and other settings for the distributed array. The width of the array includes the number of physical drives and rebuild areas that are needed to ensure redundancy if drives fail. The system can recommend a maximum of four rebuild areas for distributed arrays for SAS-attached drives and non-compressing NVMe distributed RAID arrays. Compressing NVMe distributed RAID arrays support one rebuild area. Other settings, such as drive class, are also used to determine the best configuration for arrays. For example, if you are adding storage to an existing pool, the management GUI disables any drive classes that are incompatible with drives already in the pool. It then analyzes the current array configuration in the pool to recommend stripe widths and drive counts.

After system setup, you must configure storage by creating pools and assigning storage to specific pools. Ensure that a pool or pools have been created before assigning storage. In the management GUI, select Pools > Actions > Add Storage. The Add Storage automatically configures existing drives into arrays.Use the lsarrayrecommendation command to display the system recommendations for configuring an array. For greatest control and flexibility, you can use the mkarray command-line interface (CLI) command to configure a nondistributed array on your system. To configure a distributed array, you can use the mkdistributedarray command.