550: A clustered system cannot be formed because of a lack of clustered system resources.

Explanation

The node cannot become active in a clustered system because it is unable to connect to enough clustered system resources. The clustered system resources are the nodes in the system and the active quorum disk. The node needs to be able to connect to a majority of the resources before that group forms an online clustered system. This prevents the clustered system splitting into two or more active parts, with both parts independently performing I/O.

Supplemental data that is displayed with this error code list the missing IDs for the 2145s and the quorum disk controller. Each missing node is listed by its node ID. A missing quorum disk is listed as WWWWWWWWWWWWWWWW/LL, where WWWWWWWWWWWWWWWW is a worldwide port name (WWPN) on the disk controller that contains the missing quorum disk and LL is the Logical Unit Number (LUN) of the missing quorum disk on that controller.

If the system topology is stretched and the number of operational nodes are less than half, then node error 550 is displayed. In this case, the Site Disaster Recovery feature cannot be used as the number of operational nodes is less than the quorum required to create the clustered system that uses the Site Disaster Recovery feature.

Explanation

The node canister cannot become active in a clustered system because it is unable to connect to enough clustered system resources. The clustered system resources are the node canisters in the system and the active quorum disk or drive. The node needs to be able to connect to a majority of the resources before that group forms an online clustered system. This prevents the clustered system splitting into two or more active parts, with both parts independently performing I/O.

The error data lists the missing resources. This includes a list of node canisters and optionally a drive that is operating as the quorum drive or a LUN on an external storage system that is operating as the quorum disk.

If a drive in one of the system enclosures is the missing quorum disk, it is listed as enclosure:slot[part identification] where enclosure:slot is the location of the drive when the node shutdown, enclosure is the seven-digit product serial number of the enclosure, slot is a number between 1 and 24. The part identification is the 22 character string that starts with "11S" found on a label on a drive. The part identification cannot be seen until the drive is removed from the enclosure.

If a LUN on an external storage system is the missing quorum disk, it is listed as WWWWWWWWWWWWWWWW/LL, where WWWWWWWWWWWWWWWW is a worldwide port name (WWPN) on the storage system that contains the missing quorum disk and LL is the Logical Unit Number (LUN).

Explanation

The node canister cannot become active in a clustered system because it is unable to connect to enough clustered system resources. The clustered system resources are the node canisters in the system and the active quorum disk or drive. The node needs to be able to connect to a majority of the resources before that group forms an online clustered system. This prevents the clustered system splitting into two or more active parts, with both parts independently performing I/O.

The error data lists the missing resources. This includes a list of node canisters and optionally a drive that is operating as the quorum drive or a LUN on an external storage system that is operating as the quorum disk.

If a drive in one of the system enclosures is the missing quorum disk, it is listed as enclosure:slot[part identification] where enclosure:slot is the location of the drive when the node shutdown, enclosure is the seven-digit product serial number of the enclosure, slot is a number between 1 and 24. The part identification is the 22 character string that starts with "11S" found on a label on a drive. The part identification cannot be seen until the drive is removed from the enclosure.

If a LUN on an external storage system is the missing quorum disk, it is listed as WWWWWWWWWWWWWWWW/LL, where WWWWWWWWWWWWWWWW is a worldwide port name (WWPN) on the storage system that contains the missing quorum disk and LL is the Logical Unit Number (LUN).

Explanation

The node canister cannot become active in a clustered system because it is unable to connect to enough clustered system resources. The clustered system resources are the node canisters in the system and the active quorum drive. The node canister needs to be able to connect to a majority of the resources before that group forms an online clustered system. This prevents the clustered system splitting into two or more active parts, with both parts independently performing I/O.

The error data lists the missing resources. This includes a list of node canisters and optionally a drive that is operating as the quorum drive.

If a drive in one of the system enclosures is the missing quorum disk, it is listed as enclosure:slot[part identification] where enclosure:slot is the location of the drive when the node shutdown, enclosure is the seven-digit product serial number of the enclosure, slot is a number between 1 and 24. The part identification is the 22 character string that starts with "11S" found on a label on a drive. The part identification cannot be seen until the drive is removed from the enclosure.

Explanation

The node canister cannot become active in a clustered system because it is unable to connect to enough clustered system resources. The clustered system resources are the node canisters in the system and the active quorum drive. The node canister needs to be able to connect to a majority of the resources before that group forms an online clustered system. This prevents the clustered system splitting into two or more active parts, with both parts independently performing I/O.

The error data lists the missing resources. This includes a list of node canisters and optionally a drive that is operating as the quorum drive.

If a drive in one of the system enclosures is the missing quorum disk, it is listed as enclosure:slot[part identification] where enclosure:slot is the location of the drive when the node shutdown, enclosure is the seven-digit product serial number of the enclosure, slot is a number between 1 and 24. The part identification is the 22 character string that starts with "11S" found on a label on a drive. The part identification cannot be seen until the drive is removed from the enclosure.

User Response

1. Check the status of other node canisters in the system, resolve any faults on them.
2. Check that all enclosures in the system are powered on and that the SAS cabling between the enclosures has not been disturbed. If any wiring changes have been made, check that all cables are securely connected and that the cabling rules have been followed.

   Check that all nodes in the system are shown in the service assistant or by using the service command: sainfo lsservicenodes. Investigate any missing nodes.

3. Check all nodes and quorum disks shown in the error data and check the communication links from this node to those nodes and quorum disks.
   1. If a quorum drive in a system enclosure is shown as missing, find the drive and check that it is working. The drive may have been moved from the location shown. In that case, find the drive and ensure it is installed and working. If the drive is not located in the control enclosure, try moving it to the control enclosure. A problem in SAS connectivity might be the issue.
      Note: If you are able to reestablish the system's operation, you will be able to use the extra diagnostics the system provides to diagnose problem on SAS cables and expansion enclosures.
   2. If a quorum disk on an external storage system is shown as missing, find the storage controller and confirm that the LUN is available. Check that the Fibre Channel connections between the storage controller and the 2076 are working and that any changes made to the SAN configuration and zoning have not effected the connectivity. Check the status of the Fibre Channel ports on the node and resolve any issues.
4. If all canisters have either node error 578 or 550, attempt to reestablish a clustered system by following the service procedures for the nodes showing node error 578. If this is not successful, follow the system recovery procedures.

User Response

1. Check the status of other node canisters in the system, resolve any faults on them.
2. Check that all enclosures in the system are powered on and that the SAS cabling between the enclosures has not been disturbed. If any wiring changes have been made, check that all cables are securely connected and that the cabling rules have been followed.

   Check that all nodes in the system are shown in the service assistant or by using the service command: sainfo lsservicenodes. Investigate any missing nodes.

3. Check all nodes and quorum disks shown in the error data and check the communication links from this node to those nodes and quorum disks.
   1. If a quorum drive in a system enclosure is shown as missing, find the drive and check that it is working. The drive may have been moved from the location shown. In that case, find the drive and ensure it is installed and working. If the drive is not located in the control enclosure, try moving it to the control enclosure. A problem in SAS connectivity might be the issue.
      Note: If you are able to reestablish the system's operation, you will be able to use the extra diagnostics the system provides to diagnose problem on SAS cables and expansion enclosures.
   2. If a quorum disk on an external storage system is shown as missing, find the storage controller and confirm that the LUN is available. Check that the Fibre Channel connections between the storage controller and the 2076 are working and that any changes made to the SAN configuration and zoning have not effected the connectivity. Check the status of the Fibre Channel ports on the node and resolve any issues.
4. If all canisters have either node error 578 or 550, attempt to reestablish a clustered system by following the service procedures for the nodes showing node error 578. If this is not successful, follow the system recovery procedures.

User Response

1. Check the status of other node canisters in the system, resolve any faults on them.
2. Check that all enclosures in the system are powered on and that the SAS cabling between the enclosures has not been disturbed. If any wiring changes have been made, check that all cables are securely connected and that the cabling rules have been followed.

   Check that all nodes in the system are shown in the service assistant or by using the service command: sainfo lsservicenodes. Investigate any missing nodes.

3. Check all nodes and quorum disks shown in the error data and check the communication links from this node to those nodes and quorum disks.
   1. If a quorum drive in a system enclosure is shown as missing, find the drive and check that it is working. The drive may have been moved from the location shown. In that case, find the drive and ensure it is installed and working. If the drive is not located in the control enclosure, try moving it to the control enclosure. A problem in SAS connectivity might be the issue.
      Note: If you are able to reestablish the system's operation, you will be able to use the extra diagnostics the system provides to diagnose problem on SAS cables and expansion enclosures.
   2. If a quorum disk on an external storage system is shown as missing, find the storage controller and confirm that the LUN is available. Check that the Fibre Channel connections between the storage controller and the 2076 are working and that any changes made to the SAN configuration and zoning have not effected the connectivity. Check the status of the Fibre Channel ports on the node and resolve any issues.
4. If all canisters have either node error 578 or 550, attempt to reestablish a clustered system by following the service procedures for the nodes showing node error 578. If this is not successful, follow the system recovery procedures.

User Response

Follow troubleshooting procedures to correct connectivity issues between the cluster nodes and the quorum devices.

1. Check for any node errors that indicate issues with Fibre Channel connectivity. Resolve any issues.
2. Ensure that the other systems in the cluster are powered on and operational.
3. Check the Fibre Channel port status. If any port is not active, perform the Fibre Channel port problem determination procedures.
4. Ensure that Fibre Channel network zoning changes have not restricted communication between nodes or between the nodes and the quorum disk.
5. Perform the problem determination procedures for the network.
6. The quorum disk failed or cannot be accessed. Perform the problem determination procedures for the disk controller.

User Response

Follow troubleshooting procedures to correct connectivity issues between the cluster nodes and the quorum devices.

1. Check for any node errors that indicate issues with bus or Fibre Channel connectivity. Resolve any issues.
2. Check the status of other node canisters in the system, resolve any faults on them.
3. Check all enclosures in the system are powered on and that the SAS cabling between the enclosures has not been disturbed. If any wiring changes have been made check all cables are securely connected and that the cabling rules have been followed.
4. If a quorum drive in a system enclosure is shown as missing, find the drive and check that it is working. The drive may have been moved from the location shown, in that case find the drive and ensure it is installed and working. If the drive is not located in the control enclosure, try moving it to the control enclosure, because a problem in SAS connectivity may be the issue.
   Note: If you are able to reestablish the systems operation you will be able to use the extra diagnostics the system provides to diagnose problems on SAS cables and expansion enclosures.
5. If a quorum disk on an external storage system is shown as missing, find the storage controller and confirm that the LUN is available, check the Fibre Channel connections between the storage controller and the 2076 are working and that any changes made to the SAN configuration and zoning have not effected the connectivity. Check the status of the Fibre Channel ports on the node and resolve any issues.
6. If a quorum disk on an external storage system is shown as missing, find the storage control and confirm that the LUN is available, check the Fibre Channel connections between the storage controller and the system are working and that any changes made to the SAN configuration and zoning have not effected the connectivity. Check the status of the Fibre Channel ports on the canister and resolve any issues.
7. If all canisters have either node error 578 or 550, attempt to reestablish a cluster by following the service procedures for the nodes showing node error 578. If this is not successful, follow the cluster recovery procedures.