During
normal operations in a Fibre Channel environment,
systems are defined by supported configuration and zoning rules. If a single failure causes one or more of these rules to be invalidated,
the configuration is still supported until the failure is corrected
and the configuration is brought back into a normal supported mode.
Descriptions of configuration
terms
A path is a logical connection between
two Fibre Channel ports. The path can exist only
if both of the two Fibre Channel ports are in the
same zone.
A core
switch is the switch that contains the
system ports. Because most SAN fabric traffic might flow through
the system, put the system in the core of the fabric. Some configurations
have a core switch that contains inter-switch links (ISLs) only and
a storage edge switch that contains the system ports. In this rules
summary, a storage edge switch is the same as a core
switch.
A dual-core fabric
design is an environment where two switches are both designated as
core switches in the same fabric. Every node has one Fibre
Channel port that is connected to each of the core switches.
Zoning is then used to ensure that internode traffic flows only within
a single switch wherever possible.
SAN configuration rules
The system supports
any SAN fabric configuration that is supported by the SAN vendors.
For more information about direct-attached
configuration, see the topic about planning for a direct-attached
configuration. For specific requirements when you configure direct
attachments to the system, see the Preventive Service Planning document, Direct Attachment of Storwize and SAN Volume Controller Systems.
Node connectivity via a SAN:
- Configuring SAN communication
between nodes in the same I/O group is optional. All internode communication
between ports in the same I/O group must not cross ISLs.
- Each node in the system must have
at least two ports with paths to all other nodes that are in different
enclosures in the same system. A node cannot have more than 16 paths
to another node in the same system.
- Fibre
Channel connections between the system and the switch can
vary based on fibre types and different SFPs (longwave and shortwave).
Storage system connectivity (optional):
- Connections between the system and
storage require the best available bandwidth. For optimal performance
and reliability, ensure that paths between the
system nodes and storage systems do not cross ISLs. If you use ISLs on these paths,
make sure that sufficient bandwidth is available. SAN monitoring is
required to identify faulty ISLs.
- Each system node must have a path to the same set of worldwide
port names (WWPNs) for each storage system.
- Where multiple paths exist between
the system nodes and storage systems and some of those paths cross
ISLs, use zoning to prevent the system from using the paths that cross
the ISLs.
- SAN
routing technologies between the system and storage systems are supported
when the routing stays entirely within Fibre Channel connectivity and no other transport technologies, such as Internet
Protocol (IP), are used.
- When the SAN is used for connection to an external storage system, each node canister in the same control enclosure must have
a Fibre Channel connection to the same set of Fibre Channel SANs.
Host connectivity:
- Paths between hosts and the system can support
up to three ISL hops between hosts and system nodes.
- The system supports SAN routing technologies
(including FCIP links) between the system and hosts. However, the
use of long-distance FCIP connections might degrade the performance
of any servers that are attached through this technology.
- Hosts can be connected
to system Fibre Channel ports directly or through a SAN fabric.
Intersystem
connectivity:
- For a node port to have a usable path to a port on another node
in the system, the following conditions must be met:
- The ports must be connected to the same Fibre Channel SAN or to
SANs that are linked.
- The SAN zoning must have both the ports in at least one zone.
- The system configuration on the systems must have the port number
on their system enabled for connection to partnered systems. If the
port number is not in the Fibre Channel partner port mask, a usable
path is not available.
- The system supports SAN routing technology (including
FCIP links) for intersystem connections that use Metro Mirror or Global Mirror.
General SAN configuration rules:
- To make best use of the available bandwidth between switches,
use ISL trunking (also known as port channels) on all ISLs.
- When you use Fibre Channel over IP (FCIP) or iSCSI connections, it is best to use jumbo frames
in the IP network.
- Each system can
support 0 - 4 counterpart SANs per system.
- High latency links can affect performance. Regarding the length
of Fibre Channel connections in the SAN, ensure that you conform to
the support statements of the SAN switch vendors and other connected
devices.
- All Fibre Channel devices (except for hosts)
must be connected through a SAN fabric, and must not use direct connections.
You can connect hosts directly or through a SAN fabric.
- The SAN must contain only supported switches, Fibre Channel extenders, and SAN routers. See the following Web sites for specific
firmware levels and supported hardware:
Zoning rules
Notes:
- Apply these rules to each fabric that contains system ports.
- If the edge devices contain more stringent zoning requirements,
follow the storage system rules to further restrict the system zoning
rules. For example, a IBM® DS4000™ system does not support a storage system A and storage system
B in the same zone.
Host zoning:
- The system requires single-initiator zoning for all large configurations
that contain more than 64 host objects. Each server Fibre
Channel port must be in its own zone, which contains the Fibre Channel port and system ports. In configurations of fewer than 64 hosts,
you can have up to 40 Fibre Channel ports in a
host zone if the zone contains similar HBAs and operating systems.
- For optimal performance, include a maximum of two paths per volume
per host Fibre Channel port. This ratio equates
to a zone that contains one port per system node per HBA.
- For load balancing, alternate the server Fibre Channel ports between the ports of the system. For example, the first server is zoned with ports 1 and 3 of each system node (one system port per fabric). The second server is zoned
with ports 2 and 4.
- The maximum number of supported paths to a system volume is eight.
- If a host object is not mapped to all
I/O groups, do not include in the host zone system ports from all nodes in the system. For example,
if node A is in I/O group X and the host object is mapped to I/O group
X, include only ports from node A in the host zone.
The maximum
number of hosts that are mapped to an I/O group is less than the maximum
number of hosts per system. Therefore, in configurations that might
grow to greater than the maximum number of hosts per I/O group, do
not map every host to every I/O group.
- When a dual-core SAN design is used, it is a requirement that
no internode communications use the ISL link. When you create host
zones in this type of configuration, ensure that each system port
in the host zone is attached to the same Fibre Channel switch.
Storage system
zoning:
- For most configurations, follow these rules:
- For every storage system, create one zone that contains system
ports from every node and all storage system ports, unless otherwise
stated by the zoning guidelines for that storage system.
- Single initiator zoning is not required for zones that
include the system and the storage system. The system ports are designed to log in to each other to
form the system.
System zoning:
- If the system has more than one control enclosure,
each node canister in the system must have at least two ports with
paths to all other nodes in the same system. The zoning requirement
to meet these rules is typically satisfied by other zones. However,
for clarity for each SAN fabric, you must create one zone that contains
all ports from this system in that SAN fabric.
- If the system has only one control enclosure
and one or more Fibre Channel ports that are attached to a SAN fabric,
allow communication between node canisters over the Fibre Channel
network. Creating one zone per system in each SAN fabric and adding
all ports from this system into that zone satisfies this best practice.