Generic fabric checks

We first describe a methodology for troubleshooting SAN fabrics in general terms. During the descriptions of each step, we denote whether the actions of each step are applicable in single switch SANs, multi-switch environments, or both types of SAN fabrics.

Figure 4-15 on page 149 provides a view of the flow of the basic troubleshooting steps for the problem-determination process for the SAN fabric. Additional descriptions of the individual steps follow the map.

The basic intent of these procedures is to quickly verify the health of the SAN fabric. As with the previous troubleshooting processes, we determine a logical starting point and then work outwards from that starting point until either the problem source is discovered or the SAN fabric is determined to be operating properly.

Note: The device number and the device index number are usually the same. However, the devices can be configured out of order, thus, the numbers are not always consistent. To correlate the index number for a specific device, issue the datapath query device command. At the same time as when checking the correlations, also verify that the each adapter is in the NORMAL state and ACTIVE mode.

The starting point for the SAN fabric in general is the links between the various devices that make up the fabric. Once the links are verified, the next step is to check on the connections from the fabric to the edge devices, such as servers and storage units. Finally, we explore the edge-to-edge logical connectivity by checking the zoning configurations.

In the product-specific sections we also provide the methods used for collecting additional information, which are used with the advanced problem-determination steps. Examples of this information include error logs, audit logs, and port performance data.

Figure 4-15 SAN fabric switch troubleshooting map

Step 1 of the SAN fabric troubleshooting map is a quick and simple integrity check for all devices that make up the SAN fabric. In the case where only a single switch is in the SAN fabric, this step can be skipped. Using the SAN diagram and the supporting information for fabric devices, verify that all of the

fabric devices are known to each other. This action can be a simple command at a device prompt, or can use a graphical device management application. Either way, the devices in the fabric must know of each other’s existence, and these links must be of the correct type of connection. When switches of the same type are interconnected, the end ports should be listed as E_ports (or T_Ports for INRANGE products), or expansion ports. Some switches may use different terminology, but the idea is that inter-switch links are not based on the same type of end port definitions as a port connecting an edge device.

If one or more fabric devices are missing from one location in the fabric, check other switches to see if the problem is consistent. If so, then areas for further checks are:

򐂰 Port status of connections to other fabric devices

Check the port status of all connections to fabric devices. Their conditions should be operational and the proper type of connection (as in E-port for interswitch links). If the port is not operational, use the product’s user’s guide or service guide to determine the cause. Possible causes are cabling issues, configuration changes, or hardware failures. Most of these causes are usually found quickly.

򐂰 Error log of the switch

Refer to the appropriate sections of the corresponding product manuals for information on gathering and making use of this information.

򐂰 Zoning configuration of the affected switches in comparison to other switches without the problem

We describe methods to view zoning configuration information in general terms further on in this chapter.

After verification that all SAN fabric devices are know to each other, the second step is to confirm the logical connections from the fabric to the edge devices.

Depending on the nature of the problem, some of these connections have already been checked from a subset of the edge devices. However, the

Note: Fibre Channel standards are still being implemented at various rates of progress by the different Fibre Channel switch manufacturers. Thus,

inter-switch connections between two switches of different makers may not have a sufficient level of interoperability to establish an fabric expansion link, or E_port. Refer to documentation from the various manufacturers to determine if their products are capable of establishing inter-switch links with each other’s product.

communications links are two-way. Thus, one side may be talking while the other end of a link is not listening or able to understand the conversation. This is the reason for inspecting and verifying a link at the fabric end, even though the given link has already been checked from the edge device.

Step 2 of the fabric check map (Figure 4-15 on page 149) is a detailed

examination of the links from some, or all, of the fabric devices to the attached edge devices. The main aspects of a link to investigate are that the link is established, and the link is defined as the proper type of connection. Thus, the link to an edge device should be connected to an F_port, or fabric port, at the switch’s end. Hopefully, not all edge device connections will need to be checked.

During initial rounds of the problem determination process, the reader has defined the scope of the problem. Only the ports with connections to edge devices identified during the initial understanding of the problem’s scope need to be checked. It is a rare situation where all fabric ports to edge devices will have to be checked. However, the reader should remember that intermittent problem types or the many-to-many scenario may warrant such an extreme as testing all ports.

Step 3 of the fabric check map is a verification of the zoning configuration that is implemented in the SAN fabric. This step is based on the assumption that the various fabric port checks in Steps 1 and 2 showed that all connections are properly established. This action will verify the zoning configuration of one switch for proper connectivity through the SAN fabric to the effected edge devices. If there are multiple switches in the fabric, the confirmed zoning configuration of the first checked switch is then compared to the remaining switches. Depending on the type of switch used (IBM 2109, McDATA, or INRANGE), the comparison process may be quickly accomplished, or may take a significant amount of detailed checking.