HP 5920 & 5900 Switch Series
FCoE
Configuration Guide
Part number: 5998-3375 Software version: Release2207 Document version: 6W100-20121130
Legal and notice information
© Copyright 2012 Hewlett-Packard Development Company, L.P.
No part of this documentation may be reproduced or transmitted in any form or by any means without prior written consent of Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.
HEWLETT-PACKARD COMPANY MAKES NO WARRANTY OF ANY KIND WITH REGARD TO THIS MATERIAL, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. Hewlett-Packard shall not be liable for errors contained herein or for incidental or consequential damages in connection with the furnishing, performance, or use of this material.
The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.
Contents
FCoE overview ··· 1
Storage area network ··· 1
FC SAN ··· 1 FC protocol ··· 2 Basic concepts ··· 2 Communication flow ··· 3 VSAN ··· 4 FC zone ··· 4 FCoE ··· 4 Basic concepts ··· 5
How FCoE works ··· 6
FCoE modes ··· 8
FCF mode ··· 8
NPV mode ··· 9
Protocols and standards ··· 9
FCoE configuration guidelines ··· 11
FCoE features supported by different FCoE modes ··· 11
Configuring an FCoE mode for a switch ··· 11
Configuring VFC interfaces and FIP ··· 12
VFC interfaces and FIP configuration task list ··· 12
Configuring a VFC interface ··· 12
Enabling FCoE for a VLAN and mapping a VSAN to the VLAN ··· 13
Configuration restrictions and guidelines ··· 13
Configuration procedure ··· 13
Configuring the FC-MAP value ··· 13
Configuring the FKA advertisement period value ··· 14
Role of the FKA advertisement period value ··· 14
Configuration restrictions and guidelines ··· 14
Configuring the FCF priority ··· 15
Configuring the system FCF priority ··· 15
Configuring the VFC interface FCF priority ··· 16
Displaying and maintaining VFC interfaces and FIP ··· 16
VFC interfaces and FIP configuration example ··· 16
Network requirements ··· 16
Configuration procedure ··· 17
Setting up a fabric ··· 19
Overview ··· 19
Principal switch selection ··· 19
Domain ID assignment ··· 20
FC address assignment ··· 21
Fabric setup configuration task list ··· 21
Building a fabric statically ··· 21
Building a fabric dynamically ··· 22
Enabling or disabling the fabric configuration function ··· 22
Setting a fabric name ··· 23
Setting the switch priority ··· 23
Configuring a domain ID for a switch ··· 24
Configuring the mapping between the N_Port WWN and the FC address ··· 25
Configuring the fabric timers ··· 25
Configuring the fabric timers in system view ··· 25
Configuring the fabric timers in VSAN view ··· 26
Configuring the fabric reconfiguration ··· 26
Configuring the auto fabric reconfiguration function ··· 27
Manually initiating the fabric reconfiguration ··· 27
Configuring a VFC interface to reject incoming RCF requests ··· 27
Displaying and maintaining a fabric ··· 27
Static fabric building configuration example ··· 28
Network requirements ··· 28
Configuration procedure ··· 28
Verifying the configurations ··· 29
Dynamic fabric building configuration example ··· 30
Network requirements ··· 30
Configuration procedure ··· 30
Verifying the configurations ··· 31
Configuring VSAN ··· 33
Overview ··· 33
VSAN fundamentals ··· 33
Creating a VSAN ··· 34
Configuring a trunk VSAN ··· 34
Displaying and maintaining VSAN ··· 35
VSAN configuration example ··· 35
Network requirements ··· 35
Configuration considerations ··· 35
Configuration procedure ··· 36
Verifying the configurations ··· 37
Configuring FC routing and forwarding ··· 38
Overview ··· 38
Routing table and FIB table ··· 38
Direct routes ··· 39
Static routes ··· 39
FSPF routes ··· 40
Configuring static routes for FC ··· 41
Configuration restrictions and guidelines ··· 41
Configuration procedure ··· 41
Configuring FSPF ··· 41
FSPF configuration task list ··· 42
Enabling FSPF ··· 42
Configuring the shortest SPF calculation interval ··· 42
Configuring the minimum LSR receiving interval ··· 43
Configuring the minimum LSR refresh interval ··· 43
Configuring the FSPF cost for an interface ··· 43
Configuring the hello interval for an interface ··· 44
Configuring the dead interval for an interface ··· 44
Configuring the LSR retransmission interval for interfaces ··· 44
Disabling FSPF for an interface ··· 45
Configuring FSPF GR ··· 45
Displaying and maintaining FC routing and forwarding ··· 46
Static FC routing configuration example ··· 46
Configuration procedure ··· 47
Verifying the configurations ··· 48
FSPF configuration example ··· 49
Network requirements ··· 49
Configuration procedure ··· 50
Verifying the configurations ··· 51
Configuring FC zones ··· 53 Overview ··· 53 Zone database··· 53 Distributing zones ··· 55 Zone merge ··· 57 Access control ··· 59
FC zone configuration task list ··· 59
Configuring zone aliases ··· 60
Configuring zones ··· 60
Configuring zone sets ··· 61
Configuring the default zone policy··· 61
Configuring zone distribution and merge types ··· 61
Activating a zone set and distributing it to the entire fabric ··· 62
Triggering a complete distribution ··· 62
Renaming zone aliases, zones, and zone sets ··· 63
Copying zone aliases, zones, and zone sets ··· 63
Deleting the zone database ··· 63
Displaying and maintaining FC zones ··· 64
FC zone configuration example ··· 64
Network requirements ··· 64
Configuration considerations ··· 65
Configuration procedure ··· 65
Verifying the configurations ··· 66
Configuring NPV ··· 68
Overview ··· 68
Downlink interface and downlink ··· 68
Uplink interface and uplink ··· 68
Downlink-to-uplink interface mappings ··· 69
Disruptive load balancing ··· 69
NPV configuration task list ··· 69
Configuring uplink interfaces and downlink interfaces ··· 69
Configuring uplink interfaces ··· 69
Configuring downlink interfaces ··· 70
Configuring downlink-to-uplink interface mappings ··· 70
Initiating a disruptive load-balancing process ··· 71
Displaying and maintaining NPV ··· 71
NPV configuration example ··· 71
Network requirements ··· 71
Configuration procedure ··· 72
Verifying the configurations ··· 73
Configuring FC ping ··· 75
Overview ··· 75
Configuration procedure ··· 75
FC ping configuration example ··· 75
Network requirements ··· 75
Configuring FC tracert ··· 78
Overview ··· 78
Configuration procedure ··· 79
FC tracert configuration example ··· 79
Network requirements ··· 79
Configuration procedure ··· 79
Appendixes ··· 82
Appendix A Fabric address assignment ··· 82
Appendix B Well-known fabric addresses ··· 82
Support and other resources ··· 84
Contacting HP ··· 84 Subscription service ··· 84 Related information ··· 84 Documents ··· 84 Websites ··· 84 Conventions ··· 85 Index ··· 87
FCoE overview
Storage area network
According to the Storage Networking Industry Association dictionary, "a storage area network (SAN) is any high-performance network whose primary purpose is to enable disk devices to communicate with computer systems and with each other."
A SAN enables the universal connectivity of servers and disk devices. Compared to the conventional client/server computer system, a SAN allows the servers to share data and directly access data created by one another without having to copy it, improves storage scalability, and centralizes the management of data backup, access, and security.
Most SANs use Fibre Channel (FC) or Ethernet to interconnect devices. An FC SAN uses the FC protocol suite for communication, and an Ethernet SAN uses the TCP/IP protocol suite for communication. This document covers only the FC SAN.
FC SAN
As shown in Figure 1, an FC SAN connects the data sending and receiving entities (network servers and disk devices) with fibers or copper wires in the following ways:
• Directly connects a server and a disk device, as shown in the point-to-point connection.
• Connects servers and disk devices to an FCF switched fabric, as shown in the switched fabric. In a switched fabric, the servers and disk devices are called "nodes." A fabric uses 24-bit addressing and supports thousands of devices.
Figure 1FC SAN networking
NOTE:
• An FC SAN refers to a network comprising FCF switches and nodes. • A fabric refers to a transmission network comprising FCF switches.
FC protocol
The servers, FCF switches, and disk devices in an FC SAN must all support FC.
Basic concepts
WWN
The World Wide Name (WWN) is a 64-bit address that identifies a fabric or an entity (such as an FCF switch, node, or port) in an FC SAN. The upper-layer protocol of FC uses WWNs for communication. Each entity has a factory-assigned globally unique WWN.
FC address
The FC protocol accesses communication entities in a SAN through FC addresses. An FC address is also known as an "FC_ID."
Figure 2 shows the structure of an FC address. The FC address is 24 bits long and is divided into these 8-bit fields: Domain_ID, Area_ID, and Port_ID.
• A domain represents a switch and all N_Ports connected to the switch. A Domain_ID, which is in the range of 1 to 239, uniquely identifies an FCF switch.
• One or more N_Ports on the same node can be assigned to an area, which is identified by an Area_ID.
• The Port_ID field identifies an N_Port. Figure 2Structure of an FC address
A Domain_ID can uniquely identify an FCF switch. Different FCF switches in the same fabric have different Domain_IDs.
An FC address can uniquely identify an N_Port on a node. Different N_Ports on the same node have different FC addresses. FCF switches use Domain_IDs to route messages between each other.
The FC protocol standardizes the FC address usage. For more information, see "Appendixes."
Interface modes
Figure 3Port modes
1. The interface on a node is called an "N_Port." 2. An FCF switch provides the following types of ports:
{ F_Port—Connects to an N_Port or an NP_Port on another FCF switch. { E_Port—Connects to an E_Port on another FCF switch.
{ NP_Port—Connects to an F_Port on another FCF switch. For more information about NP_Port, see "Configuring NPV."
E_Ports connect FCF switches to form a fabric, and F_Ports connect the nodes to FCF switches in the fabric.
Communication flow
FCF switches provide data transmission services. Through FCF switches, a server sends instructions and data to disk devices and reads data from disk devices.
Figure 4FC SAN communication model
The following takes a server accessing a disk device as an example to see how data communication occurs in an FC SAN.
1. The server and the disk device use the fabric login (FLOGI) protocol to register with the FCF switches, which then assign FC addresses to each directly-connected node.
2. The registered server and disk device send name service registration requests to their respective access FCF switches to register name service information, including their WWNs and FC addresses. Finally, each FCF switch in the fabric stores the name service information for all nodes. 3. To access a disk device, the server needs to send a name service query request to its
directly-connected FCF switch to obtain the list of disk devices in the fabric and their WWNs and FC addresses.
4. After the server obtains the FC address of the disk device, the server can send FC frames (with the FC address of the disk device as the destination FC address) to the FCF switch nearby.
5. When the FCF switch receives the FC frame from the server, it queries its FIB table for a data forwarding path according to the destination FC address, and forwards the FC frame to the next-hop FCF switch. The next-hop FCF switch forwards the FC frame in the same way, until the FCF switch at the last hop forwards the FC frame to the destination disk device.
NOTE:
A FIB table is generated by the FCF switch through calculation based on the FC routing protocol or configured static routes.
VSAN
In actual applications, the data is insecure because the data of all users is transmitted in the same FC SAN. You can divide one physical FC SAN into multiple Virtual Storage Area Networks (VSANs). In this manner, VSANs are separated from one another and provide independent services, enhancing adaptability and security of the network and offering more effective services for users. For more information about VSAN, see "Configuring VSAN."
FC zone
With VSAN, one physical SAN is divided into multiple logical SANs. A VSAN, however, cannot perform access control over the servers and disk devices (or the N_Ports) connected to a fabric. N_Ports in the same VSAN can access one another only if these N_Ports register name services. This creates data security risks.
Zoning can solve the preceding problem by dividing a VSAN into zones and adding N_Ports to different zones for different purposes. In this manner, N_Ports in different zones are separated to implement access control.
For more information about FC zones, see "Configuring FC zones."
FCoE
A data center using the FC SAN technology usually comprises separate local area networks (LANs) and SANs. LANs carry traditional Ethernet/IP services, and SANs carry network storage services.
To provide services for LANs and use SANs for storage simultaneously, the servers must use independent Ethernet adapters and FC adapters. In addition, the IP switches and the FCF switches are also independent and have independent network connections. Such a network needs many switches, network adapters, and cables, and it brings high investments and maintenance costs and low scalability. FCoE was introduced to solve this problem. FCoE is a protocol that carries FC over Ethernet. In an FCoE solution, the server uses an FCoE-capable Ethernet adapter, and the FCoE switch (FCoE forwarder) integrates the functions of both the traditional IP switch and FCF switch. FCoE reduces the number of network adapters, switches, and cables, and the network operation and maintenance workload. In all, FCoE reduces the total cost.
Figure 5FCoE for I/O consolidation
As shown in Figure 5, in the traditional network, the server is connected to the LAN through an Ethernet interface and to the SAN through an FC interface. In the FCoE network, the server is connected to the FCoE-capable FCF switch, and then the FCF switch is connected to the LAN through an Ethernet interface and to the SAN through an FC interface. The links between the server and the FCF switch and between FCF switches can transmit both Ethernet frames and FC frames.
For more information about FCoE, see "Configuring FCoE."
Basic concepts
As shown in Figure 6, the links between the FCF switch and the ENode (nodes that can transport FC over Ethernet, such as servers and disk devices) and between FCF switches can be used for receiving and sending both Ethernet frames and FC frames.
Figure 6FCoE network diagram
VFC interface and VN interface
A virtual fiber channel (VFC) interface is a logical interface manually created on the FCF switch to simulate the function of a physical FC interface.
You can connect either an ENode or an FCF switch to a VFC interface. VFC interfaces support E mode, F mode (default), and NP mode.
The virtual node (VN) interface is a logical interface on an ENode to simulate the function of a physical FC interface.
FIP protocol
FCoE initialization protocol (FIP) is an FCoE control protocol that establishes and maintains virtual links. FIP establishes a virtual link between the VFC interface of an FCF switch and the VN interface of an ENode or between VFC interfaces of two FCF switches to provide a physical infrastructure for transmitting FC frames over Ethernet.
FCoE frames
To transmit an FC frame over an Ethernet link, you must encapsulate the FC frame in an FCoE frame by adding an Ethernet frame header to the FC frame.
An FCoE frame uses Ethernet II encapsulation, which has the following fields in the Ethernet header: • EtherType 0x8906.
• Destination MAC address/source MAC address—For a switch, it is the FCoE MAC address of the switch (which can be displayed by using the display fcoe command). For a node, it is the fabric provided MAC address (FPMA) of the node. As shown in Figure 7, an FPMA is composed of the FC-MAP as the 24 most significant bits and the FC ID of the VN interface as the 24 least significant bits. The FC-MAP takes the value of the switch FC-MAP, 0x0EFC00 by default and confiugrable by using the fcoe fcmap command.
Figure 7FPMA composition
How FCoE works
Figure 8Block diagrams of the ENode and the FCF switch
Virtual link ENode FCF VN interface VFC interface FC layer Ethernet layer Ethernet interface Ethernet interface FC layer Ethernet layer
NOTE:
This section describes how FCoE works only on the FCF switch, rather than on the ENode.
Procedure for receiving and sending FC frames over Ethernet
An FC frame is transmitted over Ethernet using the following workflow:
• FIP establishes a virtual link between the VFC interface of the FCF switch and the VN interface of the ENode or between VFC interfaces of two FCF switches.
• After the virtual link is established, the FCF switch encapsulates the FC frame in an FCoE frame and sends it out.
• After receiving the FCoE frame, the FCF switch removes its Ethernet header to send the original FC frame to the upper layer for processing.
How FIP works
FIP sets up and maintains virtual links between a VFC interface and a VN interface or between VFC interfaces.
Two categories of packets are used in FIP: Discovery Solicitation and Discovery Advertisement. There are two types of Discovery Advertisement:
• Solicited Discovery Advertisement—A reply for a Discovery Solicitation. • Unsolicited Discovery Advertisement—Periodically sent.
The following example shows how a virtual link is set up between an FCF switch and an ENode. Figure 9FIP operation
As shown in Figure 9, the following workflow is used to set up a virtual link: 1. The ENode sends a Discovery Solicitation containing its FCoE MAC address.
2. After receiving the Discovery Solicitation, the FCF switch acts differently depending on whether the receiving VFC interface is bound to the FCoE MAC address:
{ If it is not bound, the switch learns the FCoE MAC address and replies with a solicited
(1) Send Discovery Solicitation Learn FCoE MAC address
FCF ENode
(2) Send solicited Discovery Advertisement (3) Send solicited Discovery
Advertisements periodically
(4) Send FLOGI request Check FCoE MAC address
(5) Send FLOGI LS_ACC (6) Send solicited Discovery
{ If it is bound, the switch checks whether the FCoE MAC address matches the bound FCoE MAC address. If they match, it replies with a solicited Discovery Advertisement, whose fcf priority field carries the FCF priority of the VFC interface. If they do not match, it discards the Discovery Solicitation.
3. The FCF switch periodically sends unsolicited Discovery Advertisements, whose fcf priority field carries the FCF priority of the system. The sending interval is specified by using the fcoe fka-adv-period command and defaults to 8 seconds.
4. After receiving the Discovery Advertisements, the ENode determines the FCF switch with the highest priority according to the fcf priority field and sends a FLOGI request frame to that switch for login.
5. After receiving the FLOGI request frame, the FCF checks whether the source MAC address matches its learned or bound FCoE MAC address. If they match, it sends a FLOGI LS_ACC, which indicates the setup of the virtual link. Otherwise, it discards the FLOGI frame.
6. The FCF switch also periodically sends unsolicited Discovery Advertisements to maintain
established virtual links. If the ENode fails to receive an unsolicited Discovery Advertisement within a period 2.5 times the interval specified by the fcoe fka-adv-period command, it deletes the virtual link.
FCoE modes
The switch supports the following FCoE modes:
• FCF mode—A switch operating in this mode is called an FCF switch. Its VFC interfaces support E mode (E_Port) and F mode (F_Port).
• NPV mode—A switch operating in this mode is called an N_Port Virtualizer (NPV) switch. Its VFC interfaces support F mode (F_Port) and NP mode (NP_Port).
An FCoE-capable switch can operate in the following modes:
• FCF mode—When the switch operates in this mode, it can connect to the E_Port on another FCF switch through its E_Port, or connect to the N_Port on a node or the NP_Port on an NPV switch through its F_Port.
• NPV mode—When the switch operates in this mode, it can connect to the N_Port on a node through its F_Port or to the F_Port on an FCF switch through its NP_Port.
• Non-FCoE mode—When the switch operates in this mode, it is a standard switch and does not provide any FCoE capabilities.
FCF mode
An FCF switch encapsulates FC frames in Ethernet frames and uses FCoE virtual links to simulate physical FC links. Therefore, it provides standard FC switching capabilities and features on a lossless Ethernet network.
Figure 10FCF network diagram
In an FCoE environment as shown in Figure 10, different from a pure FC network, the ENode and FCF switch communicate over Ethernet interfaces on a lossless Ethernet network. The FCoE virtual link between the ENode and FCF switch connects a VN interface to a VFC interface, and the FCoE virtual link between FCF switches connects two VFC interfaces.
Each FCF switch is assigned a domain ID. Each FC SAN supports a maximum number of 239 domain IDs, so an FC SAN cannot have more than 239 FCF switches.
NPV mode
An FC SAN needs a large number of edge switches that connect directly to nodes. N_Port Virtualization (NPV) switches are developed to expand the number of switches in an FC SAN.
Figure 11NPV network diagram
As shown in Figure 11, the NPV switch resides between nodes and the core switch on the edge of the fabric. The core switch is a switch operating in FCF mode. The NPV switch is connected to the nodes through its F_Ports and to the core switch through its NP_Port. In this manner, the NPV switch forwards traffic from its connected nodes to the core switch.
The NPV switch appears as an FCF switch to nodes and as a node to the core switch. For more information about NPV, see "Configuring NPV."
• FC-SW-5, Fibre Channel - Switch Fabric - 5 • FC-LS-2, Fibre Channel - Link Services - 2 • FC-GS-6, Fibre Channel - Generic Services - 6 • FC-BB-5, Fibre Channel - Back Bone – 5
FCoE configuration guidelines
The switch supports FCoE only when operating in advanced mode. For more information about system operating modes, see Fundamentals Configuration Guide.
FCoE features supported by different FCoE modes
The switch supports two FCoE modes: FCF mode and NPV mode. Each mode has different features as shown in Table 1. You can choose to configure different features based on the FCoE mode of a switch. Table 1FCoE functions supported by different FCoE modesFCoE mode
FCoE feature FCF mode NPV mode
Configuring VFC interfaces and
FIP Supported Supported
Setting up a fabric Supported Only the following function is supported: "Configuring the fabric timers."
Configuring VSAN Supported Supported
Configuring FC routing and
forwarding Supported
Only the following functions are supported:
• Displaying FC routing table information
• Displaying FC FIB table information
• Display FC Exchange table information
Configuring FC zones Supported Not supported
Configuring NPV Not supported Supported
Configuring FC ping Supported Not supported
Configuring FC tracert Supported Not supported
Configuring an FCoE mode for a switch
An FCoE-capable switch can operate in FCF mode, NPV mode, or non-FCoE mode. The switch can only convert from non-FCoE mode to one FCoE mode, or vice versa, and it cannot convert directly among the two FCoE modes. To convert among the two FCoE modes, first convert the switch to non-FCoE mode. After converting the switch to non-FCoE mode, FCoE-related configurations in the original FCoE mode are cleared.
To configure an FCoE mode for a switch:
Step Command Remarks
1. Enter system view. system-view N/A
2. Configure an FCoE mode
Configuring VFC interfaces and FIP
VFC interfaces and FIP configuration task list
Tasks at a glance(Required.) Configuring a VFC interface
(Required.) Enabling FCoE for a VLAN and mapping a VSAN to the VLAN
(Optional.) Configuring the FC-MAP value
(Optional.) Configuring the FKA advertisement period value
(Optional.) Configuring the FCF priority
Configuring a VFC interface
Step Command Remarks
1. Enter system view. system-view N/A
2. Create a VFC interface and
enter its view. interfacevfcinterface-number N/A
3. Configure the VFC interface
mode. fc mode { e | f | np }
By default, a VFC interface operates in F mode.
• When an FCF switch operates in FCF mode, VFC interfaces support E and F modes.
• When an FCF switch operates in NPV mode, FC interfaces support F and NP modes.
4. Bind the VFC interface to the specified Ethernet interface.
bind interfaceinterface-type
interface-number [mac
mac-address ]
By default, no Ethernet interface is bound to a VFC interface.
The VFC interface sends and receives packets through the Ethernet interface bound to it.
5. Assign the VFC interface to the specified VSAN as a
trunk interface. port trunk vsan vsan-id
By default, a VFC interface is not assigned to any VSAN as a trunk interface.
You can assign a VFC interface to a nonexistent VSAN as a trunk interface and then create the VSAN.
6. (Optional.) Configure a description for the VFC
interface. descriptiontext
By default, the description of an interface is Interface name Interface, for example, Vfc1 Interface.
Step Command Remarks 7. (Optional.) Restore the
default settings for the VFC
interface. default N/A
8. Bring up the VFC interface. undo shutdown By default, a VFC interface is up.
Enabling FCoE for a VLAN and mapping a VSAN
to the VLAN
When you use a VFC interface to transmit packets, the Ethernet interface bound to the VFC interface may allow multiple VLANs. You should enable FCoE for a VLAN and map a VSAN to the VLAN, so that the packets from the VSAN are tagged with the VLAN tag and transmitted within the VLAN.
Configuration restrictions and guidelines
Follow these restrictions and guidelines when you configure this feature: • FCoE cannot be enabled for VLAN 1.
• VSANs are mapped to VLANs on a one-to-one basis.
• You must enable FCoE for the same VLAN and map this VLAN to the same VSAN on the two ends. • Make sure the Ethernet interface bound to the VFC interface allows the FCoE-capable VLAN. After you enable FCoE for a VLAN, the following changes apply to the VLAN:
• An FCoE-capable VLAN allows only FCoE traffic.
• All member ports in an FCoE-capable VLAN are isolated. For this reason, a Layer 2 protocol enabled in the FCoE-capable VLAN runs based on the port isolation topology.
Configuration procedure
To enable FCoE for the specified VLAN and map this VLAN to the specified VSAN:
Step Command Remarks
1. Enter system view. system-view N/A
2. Enter VLAN view. vlan vlan-id N/A
3. Enable FCoE for the specified VLAN and map this VLAN to the specified VSAN.
fcoe enable [ vsan
vsan-id ]
By default, FCoE for a VLAN is disabled. Make sure that the VSAN to be mapped has been created.
Configuring the FC-MAP value
The FC-MAP value identifies an FCoE network. Switches in the same FCoE network must have the same FC-MAP value.
IMPORTANT:
After FC-MAP values are configured, VFC interfaces perform a renegotiation. The same FC-MAP value is required for two VFC interfaces to negotiate successfully.
To configure an FC-MAP value:
Step Command Remarks
1. Enter system view. system-view N/A
2. Configure an FC-MAP
value. fcoe fcmap fc-map The default setting is 0x0EFC00.
Configuring the FKA advertisement period value
Role of the FKA advertisement period value
The FKA advertisement period value functions as follows:
• After setting up a virtual link with a peer switch, a switch sends unsolicited Discovery Advertisements every FKA advertisement period on its VFC interfaces in E mode to maintain the established virtual link. The FKA advertisement period value is carried in unsolicited Discovery Advertisements. After receiving an unsolicited Discovery Advertisement, the peer switch maintains the status of the virtual link and records the FKA advertisement period value. If the peer switch fails to receive an unsolicited Discovery Advertisement within 2.5 FKA advertisement periods, it deletes the virtual link.
• After setting up a virtual link with a peer ENode, a switch sends unsolicited Discovery Advertisements every FKA advertisement period on its VFC interfaces in F mode to maintain the established virtual link. The FKA advertisement period value is carried in unsolicited Discovery Advertisements. After receiving an unsolicited Discovery Advertisement, the peer ENode maintains the status of the virtual link and records the FKA advertisement period value. If the peer ENode fails to receive an unsolicited Discovery Advertisement within 2.5 FKA advertisement periods, it deletes the virtual link. In addition, the ENode sends keepalive frames to the switch every FKA advertisement period value (this value is obtained from unsolicited Discovery Advertisements received from the switch). After receiving a keepalive frame, the switch maintains the status of the virtual link. If the switch fails to receive a keepalive frame within 2.5 FKA advertisement periods, it deletes the virtual link.
• The same as ENodes, VFC interfaces in NP mode use the FKA advertisement period value learned from the peer switch instead of that configured on the local switch.
Configuration restrictions and guidelines
When you configure the FKA advertisement period value, follow these restrictions and guidelines: • As specified in FC-BB-5, the upper limit of the FKA advertisement period value is 90 seconds. The
switch allows a maximum FKA advertisement period value of 600 seconds. When the switch interoperates with servers, storage devices, or other vendors' switches, you cannot configure the FKA advertisement period to be greater than 90 seconds.
• In normal cases, use the default FKA advertisement period value (8 seconds). In the case of an active/standby switchover or ISSU reboot on an IRF member switch with subordinate switches, HP
recommends that you increase the FKA advertisement period to a value between 60 and 90 seconds to avoid service interruption if there are many configurations on the switch. For more information about ISSU, see Fundamentals Configuration Guide.
• Use values greater than 90 in the case of an ISSU reboot on a switch operating in standalone mode or in IRF mode but without subordinate switches. In this case, the switch cannot send unsolicited Discovery Advertisements or keepalive frames for a period of time. To prevent the peer from deleting the virtual link for failing to receive an unsolicited Discovery Advertisement and avoid service interruption, HP recommends that you set the FKA advertisement period to a value between 300 and 600 seconds.
• To ensure service continuity during an active/standby switchover or ISSU reboot on an NPV switch, you must also adjust the FKA advertisement period value on the upstream FCF switch. This is because the FKA advertisement period value configured on the NPV switch affects only its VFC interfaces in F mode and connected ENodes, and its VFC interfaces in NP mode use the FKA advertisement period value learned from the upstream FCF switch.
FCoE traffic will be interrupted during an ISSU reboot on an access FCF switch or NPV switch operating in standalone mode or in IRF mode but without subordinate switches. This is because an access FCF switch or NPV switch will connect to servers, storage devices, or other vendor's switches and its FKA advertisement period cannot be longer than 90 seconds. An ISSU reboot on a switch operating in standalone mode or in IRF mode but without subordinate switches takes a longer time than 225 (2.5*90) seconds. Therefore, FCoE traffic will be interrupted during the ISSU reboot because the virtual link is deleted after the peer fails to receive an unsolicited Discovery Advertisement within 225 seconds. To configure an FKA advertisement period value:
Step Command Remarks
1. Enter system view. system-view N/A
2. Configure an FKA advertisement period value.
fcoe fka-adv-period
fka-adv-period The default setting is 8 seconds.
Configuring the FCF priority
The FCF priority includes the VFC interface FCF priority and the system FCF priority, which are used in the following scenarios:
• The VFC interface FCF priority is used in the fcf priority field in an unsolicited Discovery Advertisement.
• The system FCF priority is used in the fcf priority field in a solicited Discovery Advertisement. An ENode selects the FCF switch with the highest priority from the FCF switches sending Discovery Advertisements and sends a FLOGI request to it for login.
The FCF priority is effective only on a VFC interface connected to an ENode (VFC interface in F mode).
Configuring the system FCF priority
Step Command Remarks
Step Command Remarks 2. Configure the system
FCF priority. fcoe global fcf-priority priority
The default setting is 128.
The configuration takes effect on all VFC interfaces in F mode.
Configuring the VFC interface FCF priority
Step Command Remarks
1. Enter system view. system-view N/A
2. Enter VFC interface
view. interface vfc interface-number N/A
3. Configure the FCF priority for the VFC
interface. fcoe fcf-priority priority
The default setting is 128.
The configuration takes effect on a VFC interface only when it operates in F mode.
Displaying and maintaining VFC interfaces and FIP
Execute display commands in any view.Task Command
Display VFC interface information. display interface [ description ] ] [ vfc [ interface-number ] ] [ brief Display FCoE global configuration. display fcoe
Clear the statistics for VFC interfaces. reset counters interface [ vfc [ number ] ]
VFC interfaces and FIP configuration example
Network requirements
As shown in Figure 12, use the FCoE solution in a data center combining a LAN and a SAN to reduce the number of devices, network adapters, and cables.
Figure 12Network diagram
If the FCF switch is connected to the converged network adapter (CNA) of the server or storage device, PFC and DCBX should be configured additionally on the physical interfaces. For information about configuring PFC and DCBX, see Layer 2—LAN Switching Configuration Guide.
Configuration procedure
This section describes the configurations for VFC interfaces and FIP on the FCF switch. 1. Configure Switch A:
# Create VSAN 10 and configure Switch A to operate in FCF mode.
<SwitchA> system-view [SwitchA] fcoe-mode fcf [SwitchA] vsan 10 [SwitchA-vsan10] quit
# Create interface VFC 1, bind it to interface Ten-GigabitEthernet 1/0/1, and assign it to VSAN 10 as a trunk port.
[SwitchA] interface vfc 1
[SwitchA-Vfc1] bind interface ten-gigabitethernet1/0/1 [SwitchA-Vfc1] port trunk vsan 10
[SwitchA-Vfc1] quit
# Create interface VFC 2, configure it to operate in E mode, bind it to interface Ten-GigabitEthernet 1/0/2, and assign it to VSAN 10 as a trunk port.
[SwitchA] interface vfc 2 [SwitchA-Vfc2] fc mode e
[SwitchA-Vfc2] bind interface ten-gigabitethernet1/0/2 [SwitchA-Vfc2] port trunk vsan 10
[SwitchA-Vfc2] quit
# Configure interface Ten-GigabitEthernet 1/0/1 to allow VLAN 20.
[SwitchA] interface ten-gigabitethernet1/0/1
[SwitchA-Ten-GigabitEthernet1/0/1] port link-type trunk [SwitchA-Ten-GigabitEthernet1/0/1] port trunk permit vlan 20 [SwitchA-Ten-GigabitEthernet1/0/1] quit
[SwitchA] interface ten-gigabitethernet1/0/2
[SwitchA-Ten-GigabitEthernet1/0/2] port link-type trunk [SwitchA-Ten-GigabitEthernet1/0/2] port trunk permit vlan 20 [SwitchA-Ten-GigabitEthernet1/0/2] quit
# Enable FCoE for VLAN 20 and map it to VSAN 10.
[SwitchA] vlan 20
[SwitchA-vlan20] fcoe enable vsan 10
2. Configure Switch B:
# Create VSAN 10 and configure Switch B to operate in FCF mode.
<SwitchB> system-view [SwitchB] fcoe-mode fcf [SwitchB] vsan 10 [SwitchB-vsan10] quit
# Create interface VFC 1, configure it to operate in E mode, bind it to interface Ten-GigabitEthernet 1/0/2, and assign it to VSAN 10 as a trunk port.
[SwitchB] interface vfc 1 [SwitchB-Vfc1] fc mode e
[SwitchB-Vfc1] bind interface ten-gigabitethernet1/0/2 [SwitchB-Vfc1] port trunk vsan 10
[SwitchB-Vfc1] quit
# Configure interface Ten-GigabitEthernet 1/0/2 to allow VLAN 20.
[SwitchB] interface ten-gigabitethernet1/0/2
[SwitchB-Ten-GigabitEthernet1/0/2] port link-type trunk [SwitchB-Ten-GigabitEthernet1/0/2] port trunk permit vlan 20 [SwitchB-Ten-GigabitEthernet1/0/2] quit
# Enable FCoE for VLAN 20 and map it to VSAN 10.
[SwitchB] vlan 20
Setting up a fabric
Overview
A fabric transmits data for servers and disk devices. When setting up a fabric, you must assign a domain ID to each FCF switch in the fabric and assign an FC address to each node connected to the fabric. You can build a fabric through one of the following modes:
• Static mode—You must manually assign domain IDs to all switches in the network, and then each switch assigns FC addresses to the N_Ports connected to it. The static mode avoids network flappings, but it is applicable only to simple, small-sized networks.
• Dynamic mode—A principal switch is automatically elected to assign domain IDs to all switches in the network, and then each switch assigns FC addresses to the N_Ports connected to it. The dynamic mode enables centralized network management and is applicable to large-sized networks.
Figure 13Fabric setup workflows
The following section details each process in the fabric setup workflows.
Principal switch selection
During the dynamic fabric building process, it is the principal switch that assigns domain IDs to all switches in the network.
The switch with the highest priority is selected as the principal switch. When multiple switches have the same priority, the switch with the smallest WWN wins.
The principal switch selection process is as follows:
1. When the principal switch selection starts, each switch considers itself as the principal switch, records itself in the principal switch information, and starts the Principal Switch Selection Timer (PSST), which is 10 seconds.
2. Before the PSST times out, the switches exchange packets carrying the principal switch information to select a principal switch. On receiving such a packet, a switch compares the priority and WWN of the principal switch carried in the packet against those locally recorded. If the priority carried in the packet is higher, or the priority in the packet is the same and the WWN is smaller, the switch
replaces the locally-record principal switch information with the principal switch information recorded in the packet, and notifies the other switches. Finally, all switches in the network make an agreement on which switch is the principal switch.
3. When the PSST times out, if the locally-recorded principal switch information is the local switch, the switch becomes the principal switch.
After the principal switch is selected, the WWN of the principal switch becomes the fabric name.
NOTE:
During the principal switch selection process, if a switch receives a packet that updates the principal switch information, the switch must record the E_Port receiving the packet. The link relevant to this E_Port is called the "upstream principal link."
Domain ID assignment
A domain represents a switch and all N_Ports connected to the switch. Each domain must have a domain ID.
An FCF switch can automatically assign domain IDs. Alternatively, you can manually configure static domain IDs.
• If you manually configure static domain IDs, you must assign a unique domain ID to each switch in the fabric.
• If domain IDs are dynamically assigned, the fabric configuration process is performed to select a principal switch and assign domain IDs. After the principal switch is selected, the principal switch assigns domain IDs to all switches in the fabric. After the fabric configuration process, each switch has a unique domain ID.
The dynamic domain ID assignment process is as follows:
1. The principal switch assigns a domain ID to itself. If the principal switch has been configured with a desired domain ID, the principal switch assigns the domain ID to itself. Otherwise, the principal switch assigns the smallest available domain ID to itself. After the principal switch assigns a domain ID to itself, it notifies its downstream switches to request domain IDs from it.
2. After a downstream switch receives the notification, it starts to request a domain ID from the principal switch. If the downstream switch is configured with a desired domain ID, it requests the desired domain ID from the principal switch.
3. After the principal switch receives the domain ID request from the downstream switch, it assigns a domain ID to it and notifies the downstream switch. The principal switch assigns domain IDs following these rules:
{ If the downstream switch requests a desired domain ID that is available, the principal switch assigns the domain ID to the downstream switch.
{ If the downstream switch does not request a desired domain ID or the desired domain ID is not available, the principal switch assigns the smallest available domain ID to the downstream switch.
{ If all available domain IDs have been assigned, the principal switch notifies the downstream switch that no domain ID can be assigned to it.
4. After the downstream switch receives the domain ID assignment notification from the principal switch, it works following these rules:
{ If the downstream switch has been configured with a static domain ID and the static domain ID is different from the one assigned by the principal switch, or if the principal switch notifies the
downstream switch that no domain ID can be assigned, the downstream switch isolates its upstream principal link and brings down the relevant interface. For more information about domain ID types, see "Configuring a domain ID for a switch."
{ Otherwise, the downstream accepts the domain ID assigned by the principal switch and notifies the nearby downstream switch to request a domain ID from the principal switch. 5. Repeat steps 2 through 4 until all downstream switches have been assigned domain IDs.
NOTE:
During the dynamic domain ID assignment process, if a switch receives the domain ID request on an E_Port, the switch must record the E_Port. The link relevant to this E_Port is called the "downstream principal link."
FC address assignment
After a switch obtains a domain ID, it can assign FC addresses to N_Ports directly connected.
The Domain_ID field in the FC address is the domain ID of the switch, and it does not need assignment. The switch assigns the Area_IDs and Port_IDs following these guidelines:
• If you bind the WWN of an N_Port to an FC address, the switch assigns the bound FC address to the N_Port.
• If the N_Port itself has a desired FC address, the switch assigns the desired FC address, if available. • If the N_Port itself does not have a desired FC address or the desired FC address is unavailable, the
switch assigns the smallest available Area_ID and Port_ID to the N_Port.
Fabric setup configuration task list
When you set up a fabric, HP recommends that you use the same building mode (dynamic or static) for all switches in the fabric and then perform the following configurations depending on your building mode.
Building a fabric statically
Tasks at a glance Remarks
(Required.) Configuring VFC interfaces and FIP N/A (Required.) Enabling or disabling the fabric configuration
function To statically build a fabric, you must disable the fabric configuration function.
(Required.) Setting a fabric name
When statically building a fabric, you must manually configure the fabric name, and make sure all switches in the fabric are configured with the same fabric name. (Optional.) Configuring the allowed domain N/A
(Required.) Configuring a domain ID for a switch When statically building a fabric, you must manually configure a domain ID for each switch.
(Optional.) Configuring the mapping between the N_Port WWN and the FC address N/A
Tasks at a glance Remarks
(Optional.) Configuring the fabric timers N/A
Building a fabric dynamically
Tasks at a glance Remarks
(Required.) Configuring VFC interfaces and FIP N/A (Required.) Enabling or disabling the fabric configuration
function To dynamically build a fabric, you must enable the fabric configuration function. (Optional.) Setting the switch priority Principal switch selection relies on the switch priority.
(Optional.) Configuring the allowed domain N/A
(Optional.) Configuring a domain ID for a switch When dynamically building a fabric, you can configure desired domain IDs for switches.
(Optional.) Configuring the mapping between the N_Port WWN and the FC address N/A
(Optional.) Configuring the fabric timers N/A
(Optional.) Configuring the fabric reconfiguration N/A (Optional.) Configuring a VFC interface to reject incoming RCF
requests N/A
Enabling or disabling the fabric configuration
function
After being enabled with the fabric configuration function, FCF switches exchange messages to select the principal switch. Then the principal switch dynamically assigns domain IDs to all switches in the fabric. Therefore, to dynamically build a fabric, you must enable the fabric configuration function on switches. To statically set up a fabric, you must disable the fabric configuration function on switches and manually configure domain IDs for the switches.
To enable or disable the fabric configuration function:
Step Command Remarks
1. Enter system view. system-view N/A
2. Enter VSAN view. vsanvsan-id N/A
3. Enable the fabric configuration
function. domain configure enable Enable or disable the function for all switches in the VSAN as required. By default, the fabric configuration function is enabled.
4. Disable the fabric configuration
Setting a fabric name
The fabric name configured takes effect only on a statically-built fabric. You must configure the same fabric name for all switches in a VSAN.
To set a fabric name:
Step Command Remarks
1. Enter system view. system-view N/A
2. Enter VSAN view. vsanvsan-id N/A
3. Configure a fabric
name. fabric-namename
By default, the fabric name is null.
If the user does not configure a fabric name, the switch WWN is used as the fabric name after FCoE is enabled.
Setting the switch priority
The priority value for FCF switches is in the range of 1 to 254. The smaller the value, the higher the priority. The FCF switch with the highest priority will be elected as the principal switch.
The priority is configured on a per-VSAN basis, and one FCF switch may have different priorities in different VSANs.
In a stable fabric, the configured priority does not take effect immediately. Therefore, the running priority of a switch might be different from the configured priority. To validate the configured priority, use the domain restart disruptive command to perform a disruptive reconfiguration. After a disruptive reconfiguration, the running priority could still be different from the configured priority. See the following possibilities on the principal and a non-principal switch, depending on the configured priority value, as shown in Table 2.
Table 2Configured priority and running priority mappings
Configured priority Running priority
≦ 2 • Principal switch—Same as the configured priority.
• Non-principal switch—3.
﹥ 2 • Principal switch—2.
• Non-principal switch—Same as the configured priority. To set the switch priority:
Step Command Remarks
1. Enter system view. system-view N/A
2. Enter VSAN view. vsanvsan-id N/A
3. Configure a priority
Configuring the allowed domain ID list
Configuring the allowed domain ID list has an effect on switches as follows:• Principal switch—Can only assign domains IDs within the allowed domain ID list. If the allowed domain ID list configured does not include any of the already assigned domain IDs or manually configured domain IDs, the configuration will fail.
• Non-principal switch—The manually configured domain ID must be within the allowed domain ID list. Otherwise, the configuration will fail. The principal switch must assign the switch a domain ID within the allowed domain ID list. Otherwise, the switch refuses the assigned domain ID and isolates its interface connected to the principal switch. If the runtime domain ID for a switch is beyond the new allowed ID list, the configuration will also fail.
HP recommends that you specify the same allowed domain ID list for the member switches of a VSAN. To configure the allowed domain IDs for a switch:
Step Command Remarks
1. Enter system view. system-view N/A
2. Enter VSAN view. vsanvsan-id N/A
3. Configure the allowed
domain IDs for the switch. allowed-domain-iddomain-id-list By default, the allowed domain IDs are 1 to 239.
Configuring a domain ID for a switch
In different scenarios, the configured domain ID has different meanings.
• In a statically built fabric, the configured domain ID is the actual domain ID of the switch. • In a dynamically built fabric, the configured domain ID is desired by the switch but may not be the
actual domain ID.
To statically build a fabric, you must manually configure a domain ID for each switch.
To dynamically build a fabric, you can configure a desired domain ID for a switch, but the domain ID assigned to the switch may not be the desired one.
The configured domain ID can be of a static or preferred type. • In a statically built fabric, the two types make no difference.
• In a dynamically built fabric, when a non-principal switch fails to obtain the desired domain ID from the principal switch, the non-principal switch can use another domain ID assigned by the principal switch if the preferred type is configured. The non-principal switch will isolate the upstream link and refuse other domain IDs assigned by the principal switch if the static type is configured.
HP recommends that you configure domain IDs of the same type for all switches in a VSAN. To configure a domain ID for a switch:
Step Command Remarks
1. Enter system view. system-view N/A
Step Command Remarks 3. Configure a domain ID for
the switch. domain-idstatic } domain-id { preferred | By default, the domain ID of a switch is 0 and is of the preferred type.
Configuring the mapping between the N_Port
WWN and the FC address
If you bind the WWN of an N_Port to an FC address, when the N_Port requests an FC address, the switch assigns the bound FC address to it.
The WWN of an N_Port can be bound to only one FC address, and vice versa. The N-Port here indicates an N_Port on a node or an NP_Port on an NPV switch. To bind the WWN of an N_Port to an FC address:
Step Command Remarks
1. Enter system view. system-view N/A
2. Enter VSAN view. vsanvsan-id N/A
3. Bind the WWN of an
N_Port to an FC address. wwnarea-port-id-valuewwn-valuearea-port-id By default, no binding is configured.
Configuring the fabric timers
The fabric operation involves the following timers: • Distributed service timeout period
• Error detection timeout period • Resource allocation timeout period
For more information about these timers, see FC-related protocols and standards. You can configure fabric timers by using one of the following ways:
• Configure the timers in system view, and the configuration takes effect for all VSANs. • Configure the timers in VSAN view, and the configuration takes effect for only the VSAN.
If you perform the configuration in both system view and VSAN view, the configuration made in VSAN view applies to the VSAN.
Configuring the fabric timers in system view
Step Command Remarks
1. Enter system view. system-view N/A
2. Configure the global distributed service timeout
period. fc timer distributed-servicesvalue
By default, the distributed service timeout period is 5000
Step Command Remarks 3. Configure the global error
detection timeout period. fc timer error-detectvalue
By default, the error detection timeout period is 2000 milliseconds.
4. Configure the global resource allocation timeout
period. fc timer resource-allocationvalue
By default, the resource allocation timeout period is 10000
milliseconds.
Configuring the fabric timers in VSAN view
Step Command Remarks
1. Enter system view. system-view N/A
2. Enter VSAN view. vsanvsan-id N/A
3. Configure the distributed service timeout period for
the VSAN. timer distributed-servicesvalue
By default, the distributed service timeout period is 5000 milliseconds.
4. Configure the error detection timeout period
for the VSAN. timer error-detectvalue
By default, the error detection timeout period is 2000 milliseconds.
5. Configure the resource allocation timeout period
for the VSAN. timer resource-allocationvalue
By default, the resource allocation timeout period is 10000
milliseconds.
Configuring the fabric reconfiguration
IMPORTANT:
The fabric reconfiguration function takes effects only when the fabric configuration function is enabled. The fabric reconfiguration occurs in the case of a network reconstruction (for example, two fabrics are merged) or external intervention (for example, the administrator uses a command to initiate reconfiguration). The fabric reconfiguration triggers principal switch selection, domain ID assignment, and FC address assignment throughout the fabric.
The fabric reconfiguration includes the following categories:
• Disruptive reconfiguration—Floods the Reconfigure Fabric (RCF) frames throughout the fabric, and notifies all switches to perform a disruptive reconfiguration. During the reconfiguration procedure, each switch clears all data for renegotiation, and data transmission in the fabric is disrupted. • Non-disruptive reconfiguration—Floods the Build Fabric (BF) frames throughout the fabric, and
notifies all switches to perform a non-disruptive reconfiguration. During the reconfiguration procedure, each switch tries to save the last running data for its domain ID to remain unchanged. Thus, data transmission in the fabric is not disrupted.
Depending on the triggering conditions, the fabric reconfiguration falls into auto reconfiguration and manual reconfiguration.
• When two fabrics are merged:
{ The system automatically performs a non-disruptive reconfiguration if the principal switch information of the two fabrics is different and the domain ID lists are not empty or overlapping. { You can manually initiate a disruptive reconfiguration to trigger the fabric reconfiguration if
ports are isolated and priority values of switches are modified.
• When the principal switch in a fabric is down, the system automatically performs a non-disruptive reconfiguration.
Configuring the auto fabric reconfiguration function
Step Command Remarks
1. Enter system view. system-view N/A
2. Enter VSAN view. vsanvsan-id N/A
3. Enable the auto fabric
reconfiguration function. domain auto-reconfigure enable By default, the auto fabric reconfiguration function is disabled.
Manually initiating the fabric reconfiguration
Step Command 1. Enter system view. system-view
2. Enter VSAN view. vsanvsan-id
3. Initiate the fabric reconfiguration. domain restart [ disruptive ]
Configuring a VFC interface to reject incoming RCF
requests
In a stable fabric, to avoid unnecessary disruptive reconfigurations, you can configure a VFC interface to reject the RCF requests received from a specific VSAN by replying with a reject message. With this feature, the interface on which an RCF request is received is isolated.
To configure a VFC interface to reject the received RCF requests:
Step Command Remarks
1. Enter system view. system-view N/A
2. Enter VFC interface view. interface vfc interface-number N/A
3. Configure the VFC interface to reject the
received RCF requests. fc domain rcf-reject vsanvsan-id
By default, a VFC interface does not reject the received RCF requests.
Displaying and maintaining a fabric
Execute display commands in any view.Task Command
Display the domain information of the specified
VSAN. display fc domain [ vsanvsan-id ]
Display the list of domain IDs dynamically allocated in
the specified VSAN. display fc domain-list [ vsanvsan-id ]
Display fabric timers. display fc timerresource allocation [ distributed-services ] [ vsanvsan-id ] | error-detect | Display node login information. display fc login [ vsanvsan-id ] [ count ]
Display the SCR table for an N_Port. display fc scr-table [ vsanvsan-id ]
Display name service database information. display fc name-service databasefcid ] ] [ verbose ] [ vsanvsan-id [ fcid Display ESS negotiation results. display fc ess [ vsanvsan-id ]
Static fabric building configuration example
Network requirements
As shown in Figure 14, use the static approach to build a fabric. Figure 14Network diagram
Configuration procedure
1. Configure Switch A:# Disable the fabric configuration function and configure Switch A to operate in FCF mode.
<SwitchA> system-view [SwitchA] fcoe-mode fcf [SwitchA] vsan 1
[SwitchA-vsan1] undo domain configure enable
# Configure a name for the fabric.
[SwitchA-vsan1] fabric-name 11:11:11:11:11:11:11:11
# Configure the domain ID as 1.
[SwitchA-vsan1] domain-id 1 static
Non-disruptive reconfiguration or isolating the switch may be performed. Continu e? [Y/N]:y
2. Configure Switch B:
# Disable the fabric configuration function and configure Switch B to operate in FCF mode.
<SwitchB> system-view [SwitchB] fcoe-mode fcf [SwitchB] vsan 1
[SwitchB-vsan1] undo domain configure enable
# Configure a name for the fabric.
[SwitchA-vsan1] fabric-name 11:11:11:11:11:11:11:11
# Configure the domain ID as 2.
[SwitchB-vsan1] domain-id 2 static
Non-disruptive reconfiguration or isolating the switch may be performed. Continu e? [Y/N]:y
Verifying the configurations
1. Verify the configurations on Switch A.[SwitchA-vsan1] display fc domain vsan 1 Domain Information of VSAN 1:
Running time information: State: Stable Switch WWN: 48:33:43:2d:46:43:1A:1A Fabric name: 11:11:11:11:11:11:11:11 Priority: 128 Domain ID: 1 Configuration information: Domain configure: Disabled
Domain auto-reconfigure: Disabled Fabric name: 11:11:11:11:11:11:11:11 Priority: 128
Domain ID: 1 (static)
Principal switch running time information: Priority: 128
No interfaces available.
The output shows that the domain configuration is complete and that the runtime domain ID of Switch A is 1.
2. Verify the configurations on Switch B.
[SwitchB-vsan1] display fc domain vsan 1 Domain Information of VSAN 1:
Running time information: State: Stable Switch WWN: 48:33:43:2d:46:43:1B:1B Fabric name: 11:11:11:11:11:11:11:11 Priority: 128 Domain ID: 2 Configuration information: Domain configure: Disabled
Domain auto-reconfigure: Disabled Fabric name: 11:11:11:11:11:11:11:11 Priority: 128
Principal switch running time information: Priority: 128
No interfaces available.
The output shows that the domain configuration is complete and that the runtime domain ID of Switch B is 2.
Dynamic fabric building configuration example
Network requirements
As shown in Figure 15, use the dynamic approach to build a fabric. Figure 15Network diagram
Configuration procedure
1. Configure Switch A:# Enable the fabric configuration function (optional, because this function is enabled by default), and configure Switch A to operate in FCF mode.
<SwitchA> system-view [SwitchA] fcoe-mode fcf [SwitchA] vsan 1
[SwitchA-vsan1] domain configure enable
# Enable FCoE for VLAN 10 and map it to VSAN 1.
[SwitchA] vlan 10
[SwitchA-vlan10] fcoe enable vsan 1 [SwitchA-vlan10] quit
# Configure the domain ID as 11.
[SwitchA-vsan1] domain-id 11 preferred
Non-disruptive reconfiguration or isolating the switch may be performed. Continu e? [Y/N]:y
[SwitchA-vsan1] quit
# Create interface VFC 1, bind it to interface Ten-GigabitEthernet 4/0/2, configure it to operate in E mode, and assign it to VSAN 1 as a trunk port.
[SwitchA] interface Vfc 1
[SwitchA-Vfc1] fc mode e
[SwitchA-Vfc1] port trunk vsan 1 [SwitchA-Vfc1] quit
# Configure other physical-to-virtual interface bindings and VLAN-to-VSAN mappings in the same way configure the preceding ones. (Details not shown.)
2. Configure Switch B:
# Configure physical-to-virtual interface bindings and VLAN-to-VSAN mappings in the same way as you configure them on Switch A. (Details not shown.)
# Enable the fabric configuration function (optional, because this function is enabled by default), and configure Switch B to operate in FCF mode.
<SwitchB> system-view [SwitchB] fcoe-mode fcf [SwitchB] vsan 1
[SwitchB-vsan1] domain configure enable
# Set the priority value to 1, so that Switch B can be selected as the principal switch.
[SwitchB-vsan1] priority 1
3. Configure Switch C:
# Enable the fabric configuration function (optional, because this function is enabled by default), and configure Switch C to operate in FCF mode.
<SwitchC> system-view [SwitchB] fcoe-mode fcf [SwitchC] vsan 1
[SwitchC-vsan1] domain configure enable
# Configure the domain ID as 13.
[SwitchC-vsan1] domain-id 13 preferred
Non-disruptive reconfiguration or isolating the switch may be performed. Continu e? [Y/N]:y
4. Configure Switch D:
# Enable the fabric configuration function (optional, because this function is enabled by default), and configure Switch D to operate in FCF mode.
<SwitchD> system-view [SwitchD] fcoe-mode fcf [SwitchD] vsan 1
[SwitchD-vsan1] domain configure enable
# Configure the domain ID as 14.
[SwitchD-vsan1] domain-id 14 preferred
Non-disruptive reconfiguration or isolating the switch may be performed. Continu e? [Y/N]:y
Verifying the configurations
1. Verify the configurations on Switch A:# Display the domain information of VSAN 1.
[SwitchA-vsan1] display fc domain vsan 1 Domain Information of VSAN 1:
Running time information: State: Stable Switch WWN: 48:33:43:2d:46:43:1A:1A Fabric name: 48:33:43:2d:46:43:1B:1B Priority: 128 Domain ID: 11 Configuration information: Domain configure: Enabled
Domain auto-reconfigure: Disabled Fabric name: 48:33:43:2d:46:43:1A:1A Priority: 128
Domain ID: 11 (preferred)
Principal switch running time information: Priority: 1
Path Interface Upstream Vfc 1
Downstream Vfc 2
The output shows that the domain configuration is complete and that the principal switch assigns domain ID 11 to Switch A.
# Display the domain ID list of VSAN 1.
[SwitchA-vsan1] display fc domain-list vsan 1 Domain list of VSAN 1:
Number of domains: 4 Domain ID WWN 0x01(1) 48:33:43:2d:46:43:1B:1B [Principal] 0x0b(11) 48:33:43:2d:46:43:1A:1A [Local] 0x0d(13) 48:33:43:2d:46:43:1C:1C 0x0e(14) 48:33:43:2d:46:43:1D:1D
The output shows that Switch B becomes the principal switch and assigns the smallest domain ID 1 to itself.
Configuring VSAN
Overview
The virtual storage area network (VSAN) technology breaks a physical SAN into multiple VSANs, and provides more secure, reliable, and flexible services.
Devices in a VSAN cannot get information about any other VSAN and devices in any other VSAN. Each VSAN performs the following operations independently: selecting a principal switch, assigning domain IDs, running routing protocols, maintaining routing table and FIB table, and providing services. The VSAN technology delivers the following benefits:
• Improved security—VSANs are isolated from each other.
• Improved adaptability—Each VSAN independently runs and provides services. Different VSANs can use the same address space so that network capacity is improved.
• Flexibility—You can assign interfaces to different VSANs without changing the physical connections of the SAN.
VSAN fundamentals
VFC interfaces can only work as trunk ports. A trunk port can belong to multiple VSANs.
Trunk VSAN in an FC network
The trunk VSAN technology implements logical isolation among VSANs. The trunk VSAN works as follows: The trunk VSAN adds a Virtual Fabric Tagging Header (VFT_Header, also known as VSAN tag) to the FC frames. The VFT_Header contains a VF_ID (also known as "VSAN ID") field to indicate the VSAN of the FC frames. In this way, FC frames within different VF_IDs are contained in their respective VSANs, and different VSANs cannot communicate with each other. VSAN tags are added to and removed from an FC frame during transmission. A switch supports multiple VSANs one physical interface, thus reducing physical connections and implementing logical isolation in a physically connected SAN. Figure 16 shows a typical trunk VSAN. The F_Ports in blue on switches are configured as trunk ports and assigned to VSAN 1, and the F_Ports in purple are configured as trunk ports and assigned to VSAN 2. The E_Ports are configured with trunk VSANs 1 and 2.
When servers read the disks, the N_Ports of different servers send FC frames without VFT_Headers to the F_Ports on FC switch Switch A. Switch A searches for the outgoing interfaces in the FIB table of the VSAN that each F_Port belongs to. These F_Ports use the same E_Port as the outgoing interface. When the frames are forwarded out of the E_Port, they are tagged with the VFT_Header of VSAN 1 and VSAN 2 and travel across multiple VSAN-capable switches to the E_Port of FC switch Switch B.
According to the VFT_Headers, Switch B searches for the outgoing interfaces in the FIB tables of the VSANs, and forwards them to the F_Ports. Then, the F_Ports remove the VFT_Headers and send the frames to the N_Ports of different disk devices. The frames from the disk devices to the server are processed in the same way and finally reach the servers.