Brocade VCS Fabric Technology and NAS with NFS Validation Test

(1)

DATA CENTER

Brocade VCS Fabric Technology

and NAS with NFS Validation Test

NetApp/VMware vSphere 5.0

Red Hat Enterprise Linux

This material outlines sample configurations and associated test results

of Brocade

®

_VCS

®

_{Fabric technology with NFS file servers.}

(2)

1 PREFACE

1.1 Overview

As per the Gartner 2011 NAS Magic Quadrant report, the midrange and high-end Network-Attached Storage (NAS) market for 2010 experienced a growth rate of 33 percent over 2009 in terms of hardware vendor revenue. This favorable growth rate was a result of several factors: fast-growing unstructured file data, widespread availability of data de-duplication/compression in NAS storage solutions, ease of management, support of virtualized

environments such as VMware, and the flexibility of unified storage. 1

Additionally, IDC predicts that by 2014 more than 83 percent of enterprise storage system capacity will be shipped for file-based data, taking the Compound Annual Growth Rate (CAGR) for file serving storage capacity to 2.5 times the CAGR for block storage capacity2_.

As also discussed in the Gartner 2011 NAS Magic Quadrant report, NAS support of the VMware environment has become more prominent in the past year, as more and more NAS vendors have invested in this area to increase the appeal of their products. Additionally, NAS products use industry-standard remote file protocols, including Network File System (NFS).

The Gartner report also observes that because some applications, such as Oracle database applications and VMware, are built on files instead of blocks, NAS is increasingly used as application storage for those environments, providing ease-of-use benefits to users as compared with storage arrays that use block protocols, which may offer higher performance than NAS. As a result, many midrange and high-end NAS products are used to consolidate storage for both server applications and home directories for PC clients.

Given the tremendous growth rates in NAS and the large number of existing deployments, this presents an opportunity to demonstrate that Brocade®_VCS®_{Fabric technology interoperates with the underlying NFS protocol}

used by these systems.

1.2 Purpose of This Document

This document provides the validation of Brocade VCS Fabric technology with two implementations of the Network File System (NFS) Protocol, including both the NetApp FAS3050 NFS filer and Red Hat Enterprise Linux configured as an NFS server. This validation demonstrates that existing deployments using NFS will interoperate with Brocade VCS fabrics and exhibit resiliency to failure scenarios. This ensures that inputs/outputs (I/Os) between clients and servers operate in a non-disruptive manner.

The testing demonstrates NFS interoperability with Brocade VCS Fabric technology, while providing sample configurations and test results associated with fabric failover scenarios. This document should provide peace of mind for both network and storage administrators and architects who are already using NFS and are considering the use of Brocade VCS Fabric technology.

1.3 Audience

The content in this document is written for a technical audience, including solution architects, solutioneers, system engineers, and technical development representatives. This document assumes the audience is familiar with Brocade VCS Fabric technology.

1.4 Objectives

The objectives of this document are to evaluate NFS protocol interoperability with Brocade VCS Fabric technology in the following two test cases:

Test Case #1 with the NetApp FAS3050: This test consists of a 6-node Brocade VCS fabric with 2 ESXi hosts using Brocade 1020 FCoE (Fiber Channel over Ethernet) CNAs and the NetApp FAS3050. For this test, the Virtual Machine (VM) data store associated with the ESXi cluster resides on a volume of the NetApp FAS3050. Iometer is used as a measurement and characterization tool for this test.

(5)

Test Case #2 with Red Hat Enterprise Linux NFS server: This test consists of a 4-node Brocade VCS fabric with a Red Hat Enterprise Linux NFS server. For this test, Spirent is used as a characterization tool for the NFS clients. Spirent emulates many NFS clients accessing the Red Hat Enterprise Linux NFS server share. Each of the Spirent-emulated NFS clients mounted the NFS share of the Red Hat Enterprise Linux NFS server.

Both tests demonstrate that the topology of the network can change, with limited implication to the application, using NFS as an underlying protocol. Please note that this does not include a full configuration or validation of specific features on ESXi, Red Hat, NetApp, and VMware.

1.5 Brocade VCS Features

The following Brocade VCS features are used in the validation testing for both test cases. These features are considered best practices when utilizing NFS over a Brocade VCS fabric.

Please refer to Appendices A and B for the actual configuration procedures for these features.

1.5.1 Brocade Inter-Switch Link (ISL) Trunks

For both Test Case #1 and Test Case #2, Brocade Inter-Switch Link (ISL) Trunking is used within the Brocade VCS fabric to provide additional redundancy and load balancing between the NFS clients and NFS server.

Typically, multiple links between two switches are bundled together in a Link Aggregation Group (LAG) to provide redundancy and load balancing. Setting up a LAG requires lines of configuration on the switches and selecting a hash-based algorithm for load balancing based on source-destination IP or MAC addresses. All flows with the same hash traverse the same link, regardless of the total number of links in a LAG. This might result in some links within a LAG, such as those carrying flows to a storage target, being overutilized and packets being dropped, while other links in the LAG remain underutilized.

Instead of LAG-based switch interconnects, Brocade VCS Ethernet fabrics automatically form ISL trunks when multiple connections are added between two Brocade VDX®_{switches. Simply adding another cable increases}

bandwidth, providing linear scalability of switch-to-switch traffic, and this does not require any configuration on the switch. In addition, ISL trunks use a frame-by-frame load balancing technique, which evenly balances traffic across all members of the ISL trunk group.

1.5.2 Equal-Cost Multipath (ECMP)

A standard link-state routing protocol that runs at Layer 2 determines if there are Equal-Cost Multipaths (ECMPs) between RBridges in an Ethernet fabric and load balances the traffic to make use of all available ECMPs. If a neighbor switch is reachable via several interfaces with different bandwidths, all of them are treated as “equal-cost” paths. While it is possible to set the link cost based on the link speed, such an algorithm complicates the operation of the fabric. Simplicity is a key value of Brocade VCS Fabric technology, so an implementation is chosen in the test cases that does not consider the bandwidth of the interface when selecting equal-cost paths. This is a key feature needed to expand network capacity, to keep ahead of customer bandwidth requirements.

1.5.3 Virtual Link Aggregation Group (vLAG)

For both Test Case #1 and Test Case #2, Virtual Link Aggregation Groups (vLAGs) are used for the ESXi hosts, the NetApp FAS3050, and the Red Hat Enterprise Linux NFS server. In the case of the NetApp FAS3050, a dynamic Link Aggregation Control Protocol (LACP) vLAG is used. In the case of both ESXi hosts and the Red Hat Enterprise Linux NFS server, static LACP vLAGs are used.

While Brocade ISLs are used as interconnects between Brocade VDX switches within a Brocade VCS fabric, industry-standard LACP LAGs are supported for connecting to other network devices outside the Brocade VCS fabric. Typically, LACP LAGs can only be created using ports from a single physical switch to a second physical switch. In a Brocade VCS fabric, a vLAG can be created using ports from two Brocade VDX switches to a device to which both VDX switches are connected. This provides an additional degree of device-level redundancy, while providing active-active link-level load balancing.

(6)

1.5.4 Pause Flow Control

For these test cases, Pause Flow Control is enabled on vLAG-facing interfaces connected to the ESXi hosts, the NetApp FAS3050, and the Red Hat Enterprise Linux NFS server. Brocade VDX Series switches support the Pause Flow Control feature. IEEE 802.3x Ethernet pause and Ethernet Priority-Based Flow Control (PFC) are used to prevent dropped frames by slowing traffic at the source end of a link. When a port on a switch or host is not ready to receive more traffic from the source—perhaps due to congestion—it sends pause frames to the source to pause the traffic flow. When the congestion is cleared, the port stops requesting the source to pause traffic flow, and traffic resumes without any frame drop. When Ethernet pause is enabled, pause frames are sent to the traffic source. Similarly, when PFC is enabled, there is no frame drop; pause frames are sent to the source switch.

For configuration details on Ethernet Pause and PFC on Brocade VDX Series switches, please refer to Appendix A.

1.5.5 Ultra-Low Latency

The Brocade VDX series of switches provides industry-leading performance and ultra-low latency through wire-speed ports with 600 nanosecond port-to-port latency and hardware-based Brocade ISL Trunking. This is helpful for environments that require high availability, such as providing Ethernet storage connectivity for FCoE, Internet Small Computer Systems Interface (iSCSI), and NAS.

1.5.6 Jumbo Frames

Brocade VDX Series switches support the transport of jumbo frames. Jumbo frames are enabled by default on the Brocade ISL trunks. However, to accommodate end-to-end jumbo frame support on the network for the edge hosts, this feature can be enabled under the vLAG interface connected to the ESXi hosts, the NetApp FAS3050, and Red Hat Enterprise Linux NFS server. The default Maximum Transmission Unit (MTU) on these interfaces is 2500. This MTU is set to 9216 to optimize the network for jumbo frame support.

For additional configuration details of jumbo frames on Brocade VDX Series switches, please refer to Appendix A. For additional details on best practices for enabling jumbo frames on the host devices that transmit the frames, please reference the NetApp and VMware vSphere Storage Best Practices guide:

http://communities.netapp.com/servlet/JiveServlet/previewBody/11657-102-1-22108/TR3749NetAppandVMwarevSphereStorageBestPracticesJUL10.pdf

1.6 Summary

These results support the validation that Brocade VCS Fabric technology interoperates with two implementations of the Network File System (NFS) Protocol, including both the NetApp FAS3050 NFS filer and the Red Hat Enterprise Linux NFS server.

This validation demonstrates that existing deployments using NFS interoperate with Brocade VCS fabrics and exhibit resiliency to failure scenarios. This ensures that I/Os between clients and servers operate in a non-disruptive manner.

1.7 Related Documents

For more information about Brocade VCS Fabric technology, please see the Brocade VCS Fabric Technical Architecture brief:

http://www.brocade.com/downloads/documents/technical_briefs/vcs-technical-architecture-tb.pdf

For the Brocade Network OS (NOS) Admin Guide and NOS Command Reference:

http://www.brocade.com/downloads/documents/product_manuals/B_VDX/NOS_AdminGuide_v211.pdf http://www.brocade.com/downloads/documents/product_manuals/B_VDX/NOS_CommandRef_v211.pdf

(7)

For more information about the Brocade VDX Series of switches, please see the product Data sheets:

• Brocade VDX 6710 Data Center Switch:

http://www.brocade.com/products/all/switches/product-details/vdx-6710-dc-switches/index.page • Brocade VDX 6720 Data Center Switch:

http://www.brocade.com/products/all/switches/product-details/vdx-6720-dc-switches/index.page • Brocade VDX 6730 Data Center Switch:

http://www.brocade.com/products/all/switches/product-details/vdx-6730-dc-switches/index.page

1.8 About Brocade

As information becomes increasingly mobile and distributed across the enterprise, organizations are transitioning to a highly virtualized infrastructure, which often increases overall IT complexity. To simplify this process, organizations must have reliable, flexible network solutions that utilize IT resources whenever and wherever needed—enabling the full advantages of virtualization and cloud computing.

As a global provider of comprehensive networking solutions, Brocade has more than 15 years of experience in delivering Ethernet, storage, and converged networking technologies that are used in the world’s most mission-critical environments. Based on the Brocade One strategy, this unique approach reduces complexity and disruption by removing network layers, simplifying management, and protecting existing technology investments. As a result, organizations can utilize cloud-optimized networks to achieve their goals of non-stop operations in highly virtualized infrastructures where information and applications are available anywhere.

(8)

2. TEST CASE #1: NETAPP FAS3050

For this test, one type of NAS storage is used. NetApp FAS3050 is used as a NAS server. This storage option is commonly deployed when using NAS storage pools with VMware vSphere.

Test Case #1 with the NetApp FAS3050 consists of a 6-node Brocade VCS fabric with 2 ESXi hosts using Brocade 1020 CNAs and NetApp FAS3050. For this test, the VM data store associated with the ESXi cluster resides on a volume of the NetApp FAS3050. Iometer is used as a measurement and characterization tool for this test. The demonstration shows that the topology of the network can change with limited implication to the application using NFS as an underlying protocol. Please note that this does not include a full configuration or validation of specific features on ESXi, Red Hat, NetApp, and VMware. Lastly, please note that for this test case, the NetApp FAS3050 used is limited to Gigabit Ethernet (GbE) interfaces. Therefore, a Brocade VDX 6710 was used to extend Gigabit Ethernet to both RB1 and RB2. Assuming the NetApp FAS3050 had 10 GbE interfaces, it would simply be connected to RB1 and RB2 without the need for RB21 and RB22.

2.1 Test Case #1 Summary

All tests for Test Case #1 were performed successfully, with no issues.

This validation demonstrates that existing deployments using NFS do interoperate with Brocade VCS fabrics and exhibit resiliency to I/Os between clients and servers in a non-disruptive manner. The following tests were conducted.

1. Test 1 validated baseline I/O using the shortest equal-cost paths in the fabric, with NFS clients on RB3 and RB4 accessing the NetApp FAS3050 on the vLAG interface on RB21 and RB22.

2. Test 2 validated that I/O continues between a VM using an NFS client accessing a storage pool on a NetApp FAS3050 when failing a link in the Brocade ISL trunk.

3. Test 3 validated that I/O continues between a VM using an NFS client accessing a storage pool on a NetApp FAS3050 when a complete path fails.

4. Test 4 validated that I/O flows are not impacted between a VM using an NFS client accessing a storage pool on a NetApp FAS3050 when a switch in the Brocade VCS fabric fails.

5. Test 5 validated that there was no impact throughout the duration of a successful vMotion between ESX servers acting as NFS clients with a link/path failure.

This summarizes the test results for Test Case #1.

Test Description Results

1 Baseline of shortest paths and traffic distribution in the Brocade VCS fabric

Pass

2 Perform link failure within a Brocade ISL trunk Pass 3 Perform a complete active path failure within the

Brocade VCS fabric

Pass

4 Perform multiple switch failures within the Brocade VCS fabric

Pass

5 Perform vMotion between ESX servers as NFS clients with switch failure

(9)

2.2 Topology

Figure 1. NetApp FAS3050 NFS validation topology and components for Test Case #1.

2.3 Hardware Resources

The following equipment was used in this configuration:

Description Quantity Revisions

Brocade VDX 6710-54 2 Brocade NOS 2.1.1

(10)

2.4 Compute Resources

VMware ESX 2 Intel Xeon X5670, 2.93 GHz 2

socket, 6 cores, 32 GB RAM

VMware ESX – Continued 2 Brocade 1020

VMware vSphere Management 1 Intel Xeon X3430, 2.4 GHz Quad cores, 8 GB RAM

2.5 Software Resources

The following software was used in this configuration:

Description Revision

Brocade NOS 2.1.1

NetApp Data ONTAP® _7.3.6

VMware vSphere 5.0.0

VMware ESX 5.0.0

2.6 Test 1: I/O Verification

The purpose of test 1 is to validate baseline I/O is using the shortest equal-cost paths in the fabric, with the NFS clients on RB3 and RB4 accessing the NetApp FAS3050 on the vLAG interface on RB21 and RB22.

(11)

Figure 2. NetApp FAS3050 NFS validation topology for test 1.

2.6.1 Test Procedure

Step 1: Run Iometer on the Windows VM. Note that NIC teaming for vSwitch was configured for the “Route based on IP hash” policy. Please see Appendix A for details. Note that the I/O path shown with a green line shows valid ECMPs in the fabric between ESXi and the NetApp FAS3050. The actual flows that traverse these links vary, depending on the number of hosts, MAC addresses, and flows.

Step 2: Execute the following command on all Brocade VDX ISLs and VDX vLAG interfaces to confirm that traffic flow and I/O path is evenly distributed as expected. The following is an example:

VDX6710-RB21# do show interface port-channel 33 | in rate

2.6.2 Expected Results

The expected results for test 1 are to show distributed I/O due to an active/active NIC teaming vLAG end-to-end from ESX VM NFS clients to the NetApp server datastore.

2.6.3 Actual Results

The actual results for test 1 confirm evenly distributed I/O due to an active/active NIC teaming vLAG end-to-end from ESX VM NFS clients to the NetApp server datastore.

(12)

2.7 Test 2: Link Failure

The purpose of test 2 is to validate that I/O flows are not impacted between the NFS client VM and NetApp FAS3050 while introducing a link failure on one link in the Brocade trunk, which is an active shortest path in the fabric. The links that are failed as part of this test are highlighted below in red.

(13)

The NFS clients on RB3 and RB4 continue to access the NetApp FAS3050 on the active/active vLAG interface on RB21 and RB22.

Step 1: Run Iometer on the Windows VM for the duration of the test.

Step 2: Execute the following command on the desired Brocade VDX switches to validate baseline traffic prior to failure. The following is an example:

VDX6730-RB3# show in te 3/0/1 | in rate VDX6730-RB3# show in te 3/0/2 | in rate

Step 3: Now that active traffic on both Brocade ISLs is confirmed, an ISL in the trunk is failed. In this example, Interface TenGbE3/0/1 is failed, which is one of the active ISLs in the trunk between RB3

and RB1. VDX6730-0

RB3(config)# in te 3/0/1

VDX6730-RB3(conf-if-te-3/0/1)# shut

Step 4: Execute the following command on the desired Brocade VDX switches to observe traffic levels after interface failure. The following is an example:

Step 5: Execute the following command on the desired Brocade VDX switches to re-enable the failed ISL: VDX6730-RB3# conf

Entering configuration mode terminal VDX6730-RB3(config)# in te 3/0/1 VDX6730-RB3(conf-if-te-3/0/1)# no shut

Step 6: Execute the following command on the desired Brocade VDX switches to observe traffic levels after interface re-enabling. The following is an example:

Repeat Steps 1–6: Steps 1–6 were also performed for the other Brocade ISLs that are highlighted in red under section 2.7.

The expected results for test 2 are to confirm that distributed I/O is not impacted while failing an active Brocade member ISL within the fabric.

The NFS clients on RB3 and RB4 continue to access the NetApp FAS3050 on the active/active vLAG interface on RB21 and RB22.

(14)

All I/Os from the NFS client on RB3 and RB4 fail over to the remaining link on the Brocade trunks going into RB21 and RB22, where the active/active vLAG is located on the NFS server. Since the trunks have 20 Gbps of aggregated bandwidth, there should be no loss of throughput, unless there is more than 10 Gbps of I/O going through the fabric when a single 10 Gbps link is failed during the test. When the links in the trunk are re-enabled, the I/Os get evenly redistributed to all of the active links in the trunk.

The actual results for test 2 confirm that distributed I/O is not impacted while failing an active Brocade member ISL within the fabric.

During the link failure in the Brocade trunk, while I/O was evenly distributed between the two 10 GbE links on the 20 GbE trunk, all I/O was immediately switched to the remaining 10 GbE link. When the link was re-enabled, all I/O was immediately rebalanced between both links again.

The same results were observed between other Brocade ISL trunk link failures in the fabric. These failures were performed for the other Brocade ISLs, highlighted in red under section 2.6.5.

Again, there was no disruption of I/O from the perspective of the Windows VM/NFS client. The maximum throughput of 119 MB/sec was the same before, during, and after the link failure.

2.8 Test 3: Active Path Failure

The purpose of test 3 is to validate that I/O flows are not impacted between the NFS client VM and NetApp FAS3050 while introducing a complete active path failure. This is done by failing both interfaces in the Brocade trunk, which is an active shortest path in the fabric. The links that are failed as part of this test are highlighted below in red. Observe that all I/Os get rerouted via fabric switching in the remaining paths.

(15)

Step 2: Execute the following command on RB3 to validate fabric topology prior to path failure.

VDX6730-RB3# show fabric route topology

Total Path Count: 8

Src Dst Out Out Nbr Nbr

RB-ID RB-ID Index Interface Hops Cost Index Interface BW Trunk --- 3 1 2 Te 3/0/2 1 500 7 Te 1/0/7 20G Yes 2 6 Te 3/0/6 1 500 7 Te 2/0/7 20G Yes 4 2 Te 3/0/2 2 1000 7 Te 1/0/7 20G Yes 4 6 Te 3/0/6 2 1000 7 Te 2/0/7 20G Yes 21 2 Te 3/0/2 2 1000 7 Te 1/0/7 20G Yes 21 6 Te 3/0/6 2 1000 7 Te 2/0/7 20G Yes 22 6 Te 3/0/6 2 1000 7 Te 2/0/7 20G Yes 22 2 Te 3/0/2 2 1000 7 Te 1/0/7 20G Yes

(16)

In the base testing, it is known that the NFS clients on RB3 and RB4 take the shortest path to the NFS server. In the fabric topology prior to the path failure, note that total path count is 8, and from RB3 to RB1, note that the hop count is 1 and the cost is 500.

Step 3: In this example, interfaces TenGigE3/0/1 and TenGigE3/0/2 are failed, to fail the active path that includes both active ISLs in the trunk between RB3 and RB1.

VDX6730-RB3(config)# in te 3/0/1 VDX6730-RB3(conf-if-te-3/0/1)# shut

VDX6730-RB3(conf-if-te-3/0/1)# in te 3/0/2 VDX6730-RB3(conf-if-te-3/0/2)# shut

Step 4: Execute the following command on RB3 to observe fabric topology after path failure: VDX6730-RB3# show fabric route topology

RB-ID RB-ID Index Interface Hops Cost Index Interface BW Trunk --- 3 1 6 Te 3/0/6 2 1000 7 Te 2/0/7 20G Yes 2 6 Te 3/0/6 1 500 7 Te 2/0/7 20G Yes 4 6 Te 3/0/6 2 1000 7 Te 2/0/7 20G Yes 21 6 Te 3/0/6 2 1000 7 Te 2/0/7 20G Yes 22 6 Te 3/0/6 2 1000 7 Te 2/0/7 20G Yes

VDX6710-RB21# show interface Port-channel 33 | in rate

Step 6: Execute the following command on the desired Brocade VDX switches to re-enable the failed ISL: VDX6730-RB3# conf

Entering configuration mode terminal VDX6730-RB3(config)# in te 3/0/1 VDX6730-RB3(conf-if-te-3/0/1)# no shut VDX6730-RB3(config)# in te 3/0/2 VDX6730-RB3(conf-if-te-3/0/2)# no shut

(17)

VDX6710-RB22# show interface Port-channel 33 | in rate

The expected results for test 3 are to confirm that distributed I/O is not impacted while introducing a complete active path failure. It should be observed that all I/Os are rerouted via fabric switching in the remaining paths.

The NFS client on RB3 and RB4 should continue to access the NetApp FAS3050 on the active vLAG interface on RB21 and RB22. This test will show the resiliency of the fabric.

In this example, the shortest path on the left side from RB3 to RB21 will go down between RB1 and RB3. Through Brocade VCS Ethernet fabric switching, all of the I/Os are redirected through RB2 to RB1, in order to get to RB21.

The actual results for test 3 confirm that distributed I/O is not impacted while introducing a complete active path failure. It is observed that all I/Os are rerouted via fabric switching in the remaining paths.

As expected, during the shortest path failure on the left side from the NFS client to the NetApp FAS3050, all I/Os were immediately switched over to the alternate path. Again, there was no disruption of I/O from the perspective of the Windows VM/NFS client.

From the perspective of the Iometer application, there were no errors, and all I/Os continued to flow through the fabric. Upon re-enabling the shortest path, all I/Os failed back to the shortest path between RB3 and RB1 without any disruption.

2.9 Test 4: Switch Failure

The purpose of test 4 is to validate that I/O flows are not impacted between the NFS client VM and NetApp FAS3050 while introducing multiple switch failures in the active path of the Brocade VCS fabric. The first switch to be reloaded, RB2, is the principal switch in the Brocade VCS fabric. The second switch to be reloaded, RB3, is a switch directly connected to an active vLAG interface of the NFS client. For this test, all MACs were hashed onto the GbE 0/c of the NetApp FAS3050.

(18)

Step 1: Validate that all switches in the Brocade VCS fabric are up, and the shortest paths between the NFS client and the server are being used as depicted in test 1.

Step 2: Execute the following command on the desired Brocade VDX switches to validate fabric topology prior to failure:

VDX6720-RB4# show fabric route topology

RB-ID RB-ID Index Interface Hops Cost Index Interface BW Trunk ---

4 1 2 Te 4/0/2 1 500 10 Te 1/0/10 20G Yes

2 5 Te 4/0/5 1 500 11 Te 2/0/11 20G Yes

3 5 Te 4/0/5 2 1000 11 Te 2/0/11 20G Yes 3 2 Te 4/0/2 2 1000 10 Te 1/0/10 20G Yes

(19)

21 2 Te 4/0/2 2 1000 10 Te 1/0/10 20G Yes 21 5 Te 4/0/5 2 1000 11 Te 2/0/11 20G Yes

22 5 Te 4/0/5 2 1000 11 Te 2/0/11 20G Yes 22 2 Te 4/0/2 2 1000 10 Te 1/0/10 20G Yes

Step 3: Execute the following command on any Brocade VDX switch to validate the existing principal switch in the fabric:

VDX6720-RB2# show fabric all VCS Id: 1

Config Mode: Local-Only

Rbridge-id WWN IP Address Name

--- 1 10:00:00:05:33:46:6A:8D 10.17.83.222 “VDX6720-RB1” 2 10:00:00:05:33:67:FE:A5 10.17.83.223 >“VDX6720-RB2”* 3 10:00:00:05:33:91:3A:E4 10.17.83.224 “VDX6730-RB3” 4 10:00:00:05:33:67:CA:BC 10.17.82.237 “VDX6720-RB4” 21 10:00:00:05:33:8C:AD:91 10.17.84.31 “VDX6710-RB21” 22 10:00:00:05:33:8C:D3:FD 10.17.84.77 “VDX6710-RB22”

The Fabric has 6 Rbridge(s)

Step 5: Now that it is confirmed that the principal switch is RB2, the disabling of the RB2 switch can proceed. VDX6720-RB2# reload

Warning: Unsaved configuration will be lost. Are you sure you want to reload the switch? [y/n]:y The system is going down for reload NOW !!

Broadcast message from root Wed Jan 18 18:44:57 2012... The system is going down for reboot NOW !!

Step 6: Execute the following command on the desired Brocade VDX switches to validate the fabric topology after RB2 failure:

--- 1 10:00:00:05:33:46:6A:8D 10.17.83.222 >“VDX6720-RB1” 3 10:00:00:05:33:91:3A:E4 10.17.83.224 “VDX6730-RB3” 4 10:00:00:05:33:67:CA:BC 10.17.82.237 “VDX6720-RB4”*

(20)

21 10:00:00:05:33:8C:AD:91 10.17.84.31 “VDX6710-RB21”

Step 7: Verify that I/Os are still running after principal switch RB2 failure:

Step 8: Disable the RB3 switch. VDX6730-RB3# reload

Broadcast message from root Wed Jan 18 18:51:54 2012... The system is going down for reboot NOW !!

Step 9: Execute the following command on the desired Brocade VDX switches to validate the fabric topology after RB3 failure:

--- 1 10:00:00:05:33:46:6A:8D 10.17.83.222 >“VDX6720-RB1” 4 10:00:00:05:33:67:CA:BC 10.17.82.237 “VDX6720-RB4”* 21 10:00:00:05:33:8C:AD:91 10.17.84.31 “VDX6710-RB21”

(21)

Step 10: Verify that I/Os are still running after principal switch RB3 failure:

VDX6710-RB21# show interface port-channel 33 | in rate

The expected results for test 4 are to confirm that distributed I/O is not impacted while introducing multiple switch failures within the active path of the Brocade VCS fabric. It should be observed that all I/Os are rerouted via fabric switching in the remaining paths.

During the failure of the switches in the fabric impacting both end-device vLAGs, all Iometer I/Os will continue to flow. This will show little to no disruption of I/Os during the simultaneous failure of the RB2 principal switch, as well as RB3. Since the NFS client has redundant connections on RB3 and RB4 on its vLAG, it gets continuous access to the NetApp FAS3050. In addition to that, the NetApp FAS3050 also has redundant connections on RB21 and RB22 on the vLAG. This test shows the resiliency of Brocade VCS Fabric technology and its end devices when vLAGs are used. Note that in this scenario, when the principal switch RB2 is rebooted, RB22 should also exit the fabric. When RB3 gets rebooted, there should be a total of three switches leaving the fabric, and I/O will continue to flow from the NFS client to the server.

The actual results for test 4 confirm that distributed I/O is not impacted while introducing multiple switch failures in the active path of the Brocade VCS fabric. It is observed that all I/Os are rerouted via fabric switching in the remaining paths.

As expected, during the failure of the principal switches RB2 and RB3, all Iometer traffic continued to flow between the NFS client and server. Brocade VCS Ethernet fabric switching utilized all available paths between the end devices and failed over I/O within sub-second convergence times. The vLAGs between the NFS server and NFS client to the

(22)

fabric had redundant connections without having to connect to the same physical switch. This resulted in high availability of application traffic between the NFS client and server, even during switch failures.

2.10 Test 5: vMotion with Link/Path Failure

The purpose of test 5 is to validate that I/O flows were not impacted throughout the duration of a successful vMotion between ESX servers acting as NFS clients with a link/path failure.

In this test scenario, there are two 10 GbE ESX servers dual connected to both RBridges 3 and 4 to the Brocade VCS fabric with NIC Teaming/vLAG, using LACP/Port-Channels 44 and 55. There are two virtual machines between the ESX servers, one RedHat Linux Enterprise 5 and the other a Windows 2008 Server, serving as NFS clients using the NetApp FAS3050 as the VM datastore.

For the NFS server, there is 1 GbE NetApp storage that is dual connected to the Brocade VCS fabric with NIC Teaming/vLAG using LACP/Port-Channel 33. See Figure 6.

(23)

Step 1: Validate that all switches in the Brocade VCS fabric are up and that the shortest paths between the NFS client and server are being used as depicted in test 1.

VDX6730-RB3# show fabric route topology Total Path Count: 8

RB-ID RB-ID Index Interface Hops Cost Index Interface BW Trunk --- 3 1 2 Te 3/0/2 1 500 7 Te 1/0/7 20G Yes 2 6 Te 3/0/6 1 500 7 Te 2/0/7 20G Yes 4 6 Te 3/0/6 2 1000 7 Te 2/0/7 20G Yes 4 2 Te 3/0/2 2 1000 7 Te 1/0/7 20G Yes 21 6 Te 3/0/6 2 1000 7 Te 2/0/7 20G Yes 21 2 Te 3/0/2 2 1000 7 Te 1/0/7 20G Yes 22 6 Te 3/0/6 2 1000 7 Te 2/0/7 20G Yes 22 2 Te 3/0/2 2 1000 7 Te 1/0/7 20G Yes

(24)

Step 3: Execute the following command on any Brocade VDX switch to validate the member RBridges in the fabric: VDX6730-RB3# show fabric all

VCS Id: 1

--- 1 10:00:00:05:33:46:6A:8D 10.17.83.222 >“VDX6720-RB1” 2 10:00:00:05:33:67:FE:A5 10.17.83.223 “VDX6720-RB2” 3 10:00:00:05:33:91:3A:E4 10.17.83.224 “VDX6730-RB3”* 4 10:00:00:05:33:67:CA:BC 10.17.82.237 “VDX6720-RB4” 21 10:00:00:05:33:8C:AD:91 10.17.84.31 “VDX6710-RB21” 22 10:00:00:05:33:8C:D3:FD 10.17.84.77 “VDX6710-RB22”

Step 4: Validate the vSphere client view of the VM NFS clients and NetApp FAS3050 datastore while on ESX server #2 (on the right of the figure) before vMotion and prior to RB4 failure. Note that from the ESX server #2 on the right, vmnic2 will be disabled when RB4 is failed.

This is the vSphere view prior to RB4 failure:

Step 5: On the Windows VM, send I/Os throughout the duration of the vMotion and prior to RB4 failure, using Iometer to read/write to the NFS server.

(25)

Step 6: On the Linux VM, send I/Os before the vMotion and prior to RB4 failure, using “dd” to read/write to the NFS server

.

(26)

Step 8: Proceed with disabling the RB4 switch. VDX6720-RB4# reload

Broadcast message from root Tue Jan 23 14:22:49 2012... The system is going down for reboot NOW !!

Step 9: Execute the following command on the desired Brocade VDX switches to validate fabric topology after RB4 failure:

--- 1 10:00:00:05:33:46:6A:8D 10.17.83.222 >“VDX6720-RB1” 2 10:00:00:05:33:67:FE:A5 10.17.83.223 “VDX6720-RB2” 3 10:00:00:05:33:91:3A:E4 10.17.83.224 “VDX6730-RB3”* 21 10:00:00:05:33:8C:AD:91 10.17.84.31 “VDX6710-RB21” 22 10:00:00:05:33:8C:D3:FD 10.17.84.77 “VDX6710-RB22”

Step 10: Execute the following command on any Brocade VDX switch to validate the member RBridges in the fabric after RB4 failure:

RB-ID RB-ID Index Interface Hops Cost Index Interface BW Trunk --- 3 1 2 Te 3/0/2 1 500 7 Te 1/0/7 20G Yes 2 6 Te 3/0/6 1 500 7 Te 2/0/7 20G Yes 21 6 Te 3/0/6 2 1000 7 Te 2/0/7 20G Yes 21 2 Te 3/0/2 2 1000 7 Te 1/0/7 20G Yes 22 6 Te 3/0/6 2 1000 7 Te 2/0/7 20G Yes 22 2 Te 3/0/2 2 1000 7 Te 1/0/7 20G Yes

(27)

Step 11: Validate the vSphere client view of the VM NFS clients and NetApp FAS3050 datastore during RB4 failure and vMotion:

Step 12: Validate the vSphere client view of the VM NFS clients and NetApp FAS3050 datastore during RB4 failure and after successful vMotion:

(28)

Step 13: Verify that I/Os are still running after switch RB4 failure—that the Windows VM is sending I/Os during vMotion and RB4 failure, using Iometer to read/write to the NFS server.

Step 14: Verify that I/Os are still running after switch RB4 failure—that the Linux VM is sending I/Os during vMotion and RB4 failure, using “dd” to write to the NFS server.

(29)

The expected results for test 5 are to confirm that distributed I/O is not impacted while introducing a switch failure within the active path of a Brocade VCS fabric while VM clients are sending I/Os to the NFS server; they will be moved to another ESX server using vMotion. During the vMotion, there will be a failure on RB4, but there will be no disruption of I/Os.

The actual results for test 5 confirm that distributed I/O is not impacted while introducing a switch failure in the active path of the Brocade VCS fabric while VM clients are sending I/Os to the NFS server; they are moved to another ESX server using vMotion. Before, during, and after RB4 failure and vMotion of both VMs, while they were accessing the NFS server, there was no disruption of I/O. Note that one “ping” from the Windows VM was lost due to an “in-flight” frame that was lost during the failure. It was observed that all I/Os were rerouted via fabric switching in the remaining paths.

(30)

3. TEST CASE #2: RED HAT ENTERPRISE LINUX NFS SERVER

The topology used for validation is shown below. For this test, one type of NAS storage was used, with Red Hat Enterprise Linux configured as an NFS server.

Test Case #2 with Red Hat Enterprise Linux NFS server: This test consists of a 4-node Brocade VCS fabric with a Red Hat Enterprise Linux NFS server. For this test, Spirent is used as a characterization tool for NFS clients. Spirent will emulate many NFS clients accessing the Red Hat Enterprise Linux NFS server share. Each of the NFS clients mounted the NFS share of the Red Hat Enterprise Linux NFS server.

The purpose is to demonstrate that the topology of the network can change with limited implication to the application using NFS as an underlying protocol. Please note that this does not include a full configuration or validation of specific features on ESXi, Red Hat, NetApp, and VMware. Also, please note that for this test case, the NetApp FAS3050 has Gigabit Ethernet interfaces. Therefore, a Brocade VDX 6710 was used to extend Gigabit Ethernet to both RB1 and RB2. However, for most cases, it is recommended that you use redundant links for all fabric facing interfaces.

3.1 Test Case #2 Results

Test Case #2 was performed successfully, with no issues. The Brocade VCS fabric interoperates with the Network File System (NFS) Protocol using Red Hat Enterprise Linux NFS server.

This validation demonstrates that existing deployments using NFS will interoperate with Brocade VCS fabrics and exhibit resiliency to input/output (I/O) between clients and servers in a non-disruptive manner.

This is a summary of the tests conducted for Test Case #2.

1. Test 1 validated that baseline I/O is using the shortest equal-cost paths in the fabric. NFS clients on RB3 and RB4 will access the Linux NFS server on the active vLAG interface on RB1.

2. Test 2 validated that I/O flows are not impacted between the NFS clients emulated by Spirent Avalanche and Red Hat Enterprise Linux NFS server while introducing a link failure on one link in the Brocade trunk, which is an active shortest path in the fabric.

3. Test 4 validated that I/O flows are not impacted between the NFS clients emulated by Spirent

Avalanche and Red Hat Enterprise Linux NFS server while introducing a switch failure in the active path of the Brocade VCS fabric.

This summarizes the test results for Test Case #2.

Test Description Results

1 Baseline of shortest paths and traffic distribution in the Brocade VCS fabric

Pass

2 Perform link failure within a Brocade ISL trunk Pass

(31)

3.2 Topology

Figure 7. Red Hat Enterprise Linux NFS server validation topology and components for Test Case #2.

3.3 Hardware Resources

Brocade VDX 6720-24 3 Brocade NOS 2.1.1 Brocade VDX 6730-32 1 Brocade NOS 2.1.1

Linux NFS Server 1 Red Hat Enterprise Linux Server Release 5.2 (Tikanga)

Spirent Avalanche 1 3.6.0

3.4 Compute Resources

(32)

3.5 Software Resources

The following software was used in this configuration:

Description Revision

Brocade NOS Brocade NOS 2.1.1

Linux NFS Server Red Hat Enterprise Linux Server release 5.2 (Tikanga)

3.6 Test 1: I/O Verification

The purpose of test 1 is to validate baseline I/O is using the shortest equal-cost paths in the fabric. NFS clients on RB3 and RB4 will access the Linux NFS server on the active vLAG interface on RB1.

Figure 8. Red Hat Enterprise Linux NFS server validation topology for test 1.

Step 1: Execute the following command on all Brocade VDX ISLs and VDX vLAG interfaces to confirm traffic flow and that the I/O path is distributed as expected.

The following is an example:

VDX6730-RB3# show in te 3/0/24 | in rate

(33)

The expected results for test 1 is that all I/Os from each of the NFS clients on RB3 and RB4 will be spread evenly across the Brocade trunks going into RB1, where the active vLAG on the server is located.

The actual results for test 1 confirm that all I/Os from each of the NFS clients on RB3 and RB4 are spread evenly across the Brocade trunks going into RB1, where the active vLAG on the server is located.

The total bandwidth from the 95 Spirent clients on RB3 (3/0/24) and RB4 (4/0/24) are received on the Egress port going to the NFS server (1/0/23).

(34)

3.7 Test 2: Link Failure

The purpose of test 2 is to validate that I/O flows are not impacted between the NFS clients emulated by Spirent Avalanche and Red Hat Enterprise Linux NFS server while introducing a link failure on one link in the Brocade trunk, which is an active shortest path in the fabric. The links that are failed as part of this test are highlighted below in red.

The NFS clients on RB3 and RB4 continue to access the Red Hat Enterprise Linux NFS server on the active vLAG interface on RB1.

Step 2: Run Spirent Avalanche for the duration of the test to emulate the NFS clients.

Step 3: Now that active traffic on both Brocade ISLs is confirmed, an ISL in the trunk is failed. In this example, Interface TenGigE3/0/1 on RB3 is failed, which is one of the active ISLs in the trunk between RB3 and RB1. VDX6730-RB3# conf t

Entering configuration mode terminal VDX6730-RB3(config)# in te 3/0/1

(35)

Step 5: Execute the following command on the desired Brocade VDX switches to re-enable the failed ISL between RB3 and RB1.

VDX6730-RB3# conf t

Entering configuration mode terminal VDX6730-RB3(config)# in te 3/0/1

VDX6730-RB3(conf-if-te-3/0/1)# no shut

VDX6730-RB3# show in te 3/0/1 | in rate VDX6730-RB3# show in te 3/0/2 | in rate

Repeat Steps 1–6: Steps 1–6 were also performed for the other Brocade ISLs, highlighted in red under section 3.5.1.

The expected results for test 2 are to confirm that distributed I/O is not impacted while failing an active Brocade member ISL within the fabric.

The NFS clients on RB3 and RB4 continue to access the Red Hat Enterprise Linux NFS server on the active vLAG interface on RB1.

All I/Os from each of the NFS clients on RB3 and RB4 fail over to the remaining link on the Brocade trunk that goes into RB1, where the active vLAG on the server is located. The total bandwidth from the clients on RB3 (3/0/24) and RB4 (4/0/24) are received on the Egress port going to the NFS server (1/0/23). There should be no loss in I/O, with the exception of “in-flight” frames during the failure of the link. When the links in the trunk are re-enabled, the I/Os should be evenly redistributed to all of the active links in the trunk.

The same results should be observed between other Brocade ISL trunk link failures in the fabric. These failures were performed for the other Brocade ISLs highlighted in red under section 3.5.1.

The actual results for test 2 confirm that distributed I/O is not impacted while failing an active Brocade member ISL within the fabric.

During the link failure in the Brocade trunk while I/O was evenly distributed between the two 10 GbE links on the 20 GbE trunk; all I/O was immediately switched to the remaining 10 GbE link. When the link was re-enabled, all I/O was immediately rebalanced between both links again.

The same results were observed between other Brocade ISL trunk link failures in the fabric. These failures were performed for the other Brocade ISLs highlighted in red under section 3.5.1.

(36)

All I/Os from each of the NFS clients on RB3 and RB4 fail over to the remaining link on the Brocade trunk that goes into RB1, where the active vLAG on the server is located. The total bandwidth from the clients on RB3 (3/0/24) and RB4 (4/0/24) are received on the Egress port going to the NFS server (1/0/23).

When the links in the trunk are re-enabled, the I/O is evenly redistributed to all of the active links in the trunk. It appears that there was a loss of “in-flight” NFS frames during the failure, as highlighted in the output below.

3.8 Test 3: Switch Failure

The purpose of test 3 is to validate that I/O flows are not impacted between the NFS clients emulated by Spirent Avalanche and Red Hat Enterprise Linux NFS server, while introducing a switch failure in the active path of the Brocade VCS fabric. The switch to be reloaded, RB1, is a switch directly connected to an active vLAG interface of the Red Hat Enterprise Linux NFS server.

(37)

3.8.1 Test Procedure

Step 1: Validate that all switches in the Brocade VCS fabric are up and that the shortest paths between the NFS client and server are being used as depicted in test 1.

(38)

Note that all shortest paths are active, and there are six switches in the fabric.

Step 4: Disable RB1 to simulate switch failure, forcing the vLAG to use the standby link on RB2 to the NFS server. VDX6720-RB1# reload

Broadcast message from root Thu Dec 22 00:04:10 2011... The system is going down for reboot NOW !!

Step 5: Execute the following command on the desired Brocade VDX switches to validate fabric topology after RB1 failure:

RB-ID RB-ID Index Interface Hops Cost Index Interface BW Trunk --- 3 2 5 Te 3/0/5 1 500 6 Te 2/0/6 20G Yes 4 5 Te 3/0/5 2 1000 6 Te 2/0/6 20G Yes 22 5 Te 3/0/5 2 1000 6 Te 2/0/6 20G Yes

(39)

RB-ID RB-ID Index Interface Hops Cost Index Interface BW Trunk --- 4 2 6 Te 4/0/6 1 500 10 Te 2/0/10 20G Yes 3 6 Te 4/0/6 2 1000 10 Te 2/0/10 20G Yes 22 6 Te 4/0/6 2 1000 10 Te 2/0/10 20G Yes

Step 6: Verify that I/Os are still running after principal switch RB2 failure. The previous standby vLAG link to the NFS server on RB2 is now the active link (Egress to NFS server). The following is an example:

VDX6720-RB2# show in te 2/0/23 | in rate

Step 7: Execute the following command on the desired Brocade VDX switches to validate fabric topology after RB1 has become active:

(40)

The expected results for test 3 are to confirm that distributed I/O is not impacted while introducing a switch failure within the active path of the Brocade VCS fabric. It should be observed that all I/Os are rerouted via fabric switching in the remaining paths.

During the failure of the switches in the fabric impacting both end-device vLAGs, all Spirent client sessions will continue to flow. This will show little to no disruption of I/Os during the failure of RB1. Since the Spirent NFS server has redundant connections on RB1 and RB2 on its vLAG, it gets continuous access to the Spirent NFS clients. This test should show the resiliency of Brocade VCS Fabric technology and its end devices when vLAGs are used.

The actual results for test 3 confirm that distributed I/O is not impacted while introducing multiple switch failures in the active path of the Brocade VCS fabric. It is observed that all Spirent NFS clients are rerouted via fabric switching in the remaining paths.

As expected, during the switch failure of RB1, all Spirent client sessions continued to flow to the NFS server. Brocade VCS Ethernet fabric switching utilized all available paths between the end devices and failed over within sub-second convergence times. This test shows the resiliency of Brocade VCS fabric technology and its end devices when vLAGs are used.

(41)

APPENDIX A: TEST CASE #1

Brocade VCS Deployment Considerations: Dynamic LACP vLAG with NetApp

Step 1: Configure vLAG Port-channel Interface on Brocade VDX 6710-RB21 to NetApp FAS3050 GbE 0/c: VDX6710-RB21# conf t

Entering configuration mode terminal

VDX6710-RB21(config)# interface Port-channel 33

VDX6710-RB21(config-Port-channel-33)# vlag ignore-split VDX6710-RB21(config-Port-channel-33)# speed 1000

VDX6710-RB21(config-Port-channel-33)# description NetApp3050-VDX VDX6710-RB21(config-Port-channel-33)# switchport

VDX6710-RB21(config-Port-channel-33)# switchport mode access VDX6710-RB21(config-Port-channel-33)# switchport access vlan 1 VDX6710-RB21(config-Port-channel-33)# no shutdown

Step 2: Configure Interface Gigabit Ethernet 21/0/1 on Brocade VDX6710-RB21: VDX6710-RB21# conf t

VDX6710-RB21(config)# interface GigabitEthernet 21/0/1 VDX6710-RB21(conf-if-gi-21/0/1)# description NetApp3050_e0c

VDX6710-RB21(conf-if-gi-21/0/1)# channel-group 33 mode active type standard VDX6710-RB21(conf-if-gi-21/0/1)# lacp timeout long

VDX6710-RB21(conf-if-gi-21/0/1)# no shutdown

Step 3: Configure vLAG Port-channel Interface on Brocade VDX 6710-RB22 to NetApp FAS3050 GbE 0/d: VDX6710-RB22# conf t

VDX6710-RB22(config-Port-channel-33)# vlag ignore-split VDX6710-RB22(config-Port-channel-33)# speed 1000

VDX6710-RB22(config-Port-channel-33)# description NetApp3050-VDX VDX6710-RB22(config-Port-channel-33)# switchport

Step 4: Configure Interface Gigabit Ethernet 22/0/1 on Brocade VDX6710-RB22: VDX6710-RB22# conf t

VDX6710-RB22(config)# interface GigabitEthernet 22/0/1

VDX6710-RB22(conf-if-gi-22/0/1)# channel-group 33 mode active type standard VDX6710-RB22(conf-if-gi-22/0/1)# lacp timeout long

(42)

Step 5: Validate vLAG Port-channel Interface on Brocade VDX 6710-RB21 to NetApp FAS3050 GbE 0/c: VDX6710-RB21# show interface Port-channel 33

Port-channel 33 is up, line protocol is up Hardware is AGGREGATE, address is 0005.338c.adee Current address is 0005.338c.adee

Description: NetApp3050-VDX

Interface index (ifindex) is 671088673

Minimum number of links to bring Port-channel up is 1 MTU 2500 bytes

LineSpeed Actual : 1000 Mbit Allowed Member Speed : 1000 Mbit

Step 6: Validate Interface Gigabit Ethernet 22/0/1 on Brocade VDX6710-RB22: VDX6710-RB21# show interface gigabitethernet 21/0/1

GigabitEthernet 21/0/1 is up, line protocol is up (connected) Hardware is Ethernet, address is 0005.338c.adb6

Current address is 0005.338c.adb6 Fixed Copper RJ45 Media Present Description: NetApp3050_e0c

Interface index (ifindex) is 90597048320 MTU 2500 bytes

LineSpeed : 1000 Mbit, Duplex: Full Flowcontrol rx: off, tx: off

Step 7: Validate vLAG Port-channel Interface on Brocade VDX 6710-RB21 to NetApp FAS3050 GbE 0/c: VDX6710-RB22# show interface Port-channel 33

Port-channel 33 is up, line protocol is up Hardware is AGGREGATE, address is 0005.338c.d45a Current address is 0005.338c.d45a

Description: NetApp3050-VDX

LineSpeed Actual : 1000 Mbit Allowed Member Speed : 1000 Mbit Flowcontrol rx: off, tx: off

Step 8: Validate Interface Gigabit Ethernet 22/0/1 on Brocade VDX6710-RB22: VDX6710-RB22# show interface gigabitethernet 22/0/1

GigabitEthernet 22/0/1 is up, line protocol is up (connected) Hardware is Ethernet, address is 0005.338c.d422

Current address is 0005.338c.d422 Fixed Copper RJ45 Media Present

LineSpeed : 1000 Mbit, Duplex: Full Flowcontrol rx: off, tx: off

(43)

Brocade VDX Deployment Considerations: Static LACP vLAG with ESXi Server and

Brocade 1020 CNA

Step 1: Configure vLAG Port-channel Interface on Brocade VDX 6730-RB3 to ESXi5-vswitch1: VDX6730-RB3# conf t

VDX6730-RB3(config-Port-channel-44)# vlag ignore-split

VDX6730-RB3(config-Port-channel-44)# description ESXi5-vSwitch1 VDX6730-RB3(config-Port-channel-44)# switchport

Step 2: Configure Interface TenGigabit Ethernet 3/0/10 on Brocade VDX6720-RB3: VDX6730-RB3# conf t

VDX6730-RB3(config)# interface TenGigabitEthernet 3/0/10 VDX6730-RB3(conf-if-te-3/0/10)# description ESXi5-vmnic2

VDX6730-RB3(conf-if-te-3/0/10)# channel-group 44 mode on type standard VDX6730-RB3(conf-if-te-3/0/10)# no shutdown

Step 3: Configure vLAG Port-channel Interface on Brocade VDX 6720-RB4 to ESXi5-vswitch1: VDX6720-RB4# conf t

VDX6720-RB4(config-Port-channel-44)# vlag ignore-split

VDX6720-RB4(config-Port-channel-44)# description ESXi5-vSwitch1 VDX6720-RB4(config-Port-channel-44)# switchport

Step 4: Configure Interface TenGigabit Ethernet 4/0/10 on Brocade VDX6720-RB4: VDX6720-RB4# conf t

VDX6720-RB4(config)# interface TenGigabitEthernet 4/0/10 VDX6720-RB4(conf-if-te-4/0/10)# description ESXi5-vmnic3

VDX6720-RB4(conf-if-te-4/0/10)# channel-group 44 mode on type standard VDX6720-RB4(conf-if-te-4/0/10)# no shutdown

(44)

Step 5: Validate vLAG Port-channel Interface on Brocade VDX 6730-RB3 to to ESXi5-vswitch1: VDX6730-RB3# show interface Port-channel 44

Port-channel 44 is up, line protocol is up Hardware is AGGREGATE, address is 0005.3391.3b24 Current address is 0005.3391.3b24

Description: ESXi5-vSwitch1

Step 6: Validate Interface TenGigabit Ethernet 3/0/10 on Brocade VDX6730-RB3: VDX6730-RB3# show interface tengigabitethernet 3/0/10

TenGigabitEthernet 3/0/10 is up, line protocol is up (connected) Hardware is Ethernet, address is 0005.3391.3b12

Current address is 0005.3391.3b12 Pluggable media present

Description: ESXi5-vmnic2

LineSpeed Actual : 10000 Mbit

LineSpeed Configured : Auto, Duplex: Full Flowcontrol rx: off, tx: off

Step 7: Validate vLAG Port-channel Interface on Brocade VDX 6720-RB4 to to ESXi5-vswitch1: VDX6720-RB4# show interface Port-channel 44

Port-channel 44 is up, line protocol is up Hardware is AGGREGATE, address is 0005.3367.cafc Current address is 0005.3367.cafc

Description: ESXi5-vSwitch1

Step 8: Validate Interface TenGigabit Ethernet 4/0/10 on Brocade VDX6720-RB4: VDX6720-RB4# show interface tengigabitethernet 4/0/10

TenGigabitEthernet 4/0/10 is up, line protocol is up (connected) Hardware is Ethernet, address is 0005.3367.caea

Current address is 0005.3367.caea Pluggable media present

(45)

LineSpeed Actual : 10000 Mbit

LineSpeed Configured : Auto, Duplex: Full Flowcontrol rx: off, tx: off

Brocade VCS Deployment Considerations: Enable Brocade VDX Jumbo Frame Support

Note that this must be performed on all RBbridges where a given interface port-channel is located. In this example, interface port-channel 44 is on RBridge 3 and RBridge 4, so we will apply configurations from both RBridge 3 and RBridge 4.

Step 1: Example to enable Jumbo Frame Support on applicable VDX interfaces for which Jumbo Frame support is required:

VDX6730-RB3# conf t

VDX6730-RB3(config)# interface Port-channel 44 VDX6730-RB3(config-Port-channel-44)# mtu

(<NUMBER:1522-9216>) (9216): 9216

Brocade VCS Deployment Considerations: Enable Ethernet Pause/Flow Control Support

Note that this must be performed on all RBbridges where a given interface port-channel is located. In this example, interface port-channel 44 is on RBridge 3 and RBridge 4, so we will apply from both RBridges.

Step 1: Enable QOS Flow Control for both tx and rx on vLAG Port-channel 44 Interface of VDX 6730-RB4 to ESXi5-vswitch1:

VDX6730-RB3# conf t

VDX6730-RB3(config-Port-channel-44)# qos flowcontrol tx on rx on

Step 2: Validate vLAG Port-channel 44 Interface qos rcv-queue on Brocade VDX 6730-RB4 to to ESXi5-vswitch1 after changes have been made:

VDX6730-RB3(config-Port-channel-44)# do show running-config interface Port-channel 44 interface Port-channel 44

vlag ignore-split

description ESXi5-vSwitch1 switchport

switchport mode access switchport access vlan 1

qos flowcontrol tx on rx on

Brocade VCS Fabric Technology and NAS with NFS Validation Test

DATA CENTER