Using BLADE Switches for iSCSI
SAN Connectivity
Case Study
Balance the need for a robust/always-on IT infrastructure while keeping costs low
Choosing the right storage technology was a key priority for IT when Blade Network Technologies (BLADE) was established in February 2006.
While still a division within Nortel, BLADE engineers had already been building, shipping and supporting blade switches for the two largest blade server vendors for the past four years. More
products were in the pipeline, and due to release in June 2006. So BLADE wasn’t the typical startup—it had the needs of an established business, but had to build a new datacenter from the ground up.
As a newly independent company, BLADE could not afford a moment of downtime—customers were expecting their products on time, period. As the IT director planned the network and the data migration from Nortel into new facilities, the business imperative was clear:
Balance the need for a robust/always-on IT infrastructure with the startup’s need to keep costs low—and have it up and running in under a month.
BLADE’s business profile in February 2006
Privately held technology company Plans to increase from 40 to 80+ employees in first 6 months Revenues in the $50-$100M range
Business Operations in U.S.A, Canada, Australia, France, U.K, Germany, Hungary, Scotland, Amsterdam, Vietnam, Malaysia, India, Taiwan, Mexico, & China
Mission critical server applications span multiple functions: – Engineering (software build, compilation, defect tracking) – Finance (accounting – AP, AR, GL, etc.)
– Supply Chain Operations (Product Life Cycle Management) – Communications (Blackberry, MS Exchange, VoIP)
– Disaster Recovery at secondary data center in Canada
Choosing a SAN solution: Fibre Channel or iSCSI?
Since BLADE specializes in developing networking devices for the blade server market, the decision to build the IT infrastructure based on blade servers was a no-brainer. However, looking at the current state of blade servers in today’s market, BLADE saw many solutions that assumed reliance on dual technologies —Ethernet for LAN, and Fibre Channel for SAN. In fact, most blade servers contain slots for both Ethernet and Fibre Channel switches. BLADE’s IT director sat down with a storage expert and took a look at these two technologies.
Fibre Channel technology has been around since the late 1980s, and was originally developed to improve connectivity and cable lengths between SCSI devices on super computers. Today Fibre
Channel is considered a reliable, congestion-free, and fast method for connecting SANs. However robust the technology has proven itself to be, the IT director and storage expert saw some significant
roadblocks to widespread Fibre Channel deployment:
iSCSI is a newer SAN networking technology (ratified in 2003) that uses TCP/IP Ethernet for data transfer. The SCSI storage protocol is encapsulated into IP packets for transport through any common Ethernet interface. The simplicity of this technology is that iSCSI initiator software on the blade server, paired with iSCSI target software on SAN disk storage arrays or tape storage, performs the bulk of the work needed for transporting iSCSI data to the SAN.
Other Ethernet devices in the network require no specific configuration to handle iSCSI traffic, though special consideration for trunked uplinks to the SAN disk array can add performance. Latency was a potential issue with iSCSI, as it depended on overall Ethernet network performance, which could vary. The IT director and the storage consultant looked at the following two configuration options:
Option 1: Use Ethernet switches to connect to LAN and WAN, and Fibre Channel switches to connect to SAN.
They found the following issues with mixed Ethernet/Fibre Channel configuration:
An Ethernet and Fibre Channel solution is complex and difficult to maintain, and may require outside consulting. Most in-house IT staff do not have specialized Fibre Channel expertise
Servers can’t be re-provisioned between web serving to application serving without pulling out the server blades, and swapping NICs and HBAs
Local storage on server blades offers poor performance, adds cost, and reduces server reliability Adding on additional Fibre Channel storage arrays could require forklift upgrades, and increase
latency as additional storage arrays are daisy chained together
Clearly, this was not the right solution for a company building its IT storage infrastructure from scratch.
Option 2: Use Ethernet switches to connect to LAN and iSCSI SAN
So the IT director and the storage expert looked at implementing an all-Ethernet solution, using iSCSI protocol storage arrays:
They saw the following advantages in implementing an iSCSI SAN: Solution simplicity: both LAN and SAN run over IP/Ethernet
Servers can be dynamically re-provisioned from web serving to application serving (and visa versa) using the included storage management software
Disk shelf restriping could be done in a matter of minutes, not days
Local storage is consolidated from individual server blades onto a shared disk shelf Virtualization of the disk shelf
– Local Disk on a server blade required only for iSCSI initiator software, thus conserving blade server resources for other applications.
– Low latency, low CPU utilization on blade server, and minimal bandwidth requirements Block level data replication over the corporate WAN to remote disaster recovery cluster for faster
backups
Future support for 10Gb Ethernet meant that performance would keep up with the company’s growing storage needs
They were on to something. Implementing iSCSI SANs would make setup and scaling of BLADE’s storage arrays both manageable and flexible.
The last thing they did was compare cost between Fibre Channel and iSCSI. While cost wasn’t the determining factor in choosing which technology to implement, the cost savings (detailed below) combined with the manageability and scalability made iSCSI the clear winner.
Data Center Total Cost of Ownership
Table 1 Primary Data Center Configuration, Santa Clara, CA
Configuration Ethernet Switches for
LAN/ Fibre Channel Switches for SAN
Ethernet Switches for LAN & SAN
Connectivity
Reason for Cost Delta
Blade Server chassis + Power Supplies + Switches For LAN and SAN
$36,400 $14,000 FC SAN Switches
10 Server Blades + NICs + HBAs $42,000 $37,600 HBAs
10 TB Storage Arrays $350,000 $120,000 FC Storage Arrays
Storage Management Software & Utilities $50,000 $0 (included with hardware)
Storage Mgt. Software
Installation Cost $10,000 $4,000 Setting up FC arrays takes more
time and requires technicians who charge more than Ethernet technicians
Net Acquisition Cost After Discount $366,300 $131,700 All of the above
Maintenance Costs for 3 years – 24*7 Service
$128,520 $51,480 Maintenance cost is a % of
acquisition cost
3 Year Cost Of Ownership $494,820 $183,180 iSCSI is 37% the cost of Fibre Channel
Table 2 Disaster Recovery Data Center Configuration: Ottawa, Canada
Configuration Cost of Ethernet
Switches for LAN and Fibre Channel Switches for SAN
Cost of Ethernet Switches for LAN and SAN Connectivity
Reason for Cost Delta
Blade Server chassis + Power Supplies + Switches For LAN and SAN
$20,200 $9,000 FC SAN Switches
5 Server Blades + HBAs $21,000 $17,500 HBAs
10 TB Storage Arrays $262,500 $90,000 FC Storage Arrays
Storage Management Software & Utilities
$25,000 $0 (included with hardware)
Storage Mgt. Software
Installation Cost $5,000 $2,000 Setting up FC arrays takes
more time and requires technicians who charge more than Ethernet technicians Net Acquisition Cost After
Discount
$250,275 $90,375
Maintenance Costs for 3 years – 24*7 Service
$91,110 $34,950 Maintenance cost is a % of acquisition cost
3 Year Cost Of Ownership $341,385 $125,325 iSCSI is 37% the cost of Fibre Channel
Table 3 Cost of Ownership Summary
Cost Item For Both Data Centers (Primary and
Disaster Recovery Sites)
Using Ethernet Switches for LAN and Fibre Channel Switches for SAN
Using Ethernet Switches for LAN and SAN Connectivity
Savings Over 3 Years as a Result of Using Ethernet Switches & iSCSI SAN
Acquisition Cost $616,575 $222,075 $394,500 (64% )
3 year maintenance costs $219,630 $86,430 $133,200 (61%)
3 year Cost of Ownership $836,205 $308,505 $527,700 (63%)
Products Used
1. Blade server with single local SCSI drive. A basic blade server installation, this server acts as the “host” for virtual “guest” machines.
– VMWARE server beta (free software) installed – iSCSI initiator software installed
– Blade server switches from Blade Network Technologies
2. iSCSI Initiator to Equallogic Storage array over multiple trunked Gigabit Ethernet links 3. Equallogic Storage Arrays
The virtual machine hosted on the storage array can be moved to another location and booted without reconfiguration. The entire array contents are replicated to the Ottawa Disaster Recovery Center every night —greatly reducing recovery time, and ensuring business continuity.
Conclusion
Implementing a scalable and robust storage solution for a “startup” the size of BLADE was initially a daunting challenge. Fibre Channel had a reputation for speed and reliability, but was difficult to implement without specialized IT knowledge. There were concerns about day to day management and the cost to scale to a larger storage array. iSCSI is built around a well-known technology—Ethernet— that requires little additional expertise to setup and manage. Virtualization of disk shelf resources allows for efficient reprovisioning of disk resources. And finally, implementing an iSCSI storage solution would help the company realize 63% cost savings over three years, compared to a Fibre Channel solution. Our one-person IT department was able to implement the iSCSI SAN solution, and build the entire data center, well under the one-month deadline. Now that he’s done it once, he’s confident he could build it all over again in just one weekend.
BLADE NETWORK TECHNOLOGIES: The industry’s #1 provider of Ethernet, IP, and application switches for blade server systems. Our products
represent a strategic control point in a blade server system. They are the gateway for all traffic directed to and from a blade server system. As a strategic control point, our products have a direct influence on a blade server system’s performance, high availability, scalability, manageability, security, and total cost-of-ownership.
©2006 Blade Network Technologies, Inc. All rights reserved. Information in this document is subject to change without notice. Blade Network Technologies assumes no responsibility for any errors that may appear in this document
