VMware vsphere 5.0: 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP 3PAR Utility Storage

(1)

VMware vSphere 5.0: 8Gb/s Fibre

Channel SANs with HP ProLiant

DL380 G7 servers and HP 3PAR

Utility Storage

Create robust, highly available

vSphere 5.0 Fibre Channel

environments with best-of-breed HP

3PAR Utility Storage and HP ProLiant

DL servers

(2)

Table of contents

Emulex Solution Implementer’s Series ... 3

Executive summary ... 3

Introduction ... 4

About this guide ... 5

Solution components... 6

ESXi 5.0 ... 7

HP ProLiant servers ... 8

Deploying the solution components ... 9

Pre-installation ... 9

Updating firmware ... 9

Configuring network connectivity ... 10

Configuring storage ... 11

Using the VMFS-5 filesystem for additional functionality ... 11

Support for RDM LUNs in ESXi 5.0 ... 12

Configuring Fibre Channel connectivity ... 12

Deploying ESXi 5.0 ... 17

Post-installation ... 17

Planning the network environment for the host ... 19

Configuring the HP 82E 8Gb/s HBA ... 20

Using NPIV to identify HBA ports ... 20

Configuring the storage array ... 21

Using the OneCommand Manager plug-in for VMware vCenter ... 22

Performance comparison ... 26

Test environment ... 26

Test methodology... 28

Summary ... 33

Appendix A – Installing the CIM provider and vCenter plug-in ... 34

Installing the CIM provider ... 34

Installing the vCenter plug-in ... 36

Appendix B – Sample configurations for ESXi 5.0 host and VMs ... 37

Appendix C – Configuring BfS ... 38

(3)

3 Deploying 8Gb/s Fibre Channel SANs with HP ProLiant DL380 G7 servers and HP storage

Emulex Solution Implementer’s Series

This document is part of the Emulex Solution Implementer’s Series, providing Implementers (IT administrators and system architects) with solution and deployment information on popular server and software platforms. As a leader in I/O adapters – Fibre Channel (FC), Ethernet, iSCSI and Fibre Channel over Ethernet (FCoE) – the Emulex technology team is taking a lead in providing guidelines for implementing I/O for these solutions.

Executive summary

With ESXi 5.0, VMware continues to raise the bar for hypervisor products, introducing many new features along with support for more and larger virtual machines (VMs). ESXi 5.0 does not utilize a service console OS; VMware agents and Common Information Model (CIM) providers run directly on the hypervisor layer (VMkernel). As a result, there will be no further releases of ESX.

There are three options for communicating with VMkernel: VMware vSphere’s enhanced command-line interface (vCLI), vSphere PowerCLI or the vSphere Management Assistant (vMA) virtual appliance.

ESXi 5.0 provides many new features and enhancements in areas such as storage and networking. Indeed, VMware has indicated1 that there are over 100 new features, claiming, for example, that ESXi 5.0 can run VMs that are four-times as powerful as those supported by earlier versions. However, as shown in Table 1, the new, larger VM will place heavy demands on data center infrastructure.

Table 1. Resources supported by various VMware hypervisors

Resource Requirements for 95% of applications

ESX 1 ESX 2 VMware

Infrastructure 3.0/3.5

vSphere 4 vSphere 5

CPU 1 – 2 CPUs 1 vCPU 2 vCPUs 4 vCPUs 8 vCPUs 32 vCPUs

Memory <4GB (peak) 2GB/VM 3.6GB/VM 16 – 64GB/VM 256GB/VM 1,000GB/VM

Network <2.4Mb/s <0.5Gb/s 0.9Gb/s 9Gb/s 30Gb/s >36Gb/s

Storage <10,000IOPS <5,000IOPS 7,000IOPS 100,000IOPS 300,00IOPS 1,000,000IOPS

1

(4)

To help you transition to an infrastructure that can support the storage needed by this new generation of VMs, Emulex has validated the functionality and performance of ESXi 5.0 in conjunction with 8Gb/s Fibre Channel connectivity. The tested environment included the best-of-breed HP ProLiant DL380 G7 server equipped with a dual-channel 8Gb/s Fibre Channel

adapter produced for HP by Emulex. Storage was provided byHP 3PAR Utility Storage.

In addition to suggesting that 8Gb/s SANs are better suited for the new release of vSphere, this implementer’s guide outlines the process for deploying ESXi 5.0 with 8Gb/s Fibre Channel connectivity and outlines the results of performance tests carried out in such an environment. The testing demonstrated that an 8Gb/s Fibre Channel implementation based on the latest Emulex Fibre Channel adapter technology was able to support more VMs than a 4Gb/s

implementation, while also reducing CPU utilization on the ESXi 5.0 host. Thus, as VM density increases, the additional burden placed on storage can be mitigated by migrating from 4Gb/s Fibre Channel to 8Gb/s.

Intended audience: This document is intended for engineers, system administrators and

VMware administrators interested in deploying ESXi 5.0 on an HP ProLiant server featuring an 8Gb/s Fiber Channel adapter.

Testing performed in September-October 2011 is described.

Introduction

ESXi 5.0 supports an unprecedented number of VMs on a single host; moreover, these VMs can reach unprecedented size. As a result, Emulex expects to see more and more workloads being virtualized, as well as additional resources being assigned to existing VMs in order to meet the needs of particular workloads. As noted by VMware2, “the VM will only get bigger and faster.”

2

(5)

Figure 1 shows the progress that has been made in virtualizing business-critical applications.

Alex: Figure 1: the source is a bit blurred.

Figure 1. Comparing the percentages of business-critical application instances virtualized using VMware hypervisors

VMware expects3 more VMs to be deployed than ever before, with ESXi 5.0 allowing these VMs to grow as much as four times larger. With this increased density, virtualized environments must be able to provide additional network and storage resources in order to support the workloads.

About this guide

This implementer’s guide describes how to configure an 8Gb/s Fibre Channel storage array network (SAN) with a ProLiant DL380 G7 server in a vSphere 5.0 environment. Guidelines and instructions are provided for configuring servers, adapters and storage using technical

documentation provided by VMware and HP.

While the key objective was to describe the configuration and validation of this end-to-end solution, this guide also outlines the following:

 Implementing boot from SAN (BfS)

 Using the Emulex OneCommand Manager plug-in for VMware vCenter Server

3

(6)

Solution components

Emulex built the end-to-end test environment shown in Figure 2 in order to validate 8GB/s Fibre Channel connectivity with ESXi 5.0.

Table 2 outlines the key components deployed in this environment. Table 2. Test environment

Component Device Comments

Tested server HP ProLiant DL380 G7 Virtualization host running ESXi 5.0; 10 identically-configured VMs

Management server HP ProLiant DL160 Two VMs:

 vMA (Linux appliance)

 vSphere Client and Emulex OneCommand Manager or the Emulex OneCommand Manager plug-in for VMware vCenter

AD & DNS server Generic Microsoft Windows Server 2008 system

Support for Microsoft Active Directory (AD) and Domain Name System (DNS)

Storage HP 3PAR Utility Storage F200 Storage System (QL226B)

Fibre Channel connectivity

8GB/s host bus adapter (HBA) HP 82E 8Gb Dual Port Fibre Channel HBA

(7)

Figure 2. Lab Test environment

If this is your first time installing VMware products on a ProLiant server, it is important for you to have a basic understanding of each of the solution components so that the terminology used in this guide is clear.

ESXi 5.0

VMware’s latest hypervisor, ESXi 5.0, extends the core capabilities of vSphere 4.1 and provides the foundation for a cloud infrastructure, whether public or private. Areas where you can expect to see improvements after deploying ESXi 5.0 include server consolidation, performance, management, provisioning, and troubleshooting.

For more information on vSphere 5.0, refer to “What’s New in VMware vSphere 5.0 Platform,” available at http://www.vmware.com/files/pdf/techpaper/Whats-New-VMware-vSphere-50-Platform-Technical-Whitepaper.pdf.

(8)

HP ProLiant servers

HP ProLiant DL380 G7 continue to be the server of choice for many HP shops in the VMware space. These servers are used widely – from small and medium-sized businesses (SMBs) to large data centers – due to their high availability, scalability, CPU horsepower and memory capacity. In addition, these 2U rack-mount servers save space and power, making them ideal for large data centers moving to a cloud infrastructure.

Using Fibre Channel for shared storage

When deploying vSphere on ProLiant servers, you should always consider using a SAN so that you can take full advantage of the many hypervisor features and enhancements that require shared storage.

While VMware supports most of the more common protocols, Fibre Channel is the predominant choice for shared SAN storage. Storage protocols continue to evolve, introducing options such as iSCSI and NFS storage arrays; however, this guide focuses on Fibre Channel connectivity.

Server sizing

HP has developed an automated tool – HP Sizing Tool for VMware vSphere – that can help you size and scope a server for a particular vSphere deployment. Based on your responses to a questionnaire, this tool provides a quick, consistent method for identifying the best server for your environment. It also creates a bill of material.

For more information on this sizer and other HP solutions for VMware, visit www.hp.com/go/vmware.

HP Storage

HP 3PAR Utility Storage is a highly virtualized storage system built from the ground up to enhance the benefits of a VMware vSphere virtualization platform. This is accomplished by exceeding the storage demands of a server virtualization environment through architectural advantages that include wide-striping and a custom ASIC for true mixed workload support. HP 3PAR Utility Storage combines highly virtualized, autonomically managed and dynamically tiered storage with advanced internal virtualization capabilities to increase administrative efficiency, system utilization, and storage performance. As a result, HP 3PAR Storage boosts VMware vSphere ROI by enabling you to optimize your data center infrastructure, simplify storage administration and maximize virtualization savings.

(9)

Deploying the solution components

Having introduced the key solution component, this implementer’s guide now describes how to configure 8Gb/s Fibre Channel connectivity. Guidelines are provided for the following areas:

 Pre-installation

 Configuring network connectivity

 Configuring storage

 Deploying ESXi 5.0

 Post-installation

Pre-installation

There are several steps to consider before applying power or installing an operating system on any system. First, you need to ensure rack space and appropriate power and cooling are available. Another key step is to ensure all hardware and software are at the latest firmware levels and download any necessary patches and drivers. The pre-installation section provides links to verify the latest firmware levels for both HP and VMware. The pre-installation process provides suggestions in pre-configuring the network for an ESX host, followed by suggestions for storage systems and storage area networking.

As a best practice, HP recommends verifying with HP technical support that you are running the very latest HP firmware and drivers on components such as LAN on motherboard (LOM) and PCI adapters.

Updating firmware

You can review the latest HP and VMware recommended firmware levels at the following locations:

 HP: Visit www.hp.com/go/vmware and refer to the Certified ProLiants and Certified HP

Storage links under Tools/Resources.

 VMware: Refer to the VMware Compatibility Guides at http://www.vmware.com/resources/guides.html.

Before deploying ESXi 5.0 on a ProLiant server, you should determine the latest versions of the following firmware and, if necessary, update:

 ESXi host: o System BIOS

o LOMs and PCI adapters

 Storage array and controllers

(10)

Note

Always contact HP support to verify the latest firmware updates and drivers.

As always plan your deployment or upgrade before installing software. Read all the

documentation provided by VMware and HP before starting. Planning will speed up the process – particularly if you intend to deploy multiple servers.

With pre-installation activities complete, you can now configure your networking.

Configuring network connectivity

Before installing ESXi 5.0, you need to understand the network requirements for the particular ESXi host and the connectivity supported by the physical server. For example, while many physical servers feature LOM or integrated network interfaces, ports are typically 1Gb/s, though newer models may provide 10Gb/s Ethernet (10GbE) ports.

The ProLiant DL 380 G7 used in the test environment was equipped with two HP NC382i Integrated Dual Port PCI Express Gigabit Server Adapters (that is, four 1Gb/s ports). Thus, Emulex was able to configure the ESXi host with four network interface cards (NICs), as shown in Table 3.

Table 3. Four-NIC configuration for ESXi 5.0

Physical NIC Virtual switch Port group(s)

vmnic0 vSwitch0 Management port and vMotion

vmnic1 vSwitch0 Management port and vMotion

vmnic2 vSwitch1 VM network

vmnic3 vSwitch1 VM network

Combining management port traffic, which is relatively light, with vMotion traffic is acceptable in many four-NIC deployments. Since vMotion traffic is heavier, it is not a good practice to

combine this with VM traffic; in practice, you should consider separating vMotion and VM network traffic on different subnets.

Since management port and vMotion traffic is placed on the same virtual switch in the test environment, it is a good practice to use a virtual LAN (VLAN) to enhance security and isolate management port and vMotion traffic.

(11)

Configuring storage

This section provides information on the following topics:

 Using the new VMFS-5 filesystem to support additional storage functionality

 Support for RDM LUNs in ESXi 5.0

 Configuring Fibre Channel connectivity o Implementing fabric zoning

o Configuring the storage array o Configuring BfS

Using the VMFS-5 filesystem for additional functionality

Introduced with ESXi 5.0, the VMFS-5 filesystem provides support for VMFS volumes up to 64TB in a single extent. With VMFS-3, 32 extents would have been required to achieve 64TB.

Note

The volume creation wizard for VMFS-5 uses GUID Partition Table (GPT) format rather than Master Boot Record (MBR), which allows you to create VMFS volumes that are larger than 2TB.

GUID refers to a globally-unique identifier.

With the ability to create large VMFS volumes, you must now manage storage array queue depth as well as host queue depth. For example, the queue depth of the HP 82E 8Gb/s HBA is set by default to 30 and may be adjusted via Emulex OneCommand Manager, the

OneCommand Manager plug-in for vCenter Server or vMA. Other benefits delivered by the VMFS-5 filesystem include:

 Support for up to 130,000 files rather than 30,000 as before

 Support for 64TB physical-mode Raw Device Mapping (RDM) LUNs

 Virtual mode allows you to create snapshots, which is beneficial when a file exceeds 2TB

 For space efficiency, there can now be up to 30,000 8KB sub-blocks

 There is small-file support for files of 1KB or less; in the past, such files would have occupied entire sub-blocks

As you plan the deployment of shared storage, take into consideration the benefits of VMFS-5. For example, if you are migrating from ESXi 4.x to ESXi 5.0, you may also wish to migrate to VMFS-5 to take advantage of the new features and enhancements.

(12)

Support for RDM LUNs in ESXi 5.0

While RDM LUNs are still supported in ESXi 5.0, Emulex suggests using them sparingly due to manageability issues.

VMware testing (as shown in Figure 3) has demonstrated that RDMs deliver a negligible performance advantage.

Figure 3. Comparing IOPS performance with different LUNs

Configuring Fibre Channel connectivity

The Emulex Fibre Channel driver is in-box, making it is easy to transition 8Gb/s Fibre Channel connectivity from an earlier platform. There is no need to install the driver during the deployment of ESXi 5.0.

Thus, the configuration of 8Gb/s Fibre Channel connectivity via HP 82E 8Gb/s HBAs is a simple process in an ESXi 5.0 environment, with just the following stages:

 Implementing fabric zoning

 Configuring the storage array

 Configuring BfS

Before you begin the configuration, it is a best practice to review the appropriate VMware Compatibility Guide to ensure firmware on the storage array is at the latest level, as shown in

(13)

Figure 4.

Figure 4. Showing firmware levels specified in the VMware Compatibility Guide for HP 3PAR Storage

IMPORTANT

Whenever there is a need to update code or firmware, Emulex recommends backing up your data and configuration.

(14)

Implementing fabric zoning

Zoning has become a standard component of VMware deployments; indeed, most if not all storage array vendors recommend zoning LUNs that are presented to ESXi hosts. Fabric zones can enhance manageability while providing support for advanced features such as vMotion and Fault Tolerance that require multiple hosts to access the same LUN.

Zones can also enhance security. For example, consider what might happen if you were to connect a new server to the same SAN switch as an existing ESXi host. Without zoning or some other security measure, the new server would be able to access the same storage as the

existing host and could potentially overwrite the filesystem, obliterating VM data and files. Thus, since the test environment features two 8Gb/s HBA ports, there should ideally be two – or more – fabric switches, each configured with a zone that includes the particular HBA port.

Figure 5 shows the zoning used in the test environment.

(15)

The test environment utilizes two Brocade 8Gb/s SAN switches and a total of four zones, as shown in Table 4. The zones were added to an alias zone configuration, which was then activated.

Table 4. Zone configuration for HBA

HBA Storage controller

Alias Zone Zone

configuration

Switch

Port 0 Port A1 Zone 1 ZoneSet 1 ZoneConfig_1 1

Port B2 Zone 2 ZoneSet 1 ZoneConfig_1 2

Port 1 Port A2 Zone 3 ZoneSet 2 ZoneConfig_1 2

Port B1 Zone 4 ZoneSet 2 ZoneConfig_1 1

This zone configuration will give the ESXi host four paths to a single LUN. At this stage of the deployment, no LUNs have been created; thus, LUNs cannot yet be bound to WWN ports on the HP 82E 8GB/s HBA.

Setting the storage array

HP 3PAR Utility Storage is very popular in VMware environments due to their extensive

virtualization capabilities and ease of management. HP has documented deployment and tuning best practices for this simple-to-use yet robust and highly available array; for more information, refer to “HP 3PAR Utility Storage with VMware vSphere”

Emulex followed HP’s best practices when configuring the HP 3PAR Storage system for the test environment. The process included the following stages:

 Update storage array controller firmware and management software as needed

 Configure Virtual Domains

 Configure Virtual LUNs

 Assign host mode

 Create hosts

(16)

Figure 6. Assigning the appropriate host mode as part of the storage array set-up

If the correct zoning and host mode have been applied, LUNs will be visible to the assigned hosts. There should be four paths to each LUN for optimal performance and redundancy.

Configuring BfS

Enterprise server manufacturers such as HP continue to offer local disk storage; however, with the growth of virtualization and the increased use of BfS4, server configurations are evolving. For example, HP offers a diskless server, which would allow you to deploy ESXi via a USB flash drive.

BfS capability is often configured in an ESXi environment, where its benefits include:

 Enhanced manageability

 Faster deployment

 Easier backup5

 Enhanced disaster recovery capabilities

The process for configuring BfS via an HP 82E 8GB/s HBA is simple and can be achieved in the following stages:

 Load the latest boot code to the HP 82E 8GB/s HBA

 Configure the ESXi host

 Specify the desired boot volume

 Place the HP 82E 8GB/s HBA first in the boot order

This vendor-specific process is described in more detail in Appendix C – Configuring BfS.

Note

If you plan to install ESXi 5.0 with local storage, Emulex recommends disconnecting the Fibre Channel cables from the SAN to prevent the OS from being accidentally installed on the SAN.

Once storage has been configured – and you have verified that the hardware has been certified by VMware6 – you can deploy ESXi 5.0.

4

Whereby the storage array owns the operating system LUN

5

Since the array owns the LUN, array-based copies can be made without server intervention.

6

(17)

Deploying ESXi 5.0

This section on deployment will cover the options for installing ESXi 5.0 to an HP ProLiant server. Since ESXi 5.0 has been designed for flexibility, you have a range of new deployment options, including the following:

 Interactive installation: Suggested for fewer than five hosts

 Scripted installation: Unattended deployment for multiple hosts

 vSphere Auto Deploy installation: Suggested for a large number of ESXi hosts with vCenter Server

 Custom installation: vSphere 5 Image Builder command-line interfaces (CLIs) provide custom updates, patches and drivers

Emulex elected to use the interactive method, downloading an ESXi 5.0 image from the

VMware website to local storage. The deployment process is fairly straightforward and, in many ways, identical to the deployment of earlier versions of ESXi. Since this process is detailed in VMware’s technical documentation, it is not described in this guide.

The main points to remember before beginning the deployment are as follows:

 Check all firmware on host and adapters and make updates as needed

 Check HBA drivers

 Plan the networking configuration

 Plan the Fibre Channel storage configuration and LUNs

 Decide on local or BfS storage

 Select the most suitable deployment option for your environment It should not take more than a few minutes to install ESXi 5.0.

After you have configured the management network and, if appropriate, enabled lockdown mode, you can proceed with the post-installation process, which includes configuring the ESXi 5.0 host for 8 Gb/s Fibre Channel SAN connectivity.

Post-installation

After ESXi 5.0 has been deployed on the ESXi 5.0 host, you should review the installation and perform any necessary updates. Consider the following examples:

 A vendor may have recently released new firmware or drivers.

 Since it is not configured automatically – unless you are using a scripted installation – you should configure NIC teaming.

(18)

 If you are using local storage, remember to reconnect the Fibre Channel cables to the SAN and then verify that the host can login to the fabric and view any LUNs that have been assigned

You can now configure the ESXi 5.0 host and storage array for 8 Gb/s Fibre Channel SAN connectivity, which may involve the following activities:

 Planning the network environment for the host

 Configuring the HBA

 Using NPIV to identify HBA ports

 Configuring the storage array with features such as multipathing

Note

For host and VM configurations used in the Emulex test environment, refer to Appendix B – Sample

(19)

Planning the network environment for the host

You should plan the network environment for the host in conjunction with your network administrator.

Figure 7 shows the four-NIC configuration planned for the Emulex test environment, with ports for the VMkernel, management network and VM network.

Figure 7. Four-NIC network configuration for the ESXi host

This configuration is commonly used with 8Gb/s Fibre Channel SANs. The following connectivity is provided:

 vmnic0 – vmnic1: Redundant network adapters for the management port and vMotion

(20)

Configuring the HP 82E 8Gb/s HBA

If you are migrating to ESXi 5.0 or are installing ESXi 5.0 for the first time and have installed the HP 82E 8Gb/s HBA in an appropriate, full-length PCI slot, with the correct small form-factor pluggable (SFP) transceivers, configuration is simple.

Since ESXi 5.0 already provides an in-box VMware driver for Fibre Channel, there is no need to install a driver. After the installation of ESXi 5.0, review the Configuration tab of vCenter Server; you will see your HBAs listed under Storage Adapters, as shown in Figure 8.

Figure 8. In this example, ESXi has automatically recognized an 8Gb/s Fibre Channel HBA

If necessary, you can use vMA to remotely send commands to the ESXi host to verify that the driver has been installed.

Using NPIV to identify HBA ports

N-Port ID Virtualization (NPIV)7 is supported in ESXi 5.0. This facility allows a single Fibre Channel HBA port to register with a fabric using multiple worldwide port names (WWNs), each having a unique entity. To learn more, refer to the VMware technical note, “Configuring and Trouble-shooting N-Port ID Virtualization,” which provides specific information on Emulex adapters. The technical note is available at

http://www.vmware.com/files/pdf/techpaper/vsp_4_vsp4_41_npivconfig.pdf.

7

(21)

Configuring the storage array

In addition to the VMFS-5 filesystem, new storage-related features introduced in ESXi 5.0 include:

 vSphere Storage APIs for Array Integration (VAAI)

 vSphere Storage APIs for Storage Awareness (VASA)

 Profile-driven storage

 Storage Distributed Resource Scheduler (SDRS)

 Storage vMotion

 Software Fibre Channel over Ethernet (FCoE)

 vSphere Storage Appliance (VSA)

While these valuable new features are beyond the scope of this guide, be aware that you may be facing storage-related issues after you install ESXi 5.0. For example, after mounting a LUN and formatting it with VMFS-5, you may need to determine if additional, array-specific agents are required to support features such as VAAI or VASA.

Note

For more information on ESXi 5.0 features that are supported by a particular HP array, you are advised to consult HP technical support.

In the test environment, Emulex mounted a LUN and formatted it with VMFS-5. The next concern was multipathing.

Configuring multipathing

By default with ESXi 5.0, HP 3PAR Storage uses Fixed path policy for active/active storage arrays. This policy maximizes bandwidth utilization by designating the preferred path to each LUN through the appropriate storage controller.

According to HP documentation, HP 3PAR Storage also supports Round Robin path policy, which can improve storage performance by load-balancing I/O requests between active paths, sending a fixed number of requests through each in turn.

Note

The fixed number of I/Os is user-configurable. You are advised to consult HP technical support for their recommendations.

(22)

You might consider enabling the array’s Asymmetric Logical Unit Access (ALUA) feature, which can improve storage performance in some environments.

Using the OneCommand Manager plug-in for VMware

vCenter

Emulex OneCommand Manager for VMware vCenter integrates the in-depth management of HP 82E 8Gb Dual Port Fibre Channel HBAs into the vCenter management console.

Note

In order for the OneCommand Manager plug-in for VMware vCenter Server to function, you must also install the appropriate Emulex Common Information Model (CIM) provider on your ESXi 5.0 hosts.

For more information on installing the CIM provider and plug-in, refer to Appendix A – Installing the CIM provider

and vCenter plug-in.

Using vCenter to update Emulex adapters on multiple ESXi 5.0 hosts can be a daunting prospect for the VMware administrator. To ease this burden, Emulex has developed a plug-in for vCenter – Emulex OneCommand Manager for VMware vCenter plug-in v1.1 – that provides single-pane-of-glass manageability from within vCenter.

(23)

Figure 9 presents a view of an HP adapter provided by the plug-in.

Figure 9. In this example, the OneCommand Manager plug-in is providing information about an HP 82E 8Gb/s HBA.

The OneCommand Manager plug-in can manage a mix of Emulex-branded and Emulex OEM adapters (such as the HP 82E 8Gb/s HBA).

While OneCommand Manager delivers a range of capabilities, this guide focuses on the plug-in’s ability to perform firmware updates on a per-port basis while adapters are online, without needing to reboot the host. In addition, new logging capabilities added in version 1.1 are outlined.

Note

To activate the new firmware, you must schedule downtime in order to re-boot the host.

(24)

Using the plug-in to perform firmware updates

OneCommand Manager allows you to update the firmware on the HP 82E 8Gb/s HBA without you having to migrate VMs from the host and reboot the server. You can continue working as normal while OneCommand Manager updates the firmware on the host. At some point in the future the host will need to be rebooted so as to activate the firmware; however, you will able to schedule this reboot with the system and vCenter Server administrators.

Some environments include not only Fibre Channel adapters but also adapters created by Emulex to support other connectivity. In this case, you could, for example, select the desired protocol to only update UCNAs on each host.

Figure 10 shows firmware being updated on a particular host.

Figure 10. Firmware being updated on localhost

Best practice

(25)

Using the plug-in to log events

OneCommand Manager also provides the ability to create log files for activities associated with the HP 82E 8Gb/s HBA. After this feature is enabled, events are automatically logged; the associated files can be viewed via vCenter’s Task & Events tab.

Figure 11 shows event logging being enabled.

Figure 11. Utilizing User Preferences to enable event logging for the HP 82E 8Gb/s HBA

One application for event logging is to troubleshoot the status of a particular link – a capability that was missing in earlier versions of OneCommand Manager; however, version 1.1 of the plug-in introduced tighter integration between the plug-in and vCenter.

(26)

In the example shown in Figure 12, the Task & Events tab lists event messages indicating that both HP 82E 8Gb/s HBA links (that is, both ports) are up. The WWNs of these ports are also provided.

Figure 12. vCenter Server showing the results of enabling event logging for the HP 82E 8Gb/s HBA

Performance comparison

As VM density increases, the burden placed on storage by applications running on these VMs will also increase. As a result, Emulex has carried out a series of performance tests to

demonstrate the ability of 8Gb/s Fibre Channel storage to sustain a significantly higher workload than 4Gb/s Fibre Channel without increasing CPU utilization on the host.

Test environment

The test environment used by Emulex included the following components:

 Tested server: ProLiant DL380 G7, configured as follows:

o HBA: Either 8Gb/s (HP 82E 8Gb/s HBA) or 4Gb/s (HP PCIe 4Gb Host Bus Adapter8₎ o RAM: 16 GB

 SAN

o HP 3PAR Utility Storage

o Memory Systems RamSan solid-state drives (SSDs), models 320 and 400

Each VM was configured with two 2GB RDM volumes on the SSDs; the HP 3PAR Storage was used to host datastores for the VMs.

8

(27)

Figure 13 shows how storage was configured in the test environment.

(28)

Test methodology

The Iometer tool was used to stress 4KB/s or 8KB/s Fibre Channel storage with the workloads generated by one or more VMs. Two scenarios were used: application testing and a backup reader test.

Application testing

The following block sizes were used in the application testing:

 512B: To create a baseline for the testing

 4KB and 8KB: To simulate typical workloads such as a mail server or online transactional processing (OLTP) database

 64KB: To simulate a richer multimedia application such as video streaming

Note

This testing was designed to compare the capabilities of 4KB/s and 8KB/s Fibre Channel storage in a specific environment. Results in your environment may differ.

Emulex initiated the testing with 512B blocks to establish a baseline, determining the number of I/Os needed to saturate the Fibre Channel connection between HBA and RDM volume. For subsequent block sizes, Emulex used Iometer to reduce the number of I/Os until, in most cases, the baseline was reached.

(29)

Figure 14 presents IOPS capacities for workloads generated by one, two or four VMs.

Figure 14. Comparing the IOPS delivered by 4Gb/s and 8Gb/s Fibre Channel configurations

Significantly more IOPS capacity was available with 8Gb/s Fibre Channel. For example, with the commonly-used 8KB block size and a single VM, IOPS capacity was 100,000 with 8Gb/s Fibre Channel, compared to 50,000 with 4Gb/s Fibre Channel.

1 VM 4GB/s 1 VM 4GB/s 1 VM 4GB/s 1 VM 8Gb/s 1 VM 8Gb/s 1 VM 8Gb/s 2VMs 4Gb/s 2VMs 4Gb/s 2VMs 4Gb/s 2 VMs 8Gb/s 2 VMs 8Gb/s 2 VMs 8Gb/s 4 VMs 4Gb/s 4 VMs 4Gb/s 4 VMs 4Gb/s 4VMs 8Gb/s 4VMs 8Gb/s 4VMs 8Gb/s 0.00 20,000.00 40,000.00 60,000.00 80,000.00 100,000.00 120,000.00 140,000.00 160,000.00 180,000.00 512B 4K 8K 1 VM 4GB/s 2VMs 4Gb/s 4 VMs 4Gb/s

(30)

Emulex also considered I/O’s per percent of CPU utilization which is IOMeter’s CPU

effectiveness result. This chart illustrates how efficient the I/O subsystem is in CPU utilization. For example, the HP 82E 8Gb/s Fibre Channel adapter using an 8KB block size test shows 128% improvement over HP’s 4Gb/s Fibre Channel adapter.

Figure 15. The chart demonstrates CPU effectiveness with 8Gb/s Fibre Channel vs. 4Gb/s Fibre Channel adapters.

Video streaming and backup reader testing

Emulex also compared the impact of a video streaming and backup on 4Gb/s and 8Gb/s Fibre Channel, again using Iometer to generate the workload. The following configurations were tested:

Video streaming:

 Four VMs with 4Gb/s Fibre Channel

0.00 1,000.00

2,000.00

3,000.00

4,000.00

5,000.00

6,000.00

7,000.00

8,000.00

1 VM 4GB/s 1 VM 8Gb/s 2VMs 4Gb/s 2 VMs 8Gb/s 4 VMs 4Gb/s 4VMs 8Gb/s

(31)

 Four VMs with 8Gb/s Fibre Channel Back up reader:

 Four VMs with 4Gb/s Fibre Channel

 Eight VMs with 8Gb/s Fibre Channel

Both 64KB for video and 256KB blocks for backup were read from the SSDs.

As shown in Figure 16, throughput capacity was significantly higher with 8Gb/s Fibre Channel.

Figure 16. When comparing 8Gb/s Fibre Channel to 4Gb/s Fibre Channel, the 64KB throughput improved 126%

and the 256KB throughput improved 205%.

The HP 82E 8Gb/s HBA supports message signaled interrupts (MSI) and extensions (MSI-X), which are designed to improve host processor utilization and interrupt-handling efficiency. Thus, this HBA can reduce overhead in an MSI-enabled system such as the ProLiant DL380 G7 server. The latency recorded for average response time was 5 milliseconds (ms) when we tested both 64KB and 256KB block size for 8Gb/s Fibre Channel adapter compared to 20 ms on the 4Gb/s Fibre Channel adapter for the same block sizes. For more information, refer to

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/PCI/MSI-HOWTO.txt;hb=HEAD. 0 1000 2000 3000 4000 5000 6000 64K 64K 256K 256K 4Gb 8Gb 4Gb 8Gb

Throughput

4Gb 64K 8Gb 64K 4Gb 256K 8Gb 256K

(32)

As a result, the HP 82E 8Gb/s HBA helps to reduce backup times, as well as increasing throughput and reducing CPU utilization (as shown in Figure 17).

Figure 17. With MSI-X and 8Gb/s Fibre Channel, CPU utilization on the host was approximately 30% lower, even

though twice as many VMs were being supported

0 1000 2000 3000 4000 5000 6000 64K 64K 256K 256K 4Gb 8Gb 4Gb 8Gb

CPU/e

4Gb 64K 8Gb 64K 4Gb 256K 8Gb 256K

(33)

Summary

ESXi 5.0 clearly adds a broad range of features to the hypervisor; however, taking full advantage of these features requires an 8Gb/s SAN in your virtualization environment.

Testing carried out be Emulex indicated that an 8Gb/s Fibre Channel implementation based on the HP 82E 8Gb/s HBA was able to support more VMs than a 4Gb/s implementation, while also reducing CPU utilization on the ESXi 5.0 host.

Planning the deployment of an ESXi 5.0 host – or the migration of an existing host to ESXi 5.0 – is a simple process. Using 8Gb/s storage, the resulting infrastructure will be able to meet the demands of the larger VMs you can now create and will allow you to introduce additional business-critical applications.

(34)

Appendix A – Installing the CIM provider and vCenter plug-in

In order for the OneCommand Manager plug-in for VMware vCenter to function, you must also install the appropriate Emulex Common Information Model (CIM) provider on your ESXi 5.0 hosts.

Note

The VMware Knowledge Base provides an Emulex CIM provider overview for ESXi 5.0. Visit

http://kb.vmware.com/selfservice/microsites/search.do?la

nguage=en_US&cmd=displayKC&externalId=2001548.

The process is as follows:

Note

Ideally, the CIM provider should be installed before deploying any VMs on the particular host. The plug-in can be installed before or after deploying VMs.

1. Download the CIM provider package for ESXi 5.0 to any system that can communicate with the particular host via Secure Shell (SSH). The package is available from the Emulex Downloads site at http://www.emulex.com/downloads/emulex/vmware/vsphere-50/management.html.

2. Install the CIM provider on the ESXi 5.0 host (see Installing the CIM provider). 3. Reboot the host.

4. Install the vCenter plug-in on a remote system (see Installing the vCenter plug-in). 5. Register the host.

6. Use vSphere Client to login to vCenter.

Installing the CIM provider

1. Open the secure shell (SSH) port on the ESXi 5.0 host.

2. Use an FTP client (such as WinSCP) to move the CIM package to directory

/var/log/vmware/ on the ESXi 5.0 host.

3. Use an SSH client (such as PuTTY) to login to the vMA. 4. Login to the ESXi host.

(35)

5. As shown in Figure A-1, install the CIM provider by typing the following:

vi-admin@localhost-> esxcli -s ipaddress software vib install -d /var/log/vmware/name-of-file.zip

Figure A-1. Installing the CIM provider on the ESXi 5.0 host

6. After the CIM provider (VIB files) has been installed, reboot the host.

7. As shown in Figure A-2, verify that the CIM provider has been installed by typing the following:

vi-admin@localhost-> esxcli -s ipaddress software vib list

(36)

Installing the vCenter plug-in

After successfully installing the CIM provider on the desired ESXi 5.0 hosts, you should now install the vCenter plug-in on a remote system. Next, use vSphere Client to login to vCenter to verify that the Emulex OneCommand tab has been added, as shown in Figure A-3.

Figure A-3. The Emulex OneCommand tab has been added to vCenter

Note

There may be additional tabs if you have installed CIM providers for other management applications, such as storage.

(37)

Appendix B – Sample configurations for ESXi 5.0 host and

VMs

This appendix presents the configurations used by Emulex in conjunction to compare the performance of 4Gb/s and 8Gb/s Fibre Channel adapters.

Note

Emulex used the Iometer I/O subsystem measurement and characterization tool as part of the performance testing.

Configurations are provided for the following:

 ESXi 5.0 host – see Table B-1

 Microsoft Windows Server 2008 R2 VMs (VM1 – VM10) – see Table B-2 Table B-1. ESXi 5.0 host configuration

Component Description

Hypervisor ESXi 5.0 Build 456551

Host server HP ProLiant DL380 G7 server

HBA HP 82E 8Gb/s HBA

Local datastore Storage 1

Storage array HP 3PAR Utility Storage

SAN fabric switch Brocade 8Gb/s Fibre Channel switch

Table B-2. Typical VM configuration

Hardware Summary

Memory 2048 MB

CPUs 2

Video card Video card

SCSI controller 0 LSI Logic SAS

Hard disk 1, 2 Mapped Raw LUN

CD/DVD drive 1 ide1:0

Network adapter 1 VM Network 2

Floppy drive 1 Floppy0

(38)

Appendix C – Configuring BfS

This appendix outlines the process for configuring BfS:

1. Collaborate with your SAN administrator on provisioning a boot LUN and presenting it to the ESXi 5.0 host.

2. Download the Universal Boot Code firmware for your adapter, available at

http://h20000.www2.hp.com/bizsupport/TechSupport/ProductList.jsp?lang=en&cc=us&prodT ypeId=12169&prodSeriesId=3662826&taskId=135. For example, select HP 82E 8Gb Dual

Port PCIe Fibre Channel Host Bus AdapterCross operating system (BIOS, Firmware, Diagnostics, etc.).

3. Power on the server and, at the prompt, select Ctrl-E. 4. Install the boot code firmware.

5. Reboot the host and, at the prompt, select Ctrl-E.

6. Specify the adapter port from which the system will be booting. 7. Scan the array and select the boot LUN.

8. Save the settings and reboot the host. 9. Insert the ESXi 5.0 CD into the CD drive. 10. Initiate the install.

(39)

For more information

Migrating to VMware ESXi http://www.vmware.com/files/pdf/VMware-ESXi-41-Migration-Guide-TWP.pdf Performance best practices for VMware

vSphere 5.0

http://www.vmware.com/pdf/Perf_Best_Practic es_vSphere5.0.pdf

HP virtualization with VMware, including a section on ProLiant servers

www.hp.com/go/vmware

HP storage solutions for VMware http://h71028.www7.hp.com/enterprise/w1/en/ solutions/storage-vmware.html

HP offline bundle for ESXi 5.0

HP 3PAR Utility Storage with VMware vSphere http://h20000.www2.hp.com/bizsupport/TechS upport/SoftwareDescription.jsp?lang=en&cc=u s&prodTypeId=15351&prodSeriesId=4091412 &swItem=MTX-dd492ace50c6427389678df8be&prodNameId =4091432&swEnvOID=4115&swLang=8&taskI d=135&mode=4&idx=1 http://www.vmware.com/files/pdf/techpaper/vm w-vsphere-3par-utility-storage.pdf

To help us improve our documents, please provide feedback at implementerslab@emulex.com.

© Copyright 2012 Emulex Corporation. The information contained herein is subject to change without notice. The only warranties for Emulex products and services are set forth in the express warranty statements accompanying such products and services. Emulex shall not be liable for technical or editorial errors or omissions contained herein.

LightPulse and OneCommand are registered trademarks of Emulex Corporation. HP is a registered trademark in the U.S. and other countries.

VMware is a registered trademark of VMware Corporation.

(40)

World Headquarters 3333 Susan Street, Costa Mesa, California 92626 +1 714 662 5600 Bangalore, India +91 80 40156789 | Beijing, China +86 10 68499547

Dublin, Ireland+35 3 (0)1 652 1700 | Munich, Germany +49 (0) 89 97007 177 Paris, France +33 (0) 158 580 022 | Tokyo, Japan +81 3 5322 1348 Wokingham, United Kingdom +44 (0) 118 977 2929