Virtualizing Exchange Server 2007 Using VMware vSphere 4 on the Hitachi
Adaptable Modular Storage 2500
August 2009
Tested Deployment and Best Practices Guide
By Patricia Brailey, Patrick Flueckiger, Bob Ingram and Larry Meese
Summary
As organizations battle unoptimized storage and server infrastructures supporting their Exchange Server 2007 deployments, they seek a solution that allows them to match their application and business requirements to optimal virtual machine and storage designs; reduce costs by consolidating storage and server footprints;
deliver scalable, predictable performance that is easy to evolve, deploy and manage; and provide enhanced levels of availability, reliability and IT agility.
This white paper outlines a validated building block approach that leverages the combined capabilities of vSphere 4 and the Hitachi Adaptable Modular Storage 2500 to deliver an Exchange Server 2007 infrastructure that presents organizations scenarios to achieve those important goals.
It provides best practice recommendations based on Hitachi Data Systems’ testing of a virtual machine building block architecture that is designed to optimize memory, storage, and CPU utilization while reducing server sprawl and simplifying management of the Exchange Server 2007 infrastructure. The building block approach is applied to traditional storage configurations made up of standard RAID groups and logical units. In addition, Hitachi Dynamic Provisioning software, an intelligent storage technology that provides advanced wide-striping and thin-provisioning capabilities, is introduced and explained. Incorporating Hitachi Dynamic Provisioning software into the architecture leverages virtual machines, storage pools and virtual volumes as the fundamental building blocks.
For best results use Acrobat Reader 8.0.
Contributors
The information included in this document represents the expertise, feedback and suggestions of a number of skilled practitioners. The authors recognize and sincerely thank the following contributors and reviewers of this document:
• Heidi Biggar
• Steve Burr
• John Harker
• Ron Lee
• Lisa Pampuch
• Rob Simmons
Table of Contents
Solution Overview ... 6
Solution Components ... 9
Building Block Architecture ... 11
Multipathing ... 13
Disk Management ... 13
Scaling the Exchange Server 2007 Mailbox Server Role ... 14
Tested Deployment ... 16
Software ... 16
Hardware ... 17
Path Configuration ... 17
Storage Configuration for Standard RAID Groups and LUs ... 18
Storage Configuration Using Hitachi Dynamic Provisioning Software ... 19
Test Method ... 20
Best Practices ... 21
Conclusion ... 21
Virtualizing Exchange Server 2007 Using VMware vSphere 4 on the Hitachi Adaptable Modular Storage 2500
Tested Deployment and Best Practices Guide
Server virtualization is enjoying rapid adoption in the 21st-century data center as organizations continue to plan for, validate and realize the benefits this technology can provide. As organizations become more familiar, confident and capable with virtual machine deployments, they naturally investigate the feasibility of applying server virtualization to mission-critical applications in pursuit of similar or even enhanced benefits. E-mail is arguably a vital application for nearly all organizations operating in today’s global marketplace and Microsoft® Exchange Server 2007 is the messaging platform of choice for many of these enterprises.
Exchange Server 2007 introduced a number of new features, enhancements and advancements, such as a shift to a 64-bit platform, server roles, and architectural changes that allow it to better leverage the expanding capabilities and resources of server hardware. However, many IT operations regularly battle unoptimized storage and server infrastructures supporting their Exchange Server 2007 deployments that can result in the following problems:
• Scale-out server sprawl as messaging demands continue to grow and traditional or existing infrastructures reach their limitations
• Increased administrative burden and operational costs associated with managing isolated storage islands or silos like routinely migrating mailboxes in search of available performance resources
• Limited or no ability to scale traditional or existing infrastructure designs, especially those built upon direct-attached storage (DAS)
• Degraded service levels and end-user experiences that damage productivity levels
These factors, coupled with the advances in Exchange Server 2007 and server hardware and Microsoft’s introduction of the Server Virtualization Validation Program (SVVP), make Exchange Server 2007 an ideal choice for consolidation and virtualization initiatives. However, successfully deploying Exchange Server 2007 in a virtualized, consolidated fashion requires an advanced infrustructure consisting of industry-leading hypervisor software deployed on capable server hardware and intelligent, centralized storage hardware and software. VMware vSphere 4 and the Hitachi Adaptable Modular Storage 2500 provide an optimal environment for the management of virtual machines, and provide organizations with the technologies necessary to reap tangible business benefits that consolidation and virtualization initiatives can deliver.
This white paper outlines a validated building block approach that leverages the combined capabilities of vSphere 4 and the 2500 to deliver an Exchange Server 2007 infrastructure that presents organizations achievable scenarios to achieve these important goals:
• Match application and business requirements to optimal virtual machine and storage designs to improve resource utilization rates
• Reduce costs by consolidating storage and server footprints, thus lowering power, cooling, and management costs associated with sprawling Exchange Server 2007 mailbox infrastructures
• Deliver scalable, predictable performance that is easy to evolve, deploy and manage and that can handle I/O spikes during peak usage periods
• Provide enhanced levels of availability, reliability and IT agility
It provides best practice recommendations based on Hitachi Data Systems’ testing of a virtual machine building block architecture that is designed to optimize memory, storage, and CPU utilization while reducing server sprawl and simplifying management of the Exchange Server 2007 infrastructure. This solution aligns with best practice recommendations from both Microsoft and VMware. The building block approach is applied to traditional storage configurations made up of standard RAID groups and logical units (LUs). In addition, Hitachi Dynamic Provisioning software, an intelligent storage technology that provides advanced wide-striping and thin-provisioning capabilities, is introduced and explained.
Incorporating Hitachi Dynamic Provisioning software into the architecture leverages virtual machines, storage pools and virtual volumes (V-VOLs) as the fundamental building blocks.
This white paper is intended for use by IT professionals charged with Exchange, VMware or storage responsibilities. It assumes familiarity with VMware vSphere 4 and vCenter, Hitachi Storage Navigator Modular 2 software, Microsoft Windows Server 2008 and Microsoft Exchange Server 2007.
Solution Overview
For ease of management, scalability, and to provide predictable performance, this solution uses a building block approach, a familiar concept that Hitachi uses for current and previous generations of Exchange Server and the Adaptable Modular Storage family. A single virtual machine (VM) running Windows Server 2008 and Exchange Server 2007 Service Pack 1 mailbox server role with underlying storage from the Hitachi Adaptable Modular Storage 2500 make up a building block for purposes of this paper. VMware recommends deploying virtual machines on shared storage like the 2500 to take
advantage of VMware features such as VMotion, VMware High Availability (HA), and VMware Distributed Resource Scheduler (DRS).
Logical units (LUs) from the 2500 are allocated to the ESX hosts and formatted as VMFS volumes from which virtual disks are created. The virtual disks (vDisks) are presented to the Windows Server 2008 guest operating system and can be partitioned and used in NTFS file systems for the Exchange Server 2007 databases and transaction logs. The solution building block supports up to 1,824 very heavy users, which corresponds to a user profile of 0.576 IOPS with a 2GB mailbox quota according to Microsoft recommendations for planning and testing purposes.
Table 1 describes the building block architecture that supports up to 1,824 very heavy Exchange users in a standard provisioned environment.
Table 1. Building Bock Architecture Supporting 1,824 Very Heavy Exchange Users Using Standard LUs
Resource Details ESX hosts 2 physical hosts
32GB memory minimum 2 CPU cores minimum Virtual machines 1
16GB memory 1 CPU
Storage processor Hitachi Adaptable Modular Storage 2500 50 300GB 15K RPM SAS disks
Note: 2 disks are utilized as hot spares RAID groups 8 RAID-1+0, 2D+2D
5 RAID-1, 1D+1D
1 RAID-5, 5D+1P for OS drives
LUs SG LUs - 8 535GB on RAID-1+0
Log LUs - 5 276GB on RAID-1 OS LU - 1 80GB on RAID-5D+1P Exchange
configuration
85GB Exchange database size 38 mailboxes per storage group 6 storage groups per LU 10 logs per LU
Table 2 describes the building block architecture that supports up to 1,824 very heavy Exchange users in an environment provisioned using Hitachi Dynamic Provisioning Software.
Table 2. Building Bock Architecture Supporting 1,824 Very Heavy Exchange Users Using Dynamic Provisioned LUs
Resource Details ESX hosts 2 physical hosts
32GB memory minimum 2 CPU cores minimum Virtual machines 1
16GB memory 1 CPU
Storage processor Hitaci Adaptable Modular Storage 2500 50 300GB 15K RPM SAS disks
Note: 2 disks are utilized as hot spares RAID groups 11 RAID-1+0, 2D+2D in DP-Pool
1 RAID-5, 5D+1P for OS drives
LUs SG LUs - 8 535GB
Log LUs - 5 276GB
OS LU - 1 80GB on RAID-5D+1P Exchange
configuration
85GB Exchange database size 38 mailboxes per storage group 6 storage groups per LU 10 logs per LU
Each Exchange storage group contains one database. Each database is 85GB in size and has 38 mailboxes.
When using Hitachi Dynamic Provisioning software, use a single pool of 11 RAID-1+0 2+2 groups for the Exchange storage groups and logs. The use of a single pool provides the best performance by striping the LUs across all disks in the DP pool. Wide striping reduces or eliminates hot spots caused by skewed workloads. In cases where log LUs are less busy than database LUs, I/O is distributed across all disks and therefore more IOPS capability is available. RAID protection is provided to the DP pool through the use of multiple RAID-1+0 2D+2D groups.
The ESX hosts used in this test environment contain 128GB of memory and four quad-core AMD opteron CPUs. Two ESX hosts are used at a minimum to achieve high availability. Three VMs can be added to one ESX host along with additional RAID groups on the 2500 to scale up to 5,472 very heavy users.
Depending on the usage patterns, more VMs can be added to a single ESX host. To scale beyond this, additional ESX hosts need to be added, ensuring that sufficient ESX hosts are employed to achieve redundancy or high availability. This paper includes details regarding the test environment and methods used to validate the Exchange Server 2007 mailbox server role and storage infrastructure. For more information about how to deploy and configure the environment, see the Virtualizing Exchange Server 2007 Using VMware vSphere 4 on the Hitachi Adaptable Modular Storage 2500 Deployment Guide.
This solution focuses specifically on the Exchange Sever 2007 mailbox role. The other Exchange roles, such as hub transport, client access and the edge server role, can be deployed on virtual machines as well, however, these topics are out of scope for this paper. Deploying the full Exchange environment on VMware provides the greatest overall benefit to your organization by reducing expenditures for physical servers and data center resources and simplifying management of your data center. For more
information, see VMware’s Deployment Guide: Microsoft Exchange Solutions of VMware white paper.
Solution Components
This section describes the key components that make up this solution.
Hitachi Adaptable Modular Storage 2500
The Hitachi Adaptable Modular Storage 2500 is the only midrange storage system with dynamic load balancing and symmetric active-active controllers. It provides integrated, automated hardware-based front to back end I/O load balancing that, when coupled with VMware round-robin load balancing, eliminates many complex and time consuming tasks that storage administrators typically face. This ensures I/O traffic to back-end disk devices is dynamically managed, balanced and shared across both controllers. No other midrange storage product that scales beyond 100TB has a serial attached SCSI (SAS) drive interface. The new point-to-point back end design virtually eliminates I/O transfer delays and contention associated with Fibre Channel arbitration and provides significantly higher bandwidth and I/O
concurrency.
Although the Hitachi Adaptable Modular Storage 2500 was used in the testing of this solution, the information in this paper is relevant and the building block approach can be applied to the other 2000 family members with the proper changes to account for capacity and performance differences. The 2500 is an easy-to-use, scalable, cost effective storage system for mission-critical business applications like Exchange Server 2007. It is also a top choice for tiered and standalone storage, consolidation, business continuity, data replication, backup and archiving. The 2500 offers a rich set of features in a model that scales to 480 disk drives and delivers enterprise-class performance and capabilities at a modular price.
Table 3 lists some of the 2500’s specification options.
Table 3. Hitachi Adaptable Modular Storage 2500 Specification Options
Raw capacity 472TB SATA
210TB SAS Internal disk drives (SAS unless otherwise noted) 146GB (15K RPM)
300GB (15K RPM) 400GB (10K RPM) 450GB (15K RPM)
500GB SATA II (7200 RPM) 1TB SATA II (7200 RPM) Disk drive interfaces SAS and SATA
Host interfaces Fibre Channel: 8Gb/sec or 4Gb/sec iSCSI: GigE
Maximum host connections 16 Fibre Channel or 8 iSCSI
or 8 Fibre Channel and 4 iSCSI Maximum attached hosts through virtual ports 2,048
SAS links 32
Maximum number of LUs 4,096
Maximum LU size 60TB
Controller cache (per system) 16GB to 32GB
Hitachi Dynamic Provisioning Software
On Hitachi Adaptable Modular Storage 2000 family systems, Hitachi Dynamic Provisioning software provides a thin provisioning feature that provides virtual storage capacity to eliminate application service interruptions, reduce costs and simplify administration, including these:
• Optimizes or “right-sizes” storage performance and capacity based on business or application requirements.
• Supports deferring storage capacity upgrades to align with actual business usage.
• Simplifies and adds agility to the storage administration process.
• Provides performance improvements through automatic optimized wide striping of data across all available disks in a storage pool.
For more information, see the Hitachi Dynamic Provisioning Software datasheet.
The wide-striping technology that is fundamental to Hitachi Data Provisioning software dramatically improves performance, capacity utilization and management of your environment. By deploying your Exchange Server 2007 mailbox server using V-VOLs from Hitachi Dynamic Provisioning storage pools on the 2000 family, you can expect the following benefits:
• An improved I/O “buffer” to burst into during peak usage times or intense maintenance activities like content indexing or database integrity checks
• A smoothing effect to the Exchange workload that can eliminate hot spots, resulting in reduce mailbox moves related to performance or capacity constraints
• Elimination of excess, unutilized capacity by leveraging the combined capabilities all disks comprising a storage pool
• Elimination of the need to manage the placement of power users and worrying about which users currently use or might be getting a Blackberry or Windows Mobile device
VMware vSphere 4
VMware vSphere 4 can help reduce hardware footprints and capital expenses dramatically through server consolidation. Utilizing VMware products and features such as VMware ESX, VMware vCenter Server, High Availability (HA), Distributed Resource Scheduler (DRS) and Fault Tolerance (FT), vSphere allows for a robust environment, centralized management and gives administrators control over key capabilities.
Exchange Server 2007
Exchange Server 2007 with Service Pack 1 provides a reliable messaging system that allows users throughout an organization to access e-mail, voice mail, calendars, and contacts from any location. In this solution, the Exchange Sever mailbox role is installed on a Windows 2008 virtual machine. Multiple virtual machines are deployed on a single ESX host to accommodate the required number of users. Other Exchange Server 2007 roles — such as hub transport, client access, and the edge server — can be deployed on virtual machines as well. For more information about deploying non-mailbox Exchange server roles, see the Deployment Guide: Microsoft Exchange Solutions on VMware white paper.
Building Block Architecture
Fundamentally, designing server and storage infrastructures to support the Exchange Server 2007 mailbox server role in a virtual server environment is no different than performing the same activities for a physical environment. Deploying Exchange using a building block approach allows you to easily manage and scale your environment. Additional VMs are easy to deploy using templates, and storage can be provisioned on a per RAID group basis or by extending the HDP pool and adding DP-VOLs. Use the 1,824 user count with the specified very heavy user profile that a single building block can support as a guidepost. Adjust it accordingly to account for the factors that make each Exchange Server 2007 environment unique, such as planned and unanticipated growth, protection methods and service level agreements. Hitachi Data Systems used the heaviest profile with a large mailbox size for this architecture intentionally in an effort to demonstrate the efficiency and flexibility of the design.
The standard LU architecture uses eight RAID-1+0, 2D+2D groups created using Hitachi Storage Navigator Modular 2 software. Each RAID1+0 group contains one LU for the Exchange databases. This architecture also uses five RAID-1 groups, each containing one LU for the Exchange transaction logs.
The Hitachi Dynamic Provisioning software architecture uses 11 RAID-1+0, 2D+2D in a single DP pool, eight 535GB DP-VOLs for storage groups and 5 267GB DP-VOLs for logs. Storage for the Exchange databases and transaction logs is sized first for performance, then capacity which is no different than sizing Exchange for physical server deployments. This architecture requires the following calculations be derived from the requirements of an existing or planned Exchange Server 2007 environment:
• IOPS needed for Exchange databases and transaction logs
• Capacity needed for databases
• Capacity needed for transaction logs
• Overhead for Exchange databases
Note that overhead must accommodate the Exchange database dumpster, which stores hard-deleted items from users’ mailboxes, and white space in the Exchange database. For more information about calculating the storage capacity needed for your Exchange Server 2007 environment, see the Planning for Microsoft® Exchange Server 2007 Deployments on the Hitachi Adaptable Modular Storage 2000 Family white paper.
LUs are presented to the VMware ESX host and made into a VMFS. Figure 1 shows how the physical disks from the Hitachi Adaptable Modular Storage 2500 map to the ESX hosts.
Figure 1. Physical to Virtual Disk Mapping
. Design Goals
The building block architecture that was validated in this solution is designed to achieve the following goals:
• Reach 50 percent CPU utilization on virtual machines
• Optimize storage configuration on the 2500 for best I/O throughput and ease of management for both standard and Hitachi Dynamic Provisioning configurations
• Deliver at least 20 percent more IOPS for the Microsoft-specified user profile within 15ms and 5ms response time respectively for database reads and writes as tested, measured and reported by the Microsoft Jetstress tool as well as to accommodate burst and peak activities.
• Delliver at least 80 percent disk capacity utilization for the database volumes, while maintaining disk I/O utilization rates at less than 50 percent
In addition to the design goals, several other factors contributed to the considerations and trade-offs to arrive at the standard provisioned building block architecture, including these:
• Microsoft recommends that mailbox databases be no greater than 100GB when its native continuous replication technologies are not enabled on a storage group.
• For a very heavy user profile, Microsoft recommends a large mailbox size of 2GB.
• The maximum number of storage groups and databases an Exchange Server 2007 instance can support is 50.
• Granularity of scale is determined by the LU size; adding a single 535GB LU for the storage groups and additional storage for the transaction logs supports an additional 228 users, as follows:
– In a standard LU configuration, one RAID-1+0 group of 2D+2D for databases and one RAID-1 group of 1D+1D can be added, containing one LU each.
– In a dynamic provisioned configuration, DP-Pool space can be allocated by adding one RAID-1+0 group to the DP pool.
• VMware virtual machines support a total of 60 vDisks; smaller LUs might support more granular scaling by adding fewer users at a time, but you are then be limited by the number of users per virtual machine because you can have only 60 vDisks.
• ESX hosts support a total limit of 256 LUs and all LUs must be masked to all ESX hosts in a VMware cluster scenario.
Multipathing
To maintain a constant connection between the ESX hosts and storage, ESX supports multipathing.
Multipathing is a methodology that allows multiple physical or logical connections from the host to the storage. To support multipathing Hitachi Data Systems recommends that the physical host contain at least two HBAs that connect to at least two Fibre Channel ports on the storage processor.
In ESX, several types of multipathing policies are available through the VMware Native Multipathing Plugin (NMP):
• Fixed (Default) — Uses the designated preferred path, if it is configured. Otherwise, it uses the first working path discovered at system boot time. If the host cannot use the preferred path, it selects a random alternative available path. The host automatically reverts to the preferred path as soon as that path becomes available.
• Round-robin (Recommended) — Uses a path selection algorithm that rotates through all available paths, enabling load distribution across the paths.
Round-robin is the best choice for the Adaptable Modular Storage 2000 familly of storage systems due to their symmetric active-active controller design.
• Most recently used (MRU) — Selects the path the ESX host used most recently to access the given device. If this path becomes unavailable, the host switches to an different path and continues to use the new path while it is available.
MRU is a good choice when a single path I/O flow is desired and if the storage processor has active- active controllers.
Disk Management
ESX hosts can access LUs in two ways, through VMFS or raw device mapping (RDM). Virtual disks are files stored on a datastore, which is a logical container for the .vmdk files. The datastores are deployed on storage devices and make up the virtual machine file system (VMFS). The VMFS file system is optimized for virtual machines and hides the specifics of the underlying storage.
The VMFS can be accessed by several ESX hosts and the cluster feature allows for distributed file locking for the virtual machines. The VMFS can be extended while the client is running and can extend across multiple LUs. Using Storage vMotion, the .vmdk file can be moved to another datastore
nondisruptively.
As an alternative, LUs can be mapped as a raw device mapping (RDM). This feature allows a LU to be mapped directly to a virtual machine. RDMs are useful for command devices and any other device that requires direct communication to the storage processor. Hitachi Data Systems recommends against using RDM for Exchange because the enhanced performance and features of the VMFS are not available.
For more information about VMFS and RDM, see VMware’s Fibre Channel SAN Configuration Guide.
Scaling the Exchange Server 2007 Mailbox Server Role
To scale the Exchange Server 2007 mailbox role, you must add virtual machines and storage. Calculate the maximum number of VMs that a single ESX host will run by noting the total number of CPUs and total memory required. Do not over-subscribe CPU or memory and keep in mind that the ESX service console requires one CPU and at least 400MB of memory.
In a production implementation, it’s best to utilize the tools available to examine resource utilization:
• vCenter Performance tab — Examine the CPU and memory utilization to determine if sufficient headroom is available.
• Hitachi software — Use Hitachi Tuning Manager or Hitachi Storage Navigator Modular 2 software’s Performance Monitor feature to examine the 2500’s CPU, Fibre Channel port and disk utilization.
If you need a high availability Exchange solution, use at least two physical servers in your production environment to enable VMware HA. Use enterprise class servers that contain enough processor and memory resources to accommodate a failover situation. The servers must have enough memory and processor power to operate the entire Exchange environment in the event that a server fails.
Scaling Exchange Server 2007 Using the Building Block Approach
Using the very heavy I/O profile and 2GB mailboxes, a single 2500 scaled to approximately 16,000 to 18,000 users before consuming the available capacity from the full complement of 480 300GB 15K SAS drives in the testing configuration. Although the 2500 has ample performance capabilities available to support additional users in this scenario, the 2GB user mailbox size is the limiting factor in the ability to scale beyond 16,000 to 18,000 users. In other words, this design becomes capacity bound in scaling this particular architecture because of the chosen user profile. Table 4 lists the building block resources needed to scale the architecture to support a given number very heavy users users with 2GB mailboxes.
Table 4. Building Block Resources for 3,648, 10,944 and 16,416 Exchange Users
Resource Details for 3,648 Users Details for 10,944 Users Details for 16,416 Users Virtual machines 2
32GB memory 2 vCPU
6
96GB memory 6 vCPU
9
144GB memory 9 vCPU Storage systems 1 Hitachi Adaptable
Modular Storage 2500 86 300GB 15K RPM SAS disks
Note: 2 disks are used as hot spares
1 Hitachi Adaptable Modular Storage 2500 254 300GB 15K RPM SAS disks
Note: 2 disks are used as hot spares
1 Hitachi Adaptable Modular Storage 2500
380 300GB 15K RPM SAS disks
Note: 2 disks are used as hot spares
RAID groups 16 RAID-1+0, 2D+2D 10 RAID-1, 1D+1D
48 RAID-1+0, 2D+2D 30 RAID-1, 1D+1D
72 RAID-1+0, 2D+2D 45 RAID-1, 1D+1D
LUs 16 535GB on RAID-1+0
10 276GB on RAID-1
48 535GB on RAID-1+0 30 276GB on RAID-1
72 535GB on RAID-1+0 30 276GB on RAID-1
To support 20,000 users or more with the same user profile and mailbox size, a second 2500 is required and three to four physical ESX hosts utilizing a total of 11 mailbox building blocks. Each physical server can support up to approximately 7,296 users on four virtual machines. This provides the overhead needed in the case of planned or unplanned downtime; all virtual machines have resources available on three physical machines. Figure 2 illustrates a 20,000 user deployment.
Figure 2. 20,000 User Deployment
Tested Deployment
The following sections describe the configuration used for building and validating the virtualized Exchange deployment documented in this guide. Figure 3 illustrates the test environment topology.
Figure 3. Test Environment Topology
Software
Table 5 describes the software required for the tested deployment described in this white paper.
Table 5. Software Resources
Software Version
ESX 4.0.0, build 140815
vCenter 4.0 140742
Windows Server 2008, Service Pack 1
Exchange Server 2007, Service Pack 1
Hitachi Storage Navigator Modular 2 7.0
Hitachi Tuning Manager 6.1.0-00
In addition, for the tested deployment using Hitachi Dynamic Provisioning software, the license key was installed on the Hitachi Adaptable Modular 2500.
Hardware
The solution building block is designed to optimize resource utilization and throughput of the 2500 and the vSphere server. Table 6 lists the hardware used in testing this architecture.
Table 6. Hardware Resources
Hardware Configuration
Hitachi Adaptable Modular Storage 2500 Microcode 0870H 2 controllers 10 disk trays
15 300GB 15K RPM SAS disks per tray 16GB cache per controller
Brocade 48000 SAN director 8 4Gb Fibre Channel ports used
Two Dell R905 servers 4 quad core AMD Opteron 1.9GHz processors
128GB memory
2 Emulex LPe11002 4Gb Fibre Channel HBAs
Path Configuration
All LUs are masked and zoned to all four HBAs on each of the ESX hosts using four dedicated Fibre Channel ports on the 2500. Table 7 lists the connections between the ESX hosts and the Fibre Channel ports.
Table 7. Path Configuration
ESX Host Fibre Channel Ports
1 0A, 0B,1A, 1B
2 0C, 0D, 1C, 1D
3 0E, 0F, 1E, 1F
ESX hosts use VMware’s round-robin multipathing algorithm.
Storage Configuration for Standard RAID Groups and LUs
This virtualized Exchange solution uses one Hitachi Adaptable Modular Storage 2500, as illustrated by Figure 4.
Figure 4. Standard Provisioned Storage Building Block
The following sections describe the standard configuration for each building block.
RAID Groups and LUs
Table 8 lists the configuration of RAID groups and LUs for a single building block.
Table 8. Detailed Storage Configuration for a Single Building Block
RAID Group
LUN Size (GB)
RAID Level
RAID Type
Disk Spec Description
1 5 535 RAID-1+0 2+2 300GB 15K Storage groups 1-6 2 6 535 RAID-1+0 2+2 300GB 15K Storage groups 7-12 3 7 535 RAID-1+0 2+2 300GB 15K Storage groups 13-18 4 8 535 RAID-1+0 2+2 300GB 15K Storage groups 19-24 5 9 535 RAID-1+0 2+2 300GB 15K Storage groups 25-30 6 10 535 RAID-1+0 2+2 300GB 15K Storage groups 31-36 7 11 535 RAID-1+0 2+2 300GB 15K Storage groups 37-42 8 12 535 RAID-1+0 2+2 300GB 15K Storage groups 43-48 9 13 267 RAID-1 1+1 300GB 15K Logs 1-10
10 14 267 RAID-1 1+1 300GB 15K Logs 11-20 11 15 267 RAID-1 1+1 300GB 15K Logs 21-30
RAID Group
LUN Size (GB)
RAID Level
RAID Type
Disk Spec Description
12 16 267 RAID-1 1+1 300GB 15K Logs 31-40 13 17 267 RAID-1 1+1 300GB 15K Logs 41-48
Storage Configuration Using Hitachi Dynamic Provisioning Software
This virtualized Exchange solution uses one Hitachi Adaptable Modular Storage 2500 with the Hitachi Dynamic Provisioning software configuration for each building block described in this section.
Using Hitachi Dynamic Provisioning software, the I/O workload is distributed across all drives in the HDP pool, which consists of 11 RAID-1+0, 2+2 groups. All LUs for the building block using Hitachi Dynamic Provisioning software are allocated from this single HDP pool.
Table 9 lists the configuration of RAID groups and LUs for a single building block using Hitachi Dyanmic Provisioning software. Note that all storage is in HDP pool 0.
Table 9. Detailed Storage Configuration for a Single Building Block using Hitachi Dynamic Provisioning Software
LU Size Description
5 535GB Storage groups 1-6 6 535GB Storage groups 7-12 7 535GB Storage groups 13-18 8 535GB Storage groups 19-24 9 535GB Storage groups 25-30 10 535GB Storage groups 31-36 11 535GB Storage groups 37-42 12 535GB Storage groups 43-48
13 267GB Logs 1-10
14 267GB Logs 11-20 15 267GB Logs 21-30 16 267GB Logs 31-40 17 267GB Logs 41-48
Hitachi Data Systems recommends that you create a DP pool for each building block.
Test Method
Jetstress exercised the storage subsystem beginning with 48 storage groups for one building block and then additional VM and storage building blocks were added to scale up the configuration. Results were collected and the number of building blocks increased until the design goal was achieved. Table 10 lists the Jetstress parameters used in testing.
Table 10. Jetstress Test Parameters
Parameter Description
Test scenario Exchange mailbox profile Number of mailboxes 1824
IOPS per mailbox 0.576 Mailbox size (MB) 2048
Thread count 4
Test type Performance
Test duration 2 hours
Table 11 describes the Jetstress success criteria established by Microsoft.
Table 11. Jetstress Success Criteria
Criteria Description
Achieved IOPS 1824 x .576 = 1050.5 Database Avg. Disk sec/read Average < 20ms
Maximum < 50ms Database Avg. Disk sec/Write Average < 20ms Log Avg. Disk sec/Read Average < 20ms Maximum < 50ms Log Avg. Disk sec/Write Average < 10ms
Maximum < 50ms
Testing shows that this solution meets or exceeds all design goals.
After a successful completion of the Jetstress disk performance and stress tests in a non-production environment, you can be confident that your Exchange 2007 disk storage system is adequately sized in terms of the supplied performance criteria for the user count and user profiles you establish. However, you need to validate all Exchange Server 2007 roles and the supporting infrastructure components that make up the overall environment with the Microsoft LoadGen tool before transitioning to production.
Best Practices
These best practices for the design of the virtualized Exchange environment are based on Hitachi Data Systems’ testing of the building block architecture. For a deployment of Exchange 2007 using VMware vSphere 4 on a 2500, follow these guidelines:
• For enhanced storage utilization and usability, use Hitachi Dynamic Provisioning software.
• When using standard LUs:
– Use RAID-1+0, 2D+2D for Exchange storage groups (with one database per storage group) – RAID-1 1D+1D for logs.
• When using DP LUs:
– Use RAID-1+0 for all pools.
– Adjust the size of the RAID group for scalability. When using 300GB disks, 2D+2D RAID groups provide the best scaling option for Exchange 2007 environments.
– Create a new HDP pool when adding a building block to the Exchange environment.
– Add capacity to the HDP pool in building block increments rather than by single RAID groups; that is, add 11 RAID-1+0 2D+2D groups (44 disks). This ensures adequate capacity to avoid reaching the pool full threshold and over-commitment scenarios.
• Install all Exchange server roles on virtual machines if possible to increase the agility and availability of Exchange while lowering costs.
• Use a minimum of two vSphere servers to take full advantage of vSphere features such as High Availability, Dynamic Resource Allocation and Fault Tolerence.
• Configure at least two redundant paths to the 2500 from each ESX host.
• Use a building block architecture for easy scaling.
• Use round-robin multipathing policy to take best advantage of the 2500’s symmetric active-active controllers.
• Use virtual disks for disk management.
Conclusion
Understanding the tested deployment and following the best practice recommendations in this white paper can help your organization enjoy the benefits of virtualizing Exchange Server 2007 with VMware vSphere 4.0 and the Hitachi Adaptable Modular Storage 2500. Those benefits include reduced data center footprint, increased availability, reduced complexity, simplified management and scalability, and reduced power consumption. The building block architecture described in this document optimizes memory, storage and CPU utilization.
For more information about Hitachi products and solutions, see the Hitachi Data Systems Web site, your sales representative or a channel partner.
Corporate Headquarters 750 Central Expressway, Santa Clara, California 95050-2627 USA Contact Information: + 1 408 970 1000 www.hds.com / info@hds.com
Asia Pacific and Americas 750 Central Expressway, Santa Clara, California 95050-2627 USA Contact Information: + 1 408 970 1000 www.hds.com / info@hds.com
Europe Headquarters Sefton Park, Stoke Poges, Buckinghamshire SL2 4HD United Kingdom Contact Information: + 44 (0) 1753 618000 www.hds.com / info.uk@hds.com
Hitachi is a registered trademark of Hitachi, Ltd., in the United States and other countries. Hitachi Data Systems is a registered trademark and service mark of Hitachi, Ltd., in the United States and other countries.
All other trademarks, service marks and company names mentioned in this document or Web site are properties of their respective owners.
Notice: This document is for informational purposes only, and does not set forth any warranty, expressed or implied, concerning any equipment or service offered or to be offered by Hitachi Data Systems. This document describes some capabilities that are conditioned on a maintenance contract with Hitachi Data Systems being in effect and that may be configuration dependent, and features that may not be currently available. Contact your local Hitachi Data Systems sales office for information on feature and product availability.
© Hitachi Data Systems Corporation 2009. All Rights Reserved.
AS-014-00 August 2009
