Performance and Sizing Considerations

(1)

Performance and Sizing

Considerations

Chin, Kar Wai

Global DC Services Delivery Manager

Email : [email protected]

Http://www.trw.com

(2)

Performance and Sizing Considerations

Understanding what’s a Virtual Machine

Candidate Selections

Building your application on VM e.g. Exchange, SAP

Sizing/Performance consideration for your VM

Support from major ISV

(3)

Anatomy of a Virtual Machine

Each Virtual Machine is a complete system encapsulated in a

set of software files

Virtual Machines Unmodified Application Unmodified (X86) Physical Server ESX Server Unmodified OS Virtual Hardware

(4)

Candidate Selections

Non-Candidates

Poor-Candidates

Candidates – requires additional resources

Virtualizable Candidates

(5)

Candidate Selections

Non-Candidates

Application requires more resources than can be made available

to a single Virtual Machine

more than 4 active NIC ports, more than 64GB RAM, more than 4 physical CPUs http://www.vmware.com/pdf/vi3_35/esx_3/r35u2/vi3_35_25_u2_config_max.p http://www.vmware.com/pdf/vi3_35/esx_3/r35u2/vi3_35_25_u2_config_max.p df

OS supportability

http://www.vmware.com/pdf/GuestOS_guide.pdf

(6)

Candidate Selections

Application which uses specialized peripherals

not supported in

Virtual Machines

such as fax server cards, modems, encryption cards, other specialized PCI/expansion cards, some dongles;

Application which uses specialized peripherals for licensing

which are

not supported in VM’s

which are

not supported in VM’s

such as fax server cards, modems, encryption cards, video capture cards, other specialized PCI/expansion cards, some dongles;

(7)

Candidate Selections

Poor-Candidates

Applications that require a serial connector, or parallel port

connector for the licensing

Can mount port to VM, but locked to specific host, No Vmotion

Tape Backup Server – SCSI attached

Only Adaptec SCSI adapters

Generally highly utilized - CPU, Memory, Network and Disk

True Real-time application – one clock cycle to one event

Applications that use hard-core 3D graphics

(8)

Candidate Selections

Candidates – requires additional resources

Applications that require a USB key for licensing

USB over IP e.g. AnywhereUSB (Digi)

Applications that use licensing to a specific MAC address

can set a static MAC address in the VM’s configuration file (.vmx) can set a static MAC address in the VM’s configuration file (.vmx)

(9)

What about the

rest?

(10)

>95% Applications

(11)

Building your

Application on VM

(12)

>95% of Applications Match or Exceed Native

Performance on VMware Infrastructure

P ro p o rt io n o f A p p s P e rf o rm in g W e ll

100% ESX 2 ESX 3 ESX 3.5 Future

Overhead VM CPU • 30% - 60% • 1 vCPU • 20% - 30% • 2 vCPU • 4 vCPU • <2% - 10% • 8 vCPU • <10% - 20% ESX Versions P ro p o rt io n o f A p p s P e rf o rm in g W e ll VM Memory IO • 3.6 GB • 800 MBits • 64 GB • 100,000 IOPS • 9 GBits • 256 GB • >200,000 IOPS • 40 GBits • <10,000 IOPS • 380 MBits • 16 GB

(13)

Disk IO per Second

Database Requirements

vs. VI Capacity

VI supports 80X IO

throughput

requirement of

average 4-CPU

Support Large Oracle Databases

60K 80K 100K 100,000 DB requirements (4-CPU Oracle DB) VI Capacity

Source: VMware Capacity Planner analysis of > 700,000 servers in customer production environments

average 4-CPU

Oracle Database

20K 40K

(14)

Run SAP with <10% Overhead

# SD

Users

SAP SD Benchmark

200 400 600 800 1000 300 ₂₇₀ 480 ₄₄₅ 815 732 10% overhead 7% overhead 10% overhead 200

Native VMware Native VMware Native VMware

CentraI Instance & DB 1 vCPU 4 Gb 4 vCPU 16 Gb CentraI Instance & DB 2 vCPU 8 Gb CentraI Instance & DB 4 vCPU 16 Gb

(15)

12-Dec-2007: SAP Support for VMware

[….]

Production Support for Windows and Linux

VMware now a SAP Global Technology Partner

(16)

Server Sprawl

Development

Typical SAP Deployment

Training Dev

(17)

Virtual Machine Windows 64 Bit Application Servers SAP HR Virtual Machine SuSE Linux Application Servers SAP ERP Virtual Machine Windows 2000 Server Application Servers Test & Dev

Virtual Machine Windows 2003 Server Application Servers SAP CRM Virtual Machine Windows 64 Bit Application Servers Test & Dev

Virtual Machine Red Hat Linux Application Servers SAP ERP Virtual Machine Red Hat Linux 64 Bit Application Servers SAP BI

SAP BI Resource Pool Test & Dev Resource Pool SAP ERP Resource Pool

The Dynamic SAP Datacenter

SAP Test & Development Systems

VMware Infrastructure

SAP HR System SAP ERP Cluster SAP Business Intelligence

VMware ESX Server VMware ESX Server

VMware® VMotion™ Technology

(18)

SAP Sizing - Guidelines

Start with small units. Smaller units are better manageable and

load balancing is easier.

Scale-out instead of scale-up. e.g. deploy additional Application

Servers if necessary

Virtualization may not fit every customer, but often leaving a

highly utilized Database on a ‚“native“ host and virtualising the

remaining components is a viable option.

SD Benchmark results are around 400 SD users for 2 vCPU with

8GB and around 700 SD users for 4 vCPU with16GB.

(19)

Sizing and

Performance

Considerations

(20)

Sizing Considerations

Basic Concepts - CPU

CPU Performance

Virtual CPUs map to physical CPU cores

Multi-vCPUs from a single VM must be scheduled simultaneously

Configure only as many vCPUs as needed

Impose unnecessary scheduling constraints on ESX Server

Impose unnecessary scheduling constraints on ESX Server Waste system resources (idle looping, etc.)

Idle vCPUs will compete for system resources

Unavailable pCPUs can result in VM “ready time.”

# of total vCPUs + 1 <= # of total cores per ESX hosts (High-Performance)

Ensure application is support for multithread if assign multiple Vcpu to single

VM

(21)

Sizing Considerations

CPU-0 CPU-1 CPU-2B - 0 CPU-3E - 0 _{Schedule - 1}

C- 0 C- 1 D- 0 D- 1 _{Schedule - 2}

A - 0 A - 1 B - 0 E - 0 _{Schedule - 3} A - 0 A - 1

(22)

Sizing Considerations

CPU-0A - 0 CPU-1A - 1 CPU-2B - 0 CPU-3E - 0 CPU-4C- 0 CPU-5C- 1 CPU-6D- 0 CPU-7D- 1 _{Schedule - 1}

(23)

Sizing Considerations

Basic Concepts - Memory

Memory Performance

Service Console: 272 MB

VMkernel: 100 MB+

Per-VM memory space overhead increases with:

VA

Per-VM memory space overhead increases with:

Number of VCPUs Size of guest memory

Do not over allocate – for intensive workloads

PA

(24)

Sizing Considerations

Basic Concepts - Networking

Networking Performance

NIC teaming

Use separate NICs to avoid contention

Use same virtual switch

VSwitch VSwitch Virtual Machines Virtual NICs ESX Server VM3

Use same virtual switch

to connect communicating VMs Use vmxnet virtual device in guest Requires CPU cycles

Physical Switches 1000 Mbps 1000 Mbps Physical NICs 1000 Mbps

(25)

Performance Considerations - VM Network

Virtual Switch

Separate multiple subnet into different virtual switch

Reduce network hoop from each virtual

switches to improve network efficiency Proper planning on network infrastructure and VM guest IP

address and subnet Standardize Virtual Switch name for HA, DRS and VMotion purposes

(26)

Performance Considerations - VM Network

Ensure Physical Network Cards is slotted at PCI-X to improve network

performance

Separate the VM network traffic from Service Console and VMkernel

Reside the VMs which

require connectivity to each others into same virtual switches to improve efficiency

Multiple mode of the physical NIC must match with the switch configuration

(27)

Virtualization Storage Best Practices

Standardize the VMFS datastore size with storage best practices recommended by different storage vendors

Avoid high intensive I/O workload VM guests access to the same datastore at the same time

Assign separate Virtual Disks and Virtual Controller if require for better management capability and performance purpose

Select RDM for huge storage requirement in single volume

Avoid extra multi path if do not require as it will impact the total numbers of LUNs to be Avoid extra multi path if do not require as it will impact the total numbers of LUNs to be presented to each ESX hosts

Create separate VMkernal for Software ISCSI adaptor or select the right ISCSI HBA cards for ISCSI storage protocol

Sufficient Swap Space require in each datastore to ensure the allow the memory over commit on ESX servers

(28)

Round Robin

Load Balance across all available paths

Improve I/Os throughput

Avoid Bottleneck on dedicated path for performance

perspective perspective

(29)

Performance Considerations - Memory Reservation

Memory

Reservation

Increase physical memory respond time Improve data processing efficiency Share value impact Share value impact the resources priority to VM

Only reserve if needed

(30)

Performance Considerations - CPU Reservation

CPU Reservation

CPU reserve for High CPU contending VM Improve performance Reduce available CPU processing power to the rest of VM on same ESX Host same ESX Host Only reserve the

amount which needed Shares value will impact the

prioritization of VM in the ESX host

(31)

Performance Considerations - Advance CPU

CPU Reservation

Keep the setting to default for hyper threaded core sharing

Scheduling Affinity allow specify Vcpu to be assigned to specify physical specify physical Core in ESX servers Distribute workload to underutilize physical cores if needed Recommended to keep as default setting

(32)

Performance Considerations - DRS Best Practices

Reside VMs which require communication in same host will reduce communication latency

Separate VMs with same functionality to different hosts to maintain maximum availability

Right aggressive mode for migration threshold and fully automate Enable EVC to improve VMotion Capability

(33)

MS Kerberos implementation allows a 5 minute tolerance

Active Directory operations are critically time dependent

File Replication Services (FRS) synchronizes scripts, database

changes/updates, policies based, in part, on time-stamping

(34)

Time Synchronization

No CPU cycles needed – none given!

Clock drifts can be significant in a relatively short period

Idle cycles in a virtual machine is an Active Directory domain’s

worst enemy

How do you combat time synchronization issues?

More than a 28

minute drift!

(35)

Time Synchronization–Option A – Using W32Time

Use Windows Time Service – NOT VMware Tools

Microsoft KB article # 816042 provides instructions for this

process

1. Modify registry settings on the PDC emulator for the forest root domain:

HKLM\System\CurrentControlSet\Services\W32Time\Parameters Change Type REG_SZ value from NT5DS to NTP

Change NtpServer value from time.windows.com,0x1 to an external stratum 1 time source, i.e. tock.usno.navy.mil,0x1

tock.usno.navy.mil,0x1

HKLM\System\CurrentControlSet\Services\W32Time\Config Change AnnounceFlags REG_DWORD from 10 to 5

2. Stop and restart time service –

_{net stop w32time}

_{net start w32time}

(36)

Time Synchronization–Option B – VMware Tools

Modify Windows Time Service – Use VMware Tools

Implement Domain Controllers Group Policy to modify registry:

Modify

Enable ESX server NTP daemon to sync with external stratum NTP source

VMware Knowledge Base ID# 1339

Enable ESX server NTP daemon to sync with

external. Use VMware Tools time synchronization within the virtual machine

NOTE: VMware Tools time sync is designed

to play “catch-up”, not slow down!

(37)

Time Synchronization - Summary

Use one method or the other

Do NOT use both!!!

Decisions should be based on current time management

infrastructure or organization’s policies

Ensure Multiple NTP servers synchronize option is been

configured on each ESX Host

(38)

Exchange and MS SVVP

Ong, Kok Leong

Senior Consultant, VMware ASEAN

e

[email protected]

m +60-12-4706070

(39)

Native VMware Infrastructure

Double Capacity of Exchange 2007 Hosts

8K mailboxes

8 VM

16K mailboxes

16 core 128 GB 16 core 128 GB

(40)

Proving to be Better Than Native

The experiment: 2,000 user Exchange multi-role VMs

The hardware:

IBM x3850 M2

4 x quad-core Intel Xeon 7350

128G RAM

(41)

Scaling Exchange (Natively) On a Single Server

8 Cores Max

(42)

Single-Server Native Exchange Scaling

8K 10K 12K 14K 16K Outside of M a ilb o x e s 8 core max 2 4 6 8 10 12 14 16 8K 6K 4K 2K 5M/mailbox perf. threshold Highest Performance Outside of Microsoft recommendations M a ilb o x e s

(43)

Multi-VM Scaling of Exchange on VI3

8K 10K 12K 14K 16K M a ilb o x e s

Building blocks stay within Microsoft recommendations: • 1,000 mailboxes/core

• 5MB/mailbox

Maximum performance!

• Three building blocks breaks through pre-virtual memory boundaries

• Five building blocks shatters pre-virtual CPU limitations 5 MB/mailbox perf. 2 4 6 8 10 12 14 16 8K 6K 4K 2K M a ilb o x e s

CPUs (or cores)

• Eight building blocks enables 16,000 users

VI3 provides

native-matching

performance and

complete resource

utilization

5 MB/mailbox perf. threshold

Eight core limit recommended by Microsoft

(44)

Exchange Sizing Considerations

Considerations

Do not over-commit pCPUs when running Exchange VMs. Do not over-allocate vCPUs; try to match the exact workload.

If the exact workload is unknown, start with fewer vCPUs initially and increase

later if necessary.

Memory pages can be shared across VMs that have similar data (e.g. same

guest OS) guest OS)

Memory can be over-committed, (i.e. allocating more memory to VMs than is

physically available on the ESX Server)

(45)

Microsoft Support and Licensing for VMware

MS Licensing Allows VMotion MS Support for Apps on ESX MS Licensing Allows VMotion

>Reassign licenses between servers as frequently as needed

>Covers 41 server apps including >Exchange 2007

>SQL Server 2008 >SharePoint 2007 >Dynamics CRM 4.0

MS Support for Apps on ESX

>ESX validated as part of Microsoft SVVP program

>Microsoft ensures same technical support on ESX as on physical servers for Windows and the 33 apps

>Exchange 2007 >SQL Server 2008 >SharePoint 2007 >Dynamics CRM 4.0

(46)

MS SVVP – What does validation mean?

Customers will now get corporate support from MS when

running Windows Server on ESX 3.5 U2/U3

Windows Server 2008, Windows 2000 Server Service Pack 4,

Windows Server 2003 Service Pack 2 and subsequent service

packs are supported

Any version of Windows back to Windows Server 2000 SP4 is

VMware Confidential

also supported, including all OS roles such as Active Directory,

File Services, etc

VMware has a premier support contract with MS and can

escalate issues to MS Support on a customer’s behalf.

(47)

MS VSSP – What about the MS Applications?

As part of VSSP, Microsoft has published a list of MS Applications

that are supported. Article ID:957006

Microsoft Application Virtualization (App-V) Microsoft Operations Manager (MOM) 2005 Microsoft BizTalk Server Microsoft Search Server

Microsoft Commerce Server Windows Essential Business Server 2008 Microsoft Dynamics AX Windows Small Business Server 2008

Microsoft Dynamics CRM Microsoft SQL Server 2005 and SQL Server 2008 Microsoft Dynamics NAV Microsoft System Center Configuration Manager Microsoft Exchange Server Microsoft System Center Data Protection Manager Microsoft Exchange Server Microsoft System Center Data Protection Manager Microsoft Forefront Client Security Microsoft System Center Essentials

Microsoft Intelligent Application Gateway (IAG) Microsoft System Center Operations Manager Microsoft Forefront Security for Exchange (FSE) Microsoft System Center Virtual Machine Manager Microsoft Forefront Security for SharePoint (FSP) Microsoft Systems Management Server (SMS) Microsoft Host Integration Server Microsoft Visual Studio Team System

Microsoft Internet Security and Acceleration (ISA) Server Microsoft Windows HPC Server 2008 Microsoft Office Groove Server Windows Server 2003 Web Edition

Microsoft Office Performance Point Server Microsoft Windows Server Update Services (WSUS) Microsoft Office Project Server Windows Web Server 2008

(48)

Expanding ISV Ecosystem

(49)

Additional Resources

Visit us on the web to learn more on specific apps

http://www.vmware.com/solutions/business-critical-apps/

Best Practices, Reference Architectures, and Case Studies

Microsoft Apps (Exchange, SQL, SharePoint)

Oracle

SAP

Microsoft server software and supported virtualization

environments

http://support.microsoft.com/kb/957006

(50)

VMware - Plan and Design Service

Project Kickoff Data Collection

Week 1 Week 2 Week 3 Week 4 Week 5

Comprehensive VI architecture design for implementation

Avoid implementation pitfalls that can stall or slow deployment Accelerate the achievement of virtual infrastructure benefits

Capacity and Financial Analysis

Assessment Deliverable

Design Sessions Creation of Deliverables

• Architectural Design

• Assembly & Configuration Guide

• Standard Procedures

• Test Plan & Rollout Plan

Final Recommendations

(51)

Start with Capacity Planner

VMware Capacity Planner is…

Business and decision support tool

To perform faster, more accurate and benchmarked consolidation assessments

Capacity Planner provides:

Complete state of the datacenter (As-Is) Complete state of the datacenter (As-Is)

Comprehensive future state consolidation recommendation and roadmap (To-Be)

(52)