Intel Virtualization and Server Technology Update

(1)

1 Intel

Intel

Intel Virtualization

Virtualization

Virtualization and

and

Server Technology Update

Petar Torre

Lead Architect

Service Provider Group

(2)

Legal Disclaimer

• Intel may make changes to specifications and product descriptions at any time, without notice.

• Performance tests and ratings are measured using specific computer systems and/or components and reflect the

approximate performance of Intel products as measured by those tests. Any difference in system hardware or

software design or configuration may affect actual performance. Buyers should consult other sources of information to

evaluate the performance of systems or components they are considering purchasing. For more information on

performance tests and on the performance of Intel products, visit

Intel Performance Benchmark Limitations

• Intel does not control or audit the design or implementation of third party benchmarks or Web sites referenced in this

document. Intel encourages all of its customers to visit the referenced Web sites or others where similar performance

benchmarks are reported and confirm whether the referenced benchmarks are accurate and reflect performance of

systems available for purchase.

• Intel processor numbers are not a measure of performance. Processor numbers differentiate features within each

processor family, not across different processor families. See www.intel.com/products/processor_number for details.

2

• Intel, processors, chipsets, and desktop boards may contain design defects or errors known as errata, which may cause

the product to deviate from published specifications. Current characterized errata are available on request.

• Intel Virtualization Technology requires a computer system with a processor, chipset, BIOS, virtual machine monitor

(VMM) and applications enabled for virtualization technology. Functionality, performance or other virtualization

technology benefits will vary depending on hardware and software configurations. Virtualization technology-enabled

BIOS and VMM applications are currently in development.

• 64-bit computing on Intel architecture requires a computer system with a processor, chipset, BIOS, operating system,

device drivers and applications enabled for Intel® 64 architecture. Performance will vary depending on your hardware

and software configurations. Consult with your system vendor for more information.

• Intel, Intel Xeon, Intel Core microarchitecture, and the Intel logo are trademarks or registered trademarks of Intel

Corporation or its subsidiaries in the United States and other countries.

(3)

Agenda

• Virtualisation

–

Review

–

“Invisible” technology

• Intel® Xeon® processors in Cisco UCS

–

E5-2600

–

E7-2800

3 –

E7-2800

–

E7-4800

(4)

Virtualization: Evolving

Towards the Enterprise Cloud

Enterprise Cloud:

Virtualization 3.0

Automation and Resource Scalability

Flexible Resource Management:

4 Consolidation: Virtualization 1.0

Operational Expense Efficiency

Flexible Resource Management:

Virtualization 2.0

Dynamic Resource Allocation

(5)

Virtualization Data Center Foundation

Infrastructure

Scale

Massive Scaling

Balanced Platform

RAS is critical

Lower TCO

Network

consolidation

Multi

Multi--Tenancy

Tenancy

Resource Sharing

Security

Isolation

D

a

ta

C

e

n

te

r

R

e

q

u

ir

e

m

e

n

ts

5 Unified Network

Unified Network

(10Gb Ethernet for Storage and Networking)

Intel

®

_Xeon

®

_{Processor Based Platform}

(CPU & I/O performance, New Technologies)

D

a

ta

C

e

n

te

r

R

e

q

u

ir

e

m

e

n

ts

D

a

ta

c

e

n

te

r

F

o

u

n

d

a

ti

o

n

(6)

A Framework for Optimizing Virtualization

VMM

OS

…

… and scale out across VMs

Reduce

overhead

within each

VM…

Ways to Reduce Overheads

• VT-x Latency Reductions

• Virtual Processor IDs

• Extended Page Tables

• APIC Virtualization (Flex Priority)

• I/O Assignment via DMA Remapping (VT-d)

6 Host

• I/O Assignment via DMA Remapping (VT-d)

• VMDq

• 4GB pages

Technologies for Scaling

• Scaling from EP to EX

• Hyper Threading

• PAUSE-loop Exiting

• SR-IOV

• RAS Capabilities

Intel working with

VMware and other VMM

vendors to reduce

virtualisation overhead

(7)

Optimizing VT

Optimizing VT Transition Latencies

Transition Latencies

• Virtual Machine Control Structure (VMCS)

–

VMCS holds all essential guest and host register state

–

“Backed” by host physical memory

–

Accessed via an architectural VMREAD / VMWRITE interface

–

Enables CPU implementations to cache VMCS state on-die

• Virtual Processor IDs (VPIDs)

–

VMM-specified values used to tag microarchitectural structures (TLBs)

–

Used to avoid TLB flushes on VT transitions

7 Memory

–

Used to avoid TLB flushes on VT transitions

VMM

pCPU

VMREAD

VMWRITE

Backing

Page

in

Memory

VMCS

Significant VT latency reductions over time…

0

200

400

600

800 1000

1200

1400

1600

1800

2007

2008

2009

2010

(8)

Making Resource Sharing Safer

VM

Isolate

Intel

®

_{Virtualization and Intel}

®

_{Trusted Execution}

Technology (Intel

®

_{TXT) work together to better}

isolate VMs

Measure

Intel

®

_{TXT measures VMM for launch protection}

Encrypt

Virtual Machine Monitor

8 Encrypt

Intel

®

_{AES New instructions in Intel® Xeon®}

processor E5 and E7 families quickly encrypts data in

flight and at rest

Intel

®

_{TXT and AES New instructions in Intel}

®

_Xeon

®

_processors

(9)

Intel® Xeon® Processor Families in Cisco UCS

Top Performance / $,

Energy Efficiency, &

Flexibility

for Infrastructure Apps

E

EE

E5

5

5 Family

Family

Xeon E

Xeon E5

5--2600

2600

9 Scalable Performance,

Flexibility, & Advanced

RAS for Demanding Apps /

/

Consolidation

E7

Family

Xeon

Xeon 7500

7500 &

&

Xeon

Xeon 6500

6500

7000

Sequence

Xeon E

Xeon E7

7--4800

4800 &

&

2800

E

EE

E7

7

7 Family

Family

(10)

Typical Workloads

L

a

r

g

e

r

W

o

r

k

lo

a

d

s

E7 UCS

Servers:

B440 M2,

B230 M2,

C460, M2

C260 M2

Workload / Usage

Business Processing

(DB, ERP, CRM, batch)*

Decision Support

(data warehouse, Business Intelligence)

Large-scale Consolidation

(including Virtualization & Multi-tier)

Application Development

High Performance Computing

10

M

id

/

S

m

a

ll

e

r

W

o

r

k

lo

a

d

s

More/Larger

Instances/Users

Fewer/Smaller

Instances/Users

2S Volume

Servers

B200 M3,

C240 M3,

C220 M3

Collaboration

Web Infrastructure

IT Infrastructure

Development/Quality Assurance

* For directional guidance only. This is not a server selection guide. Actual server sizing is a relatively complex effort involving workload characterization including such considerations as type of application, size of workload, number of users, type of transaction, SLA response times, targeted utilization, level and estimation accuracy of workload baseline/peak/growth, physical or managerial constraints, need to maintain a single state at all times to ensure all users see the same results at any given time, cost to migrate to a scale-out alternative, and system availability requirements.

(11)

Introducing the Intel® Xeon® Processor

E

E5

5 Family

Family

Leadership Performance

Breakthrough I/O

Innovation

Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products.

The Heart of a Flexible, Efficient Data

Center Built to Scale

Innovation

Trusted Security

Exceptional Energy

Efficiency

(12)

Intel

®

_Xeon

®

_{2S Platform Comparison}

E5-2600 Product Family

based Platform (B200 M3)

Sandy

Bridge

8 Core

Sandy

Bridge

8 Core

DDR3

x8

x4

QPI

Xeon

®

_{5500 / 5600}

based Platform (B200 M2)

Xeon

®

₅₅₀₀

Xeon

®

₅₆₀₀

Core

Xeon

®

₅₅₀₀

Xeon

®

₅₆₀₀

Core

DDR3

QPI

up to DDR3 1333 up to 6.4 GT/s

Q

P

I

Q

P

I

up to

DDR3 1600 up to 8.0 GT/s

Up to 8 Cores

Up to 24 DIMMs @1666 Mhz

Up to 80 PCIe lanes

Two QPI links @ 8.0GT/s between CPUs

One-chip Platform Controller Hub (PCH)

Integrated PCI-E Gen 3

x8

x4

Intel C600

Series

(PCH)

Serial Attached

SCSI (SAS) 4

ports, 3Gb/s

Intel 5500

Series

(IOH)

Intel

ICH 10

Q

P

I

Q

P

I

up to

36 lanes

PCIe2

up to 40 lanes PCIe3 per socket

Up to 18 DIMMs per 2S platform

Up to 36 PCIe2 lanes

Two-chip IOH / ICH

12

(13)

Romley

Romley Memory New Features

Memory New Features

• Higher speed memory: 1600 MHz DDR 3

–

Intel

®

_Xeon

®

_{processor E5-2600/4600 product families (EP) 1600MHz}

Support:

SR/DR RDIMM 3SPC/2DPC or 1DPC

Max 16GB DIMMs

total RAM 256GB @ 1600 MHz

• LR-DIMMs

13 • LR-DIMMs

–

Load Reduced DIMMs

–

3 x 32GB DIMMs per channel supported

(14)

DIMM Types

• UDIMM

–

Address, Control & Data signals are “unbuffered”

• RDIMM

–

Address & Control signals are “buffered” in a

register

• LR-DIMM

–

Address, Control and Data signals are all “buffered”

UDIMM

D

R

A

M

D

R

A

M

D

R

A

M

D

R

A

M

D

R

A

M

D

R

A

M

D

R

A

M

D

R

A

M

D

R

A

M

Data Bus

Address

& Control

14

14 LR-DIMM

D

R

A

M

R

e

g

is

te

r

+

R

a

n

k

M

u

lt

.

L

o

g

ic

D

R

A

M

D

R

A

M

D

R

A

M

D

R

A

M

D

R

A

M

D

R

A

M

D

R

A

M

D

R

A

M

Data Bus

Address

& Control

Data Buffers

D

R

A

M

R

e

g

is

te

r

D

R

A

M

D

R

A

M

D

R

A

M

D

R

A

M

D

R

A

M

D

R

A

M

D

R

A

M

D

R

A

M

Data Bus

Address

& Control

(15)

What is an LR

What is an LR----DIMM?

DIMM?

• LR-DIMM = Load Reduced DIMM

• An LR-DIMM has additional components on the DIMM which:

a) Buffer the data bus pins

–

To reduce electrical loading on the data bus

b) Rank multiplication logic to make to logically reduce the number of ranks

visible to the memory controller

15 visible to the memory controller

–

Even though there is likely four ranks physically on the DIMM

–

Depending on the DIMM, rank multiplication can be disabled, set to 2:1 or set to 4:1 (4:1

makes a Quad ranked DIMM logically look like a single rank DIMM)

• More info about LR-DIMMs available here:

–

http://www.edn.com/article/519386-Basics_of_LRDIMM.php

(16)

Trusted Security

Intel

®®

_{Trusted Execution Technology}

Average organizational costs of a data breach over

$

$7

7M per incident

M per incident

2

2 Intel

Intel

®®

_{Advanced Encryption Standard}

New Instructions

Use of AES encryption

1

1 _{has nearly tripled in}

_{has nearly tripled in}

the last 10 quarters

1 AES Instructions refers to AES128-SHA1. Source: Akamai Second Quarter 2011 ‘State of the Internet' Report. See details and report at:

http://www.akamai.com/html/about/press/releases/2011/press_102411.html

2 Source: Ponemon Institute - 2010 Annual Study: Cost of a Data Breach . March 2011

(17)

Xeon® Processor Energy Efficiency

~50%

1

higher Performance

At Same Power

~50%

1

higher Performance

At Same Power

P

er

fo

rm

an

ce

a

nd

S

ys

te

m

P

ow

er

Peak power under load

Performance

1 Performance comparison using SPEC_Power results published as of March 6th_{, 2012. See back up for configuration details. For more information go to intel.com/performance}

R

el

at

iv

e

P

er

fo

rm

an

ce

a

nd

S

ys

te

m

X5675

E5-2660

Best Data Center Performance per Watt

1

(18)

7.59

10.88

6

8

10

12 S

c

o

r

e

@

N

u

m

b

e

r

o

f

T

il

e

s

)

**VMmark* v2.1 Performance**

Intel

®

_Xeon

®

_{Processor E5-2600 Product Family}

**Virtualized Consolidation 2.0 Performance on VMmark* 2**

Higher is better

N

o

P

u

b

li

c

a

ti

o

n

A

v

a

il

a

b

le

1.43X

• Flexible

datacenter,

multi-host

benchmark

models

application

throughput with

minimum QoS

requirements

and virtual

infrastructure

operations

• Measures

consolidated

application

throughput and

operations

completed

@10

Tiles

@7

Tiles

0

2

4 Opteron*

6282 SE

(2.6GHz, 16C, 140W)

X5690

(3.46GHz, 6C, 130W)

E5-2690

(2.9GHz, 8C, 135W)

P

e

r

fo

r

m

a

n

c

e

(

S

c

o

r

e

@

N

u

m

b

e

r

o

f

T

il

e

s

)

Grow your consolidation capabilities with the Intel® Xeon® processor E5-2600 product family

Source: Best available publications/submissions as of 6 March 2012 Configuration Details: Please reference slide speaker notes and back up slides

For more information go to http://www.intel.com/performance

18

N

o

P

u

b

li

c

a

ti

o

n

A

v

a

il

a

b

le

completed

B

e

s

t

P

u

b

li

s

h

e

d

* Other names and brands may be claimed as the property of others

B

e

s

t

P

u

b

li

s

h

e

d

(19)

Intel® Xeon® Processor E

Intel® Xeon® Processor E5

5

5 5----2600

2600

2600 Product Family

Product Family

Source: Public white paper published at: http://www.principledtechnologies.com/clients/reports/Intel/Xeon_E5-2690_consolidation_0312.pdf

For more information go to http://www.intel.com/performance

(20)

The Intel® Xeon® Processor E

The Intel® Xeon® Processor E5

5 Family

Family

Leadership Performance:

Leadership Performance: 15 New x86 World Records

Breakthrough I/O Innovation:

Breakthrough I/O Innovation: Up to 3X I/O Performance

Trusted Security:

Trusted Security: Trusted Hardware Security

Learn More at: www.intel.com/datacenter

Exceptional Energy Efficiency

Exceptional Energy Efficiency: Best performance per watt

Cisco Servers: B200 M3, C220 M3 and C240 M3

The Heart of a Flexible, Efficient Data

The Heart of a Flexible, Efficient Data Center

Center

that’s

that’s Built to Scale

Built to Scale

(21)

21

INTEL

®®®®

_XEON

®®®®

_{PROCESSOR E7}

_{PROCESSOR E}

₇

_{7 FAMILY}

_FAMILY

(22)

Intel

®

_Xeon

®

_{Processor E7 platform (Westmere-EX)}

More Efficient

• Supports 32GB DDR3 DIMMs (2TB per

4-socket system)

1

More Expandable

More Security & RAS

• 10 cores / 20 threads

• 30MB of last level cache

More Performance

WSM WSM WSM

WSM----EXEXEXEX WSMWSMWSMWSM----EXEXEXEX

SECURITY

• Intel

®

_{Advanced Encryption}

22 Delivers more Performance, Expandability and RAS

while improving Energy Efficiency

• More performance within same max

CPU TDP as Xeon 7500

• Lower partial active & idle power via

Intel Intelligent Power Technology

2

• Support for Low Voltage-DIMMs

3

• Reduced power memory buffers

4 WSM

WSM WSM

WSM----EXEXEXEX WSMWSM----EXWSMWSMEXEXEX

• Intel Advanced Encryption

Standard-New Instructions

• Intel

®

_{Trusted Execution}

Technology (TXT)

RELIABILITY, AVAILABILITY, SERVICEABILITY

• Double Device Data Correction (DDDC)

• Partial Memory Mirroring

1. Up to 64 slots per standard 4 socket system x 32GB/DIMM = 2TB

2. Uses similar core and package C6 power states enabled on Intel Xeon 5500/5600 series processors. Requires OS support.

3. Savings dependent on workload and configuration. Example: At 100% SPECpower load it can save ~0.8W for 4GB DIMM DRx8 based on early Intel internal estimates

(23)

2-Socket Platform Comparison

Efficient Performance Class

Expandable Class

IO HUB IO HUB IO HUB IO HUB IO HUB IO HUB IO HUB IO HUB

E7 E7 E7

E7----2800280028002800 E7E7E7E7----2800280028002800

Xeon®

E7-2800

Xeon®

E5-2600

Frequency Sensitive

Performance

Thread Sensitive

B200M3

C220M3

C240M3

B230M2

C260M2

E5-2600

8 Core

E5-2600

8 Core

DDR3

QPI

23

Frequency Sensitive

Performance

Thread Sensitive

Power Efficiency

Low

High

Low

Memory Capacity

High

Good

Reliability

Best

Virtualization driving need for compute, memory & reliability

(24)

Xeon

®

_{E7 Family: Meeting the}

Highest Virtualization Needs

Infrastructure

Consolidation

(of multi-tier

Applications)

Large Scale,

Mission

Critical

Virtualization

(>8GB)

Xeon

®

_E7

• 10C/20 threads per

socket

• 2-256 socket scaling

• 512GB memory per skt

• 2X I/O capacity

• Mission Critical RAS

Intel Platform Virtualization Technologies

24 Headroom

for Peak

&

Unpredictable

Demand

Live Migration

of Big

Workloads

Intel

®

_{VT Flex Migration}

Processor

Chipset

Network

Intel

®

_{VT For}

Directed I/O

Intel

®

_VT-x

_Intel

®

_{VT For}

Connectivity

Intel Platform Virtualization Technologies

(25)

Machine Check Architecture Recovery

Previously seen only in RISC, mainframe, and Itanium

Previously seen only in RISC, mainframe, and Itanium----based systems

based systems

Normal Status

With Error

Prevention

System

Recovery

with OS

System works in

conjunction with OS or

VMM to recover or

restart processes and

continue normal

operation

Error information

MCA Recovery

25 HW Un-correctable Errors

Error

Corrected

Error

Detected*

Error

Contained

HW Correctable Errors

Un-correctable Errors

Bad memory location

flagged so data will not

be used by OS or

applications

Error information

passed to OS /

VMM

*Errors detected using Patrol Scrub or Explicit Write-back from cache

(26)

MCA

MCA----R Fault Isolation with

R Fault Isolation with

R Fault Isolation with vSphere

vSphere

vSphere 5

5

5 5....0

0

0 Only Supported on Intel® Xeon® processor E

Only Supported on Intel® Xeon® processor E

Only Supported on Intel® Xeon® processor E7

7

7 7 family & VMware

family & VMware

family & VMware vSphere

family & VMware

vSphere

vSphere 5

5 5....0

5

0

0 ESXi 5.0

VM

ESXi 5.0

VM

ESXi 5.0

26 STATUS: GOOD

MCA-Recovery Provides Isolation From Otherwise Fatal System Errors

ESXi 5.0

STATUS: GOOD

Non MCA System

Cisco C460 M2

C260 M2

B230 M2

B430 M2

CORRECTABLE MEMORY ERROR

ERROR LOGGED FOR PFA

STATUS: GOOD

UNCORRECTABLE MEMORY ERROR

ERROR LOGGED FOR PFA

STATUS: GOOD

UNCORRECTABLE MEMORY ERROR

PURPLE SCREEN OF DEATH

REBOOT SERVER

REPORT TO ESXi

MARK MEMORY LOCATION BAD

RESTART FAILED VM IN POOL

VM GETS RESTARTED

REPORT TO ESXi

ISOLATE ERROR

(27)

Intel

®®®®

_Xeon

®®®®

_{Processor E7 Family and Vmware*}

vSphere* 5.0

• Vmware* vSphere* 5.0 scales to exploit the Intel® Xeon®

processor E7 platform capabilities:

• 80-cores (160 threads) physical servers

• 2 TB physical RAM

• 32-vCPU VMs w/ 1TB virtual RAM

• 512 VMs, up to 2048 vCPUs per physical host

• VM consolidation on this scale requires hypervisor focus on:

27

• VM consolidation on this scale requires hypervisor focus on:

• Scalability, Energy Efficiency, & Uptime (Reliability

)

• Especially Memory Reliability

Especially Memory Reliability

• New

New Vmware

New

Vmware

Vmware*

Vmware

* vSphere

*

vSphere

vSphere* 5.0 memory reliability features

* 5.0 memory reliability features

require Intel® Xeon® processor E7 family

(28)

The Intel® Xeon® Processor

The Intel® Xeon® Processor E5 & E7 Families

E5 & E7 Families

For Cisco UCS Servers

Leadership Performance:

Breakthrough I/O Innovation:

Breakthrough I/O Innovation: with E5

Trusted Security:

Trusted Security: Trusted Hardware Security E5 & E7

Learn More at: www.intel.com/datacenter

E5 Exceptional Energy Efficiency

E5 Exceptional Energy Efficiency: Best performance /watt

E7 Best RAS with MCA

The Heart of a Flexible, Efficient Data

The Heart of a Flexible, Efficient Data Center

Center

that’s

that’s Built to Scale

Built to Scale

(29)

29

* Other names and brands may be claimed as the property of others.

Thank