Cloud Computing Virtualized Computing Infrastructures

(1)

Cloud Computing –

Virtualized Computing Infrastructures

Erik Elmroth

Cloud computing in plain English

www.youtube.com/watch?v=QJncFirhjPg

2

Brief outline

• A Game changing trend in IT use • Revitalization of the datacenters

– Infrastructure providers for service providers • Compute Clouds

• Compute Clouds – virtualization

– datacenter infrastructure providing virtual resources as utility

• The RESERVOIR approach – business-aware federated Clouds

– Architecture & functionality

– Scenarios illustrating the RESERVOIR capabilities

A Game Changing Trend - Growth

on Service Consumer Side

• Individuals use internet-based services such as YouTube, MySpace, Google documents, Adobe Photoshop Express, etc, for managing their private and professional everyday activities

• Companies use external services such as hosted Microsoft Exchange, external mail services, external customer relations management, accounting systems, or hosting of their complete IT environments

• Explosive growths in availability of Software-as-a-Service (SaaS) (and Infrastructure-as-a-Software-as-a-Service (IaaS), Everything-as-a-Service (XaaS), …)

Crucial capacity characteristics –

to be cost-efficiently met

Extremely rapid growths (from global scale)

– MySpace: 36 months to reach 100 million users (now per day: 300 000 new users, 65 billion page views) – YouTube reached 20 million users within 16 months – App Store: Over 1000 Iphone applications. Over 160

million sold million sold

Regular/planned peaks

– banks see a major peak over a few days every month – on-line tax filing - exceptional load a few days/year – meet a rapid increase of use, e.g., after a marketing

campaign

Unexpected peaks

– News-related video streaming – Stock trading peaks at financial crises

Revitalization of the datacenters:

Service & infrastructure provider cooperation

Company or individual. Sees service, not hardware Provider for Provider for service user. Customer for infra provider Provides infra to service provider. (Datacenter) SLA SLA SLA SLA

(2)

New Requirements on Datacenters

(Infrastructure Providers)

• Today, load peaks typically managed by extensive over-provisioning

– COSTLY!!!

• Need for a new datacenter infrastructure, that – provide elasticity:

• scale quickly in response to demand increase (in • scale quickly in response to demand increase (in

minutes, not days)

• shrink dynamically to save resources (energy, other use)

– Improve network parameters by locality-awareness – manage SLAs corresponding to business

agreements

– support a variety of payment schemes (pay-per-use, pre-paid, flat-rate, etc)

• Today’s clouds provide partial solutions

Compute Clouds

• Virtual “cloud” of IT resources (within a datacenter)

• Services run on virtual resources, unaware of the physical resources

• Infrastructure – compute, storage, and network

network

• Utility model – provision on demand, charge back on use

– Notably, as power and running costs become a larger fraction of the total IT cost, the character of IT capacity become more utility-like

Before talking more Clouds…

• Virtualization

9

”Traditional” server virtualization

Applic. Applic. Applic.

Applic.

Hardware (CPU, RAM, Disk, LAN) Operating System Virtual Machine OS Applic. Virtual Machine OS Applic.

…

Applic. Virtual Machine OS Applic.

Hypervisor virtualization

Virtual machine Virtual machine Virtual machine

Hardware (CPU, RAM, Disk, LAN) Hypervisor Hypervisor OS Applic. machine Appl OS Appl machine

…

OS Applic. machine

Virtualization features

With a virtual machine you can:

• Define machine size as part of physical machine • Halt and resume execution

• Migrate between physical machines

These features can be used for many purposes!

(3)

Server Sprawl

• New application = new server

File/Print File/Print Application Application Application Application File/Print Database Database Application Application Application Application Application Database Application Application Application Database Database Application

Problems Server Sprawl

• Hardware

– Increased hardware acquisition costs – Increased infrastructure requirements – Increased hardware maintenance costs – Increased hardware replacement costs • Administration

– Patch management – Backup and recovery

– Server management and troubleshooting

Servers Deployed

26% 13% 8% 1% 100 - 249 26 - 99 10 - 25 Less than 10 18% 6% 6% 9% 13% 26% 0% 10% 20% 30% Don't know 5,000 or more 1,000 - 4,999 500 - 999 250 - 499 100 - 249

IDG Server Consolidation Research July 2006

Multiple Vendor Support

13% 14% 29% 17% 6% 5 4 3 2 1 vendor 9% 2% 2% 1% 1% 2% 4% 13% 0% 10% 20% 30% Don't know Over 25 vendors 10 - 25 9 8 7 6 5

IDG Server Consolidation Research July 2006

Biggest Challenges

42% 44% 60% 63% Server sprawl Maintenance costs Resource utilization Patch management 2% 6% 25% 27% 42% 0% 10% 20% 30% 40% 50% 60% 70% Don't know Other Downtime Interoperability Server sprawl

IDG Server Consolidation Research July 2006

Server Consolidation Strategy

No

28%

Yes

70%

Don’t

know

2%

70%

(4)

Server Consolidation

• Increase hardware

utilization

• Reduced costs

– Fewer systems

– Less power

– Less cooling

– Less administration

• Reduced

Infrastructure

– Fewer racks

– Fewer switches

Multiple OS & Applications

• Need for multiple OS

– Shared hardware

• Incompatible application

• Applications with different

OS and library

requirements

OS Support

32% 24% 21% 20% 30% 40% 14% 3% 6% 0% 10% 20%

5 or more 4 3 2 1 Don't know

IDG Server Consolidation Research July 2006

Types of OS Deployed

64% 59% 48% 72% 83% 50% 60% 70% 80% 90% 100% 1% 11% 28% 32% 48% 0% 10% 20% 30% 40% 50% Windows server Unix (AIX, Solaris, SCO) Linux (Red Hat, Caldera, Debian, SUSE) Windows 2000 Proprietary (S/390, OS/400, VMS) Windows NT

NetWare Other Don't know

IDG Server Consolidation Research July 2006

Training

• Present and reset training image

– Just reset the VM

– No need to reimage the systems

– Network isolation

Help Desk

• Increase ability to represent multiple

product environments

• Reduced testing infrastructure

– Physical systems

– Space requirements

– Power

– Cabling

• Enhanced test system accessibility

• Ability to rollback test system state

(5)

Disaster Recovery

• Fewer servers to manage and

recover/restore

– Reduces costs

• Server VMs are hardware

independent

– Can be restored to other platforms

– No need to match primary site and

secondary site hardware

• VMs are encapsulated

– Can be replicated between sites

– No need for bare-metal installs

Application Isolation

• Sandboxing

• Isolated from host • Discard changes

when finished

• Running incompatible software • Different versions of Microsoft Office • Running beta software

• Running multiple Java virtual machines

Why virtualization? – summary (1)

• Server consolidation

• Multiple OS & application support

• Lab and deployment testing

• Training

• Help desk

• Disaster recovery

• Application isolation

• Security

27

Why virtualization? – summary (2)

• Reduces administrative efforts

– Lowers operational costs

• Fewer servers to manage

– Speeds deployment

• Now 1-6 weeks (requisition, setup, software, • Now 1-6 weeks (requisition, setup, software,

test)

• Virtualization reduces this to hours

• Reduced hardware and infrastructure

costs

• Improves resource utilization

• Increases availability

28

Other levels of virtualization

(with inconsequent naming)

• Operating system virtualization

– Virtualization inside OS

– Full separation between applications, but all running in the same OS

• Application virtualization • Application virtualization

– Encapsulation of application in executable – no need for traditional installation of application in OS

– Runs as if installed on hardware but all access to OS is virtualized.

• Desktop virtualization

– As appl. virtualization but encapsulation of the whole desktop

29

What is Cloud Computing?

An emerging computing paradigm where data and services reside in massively scalable data centers

and can be ubiquitously accessed from any connected devices over the internet.

4+ billion phones by 2010 Web 2.0-enabled PCs, TVs, etc. Businesses, from startups to enterprises

(6)

Not only one type of clouds…

Three emerging Cloud

infrastructure markets

Appl-components-as-a-service Web-based services Software-as-a-service

Two existing end-user service markets delivered from clouds

Physical infrastructure-as-a-service Virtual infrastructure-as-a-service

Software-platform-as-a-service Appl-components-as-a-service

Traditional datacenter markets, such as managed hosting

Three emering cloud infrastructure markets

Different Cloud Computing Layers

(and example players)

Application Service (SaaS)

Google App Engine, Mosso, MS Live/ExchangeLabs, IBM, Google Apps, Salesforce.com Quicken Online, Zoho, Cisco

Application Platform

Server Platform

Storage Platform Amazon S3, Dell, Apple, ...

3Tera, EC2, SliceHost, GoGrid, RightScale, Linode Google App Engine, Mosso, Force.com, Engine Yard, Facebook, Heroku, AWS

The Amazon example …

(Why is an internet bookstore entering this market?)

• They already have the capability – The hardware infrastructure

– The software infrastructure: The Amazon – The software infrastructure: The Amazon

Web Services (AWS)

• They have spare capacity during non-peak

periods

34

The Amazon example …

• EC2 is a web service that provides resizable compute capacity in the cloud

• S3 provides a web services interface to store and retrieve any amount of data, at any time, from anywhere on the web

• SimpleDB is a web service for running queries on structured data in real time

on structured data in real time

• CloudFront is a web service for content delivery (software distributions, web content, media files)

• SQS offers a reliable, highly scalable, hosted queue for storing messages as they travel between computers

• Mechanical Turk is a web service for programmatically access to marketplace for work that requires human intelligence 35

Amazon EC2 Concepts

• Amazon Machine Image (AMI):

– Bootable root disk – Pre-defined or user-built – Catalog of user-built AMIs

– OS: Fedora, Centos, Gentoo, Debian, Ubuntu, Windows Server

– App Stack: LAMP, mpiBLAST, Hadoop

• Instance:

– Running copy of an AMI – Launch in less than 2 minutes – Start/stop programmatically

• Network Security Model:

– Explicit access control – Security groups

(7)

Standard EC2 Instances

• Small: 1.7 GB, 1 EC2 Compute Unit (1 virtual core), 160 GB storage, 32-bit platform • Large: 7.5 GB, 4 EC2 Compute Units (2 virtual

cores), 850 GB storage, 64-bit platform • Extra Large: 15 GB, 8 EC2 Compute Units (4

virtual cores), 1690 GB storage, 64-bit platform virtual cores), 1690 GB storage, 64-bit platform

One EC2 Compute Unit equivalent to a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor

37

Standard

Instances Linux/UNIX Windows

Small (Default) $0.10 per hour $0.125 per hour Large $0.40 per hour $0.50 per hour Extra Large $0.80 per hour $1.00 per hour

EC2 High CPU Instances

• Medium: 1.7 GB, 5 EC2 Compute Units (2 virtual cores), 350 GB storage, 32-bit platform • Extra Large: 7 GB of memory, 20 EC2

Compute Units (8 virtual cores), 1690 GB storage, 64-bit platform

38

High CPU

Instances Linux/UNIX Windows

Medium $0.20 per hour $0.30 per hour Extra Large $0.80 per hour $1.20 per hour

Pay only for what you use

On-demand capacity allocation Own capacity

Resource need

Simple Storage Service (S3)

Storage for the Internet

• Write, read, and delete 1 B to 5 GB objects • Unlimited number of objects

• Each object is stored in a bucket and retrieved via a unique, developer-assigned key.

• A bucket can be located in the United States or in Europe.

Europe.

• All objects within the bucket will be stored in the bucket’s location, but the objects can be accessed from anywhere.

• Objects can be made private or public • REST and SOAP interfaces

• Default download protocol is HTTP

• A BitTorrent protocol interface is provided to lower costs for high-scale distribution.

40

S3 pricing

Storage

$0.150 per GB – first 50 TB / month of storage used $0.140 per GB – next 50 TB / month of storage used $0.130 per GB – next 400 TB /month of storage used $0.120 per GB – storage used / month over 500 TB

Data Transfer

41

Data Transfer

$0.100 per GB – all data transfer in

$0.170 per GB – first 10 TB / month data transfer out $0.130 per GB – next 40 TB / month data transfer out $0.110 per GB – next 100 TB / month data transfer out $0.100 per GB – data transfer out / month over 150 TB

Requests

$0.01 per 1,000 PUT, COPY, POST, or LIST requests $0.01 per 10,000 GET and all other requests* * No charge for delete requests

SimpleDB

• CREATE a domain to house structured data. • GET, PUT or DELETE items in your domain,

along with the attribute-value pairs. • QUERY your data set using this simple set of

operators: =, !=, <, > <=, >=, STARTS-WITH, AND, OR, NOT, INTERSECTION AND WITH, AND, OR, NOT, INTERSECTION AND UNION.

• Pay only for the resources that you consume – Data transfer

– Compute hours – Storage

(8)

Common AWS features

• Provide application platforms, including

resources

• Accessed over the web • Simple to use

• Easy to get started (just need a credit card) • Pay-per-use

• Pay-per-use

• No contracts (committing to future use)

43

Cloud attractions

Cost – especially for peaks

Flexibility; rapid scalability and de-scalability Data replication

Easier cross-institution collaboration Any {time, place, device} access via web

browser browser

Alternative if departmental or central IT

non-responsive

Priorities: no need to focus on commodity IT Future of computing

Cloud concerns

Loss of control

Integration: enterprise & federated

authorization

Interoperability: with key enterprise apps Accessibility and user interface limitations of

web apps web apps

Reliability, performance, security Offline access

Features; changes; vendor lock-in Policy/compliance concerns (privacy) Business “surprises”; Support; More Logins Consequences of “Creative Destruction”

The RESERVOIR Vision

Next Generation Infrastructure for Service Delivery

– Federation of clouds

– Leverage locality – enable migration

– Service definition, SLA management, accounting and billing – Open specifications

– Diversity in underlying technologies (e.g., for server virtualization)

virtualization)

Analogies exists in areas outside services:

– Electrical power delivery: capacity can be shifted to guarantee supply and lower costs

– Roaming cellular communications: Talk wherever you are

Grid-aware Virtualization Service-oriented capacity provisioning across sites Virtualization-aware Grid Optimal placement of VMs on a federated cloud

Business & Service Management

Policy-based management of service-level

agreements

SOI

The Reservoir Architecture

Service Manager Service Provider

SLA SLA

SD+ SLA

• Monitors service and enforces SLA compliance by managing capacity of Service Components (VEEs) or/and size of Service Tiers

• Deals with mapping of service metrics (response time) to infrastructure metrics (VEE size)

(SM)

Infrastructure Provider = Site/Domain/Cloud VEE Management System

VEE Management Enablement Layer

Virtualized Physical Resource (e.g., Hypervisor)

infrastructure metrics (VEE size)

• Monitors VEEs and finds best VEE placement

• Deals federation of domains

VEE = Virtual Execution Environment

(9)

The Reservoir Architecture

Service Manager Service Provider SLA SLA SD+ SLA (SM) Clear separation of concern &

delegation of responsibility, e.g.,

• SM unaware of placement (local & remote)

• Primary VEEM takes the role of an SM towards remote site

Infrastructure Provider = Site/Domain/Cloud VEE Management System

VEE Management Enablement Layer

Virtualized Physical Resource (e.g., Hypervisor)

VEE = Virtual Execution Environment (VEEM)

an SM towards remote site

• Remote VEEM sees no difference between local SM and remote VEEM

Service Applications on Reservoir

One multi-VEE

application on: – One VEE host – Multiple VEE hosts – Multple sites

SM may specify placement constraints, e.g.,

– When physical nearness is needed

– For redundancy – Various types user

requests

Illustrative usage scenario

• Novel capabilities which

will enable the

deployment of commercial service scenarios which cannot be supported cannot be supported today

• Example:

The Winter Olympics Scenario

Service Definition

<service Olympic Games… > <tier web-servers … > <VEE-requirement … > <image … > <software … > <storage …> <network … > <configuration … > <tier-QoS … >

Web site service for

Olympics

Tier definition (web servers, application servers, databases) using service definition language

• Required VEEs </tier><tier-QoS … > <tier app-servers … > … </tier> <tier DB-servers … > … </tier> <inter-tier-configuration … > <service-QoS … > … </service> • Required VEEs • Software • Images • Storage • Network • Required configuration • Inter-tier relations •Required QoS.

EU Olympics Scenario – Service Deployment

•Service manager translates service specifications into infrastructure

requirements, and requests capacity from local VEEM

•The primary VEEM may, e.g., setup the required configuration on local site

<service EU-GAMES … > <tier web-servers … > <VEE-requirement … >

Primary Management Site <VEE-requirement … > <image … > <software … > <storage …> <network … > <configuration … > <tier-QoS … > </tier> <tier app-servers … > … </tier> <tier DB-servers … > … </tier> <inter-tier-configuration … > <service-QoS … > … </service> VEE Phys server

Primary Management Site

EU Olympics Scenario – Requesting remote resources

MS1

•For HA and assuring SLA, the primary VEEM requests capacity at remote sites

•Each MS deploys the service (according the

contracted resources) <deploy descr.. -<deploy descr .. <deploy descr.. -<deploy descr.. Primary Management Site

(10)

EU Olympics Scenario – HA with Live Migration

MS1

•Primary Management Site suffers electricity problems and needs to power off physical servers.

•It negotiates over the MS-MS protocol additional resources at MS1

•It evacuatesthe VEEs on the servers to be powered off,

migrating to MS1

•Live migration to maintain application servers’ states and client connections

MS2 Primary Management

Site

EU Olympics Scenario – On Demand Service Expansion • Load increases and the Primary Management Site realizes that the

available resources at the 3 sites are not enough by executing elasticity rules

• It negotiates with additional MS3 and requests resources

• MS3 deploy the service (according the contracted resources) MS1 • MS3 deploy the service (according the contracted resources) MS1

MS2 MS3

Primary Management Site

SOI: Grid Computing

Grid node or

Service Site (datacenter) Physical Resources

Service Tasks

Color of service task illustrates owning organization

SOI: Grid Computing +

Virtualization

Virtual Execution Environment

(VEE)

Improved isolation, Relax dependencies. Well defined billing units

(VEE)

Policy 1: If possible keep VEEs from the same organization in the same physical box

SOI: Grid Computing +

Virtualization + BSM

physical box

physical box Policy 2: Turn off underutilized physical boxes

(11)

Virtualization + BSM

Policy 2: Turn off underutilized physical boxes physical box

Local optimizations (within a single site): placement, power, etc.

Policy 3: If possible keep VEEs in “owning” organization

SOI: Grid Computing +

Virtualization + BSM – Boundaries

Policy 3: If possible keep VEEs in “owning” organization

SOI: Grid Computing + Virtualization + BSM – Boundaries Policy 4: If possible keep VEEs in least number of external organizations Policy 3: If possible keep VEEs in “owning” organization

Virtualization + BSM – Boundaries

Policy 4: If possible keep VEEs in least number of external organizations Policy 5: “Follow” the service customer

– Boundaries

Migration across sites Global optimizations:

placement, cost, bandwidth, etc.

Virtualize the Network

Create virtual networks connecting VEEs regardless of physical server location

(12)

Virtualize the Network and the

Storage

Enable secure access to relevant data regardless of storage location

The Evolution of the Power Grid

m u s e u m .or g / c ol le c ti on / e v e n t. p h p ? id = 3 4 5 6 8 7 6 http://www.pbase.com/rbenny/image/29116201 http://www.rootsweb.com/~nytigs/BurdenPayrollRecords.htm

The Burden Iron Works Water Wheel

h tt p :/ / ie e e -v ir tu a l-m u s e u m .or g / c ol le c ti on / e v e n t. p h p ? id =

The Pearl Street Station

•Make your own infrastructure •Not the company’s main

business but a considerable competitive advantage

•The utility industry •Metering •Limited reach

•Efficient distribution •Federation of providers •The diversity factor •Economies of scale http://www.anl.gov/Media_Center/logos22-1/electricity.htmThe US National Power Grid

The Evolution of the Compute

Grid

R E S E R V O I R

“… “… “…

“…will move towards a mix of will move towards a mix of will move towards a mix of microproductionwill move towards a mix of microproductionmicroproductionmicroproduction and and and and large utilities, with increasing numbers of large utilities, with increasing numbers of large utilities, with increasing numbers of

large utilities, with increasing numbers of smallsmallsmallsmall----scale scale scale scale producers

producers producers

producers cococo----existing with largecoexisting with largeexisting with largeexisting with large----scale regional scale regional scale regional scale regional producers

producers producers

producers, and , and , and , and load being distributed among them load being distributed among them load being distributed among them load being distributed among them

•Make your own infrastructure •Not the company’s main

business but a considerable competitive advantage

•Efficient distribution •Federation of providers •The diversity factor •Economies of scale http://www.by-star.net/techspeak/datacenter/

http://www.smcplus.com/applications.asp?id=32

http://www.informationweek.com/galleries/showImage.jhtml?galleryID=62&imageID=13 Google @ The Dulles, OR

producers producers producers

producers, and , and , and , and load being distributed among them load being distributed among them load being distributed among them load being distributed among them dynamically dynamically dynamically dynamically…”…”…”…” There’s There’s There’s

There’s Grid and then Grid and then Grid and then tharGrid and then thartharthar Clouds Clouds Clouds Clouds ---- Ian FosterIan FosterIan FosterIan Foster •The utility industry

•Metering •Limited reach

Sample Research Challenges

• Overall architectural issues

– Separation of concern, delegation of responsibility

– Minimize inter-component dependencies while preserving degrees of freedom for optimizing functionality

– Information and data models – Information and data models – Policy infrastructures – etc

Sample Research Challenges

Service Level Challenges

• Translate business concept requirements to infrastructure requirements

– E.g., response time to CPU utilization – Define a Service Definition Language to

characterize all information and context required to enable lifecycle management of required to enable lifecycle management of services across RESERVOIR sites

– Must be able to handle rollback on deployment failures

• SLA definitions

• Support multiple levels of QoS

Sample Management Challenges

• Support policy based management across

administrative domains (clouds)

– Automatically hire additional “power” from another clouds (based on pre-existing framework agreements about guaranteed capacity or best effort dynamic requests) • Create an inter-site protocol to allow for

federation of RESERVOIR sites federation of RESERVOIR sites • Protect Service Level Agreements

– Detect violations (SLA monitoring) – Predict and avoid SLA violations – Elasticy rules (definition, execution) – Provide for dynamic relocation of resources – Provide accountability

• Bill for services used, even across RESERVOIR sites

– Different billing and accounting systems may be used

(13)

Sample Infrastructure-level Challenges

• Provide for relocation of resources without

boundaries

– Live migration across subnet boundaries

– Migration to a different physical host without shared storage

• Provide standardized interfaces for lifecycle management to Virtualized Execution Environment • Analyze end-to-end performance in a virtualized • Analyze end-to-end performance in a virtualized

environment to understand bottlenecks

• Be able to handle surges in 3-5 orders of magnitude in service requests

• Provide cost-efficient VEE placement

– ”Optimal” local placement decisions – ”Optimal” remote placement decisions – Local vs. remote placement

– Note differences compared to job scheduling – services typically have ”infinite” lifetime and varying resource utlization

RESERVOIR– EU FP7 IP

3-years, started February 2008 (17 M Euro budget)

Partner Role Comment

1. IBM, Israel Technology Virtualization/SOC Infrastructure. Project coordinator. 2. Telefonica I+D, Spain Technology Service Technology, Billing Infrastructure 3. UC London, UK Technology Virtualization Technology

4. Umeå Univ., Sweden Technology Grid technology, Resource Management, Accounting 5. SAP AG, Germany Use-Cases Use-Cases, Contribution to Requirement, Standards

www.reservoir-fp7.eu

5. SAP AG, Germany Use-Cases Use-Cases, Contribution to Requirement, Standards 6. Thales, France Technology Security, Virtualization Infrastructure, Hosting 7. Sun Microsystems Use-Cases+Tech Contribution to Standards, Java Services, Monitoring 8. DATAMAT Technology Service Management Technologies 9. UC Madrid, Spain Technology Grid, Dynamic Allocation Technology 10. CETIC, Belgium Technology Security

11. Univ Lugano, Switz. Technology Monitoring and SLA Management 12. Univ. Messina, Italy Technology Grid Experience, Testbed Development 13. Open Grid Forum Technology Standardization

Grid computing @ UmU

Erik Elmroth (Associate Professor)*

Bo Kågström (Professor) Francisco Hernandez (PhD, Postdoc)* Daniel Henriksson (PhD Student)* Lars Larsson (Software Developer)* Johan Tordsson (PhD Student)* P-O Östberg (PhD Student)