(cloud) SERVICE DESIGNER

(1)

ELIS ICT Management

Alta Formazione 2011

Nel semestre CONSEL di presidenza

Hewle@-‐Packard Italiana

Relatore: Emiliano Casalicchio / Luca Silvestri

email: [email protected]

[email protected]

Quality of Service

in

(2)

Agenda

Part 1.

(E.Casalicchio -- 2h)

•

QoS in distributed systems

– 

General definition of QoS

– 

QoS in Grid, Web Services, SoA, …

– 

Governance of QoS in DS: General approaches

•

Mapping QoS concepts on the Cloud Stack

– 

IaaS QoS

– 

PaaS QoS

– 

SaaS QoS

– 

Cross layer dependencies and issues

•

QoS Governance

(3)

Agenda

Part 2.

Solutions offered by vendors (L.Silvestri -- 1.5h)

•

Iaas Providers

– 

Survey

– 

Taxonomy

•

The AWS case study

– 

AWS services

– 

EC2 Platform

(4)

Agenda

Part 1.

•

– 

QoS in Grid, Web Services, SoA, …

– 

•

– 

IaaS QoS

– 

PaaS QoS

– 

SaaS QoS

– 

•

QoS Governance

(5)

Quality of Service concept

• 

There is no common or formal definition of QoS.

• 

In the field of telephony/Comm-net, quality of service was defined

in the ITU standard X.902 as "

A set of quality requirements on

the collective behavior of one or more objects

".

• 

Quality of Service comprises requirements on all the aspects of a

connection, such as:

–

service response time,

–

loss,

–

signal-to-noise ratio,

–

interrupts,

–

frequency response,

(6)

Quality of Service concept

•

QoS deal with non-functional properties of systems that

affect the quality of the service provided

•

The main non-functional properties that impact the

service level experienced by clients and users are:

– 

performance

– 

reliability

– 

security

– 

adaptability

– 

availability

QoS dimensions

(7)

Main deﬁniPons

• 

Performance

–

the amount of useful work accomplished by a computer system

compared to the time and resources used

• 

Reliability

–

the ability of a system or component to perform its required functions

under stated conditions for a specified period of time.

• 

Availability

–

the proportion of time a system is in a functioning condition

• 

Security

–

a form of protection where a separation is created between the assets

and the threat

• 

Adaptability

–

the ability of a system (e.g., a computer system) to adapt itself

(8)

QoS Dimensions

•

Depending on the application context, different

dimensions can apply

•

More dimensions can be added

– 

scalability, capacity, performance, reliability, availability,

– 

flexibility, exception handling,

– 

integrity,

– 

regulatory, supported standard,

– 

stability,

– 

cost,

(9)

Quality of Service Concept In SW eng.

•

ISO 9126 is an international standard for

the evaluation of software.

•

Quality is defined as:

The totality of

features and characteristics of a

product or service that bear on its

ability to satisfy stated or implied

needs

–

Stated needs are explicitly declared by

the users

–

Implied needs refers to requirements

users do not know

•

Quality is a combination of three types of

qualities:

–

In use: related to the quality perceived by

the user

–

Internal: regardless of the context in

which it is used

–

External: related to context in which it is

(10)

Service Level Agreement: concepts &

deﬁniPons

• 

Web Services

–

Sla is an agreement used to guarantee web service delivery. It define the

understanding and expectations from service provider and service consumer

(HP Lab, 2002)

• 

Networking

–

An SLA is a contract between a network service provider and a customer that

specifies, usually in measurable term, what services the network service

provider will supply and what penalties will assess if the service provider will

not meet the established goals (R.P.)

• 

Internet

–

SLA constructed the legal foundation for the service delivery. All parties

involved are users of SLA. Service consumer uses SLA as legally binding

description of what provider promised to provide. The service provider uses it to

have a definite, binding record of what is to be delivered (Internet N.G.)

• 

Data Center Management

–

SLA is a formal agreemnt to promise what is possible to provide and provide

(11)

SLA components

•

Purpose

– 

Objectives to achieve by using an SLA.

•

Restrictions

– 

Necessary steps or actions that need to be taken to ensure that

the requested level of services are provided.

•

Validity period

– 

SLA working time period.

•

Scope

– 

Services that will be delivered to the consumers, and services

that will not be covered in the SLA.

•

Parties

– 

Any involved organizations or individuals involved and their

roles (e.g. provider and consumer).

(12)

SLA components

•

Service-level objectives (SLO)

– 

Levels of services which both parties agree on. Some service

level indicators such as availability, performance, and reliability

are used.

•

Penalties

– 

If delivered service does not achieve SLOs or is below the

performance measurement, some penalties will occur.

•

Optional services

– 

Services that are not mandatory but might be required.

•

Administration

– 

Processes that are used to guarantee the achievement of

SLOs and the related organizational responsibilities for

controlling these processes.

(13)

Service Level ObjecPve

• 

A Service level objective (SLO) is a key element of a service level

agreement (SLA) between a service provider and a customer.

• 

SLOs are agreed as a means of measuring the performance of the

Service Provider and are outlined as a way of avoiding disputes

between the two parties based on misunderstanding.

• 

SLOs are specific measurable characteristics of the SLA such as

–

availability

–

throughput

–

frequency

(14)

SLO

•

SLOs must be:

– 

Attainable

– 

Repeatable

– 

Measurable

– 

Understandable

– 

Meaningful

– 

Controllable

– 

Affordable

– 

Mutually acceptable

•

Examples

– 

95-percentile or response time < 3 sec.

– 

99.99% availability

(15)

Actors involved in QoS

• 

Stakeholder

–

who requests the development of a Web service to a developer defining the

quality requirements

• 

Developer

–

who develops the Web service meeting the requested quality level

• 

Provider

–

who really makes available a Web service to the consumer

• 

Consumer

–

who actually uses the Web service

• 

QoS Broker

–

at consumer side who collects information on the quality

• 

Quality Assurer

–

who monitors the quality level to state the respect for the contract between

consumer and provider

• 

Quality Manager

(16)

(17)

Actors involved in QoS: Deployment

(18)

Actors involved in QoS: InvocaPon/use

(19)

SLA Life Cycle [Ron 2001]

•

the

creation phase,

– 

in which the customers find service provider who matches their

service requirements.

•

the

operation phase,

– 

in which a customer has read-only access to the SLA.

•

the

removal phase,

– 

in which SLA is terminated and all associated configuration

information is removed from the service systems.

1. Creation

phase

2. Operaton

Phase

3 Removal

Phase

(20)

SLA Life cycle [SUN Microsystem IDG 2002]

1. Discover Service Provider 2. Define SLA 3. Establish Agreement 4 Monitor SLA violation 5. Terminate SLA 6. Enforce penalties for SLA violation

• 

Discover - service providers

–

in where service providers are located according to

consumer’s requirements.

• 

Define – SLA

–

which includes definition of services, parties, penalty

policies and QoS parameters. In this step it is possible to

negotiate between parties to reach a mutual agreement

• 

Establish – agreement

–

in which an SLA template is established and filled in by

specific agreement, and parties are starting to commit to

the agreement.

• 

Monitor – SLA violation

–

in which the provider’s delivery performance is measured

against to the contract

• 

Terminate – SLA

–

in which SLA terminates due to timeout or any party’s

violation

• 

Enforce - penalties for SLA violation

(21)

Agenda

Part 1.

•

– 

•

– 

IaaS QoS

– 

PaaS QoS

– 

SaaS QoS

– 

•

QoS Governance

(22)

Quality of Service Concept (in Web Services)

• 

D.A.Menascé define

Quality of Service

(QoS) a combination of

several qualities or properties of a service, such as:

–

Availability

is the percentage of time that a service is operating.

–

Security properties

include the existence and type of

authentication

mechanisms the service offers,

confidentiality

and

data integrity

of

messages exchanged,

nonrepudiation

of requests or messages, and

resilience

to denial-ofservice attacks.

–

Response time

is the time a service takes to respond to various types of

requests. Response time is a function of load intensity, which can be measured

in terms of arrival rates (such as requests per second) or number of concurrent

requests. QoS takes into account not only the average response time, but also

the percentile (95th percentile, for example) of the response time.

–

Throughput

is the rate at which a service can process requests. QoS

measures can include the maximum throughput or a function that describes

how throughput varies with load intensity

Menascé, D. A. 2002. QoS Issues in Web Services. IEEE

Internet Computing 6, 6 (Nov. 2002), 72-75.

(23)

Quality of Service Concept (in Web Services)

•

The QoS measure is observed by Web services users.

•

Users are not human beings but programs that send requests for services

to Web service providers. QoS issues in Web services have to be

evaluated

–

from the perspective of the providers of Web services and

–

from the perspective of the users of these services

Menascé, D. A. 2002. QoS Issues in Web Services. IEEE

Internet Computing 6, 6 (Nov. 2002), 72-75.

(24)

Quality of Service Concept (in Grid)

•

D.A. Menasé and E.Casalicchio consider three main

QoS dimensions in grid computing

– 

Latency

– 

Throughput and

– 

Availability

E.Casalicchio, D.A.Menascé, Quality of Service Aspects and Metrics in

Grid Computing, CMG Int. Conference 2005

(25)

• 

Latency: time to execute a task.

–

Application layer: time to complete immediate

quote requests; 95-percentile of the time to

respond to non-immediate queries; Elapsed

time to return a delayed quote

–

Collective layer: Average time to perform

co-reservation and co-allocation of resources to

run a RAM

–

Resource layer: Time to access a specific

resource through the Grid Resource Access

Management protocol.

–

Connectivity layer: Time to perform an

authentication on behalf of a customer to a

financial organization using the GSI.

–

Fabric layer: Time taken by the database

server at a health insurance company to reply

to a query.

(26)

• 

Throughput: units of work executed per unit

time.

–

Application layer: Number of delayed requests

processed per second.

–

Collective layer: Number of queries per second

that can be handled by the directory service

used to locate resources across different VOs.

–

Resource layer: Effective transfer rate under the

Grid-FTP protocol used to transfer files among

computers running RAMs. Queries/sec that can

be processed by the database server of a law

enforcement agency needed by a RAM

application

–

Connectivity layer: Throughput (in KB/sec) of a

secure connection between an instance of a

RAM model and a database service that

provides information about customers

–

Fabric layer: Number of CPU cycles per second

obtained from a machine involved in processing

a task that is part of a parallel RAM evaluation.

(27)

•

Availability: fraction of time a resource/application

is available for use

–

Application layer: Fraction of time that the

immediate quote requests application is available.

–

Collective layer: Fraction of a time that the directory

service used to locate resources across different

Vos is available.

–

Resource layer: Fraction of time that a specific

computing node is available for allocation to a RAM

instance.

–

Connectivity layer: Fraction of time that a proxy

authentication request succeeds in authenticating a

user with a law enforcement agency.

–

Fabric layer: Fraction of time that a computing node

(28)

QoS in the SOA (as Cloud implemenPng

technology)

Papazoglou, M. P. and Georgakopoulos, D. 2003.

Introduction. Commun. ACM 46, 10 (Oct. 2003),

24-28.

(29)

Agenda

Part 1.

•

– 

•

– 

IaaS QoS

– 

PaaS QoS

– 

SaaS QoS

– 

•

QoS Governance

(30)

QoS Governance issues

•

Mapping of SLO on the system components

•

Measuring SLO

(31)

SLO to Service level thresholds

• 

One of the main problem is to translate SLO specified in SLA into

lower level resource requirements

–

For example, what do imply an SLO

R<10 sec

in term of maximum

CPU utilization allowable, or in term of maximum arrival rate

admissible?

• 

Automatically deriving and inferring low level thresholds from high

level goals are difficult tasks due to the complexity and dynamism

inherent in such systems.

• 

The range of design choices in terms of operating systems,

middleware, shared infrastructures, software structures etc. further

complicates the problem.

–

For example, different virtualization technologies (e.g., Xen [2] or VMware [4])

can be used in a utility data center.

–

Applications can use different software structures (e.g., 2-tier PHP, 3-tier

Servlet, or 3-tier EJB [14]) to implement the same functionality.

–

Different implementations are also available for each tier (e.g., Apache or IIS

for web server; WebLogic, WebSphere, or JBoss for EJB server; Microsoft SQL

Server, Oracle, or MySQL as database server).

(32)

SLO to Service level thresholds

•

A component profile captures the component s performance

characteristics as a function of the resources that are allocated to the

component and its configuration.

–

e.g. a

component

can be an http server or an application server or a db server, etc..

•

Performance modeling captures the relationship between each single

component and the overall system performance.

SLA Decomposition: Translating Service Level Objectives to System Level Thresholds Yuan Chen, Subu Iyer, Xue Liu, Dejan Milojicic, Akhil Sahai

(33)

How to measure QoS

• 

Let us consider the latency dimension, that is

time to execute a

task, activity, process, etc..

• 

We can evaluate the

–

time series

–

probability distribution

•

pdf, CDF, CCDF

• 

We can measure

–

the instant value

–

the average value

–

the variance

(34)

QoS measures and QoS Dimensions

•

For a given dimension different type of metrics can be

proposed. For example:

Availability

: fraction of time a resource/application is available for use

–

Grid Application layer: Fraction of time that the immediate quote requests application

is available.

–

Grid Collective layer: Fraction of a time that the directory service used to locate

resources across different Vos is available.

–

SoA service provider layer: Fraction of time the service is available

(35)

How to measure QoS: the a-‐percenPle example

• 

The

a

-percentile of a r.v.

X

is

x

such that

–

Pr(X

≤

x) = a

• 

For example:

–

if the 95-percentile of the responce

time

is 20 sec this means that in the

95% of the observation the responce

time is less or equal then 20 sec.

•

An SLA on X-percentile is stronger

then on the average value

1

CDF is Pr(X

≤

x)

0 1 2 3 ... x ...

(36)

QoS Governance

•

Load balancing, load sharing

– 

Grid, web, web applications/services,

•

Caching and replication

– 

web

•

Geographical distribution

– 

web

•

Admission control

– 

communication systems, Grid, web

•

Dynamic service composition

– 

Web services, SOA

•

Dynamic VM allocation

(37)

Dynamic VM allocaPon example

Clients

Service

provider

VM

₂

S

VM

₁

S

VM

_i

S

Cloud

Dispatcher

VM

_n

S

Clients

SLA Performan ce indexes

(38)

(39)

Agenda

Part 1.

•

– 

•

– 

IaaS QoS

– 

PaaS QoS

– 

SaaS QoS

– 

•

QoS Governance

(40)

QoS in the Cloud: three layers

A Business Driven Cloud Optimization Architecture Marin Litoiu

Joanna Ng, Gabriel Iszlai

(41)

QoS in the Cloud: three layers

•

IaaS virtualizes the hardware layer and offers computing services such as

storage, CPU and memory.

–

The CPU, memory and local storage are packaged as virtual machines of

different sizes, each with a price per hour.

–

Storage is a continuous space with a price per kilobyte

•

PaaS offers platform services, such as web, application and database

servers and a programming model associated with it. Programmers can

use this environment to develop, test and deploy applications.

–

PaaS consumes the services of IaaS by requesting Virtual Machines and

storage and deploying application containers on the virtual machines. There

can be many PaaS deployed on the same IaaS.

•

SaaS layer consists of simple or composite services (applications) offered

to the end users.

–

Those applications are deployed in PaaS containers on topologies specific to

each application. There can be many applications sharing the same PaaS. In

general, a SaaS user pays a subscription.

Joanna Ng, Gabriel Iszlai

(42)

Quality of Service Concept (in Cloud)

•

Nallur et al. identify three type of cloud applications with different QoS requirements

–

Web-applications that cater to diverse users across the internet and demand fast

response times

.

–

Enterprise applications that cater to different business units across the world and require

large amounts of

secure

,

reliable

data transfer

and

high availability

(99.999%).

–

Scientific applications that need raw cpu, or enterprises that perform batch processing

•

Example requirements of a Cloud Application (CA)

–

Scalability

: Cloud providers charge for their service on the basis of cpu-hours used,

storage and bandwidth usage. Scale-up and Scale-down are mandatory for a CA

–

Availability

: Users are turned off if the service is not available due to frequent

over-loading. CA should be resilient in times of increased traffic and always be available.

–

QoS

: CA provides a lot of its services by composing several other web-services. If these

web-services fail to provide proper performance, then CA’s reputation plummets. Hence,

CA would like QoS from these web-services to be guaranteed via

Service Level

Agreements

–

Cost

: The cost of using other web-services could fluctuate depending on demand and

supply.

Self-Optimizing Architecture for Ensuring Quality Attributes in

the Cloud Vivek Nallur, Rami Bahsoon, Xin Yao

(43)

IaaS QoS

•

Goals.

Cost and revenue are functions of number of resources and price

per resource.

–

Maximizing the resource multiplexing and therefore increasing the

resource utilization

is an

important factor that can contribute to reducing cost and increasing revenue. Reports

suggest that only 10% of the IT capacity is currently used, theoretically resource utilization

can go up by a factor of 9. In practice, a factor of 5 to 7 is more feasible

–

Service requests from the layer above are for individual VMs, with performance

characteristics: memory, storage, processing capacity. To reach maximum utilization, the

management layer has to fit all the requirements into the smallest number of hardware

machines.

•

Constraints.

Capacity contracts with PaaS owners, and the available

processors, memory, storage, and networks (which may change due to

failures or equipment additions).

(44)

PaaS QoS

• 

Goals.

The revenue of this layer comes from the application/

services it hosts.

–

The cost is that of the resources (VMs, storage) it consumes from

IaaS, the cost of third party licences (Application or Data base servers,

for example) and the penalties it has to pay for SLA breaches.

–

The business goal of this layer is to maximize the profit by maximizing

the number of applications it hosts and minimizing the resources it

uses, and penalties it pays.

• 

Constraints.

SLA contracts with SaaS owners can be treated as

hard constraints instead of costs via penalties.

–

There may be provisioning delays in increasing the resources it has,

(45)

SaaS QoS

• 

Goals.

Most likely, this layer will charge the end user for

subscriptions (revenue proportional to the number of users), or

transactions (revenue proportional to the throughput).

–

Service costs include the payment for resources to PaaS which can

also be by usage or by subscription.

–

The application owner will deploy the application together with a set of

policies that capture the acquiring of resources from PaaS, the

obligation of PaaS to provide those resources, the price per resource

type, the penalties for not providing the resources, etc.

• 

Constraints.

There may be SLAs with application users that can be

(46)

QoS in the Cloud: Cross layer dependencies

• 

SLA violations a lower level impact higher levels SLA

Joanna Ng, Gabriel Iszlai

(47)

QoS in the Cloud: problems at diﬀerent levels

•

How to isolate

performance at single

node using VMs

(Isolation problem)

•

How to improve the

performance and

depensability of a

cluster (load

balancing, sharing

problem)

•

How to distribute VM

instances to increase

availability, resilience

and how to decrease

responce time (replica

placement problem)

A Break in the Clouds: Towards a Cloud De

ﬁ

nition Luis M. Vaquero , Luis Rodero-Merino , Juan Caceres ,

Maik Lindner

(48)

Examples of SLA oﬀered by Cloud Providers at

diﬀerent stack layers

•

AWS EC2 and S3 (IaaS)

•

ReliaCloud (IaaS)

•

StormOnDemand (IaaS)

•

GoogleAppEngine (PaaS)

•

Windows & co. Azure (PaaS/SaaS)

(49)

Agenda

Part 1.

•

– 

QoS in Grid, Web Services, SoA, …

– 

•

– 

IaaS QoS

– 

PaaS QoS

– 

SaaS QoS

– 

•

QoS Governance

(50)

(51)

MAPE-‐K implementaPon in Cloud based

systems

• 

Performance and workload monitoring components

–

are the sensors of the QoS-aware service provisioning system

and are in charge of collecting information on the performance

state of the computational, storage and network resources

used, as well as on the workload submitted by users (e.g.

requests rate, type, size).

–

Through Monitors relevant performance indexes (e.g.,

response time, network traffic, CPU utilization, etc.) and

workload information are collected, aggregated and elaborated.

• 

The SLA Analyzer component

–

is in charge of elaborating and analyzing data collected by

monitors. Moreover, the analyzer should trigger the Planner

component if significant changes in the data pattern, e.g. SLA

violations and abnormal workload fluctuations, are detected.

–

Data obtained by the monitors (average response time,

observed and expected arrival rate, etc.) are examined by this

component to check whether adaptive actions are needed to

guarantee the requested level of QoS.

(52)

systems

•

The Planner (or Provisioning Manager)} component.

–

This module, instrumented with a system model, evaluates the configuration

changes that have to be operated to guarantee SLAs and optimize costs.

–

Reconfiguration is triggered by the Analyzer and the system model is usually

parameterized with information collected by the performance and workload

monitors and elaborated by the Analyzer.

–

If an adaptation is needed, the Planner decides, on the bases of the system

performance model and of a resource allocation algorithm, the corrective

actions to take (e.g. how many VMs to allocate or deallocate).

(53)

systems

•

The Resource Manager (or VMs Allocator) component,

–

is in charge to actuate the strategies determined by the planner to properly

dimension and manage resources, e.g. allocate and deallocate VMs, resize the

capacity of running VMs, increase or decrease the storage.

•

The Load Balancer component

–

distributes requests among the instantiated resources, e.g. application servers,

web server, and databases, running on leased virtual machines. This module

should be capable of guaranteeing, at least, session persistency.

(54)

OpPmizaPon loop

(55)

Global and Local OpPmizaPon

(56)

IaaS

•

Sensors.

The IaaS layer has access to all hardware

counters as well as hypervisor and operating systems

counters.

– 

The sensors measure the used capacity, the available capacity,

the availability and the location of each resource.

•

Actuators:

VM allocation and storage allocation are the

main actuators to satisfy the performance requirements

and improve the utilization of the layer.

– 

Activation and deactivation of VMs can be used to increase

utilization without affecting the performance requirements.

(57)

PaaS

•

Sensors

. This layer monitors the virtual machines it

owns and the containers it deploys on the VMs.

– 

It has no access to the hardware resources performance

counters or virtualization hypervisors.

– 

It is very likely that this layer has a pool of licenses and a pay

per use licensing of third parties software and the monitoring of

those resources has to be performed as well.

•

Actuators

. This layer achieves its performance goals by

acting on several handlers:

– 

the number, size and type of VMs,

– 

the allocation of containers to the VMs (trying to fit as many in a

container, container settings, number of resources in use, etc).

(58)

SaaS

•

Sensors

. Each application monitors its Quality of

Service as it tries to optimize its goals (the number of

users and the number of transactions).

– 

QoS can include response time for each user request and

throughput of aggregated functions across multiple requests.

•

Actuators

: To reduce its cost, an application will try to

limit the number of resources it requests from PaaS.

– 

It should be able to control the number of instances of

containers, the deployment of the application on the containers,

tune application parameters.

(59)

Summary

•

– 

General definition of QoS, SLA concepts and SLA lify cycle

– 

Governance of QoS in DS: How to map and measure SLO,

mechanisms to enforce SLO

•

– 

IaaS QoS

– 

PaaS QoS

– 

SaaS QoS

– 

•

QoS Governance

– 

The MAPE-K cycle