Positioning Performance Testing to Cut Costs of Cloud Computing

(1)

1

Positioning Performance Testing to

Cut Costs of Cloud Computing

Thomas Barns

UKCMG TEC 2011

Positioning Performance Testing to Cut Costs of Cloud Computing

Agenda

• Benefits of cloud computing

• Performance, capacity and cost

• Testing and Modelling

(2)

3

Cloud Computing Benefits

• Why use cloud computing? • Reduced cost

• Increased flexibility

?

UKCMG TEC 2011

Physical Capacity Management

Compare to Requirements Capacity Plan Order hardware Provision hardware Measure hardware

(3)

5

Physical Capacity Management

Compare to Requirements Capacity Plan Order hardware Provision hardware Measure hardware UKCMG TEC 2011

Cloud Capacity Management

Compare to Requirements Capacity Plan Launch instances Measure hardware

(4)

7

Cloud Capacity Management

Compare to Requirements Capacity Plan Launch instances Measure hardware UKCMG TEC 2011

• This can even be automated

• And so…

Cloud Capacity Management

Compare to Requirements Launch instances Measure hardware

(5)

9

No more capacity planning!?

When the Associated Press (AP) wanted the flexibility for

application hosting and cloud data storage in the cloud, they turned to the Windows Azure platform from Microsoft. "Capacity planning is the thing that stands out as the biggest

advantage of the Microsoft cloud model. The Windows Azure platform takes that out of the equation for us, unlike the other cloud providers."- Jonathan Malek, Chief Architect and Director of Research, Associated Press

See how Windows Azure helped the AP develop a new global API through easy scalability that removed the need for costly

and time-consuming capacity planning.

Microsoft.com

UKCMG TEC 2011

Physical vs Cloud

0 50 100 150 200 250 300 350 400 450 Time Cloud

Demand Waste Capacity 0 50 100 150 200 250 300 350 400 450 Time Physical

(6)

11

The key questions

How many servers? How much will it cost?

Time

Cloud

Demand Waste Capacity Time

Physical

Demand Waste Capacity

UKCMG TEC 2011

The key questions

• Understand the application • Model future user demand • Model utilisation

• Understand acceptable utilisation thresholds

• Plan how many servers to buy and when

How many servers?

Time

Physical

(7)

13

The key questions

• Understand the application • Model future user demand • Model utilisation

• Understand acceptable utilisation thresholds

• Plan how much to spend and when

How much will it cost?

Time

Cloud

UKCMG TEC 2011

Capacity Planning and Budgeting

• Capacity planning is as necessary as ever • to calculate the budget

(8)

15

Testing and Modelling

Modelling Testing

Application Hardware/Cloud

Infrastructure Usage Scenarios

UKCMG TEC 2011

Testing and Modelling

• Performance Testing

• Understand the application

• Understand how the application relates to infrastructure/cost • Modelling

• Understand user behaviour

(9)

17

Performance, Capacity, Cost

Cost

Capacity

Performance

• The performance of the system impacts • Utilisation of available capacity • And cost!

UKCMG TEC 2011

Performance to Cost

Performance

• Improving performance by reducing resource costs reduces resource utilisation

Capacity

• Reducing the utilisation of resources means that less resources are required

Cost

• Reduced resource requirements should result in real cost savings

(10)

19

Case study: Background

• A client set up a mobile site to provide a better experience for customers during periods of peak demand

• This site was to be hosted as a cloud based service • The usage of the site would start off low and scale up to

unknown levels depending on the scale of high demand peaks

UKCMG TEC 2011

Performance Testing

• To ensure that

• User response times will be within performance SLAs • System will provide value for

money within budgets

Find/Fix Code Defects Determine and Optimise Response Times Determine and Optimise Costs

(11)

21

Step 1: Finding Code Defects

• Before response times and costs can be measured, defects introducing non-linearity must be found and fixed

• Memory leaks • Data growth • Logical bottlenecks • Locking

UKCMG TEC 2011

(12)

23

Step 2: Determining Response Time

• Response times are the measure of performance seen by the user

• What will they see?

• Are times within SLAs and guidelines?

UKCMG TEC 2011

Step 2: Determining Response Time

Average Before fix Average After fix 1 Average After fix 2 Average SLA

(13)

25

Step 3: Cutting Costs

• Costs are a key measure of performance seen by the business rather than the user

• What is the resource cost of key transactions?

• This needs to be modelled to understand the total forecast costs

• Improvements should be made to approach cost optimality

UKCMG TEC 2011

Step 3: Cutting Costs

• Performance tests could now achieve, on a single instance • 2 user visits a second

• With CPU at 75% • Isn’t that good enough?

(14)

27

Step 3: Cutting Costs

• We can (probably) handle initial load with one instance

• Because it’s on the cloud, we can scale it easily by adding in more instances if we need them

Time

Cloud

UKCMG TEC 2011

Step 3: Cutting Costs

• But will those instances be added in time?

• And – how much will they cost?

Time

Cloud

(15)

29

Step 3: Cutting Costs

• After optimisation, tests could achieve • 10 user visits a second

• With CPU at 15%

• So at 75% CPU we could now handle 50 user visits a second, rather than 2

UKCMG TEC 2011

Step 3: Cutting Costs

• 25 cloud servers? • Or 1?

Time

Cloud

(16)

31

Summary

• Cloud computing has benefits, if used correctly

• Capacity management is as necessary as ever

• Performance testing is key to successful outcomes and cutting

costs of cloud computing

UKCMG TEC 2011

Questions?

Thomas Barns Consultant