Testing of Cloud Services The Approach: From Risks to Test Measures

(1)

Kees Blokland and Jeroen Mengerink, Polteq

Testing of Cloud Services

The Approach: From Risks to Test

Measures

www.eurostarconferences.com

@esconfs #esconfs

(2)

Testing of Cloud Services

The Approach: From Risks to Test

Measures

Kees Blokland Jeroen Mengerink Polteq Test Services BV

(3)

(4)

searching, recording, accounting, paying, writing,

reviewing, tracking, calculating, developing, listening,

analyzing, transmitting, learning, controlling,

purchasing, testing, alarming, changing, updating,

deleting, accessing, rejecting, correcting, studying,

booking, receiving, tracing, protecting, deciding,

managing, teaching, facilitating, identifying, copying,

removing, demonstrating, checking, showing,

selecting, subscribing, unsubscribing, sharing,

mailing, communicating, reading, playing, working,

meeting, gambling, shopping, storing, cross

checking, retrieving, configuring, sketching, saving,

accelerating, enhancing, creating, growing, checking

in, checking out, finding out, reaching, denying,

talking, designing, making, verifying, measuring

(5)

5

Email Surf Transfer Develop and Test

(6)

6

Operate and Manage Store

storage claim

80% unused

redundancy limitations

(7)

7

Operate and Manage Store

standard software bandwidth

internet technology

virtualization SOA

(8)

8

searching, recording, accounting, paying, writing,

reviewing, tracking, calculating, developing, listening,

analyzing, transmitting, learning, controlling,

purchasing, testing, alarming, changing, updating,

deleting, accessing, rejecting, correcting, studying,

booking, receiving, tracing, protecting, deciding,

managing, teaching, facilitating, identifying, copying,

removing, demonstrating, checking, showing,

selecting, subscribing, unsubscribing, sharing,

mailing, communicating, reading, playing, working,

meeting, gambling, shopping, storing, cross

checking, retrieving, configuring, sketching, saving,

accelerating, enhancing, creating, growing, checking

in, checking out, finding out, reaching, denying,

talking, designing, making, verifying, measuring

(9)

(10)

US: National Institute of Standards and Technology

http://www.nist.gov

Essential characteristics



On-demand service

 Self service provisioning, pay-per-use

(11)

US: National Institute of Standards and Technology

http://www.nist.gov

Essential characteristics



On-demand service



Broad network access

 Standard mechanisms over networks

(12)

US: National Institute of Standards and Technology

http://www.nist.gov

Essential characteristics



On-demand service



Broad network access



Resource pooling

 Multi-tenant

 Storage, processing, memory, virtual machines, …

(13)

US: National Institute of Standards and Technology

http://www.nist.gov

Essential characteristics



On-demand service



Broad network access



Resource pooling



Rapid elasticity

 Rapid scale in and out

(14)

US: National Institute of Standards and Technology

http://www.nist.gov

Essential characteristics



On-demand service



Broad network access



Resource pooling



Rapid elasticity



Measured service

 Controlled resource use

(15)

US: National Institute of Standards and Technology

http://www.nist.gov

Essential characteristics



On-demand service



Broad network access



Resource pooling



Rapid elasticity



Measured service

Deployment models – private cloud – community cloud – public cloud – hybrid cloud Service Models Software as a Service Platform as a Service Infrastructure as a Service

(16)

Continuity Privacy Multi platform Legislation Cyber crime Impact organisation Standards 143

(17)

Continuïty Privacy Multi platform Legislation Cyber crime Impact organisation Standards Performance Security

Availability & Continuity

Functionality

Manageability

Legislation & Regulations

Suppliers & Outsourcing

R

i

s

k

(18)

19

Other customers

YOUR Operational Profile YOUR Operational Profile PLUS Performance Security

Functionality

Manageability

R

i

s

k

(19)

20

Everything over the web

The idea: “it’s safe”

Home ground for hackers

Performance

Security

Functionality

Manageability

R

i

s

k

(20)

21

Bring Your Own Device

No free choice of device. Endless possibilities. Performance Security

Functionality

Manageability

R

i

s

k

(21)

22

Backup and recovery

Taken care of.

Who will support me?

Performance

Security

Functionality

Manageability

R

i

s

k

(22)

23

Updates, patches, fixes, …

Planned and controlled Do I have a choice? Performance Security

Functionality

Manageability

R

i

s

k

(23)

24

Where is my data?

Is it OK?

In house. Somewhere… Performance Security

Functionality

Manageability

R

i

s

k

(24)

25

Performance

Security

Functionality

Manageability

R

i

s

k

(25)

26

Testing?

Check Intake Trial Interview Proof of concept

(26)

27

Testing!

Check Intake Trial Interview Proof of concept Testen Proef Intake Interview Proof of concept

(27)

28

Performance Testing

Security Testing

Manageability Testing

Availability & Continuity Testing

Functional Testing

Migration Testing

Testing caused by

Legislation & Regulations Testing in Production

Testing during Selection

T

e

s

t

M

e

a

s

u

r

e

s

Testen Proef Intake Interview Proof of concept

(28)

29

Performance

Security

Functionality

Manageability

R

i

s

k

s

Performance Testing Security Testing Manageability Testing

T

e

s

t

M

e

a

s

u

r

e

s

(29)

30

Performance

Security

Functionality

Manageability

R

i

s

k

s

T

e

s

t

M

e

a

s

u

r

e

s

(30)

31

Performance

Security

Functionality

Manageability

R

i

s

k

s

T

e

s

t

M

e

a

s

u

r

e

s

(31)

32

T

e

s

t

M

e

a

s

u

r

e

s

Performance Security

Functionality

Manageability

R

i

s

k

s

Architecture

From “individual” risks

to

(32)

33

Selection

Implementation

Production

T

e

s

t

M

e

a

s

u

r

e

s

(33)

34

Testing in Production

T

e

s

t

M

e

a

s

u

r

e

s

Functionality

Manageability

R

i

s

k

(34)

35

logo van Flair 17-12-2010 Polteq logo_RGB.png R G B 35 30 96 Blauw 232 62 38 Rood 108 174 68 Groen

Selection Criteria

T

e

s

t

M

e

a

s

u

r

e

s

(35)

36

Proof of Concept

T

e

s

t

M

e

a

s

u

r

e

s

(36)

37

Testing in Production Testing during Selection

T

e

s

t

M

e

a

s

u

r

e

s

Functionality

Manageability

R

i

s

k

(37)

38

Known measures

tuned and tweaked

New measures developed

T

e

s

t

M

e

a

s

u

r

e

s

(38)

39

Load Testing

YOUR Operational Profile YOUR Operational Profile PLUS ACTUAL MOMENT Performance Testing Security Testing Manageability Testing

T

e

s

t

M

e

a

s

u

r

e

s

(39)

40

Online – Offline

Use case testing. Global testing.

T

e

s

t

M

e

a

s

u

r

e

s

(40)

41

Any device – any platform

Multiplatform testing. Multiplatform testing. Performance Testing Security Testing Manageability Testing

T

e

s

t

M

e

a

s

u

r

e

s

(41)

42 Internet Explorer 6 Internet Explorer 7 Internet Explorer 8 Firefox 3.5 Firefox 3.6 Firefox 4 Safari 4 Safari 5 Chrome11 Opera11 Windows XP Windows Vista Windows 7 Windows 2003 server Windows 8 Windows CE Linux Unix Mac OS Lion Mac OS Snowleopard iOS Android Operating Systems Browsers Multiplatform Devices Computer Mobile Tablet PC Macintosh SUN NOKIA … Samsung … Windows Mobile iPhone .. Xxx … MOTOROLA… Blackberry… ASUS.. Xxx … 5-18

(42)

43

Any device – any platform

Multiplatform testing. Multiplatform testing. Performance Testing Security Testing Manageability Testing

T

e

s

t

M

e

a

s

u

r

e

s

(43)

44

Legislation + Regulations

=

Test basis

Incidental testing. Compliancy testing. Performance Testing Security Testing Manageability Testing

T

e

s

t

M

e

a

s

u

r

e

s

(44)

45

European Commissioner Kroes, Sept 27

• Strategy document to promote cloud computing

• 2,5 million jobs, 160 billion Euros

• Major barriers:

–

Many different standards

–

Contract issues

–

Many different rules and policies

(45)

46

Legislation + Regulations

=

Test basis

Incidental testing. Compliancy testing. Performance Testing Security Testing Manageability Testing

T

e

s

t

M

e

a

s

u

r

e

s

(46)

47

T

e

s

t

M

e

a

s

u

r

e

s

Functionality

Manageability

R

i

s

k

(47)

48

Continuous

End-to-End Test

Functionals

and

non-functionals

T

e

s

t

M

e

a

s

u

r

e

s

(48)

Continuity Privacy Multi platform Legislation Cyber crime Impact organisation Standards Check Intake Trial Interview Proof of concept

(49)

Continuity Privacy Multi platform Legislation Cyber crime Impact organisation Standards Check Intake Trial Interview Proof of concept