Model Based Test Automation

Accelerate. Innovate. Automate.

Model Based Test

Automation

Avoiding the maintenance trap

Hans Planken

Vice President Alliances EMEA

January 2015

Automation level below 25% even

after 30 years of software testing.

Why?

Do we need a different approach?

Evolution of Complexity

- Testing Challenges

Mainframe

Client/Server

Web

Mobile

Complexity

What’s the problem? A typical test project.

Total Cost and Effort

100%

50%

Test Case

Specification

Manual Testing

(Execution)

Automated Testing

(Setup, Maintenance

and Execution)

If you don’t know how much

test coverage you have,

you’ll end up with far too

many test cases!

Not effective enough

Automation levels are low

because of it’s difficult to

automate!

Not efficient enough

Risk Coverage

Tricentis Tosca Testsuite

Virtualize

– Orchestrated Service Virtualization

Technology Integration

Optimize

Risk Coverage

Optimizer

Manage

Test Data

Provisioning

Automate

Model-Based Test

Automation

Target Applications

Virtualize

– Service Virtualization & Orchestration

Manage

Test Data

Provisioning

Automate

Model-Based Test

Automation

Reporting & Dashboard

Tricentis Tosca Testsuite

Aligns

business priorities

to test planning & execution

Generates the

fewest test cases to maximize risk coverage

Shifts focus from

Test Coverage to Risk Coverage

Optimize

Risk Coverage

Optimizer

Source

: Customer Projects

TRICENTIS

®

Total effort/costs

100%

0%

50%

Test Case

Specification

25%

Manual

Test

Execution

75%

Automation

Finding

and

Preparing

Test Data

Execution

of

Test Cases

50 - 75%

of manual

execution costs!

insufficient control of

test data

Virtualize

– Service Virtualization & Orchestration

Automate

Model-Based Test

Automation

Reporting & Dashboard

Optimize

Risk Coverage

Optimizer

Manage

Test Data

Provisioning

Tricentis Tosca Testsuite

Provides tailored

test data basis

for test execution

Reduces effort in

manual test execution

by 40 – 50 %

Virtualize

– Service Virtualization & Orchestration

Reporting & Dashboard

Manage

Test Data

Provisioning

Optimize

Risk Coverage

Optimizer

Automate

Model-Based Test

Automation

Tricentis Tosca Testsuite

Enables

extreme automation

(rates > 90%)

Covers

UI and nonUI test automation

Tosca Testsuite – Robust, Cost-Effective, Scalable

Risk Management

Engine

: Higher Coverage,

Fewer Tests.

Model-Based Test

Automation:

Robust

Cost-Effective

Scalable

What makes it

different?

Model-Based Automation

Model-Based Test Automation

What keeps us from higher

automation levels…

• Prohibitive Maintenance

of Test Scripts

• Missing or inadequate

Object Recognition

• GUI and nonGUI tests require

different teams/skills…

• Autom. Tests require dynamic

and thorough Test Data

Efficiency: Why

are automation

levels so low?

The Evolution of Test Automation

Record/

Replay

FRAGILE

1

st

_Generation

Robust

Cost

Effective

Scalable

The Maintenance Problem with Capture/Replay

Example:

“…select invoice with the car registration number VIC-3456”

CheckBox Click

,

“/usr/cntlCONTAINER/

shellcont/

shell

[2]

/chbx

[1,3]

”

Does your business team understand this command?

Is this command stable?

Conclusion: Capture/Replay does not work!

Checkbox Name

Reg Number Invoice

John Smith

NSW-1234

90035560

Jim Courier

NSW-2345

90035561

John Cook

VIC-3456

90035562

Tim Rebhorn

VIC-4567

90035563

Dale Howard

WA-5678

90035564

The Evolution of Test Automation

Record/

Replay

FRAGILE

Test

Automation

Frameworks

EXPENSIVE

Model-Based

1

st

_Generation

₂

nd

_Generation

_{Next Generation}

Script-Based

Model-Based

Robust

Cost

Effective

Scalable

•

Cognitive Test Cases:

business language - not

scripts!

•

Requires only business

SME skills

•

Highly

stable

from a

technical as well as from a

business point of view

•

Independent

from

technologies

, HTML, Java,

.NET, SAP, Manual

CheckBox Click

,

“/usr/cntlCONTAINER/she

llcont/

shell

[2]

/chbx

[1,3]

”

Example:

Do you understand the

business

instruction?

Do you remember it?

Model Based Test Automation

How does it work?

Model-Based Automation

UI

Technology

System Under Test

Test Cases

de

coupling

Action

Action

non

UI

Business

W/S

OneView: UI/nonUI; Robustness over Change

Model

Systems

HTML

SQL

JAVA

.NET

_SWIFT

_XML

HOST

ST

SIT (E2E)

UAT

CT

SCAN &

SYNCHRONIZE

Test Cases

in plain English

CHANGE

SYNCHRONIZE

Consistent User Experience & On-Demand Staffing

TOSCA User Interface

Base Engine

decouples

DB

W

eb

Ser

v

ices

SW

IF

T

D

iesel

_Gas

…

H

P

Petr

o

l

T

H

T

M

L

Test Logic

Technology

Broad coverage or technologies & applications

TOSCA allows for easy tests across

any technology

and/or platform!

A

d

a

p

te

r

A

d

a

p

te

r

A

d

a

p

te

r

A

d

a

p

te

r

…

> 30 technology specific adapters

…

Model Based Test Automation

Customer-specific

extensions

T

e

c

h

n

o

lo

g

ie

s

SAP

3

2

7

0

,

E

mu

la

to

rs

x

B

ro

w

s

e

r

J

a

v

a

Cen

tu

ra

/Gu

p

ta

W

in

3

2

,

C++,

V

B

P

o

w

e

rBu

ild

e

r,

Del

p

h

i

P

DF

-Doc

u

me

n

ts

S

ie

b

e

l,

Ora

c

le

Wo

rd

,

X

L

,

Out

lo

o

k

.NE

T

,

W

P

F

Dat

a

b

a

s

e

s

…

A

v

a

lo

q

We

b

S

e

rv

ic

e

s

,

X

ML

Mo

b

ile

RE

S

T

J

S

ON

Traditional Automation

Traditional

automation from a

manual

to an

automated

Test Case…

Manual (Prose)

Only Tosca offers seamless test automation

Model

Finding

and

Preparing

Test Data

Execution

of

Test Cases

Total costs

100%

50%

Test Case

Specification

25%

Pre Tosca

Cost

Manual

Test

Execution

75%

Automation

Test Case

Spec.

Test

Automation

Manual

Testing

Test Case

Design

Risk-Based

Test Structure

Post

Tosca

Cost

The Bottom Line

Poor automation coverage due to

(i) maintenance trap