Test Suite Generation for Services

Test
Suite


Generation


for
Services


Kathrin
Kaschner
and
Niels
Lohmann


WESOA
2008
▪_Sydney▪_{1
December
2008}

Testing
Services:
What's
new?


classical
software
 service


location
 my
machine
 some
machine(s)


usage
 input
➙
output
 stateful,
asynchronous


communication
over
 the
net


description
 manual
/
API
/
GUI
 public
view


test
case
 function
call
/


Service
Development


1.  define
the
public
view
   publicly
observable
behavior
   interface
   exchanged
messages
 2.  implement
all
details
 3.  publish
public
view
   defines
all
correct
interacting
partners


•  the
implementation
should
conform
to
its
specification:


➙
Any
correct
partner
derived
from
the
public
view
should


also
be
a
correct
partner
of
the
implementation.


specification


Specification Implementation receive order receive login send terms of payment receive confirmation send invoice receive order update user profile send offer lookup user in database premium user standard user send invoice upgrade to premium user order ! 100 " order < 100 " pre-notify shipment customer receive order send terms of payment send invoice receive confirmation receive order update user profile premium user standard user send invoice upgrade to premium user order ! 100 " order < 100 " pre-notify shipment store order store order customer incorrect

Verification
vs.
Testing


•  Verification/White
Box
Testing
 •  [AalstLMSW2008]:
algorithm
and
tool
 specification
 abstract
BPEL,
 BPMN,
UML‐AD
 implementation
 executable
BPEL,
 Java,
.NET
 formal
model
 formal
model
 conform
 verify
 translate
 translate


Verification
vs.
Testing


•  Black
Box
Testing:
Code
too
complex
or
unknown
 formal
model
 specification
 abstract
BPEL,
 BPMN,
UML‐AD
 implementation
 executable
BPEL,
 Java,
.NET


✘

comply
 verify
 translate
 translate
 implementation
 executable
BPEL,
 Java,
.NET
 formal
model


Test
Suite
Generation


generate
 specification
 abstract
BPEL,
 BPMN,
UML‐AD
 conform
 test
 case
 test
suite
 implementation
 executable
BPEL,
 Java,
.NET
 test
 generate
 specification
 abstract
BPEL,
 BPMN,
UML‐AD
 Problem:
test
suite
must


contain
a
significant
number

Test
Suite
Generation


•  Proposed
approach
 specification
 abstract
BPEL,
 BPMN,
UML‐AD
 implementation
 executable
BPEL,
 Java,
.NET
 formal
model
 partner
service
 characterization
 conform
 translate
 compute
 test
 case
 test
suite
 test
 generate


Finitely
Characterizing
all
Partner
Services


•  edges:
   ?
=
receive
   !
=
send
 •  states:
Boolean 
annotations
 !login ! !order !login !order

?offer ! (?terms " ?invoice) !login ?terms " ?invoice ?offer !confirm ?terms ?invoice !confirm final ?invoice final ?invoice !confirm ! ?offer ?terms ?offer ?invoice ! ?offer !confirm ?offer ?offer ?invoice ?offer ?offer ?invoice ?offer ?offer ! !order !login !order ?offer !order !order Partner
if
and
only
if:
 1.  subgraph
 2.  annotations
fulfilled
 •  "operating
guidelines"


send login receive terms of payment send confirmation receive offer send order

online shop receive invoice receive invoice Operating
Guidelines Test
case
➙ !login ! !order !login !order

?offer ! (?terms " ?invoice) !login ?terms " ?invoice ?offer !confirm ?terms ?invoice !confirm final ?invoice final ?invoice !confirm ! ?offer ?terms ?offer ?invoice ! ?offer !confirm ?offer ?offer ?invoice ?offer ?offer ?invoice ?offer ?offer ! !order !login !order ?offer !order !order

11
 send login receive terms of payment send confirmation receive offer send order

online shop receive invoice receive invoice receive order send terms of payment send invoice receive confirmation receive order update user profile premium user standard user send invoice upgrade to premium user order ! 100 " order < 100 " pre-notify shipment store order store order customer receive order send invoice send terms of payment receive confirmation receive order update user profile premium user standard user send invoice upgrade to premium user order ! 100 " order < 100 " pre-notify shipment store order store order customer

✘

✔

Reducing
the
Test
Suite


•  operating
guideline
characterizes
all
test
cases


•  some
test
cases
are
redundant
 •  criterion:
annotations
 •  reduced
test
suite
is
still
complete
 !login ! !order ... !order ... !login ... !order ... !login ... !order ... !login

redundant


!login ! !order ?terms " ?invoice !confirm ?terms ?invoice !confirm final ?invoice final ?invoice ?offer ! !order !login !order ?offer !order !login ! !order

?offer ! (?terms " ?invoice) !confirm ! ?offer ?terms ?invoice ! ?offer !confirm ?invoice ?offer ! !order !login !order !login ! !order

?offer ! (?terms " ?invoice)

final !confirm ! ?offer ?terms ?invoice ! ?offer !confirm ?offer ?offer ?invoice ?offer ?offer ! !order !login !order !login ! !order

?offer ! (?terms " ?invoice)

!confirm !confirm final !confirm ! ?offer ?terms ?offer ?offer ?invoice ?offer ?offer ! !order !login !order !login ! !order

?offer ! (?terms " ?invoice) ?terms " ?invoice ?offer !confirm ?terms ?invoice !confirm final ?invoice final ?invoice ?offer ! !order !login !order !login ! !order !login !order

?offer ! (?terms " ?invoice)

!login final final !confirm ! ?offer ?terms ?invoice ! ?offer !confirm ?offer ?invoice ?offer ?offer ?invoice ?offer !login ! !order !login !order

?offer ! (?terms " ?invoice)

!login ?invoice final ?invoice final !confirm ! ?offer ?terms ?invoice ! ?offer !confirm ?offer ?offer ?invoice ?offer !login ! !order !login !order

?offer ! (?terms " ?invoice)

!login !confirm ?invoice !confirm final ?invoice final !confirm ! ?offer ?terms ?offer ?offer ?invoice ?offer !login ! !order !login !order

?offer ! (?terms " ?invoice)

!login ?terms " ?invoice ?offer !confirm ?terms ?invoice !confirm final ?invoice final ?invoice 9
of
1151
test
cases
are
sufficient

Tool
Support


specification
 abstract
BPEL
 Petri
Net
 Operating
 Guidelines
 conform
 BPEL2oWFN
 Fiona
 test
 case
 reduced
test
suite
 Fiona
 implementation
 executable
BPEL,
 Java,
.NET
 test
 oWFN2BPEL
 test
 case
 BPEL
test
suite
 •  BPEL2oWFN:
translates
BPEL
process
to
formal
model
 •  Fiona:
calculates
OG;
generates
and
reduces
test
suite


Take
home
points


15
 •  Systematic
test
suite
generation
 •  Translation
of
test
cases
into
abstract
BPEL
processes
 •  Manual
addition
of
further
implementation
details
 •  Applications:
compliance,
substitutability,
contracts,…


•  Possible
reduction
of
test
suite
(99%
of
test
cases)


•  Future
Work:


  finish
Implementation
of
test
suite
generation


  Negative
test
cases


References


•  Wil
van
der
Aalst
and
Niels
Lohmann
and
Peter


Massuthe
and
Christian
Stahl
and
Karsten
Wolf.


Multiparty
Contracts:
Agreeing
and
Implementing


Interorganizational
Processes.
Comput.
J.
(Accepted


for
publication
in
November
2008)


•  Niels
Lohmann,
Peter
Massuthe,
and
Karsten
Wolf.


Operating
Guidelines
for
Finite‐State
Services.
In
Proc.


ICATPN
2007,
volume
4546
of
LNCS,
pages
321–341, 
2007.