© Copyright by TRICENTIS®, all rights reserved 1
WWW.GOTOSCA.COM
Shift Left: Test Automation in Agile Environments
Unlock the Secrets
Shift Left - Test Automation Agile Environments
• Today we will Cover
• Challenges around
Agile
Testing
• Shift-Left – Specification by Example
• The
Agile
test-pyramide
• What are the benefits of Shift-Left & early
automation?
• Agile
customers - case studies
• How do developers need to contribute?
• Summary
© Copyright by TRICENTIS®, all rights reserved 3
Agile Methods are on the Rise
Agile
…here is where
Waterfall
they are today
…here they want to get to …
More than 2/3 of new software developments are
now being set up with an agile method.
© Copyright by TRICENTIS®, all rights reserved 5
Costs and experienced value | conventional (II)
Project aborted due to shortage of resources or the project scope
needs to be changed significantly
Project aborted!!
Costs and Experienced Value of Agile (Scrum)
10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 5% 15% 30% 55% 75% 85% 90% 95% 98% 100%0%
20%
40%
60%
80%
100%
120%
1
2
3
4
5
6
7
8
9
10
Kosten
Wert
Value
Costs
© Copyright by TRICENTIS®, all rights reserved 7
Shift
to the left
Agile
Development
Shift from the
should
to the
must
Specification by Example
© Copyright by TRICENTIS®, all rights reserved 9
TOSCA
– Agile Testing | BDD
Shift to the Left!
© Copyright by TRICENTIS®, all rights reserved 11
V-Model
Specification
Techn. design
Requirement
definition
Unit/Module testing
Acceptance testing
(UAT)
(System)
integration testing
© Copyright by TRICENTIS®, all rights reserved 13
From the waterfall to the agile method…
The size of the birds changes!
Birds remain birds!
Many small birds are considerably more maneuverable
(more flexible) than large ones.
Specification by Example - Key to Success
Specification
Techn. design
Requirement
definition
Unit/Module testing
Acceptance testing
(UAT)
(System)
integration testing
Specification
by Example
© Copyright by TRICENTIS®, all rights reserved 15
Specification by Example with Test Case Design
Test Case Design is …
sufficiently substantiated
flexible if changes occur
lean and very cost-effective (compared with the specification of test
cases)
checked
Requirements
Test Case Design
substantiated
Shift to the Left!
© Copyright by TRICENTIS®, all rights reserved 17
Test Pyramid – conventional testing procedure
Practice: main focus of efforts is on the E2E test!
Poor cost efficiency
Limited test coverage
Availability of all systems in the test?
Consistent test data in all systems?
© Copyright by TRICENTIS®, all rights reserved 19
Test Pyramid – Target State
Powerful, automated Unit und
Acceptance Tests (system tests)
Test access: non-UI
Tests via GUI only for:
Testing the display on the
screen
Testing the connection to the
underlying layers
E2E – System internal und Crossover Tests
…
System
1
…
…
System
n
System-internal Tests
E2E Tests
© Copyright by TRICENTIS®, all rights reserved 21
Testing up to the Interfaces
…
System
1
…
…
System
n
System-internal Tests
E2E Tests
System-internal tests extended right to the interfaces
Coupled systems are „stubbed“ or „mocked“
Share of functionality that can be tested without coupled systems (=
system-internally) is sufficiently extended
Testsequence UI and nonUI
- Example
Example: Create Customer, Customer
Project TRICENTIS, via WebService
Dialog/Data
SRec/Data
2
3
# of test steps: Save 75%!
Maintenance: Save 75%!
© Copyright by TRICENTIS®, all rights reserved 23
Regressiontest
Releasetest
Test streams & Test automation
| traditional
Functionality to be tested
R
2
t
…
… One-time (manually)
… Test of new functionality with regressive error potential (manually)
… Regressiontest (automated)
Releases
R
…
Regressiontest – automated:
High automation degree from
the start
Dominated by nonUI access
Test streams and Automation | agile
Functionality to be tested
nonUI
UI
© Copyright by TRICENTIS®, all rights reserved 25
Testing Challenges:
o
Move towards Agile software development
o
Cost and scheduling pressure
o
All previous attempts at automation have failed
Solution:
Model-based test automation.
Results:
o
Core regression suite implemented in less than 3 months
1,009
test cases
33,797
test actions
35
end-to-end business process scenarios, multiple day scenarios
2,511
verification points
o
Execution effort down by 80% (from 535 h to less than 100 h).
o
Annual cost reduction exceeds 50%.
GBST Shares
Settles close to 50% of all trades on the
Australian Stock Market, ~ AU$ 2.5
Testing Challenges:
o
Agile development with continuous integration
o
Very short development cycles
o
Multiple interfaces (Browser, Smartphone, rich client)
Solution:
Development provides test-accessible nonUI interface (WebServices, REST) for every single
use-case!
… enabling model-based test automation, early nonUI test automation
German Payback System vendor,
hosting payback systems for Lufthansa,
Deutsche Bahn, etc.
© Copyright by TRICENTIS®, all rights reserved 27
How Developers need to support …
Agile testing requires …
early
automated
nonUI
… testing.
How Developers need to support …
Agile developers need to provide …
… appropriate hooks:
Comprehensive, public interface for each
use-case …
… allowing to fully test business logic …
… avoiding „noisy“ communication
© Copyright by TRICENTIS®, all rights reserved 29