CONTINUOUS INTEGRATION TESTING

(1)

WELCOME TO

CONTINUOUS

INTEGRATION TESTING

Mikko Palkama

IBM Software Nordic

(2)

THE CHALLENGE

Traditional approach:

Integrate everything,

then try it out

Challenges:

Complexity lies

beneath the

surface

Challenges:

Difficult to troubleshoot

Everybody waits for fixes

Test environment setup

(3)

Requirements

Unit Tests

Sys Tests

Big Bang

TEST INTEGRATIONS EARLY

Performance

UAT

(4)

Requirements

Unit Tests

Integration Tests

Sys Tests

TEST INTEGRATIONS EARLY

Performance

UAT

(5)

Requirements

Unit Tests

Integration Tests

Sys Tests

TEST INTEGRATIONS EARLY

Performance

UAT

(6)

Requirements

Unit Tests

Integration Tests

Sys Tests

• Accelerates testing

• Reduces costs

• Lowers risk

TEST INTEGRATIONS EARLY

Performance

UAT

OT

Shifting testing to the left!

(7)

AGILITY STOPS

(8)

(9)

WHAT IS NEEDED?

Ability to

Test at interface level (test automation)

Test components in isolation

(simulation of other components or systems)

Without

Big effort

Deep programming skills

(10)

NEED BETTER TOOLING

That make it easy and fast to

1. Create automatic integration tests

2. Create stubs that simulate (sub) systems

3. Manage the use of stubs (virtual services)

Initiator

Recipient

3. Manage the use of stubs (virtual services)

in various test environments

4. Maintain and reuse the integration test

inventory

(11)

EXAMPLE USE CASES

3rd party

ESB

Database

Business logic

UI logic

Mainframe

Integration points

ESB

3rd party

(12)

3rd party

ESB

EXAMPLE USE CASES

TEST BUSINESS LOGIC ALONE

Database

Business logic

UI logic

Mainframe

ESB

3rd party

Automatic test

Virtual service

(13)

3rd party

ESB

EXAMPLE USE CASES

TEST USER INTERFACE ALONE

Database

Business logic

UI logic

Mainframe

ESB

3rd party

Automatic test

Virtual service

(14)

3rd party

ESB

EXAMPLE USE CASES

TEST USER INTERFACE ALONE

Database

Business logic

UI logic

Mainframe

ESB

3rd party

Automatic test

Virtual service

(15)

3rd party

ESB

EXAMPLE USE CASES

SIMULATE BACKENDS

Database

Business logic

UI logic

Mainframe

ESB

3rd party

Automatic test

Virtual service

(16)

TOOLSET / PRODUCTS

Rational Integration Tester

Scriptless, wizard-driven test authoring

environment with support for more than

70 technologies and protocols

Capture and model virtual services

Create functional integration tests

DEV

QA

Create functional integration tests

Reuse your functional tests for

performance testing

Rational Test Virtualization Server

Central environment for managing virtual

services and test environments (Web UI)

Make virtual services available to others

Test

Workbench

Functional Testing Integration Testing Performance Testing Regression Testing

Test

Servers

Load Agents Virtualization Agents

QA

DEV

(17)

SUPPORTED TECHNOLOGIES

• Email (SMTP, IMAP) • Files • FTP/S • HTTP/S • TCP/IP • WebSphere MQ • ActiveMQ • Sonic MQ • CentraSite • Oracle Fusion • SCA Domain

• Software AG IS, Broker, BPMS • WebSphere Application Server • WebSphere Message Broker • WebSphere MQ • Sonic ESB • .Net Objects • Bytes • COBOL Copybook • ebXML • ASC X12 • UN/EDIFACT • DFDL • Fixed Width

Messaging Protocols

SOA, ESB, Others

Message Formats

• Sonic MQ • JMS

• SAP IDoc, BAPI, RFC • Software AG’s IB & IS • TIBCO Rendezvous, Smart

Sockets & EMS • CICS TG

• IMS Connect • Java, RMI

• Sonic ESB

• System z (CICS and IMS) • TIBCO ActiveMatrix • UDDI • Web Services, WSDL • WebSphere RR • BPM • Databases • Log Files • Fixed Width • Java Objects • MIME • SOAP

• Software AG Broker Docs • SWIFT • CHIPS • FedWire • TIBCO ActiveEnterprise • XML (DTD, XSD, WSDL) • Custom

(18)

Test Virtualization Server

Workbench

EXAMPLE TOPOLOGY

Test Control

Panel

Integration

Tester

Intercept control and publication of virtual services

Agent

Stubs

Agent

SUT

Intercept /

Proxy

Client

Service

Normal traffic flow from client application to SUT via intercept technology

(19)

PERFORMANCE TESTING

Configure & Control

Execute 1700 New Virtualize

Orders Type A per second 5 Cancelled Orders per second 50 Order Edits per second 1 Heartbeat per second 750 New Orders Type B per second

(20)

PERFORMANCE TESTING

Configure & Control

Execute 1700 New Virtualize

Orders Type A per second 5 Cancelled Orders per second 50 Order Edits per second Analyse 1 Heartbeat per second 750 New Orders Type B per second

(21)

Booking Request (SOAP/HTTP)

Travel web UI

_{Hotel finder}

web services

(22)

Reduced

Cycle Time

More effective parallel development by virtualizing

different parts of the system

Test environments can be configured faster

Focus on testing, not on configuring environments

Lower

Developers have the means to test software earlier

at the Service/API level

BENEFITS

Lower

Test Lab costs

Test lab infrastructure costs

Labor involved in setting up test environments

3rd party systems for non-production use,

fee-based services

Lower

Risk

at the Service/API level

More testing can be done independently from the

User Interface, during development

(23)

Actual Service/App

Virtual Service/App

Service Virtualization is an enabler for

continuous Integration Testing

Services, applications, systems are introduced

into the continuous integration cycle in a

prioritized, controlled fashion.

RE-CAP

CONTINUOUS INTEGRATION TESTING

(24)

PART OF YOUR

CONTINUOUS DELIVERY PIPELINE

Enhance in your continuous integration / delivery pipeline:

Run automated integration tests

Automatically provision virtual services

Automatically set up and tear down test data sources

Combine with other tools: build engines, deployment

automation, cloud provisioning, test data management

Integrate with

Test management

Defect tracking

(25)

Major telecom carrier

Multiple test releases required additional test resources, increasing testing costs

Solution: Virtualized services and created

reusable test cases across multiple environments Results: Reduced testing time by 50%,

saving $800K over 3 years

Global bank

Reduces critical development path for Mortgage platform program

Solution: Simulation of front end to conduct sub-GUI database and log checks. Message based testing of Mortgages mainframe

Results: Critical path reduced by 6 weeks, estimated savings of $2.2M

SUCCESS STORIES

Global bank

Needed to improve time to market for fixed income business

Solution: Implemented agile testing process supported by continuous, automated integration testing for payments hub

Results: Project lead times halved and costs down one third; Flexibility to introduce late

requirement changes

US Healthcare Organization

Moves to continuous testing model with 100s of 3rd party interfaces

Solution: Service virtualization and early integration testing. Create stubs to simulate 3rd party interfaces. Results: Reduced environment costs for 3rd party interfaces; enabled switch to continuous delivery; reduced start-up and maintenance costs for new projects through sharing of SV and test assets

(26)

FIND OUT MORE…

Video: What is continuous integration testing?

http://www.youtube.com/watch?v=wGOogTsOtc4

Video: What is service virtualization?

http://www.youtube.com/watch?v=Np5_O43BFD4

Video: When to use service virtualization?

http://www.youtube.com/watch?v=j1f5vP3gCIM

White paper: Lessons learned from testing service-oriented architectures

https://www.ibm.com/services/forms/signup.do?source=swg-rtl-sd-wp&S_PKG=500029668

White paper. A rational approach integration testing

https://www.ibm.com/services/forms/signup.do?source=swg-rtl-sd-wp&S_PKG=500029634

Product Information:

IBM Rational Test Workbench:

http://www-01.ibm.com/software/rational/products/rtw/

IBM Rational Test Virtualization Server:

http://www-01.ibm.com/software/rational/products/rtvs/

IBM Rational Performance Test Server:

(27)

© Copyright IBM Corporation 2014. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied.

IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.