"Decoupled System Interface Testing at FedEx"

15 

Loading....

Loading....

Loading....

Loading....

Loading....

Full text

(1)

Concurrent Class

10/3/2013 3:00:00 PM

"Decoupled System Interface

Testing at FedEx"

Presented by:

Chris Reites

FedEx

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

(2)

As a Technical Principal in software quality assurance for FedEx Services, Chris Reites has

experience providing cutting edge, best-practice processes for the development and testing of

large-scale, complex, global software systems. Chris has been working for FedEx in IT for

fifteen years. Prior to joining the software testing organization within FedEx, Chris was a

software developer for several key applications within the FedEx billing system.

(3)

Decoupled Testing

Overview

Chris Reites

Technical Principal – FedEx Services

STARWEST Conference

10/03/2013

FedEx Testing – What We’re Dealing With

Application Testing and Certification

Responsible for Test Planning,

Design, Execution, and Validation

Key Shipping Products (Desktop

devices,

www.fedex.com

, etc…)

Backend Rating, Revenue, Tracking,

and Invoicing Systems

Major releases annually, as well as

weekly exception and emergency

loads

(4)

Types of Testing We Do

3

Definition: Validates core software functionality in production after regularly scheduled software loads and regularly scheduled data updates. Supports Corporate Loads, Dotcom Loads, Exceptions Loads & some Emergency loads.

Functional Testing

Performance Testing

Vulnerability Testing

Regional Testing

Definition: Evaluates the compliance of a system or component with specified functional requirements. Includes New Features, Regression, Integration and System tests. Coordinate with Revenue Testing for impacted products.

Definition: Evaluation of a system’s or component’s compliance with specified performance requirements. Includes Volume testing, Load / Stress testing, Failover / Recovery Testing and Disaster Recovery.

Definition: Utilizing security scanning tools and educated vulnerability testing through manual human intervention techniques, providing scanning and penetration testing for supported applications during regular releases.

Definition: Includes functional testing performed on behalf of the Regions by SQA Testing groups and Language Translation Testing performed by Marketing or User groups within the Regions.

Certification Services

Quality Assurance Approaches

Production Checkout

CSP Certification

Label Certification

(ensures operational excellence)

Definition: Functional tests designed to validate specific outcomes within Revenue systems. Starts with entering shipments (via INET, WSVC, CAFE, FXRS, etc.) and addition of selected scans, flows through servers and intermediate systems, and concludes with validation of results on invoices and in accounts receivables.

Definition: Consulting with third-party software providers to integrate FedEx technology into their applications and validate that their applications meet FedEx requirements from a brand, revenue and operational perspective.

Definition: Validation of Express labels submitted by automation clients, CSP providers, WebServices customers and FXRS customers who modify labels in production. Automation clients submit all labels for certification.

End-to-End Testing

Dependency to have all code ready at the same

time

Interfaces were critical…yet not well documented

or understood

Interface changes coming late into Code/Test

phases

Interface issues caused “Ripple Effect” throughout

system

Impacts our Speed To Market and causes a lack of

flexibility in testing

4

Potential Hazards in Large

System Testing

(5)

Decoupled Testing Concepts and Potential

5

Automation Revenue

Corp. Load End to End Testing

Corp. Load End to End Testing enhanced by Decoupled Test

Concept

Provides the ability to test target systems

independently by removing dependencies

on other external systems.

Divide and conquer

Reduce defect fix/validation cycle time

Mitigate Risk when introducing software

changes – comprehensive regression test

Reduce validation dependency on

End-to-End cycles

Adoption in Automation Systems

Reduced dependency on revenue test

cycles

Mini Revenue cycles within automation

systems using Decoupled Test Tool

Pilot mini Decoupled Testing cycle in one

shipping device – FY12 Q1

Mini Decoupled Testing cycles in multiple

shipping devices for Jan12 corporate

load.

Automation Revenue Devices Front End Process Back End Process Edit & Rating Invoicing Settlement Devices Front End Process Back End Process Edit & Rating Invoicing Settlement

What Makes Up Decoupled Testing?

Decoupled Testing provides the ability to test target systems

independently by removing dependencies on other external

systems.

4 Core Functions:

Interface Data Capture

:

supports the collection and storage of interface

data

Interface Data Compare:

supports on-demand, field-level comparisons of

interface data

Interface Data Injection

:

supports the ability to ‘replay’ previously

captured interface data into the target system

Interface Simulation

:

supports the virtualization of responses from

(6)

eCommerce

Express Rev Rating

Ground Rev Rating

Accounting Corp App

Revenue Back End Regions Apps Invoice In p u ts O u tp u ts GL Entries A/R Order Cash Shipment Sys Testing Researcher Input Output Expected FCIS Sys

Freight Rev Rating

End to End

Corporate Load Testing

Coupled Testing

Jump

7

eCommerce

Express Rev Rating

Ground Rev Rating

Accounting Corp App

Revenue Back End Regions Apps Invoice In p u ts O u tp u ts GL Entries A/R Order Cash Shipment Sys Testing Researcher Input Output Expected FCIS Sys

Freight Rev Rating

Coupled Testing

Jump

(7)

Express Rev Rating Accounting Corp App Revenue Back End Invoice In p u ts O u tp u ts GL Entries A/R Order Cash Shipment Sys Testing Researcher Input Output Expected FCIS Sys Freight Rev Rating

End to End Testing

Cluster Cluster Cluster Cluster Cluster Cluster Cluster eCommerce Back End eCommerce Front End Cluster Cluster

Decoupled Testing

evolve

evolve

evolve

evolve

9

eCommerce

Front End

Express

Rev Rating

Revenue

Back End

In

pu

ts

Actual Before

Actual After

O

up

ut

s

Testing Researcher A A B B C S

Expected

Decoupled Testing

10

(8)

11

Revenue Systems Example:

From Serial to Parallel Processing

12

New test data created for both system entry

points and downstream injection points

simultaneously

Regression test data injected with previously

captured results

•Compare outputs against previous results

Automation

•Inject data directly into backend system •Compare outputs against previous

results

Revenue

Common Test Data Design

Positive Effects:

Test Data

Shipping Input

Revenue Input

Reduce idle time (waiting for successful

end-to-end execution)

Increase the test coverage utilizing data

comparison analysis

Early detection of issues

(9)

Interface Simulation

13

Interface Simulation is

the ability to virtualize

responses from backend

systems.

Simulated Interfaces

remove backend

complexity from testing

environments and provide

stable, predictable

behavior to make system

testing easier and more

available.

Response times of virtual

interfaces can be varied to

simulate latency or

systems under load

Test Strategy Objectives

• From weeks to days or hours

Reduce Defect Resolution Time

• By providing data analysis solutions

• By automating validation processes

Reduce Validation Time

• By discovering and resolving defects earlier

• By more quickly fixing and revalidating defects

Improve Code Quality coming into Integration Testing

• By providing environments on demand

• By providing a self service set of test solutions

(10)

Common Goals And Success Metrics

Normal Dev/Test Phase DTT Dev/Test Phase

Agility and Time-to-Market

The measure of time required to launch a

project/feature through newly enabled testing

processes.

Defect Removal Cost Reduction

The measure of identifying and fixing defects

earlier in the testing process.

Support Resource Cost Reduction / Reallocation

The measure of resource reduction due to

functionality being delivered.

15

#

of

De

fe

ct

s

16

Defect Removal - Reduction in Cost

The measure of identifying defects earlier in the testing process.

Metrics above reveal the following:

A definite shift in time in the number of defects found and closed in earlier cycles

of testing

#

of

De

(11)

17

Decoupled Testing – Speed to Market

100.00%

67.00%

38.00%

0

0.2

0.4

0.6

0.8

1

1.2

Traditional

Decoupled Testing v1

Decoupled Testing v2

Dur

atio

n

of T

es

ting

R

elat

iv

e

To

T

raditio

n

al

Me

th

o

d

*

Testing Method

Testing Process for Certification of a Key Customer Shipping Platform

* Duration includes shipping/execution and validation

8.5%

100.0%

147.0%

100.0%

100.0%

147.0%

Traditional

Decoupled v1 Decoupled v2

#

Tes

t

Cas

es

Shipp

ed

an

d

V

al

id

at

ed

R

el

at

iv

e

to

Tr

ad

it

ion

al

Me

thod

Testing Method

Validated

Baseline

Decoupled Testing – Resource Cost Reduction

Testing Process for Certification of a Key Customer Shipping Platform

100%

1750%

3268%

Traditional Decoupled

v1

Decoupled

v2

%

Of

Tes

t

Cas

es

T

es

ted

P

er

Da

y

R

el

at

iv

e

to

Tr

ad

it

ion

al

Me

tho

d

Testing Method

For each test case: field level validation on

two transactions

Around 200 field comparisons per test case

100.00%

67.00%

38.00%

Traditional

Decoupled

Testing v1

Decoupled

Testing v2

Dur

ati

on

of

T

e

sti

n

g

R

e

la

ti

ve

To

Tr

ad

iti

on

al

M

eth

od

*

Testing Method

(12)

Case Study: FedEx Delivery Manager

19

FedEx Delivery Manager – Decoupled Testing

20

Business Challenge

Parallel development and

simultaneous delivery of

multiple FedEx

applications impacted by

the project prevented

integration testing prior

to Integration Testing.

Goals

Provide development

teams access to key

backend systems during

Unit Testing in order to

identify defects early,

and have the ability to

inject transactions into

various parts of the

system to break

dependency of needing

all systems ready at same

time.

(13)

Utilizing DTT, FedEx Delivery Manager early discovered 75% of defects and

validated 68% of test cases for the Back-End Systems

Delivery Manager Portal started having success

Utilizing injection for Back-End testing, FedEx identified:

11% of total defects during shakeout

73% of total revenue defects

82% of total credit card defects

63% of total Shipment Event Processing defects

21

Results

Why All Companies Should Be

Thinking About Decoupled Testing

As applications and systems become larger and

more complex, traditional “end to end” testing

becomes un-scalable.

With the increase in 3

rd

party service providers and

service oriented architectures, the ability to

decouple those dependencies for testing is critical.

The longer you wait to begin decoupling the testing

of your systems, the harder it is to do.

There is probably a LOT of low hanging fruit

available to start with.

(14)

Where to Begin

23

Involvement and buy-in of internal business

counterparts is critical

Identify testing dependencies that cause the most

issues (unavailability, late delivery, high associated

costs, etc…)

Find the quick wins to build confidence and

momentum

Partner with application developers and find ways to

share the decoupled testing tools

What’s Next?

User-driven, integrated data management

system

Minimal development on interface rollout

High-level reliability & instrumentation

Failure tolerant (re-connect)

Standard FedEx logging (monitor)

Unattended operation

Scalable without incurring major licensing cost

Large Capacity

Service Oriented Architecture for Integration

with other Systems

Rules-driven business logic

Synergistic Test Management System

(15)

25

Figure

Updating...

References

Updating...

Related subjects :