Unit Testing with zunit

®

IBM Software Group

© IBM Corporation

Rational Developer for System z –

Unit Testing with zUnit

IBM Trademarks and Copyrights

© Copyright IBM Corporation 2013, 2014. All rights reserved.

The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be

responsible for any damages arising out of the use of, or otherwise related to, these

materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM

software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates.

This information is based on current IBM product plans and strategy, which are subject to change by IBM without notice. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way.

IBM, the IBM logo, the on-demand business logo, Rational, the Rational logo, and other IBM Rational products and services are trademarks or registered trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other

3

Topics:

Agenda



Unit Testing Value Proposition,

zUnit Terms, Concepts and Vocabulary



Demonstration

What is Unit Testing?

Unit Testing is a software testing

method by which

individual

units of source code

, sets of

one or more computer program

modules together with

associated control data, usage

procedures, and operating

procedures are tested to

determine if they are fit for use

.

Wikipedia

An individual unit of software is a single test-able logic construct or routine within a call-able program:

• Date validation

• Credit Card number look-up • Tax computation

• Co-pay calculation

This method of testing is sometimes

5

Black Box Testing

Black Box Testing examines the functionality of an application

(what the software does)

without regard to its internal semantics

(

how

the program does what it does)

Black Box

Analyze Output

Analyze Input

Run File Update

Program

Verify Changes

to Data Files

Analyze

Data Files

QSAM

DB2 or IMS VSAM QSAM _{DB2 or IMS VSAM}

White Box (Unit) Testing and zUnit

White Box Testing is a method of testing software that tests the

internal

workings

of an application, as opposed to its functionality or its outputs.

During White Box testing,

code is run with preselected input values for

the validation of preselected output values.

White Box Tests

Address Validation Test

• Initialize Street Address

• Initialize City

• Initialize State

• Initialize Zip Code

• Call Program using…

• Evaluate test results

Date Validation Routine

Internal Table Lookup

Address Validation

State Tax Calculation

Test Harness

So ft w are Progra m So ft w are Progra m

Unit Test…

7

zUnit and the xUnit Standard



xUnit

is an industry-standard, program testing framework - that helps create and

automatically run repeatable, self-checking unit test cases.

 xUnit provides:

 Assertion-style test validation capabilities and result reporting.

 Automated execution validation - instead of independent/interactive testing activity

 At run-time, the xUnit framework distinguishes between failures and errors:  A failure is an anticipated problem (e.g., actual output does not match expected).

 An error is an unexpected, catastrophic problem (e.g., protection exception, null pointer)



jUnit

is an instance of the

xUnit architecture for Java



zUnit

is IBM’s implementation of the

xUnit architecture for System z



zUnit

provides the same

industry-standard output reports as jUnit

 You can use the same XML-reporting

dashboards and products that you

currently use with jUnit to review and analyze results

Why bother to Unit Test?

By testing individual logic routines in your programs:

• You can move through the lifecycle more quickly, because you have

precise feedback about separate logic routines

•

So you can better understand cause & effect

•

And you know

that

your code works and you know

how

your code works,

which gives you confidence to make enhancements and modifications

• Because you execute zUnit Tests through JCL:

• The testing can be

automated

• The end-to-end process takes less time than interactive debugging.

• And it can be more

systematic

• By isolating and verifying code fragments Unit Testing allows

you to understand

“cause & effect”

in your program logic

All of the above benefits allow you to catch errors earlier in

the lifecycle…

9

The value of early and extensive testing

During the Coding or Unit Testing phases $80/defect During the BUILD phase $240/defect During Quality Assurance

or the System Test

phases $960/defect Once released into production $7,600/defect +

Law suits, loss of customer trust, damage to brand

“

80% of development costs are spent identifying

and correcting defects

” **

**

National Institute of Standards & Technology Source: GBS Industry standard study Defect cost derived in assuming it takes 8 hours to find, fix and repair a defect when found in code and unit test. Defect FFR cost for other phases calculated by using the multiplier on a blended rate of $80/hr.

What do you use to Unit Test?

Unit Testing is typically done using a product or a

framework that provides:

1. Standardized

process, methodology and a component

framework (architecture)

2. Tools

for code generation or some method of simplifying the

development Unit Tests

3. Automation

- running scripts - or in the case of zUnit -

running JCL and not testing interactively

The industry leadership position in the Unit Testing space is

occupied by the

xUnit

component framework

• xUnit has been adopted by almost all of the major

computing languages/vendors/tools providers

• Including IBM

• Thus… zUnit

11

How do you Unit Test - with zUnit

Using zUnit you:

1. Develop Unit Tests for your programs

2. Execute the Unit Tests through JCL or interactively

3. Interpret/Analyze your Unit Test results

The Unit Tests you develop are COBOL programs or PL/I

procedures

•

RDz generates 95+ percent of the COBOL and PL/I Unit Testing code

• RDz provides the test harness (

Test Runner

) as an installed run-time

• And RDz provides both an interactive and batch test execution workflow

The zUnit development process is:

•

Wizard-driven

•

Evolving

- as customers adopt, and feedback next-generation

zUnit Support - as of RDz v9.1

• The call-able

(zUnit targeted)

program is a

z/OS Load Module

that

can be written in any z/OS language

- including but not limited to:

 COBOL

 PL/I

 4GL z/OS applications, generated to; COBOL or PL/I; CA-Easytrieve, SAS, IBM Report Writer, CA-Telon, NETRON/CAP™, CA-Meta COBOL, etc.

 Assembler

(see slide notes)

(what this means…)

You zUnit test at the Load Module level

:



The underlying language used to create the targeted business logic is irrelevant.



Note that this is differs significantly from jUnit and the other xUnit-based

language-dependent frameworks, and product offerings

• In RDz

v9.1

- zUnit supports Test Case + “stub module” generation,

plus automated testing and result XML generation for:



COBOL batch call-able subroutines - with or without QSAM/VSAM and DB2 file I/O

13



zUnit Test Runner

• Runs on z/OS

• Installed and configured on z/OS as part of RDz Host install and customization • Fetches and runs the Test Suite referred to in a zUnit configuration file



zUnit Wizard used to generate Test Cases

• RDz client feature

• Eclipse based wizards allow creation of:

• Template Test Cases are generated in COBOL or PL/I

• Simple pass/fail assertion API

(RDz v9.1) Complete COBOL test cases:

• Identify the interface or set of copy book(s)

• Generate XML Schema to represent the interface

• Generate XML files where you would specify test input and expected output • Generate a Test Case based on the XML file

• (Optionally) Generate stubs for called programs

 RDz viewers/editors for unit test XML results

zUnit/Unit Testing is

Not

a “Silver Bullet”

GUI 3270 Regression Testing Performance Testing Interactive Testing Integration Testing

Automated Unit Testing Test Driven Development

Rational Function Tester

Application Performance Analyzer

zUnit RDz Integrated Debugger

IBM Debug Tool

IBM recognizes this - and provides multiple testing solutions, each

optimized for different problem areas within the software testing

space:

15

xUnit Components and Terminology

Test Runner A test runner executes test cases or test suites and makes the results available in some programmatically-consumable way (e.g.,

XML,...).

Test Case Test cases execute within a test fixture and make assertions during test execution. Test cases contain any number of tests where

each test is a separate subprogram, function, method, or procedure in the test case.

Test Suite Test suites define a set of test cases for test execution.

Assertions Assertions verify the behavior and state of a test case. A failed assertion should result in an exception being thrown.

Test Fixture A test fixture is a context and/or environment that the programmer creates (setup()) before a test in a test case is executed, and

discards (teardown()) immediately after the test is run.

Test Execution Test execution involves invoking the Test Runner with input that includes, among other things, which test Cases or test suites

should be run.

All xUnit frameworks share the following basic component architecture, with some varied implementation details.

Test Execution Test Runner Test Suite Test Case Test Fixture Assertions

Your program and the program logic

to be tested Test Results

(XML File)

There are hundreds of

distributed xUnit products and implementations (jUnit, JSUnit, NUnit, XMLUnit,etc.). They all contain code frameworks in the underlying supported language. Many but not all are TAP

compliant.

Launches

Reads from

Which identifies

Which constructs

zUnit Component Implementation

Test Runner A z/OS systems program developed by the RDz team, and installed with RDz. The Test Runner orchestrates the unit testing

process

Test Case A generated COBOL or PL/I program compiled into a Load Module that calls your target Subroutine. Test Cases are executed

(Called) by the Test Runner

Test Suite An XML file that provides a list of Test Cases to be executed by the Test Runner

Assertions COBOL or PL/I conditions that test individual units of source code in your program

Test Fixture A generated set of COBOL or PL/I programs that implement the xUnit framework standard; for setting up, running and tearing

down individual Unit Tests

Test Execution z/OS JCL that invokes the Test Runner

Test Execution //JCL

Test Runner - RDz zUnit run-time

Test Suite - List of Test Cases (Load Modules)

Test Case - Generated COBOL or PL/I program

Test Fixture - Generated COBOL or PL/I programs

Assertions - Conditional Expressions

coded in COBOL or PL/I Your program and the _{logic to be tested}

Test Results (XML File)

RDz substitutes generated z/OS-specific resources for distributed technology classes, and frameworks.

EXEC PGM=

OPEN INPUT READ Test Suite Which identifies Test Case Load Module

The Test Runner CALLs (interacts with) Test Case Load Module

Which contains Test Fixtures

(COBOL subprograms or PL/I procedures Each Test Fixture sets up and CALLs

(executes) individual Unit Tests and Makes Submit JCL

17

zUnit Test Case Development Wizard

RDz provides a wizard-driven process to create your Test Cases and Test Fixture programs

The Test Case is a generated COBOL or PL/I module that contains:

• Nested programs with standard xUnit names and functionality

• 1  M generated Test Fixtures (programs) - One program for each of the Tests

defined using the wizard

You customize the Test Case

programs, and add statements

specific to your business logic and

testing requirements:

•

Initialize test input variables

•

Evaluate test outcome

RDz’s zUnit framework

takes care of the rest

UNIT0001(...) ADDTESTS(...) SETUP(...) TEARDOWN(...) TEST1(...) TESTn(...) . . . .

zUnit Test Case Module

USER.ZUNIT.LOAD(UNIT0001)

19

Topics:

Agenda



Unit Testing Value Proposition,

zUnit Terms, Concepts and Vocabulary



Demonstration

Demonstration

What you will see:

1. Develop Unit Tests for your programs

•

Use the zUnit Automated Testing

Framework to generate a Test Case

•

Which includes Test Fixtures that contain

Assertion logic that unit tests your program

•

Compile & Link the Test Case

2. Execute the Unit Test

21

IBM Rational Developer for System z version 8 21 http://www.youtube.com/watch?v=vjNY7SMtVqU&list=UUdgVL1ysbledgitPTYKyJuQ&index=1&feature=plcp

JES

Review - What you Saw - zUnit Run-time

zUnit Test Runner

(Test Harness)

READ

Test Suite

CALL (invoke)

Test Case

Invoke

Test Fixture

Interact with different nested COBOL

subprograms or PL/I procedures within your

Test Case

to:

- Setup unit test variable values - Run a test.

- Evaluate the Assertion outcome - Release system resources

Generate Results File

//

AZUCFG DD DSN=…

• Names the Test Case module(s)

//AZURES DD DSN=…

• Stores test results in XML PDS member

// EXEC PGM=Test Runner

//AZUCFG  Test Suite //AZURES  Test Results //…Test Case Load Library

Launches…

Submit JCL…

Test Case

Load Module to be unit tested

• Testable logic routine

• Testable logic routine

• Testable logic routine

Test Execution

Test Fixture

- SETUP Linkage Variables

- Run Unit Test Assertion Logic (CALL) - Evaluate outcome - Release resources

z/OS

Test Suite

Target Subroutine

Test Results

23

Topics:

Agenda



Unit Testing Value Proposition,

zUnit Terms, Concepts and Vocabulary



Demonstration

Review - zUnit Test Case Development Wizard

RDz provides a wizard-driven process to create your Test Cases and Test Fixture programs

25

Review - xUnit Vocabulary / zUnit Implementation



Test Execution:

JCL that invokes the

Test Runner

(a “supervisor program” or

“test harness” that runs your

Test Suite

)



Test Runner:

The zUnit “supervisor program” invoked by

Test Execution

, that:

 Reads a Test Configuration Test Suite  Calls your Test Cases



Test Suite

:

an XML configuration file that defines which Test Cases the

Test Runner

should execute (Note - a Test Case == a Load Module)



Test Case

: An RDz-generated COBOL or PL/I program that:

 Is called by the Test Runner

 Uses a Test Fixture to Call your Subroutine and write test outcomes to a Test Results file



Test Fixture

:

A set of generated programs nested inside your Test Case. Test Fixtures run Tests that utilize Assertion logic to Call your Subroutine, and evaluate the result.



Subroutine: The target call-able program to be Unit Tested



Test Results: An output dataset that describes the results of each test run

in xUnit standard (XML) file format



Assertion

:

A simple pass/fail predicate used to test one part of your Subroutine’s logic

27

Links - to additional Unit Testing content

http://lisacrispin.com/wp-content/uploads/2011/11/Agile-Testing-Quadrants.png http://1.bp.blogspot.com/-RQLtpTssY_o/UZ9CYqzflqI/AAAAAAAAAyo/9kIx6aGwSaU/s1600/TestingTrianglePished.png http://en.wikipedia.org/wiki/White-box_testing http://en.wikipedia.org/wiki/Black-box_testing http://en.wikipedia.org/wiki/XUnit http://en.wikipedia.org/wiki/Test-driven_development