Improving testability of object-oriented systems

Improving testability

of object-oriented systems

Dissertation

zur Erlangung des Grades eines

Doktors der Naturwissenschaften (Dr. rer. nat.)

des Fachbereichs Informatik

der FernUniversität in Hagen

Dipl.-Ing. Stefan Jungmayr

erster Berichterstatter

Univ.Prof. Dr. H.-W. Six (FernUniversität in Hagen)

Ao.Univ.Prof. Dr. G. Futschek (TU-Wien)

Jungmayr, Stefan:

Improving testability of object-oriented systems Stefan Jungmayr. – Als Ms. gedr.. –

Berlin : dissertation.de – Verlag im Internet GmbH, 2004 Zugl.: Hagen, Fernuniv., Diss., 2003

ISBN 3-89825-781-9

Bibliografische Information Der Deutschen Bibliothek

Die Deutsche Bibliothek verzeichnet diese Publikation in der Deutschen Nationalbibliografie; detaillierte bibliografische Daten sind im Internet über <http://dnb.ddb.de> abrufbar.

Copyright dissertation.de – Verlag im Internet GmbH 2004

Alle Rechte, auch das des auszugsweisen Nachdruckes, der auszugsweisen oder vollständigen Wiedergabe, der Speicherung in Datenverarbeitungsanlagen, auf Datenträgern oder im Internet und der Übersetzung, vorbehalten.

Es wird ausschließlich chlorfrei gebleichtes Papier (TCF) nach DIN-ISO 9706 verwendet. Printed in Germany.

dissertation.de - Verlag im Internet GmbH Pestalozzistraße 9

10 625 Berlin

i

Unless the LORD builds the house,

its builders labour in vain.

ii

Zusammenfassung

Software kontrolliert in zunehmendem Maße sicherheits- und geschäftskritische Systeme. Zur Sicherstellung der Qualität und Fehlerfreiheit dieser Software werden weitgehend Tests eing-esetzt. Der Testvorgang selbst ist dabei meist mit einem großem Ressourcenaufwand verbunden. Deshalb wurden bisher zahlre-iche Ansätze zur Reduktion des Testaufwandes entwickelt, die auf eine Verbesserung der eingesetzten Testmethoden und Testwerkzeuge fokusieren. Nicht zuletzt hängt der Testaufwand bzw. Testerfolg aber auch wesentlich von den Eigenschaften des zu testenden Produktes selbst ab, welche unter dem Begriff Testbarkeit zusammengefasst werden.

Die vorliegende Arbeit beschreibt einen methodischen Ansatz zur Verbesserung der Testbarkeit von objektorientierter Soft-ware. Der Schwerpunkt liegt dabei auf der Kontrolle der Abhän-gigkeiten zwischen den verschiedenen Teilen des Softwaresys-tems, beginnend in der Anforderungsermittlung bis hin zur Implementierung und dem Test. Unterstützt wird der Ansatz durch Entwurfsrichtlinien, neuartige Produktmetriken sowie einem eigens entwickelten Analysewerkzeug. Wesentliche Teile des Ansatzes werden anhand von Fallbeispielen demonstriert und validiert.

iii

Summary

Safety- and business-critical systems are often controlled by software. To guarantee the quality and correctness of the soft-ware, tests are widely used. Unfortunately, testing consumes a significant amount of resources. A number of approaches to reduce the test effort have been designed so far which mainly focus on the test methods and tools applied. However, a main influence on the test effort and test success comes from charac-teristics of the software under test, also known as testability. This thesis describes a systematic approach to improve the test-ability of object-oriented software. The main focus lies on con-trolling the dependencies between different parts of the system, from the requirements capture activity to the implementation and test activity. The approach is supported by design guidelines, new product metrics, and an analysis tool. Essential parts of the approach are demonstrated and validated based on a set of case studies.

v

Preface

This thesis covers some important aspects of testability but of course not the entire field of testability engineering. You can find additional information on the subject as well as information on how to contact the author at the following website1:

http://www.testability.de

Any comments, questions and suggestions regarding testability are welcome!

1 The author is not able to guarantee, that this website will be available for more than several years after the publication date of this work.

vii

Acknowledgements

I thank god for all what he provided to me out of his grace and outside of my control, including the numerous support by other people and the ideas concerning technical solutions.

I am grateful to Univ.Prof. Dr. Hans-Werner Six for the opportu-nity to work on this thesis, for his comments and advice on the content, and for the chance to learn from his writing experience. I’m thankful to Ao.Univ.Prof. Dr. Gerald Futschek for his encour-agement and for his willingness to act as a referee.

Appreciations go to Prof. Dr. Mario Winter for his support and comments on working drafts as well as to Edgar Merl and Alex-ander Müller who contributed significantly to this work in the course of their master theses and in terms of reviews.

Many thanks go to my dear colleagues Henrik Behrens, Andreas Homrighausen and Thorsten Ritter, which have always been willing to discuss technical issues at their doorstep.

My special thanks goes to my parents, who granted me the pos-sibility to study and who encouraged me all the time as well as to my loving wife Sandra who enabled me to work on this thesis by managing much of our private life and obligations.

Also thanks to TogetherSoft and Borland who granted a campus license of Together (a tool which has been used for this work) to the FernUniversität in Hagen.

ix

Contents

Part I

Introduction and Related Work

1 Introduction ... 3

1.1 Testing ... 3

1.2 Testability ... 4

1.2.1 Definition ... 4

1.2.2 Importance of Testability ... 5

1.2.3 Application Problem, Software Artifacts, and Testability... 6

1.3 Dependencies ... 9

1.4 Open Problems ... 11

1.5 Outline of Thesis ... 12

2 Dependencies and Test Tasks ... 15

2.1 Test Levels, Test Activities, and Test Tasks ... 15

2.2 Effects of Dependencies on Test Tasks... 17

2.2.1 Define Test Order... 17

2.2.2 Define Test Cases... 17

2.2.3 Implement Stubs, Mocks, and Drivers ... 18

2.2.4 Build System ... 18

2.2.5 Instantiate and Initialize Objects Involved in Tests ... 19

2.2.6 Execute Tests ... 19

2.2.7 Observe and Analyze Test Results... 19

2.2.8 Isolate Errors... 20

2.2.9 Identify Change Impact ... 20

2.2.10 Maintain Test Cases ... 20

2.2.11 Automate Test Cases... 21

2.3 Effects of Dependency Cycles ... 21

2.4 Effects of Indirect Dependencies ... 21

3 Related Work ... 23

3.1 Introduction ... 23

3.2 Closely Related Work... 24

3.2.1 Metric “Average Component Dependency”... 24

3.2.2 Test-First Design... 24

3.2.3 Metric for Class Interactions... 25

x Contents

3.3 Other Related Work ... 26

3.3.1 Coupling Metrics ... 26

3.3.2 ISO Standard 9126-2 ... 28

3.3.3 Design Guidelines ... 28

3.3.4 Law of Demeter... 29

Part II

Approach

4.1 Systematic Control of Test-Critical Dependencies ... 33

4.2 Life-Cycle Support... 34

4.3 Guidelines, Metrics, and Tool Support ... 34

5 Requirements Capture... 37

5.1 Introduction ... 37

5.1.1 Overview of Requirements Capture Activity... 37

5.1.2 Important Concepts... 38

5.1.3 Requirements Capture and Testability... 40

5.2 Use Case Dependencies ... 41

5.3 Use Case Dependencies and Testing... 42

5.4 Specify Use Case Criticality and Frequency ... 43

5.5 Control Use Case Dependencies... 43

5.5.1 Control Mandatory Domain Class Associations ... 43

5.5.2 Control References from Use Cases to Domain Classes .... 46

5.6 Control Dependencies to External Systems... 49

6 Analysis ... 51

6.1 Introduction ... 51

6.1.1 Overview of Analysis Activity ... 51

6.1.2 Important Concepts... 52

6.1.3 Analysis and Testability... 52

6.2 Identify Classes Relevant to Testability ... 53

6.2.1 Identify Test-Critical Classes... 53

6.2.2 Identify Hard-To-Test Classes ... 54

6.2.3 Identify Test-Sensitive Classes ... 54

6.2.4 Using Prototypes to Identify Classes Relevant to Testability55 6.2.5 Relationships Between Different Class Categories... 56

6.3 Identify Test-Critical Dependencies ... 56

6.4 Specify Testability Requirements... 57

7 Design ... 59

7.1 Introduction ... 59

Contents xi

7.1.2 Important Concepts... 60

7.1.3 Design and Testability... 60

7.2 Design Dependencies ... 61

7.2.1 Hard-Wired Dependency... 62

7.2.2 Semi-Hard-Wired Dependency ... 65

7.2.3 Type Dependency ... 66

7.3 Design Dependencies and Testing ... 67

7.3.1 Effects of Hard-Wired Dependencies... 67

7.3.2 Effects of Semi-Hard-Wired Dependency ... 69

7.3.3 Effects of Type Dependency ... 69

7.4 Control Design Dependencies ... 69

7.4.1 General Design Principles... 70

7.4.2 Limit Indirect Dependencies... 70

7.4.3 Test-Critical Dependencies Shouldn’t be Hard-Wired... 71

7.4.4 Involve Hard-To-Test and Test-Sensitive Classes on Demand 72 7.4.5 Ensure Substitutability... 72

7.4.6 Isolate Access to Technical Infrastructure ... 73

7.4.7 Put Special Attention on Object Construction ... 74

7.4.8 Make Entity Classes Self-Contained... 75

7.5 Techniques to Control Design Dependencies ... 75

7.5.1 Early Creation of Test Artifacts ... 75

7.5.2 Design Reviews ... 76

7.5.3 Design Metrics ... 76

8 Implementation... 77

8.1 Introduction ... 77

8.1.1 Overview of Implementation Activity ... 77

8.1.2 Implementation and Testability ... 78

8.2 Implementation Dependencies... 78

8.3 Control Dependency Structure ... 80

8.3.1 Identify Static Dependencies... 80

8.3.2 Evaluate Dependency Structure ... 80

8.3.3 Evaluate Dependency Cycles ... 83

8.3.4 Identify Test-Critical Dependencies ... 85

8.3.5 Identify Candidates for Refactoring... 89

8.3.6 Perform Refactorings ... 90

8.4 Control Hard-Wired Dependencies ... 94

8.4.1 Identify Hard-Wired Dependencies ... 94

8.4.2 Evaluate Hard-Wired Dependencies... 94

8.4.3 Identify Candidates for Refactoring... 94

8.4.4 Perform Refactorings ... 95

8.5 Control Semi-Hard-Wired Dependencies ... 96

8.5.1 Identify Semi-Hard-Wired Dependencies... 96

8.5.2 Evaluate Semi-Hard-Wired Dependencies ... 97

8.5.3 Perform Refactorings ... 98

xii Contents

9 Test ... 101

9.1 Introduction ... 101

9.2 Evaluate Testability ... 101

9.2.1 Test Problem Reports ... 101

9.2.2 Test Process Metrics... 102

Part III

Metrics and Tool Support

10.1 Introduction ... 105

10.2 Overview of Dependency Graph Model ... 106

10.3 Classes of Dependency Graph Model ... 108

10.3.1 Class “Class”... 108 10.3.2 Class “Member”... 111 10.3.3 Class “ClassDependency”... 111 10.3.4 Class “Generalization”... 112 10.3.5 Class “TypeDeclaration”... 113 10.3.6 Class “MemberAccess” ... 114 10.3.7 Class “System”... 116

10.4 Complete Dependency Graph Model ... 118

10.5 Constructing a Dependency Graph ... 120

11 Metrics... 123

11.1 Introduction ... 123

11.1.1 Basic Measurement Concepts ... 123

11.1.2 Overview of Metrics... 124 11.2 Class Metrics... 125 11.2.1 Metric CD ... 125 11.2.2 Metrics CDh and CDsh ... 127 11.3 System Metrics... 129 11.3.1 Metric ACD... 129

11.3.2 Metrics ACDh and ACDsh... 131

11.3.3 Metric NCDC ... 133 11.3.4 Metric NFD ... 134 11.4 Dependency Metrics ... 135 11.4.1 Metric DSTM ... 135 11.4.2 Metric DSTMh ... 136 11.5 Reduction Metrics ... 138 11.5.1 Introduction ... 138 11.5.2 Metric rACD... 139

11.5.3 Metrics rACDh and rACDsh ... 142

11.5.4 rNCDC... 143

11.5.5 Metric rNFD... 144

Contents xiii 12 Tool Support ... 151 12.1 Overview of Design2Test ... 151 12.2 Functionality of Design2Test ... 151 12.3 Architecture of Design2Test... 153 12.3.1 Process View ... 153 12.3.2 Architectural View ... 154

Part IV

Case Studies

13.1 Introduction ... 159

13.2 Investigated Systems ... 160

13.3 Investigated Issues ... 161

14 Results ... 163

14.1 Distribution of Reduction Metric Values ... 163

14.2 Correlation between Reduction Metrics ... 168

14.3 Correlation between rACD and Coupling Metrics ... 169

14.4 Metric rACD and Effect on System Structure ... 172

14.5 Metric rACD and Design Errors... 174

14.6 Metric rACD and Effect on Testing... 178

14.7 Metric rACD and Strength of Dependencies ... 179

14.8 Amount and Cause of Hard-Wired Dependencies ... 181

14.9 Semi-Hard-Wired Dependencies and Refactoring ... 183

14.10 Feedback Dependencies ... 186

14.11 Metric rACDin and Effect on System Structure ... 187

Part V

Conclusion

16 Future Work ... 195

16.1 Testability Engineering Process ... 195

16.2 Improvement of Testability Metrics ... 196

xiv Contents

Appendix

A Glossary ... 201

B Java - Syntactic Dependencies... 205

B.1 Generalization ... 205

B.2 Type Declarations ... 206

B.3 Member Access ... 208

B.4 Dependencies Involving Initializers and Inner Classes ... 210

C Coupling Metrics ... 213

D Testability and Other Quality Characteristics ... 217

D.1 Overview ... 217

D.2 Testability and Maintainability ... 218

D.3 Testability and Reliability... 219

D.4 Testability and Performance ... 220

D.5 Testability and Reusability ... 220

D.6 Testability and Traceability... 221

D.7 Testability and Usability ... 221

E Index ... 223

1

Part I Introduction and Related Work

The first part of this thesis introduces the topic addressed and motivates its importance for software development. Furthermore this part describes related work and identifies open research issues due to the shortcomings of currently existing approaches.

3

Chapter 1

Introduction

The purpose of this chapter is to give an overview of the topic of this thesis. It describes the problems addressed, why they are important and nontrivial to solve, and provides an overview of the content and structure of the document.

1.1

Testing

Dynamic testing The term testing is widely used as a synonym for the term

dynamic testing. Dynamic testing is a defect detection technique

based on executing a software system with selected inputs and comparing the observed results with the expected results. (Other defect detection techniques like reviews, inspections, and automated static analysis are referred to as static testing tech-niques.) In the remainder of this document we use the term “testing” as an abbreviation for “dynamic testing”.

Motivation

Testing is important for error detection and continuos software evolution:

• The potential impact of software errors on business, human life, and environment grows as software controls more and more critical functionality within technical products and busi-ness processes. Unfortunately, software development is an error prone process. Testing is the most widely used tech-nique to detect errors.

• If user requirements change frequently, it is important for the software developer that the software system can be adapted and extended easily. New functionality added to a system should not break existing functionality. Regression testing is one technique to assure, that existing functionality remains intact after implementation changes. Without the ability to perform regression tests quickly and easily after implemen-tation changes, the risk of undetected errors in the new

soft-4 Chapter 1: Introduction

ware release increases. Testing is therefore an enabling fac-tor for continuos and rapid software evolution.

Additionally, test cases are utilized in a new manner in the con-text of Xtreme Programming and agile software development: they act as a requirements specification. This practice is called

Test-First Design or Test-Driven Development and means, that

the required functionality is specified in terms of executable test cases before the implementation of the related functionality starts - no separate requirements specification document is cre-ated [Beck03].

Test Effort

While testing is important for software quality and evolution, it is a major cost driver as well: about 25% to 50% of an average development budget are spent on testing [Pol00] [Spil03]. The actual amount of time and money which is needed to achieve the test goals as well as the ability to achieve the test goals at all depends on several factors. These factors include e.g. human skills and the degree of test automation but also the characteris-tics of the software, the later subsumed by the term testability.

1.2

Testability

1.2.1

Definition

Testability comprises the characteristics of a software artifact1 which have an impact on the ease to achieve test goals. The term testability is defined in this work as following:

The degree to which a software artifact facilitates testing in a given test context.

The following considerations are important to our definition: • Not only source code or executables effect testing but other

software artifacts as well (e.g. requirements specification documents).

• Whether a software artifact facilitates testing or not can only be determined relative to the test context, i.e. with respect to a given set of test goals, resources, techniques and tools.

1 A software artifact is a document or product created during software de-velopment. Examples are requirement specification documents, design documents, source code, and executables.

Chapter 1: Introduction 5

Example: a software artifact may facilitate black-box testing by providing a dedicated test interface while not supporting white-box testing because of unstructured code. If achieving white-box test coverage is not a test goal, then this does not have a major negative impact on overall testability, other-wise it does.

1.2.2

Importance of Testability

Large systems Several software development and testing experts pointed out the importance of testability and design for testability, especially in the context of large systems:

“During the design of new systems we do not have only to answer the question ’can we build it?’ but also the question ’can we test it?’. Good testability of systems is becoming more and more important.” [Pol00]

“The absence of design for testability in large systems can greatly reduce testing effectiveness.” [Bind99]

“Design for testability, although rarely the first concern of smaller projects, is of paramount importance when success-fully constructing large and very large C++ systems.”

[Lako96]

Other Definitions of Testability

The relative ease and expense of revealing software faults. [Bind94]

(1) The degree to which a system or component facilitates the establishment of test criteria and the performance of tests to determine whether those criteria have been met, and

(2) the degree to which a requirement is stated in terms that permit establishment of test criteria and performance of tests to determine whether those criteria have been met. [IEEE90] A set of attributes that bear on the effort needed for validating the modified software. [ISO9126-2]

A software system is testable if 1) its components can be tested separately, 2) test cases can be identified in a system-atic manner and repeated, and 3) the test results can be observed. [Kahl98]

6 Chapter 1: Introduction

Factors contributing to importance of testability

The importance of software testability for a particular software system increases with

• the size and complexity of the system,

• the risks for life and business if errors remain undetected, • the frequency of the test activities, and

• the life-time of the system (assuming that maintenance and regression testing are permanent tasks).

Beneficiaries of testability

Testability is important for software testers and programmers because it helps them to keep the test effort under control. Addi-tionally it is relevant to customers as well: customers benefit from higher product quality and faster fixing of errors occurring at the customer site when testability features like built-in-tests, automatic failure reports, and built-in diagnostic capabilities pro-vide better and faster information to the developers about the cause of failures which accelerates problem fixing.

1.2.3

Application Problem, Software Artifacts, and Testability

The testability of a software system depends not only on the software artifacts specifying, designing, or implementing the structure and behavior of the system, but also on the original application problem to be solved and the software artifact(s) used as a test reference when defining the test cases (Figure 1).

Testability and Other Defect Detection Techniques

Testability is by definition linked to testing. Of course, testing is not the only defect detection technique which benefits from a high level of testability: improving testability is beneficial for reviews, too. Characteristics of testable software like adequate complexity, low coupling, and good separation of concerns make it more easy for reviewers to understand the software artifacts under review.

Chapter 1: Introduction 7

Figure 1 Application problem, software artifacts, and testability

Related to these three subjects of testability considerations we distinguish three main characteristics of testability: problem test-ability, test reference testtest-ability, and test object testability.

Problem testability

Problem testability describes the effect of the application

prob-lem on the ease to test the planned software system. The appli-cation problem is usually defined by the customer and seldom under the control of the software developer.

Oracle problem An application problem makes testing difficult if a good test ora-cle1 does not exist [Weyu82], e.g. because 1) the program is written to calculate an output which is difficult or impractical to calculate manually or 2) because the planned software system will have to produce so much output that it is impractical to verify it all.

Interaction with external systems

Part of the application problem is the required interaction with external systems as specified by the customer. These external systems and required interaction may generate test problems as well. test process application problem requirements specification design software artifacts testability input influence implemen-tation

test reference test object

1 A test oracle is a source of an expected test result for a test case which can be e.g. a human or another software system.

8 Chapter 1: Introduction

Test reference testability

Test reference testability is the degree to which the test

refer-ence facilitates testing in a given test context. Quantitative

requirements

A test reference facilitates testing, if it provides non-ambiguous, quantitative, complete, and consistent information which can be used to design test cases and to implement a (reliable) test ora-cle. A typical test reference is a requirements specification docu-ment. The testability of requirement specifications is “probably the most important [concern for requirements], and, at the same time, the most difficult one to achieve.” [Roma85]

Test object testability

Test object testability is the degree to which a software artifact

specifying, designing, or implementing the structure and behav-ior of the system facilitates testing in a given test context.

A number of factors contribute to test object testability. The most important factors include complexity, separation of concerns, coupling (i.e. dependencies), controllability, and observability. it is usually the software developer who specifies and imple-ments the required level of test object testability which is there-fore under his (and not the customer’s) control.

In the remainder of this document we use the term “testability” as an abbreviation for “test object testability”.

Our Focus

Testability is, like testing itself, a very broad topic. Therefore we limit our focus on selected aspects of testability:

• We deal to some extent with problem testability and describe e.g. how to reduce the application problem.

• The main focus is on test object testability and in particular on the effect of dependencies on the testability of object-ori-ented systems.

• Test reference testability is not addressed in this thesis. Note: The interested reader can find information on how to define testable requirements in [Rupp02] and informa-tion on how to define testable UML1-models to be used as a test reference in [Bind99].

Chapter 1: Introduction 9

1.3

Dependencies

Origin of dependencies Large systems are usually built from smaller units which makes it easier to understand the system, to define development tasks and to assign them to different people. Naturally, breaking a sys-tem into units leads to dependencies between these units. The majority of the software units within a large system depends on other units to provide their functionality.

Dependencies not only exist between software units but also between entities of other software artifacts like requirements specification and design documents. The dependencies between all kinds of software entities have an effect on testing. In the context of this work, we define the term dependency as follows:

Definition A dependency is a directed relationship between two entities where changes in one entity may cause changes in the other (depending) entity.

Note: The term “dependency” is used in the UML with a slightly different meaning: a (UML) dependency is an

ele-ment of a UML model which represents the relationship

between two other elements of the same model. Our defini-tion, however, applies to dependencies between entities of software artifacts, no matter whether they are represented within a model or not.

Semantic dependency Above definition describes a semantic dependency between two entities.

Syntactic dependency A syntactic dependency between two entities means that the depending entity contains a syntactic reference to the other entity.

Note: A semantic dependency may not lead to a syntactic dependency in all cases. For example, a class

A

B

concerning the meaning of particular attribute values but may not interact with class

B

A

B

10 Chapter 1: Introduction

Direct and indirect dependencies

Direct dependency Dependencies define a relation between entities, i.e. between two entities there exists at most one direct dependency in each direction.

Indirect dependency A dependency relation can be transitive (like compile dependen-cies1): if an entity

A

B

C

A

C

We further distinguish between class dependencies and model dependencies.

Class dependencies

In the context of dynamic testing and testability we are inter-ested in source code dependencies, since it is source code which is executed during testing. Within object-oriented sys-tems, source code dependencies manifest as class

dependen-cies.

Note: By classes we mean concrete classes, abstract classes, and interfaces.

Client and supplier class In the context of a class dependency from class

A

B

A

B

Model dependencies

The specification and design of a system is usually based on models. A common notation to describe such models is the UML [UML01] which provides e.g. use case models and class dia-grams. Dependencies between model entities are represented e.g. as class associations or use case relationships. Such model

dependencies are the origin of class dependencies at the code

level later on which makes them interesting to us as well.

Note: Dependencies between model entities can be further traced back to dependencies between entities of the real world. Evaluating (and perhaps changing) dependencies between real world entities is out of scope for this work. Our focus In this work, we mainly focus on syntactic class dependencies

and only to some extent on syntactic and semantic model dependencies.

Chapter 1: Introduction 11

Effect of dependencies

According to Les Hatton [Hatt99], 1) coupling caused by depen-dencies and 2) complexity are those characteristics of today’s software systems which contribute most to increasing test diffi-culties.

Dependencies indeed have a negative impact on all test tasks which will be discussed in more detail in Chapter 2.

1.4

Open Problems

A software developer who wants to evaluate and improve test-ability faces a number of problems because basic guidance is missing.

Lack of systematic approach

A general problem is the lack of a systematic approach to design and implement testability into software products. In our view the reasons for this situation are:

• Designers and developers do not face the test problems caused by their designs and implementations. Therefore they do not design and implement with testability in mind. Often the designers and developers introduce new depen-dencies into existing designs and implementations during the course of adding functionality while it is not them but the test and maintenance engineers which suffer from the result-ing negative impact on testresult-ing.

• The research on testability is still in its infancy. Researchers are often unaware of relevant related work. Testability engi-neering has not been yet established as a research field of its own (like usability engineering). Even a common terminol-ogy is still missing.

A systematic approach towards testability, however, is neces-sary to avoid a waste of resources and to enable continuos improvement.

Problems with controlling dependencies Existing approaches to

control dependencies not sufficient

Approaches to control the effect of dependencies on testability do exist, but they are not sufficient:

• Principles of good design have been developed from an implementation point of view and do not account for

require-12 Chapter 1: Introduction

ments which have their origin in tasks specific to testing (e.g. the ability to test a class in isolation).

• Existing design metrics like coupling [Fent96], abstractness and stability [Mart97] are metrics at the package or class level and do not help to identify individual dependencies which are critical for testing. Additionally, existing design metrics are general-purpose metrics: the relationship to test-ability is unclear and they are e.g. insensitive to specific test problems caused by certain categories of dependencies (see Section 7.3).

• Test-First Design (see Section ) helps to identify test prob-lems early (by developing the test code before the function-ality to be tested) but the approach is not systematic. Test problems are dealt with when they surface, leading to an iterative process and the need for frequent refactoring. While this may be a practical approach for smaller systems, the required rework effort for implementation and design arti-facts is likely to be prohibitive for large and critical software systems.

Controlling dependencies is not trivial

Controlling software dependencies in order to improve testability is nontrivial because, e.g.:

• The number of dependencies within a system is usually very large. There are no general criteria available that help to evaluate the effect of individual dependencies on testing. The question is, how to (automatically) identify dependen-cies with a potentially negative impact on testing.

• Because dependencies are transitive, a local dependency may have a global effect on the overall system structure and test effort. Such global effects are often unobvious from the local context.

• Criteria and guidelines are missing for evaluating dependen-cies during early development activities.

1.5

Outline of Thesis

This thesis describes a systematic approach to improve the test-ability of object-oriented software artifacts by controlling the effect of dependencies on testing.

Life-cycle support Our approach covers the software development activities requirements capture, analysis, design1, implementation, and test as they have been proposed by Jacobson [Jaco92] and

fur-Chapter 1: Introduction 13

ther derived by Six et al. [Six03]. For each development activity and related software artifacts (including use case models, class diagrams and source code) we describe constructive and analyt-ical measures to control the effect of dependencies.

Constructive and analytical measures

The constructive measures are supported by guidelines on how to specify testability requirements and how to deal with depen-dencies during design. The analytical measures include tech-niques to evaluate individual dependencies and to highlight potential points of improvement.

Focus The focus of our research lies on object-oriented software in the domain of conventional business applications. The software arti-facts we consider are modeled using UML and implemented using Java. Out of scope are special testability issues like those related to distributed and real-time systems or telecommunica-tion software.

Document Structure

This thesis is organized as follows:

Part I In the remainder of the first part we describe how dependencies effect common test tasks in general and discuss related work in the context of testability and dependencies.

Part II In the second part of this thesis we give an overview of our approach and describe for each development activity the neces-sary steps to control dependencies as well as supporting guide-lines, techniques, and metrics.

Part III At the beginning of the third part we introduce a dependency graph which is used as a basis to define the metrics (introduced in Part II) formally. The definition of each metric is accompanied by a motivation and discussion. A tool, which supports the com-putation of metric values, is introduced at the end of the third part.

Part IV The fourth part contains the details of three case studies con-cerning the usefulness and validity of the proposed metrics. Part V A summary and notes on future work conclude the thesis.

Appendix The appendix contains the bibliography and provides an over-view of

• important terms and abbreviations,

1 Architectural design is not considered because it has a global focus and does not describe individual dependencies.

14 Chapter 1: Introduction

• the types of dependencies within Java source code, • existing coupling metrics, and

• the relationship between testability and other software qual-ity characteristics.

15

Chapter 2

Dependencies and Test Tasks

Direct and indirect dependencies are relevant for testability because they effect common test tasks. Understanding these effects is essential if we want to evaluate and improve testability. This chapter gives an overview of common test tasks and how they are effected by dependencies.

Note to readers:

This chapter is mainly motivational, it does not introduce new concepts. You may skip this chapter, if you are familiar with test tasks and the effect of dependencies on testing.

2.1

Test Levels, Test Activities, and Test Tasks

Test tasks have to be solved at different test levels and within different test activities1. This section describes typical test levels and test activities.

Test levels

Common test levels are unit test, integration test, and system test. The test level refers to the test object which can be an indi-vidual class, a cluster of classes, a subsystem, or a complete system.

The scope of a unit test is only a single class which makes fault isolation easy.

The goal of an integration test is to test the interaction of two or more classes which may involve polymorphism as well. Nor-mally, integration testing starts, when the classes involved have already been unit tested.

During a system test, the entire system (or a major increment) is tested in order to make sure, that it fulfills the intended

16 Chapter 2: Dependencies and Test Tasks

ality as specified by the requirements. Often this is done via the (graphical) user interface of the system.

Test activities

The overall test activity consists of a number of (sub-)activities which can be found at each test level. This set of (sub-)activities includes:

• Test planning, the identification and specification of the required tests and resources as well as the scheduling of test sub-activities and test tasks.

• Test design, i.e. the specification of the test cases.

• Test preparation, i.e. the preparation of all what is needed to execute the test cases including a test framework, test tools, and stubs.

• Test execution, where the actual testing takes place.

• Test result analysis, which subsumes the collection and evaluation of test results.

• Test follow-up, the isolation and removal of errors.

• Test maintenance, which means to keep the test cases up-to-date after changes to the requirements, design, or imple-mentation.

Test tasks

Within each sub-activity a different set of concrete test tasks has to be solved. The test tasks include:

• Define the test order.

• Define the test cases to fulfill a test coverage criterion. • Implement stubs, mocks and drivers.

• Build the system (i.e. compile and link all required classes). • Instantiate and initialize the objects involved in the test. • Observe and analyze the test results.

• Locate errors.

• Identify the classes that have to be retested after a change to a particular class (change impact identification).

• Maintain test cases after program changes. • Identify reusable test cases.

• Automate the test execution.

The classes under test are the main input to most of the test tasks. Dependencies between these classes therefore have an effect on the ease to perform the test tasks.

Chapter 2: Dependencies and Test Tasks 17

2.2

Effects of Dependencies on Test Tasks

In the following we describe the test tasks in more detail and how dependencies effect the ease to perform them.

2.2.1

Define Test Order

A good test order helps to minimize the required number of stubs and drivers during unit and integration testing. The actual test order depends on the test strategy (e.g. bottom-up or top-down) and the class dependencies existing within the system. Effect of dependencies

If the classes and dependencies form a tree structure, it is trivial to identify a meaningful test order1, and the degree of freedom in choosing the test order is large.

Adding additional dependencies reduces the freedom in choos-ing the test order. Additional dependencies which introduce dependency cycles make it more difficult to identify a meaningful test order [Kung94b] [Kung94c] [Over94] [Wint99] and may require the implementation of stubs (see Section 2.2.3).

Note: Adding dependencies also has an impact on the overall test process because it reduces the opportunities for parallel testing.

2.2.2

Define Test Cases

Each test case has to describe the initial set of the classes involved, the initial state of the objects, the input parameters, global variables, and required method calls (or user interactions) as well as the expected test results like GUI output, return val-ues, object states, and exceptions thrown.

For a system of reasonable size it is practically impossible to cover all possible input combinations with help of test cases. Therefore, different test coverage criteria have been developed to help in finding a small but sufficiently large set of test cases with a high probability to uncover most of the errors.

1 In this case, the test order starts with leaf nodes and continues with nodes upwards the tree where all supplier nodes have already been tested.

18 Chapter 2: Dependencies and Test Tasks

Effect of dependencies

Adding a new dependency between a client class (the class under test) and a supplier class may for example a) increase the number of the client class’s methods and/or method parameters or b) make the control flow of the client class dependent on the state of the supplier class, which likely increases the required number of black-box test cases.

If the new dependency manifests as a change within a method body of the client class, additional white-box test cases may be required as well, especially if the state of the supplier class is rel-evant for the control flow of the class under test.

CUT = class under test Note: In the remainder, we will use the abbreviation CUT for the class under test.

2.2.3

Implement Stubs, Mocks, and Drivers

Stubs, mocks [Link01], and drivers are simple substitutes for classes (or interfaces):

• Stubs allow to test a class in isolation or to substitute classes which haven’t been tested yet.

• Mock objects additionally improve the ability to observe (intermediate) test results1.

• Drivers are used to control the CUT directly.

The need to implement stubs, mocks, and drivers depends e.g. on the integration strategy chosen.

Effect of dependencies

Introducing new dependencies increases the number of stubs required to test classes in isolation. Dependencies introducing dependency cycles require the implementation of stubs if the strategy is to break the cycles during testing.

2.2.4

Build System

Before the first test run, all executables of the classes involved in the test have to be built from the source code. If the CUT or its supplier classes have been changed before re-running a test, existing executables have to be rebuilt as well.

1 Mock objects may report intermediate test results in terms of the (se-quence of the) method calls to the mock object and the corresponding values of parameter objects.

Chapter 2: Dependencies and Test Tasks 19

Effect of dependencies

Adding dependencies means that more classes have to be re-compiled on average after program changes (see [Cold96]) which may increase the duration of the compile-test-debug cycle significantly.

2.2.5

Instantiate and Initialize Objects Involved in Tests

Objects of the classes involved in a test have to be instantiated and often initialized during test setup. Initialization may include the establishment of appropriate object links.

Effect of dependencies

An additional dependency to a supplier class often increases the number of objects that have to be instantiated and initialized. For example, if an instance of the supplier class has to be provided as a parameter object to the method under test, the test code has to create this supplier instance. If the CUT creates the instance of the supplier class itself, the test code is freed from this responsibility, but such direct creation of supplier instances can cause other test problems (see Section 7.2.1 and Section 7.3.1).

2.2.6

Execute Tests

Test case execution means to call a method of a class or to invoke user operation(s) provided by a graphical user interface (GUI). This can be done manually or automated.

Effect of dependencies

Adding a dependency from a CUT to a supplier class which involves time consuming operations like user input, access to a database, or access to the Internet increases the overall time needed for test execution.

2.2.7

Observe and Analyze Test Results

During and after test execution the intermediate and final results need to be observed. Test results include values of return parameters, the (hidden) state of objects, exceptions that have been thrown, and other kinds of system output.

20 Chapter 2: Dependencies and Test Tasks

The test results observed have to be compared with expected test results. In case of failures it is necessary to describe them with as much details as necessary to facilitate isolation of errors. Effect of dependencies

The more classes a CUT depends on the more instances will be involved in test execution and the more intermediate and final object states have to be observed as part of the overall test result (in general) which increases the related effort.

2.2.8

Isolate Errors

In case of a wrong test result it is necessary to track the failure down to the faulty class(es) and code statement(s): analytically based on supplemented information (like system traces) or empirically by adding, removing, exchanging or additionally test-ing objects of other classes involved.

Effect of dependencies

A higher number of dependencies makes fault isolation more dif-ficult because an error may hide in a larger number of classes [Cold96]. Cyclic dependencies pose a special problem because they lead to re-entrance situations [Szyp97] which make it harder to understand the code and to isolate faults.

2.2.9

Identify Change Impact

During maintenance, test cases have to be rerun in order to make sure, that no new errors have been introduced while fixing errors or adding new functionality. The number of test cases which have to be rerun during the regression tests can be reduced by identifying the classes which are actually effected by the implementation change.

Effect of dependencies

A higher number of dependencies causes more effort to deter-mine the set of classes effected by the change.

2.2.10

Maintain Test Cases

Test cases related to a given class may need to be updated if this class or one of its supplier classes changes.

Chapter 2: Dependencies and Test Tasks 21

Effect of dependencies

Adding dependencies to a system reduces the independence of unit test cases and increases the number of test cases which have to be maintained after an implementation change.

2.2.11

Automate Test Cases

Test automation means that some or all steps necessary during test execution, test result observation, and analysis are auto-mated. This requires that test results are presented in a way which facilitates comparison with expected results without the need for human intervention.

Effect of dependencies

Adding a dependency may have a negative effect on testing if the supplier class causes problems for test automation.

2.3

Effects of Dependency Cycles

A dependency cycle has a severe effect on testing which depends on the strategy chosen to deal with the cycle:

• If the cycle is not broken (using stubs) it is impossible to test the classes involved in isolation. Instead they have to be tested altogether at once.

• Additional stubs have to be implemented if the cycle shall be broken.

• A dependency cycle may cause a re-entrance1 situation dur-ing program execution which make it more difficult to under-stand the program and to isolate faults [Szyp97].

It is therefore strongly recommended to avoid dependency cycles whenever possible [Lako96].

2.4

Effects of Indirect Dependencies

Indirect dependencies to supplier classes have several effects on testing:

1 Re-entrance means that a method of an object A is invoked by another object while another method of object A is still executing.

22 Chapter 2: Dependencies and Test Tasks

• More classes have to be compiled before test execution which increases the time to rebuild the system after changes. A similar effect occurs in component-based sys-tems where re-deployment of components after implementa-tion changes is time-intensive.

• The indirect supplier classes have to be instantiated and ini-tialized during test setup (if the direct supplier classes do not take responsibility of that).

• Fault isolation becomes more difficult. A fault may have to be tracked down a long sequence of method calls until it is attributable to a particular class.

• Indirect supplier classes may slow down the test progress e.g. if they involve time-intensive computation or if they are not test-ready.

• The state of the indirect supplier classes may be relevant for test result analysis which increases the related effort.

23

Chapter 3

Related Work

In this chapter we describe related work in the context of test-ability, dependencies, and metrics.

3.1

Introduction

Testability engineering has not yet been established as a disci-pline within software engineering:

“Certain aspects of quality have been the focus of very little research. For example, although it is recognized as an issue in object-oriented development, testability has rarely been addressed.” [Bria99c]

“Testability is a compelling concept but in its infancy.”

[Whit00]

One reason for this situation is, that many researchers are not aware of work done by other researchers, leading to a lack of common terminology and slow research progress. Experience reports related to published approaches and techniques are almost completely missing.

Quite a large number of software engineering books or articles mention testability or address the topic to some extent, but there is only a limited number of publications that really go into the detail. Even fewer publications study the effect of dependencies on testing.

In the following sections we discuss 1) contributions concerning dependencies in the context of testability as well as 2) research focusing on dependencies without explicitly considering testabil-ity aspects.

24 Chapter 3: Related Work

3.2

Closely Related Work

Few authors have investigated dependencies in the context of testability so far.

3.2.1

Metric “Average Component Dependency”

Lakos John Lakos [Lako96] proposes a metric called “Average Compo-nent Dependency” (ACD) which is basically the average of the number of classes each class directly and indirectly depends on. Metric ACD is used to evaluate testability because it allows to predict the average effort required to recompile a system after a program change (before rerunning the test cases). Lakos also discusses the effect of cyclic dependencies on the dependency structure and how to avoid dependency cycles.

Related work Work related to the approach by Lakos includes reports on the use of metric ACD [Maso99] as well as tools which collect this metric like Ignominy [Tuur01] and JDepend [Clar01].

Discussion

Lakos uses metric ACD to evaluate the effect of an individual dependency on the dependency structure. This approach is purely manual and therefore practically limited to a small subset of the dependencies existing within a system. Additionally, Lakos does not take into account test tasks other then recompi-lation.

3.2.2

Test-First Design

Beck, Link, et al. One of the main techniques used in Xtreme Programming is called Test-First Design (or Test-Driven Development) which has been described in more detail by Johannes Link [Link02] and Kent Beck [Beck03]. Test-First Design means that the test cases are developed before the actual method bodies are imple-mented. Early feedback to programmers about test problems caused by class dependencies helps to improve testability from the beginning [Beck01].

Discussion

Test-First Design is the first popular approach in the software engineering field which contributes to testability. Unfortunately, testability is still a by-product of this approach and not the result of dedicated efforts: test problems are dealt with when they

sur-Chapter 3: Related Work 25

face during the implementation and test activity. Testability is not designed into the product systematically. No specific process steps are defined to achieve testability. Instead, Test-First Design is a trial-and-error approach to testability which heavily relies on refactoring. While this may be practical for smaller sys-tems, the time and effort needed to rework implementation and design artifacts is likely prohibitive for large and critical software systems.

Additionally, there is no evidence yet that Test-First Design leads to better software quality than conventional development processes. Some first experiments have been carried out to investigate this question:

• Müller et al. [Muel02] report, that Test-First Design did pro-duce less quality.

• Other authors report, that Test-First Design performed better than an ad-hoc approach to testing [Geor02] [Maxi03]. How-ever, a comparison to systematic testing would have been more relevant.

3.2.3

Metric for Class Interactions

Baudry et al. Benoit Baudry et al. focus on class interactions in the context of testability [Baud02]. A class interaction (according to Baudry) occurs, when a client class accesses one of its supplier classes concurrently via two (or more) distinct chains of associations. Such class interactions increase the possibility of side effects (which are a source of errors) and should be tested during inte-gration testing. Baudry et al. define a metric to evaluate the test complexity caused by class interactions and describe design guidelines that help to avoid concurrent class access.

Discussion

The problem addressed is relevant for integration testing and the proposed design guidelines are practical. Not sufficient yet is the motivation and validation of the proposed testability metric which is based on a (overly) complex formula.

3.2.4

Short Contributions

Some issues related to dependencies in the context of testability have been mentioned in work which does not have testability as a main focus:

• Implicit dependencies increase the risk of missing important test cases [Fowl01] [Koen99].

26 Chapter 3: Related Work

• Dependencies on static attributes with complex initialization may cause problems for test automation in subsequent test runs [Free02].

• Separating object creation from object use (e.g. using fac-tory methods or facfac-tory classes) facilitates test case reuse [Ruep97] [Dorm97]. Using a factory method e.g. allows to break an indirect dependency between the test class and a supplier class of the class under test by overriding the code responsible for object creation.

• Static dependencies on a singleton class1 make it more diffi-cult to test the client classes in isolation [Rain01].

• Hard-wired dependencies to system resources [Koen99] (like those introduced by using commands like

println()

in C++) make it more difficult to test the client classes in iso-lation.

3.3

Other Related Work

Several contributions concentrate on dependencies, but do not consider testability explicitly.

3.3.1

Coupling Metrics

Fenton and Pfleeger One of the first coupling metrics for object-oriented software has been defined by Norman E. Fenton and Shari Lawrence Pflee-ger, called “coupling between object classes” (CBO) [Fent96]. This metric is defined as the number of other classes to which a class is coupled. Other coupling metrics vary in the level of gran-ularity and the coupling aspects they take into account. A frame-work for coupling measurement has been described in [Bria96]. Discussion

A coupling metric measures the degree to which a given class is coupled to other classes within the system. Coupling metrics can be used to evaluate the effect of the dependencies of a given class on its testability. However, the shortcomings of exist-ing couplexist-ing metrics are:

1 Coupling metrics are metrics at the class level, not at the dependency level. Therefore they are not able to highlight individual dependencies which are critical. The developer

1 A client class needs to call a static method of the singleton class if it wants to get a reference to its only instance.

Chapter 3: Related Work 27

has still to decide which dependency to remove in order to reduce coupling.

2 The majority of the coupling metrics neglect indirect depen-dencies and their effects on test tasks like test setup and fault isolation.

3 If coupling metrics take into account indirect dependencies, they are insensitive to dependencies which cause a depen-dency cycle (each class within a dependepen-dency cycle has just the same coupling value) and insensitive to classes which are crucial for the overall number of indirect dependencies (like hubs1). Therefore they do not help to identify the cause of high coupling values.

Example 1

The dependency from class H to class B in Figure 2 has the big-gest impact on overall indirect coupling: it causes an increase of 30% compared to the system without this dependency (Table 1). The classes involved in this dependency do neither have the highest direct coupling values (metric CBO2) nor the highest indirect coupling values (metric CBOi3).

Figure 2 Dependency structure

1 Hubs are classes that, if removed from the system, break the dependency graph into independent dependency subgraphs.

2 The number of other classes a class depends on. This definition is used by the IDE Together (from Borland Inc.) to calculate the metric value. 3 The number of other classes a class depends on directly or indirectly.

C A B D H E F G I

28 Chapter 3: Related Work

Table 1 Classes and values of metric CBO

‰ 4 Coupling metrics are general purpose metrics. They are insensitive to characteristics of dependencies which make a difference for testing (for example related to the ability to test a class in isolation). This discussion will be continued in Sec-tion 11.5.2.

3.3.2

ISO Standard 9126-2

ISO The draft standard [ISO9126-2] defines a system-level testability metric similar to coupling metrics which does not consider the coupling between classes but the number of dependencies to other systems.

Discussion

The metric mentioned above does not help to identify critical individual dependencies (because it is a system-level metric).

3.3.3

Design Guidelines

Liskov and Martin Barbara Liskov and Robert C. Martin described and proposed a number of design guidelines related to coupling:

• A client class should be able to collaborate with any sub-class of its supplier sub-classes (“Liskov Substitution Principle”) [Lisk88].

• A high level component should not depend on the implemen-tation of low level components (“Dependency Inversion Prin-ciple”) but on the component interfaces [Mart96c].

class CBO CBOi

A 3 8 B 2 3 C 3 6 D 2 5 E 2 3 F 2 4 G 2 4 H 1 3 I 0 0

Chapter 3: Related Work 29

• The abstraction of a package (i.e. the degree to which a package consists of interfaces and abstract classes) should be in proportion to its stability (i.e. the difficulty of changing it based on incoming and outgoing dependencies) [Mart97]. This guideline is a refinement of the previous guideline. Martin additionally defined two simple metrics to measure abstractness and stability:

• Abstractness: the ratio of the number of abstract classes (and interfaces) to the overall number of all classes within a package.

• Stability: the ratio of the number of incoming dependencies to the overall number of (incoming and outgoing) dependen-cies of a package.

Discussion

The metric abstractness takes into account the type of depen-dencies because it distinguishes between dependepen-dencies to con-crete classes and dependencies to abstract classes or inter-faces. The shortcomings of the metrics abstractness and stability are:

• They are defined on package level. The developer still has to identify a meaningful set of classes and dependencies which should be refactored to obtain “good” metric values.

• The metric abstractness is based on a simple heuristic and does not take into account whether the classes of other packages actually use the interfaces defined within a given package or not. (It is easy to change the value of the metric abstractness by simply adding some interfaces to a pack-age.)

• The relationship between the metrics abstractness and sta-bility and the software characteristic testasta-bility has not been discussed by Martin.

3.3.4

Law of Demeter

Lieberherr et al. Lieberherr et al. [Lieb88] defined a design guideline called “Law of Demeter” to reduce direct coupling: A client class should not access the (“foreign”) supplier classes of its own direct supplier classes. The number of violations of this guideline can be used as a design metric.

Discussion

This guideline helps to reduce direct dependencies. However, reducing direct dependencies alone is not sufficient if indirect dependencies still remain.

31

Part II Approach

This part describes our approach to improve the testability of object-oriented systems by controlling dependencies within specification models, design models, and source code.

33

Chapter 4

Overview of the Approach

Testability has to be considered as early as possible within the development process. If testability problems are detected to late, then it is often impractical to solve them because the required effort to change major parts of the design and implementation is to high. Bruce F. Webster therefore describes one important pit-fall of software development as “thinking about testing after the fact” [Webs95].

Today’s software development projects lack a systematic approach to design testability into the product1:

“Software DFT [design for testability] is necessary but little is known at this time.” [Voas97h]

“Testing should play a much bigger role in design.” [Hatt99]

If a company instead follows a systematic approach to deploy testability within its products it gains competitive advantages:

“In the face of intense competitive pressure, a comprehen-sive and rational strategy to achieve high testability will be a strategic advantage - not a bottleneck.” [Bind95c]

This chapter provides an overview of our approach to improving the testability of object oriented software and describes its main characteristics.

4.1

Systematic Control of Test-Critical

Dependencies

Our approach is systematic by defining a sequence of steps which have to be performed in order to control the effect of soft-ware dependencies on testability. The steps cover the entire life-cycle and are both constructive and analytical.

1 The author only knows about design for testability efforts in the telecom-munication software domain.

34 Chapter 4: Overview of the Approach

Controlling test-critical dependencies

The basic idea behind our approach is to identify and control those class dependencies which have a major impact on testing, called test-critical dependencies. A dependency is test-critical, if it imposes a more negative impact on the ease to achieve test goals than the majority of all other dependencies.

4.2

Life-Cycle Support

Our approach supports testability deployment throughout the development activities requirements capture, analysis, design, implementation, and test. These activities, which have been defined in [Jaco92] and refined in [Six03], can be found in practi-cally all software development processes.

For each development activity our approach describes the steps necessary to control the effect of dependencies on testability:

• requirements capture: reduce the problem complexity with special emphasis on use case dependencies and domain classes.

• analysis: specify testability requirements and identify test-critical classes and dependencies.

• design and implementation: identify, analyze, and refactor test-critical dependencies.

• testing: evaluate the degree of testability finally achieved. The main focus in this thesis lies on the testability issues during design and implementation activity.

Note: Our approach does not include a comprehensive pro-cess definition in terms of roles, responsibilities, and propro-cess steps. However, some selected process issues which should be addressed in order to make testability efforts successful are mentioned in Section 16.1.

4.3

Guidelines, Metrics, and Tool Support

The constructive and analytical steps of our approach are based on guidelines, metrics, sensitivity analysis, and tool support. Guidelines

The constructive steps are based on design guidelines which help to avoid dependencies with a negative effect on testing.

Chapter 4: Overview of the Approach 35

Some of the guidelines refer to distinct categories of classes w.r.t. their role during testing.

Metrics and sensitivity analysis

The analytical steps of our approach are based on a set of new metrics which are linked to test tasks and help to evaluate indi-vidual class dependencies. A sensitivity analysis based on the metric set is used to evaluate the degree to which a dependency effects characteristics of the overall system and in this way to highlight potential areas of improvement.

Tool support

A metric tool implemented in the context of this thesis allows to identify class dependencies within Java source code and to cal-culate our set of metrics.

Figure 3 shows the building blocks of our approach and the input relationships.

Figure 3 The building blocks of our approach

analysis documents design code development artifacts metric tool metric analysis sensitivity analysis testability deployment metric definitions description of necessary steps design for testability design guidelines input

37

Chapter 5

Requirements Capture

During requirements capture the software developer has the unique chance to influence the user requirements towards a smaller complexity of the application problem and towards increased problem testability. This chapter describes what can be done during requirements capture 1) to reduce the complex-ity of the application problem with special emphasis on use cases, and 2) to control the influence of external systems on testability.

5.1

Introduction

In the following we give a general overview of the requirements capture activity, introduce important requirements capture con-cepts, and discuss briefly how testability relates to this activity in general.

5.1.1

Overview of Requirements Capture Activity

The requirements capture activity is devoted to the identification and specification of user requirements. Its result, the

require-ments specification, describes which functionality shall be

imple-mented, for what reasons, and on which objects and classes the functionality operates. Implementation issues are not considered during requirements capture.

The requirements specification usually contains a use case model and a domain model (i.e. a class model). They are often supplemented by models of system behavior and non-functional requirements.

At the beginning of the requirements capture, the requirements engineer acquires an overview and basic understanding of the application domain. A discussion with system users about