PRACTICAL MODEL BASED SOFTWARE TESTING A MASTER S THESIS INFORMATION TECHNOLOGY SERVICE MANAGEMENT ATILIM UNIVERSITY

(1)

PRACTICAL MODEL BASED SOFTWARE TESTING

A MASTER’S THESIS

IN

INFORMATION TECHNOLOGY SERVICE MANAGEMENT

ATILIM UNIVERSITY

BY

RENGİN ÇOŞKUN

MAY 2015

(2)

PRACTICAL MODEL BASED SOFTWARE TESTING

A THESIS SUBMITTED TO

THE GRADUATE SCHOOL OF NATURAL AND APPLIED

SCIENCES

OF

ATILIM UNIVERSITY

BY

RENGİN ÇOŞKUN

IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR

THE DEGREE OF

MASTER OF SCIENCES OF PHILOSOPHY

IN

THE DEPARTMENT OF INFORMATION TECHNOLOGY

SERVICE MANAGEMENT

(3)

Approval of the Graduate School of Natural and Applied Sciences, Atilim University.

_____________________ Prof. Dr. K. İbrahim AKMAN

Director I certify that this thesis satisfies all the requirements as a thesis for the degree of Master of Science.

_____________________ Assoc. Prof. Dr. Murat KOYUNCU Head of Department

This is to certify that we have read the thesis “Practical Model Based Software Testing” submitted by “Rengin ÇOŞKUN” and that in our opinion it is fully adequate, in scope and quality, as a thesis for the degree of Master of Science.

_____________________ Asst. Prof. Dr. Deepti MISHRA

Supervisor Examining Committee Members

Assoc. Prof. Dr. Korhan Levent Ertürk _____________________ Asst. Prof. Dr. Meltem Eryılmaz _____________________ Asst. Prof. Dr. Deepti Mishra _____________________

(4)

I declare and guarantee that all data, knowledge and information in this document has been obtained, processed and presented in accordance with academic rules and ethical conduct. Based on these rules and conduct, I have fully cited and referenced all material and results that are not original to this work.

Name, Last name: Rengin ÇOŞKUN Signature:

(5)

ABSTRACT

PRACTICAL MODEL BASED SOFTWARE TESTING

Çoşkun, Rengin

M.S., Information Technology Service Management Department Supervisor: Asst.Prof.Dr. Deepti Mishra

May 2015, 172pages

In this thesis, it is proposed a new approach using both manual and automated testing for MBT. The goal is to find successful test cases. Many different research papers are studied and UML sequence diagram examples of these research papers as a model are used in this thesis. Two new algorithms are written for manual testing. It is designed a new tree technique using depth first search path algorithm to find test path. It is written a new java junit testing code to find test paths automatically in automated testing. It is used same operation for both manual and automated testing and also, it is examined these two methods with same sequence diagram examples. However, effective results are not achieved because of determination of same results and errors in automated testing. Therefore, automated testing (java junit testing) code is improved using different methods. It is compared manual, improved automated method and research papers methods using same examples. Finally, it is obtained more successful test paths and test cases from the improved automated testing method according to the manual and research papers testing methods.

Keywords:Model Based Testing, UML Sequence Diagram, Algorithm, Intermediate Model, Depth First Search, JUnit, XML, Test Paths, Test Cases.

(6)

ÖZ

PRATİK MODEL TABANLI YAZILIM TESTİ

Çoşkun, Rengin

Yüksek Lisans, Bilgi Teknolojileri ve Hizmet Yönetimi Bölümü Tez Yöneticisi: Yrd.Doç.Dr. Deepti Mishra

Mayıs 2015, 172 sayfa

Bu tezde, model tabanlı test için, hem otomatik hem de manuel test kullanarak yeni bir yaklaşım üretilir. Amaç, başarılı test durumları bulmaktır. Bu tezde, birçok farklı araştırma makalesiyle çalışıldı ve bir model olarak bu araştırma makalelerinin sıralı diyagram örnekleri kullanıldı. Manuel test için iki yeni algoritma yazıldı. Test yollarını bulmak için, derinlik öncelikli arama yol algoritması kullanılarak yeni bir ağaç tekniği tasarlandı. Otomatik testte, test yollarını otomatik olarak bulmak için, yeni bir java birim test (junit) kodu yazıldı. Hem otomatik, hem de manuel test için, aynı işlem kullanıldı ve ayrıca aynı sıralı diyagram örnekleri ile bu iki metod test edildi. Fakat, aynı sonuçların saptanması ve otomatik testteki hatalardan dolayı, etkili sonuçlar elde edilemedi. Dolayısıyla, farklı metodlar kullanılarak, otomatik test kodu (java birim test) geliştirildi. Aynı örnekler kullanılarak, manuel, gelişmiş otomatik test metodu ve araştırma makalelerindeki metodlar karşılaştırıldı. Sonuçta, araştırma makaleleri ve manuel test metodlarına göre, geliştirilmiş otomatik test metodundan daha başarılı test yolları ve test durumları elde edildi.

Anahtar Kelimeler: Model Tabanlı Test, UML Sıra Diyagramı, Algoritma, Ara Model, Derinlik Öncelikli Arama, JUnit, XML,Test Yolları, Test Durumları.

(7)

To My Parents

(8)

ACKNOWLEDGMENTS

I express sincere appreciation to my supervisor Asst.Prof.Dr. Deepti Mishra for her guidance, help and insight throughout the research. Thanks also, to my parents, I offer sincere thanks for their continuous support and patience during this period.

(9)

TABLE OF CONTENT

ABSTRACT ...iii

ÖZ ...iv

DEDICATION...v

ACKNOWLEDGMENTS ...vi

TABLE OF CONTENTS ...vii

LIST OF TABLES ...xi

LIST OF FIGURES ...xiii

LIST OF ABBREVIATIONS...xv

CHAPTER I. INTRODUCTION ...1

1.1 Statement of the Problem ...1

1.2 Scope and Outline of the Thesis...1

1.3 Software Testing ...3

1.3.1 Software Testing Methods…………...4

1.4 Model Based Testing ...5

1.4.1 Model Based Testing Process ...6

1.4.2 Model Based Testing Advantages and Disadvantages ...8

1.5 Flow Diagram ...9

II. BACKGROUND INFORMATION AND LITERATURE SURVEY ……...10

2.1 Testing Types ………...11

2.1.1. Manual Testing Pros & Cons ……….11

2.1.2. Automated Testing Pros & Cons ………...12

2.2 Test Techniques ………....12

2.2.1 Manual Test Technique ………...13

2.2.1.1 Tree Method ……….13

2.2.1.2 Depth First Search Algorithm ………..14 vii

(10)

2.2.2 Automatic Test Technique ……….15

2.2.2.1 Supporting Tool ………...15

2.3 UML ……….17

2.3.1 Sequence Diagram ………..18

2.3.1.1 Sequence Diagram Elements ………...19

2.3.1.2 Supporting UML Tool………...21

2.4 Automation Level or Complexity ………22

2.5 Test Coverage Criterias ………22

III. PROPOSED MANUAL MODEL BASED TESTING TECHNIQUE & COMPARISON WITH AUTOMATED TESTING………...24

3.1 Proposed Approach ……….24

3.2 Manual Testing ………24

3.2.1 The Single Message Sharing Algorithm for Tree ………..25

3.2.2 Finding Test Paths Algorithm of Tree ………...26

3.3 Automated Testing ………..27

3.4 An Example For Proposed Approach in MBT………..31

3.4.1 Tree & Test Paths For Proposed Manual Testing ……….31

3.4.2 JTable & Test Paths For Proposed Automated Testing ………34

3.4.3 Test Cases Table for Both Manual & Automated Testing …….36

3.5 Testing Results With Examples ………...37

3.5.1 Example1 ………37

3.5.1.1 Manual Testing of Example 1 ………..38

3.5.1.2 Automated Testing of Example 1 ………...39

3.5.2 Example2 ………40

3.5.2.1 Manual Testing of Example 2 ……….41

3.5.3 Example3 ………43

3.5.4 Example4 ………47

3.5.4.2 Automated Testing of Example 4 ………48

3.5.5 Example5 ………50 viii

(11)

3.5.5.1 Manual Testing of Example 5 ………51

3.5.5.2 Automated Testing of Example 5 ………..…………53

3.5.6 Example 6 ………..…………55

3.5.6.1 Manual Testing of Example 6 ………..………..56

3.5.6.2 Automated Testing of Example 6 ..……….…57

3.5.7 Example7 ………...58

3.5.7.2 Automated Testing of Example 7 ………..60

3.5.8 Example 8 ………..61

3.5.9 Example 9 ………..67

3.5.10 Example10 ………...70

3.6 Comparison Table of Manual & Automated Testing ………..78

IV. COMPARISON OF MANUAL MODEL BASED TESTING TECHNIQUE WITH IMPROVED AUTOMATED MODEL BASED TESTING……….80

4.1Comparison of Manual & Automated Testing ………80

4.2 Improved Automated Model Based Testing ………...81

4.2.1 Example 1 ……….81 4.2.2 Example 2 ……….82 4.2.3 Example 3 ……….83 4.2.4 Example 4 ……….84 4.2.5 Example 5 ……….85 4.2.6 Example 6 ……….87 4.2.7 Example 7 ……….87 4.2.8 Example 8 ……….88 4.2.9 Example 9 ……….90 4.2.10 Example 10 ……….91 4.3 Comparison of Proposed Manual vs Automated & Improved Automated

(12)

Testing……….94

4.4 Comparison of Proposed Manual & Improved Automated Testing with Existing Literature………...95

V. CONCLUSION ...97

5.1 Conclusion ...97

5.2 Contributions & Limitations………..99

5.3 Directions for Further Research………...99

REFERENCES ...100

APPENDICES………..103

A. XML Format for Sequence Diagram in Example 1……….103

B. XML Format for Sequence Diagram in Example 2……….107

C. XML Format for Sequence Diagram in Example 3……….112

D. XML Format for Sequence Diagram in Example 4……….117

E. XML Format for Sequence Diagram in Example 5……….121

F. XML Format for Sequence Diagram in Example 6……….128

G. XML Format for Sequence Diagram in Example 7……….132

H. XML Format for Sequence Diagram in Example 8……….135

I. XML Format for Sequence Diagram in Example 9………...146

J. XML Format for Sequence Diagram in Example 10………151

K.XML Format for Sequence Diagram in Figure 20………...158

L.Automatic Testing Junit Code (‘JUnitCode’)………...167

M.ImprovedAutomatic Testing Junit Code (‘JunitCode2’)………....170

(13)

LIST OF TABLES

TABLES

Table 1: Comparison of Most popular Unit Test Frameworks………...17

Table 2:Modeling notations and Structural Model coverage examples as in [22]…23 Table 3: Sequence of All Test Paths in Figure 21………..33

Table 4: List of All Test Paths Name of Figure 21………33

Table 5: Test Cases Table of Example for Manual & Automated Testing…………37

Table 6: Sequence of All Test Paths in Figure 24………..38

Table 7: List of All Test Paths Name of Figure 24………38

Table 8: Test Cases Table of Example1……….40

Table 9: Sequence of All Test Paths in Figure 27………..42

Table 10: List of All Test Paths Name of Figure 27………...42

Table 11: Test Cases Table of Example2………...43

Table 12: Sequence of All Test Paths in Figure 30………45

Table 13: List of All Test Paths Name of Figure 30………..45

Table 14: Test Cases Table of Example 3………..46

Table 16: List of All Test Paths Name of Figure 33………...48

Table 24: Sequence of All Test Paths in Figure 42………59 xi

(14)

Table 35: Test Cases Table of Example 10………78

Table 36: Comparison of Results of Manual & Automatic Testing………..78

Table 37: Comparison Table of All Proposed Testing………..94

Table 38: Comparison Table of All Results………..95

(15)

LIST OF FIGURES

FIGURES

Figure 1: Flow Diagram of the Outline of the thesis ………...3

Figure 2: Software development activities and testing levels-the “V-Model" as in [22]………....4

Figure 3: Model Based Testing Process as in [12]………...7

Figure 4: Taxonomy of Model-Based Testing as in [13]……….8

Figure 5: Illustrates the progress of DFS on the graph as in [14]………..15

Figure 6: UML Diagrams as in [19]………...18

Figure 7: Lifeline of UML Sequence Diagram………...19

Figure 8: Synchronous message of UML Sequence Diagram………19

Figure 9: Asynchronous message of UML Sequence Diagram………..19

Figure 10: Return message of UML Sequence Diagram………20

Figure 11: Instantaneous message of UML Sequence Diagram……….20

Figure 12: Self message of UML Sequence Diagram………20

Figure 13: Interaction Fragments of UML Sequence Diagram [20]………..21

Figure 14: General Flow Diagram of Tree Testing Process………...25

Figure 15: Generate The Single Message Sharing Tree Algorithm………...26

Figure 16: Generate Distribution of The Message Path………26

Figure 17: General Flow Diagram of JUnit Testing Process……….28

Figure 18: Example of a well-formed XML File………29

Figure 19: ‘Table’ java code for JTable………..30

Figure 20: Example Sequence Diagram of the Hospital Management System as in [24]………...31

Figure 21: The Single Message Sharing Tree (SMST) in Figure 20………...32

Figure 22: JTable of XML File in Appendix K………..34 xiii

(16)

Figure 23: Sequence Diagram of Banking System [3]………...37

Figure 24: The Single Message Sharing Tree (SMST) of Figure23………38

Figure 25: JTable of XML File in Appendix A………..39

Figure 26: Sequence Diagram of Bank ATM System (Withdraw Money) [1]……...41

Figure 27: The Single Message Sharing Tree (SMST) of Figure 26………...41

Figure 28: JTable of XML File in Appendix B………..42

Figure 29: The Sequence Diagram for Deal Cards Scenario [10]………..44

Figure 31: JTable of XML File in Appendix C………..45

Figure 32: Sequence Diagram for Medical Consultation System [9]……….47

Figure 34: JTable of XML File in Appendix D………..48

Figure 35: Sequence Diagram of FB Authorization [2]………..50

Figure 37: JTable of XML File in Appendix E………..53

Figure 38: Sequence diagram for ATM Banking System [5]………56

Figure 39: The Single Message Sharing Tree (SMST) of Figure 38………..56

Figure 40: JTable of XML File in Appendix F……….57

Figure 41: Sequence Diagram for Bookstore Composite Web Service [6]………..58

Figure 42: The Single Message Sharing Tree (SMST) of Figure 41……….59

Figure 43: JTable of XML File in Appendix G………60

Figure 44: Sequence Diagram of Library Management System [4]……….61

Figure 46: JTable of XML File in Appendix H………64

Figure 47: Sequence diagram for Withdrawal Usecase [7]………..67

Figure 49: JTable of XML File in Appendix I……….69

Figure 50: Sequence Diagram for Operation getRegistered() [8]………71

Figure 51: The Single Message Sharing Tree (SMST) of Figure 50………72

Figure 52: JTable of XML File in Appendix J………75

(17)

LIST OF ABBREVIATIONS

MBT - Model Based Testing

UML - Unified Modeling Language

DFS - Depth First Search Algorithm

BFS - Breath First Search Algorithm

JUnit - Java Unit Testing

GA - Genetic Algoritm

SG - Sequence Graph

SDG - Sequence Dependency Graph

EFSM - Extended Finite State Machine

SDT - Sequence Dependency Table

SUT - System Under Testing

SQL - Structured Query Language

XML - Extensible Markup Language

GUI - Graphical User Interface

(18)

1

CHAPTER I

INTRODUCTION

1.1Statement of the Problem

Software tests have an increasingly effective and important role in terms of whole system in software systems. Also, it is necessary in terms of some criterias of the system such as successful, quality, cost, availability etc. The problem of the thesis, large systems has very complex structures and also finding of test cases based on a model are a difficult step in testing. Therefore, finding of test cases need more time and effort. Tools are very practical to find test cases in automated methods. Also, automated testing can be done by improving source code for software testing. Although manual testing method has some advantages, this method is more difficult than automated methods. Also, manual testing is more risky than automated testing. However, researchers prefer manual or automated testing methods according to their studies.

1.2Scope and Outline of the Thesis

The aim of this thesis is to propose a solution for the above problem. There are lots of approaches and studies about this issue. Many researchers propose a new method or an algorithm, and find test cases of the system with their methods and algorithms. On the other hand, some researchers use tools that are existing. The purpose of this thesis is to support MBT with a new technique and to prove which testing method is better. In this thesis, it is used two general testing methods with a new approach. Both manual and automated method are used for finding good test paths and also, test cases generation. In manual testing, it is produced a new algorithm based on a sequence diagram model. Then, a new technique is proposed according to the new

(19)

2

algorithm. In automated testing, it is used the same operation with manual testing because of showing whether test paths results same or different in terms of both manual and automated testing. The aim is whether test paths results are the same or test paths results include errors, manuel or automated testing should be improved.

In chapter 1, what model based testing is and how this testing is used with UML

sequence diagram are explained. Also, scope and statement of the thesis are defined. It will be mentioned about two general methods that are manual and automated testing in MBT. It will be explained the produced technique or method, algorithms, codes and tools in chapter 2. In chapter 3, it is shown which methods will be used and how the methods will be used. Furthermore, it will be explained test paths and test cases of the proposed approach with examples. In chapter 4, the proposed approach will be improved, and the proposed approach results will be compared with manual method results and research papers methods results. It is proved the success of improved test methods results. The final step chapter 5 is the conclusion of the thesis. It will be explained all results of the thesis.

(20)

3

Figure 1: Flow Diagram of the Outline of the thesis

1.3 Software Testing

Software testing is a testing model of system under testing. System under testing is a phase of maturity of the software testing for correct operations. Software testing executes all process of the whole system and also this testing provides finding errors. Software testing is based on validation and verification processes. The validation process involves requirement documents which describe what the software is supposed to do, whereas, the verification process determines whether the software is correct with regard to its design. [22]

(21)

4

1.3.1 Software Testing Methods

Software testing methods can be categorized as level of testing and method of testing.

 The level of Testing [22]

 Function/Unit testing is a unit program code of an software system.

 Module testing is whole modules or components of software system.

 Integration testing assesses the software about its subsystem design.

 System testing is an architectural design testing of the complete system.

 Acceptance testing is measurement of the system satisfies performed. Acceptance criterias that operational, process and business requirements are very important for this testing.

Figure 2: Software development activities and testing levels - the “V-Model"

as in [22]

 The method of Testing

 White-box testing is the actual source code for software. Tests design code of system with various algorithms and some structures of the source code. This type of testing is also referred to as structural testing. As in [22]

 Black-box testing is a system-under-test that is considered as inputs and the outputs of the system. Also, this testing is referred to as functional testing. As in [22]

(22)

5

1.4Model Based Testing

The model-based testing is defined as a kind of test application where test cases are generated based on a model to perform system testing. Model-based testing is a one of the software testing where test cases are derived from a model as whole or part. This testing can describe some functional aspects of the system under test (SUT). The test cases derived from models are a process of functional and structural tests of the model. MBT is a black box testing method that test cases are occured automatically using models. In other words, model based testing is a behavior model. The behavioral model specifies system requirements or conditions with the system under testing. This behavior models include requirements of system such as inputs, outputs and condition of the system ( pre-condition and post-condition). A behavior model explains test cases in a model based testing.

According to [23], there are four main approaches known as model-based testing: 1. Generating of test input data from a domain model

2. Generating of test cases from an environment model

3. Generating of test cases with oracles from a behaviour model 4. Generating of test scripts from abstract test

Generating of test input data from a domain model is related to input data of model in model-based testing. The test input information involves combination of a subset of input data values to produce test values.

Generating of test cases from an environment model uses a different model to describe the expected environment. These environment models can generate sequences with the system under test. This environment models can not predict the output values because the environment models are not behaviour models. Therfore, these environment models are very difficult to specify whether any testing has passed or failed.

Generating of test cases with oracles from a behaviour model includes oracle information for executable test cases. These are expected output values or automated check on the actual output values of the system under testing. This model is very complex models of model based testing. This models has feedback, however do not check the results. This model should identify behaviour relationship between inputs and outputs of the system under testing.

Generating of test scripts from abstract test gives abstract description of a test case, such as a UML sequence diagram, and this testing is related to transforming of

(23)

6

abstract test case for low-level test script. In this approach, the model includes information about the structure and API (Application Programming Interface) of the system under testing.

1.4.1 Model Based Testing Process

Figure 3 shows steps of MBT processes. According to [22], MBT includes five processes.

The first step is related to phases of model. A model can be an environment of the system-under test, and it is not complex than the actual implementation. The model includes requirements, design specifications or specification documents and test plan of the system.

The second step is test case generation from abstract models. This test cases is related to key aspects for testing and also many details. Also, this process follows model coverage criteria and keeps requirements for the test cases.

The third step is called implementation of tests. This step focuses on transformation of the abstract test cases with various templates and mappings. This process is supported with tools by programmers.

The fourth step is related to test execution and it needs to be adapted to execute. This process is created by using a tool commonly called as test adapter. The adapter adds specific information to the tests and tests can be executed either in online or in offline mode. This test results are directed by test adapter tool.

The fifth step is the analysis phases of the test results. This testing phase is similar to traditional test analysis. Also, this testing phase is related to test case failure and a fault in the system.

(24)

7

(25)

8

Figure 4: Taxonomy of Model-Based Testing as in [13]

1.4.2 Model Based Testing Advantages and Disadvantages

Advantages of MBT

 MBT provides high level automation for testing.

 MBT provides improvement of product quality in testing system. Also, MBT is a systematic testing to help understanding the behavior of the system. MBT provides improving quality with this property.

 Detail testing can find test cases with MBT.

 Test case changing can be managed and followed with MBT.

 MBT provides direct connection with the model so, model requirements can be updated.

 MBT provides high effective test covearge, and this testing is applied for all the functionalities of the system.

(26)

9

 MBT reduces cost of testing. This testing includes fault detection and finding error, so cost reduces as automatically.

 MBT reduces time of testing, and avoids from the system complexity. From this property, it can be reached testing results shortly with MBT.

 MBT provides traceability for model requirements and test cases.

Disadvantages of MBT

 If model complexity increases, then MBT becomes more difficult.

 In MBT, adequate skill is necessary to construct an effective model for abstract and design model.

 If any changing occurs in the model, then a different test set can occur.

 MBT can not quarantee to find test cases differences from the model.

1.5Flow Diagram

Flowchart or flow diagram is a diagram that uses graphic symbols, represents algorithms, and steps of all process order by their connecting each other. In this thesis, it is used flow diagram to represent all process with the proposed approach. Flowchart is an easy diagram in terms of both drawing and understanding. It is tried to show all steps of the system in general with this flowchart.

(27)

10

CHAPTER II

BACKGROUND INFORMATION AND LITERATURE SURVEY

Testing has a significance role for software systems. Large systems of software have very complex structures, and finding test cases are a difficult step for testing. Finding test cases is a very difficult process in terms of time, effort and cost. Many researchers use lots of different approaches, and they use different methods and algorithms in model based testing as manual or automatic technique. Besides, researchers prefer existing tools in some studies. Tools have many good properties because they run automatically. Although manual testing is more difficult than automated testing, many researchers prefer manual testing for their studies. The most important reason of this situation is that many tools can not meet the requirements of the studies.

In chapter 2, it will be given some information about contents of this thesis. Manual and automated testing are defined as test automation in MBT. Both manual and automated testing are included in this study. Also, it is explained techniques, algorithms and tools in this approach. UML sequence diagram is used as a model in MBT and it is worked with sequence diagram in different areas. UML Sequence diagrams are very common models in terms of model based testing approach. Sequence diagrams in MBT provide test path sequences with various intermediate models, graphs, tables and flow graphs. An another method for sequence diagrams in MBT is that various algorithms provide generation of test cases with unit testing. Sequence diagrams can transform from model-to-model. xUnit is a collective name of unit testing frameworks such as JUnit, SUnit etc. Test cases can find with these test code methods. Furthermore, complexity and test coverage criteria of test cases can be defined with MBT.

(28)

11

2.1 Testing Types

Manual and automated tests are the most common testing types in the software. These testing types cover some specific testing methods such as black box testing, white box testing, integration testing, system testing, performance testing, load testing, unit testing and acceptance testing. Some of these testing are very suitable for manual testing, but some testing are better for automated testing. Manual testing is executed by human without using any automated tools or scripts. A test plan document or test scenario is prepared by testers for manual testing. Automated

testing is executed with automated tools, scripts and software. Testers can use a

suitable programming language for system and also, testers can improve system for any operation in automated testing. Manual and automated tests have some pros and cons.

2.1.1. Manual Testing Pros & Cons PROS

1. The most important feature is flexibility.

2. More likely to find real user issues [15]

3. Short-term cost is lower. [15]

4. Manual testing will be useful when the test case only needs to run once or twice [17]

5. To execute the test cases every time tester requires the same amount of time [17]

6. Manual tests are very much helpful in UI testing [17]

7. To execute the Build Verification Testing (BVT) is very mundane and tiresome in manual testing [17]

8. To execute the test cases first time using manual testing will be very much useful. But it is not sure that it will catch the regression defects under frequently changing requirements [17]

CONS

1. Certain tasks are difficult to do manually [15]

2. Manual tests are unstimulating.

3. Can’t reuse manual tests [15] 4. Manual tests areless reliable.

5. Non- programmable [16]

(29)

12

7. Using manual testing,testing on different machine with different OS platform combination is not possible, concurrently. To execute such task different testers are required [17]

2.1.2. Automated Testing Pros & Cons PROS

1. Automated testing ismore reliable.

2. Automated tests are veryfast.

3. Less investment in human resources [16]

4. Programmable [16]

5. Canexecute easily and effectively.

6. Can be cost effective [15]

7. More interesting [15]

8. Results can see quickly.

9. Automation testing will be useful to execute the set of test cases frequently [17]

10. After making Automation test suites, fewer testers required to execute the test cases [17]

11. Automation testing can also be done on different machine with different OS platform combination, concurrently [17]

12. Some time it is not helpful in UI testing [17]

13. Automation testing is very useful for automating the Build Verification Testing (BVT) & it is not mundane and tiresome [17]

14. Automation testing is very much helpful regressions in testing where code changes frequently [17]

CONS

1. Cost oftools can be expensive.

2. Use oftools can take time consuming.

3. Tools can include various limitations.

2.2 Test Techniques

It is used two test techniques in this thesis. First technique is manual testing and second technique is automatic testing in the MBT. These techniques are explained with following subtitles.

(30)

13

2.2.1 Manual Test Technique

In manual testing, it is preferred intermediate model such as formal models, graphs, trees, and other hybrid types to create test case generation. These intermediate model approaches are broadly classified into four major categories: specification based testing, graph theoretic testing, heuristic testing, and direct UML specification processing. [18] However, it is used only tree approach among intermediate model structures in this paper. Tree model approach depends on graph theoretic testing methods.

Graph theory is an area of mathematics that helps deriving test cases from design models in many different ways. [18] Graphs are the structures that are used the most for abstract test cases. UML models represent graphical structures of a system properties. However, UML models are not used directly to create test cases so, UML models need some intermediate models such as trees and graphs for test case generation.

In this study, two new algorithms are proposed using UML sequence diagram model for manual test technique in MBT. First algorithm is necessary to develop a tree. The tree is created from the first algorithm using UML sequence diagram. Then, test paths are read from the tree using second algorithm. This second algorithm includes DFS algorithm. The reasons for choosing of DFS algorithms is that this algorithm focuses on to generate a set of test scenarios or basic test paths. Finally, it is derived a set of test paths and also test cases or scenarios from this test method.

2.2.1.1Tree Method

A tree is a connected acyclic simple graph. An alternate sequence of vertices and edges beginning at the root and ending in a leaf node is of interest for a scenario representation. [18]

Lots of researhers benefit from graphs, but it is used a tree technique instead of graph. Also, trees are an undirected graphs. Every tree consists of nodes and edges. In this study, it is preferred tree, becuase tree is a very practical and easy method in terms of manual testing. Also, this method is very available to find test paths immediatelly.

(31)

14

 Theorem:

Tree is T = (V, E)

 V = nodes. (number of message)

 E = edges between pairs of nodes. (message dependency)

 Captures messages relationship between objects. Let T be an tree on n nodes that message numbers.

 T is connected.

 T does not contain a cycle.

2.2.1.2 Depth First Search Algorithm

In manual testing, an algorithm is generated to find test paths in test case generation. The algorithm includes also DFS algorithm. It is preferred DFS algorithm, because DFS algorithm is very practical and common for tree tehnique in the software engineering. The other reason of using DFS algorithm is that DFS algorithm is more suitable than other existing algorithms for the proposed tree technique.

Depth first search is one of the simplest algorithms for searching a graph, and this algorithm uses also for tree structures. This algorithm controls all neighboor of a node, and provides searching deeper. DFS discovers all vertices. If any undiscovered vertices remain, then depth-first search selects one of them as a new source, and it repeats the search from that source. The algorithm repeats this entire process until it has discovered every vertex.[14]

(32)

15

Figure 5: Illustrates the progress of DFS on the graph as in [14]

2.2.2 Automatic Test Technique

In this thesis, it is used some tools to generate set of test paths for automatic test technique in MBT. It is used java as programming language. Therefore, new codes in eclipse are written. Also, junit for testing is used. Junit is better than the other unit testing frameworks so, junit is preferred.

2.2.2.1 Supporting Tool

It is used some open source tools for this approach in MBT. Especially, these tools are necessary for automatic testing. An integrated system is used for automatic testing. It is prefered java as programming language because java is a very common, usable and preferable programming language in the software engineering. In this thesis, it is used eclipse for java language and also, plug-in tools for eclipse platform is used. For automated testing, test paths and test cases are generated using unit testing.

(33)

16

Eclipse is a very common, easy and useful tool for software engineering. It is a unique model for Open Source Development. Also, the programming language of eclipse is Java, C and C++ so, many users prefer eclipse with this feature. The Eclipse is a non-commercial tool. Also, this tool can be integrated with other tools easily. Eclipse can run both Windows and Unix platforms.

 Junit Testing Features

Unit testing is used for automatic testing. JUnit is an open source framework for java programming language. Junit is an automatic language that write and run. Junit is a unit test case part for java code. Also, junit provides achievment of all desired test results quickly. Unit test case includes two test cases results as positive and negative. These two test cases results provides usable code for testing requirements. xUnit is general name of various unit testing frameworks. It is compared most popular unit testing framewors in the following Table 1. It is prefered junit testing as the most popular unit testing. Junit is better than the other unit testing frameworks because of its good properties. Also, junit provides an automatic testing for developing codes and applications. It is very useful and easy for many programmers. It provides easy understanding of code, and it is very fast for code development. Therefore, it does not cause loss of time. JUnit reduces the error rate in the code, increases code quality, and detects errors quickly. Errors are found very easy and fast in automated codes or systems. Also, junit has porting properties in software engineering. Porting is a process of adapting software. Porting is an adapting of an executable program to different environments for software / hardware.

(34)

17

Unit Test Framework

Name

JUnit NUnit CppUnit PHPUnit

xUnit    

Automated Run    

Atomic Result

PASS/FAIL

   

Language Java C# / .Net C++ PHP

Integrated    

Open Source    

Common    

Port / Based on - Junit Junit Junit

License Eclipse Public License

Zlib License BSD LGPL

Table 1: Comparison of Most popular Unit Test Frameworks

2.3 UML

The Unified Modelling Language (UML) is a visual modelling and also a general programming language of the software system. In 1994-1995, Grady Booch, Ivar Jacobson and James Rumbaugh was developed this modeling language. UML was designed to incorporate current best practice in modelling technologies and software engineering. [23] This modeling language includes various behavior and structure diagrams.

 Behavioral diagrams represent specific processes of a system. And also,

behavioral diagrams are very common diagrams. These diagrams are called as activity, use case, state machine, and interaction diagrams.

 Interactional diagrams are more than one diagram. These diagrams

includes a subset of behavioral diagrams. These diagrams are called as sequence, communication, interaction overview, timing diagrams.

(35)

18

Figure 6: UML Diagrams as in [19]

2.3.1 Sequence Diagram

A sequence diagram is an interaction diagrams of UML behavior diagrams. This diagrams consists of a specific number objects and messages. Sequence diagram is shown with vertical lines and these lines consist of objects of the diagram. Sequence diagram messages are among objects, and they act according to a sequence. Sequence diagrams are based on a scenario. Every message is send or received according to the scenario.

In MBT with UML model use various diagrams such as UML activity, use case, sequence diagrams, etc. Activity, sequence and state-machine diagrams are commonly used for MBT.

In this study, it is used sequence diagram as UML diagram for MBT because sequence diagrams are very common in UML behavior models. Sequence diagram is very suitable for scenario-based testing in MBT. In scenario-based testing, a scenario is an abstract test case. Scenario-based testing tests are usually different from test cases in that test cases are single steps and scenarios cover a number of steps. [11] Scenarios are useful to use in terms of model test requirements and test cases. Also, this testing works the best in terms of very complex system or events. In MBT, using sequence diagram can give more effective results to find test cases according to other diagrams.

(36)

19

2.3.1.1 Sequence Diagram Elements

 Object : Objects defines classes of a sequence diagram, and they shows dependency

of messages.

Figure 7: Lifeline ofUML Sequence Diagram

 Messages : A message has a name and a parameter in a sequence diagram. Messages

consist of various names such as synchronous messages, asynchronous messages, return messages, instantaneous messages and self messages.

Figure 8: Synchronous message ofUML Sequence Diagram

A synchronous message sends message between sender object and receiver object. The sender object waits to finish processing message of the receiver object.

(37)

20

The sender object does not wait to finish processing message of the receiver object in

an asynchronous message. An asynchronous message does not have a return message.

Figure 10: Return message ofUML Sequence Diagram

Return messages are solutions of sending messages and they depends on an optional.

Figure 11: Instantaneous message ofUML Sequence Diagram

The sending message arrives to receiving message, but the time is negligible in the instantaneous mesages.

Figure 12: Self message ofUML Sequence Diagram

An object sends message to itself that called self message.

 Interaction Fragment (Conditional Interaction) : The following table defines

(38)

21

Figure 13: Interaction Fragments ofUML Sequence Diagram [20]

In this thesis, it is studied with UML sequnce diagram models in model based software testing. It is studied with sequence diagrams in different areas because the aim is to achieve a different perspective in terms of test results. It is shown different results for both manual and automated testing using models in different areas. Also, according to research papers, it is achieved more power results of this approach with different sequence diagram in MBT. In this thesis, sequence diagram topics are ‘Banking System’, ‘Bank ATM System (Withdraw Money)’, ‘Deal Cards Scenario’, ‘Medical Consultation System’, ‘FaceBook Authorization’, ‘Bookstore Composite Web Service’, ‘Library Management System’, ‘Operation getRegistered()’.

2.3.1.2Supporting UML Tool

It is chosen Umlet as UML tool. Umlet has some good features such as free or open source, easy to use, multiple diagram, export diagram or file. This tool can be integrated and it has an easy GUI. This tool is a UML plug-in tool. Also, it is appropriate for UML 2 Graphical Model.

It is preferred Umlet UML tool instead of other UML tools. Umlet UML tool can be integrated easily with Eclipse. It is studied with XML files in this thesis. UML diagrams can be converted and read XML file with Umlet easily. This property provides very usable XML files. Umlet is very flexible to find test paths of a diagram with this property.

(39)

22

2.4 Automation Level or Complexity

Automation level or complexity measures and analyses rate of easiness and difficulty of the system. Automation levels change according to manual or automated systems. According to [21], In this thesis, automation level complexity criteria means:

•Low (L): Low level representsautomated or involve a simple task. Tool is

available.

•Medium (M) : Medium level represents intermediate model modeling or test data

definition.

• High (H): High level represents manual translation between models. Tool does not

support.

This thesis covers high, medium and low automation levels in MBT. The proposed manual testing covers high and medium complexity criterias because, manual testing contains intermediate model, and it does not use tool. The proposed automated testing contains low complexity criteria because it includes automatic running code. Also, this automated testing uses a tool.

2.5 Test Coverage Criterias

In MBT, models have different structural notations. These notations include structural model coverage criterias. These criteria specify the selection of tests from models in terms of coverage of the structural elements of the models. [22]

In this studies, it is used history-based model notation both manual and automated testing. This model notation includes all messages and message path coverage as structural model coverage. This criteria is evaluated all message for testing from a model. This approach is scenario-based testing and this approach supports functional system testing depending on inputs and outputs of the system. The following Table 2 shows all notations of model andstructural model coverage in MBT.

(40)

23

(41)

24

CHAPTER III

PROPOSED MANUAL MODEL BASED TESTING TECHNIQUE

& COMPARISON WITH AUTOMATED TESTING

3.1 Proposed Approach

Many researchers have produced many approaches for MBT. These studies are different from each other in terms of techniques and algorithms. Some researhers prefer using tool that using automated testing, and some researhers prefer using their algorithms with manual testing to achieve test paths in MBT. Also, some researhers prefer using their own methods. They write their own automated codes using tools and programming language, or they develop their own algorithms using manual methods. The proposed approach is based on scenario-based testing. It is designed a novel approach with a new system using UML sequence diagram models for MBT, and two testing methods are used as manually and automatically for testing in this thesis.

3.2 Manual Testing

In this thesis, it is designed a new technique and algorithm as manually for model based software testing. The following subtitles explain the proposed technique and algorithms. A new technique that is called ‘The Single Message Sharing Tree’ is developed for sequence diagram message system, and two algorithms for this system is written. The first algorithm that is called ‘The Single Message Sharing Tree Algorithm’ explains what message tree is and how a message tree is created. Second algorithm that is called as ‘Distribution of The Message Path Algorithm’ includes DFS path algorithm to read all test paths. Finally, all test paths for all test cases are reached in this thesis. In summary, desired results of test cases of the system are achieved.

(42)

25

The whole system is defined using a flow diagram. The following flow diagram shows all process of proposed manual testing.

Figure 14: General Flow Diagram of Tree Testing Process

3.2.1 The Single Message Sharing Algorithm for Tree

UML sequence diagram is used as a model for the system. This model is a. A new approach using messages of the sequence diagram is produced. The proposed technique is a tree. A road algorithm is written that is called as ‘Algorithm 1’ and then, the tree is generated according to algorithm 1. Algorithm 1 explains steps of tree. This algorithm generates nodes of tree. Also, all test paths of the tree are reached with this algorithm. Every step explains node dependency of tree according to messages of sequence diagram.

(43)

26

Figure 15: Generate The Single Message Sharing Tree Algorithm

3.2.2 Finding Test Paths Algorithm of Tree

The following algorithm is ‘Algorithm 2’ that finds all test paths of the tree. Also, algorithm 2 benefits from depth first search path algorithm.

(44)

27

3.3 Automated Testing

In this study, a second technique is designed as automatically for model based software testing. The following subtitles explain this second technique. First of all, a sequence diagram is drawn using UMLet tool inside eclipse, and sequence diagram is converted to XML file using eclipse. Secondly, a new algorithm that is called as ‘Table’ is written using java code to show all test data. All data of sequence diagram is extracted from an XML file. After then, all data is stored into a jtable with this ‘Table’ code. Thirdly, a junit unit testing code that is called as ‘JUnitCode’ is written for testing of all data. The junit testing code benefits from jtable. Furthermore, this code provides to find and show all test paths of sequence diagram in the java console. Finally, all test cases with these test paths are reached in this thesis.

The whole system is defined using a flow diagram. The following flow diagram Figure 17 shows all process of proposed automated testing. In Figure 18, well-formed of all XML files are shown. This form shows structure of all xml files.

(45)

28

(46)

29

Figure 18: Example of a well-formed XML File

Figure 19 shows the ‘Table’ java code. It is used parser method to convert from XML file to jtable in the ‘Table’ java code. The XML file is in Appendix K. In this study, java jtable method is used to extract all necessary data of sequence diagram, and all data is stored into a jtable with this code.

In chapter 3, a new testing code is called as ‘JUnitCode’ is in Appendix L. The purpose is to find all test scenario cases of sequence diagram from test paths with junit testing. This junit testing code incudes some test methods for test requirements. First method is testPossibleTestPaths() method. This method is necessary to find all possible test path sequence. Second method is called testAllPaths() to read names of sequence diagram messages according to test paths. Third method is called testReadMessages() to show all messages and also number of message of sequence diagram. Final method is called testBestPath() to show best test path of the sequence diagram. All necessary test path of the sequence diagram are found using jtable data with junit testing code in proposed automated test method.

(47)

30

(48)

31

3.4 An Example For Proposed Approach in MBT

The proposed manual and automated test method are explained with a good example. UML sequence diagram in Figure 20 is used as a model. This model includes a hospital management system. The following parts explain testing results of the proposed manual and automated methods.

Figure 20: ExampleSequence Diagram of the Hospital Management System

as in [24]

3.4.1Tree & Test Paths For Proposed Manual Testing

In Figure 21 shows the proposed tree. Tree starts root node, and connects nodes to edges from start message to end message respectively. However, after every RM, a new node from SM opens (for example; if 5. message is SM & 6. message is RM

(49)

32

node, a new node from SM opens for 7. message) and continues. RM node is called return message. Also, the final node is called the final message. Table 3 and 4 show all possible test paths from the tree. The purpose is to evaluate all sending and return messages. Figure 21 is the proposed message tree as a technique, and Algorithm 1 is the algorithm of this tree technique.

(50)

33

 Test Paths Path ID Test Paths

1 1-2-3-4-5-6 2 1-2-3-4-5-7 3 1-2-3-4-5-8-9 4 1-2-3-4-5-8-10 5 1-2-3-4-5-8-11-12-13-14 6 1-2-3-4-5-8-11-12-13-15-16-17-18-19 7 1-2-3-4-5-8-11-12-13-15-16-17-18-20-21 8 1-2-3-4-5-8-11-12-13-15-16-17-18-20-22-23-24-25

Table 3: Sequence ofAll Test Paths in Figure 21

Path ID Test Paths

1 Prescribe Medicines-Precribe Tests-CheckReports-RequestForRoom-Check Room Availability-ReturnStatus

2 Prescribe Medicines-Precribe Tests-CheckReports-RequestForRoom-Check Room Availability-ReturnStatus

3 Prescribe Medicines-Precribe Tests-CheckReports-RequestForRoom-Check Room Availability-If Available BookRoom-Return RoomNo

4 Prescribe Medicines-Precribe Tests-CheckReports-RequestForRoom-Check Room Availability-If Available BookRoom-Return RoomNo

5 Prescribe Medicines-Precribe Tests-CheckReports-RequestForRoom-Check Room Availability-If Available BookRoom-MaintainPatientDetails-Generate Bills-PayBills-Return DischargeSummary

6 Prescribe Medicines-Precribe Tests-CheckReports-RequestForRoom-Check Room Availability-If Available BookRoom-MaintainPatientDetails-GenerateBills-PayBills-AddPerson-DrawSalary-Return-DrawSalary-Return

7 Prescribe Medicines-Precribe Tests-CheckReports-RequestForRoom-Check Room Availability-If Available BookRoom-MaintainPatientDetails-Generate Bills-PayBills-AddPerson-DrawSalary-Return-DrawSalary-DrawSalary-Return

8 PrescribeMedicines-PrecribeTests-CheckReports-RequestForRoom-CheckRoom Availability-IfAvailableBookRoom-MaintainPatientDetails-GenerateBills-PayBills-AddDrawSalary-Return-DrawSalary-DrawSalary-EditPerson -Delete Person-Add Doctor-Delete Doctor

(51)

34

3.4.2JTable & Test Paths For Proposed Automated Testing

All necessary elements of XML file are loaded into jtable with ‘Table’ code. Jtable includes all necessary information of sequence diagram for test paths in Figure 22. Also, the following part shows all test paths results of proposed junit automated testing screen results in ‘JUnitCode’ code.

Figure 22: JTable of XML File in Appendix K

 All Results for Test Paths

TEST PATHS

All Possible Test Paths... 1. [1 - 2 - 3 - 4 - 5 - 6] 2. [1 - 2 - 3 - 4 - 5 - 6 - 7] 3. [1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9] 4. [1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 - 10] 5. [1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 - 10 - 11 - 12 - 13 - 14] 6. [1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 - 10 - 11 - 12 - 13 - 14 - 15 - 16 - 17 - 18 - 19]

(52)

35

7. [1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 - 10 - 11 - 12 - 13 - 14 - 15 - 16 - 17 - 18 - 19 - 20 - 21]

8. [1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 - 10 - 11 - 12 - 13 - 14 - 15 - 16 - 17 - 18 - 19 - 20 - 21 - 22 - 23 - 24 - 25]

1. [1:PrescribeMedicines() - 2:PrescribeTests() - 3:CheckReports() 4:RequestForRoom() - 5:CheckRoomAvailability() - 6:ReturnStatus] 2. [1:PrescribeMedicines() - 2:PrescribeTests() - 3:CheckReports() -

4:RequestForRoom() - 5:CheckRoomAvailability() - 6:ReturnStatus - 7:ReturnStatus] 3. [1:PrescribeMedicines() - 2:PrescribeTests() - 3:CheckReports() -

4:RequestForRoom() - 5:CheckRoomAvailability() - 6:ReturnStatus - 7:ReturnStatus - 8:IfAvailableBookRoom() - 9:ReturnRoomNo]

4. [1:PrescribeMedicines() - 2:PrescribeTests() - 3:CheckReports() -

4:RequestForRoom() - 5:CheckRoomAvailability() - 6:ReturnStatus - 7:ReturnStatus - 8:IfAvailableBookRoom() - 9:ReturnRoomNo - 10:ReturnRoomNo]

4:RequestForRoom() - 5:CheckRoomAvailability() - 6:ReturnStatus - 7:ReturnStatus - 8:IfAvailableBookRoom() - 9:ReturnRoomNo - 10:ReturnRoomNo -

11:MaintainPatientDetails() - 12:GenerateBills() - 13:GenerateBills() - 14:ReturnDischargeSummary]

11:MaintainPatientDetails() - 12:GenerateBills() - 13:GenerateBills() - 14:ReturnDischargeSummary - 15:AddPerson() - 16:DrawSalary() - 17:Return() - 18:DrawSalary() - 19:Return]

11:MaintainPatientDetails() - 12:GenerateBills() - 13:GenerateBills() - 14:ReturnDischargeSummary - 15:AddPerson() - 16:DrawSalary() - 17:Return() - 18:DrawSalary() - 19:Return - 20:DrawSalary() - 21:Return()]

11:MaintainPatientDetails() - 12:GenerateBills() - 13:GenerateBills() - 14:ReturnDischargeSummary - 15:AddPerson() - 16:DrawSalary() - 17:Return() - 18:DrawSalary() - 19:Return - 20:DrawSalary() - 21:Return() - 22:EditPerson() - 23:DeletePerson() - 24:AddDoctor() - 25:DeleteDoctor()]

BEST PATH

The Best Path: 8. Path [1:PrescribeMedicines() - 2:PrescribeTests() -

3:CheckReports() - 4:RequestForRoom() - 5:CheckRoomAvailability() - 6:ReturnStatus - 7:ReturnStatus - 8:IfAvailableBookRoom() - 9:ReturnRoomNo - 10:ReturnRoomNo - 11:MaintainPatientDetails() - 12:GenerateBills() - 13:GenerateBills() -

14:ReturnDischargeSummary - 15:AddPerson() - 16:DrawSalary() - 17:Return() - 18:DrawSalary() - 19:Return - 20:DrawSalary() - 21:Return() - 22:EditPerson() - 23:DeletePerson() - 24:AddDoctor() - 25:DeleteDoctor()]

ALL MESSAGES

All messages of sequence diagram: [1:PrescribeMedicines() -> 2:PrescribeTests() -> 3:CheckReports() -> 4:RequestForRoom() -> 5:CheckRoomAvailability() -> 6:Return Status -> 7:ReturnStatus -> 8:IfAvailableBookRoom() -> 9:ReturnRoomNo -> 10:Return RoomNo > 11:MaintainPatientDetails() > 12:GenerateBills() > 13:GenerateBills() -> 14:ReturnDischargeSummary --> 15:AddPerson() --> 16:DrawSalary () --> 17:Return() --> 18:DrawSalary() > 19:Return > 20:DrawSalary() > 21:Return() > 22:EditPerson() -> 23:DeletePerson() --> 24:AddDoctor() --> 25:DeleteDoctor()]

(53)

36

3.4.3 Test Cases Table for Both Manual & Automated Testing

The following table is a test cases table for all requirements of both manual and automated testing. The proposed test cases table includes test case id, test case sequence, activity name id, test case inputs and test case expected and actual outputs for sequence diagram. Test case sequence defines order of all test paths until the last message for each test case. Test case input defines all messages of sequence diagram. Test case output defines both expected and actual results for sequence diagram. Expected output is proposed outputs. Also, actual output is return messages for each input message. Test Case ID Test Case Sequence Test Case Activity Name ID

Test Case Activity Name (INPUT Messages) Test Case Expected OUTPUT Test Case Actual OUTPUT 1 1-2-3-4-5-6

1 Prescribe Medicines Treatment Results - 2 Prescribe Tests Test Results - 3 CheckReports Check Report -

4 RequestForRoom Return Status Return Status 5 CheckRoomAvaiability Return Status Return Status 6 Return Status

2 1-2-3-4-5-7 7 Return Status

3 1-2-3-4-5-8-9 8 IfAvailableBookRoom Return Room No Return Room No 9 Return Room No 4 1-2-3-4-5-8-10 10 Return Room No 5 1-2-3-4-5-8-11-12-13-14 11 MaintainPatientDetails Receptionist Control - 12 GenerateBills Bill

-13 PayBills Return Discharge Summary Return Discharge Summary 14 Return Discharge Summary 6 1-2-3-4-5-8-11- 12-13-15-16-17-18-19

15 AddPerson Record Patient Info -

16 DrawSalary Return Return

17 Return

19 Return

7

1-2-3-4-5-8-11- 12-13-15-16-17-18-20-21

(54)

37 8 1-2-3-4-5-8-11- 12-13-15-16-17- 18-20-22-23-24-25

22 EditPerson Record Patient Info - 23 DeletePerson Delete Patient Info - 24 AddDoctor Record Doctor Info - 25 DeleteDoctor Delete Doctor Info -

Table 5: Test Cases Table of Example for Manual & Automated Testing

3.5 Testing Results with Examples

In this thesis, the same sequence diagrams examples is used for both manual and automated testing. It is reached all test paths results in terms of proposed manual and automated methods with the following examples. In conclusion, it is achieved test cases tables from all manual and automated testing results. Also, the proposed approach is compared to approaches in research papers, and all results are shown in a different table.

3.5.1 Example1

The model is a banking system (ATM). Example 1 shows tree method and test paths results of proposed manual testing using our algorithms. Also, this example shows jtable and test paths results of proposed automated testing method using ‘Table’ and ‘JUnitCode’ programming codes. Appendix A includes XML file of automated testing. Also, a general test cases table is achieved with these methods.

(55)

38

3.5.1.1 Manual Testing of Example 1

Figure 24: The Single Message Sharing Tree (SMST) of Figure23

 Test Paths Path ID Test Paths

1 1-2-3 2 1-2-4 3 1-2-5-6-7 4 1-2-5-6-8 5 1-2-5-6-9-10-11 6 1-2-5-6-9-10-12

Table 6: Sequence ofAll Test Paths in Figure 24

Path ID Test Paths

1 Insert card-Validate(card)-Return 2 Insert card-Validate(card)-EnterPin 3 Insert card-Validate(card)-Pin-VerifyPin-Return 4 Insert card-Validate(card)-Pin-VerifyPin-EnterAmount 5 Insert card-Validate(card)-Pin-VerifyPin-Amount-Check(Balance)-Return 6 Insert card-Validate(card)-Pin-VerifyPin-Amount-Check(Balance)-TakeCash

(56)

39

3.5.1.2 Automated Testing of Example 1

Figure 25: JTable of XML File in Appendix A

 All Results for Test Paths

TEST PATHS

All Possible Test Paths... 1. [1 - 2 - 3] 2. [1 - 2 - 3 - 4] 3. [1 - 2 - 3 - 4 - 5 - 6 - 7] 4. [1 - 2 - 3 - 4 - 5 - 6 - 7 - 8] 5. [1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 - 10 - 11] 6. [1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 - 10 - 11 - 12] 1. [1:InserdCard - 2:Validate(Card) - 3:Return]

2. [1:InserdCard - 2:Validate(Card) - 3:Return - 4:EnterPin]

3. [1:InserdCard - 2:Validate(Card) - 3:Return - 4:EnterPin - 5:Pin - 6:VerifyPin - 7:Return]

4. [1:InserdCard - 2:Validate(Card) - 3:Return - 4:EnterPin - 5:Pin - 6:VerifyPin - 7:Return - 8:EnterAmount]

5. [1:InserdCard - 2:Validate(Card) - 3:Return - 4:EnterPin - 5:Pin - 6:VerifyPin - 7:Return - 8:EnterAmount - 9:Amount - 10:Check(Balance) - 11:Return]

6. [1:InserdCard - 2:Validate(Card) - 3:Return - 4:EnterPin - 5:Pin - 6:VerifyPin - 7:Return - 8:EnterAmount - 9:Amount - 10:Check(Balance) - 11:Return - 12:TakeCash]

BEST PATH

The Best Path: 6. Path [1:InserdCard - 2:Validate(Card) - 3:Return - 4:EnterPin - 5:Pin - 6:VerifyPin - 7:Return - 8:EnterAmount - 9:Amount - 10:Check(Balance) - 11:Return - 12:TakeCash]

ALL MESSAGES

All messages of sequence diagram: [1:InserdCard - 2:Validate(Card) - 3:Return - 4:EnterPin - 5:Pin - 6:VerifyPin - 7:Return - 8:EnterAmount - 9:Amount -

10:Check(Balance) - 11:Return - 12:TakeCash] Total message number of sequence diagram: 12