Process Improvement and Risk Management in Off-The-Shelf Component-Based Development

(1)

Process Improvement and Risk

Management in Off-The-Shelf

Component-Based Development

Jingyue Li

Doctoral Thesis

Submitted for the Partial Fulfilment of the Requirements for the Degree of

Philosophiae Doctor

Department of Computer and Information Science

Faculty of Information Technology, Mathematics and Electrical Engineering

Norwegian University of Science and Technology June 2006

(2)

ISBN printed 82-471-7920-2 ISBN electronic 82-471-7919-9 ISSN 1503-8181

NTNU 2006:84

(3)

TO MY PARENTS ZHONGMIN LI AND YUSHU SHEN, MY WIFE FUFEN JIN AND MY SON LELE

(4)

(5)

Abstract

Reusing software components from third-party vendors is one key technology to gain shorter time-to-market and better quality of the software system. These components, also known as OTS (Off-the-Shelf) components, come in two types: COTS (Commercial-Off-The-Shelf) and OSS (Open–Source-Software) components. To use OTS components successfully, it is necessary to know how the development processes and methods have to be adapted. Most current studies are either theoretical proposals without empirical assessment or case studies in similar project contexts. It is therefore necessary to conduct more empirical studies on how process improvement and risk management were performed and what were the results in various project contexts.

The overall research design of the work was a sequential, mixed method design that consists of several empirical studies. The thesis contains five novel main contributions (C1 to C5) and one minor contribution (C6):

C1. Better understanding on reusing in-house built components. A quantitative preliminary study shows that reusing in-house built components has the same challenges as OTS components related to requirements (re)negotiation, component documentation, and the specification of quality attributes. The informal communication between developers supplements component documentation. The component repository is not a key factor to successful component reuse.

C2: Summarizing the state-of-the-practice of development and selection processes in OTS component-based projects. A qualitative pre-study, a quantitative main study, and a follow-up study with industrial seminar show that OTS component-based development processes are typically variations of well-known process models, mixed with OTS specific activities. Concerning the OTS component selection, we find that mainly familiarity-based and Internet searches with hands-on-trials processes are applied. We summarize the state-of-the-practice of OTS processes into seven scenarios and propose how to customize the processes in OTS component-based projects.

C3: Empirical verification of the risk management proposals in OTS component-based development. The results of the main study reveal that allocating more effort into learning OTS components, performing the integration testing early, evaluating the quality of OTS components thoroughly, and following the update of OTS components have helped to mitigate corresponding risks. However, some problems, such as wrong estimation of the integration efforts and inefficient debugging, still happen frequently in practice and need further investigations.

C4: Better understanding of decision-making in OTS component-based development.

By comparing projects using COTS components with those using OSS components, we reveal who, why, and the results of using different OTS components.

C5: Empirical validation of six newly published theses from the literature, which are related to process and risk management in OTS component-based development. The results of the main study support four theses and contradict the two others.

C6. Improved understanding about performing an international survey in the ICT industry (minor contribution). The main study is probably the first software engineering survey using a representative subset (sample) of industrial companies. Lessons learned from performing such as study reveal that, at best, we can achieve a stratified-random sample of ICT companies, followed by a convenient sample of relevant projects.

(6)

(7)

Preface

This thesis is submitted to the Norwegian University of Science and Technology (NTNU) for partial fulfilment of the requirements for the degree of Philosophiae Doctor.

The work referred to has been performed at the Department of Computer and Information Science, NTNU, Trondheim, under the supervision of Professor Reidar Conradi.

The doctoral work was financed by NTNU for four years. This thesis has also been affiliated with the INCO (INcremental and COmponent-based development) done jointly by the University of Oslo and NTNU.

(8)

(9)

Acknowledgements

During the work on this thesis, I have benefited from a lot of communication with many people, who have provided valuable input to my studies. First of all, I would like to thank my supervisor Prof. Reidar Conradi for giving in valuable feedback and advice during my work. His engagement and knowledge have inspired me a lot. I would also thank Prof. Maria Letizia Jeccheri, Associate Prof. Dag Svanæs, Prof. Tor Stålhane, Dr. Parastoo Mohagheghi, Dr. Carl Fredrik Sørensen, Odd Petter N. Slyngstad, Prof. Kristen Ringdal, and Prof. Ola Listhaug at NTNU. They have all helped me with their insights and advice. Furthermore, I acknowledge Assistant Prof. Marco Torchiano and Associate Prof. Maurizio Morisio at Politecnico di Torino and Dr. Christian Bunse at Fraunhofer IESE for their support and cooperation during the international survey. Stewart Clark has been especially helpful by proof-reading this thesis. I also acknowledge all the industrial partners that participated in this work.

Several master’s students at NTNU have contributed to studies in this thesis. I would like to thank Axel Anders Kvale, Odd Are Sæhle, and Øivind Wang for their contributions. Also, thanks to present and former colleagues at the software engineering group at the Department of Computer and Information Science, NTNU, for providing a good work environment.

I would like to thank Prof. Anthony Finkelstein at University College London, for his kind hospitality and cooperation during my stay there in spring 2005.

I would also like to thank my family. A specific appreciation goes to my parents for supporting and believing in me during all these years. Their effort in taking care of my child provided me enough time to finish this work. I would especially express my thanks to my wife, Fufen Jin and my son Lele for their love and support.

(10)

(11)

Abstract... i Preface... iii Acknowledgements ... v Contents ... vii List of Figures... x List of Tables ... x Abbreviations ... xi 1 Introduction ... 1 1.1 Problem Outline 1 1.2 Research Questions 2 1.3 Research Design 3 1.4 Contributions 4 1.5 Selected Papers 9 1.6 Thesis Structure 11 2 Software Reuse and Component-Based Development ... 13

2.1 Software Engineering Definitions and Challenges 13 2.2 Software Reuse 14 2.3 Component-Based Development 15 2.4 COTS-Based Development 18 2.5 OSS-Based Development 19 2.6 OTS Component-Based Development 21 2.7 Summary and Discussion 22 3 Process Improvement in OTS Component-Based Development... 25

3.1 Software Process Improvement (SPI) 25 3.2 SPI of OTS Component-Based Development 27 3.2.1 Improvement of the Whole Life Cycle... 27

3.2.2 Improvement of the OTS Component Selection ... 28

3.3 Summary and Discussion 29 4 Risk Management in OTS Component-Based Development... 31

(12)

4.2 Risk Management in OTS Component-Based Development 33

4.3 Summary and Discussion 35

5 Research Methods... 37

5.1 Research Strategies in Empirical Software Engineering 37 5.2 Survey Methods 39 5.3 Research Design 40 5.3.1 Research Design of the Preliminary Study... 40

5.3.2 Research Design of the Pre-study... 43

5.3.3 Research Design of the Main Study ... 45

5.3.4 Research Design of the Follow-up Study ... 52

5.4 Validity Issues 53 5.5 Summary and Discussion 54 6 Results... 55

6.1 Collected Samples in Each Study 55 6.1.1 Collected Samples in the Preliminary Study ... 55

6.1.2 Collected Samples in the Pre-study... 55

6.1.3 Collected Samples in the Main Study... 56

6.1.4 Collected Samples in the Follow-up Study ... 57

6.2 Answers to Research Questions 58 6.2.1 Reusing In-House Built Components – RQ1 ... 58

6.2.2 Software Process Improvement – RQ2 ... 59

6.2.3 Risk Management – RQ3 ... 65

6.2.4 Decision Making – RQ4 ... 69

6.3 Our Lessons Learned on Performing the Studies 73 6.4 Summary and Discussion 74 7 Evaluation and Discussion ... 75

7.1 RQ1: How to Improve the Reuse of In-house Built Components? 75 7.1.1 Component-Related Requirements (Re) negotiation... 75

7.1.2 Component Repository... 76

7.1.3 Component Understanding ... 76

7.1.4 Quality Attributes of Components... 76

7.2 RQ2: How to Do SPI in OTS Component-Based Development? 76 7.2.1 How to Improve the Whole Life Cycle ... 77

7.2.2 How to Select the OTS Components... 80

7.3 RQ3: How to Mitigate Risks in OTS Component-Based Development? 81 7.3.1 How to Mitigate the Cost Estimation Risks ... 81

7.3.2 How to Mitigate the Quality Risks ... 82

7.3.3 How to Mitigate the Requirements Risks... 83

7.3.4 How to Mitigate the Debugging and Deployment Risks... 83

7.3.5 How to Mitigate the Maintenance Risks ... 84

7.3.6 How to Mitigate the Provider Support Risks... 85

7.4 RQ4: How to Make the Decision between COTS or OSS Components? 85 7.4.1 Why Did Developers Decide to Use OTS Components... 86

(13)

7.4.2 How Were the OTS Components Used in Practice? ... 86

7.4.3 COTS or OSS? Our Suggestions ... 87

7.5 Evaluation of Validity Threats 87 7.5.1 Possible Validity Threats in the Preliminary Study... 87

7.5.2 Possible Validity Threats in the Pre-study ... 88

7.5.3 Possible Validity Threats in the Main Study... 89

7.5.4 Possible Validity Threats in the Follow-up Study... 90

7.6 Summary 90 8 Conclusions and Directions for Future Work... 91

8.1 Conclusions 91 8.1.1 Describing Different Aspects of Reusing Component ... 91

8.1.2 Verifying Existing Theories and Assessing Existing Methods ... 92

8.1.3 Generating New Theories from New Perspectives... 93

8.2 Future Work 94 8.2.1 Need to Study the Cause-Effects of the SPI Issues ... 94

8.2.2 Risk Management Issues Need a Deeper Investigation... 94

8.2.3 Need to Better Understand OSS Projects and Products ... 94

References... 95

Appendix A: Selected papers ... 107

P1. Developer Attitude to Component Reuse 107 P2. Variations in COTS-Based Development Process 123 P3. Validation of Theses on OTS-Based Development 153 P4. SPI on OTS-Based Development Process 171 P5. Risk Management in OTS-Based Development 185 P6. Decision Making in OTS Component-Based Development 225 P7. Barriers to Disseminating Theory 233 P8. Reflections on Conducting a Survey 241 Appendix B: Other Papers... 259

Appendix C: Interview Guide for the Pre-study... 265

(14)

List of Figures

Figure 1-1 Research questions and corresponding studies... 3

Figure 1-2 Contributions C1 to C5 ... 5

Figure 4-1 Risk management framework ... 31

Figure 4-2 Risk management in OTS component-based development ... 35

Figure 5-1 Risk management before a project starts ... 46

Figure 5-2 Research framework for the risk management in the main study... 46

Figure 6-1 Distribution of company size... 56

Figure 6-2 Distribution of companies main business areas... 57

Figure 6-3 Distribution of application domain of the systems ... 57

Figure 6-4 The actual development process in the OTS-based project... 60

Figure 6-5 What selection and evaluation actions were performed?... 62

Figure 6-6 When was the OTS component selected?... 62

Figure 6-7 Result of thesis T5 ... 64

Figure 6-8 Occurrences of typical risks... 66

Figure 6-9 Risk management activities performed... 67

Figure 6-10 Project profiles of the COTS component-based system ... 69

Figure 6-11 Project profiles of the OSS component-based system ... 70

Figure 6-12 General motivations of using COTS components ... 70

Figure 6-13 General motivations of using OSS components ... 71

Figure 6-14 Specific motivations of using COTS components... 71

Figure 6-15 Specific motivations of using OSS components... 72

Figure 6-16 Occurrences of risks in COTS component-based projects ... 72

Figure 6-17 Occurrences of risks in OSS component-based projects ... 73

Figure 7-1 Scenarios 1 to 3... 78

Figure 7-2 Scenarios 4 to 6... 79

List of Tables

Table 1-1 Relations between research questions, contributions, and papers... 9

Table 2-1 Advantages and disadvantages of using COTS software... 19

Table 5-1 Qualitative vs. quantitative in empirical strategies ... 38

Table 5-2 Questionnaire used in the preliminary study... 42

Table 5-3 Correspondence between questions and RQs in the preliminary study ... 43

Table 5-4 Typical risks in OTS component-based projects ... 48

Table 5-5 Typical risks management activities in OTS-based projects ... 49

Table 5-6 Revised research questions to investigate the newly published theses ... 50

Table 6-1 Barriers to disseminating SPI theories to the IT industry ... 64

Table 6-2 Effective risk management activities ... 68

(15)

Abbreviations

CBD Component-Based Development CBSE Component-Based Software Engineering CMM Capability Maturity Model

CRM Customer Relationship Management COCOTS COnstructiveCOTS

CORBA Common Object Request Broker Architecture COM Component Object Model (Microsoft)

COTS Commercial-Off-The-Shelf

DCOM Distributed Component Object Model (Microsoft) EJB Enterprise Java Beans (Sun)

ERP Enterprise Resource Planning GUI Graphical User Interface

HHM Hierarchical Holographic Modelling

ICT Information and Communication Technology ISO International Standards Organization

IEC International Electrotechnical Commission LOC Lines of Code

NTNU Norwegian University of Science and Technology OO Object-Oriented

OTS Off-The-Shelf

OSS Open Source Software ROI Return On Investment RUP Rational Unified Process

SEI the Software Engineering Institute (SEI) at Carnegie Mellon University SPI Software Process Improvement

UP Unified Process

(16)

(17)

1 Introduction

In this chapter, the background to the research and the research context are briefly presented. The chapter also describes research questions, research design, and the claimed contributions. The list of papers, my specific contributions to each study, and the thesis outline are also presented.

1.1 Problem Outline

To gain competitive advantages, software organizations are forced to develop systems quickly and cost-efficiently. Reuse is therefore onekey technology to reach their goals. Reuse is known since the beginning of computer science (e.g., reuse of command sequences) and has now be raised to the level of components (i.e., a reusable piece of software that can be easily integrated with other components with relatively little effort). The components can be built in-house, be acquired from the third-party providers, or be ordered from the sub-contractors. The components acquired from third-party providers, also known as OTS (Off-the-Shelf) components, come in two different types: COTS (Commercial-Off-The-Shelf) and OSS (Open–Source-Software) components. To integrate the OTS components successfully, one major research question is: how development processes or methods have to be adapted concerning the development of systems with such components?

Although researchers and practitioners have been dealing with OTS component-based development processes for quite a time, most studies are component-based on military or aerospace projects, or similar large projects. To propose and design cost-effective OTS-based development process, it is necessary to empirically investigate how OTS-OTS-based projects are performed in different application domains, especially in small or medium-sized projects and companies.

Although using OTS component may help to shorten time-to-market and save development effort, using such external components introduces many risks. Before project managers decide to acquire an external component instead of building it in-house, they must evaluate and compare possible risks and benefits. Although several risks and risk management activities in OTS component-based development have been identified from case studies, few empirical studies have subsequently verified their

(18)

Introduction

conclusions. As a result, software project managers have few effective and well-proven guidelines on using specific risk management activities to mitigate corresponding risks.

Although both COTS and OSS components can be acquired from third-party providers, they are different in their nature and appearance. COTS components are owned by commercial vendors that often provide specific support. Users of COTS components have limited access to the source code. On the other hand, OSS components are provided by open source communities with freely accessible source code, but with no promise about specific support. When planning a new software project, project decision-makers need to decide whether they should buy a COTS component or acquire an OSS one, if it was decided to use OTS components. To make such a decision, it is important to investigate previous projects using such components. Unfortunately, the relevant decision-making processes are often intuitive and only sparsely documented. Thus, further research is warranted to examine the current state-of-the-practice and to define guidelines for making such decisions.

1.2 Research Questions

The goal of this research is to investigate state-of-the-practice of COTS and OSS component-based development from the process and organizational viewpoints. The motivation is to give guidelines on how to select and integrate COTS or OSS components based on the experiences and lessons learned from finished projects. There are four main research questions in this study. The research questions are designed and refined step by step.

The process and risk management issues of component-based development were firstly investigated through a preliminary study on reusing in-house built components. This study examined research question RQ1:

RQ1: What is the status of component-based development based on components built in-house?

Results of the preliminary study gave valuable insights on designing research

questions for later studies. We are aware that there are many common issues between reusing in-house built components and using OTS components. Therefore, we decided to focus more on using OTS components, because it will cover broader issues and some of these issues are also relevant to in-house built components.

Based on results of the preliminary study, it was decided to investigate the software process improvement (SPI) and risk management in COTS components-based development with a pre-study. This study investigated research questions RQ2 and

RQ3:

RQ2: What are the development processes and OTS component-selection processes used in OTS component-based development projects?

RQ3: Which risk management activities have been performed and what are the results of performing them?

(19)

Introduction

By talking with the respondents face-to-face in the pre-study, we got insights on SPI and risk management in OTS component-based development. We also found that OSS components should be involved in future studies, because OSS components represent an important type of OTS components. In addition to know their commonalities, it is interesting to know the differences in using these two different types of OTS components. Therefore, research question RQ4 is defined as:

RQ4: How do project managers make their acquire decisions in using COTS instead of OSS components, or vice versa?

1.3 Research Design

Figure 1-1 shows the studies performed, their date and sequence, type of studies, and their relations with the research questions.

(20)

Introduction

Empirical studies may be performed quantitatively, qualitatively or in combination. The studies reported in this thesis have been a combination of qualitative and quantitative studies. The choices of approach are decided by the research questions.

The research question RQ1 was studied by an explorative quantitative preliminary study. Data from 26 people in three Norwegian IT companies were collected by

interviews using a structured questionnaire.

Research questions RQ2 and RQ3 were first investigated by an explorative qualitative pre-study. The pre-study was performed through semi-structured interviews

on COTS component-based development. In this pre-study, we interviewed 16 COTS component-based development projects in 13 Norwegian IT companies. In addition,

RQ4 was initiated based on the results of the pre-study.

To validate the conclusions of the pre-study, a descriptive quantitative main study

was performed. Data was collected with a structured questionnaire. Results from 133 OTS component-based projects in Norway, Italy, and Germany have been collected. Results of the main study verified the conclusions of the pre-study.

In the late phase of the main study, we performed an explorative qualitative follow-up studywith an industrial seminar to present our conclusions, to discuss with industrial

partners, and to get feedback from them. The intention was to briefly understand some cause-effects of the phenomena we discovered from the previous studies.

1.4 Contributions

To manage the OTS component-based project successfully, project members need to make several decisions. These decisions may not be sequential and may need several iterations. We mainly contributed on summarizing the state-of-the-practice and giving guidelines on making several key decisions, as shown in Figure 1-2.

(21)

Introduction

Figure 1-2 Contributions C1 to C5

Contribution C6 is a secondary contribution, which comes from the lessons learned

in performing the survey. It gives guidelines on how to perform an international survey with stratified-random sample selection strategy.

Detailed descriptions of each contribution:

C1: Increased understanding on risks of reusing in-house built components

In a preliminary quantitative study, we have investigated challenges related to four key factors for development based on in-house built components, especially in development-with-reuse. These factors are component-related requirements (re)negotiation, component repository, component understanding, and components’ quality attribute specification. Another contribution is that we have compared three IT companies with different reuse levels to study the possible trend and challenges in these factors when more and more code is encapsulated as reusable components.

For component-related requirements (re)negotiation, we conclude that requirements (re)negotiation for in-house built components is important but not efficient. The

(22)

Introduction

efficiency will probably not increase with a higher reuse level (i.e., more reusable components are available in the organization).

For a component repository, we confirm that this is not a key factor for successful reuse. Furthermore, the potential value of a component repository will probably not increase with higher reuse levels.

For component understanding, we reveal that most developers are not satisfied with component documentation, and that developers’ satisfaction with component documentation will decrease with higher reuse levels. The results also show that informal communication channels (i.e., tacit knowledge), through which developers may get necessary information about the components, should be given more attention.

For components’ quality attribute specification, we discover that developers still need to spend much effort on testing the components before they start to use them, as they cannot get relevant information from component specifications.

C2: Summarizing the state-of-the-practice of development and selection processes in OTS component-based projects

The contributions to the OTS component-based development process focus on two dimensions:

- The whole software development life cycle. - OTS component selection and evaluation.

A qualitative pre-study with semi-structured interviews was first performed to examine the state-of-the-practice of the development processes in COTS component-based development. We found:

- The use of COTS components can be done as part of traditional development

processes (e.g., waterfall and evolutionary), i.e., there is no special “COTS-based development process”.

- Successful use of COTS components in such processes, however, requires that

some new activities and roles are introduced. Typical new activities are the make vs. acquire decision, COTS component selection, and COTS component integration. A new role is that of a knowledge keeper.

- Two of the new activities, the make vs. acquire decision and the COTS

component selection, can be placed in different development phases (requirements, design, or implementation). This depends on project context, especially on the familiarity with possible COTS components and the flexibility of requirements.

- A set of explanatory scenarios and guidelines have been synthesized to assist in

the customization of the traditional development processes with the new activities and roles.

Based on the results of the pre-study, a main study with a formal questionnaire was launched. This studied 133 projects from Norway, Italy, and Germany. It had a broader focus than the pre-study and also included projects using OSS components. The results confirm our findings in the pre-study. We conclude:

(23)

Introduction

- The actual OTS-based development process was the traditional process with

OTS-specific activities. The development process was dominated by the company/department rules, instead of the decision of using OTS components.

- The actual OTS component selection can be done in different phases. The main

evaluation processes are familiarity-based or hands-on-trial-based. The phase to select OTS component has relationship with the project members’ familiarity with possible component candidates.

- The proposed software development process and OTS component selection

process need further investigation to be suited to different project contexts.

By summarizing the state-of-the-practice of OTS component-based development processes into seven scenarios, we gave systematic proposals about how to adopt the OTS-based development and selection processes based on the project contexts.

C3: Empirical validation of risk management proposals in OTS component-based development

By summarizing the good practices and lessons learned in the projects investigated in the pre-study, we got new insights on managing the possible risks in OTS component-based development. In the main study, we further examined our findings. Our contributions answered the following three sub-questions:

- What are the most frequent problems occurred in OTS component-based development?

Two of the most frequent problems in OTS component-based development are the incorrect estimation of the integration effort and inefficient debugging. The least frequent problems are relevant to the negative effect of the OTS components on the quality of the whole system.

- What are the most frequent risk management activities performed in OTS component-based development?

Developers have seriously evaluated the quality of the OTS components in the selection phase. They have considered the possible learning effort in the effort estimation. They have done integration testing as early as possible and usually have internal experts to follow the update of the OTS components. However, developers have rarely involved their customers in the make vs. acquire decision and OTS component selection.

- What are the results of the risk management activities on corresponding risks?

To increase the accuracy of effort estimation, it is important to allocate enough effort to learn and understand the OTS components. To avoid the possible quality problems and to increase the efficiency of locating the defects, it is important to do black-box testing as part of OTS component selection and to do integration testing as early as possible. To mitigate the problems due to customers’ requirements changes, it is important to learn the OTS components thoroughly in the selection phases and to integrate the unfamiliar OTS components first. It is also important to be able to negotiate changed requirements with the customers. To increase the efficiency of debugging and

(24)

Introduction

system deployment, it is necessary to understand the components well and to do integration testing early. It is critical to keep and share the knowledge about OTS components and the component providers inside the organization. It shows to help mitigate the possible risks in system maintenance and evolution.

C4: Increased understanding of the decision-making practice in OTS component-based development

We compared who, why, and the results of using COTS components instead of OSS components, or vice versa. This contributed to answer the three questions:

- Who were using OTS components?

Both COTS and OSS components were used in projects with different application domains and different non-functional requirements. There was no significant difference between the profiles of COTS-based and OSS-based projects.

- Why was it decided to use OTS components?

The main expectations of using either COTS or OSS components are to obtain shorter time-to-market and less development effort. COTS users have higher expectation on COTS component quality and COTS vendor support. Possible no-cost source code is the key motivation of OSS users. OSS users prefer to have access to the source code, so that they can revise it when necessary.

- What are the possible problems of using each type of OTS components?

It is more difficult for COTS component users to follow requirement changes than OSS users. It is also more difficult for COTS users to estimate the selection effort.

C5: Empirical validation of the six newly published theses, which are related to the process and risk management in OTS component-based development

The pre-study and main study also investigated six newly published theses in COTS component-based development proposed in [Torchiano04]. Our results support four of the theses and contradict the two others. The supported theses are: OSS components were mainly used without modification in practice; custom code mainly provided additional functionality; formal COTS selection processes were seldom used; COTS component users managed to get the required changes from vendors. The unsupported theses are: standard mismatches were more frequent than architecture mismatches; COTS components were mainly selected based on architecture compliance instead of functional completeness.

C6: Improved understanding on performing an international survey in the ICT industry

In the main study, a stratified-random sample selection strategy was used to collect data in three European countries. The lessons learned from performing the main study contributed to give guidelines on performing similar empirical studies in the IT industry. We found that:

(25)

Introduction

- A pre-study often clarifies both research questions and concrete questions in a

later questionnaire. For instance, we “discovered” several variants of OTS-based development processes in our pre-study.

- A sample of national Information and Communication Technology (ICT)

companies assumes access to public data sources, like company lists from census registers, but even such data may contain many errors. Close contact with people in such organs are needed to obtain stratified samples and expedite delivery of data in electronic form.

- For a variety of reasons it seems impossible to perform the same random

sampling and data collection in different countries. The national differences may give an unknown mode effect in result interpretation between the countries.

- Random sampling and the ensuing contact process appear to be much more

expensive (5 times?) than convenient sampling. In our case, this is because we got stuck with no previous contact person, and scant pre-knowledge and “goodwill” from such companies, especially the large ones.

- If the final sample size is under 100 respondents, use phone interviews. This

applies particularly when phone contact is anyhow needed, e.g. due to a non-convenient sample. A written procedure for the contact process is anyhow needed.

- A project baseline can be useful for comparison purposes. In our case, we did

not ask the companies about non-OTS projects, so we missed this.

- Interviewing on this level requires a sound background knowledge of the

material; Company phone contact assumes at least PhD student qualifications.

1.5 Selected Papers

This thesis includes eight papers from P1 to P8 as follows. These papers show our main contributions to the research questions and are attached in Appendix A. Abstracts of other papers (from AP1 to AP7) related to work in this thesis are attached in Appendix B. All these papers can be downloaded from http://www.idi.ntnu.no/grupper/su/. The relations between the research questions, contribution, and the papers are shown in Table 1-1.

Table 1-1 Relations between research questions, contributions, and papers Research questions Contributions Papers

RQ1 C1 P1*,

RQ2 C2 P2*, P4*, P7*, AP6, AP7

RQ3 C3 P2*, P5*, P7*, AP1, AP2, AP3, AP4

RQ4 C4 P6*, AP5

RQ2&RQ3 C5 P3*

Method issues C6 P8*

* Papers attached in Appendix A

[P1] [Li04a] Jingyue Li, Reidar Conradi, Parastoo Mohagheghi, Odd Are Sæhle, Øivind

Wang, Erlend Naalsund, and Ole Anders Walseth: A Study of Developer Attitude to Component Reuse in Three IT Companies. Proceedings of 5th International Conf. on

(26)

Introduction

Product Focused Software Process Improvement (PROFES'2004), Apr. 2004, Nara area, Japan, Springer LNCS Vol. 3009, pp. 538-552.

Relevance to this thesis: This paper reports the result of the preliminary study,

which covers RQ1 and shows our main contribution to C1.

My contribution: I was the leading author and contributed about 70% of the total

work, including research design, questionnaire design, data analysis, and paper writing.

[P2] [Li06a] Jingyue Li, Finn Olav Bjørnson, Reidar Conradi, and Vigdis By

Kampenes: An Empirical Study of Variations in COTS-based Software Development Processes in Norwegian IT Industry. To appear in the Journal of Empirical Software Engineering (Special issue from Metrics 2004).

Relevance to this thesis: This paper reports the result of the pre-study, which

covers RQ2 and RQ3 and partly shows our contribution to C2 and C3.

My contribution: I was the leading author and contributed about 80% of the total

work, including research design, interview guide design, performing most interviews, data analysis, and paper writing.

[P3] [Li05a] Jingyue Li, Reidar Conradi, Odd Petter N. Slyngstad, Christian Bunse,

Umair Khan, Marco Torchiano and Maurizio Morisio: Validation of New Theses on Off-The-Shelf Component Based Development. Proceedings of the 11th IEEE International Metrics Symposium (Metrics'05), Sep. 2005, Como, Italy, IEEE Press, pp.26 (abstract), 10p.

Relevance to this thesis: This paper reports the result of the main study, which

covers RQ2 and RQ3 and shows our contribution to C5.

My contribution: I was the leading author and contributed 70% of the total work,

including research design, questionnaire design, and data collection in Norway, data analysis, and paper writing.

[P4] [Li06b] Jingyue Li, Marco Torchiano, Reidar Conradi, Odd Petter N. Slyngstad,

and Christian Bunse: A State-of-the-Practice Survey of the Off-the-Shelf Component-Based Development Processes, Proceedings of the 9th International Conference on Software Reuse, Torino, Italy, June, 2006, Springer LNCS Vol. 4039, pp. 16-28.

covers RQ2 and shows our main contribution to C2.

[P5] [Li06c] Jingyue Li, Reidar Conradi, Odd Petter N. Slyngstad, Christian Bunse,

Marco Torchiano, and Maurizio Morisio: A State-of-the-practice Study on Risk Management in OTS Component-Based Development. Submitted to IEEE Transactions on Software Engineering, June 2006.

(27)

Introduction

including research design, questionnaire design, data collection in Norway, data analysis, and paper writing.

[P6] [Li06d] Jingyue Li, Reidar Conradi, Odd Petter N. Slyngstad, Christian Bunse,

Marco Torchiano, and Maurizio Morisio: An Empirical Study on the Decision Making Process in Off-The-Shelf Component Based Development. Proceedings of the 28th International Conference on Software Engineering (ICSE 2006), 20-28 May 2006, Shanghai, P.R. China, pp. 897-900, IEEE Press.

covers RQ4 and shows our main contribution to C4.

[P7] [Li05b] Jingyue Li, Reidar Conradi, Odd Petter N. Slyngstad, Christian Bunse,

Umair Khan, Marco Torchiano and Maurizio Morisio: Barriers to Disseminating Off-The-Shelf Based Development Theories to IT Industry. Proceedings of the International Workshop on Models and Processes for the Evaluation of COTS Components

(MPEC'05 Arranged in co-location with ICSE'05), May, 2005, St. Louis, Missouri, USA, pp. 1-4. It is also published in the electronic version of ACM Software Engineering Notes, Vol. 30, No. 4, July 2005.

Relevance to this thesis: This paper reports the result of the follow-up study,

which covers RQ2 and RQ3, and shows partly our contribution to C2 and C3.

My contribution: I was the leading author and contributed on 80% of the total

work, including research design, seminar organization, data collection, data analysis and paper writing.

[P8] [Conradi05a] Reidar Conradi, Jingyue Li, Odd Petter N. Slyngstad, Christian

Bunse, Marco Torchiano and Maurizio Morisio: Reflections on conducting an international CBSE survey in ICT industry. Proceedings of the 4th International Symposium on Empirical Software Engineering (ISESE 2005), Nov. 2005, Noosa Heads, Australia, pp. 214-223, IEEE Press.

Relevance to this thesis: This paper reports our lessons learned on the main study,

and shows our main contribution to C6.

My contribution: I contributed on 40% of the total work, including sample

selection design, performing the data collection, and major comments on the data analysis and draft paper.

1.6 Thesis Structure

Chapters 2 to 5 give the state-of-the-art of relevant fields of this thesis and introduce how the research questions are designed. Chapters 6 to 8 give the results and discuss our contributions to the research questions.

Chapter 2: Software reuse and component-based development. This chapter

(28)

Introduction

development. First, we discuss the motivation of systematic reuse in software development and illustrate different kinds of reuse, such as reusing documentation, architecture, design, and test cases. Then, we introduce the intention and challenges of reusing software components. Finally, we focus on issues of reusing the third-party software components, such as COTS and OSS components.

Chapter 3: Software process improvement in OTS component-based development.

This chapter introduces the state-of-the-art of SPI in OTS component-based development. We describe not only proposed process models in COTS or OSS component-based development, but also OTS component selection processes.

Chapter 4: Risk management in OTS component-based development. This chapter

first introduces the theory of risk management. We then list the proposed specific risks and according risk management activities in OTS component-based development.

Chapter 5: Research methods and metrics. In this chapter, the research goals and

research questions are specified. We give our rationale for using different research methods to answer different research questions.

Chapter 6: Empirical investigations and main results/findings. This chapter explains

how we perform the preliminary study, the pre-study, the main study, and the follow-up study. We give detailed information on data collection methods, collected samples, and answers to research questions in each study.

Chapter 7: Evaluation and discussion. The research questions are answered in this

chapter. We summarize the results from the empirical investigation and compare them with the state-of-the-art. We also discuss possible threats to the validity of each study.

Chapter 8: Conclusion and future work. This chapter sums up the main findings from

the discussion ands outline possible future work.

Appendix A contains 8 selected papers (P1 to P8) that provide detailed results and

discussions of the individual studies.

Appendix B contains the abstracts of other seven relevant papers (AP1 to AP7) which

are not attached in this thesis.

Appendix C includes the interview guide for the pre-study. Appendix D contains the questionnaire used in the main study.

In both parts of this thesis, I have generally used the term “we” to present the work. The layout of papers in Appendix A has been changed to fit the format of the rest of the thesis.

(29)

Software reuse and component-based development

2 Software Reuse and Component-Based

Development

This chapter describes the challenges in software engineering that are the motivations behind reuse. The state-of-the-art in Component-Based Development (CBD), COTS development, and OSS-based development are then described. Finally, the whole chapter is summarized and the research challenges related to this thesis are discussed.

2.1 Software Engineering Definitions and Challenges

Software engineering is the branch of system engineering concerned with the development of large and complex software intensive systems. It is concerned with the processes, methods, and tools for the development of software intensive systems in an economic and timely manner [Finkelstein00]. Software engineering activities or phases include managing, estimating, planning, modelling, analysing, specifying, designing, implementing, testing, and maintaining [Fenton97].

The term software crisis was first used in 1968 to describe the ever-increasing burden and frustration that software development and maintenance placed on otherwise happy and productive organizations [Griss93]. The much cited Standish report on software projects [Standish95] shows “a staggering 31.1% of projects will be cancelled before they ever get completed”. Further results indicate 52.7% of projects will cost 189% of their original estimates. The cost of these failures and overruns are just the tip of the proverbial iceberg.

Philippe Kruchten [Kruchten01] discusses why software engineering differs from structural, mechanical, and electrical engineering due to the soft, but unkind nature of software. He suggests four key differentiating characteristics:

− Absence of fundamental theories or at least practically applicable theories makes is difficult to reason about software without building it.

− Ease of change encourages changes in software, but it is hard to predict the impact.

− Rapid evolution of technology does not allow proper assessment, and makes it difficult to maintain and evolve legacy systems.

− Very low manufacturing costs combined with ease of change have led the software industry into a fairly complex mess.

(30)

These characteristics have even become more extreme due to developments in Internet-speed. Internet-speed development involves rapid requirement changes and unpredictable product complexity [Baskerville03]. Software organizations have always been looking for effective strategies to develop software faster, cheaper, and better.

2.2 Software Reuse

The motivations for reuse are primarily economic: the potential of saving cost, time and effort of redundant work, increasing productivity, decreasing time to market and improving systems quality by reusing both artefacts, and the underlying engineering experience.

Software reuse means reusing the inputs, the processes, and the outputs of previous software development efforts. The idea of reuse was originally due to Doug McIlroy [McIlroy68] in his paper “Mass Produced Software Components”. Reuse is later named as a potential “silver bullet” [Brooks87]. Many software organizations around the world have reported successful reuse programs, such as at IBM, Hewlett-Packard, and Hitachi [Griss93]. Jones [Jones84] identified four types of reusable artifacts:

- Architecture reuse, which consists of standardizing a set of design and

programming conventions dealing with the logical organization of software.

- (detailed) Design reuse: for some common business application. - Data reuse, involving a standardization of data formats.

- Program reuse, which deals with reusing executable code.

The challenges facing reuse are structural, organizational, managerial, and technical [Mili95]. Many organizations treat reuse as technology-acquisition problems instead of a technology-transition problem. However, just buying technology usually does not lead to extensive reuse. Morisio et al. [Morisio02b] found that:

− Top management commitment is the prerequisite for success.

− Product family practice, common architecture, and domain engineering increase reuse capability.

− Size, development approach (object-oriented or not), rewards, repository, and reuse measurement are not decisive factors, while training is.

− The other three factors that are considered to be success factors are reuse process introduced, non-reuse process modified, and human factors.

− Successful cases tried to minimize change, to retain their existing development approach, choosing reuse technology to fit that.

Griss [Griss95] summarized the experience from industrial reuse and pointed out that:

− Reuse needs management support, since reuse involves more than one project.

− Object technologies or libraries give no improvement in reuse.

− Domain stability and experience are often more important for successful reuse than general process maturity.

(31)

Frakes et al. [Frakes95] report that most software engineers prefer to reuse rather than to build from scratch. In addition, they did not find any evidence that use of certain programming languages, CASE tools, or software repositories promote reuse. On the other hand, reuse education and a software process that promotes reuse have a positive impact on reuse. They also found that the telecom industry has higher levels of reuse than some other fields.

Software reuse includes two interrelated parts: reuse as and reuse with. Reuse as

means creating the asset for future reuse. Reuse with indicates using the reusable artefacts in the new system.

2.3 Component-Based Development

Component-based development (CBD) and Component-based Software Engineering (CBSE) facilitate reuse by providing logical units for assembly and make systematic reuse possible by demanding that components should adhere to a component model. CBD focuses on software architecture as a guideline to put pieces together, viewing components as independent units of development and deployment, and on component models. Developing components is one kind of developing for reuse, while developing systems of reusable components is developing with reuse [Karlsson95]. Many CBD concepts originate from reusability and Object-Oriented (OO) approaches.

Components, the heart of CBSE, are defined in various ways depending on the viewpoint. We introduce several theoretical definitions in this section. The component definition used in this thesis is shown in Section 2.6.

In the SEI’s report [Bachmann00] on technical aspects of CBD, a component is defined as:

− An opaque implementation of functionality.

− Subject to third-party composition.

− Conformant to component model.

Heineman and Council [Heineman01] define a software component as: A software element that conforms to a component model, which can be independently deployed and can be composed without modification according to a composition standard.

Szyperski’s [Szyperski02] definition is: A software component is an executable unit of independent production, acquisition, and deployment that can be composed into a functioning system. To enable composition, a software component adheres to a particular component model and targets a particular component platform.

What these three definitions have in common are:

− Components are units of independent development and acquisition.

− Components adhere to a component model that enables composition of components. Composition is the term used for components, instead of integration.

None of these two aspects are found in OO approach. Some other differences between component approach and OO approach are:

− Instantiation: components may be instantiated or not, and if instantiated there are usually not many instances of them [Atkinson02].

(32)

− Components may have state or not (in order to be replaceable they should not have state).

− Components are generally considerably larger than individual classes [Bosch00]. Bachman et al. [Bachman00] list the advantages of CBD as:

− Reduced time to market: even if component families are not available in the application domain, the uniform component abstractions will reduce overall development and maintenance costs.

− Independent extensions: components are units of extension and component models prescribe how extensions are made.

− Easy to acquire: there are component markets to acquire components.

− Improved predictability: component frameworks can be designed to support the quality attributes that are most important.

Advantages added by Bass et al. [Bass00] are:

− Improved productivity, which can lead to shorter time-to-market.

− Separation of skills: complexity is packaged into the component framework and new roles are added, such as developer, assembler, and deployer.

− Components provide a base for reuse, since components are a convenient way to package value. They have direct usability (may be used to build systems directly); while other approaches such as design patterns are more abstract. Component-based system development is often promoted in the literature as a low risk development strategy, which provides a simple and rapid mechanism for increasing the functionality and capability of a system. In reality, CBD carries significant risk throughout the system life cycle. Bass et al. [Bass00] mention inhibitors in CBD as being lack of available components, lack of standards for component technology, lack of certified components, and lack of engineering methods. Some challenges in each development phase and for a project as a whole are discussed here, using [Crnkovic02], [Ghosh02], [Jacobson97] and other various sources as:

− Management: decision-making on make vs. reuse vs. acquire, initiating product

families, Return on Investment (ROI), vendor interactions for cost estimates. Although component markets have grown in the recent years, there are few empirical studies that can verify increased productivity or shorter time to market due to acquiring components.

− Component based software life cycle: The life cycle of component-based

software is becoming more complex because many phases are separated in unsynchronized activities. For example, the development of components may be completely independent of the development of systems using those components. The process of engineering requirements is much more complex because the possible candidate components usually lack one of more features that the system requires.

− Composition predictability: Even if we assume that we can specify all the

(33)

attributes will determine the corresponding attributes of systems of which they may become part.

− Trusted components and component certification: One way of classifying

components is to certify them [Voas98a]. In spite of the component belief that certification means absolute trustworthiness, it in fact merely provides the results of tests performed and a description of the environment in which the tests were performed. Although certification is a standard procedure in many domains, it has not yet been established in software in general especially not for software components.

− Component configurations: As soon as we begin to work with complex

structures, problems involving structural configurations appear.

− Tools support: The purpose of software engineering is to provide practical

solutions to practical problems. Component evaluation tools, component repositories and tools for managing the repositories, component test tools, and component-based design tools, run-time analysis tools, and component configuration tools and so on.

− Dependable system and CBSE: The use of CBD in safety-critical domains,

real-time systems and different process – control systems, in which reliability requirements are particularly rigorous, is particularly challenging.

Components can be built in-house, acquired from the third-companies, or just downloaded from the websites of the OSS communities.

− In-house development

In-house development means that the provider and the consumer of the software reuse artefacts is the same person or organization. So, a component is a white-box for the person who may use it. Moreover, the component may be only used for special application and domain. Thus, it possible that the component is not based on uniform, standard coordination schemes.

− COTS-based development

Practitioners using COTS software equate components to COTS products. According to the perspective of SEI [Brownsword00], “A COTS product is: sold, leased, or licensed to the general public; offered by a vendor trying to profit from it; supported and evolved by the vendor, who retains the intellectual property rights; available in multiple, identical copies; and used without source code modification.” Here, COTS components are mostly executable black-box components.

− Open Source Software

Open Source Software (OSS) [Hissam01] makes the definition of component more complex. OSS means that the customers can get the source code of the software and they may change the source code and give back the modified artefacts. Thus, the open source component can also be regarded as a white-box.

(34)

2.4 COTS-Based Development

COTS software is generally a software that is not developed inside the project, but acquired from a vendor and used “as-is”, or with minor modifications.

Oberndorf [Oberndorf97] defines the term COTS product on the basis of the “Federal Acquisition Regulations”. It is defined as something that one can buy, ready-made, from some manufacturer’s virtual store shelf (e.g., through a catalogue or from a price list). It carries with a sense of getting, at a reasonable cost, something that already does the job. The main characteristics are 1) it exists a priori, 2) it is available to the general public, or 3) it can be bought (or leased or licenced).

Vidger et al. [Vidger96][Vidger97] define COTS as pre-existing software products; sold in many copies with minimal changes; whose customers have no control over specification, schedule, and evolution; access to source code as well as internal documentation is usually unavailable; complete and correct behavioral specifications are not available.

Basili and Boehm [Basili01] specify that COTS has the following characteristics 1) the buyer has no access to the source code, 2) the vendor controls its development, and 3) it has a nontrivial installed base.

Torchiano and Morisio [Torchiano04] give a more detailed, empirically based definition as: A COTS product is a commercially available or open source piece of software that other software projects can reuse and integrate into their own products. They also specify a COTS product:

− Is not produced in or exclusively for the project.

− Can be closed source or open source. If it is open source software, it is usually treated as if it were closed.

− Is not a commodity. It is not shipped with the operating system, provided with the development environment, or generally included in any pre-existing platform.

− Is integrated into the final delivered system. It is not a development tool.

− Is not controllable, in terms of provided features and their evolution.

COTS software can be classified into different categories. Morisio et al. [Morisio02a] summarized some of the previous work and gave a proposal on how to classify COTS. These classification attributes are supposed to be linked with software process (e.g., cost models, selection methods, architectures, testing, and validation techniques).

COTS usage promises faster time-to-market and increased productivity [Voas98c]. Developing software with as much COTS functionality as possible saves you from reinventing the wheel. With COTS software, the functionality you desire can be:

− Accessed immediately.

− Obtained at significantly lower prices than otherwise.

(35)

At the same time, COTS-based software introduces many challenges, such as unknown quality properties of the chosen COTS components can be harmful for the final product and the economic instability of the COTS vendor may terminate the maintenance support of its COTS components [Voas98b]. The advantages and disadvantages of using COTS software are summarized in [Boehm99] as shown in the Table 2-1.

Table 2-1 Advantages and disadvantages of using COTS software

Advantages Disadvantages

Immediately available; earlier payback Licensing, intellectual property procurement delays

Avoids expensive development Up-front licensing fees Avoids expensive maintenance Recurring maintenance fees Predictable, confirmable licence fees

and performance Reliability often unknown or inadequate; scale difficult to change Rich functionality Too-rich functionality compromises

usability, performance

Broadly used, mature technologies Constraints on functionality, efficiency Frequent upgrades often anticipate

organization’s needs

No control over upgrades and maintenance

Dedicated support organization Dependence on vendor

Hardware/software independence Integration not always trivial; incompatibilities among vendors

Tracks technology needs Synchronizing multiple-vendor upgrades

2.5 OSS-Based Development

The basic idea behind OSS is very simple: When programmers can read, redistribute, and modify the source code for a piece of software, the software evolves. People improve it, people adapt it, and people fix bugs. This can happen at a speed that, if one is used to the slow pace of conventional software development, seems astonishing. However, open source does not just mean access to the source code. The distribution terms of OSS must comply with the following criteria, such as free redistribution, open source code, derived work and so on[OSS04].

Many people prefer the term free software to open source software. The term free software dates from 1984, when the idea and arguments for it were first published by Richard Stallman. The idea is not that software should be free “as in beer,” or available at no charge, but that it should be free “as in speech”, so that you can review it and change it as you need to [Freesoftware05]. The term open source dates from 1997, when a group of people, including Eric Raymond, Tim O’Reilly, and Bruce Perens, decided that the term free software and some of the arguments employed in support of it were making the idea less attractive to many businesses. They decided, as a marketing decision, to emphasize technical and practical advantages of open source software rather than arguments from principle [OSS04].

(36)

OSS software has risen to great prominence. There is excellent evidence that OSS has a significant market share in numerous markets, for example,

− Netcraft’s statistics on web servers shows that Apache is the number one web server with over three times the market share of its next ranked competitor [Netcraft05]

− A 2004 InformationWeek survey found that 67% of companies use OSS products with another 16% expecting to use it in 2005, and only 17% have no near-term plans to support OSS products [Infoweek04].

OSS software offers many choices from large suites to small components. Typical OSS suites are Linux (operating system), Apache (web server), MySQL (database), PostgreSQL (database), Eclipse (development platform), and OpenOffice (office suites). Examples of software components are zlib (compression/decompression Library), SpamAssassin (spam-checker for email servers), and log4j (logging component). As of 27th April 2006, there are more than 118,615 OSS project registered at sourceforge.net and more than 40,590 project registered at freshmeat.net. The projects are classified according to the application domain of the software, such as clustering, games, networking, and security at sourceforge.net and freshmeat.net.

OSS has many proposed advantages [Fitzgerald04], [Ruffin04] [Madanmohan04]:

− OSS is usually freely available for public download. The collaborative, parallel efforts of globally distributed developers allow OSS to be developed more quickly than conventional software.

− Many OSS products are recognized for high reliability, efficiency, and robustness.

− When OSS is a de facto standard component and has a large user community, it is a lasting solution that can resist the commercial supplier uncertainties that can often abruptly end a product’s life

− Using mainstream OSS reduces cycle time for component updates and corrections. More and often free labour is available to localize and correct defects. Especially for embedded systems, OSS provides fast, new drivers and hardware-related features, even for rather exotic environments.

− Another attraction is that the marginal cost of scaling up is zero. OSS does not require additional licences as installation numbers grow.

Despite its wide appeal, OSS software faces a number of serious challenges and constraints [Fitzgerald04] [Ruffin04]:

− The popularity of OSS increases the risk that so-called net-negative-producing programmers will become involved.

− Many software tasks, such as documentation, testing, and field support are lacking in OSS projects.

(37)

− If developers produce or sell a product, which integrates OSS as source code, they may need the licensor’s permission. Otherwise, the licensor might claim for damages or force them to end the product’s further development, delivery, and sale.

− Open source systems lack the comfort zone that a commercially acquired solution provides because support is in the form of bulletin boards and the like. The studies on OSS focus on three dimensions:

− Since OSS projects have some typical characters, such as parallel development, fast release, and peer review, some studies investigated how the OSS projects are organized and managed. The intentions of these studies are to summarize the good practice in OSS projects in order to improve the development of commercial software [Feller02].

− Some studies compared the OSS products with COTS products. For example, Paulson et al. [Paulson04] validated several hypotheses about the difference between OSS and COTS products. They concluded that there is still no empirical evidence that OSS fosters faster system growth and there is also no strong evidence that OSS is more modular than COTS software.

− Other studies concentrate on using OSS in the commercial software projects in order to facilitate the development in the IT industry [Madanmohan04].

2.6 OTS Component-Based Development

The granularity of the component in practice/marketplace is generally the same or larger than the theoretical component defined in Section 2.3. The components following COM, CORBA, and EJB models, or software libraries like those in C++ or Java, have all been regarded as components in industries and component marketplace [Componentsource04].

In our study, we investigated the components with the similar granularity as in industrial practice and component marketplace. However, we exclude platform software, such as OS and DB. The OTS components definition used in our studies are:

− Component: Software components are program units of independent

production, acquisition, and deployment that can be composed into a functioning system. We limit ourselves to components that have been explicitly decided either to be built from scratch or to be acquired externally as an OTS-component. That is, to components that are not shipped with the operating system, not provided by the development environment, and not included in any pre-existing platform. That is, platform (“commodity”) software is not considered, e.g., an OS like Linux, DBMSes, various servers, or similar software. Furthermore, components usually follow some component model, as expressed by the COM, CORBA, EJB, .Net, or Web Service standards, or they can be a C++/Java library.