3.3 Introducing Test Driven Development on a FLOSS Project: a Simu-
3.3.5 Validation of Hypotheses
In order to validate our hypotheses we performed a two-sample t-test6 on the two groups of data (nonTDD and TDD). The test assesses if the means of the two distributions are statistically different or not. Since empirical evidence in this research area is not sufficiently strengthened we have used two-tails test.
Hypothesis A: The defect density injected using the TDD practice is different from that obtained without TDD.
Hypothesis B: The total number of LOCs written using TDD is different from the LOCs written without TDD.
Hypothesis C: The number of developers does not change using the TDD.
Table 3.6 formalizes the research hypotheses.
The results of the t-tests are shown in table 3.7. It can be seen that the zscore of HA and HB are greater than zcrit, so it can be concluded that the null hypotheses of HA and HB are rejected at the 95% confidence level.
6We have previously verified that the standard deviations of the two groups were similar and the size of the samples was large enough (degrees of freedom = 198). So we have used the proper formula for large independent samples and equal variance.
3.3 Introducing Test Driven Development on a FLOSS Project: a
Simulation Experiment 61
Table 3.6: Hypotheses to be tested, where b is the average value of the defect density, Locs is the average value of the produced LOCs and d is the average number of contributors to the project.
H0 (null hypothesis) H1 (alternate hypothesis)
HA bnoT dd= bT dd bnoT dd6= bT dd HB LocsnoT dd= LocsT dd LocsnoT dd6= LocsT dd
HC dnoT dd= dT dd dnoT dd6= dT dd
Table 3.7:Results of the two-sample t-test (α = 0.05).
zscore zcrit H0
HA 25.84 1.96 rejected HB 2.30 1.96 rejected HC 0.23 1.96 accepted
On the other hand, the zscore of HC is less than zcrit, so we accept the null hypothesis, and it can be concluded that the two groups do not differ significantly in this respect. This result is aligned with the HC hypothesis, which states that the number of developers is unrelated to the usage of the TDD practice.
Finally, we should emphasize that the zscore obtained for the HB is quite close to its zcritic value. For this reason, even though the sample size is large (n ≥ 30) and the null hypothesis has been rejected, further studies and experiments are needed to validate this statement.
3.3.6 Considerations
We presented a simulation model aimed to understanding whether or not the adoption of the agile practice of Test Driven Development can improve an open source development process. The starting point of our research can be summarized in the following hypothesis:
TDD yields code with superior quality, in terms of defect density per KLOC.
This hypothesis was tested and statistically validated using a simulator of a FLOSS project. We found that the defect density decreased by 40% in the case of TDD. Also we investigated two other hypotheses. We found that the number
3.3 Introducing Test Driven Development on a FLOSS Project: a
Simulation Experiment 62
of developers contributing to a FLOSS project is unrelated to the usage of TDD practice and that code productivity decreases by 9% when TDD is fully adopted.
Chapter 4
Implementing the Agile
Methodologies: three case studies
So far we’ve presented a mix of theories and related works to understand the Agile Methodologies and the issues related to its application in a distributed en-vironment. In this chapter we will describe the results of our research activities related to investigating how the agile methodologies can be applied in a distributed environment. We will show real cases of distributed projects in order to understand if agile practices can be completely applied.
We will describe and analyze three exemplar cases in which the distributed ag-ile development processes have been applied with success, they represent any real projects in order to evaluate the distributed agile methodology which is the main goal of this thesis.
The first of the cases we have studied concerns the application of an agile and distributed development process on a project which has involved a developers team distributed in the academic workspace. This study will best described in section 4.1:
MAD Project. Rather than define how project should use Agile methodologies, we will attempt to explain what has worked for us.
4.1 MAD Project 64
Section 4.2 introduces the second case study: the DART Project. This is a re-search project on Web applications and one of the main goal to achieve was analyze how Agile methodologies fit the needs of Web development. We have introduced also the process metrics and automated testing technics which are best adaptable to the Web processes.
Finally the thirst case study shows the implementation of a Agile methodology, in the specific instance Scrum, to a research project. Can a research project be agile?
In section 4.3 we describe our proposal of applying Scrum for the management of an European research project aimed at developing an agent-based software platform for European economic policy design. The use of an agile, adaptive methodology is justified because successful research projects are complex, unstable processes, which should be continuously adapted along their way. We describe in detail the roles, artifacts and practices of the proposed process, and the first steps of its adoption.
These case studies has been a wrap-up of lessons learned from which we arise some considerations in order to propose a distributed agile methodology.
4.1 MAD Project
Here we describe an experience, performed at the University of Cagliari, pub-lished in [13], [12], in which we try to use the XP process within a distributed context. We will seek to demonstrate through empirical evidence that XP values can be supported by multi-site team and we will present how to redesign some XP practices in order to fit the needs of a software distributed team.
This case study, called MAD (acronyms of “Metodologie Agili Distribuite)”
was supported by MAPS 1 and closed in July 2007. The project has involved our research group (Agile Group of DIEE - Department of Electrical and Electronic
1(Agile Methodologies for Software Production) research project, contract/grant sponsor: FIRB research fund of MIUR. , contract/grant number: RBNE01JRK8.
4.1 MAD Project 65
Engineering), CRS4 (Center for Advanced Studies, Research and Development in Sardinia), the Libera Universit`a of Bolzano, the University of Genova and the Uni-versity of Milano.
This research proposal was about a network of centers of competence on software development methodologies. Such centers, able to supply both computer science and firm organization competence, was specifically targeted to the most modern software development technologies, that is agile methodologies.
The main scientific objectives of the proposal was:
• To study, to be center of competence, and to spread agile methodologies for software development, and in particular Extreme Programming.;
• To study the agile methodologies from the point of view of the organization of the business processes, and the relationships between agile methodologies and quality certification;
• To validate empirically the software development processes, and in particular agile processes;
• To study the application of agile methodologies to software development made by programmers distributed along the Internet, with particular care to open source software development.
MAD Project is a case study included in the research project mentioned above.
An outline of the case study’s goals follows:
• To develop specific competence in software development among the students of the academic course in Electronic Engineering throughout the targeted ed-ucation and learning in the field by applying software development with the support of the coach;
4.1 MAD Project 66
• To validate a “Distributed Agile software development Model ” defined in itera-tive manner during the project and implemented in the development practices.
The development monitoring and the software measuring allowed to validate it;
The real project consisted of the development of a web portal, integrating ser-vice and handling contents through a contents management system (CMS), for the University of Cagliari.
The software development process was made up of two phases.
The first one was the development of the kernel of the system and has been carried out by a co-located core team. In this phase the system has been developed following the classic XP practices [6].
In the second phase the system has been released as an open source project, and XP methodology has been redesigned to adapt it to a distributed environment.
Moreover the project saw also a phase called “zero”. The aim of this phase was to bring the developers to common information level on the required elements in order to partecipate to the project productively. We carried out several activities (workshops, lesson courses, and so on) in order to provide a specific knowledge:
• Methodological Activity: with reference to the software development method-ologies to adopt;
• Technological Activity: with reference to specific tools and environments, such as Java, Eclipse and Sourceforge, to learn.
• Specific Activity: with reference to the installation and advanced use of Open Source projects on which the software development phase must base it.
The main goal of this experience was to understand how the pure XP approach evolves while passing from the first to the second phase. We have investigated how
4.1 MAD Project 67
the values and practices of XP must be modified, removed or tool-supported as the first co-located team becomes dislocated.