Basili (2006a) is concerned about the fact that empiricism is considered to be a basic part of scientific and engineering discplines but has not been part of the tradition of software engineering. Basili (1996) says that software engineering should follow other physical sciences in terms of experimentation, this is based on experimentation and process improvement. Basili’s approach to experimentation in software engineering is based on the idea that in any discipline progress is based on an understanding of the basic units that are used to solve a problem and requires models to be built in the application domain and an understanding of the techniques that are available for using these models to help solve problems (Basili, 1996). Likewise, Basili and Zelkowitz (2008) apply an empirical approach in order to understand important issues in software development which requires observation, model building and experimentation. In reference to these approaches, the present study considered observation, however, it was deemed inappropriate because observing participants would affect of the results of the study, especially where interactions between
82
participants are important and are studied. Thus it was deemed appropriate to adopt real experimentation.
Basili’s contribution to the empirical study of software development and software quality have included the Goal Question Metric (GQM) approach which has made measuring software possible, specifically, the approach involves deriving measures from goals, limiting data collection to what is required to answer questions, clearly stating assumptions and using appropriate models for interpreting results (Shull et al. , 2006).
The idea that the concepts and practices of software engineering which are taken from experience and observation should have empirical validation is also supported by Kitchenham and Budgen (2002) who state that validation of such ideas would help to link theory with practices. Kitchenham and Budgen (2002) also say that there is a need for empiricism, especially in academic study of software engineering projects. There are practices in software and software development processes that have received little attention which include quality, design patterns and the usability of UML, all of which are relevant to the present study because it will consider the usability of UMLsec and in reference to quality would consider security requirements.
Basili’s idea that empiricism should play a role towards the improvement of product quality and productivity in software engineering has been demonstrated in a study named ‘Using Measures
83
Safety’ (Basili, 2008). This study looked at the relationship between the processes of software
development and the resulting product characteristics and was based on the assumption that there is a relationship between processes and product characteristics. Moreover, Basili (2008) recognised that although the characteristics of a system cannot be verified during the development phase, measures for the potential characteristics can be made during development. However, the present study is not concerned with measuring the security characteristics of the developed product, it is primarily concerned with improving the development process in relation to security requirements and evaluating such improvement through the perceptions of those who are engaged in product development. Improving security requirements consideration and modelling by the development team, coordination and teamwork are hypothesised to be improved by the introduction of a Security Owner role and UMLsec.
In software engineering there is a need to model different product characteristics, such as reliability, portability and efficiency, and project characteristics, something which a concern of the present study, which include schedule and cost. Basili (1996) emphasises that it is important to understand the relationships between the process characteristics and the product characteristics. More specifically, there needs to be an investigation of what type algorithms produce efficient solutions in relation to certain variables and how certain development processes produce certain product characteristics under different conditions (Basili, 1996). The present study is concerned with the development process and its impact on the product characteristics, specifically in relation to product security. Therefore, these considerations made by Basili (1996) are relevant to this study.
84
This study uses some of the ideas posed by Basili to understand how the participants feel that the development process has an impact on the characteristic of the product. Specifically, how product security is impacted. By introducing new process factors can improve product quality in terms of security. This is determined from the opinions of the development team.
Basili (2006b) describes that in software engineering research there has been a move from studies that are based solely on quantitative approaches to experiments which include pre-experimental designs and case studies, the influence and importance of context variables and the domain in the interpretation of results has been recognised. In this study experimentation will be used to investigate the effect of variables, in this case the inclusion of UMLsec and a Security Owner role on agile development in terms of security requirements consideration. The consideration of the influence and importance of context variables has justified the use of real experimentation.
Kitchenham et al. (2002) is more relevant as it offers empirical software engineering research guidelines that will help to improve both the research and reporting. The guidelines are used in this research to improve research planning, implementation and reporting. The guidelines for experimental design included consideration of the population being studied, this was achieved through a sampling technique which included a pre-experiment questionnaire to identify suitable participants based on pre-established criteria. There is a requirement for the justification of the sampling technique, the selected technique was purposive sampling and was justified (Nachmias, 1981, Bryman, 2001, Denscombe, 2003, Vaus, 2004) in light of the fact that participants had to have a certain level of knowledge and experience in UML and agile methods. The guidelines for
85
conducting the experiment and data collection include the defining of software measures. However, this study is not concerned with measuring the software product but instead is concerned with measuring developers’ perception and feedback of the extended and improved software development process. The guidelines ensure that if the measures are subjective, in the case of this study the subjective opinions of the participants, then the measurement used should be accurate and not be subject to the bias of the researcher. To follow this guideline Likert scales are employed to increase validity and reliability of results, the present study employs a Likert scale in the post- experiment questionnaire.
Experimentation is used to observe the phenomena and an empirical approach is used to validate the success or otherwise of the framework presented in chapter four. The experiments will be concerned with the structure of the Scrum team and the interactions and communications between its members. Related work by Nagappan, Murphy and Basili (2008) conducted an empirical case study about the influence of organisational structure on software quality which will be used to help guide the methodology in this study. Although this study is not concerned with software quality it is concerned with organisational issues such as communication and interaction between the various Scrum team members and how they are impacted by UMLsec and the Security Owner role and how this affects security requirments modelling.