The participants are to take part in a scenario experiment where they were engaged in a scenario where they have to resolve an issue within a simulated agile development situation. Participants will be presented with two different scenarios, one will be using agile methods to develop software and will focus on the security aspect of the development, the other will be the same as the first but will include the addition of UMLsec and the Security Owner role, the former being the control experiment. Detail of the experiment is provided in Chapter five.
103
3.7.1 Justification for Real Experimentation
The experimentation of the Scrum method with and without the inclusion of the Security Owner is something that clearly involves experimentation of the interactions that take place between team members and between team members and the Security Owner. This was a consideration of the approach to the experimentation, specifically, whether to select a simulated online experiment or a real scenario experimentation, the latter was selected and a justification is provided here. This interaction and collaboration that is taking place in a software development scenario includes body language, tone of voice and facial expressions as part of the interaction between team members. Moreover, in reference to the principles of agile development one of the principles is that the most effective and efficient way of relaying information to and within the team is face-to-face interaction. Although these are not measured in the study, they are essential as part of the simulation of a Scrum team development including all its associated interactions and process in a real time, real life situation. The study aims to see the effect of the inclusion of the Security Owner and UMLsec, which are factors that interact in this simulated, real time situation and therefore, a simulation may be more appropriate than an online simulation where real human face-to-face interaction is lost. Here the various issues related to the decision as to whether or not real experimentation instead of online experimentation should be used are addressed.
Dandurand et al. (2008) say that one of the disadvantages of doing online experimentation, as opposed to lab methods, is that there can be a high dropout rate. Nedic et al. (2003) say that although there has been numerous software packages designed for the simulation of real experiments, they are not as effective as undertaking real experimental work and they are a poor
104
replacement for real lab work. Moreover, Nedic et al. (2003) also say that the lab environment provides opportunities for testing conceptual knowledge, working collaboratively and learning by trial and error, an another interesting issue raised by Nedic et al. (2003) which would be relevant to the present study is that in real experimentation there are time and physical boundaries, the present study aims to introduce a new role that will interact with existing Scrum team members and that this human interaction will take place in an agile development framework that has time and physical boundaries.
One of the main disadvantages of web-experimentation is that there is a reduction in the experimental control, because all of the participants are at their own location there might be environmental noise or distractions and we do not know the characteristics of the participants or their psychological states of mid (Pollanen, 2014). In the lab experiment situation the conditions can be controlled and the lab environment can be polished, scrutinised and tested.
3.7.2 Sampling for Experiment
It is preferable that the number of members in an agile team does not exceed 12; the minimum for the experimentation is five members. In light of this the number of participants ranged between 5 and 11 for the experiments. Moreover, it was necessary to have approximately half of the experiments without the inclusion of the Security Owner and UMLsec and half the experiments with the inclusion of the Security Owner and UMLsec. Chapter 5 provides more details of the actual sample numbers for each experiment.
105
The identified participants completed the questionnaire, the responses to which will identify if they are suitable for the study, the criteria for selection has been described in section 3.6.3.3.3.
This study is concerned with the perceptions of the team members working in an agile software development method, namely, Scrum using a survey approach. Similar studies have addressed developer perceptions of using agile methods. Mannaro et al. (2004) surveyed developer perceptions of agile methods in software companies, their study adopted a questionnaire distributed to 122 participants. Santos et al. (2011) assess the perception of agile team members about how agile methods contribute to the quality of software, their study used questionnaires with 109 participants. Vijayasarathy and Turk (2012) look at the drivers for the use of agile as a development process from the perspective of the advantages and disadvantages, they used an online survey with 98 participants which they stated was a reasonably good size. Therefore, in consideration of these studies and the fact that real practical experiments were being conducted and not online surveys, that the number of participants for the scenario experiment is 110.
3.7.3 Preparation Training
The training course will prepare the participant for the simulated scenario experimentation. It will familiarise them with software development using agile methods and modelling with UMLsec and explain to them the procedures for the experiment. Details of the experiment training are provided in Chapter five.
106