Now that the participants had attended the training course and the experiments were piloted, the next stage was the experimentation itself. The participants were requested to analyse the two aforementioned problem scenarios and then asked to carry out a number of tasks that form part of the Scrum agile software development method. These tasks included the following:
Organising into roles
Initially before the participants started the experiment there was a need for each participant to assume a Scrum team member role. The participants were left to decide who would assume what role by themselves. There are two main reasons for this approach; firstly, to avoid bias through the researcher selecting the participants, and secondly, to avoid ill feeling or disappointment among participants if they are assigned an inappropriate role according to their skills or experience.
Consider the security related problems / issues
The participants are presented with the two scenarios presented in the above and have to identify the security related issues for each scenario. This activity forms the initial part of the examination of whether or not the presence of the Security Owner role will have any effect, positive or otherwise, on this particular activity, specifically it answers the hypothesis that the presence of a Security Owner role helps the Scrum team to consider security issues for a given system.
168
Two simple scenarios were conducted by two different groups; these formed the experiment to evaluate the role of the Security Owner. One of the groups included a Security Owner role and the other did not, this was for comparison purposes. Each scenario experiment lasted for approximately twenty minutes, detail of the two scenarios can be found in the Appendix. All team members took one of these scenarios (e-exam system) and apply the proposed techniques for example playing the security poker game learned previously. During the scenario experimentation they presented some agile UMLsec diagrams on a flipchart. The end result of the teamwork in the scenario was a refined Product Backlog (all items are prioritised according to importance and return-on-investments, security items are included in this stack as well. The top of the stack is the most important one).
Identify security requirements
The second part in the experiment involves the participants establishing the security requirements after identification of the problems has been carried out. Specifically, there was identification and categorisation of security requirements into areas such as authentication, authorisation, confidentiality, access control and message transfer. This part of the experimentation helps to validate the second hypothesis of the study which states that the presence of a Security Owner role helps the Scrum team to identify security requirements.
169
Negotiate security requirements – through security poker game to Prioritise security
requirements
Once the security requirements have been established they need to be prioritised. The reason for prioritisation is that in any Scrum development there are limited resources in terms of time, cost and technical ability and importantly the Scrum as an agile method has to remain agile, which it cannot do if it considers all security requirements equally. This part of the experimentation validates the hypothesis that the Security Owner role will help team members to prioritise security requirements.
Develop security requirements solution using UML / UMLsec
Now that the security requirements have been established and prioritised, the solutions can be developed using UML modelling. It is important to note that one experiment group will use UML for this modelling and the other group will use UMLsec facilitated by the Security Owner role. Therefore, this part of the experiment will validate the hypothesis that UMLsec is an appropriate tool for modelling security requirements when facilitated by a Security Owner.
Represent security requirements using UML / UMLsec
In order to facilitate discussion and communication about the identification and prioritisation of security requirements and proposed solutions, for each of the two experiment types a circle group was formed. Once the aforementioned tasks were complete and analysis and design artefacts are complete quantitative/qualitative analysis of the participants’ experiences was conducted.
170
Bias
It is important to acknowledge any bias in the experiments. The participants are not told prior to the experimentation whether or not there will be the presence of UMLsec or the Security Owner. In reference to threats to validity, the experiments are conducted within the same experimental conditions, namely; in universities in Saudi Arabia and to reduce this threat to validity the experiments could be conducted in different conditions such as with professionals in the UK.
Post-experiment Questionnaire
To investigate the participants’ evaluation of the experiment ‘Modelling Security Requirements by UML in Agile Development Methods’, a questionnaire was conducted. A half an hour pre- prepared questionnaire guided session was conducted with the participants and their experiences were recorded for detailed analysis. This step evaluates the usefulness and the complexity of our addition and we did it to see if the development is according to the user aspirations.
After discussion, some of them agreed that they became more knowledgeable and have awareness of addressing potential security requirements when engaging IT projects. They referred the reason to the debate occurring while playing security poker cards turns to determine the values of security requirements in addition to the stand-up meeting and informal discussion of these items obstacles. Full results of the participants’ experiences are present in the results and analysis chapter (chapter six).
171
5.5 Chapter Summary
This experimentation is to evaluate if UMLsec is beneficial to use within and extend agile development methods, e.g. Scrum development framework including to what extent could be integrated, in accordance with objective one and whether the inclusion of UMLsec retains the agility of agile methods in accordance to research question four.
The experiments sought to see if the inclusion of a Security Owner role will improve security requirements consideration through identifying and managing security requirements, negotiation with Product Owner and supporting the Scrum team in order to improve the Scrum framework in accordance to objective one of this study. In addition, the experimentation investigated and evaluated the effect of the Security Owner role in terms of emphasizing security issues, coordination and negotiation with other Scrum team roles, this is related to research question two. We do this by adopting the current extension of UMLSec for the proposed Scrum agile method and by finding to what extent this exercise would be beneficial.
This chapter demonstrates the evaluation experiment of our research work, its aims, goals and steps. It has mentioned the sample groups of the needed participants and the environment. In addition, it has discussed the training course for those participants as well.
The next chapter will address and analyse the collected data of these experiments in order to assess/evaluate the new approach of modelling with UMLsec facilitated by the new Security
172
Owner role and its effectiveness of the security and setting interest derived by the agile team at the level of modeling security requirements.
173
CHAPTER SIX
DATA ANALYSIS AND DISCUSSION
Objectives
To present analysis of results from experimentation
To verify with analysis of results the hypotheses of the study To discuss the meaning and implications of the findings
174
6. Data Analysis and Discussion
6.1 Introduction
This chapter presents the findings of the post-experiment questionnaire that was conducted immediately after each experiment. Data analysis (chapter 3, section 3.8) of the participants’ perceptions is to test the hypotheses of this research. Both groups, those with and those without the inclusion of a Security Owner role and UMLsec, were questioned about their experience of engaging in a Scrum team situation and considering security requirements. The questions were designed to reveal if there was an effect on the management of security requirements and team work, also in relation to security requirements. This was required in order to verify the hypotheses of this study to determine the effect of the presence of a Security Owner role and the inclusion of UMLsec on security requirements in Scrum. The results include a comparison between the two different groups and correlation analysis.
Statistical analysis is carried out to check the statistical significant difference in the responses to the statements between the group that includes the Security Owner role and UMLsec and the group that does not. This is carried out using the T-test in SPSS. The T-test shows if there is a statisitcal difference in the means of the two groups towards veryfying the hypothesis of this study. Specifically, it will show if there if there is a statistical significant difference in responses from the group that included a Security Owner and UMLsec and the group that didn’t in order to veryfy the hypotheses which check if the inclusion of the Security Owner and UMLsec have an impact in terms of security requirements, teamwork and agility.
175