Evaluation of the RMIAS
4.2 Analytical Evaluation and Analysis of the Interviews
4.2.1 Simplicity of the RMIAS
Simplicity is a subjective characteristic: what is simple for one individual, may be complex for another. Objectively, simplicity may be analytically evaluated against other models. In compari-son with other models (e.g. McCumber’s cube [65], Maconachy et al. [86]), the RMIAS is more complex. The RMIAS has a wider scope than other models and, therefore, it inevitably has more elements and is less simple. However, according to the Ockham’s razor principle [143], the simple explanation or model should only be preferred until simplicity can be traded for greater explana-tory power. It may be hypothesised that the RMIAS has greater explanaexplana-tory power than the other models because it may represent more security issues and solutions, and it also makes the inter-relationships between the IAS concepts explicit. This statement is supported by the analysis of the other models summarised in Tables 2.6 and 2.7 which show that none other of the examined models covers the same range of security concepts as the RMIAS. The detailed analysis of other models of IAS is presented in Chapter 2.
The RMIAS also attempts to cover the full breadth of the IAS domain. As the result of this, in the trade-off between simplicity and completeness, in the RMIAS, the preference is given to completeness.
Despite being more complex than other analysed models according to the analytical evaluation conducted by the author, the RMIAS is considered as relatively simple and easy to grasp by the interviewed experts and even by newcomers to the IAS field as discussed in Section 4.3.2. In order to enhance its intelligibility, the RMIAS is duly accompanied by a narrative. The definitions of every element of the RMIAS are provided and the interrelationships between the elements are explained. The visual appearance also aims to improve the intelligibility of the RMIAS. During the workshops, the RMIAS was presented to the audience which had different levels of expertise in IAS. The feedback from the participants indicates that even the novices to IAS find the model simple and easy to understand. As discussed in Section 4.3.2, the novice participants along with more experienced ones successfully used the RMIAS for the development of an ISPD during the evaluation workshops.
to the use of little or no quotations [146].
In the interviews, there were two questions capturing the opinion of interviewees with regard to the simplicity of the RMIAS:
• Question 4 - Are the elements of the RMIAS simple?
• Question 5 - Are the relationships between the elements simple? (The relationships are illustrated by arrows.)
Twenty-two out of twenty-six interviewees described the elements of the RMIAS as simple. The following comments were received for question 4:
Respondent 6: "They are simple. A very easy way of expressing requirements to key stakeholders."
Respondent 9: "The diagram is quite simple, but also conveys a complex depth."
Respondent 11: "Following the explanation, the model is simple. However, its rather comprehen-sive approach possibly detracts from the optimum level of simplicity."
Respondent 13 - "Yes, it appears clear and easy to implement."
Respondent 19 - "Yes. They appear to be simple at the coarse or more abstract level."
Respondent 20: "The elements of model are simple, but the difficulty lies in how to show them (I think!). The model needs to be viewed from other perspectives for identifying its simplicity. I mean that if the model is four dimensional, can I present it as a series of three dimensional projections that progress over the fourth dimension?."
Respondent 21: "I think that the model, with regards to the dimensions is fairly simple to under-stand. I don’t think it would be as easy to implement because it will highlight such a high number of risks."
Respondent 23: "Yes, the elements are reasonably self-explanatory and straight forward to un-derstand. One comment I have is that visually the diagram gives the impression that you would complete the contents of each quadrant (so to speak) before moving onto the next. However the security development life cycle is not something you would "complete" before moving onto the information taxonomy dimension. In practice, when carrying out security requirements elicitation you would "Consider every stage of information", "Prioritise security goals", and "Select security countermeasures" before moving to the security design stage in the development life cycle. I am not convinced the development life cycle sits as a quadrant within the diagram and should per-haps sit centrally or outside the diagram. Steps 1,2 and 3 in the security development life cycle
are informed by information taxonomy, security goals and security countermeasures, so it feels incongruous to have them in a flow."
Thus, although two respondents (respondents 20 and 23) found the elements of the RMIAS simple, they suggested to change the layout and improve the visual effectiveness of the RMIAS. Respon-dent 23 interprets the role of the security development life cycle in the RMIAS exactly as it is meant, but suggests that the visual appearance of the RMIAS does not convey the view on the life cycle as a time line in the most effective way. In order to eliminate possible misinterpretations of the role of the security dimensions life cycle, the detailed explanation of the role of this dimension and of its interrelationships with other dimensions is presented in the narrative of the RMIAS in Chapter 3.
Respondents 13 and 21, although both agreed that the elements of the RMIAS are simple, had opposite opinions regarding the simplicity of the implementation of the RMIAS.
Only four respondents pointed out at the difficulty to understand the elements of the RMIAS (question 4):
Respondent 3: "To some extent, if the elements are explained."
Respondent 5: "No, the development life cycle is too abstract."
Respondent 12: "It depends on the resources that the enterprise has to understand the model. For an SME, I’d suggest no."
Respondent 26: "No! But then it is a complex area."
Answering question 5, seventeen interviewees agreed that the interrelationships between the di-mensions of the RMIAS are simple. Seven respondents did not see the interrelationships as simple and provided the following comments:
Respondent 2: "The top arrow refers to a category of information. It is not clear what is meant by a category of information."
Respondent 3: "To some extent, if the relationships are explained."
Respondent 5: "It is not clear or intuitive in terms of the flow of the model. The risk analysis, cost-effectiveness and consistency statements don’t seem to connect."
Respondent 11: "I believe the relationships within the elements are simple, but they do not neces-sarily flow between elements logically."
Respondent 17: "Not quite. Seemingly yes, but the relationship are in reality quite complex and may vary depending on the context."
Respondent 19: "Not seen as simple because the relationships could carry different meanings to different people."
Respondent 22: "They are presented as such, but in reality that is not necessarily how it works."
Two respondents were not sure about the simplicity of the interrelationships in the RMIAS (ques-tion 5):
Respondent 4: "They seem to be, but without applying in practice I am not sure."
Respondent 14: "Not sure about the life cycle."
While only four interviewees did not find the element of the RMIAS simple (question 4), nine did not see the interrelationships as simple or were not sure about them (question 5). Overall, the interrelationships between the elements of the RMIAS pose more difficulties for understanding than the elements of the model. While considering the fact that a larger number of the interviewees struggled to understand the interrelationships, it is worth noting that during the presentations the author had only limited time to present the RMIAS and was not always able to provide a detailed explanation on every aspect of the model. Also the interrelationships between security concepts are complicated and may be presented in different form. They may be perceived by security experts differently. The purpose of the evaluation process was to established whether a majority of experts would agree with the representation of the interrelationships suggested in the RMIAS.
The responses for questions 4 and 5 indicate that the majority of the interviewees found the RMIAS, both the elements and the interrelationships, simple. However, in the future further re-search is required into the improvement of the clarity of the visual appearance of the RMIAS.