Delphi Method

6.4.1 Delphi Method Analysis

The Delphi method was used to confirm empirically the MERIT IT Risk Awareness conceptual model. Empirical data on the variables of the model were collected from expert departmental managers and IT managers in the rounds of the Delphi method. This was done to validate empirically the variables stemming from the MERIT IT Systems Risk Awareness conceptual model obtained from the review of the risk management and IT risk management literature.

This empirical validation strengthened the model’s application to evaluate IT systems risk awareness in organisations. The Delphi method was selected for two reasons. One, it would collect experts’ knowledge on IT risk awareness. Two, it could be applied for policy-making based on the consensus of experts.

The selection criteria for the experts were derived from experiential knowledge of IT risk management and from reading the literature on risk management and IT risk management.

These criteria included (a) five years’ experience in a managerial role, (b) significant risk management responsibility and (c) demonstrated success in risk management, as noted in their curriculum vitae. The experts consulted were IT security engineers, CCTV experts and IT engineers.

The panel of experts independently and separately answered questionnaires in two or more rounds and in each round they should revise their answers based on an independent expert’s summary of their answers. The independent expert, the facilitator, is used to provide an anonymous summary of the experts’ forecast from the previous round and gives the reasons for their judgements. The aim is to decrease the range of answers during this process towards a standard or correct answer. The process is stopped when the predefined stop criterion is reached, usually number of rounds or achievement of consensus. The mean or median scores of the final rounds constitute the results. The method is illustrated in Figure 23.

Page | 132

Figure 23 The Delphi Method communication structure (Source: Okoli and Pawlowski, 2004)

The Delphi method has the following features: panel of experts, facilitator, structured information flow, regular feedback and anonymity of participants (Okoli and Pawlowski, 2004). These features help the participants to focus on the issue explored and provide expert knowledge on the topic under investigation. A panel of experts in the field is chosen and the questionnaire is given to them and the answers collected. There is no stipulation for the size of the panel of experts. In large panels, a panel director manages the interactions among participants by editing the information and filtering out irrelevant content. This helps to structure the information flow. The regular feedback is composed of participants´ own comments on their forecasts, the responses of others and the progress of the panel.

All the panel experts remain anonymous and their identity is not revealed, even when the final report is completed. This curtails social pressure to conform, for example the authority, personality or reputation of other experts may dominate. Anonymity prevents experts from taking defensive postures to defend their original answers. Social pressure is not used to sway experts to the majority or consensus view. This is done by enabling experts to see others’

answers and then revising their own answers. Anonymity thus facilitates free expression of opinions, open critique and admission of errors when revising earlier round answers. It is argued that anonymity also frees experts from their personal biases and minimises the

‘bandwagon effect’ or ‘halo effect’. The role of the facilitator is crucial in the Delphi method.

The facilitator manages the Delphi rounds and ensures that the participants follow instructions. The response are collected and analysed and common and conflicting answers are identified. The facilitator seeks consensus and if it is not reached the process continues until the answers synthesise towards consensus.

Page | 133 Delphi method uses scales to obtain answers. This research used scales. Examples are shown and the complete Delphi questionnaire is given in Appendix E. The Delphi method was used as the following five steps described.

Define the problem

Identify the problem that needs to be resolved. This was done by reading the literature on risk management and IT risk management, formulating the conceptual model and the mathematical model. The variables of the mathematical model required empirical validation to validate the mathematical model. These variables from the 5 mathematical functions of the model were then posed as quantitative questions in a questionnaire on IT risk awareness to the Delphi experts.

Distribute the questionnaire

Identify the Delphi experts capable of addressing the problem. Once the problem is defined it needs to be distributed to the experts. The Delphi group experts were chosen from the researcher’s experiential knowledge and contacts in private and public organisations. The researcher has over 20 years’ experience working in the police force and an active network of experienced managers in private and public organisations. Some of these expert managers and IT managers were recruited to join the Delphi group. The Delphi method does not require meetings, since experts answer the questionnaire individually in separate rounds. The number of Delphi experts chosen was 6, which is considered to be adequate. The questionnaire or problem was sent to the Delphi experts and they were asked to respond. The administration of the questionnaire was done by the researcher, ensuring that Delphi experts had received the questionnaire and prompting them to respond where necessary and that the feedback from the independent expert was communicated to the Delphi experts.

Collate the responses

The researcher collated all the responses into one list. The annotations to the questionnaire in the different rounds were made simple for Delphi and independent experts to read and the annotations were communicated to them verbally to avoid misunderstanding. Since, scale scores were being collated the likelihood of bias was minimal.

Page | 134 Give everyone the collation

The researcher sent the collated answers to the Delphi experts. They were asked to score each item on the given scales in the questionnaire. The Delphi method permits addition of further items as appropriate but no new items were needed.

Repeat as necessary

The Delphi method places no minima or maxima limit on the number of iterations of the process. The aim is to achieve consensus of answers from the Delphi experts. Two rounds were judged to be sufficient because the scores tended towards consensus levels.