Data Analysis

After the identification of the philosophical position and definition of strategies for exploratory-explanatory studies as well as (mixed) data collection methods and the research instrument design, the final section of the research methodology chapter describes the process of the appropriate data analysis mode and interpretation choice. Overall, no standard approach in science exists, but of course the choice has to be consistent with the defined research philosophy and strategy (Schmidt 1997). The analysis was conducted in electronic form and involved according to the mixed method both qualitative and quantitative data. For the data analysis of Delphi studies as well other studies decision- criteria must be fixed to govern the gathering and processing of the feedback, which especially relates in this case to consensus determination, which can be interpreted differently (Hsu and Sandford 2007).

Depending on the level of feedback and consensus, it has resulted that data of two rounds has to be processed. This has led to 28 pieces of feedback being returned via the online- tool of esurveycreator.com in the form of data recorded via the filled out questionnaire for subsequent analysis. In preparation for the next round, the ranking and mean of the previous round (e.g. importance of outsourcing is rated on average 4.8 out of 10 and ranked as number 25) is analysed and provided in the next round’s questionnaire as a statement of the whole group to stimulate reflection and consensus amongst participants. Particular comments returned are also added as anonymous quotes during the second

round for the same purpose as the mean indication. The discussed comment field additionally provides the potential of adding general further comments that are not related to any of the 25 topics. Although software is available for analysis of mixed data (e.g. NVIVO) this is not required for the limited amount of feedback that has to be processed. The suitability of the Delphi technique for a variety of statistical analysis techniques to interpret the data and a thorough execution of these statistical analyses ensures not only consensus but also that individual opinions are well represented in the final step, as a significant spread of opinions may persist (Hsu and Sandford 2007). Also, this approach allows an impartial and objective analysis and further diminishes the described issue of conformity due to group pressures (Hsu and Sandford 2007).

For the data analysis, it is important to understand the meaning of each derived value, like with the usage of Kendall’s W, to assess in a realistic manner if any consensus is achieved, if it is increasing and to determine the relative strength of agreement (Kendall and Gibbons 1990). In line with the literature, a value of Kendall’s coefficient of concordance (W) that is 0.7 or higher indicates consensus on a strong level (Schmidt 1997). Values lower than 0.7 were found in this study’s Delphi research, which still represented significant results for agreements, although being relatively weak. Table 3.3 presents an indication for the understanding of Kendall’s W values, but it does not provide strict endpoints.

Table 3.3 Kendall’s W Value Interpretation (Kendall and Gibbons 1990)

W Interpretation of Agreement Confidence in Ranks

0.1 Very Weak None

0.3 Weak Low

0.5 Moderate Fair

0.7 Strong High

0.9 Unusually Strong Very High

Kendall’s W for determining present agreement in a simple way, has higher popularity than alternatives like those assessing for example, the least absolute distance solution, which establishes various results that are all practical but therefore impractical for Delphi studies (Armstrong et al. 1982). The Kendall’s W and other statistical measures were calculated manually with the help of Excel and the guidance of related literature (Bortz 2013). The results were also carefully crosschecked with the help of the online program from Statstodo (Statstodo 2015).

This is the squared difference between total grand mean and the individual mean ranks (Di²=Ri-R)². This is also calculated and provided in the results chapter for reference, however focus is set on Kendall’s W figures.

Besides the previously discussed mechanisms for the study-administration, the usage of the electronic “fast” media internet and email supported an acceleration of the study and particularly also the data analysis before, between and after the Delphi circles. This reduced turnaround times and achieved a completion well within regular lead times of about 45 days to five months (Delbecq et al. 1975).

Again, the literature review has already identified key themes and patterns and the development of theories based on these patterns. These findings were already synthesised during the content analysis of the literature review and will be further compared, contrasted and validated by the Delphi study. The data analysis will afterwards ultimately support the drawing of conclusions that will be defined and discussed in the subsequent chapters. Delphi literature states that three iterations or cycles are sufficient, which is in line with the approach of this study’s Delphi research that consists of two rounds plus the extensive literature review that worked as round zero (or “one”) by forming the initial basis for the first structured questionnaire (Hsu and Sandford 2007).

3.6.1 Variables

The following table provides a list of variables defined for this study. This is in reference to the raw data provided in appendix V, and allows anyone to repeat this experiment and calculate values for these variables. This definition makes the experiment reproducible. Table 3.4 List of variables (n=sample size)

Variable Description Coding Value Definitions

Project Management

Response of judge about their Project Management expertise. Of the 14 judges, 9 judges had PM expertise.

PM Expertise Yes = 1 Expertise No = 0

Knowledge Management

Response of judge about their

Knowledge Management

expertise. Of the 14 judges, 6 judges had KM expertise.

KM Expertise Yes = 1 Expertise No = 0

Virtual Teamwork

Response of judge about their Virtual Teamwork expertise. Of

the 14 judges, 6 judges had VT expertise.

Expertise No = 0 E-learning

Management

Response of judge about their E- Learning Management expertise. Of the 14 judges, 5 judges had EL expertise. EL Expertise Yes = 1 Expertise No = 0 Information System Set- Up

Response of each judge about ranking of the individual IS Set- Up topic in terms of first importance and second feasibility. Six potential best practices from e- learning derived from the area of information system related risks listed. 1. IS Set-up (6 sub- sections) No Importance/ Feasibility = 0 till Highest Importance/ Feasibility = 10 Information System related Trends and

Response of each judge about ranking of the individual IS- related Trends and Potentials in terms of first importance and second feasibility. Five potential best practices from e-learning derived from the area of information system related trends and potentials listed.

2. IS-related Trends and Potentials (5 sub-sections) No Importance/ Feasibility = 0 till Highest Importance/ Feasibility = 10 Information System related Risks

Response of each judge about ranking of the individual IS- related Risks topic in terms of first importance and second feasibility. Two potential best practices from e-learning derived from the area of information system related risks listed. 3. IS-related Risks (2 sub- sections) No Importance/ Feasibility = 0 till Highest Importance/ Feasibility = 10 Management Theory, Models, and Management of Social Aspects

Response of each judge about ranking of the individual Management topic in terms of first importance and second feasibility. Twelve potential best practices from e-learning derived from the area of management theory, models, and management of social aspects listed. 4. Management (12 sub- sections) No Importance/ Feasibility = 0 till Highest Importance/ Feasibility = 10

In this Delphi study, variables are both categorical 0/1 and ordinal 1-10, and the categories are without an intrinsic order. There are no transformations. These variables presented are reviewed in the following results and discussion chapters.

3.6.2 Analysis of Open Statements

The following section elaborates on and explains the qualitative data analytical steps that were guided by the qualitative research literature (Denzin and Lincoln 2011). Although the Delphi study is mainly quantitative, it also contains highly relevant qualitative aspects. The related qualitative results, retrieved from the mentioned open comment-field, are also systematically evaluated for both rounds individually. This is done by clear coding of the experts’ comments and a categorisation of all comments into the four expert-fields (PM, KM, virtual teamwork and EL) ((Denzin and Lincoln 2011). Furthermore a thematic analysis of main themes as well as corroboration is executed for each comment. This is executed and pinpointed in two different steps: initially this is done individually in comparison with the overall results and then in relation to the individual’s expert-groups and their quantitative findings. This is to confirm and give support of the quantitative findings and the general robustness of the study. The thorough analysis of open statements also includes an examination of differences, to highlight potential discrepancies.

3.6.3 Practical and Ethical Considerations

It is prudent that every research is designed and conducted in a practical but also ethical manner. In regard to ethics, this means that data collection and analysis is executed without detrimental effects for the participants and that the study has methodical soundness. Although there are only very limited ethical issues anticipated for the case of this study, an informed consent was gained by means of providing the necessary information. As such, the researcher endeavoured to outline not only the purpose of the study to the participants, but also the intended outcomes, importance of their participation and how this is likely impacting the overall findings. This was not only facilitated through letters, emails and associated documentation, rather further clarifications were made during, pre-call discussions with the panellists of experts, prior to gaining their written acceptance for the study. Furthermore, the decision to withhold personal details and thus grant the experts anonymity prevents any anticipated ethical problems. Lastly, the ethical approval was obtained upfront from the University of Salford.