Constructs and Items - Online Survey-Questionnaire Design, Constructs-Items and Administration

3 Chapter Three: Research Methodology

3.5 Online Survey-Questionnaire Design, Constructs-Items and Administration

3.5.4 Constructs and Items

The review of the extant background-literature provided theoretical discussions of the 25 e-learning factors that were discussed in the literature-review and that form the basis of the constructs-items. This study avoids the common mistake of combining the issue- determination and issue-ranking steps, as often, panellists independently rank arbitrary numbers of issues, which is not a productive way of determining the final list of issues (Schmidt 1997). Besides this, these 25 items are sorted into the four described groups according to research literature for more structure and easier processing in the survey. As outlined before, pre-testing is an essential element to ensure reliability of the Delphi research process, so this was done together with the assigned supervisor from Salford University, prior to the launch of the first and second round. As mentioned, the survey- invitation is sent together with an explanatory glossary (See Appendix II), which provides brief explanations on how each e-learning factor is currently used. This provides a clear definition for the quantitative-qualitative basis for assessing each practice. It also eases the reconciliation and understanding of each various panellist’s feedbacks in the subsequent Delphi circles. The provision of clear understandings of each topic is essential for the data- collection management phase of ranking-type Delphi surveys (Schmidt 1997).

Figure 3.4: Extract of explanatory glossary explaining each of the 25 potential e-learning best practices

The research sought to require minimum effort on the part of the participants and endeavoured to maximise convenience for the experts. Email was therefore identified as the most suitable tool through which to ensure this, eliminating the need to travel as well as associated costs and resources. Despite verbal commitment of participation by all experts and a zero drop-out-rate, the experts had to be reminded regularly due to overdue responses. Given the multiple circles and the additional exchange via phone or email, the Delphi study is more time-consuming for participants, as compared to regular surveys. In line with general Delphi practice, it was tested and assured that no survey response required more than 30 minutes (Schmidt 1997). In addition, the average response of the second round was for example only 11 minutes. Therefore initially during the first contact presented incentives were outlined again in written form during the invitation for the second round, to sustain commitment and decrease dropouts from the busy experts. These helped to convince the experts to participate in this study, where they potentially declined already other study participations included (Okoli and Pawlowski 2004):

- Selection for a limited but diverse panel of well-chosen experts

- Chances for personal learning and takeaways from consensus building

- Additional personal visibility and given early access to the final published PhD thesis and potential future publications, via free copies

General mechanisms for the survey-administration were followed and the online questionnaire was designed carefully considering general survey design principles. In such way, the administration complied with the recognised and presented ranking-type approach, including an upfront brainstorming (in this case via literature-review) and gathering of relevant factors followed by a narrowing down and ranking to reliably identify most important and feasible topics (Okoli and Pawlowski 2004). Complying with best practices from literature, the ranking-type Delphi approach is executed in the following three steps (Schmidt 1997):

- Data collection management:

- Issue discovery (Researcher to provide single list with consolidated definitions) - Most important issue determination (Paring into groups)

- Issue ranking (Panellists to rank all issues, that are provided in random order) - Data analysis (Via suitable statistics-calculation)

- Discussion and data presentation to support the research results

As outlined, the research approach foresees that the chosen experts are presented with the synthesised e-learning aspects to assess and rank feasibility and importance of each aspect in the online survey in order to collect the quantitative results.

Figure 3.5 Screenshot from first round Delphi questionnaire with e-learning practices, presenting two example elements

Likewise the following figure explains the qualitative data collection procedure with a screenshot of the open-statement caption. These comments were also required and thoroughly analysed, as elaborated in the chapter “Analysis of Open Statements”.

Figure 3.6 Screenshot from first round Delphi questionnaire with e-learning practices, presenting the open comment field

The above discussion and screenshots have signposted the survey-questionnaire constructs-items and administration procedures. In addition to this, the following table describes all constructs-items and scales.

Table 3.2 Constructs-Items and Scales of the Delphi study

Construct Importance Scale Feasibility Scale

1. IS Set-up: 1.1. State-Of-The-Art Information Systems 0 (No importance) to 10 (Highest importance) 0 (No feasibility) to 10 (Highest feasibility) 1.2. Distinction of IT: Leading to Sustainability / Competitive Advantage 0 (No importance) to 10

(Highest importance) 0 (No feasibility) to 10 (Highest feasibility) 1.3. Governance / Learning Objects 0 (No importance) to 10 (Highest importance) 0 (No feasibility) to 10 (Highest feasibility) 1.4 Information/ Data Management and Tools

0 (No importance) to 10 (Highest importance)