4.2
The research was conducted through five phases and applied a single case study design (Yin, 2003) adapting the convergent parallel mixed method for data collection purposes (Creswell & Clark 2011). The research took place in the following five sequential phases.
Phase 1: Literature review
The literature review (Chapter 2) explored relevant literature concerning UX and contextualised the inquiry. The review was extended to include an investigation on current research concerning UX, the relationship between UX, usability, LMS, and ODL, and outlined the general factors that influence UX. The components for UX were adopted from Hassenzahl and Tractinsky (2006) as the user, the system and the context. The review reflected literature from UX research, research regarding academic lecturers’ experiences with LMSs, and literature regarding the ODL context. Phase 1 answered the first sub research question and presented the components of UX as the user, the system and the context.
E. de Kock - MSc Dissertation
67
Phase 2: Conceptual framework
Phase 2 provided answers to sub research question 2 as a draft conceptual framework that was compiled through the inspection of relevant literature documented in Phase 1 (Chapter 2) and consisted of the following:
A draft conceptual framework was compiled according to the identified UX components (adopted from Hassenzahl & Tractinsky 2006) and potential factors that could influence the UX. The factors were expanded on with aspects that are related to the specific case study, which is the academic lecturing staff using myUnisa as the LMS in Unisa as an ODL institution. An initial draft of the conceptual framework was reviewed by five expert reviewers in the fields of usability, ODL, CPD, instructional design and pedagogy. An academic lecturer that practises online teaching in several subjects, a usability expert and a UX expert were part of the reviewers. See sections 3.3 and 4.5.1.1 for a detailed discussion on the expert evaluation that validated the conceptual framework. These experts were selected from diverse domains in order to obtain more comprehensive feedback from different viewpoints, to perform a deeper evaluation, that is different aspects were taken into account as advocated by Ardito et al. (2005). The researcher conducted semi-structured interviews with four of the five experts to discuss potential confusion or terminology. There was an open invitation to the participants to give their opinion on the applicability of the factors to the certain LMS in the ODL context.
Feedback from the experts was incorporated in the initial version of the draft conceptual framework and is presented in section 3.3.
Phase 3: Data collection
In Phase 3, data collection was done by adopting the convergent parallel mixed method described by Creswell and Clark (2011). A questionnaire was derived from the conceptual framework and used to collect both qualitative and quantitative data. It was distributed within the context of an ODL institution and comprised the following:
A questionnaire (Appendix B) was compiled with the conceptual framework as a source of enquiry. The questionnaire consisted of two sections. The first section was compiled to collect the demographic information of the participants. The second section included ten questions from the System Usability Scale, SUS© (Brooke 1996 2013) which were adapted to enquire about the perceived usability of the specific LMS. The SUS questionnaire was chosen to be used as the usability metric for UX in this research. The SUS was assessed in a study by Finstad (2010) which
E. de Kock - MSc Dissertation
68 showed that the Usability Metric for User Experience (UMUX) and the SUS provided similar results.
An additional 35 questions were added. These questions consisted of the propositions in the conceptual framework that were transformed into questions. The responses of section two were to be made on a 5 point Likert scale. The last question in the questionnaire was an open ended question.
The questionnaire was piloted with three academic lecturers in the School of Computing, and reviewed by the statistician. After amendments have been made as advised, it was sent out to all the academics (1640) in Unisa. Note that not all the academics in Unisa are currently academic lecturers – some are in research or management positions. Unfortunately the specific details regarding the number of academic lecturers could not be provided by the Human Resource office in Unisa at the time of the data collection. The estimate is that there are more or less 1400 academic lecturers.
The questionnaire was sent out and administered over a period of two months.
Phase 4: Data analysis
In Phase 4 the data were analysed independently in accordance with Creswell and Clark's (2011) outline for a convergent parallel mixed method. The use of a mixed method research, also sometimes referred to as multi method research, involves the application of two or more data sources or research methods, in the investigation of a research question (Creswell & Plano Clark, 2007; Goode et al., 2014). In this study, mixed method research refers to the combined use of quantitative and qualitative data collection methods in the same research. The motivation of mixed method methodology is strengthened by the principle of triangulation, which implies that more than one measurement procedure is used when investigating a research problem in order to enhance confidence in findings. The quantitative analysis findings are presented in section 5.3 and the qualitative analysis findings are presented in section 5.4.
Phase 5: Revised conceptual framework
The conceptual framework that resulted from Phase 2 was revised and amended to present an updated contextualised framework to represent the UX of academic lecturing staff in the use of an LMS tool in an ODL institution, to answer the main research question: How can the UX of academic lecturing staff in their endeavour to facilitate courses online with the use of an LMS in an ODL institution be represented?
E. de Kock - MSc Dissertation
69 The research process in terms of the five phases applied in this study is schematically represented in Figure 4.1.
Figure 4.1: Schematic representation of the research process