4.1. Research design
4.1.5. Evaluate and analyse the data
In this section I will elaborate on what strategy has been used to analyse the case study and details of how data from each instrument have been analysed and presented.
According to Yin (2009; p 132) analytic strategies, in my research “Developing a case description”, has been adopted. In my case study there were no research propositions, as it is not yet clear what effect the intervention might have. Tellis (1997) states that “if theoretical propositions are not present, then the researcher could consider developing a descriptive framework around which the case study is organized”. The analysis and presentation of my case study will start with the main outcome of the case study (knowledge construction) to be explained in related to the unit of analysis (quantitative content analysis).
This is followed by analysis of data gathered from mixed quantitative and qualitative research (questionnaires and interviews) central to the entire case study, to show the whole picture(R Q 3-9) (Figure 22). According to (Creswell and Clark, 2011) mixed methods can be implemented concurrently (at the same time) or sequentially (different phases). In my case study, all data were collected after the end of the Growth and Development block exam, by when participants’ opinions were well formalized, particularly after attending the exam.
Figure 22: Embedded units and entire cohort data analysis
4.1.5.1.
Instruments data analysis and
presentation
In this section, I will explain how raw data from each instrument have been analysed, and the process of analysis, and how results will be presented.
R Q 1: Content analysis of the posts (knowledge
construction and social presence) process and
presentation
Posts on the online discussion forums have been saved in PDF files and then used in NVIVO for coding of the two purposes, knowledge construction and social presence.
First, indicators and categories have been created as nodes in NVIVO system (Figure 23, and Figure 24).The posts were coded based on the evaluation model. For instance, in the knowledge construction model, the whole post is the unit of coding (Gunawardena et al., 1997), while in the social presence evaluation, model coding is based on the sentence (Rourke et al., 2001).
Figure 23: Example of knowledge construction model’s phases
Figure 24: Categories and indicators of social presence model
The quantity of codes of the two evaluation models were represented in bar charts, with numerical representations based on each evaluation model’s coded unit. For instance, in the knowledge construction evaluation model, numbers represent how many posts have been coded out of the total posts (Gunawardena et al., 1997). Whereas in the social presence evaluation model, the number represents how many sentences have been coded under the model categories (Rourke et al., 2001).
R Q 2, 3, 4, 5, 6, and 7: Questionnaire and
interview data analysis and presentation
A. Questionnaire data analysis and data presentation
Data collected through questionnaires was first stored in the place they were administered. For instance, students’ questionnaire responses were saved in the VLE (Moodle) system. At the time of analysis, data were exported to Excel (Microsoft office software). The exported spreadsheet showed responses in words (i.e., strongly agree). I have replaced them with numbers (e.g., strongly agree=5) to be readable by SPSS, which is a quantitative analysis software program. The tutors’ questionnaire was distributed using a Google form. Data was stored similar to tutors’ responses (in words) in the VLE, therefore, words transferred to numbers before analysis using SPSS.The data has been analysed at the level of exploratory data analysis, applying descriptive statistics (Rugg and Petre, 2007; Cohen et al., 2007). Since the objective is to identify participants’ perception, the questionnaires were designed to collect their opinions. In other words, the objective was to gather participants’ opinions of the intervention, so questionnaires were descriptively designed and afterward descriptively analysed.
I have calculated the mean/average of participants’ responses to each item. In addition, I considered frequencies, representing, for instance, the number of students who found the integration helped them to understand the weekly problem. Finally, data were presented in tables, and striking results were further interpreted in text. Each dimension was represented separately, and in each dimension there are two tables. One shows female students’ responses and another table shows male students’ responses for comparison; in a separate section, tutors’ perception was presented.
B. Individual interviews data analysis and data
presentation
The interviews were conducted in the tutors’ offices at Qassim Medical School. They were conducted in English, as all speak English fluently. Immediately after the interview the recorded interview file was uploaded to
the Leeds University server, in ‘N drive’, in a folder that required a password and could only be accessed by the researcher.
The first step after conducting the interview was transcription (transferring data from audio recording to written text). Audio recordings were transcribed by a professional who is not related to Qassim Medical School or to any one of the tutors. I have listened to all interviews and read all transcripts before analysis commenced to ensure a reliable transcribed text. Listening to the recording and reading the transcript of the whole interview is highly recommended (Wolcott, 1994; Cohen et al., 2007; Robson, 2011). It is necessary to familiarize oneself with data and note down ideas and data will “speak for themselves” (Wolcott, 1994, P 13)
Since it is an exploratory case study, the data of the interviews has been analysed adopting thematic analysis, which is one of the most common approaches in qualitative data analysis (Bryman, 2008). It is used as a realistic methods to report meanings, experiences and reality of participants (Robson, 2011). Researchers perform thematic analysis in one of two ways: either they start analysis without predetermined themes (inductive thematic analysis) (for example in grounded theory), or the themes are determined beforehand, from the literature or based on the research question (deductive thematic analysis) (Robson, 2011; Wilkinson and Birmingham, 2003).
Based on the research questions and theoretical framework it was necessary to explore these areas that may affect the quality of the intervention and then affect the interaction of a participant. These areas are the themes that need to be explored. After the themes were determined, an opinion of two researchers (my supervisors) was considered to maximise the trustworthiness of the interviews analysis. The areas/themes were participants’ perceived satisfaction and learning and training towards the intervention. Thus, the present general and broad themes were expectations, training, advantages, limitations, motivation, interaction/collaboration and impact. Braun and Clarke (2006: p 12) remark that inductive thematic analysis “would tend to be driven by the researcher’s theoretical or analytic interest in the area, and is thus more explicitly analyst-
driven”. Additionally, subcategories were developed during the analysis. Despite the plan to determine the themes in advance, analysis was flexible so as not to neglect new information from the participants. Robson (2011) notices that predetermined themes might bias the researchers toward one aspect of the data and cause them to ignore others. This limitation has been considered in my interviewing process by increasing flexibility. To ensure flexibility and enhance reliability, an independent person (who re-coded the discussion forums) reviewed the transcript and the codes.
After transcription, all interviews were brought together in one NVIVO file. This helped in handling the data and retrieving quotes from different interviews under one theme quickly. Using NVIVO made the management and interpretation of data more efficient (Weitzman, 2000).
In NVIVO, a node is a theme (e.g., training in Figure 25), a group of quotes/data in one subcategory is a code (e.g., clarity in Figure 25), and whatever was presented in the transcript as having the same meaning was coded under such subcategory (Rubin and Rubin, 2012). This helped focus on the details. Figure 25 shows an example of the hierarchy of coding regarding part of a quote from a tutor.
Figure 25: Themes, codes and quotes in NVIVO
Finally, themes, subthemes/subcategories and quotes from the interviews were presented with their interpretations in a descriptive manner, including
comparisons between tutors’ responses if they were found. Overall, the development of the question and handling of the data from interviews in this study was led by the primary aim of using interviews to provide in-depth understanding of the tutors’ perception of the intervention. The analysis of interviews was mostly guided by the aim proposed by Rubin and Rubin (Rubin and Rubin, 2012), that the goal of interview analysis is “to find themes that both explain the research arena and fit together in a way that a reader can understand “
o Focus group data analysis and data presentation
I have analysed the focus groups using a similar approach to that which I have applied in analysis of the individual interviews: thematic analysis. Morgan (2008, p 354) states that focus groups "show many similarities with individual interviews”. However, the process of analysis was different.
Interview audio recordings were transcribed in Arabic. Male students’ interviews were transcribed without issues. However, it was necessary to have support from one of the interviewees in the female interview to indicate the speaker of each response, as she could recognize names and voices.
Firstly, it was not possible to use NVIVO to handle the transcription of focus groups, as NVIVO does not recognize Arabic. Therefore, Microsoft Word was adopted. I read and listened to the interviews several times before starting the real analysis/coding, to familiarize myself with the material. The general themes have been determined beforehand: expectations, training; advantages, limitations, motivation and impact of the intervention. Microsoft Word has been adopted for coding (comment tool). First, it was coded before translation (appendixes 16). All quotes coded were then organized in a table (appendix 17). Finally, they were translated in English. A translation of a text, according to (Esposito, 2001), cannot possibly reflect the exact meaning in cross-language research. Having realized this issue, and to enhance reliability, I asked an Arabic-speaking professional, who is an English teacher, to review the translated quotes. To ensure flexibility and enhance reliability, an independent person (who re-coded the discussion forums) reviewed the transcript and the codes.
Finally, themes, subthemes/subcategories and quotes from the interviews were presented in descriptive interpretations, including comparisons between students’ responses. Generally, the development of the question and handling of the data of focus groups in this study were led by the main aim of using interviews to provide deep understanding of the students’ perception towards the intervention.