Qualitative research is a type of educational research in which the researcher relies on the views of participants, asks broad general questions, collects data consisting largely of words (or text) from participants, describes and analyses these words for themes, and conducts inquiry in a subjective, and a biased manner (Cresswell, 2005). The qualitative research approach enables the researcher to better understand and contextualise the problem from the point of view of the participant. It includes interviews, case studies, and ethnographies – just to name a few examples. Burns (1994) recognises that quantitative research methods can produce findings of little consequence, because it tends to generalize findings rather than take account of peoples’ ability to individually interpret their own meanings. So, what is true for one participant when evaluating a specific aspect of courseware may not be true for another participant. The mixed methods approach of utilising both quantitative and qualitative approaches to the research methodology attempts to address this potential deficiency. The qualitative data utilised in this research includes information from the open-ended questionnaire data, and the courseware evaluation data – which included both interview and web-based feedback).
Whilst the quantitative methodology utilises a deductive process to pre-define sub-themes and address pre-defined sub-problems, the qualitative methodology utilises a primarily inductive process. A Grounded Theory Methodology was utilised to discover emerging sub- themes and issues that arose from the qualitative data. When it came to the questionnaire data there were some pre-existing themes that were addressed, and further insights provided behind the findings of the statistical results were discovered. In other cases new sub-themes emerged, and provided another perspective on teaching and learning online, user-friendly design or learner-friendly design, which the statistical analysis alone did not provided. Furthermore, the courseware evaluation was also analysed according to a Grounded Theory Methodology. Again, some of the same sub-themes from the questionnaire data arose, whilst other emerged. In all cases new insights and perspectives developed on courseware design and development, which the questionnaire data alone does not address. More information about the qualitative analysis is provided in the Quantitative Analysis chapter.
3.10.1 Interview
Interviewing allows for in-depth probing of attitudes and experience relating to participants use of the software (Paas and Firssova, 2004). Proponent of interviewing Irvine Seidman (1998) believes that it provides access to the context of people’s behaviour and thereby provides a way for researchers to understand the meaning of that behaviour. That is, “Interviewing allows us to put behaviour in context and provides access to understanding their action” (p.4). Obtaining the rational behind academic, student and professional’s interpretation of the courseware, greatly assists in understanding how specific aspects of learner-friendly and user-friendly design contribute (or otherwise) to enhanced learning online.
Seidman (1998) lists techniques that should aid in successful interviews, which the researcher attempted to follow, in order to maximise the potential for a more in-depth analysis of the courseware evaluation from participants:
• Listen more, talk less,
• Follow up on what the participant says, • Ask questions when you do not understand, • Ask to hear more about a subject,
• Explore, does not probe (too little exploration can leave an interviewer unsure of the
participant’s meaning in the material he/she has gathered),
• Listen more, talk less, and ask real questions (i.e. one which the interviewer doesn’t
already know or anticipate the response),
• Avoid leading questions (one that influences the direction the response will take), • Ask open-ended questions (establishes territory, and allows participant to take any
direction he/she wants),
• Follow up, does not interrupt,
• Ask the participants to talk to you as if they were somebody else, • Keep participants focused and ask for concrete details,
• Share experiences on occasion,
• Ask participants to reconstruct, not to remember, • Avoid reinforcing your participant’s response, • Explore laughter,
• Follow up your hunches,
• Use an interview guide cautiously, and • Tolerate silence.
The interview included nine open-ended questions, one before the demonstration of the courseware and the rest immediately preceding the demonstration. It is anticipated that these questions may contribute to a greater understanding of issues associated with the
implementation of specific features of learner-friendly and user-friendly design. The starting interview questions are as follows, these were adapted according to the ebb and flow of the interview:
Pre courseware demo:
Definition of Courseware
1. What does the term courseware mean to you?
Post courseware demo:
User-friendly Design
2. In your opinions does this courseware provided user-friendly design? Is it relatively easy to use?
3. Usability features of courseware are not always considered. What usability features would best help you learn?
4. The visual design of courseware is not often always considered. What do you think of the visual design?
Learner-friendly Design
5. What do you think of the amount of instructional guidance? Is it too clear, too much, too little, too vague?
6. What do you think of independent learning? Do you think this courseware can encourage this?
7. Courseware design often just gives notes, courseware should encourage people to think deeper. Do you think the courseware does this to some extent?
User-friendly and Learner-friendly Design
8. When you look at the courseware what appeals to you? What do you think is good, and what do you think needs improvement?
Question nine provided the opportunity for learners to state in their own words, any preconceived strengths or weaknesses that the courseware may have according to them. This enabled the researcher to obtain information on the courseware design, which the questionnaire and courseware evaluation hadn’t thus far obtained.
3.10.2 Participants and the Process
There were ten interview participants, and each of the groups above were represented. In some cases a single participant bridged more than one participant group. For example, one participant was an academic from Other Disciplines (information systems) and a professional web developer. Another participant was a university student from Education, and a professional instructional designer within the secondary education sector. More information about the participants is provided in the Qualitative Analysis chapter. As mentioned above, participants were invited to take part in the interview via email or face-to-face. The letter of invitation instructed participants that the interview would take approximately 30 minutes, not including 15 minutes preparation time. Interested participants were emailed information
sheets, and appointments were booked according to their availability. The interview took place at a time and location convenient to the participant. Prior to the interview, participants were emailed the URL of the courseware and asked to familiarise themselves with the courseware, and to feel free to come to the interview with any design issues that they notice in particular. During the interview participants were instructed that there was no right or wrong answer, and briefed on the interview structure. They were also given a 10 minute demonstration of the courseware, to (re)familiarise themselves with its features. Furthermore, it was emphasised that the courseware is a prototype, and the purpose of the interview is not to obtain a critique of the content, rather feedback on specific aspects of the learning and interface design. The researcher used her laptop to demonstrate the courseware offline.
3.10.3 Web Based Feedback
An invitation was emailed to four educational research listservs, inviting members to view the courseware and email the researcher feedback regarding its strengths and weaknesses. Each of these groups were chosen due to the range and scope of their audience. That is, the groups were purposely oriented towards eLearning design, thus it was anticipated by the researcher that respondents would have experience and an interest in providing relevant feedback on the courseware evaluation. These listservs included:
• IFETS (international forum for educational technology and society, for academics and
professional instructional designers),
• ITFORUM (international instructional technology forum, for academics and
professional instructional designers),
• UTAS_education (Yahoo Group mainly for postgraduate researchers at the University
of Tasmania), and
• Tas-IT Yahoo Group (Yahoo Group for Tasmanian educators interested in IT).
3.10.4 Validity and Reliability
The validity and reliability of the interview may be compromised in two ways. Firstly, if the participant is not given an opportunity to adequately construct and communicate their own meaning (Siedman, 1998). This research utilised the interviewing techniques listed above, to help ensure this does not happen. Secondly, if participants do not take the time to familiarise themselves with the courseware prior to the interview, they may suffer cognitive overload and their feedback may not be as comprehensive had they been familiar with the courseware. The validity and reliability of the web-based discussion is negatively affected if the participant does not take the time to read the splash page, which contextualises the purpose of the courseware, its target audience, and its design features. To help protect against this, the researcher informs the listserv members of these issues, before including the URL of the courseware.