Data Analysis Approach

Qualitative data analysis involves systematically searching and arranging the collected data through various methods within a study (Bogdan et al., 1998). According to Blaxter et al. (2006), the problem in qualitative data analysis is that it is accomplished mainly with words, not numbers as it usually embodies multiple meanings. It is important for the researcher to realise that there are multiple alternatives and practices to analyse social events, especially in qualitative data analysis. There is no single way or methodological framework well formulated to analyse qualitative data (Punch, 2005). According to Cresswell (2009), data collected through various methods, including interviews, observation and course documents, should be brought together by bringing some meaningful description, or in a summary form, and later by highlighting significant findings. He further suggested to store and organise the summarised data in a personal computer and back this up in digital media for analysis and descriptive writing.

Miles et al. (1994) suggested for the researcher to code the data and count codes to identify the frequency of similar codes appearing in the database. According to Zhang et al. (2009), this approach seems quantitative in the early stages, but the objective is to explore the usage of keywords in an inductive manner. Coding involves the process of data dissecting and providing labels to units of meaning, which helps the researcher to pool ideas, to cluster and later draw conclusions (Hurworth, 1996). Hsieh et al. (2005) described the process as: “A research method for the subjective interpretation of the content of the data through systematic classification process of coding and identifying themes or patterns” (p. 1278).

The literature suggested the use of computer software applications, for example, NVIVO, which is a systematic way to code the data, categorise codes and identify themes (Bazeley et al., 2013). For this research study, the researcher employs Wolcott (1994) three steps of data analysis: description, analysis and interpretation.

3.4.6.1 Description

The description is the initial phase of data analysis in this study. Activities include transcribing the audio, summarising field notes and integrating these with the course documents. This phase begins with transcribing the audio. The audio data is transcribed soon after it is collected. The process of collecting the data takes four months to complete, and it is fully transcribed nine months after collection.

89

The researcher utilises Microsoft Windows Media Player 10 for transcribing the audio- recorded interview. As English is not the researcher’s first language, the researcher requires approximately six hours to transcribe the 30 minutes of audio-recorded data. Examples of the audio transcript are presented in Appendix 2.

Miles et al. (1994) suggested using memos to help to tie together pieces of data into an identifiable cluster which later may create the general concept. Thus, the researcher takes the suggestion to combine ideas that emerge from the audio transcript of interviews and course documents. The initial emerging findings are sketched on paper using simple diagrams and tables to make it significant.

3.4.6.2 Analysis

The purpose of this phase is to reduce the pool of data. This phase is conducted in a two- stage process. The first stage involves coding and categorising, while the second stage consists of writing narratives of the participants’ (students’, lecturers’/mentors’ and employers’) experiences. The narratives of the participants are presented in both case studies in Chapter 4 and Chapter 5.

3.4.6.2.1 Coding and Categorising

Coding is a tedious process, during which patterns and themes are identified to represent the significance and meaningfulness in data (Patton, 2002). Hurworth (1996) described that computer-aided analysis helps to cut out most of the drudgery, provides systematic organisation of data, offers flexibility and permits complex testing of ideas. Thus, after careful consideration of the pros and cons of utilising computer-aided analysis, it is decided to use NVIVO software application for managing the sources, coding and clustering the interview data. It was claimed by the NVIVO developer that the software provides researchers with a set of tools to manage data, manage ideas, query data and transform it into a graphical model. The researcher employs Miles et al. (1994) bottom-up and the top-down coding approaches for coding interview data for this study. Bottom-up coding involves coding the data from scratch using key ideas that emerge from the data. On the other hand, top-down coding involves using ideas from the literature and codes that are developed during the bottom-up coding process.

Initially, a few interviews are manually coded using the bottom-up approach. The codes that emerge during the manual coding are used to code the rest of the data by using NVIVO software. However, the researcher does not limit the analysis to the initial coding, in case new themes or coding emerge from the data, revisions and refinements are needed.

90

Figure 3-4 shows the NVIVO project file that is used for analysis of the data collected for this study. On the left-hand side of the figure (navigation view), the sources of the data for both case studies, such as course documents and semi-structured interviews, are displayed. The highlighted item “PBL_Students” is elaborated in the main section of the figure (list view), which indicates the total number of codes present in each student’s interview and the total number of references made to those codes. The audio and transcript files for the students’ interviews are also displayed. In addition, the date of creation and the date of modification appear in the list view.

Figure 3-5 illustrates some of the sample codes and categories that emerge during the coding process. The NVIVO software provides detailed information on the number of times a particular code for different sources are used and the total number of references referring to all the sources. Referring to Figure 3-5, the codes that correspond to the categories of the lecturers’ PBL processes are presented. A blue rectangular box highlights the code “Reflection”. This code summarises the last stage of information from the PBL process, where the lecturer has given students something on which to reflect; it verifies the information, reflecting on the learning outcome and summarising the learning. This code is established from five different sources and is referenced eight times in those five sources.

3.4.6.2.2 Narratives

At this stage of analysis, the process starts by obtaining feedback on initial ideas and making metaphors. As the researcher has adopted a multiple case study approach in this study, it is worth noting Miles et al. (1994) argument: that the meaning of the data collected from individual interviewees tends to get lost during the process of coding. Further, Miles et al. (1994) recommended employing case analysis meetings, in order to combat this problem. Therefore, the researcher meets his supervisor once a month for at least one hour to summarise the current status for each group of participants. Each of the questions raised during the meeting reflects the researcher’s way of thinking and helps refine the findings. The narratives of the participants (students, lecturers and employers) for both case studies are presented in Chapter 4 and Chapter 5 respectively. Although the narratives give an account of what the students and lecturers have experienced in HE concerning the assessment of generic skills, it is necessary to understand the implementation of active learning settings to gain a broader sense of their learning and teaching. Besides, it is also important to distinguish the graduates’ performance of their generic skills from the employers’ perspectives. Therefore, it is necessary to proceed to the last stage of analysis: interpretation.

91

92

93 3.4.6.3 Interpretation

The final stage of data analysis involves activities to check and refine the codes and categories. By systematically identifying similarities and differences across cases, Miles et al. (1994) described cross-case analysis as allowing the researcher to identify negative cases to enhance a theory, improve generalisability or apply to other similar settings.

Therefore, in this study, the cross-case analysis is used to compare and analyse similar patterns that emerge during both case studies. The patterns which emerge during the cross- case analysis are analysed with particular reference to the research question of interest. In particular, the cross-case analysis is used to identify patterns while developing the generic skills with its assessment. These patterns are then used in understanding the practice of the assessments that emerge from 16 students and 14 lecturers. Similarly, patterns from ten employers help the researcher to identify and update the attributes that represent the intended generic skills gained in HE.

Miles et al. (1994) recommended displaying data in the final report. The display can be generated by hand or even by using a computer program, either of which may help to organise data and motivate thinking (Hurworth, 1996). Hsieh et al. (2005), for instance, suggested that a tree diagram may organise the categories into a hierarchical structure. Figure 4-2 and Figure 4-7 presented in Chapter 4 are examples of diagrams that are generated during this phase of analysis. The key ideas that emerge during this phase of analysis are used as the basis for the researcher’s ongoing discussion of his findings in Chapter 7.