The Methods

There are two main methods used for the data collections: 1. Course documents

85 3.4.5.1 Course Documents

Blaxter et al. (2006) defined documents as (public or private, official or unofficial, statistics or words,) artificial and partial accounts, and which need to be critically assessed for research purposes. For both case studies, the course documents are the guidance in a written format for the students to motivate their learning. As the course documents may define the learning objectives, the intended learning outcomes, course contents, learning activities and the assessment approaches that are used during the programme, these documents are collected for this study. The researcher only focuses on the guidelines, that relate to skills of problem- solving, verbal communication and team working. Data, such as intended learning outcomes, learning activities, graduate attributes, assessment tasks and criteria, is collected from the documents. This data is then recorded in the curriculum alignment matrix as shown in Table 4-16 and Table 5-16 respectively.

The process is conducted to ensure the learning outcomes, learning and teaching approaches, and assessments are aligned in a documentation sense. As Biggs (1999b) suggested, the appropriate verbs should be embedded in the assessment tasks and the objectives, so that judgements can be made and presented. This method is utilised to answer partly the research question; (2) How is generic skills assessment implemented?

3.4.5.2 Semi-Structured Interviews

The second method of data collection in this research is semi-structured interviews along with personal observation and informal conversations (Yin, 2003). Interviews attempt to capture the understanding of the world from the participants’ point of view; they seek to understand meaning and to uncover an individual’s lived world and paradigm, prior to scientific explanation (Kvale, 2008). Robson (2011) described that in the semi-structured interview: “The interviewer has an interview guide that serves as a checklist of topics to be covered and a default wording and order for the questions, and additional unplanned questions are asked to follow up on what the interviewee says” (p. 280). According to Cresswell (2009), in semi-structured interviews, the form of questions have the advantage that they are both closed ended, that is, they are easy to make a comparison, and open ended, which gives space for the interviewee to answer within a wider structure.

Semi-structured interview techniques are chosen because of the limited time for the interview to be conducted. The approach helps the researcher to stay focused and aids further discussion regarding any interesting relevant points if they crop up during the interview (Robson, 2002). The researcher conducts face-to-face semi-structured interviews to obtain as much in-depth information and participant understanding as possible; and at the same time it is also, obviously, be recorded precisely and in an unbiased manner. Yin (2003)

86

recommended that the questions of a semi-structured interview should be reviewed by experienced supervisors to check their alignment with research questions and objectives and to enable the reduction of bias during the process. Semi-structured interviews are widely used by researchers to obtain information and perceptions regarding key issues, and are also used to seek recommendations, specifically within generic skills perspective (for example, see Dahlmann et al., 2008; Jones, 2009; Jackling et al., 2010; Mitchell et al., 2011; Clark et al., 2012).

Before interview sessions start, respondents are asked if they are willing for the interviews to be digitally recorded as to maintain the ethical issues in the study. If not, the researcher takes notes during the interview. In addition, the researcher also notes expressions and words that may have useful meanings in the context of the issue being investigated. The recordings are used to transcribe the data. The interview framework is as shown in Table 3-2.

Table 3-2Interview framework (Source: author)

Stage Process

1. a. Interviewing the respondent Signature informed consent b. Background details

2. c. Recording the interviews

 MP3 recorder  Record interviews

3. d. Transcribing

 Listening to the recording  Translating the data  Noting from the recording  Reviewing the notes

 Writing/typing the transcription  Respondents confirming the data

4. e. Reading and coding

 Reading and classifying verse/unit by theme using NVIVO software

 Providing coding

5. f. Analysing the data

 Analysing by theme

 Relating the theme to research questions

6. g. Conclusions and reports

a) Student Interviews

The students’ interviews are done to seek their perceptions on the active learning approach and generic skills assessment, specifically examining those appertaining to their interest, motivation, ability and participation. Student perceptions are stated to have a considerable influence on student learning (Struyven et al., 2005). The interviews are also designed to track

87

students’ understanding, practice and knowledge of generic skills assessment and provide the opportunity for students to reflect on the years’ experiences being assessed by the lecturer. It is used to map whether or not the intended learning outcome, pedagogical approach and assessment have been achieved and completed throughout the module. The students are given an option either to answer in English or the Malay language, in order to capture the findings more explicitly and as it gives the students an increased confidence in their answers. The method is used to answer partly the research questions: (1) How is AL being implemented? (2) How is generic skills assessment implemented? (4) What are students’ experiences of generic skills assessment?

b) Lecturer/Mentor Interviews

Lecturer/mentor interviews are all conducted in the institution’s premises. The interviews allow lecturers/mentors to describe their own personal stories and past experiences of teaching in the AL environment and assessing students’ generic skills. This provides insight into the lecturers’/mentors’ overall understanding and practice of the generic skills assessment and their relationships with students and colleagues.

The interviews are aimed to capture a holistic picture of the lecturers’/mentors’ perceptions of their practices in assessment and their relationships to their students. It also includes the lecturers’/mentors’ reflections on experiences over the past year regarding students’ results and achievements. Their reflections on what they have done in the past, how it leads to what they are doing now and possibly how they want to assess generic skills differently in the future are some of the main aims of the interview outcomes. Similarly, to the students’ interviews, the lecturers/mentors also need to justify whether or not the intended learning outcomes, pedagogical approaches and assessments are aligned to the curriculum. The method is used to partly answer the research questions: (1) How is AL being implemented? (2) How is generic skills assessment implemented? (3) What are the lecturers’/mentors’ experiences of generic skills assessment?

c) Employer Interviews

Selected employers are interviewed, depending on their profile within the case-studies. The aim is to find out the employers’ satisfaction/dissatisfaction with the graduates; problems they face when recruiting potential employees, specifically with regards to generic skills; and up- to-date required attributes in graduates for the current technologies and working environments. A clearer understanding of essential generic skills and professional attributes is important as it is needed in the workplace (Shah et al., 2011; Rosenberg et al., 2012). The aim is also to investigate the mismatch in what students expect to learn in Higher Education

88

institutions, what they need to know, and what they should be able to do in the workplace (Mai, 2012). The method is used to answer the research question: (5) What generic skills attributes do employers expect engineering graduates to possess?