Phase II data collection

This was a ‘non-participant observation with interaction’ of a FD activity at WMS, aimed at understanding the culture and process of FD in a medical school context (setting and objectives), observing FD (delivery and methods) and using interviews to explore the views of participants on FD (reception and perception). I used the postgraduate award ‘Essentials of Clinical Education’ (ECE) course, a three day programme designed for up to 25 participants, held three times a year. The course has 24 hours contact time (8 hours daily), and 175 hours of self-directed learning supported by an online discussion forum for each cohort. Details of the course and the post course assessment are shown in Table 5.3. I chose this course as it was accredited by the HEA at an associate fellow level as described in section 5.2. The ECE course is the first module for the postgraduate certificate, diploma and masters in medical education hence there would be a variety of educators attending the course who would still be contactable six months later. The aim of the course as stated on the WMS website4 was,

“To develop the understanding of education in the field of clinical practice and to begin to apply that understanding in the workplace.”

4

ECE course aim on WMS website:

132

Participants’ views were explored by immediate informal interviews during the course and six months later by formal semi-structured interviews. Informal interviews are casual conversations with informants (without the use of a structured interview guide) while in the field as an observer (Bryman, 2012). I used jottings and brief notes to help in the recall and writing of interview notes. Semi-structured interviews are formal interviews with the use of an interview guide (a list of questions or topics that need to be discussed during the conversation) but as the researcher, I was able to follow topical trajectories in the conversation that might stray from the guide when I felt this was appropriate (Bryman, 2012).

Table 5.3: Details of the Essentials of Clinical Education Course (ECE)

Day 1 Day 2 Day 3

0900 – 1630

Getting to know you: Welcome, Introduction & Overview

On the Job Teaching

Teaching and learning in large groups

Being a clinical educator E-Learning Portfolios for learning and assessment Includes guidance on module assessment How to teach a practical skill Planning a teaching

session

Facilitating learning in small groups

Research and scholarship in clinical education

Strategies for active and

interactive learning

What next? Developing and evidencing teaching practice

Includes portfolio assessment Q&A session Assessment, evaluation and feedback Reflection and

feedback on day 2

Reflection and feedback on day 3

Reflection and feedback on day 1 Module evaluation session

Close

Post Course Assessment

Assessment is based on a reflective teaching portfolio comprising seven key tasks:

Teaching observations: Two teaching observations with reflective comments (one as teacher, one as observer) Giving feedback to learners: Reflective comments on giving feedback to students on at least three occasions. Evaluation: Evidence of collecting and using feedback from students.

Lesson planning: Ensuring effective learning in clinical settings.

Appropriate use of technology to enhance learning: Review of a learning event. Commitment to the HEA Professional Values Statements: A reflective overview. Action Plan: Future professional development as an educator.

133

Phase II data collection was carried out in three steps as described below.

Step 1: Interview of WMS FDC

I carried out a one-hour in-depth, semi-structured interview of the WMS FD co- ordinator on the range of FD activities available, participation, outcomes and the FDC views on the benefits of FD at individual and institutional levels. This was followed by specific questions on the ECE course including the aim / objectives, intentions, evolution of the course, evaluation / outcomes and, more importantly from a realist perspective, what were the possible explanations (mechanisms) for the observed outcomes. This interview was taped and transcribed for analysis prior to observation of the two courses.

Step 2: Observations and Interviews during the course

Prior to the ECE course, I developed an observed engagement scale as discussed in section 4.2.2.1. This was a 5-point engagement scale where 1 represented no

engagement and 5 represented full engagement (Table 4.2). I also designed an observation schedule (Appendix 3d) to record individual and group data as well as keep track of participants during the sessions. The observed engagement scale and the observation schedule were piloted on a different course prior to the ECE course. The course used for piloting (Becoming an Effective Teacher) runs for five days at WMS but I observed the participants over a three day period to simulate the ECE course. This allowed me to test my ability to listen, observe and take notes during the sessions and at break times. I was able to test my observed engagement scale and fine-tune my observation template prior to observing the ECE course.

134

During the actual ECE course, I observed, listened, wrote field notes and carried out interviews of participants during each course in January and April. The observations focused on individual participant as well as the interaction amongst participants (for example at their tables, during tasks and discussions), but the key focus of the observation was on participants. Observation data included session organisation, number of attendees, topics covered, teaching methods, evaluation, facilitators and specialist areas, discussions during sessions and breaks. In other words, data were collected on the physical setting as well as individual and group interactions. Each session was about one hour long and with 22 to 24 participants sitting in groups of threes and fours, observation of each table was carried out at five minutes intervals thereby providing two sets of observations for each individual per session.

I also scored participants’ engagement on the observed engagement scale (Table 4.2); the scale was not given to participants. Engagement was scored using the 5- point scale. When all three categories descriptors (behavioural, emotional and cognitive) were observed a score of 5 (full engagement) was given, 4 (mostly

engaged) if any two categories descriptors were observed, 3 (partial engagement) if only one category descriptor was observed, 2 (minimal engagement) if only one category descriptor was observed only once during the two sets of observations and a score of 1 (no engagement) if no category descriptor was observed. The scores for individual participants were recorded session by session over the three day period to document the trend in individual engagement over the three days, as well as to determine if there was a group effect (e.g. low engagement) for any particular session. In addition, participants were asked at intervals (breaks, lunch and end of

135

the day) to describe their engagement during the sessions (as discussed next under interviews).

Participants were interviewed during the course to explore reasons for their

attendance and their opinions about the course relevance, usefulness, applicability and attributes of FD. These informal interviews were carried out in order to add further information to the observations and in order to establish a foundation for a deeper understanding of what had been observed (Hammersley & Atkinson, 2007). I chose informal interviews of participants at this initial stage as they are carried out ‘on the fly’ and do not require scheduling or taping. In fact, respondents might just see this as ‘conversation’, thereby fostering ‘low pressure’ interactions and allow respondents to speak more freely and openly (Cohen & Crabtree, 2006). It was helpful in building rapport and provided understanding from the participants’ perspectives. Furthermore, according to Cohen et al. (2011), there would be candour, richness, authenticity and honesty about participants’ experiences.

Demographic data including gender, seniority, funding source, job type and previous attendance at a teaching course were also collected from all participants.

Observation notes and notes from informal interviews were transcribed after each day. Observing and interviewing two groups allowed comparative analysis, improved validity and as observation is still uncommon in this area of FD research, it provided interesting and valuable data.

The key purpose of this step two of the study was to collect immediate data through observations / short informal interviews on the three common mechanisms

136

(motivation, engagement and perception) reported in the literature as being

important in a learning process (section 2.9.2). These three mechanisms were used to develop some of the hypotheses listed in Table 3.2. I was aware that this would not explore all the possible hypotheses listed in the table; the others were explored with further data collection described in step three below.

Step 3: Interviews of Course Participants at Six Months

Six participants from each course were selected for follow up interviews six months later (total of twelve), which was within the recommended qualitative methodologic standard of at least eight participants for in-depth interviews (Kennedy et al, 2004; McCracken, 1988). Purposeful selection was based on the engagement score

(above average, average and below average), seniority and gender. The audiotaped interviews were one hour, in-depth interviews to explore the longer-term impact on learning / behavioural changes i.e. what and how the interviewees have changed, and what facilitated or hindered that change (Appendix 3e). This allowed me to understand causal mechanisms and their outcomes.

The interviews followed the principles of ‘theorising the interview’ (Pawson, 1996) (section 4.2.3). I explained the hypotheses so the educators could understand them. We then discussed the hypotheses using information from the FD course as well as information from their experience of teaching during the six months following the course. As I anticipated, the educators were able to work out their own ideas and hypotheses about the FD. I helped them develop their views through a process of checking that I had understood correctly, sometimes challenging what they said and

137

thus we engaged together in the process of conceptual refinement. The educators were generally positive about FD but there were also criticisms.

Once transcribed, all individual phase II data (observations and interviews) were sent to the relevant participant. This respondent or member validation (Bryman, 2012) was to avoid misinterpreting the meaning of what participant’s said and did as well as the perspectives they had on FD. It was also an important way for me to identify my own biases and misunderstandings of what I had observed or heard. In addition, once transcribed and analysed, the data helped in refining the interview schedule for phase III as well as gave me clarity on the acceptability, understanding and

importance of my questions. Further details of analysis are described in section 5.5.