Phase 1 Scoping exercise

The participants

The participants were volunteers from a pool of 28 anaesthetic trainees working at a

university hospital. I approached trainees whom I knew well and had expressed some interest in education; all had either completed or were about to start a voluntary educational module that I supervise. I explained verbally the purpose of the study, that participation was voluntary and any information gained would be confidential. I reassured them that any data presented or published would not identify individuals or individual institutions. They clearly could not be

anonymous to me but would be to wider audiences. All agreed to have the interviews taped and transcribed. At the initial approach I also gave them a list of possible questions I was intending to use (Appendix 3). This gave them the opportunity to consider and reflect on the subject matter and gave them the opportunity to withdraw.

These trainees had completed at least two years at registrar level and had passed their final anaesthetic examinations. The reason for selecting post examination trainees was that up until this point their future career depended on their passing the examination. Failure meant they could not progress into their third registrar year and consequently could not complete their training. Therefore, their energies would be concentrated on the examination rather than participating with research. While they came from one institution they rotated on a yearly basis to other hospitals over their five year training programme.

In total I interviewed four anaesthetic trainees. This number was determined by the extent to which new themes emerged with each interview. I stopped when saturation was reached. This small number has limitations which might cause bias. In addition, as the trainees were from one institution they might not reflect the views of trainees elsewhere, although all had worked in several other institutions. As it turned out there were two male and two female trainees with a mix of ethnic backgrounds and all had completed at least five years within the UK training scheme for anaesthesia.

This sampling strategy had both advantages and disadvantages. On the positive side, I was already part of the culture. I understood the context within which the anaesthetic registrars worked, their language and how they needed to function. Furthermore, I had easy access to them. On the negative side, this meant that I started the process with preconceptions and biases. It was possible that as the trainees’ senior in hierarchical terms, they may have felt obliged to participate and the information they shared may have depended on what they

thought I wanted to hear. They might also have felt that what they said might be used in a different forum to influence their careers. Clearly this needed to be handled with great care because it has implications for reliability and validity (Rubin and Rubin, 1995; Arksey and Knight, 1999).

Questions

The questions were a combination of open and directed questions. While the nature of the enquiry was to gain insight and understanding it was also to gain information on a specific topic. As research interviewing was a new venture for me, I listed more questions than I thought I might need as I was concerned about running out of questions. In practice this was unnecessary because all interviewees spoke freely and only required some gentle probes. The topic areas for questions came initially from the research question. These included:

1. Do they teach and if so what? (This was expanded by asking for real examples of their teaching.)

2. How have they learnt to teach?

3. How do formal courses for TTT fit in with their training?

The questions were refined by reflecting on my experience of anaesthetic trainees’ involvement in teaching, by discussion with a critical friend with a particular interest in medical education and my study supervisors. The wording of the questions actually used was much more general than those listed (Appendix 3). For example my opening question was:

‘What I’m interested in today is looking at how we learn to teach in medicine. I wondered if you did any sort of teaching at the moment.’

Location

The interviews took place within the hospital. I avoided the use of my office in order to maintain a degree of neutrality and to limit any potential advantage I might be perceived to

have as the interviewer. However this caused other difficulties. On one occasion it became necessary to move during the interview because a room that is normally unused suddenly became a thoroughfare, despite being a dead end. On another occasion it became necessary to shut the windows because of noise from ambulance sirens around the hospital site, but this made the room hot and uncomfortable.

Recording and Transcription

The literature gives a number of warnings when recording interview data (Oppenheim, 1992; Britten, 1995; Rubin and Rubin, 1995; Gillham, 2000b). In short, what can go wrong will go wrong. I had a number of specific difficulties despite careful preparation. On the first

interview the microphone sensitivity was set for ‘meeting’. However, the sensitivity was so high that there was considerable noise in the background. This meant that transcription took longer than expected (4 hours for a 30 minute interview) due to having to repeatedly replay parts of the interview and listen through the background noise. The potential risk of this poor recording was the loss of relevant content material.

The biggest technical problem was when a brand new tape snapped on the first attempt to listen to the recording. It took over an hour of disassembling an unused tape before working out how to repair the broken one. In the end there was no loss of interview material but it did reinforce the dependence on and the vagaries of any form of technical equipment. My

contemporaneous notes would have salvaged some of the themes but would have missed detailed quotes. As I had checked the tape soon after the interview I could have expanded my notes from memory.

Process of Analysis

I elected to undertake the transcription myself which gave me an opportunity to immerse myself in the data. The transcriptions did not include pauses, hesitations, or reference to body

language or the ‘feel’ of the interview. These decisions already limited the ‘view’ of the interview as an event (Arksey and Knight, 1999). This meant the process was not neutral or value free. I transcribed each interview before undertaking the next. The transcriptions of the interviews were given to the interviewees so that they could check them for accuracy and withdraw aspects if they wished. They did not make or request any changes. My initial impressions were that after four interviews there was a saturation of the emerging themes and there was little to be gained by further interviews

I approached the data analysis from three directions. My first approach was to consider the data while trying to minimise my preconceptions. Initially I read the four interviews to get an overall view of the substantive statements and some of the main themes which were

emerging. I then re-read the interviews coding each of the ideas and assigned these ideas into broader themes. Finally, I identified other themes which seemed important but were not already covered. My second approach was to consider the semi-structured questions used for the interview because by their nature they might provide some core themes. My third

approach to the analysis was to search the transcripts electronically utilising text searches. A text word search of ‘teach’ revealed a count of 229 occurrences and of ‘learn’ a count of 56. Inspection of both of these searches showed that the majority of occurrences had already been coded by one of the previously identified themes.

My ideas and themes were presented to and discussed individually with my two supervisors and critical friend together with the full transcripts. The ideas and themes were also discussed in a group consisting of my supervisors, critical friend and myself. After modification and revisiting the transcripts, the broad themes were agreed as a fair interpretation and summation of the interviews.

From a technical point of view I utilised the computer software QRS NVIVO to highlight the identified themes and as the main record keeping tool (Appendix 4). I used this as an

alternative to the more traditional approach which uses highlighter pens, paper and scissors. The computer programme did not undertake the analysis, select the ideas or develop the themes. The advantage of using this computer software was that it was easy to add, subtract and modify themes. It kept an electronic record that could be adjusted, changed, and printed. Furthermore it acted as a record of how the process developed by saving different versions of the analysis on a time basis.

If the project had consisted of only the four interviews, then learning to use the computer software would not have been an effective use of time. The process would have been quicker using pens and paper. However, when combined with the action learning group transcripts it enabled easy access between the transcripts of the various phases of the project. The power of this electronic access to the ideas and themes facilitated movement between different

transcripts. It also enabled assimilation and display of ideas and themes from different phases into single documents. In contrast, a paper approach would have been more time consuming. It is worth emphasising that NVIVO was used as a record of the processes not as an analysis tool.

Reliability

The concept of ensuring reliability from a series of interviews is complex. The word reliability is, within the positivist paradigm, concerned with the repeatability of the results found in a study. If the study is repeated by other researchers under the same stated conditions then, if the study is reliable, they will find the same results. This works in the experimental research designs where experiments are looking for a single ‘true’ answer. However, within the interpretative paradigm things are different. In this paradigm there is no expectation of a

single reality. There is the underlying acceptance that there may be multiple realities or ‘truths’. In my study it was anticipated that each trainee would have different views. However, within these differences there would also be commonalities. The interviews continued until there was saturation of the common themes identified. Repeatability between different subjects at different times was not expected or indeed looked for and thus external reliability was difficult to achieve.

It is also possible to look for internal reliability. This could be achieved by re-asking similar questions on one or more occasions throughout the interview. Alternatively the individual could be interviewed again on another occasion.

Equivalent words to reliability, in the interpretative paradigm, include dependability and authenticity (Bryman, 2001). Does the data collected convey a measure of trustworthiness? This is a particular problem when there is a large amount of data which then requires interpretation by the researcher and the original data are not presented. It is found when analysing in-depth interviews. There are no concrete facts, as in scientific experiments, which can be published to allow reanalysis by others. One suggestion (Bryman, 2001) is to

encourage others to ‘audit’ the researchers’ original data so that they can follow the researchers’ train of thought. Authenticity can be achieved by checking that the findings reflect the views of the subjects. While this is not always possible, in this study the participants received a transcript of their interview for verification.

Validity

The validity issue is equally complex within the interpretative paradigm. Internal validity requires that the research is carried out according to good practice and is submitted for peer review so that the wider research community can confirm or query the findings. The validity of the questions was addressed in this study by reflecting on my experience of anaesthetic

trainees’ involvement in teaching, by discussion with a critical friend and my supervisors. External validity is related to applicability within in a wider context. This is of concern when small studies have generated the data which have been used for the analysis and

interpretation. An area of particular importance is for the researcher to declare how they collected the data, analysed it and state their position within the context in which these took place.

The problems of reliability and validity that this study has struggled with are as follows. The study gathered qualitative data which was subjective and impressionistic. The theories of how doctors learn grew out of the data rather than being pre-determined. In-depth interviews by their very nature are difficult to replicate. Different individuals are likely to have different views, which may be different on different occasions. Replication is also dependent on the researcher because different interviewers will follow different paths in semi-structured interviews. However, the key to demonstrating reliability and validity can be summarised as making the whole process as transparent as possible. Future researchers can, at the very least, follow in the same direction if not the same footsteps.

Conclusions

This scoping exercise can be located within the framework described in the methodology section 3.2. Evidence within the literature, on how anaesthetists learn to teach was minimal, and this was confirmed by peer review on two separate occasions. The questions asked in the interviews were selected after reviewing information on research interviewing and discussion with supervisors and a critical friend. The emerging themes were developed from the data and confirmed as fitting in with the experiences of others.