Data collection methods

The choice of data collection methods is an important consideration as part of the wider study design in determining how reliable and valid the data will be. Such considerations are of importance when considering the overall worth of a study. A broad overview of the methods of data collection for each of the two studies is given within the following subsections.

85

3.6.1 Data collection method for study 1: Questionnaire administered by interview

A questionnaire, most commonly used within cross-sectional survey designs (Greenfield, 2002), was chosen as the data collection instrument in the first study. Whilst other methods of obtaining survey data have been acknowledged (e.g. observation, content analysis, in depth interviews (de Vaus, 2013)) the questionnaire was deemed the most efficient way of gaining the data required. With any research design ensuring data quality is paramount and in the

instance of a questionnaire the mode of data collection can seriously affect the quality of data obtained (Bowling, 2005). Modes of questionnaire administration include self-completion (e.g. postal), internet based, face-to-face via interview and telephone (Robson, 2011; Bowling, 2005).

The principle weakness of self-completed questionnaires (internet, in person or postal) is a low response rate which is particularly problematic as responders and non-responders may systematically differ (Mann, 2003). Furthermore, added to the high probability that some questions will be ignored, questions may also be misinterpreted and/or answers inadequately detailed (Hicks, 2004). Telephone administration has also been criticised due to the difficulty in

contacting respondents and therefore the potential for lower response rates (de Vaus, 2013). The mode of administration chosen was face-to-face interview with completion of the questionnaire by the attendant health professional. Face- to-face interviews are more likely to elicit serious, considered responses to questions and overcome respondent literacy problems (Gillham, 2000). This reduced the threat of non-response bias to data quality, and was anticipated to be the most convenient data collection approach as patients would already be attending a clinic. A further advantage in using the health professional to

86

interview patients was that medical notes could be reviewed simultaneously to ensure data was as accurate as possible. In the present study this was

especially pertinent to most accurately determine date of injury, dates and type of clinic attendance and number of healthcare appointments reducing recall bias. Face-to-face interviews have been criticised as the quality of answers can be compromised in cases where questions may be sensitive or controversial (de Vaus, 2013). However, the topic under investigation did not produce such problems.

Prospective designs identify a group of people and collect information at the time of attendance in a particular service (Hicks, 2004). Whilst prospective designs involve a longer data collection phase it has the advantage of reducing non-response bias compared to retrospective designs and can enhance data quality (Hicks, 2004). Patients were therefore identified prospectively within this study.

3.6.2 Data collection method for study 2: Non-participant direct observation

A number data collection methods could be used to determine what takes place during a clinical encounter including direct observation of the clinical encounter, review of medical notes, audiotape, interview, questionnaire and video

observation.

The review of medical notes to determine the content of clinical examination has an advantage over direct observation by reducing or negating the observer effect but poses a number of disadvantages. Large discrepancies have been found between directly observed assessment or audio recorded assessment and that reported in medical records (Sharma, 2011; Wilson and McDonald,

87

1994). Furthermore, data obtained from medical records as a method for determining assessment quality in an outpatient clinical setting has been criticised as time constraints may result in recording bias leading to an incomplete picture of the assessment (Peabody et al., 2000). The result of recording bias has been shown to underestimate clinician performance through a high false negative rate whereby a significant proportion of the clinical

examination is not recorded, but may also overestimate clinician performance in some cases due to false positive reporting (Dresselhaus et al., 2002).

Interviews and questionnaires are further data collection methods that could have been used to determine what occurred in the clinical encounter but it has been suggested these methods are of limited value in determining behaviour as the correlation between what people do, and say they do, is often low (French et al., 2001).

Audio tape recording of the clinical examination was discounted as it would not allow appreciation of the physical examination tests, a key area of interest. Video observation has been proposed as a reliable data collection method in cases where fleeting events, simultaneous interventions or brief interactions occur when direct observation may not reliably capture all data (Mackenzie and Xiao, 2003). However, in study 2 the observation was undertaken on a single patient assessment in a setting with only the clinician and patient present and therefore simultaneous events would not be encountered. Video observation also posed a number of disadvantages. It has been shown to be intrusive and influential in clinical decision making (Ram et al., 1999). Further, the presence of video cameras is unacceptable for some patients with 13% refusing to be filmed in a study within a doctors surgery (Martin and Martin, 1984). Among

88

patients who did consent, 11% disapproved of being filmed, 16% were constantly aware they were being filmed and 11% reported that the video camera made them feel nervous. In contrast to this a study undertaken in a similar setting, Servant and Matheson (1986) reported that only 10% of patients consented to having their consultation video recorded. Whilst these studies took place in a different generation and cannot be applied directly to an A&E or orthopaedic environment, they show that recording clinical examinations may lead to problems with consent and participation. Video observation also poses significant logistical difficulties; in the observational study multiple cameras would have been required to ensure that all physical examination tests could be appreciated and the clinical examination would have had to be undertaken outside of the usual environment, negating one of the potential benefits of observation.

The method chosen was direct observation of the clinical encounter.

Observation is of use in situations where the interaction of interest is hidden such as in a clinical examination whereby only the assessor and patient would normally be present. Direct observation has the advantages of providing a ‘real life’ setting (Robson, 2011; French et al., 2001) and allows appreciation of the clinical encounter in real time. Robson (2011 p316) states that ‘direct

observation…permits a lack of artificiality which is all too rare with other techniques’. A further advantage is that no effort is required from research participants (French et al., 2001).

Non-participant, as opposed to participant, observation was chosen as the method to determine what happens in the clinical assessment and is in keeping with the adopted post-positivist philosophical stance. Participant observation (associated with an ethnographic methodology) involves engagement with

89

people in the research setting with the researcher becoming a member of the group (Gray, 2013). This posed notable disadvantages; the timescale required to integrate within the groups and the potential for the researcher to change usual practice.