Chapter 3: Methodology 3.1 Introduction
4 Critique the material
3.5 Methodological rigour
Ways to reduce bias were built in to the methodology during the research design
phase of the study. Consideration of how to minimise or recognise bias was
ongoing through the different stages of the research, and is reported in the
relevant sections on data collection, analysis, findings and discussion. Table 5
provides a summary of the actions taken to reduce bias. In particular, care was
taken to establish appropriate relationships with participants through honest
explanations of the scope and purpose of the research; awareness of personal
and professional self, and the potential effects upon participants (Appendix 2
Table 5: Actions taken to reduce bias.
Type of bias Actions
In the literature review
Systematic searching; recognition of ‘saturation’ point when no new papers or concepts were uncovered; use of critical review tools to appraise publications.
In the research design
Inclusion of experts, nurses and travellers in order to examine the pre-travel consultation from different perspectives.
Selection of Donabedian’s framework to organise, examine and synthesise methods, data and analysis.
Bricolage design to select the methods most suited to answering the research questions, and to triangulate approaches to an issue. Scrutiny by both the University and the NHS REC.
Sampling and recruitment bias
Selection of domain: the locality did not have specialist travel medicine units or teaching centres likely to influence local services.
Selection of general practices to avoid known colleagues and therefore potential bias.
Data collection bias Choice of research methods, e.g. video recording of consultations enabled the researcher to remove self from the consultation. Pilot studies.
Adherence to the normal practice within surgeries, e.g. clinic and consultation times.
Setting up data collection tools and equipment to maximise reliability and validity, and to minimise impact upon participants, e.g. choice of an unobtrusive camera and tripod position. Data analysis bias Designing tools to maximise reliability and validity.
Pilot studies.
Triangulation through bricolage design enabled comparison of findings from different methods.
Coding frameworks testing: method three against RIAS and method six by a peer.
Keeping an audit trail of all data, findings and decision-making for PhD supervisors and examiners.
Biased
interpretation or conclusions
Searches for other explanations; clinical and doctoral supervision; peer testing.
Crosschecking findings from method three consultation recordings with method five interviews with travellers and method six focus groups with practice nurses.
Researcher bias Supervision from experienced primary care researchers and
research training at the University of Warwick.
Personal reflection, clinical and research training and supervision.
The concepts of reliability and validity were considered throughout the research
process. Reliability relates to how well the research has been carried out and
whether methods are consistent – for instance, whether another researcher could
replicate the research and findings using the same tools. Although it can be
cases and contexts can vary so much, it is nevertheless important to provide a
clear audit trail of research activity to show how findings and interpretations were
reached (Wisker, 2001). This study considered reliability relating to each of the six
methods; carried out pilot studies to test them; maintained an audit trail of data
collection and writing; retained data in line with University of Warwick guidelines;
kept a book of field notes and memoranda recording the research process and the
development of ideas; and used a senior nurse with experience of both research
and travel health to examine the findings and discussion using the analytical
frameworks developed for this study. Ten percent (n=3) of each set of transcripts
from the AV recordings, interviews with travellers and focus group discussions
were checked for reliability. The technique involved assigning each section of talk
into a category, then the coding was compared to that of the researcher. An inter-
rater reliability score averaging 80 percent was achieved, meaning identical
categories had been selected for eight out of ten sections of talk. Analysis of the
20 percent assigned to categories that differed from the original researcher
showed that some ambiguity was possible, e.g. talk relating to first aid kits could
be assigned to the category ‘personal safety’, or to ‘exposure to blood and body
fluids’ (see also Appendix 4). Therefore the wider context of talk needed to be
taken into account to clarify which category was the most accurate in which to
place the comment.
Validity is central to the integrity of research (Cormack, 2000; Denzin, 2006). It
refers to the strength of the findings and conclusions, and whether they are
judged to be correct or true. Like reliability, in qualitative research a single,
account different perceptions of ‘truth’. There are different perspectives to validity
to assist with this.
External validity relates to the generalisability of the findings. It is not claimed for
this study because of the small sample. Instead, the objective is to create, with
stakeholder involvement, a prototype model for pre-travel health consultations
ready for post-doctoral testing. In line with Coleman’s (2000) recommendations,
notes were kept about the response rates to look for indications of something
different about participants or non-participants that skewed involvement in the
study, and therefore also potentially skewed external validity. Each of the methods
used in the study are long established in qualitative research, and the tools for
each were developed with attention to recommendations from the published
literature on qualitative methodology. The RIAS tool has been subjected to tests
for reliability and validity (Roter and Larson 2002; Roter, 2005), and the coding
technique within the University was successfully tested for reliability with the
original developers at Johns Hopkins University, Baltimore.
Content validity was sought by drawing upon the known literature for tools such as
the audit of resources available to practice nurses, the AV recordings and RIAS
categories for analysis, and the interview schedules devised for travellers and
nurse focus groups, and piloting them with participants drawn from the groups
researched, thus validating the tools as representative of what is known. Internal
validity refers to the extent to which an instrument or tool measures what it is
intended to, so that conclusions are valid and just. For instance, in the AV
recordings of consultations, the concept of internal validity aided assessment of
whether the process of being filmed would alter the behaviour of participants.
recording for unnatural behaviour such as continual glancing at the camera, or
exaggerated actions, which may indicate such issues. Trochim (2006) suggests
the alternative term of credibility to replace internal validity, requiring the results of
qualitative research to be believable. To assess this, interpretations from the
documentary analysis were verbally checked with nurse participants;
interpretations relating to the consultation recordings were checked with the
travellers who had participated and with practice nurses during the focus groups;
and interpretations of diary entries were checked with the travellers during their
interviews.
The focus groups were also used to explore other possible explanations of
processes used within consultations. A powerful example of a credible or valid
interpretation was the description of different consultation styles in Appendix 10,
which were drawn from observations of the consultation recordings, and validated
by nurse participants in the focus groups.
An audit trail of data collection, analysis and interpretations was kept, e.g. original
recordings and transcripts were kept to allow verification with other researchers
within the NHS REC boundaries.
The bricolage design proved useful in providing triangulation of methods and
findings to strengthen concurrent validity within the study (Adami and Kiger, 2005;
Williamson, 2005). For instance, results from the established validity of the RIAS
3.6 Conclusions
The bricolage technique of combining different methods carries risks of producing
masses of data that are unwieldy to analyse, and creating breadth rather than
depth of meaning. These problems were avoided through developing a
methodology designed to answer specific questions, and an iterative analytical
process to identify and develop the main themes running through all the methods.
A summary table to show the methods chosen to answer the research questions
is shown in Table 6. Although no single method in itself can be said to be original,
the combination used in this study is believed to be an original contribution
towards understanding the phenomenon of the pre-travel health consultation. In
particular, the views and actions of different stakeholders are revealed, and are
elucidated by the structure of the thesis which now moves on to present these in
three phases: the experts, the nurses, and the travellers. The findings from all
three are then interpreted and discussed to develop a new model of pre-travel
health consultations. As all practice must start from a knowledge base, that which
comprises the official guidance from national and international sources – the
Table 6: Methods selected to answer the research questions.
Research question Method of answering it 1. What currently comprises the nurse-
led pre-travel health consultation?
Subsidiary aspects to this question include:
a. What structures, processes and outcomes are currently associated with the pre-travel health consultation?
b. How appropriate are the interventions, when mapped against the ‘expert opinion’ and
guidance available in the literature?
c. Do nurses consciously adopt a model of consultation?
d. How do travellers use the education, information, advice and interventions gained from the consultation?
a. Method 2 - Audit of
structures, and method 3 – AV consultation recordings.
b. Method 1 - Documentary analysis, and method 3 - AV consultation recordings.
c. Method 3 - AV consultation recordings, and method 6 - Focus groups.
d. Method 3 - AV consultation recordings, method 4 - Diaries, and method 5 - telephone interviews.
2. What elements ought to be incorporated