List of acronyms and synonyms
RCT Randomised controlled trial Telemonitoring group Intervention group
TM Telemonitoring
BP Blood pressure
SBP Systolic blood pressure DBP Diastolic blood pressure
GP General practitioner
HbA1c Glycated haemoglobin A1c
BMI Body mass index
DAG Directed acyclic graph
CI Confidence interval
P P-value from a statistical test
N Number of participants
SD Standard deviation
IQR Interquartile range
MCAR Missing completely at random
MAR Missing at random
MNAR Missing not at random
MI Multiple imputation
IPW Inverse probability weighting
7.1 Introduction
7.1 Introduction
7.1.1 Overview
In Chapters 7 and 8, we present a case study where we apply some of the principles
discussed in this thesis. It centres on the analysis of a randomised controlled trial (RCT) of a telemonitoring service and our aim was to provide conclusions that were more accurate and relevant than we might otherwise have delivered. We also wanted to better understand and communicate to stakeholders and clinical researchers the true level of uncertainty that remained following the analysis. This communication goal became more important when the study revealed much more missing data than was expected, and as a result, needed to be analysed as if it were an observational study.745 Although the level of missing data would
reduce the certainty of our conclusions, the human predilection for causal thinking, discussed in Chapter 4, may have left some of the staff involved believing that a causal relationship existed based primarily on their anecdotal observations during the trial. In reality, whether true or not, those causal inferences are likely to have been influenced by confounding and selection bias,745 as well as by the cognitive biases that can influence causal
judgements, including confirmation bias746 and overconfidence bias.305
Our view was that, of greatest value, might be an analysis that properly assessed the potential sources of confounding, selection bias, measurement error, and cognitive biases and, where possible, controlled for as much confounding and selection bias as could be determined, while ensuring that the level of uncertainty remaining was well understood and communicated.
To facilitate an extended discussion of bias relating to the case study, the presentation is divided into two chapters. In Chapter 7, we focus on describing the study and the data, including the measures taken in response to missing data. The overall aim is to promote an understanding of the potential biases this study is exposed to. In Chapter 8, our focus shifts to the analysis of the data and presentation of the results; using models to reduce the potential for bias and sensitivity analyses to better understand and communicate the uncertainty. We also explore the concept of time-dependent confounding in a separate analysis that uses the parametric g-formula.
More specifically, in this chapter we will:
1. briefly present the background and design of the case study 2. fully describe the data collected and the data that is missing
3. explain how we assessed and inferred aspects of the missing data mechanism 4. examine the possible effects of the missing data in terms of biased results and
increased uncertainty, and the use of multiple imputation to try to reduce such effects
5. display the causal diagrams we constructed to more easily identify and communicate potential sources of bias
7.1.2 Pragmatic trials
Telemonitoring trials, the type of trial assessed in this case study, have returned mixed results over the last two decades. There have been at least 20 randomised controlled trials (RCTs)747
–
766 and 4 observational studies767
–770 assessing either home blood glucose or blood pressure
measurement and all combined with some form of remote assessment and support. The HCF Telemonitoring RCT was a pragmatic trial that offered remote home blood glucose or blood pressure self-measurement, with associated telemonitoring by nurses. Originally introduced by Schwartz and Lellouch,771 the term ‘pragmatic trial’ refers to a randomised controlled trial
where the intervention: (a) resembles those that are already in routine use and may be combined with other interventions, as would occur in normal clinical practice; (b) where the main aim is to inform routine clinical decision making, as opposed to testing whether the intervention really can cause improvements in some people; and (c) is trialled with a broad patient group that is sufficiently representative of those encountered in normal clinical practice.772
In many cases, the analysis of a pragmatic trial relies on routinely collected data. Using such data often has substantial advantages, such as less interference with usual care and fewer expenses from a reduced need for onsite staff training regarding data collection and management.773 Relying on this type of data comes with a range of limitations, however,
7.1 Introduction
a research question. For example, data for some confounders may not be adequately
collected, such as particular diagnoses, medications or lifestyle factors, unless prompted by a voiced health concern from the participant.773
,774 This may mean that important baseline data
is not available for some, or even all, study participants. Participant outcome data is also more likely to be missing if it does not represent a major life event such as death, and this is often the case in pragmatic trials.775
7.1.3 Missing data
Missing data is one of the main concerns when using routinely collected data,773
,774 but the
mechanisms can be difficult to understand280
,776
,777 and are often not handled
adequately.776
,778
–783 The loss of information from missing baseline, intervention or outcome
data leads not only to a reduction of precision and power, but more importantly, it can also result in biased estimates.784 Whether such bias occurs depends primarily on why participant
values are missing, often called the missing data mechanism or missingness mechanism.785
From a system developed by Rubin in 1976,786 these reasons are commonly classified into
three types using the slightly confusing781
,785 terminology of Little and Rubin (1987, 2002).787
They are missing completely at random (MCAR), missing at random (MAR), and missing not at
random (MNAR). As well as having ambiguous labels, the three types are also frequently
described in noticeably different ways. Hence, to assist with clarity, each missingness mechanism type will be described in a variety of ways below. We will also avoid formal mathematical definitions, partly because this chapter is focused more on the practical application of methods and concepts, and partly because in recent years, differences have been highlighted in the way these terms have been formally defined by various authors,788
–791
but these details are beyond the scope of this case study.
Missing participant values are considered to be MCAR when they are, in effect, a random sample of the complete data.785 In this case, the missingness mechanism does not depend
on the values of any observed or unobserved variables in the causal network under study, including the missing values.792 This also implies that there are no systematic differences
between the missing and the observed values.781 Describing missing participant data as
MCAR is usually not a plausible assumption in health research, however.785
,792
When missing participant data can be explained by the observed participant data, the missingness mechanism is labelled MAR.781 In this case, systematic differences do exist
between the missing and observed values, however, conditioning on the measured values of the other variables removes the association between a value being missing and what that value would have been.785 A few statistical techniques used to handle missing data, including
multiple imputation, can provide unbiased estimates if the missingness mechanism is MAR and other assumptions are met. But if it is MNAR, such techniques may or may not provide unbiased estimates, depending on the nature of the missing data.794
If the missing participant data cannot be explained by what has been observed, then we say it is MNAR.792 This means that the probability that a participant’s value is missing is related to
the value itself,785 and that value cannot be predicted from the observed data, making
statistical adjustment not possible.552 But while this makes it more likely that particular values
are missing compared to other values, and that may lead to biased estimates, such bias is not always inevitable as it depends on the specific causal structure and the parameter being estimated.280
,792
Another term commonly encountered in the literature is ignorability, which is often used to mean that the missing data values are MAR or MCAR.792 But the formal mathematical
definition is a little different and means that inferences made from a parametric model of the observed data do not differ from inferences made from a joint model describing the
observed data and missingness mechanism.788 Missing data that is MNAR is sometimes
referred to as informative missingness, meaning that the fact that the values are missing contains information about what that value is.142
As an example, in the Glucose arm of the Telemonitoring trial, if occasional blood glucose measurements for some participants were the only missing data, and the reason was that their glucometer happened to have a defect that led to underestimated measurements, the missing data would likely have been MCAR. Alternatively, if blood glucose measurements were missing only for people who held a full-time job and it was inconvenient to take measurements sometimes, then if employment status was fully recorded the data might be described as MAR. But if blood glucose measurements were sometimes missing because participants had eaten foods that they knew would result in a high reading and thought they
7.1 Introduction
would feel embarrassed providing such a reading, then assuming there was no record of diet the data would be MNAR.
Missing data from loss to follow-up or dropout is the main mechanism by which a
randomised controlled trial can become as susceptible to selection bias as an observational study,745 so a careful assessment of missing data is essential. The vital question is whether
the results would have changed if the missing data had, instead, been obtained. In general, however, it is not possible to tell from the observed data whether the values that were missing were MAR or MNAR.792 Instead, as put by Sterne et al. (2009):781
The onus rests on the data analyst to consider all the possible reasons for missing data and assess the likelihood of missing not at random being a serious concern.
One tool that can assist in this assessment is causal diagrams, and a range of articles are now available that focus on causal diagrams for missing data.244
,279 ,280 ,795 ,796
Once the nature of the missing data has been ascertained, attempts can be made to reduce its influence. Over the last four decades, numerous authors have divided missing data
methods into two groups. Methods often labelled ad hoc include the older, simpler methods, like complete-case analysis; all developed before the advent of modern computers.787 The
more sophisticated and more recently developed methods, like multiple imputation, comprise the second group.142
,776 ,779 –781 ,787 ,793 ,797
–806 Additionally, an increasing number of
authors are now referring to this second, model-based, group as the principled missing data methods.776 ,780 ,793 ,798 ,800 ,801 ,803 –806