statistical mEthods to corrEct for confounding

in non-randomisEd (dirEct

EvidEncE) study approachEs

In this Appendix we briefly explain different methods that can be used in non-ran- domised studies to limit or adjust the potential for bias or confounding bias (see also the references in main text). 25

A. Methods used at the study design (set-up) stage

In this method, the aim is to create two groups that are similar on one or more known and major confounding variables by restricting the inclusion or exclusion criteria. For example, if age, gender or a specific disease variant are known to have a major influ- ence on the mechanism/pathway of the device’s benefits, we can restrict the two study groups only to a specific gender, age range, or disease variant. Another method is to exclude subgroups which are known a priori to influence the pathway of the device’s effect. A disadvantage of restriction is that the results may have limited applicability – reduced generalizability of device use – because they are based on a more homogene- ous and less representative study group.

A2. Matching

Matching is the process of searching, for each individual in the index (device) group, a subject in the control group who is similar on the set of most important confounding 25  Cousens S., Hargreaves J., Bonell C., Armstrong B., Thomas J., Kirkwood B.R., Hayes R., Al- ternatives to randomisation in the evaluation of public-health interventions: statistical analysis and causal inference. J Epidem Community Health 2011;65:576-581; McNamee R., Regression modelling and other methods to control confounding. Occup Environ Med 2005;62:500-506 doi:10.1136/oem.2002.001115.

variables. For example, for an index-group female, 40 years of age, no previous disease, particular disease stage, we would search and include a control-group female with a similar age range (40-45), no previous disease and same disease stage. One limitation to matching is that it is tedious work and therefore an expensive undertaking when there are many known a priori confounding variables. It is therefore often performed only for the two or three most important confounders.

B. Methods used during statistical analysis

B1. Stratification

This method involves stratifying the study participants by the main confounders for which results should be controlled (e.g. by age groups or gender), and estimating the effect of the device per stratum (e.g. per age or gender group). In order to ensure that strata with a larger group of participants receive larger weights when estimating the association between the difference in device use and the observed benefits, we simply estimate a weighted average for the benefits of the device over these strata. We then end up knowing the unbiased and valid effect of the device’s use compared to the control group, and adjust for those confounding variables. The most common statis- tical weighting approach used in this case is the Mantel-Haenszel method. Stratifi- cation also allows us to assess whether the device has different benefits in different subgroups, provided we have large enough numbers per subgroup. One disadvantage of this method is that strata with larger numbers of participants will generate more precise estimates of the device’s benefits (with a smaller standard error) than smaller strata.

B2. Regression modelling

This statistical approach is used to control for many confounders simultaneously, unlike all the above methods, which control for one or a few confounders. Regression modelling adjustment is the most widely used and perhaps the best method to adjust for other influential factors when studying the benefits of device use versus a control group in a non-randomised study. The use of this method only requires that inves- tigators predefine the known influential or confounding factors for the device and outcome being studied and subsequently measure the presence or absence of each predefined confounder in each study participant in both study groups. One disadvan- tage of these analyses is that confounders can only be controlled for if they are known

a priori and properly measured in each study participant.

26  To correct for confounding during the analysis phase, the process of designing and con- ducting a non-randomised study on the benefits of a device must meet certain requirements.

B3. Propensity score methods

This relatively novel approach – which also uses a regression modelling approach – is used most notable in Non Randomised Studies on the benefits of interventions where the number of study participants is limited and the number of confounders is rela- tively large. A propensity score is also called a multiple confounder score, with the value of each study participant’s characteristics and confounders being summarised to a single variable. It has two major stages. First, when comparing the two non-ran- domised groups – participants with the device versus those without the device – we estimate the probability that each individual had received the device according to their characteristics and confounding variables (using a logistic regression model). The probability per participant is known as the propensity score. Second, we then restrict, match or stratify the two study groups according to this propensity score variable, or use a regression model. There has been extensive research comparing the advantages and disadvantages of propensity methods for confounder control to the traditional methods described above.

appEndix vi

concisE comparison