ITT, PP and AT analyses

Analysis in the face of treatment protocol deviations may therefore take a number of forms.

Analysis according to randomisation, in other words according to how the patient was intended to be treated (hence the term “intention to treat”, ITT), ignores any such deviations, such that patients are simply analysed according to their randomised allocation regardless of whether they received or deviated from this allocation. In retaining randomised allocations, an ITT analysis maintains the balance afforded by randomisation, thus preventing selection bias and assuring a sound basis for statistical hypothesis testing. In the presence of any deviation from assigned treatment, however, the interpretation of such an analysis is limited to an assessment only of the effectiveness of treatment policy or of the treatment prescription, rather than a causal estimate of treatment received.

When the efficacy of treatment received is also of interest, alternatives to ITT are required, as inclusion of non-adherent participants in ITT analysis generally diminishes the estimated treatment effect and thereby resulting in a biased assessment of treatment efficacy (61). Analysts must however be mindful of the potential for bias resulting when analysing according to anything other than randomised allocations. A simple but statistically naïve method of analysis may involve analysis of patients in their

67

randomised group only if (or during the period during which) they follow their randomised allocation, thus excluding or censoring (terminating the time at which the participant was included in the analysis as being “at risk” of the event of interest at the point of deviation from the treatment protocol, which is possible when undertaking a survival analysis for an outcome which is time to some event) patients who deviate from randomised treatment protocol (hence the term “per protocol” (PP) analysis). Exclusion of participants from analysis in this way affects both the internal and external validity of a trial (62). A patient’s ability or willingness to persevere with treatment is highly likely to be correlated with their condition and other lifestyle factors; indeed, non-compliant patients have been shown to have worse prognosis than compliers in their respective randomised group, even when administered with placebo (9). As such, by excluding some definition of “non-compliant” participants, a PP analysis will affect the generalisability of a trial, as those who persevere with treatment protocol represent a non-random sample of the original group of trial participants.

Furthermore, such an analysis is likely to introduce selection bias and thus also affect the internal validity of a trial, because the various treatment protocols being compared will present different challenges to adherence. This is especially likely when the definition of “compliance” varies between treatment arms, reflecting the different adherence requirements of the treatment packages. As such, the residual compliant subgroups of each randomised group are unlikely to be comparable. Therefore, unless it can be demonstrated that the average prognosis of those who deviate from treatment protocol does not differ between randomised groups, a PP analysis will likely upset the balance provided by randomisation, casting doubt on the validity of its conclusions (60). However, given that those intermediate confounding factors which influence a patient’s compliance status as well as their prognosis (and hence outcome) typically remain

68

unmeasured (and may arguably even be unmeasurable), it is often impossible to compare the profiles of these factors between groups. Therefore, even if the baseline characteristics, and rates and reasons for treatment withdrawals or changes, appear relatively similar between treatment arms, it is not possible to ascertain whether the compliant subgroups remain balanced with respect to unmeasured prognostic factors. The results of any such PP analysis are therefore highly likely to be unreliable because of these hidden confounding or selection effects (63).

Exclusion of patients due to deviation from treatment protocol reduces the statistical power of a study by reducing the sample available for analysis. A variation on (but one even more flawed than) PP analysis used to overcome this problem of reduced power is analysis according to treatment received (or an “as treated” analysis). As treated (AT) analyses, whereby patients are analysed according to the (predominant) treatment received, are never likely to be valid as randomisation is disregarded entirely (57).

3.6.1. Healthy user bias

This (so called “healthy user”) bias associated with compliance analyses is supported by evidence demonstrating that those with better compliance behaviour tend to have better clinical prognosis that those with poor compliance, regardless of treatment received (e.g. active or placebo treatment). This effect has been demonstrated in varying clinical setting with disparate drug regimens, independently of whether or not there is an apparent clinical effect of drug (or of a drug/compliance interaction), suggesting generalisability of this phenomenon. (64) For example, in the Coronary Drug Project Research Group (65), non-compliers in the placebo group experienced nearly double the mortality rate compared to compliers (28% versus 15%, respectively), and in the Beta Blocker Heart Attack Trial, poor adherers had an increased risk of death whether they were taking active treatment (OR=3.1) or placebo (OR=2.5) (66).

69

Epstein (64) discusses potential reasons for this apparent relationship between compliance and outcome: first, the psychological impact of complying with medical instruction may actually enhance the wellbeing of the patient, either solely because of the placebo effect (expectation that the drug will work leads to improvement in a patient’s condition) or because of the mediating effect of patients’ changes in habits or actions which result from the positive feelings towards taking treatment (for example, if such positivity leads patients to alter other lifestyle habits which in turn influence their outcome). Secondly, the relationship between compliance and outcome may be spurious: either because of the method in which compliance information is collected (for example, if those faring better are more likely to provide compliance data or be labelled as “compliant”) or if a patient’s ability or likelihood to comply is determined directly by their prognosis (either physically or psychologically) or their innate personality traits (which in turn affect their outcome), and thus the apparent relationship between compliance and outcome merely reflects the underlying link between prognosis and outcome. Finally, the relationship between compliance and outcome evident in both treatment and placebo groups may be caused by different mechanisms – for example, in the active treatment group, this association may be due to the true effect of drug, whereas in the placebo group, other factors (such as those listed above) may have improved compliant patients’ outcomes – though it may seem unlikely that these different mechanisms would lead to similar results between non/compliers in both randomised groups.

Thus, although it may tempting to base any validity of such a comparison on assessment of any differences in baseline factors between compliers and non-compliers, an apparent similarity in baseline prognostic factors is not sufficient to justify such an assumption. Indeed, analysis exploring the relationship of baseline prognostic factors

70

with compliance in a number of trials demonstrates that adjustment for important prognostic factors often fails to explain for the variation in treatment effect attributable to compliance features (65, 67, 68).