Strategy for intention to treat analysis in randomised trials with missing outcome data

Abstract

Loss to follow-up is often hard to avoid in randomised trials. This article suggests a framework for intention to treat analysis that depends on making plausible assumptions about the missing data and including all participants in sensitivity analyses

The intention to treat principle requires all participants in a clinical trial to be included in the analysis in the groups to which they were randomised, regardless of any departures from randomised treatment.¹ This principle is a key defence against bias, since participants who depart from randomised treatment are usually a non-random subset whose exclusion can lead to serious selection bias.²

However, it is unclear how to apply the intention to treat principle when investigators are unable to follow up all randomised participants. Filling in (imputing) the missing values is often seen as the only alternative to omitting participants from the analysis.³ In particular, imputing by “last observation carried forward” is widely used,⁴ but this approach has serious drawbacks.³ For example, last observation carried forward was applied in a recent trial of a novel drug treatment in Alzheimer’s disease.⁵ The analysis was criticised because it effectively assumed that loss to follow-up halts disease progression,⁶ but the authors argued that their analysis was in fact conservative.⁷ Increasingly, trialists are expected to justify their handling of missing data and not simply rely on techniques that have been used in other clinical contexts.⁸

To guide investigators dealing with these tricky issues, we propose a four point framework for dealing with incomplete observations (box). Our aim is not to describe specific methods for analysing missing data, since these are described elsewhere,^{9 10} but to provide the framework within which methods can be chosen and implemented. We argue that all observed data should be included in the analysis, but undue focus on including all randomised participants can be unhelpful because participants with no post-randomisation data can contribute to the results only through untestable assumptions. The key issue is therefore not how to include all participants but what assumptions about the missing data are most plausibly correct, and how to perform appropriate analyses based on these assumptions. We now expand on these four points.

Strategy for intention to treat analysis with incomplete observations

• 1. Attempt to follow up all randomised participants, even if they withdraw from allocated treatment

• 2. Perform a main analysis of all observed data that are valid under a plausible assumption about the missing data

• 3. Perform sensitivity analyses to explore the effect of departures from the assumption made in the main analysis

• 4. Account for all randomised participants, at least in the sensitivity analyses

Attempt to follow up all randomised participants

Following up participants who withdraw from randomised treatment can be difficult but is important because they may differ systematically from those who remain on treatment. A trial that does not attempt to follow participants after treatment withdrawal cannot claim to follow the intention to treat principle.

Perform a plausible main analysis

When data are incomplete, all statistical analyses make untestable assumptions. The main analysis should be chosen to be valid under a plausible assumption about the missing data. For example, in the trial in Alzheimer’s disease, consider a group of participants who are lost to follow-up between 6 and 12 months and a group of participants whose outcomes up to 6 months are similar to the first group’s but who are followed at least to 12 months. It may be reasonable to assume in the main analysis that these two groups have similar changes on average from 6 to 12 months—a “missing at random” assumption, under which an analysis of all observed outcome data, with adjustment for selected covariates, is appropriate. A similar assumption underlies standard analyses of time to event data.

Possible analysis methods under a “missing at random” assumption include multiple imputation, inverse probability weighting, and mixed models. These methods, and other methods whose assumptions are less clear, are reviewed elsewhere.^{9 10}

Assumptions about the missing data can often be supported by collecting and reporting suitable information. For example, “missing at random” is often plausible if the reason for most missing data is shown to be administrative error but implausible if the reason is undocumented disease progression.

Perform sensitivity analyses

Good sensitivity analyses directly explore the effect of departures from the assumption made in the main analysis.¹¹ For example, if the main analysis assumes similarity between groups who are and are not lost to follow-up, a good sensitivity analysis might assume that the group who are lost to follow-up have systematically worse outcomes. A clinically plausible amount could be added to or subtracted from imputed outcomes, possibly using a technique such as multiple imputation.⁹ Conversely, analysts could report how large an amount should be added to or subtracted from imputed outcomes without changing the clinical interpretation of the trial. With a small proportion of missing binary outcomes, best and worst case analyses may be appropriate.¹²

Results of the sensitivity analyses should be concisely reported in a paper’s abstract, saying, for example, whether the significance of the main analysis was maintained in all sensitivity analyses or was changed in a limited or large number of sensitivity analyses.

Account for all randomised participants in the sensitivity analyses

When sensitivity analyses are carried out in this way, they should account for all randomised participants. For example, if a sensitivity analysis assumes a systematic difference between missing and observed values, then its results directly depend on the extent of missing data in the two trial arms.

Example of strategy in action

We illustrate the proposed strategy for intention to treat analysis using a recent trial comparing four doses of a new drug for obesity with two control groups.¹³ Participants had nine planned visits over 20 weeks. The trial report suggests that participants who withdrew from trial treatment were followed up (point 1 of our proposed strategy). The primary analysis (point 2) used last observation carried forward in a “modified intention to treat” population that excluded three participants with no post-randomisation measures. A sensitivity analysis used repeated measures and thus assumed the data were missing at random. Since the main analysis implicitly assumes that participants neither gained nor lost weight on average after loss to follow-up, more direct approaches to sensitivity analysis are preferable. The figure shows our proposals for a hypothetical participant who attends only four of the nine visits (solid line). The red broken line shows the imputed value under last observation carried forward, the study authors’ main analysis, while the other lines show three sensitivity analyses (point 3): sensitivity 1 shows the imputed value assuming that participants lost to follow-up returned to their baseline weight¹⁴; sensitivity 2 assumes they regained 50% of their lost weight; and sensitivity 3 assumes a larger fraction of the lost weight was regained in the intervention group.¹⁵ Participants with no post-randomisation measures could be included in these analyses by making similar assumptions about their weight gain (point 4).

Fig 1 Possible ways to impute outcome measures at visit 9 for a hypothetical participant in the obesity trial who drops out after visit 4: main analysis (last value brought forward) and three sensitivity analyses (1 assumes participants lost to follow-up return to baseline weight; 2 assumes 50% of weight regained, and 3 assumes intervention group regains a greater proportion of weight than controls)

Discussion

The ideal solution to the problems discussed here is to avoid missing data altogether. This is rarely practical, but missing data can be minimised by careful design and trial management,¹⁰ and in particular by attempting to follow up all participants.

The obesity trial illustrated our strategy applied to a trial with a repeatedly measured outcome. Analysis choices are more limited in trials with a singly measured outcome. In trials with time to event outcomes, an analysis that includes all randomised participants with censoring at the point of loss to follow-up is generally acceptable, but possible biases from informative censoring should be considered. In general, primary and sensitivity analyses should be specified in detail, ideally in the registered trial protocol and certainly before the unblinded data are seen, as a defence against claims of data driven changes to the analysis.¹⁶

Some argue for conservative analyses.¹⁷ However, methods that are conservative in some settings may not be conservative in others. For example, last observation carried forward is often claimed to be conservative, but it can be biased in favour of a new treatment.¹⁸ We have instead suggested that authors should make their most plausible assumptions the basis for their primary analysis and then provide conservatism by assessing sensitivity to departures from those assumptions.

Our proposed analysis strategy conforms to the intention to treat principle in the presence of missing outcomes and clarifies uncertainty regarding its application. It acknowledges the uncertainty introduced by missing data and therefore gives investigators an added incentive to minimise the extent of missing data.¹⁹ Such guidelines are needed given the importance placed on intention to treat analyses and the ubiquity of missing data in real world clinical trials.

Cite this as: BMJ 2011;342:d40

Related links

bmj.com/archive

• Economic evaluation using decision analytical modelling: design, conduct, analysis, and reporting (2011;342:d1548)

• Strengthening the reporting of genetic risk prediction studies: the GRIPS statement (2011;342:d631)

• Correlation in restricted ranges of data (2011;342:d556)

• Interpretation of random effects meta-analyses (2011;342:d549)