Post-randomisation exclusions: the intention to treat principle and excluding patients from analysis

When is it legitimate to exclude randomised patients from the analysis of data in clinical trials? Basing their analysis on the desirability of minimising bias and random error, the authors consider the circumstances when it may be possible to exclude patients, even in an intention to treat trial

Most clinical researchers and statisticians agree that the primary analysis of data in a randomised clinical trial should compare patients according to the group to which they were randomly allocated, regardless of patients' compliance, crossover to other treatments, or withdrawal from the study. Such an analysis is referred to as an intention to treat or an “as randomised” analysis. Proponents argue that the intention to treat approach

• Helps preserve prognostic balance in the study arms

• Limits inferences based on arbitrary or ad hoc subgroups of patients in the trial

• Emphasises greater accountability for all patients entered into the study and consequently minimises the influence of withdrawals, non-compliers, and patients lost to follow up

• Is the most cautious approach and so minimises type 1 error, and

• Allows for the greatest generalisability.^1–5

Critics say, however, that an intention to treat approach is too cautious and more susceptible to type II error.⁶,⁷ They argue that such an analysis is less likely to show a positive treatment effect, especially in studies that randomise patients who have little or no chance of benefiting from the intervention. These critics maintain that an efficacy or explanatory approach to an analysis is more important than an effectiveness or pragmatic approach.

Experts have documented the strengths and weaknesses of the different analytical approaches.⁸ However, one issue that has only rarely been addressed in the literature is post-randomisation exclusions unrelated to non-compliance, withdrawal, or losses to follow up.⁹ These exclusions occur when patients are inappropriately randomised into a clinical trial or when pre-randomisation information on patients' eligibility status is not available at the time of randomisation.

Summary points

• Trial investigators can exclude patients' data from analysis, without risking bias, when ineligible patients are mistakenly randomised into a trial

• Similarly, data on patients who were prematurely randomised and so did not receive an intervention can be excluded, as long as allocation to treatment arm cannot influence the likelihood that patients receive the intervention

• Data should be included in the analysis when patients are randomised before information is available to confirm their eligibility and when the eligibility criteria are too broad and some patients don't have the condition of interest. But investigators can do a secondary analysis that excludes such patients

• Although excluding patients from analysis in certain circumstances does not bias the results, investigators should adhere to the highest standards of methodological design and trial execution to minimise post-randomisation exclusions

Types of post-randomisation exclusion

Our approach to the acceptability of post-randomisation exclusions focuses on two primary goals: to avoid bias and to minimise random error. The best way to achieve these goals depends on whether investigators wish to address an explanatory (efficacy) or management (effectiveness) question. Ideally, investigators will avoid post-randomisation exclusions through rigorous design and pretesting of the study protocol. We address four situations, illustrated by real or hypothetical studies, that are unusual and ideally should not arise during the conduct of most clinical trials.

Patients mistakenly included who do not meet inclusion criteria

Patients may be inappropriately randomised into clinical trials as a result of human error. Many clinical trials involve acutely ill patients who require urgent interventions. Determination of patients' eligibility for inclusion in these studies must be made quickly and consent and randomisation arranged expediently. Often study personnel work in chaotic clinical environments. Time constraints may result in patients who do not meet predetermined eligibility criteria being mistakenly included (box B1).

Box 1.

Ineligible patients mistakenly included

A study compared the effects of vasopressin and adrenaline (epinephrine) on 324 patients' survival after cardiac arrest in a hospital setting¹⁰

Fifty patients included in this study were excluded from the analysis because their cardiac arrest occurred before their arrival at hospital, and thus they had been mistakenly randomised

When ineligible patients are mistakenly included, investigators could remove these patients from both study arms without risking bias. However, so that the decision to remove such patients is unbiased and not influenced by events that occurred after randomisation (and may therefore be affected by whether patients received experimental or control treatment), an independent adjudication committee blinded to treatment and outcome must systematically review each patient. Also, the adjudication committee must base its decisions solely on information that reflects the patient's status before randomisation. Investigators should clearly state the number of patients randomised but not included in the primary analysis of data and explain the circumstances under which such patients were enrolled but excluded from the analysis.

Although excluding a large number of patients may not introduce bias, it may weaken any inferences from the study, because of the decreased sample size (that is, decrease the precision of the estimates of effect). If ineligible patients have a similar response to treatment to that of eligible patients, their exclusion will reduce the power of the study. If the reason for exclusion was that they were expected to have a reduced or no response to treatment, and the expectation is correct, their inclusion will introduce random error and reduce the power of the study and the precision of the estimate of treatment effect. Furthermore, the most informative analysis will depend on whether clinicians ultimately intend to apply the study results to patients represented by those who were mistakenly randomised.

Poor or excessively broad eligibility criteria

Poorly defined or excessively broad eligibility criteria can lead to the inclusion of patients who do not have the condition of interest and are therefore unlikely to benefit from treatment. For example, studies of severe infections resulting in sepsis syndrome are often beset by difficulties in defining the condition of interest and the eligibility criteria.¹¹,¹² The diversity of clinical presentations often results in the enrolment of patients who meet eligibility criteria and receive treatment but are unlikely to benefit (box B2).

Box 2.

Excessively broad eligibility criteria

A large randomised controlled trial of a drug that modulates immune responses in severe sepsis enrolled a very diverse study population because of broad eligibility criteria¹³

A high proportion (175/893 or 20%) of enrolled patients that met the criteria did not have a confirmed infection, resulting in a study that yielded a less than optimal test of the researchers' hypothesis

Under such circumstances the primary analysis should include all randomised patients. A secondary analysis that includes only patients who had the condition of interest and that is based on data collected before randomisation can also be informative and unbiased (see Discussion).

Patients randomised before eligibility for inclusion can be confirmed

If investigators expect delays in obtaining clinical or laboratory information on patients' eligibility, they should ideally postpone randomisation until this information is available. However, even with sound methods and procedures and the best of intentions, instances when patients must be randomised before all the data needed to confirm eligibility are available will occur (box B3).

Box 3.

Randomisation of patients before data are available to confirm their eligibility

A clinical trial is designed to study the effectiveness of a new anti-influenza drug. To be effective the drug must be given within 48 hours of onset of influenza symptoms, which may be indistinguishable from symptoms of other infections

All consenting patients who present to a doctor within 48 hours of development of influenza-like symptoms are enrolled and randomised into the trial

The study protocol stipulates that only patients who later give positive results on culture or serological tests for influenza infection will be analysed in the results, meaning that a number of patients randomised to the study are excluded retrospectively

Excluding such patients has serious potential implications. For example, one study of an anti-influenza drug randomised 629 patients, of whom 255 (40%) were later found to not have influenza.¹⁴ The study reported that, in the 374 patients who were infected, the study drug reduced the duration of illness by 30% (P<0.001). However, analysis of all 629 randomised patients shows a less dramatic but still significant effect of the study drug, with a reduced duration of 22% (P=0.004). Although in this particular case the result of the intention to treat analysis was significant, exclusion of 40% of randomised patients in many trials could have a more dramatic impact on results and could transform a null result into a positive one, reflecting the biological effect of the treatment in patients with the target condition.

On the other hand, retrospective exclusion of a large number of patients who would not be expected to benefit from the treatment creates a potentially misleading impression of the overall effect (positive and negative) of the treatment on the population to whom it will be applied. For example, the antiviral drug in this study caused nausea or vomiting in 19% of all randomised patients. Presumably the 255 patients who received the drug but did not have influenza experienced the same degree of side effects, without any benefit.

This clinical scenario mirrors real life clinical situations where doctors need to treat patients before all information is available. The major issue in the interpretation of results becomes one of effectiveness versus efficacy or explanatory versus pragmatic approaches. One would want to be sure that the benefit of the study drug to patients with the underlying condition outweighs the harm to patients exposed to the drug without possibility of benefit. Therefore, the primary presentation of the results should include all the patients randomised into the study. Exclusion or failure to report outcomes of patients without the condition of interest, but whom doctors must necessarily treat, risks underestimating the negative sides of the intervention. Investigators should also conduct a secondary analysis of efficacy, particularly when the intention to treat analysis leaves uncertainty as to whether the treatment is effective. This analysis, if it adheres to the rules of blinded adjudication we described above, will lead to an unbiased estimate of treatment effect in patients who truly had the underlying condition of interest.

Patients prematurely randomised into a clinical trial

Premature randomisation occurs when clinical circumstances evolve so that the patient never receives the intervention (an issue of methodology). Trials evaluating universal prestorage leucoreduction of red blood cells before surgery show the effect of premature randomisation (box B4).

Box 4.

Premature randomisation

In one trial of leucodepletion of red blood cells patients were randomised before an operation rather than when a unit of red blood cells was requested by the surgical team.¹⁵ The point of randomisation was premature, and 36% of the patients randomised to the study did not need a transfusion

When the investigators removed these patients from the analysis, the odds ratio was 1.28 (95% confidence interval 0.88 to 1.88); when they included these patients, the odds ratio was 1.15 (0.84 to 1.57)

Excluding all randomised patients who did not receive a unit of red blood cells will not bias the analysis, as long as allocation to treatment or control arm could not influence the likelihood that patients receive a transfusion. We believe this is a secure inference. The only impact of excluding patients who did not receive a transfusion will be to enhance the precision of the estimate—and the meaningfulness of the estimate of relative risk reduction for the clinician. To ensure that allocation could not have influenced whether patients received a transfusion, investigators should report an analysis of all randomised patients, as well as baseline characteristics for all patients excluded from the analysis.

In studies in which only patients allocated to one of two arms will receive the target intervention, excluding such patients will lead to biased results. For example, in a clinical trial of epidural anaesthesia in childbirth, some women randomised to the epidural treatment arm did not need an epidural because their pain levels did not rise above their personal thresholds.¹⁶ Investigators should not exclude these patients from the analysis, as they cannot identify similar patients in the control arm.

Discussion

Ideally all information to assess patients' eligibility for inclusion in a study will be available at the time of enrolment. Unfortunately resources and logistics mean that information collected before randomisation sometimes comes to light only later. How should investigators deal with such information in their analysis? Our approach to this issue is based on the desirability of minimising bias and random error and presenting analyses that give maximum information to clinicians, whether they are interested in explanatory or management questions.

Excluding randomised patients from the primary analysis may be legitimate when

• study personnel made errors in the implementation of eligibility criteria, or

• patients never received the intervention.

In these cases excluding patients does not introduce bias and may lead to a more informative analysis if an independent, blinded adjudication committee makes this determination after evaluating all randomised patients.

In contrast, investigators should not exclude patients from the primary intention to treat analysis if the treatment given could have influenced the ultimate decision regarding exclusion, as may occur with excessively broad eligibility criteria. When patients are randomised before information is available to confirm their eligibility for inclusion, the exclusion of patients who ultimately prove not to have the target condition will lead to an unbiased assessment of treatment effect in patients who do meet inclusion criteria. However, this analysis will not address the ultimate effect of treatment in everyone who will receive it in clinical practice if clinicians cannot establish definitive eligibility requirements at the point when they must make treatment decisions. As a result, presenting only an analysis based on patients who ultimately proved to have the target condition is likely to mislead.

Although excluding patients from an analysis in certain circumstances does not bias the results, investigators must still adhere to the highest standards of methodological design and trial execution. To discourage carelessness in defining eligibility and later “tidying up” of data, investigators need to specify explicitly any foreseeable post-randomisation exclusions in the protocol.