What is meant by intention to treat analysis? Survey of published randomised controlled trials

Abstract

Objectives

To assess the methodological quality of intention to treat analysis as reported in randomised controlled trials in four large medical journals.

Design

Survey of all reports of randomised controlled trials published in 1997 in the BMJ, Lancet, JAMA, and New England Journal of Medicine.

Main outcome measures

Methods of dealing with deviations from random allocation and missing data.

Results

119 (48%) of the reports mentioned intention to treat analysis. Of these, 12 excluded any patients who did not start the allocated intervention and three did not analyse all randomised subjects as allocated. Five reports explicitly stated that there were no deviations from random allocation. The remaining 99 reports seemed to analyse according to random allocation, but only 34 of these explicitly stated this. 89 (75%) trials had some missing data on the primary outcome variable. The methods used to deal with this were generally inadequate, potentially leading to a biased treatment effect. 29 (24%) trials had more than 10% of responses missing for the primary outcome, the methods of handling the missing responses were similar in this subset.

Conclusions

The intention to treat approach is often inadequately described and inadequately applied. Authors should explicitly describe the handling of deviations from randomised allocation and missing responses and discuss the potential effect of any missing response. Readers should critically assess the validity of reported intention to treat analyses.

Key messages

• Intention to treat gives a pragmatic estimate of the benefit of a change in treatment policy rather than of potential benefit in patients who receive treatment exactly as planned

• Full application of intention to treat is possible only when complete outcome data are available for all randomised subjects

• About half of all published reports of randomised controlled trials stated that intention to treat was used, but handling of deviations from randomised allocation varied widely

• Many trials had some missing data on the primary outcome variable, and methods used to deal with this were generally inadequate, potentially leading to bias

• Intention to treat analyses are often inadequately described and inadequately applied

Introduction

“Intention to treat” is a strategy for the analysis of randomised controlled trials that compares patients in the groups to which they were originally randomly assigned. This is generally interpreted as including all patients, regardless of whether they actually satisfied the entry criteria, the treatment actually received, and subsequent withdrawal or deviation from the protocol. However there is a debate about the validity of excluding specific cases within each of these categories from an intention to treat analysis.¹ Clinical effectiveness may be overestimated if an intention to treat analysis is not done.²

The intention to treat approach has two main purposes. Firstly, the approach maintains treatment groups that are similar apart from random variation. This is the reason for randomisation, and the feature may be lost if analysis is not performed on the groups produced by the randomisation process. For example, in a trial comparing medical and surgical treatment for stable angina pectoris, some patients allocated to surgical intervention died before being operated on.³ If these deaths are not attributed to surgical intervention using an intention to treat analysis, surgery seems to have a falsely low mortality (table 1). Secondly, intention to treat analysis allows for non-compliance and deviations from policy by clinicians. There are, of course, exceptions. Some types of deviations from randomised allocation may occur only within the trial setting and would not be expected in routine practice. For example, in a trial comparing active and placebo vaccination there is the potential for placebo vaccine to be incorrectly administered in place of active, but this could not occur outside the trial and so need not be accounted for in estimates of potential efficacy. However, most types of deviations from protocol would continue to occur in routine practice and so should be included in the estimated benefit of a change in treatment policy. Intention to treat analysis is therefore most suitable for pragmatic trials⁴ of effectiveness rather than for explanatory investigations of efficacy.

Table 1.

Use of intention to treat and other methods to analyse trial of coronary artery bypass surgery and medical treatment for stable angina pectoris in 768 men.² Mortality 2 years after randomisation is shown by allocated and actual intervention^*

	Allocated (actual) intervention				Differences in mortality (95%CI) surgical v medical
	Medical (medical)	Medical (surgical)	Surgical (surgical)	Surgical (medical)
No of survivors	296	48	353	20	—
No of deaths	27	2	15	6	—
Mortality (%)	8.4%	4.0%	4.1%	23.1%	—
Intention to treat analysis	7.8% (29/373)		5.3% (21/394)		2.4% (−1.0% to 6.1%)
Per protocol analysis	8.4% (27/323)		4.1% (15/368)		4.3% (0.7% to 8.2%)
As treated analysis	9.5% (33/349)		4.1% (17/418)		5.4% (1.9% to 9.3%)

^*77 patients did not receive allocated intervention for various reasons (one was not available for follow up and is not included in table). The high death rate in the group assigned to surgery but not receiving it is due to 6 patients who died before they could be operated on.The authors correctly reported the intention to treat analysis, which shows no significant difference between the treatments. 

Deviations from randomised allocation often result in missing outcome data. A full application of the intention to treat approach is possible only when complete outcome data are available for all randomised subjects. Care must always be taken to minimise missing responses and to follow up those who withdraw from treatment, but this is particularly important for the implementation of an intention to treat analysis.⁵ No consensus exists about how missing responses should be handled in intention to treat analyses, and different approaches may be appropriate in different situations. Practice also varies over handling of false inclusions (subjects found after randomisation not to satisfy the entry criteria). Thus, there is no single definition of an intention to treat analysis, and the phrase seems to have different meanings for different authors.⁶ We carried out a survey of recently published reports to examine current application of intention to treat analysis.

Methods

We identified all reports of randomised controlled trials published in 1997 in four major medical journals: BMJ, Lancet, JAMA, and New England Journal of Medicine. All except the New England Journal of Medicine have adopted the CONSORT statement,⁷ which requires that authors indicate whether analyses were performed on an intention to treat basis. The total number of randomised controlled trials was obtained by Medline searches for publication type “randomized controlled trials” within each journal and cross checked against the Cochrane controlled trials register.⁸ The journals were then hand searched to identify trials which reported an intention to treat analysis. For articles in the BMJ and Lancet, we also carried out a full text search for “intention to treat” or “intent to treat” on the internet (www.bmj.com, www.thelancet.com).

All trials that reported an intention to treat analysis were then independently assessed by both authors. We considered deviations from random allocation, false inclusions, and missing response. For each trial we recorded whether each of these occurred, and if so, the method of analysis and whether this method was explicitly stated. The assessment of missing response was limited to the primary outcomes if any were specified. Any uncertainties or disagreements between the two assessments were resolved by consensus.

Results

About half of all the randomised controlled trials reported an analysis explicitly described as intention to treat (table 2), with similar proportions in each journal. A total of 119 reports of randomised controlled trials including an intention to treat analysis were assessed. Table 3 summarises their characteristics.

Table 2.

Randomised controlled trials identified for assessment

Journal	No of trials	No (%) reporting intention to treat
BMJ	42	18 (43)
JAMA	35	15 (43)
Lancet	84	45 (54)
N Engl J Med	88	41 (47)
Total	249	119 (48)

Table 3.

Characteristics of intention to treat analyses

	No (%) of trials (n=119)
Deviations from randomised allocation
Did not analyse as randomised	15 (13)
Stated no deviations	5 (4)
Stated analysed as randomised	34 (29)
Appeared to analyse as randomised	65 (55)
Missing outcome
Stated none missing	30 (25)
<10% missing	60 (50)
⩾10% missing	29 (24)
False inclusions
Reported as included	6 (5)
Reported as excluded	19 (16)
None reported	94 (79)

Most reports stated in the methods section that intention to treat analysis was used but did not specify how any deviations from randomised allocation, false inclusions, or missing outcomes were handled. Of the 15 reports that did not analyse according to randomised allocation, 12 specifically excluded from the analysis any patients who did not start the allocated intervention (table 4). Three papers described intention to treat analyses that do not comply with the basic principle of analysing all randomised subjects as allocated. In a report of a trial comparing conventional anterior surgery and laparoscopic surgery for repairing inguinal hernia, various patients were excluded, including those not receiving the allocated intervention:

Table 4.

Trials in which patients who did not start allocated intervention were excluded from intention to treat analysis

Study	Population	Interventions	Outcome	Exclusions from analysis*
Spruance et al9	2209 patients with history of frequent episodes of herpes simplex labialis	Topical penciclovir or vehicle control cream for recurrence of classic cold sore	Lesion healing	636 patients who did not start treatment
Fazekas et al10	150 patients with relapsing-remitting multiple sclerosis	Monthly intravenous immunoglobulin or placebo	Clinical disability	2 patients in placebo group who withdrew consent between randomisation and start of treatment
CAESAR11	1895 patients infected with HIV-1 and CD4 counts of 25-250×106/l	Addition of lamivudine, or lamivudine and loviride, or placebo to zidovudine based regimens	Progression to new protocol defined AIDS event or death	35 patients who did not start the study treatment. The study was prematurely terminated, but it is not clear whether this was the cause
Landoni et al12	343 women with newly diagnosed stage Ib and IIa cervical cancer	Radical surgery or radiotherapy	Survival	2 patients randomised to surgery: 1 for progression before operation and 1 refused any therapy after randomisation
Rutgeerts et al13	854 patients with bleeding peptic ulcer	Three endoscopic treatments: single injection of polidocanol, or single injection of fibrin glue, or repeated injection of fibrin glue	Endoscopic rebleeding	4 patients in whom, after randomisation, injection treatment turned out to be impossible
Nashan et al14	380 renal allograft recipients	Basiliximab or placebo	Acute rejection	Four patients who received study drug but did no transplant
Jacobson et al15	60 HIV positive patients with oral aphthous ulcers	Thalidomide or placebo	Complete ulcer resolution and change in HIV load	2 patients whose ulcers healed between screening and start of study treatment
Kaplan et al16	198 HIV positive patients with previously untreated, aggressive non-Hodgkin’s lymphoma	Low or standard dose chemotherapy	Survival	1 patient who was never treated due to an acute opportunistic infection and 1 lost to follow up after randomisation but before start of treatment
Englund et al17	839 children with HIV infection	Zidovudin or didanosin, or both	Time to death or progression of HIV disease	7 patients excluded because treatment was refused after randomisation
Tardif et al18	317 patients having elective angioplasty	Combinations of antioxidants or placebo, starting preoperatively	Extent of restenosis	Analysis included “all randomised patients with successful angioplasty” (11 exclusions)
Guilhot et al19	745 previously untreated patients with chronic myelogenous leukaemia	Interferon α-2b and cytarabine or interferon alone	3 year survival	3 patients who declined to participate immediately after randomisation
Daoud et al20	131 patients scheduled for elective heart surgery	Preoperative amiodarone or placebo	Clinical outcome, complications, length of hospital stay, and cost	7 patients in whom surgery was cancelled after randomisation

^*Patients excluded because they did not start the allocated intervention. 

Data on all patients who were randomly assigned ... were analysed on an intention to treat basis. In this analysis we did not include patients without hernias, those who withdrew their consent before undergoing surgery, those who at the time of surgery were found to be poor candidates for general anaesthesia, and those who did not undergo the assigned operation because of a misunderstanding resulting in an unplanned open or laparoscopic repair.²¹

This resulted in the exclusion of 57 (5%) enrolled patients.

In a trial of endometrial resection or hysterectomy for menorrhagia, the authors excluded from the intention to treat analysis 26 (13%) women who withdrew after randomisation but before surgery.²² The researchers contacted 10 of these women and found that, “of six who had been assigned endometrial resection, four had hysterectomy and two had resection, whereas three of four assigned hysterectomy chose endometrial resection and one chose hysterectomy.” In a trial of folic acid supplementation, 17 (14%) women were excluded because of non-compliance.²³ The aim of this trial was to predict the likely effect of food fortification, which would not provide the same opportunity for non-compliance as supplementation using tablets. Thus, exclusion of women who did not comply was appropriate, but it should not have been described as an intention to treat analysis.

Five reports explicitly stated that there were no deviations from random allocation. The remaining 99 reports seemed to analyse according to random allocation, but only 34 of these explicitly stated this. Of the 25 reports which stated that false inclusions had occurred, only a quarter included these cases in the reported intention to treat analysis (table 3).

Eighty nine trials had some missing data on the primary outcome variable. The most common method of handling missing data was complete case analysis (44, 49%), in which all patients with a missing response are excluded from the analysis. Twenty nine (33%) papers used all available information on each patient (28 censored at end of follow up and one used all available outcome measurements over five assessments). Fifteen (17%) imputed values for the missing response. The imputation methods used were carry forward of last observed response (seven), explicit allocation of poor outcome (four), implicit assumption of good or poor outcome by including patients with missing response in the denominator but not the numerator when calculating rates (three), and use of the group average (one). Only one paper examined the effect of using a range of methods to handle the missing responses.²⁴ Twenty nine (24%) trials had more than 10% of responses missing for the primary outcome, the methods of handling the missing responses were similar in this subset.

Discussion

Almost half the reports of randomised controlled trials included an analysis described as intention to treat. This compares with 12% of trials found in a survey of reports published in obstetric and gynaecological journals in 1990-1.²⁵ Evidence based health care encourages appraisal of research methods, and critical appraisal guides for trials usually include a question on whether follow up was complete and whether subjects have been analysed in the groups to which they were randomised.²⁶ This increased general awareness of intention to treat analysis may have contributed to its incomplete use in the analysis of randomised controlled trials. The trials may have not been planned with a complete strategy for the reduction and handling of deviations from the allocated intervention.

Failure to start intervention

The exclusion of patients who did not start the allocated intervention from the intention to treat analysis was fairly common (10%). In some situations this seems sensible and is unlikely to lead to bias—when the intended effect of an intervention depends on the occurrence of a subsequent event that cannot be influenced by the randomised allocation. For example, prophylaxis for prevention of transplant rejection can be effective only if a transplant is received; it seems unlikely that allocation to active treatment or placebo could affect this. Ideally, these situations should be avoided by randomisation after the necessary event, but this is not always possible in practice. Perhaps more could be achieved towards appropriate timing of randomisation, as illustrated by the surgeon who ensured randomisation after diagnosis by tossing a sterilised coin in the operating theatre once the patient’s abdomen was open.²⁷ Unless the possibility of bias can be confidently rejected, patients who did not start the allocated intervention should be included in the intention to treat analysis where possible.

Non-compliance

If deviations from randomised allocation are due to non-compliance of the patient, the effect of the intervention if compliance had been complete may be relevant. However, naive comparisons based on compliance may be misleading. For example, the coronary drug project²⁸ found a substantially lower five year mortality in patients who complied well with clofibrate than in those who complied poorly, which seemed to indicate clofibrate was beneficial when taken as instructed. However, when compliance was examined in the placebo group, death rates in patients with both good and poor compliance were similar to those in the clofibrate group. The authors concluded that there are serious difficulties in evaluating treatment efficacy in subgroups defined by patient responses after randomisation. Considerable work has been carried out on valid statistical analysis of the effect of compliance in clinical trials,^29–32 but this is a complex area and should be approached with care.

False inclusions

False inclusions should also generally not be excluded from an intention to treat analysis.³³ Their exclusion can be justified only if the reascertainment of the entry criteria is applied identically in each group. From a pragmatic viewpoint, if false inclusions occur in the controlled environment of a trial, it seems inevitable that misclassification will also occur in routine clinical practice.

Missing response

The main problem in the application of intention to treat seen in this survey was the handling of missing response. Inappropriate handling of missing response can produce misleading conclusions. Table 5 shows the effect of various approaches. Complete case analysis, which was the approach used in most trials, violates the principle of intention to treat and leads to bias unless data are missing at random—that is, absence of an observation is independent of the outcome.³⁵,³⁶ Partial information, such as outcome at some time points, or time to drop out, may be used to produce a more efficient analysis, but this is still potentially biased.³⁷

Table 5.

Effect of various methods of handling missing response in trial comparing balloon angioplast and stent placement for obstructed coronary bypass grafts³⁴

	Angioplasty (n=110)	Stent (n=110)	Absolute difference in restenosis rate (95% CI)
Restenosis at 6 months:
Yes	37	32	—
No	43	54	—
Unknown	30	24	—
Rate of restenosis (%):
Complete case analysis	46 (37/80)	37 (32/86)	9 (−6 to 24)
Assuming poor outcome	61 (67/110)	51 (56/110)	10 (−3 to 23)
Assuming good outcome	34 (37/110)	29 (32/110)	5 (−8 to 17)
Extreme case favouring stenting	61 (67/110)	29 (32/110)	32 (19 to 44)
Extreme case favouring angioplasty	34 (37/110)	51 (56/110)	−17 (−30 to −4)

The authors reported the complete case analysis, ignoring all those with unknown outcome. This shows no significant difference between the groups with a fairly wide confidence interval. As the amount of missing data was similar in each group, assuming all patients in both groups with unknown outcome to have had either a good or a poor outcome gives similar results. However, the extreme cases in both directions show a significant difference between the two procedures. Therefore, the large amount of missing data makes it impossible to draw a valid conclusion on the difference between the two procedures. 

Various imputation methods may be used to estimate the missing responses. However, clinical trials usually do not collect sufficient data to allow good estimation, and the only commonly feasible options are using the last observed response (carry forward) or assuming that all missing responses were constant. Extreme case analysis (for example, all patients lost to the group that fared better are assigned a poor outcome; all lost to the group that fared worse are assigned a good outcome) has also been recommended,³⁸ but this is unlikely to yield a conclusive answer in practice (Meyer K, Windeler J, 19th International Society for Clinical Biostatistics meeting, Dundee 1998). More sophisticated techniques for handling missing data are available³⁹ but depend on assumptions about the missing data mechanism which cannot be completely verified in most clinical trials. In general, imputation is used to produce a conservative estimate of treatment effect. However, no imputation method can give an unbiased estimate of the treatment effect unless the assumptions made about the missing data are valid. To fully appreciate the potential influence of missing responses, some form of sensitivity analysis is recommended, examining the effect of different strategies on the conclusions.

Recommendations for intention to treat analysis “ITT is better regarded as a complete trial strategy for design, conduct and analysis rather than as an approach to analysis alone”⁵

Design

• Decide whether the aim is pragmatic or explanatory. For pragmatic trials, intention to treat is essential

• Justify in advance any inclusion criteria which when violated would merit exclusion from intention to treat analysis

Conduct

• Minimise missing response on the primary outcome

• Follow up subjects who withdraw from treatment

Analysis

• Include all randomised subjects in the groups to which they were allocated

• Investigate the potential effect of missing response

Reporting

• Specify that intention to treat analysis has been carried out, explicitly describing the handling of deviations from randomised allocation and missing response

• Report deviations from randomised allocation and missing response

• Discuss the potential effect of missing response

• Base conclusions on the results of intention to treat analysis

Implications

Full reporting of any deviations from random allocation and missing response is essential in the assessment of the necessity and appropriateness of an intention to treat approach, as emphasised in the CONSORT guidelines on the reporting of randomised controlled trials.⁷ However, the CONSORT guidelines do not address intention to treat analysis in any detail and so we have provided recommendations for its implementation (box).

Our survey revealed that the intention to treat approach is often inadequately described and inadequately applied. We hope that future researchers will take note of our recommendations, but we advise readers to assess critically the validity of reported intention to treat analyses.