Intent-to-Treat (ITT) vs Completer or Per-Protocol Analysis in Randomized Controlled Trials

Abstract

In randomized controlled trials, randomization creates groups that are reasonably well balanced on all baseline variables, whether measured, unmeasured, or unknown. Postbaseline events disturb this balance, resulting in postrandomization biases. Drop-out is one such event. There are two main methods for data analysis when there are dropouts. One method is to analyze data from only those who complete the study (completer analysis), or only those who complete the study and also comply with all its key elements (per-protocol analysis, a special type of completer analysis). The other method is to analyze the data from all randomized patients, regardless of dropout (intent-to-treat [ITT] analysis), or all randomized patients who meet an additional criterion, such as taking at least one dose of study drug (modified ITT [mITT] analysis, a special type of ITT analysis). Completer analyses present results in the ideal situation in which patients take medications as advised. ITT analyses present results related to real-world practice, where patients may be irregular with dosing or stop taking medications. The advantages and disadvantages of each type of analysis are discussed. The handling of missing data in ITT and mITT analysis is also briefly discussed.

Randomized controlled trials (RCTs) are the gold standard for the clinical evaluation of new treatments; however, few realize the extent to which RCTs are vulnerable to bias. To start with, RCTs usually recruit a convenience sample from hospital settings; patients who seek hospital care, especially specialist care, usually differ in many ways from patients with the same diagnosis in the general population. Next, RCTs set many inclusion and exclusion criteria, and the purposive sample that emerges makes the sample even less representative of patients in the general population. Furthermore, the sampling of eligible patients is almost never random. Finally, the ethical need to recruit only consenting patients adds bias. The limitations of convenience and purposive samples were considered in an earlier article in this column. ¹

Randomization and Baseline Matching

Regardless of such biases, randomization is expected to create groups that are reasonably well-matched on all baseline variables, whether measured, unmeasured, or unknown. Measured baseline variables that can influence RCT outcomes include age, sex, baseline severity of illness, and so on. Unmeasured baseline variables that can influence outcomes include attitudes towards treatment that might influence the placebo response and psychosocial stressors that might sustain psychological dysfunction despite treatment efforts. Unknown baseline variables that can influence outcomes include genetic, neurohistological, gut microbiome, and other factors that can influence the efficacy and tolerability of the treatment under study.

Postrandomization Bias

Assuming that randomization has done its job, the treatment groups should be well-matched at baseline; this is necessary for the internal validity of the study. ² Unfortunately, threats to internal validity continue to arise in the form of postrandomization biases. ³ As examples, the use of over-the-counter medications and rescue medications may differ between groups across the course of the RCT. The rates of missed medication doses may also differ between groups. And most problematic of all, the dropout rate and the reasons for dropping out may differ between groups.

Intent-to-Treat vs Completer Analyses

Inevitably, because of dropouts, there are fewer patients at the end of the study than there were at the beginning. In a completer analysis, the data are analyzed only for patients who reach the study endpoint. The advantage of a completer analysis is that it tells us how the treatment performs in an ideal situation; that is, when patients take the treatment for the specified duration. The disadvantages, however, are many. For example, in a drug vs placebo RCT, placebo patients may drop out due to inefficacy of treatment. This means that the placebo patients who complete the study are those in whom the placebo response, for whatever reason, ⁴ is higher than the true average for the group; so, the completer analysis may fail to demonstrate an advantage for the study drug. Next, patients who receive the study drug may drop out due to adverse effects; in such a situation, a completer analysis will fail to capture the downsides of the drug. Another problem is that the sample size is attenuated in a completer analysis; this reduces the statistical power of the study. Finally, in a completer analysis, the benefits of randomization are lost because the composition of the original groups has been disturbed.

An intent-to-treat (ITT) analysis is an attempt to improve the internal validity of the study by including all randomized patients in the analysis, whether they completed the study or dropped out early. So, the sample size is not attenuated and statistical power is maintained; additionally, the integrity of randomization is preserved. The ITT sample can be defined in different ways. The basic definition is that it comprises all patients who were randomized. For practical and often justifiable reasons, modified ITT (mITT) samples may be defined. Possibilities examples of mITT samples are all patients who were randomized and took at least one dose of the study drug or all patients who were randomized and attended at least one follow-up visit. Whatever definition is used must be stated in advance, in the study protocol, and not after the study is over and the data are available for examination.

A problem in ITT analysis is that, when patients drop out, data are available for them only until the point at which they dropped out; so, what data should be entered for them for subsequent follow-up visits, including the endpoint study visit? The traditional solution is to use the “last-observation-carried-forward” (LOCF) approach; here, the ratings recorded at the last visit before dropout are entered for subsequent visits, all the way to the endpoint visit. A serious limitation of the LOCF method is that the missing data that it replaces are almost never missing at random; so the LOCF method for data imputation introduces bias into the analysis.^5,6

Handling Missing Data

Many methods for data imputation are now employed in lieu of LOCF^7,8; these have even been incorporated into statistical packages with tutorials available online. Older methods of imputation involved the substitution of the mean for the group, the mean for the subgroup, or a value generated using regression. These are simple but fallacious because they reduce the standard deviation of the group, provide a false impression of precision, and increase the risk of a Type 1 statistical error. Better methods are now available, such as multiple imputation and the maximum likelihood methods. Unfortunately, most methods of data imputation assume that the data are missing at random, or missing completely at random, rather than not missing at random, which is most commonly the case. ⁵

General Notes

Per-protocol analysis is a special type of completer analysis. In per-protocol analysis, data are analyzed only for those patients who completely adhere to the treatment protocol. This means that the patients not only reach the study endpoint without dropping out but also complete all key assessments at all study visits, show good treatment adherence, etc.

Completer analysis and especially per-protocol analysis represent results in the ideal situation in which patients take treatment as advised and for the duration advised. ITT analysis represents real-world practice where patients may or may not take their medications as advised and drop out if they feel that they are not improving or if they do not like the side effects of the drug. So, which method of analysis should be employed: ITT or completer? The answer is both! Whereas the ITT analysis is almost universally considered to be the preferred primary analysis, completer analysis can also be performed so that the reader understands how well the study drug performs when it is taken as advised.

As a parting note, there are some situations where a completer analysis is the only meaningful analysis. Consider a study of the cognitive adverse effects of unilateral vs bilateral electroconvulsive therapy. Cognitive assessments are conducted at baseline and at endpoint. The only way to answer the research question is to analyze data from only those patients who complete the endpoint cognitive assessments.