Interpreting the Results of Intention-to-Treat, Per-Protocol, and As-Treated Analyses of Clinical Trials

Nonadherence in a randomized clinical trial (RCT) occurs when study participants do not follow the randomly assigned treatment protocol. Reasons for nonadherence may include the study participant not taking trial medications, crossing over to the other intervention being studied, or accessing treatment outside of the trial. Nonadherence also may occur when the clinician is unable to complete the assigned therapy (eg, a surgical procedure) as intended.

The CABANA clinical trial published in JAMA by Packer et al¹ was difficult to interpret because of nonadherence with the treatment protocol that resulted from substantial crossover between groups. In this trial, patients with atrial fibrillation were randomized to either undergo catheter ablation or receive conventional medical therapy. Of the 1108 participants randomized to ablation, 102 (9%) did not receive the procedure. Of the 1096 patients randomized to drug therapy, 301 (27%) underwent ablation during the follow-up period, resulting in nonadherence to assigned treatment in both groups of the study. Interpretation of the effect of catheter ablation on atrial fibrillation differed based on alternate ways of analyzing the trials results. Intention-to-treat (ITT), per-protocol (PP), and as-treated (AT) approaches to analysis differ in how the included patient population and treatment assignments are defined, with important implications for interpretation of treatment effects in clinical trials.

ITT, PP, and AT Approaches to Analyses

The ITT principle is the most commonly used approach for the primary analysis of RCTs. It measures the effect of assigning patients to treatment, which includes differences in individuals’ adherence.² With the ITT approach, all randomized patients are included in the analysis, based on the groups to which they were initially randomly assigned. The PP and AT analyses estimate the effect of receiving a treatment.^3,4 Per-protocol only analyzes data from participants who follow the protocol, excluding their data after they become nonadherent. AT analyses consider the treatment actually received by the participant, without regard to adherence to their randomization assignment.

As-treated and PP analyses are not simple to interpret because of the potential loss of an important benefit of randomization: the elimination of systematic bias in treatment assignment. Selection bias and confounding in the treatment effect estimate arises if patients who are more adherent with the assigned treatment differ in ways that also influence their outcomes compared with those who are less adherent. Similarly, when comparing medical and procedure-based treatments, as occurred in the CABANA trial, patients who can have a procedure performed compared with those who cannot may have differing prognoses. Consequently, PP and AT analyses must apply the types of statistical methods used in non-randomized studies to account for potential differences among patients and mitigate the effects of confounding.

Why Are ITT, PP, and AT Approaches to Analysis Important?

Analyses that adjust for nonadherence provide important information to complement the effects estimated from the ITT approach to analysis. An ITT-based analysis estimates the effect of being assigned to a given treatment; therefore, the magnitude of the estimated effect reflects both the inherent effect of the treatment in addition to the proportion of patients that receive it.

However, methods that attempt to quantify the undiluted effect of receiving treatment can provide important information about the potential magnitude of treatment effects when patients adhere to the protocol. Both the estimated effect of being assigned to a treatment (from an ITT-based analysis) and the estimated effect of full adherence with a treatment may be useful to both clinicians and patients to fully understand the range of potential treatment effects. The treatment effect estimated from PP or AT analyses is frequently larger than the effect size estimated from the ITT-based approach, particularly when adherence to treatment is associated with larger treatment effects.

Limitations of ITT, PP, and AT Approaches to Analyses

Effect estimates based on ITT approaches depend on the amount of adherence to randomized treatment and may not generalize to settings where adherence behavior differs from that observed in the study. When a clinician and patient decide to initiate treatment, it is generally not possible to know whether that patient will adhere. Thus, the treatment outcomes for an individual patient cannot necessarily be predicted from the results of an ITT-based analysis.

Randomization is conducted with a goal of ensuring that, on average, the baseline characteristics of patients in the groups of an RCT are comparable. This balance between groups may be lost in both PP and AT analyses because patients who are and are not adherent with randomized treatment may differ in important characteristics that also influence their outcomes.² In principle, the patient characteristics associated with the decision to adhere with the assigned treatment or pursue alternative treatments and the outcome must be measured and accounted for in AT or PP analyses. However, the assumption that these characteristics can be quantified is untestable, resulting in an inability to draw firm conclusions about treatment effects from PP or AT analyses. For example, if some patients discontinue treatment because they find it ineffective, comparing their outcomes with outcomes among patients who perceive the treatment to be effective and continue treatment as instructed without addressing the reasons for discontinuation may make the treatment appear more effective than it truly is. Therefore, as appealing as it may seem to rely on PP or AT findings to isolate the effect of receiving an intervention, the beneficial effects of creating balance between groups achieved by randomization are lost with PP or AT analysis and the results should be interpreted with caution because bias from confounding can be introduced without randomization.

It is also important to note that many RCTs that analyze patients according to their randomized group do not strictly follow the ITT principle. Examples of modifications include limiting the analyses to patients who received a first dose of a medication or only analyzing patients who completed follow-up. Such modifications have the potential to introduce selection bias into the analyses.

How Did the CABANA Study's Interpretation Differ When Analyzed by ITT, PP, and AT Approaches?

The CABANA trial was analyzed by ITT, PP, and AT approaches.¹ In the PP analysis, patients in the medical therapy group who underwent ablation treatment had their follow-up results included in the medical therapy group up to the time they underwent ablation. The ablation group included patients initially randomized to ablation who underwent an ablation within 6 months. The AT analysis used a Cox model with catheter ablation included as a time-dependent covariate, including all data and updating treatment status as patients received ablation, regardless of the treatment group to which they were assigned.

Both the AT and PP analyses in the CABANA trial¹ used Cox regression with a set of prespecified baseline characteristics included as covariates to adjust for potential bias due to nonadherence.

In the CABANA trial,¹ the primary composite outcome included death, disabling stroke, serious bleeding, or cardiac arrest. The ITT-based analysis suggested no meaningful differences between trial groups based on how patients were initially assigned to treatment, irrespective of what treatments they ultimately received during the course of the study (hazard ratio [HR], 0.86 [95% CI, 0.65-1.15]). Being randomized to ablation did not significantly reduce the primary composite outcome rate compared with being randomized to drug therapy. The PP analysis suggested a potentially larger, but still not statistically significant, effect of catheter ablation (HR, 0.74 [95% CI, 0.54-1.01]). In the AT analysis, the resulting estimate of benefit of undergoing ablation reached statistical significance (HR, 0.67 [95% CI, 0.50-0.89]).

How Should the CABANA Trial Be Interpreted in Light of the Protocol Nonadherence That Occurred?

Taken together, the results of the analyses based on ITT, PP, and AT approaches suggest a possible beneficial effect of ablation when patients are able to undergo the procedure. Future analyses of CABANA considering postrandomization factors such as drug adverse effect profiles or recurrent atrial fibrillation episodes that might have influenced treatment decision may provide additional insight into the effect of ablation for treating atrial fibrillation. For example, if patients unable to undergo ablation had worse health status or more comorbid disease following randomization, that could bias the results to favor ablation because healthier patients would have undergone the procedure. Conversely, if patients randomized to medical therapy with severe atrial fibrillation symptoms during follow-up were more likely to seek ablation, this could make ablation appear less effective in a PP or AT analysis because patients with more severe disease would have received the procedure.