Beyond intention-to-treat in complex, cluster randomized trials: the difficulty of defining compliance

To the Editor

In their excellent re-analysis of a cluster-randomized trial that delivered combinations of water, sanitation, and handwashing (WASH) improvements to schools in rural Kenya, Garn and colleagues¹ showed how imperfect compliance likely attenuated some intention-to-treat estimates by comparing them with complier average causal effect estimates – the effect of the intervention among the subgroup of schools that complied with their intervention assignment. Since the subgroup of compliers cannot be observed in a parallel arm trial,² Garn and colleagues used an instrumental variables (IV) technique to estimate complier average causal effects, and should be commended for extending the original trial’s inference.

The paper illustrates two more general issues related to compliance definitions in community intervention trials. First, community trials that wish to use IV techniques to estimate complier average causal effects for infectious outcomes should define compliance at the level of randomization – individual-level definitions are generally problematic (Figure). Second, the definition of compliance for a multi-faceted intervention is centrally important for complier average causal effects, ‘per-protocol’ and ‘as treated’ parameters,³ yet defining compliance is rarely straightforward. A vaccine trial delivers treatment at specific ages or time points, which can be measured and defined unambiguously. This is not the case for many community level interventions – instead, they often have multiple components that involve behavior change, which can vary during follow-up. Garn and colleagues¹ used a pragmatic compliance definition based on a count of water, sanitation, and handwashing components present on a single visit during follow-up. Their definition weighted each intervention component equally and included no information about behavior change over the 18-month follow-up. Yet, each intervention component could be more or less important depending on the dominant pathogen mix, and actual use of the hardware should be important to the intervention’s efficacy. This example illustrates how compliance definitions can be complex, subject to data availability, and ultimately debatable. The clinical trials literature has developed compliance definitions based on pre-specified treatment regimens over time,³ and a similar approach may be helpful for community level interventions with time-varying compliance. More complex compliance definitions may reduce complier misclassification but may also open more room for debate. Since multiple compliance definitions may be reasonable in complex community interventions, and the choice has direct bearing on the magnitude and interpretation of complier average causal effects or other measures that rely on identifying compliers,³ it would be prudent for trials to pre-specify, justify, and fully report results for each definition.

Figure.

(a) Simplified causal diagram of a cluster randomized trial with allocation (A), compliance defined at the cluster level (C), potential confounders (W) and an outcome (Y). Random allocation (A) can be used as an instrumental variable to estimate the causal effect of C on Y, as illustrated by Garn and colleagues¹ (b) Extension of (a) with compliance and outcomes indexed for 2 individuals within a cluster. Had Garn and colleagues defined compliance at the pupil level then an individual’s probability of an enteric infection (Y₁) would be influenced by randomized assignment (A) through her own compliance (C₁) as well as the compliance of her peers (C₂) – a clear violation of the instrumental variable estimator’s assumptions and a common pitfall for cluster randomized trials in which one individual’s compliance affects outcomes of other individuals in the cluster, as with infectious diseases (edges from W to C₂ and Y₂ have been omitted for simplicity).