Negative Control Outcomes

Investigators have several design, measurement, and analytic tools to detect and reduce bias in epidemiological studies. One such approach, “negative controls,” has been used on an ad hoc basis for decades. A formal approach has recently been suggested for its use to detect confounding, selection, and measurement bias in epidemiological studies.^1,2 Negative controls in epidemiological studies are analogous to negative controls in laboratory experiments, in which investigators test for problems with the experimental method by leaving out an essential ingredient, inactivating the hypothesize dactive ingredient, or checking for an effect that would be impossible by the hypothesized mechanism.¹ A placebo treatment group in a randomized trial is an example of a negative control exposure (leaving out an essential ingredient) that helps remove bias that can result from participant or practitioner knowledge of an individual’s treatment assignment—the placebo treatment is susceptible to the same bias structure as the actual treatment but is causally unrelated to the outcome of interest.

Negative control outcomes are conceptually similar but are subtly different because, unlike exposures in a randomized trial, they are not under investigator control. The formal definition of a negative control outcome is one that shares the same potential sources of bias with the primary outcome but cannot plausibly be related to the treatment of interest. For example, early screening echocardiography for patent ductus arteriosus in extremely preterm infants was associated with a 4.3% absolute reduction of in-hospital mortality in a propensity-score matched analysis of a population-based cohort.³ To help check for residual bias from unmeasured confounding, investigators repeated the analysis using late-onset infections as a negative control outcome under the assumption that any sources of uncontrolled confounding in the mortality analysis would similarly lead to lower incidence of late-onset infection (an effect that would be impossible by the hypothesized mechanism). The finding of no association between echocardiography screening and the negative control outcome provided additional support for the conclusion from the primary analysis.

To date, nearly all discussions and applications of negative control outcomes have focused on use in observational studies to detect unmeasured confounding.^4,5 With sufficient sample size and proper allocation, randomized trials are protected from confounding bias when estimating an intention-to-treat effect; however, confounding, selection, and measurement bias can still threaten the validity of trials in many circumstances that regularly occur. For example, even masked trials with aplacebo control can be vulnerable to bias if the treatment has adverse effects (leading to selection bias from differential attrition or measurement bias from unblinding participants or practitioners). In this Viewpoint, we suggest that negative control outcomes can be a valuable addition to detect residual bias in randomized trials.

Confounding and selection bias are of greatest concern in clinical trials that report analyses beyond intention-to-treat. For example, trials with imperfect adherence often include as-treated and per-protocol analyses. As-treated analyses can be vulnerable to confounding bias because participants are analyzed according to the treatment regimen they actually followed irrespective of their randomized assignment, which may be confounded by prognostic factors. Per-protocol analyses restrict the analysis to participants who were adherent with their randomized assignment and can be vulnerable to selection bias because participants who are adherent are usually different from those who are not. If both treatment assignment and prognostic characteristics affect adherence, excluding those who are not adherent from the analysis induces selection bias. Comparison of observable characteristics between study participants who are adherent to the assigned intervention and those who are not adherent can help provide clues about the potential for bias. Controlling for these in as-treated and per-protocol analyses may remove the bias.

However, such analyses (ie, as treated and per protocol) no longer rely solely on randomization for inference and are effectively observational analyses. Accordingly, there is always concern that statistical adjustments are imperfect because they can only control for bias from measurable factors. A negative control outcome analysis goes one step further to help identify the presence of residual bias: if an effect is observed between the treatment and negative control outcome that is impossible by the hypothesized mechanism, this suggests that unmeasured or unmeasurable sources of bias are influencing the results.^1,2 A trial to measure the effect of flexible sigmoidoscopy screening on colorectal cancer mortality provides an illustrative example,⁶ whereby per-protocol analyses could have overestimated the benefits of screening on mortality due to “healthy screenee” selection bias if individuals assigned to regular screening were more health conscious than those who were not adherent. A negative control outcome that was affected by health consciousness but not influenced by flexible sigmoidoscopy screening, such as mortality due to noncolorectal cancers, could have been used to detect this bias. Selection bias can also threaten trial validity in other ways such as differential inclusion or exclusion protocols or differential loss to follow-up (attrition). Negative control outcomes could provide similarly useful diagnostics for the presence of selection bias from these other mechanisms.²

Measurement bias from differential outcome misclassification by treatment status is another concern in randomized trials. A large systematic review of clinical trials found evidence of systematically larger effects among unblinded trials with subjective outcomes (either patient reported or investigator assessed), likely owing to differential measurement bias from knowledge of treatment status.⁷ Negative control outcomes can be useful here as well. For example, trials of in-home water treatment ordinarily measure child diarrhea outcomes based on caregiver-reported symptoms because of the cost and logistical difficulties of collecting stool specimens and testing them for enteric pathogens. Because such trials are rarely blinded, there is concern that caregivers who receive in-home water treatment could underreport diarrhea, leading to a biased effect away from the null. To test for this potential source of bias, a trial of in-home chlorination and safe storage also asked care-givers about skin rash and ear infection symptoms along with diarrhea, with the hypothesis that these symptoms could be subject to the same source of potential reporting bias but could not be improved by drinking water treatment.⁸ A large reduction in diarrhea but not skin rash or ear infections added credibility to the trial’s primary results using reported diarrhea.

Selecting a good control outcome at the design stage of a trial that captures the bias structure of concern but is unequivocally unrelated to the treatment requires subject-matter expertise. Nevertheless, a deep understanding of the science underlies most substantive elements of epidemiological study design and analysis, so negative control outcomes are a natural addition to the approaches trialists could use. A second caveat is that negative controls typically identify the presence of bias but not necessarily its magnitude without further assumptions.¹ This is an active area of research, and it is likely that methodologic advances will enable investigators not only to detect but also to minimize bias using negative control outcomes in a similar way that a placebo group removes the placebo effect. In addition, prespecification of negative control outcomes could prevent the selective presentation of favorable results.⁴

Negative controls are a simple and powerful tool with potential for broad application. Trials have used negative control exposures (placebos) for decades to reduce bias when estimating the effects of treatment. Trialists should similarly add negative control outcomes to their approaches for study design. In particular, the use of prespecified negative control outcomes could potentially improve the quality of evidence from trials that report additional analyses beyond intention-to-treat effects and those that have weaknesses (inescapable in many settings) such as lack of blinding, subjective outcomes, or differential attrition.