The Critical Role of Compliance in Delivering Health Gains from Environmental Health Interventions

Abstract.

Gains in reducing childhood disease burden rely heavily on effective means of preventing environmental exposures. For many environmental health interventions, such as point-of-use water treatment, sanitation, or cookstoves, exposures are strongly influenced by user behaviors and the degree to which participants adhere to the prescribed preventive measures. In this commentary, we articulate the need for increased attention on user behaviors—critically, the careful measurement and inclusion of compliance—to strengthen exposure assessment and health impact trials in environmental health intervention research. We focus here on water, sanitation, and hygiene interventions to illustrate the problem with the understanding that this issue extends to other environmental health interventions.

INTRODUCTION

A landmark 1847 study demonstrated a reduction in patient mortality when washing hands before patient contact.¹ Despite the substantial reduction in patient deaths attributable to the intervention, physicians were reluctant to adopt the procedure for a range of reasons. Poor compliance resulted in patient death increases that reached pre-intervention levels, demonstrating how an efficacious intervention can have no effect.

Like handwashing, environmental health interventions are especially susceptible to subpar compliance, although compliance is rarely measured well in practice. Recent null results from cookstove^2,3 and sanitation⁴ interventions highlight the value of interrogating compliance as a critical determinant of environmental health intervention effectiveness. Even when interventions initially demonstrate health gains, these gains can diminish over time because of decreases in compliance, as has been observed with chlorine drinking-water treatment.⁵ Access to the intervention is often used as a measure of successful deployment for both point-of-use technologies and services more broadly. However, access alone is not sufficient to guarantee health improvements.

The concept that interventions not adhered to are not effective seems self-evident; drug therapy is not effective if patients do not take the drug. Much of the research in developing interventions, however, focus solely on the efficacy of a technology or practice, separate from the behaviors critical to the intervention’s success. One reason for this focus on efficacy is that intervention studies are led by scientists and engineers whose viewpoint may be technocentric and focused on experimentation under controlled conditions. For example, physicians lead studies on drug therapy, entomologists lead studies on vector control, and environmental microbiologists lead studies on water quality. As a result, less attention is paid to how effective an intervention is under real-world conditions as opposed to idealized settings. We argue for the need to develop a more holistic framework that incorporates the dynamics of behavior when assessing the effectiveness of environmental health interventions, with a particular focus on water, sanitation, and hygiene (WASH).

INDIRECT EXPOSURE MEASUREMENT IN WASH STUDIES

Observational studies on the risks associated with environmental contamination often rely on indirect exposure measurements. In the case of drinking water, exposure is based on access to a particular water source, for example, surface or piped water, and/or a particular treatment technology, for example, a water filter or chlorine. These proxy exposure variables, standardized by the Joint Monitoring Program, define improved and unimproved water and sanitation based on access and not directly on exposure.⁶ Many have demonstrated limitations of these definitions; for example, piped water can be more contaminated than surface water, and a pit latrine maintained and used can result in less exposure than an unmaintained “higher quality” latrine. Observational studies also rely on access to water treatment technologies as an exposure variable and pay little attention to the use of those technologies, that is, who and when individuals comply with the use of a particular intervention.

Importantly, compliance can vary by group. Some participants may report high compliance for normative practices, especially those that convey perceived social status (e.g., cleanliness). Others, may not comply with a given technology because of cultural, societal, and/or economic factors. Compliance can also vary by time; for example, intervention compliance may be more likely when risk is high because of heightened awareness (e.g., during outbreaks) and drop off when the risks are lower. Hand hygiene and influenza is a good example of this phenomenon. Therefore, context is an important factor that should be integrated into any characterization of compliance.^7,8

ADDRESSING COMPLIANCE IN INTERVENTION TRIALS

Low and variable compliance may fundamentally impair health impacts by allowing persistent exposures despite access to interventions. An example of how low compliance can adversely impact an intervention is the series of sanitation interventions unrolled in Odisha, India, between 2010 and 2013. Latrines were constructed and promoted as part of a government program whose effects were measured in a cluster-randomized trial.^4,9,10 Although latrine coverage increased substantially in the study area, the prevalence of diarrhea—or other outcome measures—did not vary significantly between the intervention and control groups. Notably, in spite of the intervention’s availability, latrine use remained low,¹¹ whereas open defecation was widely observed among the intervention group.¹² Like other environmental health interventions, this was really a coupled technology–behavioral intervention that required both access to and consistent use of technologies to sustain reductions in exposures. The finding of “no effect” on study outcomes—widely but incorrectly interpreted to mean that better sanitation yields no measurable health benefits to communities—underscored both access and compliance as keys to delivery of an effective intervention. Technology alone was only part of the puzzle; exposures to enteric pathogens likely remained unchanged across study arms. Fortunately, compliance was measured in this intervention trial, potentially explaining the lack of an apparent effect. In general, measurement of compliance is lacking. Given the health impact of compliance, this lack of measurement may impair the external validity of intervention trials.

Randomized controlled trials commonly use intention-to-treat analysis, where exposure status is defined as access to the intervention, possession of the intervention, or exposure to messaging or behavior change programming. Per protocol analysis, on the other hand, defines exposure based on whether or not someone actually uses the intervention.^2,13,14 The latter, which incorporates compliance to the intervention, is uncommon, partly because compliance for environmental interventions can be difficult to measure and partly because per protocol analysis results in the loss of randomization. Similarly, treatment-on-the-treated analysis uses an instrumental variable approach to estimate the effect of interventions among those who actually received/complied with the intervention.

Measurement of compliance is a particular problem for environmental trials that are generally conducted outside clinical settings, where use of interventions is controlled by individuals and may be highly variable over time. In these cases, compliance is associated with specific behaviors, including daily use of an intervention, or changes to household practices that are required to realize health relevant exposure reductions. Adherence to these practices can be difficult for researchers to measure directly and, as a result, is often not measured or is measured poorly.

Bias plays a major role in the problems associated with measuring compliance. For unblinded interventions whose effects are measured via subjective outcome measures—common in environmental health—the effect of low and variable compliance may not be detected in outcome data alone, because of well-documented recall bias.¹⁵ Self-reported compliance behaviors may also be subject to response bias, with respondents potentially overreporting safe behaviors or intervention compliance.^16–18 Over-reporting may be increased when respondents had been primed (during the intervention) with information about which behaviors were targeted as part of interventions. Promising approaches have been developed for measuring behavior and compliance based on social¹⁹ and economic^7,20 theory, as well as through microbial²¹ and sensor measurement.²²

The influence of compliance may explain heterogeneity of effect estimates in a range of environmental health intervention trials,²³ although a lack of standardized methods for measuring and reporting compliance limits our ability to systematically assess this impact. In most cases, direct measures of exposure are critical to evaluating interventions. For water and sanitation studies, this should include, at minimum, a range of direct or indirect pathogen measures in environmental media that represent key exposure pathways.^24,25 Future interventions may also benefit from the explicit inclusion of supplementary behavioral interventions designed to improve compliance.²⁶

COMPLIANCE AND RISK ASSESSMENTS

Risk estimates from microbial risk assessment models are typically a function of microbiological characteristics and the efficacy of treatment technologies. The implicit assumption of these models is that compliance to treatment is perfect, which means that for low doses there is a linear relationship between exposure and infection. In the case of a water treatment intervention, increasing efficacy from 1 to 2 log removal of bacteria will decrease the risk by 1 log, the same risk reduction as increasing from 4 to 5 log removal. As a result, optimal treatment technologies appear to be ones which have the highest possible efficacy. However, a small deviation from perfect compliance can result in diminishing returns as treatment efficacy increases.²⁷ As we increase treatment efficacy, the incremental health benefit of a unit increase in treatment efficacy is less and less when compliance is not perfect. This is because when we are attempting to decrease already low risks further, even a small amount of exposure to the untreated environment reverses any gain from the increase in efficacy. Realistically, compliance is never perfect, which suggests that the benefit of high-efficacy water treatment may be overstated. The specific effect of compliance is likely to vary by the mechanism of pathogenesis. Acute outcomes due to pathogen exposure may result in steeper diminishing returns than chronic outcomes due to repeated exposures.

Including compliance in risk modeling shifts risk estimation from an idealized efficacy assessment to a real-world effectiveness assessment.²⁷ It provides a means to explicitly address potential trade-offs that result from designing high-efficacy interventions. That is, a highly efficacious water treatment device removes or inactivates more than 99.99% of pathogens when used correctly, but may also be more difficult to use than a less efficacious device. Using such an integrated assessment allows for a more holistic risk comparison of a lower efficacy treatment device that has high uptake and compliance and a higher efficacy device that has lower uptake and compliance.^28,29 This comparison would not be possible in an efficacy-only risk assessment. Given that the results of quantitative microbial risk assessment models can drive the technical specifications of future WASH interventions, it is critical that these models conform to the reality of imperfect compliance or uptake.

CONCLUSION

We argue that most environmental health intervention trials are “technology led” and underemphasize the importance of human behaviors—including compliance—as integral to reducing exposures and, therefore, realizing health effects of interventions. Compliance can rarely be assumed or taken for granted, and doing so distorts our view of the evidence by ignoring this critical link between technology access and exposure. We advocate for future studies that measure both key We advocate that future studies measure both key compliance behaviors and incorporate direct measures of exposure, providing for a fuller assessment of the causal pathway beyond simple exposure to the technology and delivery of a distal health outcome. Although studies of this type are rare in environmental health,^16,30,31 other fields such as developmental economics have integrated compliance into studies on program cost-effectiveness.^32,33 Therefore, new studies may benefit from interdisciplinary teams that incorporate social scientists, or other experts in compliance-monitoring techniques. Doing so will help improve the epidemiological evidence base for environmental health interventions.