Spillover effects on health outcomes in low- and middle-income countries: a systematic review

Abstract

Background

Many interventions delivered to improve health may benefit not only direct recipients but also people in close physical or social proximity. Our objective was to review all published literature about the spillover effects of interventions on health outcomes in low-middle income countries and to identify methods used in estimating these effects.

Methods

We searched 19 electronic databases for articles published before 2014 and hand-searched titles from 2010 to 2013 in five relevant journals. We adapted the Cochrane Collaboration’s quality grading tool for spillover estimation and rated the quality of evidence.

Results

A total of 54 studies met inclusion criteria. We found a wide range of terminology used to describe spillovers, a lack of standardization among spillover methods and poor reporting of spillovers in many studies. We identified three primary mechanisms of spillovers: reduced disease transmission, social proximity and substitution of resources within households. We found the strongest evidence for spillovers through reduced disease transmission, particularly vaccines and mass drug administration. In general, the proportion of a population receiving an intervention was associated with improved health. Most studies were of moderate or low quality. We found evidence of publication bias for certain spillover estimates but not for total or direct effects. To facilitate improved reporting and standardization in future studies, we developed a reporting checklist adapted from the CONSORT framework specific to reporting spillover effects.

Conclusions

We found the strongest evidence for spillovers from vaccines and mass drug administration to control infectious disease. There was little high quality evidence of spillovers for other interventions.

Key Messages

• Spillovers are the effects of an intervention on individuals who did not receive the intervention but who are connected to recipients through physical or social proximity.

• Our systematic review found a wide range of terminology used to describe spillover effects, a lack of standardization among spillover measurement methods and poor reporting of spillover effects in many studies.

• The strongest evidence for spillover effects exists for studies of vaccines and mass drug administration to control infectious disease. The evidence of spillover effects for other interventions is of limited or poor quality.

• To facilitate improved reporting and standardization in future spillover studies, we developed a reporting checklist adapted from the CONSORT framework, specific to reporting of spillover effects.

Introduction

Interventions delivered to improve health are frequently targeted to specific populations. Such interventions may benefit not only direct recipients but also those who did not receive the intervention but are connected to recipients through physical or social proximity. Such effects, which we refer to as ‘spillovers’, are a component of the population-level impact of interventions. A wide range of terms has been used to describe spillovers in disciplines including economics, public health and political science: externalities,^1,2 interference,^3–7 contamination,⁸ herd immunity,^9–11 stable unit treatment value assignment (SUTVA) violations,¹² stability violations¹³ and indirect effects.^14,15

A ‘positive’ spillover is an effect in the same direction as the treatment effect (on intervention recipients); conversely, a ‘negative’ spillover is an effect in the opposite direction of the treatment effect. If positive spillovers are present, studies that only estimate treatment effects without measuring spillover effects will underestimate the effectiveness of the intervention. In addition, cost-effectiveness calculations that exclude such spillovers may underestimate intervention benefits. Conversely, if negative spillovers are present, evaluations that do not measure spillover effects may overestimate health impacts and cost-effectiveness. Furthermore, negative spillovers could attenuate the effects of an otherwise beneficial intervention. For these reasons, when an intervention is capable of diffusing through a population, information about spillover effects is an important complement to estimates of treatment effects—when spillovers are found to be large and positive, such evidence may, for example, justify national scale-up of an intervention or a public subsidy.¹⁶ The well-documented evidence of spillovers (i.e. ‘herd effects’) of many vaccines justifies the cost-effective scale-up of immunization efforts to a global level via programmes such as the Global Alliance for Vaccines and Immunization.^17,18

In the epidemiological literature focusing on trials of interventions other than vaccines, spillovers have motivated randomizing clusters rather than individuals in order to minimize the chance of spillovers into control units (i.e. ‘contamination’).^8,19,20 Outside vaccine studies, epidemiologists consider spillovers in designing studies (e.g. the expectation of spillovers may motivate cluster-randomization instead of individual-randomization^8,19,20), but they typically do not estimate spillovers explicitly alongside direct effects. Recently, spillovers have increasingly been framed as a quantity of interest themselves, particularly in economics where a growing literature describes spillovers of interventions including school-based deworming² and insecticide-treated bed nets.²¹ On the whole, methods for estimating spillovers have developed independently within disciplines.

We conducted this systematic review in order to summarize the literature about spillover effects on health in low- and middle-income countries. We restrict our review to such countries because this review was supported by the International Initiative for Impact Evaluation (3ie), which focuses on low- and middle-income countries.²² Our objective is to provide a broad summary of the types of spillover effects that have been measured to date.

Methods

Protocol and registration

We attempted to register our protocol with the Campbell Coordination International Development Coordinating Group (IDCG). However, because our protocol included a synthesis of methods in addition to a systematic review, the IDCG did not accept our protocol. Instead, the International Initiative for Impact Evaluation (3ie), which funded this endeavour, supported the development of the protocol and provided both internal and external review.

Eligibility criteria

A complete description of eligibility criteria is available in Supplement 1, as Supplementary data at IJE online. Briefly, we included studies that: (i) were conducted in low- or middle-income countries as defined by the World Bank²³ (as required by our funder); (ii) were quantitative studies evaluating an intervention; (iii) measured health outcomes; and (iv) included a comparison group with sufficient detail about the design and comparison group to determine whether there were serious threats to internal or external validity.

Information sources

We searched 19 electronic databases that contained articles on health, economics, social science, and other disciplines for articles published before 2014 (Supplement 2, available as Supplementary data at IJE online). In addition, we hand-searched all titles from 2010 through 2013 in the following journals, which we considered most likely to include relevant articles: Health Economics, The Journal of Development Effectiveness, The Lancet, PLoS Medicine and the World Bank Policy Research Working Papers.

Search

A detailed description of our search strategy is listed in Supplement 3, available as Supplementary data at IJE online. We searched reference lists of texts classified as eligible in the original search. We also identified records that cited included texts from the original search using Google Scholar. Following the search process, all records were merged, duplicates were removed and a unique ID was assigned to each record.

Study selection

At least one team member reviewed each record for relevance. Titles that were clearly not eligible for the review received no further review. We reviewed each available abstract that passed the title review for relevance. If an abstract was not available but a full text was, we reviewed the full text instead. Of the abstracts deemed relevant, we reviewed each full text for relevance. For records that were deemed not to be relevant, team members recorded the first reason for exclusion identified. If multiple versions of a paper were available, we included the most recent version of the paper.

Data collection process

We extracted data from included texts, and then a second team member independently checked all extracted data. In one case, spillover results were mentioned and disaggregated results were not listed in the publication, but the authors mentioned that results were available upon request.²⁴ We contacted the authors to request these results but did not receive a reply.

Data items

We extracted information about: interventions; outcomes measured; study site; primary study design; study design used to estimate spillovers; purported spillover mechanism; scale of spillover (e.g. household versus village); average cluster-level treatment coverage; whether or not spillover measurement was pre-specified; and direct effect, total effect, overall effect and spillover effects reported numerically in tables or text. If multiple effects or model specifications were used to estimate the direct, total or overall effects, we chose the estimate that appeared to be the primary finding reported by the author and that allowed the greatest comparability of the effect with the spillover estimates. We considered spillovers to be pre-specified if spillover estimation methods were included in the original study protocol.

Risk of bias in individual studies

We classified specific criteria related to risk of bias for each study using criteria compiled from relevant fields.^25–29 Duplicate assessment of risk of bias was performed for a 20% subsample. Classification was not blinded. Co-authors of this systematic review who had authored included studies did not participate in the classification of risk of bias criteria for any studies. For studies that performed secondary analyses, we attempted to obtain the original publication and incorporated information from the original publication(s) into our risk of bias assessment. We only assessed the risk of bias for the elements of the study that estimated effects on health outcomes. We also created an overall classification of risk of bias for individual studies by adapting the Cochrane GRADE approach^30,31 to spillover estimation (Supplement 4, available as Supplementary data at IJE online). We developed these criteria through an iterative process in which we revised our classification system after the initial risk of bias assessment for each study and discussion with multiple reviewers. We then classified each study’s overall quality of evidence as ‘very low’, ‘low’, ‘medium’ or ‘high’.

Summary measures

Due to the wide range of interventions and outcomes evaluated in included studies, we did not consider it reasonable to assume that the studies included were independent and that a common treatment effect existed across all included studies.³² Thus, we did not calculate summary measures.

Synthesis of results

For spillover types for which a sufficient number of studies reported estimates, we standardized results for binary outcomes on the relative scale. We present results as the percentage reduction in outcomes attributable to intervention {[1- relative risk (RR)] x 100%}. For results reported only as risk differences, we calculated the percentage reduction by dividing the risk difference by the probability of the outcome in the control group. We chose the relative scale instead of the additive scale because the interpretation of risk differences depends upon the risk of the outcome among the untreated; given the wide range of interventions and outcomes included in this study, we consider the relative scale to be more appropriate because it facilitates direct comparison using a single measure. We did not synthesize results for continuous outcomes because very few studies measured the same continuous outcome. To generate forest plots, we converted estimates on the additive scale to the relative scale by dividing by the mean of the outcome among individuals not receiving treatment. In plots comparing estimates across studies, we presented 95% confidence intervals for the studies for which standard errors were reported or could be estimated on the relative scale. When possible, we used adjusted effect measures in these plots because many of the included studies used observational designs or used randomized designs that conditioned on a non-randomized variable (e.g. eligibility status) to measure spillovers. Thus, we consider adjusted estimates more appropriate because they are less likely to be biased than crude estimates. We excluded studies of low or very low quality from plots comparing results across studies.

Risk of bias across studies

To assess publication bias, we produced funnel plots. We produced separate plots for studies estimating risk ratios (or 1-RR) and risk differences because insufficient information was reported to standardize measures on a single scale. Funnel plots only included studies that estimated effects for binary outcomes. We did not produce funnel plots for estimates using continuous outcomes because the number of different outcomes measured would not have allowed for comparison across a useful number of studies.

Additional analyses

We searched each included text for terms commonly used to describe spillovers and noted whether the terms appeared in each text.

Results

Study selection

We retrieved 49 749 records through our search process (Supplement 7 Figure 1, available as Supplementary data at IJE online). Following removal of duplicate records and records from non-bibliographical sources, we reviewed 31 622 titles for relevance. We reviewed relevant abstracts and full texts and classified 28 studies from the original search as eligible. We performed title, abstract and full-text review on the reference lists of the 28 eligible texts (n = 798 records) and identified one additional eligible text. We also reviewed records that cited the 28 original included texts (n = 1622 records) and identified an additional 25 eligible texts. A total of 54 records were included in this systematic review. Reasons for exclusion of full texts are listed in Supplement 5, available as Supplementary data at IJE online. We extracted data from 51 studies. We could not extract data for two studies that only reported spillover effects graphically^33,34 or for one study that did not provide numerical results for spillover estimates.²⁴

Study characteristics

Studies were conducted in 17 low- or middle-income countries. The most common study design was cluster-randomized trials (n = 13 studies; 24%) followed by re-analyses of cluster-randomized trials (n = 9; 17%) and re-analyses of individually randomized trials (n = 7; 13%). The most common interventions were vaccines (n = 22; 41%), mass drug administration for infectious disease control (n = 7; 13%) and health education (n = 5; 9%). Several programmes were commonly evaluated for spillovers: the maternal and child health programme in Matlab, Bangladesh;^35,36 the PROGRESA programme, which offered conditional cash transfers in Mexico;^37,38 cholera vaccines provided in Matlab Bangladesh;^39–44 and the Primary School Deworming Program in Busia, Kenya.^2,45–47 Studies estimated a variety of different statistical parameters to quantify spillovers; we define these parameters in Supplement 8, available as Supplementary data at IJE online.

Risk of bias within studies

Six studies (11%) had high quality evidence, 30 (56%) had moderate quality, 12 (22%) had low quality and six (11%) had very low quality evidence (Supplement 6, available as Supplementary data at IJE online). Of studies with high quality evidence, five used cluster-randomized^43,48–51 designs and one used a household secondary attack rate study design.⁵² The proportion of studies with low or very low quality evidence was similar in studies that incorporated spillover measurement into the original design (35%) compared with those which did not pre-specify spillover estimation (36%). All high quality studies were peer- reviewed.

Spillover mechanisms

We identified three primary types of spillover mechanisms in included studies.

1. Reduced disease transmission (n = 28 studies): interventions may decrease the infectiousness of an intervention recipient, and in turn, the risk that non-recipients become ill may decrease.

2. Social proximity (n = 20): interventions may create spillovers when individuals change their behaviour as a result of intervention and in turn influence the behaviour of non-recipients with whom they are in social proximity. Family members, neighbours, classmates or even residents of the same village or city could be considered socially proximate with varying degrees of closeness.

3. Substitution (n = 3): when one household member receives additional resources as a result of intervention, spillovers may occur to other household members because additional resources are available to the household. For example, if one child receives free meals at school, more food may be available for siblings to eat at home.

Results by spillover mechanism

In this section, we summarize studies by spillover mechanisms because they influence spillover magnitude, scale of spillovers, and appropriate study designs for detecting spillovers. Within each mechanism we summarize studies by intervention type. Within each of these categories, we describe results by intervention types. We excluded very low quality studies (n = 6) from this summary.

Spillovers through reduced disease transmission

Studies of spillovers through reduced disease transmission included studies of vaccines (n = 21), mass drug administration to control infectious disease (n = 6), improved water and sanitation (n = 3) and insecticide treated nets (n = 1) (Tables 1–2; Supplement 7 Table 1, available as Supplementary data at IJE online). These studies evaluated spillovers through two approaches: analyses with group-level data (i.e. ecological analyses) and with individual-level data.

Table 2.

Spillover estimates from studies that estimated spillovers through reduced transmission using individual-level data and that measured spillovers as a function of distance to treated individuals

Reference / country / quality of evidencea	Parameter type (in bold) and parameter description	Intervention	Distance / area over which spillovers were measured	Outcome / subgroup / time point	Estimate (95% CI)
Perez-Heydrich et al., 2014 Country: Bangladesh Quality: high	Spillover effect conditional on treatment density Investigators estimated the reduction in cholera risk cases per 1000 persons associated with varying levels of vaccination coverage using a counterfactual model. We report the maximum difference, which is the only quantitative estimate reported in the paper. This is the difference in risk between unvaccinated individuals in neighbourhoods with 60% vaccination coverage compared with those in neighbourhoods with 32% coverage.	Cholera vaccine	Neighbourhood of 64 individuals	Cholera risk per 1000 persons	Risk difference: 5.29 (2.61, 7.96)
Hawley et al., 2003 Country: Kenya Quality: high	Spillover effect conditional on distance among individuals The authors conducted a test of trend by comparing individuals who did not receive ITNs who lived ≥ 900 m from compounds that did receive ITNs (the reference) with individuals who lived at decreasing distances from compounds that received ITNs.	Insecticide-treated nets	0–900 m	Clinical malaria	Odds ratio: 0.92 (0.75, 1.12)
				High-density parasitaemia	Odds ratio: 0.89 (0.78, 1.01)
				Moderate anaemia	Odds ratio: 0.78 (0.69, 0.89)
				Haemogloblin level	Mean difference for one unit increase: 0.18 (0.06, 0.31)
				Child mortality	Hazard ratio: 0.94 (0.90, 0.98)
Ziegelhöfer et al., 2012 Country: Guinea Quality: moderate	Spillover effect conditional on treatment density Investigators estimated the association between diarrhoea and the fraction of people within 3 km who received water supply points.	Improved water supply	3 km	Diarrhoea	Mean difference: −0.59 (−1.32, 0.14) c
Miguel and Kremer, 2004 Country: Kenya Quality: moderate	Spillover effect conditional on treatment density Investigators estimated the association between the proportion of schoolchildren within 0–3 km and 3–6 km receiving deworming medication at schools and worm infections among untreated children. The effects reported here are for a one-unit increase in the number of children treated, controlling for the number of schoolchildren within a specific distance.d	School-based deworming	0–3 km	Moderate-heavy helminth infection (association with treatment density within 0–3 km)	Mean difference: −0.21 (−0.41, −0.01)
			3–6 km	Moderate-heavy helminth infection (association with treatment density within 3–6 km)	Mean difference: −0.05 (−0.21, 0.11)
Baird et al., 2013 Country: Kenya Quality: low	Spillover effect conditional on treatment density Investigators estimated the association between the density of deworming treatment within 6 km of each school during childhood and self-reported outcomes in adulthood, 10 years after a school-based deworming programme.	School-based deworming	6 km	Self-reported health is ‘very good’	Probability difference: 0.128 (−0.097, 0.353)
				Height	Mean difference: −1.891 (−5.158, 1.376)
				Body mass index	Mean difference: 0.317 (−0.210, 0.844)
				Number of pregnancies	Mean difference: −0.335 (−0.958, 0.288)
				Any miscarriages	Probability difference: −0.078 (−0.151, −0.005)
Shekhawat et al., 2014 Country: Tanzania Quality: low	Cluster-level spillover effecte Investigators compared trachoma prevalence in 4 birth cohorts who received multiple rounds of mass azithromycin distribution. To measure spillovers, they assessed whether the prevalence of trachoma was lower among the youngest children in later birth cohorts before their first round of mass azithromycin distribution.	Mass azithromycin distribution	Not applicable	Trachoma	No quantitative estimates were reported, but they concluded that spillovers were present
Paul et al., 1962 Country: Costa Rica Quality: low	Vaccine efficacy Investigators compared the rate of poliovirus vaccine strain excretion over time in the unvaccinated family contacts of children participating in a vaccine trial.	Polio vaccine	Not applicable	Poliovirus	No quantitative estimates were reported, but they found that the rate of excretion among household contacts increased following vaccination

^aThe quality of evidence reported here applies to each study as a whole even if multiple types of spillovers were estimated.

^bWe estimated approximate cluster-treatment coverage using available information in each paper.

^cConfidence intervals present a best-case scenario as they are not necessarily adjusted for clustering.

^dWe report findings from a replication study of the original study, which revised estimates after correcting for coding errors in the original study. Aiken AM, Davey C, Hargreaves JR, Hayes RJ. Re-analysis of health and educational impacts of a school-based deworming programme in western Kenya: a pure replication. Int J Epidemiol 2015;44:1572–80.

^eThis parameter was not explicitly estimated, but it could have been using the data collected in the study.

Table 1.

Spillover estimates from studies that estimated spillovers through reduced transmission using individual-level data measured within clusters

Reference / country / quality of evidencea	Parameter type (in bold) and parameter description	Intervention	Cluster size and treatment coverageb	Outcome / subgroup / time point	Estimate (95% CI)
Ali et al., 2013 Country: India Quality: high	Cluster-level spillover effect Investigators compared cholera incidence per 100 000 person-days among unvaccinated individuals in clusters where the cholera vaccine was delivered to the rate in control clusters.	Cholera vaccine	Size: small group of households Coverage: 36%	Cholera	(1-RR) x 100%: 0% (−59%, 37%)
Baptista et al., 2006 Country: Brazil Quality: high	Vaccine efficacy for infectiousnessc Investigators compared rates of unvaccinated individuals in households with and without vaccinated, infected individuals.	Pertussis vaccine	Size: household Coverage: 31% received the vaccine in the past 10 years	Pertussis	(1-RR) x 100%: 61.6% (12.8%, 83.1%)
	Vaccine efficacy against illness Investigators compared rates of unvaccinated individuals with rates among vaccinated individuals in households with an infected individual.			Pertussis	(1-RR) x 100%: 12.5% (−5.3%, 27.3%)
Préziosi and Halloran, 2003 Country: Senegal Quality: high	Vaccine efficacy for infectiousness Investigators compared rates of unvaccinated individuals in households with and without vaccinated, infected individuals.	Pertussis vaccine	Size: household Coverage: 72% of infected index cases	Pertussis	(1-RR) x 100%: 85% (46%, 95%)
Ozier, 2014 Country: Kenya Quality: moderate	Cluster-level spillover effect Investigators compared outcomes of younger siblings of children who received school-based deworming with younger siblings of children who were not assigned to receive deworming. The authors presented other measures of cognitive function as well; we present a subset of these outcomes here.	School-based deworming	Size: household Coverage: not applicable	Raven’s matrices score (measure of intelligence)	Mean difference: 0.220 (0.067, 0.373)
				Height	Mean difference: 0.204 (−0.378, 0.786)
				Height-for-age	Mean difference: 0.029 (−0.057, 0.115)
				Stunting (height-for-age z-score < -2)	Risk difference: 0.007 (−0.024, 0.038)
Roca et al., 2011 Country: The Gambia Quality: moderate	Spillover effect conditional on exposure to treatment before and after treatment In a population where the pneumococcal conjugate vaccine was not previously offered during routine immunization, investigators compared vaccine-type pneumococcus rates among unvaccinated individuals in villages randomized to receive partial vaccination coverage at baseline and follow-up. The authors presented results over a number of ages and time points. We only present the results of the final follow-up survey for vaccine-type pneumococcus (22 months post-vaccination). We consider vaccine-type pneumococcus most likely to be influenced by vaccination.	Pneumococcal conjugate vaccine	Size: village (80–660 inhabitants) Coverage: 5–9%	Pneumococcal nasopharyngeal carriage among children 2 to <5 years	Odds ratio: 0.28 (0.11, 0.70)
				Pneumococcal nasopharyngeal carriage among children 5 to < 15 years	Odds ratio: 0.25 (0.14, 0.46)
				Pneumococcal nasopharyngeal carriage among children 15 years or older	Odds ratio: 0.43 (0.17, 1.10)
Roca et al., 2013 Country: The Gambia Quality: moderate	Spillover effect conditional on exposure to treatment before and after treatment In a population where the pneumococcal conjugate vaccine was not previously offered during routine immunization, investigators compared vaccine-type pneumococcus rates among unvaccinated individuals in villages randomized to receive partial vaccination coverage at baseline and at 4 years follow-up. The authors presented results over a number of ages and time points. We only present the results of the final follow-up survey for vaccine-type pneumococcus (22 months post-vaccination). We consider vaccine-type pneumococcus most likely to be influenced by vaccination.	Pneumococcal conjugate vaccine	Size: village (80–660 inhabitants) Coverage: 5–9%	Pneumococcal nasopharyngeal carriage among children 2.5 to < 5 years	Odds ratio: 0.15 (0.07, 0.33)
				Pneumococcal nasopharyngeal carriage among children 5 to < 15 years	Odds ratio: 0.21 (0.10, 0.42)
				Pneumococcal nasopharyngeal carriage among children 15 years or older	Odds ratio: 0.02 (0.003, 0.18)
Khan et al., 2012 Country: Pakistan Quality: moderate	Cluster-level spillover effect Investigators compared typhoid among the unvaccinated in vaccinated vs control clusters.	Typhoid vaccine	Size: group with an average of 433 children Coverage: 38%	Typhoid incidence	(1-RR) x 100%: −10% (−116, 44)
Sur et al., 2009 Country: India Quality: moderate	Cluster-level spillover effect Investigators compared typhoid among the unvaccinated in vaccinated vs control clusters.	Typhoid vaccine	Size: group with an average of 777 individuals Coverage: 60%	Typhoid incidence	(1-RR) x 100%: 44% (2%, 69%)
Miguel and Kremer, 2004 Country: Kenya Quality: moderate	Cluster-level spillover effect Investigators compared infection rates among students who did not receive deworming but attended schools with the deworming programme, with children in schools without the programme.d	School-based deworming	Size: schools with an average of ∼ 400 pupils Coverage: ∼ 70–80%	Moderate-heavy helminth infection	Risk difference: −0.18 (−0.32,−0.04)
Hammitt et al., 2014 Country: Kenya Quality: moderate	Spillover before and after treatment Investigators compared vaccine-type pneumococcus among unvaccinated children (≥ 5 years of age) before and after a campaign was initiated.	Pneumococcal conjugate vaccine	Size: population of 260 000 (no clusters) Coverage: 79%	Pneumococcal nasopharyngeal carriage among children ≥ 5 years	Prevalence ratio: 0.34 (0.18, 0.62)
House et al., 2009 Country: Ethiopia Quality: high	Cluster-level spillover effect among ineligibles We compared the prevalence of trachoma among untreated individuals in clusters randomly allocated to treatment 12 months after mass treatment with that in control clusters.	Mass azithromycin distribution	Size: administrative unit with ∼ 1400 people Coverage: 82%	Trachoma	(1-RR) x 100%: 35% (8%, 55%)e
Chidambaram et al., 2004 Country: Ethiopia Quality: low	Cluster-level spillover effect Investigators compared the odds of trachoma among ineligible individuals in programme areas with the odds in control areas.	Mass azithromycin distribution	Size: village Coverage: 91%	Trachoma	Odds ratio: 2.9 (1.1, 7.5)
Egere et al., 2012 Country: The Gambia Quality: high	Cluster-level spillover effect among ineligibles Investigators compared pneumococcal carriage among infants too young to be vaccinated in fully vs partially vaccinated villages.	Pneumococcal conjugate vaccine	Size: village (80–660 inhabitants- Coverage: 100%	Pneumococcal carriage	Hazard ratio: 0.39 (0.26, 0.58).

^aThe quality of evidence reported here applies to each study as a whole even if multiple types of spillovers were estimated.

^bWe estimated approximate cluster-treatment coverage using available information in each paper.

^cThe manuscript labels this parameter vaccine efficacy against transmission; however, we refer to it as vaccine efficacy for infectiousness based on the definition in Halloran E, Longini IM Jr, Struchiner CJ. Design and Analysis of Vaccine Studies. New York, NY: Springer, 2010.

^dWe used estimates from the replication study published in Aiken AM, Davey C, Hargreaves JR, Hayes RJ. Re-analysis of health and educational impacts of a school-based deworming programme in western Kenya: a pure replication. Int J Epidemiol 2015;44:1572–80.

^eConfidence intervals present a best-case scenario as they are not necessarily adjusted for clustering.

Eleven of the 13 studies that evaluated spillovers using group-level data found that the risk of illness declined as treatment coverage increased, suggesting that spillovers were present. Six studies evaluated spillovers of the cholera vaccine and found that cholera risk decreased among unvaccinated individuals as vaccine coverage increased (Supplement 7 Table 1 and Figure 2 Panel A).^{39–42,44,53–60} No such pattern was evident among vaccinated individuals, suggesting that spillover effects did not yield additional protection beyond that conferred by the vaccine itself (Supplement 7 Figure 2 Panel B).^39,40,53,54 Five of these studies re-analysed data from the same trial, so their findings cannot be considered independent.^39–43 There are two significant limitations to this type of analysis in assessing spillovers. First, in observational studies or randomized trials without perfect compliance, this type of ecological comparison is likely to be confounded by factors associated with both treatment compliance and the outcome. For example, vaccination coverage may have been higher in high-income areas with better access to care, which may have partially explained lower illness levels in these areas. Second, spillover findings are likely to be highly sensitive to the definition of the area in which treatment coverage and outcomes were measured; groups of different sizes or composition may have produced different results.⁶¹ Thus, overall, we consider these findings to be of lower quality than findings from studies analysing individual-level data.

Seventeen studies evaluated spillovers through reduced disease transmission using individual-level data (Tables 1–2). We separated these studies into two categories: those that measured spillovers within clusters (e.g. households, villages), and those that measured spillovers as a function of distance from treated individuals. Among the studies measuring spillovers in clusters, we expected that spillovers would be larger in smaller-sized clusters (e.g. households) because reductions in disease transmission are most likely to impact on individuals in close proximity. In general, we did not find this to be the case. Two out of four studies of spillovers in households found relatively large spillover effects;^52,62 both studies estimated the reduction in risk associated with living in households with individuals diagnosed with pertussis who were vaccinated versus unvaccinated for pertussis (the vaccine efficacy for infectiousness); the study in Senegal estimated an 85% [95% confidence interval (CI) 46%, 95%] risk reduction,⁵² and the study in Brazil estimated a 61.6% (95% CI 12.8%, 83.1%) risk reduction.⁶² Three of the eight studies measuring spillovers in larger clusters (e.g. schools, villages) found evidence of large spillovers.^51,63,64 We expected that spillovers would be larger at higher levels of treatment coverage, and we found this to be true in the relevant studies: of the four studies with cluster-level treatment coverage under 50%, two found no spillover effects.^53,65 Of those with treatment coverage over 50%, all found evidence of spillovers, and in four studies, spillover effects were relatively large.^51,52,63,64 Among studies that measured spillovers as a function of distance from treated individuals, the magnitude of spillovers was smaller than in studies evaluating spillovers within clusters. However, this finding may be explained by intervention type—none of these studies evaluated vaccines. Spillovers decayed with distance from treated individuals in two studies.^2,48

Spillovers through social proximity

Seventeen studies evaluated spillovers through social proximity (Table 3). One found evidence of negative spillovers,⁴⁶ eight found no evidence of spillovers^66–73 and eight found evidence of spillovers for some but not all outcomes or conditions reported.^{21,37,38,50,74–77} These studies measured spillovers through four mechanisms among untreated individuals who were: (i) in areas where cash transfers were offered; (ii) in or near areas where subsidies or microloans were offered to promote certain health products or behaviours (e.g. subsidies for vaccines); (iii) socially connected to treated individuals; or (iv) in the same schools or areas as treated individuals, regardless of social links.

Table 3.

Spillover estimates from studies that estimated spillovers through social proximity

Reference / country / quality of evidencea/ scale at which spillovers were measured	Parameter type (in bold) and parameter description	Intervention	Outcome / subgroup / time point	Estimate (95% CI)
Spillovers of cash transfer interventions
Avitabile, 2012 Country: Mexico Quality: low Scale: village	Cluster-level spillover effect among ineligibles Investigators compared the difference in the difference (DID) in outcomes before and during the programme among ineligible individuals in clusters where cash transfers were offered to those of ineligibles in the control clusters.	Conditional cash transfers	Cervical cancer screening	Mean difference: 0.061 (0.022, 0.100)
			Blood sugar screening	Mean difference: 0.010 (−0.025, 0.045)
			Blood pressure screening	Mean difference: 0.025 (−0.010, 0.060)
Handa et al., 2001 Country: Mexico Quality: moderate Scale: localities	Cluster-level spillover effect among ineligibles Investigators compared the difference in the difference (DID) in outcomes before and during the programme among ineligible individuals in clusters where cash transfers were offered to those of ineligibles in clusters without cash transfers.	Conditional cash transfers	Child nutrition surveillance 6 months after programme initiation	Mean difference: 2.307 (0.817, 3.797)
			Child nutrition surveillance 12 months after programme initiation	Mean difference: 6.846 (2.632, 11.060)
Ribas et al., 2011 Country: Paraguay Quality: moderate Scale: district	Cluster-level spillover effect among ineligibles Investigators compared outcomes among ineligible individuals in clusters where cash transfers were offered to those of ineligibles in clusters without cash transfers.	Conditional cash transfers	Child growth monitoring visits	Mean difference:  −0.014 (−0.169, 0.141)
Baird et al., 2013 Country: Malawi Quality: very low Scale: ∼250 households	Cluster-level spillover effect Investigators compared psychological distress among girls who did not receive cash transfers in areas where cash transfers were offered to those in comparison areas. They also stratified by whether girls who did not receive cash transfers girls lived in a household with another treated girl.	Conditional and unconditional cash transfers	Psychological distress among all untreated girls during the intervention	Mean difference: 0.064 (0.007, 0.121)
			Psychological distress among all untreated girls after the intervention	Mean difference: 0.007 (−0.056, 0.070)
			Psychological distress among untreated girls in households without treated girls during the intervention	Mean difference: 0.099 (0.038, 0.160)
			Psychological distress among untreated girls in households without treated girls after the intervention	Mean difference: 0.001 (−0.058, 0.060)
			Psychological distress among untreated girls in households with treated girls during the intervention	Mean difference:  −0.086 (−0.188, 0.016)
			Psychological distress among untreated girls in households with treated girls after the intervention	Mean difference:  0.015 (−0.142, 0.172)
Contreras and Maitra, 2013b Country: Columbia Quality: very low Scale: household	Cluster-level spillover effect among ineligibles Investigators compared the difference in the difference (DID) in outcomes before and during the programme among ineligible individuals in households where cash transfers were offered to those of ineligibles in households without cash transfers. The authors presented results stratified by age and gender, but here we only present pooled results.	Conditional cash transfers	Self-reported to be ill at 1 year	Mean difference: −0.030 (−0.060, 0.000)
			Self-reported to be ill at 4 years	Mean difference: −0.018 (−0.055, 0.019)
			In bed as a result of illness at 1 year	Mean difference: 0.017 (−0.046, 0.080)
			In bed as a result of illness at 4 years	Mean difference: 0.042 (−0.028, 0.112)
			Hospitalized in the previous year at 1 year	Mean difference: −0.010 (−0.020, 0.009)
			Hospitalized in the previous year at 4 years	Mean difference: −0.016 (−0.031, −0.002)
Spillovers of interventions with subsidies or microloans
Bhattacharya et al., 2013 Country: Kenya Quality: low Scale: neighbourhood	Total effect conditional on treatment density Investigators estimated the association between the proportion of nearby households (within 250 m, 500 m and 1000 m) receiving subsidies and the probability of ITN use at different income thresholds of eligibility for the subsidy. The authors presented results over a large number of subsidy thresholds; we excluded some to avoid redundancy.	Subsidized insecticide-treated nets	Probability of ITN use when < 50% eligible for the subsidy	Up to a 5% decrease At most thresholds, the 95% confidence intervals did not span 0%.
			Probability of ITN use when ≥ 50% eligible for the subsidy	Up to 4.8% increase All confidence intervals did not include 0%.
Banerjee et al., 2010 Country: India Quality: high Scale: 6 km between villages	Spillover effect conditional on distance to clusters Investigators compared immunization rates in villages within 6 km of villages randomized to either an immunization campaign or an immunization campaign with incentives, with rates in villages randomized to the control group.	Immunization campaign without incentives	Number of immunizations	Relative risk: 1.18 (0.92, 1.43)
		Immunization campaign with incentives	Number of immunizations	Relative risk: 1.48 (1.18, 1.77)
		Immunization campaign without incentives	Child received ≥ 1 immunization	Relative risk: 1.00 (0.80, 1.19)
		Immunization campaign with incentives	Child received ≥ 1 immunization	Relative risk: 1.05 (0.86, 1.24)
		Immunization campaign without incentives	Child has BCG scar	Relative risk: 1.00 (0.73, 1.28)
		Immunization campaign with incentives	Child has BCG scar	Relative risk: 1.05 (0.78, 1.32)
		Immunization campaign without incentives	Child was completely immunized	Relative risk: 1.83 (0.93, 2.73)
		Immunization campaign with incentives	Child was completely immunized	Relative risk: 3.47 (2.18, 4.77)
Tontarawongsa et al., 2011 Country: India Quality: moderate Scale: village	Spillover effect conditional on number of social network links Investigators measured whether insecticide-treated net (ITN) acquisition and use among untreated individuals in areas where free ITNs were offered was associated with the percentage of respondents’ social ties in a programme offering free and subsidized ITNs. The authors presented results stratified by the type of social links, but we do not present those results here.	Free insecticide-treated nets	Recently acquired at least one ITN	Mean difference: −0.008 (−0.141, 0.125)
			Fraction of household members slept under ITN last night	Mean difference: 0.071 (−0.011, 0.153)
			Recently acquired at least one bed net	Mean difference: 0.183 (−0.091, 0.457)
			Fraction of household members slept under bed net last night	Mean difference: 0.157 (−0.014, 0.328)
	Spillover effect conditional on social network links’ ITN acquisition and use Investigators measured whether insecticide-treated net (ITN) acquisition and use among untreated individuals was associated with the acquisition and use among social ties in a programme offering free and subsidized ITNs.	Average per capita bed nets owned by peers	Recently acquired at least one ITN	Mean difference: −0.050 (−0.320, 0.220)
			Fraction of household members slept under ITN last night	Mean difference: 0.020 (−0.145, 0.185)
			Recently acquired at least one bed net	Mean difference: 0.042 (−0.350, 0.434)
			Fraction of household members slept under bed net last night	Mean difference: 0.056 (−0.216, 0.328)
		Average ITN usage the previous night among peers	Recently acquired at least one ITN	Mean difference: −0.107 (−0.283, 0.069)
			Fraction of household members slept under ITN last night	Mean difference: 0.007 (−0.115, 0.129)
			Recently acquired at least one bed net	Mean difference: −0.036 (−0.305, 0.233)
			Fraction of household members slept under bed net last night	Mean difference: 0.016 (−0.180, 0.212)
Godlonton and Thornton, 2012 Country: Malawi Quality: moderate Scale: 1 km	Overall effect conditional on outcome density Investigators assessed whether a 1% increase in the proportion of neighbours within 0–0.5 km who received HIV test results (regardless of incentives received) was associated with choosing to learn one’s HIV test results.	Incentives for voluntary counselling and testing for HIV	Probability of learning HIV test results	Mean difference: 0.106 (0.014, 0.198)
	Investigators assessed whether a 1% increase in the proportion of neighbours within 0–0.5 km who received incentives for learning their HIV test results was associated with choosing to learn one’s HIV test results. The authors stratified results by gender, the proportion of neighbours over increasing distances, distance to HIV testing centres and other variables. See the paper for the full set of results.		Probability of learning HIV test results	Mean difference: −0.064 (−0.227, 0.099)
Spillovers through social connections to treated individuals
Chong et al., 2013 Country: Colombia Quality: moderate Scale: classroom / school	Cluster-level spillover effect Investigators compared outcomes among students in classrooms that did not receive the education programme but were in schools where other classrooms received it, with outcomes in schools where the programme was not offered. The authors presented other outcomes as well. See the paper for the full set of results.	Online sexual health education	Knowledge index at 1 week	Mean difference: 0.015 (−0.073, 0.103)
			Knowledge index at 6 months	Mean difference: 0.013 (−0.148, 0.174)
			Attitude index at 1 week	Mean difference: 0.026 (−0.066, 0.118)
			Attitude index at 6 months	Mean difference: 0.023 (−0.077, 0.123)
			Condom voucher redemption at 6 months	Mean difference: 0.040 (−0.031, 0.111)
	Spillover effect conditional on treatment density Investigators estimated the difference in outcomes among students who did not participate in the programme if all vs none of their friends participated in the programme. The authors presented other outcomes as well. See the paper for the full set of results.		Knowledge index at 6 months	Mean difference: −0.074 (−0.480, 0.332)
			Attitude index at 6 months	Mean difference: −0.046 (−0.326, 0.234)
			Condom voucher redemption at 6 months	Mean difference: −0.156 (−0.274, −0.038)
German et al., 2012 Country: Thailand Quality: moderate Scale: one social network node	Spillover effect among social network members Investigators estimated the mean depression score for peers of individuals randomized to treatment intervention participants with that of peers of individuals randomized to control.	Peer support intervention	Depression score of peers of intervention recipients	Mean difference: −0.095 (−0.18, −0.01)
			Depression score of peers of controls recipients	Mean difference: −0.092 (−0.18, −0.01)
Kremer and Miguel, 2007 Country: Kenya Quality: moderate Scale: 3–6 km between schools	Overall effect conditional on number of social network links Investigators measured whether the number of social links with parents whose children previously received deworming was associated with the probability of taking deworming.	School-based deworming	Deworming consumption	Probability: −0.031 (−0.058, −0.004)
Spillovers through residence in the same area as treated individuals, regardless of social links
Janssens et al., 2006 Country: India Quality: low Scale: village	Cluster-level spillover effect Investigators compared vaccination rates among non-participants in villages with the programme with rates in the control group.	Women’s empowerment programme	Tuberculosis vaccine	Mean difference: 0.149 (0.053, 0.245)
			DTP vaccine	Mean difference: 0.122 (0.024, 0.220)
			Measles vaccine	Mean difference: 0.268 (0.170, 0.366)
	Cluster-level spillover effect in which controls are matched to the untreated Investigators compared vaccination rates among non-participants in villages with the programme, with rates among individuals in the control group who were matched to non-participants in the treatment group.		Tuberculosis vaccine	Mean difference: 0.108 (0.059, 0.157)
			DTP vaccine	Mean difference: 0.090 (0.045, 0.135)
			Measles vaccine	Mean difference: 0.089 (0.042, 0.136)
Dupas 2006 Country: Kenya Quality: moderate Scale: school	Spillover effect conditional on treatment density Investigators compared condom use among students in schools with different proportions of students who previously participated in a health education programme. The authors presented other outcomes as well; we excluded some to avoid redundancy.	Proportion of girls who received information about HIV transmission	Probability of using a condom during sex for girls	Mean difference: 0.476 (0.111, 0.841)
			Probability of using a condom during sex for boys	Mean difference: 0.109 (−0.254, 0.472)
		Proportion of boys who received information about HIV transmission	Probability of using a condom during sex for girls	Mean difference: −0.388 (−0.798, −0.022)
			Probability of using a condom during sex for boys	Mean difference: 0.042 (−0.387, 0.471)
Azad et al., 2010 Country: Bangladesh Quality: low Scale: union (population ∼15 000–35 000)	Cluster-level spillover effect conditional on exposure to treatment Investigators measured outcomes among women exposed to women’s groups who did not participate in them and women in the same villages where the groups took place who had not heard of them. We compared these rates with those in the control group (the authors did not explicitly measure spillovers). The authors assessed numerous other outcomes as well, such as the stillbirth rate and number of antenatal visits, and there was no evidence of spillovers for these outcomes.	Women’s groups and health service strengthening	Neonatal mortality among women exposed to women’s groups who did not participate in them (cluster-level spillover effect conditional on exposure to treatment)	Mean difference: −4.8 per 1000 live births c
			Neonatal mortality among women in the same villages where the groups took place, who had not heard of them (cluster-level spillover effect)	Mean difference: −4.1 per 1000 live births c
Singh 2011 Country: India Quality: moderate Scale: city block	Cluster-level spillover effect Investigators compared the difference in weight-for-age z-score among children of mothers in the intervention clusters that did not participate in the intervention between baseline and follow-up, with the difference for among children of mothers in a control group, between baseline and follow-up.	Nutrition education	Weight-for-age z-score	Difference in difference: 0.013 (−0.067, 0.093)
Björkman and Svensson, 2009 Country: Uganda Quality: moderate Scale: ∼30 km between villages	Overall effect conditional on distance Investigators assessed whether being within 10 km of a treatment clinic was associated with increased health care utilization in control areas. The authors also measured spillovers on child death. See the manuscript appendix for results.	Community monitoring and provision of health services	Rate of outpatient visits	Mean difference: 68.1 (−51.1, 187.3)
			Mean deliveries per facility per month	Mean difference: 2.6 (−5.4, 10.6)

^aThe quality of evidence reported here applies to each study as a whole even if multiple types of spillovers were estimated.

^bSubstitution is another possible mechanism of spillover in this paper.

^c95% Confidence intervals could not be calculated due to insufficient information in the paper.

We hypothesized that spillovers would be stronger for interventions involving incentives or cash transfers than for those that did not because intervention uptake might be higher and the intervention might receive more attention from untreated individuals than interventions with no transfers or incentives. We also hypothesized that studies considering spillovers through social proximity might be more likely to detect spillovers if they considered social connections between treated and untreated individuals. However, we found neither of these to be true among the studies in this review. Three of the studies measuring spillovers of cash transfers found no evidence of spillovers,^66–68 and two found evidence for some but not all outcomes measured.^37,38 Even among the outcomes for which there was evidence of spillovers, the effect sizes were small. Two of four studies evaluating spillovers of subsidies or microloans for health products found evidence of spillovers.^21,50,69,77 For example, in a study of incentives for immunization, Banerjee et al. estimated both total and spillover effects. The relative risk for the total effect on complete child immunization was 6.66 (95% CI 4.53, 8.80); for the spillover effect the relative risk was 3.47 (95% CI 2.18, 4.77).⁵⁰ Two of the three studies that measured spillovers through social links to treatment recipients found no evidence of spillovers,^70,71 and one found evidence of negative spillovers.⁴⁶ Of the five studies measuring spillovers among untreated individuals in the same schools, villages or areas as treated individuals, one found evidence of spillovers,⁷⁶ two found evidence for some but not all outcomes^74,75 and two found no evidence of spillovers.^72,73 Because the number of studies measuring spillovers through social proximity is relatively small and the types of interventions and outcomes measured varied widely, it is likely that the patterns we observed in this review do not necessarily generalize to the same interventions implemented in other contexts.

As for studies of spillovers through disease transmission, we also assessed whether spillover presence and effect sizes were associated with the size of the area in which spillovers were measured (e.g. household versus city). We did not find evidence of any patterns associated with area size.

Spillovers through substitution

Four studies measured spillovers through substitution (Table 4).^24,78–80 Three studies measured whether siblings of children participating in school nutrition programmes or whose mothers participated in nutrition education programmes experienced improved growth as a result. None of these studies found evidence of spillovers.

Table 4.

Spillover estimates from studies that estimated spillovers through substitution

Reference / country / quality of evidencea	Parameter type (in bold) and parameter description	Intervention	Outcome	Mean difference (95% CI)
Fitzsimons et al., 2012 Country: Malawi Quality: moderate	Cluster-level spillover effect among ineligibles Investigators measured several outcomes among older children who were not targeted by the programme who lived in the same households as program beneficiaries.	Information on infant nutrition and health	Height-for-age	−2.66 (−0.540, 0.008)
			Weight-for-age	−0.142 (−0.456, 0.172)
			Weight-for-height	−0.038 (−0.332, 0.256)
			Diarrhoea	0.004 (−0.055, 0.063)
			Vomiting	−0.042 (−0.134, 0.050)
			Fast breathing	−0.008 (−0.110, 0.094)
			Fever	−0.018 (−0.130, 0.094)
			Chills	−0.033 (−0.170, 0.104)
Kazianga et al., 2014 Country: Burkina Faso Quality: moderate	Cluster-level spillover effect among ineligibles This study estimated the mean difference in the difference (DID) in weight-for-age and height-for-age z-scores between baseline and follow-up. They estimated spillovers among pre-school-aged children who lived in households where school-aged children received a school feeding programme or a take-home rations programme compared with those where school-aged children received neither.	School feeding programme	Weight-for-age	0.031 (−0.230, 0.292)
			Height-for-age	0.094 (−0.218, 0.406)
		Take-home rations	Weight-for-age	0.445 (0.159, 0.731)
			Height-for-age	0.079 (−0.262, 0.420)
Zivin et al., 2009 Country: Kenya Quality: moderate	Cluster-level spillover effect Investigators compared weight-for-height z-scores of children whose parents were HIV-positive and had received more than 100 days of antiretroviral therapy, with those whose parents had received fewer than 100 days of therapy.	HIV/AIDS treatment	Weight-for-height	0.374 (−1.163, 1.911)
Buttenheim et al., 2011 Country: Laos Quality: low	Cluster-level spillover effect among ineligibles Investigators compared outcomes of younger and older siblings of children participating in a school feeding and take-home rations programme with those in a control group.	School feeding and take home rations programme	Child growth and anaemia	The authors report that they found evidence of spillovers, but they did not present disaggregated spillover results.

^aThe quality of evidence reported here applies to each study as a whole even if multiple types of spillovers were estimated.

Results pooled across intervention

Because studies estimated a wide variety of types of spillovers, results from many studies were not directly comparable. However, a sufficient number of studies estimated within-cluster spillovers to allow for comparison of results across interventions. This type of spillover compares outcomes among untreated individuals in clusters with different proportions of treatment; most commonly, the effect compares untreated individuals in clusters in which some proportion of the cluster receives treatment with those in which no one receives treatment. The three interventions for which investigators reported positive spillover estimates were mass administration of azithromycin to control trachoma (35% decrease in trachoma),⁴⁹ the typhoid vaccine in India (44% decrease in typhoid)⁸¹ and the pneumococcal conjugate vaccine (70% decrease in vaccine-type pneumococcus) (Figure 1).⁵¹ These spillovers were measured in studies with moderate and high quality. The remaining studies could not distinguish spillover estimates from the null. Within-cluster spillovers were stronger in studies in which the proportion of individuals treated within clusters was higher (Figure 2). The largest spillovers were present for a study evaluating spillovers of the pneumococcal vaccine by comparing outcomes in villages where 100% of individuals were vaccinated, with those in which only infants were vaccinated.⁵¹ Both studies with average cluster-level treatment coverage below 40% did not find evidence of spillovers.^53,65

Figure 1.

Cluster-level spillover effects. On the x-axis, the cluster-level spillover effect is shown as the % change in outcome among the untreated in the treated cluster from the mean in the control group [i.e., (1-RR) x 100%, where RR is the relative risk]. Outcomes were recoded so that a greater value of the spillover effect indicates an improvement in health (e.g., higher vaccination coverage, lower mortality) and a smaller value indicates poorer health (e.g., lower vaccination coverage, higher mortality). This figure excludes studies of low or very low quality and studies that did not report information that allowed for standardization. Statistical significance was determined based on the measures presented in the paper for the parameter on its original scale. (a) Information required to convert standard errors for risk differences to standard errors for (1-RR) x 100% was not reported, thus 95% confidence intervals are not presented. (b) These studies were conducted in the same country (India) and are subject to dependence.

Figure 2.

Cluster-level spillover effects by treatment coverage level. This figure plots cluster-level spillover estimates by the level of treatment coverage within treated clusters. We estimated treatment coverage using information available in each paper. On the y-axis, the cluster-level spillover effect is shown as the % change in outcome among the untreated in the treated cluster from the mean in the control group [i.e., (1-RR) x 100%, where RR is the relative risk]. Outcomes were recoded so that a greater value of the spillover effect indicates an improvement in health (e.g., higher vaccination coverage, lower mortality) and a smaller value indicates worse health (e.g., lower vaccination coverage, higher mortality). This figure excludes studies of low or very low quality and studies that did not report information that allowed for standardization. (a) These studies were conducted in the same country (India) and are subject to dependence. (b) Information required to convert standard errors for risk differences to standard errors for (1-RR) x 100% was not reported, thus 95% confidence intervals are not presented.

Risk of bias across studies

The funnel plots for total and direct effects suggest that publication bias was not present (Supplement 7 Figure 3). The funnel plot for spillover effects estimated with risk differences was balanced around 0, indicating minimal publication bias (Figure 3). For spillovers estimated with risk ratios, the plot was asymmetrical, with few studies producing estimates of negative spillover effects at any level of precision, indicating strong publication bias. Risk ratios were more common in public health studies of interventions that were unlikely to result in negative spillovers (e.g. vaccines). Conversely, risk differences were more common in economics studies of interventions for which the expected direction of spillover effects is less clear.

Figure 3.

Funnel plots for spillover effects. Panel A: This plot includes spillover estimates from 19 studies that reported risk differences for binary outcomes, of which all but one were from studies in the economics literature. These studies evaluated a wide range of interventions including women’s empowerment programs, mass drug administration for infectious disease control, peer group interventions, and nutrition programs. Panel B: This plot includes spillover estimates from 14 studies that reported risk ratios or protective efficacy ((1-RR) x 100%) for binary outcomes, all of which were from studies in the public health literature. These studies evaluated vaccines and mass drug administration for infectious disease control.

Additional analysis

We identified 15 terms commonly used to describe the concept of spillovers (Table 5). The most common terms were ‘indirect effect’ and ‘spillover’, followed by ‘externality/externalities’. ‘Indirect protection’ and ‘herd protection’ were other common terms.

Table 5.

Search terms related to spillover effects in included texts by academic field^b

	Economics	Geography	Public health	Total
Indirect effect*a	12	2	13	27
Spillover*a	23	0	1	24
Externalit*a	19	0	0	19
Seconda*a	3	3	10	16
Indirect protection	0	4	11	15
Herd protect*a	0	2	12	14
Diffusion	7	1	3	11
Herd immunity	1	4	5	10
Herd effect*a	0	0	10	10
Peer effect*a	9	0	0	9
Unexpected	2	0	3	5
Interference	2	0	2	4
Indirect protective	0	0	4	4
Contagion	3	0	0	3
Unexpected benefit*a	0	0	1	1

^aAsterisks at the end of search terms indicate wild-card characters allowed at the end of the search term. For example, ‘externalit*’ would retrieve search results for ‘externality’ and ‘externalities’.

^bCounts allow for multiple terms per included text.

Discussion

Summary of evidence

To our knowledge, this is the first systematic review of health-related spillovers of interventions in low- and middle-income countries. Evidence of spillovers was strongest for spillovers through reduced disease transmission, and in particular for vaccines and mass drug administration for infectious disease control. There was also strong evidence of spillovers for insecticide-treated net use on health outcomes, but only one study evaluated this association. In studies of spillovers through social proximity, there was weak evidence of spillovers in most studies with a few exceptions: for example, there was evidence that an immunization campaign with incentives increased immunization coverage among non-participants in nearby villages. There was no evidence of spillovers through substitution effects in the three relevant studies.

There are several reasons why we believe we found the strongest evidence for spillovers through reduced disease transmission. First, spillovers through reduced disease transmission are mostly a function of physical proximity. Infectious disease theory suggests that spillovers occur through reduced disease transmission when susceptible and infected individuals come into contact;⁸² such contact can modify disease transmission across different populations and pathogens. Indeed, we found evidence of spillovers through reduced disease transmission across interventions, outcomes and populations. On the other hand, spillovers through social proximity may be a function of physical proximity as well as social dynamics that are highly dependent on culture and context, and these factors may vary by population, intervention and health outcome. The relative complexity of spillovers through social proximity may make spillovers less likely to occur through this mechanism than through reduced disease transmission. Similarly, for spillovers through substitution, although an intervention may free up a fixed amount of resources in a household, whether those resources support the health of non-intervention recipients may depend on complex factors, such as education level and culture, which vary across populations.

Second, study designs may have been more appropriate for detecting spillovers through reduced disease transmission than through social proximity or substitution. A rich literature has refined study designs to estimate spillovers of vaccines;^{4,5,7,13,15,82–86} these methods can easily be extended to studies of other interventions that produce spillovers through reduced disease transmission. For spillovers through social proximity or substitution, there is no equivalent methodological literature focused on empirical measurement. As a result, in this review, study designs for detecting spillovers through these mechanisms may have been suboptimal or biased. Indeed, the proportion of studies that we classified as moderate, low, or very low quality was greater among studies measuring spillovers through social proximity or substitution than among studies of spillovers through reduced disease transmission.

Finally, it is also possible that rigorous studies to measure spillovers through social proximity or substitution simply have not been conducted yet or were missed in our search. There were five high quality studies of spillovers through reduced disease transmission compared with only one for spillovers through social proximity and none for substitution. Thus, our findings do not necessarily reflect a lack of spillovers of any particular intervention. Rather, with the exception of vaccines and mass drug administration to control infectious disease, our findings show little evidence for health-related spillovers from currently published intervention studies conducted in low- and middle-income countries.

Quality of evidence

Most studies reported moderate or low quality evidence of spillovers, and there were two overarching sources of bias. First, many study designs did not adequately minimize unmeasured confounding of spillover estimates. Only two out of the 23 studies estimating within-cluster spillover effects used double-randomized designs^67,70 which allow for the strongest inference for this type of spillover by minimizing selection bias and unmeasured confounding. In the 21 other such studies, untreated individuals in treated clusters may have been systematically different from individuals in control clusters, possibly because they were not eligible to receive the intervention or chose not to receive the intervention. Such systematic differences between the populations used to measure total effects versus spillover effects could result in biased spillover estimates relative to the estimates that would be obtained in a double-randomized design, in which measured and unmeasured confounders are balanced across both populations.

Second, in 33 out of 54 studies, spillover measurement was not pre-specified, which may have increased the chance that a study’s results were biased. We found evidence of publication bias for spillover estimates reported as risk ratios but not for total or direct effects. Pre-specification helps prevent publication bias. Without pre-specification, spillover parameters may be defined in a way that increases the chance of detecting positive spillovers, whether intentionally or not. For example, studies estimating spillovers conditional on treatment density within fixed areas may define areas in a way that increases the magnitude of spillover effects. In addition, when spillover measurement is not pre-specified, investigators may fail to measure spillovers altogether or they may be less likely to report null spillover findings.

Finally, because spillover effects are likely to be smaller than treatment effects in most cases, studies that do not pre-specify spillover measurement and incorporate them into sample size calculations may be underpowered to detect spillovers. Because spillovers tend to have smaller effect sizes relative to total or overall effects, typically larger sample sizes are required to detect them. As a result, they are more subject to selective reporting than direct effects.

The overall quality of evidence was lower for economics studies than public health studies. Most public health studies evaluated spillovers of vaccines, whereas many economics studies measured spillovers of complex interventions such as conditional cash transfer programmes. Our finding that studies of more complex interventions typically had lower quality ratings is consistent with other studies.^87,88 There are several reasons for this pattern. First, complex, realistic interventions often cannot feasibly or ethically be randomized. As a result, many observational studies, some which employed innovative methods for measuring spillover effects in realistic settings, received lower quality ratings in our adapted GRADE framework.⁸⁹ Second, evaluations of complex interventions often cannot blind participants and/or investigators, resulting in lower quality ratings. Third, many public health studies measured spillovers of outcomes directly targeted by an intervention (e.g. the impact of the cholera vaccine on cholera risk). Studies that measured outcomes indirectly affected by the intervention (e.g. the impact of childhood deworming on later miscarriage⁴⁷) were more common in economics and received a lower quality rating due to concerns about indirectness of evidence.⁹⁰ Finally, economics and other social science studies have different reporting norms compared with public health studies and do not always report information required to receive a high quality rating, such as whether randomized treatment allocation was concealed.

Reporting recommendations

We found a wide range of terminology used to describe spillovers, a lack of standardization among spillover methods, and poor reporting of spillovers in many studies. Very few studies clearly defined the specific spillover effect estimated, and in many studies insufficient information was available to compare spillover effects with direct effects or with spillover effects in other studies. More standardized, systematic reporting across disciplines, particularly in the social sciences, would increase comparability across studies and allow for more careful assessment of risk of bias in studies.⁹¹ To facilitate such standardization, we propose a checklist specific to reporting of spillover effects, adapted from the CONSORT and STROBE frameworks^92,93 (Table 6). This checklist is focused only on reporting spillover effects and is meant to complement the CONSORT⁹² and STROBE⁹⁴ checklists. We provide an explanation and examples for each item in the checklist in Supplement 9, available as Supplementary data at IJE online. By including items in the checklist that apply to both randomized and observational studies, our objective is to foster more consistent reporting of spillovers across academic disciplines in future studies.

Table 6.

Reporting checklist for studies estimating spillovers

Section/topic	No.	Checklist item
Title and abstract
Title and abstract	1	If spillovers were measured as a primary outcome of a study, mention them in the title and/or abstract. Use the term ‘spillovers’ or ‘indirect effects’ to refer to spillovers
Introduction
Background	2	Use the term ‘spillovers’ or ‘indirect effects’ to refer to spillovers
Methods
Study design	3	Indicate whether spillover estimation was pre-specified
	4	Describe whether buffers existed between treatment and control units, whether in physical or social distance
	5	If treatment or outcome density was measured within areas, describe the rationale for and method of defining these areas
	6	Describe the scale on which spillovers are expected (e.g. household, village etc.)
	7	For study designs used to estimate spillovers other than the double-randomized or the cluster-randomized design, provide a clear description of the assumptions required to estimate valid statistical parameters if SUTVA is violated
Participants	8	Provide a clear description of treatment eligibility criteria
	9	State whether individuals enrolled to measure spillovers were eligible for the treatment or not
Interventions	10	Provide a clear description of how treatment was allocated to groups and individuals
	11	Describe whether untreated individuals in treated areas were randomly assigned to not receive treatment, if they opted out of treatment, if they were ineligible for treatment or if there were other reasons they were not treated
	12	State whether the level of treatment allocation was chosen in order to measure spillovers
	13	Describe the mechanism of spillovers hypothesized and assessed for each treatment
	14	Describe whether a buffer zone was created between treatment and control units
Outcomes	15	If outcomes measured to estimate direct, total or overall effects differed from outcomes measured to estimate spillover effects, provide a rationale for the difference
Study size	16	Describe any calculations conducted to determine the sample size needed to estimate spillover parameters. If none, state that none were conducted
Statistical methods	17	Define the specific spillover parameter(s) estimated for each intervention
	18	Describe the statistical analysis methods used to estimate spillover effects
	19	Indicate whether spillovers were estimated among individuals allocated to not receive treatment vs those that chose not to take treatment (i.e. indicate whether the spillover analysis was intention-to-treat)
Results
Participant flow	20	If using a clustered design to measure spillovers, provide the number of clusters allocated to treatment and control that were included in the assessment of spillovers
	21	If using a clustered design to measure spillovers, provide the number of individuals that received and did not receive treatment within treatment and control clusters
	22	If using a clustered design to measure spillovers conditional on eligibility status, provide the number of individuals eligible to receive treatment in treated clusters and the total number of individuals in treated clusters
	23	If using a clustered design to measure spillovers, provide the number of individuals allocated to treatment within treatment clusters, allocated to not receive treatment within treated clusters, and allocated to control clusters
	24	If using a clustered design to measure spillovers, provide information about the proportion of individuals receiving treatment within each cluster
	25	If measurement occurred in buffer zones between treatment and control clusters, provide the number of individuals who did and did not receive treatment in buffer zones
	26	Describe whether loss to follow-up rates were similar among individuals measured for spillover vs direct/total/overall effects and whether the characteristics of those lost to follow-up for spillover measurement differed from those who were not lost to follow-up
Recruitment	27	If dates of data collection for spillover measures differed from dates for direct, total or overall effect measures, explain the discrepancy
Main results	28	Clearly label which results estimate each spillover parameter
	29	If multiple spillover mechanisms were hypothesized, label results according to the hypothesized spillover mechanism
	30	Present direct, total, overall and spillover effects in the same population subgroups to allow for assessment of the proportion of the total and overall effects attributable to spillovers
	31	Report whether there was any evidence that untreated individuals in the treatment or control group were exposed to treatment (e.g. if untreated individuals had heard of the intervention or knew individuals who received it)
	32	Describe any evidence of contamination of the control group
Discussion
Summary of findings / key results	33	Present theory or evidence supporting the proposed mechanism of spillover.
Limitations	34	Discuss any potential biases that may be present for spillover parameters Discuss whether these biases may also be present for direct or total effect parameters. This includes contamination of the control group
	35	Articulate whether any analyses conducted to estimate spillovers were not pre-specified
Generalizability	36	Comment on external validity of findings and whether any methods used to estimate spillover effects may have compromised external validity (e.g. matching of untreated in the treatment group to untreated in the control group)

SUTVA, stable unit treatment value assignment.

Limitations

Our search and review process was subject to several limitations. Although we made every effort to conduct a comprehensive search, since the concept of spillovers is poorly indexed, it is possible that we missed relevant articles. Greater consistency in the use of terms that describe spillovers would improve future efforts to identify relevant papers by searching electronic databases. We excluded studies from high-income countries from this review since our focus was on interventions relevant to populations in low- and middle-income countries. This focus was a requirement of our funder. However, there are relevant papers measuring health spillovers from high-income countries, many of which evaluate vaccines.^95–97 Some relevant papers which may have been eligible came to our attention after we completed our search process, so we did not include them.^98–101 In addition, some of the databases we searched (e.g. Google Scholar) do not allow for repeatable searches, so our complete search results cannot be fully replicated. During the review process, some titles and abstracts could only be reviewed by one team member, and duplicate risk of bias assessment was only possible in a subset of studies. It is possible that there was misclassification that would have been prevented by complete duplicate review.

Our synthesis of results was also subject to several limitations. The information needed to convert standard errors from the additive to the relative scale was not available in the included studies, so our comparison of estimates across studies did not take precision into account. Since there were very few studies measuring spillovers of the same intervention, our ability to summarize results by intervention type was limited. For papers on vaccines and mass drug administration for infectious disease control, results from studies included in the review may have been dependent because many studies re-analysed data from the same study populations or from the same country. Evidence of spillovers for these interventions in other populations would strengthen the generalizability of these findings.

Conclusions

This review of spillover effects on health outcomes in low- and middle-income countries found a wide range of terminology used to describe spillovers, a lack of standardization among spillover methods and poor reporting of spillovers in many studies. The strongest evidence for spillover effects was found in studies evaluating vaccines and mass drug administration to control infectious disease. There was little evidence available for other types of interventions, and the quality of evidence was moderate or poor in most studies. Future studies would benefit from incorporation of spillover measurement in the design phase and standardized reporting of spillover estimation methods and spillover findings.

Funding

Funding for this study was provided by the International Initiative for Impact Evaluation (3ie) (SR4-1084).