Systematic Review of Natural Experiments for Childhood Obesity Prevention and Control

Abstract

Context:

The National Academy of Medicine recommends childhood obesity prevention efforts making healthier options the passive choice. This review evaluated the effectiveness of population-level policies and programs from natural experiments for childhood obesity prevention.

Evidence acquistion:

The search included PubMed, CINAHL, PsycINFO, and EconLit from 2000 to 2017 for policies evaluated by natural experiments reporting childhood BMI outcomes. The studies were analyzed in 2017–2018. Interventions were classified by environmental focus (food/beverage, physical activity, or both) and stratified by setting (school, community, both). Risk of bias was evaluated for each study.

Evidence synthesis:

Of 33 natural experiments, most (73%) took place in the school setting only. The most common environmental focus in any setting was food/beverage (48%). All four studies that focused on both food/beverage and physical activity in schools demonstrated decreased prevalence of overweight/obesity or BMI z-score by 0.04–0.17. BMI decreased in all four studies in both school and community settings. The largest effect size was a decrease in BMI z-score of 0.5, but most were <0.25. The risk of bias was high for most (76%) studies. Most (63%) of the eight studies with low/medium risk of bias took place in the school setting focused on the food/beverage environment; effects on BMI were mixed.

Conclusions:

Natural experiments evaluating school-based policies focusing on both the food/beverage and physical activity environments (versus targeting only one) consistently showed improvement in BMI. However, most studies had high risk of bias, highlighting the need for improved methods for evaluation of natural experiments for childhood obesity prevention.

CONTEXT

Most children today (57%) are projected to have obesity by age 35 years.¹ Childhood obesity is associated with significant health-care costs and morbidity in childhood, and increased risk of obesity, cardiovascular disease, and other comorbidities throughout the lifespan.^2–6 Despite consensus on the need for coordinated, multicomponent interventions to address the childhood obesity epidemic, child-hood obesity rates in the U.S. continue to climb for adolescents, children of low-income backgrounds, and children of racial/ethnic minority backgrounds.⁷

Effective childhood obesity prevention and treatment strategies have been difficult to develop and implement at a population level. The National Academy of Sciences recommends childhood obesity prevention approaches that promote making the healthier choices the passive option: “food and beverage environments that ensure that healthy food and beverage options are the routine, easy choice.”⁸ The school environment has been considered an optimal setting for obesity prevention and control interventions, because children spend a substantial portion of their day at school, consume two thirds of their calories at school,⁹ and are in a structured environment that supports learning about nutrition.¹⁰ Despite the large number of school-based interventions, there is limited evidence showing an impact on childhood BMI.¹¹ A large systematic review conducted in 2015 showed that only 39% of studies had a favorable effect on adiposity measure, and that there was greater efficacy among the school-based interventions that included a home-based component.¹¹ Most of the 147 studies included in this review were experimental studies.¹¹

Few studies have evaluated structural/environmental or policy-level interventions focusing on the food/beverage or physical activity (PA) environments. Natural experiments are defined as studies in which the exposure to an intervention (i.e., policy or structural change) was not manipulated by the researcher.¹² Because much of the evidence from previous reviews was from experimental studies¹³ and did not assess policy, program, and built environment changes,^14,15 the objective of this review is to identify natural experiment studies that report effects of programs, policies, or built environment changes on childhood BMI outcomes.

EVIDENCE ACQUISITION

This review focuses on studies identified in a larger systematic review entitled, “Methods for Evaluating Natural Experiments in Obesity: A Systematic Evidence Review” (PROSPERO #CRD42017055750).¹⁶ The original review focused on evaluating the methods used in obesity natural experiments. The scope of this review extends beyond the original review by reporting on the effectiveness of the policies, programs, and built environment changes on childhood weight (i.e., BMI), dietary, and PA outcomes (Appendix Table 1, available online, for PRISMA Checklist and Appendix Table 2, available online, for Complete Search Terms).¹⁶

Data Sources and Search Strategy

A description of the methods for the original review are in the full report.¹⁶ The search was conducted in PubMed, CINAHL, PsycINFO, and EconLit from January 1, 2000 to August 24, 2017, to identify all U.S. and non–U.S. studies of programs or policies targeting obesity prevention and control in people of all ages and in any setting. This review focused on studies done in pediatric populations.

Study Selection

Two reviewers independently screened abstracts and full-text articles using prespecified inclusion and exclusion criteria to identify natural experiments evaluating a program or policy aimed at combating pediatric obesity that reported on weight measures (BMI, BMI z-score [BMIz], BMI percentile, or proportion of children with overweight or obese BMI). Studies were defined as natural experiments based on the United Kingdoms’ Medical Research Council guidance.¹² Excluded studies without a comparator group were excluded, such as an unexposed (versus exposed) or a pre- (versus post-) comparison group. The studies were analyzed in 2017–2018.

Data Extraction and Risk of Bias Assessment

Two reviewers serially extracted data on study design, setting, population and intervention characteristics, magnitude of the effect size, and p-value for weight outcomes (change in BMI, BMIz, BMI percentile, or BMI classification according to age). Behavioral outcomes (PA and dietary behaviors [vegetable and fruit intake and sugar-sweetened beverage (SSB) intake]) were not reported in every study. When reported, direction of the effect, measures of statistical significance, and details about how each outcome was assessed were extracted. For BMI, the way in which height/weight was measured, direction of effect, and magnitude of effect on BMI were extracted; BMI percentiles were converted into BMIz for the forest plot.

Using the Effective Public Health Practice Project tool, two reviewers independently assessed the risk of bias on six domains for each included study (selection bias, blinding, withdrawals and dropouts, study design, confounding, and data collection; Appendix Table 3, available online).¹⁷ Each study received a global risk of bias rating: low if no domain was rated high risk of bias, moderate if only one domain was rated high risk of bias, and high if two or more domains were rated as high risk of bias. A sensitivity analysis excluding studies that had a global rating of high risk of bias was conducted (Appendix Figure 1, available online).

Data Synthesis and Analysis

Recognizing the important distinction between school and community settings in childhood obesity prevention efforts, the results were first stratified by setting (school, community, or both) and then by their primary environmental focus. The five environmental foci were: (1) PA and physical/built environments, (2) food/beverage environment, (3) messaging environment, (4) health care and work environment, and (5) school environment, based on by the National Academy of Medicine 2012 report, Accelerating Progress in Obesity Prevention: Solving the Weight of the Nation.⁹ All of the studies in this review focused on the first three environments (PA and physical/built, food/beverage, and messaging environments). Studies with multiple primary environmental foci were categorized under multiple foci.

For each study, the effect of the programs, policies, or built environment changes on BMI were reported, as well as the changes on dietary and PA outcomes when reported. The magnitude of effect sizes for BMI outcomes were compared using a forest plot (Appendix Figure 2, available online) and compared diet and PA behaviors using an evidence map (Appendix Figure 3, available online). Studies that reported change in BMI or change in the percentage of children who had overweight or obese BMI were not included in the forest plots.

Strength of Evidence for BMI Outcomes

Two reviewers independently assessed the evidence based on the intervention focus and assessed studies’ limitations, directness, consistency, precision, and potential reporting bias using guidelines from the Agency for Healthcare Research and Quality (AHRQ; Appendix Table 4, available online).¹⁷ High strength of evidence indicates that the evidence likely reflects the true effect, moderate strength indicates that further research may change the result, low strength indicates low confidence that the evidence reflects the true effect, and insufficient indicates no confidence in the estimate of effect for the outcome.

Role of Funding Source

The NIH Office of Disease Prevention funded the larger systematic review through an interagency agreement with AHRQ and a working group convened by NIH assisted in developing the scope of the review and its key questions. Neither organization had a role in study selection, quality assessment, or synthesis. The investigators are solely responsible for the content.

EVIDENCE SYNTHESIS

Of the 156 natural experiment studies in the original systematic review, 33 natural experiments reported on childhood BMI outcomes for this review (Figure 1). Most studies (n=29, 89%) were conducted in the U.S., with the remaining from Canada (n=2) and Australia (n=2). Table 1 provides the baseline characteristics of the study population for each study, classified by setting (school, community, both) and environmental focus. Among the 29 U.S. studies, 35% evaluated local policies, 31% state/regional policies, 24% U.S./federal-level policies, and 10% (n=3) non-governmental policies. Federal-level policies included the Child Nutrition and Special Supplemental Nutrition Program for Women, Infants, and Children Reauthorization Act; state- or regional-level policies included competitive food laws and school-district food policies; and local policies included efforts to promote PA through exercise classes, sidewalks, or playgrounds^18–20 (Table 1).

Figure 1.

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) diagram of included studies.

^aSum of excluded abstracts exceeds 25,093 because reviewers were not required to agree on reasons for abstract exclusion.

^bSum of excluded articles exceeds 1,190 because reviewers were not required to agree on reasons for exclusion.

Table 1.

Population Characteristics of Natural Experiment Studies That Assessed Pediatric BMI, Stratified by Setting and Then by Environmental Focus

		Overall baseline population characteristics, range						Policy level evaluated by the studies within the stratum, %
Characteristics	Number per study, range	Study duration, months	Age, years	Female, %	White, %	Black, %	U.S., %
School setting,a n=24
Physical activity/physical and built environment, n=6
Elementary, n=2 (33%); middle school, n=3 (50%); and high school, n=3 (50%)	100–83,253	5–48	10.4–18	49.8–55	3–85	2–27.6	83.3	Local: 17 State: 33 Federal: 17 Non-government: 17 Other country: 17
Food and beverage environment, n=14
Early childhood, n=1 (7%); elementary, n=8 (57%); middle school, n=8 (57%); and high school, n=6 (43%)	431–1,065,562	18–96	2–19	45–57	8.4–84	6–36.5	85.7	Local: 43 State: 21 Federal: 21 Other country: 14
Multiple environmental targets in the school setting, n=4
Elementary, n=2 (50%); middle school, n=2 (50%); high school, n=2 (50%)	23,347 (1 study)	7–96	NR	49–50.6	13–70.3	7.4–61	100	Local: 25 State: 25 Non-government: 50
Community setting included, n=7
Community and school settings, one or multiple environmental focus, n=4	104–3,648	12–24	0–18.6	49–51.7	13–83	7.4–61	75	Local: 25 State: 25 Federal: 25 Other country: 25
Community setting, food and beverage environment, n=2	5,193–7,414	NR	10.4–11.2	49.1–53.3	37.9–48.8	14.2–31.2	100	Federal: 100
Community setting, physical activity/physical and built environment, n=1	1,443	36	2–17.9	56	22	77	100	Local: 100
Other, n=2
Increasing parental awareness of BMI, n=2	1,081–1,148,000	84	15.8–17.6	47.5–49.4	31.7	9.2	100	State: 100

^aSome studies took place in multiple grade levels.

NR, not reported.

Risk of Bias

The majority of studies (76%, 25/33) had an overall high risk of bias (indicating a low-quality study) because of high rates of withdrawal and dropouts (24/33 studies) and low quality of the study design (17/33). Seven studies had a medium risk of bias, and one study had a low risk of bias (Appendix Table 3, available online). In the sensitivity analysis, four of eight studies with low or moderate risk of bias showed favorable effects on BMI (Appendix Figure 1, available online, shows the forest plot excluding studies with high risk of bias).

Setting and Environmental Foci

The majority (n=24, 73%) of natural experiment studies were conducted in the school setting. Among school-based studies, most (n=14, 58%) focused primarily on the food/beverage environment, six (25%) primarily on the PA environment, and four (17%) on multiple environments. Most studies were conducted across several grade levels: 12 (50%) included elementary school, 13 (54%) included middle school, and 11 (46%) included high school. One study was conducted in an early childcare setting. Few studies (n=7, 21%) included a community setting, four of which included both community and school settings. Of the remaining three studies set in the community only, two focused on the food/beverage environment and the other on the PA/built environment.

Across all settings, the most common single environmental focus was the food/beverage environment (16 of 33 studies). Eight studies (24%) focused on the PA/built environment, and seven studies (21%) had multiple environmental foci. Two studies evaluated programs that increased parental awareness of their child’s BMI by adding an additional BMI screening for children at school and notifying parents of their child’s BMI.

Assessment Methods for All Outcomes

Data on height and weight were directly measured by trained study staff in 21 of 33 studies (64%), self-reported in six of 33 studies (18%), assessed by other methods (e.g., FitnessGram test results) in five of 33 studies (15%), and obtained from electronic health records in one study. For diet, seven of eight studies reporting on fruit/vegetable intake described how it was assessed: food frequency questionnaires (n=2) or other types of questionnaires (n=5). Seven of nine studies assessing SSB intake described how this outcome was assessed: food frequency questionnaires (n=1) and other types of questionnaires (n=6). PA was assessed by questionnaire (n=3), survey (n=3), observation (n=1), and pedometer (n=1) in the eight studies reporting fruit and vegetable intake.

BMI Outcomes

Appendix Figure 2 (available online) displays a forest plot summarizing weight outcomes, in terms of BMIz, and Table 2 reports all BMI outcomes data. Table 3 reports the study details and weight outcomes for the eight studies with low or medium risk of bias. Strength of evidence for the outcome of BMI was low in five of six categories of studies and insufficient for studies conducted in the community setting focusing on PA (Appendix Table 4, available online). The studies with low or medium overall risk of bias will be highlighted here as examples.

Table 2.

Summary of Outcomes Stratified by School Versus Community Setting, and Then by the Environmental Focus of the Intervention

	Outcomes,a n
Variables	BMI	F/V intakeb	SSB intakeb	PAb	Overall summary
School setting, n=24 (73%)
PA/built environment focus; U.S., n=5 (80%); Australia, n=1 (20%)	Favorable, 3/6 No effect, 3/6	—	No effect, 1/1	Favorable, 3/5 No effect, 2/5	The Australian study was the only one that improved both BMI and PA outcomes, by improving the school gymnasium and sports equipment. Two studies improved the intermediary PA without improving BMI; one of these had only 5-month follow-up, making a BMI effect less likely
F/B focus; U.S., n=12 (86%); Canada, n=2 (14%)	Favorable, 8/14 Unfavorable, 3/14 No effect, 3/14	Favorable, 1/3 Unfavorable, 1/3 No effect, 1/3	Favorable, 3/4 No effect, 1/4	—	Competitive F/B policies improved BMI, F/V, and SSB, particularly among racial and ethnic minority students. One study with short follow-up (8 months) improved SSB intake, but not BMI. Participation in the School Breakfast and National School Lunch programs affected BMI unfavorably. The two Canadian studies did not improve BMI.
Multiple environmental foci; U.S., n=4 (100%)	Favorable, 4/4	—	—	—	Policies focused on both F/B and PA. The two studies that also focused on the messaging environment achieved the largest BMI improvements.
Community settings included (community only or both community and school), n=7 (21%)
Multiple settings, single or multiple focus; U.S., n=3 (75%); Australia, n=1 (33%)	Favorable, 4/4	Favorable, 1/2 No effect, 1/2	Favorable, 1/2 No effect, 1/2	Favorable, 2/3 No effect, 1/3	The three interventions with multiple environmental foci (F/B and PA, with or without messaging) in multiple settings improved both BMI and behavioral outcomes. The fourth study did not report behavioral outcomes.
F/B focus, community; U.S., n=2 (100%)	Favorable, 1/2 No effect, 1/2	No effect, 1/2	Favorable, 1/2 Unfavorable, 1/2	—	Increased state-wide soda tax in grocery stores decreased SSB intake and BMI, especially among children at risk for obesity from low income and minority backgrounds. In one study, SNAP recipients consumed more SSB than low- income children not receiving SNAP.
PA/built environment focus, community; U.S., n=1 (100%)	Unfavorable, 1/1	—	—	—	One study increased BMIz (0.03) after an urban park was developed, with no difference by proximity to park (all participants lived within 11 miles)
Other: increasing parents’ awareness of child’s BMI, n=2 (6%)
Parental awareness of children’s weight status; U.S., n=2 (100%)	No effect, 2/2	No effect, 1/1	—	—	Interventions that alerted parents to their children’s BMI had a favorable but not significant effect on BMI; F/V was reported for one study and was not significant.

^aStatistically significant outcomes (p<0.05) were classified as favorable or unfavorable based on the direction of the effect.

^bNot reported in all studies, denominator is the number of studies included in this analysis that also reported this behavioral outcome within a given category of study.

BMIz, BMI z-score; F/B, food/beverage; F/V, fruit and vegetable; PA, physical activity; SNAP, Supplemental Nutrition Assistance Program; SSB, sugar-sweetened beverages.

Table 3.

Study, Population Characteristics, and Main Results for BMI Outcomes of Natural Experiment Studies With Low or Medium Risk of Bias

	Study characteristics		Baseline population characteristics within studies
First author, year, country	Program level; policy or built- environment change	Comparator group	n	Age or grade, range	Female, %, range	Race, %, range	Analysis, study length	Main result for BMI outcomes
Capogrossi, 2017, U.S.21	Federal; participation in both the School Breakfast Program and National School Lunch Program	Participation in both programs versus only the National School Lunch Program	3,020	Grades 1–8	45–57	Black, 7–33 Hispanic, 17–26	DID and ATT, 8 years	Increased probability of being overweight with participation in both meal programs
Hennessy, 2014, U.S.22	State; competitive food laws in schools	Strong food laws versus weak food laws versus no law (reference)	16,271	11–14 years old	51	White, 64–47 Black, 14–21 Hispanic, 15–25 Other, 8–7	Regression model, 2 years	Increased odds of overweight/obesity Weak laws: OR=1.23, 95% CI=1.05, 1.45 Strong laws: OR=1.01, 95% CI=0.798, 1.30
Heelan, 2015, U.S.23	Local; district- and school-level implementation of physical education grant program, healthier school meals, school wellness, and BMI screening	Pre-implementation versus post-implementation	2,244	Grades K–5	NR	White, 85	Pre–post, 6 years	Decreased prevalence of overweight or obesity, from 16.4% to 13.9%
Schwartz, 2016, U.S.24	Local; implementation of water jets in schools for dispensing cooled water	Schools with versus without water jets	1,065,562	Grades K–8	50	Asian, 12–14 Black, 33–36 Hispanic, 37–39 White, 13–14	DID, 5 years	Reduced BMIz in boys: OR= −0.025, 95% CI= −0.038, −0.011; in girls: OR= −0.022, 95% CI= −0.035, −0.008
Fitzpatrick, 2017, Canada25	Local; Dietary environment in the school as well as in the surrounding neighborhoods	Healthful versus unhealthful food environments	431	8–12 years	42–49	White, 100	Regression model, 2 years	No difference in BMIz Mean change= 0.06, 95% CI= −0.16, 0.28
Nanney, 2016, U.S.26	Local; policies around foods available in school vending machines and stores; PE requirements, intramural sports	Pre-implementation versus post-implementation	7,237	Grade 9	NR	Minority, 10–14	Regression model, 6 years	BMI% +0.01 (95% CI=0.00, 0.02) when less healthy food available
Goldsby, 2016, U.S.27	Local; construction of a new neighborhood park	Pre- versus post- construction of park	1,443	2–17 years	56	Black, 78 White, 22 Hispanic, 14	Pre-post, 1–3 years	Increased BMIz Mean change= +0.03, p= 0.0007
Madsen, 201128	Other; BMI screening with parental notification of BMI	Parental notification versus no parental notification	755	Grades 5, 7, 9	NR	Black, 9 White, 33 Hispanic, 40	Regression model, 7 years	No difference in BMIz Mean change= −0.01, 95% CI= −0.03, 0.01

ATT, average treatment effect in the treated; BMI%, BMI percentile; BMIz, BMI z-score; DID, difference in difference; K, kindergarten; NR, not reported; PE, physical education.

Five of the eight studies with low or medium risk of bias took place in the school setting and focused on the food/beverage environment. Three of these five (60%) studies achieved favorable effects on BMI through programs such as: implementing water jets in the school,²⁴ reducing unhealthy foods and beverages available in vending machines and school stores,²⁶ and creating a healthful food environment in the school and surrounding neighborhoods (that children travel through on the way to school).²⁵ Two of the five studies found unfavorable effects on BMI: Capogrossi et al.²¹ found that participation in both the School Breakfast Program and National School Lunch Program was associated with an increased probability of being overweight compared with participation in only the National School Lunch Program; Hennessy and colleagues²² found that weak competitive food laws were associated with an increased odds of overweight or obesity.

One of the eight studies with low or medium risk of bias took place in the school setting and focused primarily on PA and the built environment. In this study, Heelan et al.²⁹ showed a decline in the prevalence of overweight/obesity after implementation of new physical education programs, school wellness, and BMI screening.

Of the two other studies occurring in other settings, the study by Goldsby and colleagues²⁷ took place in the community setting only and focused solely on the PA environment. They found a small but statistically significant increase in BMIz after a neighborhood park was built. Another study, by Madsen et al.,²⁸ focused on screening children’s BMI and alerting parents to the results and found no effect on BMI.

Considering the entire body of evidence (n=33 studies), including those with low, medium, and high risk of bias, three of the six school-based studies that focused on the PA/built environment achieved favorable BMI outcomes.^23,30,31 Eight of the 14 school-based studies that focused on the food and beverage environment achieved favorable BMI outcomes (Table 2). All four studies that took place in a school setting and focused on multiple foci, including the food/beverage and PA/built environments, had favorable BMI outcomes.^32–35 Two of these four studies that also focused on healthy messaging achieved the largest improvement in BMI^24,35 (Table 2).

All four studies conducted in both the school and community settings showed a reduction in BMI. Three studies focused on both the PA and food/beverage environments, demonstrating a reduction in BMIz ranging from −0.02 to −0.5 and improvements in PA, dietary outcomes, or both.^19,36,37

In additional analyses, the effectiveness on BMI was assessed by the level of government policy. Among the 26 U.S. studies evaluating government policies, 15 (58%) reported favorable effects on BMI—six of seven (85%) federal policies, three of nine (33%) state/regional policies, and six of ten (60%) local policies. BMI outcomes were also assessed by age of participants by stratifying all 33 studies into those that focused on children in elementary school and younger compared with those whose participants were mostly middle school and older. Sixty-six percent of studies focused on the younger age group had favorable effects on BMI, compared with 54% of studies focused on the older age group. Of the four studies that included only high school children, 50% had favorable effect on BMI.

Diet Behaviors

Appendix Figure 3 (available online) and Table 2 display the dietary outcomes. Nine studies reported SSB intake and eight studies reported fruit/vegetable intake. None of the studies that included multiple environmental foci in the school setting reported behavioral outcomes. Three of four (75%) studies that were conducted in the school setting and focused on the food/beverage environment showed reduced SSB intake.^18,38,39 These three studies evaluated state- or school-wide policies to decrease access to SSBs. Fruit/vegetable intake was evaluated in only three of the studies conducted in the school setting that focused on the food and beverage environment. One of the three studies found favorable effects on fruit/vegetable intake after implementation of a competitive food policy⁴⁰ and another found unfavorable effects on fruit/vegetable intake after implementation of a school district–wide policy to reduce SSBs³⁸ (Appendix Figure 3, available online, and Table 2).

Two of the four studies conducted in both school and community settings reported on fruit/vegetable and SSB intake.^36,37 One study found favorable outcomes on both fruit/vegetable and SSB intake after implementation of grant-supported community-based interventions that included physical education, nutrition classes, improved cafeteria options in schools, walking trails, and PA promotion within the community settings¹⁹ (Appendix Figure 3, available online, and Table 2).

Both of the studies in the community setting focused on the food and beverage environment reported on SSB intake.^41,42 One study showed decreased SSB intake after implementation of a soda tax in grocery stores.⁴² The other study found that Supplemental Nutrition Assistance Program beneficiaries consumed more SSB than non–Supplemental Nutrition Assistance Program beneficiaries.⁴¹

Physical Activity Behaviors

Eight of 33 studies (24%) reported on PA behaviors in children (Appendix Figure 3, available online, and Table 2). Among the six school-based studies that focused primarily on the PA environment, five reported PA outcomes and three of these five showed an increase in PA (i.e., increased participation in physical education class or self-reported PA).^{30,31,43–45} For example, one study analyzed a Texas state bill requiring schools to have children in sixth to eighth grade participate in 30 minutes of structured PA daily.⁴⁴ The exposed students had a significant increase in self-assessment of PA level relative to the control participants (p=0.01).⁴⁴

Three of four studies in both the school and community settings that focused on multiple environments reported on PA outcomes. Two of these three studies (67%) found a favorable effect on PA (increased moderate and vigorous PA minutes/hour) after implementation of community-wide initiatives to improve PA in the school and community through positive messaging, walking clubs, walking trails, and more physical education time.^19,46

Analytic Approaches

Commonly used analytic approaches in these natural experiment studies included regression models (n=12), pre–post (n=11), difference-in-differences (n=7), propensity score matching (n=1), and interrupted time series (n=2).

DISCUSSION

This is the first systematic review of natural experiments for childhood obesity prevention and control that assessed the effectiveness on BMI. Among the 33 natural experiment studies that reported on child BMI outcomes, 25 studies had a high risk of bias. With only eight studies having a low or medium risk of bias, it was not possible to draw strong conclusions about this body of evidence. These eight studies were distributed between different environ- mental foci and settings, making it difficult to determine whether one environmental focus consistently produced favorable effects on BMI. Of the five studies focused on the food/beverage environment in the school setting, the majority (60%) showed favorable effects on BMI. More high-quality research is needed in order to understand the best programs, policies, and built environment changes for childhood obesity prevention and control.

Considering all 33 studies, most were conducted in the school setting and focused on the food and beverage environment. The most successful programs were multidimensional, either focusing on multiple environments within the school or taking place in multiple settings. A wide range of programs and policies from different settings and levels (local, state, national, non-governmental) were evaluated in these studies. Some studies assessed community-based initiatives focused in one small geographic area, whereas others evaluated national-level policies, such as the federal Child Nutrition and Special Supplemental Nutrition Program for Women, Infants, and Children Reauthorization Act. In this review, federal and local policies were more effective at reducing BMI than state-/regional-level policies. Yet, data are limited regarding longitudinal impacts of policy change on younger children and whether improvements in BMI may be sustained, reversing the rise in obesity rates among children.

The multidimensional aspect of these more successful programs confirms professional society guidelines that recommend population-level, multipronged interventions for childhood obesity prevention and control.^8,47,48 The National Academies of Sciences, Engineering, and Medicine; WHO; and the U.S. Preventive Services Task Force recommend screening for pediatric overweight/obesity and implementing multicomponent interventions for obesity prevention and treatment.^8,47,48

Although the effect sizes for BMI were small for most studies in this review, previous research has shown that a decrease in BMIz of only 0.15 reduces cardiovascular risk factors, and a decrease in BMIz of 0.2 in children is equivalent to 5% body weight loss in an adult.^49,50 Not all studies in this review reported BMIz, but three of the 13 studies that did report BMIz achieved a decrease in BMIz of at least 0.15 in at least one of the intervention arms.^19,32,51 Similar modest effects on BMIz (−0.16) were observed recently in a 2-year clinic-based intervention, Massachusetts Childhood Obesity Research Demonstration, which included on-site weight clinics, electronic decision support for clinicians, integration of community health workers, and pediatric weight management training for clinicians.⁵² Additionally, a recent study examining a year-long, multiprong school-based intervention in England showed no effect on BMI.⁵³ This review and these recent studies show that the medical and public health communities struggle to know how to tackle pediatric obesity. Although local programs can have an impact, the epidemic of childhood obesity needs to be addressed broadly through the entire food system.⁵⁴

The American Heart Association stresses intervening on childhood obesity to prevent losing the gains in cardiovascular disease health that have been achieved at a population level because of reduced tobacco use.⁵⁵ Recently, organizations such as the Robert Wood Johnson Foundation have focused obesity interventions on prekindergarten ages and early childhood education. Although this review did show that interventions on younger children were more likely to favorably affect BMI than interventions on adolescents, it is not known whether these effects are sustained into adolescence. However, effective policies and programs are urgently needed to stop rising obesity rates among adolescents and young adults, which are currently increasing more than in younger age groups.^7,56,57 Adolescents need to be equipped with healthy habits and knowledge to avoid excessive weight gain during the important transition into early adulthood.⁵⁶ Most of the studies in this review were conducted in elementary- and middle school–aged children, not in high school–aged children who are in this high-risk group. Because adolescents and young adults are generally healthy and not interfacing with the medical system often,⁵⁶ the focus of resources and research should be about making healthy options the passive choice in the community and school settings, and increasing preferences for healthy foods.

Limitations

A major limitation in most of the studies included in this review was a lack of clear descriptions of what was actually implemented in the policies and programs. For example, studies often did not describe changes that were made to food and beverage choices for children, beyond describing them as “healthy changes.” From what was described in the papers, it was difficult to determine what the key elements were in many of the interventions reviewed. Future research is needed to identify the core components of policies and programs that are driving effects on BMI, and how the components are delivered and interact to potentiate each other. Many of the studies in this review included self-reported height and weight data, either self-reported by the parent or by the child participant, which limits the accuracy of the anthropometric data in those studies. Additionally, the programs and policies implemented for childhood obesity control were not evaluated rigorously. The majority (76%) of studies in this review had an overall high risk of bias. The most common reason for studies to have high risk of bias was high rates of withdrawals and dropouts and lower quality of study design. Study design is something within the control of researchers, and the topic of childhood obesity deserves high-quality studies. However, the authors acknowledge that because this review focuses on natural experiments, the studies do not involve the most rigorous study designs, like clinical trials. Additionally, potential publication bias means that this review may not have captured all of the efforts made and resources used in this field. Overall, the strength of evidence for this body of literature is limited, with no category of study achieving moderate or high strength of evidence.

The AHRQ guide for conducting comparative effectiveness reviews was used to evaluate the strength of evidence for weight/BMI outcomes; however, there is no standard method for assessing strength of evidence in natural experiment studies. Studies had a range of follow-up times (range, 5 to 96 months), intervention approaches, and outcomes measures, making a true comparison of effectiveness on BMI outcomes among studies challenging. Sustainability and scalability were not evaluated in this review. Future work is needed to assess the implementation and dissemination of healthy programs and policies so that they become the norm in communities and schools. Natural experiments are an important method for evaluating such population-level policies. However, implementation of the policies in this review would be difficult, based on the descriptions given in many of the studies.

CONCLUSIONS

Despite consensus on the need for coordinated, multicomponent interventions to address the childhood obesity epidemic, reducing childhood obesity remains a challenge. Among natural experiments evaluating policies and programs for their effectiveness on BMI, the most successful ones created healthier environments for children in multiple settings or across multiple environmental foci within the school setting. However, given the high risk of bias for most studies, and overall low strength of evidence in natural experiments for childhood obesity, more research is needed in this area to address the urgent public health issue of childhood obesity.