Measuring the intensity of community programs and policies for preventing childhood obesity in a diverse sample of U.S. communities: The Healthy Communities Study

Abstract

Introduction:

Efforts to address the critical public health problem of childhood obesity are occurring across the US; however, little is known about how to characterize the intensity of these efforts.

Objectives:

To describe the intensity of community programs and policies (CPPs) to address childhood obesity in 130 U.S. communities. To examine the extent to which observed CPPs targeted multiple behaviors and employed a comprehensive array of strategies.

Methods:

To document CPPs occurring over a 10-year period, key informants were interviewed using a semi-structured interview protocol. Staff coded CPPs for key characteristics related to intensity, including reach, duration, and strategy. Three types of CPP scores were calculated for: intensity of CPPs, targeting of CPPs toward multiple behaviors, and strategies used.

Results:

9,681 CPPs were identified. On average, communities had 74 different CPPs in place (standard deviation 30), with variation in documented CPPs (range 25–295). Most communities experienced a steady, modest increase in intensity scores over 10 years. CPP targeting scores suggested that communities expanded the focus of their efforts over time to include more behaviors and strategies.

Conclusions:

Findings of this large-scale study indicate that great variation exists across communities in the intensity and focus of community interventions being implemented to address childhood obesity.

Introduction

Childhood obesity is recognized as a serious public health problem nationally and globally^1–5. Increasingly, U.S. federal initiatives and agenda-setting organizations have called for environmental and policy actions that use systems concepts and tools⁶ or incorporate multiple behavior change strategies to affect physical activity, nutrition, and healthy weight of children². These organizations recommend using a comprehensive set of community programs and policies (CPPs) that address multiple behaviors, reach intended targets through relevant sectors of the community³, are driven by communities, and benefit all segments of communities⁷.

Although multi-level interventions implemented in school and community settings can impact obesity⁸, few published accounts communicate the dose of community intervention associated with improvements in markers of healthy weight among a community’s children. In one exception, Economos et al.⁹ examined the effects of a comprehensive community initiative that resulted in improvements in body mass index (BMI) z-scores among children in the intervention versus control groups in Massachusetts. Although some activities that made up the comprehensive intervention were documented, each was treated as though it contributed equally to the BMI changes that were observed. A challenge with understanding comprehensive efforts is determining how to systematically measure the independent variable, that is, the number and intensity of programs and policies that were implemented in the community¹⁰.

Some researchers have attempted to measure the intensity—or potential dose—of comprehensive community health initiatives. For instance, Glasgow et al.¹¹ proposed using the RE-AIM model that characterizes elements, such as reach, to help understand the impact of single programs or policies. In an evaluation of the Kaiser-Permanente Community Health Initiative, Cheadle et al.¹² used the dimensions of reach, efficacy, and strength to describe and compare the “dose” of single strategies or interventions used in the community health initiatives; “high-dose” strategies, those that had high reach and high strength, were correlated with improvement in health behaviors. To measure the intensity, or dose, of multiple CPPs in a chronic disease prevention initiative, Collie-Akers et al.¹⁰ applied a weighting scheme to three key CPP attributes: behavior change strategy used, duration, and reach. The development of measures of intensity of community health initiatives—including those aimed at preventing childhood obesity—is a critical need in understanding the potential contribution of combinations of programs and policies occurring in communities¹³.

To address how community efforts may be related to childhood obesity, the National Institutes of Health initiated the Healthy Communities Study (HCS). The HCS examined the association of community programs and policies aimed at improving nutrition and/or increasing physical activity with markers of childhood obesity in a diverse sample of 130 U.S. communities ¹⁴. To measure the exposure of children to programs and policies, data were collected to identify instances of CPPs that occurred during the 10-year retrospective study period. The information on particular attributes, such as strength of behavior change strategy and estimated reach, was used to create CPP intensity scores. Consistent with recommendations that community health initiatives target multiple behaviors and use multiple strategies, two additional scores were calculated: a) a CPP target behavior score (i.e., measuring the number of different nutrition and physical activity behaviors targeted by CPPs), and b) a CPP behavior change strategy score (i.e., measuring how many different change strategies were used). The purpose of this study is to describe what was observed in these communities using these measures of intensity and targeting of CPPs addressing childhood obesity prevention.

Methods

Study design and community selection.

The HCS was an observational study involving current and retrospective data collection of child weight and height and CPPs. Data collection occurred in 130 communities between 2013 and 2015. For the purposes of this study, community (defined as a single high-school catchment area) was the unit of analysis. Selection of communities included a purposive sample of communities identified as having active interventions and a randomly-selected sample of communities stratified on the basis of socio-demographic characteristics^15,16.

Protocol for community measurement.

Using methods developed by the University of Kansas^{10, 17}, a community measurement protocol was designed to document, code, characterize, and calculate the intensity of the multiple CPPs identified in each community¹⁸. Although data collection occurred from 2013 to 2015, retrospective collection of CPP data allowed for CPPs to be documented for the prior 10 years. Supplemental figure 1 and supplemental table 1 outline this protocol for community measurement and provide examples to illustrate its use.

To identify and document instances of CPPs, trained field data collectors conducted interviews with an average of 11 key informants (KI) per community (range 4–15). KIs, such as school principals or parks and recreation directors, were selected from different sectors. They were identified for their knowledge of community efforts to address childhood obesity and their time spent living and/or working in the community. Interviews were conducted using scripted prompts to gather information intended to develop a complete list of CPPs in each community. Also, document abstraction was used to identify instances of CPPs from archived reports, web searches, and other written sources.

For a reported activity to be coded as an instance of a CPP, it had to meet five specified criteria outlined in the coding definitions and scoring instructions¹⁸: (1) it occurred (or was initiated) during the 10-year study period; (2) it was a program, policy, or other change to the environment that was implemented or occurred in the community during the study period; (3) it was related to nutrition, physical activity, and/or weight control/prevention of childhood obesity; (4) it targeted or benefited children ages four to 15 years of age; and (5) it occurred in or benefited children in the defined community. Research staff scored each documented activity as an instance (or not) of a CPP, and KU staff independently scored a randomly-selected set of 10% of identified CPPs as part of quality assurance procedures. Inter-observer agreement was assessed for all variables coded or recoded, and calculated inter-observer agreement proportions were >0.80 for all variables. Each coded instance of a CPP was further characterized by attributes used in intensity scoring (see below) as well as other attributes (e.g., goal addressed—whether nutrition, physical activity, both; target behavior addressed).

The researchers used scoring instructions to implement this measurement approach. A set of activities implemented at the same time and in the same setting was coded as one instance of a community program or policy (e.g., a Safe Routes to School intervention that included a Walk to School kickoff event, posters, etc.). To represent an instance of a CPP over time, a program or policy, such as an expanded bike path, was coded as one instance of a CPP, and it was represented as occurring in each of the years in which it was present in the environment. To add up CPPs for a community, each instance (e.g., an elementary school presentation about healthy eating and PA) was added to all others to calculate a combined total of CPPs for that year.

Methods for calculating intensity.

Several methods for estimating the intensity or potential contribution of identified CPPs were explored, each using a somewhat different combination of attributes and related formulae. The original intent of the study was to use a single intensity score for each CPP (aggregated for the community) to examine potential associations between CPPs and weight status of children. The overall community intensity score is based on the sum of the CPPs for a given community in a given year. Two other targeting indices—a CPP target behavior score and a CPP behavior change strategy score—were also calculated.

CPP Intensity Score (CPP-Int):

To characterize CPPs for intensity scoring, each coded CPP was scored for three specific attributes: (1) behavioral intervention strategy used, (2) duration of CPP, and (3) reach (i.e., the proportion of the total number of children in the catchment area estimated to be involved in or to have experienced the CPP). Each attribute was assigned a numerical value based on its relative strength (e.g., an environmental change that reached a higher percentage of the intended population received a higher weight than a small program reaching a few children).

Table 1 provides examples of CPPs observed in the study, the categorizations of attributes, and calculations used in intensity scoring. The values across all three attributes (i.e., behavior change strategy, duration, and reach) were averaged to create a single overall intensity score for each CPP. Using this method, each CPP documented could range in score from 0.1 (weakest and potentially of less influence on outcomes), to 0.55 (medium strength and influence), to 1.0 (strongest and potentially of greater influence). CPP intensity scores for distinct CPPs within a community were then summed for an overall community intensity score. The possible unstandardized scores for community intensity in a given year could range from zero (if no CPPs were documented) to the maximum number of CPPs (if all CPPs used a high behavior change strategy, long duration, and high reach). In its standardized form, the maximum value was found empirically by observing intensity scores across all communities¹⁶.

Table 1.

Distribution of community programs and policies (CPP) (n=9,681) by target behavior over the past 10 years among 130 communities in the Healthy Communities Study.

Targeted Behavior	Number (%) of unique CPPs (n=9,681)
Nutrition (n=4,107)
Increase consumption of fruits and vegetables	2826 (29.19)
Decrease consumption of high-calorie snacks, desserts, sweets, and candy	1782 (18.41)
Increase consumption of whole grain foods	1625 (16.79)
Decrease consumption of sugar sweetened beverages	1566 (16.18)
Increase consumption of water	1283 (13.25)
Decrease consumption of fat	1192 (12.31)
Decrease consumption of fast food	1116 (11.53)
Increase eating breakfast	830 (8.57)
Decrease calories from all food	729 (7.53)
Increase breastfeeding/improve infant health	137 (1.42)
Other	1110 (11.47)
Physical Activity (n=7,085)
Increase participation in community-based physical activity lessons, classes, or clubs	2697 (27.86)
Increase physical activity in after school programs	2594 (26.79)
Increase participation in home/family physical activity	2288 (23.63)
Increase participation in community-based sports teams	1460 (15.08)
Decrease TV watching	1417 (14.64)
Decrease time spent playing inactive video/handheld electronic games	1405 (14.51)
Decrease recreational computer/internet use	1325 (13.69)
Increase participation in school sports teams	1212 (12.52)
Increase physical activity during school recess or classroom instruction	1085 (11.21)
Increase exposure to physical education	733 (7.57)
Increase walking or biking to/from school	609 (6.29)
Increase moderate to vigorous physical activity in PE classes	535 (5.53)
Other	937 (9.68)

CPP Target Behavior Score (CPP-Behav):

Instances of CPPs were further characterized for attributes that might be associated with their contribution to addressing childhood obesity; that is, specific behaviors targeted for nutrition (e.g., increase consumption of fruits and vegetables) or physical activity (e.g., increase PA in after school programs). The target behavior objectives were identified by the scientific team, based on their expertise in each topic. As described in Frongillo et al.¹⁹, this targeting score was calculated using the total number of unique target behaviors across all CPPs in each community in each year. Each CPP was coded for one or more of 24 unique target behaviors (Table 1), 11 related to nutrition, and 13 related to physical activity. The possible scores in a given year could range from zero to 24.

CPP Strategy Score (CPP-Strat):

This score was calculated using the number of unique behavior change strategies across all CPPs in each community in each year. Each CPP was characterized for one or more of six different behavior change strategies: a) providing information and enhancing skills; b) enhancing services and support; c) changing consequences; d) modifying opportunities, barriers, and access; e) modifying policies and broader systems; and f) other. The possible scores could range from one to six.

Calculations of intensity and targeting scores were performed for CPPs occurring in each year of the 10-year study period using the documented year of onset and offset for each CPP. All scores for each year were standardized to a scale of zero to one (i.e., with zero being the minimum and one being the maximum). To permit a consistent interpretation, these continuous community-specific CPP intensity scores were standardized across all communities and all years.¹⁶

Analysis.

Data were summarized by compiling CPP community intensity scores over the ten-year study period. Onset and/or offset dates were missing for about 18% of the CPPs identified by key informants or document abstraction. To address missing data, the data set underwent multiple imputation by predicting the onset and offset dates of individual CPPs using community information²⁰.

Descriptive analysis was conducted to report on the presence and characterization of CPPs in 130 communities over the ten-year retrospective period. To explore whether or not the slopes of the CPP-Int, CPP-Behav, and CPP-Strat changed significantly over time, a mixed-model was used that allowed for community-specific random intercepts and slopes, along with a fixed-effect trend over time that aggregated results across all 130 communities for the overall trend.

Results

KI interviews (n=1,421) and document abstraction resulted in the identification of 9,681 distinct CPPs in the 130 communities over the 10-year study period. Of these, 2,596 (27%) focused on nutrition, 5,574 (58%) focused on physical activity, and 1,511 (16%) focused on both nutrition and physical activity. The mean number of CPPs per community was 74, with a range of 25–295.

CPP Intensity Scores (CPP-Int).

Among all CPPs (n=9,681), duration for the majority (62.9%) was scored as occurring more than once (medium) in a year, while 27.9% were scored as being ongoing or continuously present. In terms of reach, almost 80% of CPPs were scored as low or estimated to reach only 1–5% of the children in the catchment area. More than half (56.3%) of CPPs were assigned behavior change strategies that fell into the medium category (i.e., enhancing services and support or changing consequences), and 33.4% fell into the high category (i.e., modifying access, opportunities, or barriers; modifying policies and systems). For CPPs focused on physical activity, distribution of the key CPP attributes reflected the same patterns as for all CPPs (nutrition and physical activity combined), although the proportion in each of the categories was higher than for all CPPs. For nutrition-focused CPPs, the majority were medium in duration (57.4%) and low in reach (75.9%), and just over half (50.8%) scored in the high category for behavior change strategy (i.e., modifying access, opportunities, or barriers; modifying policies or systems).

Using these variables, a CPP intensity score was established for each of the 130 communities for each year of the study period. Table 2 contains information about the intensity score for all communities.

Table 2.

The standardized and unstandardized means and standard deviations per community for the community programs and policies (CPP) - Intensity Score, and CPP- Behavior Score, and CPP- Strategy Score, by study year and goal area in the Healthy Communities Study.

	CCP Intensity Score		Target Behavior Score		CPP Behavior Change Strategy Score
	Standardized	Unstandardized	Standardized	Unstandardized	Standardized	Unstandardized
	Mean(Std Dev)	Mean(Std Dev)	Mean(Std Dev)	Mean(Sth Dev)	Mean(Std Dev)	Mean(Std Dev)
Past 1 Year
Total	0.34 (0.16)	34.9 (11.5)	0.75 (0.25)	22.4 (1.64)	0.86 (0.35)	4.86 (0.35)
Physical Activity	0.35 (0.18)	23.97 (8.3)	0.88 (0.19)	12.28 (1.14)	0.85 (0.26)	4.69 (0.52)
Nutrition	0.38 (0.2)	15.75 (6.3)	0.85 (0.16)	10.1 (0.98)	0.83 (0.26)	4.66 (0.52)
Past 3 Years
Total	0.36 (0.18)	98.83 (32.36)	0.75 (0.24)	66.79 (5.01)	0.83 (0.33)	14.48 (1.03)
Physical Activity	0.34 (0.19)	68.54 (23.67)	0.87 (0.19)	36.61 (3.4)	0.83 (0.26)	13.97 (1.56)
Nutrition	0.34 (0.20)	44.07 (17.83)	0.84 (0.17)	30.13 (3.02)	0.80 (0.25)	13.81 (1.52)
Past 6 Years
Total	0.36 (0.19)	168.06 (57.29)	0.70 (0.24)	130.6 (11.04)	0.62 (0.24)	28.52 (2.15)
Physical Activity	0.35 (0.19)	119.83 (42.41)	0.82 (0.20)	71.35 (7.24)	0.66 (0.23)	27.27 (3.22)
Nutrition	0.30 (0.18)	71.95 (31.21)	0.81 (0.18)	59.06 (6.7)	0.63 (0.22)	26.80 (3.03)
Past 10 Years
Total	0.35 (0.19)	225.24 (80.22)	0.63 (0.24)	205.71 (22.77)	0.72 (0.25)	45.55 (4.06)
Physical Activity	0.35 (0.19)	165.41 (61.28)	0.73 (0.21)	112.93 (13.57)	0.71 (0.24)	42.94 (5.84)
Nutrition	0.27 (0.17)	91.99 (42.24)	0.73 (0.22)	92.43 (14.46)	0.63 (0.25)	41.75 (5.63)

CPP-Int varied considerably across communities and across years (Figure 1, panel 1). Most communities had a steady, modest increase in intensity of efforts to address childhood obesity over time. Some communities had steep and sharp increases in intensity, particularly between 2009 and 2011. Overall, there was a statistically significant (p <0.0001) increase in CPP intensity score over the ten-year study period.

Figure 1.

Distribution of unstandardized CPP-Intensity Scores; CPP-Behavior Score; and CPP-Strategy Score for each of 130 communities (gray lines) and average CPP intensity score (black line) over time during the 10-year study period.

CPP Behavior Scores (CPP-Behav).

As seen in Table 1, more of the nutrition-focused CPPs aimed to increase consumption of fruits and vegetables (29.2%) than other target behaviors. The physical activity-focused CPPs were primarily distributed among CPPs that aimed to increase participation in community physical activity opportunities (27.9%), physical activity in afterschool programs (26.8%), or physical activity at home or with family (23.6%).

Most communities expanded the number of target behaviors addressed over time, as captured in the CPP target behavior score (Figure 1, panel 2). For instance, one community began with only five behaviors targeted to be addressed (in 2006), but increased to targeting over 15 different behaviors related to nutrition and physical activity (by 2015). Beginning in 2009, it appears that many communities addressed more target behaviors. Over the 10-year study period, the range of targeted behaviors narrowed considerably, from 7–24 in 2006 to 17–24 in 2015. There was a statistically significant (p <0.0001) increase in CPP target behavior score—that is, more different behaviors related to physical activity and nutrition targeted—over the ten-year study period.

CPP-Strategy Score (CPP-Strat).

Figure 1 (panel 3) displays data for the annual CPP-Strategy Score for each of the 130 communities. In more recent years (2011 through 2015) all but a few communities were concentrated at the higher end of the available scores.

Overall, the standardized mean for CPP-Int of 0.36 did not change substantially over time; it generally ranged from 0.35 to 0.37 (Table 2). For CPP-Behav and CPP-Strat, the standardized mean for recent years was higher than for earlier years (Table 2). For the intensity score and two targeting scores, those for physical activity-focused CPPs were higher than for nutrition-focused CPPs. In addition, the CPP-Int scores, on average, were on the lower end of the possible scores, while the mean scores for the CPP-Behav and CPP-Strat measures tended to be on the higher end of the available scores.

Discussion

Communities varied greatly in the intensity and targeting of their efforts to address childhood obesity. The documentation, coding, and characterization of nearly 10,000 CPPs in 130 communities enabled examination of the dose (quantity and intensity) of childhood obesity efforts being implemented across the United States. Community efforts appear to be more comprehensive over time—to target more behaviors related to physical activity and nutrition. This is important, since the Healthy Communities Study investigators found that the CPP target behavior score was significantly associated with lower BMI in children¹⁹. The CPP intensity scores suggest that the intensity or dose of community efforts has generally grown over time. This is important since the HCS team found average BMI difference of 1.4 kg/m² (p-value<0.01) between communities with the highest and lowest observed CPP intensity scores²¹, with communities with higher CPP intensity scores having lower BMI averages than communities with lower CPP intensity scores.

This observational study found substantial increases in CPP activity, beginning near the mid-point of the study period (2009–2011). Potential explanations for this observed increase include two correlated events: a) recommendations and calls to action by agenda-setting organizations, including reports from the National Academies of Science and the U.S. Centers for Disease Control and Prevention; and b) subsequent increases in initiatives and investments in addressing childhood obesity by national and local grant-makers.

The science and practice of community health promotion requires a better understanding of what communities are doing to address important public health issues, which in turn requires a better measure of the “exposure” variable. Although the demonstrated association between intensity scoring and BMI in the HCS is encouraging, more research is needed to discover valid and cost-effective community measurement approaches. For instance, both RE-AIM and the population dose method compute a separate reach and strength/effect size measure and then multiply the two together, rather than averaging measures as was done in the HCS. Further exploration of these and other promising approaches can lead to: a) reliable capture and coding of community programs and policies, b) their characterization by key attributes (e.g., strength of strategy, reach, duration), and c) valid methods for measuring the intensity or “dose” of comprehensive community interventions. This study and further research can help us better understand what community and environmental conditions are necessary to achieve improvements in population health and health equity.

What is already known about this subject?

• Communities are implementing programs and policies to address childhood obesity.

• Little is known about how these programs and policies vary in intensity.

• Similarly, little is understood about how communities are focusing their efforts by targeting key behaviors or strategies.

What this study adds?

• This study describes the creation of three types of indices for understanding how community programs and policies may contribute to prevention of childhood obesity.

• Across 130 communities in the United States, implementation of community programs and policies has varied considerably.

• Findings suggest that, over the last 10 years, intensity and the targeting of specific behaviors and strategies have grown steadily.

Abbreviations:

HCS

Healthy Communities Study

CPP

Community programs and policies

CPP-Int

CCP Intensity Score

CPP-Behav

CPP Behavior Score

CPP-Strat

CPP Strategy Score