Strength and Comprehensiveness of District School Wellness Policies Predict Policy Implementation at the School Level

Abstract

Background

In 2006, all local education agencies in the United States participating in federal school meal programs were required to establish school wellness policies. The aim of this study was to document the strength and comprehensiveness of one state's written district policies using a quantitative coding tool, and test whether the strength and comprehensiveness of the written policy predicted school level implementation and practices.

Methods

School wellness policies from 151 Connecticut districts were evaluated using a quantitative coding system. In each district, school principal surveys were collected before and after the writing and expected implementation of wellness policies. Socio-demographic variables were assessed for each district, including enrollment, population density, political climate, racial composition and socio-economic status. Changes in school-level policy implementation before and after the federal wellness policy requirement were compared across districts by wellness policy strength, and policies were compared based on district-level demographic factors.

Results

Statewide, fuller implementation of nutrition and physical activity policies at the school level was reported after adoption of written policies in 2006. Districts with stronger, more comprehensive policies were more successful in implementing those policies at the school level. Some socio-demographic characteristics predicted the strength of wellness policies; larger, urban districts and districts with a greater ratio of registered Democrats to Republicans wrote stronger policies.

Conclusions

Written school wellness policies have the potential to promote significant improvements in the school environment. Future regulation of school wellness policies should focus on the importance of writing strong and comprehensive policies.

The Child Nutrition and Women, Infants, and Children Reauthorization Act of 2004 (Public Law 108-265)¹ required all local education agencies participating in federal food programs to create a school wellness policy by the 2006-2007 school year. This legislation required policies to include: goals for nutrition education and physical activity to promote student wellness; nutrition guidelines for all foods available on each school campus during the school day; an assurance that reimbursable school meals follow federal law; a plan for measuring implementation of the policy; and the involvement of parents, students, the school food authority, school board, school administrators, and the public in the development of the policy.

Recent research on the impact of implementing school food policies has shown some promising results. Studies have documented locations where wellness policies have been responsible for increased availability of fresh produce and lower-fat meats, ² decreased availability of candy, high-fat baked goods and sugar-sweetened beverages ^{3, 4} and a perception at the district level that competitive foods have become healthier as a result of the wellness policy. ⁵

While success stories have emerged, it is unclear whether wellness policies have resulted in systematically improved nutrition and physical activity environments in most U.S. school districts; the quality of policies is potentially an important factor. Since 2006, several studies have documented the substantial variability in the quality of written policies. ^6-11 Chriqui and colleagues examined a nationally representative sample of school districts over several years and concluded that while the majority of districts created written policies that addressed the required elements, there was striking variability across policies; many were underdeveloped, fragmented, and lacked sufficient plans for implementation and monitoring.¹² In a coding system that rates districts on a scale of 1-100, average wellness policy strength increased from scores in the low 20s to the high 20s from school years 2006-07 to 2007-08. A follow-up study of policy strength from the 2008-09 school year found continued improvement: the average policy strength was 33 (out of possible 100 points).¹³ Thus, despite improvement, scores remain very low.

Even the best written policies, however, are effective only if implemented.¹⁴ To date, the relationship between the written policy and the implementation of that policy is unclear. It is possible that some districts are hesitant to put plans down in writing, but in fact have made substantial changes in practice. At the same time, other districts may have written very extensive and progressive policies, but have not followed through in actually making the changes in schools. The objective of establishing a written policy is to create a standard against which to hold the school community accountable for making changes that may take effort and commitment. If the policy is written with clear and strong language, it may have a better chance of being implemented as intended than would policies that are written in weak or vague language. Therefore, we hypothesized a positive relationship between the strength of the school wellness policy and the likelihood of meaningful implementation of changes.

The aim of this study was twofold. First, the relationship between written district school wellness policy quality and school-level policy implementation as reported by school principals was assessed to test whether stronger written policies predict better implementation of nutrition and physical activity policies. Second, the relationship between district socio-demographic variables and the quality of the district's school wellness policy was assessed to test whether districts with particular characteristics produced stronger or weaker policies.

Methods

Three data sources contributed to this study: (a) district wellness policies, which were coded using the Wellness School Assessment Tool (WellSAT-96), (b) data regarding school practices, which were gathered via a survey completed by school principals, and (c) district demographic data, which were obtained from public sources.

Measures and Sample

District wellness policy scores

The Commissioner of the Connecticut State Department of Education (CSDE) requested that all Connecticut (CT) school districts provide copies of their written school wellness policies and procedures, as approved by the local board of education. Included in the study sample were all public school sponsors in CT that represented school districts or single schools and that participated in the National School Lunch Program (NSLP) or other federal school meal programs (N=151). Private schools, charter schools, the CT Technical High School System, and residential child care institution sponsors were excluded from the study due to lack of demographic data and lack of geographic representation. After several reminders from the CSDE, all 151 districts had submitted a policy.

Instruments

All policies were coded by trained researchers using the 96-item quantitative assessment tool called the Wellness School Assessment Tool (WellSAT-96).¹⁵ This coding system has been adapted for use in multiple studies around the country on the impact of school wellness policies.^12,16,17. It provides a score from 1-100 for the comprehensiveness and strength of the school wellness policy overall, as well as comprehensiveness and strength scores for the following subscales: nutrition education; nutrition standards for the United States Department of Agriculture (USDA) meal programs; nutrition standards for competitive foods; physical education; physical activity; communication and promotion; and evaluation. Each of the 96 items were scored on a scale from 0 – 2, where 0 represented no mention of the item in the wellness policy, 1 represented mention of the item in weak or vague language (e.g., “Vending machines should include items which are healthful”), and 2 indicated a strong and specific policy (e.g., “All items sold through vending machines shall contain no more than one serving per package, no more than 35% of calories from sugar, and no trans fat”). An abbreviated online version of this tool is also available.¹⁸

School nutrition and physical activity practices survey

In the spring semesters of the 2005-2006 and the 2006-2007 school years, a two-page questionnaire was mailed to a sample of school principals to assess the degree to which their school had implemented specific nutrition and physical activity related policies. One principal from each school level found in the district – elementary, middle, and high school – was randomly selected from each of the 151 districts. Our final sample consisted of 383 principals. The items addressed nutrition education, school food practices, physical education and physical activity, communication and promotion, and coordinated school health. In addition, respondents were asked to indicate whether or not the school experienced specific barriers to promoting a healthy school environment. Barriers fell into five categories: lack of support, lack of coordination, lack of resources, sales of unhealthy food, and lack of training.

Items on this survey reflected the policy areas assessed and coding system used in the WellSAT-96. Each item on the survey was coded on a scale of 0-2. If the principal noted that the policy was “fully in place,” the item was coded as a “2.” If it was either “partially in place” or “under development,” it was coded as “1.” If the response was “not in place,” or “don't know” it was scored a “0.” “Not applicable” items were excluded. Total practices strength scores were calculated by determining the percentage of items that obtained a “2” (i.e., were rated as “fully in place” by the principals). Scores for total number of practices addressed were calculated by determining the percentage of items that obtained either a “1” or “2” (i.e., were “under development,” “partially in place,” or “fully in place”).

During spring 2005-2006 (Time 1), we obtained a response rate of 70% (N = 269). During spring 2006-2007 (Time 2), we experienced very slight attrition (a total of eight). The number of principals who completed surveys at both Time 1 and Time 2 was N = 261 (68% response rate).

District demographics

For each school district, demographic data were collected in 2006 from the CSDE's strategic school profiles and other state government sources. The demographic variables included: percentage of the population living below the poverty line; percentage of white students; median family income; total district enrolment; total district expenditure per pupil; percent of town budget devoted to education; population density; percentage of students passing the Connecticut Mastery Test (the state's standardized testing); number of food stores and restaurants in the district; and ratio of registered Democrats to Republicans. With respect to the latter, if the ratio was less than or equal to 0.8, the district was classified as primarily republican. If the ratio was 1.2 or higher, the district was classified as primarily democrat. A “mixed” district was defined as a ration between 0.8 and 1.2.

Data Analyses

The data were analyzed using descriptive techniques, paired and independent t-tests, and linear regression models (OLS). Since the 261 schools were clustered within 118 school districts, uncertainty of coefficients was adjusted accordingly by calculating robust standard errors to allow for intragroup correlation. Analyses were carried out in Stata 11.2 (StataCorp. 2009. Stata Statistical Software: Release 11. College Station, TX: StataCorp LP.)

Results

Table 1 presents the results from coding the written school wellness policies in all districts, as well as all district-level variables. The coding of the district wellness policies revealed substantial variability across districts

Table 1. School District Variables.

N=151	M	SD	Range
School Wellness Policy Coding
Strength Total Score	38.43	13.32	7.07	76.95
Comprehensiveness Total Score	55.09	14.45	15.05	88.64
Socio-demographic Variables
Ratio Democrats/Republicans	1.76	2.00	0.31	13.59
Percentage population below poverty line	5.09	4.25	1.23	21.20
Percentage white students (2005-2006)	79.06	23.17	6.40	99.10
Median family income (in 1000)	72.68	22.37	35.95	173.78
Total District Enrollment 2005-2006 (in 1000)	5.00	4.67	0.12	21.77
Total District Expenditure per pupil 2005-2006 (in 1000)	11.29	1.45	8.20	16.13
Percentage budget for education	62.40	9.21	35.69	80.90
Population density (Pop./sq. mi. in 1000)	1.40	1.71	0.05	9.00
Percentage passed Connecticut Mastery Test	49.85	16.68	13.50	78.17
Number of stores & restaurants in district	79.18	87.73	2.00	404.00

Predicting school wellness policy quality

Correlational analyses were conducted to assess the relationship between district socio-demographic variables and policy quality. The strength of the written policy was significantly positively correlated with the percentage of students eligible for free or reduced-price lunch (r = .21, p < .05), and the population density of the town (r = .19, p < .05). Similar relationships were observed between policy comprehensiveness and these same demographic variables. Political climate also predicted policy strength. In districts where the majority of voters were Democrats (N=75), the comprehensiveness scores were significantly higher than in the districts where the majority of voters were Republican (N=42) (56.1 vs. 49.6; t (115)=2.37, p=.019). Districts with an equal mix of voters (N=34) fell in between, with a mean policy strength score of 53.7.

School practices improved in the first year of school wellness policy implementation

The school practices surveys at Time 1 and Time 2 revealed a significant improvement in both the full implementation of policies, and the frequency of fully, partially, or under-development policies. At Time 1, the average percentage of policies that were fully implemented was 27% (SD=16). This increased significantly to an average of 40% (SD=18) at Time 2 (t(260)=8.696, p<.001). The frequency of indicating a policy was “Under Development”, “Partially Implemented” or “Fully Implemented” increased from 66% (SD=19) at Time 1 to 75% (SD=17) at Time 2 (t(260)=6.527, p<.001). Implementation rates were similar among elementary, middle, and high schools.

Principals were asked to report at Time 1 and Time 2 the barriers they experienced in turning policies into practices in their schools. The total number of barriers and the frequency of each category of barriers decreased from Time 1 to Time 2. Only one barrier category, “sales of unhealthy food,” decreased significantly between Time 1 and Time 2. At Time 2, insufficient staff to implement the programs and activities (35%), lack of a key point person (34%), and the time to plan and coordinate efforts (48%) remained the most frequently endorsed barriers.

School wellness policy quality predicts implementation

Regression analyses were used to address whether the written school wellness policy quality predicted the level of implementation reported by the principal. In the model testing full implementation of practices as the outcome, predictors were school wellness policy Strength, percent of health-promoting practices fully implemented at Time 1, perceived barriers to change, school level, and several district-level covariates. In the model testing any degree of implementation/development of health-promoting practices as outcome, predictors were school wellness policy Comprehensiveness, percent of health-promoting practices at any stage of development/implementation at Time 1, perceived barriers to change, school level, and district-level covariates. Table 2 shows the results from each step of the two models. Higher written school wellness policy Strength scores predicted significantly greater full implementation of health promoting practices at the school level. The relationship between school wellness policy Comprehensiveness score and any level of consideration or implementation of school practices was also significant in the full model. The only barrier categories that predicted lower levels of implementation were lack of coordination and lack of resources.

Table 2. Implementation of School Nutrition and Physical Activity Practices as Predicted by Written School Wellness Policy (SWP) Strength and Comprehensiveness.

	Percent of practices fully implemented at Time 2			Percent of practices under development, partially or fully implemented at Time 2
SWP Strength / Comprehensiveness	0.215**	0.180*	0.182*	0.154*	0.150*	0.153*
Full/Any Implementation of School Practices (T1)		0.320***	0.217**		0.350***	0.336***
Barriers
Lack of support			-0.402			0.647
Lack of coordination			-3.289***			-3.046**
Lack of resources			-2.068*			-0.257
Sales of unhealthy food			-2.267			-1.612
Lack of training			-1.161			0.962
Control variables
School level
School type (ref.: Elementary School)			.			.
Middle			1.387			-1.499
High School			-2.584			0.841
District level
Ratio Democrats/Republicans			0.788			-0.890
Percentage population below poverty line			0.033			0.341
Percentage white students (2005-2006)			-0.067			-0.073
Median family income (in 1000)			0.092			0.137
Total District Enrollment 2005-2006 (in 1000)			0.033			-0.809
Total District Expenditure per pupil 2005-2006 (in 1000)			-0.111			-0.419
Percentage budget for education			-0.103			-0.008
Population density (Population/square mile, in 1000)			-0.826			-0.071
Percentage passed Connecticut Mastery Test			-0.072			-0.215
Number of stores & restaurants in district			-0.029			0.011
Constant	31.639	24.215	45.905	66.876	43.966	62.355
R-Square	0.026	0.107	0.246	0.018	0.183	0.258
N	261	261	252a	261	261	252 a

Note. Standard errors adjusted for within-district correlation. Significance levels:

^*p<0.05,

^**p<0.01,

^***p<0.001

^aThe sample size for fully adjusted models is decreased slightly due to missing data on Connecticut Mastery Test scores.

Discussion

The federal requirement for all districts to write school wellness policies in 2006 serves as a natural experiment demonstrating what actions districts will take when encouraged, but not required, to adopt specific policies. The present study adds to the understanding of one state's experience with the development and implementation of district policies.

A commonly raised concern – that school wellness policies might increase disparities because wealthier districts may have more resources for policy development than districts serving lower-income communities – was not borne out. In fact, Connecticut data indicate that urban districts with higher rates of free- and reduced-price lunch eligibility developed significantly stronger policies than other districts. This suggests that urban districts may have taken this task more seriously, perhaps due to greater concern about elevated rates of obesity and other health concerns among their students or greater comfort with the role of the school district as a partner in ensuring the health of children. Other studies have found a similar relationship between policy strength and demographics of the student population, at least for some policy components.¹⁹

These data also showed that having a greater proportion of Democrats than Republicans in a district predicted stronger policies. This may reflect a predominantly Democratic political belief that government has a role in ensuring health for all, or perhaps a philosophical agreement with the notion of the importance of environment over personal responsibility in improving children's nutrition and physical activity levels. ^20,21 Other research on the link between political orientation and school wellness state legislation found that states with Democratic governors and legislatures not controlled by Republicans were more likely to introduce and enact state legislation supporting obesity prevention in schools. ²²

This study suggests that stronger district wellness policies are predictive of implementation of school-level policies. The best predictor of having desired practices in place at Time 2 was having a strong wellness policy, even when controlling for the implementation of those practices at Time 1. The coding system used in the present study distinguishes between strong policies that have clear, directive language and weak policies that make suggestions or are vague. These findings suggest that the wording of policies makes a difference: if policies are written with strong language, they are more likely to be fully implemented than if they are written with weak language.

Barriers that emerged as predictive of low levels of implementation included lack of coordination and lack of resources. These barriers are not easily eradicated due to the personnel, time and funds likely required to develop sound solutions. Encouragingly, some of the barriers that seemed problematic before the policies were written, such as the sale of unhealthy food and lack of support from the school food service staff, decreased significantly over the year that policies were implemented.

The present study has a number of limitations. First, these data were derived from a single state, so findings may not be generalizable to other states. However, as noted above, many of the key findings are consistent with national studies and studies from other states. Second, implementation of practices was assessed via self-report from school principals. It is possible that administrators may wish to appear compliant with the spirit of school wellness policies and may have exaggerated the degree to which their schools had improved. Other studies that assess the school environment via observation or triangulated reports will allow for more robust evaluation of school practices. Finally, these data represent changes to the school environment only one year out from the school wellness policy mandate. Changes may well continue to accumulate over time, and more recent data may suggest this to be the case.¹³ Studies on the long term impact of school wellness policies are needed.

Implications for School Health

As the USDA finalizes their new regulations on child nutrition programs, ²³ the role of school wellness policies, will continue to evolve. This study supports the importance of writing clear, strong policies and suggests that policy strength makes a difference in likelihood of implementation and improvement of practices. These conclusions support efforts on the part of state government and non-government organizations to continue to monitor the strength of the written policies, provide feedback to districts on how to strengthen their language, and continued evaluation of the implementation of the policies into practices at the school level. Smaller school districts in politically conservative areas may need additional encouragement and education about the importance of, and support for, improving the school wellness environment. Ideally, all states will implement an ongoing system to monitor the strength of school wellness policies in much the same way that they monitor other areas of school performance.

Human Subjects' Approval

This study was deemed exempt by Yale University's Human Investigations Committee.