RE-AIM Analysis of a School-Based Nutrition Education Intervention in Kindergarteners

Abstract

BACKGROUND:

Few nutrition interventions in kindergarten classes have been evaluated, and none has been tested for program effectiveness, implementation, and dissemination. Building a Healthy Me (BHM) is a nutrition intervention for kindergarteners that is classroom-based and includes a family component. This study evaluated the public health impact of BHM in California kindergarten classrooms using the RE-AIM (reach, efficacy, adoption, implementation, and maintenance) framework.

METHODS:

A quasi-experimental design assessed pre-to-post changes in nutrition knowledge, dietary consumption, and parent behaviors of 25 intervention classrooms (414 students, 264 parents); and postintervention differences in nutrition knowledge between the intervention classrooms and 4 control classrooms measured at postintervention only (103 students).

RESULTS:

Intervention students improved in knowledge of food groups and healthy breakfast/snack options, and scored higher than control students in food group knowledge at postintervention (ps < .05). Parents of intervention group children increased their use of food labels, and intervention group children increased intake of several healthy foods and decreased intake of candy and fried potatoes (ps < .05). The BHM program reached 41% of kindergarteners attending public schools in California, and teachers implemented most lesson material.

CONCLUSIONS:

The BHM program was effective, implemented with fidelity, and broadly disseminated, highlighting its potential public health impact for kindergarteners.

The prevalence of childhood obesity has risen substantially in the past 3 decades but has recently plateaued.¹ Childhood obesity tracks into adulthood and increases risk of negative health outcomes including diabetes, cardiovascular disease, and cancer.^2–6 An increased effort has been placed on intervening among young children in primary school settings,^7–9 with evidence suggesting that these strategies can be effective at improving children’s energy balance behaviors.^10–13 However, there is a lack of systematic research evaluating the effectiveness of nutrition interventions among preschoolers and kindergarteners.^12,14–18 Costs and implementation complexities make it difficult to evaluate large-scale nutrition interventions for this age-group.¹⁹ Limited evidence suggests that theoretically based nutrition education programs, particularly ones based on Social Cognitive Theory, may have increased program efficacy among preschoolers and kindergarteners.²⁰ In addition, there is preliminary evidence indicating that multicomponent interventions may have greater success than education-only interventions or programs altering single characteristics (eg, modifying serving stations).¹⁶ In particular, family activities are associated with increased success, but family components are underutilized in current research.^19,21,22

In addition to the small number of studies in this area, several reviews have highlighted methodological limitations in research evaluating nutrition interventions in kindergarten and preschool classrooms. For example, 1 review reported that the majority of multicomponent interventions in this age-group (ie, interventions combining classroom nutrition education with an additional behavior change strategy such as parent activities) failed to include measures of children’s knowledge to evaluate the effects of the education component, and nearly half of the interventions failed to mention a theoretical basis.¹⁶ Another review observed that only 4 of 14 studies were at least quasi-experimental in nature (ie, included some type of control group).¹²

Furthermore, no studies of nutrition education among preschoolers and kindergarteners have reported the necessary information for evaluating the overall public health impact of kindergarten interventions, particularly dissemination and implementation metrics.^23,24 Interventions that potentially yield large effects on the intended outcomes can have weak public health impact if they are difficult to implement and reach few people.^23,24

One approach that has been used to assess the overall public health impact of interventions, by assessing program effectiveness and dissemination, is called RE-AIM.^25–27 RE-AIM stands for reach (proportion of target population participating), efficacy (ability to alter target outcomes), adoption (proportion of target settings the program includes), implementation (quality of program administration), and maintenance (long-term impact and sustainability). Program efficacy can be considered in combination with program reach, adoption, implementation, and maintenance to determine the program’s overall impact.

This study utilizes the RE-AIM framework to evaluate the overall public health impact of the Building a Healthy Me (BHM), a theoretically grounded, multicomponent school-based nutrition intervention for kindergarteners, disseminated across the state of California. The objectives of the BHM program were to increase students’ nutrition knowledge, nutrition behaviors, and healthy dietary intake; and to improve nutrition-related parenting practices. This study evaluated efficacy of the BHM program in these domains through a quasi-experimental research design. Program reach, adoption, implementation, and dissemination were also evaluated to assess its overall potential for large-scale public health impact.

METHODS

Participants

The study utilized a dissemination sample to evaluate the reach, adoption, and maintenance components of RE-AIM, and an evaluation sample to evaluate the efficacy and implementation of the BHM program. The dissemination sample included all 7556 kindergarten classrooms (209,660 students) that ordered BHM materials for the 2013-2014 school year. The evaluation sample included a subset of 29 classrooms. A total of 25 of these classrooms participated as an intervention group, and were measured at pre- and postsurvey (414 student surveys and 264 parent surveys matched pre-to-post). The remaining 4 classrooms participated as a control group, and were measured at postsurvey only (103 student surveys, no parent surveys were collected). An a priori power analysis using G*Power software²⁸ indicated approximately 24 classrooms (with 26 students per classroom) would have 80% power to detect medium effect sizes (d = 0.23-0.28) for changes in student knowledge and parent-reported student dietary intake over time assuming large intraclass correlations (ICC = 0.12-0.15). The intervention group was selected from classrooms already planning to implement the BHM program. Classrooms were chosen to be spread throughout California and to match the range of socioeconomic status (SES) and ethnicities of kindergarten classrooms throughout the state. A total of 25 schools were chosen. One classroom from each school was selected to the intervention group. Teachers not agreeing to participate were replaced with another selected kindergarten classroom from that school. Eligible classrooms were ones in which (1) nutrition education had not yet been taught that year, (2) the class contained only kindergarteners, (3) the school was on a traditional academic calendar, (4) and the teachers could teach the BHM program during the required months. A total of 4 kindergarten classrooms that did not order the BHM program were recruited in March, 2014 as a control group. Figure 1 shows the participant flow.

Figure 1.

Participant Flow

To evaluate the statewide representativeness of schools participating in the evaluation sample, demographic comparisons using data from all California public schools were made. The ethnic breakdown for public school students in California during the 2013-2014 school year were: 53.3% Hispanic or Latino, 25% White/Caucasian, 11.1% Asian, 6.2% African American, and 4.4% were other ethnicities. The intervention group schools were comparable to these state-level demographics with 53.5% Hispanic or Latino, 18.4% White/Caucasian, 15.4% Asian, 6.6% African American, and 6.1% other ethnicities. The demographic breakdown of the control group schools was 88.3% Hispanic or Latino, 2.9% White/Caucasian, 1.1% Asian, 5.4% African American, and 2.2% other ethnicities. In addition, 55.8% of California public school students received free or reduced-price lunches (a marker of SES), compared with 63.8% of students from the intervention group schools and 80.9% of students the control group schools. Finally, 22.3% of California public school students spoke English as a second language, compared with 34.3% and 51.5% of students from the intervention and control group schools, respectively.

Instruments

Student survey.

Student surveys were administered to the intervention group during class immediately prior to (presurvey, October) and after (postsurvey, March) the delivery of the BHM program. The control group completed the postsurvey only in March. Students were separated when possible to reduce copying. Teachers read directions aloud and instructed students to choose answers they thought were best for each question. Dairy Council staff assisted in proctoring surveys. Spanish surveys were administered in small groups when necessary.

The student survey assessed child sex, age, and nutrition knowledge, and took approximately 20 minutes to administer. Nutrition knowledge was measured with 5 questions requiring students to classify foods into food groups (fruits, vegetables, milk, proteins, and grains), 2 questions measuring breakfast knowledge, and 2 questions assessing healthy snack knowledge. All questions were picture-based. Each food group knowledge question contained pictures of 5 foods or drinks, some belonging to that food group, and others that did not. Children were asked to circle the foods belonging to that food group. For example, the fruit group question presented pictures of cheese, grapes, watermelon, bananas, and carrots. Students received 1 point for each fruit correctly circled and each nonfruit not circled. Then 2 breakfast knowledge questions were asked. The first breakfast question required children to circle the healthier breakfast option from pictures of cereal and a doughnut. The second breakfast question required children to circle the number of days each week they think they should eat breakfast, from 0 to 7, with 7 as the correct answer. Finally, the 2 questions measuring healthy snack choices included 2 pictures each and required children to circle the healthier snack option (yogurt versus potato chips, and cheese versus French fries). Pilot tests revealed adequate 7-day test-retest reliability, indicating 59-86% of answers were consistent across measurements.

Parent survey.

Parent surveys were administered prior to and after the intervention for intervention group parents only. Parent surveys were completed by parents at home, and returned to teachers by students. Parent-child surveys were matched via a unique ID number assigned to each student that was written on the parent survey prior to survey dissemination. Surveys were available in English or Spanish, and took 15-20 minutes to complete. The survey assessed parent demographics (age, relation to child, primary language, and ethnicity), child demographics (sex, whether they receive free/reduced-price lunches, which meals their child typically eats at home), children’s usual dietary intake over the prior month, usual nutrition-related parenting practices in the home (parent role modeling, parent policies regarding foods, parent use of food labels), and parents’ awareness of and satisfaction with the BHM program. In addition, parent-observed changes in child nutrition behaviors during the BHM program were assessed at postsurvey only. All participants completing parent surveys are referred to as parents for simplification purposes, although some participants held different relationships to the student (eg, grandparents).

Children’s dietary intake was measured using an adapted version of the National Cancer Institute’s Dietary Screener Questionnaire (DSQ).²⁹ The measure contained 27 items, in which 3 items were added to the original DSQ (diet soda, yogurt, and red meat intake), and an item assessing coffee or tea was removed because kindergarteners were assumed to not typically consume coffee or tea. For each food item, parents reported how frequently in the past month their child ate or drank that food (ranging from 0 = “Never” to 9 = “6-7 times per day”). In addition to testing changes of intake over time for each food item, the DSQ includes algorithms for calculating intake of fiber (grams), calcium (milligrams), sugar (teaspoons), whole grains (ounces), dairy (cups), fruit-vegetables-legumes (cups), fruit-vegetables-legumes without fries (cups), and sugar from beverages (tsp; see²⁹). However, several assumptions were required to calculate these scores. It was assumed that children did not drink any coffee or tea, and that children who ate cereal had eaten a cereal with average nutritional profile compared with other popular cereals for this age-group. Fruit Loops™ was chosen because it met this criterion, and nutrition information for Fruit Loops™ was imputed for all children who ate cereal.

An adapted version of the Home Environment Survey was used to assess nutrition-related parental role modeling and nutrition policies.³⁰ Nutrition-related role modeling was assessed via 3 items measuring the degree parents (1) “Eat healthy snacks such as fruits, vegetables, nuts, milk, yogurt, or cheese,” (2) “Eat unhealthy snacks such as candy, cookies, chocolate, or chips,”(3) and “Choose sugared drinks such as soda, Kool-Aid™, Gatorade™, Sunny Delight™, fruit punch, or other fruit drinks that are not 100% juice.” Parents rated the items on 5-point scales with anchors “Always” and “Never.” The unhealthy snacks and sugared-beverage items were reverse-scored, and the items were averaged for analyses (α = .50). A total of 7 items assessed nutrition-related parenting policies, measuring whether (1) parents use food as a reward; (2) parents eat breakfast with their child; (3) parents eat dinner with their child; (4) families have scheduled meals; (5) parents allow their child to help prepare meals; (6) parents offer healthy snacks to their child; (7) and parents allow children to eat any snack without asking. Items were assessed on 5-point scales with anchors “Always” and “Never.” The degree parents allow children to eat any snack without asking and parents using food as a reward were reverse-scored and all items were averaged (α = .46). Parent use of nutrition labels was measured with a single item adapted from the Food Behavior Checklist asking how frequently parents use food labels when choosing foods to eat on a 5-point scale with anchors “Always” and “Never.”³¹ Observed changes in child nutrition behaviors were measured at postsurvey only. Parents indicated (yes or no) whether they observed changes in their child’s nutrition behaviors for 8 specific behaviors, or whether they observed no changes at all. The behaviors were (1) asking for breakfast more often; (2) wanting to help prepare meals; (3) singing songs about food; (4) naming colors of food groups; (5) showing interest in new foods; (6) discussing the importance of eating breakfast; (7) asking for healthy snacks (fruits, vegetables, nuts, milk, yogurt or cheese); (8) and talking about food group foods. Percentage of parents reporting changes for each behavior (and no changes at all) were calculated.

Finally, the postsurvey assessed parents’ awareness, implementation, and satisfaction of the BHM program. Awareness of the BHM program was assessed with 2 items measuring whether parents were aware of the BHM program (“Yes” or “No”), and whether parents remembered seeing the BHM booklet (“Yes,” “No,” or “Unsure”). Parents also reported whether they posted the program nutrition chart on their refrigerator (“Yes” or “No”), how much of the booklet information they talked about with their child (“None of it,” “A little of it,” “Some of it,” “A lot of it”), how much of the information in the booklet influenced their food choices (“None,” “A little bit,” “Somewhat,” “A lot”), and how much their child liked each of the 4 booklet activities (“Very much liked activity,” “Somewhat liked activity,” “Did not like activity,” “Did not do activity”).

Teacher log and teacher survey.

Teacher logs were completed by teachers throughout the program. Teachers reported in the log whether they completed each lesson, completed each lesson activity, adapted each lesson, and assigned lesson homework via “Yes” or “No” response options. Teachers also reported the helpfulness of each activity and student interest during each activity on a 5-point scale with anchors “Not at all” and “Very much.” Teacher surveys were completed by intervention group teachers at postsurvey only. Teachers reported their opinion of each BHM unit, including age-appropriateness of lessons (5-point scale, anchors “Definitely too young for them” and “Definitely too advanced for them”), their overall satisfaction with BHM materials (10-point scale, anchors “Very Dissatisfied” and “Very Satisfied”), and whether they planned to teach the BHM program the following year (“Yes,” “Not sure,” and “No” response options).

Procedures

Intervention.

The BHM program was developed by the Dairy Council of California to increase nutrition knowledge and healthy eating in kindergarteners, and is offered free of charge to public and private schools in California and for purchase elsewhere. The program was designed to align with the 2010 Dietary Guidelines and the United States Department of Agriculture’s MyPlate principles for nutritious eating, and was aligned with California’s Common Core Content Standards and the California and National Health Education Standards.^32,33 A goal of the program was to encourage children to eat healthy foods, rather than focusing on foods that should be avoided. The BHM program included 8 units implemented by classroom teachers between October and March of the 2013-2014 school year. Unit 1 provided an overview of the 5 food groups, units 2-6 focused on each food group, unit 7 focused on healthy snacks, and unit 8 focused on healthy breakfast. Each unit included 4 core learning activities, and up to 4 supplemental activities. Teachers were encouraged to complete at least 3 core activities for each unit. The BHM program also included a teacher’s guide with instructions and lesson plans, a poster of nutrition information, a box of food pictures, a student workbook, and family homework to build on concepts taught during the units. The curriculum was based on an integration of the Health Belief Model and Social Cognitive Theory.^34–36

Design.

The efficacy of the BHM program was tested through a quasi-experimental design with pre- and postsurvey measurements of intervention classrooms for child and parent reported measures, and a post-survey measurement of nonrandomized control classrooms for child-reported measures only. Primary outcomes tested were child knowledge (child-reported) and child dietary intake (parent-reported). Secondary outcomes were nutrition-related parenting practices (parent-reported), and parents observing changes in their child’s nutrition behaviors (parent-reported). Changes in children’s nutrition knowledge were evaluated in 2 steps. First, intervention group pre-post scores were compared to determine changes in knowledge over the course of the program. Second, intervention and control group differences in postsurvey knowledge were tested. Parent-reported child dietary intake and nutrition-related parenting policies were tested for pre-post changes in the intervention group only. The proportion of parents observing changes in their child’s nutrition behaviors was measured at postsurvey only, and therefore, only descriptive statistics are provided for this outcome. Presurveys were completed in October, 2013 and postsurveys were completed March, 2014. Study materials were pilot-tested in 3 kindergarten classrooms in September, 2013.

Implementation was assessed with teacher program log and parent postsurveys. Variables assessed via the teacher log included the proportion of teachers implementing each lesson and accompanying units, the proportion of teachers adapting unit material, and teacher-reported age-appropriateness of the materials. Variables assessed via the parent survey included awareness of the BHM program, whether parents saw the BHM booklet, whether parents posted the BHM nutrition chart at home, the degree to which parents utilized the nutrition booklet, and the degree to which children enjoyed the BHM family activities.

Reach, adoption, and maintenance were assessed via secondary analysis of data collected by the Dairy Council of California on public school kindergarten classrooms ordering program materials during the 2013-2014 and 2014-2015 school years. Variables assessed were the number and percent of eligible kindergarteners and classrooms that ordered BHM materials in 2013-2014 (reach), the percent of teachers who ordered materials for the first time (adoption), and the percent of teachers who reordered materials the following year (maintenance).

Data Analysis

An “intention-to-treat” analysis was conducted. Multilevel models evaluated the following aims: (1) changes in child nutrition knowledge occurring between the pre- and postsurvey in the intervention group; (2) differences in child nutrition knowledge between the intervention and control group at postsurvey, and (3) changes in parent-reported child dietary intake and parent nutrition-related behaviors occurring between the pre- and postsurvey in the intervention group. Various sample subsets were used for each analysis due to missing data. Multilevel models correct for violations of independence in regression analyses, and were necessary as the study utilized data from students clustered within classrooms and repeated measures of individuals over time. Multilevel models were tested in SAS PROC GENMOD and R version 2.15.1 using the nlme package with maximum likelihood estimation to reduce bias due to missing data. School-level SES, percent Hispanic, and student sex were included as covariates in analyses of intervention and control group postsurvey differences to control for potential differences in group demographics. These covariates were unnecessary when testing intervention group pre-to-postsurvey changes because they were expected to remain invariant across measurements.

RESULTS

Program Efficacy

Student survey.

The intervention group included 604 students assessed at presurvey and 603 students at postsurvey (414 pre- and postsurveys matched). The control group included 103 students measured at postsurvey. The intervention group was on average 5.4 years old (SD = 0.50) at postsurvey and approximately 49% female. The control group was on average 5.5 years old (SD = 0.50) and approximately 47% female. The control group had a higher proportion of Hispanic or Latino students compared with the intervention group (p < .01), and no other group differences were observed on demographic variables (ps > .05).

Intervention group changes in child nutrition knowledge from pre- to postsurvey.

Intervention group changes in nutrition knowledge are presented in Table 1. For each set of food group items, children were presented 5 images of foods and instructed to circle which foods belonged to that food group. Each image was scored for correctness (5 scores per food group question). Children significantly improved on correctly categorizing all food images over time, except for categorizing grapes as a fruit (83.3% versus 88.8%; p = .110). In addition, children significantly improved on identifying the healthy breakfast option and on indicating how often they should eat breakfast. Finally, children significantly improved in identifying healthier snack options for both questions (yogurt versus potato chips; and cheese versus French fries).

Table 1.

Intervention Group Pre- to Postsurvey Changes, and Postsurvey Intervention Versus Control Group Comparison in Nutrition Knowledge

	Items	Intervention Group		Control Group	Intervention Group Pre- to Postsurvey Changes B	Group Differences at Postsurvey B
		Pre % Correct	Post % Correct	Post % Correct
Fruits food group	Grapes	83.3	88.8	88.1	0.29	0.07
	Banana	73.2	84.9	80.2	0.56**	0.08
	Melon	84.1	91.6	92.1	0.53**	−0.50
	Cheese	85.3	93.4	89.1	0.72**	0.50
	Carrots	66.7	80.0	77.2	0.72**	0.05
Vegetables food group	Carrots	73.1	87.2	63.4	0.81**	1.42**
	Corn	68.5	84.4	67.3	0.84**	0.94**
	Peas	79.2	88.1	75.3	0.47*	0.70
	Orange	61.4	85.0	61.4	1.41**	1.20**
	French fries	76.3	89.0	71.3	0.97**	1.00*
Dairy food group	Cheese	61.6	93.5	61.4	2.03**	2.23**
	Yogurt	78.5	96.0	83.2	1.76**	0.82*
	Doughnut	56.3	82.8	54.5	1.36**	1.59**
	Beans	80.2	93.4	85.2	1.39**	0.77
	Tortilla	81.2	90.9	91.9	0.88**	−0.11
Proteins food group	Peanut butter	60.5	84.2	40.6	1.19**	2.11**
	Ham	76.8	87.4	78.2	0.65**	0.53*
	Beans	73.1	83.2	77.2	0.35*	0.73**
	Broccoli	71.2	89.8	69.3	1.13**	1.72**
	Chips	66.4	89.3	68.3	1.43**	1.31**
Grains food group	Cereal	85.8	94.2	86.1	0.71**	1.12*
	Pasta	37.7	58.3	27.7	0.87**	0.84**
	Tortilla	30.9	65.0	31.7	1.31**	1.30**
	Cookies	38.0	58.8	30.7	0.91**	0.87**
	Yogurt	68.7	93.0	80.2	1.55**	1.58**
Breakfast knowledge	Cereal versus doughnut	82.5	95.4	88.9	1.37**	1.00**
	Days/week	70.1	84.7	76.0	0.90**	0.37
Healthy snack knowledge	Yogurt versus chips	75.9	89.8	88.9	0.94**	0.02
	Cheese versus French fries	64.3	87.8	61.2	1.35**	1.81**

^*p < .05;

^**p < .01.

Percentages represent the proportion of students answering each item correctly. Analyses testing postsurvey differences of intervention and control groups controlled for sex, percent of students receiving free or reduced-priced lunches, and percent of students who were Hispanic. Means for percentage correct were calculated using matched-cases: intervention group N = 414; control group N = 103.

Intervention versus control group postsurvey differences in child nutrition knowledge.

Group differences in nutrition knowledge are presented in Table 1. Analyses controlled for sex, school-level SES, and school-level percentage of Hispanic students. For the 5 food group knowledge questions (25 images total), intervention group students more accurately categorized most images¹⁶ compared with control group students (ps < .05). No differences were observed for any of the 5 fruit food group images (ps > .05). In addition, no differences were observed for categorizing peas as vegetables (p = .18) or for categorizing tortillas and beans as nondairy (p = .76 and p = .10, respectively). Intervention group students were more likely to select cereal as the healthier breakfast option compared to a doughnut, but there was no difference between groups for correctly indicating the number of days students felt they should eat breakfast (p = .32). Finally, compared with the control group, intervention group students more accurately identified cheese as healthier than French fries; however, there was no group difference in identifying yogurt as healthier than potato chips (p = .95).

Parent survey.

There were 422 presurveys and 383 postsurveys collected, with 264 surveys matched across time-points. In addition, DSQ scores were calculated for 374 participants at pre- and 322 participants at postsurvey (differences due to item-level missing data). Approximately 89% of surveys were completed by mothers, 9% by fathers, and 2% by others (eg, grandparents). Most respondents were between the ages of 25-34 (46.2%) and 35-44 (39.3%), and 54.6% reported as Hispanic. Approximately 21% of parents completed the survey in Spanish.

Intervention group changes in child dietary intake and parenting behaviors from pre- to postsurvey.

Table 2 shows pre-post changes in parent-reported child intake of 27 food items, 8 DSQ composite scores, and nutrition-related parenting practices in the intervention group. Parents reported more green leafy or lettuce salad (p = .009), beans (p = .005), and brown rice or other whole grains (p = .005), and lower consumption of fried potatoes (p = .038) and candy (p = .011) at postsurvey compared with presurvey. No differences were observed for the DSQ composite scores, which included fiber, calcium, sugar, whole grains, dairy, fruit-vegetables-and-legumes, fruit-vegetables-and-legumes without French fries, and sugar from beverages. For the nutrition-related parenting practices, intervention parents reported using nutritional labels more often at postsurvey compared with presurvey (p = .002), but no changes were observed for parental role modeling (p = .211) or parental policies (p = .965).

Table 2.

Intervention Group Pre- to Postsurvey Changes in Parent-Reported Child Dietary Intake and Nutrition-Related Parenting Practices

	Intervention Group Parents
	Presurvey Mean	Postsurvey Mean	Test of Change Over Time B
Foods
Cereal	4.45	4.42	−0.01
Milk	6.00	5.94	−0.03
Soda	1.74	1.68	−0.01
Diet soda	0.19	0.22	0.02
Fruit juices	4.42	4.36	−0.08
Sweetened drinks	2.31	2.22	−0.15
Fruit	5.62	5.61	−0.07
Green leafy or lettuce salad	3.49	3.85	0.32**
Fried potatoes	2.73	2.59	−0.156*
Potatoes not fried	2.21	2.22	0.02
Beans	2.62	2.87	0.29**
Brown rice or other whole grains	2.35	2.72	0.37**
Yogurt	4.26	4.17	−0.04
Other vegetables	4.21	4.14	0.00
Salsa	1.55	1.65	0.01
Pizza	2.18	2.17	−0.05
Tomato sauces	2.45	2.48	0.00
Cheese	4.03	4.06	0.02
Red meat (beef, pork, ham)	3.47	3.60	0.13
Processed meat	2.67	2.60	−0.06
Other meat	4.10	3.89	−0.09
Whole grain bread	4.11	4.20	0.03
Candy	3.18	3.05	−0.24*
Doughnuts	2.23	2.23	−0.04
Cookies	2.60	2.63	0.05
Ice cream	2.52	2.57	0.04
Popcorn	1.75	1.94	0.11
DSQ composite scores
Fiber (g)	11.10	11.29	0.29
Calcium (mg)	672.49	663.12	−1.64
Sugar (tsp)	8.99	8.84	−0.25
Whole grain (ounces)	0.28	0.33	0.02
Dairy (cups)	1. 37	1.30	−0.05
Fruit, vegetables, and legume (cups)	1.86	1.91	0.03
Fruit, vegetables, legume, no fries (cups)	1.82	1.86	0.03
Sugar from beverages (tsp)	1.39	1.35	−0.06
Nutrition-related parenting practices
Food label use	3.47	3.64	0.19**
Parent role model	3.75	3.81	0.04
Parent nutrition policies	4.03	4.03	0.01

DSQ, Dietary Screener Questionnaire.

^*p < .05;

^**p < .01.

Means calculated using matched-cases: N = 264.

Intervention group changes in child nutrition behaviors at the postsurvey.

Finally, intervention group parent-observed changes in child nutrition behaviors were assessed at postsurvey only, and therefore, statistical changes over time were not testable. Approximately 47% of intervention parents observed changes in the frequency their child asked for healthy snacks (fruits, vegetables, nuts, milk, and yogurt or cheese), 41% observed their child wanting to help prepare meals more often, 35% observed their child talking about foods from the food groups more often, 30% observed their child exhibit more interest in new foods, 28% observed their child naming colors of food groups, 27% observed their child talking about the importance of eating breakfast, 24% observed their child asking for breakfast more often, and 21% observed their child singing songs about food. Only 10% of parents reported no changes in their children’s behavior.

Program Implementation

All 25 intervention group teachers completed teacher surveys. All teachers were female. Teachers’ median age was 47 years, with a median of 20 years teaching experience, and all teachers reported a bachelor’s (11 teachers) or master’s degree (14 teachers) as their highest degree obtained. The racial/ethnic composition of the teachers was: 4% African American/black, 8% Asian, 68% non-Hispanic Caucasian/white, and 20% Hispanic/Latino.

Most teachers implemented all 8 units. Four of the units were taught by all 25 teachers, 3 were taught by 23 or 24 out of 25 teachers, and 1 unit was taught by 22 out of 25 teachers. Overall, teachers implemented 86.5% of the 4 core activities from each unit and 30.5% of supplemental activities. In addition, 4 units included family homework, which teachers reported assigning 65% of the time. Most teachers reported changing none (65%) or less than half of the program materials (20%). Teachers rated the degree units achieved program objectives as 4.56 out of 5, and the degree their students were interested in program activities as 4.56 out of 5. Finally, 80% of teachers reported the age-appropriateness of program materials was “about right,” and 10% of teachers reported it was “somewhat too young,” 5% of teachers reported it was “somewhat too advanced,” and 5% of teachers reported it was “definitely too young.”

In addition, 93.8% of parents reported being aware their child was learning about nutrition in school, and 68.3% of parents reported seeing the BHM booklet. However, only 38.6% of parents reported posting the BHM shopping list at home. About half of the parents reported discussing either a lot of (21.2%) or some of (34.8%) the information in the nutrition booklet with their children, while fewer parents reported discussing a little of (32.6%) or none of the information (5.7%).

Reach, Adoption, and Maintenance

The program reached 209,660 kindergarteners attending California’s public schools during the 2013-2014 school year, constituting approximately 41% of eligible students (reach). The program was ordered by 7556 kindergarten classrooms in California (~30%). About 47.4% of teachers were new adopters of the program (adoption). In addition, 24 of 25 intervention teachers indicated they planned to reorder the BHM program the following year. Dairy Council records indicated that 60.7% of classroom teachers in California who used the program during the 2013-2014 school year reordered materials the following year (maintenance).

DISCUSSION

This study used the RE-AIM framework to evaluate a school-based nutrition intervention in 29 kindergarten classrooms throughout California. The quasi-experimental design tested (1) pre-post changes in nutrition knowledge of intervention classrooms and (2) postsurvey differences in nutrition knowledge between intervention and control classrooms. The study also tested intervention group pre-post changes in parent-reported child dietary intake, parents’ nutrition-related behaviors, and parents noticing changes in their children’s nutrition behavior during the BHM program.

Intervention group students improved in categorizing foods within the 5 food groups from pre-to-post-survey for nearly all of the items. The only item failing to show significant improvement was correctly categorizing grapes as fruit. However, 83.3% and 88.8% of intervention group students answered this item correctly at pre- and postsurvey, respectively, indicating that perhaps high initial scores limited variance necessary for detecting a significant change. In addition, intervention classrooms scored higher on 19 of 29 knowledge items at postsurvey compared with controls. Similar findings were observed in other studies evaluating kindergarten (and pre-school) interventions. For example, several studies reported changes in nutrition knowledge.^16,37,38 However, some of these previous studies tested a smaller subset of nutrition knowledge items,^38,39 or tested nutrition knowledge via one-on-one interviews with students,^37,39,40 as opposed to evaluating knowledge in a group setting as this study and others have done.^41–43 Other studies also observed changes in vegetable intake.^16,44,45 This study found intervention group children increased consumption of green leafy or lettuce salad, beans, and brown rice or other whole grains, and decreased consumption of fried potatoes and candy. These observed changes are difficult to compare to past research, as many previous studies focus primarily on fruit and vegetable consumption.¹⁶

Whereas the kindergarten and preschool nutrition intervention literature is limited in general, no study has used the RE-AIM framework to evaluate program implementation and dissemination in addition to efficacy. Implementation and dissemination are critical for evaluating the overall public health impact of nutrition programs.^23–25 This study was implemented well, as most teachers administered all 8 units, and most used unaltered materials. However, one third of teachers did not assign the homework, and some indicated they do not assign homework to their kindergarteners in general. This is a potential limiting factor to the effectiveness of a parent-child component in children of this age-group. More encouragement from study staff or strategies that reframe the parent-child component as “life-skills building” or “family nutrition activities” could potentially encourage more teachers to assign the activities. Dairy Council of California reports indicated the study was disseminated to approximately 30% of eligible kindergarten classrooms in California (representing approximately 40% of eligible students) during the 2013-2014 school year. Preliminary evaluations of program maintenance indicated nearly 61% of teachers reordered materials the following year, and secondary cost-analysis estimates the program cost approximately $2.28 per student to implement.

Strengths and Limitations

The study design benefited from an intent-to-treat analysis, a theory-based program (Social Cognitive Theory and Health Belief Model), and advanced statistical procedures (multilevel modeling;maximum likelihood estimation), which have been rare in comparable studies.^12,16,19 An additional strength of the program was that it included a family component via homework.^19,21,22 The quasi-experimental study design represents an improvement over common less rigorous research designs used by previous kindergarten and preschool nutrition interventions.¹² However, with this design, there are still some limitations in the extent to which causality can be attributed to the BHM program because control classrooms potentially differed from intervention classrooms in nutrition knowledge at presurvey, which could not be assessed. Another limitation is that the control and intervention groups were selected at different times. The control group was observed to have a higher proportion of Hispanic students than the intervention group, and this variable, as well as other demographics (SES and sex), were controlled for in analyses. Also, there was no parent control group, and parent-reports of child intake may be error-prone because parents may not be aware of their child’s consumption outside the house.¹⁶ Furthermore, all cereal reportedly consumed was assumed to be a single type in order to simplify analysis, which could add noise to the data. Nevertheless, it is challenging to accurately measure food intake in kindergarten-aged students, as methods are often costly or burdensome for children.^46,47 Another limitation is the relatively short time-frame (5 months) of the evaluation, as some changes may dissipate over time. There are also potential weaknesses to using curriculum reorder information as an indicator of adoption and maintenance, as this measure does not directly indicate which teachers actually used the materials. Future studies should include follow-up measurements to evaluate long-term changes in target outcomes. Finally, statistical analyses did not adjust for multiple comparisons and type I error.

Conclusions

This study evaluated the impact of a classroom-based nutrition education program for kindergarteners using a more rigorous research design (quasi-experimental with control group) than has been used in the past. Furthermore, it is one of the first known studies to evaluate overall potential for public health impact using the RE-AIM framework, which evaluates program implementation and dissemination along with effectiveness. The current RE-AIM analysis indicated that BHM was widely disseminated, generally implemented well, and showed evidence of improving children’s nutrition knowledge, healthy eating, and parents’ use of food labels.

IMPLICATIONS FOR SCHOOL HEALTH

Study findings support the utility of school-based nutrition interventions in kindergarten-aged children. Young childhood represents an important period to positively shape children’s nutrition behaviors before unhealthy habits can be established. The findings underscore the value of multicomponent school-based nutrition interventions that incorporate classroom and family activities which have been effective and advocated for in the literature.^19,21,22 Family components can extend children’s education, and disseminate information to parents who are particularly influential in terms of purchasing and preparing healthy food in the home. However, teachers should be adequately motivated to assign family homework, as one third of teachers did not in the current study. Future programs could benefit by expanding resources directed toward the family as a whole, and for testing strategies beyond “homework,” as some teachers indicated they do not assign any homework to kindergarteners. Future research also needs to evaluate the effectiveness of school-based nutrition education for children as young as kindergarteners through randomized-controlled studies, as they are necessary for inferring causality. Adequately sized control groups should be included when budgets allow. Existing evidence is limited, but trends appear to support the utility of theory-based, multicomponent nutrition interventions in kindergartens.

Human Subjects Approval Statement

All procedures were approved by the Institutional Review Board at Independent Review Consulting Inc, review number: 13101-01. A passive consent process was used, in which children whose parents did not give permission to participate were excluded. Parents signed a separate consent form along with the parent survey for their participation.