A systematic review of the implementation of obesity prevention interventions in early childcare and education settings using the RE-AIM framework

Obesity prevention interventions within early childcare and education settings underreport data useful to replicate, disseminate, and implement such interventions in real-world settings and to inform policy.

Abstract

Previous systematic reviews have examined the efficacy of obesity prevention interventions within early childcare/education settings. Often lacking in these reviews is reporting on external validity, which continues to be underemphasized compared to internal validity. More attention to external validity would help better translate evidence-based interventions to real-world settings. This systematic review aimed to determine the availability of data on both internal and external validity across dimensions of the Reach, Efficacy/Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework in studies reporting on obesity prevention interventions in early childcare/education settings. Inclusion criteria included: randomized controlled trials, early childcare/education setting, targeted children 2–6 years old, addressed both diet and physical activity, collected measures of weight status and diet and/or physical activity, and published within the last 10 years. Searches were conducted in ERIC, PsychInfo, and PubMed; 23 studies met inclusion criteria. A validated RE-AIM abstraction tool was used to code studies. Most commonly reported dimensions were Reach (62.3%), Implementation (53.5%), and Efficacy/Effectiveness (48.7%). Adoption (21.7%) and Maintenance (11.6%) were less often reported. All studies reported on primary outcomes, but few reported on RE-AIM indicators of characteristics of participation and adoption, quality of life, methods used to identify staff, staff inclusion/exclusion criteria and adoption rates, implementation fidelity, measures of cost to start-up and deliver the intervention, and indicators of maintenance. This systematic review underscores the need for more focus on external validity to inform replication, dissemination, and implementation so that evidence-based early childcare/education obesity interventions can be generalized to real-world settings.

Implications.

Practice: Obesity prevention interventions in early childcare and education settings should consider using the Reach, Efficacy/Effectiveness, Adoption, Implementation, and Maintenance framework for planning, implementation, and evaluation to maximize effectiveness and internal and external validity data collection and reporting.

Research: Standardized reporting methods for external validity should be developed and followed by researchers to inform future translation, implementation, and evaluation of obesity prevention interventions in early childcare and education settings.

Policy: Policymakers should utilize findings from efficacy and effectiveness trials to inform the development of evidence-based diet and physical activity policy guidelines in early childcare and education settings.

BACKGROUND

Preventing overweight/obesity early in life is important for reducing the lifetime risk of overweight, obesity, and obesity-related diseases [1]. In the USA, approximately 23% of 2–5 year olds are overweight or obese [2]. Although obesity rates have decreased within this age group, racial/ethnic and socioeconomic disparities persist [2]. One potential explanation for obesity-related disparities is that underserved children are not meeting recommended guidelines for diet and physical activity. Specifically, previous research demonstrates that preschool-aged children, particularly those from racial/ethnic groups and low-income communities, do not meet the United Stated Department of Agriculture dietary recommendations for fruits, vegetables, and whole grains and exceed the recommendations for energy dense foods and sugar-sweetened beverages [3–6]. Similarly, preschool-aged children do not meet recommended guidelines for physical activity, with only half engaging in the recommended 60 min of physical activity per day [7].

Childcare settings often may provide opportunities to improve diet and increase physical activity among at-risk preschool children. In 2016, 73% of U.S. preschool-aged (3–5 years old) children attended a childcare center, Head Start, preschool, prekindergarten, or other early childcare and education settings at least 1 day/week [8]. In 2017, approximately 56% attended full-day programs [9], and children of working parents spend almost 40 hr/week in a childcare setting [10]. Further, children who attend childcare typically consume 1–2 meals and snacks at school [11]. Therefore, early childcare and education settings represent an ideal environment for obesity prevention interventions [12].

Identifying successful obesity prevention interventions in this particular setting is necessary for widespread dissemination and implementation [13]. Several reviews have focused on the efficacy, or internal validity, of obesity prevention interventions among this age group and these settings in an attempt to provide evidence of a relationship between the intervention health and/or health behavior outcomes among participants [14–19]. Most of the interventions included within these reviews reported efficacy in improving weight status and/or obesity-related health behaviors among children, demonstrating the potential positive impact of obesity prevention in these settings [14–19]. However, often lacking in these reviews is the reporting of external validity, which continues to be underemphasized in contrast to internal validity. Increased attention to the external validity of these types of intervention is required for improved translation of evidence-based interventions to real-world settings.

The Reach, Efficacy/Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework is a useful tool for understanding the development, delivery, and evaluation of health interventions within the context of external validity [20,21]. The framework assesses the reach of the target population, efficacy/effectiveness of the intervention in achieving desired outcomes, adoption and implementation of the intervention at the desired setting, and maintenance of change by participants and settings [21]. The framework can be used to translate research into practice by examining reported issues related to the design, dissemination, and implementation process [22]. Therefore, the RE-AIM framework may help strengthen the evidence base for obesity prevention interventions in early childcare and education settings.

Objective

The primary objective of this study was to determine the availability of data on both internal and external validity across dimensions of the RE-AIM framework in studies reporting on obesity prevention interventions in early childcare and education settings.

METHODS

This systematic review adheres to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [23] and is registered with PROSPERO.

Inclusion and exclusion criteria

The search protocol was developed using the Participants, Interventions, Comparisons, Outcomes, and Study Design framework for systematic reviews [24]. A study was included if it was peer reviewed, published within the past 10 years, administered in an early childcare and education setting, and a randomized controlled trial, if it targeted children 2–6 years old and included both diet and physical activity components and its measured outcomes included weight status and diet and/or physical activity, and if published in English.

Search strategy and screening

Searches were conducted in three databases (ERIC, PsychInfo, and PubMed) in July 2019; search terms and methods are available in an appendix (Supplementary Material 1). Search strings corresponded to the following five terms: (a) obesity, (b) early childcare and education setting, (c) diet, (d) physical activity, and (e) randomized controlled trial. The search was then filtered to the past 10 years. The search protocol was discussed within the study team and reviewed by an experienced subject librarian. Figure 1 illustrates the search strategy and screening process in more detail. Initial searches returned 831 results. After the removal of duplicates, 745 results were imported into RefWorks for title and abstract review. Five reviewers (J.C.S.-F., A.H., J.B., A.K. and A.B.) independently screened records against eligibility criteria in two phases: (a) title and abstract and (b) full text. Any discrepancies were resolved by an additional reviewer (M.L.F.). Upon completion of title and abstract screening, 707 records were excluded resulting in 38 full-text articles to be assessed for inclusion. Upon completion of full text of screening, 15 were excluded resulting in 23 studies to be included.

Fig 1.

Study selection flow diagram.

Data extraction, synthesis, and analysis

A validated data extraction tool based on the RE-AIM framework [25–28] was used to code studies on the degree to which indicators of internal and external validity were reported (Supplementary Material 2). The coding sheet includes a series of “yes” or “no” questions, in addition to space to include data, for each of the five indicators within the RE-AIM dimensions. Reach items include the method to identify the target population, inclusion and exclusion criteria, participation rate, and characteristics of participants and nonparticipants. Efficacy/Effectiveness items include primary outcome measure(s), analysis procedure (intent-to-treat analysis or present at follow-up), imputation procedures, quality of life measure(s), and attrition. Adoption items include methods to identify delivery agent(s), level of expertise of delivery agent(s), inclusion/exclusion criteria, participation rate, characteristics of adoption and nonadoption, and measures of cost. Implementation items include intervention type and intensity, extent protocol was delivered as intended, and measures of cost. Lastly, Maintenance items include assessment of outcomes longer than or equal to 6 months postintervention, whether the intervention was still in place, and whether the intervention was modified. In addition, the coding sheet was used to extract data on study design. Each study was coded by J.C.S.-F. and three authors (A.H., A.K., and A.B.) served as second coders. Extracted data were compared and discrepancies were resolved by a third coder (J.B. or M.L.F.) who coded studies only for indices where the discrepancy was identified. After coding completion, frequencies and percentages for each of the RE-AIM indicators were calculated for each individual study. The average reporting of RE-AIM indicators and dimensions across studies was determined using the percentages for each of the RE-AIM indicators.

RESULTS

Study design characteristics, including a brief description of intervention strategies, study duration, and outcome measures, are reported in Supplementary Material 3. Of the 23 studies, the majority were conducted in the USA (n = 12) [29–41] and others were conducted in Israel (n = 3) [42–44], Spain (n = 2) [45,46], Switzerland (n = 2) [47,48], Australia (n = 1) [49], Belgium (n = 1) [50], Italy (n = 1) [51], and Sweden (n = 1) [52]. While all studies included diet and physical activity intervention components, three studies only reported on weight status and dietary outcomes [31,33,36] and three studies only reported on weight status and physical activity outcomes [30,42,47]. Two of the studies included are of the same intervention and group of participants but report on weight status and physical activity outcomes separately [37,38]. The proportion of studies reporting on RE-AIM dimensions and indicators are reported in Table 1 and the summary of RE-AIM characteristics for studies included are reported in Supplementary Material 4.

Table 1.

Proportion of obesity prevention intervention studies in early childcare and educational settings reporting Reach, Efficacy/Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) indicators and components

RE-AIM dimensions and components	Number reporting (N = 23)	Percentage reporting (N = 23)
Reach
Method to identify target population	21	91.3%
Inclusion criteria	19	82.6%
Exclusion criteria	6	26.1%
Sample size	23	100%
Participation rate	15	65.2%
Characteristics of participation and nonparticipation	2	8.7%
Average of overall Reach dimension	14.3	62.3%
Efficacy/Effectiveness
Measures and results for at least one follow-up	23	100%
Intent-to-treat analysis utilized	10	43.5%
Imputation procedures	5	21.7%
Quality of life measure	3	13.0%
Percent attrition	16	69.6%
Average of overall Efficacy/Effectiveness dimension	11.2	48.7%
Adoption
Methods used to identify staff	3	13.0%
Description of staff and level of expertise or training provided	22	95.7%
Inclusion/exclusion criteria for setting and staff	2	8.7%
Adoption rate	2	8.7%
Characteristics of adoption/nonadoption	1	4.3%
Measures of cost (start-up)	0	0%
Average of overall Adoption dimension	5	21.7%
Implementation
Intervention type, frequency, and intensity	23	100%
Extent protocol delivered as intended (%)	6	26.1%
Measures of cost (delivery)	8	34.8%
Average of overall Implementation dimension	12.3	53.6%
Maintenance
Assessed outcomes ≥6 months postintervention	8	34.8%
Current status of intervention	0	0%
Cost of maintenance	0	0%
Average of overall Maintenance dimension	2.7	11.6%

Reach

The overall proportion of reported indicators of RE-AIM dimensions was highest for Reach at 62.3%. Information related to internal validity, such as sample size and inclusion criteria, were the most commonly reported. All studies (100%) reported on sample size. Studies often reported the sample size for both the early childcare and education setting (range: 3–40) and child participants (range: 101–2062). The method to identify target population (91.3%) and inclusion criteria (82.5%) followed as the most commonly reported Reach indicators. In contrast, exclusion criteria was less often reported (26.1%). There was mixed reporting in whether the inclusion criteria referred to the setting, participants, or both. The least common Reach indicator reported, which is related to external validity, was characteristics of participation and nonparticipation (8.7%). Studies reporting on characteristics for nonparticipation reported these data at the setting level; common characteristics reported were time constraints, health problems of the teacher, and refusal or lack of interest to participate in the study [31,48,52]. Over half of the studies (65.2%) reported on participation rate, although this varied widely across studies as to whether a participation rate was included for the setting, the participants, or both. Of the participation rates reported, the range was 25%–100% for setting and the range was 49%–98% for participants [29,31–34,36,38,41,45,48–52].

Efficacy/Effectiveness

The overall proportion reporting for Efficacy/Effectiveness was 48.7%. Similar to the Reach dimensions, studies demonstrated strong reporting on indicators associated with internal validity. These indicators include primary outcome measure(s), including behavioral outcomes, in all (100%) studies. Less than half of the studies found a significant difference between the intervention and control groups in weight status outcomes, such as body mass index z-score (BMI z-score), waist circumference, percent body fat or sum of skinfolds [29,33,42,44–46,48–50]. In terms of behavioral outcomes, less than half of studies reported a significant difference between the intervention and control groups for dietary-related behavior, knowledge, or policies [29,31,32,34,43,44,48,49,52] and less than half of the studies reported a significant difference between groups for physical activity-related behavior, knowledge, or policies [29,37,41,42,44–48]. Percent attrition was reported on over half (69.6%) of the studies and was reported at the setting or participant levels or both. Many studies provided background information to their attrition rates at both the setting and participant levels. For example, a common reason for attrition at the participant level was that the child no longer attended the early childcare and education program or was absent the day of data collection [30,45]. The next set of Efficacy/Effectiveness indicators that were commonly reported were the use of an intent-to-treat analysis (43.5%) and imputation procedures (21.7%). Quality of life, including satisfaction, an external validity factor, was reported in three studies (13%). In addition, four studies reported that there were no adverse events for participants as a result of the intervention [40,42,48,52].

Adoption

The average proportion for reporting of the Adoption dimension across all studies was 21.7%. The description of the staff who delivered the intervention and their level of expertise or training provided was most commonly reported (95.7%). All but one study reported utilizing early childcare and education program staff, primarily teachers, to deliver the intervention. Therefore, information regarding teacher training was often reported. The one study that did not use early childcare and education program staff reported using trained childcare health consultants [29] to deliver the intervention. Some studies reported using outside professionals, such as registered dietitians, in addition to early childcare and education program staff, to assist with intervention delivery [32,34,36,42–44,48]. Methods used to identify staff to deliver the intervention (13%), inclusion/exclusion criteria for staff (8.7%), adoption rate (8.7%), and characteristics of adoption and nonadoption (4.3%) were least likely to be reported. No study (0%) reported on measures of cost to start up the intervention.

Implementation

The average reported proportion of Implementation indicators was 53.6%. All studies (100%) reported on the intervention type, frequency, and intensity. Most common intervention approaches included classroom-based lessons on diet and physical activity. Many studies often referenced a separate intervention design or protocol paper [32,33,37–41,46–49], suggesting readers refer to that paper for additional details on intervention type, frequency, and intensity. Less than half of the studies (26.1%) reported on the extent protocol was delivered as intended. These data were often reported as the proportion of overall lessons completed by early childcare and education program staff [30,32,41,48,52]. A few studies reported that data were collected on the extent to which the protocol was delivered as intended; however, outcomes of these data were not reported [33,43,44]. Less than half of the studies (34.8%) reported on measures of cost for delivery; however, these studies mainly reported on incentives provided to the setting or participants [30–32,36,39–41,49,50].

Maintenance

The average reported proportion of Maintenance indicators was lowest (11.6%) across all studies. Less than half of studies (34.8%) reported on outcomes, including behavioral outcomes, assessed at longer than or equal to 6 months postintervention. Of these studies, one reported significant changes in weight status between intervention and control groups [39] and three reported on differences in dietary or physical activity behavior [39,51,52]. Two studies reported no changes in weight status [34,40]. No studies (0%) reported on the current status of the intervention or the cost of maintenance.

DISCUSSION

To our knowledge, this is the first systematic review of obesity prevention interventions targeting diet and physical activity behaviors in early childcare and education settings using the RE-AIM framework. As illustrated in this review, the current methods used to conduct, report, and evaluate obesity interventions in early childcare settings makes it difficult to translate the findings into “real-world,” nonresearch settings. Traditionally, the emphasis is placed on internal rather than external validity. However, the focus on internal validity is not unique to the study of obesity prevention but rather pervasive across the field of public health [13,53,54]. The underreporting reflects a gap between current obesity intervention research and the emerging fields of dissemination, implementation, and translational science [55]. To move the obesity prevention field forward in terms of public health impact, practice, and policy, future research should implement more standard methods for more transparent reporting of original research [56] to systematically identify areas that could improve intervention dissemination, generalizability, and translation of findings to nonacademic settings. One method to increase transparent and standard reporting is to use the RE-AIM framework during intervention planning. Recent intervention studies, utilizing the RE-AIM framework during the planning phase, successfully collected, reported, and disseminated information on both internal and external validity within each RE-AIM dimension [57–59]. One commonality within these studies is the use of mixed-methods approaches to collect internal and external validity data [57–59].

While demonstrating intervention efficacy and effectiveness is vital to evidence-based practice, insufficient reporting of factors, such as Reach or Adoption (based on RE-AIM framework), which supports the translation of evidence-based intervention in real-world contexts, is currently lacking [60]. From a health equity perspective, issues may arise in assessing whether an intervention reached the population that may have needed the intervention the most [61]. Furthermore, it may be difficult to determine the applicability or relevance of an intervention to one’s own target population or setting without gathering these data [62].

The studies varied in the degree to which indicators in the Reach and Adoption dimension issues were reported, with particularly limited reporting in the Adoption dimension. To address issues in reporting on Reach and Adoption, an expansion to the current CONSORT figure [63] has been proposed [64]. This expanded CONSORT figure, which is informed by the RE-AIM framework, would allow for the inclusion of data on participation, nonparticipation, and representativeness at the levels of the participant, setting, and staff, in addition to data on intervention maintenance [64]. The expanded CONSORT figure supports comprehensive quantitative data while also providing space to summarize key qualitative data, which can be supported by additional discussion within the text, an appendix, or a companion article [64]. Providing details within the Reach and Adoption dimensions allows for additional understanding of important issues surrounding context, which supports the planning, replication, implementation, and evaluation of evidence-based interventions [64–66]. This is particularly important for early childcare and education settings where there may be variations in the number of staff [67], resources, and/or policies [68,69] available to either support or hinder obesity prevention interventions, which then may impact adoption. Additionally, there may be multilevel contextual issues [70] that may affect adoption. For example, early childcare centers are usually nested within a larger organization, such as a public-school system. This context may involve constraints to intervention implementation that goes beyond the early childhood center. To gain additional data on such contextual details, quantitative and qualitative approaches could be utilized [71]. Such mixed-methods data could provide useful information surrounding adoption or nonadoption and assist researchers in translating research into practice. In terms of increasing adoption, detailed protocols and guides [22], in addition to technical assistance including auditing and feedback [72], should be provided to settings and staff to assist in intervention adoption, implementation, and maintenance.

The studies included in this review provided limited information regarding the context of each early childcare and education setting. Although all included studies noted that the intervention took place in an early childcare and education setting, very few provided data on participation rates for both the setting and staff levels, staff inclusion/exclusion criteria, and characteristics of adoption or nonadoption. With the availability of more detailed data, whether via the expanded CONSORT diagram or through using the RE-AIM framework [56], we may begin to better understand what interventions strategies and modes of delivery are most or least effective in these settings [64].

In terms of Implementation, all studies provided at least a brief description of intervention components; however, less than half reported on the extent to which the intervention protocol was delivered as intended (i.e., fidelity). Evidence-based interventions that are not implemented as intended and at the appropriate dose will be less effective [73,74]. Fidelity can be conceptualized with various components including dose, adherence to specified intervention components, and quality of delivery [21,75,76]. For example, learning about factors that may affect fidelity, such as the quality of training and support for staff implementing the intervention and/or staff constraints that may limit their ability to deliver the intervention [77], can provide insight into what factors need to be improved to maximize implementation fidelity. Such implementation fidelity data are critical for determining implementation feasibility and applicability [78] to one’s own early childcare and education setting. Additionally, one can use implementation fidelity data to inform the design and planning of an early childcare and education obesity intervention by either reducing or increasing intervention components based on the extent to which the protocol was delivered as intended. Iterative applications of the RE-AIM framework can be used in conjunction with implementation fidelity data to inform adaptive interventions to better meet the needs of their setting and/or target population as intervention challenges arise [22]. Implementation fidelity data could also be used as part of a participatory approach to intervention planning and development [22], with setting staff and/or members of the target population providing insight as to which intervention strategies may be most feasible, appropriate, and achievable. To enhance implementation fidelity reporting, measurement protocols should be developed and disseminated so that researchers can more easily report these data and facilitate translation of research into practice [79]. In addition to the reporting of implementation fidelity and descriptions of implementation strategies and contexts, researchers may consider mediation analyses to test the relationships between setting, implementation strategies, and implementation and participant outcomes [80]. This would allow for the examination of fit between the characteristics of an intervention, setting, and outcomes to determine what is working, when, and why or to better understand which intervention components are generalizable and which are not [65]. As the obesity prevention intervention field advances, so does the need for sophisticated implementation and evaluation efforts to account for increasing complexity [62].

Strengths and limitations

This is the first systematic review of obesity interventions in early childcare and education settings to focus primarily on external validity using the RE-AIM framework. This review is an important step in the progress of translational research for obesity prevention interventions for preschool-aged children. Additionally, this review focused on randomized controlled trials, which are considered the gold standard study design. Lastly, a team of reviewers were involved during the selection and data extraction process, which minimized the potential for selection bias.

However, this systematic review does have limitations. The articles focused on interventions targeting both diet and physical activity behaviors, thereby excluding studies that may have targeted just one of those behaviors or another related behavior, such as sedentary behavior. Additionally, this review was limited to the past 10 years, excluding older studies, which may have provided additional data. Lastly, data were not reported consistently across the 23 studies, making some comparison difficult.

CONCLUSION

Despite its limitations, this is the first systematic review of early childcare and education obesity prevention interventions using the RE-AIM framework and suggests that more consistent reporting on external validity needs to be conducted. In an effort to advance the field and provide a more comprehensive evaluation of obesity prevention interventions, future studies should include the expanded CONSORT diagram to ensure inclusion of data related to external validity to inform replication, dissemination, and implementation, particularly to meet the needs of diverse and low-resource settings. As such, intervention studies conducted with racially and ethnically diverse and/or low-income populations guided by the RE-AIM framework are needed. The increasing emphasis on external validity will require effort on the part of researchers, journals, and funders who have traditionally focused on the efficacy of randomized controlled trials, with little attention to how tightly controlled evidence-based trials could transition into more “real-world” settings or have potential practice and policy implications. More detailed reporting on factors influencing external validity may also serve to close practice to policy translation gaps, as findings from efficacy studies and reviews of efficacy studies may inform the development of evidence-based diet and physical activity policy guidelines in early childcare and education settings.

Funding:

This study was funded by the National Cancer Institute Training Program: Cancer Education and Career Development Program (T32CA057699).

Compliance with Ethical Standards

Conflicts of Interest: All authors declare that they have no conflicts of interest.

Authors’ Contributions: J.C.S.-F. conceived the study, implemented the search strategy, led the screening and coding process and the preparation of the manuscript. A.H. and A.B. served as screening and coding reviewers and provided feedback during the preparation of the manuscript. J.B. and A.K. served as screening and coding reviewers and provided critical feedback during conceptualization and preparation of the manuscript. M.L.F. mentored J.C.S.-F. on this study, served as a coding and screening reviewer, and provided critical feedback during conceptualization and preparation of the manuscript.

Ethical Approval: This article does not contain any studies with human participants performed by any of the authors. This article does not contain any studies with animals performed by any of the authors.

Informed Consent: This study does not involve human participants and informed consent was, therefore, not required.