Abstract
Objective
The National Early Care and Education Learning Collaboratives (ECELC) Project aims to promote healthy physical activity and nutrition environments, policies and practices in early care and education (ECE) programmes across multiple states. The present pilot study sought to assess changes to the physical activity and nutrition practices in a sub-sample of ECE programmes participating in the ECELC using the Environment and Policy Assessment and Observation (EPAO). Additionally, it sought to compare results with the Nutrition and Physical Activity Self-Assessment for Child Care (NAP SACC).
Design
Quasi-experimental pre–post pilot study where paired-sample t tests examined changes to physical activity and nutrition practices from pre-assessment to post-assessment (P<0·05). Pearson correlation coefficients examined change scores from EPAO compared with NAP SACC with statistical significance set at a two-sided α level of P<0·10 to account for sample size.
Setting
The study occurred among ECE programmes.
Subjects
Pre-school classrooms in nineteen ECE programmes across four US states were observed.
Results
EPAO data demonstrated an increase in total score from pre-assessment to post-assessment (150 (sd 30) to 176 (sd 35)). NAP SACC change scores demonstrated little relationship with EPAO domain change scores, with exceptions in Nutrition Policy and Physical Activity Policy (r=−0·4 and −0·6, respectively).
Conclusions
The overall improvements reported through the EPAO suggest participation in the ECELC resulted in changes in critical nutrition- and physical activity-related practices. However, considerable differences in data reported using the NAP SACC compared with the EPAO suggest subjective data should be interpreted with caution and objective measurement should be used when feasible.
Keywords: Child day-care centres, Childcare, Nutrition policy, Physical activity
One out of four (24 %) children in the USA aged 5 years or younger spends time in an organized care facility( 1 ). Early care and education (ECE) programmes, which are facilities (including classroom environments, staff, policies and practices) that provide nurturing care, support for development and learning experiences for children aged 5 years or younger, are a strategic setting for implementing strategies to prevent obesity( 2 ). Preliminary evidence suggests environmental-level strategies in ECE, such as improving policies and practices related to eating, physical activity and sedentary behaviours, appear to directly influence children enrolled in these programmes( 3 , 4 ).
In 2007, Nemours Children’s Health System (Nemours) implemented an intervention in Delaware to promote healthy eating and physical activity among children and young children ranging in age from 0 to 5 years in a variety of settings, including ECE. A key part of the initiative included the establishment of ‘learning collaboratives’ and ‘train-the-trainer’ models with ECE programmes, which helped these programmes identify and implement healthy eating and physical activity practices and policies( 5 ). These practices improved significantly in 81 % of the twenty-eight participating ECE programmes( 5 ), suggesting that continued work in this area is warranted.
The original Delaware model was adapted for a multi-state implementation effort in 2012 by Nemours in collaboration with the Centers for Disease Control and Prevention. The resulting National Early Care and Education Learning Collaboratives (ECELC) Project (in its fourth year at the time of writing) aims to promote healthy environments, policies and practices with regard to the following areas: Breast-feeding & Infant Feeding, Child Nutrition, Infant & Child Physical Activity, Outdoor Play & Learning, and Screen Time. To the best of our knowledge, it is the first large-scale effort aimed at improving these types of policies and practices in ECE programmes across multiple states( 6 , 7 ). Although data have shown promise for broad implementation of projects that promote healthy eating, physical activity and reduction in screen time in childcare settings (TM Smith, DJ Schober, J Shuell et al., unpublished results), most data collected have been self-reported (e.g. using the Nutrition and Physical Activity Self-Assessment for Child Care (NAP SACC)) and not verified through objective measures. While one study indicated that the NAP SACC assessment tool is a stable and reasonably accurate instrument for use in childcare interventions, if funds allow, a more robust, less subjective measure may be more appropriate for researchers seeking an outcome measure to assess intervention impact( 8 ).
The current pilot study was exploratory in nature and aimed to objectively measure a small sub-sample of pre-school classrooms using the Environment and Policy Assessment and Observation (EPAO)( 9 ) measurement tool to assess change from before to after participating in the ECELC with regard to nutrition and physical activity. Specifically, the present study had two aims: (i) to determine how participation in the second cohort of the ECELC influenced eating and physical activity at the ECE programme level, as measured by the EPAO for sub-sampled programmes; and (ii) to compare outcomes derived from the collected EPAO data with matched outcomes derived from the collected NAP SACC data.
Methods
The ECELC intervention consisted of five main strategies: (i) self-assessment; (ii) in-person peer learning sessions; (iii) action planning and implementation; (iv) technical assistance; and (v) reassessment. ECE programme staff participated in learning sessions that included didactic presentations on content, interactive activities, and peer sharing and support. In between learning sessions, ECE programmes received action planning tasks, which encouraged them to share what they learned with their programme staff and build staff support for implementing best practices across topic areas on-site. Finally, each ECE programme received individualized technical assistance in between learning sessions in order to support programmes during their action planning phases. Reassessment (post-assessment) occurred after the fifth and final learning session approximately 10 months later.
The methodology of the present study was based on a study published in 2015 by Benjamin Neelon and colleagues. That study employed a randomized control trial to test the effect of an intervention targeting ECE programmes serving children less than 2 years of age to improve the nutrition and physical activity environments, as outlined in the Baby NAP SACC for ECE programmes( 10 ). While the Benjamin Neelon study differed from the current study in that it compared data collected using the EPAO among intervention and control groups (and also that it utilized Baby NAP SACC v. the regular full NAP SACC), it provided a framework for the current study on conducting pre–post analysis and the comparison of tools.
Participants
To be eligible to participate in the ECELC, programmes had to serve at least fifty children, develop a Leadership Team of at least three staff (e.g. owner or director, teacher, cook) and attend each of the five in-person learning sessions. Some data were collected via an enrolment form administered electronically and included contact information, programme characteristics (e.g. number of children served) and state characteristics (e.g. presence of a Quality Rating and Improvement System (QRIS), an approach to assess, improve and communicate the level of quality in ECE programmes)( 11 ). Seven sites (North/Central Florida, South Florida, Indiana, Missouri, New Jersey, Kansas and Arizona) participated in the second phase of the second cohort of the ECELC; however, only four (North/Central Florida, Indiana, Missouri and New Jersey) participated in the current study. Kansas and Arizona were ineligible for participation due to having an early start in the ECELC, and South Florida was ineligible because investigators were not granted access to classrooms. Because the current pilot study was exploratory in nature, and due to budget constraints, six programmes from a pool of 559 ECE programmes enrolled in the second phase of the second cohort of the ECELC were randomly selected from each of the four sites to be observed using the EPAO (n 24). Programmes were excluded from analysis if they were unable to complete an EPAO observation at both pre-assessment and post-assessment or if they did not complete all aspects of the ECELC (five programmes), resulting in a final analytical sample of nineteen programmes.
Trained observers (one observer per classroom) conducted observations using the EPAO in the same pre-school-aged classroom for two consecutive days at pre-assessment (August through October 2014, prior to the launch of the ECELC) and again at post-assessment (August and September 2015, two to four months after the completion of intervention activities). Assessments occurred two to four months after the completion of the intervention activities for several reasons, chiefly due to the ‘real world’ challenges associated with conducting the study within a set budget. One specific example was that travel and schedules of several trained observers had to be coordinated in a way where observations could be conducted on consecutive days among ECE programmes within each site. Assessments were conducted in pre-school-aged classrooms as it was the largest proportion of children served across the programmes (56 %), although it ranged across sites from 33 % (Missouri) to 69 % (New Jersey). Additionally, most NAP SACC items apply to children of pre-school age.
EPAO measurement tool and scoring
The EPAO was developed to objectively assess environments of ECE programmes( 9 ). All items in the EPAO were utilized for this analysis and were divided into two subgroups (Nutrition and Physical Activity) comprised of sixteen separate domains. Domains in the Nutrition subgroup included Fruits and Vegetables (ten items), Whole Grains and Low Fat Meats (six items), High Sugar/High Fat Foods (nine items), Beverages (twelve items), Staff Behaviours Regarding Nutrition (seven items), Nutrition Environment (four items), Nutrition Training and Education (six items) and Nutrition Policy (fourteen items). Domains in the Physical Activity subgroup included Active Opportunities (five items), Sedentary Opportunities (four items), Sedentary Environment (three items), Portable Play Environment (seven items), Fixed Play Environment (eight items), Staff Behaviours Regarding Physical Activity (five items), Physical Activity Training and Education (five items) and Physical Activity Policy (six items). Per EPAO protocol, item responses were coded on a three-point scale and scored as 0 (best practice not met), 1 (close to best practice) or 2 (best practice met). Scores were totalled within a given domain, for a total of 20 possible points per domain; a higher score translated to a greater number of best practices being observed. Scores were summed for each domain to calculate a total EPAO score (0–320 points) made up of a Nutrition sub-score (0–160 points) and Physical Activity sub-score (0–160 points).
Comparing EPAO change scores with NAP SACC change scores
ECE programmes completed the NAP SACC instrument following the first learning session and post-assessment occurred during the action period prior to the last learning session. The NAP SACC consisted of four topic areas: Breast-feeding & Infant Feeding (twenty-three items), Child Nutrition (forty-four items), Infant & Child Physical Activity (twenty-two items) and Screen Time (twelve items)( 12 ). The evaluation crosswalk method was used to identify the domains with which each NAP SACC item most aligned (see online supplementary material, Supplemental Table 1)( 13 ). Two researchers independently assigned each individual NAP SACC item to an EPAO domain that fit the objective of both the NAP SACC item and the corresponding EPAO domain. In cases of disagreement, items were discussed among the research team until consensus was reached. Because the observations occurred only in pre-school classrooms, NAP SACC items were excluded if they did not apply to pre-school-aged children (e.g. the Breast-feeding & Infant Feeding assessment), resulting in eighty-nine of the 121 NAP SACC items being utilized in this analysis. Each item had four response options, ranging from non-compliance with a particular best practice to total compliance with said best practice. When the response option representing total compliance with a given best practice was selected, the best practice was considered being met (1=best practice met). All other responses were considered to mean the best practice was not being met (0=best practice not met). The raw NAP SACC composite scores for each domain were calculated as the sum of outcomes of each applicable NAP SACC item. Because the number of NAP SACC items differs from the number of EPAO items for each domain, the raw scores were then scaled to be directly comparable to their respective EPAO domain as 20 times the raw score divided by the number of items. This resulted in similar maximum scores (20 points) per domain of the EPAO and composite scores of the NAP SACC, aiding in interpretation across the findings from the two measurement tools.
Statistical analysis
The SAS statistical software package version 9.4 was used for all statistical analyses. Descriptive statistics were calculated across all EPAO domains and NAP SACC composites at pre-assessment and post-assessment. A change score for each domain was calculated by subtracting the pre-assessment score from the post-assessment score. A paired-sample t test was utilized to examine whether mean scores changed from pre-assessment to post-assessment across the sub-sampled ECE programmes. In order to test if the data collected using the EPAO resulted in similar findings as data collected using the NAP SACC, Pearson correlation coefficients were used to measure the linear correlation between the change scores for each of the sixteen EPAO domains and corresponding NAP SACC composites. To accommodate the sample size of nineteen programmes, statistical significance was set at a two-sided α level of P<0·10, which enabled the power to approach 0·8 (β=0·22)( 14 , 15 ).
Results
As described in the ‘Methods’ section, twenty-four programmes were randomly selected to be observed using the EPAO for the current study. Five programmes were unable to complete an EPAO observation at both pre-assessment and post-assessment (e.g. at least one programme closed for business between pre-assessment and post-assessment) or did not complete all aspects of the ECELC, resulting in a final sample of nineteen programmes. The majority of participating ECE programmes in the present sub-study were non-profit (68 %), most programmes (74 %) provided only a full day of care, and about half of ECE programmes participated in the US Department of Agriculture’s Child and Adult Care Food Program (47 %; Table 1). Overall, programme accreditation was low (32 %). Of all nineteen participating programmes, only six (32 %) participated in their state’s QRIS. All programmes provided some type of meal or snack throughout the day; the majority provided both meals and snacks (90 %). Most of the programmes prepared meals or snacks on-site (84 %), while relatively few only catered (11 %), and one programme used a combination of preparation on-site and catered.
Table 1.
Characteristics of early care and education (ECE) programmes participating in the environment and policy assessment and observation sub-study (n 19)
| Stratified by site | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Overall | North/Central Florida | Indiana | Missouri | New Jersey | ||||||
| n | % | n | % | n | % | n | % | n | % | |
| Total ECE programmes | 19 | 100·0 | 3 | 15·8 | 6 | 31·6 | 5 | 26·3 | 5 | 26·3 |
| Profit status | ||||||||||
| Non-profit | 13 | 68·4 | 3 | 100·0 | 5 | 83·3 | 1 | 20·0 | 4 | 80·0 |
| For-profit or private | 6 | 31·6 | 0 | 0·0 | 1 | 16·7 | 4 | 80·0 | 1 | 20·0 |
| Head Start/Early Start | ||||||||||
| Participate | 4 | 21·1 | 0 | 0·0 | 2 | 33·3 | 0 | 0·0 | 2 | 40·0 |
| Do not participate | 15 | 79·0 | 3 | 100·0 | 4 | 66·7 | 5 | 100·0 | 3 | 60·0 |
| School-based | ||||||||||
| Yes | 2 | 10·5 | 0 | 0·0 | 2 | 33·3 | 0 | 0·0 | 0 | 0·0 |
| No | 17 | 89·5 | 3 | 100·0 | 4 | 66·7 | 5 | 100·0 | 5 | 100·0 |
| Faith-based | ||||||||||
| Yes | 6 | 31·6 | 3 | 100·0 | 2 | 33·3 | 0 | 0·0 | 1 | 20·0 |
| No | 13 | 68·4 | 0 | 0·0 | 4 | 66·7 | 5 | 100·0 | 4 | 80·0 |
| Full or half day care | ||||||||||
| Full day and half day | 4 | 21·1 | 1 | 33·3 | 2 | 33·3 | 0 | 0·0 | 1 | 20·0 |
| Full day only | 14 | 73·7 | 2 | 66·7 | 3 | 50·0 | 5 | 100·0 | 4 | 80·0 |
| Half day only | 1 | 5·3 | 0 | 0·0 | 1 | 16·7 | 0 | 0·0 | 0 | 0·0 |
| CACFP | ||||||||||
| Participate | 9 | 47·4 | 1 | 33·3 | 2 | 33·3 | 3 | 60·0 | 3 | 60·0 |
| Do not participate | 10 | 52·6 | 2 | 66·7 | 4 | 66·7 | 2 | 40·0 | 2 | 40·0 |
| Accreditation | ||||||||||
| Accredited | 6 | 31·6 | 2 | 66·7 | 1 | 16·7 | 1 | 20·0 | 2 | 40·0 |
| Not accredited | 13 | 68·4 | 1 | 33·3 | 5 | 83·3 | 4 | 80·0 | 3 | 60·0 |
| Quality rating and improvement systems | ||||||||||
| Participate | 6 | 31·6 | 1 | 33·3 | 4 | 66·7 | 0 | 0·0 | 1 | 20·0 |
| Do not participate | 13 | 68·4 | 2 | 66·7 | 2 | 33·3 | 5 | 100·0 | 4 | 80·0 |
| Meals and snacks provided | ||||||||||
| Meals and snacks | 17 | 89·5 | 3 | 100·0 | 6 | 100·0 | 5 | 100·0 | 3 | 60·0 |
| Meals only | 1 | 5·3 | 0 | 00·0 | 0 | 00·0 | 0 | 00·0 | 1 | 20·0 |
| Snacks only | 1 | 5·3 | 0 | 00·0 | 0 | 00·0 | 0 | 00·0 | 1 | 20·0 |
| Preparation of meals/snacks | ||||||||||
| Prepared on-site and catered | 1 | 5·3 | 0 | 00·0 | 1 | 16·7 | 0 | 00·0 | 0 | 00·0 |
| Catered only | 2 | 10·5 | 0 | 00·0 | 1 | 16·7 | 0 | 00·0 | 1 | 20·0 |
| Prepared on-site only | 16 | 84·2 | 3 | 100·0 | 4 | 66·7 | 5 | 100·0 | 4 | 80·0 |
| Family provision of meals/snacks allowed | ||||||||||
| Meals and snacks | 1 | 5·3 | 0 | 00·0 | 0 | 00·0 | 0 | 00·0 | 1 | 20·0 |
| Meals only | 2 | 10·5 | 1 | 33·3 | 1 | 16·7 | 0 | 00·0 | 0 | 00·0 |
| Snacks only | 1 | 5·3 | 0 | 00·0 | 0 | 00·0 | 0 | 00·0 | 1 | 20·0 |
| Not allowed | 15 | 79·0 | 2 | 66·7 | 5 | 83·3 | 5 | 100·0 | 3 | 60·0 |
CACFP, Child and Adult Care Food Program.
Data were collected from pre-school classrooms in nineteen programmes across four US states in 2014.
The mean total EPAO change score (for both Nutrition and Physical Activity) across all programmes was 26 (sd 38) points, which was a 17 % increase from pre-assessment to post-assessment (P=0·008; Table 2). The Nutrition sub-score contributed the most to the overall score with a change score of 19 points, which was a 25 % increase from pre-assessment to post-assessment (P<0·001). Physical Activity contributed 8 points to the overall change but was not significant. At the domain level, five of the nutrition domains underwent a statistically significant change, with Nutrition Environment showing the greatest improvement of 6 points (P<0·001). One of the physical activity domains, Active Opportunities, showed a significant improvement with an increase of 2 points (P=0·001).
Table 2.
Change in Environment and Policy Assessment and Observation (EPAO) score from pre-assessment to post-assessment (n 19)
| EPAO score | Pre-assessment | Post-assessment | Change | P value |
|---|---|---|---|---|
| Total score | 149·79 | 176·03 | 26·25 | 0·008*** |
| Nutrition sub-score | 74·64 | 93·17 | 18·52 | <0·001**** |
| Fruits and Vegetables | 12·54 | 11·97 | −0·57 | 0·456 |
| Whole Grains and Low Fat Meats | 7·28 | 9·94 | 2·66 | 0·040** |
| High Sugar/High Fat Foods | 12·68 | 14·51 | 1·83 | 0·021* |
| Beverages | 11·97 | 12·95 | 0·98 | 0·354 |
| Staff Behaviours Regarding Nutrition | 12·26 | 14·78 | 2·53 | 0·001*** |
| Nutrition Environment | 8·68 | 15·09 | 6·40 | <0·001*** |
| Nutrition Training and Education | 5·04 | 9·04 | 4·00 | 0·003*** |
| Nutrition Policy | 4·20 | 4·89 | 0·70 | 0·239 |
| Physical Activity sub-score | 75·14 | 82·87 | 7·72 | 0·226 |
| Active Opportunities | 6·37 | 8·05 | 1·68 | 0·001** |
| Sedentary Opportunities | 11·93 | 11·71 | −0·22 | 0·837 |
| Sedentary Environment | 10·35 | 12·81 | 2·46 | 0·115 |
| Portable Play Environment | 13·01 | 13·08 | 0·08 | 0·963 |
| Fixed Play Environment | 13·17 | 12·30 | −0·87 | 0·415 |
| Staff Behaviours Regarding Physical Activity | 11·79 | 13·89 | 2·11 | 0·157 |
| Physical Activity Training and Education | 5·72 | 7·16 | 1·44 | 0·404 |
| Physical Activity Policy | 3·14 | 4·22 | 1·08 | 0·411 |
Data were collected from pre-school classrooms in nineteen programmes across four US states from 2014 to 2015.
*P<0·10, **P<0·05, ***P<0·01, ****P<0·001.
Scores for the EPAO domains and their corresponding NAP SACC composites are shown in Table 3, as well as the Pearson correlation coefficients. Ten of the sixteen domain–composite pairs shared directionality in their change scores indicating both the EPAO and NAP SACC were able to detect similar changes when programmes were measured as a group. However, when comparing the EPAO change score with the NAP SACC change score for each programme, the correlation coefficients and P values demonstrate less of a relationship between the two measurement tools. Only the domains and composites of Nutrition Policy (r=−0·4, P=0·06) and Physical Activity Policy (r=−0·6, P=0·02) were found to have a statistically significant correlation. Interestingly, the correlations implied a negative relationship between the two scores, implying that as one increased the other decreased. No other relationships between EPAO domains and NAP SACC composites were significantly related.
Table 3.
Mean change scores for Environment and Policy Assessment and Observation (EPAO) domains and Nutrition and Physical Activity Self-Assessment for Child Care (NAP SACC) composites and Pearson correlation coefficients
| Domain | EPAO change score† | NAP SACC change score† | Pearson correlation coefficient† | P value |
|---|---|---|---|---|
| Fruits and Vegetables | −0·57 | −1·58 | 0·056 | 0·821 |
| Whole Grains and Low Fat Meats | 2·66 | −2·11 | −0·056 | 0·818 |
| High Sugar/High Fat Foods | 1·83 | 0·30 | −0·275 | 0·254 |
| Beverages | 0·98 | −0·84 | 0·100 | 0·685 |
| Staff Behaviours Regarding Nutrition | 2·53 | 1·75 | 0·303 | 0·208 |
| Nutrition Environment | 6·40 | −1·40 | 0·109 | 0·657 |
| Nutrition Training and Education | 4·00 | 4·21 | −0·268 | 0·268 |
| Nutrition Policy | 0·70 | −1·40 | −0·434 | 0·063* |
| Active Opportunities | 1·68 | 2·11 | −0·142 | 0·563 |
| Sedentary Opportunities | −0·22 | −2·63 | −0·052 | 0·832 |
| Sedentary Environment | 2·46 | −0·26 | −0·006 | 0·979 |
| Portable Play Environment | 0·08 | 0·53 | −0·280 | 0·247 |
| Fixed Play Environment | −0·87 | 1·71 | 0·138 | 0·574 |
| Staff Behaviours Regarding Physical Activity | 2·11 | 3·16 | 0·125 | 0·609 |
| Physical Activity Training and Education | 1·44 | 4·08 | −0·125 | 0·610 |
| Physical Activity Policy | 1·08 | 1·26 | −0·552 | 0·022** |
Data were collected from pre-school classrooms in nineteen programmes across four US states from 2014 to 2015.
*P<0·10, **P<0·05.
Correlation coefficients are calculated on programme-level change scores for each domain.
Discussion
Although the present study was a pilot study with a small sample size, we did find that programmes assessed using the EPAO changed with regard to several physical activity and nutrition environment best practices( 9 ). Overall, the programmes improved by about 17 % of the total EPAO score, with the majority of improvements occurring in the nutrition-focused domain. Programmes improved in the areas of Whole Grains and Low Fat Meats, High Sugar/High Fat Foods, Staff Behaviours Regarding Nutrition, the Nutrition Environment, and Nutrition Training and Education. However, programmes did not improve significantly in the areas of Fruits and Vegetables, Beverages, or Nutrition Policy. For Physical Activity, programmes improved significantly in the areas of Active Opportunities, but no other domains. Results of similar studies promoting healthy eating and physical activity in childcare settings have also shown interventions involving self-assessment and action planning enable change in programme-level practices( 6 , 7 , 16 ).
The original intent of the NAP SACC measurement tool was to serve as an aid in self-assessment and action planning and it was not intended as an objective outcome measure( 6 , 8 , 9 , 12 ). Accordingly, all ECE programmes participating in the ECELC developed personalized action plans based on their self-assessments and were given autonomy to focus on their most desired changes related to any of the five NAP SACC sections, including Breast-feeding & Infant Feeding. This is important to consider because domains that resulted in little or no change scored relatively higher at pre-assessment when compared with domains that resulted in significant change. For example, even though there was no significant change to the domain of Portable Play Environment, the score from the pre-assessment was 13·01, which was the second highest score at pre-assessment. It is unknown exactly how ECE programmes chose to focus their targeted action planning, although it is likely they selected areas of higher need, which may have been domains with relatively lower scores at pre-assessment and potentially why programmes did not improve significantly in several areas (i.e. fruits and vegetables, beverages, reducing sedentary opportunities, improving portable play environment or the fixed play environment, or enhancing staff behaviours regarding physical activity) as measured by the EPAO.
The EPAO generally resulted in different outcomes from the NAP SACC. Only the EPAO domains of Nutrition Policy and Physical Activity Policy correlated with their NAP SACC counterparts, although it was a negative, and therefore an unexpected, correlation. Another study that aimed to validate the NAP SACC using the EPAO as the gold standard assessed the relationship of cross-sectional scores of each measurement tool( 8 ). Kappa statistics ranged from −0·01 to 0·79, which was considered poor to substantial agreement. While the prior study tested the relationship between the EPAO and the NAP SACC using cross-sectional scores, the current study aimed to determine if the data collected using the EPAO resulted in the same outcomes as data collected using the NAP SACC. Accordingly, change scores were used to test the relationship between the two measurement tools. Regardless, our study corroborates the findings of the previous study, suggesting the NAP SACC may tap into slightly different constructs from those from the EPAO, especially given that the former is a self-assessment and the latter is an observational measure.
Since the NAP SACC is a self-assessment, there is greater opportunity for bias due to social desirability, which is common among nutrition- and physical activity-based self-reporting( 17 – 21 ). The EPAO is completed by an independent observer so there is far less chance for social desirability bias. Additionally, the NAP SACC pre-assessment is completed early in the ECELC, when ECE programme staff may have less knowledge about topics related to nutrition and physical activity than at post-assessment. Therefore, rather than a reflection of true change, the NAP SACC may sometimes be a reflection of increased knowledge and more accurate responses; similar interventions have promoted an increase in health-related knowledge among childcare providers( 22 ). Future research could test the NAP SACC and EPAO in a larger sample, which would increase the power of the analysis and potentially lead to more consistency between change scores of the two tools. However, the EPAO is costly and requires additional staff time, and therefore the NAP SACC may be more feasible as an assessment tool to implement on a wide scale.
Our study has some limitations to note. The small sample size did not allow for enough power to adequately detect relationships between the EPAO and the NAP SACC at the traditional level of significance (α=0·05). Additionally, since outcome data were all quantitative, we did not have the contextual information to assist in explaining and interpreting findings. Further, state-based healthy eating and physical activity initiatives outside the ECELC like the Missouri Eat Smart and MOve Smart Guidelines for Child Care( 23 ) may have influenced change. It should also be discussed that the post-assessment ideally would have occurred directly after the completion of the intervention. However, as described earlier, this evaluation required utmost coordination among the study team and participating ECE programmes, which led to post-assessment occurring two to four months after the completion of the evaluation. Nevertheless, data from a study currently in review suggested that best practices and policies related to breast-feeding support, child nutrition, physical activity, and screen time reduction were sustained one year after this intervention ended (TM Smith, C Blaser, C Geno Rasmussen et al., unpublished results), so we expect that changes made during the intervention would be unchanged two to four months after the completion of the intervention. Despite these limitations, the current study helps elucidate nuances in the implementation, practicalities and ultimately the comparison of outcome data using the NAP SACC and EPAO. These changes, while just one step in potentially reducing obesity among children aged 5 years or younger, can help inform other obesity prevention interventions in ECE programmes moving forward.
The overall significant improvements reported through the EPAO measurement tool in the present pilot study suggest that participation in the ECELC resulted in changes in some, but not all, critical nutrition- and physical activity-related practices in ECE programmes. However, considerable differences in data reported using the NAP SACC compared with objective data using the EPAO suggest NAP SACC data should be interpreted with caution and objective measurement should be used when feasible. Self-assessment and observational methods for assessing nutrition and physical activity practices in ECE programmes have strengths and limitations; factors such as cost and feasibility should be taken into account when choosing a measure. However, both methods have merit and are important to assess and advance environmental interventions intended to change nutrition and physical activity policies/practices in ECE.
Acknowledgements
Acknowledgements: The authors would like to thank all of the National Early Care and Education Learning Collaboratives Project stakeholders, including State Implementing Partners, State Project Coordinators, Trainers and Leadership Team members, for their participation in this evaluation. Also, they would like to thank Dr Dan Schober, Dr Mary Story, Dr Dianne Ward and Stephanie Mazzuca for providing expertise on study design and measurement protocol. Finally, the authors would like to thank staff at the Gretchen Swanson Center for Nutrition for their support on this evaluation. Financial support: Funding for the National Early Care and Education Learning Collaboratives Project was provided by a five-year Cooperative Agreement (1U58DP004102-01) to the Nemours Children’s Health System from the Centers for Disease Control and Prevention. The funders advised on the design and analysis, but did not have a role in the writing of this article. Conflict of interest: None. Authorship: J.S. and A.L.Y. conceived and supervised the study; C.B. completed the analysis; T.M.S. led the writing; C.P. and T.M.S. assisted with the study; C.G.R. advised on the study; all authors contributed to the writing. Ethics of human subject participation: This study was approved by the Nemours Institutional Review Board.
Supplementary material
For supplementary material accompanying this paper visit https://doi.org/10.1017/S1368980017000155.
click here to view supplementary material
References
- 1. Laughlin L (2013) Who’s Minding the Kids? Child Care Arrangements: Spring 2011. Current Population Reports, P70–135. Washington, DC: US Census Bureau. [Google Scholar]
- 2. Birch LL, Parker L & Burns A (2011) Early Childhood Obesity Prevention Policies. Washington, DC: National Academies Press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Benjamin SE, Cradock A, Walker EM et al. (2008) Obesity prevention in child care: a review of US state regulations. BMC Public Health 8, 188. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. McWilliams C, Ball SC, Benjamin SE et al. (2009) Best-practice guidelines for physical activity at child care. Pediatrics 124, 1650–1659. [DOI] [PubMed] [Google Scholar]
- 5. Nemours Health and Prevention Services (2012) Evaluation of School and Child-Care Sector Strategies in Delaware: Final Evaluation Report. http://www.rwjf.org/content/dam/farm/reports/evaluations/2012/rwjf401734 (accessed February 2017).
- 6. Benjamin SE, Ammerman A, Sommers J et al. (2007) Nutrition and physical activity self-assessment for child care (NAP SACC): results from a pilot intervention. J Nutr Educ Behav 39, 142–149. [DOI] [PubMed] [Google Scholar]
- 7. Trost SG, Messner L, Fitzgerald K et al. (2011) A nutrition and physical activity intervention for family child care homes. Am J Prev Med 41, 392–398. [DOI] [PubMed] [Google Scholar]
- 8. Benjamin SE, Neelon B, Ball SC et al. (2007) Reliability and validity of a nutrition and physical activity environmental self-assessment for child care. Int J Behav Nutr Phys Act 4, 29. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Ward D, Hales D, Haverly K et al. (2008) An instrument to assess the obesogenic environment of child care centers. Am J Health Behav 32, 380–386. [DOI] [PubMed] [Google Scholar]
- 10. Blaine RE, Davison KK, Hesketh K et al. (2015) Child care provider adherence to infant and toddler feeding recommendations: findings from the Baby Nutrition and Physical Activity Self-Assessment for Child Care (Baby NAP SACC) Study. Child Obes 11, 304–313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. National Center on Early Childhood Quality Assurance (n.d.) About QRIS. https://qrisguide.acf.hhs.gov/index.cfm?do=qrisabout (accessed September 2016).
- 12. Benjamin SE, Ammerman AS, Ward DS et al. (2007) An intervention to promote healthy weight: Nutrition and Physical Activity Self-Assessment for Child Care (NAP SACC) theory and design. Prev Chronic Dis 4, A67. [PMC free article] [PubMed] [Google Scholar]
- 13. O’Sullivan RG (1991) Improving evaluation design and use through the ‘evaluation crosswalk’ method. Natl Forum Appl Educ Res J 4, 43–49. [Google Scholar]
- 14. Schoenfeld D (1980) Statistical considerations for pilot studies. Int J Radiat Oncol Biol Phys 6, 371–374. [DOI] [PubMed] [Google Scholar]
- 15. Moore CG, Carter RE, Nietert PJ et al. (2011) Recommendations for planning pilot studies in clinical and translational research. Clin Transl Sci 4, 332–337. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Davison KK, Falbe J, Taveras EM et al. (2015) Evaluation overview for the Massachusetts Childhood Obesity Research Demonstration (MA-CORD) Project. Child Obes 11, 23–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Hebert JR, Ma Y, Clemow L et al. (1997) Gender differences in social desirability and social approval bias in dietary self-report. Am J Epidemiol 146, 1046–1055. [DOI] [PubMed] [Google Scholar]
- 18. Van de Mortel TF (2008) Faking it: social desirability response bias in self-report research. Aust J Adv Nurs 25, 40–48. [Google Scholar]
- 19. Prince SA, Adamo KB, Hamel ME et al. (2008) A comparison of direct versus self-report measures for assessing physical activity in adults: a systematic review. Int J Behav Nutr Phys Act 5, 56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Adams SA, Matthews CE, Ebbeling CB et al. (2005) The effect of social desirability and social approval on self-reports of physical activity. Am J Epidemiol 161, 389–398. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Hebert JR, Clemow L, Pbert L et al. (1995) Social desirability bias in dietary self-report may compromise the validity of dietary intake measures. Int J Epidemiol 24, 389–398. [DOI] [PubMed] [Google Scholar]
- 22. Bonis M, Loftin M, Ward D et al. (2014) Improving physical activity in daycare interventions. Child Obes 10, 334–341. [DOI] [PubMed] [Google Scholar]
- 23. Missouri Department of Health & Senior Services (2014) Missouri Eat Smart Guidelines for Child Care. http://health.mo.gov/living/wellness/nutrition/eatsmartguidelines/pdf/MO_Eat_Smart_Guidelines-Child_Care.pdf (accessed February 2017).
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
For supplementary material accompanying this paper visit https://doi.org/10.1017/S1368980017000155.
click here to view supplementary material