The Healthy Afterschool Activity and Nutrition Documentation Instrument

Abstract

Background

Policies call on afterschool programs to improve the physical activity and nutrition habits of youth attending. No tool exists to assess the extent to which the afterschool program environment meets physical activity and nutrition policies.

Purpose

To describe the development of the Healthy Afterschool Activity and Nutrition Documentation (HAAND) instrument, which consists of two subscales: Healthy Afterschool Program Index for Physical Activity (HAPI-PA) and the HAPI-Nutrition (HAPI-N).

Methods

Thirty-nine afterschool programs took part in the HAAND evaluation during fall/spring 2010–2011. Inter-rater reliability data were collected at 20 afterschool programs during a single site visit via direct observation, personal interview and written document review. Validity of the HAPI-PA was established by comparing HAPI-PA scores to pedometer steps collected in a subsample of 934 children attending 25 of the afterschool programs. Validity of the HAPI-N scores was compared against the mean number of times/week that fruits/vegetables (FV) and whole grains were served in the program.

Results

Data were analyzed in June/July 2011. Inter-rater percent agreement was 85%–100% across all items. Increased pedometer steps were associated with the presence of a written policy related to physical activity, amount/quality of staff training, use of a physical activity curriculum, and offering activities that appeal to both genders. Higher servings of FV and whole grains per week were associated with the presence of a written policy regarding the nutritional quality of snacks.

Conclusions

The HAAND instrument is a reliable and valid measurement tool that can be used to assess the physical activity and nutritional environment of afterschool programs.

Introduction

Nationwide, 8.4 million youth attend afterschool programs for an average of 8.1 hours/week.¹ As a result of this extensive reach and contact time, afterschool programs are seen as an environment in which to combat childhood obesity² through increased physical activity and improvements in the nutritional quality of snacks served.³ The importance of this is reflected in recent state and national policies that outline minimal requirements for the amount of physical activity children should accumulate while attending afterschool programs, specify the nutritional quality of foods and beverages to serve during snack time, and describe the core competencies afterschool program staff should exhibit as they pertain to promoting the physical activity and nutrition of children.^4–6 Although afterschool programs are seen as settings that can potentially have a positive influence on child health, few studies have evaluated afterschool program environmental characteristics (e.g., indoor/outdoor activity opportunities, type of activity session – free-play vs organized, type of snacks offered, access to vending machines) and the extent to which afterschool programs do or do not promote physical activity and healthy eating.^{7, 8}

A wide array of quality-rating scales designed to characterize the afterschool program environment have been developed by practitioners, licensing and accreditation organizations.^9–11 At their core, quality-rating scales focus on ten essential elements related to high-quality afterschool programming. These are: (1) environment/climate, (2) relationships, (3) professional development, (4) programming/activities, (5) linkages between school and afterschool, (6) youth participation/engagement, (7) parent, family, and community partnerships, (8) program growth/sustainability, (9) measuring outcomes, and (10) evaluation.

More recently, the addition of physical activity and nutrition elements associated with afterschool program environmental quality have been endorsed by national organizations.⁶ Although there is growing evidence pointing to the influence of both the social and physical environment on children’s physical activity level and nutrition,^{12, 13} there are a limited number of tools that assess the physical activity and nutritional environment in childcare settings and afterschool programs.^13–17 Moreover, of the existing scales that include physical activity and/or nutrition, none is designed to measure the extent to which the afterschool program environment is aligned with existing physical activity and nutrition policies/standards for afterschool programs.

Although promoting physical activity in afterschool programs can lead to improvements in children’s activity,² the amount of activity children accumulate within afterschool programs is well below policy-recommended levels.^18–20 Likewise, the nutritional value of snacks served at afterschool programs falls short of existing standards,⁴ with the majority of the afterschool programs serving low–nutrient density items (e.g., chips, cookies, and sugar-sweetened beverages).²¹ If afterschool programs are to play a major role in promoting healthy lifestyles, as policies and standards^4,5 would indicate, the ability to characterize the “quality” of the physical activity and nutrition environment of afterschool programs is essential. The purpose of this paper is to describe the development of the Healthy Afterschool Activity and Nutrition Documentation (HAAND) instrument designed specifically to measure the obesogenic environment of afterschool programs.

Methods

A total of 39 afterschool programs from Columbia and Lexington SC and Omaha NE serving a total of 2073 children (kindergarten to 5^th grade) with a 1:12 staff:student ratio took part in the evaluation of the instrument. Afterschool program sites ranged in organizations (e.g., YMCA, Boys and Girls Club, individually operated) and locations (i.e., school-based, faith-based, community-based). The average duration of the afterschool program was 3 hours per day. All procedures were approved by the IRB at the University of South Carolina and the University of Nebraska Medical Center.

The HAAND was developed to assess the physical activity and nutritional environment of afterschool programs. Development of the HAAND included an extensive review of existing physical activity and nutrition environmental-quality-rating scales in both childcare and afterschool program settings,^9,10,15,17 a review of standards and policies from state and national organizations related to promoting physical activity and nutrition,^{1,4–6,22,23} and input from individuals with expertise in the field of afterschool child care. The final items included in the HAAND were aligned with existing recommendations, accreditations, and policies from the Council on Accreditation (COA),¹¹ the School-Age Care Environment Rating Scale (SACERS),¹⁰ the New York State Afterschool Network (NYSAN) Program Quality Self-Assessment Tool,⁹ and recently endorsed physical activity and nutrition standards from the National Afterschool Association.⁶

The HAAND was designed to be collected via direct observation, a brief interview with the afterschool program leader/site director, and written document review during a single day’s visit at an afterschool program. This is consistent with single-day evaluations used to evaluate child care centers and those for school-aged children for accreditation purposes (ncchildcaresearch.dhhs.state.nc.us and www.coastandards.org).

The assessment of the afterschool program environment was divided into three sections (revised HAAND tool is available in Appendix A, available at www.ajpmonline.org). The first section collects basic information regarding the program name, location (e.g., school, faith or community), weather condition during visit, the number of children attending the program, and the number of staff present the day of the visit. Included in this section are four questions related to the type of snack served at the day of the visit, whether children brought foods or beverages outside (i.e., foods not provided by the afterschool program), whether staff were observed eating or drinking foods in front of children that were not available to the children (e.g., fast food, soda), and the total time allocated for physical activity opportunities in the afterschool program schedule.

The next two sections of the HAAND consist of two separate scales: the Healthy Afterschool Program Index for Physical Activity (HAPI-PA) and nutrition (HAPI-N). These scales were based on a rubric scoring procedure with scores for each item ranging from 0 to a maximum of 4 tables (items operational definitions are given in Appendixes B and C, available online at www.ajpmonline.org). The items on the scales were derived from the quality-rating scales, policy/standards documents, and afterschool program professionals (a table showing the alignment of HAAND domain/item with existing standards is available in Appendix D, available online at www.ajpmonline.org).

For the HAPI-PA, a total of six categories consisting of ten total items were developed and for the HAPI-N, a total of seven categories, consisting of 11 items were developed (original HAAND tool). The total score for the HAPI-PA ranged from 0 to 23, whereas the total score for the HAPI-N ranged from 0 to 30. Consistent with state quality-rating scales for the child care setting,^9–11 the total score for each scale is presented as either a continuous measure (e.g., 0–23 or 0–30) or presented as an ordinal rating based on a star system, where higher scores indicate a more supportive environment. The stars ratings for the HAPI-PA were 1 to 5 = 1 star, 6 to 10 = 2 stars, 11–15 = 3 stars, 16–20 = 4 stars, and 21–23 = 5 stars, while the star ratings for the HAPI-N were 1 to 6 = 1 star, 7 to 13 = 2 stars, 14–20 = 3 stars, 21–27 = 4 stars, and 28–30 = 5 stars.

Procedures

Prior to data collection at all the afterschool program sites, the HAAND was pilot tested by two trained observers in an afterschool program to determine whether items were clear, understandable, and able to be answered by observation, interview, and/or document review. After this, revisions were made to clarify questions and items. Once the HAAND was finalized, all field observers (n=7) attended a 1-hour training session that included a review of the HAAND items and scoring system.

All 39 afterschool programs were visited once by at least one trained research assistant. Visits were scheduled with the program unit director at each site, and data were collected in the participating afterschool programs between fall 2010 and spring 2011. The document review involved examining the afterschool program schedule, parent and staff handbooks, physical activity and nutritional training documents (if available), curricula, and policy documents (if available). All the domains were covered during the personal interview. Information from the personal interview, observation, and review documents were used to score HAPI-PA and HAPI-N sections of the HAAND.

Reliability Testing

Inter-rater reliability testing was conducted on the HAAND to assess the ability of the instrument to yield consistent results across multiple raters. Concurrent HAAND data collection was carried out by two field observers at 20 of the 39 afterschool programs (20 pairs of observations, 51% of afterschool programs). Program unit directors were interviewed by one field observer while responses were recorded by both field observers present during the site visits. After the completion of the interview and direct observation, each field observer reviewed available documents from each site and independently completed the HAAND.

Validity Testing

To assess the construct validity of the HAPI-PA scale as a tool to characterize the physical activity environment in afterschool programs, the HAPI-PA items scores were compared to pedometer-determined physical activity (i.e., step counts) collected in a subsample of 934 children attending 25 afterschool programs.¹⁸ The pedometer data were collected during the program time over a period of 4 days during the same time period the HAAND data were collected. Pedometer-determined physical activity was measured using the Walk4Life MVPa pedometer, following established protocol in afterschool programs^18,24,25. The Walk4Life pedometer has acceptable reliability and validity in children^26,27. Pedometer-determined step-counts data were collected over 4 consecutive days, Monday to Thursday, with each child having an opportunity to wear a pedometer for a total of 4 days. Valid pedometer data were defined as total time of attendances 60 minutes per day and a minimum of 500 steps per day.²⁴ All 25 sites had data for all 4 days.

To assess the HAPI-N scale validity, observed snack data for the day of the site visit were used to confirm the snack menu for that week. For construct validity assessment, item scores for policy, child feedback, staff training amount and quality, parent workshops, curricula, and evaluation were compared against the mean number of times that FV and whole grains were served in the program per week collected via a program’s snack menu. HAPI-N items related to sugar-sweetened beverages and vending machine access were not compared to FV or whole grain servings per week.

Statistical Analysis

Data were analyzed in summer 2011. Inter-rater reliability was calculated to assess overall agreement for each item on the HAAND using the kappa statistic²⁸ and percent agreement.²⁹ To determine the validity of the HAPI-PA scale, means and SDs were calculated on data from 25 afterschool programs where both HAAND and pedometer steps were collected for boys and girls. Stratified analysis was conducted and comparisons were made among responses on each item on the HAPI-PA (treated as dummy variables with the lowest possible score serving as the reference group) with pedometer step count for boys and girls, separately.

All analyses used a three-level random intercept model (days nested within children nested within afterschool programs). For the HAPI-N scale, means and SDs were calculated on data from 39 afterschool programs and comparisons were made among responses using one-way ANOVAs with each items’ response categories on the HAPI-N as the independent variable and the average number of times that FV and whole grains were served in the program per week as the dependent variable. Significance was set at p<0.01.

Results

Reliability and Validity

Results for the inter-rater reliability measures are reported in Tables 1 and 2. Kappa statistics ranged from 0.70 to 1.00 (median kappa = 0.92) representing substantial to almost perfect agreement,²⁸ with percent agreement ranging from 85% to 100% (median percent agreement = 95%) across all items in the HAAND instrument. The validity of the HAPI-PA and HAPI-N items are presented in Tables 1 and 2. For physical activity, every 1-unit increase in total HAPI-PA scores was associated with an increase in the pedometers-determined step counts (70 steps, 95% CI=4, 135).

Table 1.

Description of items, inter-rater reliability, and construct validity of the HAPI-PA

Items	Level	Observeda		Reliabilityb		Construct validity: pedometer stepsc
						Girls			Boys
		n	%	% agreement	Kappa	n	M	SD	n	M	SD
Policy	No written policies	15	38.5	90	0.81	10	2407 *	1274	10	3104	1720
	Written policies, nonspecific language	23	56			14	2980 *	1940	14	3184	1943
	Written policies, explicit language (i.e., measurable)	1	2.6			1	3190	—	1	2748	—
Child feedback	None	13	33.	95	0.90	13	3149 *,†	1966	13	3496 *,†	2037
	Informal collection	20	51.3			9	2299 *	1332	9	2851 *	1736
	Formal collection	6	15.4			3	2325 †	1211	3	2956 †	1573
Staff training amount	No training for physical activity promotion	11	28.2	95	0.93	9	2437 *	1429	9	2734 *,†	1481
	Less than1 hour devoted to physical activity/year	2	5.1			—	—	—	—	—	—
	1–4 hours devoted to physical activity/yr	12	30.8			6	2382 †	1263	6	3251 *	1813
	+4 hours/year	14	35.9			10	3046 *,†	1958	10	3389 †	2087
Staff training quality	No training for physical activity	11	28.2	90	0.70	9	2437 *	1429	9	2734 *	1481
	Training delivered by noncertified personnel	20	51.3			10	2349 †	1310	10	2940 †	1729
	Training delivered by qualified professional	8	20.5			6	3544 *,†	2135	6	3915 *,†	2187
Parent workshop	None	38	97.4	100	1.00	25	2331	1740	25	3153	1867
	1/year	1	2.6			—	—	—	—	—	—
	+2/year	0	0			—	—	—	—	—	—
Curricula	None	15	38.5	100	1.00	7	2437 *	1,423	7	2894 *	1619
	Non–evidence based curriculum	15	38.5			11	2331 †	1,309	11	2806 †	1629
	Evidence-based curriculum	9	23.1			7	3499 *,†	2,104	7	3842 *,†	2154
Evaluation	None	28	71.8	100	1.00	14	2328 *	1309	14	2809 *	1599
	Limited evaluation/nonvalid methods	9	23.1			9	3305 *	2047	9	3605 *	2101
	Ongoing evaluation/nonvalid methods	2	5.1			2	2732	1387	2	3214	1711
	Limited evaluation/valid methods	0	0			—	—	—	—	—	—
	Ongoing evaluation/valid methods	0	0			—	—	—	—	—	—
						—	—	—	—	—	—
Time Allocated for physical activity	No scheduled time	0	0	90	0.80
	Less than 25% of afterschool program	7	18			2	2625	1500	2	2352 *	1340
	25%–49% of the scheduled	28	71.8			19	2817	1809	19	3222 *	1916
	50% or more of schedule	4	10.3			4	2455	1344	4	3033	1588
Types of Activities	Free play	2	5.1	85	0.73	2	2087	1101	2	2961	1449
	Limited # activities	22	56.4			11	2307 *	1357	11	2781 *	1688
	Diverse range of activities	15	38.5			12	3105 *	1907	12	3539 *	2011
Equity	Activities favor one gender	20	51.3	90	0.80	11	2235 *	1,231	11	2882 *	1723
	Activities appeal to both genders	19	48.7			14	3364 *	2,016	14	3407 *	1960

^aBased on a total of 39 afterschool programs observed. “n” = number of afterschool programs observed for a given rating. “%” = proportion of the total (N=39) afterschool programs with a given rating.

^bReliability between two observers based on a total of 20 pairs of observations.

^cPedometer step counts from 25 afterschool programs were compared using three-level random intercept models with (days within children within afterschool programs).

Note: Step counts with the same symbol indicates significance difference (p<0.01) between the means; for example, girls attending afterschool programs with a written policy with nonspecific language accumulated more steps than girls attending afterschool programs with no written policies (2980 vs 2407, p<0.01)

HAPI-PA, Healthy Afterschool Program Index for Physical Activity

Table 2.

Description of items, inter-rater reliability, and construct validity of the HAPI-N

Items	Level					Construct validity
		Observeda		Reliabilityb		Fruits and vegetables		Whole grains
		n	%	% Agreement	Kappa	M	SD	M	SD
Policy	No written policies	10	25.6	95	0.92	1.2 *	0.92	1.6 *	1.07
	Written policies, nonspecific language	13	30.8			2.4	1.39	2.85 *	0.80
	Written policies, explicit language (i.e., measurable)	16	43.6			2.6 *	0.98	2.19	0.66
Child feedback	None	22	56.4	86	0.76	2.0	1.11	2.0	0.76
	Informal collection	15	38.5			2.2	1.42	2.6	1.12
	Formal collection	2	5.1			3.5	0.71	2.5	0.71
Staff training amount	No training for healthy nutrition promotion	20	51.3	90	0.86	2.05	1.39	2.25	1.16
	Less than 1 hour devoted to physical activity/year	5	12.8			2.2	0.84	2.4	0.89
	1–4 hours devoted to physical activity/year	7	18			2.14	1.07	2.57	0.53
	+4 hours/year	7	18			2.43	1.40	1.86	0.38
Staff training quality	No training for physical activity/healthy nutrition promotion	20	51.3	90	0.84	2.05	1.39	2.25	1.16
	Training delivered by noncertified personnel	15	38.5			2.47	0.83	2.27	0.7
	Training delivered by qualified professional	4	10.3			1.5	1.73	2.25	0.5
Parent workshop	None	36	92.3	100	1.00	2.25	1.18	2.13	0.87
	1/year	2	5.1			0	0	4	0
	+2/year	1	2.6			3	—	3	—
Curricula	None	23	59	100	1.00	2.26	1.36	2.21	1.04
	Non–evidence based curriculum	16	41			2.0	1.10	2.31	0.79
	Evidence-based curriculum	0	0			—	—	—	—
Evaluation	None	27	69.2	100	1.00	1.78	1.28	2.26	1.06
	Limited evaluation/nonvalid methods	8	20.5			2.75	0.46	2	0
	Ongoing evaluation/nonvalid methods	3	7.7			3.67	0.58	2.67	1.15
	Limited evaluation/valid methods	1	2.6			3	—	3	—
	Ongoing evaluation/valid methods	—	—			—	—	—	—
Access to vending machines	No Access	7	18	100	1.00	—	—	—	—
	Full Access	32	82.1			—	—	—	—
Fruits and vegetables (number of times per week served)	0	6	15.4	95	0.94	—	—	—	—
	1	4	10.3			—	—	—	—
	2	12	30.8			—	—	—	—
	3	12	30.7			—	—	—	—
	≥4	5	12.8			—	—	—	—
Sugar-sweetened beverages (number of times per week served)	≥4	3	7.7	100	1.00	—	—	—	—
	3	0	0			—	—	—	—
	2	2	5.1			—	—	—	—
	1	1	2.6			—	—	—	—
	0	33	84.6			—	—	—	—
Whole grains (number of times per week served)	0	2	5.1	85	0.81	—	—	—	—
	1	3	7.7			—	—	—	—
	2	21	53.9			—	—	—	—
	3	9	23.1			—	—	—	—
	≥4	4	10.3			—	—	—	—

^aBased on a total of 39 afterschool programs observed. “n” = number of afterschool programs observed for a given rating. “%” = proportion of the total (N=39) afterschool programs with a given rating.

^bReliability between two observers based on a total of 20 pairs of observations.

^*Significant difference (p<0.01) between the means; for example, afterschool programs with written policies with explicit language served more fruits and vegetables compared with afterschool programs with no written policy (1.2 vs 2.6, p<0.01).

Increased pedometer steps were associated with the following items on the HAPI-PA: the presence of a written policy related to physical activity (either nonspecific or explicit), staff training received (+4 hours/year), the quality of staff training (delivered by qualified professionals), the use of an evidence-based curriculum focused on physical activity, allocating more than 25% of the daily afterschool program schedule for physical activity, providing a diverse range of physical activities, evaluation of children’s physical activity levels, and providing activities that appeal to both boys and girls. Decreased pedometer steps count was unexpectedly associated with child feedback (Table 1). For nutrition, every 1-unit increase in the total HAPI-N scores was associated with an increase in the number of FV served per week (0.21 FV servings/week, 95% CI=0.13, 0.28). Observed snack data confirmed reported snack menu offerings for the day of observation in all the participating afterschool programs. Across the 39 afterschool programs, an increase in servings of FV and whole grains per week was associated only with having some form of written policy, either nonspecific or explicit (Table 2).

Ratings of the Afterschool Programs

Using data obtained from the HAAND instrument (Tables 1 and 2), the mean total score for physical activity (HAPI-PA) was 9.0 (range: 3–14) and for nutrition (HAPI-N) 13.0 (range: 0– 20). Using a star rating system, the number of programs associated with the ratings was: 1 star (8); 2 stars (17); 3 stars (14); and no programs received either a 4-star or 5-star rating for physical activity. For nutrition, two programs were rated as 1-star, 20 as 2-star, 16 as 3-star, and no programs were rated as 4- or 5-star. The HAPI-PA indicated that the majority of the afterschool programs had nonspecific written policies (e.g., children engage in daily fitness program or enjoy age-appropriate exercise activities) regarding physical activity, collected informal feedback from children regarding the physical activity programs offered, dedicated approximately 25%–49% of the afterschool program time to physical activity, offered a limited number of activities, offered activities that favor one gender (boys), offered >4 hours of training for physical activity promotion, with training delivered by noncertified personnel. There was an equal percentage of afterschool programs that either used no physical activity curriculum or used one that was not evidence-based. The majority of the afterschool programs in this study were not conducting any form of evaluation to assess the level of physical activity of the children attending the programs and only one program indicated involving parents.

The HAPI-N scale indicated that almost half of the afterschool programs had written policies with explicit language regarding the nutritional aspects of the snacks provided at the program (e.g., snack must be made of at least two of the following four food groups; milk, bread, fruit/vegetable or 100% juice, and protein). Of the 39 afterschool programs in the study, 14 programs were enrolled in some form of reimbursement program such as those administered by the State Department of Social Services (DSS) and the U.S. Department of Agriculture (USDA). The majority of the afterschool programs did not collect child feedback regarding the snacks served at the program and reported children had full access to vending machines while at the program. One third of the afterschool programs served FV either twice (the minimal recommendations of FV per week based on the USDA’s afterschool program snack reimbursement program²³) or three times per week and only five of the afterschool programs served FV daily based on the National Afterschool Association nutrition standards,⁶ while whole grains were served two times per week at half of the afterschool programs.

The vast majority of the afterschool programs did not serve sugar-sweetened beverages. Half of the afterschool programs offered no nutritional training to staff, and of the remaining programs offering staff training on nutrition, one third of the training was delivered by noncertified personnel. Almost all of the afterschool programs did not offer any parental workshop to promote healthy eating habits. The majority of the afterschool programs did not follow any curriculum regarding nutrition education and over two thirds did not evaluate the nutritional aspect of the snacks served to the children in the program.

Discussion

The aim of this paper was to describe the development of a new environmental audit instrument (HAAND) specifically designed to measure the extent to which the afterschool program environment supports physical activity and healthy nutrition. The findings of this study provide evidence of the reliability and validity of the HAAND instrument through the collection of scores from multiple observers and a comparison of HAAND scores to objective measures in a large number of diverse afterschool programs. The majority of the items in the HAPI-N lacked validity evidence in relation to the number of FV and whole grains served per week; however, all the items in the HAPI-N scale exhibited strong content validity as the items directly reflect existing afterschool program nutrition policies/standards. Based on these findings, the HAAND instrument can be used to evaluate the extent to which afterschool program environments support physical activity and nutrition standards, in addition to identifying afterschool programs where additional assistance is required.

The strength of the HAAND as an environmental audit lies in the alignment of the HAPI-PA and HAPI-N items with current physical activity and nutrition policies and standards. The documents used to inform the items were drawn from established policies or standards from recognized organizations specifically serving afterschool programs. As part of the initial refinement of the instrument, feedback from afterschool program leaders was also obtained.

Additionally, most of the items on both the HAPI-PA and HAPI-N consistently differentiated children’s objectively monitored physical activity and the number of times FV and whole grains were served weekly. Hence, the HAAND demonstrates both strong content and construct validity. Further, the use of an objective measure of physical activity to validate the HAPI-PA environment items adds strength to utility of the HAAND instrument. Specifically, studies developing and validating environmental audits for other settings (e.g., child care) have not compared audit scores to objectively measured criteria, in this case physical activity.^15,16 Thus, this study represents one of the first attempts to validate an environmental audit with an objective measure. Despite the expected associations of many of the HAPI-PA and HAPI-N items, the presence of these characteristics was insufficient to bring physical activity or the quality of snacks served to a level comparable with existing policies/standards.^4,5 This suggests that, in addition to these characteristics, efforts targeting other afterschool program features or the synergy of all elements on the HAPI-PA need to be performed.³⁰

Several limitations of the HAAND scale need to be noted. Some of the HAPI-PA and HAPI-N items did not differentiate children’s physical activity or the quality of snacks served, such as obtaining child feedback. Additionally, afterschool programs that allocated 50% or more of their schedule to physical activity did not have more-active children. This observation indicates that scheduling physical activity opportunities during the afterschool program does not always translate to increased physical activity levels.

For the HAPI-N, items such as child feedback, level of staff training, staff training quality, and curricula were not related to FV or whole grain servings. This could be due to budget constraints that can influence the choice of the type of snacks served during the program, regardless of providing additional training or curricula. Also, the lack of differences could be due to low statistical power, with some HAPI-PA and HAPI-N responses having three or fewer afterschool programs. The USDA updated their standards at the end of this study by differentiating FV as separate items. Thus, FV were not separated in the original HAAND. This has been updated in the revised HAAND tool (Appendix A, available online at www.ajpmonline.org).

Likewise, TV and electronic media use during the afterschool programs were not included in the original HAAND. This was omitted based on the few afterschool programs that had access to TV/electronic media. Nevertheless, recent guidelines call for minimizing or excluding these altogether. These issues have also been updated in the revised HAAND tool. Finally, the use of pedometers-determined step counts to validate HAPI-PA limits the study finding to pedometer-determined steps. However, the accumulating evidence indicates that pedometers provide reliable and valid measurement of physical activity^{26, 27, 31, 32} and are strongly correlated with accelerometers.³¹

Conclusion

The results indicate that the HAAND instrument is a useful, reliable, and valid measurement tool that can be used by researchers and practitioners to assess the physical activity and nutritional environment of afterschool programs. Future studies need to focus on the training of afterschool program professionals to self-evaluate their afterschool program environment for widespread evaluation of afterschool programs nationwide.