Table 2.

Summary of Tools Reporting Psychometric Properties

Author (year)Tool name	Reliability			Validity
	Type	Analysis	Findings	Type	Analysis	Findings
Ward (2008) Bower (2008) EPAO	Interobserver (Concurrent)	For all Item: Percent agreement For Subscale: ICC¥(one-way ANOVA)	Mean percent agreement was 87.26% for observation section and 79.29% for document review section ICC values ranged from 0.45 to 0.97	Construct comparing EPAO subscales with mean activity level and % MVPA using OSRAP§	Pearson correlation	Pearson correlation: Strongest correlation between mean PA and %MVPA PA policy had weak correlation with estimate of PA (r=−0.076 to 0.157)
Benjamin (2007) NAP SACC	Test-retest (2 time over 3 wk. period)	kappa coefficients & percent agreement	Test-retest: Kappa ranged from 0.07 to 1.00; interquartile ranged from 0.27 to 0.45 percent agreement ranged from 34.3% to 100%	Face and content	Conducting a comprehensive literature and resource review	Reasonable face and content validity
	Inter-rater (concurrently using 50 triad and 9 dyads)3	kappa coefficients & percent agreement	Inter-rater: Kappa ranged from 0.20 to 1.00; Interquartile ranged from 0.45 to 0.63 and percent agreement ranged from 52.6% to 100%	Construct Expert review from Jan to April 2004		validity was reported to be established through National expert review
				Criterion comparing each question from the NAP SACC to the EPAO data from 69 childcare centers)	Weighted Kappa coefficients & percent agreement	Kappa ranged from −0.01 to 0.79 & percent agreement ranged from 0 to 93.65%
Henderson (2011) Child Care Nutrition and Physical Activity Assessment Survey				Criterion For policy & practice items survey answers were compared with in-person interview with mirroring items For Practice & environment items survey answers to direct observation data For nutrition quality items survey answers were compared to a measurement tool created for this project.	Percent agreement	Percent agreement 39% - 97% (62% item achieved ≥ 80%)
Falbe (2011) WellCCAT	Inter-rater (18 random documents coded by 2 raters independently)	ICC	For total comprehensiveness and strength score ICC was 0.98 and 0.94 respectively For Subscale ICC ranged from 0.84-0.99 respectively.	Construct compared policy quality scores for Head Stare centers to those of non-Head Stare centers and centers accredited by the National Association For Education of Young Children	simple t test	Comprehensiveness and strength scores were higher for head start centers than non-head start centers across most domains and higher for national association for education of young children accredited centers than non-accredited centers across some domain
	Internal Consistency	Cronbach's α coefficients	Cronbach's α ranged from = 0.53 to 0.83
Brener (2003) SHPP 2000	Test-retest (2 interviews) 1st interview was computer assisted 2nd interview field staff led Interview conducted 10 to 20 days apart)	kappa coefficients & Pearson correlation	School level PE Kappa ranged from 51.4% to 80.7% Classroom PE kappa ranged from 51% to 74.4% Person correlations for both school and classroom level PE questions ranged from 0.39% to 0.67% Food service, Kappa ranged from 36.6% to 88.5% and Pearson correlation coefficient ranged from 0.45 to 0.75	Construct only for the state and district level questionnaires (through a follow up a telephone interview with a subsample of the original state and district level respondent)	Comparison between the questionnaire data and interview data	Interviews with the state and district level respondents indicated that overall the questionnaire produced valid data
Lounsbery (2012) S-PAPA	Test-retest (measured 14 days apart)	kappa, percent agreement, Phi and Chi Square tests	PE module Kappa ranged from 0.14 to 0.99 and first and second administration responds had significant x2 association p values ranging from 0.001 to 0.04 with percent agreement ranging from 67% to 87% Recess module Kappa ranged from 0.33 to 0.81and first and second administration responds had significant x2 association p values ranging from <0.001 to 0.034 with percent agreement ranging from 71% to 97% For before, during and after school program kappa ranged from 0.31 to 0.84 and first and second administration responds had mostly significant x2 association p values ranging from <0.001 to 0.065 with percent agreement ranging from 61% to 87%	Content	Instrument review by content expert and PE teachers	Draft instrument was reviewed by content expert, revision was made then the revised instrument was resent to the content expert and a third draft was prepared. This draft was sent to 4 PE teachers and based on their feedback a final fourth instrument was prepared resent to PE teachers and based on their feedback final instrument was completed.
Bullock ( 2010) FoodBEAM	Inter-rater ( for researcher to researcher (4 dyads) and researcher non-researcher (5 dyads)	ICC	For both food and beverages researcher versus researcher and researcher versus non- researcher ICC ranged from 0.972 to 0.987	Convergent Comparing FoodBEAMs to the school environmental assessment tool (Samuels, 2008)	ICC scatterplot of EAT*FoodBEAMS versus percent adherence by venue to California state standards for Beverages and Food	ICC for Beverages = 0.982 and for food = 0.975 and shows that the FoodBEAMS is a valid method for collected this type of data.
Schwartz (2209) WellSAT	Inter-rater (by pairs of researcher 1 in-state and 1 out-of-state)	ICC Cronbach's alpha	For total comprehensiveness and strength ICC = 0.82 For subscale scores was 0.70 For Individual items ICC was 0.72. Cronbach's alpha ranged from 0.90 to 0.93
Kim (2010) CHLI	Inter-rater (4 sites with two interviews)	Percent agreement	93.0% school items & 84.9% afterschool items showed substantial to almost prefect agreement
Ajja (2012) HAAND	Inter-rater (concurrently)	Percent agreement kappa statistic	Percent agreement raged from 85% to 100% across all items. Kappa statistics ranged from 0.73 to 1.00 for HAPI-PA( Healthy Afterschool Program Index-Physical activity ) and 0.76 to 1.00 for HAPI-N (Healthy Afterschool Program Index-Nutrition)	Content	Items of HAAND tool were developed based on extensive literature review of the existing PA& nutrition environment quality rating, standards and policies from state and national organization and input from expertise in childcare and afterschool field	Good content validity
				Construct Pedometer step counts were compared to the HAPI-PA scores Menu from observation day was compared to number of time FV Whole grains and Sugar sweeten beverages reported on the HAPI-N	Means and standard deviation calculated and one-way ANOVA test used	HAPI-PA, → pedometer steps were significantly associated with presence of a written policy related to PA, amount/quality of staff training use of PA curriculum and offering activity that appeal to both genders For HAPI-N, higher servings of FV and whole grains per week were significantly associated with the presence of a written policy regarding the nutritional quality of snacks
Nathan (2013). (SEAT)				Construct Principals self-report using the SEAT was compared with scores from direct observations by research staff	Kappa/ PABAK coefficients & percent agreement	Percent agreement = 37% to 100% PABAK = −0.06 to 1.00

^¥ICC: intraclass correlation coefficient

^§OSRAP:_observation system for recording activity in preschools