Physical activity and healthy eating environmental audit tools in youth care settings: A systematic review

Abstract

Background

There is a growing interest in evaluating the physical activity (PA) and healthy eating (HE) policy and practice environment characteristics in settings frequented by youth (≤18 years).

Objective

This review evaluates the measurement properties of audit tools designed to assess PA and HE policy and practice environmental characteristics in settings that care for youth (e.g., childcare, school, afterschool, summer camp).

Method

Three electronic databases, reference lists, educational department and national health organizations’ web pages were searched between January 1980 and February 2014 to identify tools assessing PA and/or HE policy and practice environments in settings that care for youth (≤18 years).

Results

Sixty-five audit tools were identified of which 53 individual tools met the inclusion criteria. Thirty-three tools assessed both the PA and HE domains, 6 assessed PA domain and 14 assessed HE domain solely. The majority of the tools were self-assessment tools (n=40), and were developed to assess the PA and/or HE environment in school settings (n=33), childcare (n=12), and afterschool programs (n=4). Four tools assessed the community at-large and had sections for assessing preschool, school and/or afterschool settings within the tool. The majority of audit tools lacked validity and/or reliability data (n=42). Inter-rater reliability and construct validity were the most frequently reported reliability (n= 7) and validity types (n=5).

Conclusions

Limited attention has been given to establishing the reliability and validity of audit tools for settings that care for youth. Future efforts should be directed towards establishing a strong measurement foundation for these important environmental audit tools.

Context

From childhood to adolescence, children (age 3-18 years) are exposed to a variety of settings such as preschool, school, afterschool and summer camp. Nearly 60% of children age 3-5 years attend some type of childcare center and over 95% of youth age 5-17 years are enrolled in public/private school.¹ Additionally, over 10 million school-age children are enrolled in afterschool programs² and over 14 million youth (≤ 18 years) attend summer day camps annually.³ Given the extended contact youth have with these settings, whether these environments support or hinder physical activity and healthy eating habits is of critical importance.

In recent decades there has been an increased recognition of the role the physical environment characteristics and the policy and practice environmental characteristics plays in shaping the physical activity levels and eating habits of youth.^{4, 5} In the context of this review, physical environment characteristics refers to factors such as size and quality of structures of fixed and portable playgrounds, green fields, facility designs, esthetic etc.^6-8 Whereas, policy and practice environmental characteristics includes characteristics such as, having supportive physical activity and/or healthy eating written policy, provision of professional training on physical activity and/or healthy eating promotion to staff, scheduling of physical activity, quality of physical activity and food served, and monitoring and evaluation processes.^9-11 More recently, there has been a visible increase in the prevalence of policies and standards designed to influence settings that care for youth to be more supportive of physical activity and healthy eating habits.^{1, 10, 11} Examples of these include “wellness” policies in school settings that dictate the amount and quality of daily physical education students must receive per week during the school year and/or the type of foods and beverages sold or served at schools.

In response, a wide array of audit tools designed to assess policy and practice environmental characteristics have been developed. Audit tools come in a variety of forms, such as questionnaires, checklists, observation scales, and surveys. These tools are designed to capture information pertaining to the alignment or presence of physical activity and healthy eating environmental characteristics of a given setting with existing state or national policies, standards, or scientific position statements.^{7, 12-14} The extent to which audit tools designed to assess policy and practice environmental characteristics provide an accurate reflection of such settings, however, remains unknown.

If audit tools are to provide credible information aimed at informing current and future policy decisions regarding the adoption or implementation of supportive policy and practice physical activity and healthy eating interventions,^15-18 it is of critical importance that such tools demonstrate: (1) an acceptable level of reliability (defined as the ability of the tools to consistently capture the same information with repeated use and/or when used by two or more users) and (2) validity (referred to as the ability of the tools to accurately measure what they were designed or intended to measure)¹⁹. To the authors’ knowledge, no reviews have examined audit tools designed to assess policy and practice environmental characteristics in the wide range of settings that care for youth. Therefore, the aim of this review is to identify and examine the quality of policy and practice environmental audit tools currently in use at various settings caring for youth.

Evidence acquisition

Literature Search

A systematic literature search was conducted to identify tools assessing policy and practice environmental characteristics related to physical activity and healthy eating in settings that care for youth (3-18 years). Three electronic databases: PubMed, Web of Science, and CINAHL were searched for all relevant articles published between January 1980 and February 2014. Search strategies for the databases included the following key words: population (child, youth, adolescent); settings [(preschool, childcare, homecare (residential children homes)], school, afterschool, summer camp); apparatus (tool, kit, instrument, index, survey, questionnaire, checklist, audit); quality (assessment, development, validity, reliability); and area (environmental, policy, standards, benchmarking, physical activity and nutrition). In addition to database searches, reference lists of identified articles were screened in order to identify additional tools to include in the review.^{12-14, 20-27}

Tools were also sourced from the following national education departments and health organizations’ web pages: National Cancer Institute, Active Living Research, Robert Wood Johnson Foundation, Center for Diseases Control and Prevention (CDC), Yale Rudd Center for Food Policy and Obesity, National Association of School Nurses, USDA's “Changing the Scene” and National Association of State Boards of Education (NASBE). The following keyword combinations were used when conducting an electronic search of national education departments and health organization web pages: wellness, policies, tool (kit), audit, assessment, resources, measurements, school (pre-, after-), summer camp, and homecare (i.e., residential children homes).

Eligibility Criteria

Tools were included in the review if they met the following inclusion criteria: (1) the tool as a whole or sections of the tool assessed physical activity and/or healthy eating policy and practice environmental characteristics (e.g., written policies, provision of professional training on physical activity and/or healthy eating promotion and the credentials of staff delivering the training, scheduling of physical activity and/or snack/meals, quality of physical activity and food served, monitoring and evaluation processes), (2) the setting assessed included one or more of the following: preschool, school, afterschool, summer camp, residential children homes, (3) the tool could be used by researchers and/or non-research affiliated staff in the field, (4) it was an English language publication, and (5) the tool was available electronically or through communication with the authors. Two independent reviewers (RA and JC) screened and selected the audit tools included in the review based on the above inclusion criteria. Tools were excluded from this review if they (1) only assessed the physical environment (e.g., facilities, room space, playground features, green field, etc.), (2) were designed to evaluate strategies for meeting national/state policy recommendations, or (3) were a non-English publication. For the purpose of this review, we only included articles reporting psychometric properties as part of the tool development/testing procedure.

Selection of Tools

The electronic search strategies were executed by two independent researchers (RA and JC). Disagreements were discussed and resolved, and, if required, a third reviewer (MWB) was consulted. A copy of the latest version of the tools included in the review was retrieved, and when available, the full text papers reporting on tool measurement properties that fulfilled the inclusion criteria were also retrieved.

Description of Tools

The following information was extracted from the tools included in this review: (i) name of the tool, (ii) developer; (iii) the purpose of the tool development; (iv) setting; (v) intended users; (vi) data collection method; (vii); time frame needed to complete the tool; (viii) number of items in the tool; and (ix) domains (e.g. written policy, child feedback, time allocated for physical activity, type of activity, staff professional training, screen time, time allocated for snack/meals, meal quality, evaluation, etc.,) assessed by the tool. In addition, when psychometric (i.e., reliability and/or validity) information of the tool was available, the following information was extracted: (i) type of validity and or/reliability evaluated; (ii) time frame for reliability testing (test-retest); (iii) type of analysis used; (iv) validity comparison, and (v) reliability and validity findings.

Evidence synthesis

Description of Tools

A total of 123 tools were identified from the initial search of the three databases, review of references from these articles, and from a search of national health organizations/agencies’ web pages. After excluding duplicates, 65 tools were retained, of which 53 tools were included in this review based on the inclusion/exclusion criteria (Figure 1).

Figure 1.

Tool selection process.

Table 1 presents summaries of the audit tools included in this review. Policy and practice environmental characteristics were evaluated solely in 34 tools^28-62 compared to 19 tools^{27, 63-80} which assessed both policy and practice environmental characteristics and the physical characteristics. Physical activity and healthy eating domains were assessed in 33 tools27-29, 34-36, 39-44, 46-52, 54, 61, 63-69, 72, 75, 78-80 compared to six tools^{33, 38, 70, 74, 76, 77} that assessed only physical activity domain and 14 tools^{30-32, 37, 45, 53, 55-59, 62, 71, 73} that assessed only healthy eating domain.

Table 1.

Description of Environmental Audit Tools assessing Healthy Eating and Physical Activity

Tool Name	Setting	Developer	Purpose	User
				Staff/community member (Self-assessment)	Researcher
Child Care Nutrition and Physical Activity Assessment Survey	Childcare	Rudd Center for Food Policy and Obesity, Yale University.	To evaluate nutrition and physical activity environment of child care centers.	•
Child Care Nutrition and Physical Activity Policies- Communication & Promotion	Childcare	Connecticut State Department of Education.	To assess communication level and health promotion strategies of childcare centers in the state of Connecticut.	•
Child Care Nutrition and Physical Activity Policies- Eating Environment	Childcare	Connecticut State Department of Education.	To assess nutrition standards of childcare centers in the state of Connecticut.	•
Child Care Nutrition and Physical Activity Policies- Evaluation	Childcare	Connecticut State Department of Education.	To assess evaluation policies of childcare centers in the state of Connecticut.	•
Child Care Nutrition and Physical Activity Policies- Nutrition Education	Childcare	Connecticut State Department of Education.	To assess nutrition education of childcare centers in the state of Connecticut.	•
Child Care Nutrition and Physical Activity Policies- Nutrition Standard	Childcare	Connecticut State Department of Education.	To assess nutrition standards of childcare centers in the state of Connecticut.	•
Child Care Nutrition and Physical Activity Policies- Physical Activity	Childcare	Connecticut State Department of Education.	To assess nutrition standards of childcare centers in the state of Connecticut.	•
Childcare director interview	Childcare	Rudd Center for Food Policy and Obesity, Yale University.	To assess nutrition and physical activity environment at childcare settings.		•
Environment and Policy Assessment and Observation (EPAO)	Childcare	Ward et al., Center for Health Promotion and Disease Prevention, University of North Carolina at Chapel Hill.	To evaluate the Nutrition and Physical Activity Self-Assessment for Child Care Program (NAP SACC).		•
Nutrition and Physical Activity Self-Assessment for Child Care Program (NAP SACC)	Childcare	Ward, et al., Nutrition and Physical Activity Self-Assessment for Child Care (NAP SACC), Center for Health Promotion and Disease Prevention and Department of Nutrition, University of North Carolina at Chapel Hill.	Developed for the Nutrition and Physical Activity Self-Assessment for Child Care Program (NAP SACC) intervention.	•
Study of Healthy Activity and Eating Practices and Environments in Head Start (SHAPES) Self-assessment Survey	Childcare	Whitaker, et al, Department of Public Health and Pediatrics, Center for Obesity Research and Education, Temple University.	To evaluate nutrition and physical activity environments in childcare setting.	•
Wellness Child Care Assessment Tool (WellCCAT)	Childcare	Falbe et al., Rudd Center for Food Policy and Obesity, Yale University.	To assess written health-related polices (i.e., nutrition and physical activity and wellness polices).		•
Abbreviated Wellness School Assessment Tool (WellSAT)	School	Robert Wood Johnson Foundation Healthy Eating Research Program, Working Group 1.	To evaluate the quality of existing schools' district wellness policies.		•
Competitive Foods and Beverages Toolkit	School	Alliance for Healthier Generation.	To help schools evaluate the presence of competitive food and Beverages at their schools.	•
Food and Beverage Environment Analysis and Monitoring System (FoodBEAM)	School	Samuels & Associates.	Developed to capture the following: •Venues where competitive foods and beverages are sold. •Types of foods and beverages sold. Compliance of foods and beverages with the California school nutrition standards for competitive foods.		•
Food and Fitness School Health Policies and Practices Questionnaire	School	Turner, Bridging the Gap Research Program.	Developed as part of a study to assess school s' health policy and programs.	•
Gold Medal Rating Scale – Elementary School	School	Massachusetts Action for Healthy Kids supported by the MetroWest Community Health Care Foundation.	Developed as part of Action for Healthy Kids initiative for schools to assess their local wellness policies.	•
Gold Medal Rating Scale – Middle & High School	School	Massachusetts Action for Healthy Kids supported by the MetroWest Community Health Care Foundation.	Developed as part of Action for Healthy Kids initiative for schools to assess their local wellness policies.	•
Illinois Needs Assessment & Evaluation Tool	School	Illinois State Board of Education.	Developed to evaluate the local schools wellness polices and practice in Illinois.	•
Local Wellness Policy	School	National Team Nutrition Office for the Colorado Healthy Schools Summit.	Developed to assess local schools wellness programs.	•
Michigan's Healthy School Action Tools (HSAT)- Nutrition service	School		Developed for Michigan schools to create healthier environments initiative.	•
Michigan's Healthy School Action Tools (HSAT)- Physical education and other physical activity opportunities	School		Developed for Michigan schools to create healthier environments initiative.	•
Michigan's Healthy School Action Tools (HSAT)- School Health & Safety Policies	School		Developed for Michigan schools to create healthier environments initiative.	•
Mississippi School Nutrition and Physical Activity Environment Assessment	School	Mississippi Department of Education.	Developed for Mississippi schools to evaluate their health and wellness environment.	•
Neumark-Sztainer Food Policies and Practices questionnaire	School	Developed by the Minnesota Association of Secondary School Principals and the University of Minnesota, Division of Epidemiology.	To evaluate high school food policy and practice environment.	•
New Hampshire School Wellness Policy Assessment Form	School	New Hampshire Department of Education Local.	Developed to evaluate the New Hampshire schools wellness policies and practice complete school environment.	•
Policy and Systems Toolkit	School	Alliance for Healthier Generation.	Developed to be used by schools enrolled in the Healthy Schools Program	•
Principals Survey	School	Lytle et al, Division of Epidemiology, University of Minnesota.	Developed as part of the TEENS intervention project.		•
Rhode Island Nutrition & PA survey	School	Rhode Island Healthy Schools Coalition.	Developed for Rhode Island schools to assess their school environment with respect to nutrition and physical activity.	•
Rhode Island Needs Assessment Tool (RINAT)	School	Pearlman, Rhode Island Department of Health..	Developed as part of needs assessment and intervention project in Rhode Island schools.		•
School Environment Assessment Tool (SEAT)	School	Nathan et al, Hunter New England Population Health and School of Medicine and Public Health, The University of Newcastle, Australia.	Developed to assess quality of school food and physical activity environment.		•
School food policies and practices: a state-wide survey of secondary school principals	School	French 2002. University of Minnesota, Division of Epidemiology.	To evaluate food related policies and practices in secondary schools in Minnesota.	•
School Health Index (SHI) (2012) - Elementary School	School	Centers for Disease Control and Prevention.	Developed for schools to assess health and safety policy and for planning.	•
School Health Index (SHI) (2012) - Middle/High school	School	Centers for Disease Control and Prevention.	Developed for schools to assess health and safety policy and for planning.	•
School Health Policies and Practices Study (SHPPS 2006) questionnaire- Nutrition	School	Centers for Disease Control and Prevention.	Developed for the School Health Policies and Practices study.		•
School Health Policies and Practices Study (SHPPS 2006) questionnaires- Physical Education and Activity	School	Centers for Disease Control and Prevention.	Developed for the School Health Policies and Practices study.		•
School Health Policies and Practices Study (SHPPS 2006) questionnaires- School Policy & Environment	School	Centers for Disease Control and Prevention.	Developed for the School Health Policies and Practices study.		•
School Meals Program Toolkit	School	Alliance for Healthier Generation.	Developed to be used by schools to evaluate the quality of the school meals provided as part of assessing school wellness policies and practice programs.	•
School Nutrition by Design	School	California Department of Education Nutrition Services Division.	Developed as part of the recommendation of State Superintendent Advisory Committee on Nutrition Implementation Strategies.	•
School Physical Activity Policy Assessment (S-PAPA)	School	Lounsbery et al., University of Nevada, Las Vegas	Developed to assesses physical activity policy at the district & school level.	•
Student Wellness Toolkit – Elementary school	School	Alliance for Healthier Generation.	Developed to be used by elementary schools to evaluate overall school wellness policies and practice	•
Student Wellness Toolkit – High School	School	Alliance for Healthier Generation.	Developed to be used by high schools to evaluate overall school wellness policies and practice	•
Student Wellness Toolkit – Middle School	School	Alliance for Healthier Generation.	Developed to be used by middle schools to evaluate overall school wellness policies and practice	•
Survey of school vending machines	School	Johanson and Wootan. Center for Science in the Public Interest (CSPI).	Developed as part of the CSPI nutrition policy project to evaluate the nutrition quality of food in school vending machines.	•
Wellness School Assessment Tool (WellSAT-96)	School	Robert Wood Johnson Foundation Healthy Eating Research Program, Working Group 1.	To provide a standard method for assessing school district wellness policies.		•
Community Healthy Living Index (CHLI)	Community	Y-USA collaborated with Stanford, Harvard, and St. Louis Universities.	To examine environmental and social supports for healthy eating and active living.	•
Healthy Community Checklist	community	Michigan Healthy Communities Collaborative.	To assess community's health environment with regard to promoting and supporting: •Physical Activity. •Healthy Eating & Healthy Weight. Smoke-Free Environments & Tobacco-Free Lifestyles.	•
Nutrition Environment Assessment Tool (NEAT) – section 3 (school)	Community	Michigan Healthy Community Collaboration.	Developed to help communities assess how supportive their environment is to healthy eating.	•
The Environmental Nutrition and Physical Activity Community Tool (ENACT)	Community	Strategic Alliance (California).	Developed to help community assess current policy status and develop an action plan.	•
2 Minute Program Assessment	Afterschool	Harvard School of Public Health Prevention Research Center as part of the Out of School Nutrition and Physical Activity (OSNAP) Initiative.	To assess how closely program adheres to the OSNAP nutrition and physical activity environmental standards.	•
Healthy Afterschool Activity and Nutrition Documentation Instrument (HAAND)	Afterschool	Ajja et al., Arnold School of Public Health, University of South Carolina, Columbia.	To assess the extent to which the afterschool environment meets current physical activity and nutrition policies.	•	•
Policy assessment tool	Afterschool	Harvard School of Public Health Prevention Research Center as part of the Out of School Nutrition and Physical Activity (OSNAP) Initiative.	To identify existing nutrition, physical activity and screen time polices.	•
Program self-assessment observation tool	Afterschool	Harvard School of Public Health Prevention Research Center as part of the Out of School Nutrition and Physical Activity (OSNAP) Initiative.	To assess the nutrition and physical activity of program during the OSNAP intervention.	•

Data collection method				Time frame	No. of items	Domain of physical activity environment covered		Domain of nutrition environment covered		Note
Observation	Document review	Interview	Self-report			Environmental (Policy/ practice) characteristics	Physical characteristics	Environmental (Policy/practice) characteristics	Physical characteristics
•	•			1 day site visit	43	Policy, activity types , screen time, staff behavior, training barriers and support	Equipment, space	Policy, menu quality, meal schedule, food safety, staff behavior, training, curriculum, access (water)		Close-ended questions with appropriate responses as follows: - Choose one response category from several possible answers.
	•			Not reported	12	Policy, staff behavior		Policy, staff behavior, advertising		Close-ended questions with appropriate responses as follows: - Full/Partial/None/NA
	•			Not reported	31			Policy, meal schedule, food safety, staff behavior, training		Close-ended questions with appropriate responses as follows: - Full/Partial/None/NA
	•			Not reported	6	Policy, evaluation		Policy, evaluation		Close-ended questions with appropriate responses as follows: Full/Partial/None/NA
	•			Not reported	17			Policy, staff behavior, curriculum, advertising		Close-ended questions with appropriate responses as follows: - Full/Partial/None/NA
	•			Not reported	51			Policy, menu quality, meal schedule, food safety, access (water, vending machines), fundraising		Close-ended question with appropriate responses as follows: - Full/Partial/None/NA.
	•			Not reported	45	Policy, amount of time allocated, activity types , screen time, staff behavior, curriculum	Equipment, space, safety			Close-ended questions with appropriate responses as follows: - Full/Partial/None/NA
		•		Not reported	73	Policy, amount of time allocated, screen time, staff behavior, training barriers and support		Policy, staff behavior/modeling and training, nutrition curriculum, barriers and support, fundraising		Close-ended questions with appropriate responses as follows: - Yes/No - Choose one response category from several possible answers.
•	•			1 full day visit	192	Policy, amount of time allocated, activity types, screen time, staff behavior and training, curriculum	Space, equipment, safety	Policy, menu quality, meal schedule, staff behavior/modeling, training, nutrition curriculum, access (water, vending machines), fundraising	Vending machine location	Close-ended questions with appropriate responses as follow: - yes/no - Choose one response category from several arranged in hierarchical order.
•	•			1 full day visit	56	Policy, amount of time allocated, activity types, screen time, staff behavior, training	Equipment, space	Policy, menu quality, meal schedule, staff behavior, training, access (water/vending machines), fundraising		Close-ended questions. - Each question has 4 possible response options ranging from minimum standard to best practice).
			•	30 min	90	Policy, amount of time allocated, screen time, curriculum	Space, equipment	Policy, menu quality, staff behavior/modeling, curriculum, fundraising		Close-ended questions with appropriate responses as follows: - Yes/No
	•			N/A	64	Policy, amount of time allocated, activity types, screen time, staff behavior, training, curriculum, evaluation	Safety, space	Policy, menu quality, meal schedule, staff behavior , training, access (water), curriculum, evaluation, fundraising		Close-ended questions with 4 possible response options ranging from: - 0, 1, 2, NA.
	•			45 min	50	Policy, amount of time allocated, activity types, , staff behavior, training and credentials, curriculum, evaluation,	Equipment	Policy, child involvement, menu quality, meal schedule, staff behavior, training and credentials, food safety, access (water, vending machines), curriculum, advertising, fundraising,		School policy statement are rated “0” “1” or “2” rating “3” “4” only apply to specific section 3
			•	Not reported	8			Policy, access (vending machines).		Uses best practice framework of criteria at the bronze, silver and gold levels as a way of scoring the program policy.
•				Varies based on school size and number, location where food is sold	N/A			Policy, menu quality, access (vending machines), advertising, fundraising.		Online assessment tool. Data collector enters the information guided by the software that matches the information items with nutrient profile which is housed in the nutrient database in imbedded in the software. Software has a drop-down menu as well as the ability to add new items not in the database.
	•			Not reported	100	Policy, amount of time allocated, activity types, staff credentials, curriculum, barriers and support.	Equipment	Policy, menu quality, meal schedule, access (vending machines), curriculum, fundraising.		Combination of close- and open-ended questions: Close-ended questions: possible responses - Yes/ No - Choose one response category from several possible answers Likert scale
			•	Not reported	29	Policy, amount of time allocated, activity types, staff behavior, training.	Safety			Close-ended questions with possible responses. Choose one response category from several arranged in hierarchical order.
			•	Not reported	33	Policy, amount of time allocated, staff behavior, training, curriculum.	Safety			Close-ended questions with possible responses. Choose one response category from several arranged in hierarchical order.
			•	Not reported	49	Policy, amount of time allocated, staff behavior training.	Safety	Policy, child involvement, menu quality, meal schedule, food safety, staff behavior, training, advertising.	Food facility	Close-ended questions with possible responses as follows: - OK/Need For Improvement
			•	Not reported	40	Amount of time allocated, staff behavior and credentials, curriculum.	Space	Menu quality, meal schedule, food facility, staff behavior, training, access (water), fundraising.	Food facility	Close-ended questions with possible responses as follows: - fully implemented - partially implemented - still in planning - not applicable - Don't know
			•	2 to 7 hours	51			Policy, menu quality, meal schedule, staff behavior , training and credentials, fundraising.		Combination of open ended and close ended questions with possible reposes ranging from: - Yes/No Choose the most appropriate statement
			•	2 to 7 hours	92	Policy, amount of time allocated, activity types, staff behavior, training and credentials, curriculum.	Equipment			Combination of open-ended and close-ended questions with possible responses ranging from: - Yes/No. Choose the most appropriate statement.
			•	2 to 7 hours	353	Child involvement, amount of time allocated, activity types, staff behavior, training, evaluation.	Equipment, space, safety	Policy, child involvement, menu quality, meal schedule, food safety, staff behavior, training, access (water, vending machine), evaluation, food safety, advertising, fundraising.	Food facility	Combination of open-ended and close-ended questions with possible responses ranging from: - Yes/No. Choose the most appropriate statement.
			•	Not reported	57	Amount of time allocated, screen time, curriculum.	Equipment	Policy, child involvement, menu quality, meal schedule, food safety, food facility, staff training, access (water, vending machines), curriculum, advertising		Close-ended questions: Choose one response category from several arranged in hierarchical order.
			•	Not reported	36			Policy, access (vending machines).		Close-ended questions possible responses: - Yes/No - Likert scale. - Choose one response category from several arranged in hierarchical order.
			•	Not reported	50	Policy, type, staff behavior, training, credentials, evaluation		Policy, nutrition curriculum, staff behavior, , training, meal schedule, evaluation, fundraising.		Close-ended questions with - Yes /No response Points reported as numeric scores and percentages
			•	Not reported	8	Policy, child involvement.		Policy, menu quality, food facility, staff behavior, training.		Uses best practice framework of criteria at the bronze, silver and gold levels as a way of scoring the program policy.
		•		Not reposted	22			Policy, menu quality, staff training, advertising		Combination of open-ended and close-ended questions: - Close-ended questions possible responses: Yes/No/Don't know
			•	Not reported	49	Amount of time allocated, staff behavior, staff credentials.	Equipment, space,	Policy, menu quality, meal schedule, staff behavior, training, access (vending machine), curriculum		Combination of close- and open-ended questions. Close-ended questions possible responses: - Yes/No
		•		Not reported	40	Policy, child involvement, amount of time allocated, barriers and support.	Space	Policy, child involvement, barriers, advertising, access (vending machines).		Combination of open-ended and close-ended questions: Close-ended questions possible response answers: - Yes/No - Likert scale
		•		20 min	65	Amount of time allocated, screen time.	Facility, equipment,	Menu quality, access (water, vending machines), fundraising.		Close-ended questions with possible responses: Yes/ No/Don't know
			•	Not reported	36			Polices, menu quality, access (vending machines), attitudes, advertising, fundraising.		Close-ended questions
			•	6 hr	105	Policy, amount of time allocated, activity types, staff behavior, training and credentials	Space, safety	Policy, menu quality, meal schedule, food safety, staff behavior, training and credentials, access (water), curriculum, evaluation, advertising, fundraising		Close-ended questions with possible responses: Fully in place/partially in place/underdeveloped/not in place
			•	6 hr	122	Policy, amount of time allocated, activity types , staff training and credentials, curriculum	Space, safety	Policy, menu quality, meal schedule, food safety, staff behavior, training and credential, access (water), curriculum, evaluation, advertising, fundraising		Close-ended questions with possible responses: Fully in place/partially in place/underdeveloped/not in place
			•	40 min	88			Child involvement, menu quality, meal schedule, food safety, staff training, staff credentials, access (water), evaluation		Combination of open-ended and close-ended questions: - Close ended questions possible responses: Yes / No, Likert scale
			•	60 min	114	Amount of time allocated, activity types , staff training and credential, evaluation				Combination of open-ended and close-ended questions: - Close-ended questions with possible responses: Yes/No, Likert scale
			•	60 min	201	Policy, amount of time allocated		Policy, menu quality, meal schedule, access (water) evaluation, fundraising		Combination of open-ended and close-ended questions: Close-ended questions possible responses: - Yes/No - Likert scale
			•	Not reported	29			Policy, menu quality, food facility, staff behavior, training.		Uses best practice framework of criteria at the bronze, silver and gold levels as a way of scoring the program policy.
			•	Not reported	36			Policy, child involvement, staff behavior, training, access (vending machine), curriculum, evaluation, fundraising.		Close-ended questions. Choose one response category from three categories arranged in hierarchical order
•	•			30 min	96	Policy, amount of time allocated, activity types, staff behavior, training and credential, curriculum, evaluation.	Space			Close ended questions with possible responses: - Yes/No/Don't know - No/Partially/Yes Choose one response category from several arranged in hierarchical order
			•	Not reported	11	Policy, amount of time allocated, activity types, staff training, curriculum		Policy, staff training, curriculum.		Uses best practice framework of criteria at the bronze, silver and gold levels as a way of scoring the program policy. Close-ended response option: choose the most appropriate statement.
			•	Not reported	10	Policy, amount of time allocated, activity types, staff training, curriculum.		Policy, menu quality, curriculum.		Uses best practice framework of criteria at the bronze, silver and gold levels as a way of scoring the program policy.
			•	Not reported	8	Policy, amount of time allocated, activity types, curriculum.				Uses best practice framework of criteria at the bronze, silver and gold levels as a way of scoring the program policy.
•				Not reported	18			Snack/beverages quality.		Open-ended questions (listing how many selections of each type of food and drink items available)
	•			Not reported	96	Policy, amount of time allocated, activity types, staff behavior, training and credentials, curriculum, evaluation	Equipment, safety, space	Policy, child involvement, menu quality, meal schedule, staff behavior, training and credentials, food safety, access (water, vending machines) curriculum, advertising, fundraising		School policy statement are rated “0” “1” or “2”.
	•			Not reported	160 (childcare) 123 (school) 110 (afterschool)	Policy, child/parent involvement, amount of time allocated activity types, screen time, staff behavior, training and credentials, curriculum evaluation.	Equipment, safety, space	Policy, menu quality, meal schedule, staff training, access (water), fundraising.		Combination of close- and open-ended questions. Close-ended questions with possible responses: Choose one response category from several arranged in hierarchical order - Yes/No
	•			Not reported	8 (schools)	Polices.		Policy		Combination of close- and open-ended questions. Close-ended questions possible responses: Choose one response category from several arranged in hierarchical order. Choose all response categories that apply
	•			1 to 4 hours	37(school)			Policy, access (water), menu quality, staff training, advertising.		Combination of open- and close-ended questions. Close-ended questions with possible responses. Choose all that applies to your program
			•	Not reported	8 (Childcare) 6 (school) 7 (afterschool)	Policy, amount of time allocated, activity types, staff credentials.	Space	Policy, menu quality, meal schedule, staff training, access (water), fundraising.		Close-ended questions with possible responses. Choose one response category from several arranged in hierarchical order ranging from 1 (elements not in place) to 5 (elements are culturally appropriate, accessible and available).
			•	2 min	9	Amount of time allocated, screen time.		Menu quality, access (water)		Close-ended questions with appropriate response as follows: - Yes/No/In Progress/Unsure answer
•	•		•	1 day site visit	23	Policy, child involvement, amount of time allocated, activity types, gender equity, screen time, staff training, credentials, curriculum, evaluation.		Policy, child involvement, menu quality, access (vending machines), staff training, credentials, curriculum, evaluation.		Close-ended questions. Choose one response category from several arranged in hierarchical order.
	•			Not reported	10	Policy, amount of time allocated, activity types, screen time.		Policy, menu quality, access (water)		Close-ended questions with appropriate response as follows with appropriate response: - Yes/No/In Progress/Unsure answer
•				Program length	27	Amount of time allocated, activity types, screen time.		Menu quality, access (water)		Combination of close-ended and open-ended questions. Close-ended questions with appropriate response as follows: - Yes/No

School was the setting with the most tools assessing physical activity and/or healthy eating environments (n= 33)^{27, 36-41, 45-47, 54-56, 58, 59, 61-64, 69-78, 81} followed by childcare settings (n= 12).^{28-35, 65-68} There were 4 tools^48-51 evaluating afterschool settings and 4 tools^{52, 53, 79, 80} evaluating community settings with sections dedicated to evaluating childcare, school, and/or the afterschool setting. Forty out of the 53 tools30-35, 42-47, 49-56, 58, 61, 63-66, 68-70, 72-74, 76-80 were categorized as self-assessment tools designed to be used by staff/community members, 12 tools^{27-29, 36-41, 59, 67, 71} were designed to be completed by researchers/public health practitioners for research purposes or for assessments within specific projects, and a single tool⁴⁸ was intended to be used by both researchers and staff members.

The majority of the tools assessing physical activity focused on items such as written policies (n=31) and time allocation (n=31). A considerable number of tools included items such as activity types (n=26), staff training (n=20), curriculum (n=19), staff behavior (n=16), staff credentials (n=16), and screen time (n=14). Fewer tools included items such as evaluation and monitoring process (n=10), parent workshop (n=8), child involvement (n=5), and barriers and support (n=4). When healthy eating was evaluated, the majority of tools focused on written policies (n=40) and menu quality (n=30). The majority of tools included staff training (n=26), behavior (n=19), access to water (n=21), access to vending machines (n=18), curriculum (n=18), food safety (n=12) and child involvement (n=12). Fewer tools included meals/snack schedules (n=10), parent workshops (n=10), evaluation (n=10), staff credentials (n=9), and barriers and support (n=2).

Reliability

Inter-rater reliability (Table 2) was the most commonly tested type of reliability (n=7)^{13, 21, 23-26} followed by test-retest (n=3),^{21, 82, 83} and internal consistency (n=1).²⁵ For reliability assessment, studies reported Pearson correlation, Cronbach's α, kappa coefficient, percent agreement and/or interclass correlation coefficient (ICC) scores. The following tools had the highest reliability coefficients: the Wellness Child Care Assessment Tool (WellCCAT)²⁵ with an ICC ranging from 0.84-0.99; the Food and Beverage Environment Analysis and Monitoring System (FoodBEAM),²⁶ with an ICC ranging from 0.97-0.99; the Community Healthy Living Index (CHLI),²³ with percent agreement ranging from 84%-93%; and the Healthy Afterschool Activity and Nutrition Documentation (HAAND)¹³ with percent agreement ranging from 85%-100% and kappa coefficients ranging from 0.73-1.00.

Table 2.

Summary of Tools Reporting Psychometric Properties

Author (year)Tool name	Reliability			Validity
	Type	Analysis	Findings	Type	Analysis	Findings
Ward (2008) Bower (2008) EPAO	Interobserver (Concurrent)	For all Item: Percent agreement For Subscale: ICC¥(one-way ANOVA)	Mean percent agreement was 87.26% for observation section and 79.29% for document review section ICC values ranged from 0.45 to 0.97	Construct comparing EPAO subscales with mean activity level and % MVPA using OSRAP§	Pearson correlation	Pearson correlation: Strongest correlation between mean PA and %MVPA PA policy had weak correlation with estimate of PA (r=−0.076 to 0.157)
Benjamin (2007) NAP SACC	Test-retest (2 time over 3 wk. period)	kappa coefficients & percent agreement	Test-retest: Kappa ranged from 0.07 to 1.00; interquartile ranged from 0.27 to 0.45 percent agreement ranged from 34.3% to 100%	Face and content	Conducting a comprehensive literature and resource review	Reasonable face and content validity
	Inter-rater (concurrently using 50 triad and 9 dyads)3	kappa coefficients & percent agreement	Inter-rater: Kappa ranged from 0.20 to 1.00; Interquartile ranged from 0.45 to 0.63 and percent agreement ranged from 52.6% to 100%	Construct Expert review from Jan to April 2004		validity was reported to be established through National expert review
				Criterion comparing each question from the NAP SACC to the EPAO data from 69 childcare centers)	Weighted Kappa coefficients & percent agreement	Kappa ranged from −0.01 to 0.79 & percent agreement ranged from 0 to 93.65%
Henderson (2011) Child Care Nutrition and Physical Activity Assessment Survey				Criterion For policy & practice items survey answers were compared with in-person interview with mirroring items For Practice & environment items survey answers to direct observation data For nutrition quality items survey answers were compared to a measurement tool created for this project.	Percent agreement	Percent agreement 39% - 97% (62% item achieved ≥ 80%)
Falbe (2011) WellCCAT	Inter-rater (18 random documents coded by 2 raters independently)	ICC	For total comprehensiveness and strength score ICC was 0.98 and 0.94 respectively For Subscale ICC ranged from 0.84-0.99 respectively.	Construct compared policy quality scores for Head Stare centers to those of non-Head Stare centers and centers accredited by the National Association For Education of Young Children	simple t test	Comprehensiveness and strength scores were higher for head start centers than non-head start centers across most domains and higher for national association for education of young children accredited centers than non-accredited centers across some domain
	Internal Consistency	Cronbach's α coefficients	Cronbach's α ranged from = 0.53 to 0.83
Brener (2003) SHPP 2000	Test-retest (2 interviews) 1st interview was computer assisted 2nd interview field staff led Interview conducted 10 to 20 days apart)	kappa coefficients & Pearson correlation	School level PE Kappa ranged from 51.4% to 80.7% Classroom PE kappa ranged from 51% to 74.4% Person correlations for both school and classroom level PE questions ranged from 0.39% to 0.67% Food service, Kappa ranged from 36.6% to 88.5% and Pearson correlation coefficient ranged from 0.45 to 0.75	Construct only for the state and district level questionnaires (through a follow up a telephone interview with a subsample of the original state and district level respondent)	Comparison between the questionnaire data and interview data	Interviews with the state and district level respondents indicated that overall the questionnaire produced valid data
Lounsbery (2012) S-PAPA	Test-retest (measured 14 days apart)	kappa, percent agreement, Phi and Chi Square tests	PE module Kappa ranged from 0.14 to 0.99 and first and second administration responds had significant x2 association p values ranging from 0.001 to 0.04 with percent agreement ranging from 67% to 87% Recess module Kappa ranged from 0.33 to 0.81and first and second administration responds had significant x2 association p values ranging from <0.001 to 0.034 with percent agreement ranging from 71% to 97% For before, during and after school program kappa ranged from 0.31 to 0.84 and first and second administration responds had mostly significant x2 association p values ranging from <0.001 to 0.065 with percent agreement ranging from 61% to 87%	Content	Instrument review by content expert and PE teachers	Draft instrument was reviewed by content expert, revision was made then the revised instrument was resent to the content expert and a third draft was prepared. This draft was sent to 4 PE teachers and based on their feedback a final fourth instrument was prepared resent to PE teachers and based on their feedback final instrument was completed.
Bullock ( 2010) FoodBEAM	Inter-rater ( for researcher to researcher (4 dyads) and researcher non-researcher (5 dyads)	ICC	For both food and beverages researcher versus researcher and researcher versus non- researcher ICC ranged from 0.972 to 0.987	Convergent Comparing FoodBEAMs to the school environmental assessment tool (Samuels, 2008)	ICC scatterplot of EAT*FoodBEAMS versus percent adherence by venue to California state standards for Beverages and Food	ICC for Beverages = 0.982 and for food = 0.975 and shows that the FoodBEAMS is a valid method for collected this type of data.
Schwartz (2209) WellSAT	Inter-rater (by pairs of researcher 1 in-state and 1 out-of-state)	ICC Cronbach's alpha	For total comprehensiveness and strength ICC = 0.82 For subscale scores was 0.70 For Individual items ICC was 0.72. Cronbach's alpha ranged from 0.90 to 0.93
Kim (2010) CHLI	Inter-rater (4 sites with two interviews)	Percent agreement	93.0% school items & 84.9% afterschool items showed substantial to almost prefect agreement
Ajja (2012) HAAND	Inter-rater (concurrently)	Percent agreement kappa statistic	Percent agreement raged from 85% to 100% across all items. Kappa statistics ranged from 0.73 to 1.00 for HAPI-PA( Healthy Afterschool Program Index-Physical activity ) and 0.76 to 1.00 for HAPI-N (Healthy Afterschool Program Index-Nutrition)	Content	Items of HAAND tool were developed based on extensive literature review of the existing PA& nutrition environment quality rating, standards and policies from state and national organization and input from expertise in childcare and afterschool field	Good content validity
				Construct Pedometer step counts were compared to the HAPI-PA scores Menu from observation day was compared to number of time FV Whole grains and Sugar sweeten beverages reported on the HAPI-N	Means and standard deviation calculated and one-way ANOVA test used	HAPI-PA, → pedometer steps were significantly associated with presence of a written policy related to PA, amount/quality of staff training use of PA curriculum and offering activity that appeal to both genders For HAPI-N, higher servings of FV and whole grains per week were significantly associated with the presence of a written policy regarding the nutritional quality of snacks
Nathan (2013). (SEAT)				Construct Principals self-report using the SEAT was compared with scores from direct observations by research staff	Kappa/ PABAK coefficients & percent agreement	Percent agreement = 37% to 100% PABAK = −0.06 to 1.00

^¥ICC: intraclass correlation coefficient

^§OSRAP:_observation system for recording activity in preschools

Validity

Construct validity (table 2) was the most reported type of validity (n=5),^{13, 21, 24, 25, 82} followed by face and/or content validity (n=3),^{13, 21, 83} criterion validity (n=2)^{21, 84} and convergent validity (n=1).²⁶ Construct validity comparisons were made: against national expert review,²¹ comparison to environmental characteristic scores among sites using groups expected to differ due to known characteristics,^{24, 25} and objective measures of child-level physical activity such as pedometers¹³ and direct observation.²⁷ For validity assessment, studies reported Pearson correlation coefficient (r), weighted kappa coefficient, percent agreement, means and standard deviation, multi-level modeling and one-way ANOVA. The following tools reported the highest validity coefficients: the WellCCAT,²⁵ with centers known to have supportive environmental characteristics scoring significantly higher than centers with less supportive environments; the Child Care Nutrition and Physical Activity Assessment Survey,⁸⁴ with 62% of the items reporting ≥ 80% agreement between item scores and criterion measures such as in-person interviews, direct observations, and a newly-developed tool to assess menu items; and the HAAND tool,¹³ with physical activity items having significant positive associations with pedometer step counts.

Discussion

The purpose of this review was to examine the measurement properties of audit tools currently used to evaluate environmental characteristics at various settings caring for youth (≤18 years). Fifty-three tools evaluating the physical activity and healthy eating environmental characteristics in a variety of youth care settings were included in this review. The findings from this review indicate that although a considerable number of tools have been developed over the past decade, relatively little work has been devoted to establishing their reliability and/or validity, with only 11 out of 53 tools having measurement properties information reported.

This review highlights several key issues regarding the utility and the quality of the data collected by the audit tools identified. Several tools (n=7) were developed to assess a specific project or environmental interventions^{35-37, 87, 93} or to evaluate the validity of another pre-existing audit tool.⁸⁵ For example, the Policy Assessment Tool, the 2-minute Program Assessment, and the Program Assessment Tools were all developed to assess the Out of School Nutrition and Physical Activity (OSNAP) intervention in the afterschool setting.⁸⁶ Another example is the Principal's Survey Tool⁸⁵ which was developed as part of evaluating the Teens Eating for Energy and Nutrition at School (TEENS) intervention. As a result, the generalizability of such tools is limited to the projects/interventions they were developed to evaluate and may therefore not provide accurate reflection of practice when used to assess alignment with national and state level physical activity and healthy eating environmental characteristic recommendations.

Psychometric properties

Reliability

Inter-rater reliability was the most reported type of reliability. Assessing tool test-retest and internal consistency reliability is an essential step in establishing measurement properties in the early stages of audit tool development. It is especially important to establish this characteristic in self-assessment tools as it provides critical information about the stability of the item scores on multiple administrations (test-retest reliability) and the extent to which items in the tools all measure the same underlying construct (internal consistency reliability).⁸⁷ However, for observational audit tools, inter-rater reliability is most critical as it will confirm that individuals using the tools observe the same items. For instance, do multiple evaluators assign similar scores to items with respect to the presence or absence of environmental characteristics? An example might be “does the school have a written policy banning cafeteria from serving sugar–sweetened beverages?”.

For continuous data, the intraclass correlation coefficient (ICC) is recognized as the most preferred analysis, whereas for ordinal/categorical data, the recommended analysis is kappa statistics.^{88, 89} An ICC and kappa coefficient of ≥0.7 is considered an acceptable reliability coefficient^90,91 while use of Pearson correlation coefficient (r) is not recommended to assess test-retest reliability as correlations are considered a measure of association and not a measure of agreement.⁹² In this review, only a single study reported using a Pearson correlation coefficient (r) to evaluate test-retest reliability¹⁴. Overall, there are large variations in the reported reliability coefficients, with reliability coefficient values ranging from poor agreement (i.e., ≤ 0.2) to almost perfect (0.8 to 1.00) for kappa while many of the items across the tools reviewed failed to reach the acceptable level for reported reliability (i.e., Kappa > 0.70).

This review found that although the majority of the tools assessing the physical activity and/or healthy eating environmental characteristics were designed to be used by staff/community members (i.e., self-assessment tools), only two studies^{23, 93} evaluated the inter-rater reliability of the tool when used by different groups (i.e., among non-research affiliated staff/community members and/or when compared to research staff). The first study was conducted by Kim et al²³ to evaluate the reliability of the CHLI tool. They reported that the items in the audit tool showed substantial to almost perfect agreement between staff/community members. The second study was done by Bullock et al,²⁶ to evaluate researcher–to-researcher and researcher-to-non-researcher inter-reliability of the FoodBEAMS tool. In this study, they reported perfect agreement between researchers as well as between researchers and non-researcher staff. The ability of the staff/community members to rate the environmental characteristics as accurately as researchers is an essential step in tool development for several reasons. Audit tools designed to evaluate the policy and practice environmental characteristics are often definition-dense, with terminology that does not easily lend itself to use by community members. In addition, one cannot assume that establishing inter-rater reliability across researchers will necessary translate to inter-rater reliability when used by staff/community members. Therefore, adequate training to intended users of these tools is required if such tools are to yield accurate data. Future research should focus on establishing accuracy of newly developed tools when used by intended audiences (i.e., staff/community members).

Validity

Establishing all types of validity (e.g., content, face, criterion, and construct) is an essential step in new tool development.⁹⁴ Construct validity is particularly important as it provides important details as to whether or not a tool actually measures the construct it intends to measure. An important question is “do the items in the tool consistently follow a predicted pattern or theory?”^97,98. An example of this type of validity would be settings which score higher in physical activity-promoting policies having a higher participant physical activity levels when an objective measurement is used, such as accelerometers/pedometer.

The use of Pearson correlation coefficient (r), ICC, percent agreement, scatter plots of interest differences versus means (i.e., visual inspection), and one-way ANOVA are considered acceptable analyses for reporting on the validity of continuous measures.⁹⁵ For ordinal continuous data, the use of Spearman rank correlation coefficient (r_s) is recommended and for categorical (ordered) data, weighted kappa statistics are often recommended.⁹⁶ When a tool's validity coefficients were reported, there were wide variations in the reported values across tool items within each tool, with many of the studies reporting that tools demonstrate good to acceptable validity coefficients, despite the fact that multiple items within those tools fail to reach acceptable coefficient values. Overall this review found that the majority of the studies evaluating measurement properties used appropriate terminology when reporting on the type of validity evaluated. However, only a single study¹² reported criterion validity by using a follow-up interview with the site director who completed the original assessment as a criterion comparision to evalate policy and practice items of the tool. Accurate use of terminology is of critical importance as misclassification of the type of measurement evaluated will impact the quality of the data collected.

In this review, apart from the study by Lounsbery et al.,⁸³ which only reported on content validity for the S-PAPA tool, all the other studies examined additional validity types such as construct or criterion validity to establish stronger measurement proprieties of the newly developed tools. When validity was tested, construct validity was the most often reported validity type, which is an essential measurement property to establish if audit tools are expected to be used to evaluate the environmental characteristics in relation to health outcomes.⁹⁷

These elements, reliability and validity, are fundamental measurement properties necessary for the collection of accurate information on policy and practice environmental characteristics of settings that serve youth. This review shows the lack of consistency when reporting on measurement properties of such tools, with 7 studies out of 11 reporting both validity and reliability properties of environmental characteristics audit tools, and 4 studies reporting on either validity or reliability properties of such tools. For example, Kim et al.,²³ and Schwartz et al.,²² reported only the reliability of the CHLI and the WellSAT tools, respectively. Henderson et al.,¹² reported on only the validity of their newly developed tool. Validity testing of newly developed tools is an important first step in establishing the measurement quality of newly developed tools prior to establishing tool reliability. However, this review indicates that, when measurement properties were tested, the focus was more on reliability testing than validity testing, with reliability reported more often than validity when assessing newly developed instruments, which is in line with current literature findings⁹⁸. Future studies, should address the cause for this apparent lack of validity reporting in the field.

Limitation

Despite our efforts to identify current environmental audit tools used in youth care setting, the authors understand that some tools could have been overlooked. In addition, as indicated in our review, many of the tools were developed for specific projects and were not intended for publication making their identification more difficult.

Recommendations regarding future audit tool development

Audit tools designed to evaluate the environmental characteristics of settings that care for children must demonstrate minimal acceptable levels of reliability and validity evidence. This is critical as information gathered from such tools is being used to inform policy makers’ decisions regarding the impact or effectiveness of environmental characteristics interventions and to, in turn, formulate future strategies regarding the promotion of physical activity and healthy eating habits among youth. Saelens et al.,¹⁹ put forward a set of guidelines for reporting on newly developed instruments. These guidelines include: (1) the rationale and justification for developing the tool and how it differs from existing tools, (2) the construct measured by the tool, (3) reliability and validity of the tool, (4) detailed protocols on how to use the tool, (5) scoring and scaling of the tool, (6) modifications made to the tool, (7) the setting, geographical area, and population or environments where the tool was used, and (8) ways to access the tool.

In the future, when developing new audit tools to assess the environmental characteristics, we recommend that the guidelines put forward by Saelens et al.,¹⁹ be followed when evaluating new audit tools designed to measure environmental characteristics. In addition, we propose that when developing such audit tools, (1) greater efforts must be put towards evaluating inter-rater reliability between researchers and intended users of the tool (e.g., staff/community members, researchers); (2) establishment of construct validity should be given a high priority; and (3) reliability and validity coefficient scores across items of newly developed tools should be reported.

Conclusion

Little attention has been given to establishing reliability and validity evidence of newly developed tools designed to assess physical activity and/or healthy eating environmental characteristics in settings caring for youth. Future efforts should be directed towards establishing a strong measurement foundation for these important environmental audit tools in order to maximize understanding of the health-promoting potential of these critical developmental settings.

Highlights.

• Considerable number of policy and practice environment audit tools have been developed

• Majority of the tools were developed to assess school settings and were based on self-report

• Little attention has been given to establishing measurement properties of newly developed tools

• Majority of the currently available tools lack validity and/or reliability information