Measuring the Environment for Friendliness Toward Physical Activity: A Comparison of the Reliability of 3 Questionnaires

Abstract

Objectives. We tested the reliability of 3 instruments that assessed social and physical environments.

Methods. We conducted a test–retest study among US adults (n = 289). We used telephone survey methods to measure suitableness of the perceived (vs objective) environment for recreational physical activity and nonmotorized transportation.

Results. Most questions in our surveys that attempted to measure specific characteristics of the built environment showed moderate to high reliability. Questions about the social environment showed lower reliability than those that assessed the physical environment. Certain blocks of questions appeared to be selectively more reliable for urban or rural respondents.

Conclusions. Despite differences in content and in response formats, all 3 surveys showed evidence of reliability, and most items are now ready for use in research and in public health surveillance.

An estimated 200 000 to 300 000 premature deaths occur each year in the United States because of physical inactivity.1–4 Accordingly, the goal of increasing physical activity is one of 10 “leading indicator” areas within the national health objectives of Healthy People 2010.5 Even with the known health benefits of physical activity, more than one quarter of the American population remains completely inactive, and US trends in activity showed little improvement from 1990 to 1998.6 More than 60% of the world’s population is not physically active enough to achieve health benefits.7

The physical, or built, environment is important in providing cues and opportunities for activity,8 and it is associated with rates of physical activity in intervention studies and in large population-based surveys.9 Support for the importance of the environment is derived from 2 distinct literatures. A review of 19 studies in the physical activity and health literature showed consistent associations of accessibility of recreational facilities, opportunities to be active, and certain aesthetic qualities with physical activity in adults.10 Researchers in the transportation and urban planning fields have examined the relationship between community design variables and walking or cycling for transportation. Fourteen studies have consistently shown that people walk and cycle more when their neighborhoods have higher residential density, a mixture of land uses (e.g., shops are within walking distance of homes), and connected streets (e.g., gridlike pattern instead of many cul-de-sacs).11 Other community design characteristics, such as the condition of sidewalks, the presence of bike paths, street design, traffic volume and speed, and crime, are hypothesized to be related to physical activity12,13 but have not been systematically examined. In addition, rural areas have important differences from urban areas in their activity-related design features14–16 and are generally understudied.11

Multiple questionnaires have been developed to assess physical activity—measurement properties (i.e., reliability/validity) are documented for many of these. For example, Ainsworth et al.17 reported on the measurement properties of 39 questionnaires, and Kriska and Caspersen18 described the validity, reliability, and comprehensiveness of 32 instruments. In contrast, considering the apparent importance of the built environment, there is limited information in the literature on how best to measure various aspects, such as the presence of well-maintained sidewalks or whether shopping venues are within walking distance.19 One method of measuring the perceived physical environment is through population-based surveys and surveillance systems.20 Individual responses from these surveys can be aggregated to identify patterns in important design/neighborhood features (e.g., lack of access to sidewalks in rural areas) and to determine associations between these design features and behavior.21–23 As yet, it is unclear whether the objective environment (e.g., actual counts of traffic) or the perceived environment (e.g., an individual’s self-reported perception of crime in his/her neighborhood) is more important in explaining physical activity.10,11

As measures of perceived environments are developed, it is important to ensure that they can be administered by multiple modes (e.g., self and interviewer administered) and are reliable for broad populations. Our study reports the results of reliability testing of 3 instruments among urban and rural residents across the United States. A major focus of the instruments tested was the assessment of environmental characteristics that are believed to be related to recreational physical activity and nonmotorized transportation, although some instruments assessed other related variables.

METHODS

Sampling Plan

Data were collected through telephone surveys of people aged 18 years and older who lived in the continental United States. We used a modified version of the Behavioral Risk Factor Surveillance System (BRFSS) sampling plan24,25 in which a random-digit sample was purchased from a database company; 50% of the telephone numbers were from rural areas and 50% were from urban areas. Rural or urban residences were defined by US Census Bureau categories. The Census Bureau classifies as urban all territory, population, and housing units located within an urbanized area or an urban cluster. It delineates urban area and urban cluster boundaries to encompass densely settled territory, which consists of census blocks (e.g., a block bounded by city streets) that have a population density of at least 1000 people per square mile and have surrounding census blocks with an overall density of at least 500 people per square mile. The Census Bureau’s classification of rural includes all territory, population, and housing units located outside of urban areas and urban clusters.26

Because the purpose of our study was to determine test–retest reliability, respondents who completed the survey were asked if they would be willing to complete the survey again in 7 to 21 days, and they were asked for the most convenient time to call for the resurvey. The second calls were made within the 7-to-21-day time frame, and the survey was readministered. This time frame is often used in test–retest studies because it is a long enough period so that respondents are unlikely to remember their answers to the original survey, yet the time frame is short enough so that changes in behavior (e.g., seasonal changes in physical activity) are unlikely to have occurred. Each survey participant was assigned randomly to 1 of the 3 questionnaires.

Questionnaires

Three questionnaires were used: the San Diego instrument, the South Carolina instrument, and the St Louis instrument. Each of these had been previously tested for some psychometric properties but had not been tested for reliability side by side across a nationwide population. All 3 questionnaires, decribed in the following paragraphs, used the same sociodemographic questions (i.e., race/ethnicity, age, gender, education level, income, employment status). A copy of each questionnaire is available from the lead investigator at each site: San Diego—J.F.S. (http://www.rohan.sdsu.edu/faculty/sallis/index.html), South Carolina—B.E.A. (http://prevention.sph.sc.edu), and St Louis—R.C.B. (http://prc.slu.edu).

The San Diego instrument (also called the Neighborhood Environment Walkability Survey).

This 98-question instrument was developed by Sallis et al. to determine the perception of neighborhood design features hypothesized to be related to physical activity. The questionnaire includes questions about types of residences (to assess density), proximity of stores and facilities in the neighborhood, perceived access to these places, street characteristics (to assess connectivity), facilities for walking and cycling, neighborhood aesthetics, and safety regarding traffic and crime. The San Diego instrument was originally developed for self-administration and was therefore adapted for telephone administration in our study. A reliability study of a self-administered version of this instrument was completed in San Diego.27

The South Carolina instrument.

This 61-question instrument was developed by Ainsworth et al. and includes an assessment of the physical and social environments, including perceptions of the community environment (e.g., whether the neighborhood is pleasant), safety, access to recreation and shopping destinations, and conditions of the neighborhood and facilities. Thirteen items focus on the neighborhood, which is defined as a half-mile radius or a 10-minute walk from the respondent’s home, and 13 items focus on the community, which is defined as a 10-mile radius or a 20-minute drive from the respondent’s home. Additional physical activity questions from the BRFSS incorporate an assessment of employment activity as well as moderate and vigorous physical activities and global walking behaviors. This instrument was previously tested for reliability and validity among 1200 adults who lived in Sumter County, SC.28

The St. Louis instrument.

This 104-question survey was developed by Brownson et al. to measure physical activity and environmental influences on physical activity across the United States.22,29,30 Several constructs in the St Louis instrument were used to develop and evaluate physical activity interventions in rural settings.14,31 The questionnaire includes a detailed assessment of walking behavior, places to walk, barriers to being physically active, neighborhood infrastructure for walking and cycling, perceptions about places for walking, social assets, social support for physical activity, community assets, policy attitudes, and sedentary behaviors. An earlier version of this instrument was tested for reliability in a US sample of ethnically diverse women aged 40 years and older.32

Data Collection

Telephone calls were made in April and May 2002 by experienced interviewers who had at least 8 hours of specific training for this project. Calls were made between 5:00 p.m. and 9:00 p.m. on weekdays and between 12:00 p.m. and 6:00 p.m. on weekends. The person who answered the phone, if eligible, responded to the survey. If the person who answered the phone was under 18, he or she was asked to give the phone to an available adult who lived in the household. The baseline response rate for the original survey was 36.3%, which was calculated with the method of the Council of American Survey Research Organizations,33 and the retest survey had a response rate of 63.9%. The average interview administration times for each of 3 instruments were as follows: San Diego—24 minutes, South Carolina—25 minutes, and St Louis—30 minutes. These administration times provide only a rough estimate of time needed to complete surveys of environmental features, because questionnaires varied in their number of questions related to factors other than the physical environment (e.g., the St Louis instrument included a substantial block of questions about social and community assets).

Analyses

After cleaning and editing the data, the reliability of each variable from time 1 to time 2 was assessed with the 1-way model intraclass correlation coefficient (ICC).34 The ICC is derived from a 1-way analysis of variance, and it represents the proportion of total variation accounted for by the variability between, rather than within, subjects. As a rough guide, we followed the adjectival ratings suggested by Landis and Koch35 in the following categories: 1.0 to 0.8 (almost perfect agreement), 0.8 to 0.6 (substantial agreement), 0.6 to 0.4 (moderate agreement), 0.4 to 0.2 (fair agreement), and 0.2 to 0.0 (poor agreement). Agreement for each of the 3 questionnaires also was considered separately for urban versus rural respondents, because earlier studies have shown that patterns in these environmental characteristics can vary widely depending on urban or rural residence.15,16

RESULTS

Compared with the overall US population,26 our sample tended to overrepresent females, Whites, and persons who had more than a high school education. Statistically significant differences (P < 0.01) across the 3 samples were present for age group, education level, and employment status (Table 1 ▶). Coefficients of reliability were consistently high for sociodemographic variables (i.e., gender, race/ethnicity, age, education level, income, employment status); the ICC values ranged from 0.98 for gender and age to 0.82 for annual household income. These coefficients were similarly high across all 3 questionnaires.

TABLE 1—

Characteristics of Participants in the Study of Reliability of 3 Questionnaires: United States, 2002

	San Diego (n = 93)		South Carolina (n = 97)		St Louis (n = 99)		Total Sample Test–Retest Agreement/Reliability
Characteristic	n	%	n	%	n	%	%	ICC
Gender
Female	66	71.0	59	62.8	57	60.6	99.2	0.98
Male	27	2.0	38	37.2	42	39.4
Race
White	78	83.9	73	75.3	78	79.6	95.4	0.85
Black or African American	11	11.8	20	20.6	18	18.4
Othera	4	4.4	4	4.1	2	2.0
Age, y
< 30	16	17.2	23	23.7	17	17.2	98.2	0.98
30–39	24	25.8	10	10.3	19	19.2
40–49	18	19.4	13	13.4	17	17.1
50–59	22	23.6	21	21.7	16	16.2
> 60	13	14.0	30	30.9	32	30.3
Education
Less than high school	4	4.3	14	14.5	11	11.2	72.3	0.87
High school or GED	21	22.6	31	32.0	29	29.6
Some college/technical school	30	32.3	24	24.7	33	33.7
College graduate	21	22.6	19	19.6	16	16.3
Post graduate/professional degree	17	18.3	9	9.3	9	9.2
Annual income
< $20 000	13	15.2	19	21.6	20	23.3	73.6	0.82
$20 000–$34 999	21	24.7	30	34.1	25	29.1
$35 000–< $49 999	17	20.0	14	15.9	15	17.4
> $50 000	34	40.0	25	28.4	26	30.3
Employment status
Employed for wages—full-time	46	49.5	35	36.1	37	37.8	73.7	0.86
Employed for wages—part-time	12	12.9	13	13.4	9	9.2
Self-employed	12	12.9	2	2.1	8	8.2
Out of work, retired, disabled or unable to work	15	16.1	36	37.2	29	29.6
Homemaker/student	8	8.7	11	11.4	15	15.3

Note: ICC = intraclass correlation coefficient.

^aOther includes Asians/Pacific Islanders and American Indians/Alaska Natives.

For the San Diego questionnaire (Table 2 ▶), more than 70 different individual questions were used to form 8 major constructs. All means and frequencies in Tables 2 ▶, 3 ▶, and 4 ▶ are based on the second administration of the questionnaire because it represented the final sample after attrition. Across the various constructs, reliability coefficients were nearly always at the moderate level or higher. Within the construct “land use mix—diversity,” the largest proportion of coefficients greater than or equal to 0.60 was found. Across all constructs, only 1 question resulted in poor reliability: the neighborhood characteristic of not many/any cul-de-sacs. The San Diego instrument also was constructed to allow calculation of scale reliabilities for the 8 domains (on the basis of the mean of questions within a scale). The ICC values for the 8 scales ranged from 0.41 for “street/walking environment” to 0.93 for “land use mix—diversity.” Most scales were in the substantial agreement category. Reliabilities also were assessed separately for urban versus rural respondents (data not shown). Most constructs for the San Diego instrument measured the physical environment with about equal reliability in urban versus rural areas. Two constructs (“land use mix—diversity” and “neighborhood satisfaction”) had higher ICC values for urban compared with rural respondents. For 1 construct (“neighborhood safety”), reliability tended to be higher for rural compared with urban respondents. Many differences in coefficients were small; however, when large differences (> 50%) in coefficients were found, urban respondents nearly always showed higher reliability than rural respondents.

TABLE 2—

Reproducibility of Items and Scales That Assessed Physical Environments: San Diego Instrument, 2002 (n = 93)

Travel Time to Location
Instrument Domain and Characteristic	1–5 Min, %	6–10 Min, %	11–20 Min, %	21–30 Min, %	≥ 31 Min, %	Mean (SD)	Observed Agreement, %	ICC
Land use mix—diversity								0.93
Convenience/small grocery store	29.0	20.4	25.8	8.6	16.1	3.4 (1.4)	68.5	0.85
Supermarket	8.8	12.1	25.3	14.3	39.6	2.4 (1.4)	54.8	0.69
Hardware store	11.4	6.8	22.7	11.4	47.7	2.2 (1.5)	57.2	0.74
Fruit/vegetable market	10.0	11.3	18.8	12.5	47.5	2.1 (1.6)	53.4	0.46
Laundry/dry cleaners	17.6	12.1	14.3	19.8	36.3	2.7 (1.5)	63.2	0.64
Clothing store	5.9	12.9	16.5	11.8	52.9	2.0 (1.5)	60.5	0.60
Post office	13.2	15.4	17.6	17.6	36.3	2.6 (1.5)	54.9	0.80
Library	16.3	10.9	15.2	17.4	40.2	2.6 (1.5)	65.4	0.79
Elementary school	21.5	18.3	21.5	11.8	26.9	3.1 (1.5)	64.3	0.80
Schools other than elementary	10.8	19.4	16.1	12.9	40.0	2.6 (1.5)	53.5	0.60
Bookstore	6.3	10.0	15.0	10.0	58.8	1.9 (1.5)	58.4	0.73
Fast-food restaurant	16.7	17.8	25.6	12.2	27.8	2.6 (1.5)	56.0	0.72
Coffee place	20.5	9.6	16.9	12.0	41.0	2.6 (1.7)	54.9	0.62
Bank/credit union	21.5	18.3	18.3	12.9	29.0	3.0 (1.5)	62.0	0.74
Non–fast-food restaurant	16.5	15.4	23.1	12.1	33.0	2.8 (1.5)	55.5	0.68
Video store	14.3	9.9	20.9	16.5	38.5	2.5 (1.5)	62.2	0.83
Pharmacy/drug store	11.2	14.6	23.6	15.7	34.8	2.5 (1.5)	56.1	0.79
Salon/barber shop	25.3	13.2	19.8	17.6	24.4	3.1 (1.5)	64.3	0.67
Your job	3.9	3.9	3.9	9.1	79.2	1.3 (1.3)	70.3	0.60
Bus/trolley stop	49.7	16.7	6.4	2.6	34.6	3.3 (2.0)	67.2	0.77
Park	29.7	19.8	14.3	9.9	26.4	3.2 (1.6)	60.8	0.80
Recreation center	16.0	8.6	18.5	14.8	42.0	2.3 (1.7)	46.2	0.44
Gym/fitness facility	8.1	16.3	18.6	11.6	45.3	2.3 (1.5)	52.0	0.58

Presence of Housing Type in Neighborhood
Characteristic	None, %	Few, %	Some, %	Most, %	All, %	Mean (SD)	Observed Agreement, %	ICC
Residential density								0.78a
Detached single-family residences	6.5	17.2	15.1	23.7	37.6	2.7 (1.3)	57.2	0.52
Townhouses/row houses of 1–3 stories	54.8	20.4	10.8	10.8	3.2	1.9 (1.2)	62.0	0.56
Apartments/condos 1–3 stories	57.6	20.7	13.0	6.5	2.2	1.8 (1.1)	64.2	0.70
Apartments/condos 4–6 stories	89.2	6.5	3.2	1.1	. . .	1.2 (0.5)	83.9	0.51
Apartments/condos 7–12 stories	93.5	4.3	1.1	1.1	. . .	1.1 (0.4)	96.9	0.89
Apartments/condos > 13 stories	97.8	1.1	. . .	1.1	. . .	1.0 (0.3)	97.8	. . .

Acceptance of Statements
Characteristic	Strongly Agree, %	Somewhat Agree, %	Somewhat Disagree, %	Strongly Disagree, %	Mean (SD)	Observed Agreement, %	ICC
Land use mix—access							0.77
Shopping at local stores	31.2	20.4	12.9	35.5	2.5 (1.3)	53.2	0.57
Stores within walking distance	25.8	30.1	8.6	35.5	2.5 (1.2)	61.4	0.71
Parking difficult in shopping areas	7.5	14.0	17.2	61.3	1.7 (1.0)	41.8	0.22
Many places within walking distance	33.7	26.1	8.7	31.5	2.6 (1.3)	44.6	0.53
Easy walking to transit stop	40.5	16.7	2.4	40.5	2.6 (1.4)	72.5	0.78
Hilly streets difficult to walk	6.5	14.0	17.2	62.4	1.7 (1.0)	54.4	0.48
Canyons/hillsides limit the number of traveling routes	2.2	7.5	12.9	77.4	1.3 (0.7)	60.3	0.25
Street/walking environment							0.41
Not many/any cul-de-sacs	38.7	24.7	6.5	30.1	2.7 (1.3)	33.3	0.18
Walkways connect cul-de-sacs to streets	15.2	15.2	9.8	59.8	1.9 (1.2)	49.9	0.26
Short distance between intersections	44.4	21.1	10.0	24.4	2.9 (1.2)	41.5	0.22
Many 4-way intersections	35.5	20.4	10.8	33.3	2.6 (1.3)	47.4	0.51
Many alternative travel routes	48.4	28.0	5.4	18.3	3.1 (1.1)	56.0	0.66
Infrastructure for walking/cycling							0.76
Sidewalks on most streets	48.4	18.3	3.2	30.1	2.9 (1.3)	63.5	0.77
Well-maintained sidewalks	29.3	27.2	7.6	35.9	2.5 (1.3)	60.7	0.69
Bike/pedestrian trails accessible	17.4	17.4	9.8	55.4	2.0 (1.2)	47.8	0.52
Sidewalks separated from the roads by parked cars	22.6	16.1	7.5	53.8	2.1 (1.3)	52.8	0.56
Grass/dirt strip separates the streets from sidewalks	38.7	12.9	9.7	38.7	2.5 (1.4)	53.3	0.65
Neighborhood safe for biking	58.1	30.1	2.2	9.7	3.4 (0.9)	49.5	0.45
Neighborhood aesthetics							0.66
Trees along the streets	58.1	25.8	2.2	14.0	3.3 (1.1)	53.9	0.57
Trees/canopy cover along the sidewalks	17.6	24.2	3.3	54.9	2.0 (1.2)	42.9	0.49
Many interesting sights while walking	38.7	36.6	12.9	11.8	3.0 (1.0)	57.0	0.64
Neighborhood free from litter	50.5	34.4	6.5	8.6	3.3 (0.9)	51.1	0.46
Attractive views/landscape	35.9	27.2	20.7	16.3	2.8 (1.1)	46.8	0.54
Attractive buildings/homes	39.8	41.9	9.7	8.6	3.1 (0.9)	51.1	0.64
Neighborhood safety							0.69,b 0.64c
Too much traffic to walk along the street where you live	7.5	14.0	17.2	61.3	1.7 (1.0)	52.8	0.59
Too much traffic to walk along the nearby street	9.7	15.1	17.2	58.1	1.8 (1.0)	45.1	0.46
Slow traffic speed on the street where you live	48.4	24.7	12.9	14.0	3.1 (1.1)	39.1	0.34
Slow traffic speed on the nearby street	36.6	28.0	18.3	17.2	2.8 (1.1)	41.8	0.39
Streets well lit at night	35.5	32.3	9.7	22.6	2.8 (1.2)	48.8	0.44
Most drivers exceed the posted speed limit while driving in the neighborhood	35.9	23.9	20.7	19.6	2.8 (1.1)	49.5	0.56
Walkers/bikers visible in the neighborhood	46.2	35.2	6.6	12.1	3.2 (1.0)	51.7	0.57
Crosswalks/pedestrian signals to help crossing busy streets	33.3	18.3	7.5	40.9	2.4 (1.3)	49.6	0.56
Crosswalks help walkers feel safe crossing busy street	27.2	23.9	8.7	40.2	2.4 (1.3)	53.5	0.61
Lots of exhaust fumes when walking	4.3	18.3	18.3	59.1	1.7 (0.9)	59.1	0.63
Greet people while walking	48.9	35.9	8.7	6.5	3.3 (0.9)	56.6	0.44
High crime rate in the neighborhood	6.5	6.5	20.7	66.3	1.5 (0.9)	63.1	0.61
Unsafe walking during the day because of crime rate	0.0	5.4	22.6	72.0	1.3 (0.6)	61.3	0.31
Unsafe walking at night due to crime rate	12.0	13.0	16.3	58.7	1.8 (1.1)	59.7	0.69
Safe for a kid to walk around the block alone during the day	55.9	26.9	8.6	8.6	3.3 (1.0)	56.9	0.49
Neighborhood satisfaction							0.65

Acceptance of Statements
Characteristic	Strongly Agree, %	Agree, %	Slightly Agree, %	Neither Agree nor Disagree, %	Slightly Disagree, %	Disagree, %	Strongly Disagree, %	Mean (SD)	Observed Agreement, %	ICC
Note: ICC = intraclass correlation coefficient.
aWeighted by the following formula: (1 × single-family detached) + (12 × row houses/townhouses 1–3 stories) + (10 × apartments/condos 1–3 stories) + (25 × apartments/condos 4–6 stories) + (50 × apartments/condos 7–12 stories) + (75 × apartments/condos ≥ 13 stories).
bFor traffic section within the Neighborhood safety domain.
cFor crime section within the Neighborhood safety domain.
Freeway access from home	32.3	25.8	12.9	16.1	4.3	2.2	6.5	5.3 (1.7)	32.4	0.50
Access to the bus/trolley system	22.6	24.7	6.5	19.4	2.2	2.2	22.6	4.5 (2.2)	43.6	0.61
Commuting time to work/school	46.2	24.7	9.7	7.5	4.3	2.2	5.4	5.7 (1.7)	46.4	0.44
Access to shopping	36.6	28.0	11.8	1.1	8.6	4.3	9.7	5.3 (2.0)	39.8	0.51
Number of neighborhood friends	37.6	30.1	14.0	2.2	3.2	6.5	6.5	5.5 (1.8)	45.2	0.63
ANumber of neighborhood acquaintances	40.9	34.4	10.8	4.3	1.1	2.2	6.5	5.8 (1.7)	43.1	0.52
How easy/pleasant to walk in the neighborhood	50.5	29.0	8.6	3.2	1.1	2.2	5.4	6.0 (1.6)	45.2	0.67
How easy/pleasant to bike in the neighborhood	41.9	29.0	8.6	5.4	4.3	5.4	5.4	5.6 (1.8)	43.1	0.56
Neighborhood a good place to raise children	55.9	20.4	7.5	3.2	1.1	6.5	5.4	5.9 (1.8)	49.6	0.69
Neighborhood a good place to live	65.6	20.4	5.4	2.2	1.1	2.2	3.2	6.3 (1.4)	57.0	0.73

TABLE 3—

Reproducibility of Items That Assessed Community and Physical Environments: South Carolina instrument, 2002 (n = 97)

Characteristic	n	Group Prevalence, %	Observed Agreement, %	ICC
Community environment
Neighborhood physical activity level
Very active	7	7.7	60.9	0.49
Somewhat active	55	60.4
Not very active	23	25.3
Not at all active	6	6.6
Neighborhood as a place to work
Very pleasant	50	53.2	67.1	0.57
Somewhat pleasant	40	42.6
Not very pleasant	3	3.2
Not at all pleasant	1	1.1
Problem with unattended dogs
A big problem	6	6.2	64.6	0.65
Somewhat of a problem	18	18.6
Not very much of a problem	20	20.6
Not a problem at all	53	54.6
Neighborhood safety
Extremely safe	21	21.6	67.9	0.62
Quite safe	54	55.7
Slightly safe	19	19.6
Not at all safe	3	3.1
Traffic in the neighborhood
Heavy	18	18.6	70.8	0.63
Moderate	44	45.4
Light	35	36.1
Safety of public recreation facilities
Very safe	42	56.0	67.6	0.42
Somewhat safe	32	42.7
Somewhat unsafe	1	1.3
Physical environment
Sidewalks in the neighborhood
Yes	58	59.8	93.8	0.87
Condition of sidewalks
Very well maintained	27	46.6	72.7	0.66
Somewhat maintained	23	39.7
Not very well maintained	6	10.3
Not at all maintained	2	3.4
Condition of streetlight at night
Very good	11	11.7	46.5	0.79
Good	28	29.8
Fair	22	23.4
Poor	12	12.8
Very poor	21	22.3
Access to public recreation facilities
Yes	56	57.7	76.1	0.50
Condition of public recreation facilities
Excellent	19	35.2	69.5	0.62
Good	22	40.7
Fair	12	22.2
Poor	1	1.9
Use of walking trail
Yes	35	36.5	67.3	0.66
No	29	30.2
No community walking trail available	32	33.3
Use of public swimming pools
Yes	14	14.9	64.9	0.59
No	52	55.3
No community public swimming pools available	28	29.8
Use of public recreation centers
Yes	22	23.4	64.5	0.51
No	40	42.6
No community public recreation centers available	32	34.0
Use of bicycle paths/trails
Yes	37	25.3	57.7	0.47
No	60	41.1
No community bicycle paths/trails available	49	33.6
Use of parks/playgrounds/sports fields
Yes	52	54.2	65.2	0.47
No	34	35.4
No community parks/playgrounds/sports fields available	10	10.4
Use of schools with public recreation activities
Yes	20	22.7	53.7	0.44
No	38	43.2
Schools with public recreation activities not open to public	30	34.1
Use of shopping mall for physical activities
Yes	15	15.5	67.9	0.42
No	66	68.0
No community shopping mall available	16	16.5
Equal access to public recreation facilities
Yes	76	79.2	79.8	0.39
No	4	4.2
No community public recreation facilities available	16	16.7

Note: ICC = intraclass correlation coefficient.

TABLE 4—

Reproducibility of Data Items That Assessed Community and Physical Environments: St Louis Instrument, 2002 (n = 99)

Characteristic	n	Group Prevalence, %	Observed Agreement, %	ICC
Community environment
Availability of walking trails/tracks/paths
Yes	13	36.1	96.0	0.92
How safe while walking (n = 42)
Extremely safe	26	61.9	69.7	0.60
Quite safe	13	31.0
Slightly safe	3	7.1
Most liked feature of the walking facilities (n = 40)
Location/convenience	16	40.0	43.8	0.19
Scenic beauty	13	32.5
Othera	11	27.5
Least liked feature of the walking facilities (n = 41)
Unsafe surface	2	4.9	54.5	0.58
Poor lighting	2	4.9
Unattended animals	2	4.9
Like everything about the place	14	34.1
Otherb	21	51.0
Neighbors are physically active
Not at all true	10	17.9	56.7	0.58
Somewhat true	25	44.6
True	15	26.8
Very true	6	10.7
How safe from crime in the neighborhood
Extremely safe	33	33.7	69.4	0.58
Quite safe	52	53.1
Slightly safe	10	10.2
Not at all safe	3	3.1
Workplace environment
Employer gives incentives to exercise
Yes	23	41.8	85.1	0.70
Types of workplace support
Breaks for exercise	8	32.0	40.0	0.44
Facilities to exercise	7	28.0
Offer group services	3	12.0
Subsidize health club membership	4	16.0
Sponsor sports teams	2	8.0
Monetary incentives	1	4.0
Safe stairways at work
Yes	28	60.9	67.4	0.42
No safe stairways for use	3	6.5
No stairs at all	15	32.6
Sedentary behaviors
Hours spent sitting/lying down (watch TV, reading, etc.) per week
1–10	35	35.4	54.1	0.37
11–20	26	26.3
21–30	15	15.2
> 30	23	23.2
Hours spent using a computer per week
1–10	31	35.6	77.7	0.79
11–20	8	9.2
21–30	1	1.1
> 30	47	54.0
Hours spent driving to work per week
1–10	14	14.6	68.5	0.29
11–20	6	6.3
21–30	10	10.4
> 30	66	68.8
Hours spent driving to shop per week
1–10	25	25.5	51.5	0.27
11–20	18	18.4
21–30	7	7.1
> 30	48	49.0
Hours spent driving for delivery/picking up kids/errands per week
1–10	12	13.0	58.4	0.17
11–20	9	9.8
21–30	9	9.8
> 30	62	67.4

Acceptance of Statements
Characteristic	Strongly Agree, %	Somewhat Agree, %	Somewhat Disagree, %	Strongly Disagree, %	Observed Agreement, %	ICC
Note: ICC = intraclass correlation coefficient.
aOther = free place to exercise, trail design, safe surface, no crowds, etc.
bOther = traffic, crossing busy street, trail design, poor lighting, etc.
Infrastructure for walking and cycling
Sidewalks on most streets of the neighborhood	38.4	5.1	14.1	42.4	70.7	0.75
Sidewalks well maintained	37.2	7.4	21.3	34.0	56.7	0.64
Accessible bike/walking trails	41.4	13.1	13.1	32.3	62.2	0.62
Difficult hilly street for walking	46.5	17.2	18.2	18.2	55.6	0.51
Neighborhood surroundings
Many attractive natural sites in my neighborhood	28.3	12.1	23.2	36.4	44.5	0.42
Neighborhood safety
Much traffic, barrier for walking	60.6	17.2	12.1	10.1	51.4	0.44
Most drivers exceed speed limit in the neighborhood	23.2	17.2	24.2	35.4	47.5	0.57
Street well lit at night	33.3	8.1	24.2	34.3	66.7	0.80
Unsafe walking during the day due to crime rate	69.7	17.2	8.1	5.1	63.5	0.36
Unsafe walking at night due to crime rate	52.5	14.1	24.2	9.1	58.7	0.59
Lots of exhaust fumes	52.5	14.1	19.2	14.1	56.5	0.52
Many unattended dogs	50.5	20.2	16.2	13.1	58.7	0.48
Lots of people walking and biking in the neighborhood	20.6	10.3	27.8	41.2	58.7	0.63

Results for 19 questions about the community and physical environment are shown for the South Carolina instrument (Table 3 ▶). Among these variables, most questions (n = 10) were classified in the moderate agreement category, and 7 questions showed substantial reliability. The highest reliability was shown for the question regarding the presence of sidewalks (ICC = 0.87), and the lowest reliability was for the question regarding equal access to public recreation facilities (ICC = 0.39). Reliability was higher for urban respondents on 10 questions and for rural residents on 9 questions. In most cases, coefficient differences were small. Four exceptions were “problem with unattended dogs” (urban ICC = 0.76, rural ICC = 0.37), “safety of the public recreation facilities” (urban ICC = 0.37, rural ICC = −0.04), “condition of public recreation facilities” (urban ICC = 0.74, rural ICC = −0.07), and “use of shopping mall for physical activities” (urban ICC = 0.49, rural ICC = 0.26).

The highest proportion of questions within the St Louis questionnaire (Table 4 ▶) showed moderate reliability (n = 12). Only 2 questions (“most-liked feature of the walking facilities” and “hours spent driving for delivery/picking up kids/errands per week”) resulted in poor agreement. Except for the question about hours spent using a computer, each of the questions that attempted to measure sedentary behaviors showed only fair or poor agreement. Differences in reliabilities for urban versus rural respondents were more distinct for the St Louis instrument than for the other 2 questionnaires. For questions that assessed the community environment, 1 question (“least-liked feature of the walking facility”) was more reliable for urban respondents, and all other questions showed higher reproducibility among rural respondents. Of the 5 questions that assessed sedentary behaviors, 4 showed higher reliability for urban compared with rural respondents, and 1 question in this domain (“hours spend driving for delivery/picking up kids/errands”) showed no urban–rural variation. Finally, for neighborhood safety, 7 of 8 questions were more reliable for rural compared with urban respondents.

DISCUSSION

There is growing recognition that it is essential to understand, and eventually intervene on, environmental and policy factors if we are to increase population rates of physical activity.9,10,36 To conduct research studies that test these environmental hypotheses, it is essential to improve measurement of environmental variables. There are at least 2 ways in which these environmental factors can be measured. First, unobtrusive indicators or measures are those on which data can be collected without an individual’s or community’s awareness.20 They often include examining physical surroundings, archival (public) records, sales records, institutional records, and personal documents, as well as observational measures recorded for specific events.37–40 Recently, systematic direct observations of features of the physical environment within communities has been shown to be a useful and reliable method for collecting data.41 Regardless of how data are collected, they can be mapped and analyzed with geographic information system technologies.42,43

The other main source of environmental measures is from survey or surveillance data on individuals that can be aggregated to some larger unit (e.g., zip code) and compared across subgroups. The 3 instruments used in our study are useful for this type of data collection and analysis. Although each instrument was designed for a slightly different purpose, most of the variables were reasonably reliable in a diverse sample of adults. In spite of differences in content and response formats, all 3 surveys showed evidence of reliability, and most items are now ready for use in research and in public health surveillance. Several patterns in our data deserve mention:

• Most questions in our surveys that attempted to measure specific characteristics of the built environment (e.g., distance between destinations, presence of sidewalks) showed moderate to high reliability.

• Questions about the social environment (e.g., perceived safety in one’s neighborhood) showed lower reliability than those that assessed the built environment.

• Certain blocks of questions appeared to be selectively more reliable for urban or rural respondents (e.g., residential density was more reliably reported among urban respondents). We expected many of these items to perform better in urban samples, because most environmental constructs were derived from studies or from considerations of urban settings. Some variables may be largely irrelevant for rural environments. One research priority is to identify environmental variables in rural settings that might be related to physical activity.

• When the reliability of entire scales were tested, ICC values tended to fall in the substantial agreement (0.6–0.8) category.

• Additionally, some questionnaire items with responses that vary on the time of the day, such as availability of parking at stores and traffic patterns, may not be suitable for point-in-time reliability testing.

When determining which questionnaire scales to use in a particular study, there is a trade-off between the ability to comprehensively measure all domains and the feasibility of collecting data efficiently. It is necessary to match environmental variables with the physical activity outcomes of interest, and very specific hypotheses may need to be developed. For example, walking for transportation is likely to be related to the presence of shops nearby, and walking for recreation may be more related to neighborhood aesthetics. Bicycling is expected to be related to accessibility of cycling facilities, and other types of recreational physical activity may be related to presence, condition, and accessibility of recreational facilities. The 3 questionnaires we evaluated can assess a wide range of environmental variables that allow researchers to test multiple hypotheses.

The next research priority is to test hypotheses about the relationship between environmental variables and physical activity. Because it is not clear whether perceived or objectively measured environmental variables provide more explanatory power, the use of triangulation—applying multiple methods of data collection to determine points of concordance or disagreement44,45—is recommended. A broad range of populations should be studied for several reasons; for example, children and older adults are likely to do physical activity in different settings. Both cross-sectional and longitudinal studies are needed in multiple settings that range from urban to rural locales. To adequately explain physical activity, researchers should examine the separate and interactive contributions of psychological, social, and environmental variables.12 As consensus is reached on the most important correlates or predictors of physical activity, these variables can be incorporated into national surveillance systems.

There are several limitations to our study that deserve mention. We relied on selfreported telephone survey data for which there are several potential biases (e.g., possible underrepresentation of lower socioeconomic status segments of the population).46–48 Our questions about the environment were selfreported and did not include separate objective measures that would allow assessment of validity (i.e., presence of some “gold standard”). However, 1 recent study found statistically significant associations between selfreported and objectively measured (with geographic information systems) characteristics of trails that may influence physical activity.43 We do not intend to imply that perceived environment measures are preferred over objective measures. At this early phase in this field of research, it is important to evaluate both perceived and objective measures of the environment as they relate to physical activity. Our response rate for the initial survey was lower than anticipated. However, because our study was not developed to measure prevalence, and because the follow-up response rate was reasonable (64%), our reliability results should not be subject to substantial bias. Other similar reliability studies of questionnaires on physical activity have shown response rates from 13% to 54%,49–51 and many reliability studies have relied on convenience samples. Both the length of the survey/completion time and the content areas may be factors in the low baseline response rate.

Surveillance of chronic diseases has focused primarily on the diseases themselves until recently, when national systems began tracking behavioral risk factors and changes in preventive health practices.25,52–54 Our study suggests that numerous dimensions of the physical environment can be measured reliably with telephone survey methods. Multisite collaborations such as ours allow for the testing of multiple instruments simultaneously. Surveillance systems need to begin capturing key aspects of the physical and social environments in addition to the main focus on the behavior of physical activity.20,54

CONCLUSIONS

Our study contributes to the growing understanding about the ability to measure people’s perceptions of their physical and social environments in community settings. These surveys have been shown to be reliable in diverse adult samples and are now available for use in further studies. Additional studies are needed to establish the validity of perceptions about environmental variables that may be related to physical activity.