Factorial Invariance of the Physical Activity Neighborhood Environment Survey Among Single vs. Multi-family Housing Residents

Abstract

Purpose

Individual perceptions of one’s neighborhood environment influence decisions about physical activity (PA) participation. Differences between single-family housing neighborhoods vs. multi-family housing neighborhoods may affect perceptions and lead to varying responses on surveys designed to assess perceptions of the neighborhood environment for PA. This study tested the factorial invariance for the Physical Activity Neighborhood Environment Survey (PANES) between residents of single-family vs. multi-family housing neighborhoods.

Method

This study was a secondary data analysis of PANES ratings from African American and Hispanic or Latina women (n=324) who participated in the Health is Power study (NCI R01CA109403), a multi-site, community based trial to investigate the relationship between neighborhood factors and physical activity adoption and maintenance. Factorial invariance was tested using a series of nested confirmatory factor analysis models.

Results

The final model was a second-order factor structure with partial invariance of item intercepts. The second-order factor structure and the relationships of the PANES items to the first-order factors (Amenable, Unsafe, and Walkable) and of the first-order factors to the second-order factor (Environment) were invariant between the single-family and multi-family housing neighborhoods groups.

Conclusion

These findings support the construct validity of PANES, which can be considered valid for measuring neighborhood perceptions among residents of neighborhoods with different housing types.

According to Healthy People 2020 (U.S. Department of Health and Human Services), more than 80% of US adults currently do not meet the recommended guidelines for physical activity. Individual perceptions of one’s neighborhood environment influence attitudes and decisions about participating in physical activity (Duncan, Spence, & Mummery, 2005; Poortinga, 2006; Velasquez, Holahan, & You, 2009). Perceptions can be based on many factors, such as crime, incivilities and density (Brownson, Hoehner, Day, Forsyth, & Sallis, 2009). Although many studies have associated population density with physical activity (Ewing, Schmid, Killingsworth, Zlot, & Raudenbush, 2003; James et al., 2013; Yang, Spears, Zhang, Lee, & Himler, 2012), none have investigated how population density impacts individual perceptions of the neighborhood.

Population density may be characterized by the predominant types of housing present in the neighborhood. Low-density neighborhoods have mostly single-family houses, whereas high-density neighborhoods have mainly multiple-family townhouses or apartments. The myriad of differences between these neighborhood types may affect residents’ perceptions and affect responses on instruments that are designed to assess neighborhood environment, such as the Physical Activity Neighborhood Environment Survey (PANES) (Sallis et al., 2010). When these instruments have inconsistent factor structures or measurement properties, interpretation and application becomes challenging, particularly when comparing scores across groups (Byrne, Shavelson, & Muthen, 1989; Byrne & van de Vijver, 2010). Consistency of factor structure and measurement properties across groups is known as factorial invariance.

Previous studies have investigated the psychometric properties of the PANES (Bergman, Grjibovski, Hagstromer, Sallis, & Sjostrom, 2009; Sallis et al., 2009; Sallis et al., 2010). To date, however, no one has evaluated the PANES factor structure or its invariance. The purpose of this study was to test a hypothesized second-order factor structure and the factorial invariance of the Physical Activity Neighborhood Environment Survey (PANES) between single-family vs. multi-family housing neighborhoods.

Methods

Participants

This study conducted a secondary data analysis on PANES ratings from participants in the Health is Power study (NCI R01CA109403) (Lee, Medina, et al., 2011), a multi-site, community based randomized controlled trial to increase physical activity or improve dietary habits and to investigate the relationship between neighborhood factors and physical activity in community-dwelling women residing in Harris or Travis County, Texas. Participants self-identified as African American (n=258) or Hispanic or Latina (n=152) and were between the ages of 25 and 60 years old, able to read, speak, and write in English or Spanish, not pregnant or planning to become pregnant within 12 months, not planning on moving within 12 months, physically inactive (< 30 min of physical activity per day on > 3 days per week), and free from health conditions that would be aggravated by physical activity. All study procedures were approved by the Committee for the Protection of Human Subjects at the University of Houston and informed consent was obtained from all participants. Data for this study were collected from 2006–2008, and analyses for this study were conducted in 2013. Demographic, anthropometric, and health measures were collected using the previously described protocols (Lee, Mama, et al., 2011; Pickering et al., 2005).

The Physical Activity Neighborhood Environment Survey (PANES) was administered at baseline data collection of the Health is Power study. The PANES is a 17-item survey assessing perceptions of the neighborhood environment for walking and bicycling (see Table 1 for paraphrased item content). This survey was developed for the International Prevalence Study of Physical Activity (IPS) (Bauman et al., 2009) and was previously referred to as the IPS environmental module (Alexander, Bergman, Hagstromer, & Sjostrom, 2006). The items have been shown to have high reliability, content/face validity and criterion validity (Sallis et al., 2010), and construct validity (Sallis et al., 2009).

Table 1.

Observed PANES mean item scores by neighborhood type.

Item Number and Content	Single-Family Housing (N=147) Mean (SD)	Multi-Family Housing (N=177) Mean (SD)	Effect Size*
PANES 2: Places to buy things are within easy walking distance of home.	2.2 (1.2)	2.8 (1.1)	0.23
PANES 3: 10-15 minutes walk to a transit stop from home.	2.9 (1.3)	3.4 (1.0)	0.24
PANES 4: Sidewalks on most of streets in neighborhood.	3.3 (1.1)	3.5 (0.9)	0.07
PANES 5: Facilities to bicycle in or near neighborhood.	2.5 (1.3)	2.6 (1.2)	0.01
PANES 6: Neighborhood has several free or low cost recreation facilities.	3.1 (1.1)	3.0 (1.1)	0.03
PANES 7: Crime rate in neighborhood makes it unsafe to walk at night.	2.5 (1.2)	2.8 (1.1)	0.14
PANES 8: Traffic makes it difficult or unpleasant to walk in neighborhood.	2.1 (1.1)	2.4 (1.1)	0.17
PANES 9: See many people being physically active in neighborhood.	2.9 (1.0)	3.0 (1.0)	0.04
PANES 10: Many interesting things to look at while walking in neighborhood.	2.4 (1.1)	2.5 (1.1)	0.06
PANES 12: Many four-way intersections in neighborhood.	2.8 (1.1)	2.9 (1.0)	0.05
PANES 13: Sidewalks in neighborhood are maintained and not obstructed.	2.9 (1.2)	2.7 (1.1)	0.10
PANES 14: Places for bicycling in neighborhood are maintained and not obstructed.	2.4 (1.3)	2.3 (1.1)	0.02
PANES 15: Traffic makes it difficult or unpleasant to bicycle in neighborhood.	2.2 (1.1)	2.7 (1.1)	0.23
PANES 16: Crime in neighborhood makes it unsafe to walk during the day.	1.8 (0.9)	2.2 (1.0)	0.22
PANES 17: Many places to go within easy walking distance of home.	2.6 (1.2)	2.8 (1.1)	0.10

Items are rated by respondents using a four-category Likert-type scale from Strongly Disagree to Strongly Agree.

^*Effect size for Wilcoxon rank sum comparisons between neighborhood types: “small” effect = 0.10–0.29; “medium” effect = 0.30–0.49; “large” effect ≥0.50.

To determine neighborhood type, PANES item 1, “What is the main type of housing in your neighborhood?” was used. Participants selected one of the following options, (a) Detached single-family housing, (b) Townhouses, row houses, apartments, or condos of 2–3 stories, (c) Mix of single-family residences and townhouses, row houses, apartments or condos, (d) Apartments or condos of 4–12 stories, (e) Apartments or condos of more than 12 stories, or (f) Don't know/Not sure. Participants who selected “detached single-family housing” were categorized as single-family housing neighborhood, and all others were categorized as multi-family housing neighborhood.

Data Analysis

Only participants with complete answers for the entire PANES were included (n=324). One item on the PANES was excluded (#11, number of motor vehicles at the household) because it was participant-level rather about the neighborhood environment (Bergman et al., 2009). Descriptive statistics were computed for participant characteristics and for the PANES items. PANES ratings were compared between neighborhood types using adjusted Mann-Whitney U tests.

Factorial invariance of a second-order factor model was tested using a series of nested confirmatory factor analysis (CFA) models (Byrne & Stewart, 2006; Chen, Sousa, & West, 2005; Millsap & Yun-Tein, 2004). Testing of each model required showing invariance for all preceding models:

• Model 1 -

  Configural invariance specifies only the second-order factor structure as invariant between groups, allowing all model parameters to vary between groups;

• Model 2 -

  First-order factor loadings invariance tested whether that the correlations of items and first-order latent factors are equivalent across groups, indicating measurement unit equivalence;

• Model 3 -

  Second-order factor loadings invariance tested whether the associations of first-order factors with the second-order factor were equivalent across groups;

• Model 4 -

  First-order (item) intercepts invariance tested whether groups differ on the item means, conditional on the distributions of the respective first-order factors;

• Model 5 -

  Second-order (factor) intercepts invariance tested whether groups differ on the first-order factors, conditional on the distribution of the second-order factor.

A preliminary configural model was specified using a previously reported four-component model (Bergman et al., 2009) as a starting point for developing the second-order configural model. This initial model was modified using a standard strategy (Jöreskog, 1993) to obtain the second-order structure. One item loading for each first-order factor was set equal to 1.0 to provide a scale for the first-order factors, and the same strategy was used for the second-order loadings to provide a scale for the second-order factor (Chen et al., 2005).

The models were fitted using multi-group CFA with robust maximum likelihood for ordinal data (Finney & DiStefano, 2006; Jöreskog & Sörbom, 2000). In brief, the observed ordinal item responses are characterized as discrete manifestations of a latent continuous distribution, which allows estimation of the group-specific matrices required to obtain the robust ML solution (Jöreskog & Sörbom, 2000). Model fit was evaluated using indices based on the Satorra-Bentler (SB) scaled chi-square (_SBχ²) statistic (Satorra & Bentler, 1994): non-normed fit index (NNFI), comparative fit index (CFI), root mean square error of approximation (RMSEA) with 90% confidence interval, and the standardized root mean square residual (SRMR).

To test the invariance models, Models 2 and higher were each compared to Model 1 using the following criteria (Hu & Bentler, 1999): (a) small changes (≤ .01) in NNFI (ΔNNFI) and CFI (ΔCFI); (b) small values for RMSEA (≤ .05) and SRMR (≤ .10); (c) upper bound of the 90% confidence interval for RMSEA ≤ .08; and (d) a p-value > .05 for the closeness-of-fit test, the probability that RMSEA ≤ .05.

Results

The average participant in the overall sample of both housing groups was middle-aged, obese, normotensive, and had completed some college (Table 2). The effect sizes for the differences in ratings between the housing groups for seven of the PANES items were small (0.10 to 0.24 whereas effect sizes of the other seven items were negligible < 0.10) (Table 1).

Table 2.

Sample characteristics by neighborhood type.

	Single-Family Housing (N=147)	Multi-Family Housing (N=177)	Total Sample (N=324)
	Mean (SD)	Mean (SD)	Mean (SD)
Age	45.3 (10.0)	42.4 (15.30)	43.7 (13.25)
BMI (kg/m2)	34.9 (10.0)	31.6 (8.4)	33.0 (9.3)
Systolic Blood Pressure (mmHg)	125.0 (15.3)	122.0 (15.6)	123.3 (15.5)
Diastolic Blood Pressure (mmHg)	78.5 (10.7)	75.0 (11.2)	76.6 (11.1)
Resting Heart Rate (bpm)	73.2 (10.2)	74.3 (10.0)	73.8 (10.1)
Body Fat (%)	42.0% (10.0%)	36.7% (11.6%)	39.1% (11.2%)
	Median	Median	Median
Education Category	Some College	High School	Some College
FPL Income Category	4.5	2	3

SD = standard deviation.

FPL = Federal poverty level, standardized to represent a family of four; 100% FPL represents a household income equal to the FPL. Categories: 1 = 0%–100% FPL; 2 = 101%–200% FPL; 3 = 201%–300% FPL; 4 = 301%–400% FPL; 5 = > 400% FPL.

Configural Model

We identified a well-fitted configural model with a second-order factor structure (Figure 1). By examining the content of the items that loaded to each of the factors, the three first-order factors representing different physical activity-related characteristics of the neighborhood were named “Amenable,” “Unsafe,” and “Walkable.” These three first-order factors loaded to the single second-order factor, named “Environment,” which represented the overall neighborhood physical activity environment.

Figure 1.

Configural invariance model (Model 1) showing a second-order factor structure for the PANES. All model parameters are represented by arrows: (A) first-order factor loadings, (B) second-order factor loadings, (C) item-level residuals, and (D) first-order factor residuals. All parameters were freely estimated between the two neighborhood types. First-order factor names in ellipses were developed based on the content of the items loading to each respective factor (see Table 1). Correlated errors among five pairs of items (items 2 and 3, 3 and 4, 5 and 10, 13 and 14, and 2 and 17) are not shown to simplify presentation.

Model 1 included several statistically significantly correlated item error terms specifically between items 2 and 3, 3 and 4, 5 and 10, 13 and 14, and 2 and 17. Inclusion of the correlated errors did not materially affect model parameter estimates but did improve model fit since they accounted for additional systematic variation. These same correlated errors were included in all subsequent models.

Invariance Testing

Model 2, invariance of first-order factor loadings, had good fit (Table 3), indicating that the associations of the PANES items with the respective first-order factors were invariant across groups. The SRMR value slightly exceeded the criterion value, but, given the acceptability of the other criteria, invariance testing proceeded to the next model.

Table 3.

Results of testing sequentially nested models for increasingly more restrictive forms of factorial invariance.

Model	Type of Invariance	NNFI	CFI	SRMR	RMSEA (90% CI)	Closeness-of-fit p-value	df	SBχ2*
1	Configural	.983	.987	.088	.038 (.016 – .055)	.871	162	200.13
2	First-order loadings	.974	.978	.106	.048 (.031 – .062)	.596	173	235.80
Model 2 vs. 1		−.009	−.009				11	98.76
3	Second-order loadings	.969	.974	.117	.052 (.037 – .066)	.410	176	251.81
Model 3 vs. 1		−.002	−.013				14	132.37
4	Item intercepts	.954	.958	.119	.063 (.063 – .075)	.047	191	313.23
Model 4 vs. 1		−.018	−.029				29	358.83
4A	Partial invariance of item intercepts	.964	.968	.116	.056 (.042 – .069)	.226	187	281.45
Model 4A vs. 1		−.006	−.019				25	205.53

^*Satorra-Bentler scaled chi square, which was not used directly in testing model fit, but was used in the computation of the other fit indices.

Model 3, invariance of second-order factor loadings, also demonstrated good fit (Table 3), indicating that the associations of the first-order with the second-order factor were invariant across groups. Similar to Model 2, only SRMR slightly exceeded the criterion value.

Model 4, invariance of item intercepts, had unacceptable fit (Table 3), indicating that the groups had responded differently to some of the items despite having similar perceptions of the respective characteristics. The modification indices for the item intercepts were used to develop Model 4A, a partial invariance model. In Model 4A, the item intercepts for the single family housing group were found to be lower for PANES item 2, item 15, and item 16, and higher for item 3. Model 4A (Figure 2), partial invariance of item intercepts, demonstrated acceptable fit. Since testing higher levels of invariance depend on demonstrating invariance at all preceding lower levels, invariance testing concluded with Model 4A. Model 5, invariance of second-order intercepts (i.e., differences in first-order factor means) was not tested.

Figure 2.

Final second-order model (Model 4A) showing invariance of all first and second-order factor loadings, providing strong evidence of construct validity, with partial invariance of item intercepts. All unstandardized parameter estimates shown in this figure were constrained to be equal for both neighborhood type groups; standard errors of these estimates are shown in parentheses. The parameters marked with asterisks (*) are residual terms that were allowed to vary between the two neighborhood type groups; invariance of those parameters were not required for this model. The same correlated errors as described for Figure 1 were included but are not shown to simplify presentation.

Discussion

This study provides strong evidence that the PANES has factorial invariance (i.e., construct validity) between neighborhood types with different housing densities. The analyses confirmed the hypothesized second-order factor structure and invariance between neighborhood types for the relationships of the PANES items to the first-order factors and for the relationships of the first-order factors to a single second-order factor. Thus, researchers can be confident that the PANES is measuring the same constructs for perceptions of neighborhood characteristics related to physical activity across neighborhood types.

The PANES measures a higher-order factor (Environment) through three lower-order factors (Amenable, Unsafe, Walkable). The lower-order factors represent specific neighborhood characteristics, whereas the complete set of items may be considered a general measure of a neighborhood’s physical activity-related environment. This is consistent with the intent and design of PANES (Bauman et al., 2009; Oyeyemi, Adegoke, Oyeyemi, & Fatudimu, 2008; Sallis et al., 2010).

The partial invariance of some item intercepts indicate differences in the groups’ average responses, meaning that certain items function somewhat differently for inhabitants of single-family housing and multiple-family housing neighborhoods despite having similar average perceptions of the neighborhood. The single-family housing neighborhood group had lower intercept values for PANES items 2, 15, and 16. Despite similar perceptions of their neighborhood as being Walkable and Unsafe, the single-family housing neighborhood group reported lower ratings for these items compared to those in the multi-family housing neighborhood group. For example, for item 16 participants in the single-family housing neighborhood group would require a higher average perception of their neighborhood as Unsafe before agreeing that “the crime rate…makes it unsafe to go on walks during the day”. Perhaps there are unmeasured individual or neighborhood factors, such as differences in neighborhood cohesion or sensitivity to the actual surroundings that modify perception of these items. Alternately, crime may have been lower in the single-family housing neighborhoods, such that despite similar perceptions of it being Unsafe the neighborhood’s low crime does not actually interfere with walking during the day. The intercept for item 3 was lower in the multiple-family housing neighborhood group, so people in the multiple-family housing neighborhood group were more likely to agree that transit stops are within a 10–15 minute walk from home despite having a low perception of the neighborhood as Walkable. This likely reflects that neighborhoods with higher housing density often also have a higher density of public transit service stops although the sidewalks may be in disrepair (items 4 and 13), making it less “Walkable”.

The initial and all subsequent models included the same set of correlated errors among some of the PANES items for both groups, representing systematic variation among items that is not captured by the model’s latent variable structure (Byrne et al., 1989). Correlated errors may arise from similar phrasing of items, such as PANES items 13 and 14 both query whether aspects of the “neighborhood are well maintained.” Correlated errors may also indicate associations that are unaccounted for by the hypothesized constructs. In the current study, the correlated errors between item 5 (bicycling paths) and item 10 (“interesting things to look at while walking”) may indicate that bicycling paths tend to occur with walking paths in neighborhoods with pleasant surroundings (e.g., riverside, wooded areas).

What Does This Study Add?

Evaluation and interpretation of group differences on PANES scores may be informed by our results. The partial invariance of intercepts among some items suggests some minor differences in how inhabitants of single-family housing neighborhoods and multi-family housing neighborhoods respond to some items. The practical consequence of partial invariance of the item intercepts is that the PANES observed scores may differ slightly between single-family housing neighborhoods and multi-family housing neighborhoods as a result of differential item response rather than actual differences in the underlying factors (Byrne & Stewart, 2006). For this reason, Model 5 (invariance of first-order factor means) was not tested. Comparison of the first-order factor intercepts is likely to show group differences, although such differences may be caused by a lack of invariance (i.e., differences relating the second-order factor distribution to the first-order means), by the differential function of 4 items (i.e., lack of invariance of item intercepts), true mean differences between groups, or some combination. However, the invariance of 11 of the 15 item intercepts and all of the first- and second-order loadings provides strong evidence of partial factorial invariance and construct validity of the PANES. Future work in more diverse samples and using additional information about the sampled neighborhoods and participants may further clarify the psychometric properties of the PANES.