Does neighborhood collective efficacy for families change over time? The Boston Neighborhood Survey

Abstract

There is an increased interest in how neighborhood social processes, such as collective efficacy, may protect mental health. Yet little is known about how stable these neighborhood processes are over time, or how to change them to influence other downstream factors. We used a population-based, repeat cross-sectional study of adults (n=5135) to assess stability of collective efficacy for families in 38 Boston neighborhoods across 4 years (2006, 2008, 2010) (the Boston Neighborhood Survey). We test temporal stability of collective efficacy for families across and within neighborhoods using 2-level random effects linear regression, fixed effects linear regression, T-tests, and Wilcoxon rank tests. Across the different methods, neighborhood collective efficacy for families remained stable across 4 years, after adjustment for neighborhood composition. If neighborhood collective efficacy is measured within 4 years of the exposure period of interest, assuming temporal stability may be valid.

Introduction

A burgeoning literature has documented the association between individual mental health and neighborhood social and structural factors (Kim, 2008; Mair, Roux, & Galea, 2008; Osypuk, Schmidt, et al., 2012; Osypuk, Tchetgen Tchetgen, et al., 2012; Simons et al., 2002; Xue, Leventhal, Brooks-Gunn, & Earls, 2005). Researchers posit three ways in which neighborhoods can affect mental health: 1) norms/social processes, such as neighborhood collective efficacy, minimize negative processes that could affect mental health (Ingoldsby & Shaw, 2002; Leventhal & Brooks-Gunn, 2000); 2) institutional resources, such as youth programs, facilitate healthy development (Leventhal, Dupéré, & Gunn, 2009; McLoyd, Jayaratne-Epstein, Ceballo, & Borquez, 1994); or 3) neighborhood context shapes parental well-being, household processes, and parenting styles, such as parental monitoring, that affect child and adolescent development (Caldwell, Rafferty, Reischl, De Loney, & Brooks, 2010; Earls & Carlson, 2001).

The bulk of this literature is relatively new, especially in regards to measuring social processes. Neighborhood collective efficacy is one such social process that has recently been linked to mental health. Studies have shown that collective efficacy interacts with other variables to predict depression (Vega, Ang, Rodriquez, & Finch, 2011) and suicide attempts (Maimon, Browning, & Brooks-Gunn, 2010), and indirectly relates to internalizing behaviors (Dupéré, Leventhal, & Vitaro, 2012). To the extent that neighborhood processes like collective efficacy remain stable over time, it could mean lasting poor mental health outcomes among residents in affected neighborhoods. Prior studies have not demonstrated how to change collective efficacy or other social processes, so even if they causally influence mental health or other outcomes, it may represent a fruitless direction for upstream prevention.

A first step in understanding this causal process is to assess whether neighborhood collective efficacy exposures change over time, including to move beyond the predominantly cross-sectional research evidence. Some evidence of neighborhood effects on mental (and physical) health comes from longitudinal studies (see e.g., Cerdá, Diez-Roux, Tchetgen Tchetgen, Gordon-Larsen, & Kiefe, 2010; Nandi et al., 2010), including one experimental study of housing mobility (see e.g., Ludwig et al., 2011; Orr et al., 2003; Osypuk, Schmidt, et al., 2012; Osypuk, Tchetgen Tchetgen, et al., 2012). However, the majority of studies have been cross-sectional (Kim, 2008; Mair, et al., 2008).

Theoretically, changes in neighborhood exposures may be generated by virtue of residents themselves moving to different neighborhoods (residential mobility), by place-based fluctuations in resident composition (driven by residential turnover), or by place-based physical or social changes (e.g. neighborhood revitalization), indicating several different approaches for modeling change. Although prior work has demonstrated that most neighborhoods are stable social systems that change slowly unless disrupted by ‘triggering events’ such as disinvestment or demolition (Skogan, 1986), researchers testing the stability of neighborhood characteristics have primarily examined sociodemographic features (Alba, Denton, Leung, & Logan, 1995; Jackson & Mare, 2007; Rohe & Stewart, 1996) facilitated by the collection and dissemination of decennial census data in administrative units (e.g., census tracts) used as proxies for neighborhoods (Chaix, 2009; Messer & Kaufman, 2006). Given the intrinsic limitations of census data for capturing the social characteristics of neighborhoods as they are experienced by residents, many studies of neighborhood effects on health have shifted away from a sole reliance on such administrative data operationalizing neighborhood context using compositional variables (e.g., poverty rate or ethnic composition of census tracts), towards using resident surveys to operationalize neighborhood context. Indeed, such survey-based data may represent the most nuanced means of measuring social processes and social resources in neighborhoods, including collective efficacy (Raudenbush & Sampson, 1999).

Collective efficacy is a measure of how confident residents are that their neighbors trust each other and will work together for the good of the neighborhood (Sampson, Raudenbush, & Earls, 1997). Collective efficacy has roots in Albert Bandura’s work on perceived self-efficacy; extending the notion of “efficacy” to communities, Bandura postulated that the strength of a community could be at least partially attributed to the residents’ beliefs that collective problems can be solved by working together. The stronger this belief, the more effort is invested in collective behaviors for the good of the community (Bandura, 1995). A lesser studied component of collective efficacy that may be particularly relevant to raising children is intergenerational closure, defined as the degree to which parents and children in a neighborhood know and interact with each other (Sampson, Morenoff, & Earls, 1999). We include intergenerational closure in our operationalization of collective efficacy, which we call collective efficacy for families, because we are interested in facets of interconnectedness between adults and youth in communities that are not captured in the traditional measure.

Researchers applying ecometric methods (Raudenbush & Sampson, 1999) to understand neighborhood health effects have documented that neighborhood-level social exposure variables derived from resident surveys are associated with individual outcomes. Empirically, higher neighborhood collective efficacy has been associated with better outcomes for youth and families across a number of domains, including decreased violence in neighborhoods with similar socioeconomic status (Morenoff, Sampson, & Raudenbush, 2001; Sampson, et al., 1997), less gun carrying by youth (Molnar, Miller, Azrael, & Buka, 2004), less delinquency and aggression among youth (Molnar, Cerda, Roberts, & Buka, 2008; Odgers et al., 2009), decreased intimate partner homicide and non-lethal partner violence (Browning, 2002), and decreased perpetration of adolescent dating violence (Jain, Buka, Subramanian, & Molnar, 2010; Rothman et al., 2011). Collective efficacy has also been associated with multiple educational and health-related outcomes (Kawachi & Berkman, 2003), including higher passing rates on standardized math tests among low-income adolescents (Emory, Caughy, Harris, & Franzini, 2008), improved school and substance use outcomes (Coley, Morris, & Hernandez, 2004), later onset of first sexual activity (Browning, Leventhal, & Brooks-Gunn, 2004), better internalizing behavior scores (Xue, et al., 2005), higher birth weight (Buka, Brennan, Rich-Edwards, Raudenbush, & Earls, 2003), better self-rated health (Browning & Cagney, 2002; Wen, Browning, & Cagney, 2003), and reduction in asthma (Cagney & Browning, 2004).

Neighborhood collective efficacy may be dynamic across time as social relationships change or people in the neighborhood move, yet most studies using neighborhood resident surveys to operationalize social context are cross-sectional (Sampson, Morenoff, & Gannon-Rowley, 2002), and are rarely contemporaneous with the measurement of the health outcomes. For example, Buka et al. (2003), Cagney and Browning (2004), Xue et al. (2005), Maimon et al. (2010), and Dupéré et al. (2012) all use data from the Project for Human Development in Chicago Neighborhoods (PHDCN), which collected exposure measures of neighborhood social processes in 1994–1995 across neighborhoods in the city of Chicago. Outcomes, including mental health, whether from PHDCN or alternative sources, were measured anywhere from one to five years after the neighborhood data, requiring the assumption that either 1) neighborhood social context is stable over time, or, 2) that the neighborhood exposure impacts the health outcome across that particular latency period. However, these assumptions have not been tested to our knowledge, and if they are invalid, conclusions drawn from studies that rely on these assumptions may be dubious.

To date, only one study has examined the stability of neighborhood social processes over time. In a recent book on Chicago neighborhoods using data from the PHDCN, Sampson examined whether collective efficacy of Chicago neighborhoods remained stable or changed from 1995 to 2002 using two waves of data (Sampson, 2012). Sampson found that collective efficacy remained stable over this 7-year period, as did the structural factors (e.g., inequality) that are hypothesized to drive changes in collective efficacy over time. However, having only two waves of data inherently constrained Sampson’s ability to assess neighborhood stability over time and neighborhood processes or structure in Chicago may differ markedly from those in other US cities (Osypuk, Galea, McArdle, & Acevedo-Garcia, 2009).

The overarching objective of this paper is to test whether neighborhood collective efficacy for families (Sampson, et al., 1999; Sampson, et al., 1997) remains stable across time using a unique source of population-based neighborhood resident survey data in Boston, MA, comprised of three waves of data across a 4-year period. This study makes several contributions to the literature. First, the availability of three waves of data allows us to employ an array of simple to sophisticated methodological techniques to test our hypotheses, over and above the correlations reported in Sampson’s analysis of Chicago neighborhoods, thereby providing evidence beyond that of cross-sectional or pre-post designs. Second, we examine a frequently studied social construct --collective efficacy-- that has been linked to mental health outcomes, in a city other than Chicago. Finally, we include an additional validated scale (intergenerational closure) to operationalize collective efficacy that is specific for neighborhood processes influencing children and families.

Methods

Data for this study come from the 2006, 2008, and 2010 Boston Neighborhood Survey (BNS). The BNS is a repeat cross-sectional, random-digit-dial telephone survey conducted by the opinion research firm Fact Finders, Inc (http://www.factfinders.com/) on behalf of the Harvard Injury Control Research Center. One adult 18 years or older was randomly selected from each household to answer questions about neighborhood norms and processes (Azrael et al., 2009) (n=1707 adults in 2006, n=1710 in 2008, and n=1718 in 2010 for a total of 5135). Potential respondents were stratified by each of Boston’s 16 larger neighborhoods (e.g., Roxbury, Dorchester, South Boston), with sampling proportional to neighborhood population size (Azrael, et al., 2009). Interviewers administered the survey in English and Spanish. Interviewers obtained verbal informed consent prior to administering the survey.

Measures

Collective efficacy for families was measured by combining three validated neighborhood scales (social cohesion, informal social control, and intergenerational closure) (Sampson, et al., 1999; Sampson, et al., 1997). Table 1 lists the coding scheme, number and content of scale items, scale internal consistency reliability, and response categories for each item. Responses of don’t know/no opinion were coded as missing because we could not differentiate between a respondent with a neutral opinion/no opinion and one who did not know enough about their neighborhood to respond.

Table 1.

Collective Efficacy Scale & Composite Scales Variable Coding

Scale	Higher Values Indicate	No. of Items	Cronbach's Alpha	Items	Rating Scale
			All Years
Social Cohesion	More cohesion	5	0.85	In my neighborhood: people can be trusted; people are willing to help their neighbors; people know and like each other; people get along with each other; people share the same beliefs about what is right and wrong	1=strongly agree to 4=strongly disagree; reverse coded
Informal Social Control	More social control	5	0.82	In your neighborhood, how likely is it that your neighbors would: organize together to keep a fire station open that was going to close because of budget cuts; do something about neighborhood children skipping school and hanging out on a street corner; do something about a child showing disrespect to an adult; do something about a child spray-painting graffiti on a local building; do something if there was a fight in your neighborhood and someone was being beaten or threatened	1=very likely to 4=very unlikely; reverse coded
Intergenerational Closure	More connection between adults/children	4	0.82	In my neighborhood: there are adults that children can look up to; you can count on adults to watch out that children and teenagers are safe and stay out of trouble; parents know one another; parents know their children's friends	1=strongly agree to 4=strongly disagree; reverse coded
Collective Efficacy for families	More collective efficacy	14	0.93	All items described above	Described above*

Table Note

^*Scores for the Collective Efficacy for Families scale were recoded from the original scale items using IRT methods for the final models.

To confirm that the 14 items across the 3 scales were unidimensional and reliable, we ran a factor analysis and calculated Cronbach’s alpha. Internal consistency reliability was very high, α=.93; moreover, items loaded on a single factor, with the first factor explaining 96% of the variance.

We then applied Item Response Theory (IRT) methods to calculate the collective efficacy measure, combining the three ordinal scales into a factor score with an approximately standard normal distribution. As such, a one-unit change in collective efficacy in our models corresponds to an approximately one standard deviation (SD) change (mean(SD)= −1.38E-04(.95)). IRT is a more flexible latent variable method for creating scales compared with simply summing items; IRT gives greater weight to items having a stronger relationship to the underlying construct and therefore decreases measurement error compared to simple mean scales (Hambleton & Swaminathan, 1985). We estimated the IRT score using Mplus software and had very little missing data on our outcome (0.2%).

We adjusted for year linearly (2006=0, 2008=1, 2010=2), sex, age, race/ethnicity, nativity (foreign-born vs. US-born), education, income, length of time in neighborhood (neighborhood tenure), and home ownership in multivariate analyses. All covariates had <2% missing except income (19%), which was modeled using a missing indicator. Cases with missing covariate data (except income), missing outcome data, or missing a neighborhood level identifier were excluded from multivariate analyses (n=232, 4.5% of sample).

Neighborhood Definition

The City of Boston conventionally divides its geography into 16 large neighborhoods. These divisions, which range in size up to 90,000 residents, are far too large for understanding the relationships of social processes such as collective efficacy, which are hypothesized to operate at smaller spatial scales. As such, as part of the larger project in which the BNS is embedded, the research team worked with key informants in sub-neighborhoods of the city who inspected maps and used their local knowledge to define 38 socially relevant “neighborhood clusters” comprised of multiple contiguous census blocks. Details of this neighborhood formulation process are described elsewhere (Azrael, et al., 2009). The BNS survey achieved a mean sample size of 45 respondents in each of these 38 neighborhoods (SD=42; median=28–31 respondents across years), with a minimum neighborhood sample size of 4 and a maximum of 203, representing sufficient sample sizes for estimating neighborhood ecometric constructs (Raudenbush & Sampson, 1999).

Analysis

We applied multiple methods to test the temporal stability of neighborhood collective efficacy for families, including comparing means over time, between and within neighborhoods. All hypothesis tests were conducted at nominal type 1 error=0.05. Individual-level data were used for some analyses; neighborhood-level measures were created for other analyses by aggregating data to calculate a mean collective efficacy value for each of the 38 neighborhood clusters for each year (2006, 2008, and 2010). We began with a basic test of difference across time, using a paired t-test to assess the change in collective efficacy from 2006 to 2010 across neighborhoods (i.e., using neighborhood-level data) and then separate t-tests within each of the 38 neighborhoods (i.e., using individual-level data stratified by neighborhood). Next, we used a Wilcoxon rank sum test to assess whether the rank ordering of neighborhoods changed for collective efficacy from 2006 to 2010 (i.e., using neighborhood-level data). Although these analyses provide simple tests of the null hypothesis that neighborhood collective efficacy for families remained stable over time, these analyses fail to adjust for covariates that might influence respondents’ reporting of their neighborhood social environment.

To obtain a more powerful test of neighborhood change, we employed a two-level random effects multi-level model, which explicitly accounted for the nesting of individuals within neighborhoods, to test whether there was neighborhood variation in change in collective efficacy over time. For each neighborhood, we had measurements of collective efficacy in 2006, 2008, and 2010, so we estimated a two-level model using SAS proc mixed, with respondents (level 1) nested in neighborhoods (level 2). We estimated both a crude model and a covariate-adjusted model to test for compositional effects of neighborhoods. The model contained a random intercept for neighborhoods, a fixed effect for year, and a random slope for year, which allows for neighborhood-specific rates of change that captures heterogeneity across neighborhoods. The equation for this model is:

$${\mathit{\text{Collective Efficacy for Families}}}_{ij}={\gamma }_{00}+{\gamma }_{10}{\mathit{\text{Year}}}_{ij}+{\zeta }_{0i}+{\zeta }_{1i}{\mathit{\text{Year}}}_{ij}+{\epsilon }_{ij}$$ (1)

where γ₀₀ is the overall neighborhood intercept, γ₁₀ is the overall rate of change (fixed effect of year; a one unit change in this time variable is equal to a 2-year change), ζ_0i is the level 2 neighborhood random intercept variance, ζ_1i is the level 2 year random slope (a one unit change in this time variable is equal to a 2-year change), and Ɛ_ij is the level 1 residual error. The parameters of interest are the fixed effect of year and the variance-covariance matrix of random effects. We are particularly interested in whether there is evidence of moderate to large heterogeneity in the rate of change across neighborhoods, which would indicate that the time trend for some neighborhoods is significantly different from the average (fixed effect) of the time trend in collective efficacy for all of Boston.

The random effects model is a powerful framework for evaluating change, but it relies on the assumptions that unobserved heterogeneity is uncorrelated with other variables in the model and that the residuals are normally distributed (Gardiner, Luo, & Roman, 2009; Snijders, 2005). We tested multilevel model assumptions (functional form, normality, and homoscedasticity) (Singer & Willett, 2003), and we found no evidence of a violation of the assumptions. However, it is conceivable that any unobserved heterogeneity is not independent of the explanatory variables, which could lead to inconsistent random effects (Gardiner, et al., 2009). As a supplemental analysis, we next estimated a neighborhood fixed effects model, which effectively removes unobserved heterogeneity from the model by including interactions between the explanatory variable (year) and the level 2 units (neighborhoods) (Snijders, 2005). We first estimated a crude model, and then a covariate-adjusted model, to test for compositional effects, which included the aforementioned covariates, a linear variable for survey year, an indicator for each of the 37 neighborhoods (vs. a reference 38^th neighborhood), and an interaction term between year and each neighborhood indicator. The focus here was on the joint test of significance for the interaction terms, which tests the null hypothesis that there is no overall change across years across neighborhoods.

Results

Table 2 provides descriptive information on the pooled year sample (see Appendix Table 1 for descriptive information by year). The sample was predominately female (59%) and the average age of respondents was approximately 53 years. The sample was 20% black, 63% white, 9% Hispanic, and 8% other race; 21% of respondents were foreign born. The modal education level was some graduate level education (30%). Of those who reported income, the modal income level was $40–80,000 (26%). Respondents reported living in their neighborhood for an average of 20 years; 59% were homeowners and 37% were renters. The distribution of some demographic variables significantly changed from 2006 to 2010 (see Appendix Table 2); in 2010, the sample was more female, slightly older, reported slightly higher income, had lived in their neighborhood longer, and were more likely to be homeowners.

Table 2.

Individual-level descriptive statistics for Boston Neighborhoods Survey - 2006, 2008, and 2010

Variable		Subgroup n	Total n	Mean	SD	Min	Max
Male		2123	5135	0.41	0.49	0	1
Female		3012	5135	0.59	0.49	0	1
Age		--	5074	53.39	16.61	18	99
Race/Ethnicity
	Black	1013	5032	0.20	0.40	0	1
	White	3145	5032	0.63	0.48	0	1
	Hispanic	462	5032	0.09	0.29	0	1
	Other	412	5032	0.08	0.27	0	1
Foreign Born		1078	5126	0.21	0.41	0	1
Education
	< HS	364	5096	0.07	0.26	0	1
High school diploma or GED		1049	5096	0.21	0.40	0	1
	Some College	1205	5096	0.24	0.42	0	1
	Bachelors	971	5096	0.19	0.39	0	1
	Graduate Study	1507	5096	0.30	0.46	0	1
Annual Income
	<20K	790	5135	0.15	0.36	0	1
	20–40K	875	5135	0.17	0.38	0	1
	40–80K	1096	5135	0.21	0.41	0	1
	80–100K	452	5135	0.09	0.28	0	1
	>100K	944	5135	0.18	0.39	0	1
	Missing Income	978	5135	0.19	0.39	0	1
Neighborhood Tenure		--	5119	20.02	17.59	0	94
Home Owner		3016	5090	0.59	0.49	0	1
Renter		1865	5090	0.37	0.48	0	1
Social Cohesion a		--	5135	3.63	0.66	1	5
Informal Social Control a		--	5135	3.81	0.84	1	5
Intergenerational Closure a		--	5135	3.61	0.67	1	5
Collective Efficacy for Families IRT
Score		--	5125	−1.38E-04	0.95	−3.59	2.41

^aBefore creating the mean score, items were recoded to a 5-point scale and missing data was imputed to the row-column mean (on average, 13% of scale items were missing).

Neighborhood Stability Analyses

We began with t-tests to examine differences in collective efficacy for families using data from 2006 and 2010, both between and within neighborhood clusters (Table 3). Across neighborhood clusters (i.e., Panel 3a, using neighborhood-level data), there was a significant increase in collective efficacy in Boston from 2006 to 2010 (paired t-test p=.04). Within neighborhood clusters (i.e., Panel 3b, using individual-level data stratified by neighborhood cluster), we found a statistically significant difference in 3 neighborhood clusters (8%), without correcting for multiple testing (i.e., running 38 separate t-tests; this difference was greater than the 5% expected by chance). However, once we adjusted for multiple testing using a Bonferroni correction (adjusted p=.05/38=.001), none of the t-tests for neighborhood collective efficacy change achieved the Bonferroni-corrected significance level.

Table 3.

T-tests between and within 38 Boston neighborhoods; Testing change in collective efficacy for families 2006 to 2010

3a. Between Neighborhoods Paired t-test; N=38
Mean (SD)
2006	2010	t	p-value	p
−.13(.42)	−.02(.37)	−2.15	0.04	*

3b. Within Neighborhoods t-test; N=5125, stratified by neighborhood
Without Bonferroni			With Bonferroni†
#	%	p-value	#	%	p-value
significant tests (of 38)	significant tests		significant tests (of 38)	significant tests
3	8%	0.05	0.00	0%	0.001

^#p<.10

^*p<.05

^**p<.01

^***p<.001

^†Bonferroni correction is .05 divided by the number of tests; for each neighborhood cluster (n=38) we are testing one outcome for 38 neighborhoods, so .05 is divided by 38.

NOTES: The sample size for section 3a is 38, because the t-tests are done using neighborhood level data; the sample size for section 3b is 5125, because the t-tests are done at the individual level, but stratified by neighborhood -- the neighborhood specific sample sizes range from 4 to 203 cases.

A Wilcoxon rank sum test examined whether any neighborhoods differed in rank ordering on collective efficacy from 2006 to 2010, using neighborhood-level means. The test was non-significant (z-statistic= −1.12, p=.26), indicating that neighborhoods achieved no significant change in their neighborhood ranking across Boston during this time.

Table 4 presents the results from crude and covariate-adjusted two-level random effects models. There was a significant random intercept for neighborhood (adjusted σ₀²(SE)=.049(.018)), p=.004) indicating that neighborhoods significantly varied from one another in their 2006 cross-sectional collective efficacy score. However, we found no statistical evidence of a random slope for year (adjusted σ₁²(SE)=.001(.003), p=.370), and no significant covariance between intercept and year slope (adjusted σ₀₁(SE)=-.002(.006), p=.805), indicating no change in collective efficacy across neighborhoods over time. We did find a statistically significant crude estimate of the fixed effect of year (b(SE)=.029(.015), p=.05) on collective efficacy, which suggests collective efficacy did change in Boston across this time; however, this effect disappeared upon adjusting for individual-level covariates (b(SE)=.004(.017), p=.827). Ten percent of the variation in the outcome was due to differences between neighborhood clusters (Intra-cluster correlation (ICC)=.103; calculated from unadjusted random intercept model). Although this is a larger ICC for a neighborhood study, it does indicate that most of the variation in collective efficacy for families was due to differences between individuals within neighborhoods, in line with other neighborhood ecometric studies (Hox, 2002; Morenoff, 2003; Morenoff, Diez Roux, Osypuk, & Hansen, 2006; Raudenbush & Sampson, 1999; Snijders & Bosker, 1999).

Table 4.

Two-level random effects model; Testing change in collective efficacy for families 2006, 2008, and 2010

	Unadjusted Model				Covariate-Adjusted Model
	b	SE	p-value	p	b	SE	p-value	p
Variance Parameter Estimates
Random Neighborhood Intercept	0.097	0.029	0.001	***	0.049	0.018	0.004	**
Random Year Slope	−0.001	0.002	0.602		0.001	0.003	0.370
Covariance between Intercept-Slope	−0.001	0.007	0.907		−0.002	0.006	0.805
Residual	0.829	0.017	<.0001	***	0.772	0.016	<.0001	***
Fixed Effects
Year	0.029	0.015	0.05	*	0.004	0.017	0.827
AIC		13102.9				12806.1
N		4903				4903

^#p<.10

^*p<.05

^**p<.01

^***p<.001

NOTE: Covariate-adjusted model adjusted for gender, age, race/ethnicity, foreign born status, education, income, neighborhood tenure, and home ownership

Finally, we estimated fixed effects unadjusted and covariate-adjusted linear regression models, with an indicator for year, for each neighborhood, and for neighborhood-year interactions. Table 5 summarizes the F-test comparing the full (with neighborhood-year interactions) and reduced (without interactions) models. The F-test was not significant in the unadjusted models (p=.22). However the F-test was marginally significant for the covariate adjusted model (p=.07), suggesting that, after controlling for all unmeasured time-stable characteristics of neighborhoods, as well as differences in measured respondent characteristics, the neighborhoods marginally differed in the slope change of collective efficacy for families from 2006 to 2010.

Table 5.

Neighborhood fixed effects model; Testing change in collective efficacy for families 2006, 2008, and 2010

	Unadjusted Model			Covariate-Adjusted Model
	Full		Reduced	Full		Reduced
Linear Effect of Year b(SE)	−.016(.046)		.031(.016)	−0.064(.045)		.001(.016)
Linear Effect of Year p-value	0.74		.05 #	0.16		0.93
R2	0.09		0.09	0.16		0.15
Joint F-Test of Interactions		1.17			1.36
p-value		0.22			.07 #
N		4907			4907

^#p<.10

^*p<.05

^**p<.01

^***p<.001

NOTES: Covariate-adjusted model adjusted for gender, age, race/ethnicity, foreign born status, education, income, neighborhood tenure, and home ownership. The F-test above is comparing a full model to a reduced model. The null hypothesis test is that the interaction terms do not significantly improve the model. In other words, the joint effect of the year*neighborhood interactions is equal to zero. The full model was estimated with post-estimation commands that give a sub-group analysis comparing, within each neighborhood, the effect of time on the construct of interest. The critical value with F(α=.05; df=37, 4907)=1.41. We tested for nonlinearity of year using a squared term and found no evidence of a non-linear effect of year.

Discussion

Our analysis of population-based neighborhood survey data in Boston tested whether collective efficacy for families remained stable across time, which has implications for causal questions regarding how neighborhood collective efficacy can potentially prevent mental health problems or other individual outcomes. Overall, our analyses could not reject the null hypothesis that collective efficacy for families in Boston neighborhoods remained stable over a 4-year period.

Although we found that Boston residents reported significantly higher collective efficacy in 2010 vs. 2006 in unadjusted analyses (mean comparison and fixed effect of year in multilevel regression), the time pattern was accounted for by compositional differences across time, which may be due to true population change across time, or to changes in BNS sample response patterns across time. Using covariate-adjusted regression, we found no statistical evidence of significant change in collective efficacy for families overall across neighborhood clusters. This suggests that it is important to adjust for population composition when assessing change not due to variation in person-level characteristics. However there also may be some argument for not adjusting for population composition. For example, if changing the population composition is one feasible way to increase (or decrease) collective efficacy, (suggesting that changing population composition is a cause of collective efficacy), then researchers may want to model it in its naturally occurring form.

The neighborhood fixed effects model was suggestive of potential variation across neighborhood cluster-years in change in collective efficacy for families 2006–2010 given the marginally-significant F-test result, but this was not confirmed by the random effects model. The fixed and random effects models are closely related in that they both account for the clustered nature of the data and incorporate the fact that observations within clusters are likely to be correlated. They differ, however, in one important respect: the random effects model incorporates intra-cluster correlation and assumes that it is uncorrelated with other variables in the model, and makes across neighborhood comparisons (Petersen, 2004). It also is drawn from a common random effect distribution, suggesting that random effect-neighborhoods are exchangeable (i.e., that the effects are unconfounded) (Oakes, 2004), which may or may not be a reasonable assumption. The fixed effects model, on the other hand, removes all of the unobserved, time stable characteristics of neighborhoods from the model, and makes within neighborhood comparisons (Petersen, 2004). This model adjusts for possible lack of exchangeability by removing neighborhood level confounding (measured and unmeasured) that is stable over time. Although this provides a more consistent test of the stability of collective efficacy for families, it also estimates the random slope for year as if it were a fixed value and thus, to the extent that there is variability in change across neighborhoods, it may be underestimating the variance of this effect.

Given the different assumptions and treatment of unobserved heterogeneity in fixed versus random effects models, it is not surprising that we found slightly divergent results in our models. Despite the divergence, the neighborhood-time fixed effect test was only marginally significant, so we cannot rule out that it may be due to chance. Taken together, these results suggest that collective efficacy for families is stable over this 4-year, 3-wave period in the city of Boston. This is consistent with recent findings for collective efficacy over a 7-year, 2-wave period in Chicago (Sampson, 2012).

One potential explanation for our finding that collective efficacy for families was stable across time is that four years may be insufficient time for the construct to change, suggesting that studies that assume the stability of collective efficacy – at least over relatively short periods of time – may be justified in doing so. Higher levels of collective efficacy have been linked to positive neighborhood structural factors, such as having more parks and fewer alcohol outlets, after controlling for tract-level disadvantage and sociodemographic variables of individuals (Cohen, Inagami, & Finch, 2008). Thus, collective efficacy may be stable because of its covariation with structural components that do not change dramatically over a short period of time. This explanation is supported by Sampson’s recent work (2012) that found stability in both collective efficacy and the underlying structural components that influence collective efficacy (e.g., concentrated poverty).

Collective efficacy is thought to be a modifiable neighborhood process (Earls, Raviola, & Carlson, 2008). Indeed, interventions aimed at increasing collective efficacy are being attempted in some communities, including, most recently, Sub-Saharan Africa (Carlson, Brennan, & Earls, 2012; Earls, et al., 2008). However, to our knowledge only one study has documented that collective efficacy can indeed be changed with explicit, directed intervention. Even then, this intervention changed one very specific dimension of collective efficacy – youth collective efficacy related to HIV communication (as perceived by adults in the neighborhood) – but the intervention did not change a more general, global measure of neighborhood collective efficacy, although it attempted to do so (Carlson, et al., 2012). Aside from this intervention, most current evidence on collective efficacy derives from studies that typically examine between-neighborhood cross-sectional differences in the construct. As the field moves toward interventions, more evaluation studies are needed to assess whether efforts to mobilize the participation of community members for the collective good can modify collective efficacy.

Researchers commonly operationalize collective efficacy with two scales: social cohesion and informal social control (Sampson, et al., 1997). In sensitivity analyses, we used a traditional 10-item measure of collective efficacy (i.e., comprised of social cohesion and informal social control, Cronbach’s α=.89; see Sampson et al., 1997). The results were comparable. In this study, however, we used a more inclusive measure, incorporating a scale of intergenerational closure in order to tap an aspect of collective efficacy relevant to families and raising children (Sampson, et al., 1999). Intergenerational closure assesses the degree to which adults and children in a neighborhood are interconnected, which could have important consequences for the health of neighborhood residents, both children and adults. The three scales were very highly correlated (at .93) and loaded clearly on one factor, so adding this third scale to collective efficacy may improve measurement.

Limitations

We used a population-based survey, but cooperation rates (CR) were low, e.g. 39% in 2006, 32% in 2008, and 33% in 2010, calculated thus:

$$(\mathrm{C}\mathrm{R}=\frac{\mathit{\text{completed}}}{\mathit{\text{completed}}+\mathit{\text{partial}}+\mathit{\text{refused}}})$$ (2)

We recognize that low cooperation rates have the potential to bias results, especially with respect to external validity (Galea & Tracy, 2007). However these cooperation rates are consistent with a strong secular decreasing trend in survey participation (Galea & Tracy, 2007). With this limitation in mind, we assessed the generalizability of the BNS sample to the city of Boston by comparing demographic characteristics of BNS respondents with those obtained for the city of Boston from the 2005–2009 American Community Survey (ACS) conducted by the US Census Bureau (see Appendix Table 2). We then adjusted for all covariates in our sample that exhibited significantly different distributions from ACS estimates of the Boston population distributions. Thus we can still generalize our findings to Boston conditional on these covariates provided that these covariates account for generalizability differences between the BNS and ACS. Notably, empirical research has demonstrated that nonresponse bias is unlikely to substantially bias measures of interest (Galea & Tracy, 2007). Moreover, there is evidence that even studies that use volunteers, rather than random samples, can provide samples representative at the group level to study neighborhood effects (Oakes, Forsyth, Hearst, & Schmitz, 2009).

Our study also may not be generalizable to cities other than Boston. However, our finding that collective efficacy was stable in Boston neighborhoods across the 4-year period of our study, coupled with Sampson’s similar finding in Chicago over 7 years (Sampson 2012), suggests that studies that assume that collective efficacy is stable over relatively short periods of time may be justified in doing so.

Analyzing change with mean-difference tests carries with it some problems (Cronbach & Furby, 1970), including regression to the mean and the association between raw scores and random/measurement error. They also cannot account for covariates or for clustered data. We address these issues by also estimating more sophisticated neighborhood random and fixed effects models, which overcome many of those limitations.

Conclusion

Prior literature has provided very little information on how, and if, neighborhood social processes change over time (Sampson, et al., 2002), which is one criteria for demonstrating that such processes may be upstream causes of mental health or other individual outcomes (Hill, 1965). This study attempts to fill this gap by explicitly testing the stability of collective efficacy, a frequently studied neighborhood social process. We applied a range of tests, ranging from simple to more sophisticated methods including multi-level analysis, and we find that collective efficacy for families remains stable in Boston neighborhoods over a four-year period. These findings suggest that using existing cross-sectional community surveys to examine the links between social processes and health outcomes may be a reasonable alternative to developing community-level panel data (Bellair, 2000) to study the population level effects of community processes on mental and physical health.