Abstract
Background
In multilevel studies, strong correlations of neighbourhood exposures with individual and neighbourhood confounders may generate problems with non-positivity (ie, inferences that are `off-support'). The authors used propensity restriction and matching to (1) assess the utility of propensity restriction to ensure analyses are `on-support' and (2) examine the relation between collective efficacy and violence in a previously unstudied city.
Methods
Associations between neighbourhood collective efficacy and violent victimisation were estimated in data from New York City in 2005 (n=4000) using marginal models and propensity matching.
Results
In marginal models adjusted for individual confounders and limited to observations `on-support', under conditions of high collective efficacy, the estimated prevalence of violent victimisation was 3.5/100, while under conditions of low collective efficacy, it was 7.5/100, resulting in a difference of 4.0/100 (95% CI 2.6 to 5.8). In propensity-matched analysis, the comparable difference was 4.0/100 (95% CI 2.1 to 5.9). In analyses adjusted for individual and neighbourhood confounders and limited to observations 'on-support', the difference in violent victimisation associated with collective efficacy was 3.1/100 (95% CI 1.2 to 5.2) in marginal models and 2.4/100 (95% CI 0.2 to 4.5) in propensity-matched analysis. Analyses without support restrictions produced surprisingly similar results.
Conclusions
Under conditions of high collective efficacy, there was about half the prevalence of violence compared with low collective efficacy. The results contribute to a growing body of evidence that suggests collective efficacy may shape violence, and illustrate how careful techniques can be used to disentangle exposures from highly correlated confounders without relying on model extrapolation.
INTRODUCTION
Multilevel studies allow assessment of the relations between neighbourhood exposures and individual outcomes while accounting for individual and neighbourhood confounders and are thus commonly used in social epidemiology.1,2 However, when neighbourhood exposures have strong correlations with individual and neighbourhood confounders, some population subgroups (defined by combinations of confounders) may only experience one level of the exposure. This strong confounding due to social stratification has been described as structural confounding, and the population subgroups that only experience one level of exposure are described as `off-support' or non-positive.3–5 For example, in a multilevel study examining the relation of neighbourhood collective efficacy with violence, it might be that young men, who have low income, live in high-poverty neighbourhoods and share a set of other covariates, only experience low collective efficacy. Analyses of data with subgroups that are `off-support' rely on model extrapolation and are thus of questionable interpretability.3–5 To estimate the relation between collective efficacy and violence for the young men (described above), who only experience low collective efficacy, it is necessary to extrapolate based on the relation estimated among subgroups that do experience both low and high collective efficacy.
Propensity score restriction is one approach that can be used to assure inferences are `on-support' when non-positivity is a problem;6 analyses can be restricted to propensity score values for which there are different levels of exposure.7,8 For example, the young men described above would have propensity of 1.0 for low collective efficacy (none experienced high collective efficacy). If the highest propensity value in the high collective efficacy group were 0.95, then those in the low collective efficacy group with propensities above 0.95 (including the aforementioned young men) would be excluded from analysis. In this paper, we use propensity restriction in an analysis of a classic problem in social epidemiology: the relation between neighbourhood collective efficacy and violence. Our goals were to assess the utility of propensity restriction as applied to an important question and to assess the relation between collective efficacy and violence in a previously unstudied city. While propensity score matching applications are on the rise in social epidemiology, few illustrations have been published on the ways that propensity restriction can be used to ensure inferences from neighbourhood analyses are `on-support'.9–12 Propensity restriction combined with analysis approaches that account for clustering by neighbourhood offer an alternative to propensity stratification or matching for multilevel studies; there are some propensity score approaches that accommodate multilevel clustering, but they are not widely or easily implemented.13–15
Based on foundational sociological work on violence in urban communities,16–21 collective efficacy captures the social capacity that deteriorates in communities that have been marginalised and isolated by conditions of structural disadvantage.22 Communities suffering conditions of structural disadvantage no longer possess what WJ Wilson called “…social organization—a sense of community, positive neighborhood identification, and explicit norms and sanctions against aberrant behavior”.16 Collective efficacy has been conceptualised as encompassing the two components of social cohesion, defined as mutual trust and shared values, and informal social control, defined as willingness to intervene for the common good.22 This combination of mutual trust and willingness to intervene has been hypothesised to be a critical capacity for controlling violent behaviour.22 Collective efficacy may be a key link between structural disadvantage and rates of violent crime in urban areas,22 and it has been inversely associated with youth carrying firearms,23 dating violence victimisation24 and adolescent suicide attempts.25 Building on this work in Chicago,22,26,27 studies have documented relations between collective efficacy and community violent victimisation in Stockholm, Sweden,28 and Brisbane, Australia.29
There have been notable efforts in collective efficacy research to separate the associations of collective efficacy with violence from those of structural disadvantage measures that are markers of the historical processes that deteriorated collective efficacy.22,26,30,31 Supporting these efforts, recent work suggests that there are a variety of plausible causal structures that interrelate a neighbourhood exposure such as collective efficacy with neighbourhood structural disadvantage and an outcome such as violence; in some situations, neighbourhood structural disadvantage indicators are confounders so control is appropriate, while in others, they are colliders or on the causal pathway, and in those cases, control in a regression model is inappropriate.32 Since we cannot know which causal structure is `true', the best approach (when measurements over time are not available to disentangle causal inter-relations) is to present analyses of neighbourhood exposures with and without adjustment for neighbourhood structural disadvantage.
However, the documented strong correlations of collective efficacy with individual characteristics and neighbourhood structural disadvantage can generate problems with non-positivity.22,26 Therefore, in this paper, we examined the relation between neighbourhood collective efficacy and community violent victimisation in New York City (NYC) using propensity score approaches to assess positivity. Marginal models and propensity-matched analyses were conducted with restrictions based on the propensity score to assure inferences were `on-support', and results were compared. Marginal models without restrictions were also implemented to examine the impact that extrapolation may have on study conclusions.
METHODS
Data
Analyses were conducted with data from the New York Social Environment Study (NYSES), a multilevel study designed to examine neighbourhood-level exposures and mental and behavioural health outcomes. We used random-digit-dial methods to contact and interview 4000 residents of NYC in 2005. One adult 18 years or older was interviewed by telephone in each household. Fifty-four per cent of those contacted agreed to participate in the study. The study protocol received human subjects approval. Further study details are available elsewhere.33–36
Neighbourhoods
Respondents were geocoded and linked to their neighbourhoods of residence.34 The neighbourhoods were the 59 community districts in NYC. Community districts are recognisable neighbourhood areas, and many characteristics of these neighbourhoods have been associated with health indicators.33,34,36
Measures
Respondents were interviewed with a structured questionnaire that included questions on potential confounders including age, race/ethnicity, sex, marital status, place of birth, education, income, years lived in the current neighbourhood and interview language.
Neighbourhood collective efficacy was measured using the standard scale that includes social cohesion and informal social control subscales.22 The social cohesion subscale assesses residents' perceptions of the extent to which their neighbours are close-knit, are helpful, get along, share values and are trustworthy. The informal social control subscale measures perceptions of the likelihood that neighbours would intervene if children skipped school, sprayed graffiti or disrespected an adult; if there were a fight or if the city was closing a firehouse. The neighbourhood-level measure of collective efficacy was the average of all respondents in each neighbourhood. Results of analyses replicated with a measure of collective efficacy calculated with individuals who reported violent victimisation removed were similar. Cronbach's α for the collective efficacy scale was 0.77, consistent with previous reports.22,37 Neighbourhood collective efficacy was dichotomised above and below the median for all analyses because most propensity score approaches are based on binary exposure; extensions to categorical and continuous exposures have been developed but are not widely used.38–41 Results were similar with collective efficacy dichotomisation shifted by five neighbourhoods higher and lower. Individual perception of collective efficacy reported by each resident was adjusted in all analyses to distinguish the effect of the neighbourhood-level measure from that of individual perception of that characteristic.
Violent victimisation in the neighbourhood was assessed with the question `In the past 12 months, has anyone used violence, such as in a mugging, fight, or sexual assault, against you or any member of your household anywhere in your neighbourhood?'.22,42 This is the same violence question used in the Chicago research, with the modification to query the past 12 months rather than `ever' to capture violence in a recent timeframe.
Neighbourhood structural disadvantage measures were developed based on 2000 US census data. Indicators of racial/ethnic composition, poverty, residential stability and immigrant composition were included in a factor analysis to estimate composite measures of dimensions of neighbourhood structure that would not be highly correlated (see online appendix A for details). Four factors were indicated: concentrated poverty, black non-immigrant, immigrant and residential stability.
Analysis
Individual demographic and socioeconomic characteristics that were conceptually considered confounders (listed in measures) as well as individual perception of collective efficacy were controlled in all analyses. Multiple imputation and a missingness indicator approach were applied to variables where some respondents declined to answer, and results of analyses that applied these two approaches were compared; differences in results between the approaches were negligible, so the missingness indicator approach was used in all analyses for simplicity. Analyses were conducted with Stata V.12, and the psmatch2 package was used for propensity matching.
Propensities for living in low as compared with high collective efficacy neighbourhoods were estimated as a function of confounders using logistic regression; they were estimated first based on individual confounders and next based on individual and neighbourhood confounders. Observations were determined to be `off-support' if they had propensity values higher than the maximum or lower than the minimum in the other exposure group.
Marginal models and propensity-matched analyses were conducted with propensity restrictions to assure inferences were `on-support' first for individual confounders (model 1A) and then for individual and neighbourhood confounders (model 2A). Marginal models without restrictions were also implemented to examine the impact that extrapolation may have on study conclusions (models 1B and 2B).
Marginal models' estimation started with logistic generalised estimating equation regression models that account for potential clustering by neighbourhood and estimate population-averaged parameter estimates with robust SEs.43 These regression models were then used to estimate marginal associations on the additive scale comparable to the `average treatment effect' estimated in propensity-matched analyses (see below).35,36 We estimated the difference in the prevalence of violent victimisation if all residents had lived in neighbourhoods with high compared with low collective efficacy. We estimated the following parameters: θ (low) = EW{E[Y|A = low,W]}, θ(high) = EW{E[Y|A = high,W]}, where A is collective efficacy, W is the vector of confounders and Y is violent victimisation. We then compared low with high to estimate the marginal association between collective efficacy and violent victimisation: θ(low − high) = EW{E[Y|A = low,W] − E[Y|A = high,W]}. CIs were bootstrapped.44
In the propensity-matched analyses, one-to-one matching was employed because it has been shown to best minimise bias in simulations,45 and the calliper was set at 0.01 based on simulations that indicated an optimal width of 0.2 SD of the propensity score logit.46 Callipers 0.005 and 0.02 produced equivalent results. The `average treatment effect' (ie, marginal association in the whole population) was estimated, and CIs were bootstrapped.44 Results were not sensitive to the order of the observations.
RESULTS
The NYSES respondents are described in table 1. Violent victimisation was reported by 5.6% of the respondents. Neighbourhood collective efficacy had a median of 3.6 and a range of 2.7–4.0; a value of 3 means on average respondents `neither agree nor disagree' that the neighbourhood is cohesive and a value of 4 means on average respondents `somewhat agree' that the neighbourhood is cohesive. Collective efficacy was strongly correlated with neighbourhood concentrated disadvantage (r=−0.82) and had weak correlations with the other neighbourhood measures (black non-immigrant (r=−0.18), immigrant (r=−0.06) and residential stability (r=0.13)).
Table 1.
New York Social Environment Study characteristics and bivariable associations with community violent victimisation, NYC 2005 (n=4000)
| N (%) | Violent victimisation N (%) | p Value* | |
|---|---|---|---|
| Total | 4000 | 222 (5.6) | |
| Age (years) | <0.01 | ||
| 18–24 | 350 (8.8) | 31 (8.9) | |
| 25–34 | 685 (17.3) | 45 (6.6) | |
| 35–44 | 815 (20.6) | 43 (5.3) | |
| 45–54 | 808 (20.4) | 57 (7.1) | |
| 55–64 | 612 (15.5) | 22 (3.6) | |
| ≥65 | 690 (17.4) | 22 (3.2) | |
| Race/ethnicity | <0.01 | ||
| White | 1616 (41.6) | 61 (3.8) | |
| African–American | 1055 (27.1) | 58 (5.5) | |
| Asian | 164 (4.2) | 12 (7.4) | |
| Hispanic | 958 (24.6) | 82 (8.6) | |
| Other | 95 (2.4) | 5 (5.5) | |
| Sex | 0.94 | ||
| Male | 1880 (47.0) | 105 (5.6) | |
| Female | 2120 (53.0) | 117 (5.6) | |
| Marital status | 0.39 | ||
| Married | 1632 (41.4) | 83 (5.1) | |
| Divorced | 479 (12.2) | 25 (5.2) | |
| Separated | 208 (5.3) | 17 (8.2) | |
| Widowed | 354 (9.0) | 23 (6.5) | |
| Never married | 1270 (32.2) | 70 (5.5) | |
| Education | <0.01 | ||
| Less than high school | 508 (13.0) | 43 (8.5) | |
| High school/GED | 923 (23.5) | 64 (7.0) | |
| Some college | 879 (22.4) | 51 (5.8) | |
| College graduate | 883 (22.5) | 35 (4.0) | |
| Graduate work | 730 (18.6) | 24 (3.3) | |
| Birth place | 0.04 | ||
| NYC | 1810 (45.9) | 97 (5.4) | |
| Other US location | 731 (18.5) | 30 (4.1) | |
| Different country | 1406 (35.6) | 94 (6.7) | |
| Interview language | <0.01 | ||
| English | 3545 (88.6) | 179 (5.1) | |
| Spanish | 455 (11.4) | 43 (9.5) | |
| Income | <0.01 | ||
| ≤$40 000 | 1605 (40.1) | 26 (7.8) | |
| $40 001–$80 000 | 1093 (27.3) | 125 (4.6) | |
| >$80000 | 722 (18.1) | 50 (2.9) | |
| Unemployed | 0.77 | ||
| No | 3658 (92.0) | 203 (5.5) | |
| Yes | 321 (8.0) | 19 (5.9) | |
| Years lived in neighbourhood (years) | 0.06 | ||
| <8 | 1330 (33.4) | 90 (6.8) | |
| 8–21 | 1318 (33.1) | 65 (5.0) | |
| >21 | 1335 (33.5) | 66 (5.0) | |
| Neighbourhood collective efficacy | <0.01 | ||
| Q1 | 803 (20.1) | 80 (10.0) | |
| Q2 | 1023 (25.6) | 76 (7.5) | |
| Q3 | 1067 (26.7) | 38 (3.6) | |
| Q4 | 1107 (27.7) | 28 (2.5) |
χ2 p value comparing violent victimisation by covariate categories.
GED, general equivalency diploma; NYC, New York City.
Examination of propensity distributions based on individual confounders suggested no problems with positivity; only 0.5% of observations were `off-support' (table 2, see online appendix B for propensity plots). In contrast, the propensity distributions based on individual and neighbourhood confounders suggested substantial problems with positivity; 30.7% of observations were `off-support'. Those `off-support' included more people who were older, of white and African–American race/ethnicity, born in NYC, and higher income earners (table 3).
Table 2.
Distribution of propensities for living in a low collective efficacy neighbourhood by actual neighbourhood collective efficacy, New York Social Environment Study, New York City 2005 (n=4000)
| Minimum | 5% | 50% | 95% | Maximum | Observations `off-support'* | |
|---|---|---|---|---|---|---|
| Propensities based on individual confounders† | ||||||
| Collective efficacy | ||||||
| High | 0.059 | 0.134 | 0.326 | 0.729 | 0.929 | 0.5% |
| Low | 0.095 | 0.207 | 0.590 | 0.838 | 0.933 | |
| Propensities based on individual and neighbourhood confounders‡ | ||||||
| Collective efficacy | ||||||
| High | 0.000 | 0.000 | 0.081 | 0.788 | 0.954 | 30.7% |
| Low | 0.019 | 0.189 | 0.869 | 0.998 | 1.000 | |
An observation is `off-support' if its propensity value is higher than the maximum or lower than the minimum value of observations in the other exposure group.
Confounders used to estimate propensities were age, race/ethnicity, gender, marital status, place of birth, education, income, years lived in the current neighbourhood, interview language and individual perception of collective efficacy.
Confounders used to estimate propensities were age, race/ethnicity, gender, marital status, place of birth, education, income, years lived in the current neighbourhood, interview language, individual perception of collective efficacy, and four-factor analysis derived scores based on 2000 Census data: concentrated poverty, black non-immigrant, immigrant and residential stability.
Table 3.
Comparison of observations `on-support' and `off-support' in the analysis adjusted for individual and neighbourhood confounders,* New York Social Environment Study, NYC 2005 (n=4000)
| Off-support† N (%) | On-support N (%) | p Value | |
|---|---|---|---|
| Total | 1224 (30.7) | 2758 (69.3) | |
| Age (years) | 0.07 | ||
| 18–24 | 97 (8.0) | 252 (9.2) | |
| 25–34 | 185 (15.3) | 498 (18.2) | |
| 35–44 | 258 (21.3) | 555 (20.3) | |
| 45–54 | 241 (19.9) | 561 (20.6) | |
| 55–64 | 203 (16.7) | 408 (15.0) | |
| ≥65 | 229 (18.9) | 456 (16.7) | |
| Race | <0.01 | ||
| White | 502 (42.3) | 1111 (41.4) | |
| African–American | 421 (35.5) | 627 (23.4) | |
| Asian | 30 (2.5) | 132 (4.9) | |
| Hispanic | 213 (18.0) | 743 (27.7) | |
| Other | 20 (1.7) | 71 (2.7) | |
| Sex | 0.35 | ||
| Male | 562 (45.9) | 1311 (47.5) | |
| Female | 662 (54.1) | 1447 (52.5) | |
| Marital status | 0.02 | ||
| Married | 462 (38.2) | 1161 (42.8) | |
| Divorced | 149 (12.3) | 328 (12.1) | |
| Separated | 75 (6.2) | 132 (4.9) | |
| Widowed | 128 (10.6) | 225 (8.3) | |
| Never married | 395 (32.7) | 870 (32.0) | |
| Education | 0.07 | ||
| Less than high school | 136 (11.3) | 370 (13.7) | |
| High school/GED | 274 (22.7) | 645 (23.9) | |
| Some college | 271 (22.5) | 604 (22.4) | |
| College graduate | 276 (22.9) | 603 (22.4) | |
| Graduate work | 250 (20.7) | 476 (17.6) | |
| Birth place | <0.01 | ||
| NYC | 603 (49.7) | 1203 (44.3) | |
| Other US location | 272 (22.4) | 454 (16.7) | |
| Different country | 339 (27.9) | 1058 (39.0) | |
| Interview language | <0.01 | ||
| English | 1161 (94.9) | 2366 (85.8) | |
| Spanish | 63 (5.2) | 392 (14.2) | |
| Income | <0.01 | ||
| ≤$40 000 | 482 (39.4) | 1115 (40.4) | |
| $40 001–$80 000 | 348 (28.4) | 740 (26.8) | |
| <$80 000 | 248 (20.3) | 472 (17.1) | |
| Missing | 146 (11.9) | 431 (15.6) | |
| Unemployed | 0.96 | ||
| No | 1126 (92.0) | 2536 (92.0) | |
| Yes | 98 (8.0) | 222 (8.1) | |
| Time lived in neighbourhood (years) | 0.65 | ||
| <8 | 420 (34.4) | 903 (32.9) | |
| 8–21 | 396 (32.4) | 918 (33.5) | |
| >21 | 406 (33.2) | 922 (33.6) | |
| Neighbourhood collective efficacy | <0.01 | ||
| Q1 | 448 (36.6) | 352 (12.8) | |
| Q2 | 138 (11.3) | 880 (31.9) | |
| Q3 | 229 (18.7) | 833 (30.2) | |
| Q4 | 409 (33.4) | 693 (25.1) |
Confounders used to estimate propensities were age, race/ethnicity, gender, marital status, place of birth, education, income, years lived in the current neighbourhood, interview language and individual perception of collective efficacy, as well as neighbourhood-level structural disadvantage measures (concentrated poverty, black non-immigrant, immigrant and residential stability).
An observation is `off-support' if its propensity value is higher than the maximum or lower than the minimum value of observations in the other exposure group (see table 2 for the propensity distributions).
NYC, New York City.
Estimates of the relations between collective efficacy and violent victmisation based on marginal models and propensity-matched analyses are presented in table 4 (see online appendix C for logistic generalised estimating equation model results). Overall, marginal model and propensity-matched results were similar.
Table 4.
Marginal models and propensity-matched analysis of the relation between neighbourhood collective efficacy and community violent victimisation, New York Social Environment Study, New York City 2005 (n=4000)
| Marginal models* |
Propensity matched† |
|||||
|---|---|---|---|---|---|---|
| θ(low) | θ(high) | θ(low − high) | 95% CI‡ | ATE | 95% CI‡ | |
| Model 1A: `on-support' for individual§ confounders—adjusted for individual confounders, n=3962 | 7.5/100 | 3.5/100 | 4.0/100 | 2.6 to 5.8 | 4.0/100 | 2.1 to 5.9 |
| Model 1B: no restrictions—adjusted for individual§ confounders, n=3982 | 7.5/100 | 3.5/100 | 4.0/100 | 2.3 to 5.5 | ||
| Model 2A: `on-support' for individual§ arid neighbourhood¶ confounders—adjusted for individual and neighbourhood confounders, n=2758 | 6.7/100 | 3.6/100 | 3.1/100 | 1.2 to 5.2 | 2.4/100 | 0.2 to 4.5 |
| Model 2B: no restrictions—adjusted for individual§ and neighbourhood¶ confounders, n=3982 | 7.0/100 | 3.8/100 | 3.2/100 | 1.3 to 5.2 | ||
Marginal models estimate and compare the prevalence of violent victimisation under low and high collective efficacy conditions—specific parameters estimated are as follows: θ(low) = EW{E[Y|A = low,W]}, θ(high) = EW{E[Y|A = high,W]}, (θ{low − high} = EW{E[Y|A = low,W] − E[Y|A = high,W]}, where A is collective efficacy, W is the vector of confounders and Y is violent victimisation.
Propensity-matched ATE or `average treatment effect' using a 0.01 calliper.
Bias corrected bootstrapped CIs, units are per 100.
Individual confounders: age, race/ethnicity, gender, marital status, place of birth, education, income, years lived in the current neighbourhood, interview language and individual perception of collective efficacy.
Neighbourhood confounders are four-factor analysis derived scores based on 2000 Census data: concentrated poverty, black non-immigrant, immigrant and residential stability.
ATE, average treatment effect.
In the analyses that adjusted for individual confounders (models 1A and 1B), results were similar regardless of support restrictions; this was not surprising because <1% of data were `off-support'. In model 1A marginal models, under conditions of high collective efficacy (above the median), the estimated prevalence of community violent victimisation would have been 3.5/100 (estimated by (high)). Under conditions of low collective efficacy (below the median), the violent victimisation prevalence would have been 7.5/100 (estimated by (low)). This results in a difference of 4.0/100 in violent victimisation prevalence associated with low compared with high collective efficacy (estimated by (low − high), 95% CI 2.6 to 5.8). In the propensity-matched analysis, the comparable `average treatment effect' was 4.0/100 (95% CI 2.1 to 5.9).
In the analyses that adjusted for individual and neighbourhood confounders (models 2A and 2B), results were also similar regardless of support restrictions; this was surprising given that 30% of data were off-support. In model 2A that includes data `on-support' for individual and neighbourhood confounders, the marginal models estimated a difference of 3.1/100 in violent victimisation prevalence for low compared with high collective efficacy (95% CI 1.2 to 5.2). In model 2B that did not have support restriction, the difference in violent victimisation associated with collective efficacy was almost equivalent ((low − high): 3.2/100, 95% CI to 1.3, 5.2). In the propensity-matched analysis, the comparable `average treatment effect' was slightly smaller than the marginal model estimates, but the CI still excluded the null value of 0 (`average treatment effect': 2.4/100, 95% CI 0.2 to 4.5).
DISCUSSION
In an analysis of neighbourhood collective efficacy and violence in NYC, we found that under conditions of high collective efficacy, there was about half the prevalence of violence compared with low collective efficacy in analyses that controlled for individual confounders. The strength of the association was reduced in analyses that additionally controlled for neighbourhood structural disadvantage, but associations were still substantial and CIs excluded the null. Notably, analyses were adjusted for individual perception of collective efficacy (in addition to many other confounders) and thus represent conservative estimates. The analysis populations for the marginal models and propensity-matched analyses were restricted using the propensity score to assure all inferences were `on-support'. The magnitudes of the associations estimated in marginal models were similar to propensity-matched analyses; this suggests that marginal models with propensity restriction were a valid alternative for this analysis. Furthermore, the marginal models accounted for clustering by neighbourhood and thus provide more appropriate inference than matched analyses that did not account for clustered data. Surprisingly, models without the support restriction estimated associations between neighbourhood collective efficacy and violence that were similar to the restricted analyses; we would not expect that this would typically be the case and caution against any conclusion that problems of positivity can be ignored.
While the positivity restriction assures inferences are `on-support', it also changes the parameter that is estimated. Unrestricted marginal analyses estimate the association in the total study population, while analyses of those `on-support' are only applicable to the population subset for whom there was experimentation in the exposure. Nonetheless, this parameter is informative because it tells us that for population subgroups (defined by the confounders in the model), who experienced high and low levels of collective efficacy, there is an association of meaningful magnitude.
The assumptions necessary for causal interpretation of associations in observational research generally and observational neighbourhood research specifically have been well elaborated elsewhere47,48; we address each assumption below. Temporal ordering: given the cross-sectional design of our study, we cannot establish temporal ordering between the exposure and outcome. For a causal interpretation, we must assume that collective efficacy precedes violence; this is a reasonable assumption, but the reverse is also likely true to some extent. Longitudinal consideration of these relations will be necessary to establish temporality. Ignorability: although we controlled many confounders, notably individual perception of collective efficacy, the assumption of no unmeasured confounding cannot be assessed empirically and can only ever be approximated with observational data. Neighbourhood-level stability assumption: the assumption that exposures in one neighbourhood cannot affect the potential outcomes of individuals in another is not unreasonable because collective efficacy in one area does not seem likely to alter the effects of collective efficacy on potential outcomes in another area.
There are several limitations to this study. The NYSES had a cooperation percentage of 54%, which is consistent with many other recent telephone-based studies49 but does raise concern about potential differences between respondents and non-respondents. The neighbourhood collective efficacy measure captures aggregate perception of the potential for collective efficacy.22 Perceptions are affected by individual characteristics including experience of violence in the neighbourhood. It is reassuring in that regard that our findings were similar when we used a measure of collective efficacy calculated with individuals who reported violent victimisation removed.
Among several strengths, the NYSES includes a large population-based sample. Analyses were adjusted for individual perception of collective efficacy, allowing documentation of an association of neighbourhood collective efficacy with violent victimisation that was independent of individual perceptions; this approach provides a conservative estimate. In addition, we found associations of similar magnitude using both marginal models and propensity-matched analyses and used restriction to assure analyses did not rely on extrapolation.
The findings indicate several directions for future methodological investigation. A general assessment of the performance of marginal models restricted by the propensity score versus propensity matching would be informative. It is easier to account for clustering in marginal models (and other regression-based approaches) than in propensity matching, and marginal models thus offer strengths for multilevel studies. Furthermore, marginal models would accommodate complex sampling designs. The surprising finding here that non-positivity did not produce substantial bias suggests that it would be informative to examine the conditions under which non-positivity does and does not lead to bias more generally.
Overall, our findings in NYC are consistent with research in Chicago and major cities in other high-income countries (Sweden and Australia) that have found relations between collective efficacy and violence.22–25,28,29 Hence, our results contribute to a growing body of evidence that suggests collective efficacy may be a generalisable social mechanism that serves a protective function against violent victimisation. Our work also illustrates the utility of propensity score-related methods to assess the question of positivity and assure that inferences are `on-support'.
What is already known on this subject.
In neighbourhood studies, strong correlations of neighbourhood exposures with individual and neighbourhood confounders can create problems with non-positivity (ie, data that are `off-support'), defined as the situation in which some population subgroups experience only one level of the exposure. Analyses with data that are `off-support' rely on extrapolation to estimate associations. Propensity restriction to assure inferences are `on-support', combined with analysis approaches that account for clustering by neighbourhood, offer an alternative to propensity stratification or matching for multilevel studies.
What this study adds.
We assessed the potential utility of propensity restriction to examine the relation between collective efficacy and violence in NYC where it has not yet been assessed. Propensity-restricted marginal model and propensity-matched parameters were similar in magnitude, suggesting that in this analysis, the marginal modelling with propensity restriction was a viable alternative to propensity matching. We found strong associations between collective efficacy and violence in analyses that controlled for individual confounders; the strength of the association was reduced but still substantial after additional control for neigh-bourhood structural disadvantage. The results contribute to a growing body of evidence that suggests collective efficacy may shape violence, and illustrate how the utility of propensity score-related methods to assess the question of positivity and assure that inferences are `on-support'.
Acknowledgments
Funding Support for this work was provided in part by the National Institute on Drug Abuse at the National Institutes of Health (DA 017642 and DA 06534).
Footnotes
▶Additional appendices are published online only. To view these files please visit the journal online (http://dx.doi.org/10.1136/jech-2012-201317).
Contributors JA collaborated on the design and implementation of the study, designed and implemented the analysis, conducted the literature review and wrote the manuscript. MC collaborated on the analysis plan, drafted sections of the manuscript and substantially edited all sections of the manuscript. SAL collaborated on the literature review, provided input on the analysis plan and substantially edited all sections of the manuscript. KJT obtained study funding and provided input on the analysis plan and on the manuscript. DV obtained study funding and provided input on the analysis plan and on the manuscript. SG obtained study funding, collaborated on the design and implementation of the study, and substantially edited all sections of the manuscript. All authors have approved the final version of the manuscript.
Competing interests None.
Ethics approval The study protocol was approved by the institutional review boards of the New York Academy of Medicine, the University of Michigan and the University of California, Berkeley.
Provenance and peer review Not commissioned; externally peer reviewed.
REFERENCES
- 1.Greenland S. Ecologic versus individual-level sources of bias in ecologic estimates of contextual health effects. Int J Epidemiol. 2001;30:1343–50. doi: 10.1093/ije/30.6.1343. [DOI] [PubMed] [Google Scholar]
- 2.Diez Roux AV. Multilevel analysis in public health research. Annu Rev Public Health. 2000;21:171–92. doi: 10.1146/annurev.publhealth.21.1.171. [DOI] [PubMed] [Google Scholar]
- 3.Oakes JM. The (mis)estimation of neighborhood effects: causal inference for a practicable social epidemiology. Soc Sci Med. 2004;58:1929–52. doi: 10.1016/j.socscimed.2003.08.004. [DOI] [PubMed] [Google Scholar]
- 4.Diez Roux AV. Estimating neighborhood health effects: the challenges of causal inference in a complex world. Soc Sci Med. 2004;58:1953–60. doi: 10.1016/S0277-9536(03)00414-3. [DOI] [PubMed] [Google Scholar]
- 5.Messer LC, Oakes JM, Mason S. Effects of socioeconomic and racial residential segregation on preterm birth: a cautionary tale of structural confounding. Am J Epidemiol. 2010;171:664–73. doi: 10.1093/aje/kwp435. [DOI] [PubMed] [Google Scholar]
- 6.Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70:41–55. [Google Scholar]
- 7.Zanutto E, Lu B, Hornik R. Using propensity score subclassification for multiple treatment doses to evaluate a national antidrug media campaign. J Educ Behav Stat. 2005;30:59–73. [Google Scholar]
- 8.Dehejia RH, Wahba S. Causal effects in nonexperimental studies: reevaluating the evaluation of training programs. J Am Stat Assoc. 1999;94:1053–62. [Google Scholar]
- 9.Leal C, Bean K, Thomas F, et al. Are associations between neighborhood socioeconomic characteristics and body mass index or waist circumference based on model extrapolations? Epidemiology. 2011;22:694–703. doi: 10.1097/EDE.0b013e3182257784. [DOI] [PubMed] [Google Scholar]
- 10.Diez Roux AV, Borrell LN, Haan M, et al. Neighbourhood environments and mortality in an elderly cohort: results from the cardiovascular health study. J Epidemiol Community Health. 2004;58:917–23. doi: 10.1136/jech.2003.019596. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Johnson PJ, Oakes JM, Anderton DL. Neighborhood poverty and American Indian infant death: are the effects identifiable? Ann Epidemiol. 2008;18:552–9. doi: 10.1016/j.annepidem.2008.02.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Hearst MO, Oakes JM, Johnson PJ. The effect of racial residential segregation on black infant mortality. Am J Epidemiol. 2008;168:1247–54. doi: 10.1093/aje/kwn291. [DOI] [PubMed] [Google Scholar]
- 13.Hong G, Raudenbush SW. Evaluating kindergarten retention policy: a case study of causal inference for multilevel observational data. J Am Stat Assoc. 2006;101:901–10. [Google Scholar]
- 14.Hansen BB, Bowers J. Covariate balance in simple, stratified and clustered comparative studies. Stat Sci. 2008;23:219–36. [Google Scholar]
- 15.Preisser JS, Young ML, Zaccaro DJ, et al. An integrated population-averaged approach to the design, analysis and sample size determination of cluster-unit trials. Stat Med. 2003;22:1235–54. doi: 10.1002/sim.1379. [DOI] [PubMed] [Google Scholar]
- 16.Wilson W. The Truly Disadvantaged: The Inner City, the Underclass, and Public Policy. University of Chicago Press; Chicago: 1987. [Google Scholar]
- 17.Shaw CR, McKay HD. Juvenile Delinquency and Urban Areas. The University of Chicago Press; Chicago: 1942. [Google Scholar]
- 18.Kornhauser R. Social Sources of Delinquency. University of Chicago Press; Chicago: 1978. [Google Scholar]
- 19.Schulz AJ, Zenk SN, Israel BA, et al. Do neighborhood economic characteristics, racial composition, and residential stability predict perceptions of stress associated with the physical and social environment? Findings from a multilevel analysis in Detroit. J Urban Health. 2008;85:642–61. doi: 10.1007/s11524-008-9288-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Sampson RJ. The neighborhood context of well-being. Perspect Biol Med. 2003;46:S53–64. [PubMed] [Google Scholar]
- 21.Massey DS. The age of extremes: concentrated affluence and poverty in the twenty-first century. Demography. 1996;33:395–412. discussion 3–6. [PubMed] [Google Scholar]
- 22.Sampson RJ, Raudenbush SW, Earls F. Neighborhoods and violent crime: a multilevel study of collective efficacy. Science. 1997;277:918–24. doi: 10.1126/science.277.5328.918. [DOI] [PubMed] [Google Scholar]
- 23.Molnar BE, Miller MJ, Azrael D, et al. Neighborhood predictors of concealed firearm carrying among children and adolescents: results from the project on human development in Chicago neighborhoods. Arch Pediatr Adolesc Med. 2004;158:657–64. doi: 10.1001/archpedi.158.7.657. [DOI] [PubMed] [Google Scholar]
- 24.Jain S, Buka SL, Subramanian SV, et al. Neighborhood predictors of dating violence victimization and perpetration in young adulthood: a multilevel study. Am J Public Health. 2010;100:1737–44. doi: 10.2105/AJPH.2009.169730. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Maimon D, Browning CR, Brooks-Gunn J. Collective efficacy, family attachment, and urban adolescent suicide attempts. J Health Soc Behav. 2010;51:307–24. doi: 10.1177/0022146510377878. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Morenoff JD, Sampson RJ, Raudenbush SW. Neighborhood inequality, collective efficacy, and the spatial dynamics of urban violence. Criminology. 2001;39:517–59. [Google Scholar]
- 27.Sampson RJ, Morenoff JD, Earls F. Beyond social capital: spatial dynamics of collective efficacy for children. Am Sociol Rev. 1999;64:633–60. [Google Scholar]
- 28.Sampson RJ, Wikstrom PO. The social order of violence in Chicago and Stockholm neighborhoods: a comparative inquiry. In: Shapiro I, Kalyvas S, Masoud T, editors. Order, Conflict, and Violence. Cambridge University Press; New York: 2007. [Google Scholar]
- 29.Mazerolle L, Wickes R, McBroom J. Community variations in violence: the role of social ties and collective efficacy in comparative context. J Res Crime Delinq. 2010;47:3–30. [Google Scholar]
- 30.Browning CR. The span of collective efficacy: extending social disorganization theory to partner violence. J Marriage Fam. 2002;64:833–50. [Google Scholar]
- 31.Browning CR, Feinberg SL, Dietz RD. The paradox of social organization: networks, collective efficacy, and violent crime in urban neighborhoods. Soc Forces. 2004;83:503–34. [Google Scholar]
- 32.Chaix B, Leal C, Evans D. Neighborhood-level confounding in epidemiologic studies: unavoidable challenges, uncertain solutions. Epidemiology. 2010;21:124–7. doi: 10.1097/EDE.0b013e3181c04e70. [DOI] [PubMed] [Google Scholar]
- 33.Ahern J, Galea S, Hubbard A, et al. “Culture of drinking” and individual problems with alcohol use. Am J Epidemiol. 2008;167:1041–9. doi: 10.1093/aje/kwn022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Ahern J, Galea S, Hubbard A, et al. Neighborhood smoking norms modify the relation between collective efficacy and smoking behavior. Drug Alcohol Depend. 2009;100:138–45. doi: 10.1016/j.drugalcdep.2008.09.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Ahern J, Hubbard A, Galea S. Estimating the effects of potential public health interventions on population disease burden: a step-by-step illustration of causal inference methods. Am J Epidemiol. 2009;169:1140–7. doi: 10.1093/aje/kwp015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Ahern J, Galea S. Collective efficacy and major depression in urban neighborhoods. Am J Epidemiol. 2011;173:1453–62. doi: 10.1093/aje/kwr030. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Echeverria SE, Diez-Roux AV, Link BG. Reliability of self-reported neighborhood characteristics. J Urban Health. 2004;81:682–701. doi: 10.1093/jurban/jth151. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Imai K, van Dyk DA. Causal inference with general treatment regimes: generalizing the propensity score. J Am Stat Assoc. 2004;99:854–66. [Google Scholar]
- 39.Imbens GW. The role of the propensity score in estimating dose-response functions. Biometrika. 2000;87:706–10. [Google Scholar]
- 40.Lu B, Zanutto E, Hornik R, et al. Matching with doses in an observational study of a media campaign against drug abuse. J Am Stat Assoc. 2001;96:1245–53. doi: 10.1198/016214501753381896. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Hirano K, Imbens GW. The propensity score with continuous treatments. In: Gelman A, Meng XL, editors. Applied Bayesian Modeling and Causal Inference from Incomplete Data Perspectives. John Wiley & Sons, Ltd; Hoboken, NJ: 2004. [Google Scholar]
- 42.Krug EG, Dahlberg LL, Mercy JA, et al., editors. World Report on Violence and Health. World Health Organization; Geneva: 2002. [Google Scholar]
- 43.Zeger SL, Liang KY, Albert PS. Models for longitudinal data: a generalized estimating equation approach. Biometrics. 1988;44:1049–60. [PubMed] [Google Scholar]
- 44.Efron B, Tibshirani R. Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Stat Sci. 1986;1:54–77. [Google Scholar]
- 45.Austin PC. Statistical criteria for selecting the optimal number of untreated subjects matched to each treated subject when using many-to-one matching on the propensity score. Am J Epidemiol. 2010;172:1092–7. doi: 10.1093/aje/kwq224. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Austin PC. Some methods of propensity-score matching had superior performance to others: results of an empirical investigation and Monte Carlo simulations. Biom J. 2009;51:171–84. doi: 10.1002/bimj.200810488. [DOI] [PubMed] [Google Scholar]
- 47.van der Laan MJ, Robins JM. Unified Methods for Censored Longitudinal Data and Causality. Springer Verlag; New York: 2003. [Google Scholar]
- 48.Vanderweele TJ. Ignorability and stability assumptions in neighborhood effects research. Stat Med. 2008;27:1934–43. doi: 10.1002/sim.3139. [DOI] [PubMed] [Google Scholar]
- 49.Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol. 2007;17:643–53. doi: 10.1016/j.annepidem.2007.03.013. [DOI] [PubMed] [Google Scholar]