Do changes in neighborhood social context mediate the effects of the Moving to Opportunity experiment on adolescent mental health?

Abstract

This study investigated whether changes in neighborhood context induced by neighborhood relocation mediated the impact of the Moving to Opportunity (MTO) housing voucher experiment on adolescent mental health. Mediators included participant-reported neighborhood safety, social control, disorder, and externally-collected neighborhood collective efficacy. For treatment group members, improvement in neighborhood disorder and drug activity partially explained MTO’s beneficial effects on girls’ distress. Improvement in neighborhood disorder, violent victimization, and informal social control helped counteract MTO’s adverse effects on boys’ behavioral problems, but not distress. Housing mobility policy targeting neighborhood improvements may improve mental health for adolescent girls, and mitigate harmful effects for boys.

INTRODUCTION

A growing body of evidence is documenting how neighborhood context influences the health of residents (Kim, 2008, Mair et al., 2008, Truong and Ma, 2006, Leventhal et al., 2009, Wen et al., 2006, Diez Roux et al., 1997, Diez Roux et al., 2003), including small literature documenting the negative long-term health effects of poor neighborhood conditions over the life course (Jivraj et al., 2019). However, research remains hampered by observational designs. A recent synthesis of neighborhood effects research documented that the overwhelming majority of the 1170 empirical neighborhood effect studies used observational data (Oakes et al., 2015). The authors noted that many of these papers used multilevel modeling, but rarely did they explicitly address the assumptions underlying the application of causal multilevel methods. More recently researchers are applying causal methods to assess neighborhood effects, such as inverse probability weighting (IPW) and propensity score analysis, but results from these studies have been conflicting (Oakes et al., 2015). To date, the relative contribution of neighborhoods’ impacts on health, and the extent to which methodological considerations play a role, are unclear (Oakes et al., 2015).

Experimental research designs provide many advantages for identifying how neighborhood changes influence health, since they nearly eliminate confounding whether measured or not (i.e., ensuring exchangeability) and they assign a specific treatment (i.e., ensuring consistency), meeting two assumptions necessary for causal inference (Sobel, 2006). Despite these advantages, some findings from experimental studies of neighborhood relocation, like the Moving to Opportunity for Fair Housing Demonstration Project (MTO) (Goering and Feins, 2003), have not been straightforward (Oakes et al., 2015). The MTO study randomly assigned low-income families residing in public housing to receive a rental housing voucher to move to private apartments in lower poverty neighborhoods in an effort to improve social and economic outcomes (Orr et al., 2003). Health was not anticipated as an outcome in the original MTO logic model, but health, particularly mental health, emerged as a potentially important outcome based on early evidence (Leventhal and Brooks-Gunn, 2003), which was borne out in several evaluations for the single mother household heads and their daughters. However, one puzzling finding has been the opposite effects of the MTO policy on the mental health of girls and boys at the interim survey, 4–7 years after random assignment. Compared to the control group, the older cohort of treatment group girls experienced decreased psychological distress, while the older cohort of boys experienced increased psychological distress and behavior problems (Orr et al., 2003, Kling et al., 2007, Author et al., 2012b, Author et al., 2012a). The beneficial treatment effects on girls’ mental health persisted at the final survey, 10–15 years after random assignment, but there were no significant effects for boys’ mental health (Sanbonmatsu et al., 2011). Although a few qualitative studies have delved into why girls benefitted and boys did not (Popkin et al., 2008, Clampet-Lundquist et al., 2011), few studies have empirically tested what mediates these unexpected opposite effects on mental health. One study found comorbid substance use to be a mediator of boys’ behavior problems, (Author et al., 2017) but no studies have explored neighborhood social context as a mediator. Understanding the mechanisms by which the MTO treatment operated can increase the number of intervention points to maximize the benefit, and minimize the harm, of this expensive and expansive housing policy investment.

The primary aim of this study was to investigate whether changes in neighborhood context induced by neighborhood relocation mediated the impact of MTO on adolescent mental health. Many aspects of neighborhood context have been linked with mental health, and therefore could act as potential mediating mechanisms. For example, neighborhood disorder, exemplified by the presence of graffiti, loitering, and public drinking, as well as violence and violent victimization have been linked to depression, behavior problems, and poor mental health outcomes (Browning et al., 2013, Strohschein and Matthew, 2015, Schulz et al., 2006, Aneshensel and Sucoff, 1996, Foster and Brooks-Gunn, 2009). The prevailing theory is that poor neighborhood conditions, such as exposure to neighborhood disorder, violence, and victimization, cause stress that can lead to mental health problems (Browning et al., 2013, Pearlin, 1989, Aneshensel, 1992). Positive aspects of neighborhoods, like collective efficacy, on the other hand, may mitigate the impacts of stressors (Foster and Brooks-Gunn, 2009), and therefore be associated with fewer mental health problems (Browning et al., 2013, Aneshensel and Sucoff, 1996, Ahern and Galea, 2011).

A secondary aim of this study is to examine whether mediation pathways are gender-specific. Gender modification of neighborhood associations on health outcomes is common (Kling et al., 2005, Ramirez-Valles et al., 2002, Chuang and Chuang, 2008, Berke et al., 2007), although rarely unpacked with mediation analyses. The environment for low-income female adolescents may be markedly different than that for their male counterparts (Rawlings, 2015), in particular with respect to the role that neighborhood disorder, lack of safety, and concerns around victimization play in perpetuating mental health issues for girls and women (Popkin et al., 2002, Popkin et al., 2008, Browning et al., 2013, Schulz et al., 2006).

Even when boys and girls experience the same objective environment, it may impact mental health differentially by gender because boys and girls are socially conditioned to navigate neighborhoods in very different ways. For example, in order to successfully navigate poor neighborhoods, boys may adopt a street posture, which may interfere with how they assimilate into low poverty neighborhoods (Anderson, 1990). Moreover, boys are more likely to hang out outdoors in neighborhoods, inadvertently exposing boys who moved in MTO to both informal and formal supervision, potentially increasing their likelihood of getting in trouble (Clampet-Lundquist et al., 2011). This difficulty assimilating into low poverty neighborhoods may negatively impact boys’ mental health. Girls, on the other hand, may be more adept at navigating these new social contexts, to the extent that they are better able to leverage dominant cultural capital (i.e., the expectations for behavior among white, middle class society) that may be essential for advancing socially, academically, and economically in low poverty neighborhoods (Carter, 2003).

Given this literature, we hypothesized that neighborhood social context, including disorder (physical and social), safety, exposure to violence, informal social control, and collective efficacy, would mediate MTO effects on adolescent mental health. Secondarily, we hypothesized that neighborhood social context would mediate the MTO voucher treatment on mental health differently for girls than for boys. We leverage the MTO housing voucher experiment to test these hypotheses.

METHODS

MTO was a randomized controlled trial (RCT) implemented by the U.S. Department of Housing and Urban Development (U.S. Department of Housing and Urban Development, 1996) in Boston, Baltimore, Chicago, Los Angeles, and New York. Low income families volunteered to participate in the MTO experiment, and were eligible if they had children under age 18, qualified for rental assistance, and lived in public housing or project-based assisted housing in high poverty neighborhoods; 4610 of 5301 volunteer families were eligible (Orr et al., 2003). This federally-funded housing experiment also partnered with local housing authorities and nonprofit housing counseling organizations to execute the study (Orr et al., 2003).

Treatment assignment.

Families were randomized using specialized software to one of three treatment groups in 1994–1998: 1) the “low poverty” group received a Section 8 housing voucher to subsidize renting a private market apartment in neighborhoods with <10% of households in the census tract living in poverty, and housing counseling to assist in relocation; 2) the “Section 8” treatment group received a traditional Section 8 housing voucher with no constraints on neighborhood poverty and no housing counseling; and 3) the control group could remain in public housing but received no further assistance (Goering et al., 1999). The low poverty neighborhood group had to remain in a low poverty neighborhood for one year, at which point families could move to another apartment using their voucher, regardless of its tract poverty level. Treatment effects on mental health were homogenous within gender, so we combined the two voucher groups for parsimony (Author et al., 2012b, Author et al., 2012a).

Assessments.

Household heads and up to two randomly-selected children completed surveys at baseline (1994–1998), the 4–7 year interim follow-up (2001–2002), and up to three randomly selected children completed the final follow-up survey (2008–2010) using computer-assisted interviewing technology. We focus on adolescents 12–19 years old at the interim follow-up who were randomized through 12/31/97 in the MTO Tier 1 Restricted Access Data, because MTO effects on mental health at interim showed opposite effects by gender and were consistent across multiple mental health outcomes (N=2829, effective response rate = 89.3%) (Orr et al., 2003). The final survey showed significant effects for some mental health outcomes for girls, but no significant effects for boys’ mental health (Sanbonmatsu et al., 2011). Adults provided informed written consent for themselves and their children (Orr et al., 2003, Goering et al., 1999).

Mental health outcomes.

Past-month psychological distress was measured at interim (2001–2002) using the Kessler K-6 scale (K6), a broad-gauged dimensional screening tool for mental illness (Kessler et al., 2002). The K6 includes 5-item Likert responses (none of the time, a little, some, most, or all of the time during the past 30 days) to 6 items: so depressed nothing could cheer you up; nervous; restless or fidgety; hopeless; everything was an effort; worthless. Externalizing behaviors were assessed using an 11-item scale adapted from the Behavior Problems Index (BPI) (Zill, 1990); responses to items such as “I lie or cheat” and “I have a hot temper” ranged from 0 (not true) to 2 (often true). We scored the K6 and BPI using two-parameter binary Item Response Theory (IRT) methods (Kessler et al., 2002, Author et al., 2012b, Author et al., 2012a).

Neighborhood Mediators.

Neighborhood mediators were derived from 2 sources: participant reports in the MTO interim survey, and externally measured neighborhood surveys.

MTO-Reported Neighborhood Quality.

The 2002 MTO Interim Survey was administered to both household heads and adolescents. We measured household head-reported neighborhood environment using neighborhood physical and social disorder, social control, and safety; most of these measures are validated measures taken from prior neighborhood surveys (Sampson et al., 1997, Kawachi, 1999, Galster and Killen, 1995, de Souza Briggs, 2006, Author et al., 2009). We also tested two individual items: violent crime victimization, and the presence of illegal drugs. Neighborhood disorder included public drinking, loitering, police not coming when called, trash/litter, graffiti, and abandoned buildings (not at all, somewhat, or a big problem; Cronbach’s alpha = .87). Subscales measured social disorder (the first three items; alpha = .75) and physical disorder (the last three items; alpha = .80). Informal social control included how likely (1=very unlikely to 5=very likely) it is that neighbors would intervene if 1) children were skipping school/hanging out on street corner, or 2) children were spray-painting graffiti on a local building (alpha = .81). Neighborhood safety was assessed with two items including how safe household heads felt in their neighborhoods at night and during the day (1=very unsafe to 4=very safe) (alpha = .86). Violent crime victimization captured whether any household member reported being a victim of any (versus none) of the following in the past 6 months: purse/wallet/jewelry snatched; threatened with knife or gun; beaten or assaulted; stabbed or shot; break-in. Finally, household heads reported whether they saw illegal drug use or sales in their neighborhood in the past 30 days, classified as once a week or more versus never. We linked these household-head-reported measures to their child’s outcomes.

Adolescent-reported neighborhood environment included two items tapping neighborhood social disorder: youth heard gunshots at least once a week in the past month or saw illegal drug use or sales. Reliability was low when combined in a scale (alpha=.52), so we modeled these items separately. Youth also reported exposure to violence in the past 12 months as seeing one or more of the following (versus none): someone shoot/stab another person; someone pull a knife/gun on child; someone cut, shot, stabbed, or jumped the child.

Externally-Measured Neighborhood Quality.

We created neighborhood-level measures of social context using independent community surveys fielded for all neighborhoods in 3 of the MTO cities. We merged these neighborhood-level measures with the MTO participants’ census tract of residence in 1997, the first post-baseline residence documented for the entire MTO sample. Community surveys included the Boston Neighborhood Survey (BNS 2006), the Project on Human Development in Chicago Neighborhoods (PHDCN 1994–1995), and the New York Social Environment Survey (NYSES 2005). These are population-based surveys designed to characterize neighborhood context throughout the entire city, details of which are documented elsewhere (Azrael et al., 2009, Sampson et al., 1997, Ahern et al., 2008). Similar community surveys for Baltimore and Los Angeles were either unavailable, or did not match the MTO residential neighborhoods, so these sites were excluded from these analyses. Using these surveys, we derived neighborhood-level measures of collective efficacy and its subscales, informal social control and social cohesion. Collective efficacy is a 10-item scale assessing 1) the ability of residents to regulate behavior, maintain public order, and organize to better the community (informal social control, 5 items; e.g., neighbors would do something about a child skipping school, spraying graffiti), and 2) the mutual trust and reciprocity among residents (social cohesion, 5 items; e.g., neighbors can be trusted, are willing to help each other).(Sampson et al., 1997) Higher scores indicate higher collective efficacy.

We constructed the external neighborhood-level measures using 3-level (items nested within individuals within neighborhoods) hierarchical linear models (HLM) implemented via HLM 7 software (Raudenbush and Bryk, 2002, Raudenbush and Sampson, 1999, Mujahid et al., 2007). We attempted to obtain external survey-based measures close to the 1997 to 2002 range of the MTO address data. Although this was not always possible given data constraints, research suggests that neighborhoods change slowly, thus assuming stability in neighborhood social context is valid, both in the short- (Author et al., 2014, Sampson, 2012, Jivraj, 2012) and long-term (Dorling et al., 2007). Of note, the community surveys only cover the city proper, so we have missing data on these measures for subjects who moved to suburban areas. We detail below sensitivity analyses using multiple imputation to adjust for this potential bias.

Analytic Approach.

We first estimated intent to treat (ITT) models for the effect of MTO treatment on each proposed neighborhood mediator, using linear or logistic regression as appropriate. We confirmed that treatment-mediator effects were statistically homogenous by gender. Second, we estimated the total effect of the MTO treatment on youth psychological distress and BPI, stratified by gender, given gender heterogeneity in treatment effects on mental health (Author et al., 2012a, Author et al., 2012b). Third, we used effect decomposition to assess mediation, stratified by gender, testing natural direct and indirect effects (Pearl, 2001) of the MTO treatment on mental health. We assessed mediation for the total effects of MTO on boys’ and girls’ psychological distress, and boys’ behavior problems; we did not test mediation for girls’ behavior problems because treatment effects were near zero (Author et al., 2012a, Author et al., 2012b). Mediation was evidenced by a statistically significant indirect effect.

Mediation methods.

Effect decomposition separates the total effect of MTO on mental health into the direct effect, i.e., the effect not operating through the tested mediator, and the indirect effect, i.e., the effect operating through the tested mediator. We apply two methods of treatment effect decomposition, with differing sets of assumptions, strengths, and limitations: 1) the product method (Baron and Kenny, 1986, Valeri and VanderWeele, 2013), and 2) Inverse Odds Weighting (IOW) (Tchetgen Tchetgen, 2013).

The product method assumes no unmeasured confounding of the exposure-outcome or the mediator-outcome association. By design, RCTs eliminate the first source of confounding, and the second source can be minimized by the inclusion of covariates that are significantly related to the outcome (See Table 1) (Valeri and VanderWeele, 2013). Moreover, it assumes no interactions between the treatment and mediator on the outcome. Finally, the product method is invalid for decomposing the effect of nonlinear mediators or outcomes, limiting its application, for example, to common, dichotomous outcomes (Valeri and VanderWeele, 2013, Author et al., 2015).

Table 1.

MTO Youth, Baseline Variables, Overall and by Treatment Group

Construct	Overall	Treatment	Control	pa
Total in interim survey in 2002, N	2829	1950	879
Family characteristics
Health, %
Household member had disability, health, or developmental problem	43.0	43.6	41.5
Household member had a disability	17.2	17.8	15.8
Site, %
Baltimore	15.5	16.0	14.2
Boston	18.9	18.1	20.7
Chicago	22.4	23.3	20.4
Los Angeles	18.6	17.5	21.2
New York	24.6	25.1	23.5
Youth characteristics
Age, mean, y	9.9	10.0	9.9
Gender, %
Male	49.9	49.5	51.0
Female	50.1	50.5	49.0
Race/ethnicity, %
Black	62.8	63.2	62.1
Hispanic ethnicity, any race	30.0	30.3	29.5
White	1.1	1.0	1.2
Other race	2.2	2.4	1.9
Missing race	3.8	3.2	5.3
Gifted, %
Special class for gifted students or did advanced work	15.4	14.7	16.8
Developmental problems, %
Special school, class, or help for learning problem in past 2 y	16.6	16.7	16.3
Special school, class, or help for behavioral or emotional problems in past 2 y	7.7	8.7	5.3	*
School asked to talk about problems child having with schoolwork or behavior in past 2 y	26.3	26.7	25.4
Expelled, %
Suspended or expelled from school in past 2 y	10.4	11.5	7.7	*
Household head characteristics
Family structure, %
Never married	55.9	55.2	57.5
No teens in household	47.3	46.1	50.1
Socioeconomic status, %
Employed	25.8	26.1	25.3
On AFDC (welfare)	76.0	75.5	76.9
Education, %
Less than high school	47.1	47.2	46.7
High school diploma	36.2	36.6	35.3
GED	16.7	16.1	17.9
In school	13.9	14.4	12.6
Neighborhood/mobility variables, %
Streets near home very unsafe at night	49.1	49.0	49.3
Lived in neighborhood ≥ 5 y	65.7	65.8	65.5
Moved > 3 times in past 5 y	8.0	7.4	9.4

All variables range between 0 and 1 except baseline age (5–16). The analysis was weighted for varying intervention random assignment ratios across time and for attrition. All tests were adjusted for clustering at the family level. Missing baseline covariate data were imputed to site-specific means (<5 missing) or modeled with missing indicators. All covariates measured at baseline, except site and gender, were included in mediation models.

Abbreviations: AFDC, Aid to Families with Dependent Children; GED, General Equivalency Diploma.

^aP value for test of treatment group differences calculated from Wald X² tests outputted from logistic regression for dichotomous baseline characteristics and multinomial logistic regression for categorical characteristics. F tests were used with linear regression for continuous variables.

The null hypothesis was that the treatment and control group proportions or means did not differ.

^*P < .05

The Inverse Odds Weighting (IOW) method is a semiparametric alternative to the product method. IOW condenses information on the relationship between the treatment and mediators, conditioned on covariates, into a weight, which isolates the direct effect by deactivating all indirect pathways. This is achieved by down-weighting people with a strong treatment-mediator relationship (i.e., they have less influence), and up-weighting people with a weak treatment-mediator relationship (i.e., they have more influence). Like the product method, the IOW method assumes no residual confounding of the effects of the 1) exposure on the mediator, 2) mediator on the outcome, or 3) exposure on the outcome upon conditioning on pre-exposure confounders, as well as no confounders that are affected by exposure (Tchetgen Tchetgen, 2013, Author et al., 2015). Strengths over the product method include that it can accommodate multiple mediators in one model, it can accommodate mediators and outcomes of any functional form, and it can be used with any regression estimator that accommodates weights (Author et al., 2015). IOW also circumvents the complexity of having to specify the mediator density function (i.e., estimating the regression of mediator on exposure and covariates, and the variance of each residual and correlation between residuals), which can be cumbersome with multiple mediators (Author et al., 2015). Despite these advantages, IOW may produce larger standard errors than the product method when its assumptions are met. Triangulation using these different methods, with differing assumptions and strengths, may give us more confidence in our findings, to the extent that results converge.

The overall total effect of treatment on mental health was derived from a linear regression ITT model, unadjusted for mediators. We estimated product method direct effects of MTO treatment on mental health using a linear regression model, adjusted for mediators, then calculated product method indirect effects by taking the difference between the total and product method direct effects. A detailed account of the application of IOW mediation has been published (Author et al., 2015); briefly, IOW is implemented in three steps. In step 1, we recovered the predicted odds for each individual from a logistic regression model predicting treatment from all mediators and covariates. In step 2, we created an inverse odds weight by taking the inverse of the predicted odds for every participant derived from step 1. In step 3, we estimated IOW direct effects using a weighted linear regression model, adjusting for the proposed mediators by applying the IOW from step 2, then calculated IOW indirect effects by taking the difference between the total and IOW direct effects. For both methods, effect estimates were bootstrapped 500 times to derive standard errors, and we estimated the magnitude of mediation by dividing the direct effect by the total effect. We lastly tested whether the indirect effects significantly differed between the product and IOW method; no tests were significant.

Missing Data.

Missing data for the MTO survey measures ranged from 0.2% (youth exposure to violence) to 7% (informal social control), and we employed listwise deletion for each model. Since the original MTO investigators adjusted for attrition in their two-staged interim survey design and constructed post-estimation weights which we apply in our analysis, we are less concerned about bias from attrition in the MTO survey. However, the missing data for external neighborhood scales (7.8%) was patterned by place (missing for suburban areas because some subjects moved outside the central city, and this was patterned by treatment group). We imputed missing external survey data using multiple imputation by chained equations (MICE) to impute missing suburban neighborhood values from census tract and other neighborhood characteristics; unimputed results are also reported for comparison. Analyses were performed in STATA/SE 14 (StataCorp LP, College Station, TX). Institutional Review Boards at the authors’ institutions approved this study.

RESULTS

Effects of MTO on Neighborhood Context.

Table 1 displays baseline characteristics of the MTO adolescent sample overall and by treatment group. Baseline characteristics are equal across treatment groups, with the exception of youth baseline behavioral/emotional problems and being suspended/expelled from school (both are higher in the treatment group). Table 2 demonstrates that MTO treatment caused improvements (vs. controls) in most neighborhood factors tested for the youth at follow-up (4–7 years after randomization), including reductions in household head-reported physical and social disorder, violent victimization, and drug activity, and increases in informal social control and perceived safety. MTO treatment also significantly reduced youth reports of hearing gun shots and neighborhood drug activity, but did not affect youth-reported exposure to violence. MTO voucher treatment families also moved to neighborhoods that independently-sampled community members from non-MTO data sources reported as higher in collective efficacy, informal social control, and social cohesion, compared to control group families.

Table 2.

Intent to Treat Estimates of Moving to Opportunity Random Assignment Voucher Treatment Effects on Neighborhood Context.

Neighborhood Context	B	SE	LCI	UCI	P
MTO Head of Householda Self-Report
Social & Physical Disorder Scale	−0.199	0.036	−0.270	−0.128	< 0.001
Social Disorder Subscale	−0.252	0.040	−0.331	−0.173	< 0.001
Physical Disorder Subscale	−0.136	0.037	−0.208	−0.065	< 0.001
Informal Social Control Scale	0.370	0.089	0.196	0.544	< 0.001
Violent Victimization	−0.450	0.147	−0.737	−0.163	0.002
Safety Scale	0.303	0.051	0.203	0.403	< 0.001
Saw Drug Use/Selling	−0.584	0.132	−0.842	−0.325	< 0.001
MTO Youtha Self Report
Heard Gun Shots	−0.575	0.167	−0.902	−0.248	0.001
Saw Drug Use/Selling	−0.305	0.115	−0.530	−0.079	0.008
Violent Victimization	−0.023	0.117	−0.252	0.207	0.847
External Community Surveyb Measures
Collective Efficacy Scale	0.411	0.066	0.281	0.541	< 0.001
Social Cohesion Scale	0.443	0.065	0.315	0.572	< 0.001
Informal Social Control Scale	0.382	0.068	0.249	0.516	< 0.001

The analysis was weighted for varying intervention random assignment ratios across time and for attrition. All tests were adjusted for youth age, black race, Hispanic ethnicity, gender, site, and clustering at the family level. Dichotomous neighborhood context variables tested with logistic regression models, continuous variables tested with linear regression models. LCI= Lower 95% Confidence Interval; UCI = Upper 95% Confidence Interval.

^aSample size varies with missingness on the neighborhood variable, which varies from 3–10%: N=1426 girls, N=1403 boys

^bData sources: Boston Neighborhoods Survey (2000); Project on Human Development in Chicago Neighborhoods (1994–1995); New York Social Environment Survey (2005). N=878 girls, N=828 boys

Mediation Analyses.

For girls, the total effect of MTO treatment (vs. control) was beneficial for psychological distress (B= −0.12; p=0.05) and nonsignificant for BPI (B= −0.03; p= 0.60). For boys, the total treatment effect was harmful for both psychological distress (B= 0.14; p=0.03) and BPI (B= 0.18; p=0.003). Tables 3, 4, 5 display the mediation results testing potential neighborhood mediators, by gender and mental health outcome. Since the overall treatment effect for girls was negative (beneficial), a negative indirect effect for girls indicates that a mediator partly accounted for the beneficial effect (i.e., mediation in the expected direction). However, since the overall treatment effect for boys was positive (harmful), a negative indirect effect for boys indicates that a mediator decreased the harmful effect (i.e., countervailing mediation); in other words, after accounting for the mediator, the direct effect of treatment was more harmful than the total effect. Several neighborhood variables mediated the MTO effects on adolescent mental health, as indicated by significant indirect effects, particularly for girls’ distress and boys’ BPI. Significant indirect effects were more apparent for the product method than for IOW, since IOW had larger (almost double the size) standard errors, however the differences between the indirect effect coefficients estimated by the product method compared to IOW were not statistically significant for any model.

Table 3.

Indirect Effects from Mediation of MTO Voucher Treatment on Mental Health by Subjective Neighborhood Context - Head of Household, 2002 MTO Interim Survey^a

		Girls’ Distress		Boys’ Distress		Boys’ BPI
Mediators	Mediation Method	Beta (SE)	P	Beta (SE)	P	Beta (SE)	P
Neighborhood Disorder	PM	−0.029(0.011)	0.009	−0.007(0.010)	0.482	−0.017(0.010)	0.089
	IOW	−0.034(0.019)	0.072	−0.006(0.020)	0.764	0.003(0.022)	0.880
Social Disorder	PM	−0.040(0.012)	0.001	−0.012(0.011)	0.256	−0.022(0.011)	0.042
	IOW	−0.045(0.020)	0.024	−0.015(0.023)	0.503	−0.005(0.023)	0.837
Physical Disorder	PM	−0.013(0.008)	0.119	−0.001(0.007)	0.893	−0.008(0.007)	0.252
	IOW	−0.015(0.017)	0.380	−0.003(0.020)	0.893	0.017(0.020)	0.408
Informal Social Control	PM	−0.012(0.012)	0.306	−0.002(0.005)	0.712	0.006(0.005)	0.243
	IOW	−0.006(0.020)	0.754	−0.013(0.018)	0.48	0.028(0.018)	0.131
Violent Victimization	PM	−0.010(0.007)	0.128	0.000(0.006)	0.982	−0.017(0.009)	0.051
	IOW	−0.001(0.017)	0.943	−0.004(0.017)	0.83	0.011(0.019)	0.551
Safety	PM	−0.015(0.011)	0.186	0.003(0.010)	0.768	−0.016(0.010)	0.105
	IOW	−0.011(0.020)	0.579	0.002(0.023)	0.922	0.007(0.022)	0.766
Saw Drug Use/Selling	PM	0.006(0.009)	0.460	0.002(0.007)	0.768	−0.002(0.007)	0.749
	IOW	0.009 (0.018)	0.615	−0.005(0.020)	0.807	0.026(0.021)	0.222

^aSample size varies with missingness on the mediator which varies from 3.5–7%: N=1426 girls, N=1403 boys

Notes: Linear regression models adjusted for: youth age, black race, Hispanic ethnicity, gifted student, learning problems, behavioral problems, school asked to talk about problems youth was having with schoolwork or behavior, youth expelled from school, parental education, parental marital status, parental employment status, parental school enrollment status, parental receipt of public assistance/welfare, no teens present in baseline household, lived in baseline neighborhood for 5 years or move, parent believed streets in baseline neighborhood were very unsafe at night, and moved more than 3 times prior to baseline.

PM=Product Method; IOW=Inverse Odds Weight

Table 4.

Indirect Effects from Mediation of MTO Treatment on Mental Health by Subjective Neighborhood Context - Youth, 2002 MTO Interim Survey^a

		Girls’ Distress		Boys’ Distress		Boys’ Behavior Problems
Mediators	Method	Beta (SE)	P	Beta (SE)	P	Beta (SE)	P
Youth Heard Gun Shots	PM	−0.011(0.007)	0.134	−0.010(0.007)	0.13	−0.018(0.008)	0.024
	IOW	0.000(0.018)	0.988	−0.001(0.018)	0.946	0.004(0.017)	0.817
Youth Saw Drug Use/Selling	PM	−0.018(0.009)	0.051	−0.003(0.004)	0.515	−0.011(0.013)	0.369
	IOW	−0.006(0.018)	0.742	0.005(0.017)	0.766	−0.006(0.020)	0.755
Youth Violent Victimization	PM	−0.004(0.012)	0.767	−0.003(0.013)	0.816	−0.004(0.017)	0.814
	IOW	0.004(0.020)	0.836	0.008(0.021)	0.714	0.024(0.023)	0.31

^aSample size varies with missingness on the mediator which varies from 0.2-63-10%: N=1426 girls, N=1403 boys

Notes: Linear regression models adjusted for: youth age, black race, Hispanic ethnicity, gifted student, learning problems, behavioral problems, school asked to talk about problems youth was having with schoolwork or behavior, youth expelled from school, parental education, parental marital status, parental employment status, parental school enrollment status, parental receipt of public assistance/welfare, no teens present in baseline household, lived in baseline neighborhood for 5 years or move, parent believed streets in baseline neighborhood were very unsafe at night, and moved more than 3 times prior to baseline.

PM=Product Method; IOW=Inverse Odds Weight

Table 5.

Indirect Effects from Mediation of MTO Treatment Effects on Mental Health by External Measures of Collective Efficacy and Subscales in Boston, Chicago, and New York; Unimputed and Imputed Models^a,b,c

GIRLS’ DISTRESS		Unimputed		Imputed
Mediators	Method	Beta (SE)	P	Beta (SE)	P
Collective Efficacy	PM	0.005 (0.016)	0.758	0.014 (0.019)	0.470
	IOW	−0.003 (0.031)	0.915	−0.007 (0.031)	0.829
Informal Social Control	PM	0.002 (0.014)	0.896	−0.045 (0.021)	0.031
	IOW	−0.002 (0.030)	0.948	−0.006 (0.030)	0.833
Social Cohesion	PM	0.011 (0.018)	0.512	0.021 (0.018)	0.241
	IOW	0.002 (0.031)	0.939	−0.0001 (0.032)	0.997
BOYS’ DISTRESS		Unimputed		Imputed
Mediators	Method	Beta (SE)	P	Beta (SE)	P
Collective Efficacy	PM	0.008 (0.016)	0.615	0.020 (0.016)	0.213
	IOW	0.009 (0.030)	0.771	0.015 (0.029)	0.613
Informal Social Control	PM	0.001 (0.015)	0.954	0.002 (0.014)	0.916
	IOW	0.001 (0.031)	0.978	0.005 (0.029)	0.852
Social Cohesion	PM	0.021 (0.018)	0.235	0.024 (0.018)	0.175
	IOW	0.018 (0.031)	0.555	0.025 (0.029)	0.387
BOYS’ BPI		Unimputed		Imputed
Mediators	Method	Beta (SE)	P	Beta (SE)	P
Collective Efficacy	PM	−0.022 (0.015)	0.130	−0.015 (0.014)	0.301
	IOW	0.008 (0.032)	0.806	0.011 (0.031)	0.711
Informal Social Control	PM	−0.016 (0.013)	0.226	−0.010 (0.014)	0.471
	IOW	0.007 (0.031)	0.830	0.009 (0.031)	0.763
Social Cohesion	PM	−0.027 (0.016)	0.093	−0.021 (0.016)	0.180
	IOW	0.007 (0.033)	0.830	0.011 (0.031)	0.722

^aData sources: Boston Neighborhoods Survey (2000); Project on Human Development in Chicago Neighborhoods (1994–1995); New York Social Environment Survey (2005).

^bUnimputed sample size: N=878 girls, N=828 boys

^cMultiple imputation via chained equations was used to impute missing data on mediators using nonmissing data on the other mediators. Sample size with multiple imputation: N=953 girls, 898 boys

Notes: Linear regression models adjusted for: youth age, black race, Hispanic ethnicity, gifted student, learning problems, behavioral problems, school asked to talk about problems youth was having with schoolwork or behavior, youth expelled from school, parental education, parental marital status, parental employment status, parental school enrollment status, parental receipt of public assistance/welfare, no teens present in baseline household, lived in baseline neighborhood for 5 years or move, parent believed streets in baseline neighborhood were very unsafe at night, and moved more than 3 times prior to baseline.

PM=Product Method; IOW=Inverse Odds Weight

Girls’ Distress.

Household head-reported neighborhood disorder, and its social disorder subscale, mediated the beneficial effect of MTO on girls’ distress (Table 3). For example, after adjusting for social disorder as a mediator, the product method demonstrated a reduction in the total effect of MTO on girls’ distress by 43% (indirect effect (IE) B(se) = −.040 (.012), p=.001), and IOW demonstrated a reduction in the total effect by 49% (IE B(se) = −.045(.020), p=.024). Youth witnessing drug activity also marginally mediated the effect of MTO on girls’ distress using the product method (IE B(se) = −.018(.009), p=.051), reducing the total effect by 16% (Table 4). Informal social control, operationalized from independent neighbor surveys, emerged as a significant mediator (IE B(se) = −.045(.021), p=.031; Table 5) when missing suburban values were imputed, although in the original unimputed results the IE was close to zero.

Boys’ Behavioral Problems.

Household head-reported neighborhood disorder, and its social disorder subscale, also mediated the adverse effect of MTO on boys’ behavioral problems using the product method (Table 3). Significant mediation here in the presence of adverse (positive) total and direct effects indicates countervailing mediation, or suppression effects. Again, using social disorder as an example, the total effect increased by 12% (i.e., the direct effect is more positive than the total effect) after adjusting for this mediator (social disorder IE B(se) = −0.022 (0.011) p=.042); this means that if MTO had not improved boys’ neighborhood social disorder the treatment effect would have been more harmful. Violent victimization (IE B(se) = −0.017 (0.009) p=.051; Table 3), youth hearing gun shots (IE B(se) = −0.018 (0.008) p=.024; Table 4), and social cohesion (IE B(se) = −.027 (.016) p=.093; Table 5) also exhibited countervailing mediation for boys’ BPI, with the total effect increasing by 10%, 10%, and 8% respectively after accounting for these mediators. The IOW method did not identify any significant mediators for boys’ BPI, but formal tests of the difference in indirect effect coefficients between the two mediation methods were nonsignificant. No external measures of neighborhood social context mediated MTO effects on BPI for boys when suburban values were imputed; however original unimputed results identified significant indirect effects for social cohesion.

Mediation: Boys’ Distress.

We identified no significant mediation of MTO’s harmful effect on boys’ distress (Tables 3–5).

DISCUSSION

Random assignment to receive a housing voucher to move out of high poverty public housing and into a private rental apartment significantly improved most dimensions of neighborhood context to which families were exposed, compared to their public housing control group counterparts, over a 4–7 year period. Improvements occurred in MTO participant-reports of disorder, violent victimization, informal social control and perceived safety, as well as externally-measured neighborhood collective efficacy and its subscales informal social control, and social cohesion. Some of these improved neighborhood characteristics mediated MTO treatment effects on youth mental health.

In particular, social disorder emerged as a consistent neighborhood mediator of the MTO treatment effects on youth mental health. For treatment group girls, the reduction in social disorder stemming from moves to lower poverty neighborhoods significantly accounted for the beneficial effect of MTO, a finding that was consistent across methods (product method vs. IOW); the reduction between the total and direct effect was 43–49% respectively. This finding is consistent with prior literature documenting that social disorder is positively associated with internalizing behaviors among girls (Browning et al., 2013) and women (Schulz et al., 2006).

Social disorder exhibited countervailing mediation for boys’ BPI, where the harmful total effect on behavior problems increased after adjusting for social disorder. In other words, improvements in social disorder mitigated the harmful effect of MTO on boys’ behavior problems. This is consistent with research documenting a positive relationship between social disorder and behavior problems (Strohschein and Matthew, 2015). That social disorder did not mediate effects on boys’ psychological distress is not surprising given some evidence that social disorder may not be related to internalizing behaviors among boys (Browning et al., 2013). We did not find that safety or victimization mediated the beneficial MTO effect on girls’ distress, contrary to qualitative evidence stating this as a key mediator, and despite the fact that safety was one of the leading motivations for family participation in the MTO program (Popkin et al., 2002, Popkin et al., 2008).

A multi-level, ecometric¹ measure of neighborhood collective efficacy and its subscales, measured using independently-fielded community surveys, emerged as a potential mediator only for girls’ distress. Given that missing data for neighborhood measures in suburban areas was patterned by treatment in this RCT, we believe the imputed models are less biased. However, these results should be interpreted with caution, as they were not robust in unimputed models.

Although we found countervailing mediation on boys’ behavior problems, we did not identify any neighborhood contextual mediators that worsened after treatment that could explain the harmful effect. So, although we identified mediators that seemed to dampen the harmful effect of treatment, the causal mechanisms of the harmful effects were not captured by neighborhood environments. For example, perhaps relevant mediating mechanisms occur in friendship networks that are not spatially patterned, or are patterned by other contexts such as families or schools. Adult male role models are essential for preventing risky behaviors among minority boys (Caldwell et al., 2010), and qualitative work with the MTO sample suggests that treatment group boys are less likely to report the presence of a father figure (Clampet-Lundquist et al., 2011). Evidence suggests that substance use may also be an important mediating mechanism to explain the harmful effects of MTO on boys’ behavior problems (Author et al., 2017).

Overall, the two mediation methods we tested generated similar results, however, IOW is less efficient, and may require larger total effect sizes to identify mediating variables. Although they both estimate natural direct and indirect effects and rely on the assumption of no unmeasured confounding, they otherwise have different assumptions. The product method requires linear mediators and outcomes, and assumes no treatment-mediator interaction effects on the outcome. IOW does not carry these assumptions, thus this semiparametric method is more flexible in accommodating nonlinearities and treatment-mediator interactions. That we see consistency between these two methods, despite their differing approaches, gives us more confidence in our findings.

Study strengths and limitations

The most important limitations from our study concern power and multiple testing. As with most RCTs, the MTO study was not powered for mediation, or for effect modification (e.g. by gender) either. Therefore, we used a liberal threshold for Type 1 error. We also did not explicitly adjust for multiple comparisons, which could lead to false discovery. Our results may therefore be viewed as exploratory. Despite this, we felt it was important to test these hypotheses to inform etiology of neighborhood effects on health (which are somewhat black box, estimated typically with weak study designs), as well as to inform practice including future voucher study design.

Data from the interim survey included self-reports of neighborhood conditions which likely are measured with error. There is scant evidence on the reliability of youth reports of neighborhood constructs, however, available research indicates youth reports are reliable. One study demonstrated acceptable reliability of youth reports of neighborhood social constructs (e.g., social cohesion reliability = .76, physical disorder = .77) (Martin et al., 2017), while another showed good internal consistency (.68 to .93) and test-retest reliability (.75 to .94) for youth self-reported exposure to violence (e.g., hearing gun shots) (Selner-O’Hagan et al., 1998). Because the outcomes were also self-report, errors may be dependent. For instance, adolescents with higher BPI may systematically under- or over- report on neighborhood problems. These biases can pull effect estimates away from the null (Lash and Fink, 2003).

We attempted to mitigate these measurement limitations by testing mediators from different reporters, e.g., youth self-reports, household head reports, and externally-measured neighborhood context. This study focused only on subjective measures of neighborhood social context. Objective measures of neighborhood context, such as administrative data, may have important associations with health (Weden et al., 2008). However, we have chosen to focus on subjective neighborhood context because 1) they likely tap aspects of neighborhood context that are most important to residents (Mayes and Lewis, 2012), and 2) associations with health may be stronger for subjective than objective measures (Weden et al., 2008).

There are other possible sources of error for subjective neighborhood measures. For example, it is possible that perceptions of safety or social disorder changed over time, irrespective of household moves, from baseline to 2002 as children aged. If there was an imbalance in age across treatment groups, this could lead to differential measurement in perceptions of neighborhood context across treatment group. However, there were no treatment group differences in the distribution of children’s age at baseline or the proportion of teenagers in the home, therefore, we do not think this bias is operating.

It should be noted that MTO is a people-based policy focused on providing affordable housing and expanding neighborhood opportunity among low-income families, rather than a place-based neighborhood policy focused on improving neighborhood conditions (Author, 2010). Therefore, MTO cannot provide evidence as to how improving the neighborhood context for everyone would impact the lives of all individuals living in a neighborhood. However, housing mobility policy focuses limited resources on the neediest families, while increasing housing affordability and allowing low-income families to exercise locational choice to access higher opportunity neighborhoods, making MTO highly policy-relevant (Author, 2010). Ideally, both people-based and place-based policies should be used to combat inequalities in neighborhoods and housing (Author, 2010, Katz, 2004).

Conclusion

Although measuring neighborhood social context (as opposed to measures of composition, like neighborhood SES) is more difficult and likely incurs more measurement error, it remains important for research to unpack why neighborhoods matter, particularly when many dimensions of neighborhood context co-occur. In this paper, we tested neighborhood social context as mediators of the MTO treatment effects on youth psychological distress and behavior problems. Social disorder emerged as a primary mediator explaining the beneficial effects on girls’ distress, whereas improving social disorder dampens the harmful effects of housing mobility on boys’ behavior problems. These findings suggest that using neighborhood poverty alone to define improved neighborhood context is insufficient to effectively improve health and wellbeing among low, income families. The next generation of housing policy may benefit from incorporating a focus on the broader social context, to facilitate moves to lower disorder neighborhoods.

HIGHLIGHTS.

• MTO treatment improved neighborhood social context for adolescents of both genders.

• For girls’ distress, neighborhood disorder and drug activity were partial mediators.

• For boys’ behavior problems, disorder, victimization, and informal social control exhibited countervailing mediation.

• Neighborhood social context did not mediate harmful effects on boys’ distress.

• Housing mobility policy may improve mental health via neighborhood social context.