Abstract
Objective:
Research implicates environmental risk factors, including correlates of urbanicity, deprivation, and environmental toxins, in psychotic-like experiences (PLEs). The current study examined associations between several types of environmental risk factors and PLEs in school-age children, whether these associations were specific to PLEs or generalized to other psychopathology, and examined possible neural mechanisms for significant associations.
Method:
The current study used cross-sectional data from 10,328 9-10-year-olds from the Adolescent Brain Cognitive Development℠ study. Hierarchical linear models examined associations between PLEs and geocoded environmental risk factors, and whether associations generalized to internalizing/externalizing symptoms. Mediation models examined evidence of structural MRI abnormalities (e.g., intracranial volume) potentially mediating associations between PLEs and environmental risk factors.
Results:
The results found specific types of environmental risk factors, namely measures of urbanicity (e.g., drug offense exposure, less perception of neighborhood safety), deprivation (e.g., overall deprivation, poverty rate), and lead exposure risk, were associated with PLEs. These associations showed evidence of stronger associations with PLEs than internalizing/externalizing symptoms (especially overall deprivation, poverty, drug offense exposure, and lead exposure risk). There was evidence that brain volume mediated between 11-25% of associations between poverty, perception of neighborhood safety, and lead exposure risk with PLEs.
Conclusions:
Although in the context of cross-sectional analyses, this evidence is consistent with neural measures partially mediating the association between PLEs and environmental exposures. This study also replicated and extended recent findings of associations between PLEs and environmental exposures, finding evidence for specific associations with correlates of urbanicity, deprivation, and lead exposure risk.
Keywords: psychotic-like experiences, urbanicity, deprivation, lead exposure, MRI
Lay Summary:
Examining 10,328 9-10-year-olds using Adolescent Brain Cognitive Development℠ Study cross-sectional data, childhood psychotic-like experiences (PLEs) were specifically associated with several environmental risk factors, including measures of urbanicity, deprivation, and lead exposure risk. Further, we found evidence that associations with brain volume may account for approximately 11-25% of the association between these environmental risk factors and PLEs. The current study clarifies the nature of the association between PLEs and environmental exposures, including finding possible neural mechanisms for these associations.
Research investigating associations between environmental characteristics and psychosis1 indicates exposure to urban environments is associated with two-to-three-fold increased psychosis risk.2 However, only recently has research examined associations between environmental risk factors and earlier markers of psychosis risk, including associations with psychotic-like experiences (PLEs).3 PLEs, or nonclinical schizophrenia spectrum symptoms (e.g., perceptual abnormalities, mild delusional thoughts), in childhood are associated with greater odds of developing psychiatric disorders, including psychotic disorders, during adulthood.4 The current study will examine associations between these environmental factors and PLEs in school-age children.
Few studies have examined which specific environmental factors, including urbanicity (e.g., population density, crime rates),5 toxins (including air pollution),3 or increased poverty/deprivation,6 are associated with psychosis risk when accounting for confounding factors, including financial adversity and psychiatric family history (Figure 1).3 A wealth of evidence suggests that exposure to socio-environmental adversity, including deprivation (e.g., low income, employment, education) is associated with increased rates of psychosis.7 Research has also found evidence for associations of environmental toxins and pollutants (e.g., particulate matter (PM) and nitrogen dioxide (NO2)) with markers of psychosis risk,8 though research also implicates these pollutants in other psychiatric problems including internalizing and externalizing symptoms.9 However, a commentary on prior research3 suggested that the association between pollutants and PLEs may be at least partially attributable to lead exposure,10 consistent with other previous work.11 One reason to suspect exposure to toxins may contribute to associations between environmental risk factors and psychosis is that exposure to air pollution or lead may result in increased oxidative stress and/or systemic inflammation, which have been suggested as causal factors in psychosis development.12 Furthermore, the authors of a recent study suggested that air pollution may increase risk for PLEs by directly influencing brain structure or function due to increased inflammation or stress.3
The idea that environmental factors may be associated with increased PLEs through a specific influence on brain development is consistent with research linking reductions in volume, surface area, and cortical thickness to psychosis spectrum symptoms.13,14 Furthermore, research indicates that especially deprivation15 and exposure to environmental toxins16,17 are associated with a host of structural abnormalities, including global volumetric reductions and altered cortical thickness. However, there is less research regarding the interrelationships between environmental risk factors, structural abnormalities, and markers of psychosis risk. There is evidence that urban exposure is associated with reduced grey matter volume in males with psychosis,18 although another study found urbanicity did not influence the association between genetic risk for psychosis and cortical thickness.19 To our knowledge, no studies have explicitly examined whether there is evidence consistent with structural brain measures mediate associations between specific environmental risk factors and PLEs as will be examined in the current study (Figure 1).
The current study examined associations between PLEs and environmental risk factors using cross-sectional data from 9-10-year-olds in the Adolescent Brain Cognitive Development℠ study (ABCD Study®). This age range provides several important advantages, including providing evidence that early exposure to toxins and deprivation may have an influence on pre-pubertal neurodevelopment related to PLEs.20 We examined whether, as expected, PLEs were associated with exposure to specific types of environmental risk factors, including urbanicity (population density and crime exposure), deprivation (including neighborhood adversity), and toxins. Furthermore, given that environmental risk factors are also associated with other psychiatric difficulties (e.g., internalizing and externalizing symptoms),21 the current study also examined evidence of specificity of associations with environmental risk factors. Lastly, we examined possible mechanisms for significant associations, specifically whether there is evidence consistent with the hypotheses that structural brain metrics (e.g., volume, surface area, cortical thickness) mediate associations between environmental risk factors and PLEs. Research indicates that pre-adolescence is associated with ongoing brain changes, including increases in both grey and white matter preceding the post-adolescent decreases in cortical grey matter.22 Additionally, research indicates that environmental risk factors are associated with reductions in brain structure and function.15 It was therefore expected that environmental risk factors would be associated with global reductions in brain volume, surface area, and thickness.15-17 Further, as PLEs in general are associated with reduced global volume, surface area, and cortical thickness,13,14 we hypothesized that we would find evidence consistent with reduced brain volume, surface area, and cortical thickness mediating the associations between environmental risk factors and PLEs.
Methods
Participants
A sample of 11,875 individuals was obtained from the ABCD Study® (Data Release 2.0.1), a large-scale study tracking 9-10-years-olds recruited from 21 research sites across the United States.23 Potential participants were excluded from participating in the ABCD Study® for the following reasons: child not fluent in English, MRI contraindication (e.g., irremovable ferromagnetic implants or dental appliances, claustrophobia, pregnant), major neurological disorder, gestational age less than 28 weeks or birthweight less than 1,200 grams, history of traumatic brain injury, or had a current diagnosis of schizophrenia, autism spectrum disorder (moderate, severe), mental retardation/intellectual disability, or alcohol/substance use disorder.
ABCD Study® data were accessed from the National Institutes of Mental Health Data Archive (see Acknowledgments). All measures were collected at the ABCD Study® baseline assessment wave. Participants were removed from analyses due to missing data (n=1,546; Table S1). The final sample size was 10,328 individuals (47.4% female; 53.2% White, 20.1% Hispanic, 14.0% African American, 2.1% Asian, and 10.6% Other).
Measures
Symptom Measures
Child participants completed the Prodromal Questionnaire-Brief Child Version (PQ-BC), a 21-item self-report questionnaire previously validated for use with school-age children using the ABCD Study® sample.24 Consistent with this previous research,24 distress scores were calculated as the total number of endorsed questions weighted by level of distress [i.e., 0=no, 1=yes (but no distress), 2-6=yes (1+score on distress scale)]. Parental PLEs were assessed as in previous research24 using the summation of parent’s responses to four questions from the Achenbach Adult Self Report.25 Internalizing symptoms were examined using a) the validated and computerized Kiddie-Structured Assessment for Affective Disorders and Schizophrenia (KSADS) for DSM-526 using summations of child-rated current depression and generalized anxiety disorder symptoms, and b) parent-rated Child Behavior Checklist (CBCL) internalizing measure t-scores.25 We also examined the child’s externalizing symptoms using the parent-rated CBCL externalizing measure t-scores.25
Family history of psychiatric disorder was assessed using the parent-rated Family History Assessment Module Screener,27 and scored as the proportion of family members (i.e., the participant’s mother, father, or maternal grandmother, grandfather, uncles, aunts) endorsing a history of psychosis, depression, mania, suicidality, previous hospitalization or professional help for mental health issues.3 Financial adversity was measured as the summation of endorsement of seven parent-rated questions of financial difficulties from a demographic questionnaire. This measure included the following questions: 1) Needed food but couldn't afford to buy it or couldn't afford to go out to get it?, 2) Were without telephone service because you could not afford it?, 3) Didn't pay the full amount of the rent or mortgage because you could not afford it?, 4) Were evicted from your home for not paying the rent or mortgage?, 5) Had services turned off by the gas or electric company, or the oil company wouldn't deliver oil because payments were not made?, 6) Had someone who needed to see a doctor or go to the hospital but didn't go because you could not afford it?, 7) Had someone who needed a dentist but couldn't go because you could not afford it?
Environmental Risk Factors
A number of environmental risk factors were retrieved based on the child’s primary address coordinates (Figure 1a; Table S2 for correlations between environmental risk factors).
Urbanicity, Safety and Crime.
Based on current residential address at baseline, population density, a neighborhood walkability index, parent-rated perception of neighborhood safety, and crime exposure estimates were examined.
Population density was calculated based on estimates from the Socioeconomic Data and Applications Center (SEDAC), calculated based on the 2010 census tracts and adjusted to United Nations (UN) estimates of national-level population counts. A neighborhood walkability index was created based on data obtained from the EPA (https://www.epa.gov/smartgrowth/smart-location-mapping#walkability) and 2010 census tract estimates. Perception of neighborhood safety was calculated as a summation of three parent-rated questions (i.e., “I feel safe walking in my neighborhood, day or night”; “Violence is not a problem in my neighborhood”; “My neighborhood is safe from crime”; each was rated on a scale from 1-5, 1=strongly disagree, 5=strongly agree).
Next, crime exposure information was obtained from the Uniform Crime Report from FBI, compiled by Inter-university Consortium for Political and Social Research,28 averaged from 2010-2012 to create stable county-level estimates. Crime exposure estimates included grand total offenses, total violent offenses, total drug offenses, and total Driving Under the Influence (DUIs).
Deprivation.
Overall deprivation was defined as the Area Deprivation Index (ADI) national percentile scores for the current residential address at baseline, calculated from the 2011-2015 American Community Survey 5-year summary (Table S5 for ADI subscores).29 We also examined the proportion of individuals living in poverty (−125% of poverty level), average age of the home within the area, and number of years at current residence.
Environmental toxins.
Based on current residential address at baseline, estimates of air pollution exposure and lead exposure were examined. Estimates of air pollution, including nitrogen dioxide (N02) levels (primarily obtained from car emissions) were obtained from NASA SEDAC, based on satellite reports averaged over 3 years (2010-2012) with a resolution of 100 km2. In addition, a 2016 annual average of daily particulate matter 2.5 (PM2.5; consisting of inhalable particles, with diameters generally ≤2.5 micrometers) was created at 1 km2 spatial resolution.30 Estimates of lead exposure risk were obtained by first geocoding the participant’s address at the census tract-level and then calculating risk scores based on data obtained from vox.com (https://www.vox.com/a/lead-exposure-risk-map). Estimated lead exposure risk scores (1-10, 10 being the most at risk) were calculated using proportion of individuals living in poverty and average age of the home (see Deprivation section above).
Structural MRI Measures
For the current study, structural MRI measures include volume,31 area,32 and cortical thickness.33 All children were run on a 3T scanner- Prisma (Siemens Healthineers, Munich, Germany), Discovery MR750 (GE Healthcare, Chicago, IL), or Achieva dStream or Ingenia CX (Philips Healthcare, Andover, MA)- with a 32-channel head coil and completed T1-weighted and T2-weighted structural scans (1mm isotropic). Structural neuroimaging processing was completed using FreeSurfer version 5.3.0 through standardized processing pipelines.34 Participants that did not pass FreeSurfer Quality Control measure (i.e., at least one T1 scan that passed all quality control metrics) were excluded from analyses (n= 69). Cortical reconstruction and volumetric segmentation was performed by the ABCD Study® Data Acquisition and Integration Core using the FreeSurfer image analysis suite (http://surfer.nmr.mgh.harvard.edu/). This pre-processing includes removal of non-brain tissue using a hybrid watershed/surface deformation procedure,35 automated Talairach transformation, segmentation of the subcortical white matter and deep gray matter volumetric structures, intensity normalization, tessellation of the gray/white matter boundary, automated topology correction, and surface deformation following intensity gradients.33 Images were registered to an atlas, which was based on individual cortical folding patterns to match cortical geometry across subjects. The cerebral cortex was parcellated into 34 regions per hemisphere based on the gyral and sulcal structure. The subcortical white matter and deep gray matter volumetric structures were segmented into 22 regions.36
For the current study, we specifically examined the following global structural MRI metrics (see Supplement 1 for analyses with each of the parcellated cortical and subcortical Desikan regions, available online): intracranial volume, total cortical brain volume, total subcortical gray matter volume, total surface area, total cortical thickness, as well as hippocampal volume due to the wealth of research linking hippocampal volume to both environmental risk factors and PLEs.15,37
Statistical Analyses
The analyses used hierarchical linear models (HLMs), with all multiple comparisons False Discovery Rate (FDR) corrected across 15 models. We employed FDR as opposed to more conservative approaches (e.g., Bonferroni) as a way to effectively balance the risk of both Type I and Type II error in this relatively large sample. All analyses were conducted in R lme4 package38 (multcomp package for multiple comparison analyses39), with family unit and the 21 ABCD Study® research site modeled as random intercepts. We included covariates consistent with previous analyses (Figure 1a).3 For imaging analyses, covariates also included scanner type and average motion rating. Results are expressed as standardized estimates (βs) with 95% bootstrapped (5000 iterations) confidence intervals (CIs).
HLMs examined the association between PLEs and each of the environmental risk factors (Figure 1a). Due to significant skew and zero inflation of PLEs and child-rated internalizing symptoms, negative binomials were also conducted, with results remaining consistent. We also examined whether associations between PLEs and environmental risk factors exhibited specificity, or whether these variables were also (or even more strongly) associated with internalizing and/or externalizing symptoms. Differences between correlations were examined using Meng’s z-test procedures.40 Lastly, we performed hierarchical mediation analyses using the lavaan package in R41 to examine evidence for structural MRI abnormalities potentially mediating associations between PLEs and significant environmental risk factors.
Results
Associations between PLEs and Environmental Risk Factors
Urbanicity, Safety and Crime.
Even when accounting for covariates (Table 1 and Figure 1a; Table S3 for associations with individual PQ-BC items and Table S4 for associations with covariates; see Supplement 1 for analyses only containing Caucasian participants), greater total drug offense exposure (R2=.022) and less perception of neighborhood safety (R2=.019) were associated with greater PLEs.
Table 1.
Risk Factor | β | Lower 95% CI |
Upper 95% CI |
SE | tb | FDR- corrected p |
R2 |
---|---|---|---|---|---|---|---|
Urbanicity, Crime, and Safety | |||||||
Population Density | 0.006 | −0.016 | 0.028 | 0.011 | 0.516 | .61 | 0.019 |
Walkability | −0.010 | −0.034 | 0.015 | 0.012 | −0.804 | .52 | 0.019 |
Perception of Neighborhood Safety | −0.054 | 0.033 | 0.075 | 0.011 | 5.084 | <.001 | 0.019 |
Crime Exposure Indexes | |||||||
Grand Total Offenses Exposure | 0.048 | −0.006 | 0.103 | 0.027 | 1.783 | .15 | 0.019 |
Adult Violent Offenses Exposure | −0.092 | −0.270 | 0.086 | 0.090 | −1.021 | .41 | 0.018 |
Total Drug Offenses Exposure | 0.227 | 0.057 | 0.402 | 0.086 | 2.648 | .02 | 0.022 |
DUI Offenses Exposure | −0.118 | −0.289 | 0.052 | 0.085 | −1.389 | .27 | 0.018 |
Deprivation | |||||||
Overall Deprivation (ADI Percentile Score) | 0.146 | 0.121 | 0.172 | 0.013 | 11.214 | <.001 | 0.034 |
Years at Residence | −0.056 | −0.076 | −0.036 | 0.010 | −5.491 | <.001 | 0.021 |
Rate of Poverty | 0.075 | 0.051 | 0.098 | 0.012 | 6.269 | <.001 | 0.024 |
Housing Age | −0.007 | −0.030 | 0.017 | 0.012 | −0.573 | .61 | 0.019 |
Environmental Toxins | |||||||
2016 PM2.5 levels | 0.021 | −0.012 | 0.053 | 0.016 | 1.254 | .31 | 0.020 |
N02 levels | −0.036 | −0.081 | 0.009 | 0.022 | −1.603 | .19 | 0.019 |
Proximity to a Major Road | −0.005 | −0.025 | 0.015 | 0.010 | −0.525 | .61 | 0.019 |
Lead Exposure Risk | 0.029 | 0.006 | 0.053 | 0.012 | 2.467 | .03 | 0.019 |
Abbreviations: PLEs=Psychotic-like Experiences; β=standardized regression coefficient; CI=95% Confidence Interval; SE=standard error; t=t-test test statistic; p=p-value; FDR=False Discovery Rate-corrected for multiple comparisons; R2=R-squared, or the proportion of variance explained; ADI=Area Deprivation Index; DUI=driving under the influence; PM2.5=particulate matter with inhalable particles with diameters generally 2.5 micrometers and smaller; N02=nitrogen dioxide. Significant estimates in bold.
Models examining significant environmental risk factors (i.e., those risk factors in bold) remained significant even when including both child-rated internalizing as well as parent-rated internalizing and externalizing symptoms, FDR-corrected ps<.05.
two-tailed.
Deprivation.
Greater overall deprivation as assessed by ADI percentile score (R2=.034), rate of poverty (R2=.024), and fewer years at residence (R2=.021) were all associated with greater PLEs (Table S5 for specific ADI sub-score associations with PLEs), even when accounting for covariates (Table 1 and Figure 1a).
Environmental toxins.
When accounting for covariates (Table 1 and Figure 1a), only increased lead exposure risk (R2=.019) was significantly associated with increased PLEs.
Specificity of Associations between PLEs and Environmental Risk Factors
We also examined whether associations between PLEs and environmental risk factors were specific to PLEs or whether these variables were also associated with internalizing and/or externalizing symptoms. When including PLEs in the model (Table 2; Supplement 1 for additional results), both parent-rated internalizing and externalizing symptoms were significantly associated with less perception of neighborhood safety. In addition, parent-rated externalizing symptoms were significantly associated with fewer years at residence. Internalizing and externalizing symptoms were not significantly associated with the other risk factors, including total drug offense exposure, overall deprivation, rate of poverty, and lead exposure risk. Importantly, PLEs remained significantly associated with all aforementioned environmental risk factor correlates even when including internalizing and externalizing symptoms in the model.
Table 2.
Child-Rated Internalizing Symptoms |
Parent-Rated Internalizing Symptoms |
Parent-Rated Externalizing Symptoms |
|||||||
---|---|---|---|---|---|---|---|---|---|
Risk Factor | β | ta | FDR- corrected p |
β | ta | FDR- corrected p |
β | ta | FDR- corrected p |
Urbanicity | |||||||||
Population Density | 0.001 | 0.370 | .83 | 0.005 | 1.584 | .24 | 0.003 | 1.063 | .70 |
Walkability | 0.004 | 0.978 | .73 | 0.000 | −0.043 | .99 | 0.002 | 0.464 | .88 |
Perception of Neighborhood Safety | −0.002 | −0.282 | .83 | −0.041 | −4.780 | <.001 | −0.035 | −4.271 | <.001 |
Crime Exposure Indexes | |||||||||
Grand Total Offenses Exposure | 0.000 | −0.201 | .84 | −0.006 | −2.708 | .06 | −0.002 | −1.072 | .70 |
Adult Violent Offenses Exposure | −0.001 | −1.271 | .73 | 0.000 | −0.586 | .73 | 0.000 | −0.283 | .88 |
Total Drug Offenses Exposure | 0.000 | −0.819 | .73 | −0.001 | −1.386 | .26 | 0.000 | −0.694 | .75 |
DUI Offenses Exposure | −0.001 | −0.949 | .73 | 0.000 | −0.202 | .95 | 0.000 | 0.284 | .88 |
Deprivation | |||||||||
Overall Deprivation (ADI Percentile Score) | 0.007 | 1.846 | .47 | −0.006 | −1.224 | .31 | 0.005 | 1.243 | .70 |
Years at Residence | −0.005 | −1.013 | .73 | −0.012 | −1.918 | .15 | −0.025 | −4.436 | <.001 |
Rate of poverty | 0.008 | 1.738 | .47 | −0.014 | −2.545 | .06 | −0.004 | −0.724 | .75 |
Housing Age | −0.001 | −0.295 | .83 | −0.010 | −1.877 | .15 | −0.004 | −0.935 | .72 |
Environmental Toxins | |||||||||
2016 PM2.5 levels | 0.001 | 0.489 | .82 | 0.002 | 0.493 | .76 | −0.001 | −0.175 | .92 |
N02 levels | 0.001 | 0.669 | .78 | −0.003 | −1.864 | .15 | −0.003 | −2.059 | .22 |
Proximity to a major road | 0.003 | 0.589 | .79 | 0.008 | 1.460 | .26 | 0.000 | −0.049 | .96 |
Estimated lead risk | 0.003 | 0.790 | .73 | −0.012 | −2.420 | .07 | −0.004 | −0.879 | .72 |
Abbreviations: PLEs=Psychotic-like Experiences; β=standardized regression coefficient; t=t-test test statistic; p=p-value; FDR=False Discovery Rate-corrected for multiple comparisons; ADI=Area Deprivation Index; DUI=driving under the influence; PM2.5=particulate matter with inhalable particles with diameters generally 2.5 micrometers and smaller; N02=nitrogen dioxide. Significant estimates in bold.
two-tailed.
Next, we examined whether PLEs showed significantly stronger associations with environmental risk factors in comparison to internalizing/externalizing symptoms. In comparison to internalizing and externalizing symptoms, PLEs showed significantly stronger associations with total drug offense exposure (Zs>10.12, ps<.001), greater deprivation (Zs>6.04, ps<.001), and rate of poverty (Zs>3.10, ps<.001). PLEs also showed significantly stronger associations with fewer years at residence than internalizing symptoms (Zs>−2.31, ps<.05; externalizing symptoms: Z=−0.90, p=.18), PLEs showed significantly stronger associations with perception of neighborhood safety than child-rated internalizing symptoms (Z=2.01, p<.05; parent-rated symptoms: Zs<−0.69, ps>.25), and PLEs showed significantly stronger associations with lead exposure risk than parent-rated internalizing and externalizing symptoms (Zs>3.02, p<.005; child-rated symptoms: Z=1.50, p=.07).
Associations between PLEs, Environmental Risk Factors, and Structural Abnormalities
Each of the structural MRI variables (i.e., ICV, total subcortical gray volume, total cortical volume, total cortical thickness, total surface area, and total hippocampal volume) were significantly associated with PLEs (Table 3) except for hippocampal volume. Furthermore, these structural MRI variables were also associated with all of the environmental risk factors that were significantly associated with PLEs (Table 4), including perception of neighborhood safety, all deprivation indices, and lead exposure risk. The exception was that total drug offense exposure was not significantly associated with cortical thickness.
Table 3.
MRI metric | β | Lower CI | Upper CI | SE | ta | FDR- corrected p |
---|---|---|---|---|---|---|
ICV | −0.028 | −0.045 | −0.011 | 0.009 | −3.196 | .002 |
Total subcortical gray volume | −0.034 | −0.052 | −0.015 | 0.009 | −3.569 | <.001 |
Total cortical volume | −0.042 | −0.060 | −0.023 | 0.009 | −4.409 | <.001 |
Total cortical thickness | −0.035 | −0.053 | −0.017 | 0.009 | −3.739 | <.001 |
Total surface area | −0.019 | −0.035 | −0.003 | 0.008 | −2.300 | .03 |
Total hippocampal volume | −0.015 | −0.032 | 0.003 | 0.009 | −1.641 | .10 |
Abbreviations: PLEs=Psychotic-like Experiences; β=standardized regression coefficient; CI=95% Confidence Interval; SE=standard error; t=t-test test statistic; p=p-value; FDR=False Discovery Rate-corrected for multiple comparisons; ICV=intracranial volume.
two-tailed.
Table 4.
ICV | Subcortical Gray Volume |
Total Volume | Total Surface Area | Average Cortical Thickness |
|||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Risk factor | β | ta | p | β | ta | p | β | ta | p | β | ta | p | β | ta | p |
Urbanicity, Crime, and Safety | |||||||||||||||
Perception of Neighborhood Safety | 0.076 | 7.695 | <.001 | 0.091 | 8.508 | <.001 | 0.106 | 9.935 | <.001 | 0.091 | 8.685 | <.001 | 0.041 | 4.483 | <.001 |
Total Drug Offenses Exposure | −0.344 | −4.456 | <.001 | −0.247 | −2.995 | <.001 | −0.392 | −4.608 | <.001 | −0.355 | −4.307 | <.001 | −0.099 | −1.373 | .17 |
Deprivation | |||||||||||||||
Overall Deprivation (ADI Percentile Score) | −0.148 | −12.097 | <.001 | −0.163 | −12.460 | <.001 | −0.199 | −15.053 | <.001 | −0.161 | −12.374 | <.001 | −0.088 | −7.858 | <.001 |
Years at Residence | 0.047 | 4.892 | <.001 | 0.036 | 3.451 | <.001 | 0.065 | 6.295 | <.001 | 0.046 | 4.546 | <.001 | 0.044 | 4.925 | <.001 |
Rate of Poverty | −0.111 | −10.104 | <.001 | −0.121 | −10.185 | <.001 | −0.148 | −12.417 | <.001 | −0.126 | −10.764 | <.001 | −0.055 | −5.418 | <.001 |
Environmental Toxins | |||||||||||||||
Lead Exposure Risk | −0.051 | −4.587 | <.001 | −0.059 | −4.937 | <.001 | −0.068 | −5.629 | <.001 | −0.061 | −5.129 | <.001 | −0.021 | −2.059 | .047 |
Abbreviations: β=standardized regression coefficient; CI=95% Confidence Interval; SE=standard error; t=t-test test statistic; p=p-value False Discovery Rate-corrected for multiple comparisons; ICV=intracranial volume; ADI=Area Deprivation Index.
two-tailed.
We next examined evidence for these structural MRI variables mediating the association between environmental risk factors and PLEs. There was evidence that cortical volume partially mediated the association between deprivation as measured by rate of poverty (and to a lesser extent, overall deprivation, see Table S6) and PLEs (indirect effect [path ab] bias-corrected 95% confidence interval [CI], 0.004-0.01; proportion mediated=16.42%; note similar evidence of mediation was found for the other volume indices, see Table S6). There was evidence that total cortical volume also partially mediated the relationship between perception of neighborhood safety (an index of urbanicity, safety, and crime) and PLEs (indirect effect [path ab] bias-corrected 95% confidence interval [CI], 0.004-0.008; proportion mediated=16.33%; note similar evidence of mediation was found for the other volume indices, see Table S6). There was also evidence that cortical volume mediated the association between lead exposure risk and PLEs (indirect effect [path ab] bias-corrected 95% confidence interval [CI], 0.004-0.01; proportion mediated=29.17%; note evidence of mediation was found for each of the other structural indices, see Table S6). The other environmental risk factors (e.g., drug offense exposure, years at residence), either did not show significant evidence of mediation (i.e., drug offense exposure) or the proportion mediated by the structural MRI variables was <10% (Table S6; Tables S7-10 for results with individual Desikan regions).
Discussion
Overall, the current study moves forward our understanding of the nature of associations between PLEs and environmental risk factors and makes first steps in understanding the possible neural mechanisms contributing to the association between both deprivation and urbanicity and PLEs. As expected, PLEs were associated with several specific types of environmental risk factors, namely measures of urbanicity (i.e., drug offense exposure, less perception of neighborhood safety), deprivation (including overall deprivation, rate of poverty, fewer years at residence), and lead exposure risk. These associations showed evidence of being more strongly associated with PLEs compared to internalizing or externalizing symptoms, especially for overall deprivation, poverty, and lead exposure risk (see Supplement 1 for additional analyses). Both PLEs and environmental risk factors were generally associated with reduced global brain structural metrics, including reduced volume and surface area. Further, there was some evidence consistent with brain volume partially mediating the association between deprivation (i.e., both rate of poverty and overall deprivation), perceptions of neighborhood safety, and lead exposure risk with PLEs.
In terms of associations between PLEs and specific environmental risk factors, consistent with the majority of previous work, we found that deprivation and poverty were associated with PLEs.42 In addition, fewer years at residence, which may be associated with greater residential mobility during childhood, was associated with increased PLEs.43 Consistent with previous research,5 we also found an association between increased exposure to crime, specifically drug offense exposure, and PLEs. This association may be attributable to several factors, including increased stress.44 Lastly, while the current study failed to replicate recent findings of an association between air pollution and increased PLEs,3 we found an association between PLEs and lead exposure risk. This finding is consistent with previous work,11 including a recent commentary on the previous pollution findings suggesting that such results might reflect lead exposure.10 Lead exposure risk may be a proxy for unsafe environmental conditions leading to stress and thereby leading to PLEs, or lead exposure may be associated with PLEs in a negative cascade, whereby increased exposure leads to increased inflammation, leading to a host of negative outcomes (e.g., cognitive impairments, distress) and then increased PLEs.11,45 Notably, associations with environmental risk factors were not specific to PLEs measuring suspiciousness (Table S3). The current study helps to clarify that associations between PLEs with these environmental risk factors remain even when accounting for covariates (Figure 1a). However, further work is needed to specify exact mechanistic pathways.
Importantly, the current study found evidence that associations between environmental risk factors and PLEs exist over and above other markers of psychopathology. First, only PLEs, and not internalizing/externalizing symptoms, were significantly associated with increased drug offense exposure and increased lead exposure risk, indicating perhaps a unique association with PLEs. Furthermore, there was greater evidence for an association between PLEs with deprivation and poverty than internalizing/externalizing symptoms. This may indicate that poverty is associated with psychopathology in a graded fashion, wherein greater deprivation is more strongly associated with more severe forms of psychopathology (e.g., increased PLEs). Overall, the current study provides evidence that PLEs are more strongly associated with several environmental risk factor metrics, although other environmental risk factors (e.g., fewer years at residence, perception of neighborhood safety) were not clearly more strongly associated with PLEs than internalizing/externalizing symptoms, and therefore may represent more general associations with psychopathology.
The current study was the first study to examine cross-sectional evidence that structural neural indices (e.g., volume, area, thickness) mediate associations between environmental risk factors and PLEs. Reduced volume (with the exception of hippocampal volume not being significantly associated with PLEs) and area metrics were all significantly associated with both increased PLEs and increased environmental risk factors. The current study also found PLEs were associated with several structural neural metrics (see Supplement 1 for regional structural MRI analyses) located in regions implicated in resting state functional connectivity networks associated with PLEs in our prior research using the ABCD Study®,46 perhaps indicating that subtle neural alterations in higher-order cognitive regions may be possible mechanisms underlying PLEs. Such findings are also consistent with prior research finding exposure to urbanicity, deprivation, and environmental toxins can all have a detrimental effect on the developing brain pre-adolescence,15 which may have implications for the critical pruning processes occurring during adolescence.22 Importantly, the results potentially indicate that global structural brain metrics, and especially volume, may partially mediate associations between deprivation (i.e., rate of poverty, overall deprivation, and to a lesser extent, years at residence) with PLEs. This supports the theory that one pathway by which deprivation is associated with increased psychosis risk is through neural impairments.47 Interestingly, we found evidence consistent with brain volume partially mediating the association between reduced perceptions of neighborhood safety and PLEs. Perception of neighborhood safety may be a proxy for perceived stress associated with living in that neighborhood, which would be entirely consistent with previous findings regarding interrelationships between chronic stress, volume, and psychosis.48 We also found evidence consistent with lead risk exposure mediating the associated between reduced cortical volume and increased PLEs (and perhaps particularly middle temporal volume, see Supplement 1), in line with previous research.17 Overall, these mediation findings are potentially consistent with the notion that underlying pathophysiology in conjunction with exposure to negative environments (e.g., urbanicity, poverty, toxins) may in turn further exacerbate neurobiological impairments, leading to increased risk for psychosis spectrum symptoms, but longitudinal work is needed to generate further evidence.
The current study has a number of limitations. The fact that all measures were collected at the ABCD Study® baseline assessment limits the conclusions that can be drawn from the current study. It is possible that structural abnormalities were present prior to any environmental risk exposure. Along these lines, we were not able to examine exposure prior to age 9. Future research should conduct longitudinal analyses to further clarify these associations. Next, we do not have information about the degree of each individual’s exposure to environmental risk factors (e.g., amount of exposure to drug offenses, exact degree of lead exposure, etc.), which again limits the conclusions that can be drawn from these analyses and require future research to examine dose-response associations. Third, self-report of PLEs were not followed up with a clinical interview, although research indicates that self-reported PLEs, even those not confirmed with clinical interview, are still clinically relevant and associated with higher rates of psychopathology.49 Fourth, associations with other self-reports were in the small-moderate range (βs≤.23), as is expected given the non-clinical sample and has been previously found with PLEs in the ABCD Study® sample 24. Fifth, a number of participants (n=1546) had missing data and therefore were not included in analyses. These participants significantly differed from the included participants on a number of measures, including demographics and PLEs. However, when including participants with partial data (e.g., data for some environmental risk factors but not others), results remained consistent. Lastly, we did not include race/ethnicity as a covariate, due to the all too frequent confounding of minority status with other relevant factors involved in the current study (e.g., deprivation, increased exposure to offenses, reduced access to resources).50 Future research should disentangle associations between these environmental risk factors and race/ethnicity.
The current research makes an important contribution to understanding the nature of associations between environmental risk factors and PLEs, including significant associations with exposure to drug offenses, perception of neighborhood safety, overall deprivation, poverty, number of years at residence, and lead exposure risk. Furthermore, there is some evidence that several of the associations, especially with poverty, overall deprivation, total drug offense exposure, and lead exposure risk were more strongly associated with PLEs than other psychopathology. Lastly, we found evidence consistent with the possibility that structural brain metrics partially mediated associations between both deprivation, neighborhood safety, and lead exposure risk with PLEs, which may have important clinical implications. Future clinical interventions and public health policies to reduce exposure to deprivation, correlates of urbanicity (i.e., reduced perception of neighborhood safety, increased exposure to drug offenses), and environmental toxins will be important for reducing negative effects of exposure on psychosis risk.
Supplementary Material
Acknowledgments
We are grateful to Dr. Wesley Thompson, PhD, of the University of California San Diego, for his valuable and constructive suggestions during the revision of this research work. We also thank the families participating in the Adolescent Brain and Cognitive Development study.
Data used in the preparation of this article were obtained from the Adolescent Brain Cognitive Development℠ study (ABCD Study®) (https://abcdstudy.org), held in the NIMH Data Archive (NDA). This is a multisite, longitudinal study designed to recruit more than 10,000 children age 9-10 and follow them over 10 years into early adulthood. The ABCD Study® is supported by the National Institutes of Health and additional federal partners under award numbers U01DA041022, U01DA041028, U01DA041048, U01DA041089, U01DA041106, U01DA041117, U01DA041120, U01DA041134, U01DA041148, U01DA041156, U01DA041174, U24DA041123, U24DA041147, U01DA041093, and U01DA041025. A full list of supporters is available at https://abcdstudy.org/federal-partners.html. A listing of participating sites and a complete listing of the study investigators can be found at https://abcdstudy.org/Consortium_Members.pdf. ABCD Study® consortium investigators designed and implemented the study and/or provided data but did not necessarily participate in analysis or writing of this report. This manuscript reflects the views of the authors and may not reflect the opinions or views of the NIH or ABCD Study® consortium investigators.
The ABCD Study® data repository grows and changes over time. The ABCD Study® data used in this report came from DOI 10.15154/1503209.
This work was supported by National Institute on Drug Abuse grant U01 DA041120 (DMB), National Institute of Health grants K23MH121792-01 and L30 MH120574-01 (NRK), R01-MH112612 and R34-MH110506 (JS), and the Behavioral Health Administration, Maryland Department of Health and Mental Hygiene, through the Center for Excellence on Early Intervention for Serious Mental Illness (OPASS 14-13717G/M00B4400214) (JS).
Footnotes
Conflict of Interest Disclosures
Authors report no conflicts of interest.
References
- 1.Faris REL, Dunham HW. Mental disorders in urban areas: an ecological study of schizophrenia and other psychoses. 1939. [Google Scholar]
- 2.Fan CC, McGrath JJ, Appadurai V, et al. Spatial fine-mapping for gene-by-environment effects identifies risk hot spots for schizophrenia. Nature communications. 2018;9(1):5296. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Newbury JB, Arseneault L, Beevers S, et al. Association of Air Pollution Exposure With Psychotic Experiences During Adolescence. JAMA Psychiatry. 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Poulton R, Caspi A, Moffitt TE, Cannon M, Murray R, Harrington H. Children's self-reported psychotic symptoms and adult schizophreniform disorder: a 15-year longitudinal study. Arch Gen Psychiatry. 2000;57(11):1053–1058. [DOI] [PubMed] [Google Scholar]
- 5.Newbury J, Arseneault L, Caspi A, Moffitt TE, Odgers CL, Fisher HL. Why Are Children in Urban Neighborhoods at Increased Risk for Psychotic Symptoms? Findings From a UK Longitudinal Cohort Study. Schizophr Bull. 2016;42(6):1372–1383. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Kirkbride JB, Jones PB, Ullrich S, Coid JW. Social deprivation, inequality, and the neighborhood-level incidence of psychotic syndromes in East London. Schizophr Bull. 2014;40(1):169–180. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Fett AJ, Lemmers-Jansen ILJ, Krabbendam L. Psychosis and urbanicity: a review of the recent literature from epidemiology to neurourbanism. Current opinion in psychiatry. 2019;32(3):232–241. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Khan A, Plana-Ripoll O, Antonsen S, et al. Environmental pollution is associated with increased risk of psychiatric disorders in the US and Denmark. PLoS Biol. 2019;17(8):e3000353. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Perera FP, Tang D, Wang S, et al. Prenatal polycyclic aromatic hydrocarbon (PAH) exposure and child behavior at age 6-7 years. Environmental health perspectives. 2012;120(6):921–926. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Fuller-Thomson E, Munro AP. Lead Exposure as a Confounding Factor in the Association of Air Pollution Exposure and Psychotic Experiences. JAMA Psychiatry. 2019. [DOI] [PubMed] [Google Scholar]
- 11.Reuben A, Schaefer JD, Moffitt TE, et al. Association of Childhood Lead Exposure With Adult Personality Traits and Lifelong Mental Health. JAMA Psychiatry. 2019;76(4):418–425. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Flatow J, Buckley P, Miller BJ. Meta-analysis of oxidative stress in schizophrenia. Biol Psychiatry. 2013;74(6):400–409. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Drakesmith M, Dutt A, Fonville L, et al. Volumetric, relaxometric and diffusometric correlates of psychotic experiences in a non-clinical sample of young adults. NeuroImage Clinical. 2016;12:550–558. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.van Erp TGM, Walton E, Hibar DP, et al. Cortical Brain Abnormalities in 4474 Individuals With Schizophrenia and 5098 Control Subjects via the Enhancing Neuro Imaging Genetics Through Meta Analysis (ENIGMA) Consortium. Biol Psychiatry. 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Johnson SB, Riis JL, Noble KG. State of the Art Review: Poverty and the Developing Brain. Pediatrics. 2016;137(4). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.de Prado Bert P, Mercader EMH, Pujol J, Sunyer J, Mortamais M. The Effects of Air Pollution on the Brain: a Review of Studies Interfacing Environmental Epidemiology and Neuroimaging. Current environmental health reports. 2018;5(3):351–364. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Marshall AT, Betts S, Kan EC, McConnell R, Lanphear BP, Sowell ER. Association of lead-exposure risk and family income with childhood brain outcomes. Nature Medicine. 2020;26(1):91–97. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Frissen A, van Os J, Peeters S, Gronenschild E, Marcelis M. Evidence that reduced gray matter volume in psychotic disorder is associated with exposure to environmental risk factors. Psychiatry research Neuroimaging. 2018;271:100–110. [DOI] [PubMed] [Google Scholar]
- 19.Frissen A, van Os J, Habets P, Gronenschild E, Marcelis M. No Evidence of Association between Childhood Urban Environment and Cortical Thinning in Psychotic Disorder. PLoS One. 2017;12(1):e0166651. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Rees N Danger in the air: How air pollution may be affecting the brain development of young children around the world. UNICEF. 2017:Available at https://www.unicef.org/environment/files/Danger_in_the_Air.pdf. [Google Scholar]
- 21.Slopen N, Fitzmaurice G, Williams DR, Gilman SE. Poverty, food insecurity, and the behavior for childhood internalizing and externalizing disorders. J Am Acad Child Adolesc Psychiatry. 2010;49(5):444–452. [DOI] [PubMed] [Google Scholar]
- 22.Giedd JN, Blumenthal J, Jeffries NO, et al. Brain development during childhood and adolescence: a longitudinal MRI study. Nature neuroscience. 1999;2(10):861–863. [DOI] [PubMed] [Google Scholar]
- 23.Barch DM, Albaugh MD, Avenevoli S, et al. Special Issue: Adolescent Brain and Cognitive Development Demographic, Physical and Mental Health Assessments in the Adolescent Brain and Cognitive Development Study: Rationale and Description. Developmental Cognitive Neuroscience.In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Karcher NR, Barch DM, Avenevoli S, et al. Assessment of the Prodromal Questionnaire-Brief Child Version for Measurement of Self-reported Psychoticlike Experiences in Childhood. JAMA Psychiatry. 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Achenbach TM. The Achenbach System of Emprically Based Assessment (ASEBA): Development, Findings, Theory and Applications. Burlington, VT: University of Vermont Research Center for Children, Youth, and Families; 2009. [Google Scholar]
- 26.Kaufman J, Birmaher B, Brent D, et al. Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime Version (K-SADS-PL): initial reliability and validity data. J Am Acad Child Adolesc Psychiatry. 1997;36(7):980–988. [DOI] [PubMed] [Google Scholar]
- 27.Rice JP, Reich T, Bucholz KK, et al. Comparison of direct interview and family history diagnoses of alcohol dependence. Alcoholism, clinical and experimental research. 1995;19(4):1018–1023. [DOI] [PubMed] [Google Scholar]
- 28.Investigation USDoJOoJPFBo. Uniform Crime Reporting Program Data: County-Level Detailed Arrest and Offense Data, United States, 2010. In: Inter-university Consortium for Political and Social Research [distributor]; 2014. [Google Scholar]
- 29.Kind AJ, Jencks S, Brock J, et al. Neighborhood socioeconomic disadvantage and 30-day rehospitalization: a retrospective cohort study. Annals of internal medicine. 2014;161(11):765–774. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Di Q, Kloog I, Koutrakis P, Lyapustin A, Wang Y, Schwartz J. Assessing PM2.5 Exposures with High Spatiotemporal Resolution across the Continental United States. Environmental science & technology. 2016;50(9):4712–4721. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Fischl B, Sereno MI, Dale AM. Cortical surface-based analysis. II: Inflation, flattening, and a surface-based coordinate system. Neuroimage. 1999;9(2):195–207. [DOI] [PubMed] [Google Scholar]
- 32.Chen CH, Gutierrez ED, Thompson W, et al. Hierarchical genetic organization of human cortical surface area. Science. 2012;335(6076):1634–1636. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Fischl B, Dale AM. Measuring the thickness of the human cerebral cortex from magnetic resonance images. Proceedings of the National Academy of Sciences of the United States of America. 2000;97(20):11050–11055. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Hagler DJ, Hatton SN, Makowski C, et al. Image processing and analysis methods for the Adolescent Brain Cognitive Development Study. bioRxiv. 2018:457739. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Segonne F, Dale AM, Busa E, et al. A hybrid approach to the skull stripping problem in MRI. Neuroimage. 2004;22(3):1060–1075. [DOI] [PubMed] [Google Scholar]
- 36.Desikan RS, Segonne F, Fischl B, et al. An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage. 2006;31(3):968–980. [DOI] [PubMed] [Google Scholar]
- 37.Nakahara S, Matsumoto M, van Erp TGM. Hippocampal subregion abnormalities in schizophrenia: A systematic review of structural and physiological imaging studies. Neuropsychopharmacology reports. 2018;38(4):156–166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Bates D, Mächler M, Bolker B, Walker S. Fitting linear mixed-effects models using lme4. arXiv preprint arXiv:14065823. 2014. [Google Scholar]
- 39.Hothorn T, Bretz F, Westfall P. Simultaneous inference in general parametric models. Biometrical journal. 2008;50(3):346–363. [DOI] [PubMed] [Google Scholar]
- 40.Meng X-L, Rosenthal R, Rubin DB. Comparing correlated correlation coefficients. Psychological bulletin. 1992;111(1):172. [Google Scholar]
- 41.Rosseel Y lavaan: An R Package for Structural Equation Modeling. J Stat Softw. 2012;48(2):1–36. [Google Scholar]
- 42.O'Donoghue B, Roche E, Lane A. Neighbourhood level social deprivation and the risk of psychotic disorders: a systematic review. Social psychiatry and psychiatric epidemiology. 2016;51(7):941–950. [DOI] [PubMed] [Google Scholar]
- 43.Paksarian D, Eaton WW, Mortensen PB, Pedersen CB. Childhood residential mobility, schizophrenia, and bipolar disorder: a population-based study in Denmark. Schizophr Bull. 2015;41(2):346–354. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.van Winkel R, Stefanis NC, Myin-Germeys I. Psychosocial stress and psychosis. A review of the neurobiological mechanisms and the evidence for gene-stress interaction. Schizophr Bull. 2008;34(6):1095–1105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Cabrera B, Bioque M, Penades R, et al. Cognition and psychopathology in first-episode psychosis: are they related to inflammation? Psychol Med. 2016;46(10):2133–2144. [DOI] [PubMed] [Google Scholar]
- 46.Karcher NR, O'Brien KJ, Kandala S, Barch DM. Resting-State Functional Connectivity and Psychotic-like Experiences in Childhood: Results From the Adolescent Brain Cognitive Development Study. Biol Psychiatry. 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.van Os J, Kenis G, Rutten BP. The environment and schizophrenia. Nature. 2010;468(7321):203–212. [DOI] [PubMed] [Google Scholar]
- 48.Holtzman CW, Trotman HD, Goulding SM, et al. Stress and neurodevelopmental processes in the emergence of psychosis. Neuroscience. 2013;249:172–191. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Rimvall MK, Gundersen S, Clemmensen L, et al. Evidence that self-reported psychotic experiences in children are clinically relevant. Schizophr Res. 2019;204:415–416. [DOI] [PubMed] [Google Scholar]
- 50.Browning CR, Calder CA, Ford JL, Boettner B, Smith AL, Haynie D. Understanding Racial Differences in Exposure to Violent Areas: Integrating Survey, Smartphone, and Administrative Data Resources. The Annals of the American Academy of Political and Social Science. 2017;669(1):41–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.