Abstract
Knowledge about early risk factors for major depressive disorder (MDD) is critical to identify those who are at high risk. A multivariable model to predict adolescents’ individual risk of future MDD has recently been developed however its performance in a UK sample was far from perfect. Given the potential role of air pollution in the aetiology of depression, we investigate whether including childhood exposure to air pollution as an additional predictor in the risk prediction model improves the identification of UK adolescents who are at greatest risk for developing MDD. We used data from the Environmental Risk (E-Risk) Longitudinal Twin Study, a nationally representative UK birth cohort of 2,232 children followed to age 18 with 93% retention. Annual exposure to four pollutants – nitrogen dioxide (NO2), nitrogen oxides (NOX), particulate matter <2.5μm (PM2.5) and <10μm (PM10) – were estimated at address-level when children were aged 10. MDD was assessed via interviews at age 18. The risk of developing MDD was elevated most for participants with the highest (top quartile) level of annual exposure to NOX (adjusted OR=1.43, 95% CI=0.96-2.13) and PM2.5 (adjusted OR=1.35, 95% CI=0.95-1.92). The separate inclusion of these ambient pollution estimates into the risk prediction model improved model specificity but reduced model sensitivity – resulting in minimal net improvement in model performance. Findings indicate a potential role for childhood ambient air pollution exposure in the development of adolescent MDD but suggest that inclusion of risk factors other than this may be important for improving the performance of the risk prediction model.
Keywords: Environment, Mental health, Net Reclassification Improvement, Prediction model, Psychopathology, Risk calculator
Introduction
Major depressive disorder (MDD) is a leading contributor to global disease burden (Gore et al., 2011; Whiteford et al., 2013). MDD has a pervasive negative impact and is especially debilitating because of its common onset during adolescence and young adulthood and its usual chronic course throughout life (Thapar et al., 2012). Knowledge about early risk factors for the development of MDD is therefore critical to inform effective and targeted early intervention to prevent its onset and a lifetime of suffering.
There is a well-established evidence base for a range of early risk factors for MDD including, but not limited to, female sex (Altemus et al., 2014), parental history of depression (Weissman et al., 1997), childhood maltreatment (Li et al., 2016), and negative relationships with family (Yap et al., 2014) and peers (Moore et al., 2017). Despite recognition that multiple risk factors will combine to increase the probability of developing MDD, studies typically investigate only one (or a few) risk factors at a time. Moreover, these factors increase the average risk for MDD, but much less is known about whether they accurately predict onset for a particular individual. Knowledge about the combination of factors that best predict which individuals will develop MDD is therefore lacking. Recent work has begun to address this gap: Rocha et al. (2021) have developed a multivariable prognostic model to calculate individualised risk in early adolescence of developing MDD at age-18. Risk prediction is an important part of medical care and public health and models are widely used within medical practice – for example to predict individual risk of cardiovascular disease (D’Agostino et al., 2008). However, their application to psychiatric outcomes remains quite novel despite the potential to aid clinical practice by identifying who to target with preventive interventions (i.e., those at high risk). Rocha and colleagues’ depression risk prediction model was developed using data from the Brazilian Pelotas birth cohort and internal validation demonstrated an acceptable level of discrimination (C-statistic=0.71) and good calibration. The model was then refitted and evaluated in the UK Environmental Risk (E-Risk) Longitudinal Twin Cohort – a nationally-representative 18-year study. Performance in this external sample was reduced (C-statistic=0.62) suggesting that other context-specific factors involved in the development of adolescent MDD might be needed to improve the model’s predictive ability in the UK.
One potential factor is exposure to ambient air pollution – harmful pollutants emitted by industries, households, and road traffic. Inhalation of air pollutants is known to cause systemic inflammation and oxidative stress (Kelly, 2003; Pope et al., 2016) and is a major cause of premature death and disease largely due to cardiovascular and respiratory conditions (World Health Organization [WHO], 2013). Inflammatory processes have been implicated in the aetiology of psychiatric disorders (Danese & Baldwin, 2017; Miller et al., 2019) suggesting a possible biological mechanism linking air pollution with mental health. Accumulating evidence does indeed indicate associations between air pollution exposure and poorer mental health (Bakolis et al., 2020; Klompmaker et al., 2019; Newbury et al., 2019; Oudin et al., 2016; Power et al., 2015) including depression (Braithwaite et al., 2019; Fan et al., 2020; Szyskowicz et al., 2009; Wang et al., 2020; Zhao et al., 2020).
Childhood is an important period for identifying early risk factors for depression. This may also be an especially vulnerable time for air pollution exposure because children’s lungs, brain, and immune system are still developing and because they may inhale higher doses of air pollutants than adults due to their faster breathing (WHO, 2018). Existing evidence of a link between children’s air pollution exposure and depressive symptoms is scarce and results are mixed. For example, in the US pre-natal air pollutant exposure has been associated with children’s symptoms of depression at age 6-7 years (Perera et al., 2012). On the other hand, a study of multiple European birth cohorts found no association between pre- or post-natal exposure to air pollution and children’s depressive symptoms between ages 7 and 11 years (Jorcano et al., 2019). Likewise, a US study of early-life air pollution exposure and pre-schoolers’ internalising behaviour found no association (Loftus et al., 2020).
Few studies have examined longitudinal links between childhood air pollution exposure and adolescent depression. A US study of traffic-related pollution exposure (Yolton et al., 2019) found exposure during early life and across childhood was associated with elevated symptoms of depression at age 12. Similarly, a pilot study using a London-based subsample of the UK E-Risk cohort (N=284) found modest but robust associations of childhood exposure to particulate matter with aerodynamic diameters of less than 2.5μm (PM2.5) and nitrogen dioxide (NO2) with MDD at age 18 (Roberts et al., 2019).
Drawing on this literature of associations between childhood air pollution exposure and MDD at the group-level, we aim to examine whether air pollution exposure can contribute to accurate individual-level MDD risk prediction. It is important not to just discard risk prediction models that have been developed but need improvement, but instead undertake model revision (Moons et al., 2009). Therefore, we will test whether the inclusion of childhood air pollution exposure as an additional predictor in Rocha and colleagues’ multivariable depression risk prediction model improves the identification adolescents who are at greatest risk for developing MDD in the UK. Utilising the full E-Risk cohort, we focus on age-10 exposure to four ambient air pollutants (NO2; NOX [nitrogen oxides]; PM2.5; and PM10 [particulate matter with aerodynamic diameters <10μm]). As a preliminary step, before including age-10 air pollution exposure into the risk prediction model, we test longitudinal associations between each pollutant and age-18 MDD.
Material and Methods
Study Cohort
Participants were members of the Environmental Risk (E-Risk) Longitudinal Twin Study, which tracks the development of a nationally representative birth cohort of 2,232 British twin children. Full details about the sample are reported elsewhere (Moffitt and E-Risk Study Team, 2002) and in Supplementary Material. Briefly, the E-Risk sample was constructed in 1999-2000 when 1,116 families (93% of those eligible) with same-sex 5-year-old twins participated in home-visit assessments. This sample comprised 56% monozygotic (MZ) and 44% dizygotic (DZ) twin pairs; sex was evenly distributed within zygosity (49% male). Families were recruited to represent the UK population of families with newborns in the 1990s, on the basis of residential location throughout England and Wales and mother’s age.
Follow-up home-visits were conducted when the participants were aged 7, 10, 12 and 18 years (participation rates were 98%, 96%, 96%, and 93%, respectively). There were 2,066 E-Risk participants who were assessed at age 18. The average age of the participants at the time of the assessment was 18.4 years (SD = 0.36); all interviews were conducted after the 18th birthday. There were no differences between those who did and did not take part at age 18 in terms of socioeconomic status (SES) assessed when the cohort was initially defined (χ2=0.86, p=0.65), age-5 IQ scores (t=0.98, p=0.33), age-5 behavioural (t=0.40, p=0.69) or emotional (t=0.41, p=0.68) problems, or childhood poly-victimisation (z=0.51, p=0.61). The cohort’s neighbourhoods represent the full range of socioeconomic conditions in Great Britain. Supplementary Figure 1 shows E-Risk families’ addresses are a near-perfect match to the deciles of the UK’s 2015 Lower-layer Super Output Area (LSOA) Index of Multiple Deprivation (IMD) which averages 1,500 residents; approximately 10% of the cohort fills each of IMD’s 10% bands.
The Joint South London and Maudsley and the Institute of Psychiatry Research Ethics Committee approved each phase of the study. Parents gave informed consent and twins gave assent between 5-12 years and then informed consent at age 18.
Measures
Age-10 ambient air pollution exposure.
Pollution exposure estimates were modelled for the year 2004 and linked to the latitude-longitude coordinates of participants’ residential addresses at age 10. Pollution exposure estimates were modelled at the local-scale using the Community Multiscale Air Quality (CMAQ-urban) Modelling System, a regional chemical transport model coupled to street-scale dispersion model. CMAQ-urban uses a new generation of road traffic emissions inventory in the UK to model air quality down to individual streets, providing hourly estimates of pollutants at 20×20-metre grid points throughout the UK (i.e., address level). Full details on the creation and validation of this model have been described previously (Beevers et al., 2012; Carslaw, 2011) and model evaluation information is provided in Supplementary Material.
Participants’ average exposure to four pollutants across 2004 was estimated: NO2 (regulated gaseous pollutant), NOX (regulated gaseous pollutant, composed of NO2 and nitric oxide), and PM2.5 and PM10 (regulated pollutants composed of inorganic aerosols, carbonaceous aerosols, and dusts). To index the worst concentrations of air pollution while retaining statistical power and ensuring parity between the measures, air pollutants were dichotomised at the top quartile of exposure in this sample. These quartile cut-offs in micrograms per cubic metre were: 33.1μg/m3 for NO2, 45.4μg/m3 for NOX, 13.3μg/m3 for PM2.5, and 18.6μg/m3 for PM10. Air pollutants were moderately to highly correlated (r=0.43-0.98; p<.001). We examined the individual associations of each pollutant with adolescent MDD as they may have differential effects. Just over 64% of E-Risk participants remained at the same home address in the years preceding 2004 (ages 5-10) suggesting reasonably consistent levels of air pollution throughout childhood, with the caveat that changes in pollution levels in the same area may occur over time.
Age-18 major depressive disorder.
We assessed current depressive symptoms using the Diagnostic Interview Schedule (Robins et al., 1995). The interview began with four screening questions to identify participants who had experienced at least 2 weeks of persistent low mood, anhedonia, or irritability in the past year, or those who had been prescribed medication for depression. Participants who answered positively to any of the screening items were asked a further 24 questions designed to map onto the nine symptom-criteria of a major depressive episode specified in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV; American Psychiatric Association, 1994). We created a scale based on the total number of symptom-criteria present. To identify participants with clinically significant depression we used a diagnostic cut-off based on the presence of at least five symptom-criteria plus interference in daily functioning. At age 18, 20% of E-Risk Study participants met these criteria for MDD.
Covariates.
We account for several key covariates in our preliminary analyses of associations between age-10 air pollution exposure and age-18 MDD. Due to potential associations with both air pollution and MDD we include measures of family SES, neighbourhood SES, and urbanicity. Smoking status is included to account for the possible confound with air pollution exposure. Sex, family psychiatric history, and age-10 depressive symptoms are included due to their association with MDD. All covariates are detailed in Supplementary Material.
Risk prediction model.
Full details of the depression risk prediction model are described by Rocha et al. (2021). In brief, the baseline model was developed using data from the 1993 Pelotas birth cohort with sociodemographic variables to predict individual risk of MDD at age 18 for adolescents with no previous depressive symptoms. The performance of the risk prediction model was then evaluated in the UK context using the E-Risk cohort. Participants were included if they were assessed for MDD at age 18 and had data for all predictors, but were excluded if they had an intelligence quotient <70 and/or there was evidence of an MDD diagnosis before age 12 (N=1,144). Model predictors included: biological sex (male/female); skin colour (white/non-white); any drug use (yes/no); school failure (yes/no); social isolation (yes/no); fight involvement (yes/no); ever ran away from home (yes/no); childhood maltreatment (none/probable/severe); and interactions of each of these with biological sex (for measurement details see Supplementary Table 2). Consistent with methodological recommendations (Moons et al., 2012), the Pelotas model intercept was corrected for the E-Risk cohort to take account of the differing MDD prevalence rates (model recalibration). Model coefficients were also optimised for the E-Risk cohort to account for differences in the strength of predictors. Performance statistics for this refitted model (as reported by Rocha et al., 2021) are shown in Table 2.
Table 2.
Performance measure | Depression risk prediction model | |||
---|---|---|---|---|
Original modela | Original model + NOXb | Original model + PM2.5c | Original model + PM2.5 or NOXd | |
C-statistic | 0.62 | 0.63 | 0.62 | 0.62 |
Intercept | 0.00 | 0.00 | 0.00 | 0.00 |
Slope | 1.20 | 1.00 | 1.00 | 1.00 |
R2 | 0.04 | 0.05 | 0.04 | 0.05 |
Brier Score | 0.14 | 0.14 | 0.14 | 0.14 |
Correctly identified cases (n) | 117/196 | 99/196 | 89/196 | 102/196 |
Correctly identified non-cases (n) | 495/915 | 616/915 | 654/915 | 610/915 |
Total correctly identified (n) | 612/1111 | 715/1111 | 743/1111 | 712/1111 |
NRI (95% CI) | - | 0.04 (−0.03 – 0.11) |
0.03 (−0.03 – 0.09) |
0.05 (−0.02 – 0.12) |
NRI+ (95% CI) | - | −0.09 (−0.15 – −0.03) |
−0.14 (−0.20 – −0.09) |
−0.08 (−0.14 – −0.01) |
NRI− (95% CI) | - | 0.13 (0.10 – 0.16) |
0.17 (0.15 – 0.20) |
0.13 (0.09 – 0.16) |
Note. CI=confidence interval; NOX=nitrogen oxides, PM2.5=particulate matter with aerodynamic diameters of less than 2.5μm. Prediction models include participants meeting inclusion/exclusion criteria with full data for original model predictors and pollution estimates, N=1,111.
Original model (without pollution predictors) reported by Rocha et al. (2021).
Prediction model with NOX (top quartile vs lower quartiles) included as an additional predictor.
Prediction model with PM2.5 (top quartile vs lower quartiles) included as an additional predictor.
Prediction model with an additional predictor included indicating top quartile exposure to either NOX or PM2.5 (versus lower quartiles of both). NRI=net reclassification improvement at the event rate compared to the original model. NRI+=net proportion of persons with major depressive disorder (MDD) at age 18 correctly reclassified; NRI−=net proportion of persons without MDD at age 18 correctly reclassified. Better model performance is indicated by higher values for C-statistic, R2, and NRI, positive values of NRI+ and NRI−, lower values for Brier score, values closer to 0 for calibration intercept, and values closer to 1 for calibration slope.
Statistical Analyses
As a preliminary step we conducted binary logistic regression using Stata (v.15) to test associations between each ambient air pollution exposure estimate at age 10 and MDD at age 18. Unadjusted models controlled only for the non-independence of twin observations using the Huber-White variance estimator (Williams, 2000). To test the robustness of these associations, we then (i) adjusted models for sex, neighbourhood SES, urbanicity, smoking status, family SES, family psychiatric history, and age-10 depressive symptoms, (ii) used pollution variables categorised at different thresholds to check the sensitivity of our highest quartile cut-off, (iii) used continuous pollution variables, and (iv) limited the analysis to the 63.6% of participants who did not move residence between ages 10 and 18 to keep air pollution exposure as consistent as possible over time.
Next, we utilised the depression risk prediction model developed and refitted to the E-Risk cohort by Rocha et al. (2021). Each ambient air pollutant that was associated with depression in our binary logistic regression models was separately included as an additional predictor in this risk prediction model. We checked for change in model performance first by comparing the new prediction models (with pollution included) to the original model (without pollution) on the following measures: (i) discrimination – the model’s ability to separate adolescents with and without MDD at age 18 – using the C-statistic; (ii) calibration – the agreement between the observed and predicted outcomes – by examining the calibration intercept and the slope, (iii) the Brier score – a combination of discrimination and calibration, and (iv) R2 – an overall goodness of fit measure (see Supplementary Material for explanation of these metrics).
To fully appreciate the contribution of the air pollution predictor in the context of the established model predictors we also used the Net Reclassification Improvement (NRI) method (Pencina et al., 2008). NRI quantifies the extent to which the new model correctly reclassifies participants as high or low risk (using a cut-off equivalent to the event-rate) (Pencina et al., 2017) compared to the original model. It is the sum of two components: NRI+ and NRI−. The NRI+ is the net proportion of ‘cases’ (participants with MDD) reclassified from low to high risk by the new model. This can be interpreted as the change in true positive rate – the change in the proportion of participants with MDD correctly identified as being likely to develop MDD. A positive NRI+ value therefore indicates model improvement (improved sensitivity) whereas a negative value indicates poorer performance than the original model. The NRI− is the net proportion of non-cases (participants without MDD) reclassified from high to low risk by the new model. This is the change in false positive rate – the change in the proportion of non-depressed participants incorrectly identified as likely to develop MDD. A positive NRI− value indicates improved model performance (improved specificity); a negative value indicates poorer performance than the original model. The overall NRI value represents the change in sensitivity accounting for the change in specificity. As recommended, we report the overall NRI as well as its components (Leening et al., 2014; Kerr et al., 2015). The risk prediction model and NRI analyses were conducted using R (v.3.5.1).
Results
Preliminary Analyses
The mean annualised concentration estimate in the E-Risk cohort in 2004 for NO2 was 26.04μg/m3 (SD=10.12, IQR=18.87–33.02), for NOX was 35.19μg/m3 (SD=17.44, IQR=22.71–45.35), for PM2.5 was 12.07μg/m3 (SD=2.18, IQR=11.27–13.30), and for PM10 was 17.47μg/m3 (SD=2.87, IQR=15.92–18.57). Mean levels of PM2.5 in the E-Risk cohort exceeded current WHO air quality guidelines (>10μg/m3; WHO, 2005). Figure 1 shows the mean annualised air pollution concentrations by quartile.
The prevalence of adolescent MDD according to estimated exposure to NO2, NOX, PM2.5, and PM10 at age 10 is shown in Figure 2. Children exposed to the highest (top quartile) annual levels of NOX and PM2.5 had higher odds of developing MDD at age 18 compared to those exposed to lower (bottom three quartiles) pollution levels (Table 1). Effect sizes were not attenuated after adjusting for covariates, though associations were no longer statistically significant. We found no significant associations between childhood exposure to NO2 or PM10 and adolescent MDD.
Table 1.
Pollutant | Model | OR | 95% CI |
---|---|---|---|
NO2 | Unadjusteda | 1.20 | 0.92 – 1.57 |
Adjustedb | 1.16 | 0.78 – 1.70 | |
NOX | Unadjusteda | 1.36* | 1.04 – 1.77 |
Adjustedb | 1.43 | 0.96 – 2.13 | |
PM2.5 | Unadjusteda | 1.32* | 1.01 – 1.73 |
Adjustedb | 1.35 | 0.95 – 1.92 | |
PM10 | Unadjusteda | 0.97 | 0.74 – 1.28 |
Adjustedb | 0.91 | 0.67 – 1.22 |
Note. CI=confidence interval; NO2=nitrogen dioxide; NOX=nitrogen oxides, OR=odds ratio; PM2.5=particulate matter with aerodynamic diameters of less than 2.5μm; PM10=particulate matter with aerodynamic diameters of less than 10μm.
p<.05.
Models control for the non-independence of twin observations (participants with full data included, N=1,988).
Models were adjusted for the confounding effects of sex, neighbourhood socio-economic status (SES), urbanicity, smoking status, family SES, family psychiatric history, age-10 depressive symptoms, and the non-independence of twin observations (participants with full data included, N=1,879).
Using pollution variables dichotomised at the mean or at WHO thresholds or using pollutants as continuous variables showed a comparable pattern of findings for associations of NOX with age-18 MDD (Supplementary Tables 3 and 4). For example, after adjusting for covariates, the odds of adolescent MDD were elevated whether NOX was dichotomised at the mean (OR=1.31; 95% CI=0.94-1.82) or at WHO threshold (OR=1.35; 95% CI=0.93-1.95) or continuous (OR=1.26, 95% CI=1.00-1.59). Odds for NO2, a component of NOX, were significantly elevated when dichotomised at the mean (adjusted OR=1.60; 95% CI=1.16-2.21). However, the associations for PM2.5 were attenuated when using these alternative cut-offs and the continuous variable (Supplementary Tables 3 and 4).
Repeating our analyses using participants who lived at the same address between ages 10 and 18 showed a similar pattern to our original findings (Supplementary Table 5), albeit with smaller adjusted effect sizes for NOX.
Does childhood ambient air pollution exposure improve the performance of the adolescent depression risk prediction model in the E-Risk cohort?
Given the observed association of age-10 NOX and PM2.5 exposure (when dichotomised at the top quartile) with age-18 MDD, we focused on these two ambient air pollutants. There was considerable overlap between exposure to the highest levels of NOX and PM2.5 – 82% of participants who were exposed to the highest levels of PM2.5 at age 10 were also exposed to the highest levels of NOX, and vice versa. Accordingly, we created a variable to reflect childhood exposure to either the highest (top quartile) annualised mean levels of NOX or PM2.5 versus lowest (bottom three quartiles) annualised mean levels of both.
Table 2 shows the performance measures of the depression risk prediction model (i) without pollution, (ii) with childhood NOX exposure included, (iii) with childhood PM2.5 exposure included, and (iv) with childhood exposure to NOX or PM2.5 included. The C-statistic, calibration slope and intercept, R2, and Brier score were largely unchanged by the inclusion of age-10 exposure to NOX or PM2.5 as an additional predictor. This suggests no notable change in model discrimination, calibration or overall fit – the model still predicted age-18 MDD better than chance and had a high degree of calibration.
NRI analyses (Table 2) revealed a higher total number of correct predictions by the models with air pollution exposure compared to the original model without air pollution. This improvement was due to the increased number of correctly identified non-cases (increased specificity). By contrast, the number of correctly identified MDD cases was lower among the models with air pollution included compared to the original model (reduced sensitivity). More specifically, NRI showed that the inclusion of NOX exposure as a predictor in the risk prediction model reduced the rate of false positives by 13%. That is, the proportion of participants that were incorrectly identified as being at high risk for adolescent MDD was substantially lower than the model without NOX. However, the inclusion of NOX also reduced the rate of true positives by 9%. That is, the proportion of participants that were correctly identified as being at high risk for MDD was lower than the original model. This trade-off therefore resulted in very minimal overall NRI when childhood exposure to NOX was included in the model (Table 2). A similar pattern was evident when PM2.5 exposure was included as a predictor (Table 2). The false positive rate was reduced by 17%; however, the true positive rate was also reduced by a similar amount (14%). Thus, the overall NRI was minimal. Finally, the inclusion of a predictor reflecting high levels of exposure to NOX or PM2.5 reduced the false positive rate by 13% and the true positive rate by 8% (Table 2). Therefore, the overall NRI was minimal. Together these results show a trade-off between improved model specificity and reduced model sensitivity when childhood air pollution exposure is added as a predictor of adolescent MDD.
Discussion
We found that the odds of developing MDD at age 18 were elevated for those with the highest level of annual exposure to NOX and PM2.5 at age 10. However, inclusion of these ambient air pollution exposure estimates into the risk prediction model produced minimal overall improvement since model specificity increased but model sensitivity decreased. We discuss these results, note study strengths and limitations, and propose directions for future research.
Our preliminary finding that exposure to high levels of NOX and PM2.5 at age 10 was associated with greater odds of MDD at age 18 (though adjusted ORs fell below conventional levels of statistical significance) adds to the growing literature suggesting a link between air pollution and psychopathology (Klompmaker et al., 2019). Indeed, our results are consistent with a recent meta-analysis of particulate matter exposure which showed increased odds of depression following long term (>6 months) exposure to PM2.5 but not PM10 (Braithwaite et al., 2019). We also advance the limited literature on childhood pollution exposure and adolescent depression though more studies are needed to draw firm conclusions. Consistent with previous pilot work in London (Roberts et al., 2019), we find elevated odds for MDD at age 18 among children exposed to higher levels of PM2.5 across additional rural and urban areas of England and Wales. Although we did not replicate the association with NO2 exposure found in the London sample, we did find that NOX – which is comprised of NO2 and nitric oxide – was associated with elevated rates of MDD at age 18. NOX is strongly associated with road traffic emissions (National Atmospheric Emissions Inventory, 2019) therefore our finding of a link with adolescent MDD is consistent with previous findings of an association between elemental carbon attributable to traffic exposure during childhood and depressive symptoms at age 12 (Yolton et al., 2019). Together these findings suggest a possible role for road traffic pollution exposure in the aetiology of adolescent depression. Potential mechanisms through which childhood air pollution exposure may increase risk for adolescent MDD include increased inflammation (Block & Calderón-Garcidueñas, 2009) and altered gene regulation (Reuben et al., 2020).
The current study also contributes to the growing interest in multivariable risk prediction models in psychiatry in general (Bernardini et al., 2017; Worthington et al., 2020; Meehan et al., 2020) and of depressive disorders in particular (Brathwaite et al., 2021; Hafeman et al., 2017; Rocha et al., 2021). Our findings showed that the depression risk prediction model developed and applied to the UK context by Rocha and colleagues (2021) was not substantially improved by the inclusion of estimated childhood exposure to high levels of NOX or PM2.5. This may be because the air pollution measures are a proxy for important factors – for example, deprivation – that are already partially captured by the existing model predictors. The inclusion of childhood NOX or PM2.5 exposure was better at improving model specificity than sensitivity. Ideally both would be optimised however, the context in which the model is used may influence whether greater sensitivity or specificity is preferred. For example, if resources are limited or the intervention has potential side effects, greater specificity may be preferable so that only those we are confident are at high risk are targeted.
Taken together, our results suggest that whilst childhood air pollution exposure may be a risk factor for adolescent MDD at the average – or group – level, it does not contribute much more to individual MDD risk prediction than that already provided by the existing model predictors. Risk factors other than childhood air pollution exposure may therefore be important for improving the prediction of MDD in UK adolescents. Including other socio-environmental factors known to be associated with depression such as stressful childhood experiences (Berg et al., 2016) or poor quality parental relationships (Yap et al., 2014) may improve the model’s ability to correctly identify adolescents at risk of MDD. Furthermore, given that MDD does not result from socio-environmental influences alone but also has a well-established genetic component (Sullivan et al., 2000), adding a genetic predictor (e.g. polygenic risk scores) to the model may be beneficial. The potential utility of prediction models that combine both genetic and socio-environmental risk has been recognised (Lewis & Vassos, 2020) but little explored. This is a key avenue for future research, though it is important to consider the availability of polygenic risk scores to clinicians and, thus, how feasible their inclusion would be in practice.
Strengths and Limitations
Strengths of the study include our use of high-resolution air pollution concentration exposure estimates combined with a richly phenotyped UK population cohort; prospective longitudinal assessment of depressive symptoms and covariates; clinical interview-based diagnosis of MDD; and utilisation of a published depression risk prediction model. However, we also acknowledge limitations. First, childhood air pollution exposure estimates were based only on children’s home addresses at one time-point. Exposure estimates that include other locations where children spend a large amount of time (e.g., school) or use of personal monitoring devices with data collected over several points across childhood would provide a more comprehensive account of childhood air pollution exposure. Pre-natal air pollution exposure may also be important to consider (Perera et al., 2012). Second, we were unable to control for all confounds in our preliminary analyses. We do not have measures of exposure to noise pollution from traffic or indoor air pollution therefore exposure to noise, open fires, and parental smoking were not controlled for. NOX and PM2.5 are both formed by motor vehicles thereby implicating air pollution but potentially also noise pollution from traffic. There has been some indication of cross-sectional associations between traffic noise and depression in adults though a recent meta-analysis concluded that the evidence was of very low quality (Dzhambov & Lercher, 2019). Nonetheless, we were unable to rule out the possibility that the association of NOX and PM2.5 with adolescent MDD may be due to a link with traffic noise. Third, the depression risk prediction model was developed to predict the risk of age-18 MDD among adolescents with no previous evidence of depression therefore participants with depressive symptoms prior to age 12 were excluded. The timing of our air pollution measurement – at age 10 – may therefore limit its ability to contribute to MDD risk prediction if this pollution exposure is already impacting depression by age 12. Lastly, our sample comprised twins and the extent to which findings from twins can be generalised to non-twins is sometimes questioned. However, the prevalence of mental health problems has been shown to be comparable for twins and non-twins (Kendler et al., 1995) and the E-Risk sample is representative of UK families in terms of geographic and socioeconomic distribution (Odgers et al., 2012).
Conclusion
Childhood exposure to NOX and PM2.5 was associated with the development of MDD in late adolescence suggesting a potential role for these ambient air pollutants in the aetiology of MDD. However, their inclusion in an existing risk prediction model to identify individual UK adolescents at high risk of MDD onset improved model specificity but not sensitivity. Future research should investigate whether the inclusion of genetic liability predictors alongside, and in interaction with, the socio-environmental predictors currently included in the depression risk prediction model improve its performance in the UK context.
Supplementary Material
Acknowledgements
We are grateful to the study mothers and fathers, the twins, and the twins' teachers and neighbors for their participation. Our thanks to Professor Avshalom Caspi, one of the founders of the E-Risk study, Dr Nutthida Kitwiroon for assistance with modelling the air pollution data, CACI, Inc., the UK Ministry of Justice, and to the E-Risk team for their dedication, hard work, and insights.
Role of the funding source
The E-Risk Study is funded by the Medical Research Council (UK MRC) [G1002190]. Additional support was provided by the U.S National Institute of Child Health and Human Development (NICHD) [HD077482]; the Jacobs Foundation; the King’s Together Multi and Interdisciplinary Research Scheme (Wellcome Trust Institutional Strategic Support Fund; grant 204823/Z/16/Z); MQ Transforming Mental Health Charity, Brighter Futures grant named “Identifying Depression Early in Adolescence” [MQBF/1 IDEA]; plus the UK MRC [MC_PC_MR/R019460/1] and the Academy of Medical Sciences [GCRFNG\100281] under the Global Challenges Research Fund. Helen L. Fisher was supported by the Economic and Social Research Council (ESRC) Centre for Society and Mental Health at King’s College London [ES/S012567/1]. Louise Arseneault is the Mental Health Leadership Fellow for the UK ESRC. Andrea Danese and Valeria Mondelli were part funded by the National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London. Joanne Newbury was supported by a Sir Henry Wellcome Trust Postdoctoral Fellowship from the Wellcome Trust [218632/Z/19/Z]. Christian Kieling has received support from Brazilian governmental research funding agencies (Conselho Nacional de Desenvolvimento Científico e Tecnológico [CNPq], Coordenação de Aperfeiçoamento de Pessoal de Nível Superior [CAPES], and Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul [Fapergs]) and is an Academy of Medical Sciences Newton Advanced Fellow. Brandon Kohrt has received support from the US National Institute of Mental Health (NIMH; grant R01MH120649). Aaron Reuben was supported by a grant from the National Institute of Environmental Health Sciences (NIEHS; grant F31ES029358). The views expressed are those of the authors and not necessarily those of the National Health Service, the NIHR, the Department of Health and Social Care, the ESRC or King’s College London. These funders played no role in study design; in the collection, analysis and interpretation of data; in the writing of the report; nor in the decision to submit this article for publication.
Footnotes
Declarations of interest: None
References
- Altemus M, Sarvaiya N, Epperson CN, 2014. Sex differences in anxiety and depression clinical perspectives. Front. Neuroendocrin 35, 320–330. 10.1016/j.yfrne.2014.05.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- American Psychiatric Association, 1994. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition. American Psychiatric Association, Washington. [Google Scholar]
- Bakolis I, Hammoud R, Stewart R, Beevers S, Dajnak D, MacCrimmon S, Broadbent M, Pritchard M, Shiode N, Fecht D, Gulliver J, Hotopf M, Hatch SL, Mudway IS, 2020. Mental health consequences of urban air pollution: prospective population-based longitudinal survey. Soc. Psychiatry Psychiatr. Epidemiol 10.1007/s00127-020-01966-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Beevers SD, Kitwiroon N, Williams ML, Carslaw DC, 2012. One way coupling of CMAQ and a road source dispersion model for fine scale air pollution predictions. Atmos. Environ 59, 47–58. 10.1016/j.atmosenv.2012.05.034 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Berg L, Rostila M, Hjern A, 2016. Parental death during childhood and depression in young adults–a national cohort study. J. Child. Psychol. Psychiatry 57, 1092–1098. 10.1111/jcpp.12560 [DOI] [PubMed] [Google Scholar]
- Bernardini F, Attademo L, Cleary SD, Luther C, Shim RS, Quartesan R, Compton MT, 2017. Risk prediction models in psychiatry: toward a new frontier for the prevention of mental illnesses. J. Clin. Psychiat 78, 572–583. 10.4088/jcp.15r10003 [DOI] [PubMed] [Google Scholar]
- Block ML, Calderón-Garcidueñas L 2009. Air pollution: mechanisms of neuroinflammation and CNS disease. Trends Neurosci. 32, 506–516. 10.1016/j.tins.2009.05.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Braithwaite I, Zhang S, Kirkbride JB, Osborn DP, Hayes JF, 2019. Air pollution (particulate matter) exposure and associations with depression, anxiety, bipolar, psychosis and suicide risk: a systematic review and meta-analysis. Environ. Health Perspect 127, 126002. 10.1289/EHP4595 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brathwaite R, Rocha TBM, Kieling C, Gautam K, Koirala S, Mondelli V, Kohrt B, Fisher HL, 2021. Predicting the risk of depression among adolescents in Nepal using a model developed in Brazil: the IDEA Project. Eur. Child Adoles. Psy 30(2), 213–223. 10.1007/s00787-020-01505-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carslaw DC, 2011. Defra urban model evaluation analysis–Phase 1. https://uk-air.defra.gov.uk/library/reports?report_id=654. Accessed July 15, 2020. [Google Scholar]
- D’Agostino RB, Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM, Kannel WB 2008. General cardiovascular risk profile for use in primary care: The Framingham Heart Study. Circulation. 117, 743–753. 10.1161/CIRCULATIONAHA.107.699579 [DOI] [PubMed] [Google Scholar]
- Danese A, Baldwin JR, 2017. Hidden wounds? Inflammatory links between childhood trauma and psychopathology. Annu. Rev. Psychol 68, 517–544. 10.1146/annurev-psych-010416-044208 [DOI] [PubMed] [Google Scholar]
- Dzhambov AM, Lercher P 2019. Road traffic noise exposure and depression/anxiety: an updated systematic review and meta-analysis. Int. J. Environ. Res. Public Health 16, 4134. 10.3390/ijerph16214134 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fan SJ, Heinrich J, Bloom MS, Zhao TY, Shi TX, Feng WR, Sun Y, Shen JC, Yang ZC, Yang BY, Dong GH, 2020. Ambient air pollution and depression: A systematic review with meta-analysis up to 2019. Sci. Total Environ 701, 134721. 10.1016/j.scitotenv.2019.134721 [DOI] [PubMed] [Google Scholar]
- Gore FM, Bloem PJN, Patton GC, Ferguson J, Joseph V, Coffey C, Sawyer SM, Mathers CD, 2011. Global burden of disease in young people aged 10–24 years: a systematic analysis. Lancet 377 (9783), 2093–2102. 10.1016/S0140-6736(11)60512-6 [DOI] [PubMed] [Google Scholar]
- Hafeman DM, Merranko J, Goldstein TR, Axelson D, Goldstein BI, Monk K, et al. , 2017. Assessment of a person-level risk calculator to predict new-onset bipolar spectrum disorder in youth at familial risk. JAMA Psychiatry 74, 841–847. 10.1001/jamapsychiatry.2017.1763 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jorcano A, Lubczyńska MJ, Pierotti L, Altug H, Ballester F, Cesaroni G, et al. , 2019. Prenatal and postnatal exposure to air pollution and emotional and aggressive symptoms in children from 8 European birth cohorts. Environ. Int 131, 104927. 10.1016/j.envint.2019.104927 [DOI] [PubMed] [Google Scholar]
- Kelly FJ, 2003. Oxidative stress: its role in air pollution and adverse health effects. Occup. Environ. Med 60, 612–616. 10.1136/oem.60.8.612 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kendler KS, Martin NG, Heath AC, Eaves LJ, 1995. Self-report psychiatric symptoms in twins and their nontwin relatives: Are twins different? Am. J. Med. Genet 60, 588–591. 10.1002/ajmg.1320600622 [DOI] [PubMed] [Google Scholar]
- Kerr KF, Wang Z, Janes H, McClelland RL, Psaty BM, Pepe MS, 2014. Net reclassification indices for evaluating risk-prediction instruments: a critical review. Epidemiology 25, 114–121. 10.1097/EDE.0000000000000018 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Klompmaker JO, Hoek G, Bloemsma LD, Wijga AH, van den Brink C, Brunekreef B, Lebret E, Gehring U, Janssen NAH, 2019. Associations of combined exposures to surrounding green, air pollution and traffic noise on mental health. Environ. Int 129, 525–537. 10.1016/j.envint.2019.05.040 [DOI] [PubMed] [Google Scholar]
- Leening MJ, Vedder MM, Witteman JC, Pencina MJ, Steyerberg EW, 2014. Net reclassification improvement: computation, interpretation, and controversies: a literature review and clinician's guide. Ann. Intern. Med 160, 122–131. 10.7326/M13-1522 [DOI] [PubMed] [Google Scholar]
- Lewis CM, Vassos E, 2020. Polygenic risk scores: from research tools to clinical instruments. Genome Med. 12, 1–11. 10.1186/s13073-020-00742-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Li M, D'arcy C, Meng X, 2016. Maltreatment in childhood substantially increases the risk of adult depression and anxiety in prospective cohort studies: systematic review, meta-analysis, and proportional attributable fractions. Psychol. Med 46, 717–730. 10.1017/S0033291715002743 [DOI] [PubMed] [Google Scholar]
- Loftus CT, Ni Y, Szpiro AA, Hazlehurst MF, Tylavsky FA, Bush NR, Sathyanarayana S, Carroll KN, Young M, Karr CJ, LeWinn KZ, 2020. Exposure to ambient air pollution and early childhood behavior: A longitudinal cohort study. Environ. Res 183, 109075. 10.1016/j.envres.2019.109075 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Meehan A, Latham RM, Arseneault L, Stahl D, Fisher HL, Danese A, 2020. Developing an individualized risk calculator for psychopathology among young people victimized during childhood: A population-representative cohort study. J. Affec. Disord 262, 90–98. 10.1016/j.jad.2019.10.034 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Miller AH, Maletic V, Raison CL 2019. Inflammation and its discontents: the role of cytokines in the pathophysiology of major depression. Biol. Psychiatry 65, 732–41. 10.1016/j.biopsych.2008.11.029 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moffitt TE, E-Risk Study Team, 2002. Teen-aged mothers in contemporary Britain. J. Child Psychol. Psychiatry 43, 727–742. 10.1111/1469-7610.00082 [DOI] [PubMed] [Google Scholar]
- Moons KG, Altman DG, Vergouwe Y, Royston P 2009. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ. 338, b606. 10.1136/bmj.b606 [DOI] [PubMed] [Google Scholar]
- Moons KG, Kengne AP, Grobbee DE, Royston P, Vergouwe Y, Altman DG, Woodward M, 2012. Risk prediction models: II. External validation, model updating, and impact assessment. Heart 98, 691–698. 10.1136/heartjnl-2011-301247 [DOI] [PubMed] [Google Scholar]
- Moore SE, Norman RE, Suetani S, Thomas HJ, Sly PD, Scott JG, 2017. Consequences of bullying victimization in childhood and adolescence: A systematic review and meta-analysis. World J. Psychiatr 7, 60–76. 10.5498/wjp.v7.i1.60 [DOI] [PMC free article] [PubMed] [Google Scholar]
- National Atmospheric Emissions Inventory, 2019. Air pollutant inventories for England, Scotland, Wales, and Northern Ireland: 1990–2017. https://uk-air.defra.gov.uk/assets/documents/reports/cat09/2004151035_DA_Air_Pollutant_Inventories_1990-2017_Issue_1.2.pdf (accessed 09 July 2020). [Google Scholar]
- Newbury JB, Arseneault L, Beevers S, Kitwiroon N, Roberts S, Pariante CM, Kelly FJ, Fisher HL, 2019. Association of air pollution exposure with psychotic experiences during adolescence. JAMA Psychiatry 76, 614–623. 10.1001/jamapsychiatry.2019.0056 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Odgers CL, Caspi A, Russell MA, Sampson RJ, Arsenault L, Moffitt TE, 2012. Supportive parenting mediates widening neighborhood socioeconomic disparities in children’s antisocial behavior from ages 5 to 12. Dev. Psychopathol 24, 705–721. 10.1017/S0954579412000326 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Oudin A, Bråbäck L, Åström DO, Strömgren M, Forsberg B, 2016. Association between neighbourhood air pollution concentrations and dispensed medication for psychiatric disorders in a large longitudinal cohort of Swedish children and adolescents. BMJ Open 6, e010004. 10.1136/bmjopen-2015-010004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pencina MJ, D’Agostino RB Sr., D’Agostino RB Jr., Vasan RS, 2008. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 27, 157–72. 10.1002/sim.2929 [DOI] [PubMed] [Google Scholar]
- Pencina MJ, Steyerberg EW, & D'Agostino RB Sr, 2017. Net reclassification index at event rate: properties and relationships. Stat. Med 36, 4455–4467. 10.1002/sim.7041 [DOI] [PubMed] [Google Scholar]
- Perera FP, Tang D, Wang S, Vishnevetsky J, Zhang B, Diaz D, Camann D, Rauh V, 2012. Prenatal polycyclic aromatic hydrocarbon (PAH) exposure and child behavior at age 6–7 years. Environ. Health Perspect 120, 921–926. 10.1289/ehp.1104315 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pope CA III, Bhatnagar A, McCracken JP, Abplanalp W, Conklin DJ, O’Toole T, 2016. Exposure to fine particulate air pollution is associated with endothelial injury and systemic inflammation. Circ. Res 119, 1204–1214. 10.1161/CIRCRESAHA.116.309279 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Power MC, Kioumourtzoglou MA, Hart JE, Okereke OI, Laden F, Weisskopf MG, 2015. The relation between past exposure to fine particulate air pollution and prevalent anxiety: observational cohort study. Br. Med. J 350, h1111. 10.1136/bmj.h1111 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Reuben A, Sugden K, Arseneault L, Corcoran DL, Danese A, Fisher HL, et al. , 2020. Association of neighborhood disadvantage in childhood with DNA methylation in young adulthood. JAMA Netw. Open, 3, e206095. doi: 10.1001/jamanetworkopen.2020.6095 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Robins L, Cottler L, Bucholz K, Compton W, 1995. Diagnostic Interview Schedule for DSM-IV. Washington University School of Medicine: St Louis, MO. [Google Scholar]
- Roberts S, Arseneault L, Barratt B, Beevers S, Danese A, Odgers CL, Moffitt TE, Reuben A, Kelly FJ, Fisher HL, 2019. Exploration of NO2 and PM2.5 air pollution and mental health problems using high-resolution data in London-based children from a UK longitudinal cohort study. Psychiatry Res. 272, 8–17. 10.1016/j.psychres.2018.12.050 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rocha TBM, Fisher HL, Caye A, Anselmi L, Arseneault L, Barros FC, et al. , 2021. Identifying adolescents at risk for depression: a prediction score performance in cohorts based in three different continents. J. Am. Acad. Child Adolesc. Psychiatry 60, 262–273. 10.1016/j.jaac.2019.12.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sullivan PF, Neale MC, Kendler KS, 2000. Genetic epidemiology of major depression: review and meta-analysis. Am. J. Psychiatry 157, 1552–1562. 10.1176/appi.ajp.157.10.1552 [DOI] [PubMed] [Google Scholar]
- Szyszkowicz M, Rowe BH, Colman I, 2009. Air pollution and daily emergency department visits for depression. Int. J. Occup. Med. Environ. Health 22, 355–362. 10.2478/v10001-009-0031-6 [DOI] [PubMed] [Google Scholar]
- Thapar A, Collishaw S, Pine DS, Thapar AK, 2012. Depression in adolescence. Lancet 379 (9820), 1056–1067. 10.1016/S0140-6736(11)60871-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang R, Yang B, Liu P, Zhang J, Liu Y, Yao Y, Lu Y, 2020. The longitudinal relationship between exposure to air pollution and depression in older adults. Int. J. Geriatr. Psychiatry 35, 610–616. 10.1002/gps.5277 [DOI] [PubMed] [Google Scholar]
- Weissman MM, Warner V, Wickramaratne P, Moreau D, Olfson M, 1997. Offspring of depressed parents: 10 years later. Arch. Gen. Psychiatry 54, 932–940. 10.1001/archpsyc.1997.01830220054009 [DOI] [PubMed] [Google Scholar]
- Whiteford HA, Degenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, Charlson FJ, Norman RE, Flaxman AD, Johns NJ, Burstein R, Murray CJL, Vos T, 2013. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet 382 (9904), 1575–1586. 10.1016/S0140-6736(13)61611-6 [DOI] [PubMed] [Google Scholar]
- Williams RL, 2000. A note on robust variance estimation for cluster-correlated data. Biometrics 56, 645–646. 10.1111/j.0006-341X.2000.00645.x [DOI] [PubMed] [Google Scholar]
- World Health Organization, 2005. Air quality guidelines for particulate matter, ozone, nitrogen dioxide and sulfur dioxide – global update 2005. Summary of risk assessment. https://www.who.int/phe/health_topics/outdoorair/outdoorair_aqg/en/ (accessed 03 August 2020). [Google Scholar]
- World Health Organization, 2013. Review of Evidence on Health Aspects of Air Pollution - REVIHAAP Project: Final Technical Report. WHO Regional Office for Europe, Copenhagen. [PubMed] [Google Scholar]
- World Health Organization, 2018. Air pollution and child health: prescribing clean air. https://www.who.int/ceh/publications/air-pollution-child-health/en/ (accessed 09 July 2020). [Google Scholar]
- Worthington MA, Walker EF, Addington J, Bearden CE, Cadenhead KS, Cornblatt BA, Mathalon DH, McGlashan TH, O’Perkins DO, Seidman LJ, Tsuang MT, Woods SW, & Cannon TD, 2020. Incorporating cortisol into the NAPLS2 individualized risk calculator for prediction of psychosis. Schizophr. Res. Online ahead of print 10.1016/j.schres.2020.09.022 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yap MBH, Pilkington PD, Ryan SM, Jorm AF, 2014. Parental factors associated with depression and anxiety in young people: A systematic review and meta-analysis. J. Affect. Disord 156, 8–23. 10.1016/j.jad.2013.11.007 [DOI] [PubMed] [Google Scholar]
- Yolton K, Khoury JC, Burkle J, LeMasters G, Cecil K, Ryan P, 2019. Lifetime exposure to traffic-related air pollution and symptoms of depression and anxiety at age 12 years. Environ. Res 173, 199–206. 10.1016/j.envres.2019.03.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zhao T, Tesch F, Markevych I, Baumbach C, Janßen C, Schmitt J, Romanos M, Nowak D, Heinrich J, 2020. Depression and anxiety with exposure to ozone and particulate matter: An epidemiological claims data analysis. Int. J. Hyg. Environ. Health 228, 113562. 10.1016/j.ijheh.2020.113562 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.