Uncovering heterogeneous associations of disaster-related traumatic experiences with subsequent mental health problems: A machine learning approach

Abstract

Aim:

Understanding the differential mental health effects of traumatic experiences is important to identify particularly vulnerable subpopulations. We examined the heterogeneous associations between disaster-related traumatic experiences and postdisaster mental health, using a novel machine learning–based causal inference approach.

Methods:

Data were from a prospective cohort study of Japanese older adults in an area severely affected by the 2011 Great East Japan Earthquake. The baseline survey was conducted 7 months before the disaster and the 2 follow-up surveys were conducted 2.5 and 5.5 years after (n = 1150 to n = 1644 depending on the exposure-outcome combinations). As disaster-related traumatic experiences, we assessed complete home loss and loss of loved ones. Using the generalized random forest algorithm, we estimated conditional average treatment effects (CATEs) of the disaster damages on postdisaster mental health outcomes to examine the heterogeneous associations by 51 predisaster characteristics of the individuals.

Results:

We found that, even when there was no population average association between disaster-related trauma and subsequent mental health outcomes, some subgroups experienced severe impacts. We also identified and compared characteristics of the most and least vulnerable groups (ie, top versus bottom deciles of the estimated CATEs). While there were some unique patterns specific to each exposure-outcome combination, the most vulnerable group tended to be from lower socioeconomic backgrounds with preexisting depressive symptoms for many exposure-outcome combinations.

Conclusions:

We found considerable heterogeneity in the association between disaster-related traumatic experiences and subsequent mental health problems.

Increased incidence of psychopathology, including depression and posttraumatic stress disorder (PTSD), has been well-documented in the wake of major disaster.^1–4 Disasters often involve 3 sets of exposures that increase the risk of psychopathology: (1) traumatic stressors, including personal injuries and loss of loved ones; (2) resource loss, such as property damage and job loss; and (3) ongoing adversities, including relocation, displacement, and social isolation.^5–7 At the same time, there is considerable heterogeneity in individual responses to traumatic exposure.^1,8 For example, it has been documented that even given the same traumatic experiences, only a fraction of exposed individuals develop postdisaster mental health problems, while the majority stays “resilient” and maintains their mental health or quickly returns to normal functioning after experiencing distress for a short period.^9,10 Although studying the population “average” effects of traumatic experiences on mental health problems has been common, such heterogeneity in the impacts of traumatic experiences has also been an active area of research.¹¹

Past studies have identified characteristics of individuals who are more likely to develop mental health problems after disasters, including but not limited to female sex, “predisaster” psychiatric conditions, lack of social support, and lower (SES).^2,6,12,13 A better understanding of the heterogeneous effects of disasters will likely help clinicians and policymakers to identify subpopulations of disaster survivors for whom postdisaster mental health interventions should be prioritized.¹⁴

Nonetheless, assessing the heterogenous effects of disasters on subsequent mental health has been hampered by 2 methodological challenges. First, there is a lack of information on the characteristics of disaster survivors predating disaster onset. Most studies of disaster survivors collect data on these characteristics retrospectively and are therefore subject to recall bias.¹ Second, most studies assessing heterogeneity rely on a deductive approach, where the researchers select a limited set of predictors as sources of heterogeneity and statistically test interactions, each factor at a time.¹⁵ While the deductive approach is useful for testing substantive theory, this approach is prone to miss patterns embedded in the data. For example, although multiple factors can interact with each other and modify the impacts of disasters simultaneously, the deductive approach will not detect such a complex heterogeneity should the researcher not explicitly search for them.

The present study aimed to estimate the heterogeneous effects of traumatic experiences on subsequent mental health problems of survivors using a recently developed machine learning (ML) algorithm. This algorithm enabled us to flexibly and inductively assess effect heterogeneity.^16,17 Although several studies have applied ML algorithms to predict PTSD, our study is distinct because these studies focused on pure prediction of PTSD rather than estimating effect heterogeneity, which is our focus.^18–21 We leveraged a unique natural experiment setting stemming from the 2011 Great East Japan Earthquake and Tsunami, where a longitudinal cohort study of Japanese older adults established 7 months before the earthquake onset offered an opportunity to collect rich predisaster data of the survivors.²²

Methods

Data

The Iwanuma Study is part of a nationwide cohort study of Japanese older adults, called JAGES (Japan Gerontological Evaluation Study), which was established in 2010.²³ Iwanuma city was one of the field sites of JAGES located in Miyagi Prefecture (population 44 187 in 2010), located ≈80 km (128 miles) from the epicenter of the 2011 Great East Japan Earthquake. Importantly, the baseline survey of the Iwanuma Study was conducted by mail in August 2010, 7 months before the disaster onset. JAGES conducted a census of all residents 65 years or older in Iwanuma city using the official residential register (n = 8576) and obtained valid responses from a total of 4957 residents (response rate = 58%).

The Great East Japan Earthquake (the Richter scale: 9.0) occurred on March 11, 2011. The earthquake and the subsequent tsunami caused devastating damage to coastal areas of northeastern Japan, including the city of Iwanuma. In this city, the tsunami killed 180 residents, damaged 5542 houses, and inundated 48% of the land area in Iwanuma (Figure S1).²⁴

There were 2 follow-up mail surveys targeting disaster survivors in Iwanuma. The first follow-up survey was conducted in October 2013, ≈2.5 years after the disaster. JAGES identified the home addresses of 99.7% of the original sample. Of the eligible survivors who were healthy enough to participate and still lived in Iwanuma at the time of follow-up (n = 4380; 88.4% of the disaster survivors), we obtained valid responses from 3567 individuals (response rate = 81.4%). In November 2016 (5.5 years after the disaster), we conducted the second follow-up survey and recontacted all respondents of the previous wave. Of the 3323 eligible study participants (follow-up rate = 93.2%), we obtained valid responses from 2781 individuals (response rate = 83.7%). We excluded observations with missing data and obtained the final analytic samples (sample size ranged from n = 1559 to n = 1644 for the outcomes in 2013 and from n = 1150 to n = 1282 for the outcomes in 2016 (see Figure S2).

All procedures involving human participants were approved by the ethics committees of the Harvard T. H. Chan School of Public Health (P23143-101). Written informed consent was obtained from all participants.

Measurement

Outcome

Our outcome of interest was depressive symptoms and posttraumatic stress symptoms (PTSS) assessed in 2013 and 2016. We used the Japanese short version of the Geriatric Depression Scale (GDS), which was originally developed in English, to assess depressive symptoms.^25,26 The scale consists of 15 binary items asking about one’s depressive symptoms, and the overall summed score could range from 0 to 15, where higher scores indicate increasing depressive symptoms. The GDS, with a cutoff of ≥5 to indicate major depression, has been used as a screening tool for depression among older adults and validated (a sensitivity of 92% and a specificity of 81%) against the Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders, Third Edition, Revised.²⁷ We measured PTSS using the Screening Questionnaire for Disaster Mental Health (SQD), which was developed and validated in an older population of Japanese survivors in the aftermath of the 1955 Great Hanshin-Awaji Earthquake.²⁸ The scale has 9 binary items, and the total score could range from 0 to 9, and a score of ≥6 has been used as a cutoff to indicate possible PTSD. The SQD has been validated against the Clinician Administered PTSD Scale (receiver operating characteristic area under the curve = 0.91) and the Impact of Event Scale Revised (receiver operating characteristic area under the curve = 0.95).²⁸ We used both GDS score and SQD score as continuous outcomes.

Exposures

We assessed 2 types of traumatic events stemming from the disaster (hereafter, “disaster damage”): home loss and loss of loved ones to the disaster. Housing damage was reported in the 2013 follow-up survey and externally assessed by property inspectors and classified into 5 levels: (1) no damage, (2) partial damage, (3) minor damage, (4) major damage, and (5) complete destruction.²⁹ We created a binary variable representing home loss (1 = “complete destruction” and 0 = “no damage/less severe damage”) because previous evidence has documented that complete home loss was a unique predictor of deteriorated mental health and other health outcomes after the disaster.^12,30 Respondents also reported disaster-related loss of loved ones (close friends and/or relatives) in the 2013 wave (1 = yes, 0 = no).

Covariates

We selected 51 predisaster factors from the baseline (2010) survey wave, including 4 demographic characteristics, 3 SES measures, 24 health conditions, 14 psychosocial factors, and 6 behavioral factors (see Table S1 for the full list of the selected variables). We identified these factors because they were likely to operate as confounders (ie, predictors of postdisaster psychopathology distributed differently among the levels of the traumatic experiences) and/or effect modifiers (ie, associations between traumatic experiences and postdisaster psychopathology differ by the levels of these factors). We chose the predisaster factors not via a data-driven approach but rather based on the subject-matter knowledge and the literature on postdisaster psychopathology.¹

Statistical analysis

To examine heterogeneous effects, we estimated conditional average treatment effects (CATEs) of the disaster damages on depressive symptoms and PTSS. Formally, CATE is the effect of an exposure conditional on a set of covariates;

$$E[Y_{a=1}-Y_{a=0}|L]$$

where Y_a is the potential outcome Y under the binary treatment A = a and L is a set of covariates (confounders and/or effect modifiers). In other words, CATE is the average effect of the exposure among individuals with identical covariate values. The interpretation of CATE for each group of individuals with identical covariate values is the difference in mean mental health outcome that would have been counterfactually observed had everyone in the group been exposed (versus had nobody in the same group been exposed).

We applied a novel ML approach called the generalized random forest (GRF) algorithm to estimate CATEs of the disaster damages on depressive symptoms and PTSS.¹⁶ GRF extends the random forest algorithm, a common nonparametric algorithm designed to predict conditional expectation E[Y|X] conditional on a set of covariates X.³¹ Random forest grows many regression trees, which is a type of decision tree, by partitioning bootstrapped samples based on the values of covariates and compute a weighted average of outcome in each leaf (ie, subsamples defined by the same combinations of covariate values) of a tree. GRF targets and assesses the contrast in the average outcome between the exposed versus unexposed individuals in each leaf (ie, CATEs), rather than predicting the average outcome itself. Because the estimated mean outcomes are conditional on observed covariates, causal identification of CATEs from observational data is possible if, although unverifiable, the covariate set L used in the algorithm includes a valid adjustment set. Such a set contains variables that suffice to control for all confounding and selection bias.³² Estimating CATEs via GRF is advantageous in that it does not suffer from the model specification assumptions of a deductively and parametrically specified statistical model.

We conducted the following analyses. First, we estimated population average treatment effects (ATEs) of the disaster damages on the GDS and SQD scores in 2013 and 2016 using the doubly robust targeted maximum likelihood estimation.^33,34 This approach estimates both the exposure and the outcome mechanisms and yields unbiased estimates for ATEs if either of the 2 mechanisms is consistently estimated. Hence, the approach is more robust to model misspecification. ATEs quantify the difference in the mean outcomes had everyone in the population (ie, older adults in Iwanuma) been exposed to the damages versus had nobody been exposed. Second, we used GRF and estimated CATE for each individual in the sample to assess the distributions of the CATEs. Third, to compare the most salient heterogeneity, we compared characteristics of the top 10% and the bottom 10% of the CATE distributions. The top 10% (ie, those with the largest CATEs) were labeled as the “vulnerable” group, and the bottom 10% (ie, those with the smallest CATEs) were labeled as the “resilient” group. We compared sociodemographic variables and key sources of heterogeneity identified by the variable importance feature of GRF. Last, as a sensitivity analysis, we dichotomized the outcomes (≥5 points for the GDS scores and ≥6 points for the SQD scores as described above) and performed GRF to assess distributions of CATEs in the prevalence difference scale (ie, Pr[D = 1_a=1|L] − Pr[D = 1_a=0|L], where D = 1 denotes the presence of the binary outcomes).

Results

Table 1 shows baseline characteristics of the study participants and their mental health outcomes in the follow-up waves by levels of the disaster damages. Those who experienced home loss (versus no home loss or less severe damage) reported higher GDS and SQD scores both in 2013 (2.5 years after the onset) and 2016 (5.5 years after). While participants with loss of loved ones (versus no loss) reported higher SQD scores in 2013 and 2016, mean GDS scores were comparable among groups. Those who experienced home loss tended to have lower SES (shorter years of education and lower household income) than the unexposed individuals, but we did not find such difference in SES for loss of loved ones.

Table 1.

Mental health outcomes in 2013 and 2016 and baseline sociodemographic characteristics of study participants in 2010 by levels of disaster damage

		Home loss		Loss of love ones
Baseline characteristic	Overall (N = 3567)	Yes (n = 159)	No (n = 3307)	Yes (n = 1329)	No (n = 2238)
GDS in 2013, mean (SD)†	3.78 (3.38)	5.28 (3.44)	3.68 (3.34)	3.77 (3.39)	3.78 (3.37)
PTSS in 2013, mean (SD)‡	2.31 (2.27)	3.89 (2.53)	2.24 (2.23)	2.80 (2.45)	2.01 (2.10)
GDS in 2016, mean (SD)†	3.59 (3.47)	4.21 (3.58)	3.52 (3.44)	3.48 (3.47)	3.66 (3.46)
PTSS in 2016, mean (SD)‡	2.25 (2.17)	3.77 (2.50)	2.19 (2.14)	2.51 (2.31)	2.08 (2.05)
Age, mean (SD)	73.6 (6.29)	73.9 (6.60)	73.6 (6.28)	73.1 (6.12)	74.0 (6.36)
Sex, n (%)
Men	2015 (56.5)	97 (61.0)	1850 (55.9)	779 (58.6)	1236 (55.2)
Women	1552 (43.5)	62 (39.0)	1457 (44.1)	550 (41.4)	1002 (44.8)
Marital status, n (%)
Married	2460 (69.0)	103 (64.8)	2302 (69.6)	926 (69.7)	1534 (68.5)
Widowed	833 (23.4)	35 (22.0)	771 (23.3)	306 (23.0)	527 (23.5)
Divorced	87 (2.4)	2(1.3)	81 (2.4)	30 (2.3)	57 (2.5)
Single	43 (1.2)	0 (0)	40 (1.2)	12 (0.9)	31 (1.4)
Other	21 (0.6)	2(1.3)	17 (0.5)	10 (0.8)	11 (0.5%)
Living alone, n (%)
No	3148 (88.3)	144 (90.6)	2921 (88.3%)	1204 (90.6%)	1944 (86.9%)
Yes	323 (9.1)	5(3.1)	310 (9.4%)	96 (7.2%)	227 (10.1%)
Education, n (%)
<6 y	47 (1.3)	1 (0.6)	46 (1.4)	17(1.3)	30 (1.3)
6–9 y	1183 (33.2)	98 (61.6)	1038 (31.4)	504 (37.9)	679 (30.3)
10–12y	1486 (41.7)	38 (23.9)	1408 (42.6)	518(39.0)	968 (43.3)
≥13 y	713 (20.0)	8 (5.0)	696 (21.0)	245 (18.4)	468 (20.9)
Other	31 (0.9)	2(1.3)	28 (0.8)	13 (1.0)	18 (0.8)
Household income [10,000 yen], mean (SD)§	230 (142)	168 (124)	234 (142)	226 (142)	233 (142)
Self-rated health, n (%)
Bad	423 (11.9)	25 (15.7)	389 (11.8)	160(12.0)	263 (11.8)
Not good	2420 (67.8)	96 (60.4)	2260 (68.3)	908 (68.3)	1512 (67.6)
Good	537 (15.1)	19 (11.9)	502 (15.2)	190(14.3)	347 (15.5)
Very good	121 (3.4)	13 (8.2)	100 (3.0)	44 (3.3)	77 (3.4)
Body mass index, mean (SD)	23.5 (3.12)	23.9 (2.85)	23.5 (3.10)	23.6 (3.08)	23.5 (3.15)
ADL, n (%)
No help needed	3354 (94.0)	137 (86.2)	3127 (94.6)	1259 (94.7)	2095 (93.6)
Partially needed	118 (3.3)	8 (5.0)	106 (3.2)	40 (3.0)	78 (3.5)
Completely needed	21 (0.6)	4 (2.5)	14 (0.4)	4 (0.3)	17 (0.8)
GDS in 2010, mean (SD)†	3.66 (3.44)	4.09 (3.44)	3.60 (3.41)	3.64 (3.45)	3.68 (3.44)
IADL, mean (SD)‡	11.6 (2.30)	11.1 (2.81)	11.6 (2.25)	11.7 (2.22)	11.5 (2.34)
Treatment for major diseases, n (%)	1.54 (1.41)	1.41 (1.26)	1.53 (1.41)	1.52 (1.38)	1.55 (1.44)

^†We used the Geriatric Depression Scale (GDS; range, 0–15 points) to assess depressive symptoms.

^‡We used the Screening Questionnaire for Disaster Mental Health (range, 0–9 points) to assess posttraumatic stress symptoms (PTSS).

^§Annual household income was divided by the square root of the number of household members to account for household size.

ADL = activities of daily living; IADL, instrumental activities of daily living.

Figure 1 shows estimated ATEs of the disaster damages on postdisaster mental health problems (see Table S2 for the exact values). For depressive symptoms, home loss was associated with an absolute increase in GDS scores in 2013 (estimate, 1.39; 95% CI, 0.58–2.19) but not in 2016 (estimate, 0.25; 95% CI, −0.53 to 1.04), after adjusting for the 51 predisaster factors. There was no strong evidence that loss of loved ones was on average associated with GDS scores (estimate, 0.06 [95% CI, −0.19 to 0.30] for 2013 and −0.04 [95% CI, −0.35 to 0.26] for 2016). As for PTSS, both home loss and loss of loved ones were associated with increased SQD scores in 2013 (estimate, 1.42 [95% CI, 0.72–2.13] for home loss and 0.66 [95% CI, 0.45–0.87] for loss of loved ones). In 2016, home loss maintained the association with increased SQD scores (estimate, 1.53; 95% CI, 0.87–2.20), while the association between the loss of loved ones and SQD scores attenuated (estimate, 0.21; 95% CI, −0.01 to 0.43).

Fig.1.

Estimated population average treatment effects of disaster damages on depressive symptoms and posttraumatic stress symptoms (PTSS) in 2013 and 2016. Population average effects (ie, average treatment effects) of the exposures were estimated via doubly robust targeted maximum likelihood estimation. We used Geriatric Depression Scale (range, 0–15 points; higher scores indicate more depressive symptoms) to assess depressive symptoms. We used the Screening Questionnaire for Disaster Mental Health (range, 0–9 points; higher scores indicate more PTSS) to assess PTSS.

Figure 2 (see Table S2 for the summary statistics) shows the distributions of the CATEs estimated via GRF. We found that there was heterogeneity in the CATEs among individuals. For example, while the estimated average treatment effects of loss of loved ones on depressive symptoms in 2013 was not statistically significant, the corresponding CATEs were estimated to be positive for approximately half of the analytic sample, with the estimates ranging from a 0.52-point decrease to a 0.74-point increase in GDS scores (SD, 0.15). Likewise, the sensitivity analysis using binary outcomes showed heterogeneous CATEs among individuals (Figure S3). Based on the variable importance ranking in GRF, we identified predisaster characteristics of the survivors as key sources of heterogeneity, including body mass index, household income, self-rated health, sense of coherence, GDS score, and age (Figure S4)

Fig.2.

Distributions of estimated conditional average treatment effects of disaster damages on depressive symptoms/posttraumatic stress symptoms (PTSS) in (a) 2013 and (b) 2016. We used the Geriatric Depression Scale (range, 0–15 points; higher scores indicate more depressive symptoms) to assess depressive symptoms and the Screening Questionnaire for Disaster Mental Health (range, 0–9 points; higher scores indicate more PTSS) to assess PTSS. The heterogeneous effects (ie, conditional average treatment effects) were estimated using the generalized random forest algorithm.

In 2013, compared with the resilient group (bottom decile of CATE estimates), those in the top decile of vulnerability for mental health outcomes following the experience of home loss tended to have higher household incomes, lower GDS scores, and a higher sense of coherence before the disaster (Table 2). On the other hand, the most vulnerable groups following the loss of loved ones tended to have lower educational attainment, lower household income, and higher GDS scores at baseline compared with the resilient group.

Table 2.

Sociodemographic characteristics and key effect modifiers among individuals at the top 10% versus the bottom 10% of the estimated conditional average treatment effect of disaster damages on depressive symptoms and PTSS in 2013^†,‡

Outcome	Depressive symptoms§				PTSS¶
Exposure	Home loss (n = 1559)		Loss of loved ones (n = 1583)		Home loss (n = 1630)		Loss of loved ones (n = 1644)
Group	Vulnerable (n = 156)	Resilient (n = 156)	Vulnerable (n = 159)	Resilient (n = 159)	Vulnerable (n = 163)	Resilient (n = 163)	Vulnerable (n = 165)	Resilient (n = 165)
CATE estimates, mean (SD)††	1.75 (0.16)	0.98 (0.19)	0.37 (0.10)	−0.18 (0.06)	1.70 (0.06)	0.81 (0.13)	0.87 (0.05)	0.44 (0.05)
Age, mean (SD), y	73.0 (6.40)	74.0 (6.52)	73.1 (6.72)	72.9 (5.81)	70.6 (5.14)	76.3 (7.47)	73.2 (5.99)	70.8 (4.71)
Sex, n (%)
Men	66 (42.3)	87 (55.8)	64 (40.3)	78 (49.1)	76 (46.6)	103 (63.2)	88 (53.3)	92 (55.8)
Women	90 (57.7)	69 (44.2)	95 (59.7)	81 (50.9)	87 (53.4)	60 (36.8)	77 (46.7)	73 (44.2)
Marital status, n (%)
Married	125 (80.1)	97 (62.2)	112(70.4)	132 (83.0)	147 (90.2)	88 (54.0)	111 (67.3)	133 (80.6)
Widowed	31 (19.9)	52 (33.3)	42 (26.4)	18 (11.3)	12 (7.4)	65 (39.9)	47 (28.5)	23 (13.9)
Divorced	0(0)	4 (2.6)	2(1.3)	3(1.9)	1 (0.6)	3(1.8)	5 (3.0)	3(1.8)
Single	0 (0)	3 (1.9)	2(1.3)	6 (3.8)	2(1.2)	7 (4.3)	1 (0.6)	6 (3.6)
Other	0 (0)	0(0)	1 (0.6)	0 (0)	1 (0.6)	0(0)	1 (0.6)	0(0)
Living alone, n (%)
Not living alone	146 (93.6)	134 (85.9)	147 (92.5)	151 (95.0)	158 (96.9)	143 (87.7)	147 (89.1)	155 (93.9)
Living alone	10 (6.4)	22 (14.1)	12 (7.5)	8 (5.0)	5(3.1)	20(12.3)	18 (10.9)	10 (6.1)
Education, n (%)
<6 y	0(0)	5 (3.2)	6 (3.8)	0(0)	1 (0.6)	10(6.1)	2(1.2)	1 (0.6)
6–9 y	40 (25.6)	41 (26.3)	85 (53.5)	22 (13.8)	11 (6.7)	75 (46.0)	50 (30.3)	22 (13.3)
10–12y	72 (46.2)	72 (46.2)	56 (35.2)	74 (46.5)	88 (54.0)	51 (31.3)	84 (50.9)	96 (58.2)
≥13 yr	44 (28.2)	38 (24.4)	12 (7.5)	63 (39.6)	63 (38.7)	27 (16.6)	28 (17.0)	46 (27.9)
Other	0 (0)	0(0)	0(0)	0(0)	0 (0)	0 (0)	1 (0.6)	0(0)
Job, n (%)
Working	40 (25.6)	20 (12.8)	33 (20.8)	23 (14.5)	35 (21.5)	24 (14.7)	41 (24.8)	26 (15.8)
Retired	93 (59.6)	110 (70.5)	101 (63.5)	100 (62.9)	102 (62.6)	99 (60.7)	96 (58.2)	116 (70.3)
Never worked	23 (14.7)	26 (16.7)	25 (15.7)	36 (22.6)	26 (16.0)	40 (24.5)	28 (17.0)	23 (13.9)
Household income [10 000 yen], mean (SD)‡‡	329 (249)	225 (117)	186 (138)	263 (124)	282 (108)	200 (175)	155 (97.2)	294 (125)
Body mass index, mean (SD)	23.5 (2.66)	22.0 (3.98)	23.7 (3.11)	23.6 (2.82)	22.7 (1.99)	23.5 (3.68)	24.0 (2.77)	23.1 (3.05)
Sense of coherence, mean (SD)	22.3 (3.54)	19.6 (4.78)	19.2 (4.34)	22.9 (3.52)	26.8 (2.13)	17.5 (3.39)	22.2 (3.84)	22.3 (4.20)
Baseline GDS score, mean (SD)	3.04 (2.69)	5.70 (4.45)	6.63 (4.21)	2.96 (2.78)	0.87 (1.16)	7.79 (3.78)	3.41 (3.25)	2.84 (3.06)
Self-rated health, n (%)
Bad	21 (13.5)	7 (4.5)	19(11.9)	17 (10.7)	34 (20.9)	10(6.1)	4 (2.4)	42 (25.5)
Not good	116 (74.4)	76 (48.7)	91 (57.2)	95 (59.7)	126 (77.3)	79 (48.5)	131 (79.4)	112 (67.9)
Good	17 (10.9)	45 (28.8)	38 (23.9)	42 (26.4)	3(1.8)	57 (35.0)	28 (17.0)	9 (5.5)
Very good	2(1.3)	28 (17.9)	11 (6.9)	5(3.1)	0 (0)	17(10.4)	2(1.2)	2(1.2)

^†As key effect modifiers, we chose the top 3 variables in the variable importance ranking from generalized random forest.

^‡The top 10% of the distributions of individual effects were labeled as the “vulnerable” group because they showed greater associations between disaster damages and increased depressive symptoms/posttraumatic stress symptoms (PTSS). The bottom 10% of the distributions of individual effects were labeled as the “resilient” group because they showed weaker associations between disaster damages and increased depressive symptoms/PTSS.

^§We used the Geriatric Depression Scale (GDS; range, 0–15 points; higher scores indicate more PTSS) to assess depressive symptoms.

^¶We used the Screening Questionnaire for Disaster Mental Health (range, 0–9 points; higher scores indicate more PTSS) to assess PTSS.

^††Conditional average treatment effects (CATEs) were estimated via the generalized random forest algorithm.

^‡‡Annual household income (unit: 10 000 yen) was divided by the square root of the number of household members to account for household size.

In 2016, the most vulnerable groups for the CATEs of home loss on both outcomes were more likely to have lower household incomes and higher GDS scores before the disaster (Table 3). The most vulnerable groups following the loss of loved ones were more likely to be individuals who had more depressive symptoms before the disaster. For PTSS in 2016, the vulnerable groups for both types of traumatic exposures tended to be women and highly educated, while they simultaneously reported higher baseline GDS scores and lower household income.

Table 3.

Sociodemographic characteristics and key effect modifiers among individuals at the top 10% versus the bottom 10% of the estimated conditional average treatment effect of disaster damages on depressive symptoms and PTSS in 2016^†,‡

Outcome	Depressive symptoms§				PTSS¶
Exposure	Home loss (n = 1150)		Loss of loved ones (n = 1165)		Home loss (n = 1262)		Loss of loved ones (n = 1282)
Group	Vulnerable (n = 115)	Resilient (n = 115)	Vulnerable (n = 117)	Resilient (n = 117)	Vulnerable (n = 127)	Resilient (n = 127)	Vulnerable (n = 129)	Resilient (n = 129)
CATE estimates, mean (SD)††	0.43 (0.07)	0.00 (0.06)	0.23 (0.05)	−0.15 (0.05)	1.44 (0.07)	1.14 (0.06)	0.37 (0.04)	0.03 (0.05)
Age, mean (SD), y	72.4 (5.92)	71.3 (5.30)	73.8 (6.16)	70.5 (4.80)	71.6 (4.19)	72.7 (6.51)	72.4 (5.15)	70.7 (5.51)
Sex, n (%)
Men	63 (54.8)	58 (50.4)	48 (41.0)	55 (47.0)	47 (37.0)	78 (61.4)	32 (24.8)	103 (79.8)
Women	52 (45.2)	57 (49.6)	69 (59.0)	62 (53.0)	80 (63.0)	49 (38.6)	97 (75.2)	26 (20.2)
Marital status, n (%)
Married	85 (73.9)	86 (74.8)	97 (82.9)	87 (74.4)	102 (80.3)	88 (69.3)	113 (87.6)	76 (58.9)
Widowed	25 (21.7)	24 (20.9)	19 (16.2)	21 (17.9)	17 (13.4)	35 (27.6)	13 (10.1)	46 (35.7)
Divorced	2(1.7)	4 (3.5)	0(0)	5 (4.3)	4(3.1)	2(1.6)	1 (0.8)	4(3.1)
Single	2(1.7)	1 (0.9)	0 (0)	3 (2.6)	3 (2.4)	1 (0.8)	2(1.6)	2(1.6)
Others	1 (0.9)	0(0)	1 (0.9)	1 (0.9)	1 (0.8)	1 (0.8)	0 (0)	1 (0.8)
Living alone, n (%)
Not living alone	105 (91.3)	104 (90.4)	114 (97.4)	101 (86.3)	115 (90.6)	112 (88.2)	125 (96.9)	112(86.8)
Living alone	10 (8.7)	11 (9.6)	3 (2.6)	16 (13.7)	12 (9.4)	15 (11.8)	4(3.1)	17(13.2)
Education, n (%)
<6 y	3 (2.6)	0 (0)	4 (3.4)	0(0)	0(0)	5 (3.9)	1 (0.8)	4(3.1)
6–9 y	26 (22.6)	26 (22.6)	19 (16.2)	54 (46.2)	13 (10.2)	63 (49.6)	17 (13.2)	42 (32.6)
10–12y	59 (51.3)	52 (45.2)	67 (57.3)	35 (29.9)	71 (55.9)	39 (30.7)	68 (52.7)	58 (45.0)
≥13 y	27 (23.5)	37 (32.2)	27 (23.1)	28 (23.9)	42 (33.1)	20 (15.7)	43 (33.3)	23 (17.8)
Others	0(0)	0(0)	0(0)	0(0)	1 (0.8)	0(0)	0(0)	2(1.6)
Job, n (%)
Working	11 (9.6)	40 (34.8)	17 (14.5)	30 (25.6)	22 (17.3)	30 (23.6)	28 (21.7)	25 (19.4)
Retired	89 (77.4)	64 (55.7)	83 (70.9)	72 (61.5)	95 (74.8)	63 (49.6)	100 (77.5)	55 (42.6)
Never worked	15 (13.0)	11 (9.6)	17 (14.5)	15 (12.8)	10 (7.9)	34 (26.8)	1 (0.8)	49 (38.0)
Household income [10,000 yen], mean (SD)‡‡	233 (99.7)	317 (270)	243 (127)	202 (108)	197 (127)	236 (139)	247 (94.1)	253 (164)
Body mass index, mean (SD)	21.5 (3.22)	25.6 (3.60)	22.6 (3.22)	23.8 (3.00)	24.6 (3.66)	23.7 (2.90)	23.0 (2.06)	25.0 (3.52)
Sense of coherence, mean (SD)	18.3 (3.74)	24.7 (3.50)	20.8 (4.69)	21.6 (3.92)	21.6 (4.63)	22.5 (3.97)	20.6 (3.16)	23.0 (4.48)
Baseline GDS score, mean (SD)	6.43 (3.52)	1.73 (2.17)	4.96 (3.40)	4.23 (3.71)	3.91 (4.14)	3.38 (3.27)	4.44 (2.64)	3.17 (3.74)
Self-rated health, n (%)
Bad	8 (7.0)	23 (20.0)	8 (6.8)	16 (13.7)	14 (11.0)	11 (8.7)	8 (6.2)	22(17.1)
Not good	80 (69.6)	81 (70.4)	88 (75.2)	81 (69.2)	94 (74.0)	93 (73.2)	100 (77.5)	85 (65.9)
Good	18 (15.7)	10 (8.7)	18 (15.4)	16 (13.7)	16 (12.6)	19 (15.0)	21 (16.3)	19 (14.7)
Very good	9 (7.8)	1 (0.9)	3 (2.6)	4 (3.4)	3 (2.4)	4(3.1)	0(0)	3 (2.3)

^†As key effect modifiers, we chose the top 3 variables in the variable importance ranking from generalized random forest.

^‡The top 10% of the distributions of individual effects were labeled as the “vulnerable” group because they showed greater associations between disaster damages and increased depressive symptoms/posttraumatic stress symptoms (PTSS). The bottom 10% of the distributions of individual effects were labeled as the “resilient” group because they showed weaker associations between disaster damages and increased depressive symptoms/PTSS.

^§We used the Geriatric Depression Scale (GDS; range, 0–15 points; higher scores indicate more depressive symptoms) to assess depressive symptoms.

^¶We used the Screening Questionnaire for Disaster Mental Health (range, 0–9 points; higher scores indicate more PTSS) to assess PTSS.

^††Conditional average treatment effects (CATEs) were estimated via the generalized random forest algorithm.

^‡‡Annual household income (unit: 10 000 yen) was divided by the square root of the number of household members to account for household size.

Discussion

Limitations of existing studies on the impact of traumatic experiences on mental health include their focus on population average effects and a deductive approach of identifying treatment effect heterogeneity, and the use of data collected after the exposure. This prospective study of disaster survivors attempted to overcome these limitations by applying ML-based causal inference methods to estimate effect heterogeneity inductively to data with a rich set of information on survivors predating the disaster.

Our study has 2 main findings. First, while we found some evidence of population average effects of home loss and loss of loved ones on a deterioration in mental health 2.5 and 5.5 years after the disaster onset, we also found considerable heterogeneity in the adverse impacts of the traumatic experiences between individuals. Second, we inductively identified unique patterns in predisaster characteristics of individuals who were more vulnerable to the adverse impacts of the disaster damages.

Our finding that disaster damages, particularly home loss, on average, lead to increased PTSS is consistent with existing evidence.^35–37 Despite the heterogeneity among individuals, the CATE estimates for the PTSS outcomes were greater than zero for most people. This trend suggests that home loss and loss of loved ones may increase PTSS among all disaster survivors, although the magnitude of the effect differs among individuals.

However, when assessing depressive symptoms as an outcome, we did not find evidence that the loss of loved ones, on average, was linked to increased depressive symptoms. Similarly, we found no evidence for the average treatment effects of home loss on depressive symptoms in 2016. While these trends in population average effects were consistent with existing evidence, the observed effect heterogeneity represented in the widespread distributions of the estimated CATEs suggests that there were subpopulations for whom experiences of the disaster led to considerable suffering.^10,12,37 These findings underscore the importance of assessing heterogeneous health effects of traumatic experiences because examining only average relationships may mask important heterogeneity and result in the misleading conclusion that traumatic experiences do not affect the mental health of survivors.

Predisaster characteristics of vulnerable individuals—those whom the traumatic experiences were estimated to be more detrimental—were mostly consistent with the existing evidence from deductive tests of heterogeneity.^2,6,13,38 That is, individuals who were more vulnerable tended to be from lower SES backgrounds, report preexisting depressive symptoms, as well as a lower sense of coherence before the disaster.

Interestingly, we found the opposite pattern when we looked at the CATEs of home loss on depressive symptoms and PTSS in 2013. Namely, the most vulnerable group (ie, those who would have experienced greater deterioration in mental health if they had experienced home loss) was characterized by higher SES, fewer depressive symptoms, and a higher sense of coherence before the disaster. This seemingly paradoxical finding may be attributable to the fact that socioeconomically advantaged individuals had more to lose during the widespread property destruction associated with the tsunami. There may also have been a mismatch between people’s psychological coping styles (ie, higher sense of coherence) and the devastation that occurred on the ground on March 11, 2011. During normal times, individuals with a high sense of coherence are better equipped to deal with adversity.³⁸ But this coping style may be a disadvantage in extreme situations, when individuals who expect the world to be predictable and controllable (ie, individuals with a high sense of coherence) may experience the most significant gap between expectations and reality.³⁹ Notably, when assessing mental health in 2016 (5.5 years after the disaster), the most vulnerable group was characterized by lower SES and preexisting depressive symptoms. These results suggest that, by 2016, individuals with more resources and better predisaster mental health eventually adjusted to their postdisaster conditions and that their earlier vulnerability attributable to mismatch had disappeared. On the other hand, individuals from lower SES backgrounds may have continued to struggle from the disaster damages in the long run.

We also obtained a new insight that the deductive approach—a commonly used method examining a single source of heterogeneity at a time—could have missed. The vulnerable groups for the PTSS outcome in 2016 tended to report lower household income and greater depressive symptoms at baseline but also be more highly educated. In prior work examining heterogeneity deductively, higher SES such as higher educational attainment alone is typically linked to resilience to trauma.¹ Our ML-based inductive approach for effect heterogeneity allowed interactions between multiple characteristics and revealed that higher education could backfire and make people more vulnerable when coupled with other adverse backgrounds such as economic hardship and predisaster mental health problems.

Five limitations should be noted. First, our average treatment effect and CATE estimates are, as is the case in any observational studies, based on the assumption that the 51 covariates we included in GRF sufficed to control for confounding. Although we cannot rule out the possibility of unmeasured confounders, we conducted rigorous adjustment of the survivors’ predisaster characteristics by leveraging our natural experimental design. Second, the current findings do not tell us which characteristics we can intervene on to mitigate the effects of disaster damages because we chose the covariates to adjust for confounding between the disaster damages and the mental health problems, but they do not necessarily suffice to adjust for confounding between each predisaster characteristic and the mental health outcomes.⁴⁰ Rather, the results can inform us about which subpopulations are at particularly high risk of worse postdisaster mental health. Therefore, our method is particularly useful in identifying where preventive efforts to preserve mental health may best be directed. For example, even though body mass index was consistently ranked highly among the models in terms of variable importance from GRF, it does not necessarily mean that intervening on body mass index will reduce vulnerability to the adverse mental health effects of traumatic experiences. Rather, body mass index can be used to identify potentially vulnerable subpopulations for whom other preventive interventions (eg, providing cognitive behavioral therapy) need to be prioritized. Third, our exposure assessment was relatively crude and may have overlooked additional variation among individuals. For example, the effects of home loss may differ depending on its value, but we did not have such information. For example, the difference in the amount of wealth lost as a result of home loss may partly explain why the vulnerable groups for the CATEs of home loss on mental health in 2013 tended to have higher SES. Fourth, individuals who moved out of Iwanuma city were not eligible for the follow-up waves (n = 92 for the 2013 wave and n = 1 for the 2016 wave). Because such moving is likely to be affected by home loss and share common causes with the mental health outcomes (eg, existing health conditions), excluding those individuals may cause selection bias.⁴¹ However, the number of excluded individuals was relatively small, and, by leveraging our natural experiment design, we were able to adjust for 51 predisaster characteristics to increase the comparability of the exposed and unexposed groups. Last, the current study was based on a specific population (ie, older adult survivors affected by the 2011 Great East Japan Earthquake) and, thus, the generalizability of our findings to other groups may be limited. Specifically, our findings may not be generalizable to other populations when distributions of the predisaster characteristics linked to the heterogeneity (eg, household income) are different among populations.⁴²

In conclusion, our natural experiment study demonstrated considerable heterogeneity in the adverse impacts of the traumatic experiences from the 2011 Great East Japan Earthquake on survivors’ mental health. Our findings identified subpopulations for whom the same traumatic experiences may be particularly toxic, which would be overlooked had we estimated only the population average effects. We also demonstrated that the inductive estimation of effect heterogeneity based on the ML technique allows complex interactions between characteristics and identifies heterogeneity that the conventional deductive approach can miss. Assessing such heterogeneity can contribute to planning more effective and efficient postdisaster public health interventions to maintain survivors’ mental health.

Supplementary Material

Supplementary material

Figure S1. Map of Iwanuma city.

Figure S2. Flow chart of analytic sample selection. Complete cases with no missing in the exposure, outcome, and covariates were used as an analytic sample. For example, we analyzed 1559 individuals when assessing the association between home loss exposure and depressive symptoms in 2013.

Figure S3. Distributions of estimated conditional average treatment effects of disaster damages on binary depressive symptoms/posttraumatic stress symptoms (PTSS) in (a) 2013 and (b) 2016. We dichotomized Geriatric Depression Scale (range, 0–15 points; 1 = ≥5 points and 0 = ≤4 points) to assess depressive symptoms and the Screening Questionnaire for Disaster Mental Health (range, 0–9 points; 1 = ≥6 points and 0 = ≤5 points) to assess PTSS. The heterogeneous effects (ie, conditional average treatment effects) were estimated using the generalized random forest algorithm.

Figure S4. Top 3 variables from the variable importance rankings

Table S1. List of Covariates Used for Generalized Random Forest

All covariates were measured at the 2010 wave, 7 months before the disaster onset. These variables were used as covariates to control for confounding and also to estimate effect heterogeneity.

Table S2. Estimates of Population Average Treatment Effects and Conditional Average Treatment Effects of Disaster Damages on Depressive Symptoms and PTSS in 2013 and 2016. ^a Population average effects (ie, average treatment effects) of the exposures were estimated via doubly robust targeted maximum likelihood estimation. ^b Conditional average treatment effects of the exposures were estimated via the generalized random forest approach. ^c We used the Geriatric Depression Scale (range, 0–15 points; higher scores indicate more depressive symptoms) to assess depressive symptoms. ^d We used the Screening Questionnaire for Disaster Mental Health (range, 0–9 points; higher scores indicate more posttraumatic stress symptoms) to assess PTSS.