Skip to main content
PMC home page
Elsevier - PMC COVID-19 Collection logoLink to Elsevier - PMC COVID-19 Collection
. 2021 Jan 29;298:113773. doi: 10.1016/j.psychres.2021.113773

Prospective prediction of PTSD and depressive symptoms during social unrest and COVID-19 using a brief online tool

Stephanie MY Wong a,1, Christy LM Hui a, Corine SM Wong a, YN Suen a, Sherry KW Chan a,b, Edwin HM Lee a, WC Chang a,b, Eric YH Chen a,1,⁎
PMCID: PMC9754713  PMID: 33545423

Abstract

Large-scale protracted population stressors, such as social unrest and the coronavirus disease 2019 (COVID-19), are associated with increased symptoms of post-traumatic stress disorder (PTSD) and depression. Cost-effective mental health screening is prerequisite for timely intervention. We developed an online tool to identify prospective predictors of PTSD and depressive symptoms in the context of co-occurring social unrest and COVID-19 in Hong Kong. 150 participants completed baseline and follow-up assessments, with a median duration of 29 days. Three logistic regression models were constructed to assess its discriminative power in predicting PTSD and depressive symptoms at one month. Receiver-operating characteristic analysis was performed for each model to determine their optimal decision thresholds. Sensitivity and specificity of the models were 87.1% and 53.8% for probable PTSD, 77.5% and 63.3% for high-risk depressive symptoms, and 44.7% and 96.4% for no significant depressive symptoms. The models performed well in discriminating outcomes (AUCs range: 0.769–0.811). Probable PTSD was predicted by social unrest-related traumatic events, high rumination, and low resilience. Rumination and resilience also predicted high-risk and no significant depressive symptoms, with COVID-19-related events also predicting no significant depression risk. Accessible screening of probable mental health outcomes with good predictive capability may be important for early intervention opportunities.

Keywords: PTSD symptoms, Depressive symptoms, Trauma exposure, COVID-19, Mass screening, Risk assessment

1. Introduction

Large-scale population-level crises, such as social unrest (Ni et al., 2020b) and viral epidemics (Rajkumar, 2020; Tang et al., 2020), are associated with increases in symptoms of post-traumatic stress disorder (PTSD) and depression. The detection of those at higher risk, i.e., those whose symptoms do not improve over time, is important for timely intervention (Kessler et al., 2014). Real-life longitudinal data are required to inform the development of mental health screening tools with good predictive capability. Making such tools accessible in different settings is important, particularly in the recent global context.

A series of escalating social protests and conflicts have taken place in Hong Kong since June 2019. Rubber bullets, live rounds, tear gas, and petrol bombs were involved, with thousands of arrests made (Robles, 2019). Elevated levels of PTSD symptoms and depressive symptoms have been observed even from the initial months (Ni et al., 2020b). The outbreak of coronavirus 2019 (COVID-19) since January 2020 inflicted further turmoil on the population. Apart from pandemic-related concerns and social distancing measures, help-seeking via conventional pathways has also been compromised by concerns over potential leakage of sensitive personal information, for the fear of persecution or retaliation. Of note, the adverse combination of prolonged pandemic-related stress and societal conflicts are being increasingly encountered worldwide (Bender and Winning, 2020; Ni et al., 2020a).

Past studies have linked exposures to traumatic events in social conflicts to both PTSD and depressive symptoms (Housen et al., 2017; Ni et al., 2020b). The longitudinal impact of repeated experiences of conflict on mental health has been reported (Betancourt et al., 2013; Silove et al., 2014). In particular, rumination about conflict-related sentiments has been identified to be one of the key predictors of symptom outcomes (Silove et al., 2014). Meanwhile, since its emergence, the COVID-19 pandemic has also been observed to trigger PTSD and depressive symptoms (Liu et al., 2020; Torales et al., 2020). Preliminary cross-sectional findings have suggested the roles of resilience and rumination during COVID-19 on mental health (Killgore et al., 2020; Ye et al., 2020). Few studies, however, addressed the combined effects of both social unrest and COVID-19 in addition to other individual-level factors, especially using a longitudinal design. Data sensitive not only to one-off but also changes in symptomatology over time would be important to identify individuals who may need (1) professional attention (for high-risk individuals), (2) continued symptom monitoring (for mild-to-moderate risk individuals), as well as (3) reassurance (for those who are unlikely to deteriorate).

Smartphone-based assessments enable such accessibility for addressing the mental health impacts particularly during ongoing population-level stressful events, but require both brevity and comprehensiveness to be engaging so that informative data can be acquired. We compiled a brief online tool to capture information about exposure to population trauma, pandemic stress, personal life stressors, as well as key background risk and protective factors using abridged items from validated scales whenever possible. The current study evaluated the longitudinal predictive performance of this tool.

1.1. Aims of the study

We aimed to investigate the effectiveness of the online screening tool in the prospective prediction of symptoms for potential application in contexts undergoing ongoing population-level stress. The study tested the hypotheses that (1) baseline measures of stress exposure, risk and protective factors, and background factors will predict one-month PTSD and depressive symptom outcomes and that (2) the tool would have good predictive discrimination ability.

2. Methods

2.1. Sample

Longitudinal data were obtained from individuals who had completed an initial assessment and a subsequent follow-up assessment using a mental health screening tool that was freely accessible online. The online self-assessment tool, named the “Flow Tool”, was developed for increasing mental health awareness and helping-seeking using the Qualtrics survey system (available at https://www.psychiatry.hku.hk/flow.html). Users of the tool were invited to participate in a more comprehensive survey of risk and protective factors (reported in Wong et al., 2020). At baseline, the tool evaluated PTSD symptoms, depressive symptoms, exposure to social unrest-related traumatic events (TEs), COVID-19 pandemic-related events (PEs), personal stressful life events (SLEs), rumination, resilience, and other risk and protective factors. Participants who completed the baseline assessment were invited to participate in a follow-up assessment at one month. In response to the sensitive environment at the time of the study, the tool was designed to be particularly non-intrusive. Contact information was not demanded from participants. Follow-up contact was entirely voluntary and strictly initiated by the participant. Feedback about individual mental health conditions and advice for symptom management and help-seeking were provided to all participants after completion. No reimbursement was given for participation. Data of the initial assessment were collected between February 21, 2020, and March 6, 2020. A total of 10,110 individuals completed the initial assessment, of which 2,063 indicated they may consider participating in the follow-up assessment. The study was approved by the Institutional Review Board of the University of Hong Kong/Hospital Authority Hong Kong West Cluster.

2.2. Demographic information and background factors

Demographic information included gender (female, male, others), age (<25 years, 25–44 years, ≥45 years), and education level (primary education or below, secondary education, bachelor's degree or above). Past adversity experienced before the age of ten was assessed using a checklist that included “parents absent from home continuously for a significant period of time” to “bullied at school”, “loss of a parent”, “physical abuse”, “emotional abuse”, “sexual abuse”, and “others”. Past psychiatric history was also noted.

2.3. PTSD and depressive symptoms

PTSD symptoms were assessed using the ten-item Traumatic Screening Questionnaire (TSQ; Brewin et al., 2002). For those who identified a target traumatic event, a cut-off of six points was adopted in the current study to differentiate between low and high risk for PTSD with reference to previous studies (Brewin et al., 2002; Gouweloos-Trines et al., 2019). The TSQ has been demonstrated to have good internal consistency in a Hong Kong epidemiological sample (α=0.93), with a sensitivity of 0.93 and specificity of 0.68 (Wu et al., 2019). Depressive symptoms were assessed using the depression subscale of the Depression, Anxiety and Stress Scales (DASS-D; Lovibond and Lovibond, 1995a). The Chinese version of DASS-D has also been demonstrated to have good reliability (α=0.82; Lu et al., 2018). A score of 13 or below indicated normal to mild symptoms, 14 to 20 indicated moderate severity, and 21 or above indicated severe to very severe symptoms (Lovibond and Lovibond, 1995b). We adopted the cut-off of 13 or below in the DASS-D to define symptoms suggesting no significant risk for depression, and 21 or above to define high-risk depressive symptoms. In the current study, the internal consistency of TSQ and DASS-D was good at both baseline (TSQ: α=0.89, DASS-D: α=0.87) and follow-up (TSQ: α=0.89, DASS-D: α=0.89).

2.4. External triggering events

Context-relevant checklists were constructed after consultations with local community members and mental health professionals to assess experiences of (1) TEs, (2) PEs, and (3) SLEs. For instance, TEs were asked using the item “Have you experienced any of the following events since June 2019?”, with the checklist options including “personal experience of physical attack”, “sexual violence”, “arrest or detention”, “targeted verbal assaults or threats”, “crowd dispersal by the use of force”, “witnessing violent attacks on others”, “media viewing of disruptive actions”, and “media viewing of others being attacked”. Options for PEs (since January 2020) included “lack of protective gear”, “working in high-risk environments”, “placed under quarantine”, “family members or significant others affected”, “feeling lost about the future”, and “others”. SLEs (since June 2019) included “conflict with family”, “conflict with friends”, “being bullied”, “being abused (physical, emotional, sexual)”, “self-initiated termination of employment or studies”, “dismissal from job or expelled from school”, “serious physical health conditions”, “passing away of a significant other”, “experience of legal actions or being charged”, and “others”. A composite score was generated for each of these three triggering event categories, respectively, by summing the number of experiences endorsed.

2.5. Psychological factors

Rumination in the current study was defined as frequent repetitive thoughts to the extent that disruptions to other current tasks are caused. Four specific themes of ruminative thoughts were identified through interviews with 36 laypersons and mental health professionals, namely anger, injustice, guilt, and insecurity. For each of these four themes, one item was first used to determine the extent to which the psychological reaction experienced, with a second item assessing the frequency which the ruminative thoughts relating to the theme was experienced (see Supplementary Material S1). The sum of the four rumination items was used to compute a final composite rumination score. The internal consistency of the rumination items was good in this study (α=0.81).

The content and face validity of the rumination items over their clarity, relevance, understandability, and importance was assessed by eleven panel members (consisting of five experts and six laypersons). The ratings are summarized into indexes according to the following established criteria: (1) content validity ratio (CVR)≥0.62 (Lawshe, 1975); (2) item-level content validity index (I-CVI)>0.79 (Polit and Beck, 2006); (3) scale-level CVI (S-CVI)≥0.90 (Polit and Beck, 2006), and (4) impact score≥1.5 (Lacasse et al., 2002). All rumination items showed good content and face validity (CVR range=0.64–1.00; I-CVI range=0.91–1.00; S-CVI=0.95; impact score range=3.42–4.73).

Resilience was assessed with an abridge version of the Connor-Davidson Resilience Scale (Connor and Davidson, 2003) with three items (which we refer to as CD-RISC3 here): “able to adapt to change" (item 1), "can deal with whatever comes" (item 4), and "not easily discouraged by failure" (item 16). Each item was rated on a five-point Likert scale (from 0 “not true at all” to 4 “true nearly all of the time). The 10-item CD-RISC (CD-RISC-10) has been validated in Chinese populations (Yu and Zhang, 2007).

The internal consistency and construct validity of the CD-RISC3 used in the current study have been tested using data from a separate community youth sample in Hong Kong by our team (n=84, mean age=20.02, SD=2.80). Cronbach's α for CD-RISC3 was 0.74 in this epidemiological sample. The three-item scale was strongly correlated with CD-RISC-10 (r=0.88, p<0.001). As in previous studies (van der Meer et al., 2018; Yu and Zhang, 2007), convergent and discriminant validity were assessed against depressive symptoms (using the Patient Health Questionnaire-9; Kroenke et al., 2001) and PTSD symptoms (TSQ; Brewin et al., 2002). Significant negative associations were demonstrated between the CD-RISC3 and both depressive (r=-0.33, p<0.01) and PTSD (r=-0.23, p=0.037) symptoms.

2.6. Data analysis

All statistical analyses were performed using SPSS 25.0 (IBM SPSS Statistics, New York, United States). We used logistic regression models to identify baseline predictors of follow-up outcomes. Binary outcomes for PTSD and depressive symptoms were derived from follow-up TSQ and DASS-D scores (probable PTSD: TSQ≥6; high-risk depressive symptoms: DASS-D≥21; no significant depressive symptoms: DASS-D<14). Separate regression models were constructed for each of these symptom outcomes.

As depressive symptoms could present in the community over a wide spectrum of severity, from “near to none” to “severe”, the screening objective was to identify (1) those predicted to develop severe symptoms, who should be advised to seek professional help, and (2) those predicted to have no significant symptoms at follow-up. We therefore used two separate models for (1) high-risk depressive symptoms and (2) no significant depressive symptoms using the conventional cut-offs in DASS-D. For those in between these two thresholds (i.e., those who may present mild-to-moderate risk for depression), further monitoring would be advised (Steyerberg, 2019).

Candidate predictor variables for the three models were selected based on a priori theoretical grounds, which included TEs, PEs, SLEs, rumination, resilience, and past adversity (potential predictors), as well as age, gender, and past psychiatric history (controlled variables). Due to the small number of participants aged 45 years or above, the three age groups were conflated into two groups of below 25 years and 25 years or above. For the same reason, those who reported neither females nor males for gender were excluded in the current analyses.

Descriptive statistics were generated for all variables. Chi-square test and t-test were performed with the potential predictors on each three binary symptom outcomes at the significance level of 0.05. One-sample t-test was also performed to compare baseline and follow-up symptom scores.

Variables for inclusion in the final models were selected according to the procedures of the purposeful variable selection method (Hosmer Jr and Lemeshow, 2000). We adopted this methodology because the steps involved in the procedure allows for retaining not only significant variables in the final model, but also those with confounding effects (Bursac et al., 2008), which has the potential for building more comprehensive prediction models. In the first step, potential predictor variables significant at the p<0.25 level in the univariate analyses were included in the initial multivariate regression models for further testing. Second, the significance of these individual variables was tested in the multivariate context. Variables significant at the p<0.10 level, or if their exclusion resulted in a magnitude change of over 20% in any other coefficients, were retained in the final logistic regression models (Hosmer Jr et al., 2013). Third, variables not significant in the first step (p>0.25 in univariate analyses) were individually added back to the respective multivariate model for testing and were retained if above criteria were met. Age, gender, and past psychiatric history were included in the final models as controlled variables.

Following the above procedures, the potential predictors included in the final regression models were TEs, PEs, SLEs, rumination, and resilience for probable PTSD. The same set of potential predictors was identified for no significant depressive symptoms. Meanwhile, rumination, resilience, and past adversity were included in the final model for high-risk depressive symptoms (Table 2).

Table 2.

Logistic regression models for probable PTSD, high-risk depressive symptoms, and no significant depressive symptoms.

Probable PTSD
 High-risk depressive symptoms
 No significant depressive symptoms
OR 95% CI p OR 95% CI p OR 95% CI p
≥25 years Ref Ref Ref
<25 years 2.02 0.91–4.49 0.085 2.17 0.98–4.84 0.058 0.96 0.39–2.38 0.933
Male Ref Ref Ref
Female 0.86 0.34–2.15 0.74 0.36 0.15–0.88 0.024 3.41 1.07–10.93 0.039
No psychiatric history Ref Ref Ref
Has psychiatric history 0.81 0.30–2.23 0.69 0.84 0.31–2.28 0.732 2.62 0.86–7.99 0.090
TEs 1.49 1.11–2.00 0.01 - - - 0.86 0.62–1.18 0.341
PEs 1.25 0.77–2.04 0.37 - - - 0.43 0.23–0.78 0.006
SLEs 1.36 0.94–1.96 0.11 - - - 0.79 0.49–1.25 0.308
Rumination 1.19 1.02–1.38 0.03 1.38 1.18–1.61 0.000 0.81 0.68–0.96 0.017
Resilience 0.82 0.68–0.98 0.03 0.84 0.71–0.99 0.041 1.37 1.10–1.71 0.005
Past adversity - - - 0.80 0.58–1.12 0.199 - - -
Open in a new tab

PEs=coronavirus 2019 pandemic-related events; PTSD=post-traumatic stress disorder; SLEs=personal stressful life events; TEs=social unrest-related traumatic events.

A receiver-operating characteristic (ROC) curve was constructed for each of the three models, with the area under the curve (AUC) evaluated to assess their discriminatory power. Sensitivity (true positive rate), specificity (true negative rate), positive predictive value (PPV), and negative predictive value (NPV) were also calculated. False-negative rate (FPR) and false-positive rate (FNR) were generated with the following formulae: FPR=1−Specificity; FNR=1−Sensitivity, respectively.

To identify the optimal decision threshold for each model, we took reference from the Youden index (Sensitivity+Specificity−1; Youden, 1950), which is a summary measure of ROC that offers an indicator for a threshold value that maximizes sensitivity and specificity (Böhning et al., 2008). Nonetheless, particularly in the community setting, avoiding false-negative errors is arguably more important than avoiding false-positives (Steyerberg, 2019). We therefore applied this principle to determine the cut-off for probable PTSD. Meanwhile for depressive symptoms, we executed a two-stage process to predict (1) those with persistent high-risk depressive symptoms, and (2) those with no significant depressive symptoms.

3. Results

3.1. Descriptive statistics of the sample

150 participants completed the follow-up survey. The median follow-up time since baseline assessment was 29.5 days (IQR=24–48). Among the participants, 67.3% (n=101) aged below 25 years and 74.7% (n=112) were females. Forty-nine percent (n=74) reported secondary education as their highest education level and 50% (n=75) reported having a bachelor's degree or above. Descriptive statistics of other predictors are presented in Table 1 . Proportions of those reporting exposure to TEs, PEs, and SLEs are included in Supplementary Material Table S2. For PTSD symptoms, the mean TSQ score was 5.29 (SD=3.44) at baseline and 4.89 (SD=3.50) at follow-up (p<0.001). Their mean DASS-D score was 22.93 (SD=9.79) at baseline and 20.85 (SD=10.29) at follow-up (p<0.001). A higher proportion of females presented with no significant depressive symptoms at baseline compared to males (17.9% vs 5%), X2(1, N=150)=3.596, p=0.058.

Table 1.

Sample characteristics at baseline for each PTSD and depressive symptom subgroup.

Overall (n=150) PTSD symptoms (TSQ<6 vs ≥6)
 Depressive symptoms (DASS-D<21 vs ≥21)
 Depressive symptoms (DASS-D<14 vs≥14)
TSQ<6 (n=65) TSQ≥6 (n=85) p DASS-D<21 (n=79) DASS-D≥21 (n=71) p DASS-D<14 (n=38) DASS-D≥14 (n=112) p
Demographic factors, n (%)
Aged <25 years 101 (67.3) 36 (35.6) 65 (64.4) 0.006 46 (45.5) 55 (54.5) 0.012 23 (22.8) 78 (77.2) 0.300
Female gender 112 (74.7) 50 (44.6) 62 (55.4) 0.578 64 (57.1) 48 (42.9) 0.059 32 (28.6) 80 (71.4) 0.117
Psychiatric history 31 (20.7) 10 (32.3) 21 (67.7) 0.162 14 (45.2) 17 (54.8) 0.347 8 (25.8) 23 (74.2) 0.946
Stress exposure
TEs 3.35 (1.56) 2.77 (1.43) 3.80 (1.51) 0.000 3.14 (1.47) 3.59 (1.63) 0.076 2.87 (1.42) 3.52 (1.58) 0.026
PEs 1.58 (0.78) 1.51 (0.79) 1.64 (0.77) 0.322 1.57 (0.80) 1.59 (0.77) 0.864 1.26 (0.69) 1.69 (0.78) 0.003
SLEs 1.51 (1.15) 1.18 (0.95) 1.75 (1.22) 0.002 1.38 (1.04) 1.65 (1.24) 0.153 1.18 (1.06) 1.62 (1.16) 0.044
Psychological factors
Rumination 5.56 (2.77) 4.52 (2.62) 6.35 (2.63) 0.000 4.56 (2.44) 6.68 (2.70) 0.000 4.21 (2.64) 6.02 (2.67) 0.000
Resilience 5.73 (2.31) 6.34 (2.20) 5.27 (2.30) 0.005 6.14 (2.02) 5.28 (2.53) 0.024 6.66 (1.89) 5.42 (2.36) 0.004
Background factor
Past adversity 1.13 (1.21) 0.97 (1.10) 1.26 (1.27) 0.146 1.15 (1.14) 1.11 (1.28) 0.843 1.08 (1.24) 1.15 (1.20) 0.749
Open in a new tab

Group comparisons were performed using chi-square test for categorical variables and independent t-test for continuous variables. Statistics are presented in the form of means and standard deviations, unless stated otherwise. DASS-D=depression subscale of the Depression, Anxiety and Stress Scales; PEs=coronavirus 2019 pandemic-related events; PTSD=post-traumatic stress disorder; SLEs=personal stressful life events; TEs=social unrest-related traumatic events; TSQ=Trauma Screening Questionnaire.

3.2. Prospective prediction of probable PTSD

We conducted a logistic regression to see how well baseline measures can predict higher levels of PTSD symptoms (TSQ≥6) after one month. Prospective predictors of probable PTSD were first identified using a logistic regression model (Table 2 ). Significant predictors PTSD included TEs (odds ratio [OR]=1.49, 95% CI, 1.111–1.998, p=0.008), rumination (OR=1.187, CI, 1.02–1.383, p=0.027), and resilience (OR=0.817, CI, 0.682–0.98, p=0.029). The AUC of the model (Fig. 1 a) was 0.778 (CI, 0.703–0.851), suggesting acceptable discrimination (Hosmer Jr et al., 2013). Using the optimal cut-off of 0.422 identified, the accuracy of the model was 72.7%, with 87.1% sensitivity (12.9% FNR), 53.8% specificity (46.2% FPR), 71.2% PPV, and 76.1% NPV. Using baseline data, it was possible to correctly identify 87.1% of those who had high-risk PTSD symptoms one month later.

Fig. 1.

Fig 1

Open in a new tab

Receiver-operating characteristic (ROC) curves for probable PTSD (a); high-risk depressive symptoms (b); and no significant depressive symptoms (c).

3.3. Prospective prediction of high-risk depressive symptoms

A separate logistic regression was conducted to examine how well baseline measures could predict higher risks of depressive symptoms (DASS≥21) after one month. For high-risk depressive symptoms, rumination (OR=1.377, CI, 1.177–1.612, p<0.001), resilience (OR=0.839, CI, 0.709–0.993, p=0.041), and female gender (OR=0.362, CI, 0.149–0.877, p=0.024) were significant predictors (Table 2). Age was marginally significant (OR=2.172, CI, 0.975–4.839, p=0.058). The discriminatory ability of the model was acceptable, indicated by an AUC of 0.769 (CI, 0.694–0.844; Fig. 1b). Using the cut-off of 0.426, the model demonstrated 70.0% accuracy, 77.5% sensitivity (22.5% FNR), 63.3% specificity (36.7% FPR), 65.5% PPV, and 75.8% NPV. The findings showed that baseline data could correctly identify 77.5% of those who had high depressive symptoms one month later.

3.4. Prospective prediction of no significant depressive symptoms

In the third model, we aimed to identify those with no significant depressive symptoms after one month using baseline variables. In this model, we were cautious to choose a discriminant threshold to minimize the chance of falsely predicting individuals as having no significant risk, yet turn out to present significant symptoms one month later. We found that PEs (OR=0.428, CI, 0.234–0.782, p=0.006), rumination (OR=0.805, CI, 0.675–0.962, p=0.017), resilience (OR=1.372, CI, 1.103–1.708, p=0.005), and female gender (OR=3.413, CI, 1.066–10.934, p=0.039) were significant predictors of no significant depressive symptoms (Table 2). The discriminatory ability of the model was excellent (Hosmer Jr et al., 2013) with an AUC of 0.811 (CI, 0.694–0.844; Fig. 1c). With the optimal threshold of 0.495, the accuracy of the model was 83.3%, with 44.7% sensitivity (55.3% FNR), 96.4% specificity (3.6% FPR), 81% PPV, and 83.7% NPV. The model correctly identified 44.7% of those with no significant depressive symptoms at one month, such that many who were initially predicted to have increased symptomatology turned out to have no significant symptoms at follow-up. However, importantly, for those predicted to have no significant symptoms at one month, 96.4% actually had no significant symptoms at one month (i.e., only 3.6% of those predicted to have low risk turned out to present significant symptoms).

4. Discussion

The unique feature of this study was the use of a highly efficient online survey tool to generate prospective estimations of future clinical states. We assessed the ability of the tool to identify those with specific ranges of future symptomatic outcomes following exposure to different types of community-level and personal-level stress events (TEs, PEs, and SLEs). We showed that the simple tool was able to predict one-month outcomes with acceptable to excellent discrimination.

Overall, our goals were to identify (1) those likely to develop probable PTSD symptomatology, (2) those likely to develop high-risk depressive symptomatology, and (3) those likely to be free from significant depressive symptomatology (low risk). We used two different thresholds for predicting depressive symptoms to inform subsequent follow-up actions: (a) for those at high risk, we hoped to attain a model with high sensitivity to minimize the chance of missing severe cases, so that prompt professional help can be advised, and (b) for those with no significant symptoms, good model specificity is desired so that the majority would be true low-risk cases, for whom reassurance could be given. Meanwhile, for those who were in between (a) and (b), we advise further observations and evaluation.

In relation to these goals, our tool was able to (1) correctly predict 87.1% (sensitivity) of those with probable PTSD in one-month (which was close to that observed for TSQ in a previous study; Mouthaan et al., 2014) and (2) correctly predict 77.5% of those likely to develop high-risk depressive symptoms. In addition, of those who were predicted to have no significant depressive symptoms, 96.4% actually turned out not to have significant symptoms at one month.

Through the three regression models, we found that higher baseline levels of rumination and lower resilience were predictive of probable PTSD one month later, while the reverse (i.e., lower rumination and higher resilience) predicted absence of significant depressive symptoms. These findings are consistent with previous observations that rumination may be a significant risk factor (Ehring et al., 2008; Nolen-Hoeksema et al., 2008), and resilience a protective factor (Ying et al., 2014) for PTSD and depression. Our study extended these observations to show their roles in predicting one-month outcomes in community members using an accessible online tool. As both rumination and resilience have been suggested to be transdiagnostic and modifiable factors (Hoge et al., 2007; Nolen-Hoeksema and Watkins, 2011), our tool may help to inform early detection of those at greater mental health risks, as well as have implications for earlier intervention opportunities. Preventative psychological interventions to reduce rumination, and also to enhance resilience, may help to reduce distress due to PTSD and depressive symptoms and the emergence of full-blown disorders in the respective domains. This would be particularly important in the context of protracted ongoing population stress.

Interestingly, greater exposure to TEs predicted probable PTSD and not depressive symptoms, while experiences of PEs were related to depressive symptoms, but not PTSD symptoms. The specific TEs-PTSD and PEs-depressive relationships suggest that while some mechanisms may be shared between the two symptom clusters (i.e., rumination and resilience), differential underlying pathways may also be implicated in the manifestation of PTSD and depressive symptomatology as a result of TEs and PEs, such as the involvement of fear and threat brain systems in PTSD (Gray et al., 2004), together with neural pathways related to learned helplessness in depression (Hammen, 2005), respectively.

In addition, we found that in our study context, females were less likely than males to show continued expression of high-risk depressive symptoms. This may relate to the observation that males who participated in the follow-up survey had a higher level of baseline depressive symptoms. This may be a reflection of the gender difference in help-seeking that is also seen in other studies (Barney et al., 2006; Henderson et al., 2013), including online interventions (Crisp and Griffiths, 2014).

4.1. Strengths and limitations

Our use of an online platform enabled us to engage populations otherwise difficult to reach (Batterham, 2014). It was also highly cost-effective and could facilitate timely assessment in response to population-level stress events. Such advantages appeared to be particularly important in rapidly changing circumstances, such as COVID-19 and social unrest, which are being encountered in an increasing number of communities (Heiervang and Goodman, 2011; McVeigh, 2020). Nonetheless, because of the online nature of this study, participants were from a non-random sample. Although robust patterns of association between variables have been demonstrated in Internet-based studies (Heiervang and Goodman, 2011), we still note that caution should be taken in generalizing the study findings to the population. Due to the small sample size, we were unable to include gender non-binary individuals in this study. It is possible that these individuals may be more vulnerable to stressful events, including in times of conflict (Moore and Barner, 2017), and at a greater risk of more severe distress (Chan et al., 2020; Russell and Fish, 2016). How these individuals might be impacted by multiple population-level stressors in addition to personal stress requires further study.

While more in-depth psychopathological assessments would undoubtedly be required for greater accuracy, their implementation is much more difficult with the constraints of high costs, low accessibility, and expert manpower required. Online screening and assessment could alleviate the burden in these aspects and overcome the issue of restricted face-to-face contact, especially in times of prolonged social unrest and global pandemic.

We acknowledge that prospective models for other purposes, such as clinical diagnosis, may involve greater considerations for FPR, since erroneous diagnostic decisions could increase costs to the individual (e.g., stigma, intervention costs) as well as to the healthcare system (e.g., increased unnecessary intervention for low-risk cases when resources are limited). In the case of a prospective screening model, the cost/benefit considerations may favor putting more weight on FNR rather than FPR, as the screening outcome does not confer a diagnosis leading to treatment, but a recommendation for professional evaluation which can have a gatekeeping function for interventional resources. The current study established the feasibility and initial predictive capability of such a screening tool. It would be desirable to further validate the models in other samples and contexts.

In conclusion, we demonstrated the potential utility of a brief, low-cost online tool for predicting prospective mental health outcomes. Our study provided insights into the assessment of PTSD and depressive symptoms in the community amid ongoing community stress. The factors we identified could facilitate prediction of symptom persistence and escalation. Especially in times of widespread uncertainties, early detection of individual risk and needs is vital for efficient allocation of limited resources in the community.

Funding

The authors received no financial support for the research, authorship, and publication of this article.

Declaration of Competing Interest

EYHC has received speaker honoraria from Otsuka and DSK BioPharma, research funding from Otsuka, participated in paid advisory boards for Janssen and DSK BioPharma, and received funding to attend conferences from Otsuka and DSK BioPharma. All other authors declare no competing interests.

Acknowledgement

We would like to express our gratitude to all community members and professionals who have shared with us their thoughts and comments on the mental health survey and the individuals who have completed the survey.

Footnotes

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.psychres.2021.113773.

Appendix. Supplementary materials

mmc1.docx (19.9KB, docx)

References

  1. Barney L.J., Griffiths K.M., Jorm A.F., Christensen H. Stigma about depression and its impact on help-seeking intentions. Aust. N. Z. J. Psychiatry. 2006;40(1):51–54. doi: 10.1080/j.1440-1614.2006.01741.x. [DOI] [PubMed] [Google Scholar]
  2. Batterham P.J. Recruitment of mental health survey participants using Internet advertising: content, characteristics and cost effectiveness. Int. J. Methods Psychiatr. Res. 2014;23(2):184–191. doi: 10.1002/mpr.1421. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bender R., Winning D. Antiracism protests erupt around the world in wake of George Floyd killing. Wall Street J. 2020;2020 https://www.wsj.com/articles/protests-over-george-floyd-killing-spread-to-cities-around-the-world-11591524556/ accessed Sept 12. Last updated Jun 7, 2020. [Google Scholar]
  4. Betancourt T.S., McBain R., Newnham E.A., Brennan R.T. Trajectories of internalizing problems in war-affected Sierra Leonean youth: examining conflict and postconflict factors. Child Dev. 2013;84(2):455–470. doi: 10.1111/j.1467-8624.2012.01861.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Böhning D., Böhning W., Holling H. Revisiting Youden's index as a useful measure of the misclassification error in meta-analysis of diagnostic studies. Stat. Methods Med. Res. 2008;17(6):543–554. doi: 10.1177/0962280207081867. [DOI] [PubMed] [Google Scholar]
  6. Brewin C.R., Rose S., Andrews B., Green J., Tata P., McEvedy C., Turner S., Foa E.B. Brief screening instrument for post-traumatic stress disorder. Br. J. Psychiatry. 2002;181(3):158–162. doi: 10.1017/s0007125000161896. [DOI] [PubMed] [Google Scholar]
  7. Bursac Z., Gauss C.H., Williams D.K., Hosmer D.W. Purposeful selection of variables in logistic regression. Source Code Biol. Med. 2008;3(1):17. doi: 10.1186/1751-0473-3-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Chan R.C., Operario D., Mak W.W. Bisexual individuals are at greater risk of poor mental health than lesbians and gay men: the mediating role of sexual identity stress at multiple levels. J. Affect. Disord. 2020;260:292–301. doi: 10.1016/j.jad.2019.09.020. [DOI] [PubMed] [Google Scholar]
  9. Connor K.M., Davidson J.R. Development of a new resilience scale: the Connor-Davidson Resilience Scale (CD-RISC) Depress. Anxiety. 2003;18(2):76–82. doi: 10.1002/da.10113. [DOI] [PubMed] [Google Scholar]
  10. Crisp D.A., Griffiths K.M. Participating in online mental health interventions: who is most likely to sign up and why? Depress. Res. Treat. 2014;2014 doi: 10.1155/2014/790457. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Ehring T., Frank S., Ehlers A. The role of rumination and reduced concreteness in the maintenance of posttraumatic stress disorder and depression following trauma. Cognit. Ther. Res. 2008;32(4):488–506. doi: 10.1007/s10608-006-9089-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Gouweloos-Trines J., Te Brake H., Sijbrandij M., Boelen P.A., Brewin C.R., Kleber R.J. A longitudinal evaluation of active outreach after an aeroplane crash: screening for post-traumatic stress disorder and depression and assessment of self-reported treatment needs. Eur. J. Psychotraumatol. 2019;10(1) doi: 10.1080/20008198.2018.1554406. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Gray M.J., Litz B.T., Hsu J.L., Lombardo T.W. Psychometric properties of the life events checklist. Assessment. 2004;11(4):330–341. doi: 10.1177/1073191104269954. [DOI] [PubMed] [Google Scholar]
  14. Hammen C. 2005. Stress and depression. Annu. Rev. Clin. Psychol. 2005;1:293–319. doi: 10.1146/annurev.clinpsy.1.102803.143938. [DOI] [PubMed] [Google Scholar]
  15. Heiervang E., Goodman R. Advantages and limitations of web-based surveys: evidence from a child mental health survey. Soc. Psychiatry Psychiatr. Epidemiol. 2011;46(1):69–76. doi: 10.1007/s00127-009-0171-9. [DOI] [PubMed] [Google Scholar]
  16. Henderson C., Evans-Lacko S., Thornicroft G. Mental illness stigma, help seeking, and public health programs. Am. J. Public Health. 2013;103(5):777–780. doi: 10.2105/AJPH.2012.301056. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Hoge E.A., Austin E.D., Pollack M.H. Resilience: research evidence and conceptual considerations for posttraumatic stress disorder. Depress. Anxiety. 2007;24(2):139–152. doi: 10.1002/da.20175. [DOI] [PubMed] [Google Scholar]
  18. Housen T., Lenglet A., Ariti C., Shah S., Shah H., Ara S., Pintaldi G. Prevalence of anxiety, depression and post-traumatic stress disorder in the Kashmir Valley. BMJ Glob. Health. 2017;2 doi: 10.1136/bmjgh-2017-000419. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hosmer D.W., Jr., Lemeshow S. second ed. Wiley; New York: 2000. Applied Logistic Regression. [Google Scholar]
  20. Hosmer D.W., Jr, Lemeshow S., Sturdivant R.X. third ed. John Wiley & Sons; Hoboken, NJ: 2013. Applied Logistic Regression. [Google Scholar]
  21. Kessler R.C., Rose S., Koenen K.C., Karam E.G., Stang P.E., Stein D.J., Carmen Viana M. How well can post-traumatic stress disorder be predicted from pre-trauma risk factors? An exploratory study in the WHO World Mental Health Surveys. World Psychiatry. 2014;13(3):265–274. doi: 10.1002/wps.20150. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Killgore W.D., Taylor E.C., Cloonan S.A., Dailey N.S. Psychological resilience during the COVID-19 lockdown. Psychiatry Res. 2020;291 doi: 10.1016/j.psychres.2020.113216. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Kroenke K., Spitzer R.L., Williams J.B. The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 2001;16:606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Lacasse Y., Godbout C., Series F. Health-related quality of life in obstructive sleep apnoea. Eur. Respir. J. 2002;19(3):499–503. doi: 10.1183/09031936.02.00216902. [DOI] [PubMed] [Google Scholar]
  25. Lawshe C.H. A quantitative approach to content validity. Pers Psychol. 1975;28(4):563–575. [Google Scholar]
  26. Liu C.H., Zhang E., Wong G.T.F., Hyun S. Factors associated with depression, anxiety, and PTSD symptomatology during the COVID-19 pandemic: clinical implications for US young adult mental health. Psychiatry Res. 2020;290 doi: 10.1016/j.psychres.2020.113172. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Lovibond P.F., Lovibond S.H. The structure of negative emotional states: comparison of the Depression Anxiety Stress Scales (DASS) with the beck depression and anxiety inventories. Behav. Res. Ther. 1995;33(3):335–343. doi: 10.1016/0005-7967(94)00075-u. [DOI] [PubMed] [Google Scholar]
  28. Lovibond S.H., Lovibond P.F. second ed. Psychology Foundation; Sydney: 1995. Manual for the Depression Anxiety Stress Scales. [Google Scholar]
  29. Lu S., Hu S., Guan Y., Xiao J., Cai D., Gao Z., Sang Z., Wei J., Zhang X., Margraf J. Measurement Invariance of the Depression Anxiety Stress Scales-21 across gender in a sample of Chinese university students. Front. Psychol. 2018;9:2064. doi: 10.3389/fpsyg.2018.02064. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. McVeigh K. The Guardian; 2020. Protests Predicted to Surge Globally as COVID-19 Drives Unrest.https://www.theguardian.com/global-development/2020/jul/17/protests-predicted-to-surge-globally-as-covid-19-drives-unrest/ accessed 18 September 2020. [Google Scholar]
  31. Moore M.W., Barner J.R. Sexual minorities in conflict zones: a review of the literature. Aggress. Violent Behav. 2017;35:33–37. [Google Scholar]
  32. Mouthaan J., Sijbrandij M., Reitsma J.B., Gersons B.P., Olff M. Comparing screening instruments to predict posttraumatic stress disorder. PLoS ONE. 2014;9(5):e97183. doi: 10.1371/journal.pone.0097183. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Ni M.Y., Kim Y., McDowell I., Wong S., Qiu H., Wong I.O., Galea S., Leung G.M. Mental health during and after protests, riots and revolutions: a systematic review. Aust. N. Z. J. Psychiatry. 2020;54(3):232–243. doi: 10.1177/0004867419899165. [DOI] [PubMed] [Google Scholar]
  34. Ni M.Y., Yao X.I., Leung K.S., Yau C., Leung C.M., Lun P., Flores F.P., Chang W.C., Cowling B.J., Leung G.M. Depression and post-traumatic stress during major social unrest in Hong Kong: a 10-year prospective cohort study. Lancet. 2020;395(10220):273–284. doi: 10.1016/S0140-6736(19)33160-5. [DOI] [PubMed] [Google Scholar]
  35. Nolen-Hoeksema S., Watkins E.R. A heuristic for developing transdiagnostic models of psychopathology: explaining multifinality and divergent trajectories. Perspect. Psychol. Sci. 2011;6(6):589–609. doi: 10.1177/1745691611419672. [DOI] [PubMed] [Google Scholar]
  36. Nolen-Hoeksema S., Wisco B.E., Lyubomirsky S. Rethinking rumination. Perspect. Psychol. Sci. 2008;3(5):400–424. doi: 10.1111/j.1745-6924.2008.00088.x. [DOI] [PubMed] [Google Scholar]
  37. Polit D.F., Beck C.T. The content validity index: are you sure you know what's being reported? Critique and recommendations. Res. Nurs. Health. 2006;29(5):489–497. doi: 10.1002/nur.20147. [DOI] [PubMed] [Google Scholar]
  38. Rajkumar R.P. COVID-19 and mental health: a review of the existing literature. Asian J. Psychiatr. 2020;52 doi: 10.1016/j.ajp.2020.102066. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Robles, P. Hong Kong protests: key events from Hong Kong's anti-government protests. South China Morning Post; 2019 [accessed Jan 9, 2021]. https://multimedia.scmp.com/infographics/news/hong-kong/article/3032146/hong-kong-protests/index.html Last updated Dec 9, 2019.
  40. Russell S.T., Fish J.N. Mental health in lesbian, gay, bisexual, and transgender (LGBT) youth. Annu. Rev. Clin. Psychol. 2016;12:465–487. doi: 10.1146/annurev-clinpsy-021815-093153. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Silove D., Liddell B., Rees S., Chey T., Nickerson A., Tam N., Steel Z. Effects of recurrent violence on post-traumatic stress disorder and severe distress in conflict-affected Timor-Leste: a 6-year longitudinal study. Lancet Glob. Health. 2014;2(5):e293–e300. doi: 10.1016/S2214-109X(14)70196-2. [DOI] [PubMed] [Google Scholar]
  42. Steyerberg E.W. second ed. Springer Nature; Cham, Switzerland: 2019. Clinical Prediction Models: a Practical Approach to Development, Validation, and Updating. [Google Scholar]
  43. Tang W., Hu T., Hu B., Jin C., Wang G., Xie C., Chen S., Xu J. Prevalence and correlates of PTSD and depressive symptoms one month after the outbreak of the COVID-19 epidemic in a sample of home-quarantined Chinese university students. J. Affect. Disord. 2020;274:1–7. doi: 10.1016/j.jad.2020.05.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Torales J., O'Higgins M., Castaldelli-Maia J.M., Ventriglio A. The outbreak of COVID-19 coronavirus and its impact on global mental health. Int. J. Soc. Psychiatry. 2020;66(4):317–320. doi: 10.1177/0020764020915212. [DOI] [PubMed] [Google Scholar]
  45. van der Meer C.A.I, te Brake H., van der Aa N., Dashtgard P., Bakker A., Olff M. Assessing psychological resilience: development and psychometric properties of the English and Dutch version of the resilience evaluation scale (RES) Front. Psychiatry. 2018;9:169. doi: 10.3389/fpsyt.2018.00169. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Wong S.M., Hui C.L., Wong C.S., Suen Y.N., Chan S.K., Lee E.H., Chen E.Y. Mental health risks after repeated exposure to multiple stressful events during ongoing social unrest and pandemic in Hong Kong: the role of rumination: Risques pour la santé mentale après une exposition répétée à de multiples événements stressants d’agitation sociale durable et de pandémie à Hong Kong: le rôle de la rumination. Can. J. Psychiatry. 2020 doi: 10.1177/0706743720979920. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Wu K.K., Leung P.W., Wong C.S., Yu P.M., Luk B.T., Cheng J.P., Wong R.M., Wong P.P., Lui J.C., Ngan J.C. The Hong Kong survey on the epidemiology of trauma exposure and posttraumatic stress disorder. J. Trauma. Stress. 2019;32(5):664–676. doi: 10.1002/jts.22430. [DOI] [PubMed] [Google Scholar]
  48. Ye B., Zhou X., Im H., Liu M., Wang X.Q., Yang Q. Epidemic rumination and resilience on college students' depressive symptoms during the COVID-19 pandemic: the mediating role of fatigue. Front. Public Health. 2020;8 doi: 10.3389/fpubh.2020.560983. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Ying L., Wu X., Lin C., Jiang L. Traumatic severity and trait resilience as predictors of posttraumatic stress disorder and depressive symptoms among adolescent survivors of the Wenchuan earthquake. PLoS ONE. 2014;9(2):e89401. doi: 10.1371/journal.pone.0089401. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Youden W.J. Index for rating diagnostic tests. Cancer. 1950;3(1):32–35. doi: 10.1002/1097-0142(1950)3:1<32::aid-cncr2820030106>3.0.co;2-3. [DOI] [PubMed] [Google Scholar]
  51. Yu X., Zhang J. Factor analysis and psychometric evaluation of the Connor-Davidson Resilience Scale (CD-RISC) with Chinese people. Soc. Behav. Pers. 2007;35(1):19–30. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

mmc1.docx (19.9KB, docx)

Articles from Psychiatry Research are provided here courtesy of Elsevier

RESOURCES