Assessment of early neurocognitive functioning increases the accuracy of predicting chronic PTSD risk

Abstract

Post-traumatic stress disorder (PTSD) is a protracted and debilitating consequence of traumatic events. Identifying early predictors of PTSD can inform the disorder’s risk stratification and prevention. We used advanced computational models to evaluate the contribution of early neurocognitive performance measures to the accuracy of predicting chronic PTSD from demographics and early clinical features. We consecutively enrolled adult trauma survivors seen in a general hospital emergency department (ED) to a 14-month long prospective panel study. Extreme Gradient Boosting algorithm evaluated the incremental contribution to 14 months PTSD risk of demographic variables, 1-month clinical variables, and concurrent neurocognitive performance. The main outcome variable was PTSD diagnosis, 14 months after ED admission, obtained by trained clinicians using the Clinician-Administered PTSD Scale (CAPS). N = 138 trauma survivors (mean age = 34.25 ± 11.73, range = 18–64; n = 73 [53%] women) were evaluated 1 month after ED admission and followed for 14 months, at which time n = 33 (24%) met PTSD diagnosis. Demographics and clinical variables yielded a discriminatory accuracy of AUC = 0.68 in classifying PTSD diagnostic status. Adding neurocognitive functioning improved the discriminatory accuracy (AUC = 0.88); the largest contribution emanating from poorer cognitive flexibility, processing speed, motor coordination, controlled and sustained attention, emotional bias, and higher response inhibition, and recall memory. Impaired cognitive functioning 1-month after trauma exposure is a significant and independent risk factor for PTSD. Evaluating cognitive performance could improve early screening and prevention.

INTRODUCTION

About 20% of patients who are admitted to the emergency department (ED) after potentially traumatic event develop non-remitting, long-lasting, burdensome post-traumatic stress disorder (PTSD) [1, 2]. The early aftermath of trauma exposure might offer a window of opportunity for preventive interventions [3], which optimally should target survivors at significant risk [3]. Evaluating PTSD risk, however, is compounded by the disorder’s complexity, heterogeneous course [4, 5] and uncertainty regarding underlying mechanisms [6]. Published predictive models to-date encompassed demographics (e.g., gender, education, lifetime mental disorders) [6] and early clinical manifestations (such as PTSD, depression or anxiety symptoms’ severity) [1, 7-10]. Studies have additionally shown that neurocognitive dysfunction, shortly after trauma exposure contributes to PTSD risk [6, 11-13]. Dysfunctions associated with PTSD risk included verbal learning, short-term and declarative memory, working memory, information processing speed [14, 15], attention, executive functioning [16, 17], altered response inhibition, attentional switching, and cognitive flexibility [18-22]. It is unclear, however, if the contribution of neurocognitive factors to PTSD risk is redundant with that of demographics (e.g., education level) and early symptoms severity, or else if they, or some of which, contribute independently, properly increasing PTSD likelihood, and thus potentially informing its pathogenesis.

Examining the contribution of multiple heterogeneous predictors and determining the unique non-redundant contribution of each requires statistical methods that accommodate predictors’ eventual multicollinearity, and account for non-linear associations among them [23]. Advanced computational approaches in statistical learning [23], such as extreme gradient boosting (XGB) algorithm [24], can effectively address these challenges [23], and thereby reveal and validate the independent contribution of single- or sets of candidate features. Evaluating the predictive value of diverse predictors additionally requires labor-intensive and methodologically challenging longitudinal studies of sufficient length (from acute responses to chronic PTSD), sufficient sample size, limited attrition, documented retention-bias, and reliable, clinically informative metrics (e.g., structured clinical interviews rather than self-report questionnaires) [25-27]. To date, these requirements have rarely been met.

The present study addresses these computational- and study-design limitations by following 138 intensely documented trauma survivors for 14 months, carefully accounting for selection and attrition biases, and using previously validated, clinically meaningful metrics. Specifically, it evaluates the extent to which a panel of neurocognitive performance indicators uniquely contributes to the discriminatory accuracy of classifying PTSD status by demographics and clinical predictors. We also evaluated which neurocognitive domains specifically contribute to PTSD risk, and whether such unique contribution remains significant in a subgroup of individuals who meet 1-month PTSD symptoms criteria—a group previously shown at high risk for chronic PTSD and likely beneficial of early interventions [28, 29].

METHODS AND MATERIALS

Data informing this report was collected as part of the NIMH-funded Neurobehavioral Moderators of Post-traumatic Disease Trajectories study (MH103287). The data was collected between January 2015 and March 2020. The study’s design and methodologies have been previously published [27] and those informing this work are summarized below.

Participants

Inclusion per ED records.

Subject participants in this study were 18 to 65 years old adult civilians consecutively admitted to Tel Aviv Sourasky Medical Center (TASMC) ED between January 2015 and November 2019 after one of the following events: motor-vehicle accident, bicycle accident, physical assault, robbery, hostilities, electric shock, fire, drowning, work accident, terror attack or a large-scale disaster.

Participants were included in the study if they expressed distressing PTSD criteria 1 month after the traumatic event and provided informed consent. Participants were not included if they had an ED notation of severe head injury, coma upon ED admission, medical condition that interfered with their ability to provide informed consent or apprehend the study’s procedures, a diagnosis of PTSD prior to ED admission, current substance abuse disorder, current suicidal ideations, lifetime psychotic illness, conditions precluding MRI scanning (pacemaker, metal implants, known claustrophobia, permanent makeup or large tattoos) or medical/psychological conditions that constituted treatment priority. Participants provided an oral assent to the study’s screening telephone interview and written informed consent upon attending a subsequent diagnostic and eligibility ascertainment clinical interview.

Instruments

Diagnostics.

The Clinician Administered PTSD Scale (CAPS) provided PTSD diagnosis and symptom severity estimates. To maintain continuity with decades of DSM-IV-based PTSD research, and following evidence of non-overlapping PTSD-diagnosed groups selected DSM IV or DSM 5 criteria, and a consequent recommendation to use “broader” cross-templates PTSD definitions for empirical research [30-32] we administered the CAPS-IV and CAPS 5 simultaneously, using a combined clinical interview schedule, and scored both DSM-IV and DSM 5 CAPS’ items (CAPS-IV and CAPS-5 [33, 34]). A positive diagnosis of PTSD was inferred when a participant met either DSM-IV or DSM-5 PTSD diagnostic criteria or, in line with previous recommendations Weathers, Ruscio [35] endorsed CAPS-IV symptom severity of ≥40.

The structured clinical interview for DSM IV (SCID-IV) [36] was used to identify current and lifetime Axis I disorders [37, 38], of which current major depression and any current anxiety disorder were considered for this study. All diagnostic interviews were administered by trained and certified clinicians and corroborated, when required, by bi-weekly consensus meeting with the study PI (AS).

Psychometrics.

The PTSD checklist IV for civilians (PCL) [39] evaluated concurrent PTSD symptoms severity during clinical assessment.

The beck depression inventory (BDI-II) [40] evaluated current depressive symptoms’ severity.

The beck anxiety inventory (BAI) [41] evaluated current anxiety symptoms.

The clinical global impression instrument (CGI-P) [42] evaluated patients’ perception of their symptom severity on a 1 to 7 scale (respectively, “normal feeling” to 7 “the worst feeling there is”).

Neurocognitive assessment.

The WebNeuro, a previously validated web-based cognitive functioning testing battery, evaluated participants’ performance in 11 cognitive domains: Motor Coordination, Processing Speed, Sustained Attention, Controlled Attention, Cognitive Flexibility, Response Inhibition, Working Memory, Recall Memory, Executive Function, Emotion Identification, and Emotional Bias [43] (detailed in [27]). WebNeuro tests were administered in the laboratory. Cognitive functioning performance were calculated by WebNeuro’s dedicated software, standardized for age and years of education and expressed as z-scores (see table 1 in ref. [27]).

Procedure

The hospital’s ED electronic records were available to the study personnel within 24 h of ED admission. Within these records, an ED “trauma” notation generated initial screening contacts starting 3 days after the ED.

Telephone screening.

Eligible ED participants were contacted by telephone within 10–14 days of ED admission. Oral assent was obtained prior to starting the interview, else the interviewer thanked the subject and terminated the interview. The interviewers firstly ascertaining the occurrence of a psychologically traumatic event, associated distress, availability for the study and salient exclusion criteria specified above (alias a 5–10 min “short interview”). Individuals with ascertained traumatic event, continued to 20–30 min “long interview.” The long interviews evaluated PTSD symptom severity using the PCL, study availability and exclusion criteria as specified above. Participants were invited for the clinical assessments if they endorsed distressing early PTSD symptoms in the telephone screening interview.

Clinical interviews.

The clinical interviews took place 23.9 ± 8.20 after ED admission by trained clinicians. PTSD symptoms severity and status were identified using the Clinician Administered PTSD Scale. Participants were invited to participate in the study if they had CAPS-based full or subthreshold (3 out of 4 symptom criteria) PTSD in the clinical assessments.

Current and lifetime disorders were identified using the SCID as above. Additionally, the interviewers, re-assessed, in-person, the presence of previously unnoticed study exclusion criteria. Eligible participants were invited to continue the study (i.e., neurocognitive assessment and functional magnetic resonance imaging (fMRI) session (scheduled a week later).

Statistical analysis

Outcome measure.

The study’s primary outcome is the PTSD diagnostic status 14 months after ED admission per diagnostic criteria specified above.

Predictor variables.

Predictor variables include demographics, clinical and neurocognitive measures collected 1-month after trauma exposure, evaluated stepwise as specified below: The first step included demographic information (age, gender, marital status, years of education, and trauma type) and clinical data (self-report measures of PCL, CGI-P, BDI-II, BAI; present or past major depression or any anxiety disorder). In a second step, we added patients’ cognitive performance scores in the eleven neurocognitive domains specified above. The stepwise approach aims to facilitate comparisons between distinct sets of candidate predictors such as sociodemographic-clinical vs. cognitive functioning and to examine the added discriminatory accuracy of neurocognitive functioning in addition to sociodemographic-clinical information alone.

Data pre-processing.

Categorical variables were transformed into numeric values and transformed into one-hot encoded numeric arrays using the OneHotEncoder function scikit-learn [44]. Continuous variables were standardized using the function StandardScaler in scikit-learn and missing values were imputed using k-nearest neighbor estimation (KNNImputer in scikit-learn) with k equals 5.

Model development.

We used XGB algorithm [24] in scikit-learn package 0.23.2 [44] using python 3.7.9. We randomly split the data into a model discovery dataset including 75% of the cases (“training set”) and a separate dataset including the remaining 25% of the cases for model validation (“holdout test set”). The random split was stratified for the outcome variable. We used 10-fold cross-validation on the training set to assess the bias-variance trade-off and to gauge the extent of potential “overfitting” of the model [45].

Model evaluation.

The predictive performance was assessed in terms of discriminatory accuracy and calibration. Discrimination is reported in terms of precision, recall, f1, confusion matrix, and the Area under the ROC Curve (AUC). Discriminatory accuracy was further evaluated using calibration plots and Hosmer-Lemeshow statistic [46]. In the stepwise approach, we first compared the predictions based on demographic and clinical data (alias “clinical model”) with the predictions of a second prediction model (alias: “full model”) adding neurocognitive performance in the eleven domains as candidate predictors in addition to the demographic and clinical data.

Given the importance of predicting from acute PTSD (i.e., the ones likely to raise early clinical concern), we repeated this analysis for a subgroup of participants who met PTSD diagnostic criteria specified above at the first (1-month) assessment session.

To formally test whether the full model is significantly better than the clinical model, we used a bootstrap resampling-based significance test [47, 48] to compare the discriminatory performance of the two models.

Variable importance.

To further describe which features of each information set (demographic, clinical data, and neurocognitive data) are most influential in predicting 14 months’ PTSD, we calculate a variable importance metric using SHapley Additive exPlanation (SHAP) values [49]. This approach ranks all variables in order of their contribution to prediction and thereby illustrates how distinct features can inform individual predictions.

RESULTS

Screening, enrollment, and attrition

Figure 1 summarized the study’s participants identification, screening, and successive evaluations. Informed by the ED electronic records, we attempted 4058 telephone calls within 14 days of ED admission, reached and obtained informed assent from 3476 participants, confirmed the occurrence of psychologically traumatic event, participants availability for the study and absence of exclusion criteria in n = 1351, and invited 435 participants to attend clinical interviews (see Fig. 1 for details). Of the latter, 300 attended the interviews and 135 declined. Of those 300 individuals, n = 171 attended the first fMRI assessment and were formally enrolled, and n = 138 attended the 14 months’ clinical and fMRI assessment and are included in the current analysis (alias’ study completers’).

Fig. 1. Consort diagram.

Flow chart depicting the inclusion and exclusion of patients.

*Other exclusions (n=10) include: Serious medical/surgical condition requiring clinical attention (n=5). Chronic PTSD before current event (n=2), Substance use disorder - current (n=l). Head injury (n=l) and No traumatic event *n=l)

Evaluating sampling bias

The study sample’s demographic and clinical characteristics are presented in Table 1. By design, and towards obtaining large-enough number of PTSD patients at 14 months, participants with more severe PTSD symptoms were elected for inclusion in early screening and clinical assessment. To evaluate, however, eventual sampling bias as to age, gender, trauma type, we performed a series of comparisons between individuals included and those not included at each step on variable known to contribute to PTSD risk. Among participants reached for telephone assessments (n = 3476), those included in the long telephone interview (n = 1351) did not differ from those screened and not included (n = 2707) in age (35.51 ± 11.91 vs. 35.59 ± 11.88 years), gender distribution (55.3% females vs. 56.2%), and Trauma Type (92.8% MVA’s vs. 92.6%; all p-values > 0.05).

Table 1.

Participants’ demographic and clinical characteristics.

Demographics (at study onset)
Measure	M	SD
Age at onset	34.25	11.73
Gender (F:M)	73 (52.9%):65 (47.1%)
Marital status (S:M:D)	89 (69.0%):27 (20.9%):13 (10.1%)
Education (years)	14.35	2.67
Trauma type (%)	MVA: n = 122 (88.4%) Assault n = 9 (6.5%) 7 Other events n = 7 (5.1%)
Clinical variables	1-month post-trauma		14-month post-trauma
Measure	M	SD	M	SD
PCL total score	45.55	14.10	30.23	13.19
BDI-II total score	16.16	8.63	9.73	8.61
BAI total score	19.69	11.84	10.55	11.17
CGI-P total score	3.44	1.43	2.47	1.61
CAPS-IV total score	50.85	22.61	21.89	20.8
CAPS-5 total score	24.35	11.70	10.78	10.38
Met PTSD diagnosis	103 (74.6%)		33 (23.9%)

N = 9 participants did not provide information about their marital status. MVA motor vehicle accidents, PCL PTSD checklist, BDI beck depression inventory, BAI beck anxiety inventory, CGI clinical global impression, CAPS Clinician Administered PTSD Scale, IV for DSM IV and 5 for DSM 5.

Similarly, individuals invited for a first clinical interview (n = 435) did not differ from those who were not invited (n = 916) in age (34.11 ± 11.16 vs. 34.82 ± 12.01 years), gender distribution (50.9% vs. 51.5% females) and trauma type (89.3% vs. 90.1% MVA’s; all p-values > 0.05).

Finally, among 171 participants enrolled at 1 month, study completers (n = 138; i.e., those who were seen again 14 months after ED admission), did not differ from those lost to follow-up (n = 33) in age (33.97 ± 10.48 years vs. 34.25 ± 11.73), trauma type (87.9% MVA’s vs. 88.4% MVA’s), gender distribution (53%; (73 of 138) vs. 39% (13 of 33) females), education (14.14 ± 2.57 vs. 14.35 ± 2.67 years of schooling;all p-values > 0.05), and 1-month PTSD symptom severity (CAPS-IV total score = 56.52 ± 20.48 and 50.85 ± 22.62 (t (168) = 1.31, p = 0.191).

Comparing clinical-demographic model with the full model

In the hold-out set (n = 35) of n = 138 completers, the XGB algorithm applied to demographic and clinical variables at 1-month achieved a discriminatory accuracy for classifying PTSD diagnosis 14 months after ED admission with an AUC of 0.68 (weighted average precision = 0.62, weighted average recall = 0.63, weighted average f1-score = 0.62) (Fig. 2A). Using the full model (demographics, clinical and neurocognitive performance) the discriminatory accuracy improved to an AUC of 0.88 (weighted average precision = 0.83, weighted average recall = 0.77, weighted average f1-score = 0.79). Based on 10,000 bootstrapped resamples, the bootstrap resampling significance test was D = 1.84, p = 0.066. The model predictions were assessed for calibration within 10 bins of probability deciles to gauge the agreement between observed PTSD diagnosis at 14 months follow-up and the predictions. The non-significant Hosmer–Lemeshow test (X² = 10.76, df = 8, p = 0.22, g = 10) shows that there is no evidence of poor fit.

Fig. 2. Discriminatory accuracy for classifying PTSD diagnosis.

A ROC curve on the holdout test set for predicting PTSD diagnosis at 14 months by using all variables and only demographic and clinical variables (alias “clinical variables” in Fig. 2). B ROC curve for predicting PTSD diagnosis at 14 months by using all variables vs. only demographic and clinical variables in the subset of participants who met PTSD diagnostic criteria at 1-month post-trauma in the holdout test set.

Examining a subset of participants who met 1-month PTSD symptom criteria (N = 103), the predictive accuracy of demographics and clinical variable at 1-month for classifying PTSD diagnosis at 14 months yielded an AUC of 0.53 (weighted average precision = 0.55, weighted average recall = 0.52, weighted average f1-score = 0.53) and, upon including neurocognitive functioning had an AUC of 0.83 (weighted average precision = 0.78, weighted average recall = 0.78, weighted average f1-score = 0.78). The difference between these two models (demographic and clinical data with versus without neurocognitive data; see Fig. 2B) was significant (D = 2.11, p = 0.035, based on 10,000 bootstrapped resamples). A non-significant Hosmer-Lemeshow test (X² = 3.29 df = 8, p = 0.91, g = 10) has shown no evidence of poor fit.

The 10-fold cross-validation yielded an AUC of 0.78 for the full model among all participants, and an AUC of 0.72 for the full model among the subset of participants who met the diagnostic criteria for PTSD 1 month after ED admission.

Variable importance

The variable importance is presented in Figs. 3 and 4. In both study completers and study completers with PTSD, the same neurocognitive domains were consistently the most relevant for identifying patients with high risk for chronic PTSD. Poorer cognitive flexibility, processing speed, motor coordination, controlled and sustained attention, emotional bias, higher response inhibition, and higher recall memory at 1-month post-trauma were associated with a higher likelihood for PTSD 14-month post-trauma.

Fig. 3. Variable importance.

SHAP summary dot plot of the XGB model based on sociodemographic, clinical, and neurocognitive predictor variables for the training set (A) and hold-out test set (B). The higher the SHAP value of a feature, the higher the log odds of the PTSD diagnosis. On the y-axis, the features are sorted by their general feature importance. For each participant and variable, the dots represent how the given value of a variable impacts the classification of the patient to one of the two outcome classes. Dots that are far on the left side shift the classification of a given patient to the class “no PTSD”, whereas dots on the far-right side of the x-axis shift the classification of participants to the class “PTSD”. The color represents the range of the feature values from low (blue) to high (red).

Fig. 4. Variable importance for the subset of participants who met 1-month PTSD symptom criteria.

SHAP summary dot plot of the XGB model based on sociodemographic, clinical, and neurocognitive predictor variables for the training set (A) and hold-out set (B) for the subset of participants who met PTSD diagnostic criteria at 1-month post-trauma. The higher the SHAP value of a feature, the higher the log odds of the PTSD diagnosis. On the y-axis, the features are sorted by their general feature importance. For each participant and variable, the dots represent how the given value of a variable impacts the classification of the patient to one of the two outcome classes. Dots that are far on the left side shift the classification of a given patient to the class “no PTSD”, whereas dots on the far-right side of the x-axis shift the classification of participants to the class “PTSD”. The color represents the range of the feature values from low (blue) to high (red).

DISCUSSION

Our results show that 1 month after trauma exposure, measures of neurocognitive functioning improve the performance of demographic information and clinical symptoms in predicting 14 months PTSD status. Impaired neurocognitive functioning in eight specific domains contributed to the difference. The added contribution of neurocognitive functioning impairment was particularly striking among highly symptomatic trauma-survivors.

These results are in line with previous research findings of an association between early lower neurocognitive performance and higher risk for PTSD development [13]. For example, several studies suggested that deficits in processing speed, controlled- and sustained attention (as found here) are closely linked to PTSD progression [16, 50, 51]. Furthermore, lower cognitive flexibility shortly after trauma exposure predicted more severe PTSD symptoms a year later, and intervention-induced improvement in this domain were associated with greater subsequent clinical improvement [21]. While this work cannot infer on the mechanistic meaning of these neurocognitive predictors, we encourage future research examining their potential in preventive interventions for PTSD.

From a risk prediction perspective, our findings highlight the independent prognostic relevance of early neurocognitive functioning, a finding that emphasizes the relevance of assessing individuals’ cognitive functioning in addition to clinical and demographic variables and as a way to improve the early screening and identification of ED trauma survivors at risk. Importantly, a contribution of neurocognitive evaluation to PTSD prediction particularly concerned survivors with salient 1-month PTSD symptoms, who are at higher current distress and long-term PTSD risk.

Additionally, the association of early neurocognitive deficits with 14 months PTSD implies a mechanistic contribution to PTSD emergence and persistence. Such contribution is in line with previous findings of early symptoms persistence among survivors with mild traumatic brain injury (MTBI) and with the replicated link between education levels and PTSD risk [52, 53]. Moreover, such deficits likely implicate brain circuits and functions that extend beyond those mediating fear learning and extinction (i.e., the amygdala and vmPFC) in PTSD development;in line with published models of PTSD pathogenesis that include the hippocampus, dorsolateral and prefrontal cortical regions others [6, 54, 55].

While this work cannot separate pre-trauma neurocognitive traits from trauma-induced, or amplified, neurocognitive difficulties, it nonetheless shows a significant contribution, worthy of early detection and informing potential therapy or palliation. Either way, our finding suggests that screening for trauma type, gender, and early clinical symptoms does not exhaust the potential of early information to predict PTSD, and thus the addition of neurocognitive functioning yields clinically important information for the prognosis of the symptom development of trauma-survivors.

While prior studies have discussed neurocognitive deficits as predictors of poorer outcome of psychotherapy [56-58], only a few interventions exist that directly target cognitive functioning in PTSD patients [13, 59]. Notwithstanding, studies have shown that it is, in principle, possible to modulate neurocognitive performance [21, 60] and that neurocognitive deficits are ameliorated after successful trauma-focused therapy [61]. Our results raise the possibility that improving early neurocognitive functioning could mitigate PTSD risk. They thereby identify hypothetical mechanistic targets for preventive interventions for PTSD.

A major barrier to conducting early neurocognitive assessment is its eventual cost and the dearth of qualified personnel. As this work has shown, web-based neurocognitive assessment batteries efficiently captured an incremental PTSD risk. This is in line with new developments of web-based digital biomarkers and digital phenotyping tools [62] that require limited resources and personnel qualification and time. These new computational advancements may facilitate the implementation of such neurocognitive assessments shortly after trauma exposure. Such methods should be further validated, towards offering a promising opportunity for larger scale implementation.

Strength and limitations

This work is limited by addressing a sample of ED-admitted symptomatic civilians enrolled in a single medical center after single-impact short traumatic events. Additionally, participants with head trauma and severe lifetime and mental disorders were not included. Finally, the use of web-based neurocognitive instrument not-especially built to measure traumatic stress consequences is another limitation.

Within these limitations, however, sampling and retention biases have been reasonably good and thus participants included in this work fairly well represent our ED trauma admissions’ age, gender and trauma type, and those lost to follow up (n = 33) do not differ from those who remained in the study (n = 138). Participants were screened for early PTSD symptoms to enrich the sample and increase the likelihood of endpoint PTSD. Further, whilst the diagnosis of PTSD varies upon DSM taxonomic classification and is frequently criticized [63, 64], we attempted to optimize participants’ characterization by combining two “gold standard” structured clinical instruments capturing both DSM IV and 5 PTSD. We additionally followed subjects for 14 months—a time period indicative of a chronic course [65, 66]. We therefore believe that our results can be reliably generalized to ED trauma among adults; a frequent modality of trauma exposure among civilians, and that they can safely guide further explorations.

CONCLUSION

This work shows an independent contribution of early neurocognitive functioning to chronic PTSD risk. Further studies are needed to better understand the origin of the observed neurocognitive deficits, i.e., the extent to which they represent pre-trauma, “vulnerability” attribute, a reaction to traumatic stress or a mixture of both.