Are We There Yet? Evaluating the Evidence for Neuroimaging-Based Biotypes of Psychiatric Vulnerability in the Acute Aftermath of Trauma

Abstract

Objective:

The weak link between subjective symptom-based diagnostic methods for post-traumatic psychopathology and objectively measured neurobiological indices forms a barrier to the development of effective personalized treatments. In response, recent studies aimed to stratify psychiatric disorders by identifying consistent subgroups based on objective neural markers. Along these lines, a promising study by Stevens et al. (2021) identified distinct brain-based biotypes associated with different longitudinal patterns of post-traumatic symptoms. Here, we conducted a conceptual non-exact replication of this work, using a comparable dataset from a multimodal longitudinal study of recent trauma survivors.

Methods:

A total of 130 participants (age=33.61±11.21 years,48% females) admitted to a general hospital emergency department following traumatic exposures underwent demographic, clinical, and neural assessments at 1-, 6-, and 14-months post-trauma. All analyses followed the pipeline as outlined in the original study and in collaboration with its authors.

Results:

Task-based functional MRI obtained one-month post-trauma identified four clusters of individuals based on profiles of neural activity reflecting threat and reward reactivity. These clusters were not identical to the previously identified brain-based biotypes and were not associated with prospective symptoms of post-traumatic psychopathology.

Conclusions:

Overall, our findings suggest that the original brain-based biotypes of trauma resilience and psychopathology may not generalize. Thus, caution is warranted when attempting to define subtypes of psychiatric vulnerability using neural indices before treatment implications can be fully realized. Future replication studies are needed to identify more stable and generalizable neuroimaging-based biotypes of post-traumatic psychopathology.

Introduction

As opposed to most fields of medicine, diagnosis in psychiatry remains restricted to subjective self-reports and observable symptoms¹. The biological mechanisms underlying psychiatric disorders are complex and still poorly understood. The weak link between established diagnostic methods and objectively measured biological indices² forms a barrier to the development of effective personalized treatments³. In response, many studies have aimed to stratify psychiatric disorders, attempting to identify consistent subgroups based on objective biological markers^4,5.

Accordingly, recent innovative work by Stevens et al. (2021)⁶ aimed to discover brain-based biotypes of trauma resilience and psychopathology in the acute aftermath of trauma, using data from the AURORA⁷ longitudinal study of trauma survivors. In a discovery cohort (n=69), using functional MRI (fMRI) data of simple and widely used tasks probing threat reactivity⁸, reward reactivity⁹, and inhibitory engagement¹⁰, the authors identified four different clusters of individuals. In an internal replication cohort (n=77), three clusters were replicated: ‘Reactive/Disinhibited’, ‘Low Reward/High Threat’, and ‘Inhibited’ cluster (see Fig. 2 in original paper⁶). Those replicated clusters were associated with different longitudinal symptom trajectories of post-traumatic stress disorder (PTSD) and anxiety. Interestingly, the cluster of individuals showing heightened reactivity to both threat and reward was associated with the highest levels of subsequent PTSD and anxiety symptoms⁶.

Figure 2. fMRI Profiles of the Four Clusters among Recent Trauma Survivors (n=130).

Panel a show the region-of-interests (ROIs) covariance matrix revealing linear associations between z-scored contrast estimates extracted from the seven ROIs across the two fMRI tasks. For threat reactivity (‘T’), fMRI activation was extracted bilaterally from the amygdala (Amy), insula, dorsal and subgenual anterior cingulate cortex (dACC and sgACC, respectively) for the contrast of fearful > neutral faces. For reward reactivity (‘R’), bilateral activation was extracted from the nucleus accumbens (NAcc), amygdala (Amy) and orbitofrontal cortex (OFC) for the contrast of rewards > punishments. The matrix is ordered in the exact order as in Fig. 2 panels A and B in the original paper⁶. Correlation coefficients (R-values) are indicated on a scale ranging from −1 (blue) to +1 (red). Panel b shows the dendrogram illustrating the final cluster solution with four clusters (marked by different colors). Panel c show Cluster differences (mean and standard deviation) for standardized contrast estimates extracted from the ROIs across the threat (fearful > neutral faces) and reward (rewards > punishments) contrasts.

Multimodal longitudinal studies of the post-traumatic stress response are rare since they present formidable technical and conceptual challenges¹¹. These include obtaining a large enough sample of recent trauma survivors; optimal timing of assessments and sufficient follow-up duration to address the critical stages of PTSD longitudinal development¹²; minimizing subjects’ burden, and reaching out to, enrolling, evaluating, and retaining sensitive clinical populations. To the best of our knowledge, the only existing dataset comparable to the AURORA study to date comes from the “Neurobehavioral Moderators of Post-traumatic Disease Trajectories” study (NIMH-R01-MH103287, 2015–2020).

Here, using this similar dataset of recent trauma survivors and closely matched analytic processes, we aimed to perform a conceptual non-exact replication of Stevens et al.⁶ work. Independent replications are needed as they represent a fundamental part of science, leading to greater confidence in previously reported findings^13,14. This is particularly relevant to neuroimaging studies due to their large analytical variability¹⁵, and to the field of psychiatry suffering from the known heterogeneity problem¹⁶. Nevertheless, engaging in replication studies is often undervalued, can be difficult to publish, and has few direct incentives for researchers¹⁷.

The main objective of this work was to examine whether the previously identified brain-based clusters can generalize to our independent sample of trauma survivors. If replicated, we aimed to test whether a cluster characterized by heightened reactivity to both threat and reward would be similarly associated with increased subsequent symptoms of PTSD and anxiety, thus demonstrating the stability of a biological phenomenon across similar measures of psychopathology.

Methods

Data informing this work was collected between 2015 to 2020 as part of the NIMH-funded “Neurobehavioral Moderators of Post-traumatic Disease Trajectories” study (MH103287). The study’s design and detailed methodologies have been previously published¹¹, and those informing this work are summarized below. Overall, we conducted all analyses as close as possible to the published analytic pipeline of the original work, which aimed to replicate⁶, including similar preprocessing and analysis of neuroimaging data using fMRIprep¹⁸ and SPM-12¹⁹, same anatomical regions-of-interest (ROIs), and identical clustering analysis. We used an adapted version of the R code applied in the original work⁶, kindly provided by the corresponding author (JS). The methods section was organized in a similar way to the original work⁶ to further facilitate the comparison between the two studies.

Participants.

Potential subjects for this study were 18 to 65 years old adult civilians consecutively admitted to a general hospital emergency department (ED) following one of the following events: motor-vehicle accident (MVA), bicycle accident, physical assault, robbery, hostilities, electric shock, fire, drowning, work accident, terror attack or a large-scale disaster. Individuals who sustained head injuries, were in a coma upon ED admission, or were not able to provide informed consent or apprehend the study’s procedures were excluded from the study. Participants with conditions precluding MRI scanning (e.g., pacemaker, metal implants, large tattoos), those with current substance use disorder, current suicidal ideations, or lifetime psychotic disorder were similarly excluded. In contrast to the AURORA study, this study excluded individuals with a prior diagnosis of PTSD. All participants provided oral consent to the study’s screening telephone interview, and written informed consent upon attending a subsequent diagnostic and eligibility ascertainment clinical interview.

Clinical Assessments.

A comprehensive clinical interview was conducted by trained and certified clinical interviewers, using the Clinician-Administered PTSD Scale (CAPS)^20,21, to assess PTSD diagnosis and severity at each time-point. To maintain continuity with decades of DSM-IV-based PTSD research, following evidence of nonoverlapping PTSD-diagnosed groups selected DSM-IV or DSM-5 criteria, and a consequent recommendation to use “broader” cross-templates PTSD definitions for empirical research^22–24, we administered a combined clinical interview scoring both CAPS-4 and CAPS-5 items^20,21. A positive diagnosis of PTSD was inferred when a participant met either DSM-IV or DSM-5 PTSD diagnostic criteria or, in line with previous recommendations²⁵, endorsed CAPS-4 total score of ≥ 40. As a secondary continuous measure of PTSD symptom severity, participants completed the PTSD Checklist (PCL)²⁶, a 17-item self-report questionnaire corresponding to the DSM symptom criteria for PTSD. As a continuous measure of anxiety symptom severity, participants completed the Beck Anxiety Inventory (BAI)²⁷, a 21-item self-report questionnaire measuring physical and cognitive anxiety symptoms.

MRI.

Acquisition.

Whole-brain functional and anatomical images were acquired using a 3.0 Tesla Siemens MRI system (MAGNETOM Prisma, Germany) with a 20-channel head coil at the Sagol Brain Institute, Wohl Institute for Advanced Imaging, Tel-Aviv Sourasky Medical Center. Functional images were acquired in an interleaved order (anterior to posterior), using a T2*-weighted gradient-echo planar imaging pulse sequence (TR/TE=2000/28ms, flip angle= 90°, voxel size 2.2mm³, FOV=220×220mm, slice thickness=3mm, 36 slices per volume). A T1-weighted three-dimensional anatomical image was also collected, using a magnetization prepared rapid gradient echo (MPRAGE) sequence (TR/TE=2400/2.29ms, flip angle=8°, voxel size 0.7mm³, FOV=224×224mm), enabling optimal localization of the functional effects.

fMRI Tasks.

The face matching task²⁸ probed threat reactivity (similarly to Stevens et al.⁶), and the “Safe-or-Risky Domino Choice” paradigm²⁹ measured reward reactivity (compared to a simpler reward task in Stevens et al.⁶). Nevertheless, both reward reactivity tasks showed similar neural activations in key regions associated with response to reward in the brain (see supplementary results). Unlike the original work, we did not have an fMRI task measuring response inhibition (for a full description of the tasks, see supplementary information).

fMRI Data Preprocessing.

Functional images were preprocessed with fMRIPrep version 1.5.8¹⁸. Functional imaging scans were co-registered to the anatomical T1-weighted images, corrected for motion, spatially realigned, slice-time corrected, normalized to the 2009 ICBM-152 template, and smoothed with a 6-mm kernel (for full details, see supplementary methods).

fMRI Data Analysis.

Similar to the original work ⁶, analysis was performed using SPM version 12¹⁹. The final sample was restricted to participants with good-quality data across all fMRI tasks (n=130; see ‘Procedure’). The ROIs were the same ones as in the original work by Stevens et al.⁶, kindly provided by the corresponding author (JS). They were defined anatomically and included the left and right amygdala, insula, subgenual and dorsal anterior cingulate cortex (sgACC and dACC, respectively), nucleus accumbens (NAcc), and orbitofrontal cortex (OFC) (for full details, see supplementary methods).

Procedure.

A total of 4,058 consecutive ED-admitted trauma survivors were contacted by telephone within 10–14 days after trauma exposure, given information about the study, and provided informed assent (see consort diagram, Fig. 1). Of those, n=3,476 underwent initial screening (i.e., “short” interview), confirmed the occurrence of a psychologically traumatic event and related symptoms, and n=1,351 underwent eligibility assessment (i.e., “long” interview), further assessing acute stress symptoms, suggestive of indicative of chronic PTSD risk³⁰. A total of 435 individuals met the inclusion criteria for this study and did not meet any of the exclusion criteria (see ‘Participants’), hence were invited for an in-person clinical interview. Of the latter, 300 attended the interviews, of which 171 attended an fMRI assessment, both within one-month post-trauma. Of these, 41 individuals were excluded for the following reasons: missing (n=16) or partial (n=5) functional scans of the threat or reward reactivity tasks; poor quality of the functional scans (e.g., movements, artifacts, etc.) (n=6); missing or poor structural scans (n=5); missing or partial behavioral data of the tasks (n=5); not understanding the instructions properly (n=1); or missing clinical data (n=3). A final sample of 130 individuals with valid anatomical and functional brain data from both fMRI tasks were included in this report (see Table 1).

Figure 1. Consort Diagram.

Flow chart depicting the inclusion and exclusion of participants in this report.

Table 1.

Participants’ Demographic and Clinical Characteristics.

Demographic & Clinical Characteristics (n=130)
Measure	M	SD
Age (Years)	36.61	11.21
Education (Years)*	14.25	2.74
CAPS-5 (Total Score)	24.58	11.41
CAPS-4 (Total Score)	51.30	22.45
PCL (Total Score)	45.77	14.39
BAI (Total Score)	19.91	12.63
Measure	n [%]
Gender (F:M)	62 [48%] : 68 [52%]
Marital Status (S:M:D)**	89 [71%] : 24 [19%] : 12 [10%]
Trauma Type	MVA: 115 [88%]
	Assault: 10 [8%]
	Other: 5 [4%]

^*n=11 participants did not report their education level (in years).

^*n=5 participants did not provide information about their marital status.

MVA = Motor vehicle accidents; S=Single, M=Married, D=Divorced; PCL = PTSD Checklist; BDI = Beck Depression Inventory; BAI = Beck Anxiety Inventory; CGI = Clinical Global Impression; CAPS = Clinician Administered PTSD Scale, 4 for DSM-IV and 5 for DSM-5.

Clustering Analysis.

As noted above, we used an adapted version R code applied in the original work⁶ with R version 4.1.1 and RStudio version 1.4.1717. Clustering was conducted on data from the ROIs extracted from the two fMRI tasks (i.e., threat and reward reactivity fMRI tasks), using hierarchical agglomerated clustering, with the cluster package (version 2.1.2) following Ward’s criterion (agnes function). This method is a bottom-up method designed to preserve the existing data structure, without imposing any assumptions of linearity, hence is appropriate for exploratory analysis. The optimal number of clusters was determined using both Silhouette width metric³¹ and Hartigan’s distance metric³².

Cluster analytic algorithms are prone to find clusters even when the underlying data does not contain clusters but is multivariate and normally distributed³³. Therefore, we expanded the analytic plan of Stevens et al.⁶, by testing whether the results of our cluster analysis meaningfully differed from the null hypothesis (H0) that our data did not contain clusters. For this purpose, we used a procedure reported in Dinga et al. (2019)³⁴, based on the one originally proposed in Liu et al. (2008)³³. In this procedure, the null hypothesis is that the data come from a single multi-dimensional Gaussian distribution, that is a distribution with no underlying clusters, with the number of dimensions equal to the number of features included in the clustering analysis (for full details, see supplementary methods).

Analysis of Posttrauma Outcome by Cluster.

First, as different demographic characteristics might influence the cluster solution due to the unconstrained analytic approach, Chi-square tests (categorical variables) and ANOVAs (continuous variables) were used to assess whether demographic characteristics differed between the clusters. Second, Chi-square tests (categorical variables) and ANOVAs (continuous variables) were used to assess whether the clusters are significantly different in 1) PTSD dichotomous diagnosis (PTSD/No-PTSD), 2) PTSD symptom severity (CAPS-4 and CAPS-5 total scores), 3) self-reported PTSD symptoms (PCL), 4) self-reported anxiety symptoms (BAI). Benjamini-Hochberg³⁵ False Discovery Rate (FDR) correction (q<0.05) was calculated to control for multiple comparisons of these different clinical measures. All tests were two-tailed and used a significance threshold of p=0.05.

Results

Participants’ Demographic and Clinical Characteristics.

A sample of 130 recent trauma survivors (mean age=33.61±11.21 years, range=18–64 years, n=62 [48%] females) were included in all the analyses reported below. The most common trauma type among participants was MVA’s (n=115, 88%), while n=10 (8%) experienced an assault/brawl, and n=5 (4%) experienced other traumatic events (for full demographic and clinical characteristics, see Table 1). Similarities and differences in these characteristics between our sample (n=130) and the original work⁶ (discovery: n=69, replication: n=77) are reported in the supplemental results.

Covariance Among fMRI Tasks and Regions of Interest (ROIs).

To assess for feature redundancy, we examined the covariance structure between the tasks and ROIs. In line with Stevens et al.⁶, different ROIs showed positive within-task covariance, but not between tasks (see Fig. 2a). Different ROIs within the same task showed moderate to high correlations, with the highest correlations being between the amygdala’s reactivity to threat and both sgACC and dACC reactivity to threat (for both: r=0.56, p<0.01; Fig. 2a). As in the original paper⁶, similar regions were uncorrelated from one task to another. Specifically, participants’ amygdala reactivity to threat reactivity was not correlated with its reward reactivity (r=0.08, p=0.77; Fig. 2a).

Clustering of Individuals Using Task-Based fMRI at One-Month Post-Trauma.

Hierarchical clustering performed on all 130 participants suggested an optimal solution of k=4 clusters according to Hartigan’s distance metric (Fig. S1.a), and k=2 clusters according to the Silhouette width metric (Fig. S1.b), therefore both solutions were tested. To avoid redundancy, and in line with Stevens et al.⁶, we report the 4-cluster solution here (Fig. 2b), and the 2-cluster solution in the supplemental results (Fig. S2).

Assessment of different fMRI activation patterns among the clusters revealed a subgroup of n=18 individuals showing high reactivity of all brain regions to threat, predominated by the dACC and sgACC, and to reward, predominated by the NAcc (cluster 4 in Fig. 2c). The three other clusters were more similar to each other and were indeed part of the same cluster according to the twocluster solution (clusters 1–3, Fig. 2c). While individuals in cluster 1 (n=28) showed low reactivity to both threat (namely in the dACC and sgACC) and reward (namely in the amygdala), those in cluster 2 (n=44) showed only low reward reactivity (predominated by NAcc) and those in cluster 3 (n=40) showed only high reward reactivity (similarly across the 3 regions), with relatively no threat reactivity (clusters 1–3 in Fig. 2C).

Importantly, as in the original work, the clusters were unrelated to any of the demographic characteristics. There was no significant association between cluster assignment and participants’ age (F=0.492, p=0.688), education years (F=0.412, p=0.745), gender (χ²=0.865, p=0.834), or marital status (χ²=2.032, p=0.236).

Finally, following Dinga et al. (2019)³⁴ and Liu et al. (2008)³³, we tested the statistical significance of the observed Hartigan distance index. In our dataset, the 4-cluster solution showed the optimal Hartigan’s distance index. Using a simulation approach (described in the methods section), we found that this index was not statistically significant (Hartigan’s distance index=18.15, p=0.371; Fig. S1.c). In other words, it is not unusual to observe such an index even when the hierarchical clustering is performed on a multivariate normally distributed dataset with no clusters.

Prospective Trajectories of Mental Health Among the Different Clusters.

The four clusters did not differ in prospective 6-months PTSD dichotomous diagnosis (PTSD/No-PTSD), PTSD symptom severity (CAPS-4 or CAPS-5), self-reported PTSD (PCL) or anxiety (BAI) symptom severity (Fig. 3 & Table 2). Similarly, they did not differ in any clinical measure even later at 14-months post-trauma (Fig. S3 & Table 2). Statistical significance further decreased after applying FDR correction for multiple comparisons³⁵ (for all comparisons: 0.907≤p_FDR≤1.000; Table 2). In summary, there was no association between individuals’ clustering membership and PTSD or anxiety at 6-months (original study’s endpoint) or 14-months post-trauma (this study’s endpoint).

Figure 3. PTSD and Anxiety at 6-months Post-trauma among the Four Clusters of Recent Trauma Survivors (n=130).

Boxplots presenting the four clusters created based on neuroimaging data at 1-month post-trauma and their future clinical symptoms at 6-months post-trauma: Total scores of The Clinician‐Administered PTSD Scale (CAPS-4 in panel a; CAPS-5 in panel b), total scores of the PTSD Checklist List (PCL in panel c), and total scores of Beck Anxiety Inventory (BAI in panel d). T2 = 6-months following ED admission.

*As requested, figures are uploaded to the system as separate files (PDF format).

*Other exclusions (n=10): Serious medical condition requiring clinical attention (n=5), Chronic PTSD before current event (n=2), Current substance use disorder (n=1), Head injury (n=1), No traumatic event (n=1)

Table 2. Prospective Clinical Outcomes among the Four Different Clusters.

The table report results of Chi-square tests (categorical variables) or ANOVAs (continuous variables) that were used to assess clinical diagnosis (PTSD/No-PTSD) and symptom severity (total scores of CAPS4, CAPS-5, PCL, and BAI) differences between the four suggested clusters.

	6-months post-trauma	14-months post-trauma
PTSD Diagnosis (PTSD/No-PTSD)	χ2=2.285, p=0.515, pFDR=1	χ2=0.011, p=0.999, pFDR=0.999
CAPS-4 Total	F=0.411, p=0.746, pFDR=1	F=0.505, p=0.679, pFDR=1
CAPS-5 Total	F=0.558, p=0.644, pFDR=1	F=0.335, p=0.800, pFDR=1
PCL Total	F=0.502, p=0.682, pFDR=1	F=0.443, p=0.723, pFDR=1
BAI Total	F=0.555, p=0.646, pFDR=1	F=0.312, p=0.816, pFDR=0.907

Discussion

In this conceptual non-exact replication and extension of Stevens et al.⁶ work, we failed to replicate the previously identified neuroimaging-based biotypes⁶ or their association to prospective post-traumatic stress symptoms. Despite overall similarities in study design and aims, participants’ characteristics, and fMRI probes, this study is not an exact replication of the original paper⁶. Nevertheless, non-exact replications can provide strong evidence for robustness and external validity of previous findings, demonstrating generalization beyond specific choices of study design and population³⁶. On the other hand, when original findings are not replicated, it is hard to determine whether it is because of the methodological differences or because the original findings were false positive.

There are several potential explanations for our inability to replicate the results of Stevens et al. ⁶, mainly due to methodological differences between the studies. First, replication data was obtained from a single site in Israel, compared to original data collected in several different sites across the US. While some demographic characteristics were similar in both studies (e.g., participant’s age), others differed (e.g., gender and trauma type), and some were not collected in our work (e.g., race/ethnicity, childhood trauma) (see supplemental results & Table 1). Second, while this study specifically screened participants for early PTSD post-traumatic stress symptoms and excluded individuals with prior PTSD diagnoses, the original work did not require such constraints. Third, while our neural data was collected at one-month post-trauma (30.43±9.54 days post-trauma), MRI data in the original work was obtained slightly earlier (21±6 days posttrauma). Fourth, while Stevens et al.⁶ assessed symptoms based on abbreviated self-report tools (a limitation noted by the authors), we used gold-standard structured interviews (CAPS) administered by trained and certified clinicians. These assessments were performed at 1-, 6- and 14-months following trauma exposure, whereas the original study’s data ranged from 30 days pre-trauma (queried retrospectively in the ED) to 6-months post-trauma (5 total assessments). Importantly, a 6-month follow-up duration is a dynamic time-point in the course towards the tangible chronic PTSD subtype³⁷, compared to a 14-month follow-up which is clinically stable and indicative of the chronic disease, as further recovery is marginal¹². Finally, while clustering was originally performed on nine neural measures from three different tasks, here it was based on seven measures from two different tasks (as we did not include an fMRI inhibition task). Further, while both studies used the same fMRI task to probe threat reactivity, different tasks assessed reward reactivity. Nevertheless, both reward tasks showed similar whole-brain activations in the contrast of reward/gain vs. punishment/loss (supplemental results).

The large clinical heterogeneity of post-traumatic psychopathology, together with recent advances in statistical and computational methods, motivated the search for homogeneous PTSD subtypes through data-driven approaches³⁸. Nevertheless, the presumption of distinct and homogeneous subgroups might not be clinically useful nor represent PTSD underlying biology³⁴. For example, most clustering approaches will always yield clusters per definition, regardless of the data structure, even if there are no clusters at all³³. Hence, it is crucial to distinguish biologically/clinically meaningful subtypes from random data fluctuations or noise³³. Here, following a procedure by Dinga et al. (2019)³⁴, we showed that the 4-cluster solution could happen even if the data came from a single Gaussian distribution with no underlying clusters, supporting the fact that four clusters (as shown in Fig. 2c) were not more likely than no clusters at all. This statistical test was not performed by Stevens et al.⁶. Another way to deal with the disadvantages of data-driven approaches is the use of hybrid analytic methods¹⁶, which combine prior knowledge and assumptions (theory-driven, supervised) with data-driven (unsupervised) approaches³⁹.

In conclusion, our results highlight that slight changes in sample characteristics or experimental tasks can have a critical impact on the replicability of neuroimaging-based biotypes and their association to post-traumatic stress symptoms. This is in line with recent work suggesting smaller than expected brain-phenotype associations and large variability across population subsamples⁴⁰, as would be expected from a disorder with over 600,000 potential phenotypes⁴¹. Therefore, studies should carefully specify their design and methodology, to define to which populations results could be generalized until more stable and unified measures are established in psychiatry. Importantly, caution is warranted when attempting to define PTSD subtypes using neuroimaging data, before treatment implications can be fully realized. Future replication studies may assist in closing the translational gap between basic psychiatric research and practice, advancing the development of meaningful biological tools to assist diagnosis or predict clinical outcomes³.