Examination of Posttraumatic Stress Disorder Symptom Networks Using Clinician-Rated and Patient-Rated Data

Abstract

Network theory, which conceptualizes psychiatric disorders as networks of interacting symptoms, may provide a useful framework for understanding psychopathology. However, questions have arisen regarding the stability and generalizability of network analytic methods, with some researchers arguing that symptom networks have limited replicability. The aim of this study was to evaluate assessment modality as one possible source of instability in the estimation of posttraumatic stress disorder (PTSD) symptom networks. We estimated two cross-sectional DSM–5 PTSD symptom networks in 378 U.S. veterans: one using data from a clinician-rated assessment instrument (Clinician-Administered PTSD Scale for DSM–5; CAPS-5) and one using data from a self-rated questionnaire (the PTSD Checklist for DSM–5; PCL-5). We calculated centrality indices, conducted community structure analyses, and compared the strength and structure of the networks. The CAPS-5 and PCL-5 symptom networks were highly similar, challenging the notion that network methods produce unreliable results due to estimations consisting primarily of measurement error. Furthermore, each network contained distinct symptom communities that only partially overlapped with the DSM–5 PTSD symptom clusters. These findings may provide guidance for future revisions of the DSM, suggest hypotheses about how PTSD symptoms interact, and inform recent debate about replicability of psychopathology symptom networks.

The pathophysiology of posttraumatic stress disorder (PTSD) is a topic of growing interest and concern, yet individual differences in symptom presentation, treatment response, and course of illness are not well understood (Steenkamp, Litz, Hoge, & Marmar, 2015; Steinert, Hofmann, Leichsenring, & Kruse, 2015). Recent work suggests that the limited progress in understanding PTSD and other psychiatric conditions may reflect flawed conceptualizations of mental disorders (McNally et al., 2015). Historically, symptoms of mental disorders have been viewed as the product of an underlying disease entity. Network theory offers an alternative approach to this common cause model, conceptualizing psychiatric disorders as networks of symptoms that have causal associations with one another (Borsboom, 2017). In the case of PTSD, a symptom such as exaggerated negative beliefs (e.g., the world is completely dangerous) following trauma may cause hypervigilance, which may then cause difficulty concentrating and disrupted sleep. Within network theory, symptoms are referred to as nodes and the associations between symptoms are referred to as edges. Recent developments in computational methods allow for visual mapping of the symptom network, evaluation of the centrality (i.e., influence) of specific nodes, and identification of clusters of nodes within a larger network. These tools have potential to improve understanding of the development and maintenance of PTSD, identify risk and resilience factors, and elucidate optimal treatment targets.

Amid the growing popularity of these methods, the replicability of network models has been identified as a key challenge (Fried & Cramer, 2017). Network analysis is exploratory, involves estimation of a large number of parameters, and typically relies on single-item symptom measures, raising the possibility that network models are prone to overfitting (Fried & Cramer, 2017). Forbes and colleagues (Forbes, Wright, Markon, & Krueger 2017) explored this issue by comparing symptom networks in two large epidemiological samples. They reported that conditional independence networks demonstrated poor consistency across and within the two samples, concluding that psychopathology networks have limited replicability as a result of measurement error intrinsic to the data. In a responding commentary, Borsboom and colleagues (2017) characterized this work as containing statistical inaccuracies and debatable methodology, and reported that their analysis of the same data indicated a high degree of consistency in the resulting networks.

Despite diverging interpretations of existing data, a consensus exists that stability and generalizability of networks are paramount to the future of psychopathology network research. No single study will be able to determine to what extent symptom networks can be reliably replicated, but this question can be informed by ongoing evaluation of the conditions that influence generalizability. In this study, we address one question that has potential to advance this conversation: Do psychopathology networks remain stable within the same sample when using different measures?

Although this question is relevant to psychopathology network research more generally, it is particularly important to address in the PTSD network literature. A growing number of studies provide insight into the PTSD symptom network (e.g., Armour, Fried, Deserno, Tsai, & Pietrzak, 2017; Epskamp, Borsboom, & Fried, 2018; Fried et al., 2018; McNally et al., 2015; Mitchell et al., 2017; von Stockert, Fried, Armour, & Pietrzak, 2018). However, all but one (Bryant et al., 2017) have used data from self-rated measures, such as the PTSD Checklist (PCL), and no studies have used a clinician-rated measure based on the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM–5; American Psychiatric Association, 2013). Self-rated measures such as the PCL show evidence of strong psychometric properties and provide ease and efficiency when collecting data. However, researchers and clinicians have voiced concern regarding sole reliance on self-rated measures in the assessment of PTSD, and often recommend that such measures be supplemented with structured interviews such as the Clinician-Administered PTSD Scale (CAPS; e.g., McDonald & Calhoun, 2010). Although correlations between total PCL and CAPS severity scores are generally strong, the two measures demonstrate variable correlation strength when examined at the individual symptom level (Forbes, Creamer, & Biddle, 2001). Because network analysis is often conducted at the symptom level, variable item-level associations across clinician-rated and self-rated measures may influence the accuracy and validity of PTSD network estimations.

Therefore, in this study we aimed to assess the stability of the PTSD symptom network when using clinician-rated compared with self-rated assessment measures. To achieve these aims, we conducted network analysis of the PTSD symptom network using data from the CAPS for DSM–5 (CAPS-5). We then compared this clinician-rated symptom network to a network using data from a self-rated measure (the PCL for DSM–5; PCL-5) obtained simultaneously in the same sample.

Method

Participants

Participants were 378 veterans enrolled in two studies. Study 1 (n = 149) was designed to evaluate the psychometric properties of the CAPS-5 and PCL-5, enrolling veterans who endorsed at least one lifetime DSM–5 Criterion A traumatic event and at least one PTSD symptom in the previous month. Study 2 was a clinical trial for PTSD with 229 treatment-seeking male veterans who completed a baseline assessment (Sloan, Unger, & Beck, 2016). Participants were required to be age 18 or older and able to read and speak English.

Measures

The CAPS-5 (Weathers et al., 2013a) is a structured interview that provides severity scores ranging from 0 (Absent) to 4 (Extreme) for each DSM–5 PTSD symptom. The PCL-5 (Weathers et al., 2013b) is a self-rated measure of PTSD symptom severity corresponding to the DSM–5 PTSD symptoms. Respondents rate how much each symptom has bothered them in the past month from 0 (Not at all) to 4 (Extremely). Psychometric evaluations have provided strong support for both measures (Bovin et al., 2016; Weathers et al., 2018).

Procedures

Participants were recruited from a VA Medical Center. After providing written informed consent, the CAPS-5 and PCL-5 were administered as part of the larger assessment battery (Sloan et al., 2016; Weathers et al., 2018). Participants completed the CAPS-5 and PCL-5 in relation to the same (worst) traumatic event between zero and six days of one another. The VA Boston Healthcare System Institutional Review Board approved all procedures (protocol numbers 2625 and 2650).

Statistical Analyses

Two PTSD symptom networks were estimated via Graphical Gaussian Models (GGMs): one using CAPS-5 data and one using PCL-5 data. Each network node represents one of the 20 DSM–5 PTSD symptoms, and each edge weight between two nodes represents the strength of the partial correlation between those symptoms controlling for all other network symptoms. Graphical LASSO regularization was applied with extended Bayesian Information Criterion model selection. The GGMs were estimated and plotted using the R package qgraph (Epskamp, Cramer, Waldorp, Schmittmann, & Borsboom 2012). We calculated strength centrality (the sum of the absolute values of all edge weights connected to a node) for each node using qgraph. We evaluated the accuracy of estimated edge weights and stability of the centrality indices using the R package bootnet (Epskamp et al., 2018). Based on these analyses, closeness and betweenness centrality were not stable enough for interpretation (Epskamp et al., 2018). We examined each network’s community structure using the spin glass community detection algorithm (1,000 iterations), which identifies clusters of nodes within a larger network (Reichardt & Bornholdt, 2006).

The CAPS-5 and PCL-5 assess identical symptoms and use the same rating scales (ranging from 0 to 4), allowing for direct comparison of the resulting networks. We compared the CAPS-5 and PCL-5 network global strength (i.e., the sum of all edge weights in a network), structure (i.e., whether the networks are identical), and strength of individual edge weights using the Network Comparison Test (NCT), a permutation-based hypothesis test (van Borkulo et al., 2015; van Borkulo et al., 2017). See the online supplemental materials for additional information about measures and analyses.

Results

Participant characteristics are presented in Table 1, and mean CAPS-5 and PCL-5 item scores are presented in Table 2. Each item was rated as more severe on the PCL-5 relative to the CAPS-5 (p values <.002), except for intrusive memories (p = .20).

Table 1.

Participant Characteristics

Variable	Value
Age: M (SD)	55 (12)
Gender: n (%)
Male	354 (93.6%)
Female	19 (5.0%)
Not reported	5 (1.3%)
Race: n (%)
African American	81 (21.4%)
American Indian/Alaskan Native	5 (1.3%)
Asian/Pacific Islander	2 (.5%)
Caucasian	261 (69.1%)
Other	14 (3.7%)
Ethnicity: n (%)
Hispanic	21 (5.6%)
Non-Hispanic	343 (90.7%)
Current PTSD: n (%)	272 (72%)
Duration of PTSD symptoms, in months: M (SD)	301 (199)
Number of lifetime trauma type categories endorsed: M (SD)a
Sample 1	6.95 (3.34)
Sample 2	8.90 (3.40)

^aDifferent measures were used in Samples 1 and 2; see online supplemental materials for more detail. PTSD = posttraumatic stress disorder.

Table 2.

CAPS-5 and PCL-5 Item and Total Scores

Symptom	CAPS-5 Mean (SD)	PCL-5a Mean (SD)
B1. Intrusive memories	2.38 (.96)	2.46 (1.02)
B2. Nightmares	1.75 (1.34)	2.09 (1.23)
B3. Flashbacks	.54 (.95)	1.57 (1.26)
B4. Psychological distress in response to reminders	2.12 (.97)	2.56 (1.07)
B5. Physiological reactions to reminders	1.87 (1.09)	2.28 (1.15)
C1. Avoidance of memories, thoughts, feelings	2.26 (1.12)	2.61 (1.17)
C2. Avoidance of external reminders	1.99 (1.25)	2.60 (1.22)
D1. Amnesia	.61 (1.06)	1.51 (1.49)
D2. Distorted negative beliefs	1.78 (1.38)	2.06 (1.38)
D3. Distorted blame	1.44 (1.43)	2.04 (1.38)
D4. Persistent negative emotions	2.02 (1.09)	2.45 (1.27)
D5. Loss of interest	1.91 (1.43)	2.44 (1.26)
D6. Detachment	2.18 (1.30)	2.47 (1.23)
D7. Inability to experience positive emotions	1.85 (1.40)	2.27 (1.33)
E1. Irritable or angry behavior	1.32 (1.12)	2.08 (1.21)
E2. Reckless behavior	.37 (.84)	1.06 (1.14)
E3. Hypervigilance	2.23 (1.21)	2.74 (1.23)
E4. Exaggerated Startle	1.40 (1.14)	2.32 (1.30)
E5. Concentration problems	1.75 (1.21)	2.42 (1.26)
E6. Sleep disturbance	2.26 (1.37)	2.86 (1.27)
Total Score	34.17 (13.41)	44.92 (15.56)

Note. CAPS-5 = Clinician Administered PTSD Scale for DSM–5; PCL-5 = PTSD Checklist for DSM–5.

^aAll PCL-5 mean item and total values were significantly greater than the corresponding CAPS-5 value (with Bonferroni corrected alpha), with the exception of symptom B1.

Network Visualization

The CAPS-5-based network consisted of 105 edges representing statistically significant associations between symptoms (see Figure 1). All but four associations were positive. The largest edge weights in the network were between detachment and inability to experience positive emotions, psychological distress to reminders and physiological reactions to reminders, psychological distress to reminders and intrusive memories, and detachment and loss of interest. These four edges were significantly stronger than nearly all other edges in the network, but most edges did not differ significantly in strength.

Figure 1.

GLASSO-regularized partial correlation networks for CAPS-5 and PCL-5 items. Nodes are coded by color to correspond to the network community membership as identified by the spin glass algorithm. Positive edges appear in purple and negative edges in red, and node placement within the network was determined using the average layout between the CAPS-5 and PCL-5 networks. Network edges with a weight of .05 or higher are presented for ease of interpretation. Edge weights are scaled equally across the two networks (maximum = 1.0, cutoff = 0). To aid visual interpretation, the thickness of edges in the networks reflect the relative strength of the edge weight. CAPS-5 = Clinician Administered PTSD Scale for DSM–5; PCL-5 = PTSD Checklist for DSM–5; amn = amnesia; anh = loss of interest; avm = avoidance of memories, thoughts, and feelings; avx = avoidance of external reminders; blf = distorted negative beliefs; blm = distorted blame; cnc = concentration problems; det = detachment; drm = nightmares; fls = flashbacks; hyp = hypervigilance; irr = irritable or angry behavior; mem = intrusive memories; neg = persistent negative emotions; phy = physiological reactions to reminders; num = inability to experience positive emotions; rsk = reckless or self-destructive behavior; slp = sleep disturbance; str = exaggerated startle; ups = psychological distress in response to reminders.

The PCL-5-based network contained 96 edges (see Figure 1), with only one negative edge. The strongest edges in the network were between the two avoidance symptoms and between hypervigilance and exaggerated startle, followed by the edges between anhedonia and detachment, detachment and inability to experience positive emotions, and intrusive memories and nightmares. These five edges were significantly stronger than nearly all others in the network. Few other edges differed significantly in strength.

Strength Centrality Results

The strength centrality index was stable for both the CAPS-5 and PCL-5 networks; centrality stability(cor) =.52 and .60, respectively. Figure 2 depicts node strength in both networks. In the CAPS-5 network, intrusive memories was significantly stronger than most other nodes. In the PCL-5 network, detachment was significantly stronger than all nodes except persistent negative emotions and psychological distress to reminders, and traumatic amnesia was significantly weaker than most nodes. However, within both networks, the strength centrality of most nodes did not differ significantly from one another.

Figure 2.

Standardized strength scores for nodes in CAPS-5 and PCL-5 networks. CAPS = Clinician Administered PTSD Scale for DSM–5; PCL = PTSD Checklist for DSM–5.

CAPS-5 and PCL-5 Network Comparison Test Results

The NCT of global structure indicated that the CAPS-5 and PCL-5 networks were not statistically identical (test statistic M =.39, p =.00). However, there was no significant difference in the global strength of the two networks (CAPS-5 global strength = 8.91, PCL-5 global strength 9.05, p =.79, γ = 0.5). Additionally, only two edge weights (of 190) differed significantly between networks. The edge weights between avoidance of internal and external reminders and between hypervigilance and exaggerated startle were significantly greater in the PCL-5 network relative to the CAPS-5 network.

Network Cluster Results

In the CAPS-5 network, the spin glass algorithm detected a fourcluster solution in 82% of iterations. The most frequently occurring solution (30% of iterations) identified four clusters consisting of (a) all five intrusion symptoms, (b) both avoidance symptoms, distorted negative beliefs, hypervigilance, and exaggerated startle, (c) anhedonia, detachment, inability to experience positive feelings, trauma-related amnesia, sleep disturbance, and concentration difficulties, and (d) irritable or angry behavior, reckless behavior, distorted blame, and persistent negative emotions. Because no single solution was overwhelmingly favored in the 1,000 spin glass iterations, we calculated the Adjusted Rand Index (ARI; Hubert & Arabie, 1985) to evaluate the similarity of this solution with others that were also detected frequently. Results suggested that the most frequently detected solutions were highly similar, and examination of the specific differences across the 1,000 iterations showed that most symptoms grouped together consistently, but that distorted negative beliefs and sleep disturbance were less stable in cluster membership across iterations.

In the PCL-5 network, the spin glass yielded a 5-cluster solution in 99% of iterations. The most frequently detected solution (46% of iterations) consisted of (a) all five intrusion symptoms and sleep disturbance, (b) both avoidance symptoms and trauma-related amnesia, (c) anhedonia, detachment, inability to experience positive feelings, and concentration difficulties, (d) distorted blame, distorted negative beliefs, persistent negative emotions, irritable or angry behavior, and reckless behavior, and (e) hypervigilance and exaggerated startle. ARI calculations showed that this solution was highly consistent with the other frequently detected solutions in the 1,000 iterations. Examination of specific differences across iterations suggested variation primarily in clustering of arousal symptoms. Exaggerated startle and hypervigilance formed their own cluster in approximately half of iterations, and were grouped with irritable or angry behavior and reckless behavior in most other iterations. See the online supplemental materials for complete results.

Discussion

This is the first study to estimate the DSM–5 PTSD network based on structured clinical interview data, as well as the first to compare a clinician-rated network with one based on a self-rated measure, the PCL-5. Results demonstrate strong consistency across the clinician-rated and patient-rated symptom networks. Although the two networks were not statistically identical, there was no difference in the global strength of the networks (i.e., the overall connectivity among symptoms). Furthermore, comparison of individual edge weights found that of 190 edge weights, only two differed significantly across networks. Specifically, relative to the CAPS-5 network, the PCL-5 network demonstrated stronger partial correlations between hypervigilance and exaggerated startle, and avoidance of internal versus external reminders.

Although structured interviews such as the CAPS-5 are often referred to as “gold standards” for assessment and diagnosis of PTSD (Forbes et al., 2001; McDonald & Calhoun, 2010), our results suggest. that network analysis using brief, self-rated measures of PTSD yields comparable results. These findings are encouraging, given that self-rated measures are less burdensome for participants and are more feasibly administered in the large sample sizes needed for network analysis. However, even seemingly localized differences in symptom associations may influence analyses involving the larger symptom network. For instance, despite substantial overlap in the community structure of the CAPS-5 and PCL-5 networks, we also found some differences. The spin glass algorithm yielded four clusters of nodes in the CAPS-5 network, and five in the PCL-5 network. These differences may have resulted from the very strong associations among hypervigilance and exaggerated startle and avoidance of internal and external reminders on the PCL-5. Although these symptom pairs both belonged to a larger cluster in the CAPS-5 network, they comprised smaller, separate clusters in the PCL-5 network.

Several differences between the CAPS-5 and PCL-5 could explain potential differences in their resulting symptom networks. PCL-5 ratings are based on subjective distress, whereas CAPS-5 ratings consider the intensity and frequency of symptoms. Additionally, intensity ratings on the CAPS-5 are not limited to subjective distress; for example, assessment of hypervigilance focuses on the nature and pervasiveness of hypervigilant behavior. However, these explanations would likely yield more pervasive differences between networks, in contrast to our finding that only two edge weights differed. An alternative, perhaps more likely, possibility is that the specific symptom pairs of hypervigilance and exaggerated startle, and avoidance of internal and external reminders, are particularly challenging for nonclinicians to distinguish between, especially in the context of a brief checklist with minimal symptom descriptions. This could lead respondents to rate these pairs of symptoms similarly, whereas the clinician’s knowledge of the symptoms and the specific prompts and descriptions provided within the CAPS could buffer this response style.

The results of this study also have implications for the recent debate regarding the replicability of psychopathology networks. In contrast to Forbes and colleagues’ (2017) argument that conditional independence networks are “likely highly influenced by noise” and are therefore not replicable (p. 972), our findings demonstrated strong correspondence between PTSD symptom networks estimated from two different measures within the same sample. Results of other recent investigations have also supported the stability and generalizability of network estimations. von Stockert and colleagues (2018) found evidence of temporal stability of PTSD symptom networks over a 3-year period. Fried and colleagues (2018) compared the PTSD symptom network in four trauma patient samples that varied in cultural background, trauma type, and severity. The resulting networks were not statistically identical, but showed moderate to strong correlations among network edges and among strength centrality coefficients. By evaluating the influence of assessment modality on the PTSD symptom network, our study adds to this literature that, when taken together, suggests that symptom networks can be stable over time and robust to differences in measurement method and study sample.

The results of our community structure analyses may have implications for the ongoing debate regarding the structure of PTSD. Symptoms in the CAPS-5 and PCL-5 networks clustered into communities that resemble the DSM–5 PTSD symptom clusters in some ways. For example, all intrusion symptoms (Cluster B in DSM–5) belonged to a single community, and both avoidance (Cluster C) symptoms clustered together. However, there were some critical differences in the network clusters compared with the DSM–5 clusters. Nearly all communities in both networks contained symptoms from multiple DSM–5 clusters, suggesting that there are important relationships among symptoms that may be overlooked when relying on the DSM–5 organization. The organizational structure of the PTSD criteria in DSM–5 has already received criticism after numerous factor analytic studies supported alternative factor models (e.g., Bovin et al., 2016; Liu et al., 2014). Our findings further call into question the validity and utility of the DSM–5 symptom clusters, particularly when subscribing to a network conceptualization of mental disorders. Community structure analyses such as those conducted in this study may inform future revision of the DSM clusters, offering a perspective that reflects how symptoms relate to one another rather than how they relate to higher order latent constructs.

Despite this, some limitations to the spin glass algorithm should be recognized. Although rarely acknowledged in the psychopathology literature, the algorithm is stochastic in nature, meaning that it can produce different results each time it is run. Research is therefore needed to better understand how to interpret inconsistency across results. For instance, our CAPS-5 data showed that sleep disturbance and distorted negative beliefs had inconsistent cluster membership across 1,000 iterations of the algorithm. This could indicate that these symptoms do not fit well into any cluster, or conversely, that these symptoms are strongly associated with several clusters. This highlights another limitation of the algorithm: It may impose assumptions on the data that are inaccurate. For instance, the algorithm cannot allow for symptom membership to more than one cluster, and nearly always assigns a symptom to a cluster.

Several other limitations should be acknowledged. Our sample consisted of mostly Caucasian, male veterans recruited from a VA health care system who endorsed a high degree of trauma exposure and symptom chronicity, and therefore, findings may not generalize to other samples. As mentioned previously, analyses focused on the average associations between symptoms across participants, and therefore may obscure ways in which the symptom network differs between individuals.

Conclusion

The evaluation of the clinician-rated DSM–5 PTSD symptom network relative to a self-rated symptom network indicates that both types of measures yield highly similar results. These findings suggest that estimated symptom networks can be robust to measurement methods, and provide support for continued use of self-rated measures in network analytic studies of PTSD. Results from both symptom networks revealed the existence of distinct communities of symptoms within the larger PTSD network that correspond only partially with DSM–5 clusters and factor analytic results. These results require replication and further study using longitudinal data, but may inform revision of the DSM–5 PTSD symptom cluster organization and the ongoing debate about replicability of psychopathology symptom networks.

General Scientific Summary.

Network analysis, a set of statistical methods that evaluate associations among psychiatric symptoms may improve understanding of disorders such as posttraumatic stress disorder (PTSD). Our study found that network methods yield similar results when using both comprehensive clinician-rated interviews and brief self-rated questionnaires to assess PTSD, informing recent debate over the replicability of symptom networks. Results also suggest that symptoms in the network cluster together in ways that are not captured by the current diagnostic criteria for PTSD.