Abstract
Researchers have proposed three core deficits of Borderline Personality Disorder (BPD): emotion dysregulation, interpersonal problems, and self-identity disturbance. Previous methods for testing these deficits rest on problematic assumptions (e.g., the assumption that observable/measured features of BPD, such as chaotic relationships and affective intensity, occur independently). A network model of psychopathology assumes that observable features of disorders directly interact, and network analytic methods quantify how central each feature is. We conducted a network analysis of core deficits of BPD features using a large (n = 4386) sample of participants with a range of BPD features. The most central features of participants in the High BPD group were loneliness, recklessness/impulsivity, and intense moods, supporting models of emotion dysregulation and interpersonal problems. The networks of BPD features did not differ between men and women. We provide directions for future research to enhance our understanding of how networks of BPD features change over time.
Keywords: borderline personality disorder, network analysis, emotion regulation, interpersonal functioning, identity disturbance
Borderline Personality Disorder (BPD) is a severe psychiatric disorder, characterized by chronic suicidality, fluctuating and intense emotions, and unstable interpersonal relationships (American Psychiatric Association, 2013b). It is relatively common, with a nationally representative study identifying a lifetime prevalence rate of 5.9% (Grant et al., 2008). It is also a heterogeneous disorder; people diagnosed with BPD may present with one of 256 unique combinations of five of nine DSM-5 criteria (APA, 2013b). Such heterogeneity calls into question whether a single diagnosis is appropriate for such a diverse array of presentations. Researchers have attempted to address the problem of heterogeneity by developing theoretical models that identify core deficits in BPD. Three such core deficits have been particularly well-articulated and incorporated into treatments for BPD: emotion dysregulation (e.g., Linehan, 1993), problematic relationships (e.g., Gunderson, 2007), and the lack of an integrated sense of self (e.g., Kernberg & Michaels, 2009).
Emotion Dysregulation
Linehan (1993) characterized BPD as “primarily a disorder of the emotion regulation system” (p. 43). According to her biosocial theory, an invalidating childhood environment transacts with biological vulnerabilities to experience strong emotions producing greater dysregulation of emotional experiences. This dysregulation includes the tendency to more quickly experience more intense emotions that last longer compared to people without BPD. According to the biosocial theory, problematic relationships and a lack of an integrated sense of self result from maladaptive attempts to regulate intense and unstable emotions. Emotion dysregulation in the form of frequent outbursts of anger or greater sensitivity to rejection cues may lead to difficulties forming and maintaining relationships, while unpredictable, difficult to manage emotions and impulsive behaviors may provide inconsistent information regarding one’s preferences or sense of self.
Several researchers have found evidence for the centrality of emotion dysregulation to BPD. In a combined student and community sample, self-reported emotion dysregulation, but not interpersonal difficulties or deficits in sense of self, was associated with BPD features among those with elevated BPD features (Cheavens, Strunk, & Chriki, 2012). Similarly, across two student samples, emotion dysregulation was associated with BPD features over and above symptoms of depression, anxiety, and negative affect (Glenn & Klonsky, 2009). Clifton and Pilkonis (2007) conducted a factor analysis of a mixed clinical and nonclinical sample based on standardized clinical interviews. The item with the highest factor loading on the latent BPD factor was affective instability, with identity disturbance demonstrating one of the lowest factor loadings.
Problematic Relationships
In attachment theory, Bowlby (1983) proposed that children engage in interpersonal behaviors to ensure access to their caregivers and that children may develop certain patterns or styles of interpersonal behaviors. Children characterized by the “anxious/ambivalent” style check for proximity of their caregiver, plead for others or call for help, and cling to caregivers when possible (Ainsworth, 1978), while children characterized by the “disorganized/disoriented” style deny their need for others, exhibit no separation anxiety, and are reluctant to develop attachment with anyone (Main & Solomon, 1990). Each of these attachment styles are thought to be present among people with BPD (Gunderson, 1996). Vacillating between these two attachment styles and interpersonal behaviors may escalate dysregulated emotional responses of anger or anxiety when a caregiver is absent and lead to a fractured sense of self due to the competing goals of over-attachment and isolation.
Researchers have demonstrated the centrality of problematic relationships to BPD, especially to more severe presentations of BPD. Across two samples using an item-response theory (IRT) analysis, the presence of unstable relationships (Conway, Hammen, & Brennan, 2012) and avoidance of abandonment (Feske, Kirisci, Tarter, & Pilkonis, 2007) were two of the best indicators of more severe presentations of BPD. In a large sample of non-clinical adults, an insecure attachment style was associated with BPD features, over and above measures of identity disturbance, personality, and impulsivity (Fossati, Borroni, Feeney, & Maffei, 2012). Similarly, unstable relationships were most strongly related to a single BPD factor in a large study of people with BPD and clinical controls (Spitzer, Endicott, & Gibbon, 1979).
Self-Identity Disturbance
According to psychodynamic theory, people with BPD experience extreme unconscious aggressive impulses that conflict with their positive self-image (Kernberg, 1967). People with BPD then “split” their self-concept to preserve this positive self-image, leading to a lack of an integrated sense of self (Bender & Skodol, 2007). Moving between extreme negative and positive self-images, then, is thought to produce affective instability and emotion dysregulation, while a less coherent sense of self also produces unrealistic expectations of others, leading to problematic relationships (Levy, Clarkin, Yeomans, Scott, Wasserman, & Kernberg, 2006).
Few researchers to our knowledge have found that self-identity disturbance is strongly and uniquely associated with BPD features. However, two independent research groups conducted factor analyses and found that identity disturbances were as strongly related to a single BPD factor as problematic relationships (Fossati, Maffei, Bagnato, Donati, Namia, & Novella, 1999; Hawkins, Furr, Arnold, Law, Mneimne, & Fleeson, 2014) while a third set of researchers found similarly strong associations with BPD among identity disturbance, problematic relationships, and affective instability (Johansen, Karterud, Pedersen, Gude, & Falkum, 2004).
Network Analyses to Assess Symptom Centrality
Many of the studies cited here used factor analytic techniques to determine if BPD criteria are most accurately characterized as manifestations of a single BPD construct (i.e., a unitary latent dimension) or if they represent multiple underlying dimensions that together characterize BPD. This technique is most useful for aggregating a large number of symptoms into a smaller number of meaningful, overarching dimensions. However, a limitation of this approach is that correlations among the individual, observed items (e.g., BPD criteria) are not estimated, because it is assumed that such correlations are accounted for by the latent factor. This assumption may be less useful for psychotherapy researchers and clinicians who are targeting clients’ specific, observable behaviors for change.
A complementary approach, the network model of psychopathology, suggests that symptoms interact with and cause changes in each other (Fried, van Borkulo, Cramer, Boschloo, Schoevers, & Borsboom, 2017). In this model, symptoms such as chaotic relationships, affective intensity, and impulsive self-harming behaviors may transact over time to produce the phenomena of BPD, rather than each symptom resulting from a higher order trait or factor. Conceptualizing personality and psychopathology as networks of symptoms further allows researchers to identify which symptoms are more or less central to the network. Centrality, in a broad sense, indicates that a symptom is an important link to other symptoms in a network. Changes in central symptoms may cause changes in many other symptoms or changes in a large number of more peripheral symptoms may be expected to cause changes in more central symptoms.
Certain BPD symptoms may be more central (i.e., strongly connected) than others for several reasons. First, more central BPD symptoms may be more severe or impairing (e.g., suicidality or self-harm). Given the multiple interpersonal, behavioral, and affective precipitants of suicidality (e.g., Joiner, 2005), it is possible that this feature may be strongly connected to many aspects of BPD. Second, more central BPD symptoms may be more common. That is, because certain symptoms (e.g., affective instability, difficulties controlling anger, and impulsivity; Sanislow et al., 2002) are experienced more regularly by people, there may be more opportunities for these symptoms to influence other symptoms. Third, more central BPD symptoms may be more persistent. In a ten-year study of patients with BPD, affective symptoms (e.g., loneliness/emptiness, anger, depression, anxiety) demonstrated the longest time to remission, while suicidal/self-harming behaviors and identity disturbance resolved relatively quickly (Zanarini, Frankenburg, Reich, Silk, Hudson, & McSweeney, 2007). Symptoms may persist and be maintained because they are so interconnected with and thus easily “turned on” by many other symptoms. While we cannot differentially support one of these hypotheses based on the results of a network analysis, we can identify which specific features are more likely to function in any of these ways.
Analyzing the networks of BPD features may be a compelling data-driven way to address the competing conceptions of core deficits in BPD. By not assuming symptoms are caused by an unobserved construct, we can better understand the relations among symptoms. Finally, by considering how features in one domain of BPD (e.g., self-identity) relate to other relevant domains of functioning (e.g., emotion regulation or interpersonal functioning), we can develop a more comprehensive understanding of how deficits in one domain are related to multiple areas of functioning.
In the only network analysis of BPD features to our knowledge, Richetin, Preti, Costantini and de Panfilis (2017) found that affective instability, identity disturbance, and efforts to avoid abandonment were the most central BPD features. This pattern of results was similar between a university student sample and a personality disordered clinical sample, although only 32.3% of their clinical sample (n = 31) were diagnosed with BPD. Further, although the researchers collected data on 47 BPD features based on the Borderline Personality Disorder Checklist (Arntz, van den Hoorn, Cornelis, Verheul, van den Bosch, & de Bie, 2003), they aggregated these items to analyze the networks of the nine DSM-5 BPD criteria. While this method provides greater convergent validity of each BPD feature by relying on multiple responses, it limits our understanding of BPD-relevant features to established diagnostic criteria which do not capture the dimensional nature of personality disorders.
Personality Disorder Features: From Diagnostic Criteria to Dimensional Models
Recently, the American Psychiatric Association (2013a) and the National Institute of Mental Health (Insel, 2013) have both advocated for the development of dimensional models of personality and personality disorders. To better understand the dimensional nature of personality, researchers must include in their samples people who meet current diagnostic criteria as well as those who do not meet criteria for any diagnosis (Cuthbert, 2014). Analyzing the network of features in a sample of people all along the dimension of BPD (i.e., those both above and below the diagnostic threshold) is in line with these goals and reflective of the dimensional (Conway, Hammen, & Brennan, 2012) or hybrid categorical-dimensional (Hallquist & Pilkonis, 2012) nature of BPD.
Along with analyzing the networks of BPD features in a mixed healthy and clinical sample, it is also important to compare gender differences in networks of BPD features. Although 75% of those diagnosed with BPD in clinical research trials are women (APA, 2013b), Grant et al. (2008) conducted a large, nationally-representative study in which they found no gender differences among those diagnosed with BPD. These contrasting findings may suggest that women with BPD present for treatment more often than men with BPD. Alternatively, clinicians may over-diagnose BPD in women compared to men. Because of these contrasting findings, it is important that any comprehensive network analysis of BPD features compare the structure of BPD features between men and women.
Current Study
In the current study, we conducted a network analysis of BPD features in a large mixed sample of clinical and non-clinical participants. We had four aims with these analyses. First, we aimed to examine the network structure and most central features of BPD. Second, we aimed to compare the network structure of BPD features between participants high in BPD features and low in BPD features. Third, we aimed to compare the network structure of BPD features between men and women. Finally, we aimed to compare the centrality of core deficits related to BPD features by conducting a network analysis of BPD features, emotion dysregulation, and interpersonal difficulties and comparing these networks between participants high and low in BPD features.
Materials and Methods
Participants
Participants (N = 4636) were enrolled in one of 18 studies conducted between May 2012 and November 2017 within the same research laboratory at a large Midwestern university and included undergraduate students, community members, and those seeking psychotherapy treatment (Benitez, 2014, 2017; Cheavens, Lazarus, & Herr, 2014; Cheavens, Strunk, & Chriki, 2012; Forsythe, 2011, 2014; Heiy, 2014; Heiy & Cheavens, 2014; Lazarus, 2015; Lazarus, Southward, & Cheavens, 2016). The average participant age was 22.62 years old (SD = 8.30), with the majority of participants identifying as female (61.1%) and Caucasian (74.2%). Participants under 18 years old were excluded. All participants were reimbursed for their time, either with partial course credit or financial compensation. All procedures from each study were IRB-approved and in accordance with the provisions of the World Medical Association Declaration of Helsinki, and all participants provided informed consent.
Materials
Demographics.
In each study, participants reported their age, gender, and racial/ethnic background.
Difficulties in Emotion Regulation Scale (DERS; Gratz & Roemer, 2004).
The DERS is a 36-item self-report measure designed to assess difficulties regulating emotions in six domains: non-acceptance of emotional responses, difficulties engaging in goal-directed behavior, impulse control difficulties, lack of emotional awareness, limited access to emotion regulation strategies, and lack of emotional clarity. Participants indicate how often they experience each item on a five-point Likert-type scale, resulting in a total score and six subscale scores. In this sample, DERS items demonstrated acceptable to excellent internal consistency for each of the six subscales (Cronbach’s alphas: .76 - .92).
Inventory of Interpersonal Problems (IIP; Pilkonis, Kim, Proietti, & Barkham, 1996).
The IIP is a 47-item self-report measure designed to measure a person’s specific patterns of behavior in response to interpersonal problems across five domains: aggression, ambivalence, lack of sociability, need for social approval, and interpersonal sensitivity. Participants indicate the degree to which each item has been distressing in their relationships on a five-point Likert-type scale. In this sample, IIP items demonstrated good to excellent internal consistency for each of the five subscales (Cronbach’s alphas: .86 - .91).
Personality Assessment Inventory – Borderline Subscale (PAI-BOR; Morey, 1991).
The PAI-BOR is a 24-item self-report measure designed to assess BPD features. It includes items that directly map on to DSM-5 (APA, 2013b) diagnostic criteria for BPD (e.g., DSM-5: Chronic feelings of emptiness; PAI-BOR: “Sometimes I feel terribly empty inside”) and items that describe features relevant to but not identical to these criteria (e.g., PAI-BOR: “People once close to me have let me down”). Participants indicate the degree to which each item is true of them on a four-point Likert-type scale, resulting in a total score and four subscales: Affective Instability, Identity Disturbance, Negative Relationships, and Self-Harm. In this sample, PAI-BOR items demonstrated good internal consistency as a total score (Cronbach’s alpha = .89) and acceptable internal consistency across each of the four subscales (Cronbach’s alphas: .69 - .78).
Morey (1991) and Trull (1995) demonstrated that a threshold score of 38 or greater most effectively distinguished people likely to meet criteria for BPD from those who were not likely to meet BPD criteria. Using this threshold, we created two subsamples of participants: those scoring ≥ 38 on the PAI-BOR (High BPD; n = 1023; 23.3% of the full sample) and those scoring < 38 on the PAI-BOR (Low BPD, n = 3363, 76.7% of the full sample). In this dataset, a subsample of participants (n = 229) completed both the PAI-BOR and a structured diagnostic interview based on DSM-IV-TR personality disorder criteria (SCID-II; First, Gibbon, Spitzer, Williams, & Benjamin, 1997). Of these, 108 participants met criteria for BPD while 121 participants did not. Morey’s proposed threshold score exhibited high sensitivity (88.0%), specificity (88.4%), and area under the curve (AUC; .95) among these participants; this increased our confidence that the High BPD group was representative of those meeting criteria for a BPD diagnosis.
Demographic and Mean-Level PAI-BOR Group Comparisons
High BPD vs. Low BPD.
There were some demographic differences between participants in the High BPD group (n = 1023) and those in the Low BPD group (n = 3363). The High BPD group was 23.77 years old (SD = 8.95) on average and was mostly female (66.9%) and Caucasian (72.1%). The Low BPD group was 22.08 years old (SD = 7.89) on average and was also mostly female (59.5%) and Caucasian (75.2%). Although the High BPD group was significantly older than the Low BPD group, t(1227.95) = 5.04, p < .01, 95% CI [1.04, 2.35], age was only weakly correlated with PAI-BOR total scores in the full sample, r = .04, p = .01, 95% CI [.01, .07]. There was also a greater proportion of women in the High BPD group than in the Low BPD group, χ2(1) = 15.87, p < .01, 95% CI [3.72, 10.84], but there were no significant differences in the proportion of participants who identified as Caucasian between the two groups, χ2(1) = 3.26, p = .07, 95% CI [−.33, 6.52].
Women vs. men.
When comparing women (n = 2568) to men (n = 1636), women (M = 28.56, SD = 12.33) reported higher PAI-BOR scores on average than men (M = 26.62, SD = 11.69), t(3618.37) = 5.14, p < .01, 95% CI [1.20, 2.68]. Women in this sample (M = 22.99, SD = 8.96) were also significantly older than men (M = 21.53, SD = 6.48), t(3972.36) = 6.01, p < .01, 95% CI [.98, 1.93], but there were no significant differences between women (74.0%) and men (75.3%) in the proportion of participants who identified as Caucasian, χ2(1) = .85, p = .36, 95% CI [−1.51, 4.08].
Diagnostic assessment vs. no diagnostic assessment.
Participants (n = 229) in a subset of studies completed SCID-II interviews (First, Gibbon, Spitzer, Williams, & Benjamin, 1997) to assess the presence of BPD. We first compared the demographics and PAI-BOR total scores between participants who completed a SCID-II interview and were diagnosed with BPD and participants who did not complete a SCID-II interview but scored above Morey’s (1991) threshold on the PAI-BOR, suggesting the likely presence of BPD. Participants diagnosed with BPD (M = 30.58, SD = 10.42) were significantly older than those who only scored above Morey’s threshold (M = 22.80, SD = 8.31), t(126.01) = 7.39, p < .01, 95% CI [5.70, 9.86]. There was a greater proportion of women among participants diagnosed with BPD (87.9%) than those scoring above Morey’s threshold (64.0%), χ2(1) = 24.16, p < .01, 95% CI [15.43, 30.37]. However, a similar proportion of participants identified as Caucasian among those diagnosed with BPD (68.3%) as those scoring above Morey’s threshold (72.5%), χ2(1) = 0.80, p = .37, 95% CI [−5.19, 14.57]. Participants diagnosed with BPD (M = 47.01, SD = 9.09) also reported similar total PAI-BOR scores as those scoring above Morey’s threshold (M = 45.36, SD = 6.39), t(119.81) = 1.83, p = .07, 95% CI [−0.14, 3.43].
We then compared the demographics and PAI-BOR total scores between participants who completed a SCID-II interview and were not diagnosed with BPD and participants who did not complete a SCID-II interview but scored below Morey’s (1991) threshold on the PAI-BOR, suggesting the likely absence of BPD. Participants who were not diagnosed with BPD were significantly older (M = 29.94, SD = 10.89) than those scoring lower than Morey’s threshold (M = 21.78, SD = 7.59), t(121.35) = 8.07, p < .01, 95% CI [6.16, 10.16]. There was also a greater proportion of women among participants who were not diagnosed with BPD (97.5%) than among those scoring lower than Morey’s threshold (58.1%), χ2(1) = 73.04, p < .01, 95% CI [34.32, 41.99]. A significantly smaller proportion of participants who were not diagnosed with BPD identified as Caucasian (66.9%) than those scoring lower than Morey’s threshold (75.5%), χ2(1) = 4.47, p = .03, 95% CI [0.04, 17.95]. However, participants without BPD (M = 22.42, SD = 12.00) reported similar total PAI-BOR scores as those scoring below Morey’s threshold (M = 23.19, SD = 8.44), t(124.47) = 0.70, p = .48, 95% CI [−1.41, 2.95].
Analytic Method
Missing data analyses.
We first conducted multiple imputation using the mice package (van Buuren & Groothuis-Oudshoorn, 2011) in R version 3.4.1 (R Core Team, 2017), selecting the first full dataset produced from the procedure for analyses. We imputed data when 20% or fewer of the items from a given measure were missing. If greater than 20% of the items from a given measure were missing from a participant, this case was dropped from the relevant analyses.
When applying these criteria to the PAI-BOR items, 238 participants (5.1%) were excluded for having fully missing data. A further 12 participants (0.3%) were excluded for having > 20% missing data. Of the remaining 4386 participants, there were 767 missing PAI-BOR items (0.7%). The remaining participants were significantly younger (M = 22.44, SD = 8.15) than the excluded participants (M = 27.75, SD = 10.87), t(200.11) = 6.67, p < .01, 95% CI [3.74, 6.89], composed of a significantly greater proportion of people who identified as Caucasian (74.5%) than the excluded participants (67.7%), χ2(1) = 4.44, p = .04, 95% CI [.11, 14.04], but composed of a similar proportion of women (61.1%) as the excluded participants (62.5%), χ2(1) = .85, p = .36, 95% CI [−1.51, 4.08].
When applying these criteria to the PAI-BOR, DERS, and IIP collectively, 178 participants (3.8%) were excluded for having fully missing data. A further 2664 participants (57.5%) were excluded for having > 20% missing data on the PAI-BOR, DERS, or IIP. Of the remaining 1794 participants, there were 1385 missing items across these three measures (< 0.1%). The remaining participants were significantly older (M = 25.11, SD = 10.13) than the excluded participants (M = 20.92, SD = 6.25), t(2752.76) = 15.52, p < .01, 95% CI [3.66, 4.73], but were composed of a similar proportion of women (60.3%) as the excluded participants (61.8%), χ2(1) = .98, p = .32, 95% CI [−1.49, 4.45], and a similar proportion of participants who identified as Caucasian (74.4%) as those who were excluded (74.1%), χ2(1) = .04, p = .85, 95% CI [−2.46, 2.95]. Remaining participants also reported a lower total PAI-BOR score on average (M = 27.56, SD = 12.11) than excluded participants (M = 28.95, SD = 12.76), t(3979.67) = 3.66, p < .01, 95% CI [.65, 2.14].
Network analyses.
We conducted all network analyses using the qgraph package (Epskamp, Cramer, Waldorp, Schmittmann, & Borsboom, 2012) in R version 3.4.1 (R Core Team, 2017). This package estimates regularized partial correlation networks (also called Gaussian Graphical Models; Costantini et al., 2015; Epskamp & Fried, 2017; Lauritzen, 1996) in which the association (or edge) between two items (or nodes) is estimated controlling for all other items in the network. Because it is rare for a partial correlation between two nodes to be exactly zero, partial correlation networks can produce unstable estimates that overfit the data (Babyak, 2004). To address this issue, qgraph implements a regularization technique called the graphical least absolute shrinkage and selection operator (GLASSO; Friedman, Hastie, & Tibshirani, 2008) which estimates the network of partial correlations that maximizes the log-likelihood function with a penalty term. Consistent with the procedures outlined by Costantini, Richetin, Preti, Casini, Epskamp, and Perugini (2017), we determined this penalty term using the Extended Bayesian Information Criterion (EBIC; Foygel & Drton, 2010) by setting γ = 0.5.
Centrality analyses.
We examined three indices of centrality (Costantini et al., 2015): node strength, a measure of the associations between an item and all other items it is directly connected to in the network; closeness, a measure of the sum of the smallest associations among an item and all other items in the network; and betweenness, a measure of how important an item is to the average association between items (i.e., if a high-betweenness item is removed, the average association between other items is likely to decrease). We then tested the stability of each centrality index. We used the bootnet package (Epskamp et al., 2017) in R to estimate correlation stability coefficients (CS-coefficients) for each centrality index in each network. This package uses a case-dropping subset bootstrapping procedure in which the network is re-estimated on a random subset of cases comprising a set percentage of the total sample (Chernik, 2011). The network is repeatedly re-estimated with a larger and larger percentage of cases dropped. This procedure results in CS-coefficients representing the maximum proportion of cases than can be dropped for the correlation between the centrality indices of the full sample and the centrality indices of the subset of cases to correlate at r = .70 with 95% confidence1. Based on a simulation study (Epskamp et al., 2017), the researchers suggested that CS-coefficients should be greater than .50 for centrality estimates to be interpretable. Additionally, correlation coefficients may be biased if the variance of one or both items is restricted (Goodwin & Leech, 2006), which may bias networks of items (Terluin, de Boer, & de Vet, 2016). To address this concern, we correlated the centrality and standard deviation of each item.
Borgatti (2005) has shown that measures of closeness and betweenness only accurately characterize items when the overall network exhibits certain properties (e.g., an effect can only follow one path at a time; effects always travel the shortest path/strongest association). Because these properties may or may not characterize networks of psychopathology symptoms, we have included the closeness and betweenness results in the Supplemental Materials.
Node strength is typically calculated as a sum of the absolute value of the associations of an item with its direct neighbors. When many of these associations are negative, Robinaugh, Miller, and McNally (2016) have shown that it may be more accurate to include the sign of each association in the sum. We report their measure of node strength, expected influence, in addition to the more common node strength for each analysis for a more comprehensive understanding of these results.
In order to compare the networks of subgroups within our dataset (i.e., those with high vs. low BPD features; men vs. women), we used the EstimateGroupNetwork package (Costantini & Epskamp, 2017) in R. This package utilizes the Fused Graphical Lasso (FGL) method described by Costantini et al. (2017). The FGL applies a further penalty when maximizing the log-likelihood function for a model that is a function of the differences between correlation matrices in the groups to be compared. The FGL uses data more efficiently than calculating two separate networks because it determines which edges are nearly identical between groups without worsening model fit. This process reduces the number of parameters to be estimated and improves the accuracy of these nearly identical edges by using the full sample to estimate them (Danaher, Wang, & Witten, 2014).
One current limitation of FGL estimation methods is the lack of explicit statistical tests of network comparisons. Although it may be assumed that any differences in network structure or edge weights are real differences due to the model search and penalty application procedures, these properties have not been demonstrated. However, van Borkulo, Epskamp, and Millner (2016) developed the NetworkComparisonTest package to conduct hypothesis tests on differences in the structure and edge weights between two independent networks while accounting for differing sample sizes. While our primary analyses and interpretations rely on the networks generated from the FGL, we supplement these analyses with statistical comparisons of independent networks calculated with the NetworkComparisonTest package using 10,000 permutations.
Finally, in order to test if node centrality was related to the mean level of each item, we calculated Spearman’s ρ between the rank-ordered node centrality and rank-ordered mean level of each item for each subgroup analyzed. For the relevant pairs of subgroups (i.e., High BPD and Low BPD; women and men), we compared these associations using Fisher’s r-to-z transformation, then comparing z-scores using Cohen and Cohen’s (1983) formula as implemented by Preacher (2002). We entered each Spearman’s ρ as a Pearson’s r in this procedure because previous simulations have demonstrated the relative robustness of this approach to Type I errors (Myers & Sirois, 2006).
Results
Aim 1: The Network Structure of BPD Features
A graphical depiction of the network of 24 items of BPD features from the PAI-BOR in the full sample is presented in Figure 1A, with centrality estimates displayed in Figure 1B. Table 1 contains item abbreviations, corresponding descriptions, sample means, and standard deviations. The most central node of the network in terms of node strength, the weighted sum of the direct associations between nodes, was BPD5 (chronic feelings of emptiness) from the Identity Disturbance subscale (z = 2.31). BPD4 (intense moods; z = 1.56) and BPD1 (sudden mood shifts; z = 0.97), both from the Affective Instability subscale, represented the next strongest nodes. Based on 1000 bootstrap resamples, BPD5 (chronic emptiness) demonstrated a significantly greater strength score than all other nodes except BPD17 (hurt self when upset), ps < .05 (Figure S1, Supplemental Material). The same items (BPD5, BPD4, BPD1) also demonstrated the greatest expected influence.
Figure 1.
A) Network structure of BPD features in full sample. B) Strength centrality indices (z-scores) for all BPD features in full sample. C) Strength (expected influence) centrality indices (z-scores).
Table 1.
Item abbreviations, item content, means, and standard deviations for PAI-BOR items.
Full sample | Low BPD | High BPD | Women | Men | ||||
---|---|---|---|---|---|---|---|---|
(N = 4386) | (n = 3363) | (n = 1023) | (n = 2568) | (n = 1636) | ||||
Item | Item content | M (SD) | M (SD) | M (SD) | t | M (SD) | M (SD) | t |
BPD1 | Sudden mood shifts | 1.35 (0.94) | 1.08 (0.81) | 2.25 (0.77) | 42.28** | 1.41 (0.95) | 1.20 (0.90) | 7.15** |
BPD2 | Attitude about self changes | 1.15 (0.94) | 0.90 (0.82) | 1.96 (0.86) | 35.85** | 1.18 (0.95) | 1.03 (0.90) | 5.17** |
BPD3 | Stormy relationships | 0.95 (0.99) | 0.66 (0.80) | 1.92 (0.94) | 39.04** | 0.97 (1.00) | 0.85 (0.93) | 3.64** |
BPD4 | Intense moods | 1.26 (1.02) | 0.94 (0.86) | 2.32 (0.77) | 46.00** | 1.30 (1.03) | 1.12 (0.97) | 5.64** |
BPD5 | Chronic emptiness | 1.20 (1.11) | 0.85 (0.94) | 2.34 (0.82) | 48.85** | 1.22 (1.11) | 1.07 (1.05) | 4.40** |
BPD6 | Let people know they’ve hurt me | 1.18 (1.06) | 0.92 (0.94) | 2.05 (0.99) | 32.48** | 1.24 (1.06) | 1.01 (1.02) | 7.04** |
BPD7 | Steady mood (RS) | 1.60 (0.95) | 1.40 (0.89) | 2.24 (0.87) | 26.95** | 1.63 (0.96) | 1.46 (0.92) | 5.59** |
BPD8 | Worry about people leaving | 1.17 (1.06) | 0.89 (0.94) | 2.09 (0.93) | 35.70** | 1.24 (1.07) | 1.03 (1.02) | 6.23** |
BPD9 | People let me down | 1.63 (1.07) | 1.37 (1.03) | 2.48 (0.75) | 37.59** | 1.75 (1.08) | 1.38 (1.03) | 10.96** |
BPD10 | Little control over anger | 0.75 (0.89) | 0.52 (0.72) | 1.52 (0.93) | 31.75** | 0.74 (0.89) | 0.72 (0.85) | 0.47 |
BPD11 | Wonder about life | 1.57 (1.07) | 1.35 (1.03) | 2.29 (0.86) | 29.34** | 1.56 (1.08) | 1.51 (1.05) | 1.60 |
BPD12 | Rarely lonely (RS) | 1.62 (1.05) | 1.45 (1.02) | 2.17 (0.97) | 20.85** | 1.65 (1.05) | 1.50 (1.03) | 4.41** |
BPD13 | Do things impulsively | 0.79 (0.91) | 0.59 (0.77) | 1.45 (1.02) | 24.85** | 0.71 (0.90) | 0.89 (0.90) | –6.39** |
BPD14 | Happy person (RS) | 1.32 (1.02) | 1.14 (0.97) | 1.91 (0.95) | 22.06** | 1.31 (1.03) | 1.24 (0.98) | 2.15 |
BPD15 | Can’t handle separation | 1.28 (1.01) | 1.09 (0.94) | 1.90 (0.98) | 23.65** | 1.36 (1.03) | 1.13 (0.95) | 7.29** |
BPD16 | Mistakes in picking friends | 0.94 (1.00) | 0.69 (0.86) | 1.75 (1.02) | 30.07** | 0.98 (1.02) | 0.79 (0.91) | 6.02** |
BPD17 | Hurt self when upset | 0.38 (0.75) | 0.22 (0.56) | 0.91 (1.01) | 21.01** | 0.36 (0.73) | 0.38 (0.75) | –1.12 |
BPD18 | Can’t express all of anger | 0.99 (1.04) | 0.71 (0.87) | 1.91 (1.02) | 34.11** | 0.94 (1.03) | 0.98 (1.01) | –1.24 |
BPD19 | Don’t get bored (RS) | 1.79 (0.98) | 1.67 (0.96) | 2.18 (0.96) | 14.83** | 1.78 (0.99) | 1.76 (0.96) | 0.57 |
BPD20 | Stay friends with people (RS) | 1.42 (0.85) | 1.33 (0.83) | 1.70 (0.86) | 12.23** | 1.42 (0.86) | 1.37 (0.83) | 1.75 |
BPD21 | Too impulsive | 0.72 (0.88) | 0.52 (0.74) | 1.38 (0.98) | 25.80** | 0.65 (0.86) | 0.79 (0.88) | –5.03** |
BPD22 | Spend money easily | 1.25 (1.06) | 1.05 (0.98) | 1.92 (1.04) | 23.54** | 1.22 (1.06) | 1.25 (1.04) | –0.69 |
BPD23 | Reckless person | 0.60 (0.81) | 0.42 (0.66) | 1.19 (0.97) | 23.97** | 0.51 (0.77) | 0.68 (0.83) | –6.84** |
BPD24 | Careful about money (RS) | 1.47 (0.98) | 1.36 (0.96) | 1.85 (0.96) | 14.35** | 1.46 (0.97) | 1.44 (0.98) | 0.43 |
Note. PAI-BOR = Personality Assessment Inventory - Borderline subscale. Bonferroni correction (.05/24) for significance of t-tests applied
p < .002,
p < .0004.
These estimates were relatively stable, with the CS-coefficient for strength (.74) above .50. Node centrality was relatively unrelated to item variability: the correlation between item standard deviation and node strength (r = .17, p = .43) was a small sized association (Cohen, 1977). Rank-ordered node strength was also negatively related to rank-ordered item mean levels (ρ = −.24, p = .25), exhibiting a small-to-medium sized association (Cohen, 1977).
Aim 2: Comparing the Network Structure of BPD Features Between Participants with Low and High BPD Features
We plotted graphical depictions of the networks of 24 items of BPD features from the PAI-BOR in the Low BPD group (Figure 2A) and the High BPD group (Figure 2B). We also compared node strength (Figure 2C) and expected influence (Figure 2D) between the two groups. When compared as independent networks, the maximum difference in any of the associations of the High BPD group was significantly different than the Low BPD group, M = 0.16, p < .01, suggesting the structure of the PAI-BOR items differed between groups. However, only 2 associations (0.7%) were significantly different (ps < .05) between groups after applying a Holm-Bonferroni correction for multiple tests. The network of the High BPD group (global strength = 6.97) demonstrated lower global strength than the network of the Low BPD group (global strength = 10.00), S = 3.03, p < .01, suggesting greater connectivity in the Low BPD group than in the High BPD group (van Borkulo et al., 2017)2.
Figure 2.
A) Network structure of BPD features in the Low BPD group (PAI-BOR < 38). B) Network structure of BPD features in the High BPD group (PAI-BOR ≥ 38). C) Node strength centrality indices for BPD features in both groups. D) Node strength (expected influence) centrality indices for BPD features in both groups. PAI-BOR = Personality Assessment Inventory – Borderline subscale.
The items with the greatest node strength in the High BPD group were BPD12 (rarely lonely [reverse-scored]; z = 1.71) from the Negative Relationships subscale, BPD23 (reckless person; z = 1.35) and BPD21 (too impulsive; z = 1.09) from the Self-Harm subscale, and BPD4 (intense moods; z = 1.08) from the Affective Instability subscale. In contrast, the items with the greatest node strength in the Low BPD group were identical to the full sample: BPD5 (chronic emptiness; z = 2.14) from the Identity Disturbance subscale, and BPD4 (intense moods; z = 1.38), BPD1 (mood shifts; z = 1.13), and BPD14 (happy person [reverse-scored]; z = 1.07) from the Affective Instability subscale. Thus, only one item, BPD4 (intense moods), demonstrated relatively greater node strength in both groups, and, after bootstrapping the network, only BPD5 (chronic emptiness) demonstrated significantly greater node strength (p < .05) than other items in the Low BPD group but not in the High BPD group (Figure S2, Supplemental Material).
The items with the greatest expected influence in the High BPD group were BPD4 (intense moods; z = 1.78), BPD21 (too impulsive, z = 1.36), BPD7 (steady mood [reverse-scored]; z = 1.16) from the Affective Instability subscale, and BPD23 (reckless person; z = 1.02). The items with the greatest expected influence in the Low BPD group were the same as those with the greatest node strength.
These estimates were relatively stable, with CS-coefficients for strength in the High BPD (.75) and the Low BPD group (.74) both above .50. Node strength was relatively unrelated or negatively related to item standard deviation in both the High BPD group (r = .09, p = .66) and the Low BPD group (r = −.23, p = .28). Rank-ordered node strength was more strongly negatively associated with rank-ordered item mean levels among the Low BPD group (ρ = −.39, p = .06) than the High BPD group (ρ = −.12, p = .58), z = −8.15, p < .01, exhibiting medium-sized and small-sized associations, respectively (Cohen, 1977).
Aim 3: Comparing the Network Structures of BPD Features Between Women and Men
To test our third aim, we first plotted graphical depictions of the networks of 24 items of BPD features from the PAI-BOR for women and men (Figures S11A and S11B, respectively, Supplemental Material)3. When compared as independent networks, the maximum difference in any of the associations of the women’s network was significantly different than the men’s network, M = 0.12, p = .02, suggesting the structure of the PAI-BOR items differs between groups. However, none of the individual bootstrapped associations were significantly different between groups (ps > .05) after applying a Holm-Bonferroni correction for multiple tests. Further, the women’s PAI-BOR network (global strength = 10.62) demonstrated similar global strength as the men’s PAI-BOR network (global strength = 10.48), S = 0.14, p = .77, suggesting similar patterns of connectivity between the women’s and men’s networks.
We then examined centrality indices (Figure S11C, Supplemental Material). Among both women and men, the items with the highest node strength were BPD5 (chronic emptiness; zwomen = 2.18; zmen = 2.15) from the Identity Disturbance subscale, and BPD4 (intense moods; zwomen = 1.74; zmen = 1.46) and BPD1 (sudden mood shifts; zwomen = 1.04; zmen = 1.12) from the Affective Instability subscale. After bootstrapping the network, there was evidence that BPD5 (chronic emptiness) demonstrated greater node strength than all but one item among women (ps < .05; Figure S3A, Supplemental Material). Among men, BPD17 (hurt self when upset) demonstrated greater node strength than all but three items (ps < .05) and BPD5 (chronic emptiness) demonstrated greater node strength than all but four items (ps < .05), while BPD11 (wonder about life) demonstrated significantly lower node strength than all but one item (ps < .05; Figure S3B, Supplemental Material). These estimates were relatively stable, with CS-coefficients for strength among women (.74) and men (.75) above .50.
Node centrality was relatively unrelated to item variability for both women and men. Correlations between item standard deviation and node strength for women (r = .15, p = .48) and men (r = .07, p = .76) were small sized associations (Cohen, 1977). Rank-ordered node strength was more strongly negatively associated with rank-ordered mean item level among men (ρ = −.47, p = .02) than among women (ρ = −.36, p = .08), z = 4.21, p < .01, exhibiting medium-to-large sized associations (Cohen, 1977).
Aim 4: Comparing the Networks of BPD Features in Relation to Emotion Dysregulation and Interpersonal Problems Between Participants with Low and High BPD Features
To test our final aim, we conducted a network analysis using a representative item from each of the subscales of the PAI-BOR, DERS, and IIP. We used a representative item from each subscale rather than including all items from all three measures because scores rather than item scores because only a subset of participants completed all three measures (n = 1224). Estimating a network including all items from all three scales would have resulted in estimating 5778 parameters (107 items total; 107 threshold parameters; 107 × 106/2 pairwise association parameters), reducing the stability of the network. We also used a representative item from each subscale rather than the sum score of items for each subscale in order to remain consistent with the analyses conducted on the PAI-BOR items.
We identified these representative items by conducting three confirmatory factor analyses, one on each measure, using proc calis in SAS software Version 9.4. We estimated the factor loadings of each item on its previously identified subscale, while setting all other loadings for that item to 0 and allowing the factors to correlate. The item with the highest loading on each factor was used as the representative item from that scale. For the PAI-BOR, the subscales and representative items were as follows:
AI (Affective Instability) – BPD4 (intense moods)
ID (Identity Disturbance) – BPD5 (chronic emptiness)
NR (Negative Relationships) – BPD3 (stormy relationships)
SH (Self-Harm) – BPD21 (too impulsive).
For the DERS, the subscales and representative items were as follows:
Aware – DERS34 (When upset, I figure out what I feel [reverse-scored])
Clarity – DERS1 (I am clear about my feelings)
Goals – DERS18 (When upset, I have difficulty focusing)
Impulse – DERS14 (When upset, I am out of control)
Nonaccept – DERS29 (When upset, I become irritated with myself)
Strategies – DERS15 (When upset, I believe I will remain upset).
For the IIP, the subscales and representative items were as follows:
Aggressive – IIP30 (I am too aggressive)
Ambivalence – IIP21 (It is hard to feel good about others’ happiness)
Approval – IIP34 (I am affected by others’ moods too much)
Sensitive – IIP27 (I am too sensitive)
Sociability – IIP23 (It is hard to ask others to get together with me).
We first plotted graphical depictions of the networks of the 15 items representing BPD features, emotion dysregulation, and interpersonal problems for participants in the Low BPD group (Figure 3A) and those in the High BPD group (Figure 3B). When compared as independent networks, the maximum difference in any of the associations of the High BPD group was significantly lower than the Low BPD group, M = 0.19, p = .01, suggesting the structure of these networks differs between groups. However, none of the individual bootstrapped associations were significantly different between groups (ps > .05) after applying a Holm-Bonferroni correction for multiple tests. The Low BPD network (global strength = 5.80) demonstrated greater global strength than the High BPD network (global strength = 4.52), S = 1.28, p < .01, suggesting greater connectivity in the Low BPD group than in the High BPD group.
Figure 3.
A) Network structure of BPD features, emotion dysregulation (DERS), and interpersonal problems (IIP) for the Low BPD group. B) Network structure of BPD features, emotion dysregulation, and interpersonal problems for the High BPD group. C) Node strength indices for BPD features, emotion dysregulation, and interpersonal problems in both groups. D) Node strength (expected influence) indices for both groups. DERS = Difficulties in Emotion Regulation Scale. IIP = Inventory of Interpersonal Problems. PAI-BOR = Personality Assessment Inventory – Borderline subscale. AI = Affective Instability. ID = Identity Disturbance. NR = Negative Relationships. SH = Self-Harm.
We then compared centrality indices (Figure 3C). In both groups, DERS-Strategies (limited access to emotion regulation strategies; zHigh BPD = 1.66; zLow BPD = 1.64) and DERS-Nonaccept (nonacceptance of emotional responses; zHigh BPD = 1.30; zLow BPD = 0.81) demonstrated relatively high node strength. In the High BPD group, DERS-Impulse (impulse control difficulties) also demonstrated relatively high node strength (z = 1.47), while in the Low BPD group, IIP-Sociability (problems socializing and distress with others) demonstrated relatively high node strength (z = 0.75). After bootstrapping the network, DERS-Strategies demonstrated significantly greater node strength than all other items except DERS-Impulse and DERS-Nonaccept in the High BPD group, ps < .05. In the Low BPD group, DERS-Strategies demonstrated significantly greater node strength than all other items, ps < .054. The items with the greatest expected influence in each group were the same, except IIP-Approval (need for social approval) demonstrated relatively higher expected influence (z = 0.65) than IIP-Sociability (z = 0.62) in the Low BPD group.
These estimates were relatively stable, with CS-coefficients for node strength in both the High BPD group (.75) and the Low BPD group (.75) above .50. Item variability, however, demonstrated medium-to-large sized associations (Cohen, 1977) with node strength in both the High BPD group (r = .58, p = .02) and the Low BPD group (r = .51, p = .05).
Finally, because these measures assess related but distinct constructs, we tested for the presence of bridge symptoms among them. Bridge symptoms are defined as “symptoms that do not clearly belong to one or another [construct], because they receive and send out effects to the symptoms in both of the [constructs]” (Cramer, Waldorp, van der Maas, & Borsboom, 2010). To test for which items may function as bridge symptoms between the PAI-BOR, DERS, and IIP, we used the networktools package (Jones, 2017) in R. We calculated the bridge strength for the representative item of each measure’s subscales. Bridge strength is defined as the sum of the absolute value of all associations that exist between an item (e.g., ID) and all items that are not in the same community as that item (e.g., all 6 DERS items and all 5 IIP items). In the High BPD group, DERS-Nonaccept had the highest bridge strength (129), while in the Low BPD group, PAIBOR-ID had the highest bridge strength (242). In the High BPD group, nonacceptance of one’s emotions was the most central link between interpersonal problems and BPD features and between interpersonal problems and emotion dysregulation, while in the Low BPD group, identity disturbance was the most central link between interpersonal problems and emotion dysregulation and between interpersonal problems and BPD features.
Discussion
In a large and symptomatically diverse sample, we used network analytic techniques to compare the centrality of three potential core deficits in BPD: emotion dysregulation, interpersonal problems, and identity disturbance. Using a well-validated measure of BPD features, we conducted a series of network analyses to identify core aspects of BPD at levels of self-reported features both above and below the threshold for likely BPD diagnoses. We found that loneliness, recklessness, and mood instability were the most central features for those higher in BPD features, while chronic emptiness, mood intensity, and mood instability were more central for those lower in BPD features. These findings provide evidence that the associations among BPD features differ between people relatively higher and lower in BPD features.
These results provide data-driven insights into the theoretical debate over core deficits in BPD. Taken together, our results suggest BPD is defined by multiple deficits that are strongly associated with deficits in other domains. Emotion dysregulation (especially affective instability, a perceived lack of access to emotion regulation strategies, and nonacceptance of one’s emotions), impulsivity (especially recklessness and impulse control difficulties), and interpersonal problems (especially loneliness) constitute the primary areas of dysfunction for those scoring above Morey’s (1991) threshold for likely BPD status. The most central BPD features for people scoring below Morey’s threshold included emotion dysregulation (especially affective intensity and instability, unhappiness, and a perceived lack of access to emotion regulation strategies), identity problems (especially chronic emptiness), and interpersonal difficulties (especially socializing with others). Together, these findings suggest that while emotion dysregulation is a central feature for people across the range of BPD severity, impulsivity and loneliness are relatively more central among those higher in BPD features, while identity disturbance and difficulties socializing are relatively more central among those lower in BPD features.
These results expand on Richetin, Preti, Costantini, and de Panfilis’s (2017) network analysis of BPD symptoms. The authors reported that, among clinical patients and students, affective instability, identity disturbance, and efforts to avoid abandonment were the most central symptoms. These results most closely resemble our findings in the Low BPD group. This may be expected, given that only 31 participants in Richetin et al.’s clinical group (32.3%) were diagnosed with BPD. In contrast, our High BPD group consisted of 108 participants with diagnosed BPD and a further 915 participants scoring at or above an established threshold of BPD-specific severity, which may have provided us more power to detect unique properties of the networks of these participants.
By testing for bridge symptoms among the constructs of BPD features, emotion dysregulation, and interpersonal problems, we also provided preliminary evidence of how the cross-sectional associations among these constructs are maintained. While the nonacceptance of one’s emotions functioned as a bridge between emotion dysregulation and both BPD features and interpersonal problems among those with elevated BPD features, disturbance in one’s identity, specifically chronic emptiness, functioned as a bridge between BPD features and both emotion dysregulation and interpersonal problems among those with lower BPD features. Given the heterogeneity of BPD, these specific symptoms may be particularly important areas to consider when conceptualizing the maintenance of this broader network.
Because these data are between-person and cross-sectional, we cannot examine causality or directionality (Bos et al., 2017). Rather, these results indicate how BPD features, emotion dysregulation, and interpersonal problems co-occur at a given point in time, and future researchers should examine these networks longitudinally. Thus, it may be that among those higher in BPD features, loneliness directly influences affective instability, unhappiness, and boredom; loneliness is directly influenced by these three features; or any number of other possible influential relations among these four variables. Future researchers should examine these networks with repeated measures in a longitudinal design to test the directed structure of BPD features. Further, because these data are at the between-person level, we cannot say whether the structure of these networks remains constant for any person over time. For instance, treatment for BPD may reduce mean levels of BPD symptomatology (McMain et al., 2009; Rizvi, Hughes, Hittman, & Vieira Oliveira, 2017) and alter the network structure of symptoms to be more similar to those lower in BPD features. Alternatively, treatment for BPD may reduce mean levels of BPD symptomatology while leaving the structure of the network of symptoms unaffected. Future researchers should examine this question in clinical trials.
Longitudinal network analyses should also be used to test etiological and developmental courses of BPD. Using longitudinal designs, researchers should test whether the network structure of BPD features functions as a risk factor for the development of BPD. For instance, it may be possible for a given person to have a low mean level of BPD features but a network structure more similar to those with high BPD features. A network that is more similar to those with high BPD features may be more likely to allow changes in central symptoms to propagate to other symptoms, leading to the development of a full presentation of BPD. At present, this is a speculative hypothesis that researchers could examine in existing developmental datasets.
To address the discrepancy in the literature concerning gender differences in BPD (cf. Grant et al., 2008; Widiger & Trull, 1993), we compared the network structures of BPD features between men and women. We found almost no differences in the centrality estimates of these networks, although women exhibited higher scores on overall BPD features and half of the PAI-BOR items, in line with Silberschmidt, Lee, Zanarini, and Schulz’s (2014) findings. This suggests that, while women may have greater levels of BPD symptomatology, the associations among BPD features are no different between men and women. That is, for someone with elevated BPD features, certain BPD features (e.g., loneliness) are more connected to other features (e.g., unhappiness, mood instability, boredom) regardless of the person’s gender. These findings also suggest that clinical theories of core deficits in BPD based primarily on female clients are likely generalize to male clients as well.
Finally, we found negative associations between rank-ordered node strength and rank-ordered mean levels of each item from the PAI-BOR. These associations ranged from small- to large-sized and were stronger among those in the Low BPD group than the High BPD group and among men than women. These results suggest that node strength is not redundant with mean item levels. Because PAI-BOR items are rated on a scale from not at all true about me to very true about me, these results suggest that the centrality of items to the network of BPD features may be negatively related or relatively unrelated to how true these features seem of people. Future researchers should explicitly test competing theories of node strength (e.g., node strength indicates item severity, prevalence, or persistence).
These results should be considered in light of this study’s limitations. The primary statistical limitation of our study is the creation of the High and Low BPD groups. Muthén (1989) noted that if a sample is divided into groups based on a total score and then a factor analysis, which is mathematically equivalent to a network analysis (Epskamp, Rhemtella, & Borsboom, 2017), is conducted on the items comprising that total score, the factor loadings may be biased. However, as of now there are no satisfactory solutions to this bias to our knowledge, so we encourage caution when interpreting the results comparing the High and Low BPD groups. Additionally, because networks can change substantially depending on what symptoms are included or excluded (Epskamp, Borsboom, & Fried, 2017), future researchers should administer a separate validated measure of identity disturbance to provide a more accurate assessment of its associations with BPD features. We relied on the items representing the Identity Disturbance subscale of the PAI-BOR to test this core deficit, but because we also administered separate measures of emotion dysregulation and interpersonal problems, we may have included more evidence for the associations with these latter measures. Future researchers should also include a wider variety of measures of psychopathology to better assess how specific aspects of emotional disorders relate to each other, given concerns with the current diagnostic taxonomy (McNally, 2011). Finally, we did not account for the between-study variability that may have resulted from aggregating data across multiple studies, because we found no viable way to do so with these data within a network analytic framework.
Although data on income, education, or socioeconomic status was collected in each study, these measures were not consistent across studies and are not reported here. The generalizability of these findings may be limited by the relatively young average age of participants and the relatively large proportion of Caucasian participants. Because BPD is a heterogeneous disorder, fitting group-level models to BPD features may obscure differential associations based on differential symptom presentations. Because we only collected diagnostic information on a subset of participants, it is possible that our measure of BPD features (i.e., PAI-BOR scores ≥ 38) does not accurately characterize clinically diagnosed BPD. However, we believe this is unlikely, given the sensitivity and specificity of Morey’s (1991) suggested threshold score on the PAI-BOR to BPD diagnosis in this sample.
This is one of the first studies to our knowledge to assess the network structure of BPD features in a large sample representing the full range of BPD severity. Using the most recent tools in network analysis, we identified differential patterns of core deficits among people higher and lower in BPD features. We found gender differences in mean levels but not in the network structure of BPD features. We also tested networks of BPD features and two related constructs (emotion dysregulation and interpersonal problems) to probe core deficits in a broader network of features and to identify bridge symptoms between constructs. Finally, we provided results and recommendations for longitudinal and treatment researchers to build on in order to enhance our understanding of the network structure of BPD.
Supplementary Material
Acknowledgments:
This work was made possible through the data collection efforts of Erin Altenburger, Cinthia Benitez, Abbey Chappell, Lyvia Chriki, Olivia Fitzpatrick, Vibh Forsythe, Madison Guter, Jane Heiy, Sophie Lazarus, Sara Moss, and Ciara Stigen. We would also like to thank Paul De Boeck for his statistical consultation on this project.
Funding
This work was partially supported by The Ohio State University Center for Clinical and Translational Science under Grant #TL1TR001069 to M.W.S. The funding source had no involvement in the conduct or preparation of the research.
Footnotes
Epskamp et al. (2017) note that they chose a correlation of r = .70 as a default for the CS-coefficient because it corresponds to a large effect size (Cohen, 1977), although both Cohen’s designation and Epskamp et al.’s use of this default are largely arbitrary. We utilized this correlation to remain consistent with and provide a direct comparison to Epskamp et al.’s results.
Due to concerns about subsetting a sample based on a total score and conducting a network analysis on the items that comprise that total score (Muthén, 1989), we conducted two further network comparison tests. First, the structure of the network of participants who completed structured diagnostic interviews and were diagnosed with BPD did not differ from the structure of the network of participants who did not complete diagnostic interviews and who scored ≥ 38 on the PAI-BOR, M = .26, p = .95, 0 associations (0%) significantly different (ps > .05). Second, the structure of the network of participants who completed structured diagnostic interviews and were not diagnosed with BPD also did not differ from the structure of the network of participants who did not complete diagnostic interviews and who scored < 38 on the PAI-BOR, M = .25, p = .99, 7 associations (2.5%) significantly different (ps < .05).
The graphical layout of these networks looks somewhat different than that in Figures 1A, 2A, and 2B. Networks can only be computed on complete datasets, and a subset of participants (n = 180; 3.9%) did not provide their gender. Participants who did not identify their gender were similar in age (M = 23.33, SD = 23.50) to those who did identify their gender (M = 22.43, SD = 8.13), t(2) = .07, p = .95, 95% CI [–59.27, 57.47]. Participants who did not identify their gender were also composed of a similar proportion of Caucasian participants (100%) as those who did identify their ethnicity (74.5%), χ2(1) = .68, p = .41, 95% CI [–58.70, 26.87]. However, participants who did not identify their gender had significantly greater PAI-BOR scores (M = 41.71, SD = 14.12) than those who identified their gender (M = 27.81, SD = 12.12), t(4384) = 14.96, p < .01, 95% CI [12.08, 15.72]. Given the stability of the networks identified above, the relatively small proportion of participants who did not provide their gender, and the focus in these analyses on comparing the networks of men to women, we do not believe this substantially limits the accuracy or generalizability of these results.
We further compared the strength of each construct (i.e., BPD features, emotion dysregulation, interpersonal problems) within the High BPD group and the Low BPD group, respectively, using permutation tests. Emotion dysregulation and interpersonal problems each demonstrated greater strength than BPD features in both groups, ps < .01. However, there were no significant differences between the strength of emotion dysregulation and interpersonal problems in either group, ps > .20.
References
- Ainsworth MDS, Blehar MC, Waters E, & Wall SN (1978). Patterns of attachment: A psychological study of the strange situation. Hillsdale, NJ: Lawrence Erlbaum Associates. [Google Scholar]
- American Psychiatric Association. (2013a). Alternative DSM-5 model for personality disorders In Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author; 10.1176/appi.books.9780890425596.AlternatePersonalityDisorders [DOI] [Google Scholar]
- American Psychiatric Association. (2013b). Personality disorders In Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author; 10.1176/appi.books.9780890425596.dsm18 [DOI] [Google Scholar]
- Arntz A, van den Hoorn M, Cornelius J, Verheul R, van den Bosch WM, & de Bie AJ (2003). Reliability and validity of the borderline personality disorder severity index. Journal of Personality Disorders, 17, 45–49. 10.1521/pedi.17.1.45.24053 [DOI] [PubMed] [Google Scholar]
- Babyak MA (2004). What you see may not be what you get: A brief, nontechnical introduction to overfitting in regression-type models. Psychosomatic Medicine, 66, 411–421. 10.1097/00006842-200405000-00021 [DOI] [PubMed] [Google Scholar]
- Bender DS, & Skodol AE (2007). Borderline personality as a self-other representational disturbance. Journal of Personality Disorders, 21, 500–517. 10.1521/pedi.2007.21.5.500 [DOI] [PubMed] [Google Scholar]
- Benitez C (2014). An experimental manipulation of validating and invalidating responses: Impact on social-problem solving (Master’s thesis). Retrieved from OhioLINK. (osu1417185959) http://rave.ohiolink.edu/etdc/view?acc_num=osu1417185959 [Google Scholar]
- Benitez C (2017). An experimental manipulation of validating and invalidating responses: Impact on affective and cognitive flexibility [Google Scholar]
- Borgatti SP (2005). Centrality and network flow. Social Networks, 27, 55–71. 10.1016/j.socnet.2004.11.008 [DOI] [Google Scholar]
- Bos FM, Snippe E, de Vos S, Hartmann JA, Simons CJP, van der Krieke L, de Jonge P, & Wichers M (2017). Can we jump from cross-sectional to dynamic interpretations of networks? Implications for the network perspective in psychiatry. Psychotherapy and Psychosomatics, 86, 175–177. 10.1159/000453583 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bowlby J (1983). Attachment: Attachment and loss (Vol. 1) (2nd ed.). New York, NY: Basic Books. [Google Scholar]
- Cheavens JS, Lazarus SA, & Herr NR (2014). Interpersonal partner choice by individuals with elevated features of borderline personality disorder. Journal of Personality Disorders, 28, 594–606. 10.1521/pedi_2013_27_069 [DOI] [PubMed] [Google Scholar]
- Cheavens JS, Strunk DR, & Chriki L (2012). A comparison of three theoretically important constructs: What accounts for symptoms of borderline personality disorder? Journal of Clinical Psychology, 68, 477–486. 10.1002/jclp.20870 [DOI] [PubMed] [Google Scholar]
- Chernik MR (2011). Bootstrap methods: A guide for practitioners and researchers. New York: Wiley. [Google Scholar]
- Choi-Kain LW, Zanarini MC, Frankenburg FR, Fitzmaurice GM, & Reich DB (2010). A longitudinal study of the 10-year course of interpersonal features in borderline personality disorder. Journal of Personality Disorders, 24, 365–376. 10.1521/pedi.2010.24.3.365 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Clifton A, & Pilkonis PA (2007). Evidence for a single latent class of Diagnostic and Statistical Manual of Mental Disorders borderline personality pathology. Comprehensive Psychiatry, 48, 70–78. 10.1016/j.comppsych.2006.07.002 [DOI] [PubMed] [Google Scholar]
- Cohen J (1977). Statistical power analysis for the behavioral sciences. New York: Academic Press. [Google Scholar]
- Cohen J, & Cohen P (1983). Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum. [Google Scholar]
- Conway C, Hammen C, & Brennan P (2012). A comparison of latent class, latent trait, and factor mixture models of DSM-IV borderline personality disorder criteria in a community setting: Implications for DSM-5. Journal of Personality Disorders, 26, 793–803. 10.1521/pedi.2012.26.5.793 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Costantini G, & Epskamp S (2017). EstimateGroupNetwork: Perform the joint graphical lasso and selects tuning parameters (Version 0.1.2). Available from https://CRAN.R-project.org/package=EstimateGroupNetwork
- Costantini G, Epskamp S, Borsboom D, Perugini M, Mõttus R, Waldorp LJ, & Cramer AJO (2015). State of the art personality research: A tutorial on network analysis of personality data in R. Journal of Research in Personality, 54, 13–29. 10.1016/j.jrp.2014.07.003 [DOI] [Google Scholar]
- Costantini G, Richetin J, Preti E, Casini E, Epskamp S, & Perugini M (2017). Stability and variability of personality networks: A tutorial on recent developments in network psychometrics. Personality and Individual Differences. Advance online publication. 10.1016/j.paid.2017.06.011 [DOI] [Google Scholar]
- Cramer AOJ, Waldorp LJ, van der Maas HLJ, & Borsboom D (2010). Comorbidity: A network perspective. Behavioral and Brain Sciences, 33, 137–193. 10.1017/S0140525X09991567 [DOI] [PubMed] [Google Scholar]
- Cuthbert BN (2014). The RDoC framework: Facilitating transition from ICD/DSM to dimensional approaches that integrate neuroscience and psychopathology. World Psychiatry, 13, 28–35. 10.1002/wps.20087 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Danaher P, Wang P, & Witten DM (2014). The joint graphical lasso for inverse covariance estimation across multiple classes. Journal of the Royal Statistical Society, Series B (Statistical Methodology), 76, 373–397. 10.1111/rssb.12033 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Epskamp S, Borsboom D, & Fried E (2017). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods. Advance online publication. 10.3758/s13428-017-0862-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Epskamp S, Cramer AOJ, Waldorp LJ, Schmittmann VD, & Borsboom D (2012). Qgraph: Network visualizations of relationships in psychometric data. Journal of Statistical Software, 48, 1–18. Available from http://www.jstatsoft.org/v48/i04/ [Google Scholar]
- Epskamp S, Rhemtella MT, & Borsboom D (2017). Generalized network psychometrics: Combining network and latent variable models. Psychometrika, 82, 904–927. 10.1007/s11336-017-9557-x [DOI] [PubMed] [Google Scholar]
- Feske U, Kirisci L, Tarter RE, & Pilkonis PA (2007). An application of item response theory to the DSM-III-R criteria for borderline personality disorder. Journal of Personality Disorders, 21, 418–433. 10.1521/pedi.2007.21.4.418 [DOI] [PubMed] [Google Scholar]
- First MB, Spitzer RL, Gibbon M, Williams JBW, & Benjamin LS (1997). Structured clinical interview for DSM-IV axis II personality disorders (SCID-II). Washington, DC: American Psychiatric Press. [Google Scholar]
- Forsythe VA (2011). The moderating effect of borderline personality disorder symptoms on the relationship between interpersonal emotion regulation and the perception of social support (Master’s thesis). Retrieved from OhioLINK. (osu1299529857) http://rave.ohiolink.edu/etdc/view?acc_num=osu1299529857 [Google Scholar]
- Forsythe VA (2014). Choosing emotion regulation strategies: The effects of interpersonal cues and symptoms of borderline personality disorder (Doctoral dissertation). Retrieved from OhioLINK (osu1405095742) http://rave.ohiolink.edu/etdc/view?acc_num=osu1405095742 [Google Scholar]
- Fossati A, Borroni S, Feeney J, & Maffei C (2012). Predicting borderline personality disorder features from personality traits, identity orientation, and attachment styles in Italian nonclinical adults: Issues of consistency across age ranges. Journal of Personality Disorders, 26, 280–297. 10.1521/pedi.2012.26.2.280 [DOI] [PubMed] [Google Scholar]
- Fossati A, Maffei C, Bagnato M, Donati D, Namia C, & Novella L (1999). Latent structure analysis of DSM-IV borderline personality disorder criteria. Comprehensive Psychiatry, 40, 72–79. 10.1016/S0010-440X(99)90080-9 [DOI] [PubMed] [Google Scholar]
- Foygel R, & Drton M (2010). Extended Bayesian information criteria for Gaussian graphical models. Advances in Neural Information Processing Systems, 22, 2020–2028. [Google Scholar]
- Fried EI, van Borkulo CD, Cramer AJO, Boschloo L, Schoevers RA, & Borsboom D (2017). Mental disorders as networks of problems: A review of recent insights. Social Psychiatry and Psychiatric Epidemiology, 52, 1–10. 10.1007/s00127-016-1319-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- Friedman J, Hastie T, & Tibshirani R (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9, 432–441. 10.1093/biostatistics/kxm045 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Glenn CR, & Klonsky ED (2009). Emotion dysregulation as a core feature of borderline personality disorder. Journal of Personality Disorders, 23, 20–28. 10.1521/pedi.2009.23.1.20 [DOI] [PubMed] [Google Scholar]
- Goodwin LD, & Leech NL (2006). Understanding correlation: Factors that affect the size of r. The Journal of Experimental Education, 74, 251–266. 10.3200/JEXE.74.3.249-266 [DOI] [Google Scholar]
- Grant BF, Chou SP, Goldstein RB, Huang B, Stinson FS, Saha TD, … Ruan WJ (2008). Prevalence, correlates, disability, and comorbidity of DSM-IV borderline personality disorder: Results from the Wave 2 National Epidemiologic Survey on Alcohol and Related Conditions. Journal of Clinical Psychiatry, 69, 533–545. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gratz KL, & Roemer L (2004). Multidimensional assessment of emotion regulation and dysregulation: Development, factor structure, and initial validation of the Difficulties in Emotion Regulation Scale. Journal of Psychopathology and Behavioral Assessment, 26, 41–54. 10.1023/B:JOBA.0000007455.08539.94 [DOI] [Google Scholar]
- Gunderson JG (1996). The borderline patient’s intolerance of aloneness: Insecure attachments and therapist availability. American Journal of Psychiatry, 153, 752–758. 10.1176/ajp.153.6.752 [DOI] [PubMed] [Google Scholar]
- Gunderson JG (2007). Disturbed relationships as a phenotype for borderline personality disorder. American Journal of Psychiatry, 164, 1637–1640. 10.1176/appi.ajp.2007.07071125 [DOI] [PubMed] [Google Scholar]
- Hallquist MN, & Pilkonis PA (2012). Refining the phenotype of borderline personality disorder: Diagnostic criteria and beyond. Personality Disorders: Theory, Research, and Treatment, 3, 228–246. 10.1037/a0027953 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hawkins AA, Furr RM, Arnold EM, Law MK, Mneimne M, & Fleeson W (2014). The structure of borderline personality disorder symptoms: A multi-method, multi-sample examination. Personality Disorders: Theory, Research, and Treatment, 5, 380–389. 10.1037/per0000086 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Heiy JE (2014). A brief intervention on treatment-seeking: Barriers to mental health treatment in primary care (Doctoral dissertation). (osu1401189193) Retrieved from OhioLINK. http://rave.ohiolink.edu/etdc/view?acc_num=osu1401189193 [Google Scholar]
- Heiy JE, & Cheavens JS (2014). Back to basics: A naturalistic assessment of the experience and regulation of emotion. Emotion, 14, 878–891. 10.1037/a0037231 [DOI] [PubMed] [Google Scholar]
- Insel TR (2013, April 29). Transforming diagnosis. Retrieved from https://www.nimh.nih.gov/about/directors/thomas-insel/blog/2013/transforming-diagnosis.shtml [Google Scholar]
- Johansen M, Karterud S, Pedersen G, Gude T, & Falkum E (2004). An investigation of the prototype validity of the borderline DSM-IV construct. Acta Psychiatrica Scandinavica, 109, 289–298. 10.1046/j.1600-0447.2003.00268.x [DOI] [PubMed] [Google Scholar]
- Joiner TE (2005). Why people die by suicide. Cambridge, MA: Harvard University Press. [Google Scholar]
- Jones P (2017). networktools: Tools for identifying important nodes in networks (Version 1.1.0). Available from https://CRAN.R-project.org/package=networktools [Google Scholar]
- Kernberg OF (1967). Borderline personality organization. Journal of the American Psychoanalytic Association, 15, 641–685. 10.1177/000306516701500309 [DOI] [PubMed] [Google Scholar]
- Kernberg OF, & Michaels R (2009). Borderline personality disorder. American Journal of Psychiatry, 166, 505–508. 10.1176/appi.ajp.2009.09020263 [DOI] [PubMed] [Google Scholar]
- Lauritzen SL (1996). Graphical models. New York, NY: Oxford University Press, Inc. [Google Scholar]
- Lazarus SA (2015). Social network stability in borderline personality disorder: A longitudinal analysis (Doctoral dissertation). (osu1433783752) http://rave.ohiolink.edu/etdc/view?acc_num=osu1433783752 [Google Scholar]
- Lazarus SA, Southward MW, & Cheavens JS (2016). Do borderline personality disorder features and rejection sensitivity predict social network outcomes over time? Personality and Individual Differences, 100, 62–67. 10.1016/j.paid.2016.02.032 [DOI] [Google Scholar]
- Levy KN, Clarkin JF, Yeomans FE, Scott LN, Wasserman RH, & Kernberg OF (2006). The mechanisms of change in the treatment of borderline personality disorder with transference focused psychotherapy. Journal of Clinical Psychology, 62, 481–501. 10.1002/jclp.20239 [DOI] [PubMed] [Google Scholar]
- Linehan MM (1993). Cognitive-behavioral treatment of borderline personality disorder. New York, NY: Guilford Press. [Google Scholar]
- Main M, & Solomon J (1990). Procedures for identifying infants as disorganized/disoriented during the Ainsworth Strange Situation In Greenberg MT, Ciccheti D, & Cummings EM (Eds.), Attachment in the preschool years (pp. 121–160). Chicago, IL: University of Chicago Press. [Google Scholar]
- McMain SF, Links PS, Gnam WH, Guimond T, Cardish RJ, Korman L, & Streiner DL (2009). A randomized trial of dialectical behavior therapy versus general psychiatric management for borderline personality disorder. American Journal of Psychiatry, 166, 1365–1374. 10.1176/appi.ajp.2009.09010039 [DOI] [PubMed] [Google Scholar]
- McNally RJ (2011). What is mental illness? Cambridge, MA: Belknap Press of Harvard University Press. [Google Scholar]
- Morey LC (1991). The personality assessment inventory professional manual. Odessa, FL: Psychological Assessment Resources. [Google Scholar]
- Muthén BO (1989). Factor structure in groups selected on observed scores. British Journal of Mathematical and Statistical Psychology, 42, 81–90. 10.1111/j.2044-8317.1989.tb01116.x [DOI] [Google Scholar]
- Myers L, & Sirois MJ (2006). Spearman correlation coefficients, differences between. Encyclopedia of Statistical Sciences, 12 10.1002/0471667196.ess5050.pub2 [DOI] [Google Scholar]
- Pilkonis PA, Kim Y, Proietti JM, & Barkham M (1996). Scales for personality disorders developed from the Inventory of Interpersonal Problems. Journal of Personality Disorders, 10, 355–369. 10.1521/pedi.1996.10.4.355 [DOI] [Google Scholar]
- Preacher KJ (2002). Calculation for the test of the difference between two independent correlation coefficients [Computer software]. Available from http://quantpsy.org. [Google Scholar]
- R Core Team. (2017). R: A language and environment for statistical computing (Version 3.4.1) [Software]. Vienna, Austria: R Foundation for Statistical Computing; Available from https://www.R-project.org/. [Google Scholar]
- Richetin J, Preti E, Costantini G, & de Panfilis C (2017). The centrality of affective instability and identity in borderline personality disorder: Evidence from network analysis. PLoS ONE, 12: e0186695 10.1371/journal.pone.0186695 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rizvi SL, Hughes CD, Hittman AD, Vieira Oliveira P (2017). Can trainees effectively deliver dialectical behavior therapy for individuals with borderline personality disorder? Outcomes from a training clinic. Journal of Clinical Psychology, 73, 1599–1611. 10.1002/jclp.22467 [DOI] [PubMed] [Google Scholar]
- Robinaugh DJ, Miller AJ, & McNally RJ (2016). Identifying highly influential nodes in the complicated grief network. Journal of Abnormal Psychology, 125, 747–757. 10.1037/abn0000181 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sanislow CA, Grilo CM, Morey LC, Bender DS, Skodol AE, Gunderson JG, … McGlashan TH (2002). Confirmatory factor analysis of DSM-IV criteria for borderline personality disorder: Findings from the Collaborative Longitudinal Personality Disorders Study. American Journal of Psychiatry, 159, 284–290. 10.1176/appi.ajp.159.2.284 [DOI] [PubMed] [Google Scholar]
- Silberschmidt A, Lee S, Zanarini M, & Schulz SC (2015). Gender differences in borderline personality disorder: Results from a multinational, clinical trial sample. Journal of Personality Disorders, 29, 828–838. 10.1521/pedi_2014_28_175 [DOI] [PubMed] [Google Scholar]
- Spitzer RL, Endicott J, & Gibbon M (1979). Crossing the border into borderline personality and borderline schizophrenia. Archives of General Psychiatry, 36, 17–24. 10.1001/archpsyc.1979.01780010023001 [DOI] [PubMed] [Google Scholar]
- Terluin B, de Boer MR, & de Vet HCW (2016). Differences in connection strength between mental symptoms might be explained by differences in variance: Reanalysis of network data did not confirm staging. PLoS One, 11, e0155205 10.1371/journal.pone.0155205 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Trull TJ (1995). Borderline Personality Disorder features in nonclinical young adults: 1. identification and validation. Psychological Assessment, 7, 33–41. 10.1037/1040-3590.7.1.33 [DOI] [Google Scholar]
- van Borkulo CD (2016). NetworkComparisonTest: Statistical comparison of two networks based on three invariance measures (Version 2.0.1). Available from https://CRAN.R-project.org/package=NetworkComparisonTest
- van Borkulo CD, Boschloo L, Kossakowski JJ, Tio P, Schoevers RA, Borsboom D, & Waldorp LJ (2017). Comparing network structures on three aspects: A permutation test. Manuscript in preparation. [DOI] [PubMed] [Google Scholar]
- van Buuren S, & Groothuis-Oudshoorn K (2011). mice: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45, 1–67. 10.18637/jss.v045.i03 [DOI] [Google Scholar]
- Widiger TA, & Trull T (1993). Borderline and narcissistic personality disorders In Sutker P & Adams H, (Eds.), Comprehensive textbook of psychopathology (2nd ed., pp. 371–394). New York, NY: Plenum. [Google Scholar]
- Zanarini MC, Frankenburg FR, Reich DB, Silk KR, Hudson JI, & McSweeney LB (2007). The subsyndromal phenomenology of borderline personality disorder: A 10-year follow-up study. American Journal of Psychiatry, 164, 929–935. 10.1176/ajp.2007.164.6.929 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.