Abstract
Bipolar disorder is typified by episodes of manic/hypomanic and depressive symptoms, either distinctly or concurrently as mixed symptoms. While depressive symptoms are the major driver of risk, it is unclear whether specific combinations of manic and anxiety symptoms contribute differentially to suicidal ideation and behavior in individuals with bipolar disorder during a depressive state. This study uses a quantitative application of Rothman’s theoretical framework of causation, or ‘causal pies’ model. Data were obtained from the National Network of Depression Centers Mood Outcomes Program for 1028 visits from 626 individuals with bipolar disorder with current moderate-to-severe depressive symptoms, operationalized as a Patient Health Questionnaire-8 (PHQ-8) score ≥10. Mania symptoms were captured using the Altman Self-Rating Mania scale (ASRM) and anxiety symptoms were captured using the Generalized Anxiety Disorder-7 scale (GAD-7). The outcome of suicidal ideation or behavior was captured using the Columbia Suicide Severity Rating Scale (C-SSRS). In this cohort of individuals with bipolar disorder and at least moderate depressive symptoms, we found no increased risk of suicidal ideation or behavior attributable to manic and anxiety symptom clusters in individuals with bipolar disorder during depressive state. A small amount (4%) of risk was attributable to having severe depressive symptoms. These findings, however, may be influenced by limitations in sample size and measurement instruments. Future studies would benefit from larger samples and more rigorous assessments, including clinician-rated measures.
Keywords: Suicide, Mania, Depression, Anxiety, Bipolar Disorder
1. Introduction
Bipolar disorder is defined by episodes of manic/hypomanic and depressive symptoms, which may occur either during discrete time periods or concurrently, as mixed symptoms.(Swann et al., 2013) Depressive symptoms are the major driver of suicide risk in bipolar disorder. A recent series of analyses demonstrate that mixed symptoms are not associated with increased risk of suicidal ideation or behavior beyond that attributable to the depressive symptom component (Fiedorowicz et al., 2020; Fiedorowicz et al., 2019; Persons et al., 2018). In the first of this series, Persons et al. examine a cohort of 429 individuals with bipolar disorder from the Collaborative Depression Study (CDS), a multicenter cohort followed for a mean of 18 years, and conclude that individuals with bipolar disorder who experience mixed states have a greater burden of depressive symptoms, however mixed symptoms do not increase risk of suicidal behavior beyond that attributable to the depressive symptom component (Persons et al., 2018). A follow-up study by Fiedorowicz et al. using a cohort of 290 patients with bipolar disorder from the National Network of Depression Centers (NNDC) Clinical Care Registry, a multicenter real-world clinical sample, similarly found depressive symptoms to be most strongly associated with suicidal ideation or behavior (Fiedorowicz et al., 2019). In the third of this series, Fiedorowicz et al. replicate these findings in a much larger cohort of 6105 individuals with major depression and bipolar disorder prospectively followed through the NNDC Mood Outcomes Program (MOP), confirming a strong association between depressive symptoms and suicidal ideation or behavior but no evidence of synergy between manic and depressive symptoms (Fiedorowicz et al., 2020).
In these previous studies, the contribution of manic symptoms to risk of suicidal ideation or behavior during a depressive state was explored in terms of overall manic symptom burden. This approach is limited in that two individuals may show equal total scores on a mania questionnaire but experience different symptom profiles. Of note, mania is a heterogeneous state comprised of varying combinations of elevated mood, excess energy, and erratic thought and behavior (Miller et al., 2016; Parker and Graham, 2015); as such, the question arises whether manic symptom clusters differentially contribute to risk of suicidal ideation or behavior in individuals with bipolar disorder during a depressive state. These previous studies also did not look at anxiety symptoms, which are commonly reported in mixed states (Cassidy, 2010). To the authors’ knowledge, these questions has not been previously examined in the existing body of literature. This paper aims to build upon the current understanding of depressive symptoms as the major driver of suicide risk in bipolar disorder with no significant contribution from overall manic symptom burden by quantifying the contribution of manic and anxiety symptom clusters to risk of suicidal ideation or behavior in individuals with bipolar disorder during a depressive state.
To answer this question, this study invokes Rothman’s theoretical framework of causation, which posits that outcomes are the product of sufficient cause, which in turn consists of a number of component causes that must be present for the outcome to occur (Hoffmann et al., 2006; Rothman, 1976; Wensink et al., 2014). Historically, these have been represented graphically as the ‘causal pie’. A sufficient cause is the minimal set of conditions under which an outcome develops and may arise from a number of different causal pies, each representing a distinct complementary set of component causes (Figure 1). Rothman’s theoretical framework can be used to reveal the circumstances (i.e., particular risk factors or combinations thereof) sufficient to produce an outcome of interest.
Figure 1: Sufficient Cause Illustrated Example.

A sufficient cause is the minimal set of conditions under which an outcome develops, which may arise from a number of different causal pies each representing a distinct complementary set of component causes. This series of images illustrates potential combinations of the hypothetical component causes A, B, C, D, E, F, and G that are sufficient to produce an outcome.
Image adapted from: Rothman, Causes, 1976
In the context of bipolar disorder, the utility of Rothman’s theoretical framework is that it can account for heterogeneity of mood symptoms (Madsen et al., 2011) and offers a way to evaluate the composition of a seemingly homogenous condition (i.e., the manic/hypomanic component of mixed symptoms) that reveals its underlying heterogeneous nature and paves the way for more sophisticated statistical interpretation of risk within that population (Wensink et al., 2014).
In practical application of Rothman’s model a proportion of the components comprising a sufficient cause may be unknown or unmeasured, which can be left as unlabeled segments of the causal pie (denoted as U). A causal pie that includes unknown components is thusly considered a class of sufficient causes because they are known to share the identified components in common, but may differ in the composition of their unknown components (Figure 2) (Hoffmann et al., 2006).
Figure 2: Classes of Sufficient Cause Illustrated Example.

A class of sufficient causes is a complementary set of component causes, including unknown or unmeasured factors, sufficient to produce an outcome of interest. ‘Classes of sufficient cause’ differ conceptually from ‘sufficient cause’ in that they account for unknown contributors to the outcome; classes of sufficient cause are known to share their identified components, but may differ in their composition of unknown components. This series of images illustrates potential combinations of hypothetical component causes A and B sufficient to produce an outcome, accounting for unknown/unmeasured factors (denoted as U). In this example, the first class of sufficient causes includes component cause A and unknown/unmeasured factors U, which is known only to not include component cause B. The second class of component causes includes component cause B and unknown/unmeasured factors U, which is known only to not include component cause A. The third class of sufficient causes includes component cause A, component cause B, and unknown/unmeasured factors U.
Image adapted from: Hoffman, Estimating the Proportion of Disease due to Classes of Sufficient Causes, 2005
A statistical approach to Rothman’s theoretical framework has been proposed by Hoffman et al., who denote the proportion of risk attributable to a class of sufficient causes as ‘proportion of disease due to a class of sufficient cause’ (PDC) and offer the following equation (Hoffmann et al., 2006):
where RR is the adjusted relative risk in exposed versus unexposed individuals and P(E|D) is the conditional probability of having the exposure of interest given that the outcome has occurred.
The equation offered by Hoffman et al. draws upon the concept of the population attributable fraction (PAF), a statistic traditionally used within public health research to describe the burden of disease attributable to a particular modifiable risk factor to quantify the proportion of disease that would be prevented by eliminating that risk factor (Spiegelman et al., 2007). The utility of the approach outlined by Hoffman et al. in quantifying the component contributions to classes of sufficient cause is that it accounts for both the relative risk and the prevalence of each risk factor to estimate the contribution of each of a number of proposed component causes toward development of the outcome of interest.
Here, we attempt to estimate the proportion of risk of suicidal ideation or behavior due to different manic and anxiety symptom clusters in individuals with bipolar disorder who are currently experiencing at least moderate depressive symptoms, while also accounting the presence of severe depressive symptoms where applicable.
2. Method
2.1. Study Sample
This study used real-world clinical data from the National Network of Depression Centers (NNDC) Mood Outcomes Program (MOP). The registry includes English-speaking adult patients with a mood disorder who received treatment at one of the 26 participating U.S. academic depression centers. Our study sample overlaps with that of our previous paper using the NNDC MOP data (Fiedorowicz et al., 2020). Rather than including individuals with major depression, the current sample is restricted to patients with a diagnosis of bipolar disorder I, bipolar disorder II, or bipolar disorder NOS for this analysis. Our unit of analysis is the patient visit, for which multiple visits may be provided by a single patient. Through measurement-based care, patients had a standardized assessment package of self-reported scales at each visit, including the 9-item Patient Health Questionnaire (PHQ-9), the Generalized Anxiety Disorder (GAD-7) scale, the Altman Self-Rating Mania (ASRM) scale, and the Columbia Suicide Severity Rating Scale (C-SSRS). As with our previous study (Fiedorowicz et al., 2020), this current study uses the PHQ-8, which consists of the first 8 items of the PHQ-9 with the 9th item (‘Thoughts that you would be better off dead’) removed to avoid conflation with our later described primary outcome. As this study sought to evaluate the influence of manic and anxiety symptom clusters on risk of suicidal ideation or behavior in individuals with bipolar disorder during a depressive state, visits were included only for which depressive symptoms met a cut-point of PHQ-8 ≥10, a previously validated threshold for at least moderate depressive symptoms that has 88% sensitivity and 88% specificity for major depression (Kroenke et al., 2009).
2.2. Measurement Instruments
2.2.1. Manic symptoms
Manic symptoms were captured using the ASRM, which assesses five manic symptoms (elevated mood; increased self-esteem; decreased need for sleep; talkativity; increased activity) on a 0-4 Likert scale, with scores ranging from 0-20. The ASRM is a valid and reliable measure of mania in patients with an existing diagnosis of bipolar disorder, demonstrating high sensitivity but variable specificity (33-87%) for the cutpoint ASRM>5 for acute mania/hypomania (Altman et al., 2001; Altman et al., 1997).
2.2.2. Anxiety symptoms
Because the ASRM only captures manic symptoms related to elevated mood and increased energy, this study attempts to capture a more comprehensive catalog of relevant symptoms using the GAD-7, which assesses anxiety using seven items (feeling nervous, anxious, or on edge; not being able to stop or control worrying; worrying too much about different things; trouble relaxing; being so restless that it is hard to sit still; becoming easily annoyed or irritable; feeling afraid, as if something awful might happen) on a 0-3 Likert scale, with scores ranging from 0-21. The GAD-7 is a reliable and valid measure of anxiety in clinical populations (Spitzer et al., 2006).
2.2.3. Suicidal ideation or behavior
As with our previous analyses (Fiedorowicz et al., 2020; Fiedorowicz et al., 2019), the primary outcome of interest is a composite of suicidal ideation or behavior as reported on the Columbia-Suicide Severity Rating Scale (C-SSRS), a self-report questionnaire with high sensitivity and specificity that has been validated in both clinical and research samples (Posner et al., 2011). Suicidal ideation was defined as any thoughts of killing oneself or wishing oneself dead and suicidal behavior was defined as any suicide attempt or preparations for an attempt.
2.3. Statistical Analysis
This study utilizes a quantitative adaptation of Rothman’s causal pie models described by Hoffman et al (Hoffmann et al., 2006). In keeping with Hoffman et al., we employed the following approach:
Define dichotomous indicator variables for each class of sufficient cause other than the class for which all component causes are unknown (this will serve as the reference group)
Define model covariates
Calculate P(E|D) for each indicator variable (i.e., the conditional probability of having the indicator given presence of the outcome)
Estimate the adjusted relative risk for each indicator
Insert estimates into the equation: PDC=P(E|D) (RR-1)/RR
2.3.1. Indicator Variable Selection
Hoffman et al. recommend using all possible combinations of the risk factors of interest to create the indicator variables because the PDC calculated for individual risk factors are not strictly additive. Each combination should be prevalent among participants and the outcome should also have occurred in order to determine the relative risk. An issue that arises is that a very large sample size is needed in order to generate an adequate N for each possible combination of risk factors. In calculating the prevalence for all possible combinations of ASRM items, we identified a clear issue of low cell counts (Supplemental Table 1). Increasing the number of risk factors increases the number of possible combinations exponentially (2# risk factors), so to circumvent the low cell counts that would inevitably result from using all 4096 possible combinations of the twelve GAD-7 and ASRM items as indicator variables we pursued variable reduction via latent class analysis to identify distinct groups of participants typified by meaningful collections of mood symptoms (the latent classes) rather than exploring all possible combinations of mood symptoms.
Latent class analysis was conducted using Latent Gold (Vermunt and Magidson, 2013) to identify a set of clusters exhibiting distinct response patterns for the twelve ordinal ASRM and GAD-7 items. A multilevel latent class model with a random intercept was used to account for individuals possibly having contributed more than one visit. The optimal number of latent classes was determined based on the model with the lowest AIC, BIC, AIC3 & CAIC. The model having the lowest value on most of these fit indices was chosen as the best fitting model. The parsimony principle was used to select between two models showing the best fit according to an equal number of fit indices.
We also examined the influence of the presence of severe depressive symptoms on suicidal ideation or behavior, using a previously validated threshold defining moderate depressive symptoms as a PHQ-8 score 10-19 and severe depressive symptoms as PHQ-8 score ≥20 (Kroenke et al., 2001).
2.3.2. Relative Risk Calculation
Subsequent analyses were conducted using SAS 9.4 (SAS Institute, Inc., Cary, NC). Because individual patients may contribute multiple visits, the unit of analysis is the patient visit. Relative risks for the classes of component cause models were obtained using multivariable-adjusted regression analysis, employing a modified Poisson approach with robust error variances as described by Zou et al. (GENMOD – Poisson distribution, log link, repeated subject) to account for correlation between repeated observations (Zou, 2004). The approach outlined by Hoffman et al. defines the comparison group for relative risk calculations as the patient group with the lowest overall symptom burden, which for the purposes of this study is the Cluster 1 (lowest average manic and anxiety symptom score) with moderate depressive symptoms (PHQ-8 scores 10-19) patient group. Missing values for suicidal ideation/behavior, GAD-7 items, and ASRM items were imputed as 0 to maximize the number of observations available for analysis. All regression models were adjusted for age and gender. In keeping with the approach outlined by Hoffman et al., only models that returned a statistically significant adjusted relative risk estimate >1 were then input into the final PDC calculations.
2.3.3. Network analysis
Given that the causal pie models demonstrated no increased risk of suicidal ideation or behavior attributable to clusters of manic/hypomanic and anxiety symptoms, we pursued a follow-up network analysis to attempt to identify individual manic/hypomanic and anxiety symptoms that correlate with suicidal ideation after adjusting for the influence of all other symptoms in the network. We used network analysis to examine the interrelatedness of individual items of the ASRM and GAD-7 and their relationship to suicidal ideation as represented by the 9th item of the PHQ-9 (‘Thoughts that you would be better off dead’), adjusting for the remaining items of the PHQ. The decision to use the 9th item of the PHQ-9 rather than the C-SSRS to capture our outcome of suicidal ideation was made because network analysis requires that items be either all ordinal or all binary, and selection of the binary C-SSRS outcome of suicidal ideation or behavior, as was employed in the causal pies analyses, would have prevented use of the ordinal ASRM and GAD-7 items. We estimated a regularized partial correlation network on these ordinal symptom scores, meaning that connections between symptoms are adjusted for the influence of all other symptoms and that very small connections between network nodes are constrained to zero. The network was estimated using the R software (version 4.0.3) and the R-package qgraph (Epskamp et al., 2012).
3. Results
This study included 626 individual participants with bipolar disorder from the NNDC Mood Outcomes Program for a total of 1028 visits. The mean number of visits for participants was 1.6 (SD 1.8; range 1-17). Mean participant age was 41.3 years (SD 14.7; range 18-87 years); 401 participants (65%) were women. By bipolar subtype, there were 294 participants (47.0%) with bipolar 1 disorder, 196 participants (31.3%) with bipolar 2 disorder, and 136 participants (21.7%) with bipolar disorder NOS. Mean PHQ-8 score across visits was 16.0 (SD 4.3; range 1024), mean ASRM score was 3.3 (SD 3.5; range 0-19), and mean GAD-7 score was 12.9 (SD 5.6; range 0-21). Suicidal ideation or behavior was present at 505 visits (49% of total visits). There were 92 visits with suicidal behavior, which represented 18% of the outcomes.
3.1. Latent Class Analysis
Latent class analysis of the five ordinal ASRM items and seven ordinal GAD-7 items identified 7 latent clusters which were included in our final models: Cluster 1, characterized by overall low symptom burden (n=218 visits); Cluster 2, characterized by low manic symptom burden and high anxiety symptoms with more prominent nervousness, worrying, and trouble relaxing (n=179 visits); Cluster 3, characterized by low manic symptoms with more prominent talkativity and reduced need for sleep, and high anxiety symptoms with more prominent nervousness, worrying, and trouble relaxing (n=151 visits); Cluster 4, characterized by low-to-moderate manic symptoms with more prominent reduced need for sleep, talkativity, and increased activity, and high anxiety symptoms with more prominent nervousness, worrying, and trouble relaxing (n=218 visits); Cluster 5, characterized by low-to-moderate manic symptom burden and moderate anxiety symptoms with more prominent nervousness, trouble relaxing, and irritability (n=141 visits); Cluster 6, characterized by low manic symptom burden and low-to-moderate anxiety symptoms with more prominent nervousness, worrying, trouble relaxing, and irritability (n= 70 visits); and Cluster 7, characterized by moderate manic symptoms with more prominent reduced need for sleep, talkativity, and increased activity, and high anxiety symptoms with more prominent nervousness, worrying, and trouble relaxing (n=51 visits) (Table 1). Cluster 1 with moderate depressive symptoms had the lowest overall symptom burden and was therefore selected as the comparison group for relative risk calculations.
Table 1.
Latent Class Analysis
| Clusters | BIC(LL) | AIC(LL) | AIC3(LL) | CAIC(LL) | Parameters |
|---|---|---|---|---|---|
| 1 | 30426.3 | 30224 | 30265 | 30467.3 | 41 |
| 2 | 27622.5 | 27351 | 27406 | 27677.5 | 55 |
| 3 | 26736.7 | 26396.2 | 26465.2 | 26805.7 | 69 |
| 4 | 26216.7 | 25807.1 | 25890 | 26299.7 | 83 |
| 5 | 25908.5 | 25429.8 | 25526.8 | 26005.5 | 97 |
| 6 | 25735.4 | 25187.6 | 25298.6 | 25846.4 | 111 |
| 7 | 25619.5 | 25002.6 | 25127.6 | 25744.5 | 125 |
| 8 | 25622.2 | 24936.2 | 25075.2 | 25761.2 | 139 |
| 9 | 25598.6 | 24843.4 | 24996.5 | 25751.6 | 153 |
| 10 | 25605.9 | 24781.7 | 24948.7 | 25772.9 | 167 |
| 11 | 25611.6 | 24718.3 | 24899.3 | 25792.6 | 181 |
| 12 | 25634.3 | 24671.9 | 24866.9 | 25829.3 | 195 |
| 13 | 25647.6 | 24616.1 | 24825.1 | 25856.6 | 209 |
| 14 | 25718.4 | 24617.9 | 24840.9 | 25941.4 | 223 |
| |||||
| Cluster 1 | Low symptom burden | ||||
| Cluster 2 | Low manic symptom burden | Moderate-High anxiety symptoms with more prominent nervousness, worrying, and trouble relaxing | |||
| Cluster 3 | Low manic symptoms with more prominent talkativity and reduced need for sleep | High anxiety symptoms with more prominent nervousness, worrying, and trouble relaxing | |||
| Cluster 4 | Low-Moderate manic symptoms with more prominent reduced need for sleep, talkativity, and increased activity | High anxiety symptoms with more prominent nervousness, worrying, and trouble relaxing | |||
| Cluster 5 | Low-Moderate manic symptom burden | Moderate anxiety symptoms with more prominent nervousness, trouble relaxing, and irritability | |||
| Cluster 6 | Low manic symptom burden | Low-Moderate anxiety symptoms with more prominent nervousness, worrying, trouble relaxing, and irritability | |||
| Cluster 7 | Moderate manic symptoms with more prominent reduced need for sleep, talkativity, and increased activity | High anxiety symptoms with more prominent nervousness, worrying, and trouble relaxing | |||
3.2. Causal Pie Models
Table 2 displays counts for the total number of patient visits representing each mood symptom cluster, the number of patient visits within each mood symptom cluster during which the outcome of suicidal ideation or behavior was reported, the calculated conditional probability of having the exposure of interest given that the outcome has occurred, and adjusted relative risk. There were zero outcomes for Cluster 6 with severe depressive symptoms, so no further calculations were performed for this group.
Table 2.
Prevalence and Relative Risk of Suicidal Ideation or Behavior for Mania and Anxiety Symptom Clusters
| N | P(E|D)1 | RR (95% CI)2 | |||
|---|---|---|---|---|---|
| Latent Class | Depressive Symptom Severity | Total (n=1028) | With Outcome (n=505) | ||
| Cluster 1 Low symptom burden |
Moderate | 107 | 56 | 0.11 | ref3 |
| Severe | 111 | 77 | 0.15 | 1.37 (1.08-1.73) | |
| Cluster 2 • Low manic symptom burden • High anxiety symptoms with more prominent nervousness, worrying, and trouble relaxing |
Moderate | 152 | 66 | 0.13 | 0.78 (0.62-0.98) |
| Severe | 27 | 16 | 0.03 | 1.02 (0.75-1.40) | |
| Cluster 3 • Low manic symptoms with more prominent talkativity and reduced need for sleep • High anxiety symptoms with more prominent nervousness, worrying, and trouble relaxing |
Moderate | 137 | 61 | 0.12 | 0.83 (0.63-1.07) |
| Severe | 14 | 5 | 0.01 | 0.65 (0.36-1.17) | |
| Cluster 4 • Low-Moderate manic symptoms with more prominent reduced need for sleep, talkativity, and increased activity • High anxiety symptoms with more prominent nervousness, worrying, and trouble relaxing |
Moderate | 152 | 86 | 0.17 | 1.03 (0.81-1.29) |
| Severe | 66 | 44 | 0.09 | 1.15 (0.87-1.51) | |
| Cluster 5 • Low-Moderate manic symptom burden • Moderate anxiety symptoms with more prominent nervousness, trouble relaxing, and irritability |
Moderate | 126 | 51 | 0.10 | 0.66 (0.49-0.90) |
| Severe | 15 | 10 | 0.02 | 1.30 (0.84-2.02) | |
| Cluster 6 • Low manic symptom burden • Low-Moderate anxiety symptoms with more prominent nervousness, worrying, trouble relaxing, and irritability |
Moderate | 62 | 7 | 0.01 | 0.21 (0.10-0.45) |
| Severe | 8 | 0 | -- | -- | |
| Cluster 7 • Moderate manic symptoms with more prominent reduced need for sleep, talkativity, and increased activity • High anxiety symptoms with more prominent nervousness, worrying, and trouble relaxing |
Moderate | 34 | 14 | 0.03 | 1.20 (0.83-1.72) |
| Severe | 17 | 12 | 0.02 | 1.30 (0.91-1.86) | |
P(E|D) is the conditional probability of having the exposure of interest (e.g. membership in a particular latent class and depressive symptom severity group) given that the outcome (e.g. suicidal ideation or behavior as captured by the C-SSRS) has occurred
Relative risk estimates in bold indicate statistical significance at the α=0.05 level (estimates for which the 95% confidence interval does not contain 1). Relative risk estimates were obtained using multivariable-adjusted modified Poisson regression with robust error variances. Regression models were adjusted for age and gender.
The referent group for all relative risk calculations was the Cluster 1 with Moderate Depressive Symptoms group. This group was selected in keeping with the approach outlined by Hoffman et al., which defines the referent group as the patient group with the lowest overall symptom burden.
In calculation of adjusted relative risk, the only group to return a statistically significant relative risk > 1 was the overall low manic and anxiety symptom group (Cluster 1) with severe depressive symptoms (RR: 1.37, 95% CI: 1.08-1.73). No other groups demonstrated higher risk of suicidal ideation or behavior relative to the comparison group (Cluster 1 with moderate depressive symptoms). The P(E|D) and RR were next used to quantify the PDC, the proportion of our outcome of suicidal ideation or behavior attributable to membership in the Cluster 1 with severe depressive symptoms group. We found a small (4%) proportion of the risk of suicidal ideation or behavior to be attributable to crossing the threshold from moderate to severe depressive symptoms, as captured by PHQ-8 score 10-19 vs. PHQ-8 ≥ 20 (Figure 3).
Figure 3: Classes of Sufficient Cause for Suicidal Ideation or Behavior.

This figure illustrates the classes of sufficient cause (i.e., the complementary set of component causes, including unknown or unmeasured factors, sufficient to produce an outcome of interest) to produce the outcome of suicidal ideation or behavior. Within our study sample of individuals with bipolar disorder currently experiencing at least moderate depressive symptoms, only crossing the threshold from moderate depressive symptoms (PHQ-8 10-19) to severe depressive symptoms (PHQ-8 ≥ 20) was identified as a significant risk factor, for which 4% of the risk of suicidal ideation or behavior can be attributed. In this illustration, the class of sufficient cause includes the component causes of ‘severe depressive symptoms’ and ‘unknown/unmeasured factors’, which are known only to not include the other risk factors captured in this study (i.e., the 5 items of the ASRM and 7 items of the GAD-7).
3.3. Network Analysis
The cross-sectional partial correlation network for all items of the ASRM, GAD-7, and PHQ-9 (Figure 4) demonstrates no associations between the 9th item of the PHQ and any items of the ASRM or GAD-7. The model does, however, demonstrate an association between the 9th item of the PHQ and the 2nd and 6th PHQ items (‘feeling down, depressed or hopeless’ & ‘feeling bad about yourself’).
Figure 4: Network Analysis of ASRM, GAD-7, and PHQ-9 Items.

This figure illustrates the regularized partial correlation network of the individual items of the ASRM, GAD-7, and PHQ-9. The goal of this cross-sectional partial correlation analysis was to examine the interrelatedness of the ASRM and the GAD-7 and their relationship to the node of interest, the 9th item of the PHQ-9, adjusting for the remaining items of the PHQ. Positive partial correlations are represented by solid lines and negative partial correlations are represented by dashed lines.
ASRM=Altman Self-Report Mania Rating scale
GAD-7= 7-item Generalized Anxiety Disorder questionnaire
PHQ-9 = 9-item Patient Health Questionnaire
4. Discussion
This study sought to use a novel approach to evaluate the influence of manic and anxiety symptom clusters on risk of suicidal ideation or behavior in individuals with bipolar disorder during a depressive state. Rothman’s theoretical model of component cause is a conceptual framework for illustrating the variety of factors that may contribute to the occurrence of a particular outcome of interest. While Rothman’s causal pies traditionally have been used in public health research, it is reasonable to adopt this model for use in understanding the complex relationship between mood symptoms and suicidality because it both reveals direct associations between risk factors and the outcome of interest and accounts for the contribution of unmeasured potential contributors to the outcome.
In this paper, we utilize a quantitative adaptation of Rothman’s theoretical model of component cause proposed by Hoffman et al., which draws upon the mathematical relationship between the proportion of a disease due to a class of sufficient cause (PDC) and the population attributable fraction (PAF) to create a statistical formula for estimating the risk attributable to classes of component cause (Hoffmann et al., 2006).
In this cohort of individuals with bipolar disorder and at least moderate depressive symptoms, we found no increased risk of suicidal ideation or behavior attributable to clusters of mania/hypomania and anxiety symptoms, as represented by latent classes comprised of the individual items of the ASRM and the GAD-7, nor did network analysis suggest any relationship between individual ASRM or GAD-7 items and the suicidal ideation node. We found a very small proportion of the risk of suicidal ideation or behavior to be attributable to crossing the threshold for severe depressive symptoms, as captured by PHQ-8 score ≥20. Relatedly, network analysis demonstrated an association between suicidal ideation and the depressive symptoms ‘feeling down’ and ‘feeling bad about yourself’.
The largely negative results of this study are consistent with our prior analyses, which did not demonstrate any synergy between depressive and manic symptoms and clearly identified depressive symptoms as the primary driver of risk of suicidal ideation or behavior in bipolar disorder. That this analysis demonstrated a vast proportion of unmeasured risk highlights an important challenge inherent to studying suicidal ideation and behavior. Suicidal ideation and behavior is a complex and multifactorial outcome – beyond mood and anxiety symptoms, there are many biopsychosocial risk factors and contextual considerations that may be relevant to risk such as impulsivity, aggression, and maladaptive responses to stress, which were not captured by this current analysis (Turecki et al., 2019).
4.1. Strengths and Limitations
This study has several limitations that may influence the interpretability of our findings. In spite of our variable reduction effort through latent class analysis, inadequate sample size remains a relevant limitation. Hoffman et al. note that “the simultaneous consideration of a lot of risk factors and the differentiation among many classes of sufficient causes require a high number of cases. If the sample size and especially the number of cases are too low, stepwise regression will exclude many indicator variables, and the estimated PDC of some important classes of sufficient causes may not be significantly different from zero” and further note that their use of the EPIC-Potsdam Study cohort was of insufficient size to support their four risk factors of interest (Hoffmann et al., 2006). Although the EPIC-Potsdam study sample employed by Hoffman et al. was large (n=26,972), the prevalence of their outcome of interest, myocardial infarction, was very low (0.6%), leading to a sample that was of insufficient size to support the PDC models. Although our outcome prevalence was considerably higher at 49%, our sample size of 1,028 patient visits would have been inadequate to support the 4096 possible permutations of our original 12 risk factors of interest in this research question, resulting in low cell counts and the need to employ variable reduction as was done in our sample.
In addition, Flegal et al note that calculation of PAF (and thereby the PDC) involves several practical considerations: appropriate definition and characterization of the exposure, selection of the counterfactual, calculation of the relative risks, selection of computational model, and interpretation of results (Flegal et al., 2015).
Appropriate definition and characterization of the exposure:
Our inability to detect a contribution of any clusters of mania/hypomania and anxiety symptoms to risk of suicidal ideation or behavior in bipolar depression may suggest that there are other unmeasured factors driving the vast majority of risk. A major limitation of the ASRM as used for our purposes is that it may fail to accurately and comprehensively capture all symptoms of mania. The ASRM only captures symptoms related to the facets of elevated mood and increased energy, thereby neglecting other components such as irritability, impulsivity, or psychosis, although the GAD-7 does include an item measuring irritability. These symptoms are of particular interest given their more negative valence and potential connection to suicide risk. Several studies have demonstrated a link between irritability and increased suicide risk. In an observational study of 239 outpatients with bipolar I or schizoaffective disorder, suicidality was higher among individuals with a symptom profile that included irritability (Berk et al., 2017). Similarly, Eberhard et al. examined the association between anxiety, irritability, and agitation (AIA symptoms) and suicidality among 348 patients with bipolar I disorder during a manic episode with mixed features, and found that those with more severe AIA symptoms had more frequent suicidal ideation and suicidal behavior (Eberhard and Weiller, 2016). In a sample of individuals with a non-violent suicide attempt within the past 24 hours, Balazs et al. noted that more than 90% of individuals with mixed symptoms (57% of total study cohort) had a symptom profile characterized by irritability, distractibility, and psychomotor agitation (i.e., negatively valenced symptoms) (Balazs et al., 2006). Studies have also noted an association between impulsivity (Jimenez et al., 2016; Johnson et al., 2017; Reich et al., 2019) and psychotic symptoms (Belteczki et al., 2018) and increased suicide risk among individuals with bipolar disorder. In addition, because the ASRM is a self-report instrument, impaired insight may introduce the potential for inaccurate reporting of manic symptoms (Gazalle et al., 2007). Given these considerations, we would greatly benefit from a more complete symptom catalog, such as through the structured clinical interview. To this end, however, it is worth noting that our previous investigation of suicide risk in bipolar mixed state in the CDS cohort used clinician-administered mania assessment and yielded similar findings to those in replication studies using the NNDC CCR and MOP cohorts, suggesting that self-report instruments are adequate for capturing mania when clinician-administered assessment tools are not available (Fiedorowicz et al., 2020; Fiedorowicz et al., 2019; Persons et al., 2018).
Hoffman’s adaptation of the component cause model is also limited by requiring dichotomization of risk factors. Hoffman et al. note that problems may arise for continuous variables that are dichotomized for use in the model because the use of different cutoff values may influence the results (Hoffmann et al., 2006). This may in part explain the limited apparent role of severe depressive symptoms noted in our findings, as our models are quantifying the risk in crossing the threshold from moderate depressive symptoms (PHQ-8 10-19) to severe depressive symptoms (PHQ-8 ≥20).
Appropriate selection of the counterfactual:
Flegal et al. note that an important challenge in calculation and interpretation of the PAF is appropriate approximation of the counterfactual – that is, the unknowable alternate reality in which the exposure had not occurred – because the choice of comparison group can change the PAF estimate (Flegal et al., 2015). Our inability to identify any increased risk of suicidal ideation or behavior attributable to individual symptoms of mania or anxiety may suggest that our selection of comparison group (i.e., the low symptom burden group) is inappropriate. With the manifest risk factors typically used in the causal pie model, the comparison group naturally concerns participants with an absence of all risk factors. However, because the latent risk factors included in our model did not have a clear absence category, our comparison group did not show a complete absence of manic and anxiety symptoms. Of the seven identified symptom clusters, none showed the lowest symptom burden on all mania and anxiety items. Although we chose the comparison group with lowest average symptom burden, the mania scores in this Cluster 1 group were slightly higher than those in the Cluster 6 group (but the anxiety scores were considerably lower).
In estimation of the PAF, the comparison group is defined as a group that does not have the exposures that are being measured, so their entire causal pie is U, or unknown contributors to risk. Therefore, in the event that unknown contributors are the major drivers of risk, the low-exposure group may not actually be a low-risk group. There is potential for this issue because our lowest risk group in terms of outcomes is not our comparison group, but rather our Cluster 6 group (Low manic symptom burden, Low-Moderate anxiety symptoms with more prominent nervousness, worrying, trouble relaxing, and irritability).
Appropriate interpretation of results:
A related issue to the appropriate selection of a counterfactual group is the appropriate interpretation of results. Care should be taken to interpret these findings as individuals with bipolar disorder during depressive state that also includes X manic and/or anxiety symptom(s) are at no elevated risk of suicidal ideation or behavior relative to individuals with minimal symptoms (i.e., moderate depressive symptoms, low manic symptoms, low anxiety symptoms). In addition, using a composite exposure and a composite outcome can impede interpretability of our study findings. The benefit of composite exposures and composite outcomes is that they increase statistical power, which can however come at the cost of clarity (through use of latent categories) or precision (by data reduction). It should be noted here that a limitation of this study is the inability to study the composite outcome of suicidal ideation or behavior as its distinct components, as suicidal ideation is an imperfect surrogate for attempted suicide or suicide death (DeJong et al., 2010; Klonsky et al., 2016).
Appropriate calculation of relative risks:
It should be noted as a strength of this current study that the prevalence estimates and the relative risks are estimated from the same study. In calculating the PAF, the exposure prevalence can either be derived from the same study that will be used to estimate the relative risks, or the prevalence estimates can be derived from the target population and the relative risks can be calculated from the study sample, or ‘derivation cohort’ (Flegal et al., 2015). Potential problems that can arise in using different population samples to estimate the prevalence and the relative risk are that the definitions of exposure and outcome between samples may not be consistent, or that the study cohorts themselves may not be representative of the same population.
Appropriate selection of computational model:
Another strength of this study is careful selection of the computational model in consideration of the features of the study cohort and the use of adjusted relative risk. The formula used in this study is appropriate for use with adjusted relative risks, whereas other available models are intended for use only with unadjusted relative risks (Flegal et al., 2015).
4.2. Future Directions
Overall, future studies would greatly benefit from a sufficiently large sample size with a sufficient number of outcomes in each subset of the data showing a distinct combination of risk factors. It should be noted, however, that sufficiently large sample sizes may not be possible for studies with more than five risk factors of interest, given that for X risk factors there are 2X possible combinations of risk factors. For instances in which many risk factors are of interest, such as the individual items of a questionnaire, variable reduction may serve as a viable and likely necessary solution for lowering the number of possible combinations.
The inability of this study to detect a contribution of any clusters of mania/hypomania and anxiety symptoms to risk of suicidal ideation or behavior, as well as the large proportion of unmeasured risk of suicidal ideation and behavior, illustrates a need for future studies to consider the influence of other factors relevant to risk that were not captured by this current analysis. From a mood symptoms standpoint, future studies would also greatly benefit from a comprehensive, clinician-administered mood symptoms assessment designed to capture a complete inventory of manic symptoms, including those of negative valence and potentially linked to suicide risk, and circumvent issue related to impaired insight with mania.
In this current study, we employ a composite outcome of suicidal ideation or behavior and recognize this as an imperfect surrogate outcome for suicide. While it is important to extend our findings to suicide attempts and deaths, it would be an unusual dataset that would be able to apply these innovative methods to such an infrequent outcome, as exemplified by the EPIC-Potsdam study of myocardial infarction.
Nonetheless, the current analysis provides yet another layer of evidence of the limited impact on risk of suicidal ideation or behavior for manic symptoms in those already depressed and extends this finding to anxiety symptoms. This analysis also illustrates the role of the many risk factors that were not included in this analysis of such a complex and multidetermined outcome that is suicide.
Supplementary Material
Highlights.
There was no increased risk of suicidal ideation or behavior attributable to manic and anxiety symptom clusters.
A small amount of risk was attributable to severe depressive symptoms.
Future studies should include larger samples and more rigorous assessments.
Role of Funding Source
This work was funded by a National Network of Depression Centers 2019 Momentum Grant. The National Network of Depression Centers (NNDC) provided feedback on the study design and provided access to the data.
Declaration of Competing Interests
Jess Fiedorowicz received research support for a project with Myriad Genetics, Inc. Dr. Fiedorowicz was funded by the National Institute of Mental Health (R01MH111578) and Institute for Clinical and Translational Science at the University of Iowa (U54TR001356). John Nurnberger and William Coryell have received research support from Janssen Pharmaceuticals. The authors have no other potential conflicts of interest to disclose.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- Altman E, Hedeker D, Peterson JL, Davis JM, 2001. A comparative evaluation of three self-rating scales for acute mania. Biol Psychiatry 50 (6), 468–471. [DOI] [PubMed] [Google Scholar]
- Altman EG, Hedeker D, Peterson JL, Davis JM, 1997. The Altman Self-Rating Mania Scale. Biol Psychiatry 42 (10), 948–955. [DOI] [PubMed] [Google Scholar]
- Balazs J, Benazzi F, Rihmer Z, Rihmer A, Akiskal KK, Akiskal HS, 2006. The close link between suicide attempts and mixed (bipolar) depression: implications for suicide prevention. J Affect Disord 91 (2-3), 133–138. [DOI] [PubMed] [Google Scholar]
- Belteczki Z, Rihmer Z, Ujvari J, Lamis DA, Dome P, 2018. Differences in clinical characteristics between bipolar patients with current psychotic symptoms and those who have never been psychotic. Psychiatr Danub 30 (2), 183–188. [DOI] [PubMed] [Google Scholar]
- Berk L, Hallam KT, Venugopal K, Lewis AJ, Austin DW, Kulkarni J, Dodd S, de Castella A, Fitzgerald PB, Berk M, 2017. Impact of irritability: a 2-year observational study of outpatients with bipolar I or schizoaffective disorder. Bipolar Disord 19 (3), 184–197. [DOI] [PubMed] [Google Scholar]
- Cassidy F, 2010. Anxiety as a symptom of mixed mania: implications for DSM-5. Bipolar Disord 12 (4), 437–439. [DOI] [PubMed] [Google Scholar]
- DeJong TM, Overholser JC, Stockmeier CA, 2010. Apples to oranges?: a direct comparison between suicide attempters and suicide completers. J Affect Disord 124 (1-2), 90–97. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Eberhard J, Weiller E, 2016. Suicidality and symptoms of anxiety, irritability, and agitation in patients experiencing manic episodes with depressive symptoms: a naturalistic study. Neuropsychiatr Dis Treat 12, 2265–2271. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Epskamp S, Cramer AO, Waldorp LJ, 2012. qgraph: Network Visualizations of Relationships in Psychometric Data. Journal of Statistical Software 448 (4), 18. [Google Scholar]
- Fiedorowicz JG, Persons JE, Assari S, Ostacher MJ, Goes FS, Nurnberger JI, Coryell WH, 2020. Moderators of the association between depressive, manic, and mixed mood symptoms and suicidal ideation and behavior: An analysis of the National Network of Depression Centers Mood Outcomes Program. J Affect Disord. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fiedorowicz JG, Persons JE, Assari S, Ostacher MJ, Zandi P, Wang PW, Thase ME, Frye MA, Coryell W, of the National Network of Depression Centers Bipolar Disorders Interest, G., 2019. Depressive symptoms carry an increased risk for suicidal ideation and behavior in bipolar disorder without any additional contribution of mixed symptoms. J Affect Disord 246, 775–782. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Flegal KM, Panagiotou OA, Graubard BI, 2015. Estimating population attributable fractions to quantify the health burden of obesity. Ann Epidemiol 25 (3), 201–207. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gazalle FK, Frey BN, Hallal PC, Andreazza AC, Cunha AB, Santin A, Kapczinski F, 2007. Mismatch between self-reported quality of life and functional assessment in acute mania: a matter of unawareness of illness? J Affect Disord 103 (1-3), 247–252. [DOI] [PubMed] [Google Scholar]
- Hoffmann K, Heidemann C, Weikert C, Schulze MB, Boeing H, 2006. Estimating the proportion of disease due to classes of sufficient causes. Am J Epidemiol 163 (1), 76–83. [DOI] [PubMed] [Google Scholar]
- Jimenez E, Arias B, Mitjans M, Goikolea JM, Ruiz V, Brat M, Saiz PA, Garcia-Portilla MP, Buron P, Bobes J, Oquendo MA, Vieta E, Benabarre A, 2016. Clinical features, impulsivity, temperament and functioning and their role in suicidality in patients with bipolar disorder. Acta Psychiatr Scand 133 (4), 266–276. [DOI] [PubMed] [Google Scholar]
- Johnson SL, Carver CS, Tharp JA, 2017. Suicidality in Bipolar Disorder: The Role of Emotion-Triggered Impulsivity. Suicide Life Threat Behav 47 (2), 177–192. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Klonsky ED, May AM, Saffer BY, 2016. Suicide, Suicide Attempts, and Suicidal Ideation. Annu Rev Clin Psychol 12, 307–330. [DOI] [PubMed] [Google Scholar]
- Kroenke K, Spitzer RL, Williams JB, 2001. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 16 (9), 606–613. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kroenke K, Strine TW, Spitzer RL, Williams JB, Berry JT, Mokdad AH, 2009. The PHQ-8 as a measure of current depression in the general population. J Affect Disord 114 (1–3), 163–173. [DOI] [PubMed] [Google Scholar]
- Madsen AM, Hodge SE, Ottman R, 2011. Causal models for investigating complex disease: I. A primer. Hum Hered 72 (1), 54–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Miller S, Dennehy EB, Suppes T, 2016. The Prevalence and Diagnostic Validity of Short-Duration Hypomanic Episodes and Major Depressive Episodes. Curr Psychiatry Rep 18 (3), 27. [DOI] [PubMed] [Google Scholar]
- Parker GB, Graham RK, 2015. An evaluation of the DSM-5 rules defining mania and hypomania with identical symptom criteria. J Affect Disord 170, 91–94. [DOI] [PubMed] [Google Scholar]
- Persons JE, Coryell WH, Solomon DA, Keller MB, Endicott J, Fiedorowicz JG, 2018. Mixed state and suicide: Is the effect of mixed state on suicidal behavior more than the sum of its parts? Bipolar Disord 20 (1), 35–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, Currier GW, Melvin GA, Greenhill L, Shen S, Mann JJ, 2011. The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry 168 (12), 1266–1277. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Reich R, Gilbert A, Clari R, Burdick KE, Szeszko PR, 2019. A preliminary investigation of impulsivity, aggression and white matter in patients with bipolar disorder and a suicide attempt history. J Affect Disord 247, 88–96. [DOI] [PubMed] [Google Scholar]
- Rothman KJ, 1976. Causes. Am J Epidemiol 104 (6), 587–592. [DOI] [PubMed] [Google Scholar]
- Spiegelman D, Hertzmark E, Wand HC, 2007. Point and interval estimates of partial population attributable risks in cohort studies: examples and software. Cancer Causes Control 18 (5), 571–579. [DOI] [PubMed] [Google Scholar]
- Spitzer RL, Kroenke K, Williams JB, Lowe B, 2006. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 166 (10), 1092–1097. [DOI] [PubMed] [Google Scholar]
- Swann AC, Lafer B, Perugi G, Frye MA, Bauer M, Bahk WM, Scott J, Ha K, Suppes T, 2013. Bipolar mixed states: an international society for bipolar disorders task force report of symptom structure, course of illness, and diagnosis. Am J Psychiatry 170 (1), 31–42. [DOI] [PubMed] [Google Scholar]
- Turecki G, Brent DA, Gunnell D, O’Connor RC, Oquendo MA, Pirkis J, Stanley BH, 2019. Suicide and suicide risk. Nat Rev Dis Primers 5 (1), 74. [DOI] [PubMed] [Google Scholar]
- Vermunt JK, Magidson J, 2013. Latent GOLD 5.0 Upgrade Manual. Belmont, MA: Statistical Innovations Inc. [Google Scholar]
- Wensink M, Westendorp RG, Baudisch A, 2014. The causal pie model: an epidemiological method applied to evolutionary biology and ecology. Ecol Evol 4 (10), 1924–1930. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zou G, 2004. A modified poisson regression approach to prospective studies with binary data. Am J Epidemiol 159 (7), 702–706. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
