Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Jul 1.
Published in final edited form as: Am J Psychiatry. 2016 Feb 19;173(7):695–704. doi: 10.1176/appi.ajp.2015.15040414

Toward the definition of a bipolar prodrome: Dimensional predictors of bipolar spectrum disorder in at-risk youth

Danella M Hafeman 1, John Merranko 1, David Axelson 2, Benjamin I Goldstein 3, Tina Goldstein 1, Kelly Monk 1, Mary Beth Hickey 1, Dara Sakolsky 1, Rasim Diler 1, Satish Iyengar 4, David Brent 1, David Kupfer 1, Boris Birmaher 1
PMCID: PMC4930393  NIHMSID: NIHMS796658  PMID: 26892940

Abstract

Objective

We aimed to assess dimensional symptomatic predictors of new-onset bipolar spectrum disorder in youth at familial risk of bipolar disorder (“at-risk” youth).

Method

Offspring aged 6–18 of parents with bipolar-I/II disorder (n=391) and offspring of community controls (n=248) were recruited without regard to non-bipolar psychopathology. At baseline, 8.4% (33/391) of offspring of bipolar parents had bipolar spectrum; 14.7% (44/299) of offspring with follow-up developed new-onset bipolar spectrum (15 with bipolar-I/II) over eight years. Scales collected at baseline and follow-up were reduced using factor analyses; factors (both at baseline and visit proximal to conversion or last contact) were then assessed as predictors of new-onset bipolar spectrum.

Results

Relative to community control offspring, at-risk and bipolar offspring had higher baseline levels of anxiety/depression, inattention/disinhibition, externalizing, subsydromal manic, and affective lability symptoms (p<.05). The strongest predictors of new-onset bipolar spectrum were: baseline anxiety/depression, baseline and proximal affective lability, and proximal subsyndromal manic symptoms (p<.05). While affective lability and anxiety/depression were elevated throughout follow-up in those who later developed bipolar spectrum, manic symptoms increased up to the point of conversion. A path analysis supported the hypothesized model that affective lability at baseline predicted new-onset bipolar spectrum, in part, through increased manic symptoms at the visit prior to conversion; earlier parental age of mood disorder onset also significantly increased risk of conversion (p<.001). While youth without anxiety/depression, affective lability, and mania (and with a parent with older age of mood disorder onset) had a 2% predicted chance of conversion to bipolar spectrum, those with all risk factors had a 49% predicted chance of conversion.

Conclusions

Dimensional measures of anxiety/depression, affective lability, and mania are important predictors of new-onset bipolar spectrum in this population of at-risk youth. These symptoms emerged from among numerous other candidates, underscoring the potential clinical and research utility of these findings.


The average individual with bipolar disorder experiences impairing mood symptoms for about 10 years before obtaining an accurate diagnosis (13). While retrospective studies of adults with bipolar disorder indicate symptom onset during childhood or adolescence, few were diagnosed before 18 years old (4,5). Diagnostic delays have detrimental consequences including inappropriate treatments, increased hospitalization, and increased suicide risk (6). Thus it is crucial to better characterize the prodromal symptoms preceding bipolar disorder onset.

Multiple lines of evidence indicate the presence of significant psychopathology preceding bipolar onset. Based on retrospective studies of both adults and children, sleep disturbances, anxiety, depressive symptoms, affective lability, subthreshold hypomanic symptoms, behavioral dyscontrol, and irritability have been reported to precede bipolar disorder (3,79). Many of these characteristics have also been identified in youth at genetic risk for bipolar disorder (1018).

While the above findings indicate the presence of prodromal symptoms, non-specificity limits their clinical and research utility. To identify a prodrome that might predict bipolar disorder, parallel to the concept of ultra-high risk population in the schizophrenia literature (19), prospective studies are imperative. To date, extant prospective studies have focused primarily on categorical predictors of bipolar disorder, including both subsyndromal and syndromal diagnoses. The most important result to emerge from such studies is that subthreshold hypomanic episodes are an important predictor of bipolar spectrum in depressed adults (20), depressed adolescents (21,22), and offspring of bipolar parents (23). Major depressive episodes (23,24) and disruptive behavioral disorders (23) also predict bipolar spectrum onset in genetically at-risk youth. Anxiety disorders precede onset of mood disorder in at-risk youth (25,26), and are hypothesized to represent an early stage in the development of bipolar disorder (27).

One way to improve the characterization of prodromal symptoms is to move from a diagnostic perspective to a dimensional framework, assessing symptomatology on a continuum. The current study prospectively assesses the predictive value of several dimensional measures administered at baseline and follow-up. To our knowledge, only one study has assessed the prospective impact of dimensional measures, and interpretation was limited by small number of converters (n=9) (28).

The Pittsburgh Bipolar Offspring Study (BIOS) recently assessed categorical predictors of bipolar disorder and showed that disruptive disorders, major depressive episodes, and in particular subthreshold manic episodes were associated with developing bipolar disorder in at-risk offspring (23). Instead of focusing on mood episodes and categorical disorders, we use the same sample to assess whether dimensions are predictive of new-onset bipolar spectrum in at-risk offspring. This analysis first focuses on the impact of dimensional scales at baseline, to answer the following important clinical question: which aspects of clinical presentation from a single encounter predict new-onset bipolar spectrum? Next, we assess which dimensions are proximal predictors of new-onset bipolar spectrum, and we examine the trajectory of each significant factor prior to conversion (or last contact). Finally, we combine these predictors into a path analysis, to test a model for how significant independent predictors, both at baseline and proximal visit, lead to bipolar onset. We hypothesized that symptoms at baseline would impact the risk of bipolar spectrum, in part, through more proximal symptoms.

Methods

The methods of BIOS have been described in detail in prior reports (23,29). All procedures were approved by the University of Pittsburgh Institutional Review Board prior to the start of the study.

Sample

Parents with bipolar-I/II were recruited via advertisement, research studies, and outpatient clinics. Exclusion criteria were a lifetime diagnosis of schizophrenia, mental retardation, or a mood disorder secondary to medical illness, substance or medication use. Control parents were recruited from the community without regard to non-bipolar psychopathology, group-matched by age, sex, and neighborhood. In addition to the above exclusion criteria, control parents could not have a first-degree relative with bipolar disorder. The study included all offspring aged 6 to 18 years, unless the child had mental retardation. We used the entire sample for the factor analysis and baseline comparisons. For analyses predicting new-onset bipolar spectrum, we only used offspring of bipolar parents without bipolar spectrum at baseline (at-risk offspring).

Procedures

Informed consent from the parents and assent from the children were obtained. Parents and participating biological co-parents (31%) were assessed by direct interview using the Structured Clinical Interview for DSM-IV. The psychiatric history of non-participating biological co-parents was obtained from the participant parent using the Family-History Research Diagnostic Criteria (30).

At baseline and during follow-up visits, parents and their offspring were interviewed using the Schedule for Affective Disorders and Schizophrenia for School Age Children-Present and Lifetime Version (K-SADS P/L) for non-mood disorders and the K-SADS Mania Rating Scale and the depression items from the KSADS-Present Version present versions, which assess symptoms during the worst week over the past month (31,32). Assessments were performed by interviewers trained with the diagnostic instruments, and were reviewed by a child psychiatrist; all were blind to parental diagnoses. Summary scores were obtained using clinical consensus, integrating parent and offspring interviews. Parents and offspring completed several rating scales covering a range of psychopathology including, among others, the Child Affective Lability Scale (33) and Child Behavioral Checklist (34) (Table 1, eMethods). Socioeconomic status was determined using the Hollingshead scale (35).

Table 1.

Results of factor analyses conducted separately for Parent-Report, Child-Report, and Depression Rating Scale

Factor Items
Parent-Report Internalizing Mood and Feelings Questionnaire, Screen for Child Anxiety Related Disorders (all subscales), Child Behavioral Checklist (anxious/depressed, withdrawn, somatic complaints, social problems, thought problems)
Externalizing Child Affective Lability Scale (irritability subscale), Child Affective Dysregulation Scale (all subscales), Disruptive Behavioral Disorders Rating Scale (opposition/defiance, crime), Children’s Hostility Inventory (all subscales), Child Behavioral Checklist (rule-breaking behavior, aggressive behavior)
Inattention/Disinhibition Child Affective Lability Scale (mania subscale), Disruptive Behavioral Disorders Rating Scale (inattention, hyperactivity), Child Behavioral Checklist (attention problems)
Child-Report Internalizing Mood and Feelings Questionnaire, Screen for Child Anxiety Related Disorders (all subscales)
Externalizing Children’s Hostility Inventory (all subscales)
Affective Lability Child Affective Lability Scale (all subscales)
Depression Rating Scale Depressive/Atypical Symptoms Depressed mood, irritability/anger, reactivity, diurnal mood variation, guilt, negative self-image, hopelessness, aches and pains, anhedonia, fatigue, poor concentration, psychomotor retardation, social withdrawal, daytime sleepiness, hypersomnia, anorexia, increased appetite, craving for sweets, weight gain, leaden paralysis, rejection sensitivity
Sleep Problems Initial/middle/terminal insomnia, circadian reversal, non-restorative sleep
Suicidality Suicidal ideation, number/seriousness/lethality of suicide attempts, recurrent thoughts of death

Follow-up evaluations were performed every two years to assess for onset of DSM-IV disorders. Kappa coefficients for all disorders were ≥0.70. Date of bipolar onset was set to be the first time the participant met criteria for bipolar disorder, not-otherwise-specified or DSM-IV criteria for a manic, mixed or hypomanic episode. As detailed elsewhere (and described in eMethods), operationalized criteria were used for bipolar disorder, not-otherwise-specified (36). Youth with this diagnosis have comparable family history of bipolar disorder, suicidality, risk for substance abuse, and psychosocial impairment to those with Bipolar-I/II (29,3638), and have roughly 50% chance of progressing to Bipolar-I/II within five years (23,39).

Statistical Analyses

Baseline scales were reduced using maximum-likelihood factor analyses in SAS 9.4. The Kaiser rule, scree test, and Horn’s Parallel Analysis were used to choose optimal factor solutions. Several rotations were attempted, with the goal of optimizing separation of factors and minimizing items that did not load onto any factors. While all analyses yielded similar factor structures, the final solution included four factor analyses [Parent-Report, Child-Report, Depression Rating Scale, and Mania Rating Scale] conducted on the entire population utilizing an oblimin rotation. For the Depression and Mania Rating Scales, individual items were entered into the factor analyses; for the Parent- and Child-Report factor analyses, we used either full scale scores or, if available, subscale scores based on previous factor analyses (Table 1, eMethods).

To mitigate the impact of missing data, we imputed results using Multivariate Imputation by Chained Equations. Offspring who did not have data for an entire factor analysis (n=46) were excluded. Factor structure did not change with imputation. Extremely rare items (<10 positive responses) were excluded from the factor analysis. If an item loaded on more than one factor (weight >.3), clinical interpretation was used to determine the appropriate factor. The Parent-Report, Child-Report, and Depression Rating Scale factor analyses yielded three factors; the Mania Rating Scale factor analysis did not yield a statistically or conceptually meaningful separation, and so was analyzed as a single factor (Table 1, eTables 1–3). Factor scores were derived by multiplying each standardized item score by the corresponding factor loading, and then summing the products under each factor.

Differences in demographic and clinical characteristics at baseline among bipolar parents with at least one offspring with bipolar spectrum, bipolar parents without bipolar offspring, and control parents were assessed using standard statistical methods. Characteristics of offspring were compared using mixed effects regression models, controlling for within-family correlation. Mixed models were also used to evaluate differences in factors across these three offspring groups. Demographic covariates that differed between the three groups (p<.2) were entered in the analysis; covariates that remained predictors in the multivariate model (p<.2) were retained. Three control offspring had bipolar spectrum at baseline; these youth were excluded from all analyses. Analyses were conducted both with and without adjustment for non-bipolar psychopathology of both biological parents that met the above threshold criteria.

Cox regression was used to determine which factors at baseline were individually predictive of new-onset bipolar spectrum within at-risk offspring, after adjusting for covariates that met the above retention criteria. This method models events according to duration of follow-up, thus indicating the impact of each factor on time to disorder onset. Analyses were adjusted for demographics, parental non-bipolar categorical diagnoses, and offspring non-bipolar categorical diagnoses (listed in eMethods). To assess whether factors were similarly predictive of bipolar-I/II, we conducted a sensitivity analysis, removing individuals who had bipolar not-otherwise-specified at the time of right censorship (i.e. last visit). To determine which factors explained a significant amount of unique variance, a penalized regression model (Lasso; Least Absolute Shrinkage and Selection Operator; see eMethods) including all individually significant predictors of bipolar spectrum was used. We also assessed for interactions between child non-bipolar categorical diagnoses and factors to predict new-onset bipolar spectrum. Relevant regression models were used to determine which scales/subscales within each of the predictive factors were driving the observed relationship. All results were adjusted for within-family correlations, using frailty models.

We next assessed factor scores at the visit preceding either bipolar onset or right censorship, using logistic regression to evaluate proximal predictors of bipolar conversion. Similar to intake models, we first assessed whether each factor was individually predictive, and then used Lasso regression to determine which factors were independently predictive. All analyses were adjusted for multiple comparisons, covariates that met the above statistical threshold (demographics, parental diagnoses, and child diagnoses), and within-family correlation. To assess whether group differences persisted across time, we graphed trajectories of independently predictive factor scores up to the point prior to bipolar conversion (or right censorship). Finally, we used a path analysis to test the pathways by which significant baseline and proximal predictors predicted bipolar spectrum onset, entering variables that were significant predictors in intake and/or proximal models. Of note, 25 participants only had one visit prior to either bipolar spectrum conversion or right censorship; this visit was used for both the intake and proximal models, but these individuals only contributed to the proximal time point in the path analysis.

Results

Sample Characteristics

Parents

Compared to controls, both parent groups with bipolar disorder were less likely to be married at baseline (p<.05) and had higher rates of all DSM-IV disorders (p <.0001). Compared to controls, bipolar parents with bipolar offspring were younger (p=.006) and had lower socioeconomic status (p=.04). Bipolar parents with vs. without bipolar offspring did not significantly differ according to demographics or co-morbidity (eTable 4). Co-parent depression differed across group (p=.02), and was highest in co-parents of bipolar parents without bipolar offspring (eTable 5).

Offspring

Offspring characteristics have been described previously (23). Briefly, at baseline the mean age for all offspring, including bipolar (n=33), at-risk (n=326), and control offspring (n=220), was 11.7±3.5 years at baseline (19.6±4.5 years at last assessment), 48% of offspring were male, and average Hollingshead Socioeconomic Status score was 35.1±13.6 (middle class). Over 95% of the offspring (n=553) had data from at least one follow-up visit, with an average of 3.6±1.2 follow-ups (median=4, range 1–6) over a mean duration of 8.3±2.4 years. Loss to follow-up did not differ between offspring groups. Compared to at-risk and control offspring, bipolar offspring were born to younger mothers (p<.005) and less likely to live with both biological parents (p<.05). Except for substance use disorders, which did not significantly differ across groups, non-bipolar psychopathology was most prevalent in bipolar offspring followed by at-risk and then control offspring (most p-values<.01) (eTable 6). In addition to the 33 offspring with bipolar spectrum at baseline, 44 at-risk offspring developed new-onset bipolar spectrum during follow-up; mean age of conversion was 14.8±4.0 years old.

Factors across Baseline-Defined Groups

Compared with at-risk and control offspring, bipolar offspring had higher scores on all factors at baseline (most p-values<.0001) (Figure 1, eTable 7). At-risk offspring had significantly higher scores than control offspring on all factors except for Sleep Problems (from the Depression Rating Scale); most associations remained significant after adjustment for parental non-bipolar psychopathology (eTable 8).

Figure 1.

Figure 1

Baseline Differences in each factor across groups, adjusting for demographics. All two-group comparisons are significant (p<.05) except for Sleep scores between at-risk and control offspring.

Baseline predictors of new-onset bipolar spectrum in at-risk offspring

Of the 299 at-risk offspring with follow-up data available, 44 developed new-onset bipolar spectrum (15 with bipolar-I/II) over a mean of approximately eight years. Conversion to bipolar spectrum (regardless of sub-type) was associated with an increase in manic and depressive symptoms, and decreased global functioning (eTable 9; eFigure 1). Though youth with bipolar disorder not-otherwise-specified by definition had never meet full criteria for a (hypo)manic episode, 84% had met subthreshold manic criteria for at least 30 lifetime days, and 72% reported at least two days meeting criteria during a single week.

Internalizing symptoms, externalizing symptoms, and affective lability significantly predicted new-onset bipolar spectrum, even after taking into account categorical diagnoses (Table 2). These symptoms were similarly predictive of bipolar-I/II (excluding youth with bipolar disorder not-otherwise-specified) (Table 2). When all baseline factors were entered together into a penalized (Lasso) Cox regression, Parent-Reported Internalizing [X2=6.75,p=.009] and Child-Reported Affective Lability [X2=4.00,p=.046] significantly predicted new-onset bipolar spectrum (Table 3). History of a depressive disorder [X2=7.04,p=.008], co-parent with bipolar disorder [X2=5.49,p=.02], and earlier parental age of mood disorder onset [X2=8.42,p=.004] were also predictive. There were no significant interactions between factors and baseline history of categorical disorders.

Table 2.

Impact of individual baseline factors on the hazard of developing bipolar spectrum (and bipolar-I/II) over follow-up, adjusting for demographics, parental non-bipolar categorical disorders, and offspring non-bipolar categorical disorders. Hazard ratios associated with a one standard-deviation increase in each factor are given.

Bipolar Spectrum (n=44) Bipolar I/II (n=15)

X2 Hazard Ratio 95% CI p X2 Hazard Ratio 95% CI P

 Parent
 Reports
Internalizinga 18.70 1.78 1.37, 2.31 <0.0001b 11.85 2.07 1.37, 3.12 0.0006b
Externalizing 4.11 1.41 1.01, 1.95 0.04 3.91 1.69 1.00, 2.84 0.05
Inattention/Disinhibition 2.36 1.16 0.96, 1.41 0.1 1.26 1.16 0.90, 1.51 0.3

 Child
 Reports
Internalizing 8.81 1.54 1.16, 2.06 0.003b 3.15 1.60 0.95, 2.69 0.08
Externalizing 8.54 1.78 1.21, 2.63 0.004b 4.62 2.07 1.07, 4.01 0.03
Affective Labilitya 13.58 1.66 1.27, 2.18 0.0002b 13.97 2.24 1.47, 3.42 0.0002b

 Depression
 Rating Scale
Depressive/Atypical Symptoms 1.95 1.24 0.92, 1.67 0.2 0.13 1.09 0.68, 1.77 0.7
Sleep Problems 0.03 0.98 0.75, 1.28 0.9 0.02 0.97 0.64, 1.48 0.9
Suicidality 5.70 1.23 1.04, 1.47 0.02b 1.86 1.22 0.92, 1.62 0.2

Mania Rating Scale Total Score 2.03 1.21 0.93, 1.58 0.2 0.53 1.19 0.75, 1.90 0.5
a

Significant in the penalized (Lasso) Cox Regression Model

b

Significant after correcting for multiple comparisons (False Discovery Rate)

Table 3.

Impact of individual proximal factors on the hazard of developing bipolar spectrum (and bipolar-I/II) over follow-up, adjusting for demographics, parental non-bipolar categorical disorders, and offspring non-bipolar categorical disorders. Odds ratios associated with a one standard-deviation increase in each factor are given.

Bipolar Spectrum (n=44) Bipolar I/II (n=15)

X2 Odds Ratio 95% CI p X2 Odds Ratio 95% CI p

 Parent
 Reports
Internalizing 11.15 1.76 1.26, 2.45 0.0008b 3.18 1.57 0.96, 2.57 0.07
Externalizing 13.80 1.91 1.36, 2.68 0.0002b 2.72 1.55 0.92, 2.60 0.1
Inattention/Disinhibition 8.81 1.67 1.19, 2.35 0.003b 2.65 1.55 0.91, 2.62 0.1

 Child
 Reports
Internalizing 6.54 1.65 1.12, 2.41 0.01b 2.90 1.76 0.92, 3.35 0.09
Externalizing 7.52 1.76 1.18, 2.65 0.006b 0.002 1.02 0.53, 1.97 0.96
Affective Labilitya 12.78 1.82 1.31, 2.52 0.0004b 7.80 2.09 1.25, 3.51 0.005 b

 Depression
 Rating Scale
Depressive/Atypical Symptoms 2.78 1.29 0.96, 1.73 0.1 0.01 0.97 0.56, 1.68 0.9
Sleep Problems 5.06 1.40 1.04, 1.88 0.02b 1.10 1.28 0.81, 2.02 0.3
Suicidality 3.57 1.30 0.99, 1.71 0.06 3.38 1.37 0.98, 1.91 0.07

Mania Rating Scale Total Scorea 21.73 2.14 1.55, 2.94 <0.0001b 4.01 1.48 1.01, 2.17 0.05
a

Significant in the penalized (Lasso) Logistic Regression Model

b

Significant after correcting for multiple comparisons (False Discovery Rate)

The scale that best accounted for the observed relationship between Parent-Reported Internalizing and bipolar onset were the internalizing subscales of the Child Behavioral Checklist (eTable 10). Child-Reported Affective Lability was derived from the three Child Affective Lability Scale subscales (irritability, mania, and anxiety/depression); irritability was the best independent predictor, but anxiety/depression was also a highly significant individual predictor (eTable 11).

Proximal Predictors of new-onset bipolar spectrum in at-risk offspring

Proximal predictors of bipolar spectrum onset were similar to those observed at baseline, with an important exception. Manic symptomatology, which did not significantly predict new-onset bipolar spectrum at baseline, was a strong proximal predictor of conversion (Table 3). A similar pattern was seen when individuals with a final diagnosis of bipolar disorder not-otherwise-specified were excluded (Table 2). When all factors were entered into a penalized (Lasso) logistic regression, elevated Child-Reported Affective Lability [X2=3.85,p=.0498] and Mania Rating Scale [X2=13.49,p=.0002] emerged as significant predictors of conversion at next visit (vs. right censorship), even after taking into account categorical diagnoses. Lifetime diagnosis of an anxiety disorder [X2=5.35,p=.021] and earlier parental age of mood disorder onset [X2=5.96,p=.014] were also significant predictors. Because of the way we selected the “proximal visit”, youth who converted to BD were on average younger at this proximal visit than those who were right censored; thus age at proximal visit was retained as a nuisance covariate. There were no significant interactions between factors and lifetime history of categorical disorders.

Of the three affective lability subscales, irritability was the most important proximal predictor of new-onset bipolar spectrum (eTable 12). Most items in the mania rating scale were significant proximal predictors of bipolar onset. The most significant independent predictors within this scale (using Lasso regression) were irritability, hyperactivity, and distractibility; elation was not significant in the combined model, but was a highly significant individual predictor (p<.0001) of new-onset bipolar spectrum (eTable 13).

Factor Trajectories Prior to Conversion (or Right Censorship)

Trajectories for significant independent baseline predictors (Parent-Reported Internalizing and Child-Reported Affective Lability) indicated that group differences were robust throughout follow-up (Figure 2a and 2b). In contrast, Mania Rating Scale increased across time in youth who would go on to develop new-onset bipolar spectrum (Figure 2c), consistent with manic symptoms as a proximal predictor of conversion. Trajectories for all other factors can be found in the eSupplement (eFigure 2).

Figure 2.

Figure 2

Trajectories for dimensions graphed up to final eligible visit (either the visit prior to conversion or the visit prior to right censorship). a. Parent-Reported Internalizing, b. Child-reported affective lability, c. manic rating scale.

Path Analysis

Based on previous work indicating that mood lability predicts the development of subthreshold manic symptoms (40) and hypomania (41), we hypothesized that baseline Child-Reported Affective Lability would contribute to increased Manic Rating Scale at the proximal visit. The model was consistent with our hypothesis: all paths tested were highly significant (most p<.001) and the root mean square error of approximation was 0.000, indicating excellent fit. Of the covariates independently predictive in the above models (parental age of mood disorder onset, co-parent with bipolar disorder, history of depressive disorder at baseline, and lifetime anxiety disorder), only earlier parental age of mood disorder onset had a significant direct effect on the outcome, when taking into account dimensional measures (Figure 3). In the path model, history of depressive disorder at baseline and lifetime anxiety disorder were not significant predictors of new-onset bipolar spectrum. Adjustment for age at proximal visit did not appreciably alter results, and is included as a nuisance covariate.

Figure 3.

Figure 3

Path analysis showing significant predictors of new-onset bipolar spectrum from baseline and proximal visits. Baseline history of depressive disorders, lifetime anxiety disorders, and co-parent with bipolar disorder were significant in individual models, but did not directly predict bipolar spectrum onset in the final probit model. Model is adjusted for age at proximal visit, a nuisance covariate in this analysis. Correlations and beta coefficients between variables are included. ***p<.001, **p<.01

To assess clinical significance of results, we used the underlying probit regression model to calculate predicted risk of new-onset bipolar spectrum according to independent predictors of the outcome. We found that a participant with low levels of affective lability, anxiety/depression, and manic symptoms (one S.D. below the mean), whose parent had older age of mood disorder onset (one S.D. above the mean), had only a 2% predicted chance of conversion over the course of follow-up. In contrast, a participant with high levels of these symptoms (all one S.D. above the mean), whose parent had younger age of onset (one S.D. below the mean), had a 49% predicted chance of conversion (a 24-fold increase in risk) over follow-up (Figure 4). Running the model using parental age of mood onset as a dichotomous variable showed that defining early parental onset at age 18 yielded similar results.

Figure 4.

Figure 4

Predicted probability of new-onset bipolar spectrum for risk profiles defined by significant predictors in the overall probit model. Predicted risk differed substantially between someone with no symptoms (baseline anxiety/depression, proximal affective lability, and proximal manic symptoms all 1 S.D. below the mean) vs. someone with all of these symptoms (1 S.D. above the mean). Predicted risk also differed according to parental age of mood disorder onset, looking in particular at an individual 1 S.D. below the mean (parent proband developed mood disorder at 11 years old) vs. 1 S.D. above the mean (parent proband developed mood disorder at 29 years old). Results are adjusted for age at proximal visit, a nuisance covariate in this analysis.

Discussion

In this sample of at-risk offspring, the most important prospective dimensional predictors of new-onset bipolar spectrum were anxiety/depressive symptoms (baseline), affective lability (baseline and proximal), and subthreshold manic symptoms (proximal). Consistent with previous work (42,43), we also found an increased risk of new-onset bipolar spectrum with earlier parental age of mood disorder onset (e.g., ≤ 18 y.o). The predicted risk for an individual with all of these risk factors was over 24-fold higher than the predicted risk for an individual with none of these risk factors. These predictors were significant above and beyond categorical disorders, and in fact, the disorders were no longer significant predictors of bipolar spectrum onset after taking into account dimensions. Interactions between dimensions and disorders were also not significant, meaning that the effect of dimensions did not differ according to diagnostic category. Trajectory and path analyses indicated that anxiety/depression and affective lability were initial predictors of new-onset bipolar spectrum, and remained consistently elevated in those who would go on to convert. In contrast, manic symptoms increased up to the visit prior to conversion; affective lability at baseline predicted new-onset bipolar spectrum, in part, through the increase in manic symptoms at the proximal visit.

While affective lability emerged as an important predictor of new-onset bipolar spectrum in this analysis, this symptom might not be regularly assessed by clinicians. In this study, and in previous work (17), we used the Child Affective Lability Scale (parent and child report) to assess this domain, which factors into three symptom categories: depression/anxiety, irritability, and subthreshold mania. Thus, this freely available self-report may be used to screen offspring of parents with bipolar who are at risk to develop this disorder.

Although child and parent reports were found to be important for different domains, the current study does not provide evidence that informant is relevant, but rather is likely an artifact of collected scales. Regarding internalizing symptoms, we did not have a child equivalent of the Child Behavior Checklist, which was driving the association of parent-report internalizing with new-onset bipolar disorder. While both parents and children completed the Child Affective Lability Scale, the parent-report factored into separate domains, while the child-report factored together; thus there was no parent-reported Affective Lability factor, per se. Thus, we draw our conclusions about the domains rather than the informants.

These findings build on a recent analysis from the BIOS study, which identified subsyndromal manic episodes as an important categorical predictors of bipolar disorder (23). We add to this work by finding that subsyndromal manic symptoms (even in the absence of a mood episode) predict bipolar spectrum onset in at-risk youth. Our results are also consistent with findings from retrospective and at-risk studies that point to a wide-ranging set of prodromal symptoms, in particular, anxiety/depression (26,27), affective lability (17,44,45), and subthreshold manic symptoms (45,46) [reviewed in (9)]. We find that almost all dimensions are elevated in youth at-risk for disorder (as compared to community control offspring). However, we add to these previous findings by assessing the degree to which each dimension prospectively and independently predicts bipolar onset, even after adjustment for parental and offspring non-bipolar disorders. Using longitudinal data, we also begin to define both an “initial” prodrome for bipolar disorder (which can occur up to seven years prior to disorder (47)) and a “proximal” prodrome (two years prior to onset). From a single clinical encounter, anxiety/depression and affective lability are the best predictors of future new-onset bipolar spectrum (“initial” prodrome). Progressively increased subsyndromal manic symptoms (along with affective lability) emerge as the most important predictors of conversion within the next two years (“proximal” prodrome). Of note, over half of youth with these symptoms did not develop bipolar spectrum within the follow-up period; thus the presence of this prodrome does not imply that they will necessarily develop disorder, but rather identifies the youth who are at highest risk of conversion.

This study has several strengths on which we have capitalized in this analysis. First, the sample size and length of follow-up have led to adequate numbers of youth developing de novo bipolar spectrum to prospectively assess predictors of onset, differentiating them from consequences or correlates of disorder. Second, we collected data on a large number of both self-report and clinician-administered scales at baseline, allowing for a comprehensive assessment of mood, anxiety, and behavioral dimensions that could be potentially predictive of disorder. Thus, we did not constrain our analyses based on theory, but rather used a data-driven approach to identify independent predictors of disorder. Third, data were available regarding parental and offspring demographic and clinical characteristics. Adjustment for such variables established that observed associations were related to bipolar spectrum, and not confounded by these factors.

This study also has limitations, which should be kept in mind when interpreting results. First, our results focused on the predictors of bipolar onset within at-risk offspring; thus we do not know if results would generalize to a population without such a familial risk. Second, visits were scheduled every two years, so the “proximal” time point was often 1–2 years prior to bipolar spectrum conversion. Because of this, our analyses might have missed prodromal symptoms appearing within only months of disorder onset. Third, while we had adequate numbers of new-onset bipolar spectrum to assess predictors, we had relatively few youth with bipolar-I/II. However, we had enough power to conduct a sensitivity analysis, which revealed consistent findings to the primary model, thus mitigating this concern to some extent. Power to test interactions between dimensions and categorical disorders was also limited, rendering this analysis exploratory. Fourth, our average age at baseline was under twelve years old, and many of the at-risk offspring might yet develop bipolar disorder (particularly those with major depression), since some are only entering the high-risk period for developing the disorder. Thus our findings might apply preferentially to cases with earlier onset, as opposed to those who develop bipolar spectrum during adulthood. Young age at baseline might also explain discrepancies between our sample and other at-risk cohorts, such as the fact that substance abuse did not differ across baseline groups [see (23) for a full discussion]. Fifth, our “Sleep” variable consisted only of items from the KSADS Depression Rating Scale, and thus did not rigorously characterize sleep. Circadian dysfunction, when measured more directly, might predict new-onset bipolar disorder.

Despite limitations, these findings have important implications. We find that a diverse array of dimensional psychopathology is associated with family history of bipolar disorder. However, a smaller subset of symptoms predict bipolar onset, above and beyond the presence of categorical diagnoses. From a single assessment, anxiety/depression and affective lability should raise clinical suspicion that at-risk youth will develop bipolar spectrum in the future, particularly in those whose parent(s) developed a mood disorder at an earlier age. As these youth are followed in time, the persistence of affective lability and emergence of manic symptoms markedly increase the conversion to bipolar spectrum within the next few years. Clinically, this more specific set of prodromal symptoms might identify youth who would benefit particularly from early pharmacological and/or psychosocial interventions and increased surveillance. From a research perspective, the definition of an “ultra-high risk” population might facilitate the identification of biomarkers and the evaluation of early interventions.

Supplementary Material

supplements

Acknowledgments

DISCLOSURES: Dr. Axelson has served as a consultant for Janssen Research and received royalties from UpToDate. Dr. Tina Goldstein has received grant funding from NIMH, NIDA/NIAAA, the National Institute of Child Health and Human Development, the American Foundation for Suicide Prevention, and the Pittsburgh Foundation and receives royalties from Guilford Press. Dr. Sakolsky has received research support from NIMH and NARSAD, has received an honorarium from the American Academy of Child and Adolescent Psychiatry for CME work, and serves as an editorial board member of Child and Adolescent Psychopharmacology News and as a specialty consultant for Prescriber’s Letter. Dr. Brent has received grants from NIMH, fees from CME events, royalties from Guilford Press and ERT, Inc., and fees from UpToDate. Dr. Kupfer has served as a consultant to the American Psychiatric Association (as Chair of the DSM-5 Task Force), has joint ownership of copyright for the Pittsburgh Sleep Quality Index, is a member of the Valdoxan Advisory Board of Servier International, and is a stockholder in AliphCom, Healthrhythms, Inc., and Psychiatric Assessments, Inc. Dr. Birmaher receives royalties from Random House, Lippincott Williams & Wilkins, and UpToDate.

Footnotes

The other authors report no financial relationships with commercial interests.

References

  • 1.Berk M, Dodd S, Callaly P, Berk L, Fitzgerald P, de Castella AR, et al. History of illness prior to a diagnosis of bipolar disorder or schizoaffective disorder. J Affect Disord. 2007;103(1–3):181–6. doi: 10.1016/j.jad.2007.01.027. [DOI] [PubMed] [Google Scholar]
  • 2.Hirschfeld RM, Lewis L, Vornik LA. Perceptions and impact of bipolar disorder: how far have we really come? Results of the national depressive and manic-depressive association 2000 survey of individuals with bipolar disorder. J Clin Psychiatry. 2003;64(2):161–74. [PubMed] [Google Scholar]
  • 3.Egeland JA, Hostetter AM, Pauls DL, Sussex JN. Prodromal symptoms before onset of manic-depressive disorder suggested by first hospital admission histories. J Am Acad Child Adolesc Psychiatry. 2000;39(10):1245–52. doi: 10.1097/00004583-200010000-00011. [DOI] [PubMed] [Google Scholar]
  • 4.Beesdo K, Hofler M, Leibenluft E, Lieb R, Bauer M, Pfennig A. Mood episodes and mood disorders: patterns of incidence and conversion in the first three decades of life. Bipolar Disord. 2009;11(6):637–49. doi: 10.1111/j.1399-5618.2009.00738.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Kupfer DJ, Frank E, Grochocinski VJ, Cluss PA, Houck PR, Stapf DA. Demographic and clinical characteristics of individuals in a bipolar disorder case registry. J Clin Psychiatry. 2002;63(2):120–5. doi: 10.4088/jcp.v63n0206. [DOI] [PubMed] [Google Scholar]
  • 6.Keck PE, Jr, Kessler RC, Ross R. Clinical and economic effects of unrecognized or inadequately treated bipolar disorder. J Psychiatr Pract. 2008;14(Suppl 2):31–8. doi: 10.1097/01.pra.0000320124.91799.2a. [DOI] [PubMed] [Google Scholar]
  • 7.Lish JD, Dime-Meenan S, Whybrow PC, Price RA, Hirschfeld RM. The National Depressive and Manic-depressive Association (DMDA) survey of bipolar members. J Affect Disord. 1994;31(4):281–94. doi: 10.1016/0165-0327(94)90104-x. [DOI] [PubMed] [Google Scholar]
  • 8.Fergus EL, Miller RB, Luckenbaugh DA, Leverich GS, Findling RL, Speer AM, et al. Is there progression from irritability/dyscontrol to major depressive and manic symptoms? A retrospective community survey of parents of bipolar children. J Affect Disord. 2003;77(1):71–8. doi: 10.1016/s0165-0327(02)00176-3. [DOI] [PubMed] [Google Scholar]
  • 9.Hauser M, Correll CU. The significance of at-risk or prodromal symptoms for bipolar I disorder in children and adolescents. Can J Psychiatry. 2013;58(1):22–31. doi: 10.1177/070674371305800106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Sparks GM, Axelson DA, Yu H, Ha W, Ballester J, Diler RS, et al. Disruptive mood dysregulation disorder and chronic irritability in youth at familial risk for bipolar disorder. J Am Acad Child Adolesc Psychiatry. 2014;53(4):408–16. doi: 10.1016/j.jaac.2013.12.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Egeland JA, Shaw JA, Endicott J, Pauls DL, Allen CR, Hostetter AM, et al. Prospective study of prodromal features for bipolarity in well Amish children. J Am Acad Child Adolesc Psychiatry. 2003;42(7):786–96. doi: 10.1097/01.CHI.0000046878.27264.12. [DOI] [PubMed] [Google Scholar]
  • 12.Duffy A, Alda M, Crawford L, Milin R, Grof P. The early manifestations of bipolar disorder: a longitudinal prospective study of the offspring of bipolar parents. Bipolar Disord. 2007;9(8):828–38. doi: 10.1111/j.1399-5618.2007.00421.x. [DOI] [PubMed] [Google Scholar]
  • 13.Giles LL, DelBello MP, Stanford KE, Strakowski SM. Child behavior checklist profiles of children and adolescents with and at high risk for developing bipolar disorder. Child Psychiatry Hum Dev. 2007;38(1):47–55. doi: 10.1007/s10578-006-0041-6. [DOI] [PubMed] [Google Scholar]
  • 14.Wals M, Hillegers MH, Reichart CG, Ormel J, Nolen WA, Verhulst FC. Prevalence of psychopathology in children of a bipolar parent. J Am Acad Child Adolesc Psychiatry. 2001;40(9):1094–102. doi: 10.1097/00004583-200109000-00019. [DOI] [PubMed] [Google Scholar]
  • 15.Hirshfeld-Becker DR, Biederman J, Henin A, Faraone SV, Cayton GA, Rosenbaum JF. Laboratory-observed behavioral disinhibition in the young offspring of parents with bipolar disorder: a high-risk pilot study. Am J Psychiatry. 2006;163(2):265–71. doi: 10.1176/appi.ajp.163.2.265. [DOI] [PubMed] [Google Scholar]
  • 16.Diler RS, Birmaher B, Axelson D, Obreja M, Monk K, Hickey MB, et al. Dimensional psychopathology in offspring of parents with bipolar disorder. Bipolar Disord. 2011;13(7–8):670–8. doi: 10.1111/j.1399-5618.2011.00966.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Birmaher B, Goldstein BI, Axelson DA, Monk K, Hickey MB, Fan J, et al. Mood lability among offspring of parents with bipolar disorder and community controls. Bipolar Disord. 2013;15(3):253–63. doi: 10.1111/bdi.12060. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Lombardo LE, Bearden CE, Barrett J, Brumbaugh MS, Pittman B, Frangou S, et al. Trait impulsivity as an endophenotype for bipolar I disorder. Bipolar Disord. 2012;14(5):565–70. doi: 10.1111/j.1399-5618.2012.01035.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Cannon TD, Cadenhead K, Cornblatt B, Woods SW, Addington J, Walker E, et al. Prediction of psychosis in youth at high clinical risk: a multisite longitudinal study in North America. Arch Gen Psychiatry. 2008;65(1):28–37. doi: 10.1001/archgenpsychiatry.2007.3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Fiedorowicz JG, Endicott J, Leon AC, Solomon DA, Keller MB, Coryell WH. Subthreshold hypomanic symptoms in progression from unipolar major depression to bipolar disorder. Am J Psychiatry. 2011;168(1):40–8. doi: 10.1176/appi.ajp.2010.10030328. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Uchida M, Serra G, Zayas L, Kenworthy T, Hughes B, Koster A, et al. Can manic switches be predicted in pediatric major depression? A systematic literature review. J Affect Disord. 2014;172C:300–6. doi: 10.1016/j.jad.2014.09.046. [DOI] [PubMed] [Google Scholar]
  • 22.Biederman J, Wozniak J, Tarko L, Serra G, Hernandez M, McDermott K, et al. Re-examining the risk for switch from unipolar to bipolar major depressive disorder in youth with ADHD: a long term prospective longitudinal controlled study. J Affect Disord. 2014:152–154. 347–51. doi: 10.1016/j.jad.2013.09.036. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Axelson D, Goldstein B, Goldstein T, Monk K, Yu H, Hickey MB, et al. Diagnostic Precursors to Bipolar Disorder in Offspring of Parents With Bipolar Disorder: A Longitudinal Study. Am J Psychiatry. 2015;172(7):638–46. doi: 10.1176/appi.ajp.2014.14010035. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Mesman E, Nolen WA, Reichart CG, Wals M, Hillegers MH. The Dutch bipolar offspring study: 12-year follow-up. Am J Psychiatry. 2013;170(5):542–9. doi: 10.1176/appi.ajp.2012.12030401. [DOI] [PubMed] [Google Scholar]
  • 25.Duffy A, Alda M, Hajek T, Sherry SB, Grof P. Early stages in the development of bipolar disorder. J Affect Disord. 2010;121(1–2):127–35. doi: 10.1016/j.jad.2009.05.022. [DOI] [PubMed] [Google Scholar]
  • 26.Nurnberger JI, Jr, McInnis M, Reich W, Kastelic E, Wilcox HC, Glowinski A, et al. A high-risk study of bipolar disorder. Childhood clinical phenotypes as precursors of major mood disorders. Arch Gen Psychiatry. 2011;68(10):1012–20. doi: 10.1001/archgenpsychiatry.2011.126. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Duffy A, Horrocks J, Doucette S, Keown-Stoneman C, McCloskey S, Grof P. The developmental trajectory of bipolar disorder. Br J Psychiatry. 2014;204(2):122–8. doi: 10.1192/bjp.bp.113.126706. [DOI] [PubMed] [Google Scholar]
  • 28.Egeland JA, Endicott J, Hostetter AM, Allen CR, Pauls DL, Shaw JA. A 16-year prospective study of prodromal features prior to BPI onset in well Amish children. J Affect Disord. 2012;142(1–3):186–92. doi: 10.1016/j.jad.2012.04.023. [DOI] [PubMed] [Google Scholar]
  • 29.Goldstein BI, Shamseddeen W, Axelson DA, Kalas C, Monk K, Brent DA, et al. Clinical, demographic, and familial correlates of bipolar spectrum disorders among offspring of parents with bipolar disorder. J Am Acad Child Adolesc Psychiatry. 2010;49(4):388–96. [PMC free article] [PubMed] [Google Scholar]
  • 30.Andreasen NC, Endicott J, Spitzer RL, Winokur G. The family history method using diagnostic criteria. Reliability and validity. Arch Gen Psychiatry. 1977;34(10):1229–35. doi: 10.1001/archpsyc.1977.01770220111013. [DOI] [PubMed] [Google Scholar]
  • 31.Axelson D, Birmaher BJ, Brent D, Wassick S, Hoover C, Bridge J, et al. A preliminary study of the Kiddie Schedule for Affective Disorders and Schizophrenia for School-Age Children mania rating scale for children and adolescents. J Child Adolesc Psychopharmacol. 2003;13(4):463–70. doi: 10.1089/104454603322724850. [DOI] [PubMed] [Google Scholar]
  • 32.Kaufman J, Birmaher B, Brent D, Rao U, Flynn C, Moreci P, et al. Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime Version (K-SADS-PL): initial reliability and validity data. J Am Acad Child Adolesc Psychiatry. 1997;36(7):980–8. doi: 10.1097/00004583-199707000-00021. [DOI] [PubMed] [Google Scholar]
  • 33.Gerson AC, Gerring JP, Freund L, Joshi PT, Capozzoli J, Brady K, et al. The Children’s Affective Lability Scale: a psychometric evaluation of reliability. Psychiatry Res. 1996;65(3):189–98. doi: 10.1016/s0165-1781(96)02851-x. [DOI] [PubMed] [Google Scholar]
  • 34.Achenbach TM. Manual for the Child Behavior Checklist/4-18 and 1991 profile. Burlington, VT: University of Vermont Department of Psychiatry; 1991. [Google Scholar]
  • 35.Hollingshead AB. Four-Factor Index of Social Status. New Haven, Connecticut: Yale University Department of Sociology; 1975. [Google Scholar]
  • 36.Axelson D, Birmaher B, Strober M, Gill MK, Valeri S, Chiappetta L, et al. Phenomenology of children and adolescents with bipolar spectrum disorders. Arch Gen Psychiatry. 2006;63(10):1139–48. doi: 10.1001/archpsyc.63.10.1139. [DOI] [PubMed] [Google Scholar]
  • 37.Hafeman D, Axelson D, Demeter C, Findling RL, Fristad MA, Kowatch RA, et al. Phenomenology of bipolar disorder not otherwise specified in youth: a comparison of clinical characteristics across the spectrum of manic symptoms. Bipolar Disord. 2013;15(3):240–52. doi: 10.1111/bdi.12054. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Goldstein TR, Obreja M, Shamseddeen W, Iyengar S, Axelson DA, Goldstein BI, et al. Risk for suicidal ideation among the offspring of bipolar parents: results from the Bipolar Offspring Study (BIOS) Arch Suicide Res. 2011;15(3):207–22. doi: 10.1080/13811118.2011.589699. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Axelson DA, Birmaher B, Strober MA, Goldstein BI, Ha W, Gill MK, et al. Course of subthreshold bipolar disorder in youth: diagnostic progression from bipolar disorder not otherwise specified. J Am Acad Child Adolesc Psychiatry. 2011;50(10):1001–16 e3. doi: 10.1016/j.jaac.2011.07.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Zeschel E, Bingmann T, Bechdolf A, Kruger-Oezguerdal S, Correll CU, Leopold K, et al. Temperament and prodromal symptoms prior to first manic/hypomanic episodes: results from a pilot study. J Affect Disord. 2015;173:39–44. doi: 10.1016/j.jad.2014.10.031. [DOI] [PubMed] [Google Scholar]
  • 41.Akiskal HS, Maser JD, Zeller PJ, et al. Switching from ‘Unipolar’ to Bipolar II: An 11-year prospective study of clinical and temperamental predictors in 559 patients. Archives of General Psychiatry. 1995;52(2):114–23. doi: 10.1001/archpsyc.1995.03950140032004. [DOI] [PubMed] [Google Scholar]
  • 42.Rice J, Reich T, Andreasen NC, et al. THe familial transmission of bipolar illness. Archives of General Psychiatry. 1987;44(5):441–7. doi: 10.1001/archpsyc.1987.01800170063009. [DOI] [PubMed] [Google Scholar]
  • 43.Chang KD, Steiner H, Ketter TA. Psychiatric phenomenology of child and adolescent bipolar offspring. J Am Acad Child Adolesc Psychiatry. 2000;39(4):453–60. doi: 10.1097/00004583-200004000-00014. [DOI] [PubMed] [Google Scholar]
  • 44.Doucette S, Horrocks J, Grof P, Keown-Stoneman C, Duffy A. Attachment and temperament profiles among the offspring of a parent with bipolar disorder. J Affect Disord. 2013;150(2):522–6. doi: 10.1016/j.jad.2013.01.023. [DOI] [PubMed] [Google Scholar]
  • 45.Findling RL, Youngstrom EA, McNamara NK, Stansbrey RJ, Demeter CA, Bedoya D, et al. Early symptoms of mania and the role of parental risk. Bipolar Disord. 2005;7(6):623–34. doi: 10.1111/j.1399-5618.2005.00260.x. [DOI] [PubMed] [Google Scholar]
  • 46.Bechdolf A, Ratheesh A, Cotton SM, Nelson B, Chanen AM, Betts J, et al. The predictive validity of bipolar at-risk (prodromal) criteria in help-seeking adolescents and young adults: a prospective study. Bipolar Disord. 2014;16(5):493–504. doi: 10.1111/bdi.12205. [DOI] [PubMed] [Google Scholar]
  • 47.Skjelstad DV, Malt UF, Holte A. Symptoms and signs of the initial prodrome of bipolar disorder: A systematic review. Journal of Affective Disorders. 2010;126(1–2):1–13. doi: 10.1016/j.jad.2009.10.003. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

supplements

RESOURCES