High-Risk Phenotypes of Early Psychiatric Readmission in Bipolar Disorder with Comorbid Medical Illness

Abstract

Objective

This study aims to identify modifiable predictors of psychiatric readmission among persons with comorbid bipolar disorder and medical illness. This goal was accomplished by using electronic health records and applying objective variable selection via machine learning techniques.

Method

This is a retrospective analysis of electronic health record data derived from 77,296 episodes of care from 2006–2016 within the University of California Health Care System. Data included 1,250 episodes of care involving patients with bipolar disorder and serious comorbid medical illness (defined by transfer between medicine and psychiatry services, or concomitant primary medical and psychiatric admission diagnoses). Machine learning (classification trees) was used to identify potential predictors of 30-day psychiatric readmission across 1,250 hospital encounters. Predictors included demographics, medical and psychiatric diagnoses, medication regimen, and disposition. The algorithm was internally validated using 10-fold cross-validation.

Results

The model predicted 30-day readmission with high accuracy (98% unbalanced model, 88% balanced model). Modifiable predictors of readmission were: length of stay, transfers between medical and psychiatric services, discharge disposition to home, and all-cause acute health service utilization in the year prior to the index hospitalization.

Conclusion

Among bipolar disorder patients with comorbid medical conditions, characteristics of the index hospitalization (e.g., duration, transfer, and disposition) emerge as more predictive than static properties of the patient (e.g., sociodemographic factors and psychiatric comorbidity burden). Findings identify phenotypes of patients at high risk for rehospitalization and suggest potential ways of modifying risk of early readmission.

INTRODUCTION

Psychiatric hospital readmission is a pivotal metric of quality of care (1,2). Rehospitalization is common in serious mental illness, leading to increased healthcare costs and adversity for the individual (3). Medical comorbidities are an important consideration for psychiatric readmission because they add additional functional disruption to mental illness (4). Constructing a clinically useful set of predictors of psychiatric readmission for multimorbid populations is complicated by vulnerabilities accrued from coexisting medical conditions, including the influence of polypharmacy (5), access to providers (6), and pain (7).

Predictive factors of psychiatric readmission of patients with serious mental illness and co-occurring severe medical illness are complex and not yet well understood (8). Over the last two decades, hospitalization for mental disorders has increased at a faster rate than hospitalization for any other cause (9). This shift underscores the need to identify modifiable predictors psychiatric readmission, particularly for multimorbid populations, which are both costly and vulnerable.

Bipolar disorder is a serious mental illness for which the risk of readmission and rate of comorbid medical illness are strikingly high (10). The morbidity and mortality associated with bipolar disorder reflects both the psychiatric illness and the burden of comorbid medical conditions. Individuals with bipolar disorder have nearly twice the risk of cardiovascular mortality compared with the general population (11,12). All-cause mortality in bipolar disorder is increased two-fold relative to healthy individuals, corresponding to a decreased life expectancy of nearly nine years (13–16). Bipolar disorder increases the risk of early medical readmission and mortality from diabetes (4), heart failure (17), acute myocardial infarction (18), pulmonary disease (9), and other medical conditions. Contributors to mortality risk associated with bipolar disorder appear to be partly modifiable, e.g., patients with more timely medical diagnosis have a mortality risk approaching that of the general population (11). From a public health perspective, the cost of bipolar disorder is further increased by enormous medical care costs (17).

Models of hospital readmission do not fully capture the integration of psychiatric and general medical care and are not yet adapted to multimorbid populations. Psychiatric conditions increase the risk of medical readmission (4,18,21) and medical comorbidity may in turn predict psychiatric readmission (8). When individuals with co-existing serious mental and medical illness are admitted to the hospital, it is often unclear whether the individual should be admitted to the medical or psychiatric inpatient service. These individuals may be admitted to psychiatry too frequently, or not frequently enough. Research on important modifiable variables, such as length of stay, has yielded mixed results (5,22–26) and may not be generalizable to multimorbid populations. To our knowledge, no model of readmission exists which is specifically adapted for individuals with co-existing serious mental and medical illness. Moreover, predictors unique to this population remain unstudied, such as transfer of care between inpatient psychiatric and medical teams.

Here, we address a complex network of predictors of psychiatric readmission in patients with bipolar disorder and comorbid medical illness. Because of previous heterogeneous findings with predictors of admission, we use a machine learning algorithm capable of addressing complex data structures and nonlinear relations among variables. Machine learning is the automated and incremental optimization of a statistical model that is not explicitly programmed (27). Machine learning approaches bring advantages over traditional statistical methods, including the capacity to generate predictive models from high-dimensional data and automate feature (i.e., variable) selection, and are widely used to generate predictive models in health informatics. Machine learning mitigates bias generated from hand-selecting subsets of potential predictors, as is typically necessary in traditional regression models (27).

This study develops a model for psychiatric readmission in a population with cooccurring serious mental and medical illness using a large multi-site electronic health record (EHR) dataset. We model the risk of readmission in this population by applying a generalizable, and clinically-interpretable, machine learning algorithm. Specifically, we adopt learning-based decision trees, thus enabling stratification of specific subgroups of patients with comorbid bipolar disorder and medical illness at risk for early readmission. The algorithm is internally validated using 10-fold cross-validation, and clinical interpretability is maximized by comparing low false-negative to low false-positive models. Thus, we employ a novel approach to an understudied and vulnerable population of medically and psychiatrically ill individuals.

METHOD

Design

We performed a population-based cohort study to derive and validate a risk prediction model of 30-day psychiatric readmission among persons admitted for co-occurring bipolar disorder and serious medical illness. EHR data were extracted from two academic medical centers within the University of California, Los Angeles (UCLA) Healthcare System, in collaboration with the UCLA Clinical and Translational Science Institute. The UCLA Institutional Review Board approved this study.

Subjects

We identified individuals (≥18 years-old) who had an emergency department (ED) encounter from February 1, 2006 through April 1, 2016. Data were extracted for patients diagnosed with bipolar disorder (ICD-9 296.00–296.06, 296.40–296.89, 301.13; ICD-10 F31.0–31.9) and at least one index hospitalization during the study period. The index hospitalization was defined as (1) an all-cause hospitalization with a primary medical (non-psychiatric diagnosis) and a primary psychiatric diagnosis at time of admission, (2) a medical hospitalization with a primary medical diagnosis that resulted in a transfer to psychiatry, or (3) a psychiatric hospitalization with a primary psychiatric diagnosis that resulted in a transfer to medical service. The first day of the hospitalization was defined as the admission date to the ED. Descriptive characteristics of the sample are presented in Table 1.

Table 1.

Descriptive Characteristics of the Sample

Total	N 552	% 100
Sex
Female	311	56.3
Male	241	46.3
Race/Ethnicity
White, Non-Hispanic	424	76.8
Black, Non-Hispanic	70	12.7
Hispanic	60	10.9
Asian	21	3.8
Native Hawaiian / Pacific Islander	1	0.2
Other	21	3.8
Age
18–39	164	29.7
40–64	231	41.8
65+	157	28.4
Primary diagnosis
Bipolar I disorder	296	53.6
Bipolar II disorder	76	13.7
Unspecified Bipolar disorder /Bipolar disorder not otherwise specified	180	32.6
Comorbid medical conditions1
Cardiovascular disease	361	65.4
Fluid or electrolyte disorder	230	41.7
Anemia	171	31.0
Neurological disorder	136	24.6
Chronic Obstructive Pulmonary Disease	134
		24.3
Elixhauser comorbidity index2
≥10	181	32.8
6–9	104	18.8
≤5	151	27.3
Substance use disorder
Drug abuse	141	28.6
Alcohol abuse	84	15.2
Nicotine use	102	18.4
Transfer from medicine to psychiatry	201	36.4

¹Rates of common comorbid conditions are presented in further detail in Supplementary Figure 1.

²Elixhauser comorbidity index is condensed to a single numeric score that summarizes disease burden using the Van Walraven system (29). This score discriminates risk of in-hospital mortality. Scores ≤ 5 correspond to an in-hospital mortality risk of approximately 0-5%, scores 6–9 a mortality risk of approximately 5–10%, and scores ≥ 10 a mortality risk of greater than 10%. This score was unable to be calculated for N=116 due to missing data.

Outcome Measures

Readmission was defined as the occurrence of any of the following within 30 days of discharge from the index hospitalization: (1) any psychiatric readmission within the study sites (including admission to medicine service and subsequent transfer to psychiatry during the same admission); (2) presentation to the ED and subsequent transfer to an outside psychiatric hospital. Planned readmissions were excluded. All-cause unplanned readmissions occurring within 30 days of discharge from the index (initial) admission was selected as the primary outcome in accordance with Center for Medicaid and Medicare Services measures.

Predictor variables

Predictor categories are presented in Table 2 and the complete list of predictors is available in Supplemental Table 1. Time-varying predictors were re-coded to represent density of occurrence within 365 days from the index encounter (e.g. number of encounters with a given chief complaint). The number of hospitalizations, ambulatory visits, ED visits, and procedures, modal pain score and financial charges were extracted for each 365-day interval prior to the index encounter. Medications were categorized using WHODrug ATC classification system (28).

Table 2.

Extracted Electronic Health Record Data

Encrypted patient identifier

Encrypted encounter identifier

Age at first admission

Age at encounter

Sex

Race

Ethnicity

International Classification of Diseases (ICD) 9 and 10 diagnoses

Diagnosis date

Primary diagnosis associated with admission

Type of encounter (ambulatory, emergency department, inpatient)

Disposition (home with self-care, home with home health, psychiatric admission, acute care hospital, hospice, skilled nursing facility, eloped, against medical advice, inpatient rehab facility, longterm care)

In-hospital mortality

Vital signs (heart rate, blood pressure, temperature, respiration rate, oxygen saturation, pain score)

Procedure and procedure code

Financial charges associated with the encounter

Service date of charges

Medication regimen at each encounter

Chief complaint

Date of admission

Date of discharge

Psychiatric and medical comorbidities were identified by ICD-9 and −10 codes. Elixhauser Comorbidity Index (29) was used as a measure of medical illness burden based on ICD codes. Elixhauser Comorbidity Index was condensed to a single numeric score that summarizes disease burden using the Van Walraven system (29). This score discriminates risk of in-hospital mortality. Scores ≤ 5 correspond to an in-hospital mortality risk of approximately 0–5%, scores 6–9 a mortality risk of approximately 5–10%, and scores ≥10 a mortality risk of greater than 10%.

Psychiatric conditions were grouped into the following categories: anxiety disorders (300.00–300.02, 300.21–300.24, 300.3, 309.21, 309.24, 309.81; F41.x), personality disorders (301.0–301.9; F60.0–60.9), dementia (290.xx, 780.93; F01– 03.x, G30.x), depressive disorders (296.20–296.36, 300.4, 309.0, 309.28, 311; F32–33.x), schizophrenia (295.xx; F20.x), other psychoses (297.1, 297.3, 298.8–298.9, 301.20–301.22; F21–29.x), and substance use disorders (e.g., 291.xx, 292.xx, 303,xx, 305.xx; F10–19.x). Length of stay was defined as the number of days from admission to discharge (across contiguous hospitalizations).

Classification and Analyses

Classification and regression trees (CART) generate hierarchical rule sequences to classify data (30–32). CART sequentially partitions data based on an outcome of interest. The first branching point is the value of the variable that best partitions the data into two subgroups discriminated by outcome. For example, if the outcome is likelihood of readmission (1=likely, 0=not likely), then age (the variable) > 65 (the value) may best partition data: patients > 65 are more likely to be readmitted than patients <= 65. Each subgroup itself is then partitioned, and so on. CART has several advantages over traditional statistical models of binary outcomes, such as logistic regression. First, CART is able to process large numbers of predictive variables. As we anticipated over 300 predictive variables, a traditional regression framework would not be feasible. Second, CART is robust to missing and partially missing data via the generation of surrogates that replicate the performance of primary variables when data is not available. Third, compared with other machine learning methods, CART is a highly transparent process generating output (a decision-tree) that is readily interpretable by clinicians, thus avoiding the “black box” limitation of many machine learning models.

Models identifying rare outcomes often differ depending on the intended use case of the model, i.e., whether the model is optimized for high specificity (“diagnostic test”) or high sensitivity (“early warning test”). As we anticipated psychiatric readmission to be a relatively rare outcome, we generated two models: one optimizing specificity and one optimizing sensitivity. In CART, this is represented by generation of unbalanced (each class is weighted equally) and balanced trees (each class’s weight is balanced by the inverse proportion of that class’s frequency within a node).

Tree development was guided by constraining the tree size and shape. To optimize for clinical interpretability, we employed a semi-automated approach to variable selection, using clinical expertise and joint variable selection. We systematically categorized variables using predefined groups that matched the clinical approach to identifying critical parameters. Where predefined groups were not available, consensus coding was performed. The model was run with a maximum tree depth of 5 and a minimum leaf size of 5. We imputed missing predictor values with corresponding medians. Python 3.7.0, SciPy machine learning package version 0.19.1 was used for computational and statistical analysis (33).

Validation

Once the classification algorithm was established, it was applied to the entire dataset. The algorithm assigned a probability of 30-day readmission to each episode of care using 10-fold cross-validation. 10-fold cross-validation is a resampling procedure for estimating the quality of the machine learning model. This approach involves randomly dividing the set of observations into 10 groups of equal size. The first group is treated as a validation set, and the method is fit on the remaining nine groups (34). To validate the performance of the algorithm, we compared the following fit statistics: area under the receiver operating characteristic curve (AUC; a measure of the trade-off in diagnostic ability as the discrimination threshold is varied), accuracy (number of correct predictions divided by the total number of predictions), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

RESULTS

Data were available from 206,129 encounters, which were combined to single episodes of care (N=77,296) if multiple encounters occurred during the same date, or contiguous dates. 1,250 hospital episodes of care contained information from patients with bipolar disorder, a comorbid medical illness, and at least one qualifying index hospitalization (as defined above). The selection flowchart is presented in Supplemental Figure 1. Two readmissions were planned and these were excluded from the analyses.

Sample characteristics

The sample consisted of 1,250 episodes of care (552 patients). A large number of individuals had multiple medical comorbidities, and in many cases had prior medical hospitalizations, all of which indicated a high disease burden. Common comorbid medical conditions are presented in Supplemental Figure 2. Mean hospitalizations per individual was 2.3. Median length of stay was 5 days (SD 16.4). Across all hospitalizations, median days to psychiatric readmission was 243 (range 1–3,164); median days to all-cause readmission was 84 (range 1–3,380). There were two planned readmissions that were excluded from the sample.

Complete data on readmission were available for 323 episodes of care (i.e. data from 927 encounters were partially missing). Encounters with incomplete data were included on the basis of the robust capacity of CART to handle missing and partially missing data via imputation, i.e. creation of surrogates that replicate the performance of primary variables when data is not available (30). The psychiatric readmission rate was 3.0% for 7-day readmission, 8.6% for 30-day readmission, and 23.5% for 365-day readmission. The majority of encounters were not followed by a readmission, thus both balanced and unbalanced were compared to ascertain trade-offs in sensitivity and specificity for each use case.

Model Performance

Unbalanced tree.

The unbalanced tree was generated to optimize for specificity (“diagnostic test” for readmission). In the unbalanced tree, each class was weighted equally, disregarding the frequency with which each class occurred. The decision tree is presented in Figure 1. Patients were at the highest risk of 30-day readmission if they had a length of stay of six or more days, three or more episodes of care in the previous year, male gender, and high comorbidity burden (greater than 9 Elixhauser comorbidity category diagnoses). The unbalanced tree predicted no readmission for most patients, reflective of low posterior probability of readmission within 30 days. Fit statistics are presented in Table 3 and ROC curve in Supplemental Figure 3. Accuracy (98), AUC (88), NPV (98.2), and specificity (99.7) were high. PPV was 66.7. Sensitivity was 21.4. Overall, the unbalanced model had high specificity, detecting readmissions with a low false positive rate.

Figure 1.

Unbalanced Decision Tree. Shaded node represents patients with >50% likelihood of 30-day readmission, classified from 1st to 5th node: Length of stay of 6+ days, 3+ episodes of care in previous year, male, and 9+ Elixhauser Comorbidity diagnoses. Maximum tree depth of 5 and a minimum leaf size of 5. Dashed border indicates classification as low likelihood of readmission. Bold border indicates classification as high likelihood of readmission.

Table 3.

Model Performance. Comparison of fit statistics for unbalanced and balanced decision tree models.

MODEL	TP	FN	TN	FP	SENS	SPEC	PPV	NPV	ACC.	AUC
UNBALANCED	6	22	1219	3	21.4	99.7	66.7	98.2	0.98	0.88
BALANCED	28	0	1073	149	100	87.8	15.8	100	0.88	0.87

TP = True Positives; FN = False Negatives; TN = True Negatives; FP = False Positives; Sens = Sensitivity; Spec = Specificity; PPV = Positive Predictive Value; NPV = Negative Predictive Value; Acc = Accuracy; AUC = Area Under the Curve.

Balanced tree.

The second model was run with tree balancing, in which each class was weighted by the inverse proportion of that class’ frequency. The balanced tree was generated to optimize for sensitivity (“early warning” for readmission). The decision tree is presented in Figure 2. There were three circumstances that led to likely readmission within 30 days: (1) patients with a length of stay greater than five days, a primary medical diagnosis at index encounter, discharge disposition other than to skilled nursing facility, and high comorbidity burden (3 comorbid conditions); (2) patients with a length of stay less than or equal to five days, without transfer between medical and psychiatric services, who were Caucasian, not Hispanic or Latino, and had at least 13 ambulatory or emergency department visits in the previous year; (3) patients with transfer between medical and psychiatric services, a length of stay 1–2 days, and male gender. Fit statistics are presented in Table 3 and ROC curve in Supplemental Figure 4. Accuracy (88), AUC (87), NPV (100), specificity (87.8), and sensitivity (100) were high. PPV was 15.8. Of note, as the model was optimized for sensitivity, the terminal leaf nodes of the decision tree that did not fully differentiate tended to make a Type II error (i.e. falsely predict that a patient will be readmitted).

Figure 2.

Balanced Decision Tree. Shaded node represents patients with >50% likelihood of 30-day readmission. E.g. One classification from 1st to 5th node: Length of stay of 6+ days, primary medical diagnosis, not discharged to skilled nursing facility (SNF), and 3+ Elixhauser Comorbidity diagnoses. Maximum tree depth of 5 and a minimum leaf size of 5. Dashed border indicates classification as low likelihood of readmission. Bold border indicates classification as high likelihood of readmission.

DISCUSSION

This study empirically derived and internally validated a risk prediction algorithm to stratify likelihood of early psychiatric readmission among individuals with bipolar disorder and comorbid medical illness. We utilized CART to perform semi-automated feature selection and reduce hundreds of variables to a clinically interpretable subset, suggesting that modifiable predictors at index hospitalization may determine risk of early psychiatric readmission.

Our results suggest that the relations between individual predictors and readmission are frequently nonlinear and modulated by multiple competing factors, which reflects the extant heterogeneity in the current literature on readmissions (35,36). Specific modifiable predictors (i.e., transfer between psychiatric and medical services, discharge disposition, and length of stay) had greater predictive power than some static properties of the patient (e.g., sociodemographic factors and prior hospitalizations). To our knowledge, this is the first study to develop a model for psychiatric rehospitalization among individuals with co-occurring serious mental and medical illness, and is one of the first to use machine learning to predict psychiatric readmissions.

Medical comorbidity was a risk factor for future psychiatric admission. Risk indices to predict psychiatric readmission have only recently incorporated medical comorbidity to enhance discriminative capacity (e.g., Vigod et al’s READMIT index, (8)). Our analysis reinforces the conceptualization of medical illness as both a precipitant and indicator of poor psychiatric health. The low psychiatric readmission rate observed in this study, compared with estimates of 10–25% in the work (37–39), may reflect that patients with significant medical comorbidity tend to be readmitted to non-behavioral medical services rather than psychiatry. This pattern may arise because multimorbid patients are at risk of simultaneous psychiatric and medical decompensation. When destabilization occurs, due to difficulty in managing acute medical conditions and specialist services on an inpatient psychiatric unit, these individuals may be more likely to be admitted to medical services, rather than psychiatry. Although rates of psychiatric readmission within 30 days were not prevalent in the sample, this is not reflective of a low burden of morbidity.

Notably, in our sample of patients diagnosed with bipolar disorder and medical illness, approximately two-thirds of the sample had a diagnosis of cardiovascular disease. It is unclear whether the high prevalence of cardiovascular disease is an epiphenomenon of bipolar disorder or reflects a past history of treatment with medications that have metabolic side effects. Additional work is needed to determine whether cardiovascular disease is a modifiable predictor of psychiatric readmission, that is, whether prevention or treatment of cardiovascular disease may result in reduced psychiatric hospitalizations.

Studies have suggested that both long (25), and short (23,40) hospital stays are associated with risk of early readmission, and others find no significant association (5,22,26,41,42). Our findings suggest that length of stay is conditionally informative, with utility only in the context of knowledge of other factors, including comorbidity, diagnosis, sociodemographics, and transfer status. In the balanced model, a length of stay greater than five days was a risk factor for early rehospitalization. However, a length of stay of five or less days increased risk of readmission among patients who had frequent care utilization in the previous year, particularly if the index stay was very short (1–2 days). This finding may reflect the need for more intensive psychosocial management or even psychotropic stabilization for certain groups of patients who have repeated very brief psychiatric hospitalizations.

Transfer of care between medical and psychiatric services is an important predictor of readmission. Patients with bipolar disorder and a short length of stay (1–2 days) are more likely to be readmitted if transferred between services during index hospitalization. For those with longer length of stay (> 5 days), transfer of care did not emerge as a significant predictor. Individuals transferred between services are a vulnerable population as they experience: (1) a burden of comorbid illness acute enough to merit hospitalization on both a medical and psychiatric inpatient service, (2) a hand-off in care between providers, and (3) treatment by providers who are managing a condition outside of the usual scope of practice (e.g. serious mental illness on a medical ward, or a medical illness on a psychiatric ward), requiring coordination of care with consultants. Given the above, individuals with short length of stay and rapid transfer may be more vulnerable to gaps in care and subsequent decompensation. For these patients, interventions such as early consultation with psychiatry, involvement of interdisciplinary teams to address psychosocial needs, ongoing medical care during psychiatric hospitalization, and coordination of robust psychiatric and medical aftercare services may be particularly important to mitigate this vulnerability.

Machine learning is a promising method in a world increasingly shaped by highdimensional data. Despite efforts to create a metric for all-cause readmission risk (e.g. LACE, HOSPITAL), models of readmission vary widely in predictive ability (c-statistic 2–83) and have seldom been adapted to psychiatric populations (43–46). A systematic review identified 73 unique models of all-cause hospital readmission (47). Of these, only one (READMIT) has been adapted for psychiatric readmissions (8), with moderate discriminative capacity (AUC 63). Despite low probability of readmission, our model has good discriminative capacity (area under the curve of 87–88). However, replication including additional hospital systems and a larger sample of readmitted patients would permit greater confidence in our findings.

There are several limitations of this study. Because our data are correlational, we are unable to infer causal pathways. We strove to mitigate confounding by using expert- and data-driven variable selection. High-dimensional sets of variables, such as comorbidities, were projected onto lower-dimensional representations to counter selection of subgroups that may not generalize. Generalizability is a limitation to single-center observational studies, where divergent results can arise from ostensibly similar analyses (27). Another limitation is that our data were collected in a hospital system that usually requires insurance for psychiatric admission, and it was not possible to verify continuous enrollment in insurance during the study period. To minimize insurance-based effects, we included patients who were transferred to outside psychiatric hospitals (because of insurance or bed availability).

Conclusions

Bipolar disorder is a leading cause of global disability associated with a decreased life expectancy of nearly a decade. In this study, we investigate the vulnerable population of individuals with comorbid bipolar disorder and medical illness, and identify specific subgroups at increased risk of early psychiatric readmission. Under the Affordable Care Act, financial penalties for readmission have led to increased interest in understanding why patients are readmitted. To our knowledge, this is the first study to apply a learning-based approach to model psychiatric rehospitalization in a multimorbid population. In this study, potentially modifiable characteristics of the index hospitalization (including length, transfer, and disposition) emerge as important predictors. As learning- based approaches are more broadly adopted for health services research in psychiatry, it is important to consider the implications these algorithms may have when patients are identified as ‘high risk’ for outcomes such as readmission. Further work is needed to establish external validity of the algorithm in other health systems, and incorporate external measures of algorithm performance, such as by comparing trade-offs in cost, adherence, and in-hospital mortality.

Supplementary Material

Supplemental Figure 1.

Frequency of Comorbid Medical Conditions in the Study Population. Frequencies of the nine most common medical comorbidities reported in the study sample at time of initial encounter. Cardiovascular disease was the most common medical comorbidity (N=361/552; 65.3%), followed by fluid and electrolyte disorders (N=230/552; 41.7%) and anemia (N=171/552; 30.9%). Dashed border indicates classification as low likelihood of readmission. Bold border indicates classification as high likelihood of readmission.

Supplemental Figure 2.

Flowchart for study inclusion. Encounters occurring on the same date or during contiguous date ranges were condensed to a single episode of care. We selected individuals who had an ICD-9 or 10 diagnosis of bipolar disorder and at least one index hospitalization. *Index hospitalization was defined as (1) an all-cause hospitalization with a primary medical (non-behavioral diagnosis) and a primary psychiatric diagnosis at time of admission, or (2) a medical hospitalization with a primary medical diagnosis that resulted in a transfer to psychiatry, or (3) a psychiatric hospitalization with a primary psychiatric diagnosis that resulted in a transfer to medicine. Dashed border indicates classification as low likelihood of readmission. Bold border indicates classification as high likelihood of readmission.

Supplemental Figure 3.

Receiver Operator Characteristic Curve for Unbalanced Decision Tree.

Supplemental Figure 4.

Receiver Operator Characteristic Curve for Balanced Decision Tree.