Predicting Suicidal Behavior and Self-harm after General Hospitalization of Adults with Serious Mental Illness

Abstract

Individuals with psychiatric disorders are vulnerable to adverse mental health outcomes following physical illness. This longitudinal cohort study defined risk profiles for readmission for suicidal behavior and self-harm after general hospitalization of adults with serious mental illness. Structured electronic health record data were analyzed from 15,644 general non-psychiatric index hospitalizations of individuals with depression, bipolar, and psychotic disorders admitted to an urban health system in the southwestern United States between 2006–2017. Using data from one-year prior to and including index hospitalization, supervised machine learning was implemented to predict risk of readmission for suicide attempt and self-harm in the following year. The Classification and Regression Tree algorithm produced a classification prediction with an area under the receiver operating curve (AUC) of 0.86 (95% confidence interval (CI) 0.74–0.97). Incidence of suicide-related behavior was highest after general non-psychiatric hospitalizations of individuals with prior suicide attempt or self-harm (18%; 69 cases/389 hospitalizations) and lowest after hospitalizations associated with very high medical morbidity burden (0 cases/3,090 hospitalizations). Predictor combinations, rather than single risk factors, explained the majority of risk, including concomitant alcohol use disorder with moderate medical morbidity, and age ≤ 55-years-old with low medical morbidity. Findings suggest that applying an efficient and highly interpretable machine learning algorithm to electronic health record data may inform general hospital clinical decision support, resource allocation, and preventative interventions for medically ill adults with serious mental illness.

Introduction

Suicide is among the top twenty leading causes of death worldwide (Curtin et al., 2016; Miron et al., 2019). Suicide attempts and self-harm among people with serious mental illness who have medical comorbidities are of increasing societal concern, because of increased recognition of the compounding vulnerabilities of physical and psychiatric illness (Firth et al., 2019; Rosenbaum, 2016). Hospitalization is associated with an elevated risk of suicide attempts and suicide death after discharge (Ballard et al., 2014; Erlangsen et al., 2005).

Mental illness and physical illness independently increase risk of suicide attempts and suicide death (Barak-Corren et al., 2017; Kavalidou et al., 2017). However, to date there has been sparse work devoted to distinguishing risk of suicide-related behavior in populations with serious mental illness and medical comorbidity. The majority of research in post-discharge suicide-related behavior has focused on psychiatric, as opposed to medical, hospitalizations (e.g. Kessler et al., 2015; Qin and Nordentoft, 2005). After acute physical illness, individuals experience a unique and complex array of vulnerabilities that may destabilize mental health and promote suicidal thoughts and behavior. Examples of these vulnerabilities include medical trauma (Zatzick et al., 2007), pain (Kannampallil et al., 2016), functional impairment (Davydow et al., 2013), financial stressors (Dobkin et al., 2018), and exacerbation of chronic mental illness (Prince et al., 2018). Not surprisingly, the proximity to and frequency of medical hospital admissions predict risk of serious suicide attempts and suicide completion (Qin et al., 2013). Few suicide prevention efforts have been successfully geared specifically to populations with physical illness (Druss and Pincus, 2000; Pompili et al., 2012; Henson et al., 2019), and adults with psychiatric disorders are often not routinely screened during or after medical hospitalization for emergence of suicidal thoughts or behaviors (King et al., 2017).

Post-discharge suicide-related behavior among medical inpatients with serious mental illness remains largely understudied, and prediction of this behavior important gap in knowledge for potential life- and cost-saving interventions. Applying machine learning (ML) techniques to electronic health record (EHR) data has emerged as a promising means by which to enable prediction of psychiatric outcomes (Galatzer-Levy et al., 2014; Ioannidis et al., 2016), usually with very few assumptions about the data (Ruppert, 2004). In our study, we sought to derive and validate a risk prediction model of readmission for suicide-related behavior in the year following general hospitalization among adults with serious mental illness. We focus on adults with major affective and chronic psychotic disorders given the high prevalence of co-occurring medical comorbidity and suicidal behavior in this population (Kavalidou et al., 2017; Prince et al., 2007; Qin et al., 2013; Rosenbaum, 2016; Weiner et al., 2011). We hypothesized that information routinely collected in the course of care would allow for construction of an algorithm for the prediction of readmission for suicide-related behavior. To test this hypothesis, we applied a supervised ML approach to EHR data from a large urban health system.

Material and Methods

Study design

This was an EHR-based retrospective cohort study conducted in accordance with the Reporting of studies Conducted using Observational Routinely Collected Data (RECORD) statement guidelines (Supplemental Table 1). Following extraction, data were analyzed using Classification and Regression Tree (CART) modeling. This study focused on adults with serious mental illness as these individuals experience both a high propensity for medical illness and an elevated risk of serious suicide attempts and self-harm, and thus represent a population particularly vulnerable to adverse outcomes after medical hospitalization. Although all cases of suicidal behavior are important, this study focused on general hospital readmissions for suicide attempt and self-harm to capture behavior requiring inpatient medical care for physical sequelae.

This study is an analysis of existing deidentified health records, and no direct interaction with human participants occurred. The need for ethical approval and participant consent was waived by the University of California Los Angeles (UCLA) Institutional Review Board (IRB#15–001653).

Dataset description

Longitudinal EHR data was extracted from the University of California, Los Angeles (UCLA) Clinical and Research Data Repository (xDR). UCLA xDR is an EHR datamart with information on inpatient, outpatient, and emergency department encounters from two large general hospitals and affiliated clinics within the UCLA Health System. From this datamart, we limited data extraction to patients (1) ≥18 years-old, (2) with International Classification of Disease (ICD) codes for a depressive disorder (ICD-9 296.20–296.36; ICD-10 F32–33.x), bipolar disorder (ICD-9 296.00–296.89, 296.40– 296.89; ICD-10 F31.0–31.9), schizophrenia or schizoaffective disorder (ICD-9 295.xx; ICD-10 F20.x, F25.x), and (3) two or more general hospitalizations. Diagnoses were restricted to major affective and psychotic disorders to maintain consistent consistent with previous investigations of co-occurring physical illness and serious mental illness (e.g. Lee et al., 2012; Šprah et al., 2017). At least one all-cause readmission within our health system was necessary for inclusion. The index hospitalization was defined as the first general hospitalization during the study period for a primary medical (non-psychiatric) diagnosis. Data were available from January 1^st, 2006 through December 31^st, 2017. To capture longitudinal information one year before and after index hospitalization, index hospitalizations were restricted to admissions between January 1^st, 2007 through December 31^st, 2016. A random sample of 15% of de-identified records were manually reviewed by the study’s authors to ensure extracted data met the above study inclusion criteria.

Predictor selection

To forecast readmission for suicide attempt or self-harm, we extracted information on patient demographics, diagnoses, medications, disposition, chief complaint, length of stay, and prior outpatient, inpatient, and emergency department service utilization. Categories of predictors are presented in Table 1 and a complete list of predictors included in each category is available in Supplemental Table 2.

Table 1.

Extracted Electronic Health Record Data

Anonymized patient identifier

Anonymized encounter identifier

Age at first admission

Age at encounter

Sex

Race

Ethnicity

International Classification of Diseases (ICD) 9 and 10 diagnoses

Diagnosis date

Primary diagnosis associated with admission

Medications

Type of encounter (ambulatory, emergency department, inpatient)

Disposition (home with self-care, home with home health, psychiatric admission, acute care hospital, hospice, skilled nursing facility, eloped, against medical advice, inpatient rehab facility, long- term care)

Transfer to psychiatric care

In-hospital mortality

Vital signs (heart rate, blood pressure, temperature, respiration rate, oxygen saturation, pain score)

Procedure and procedure code

Financial charges associated with the encounter

Service date of charges

Chief complaint

Date of admission

Date of discharge

Medical and psychiatric diagnoses were identified and categorized by ICD-9 and −10 codes. Medical diagnoses were classified using the Elixhauser comorbidity system, an established method of categorizing ICD codes into a comprehensive set of 30 comorbidity measures corresponding to common disease conditions (Elixhauser et al., 1998). The Elixhauser comorbidity classification system was condensed into a single numeric score (the van Walraven score) that summarizes disease burden and is associated with risk of disease burden in the hospital (Van Walraven et al., 2009). To ascertain psychiatric comorbidity, psychiatric conditions were grouped into depressive disorders (ICD-9 296.20–296.36; ICD-10 F32–33.x), bipolar disorders (ICD-9 296.00–296.89, 296.40– 296.89; ICD-10 F31.0–31.9), schizophrenia or schizoaffective disorder (ICD-9 295.xx; ICD-10 F20.x, F25.x), and anxiety disorders (ICD-9 300.0–300.3; ICD-10 F40–41.x). Personality disorders were categorized by type (ICD-9 301.0–301.9; ICD-10 F60.0–60.9) and substance use disorders were categorized by substance (ICD-9 291.xx, 292.xx, 303,xx, 305.xx; ICD-10 F10–19.x).

Clinical outcomes

The primary outcome was hospital readmission associated with at least one diagnostic code for suicide attempt or self-harm in the 365 days following discharge from a medical hospitalization. Suicide attempt and intentional self-harm were defined by ICD-9 and ICD-10 codes specified in the 2018 National Health Statistics Report of the Center for Disease Control and Prevention (Hedegaard et al., 2018) (ICD-9: E950.0-E959; ICD-10: X71.0xx-X83.8xx, T36.2-T71.232, T14.91).

Data preparation

Static patient-level predictors were used in their extracted form (e.g. natal sex). Encounter-level predictors were compressed to presence (for categorical variables such as medications) or quantity (for continuous variables such as number of hospitalizations) within the 365 days prior to, and including the duration of, the index hospitalization. Length of stay was calculated as the number of days from admission to discharge (across contiguous hospitalizations).

Because we aimed to capture risk associated with each hospitalization, analyses were performed at the level of hospital encounter. This approach allowed capture of time-varying predictors, such as number of ambulatory visits in the year preceding index hospitalization, and a dynamic assessment of risks specific to each hospitalization. We anticipated a subset of the sample would be repeatedly hospitalized. To detect model bias for patients with repeated hospitalizations, the number of hospitalizations in the 365 days prior to index hospitalization was included as a predictor in the classification model. If an individual was hospitalized repeatedly within a 365-day period, the outcome was considered present if any medical hospitalization associated with suicide attempt or self-harm occurred (i.e. we examined the entire 365-day post-discharge period for hospitalizations related to suicide attempt or self-harm). Missing predictor values were imputed via corresponding medians (Acuña and Rodriguez, 2004).

Machine learning approach

The machine learning approach in this work involved several steps that informed the discovery of potentially predictive features and evaluation of the predictive value of features selected by the classification algorithm. For clarity, the following steps are presented in a sequential manner, though in reality the methods were developed iteratively and somewhat in parallel, consistent with other successful applications of machine learning to psychiatric clinical data (e.g. Galatzer-Levy et al., 2014).

CART Model

The CART algorithm that was used in this study was implemented using the Scikit-learn Python toolbox sklearn.tree (Pedregosa et al., 2011). Due to their highly interpretable representations and robustness to highly complex and nonparametric data, “tree” models are used in a variety of data mining and machine learning applications (Fischer et al., 2006; Stel et al., 2003). CART modeling was chosen to examine predictor combinations, automate feature selection, and produce a model optimizing for clinical interpretability in a computationally efficient manner. The CART model was run with equal-weighted priors to account for anticipated class imbalance in scipy notation: class_weight=‘balanced’. The Gini index was used to determine tree splits. Tree balancing maximized sensitivity in detecting occurrence of the outcome and the false positive rate was expected to correspondingly increase. The relative trade-off in risk of over-detection (unnecessary screening or prevention efforts) compared with risk of under-detection supported the use of tree balancing and optimization for sensitivity.

Tree Pruning

CART provides a cost-complexity based tree pruning strategy, i.e. a complexity parameter (cp), to optimize the trade-off between the cost of misclassification and the tree complexity. The cp is a hyperparameter used to control the size of the decision tree and select the optimal tree size. Trees were constructed for a sequence of values of cp, and the final cp was chosen to correspond to the value that yielded a prediction error one standard error larger than the minimum estimated by cross-validation (Therneau et al., 2015). The complexity parameter was calculated using R 3.6.1 rpart via the Python interface to the R language (rpy2). The cross-validation procedure for cp selection was in addition to, rather than embedded in, the CART algorithm.

Cross-validation

All analyses were conducted using an objective 10-folds cross validation (Stone, 1974). At each replication the sample was divided into complementary training and testing sub-samples. Each hospitalization was randomly assigned into ten non-overlapping subsets containing a similar number of cases and non-cases. A set seed was placed to enable replicability of results (‘random_state=seed’). The classification algorithm was trained in nine of the ten data sub-samples and then independently tested in the remaining tenth subset. This procedure was iteratively conducted resulting in all tenths of the data used for both training and testing the algorithm, i.e. 10-folds. The cross-validation algorithm was written in Python version 3.7.1 to randomize cases into each fold. The cp was computed separately for each fold.

Accuracy metrics

Classification tree performance was measured by the area under the receiver operating curve (AUC), where AUC of 1.0 indicates the classifier performs perfectly, and AUC of 0.5 indicates the classifier performs at chance (Ranganathan, 2019). The algorithm sensitivity, specificity, accuracy, positive predictive value (PPV), negative predictive value (NPV), and F-statistic were also examined. The mean and 95% confidence interval were calculated for each metric.

Results

Sample

The dataset contained information from inpatient encounters (N=77,296). Inpatient encounters that occurred on the same or contiguous dates were combined into hospital episodes of care (N=24,908). Hospital episodes of care were restricted to those followed by an all-cause readmission occurring within 365 days, yielding a final sample of 15,644 index hospitalizations (3,091 patients). The flowchart for encounter selection is presented in Supplemental Figure 1.

Descriptive measures of the sample are presented in Table 2. Patients had a mean age of 56.5 years (SD = 19.0) and 47.3% were male. The most common psychiatric diagnosis present at index hospitalization was a non-affective psychotic disorder (53.6%), followed by depression (43.1%), bipolar disorder (16.2%), and schizoaffective disorder (1.0%). Median van Walraven score was 24 (SD 13.2). Median length of stay was 4 days (interquartile range [IQR]: 2–7). Median days to hospital readmission was 38 (IQR: 13–105). Four percent of medical admissions resulted in transfer to psychiatry. The most commonly prescribed medication was lorazepam (48.7%), followed by trazodone (28.7%), quetiapine (25.1%), olanzapine (16.6%), mirtazapine (15.9%), haloperidol (12.9%), and sertraline (10.4%).

Table 2.

Sample Descriptives

		N	%
Total		3,091	100.0%
Sex
	Female	1,628	52.7%
	Male	1,463	47.3%
Race/Ethnicity
	White, Non-Hispanic	2,094	67.7%
	Hispanic	437	14.1%
	Black, Non-Hispanic	338	10.9%
	Asian	140	4.5%
	Native American / Alaska Native	13	0.4%
	Native Hawaiian / Pacific Islander	4	0.1%
	Other	143	4.6%
Age
	18–39	487	15.8%
	40–64	1,275	41.2%
	≥ 65	1,329	43.0%
Psychiatric diagnoses
	Psychotic disorder1	1,689	54.6%
	Depression2	1,332	43.1%
	Bipolar disorder3	502	16.2%
Comorbid medical conditions
	Hypertension	1,712	55.4%
	Fluid or electrolyte disorder	1,398	45.2%
	Cardiac arrhythmia	1,225	39.6%
	Anemia	1,189	38.5%
	Chronic ischemic heart disease	1,006	32.5%
	Renal failure	813	26.3%
	Chronic pulmonary disease	750	24.3%
	Congestive heart failure	519	16.8%
van Walraven Score5
	≥ 20	1,299	42.0%
	10–19	682	22.1%
	0–10	857	27.7%
Substance use disorder
	Nicotine use	359	11.6%
	Drug abuse	343	11.1%
	Alcohol abuse	254	8.2%
Medications6
	Antidepressant	1,060	67.3%
	Anxiolytic	928	58.9%
	Antipsychotic	745	47.3%
	Mood stabilizer	150	9.5%

¹Corresponding to ICD-9 296.00–296.89, 296.40– 296.89; ICD-10 F31.0–31.9

²Corresponding to ICD-9 296.20–296.36; ICD-10 F32–33.x

³Corresponding to ICD-9 295.xx; ICD-10 F20.x, F25.x.

⁵Summary numeric score derived from Elixhauser comorbidity classification system corresponding to overall disease burden and in-hospital mortality

⁶Information on medications was only available for 1,576 patients. Classification tree analyses used median imputation for missing values.

The readmission rate was 1.4% (N=218/15,644 hospitalizations) for suicide attempt or self-harm, 4.2% for suicidal ideation, and 5.5% for any suicidality (overlap occurred as some patients were simultaneously admitted for both suicide attempt and suicidal ideation). Readmission for a medically serious suicide attempt occurred after a median of 36.5 days (IQR: 13–102) and lasted a median of 3 days (IQR: 3–6). With respect to disposition after hospitalization for suicide attempt or self-harm, 50% of patients were discharged home, 7.3% were transferred to psychiatry, 7.8% were transferred to a step-down acute care setting, 3.1% were transferred to hospice, 3.6% left against medical advice, and 1.8% were transferred to a drug or alcohol use rehabilitation center. Of those readmitted for serious suicide attempt or self-harm, 9.6% (N=21/218) of index hospitalizations were associated with suicide attempt or self-harm, whereas the remaining 90.4% (N=197/218) were for other medical conditions.

Model performance

Accuracy metrics are presented in Table 3. The model identified 181/218 rehospitalizations for suicide-related behavior with AUC 0.86 (95% CI 0.74–0.97), sensitivity 81.9% (95%CI 58.6–100.0), specificity 79.7% (95%CI 72.1–87.2), and accuracy 79.7% (95% CI 72.4–86.9). PPV was low (5.44, 95%CI 3.23–7.66) as expected given rarity of outcome. The classification tree is displayed in Figure 1. (The classification tree with cohort counts at each node is included in Supplemental Figure 2.) History of suicide attempt or self-harm, medical comorbidity, in-hospital mortality score, age, number of medical hospitalizations in the previous year, alcohol use disorder, and bipolar disorder emerged as the most important predictors.

Table 3.

Classification Tree Performance

	Fit Metrics
	Sensitivity	Specificity	PPV	NPV	Accuracy	F-Statistic	ROC-AUC	PR-AUC
Mean	81.91	79.66	5.44	99.68	0.80	0.87	0.86	0.25
95% CI	58.61–100.0	72.11–87.22	3.23–7.66	99.28–100.0	0.72–0.87	0.83–0.92	0.75–0.97	0.05–0.45
	Contingency Matrix
	TP	FN	FP	TN
	181	37	2923	12503

PPV=Positive Predictive Value, NPV=Negative Predictive Value, ROC-AUC=Area Under the Receiver Curve, PR-AUC=Precision-Recall Area Under the Curve, TP=True Positives, FN=False Negatives, FP=False Positives, TN=True Negatives, 95%CI=95% Confidence Interval. F-statistic is reported as weighted average.

Figure 1: Decision tree for risk stratification of readmission for suicide attempt and self-harm following medical hospitalization.

The figure visually depicts a binary tree stratifying risk of readmission for suicide attempt or self-harm following medical hospitalization among individuals with serious mental illness (blue = low risk, red = high risk). Each path from root to leaf node can be translated into a series of ‘if-then’ rules that can be applied to classify observations. Each leaf node is associated with a decision rule, corresponding to the most frequent class label (i.e. attempt vs no attempt) of the observations belonging to that node. The denominator represents the number of total hospitalizations corresponding to that decision rule, and the numerator represents the number of readmissions for suicide attempt or self-injury in the year following the hospitalization. Elixhauser category diagnoses refers to the number of common disease conditions. In-hospital mortality score refers to the van Walraven score, a standardized method of condensing medical comorbidities into a single numeric score discriminating risk of death in the hospital. Top right diagram visually represents risk stratification, where circle size represents the size of the subgroup and shading represents the risk of suicide attempt or self-harm. An example of a tree path (right-most branch): 15,644 index hospitalizations were followed by an all-cause readmission, of these, 389 index hospitalizations were of individuals with a prior suicide attempt or self-harm, of these, 69 (18%) index hospitalizations were followed by a readmission for suicide attempt or self-harm. SMI = Serious Mental Illness.

The highest risk group comprised hospitalizations of individuals with a history of suicide attempt or self-harm. These hospitalizations had a 18% risk (N = 69/389 hospitalizations) of readmission for suicide-related behavior in the following year, compared with 1% risk (149/15,255) amongst hospitalizations of individuals without a history of suicide attempt or self-harm. Among hospitalizations for individuals with at least two medical comorbidities, alcohol use disorder was associated with increased risk (31/1252, 2.5% risk) (Supplemental Figure 2). The second highest risk subgroup (28/221, 12.7%) included hospitalizations of individuals with no history of suicide attempt or self-harm, two to eleven medical comorbidities, a history of alcohol use disorder, and more than five medical hospitalizations in the past year. Hospitalizations of individuals with a moderate in-hospital mortality score (van Walraven score 12–26) and bipolar disorder also were associated with an increased risk (17/337, 5.0%).

Among hospitalizations of individuals with no history of suicide attempt or self-harm and less than two medical comorbidities, age at time of hospitalization moderated risk: hospitalizations of individuals under 55 years old were at increased risk (43/482, 8.9%) compared with hospitalizations of individuals 55 years and older (3/478, 0.6%). Among hospitalizations of individuals with at least two medical comorbidities, no alcohol use disorder, and a high-to-moderate in-hospital mortality score (van Walraven score 26–34) those occurring at age 54 years or older and preceded by five or more outpatient visits in the prior year were associated with high risk (2.6%).

Several large subgroups were notable for the absence of any readmissions for suicide attempt or self-harm (0/4,523): (1) Hospitalizations of individuals with a very high in-hospital mortality score (van Walraven score greater than 35) and no history of suicide attempt or alcohol use disorder (0/3,090), (2) Hospitalizations occurring at age 54 years or younger and associated with a high-to-moderate in-hospital mortality score (van Walraven score 26–34) and no history of suicide attempt or alcohol use disorder (0/1,116), (3) Hospitalizations occurring at age 55 years or older and associated with high-to-moderate in-hospital mortality score (van Walraven score 26–34), no history of suicide attempt or alcohol use disorder, and less than 5 outpatient visits in the past year (0/317).

Discussion

Medical hospitalization imposes extraordinary stress on an already ill person. Medical inpatients are at increased risk of serious suicide attempts and suicide death after hospital discharge compared with the general population (Erlangsen et al., 2005; Qin et al., 2013). In this study, we applied a classification algorithm to structured EHR data from an urban health system and derived a model differentiating risk of readmission for suicide attempt and self-harm after medical hospitalization among adults with serious mental illness. The study population had a moderate to high burden of acute and chronic physical illness and co-existing psychiatric illness. A small but clinically significant subset of general non-psychiatric hospitalizations (1.4%) were followed by a medical readmission for suicide attempt or self-harm. The classification tree performed well in identifying hospitalizations likely to be followed by a readmission for suicide attempt or self-harm, and was in keeping with performance of such models in other populations (e.g. Belsher et al., 2019). We found that history of prior suicidal behavior, medical comorbidity, age, and alcohol use disorder differentiated risk of readmission for serious suicide attempt or self-harm in the year following medical hospitalization. To our knowledge, this is the first study to investigate serious suicide-related behavior after acute medical care utilization in a population with this degree of medical and psychiatric comorbidity.

The most important predictor (highest branch point of the decision tree) was prior suicide attempt or self-harm. Our model identified that among adults with serious mental illness and prior suicide attempt or self-harm, nearly 1 in 5 general (non-psychiatric) hospitalizations were followed by a readmission for suicide attempt or self-harm. A general hospitalization was associated with 12.8x the risk of readmission for suicide attempt or self-harm if the individual hospitalized had a history of suicide attempt or self-harm (18% vs 1.4%). This finding is consistent with patterns of recurrent suicidality in other populations (Anderson et al., 2015; Barak-Corren et al., 2017). However, the majority of medical inpatients, even those with comorbid serious mental illness, remain unscreened during hospitalization (Pezzia et al., 2018). This study’s finding supports efforts to increase screening for prior self-directed violence among medical inpatients, particularly those with serious psychiatric comorbidity.

Our results suggest that medical morbidity is an important risk modulator for suicide-related behavior following medical hospitalization among adults with serious mental illness. This finding builds upon prior population studies of suicide risk in medically ill populations unselected for psychiatric comorbidity (Qin et al., 2013). Medical morbidity was a high branch point in the classification model. However, rather than differentiating risk itself, medical morbidity moderated which subsequent branches (predictors) were important for classifying risk. For example, in our study population, hospitalizations of individuals with few or no comorbid medical diagnoses and less than 55 years old at time of admission were associated with an elevated risk of suicide attempt or self-harm (8.9%) whereas hospitalizations of those over 55 years old were associated with a low risk of attempt or self-harm (0.6%). By contrast, hospitalizations of individuals with moderate medical comorbidity and above age 55 years old were at higher risk of attempt or self-harm (2.6%). This risk pattern may reflect that, as most serious mental illnesses are first diagnosed in adolescence or young adulthood, younger individuals in our sample are temporally closer to their initial diagnosis of serious mental illness, thus a second new serious problem (i.e. a medical hospitalization occurring in the absence of prior medical diagnoses) may be particularly destabilizing. By contrast, older individuals with serious mental illness, likely for decades preceding medical hospitalization, may be more vulnerable to suicidality when a medical hospitalization occurs in the context of pre-existing disease burden.

Alcohol use disorder emerged as an important risk factor of suicide attempt and self-harm after medical hospitalization. Notably, among hospitalizations of individuals without a history of suicide attempt or self-harm and with at least moderate medical comorbidity, the presence of alcohol use disorder conferred increased risk of readmission for suicide attempt. Individuals with alcohol use disorder and frequent hospitalizations were particularly vulnerable. Surprisingly, although alcohol use, serious mental illness, and frequent medical hospitalization are all risk factors for suicide, few studies have evaluated interventions to increase referral to substance use treatment as potentially efficacious in reducing risk of post-discharge suicide after medical hospitalization (Esang and Ahmed, 2018). Our results suggest that efforts to increase universal screening for suicide risk among medical inpatients may benefit from concurrent inclusion of screening and potentially brief intervention for alcohol use disorder. To account for the nesting of hospitalizations within patients, the number of hospitalizations in the previous year was included as a predictor and emerged as significant in one sub-branch of the tree (see Figure 1), suggesting that among individuals with alcohol use disorder, predictions may be weighted toward patients with more frequent general hospitalizations.

Nearly a quarter of the sample was stratified into low-risk groups with no medical readmissions for suicide attempts or self-harm behaviors. The largest portion of these individuals (N=3,090) had a very high medical comorbidity score (van Walraven score ≥35). While moderate to high medical comorbidity tended to confer higher risk of suicidality, this extremely medically ill subpopulation appeared to be at lower risk. There are a number of potential reasons why very ill people did not return as frequently for suicide attempt: people who are very medically ill may be less likely to have medical visits coded for suicide-related complaints, may be more likely to die from suicide attempts, or may die of medical illness and not return to care. Although we excluded individuals who died during hospitalization, EHR records do not capture death outside of the hospital. Nevertheless, our finding prompts consideration of the nonlinear association between medical comorbidity and suicide risk, as well as the importance of further study of suicide risk at the end-stage of medical illness in mentally ill individuals.

Limitations

There are several limitations to the present study. Our study focused on a specific population of individuals with serious mental illness and multiple general medical hospitalizations, and the results may not be generalizable to other populations. Data were derived from one health system: hospitalizations at other institutions, suicide deaths outside of the hospital, and variants of suicidal behavior sub-threshold for admission were not captured in our dataset. Our study is limited by the open system issues common to EHR data and patients may have moved between institutions during the study period, resulting in data loss. To mitigate such right censoring, we restricted our sample to hospitalizations followed by at least one readmission. Time-varying care utilization metrics (such as number of ambulatory visits in the previous year) may have been confounded with clinical severity not captured by ICD codes and subject to left censoring in a manner that was not possible to track using EHR data. The relative value of benefits (increased screening for true positive cases) compared with harms (increased screening for false positive cases) supported optimization for sensitivity, and the model produced a large number of false positives. As encounters for diagnosis codes of suicidal ideation or attempt are undercoded, restricting our outcome to these encounters may have missed cases of suicidal behavior. Moreover, mental disorders were not diagnosed based on structured diagnostic interviews, and thus the reliability of the disorders is unclear. Cohort discovery of suicide-related behaviors from EHRs remains imperfect and is an important area of development within psychiatry. Although we attempted to prevent overfitting by use of standardized procedures to favor a parsimonious and conservative model, replication in larger multi-institutional datasets is key to validation.

Future research

Suicide is a public health crisis, and many gaps remain to developing effective means to prevent and intervene at various points in the continuum from suicidal behavior to completed suicide. The ultimate goal is to identify all high-risk individuals prior to discharge from a medical hospitalization, with the aim of connecting those individuals to appropriate services, but until then, we can guide screening efforts by stratifying populations using risk variables that are readily available and quantifiable in medical record data. In the immediate term, our study provides a first step toward forecasting whether a specific medical hospitalization will be followed by a readmission for serious suicide attempt. The success of this general approach depends critically on collection and sharing of large-scale, diverse datasets from external hospital systems. Future studies should expand to broader hospital settings, include linked sources of information such as data on social determinants of health, target understudied subpopulations such as women and minorities, and begin to incorporate codified universal screening measures and unstructured documentation data.

Data Reference

The data that support the findings of this study are available from UCLA Integrated Clinical and Research Data Repository (xDR), a large-scale clinical data warehouse supporting data analyses and extractions for research. Restrictions apply to the availability of these data. Third-party UCLA Health data agreements can be established to share UCLA health data for projects that have the potential to improve human health and benefit society. Such agreements require approval by the Health Data Oversight Committee to ensure strategic and responsible third-party data sharing (https://www.ctsi.ucla.edu/researcher-resources/pages/third_party).

Highlights.

1. A subset of general hospitalizations of adults with serious mental illness are followed by readmission for suicide attempt and self-harm.

2. Information routinely collected in the course of hospital care and stored in the electronic health record may be leveraged to classify post-discharge risk.

3. Findings support the importance of screening during medical hospitalization for prior suicidal behavior and emphasize the need for efficient modeling of interactions to understand the emergence of suicidality after medical hospitalization among people with pre-existing psychiatric illness.