Construction and validation of a predictive model for suicidal ideation in non-psychiatric elderly inpatients

Abstract

Background

Suicide poses a substantial public health challenge globally, with the elderly population being particularly vulnerable. Research into suicide risk factors among elderly inpatients with non-psychiatric disorders remains limited. This investigation focused on crafting a machine learning-based prediction model for suicidal ideation (SI) in this population to aid suicide prevention efforts in general hospitals.

Methods

A total of 807 non-psychiatric elderly inpatients aged over 60 were assessed using demographic and clinical data, and SI was measured using the Patient Health Questionnaire-9 (PHQ-9). Data were processed utilizing machine learning algorithms, and predictive models were developed using multiple logistic regression, Nomogram, and Random Forest models.

Results

Key predictors included PHQ-8, Athens Insomnia Scale, hospitalization frequency, Perceived Social Support from Family scale, comorbidities, income, and employment status. Both models demonstrated excellent predictive performance, with AUC values exceeding 0.9 for both training and test sets. Notably, the Random Forest model outperformed others, achieving an AUC of 0.958, with high accuracy (0.952), precision (0.962), sensitivity (0.987), and an F1 score of 0.974.

Conclusion

These models offer valuable tools for suicide risk prediction in elderly non-psychiatric inpatients, supporting clinical prevention strategies.

Background

Globally, suicide represents a critical public health challenge and is identified as the 14th leading cause of death [1, 2]. The susceptibility to suicide escalates with age, showing markedly higher rates among the elderly population. Suicide in the elderly is a complex and multifaceted behavior. Access to deadly means, presence of depression, disease, disability, and social disconnection are factors that increase vulnerability [3]. Data from the Global Burden of Disease estimates an annual suicide rate of 27.45 per 100,000 for individuals over the age of 70, compared to 16.17 per 100,000 for those aged 50–69, and 11.6 per 100,000 for individuals aged 15–49 [1, 2]. In China, the frequency of suicide attempts or completed suicides among general hospital inpatients stands at 3.26 per 100,000. The rates are higher among inpatients aged 60 and older, those with education levels of junior high school or lower, patients in Grade II comprehensive hospitals, and those with malignant tumors or chronic diseases [4]. Research indicates that hospitalized patients face an eightfold higher risk of suicide compared to the general population [5].

Non-psychiatric illnesses refer to medical conditions not diagnosed as mental illnesses. In this study, non-psychiatric inpatients include those admitted for various reasons, such as organic diseases or trauma. These patients may experience psychological issues, including stress and mood disturbances due to their physical health conditions, which can influence suicidal behavior. The suicide risk for non-psychiatric inpatients at admission is estimated at 2.7% [6]. Suicidal behavior in elderly non-psychiatric inpatients poses serious harm not only to individuals and their families but also to society at large [7–9]. It leads to emotional trauma and significant financial burdens for both patients and their families [9]. Additionally, medical staff who encounter inpatient suicides often experience negative emotions, with sadness (61.95%), shock (48.91%), and guilt (25%) being the most common, which can impact their professional and personal lives [10, 11]. Inpatient suicide can also strain the doctor-patient relationship and lead to medical disputes [9]. This situation may garner public attention regarding the elderly population and exert a broader societal impact. Therefore, preventing suicide among elderly non-psychiatric inpatients is of considerable importance.

SI is a broad term that encompasses thoughts, desires, and concerns about death and suicide [12]. It is often a preliminary stage before engaging in actual suicidal actions and typically involves clear self-harm intentions without detailed planning or execution [13]. In general hospitals, medical staff often hesitate to assess patients for SI, fearing that directly asking about suicidal thoughts may inadvertently lead patients to consider suicide or because such inquiries are perceived as impolite [14, 15]. Additionally, there is a societal stigma surrounding suicide [16], which makes clinicians uncomfortable when asking patients about suicidal tendencies. Consequently, SI in elderly patients is frequently overlooked, despite evidence suggesting that SI tends to increase with age, along with the incidence of suicide-related behaviors [17–19]. For instance, research from nursing homes in Hunan Province revealed a 17.9% prevalence rate of SI among the elderly [20]. The first step in suicide prevention is identifying patients at high risk [21]. Since SI is a critical risk factor for suicide [21, 22], it serves as a vital indicator in assessing suicide risk. Thus, accurately evaluating SI is essential for effective suicide prevention [23, 24].

Although awareness of suicide in the elderly has increased in recent years, existing studies highlight several factors associated with SI, including social support [20, 25], economic status [26], sleep quality [27], and psychological conditions like depression, anxiety, and loneliness [25, 28, 29]. However, most of these studies have been conducted among elderly populations in communities or nursing homes. There remains a lack of specific research focusing on SI among elderly inpatients without psychiatric illnesses. The necessity for targeted suicide prevention strategies for this demographic is critical yet underexplored. A comprehensive grasp of SI within this group remains elusive, impeding the development of effective interventions. This study aims to develop a predictive model for SI in non-psychiatric elderly inpatients and to enhance early identification and prevention of suicide risk in this population. By leveraging statistical and machine learning approaches, the model will provide healthcare providers with a clinically actionable decision-support tool for timely intervention. The ultimate goal is to reduce SI related morbidity and improve patient safety and care quality in general hospital settings. Specifically, we hypothesize that specific clinical features of non-psychiatric elderly inpatients are significantly associated with SI and may serve as key variables in predictive models for suicide risk assessment. We hypothesize that non-psychiatric elderly hospitalized patients exhibit significant associations between mental health conditions and SI, with these psychological factors serving as critical predictors in risk models. We hypothesize that insufficient family support independently predicts SI in non-psychiatric elderly inpatients, with lower levels of familial engagement correlating with heightened risk. We hypothesize that a novel SI prediction model, is expected to demonstrate superior accuracy, robustness, and generalizability compared to conventional screening tools.

Methods

This study included elderly patients with non-psychiatric illnesses who were hospitalized at Guangdong Provincial Hospital of Chinese Medicine between December 2023 and June 2024. All departments participated in the survey except for gynecology and obstetrics, surgery, and the department of infectious diseases, which were excluded due to the specific physical characteristics or age of their patients. Obstetric/gynecological cases were excluded due to pregnancy or childbirth-related physiological states, as these physiological characteristics conflict with the non-psychiatric geriatric focus. Patients with gynecological malignancies (routinely managed through the oncology department) were not excluded. Surgical cases requiring emergency interventions, trauma care, or prolonged postoperative management were excluded due to heterogeneous surgical pathologies, unpredictable complications, and recovery timelines. Infectious disease cases necessitating isolation protocols were excluded due to contagious pathologies and infection control requirements misaligned with the study’s non-communicable disease objectives.

Inclusion criteria: Eligible participants included individuals aged 60 years or older, exhibited stable vital signs, possessed the ability to communicate effectively, could cooperate in completing assessments and investigations, and voluntarily participated in the study by signing informed consent.

1. Exclusion criteria: Patients with any of the following conditions were not included: diagnoses of bipolar disorder, schizophrenia, or other mental illnesses(Patients with documented psychiatric disorders confirmed through clinical diagnosis were excluded following a systematic review of psychiatric consultation records in the hospital’s electronic medical system; Participants scoring ≥ 10 on the PHQ-9 underwent further evaluation by psychiatrists to exclude individuals with acute psychiatric symptoms (e.g., hallucinations, manic episodes) or active mental illnesses); those with severe or unstable cardiovascular, respiratory, neurological, endocrine, or other physical conditions; Aphasia, uncorrected severe hearing impairment, and functional communication disorders, etc., which affected the completion of the investigation (including but not limited to: understanding interview questions, accurately expressing subjective experiences, and filling out questionnaires).

Observation and measurement

Observational data included gender, age, smoking status, existing diseases, family status, BMI, albumin (ALB), hemoglobin (Hb), and lipid levels, all of which were analyzed for their association with SI in elderly non-psychiatric inpatients.

Participants’ personality traits (extraversion/introversion/ambiversion) were categorized through self-reported behavioral patterns and contextual interviews. Extraversion is characterized by a high propensity for social interaction, positive emotionality, and responsiveness to external stimuli [30–32]. Introversion is characterized by energy restoration through solitary reflection rather than external social stimulation, with core features encompassing a preference for deep thinking, reduced social engagement needs, and heightened sensitivity to excessive environmental stimuli [30–32]. Ambiversion is characterized by a midrange positioning on the extraversion introversion continuum, with the capacity to adapt social behaviors flexibly according to contextual demands [33].

Comorbidities were identified by extracting patients’ admission diagnoses, medication records, and laboratory/imaging reports from the hospital’s electronic medical record system.

The Perceived Social Support from Family (PSS-Fa) scale is a self-reported measure of family support, originally developed by Procidano et al. in 1983 and adapted to fit the context of China [34–36]. This scale evaluates the support, behaviors, and attitudes of caregivers towards patients and has been widely used in clinical assessments of family support. The PSS-Fa consists of 15 items, with a “yes” response scoring 1 point and a “no” response scoring 0 points. The maximum score is 15, with some items being reverse-scored. A higher total score signifies greater family support, with scores categorized as low (0–5), moderate (6–10), or high (11–15) levels of family support [35, 36].

The Barthel Index (BI) evaluates the independence of patients in daily activities. It encompasses ten activities: eating, bathing, grooming, dressing, managing bowel and bladder functions, toileting, transferring between positions, walking, and climbing stairs. The scoring system ranges from 0 to 100, where higher scores denote greater autonomy in daily functioning [37, 38].

PHQ-8 is a reliable and valid self-report tool for screening and assessing the severity of depressive symptoms [39]. It asks participants to reflect on their symptom frequency over the last two weeks, scoring each item from 0 (none) to 3 (nearly constant). Overall scores span from 0 to 24, with higher totals reflecting more pronounced depressive symptoms.

Athens Insomnia Scale (AIS) measures the severity of sleep disturbances and serves as a tool for insomnia screening. It is recognized for its consistency, reliability, and accuracy [40–42]. The scale scores insomnia severity based on the past month’s symptoms, categorizing scores into no insomnia (0–5), mild insomnia (6–9), moderate insomnia (10–15), and severe insomnia (16–24).

The ninth item of the PHQ-9 was adopted to check the patient’s thoughts of self-harm or suicide within the past two weeks, serving as a criterion for determining SI [43–45]. A response indicating suicidal ideation was classified as “SI,” while the absence of such thoughts was classified as “no SI.” Numerous studies have demonstrated that using the PHQ-9 to screen individuals for SI is effective, accurate, and reliable [46–49].

Procedure

Research Staff Training: All research assistants were required to complete a standardized 2-day training program. Key components included: Standardized protocols for administering and clarifying assessment tools; training to avoid leading language and ensure neutral participant engagement; emergency psychological crisis protocols, including escalation pathways for participants endorsing SI.

Data collection was conducted through a mixed-mode survey methodology that systematically integrated electronic questionnaires with paper-based assessments. The surveys were administered in person by well-trained researchers employing neutral, standardized language to ensure consistency. Subjects were selected strictly according to the sampling method and inclusion criteria. During the field investigation, researchers were instructed to ask relevant questions thoroughly and professionally. Patients were not permitted to complete the questionnaire themselves to ensure the accuracy and authenticity of the responses. After each survey, the questionnaire was promptly reviewed to ensure completeness, and it was assigned a questionnaire number and signed by the researcher.

Quality control

Prior to the investigation, all researchers involved received thorough training to ensure they were fully informed about the content and process of the survey. Repeated practice sessions ensured consistency among investigators, with the required standard of agreement (Kappa > 0.85) achieved before the official survey commenced. After ensuring uniformity in approach and content across interviewers, a pilot study was conducted. During the survey, the data’s accuracy and completeness were checked on-site, and any errors or omissions were corrected promptly. Each evening, the collected survey data were reviewed for quality assurance. After the survey was completed, double data entry was used to identify and eliminate questionnaires with more than 5% missing data.

Statistical analysis

Statistical analyses were completed utilizing R software (version 4.3.0). Data underwent tests for normality and variance homogeneity. Measurements adhering to a normal or near-normal distribution were described utilizing mean ± standard deviation (x ± s), while data not following were presented utilizing median and interquartile range (IQR). Normally distributed data with homogeneous variance were compared employing the t-test, and the corrected t-test was applied when variances were unequal. The Mann-Whitney U test was applied to compare non-normally distributed datasets. Pearson’s Chi-squared test was employed for categorical data comparison, while Fisher’s Exact Test with simulated p-values (based on 2,000 simulations) was utilized for precision in cases of small or unbalanced samples. The Wilcoxon rank-sum test was adopted to compare ranked data. P < 0.05 was deemed statistically significant.

Results

Demographic and clinical characteristics of participants

Table 1 presents the demographic characteristics of the enrolled subjects. The cohort consisted of 807 elderly non-psychiatric inpatients were included, with a prevalence of SI of 8.55%. Participants were categorized into two groups: 69 patients with SI and 738 patients without. The SI group contained a smaller percentage of males and a larger percentage of females (P = 0.008). Additionally, the SI group exhibited higher rates of chronic obstructive pulmonary disease (COPD), chronic kidney disease, and cancers (P < 0.001), more frequent hospitalizations (P < 0.001), higher rates of unemployment and liberal professions (P = 0.007), a higher prevalence of widowhood (p = 0.003), fewer children or no children (P = 0.030), and more individuals living alone (P = 0.031). The SI group also had a lower proportion of high-income families (P = 0.017), a higher prevalence of introverted or hybrid personalities (P = 0.035), and lower rates of no religious affiliation (P = 0.035). Furthermore, the SI group had a higher proportion of patients with more than four comorbidities (P < 0.001), more severe perceived illness (P < 0.001), lower albumin levels (p = 0.001), lower BMI (P = 0.007), lower hemoglobin levels (p = 0.007), higher phosphorus levels (p = 0.032), higher PHQ-8 scores (P < 0.001), lower BI scores (p < 0.001), and higher AIS scores (P < 0.001). Social support from family, measured by the PSS-Fa, also differed significantly (P = 0.006), and the wider range of family support scores in the SI group suggested greater variability.

Table 1.

Baseline characteristics

Variable	Overall, N = 8071	No SI N = 7381	SI N = 691	p-value2
Gender				0.008
male	450 (56%)	422 (57%)	28 (41%)
female	357 (44%)	316 (43%)	41 (59%)
Age	71 (66, 78)	71 (66, 77)	74 (69, 80)	0.056
Diseases				< 0.001
COPD	100 (12%)	89 (12%)	11 (16%)
Chronic kidney disease	99 (12%)	76 (10%)	23 (33%)
Angiocardiopathy	88 (11%)	82 (11%)	6 (8.7%)
Pneumonia	285 (35%)	272 (37%)	13 (19%)
Tumour	124 (15%)	112 (15%)	12 (17%)
Cerebrovascular disease	32 (4.00%)	31 (4.20%)	1 (1.40%)
Bronchiectasia	21 (2.60%)	20 (2.70%)	1 (1.40%)
Osteoarthropathy	50 (6.20%)	49 (6.60%)	1 (1.40%)
Asthma	8 (1.00%)	7 (0.90%)	1 (1.40%)
Frequency of hospitalization	2(1, 4)	2(1, 3)	3 (2, 12)	< 0.001
Education level				0.300
Primary school	310 (38%)	276 (37%)	34 (49%)
Junior high school	220 (27%)	202 (27%)	18 (26%)
Senior high school or secondary school	211 (26%)	199 (27%)	12 (17%)
Junior college/undergraduate	63 (7.80%)	58 (7.90%)	5 (7.20%)
Master	3 (0.40%)	3 (0.40%)	0 (0.00%)
Smoke				0.100
NO	573 (71%)	518 (70%)	55 (80%)
Yes	234 (29%)	220 (30%)	14 (20%)
Drink				0.200
NO	687 (85%)	625 (85%)	62 (90%)
Yes	120 (15%)	113 (15%)	7 (10%)
place of residence				0.600
countryside	37 (4.60%)	33 (4.50%)	4 (5.80%)
town	34 (4.20%)	30 (4.10%)	4 (5.80%)
City	736 (91%)	675 (91%)	61 (88%)
Working condition				0.007
incumbent	4 (0.50%)	4 (0.50%)	0 (0.00%)
retirement	698 (86%)	644 (87%)	54 (78%)
unemployment	5 (0.60%)	2 (0.30%)	3 (4.30%)
liberal profession	100 (12%)	88 (12%)	12 (17%)
Marital status				0.003
unmarried	6 (0.70%)	6 (0.80%)	0 (0.00%)
married	647 (80%)	603 (82%)	44 (64%)
divorced	18 (2.20%)	16 (2.20%)	2 (2.90%)
widowed	136 (17%)	113 (15%)	23 (33%)
Children (number)				0.030
0	19 (2.40%)	16 (2.20%)	3 (4.30%)
1	351 (43%)	328 (44%)	23 (33%)
2	236 (29%)	212 (29%)	24 (35%)
3	103 (13%)	95 (13%)	8 (12%)
4	64 (7.90%)	58 (7.90%)	6 (8.70%)
5	24 (3.00%)	21 (2.80%)	3 (4.30%)
6	7 (0.90%)	7 (0.90%)	0 (0.00%)
7	2 (0.20%)	0 (0.00%)	2 (2.90%)
9	1 (0.10%)	1 (0.10%)	0 (0.00%)
Living status				0.031
With spouse and children	268 (33%)	251 (34%)	17 (25%)
With spouse	305 (38%)	284 (38%)	21 (30%)
With children	144 (18%)	126 (17%)	18 (26%)
Alone	74 (9.20%)	64 (8.70%)	10 (14%)
In Nursing Home	16 (2.00%)	13 (1.80%)	3 (4.30%)
Per capita monthly household income				0.017
Less than 1500 yuan	60 (7.40%)	57 (7.70%)	3 (4.30%)
1500–2900 yuan	44 (5.50%)	35 (4.70%)	9 (13%)
3000–5000 yuan	267 (33%)	240 (33%)	27 (39%)
More than 5000 yuan	436 (54%)	406 (55%)	30 (43%)
Self-evaluation of personality				0.035
extroverted	253 (31%)	240 (33%)	13 (19%)
introverted	110 (14%)	96 (13%)	14 (20%)
ambiversion	444 (55%)	402 (54%)	42 (61%)
Religious belief				0.047
NO	751 (93%)	691 (94%)	60 (87%)
Yes	56 (6.90%)	47 (6.40%)	9 (13%)
Comorbidity				< 0.001
Less than 2 kinds of diseases	367 (45%)	355 (48%)	12 (17%)
2–3 kinds of diseases	369 (46%)	332 (45%)	37 (54%)
Greater or equal to 4 kinds of diseases	71 (8.80%)	51 (6.90%)	20 (29%)
Perceived severity of illness				< 0.001
Mild	111 (14%)	111 (15%)	0 (0%)
Moderate	289 (36%)	282 (38%)	7 (10%)
Severe	407 (50%)	345 (47%)	62 (90%)
CRP	10 (2, 43)	9 (2, 42)	17 (4, 46)	0.100
ALB	36.10 (31.90, 39.10)	36.20 (32.10,39.20)	33.60 (29.10, 37.70)	< 0.001
BMI	22.00 (19.60, 24.2)	22.10 (19.70, 24.20)	20.90 (18.40 23.20)	0.007
Hb	123 (107, 133)	124 (109, 134)	107 (95, 124)	< 0.001
GLU	5.85 (4.95, 7.47)	5.82 (4.94, 7.42)	6.12 (5.17, 8.11)	0.150
TG	1.13 (0.85, 1.65)	1.13 (0.84, 1.64)	1.04 (0.88, 1.65)	> 0.90
Ca	2.14 (2.04, 2.22)	2.14 (2.05, 2.23)	2.12 (1.99, 2.20)	0.300
P	1.08 (0.95,1.22)	1.08 (0.95, 1.21)	1.13 (0.95, 1.50)	0.032
UC	327 (249,415)	327 (251, 414)	332 (238, 438)	> 0.900
TC	4.22 (3.55, 5.13)	4.24 (3.60, 5.14)	4.09 (3.33, 5.02)	0.300
HDL-C	1.11 (0.90, 1.38)	1.12 (0.91, 1.38)	1.05 (0.87, 1.36)	0.400
LDL-C	2.54 (1.95, 3.26)	2.56 (1.96, 3.27)	2.35 (1.80, 3.12)	0.110
NHHR	2.81 (2.09, 3.64)	2.81 (2.08, 3.66)	2.81 (2.22, 3.37)	0.900
PSS-Fa	11(10, 11)	11(10, 11)	11(6, 11)	0.006
BI	95 (75, 100)	100 (80, 100)	70 (50, 90)	< 0.001
PHQ-8	3 (1.00, 6.50)	3 (1.00, 6.00)	13 (9.00, 18.00)	< 0.001
AIS	8(4, 10)	6(4, 10)	14(12, 18)	< 0.001

¹ n (%); Median (IQR)

² Pearson’s Chi-squared test; Wilcoxon rank sum test; Fisher’s Exact Test for Count Data with simulated p-value (based on 2000 replicates); Fisher’s exact test

Multiple logistic regression was implemented with the presence of SI as the dependent variable. The model integrated independent variables that emerged as significant predictors from the initial univariate analysis. These included scores from the PHQ-8, AIS, frequency of hospitalization, PSS-Fa, comorbidities, family income, and employment status (Table 2).

Table 2.

Multivariate logistic regression analysis

	B	Std.Error	t	P	OR(95% CI)
Frequency of hospitalization	0.007	0.002	2.945	0.003	1.007(1.002,1.011)
Working condition
Unemployment	0.347	0.164	2.117	0.035	1.415(1.026,1.950)
Per capita monthly household income 1500–2900¥	0.106	0.049	2.142	0.033	1.112(1.009,1.225)
Comorbidity
Greater or equal to 4 kinds of diseases	0.084	0.041	2.055	0.040	1.088(1.004,1.178)
PSS-Fa	-0.013	0.005	-2.880	0.004	0.987(0.978,0.996)
PHQ-8	0.021	0.002	8.582	0.000	1.022(1.017,1.027)
AIS	0.005	0.002	2.409	0.016	1.005(1.001,1.009)

Nomogram for predicting SI in elderly non-psychiatric inpatients

A nomogram was constructed utilizing the “Nomogram” function in R (Fig. 1). In this model, individual predictors were quantified and assigned specific scores. The aggregate of these scores for each patient determined their total score, which directly correlated with their projected risk of SI on the nomogram’s risk scale. Consequently, a higher cumulative score on the nomogram indicates an increased risk of SI for the patient.A visualized nomogram for suicide risk was constructed using these predictors to provide medical staff and patients with quantitative risk estimates based on the patient’s current health status. Each predictor was displayed on the nomogram as columns, with a 0-100 scoring scale on the top axis. The corresponding range of values for each variable was represented in rows, with the length of each line segment denoting the relative significance of that variable in predicting risk. The predicted suicide risk for elderly non-psychiatric inpatients was represented by the total score on the risk axis, offering a clear and intuitive visual tool for risk assessment.

Fig. 1.

Nomogram plot predicting SI in older non-psychiatric

Comparison of general information of patients in the training and test sets

Totally 807 patients were randomly divided into training and test sets utilizing a computerized random number generator, with an 8:2 ratio. An analysis of the general characteristics between the two sets revealed no significant statistical differences (P > 0.05), indicating effective randomization. The analysis of differences in factors included in the training and test sets is shown in Table 3.

Table 3.

Baseline characteristics

Variable	Overall, N = 8071	train N = 6461	test N = 1611	p-value2
Frequency of hospitalization	2 (1, 4)	2 (1, 4)	2(1, 4)	> 0.900
Working condition				0.500
incumbent	4 (0.50%)	4 (0.60%)	0 (0%)
retirement	698 (86%)	553 (86%)	145 (90%)
unemployment	5 (0.60%)	5 (0.80%)	0 (0%)
liberal profession	100 (12%)	84 (13%)	16 (9.90%)
Per capita monthly household income				0.800
Less than 1500¥	60 (7.40%)	51 (7.90%)	9 (5.60%)
1500–2900¥	44 (5.50%)	35 (5.40%)	9 (5.60%)
3000–5000¥	267 (33%)	215 (33%)	52 (32%)
More than 5000¥	436 (54%)	345 (53%)	91 (57%)
Comorbidity				> 0.900
Less than 2 kinds of diseases	367 (45%)	295 (46%)	72 (45%)
2–3 kinds of diseases	369 (46%)	294 (46%)	75 (47%)
Greater or equal to 4 kinds of diseases	71 (8.80%)	57 (8.80%)	14 (8.70%)
PSS-Fa	11 (10, 11)	11 (10, 11)	11 (10, 11)	0.500
PHQ-8	3 (1.00, 6.50)	3 (1.00, 7.00)	3 (1.00, 6.00)	0.700
AIS	8 (4, 10)	8(4, 10)	6 (4, 10)	0.150

¹Median (IQR); n (%)

²Wilcoxon rank sum test; Fisher’s exact test; Pearson’s Chi-squared test

Evaluation of the nomogram prediction model and random forest prediction model based on independent predictors

The Nomogram prediction model was developed utilizing the glm function in R, and its discriminatory power was assessed by calculating the AUC. For SI among elderly non-psychiatric inpatients, the model demonstrated a AUC of 0.937 (95% CI: 0.904–0.970) in the training set and 0.939 (95% CI: 0.886–0.992) in the test set (Fig. 2), reflecting its high discriminatory capacity in both datasets (Fig. 2).

Fig. 2.

ROC Curve for Nomogram model of SI in elderly non-psychiatric patients

Using the Calibrate function in R, Bootstrap sampling was implemented on the training and test sets. The calibration curve plots, obtained after eliminating selective bias, showed that the curves closely overlapped with the diagonal line, indicating a high degree of calibration and internal consistency. The model’s calibration was further endorsed by its conformity in the Hosmer-Lemeshow test, where it showed no significant deviation (P > 0.05) (Fig. 3).

Fig. 3.

Calibration plot of the Nomogram model of SI risk in elderly non-psychiatric patients

For a more intuitive presentation of the model’s accuracy, a confusion matrix heat map was constructed for the test set, visually confirming the model’s effectiveness in predicting the correct outcomes (Fig. 4).

Fig. 4.

Confusion matrix graph for the Nomogram model test set

The Nomogram model’s predictive capabilities were quantified utilizing various metrics, including accuracy, recall, precision, sensitivity, and the F1 score, across both datasets (Table 4). The analysis confirms the model’s exceptional predictive accuracy.

Table 4.

Model evaluation results of nomogram prediction model training set and test set

	Accuracy	Recall	Precision	F1
Training set	0.944	0.985	0.956	0.970
Test set	0.919	0.986	0.928	0.956

Construction and evaluation of random forest prediction models

Using the previously collected data, a Random Forest prediction model was constructed in Python, utilizing the Random Forest Classifier from the open-source machine learning toolkit, Scikit-learn. The importance of each independent predictor contributing to the risk of SI was calculated through feature importance evaluation. The predictors were ranked in descending order of importance as follows: PHQ-8, AIS, frequency of hospitalization, PSS-Fa, comorbidities, family income, and employment status (Fig. 5). The Random Forest model demonstrated its predictive strength with an AUC of 0.958 (95% CI: 0.934–0.982) in the training dataset, affirming its robust discriminative power. In the test set, the model maintained a high level of performance, recording an AUC of 0.905 (95% CI: 0.822–0.989), showcasing its consistent ability to differentiate outcomes across both datasets (Fig. 6).

Fig. 5.

Random forest feature importance ranking

Fig. 6.

Random forest prediction model ROC plot

A confusion matrix heat map for the test set was generated to summarize and visualize the prediction results of the Random Forest model. The heat map demonstrates that the model accurately predicted SI across the test set samples (Fig. 7). The model’s predictive capabilities were further assessed using key performance metrics such as accuracy, recall, precision, and the F1 score, for both datasets (Table 5). The model demonstrated strong predictive performance, with high values across all metrics in both sets.

Fig. 7.

Confusion matrix graph for Random Forest model test set

Table 5.

Model evaluation results for random forest model training set and test set

	Accuracy	Recall	Precision	F1
training set	0.952	0.987	0.962	0.974
test set	0.932	0.986	0.940	0.963

Discussion

This study developed a multidimensional predictive model of suicide risk for elderly non-psychiatric inpatients, incorporating various factors such as demographic information, PHQ-8, AIS, BI, family income, comorbidities, and clinical laboratory data. Through feature importance analysis, we identified seven clinical variables affecting the prediction of suicidal ideation, in order of decreasing importance: PHQ-8, AIS, frequency of hospitalization, PSS-Fa, comorbidities, family income, and employment status. The comprehensiveness of these predictive factors adds significant value for clinical guidance and intervention. Upon reviewing the literature, we found that few studies have applied machine learning to construct predictive models for SI in hospitalized patients without psychiatric disorders. For example, Korean scholars [50] highlighted a model with an AUC below 0.9, which incorporated factors like depression, physical and mental stress, psychotropic medication use, caregiving responsibilities, future concerns, family size, oral health, basic needs fulfillment, and life frustrations. However, these factors are less applicable to elderly non-psychiatric inpatients.

In this study, a predictive model for SI in elderly non-psychiatric patients was developed and validated using machine learning algorithms. The dataset included 807 elderly non-psychiatric inpatients, with 646 cases in the training set and 161 cases in the test set. The efficacy of the models was assessed by comparing performance metrics such as accuracy to identify the optimal prediction algorithm. The AUC values of both the Nomogram and Random Forest models exceeded 0.9 for the training and test sets, indicating good discriminatory ability.

In machine learning, evaluating the performance of a model is critical. Accuracy, Precision, Sensitivity, and the F1 Score are commonly used performance metrics that assess the effectiveness of classification models from different perspectives. Accuracy represents the ratio of correctly predicted samples to the total number of samples, focusing on the overall prediction accuracy. Precision focuses on the accuracy of samples predicted as positive, while Sensitivity primarily evaluates the model’s ability to identify positive samples. The F1 Score, on the other hand, provides a balanced consideration of both Precision and Sensitivity. A higher F1 Score indicates better performance in both Precision and Sensitivity, making it a more comprehensive evaluation metric. In this study, accuracy, precision, F1, and sensitivity values for both models were above 0.9, demonstrating excellent predictive performance. Specifically, the Random Forest model demonstrated superior predictive capabilities, registering an AUC of 0.958 in the training dataset (95% CI: 0.934–0.982), with an accuracy of 0.952, precision of 0.962, and sensitivity of 0.987, outperforming the Nomogram model in predictive performance.

In the model, the PHQ-8 score (OR = 1.022, 95% CI: 1.017–1.027) emerged as the most significant predictor, ranking as the most important risk factor. Each incremental increase in the PHQ-8 score was associated with a 1.022-fold increase in the risk of SI. Notably, while the study focused on elderly patients with non-psychiatric illnesses, the results highlighted a marked linkage between SI and the presence of depressive symptoms among participants. A total PHQ-8 score of ≥ 10 is typically indicative of depressive symptoms, suggesting that some elderly patients in this study may have experienced psychological distress or depression. Suicide screening and mental health assessment were low in older elderly [3]. Therefore, to prevent suicide in elderly non-psychiatric patients, it is crucial to enhance psychological screening and evaluation, paying close attention to depressive symptoms and mental health issues in this population.

In the model, the AIS score (OR = 1.005, 95% CI: 1.001–1.009) was identified as a critical risk factor for SI. There is extensive research indicating that insomnia is a key risk factor that may elevate the likelihood of suicide, and it is considered a modifiable risk factor [51–55]. Moreover, insomnia accompanied by short sleep duration has been shown to elevate the risk of suicide, indicating the importance of assessing both abnormal sleep duration and insomnia symptoms during suicide risk evaluations [56]. The AIS score, which provides a reliable measure for diagnosing insomnia and assessing its severity, includes key aspects such as total sleep time and time to fall asleep. Given its simplicity, the AIS score could be incorporated into the suicide risk assessment framework in general hospitals to improve monitoring and intervention for suicide risk.

Unemployment was also significantly associated with SI and emerged as an independent predictor in the model. Unemployment may increase feelings of despair among elderly patients, thereby elevating their risk of suicide [57]. Similarly, low family income was significantly associated with SI, as it negatively impacts patients’ mental health [58, 59]. Both unemployment and lower family income contribute to psychological distress in elderly inpatients, potentially increasing their suicide risk.

A novel finding in our study was that the frequency of hospitalization (OR = 1.007, 95% CI: 1.002–1.011) was linked to SI and was an independent predictor in the model. Frequent hospitalizations may exacerbate the economic, familial, and psychological burdens on elderly patients, leading to greater emotional strain. The added financial and familial pressures may elevate the risk of suicide in this population [57].

Comorbidity (OR = 1.088, 95% CI: 1.004–1.178) of four or more illnesses was significantly associated with SI. Comorbidity, defined as the coexistence of two or more chronic health conditions, poses a substantial risk. Older adults with multiple chronic diseases experience severe psychological distress, which may elevate the risk of SI and potential suicide attempts due to factors such as social exclusion, functional limitations, perceived burden, and financial strain [60]. As the number of illnesses increases, the risk of SI also rises [60, 61]. Patients with multiple comorbidities face heavy financial burdens, which may further increase their risk of suicide [57]. Additionally, the complexity of managing multiple conditions, which often requires specialized treatment from multiple healthcare providers, can heighten feelings of despair and anxiety, leading to an increased risk of suicide [62].

PSS-Fa (OR = 0.987, 95% CI: 0.978–0.996) was linked to SI. Family support is a critical protective factor against SI in elderly patients [35]. As individuals age, their families often become their primary source of both practical and emotional support, which is essential for their well-being [63]. Low levels of family support, coupled with the psychological burden many elderly patients face, tend to increase the risk of suicide.

In our analysis, measures such as triglycerides, total cholesterol, and low-density lipoprotein cholesterol (LDL-C) did not show a significant correlation with SI. This finding contrasts with other studies; for instance, American researchers have noted that a higher ratio of non-high-density lipoprotein cholesterol to high-density lipoprotein cholesterol (NHHR) correlates with an increased likelihood of SI [64]. Similarly, findings from Korea indicate a significant association between lower triglyceride levels and a decreased risk of SI in men [65], and low levels of LDL-C were linked to a higher risk of SI among Korean men over the age of 19 [66]. The discrepancies between our results and those from other regions may stem from variations in demographic or regional characteristics.

Conclusion

This study is innovative in that it applies machine learning algorithms to construct a visual nomogram for predicting SI in elderly non-psychiatric inpatients, offering a simple and intuitive tool for clinical use. This model can assist in early screening for individuals at risk of SI and provide quantitative risk assessments for healthcare providers. Additionally, we developed a Random Forest prediction model for SI in this population. Both the Nomogram and Random Forest models were validated, demonstrating high accuracy, sensitivity, and precision. These models provide a promising method for suicide prevention among elderly non-psychiatric inpatients in general hospitals, enabling targeted psychological interventions to prevent suicide and other adverse safety events.

However, the clinical interpretation of these results requires cautious optimism. Notably, existing evidence suggests that the predictive validity of SI for actual suicidal behaviors is limited.Meta-analyses indicate that even among individuals expressing SI, the transition rates to suicide attempts or deaths are low, particularly in non-psychiatric populations. In the non-psychiatric population with suicidal ideation (SI), the 12-month cumulative suicide risk was 0.23% (95% CI 0.10–0.54) [67, 68]. These findings highlight the critical need to contextualize model predictions within broader risk assessment frameworks, rather than relying solely on SI as a definitive marker.

This research has certain constraints that must be acknowledged. First, the dataset for participants reporting SI (n = 69) and stroke survivors (n = 32) was limited in size. The limited sample sizes of SI and stroke survivor may have elevated overfitting risks in the machine learning. While Fisher’s exact test and internal validation procedures partially mitigated this limitation, the external validity of these findings requires confirmation through future research employing larger, clinically diverse cohorts with longitudinal follow-up. Second, the single-center design (conducted in an urban tertiary hospital) may limit the applicability of findings to healthcare settings with disparities in resource distribution or infrastructure capacity. The exclusion of departments such as surgery, infectious diseases, and obstetrics could reduce the external validity of results for hospitals with different departmental configurations. Urban-focused cohort selection further restricts rural population applicability, given urban-rural disparities in socioeconomic factors and healthcare access that may impact clinical outcomes. Third, while suicide risk is multifactorial in nature, with psychache (e.g., feelings of hopelessness, existential distress) serving as a core driver [69], this study did not incorporate dedicated instruments to quantify psychache. Although depressive symptoms were evaluated using the PHQ-9, the absence of psychache-specific measures may have limited the mechanistic interpretation of psychological risk dimensions. Furthermore, pandemic-specific stressors (e.g., COVID-19-related social isolation, healthcare disruptions) were not systematically evaluated, despite evidence linking heightened elderly suicide risk during the pandemic to isolation, reduced mental healthcare access, and crisis-exacerbated economic instability [70]. Future research should explicitly examine how cross-cultural differences in crisis response protocols moderate these effects. Additionally, the cross-sectional nature of this study restricts our ability to explore or establish any temporal or causal relationships, potentially leading to biases in interpreting the results. Importantly, while our models achieved high statistical performance, their clinical utility may be constrained by the inherently low base rate of suicidal outcomes in this population, necessitating further validation in larger, prospective cohorts.

In the future, we plan to develop a suicide risk prediction app based on this model for use in large-scale clinical practice. To enhance real-world applicability, this tool will integrate multimodal risk indicators beyond SI. Such an approach may help healthcare professionals make more informed decisions, maximizing the clinical value of predictive models while mitigating over-reliance on single risk factors.

Abbreviations

WHO

World Health Organization

SI

Suicide Ideation

MMSE

Mini-mental state examination

MMSE

Mini-mental state examination

PHQ-9

Patient Health Questionnaire-9

PHQ-8

Patient Health Questionnaire-8

PSS-fa

Perceived Social Support from family scale

AUC

Area Under Curve

BI

Barthel Index

CI

Interval

ROC

Receiver Operating Characteristic

COPD

Chronic Obstructive Pulmonary Disease

Hb

Hemoglobin

BMI

Body Mass Index

ALB

Albumin

CRP

C Reactive Protein

Ca

Calcium

P

Phosphorus

UA

Uric Acid

GLU

Glucose

TG

Triglyceride

TC

Total cholesterol

HDL-C

High density lipoprotein cholesterol

LDL-C

Low density lipoprotein cholesterol

NHHR

non-High-density lipoprotein cholesterol to High density lipoprotein cholesterol ratio

Author contributions

S.Y.X. has full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. J.C.C. and D.X.S. are responsible for the concept of the study. J. L.Z., Y.L., D.N.L., L.X.H., X.H.C. M.Z.D. and C.Y.T. conducted the questionnaire. S.Y.X.,D.X.S. and C.Y.T. are responsible for the statistical analysis. All authors made substantial contributions to the study design, analysis and interpretation of data, and the writing of the manuscript.

Funding

This study was funded by Guangzhou Municipal Science and Technology Bureau (SL2023A03J00112).

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author by request.

Declarations

Ethical approval and consent to participate

The study was conducted in accordance with the guidelines of the Declaration of Helsinki and approved by the Medical Ethics Committee of Guangdong Provincial Hospital of Chinese Medicine (number: ZF2023-453-01). Informed consent was obtained from each participant with the right to withdraw from the study at any time and their names and other confidential information were protected. The participants were not harmed.

Consent for publication

Not Applicable.

Competing interests

The authors declare no competing interests.