Machine learning modeling for outcome prediction of hospitalized patients with aneurysmal subarachnoid hemorrhage

Abstract

Purpose

Aneurysmal rupture and subarachnoid hemorrhage (SAH) have an exceptionally high mortality and morbidity burden. The aim of this study was to develop interpretable machine learning models for predicting short-term poor outcomes defined by the National Inpatient Sample Subarachnoid Hemorrhage Outcome Measure (NIS-SOM).

Methods

The National Inpatient Sample (NIS) database was queried from 2008 to 2018 to identify patients diagnosed with SAH who had undergone endovascular coiling or clipping for intracranial aneurysm. Demographic, comorbidity, risk factor, and hospital characteristic variables were recorded. Variables were preprocessed, and the feature space was reduced to include the most important features. To predict poor outcomes, machine learning models were trained and cross-validated before being evaluated on a separate testing set. Shapley Additive exPlanations of the best performing model was used for general and local model interpretation.

Results

Among 18,149 admissions (mean age 55 ± 14 years, 68.8% women), 52.9% had a poor outcome. Test-set AUCs ranged 0.74–0.80; a multilayer perceptron performed best (AUC 0.80, precision 0.74, recall 0.82). SHAP ranked the ten most influential variables: age, neurological comorbidity, paralysis, Medicare insurance, smoking status, Elixhauser burden, fluid-electrolyte disorders, weight loss, arrhythmia, and heart failure.

Conclusions

The modeling predicted nationwide aSAH prognosis with decent accuracy and highlighted clinical, socioeconomic, and system-level drivers of determinants of poor short-term outcome. These results support the potential of explainable ML tools as complementary tools for early risk stratification, guiding resource allocation, and informing prospective multi-center validation and implementation studies.

Introduction

Aneurysmal subarachnoid hemorrhage (aSAH) accounts for roughly 5–10% of all strokes and disproportionately affects younger, working-age individuals. ¹ Despite advances in critical care and aneurysm-securement techniques, aSAH still carries approximately 40% case-fatality within 30 days and leaves many survivors with long-term neurological disability. ¹ National trend analyses show that US in-hospital mortality rates have plateaued since 2010 despite rising use of endovascular therapy, ² and a recent Nationwide Inpatient Sample (NIS) study documented marked geographic and socioeconomic disparities in treatment access and outcomes for high-grade aSAH. ³ These observations underscore the need for accurate, early, and widely generalizable prognostic tools.

Bedside grading systems such as Hunt–Hess, WFNS, and SAHIT explain only a fraction of outcome variance and perform inconsistently outside their derivation cohorts. At the national-database level, Washington et al. introduced the NIS-SAH Severity Score (NIS-SSS) and its companion Outcome Measure (NIS-SOM), providing a validated composite endpoint for large-scale studies. ⁴ Yet even with these metrics, conventional modeling struggles to capture the nonlinear interplay of demographic, comorbidity, and systems-of-care factors that might influence prognosis.

Machine-learning (ML) approaches have therefore attracted increasing interest. Early studies were small and single-center: Tabaie et al. employed a recurrent neural network in 2467 patients, achieving an AUC of 0.83 for discharge modified Rankin Scale. ⁵ Maldaner et al. developed a complication-aware gradient-boosting model in 1587 cases but without external validation. ⁶ A 2025 systematic review of 8445 pooled participants confirmed that most published ML models enrolled <3000 patients and rarely addressed external validation or interpretability. ⁷

Larger-scale efforts also emerged. De Jong et al. demonstrated an artificial neural network with an AUC of 0.85–0.88 using approximately 450 patients and outperformed SAHIT scoring. ⁸ Zhu et al. leveraged the Cerner Health-Facts EHR to train gradient-boosting models on 6728 admissions (AUC ≈ 0.80 for in-hospital mortality). ⁹ An Australian registry study of 12,070 aSAH cases used ML-based risk stratification to create a clinically deployable scoring system. ¹⁰ CatBoost and deep neural networks have shown superiority over logistic regression in mid-sized cohorts (≈3000 patients), ¹¹ while convolutional-neural-network models using admission CT imaging report AUCs up to 0.84 for early mortality. ¹² Explainable pipelines have also gained traction. Shu et al. applied SHapley Additive exPlanations (SHAP) to highlight modifiable risk factors in high-grade aSAH. ¹³ Nevertheless, addressing sample size considerations, exploring different input variables, and leakage-free model development remain important to reach more robust prognostic modeling.

The present study aimed to analyze a decade of aSAH admissions from 2008 to 2018 using a large nationwide database, the National Inpatient Sample (NIS). We trained and compared ML algorithms for predicting the validated NIS SOM composite endpoint, implemented leakage-free feature selection entirely within cross-validation folds, and included both general and individual-level interpretability. By combining national scale data with a transparent modeling pipeline, our work seeks to provide outcome modeling insights to support prognostication and resource allocation in everyday aSAH care.

Materials and methods

Patient database

The data that support the findings of this study are available from the corresponding author upon reasonable request. The National Inpatient Sample (NIS), developed for the Healthcare Cost and Utilization Project (HCUP), was queried for this study from 2008 to 2018. NIS was created as a resource for studying national estimates of healthcare utilization, cost, and quality outcomes. It represents an approximate 20% stratified sample of all discharges from HCUP-participating hospitals, which covers over 97% of the United States population and includes over 7 million national hospital discharges annually. ¹⁴ As a result of the deidentified nature of the database, this study was exempt from Institutional Review Board (IRB) approval.

Cohort selection

The International Classification of Diseases 9th edition (ICD-9) and ICD 10th edition (ICD-10) were used to identify patients diagnosed with SAH from the database. Patients undergoing coiling or clipping were then identified to distinguish aneurysmal from non-aneurysmal SAH. Patients concurrently diagnosed with arteriovenous malformations (AVMs) or head trauma were excluded. Therefore, SAH resulting from AVMs, dural arteriovenous fistulas, and mycotic aneurysms was excluded. To deal with missing values, a threshold was estimated to retain most variables of interest. These methods were previously published and validated in several studies to discern patients with aSAH.^2,15,16 ICD codes used to identify these procedures can be seen in Supplemental Table 1.

Variable selection

The following demographic variables were recorded: age, sex, race, insurance type, and Elixhauser comorbidity indices. Elixhauser comorbidity indices are based on 31 predefined comorbidities that use ICD-9 and ICD-10 codes to identify risks of poor outcomes. ¹⁷ Uncomplicated and complicated hypertension were combined to limit discrepancies, as the ICD-10 definitions for the two categories are more specific. Current and former smokers were identified by using corresponding ICD-9 and ICD-10 codes highlighted in Supplemental Table 1. The following hospital characteristics data were collected: bed size, location (rural or urban), teaching status, and region (Northeast, Midwest, South, or West). The corresponding ICD codes for each variable included in the NIS-SSS can be found in Supplemental Table 1.

The primary outcome of interest was the NIS-SAH Outcome Measure (NIS-SOM), a previously validated dichotomous variable for assessing a good vs a poor outcome for patients hospitalized with aneurysmal SAH in the NIS. A good outcome was defined as discharge to home or rehabilitation facility and/or hospital. A poor outcome was defined as (a) discharge to a nursing facility, extended care facility, or hospice care; (b) placement of a tracheostomy tube and/or gastrostomy tube; or (c) in-hospital mortality. ⁴

Feature extraction, processing, and selection

The processing and modeling of the collected features were done using Python version 3.8. All subsequent preprocessing steps were performed using parameters derived only from the training set to prevent data leakage. The features were min-max scaled to range between 0 and 1 using scaling parameters computed from the training set. To enhance discriminative capacity and reduce feature space, dimensionality reduction was applied through Maximum Relevance—Minimum Redundancy (MRMR) feature selection, performed exclusively on the training set to select the top 20% of variables. The same training-set-derived features selected were then used for the test set evaluation to avoid information leakage.

Statistical analysis

Demographic and clinical variables were analyzed between the two patient outcome groups using SciPy (version 1.6.2), a scientific computing library, and the Python programming language. Variables were described using means with standard deviations for continuous variables and proportions for categorical variables. A univariate statistical comparison between the two patient groups in relation to poor outcome was performed. Continuous quantitative variables were assessed, according to their distribution normality, using the Student t-test or the Kruskal–Wallis test. Chi-Square test was used to compare the rest of the variables on a categorical basis between the outcome groups. P-values of less than 0.05 were considered statistically significant.

Machine learning modeling and interpretation

Machine learning models were developed to predict poor outcome disposition using the selected features. After splitting the dataset into a training set of 75% and a test set of 25%, the models were trained and validated using a 10-fold cross-validation within the training set. The built benchmark models were as follows: Decision Tree (DT), Gaussian Naïve Bayes (NB), Multilayer Perceptron (MLP), K-Nearest Neighbors (KNN), Random Forest (RF), Bagging Classifier (BAG), Gradient Boosting (BG), and XGBoost (XGB). A grid-search approach was used for hyperparameter tuning. To account for all aspects of model evaluation, performance was measured and compared between the different algorithms using area under the receiver operating characteristic curve (ROC-AUC), accuracy, F1 score, precision (1-specificity), and recall (sensitivity) using the test set. Finally, Shapley Additive exPlanations (SHAP), a game-theory-based method, was used to evaluate model explainability and feature influence on the predictions of the best-performing model.^18,19 General interpretation of the test set predictions, as well as local interpretation of individual observation examples of outcome predictions, were investigated for clinical insight into the model's predictions. Force plots were generated to determine the threshold at which the continuous variable values were in favor of predicting a poor outcome.

Results

Overall, 203,260 observations and 430 variables were initially extracted. Following inclusion of the features of interest and excluding variables with missing data in >80% of observations, a total of 18,149 patients and 57 features were ultimately included in the analysis. Overall, the mean patient age was 55 years, with females accounting for 68.8%, and 53% having a poor short-term outcome.

Univariate analysis was performed to compare the characteristics of patients who experienced a poor outcome. The univariates significantly associated with worse outcome were as follows: age, Elixhauser sum, year, non-elective admission, male sex, Medicare and Medicaid primary payer, transfer in, Midwest and Northeast hospital region, private hospitals, aneurysm clipping, stupor, congestive heart failure, cardiac arrhythmias, valvular disease, pulmonary circulation disorders, hypertension, paralysis, other neurological disorders, chronic pulmonary disease, diabetes, hypothyroidism, renal failure, liver disease, peptic ulcer disease without bleeding, lymphoma, metastatic cancer, solid tumor without metastasis, rheumatoid arthritis/collagen vascular, coagulopathy, obesity, weight loss, fluid and electrolyte disorders, blood loss anemia, drug abuse, white race, and never smoking status. Univariate statistical analysis of inpatients with and without poor outcomes is summarized in Supplemental Table 2.

Following the selection of the most important features with the least redundant information, 12 were retained for machine learning modeling. The different models had an AUC range of (0.74–0.80). MLP was the best performing model, achieving the following metrics predicting poor outcome: AUROC: 0.80, accuracy: 73%, F1-score: 0.75, precision (1-specificity): 0.74, and recall (sensitivity): 0.82. Performance of all evaluated models is provided in Figure 1.

Figure 1.

(A) Receiver operating characteristic curves (B) and evaluation metrics matrix for the prediction of poor outcomes of inpatients with SAH of aneurysmal origin.

SHAP interpretation of the best model performance on the test set produced ordered by feature importance: Age, Elixhauser sum, never smoker status, neurological comorbidity, paralysis, primary payer: Medicare, days from admission to intervention, fluid and electrolyte disorders, weight loss, cardiac arrhythmias, primary payer: self-pay, and congestive heart failure.

SHAP summary plot and heatmap are illustrated in Figure 2. Feature importance rank according to both the mean as well as the maximal absolute SHAP values is summarized in SHAP bar plots shown in Figure 3. For interpretability purposes, force plot as well as local examples of individual observations relating to outcome prediction following SAH events were generated for the best performing model on the test set, as illustrated in Figure 4. Force plot assessing the individual effects of the continuous variables on the best predictive model in the test set is shown in Figure 5, demonstrating progressive shifts to poor outcome prediction in relation to age, Elixhauser sum, and days from admission to intervention. With a mean age progressively transitioning to poor outcome at 54 years, Elixhauser sum transitioning at 2.5 and higher, and days from admission to intervention becoming increasingly predictive of poor outcome at 3 days and higher.

Figure 2.

Interpretation of the best performing predictive model applied to the test set instances. (A) SHAP summary plot with the colors signifying feature value and (B) SHAP heatmaps where the red and blue refer to positive and negative SHAP values.

Figure 3.

SHAP bar plot of feature importance with (A) the mean absolute SHAP value of each feature and (B) the maximal absolute SHAP value of each feature of the test set best performing model.

Figure 4.

(A) General interpretation force plot of the best performing model on the test set with examples of (B) individual local explanations of patients without poor outcome prediction and (C) with poor outcome prediction following SAH event.

Figure 5.

Force plot of the analyzed continuous variables with their individual effects on the best model prediction in the test set, demonstrating progressive transition to poor outcome prediction in relation to (A) age, (B) Elixhauser Sum, and (C) days from admission to intervention.

Discussion

The present study sought to develop machine-learning models to predict poor outcomes in a national cohort of patients admitted with aneurysmal subarachnoid hemorrhage (aSAH). Our best model, a multilayer perceptron (MLP), achieved an AUC of 0.80. In addition, model interpretability analyses identified age, overall comorbidity burden, preexisting neurological disease, paralysis, insurance status, and smoking status as influential features in our cohort. These findings provide preliminary evidence supporting the feasibility of using routinely available data to stratify short-term risk after aSAH.

Our model's discrimination is consistent with recent literature. A nationwide electronic health record (EHR) study of 6728 aSAH admissions reported an AUROC ≈ 0.805 for in-hospital mortality using only the first 24 h of data, closely matching our performance. ⁹ Likewise, a contemporary systematic review and meta-analysis synthesizing 12 studies (8445 patients) found a pooled AUC ≈ 0.82 for ML models predicting poor outcomes after aSAH. ⁷ Single-center efforts have also shown comparable discrimination: in a prospective high-grade aSAH registry, a random-forest model achieved AUC ≈ 0.87 at 12 months, with SHAP analysis highlighting clinical grade, hemorrhage burden on CT, and age as dominant predictors; undergoing endovascular coiling was associated with better outcomes. ¹³ A machine-learning-guided “scorecard” using five variables reported internal AUC ≈ 0.79, with external-cohort accuracy ≈0.79 rather than an external AUC, underscoring the importance of distinguishing between internal and external performance metrics when interpreting model generalizability. ²⁰ At the same time, very high AUROCs (>0.90) occasionally observed in small cohorts should be interpreted with caution; for example, one single-center study (n≈351) reported AUC 0.95, a setting in which overfitting and limited transportability are plausible. ⁹ In aggregate, an AUC in the 0.75–0.85 range appears typical for ML prognostic models in aSAH, and substantially exceeding this range generally requires richer, high-resolution inputs and careful external validation.^7,9

Prior ML work in intracranial aneurysm populations has largely examined post-treatment neurological outcomes, aligning with our focus on clinically meaningful endpoints. A recurrent neural network trained on longitudinal data predicted 90-day modified Rankin Scale (mRS) after aSAH and outperformed simpler temporal models. ⁵ Maldaner et al. built treatment- and complication-aware models to predict discharge mRS. ⁶ Zafar et al., using electronic health data and multivariable modeling, identified several physiologic and care-process features—such as levetiracetam exposure, mechanical ventilation, and white blood cell count—as significant predictors of discharge Glasgow Outcome Scale (GOS). ²¹ In unruptured aneurysm care, both single-center and multicenter studies have shown moderate-to-strong performance for functional outcome prediction after microsurgical or endovascular treatment.^22,23 Taken together, these reports support the potential value of ML for outcome stratification in aneurysm care using data elements that are typically available in routine practice.

Beyond raw performance, interpretability and clinical integration are crucial. We used SHAP values to delineate global and patient-specific drivers of risk—an approach increasingly recognized for translating complex models into clinically digestible explanations.^18,19 Our top features (e.g., age; comorbidity burden via Elixhauser index; paralysis/neurologic deficits; insurance status; smoking) are concordant with known aSAH risk factors and social determinants, lending face validity to the model. The literature echoes this emphasis on explainability. In high-grade aSAH, SHAP analyses have shown that higher WFNS grade, greater hemorrhage burden (modified Fisher score), and older age are strongly associated with poor 12-month outcome, whereas undergoing endovascular coiling tracks with better outcomes. ¹³ In post-coiling cohorts, SHAP has highlighted postoperative GCS, aneurysm size, and age among the leading drivers of 6-month outcomes. ²⁴ In our study, identification of systemic complications (e.g., fluid/electrolyte disorders, weight loss) and hospital-system features (e.g., insurance type, interfacility transfer) alongside neurological status suggests that outcomes may hinge on both clinical severity and care context. Such interpretable insights can guide targeted interventions (e.g., correcting metabolic derangements, expediting transfers) and foster clinician trust by allowing providers to see why a given prediction is high-risk.

Inequity of access-relevant findings merits attention. SHAP analyses from our MLP indicated a higher risk among uninsured patients. While observational and not causal, this aligns with evidence that uninsured or Medicaid patients are less likely to receive post-acute rehabilitation after stroke and traumatic injuries.^25,26 In a national aSAH analysis, Medicaid and uninsured patients had lower odds of non-routine discharge (i.e., more often discharged home) yet longer length of stay than privately insured patients, consistent with barriers to rehabilitation placement despite medical eligibility. ²⁷ Our model also associated Medicare coverage with worse outcomes, which may partly reflect age-related risk; however, our feature-selection process considered age and payer simultaneously, indicating non-redundant contributions from these variables.

Several comorbidities emerged as important predictors. Coexisting neurologic conditions and paralysis were among the top features in our model and are plausibly linked to worse functional recovery after aSAH. Independent clinical and physiologic predictors observed in prior work (e.g., ventilation, leukocytosis, and antiseizure medication exposure) reinforce the salience of acute severity and systemic stress responses in shaping discharge outcomes.^21,28 Age and comorbidity burden—two of our model's strongest signals—are likewise consistently associated with poorer outcomes after aSAH.^17,28 Intriguingly, our SHAP analysis suggested a slightly inverse association between current smoking and poor outcome. While smoking unambiguously increases the risk of aneurysm formation and rupture,^29,30 its relationship to post-hemorrhage outcomes is mixed: some studies report higher rates of vasospasm and delayed neurologic deficits in smokers, whereas others have observed paradoxically similar or even better functional outcomes, potentially due to residual confounding or selection effects.^31–33 These conflicting signals underscore the need for cautious interpretation and, ideally, prospective measurement of granular smoking exposure and cessation timing.

With respect to implementation, our training on a large, diverse, nationwide dataset increases the likelihood of transportability across hospital types and regions; nonetheless, true external validation remains essential. Administrative datasets offer breadth and representation but can lack certain clinical details, and quality/completeness can vary across institutions. ¹⁴ Future work should prospectively validate calibration and discrimination in independent, multi-center cohorts and assess real-world impact when embedded in EHR workflows for early risk stratification. For maximal clinical utility, predictors must be available early in the hospital course, and outputs must be interpretable and actionable. Many of our inputs (demographics, comorbidities, initial exam) are available at or near admission, which is advantageous; however, some features used here (e.g., discharge-coded complications or total length of stay) are not known upfront. Restricting features to prospectively available variables will be important for a real-time tool. Finally, the clinical use case should weigh model complexity and marginal gains in AUC against simpler, validated bedside instruments. Classic clinical grading systems (e.g., Hunt–Hess) and multi-variable tools (e.g., SAHIT) are easy to apply but achieve only moderate discrimination; ML can offer incremental accuracy—especially when leveraging nonlinear interactions—provided that transparency and external validity are maintained.^34–36

Limitations

This study has several important limitations. First, the NIS is an administrative database that relies on ICD coding, therefore inherent risks of miscoding of aSAH, comorbidities, and procedures are possible and granular clinical details (Hunt–Hess grade, aneurysm morphology, location, imaging findings, delayed cerebral ischemia, physiologic variables) are unavailable; consequently, some clinically relevant predictors could not be modeled, and residual confounding may remain. Second, our composite outcome (NIS-SOM) captures in-hospital mortality or severe discharge disability only, providing no insight into functional recovery beyond discharge; future work should extend follow-up to 3–6 months. Third, all data came from the same source (US hospitals, 2008–2018) and era; therefore, external validation on contemporary multi-center cohorts is essential to confirm generalizability, calibration, and fairness across subgroups, especially given the temporal drift and how treatment approaches and devices have since evolved. Prospective deployment studies are needed to monitor real-world performance and bias. Finally, predictors identified by SHAP are associative rather than causal and may offer limited direct clinical actionability, underscoring the importance of combining interpretable ML outputs with clinician judgment and of refining models that incorporate prospectively available, physiologic and imaging features for real-time decision support.

Conclusion

Leveraging a decade of nationwide admissions, we show that interpretable machine-learning models can predict short-term poor outcomes after aneurysmal SAH with solid discrimination (AUC 0.80) while maintaining transparency through SHAP-based explanations. The analysis confirms well-known clinical predictors (age, neurological comorbidity, paralysis) and exposes health-system factors (insurance status, transfer patterns) that traditional scores often overlook. Because all input features are routinely available in administrative or electronic records, the model could be integrated into hospital dashboards to flag high-risk patients for early neuro-critical-care escalation or targeted rehabilitation planning. Future work should focus on prospective, multi-institutional validation, incorporation of imaging and physiologic time-series data for even richer prediction, and assessment of whether ML-guided interventions translate into measurable improvements in functional recovery and equity of care.