Machine learning–based prediction of 3–6-month post-stroke cognitive impairment using acute-phase clinical data: a two-center retrospective prognostic modeling study

Abstract

Background

Post-stroke cognitive impairment (PSCI) is common and disabling, but identifying patients at risk early remains difficult. We developed and externally validated a retrospective machine-learning prognostic model using acute-phase predictors collected within 1 week after stroke onset to estimate PSCI risk assessed at 3–6 months after stroke.

Methods

Two inpatient rehabilitation cohorts were retrospectively analyzed: Center 1 (n = 1,369; training/internal test) and Center 2 (n = 341; external validation). Cognitive status at 3–6 months was the outcome. Features were selected with Boruta and LASSO; eight algorithms were trained with 10-fold cross-validation. Performance was assessed by AUROC, average precision (AP), accuracy, sensitivity, specificity, precision, F1, calibration (Brier score, calibration intercept, calibration slope and plots), and decision-curve analysis (DCA). SHAP was used for interpretability, and a web-based proof-of-concept prototype was developed to display model-derived risk estimates.

Results

Ten predictors were ultimately selected. In external validation, XGBoost achieved the highest AUROC (0.827), with an average precision of 0.6668, accuracy of 0.8328, sensitivity of 0.6250, specificity of 0.9241, precision of 0.7831, and an F1-score of 0.6952. Calibration was acceptable (Brier score = 0.1366; calibration intercept = − 0.027; calibration slope = 0.916), and DCA suggested positive net benefit within the 15%–40% threshold-probability range. The largest SHAP contributors to the XGBoost model output included diabetes, NIHSS, Hgb, Hcy, apoA1, TG, ALB, WMH, smoking, and age.

Conclusion

This model may support PSCI risk stratification in post-stroke patients admitted for inpatient rehabilitation. However, further validation in broader and more representative stroke populations is required before wider application.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s12911-026-03703-x.

Introduction

Stroke remains a major global health challenge and a leading cause of long-term disability [1]. Cognitive impairment is one of the most common and disabling consequences of stroke. Post-stroke cognitive impairment (PSCI), defined as new-onset cognitive dysfunction within 3–6 months after stroke, includes cognitive impairment without dementia and post-stroke dementia (PSD) [2]. Its reported prevalence varies widely across populations and stroke subtypes [3, 4]. PSCI is associated with poorer quality of life, a higher risk of recurrent stroke, shorter survival, and progression to post-stroke dementia [5–9]. Early risk stratification and targeted monitoring are important because cognitive symptoms may be subtle and easily overlooked. PSCI has a multifactorial basis. Previous studies have linked its development to pre-stroke cognitive status, demographic and vascular factors, including age, educational attainment, hypertension, diabetes, and atrial fibrillation [10–13]. Stroke severity, lesion characteristics, white matter abnormalities, and recurrent stroke have also been associated with subsequent cognitive impairment [14]. In addition, metabolic and inflammatory biomarkers may provide further predictive information [15–17]. These factors often coexist and interact, making individual risk difficult to estimate.

Machine learning (ML) is increasingly used in clinical medicine [18], including stroke imaging, rehabilitation prognostication, and early diagnosis [19]. Previous studies have developed machine-learning models for PSCI using demographic, clinical, laboratory, and imaging data, with reported AUROC values ranging from 0.73 to 0.93 [20–24]. Systematic reviews and meta-analyses have also examined ML models for PSCI prediction [25]. However, many of these studies were based on modest single-center samples, included only a limited range of routinely available laboratory variables, or lacked independent external validation. Evidence regarding model calibration, clinical net benefit, interpretability, and potential implementation also remains limited.

To address these gaps, we conducted a two-center retrospective prognostic modeling study to develop and externally validate machine-learning models that use acute-phase clinical, laboratory, and imaging predictors collected within 1 week after stroke onset to estimate PSCI risk assessed at 3–6 months after stroke. We further evaluated model discrimination, calibration, clinical net benefit, and interpretability.

Methods

Study design and population

This was a two-center retrospective prognostic modeling study. The prediction target was PSCI status assessed at rehabilitation admission 3–6 months after stroke onset. Candidate predictors were restricted to variables available during the acute-phase hospitalization, defined as within 1 week after stroke onset, including demographic characteristics, stroke-related clinical features, laboratory tests, and imaging variables. These predictors were extracted retrospectively from electronic medical records of the index acute stroke hospitalization. Therefore, the temporal structure of the model was defined as follows: acute-phase variables collected within 1 week after stroke onset were used to estimate later PSCI status assessed at 3–6 months after stroke. No information obtained at the 3–6-month cognitive assessment, except the outcome itself, was used as a predictor.

The model training and internal test cohorts were derived from Center 1, comprising patients hospitalized in the Department of Rehabilitation Medicine at Shijiazhuang People’s Hospital between January 2021 and December 2024. The external validation cohort was derived from Center 2, comprising contemporaneous inpatients admitted to Shijiazhuang Hospital of Traditional Chinese Medicine during the same period. The study was conducted in accordance with the Declaration of Helsinki, was approved by the ethics committees of both institutions, and written informed consent was obtained from all participants.

Inclusion criteria were as follows: (1) age ≥ 18 years; (2) admission for inpatient rehabilitation with a principal diagnosis of ischemic stroke or intracerebral hemorrhage, confirmed by brain MRI or CT; and (3) a time interval of 3–6 months since stroke onset at the time of rehabilitation admission.

Exclusion criteria were as follows: (1) cognitive impairment attributable to other etiologies (e.g., Alzheimer’s disease, Parkinson’s disease, brain tumor, metabolic or infectious disorders); (2) severe dysfunction of major organs, including advanced cardiac, hepatic, or renal failure; (3) active malignancy; (4) inability to undergo valid cognitive assessment due to psychiatric disorders, severe aphasia or dysarthria, or severe visual or hearing impairment; (5) pregnancy or lactation; (6) current use of psychotropic medications likely to substantially affect cognitive function; and (7) incomplete clinical information, including missing laboratory tests or imaging data.

Because all participants were recruited from inpatient rehabilitation settings 3–6 months after stroke onset, the study cohort represented a rehabilitation-enriched post-stroke population rather than an unselected general stroke population.

Cognitive status assessment

After enrollment, cognitive status at 3–6 months post-stroke was evaluated using the Montreal Cognitive Assessment (MoCA) and the Mini-Mental State Examination (MMSE). Assessments were administered by trained cognitive rehabilitation therapists using a standardized protocol in a quiet room. Testing was discontinued if emotional distress, agitation, or poor cooperation prevented a valid assessment. MoCA scores were corrected by adding 1 point for participants with ≤ 12 years of education, and an education-corrected score < 26 was considered abnormal [26]. The MMSE was interpreted using education-adjusted Chinese cutoffs: ≤17 for illiterate participants, ≤ 20 for those with primary school education, and ≤ 24 for those with junior high school education or above [27]. Participants with normal results on both tests were classified as having post-stroke normal cognition (PSNC), whereas an abnormal result on either test was sufficient for classification as post-stroke cognitive impairment (PSCI). Therefore, when MoCA and MMSE classifications were discordant, impairment on either test was sufficient for PSCI classification, and neither test overruled the other.

PSCI was modeled as a binary composite outcome. We did not separately classify PSCI without dementia and post-stroke dementia (PSD), because standardized dementia diagnosis, functional-dependence criteria, and domain-specific neuropsychological data were not consistently available for all participants in this retrospective dataset. Thus, the model predicts MoCA/MMSE-defined PSCI but does not distinguish PSCI severity subtypes.

Predictor variables and data collection

For enrolled patients, acute-phase data were retrospectively obtained through patient interviews and review of electronic medical records from the index acute stroke hospitalization. Acute-phase predictors were defined as variables recorded within 1 week after stroke onset. Demographic and medical history variables were extracted from admission records, structured medical history documentation, medication records, and clinician-recorded diagnoses. Smoking and drinking were coded as present if current or previous regular smoking or alcohol consumption was documented. Stroke-related clinical variables, including stroke type, time from stroke onset to acute-phase assessment, and the National Institutes of Health Stroke Scale (NIHSS) score, were obtained from acute hospitalization records; when repeated NIHSS assessments were available, the earliest documented NIHSS score within the acute-phase window was used. Laboratory variables were extracted from routine blood tests performed within 1 week after stroke onset; when multiple measurements were available, the first available result within this window was used rather than peak or nadir values. Imaging variables, including white matter hyperintensities (WMH), were obtained from routine cranial CT and/or MRI reports during the acute hospitalization. WMH was coded as a binary presence/absence variable based on radiologic descriptions of white matter hyperintensities, leukoaraiosis, or chronic ischemic white-matter changes. No information collected at the 3–6-month cognitive assessment, except the outcome itself, was used as a predictor.

A total of 65 candidate predictors were extracted, encompassing demographic, stroke-related clinical features, laboratory, and imaging variables. Demographic variables included sex, age, height, weight, body mass index (BMI), and educational attainment. Stroke-related clinical features included the acute-phase National Institutes of Health Stroke Scale (NIHSS) score, stroke type, time from symptom onset to acute-phase assessment, smoking, drinking, hypertension, diabetes, coronary heart disease, history of stroke, atrial fibrillation, and liver or kidney diseases.

Laboratory variables were grouped as follows:

(1) Hematology — red blood cell count (RBC), hemoglobin (Hgb), white blood cell count (WBC), neutrophil count (NE), lymphocyte count (LY), and platelet count (PLT). (2) Coagulation — prothrombin time (PT), activated partial thromboplastin time (APTT), international normalized ratio (INR), fibrinogen (FIB), thrombin time (TT), and D-dimer. (3) Liver function — albumin (ALB), alanine aminotransferase (ALT), aspartate aminotransferase (AST), γ-glutamyl transferase (GGT), alkaline phosphatase (ALP), cholinesterase (CHE), α-L-fucosidase (AFU), total bile acids (TBA), total bilirubin (TBIL), direct bilirubin (DBIL), and indirect bilirubin (IBIL). (4) Renal function — urea, creatinine (CREA), and uric acid (UA). (5) Myocardial/skeletal muscle injury markers — lactate dehydrogenase (LDH), α-hydroxybutyrate dehydrogenase (HBDH), creatine kinase (CK), and creatine kinase-MB (CK-MB). (6) Metabolic indices — homocysteine (Hcy), glucose (GLU), total cholesterol (TC), triglycerides (TG), high-density lipoprotein cholesterol (HDL-C), low-density lipoprotein cholesterol (LDL-C), apolipoprotein A-I (apoA1), apolipoprotein B (ApoB), apolipoprotein E (ApoE), lipoprotein(a) [Lp(a)], adenosine deaminase (ADA), and β2-microglobulin (β2-MG). (7) Inflammatory markers — C-reactive protein (CRP), procalcitonin (PCT), and erythrocyte sedimentation rate (ESR). Imaging variables included stroke type (ischemic or hemorrhagic) and the presence of white matter hyperintensities (WMH) on cranial MRI or CT [28, 29].

Missing data handling and sensitivity analysis

Before complete-case exclusion, missingness across the 65 candidate predictors was summarized among otherwise eligible patients, because missing data are common in machine-learning prediction studies and complete-case deletion may introduce bias and reduce statistical power [30]. The missing-data assessment included the overall number and proportion of patients with missing predictor values, variable-level missingness, and the patient-level number of missing predictors. Complete cases were defined as patients with non-missing values for all candidate predictors. The primary analysis was conducted using the complete-case cohort.

To evaluate whether complete-case exclusion might have biased model development or performance estimation, we further performed a random forest–based imputation sensitivity analysis. Patients with one or more missing predictor values were included where applicable after imputation, whereas patients without valid cognitive outcome classification were not included. Missing values were imputed using the missForest algorithm, with prespecified categorical variables treated as factors and continuous variables as numeric. Imputation was performed with a fixed random seed, a maximum of 10 iterations, and 500 trees, and variable-wise out-of-bag errors were recorded [31]. After generating the imputed dataset, we repeated the same Boruta→LASSO feature-selection strategy, model-training procedure, hyperparameter tuning framework, and evaluation metrics. The results from the random forest imputation–based sensitivity analysis were then compared with those from the primary complete-case analysis to assess the robustness of the main findings.

Screening of variables

A two-stage feature-selection strategy consisting of Boruta followed by LASSO logistic regression was used to reduce irrelevant and redundant predictors, mitigate multicollinearity, and obtain a parsimonious feature set. Boruta was used as an all-relevant method capable of retaining predictors with potential nonlinear or interaction-related signals, whereas LASSO was used to further reduce redundant features and model complexity. All feature-selection procedures were performed exclusively in the training cohort. The internal test and external validation cohorts were not used for feature selection, hyperparameter tuning, threshold selection, or model fitting. Boruta and LASSO were applied once to the full training cohort and were not repeated within each cross-validation fold used for subsequent model tuning. Model performance was therefore evaluated primarily in the independent internal test and external validation cohorts. The outcome was coded as a binary factor, with PSCI as the positive class. Prespecified categorical predictors were encoded as factors, whereas the remaining variables were treated as continuous predictors.

Boruta [32] was first applied with maxRuns = 350 and ntree = 1500. Tentative features were resolved using the TentativeRoughFix procedure. Variables retained by Boruta were entered into a Least Absolute Shrinkage and Selection Operator (LASSO) logistic regression model [33] for further refinement. The LASSO penalty parameter was selected using 10-fold cross-validation based on binomial deviance. Two candidate feature sets were obtained using λ_min, which minimized the cross-validated deviance, and λ_1se, which represented the most parsimonious solution within one standard error of the minimum. Both feature sets were evaluated using the same downstream model-development and validation pipeline. When predictive performance was similar, the smaller feature set was preferred to reduce model complexity, improve interpretability, and facilitate potential implementation.

Model construction and verification

After feature selection, models were trained, internally tested, and externally validated. The Center 1 cohort served as the development dataset and was divided into training and internal test cohorts by stratified random sampling according to PSCI status. The random seed was set to 2025 to ensure reproducibility. The split was fixed once and was not repeatedly resampled. The final allocation was 855 patients in the training cohort and 514 patients in the internal test cohort, corresponding to an approximate 6:4 split. The ratio was chosen to retain enough data for feature selection and tuning while preserving a separate internal test cohort.

Because PSCI accounted for approximately 30% of the development cohort, stratified sampling was used to maintain a similar outcome distribution and an adequate number of PSCI cases in both cohorts. Class imbalance was addressed through the study design and performance evaluation rather than artificial resampling. Oversampling, undersampling, SMOTE, and class-weighted loss were not used in the primary analysis, thereby preserving the observed outcome prevalence and probability calibration. AUROC was used as the primary criterion for hyperparameter tuning, and average precision was additionally evaluated because it is informative under class imbalance. Threshold-dependent performance was assessed separately using sensitivity, specificity, precision, F1-score, and balanced accuracy, as described in the model-performance section. The Center 2 cohort was kept separate for external evaluation and was not used for preprocessing, feature selection, hyperparameter tuning, threshold derivation, or model fitting.

Eight ML algorithms were developed to predict PSCI: Logistic Regression (LR) [34], K-Nearest Neighbors (KNN) [35], Decision Tree (DT) [36], Random Forest (RF) [37], Support Vector Machine (SVM) [38], Neural Network (NN) [39], Extreme Gradient Boosting (XGBoost) [40], and Light Gradient Boosting Machine (LightGBM) [41]. A standardized preprocessing pipeline was incorporated into model development to ensure comparability and minimize information leakage. The outcome was encoded as a binary factor, with PSCI designated as the positive class. Categorical predictors were treated as factors and converted to dummy variables when numeric matrix inputs were required, using levels defined in the training cohort and fixed for application to the internal test and external validation cohorts. Continuous predictors were retained on their original scale for DT, RF, XGBoost, and LightGBM, whereas they were centered and scaled to zero mean and unit variance for KNN, SVM, and NN.

All preprocessing and hyperparameter tuning were conducted using the training cohort only. During 10-fold cross-validation, centering and scaling parameters were estimated separately within each training fold and applied to the corresponding held-out fold. Hyperparameters were optimized by grid search using AUROC for the positive PSCI class as the primary selection criterion. Models implemented through the caret framework used 10-fold cross-validation with class probabilities enabled, whereas LightGBM used stratified cross-validation with early stopping to determine the optimal number of boosting iterations. LR was fitted as an unregularized generalized linear model with a binomial distribution and logit link and therefore required no hyperparameter tuning. After tuning, the final preprocessing parameters were re-estimated using the full training cohort and fixed for application to the internal test and external validation cohorts. Detailed preprocessing procedures, tuning strategies, and final model specifications are provided in Supplementary Table 1.

After the optimal hyperparameter configuration was selected, each final model was refitted using the full training cohort. The fitted models were then evaluated in the independent internal test cohort to assess discrimination, calibration, and overall predictive performance, followed by external validation in the independent Center 2 cohort. No internal test or external validation data were used to estimate preprocessing parameters, select features, tune hyperparameters, derive operating thresholds, or fit the models.

Model performance, selection, and interpretation

Model performance was evaluated in the internal test and external validation cohorts. AUROC was used as the primary threshold-independent metric to compare model discrimination and assess generalizability. Average precision (AP), Brier score, calibration intercept, calibration slope, calibration plots, and decision-curve analysis (DCA) were also evaluated. In addition, threshold-dependent metrics, including accuracy, sensitivity, specificity, precision, negative predictive value (NPV), F1-score, balanced accuracy, and confusion-matrix components, were evaluated.

AUROC summarized threshold-independent discrimination, and AP summarized precision–recall performance under class imbalance. Accuracy represented the overall proportion of correct classifications; sensitivity and specificity reflected the correct identification of PSCI and PSNC, respectively; precision and negative predictive value represented the reliability of positive and negative predictions; the F1-score summarized the balance between precision and sensitivity; and balanced accuracy was the mean of sensitivity and specificity. The Brier score assessed overall probabilistic accuracy, with lower values indicating better performance, while calibration was evaluated using calibration plots, calibration intercept, and calibration slope. Model selection was based primarily on external-validation AUROC, while operating-threshold analyses were used to define clinically relevant use points. AP, calibration, Brier score, DCA net benefit, interpretability, and implementation feasibility were considered as secondary criteria. When discrimination was similar, sensitivity–specificity trade-offs, false-positive burden, and performance at the sensitivity-oriented threshold were also evaluated.

Threshold-dependent performance was evaluated at three operating points: the conventional fixed threshold of 0.50, a training-derived Youden-index threshold, and a sensitivity-oriented screening threshold. The latter two thresholds were derived exclusively from the training cohort and fixed before evaluation in the internal test and external validation cohorts. The sensitivity-oriented threshold was selected to achieve a training sensitivity of at least 0.80 while retaining the highest available specificity. This reflected the intended use of the model for early risk stratification rather than definitive diagnosis, for which reducing missed PSCI cases was considered clinically important.

To assess regional transportability, each tuned model was applied to the independent external validation cohort using fixed hyperparameters and operating thresholds, without further recalibration. DCA compared the net benefit of model-guided risk stratification with treat-all and treat-none strategies. The clinical decision context was whether high-risk classification should prompt enhanced cognitive monitoring, structured cognitive assessment, early cognitive-rehabilitation planning, optimization of modifiable risk factors, and closer follow-up. Because these follow-up actions are low risk and the model was intended for screening, DCA was interpreted over threshold probabilities of 15%–40%.

Shapley Additive Explanations (SHAP) were used to interpret the final implementation model. SHAP values were calculated at both the global and individual levels to quantify the direction and magnitude of each predictor’s contribution to the model output. Global SHAP summaries described overall feature contributions and effect patterns, whereas individual SHAP profiles illustrated patient-level prediction attribution. SHAP values were interpreted strictly as post hoc explanations of model predictions and not as estimates of causal effects, biological priority, or independent etiological importance. Finally, a browser-based proof-of-concept prototype was developed using an interactive R Shiny application to demonstrate how individualized PSCI risk estimates and SHAP-based explanations could be presented.

Statistical analysis

All analyses were performed in R (version 4.4.2) for data processing, statistical analysis, and machine-learning modeling. Continuous variables were summarized as mean ± standard deviation, and categorical variables as counts and percentages, n (%). Between-group comparisons for continuous variables used either the Student’s t test or the Wilcoxon rank-sum (Mann–Whitney U) test, as appropriate. Categorical variables were compared using the chi-square test or Fisher’s exact test. For comparisons across three groups, one-way analysis of variance (ANOVA) with Tukey’s post hoc tests was applied when normality and homoscedasticity assumptions were satisfied; otherwise, the Kruskal–Wallis test with Dunn’s post hoc comparisons was used. All tests were two-sided, and P < 0.05 was considered statistically significant.

Results

Characteristics of patients

From January 2021 to December 2024, 4,064 post-stroke inpatients admitted for inpatient rehabilitation at Center 1 were screened. According to the prespecified inclusion and exclusion criteria, 2,695 patients were excluded, including 1,638 patients with an ineligible time interval from stroke onset, 558 with incomplete clinical information, 244 unable to complete cognitive testing, 204 with psychotropic medication use, 28 with Alzheimer’s disease or Parkinson’s disease, and 23 with malignancy. The remaining 1,369 eligible patients were included as the development cohort. According to the predefined MoCA/MMSE-based decision rule, 420 patients were classified as having PSCI and 949 as having post-stroke normal cognition (PSNC).

At the independent external center, 1034 post-stroke inpatients admitted for inpatient rehabilitation at Center 2 during the same period were screened. Of these, 693 patients were excluded, including 219 patients with an ineligible time interval from stroke onset, 274 with incomplete clinical information, 154 unable to complete cognitive testing, 31 with psychotropic medication use, 7 with Alzheimer’s disease or Parkinson’s disease, and 8 with malignancy. Finally, 341 eligible patients were included as the external validation cohort, including 104 patients with PSCI and 237 with PSNC.

Before complete-case exclusion, 2,542 otherwise eligible patients were assessed for missingness across the 65 candidate predictors. Among them, 1,710 patients had complete data for all candidate predictors and constituted the complete-case cohort for the primary analysis, whereas 832 patients had at least one missing predictor value. The overall complete-case rate was 67.27%, and 32.73% of patients had at least one missing predictor. Missingness was observed in 38 of the 65 candidate predictors. The highest variable-level missingness was observed for adenosine deaminase (ADA; 16.01%), followed by cholinesterase (CHE; 15.89%), LDH (15.77%), TBA (15.74%), and TBIL (15.70%). The detailed variable-level missingness and patient-level distribution of the number of missing predictors are provided in Supplementary Tables 2–4.

The development cohort was then randomly divided into a training cohort (n = 855) and an internal test cohort (n = 514) using a fixed stratified random split by PSCI status. The study flow is shown in Fig. 1. An additional 341 patients from an independent second center were included as the external validation cohort. Baseline demographic characteristics, stroke-related clinical features, laboratory variables, and imaging findings across the training, internal test, and external validation cohorts are summarized in Tables 1 and 2. The training and internal test cohorts had similar baseline characteristics after stratified random splitting.

Fig. 1.

Study workflow for development, internal testing, and external validation of machine-learning models for PSCI prediction. Post-stroke inpatients from Center 1 were screened according to the predefined inclusion and exclusion criteria and then divided into training and internal testing cohorts. Patients from Center 2 were used as an independent external validation cohort. Feature selection was performed in the training cohort using Boruta and LASSO, followed by model development with eight machine-learning algorithms. Model performance was evaluated in the internal testing and external validation cohorts using discrimination, precision–recall performance, calibration, decision-curve analysis, and confusion-matrix-based metrics. The final model was further interpreted using SHAP analysis and implemented as a web-based proof-of-concept prototype. PSCI, post-stroke cognitive impairment; PSNC, post-stroke normal cognition; LASSO, least absolute shrinkage and selection operator; SHAP, Shapley additive explanations

Table 1.

Demographic and clinical characteristics across the training, internal test, and external validation cohorts

Variables		Modeling cohort (n = 1369)			P values
		Training cohort (n = 855)	Testing cohort (n = 514)	Validation cohort (n = 341)
Gender, n (%)	Male	619 (72.40%)	349 (67.90%)	235 (68.91%)	0.087Δ/0.171#
	Female	236 (27.60%)	165 (32.10%)	106 (31.09%)
Stroke subtype (%)	Infarction	558 (65.26%)	347 (67.51%)	243 (71.26%)	0.429Δ/0.134#
	Hemorrhage	297 (34.74%)	167 (32.49%)	98 (28.74%)
Education, n (%)	Primary school	361 (42.22%)	239 (46.50%)	154 (45.16%)	0.288Δ/0.502#
	Secondary school	393 (45.96%)	216 (42.02%)	143 (41.94%)
	University or above	101 (11.81%)	59 (11.48%)	44 (12.90%)
Age (year)		60.21 ± 13.25	60.91 ± 13.63	60.87 ± 12.73	0.361Δ/0.600#
Height (cm)		166.00 ± 8.55	165.00 ± 8.63	165.00 ± 8.39	0.078Δ/0.211#
Weight (kg)		70.23 ± 11.44	69.16 ± 11.60	70.02 ± 11.23	0.093Δ/0.234#
BMI (kg/m2)		25.38 ± 1.88	25.24 ± 1.91	25.42 ± 1.77	0.132Δ/0.313#
Duration of stroke (days)		136.24 ± 25.91	133.65 ± 25.98	135.73 ± 25.00	0.074Δ/0.189#
Drinking, n (%)		302 (35.32%)	206 (40.08%)	109 (31.96%)	0.088Δ/0.043#
Smoking, n (%)		489 (57.19%)	277 (53.89%)	192 (56.30%)	0.256Δ/0.488#
Hypertension, n (%)		729 (85.26%)	425 (82.68%)	294 (86.22%)	0.233Δ/0.298#
Diabetes mellitus, n (%)		278 (32.51%)	163 (31.71%)	114 (33.43%)	0.804Δ/0.870#
Coronary heart disease, n (%)		178 (20.82%)	126 (24.51%)	67 (19.65%)	0.127Δ/0.163#
History of stroke, n (%)		137 (16.02%)	82 (15.95%)	46 (13.49%)	0.912Δ/0.519#
White matter hyperintensities, n (%)		149 (17.43%)	108 (21.01%)	56 (16.42%)	0.116Δ/0.152#
Hyperlipidemia, n (%)		444 (51.93%)	253 (49.22%)	165 (48.39%)	0.360Δ/0.441#
Atrial fibrillation, n (%)		46 (5.38%)	35 (6.81%)	18 (5.28%)	0.334Δ/0.495#
Kidney disease, n (%)		11 (1.29%)	7 (1.36%)	7 (2.05%)	0.992Δ/0.593#
Liver disease, n (%)		47 (5.50%)	37 (7.20%)	18 (5.28%)	0.249Δ/0.365#
NIHSS score		12.38 ± 2.73	12.19 ± 2.78	12.04 ± 2.62	0.199Δ/0.092#

PSCI, post-stroke cognitive impairment; PSNC, post-stroke normal cognition; BMI, body mass index; NIHSS, National Institutes of Health Stroke Scale; Duration of stroke denotes the number of days from index stroke onset to the baseline assessment. ^ΔTraining cohort vs. Testing cohort. ^#Training cohort vs. Testing cohort vs. Validation cohort

Table 2.

Laboratory characteristics across the training, internal test, and external validation cohorts

Laboratory test parameters	Modeling cohort (n = 1369)			P values
	Training cohort (n = 855)	Testing cohort (n = 514)	Validation cohort (n = 341)
RBC (1012/L)	4.40 ± 0.72	4.37 ± 0.71	4.32 ± 0.66	0.411Δ/0.227#
Hgb (g/L)	130.12 ± 12.87	129.15 ± 13.03	128.43 ± 12.22	0.170Δ/0.084#
WBC (109/L)	8.04 ± 2.92	7.83 ± 2.90	8.14 ± 2.88	0.179Δ/0.252#
LY (109/L)	1.79 ± 0.66	1.74 ± 0.67	1.81 ± 0.65	0.247Δ/0.302#
NE (109/L)	5.06 ± 1.72	4.90 ± 1.74	5.03 ± 1.68	0.094Δ/0.236#
PLT (109/L)	253.01 ± 86.72	245.96 ± 87.57	254.72 ± 85.58	0.140Δ/0.238#
PT (s)	11.69 ± 1.33	11.75 ± 1.33	11.74 ± 1.34	0.447Δ/0.702#
INR	1.10 ± 0.17	1.09 ± 0.17	1.10 ± 0.17	0.079Δ/0.188#
APTT (s)	32.06 ± 3.45	31.97 ± 3.55	32.17 ± 3.41	0.629Δ/0.698#
FIB (g/L)	4.23 ± 1.07	4.16 ± 1.07	4.15 ± 1.08	0.288Δ/0.389#
TT (s)	13.59 ± 2.00	13.40 ± 2.03	13.58 ± 2.01	0.103Δ/0.227#
D-dimer (ng/mL)	429.97 ± 227.15	429.94 ± 219.48	426.79 ± 224.46	0.982Δ/0.963#
Hcy (mol/L)	14.38 ± 3.91	14.12 ± 3.92	13.77 ± 3.81	0.230Δ/0.042#
ALB (g/L)	35.55 ± 4.28	35.31 ± 4.28	34.84 ± 3.96	0.310Δ/0.055#
ALT (U/L)	45.59 ± 20.04	44.32 ± 20.26	45.63 ± 20.26	0.265Δ/0.487#
AST (U/L)	35.09 ± 14.11	34.03 ± 14.40	34.95 ± 13.94	0.190Δ/0.397#
GGT (U/L)	39.38 ± 17.92	37.80 ± 18.02	40.39 ± 18.06	0.113Δ/0.093#
ALP (U/L)	87.79 ± 24.33	85.27 ± 24.77	85.70 ± 24.06	0.065Δ/0.134#
CHE (U/L)	8499.65 ± 1772.08	8379.45 ± 1751.60	8226.40 ± 1605.33	0.235Δ/0.046#
AFU (U/L)	20.59 ± 11.30	19.64 ± 11.38	20.14 ± 10.95	0.133Δ/0.319#
TBA (mol/L)	5.15 ± 2.86	4.88 ± 2.90	5.10 ± 2.81	0.098Δ/0.242#
TBIL (mol/L)	16.66 ± 4.86	16.20 ± 4.93	16.57 ± 4.77	0.094Δ/0.236#
DBIL (mol/L)	6.10 ± 1.72	5.94 ± 1.74	6.07 ± 1.68	0.092Δ/0.231#
IBIL (mol/L)	11.17 ± 4.01	10.80 ± 4.06	11.10 ± 3.93	0.098Δ/0.245#
GLU (mmol/L)	6.59 ± 1.56	6.47 ± 1.62	6.59 ± 1.51	0.195Δ/0.376#
Urea (mmol/L)	6.10 ± 1.77	5.97 ± 1.78	5.86 ± 1.66	0.179Δ/0.080#
CREA (mol/L)	55.66 ± 15.97	54.66 ± 15.59	54.19 ± 14.80	0.228Δ/0.229#
URIC (mol/L)	305.38 ± 121.90	301.92 ± 120.63	297.35 ± 120.01	0.606Δ/0.564#
LDH (U/L)	201.64 ± 45.76	197.34 ± 46.40	200.85 ± 44.91	0.093Δ/0.235#
HBDH (U/L)	128.73 ± 31.50	125.79 ± 31.95	128.21 ± 30.85	0.096Δ/0.238#
CK (U/L)	117.53 ± 60.09	111.91 ± 60.95	116.49 ± 58.93	0.096Δ/0.239#
CK-MB (U/L)	2.56 ± 1.43	2.42 ± 1.45	2.53 ± 1.40	0.095Δ/0.238#
TC (mmol/L)	3.59 ± 1.43	3.47 ± 1.44	3.45 ± 1.38	0.119Δ/0.152#
TG (mmol/L)	1.89 ± 0.70	1.88 ± 0.68	1.77 ± 0.67	0.835Δ/0.060#
HDL-C (mmol/L)	1.10 ± 0.18	1.09 ± 0.18	1.08 ± 0.18	0.568Δ/0.380#
LDL-C (mmol/L)	2.35 ± 0.71	2.32 ± 0.70	2.24 ± 0.65	0.383Δ/0.050#
apoA1 (g/L)	1.22 ± 0.23	1.21 ± 0.23	1.19 ± 0.22	0.581Δ/0.095#
ApoB (g/L)	0.96 ± 0.15	0.95 ± 0.15	0.96 ± 0.15	0.152Δ/0.354#
ApoE (mg/L)	48.36 ± 12.56	47.45 ± 12.66	46.96 ± 11.88	0.202Δ/0.164#
LP(a) (mg/L)	275.04 ± 116.64	275.62 ± 117.43	268.96 ± 114.10	0.987Δ/0.636#
ADA (U/L)	14.41 ± 5.11	14.11 ± 5.10	14.33 ± 4.96	0.283Δ/0.566#
β2-MG (g/ml)	2.43 ± 0.79	2.42 ± 0.79	2.39 ± 0.76	0.733Δ/0.631#
CRP (mg/L)	8.64 ± 4.19	8.88 ± 4.30	8.35 ± 4.35	0.310Δ/0.206#
PCT (ng/ml)	0.25 ± 0.14	0.24 ± 0.14	0.23 ± 0.13	0.180Δ/0.072#
ESR (mm/h)	17.01 ± 8.30	16.85 ± 7.97	17.12 ± 8.40	0.761Δ/0.915#

PSCI, post-stroke cognitive impairment; PSNC, post-stroke normal cognition. RBC, red blood cell count; Hgb, hemoglobin; WBC, white blood cell count; LY, lymphocyte count; NE, neutrophil count; PLT, platelet count; PT, prothrombin time; INR, international normalized ratio; APTT, activated partial thromboplastin time; FIB, fibrinogen; TT, thrombin time; Hcy, homocysteine; ALB, albumin; ALT, alanine aminotransferase; AST, aspartate aminotransferase; GGT, γ-glutamyl transferase; ALP, alkaline phosphatase; CHE, (serum) cholinesterase; AFU, α-L-fucosidase; TBA, total bile acid; TBIL, total bilirubin; DBIL, direct bilirubin; IBIL, indirect bilirubin; GLU, glucose; CREA, creatinine; URIC, uric acid; LDH, lactate dehydrogenase; HBDH, α-hydroxybutyrate dehydrogenase; CK, creatine kinase; CK-MB, creatine kinase-MB; TC, total cholesterol; TG, triglyceride; HDL-C, high-density lipoprotein cholesterol; LDL-C, low-density lipoprotein cholesterol; apoA1, apolipoprotein A-I; ApoB, apolipoprotein B; ApoE, apolipoprotein E; LP(a), lipoprotein(a); ADA, adenosine deaminase; β2-MG, β2-microglobulin; CRP, C-reactive protein; PCT, procalcitonin; ESR, erythrocyte sedimentation rate. ^ΔTraining cohort vs. Testing cohort. ^#Training cohort vs. Testing cohort vs. Validation cohort

We also compared the combined development cohort with the external validation cohort. These comparisons are provided in Supplementary Tables 5 and 6. Most demographic and clinical variables were similar between cohorts, although Hcy, ALB, CHE, TG, LDL-C, and apoA1 differed significantly, suggesting a degree of real-world between-center heterogeneity. Therefore, the external validation analysis was interpreted as an assessment of regional transportability rather than validation in a fully homogeneous population. Detailed comparisons stratified by PSCI versus PSNC are provided in Supplementary Tables 7–10. To further clarify the outcome definition, patients were categorized into four MoCA/MMSE classification patterns: both tests normal, MoCA-only abnormal, MMSE-only abnormal, and both tests abnormal. Participants with both MoCA and MMSE within the normal range were classified as PSNC, whereas those with abnormality on either MoCA, MMSE, or both tests were classified as PSCI. The distribution of these cognitive-classification categories across the training, internal test, and external validation cohorts is shown in Supplementary Table 11.

Feature selection

In the first feature-selection step, Boruta was applied exclusively within the training cohort to screen the 65 candidate variables and retained 53 predictors as potentially informative (Fig. 2A). These included age, time from stroke onset to assessment, height, weight, BMI, smoking, diabetes, cardiovascular disease, WMH, hyperlipidemia, RBC, Hgb, WBC, LY, NE, PLT, INR, APTT, FIB, TT, Hcy, ALB, ALT, AST, GGT, ALP, CHE, AFU, TBA, TBIL, DBIL, IBIL, GLU, UREA, CREA, LDH, HBDH, CK, CK-MB, TC, TG, HDL-C, LDL-C, apoA1, ApoB, ApoE, Lp(a), ADA, β2-MG, CRP, PCT, ESR, and NIHSS.

Fig. 2.

Feature selection using Boruta and LASSO. (A) Boruta importance ranked across candidate predictors. Boxplots show the distribution of Z-scores from iterative random-forest runs; the horizontal line denotes the median importance. Color coding: green = confirmed important attributes, blue = tentative, red = rejected. Shadow attributes provide the null benchmark for importance. (B) LASSO model selection by cross-validation. The mean binomial deviance (red curve) with standard-error bars is plotted versus log(λ). Vertical dashed lines indicate the optimal penalty at minimum error (λ_min) and the more parsimonious choice within 1 standard error (λ_1se). Numbers above the curve denote the count of non-zero coefficients at each λ. (C) LASSO coefficient profiles for all candidate variables as a function of log(λ). Progressive shrinkage drives small coefficients toward zero; sparser models emerge at larger penalties. LASSO, least absolute shrinkage and selection operator

We then fitted a LASSO-penalized logistic regression model on the Boruta-retained features and selected the penalty parameter λ using 10-fold cross-validation. The λ_min criterion selected 21 predictors, whereas the λ_1se criterion selected 10 predictors, corresponding to a 52.4% reduction in the number of predictors. To examine whether the more parsimonious λ_1se-derived feature set compromised predictive performance, we compared the 10-variable feature set with the 21-variable λ_min-derived feature set across the candidate models. The 10-variable feature set achieved a slightly higher mean internal test AUROC than the 21-variable feature set (0.839 vs. 0.830), whereas the mean external validation AUROCs were nearly identical (0.811 vs. 0.813). Similar findings were observed for average precision. The 10-variable feature set showed a slightly higher mean internal test AP (0.697 vs. 0.694) and a higher mean external validation AP (0.678 vs. 0.664) than the 21-variable feature set. The best internal test AUROCs were also highly comparable between the two feature sets (0.850, 95% CI: 0.811–0.889 vs. 0.854, 95% CI: 0.814–0.893), although the best external validation AUROC was numerically higher for the 21-variable feature set (0.851, 95% CI: 0.803–0.899 vs. 0.827, 95% CI: 0.776–0.878). The 10-variable feature set reduced model complexity without a meaningful loss in discrimination or AP (Supplementary Table 12). Therefore, consistent with the predefined λ_1se criterion favoring parsimony, the 10-variable feature set was retained as the primary feature set for final model development and evaluation. The final set of 10 predictors was age, smoking, diabetes, WMH, Hgb, Hcy, ALB, TG, apoA1, and NIHSS (Fig. 2B and C).

Model construction and verification

In the training cohort, KNN showed the highest apparent performance, with an AUROC of 0.982 (95% CI, 0.976–0.988), AP of 0.9638, accuracy of 0.9146, sensitivity of 0.7786, specificity of 0.9747, precision of 0.9315, F1-score of 0.8482, and Brier score of 0.0567. However, its calibration slope was 2.030, indicating suboptimal calibration, with predictions that were not sufficiently extreme in the training cohort. Random Forest also showed high apparent discrimination, with an AUROC of 0.962 (95% CI, 0.951–0.972) and AP of 0.9237, but its calibration was less favorable, with a calibration intercept of 1.180 and calibration slope of 1.321 (Fig. 3A and D; Table 3).

Fig. 3.

Discrimination and precision–recall performance of eight machine-learning classifiers for predicting PSCI across training, internal testing, and external validation cohorts. (A–C) ROC curves plotted as sensitivity versus 1–specificity for the training set (A), internal test set (B), and external validation set (C). (D–F) PR curves for the same three cohorts. Average precision for each model is shown in the panel legends. AUC, area under the ROC curve; AP, average precision

Table 3.

Model performance and calibration in the complete-case analysis

Cohorts	Models	AUC (95%CI)	AP	Accuracy	Sensitivity	Specificity	Precision	F1 score	Brier score	Calibration intercept	Calibration slope
Training Cohort	Decision Tree	0.887(0.860–0.914)	0.8403	0.8643	0.7099	0.9325	0.8230	0.7623	0.1030	0.000	1.000
	KNN	0.982(0.976–0.988)	0.9638	0.9146	0.7786	0.9747	0.9315	0.8482	0.0567	0.080	2.030
	LightGBM	0.924(0.903–0.945)	0.8743	0.8561	0.5954	0.9713	0.9017	0.7172	0.1229	0.005	2.810
	Logistic Regression	0.871(0.842-0.900)	0.7792	0.8632	0.7061	0.9325	0.8222	0.7598	0.1127	0.000	1.000
	Neural Network	0.887(0.860–0.914)	0.7988	0.9041	0.7786	0.9595	0.8947	0.8327	0.0892	0.030	0.850
	Random Forest	0.962(0.951–0.972)	0.9237	0.8819	0.645	0.9865	0.9548	0.7699	0.1030	1.180	1.321
	SVM	0.881(0.852–0.910)	0.8397	0.8772	0.7366	0.9393	0.8428	0.7862	0.1001	-0.093	1.121
	XGBoost	0.919(0.898–0.940)	0.8622	0.8784	0.7061	0.9545	0.8726	0.7806	0.0983	0.008	1.374
Testing cohort	Decision Tree	0.814(0.771–0.857)	0.6665	0.7821	0.5759	0.8736	0.6691	0.6190	0.1477	-0.062	0.571
	KNN	0.833(0.793–0.873)	0.6713	0.8424	0.6835	0.9129	0.7770	0.7273	0.1410	0.189	0.210
	LightGBM	0.838(0.797–0.879)	0.7063	0.7977	0.4873	0.9354	0.7700	0.5969	0.1483	-0.004	1.892
	Logistic Regression	0.845(0.804–0.887)	0.7252	0.8599	0.7405	0.9129	0.7905	0.7647	0.1257	-0.078	0.885
	Neural Network	0.849(0.810–0.887)	0.7012	0.8405	0.6899	0.9073	0.7676	0.7267	0.1371	-0.098	0.593
	Random Forest	0.842(0.801–0.883)	0.6969	0.7860	0.4620	0.9298	0.7449	0.5703	0.1430	1.006	0.778
	SVM	0.850(0.811–0.889)	0.6927	0.8424	0.6835	0.9129	0.7770	0.7273	0.1285	-0.110	0.837
	XGBoost	0.841(0.800-0.882)	0.7120	0.8191	0.6013	0.9157	0.7600	0.6714	0.1350	-0.042	0.904
Validation cohort	Decision Tree	0.809(0.754–0.864)	0.7094	0.7977	0.5385	0.9114	0.7273	0.6188	0.1383	0.069	0.755
	KNN	0.787(0.729–0.846)	0.6495	0.8328	0.6635	0.9072	0.7582	0.7077	0.1526	0.055	0.165
	LightGBM	0.819(0.764–0.874)	0.6947	0.7859	0.4231	0.9451	0.7719	0.5466	0.1502	0.016	1.970
	Logistic Regression	0.808(0.75–0.865)	0.6468	0.8240	0.6346	0.9072	0.7500	0.6875	0.1408	-0.008	0.790
	Neural Network	0.79(0.732–0.848)	0.6639	0.8211	0.6346	0.9030	0.7416	0.6839	0.1498	-0.008	0.566
	Random Forest	0.823(0.767–0.878)	0.6931	0.7947	0.4423	0.9494	0.7931	0.5679	0.1455	1.068	0.702
	SVM	0.823(0.768–0.878)	0.7012	0.8358	0.6635	0.9114	0.7667	0.7113	0.1287	-0.083	0.869
	XGBoost	0.827(0.776–0.878)	0.6668	0.8328	0.6250	0.9241	0.7831	0.6952	0.1366	-0.027	0.916

AUC values are presented with 95% confidence intervals. Classification metrics were calculated at a fixed probability threshold of 0.5. Calibration was evaluated using the Brier score, calibration intercept, and calibration slope. A calibration intercept close to 0 and a calibration slope close to 1 indicate better agreement between predicted and observed risks. AP, average precision; AUC, area under the receiver operating characteristic curve; CI, confidence interval; KNN, k-nearest neighbors; SVM, support vector machine; XGBoost, extreme gradient boosting; LightGBM, light gradient boosting machine

In the internal test cohort, SVM and neural network achieved the highest AUROC values. SVM yielded an AUROC of 0.850 (95% CI, 0.811–0.889), AP of 0.6927, accuracy of 0.8424, sensitivity of 0.6835, specificity of 0.9129, precision of 0.7770, F1-score of 0.7273, Brier score of 0.1285, calibration intercept of − 0.110, and calibration slope of 0.837. The neural network achieved an AUROC of 0.849 (95% CI, 0.810–0.887), AP of 0.7012, accuracy of 0.8405, sensitivity of 0.6899, specificity of 0.9073, precision of 0.7676, F1-score of 0.7267, Brier score of 0.1371, calibration intercept of − 0.098, and calibration slope of 0.593 (Fig. 3B and E; Table 3).

In the external validation cohort, XGBoost achieved the highest AUROC and showed a favorable overall balance between discrimination, calibration, and clinical operating characteristics. XGBoost produced an AUROC of 0.827 (95% CI, 0.776–0.878), AP of 0.6668, accuracy of 0.8328, sensitivity of 0.6250, specificity of 0.9241, precision of 0.7831, F1-score of 0.6952, Brier score of 0.1366, calibration intercept of − 0.027, and calibration slope of 0.916. SVM showed comparable external discrimination and slightly better AP and Brier score, with an AUROC of 0.823 (95% CI, 0.768–0.878), AP of 0.7012, accuracy of 0.8358, sensitivity of 0.6635, specificity of 0.9114, precision of 0.7667, F1-score of 0.7113, Brier score of 0.1287, calibration intercept of − 0.083, and calibration slope of 0.869 (Fig. 3C and F; Table 3).

To assess whether complete-case exclusion materially affected model performance, we compared the primary complete-case analysis with the random forest imputation–based sensitivity analysis. Performance was broadly consistent between the complete-case and imputation-based analyses, with no substantial deterioration in discrimination, calibration, or overall predictive accuracy. These findings suggest that the primary complete-case analysis was not materially weakened by missing predictor values, supporting the robustness of the main modeling results. Detailed performance comparisons between the complete-case and imputation-based analyses are provided in Supplementary Table 13.

We compared the eight ML algorithms using the same evaluation metrics in the internal test and external validation cohorts. SVM and XGBoost had relatively low Brier scores, calibration intercepts near 0, calibration slopes near 1, and calibration plots that were close to the identity line. Specifically, in the internal test and external validation cohorts, SVM achieved Brier scores of 0.1285 and 0.1287, calibration intercepts of − 0.110 and − 0.083, and calibration slopes of 0.837 and 0.869, respectively. XGBoost achieved Brier scores of 0.1350 and 0.1366, calibration intercepts of − 0.042 and − 0.027, and calibration slopes of 0.904 and 0.916, respectively. Predicted and observed risks were reasonably aligned for both models, and XGBoost had calibration slopes closer to 1 in both cohorts (Supplementary Fig. 1, Table 3). In DCA, the models were evaluated in the context of identifying patients who might benefit from enhanced cognitive monitoring and early rehabilitation planning. The clinically plausible threshold-probability range of 15%–40% was marked by the shaded area and vertical dotted lines in the DCA plots. Within this range, most models, including XGBoost and SVM, showed positive net benefit compared with the treat-none strategy and generally remained above or comparable to the treat-all strategy in the internal test and external validation cohorts. These findings suggest that model-guided PSCI risk stratification may provide potential clinical value for prioritizing patients for closer cognitive surveillance and early rehabilitation planning. Because the analysis was retrospective and several net-benefit curves were close, the DCA results are exploratory and do not establish clinical effectiveness (Supplementary Fig. 2A–2 C).

Model and operating-threshold selection

Using the conventional 0.50 threshold as the reference operating point, XGBoost and SVM showed comparable external-validation performance. Although SVM showed slightly higher average precision, sensitivity, F1-score, and a lower Brier score, XGBoost achieved a marginally higher AUROC and maintained favorable specificity, precision, and calibration (Fig. 3C and F; Table 3). Because AUROC was prespecified as the primary model-selection criterion and XGBoost was suitable for subsequent SHAP interpretation and web-based implementation, XGBoost was selected as the final implementation model, with SVM retained as a close competing model.

Operating-threshold selection was then considered separately from model selection. For the final XGBoost model, threshold-dependent performance was evaluated under the conventional 0.50 threshold, the training-derived Youden-index threshold, and the sensitivity-oriented Screen-80 threshold (Table 4). The 0.50 threshold was retained as a conservative high-specificity reference threshold. The Screen-80 threshold increased sensitivity and reduced missed PSCI cases while retaining acceptable specificity. In the external validation cohort, the Screen-80 threshold increased XGBoost sensitivity from 0.6250 to 0.7500 and reduced false negatives from 39 to 26, while maintaining specificity of 0.8692 and balanced accuracy of 0.8096 (Table 4). Threshold-dependent comparisons between XGBoost and SVM are shown in Table 5, and the complete threshold-dependent results for all eight models are provided in Supplementary Table 14.

Table 4.

Threshold-dependent performance of final XGBoost model

Cohort	Threshold type	Threshold	Accuracy	Sensitivity	Specificity	Precision	NPV	F1-score	Balanced accuracy	TP	FP	FN	TN
Testing	Default	0.5000	0.8191	0.6013	0.9157	0.7600	0.8380	0.6714	0.7585	95	30	63	326
	Youden	0.2187	0.8035	0.7911	0.8090	0.6477	0.8972	0.7123	0.8001	125	68	33	288
	Screen-80	0.2376	0.8230	0.7785	0.8427	0.6872	0.8955	0.7300	0.8106	123	56	35	300
Validation	Default	0.5000	0.8328	0.6250	0.9241	0.7831	0.8488	0.6952	0.7745	65	18	39	219
	Youden	0.2187	0.8035	0.7500	0.8270	0.6555	0.8829	0.6996	0.7885	78	41	26	196
	Screen-80	0.2376	0.8328	0.7500	0.8692	0.7156	0.8879	0.7324	0.8096	78	31	26	206

Default refers to the conventional 0.50 probability threshold. Youden refers to the threshold maximizing the Youden index in the training cohort. Screen-80 refers to the sensitivity-oriented screening threshold selected in the training cohort to achieve sensitivity ≥ 0.80 while preserving specificity. The Youden and Screen-80 thresholds were derived exclusively from the training cohort and then fixed before evaluation in the internal test and external validation cohorts

Table 5.

Threshold-dependent comparison between XGBoost and SVM in the internal test and external validation cohorts

Model	Cohort	Threshold type	Threshold	Accuracy	Sensitivity	Specificity	Precision	NPV	F1-score	Balanced accuracy	TP	FP	FN	TN
XGBoost	Testing	Default	0.5000	0.8191	0.6013	0.9157	0.7600	0.8380	0.6714	0.7585	95	30	63	326
		Youden	0.2187	0.8035	0.7911	0.8090	0.6477	0.8972	0.7123	0.8001	125	68	33	288
		Screen-80	0.2376	0.8230	0.7785	0.8427	0.6872	0.8955	0.7300	0.8106	123	56	35	300
SVM	Testing	Default	0.5000	0.8424	0.6835	0.9129	0.7770	0.8667	0.7273	0.7982	108	31	50	325
		Youden	0.1623	0.8599	0.7785	0.8961	0.7688	0.9011	0.7736	0.8373	123	37	35	319
		Screen-80	0.1330	0.8268	0.8038	0.8371	0.6865	0.9058	0.7405	0.8204	127	58	31	298
XGBoost	Validation	Default	0.5000	0.8328	0.6250	0.9241	0.7831	0.8488	0.6952	0.7745	65	18	39	219
		Youden	0.2187	0.8035	0.7500	0.8270	0.6555	0.8829	0.6996	0.7885	78	41	26	196
		Screen-80	0.2376	0.8328	0.7500	0.8692	0.7156	0.8879	0.7324	0.8096	78	31	26	206
SVM	Validation	Default	0.5000	0.8358	0.6635	0.9114	0.7667	0.8606	0.7113	0.7874	69	21	35	216
		Youden	0.1623	0.8475	0.7692	0.8819	0.7407	0.8970	0.7547	0.8255	80	28	24	209
		Screen-80	0.1330	0.8035	0.7788	0.8143	0.6480	0.8935	0.7074	0.7966	81	44	23	193

Testing indicates the internal test cohort, and Validation indicates the external validation cohort. Default refers to the fixed 0.50 threshold. Youden refers to the training-derived threshold maximizing the Youden index. Screen-80 refers to the training-derived sensitivity-oriented threshold targeting sensitivity ≥ 0.80. Optimized thresholds were determined only in the training cohort and then fixed for testing and external validation. Precision is equivalent to positive predictive value. NPV, negative predictive value; TP, true positive; FP, false positive; FN, false negative; TN, true negative

SHAP analysis for model interpretation

To interpret the final XGBoost model, we applied SHAP analysis to quantify the contribution of each selected predictor to the model output. The global SHAP importance plot showed that diabetes had the largest mean absolute SHAP value (0.396), indicating the greatest average contribution to model prediction, followed by NIHSS (0.262), Hgb (0.247), Hcy (0.221), apoA1 (0.197), TG (0.167), ALB (0.155), WMH (0.122), smoking (0.116), and age (0.094) (Fig. 4A).

Fig. 4.

SHAP-based interpretation and classification performance of the final XGBoost model. (A) Global feature importance ranked by mean absolute SHAP value. (B) SHAP summary plot showing the distribution and direction of feature contributions across individual patients. Each point represents one patient; the horizontal position represents the SHAP value, and color represents the feature value, with yellow indicating higher values and purple indicating lower values. Positive SHAP values increase the predicted PSCI risk, whereas negative SHAP values decrease the predicted risk. (C) Confusion matrix of the XGBoost model in the external validation cohort under the conventional 0.50 threshold. Columns represent reference labels, and rows represent model predictions. Detailed threshold-dependent classification results under the 0.50, Youden-index, and Screen-80 thresholds are provided in Tables 4 and 5 and Supplementary Table 14. SHAP, Shapley additive explanations; XGBoost, extreme gradient boosting; PSCI, post-stroke cognitive impairment; NIHSS, National Institutes of Health Stroke Scale; Hgb, hemoglobin; Hcy, homocysteine; apoA1, apolipoprotein A1; TG, triglycerides; ALB, albumin; WMH, white matter hyperintensities

The SHAP summary plot showed the distribution and direction of feature contributions across patients; the direction and magnitude of a predictor’s contribution varied across individual profiles (Fig. 4B). These SHAP results were interpreted as post hoc explanations of model behavior rather than evidence of causal or independent etiological effects.

For reference, the confusion matrix of the final XGBoost model in the external validation cohort at the conventional 0.50 threshold is shown in Fig. 4C. Detailed threshold-dependent classification results under the 0.50, Youden-index, and Screen-80 thresholds are reported in Tables 4 and 5 and Supplementary Table 14.

Model application

We developed a browser-based proof-of-concept prototype using R Shiny to demonstrate how the final model could present individualized PSCI risk estimates and feature-level explanations (Fig. 5). After entering the required predictor variables, the interface returns a model-derived risk estimate and a corresponding risk category. This tool was intended only as a prototype for visualizing and communicating model outputs. It was not prospectively tested in clinical workflow, was not recalibrated for real-time use, and was not evaluated for usability, clinician adherence, decision impact, or patient outcomes. Therefore, it should not be interpreted as a clinically deployable decision-support system at this stage.

Fig. 5.

Web-based proof-of-concept prototype for individualized PSCI risk prediction using the final XGBoost model. The left panel shows the questionnaire interface for entering the 10 selected predictors: age, smoking, diabetes, WMH, Hgb, Hcy, ALB, TG, apoA1, and NIHSS. The decision threshold is adjustable, with the Screen-80 threshold of 0.238 shown as the default screening-oriented operating point. The right panel displays the model-derived predicted probability, binary risk classification, and per-feature local contribution to the predicted probability. The prototype is intended only to demonstrate potential model implementation and has not been prospectively validated for clinical decision-making. PSCI, post-stroke cognitive impairment; XGBoost, extreme gradient boosting; NIHSS, National Institutes of Health Stroke Scale; Hgb, hemoglobin; Hcy, homocysteine; apoA1, apolipoprotein A1; TG, triglycerides; ALB, albumin; WMH, white matter hyperintensities

Discussion

This two-center retrospective study used acute-phase (≤ 1 week after stroke onset) clinical, laboratory, and imaging data to predict PSCI assessed at 3–6 months after stroke. A two-stage Boruta–LASSO procedure selected 10 predictors from 65 candidate variables, and eight machine-learning models were evaluated in independent internal test and external validation cohorts. In the external validation cohort, XGBoost achieved the highest AUROC (0.827; 95% CI, 0.776–0.878) and showed acceptable calibration, with a Brier score of 0.1366, a calibration intercept of − 0.027, and a calibration slope of 0.916. These results support the use of routinely available acute-phase data for PSCI risk estimation in inpatient rehabilitation populations. However, because both cohorts were drawn from rehabilitation settings in the same city, the findings support preliminary regional transportability rather than broad generalizability.

The two-stage feature-selection strategy reduced the number of predictors while retaining predictive performance. Boruta first retained potentially relevant variables, after which LASSO reduced redundancy and produced a smaller predictor set. Similar combined approaches have been used in other clinical prediction studies [42, 43]. The 10-variable set achieved performance comparable to that of the 21-variable set while using fewer predictors and was therefore selected for final model development. Among the evaluated algorithms, XGBoost achieved the highest external-validation AUROC, although its overall performance was similar to that of SVM across several metrics. XGBoost was ultimately retained because it combined the best external discrimination with acceptable calibration and compatibility with SHAP-based interpretation. SHAP analysis showed that diabetes had the largest mean absolute contribution, followed by NIHSS, Hgb, Hcy, apoA1, TG, ALB, WMH, smoking, and age.

The model is intended as a screening-oriented risk-stratification tool rather than a diagnostic instrument. A high-risk classification would not confirm PSCI or mandate treatment. It could instead prompt formal cognitive assessment, repeated monitoring, rehabilitation referral when appropriate, management of modifiable vascular and metabolic risk factors, patient and caregiver education, and post-discharge follow-up. These follow-up measures are generally non-invasive and carry relatively little risk. A sensitivity-oriented threshold may therefore be appropriate when the main aim is to reduce missed PSCI cases, although it would also result in more false-positive classifications and additional assessments. Within the prespecified threshold-probability range of 15%–40%, DCA suggested that model-guided risk stratification could provide greater net benefit than treating no patients and, across parts of this range, treating all patients as high risk. However, DCA does not show that model-guided care improves PSCI detection, treatment timing, or cognitive outcomes. The Shiny application is a proof-of-concept display of individualized risk estimates and feature contributions; it is not a clinical decision-support system. The prototype requires prospective workflow evaluation before use.

Among the selected predictors, diabetes, age, smoking, NIHSS, and WMH capture different aspects of vascular and metabolic burden, stroke severity, and cerebral small-vessel injury. Previous studies have linked diabetes to PSCI and to cognitive impairment or dementia in broader populations [4, 44, 45]. Diabetes has also been associated with cerebral atrophy, cerebral small-vessel disease, poorer neurological recovery, recurrent stroke, and subsequent dementia [46–53]. In the present study, diabetes status contributed more strongly to model prediction than a single acute-phase glucose measurement. This may be because a documented diagnosis of diabetes better reflects cumulative glycemic and vascular exposure, whereas a single glucose value may be influenced by acute stress responses, treatment, and variation in sampling time. However, diabetes duration, long-term glycemic control, and treatment history were not available, although longer diabetes duration has previously been associated with a higher risk of PSCI [12]. WMH are widely regarded as an imaging marker of cerebral small-vessel disease and have been associated with chronic ischemic injury, demyelination, blood–brain barrier dysfunction, and adverse cognitive outcomes after stroke [54, 55]. Their development and progression are also related to vascular risk factors such as diabetes, smoking, hypertension, and atherosclerosis [56, 57]. In the present study, however, WMH was recorded only as present or absent based on routine radiology reports. We were therefore unable to assess whether Fazekas grade, lesion volume, severity, or anatomical distribution, including periventricular versus deep/subcortical involvement, provided additional predictive information, although these characteristics have previously been associated with PSCI and domain-specific cognitive deficits [58, 59]. Because WMH may progress over time [60], future studies should evaluate whether longitudinal changes provide additional predictive information. Accordingly, WMH should be interpreted as a routinely available predictive imaging marker rather than evidence of a specific biological mechanism.

Hgb, ALB, Hcy, apoA1, and TG may reflect related aspects of hematological, nutritional, and metabolic status. Lower Hgb may indicate anemia and reduced oxygen-carrying capacity, which could increase susceptibility to cerebral ischemic injury. Previous multivariable analyses have associated anemia with PSCI, although the underlying mechanisms were not directly examined in our study [61]. ALB is commonly used as an indicator of nutritional and systemic health, and albumin-based measures such as the Geriatric Nutritional Risk Index and Prognostic Nutritional Index have been associated with cognitive performance and PSCI in stroke populations [62, 63]. Hcy metabolism is influenced by folate and vitamins B6 and B12 [64], and elevated Hcy has been associated with cardiovascular and cerebrovascular disease, neurodegenerative conditions, cognitive decline, and cortical or hippocampal atrophy [65–68]. Nevertheless, evidence regarding its relationship with PSCI remains heterogeneous, with associations varying across follow-up periods and patient subgroups [69–71]. Evidence for conventional lipid markers is similarly inconsistent. Some studies have found no clear relationship between cholesterol levels and PSCI [72, 73], whereas others have reported associations involving LDL-C, HDL-C, and the acute-phase TG/HDL-C ratio [74, 75]. The apoA1 and TG SHAP values describe predictive contributions only; they do not establish a causal role for lipid dysregulation. Their contributions, together with that of Hcy, may also have been influenced by fasting status, vitamin levels, lipid-lowering or antidiabetic treatment, metabolic syndrome, diabetes, and nutritional status, which were not fully characterized in the retrospective dataset.

Together, the selected variables reflected stroke severity, vascular burden, and systemic metabolic or nutritional status. Their inclusion is consistent with the multifactorial nature of PSCI. However, SHAP values only show how individual variables contribute to model predictions. They do not represent independent associations or causal effects, nor do they identify modifiable biological pathways. These predictor contributions should be interpreted cautiously. Prospective studies are needed to determine whether these predictors remain stable over time and across clinical settings.

This study has several limitations. First, this was a retrospective study based primarily on electronic medical records. As a result, the retrospective records may contain inaccuracies, missing details, or other sources of information bias. Although candidate predictors were restricted to variables recorded during the acute-phase hospitalization within 1 week after stroke onset, the exact timing of laboratory tests, imaging examinations, and clinical assessments may have varied across patients. We attempted to reduce this variability by using the earliest available acute-phase records rather than peak or nadir values, but measurement timing still varied among patients.

Second, the study population was derived from inpatient rehabilitation settings 3–6 months after stroke onset. This rehabilitation-based sampling frame may have introduced selection bias by excluding patients with mild stroke, rapid recovery, no need for inpatient rehabilitation, or limited access to rehabilitation services. The model is therefore most applicable to similar inpatient rehabilitation populations, not to the general stroke population. Although the model was externally validated in an independent center, both centers were located in the same city. Similar care pathways, referral patterns, and population characteristics may limit the heterogeneity of the validation cohort. The model should be validated in multicenter acute-stroke and community cohorts from other regions and healthcare systems before broader application.

Third, missing data remain an important limitation. Complete-case analysis may introduce selection bias if missingness is related to stroke severity, frailty, care pathways, or the availability of laboratory and imaging assessments, and it may also reduce real-world usability. To address this concern, we quantified the extent and pattern of missingness and conducted a random forest imputation–based sensitivity analysis. Model performance was similar after patients with missing predictors were included through imputation. This suggests that the main results were not driven solely by the complete-case analysis. The mechanism underlying the missing data could not be determined from the retrospective electronic records. Complete-case selection may therefore have introduced residual bias. Future prospective studies with standardized data collection and prespecified missing-data strategies, including multiple imputation where appropriate, are needed to further evaluate model robustness.

Fourth, several clinically important variables were unavailable or insufficiently detailed. These included stroke lesion location and extent, quantitative WMH burden and distribution, domain-specific cognitive scores, medication exposure, diabetes duration, and other potential confounders. In particular, WMH was coded only as a binary presence/absence variable based on routine radiology reports, without information on Fazekas grade, lesion volume, severity, or periventricular versus deep/subcortical distribution. Therefore, the biological interpretation of WMH in this model is limited. Education may have contributed less because education-adjusted MoCA and MMSE thresholds were used to define PSCI. Its relationship with cognitive reserve and PSCI risk remains uncertain.

Fifth, although all feature selection and hyperparameter tuning were restricted to the training cohort, the Boruta→LASSO feature-selection procedure was not repeatedly nested within each cross-validation fold. In addition, the development cohort was divided into training and internal test cohorts using a single fixed stratified random split. Although 10-fold cross-validation within the training cohort, independent internal testing, and external validation were used to reduce optimism, some instability in internal performance estimates cannot be fully excluded. Repeated nested resampling or bootstrap validation would provide more stable internal estimates, while prospective multicenter studies are needed to assess calibration outside the current centers. Performance across sex, age groups, stroke subtypes, and metabolic comorbidities was not fully assessed. The model’s consistency across these subgroups therefore remains uncertain.

Sixth, PSCI was modeled as a binary composite outcome based on MoCA and/or MMSE criteria. Because standardized dementia diagnosis, functional-dependence information, domain-specific neuropsychological data, and comprehensive cognitive batteries were not consistently available, the model predicts MoCA/MMSE-defined PSCI and does not distinguish PSCI without dementia from post-stroke dementia. Future studies should validate the model against comprehensive neuropsychological assessments and evaluate performance across different severities and subtypes of post-stroke cognitive impairment.

Finally, although a web-based Shiny prototype was developed to demonstrate potential clinical use, it has not been prospectively validated or tested in real clinical workflows. The operating thresholds were derived retrospectively and have not been calibrated for local implementation. Usability, clinician adherence, workflow burden, alert fatigue, and clinical impact were not assessed. Therefore, the application remains a proof-of-concept prototype and is not ready for clinical deployment. Prospective, real-time validation in acute-care and rehabilitation settings is required before clinical implementation.

Conclusions

Using routinely available acute-phase clinical, laboratory, and imaging variables, the XGBoost model showed good discrimination and acceptable calibration for predicting PSCI assessed at 3–6 months among patients admitted for inpatient rehabilitation. The model may support early risk stratification and help prioritize high-risk patients for formal cognitive assessment, closer monitoring, and timely rehabilitation planning, but not for diagnosis. Before routine use, the model requires prospective multicenter validation, local recalibration, and evaluation of its clinical impact.

Supplementary Information

Below is the link to the electronic supplementary material.

Abbreviations

PSCI

Post-stroke cognitive impairment

PSD

Post stroke dementia

PSNC

Post-stroke normal cognition

LASSO

Least Absolute Shrinkage and Selection Operator

AUROC

Area Under the Receiver Operating Characteristic Curve

AP

Average precision

DCA

Decision-curve analysis

SHAP

Shapley additive explanations

MoCA

Montreal Cognitive Assessment

MMSE

Mini-Mental State Examination

RBC

Red blood cell count

Hgb

Hemoglobin

WBC

White blood cell count

NE

Neutrophil count

LY

Lymphocyte count

PLT

Platelet count

PT

Prothrombin time

APTT

Activated partial thromboplastin time

INR

International normalized ratio

FIB

Fibrinogen

TT

Thrombin time

ALB

Albumin

ALT

Alanine aminotransferase

AST

Aspartate aminotransferase

GGT

γ-glutamyl transferase

ALP

Alkaline phosphatase

CHE

Cholinesterase

AFU

α-L-fucosidase

TBA

Total bile acids

TBIL

Total bilirubin

DBIL

Direct bilirubin

IBIL

Indirect bilirubin

CREA

Creatinine

UA

Uric acid

LDH

Lactate dehydrogenase

HBDH

α-hydroxybutyrate dehydrogenase

CK

Creatine Kinase

CK-MB

Creatine Kinase-MB

Hcy

Homocysteine

GLU

Glucose

TC

Total cholesterol

TG

Triglycerides

HDL-C

High-density lipoprotein cholesterol

LDL-C

Low-density lipoprotein cholesterol

apoA1

Apolipoprotein A-I

ApoB

Apolipoprotein B

ApoE

Apolipoprotein E

Lp(a)

Lipoprotein(a)

ADA

Adenosine deaminase

β2-MG

β2-microglobulin

CRP

C-reactive protein

PCT

Procalcitonin

ESR

Erythrocyte sedimentation rate

WMH

White matter hyperintensities

LR

Logistic Regression

KNN

K-Nearest Neighbors

DT

Decision Tree

RF

Random Forest

SVM

Support Vector Machine

NN

Neural Network

XGBoost

Extreme Gradient Boosting

LightGBM

Light Gradient Boosting Machine

BMI

Body Mass Index

NPV

Negative Predictive Value

ML

Machine Learning

Author contributions

JX conceived and supervised the study and obtained funding, with additional funding secured by JM. JX, XC, and JM designed the methodology. XC, JM, YC, XL, and SM conducted the investigation; YC and XL provided resources. JM, YC, HW, XL, and QL curated the data, and QL developed the software. XC, HW, and JX performed the formal analyses, and XC, HW, JZ, and JX prepared the visualizations. XC, JM, and JX drafted the original manuscript. All authors interpreted the findings, contributed to writing—review and editing, and approved the final version.

Funding

This work was supported by the Research Project of Traditional Chinese Medicine of the Hebei Administration of Traditional Chinese Medicine (Grant No. 2024129); Hebei Provincial Medical Science Research Program (Grant No. 20250207); and the S&T Program of Hebei (Grant No. 2037727D).

Data availability

The de-identified datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request, subject to institutional and ethical requirements.

Declarations

Ethics approval and consent to participate

This study was conducted in accordance with the principles of the Declaration of Helsinki. Written informed consent was obtained from all participants. The study protocol was approved by the Medical Ethics Committee of Shijiazhuang People’s Hospital and the Ethics Review Board of Shijiazhuang Traditional Chinese Medicine Hospital.

Consent for publication

Not applicable.

Generative AI statement

An AI-based assistant (ChatGPT) was used to improve readability and language only. All content was subsequently reviewed and edited by the authors, who accept full responsibility for the final manuscript.

Competing interests

The authors declare no competing interests.