Machine learning mortality prediction model for cyclosporine therapy in pediatric aplastic anemia

Abstract

The outcomes of children with aplastic anemia receiving cyclosporine monotherapy vary significantly in terms of mortality risk; therefore, a prognostic model for predicting mortality risk was constructed to optimize risk-stratified treatment strategies. This retrospective cohort study included children with acquired AA receiving cyclosporine-based immunosuppression, stratified by disease severity (vSAA/SAA/NSAA) and randomly split into training (70%) and validation (30%) cohorts. Ten machine learning models were developed; hyperparameters were optimized via grid search with 10-fold cross-validation exclusively within the training cohort to prevent data leakage. Model performance was evaluated using area under the ROC curve (AUC), accuracy, recall, specificity, precision, F1 score, and Brier score. Decision curve analysis (DCA) quantified clinical net benefit. The calibration curve was used to evaluate the reliability of the predicted probabilities. The SHapley Additive exPlanations (SHAP) framework was used to interpret feature contributions and ensure model transparency. Least absolute shrinkage and selection operator (LASSO) regression on the training cohort identified 5 predictors: reticulocyte count (RC), platelet count (PLT), disease subtype (vSAA/SAA/NSAA), total bilirubin (TB), and bone marrow myeloid proportion. The CatBoost model achieved the highest performance: AUC 0.834 (95% CI: 0.774–0.895) in training and 0.826 (95% CI: 0.743–0.910) in validation, with acceptable calibration (Brier score: 0.206 in training cohort, 0.207 in validation cohort). SHAP analysis confirmed RC as the top contributor, with lower RC values associated with higher predicted mortality risk. The CatBoost model demonstrates robust performance and transparency for predicting mortality risk in children with AA after cyclosporine treatment. Adherence to TRIPOD + AI guidelines ensures methodological rigor, supporting its potential as a clinical decision tool to stratify patients into distinct mortality risk groups and optimize individualized treatment strategies.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00277-026-06842-3.

Introduction

Aplastic anemia (AA) is a serious blood disease characterized by peripheral blood pancytopenia and bone marrow hematopoietic failure [1]. Cyclosporine (CsA)-based immunosuppressive therapy (IST) serves as the standard treatment for acquired aplastic anemia (aAA) [2]. In severe aplastic anemia (SAA), international guidelines advocate combined IST with CsA and antithymocyte globulin (ATG) as the first-line option for patients lacking eligible hematopoietic stem cell transplantation (HSCT) donors [3, 4]. In clinical practice, cyclosporine monotherapy remains a critical therapeutic option for patients diagnosed with non-severe aplastic anemia (NSAA). This approach is also indicated for individuals concerned about the risks associated with hematopoietic stem cell transplantation (HSCT) and combined immunosuppressive therapy, or those facing financial constraints that limit access to these treatment modalities. A retrospective analysis of 912 AA patients receiving immune suppression therapy revealed striking survival disparities: pediatric patients exhibited a significantly higher survival rate than adults (81% vs. 70%, p = 0.001), particularly among those with very severe AA (vSAA) (83% vs. 62%, p = 0.0002). Concurrently, combination therapy outperformed monotherapy, with survival rates of 77% versus 62% (p = 0.002) [5]. Studies have shown that cyclosporine monotherapy for NSAA patients can achieve a 57.9% overall response rate, but its early efficacy for SAA and vSAA patients is relatively limited [6, 7]. Although HSCT and combined IST have improved the prognosis of patients with SAA and vSAA, their mortality risk remains higher than that of NSAA patients.At present there is still a lack of support for large cohort studies on key issues, such as the long-term outcomes of patients treated with the monotherapy regimen and the independent risk factors for mortality. The establishment of a mortality prediction model based on multidimensional clinical indicators may provide new ideas for developing individualized treatment regimens.

In recent years, machine learning has become more widely used in the biomedical field, especially in early disease diagnosis, outcome evaluation, drug development, and genomics analysis, offering technological support to promoting the development of precision medicine and individualized treatment plans [8, 9]. In the diagnosis of bone marrow failure syndrome (BMFS), existing studies have successfully applied machine learning algorithms to develop a diagnostic prediction model on the basis of multidimensional clinical data [10, 11]. A machine learning outcome prediction model with practical clinical value was also established for SAA patients receiving combined with IST [12]. To our knowledge, few studies have explored mortality prediction among AA children being managed with cyclosporine monotherapy. This study was conducted on the basis of the big data platform for clinical scientific research at our hospital, in which the real-world clinical data of AA children were used to systematically analyze the relationships between clinical characteristics and the outcomes of cyclosporine monotherapy and construct a mortality prediction model on the basis of machine learning. All procedures strictly adhere to the TRIPOD + AI guidelines to ensure transparency in model construction and completeness of reporting [13].

Materials and methods

Subjects

Guidelines for the diagnosis and treatment of AA children [14] and the Camitta criteria classification [15] were used in this study. The data of 437 patients who were newly diagnosed with acquired AA and received cyclosporine immunosuppressive treatment in our hospital between January 2009 and December 2023, excluding patients with inherited bone marrow failure syndromes and those undergoing HSCT, were included in the study. All patients were treated with cyclosporine at an initial dose of 4–6 mg/kg/d, which was subsequently adjusted to reach whole blood trough concentration of 150–200 ng/ml.

Data collection and variable screening

This was an observational retrospective cohort study. The study end point was patient survival at the end of the follow-up. Clinically relevant indicators of AA, including demographic information, disease type, clinical features and laboratory test results were collected as candidate predictor variables. Initially, candidate predictor variables were screened on the basis of the principle of data completeness; specifically, indicators with a missing data rate > 30% were excluded, resulting in the retention of 32 candidate variables (e.g., those with a missing data rate ≤ 30%), which were then subjected to multiple imputation by chain equations (MICE) [16] for imputing the missing values. Least absolute shrinkage and selection operator (LASSO) regression was performed on the training cohort only to identify significant clinical variables (p < 0.05), with hyperparameters (λ) optimized via 10-fold cross-validation within the training set to avoid data leakage.This algorithm compresses the regression coefficients of nonsignificant predictors to zero (p < 0.05) by adjusting the regularization parameter λ while retaining features with independent prognostic value; in this way, the predictive ability of machine learning models can be improved [17, 18].

Model development

The patients were divided into a training cohort (n = 307) and an internal validation cohort (n = 130) at a ratio of 7:3 via a random stratified sampling method that ensured that the two groups had similar distributions of key clinical characteristics. The data of the training cohort were used for model parameter optimization and algorithm training, whereas those of the internal validation cohort were used to evaluate the performance of the model. A multimodel comparison strategy was used in this model to systematically evaluate the predictive performance of models constructed with ten classic machine learning algorithms, including logistic regression (LR), support vector machine (SVM), gradient boosting machine (GBM), neural network (NN), random forest (RF), extreme gradient boosting (XGBoost), K-nearest neighbors (KNN), adaptive boosting (AdaBoost), light gradient boosting machine (LightGBM) and categorical boosting (CatBoost). The variables identified by the LASSO regression algorithm were included in the construction of all the models [19].

Grid searching and a 10-fold cross-validation strategy were used to find the optimal parameters for constructing the 10 models [19]. During the parameter adjustment step, the area under the curve (AUC) was used as the optimization index for model selection; specifically, the parameter combination that yielded the optimal AUC value was selected to construct the prediction model. In addition to the AUC, the accuracy, recall, specificity, precision, F1 score, and Brier score were used to evaluate model performance. The net clinical benefits of the model were quantified using decision curve analysis (DCA). The calibration curve is used to evaluate the reliability of prediction probabilities [20, 21]. To improve the interpretability of the model, we used the SHapley Additive exPlanations (SHAP) framework to analyze the contributions of the features to the optimal model [22, 23]. Figure 1 shows the flow chart of the study.

Fig. 1.

Flowchart of the article

Statistical analysis

R statistical software (version 4.4.2) was used for statistical analysis. The Shapiro‒Wilk test was used to assess the normality of the distributions of continuous variables. Variables that conformed to a normal distribution are presented as the means ± standard deviations (means ± SDs). For normally distributed data that also satisfied the assumption of homogeneity of variance, between-group comparisons were performed via the independent samples t test. Variables that did not conform to a normal distribution are presented as the median and interquartile range (IQR), and between-group differences were analyzed with the Mann‒Whitney U test. Categorical variables are expressed as frequencies and component ratios (n, %), and between-group differences were assessed with the chi-square test. P < 0.05 was considered to indicate statistical significance.

Results

General characteristics

This study enrolled 437 children newly diagnosed with AA, including 207 boys (47.4%) and 230 girls (52.6%). The median age of the patients was 92 months (IQR: 64–128 months). The follow-up period concluded on December 31, 2024, with a median duration of 738 days (range, 4–5684 days).As of the end of the follow-up period, OS analysis revealed that 358 patients were alive (81.9%), and 79 patients had died (18.1%). The SAA and vSAA subtypes accounted for 88.6%(70/79)of the total deaths. The chi-square test and the Mann‒Whitney U test was used to assess the balance in the baseline characteristics between the two groups (Table 1). The results revealed that the differences between the training cohort and the validation cohort were significant in terms of age (P = 0.049), serum lactate dehydrogenase (LDH) level (P = 0.030), proportion of myeloid cells (P = 0.033) and proportion of lymphocytes (P = 0.046, P < 0.05), whereas the differences in the remaining 28 observation indicators, including sex, disease classification, platelet count, and absolute neutrophil count, were not significant (P > 0.05). The cohort was randomly split into training and validation sets per standard protocol. However, as a retrospective study, missing data were present. Multiple imputation was used to address this, though residual bias could not be fully eliminated.

Table 1.

Baseline characteristics of AA patients between training cohort and validation cohort

Variable	Training cohort (n = 307)	Validation cohort (n = 130)	χ2/Z值	P值
Gender	n,%/M(P25,P75)/ M ± SD		0.921	0.337
Female	150(48.9)	57(43.8)
Male	157(51.5)	73(56.2)
Age(month)	88(49,128)	100(66,136)	1.967	0.049
Type			0.330	0.848
NSAA	90(29.3)	41(31.5)
SAA	132(43.0)	56(43.1)
VSAA	85(27.7)	33(25.4)
Fever			1.516	0.218
No	155(50.5)	74(56.9)
Yes	152(49.5)	56(43.1)
Cyclosporine(day)			0.824	0.364
< 7	212(69.1)	84(64.6)
≥ 7	95(30.9)	46(35.4)
PLT(×109/L)	11(6,18)	11(6,17)	0.001	0.999
WBC(×109/L)	3(2,4)	3(2,3)	-0.756	0.450
ANC(×109/L)	0.4(0.2,0.9)	0.5(0.2,0.7)	-0.081	0.936
LYM(×109/L)	2(1,3)	2(1,3)	-0.247	0.805
Hb(g/L)	74(63,85)	76(65,85)	1.180	0.238
RC(×1012/L)	0.02(0.01,0.04)	0.02(0.01,0.04)	0.822	0.411
CRP (mg/dL)			0.747	0.383
≤ 8	234(76.2)	94(72.3)
> 8	73(23.8)	36(27.7)
ALT(U/L)	19(13,28)	16(11,29)	-1.095	0.274
LDH(U/L)	213(175,249)	221(185,267)	2.171	0.030
TB(µmol/L)	9(5,13)	9(6,12)	0.860	0.390
M/E			0.205	0.650
2–4:1	58(18.9)	27(20.8)
≠ 2–4:1	249(81.1)	103(79.2)
Myeloid (%)	20(10,33)	18(8,27)	-2.135	0.033
Lymphocyte (%)	58(37,75)	62(44,78)	2.000	0.046
Megakaryocyte (n)			0.752	0.687
0	168(54.7)	76(58.5)
1–6	98(31.9)	40(30.8)
≥ 7	41(13.4)	14(10.8)
Hematopoietic cells(%)			0.809	0.368
≥ 25	48(15.6)	16(12.3)
< 25	259(84.4)	114(87.7)
Adipocyte (%)			0.445	0.505
≤ 50	131(42.7)	51(39.2)
> 50	176(57.3)	79(60.8)

Cyclosporine time from diagnosis to cyclosporine treatment, PLT platelet, WBC white blood cell, ANC absolute neutrophil count, LYM absolute lymphocyte count, Hb hemoglobin, RC reticulocyte count, CRP C-reactive protein, ALT alanine aminotransferase, LDH lactic dehydrogenase, TB serum total bilirubin, M/E myeloid-to-erythroid ratio in bone marrow

Feature screening

This study enrolled 437 patients with 32 variables analyzed. Within the training cohort (n = 307), 295 samples (96.10%) were complete without missing values, while in the validation cohort (n = 130), 128 samples (98.46%) had no missing values. All remaining missing data were imputed using the MICC method. Evaluate the multicollinearity among candidate predictor variables. The following pairs of highly correlated variables were identified: WBC and LYM (r = 0.86); MCV and MCH (r = 0.92); ALT and AST (r = 0.94); TB and DB (r = 0.89); and Myeloid (%) and bone marrow Lymphocyte ratio (%) (r = -0.82). These correlated variable pairs were excluded from further analysis (Fig. 2)

Fig. 2.

Multicollinearity analysis of candidate variables

The LASSO algorithm was applied for variable selection.The optimal regularization parameter was determined to be λ = 20.0295 through 10-fold cross-validation. When the penalty coefficient λ increased to the threshold along the regularization path, the regression coefficients of noncritical variables were compressed to zero, which eventually reduces the number of important variables and simplifies the model. Five core predictors with clinical explanatory power were ultimately identified in this manner: reticulocyte count (RC), platelet count (PLT), disease classification (type), proportion of bone marrow myeloid cells, and total serum bilirubin (TB) level.

Model development and screening

Each model was trained on the data from the entire training cohort, and the validation cohort was used for assessment. To comprehensively evaluate model performance, we calculated the accuracy, sensitivity, specificity, F1 score, Brier score (Table S1, Table S2). Receiver operating characteristic (ROC) curve was performed to evaluate the discriminative performance of the models. The performance of ten machine learning models was compared, among which the CatBoost algorithm yielded an AUC of 0.834 (95% CI: 0.774–0.895) in the training cohort and 0.826 (95% CI: 0.743–0.910) in the validation cohort. This model performed the best in predicting the mortality of AA patients, followed by the GBM model; although the latter model exhibited an AUC of 0.846 (95% CI: 0.791–0.902) in the training cohort, its discriminative performance decreased to 0.781 (95% CI: 0.674–0.887) in the validation cohort. On the basis of the principle of maximum AUC and model stability considerations, the CatBoost algorithm was selected in this study to construct the mortality prediction model for AA children treated with cyclosporine (Fig. 3). DCA curve was performed to evaluate the value of the ten machine learning models for clinical decision making. The DCA curves shown in Fig. 4 reveal the net clinical benefit of each model under different risk thresholds. The plot shows that the net benefit curve of the CatBoost model was higher on the y-axis than that of the other algorithms, suggesting that this model has greater applicability in predicting mortality. The calibration curve compares the agreement between the predicted mortality probabilities and the actual mortality frequencies across multiple models (Fig. 5)

Fig. 3.

ROC curves of 10 models (a) training cohort, (b) validation cohort

Fig. 4.

DCA curves of 10 models (a) training cohort, (b) validation cohort

Fig. 5.

Calibration curve of 10 models (a) training cohort, (b) validation cohort

Visualization of feature importance

In this study, the tree-explainer algorithm in the SHAP framework was used to systematically analyze the prediction mechanism of the CatBoost model. The results of the multidimensional visualization analysis (Fig. 6) revealed that the five key predictors that were significantly associated with patient survival were RC, PLT, disease type (type), TB, and the proportion of myeloid cells in the bone marrow (myeloid).

Fig. 6.

SHAP explanation of Catboost model (a) feature importance ranking, (b) impact of each feature on model prediction

This picture showed the ranking of SHAP feature importance (Fig. 6a).Each dot per row represents a patient, where dot’s color represents the relative feature value: orange represents high values, and purple represents low values. The orange and purple bars represent risk factors and protective factors, respectively, and bar length is indicative of the importance of the feature. In the decomposition diagram (Fig. 6b), the positively associated factors (risk factors) and the negatively associated factors (protective factors) are distinguished by the polarity of the color band, and the strength of the characteristic effect is positively correlated with the length of the color band. Through SHAP value analysis, RC was identified as the feature with the greatest contribution to model predictions; specifically, a decrease in RC values significantly increases the model’s predicted probability of mortality risk in AA patients. Similarly, reduced platelet count, diagnosis of vSAA, elevated TB, and decreased bone marrow myeloid ratio were all validated via SHAP values as key features positively associated with mortality risk prediction.

Discussion

Cyclosporine is an indispensable core therapeutic drug for all types of AA, including SAA, vSAA and NSAA [24]. Nao Yoshida et al. retrospectively analyzed 599 pediatric severe aplastic anemia patients (age < 17 years). While overall survival did not differ between IST and bone marrow transplantation (BMT) [88% (95%CI: 86–90) vs. 92% (90–94)], IST was associated with significantly lower failure-free survival than BMT [56% (54–59) vs. 87% (85–90); P < 0.0001]. Thus, BMT is recommended for pediatric SAA when an HLA-matched family donor is available [25]. However, in real-world practice, some patients with SAA, the majority of patients with NSAA, or those who cannot receive combination therapy often receive cyclosporine monotherapy. In recent years, with the gradual application of TPO-RAs, the combination of TPO-RAs and immunosuppressants has also achieved good efficacy in the treatment of aplastic anemia. In a phase 2 ESCALATE trial, the combination of eltrombopag and cyclosporine A, with or without the addition of horse antithymocyte globulin, achieved a combined ORR of 54.9% at 26 weeks, with 71.4% and 48.6% in the R/R group and the previously untreated group [26]. A prospective phase 1–2 study involving 92 patients receiving combined IST with Eltrombopag showed that the hematological remission rate was significantly higher in patients treated with Eltrombopag plus IST than in the historical cohort [27].In the real world, however, few studies have investigated the efficacy of and prognostic factors related to cyclosporine immunosuppressive therapy alone, and most published studies involved the use of conventional statistical methods [28, 29]. At present, the increasing use machine learning algorithms in clinical research has led to the development of different types of machine learning algorithms for predicting survival risk factors. Different machine learning algorithms have their own advantages and disadvantages in survival risk prediction. Qi et al. [30] used the cell population data (CPD) parameters of 160 patients diagnosed with AA or myelodysplasia syndromes (MDS) in comparing the performance of six machine learning algorithms for predicting the probability of these diseases. Ultimately, the LR model was found to be the most suitable candidate. Seo et al. [10] successfully developed a prediction model for a diagnosis of BMFS on the basis of age, sex and complete blood count (CBC) data via the XGBoost classifier, achieving high accuracy in the differentiation of AA and MDS patients in the control group. Chang used four machines learning to develop a prediction model for the efficacy of immunosuppressant treatment in patients with SAA, and found that the white blood cell count, lymphocyte count, absolute reticulocyte count, percentage of lymphocytes in bone marrow smears, and C-reactive protein, IL-6, IL-8 and vitamin B12 levels were closely related to long-term efficacy (P < 0.05) [12].

In this study, 10 machine learning algorithms were used to develop models for predicting mortality in AA children. Following application of LASSO regression for screening features, a total of five predictors were included for model construction. The predictive performance of the 10 machine learning methods was compared, and the results indicated that the CatBoost model had the optimal predictive performance. As an efficient gradient boosting decision tree (GBDT) algorithm, CatBoost includes native support for categorical features, overfitting control techniques (ordered boosting and symmetric tree) and efficient GPU acceleration, making it a common choice for model building in the context of complex data. For tasks whose features are mainly category-based and for which stability is important, the performance of CatBoost is better than that of similar algorithms [31]. In previous studies, CatBoost models demonstrated advantages in the diagnosis, treatment, and prognostic prediction of different diseases. An et al. [32] used 5 supervised machine learning algorithms for decision-making in the intra-arterial treatment of unresectable HCC, including XGBoost, CatBoost, GBDT, LightGBM and RF. In the IC group in the test cohort, the CatBoost algorithm achieved optimal differentiation when 30 input variables were used in model construction, with an AUC of 0.776 (95% confidence interval [CI], 0.833–0.868) [31]. Huang used 10 machine learning algorithms, including CatBoost, RF, SVM, NN, GBM, KNN, MLP, naive Bayes (NB), XGBoost, and LR to construct predictive models. The results showed that the CatBoost model had the best performance in predicting mortality in elderly patients with severe ischemic stroke admitted to the ICU [33]. However, at present, there is a lack of mortality prediction model for AA after IST. In this study, the AUC of the CatBoost model in the validation set was 0.826. On the basis of its sensitivity and specificity, the CatBoost model could be used to predict mortality in AA patients after immunotherapy.

SHAP is a model interpretation method based that aims to fairly distribute the contribution of each feature to the model prediction. The SHAP value of a feature is determined by calculating the mean contribution margin of all possible feature combinations. By visualizing the results, the SHAP system not only transforms the interpretability of the model decision process but also provides a biomarker basis with causal inference value for clinical decision-making through the explicit presentation of the feature interaction effect [34]. SHAP analysis was performed to validate the performance and improve the clinical interpretability of the model; the results can help doctors better understand the decision process of the model and thus encourage the use of the prediction results. Although global hematopoietic failure characterizes AA, substantial residual HSPC heterogeneity persists across patients [35, 36]. Patients harboring greater HSPC reserves exhibit enhanced responses to IST (e.g., ATG) or hematopoietic agents (e.g., Eltrombopag), demonstrating superior survival [37].

In this study, SHapley Additive exPlanations (SHAP) were employed to interpret the predictive logic of the CatBoost model. Cross-model feature importance consensus analysis identified reticulocyte count (RC), platelet count (PLT), disease subtype (Type), total bilirubin (TB), and bone marrow myeloid proportion as key variables associated with survival outcomes. SHAP value decomposition confirmed RC as the top-ranked feature, with lower RC values driving significantly higher predicted mortality risk in AA patients. Consistent with this pattern, SHAP validation further revealed that decreased PLT counts, presence of very severe AA (vSAA), elevated TB levels, and reduced bone marrow myeloid proportion were each linked to increased predicted death risk.

This study was designed as a single-center retrospective investigation, which may introduce selection bias. Specifically, the patient cohort predominantly comprises severe cases, potentially leading to overestimation of the model’s predictive performance in non-severe cases. In the retrospective setting, the completeness of variable collection relies on electronic medical records and telephone follow-up, and partial missing data may compromise model robustness. Due to data accessibility constraints, external validation has not yet been performed; however, internal stability was evaluated via 10-fold cross-validation, and multicenter collaboration for external validation is planned for future studies. IST response, transfusion dependency, clonal evolution, and relapse are key prognostic factors in aplastic anemia. However, these variables were not evaluated in this study. As a retrospective analysis spanning a long period, most patients lacked regular follow-up. Future work will focus on enhancing the long-term management of aplastic anemia patients, especially post-discharge follow-up protocols, to minimize loss to follow-up.

Conclusion

The CatBoost model was shown to have the best prediction performance for mortality in AA patients after cyclosporine treatment with an AUC of 0.834 in training and 0.826 in validation. This model provides clinicians with an individualized prognostic tool to stratify patients into distinct mortality risk groups, guiding early intervention strategies (e.g., escalation to combined immunosuppressive therapy or hematopoietic stem cell transplantation for high-risk patients), thereby potentially improving long-term survival outcomes and reducing treatment-related complications.

Supplementary Information

Below is the link to the electronic supplementary material.

Author contributions

X.W. designed the study, managed the patients and wrote the manuscript; L.X. performed statistical analysis and model construction. D. L. performed the data collection and proofreading. M.L.performed the data collection and proofreading. Y. L., Q. L., X. G. and Y. D. managed the patients and follow-up of patients. Z.H.designed the study and revised the manuscript. All authors read and approved the final manuscript.

Funding

This work was supported by the Medical Research Project of Chongqing Municipal Health Commission, 2024WSJK008).

Data availability

Data supporting the findings of this study are presented in this article and Supplementary material. For further inquiries, please contact the corresponding author Ziyu Hua, at h_ziyu0517@163.com.

Declarations

Competing interests

The authors declare no competing interests.