Comprehensive validation of machine learning models predicting chemotherapy related electrolyte disorders in a multicenter study

Abstract

Background

Electrolyte abnormalities following chemotherapy are common and clinically significant complications in cancer patients and are often associated with treatment delays and adverse outcomes. This study aims to develop and validate machine learning models to predict eight electrolyte abnormalities in cancer patients receiving chemotherapy.

Methods

We retrospectively analyze medical records of cancer patients from two tertiary hospitals in Korea (n = 11,227). Four machine learning algorithms are used to predict eight electrolyte abnormalities occurring within 4 weeks of chemotherapy initiation. Model performance is evaluated using a comprehensive validation framework, including internal, external, and temporal validation, and model interpretability is assessed using Shapley additive explanations.

Results

Here we show that electrolyte abnormalities occur in 4451 patients (74.0%) in the internal cohort and 4414 patients (84.7%) in the external cohort, with in-hospital mortality rates of 35.9% and 32.8%, respectively. The best-performing models achieve an average area under the receiver operating characteristic curve of 0.798, with an average performance decline of 0.11 during external validation. Model interpretation identifies serum albumin, heart rate, and estimated glomerular filtration rate as the most important predictors across models. Risk stratification demonstrates that patients in the highest-risk quintile have 4.15-fold greater odds of developing electrolyte abnormalities, and a 2.02-fold higher risk of mortality compared with the moderate-risk.

Conclusions

These machine learning models provide an effective approach for predicting electrolyte abnormalities in cancer patients receiving chemotherapy and may support risk stratification and monitoring prioritization. Future studies are needed to validate these models in more diverse populations, incorporate additional biomarkers, and explore their integration into clinical decision-support systems.

Cho, Jeong et al. develop and validate machine learning models to predict chemotherapy-related electrolyte abnormalities using multicenter real-world clinical data. The models accurately stratify patients by risk, show stable performance across validation settings, and identify individuals at higher risk of electrolyte disturbances and mortality.

Plain language summary

Cancer patients who receive chemotherapy often develop problems with body salts, such as sodium or potassium imbalances. These problems can delay treatment and increase the risk of serious complications. However, it is difficult to know in advance which patients are most likely to be affected. In this study, we analyzed medical records from over 11,000 cancer patients treated at two large hospitals in Korea. We used computer-based prediction models to estimate the risk of developing electrolyte problems within 4 weeks after starting chemotherapy. The models were carefully tested using data from different time periods and hospitals. We found that the models could identify patients at higher risk of electrolyte problems and death. These findings may help doctors monitor vulnerable patients more closely and prevent complications during cancer treatment.

Introduction

Onconephrology plays an increasingly important role in preventing and managing kidney-related complications commonly observed in cancer patients, including renal failure, proteinuria, and electrolyte disorders^1,2. However, advances in cancer treatment have exposed limitations in existing research, highlighting the need for new evidence and contributing to insufficient clinical guidelines^3,4. Consequently, the relationship between cancer therapy and kidney disease remains critically underexplored, posing significant challenges for optimal patient care^5,6.

In this study, electrolyte disorders refer to chemotherapy-related electrolyte abnormalities identified based on laboratory criteria. Among kidney-related complications in cancer patients, electrolyte abnormalities occur frequently in those receiving chemotherapy and may be life-threatening⁷. These electrolyte abnormalities arise through multiple mechanisms, including direct effects of cancer, dehydration, nutritional imbalance, direct kidney toxicity from anticancer drugs, tumor lysis syndrome, and cytotoxic effects of chemotherapy^8,9. Electrolyte abnormalities may lead to treatment delays and discontinuation, affecting clinical outcomes. Therefore, early detection or prediction is crucial, as anticipating acute electrolyte abnormalities after chemotherapy may help improve treatment adherence.

As cancer survivorship continues to increase, early prediction and proactive management have become increasingly important^10,11. Recent machine learning (ML) approaches in healthcare offer new opportunities for early risk prediction and proactive management^12,13. Although various ML methods have been proposed to predict electrolyte abnormalities^14–17, most existing studies focus on non-cancer populations, require additional diagnostic tests, or are restricted to specific anticancer agents, cancer types, or single electrolyte disorders, limiting their clinical applicability.

Moreover, current ML research on electrolyte prediction, as well as in broader medical applications, exhibits substantial methodological limitations in model validation^18,19. Several studies have demonstrated that many published models lack essential quality assurance elements^20,21. External validation, essential for establishing generalizability, has been performed in less than 1% of published ML studies, and temporal stability is rarely evaluated. In addition, calibration analysis and decision curve analysis (DCA) are infrequently conducted, limiting evaluation of model reliability and clinical utility^22,23. Performance across clinically relevant patient subgroups is also insufficiently examined, hindering the development of personalized risk prediction models²⁴.

These methodological limitations not only compromise individual study validity but also hinder clinical implementation of machine learning, creating barriers to practical application²⁵. Rigorous validation of predictive models is particularly critical in cancer patients^26–28. The complexity of cancer treatment and substantial inter-individual variability make model generalizability essential. In addition, prediction errors may directly influence treatment decisions, underscoring the importance of model reliability. Furthermore, cancer patients frequently have multiple comorbidities and drug interactions, necessitating stable and consistent predictive performance.

Here, we develop and validate machine learning models to predict eight chemotherapy-related electrolyte abnormalities using multicenter real-world data. We demonstrate that these models achieve stable performance across internal, external, and temporal validations, and reliably stratify patients according to clinical risk. By incorporating calibration, decision curve analysis, and subgroup evaluation, this study highlights the importance of comprehensive validation for clinically applicable machine learning models in onconephrology. The overall study workflow is summarized in Fig. 1.

Fig. 1. Study workflow and clinical translation pathway for predicting chemotherapy-related electrolyte abnormalities.

Figure 1 shows the overall study workflow and intended clinical translation pathway. The study included internal and external patient cohorts, routinely available baseline clinical features, and multiple machine learning models with internal, external, temporal, and subgroup validation. Model outputs were stratified into categorical risk groups and linked to a web-based prediction tool to support risk stratification and monitoring prioritization in clinical practice.

Methods

Study population

This retrospective cohort study included patients admitted to Korea University Anam Hospital between 2015 and 2021 who received chemotherapy. For external validation, additional data were obtained from patients admitted to Soonchunhyang University Cheonan Hospital between 2016 and 2021 with a record of chemotherapy administration. For patients with multiple admissions or repeated chemotherapy courses, the index date was defined as the initiation of the first chemotherapy regimen during the study period. Subsequent admissions were not treated as independent observations.

Inclusion criteria were as follows: (1) age ≥19 years; (2) documented diagnosis of malignant neoplasm (ICD-10 codes C00–C97); and (3) at least one chemotherapy prescription during hospitalization. Exclusion criteria included: (1) patients diagnosed solely with metastatic cancer (ICD-10 codes C77–C79) without a primary malignancy; (2) observation periods shorter than 4 weeks; and (3) the presence of any electrolyte abnormality within 24 h of the initial chemotherapy administration, regardless of type. After applying these criteria, 6015 patients from the internal cohort and 5212 from the external cohort were included in the final analysis (Fig. 2).

Fig. 2.

Flowchart of patient inclusion and exclusion criteria used in the final cohort selection.

The study was approved by the Institutional Review Board of Korea University Anam Hospital (IRB No. 2023AN0145) and the Institutional Review Board of Soonchunhyang University Cheonan Hospital (IRB No. 2019-10-023). The requirement for informed consent was waived by both institutional review boards due to the retrospective study design and the use of de-identified electronic medical records.

Outcome definition

The primary prediction task was the occurrence of eight binary electrolyte abnormalities within 4 weeks of the initial chemotherapy prescription. These electrolytes are routinely and frequently measured during hospitalization, and the 4-week prediction window was selected to capture acute chemotherapy-related effects while minimizing bias associated with longer observation periods. Electrolyte abnormalities were defined as shown in Table 1.

Table 1.

Definitions of the eight electrolyte abnormalities

Outcome	Definition
Hypercalcemia	corrected calcium >10.5 mg/dL
Hypocalcemia	corrected calcium <8.5 mg/dL
Hyperkalemia	serum potassium >5.5 mEq/L
Hypokalemia	serum potassium <3.5 mEq/L
Hyperphosphatemia	serum phosphate >4.5 mg/dL
Hypophosphatemia	serum phosphate <2.5 mg/dL
Hypernatremia	serum sodium >145 mEq/L
Hyponatremia	serum sodium <135 mEq/L

Corrected calcium (mg/dL) = Total calcium + 0.8 × (4.0 − serum albumin in g/dL).

Data collection and preprocessing

Demographic data, vital signs, laboratory results, and chemotherapy information were extracted from each institution’s hospital information system. Cancer types were categorized into 12 groups (liver, thyroid, colorectal, biliary/pancreatic, bladder, gastric, breast, cervical, prostate, lung, renal, other) using ICD-10 codes.

For each patient, baseline vital signs and laboratory values were defined as the measurements obtained closest to chemotherapy initiation within the preceding 4 weeks. To reflect acute physiological changes, the maximum value of each vital sign within 24 h post-chemotherapy was also collected. Estimated glomerular filtration rate (eGFR) was calculated using the CKD-EPI 2021 equation²⁹. Only chemotherapeutic agents prescribed at least 10 times at both hospitals were analyzed. Data preprocessing and model development were performed with strict separation between the internal and external cohorts to prevent information leakage. In the internal cohort, variables with ≥30% missingness were excluded from the analysis, and the remaining missing values were imputed using Multiple Imputation by Chained Equations (MICE). Continuous variables were standardized using a Robust Scaler fitted to the internal dataset. For external validation, the external cohort was processed using the same set of variables selected from the internal cohort. The MICE imputation model and scaling parameters learned from the internal cohort were applied directly to the external dataset without refitting. To address class imbalance, class weights inversely proportional to outcome prevalence were applied during model training. This procedure ensured that no information from the external cohort influenced model development and effectively prevented data leakage.

Machine learning model development

Four commonly used machine learning algorithms—Logistic Regression (LR), XGBoost (XGB), CatBoost (CAT), and Multi-Layer Perceptron—were evaluated to develop prediction models. These algorithms were selected to provide a range of model types with differing complexity and are widely applied in clinical prediction studies. For each of the eight electrolyte abnormalities, separate models were constructed for hypo- and hyper-states, resulting in a total of 32 predictive models.

Model discrimination was primarily assessed using the area under the receiver operating characteristic curve (AUROC). Given the presence of substantial class imbalance across several electrolyte abnormalities, additional confusion matrix-derived metrics, including sensitivity, specificity, positive predictive value, negative predictive value, and F1 score, were evaluated to provide a more clinically interpretable assessment of model error profiles. A single operating threshold was determined a priori using the Youden index based on internal out-of-fold predictions and was subsequently fixed and applied unchanged during external validation to enable transparent comparison of performance across cohorts.

Hyperparameters for each algorithm were predefined based on prior internal optimization and were fixed before validation. For each electrolyte abnormality, a single representative model was selected based on internal validation performance and subsequently evaluated using the same fixed configuration across all validation settings. For model interpretability, SHapley Additive exPlanations (SHAP) values were computed to examine feature contributions to model predictions. SHAP analysis was performed exclusively on the final representative model trained on the internal cohort. This analysis was intended to provide descriptive insight into model decision patterns and relative feature contributions, rather than to infer causality or identify generalizable biomarkers.

Furthermore, predicted risks of experiencing at least one electrolyte abnormality (across the eight modeled electrolytes) within 4 weeks of chemotherapy initiation were aggregated and stratified into five equal-sized risk groups (very low, low, moderate, high, and very high). Quintile-based stratification was chosen to ensure balanced group sizes and to facilitate clinically interpretable comparison across increasing levels of predicted risk without assuming a specific risk threshold. Associations between predicted risk groups, electrolyte abnormalities, and all-cause mortality were primarily evaluated using odds ratios (ORs), with the moderate-risk group serving as the reference. Although time-to-event analysis using Cox proportional hazards models may be appropriate in some settings, we adopted an OR-based approach to minimize interpretational bias arising from uncertain outcome timing and label ambiguity beyond the predefined 4-week prediction window. By restricting outcomes to a fixed short-term horizon and using ORs, we aimed to provide a more conservative assessment of risk associations and to avoid overinterpretation of temporal effects. Hazard ratios (HRs) were additionally calculated as supportive analyses for mortality outcomes.

Comprehensive validation framework

To ensure clinical applicability, we employed a four-dimensional comprehensive validation framework. Internal validation was conducted using five-fold stratified cross-validation, and external validation assessed generalizability using a geographically distinct cohort. Temporal stability was evaluated by assessing model performance across calendar years (2015–2021), with 2019 used as a cutoff to define early (≤ 2018) and late (≥ 2019) periods based on changes in cancer treatment patterns following the widespread adoption of immune checkpoint inhibitors (ICI). Subgroup analyses were performed across age (< 65 vs. ≥65 years), sex (female vs. male), and cancer type (subgroups with ≥10 cases). Model calibration and clinical utility were evaluated using the Brier score, Expected Calibration Error (ECE), and decision curve analysis comparing net benefit against treat-all and treat-none strategies³⁰.

Statistics and reproducibility

Descriptive statistics and univariate comparisons were performed for each of the eight electrolyte abnormalities. For patients with an event, chemotherapy exposure and clinical variables were evaluated within 4 weeks of the initial abnormality, whereas for non-event patients, median values across the observation period were used. Between-group comparisons were conducted using two-tailed t-tests for continuous variables and chi-square or Fisher’s exact tests for categorical variables, as appropriate. Variables with an expected count of fewer than five in any cell were excluded from analysis.

Within the internal cohort, ORs were calculated for each chemotherapeutic agent associated with electrolyte abnormalities occurring within 4 weeks of administration. Agents with fewer than ten prescriptions or electrolyte outcomes with fewer than 100 events were excluded from this analysis. Multivariate logistic regression and Cox proportional hazards models were adjusted for sex, age, body mass index, cancer type, Charlson Comorbidity Index, baseline creatinine, and albumin. For calcium-related models, corrected calcium was used, and albumin was not included as a covariate.

Each patient was treated as a single independent observation, with the index date defined as the first chemotherapy initiation during the study period. Sample sizes for the internal (n = 6015) and external (n = 5212) cohorts were determined by predefined inclusion and exclusion criteria. Model development employed five-fold stratified cross-validation within the internal cohort, while generalizability was assessed using an independent external cohort and temporal validation across calendar years. All preprocessing, model training, and validation procedures were predefined and applied consistently to ensure reproducibility of the analyses. All analyses were conducted using Python version 3.8 and R version 4.1.1. Statistical significance was defined as a two-sided p value < 0.05.

Results

Study population and baseline characteristics

In-hospital mortality was 35.9% (2156 of 6015) and 32.8% (1709 of 5212) in the internal and external cohorts, respectively. Information on administered chemotherapeutic agents, variable-level missingness in the internal and external datasets, patient characteristics, descriptive statistics, and the incidence of individual electrolyte abnormalities is summarized in Table S1 and Supplementary Data 1. In the internal and external cohorts, 47.9% (2879 of 6015) and 47.1% (2452 of 5212) of patients were female, respectively, and the mean age was 60.3 ± 12.5 years and 61.4 ± 12.7 years. During the follow-up period, 4451 patients (74.0%) in the internal cohort and 4414 patients (84.7%) in the external cohort developed at least one electrolyte abnormality.

The most common malignancies in the internal cohort were colorectal cancer and breast cancer, followed by lung cancer and liver cancer. In contrast, gastric cancer and colorectal cancer were most prevalent in the external cohort (Fig. S1A). Patterns of anticancer drug use also varied across institutions. In the internal cohort, the most frequently prescribed agents were fluorouracil, oxaliplatin, and doxorubicin, followed by cisplatin and cyclophosphamide. In the external cohort, cisplatin and fluorouracil were most common, followed by doxorubicin, cyclophosphamide, and docetaxel (Fig. S1A). Electrolyte abnormalities most frequently involved hyponatremia and hypokalemia (Fig. S1B). Cisplatin was associated with hyponatremia in 72.7% of patients in the internal cohort and 62.2% in the external cohort. Similarly, 54.5% and 46.1% of fluorouracil-treated patients developed hyponatremia in the respective cohorts. Drug-specific adjusted odds ratios for the 4-week incidence of electrolyte abnormalities showed that fluorouracil was associated with lower odds of hyponatremia (OR 0.48, 95% CI 0.40–0.58). In contrast, anthracyclines, including daunorubicin (OR 21.30) and idarubicin (OR 17.02), were associated with higher odds of electrolyte abnormalities (Fig. S2).

Model performance and validation

Table 2 summarizes the best-performing models in internal validation. Complete results are provided in Supplementary Table S2. XGB achieved the highest performance for hypernatremia in internal validation (0.738), whereas logistic regression demonstrated better generalizability externally (0.620). For hyponatremia, logistic regression consistently outperformed other approaches in both internal (0.799) and external (0.774) evaluations.

Table 2.

Summary of AUROC-based performance of machine learning models for each of the eight electrolyte abnormalities

Target	Model	Base	Internal	External
Hypercalcemia	CAT	0.678 (0.597–0.759)	0.907 (0.819–0.994)	0.930
Hypocalcemia	LR	0.691 (0.663–0.719)	0.781 (0.750–0.812)	0.657
Hyperkalemia	LR	0.626 (0.535–0.717)	0.791 (0.662–0.920)	0.773
Hypokalemia	LR	0.705 (0.688–0.723)	0.785 (0.765–0.806)	0.674
Hyperphosphatemia	LR	0.711 (0.691–0.730)	0.753 (0.733–0.774)	0.653
Hypophosphatemia	LR	0.728 (0.712–0.745)	0.826 (0.810–0.842)	0.672
Hypernatremia	XGB	0.588 (0.530–0.646)	0.738 (0.574–0.902)	0.534
Hyponatremia	LR	0.681 (0.671–0.692)	0.799 (0.722–0.826)	0.774

Base indicates model performance without hyperparameter tuning, presented as mean and 95% confidence interval where applicable. Internal and External refer to validation performances in the internal and external cohorts, respectively.

CAT categorical boosting, LR logistic regression, XGB extreme gradient boosting, AUROC area under the receiver operating characteristic curve.

LR exhibited the most consistent performance for hyperkalemia across both cohorts, with AUROC values of 0.791 internally and 0.773 externally. This model was also optimal for hypokalemia prediction within the internal dataset (0.785; 95% CI, 0.765 to 0.806), although CAT demonstrated better generalizability in external validation (0.691).

For phosphate abnormalities, logistic regression achieved the best results for hyperphosphatemia prediction internally (0.753; 95% CI, 0.733 to 0.774), while CAT achieved superior external performance (0.683). Hypophosphatemia prediction followed a similar pattern, with logistic regression reaching 0.826 in internal validation and CAT slightly outperforming in external testing (0.673).

Overall, ML models demonstrated substantial improvements of 0.08 to 0.20 over baseline approaches in internal validation. Despite expected performance attenuation across cohorts, these models retained meaningful advantages of 0.05 to 0.15 in external validation.

Across the 7-year study period (2015 to 2021), model performance remained largely stable with minimal variation. The standard deviation of yearly AUROCs was 0.02 (Fig. 3A). The mean AUROC difference between the early period ( ≤2018) and the late period (≥2019) was only 0.009 (range, 0.000 to 0.091; P = 0.723), indicating no statistically significant temporal degradation (Fig. 3B).

Fig. 3. Temporal validation of electrolyte abnormality prediction models over a 7-year period (2015–2021).

Temporal validation of electrolyte abnormality prediction models over a 7-year period (2015–2021). Early period includes years 2015–2018; late period includes years 2019–2021. n = 56 biologically independent model evaluations. A Annual trends in model performance, measured by the area under the receiver operating characteristic curve (AUROC), for each of the eight electrolyte abnormalities. Each point represents the AUROC for a given electrolyte abnormality in a specific calendar year. B Comparison of model performance distributions between the early period (≤ 2018) and late period (≥ 2019). Boxplots show the median (center line) and interquartile range (box), with whiskers indicating 1.5× the interquartile range; points represent individual annual AUROC values. Comparisons between periods were performed using a two-sided Welch’s t-test, with no adjustment for multiple comparisons. AUROC, area under the receiver operating characteristic curve.

In addition to AUROC, threshold-based performance metrics derived from confusion matrices are summarized in Table 3, providing transparent insight into sensitivity–specificity trade-offs and positive-class performance under class imbalance. As expected under substantial class imbalance, negative predictive value was consistently high across all electrolyte abnormalities (0.887–0.999), reflecting strong performance in ruling out short-term events. In contrast, positive predictive value remained low (0.036–0.482), indicating a high false-positive rate at the selected operating thresholds.

Table 3.

Threshold-based performance metrics of representative models

Target	Threshold	Cohort	Sensitivity	Specificity	PPV	NPV	F1 score
Hypercalcemia	0.005	Internal	0.875	0.843	0.046	0.999	0.088
		External	0.889	0.849	0.036	0.999	0.069
Hypocalcemia	0.400	Internal	0.795	0.638	0.161	0.973	0.268
		External	0.685	0.530	0.092	0.960	0.161
Hyperkalemia	0.344	Internal	0.816	0.645	0.078	0.995	0.144
		External	0.779	0.627	0.053	0.991	0.099
Hypokalemia	0.562	Internal	0.661	0.811	0.311	0.949	0.419
		External	0.453	0.775	0.216	0.912	0.292
Hyperphosphatemia	0.418	Internal	0.763	0.623	0.166	0.964	0.273
		External	0.725	0.449	0.104	0.949	0.182
Hypophosphatemia	0.470	Internal	0.777	0.736	0.276	0.966	0.411
		External	0.519	0.735	0.188	0.928	0.276
Hypernatremia	0.408	Internal	0.550	0.859	0.046	0.994	0.084
		External	0.401	0.854	0.076	0.979	0.128
Hyponatremia	0.477	Internal	0.722	0.737	0.482	0.887	0.579
		External	0.706	0.695	0.301	0.927	0.422

Table 3 presents confusion matrix-derived performance metrics of the representative model for each electrolyte abnormality, evaluated at a fixed operating threshold determined a priori from internal out-of-fold predictions and applied unchanged to the external cohort to allow transparent assessment under class imbalance.

Calibration, subgroup analysis, and clinical utility

Calibration performance was strongest for hypercalcemia (Brier score: 0.005; ECE: 0.002), followed by hypernatremia and hyperkalemia. In contrast, calibration was suboptimal in high-prevalence conditions such as hyponatremia (Brier score: 0.178; ECE: 0.192), hypophosphatemia (Brier score: 0.161), and hypocalcemia (Brier score: 0.174). DCA demonstrated clinically meaningful net benefit for several models, particularly for hyponatremia (maximum net benefit: 0.221; threshold probability range: 0.01 to 0.37), followed by hypokalemia (0.088), hypophosphatemia (0.074), and hyperphosphatemia (0.068). Models for hyperkalemia and hypernatremia showed minimal net clinical benefit (Table S3).

Subgroup analyses revealed improved model performance in patients younger than 65 years, particularly for hypercalcemia (AUROC 0.953 vs. 0.821) and hyperkalemia (0.821 vs. 0.686). Female patients demonstrated superior predictive performance for most targets, including hypercalcemia (AUROC 0.897 vs. 0.887), hypokalemia, and hyponatremia, whereas male patients performed better in predicting hypernatremia (AUROC 0.734 vs. 0.615) and hyperphosphatemia (0.767 vs. 0.734). Performance also varied by cancer type. For hypercalcemia, near-perfect AUROCs were observed in patients with liver (0.990), bladder (0.990), breast (0.999), and cervical cancer (0.988). In contrast, prediction for hypernatremia in patients with liver cancer was notably poor (AUROC 0.217) (Table 4).

Table 4.

Model performance across major patient subgroups stratified by age, sex, and cancer type

Subgroup	Sodium		Potassium		Phosphate		Calcium
	Hyper	Hypo	Hyper	Hypo	Hyper	Hypo	Hyper	Hypo
Age <65	0.953	0.773	0.821	0.794	0.763	0.835	0.687	0.815
Age ≥65	0.821	0.789	0.686	0.764	0.709	0.802	0.718	0.751
Male	0.887	0.781	0.761	0.767	0.767	0.818	0.734	0.774
Female	0.897	0.779	0.786	0.799	0.734	0.831	0.615	0.814
Colon	0.909	0.741	0.740	0.721	0.702	0.787	0.614	0.736
Liver	0.990	0.667	0.807	0.632	0.530	0.725	0.217	0.713
Thyroid	–	0.856	1.000	0.807	0.773	0.703	–	0.779
GB_Pancreas	–	0.719	0.510	0.732	0.702	0.761	–	0.774
Bladder	0.990	0.616	–	0.678	0.527	0.814	0.889	0.747
Stomach	–	0.663	0.668	0.739	0.635	0.833	0.896	0.767
Breast	0.999	0.644	–	0.709	0.573	0.709	0.572	0.741
Cervix	0.988	0.765	0.865	0.786	0.524	0.644	–	0.693
Prostate	0.677	0.581	–	0.848	0.778	0.707	0.431	0.700
Lung	0.877	0.718	0.700	0.658	0.611	0.708	–	0.790
Others	0.795	0.741	0.787	0.724	0.664	0.774	0.727	0.772

Model performance is presented as AUROC values across major patient subgroups stratified by age, sex, and cancer type. Missing values (–) indicate subgroup-model combinations with insufficient events for evaluation.

Cancer types: Colon, colon cancer; Liver, liver cancer; Thyroid, thyroid cancer; GB_Pancreas, gallbladder and pancreatic cancer; Bladder, bladder cancer; Stomach, stomach (gastric) cancer; Breast, breast cancer; Cervix, cervical cancer; Prostate, prostate cancer; Lung, lung cancer; Others, all other cancer types not categorized above.

AUROC area under the receiver operating characteristic curve.

Patients were stratified into five risk groups based on their predicted probability of developing at least one electrolyte abnormality within 4 weeks of chemotherapy. The incidence of electrolyte abnormalities increased progressively across risk groups. Compared with the moderate-risk group (reference), patients in the highest risk quintile had 4.15-fold higher odds of experiencing at least one electrolyte abnormality (95% CI, 3.66 to 4.70; P < 0.001), whereas those in the lowest quintile had significantly lower odds (OR, 0.39; 95% CI, 0.34 to 0.46) (Fig. 4A).

Fig. 4. Risk stratification of patients based on predicted electrolyte abnormality risk.

Risk stratification of patients based on the predicted probability of developing at least one electrolyte abnormality within 4 weeks of chemotherapy initiation. Patients were categorized into five risk groups: very low, low, moderate, high, and very high. Error bars in the forest plots indicate 95% confidence intervals. Points represent adjusted odds ratios (Fig. 4A) or adjusted hazard ratios (Fig. 4B), with the moderate-risk group serving as the reference. Sample sizes for each risk group are shown below the x-axis; n = 11,227 biologically independent patients. A The top panel illustrates the incidence proportion of at least one electrolyte abnormality across the five stratified risk groups. The bottom panel presents adjusted odds ratios with 95% confidence intervals for electrolyte abnormality occurrence, using the moderate-risk group as the reference, displayed as a forest plot. B The top panel shows the incidence proportion of all-cause mortality within each risk group. The bottom panel displays adjusted hazard ratios with 95% confidence intervals for all-cause mortality, using the moderate-risk group as the reference. CI, confidence interval

The five risk groups were further evaluated for their association with all-cause mortality. Patients in the highest predicted risk group had 2.02-fold higher odds of all-cause mortality compared with the moderate-risk group (95% CI, 1.75 to 2.33). In supportive time-to-event analyses, patients in the lowest-risk quintile demonstrated a significantly reduced risk of mortality (hazard ratio [HR], 0.27; 95% CI, 0.22 to 0.34) (Fig. 4B).

Model interpretability

SHAP-based analyses showed that serum albumin was the most consistently important feature across models, followed by heart rate and renal function–related markers, including blood urea nitrogen, creatinine, and eGFR. Lower albumin levels were generally associated with higher predicted risk across electrolyte abnormalities, with the exception of hypocalcemia and hyperphosphatemia. Heart rate contributed heterogeneously across outcomes, with lower values associated with hypernatremia. Renal function markers were prominently learned in models predicting potassium-, calcium-, and phosphate-related abnormalities. Several chemotherapy agents, including cisplatin, fluorouracil, oxaliplatin, and docetaxel, were identified as influential features in specific models. Cancer type was consistently important but excluded from SHAP summary due to interpretative complexity. SHAP summary plots are presented in Fig. S3.

Discussion

Although serum electrolyte measurements are rapid, inexpensive, and accurate, they are inherently episodic and reflect only a patient’s status at the time of testing. In routine clinical practice, electrolyte monitoring is performed at discrete intervals, leaving unobserved periods during which clinically significant abnormalities may develop. In this context, prediction models are not intended to replace laboratory testing, but rather to complement it by providing early warning signals during monitoring gaps, supporting pre-treatment risk stratification, and prioritizing patients who may benefit from closer surveillance. The models developed in this study are intended to support risk stratification and monitoring prioritization, enabling stratification of patients according to short-term risk and prioritization of those who may benefit from closer surveillance.

Based on this clinical need, we developed machine learning models to predict chemotherapy-related electrolyte abnormalities. In internal validation, these models demonstrated favorable discrimination performance. Hypercalcemia prediction showed the highest performance with an AUROC of 0.907 (95% CI, 0.819–0.994), while hypophosphatemia and hyponatremia predictions achieved excellent performance with AUROCs of 0.826 (95% CI, 0.810–0.842) and 0.799 (95% CI, 0.722–0.876), respectively.

Clinical applicability was suggested in risk stratification results. Risk stratification showed that patients in the highest-risk quintile had a 4.15-fold higher odds of developing electrolyte abnormalities and a 2.02-fold higher mortality risk compared with the moderate-risk group. This emphasizes the model’s ability not only to predict biochemical abnormalities but also to identify patients at risk for adverse clinical outcomes. Timely identification of high-risk patients enables proactive interventions to improve outcomes. These models may support clinical decision-making by identifying high-risk patients for enhanced monitoring, with the potential to reduce preventable electrolyte-related complications and treatment delays^31–33.

Compared to baseline models including only age, sex, and cancer type, ML models showed an average AUROC improvement of 0.098 (range: 0.052–0.187), validating the value of multidimensional clinical data. These findings support a precision medicine paradigm that integrates vital signs, laboratory values, and treatment data for individualized prediction and management strategies^34,35.

As expected under substantial class imbalance, negative predictive value was consistently high, indicating reliable identification of patients unlikely to experience short-term electrolyte abnormalities. In contrast, positive predictive value remained low, reflecting the low prevalence of events and the trade-offs inherent to threshold-based classification under severe class imbalance, where acceptable ranking performance does not necessarily translate into effective identification of true positive cases. In this context, the models appear more suitable for risk stratification and exclusion of low-risk patients than for definitive identification of positive cases at fixed operating thresholds. Accordingly, they should be interpreted as probabilistic prioritization tools rather than stand-alone diagnostic systems. SHAP-based analyses highlighted recurring patterns in how the models utilized available clinical variables, with serum albumin, vital signs, and renal function markers consistently contributing to risk estimation across outcomes. These findings reflect associations learned from structured clinical data rather than causal explanations of electrolyte pathophysiology.

Cancer treatment methods and medications can change over time. A representative example is ICI, which have been actively used since 2019. Therefore, we performed temporal validation using 2019 as the cutoff. The developed models demonstrated robust temporal stability, with annual AUROC standard deviation of 0.02 and mean difference of 0.009 between early (≤2018) and late (≥2019) periods (P = 0.723).

In contrast, external validation showed varied patterns of performance degradation (AUROC decrease 0.05–0.20), with hypercalcemia (AUROC 0.907 vs. 0.930) and hyperkalemia (AUROC 0.791 vs. 0.773) maintaining excellent external performance, while hypernatremia showed the greatest decline (AUROC 0.738 vs. 0.534). These findings contrast with several recent electrolyte abnormality prediction studies. In 2021, Kwon et al.’s ECG-based study of approximately 90,000 general patients showed stable performance decrease of only 0.05–0.08 on average, from internal validation AUROC 0.866–0.945 to external validation 0.813–0.873³⁶. In 2024, An et al.’s ECG-based potassium abnormality prediction study of approximately 330,000 patients showed stable performance with internal validation AUROC 0.916–0.941 and external validation 0.914–0.937³⁷. However, most studies do not include external validation, making systematic comparison impossible, and ECG-based studies aim to non-invasively identify electrolyte abnormalities when they occur, which differs from our study’s purpose.

Beyond external validation, calibration analysis and decision curve analysis are necessary due to the complexity of clinical decision-making and patient safety considerations^26,30. In model calibration analysis, hypercalcemia showed the best calibration performance (Brier score 0.005, ECE 0.002). In contrast, hypocalcemia and hyperphosphatemia showed relatively poor calibration (ECE 0.310, 0.324). These differences in calibration performance directly affect the reliability of model-predicted probabilities. When clinicians receive a prediction that “this patient has an 85% probability of developing electrolyte abnormalities,” similar probability patient groups should actually show approximately 85% incidence rates. Poorly calibrated models can mislead clinical decision-making despite high AUROC through inaccurate probability estimates. DCA results further clarify this complexity. The hyponatremia model achieved the highest net benefit (0.221) and widest threshold probability range (0.01–0.37), showing potential for decision support across various clinical situations. Hypokalemia (net benefit 0.088, threshold probability 0.03–0.10), hypophosphatemia (0.074, 0.01–0.09), hyperphosphatemia (0.068, 0.01–0.07), and hypocalcemia (0.058, 0.04–0.06) models showed modest net benefits within restricted threshold ranges, suggesting potential clinical usefulness only under carefully selected operating conditions. Notably, hypercalcemia showed low net benefit (0.003) despite excellent calibration, while hyperkalemia and hypernatremia models failed to achieve clinically useful net benefits.

These findings highlight that favorable discrimination performance alone may overestimate actionable clinical utility, particularly in low-prevalence settings where threshold-based metrics such as precision and F1-score remain limited^38,39. The hypercalcemia model shows near-perfect calibration performance, but due to the rare disease characteristics, using the model provides only minimal improvement compared to a “treat none” strategy. This directly relates to cost-effectiveness considerations when implementing models in actual healthcare settings. Clinical decision-making requires differential approaches based on individual patient risk levels, beyond simply discriminating disease presence or absence. For example, patients with 10% risk versus 90% risk require completely different monitoring strategies and treatment intensities. Therefore, model-predicted probabilities must be proportional to actual risk, which can only be confirmed through calibration analysis. Additionally, DCA quantifies how many patients benefit from model use compared to existing strategies at specific threshold probabilities, enabling assessment of actual clinical implementation value²³.

Subgroup analysis showed improved model performance in patients under 65 years, particularly for hypercalcemia (AUROC 0.953 vs. 0.821), likely due to clearer biochemical profiles in younger individuals. In elderly patients, multimorbidity and polypharmacy may introduce confounding factors that reduce prediction accuracy. Gender-stratified analysis showed varied patterns depending on the target. Notably, hypernatremia prediction was significantly better in males (AUROC 0.734 vs. 0.615), potentially reflecting differences in fluid regulation or drug responses between genders. Cancer-specific performance showed remarkably varied patterns by cancer type, which is noteworthy^6,8,40. However, models showed overly optimistic hypercalcemia predictions in certain cancers due to small sample sizes (n < 50).

Importantly, the majority of input variables used in our models including laboratory results, vital signs, comorbidity indices, and medication records are routinely collected and automatically available in electronic medical record systems, without requiring additional testing or clinician workload. This suggests potential integration as background clinical decision-support tools, although real-world usefulness will depend on careful threshold selection and management of false-positive burden. From a clinical workflow perspective, predicted risk scores could be automatically updated and presented to clinicians as categorical risk levels rather than exact probabilities, facilitating prioritization of high-risk patients for closer monitoring or earlier laboratory reassessment. For example, when the predicted risk of hyperkalemia—an electrolyte abnormality associated with a high risk of cardiac arrhythmia—increases, the model may support clinical decision-making by prompting adjustment of electrolyte monitoring intervals or enabling proactive management, such as correction of modifiable risk factors or preemptive administration of potassium-binding agents.

The models developed in this study are currently available through a web-based application (https://oncolyte.demo.cms.pe.kr/) and may assist risk stratification and monitoring prioritization for patients at elevated predicted risk of developing chemotherapy-related electrolyte abnormalities⁴¹. In particular, prediction models for hyponatremia, hyperkalemia, and hypercalcemia, which demonstrated robust performance across multiple validation settings, may have broader potential utility. Figure S4 illustrates representative usage examples of the web-based system.

Nevertheless, we acknowledge that model complexity may affect scalability, and caution is warranted in interpretation and application due to limited sample sizes in certain cancer types or electrolyte abnormalities. For example, hypernatremia, which exhibited the greatest performance decline during external validation, showed marked class imbalance (0.47% in the internal cohort and 2.17% in the external cohort). Although hypercalcemia also demonstrated severe imbalance, its external performance remained stable, likely because predictive performance was consistently high across most cancer types except prostate cancer. In contrast, hypernatremia prediction performance was particularly poor in liver (AUROC 0.217), breast (0.572), and prostate cancer (0.431). Therefore, external validation results and subgroup-specific performance by age, sex, and cancer type should be carefully considered when applying these models in clinical settings. Potential barriers to adoption include alert fatigue, interpretability concerns, and variability in institutional workflows, underscoring the need for prospective implementation studies. Future work should focus on improving positive-case identification in low-prevalence outcomes through approaches such as threshold recalibration, cost-sensitive learning, dynamic risk updating, or integration with sequential clinical assessments.

This study has several limitations. First, this study was limited to Korean patients from two institutions, which may restrict generalizability to other ethnic populations and global healthcare systems^42,43. Second, this study focused exclusively on electrolyte abnormalities, excluding other important anticancer therapy-related nephrotoxicity. Third, our inclusion criteria required at least 4 weeks of observation and excluded patients who developed electrolyte abnormalities within 24 h of chemotherapy initiation. While this design choice may introduce selection bias by underrepresenting patients with extremely favorable or unfavorable prognoses, it was intentionally adopted to align with the clinical objective of risk prediction. Patients who develop electrolyte abnormalities within the first 24 h are often already clinically unstable, leaving limited opportunity for preventive intervention. Fourth, detailed treatment-related variables such as treatment intent, cancer stage, chemotherapy dosage, combination regimens, and cumulative exposure were unavailable in this study. These factors are clinically important determinants of electrolyte abnormalities; however, incorporating such complex treatment information would require substantially larger datasets to avoid overfitting, particularly given the limited sample sizes observed in certain cancer subgroups. Therefore, the absence of these variables represents a clear limitation of the present study and should be addressed in future research using larger and more diverse patient populations. Fifth, considering frequent dynamic changes in treatment, the need for dynamic prediction model development emerges. Finally, since this was a retrospective observational study, causal inference is limited, and prospective randomized trials are needed to confirm the clinical utility and safety of these models^44,45.

Conclusions

Despite these limitations, this study represents an important step to develop and validate a large-scale machine learning framework for predicting electrolyte abnormalities in patients receiving anticancer therapy. Our findings support the potential value of computational approaches in onconephrology while emphasizing the importance of cautious interpretation in low-prevalence prediction settings. The proposed prediction models demonstrated favorable discrimination performance but variable threshold-based clinical utility. However, validation results also revealed challenges that may limit immediate clinical implementation, underscoring the need for cautious interpretation. These models may serve as useful benchmarks for future studies on electrolyte abnormalities in cancer patients. Future research should incorporate larger and more diverse patient populations and evaluate clinical impact through prospective implementation studies.

Author contributions

N.C., I.J., S.A., S.O., and H.G. defined the study design. All authors contributed to the acquisition, analysis, and interpretation of the data. The manuscript was drafted by N.C., I.J., and S.A., with all authors critically reviewing it for important intellectual content. H.L. and H.G. had full access to all the data in the study and took responsibility for the integrity of the data and the accuracy of the data analysis. Statistical analysis was performed by N.C. and I.J. Funding was provided by H.L., while Y.H.K., Y.M.K., J.P., J.K., B.K., and S.O. offered administrative, technical, or material support. Supervision was provided by H.G. and H.L.

Peer review

Peer review information

Communications Medicine thanks Yahui Li, Enzhao Zhu and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Data availability

The study used de-identified electronic medical record data obtained from Korea University Anam Hospital and Soonchunhyang University Cheonan Hospital. Due to ethical restrictions, patient confidentiality, and institutional policies, the individual-level clinical data and trained machine learning models are not publicly available. The source data underlying the main figures (Figs. 3 and 4) are provided as Supplementary Data 2 and 3. Other relevant data supporting the findings of this study are stored on secure institutional servers and are available from the corresponding authors upon reasonable request. Requests for data access will be reviewed in accordance with institutional review board regulations and data use agreements, and responses will be provided within 4 weeks by the corresponding authors (Hwamin Lee, hwamin@korea.ac.kr).

Code availability

Custom code was used for data preprocessing, feature engineering, model development, validation, and statistical analysis. Analyses were performed using Python (version 3.8) and R (version 4.1.1), with standard machine learning and statistical libraries. Due to institutional policies and data security considerations related to patient confidentiality, the analysis code is not publicly available. The code may be made available from the corresponding authors upon reasonable request for research purposes, subject to institutional review and data use agreements.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s43856-026-01589-4.