Abstract
Background
Effective risk stratification is crucial for managing cervical lesions. This study aimed to develop and evaluate machine learning models to improve the detection of cervical intraepithelial neoplasia (CIN) grade 2 or worse (CIN2+) and CIN3+.
Methods
This retrospective study included 2,863 participants. We developed three models: a Logistic Regression (Logit.Model), a Random Forest (RF.Model), and an XGBoost (XGBoost.Model) to predict CIN2 + and CIN3 + status. The dataset was split into training (60%) and test (40%) sets. Model performance was assessed using AUC, calibration curves, decision curve analysis (DCA), net reclassification improvement (NRI), and integrated discrimination improvement (IDI). The SHapley Additive explanation (SHAP) method was employed for model interpretation.
Results
XGBoost.Model demonstrated robust performance, achieving the highest test set AUCs of 0.735 for CIN2 + and 0.841 for CIN3+. It showed significantly higher AUC for CIN3 + detection compared to RF.Model (P < 0.001). XGBoost.Model also provided significant NRI (13.1% for CIN2+, 28.0% for CIN3+) and IDI (12.1% for CIN2+, 11.0% for CIN3+) over the Logit.Model (all P < 0.05). SHAP analysis confirmed the model’s interpretability, highlighting key predictive features such as cytology and specific HPV genotypes.
Conclusions
The XGBoost.Model exhibited superior and consistent performance, achieving the highest test set AUC and providing a significant NRI and IDI over the logistic regression model.
Clinical trial number
Not applicable.
Supplementary Information
The online version contains supplementary material available at 10.1186/s12911-026-03413-4.
Keywords: Cytology, HPV, Cervical cancer, Cervical intraepithelial neoplasia, Machine learning, XGBoost
Background
Cervical cancer remains a significant global health burden, despite being largely preventable through effective screening and management of precancerous lesions [1, 2]. The cornerstone of cervical cancer prevention is the accurate identification of high-grade cervical intraepithelial neoplasia (CIN), specifically CIN2 and CIN3, which require clinical intervention [3]. Current primary screening often relies on high-risk Human papillomavirus (hrHPV) testing, sometimes combined with cytology [4]. While highly sensitive, hrHPV testing has limited specificity, leading to a high number of colposcopy referrals for transient infections or low-grade abnormalities [5]. Cytology suffers from subjective interpretation, particularly for glandular lesions [6]. This diagnostic gap highlights the critical need for improved risk stratification tools to optimize patient management, reduce unnecessary procedures, and focus resources on those at highest risk.
Machine learning (ML) models offer a promising avenue to enhance predictive accuracy by integrating complex, non-linear relationships between multiple clinical and molecular variables [7]. Techniques such as logistic regression, random forests, and gradient boosting (e.g., XGBoost) have been increasingly applied in medical diagnostics. XGBoost, in particular, is renowned for its high performance and efficiency with structured data [8]. However, the clinical adoption of such “black-box” models is often hindered by a lack of interpretability, making it difficult for clinicians to understand and trust the model’s predictions [9]. The SHapley Additive exPlanations (SHAP) framework addresses this by quantifying the contribution of each feature to individual predictions, thereby bridging the gap between model complexity and clinical comprehensibility [10].
While previous studies have explored ML for cervical lesion classification [11, 12], direct comparisons of different model architectures—particularly assessing their clinical utility via robust metrics like net reclassification improvement (NRI) and decision curve analysis (DCA)—alongside comprehensive interpretability analyses, are less common. Therefore, this retrospective study aims to (1) develop and compare three ML models (Logistic Regression, Random Forest, and XGBoost) for the detection of CIN2 + and CIN3+; (2) rigorously evaluate their performance using discrimination, calibration, and clinical net benefit; and (3) employ SHAP analysis to elucidate the decision-making process of the optimal model, enhancing its transparency and potential for integration into clinical decision-support systems.
Methods
Study design
This was a retrospective diagnostic study designed to develop and validate machine learning models for predicting histologically confirmed high-grade cervical intraepithelial neoplasia (CIN2 + and CIN3+) using routinely collected screening data. This study was approved by the Research and Clinical Trial Ethics Committee of the Changsha Hospital for Maternal & Child Health Care.
Participants
A total of 2,863 women who underwent cervical cancer screening at the Changsha Hospital for Maternal & Child Health Care between June 2023 and October 2025 were included in this study. The inclusion criteria were: (1) Women undergoing cervical cancer screening, (2) availability of concurrent high-risk human papillomavirus (hrHPV) genotyping and liquid-based cytology (LBC) results, and (3) availability of a conclusive histological diagnosis obtained by colposcopy-directed biopsy or surgical excision (including loop electrosurgical excision procedure or cold knife conization). Participants with a history of total hysterectomy or treatment for cervical precancer or cancer were excluded. The study flowchart detailing participant selection is presented in Fig. 1.
Fig. 1.
Study flowchart. HPV, Human papillomavirus; E6E7, HPV E6E7 mRNA test; HC2, HPV Hybrid Capture 2 test; DNA, Deoxyribonucleic acid; CIN1, Cervical intraepithelial neoplasia grade 1; CIN2, Cervical intraepithelial neoplasia grade 2; CIN3, Cervical intraepithelial neoplasia grade 3; SCC, Cervical squamous cell carcinoma; AC, Cervical adenocarcinoma
Data collection
Data were extracted from the hospital’s electronic medical records and laboratory information system. The following variables were collected for each participant:
(1) Demographic Information: Age at the time of screening; (2) HPV Genotyping Results: Results from a PCR-based assay for 14 high-risk HPV genotypes (HPV16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, 68). Results were recorded as positive or negative for each genotype; (3) Cytology Results: Liquid-based cytology results were categorized according to the Bethesda System: Negative for intraepithelial lesion or malignancy (NILM), Atypical squamous cells of undetermined significance (ASC-US), Low-grade squamous intraepithelial lesion (LSIL), Atypical squamous cells cannot exclude HSIL (ASC-H), Atypical glandular cells (AGC), High-grade squamous intraepithelial lesion (HSIL), or cervical cancer (CC); (4) Histopathological Outcomes (Gold Standard): The final histological diagnosis from biopsy or surgical specimen was used as the reference standard. Diagnoses were classified as: Normal, CIN1, CIN2, CIN3, SCC, AC). For model development, outcomes were binarized into < CIN2 (Normal/CIN1) vs. CIN2+ (CIN2, CIN3, SCC, AC). Data collection was performed by two researchers (Z.T. and S.M.), and any discrepancies were resolved through discussion with the senior investigator (X.F.).
Statistical analysis
The sample size adequacy for the multivariable model was guided by the “Events Per Variable (EVP)” principle. Our pre-specified predictors (n = 21) included age (continuous), 14 hrHPV genotypes (dichotomous, each), and a categorical cytology variable with six levels. The outcome of interest was histologically confirmed CIN2+. Applying the common threshold of 10 events per variable, a minimum of 210 CIN2 + cases was deemed necessary. The train set, derived from random allocation, comprised 1717 participants, of whom 279 had a CIN2 + outcome. The actual EVP ratio in our train data was therefore approximately 13.3 (279 events / 21 variables), which surpasses the minimum recommended ratio of 10:1, indicating sufficient statistical power for reliable model construction.
We compared the age, HPV genotyping, cytology, and pathological results between the train set and test set. The continuous variable age is presented as mean ± standard deviation (SD) and was analyzed using the Student’s t-test. HPV genotyping, cytology, and pathological results are categorical variables, expressed as n (%), and were analyzed using the chi-square test. All analyses were performed using R version 4.5.1 (Vienna, Austria).
Model development: The derived cohort was randomly divided into a train set (60%) and a test set (40%). Cases of CIN2 + were labeled as positive samples, and cases of < CIN2 were labeled as negative samples.
Logit.Model: Two variables, cytology and HPV, were included. Cytology was categorized into three groups: NILM/ASC-US, LSIL/ASC-H/AGC, and HSIL/CC. HPV results were categorized into three groups: negative, other 12 hrHPV (+), and HPV16/18 (+). Here, "other 12 hrHPV (+)" indicates positivity for any one or more of the following genotypes: HPV31, HPV33, HPV35, HPV39, HPV45, HPV51, HPV52, HPV56, HPV58, HPV59, HPV66, HPV68. The Logit.Model was developed using stats::glm(binomial(link = "logit")) (Supplementary Table 1, S1 Logistic Regression Model).
RF.Model: Sixteen variables were included: Age, HPV16, HPV18, HPV31, HPV33, HPV35, HPV39, HPV45, HPV51, HPV52, HPV56, HPV58, HPV59, HPV66, HPV68, and Cytology. The model was developed using randomForest::randomForest() (S2 RandomForest Model).
XGBoost.Model: Sixteen variables were included: Age, HPV16, HPV18, HPV31, HPV33, HPV35, HPV39, HPV45, HPV51, HPV52, HPV56, HPV58, HPV59, HPV66, HPV68, and Cytology. The model was developed using xgboost::xgb.train() (S3 XGBoost Model).
Model evaluation: The predicted probabilities for CIN2 + were obtained in both the train set and test set using stats::predict(). ROC curves for HPV, cytology, and the models for detecting CIN2 + in the train and test sets were plotted using pROC::roc(), ggroc(), and ggplot2. Calibration curves for the models were plotted using caret::calibration(). Decision curve analysis (DCA) was performed using rmda::decision_curve() and plot_decision_curve(). The cutoff values for detecting CIN2 + were determined based on the maximum Youden index. The cutoff values for Logit.Model, RF.Model, and XGBoost.Model were 0.121, 0.047, and 0.175, respectively. Sensitivity, specificity, positive predictive value (PPV), Negative predictive value (NPV), and their 95% CIs for tests and models were calculated using DTComPair::acc.paired(). Comparisons of sensitivity and specificity were performed using DTComPair::sesp.mcnemar(), comparisons of PPV and NPV were performed using DTComPair::pv.rpv(), Comparisons of AUC were performed using pROC:: roc.test (method="delong”)), The calculations for NRI and IDI followed the method provided by Kerr, Kathleen F, et al. [13].
Model interpretation: Model interpretability analysis of features was performed using SHapley Additive explanation (SHAP) via shapviz::shapviz(). Feature importance plots, individual case waterfall plots, and dependence plots for important features were generated using shapviz::sv_importance(), sv_waterfall(), and sv_dependence().
All p-values were two-sided, and a p-value < 0.05 was considered statistically significant.
Results
Patient characteristics
This retrospective study included a total of 2,863 participants, with a mean age of 44.5 ± 11.89 years. The highest infection rate was observed for HPV52 (29.2%), followed by HPV16 (12.7%). Among cytological results, ASC-US was the most common, accounting for 43.1% (1,235 cases), followed by LSIL at 32.3% (924 cases). Pathological findings were distributed as follows: Normal (52.0%), CIN1 (31.5%), CIN2 (10.4%), CIN3 (4.8%), SCC (0.9%), and AC (0.4%). After random allocation into train and test sets in a 6:4 ratio, only the distribution of pathological results showed a statistically significant difference between the two groups (P = 0.004). Detailed characteristics of the participants are presented in Table 1.
Table 1.
Characteristics of participants
| Characteristic | level | Overall | Train set | Test set | P |
|---|---|---|---|---|---|
| n | 2863 | 1717 | 1146 | ||
| Age (mean (SD)) | 44.5 (11.89) | 44.7 (11.90) | 44.3 (11.87) | 0.329 | |
| HPV16 (%) | Positive | 364 (12.7) | 229 (13.3) | 135 (11.8) | 0.243 |
| HPV18 (%) | Positive | 135 (4.7) | 91 (5.3) | 44 (3.8) | 0.086 |
| HPV31 (%) | Positive | 82 (2.9) | 47 (2.7) | 35 (3.1) | 0.701 |
| HPV33 (%) | Positive | 169 (5.9) | 103 (6.0) | 66 (5.8) | 0.853 |
| HPV35 (%) | Positive | 55 (1.9) | 32 (1.9) | 23 (2.0) | 0.893 |
| HPV39 (%) | Positive | 225 (7.9) | 128 (7.5) | 97 (8.5) | 0.362 |
| HPV45 (%) | Positive | 46 (1.6) | 30 (1.7) | 16 (1.4) | 0.562 |
| HPV51 (%) | Positive | 267 (9.3) | 157 (9.1) | 110 (9.6) | 0.731 |
| HPV52 (%) | Positive | 836 (29.2) | 483 (28.1) | 353 (30.8) | 0.134 |
| HPV56 (%) | Positive | 190 (6.6) | 116 (6.8) | 74 (6.5) | 0.812 |
| HPV58 (%) | Positive | 487 (17.0) | 296 (17.2) | 191 (16.7) | 0.727 |
| HPV59 (%) | Positive | 181 (6.3) | 102 (5.9) | 79 (6.9) | 0.343 |
| HPV66 (%) | Positive | 173 (6.0) | 103 (6.0) | 70 (6.1) | 0.968 |
| HPV68 (%) | Positive | 178 (6.2) | 99 (5.8) | 79 (6.9) | 0.252 |
| Cytology (%) | NILM | 340 (11.9) | 215 (12.5) | 125 (10.9) | 0.114 |
| ASC-US | 1235 (43.1) | 738 (43.0) | 497 (43.4) | ||
| LSIL | 924 (32.3) | 538 (31.3) | 386 (33.7) | ||
| AGC | 235 (8.2) | 144 (8.4) | 91 (7.9) | ||
| ASC-H | 16 (0.6) | 10 (0.6) | 6 (0.5) | ||
| HSIL | 102 (3.6) | 61 (3.6) | 41 (3.6) | ||
| CC | 11 (0.4) | 11 (0.6) | 0 (0.0) | ||
| Pathology (%) | Normal | 1490 (52.0) | 932 (54.3) | 558 (48.7) | 0.004 |
| CIN1 | 903 (31.5) | 506 (29.5) | 397 (34.6) | ||
| CIN2 | 297 (10.4) | 168 (9.8) | 129 (11.3) | ||
| CIN3 | 137 (4.8) | 83 (4.8) | 54 (4.7) | ||
| SCC | 25 (0.9) | 21 (1.2) | 4 (0.35) | ||
| AC | 11 (0.4) | 7 (0.4) | 4 (0.35) |
Abbreviation: SD, Standard deviation; HPV, human papillomavirus; NILM, No intraepithelial lesions or malignancy; ASC-US, Atypical squamous cells of undermined significance; LSIL, Low-grade squamous intraepithelial lesion; AGC, Atypical glandular cells; ASC-H, Atypical squamous cells cannot exclude HSIL; HSIL, High-grade squamous intraepithelial lesion; CC, Cervical cancer; CIN1, Cervical intraepithelial neoplasia grade 1; CIN2, Cervical intraepithelial neoplasia grade 2; CIN3, Cervical intraepithelial neoplasia grade 3; SCC, Cervical squamous cell carcinoma; AC, Cervical adenocarcinoma
Performance of the models
The models achieved the following AUC performance on the train set: Logit.Model at 0.776, RF.Model at 0.945, and XGBoost.Model at 0.894 (Fig. 2A); on the test set: Logit.Model at 0.725, RF.Model at 0.768, and XGBoost.Model at 0.829 (Fig. 2B). XGBoost.Model demonstrated a higher AUC and more robust performance.
Fig. 2.
ROCs, calibration curves and DCA plots. ROCs for detecting CIN2 + of two tests and three models in train set (A) and test set (B); Calibration curves of three models in train set (C) and test set (D); The DCA of the three models for the train set (E) and test set (F). ROC, Receiver operating characteristic curve; AUC, Area under the ROC; HPV, Human papillomavirus; DCA, decision curve analysis
The calibration curves of the models in the train and test sets (Fig. 2C and D) evaluated the agreement between predicted and observed outcomes, with XGBoost.Model showing higher consistency. Figure 2E and F display the Decision Curve Analysis (DCA) plots for the models in the train and test sets. At risk thresholds of 0.2–0.6, both RF.Model and XGBoost.Model yielded higher net benefits than Logit.Model, and these two models exhibited similar net benefits in the test set.
XGBoost.Model achieved the highest AUC for detecting CIN2 + and CIN3+, with values of 0.735 (95% CI = 0.699–0.772) and 0.841 (95% CI = 0.795–0.886), respectively. Additionally, its AUC for detecting CIN3 + showed a statistically significant difference compared to RF.Model’s AUC (0.778, 95%CI = 0.729–0.828; P < 0.001; Table 2). Supplementary Table 2 presents the clinical performance of HPV, cytology, and the models in the train set.
Table 2.
Clinical performance in test set
| Test / model | Cut-off / compared groups | Sensitivity % (95%CI) | Specificity % (95%CI) | PPV % (95%CI) | NPV % (95%CI) | AUC % (95%CI) |
|---|---|---|---|---|---|---|
| CIN2+ | ||||||
| hrHPV | hrHPV (+) | 93.7 (90.3–97.2) | 10.6 (8.6–12.5) | 17.3 (15.0–19.6) | 89.4 (83.7–95.1) | 0.522 (0.502–0.541) |
| Cytology | ASC-US+ | 96.3 (93.7–99.0) | 12.4 (10.3–14.5) | 18.0 (15.7–20.4) | 94.4 (90.4–98.4) | 0.544 (0.526–0.561) |
| P |
hrHPV (+) vs. Cytology (ASC-US+) |
0.251 | 0.235 | 0.105 | 0.161 | 0.107 |
| HPV16/18 | HPV16/18 (+) | 26.2 (19.9–32.4) | 86.7 (84.5–88.9) | 28.2 (21.6–34.9) | 85.4 (83.2–87.7) | 0.564 (0.531–0.598) |
| Cytology | LSIL+ | 71.2 (64.8–77.6) | 59.4 (56.3–62.5) | 26.0 (22.2–29.7) | 91.2 (88.9–93.4) | 0.653 (0.617–0.689) |
| P | HPV16/18 vs. Cytology (LSIL+) | < 0.001 | < 0.001 | 0.481 | < 0.001 | < 0.001 |
| Logit.Model | 0.121 | 79.1 (73.3–84.8) | 54.9 (51.7–58.0) | 25.9 (22.4–29.5) | 92.9 (90.8–95.0) | 0.670 (0.637–0.703) |
| P | HPV16/18 vs. Logit.Model | < 0.001 | < 0.001 | 0.389 | < 0.001 | < 0.001 |
| P | Cytology (LSIL+) vs. Logit.Model | 0.003 | < 0.001 | 0.992 | 0.025 | 0.243 |
| RF.Model | 0.047 | 64.9 (58.1–71.7) | 77.5 (74.8–80.1) | 36.6 (31.4–41.7) | 91.7 (89.8–93.6) | 0.712 (0.676–0.749) |
| P | RF.Model vs. Logit.Model | < 0.001 | < 0.001 | < 0.001 | 0.194 | 0.030 |
| XGB.Model | 0.175 | 60.2 (53.3–67.2) | 86.8 (84.6–89.0) | 47.7 (41.4–54.0) | 91.6 (89.8–93.4) | 0.735 (0.699–0.772) |
| P | XGB.Model vs. Logit.Model | < 0.001 | < 0.001 | < 0.001 | 0.126 | < 0.001 |
| P | XGB.Model vs. RF.Model | 0.029 | < 0.001 | < 0.001 | 0.834 | 0.058 |
| CIN3+ | ||||||
| hrHPV | hrHPV (+) | 91.9 (85.1–98.7) | 10.0 (8.2–11.8) | 5.5 (4.1–6.9) | 95.6 (91.8–99.4) | 0.510 (0.474–0.545) |
| Cytology | ASC-US+ | 96.8 (92.4–100.0) | 11.3 (9.4–13.2) | 5.9 (4.4–7.3) | 98.4 (96.2–100.0) | 0.541 (0.516–0.565) |
| P |
hrHPV (+) vs. Cytology (ASC-US+) |
0.257 | 0.309 | 0.162 | 0.210 | 0.164 |
| HPV16/18 | HPV16/18 (+) | 37.1 (25.1–49.1) | 85.8 (83.7–87.9) | 13.0 (8.0–17.9) | 96.0 (94.7–97.2) | 0.615 (0.553–0.676) |
| Cytology | LSIL+ | 82.3 (72.7–91.8) | 56.3 (53.4–59.3) | 9.7 (7.2–12.3) | 98.2 (97.2–99.3) | 0.693 (0.643–0.743) |
| P | HPV16/18 vs. Cytology (LSIL+) | < 0.001 | < 0.001 | 0.105 | 0.002 | 0.071 |
| Logit.Model | 0.121 | 88.7 (80.8–96.6) | 51.4 (48.4–54.4) | 9.5 (7.1–11.8) | 98.8 (97.8–99.7) | 0.701 (0.658–0.743) |
| P | HPV16/18 vs. Logit.Model | < 0.001 | < 0.001 | 0.034 | < 0.001 | 0.008 |
| P | Cytology (LSIL+) vs. Logit.Model | 0.157 | < 0.001 | 0.583 | 0.254 | 0.753 |
| RF.Model | 0.047 | 82.3 (72.7–91.8) | 73.4 (70.8–76.1) | 15.0 (11.2–18.8) | 98.6 (97.8–99.4) | 0.778 (0.729–0.828) |
| P | RF.Model vs. Logit.Model | 0.206 | < 0.001 | < 0.001 | 0.790 | 0.004 |
| XGB.Model | 0.175 | 85.5 (76.7–94.3) | 82.7 (80.4–84.9) | 22.0 (16.8–27.2) | 99.0 (98.3–99.7) | 0.841 (0.795–0.886) |
| P | XGB.Model vs. Logit.Model | 0.414 | < 0.001 | < 0.001 | 0.498 | < 0.001 |
| P | XGB.Model vs. RF.Model | 0.317 | < 0.001 | < 0.005 | 0.128 | < 0.001 |
Abbreviation: CI, Confidence interval; PPV, Positive predictive value; NPV, Negative predictive value; AUC, Area under the receiver operating characteristic curve; CIN2+, Cervical intraepithelial neoplasia grade 2 or worse; hrHPV, High-risk human papillomavirus; ASC-US+, Atypical squamous cells of undermined significance or worse; HPV16/18, Human papillomavirus genotype 16 or 18; LSIL+, Low-grade squamous intraepithelial lesion or worse; CIN3+, Cervical intraepithelial neoplasia grade 3 or worse
Table 3 presents the NRI and IDI values of the models. In the test set, compared to Logit.Model, XGB.Model showed an NRI of 13.1 (95% CI = 5.2–21.0; P = 0.001) and an IDI of 12.1 (95% CI = 5.8–18.4; P < 0.001) for detecting CIN2+; and an NRI of 28.0 (95% CI = 19.5–36.5; P < 0.001) and an IDI of 11.0 (95% CI = 4.5–17.5; P = 0.001) for detecting CIN3+. Compared to RF.Model, XGB.Model showed an NRI of 4.6 (95% CI = -0.2–9.4; P = 0.062) and an IDI of 6.5 (95% CI = 1.2–11.8; P = 0.015) for detecting CIN2+; and an NRI of 12.5 (95% CI = 5.8–19.2; P < 0.001) and an IDI of 6.7 (95% CI = 1.0–12.4; P = 0.020) for detecting CIN3+.
Table 3.
NRIs and IDIs of models in train and test set
| Model | NRI % (95%CI) | P | IDI % (95%CI) | P | NRI % (95%CI) | P | IDI % (95%CI) | P |
|---|---|---|---|---|---|---|---|---|
| Train set (For CIN2+) | ||||||||
| Logit.Model | Reference | – | Reference | – | – | – | – | – |
| RF.Model | 35.5 (29.6–41.4) | < 0.001 | 28.6 (23.5–33.7) | < 0.001 | Reference | – | Reference | – |
| XGB.Model | 22.5 (17.1–27.9) | < 0.001 | 20.7 (15.4–26.0) | < 0.001 | -13.0 (-18.4– -7.6) | < 0.001 | -7.9 (-13.4– -2.4) | 0.005 |
| Train set (For CIN3+) | ||||||||
| Logit.Model | Reference | – | Reference | – | – | – | – | – |
| RF.Model | 27.4 (20.8–34.0) | < 0.001 | 10.0 (5.6–14.4) | < 0.001 | Reference | – | Reference | – |
| XGB.Model | 24.7 (18.4–31.0) | < 0.001 | 11.0 (5.9–16.1) | < 0.001 | -2.7 (-8.8–3.4) | 0.383 | 1.0 (-4.1–6.1) | 0.699 |
| Test set (For CIN2+) | ||||||||
| Logit.Model | Reference | – | Reference | – | – | – | – | – |
| RF.Model | 8.5 (0.5–16.5) | 0.038 | 5.6 (0.1–11.1) | 0.045 | Reference | – | Reference | – |
| XGB.Model | 13.1 (5.2–21.0) | 0.001 | 12.1 (5.8–18.4) | < 0.001 | 4.6 (-0.2–9.4) | 0.062 | 6.5 (1.2–11.8) | 0.015 |
| Test set (For CIN3+) | ||||||||
| Logit.Model | Reference | – | Reference | – | – | – | – | – |
| RF.Model | 15.6 (5.0–26.2) | 0.004 | 4.3 (-1.4–10.0) | 0.141 | Reference | – | Reference | – |
| XGB.Model | 28.0 (19.5–36.5) | < 0.001 | 11.0 (4.5–17.5) | 0.001 | 12.5 (5.8–19.2) | < 0.001 | 6.7 (1.0–12.4) | 0.020 |
Abbreviation: NRI, Net reclassification improvement; IDI, Integrated discrimination improvement; CI, Confidence interval; CIN2+, Cervical intraepithelial neoplasia grade 2 or worse; CIN3+, Cervical intraepithelial neoplasia grade 3 or worse
Model interpretations
Due to the high consistency in the clinical performance of the XGBoost.Model across both the train set and test set, we further explored its interpretability using the SHAP method (Fig. 3A). A positive SHAP value indicates that the corresponding feature increases the probability of CIN2+. The mean of the absolute SHAP values reflects the contribution of each feature to the model, with a larger mean indicating a greater contribution (Fig. 3B).
Fig. 3.
Model interpretability analysis of features. SHAP value (A) and mean |SHAP value| (B) of the XGBoost.Model; Individual SHAP waterfall plot for XGBoost.Model predictions of diagnosing < CIN2 (C) and (CIN2+) success: (E) Dependency plot of top four features. SHAP, SHapley Additive explanation; HPV, Human papillomavirus
To further illustrate interpretability at the individual level, SHAP force plots for two representative patients are presented: one correctly predicted as CIN2+ (Fig. 3C) and the other correctly predicted as < CIN2 (Fig. 3D). These plots highlight the contribution of key features to the model’s predictions.
The SHAP dependence plots (Fig. 3E) demonstrate how the top four features influence the model’s output. For example, cytology results of 0 (NILM), 1 (ASC-US), and 2 (LSIL) correspond to SHAP values less than 0, while results of 3 (ASC-H), 4 (AGC), and 5 (HSIL+) correspond to SHAP values greater than 0, leading to an increased predicted risk of CIN2+.
Discussion
In this study, we developed and rigorously validated three machine learning models to improve the detection of CIN2 + and CIN3 + within a cohort of 2,863 women. Our results demonstrate that the XGBoost.Model consistently outperformed both the Logit.Model and the RF.Model in terms of discrimination, calibration, and clinical net benefit. Specifically, the XGBoost.Model achieved the highest AUC (0.841) in the test set for CIN3+. More importantly, it demonstrated superior ability to correctly reclassify risk, as evidenced by significant NRI and IDI values compared to the Logit.Model, and also showed incremental improvements over the RF.Model for CIN3 + detection. The use of SHAP analysis successfully demystified the “black-box” nature of the optimal model, identifying cytology and specific hrHPV genotypes as the most influential predictors and providing intuitive visual explanations for both population-level and individual-level predictions.
The superior performance of the XGBoost.Model aligns with its theoretical strengths in handling complex, non-linear interactions within structured data [14]. While the RF.Model showed excellent performance on the training set (AUC: 0.945), its test set performance (AUC: 0.768 for CIN2+) indicated a degree of overfitting, which was less pronounced in the XGBoost.Model (Fig. 2A and B). Supplementary Table 3 shows the AUCs of 5-fold cross-validation of RF.Model and XGBoost.Model. The robust test set performance of XGBoost, coupled with favorable calibration and decision curve analysis, suggests it is a more generalizable tool. The interaction among risk factors for cervical lesions may be complex and nonlinear, and XGBoost happens to be good at capturing this structure. Compared with RF, its regularization technique may better control overfitting. From a clinical perspective, the DCA indicated that both XGBoost.Model and RF.Model provided higher net benefit than the traditional Logit.Model across a range of clinically relevant risk thresholds (0.2–0.6). This translates to a potential reduction in unnecessary colposcopies for a given sensitivity, or an increase in high-grade lesion detection for a given specificity, thereby optimizing resource allocation. The model’s high specificity (86.8%) and PPV (47.7%) for CIN2 + at its optimal threshold are particularly valuable for a triage tool, aiming to identify women who would most benefit from immediate colposcopic referral.
Our findings corroborate and extend previous research exploring ML in cervical cancer screening. Several studies have reported the high accuracy of ensemble methods like Random Forest and Gradient Boosting for classifying cervical lesions [11, 15]. In a machine learning model based on routine blood analysis [11], the XGBoost model achieved the highest performance in predicting cervical cancer, with an AUC of 0.964. However, we know from clinical practice that the key to cervical cancer prevention lies in detecting high-grade cervical intraepithelial neoplasia (CIN) for timely intervention. The primary screening methods used in practice are hrHPV testing and cytology, whereas routine blood analysis results are relatively difficult to obtain. Another deep learning-based liquid-based cytology model for detecting cervical precancerous lesions and cancer [15] demonstrated that deep learning-assisted performance was superior to evaluation by junior cytopathologists alone only when classifying HPV-positive cases, yet it did not fully utilize hrHPV genotyping. Our study contributes to this field by conducting a direct, head-to-head comparison of three distinct model architectures based on cervical cancer screening results—cytology and hrHPV genotyping—using a consistent dataset and a comprehensive evaluation framework that goes beyond mere discrimination. Moreover, the machine learning model [16] have incorporated gene expression as a predictor variable. This approach requires additional molecular test for unstable mRNA, which undermines the practicality of such model tools. NRI and IDI provide a more nuanced understanding of the models’ added value in risk stratification [17, 18], a metric less frequently reported in similar studies [11, 19–21]. Furthermore, while some studies have employed SHAP for model interpretation in oncology [22–24], its application to explain an XGBoost model for CIN2+/CIN3 + prediction, linking features like detailed cytology categories and individual HPV genotypes to model output, adds a significant layer of transparency that is crucial for clinical adoption.
The SHAP analysis was pivotal in translating model performance into clinically actionable insights. The top features identified—cytology result and hrHPV genotypes (notably HPV16, HPV52, HPV58)—are well-established risk factors, validating the model’s alignment with biological and clinical knowledge [25–27]. The SHAP summary plot quantitatively confirms the primary role of cytology and HPV16/52/58, validating established clinical understanding and building trust. More informatively, the dependence plots quantified the non-linear impact of these features. For instance, the dependence plot for cytology reveals a clear risk threshold (ASC-H/AGC and above), which visually reinforces and could help operationalize existing clinical management guidelines (Fig. 3E). The individual force plots demonstrate how the model synthesizes multiple risk factors for a patient, which could be used during clinician review to explain a why behind a high-risk score, potentially reducing referral of borderline cases. This interpretability bridges the gap between algorithmic prediction and clinical reasoning, allowing physicians to understand why a patient received a high-risk score, thereby fostering trust and facilitating integration into decision-support systems.
This study has several limitations. First, the retrospective, single-center design limits the generalizability of our models. The performance may vary in populations with different demographic profiles, HPV genotype prevalence, screening protocols, or cytology interpretation standards. External validation in independent, multi-center cohorts is essential before clinical application. Second, despite random splitting, a significant class imbalance exists, particularly for CIN3 + outcomes. While the Event Per Variable ratio was adequate for CIN2+, the smaller number of CIN3 + cases may affect the precision of estimates for this endpoint and model calibration in the high-risk range. Future studies with larger, balanced cohorts are needed to refine predictions for CIN3+. Third, Multiple indicators (AUC, NRI, IDI) were reported, but no correction was made for multiple comparisons. This might exaggerate the illusion that XGBoost is significantly superior to other algorithms. Fourth, the models were developed using clinician-dependent variables (cytology). Future work could explore incorporating more objective biomarkers or imaging features (e.g. objective digital cytological image features, DNA methylation, DNA ploidy). Finally, while SHAP enhances interpretability, the model’s practical implementation in a real-time clinical workflow requires the development of user-friendly interfaces and further studies on its impact on clinical decision-making and patient outcomes.
Conclusion
We developed an interpretable XGBoost-based machine learning model that demonstrates superior and consistent performance for detecting high-grade cervical lesions (CIN2 + and CIN3+) compared to logistic regression and random forest models. By rigorously evaluating discrimination, calibration, and clinical utility, and by employing SHAP analysis to ensure transparency. This study provides a proof-of-concept for an interpretable XGBoost model. This model presents a potential decision-support tool. Future work should focus on developing a user-friendly interface, integrating it into a clinical workflow for pilot testing, and prospectively evaluating its impact on clinician decision-making and patient outcomes. Prospective validation in independent, multi-center cohorts is warranted to confirm its generalizability and clinical effectiveness.
Supplementary Information
Below is the link to the electronic supplementary material.
Acknowledgements
Not applicable.
Abbreviations
- AC
Cervical adenocarcinoma
- AGC
Atypical glandular cells
- ASC-H
Atypical squamous cells cannot exclude HSIL
- ASC-US
Atypical squamous cells of undermined significance
- ASC-US+
Atypical squamous cells of undermined significance or worse
- AUC
Area under the ROC
- CC
Cervical cancer
- CI
Confidence interval
- CIN1
Cervical intraepithelial neoplasia grade 1
- CIN2
Cervical intraepithelial neoplasia grade 2
- CIN2+
Cervical intraepithelial neoplasia grade 2 or worse
- CIN3
Cervical intraepithelial neoplasia grade 3
- CIN3+
Cervical intraepithelial neoplasia grade 3 or worse
- DCA
Decision curve analysis
- DNA
Deoxyribonucleic acid
- E6E7
HPV E6E7 mRNA test
- HC2
HPV Hybrid Capture 2 test
- HSIL
High-grade squamous intraepithelial lesion
- HPV
Human papillomavirus
- HPV16/18
Human papillomavirus genotype 16 or 18
- hrHPV
High-risk human papillomavirus
- IDI
Integrated discrimination improvement
- LSIL
Low-grade squamous intraepithelial lesion
- LSIL+
Low-grade squamous intraepithelial lesion or worse
- NILM
No intraepithelial lesions or malignancy
- NPV
Negative predictive value
- NRI
Net reclassification improvement
- PPV
Positive predictive value
- ROC
Receiver operating characteristic curve
- SCC
Cervical squamous cell carcinoma
- SD
Standard deviation
- SHAP
SHapley Additive explanation
Author contributions
Hong Tao carried out the experiment of the study, read the relevant literatures, performed the statistical analyses, and drafted the manuscript. Zhengjiao Tong collected data, interpreted the results and drafted the manuscripts; Hongtao Li interpreted the results and performed the statistical analyses; Saiping Mao collects data; Xing Fan were involved in the conception and design of the study and direct the progress of the study. All authors read and approved the final manuscript.
Funding
Not applicable.
Data availability
The datasets used during the current study are available from the corresponding author upon reasonable request.
Declarations
Ethical approval
This study complies with the Declaration of Helsinki and has been approved by the Research and Clinical Trials Ethics Committee of Changsha Maternal and Child Health Hospital. Since this study only involves the collection of clinical data, patient informed consent has been waived.
Consent for publication
Not applicable.
Competing interests
The authors declare no competing interests.
Footnotes
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Hong Tao and Zhengjiao Tong contributed equally to this work.
References
- 1.Liang Y, Dai X, Chen J, Zeng X, Qing X, Huang J, Ren L, Zhang X, Zhang W, Ruan X. Global burden and trends in pre- and post-menopausal gynecological cancer from 1990 to 2019, with projections to 2040: a cross-sectional study. Int J Surg (London England). 2025;111(1):891–903. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Xu M, Cao C, Wu P, Huang X, Ma D. Advances in cervical cancer: current insights and future directions. Cancer Commun (London England). 2025;45(2):77–109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Gupta S, Nagtode N, Chandra V, Gomase K. From diagnosis to treatment: exploring the latest management trends in cervical intraepithelial neoplasia. Cureus. 2023;15(12):e50291. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.WHO guideline for screening. and treatment of cervical pre-cancer lesions for cervical cancer prevention, second edition. In: WHO guideline for screening and treatment of cervical pre-cancer lesions for cervical cancer prevention, second edition. edn. Geneva; 2021. [DOI] [PubMed]
- 5.Pillay J, Gates A, Guitard S, Zakher B, Sim S, Vandermeer B, Hartling L. Screening for the prevention and early detection of cervical cancer: systematic reviews to inform an update to recommendations by the Canadian Task Force on Preventive Health Care. Syst reviews. 2025;14(1):206. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Gupta R, Kumar N, Bansal S, Singh S, Sood N, Gupta S. Artificial intelligence-driven digital cytology-based cervical cancer screening: is the time ripe to adopt this disruptive technology in resource-constrained settings? A literature review. J Digit Imaging. 2023;36(4):1643–52. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Meza Ramirez CA, Greenop M, Almoshawah YA, Martin Hirsch PL, Rehman IU. Advancing cervical cancer diagnosis and screening with spectroscopy and machine learning. Expert Rev Mol Diagn. 2023;23(5):375–90. [DOI] [PubMed] [Google Scholar]
- 8.Ramos-Casallas A, Cardona-Mendoza A, Perdomo-Lara SJ, Rico-Mendoza A, Porras-Ramírez A. Performance evaluation of machine learning models in cervical cancer diagnosis: Systematic review and meta-analysis. Eur J Cancer. 2025;229:115768. [DOI] [PubMed] [Google Scholar]
- 9.Aravazhi PS, Gunasekaran P, Benjamin NZY, Thai A, Chandrasekar KK, Kolanu ND, Prajjwal P, Tekuru Y, Brito LV, Inban P. The integration of artificial intelligence into clinical medicine: Trends, challenges, and future directions. Volume 71. Disease-a-month: DM; 2025. p. 101882. 6. [DOI] [PubMed] [Google Scholar]
- 10.El Arab RA, Almoosa Z, Alkhunaizi M, Abuadas FH, Somerville J. Artificial intelligence in hospital infection prevention: an integrative review. Front public health. 2025;13:1547450. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Su J, Lu H, Zhang R, Cui N, Chen C, Si Q, Song B. Cervical cancer prediction using machine learning models based on routine blood analysis. Sci Rep. 2025;15(1):22655. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Ali MM, Ahmed K, Bui FM, Paul BK, Ibrahim SM, Quinn JMW, Moni MA. Machine learning-based statistical analysis for early stage detection of cervical cancer. Comput Biol Med. 2021;139:104985. [DOI] [PubMed] [Google Scholar]
- 13.Kerr KF, Wang Z, Janes H, McClelland RL, Psaty BM, Pepe MS. Net reclassification indices for evaluating risk prediction instruments: a critical review. Epidemiol (Cambridge Mass). 2014;25(1):114–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Li T, Chen J, Zhang X, Wang K, Zhao X, Cao Y, Xu Z, Wang S, Su P, He X, et al. A machine learning model for predicting the risk of diabetic nephropathy in individuals with type 2 diabetes mellitus. Front Endocrinol. 2025;16:1587932. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Xue P, Dang L, Kong LH, Tang HP, Xu HM, Weng HY, Wang Z, Wei RG, Xu L, Li HX, et al. Deep learning enabled liquid-based cytology model for cervical precancer and cancer detection. Nat Commun. 2025;16(1):3506. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Kamble P, Dubey K, Mukherjee A, Jain R, Roy I, Puri V, Garg P. Decoding cervical cancer biomarkers: an integrated framework of bioinformatics, machine learning, and experimental confirmation. Cancer Invest. 2025;43(10):882–902. [DOI] [PubMed] [Google Scholar]
- 17.Oexner RR, Ahn H, Theofilatos K, Shah RA, Schmitt R, Chowienczyk P, Zoccarato A, Shah AM. Serum metabolomics improves risk stratification for incident heart failure. Eur J Heart Fail. 2024;26(4):829–40. [DOI] [PubMed] [Google Scholar]
- 18.Xie R, Sha S, Brenner H, Schöttker B. Circulating inflammation-related proteome improves cardiovascular risk prediction. Results from two large European cohort studies. Eur J Epidemiol. 2025;40(10):1191–203. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Mehmood M, Rizwan M, Gregus Ml M, Abbas S. Machine learning assisted cervical cancer detection. Front public health. 2021;9:788376. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Wentzensen N, Lahrmann B, Clarke MA, Kinney W, Tokugawa D, Poitras N, Locke A, Bartels L, Krauthoff A, Walker J, et al. Accuracy and efficiency of deep-learning-based automation of dual stain cytology in cervical cancer screening. J Natl Cancer Inst. 2021;113(1):72–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Xue P, Xu HM, Tang HP, Wu WQ, Seery S, Han X, Ye H, Jiang Y, Qiao YL. Assessing artificial intelligence enabled liquid-based cytology for triaging HPV-positive women: a population-based cross-sectional study. Acta Obstet Gynecol Scand. 2023;102(8):1026–33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Qi X, Wang S, Fang C, Jia J, Lin L, Yuan T. Machine learning and SHAP value interpretation for predicting comorbidity of cardiovascular disease and cancer with dietary antioxidants. Redox Biol. 2025;79:103470. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Hsu WH, Ko AT, Weng CS, Chang CL, Jan YT, Lin JB, Chien HJ, Lin WC, Sun FJ, Wu KP, et al. Explainable machine learning model for predicting skeletal muscle loss during surgery and adjuvant chemotherapy in ovarian cancer. J cachexia sarcopenia muscle. 2023;14(5):2044–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Ladbury C, Zarinshenas R, Semwal H, Tam A, Vaidehi N, Rodin AS, Liu A, Glaser S, Salgia R, Amini A. Utilization of model-agnostic explainable artificial intelligence frameworks in oncology: a narrative review. Translational cancer Res. 2022;11(10):3853–68. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Egemen D, Cheung LC, Chen X, Demarco M, Perkins RB, Kinney W, Poitras N, Befano B, Locke A, Guido RS, et al. Risk estimates supporting the 2019 ASCCP risk-based management consensus guidelines. J Low Genit Tract Dis. 2020;24(2):132–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Massad SL, Markwell S, Cejtin HE, Collins Y. Risk of high-grade cervical intraepithelial neoplasia among young women with abnormal screening cytology. J Low Genit Tract Dis. 2005;9(4):225–9. [DOI] [PubMed] [Google Scholar]
- 27.Matsumoto K, Yoshikawa H. Human papillomavirus infection and the risk of cervical cancer in Japan. J Obstet Gynaecol Res. 2013;39(1):7–17. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The datasets used during the current study are available from the corresponding author upon reasonable request.