Summary
Identifying patients at highest risk of serious adverse prognostic events (AE) in neurosyphilis could enable risk-stratified treatment beyond clinical judgment. We developed machine-learning models using electronic health records from six Chinese infectious-diseases hospitals, with two centers for external validation and four for discovery. Five models incorporated demographic, clinical, laboratory, and treatment variables from 602 observations (402 discovery, 200 validation). AE occurred in 20.90% and 20.50%, respectively. DEFEAT-NS-M1 achieved AUROC 0.975 (95% CI 0.949–0.995) internally and 0.863 (0.801–0.920) externally, with Brier scores 0.027 and 0.128. Decision curve analysis demonstrated favorable clinical utility; treating 1–2 high-risk patients prevents one AE. DEFEAT-NS-M1 supports population-level risk estimation and stratified care, potentially guiding targeted monitoring and therapy. Further external validation and health-economic assessment are warranted.
Subject areas: Neurology, Artificial intelligence, Preventive medicine
Graphical abstract
Highlights
-
•
First prognostic ML study for neurosyphilis with external validation
-
•
DEFEAT-NS-M1 shows strong discrimination and calibration across cohorts
-
•
Model stratifies AE risk to support targeted monitoring and treatment
-
•
Potential to guide trials and evaluate cost-effectiveness of care
Neurology; Artificial intelligence; Preventive medicine
Introduction
Neurosyphilis (NS), the clinical outcomes of central nervous system infection of Treponema pallidum, could cf. serious adverse prognostic events (AE) late in the disease course.1,2,3 These AE include ongoing syphilitic meningitis, meningovascular syphilis, parenchymatous neurosyphilis (general paresis and tabes dorsalis) and death (if untreated),1,4,5 which can severely impact the quality of life of affected individuals. To interrupt transmission of syphilis and prevent AE of NS, bold innovative strategies to harness innovative techniques and advances in artificial intelligence (AI) are needed to bolster or disrupt current paradigms of early diagnosis, treatment, and follow-up care of patients with NS.
Our previous work has demonstrated that the use of a machine learning (ML) algorithm to accurately diagnose NS in patients with syphilis is feasible and could be generalizable and readily available across clinical settings.6 This is particularly important given the lack of a gold standard test for NS diagnosis and the complexity of current diagnostic guidelines.6,7 We further observed that this approach can be extended to predictive modeling of AE in patients with NS. However, a comprehensive review of the existing literature on PubMed and other academic databases3 has revealed no studies so far specifically addressing prognostic prediction in NS, nor any investigations employing AI-based models for the prediction of AE in patients with NS. Due to the paucity of models on prognostic prediction in NS, clinicians currently rely on a combination of neurological features, laboratory tests (e.g., cerebrospinal fluid findings), along with traditional statistical analysis and clinical judgment to assess the risk of AE in patients with NS.3,8
Our recent protocol proposed a retrospective, multicenter, longitudinal cohort study (the DEFEAT-NS study)3 to curate a knowledge base of patients with NS and develop and validate AI-based models for prognostic prediction of NS. Here, we applied this AI framework to preliminary cross-sectional data (including demographic, clinical, laboratory, and medications and treatment history) from the DEFEAT-NS study to develop and validate an ML model, DEFEAT-NS-M1, to generate absolute risk probabilities for AE in patients with NS.
Results
Characteristics of the patients
Patient characteristics and comparisons of patients with AE and non-AE in the discovery cohort (n = 402) and external validation cohort (n = 200) were presented in Figure 1, Table 1 and Tables S1 and S2. In both cohorts, patients with AE were characterized by older age at first syphilis diagnosis, higher percentage of smoking, comorbidities (including hypertension and heart disease), history of parenchymatous neurosyphilis, syphilis treatment history, and penicillin allergy, but shorter hospitalization durations and lower rates of receiving neurosyphilis treatment at the visit. They also had older age at the visit, higher alcohol use and higher percentage of late syphilis stage, which were all significant differences compared with patients with non-AE in the discovery cohort but not in the external validation cohort.
Figure 1.
Study flowchart
Table 1.
Baseline characteristics of the discovery cohort
| Characteristics | Non-AE (N∗ = 318) | AE (N∗ = 84) | p value | Total (N∗ = 402) |
|---|---|---|---|---|
| Demographics | ||||
| Sex | 0.052 | |||
| Male | 258 (81.13) | 60 (71.43) | 318 (79.10) | |
| Female | 60 (18.87) | 24 (28.57) | 84 (20.90) | |
| Age at the visit (years) | 41 (30, 56) | 50 (38, 61) | 0.003 | 43 (31, 56) |
| Smoking | <0.001 | |||
| Yes | 39 (12.26) | 24 (28.57) | 63 (15.67) | |
| No | 202 (63.52) | 52 (61.90) | 254 (63.18) | |
| Unknown | 77 (24.21) | 8 (9.52) | 85 (21.14) | |
| Alcohol use | 0.001 | |||
| Yes | 23 (7.23) | 14 (16.67) | 37 (9.20) | |
| No | 218 (68.55) | 62 (73.81) | 280 (69.65) | |
| Unknown | 77 (24.21) | 8 (9.52) | 85 (21.14) | |
| Clinical history | ||||
| First diagnosis of syphilis at that visit | 0.003 | |||
| Yes | 56 (17.61) | 16 (19.05) | 72 (17.91) | |
| No | 259 (81.45) | 62 (73.81) | 321 (79.85) | |
| Unknown | 3 (0.94) | 6 (7.14) | 9 (2.24) | |
| Stage of syphilis at the visit | 0.004 | |||
| Early syphilis | 86 (27.04) | 11 (13.10) | 97 (24.13) | |
| Late syphilis | 139 (43.71) | 53 (63.10) | 192 (47.76) | |
| Unknown | 93 (29.25) | 20 (23.81) | 113 (28.11) | |
| First diagnosis of neurosyphilis at that visit | 0.791 | |||
| Yes | 134 (42.14) | 34 (40.48) | 168 (41.79) | |
| No | 177 (55.66) | 49 (58.33) | 226 (56.22) | |
| Unknown | 7 (2.20) | 1 (1.19) | 8 (1.99) | |
| Syphilitic meningitis | 0.013 | |||
| Yes | 87 (27.36) | 12 (14.29) | 99 (24.63) | |
| No | 231 (72.64) | 72 (85.71) | 303 (75.37) | |
| Meningovascular syphilis | 0.008 | |||
| Yes | 45 (14.15) | 22 (26.19) | 67 (16.67) | |
| No | 273 (85.85) | 62 (73.81) | 335 (83.33) | |
| Parenchymatous neurosyphilis | <0.001 | |||
| Yes | 107 (33.65) | 47 (55.95) | 154 (38.31) | |
| No | 211 (66.35) | 37 (44.05) | 248 (61.69) | |
| Asymptomatic neurosyphilis | 0.252 | |||
| Yes | 112 (35.22) | 24 (28.57) | 136 (33.83) | |
| No | 206 (64.78) | 60 (71.43) | 266 (66.17) | |
| Cancer | 0.001 | |||
| Yes | 3 (0.94) | 3 (3.57) | 6 (1.49) | |
| No | 249 (78.30) | 77 (91.67) | 326 (81.09) | |
| Unknown | 66 (20.75) | 4 (4.76) | 70 (17.41) | |
| HIV infection | <0.001 | |||
| Yes | 194 (61.01) | 25 (29.76) | 219 (54.48) | |
| No | 124 (38.99) | 57 (67.86) | 181 (45.02) | |
| Unknown | 0 (0.00) | 2 (2.38) | 2 (0.50) | |
| Pregnancy | <0.001 | |||
| Yes | 1 (0.31) | 0 (0.00) | 1 (0.25) | |
| No | 251 (78.93) | 82 (97.62) | 333 (82.84) | |
| Unknown | 66 (20.75) | 2 (2.38) | 68 (16.92) | |
| Hypertension and heart disease | <0.001 | |||
| Yes | 63 (19.81) | 30 (35.71) | 93 (23.13) | |
| No | 200 (62.89) | 52 (61.90) | 252 (62.69) | |
| Unknown | 55 (17.30) | 2 (2.38) | 57 (14.18) | |
| Diabetes | 0.001 | |||
| Yes | 34 (10.69) | 16 (19.05) | 50 (12.44) | |
| No | 228 (71.70) | 66 (78.57) | 294 (73.13) | |
| Unknown | 56 (17.61) | 2 (2.38) | 58 (14.43) | |
| Digestive disorders | <0.001 | |||
| Yes | 11 (3.46) | 8 (9.52) | 19 (4.73) | |
| No | 241 (75.79) | 72 (85.71) | 313 (77.86) | |
| Unknown | 66 (20.75) | 4 (4.76) | 70 (17.41) | |
| Liver disease | <0.001 | |||
| Yes | 23 (7.23) | 15 (17.86) | 38 (9.45) | |
| No | 231 (72.64) | 65 (77.38) | 296 (73.63) | |
| Unknown | 64 (20.13) | 4 (4.76) | 68 (16.92) | |
| Pulmonary disorders | 0.001 | |||
| Yes | 18 (5.66) | 10 (11.90) | 28 (6.97) | |
| No | 234 (73.58) | 70 (83.33) | 304 (75.62) | |
| Unknown | 66 (20.75) | 4 (4.76) | 70 (17.41) | |
| Renal disease | <0.001 | |||
| Yes | 10 (3.14) | 9 (10.71) | 19 (4.73) | |
| No | 242 (76.10) | 71 (84.52) | 313 (77.86) | |
| Unknown | 66 (20.75) | 4 (4.76) | 70 (17.41) | |
| Hematological disorders | 0.002 | |||
| Yes | 3 (0.94) | 0 (0.00) | 3 (0.75) | |
| No | 249 (78.30) | 80 (95.24) | 329 (81.84) | |
| Unknown | 66 (20.75) | 4 (4.76) | 70 (17.41) | |
| Rheumatic disease | <0.001 | |||
| Yes | 3 (0.94) | 5 (5.95) | 8 (1.99) | |
| No | 249 (78.30) | 75 (89.29) | 324 (80.60) | |
| Unknown | 66 (20.75) | 4 (4.76) | 70 (17.41) | |
| Other diseases | <0.001 | |||
| Yes | 69 (21.70) | 44 (52.38) | 113 (28.11) | |
| No | 186 (58.49) | 38 (45.24) | 224 (55.72) | |
| Unknown | 63 (19.81) | 2 (2.38) | 65 (16.17) | |
| Laboratory findings | ||||
| Serum TRUST | <0.001 | |||
| Positive | 206 (64.78) | 28 (33.33) | 234 (58.21) | |
| Negative | 6 (1.89) | 1 (1.19) | 7 (1.74) | |
| No examination | 106 (33.33) | 55 (65.48) | 161 (40.05) | |
| Serum RPR | <0.001 | |||
| Positive | 62 (19.50) | 40 (47.62) | 102 (25.37) | |
| Negative | 1 (0.31) | 2 (2.38) | 3 (0.75) | |
| No examination | 255 (80.19) | 42 (50.00) | 297 (73.88) | |
| CSF protein | 0.070 | |||
| ≤0.58 | 91 (28.62) | 35 (41.67) | 126 (31.34) | |
| >0.58 | 102 (32.08) | 23 (27.38) | 125 (31.09) | |
| No examination | 125 (39.31) | 26 (30.95) | 151 (37.56) | |
| CSF WBC | 0.325 | |||
| ≤9 | 97 (30.50) | 28 (33.33) | 125 (31.09) | |
| >9 | 91 (28.62) | 29 (34.52) | 120 (29.85) | |
| No examination | 130 (40.88) | 27 (32.14) | 157 (39.05) | |
| CSF RPR | <0.001 | |||
| Positive | 40 (12.58) | 15 (17.86) | 55 (13.68) | |
| Negative | 10 (3.14) | 12 (14.29) | 22 (5.47) | |
| No examination | 268 (84.28) | 57 (67.86) | 325 (80.85) | |
| CSF TRUST | 0.029 | |||
| Positive | 71 (22.33) | 8 (9.52) | 79 (19.65) | |
| Negative | 87 (27.36) | 25 (29.76) | 112 (27.86) | |
| No examination | 160 (50.31) | 51 (60.71) | 211 (52.49) | |
| CSF TPPA | 0.296 | |||
| Positive | 187 (58.81) | 57 (67.86) | 244 (60.70) | |
| Negative | 7 (2.20) | 2 (2.38) | 9 (2.24) | |
| No examination | 124 (38.99) | 25 (29.76) | 149 (37.06) | |
| CSF TPAB | <0.001 | |||
| Positive | 27 (8.49) | 14 (16.67) | 41 (10.20) | |
| Negative | 0 (0.00) | 3 (3.57) | 3 (0.75) | |
| No examination | 291 (91.51) | 67 (79.76) | 358 (89.05) | |
| Medications and treatment | ||||
| Treatment history for syphilis | 0.284 | |||
| Yes | 217 (68.24) | 63 (75.00) | 280 (69.65) | |
| No | 94 (29.56) | 18 (21.43) | 112 (27.86) | |
| Unknown | 7 (2.20) | 3 (3.57) | 10 (2.49) | |
| History of penicillin allergy | <0.001 | |||
| Yes | 10 (3.14) | 13 (15.48) | 23 (5.72) | |
| No | 304 (95.60) | 71 (84.52) | 375 (93.28) | |
| Unknown | 4 (1.26) | 0 (0.00) | 4 (1.00) | |
| Treatment at that visit | <0.001 | |||
| Yes | 298 (93.71) | 54 (64.29) | 352 (87.56) | |
| No | 18 (5.66) | 30 (35.71) | 48 (11.94) | |
| Unknown | 2 (0.63) | 0 (0.00) | 2 (0.50) | |
| Hospitalization days | <0.001 | |||
| ≤7 | 72 (22.64) | 50 (59.52) | 122 (30.35) | |
| (7, 14) | 82 (25.79) | 15 (17.86) | 97 (24.13) | |
| (14, 21) | 114 (35.85) | 11 (13.10) | 125 (31.09) | |
| (21, 28) | 17 (5.35) | 4 (4.76) | 21 (5.22) | |
| >28 | 31 (9.75) | 3 (3.57) | 34 (8.46) | |
| Unknown | 2 (0.63) | 1 (1.19) | 3 (0.75) | |
For brevity, extended patient characteristics of the discovery cohort were presented in Table S1. N∗ represents EHRs of patients with NS during their clinical visits and routinely collected clinical data. Continuous variables were compared using Wilcoxon rank-sum test, and categorical variables were compared using chi-squared test or Fisher’s exact test.
CSF, cerebrospinal fluid; RPR, rapid plasma reagin; TPAB, treponema pallidum antibody; TPPA, treponema pallidum particle agglutination; TRUST, toluidinered unheated serum test; WBC, white blood cell.
Key features
All three strategies for handling imbalanced classification led to slightly worse performance; thus, we used the original (unmodified) data for subsequent analyses. Figures 2A and 2B showed the feature importance of the top 20 predictors of AE in patients with NS. Based on the feature ranking, data availability, and clinical consensus of our study steering committee, we selected a panel of the top 15 key predictors for model simplification. These included the treatment at the visit, other comorbidities, serum TRUST (toluidinered unheated serum test), serum RPR (rapid plasma reagin), treatment regimen for neurosyphilis at the visit, age at first syphilis diagnosis, HIV status, syphilitic meningitis history, smoking status, previous syphilis treatment regimen, age at the visit, marital status, liver disease, first diagnosis of neurosyphilis at the visit, and parenchymatous neurosyphilis history.
Figure 2.
Feature importance, model performance, and clinical utility evaluation of the five simplified models
(A) Global bar plot for top 20 predictors.
(B) Beeswarm plot for top 20 predictors.
(C) Discrimination performance on AUCs and 95% CIs for five models in internal validation.
(D) Discrimination performance on AUCs and 95% CIs for five models in external validation.
(E) DCA curves for five models in internal validation.
(F) DCA curves for five models in external validation.
Model performance
We assessed the discrimination and calibration performance of the five simplified models on the internal validation and external validation datasets, which were presented in Figures 2C and 2D and Table 2. The RF model showed the best overall performance than the other machine learning models in both validation datasets. The AUROCs of the RF model were 0.975 (0.949–0.995) for the internal validation dataset and 0.863 (0.801–0.920) in the external validation cohort (Figures 2C and 2D). Detailed evaluation metrics were listed in Table 2. Comparing Brier scores, the RF model also had the lowest probability deviation in both types of validation (0.027 [0.014–0.041] for internal validation and 0.128 [0.092–0.167] for external validation) (Table 2). We thus chose the simplified RF model as the final model and named it DEFEAT-NS-M1.
Table 2.
Prediction performance of discrimination and calibration of the simplified machine learning models in two types of validation
| Metrics and 95% CIs |
The simplified machine learning models |
||||
|---|---|---|---|---|---|
| RF | XGBoost | SVM | LR | NB | |
| Internal validation | |||||
| AUROC | 0.975 (0.949–0.995) | 0.912 (0.877–0.946) | 0.889 (0.845–0.931) | 0.869 (0.824–0.913) | 0.802 (0.744–0.858) |
| AUPRC | 0.929 (0.868–0.976) | 0.690 (0.585–0.792) | 0.714 (0.613–0.803) | 0.601 (0.490–0.716) | 0.527 (0.426–0.635) |
| Sensitivity | 0.967 (0.918–1.000) | 0.929 (0.866–0.984) | 0.797 (0.704–0.889) | 0.606 (0.500–0.719) | 0.535 (0.416–0.654) |
| Specificity | 0.944 (0.915–0.968) | 0.732 (0.680–0.782) | 0.925 (0.894–0.954) | 0.922 (0.891–0.952) | 0.928 (0.895–0.957) |
| Accuracy | 0.947 (0.922–0.970) | 0.773 (0.729–0.817) | 0.898 (0.867–0.928) | 0.856 (0.823–0.892) | 0.845 (0.809–0.884) |
| F1 score | 0.885 (0.829–0.933) | 0.631 (0.560–0.702) | 0.765 (0.687–0.833) | 0.637 (0.552–0.725) | 0.592 (0.479–0.686) |
| PPV | 0.817 (0.736–0.894) | 0.476 (0.402–0.558) | 0.735 (0.636–0.833) | 0.672 (0.571–0.776) | 0.661 (0.542–0.782) |
| NPV | 0.992 (0.977–1.000) | 0.976 (0.953–0.995) | 0.946 (0.917–0.972) | 0.900 (0.863–0.934) | 0.883 (0.845–0.920) |
| Brier score | 0.027 (0.014–0.041) | 0.096 (0.071–0.120) | 0.084 (0.064–0.106) | 0.113 (0.089–0.139) | 0.134 (0.102–0.163) |
| External validation | |||||
| AUROC | 0.863 (0.801–0.920) | 0.820 (0.744–0.887) | 0.800 (0.718–0.872) | 0.785 (0.704–0.856) | 0.760 (0.674–0.841) |
| AUPRC | 0.607 (0.449–0.736) | 0.498 (0.352–0.652) | 0.499 (0.350–0.648) | 0.463 (0.320–0.637) | 0.468 (0.322–0.620) |
| Sensitivity | 0.886 (0.769–0.972) | 0.882 (0.771–0.972) | 0.659 (0.513–0.804) | 0.514 (0.355–0.667) | 0.486 (0.325–0.647) |
| Specificity | 0.783 (0.714–0.849) | 0.676 (0.599–0.75) | 0.812 (0.745–0.876) | 0.845 (0.784–0.900) | 0.895 (0.839–0.941) |
| Accuracy | 0.800 (0.744–0.861) | 0.717 (0.650–0.778) | 0.783 (0.722–0.839) | 0.778 (0.711–0.833) | 0.811 (0.756–0.867) |
| F1 score | 0.646 (0.532–0.748) | 0.555 (0.444–0.662) | 0.553 (0.429–0.667) | 0.486 (0.337–0.617) | 0.514 (0.364–0.638) |
| PPV | 0.509 (0.389–0.636) | 0.405 (0.305–0.526) | 0.475 (0.345–0.615) | 0.457 (0.306–0.622) | 0.543 (0.371–0.700) |
| NPV | 0.964 (0.922–0.991) | 0.958 (0.913–0.990) | 0.903 (0.850–0.950) | 0.871 (0.811–0.926) | 0.872 (0.816–0.926) |
| Brier score | 0.128 (0.092–0.167) | 0.146 (0.103–0.192) | 0.138 (0.101–0.179) | 0.157 (0.116–0.202) | 0.156 (0.111–0.205) |
CI, confidence interval; RF, random forest; XGBoost, eXtreme gradient boosting; SVM, support vector machine; LR, logistic regression; NB, naive Bayes; AUROC, area under the receiver operating characteristics curve; AUPRC, area under the precision-recall curve; PPV, positive predictive value; NPV, negative predictive value.
In Figures 2E and 2F, DCA curves showed that the simplified NB model was associated with the lowest net benefit compared with the other four models in both validation datasets. All other models had a similar or better net benefit association compared with the treat-all or clinically unrealistic treat-none strategies across a wide range of threshold probabilities. As the risk probabilities predicted by the DEFEAT-NS-M1 model increased, the absolute risk of AE in patients with NS increased (Table 3). On the basis of the result from discrimination, calibration, and clinical utility evaluation, predicted risk probabilities from the DEFEAT-NS-M1 model were stratified into two risk categories: low risk (<25%) and high risk (≥25%). Accordingly, in the discovery cohort, 75.37% of patients with NS were considered to be at low risk, with an observed AE rate of 1.00%, whereas 24.63% were classified as high risk, with an observed AE rate of 81.82%. Overall, patients at high risk accounted for 96.43% of all AE cases in the discovery cohort. The NTTs in patients at low risk and those at high risk were 101 and 1, respectively. Similar findings were observed in the external validation cohort, patients at high risk accounted for 87.80% of all AE cases. The respective NTTs in patients at low risk and those at high risk were 26 and 2 (Table 3).
Table 3.
Risk probabilities of AE determined by the DEFEAT-NS-M1 model and risk category
| Risk probability | Risk category | Total number (%) | AE number | NTT |
|---|---|---|---|---|
| Discovery cohort | 402 (100.00) | 84 (20.90) | ||
| <25% | Low | 303 (75.37) | 3 (1.00) | 101 |
| ≥25% | High | 99 (24.63) | 81 (81.82) | 1 |
| External validation cohort | 200 (100.00) | 41 (20.50) | ||
| <25% | Low | 129 (64.50) | 5 (3.88) | 26 |
| ≥25% | High | 71 (35.50) | 36 (50.70) | 2 |
AE, adverse prognostic events; NTT, the number needed to treat.
Discussion
This study represents the special effort to develop clinical prediction models for patients with NS. In addition, it constitutes the largest prognostic investigation to date estimating the risk of adverse events within the NS population, attributable to the disease’s complexity and the challenges associated with patient recruitment. In this retrospective, observational, multicenter cohort study, we developed, calibrated, and validated the DEFEAT-NS-M1 model to predict AE in patients with NS using observations during their clinical visits and routinely collected clinical data. The DEFEAT-NS-M1 model appeared the most clinically useful, on the basis of its high discrimination, good calibration, and association with favorable net benefit in both discovery and external validation cohorts. Excellent clinical impact was demonstrated by the low NTTs in patients at high risk of AE across both cohorts. Potential model uses include identification of patients at high risk of developing serious AE, expansion of access to screening and early treatment, modification of monitoring and treatment strategies, or enrollment in clinical trials for risk-stratified management of NS or evaluation of innovative therapeutics.
NS accounts for 1.8%–3.5% of syphilis cases and its diagnosis and treatment are often challenging because of the lack of a gold standard test, polymorphic clinical manifestations, and strong association with HIV infection.1,6,9,10 The heterogeneity of NS and its AE has important clinical implications for risk-stratified screening, treatment or monitoring strategies. First, variation in AE across NS subtypes has motivated us to include the NS subtype information1,10 as candidate predictors in our modeling and to develop and assess risk-stratified management strategies supported by the DEFEAT-NS-M1 model. Thus far, no prognostic models have been developed to predict AE and other prognostic outcomes of this disease in such subgroups.3 Second, the complexity of NS heterogeneity causes large sample sizes to be required for rigorous modeling studies or comparative trials on treatment regimens and monitoring strategies for NS, in addition to decades of clinical expertise. Along with European, US, and UK guidelines,5,11,12 previous observational studies10,13,14,15 have recommended different treatment regimens for NS, including intravenous benzylpenicillin (24 million international units per day for 10–14 days), procaine penicillin G (2.4 million units IM once daily) plus probenecid (500 mg orally 4 times per day) for 10 to 14 days, ceftriaxone (at a dose of 1–2 g IV daily for 10–14 days), doxycycline (200 mg twice daily for 28 days). The present study had an explicit focus on including treatment regimen for neurosyphilis at the visit as one of the top 15 key predictors in the DEFEAT-NS-M1 model, which could help to identify the safety and efficacy of different treatment regimens for patients with NS in different risk categories. Third, the DEFEAT-NS-M1 model could be used to identify patients at high risk of AE who may benefit from more intensive monitoring and treatment strategies, thereby optimizing resource allocation and improving patient outcomes. Oppositely, those at low risk of AE may benefit from less intensive monitoring and treatment, thereby reducing healthcare costs and minimizing unnecessary interventions. The DEFEAT-NS-M1 model could thus facilitate risk-stratified management of NS in clinical practice.
A thorough understanding of the risk predictors of AE in patients with NS is crucial for identifying high-risk individuals and implementing targeted interventions. In the present study, treatment information for syphilis and NS at patient visits (including treatment regimen, timing, and adherence) were among the top predictors that were significantly associated with the absolute risk of AE in patients of NS, underscoring the importance of timely and appropriate treatment in preventing AE. This finding is consistent with existing literature that highlights the critical role of treatment in influencing disease progression and outcomes in NS. Given the NS heterogeneity and complexity, more rigorous comparative trials are needed to evaluate the safety and efficacy of different treatment regimens for NS in patients at varying levels of risk for AE predicted by the DEFEAT-NS-M1 model in future studies. In other words, the DEFEAT-NS-M1 model could be used to identify patients who may benefit from specific treatment regimens based on their risk profile, thereby facilitating personalized treatment approaches and duration for NS and improving design, recruitment, and subgroup analyses of future clinical trials on NS treatment.16 Of note, aging and comorbidities (e.g., liver disease and some other comorbidities) were also identified as important predictors of AE. The elderly and those with comorbidities may have a higher risk of AE due to safety and tolerability of drug-induced adverse events, drug-drug interactions, and age-related physiological changes.17,18 This finding highlights the need for tailored management strategies that consider the unique needs and challenges faced by these patient subpopulations. In addition, lifestyle factors such as smoking and family history were also found to be associated with an increased risk of AE in patients with NS. This suggests that modifiable risk factors may play a role in disease progression and outcomes, and interventions targeting these factors could potentially improve patient outcomes. For example, smoking cessation programs and lifestyle modifications could be integrated into the management of patients with NS to reduce the risk of AE. In this study, almost all known key risk predictors for AE in patients with NS were included in the analyses, and the DEFEAT-NS-M1 model demonstrated excellent performance in predicting AE across both internal and external validation cohorts. Therefore, when the intended use of the DEFEAT-NS-M1 model is to predict AE in patients with NS at the time of their clinical visits and identify those at high risk of AE, the model is believed to be reliable and robust, and thus, has the generalizable potential to be applied in the diverse real-world clinical settings to guide risk-stratified management of NS.
Limitations of the study
Study strengths include the large sample size in a rare disease, the multicenter design, the setup and use of the study steering committee to ascertain outcomes and predictors, and the evaluation strategy. However, this study also has several limitations. First, the retrospective design may introduce selection bias and limit the ability to ascribe any causal interpretation. However, the use of a multicenter cohort and rigorous data quality assessment helps to mitigate these bias concerns. Second, the generalizability of the DEFEAT-NS-M1 model to other populations and settings needs to be further validated in prospective studies and diverse clinical environments outside of China. We compared baseline characteristics between our cohorts. There were some differences in clinical variables (e.g., age at visit, alcohol use, and late syphilis stage). These differences were explicitly reported and are a key reason we used a center-based external validation and calibration (isotonic regression) to assess transportability. The model retained good discrimination and calibration in the external validation cohort (AUROC 0.863 [0.801–0.920]; Brier 0.128 [0.092–0.167]), suggesting that the observed differences did not materially compromise performance. Third, the inability to incorporate genetic risk factors, due to the lack of genetic information in the source data, may limit the model’s predictive performance. Thus this study sought to develop a clinically useful model based on routinely collected clinical data only that would be readily available and accessible in real-world clinical practice. We envision the DEFEAT-NS-M1 model could inform the identification of patients at high risk of AE, in conjunction with clinical judgment, and beyond the current standard of care. Other limitation includes reliance on EHRs, which may be subject to inaccuracies for some predictor values. Finally, this study only used data from six centers in China, as such, the findings may not necessarily generalize to other countries.
In conclusion, this study explored five models to predict AE in patients with NS. The DEFEAT-NS-M1 model was deemed the most clinically useful, on the basis of its high discrimination, good calibration, and association with favorable net benefit in both discovery and external validation cohorts. Accurate tool that can identify patients at high risk of developing serious AE could inform efficient targeting of patients most likely to benefit from early treatment and intensive monitoring, personalized management strategies, or enrollment into trials of innovative therapeutics. This study provides a foundation for future prospective studies to further validate and refine the DEFEAT-NS-M1 model and to explore its potential applications in risk-based clinical practice and research on NS. Future work should include more external validation outside of China and health economic evaluation before clinical implementation.
Resource availability
Lead contact
Requests for further information and resources should be directed to and will be fulfilled by the lead contact, Huachun Zou (zouhuachun@fudan.edu.cn).
Materials availability
This study did not generate new unique reagents.
Data and code availability
-
•
All data reported in this paper will be shared by the lead contact upon request.
-
•
This paper does not report original code.
-
•
Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.
Acknowledgments
This study was supported by the National Key Technologies R&D Program of China (2023YFC2306700), the Prevention and Control of Emerging and Major Infectious Diseases-National Science and Technology Major Project (2025ZD01905200), and the Natural Science Foundation of China General Program [82574167, 82473539], Sanming Project of Medicine in Shenzhen (No. SZSM202311033), the Eastern Talent Plan Youth Project 2024, and Development and Clinical Application of a Multimodal Early Warning Model for Neurological Syphilis and Associated Paralytic Dementia (AF7320007, YG2025ZD29). All funding parties did not have any role in the design of the study or in the explanation of the data. We thank all contributors and all participating centers for their efforts in the DEFEAT-NS study. We also thank all funding sources for their support of this work.
Author contributions
Conceptualization, Z.L. and H. Zou; data curation, J.Z., H. Zhang, M.Z., Y.F., H.S., W.W., B.X., R.W., X.Y., L.G., J.L., L.Y., and J.C.; formal analysis, Z.L. and H. Zou; funding acquisition, H. Zou, S.L., Y.C., J.C., L.Y., and J.C.; investigation, Z.L., J.Z., H. Zhang, M.Z., Y.H., R.Z., W.W., J.C., S.L., Y.C., Z.P., L.Y., J.C., and H. Zou; methodology, Z.L. and H. Zou; project administration, H. Zou, J.Z., M.Z., Y.F., J.L., L.Y., and J.C.; resources, H. Zou, J.Z., M.Z., Y.F., J.L., L.Y., and J.C.; software, Z.L.; supervision, H. Zou, J.Z., M.Z., Y.F., J.L., L.Y., J.C., H. Zhang, R.Z., W.W., S.L., Y.C., and Z.P.; validation, H. Zhang, J.Z., M.Z., Y.F., J.L., L.Y., J.C., H. Zou, R.Z., W.W., S.L., Y.C., and Z.P.; visualization, Z.L.; writing – original draft, Z.L.; writing – review & editing, all authors.
Declaration of interests
The authors declare no competing interests.
STAR★Methods
Key resources table
| REAGENT or RESOURCE | SOURCE | IDENTIFIER |
|---|---|---|
| Software and algorithms | ||
| Python (version 3.11.6) | Python Software Foundation | https://www.python.org/psf/ |
| R (version 4.3.2) | The R Foundation | https://www.r-project.org/foundation/ |
Experimental model and study participant details
Data sources and participants
As stated in our previous protocol,3 the DEFEAT-NS study is a retrospective, multicenter cohort study. The six prespecified participating centers across China include Peking Union Medical College Hospital, Shanghai Public Health Clinical Center, The Fourth People’s Hospital of Nanning, The Third People’s Hospital of Shenzhen, Guiyang Public Health Clinical Center, and Nantong Third People’s Hospital (Figure 1). The study steering committee comprises a multidisciplinary panel of experienced specialist physicians, biostatisticians, and epidemiologists from each center. Participants were eligible for inclusion if they were diagnosed with NS according to the published diagnostic criteria3,6 and had available electronic health records (EHRs). No further exclusion criteria were applied. The inclusion and exclusion criteria were intentionally broad to represent the real-world clinical setting and the target population of interest and to ensure the generalizability of the model.19 We performed a retrospective cohort study using EHRs of patients with NS during their clinical visits and routinely collected clinical data.
Ethical considerations
The DEFEAT-NS study is a retrospective analysis of existing EHRs and does not involve any direct patient contact or intervention. This study was approved by the ethical committee of the School of Public Health, Fudan University (2025-TYSQ-02-478). The study protocol has been peer-reviewed and published.3 The study adheres to the Declaration of Helsinki and standards of Good Clinical Practice and complies with local regulations regarding data protection. All data were anonymized before analysis to ensure patient privacy and confidentiality.
Randomization, masking and data preprocessing
Given the retrospective nature of the DEFEAT-NS study, randomization was not applicable. Although the data are cross-sectional at the visit level, we formed the discovery and external validation cohorts by center to provide a true geographic/setting-based external validation. Specifically, we randomly selected two of the six centers to constitute the external validation cohort and used the remaining four centers as the discovery cohort (Figure 1). Blinding of predictor assessors to outcome information occurs naturally in our prognostic study when prognostic predictors are assessed before the outcome occurs. No masking was applied during data collection or analysis.
The EHRs are systematically and validly collected for the purpose of research and deidentified before being shared with the research team. Additionally, we undertook a comprehensive data quality assessment to ensure the accuracy and completeness of the participant data. Predictors were extracted and assessed in the same way for all study participants to reduce potential risk of bias. We ensured to minimize differences in data collection and processing across centers by using a standard form on which the items and possible responses were predefined, and by standardizing the data extraction and preprocessing procedures.
Method details
Predictors
As outlined in our previous protocol,3 we pre-specified a comprehensive set of candidate predictors based on previous research evidence1,4,20,21 and consensus from a panel of experienced clinicians in neurosyphilis, epidemiologists, and biostatisticians. A directed acyclic graph guided the selection of candidate features.3 After data preprocessing, we entered all of them into the initial model development.6,22 These predictors encompassed demographic information, clinical history, laboratory test results, medication and treatment details. List of candidate predictors are provided in Tables 1 and S1. All candidate predictors are available at the time our model is intended to be used in real-world clinical practice.
Outcome
The primary outcome of interest was prespecified in our protocol3 as the occurrence of any AE in patients with NS, including ongoing syphilitic meningitis, meningovascular syphilis, parenchymatous neurosyphilis (general paresis and tabes dorsalis), and death (if untreated).1,4,5 The outcome required determination by experienced clinicians using well-established criteria and appropriate measures.3,6 Risk of bias in the time interval between predictor assessment and outcome determination was minimized because the same clinic visit was used to measure predictors from patient inquiries and physical examinations and to collect biological samples for determination of the outcome of AE.19 In this multicenter study, the study steering committee must reach a consensus on the outcome adjudication to ensure that the outcome determinations classified the outcome status correctly in all study participants and consistently across all centers.
Study design and modeling
The study design consisted of five steps3,6,22: development, calibration, validation, clinical utility evaluation, and explanation. We developed five predictive models with a training dataset in the discovery cohort. Thereafter, we calibrated the models using a smaller calibration dataset in the discovery cohort. Next, we performed internal validation and external validation using data from the discovery cohort and external validation cohort, respectively. We compared model performance against other four machine learning algorithms. Next, we performed clinical utility evaluation using decision curve analysis (DCA).6 Finally, we evaluated model interpretability by examining the number needed to treat (NTT). The best-performing model was selected as the final model and named DEFEAT-NS-M1. This study follows the TRIPOD+AI statement.23
As per our previous work,22 from the discovery cohort, we randomly and proportionally sampled 80% patients to train models and tune hyperparameters, 10% patients to calibrate the models, and the remaining 10% patients to perform internal validation. Thus, modeling was not performed on the entire discovery cohort without a test split; internal validation was actually served as the test dataset. We used five machine learning algorithms to develop the models: random forest (RF), eXtreme Gradient Boosting (XGBoost), support vector machine (SVM), logistic regression (LR), and naive Bayes (NB). The five algorithms were selected because they are widely used, well-validated methods for clinical prediction tasks and provide complementary modeling assumptions (linear vs nonlinear, probabilistic vs ensemble/tree-based),6,24,25,26 allowing a balanced comparison of performance and interpretability. Optimization of hyperparameters was performed using 10-fold cross-validation and Bayesian optimization.6,27 We experimented with three strategies to handle class imbalance: oversampling, undersampling, and class-weight adjustment.28 As the prevalence of AE in patients with NS in the six study centers were different, we calibrated the models using an isotonic regression algorithm22 in order to obtain much more relevant risk probabilities to the true observed incidence of AE in patients with NS. Although the discovery cohort pooled four centers, AE prevalence still differed across centers; therefore, we applied isotonic regression to ensure that predicted risks accurately reflected actual risks across different settings and improve transportability. Calibration performance was assessed by Brier score.6 To assess feature importance, we used the SHapley Additive exPlanations (SHAP) method to identify the relative contribution of each feature to the final prediction.6,29 SHAP values were calculated to explain whether candidate predictors had a positive or negative association with AE in patients with NS.30 Feature importance was then assessed using SHAP, and—based on the importance ranking, data availability/accessibility, and clinical consensus from the study steering committee—we selected the top 15 predictors to build simplified models.6,22
We used eight metrics25,26,31 to evaluate model performance, including AUROC (area under the receiver operating characteristics curve), AUPRC (area under the precision-recall curve), sensitivity, specificity, accuracy, F1 score, PPV (positive predictive value), and NPV (negative predictive value). We selected AUROC and AUPRC for discriminative performance evaluation.25 We compared the performance of the five simplified models on the internal validation and external validation datasets. DCA is a method for evaluating prediction models that incorporates clinical consequences, thus we used it to evaluate the clinical utility of the models by quantifying the net benefits at different threshold probabilities.6 The NTT32 was defined as the number of patients who would need to be treated to prevent one AE in patients with NS. We calculated the NTT to assess the clinical impact of using the model to guide treatment decisions. This optimal threshold of the DEFEAT-NS-M1 model was identified by maximizing the Youden index25 in the discovery cohort. We used the threshold for stratification, the low- vs high-risk grouping based on the model-predicted probability of AE, to compare observed AE rates and NNT across risk strata.
Quantification and statistical analysis
Summary statistics were presented as medians (Q1, Q3) and frequencies (percentages), as appropriate. Continuous variables were compared using Wilcoxon rank-sum test, and categorical variables were compared using chi-squared test or Fisher’s exact test. Findings were considered statistically significant at a two-sided P value of less than 0.05. We used nonparametric bootstrap resampling6,33 of the original sets: each of the 500 resamples was drawn with replacement from the full original cohort (same sample size as the original set), and performance metrics were computed per resample to derive 95% confidence intervals (CIs). All analyses were conducted in Python (version 3.11.6) and R (version 4.3.2).
Published: February 20, 2026
Footnotes
Supplemental information can be found online at https://doi.org/10.1016/j.isci.2026.115104.
Contributor Information
Zhihang Peng, Email: zhihangpeng@163.com.
Jun Li, Email: lijun35@hotmail.com.
Liuqing Yang, Email: 350281813@qq.com.
Jun Chen, Email: qtchenjun@163.com.
Huachun Zou, Email: zouhuachun@fudan.edu.cn.
Supplemental information
References
- 1.Ropper A.H. Neurosyphilis. N. Engl. J. Med. 2019;381:1358–1363. doi: 10.1056/NEJMra1906228. [DOI] [PubMed] [Google Scholar]
- 2.Ghanem K.G., Ram S., Rice P.A. The modern epidemic of syphilis. N. Engl. J. Med. 2020;382:845–854. doi: 10.1056/NEJMra1901593. [DOI] [PubMed] [Google Scholar]
- 3.Lu Z., Yang L., Li J., Wang J., Wu W., Fu L., Wang B., Tian T., Zhang H., Peng Z., et al. Curating a knowledge base for patients with neurosyphilis: a study protocol of a DEep learning Framework for pErsonalized prediction of Adverse prognosTic events in NeuroSyphilis (DEFEAT-NS) BMJ Open. 2025;15 doi: 10.1136/bmjopen-2024-092248. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Hamill M.M., Ghanem K.G., Tuddenham S. State-of-the-Art Review: Neurosyphilis. Clin. Infect. Dis. 2024;78:e57–e68. doi: 10.1093/cid/ciad437. [DOI] [PubMed] [Google Scholar]
- 5.Workowski K.A., Bachmann L.H., Chan P.A., Johnston C.M., Muzny C.A., Park I., Reno H., Zenilman J.M., Bolan G.A. Sexually transmitted infections treatment guidelines, 2021. MMWR Recomm. Rep. (Morb. Mortal. Wkly. Rep.) 2021;70:1–187. doi: 10.15585/mmwr.rr7004a1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Zou H., Lu Z., Weng W., Yang L., Yang L., Leng X., Wang J., Lin Y.F., Wu J., Fu L., et al. Diagnosis of neurosyphilis in HIV-negative patients with syphilis: development, validation, and clinical utility of a suite of machine learning models. eClinicalMedicine. 2023;62 doi: 10.1016/j.eclinm.2023.102080. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Tuddenham S., Ghanem K.G. Neurosyphilis: knowledge gaps and controversies. Sex. Transm. Dis. 2018;45:147–151. doi: 10.1097/OLQ.0000000000000723. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Smyth M., Diaz M., Saylor D. Chronic meningitis: diagnostic and therapeutic challenges. Curr. Opin. Infect. Dis. 2025;38:252–260. doi: 10.1097/qco.0000000000001107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Gordon S.M., Eaton M.E., George R., Larsen S., Lukehart S.A., Kuypers J., Marra C.M., Thompson S. The Response of Symptomatic Neurosyphilis to High-Dose Intravenous Penicillin G in Patients with Human Immunodeficiency Virus Infection. N. Engl. J. Med. 1994;331:1469–1473. doi: 10.1056/NEJM199412013312201. [DOI] [PubMed] [Google Scholar]
- 10.Bettuzzi T., Jourdes A., Robineau O., Alcaraz I., Manda V., Molina J.M., Mehlen M., Cazanave C., Tattevin P., Mensi S., et al. Ceftriaxone compared with benzylpenicillin in the treatment of neurosyphilis in France: a retrospective multicentre study. Lancet Infect. Dis. 2021;21:1441–1447. doi: 10.1016/S1473-3099(20)30857-4. [DOI] [PubMed] [Google Scholar]
- 11.Janier M., Unemo M., Dupin N., Tiplica G.S., Potočnik M., Patel R. 2020 European guideline on the management of syphilis. J. Eur. Acad. Dermatol. Venereol. 2021;35:574–588. doi: 10.1111/jdv.16946. [DOI] [PubMed] [Google Scholar]
- 12.Kingston M., Apea V., Evans C., Fifer H., Foster K., Patrick P., Grant A., Manns V., Ramsden S., Sinka K., et al. BASHH UK guidelines for the management of syphilis 2024. Int. J. STD AIDS. 2024;35:1142–1160. doi: 10.1177/09564624241280406. [DOI] [PubMed] [Google Scholar]
- 13.Dunaway S.B., Maxwell C.L., Tantalo L.C., Sahi S.K., Marra C.M. Neurosyphilis Treatment Outcomes After Intravenous Penicillin G Versus Intramuscular Procaine Penicillin Plus Oral Probenecid. Clin. Infect. Dis. 2020;71:267–273. doi: 10.1093/cid/ciz795. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Girometti N., Junejo M.H., Nugent D., McOwan A., Whitlock G., 56 Dean Street Collaborative Group Clinical and serological outcomes in patients treated with oral doxycycline for early neurosyphilis. J. Antimicrob. Chemother. 2021;76:1916–1919. doi: 10.1093/jac/dkab100. [DOI] [PubMed] [Google Scholar]
- 15.Bao Y.K., Hwang J., Long C., Davar K., Kapadia N., Spellberg B., Wong B., Rao N., Toy B.C. Outpatient Oral Doxycycline Therapy for Ocular Syphilis. JAMA Netw. Open. 2024;7:e2449364. doi: 10.1001/jamanetworkopen.2024.49364. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Hook E.W., 3rd, Dionne J.A., Workowski K., McNeil C.J., Taylor S.N., Batteiger T.A., Dombrowski J.C., Mayer K.H., Seña A.C., Hamill M.M., et al. One Dose versus Three Doses of Benzathine Penicillin G in Early Syphilis. N. Engl. J. Med. 2025;393:869–878. doi: 10.1056/NEJMoa2401802. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Sipos M., Farcas A., Leucuta D.C., Bucsa C., Huruba M., Mogosan C. Second-Generation Cephalosporins-Associated Drug-Induced Liver Disease: A Study in VigiBase with a Focus on the Elderly. Pharmaceuticals. 2021;14:441. doi: 10.3390/ph14050441. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Chen Y.-C., Fan J.-S., Chen M.-H., Hsu T.-F., Huang H.-H., Cheng K.-W., Yen D.H.-T., Huang C.-I., Chen L.-K., Yang C.-C. Risk factors associated with adverse drug events among older adults in emergency department. Eur. J. Intern. Med. 2014;25:49–55. doi: 10.1016/j.ejim.2013.10.006. [DOI] [PubMed] [Google Scholar]
- 19.Moons K.G.M., Wolff R.F., Riley R.D., Whiting P.F., Westwood M., Collins G.S., Reitsma J.B., Kleijnen J., Mallett S. PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration. Ann. Intern. Med. 2019;170:W1–W33. doi: 10.7326/M18-1377. [DOI] [PubMed] [Google Scholar]
- 20.Berger J.R., Dean D. Neurosyphilis. Handb. Clin. Neurol. 2014;121:1461–1472. doi: 10.1016/B978-0-7020-4088-7.00098-5. [DOI] [PubMed] [Google Scholar]
- 21.Workowski K.A., Bachmann L.H. Centers for Disease Control and Prevention's Sexually Transmitted Diseases Infection Guidelines. Clin. Infect. Dis. 2022;74:S89–S94. doi: 10.1093/cid/ciab1055. [DOI] [PubMed] [Google Scholar]
- 22.Lu Z., Dong B., Cai H., Tian T., Wang J., Fu L., Wang B., Zhang W., Lin S., Tuo X., et al. Identifying Data-Driven Clinical Subgroups for Cervical Cancer Prevention With Machine Learning: Population-Based, External, and Diagnostic Validation Study. JMIR Public Health Surveill. 2025;11 doi: 10.2196/67840. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Collins G.S., Moons K.G.M., Dhiman P., Riley R.D., Beam A.L., Van Calster B., Ghassemi M., Liu X., Reitsma J.B., van Smeden M., et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385 doi: 10.1136/bmj-2023-078378. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Lu Z., Zhou X., Fu L., Li Y., Tian T., Liu Q., Zou H. Prognostic prediction models for oropharyngeal squamous cell carcinoma (OPSCC): a protocol for systematic review, critical appraisal and meta-analysis. BMJ Open. 2023;13 doi: 10.1136/bmjopen-2023-073375. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Alonzo T.A. Oxford University Press; 2009. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating: by Ewout W. Steyerberg. [Google Scholar]
- 26.Kuhn M., Johnson K. Springer; 2013. Applied Predictive Modeling. [Google Scholar]
- 27.Al’Aref S.J., Maliakal G., Singh G., van Rosendael A.R., Ma X., Xu Z., Alawamlh O.A.H., Lee B., Pandey M., Achenbach S., et al. Machine learning of clinical variables and coronary artery calcium scoring for the prediction of obstructive coronary artery disease on coronary computed tomography angiography: analysis from the CONFIRM registry. Eur. Heart J. 2020;41:359–367. doi: 10.1093/eurheartj/ehz565. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Liu X., Hu P., Yeung W., Zhang Z., Ho V., Liu C., Dumontier C., Thoral P.J., Mao Z., Cao D., et al. Illness severity assessment of older adults in critical illness using machine learning (ELDER-ICU): an international multicentre study with subgroup bias evaluation. Lancet Digit. Health. 2023;5:e657–e667. doi: 10.1016/s2589-7500(23)00128-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Lundberg S.M., Erion G., Chen H., DeGrave A., Prutkin J.M., Nair B., Katz R., Himmelfarb J., Bansal N., Lee S.I. From Local Explanations to Global Understanding with Explainable AI for Trees. Nat. Mach. Intell. 2020;2:56–67. doi: 10.1038/s42256-019-0138-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Lundberg S.M., Nair B., Vavilala M.S., Horibe M., Eisses M.J., Adams T., Liston D.E., Low D.K.W., Newman S.F., Kim J., Lee S.I. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nat. Biomed. Eng. 2018;2:749–760. doi: 10.1038/s41551-018-0304-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Han J. 4th edition Edition. Elsevier; 2023. Data Mining Concepts and Techniques. [Google Scholar]
- 32.Gallin J.I., Ognibene F.P. Academic Press; 2012. Principles and Practice of Clinical Research. [Google Scholar]
- 33.Sanchez-Gendriz I., Carvalho D.D.A., Galvão-Lima L.J., Sales-Moioli A.I.L., Brito T., Fernandes F., Henriques J., Lima T., Guedes L.A., Cruz A.S., et al. Digital dual test syphilis/HIV detection based on Fourier Descriptors of Cyclic Voltammetry curves. Comput. Biol. Med. 2024;174 doi: 10.1016/j.compbiomed.2024.108454. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
-
•
All data reported in this paper will be shared by the lead contact upon request.
-
•
This paper does not report original code.
-
•
Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.