Table 2.
Performance Metrics for the Models Tested for INSIGHT-CM Based on the Strongest Threshold for Each
| PPV (%) (95% CI) |
NPV (%) (95% CI) |
Sensitivity (%) (95% CI) |
Specificity (%) (95% CI) |
F1 (%) (95% CI) |
AUROCa (%) (95% CI) |
Screened Positive (%) (95% CI) |
|
|---|---|---|---|---|---|---|---|
| Clinical ModernBERT | 19.9 (17.6-22.36) | 99.6 (99.5-99.7) | 77.5 (75.9-82.8) | 94.745 (94.4-95.0) | 31.71 (28.5-34.8) | 92.4 (90.5-94.3) | 6.5 (6.1-7.1) |
| BRNN | 19.5 (17.4-21.2) | 99.6 (98-100) | 77.2 (75.5-81.5) | 94.4 (90-97.5) | 31.69 (30.5-33.5) | 92.5 (90.5-94.1) | 6.6 (6.4-7.2) |
| LGBM | 17.1 (14.3-18.5) | 99.8 (99.8-99.9) | 91.9 (90.6-93.6) | 92.4 (88.9-93.0) | 28.8 (27.0-30.0) | 97.2 (95.4-98.6) | 9.1 (8.6-9.4) |
| HGBC | 17.0 (15.4-18.4) | 99.8 (99.7-99.9) | 90.7 (89.7-92.8) | 92.5 (91.3-94.4) | 28.6 (26.5-29.6) | 97.2 (94.5-97.7) | 8.9 (8.5-9.3) |
| RFC | 15.9 (9.2-17.2) | 99.6 (99.5-99.7) | 79.8 (78.2-81.2) | 89.6 (88.9-92.0) | 26.2 (22.7-28.7) | 92.4 (91.2-94.3) | 11.6 (11.1-12.0) |
| BBC | 14.1 (12.6-15.7) | 99.6 (99.6-99.7) | 69.9 (67.8-70.9) | 88.2 (85.7-88.7) | 23.5 (16.7-24.8) | 91.1 (90.6-93.7) | 12.8 (12.3-13.2) |
| XGB | 13.1 (9.0-15.1) | 99.8 (99.8-100.0) | 90.7 (90.4-93.5) | 89.8 (84.4-90.4) | 22.9 (15.9-23.4) | 96.3 (94.2-97.5) | 11.5 (11.1-12.0) |
AUROC = area under the receiver-operating characteristic curve; BBC = Balanced Bagging Classifier; BRNN = Bidirectional Recurrent Neural Network; HGBC = Hist Gradient Boosting Classifier; LGBM = Light Gradient Boosting Classifier; NPV = negative predictive value; PPV = positive predictive value; RFC = Random Forest Classifier; XGB = Extreme Gradient Boosting Classifier.
a
Based on continuous model.