Table 3.
Model performances by algorithm and variable group.
| Algorithm and variable group | AUROC (95% CI) | Sensitivity (95% CI) | Specificity (95% CI) | Positive predictive value (95% CI) | Negative predictive value (95% CI) | F1 score (95% CI) | |||||||
| Logistic regression | |||||||||||||
|
|
1 | 0.802 (0.802-0.802) | 0.907 (0.907-0.907) | 0.451 (0.451-0.451) | 0.293 (0.293-0.293) | 0.951 (0.951-0.951) | 0.443 (0.443-0.443) | ||||||
|
|
2 | 0.8 (0.8-0.8) | 0.901 (0.901-0.901) | 0.479 (0.479-0.479) | 0.303 (0.303-0.303) | 0.95 (0.95-0.95) | 0.453 (0.453-0.453) | ||||||
|
|
3 | 0.788 (0.788-0.788) | 0.901 (0.901-0.901) | 0.521 (0.521-0.521) | 0.321 (0.321-0.321) | 0.954 (0.954-0.954) | 0.473 (0.473-0.473) | ||||||
|
|
4 | 0.771 (0.771-0.771) | 0.901 (0.901-0.901) | 0.473 (0.473-0.473) | 0.3 (0.3-0.3) | 0.95 (0.95-0.95) | 0.45 (0.45-0.45) | ||||||
| Random forest | |||||||||||||
|
|
1a | 0.889 (0.888-0.889) | 0.901 (0.901-0.902) | 0.722 (0.719-0.724) | 0.449 (0.447-0.451) | 0.967 (0.967-0.967) | 0.599 (0.597-0.601) | ||||||
|
|
2 | 0.872 (0.872-0.873) | 0.901 (0.901-0.902) | 0.669 (0.667-0.672) | 0.407 (0.405-0.409) | 0.964 (0.964-0.964) | 0.56 (0.559-0.562) | ||||||
|
|
3 | 0.869 (0.869-0.87) | 0.902 (0.901-0.902) | 0.642 (0.639-0.645) | 0.388 (0.386-0.39) | 0.963 (0.963-0.963) | 0.542 (0.54-0.544) | ||||||
|
|
4 | 0.87 (0.87-0.871) | 0.901 (0.901-0.901) | 0.669 (0.666-0.672) | 0.407 (0.404-0.409) | 0.964 (0.964-0.964) | 0.56 (0.558-0.562) | ||||||
| Extra treeb | |||||||||||||
|
|
1 | 0.881 (0.88-0.881) | 0.901 (0.901-0.902) | 0.67 (0.665-0.675) | 0.408 (0.404-0.412) | 0.964 (0.964-0.965) | 0.561 (0.558-0.565) | ||||||
|
|
2 | 0.882 (0.881-0.882) | 0.902 (0.901-0.902) | 0.715 (0.711-0.719) | 0.443 (0.44-0.447) | 0.967 (0.966-0.967) | 0.594 (0.591-0.597) | ||||||
|
|
3 | 0.879 (0.879-0.88) | 0.901 (0.901-0.901) | 0.708 (0.703-0.713) | 0.438 (0.434-0.442) | 0.966 (0.966-0.966) | 0.589 (0.585-0.592) | ||||||
|
|
4 | 0.879 (0.878-0.879) | 0.902 (0.901-0.903) | 0.717 (0.713-0.721) | 0.445 (0.442-0.448) | 0.967 (0.967-0.967) | 0.596 (0.593-0.599) | ||||||
| Gradient boosting | |||||||||||||
|
|
1 | 0.861 (0.861-0.862) | 0.901 (0.901-0.901) | 0.621 (0.621-0.621) | 0.374 (0.374-0.374) | 0.961 (0.961-0.961) | 0.528 (0.528-0.528) | ||||||
|
|
2 | 0.847 (0.847-0.847) | 0.901 (0.901-0.901) | 0.593 (0.592-0.593) | 0.357 (0.357-0.357) | 0.96 (0.96-0.96) | 0.511 (0.511-0.512) | ||||||
|
|
3 | 0.846 (0.846-0.846) | 0.901 (0.901-0.901) | 0.573 (0.573-0.573) | 0.346 (0.346-0.346) | 0.958 (0.958-0.958) | 0.5 (0.5-0.5) | ||||||
|
|
4 | 0.839 (0.839-0.839) | 0.901 (0.901-0.901) | 0.562 (0.562-0.563) | 0.341 (0.341-0.341) | 0.958 (0.957-0.958) | 0.495 (0.494-0.495) | ||||||
| Deep neural network | |||||||||||||
|
|
1 | 0.82 (0.818-0.821) | 0.901 (0.901-0.902) | 0.499 (0.494-0.504) | 0.312 (0.309-0.314) | 0.953 (0.952-0.953) | 0.463 (0.461-0.466) | ||||||
|
|
2 | 0.799 (0.797-0.801) | 0.902 (0.901-0.902) | 0.505 (0.5-0.51) | 0.314 (0.312-0.317) | 0.953 (0.953-0.954) | 0.466 (0.464-0.469) | ||||||
|
|
3 | 0.809 (0.807-0.811) | 0.902 (0.901-0.902) | 0.525 (0.516-0.534) | 0.324 (0.32-0.327) | 0.954 (0.953-0.956) | 0.476 (0.472-0.48) | ||||||
|
|
4 | 0.807 (0.804-0.81) | 0.901 (0.901-0.902) | 0.508 (0.499-0.517) | 0.316 (0.313-0.32) | 0.953 (0.952-0.954) | 0.468 (0.464-0.472) | ||||||
aCanDETEC model.
bAutomated machine learning.