Table 3. Comparison of the N-SRS, the R-FSRS, and other machine learning methods performance on the testing set of the Framingham datasets.
A) Framingham Dataset 1 (FD1) | ||||||||
N-SRS | R-FSRS (both genders) | R-FSRS (men) | R-FSRS (women) | Log. Reg | CART | Random Forest | XGBoost | |
Sensitivity | 0.9142 | 0.8510 | 0.8461 | 0.8554 | 0.8933 | 0.8802 | 0.9175 | 0.9167 |
Specificity | 0.7238 | 0.6902 | 0.6890 | 0.7043 | 0.7102 | 0.7099 | 0.7161 | 0.7354 |
Precision | 0.9408 | 0.9620 | 0.9353 | 0.9758 | 0.9701 | 0.9736 | 0.9605 | 0.9412 |
NPV | 0.0592 | 0.0380 | 0.0647 | 0.0242 | 0.0423 | 0.0380 | 0.0863 | 0.0588 |
PPV | 0.9408 | 0.9620 | 0.9353 | 0.9758 | 0.9621 | 0.9736 | 0.9137 | 0.9412 |
AUC | 0.8743 | 0.7374 | 0.7188 | 0.7552 | 0.8065 | 0.7981 | 0.8829 | 0.8846 |
AUC 95% CI | 0.8569–0.9014 | 0.6976–0.7619 | 0.6765–0.7636 | 0.7081–0.8102 | 0.772–0.8351 | 0.7676–0.8287 | 0.8578–0.9081 | 0.8643–0.9048 |
calibration χ2 | 1.96 | 8.05 | 11.98 | 5.44 | 2.88 | 3.04 | 1.43 | 1.58 |
B) Framingham Dataset 2 (FD2) |
||||||||
N-SRS |
R-FSRS (both genders) |
R-FSRS (men) | R-FSRS (women) | Log. Reg | CART | Random Forest | XGBoost | |
Sensitivity | 0.8948 | 0.8533 | 0.8605 | 0.8487 | 0.8763 | 0.8504 | 0.8938 | 0.8934 |
Specificity | 0.5097 | 0.4217 | 0.4066 | 0.4800 | 0.4867 | 0.2505 | 0.4994 | 0.5110 |
Precision | 0.9693 | 0.9617 | 0.9531 | 0.9712 | 0.9688 | 0.9393 | 0.9816 | 0.9700 |
NPV | 0.3973 | 0.2233 | 0.2321 | 0.1933 | 0.2576 | 0.1704 | 0.3804 | 0.4053 |
PPV | 0.9693 | 0.9617 | 0.9531 | 0.9712 | 0.9401 | 0.9486 | 0.9535 | 0.9700 |
AUC | 0.8238 | 0.7488 | 0.7281 | 0.7677 | 0.7754 | 0.6884 | 0.8216 | 0.8260 |
AUC (95% CI) | 0.791–0.8558 | 0.7145–0.7831 | 0.6775–0.7788 | 0.7149–0.8204 | 0.738–0.8119 | 0.6435–0.7333 | 0.7881–0.8536 | 0.7938–0.8567 |
calibration χ2 | 2.75 | 7.3 | 12.1 | 4.1 | 6.5 | 20.34 | 2.81 | 2.7 |