Table 4.
Performance indices for machine learning models in predicting recurrence within 10 years after breast cancer surgery (unit: %).
| Models | Sensitivity | Specificity | PPV | NPV | Accuracy | AUROC |
|---|---|---|---|---|---|---|
| Training dataset (n = 798) | ||||||
| ANN | 95.89 (94.68–97.09) | 99.54 (99.43–99.66) | 97.90 (96.04–99.75) | 99.08 (98.90–99.26) | 98.87 (98.12–99.63) | 97.62 (96.87–98.37) |
| KNN | 75.00 (71.67–78.34) | 94.97 (92.58–97.36) | 78.94 (75.12–82.76) | 93.78 (92.67–94.89) | 90.95 (89.47–92.44) | 85.00 (83.78–86.23) |
| SVM | 81.29 (80.04–82.57) | 96.61 (95.49–97.73) | 85.80 (83.78–87.82) | 95.35 (94.22–96.48) | 95.53 (94.87–96.19) | 88.90 (86.76–91.04) |
| NBC | 100.00 (99.91–100.00) | 0.00 (0.00–0.00) | 79.88 (76.47–83.29) | 0.00 (0.00–0.00) | 79.88 (76.23–83.53) | 50.00 (42.65–57.36) |
| COX | 34.93 (24.91–44.99) | 0.00 (0.00–0.00) | 7.30 (4.37–10.24) | 0.00 (0.00–0.00) | 6.42 (5.78–7.06) | 17.50 (12.14–22.86) |
| p value | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 |
| Testing dataset (n = 171) | ||||||
| ANN | 95.35 (94.47–96.23) | 100.00 (99.94–100.00) | 100.00 (99.97–100.00) | 98.45 (97.87–99.03) | 98.82 (97.67–99.96) | 99.81 (99.43–99.99) |
| KNN | 46.15 (36.21–56.09) | 76.66 (73.69–79.63) | 46.15 (35.98–56.32) | 76.67 (72.41–80.93) | 67.44 (55.67–79.21) | 61.40 (47.93–74.87) |
| SVM | 70.37 (65.45–75.29) | 94.35 (93.35–95.37) | 75.50 (71.41–79.59) | 93.13 (91.47–94.79) | 89.79 (87.77–91.80) | 82.40 (80.34–84.46) |
| NBC | 100.00 (99.91–100.00) | 0.00 (0.00–0.00) | 19.01 (10.98–27.04) | 0.00 (0.00–0.00) | 19.01 (10.23–27.79) | 50.00 (40.67–59.34) |
| COX | 20.93 (8.79–33.07) | 0.00 (0.00–0.00) | 6.62 (3.78–9.46) | 0.00 (0.00–0.00) | 5.29 (3.02–7.47) | 10.50 (4.71–16.29) |
| p value | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 |
ANN: artificial neural network; KNN: k-nearest neighbor; SVM: support vector machine; NBC: naïve Bayesian classifier; PPV: positive predictive value; NPV: negative predictive value; AUROC: area under the receiver operating characteristic curve.* 1000 pairs of forecasting models with bootstrapping methods were compared in terms of accuracy in predicting recurrence within 10 years after breast cancer surgery.