Table 3.
Results of algorithms of KNN, LR and SVM
| Algorithm | Model | Cohort | AUC (95% CI) | Accuracy | Sensitivity | Specificity |
|---|---|---|---|---|---|---|
| KNN | M-S | Train | 0.786 (0.710–0.844) | 0.701 | 0.614 | 0.783 |
| Validation | 0.712 (0.529–0.833) | 0.613 | 0.533 | 0.688 | ||
| M-C | Train | 0.805 (0.741–0.863) | 0.718 | 0.544 | 0.883 | |
| Validation | 0.771 (0.621–0.886) | 0.645 | 0.533 | 0.750 | ||
| M-S-C | Train | 0.866 (0.809–0.915) | 0.786 | 0.667 | 0.900 | |
| Validation | 0.860 (0.724–0.952) | 0.839 | 0.733 | 0.938 | ||
| L-S | Train | 0.832 (0.773–0.888) | 0.744 | 0.667 | 0.817 | |
| Validation | 0.706 (0.530–0.83) | 0.677 | 0.533 | 0.875 | ||
| L-C | Train | 0.835 (0.770–0.886) | 0.735 | 0.649 | 0.817 | |
| Validation | 0.752 (0.592–0.895) | 0.710 | 0.667 | 0.750 | ||
| L-S-C | Train | 0.867 (0.890–0.912) | 0.778 | 0.632 | 0.917 | |
| Validation | 0.796 (0.625–0.931) | 0.839 | 0.733 | 0.938 | ||
| P-S | Train | 0.774 (0.707–0.844) | 0.667 | 0.561 | 0.767 | |
| Validation | 0.694 (0.559–0.867) | 0.677 | 0.553 | 0.813 | ||
| P–T | Train | 0.834 (0.762–0.889) | 0.769 | 0.702 | 0.833 | |
| Validation | 0.721 (0.598–0.891) | 0.677 | 0.600 | 0.750 | ||
| P-S-T | Train | 0.846 (0.786–0.86) | 0.769 | 0.684 | 0.850 | |
| Validation | 0.950 (0.890–0.993) | 0.903 | 0.933 | 0.875 | ||
| Final-M | Train | 0.927 (0.878–0.960) | 0.880 | 0.789 | 0.967 | |
| Validation | 0.938 (0.862–0.988) | 0.839 | 0.800 | 0.875 | ||
| Clnc-M | Train | 0.695 (0.622–0.762) | 0.684 | 0.632 | 0.733 | |
| Validation | 0.692 (0.531–0.827) | 0.642 | 0.600 | 0.688 | ||
| LR | M-S | Train | 0.813 (0.736–0.872) | 0.718 | 0.737 | 0.700 |
| Validation | 0.883 (0.745–0.996) | 0.742 | 0.733 | 0.750 | ||
| M-C | Train | 0.774 (0.696–0.840) | 0.726 | 0.684 | 0.767 | |
| Validation | 0.733 (0.567–0.885) | 0.710 | 0.800 | 0.625 | ||
| M-S-C | Train | 0.830 (0.759–0.885) | 0.744 | 0.719 | 0.767 | |
| Validation | 0.875 (0.754–0.962) | 0.774 | 0.733 | 0.813 | ||
| L-S | Train | 0.876 (0.819–0.924) | 0.795 | 0.789 | 0.800 | |
| Validation | 0.913 (0.804–0.983) | 0.806 | 0.733 | 0.875 | ||
| L-C | Train | 0.839 (0.771–0.895) | 0.761 | 0.702 | 0.817 | |
| Validation | 0.842 (0.704–0.950) | 0.742 | 0.800 | 0.688 | ||
| L-S-C | Train | 0.917 (0.873–0.952) | 0.821 | 0.789 | 0.850 | |
| Validation | 0.938 (0.857–0.991) | 0.871 | 0.800 | 0.938 | ||
| P-S | Train | 0.884 (0.829–0.931) | 0.786 | 0.754 | 0.817 | |
| Validation | 0.883 (0.858–0.992) | 0.806 | 0.867 | 0.750 | ||
| P–T | Train | 0.885 (0.832–0.933) | 0.821 | 0.754 | 0.883 | |
| Validation | 0.908 (0.836–0.982) | 0.742 | 0.667 | 0.813 | ||
| P-S-T | Train | 0.977 (0.957–0.993) | 0.932 | 0.947 | 0.917 | |
| Validation | 0.921 (0.906–1.000) | 0.806 | 0.867 | 0.750 | ||
| Final-M | Train | 0.984 (0.969–0.995) | 0.940 | 0.877 | 1.000 | |
| Validation | 0.983 (0.957–1.000) | 0.968 | 1.000 | 0.938 | ||
| Clnc-M | Train | 0.684 (0.599–0.751) | 0.684 | 0.544 | 0.817 | |
| Validation | 0.644 (0.451–9.782) | 0.645 | 0.533 | 0.451 | ||
| SVM | M-S | Train | 0.829 (0.752–0.885) | 0.752 | 0.737 | 0.767 |
| Validation | 0.708 (0.521–0.850) | 0.645 | 0.600 | 0.689 | ||
| M-C | Train | 0.883 (0.826–0.929) | 0.821 | 0.772 | 0.867 | |
| Validation | 0.792 (0.647–0.919) | 0.742 | 0.800 | 0.689 | ||
| M-S-C | Train | 0.885 (0.822–0.931) | 0.769 | 0.737 | 0.800 | |
| Validation | 0.817 (0.649–0.929) | 0.710 | 0.733 | 0.688 | ||
| L-S | Train | 0.920 (0.881–0.953) | 0.821 | 0.789 | 0.850 | |
| Validation | 0.838 (0.693–0.940) | 0.774 | 0.667 | 0.875 | ||
| L-C | Train | 0.888 (0.833–0.930) | 0.786 | 0.719 | 0.850 | |
| Validation | 0.829 (0.675–0.947) | 0.806 | 0.800 | 0.813 | ||
| L-S-C | Train | 0.941 (0.905–0.970) | 0.821 | 0.754 | 0.883 | |
| Validation | 0.896 (0.765–1.000) | 0.839 | 0.800 | 0.875 | ||
| P-S | Train | 0.923 (0.883–0.959) | 0.821 | 0.772 | 0.867 | |
| Validation | 0.867 (0.832–0.986) | 0.806 | 0.733 | 0.875 | ||
| P–T | Train | 0.915 (0.864–0.953) | 0.838 | 0.789 | 0.883 | |
| Validation | 0.858 (0.744–0.975) | 0.806 | 0.733 | 0.875 | ||
| P-S-T | Train | 0.956 (0.927–0.978) | 0.846 | 0.789 | 0.900 | |
| Validation | 0.879 (0.827–1.000) | 0.806 | 0.733 | 0.875 | ||
| Final-M | Train | 0.984 (0.968–0.996) | 0.940 | 0.877 | 1.000 | |
| Validation | 0.958 (0.895–1.000) | 0.935 | 0.933 | 0.938 | ||
| Clnc-M | Train | 0.747 (0.674–0.815) | 0.667 | 0.667 | 0.667 | |
| Validation | 0.715 (0.548–0.860) | 0.710 | 0.733 | 0.688 |