Table 1.
Performance metrics of machine learning algorithms.
| Min | 1st.Qu | Median | Mean | 3rd.Qu | Max | NA.s | ||
|---|---|---|---|---|---|---|---|---|
| Accuracy | CART | 0.850 | 0.950 | 0.955 | 0.966 | 1.000 | 1.000 | 0 |
| Accuracy | CTREE | 0.818 | 0.950 | 0.952 | 0.958 | 1.000 | 1.000 | 0 |
| Accuracy | LDA | 0.650 | 0.818 | 0.857 | 0.860 | 0.905 | 1.000 | 0 |
| Accuracy | SVM | 0.800 | 0.900 | 0.905 | 0.912 | 0.951 | 1.000 | 0 |
| Accuracy | KNN | 0.571 | 0.724 | 0.773 | 0.776 | 0.818 | 0.950 | 0 |
| Accuracy | RF | 0.850 | 0.952 | 1.000 | 0.974 | 1.000 | 1.000 | 0 |
| Kappa | CART | 0.752 | 0.911 | 0.918 | 0.939 | 1.000 | 1.000 | 0 |
| Kappa | CTREE | 0.648 | 0.906 | 0.914 | 0.923 | 1.000 | 1.000 | 0 |
| Kappa | LDA | 0.426 | 0.667 | 0.732 | 0.740 | 0.820 | 1.000 | 0 |
| Kappa | SVM | 0.592 | 0.801 | 0.821 | 0.832 | 0.907 | 1.000 | 0 |
| Kappa | KNN | 0.050 | 0.405 | 0.521 | 0.521 | 0.634 | 0.906 | 0 |
| Kappa | RF | 0.752 | 0.912 | 1.000 | 0.953 | 1.000 | 1.000 | 0 |
| Mean_Balanced_Accuracy | CART | 0.849 | 0.967 | 0.978 | 0.975 | 1.000 | 1.000 | 0 |
| Mean_Balanced_Accuracy | CTREE | 0.755 | 0.951 | 0.977 | 0.964 | 1.000 | 1.000 | 0 |
| Mean_Balanced_Accuracy | LDA | 0.690 | 0.809 | 0.855 | 0.851 | 0.905 | 1.000 | 0 |
| Mean_Balanced_Accuracy | SVM | 0.712 | 0.810 | 0.868 | 0.865 | 0.906 | 1.000 | 0 |
| Mean_Balanced_Accuracy | KNN | 0.513 | 0.634 | 0.683 | 0.692 | 0.739 | 0.905 | 0 |
| Mean_Balanced_Accuracy | RF | 0.801 | 0.973 | 1.000 | 0.977 | 1.000 | 1.000 | 0 |
| Mean_F1 | CART | 0.753 | 0.907 | 0.964 | 0.949 | 1.000 | 1.000 | 0 |
| Mean_F1 | CTREE | 0.722 | 0.881 | 0.961 | 0.945 | 1.000 | 1.000 | 3 |
| Mean_F1 | LDA | 0.563 | 0.730 | 0.796 | 0.809 | 0.864 | 1.000 | 20 |
| Mean_F1 | SVM | 0.694 | 0.828 | 0.863 | 0.876 | 0.957 | 1.000 | 39 |
| Mean_F1 | KNN | 0.686 | 0.741 | 0.785 | 0.783 | 0.820 | 0.863 | 87 |
| Mean_F1 | RF | 0.786 | 0.919 | 1.000 | 0.956 | 1.000 | 1.000 | 1 |
| Mean_Precision | CART | 0.750 | 0.889 | 0.958 | 0.948 | 1.000 | 1.000 | 0 |
| Mean_Precision | CTREE | 0.556 | 0.889 | 0.958 | 0.939 | 1.000 | 1.000 | 1 |
| Mean_Precision | LDA | 0.498 | 0.733 | 0.838 | 0.806 | 0.917 | 1.000 | 12 |
| Mean_Precision | SVM | 0.542 | 0.917 | 0.952 | 0.912 | 0.956 | 1.000 | 35 |
| Mean_Precision | KNN | 0.795 | 0.871 | 0.922 | 0.903 | 0.938 | 0.956 | 87 |
| Mean_Precision | RF | 0.786 | 0.889 | 1.000 | 0.956 | 1.000 | 1.000 | 1 |
| Mean_Recall | CART | 0.760 | 0.951 | 0.974 | 0.964 | 1.000 | 1.000 | 0 |
| Mean_Recall | CTREE | 0.593 | 0.944 | 0.974 | 0.949 | 1.000 | 1.000 | 0 |
| Mean_Recall | LDA | 0.504 | 0.712 | 0.778 | 0.774 | 0.889 | 1.000 | 0 |
| Mean_Recall | SVM | 0.556 | 0.667 | 0.786 | 0.775 | 0.833 | 1.000 | 0 |
| Mean_Recall | KNN | 0.333 | 0.474 | 0.529 | 0.546 | 0.598 | 0.833 | 0 |
| Mean_Recall | RF | 0.667 | 0.967 | 1.000 | 0.966 | 1.000 | 1.000 | 0 |
| Mean_Sensitivity | CART | 0.760 | 0.951 | 0.974 | 0.964 | 1.000 | 1.000 | 0 |
| Mean_Sensitivity | CTREE | 0.593 | 0.944 | 0.974 | 0.949 | 1.000 | 1.000 | 0 |
| Mean_Sensitivity | LDA | 0.504 | 0.712 | 0.778 | 0.774 | 0.889 | 1.000 | 0 |
| Mean_Sensitivity | SVM | 0.556 | 0.667 | 0.786 | 0.775 | 0.833 | 1.000 | 0 |
| Mean_Sensitivity | KNN | 0.333 | 0.474 | 0.529 | 0.546 | 0.598 | 0.833 | 0 |
| Mean_Sensitivity | RF | 0.667 | 0.967 | 1.000 | 0.966 | 1.000 | 1.000 | 0 |
| Mean_Specificity | CART | 0.939 | 0.978 | 0.983 | 0.985 | 1.000 | 1.000 | 0 |
| Mean_Specificity | CTREE | 0.867 | 0.964 | 0.982 | 0.979 | 1.000 | 1.000 | 0 |
| Mean_Specificity | LDA | 0.821 | 0.909 | 0.933 | 0.928 | 0.956 | 1.000 | 0 |
| Mean_Specificity | SVM | 0.869 | 0.935 | 0.956 | 0.954 | 0.976 | 1.000 | 0 |
| Mean_Specificity | KNN | 0.693 | 0.792 | 0.841 | 0.839 | 0.879 | 0.976 | 0 |
| Mean_Specificity | RF | 0.935 | 0.981 | 1.000 | 0.989 | 1.000 | 1.000 | 0 |
Pre-processing: centered (55), scaled (55), Resampling: Cross-Validated (tenfold, repeated 10 times).
CART classification and regression trees (Complexity parameter = 0.176), CTREE conditional inference tree (mincriterion = 0.9), LDA linear discriminant analysis, SVM support vector machines (sigma = 0.01225348 and C = 2), KNN K-nearest neighbor (k = 13), RF Random Forest (mtry = 28), By R 4.2.2 with package 'caret', 207 samples 55 predictor 3 classes: ‘HS’, ‘S,’ ‘N’, N No significant relapse, S significant relapse, HS highly significant relapse.