Table 1.
Comparison of different classifier designs on five-fold cross-validation.
| Performance Measures | Predictors Trained and Test on Original Dataset | Feature Selection on Original Dataset and Training/Testing on Balanced Original and Reversed Dataset | |||||||
|---|---|---|---|---|---|---|---|---|---|
| 3-Class RF with All Features a | 3-Class RF with 8 Selected Features | 2-Layer Predictor with All Features | 2-Layer Predictor with 10 Selected Features (PON-tstab) | 3-Class RF with All Features | 3-Class RF with 5 Selected Features | 2-Layer Predictor on 3 Classifiers with All Features | 2-Layer Predictor on 3 Classifiers with Selected Features | ||
| TP b | + | 21.6/43.5 | 9.6/19.0 | 19/38.3 | 23.2/46.3 | 40.8 | 44.8 | 42.8 | 43.4 |
| − | 74.8/40.2 | 124.4/67.2 | 90.6/48.1 | 91.4/49.0 | 43.4 | 42 | 41.6 | 37.4 | |
| no | 34 | 26.4 | 28.8 | 30.8 | 36.2 | 37.2 | 37.8 | 40 | |
| TN | + | 180.6/123.6 | 32.2/64.4 | 183.4/125.7 | 186.2/127.9 | 125.4 | 122.2 | 125.4 | 123.8 |
| − | 95/130.4 | 30.2/16.2 | 85.8/118.1 | 88.2/122.7 | 125.4 | 129.2 | 126.4 | 131.4 | |
| no | 134.6/113.9 | 57 | 149/121.6 | 150.8/125.6 | 121.4 | 123.4 | 121.2 | 116.4 | |
| FP | + | 57.4/43.2 | 10.6/8.0 | 54.6/41.1 | 51.8/38.9 | 41.8 | 45 | 41.8 | 43.4 |
| − | 30.2/36.4 | 71.8/91.6 | 39.4/48.7 | 37/44.1 | 42.8 | 38 | 40.8 | 35.8 | |
| no | 61.8/52.9 | 37/38.0 | 47.4/45.2 | 45.6/41.2 | 45.8 | 43.8 | 46 | 50.8 | |
| FN | + | 20.2/39.9 | 227.4/158.7 | 22.8/45.1 | 18.6/37.1 | 42.8 | 38.8 | 40.8 | 40.2 |
| − | 79.8/43.2 | 53.4/75.2 | 64/35.3 | 63.2/34.4 | 40.2 | 41.6 | 42 | 46.2 | |
| no | 49.4 | 159.4/128.8 | 54.6 | 52.6 | 47.4 | 46.4 | 45.8 | 43.6 | |
| Sensitivity | + | 0.516 | 0.228 | 0.455 | 0.554 | 0.488 | 0.537 | 0.511 | 0.518 |
| − | 0.483 | 0.805 | 0.583 | 0.59 | 0.52 | 0.504 | 0.498 | 0.449 | |
| no | 0.406 | 0.318 | 0.339 | 0.367 | 0.432 | 0.445 | 0.451 | 0.478 | |
| Specificity | + | 0.759/0.744 | 0.955/0.953 | 0.771/0.756 | 0.783/0.769 | 0.75 | 0.732 | 0.75 | 0.741 |
| − | 0.755/0.776 | 0.427/0.451 | 0.68/0.700 | 0.701/0.730 | 0.743 | 0.772 | 0.755 | 0.785 | |
| no | 0.686/0.683 | 0.812/0.772 | 0.757/0.732 | 0.767/0.756 | 0.727 | 0.739 | 0.725 | 0.697 | |
| PPV | + | 0.271/0.498 | 0.463/0.691 | 0.258/0.481 | 0.318/0.551 | 0.492 | 0.505 | 0.505 | 0.502 |
| − | 0.714/0.527 | 0.635/0.424 | 0.701/0.503 | 0.715/0.530 | 0.505 | 0.528 | 0.548 | 0.513 | |
| no | 0.354/0.388 | 0.421/0.413 | 0.371/0.386 | 0.399/0.428 | 0.445 | 0.462 | 0.452 | 0.444 | |
| NPV | + | 0.9/0.757 | 0.876/0.712 | 0.89/0.763 | 0.909/0.774 | 0.746 | 0.76 | 0.755 | 0.756 |
| − | 0.543/0.75 | 0.643/0.824 | 0.57/0.771 | 0.58/0.781 | 0.756 | 0.758 | 0.75 | 0.741 | |
| no | 0.732/0.698 | 0.737/0.694 | 0.731/0.690 | 0.741/0.705 | 0.719 | 0.727 | 0.726 | 0.728 | |
| GC2 | 0.172/0.078 | 0.101/0.281 | 0.121/0.085 | 0.162/0.112 | 0.063 | 0.08 | 0.068 | 0.071 | |
| CPR | 0.466/0.469 | 0.573/0.450 | 0.495/0.459 | 0.520/0.503 | 0.48 | 0.495 | 0.487 | 0.481 | |
a Normalized performance values are separated by a slash if they are different from the original ones. b GC2, generalized squared correction; CPR, correct prediction ratio; FN, false negative; FP, false positive; NPV, negative predictive value; PPV, positive predictive value; RF, random forest; TN, true negative; TP, true positive.