Table 6.
Mean ± standard deviation of the performance (accuracy, specificity, sensitivity, F1-score) provided by five kind of models (second column) on 50 test sets. The goal is to determine resistance of Pseudomonas to six families of antimicrobials (first column). Two FS approaches have been considered: clinical knowledge (FS1) and mutual information (FS2). Bold figures refer to the highest performance per figure of merit and antimicrobial family.
| Family | Model | Accuracy | Specitivity | Sensitivity | F1-score | ||||
|---|---|---|---|---|---|---|---|---|---|
| FS1 | FS2 | FS1 | FS2 | FS1 | FS2 | FS1 | FS2 | ||
| AMG | LR | 78.2 ± 1.2 | 75.3 ± 1.7 | 80.0 ± 1.9 | 77.2 ± 2.7 | 76.5 ± 1.8 | 73.5 ± 2.4 | 77.8 ± 1.2 | 74.8 ± 1.7 |
| k-nn | 79.3 ± 1.6 | 83.3 ± 1.9 | 84.0 ± 2.5 | 86.5 ± 2.2 | 74.5 ± 2.1 | 80.3 ± 2.0 | 78.1 ± 1.3 | 82.7 ± 2.0 | |
| DT | 77.0 ± 1.2 | 78.6 ± 2.3 | 78.0 ± 3.7 | 81.7 ± 2.5 | 75.9 ± 2.6 | 76.6 ± 2.7 | 76.5 ± 1.1 | 78.0 ± 2.6 | |
| RF | 80.1 ± 1.6 | 80.8 ± 1.2 | 80.0 ± 2.3 | 81.3 ± 2.4 | 80.0 ± 2.0 | 80.2 ± 2.0 | 80.0 ± 1.6 | 80.5 ± 1.5 | |
| MLP | 80.8 ± 1.3 | 78.3 ± 1.0 | 83.0 ± 2.1 | 82.0 ± 1.7 | 78.6 ± 1.7 | 75.0 ± 1.2 | 80.1 ± 1.3 | 77.9 ± 1.1 | |
| CAR | LR | 77.3 ± 1.4 | 74.8 ± 2.3 | 76.0 ± 3.0 | 72.0 ± 3.4 | 78.7 ± 2.8 | 77.8 ± 3.1 | 77.6 ± 1.8 | 75.5 ± 2.3 |
| k-nn | 79.9 ± 1.6 | 81.5 ± 1.4 | 80.0 ± 2.8 | 80.3 ± 2.5 | 80.1 ± 2.5 | 82.7 ± 2.2 | 79.8 ± 1.7 | 81.6 ± 1.7 | |
| DT | 78.1 ± 2.2 | 79.4 ± 2.0 | 80.0 ± 2.6 | 83.7 ± 3.4 | 75.8 ± 2.7 | 76.2 ± 3.2 | 77.3 ± 2.7 | 78.9 ± 2.0 | |
| RF | 82.4 ± 1.7 | 82.2 ± 1.7 | 78.0 ± 4.0 | 82.0 ± 3.1 | 86.8 ± 3.1 | 82.5 ± 2.6 | 83.2 ± 1.5 | 82.5 ± 1.6 | |
| MLP | 81.9 ± 1.5 | 79.0 ± 1.9 | 81.0 ± 3.3 | 78.6 ± 2.5 | 82.6 ± 2.9 | 80.2 ± 2.1 | 82.3 ± 1.5 | 79.2 ± 1.8 | |
| CF4 | LR | 68.7 ± 2.0 | 67.8 ± 1.3 | 70.0 ± 3.1 | 68.1 ± 2.3 | 67.9 ± 2.6 | 67.1 ± 2.1 | 68.2 ± 2.3 | 67.3 ± 1.8 |
| k-nn | 77.7 ± 1.5 | 75.6 ± 1.6 | 80.0 ± 2.7 | 78.9 ± 2.6 | 75.2 ± 2.4 | 72.9 ± 2.6 | 77.0 ± 1.4 | 74.7 ± 2.0 | |
| DT | 71.0 ± 1.1 | 74.6 ± 2.2 | 74.4 ± 3.7 | 78.0 ± 3.6 | 67.9 ± 3.2 | 71.8 ± 3.1 | 70.3 ± 1.4 | 73.9 ± 2.4 | |
| RF | 75.8 ± 1.9 | 78.0 ± 1.8 | 72.0 ± 4.3 | 77.2 ± 3.7 | 79.4 ± 3.0 | 79.6 ± 3.3 | 76.7 ± 1.7 | 78.2 ± 1.8 | |
| MLP | 77.0 ± 1.5 | 75.8 ± 1.4 | 81.0 ± 4.5 | 77.4 ± 3.6 | 73.4 ± 3.7 | 75.1 ± 2.7 | 76.6 ± 1.1 | 75.3 ± 1.9 | |
| PAP | LR | 69.0 ± 1.4 | 67.7 ± 1.4 | 68.0 ± 2.8 | 65.9 ± 3.0 | 70.4 ± 2.5 | 70.2 ± 2.5 | 69.1 ± 1.6 | 68.1 ± 1.4 |
| k-nn | 78.3 ± 1.6 | 78.4 ± 1.4 | 82.0 ± 2.5 | 81.3 ± 2.2 | 74.8 ± 2.4 | 76.3 ± 2.7 | 77.4 ± 1.9 | 77.7 ± 1.7 | |
| DT | 72.6 ± 2.2 | 74.6 ± 2.0 | 74.0 ± 3.5 | 78.1 ± 4.1 | 70.9 ± 2.6 | 71.1 ± 3.5 | 71.7 ± 2.5 | 73.4 ± 2.3 | |
| RF | 75.2 ± 1.5 | 75.6 ± 1.2 | 71.0 ± 3.0 | 73.0 ± 3.0 | 79.5 ± 2.7 | 78.3 ± 2.4 | 75.7 ± 1.8 | 76.1 ± 1.4 | |
| MLP | 78.2 ± 1.7 | 75.4 ± 1.4 | 78.0 ± 1.8 | 76.3 ± 2.3 | 78.0 ± 3.3 | 74.4 ± 1.9 | 78.1 ± 2.4 | 75.1 ± 1.8 | |
| POL | LR | 68.1 ± 6.3 | 70.95 ± 6.8 | 63.0 ± 7.7 | 65.1 ± 10.0 | 73.4 ± 6.3 | 76.3 ± 8.4 | 69.6 ± 6.6 | 71.4 ± 6.9 |
| k-nn | 63.9 ± 7.2 | 70.5 ± 7.0 | 68.0 ± 12.5 | 74.0 ± 6.7 | 61.4 ± 11.8 | 67.4 ± 7.4 | 63.0 ± 9.3 | 69.2 ± 9.0 | |
| DT | 60.6 ± 5.2 | 72.1 ± 6.9 | 65.0 ± 12.2 | 69.3 ± 14.6 | 57.0 ± 11.6 | 75.5 ± 7.9 | 58.3 ± 7.8 | 71.5 ± 7.0 | |
| RF | 65.3 ± 7.9 | 68.9 ± 6.9 | 58.0 ± 15.0 | 66.4 ± 11.2 | 74.2 ± 11.2 | 73.4 ± 6.6 | 68.4 ± 6.9 | 70.0 ± 6.2 | |
| MLP | 67.7 ± 3.5 | 70.8 ± 6.2 | 75.0 ± 7.6 | 71.0 ± 12.4 | 60.5 ± 9.2 | 71.5 ± 9.5 | 64.4 ± 2.3 | 71.3 ± 5.8 | |
| QUI | LR | 72.1 ± 2.1 | 71.8 ± 1.5 | 70.0 ± 2.8 | 71.6 ± 2.4 | 74.2 ± 2.5 | 72.2 ± 2.6 | 72.7 ± 2.1 | 71.9 ± 1.9 |
| k-nn | 86.8 ± 1.1 | 90.1 ± 1.3 | 86.0 ± 1.6 | 90.2 ± 1.9 | 87.6 ± 1.7 | 90.5 ± 1.7 | 86.8 ± 1.3 | 90.0 ± 1.4 | |
| DT | 81.8 ± 1.7 | 82.3 ± 1.5 | 83.7 ± 2.8 | 85.0 ± 2.8 | 80.1 ± 2.2 | 79.6 ± 2.2 | 81.4 ± 2.1 | 81.7 ± 1.7 | |
| RF | 82.5 ± 1.8 | 83.6 ± 1.4 | 79.0 ± 2.9 | 84.6 ± 2.4 | 86.0 ± 2.9 | 83.6 ± 2.0 | 82.9 ± 1.7 | 83.7 ± 1.3 | |
| MLP | 87.1 ± 1.4 | 83.4 ± 2.0 | 87.0 ± 0.8 | 82.7 ± 1.8 | 87.0 ± 1.0 | 84.8 ± 1.7 | 86.7 ± 1.7 | 83.7 ± 2.0 | |