Table 7.
Statistical significance between two methods for Datasets 3–38
| a) MAS | Inferior | ||||||||
| WAD | AD | FC | RP | modT | samT | shrinkT | ibmT | ||
| Superior | WAD | - | 2.1E-07 | 6.7E-07 | 2.3E-06 | 2.2E-02 | 1.7E-02 | 2.0E-02 | 1.8E-01 |
| AD | 1.0E+00 | - | 8.1E-01 | 2.9E-04 | 1.0E+00 | 1.0E+00 | 1.0E+00 | 1.0E+00 | |
| FC | 1.0E+00 | 1.9E-01 | - | 2.6E-04 | 1.0E+00 | 1.0E+00 | 1.0E+00 | 1.0E+00 | |
| RP | 1.0E+00 | 1.0E+00 | 1.0E+00 | - | 1.0E+00 | 1.0E+00 | 1.0E+00 | 1.0E+00 | |
| modT | 9.8E-01 | 8.6E-04 | 4.0E-03 | 1.4E-04 | - | 4.7E-01 | 9.0E-01 | 1.0E+00 | |
| samT | 9.8E-01 | 2.5E-04 | 2.0E-03 | 6.6E-05 | 5.3E-01 | - | 6.9E-01 | 1.0E+00 | |
| shrinkT | 9.8E-01 | 4.0E-04 | 2.2E-03 | 9.0E-05 | 1.0E-01 | 3.1E-01 | - | 1.0E+00 | |
| ibmT | 8.2E-01 | 4.7E-05 | 2.6E-04 | 2.6E-05 | 2.2E-04 | 2.3E-03 | 2.9E-04 | - | |
| (b) RMA | Inferior | ||||||||
| WAD | AD | FC | RP | modT | samT | shrinkT | ibmT | ||
| Superior | WAD | - | 9.8E-01 | 9.8E-01 | 8.9E-01 | 3.0E-01 | 3.1E-01 | 2.5E-01 | 4.4E-01 |
| AD | 2.3E-02 | - | 4.7E-01 | 8.3E-02 | 9.2E-03 | 1.1E-02 | 7.2E-03 | 2.8E-02 | |
| FC | 2.4E-02 | 5.3E-01 | - | 8.8E-02 | 1.1E-02 | 1.3E-02 | 8.4E-03 | 3.1E-02 | |
| RP | 1.1E-01 | 9.2E-01 | 9.1E-01 | - | 8.4E-02 | 9.7E-02 | 6.6E-02 | 1.7E-01 | |
| modT | 7.0E-01 | 9.9E-01 | 9.9E-01 | 9.2E-01 | - | 5.6E-01 | 6.5E-02 | 1.0E+00 | |
| samT | 6.9E-01 | 9.9E-01 | 9.9E-01 | 9.0E-01 | 4.4E-01 | - | 2.1E-01 | 8.3E-01 | |
| shrinkT | 7.5E-01 | 9.9E-01 | 9.9E-01 | 9.3E-01 | 9.4E-01 | 7.9E-01 | - | 1.0E+00 | |
| ibmT | 5.6E-01 | 9.7E-01 | 9.7E-01 | 8.3E-01 | 3.2E-03 | 1.7E-01 | 1.9E-03 | - | |
| (c) DFW | Inferior | ||||||||
| WAD | AD | FC | RP | modT | samT | shrinkT | ibmT | ||
| Superior | WAD | - | 1.0E+00 | 1.0E+00 | 1.0E+00 | 1.3E-01 | 1.2E-01 | 4.5E-01 | 1.6E-01 |
| AD | 2.5E-03 | - | 1.6E-01 | 9.6E-01 | 5.1E-02 | 4.7E-02 | 2.1E-01 | 6.9E-02 | |
| FC | 2.6E-03 | 8.4E-01 | - | 9.6E-01 | 5.1E-02 | 4.7E-02 | 2.1E-01 | 6.9E-02 | |
| RP | 8.7E-04 | 4.2E-02 | 4.1E-02 | - | 3.0E-02 | 3.0E-02 | 1.1E-01 | 4.4E-02 | |
| modT | 8.7E-01 | 9.5E-01 | 9.5E-01 | 9.7E-01 | - | 8.6E-02 | 9.9E-01 | 8.5E-01 | |
| samT | 8.8E-01 | 9.5E-01 | 9.5E-01 | 9.7E-01 | 9.1E-01 | - | 9.9E-01 | 1.0E+00 | |
| shrinkT | 5.5E-01 | 7.9E-01 | 7.9E-01 | 8.9E-01 | 6.1E-03 | 1.0E-02 | - | 2.6E-02 | |
| ibmT | 8.4E-01 | 9.3E-01 | 9.3E-01 | 9.6E-01 | 1.5E-01 | 5.2E-04 | 9.7E-01 | - | |
The p-values between the 36 AUC values from a possibly superior method and those from a possibly inferior method were calculated by a one-tail paired t-test. The null hypothesis is that the mean of the 36 AUC values for one method is the same as that for the other method. There are two p-values for two methods compared. For example, in (a) MAS-preprocessed data, the p-value is 1.8E-01 when the alternative hypothesis is that the mean of the 36 AUC values for WAD is greater than that for ibmT while the p-value is 8.2E-01 when the alternative hypothesis is that the mean of the 36 AUC values for ibmT is greater than that for WAD. Combinations having p < 0.05 are highlighted in bold.