Table 4.
Mean ± standard deviations of the evaluation metrics with different features in the training and validation sets. The results of deep features from each column are shown in bold. The p values of paired t-tests among different features in the validation set are listed in the lower half of the table. Calculations of the sensitivity and specificity of handcrafted and deep feature sets are provided in the Supplementary Information.
| Training set | Validation set | |||||||
|---|---|---|---|---|---|---|---|---|
| Type | AUC | Se | Sp | Acc | AUC | Se | Sp | Acc |
| FLAIR | 0.9429 ± 0.0037 | 0.7936 ± 0.0129 | 0.8738 ± 0.0044 | 0.8598 ± 0.0036 | 0.9271 ± 0.0047 | 0.7826 ± 0.0157 | 0.8421 ± 0.0062 | 0.8304 ± 0.0052 |
| T1C | 0.8980 ± 0.0053 | 0.6912 ± 0.0117 | 0.8455 ± 0.0053 | 0.8094 ± 0.0043 | 0.8771 ± 0.0065 | 0.7153 ± 0.0157 | 0.8032 ± 0.0072 | 0.7854 ± 0.006 |
| T1 | 0.9783 ± 0.0017 | 0.8687 ± 0.0103 | 0.9284 ± 0.0034 | 0.9179 ± 0.0029 | 0.9696 ± 0.0024 | 0.8529 ± 0.0108 | 0.9077 ± 0.005 | 0.8960 ± 0.0043 |
| T2 | 0.9182 ± 0.0038 | 0.8109 ± 0.0115 | 0.8290 ± 0.0044 | 0.8264 ± 0.0037 | 0.8994 ± 0.0049 | 0.8019 ± 0.0144 | 0.7905 ± 0.0061 | 0.7909 ± 0.0051 |
| Multimodality | 0.9722 ± 0.0029 | 0.8849 ± 0.0109 | 0.9190 ± 0.0035 | 0.9172 ± 0.0033 | 0.9624 ± 0.0038 | 0.8497 ± 0.0133 | 0.9083 ± 0.0052 | .8960 ± 0.0047 |
| AlexNet | 0.9995 ± 0.0002 | 0.9996 ± 0.0004 | 0.9870 ± 0.0015 | 0.9892 ± 0.0012 | 0.9993 ± 0.0003 | 0.9994 ± 0.0006 | 0.9801 ± 0.0022 | 0.9833 ± 0.0018 |
| Inception v3 | 0.9941 ± 0.0012 | 0.9913 ± 0.0034 | 0.9615 ± 0.0039 | 0.9669 ± 0.0033 | 0.9914 ± 0.0017 | 0.9884 ± 0.0042 | 0.9436 ± 0.0054 | 0.9509 ± 0.0047 |
| Fusion AlexNet | 0.9988 ± 0.0005 | 0.9957 ± 0.0021 | 0.9838 ± 0.002 | 0.9860 ± 0.0017 | 0.9982 ± 0.0007 | 0.9941 ± 0.0028 | 0.9755 ± 0.0029 | 0.9786 ± 0.0025 |
| Fusion Inception v3 | 0.9992 ± 0.0004 | 0.9933 ± 0.0025 | 0.9863 ± 0.0019 | 0.9874 ± 0.0017 | 0.9988 ± 0.0006 | 0.9907 ± 0.0034 | 0.9793 ± 0.0028 | 0.9809 ± 0.0025 |
| Single-modality handcrafted features compared to multimodality handcrafted features (p values) | ||||||||
| T1 | — | — | — | — | 5.35 × 10−39 | 1.71 × 10−27 | 1.56 × 10−28 | 6.93 × 10−59 |
| T2 | — | — | — | — | 3.57 × 10−16 | 0.1832 | 6.0 × 10−26 | 1.22 × 10−22 |
| T1C | — | — | — | — | 3.10 × 10−23 | 0.02 | 9.71 × 10−17 | 1.68 × 10−22 |
| FLAIR | — | — | — | — | 0.03 | 0.4863 | 0.02 | 0.0099 |
| Deep features compared to multimodality handcrafted features (p values) | ||||||||
| AlexNet | — | — | — | — | 3.88 × 10−138 | 6.40 × 10−117 | 8.37 × 10−208 | 7.99 × 10−301 |
| Inception v3 | — | — | — | — | 3.60 × 10−99 | 1.72 × 10−103 | 3.97 × 10−98 | 1.32 × 10−162 |
| Fusion AlexNet | — | — | — | — | 1.56 × 10−134 | 5.76 × 10−109 | 1.47 × 10−193 | 5.44 × 10−277 |
| Fusion Inception v3 | — | — | — | — | 2.81 × 10−135 | 1.02 × 10−102 | 2.94 × 10−190 | 1.49 × 10−269 |
| AlexNet features compared to Inception v3 and fusion AlexNet features, respectively (p values) | ||||||||
| Inception v3 | — | — | — | — | 1.35 × 10−18 | 8.17 × 10−8 | 8.21 × 10−31 | 2.41 × 10−33 |
| Fusion AlexNet | — | — | — | — | 0.01 | 1.11 × 10−4 | 0.08 | 0.02 |
| Fusion Inception v3 features compared to Inception v3 and fusion AlexNet features (p values) | ||||||||
| Inception v3 | — | — | — | — | 2.18 × 10−14 | 0.88 | 1.81 × 10−25 | 2.09 × 10−23 |
| Fusion AlexNet | — | — | — | — | 0.8913 | 0.03 | 0.42 | 0.9663 |
Se: sensitivity; Sp: specificity; Acc: accuracy. “—” in the table indicates the item was not calculated to correspond to paired t-test value in the training set.