Table 3. Performance metrics achieved by the coarse models using the CheXpert test set.
The coarse models are initialized with ImageNet pretrained weights (conventional transfer learning) and trained end-to-end to learn CXR modality-specific weights using the CheXpert data set to classify the CXRs into normal and abnormal classes. The custom CNN is initialized with random weights. Data in parenthesis are 95% CI for the AUC values that were calculated to be the Wilson score interval which corresponds to separate 2-sided confidence intervals with individual coverage probabilities of sqrt(0.95).
Model | Accuracy | AUC | Sensitivity | Specificity | F measure | MCC |
---|---|---|---|---|---|---|
Custom CNN | 0.8018 | 0.8898 (0.8813, 0.8983) | 0.9030 | 0.5980 | 0.7952 | 0.5356 |
VGG-16 | 0.8904 | 0.9649(0.9599, 0.9699) | 0.9173 | 0.8448 | 0.8904 | 0.7530 |
VGG-19 | 0.8799 | 0.9432 (0.9369, 0.9495) | 0.9115 | 0.8165 | 0.8798 | 0.7288 |
Inception-V3 | 0.8835 | 0.9571 (0.9516, 0.9626) | 0.9028 | 0.8363 | 0.8840 | 0.7402 |
Xception | 0.8720 | 0.9401 (0.9337, 0.9465) | 0.9005 | 0.8148 | 0.8723 | 0.7126 |
DenseNet-121 | 0.8839 | 0.9493 (0.9434, 0.9552) | 0.9140 | 0.8233 | 0.8838 | 0.7378 |
MobileNet | 0.8797 | 0.9456 (0.9395, 0.9517) | 0.9073 | 0.8244 | 0.8799 | 0.7295 |
NASNet-mobile | 0.8824 | 0.9552 (0.9496, 0.9608) | 0.9045 | 0.8380 | 0.8828 | 0.7369 |
Notes.
Bold values indicate superior performance.