Table 3.
Classification results of different networks (N = 46).
| Network model | Sensitivity | Specificity | ROC-AUC | PR-AUC |
|---|---|---|---|---|
| Classification model of DCNN | ||||
| VGG16 | ||||
| Conventional | 0.42 (0.17–0.75) | 0.82 (0.68–0.94) | 0.64 (0.45–0.84) | 0.50 (0.25–0.78) |
| + CUReT | 0.67 (0.42–0.92) | 0.94 (0.91–0.97) | 0.81 (0.65–0.98) | 0.75 (0.52–0.97) |
| + CUReT + L2-constrained softmax loss | 0.58 (0.25–0.83) | 0.94 (0.85–1.00) | 0.70 (0.47–0.93) | 0.68 (0.45–0.92) |
| + CUReT + L2-constrained softmax loss + OCSVM | 0.58 (0.33–0.83) | 0.94 (0.85–1.00) | 0.77 (0.57–0.97) | 0.71 (0.50–0.94) |
| + CUReT + L2-constrained softmax loss + LOF | 0.75 (0.50–1.00) | 0.82 (0.68–0.94) | 0.86 (0.73–0.99) | 0.77 (0.57–0.98) |
| MobileNet | ||||
| Conventional | 0.50 (0.25–0.83) | 0.91 (0.82–1.00) | 0.60 (0.38–0.83) | 0.50 (0.24–0.77) |
| + CUReT | 0.67 (0.42–0.92) | 0.65 (0.47–0.79) | 0.70 (0.52–0.88) | 0.52 (0.30–0.76) |
| + CUReT + L2-constrained softmax loss | 0.41 (0.17–0.67) | 0.85 (0.73–0.97) | 0.57 (0.34–0.79) | 0.48 (0.24–0.72) |
| + CUReT + L2-constrained softmax loss + OCSVM | 0.66 (0.42–0.92) | 0.88 (0.76–0.97) | 0.79 (0.63–0.95) | 0.63 (0.40–0.89) |
| + CUReT + L2-constrained softmax loss + LOF | 0.66 (0.42–0.92) | 0.85 (0.74–0.97) | 0.80 (0.65–0.96) | 0.70 (0.49–0.93) |
| ResNet50 | ||||
| Conventional | 0.67 (0.53–0.82) | 0.68 (0.53–0.82) | 0.51 (0.29–0.73) | 0.38 (0.14–0.56) |
| + CUReT | 0.56 (0.38–0.71) | 0.83 (0.58–1.00) | 0.74 (0.58–0.90) | 0.47 (0.24–0.77) |
| + CUReT + L2-constrained softmax loss | 0.58 (0.33–0.83) | 0.68 (0.50–0.82) | 0.51 (0.29–0.73) | 0.31 (0.14–0.56) |
| + CUReT + L2-constrained softmax loss + OCSVM | 0.91 (0.75–1.00) | 0.50 (0.32–0.68) | 0.70 (0.54–0.86) | 0.42 (0.22–0.72) |
| + CUReT + L2-constrained softmax loss + LOF | 0.92 (0.75–1.00) | 0.59 (0.44–0.73) | 0.75 (0.59–0.90) | 0.47 (0.26–0.78) |
| CVAE | 1.00 (1.00–1.00) | 0.38 (0.24–0.56) | 0.68 (0.52–0.84) | 0.39 (0.20–0.67) |
| Radiologista | 0.83 (0.58–1.00) | 0.56 (0.38–0.71) | 0.74 (0.58–0.90) | 0.51 (0.29–0.76) |
ROC-AUC area under the curve of receiver operating characteristic curves, PR-AUC area under the curve of precision-recall curves, DCNN deep convolutional neural network, CUReT Columbia-Utrecht Reflectance and Texture Database, OCSVM one class support vector machine, LOF local outlier factor, CVAE convolutional variational autoencoder.
aThe radiologist was board-certified.