Table 5.
Effect of training set on disease detection with Dense-121 tested on MIMIC-CXR.
| Training set | No finding |
||||
|---|---|---|---|---|---|
| White | Asian | Black | Female | Male | |
| AUC (95% CI) | |||||
| MIMIC-CXR | 0.85 (0.84–0.85) | 0.86 (0.84–0.88) | 0.85 (0.84–0.86) | 0.86 (0.85–0.86) | 0.84 (0.83–0.84) |
| CheXpert | 0.82 (0.82–0.83) | 0.83 (0.81–0.85) | 0.83 (0.82–0.84) | 0.84 (0.83–0.84) | 0.81 (0.81–0.82) |
| TPR (95% CI) | |||||
| MIMIC-CXR | 0.75 (0.74–0.75) | 0.74 (0.71–0.77) | 0.80 (0.79–0.82) | 0.78 (0.77–0.79) | 0.74 (0.72–0.74) |
| CheXpert | 0.69 (0.68–0.70) | 0.73 (0.69–0.76) | 0.75 (0.74–0.77) | 0.74 (0.73–0.75) | 0.68 (0.67–0.69) |
| FPR (95% CI) | |||||
| MIMIC-CXR | 0.19 (0.19–0.19) | 0.17 (0.15–0.19) | 0.25 (0.24–0.26) | 0.21 (0.21–0.21) | 0.19 (0.19–0.20) |
| CheXpert | 0.19 (0.19–0.19) | 0.19 (0.17–0.21) | 0.25 (0.24–0.26) | 0.21 (0.21–0.21) | 0.19 (0.19–0.20) |
| Youden's J statistic (95% CI) | |||||
| MIMIC-CXR | 0.55 (0.54–0.56) | 0.58 (0.54–0.61) | 0.55 (0.54–0.57) | 0.57 (0.56–0.58) | 0.54 (0.53–0.55) |
| CheXpert | 0.50 (0.49–0.51) | 0.53 (0.49–0.57) | 0.51 (0.49–0.52) | 0.53 (0.52–0.54) | 0.49 (0.47–0.50) |
| Training set | Pleural effusion |
||||
|---|---|---|---|---|---|
| White | Asian | Black | Female | Male | |
| AUC (95% CI) | |||||
| MIMIC-CXR | 0.89 (0.89–0.89) | 0.90 (0.88–0.91) | 0.91 (0.90–0.91) | 0.91 (0.90–0.91) | 0.89 (0.88–0.89) |
| CheXpert | 0.88 (0.88–0.88) | 0.88 (0.87–0.90) | 0.90 (0.89–0.90) | 0.89 (0.89–0.90) | 0.88 (0.87–0.88) |
| TPR (95% CI) | |||||
| MIMIC-CXR | 0.84 (0.84–0.85) | 0.84 (0.81–0.87) | 0.79 (0.77–0.81) | 0.83 (0.82–0.85) | 0.84 (0.83–0.85) |
| CheXpert | 0.82 (0.82–0.83) | 0.82 (0.79–0.85) | 0.75 (0.73–0.77) | 0.82 (0.81–0.83) | 0.81 (0.80–0.82) |
| FPR (95% CI) | |||||
| MIMIC-CXR | 0.22 (0.21–0.22) | 0.20 (0.18–0.22) | 0.15 (0.14–0.15) | 0.18 (0.18–0.18) | 0.22 (0.21–0.22) |
| CheXpert | 0.22 (0.21–0.22) | 0.21 (0.19–0.23) | 0.14 (0.13–0.15) | 0.19 (0.18–0.19) | 0.21 (0.21–0.22) |
| Youden's J statistic (95% CI) | |||||
| MIMIC-CXR | 0.63 (0.62–0.64) | 0.64 (0.60–0.67) | 0.64 (0.62–0.66) | 0.65 (0.64–0.67) | 0.62 (0.61–0.63) |
| CheXpert | 0.61 (0.60–0.62) | 0.61 (0.57–0.64) | 0.61 (0.59–0.63) | 0.63 (0.62–0.64) | 0.60 (0.59–0.61) |
Disease detection results reported separately for each race group and biological sex for ‘no finding’ (top) and ‘pleural effusion’ (bottom). TPR and FPR in subgroups are determined using a fixed decision threshold optimized over the whole patient population for a target FPR of 0.20.