Table 4.
Effect of training set on disease detection with Dense-121 tested on CheXpert.
| Training set | No finding |
||||
|---|---|---|---|---|---|
| White | Asian | Black | Female | Male | |
| AUC (95% CI) | |||||
| CheXpert | 0.87 (0.86–0.88) | 0.88 (0.86–0.89) | 0.88 (0.87–0.90) | 0.87 (0.86–0.88) | 0.87 (0.86–0.88) |
| MIMIC-CXR | 0.85 (0.85–0.86) | 0.86 (0.84–0.88) | 0.88 (0.86–0.90) | 0.85 (0.84–0.86) | 0.86 (0.85–0.87) |
| TPR (95% CI) | |||||
| CheXpert | 0.79 (0.77–0.80) | 0.80 (0.76–0.83) | 0.84 (0.80–0.88) | 0.79 (0.77–0.82) | 0.79 (0.77–0.81) |
| MIMIC-CXR | 0.77 (0.76–0.79) | 0.78 (0.75–0.82) | 0.83 (0.79–0.87) | 0.78 (0.76–0.80) | 0.78 (0.76–0.80) |
| FPR (95% CI) | |||||
| CheXpert | 0.20 (0.20–0.20) | 0.20 (0.19–0.21) | 0.23 (0.21–0.24) | 0.20 (0.20–0.21) | 0.20 (0.20–0.20) |
| MIMIC-CXR | 0.20 (0.20–0.20) | 0.19 (0.18–0.20) | 0.23 (0.22–0.25) | 0.21 (0.20–0.21) | 0.19 (0.19–0.20) |
| Youden's J statistic (95% CI) | |||||
| CheXpert | 0.59 (0.57–0.60) | 0.60 (0.56–0.64) | 0.61 (0.57–0.65) | 0.59 (0.57–0.61) | 0.59 (0.57–0.61) |
| MIMIC-CXR | 0.57 (0.56–0.59) | 0.59 (0.55–0.63) | 0.60 (0.55–0.64) | 0.57 (0.55–0.59) | 0.59 (0.57–0.61) |
| Training set | Pleural effusion |
||||
|---|---|---|---|---|---|
| White | Asian | Black | Female | Male | |
| AUC (95% CI) | |||||
| CheXpert | 0.86 (0.86–0.87) | 0.88 (0.87–0.89) | 0.86 (0.85–0.88) | 0.87 (0.86–0.87) | 0.86 (0.86–0.87) |
| MIMIC-CXR | 0.85 (0.85–0.85) | 0.87 (0.86–0.88) | 0.85 (0.84–0.87) | 0.86 (0.85–0.86) | 0.85 (0.85–0.86) |
| TPR (95% CI) | |||||
| CheXpert | 0.77 (0.76–0.78) | 0.78 (0.76–0.80) | 0.71 (0.68–0.74) | 0.76 (0.75–0.78) | 0.77 (0.76–0.78) |
| MIMIC-CXR | 0.74 (0.73–0.75) | 0.76 (0.75–0.78) | 0.69 (0.66–0.72) | 0.75 (0.74–0.76) | 0.73 (0.72–0.74) |
| FPR (95% CI) | |||||
| CheXpert | 0.21 (0.20–0.21) | 0.19 (0.18–0.20) | 0.16 (0.14–0.17) | 0.20 (0.19–0.20) | 0.20 (0.20–0.21) |
| MIMIC-CXR | 0.20 (0.20–0.21) | 0.20 (0.19–0.22) | 0.17 (0.15–0.19) | 0.20 (0.20–0.21) | 0.20 (0.19–0.20) |
| Youden's J statistic (95% CI) | |||||
| CheXpert | 0.56 (0.55–0.57) | 0.59 (0.57–0.61) | 0.55 (0.52–0.59) | 0.57 (0.55–0.58) | 0.57 (0.55–0.58) |
| MIMIC-CXR | 0.54 (0.52–0.55) | 0.56 (0.54–0.58) | 0.52 (0.49–0.56) | 0.55 (0.53–0.56) | 0.53 (0.52–0.55) |
Disease detection results reported separately for each race group and biological sex for ‘no finding’ (top) and ‘pleural effusion’ (bottom). TPR and FPR in subgroups are determined using a fixed decision threshold optimized over the whole patient population for a target FPR of 0.20.