Table 2.
Metrics of the AI system.
| Task | AUC (95% CI) | Sensitivity (95% CI) | Specificity (95% CI) |
|---|---|---|---|
| a | |||
| Non-pneumonia diagnosis | 0.9752 (0.9726–0.9783) | 0.9343 (0.9290–0.9429) | 0.9801 (0.9778–0.9827) |
| CAP diagnosis | 0.9804 (0.9776–0.9837) | 0.9687 (0.9634–0.9741) | 0.9407 (0.9366–0.9448) |
| Influenza-A/B diagnosis | 0.9885 (0.9861–0.9928) | 0.8307 (0.7962–0.8696) | 0.9945 (0.9936–0.9960) |
| COVID-19 diagnosis | 0.9745 (0.9722–0.9771) | 0.8703 (0.8620–0.8784) | 0.9660 (0.9629–0.9693) |
| Multi-way metrics | 0.9781 (0.9756–0.9804) | 0.9151 (0.9115–0.9193) (the metric changed to accuracy) | |
| b | |||
| Normal diagnosis | 0.9541 (0.9511–0.9574) | 0.8561 (0.8471–0.8661) | 0.9524 (0.9494–0.9563) |
| Common pneumonia diagnosis | 0.9098 (0.9058–0.9139) | 0.8823 (0.8759–0.8904) | 0.8685 (0.8628–0.8745) |
| COVID-19 diagnosis | 0.9212 (0.9175–0.9255) | 0.7799 (0.7706–0.7893) | 0.9355 (0.9315–0.9397) |
| Multi-way metrics | 0.9299 (0.927–0.933) | 0.8435 (0.8391–0.8483) (the metric changed to accuracy) | |
| c | |||
| COVID-19 diagnosis on MosMedData cohort | 0.9325 (0.9257–0.9382) | 0.9446 (0.9379–0.9510) | 0.6613 (0.6359–0.6855) |
| d | |||
| CT COVID-19 diagnosis | 0.9847 (0.9822–0.9877) | 0.9762 (0.9718–0.98110) | 0.91250 (0.8975–0.9301) |
| CXR COVID-19 diagnosis | 0.9527 (0.9474–0.9583) | 0.9623 (0.9570–0.9673) | 0.7155 (0.6918–0.7436) |
| Result-level fusion | 0.9894 (0.9873–0.9917) | 0.9469 (0.9399–0.9543) | 0.9503 (0.9371–0.9627) |
| e | |||
| Pneumonia-or-non-pneumonia cohort | 0.9869 (0.9818–0.9993) | 0.9404 (0.9210–0.9756) | 1.0000 (1.0000–1.0000) |
| CAP-or-COVID-19 cohort | 0.9727 (0.9637–0.9825) | 0.9591 (0.9459–0.9767) | 0.9199 (0.8947–0.9512) |
| Influenza-or-COVID-19 cohort | 0.9585 (0.9413–0.9813) | 0.94961 (0.93333–1.0) | 0.8331 (0.7826–0.8846) |
| f | |||
| Lung segmentation | 0.9255 (0.6018–0.9732) | 0.9660 (0.7553–0.9918) | 0.9956 (0.9787–0.9983) |
| g | |||
| COVID-infectious slices locating | 0.9559 (0.9532–0.9586) | 0.8009 (0.79323–0.8094) | 0.9636 (0.9607–0.9666) |
a Metrics on test cohort. b Metrics on CC-CCII test cohort. c Metrics on MosMedData test cohort. d Metrics on comparation between CT and XCT on paired data of test cohort. e Metrics on three reader study tasks. f Lung segmentation performances. g COVID-infectious slices locating.