. 2021 Oct 14;11:20384. doi: 10.1038/s41598-021-99986-3

Table 3.

Comparison of model performance with Radiologist labels.

Class	Radiologist 1	Radiologist 2	Radiologist 3	Radiologist 4	Radiologist Consensus	CovIx
Accuracy
Normal	0.92	0.79	0.77	0.87	0.88	0.86
Abnormal	0.74	0.64	0.71	0.71	0.72	0.67
Pneumonia	0.91	0.84	0.82	0.88	0.88	0.73
COVID-19	0.84	0.86	0.87	0.84	0.86	0.94
Sensitivity
Normal	0.91	0.91	0.94	0.88	0.94	0.62
Abnormal	0.60	0.40	0.44	0.60	0.63	0.42
Pneumonia	0.25	0.25	0.38	0.38	0.25	1.0
COVID-19	0.71	0.41	0.35	0.41	0.41	0.82
F1-score
Normal	0.88	0.73	0.72	0.81	0.83	0.74
Abnormal	0.67	0.49	0.57	0.64	0.66	0.52
Pneumonia	0.31	0.20	0.25	0.33	0.25	0.37
COVID-19	0.60	0.50	0.48	0.47	0.50	0.82
PPV
Normal	0.85	0.62	0.59	0.76	0.75	0.91
Abnormal	0.74	0.63	0.79	0.68	0.69	0.69
Pneumonia	0.40	0.17	0.19	0.30	0.25	0.23
COVID-19	0.52	0.64	0.75	0.54	0.64	0.82