Table 2.

Inter-Rater Agreement Between Chest Radiologists According to the RSNA Expert Consensus Statement on Reporting Chest CT Findings Related to COVID-19.

COVID-19 appearance	Agreement	Fleiss kappa	Standard error	P value	Interpretation
Typical COVID-19 appearance
Radiologist 1 and radiologist 2 and radiologist 3		0.815		<.0001	Almost perfect agreement
Radiologist 1 and radiologist 2	90.1%	0.803	0.034	<.0001	Almost perfect agreement
Radiologist 1 and radiologist 3	91.1%	0.821	0.033	<.0001	Almost perfect agreement
Radiologist 2 and radiologist 3	91.1%	0.823	0.032	<.0001	Almost perfect agreement
Indeterminate COVID-19 appearance
Radiologist 1 and radiologist 2 and radiologist 3		0.636		<.0001	Substantial agreement
Radiologist 1 and radiologist 2	89.1%	0.597	0.063	<.0001	Moderate agreement
Radiologist 1 and radiologist 3	89.4%	0.668	0.054	<.0001	Substantial agreement
Radiologist 2 and radiologist 3	89.8%	0.641	0.058	<.0001	Substantial agreement
Atypical COVID-19 appearance
Radiologist 1 and radiologist 2 and radiologist 3		0.806		<.0001	Almost perfect agreement
Radiologist 1 and radiologist 2	98.0%	0.823	0.071	<.0001	Almost perfect agreement
Radiologist 1 and radiologist 3	98.3%	0.830	0.074	<.0001	Almost perfect agreement
Radiologist 2 and radiologist 3	97.7%	0.762	0.086	<.0001	Substantial agreement
Negative for pneumonia
Radiologist 1 and radiologist 2 and radiologist 3		0.962		<.0001	Almost perfect agreement
Radiologist 1 and radiologist 2	98.3%	0.960	0.018	<.0001	Almost perfect agreement
Radiologist 1 and radiologist 3	98.7%	0.968	0.016	<.0001	Almost perfect agreement
Radiologist 2 and radiologist 3	98.3%	0.960	0.018	<.0001	Almost perfect agreement

Abbreviations: CT, computed tomography; RSNA, Radiological Society of North America.