Table 3.

Results of four pulmonologists without and with AI assistance on test set in distinguishing COVID‐19 from community‐acquired pneumonia.

	Evaluation metrics									Binomial distribution (McNemar)
	Accuracy			Sensitivity			Specificity
	Without AI	With AI	Delta	Without AI	With AI	Delta	Without AI	With AI	Delta	p
Pulmonologist‐1	0.825	0.916	0.091	0.826	0.853	0.027	0.824	0.965	0.141	<0.001
Pulmonologist‐2	0.793	0.904	0.111	0.587	0.798	0.211	0.951	0.986	0.035	0.031
Pulmonologist‐3	0.781	0.880	0.099	0.716	0.807	0.091	0.831	0.937	0.106	0.006
Pulmonologist‐4	0.797	0.857	0.060	0.670	0.706	0.036	0.894	0.972	0.078	0.007
Pulmonologists average	0.799	0.889	0.090	0.700	0.791	0.091	0.875	0.965	0.090	<0.001

Abbreviations: AI, artificial intelligence; COVID‐19, coronavirus disease 2019.