Chest CT accuracy in diagnosing COVID-19 during the peak of the Italian epidemic: A retrospective correlation with RT-PCR testing and analysis of discordant cases

Highlights

• •In our experience chest CT had a significantly higher specificity and accuracy in detecting COVID-19 pneumonia than previously reported.
• •Chest CT and RT-PCR positive rates were 485/773 (62.7 %) and 462/773 (59.7 %), respectively.
• •CT sensitivity and specificity for COVID 19 with RT-PCR as reference were 90.7 % and 78.8 % respectively.
• •CT PPV, NPV and accuracy were 86.4 %, 85.1 % and 85.9 % respectively.

Abstract

Objectives

The goal of this study was to assess chest computed tomography (CT) diagnostic accuracy in clinical practice using RT-PCR as standard of reference.

Methods

From March 4th to April 9th 2020, during the peak of the Italian COVID-19 epidemic, we enrolled a series of 773 patients that performed both non-contrast chest CT and RT-PCR with a time interval no longer than a week due to suspected SARS-CoV-2 infection. The diagnostic performance of CT was evaluated according to sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and diagnostic accuracy, considering RT-PCR as the reference standard. An analysis on the patients with discrepant CT scan and RT-PCR result and on the patient with both negative tests was performed.

Results

RT-PCR testing showed an overall positive rate of 59.8 %. CT sensitivity, specificity, PPV, NPV, and accuracy for SARS-CoV-2 infection were 90.7 % [95 % IC, 87.7%–93.2%], 78.8 % [95 % IC, 73.8−83.2%], 86.4 % [95 % IC, 76.1 %–88.9 %], 85.1 % [95 % IC, 81.0 %–88.4] and 85.9 % [95 % IC 83.2−88.3%], respectively. Twenty-five/66 (37.6 %) patients with positive CT and negative RT-PCR results and 12/245 (4.9 %) patients with both negative tests were nevertheless judged as positive cases by the clinicians based on clinical and epidemiological criteria and consequently treated.

Conclusions

In our experience, in a context of high pre-test probability, CT scan shows good sensitivity and a consistently higher specificity for the diagnosis of COVID-19 pneumonia than what reported by previous studies, especially when clinical and epidemiological features are taken into account.

1. Introduction

On January 9, 2020, the WHO declared the detection of a new coronavirus strain never identified before in humans, called severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) [1], later shown to be the etiological agent of the respiratory disease COVID-19 (Coronavirus Disease 2019) [2].

In light of the elevated infectiousness, diagnosing COVID-19 at early stages of infection is crucial to immediately isolate the infected subjects from the healthy population and prevent further viral spread. Although imaging plays an important role in diagnosing and evaluating COVID-19 [3,4], the definitive diagnosis relies on tests by real-time reverse-transcriptase-polymerase chain reaction (RT-PCR) [5,6]. However, it is currently unclear how often false negative RT-PCR results can occur. This issue is further complicated by the fact that the acquisition of RT-PCR findings requires several hours, whereas CT scan results can be obtained in a matter of minutes [7].

The RT-PCR tests performed during the Wuhan outbreak have shown various degrees of sensitivity, ranging from 37 to 71 % [8,9], presumably due to several factors, such as throat swab adequacy, sample type, and sample acquisition time [10,11]. Currently, the WHO does not recommend the use of serological tests to diagnose patients with ongoing SARS-CoV-2 infection [12].

On the other hand, chest computed tomography (CT) appears to be reliable in diagnosing COVID-19, especially in areas with high incidence and prevalence of this disease. The radiographic features of infected lungs include ground-glass opacities (GGOs), multifocal patchy consolidation, and/or interstitial changes with a peripheral distribution. Noteworthy, these features can also be observed in symptomatic patients with negative RT-PCR results [13].

In this scenario, the largest correlation study of chest CT and RT-PCR testing, including 1014 patients, has revealed a CT sensitivity in diagnosing COVID-19 of 97 % [8]. In another Italian study, including 158 patients, the sensitivity, specificity, and accuracy of CT for COVID-19 were 97 %, 56 %, and 72 %, respectively [14].

While some authors [15] support the use of chest CT in the screening for COVD-19 in patients, the latest Fleischner Society Statement issued on April 2020 [16] does not recommend performing imaging for medical triage of asymptomatic or in mildly symptomatic patients. Instead, they advocate the use of chest CT in COVID-19 patients with a worsening respiratory status or moderate to severe COVID-19 features regardless of RT-PCR test results. Furthermore, chest imaging is deemed appropriate in patients presenting with functional impairment and/or hypoxemia after recovery. In the period of time in which our study was conducted we found ourselves in a scenario were the pre-test probability of COVID-19 was high and the resources were constrained. The clinicians responsible for the Emergency Department (ED), in such rapidly evolving and difficult situation, felt that chest CT could help the assessment of patients suspected for COVID-19 infection. In particular the low sensitivity of a single RT-PCR test and the necessity to quickly address a COVID-19 positive patient towards isolation and proper treatment in order to improve the outcome and reduce the spread of the disease drove the execution of a large number of CT scans.

Our main goal was to assess chest CT accuracy in diagnosing COVID-19 pneumonia using RT-PCR as standard of reference, according to the model developed by Ai et al. [8]. We also sought to determine the correlation between CT findings and positive RT-PCR results.

2. Materials and methods

2.1. Patient population and study design

This retrospective study was approved by the Institute Research Medical Ethics Committee at AOU Maggiore della Carità, Novara, Italy; protocol number CE 123/20 Written informed consent was waived.

We employed a study design similar to that published by Ai et al. [8]. To this end, we enrolled a series of 773 patients admitted at the ED of our hospital from March 3rd to April 9th 2020, during the peak of the Italian COVID-19 epidemic.

The patient inclusion criteria were as follows:

• 1Clinically suspected SARS-CoV-2 infection;
• 2Having performed a chest CT scan in the ED;
• 3Having undergone 1 or 2 RT-PCR assays within 7 days of the CT scan.

The exclusion criteria were:

• 1Presence of severe motion artifacts in the CT scan;
• 2Having undergone a chest CT and an RT-PCR test with a time interval >7 days;
• 3Unknown RT-PCR date.

In our institution a patient was defined as clinically suspected for COVID-19 when one or more of this conditions were met: (a) presence of fever (i.e., temperature >37.5 °C), cough and dyspnea; (b) presence of mild symptoms and ascertained close contact with a confirmed COVID-19 patient; (c) one previously positive laboratory test result. A flowchart of the study is shown in Fig. 1 .

Fig. 1.

Flowchart of the study.

Abbreviation: RT-PCR, reversal transcription polymerase chain reaction.

Notably all the patients with the above conditions were examined at the presentation in our ED during the peak of the epidemic, at the time when the symptoms determined their access to the hospital, regardless to the severity and to the time of onset of the clinical signs. This parameters were therefore not collected in the majority of cases and not considered in the present study.

2.2. Personal data

For each patient, information such as gender, date of birth, date of the RT-PCR test and date of the chest CT were recorded.

Enrolled patients were assigned a progressive identification number (ID), which was later used to collect their personal data. This procedure guaranteed anonymity, allowing non-disclosure of sensitive data.

2.3. RT-PCR: swabs and laboratory tests

The RT-PCR results were obtained from the patient’s electronic medical records stored in our hospital information system.

Some patients (n = 51) with positive chest CT and negative RT-PCR findings repeated the nasopharyngeal swab within 7 days of the CT scan. Additionally, 41 patients with negative CT scan and negative RT-PCR repeated the swab within the allowed time interval.

The positivity to SARS-CoV-2 was then confirmed by at least one RT-PCR. These assays were performed by means of nasopharyngeal swabs (Xpert® Nasopharyngeal Sample Collection Kit for Viruses or eNAT® Transport and Preservation Media plus One pernasal applicator swab). The viral RNA was first extracted (Seegene Nimbus IVD, Hamilton ELITe Galaxy, Abbott m2000sp or Biomèrieux NUCLISENS® EASYMAG®) and then amplified and detected (Applied Biosystems™ 7500 Fast Dx Real-Time PCR Instrument or Abbott m2000rt) using two different kits (GeneFinderTM COVID-19 Plus RealAmp Kit or Allplex™ 2019-nCoV Assay). Moreover, in our Institute a fast direct sample amplification instrument was also used (DiaSorin LIAISON® MDX). All these procedures are in full compliance with the WHO Guidelines [17].

2.4. CT room staff organization and decontamination technique

Our CT exams on patients suspected of COVID-19 were performed in the ED CT room.

Decontamination of the room was achieved by disinfecting environment surfaces with 5% NaClO. After each chest CT examination, passive air exchange was performed. Other patients arriving at the ED, not clinically suspected of COVID-19, performed the CT scanning on another device present in our Radiology Department [18].

2.5. CT acquisition technique and image analysis

All Chest CT scans were performed during single full inspiratory breath hold in supine position on a 128-slice CT (Philips Ingenuity Core, Philips Healthcare, Netherlands). The scan technical average parameters were: tube voltage: 120 kV; tube current modulation: 226 mAs; spiral pitch factor: 1.08; collimation width 0.625, matrix 512 (mediastinal window) and 768 (lung window). All images were reconstructed with a slice thickness of 1 mm. For younger subjects (<40 years) tube voltage was adequately reduced to 80–100 KV and the scanner automatically adjusted the tube current modulation on the basis of the scout image in order to administer an adequate dose amount.

Two radiologists with more than 10 years of thoracic imaging experience evaluated the images in consensus, blindly to the RT-PCR results. The patients were defined either CT-positive or CT-negative based on the presence of CT findings compatible with COVID-19 infection as defined by the STR/ACR/RSNA consensus statement [19]; CT scans with “typical” or “indeterminate” features were considered positive, while CT scans with “atypical” or “negative” features were considered negative.

As suggested by Chung et al. in their recent paper [3], the following CT features were recorded: (a) number of lobes with GGOs; (b) number of lobes with consolidation, (c) presence of crazy paving; (d) presence of reversed halo sign: (e) bilateral distribution, (f) lymphadenopathy, defined as at least one lymph node with short axis >10 mm; and (g) pleural or (h) pericardial effusion (Fig. 2, Fig. 3 ).

Fig. 2.

Chest CT images of the various pattern of COVID-19 pneumonia in true positive (TP) patients. A) A 36-year-old man with bilateral, peripheral patchy areas of ground-glass opacity (GGO) (axial view). B) A 42-year-old man with bilateral diffuse areas of GGO (axial view). C) A 45-year-old woman with bilateral diffuse areas of consolidation associated with GGO (axial view).

Fig. 3.

Chest CT features consistent with COVID-19 pneumonia in TP patients. A) A 60-year-old woman with bilateral patchy areas of GGO (coronal view). B) A 43-year-old woman with bilateral diffuse areas of consolidation associated with GGO (coronal view). C) A 63-year-old man with bilateral diffuse areas of crazy paving (axial view). D) A 39-year-old man with reversed halo sign in the left inferior lobe (axial view).

2.6. Standard of reference

The RT-PCR performed with a time interval not longer than 7 days from the chest CT was our standard of reference. Several patients underwent two or more RT-PCR tests during the considered time interval. The result of the one or two performed tests were considered to determine the overall RT-PCR positivity, as follows:

• •1/1 positive swab: overall positive RT-PCR
• •1/1 negative swab: overall negative RT-PCR
• •2/2 positive swabs : overall positive RT-PCR
• •2/2 negative swabs: overall negative RT-PCR
• •1/2 positive swabs (negative to positive or positive to negative): overall positive RT-PCR

We utilized this standard of reference to classify the patients into true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN). The categories were so defined:

• •TP: typical or indeterminate CT AND overall positive RT-PCR
• •TN: atypical or negative CT AND overall negative RT-PCR
• •FP: typical or indeterminate CT AND overall negative RT-PCR
• •FN: atypical or negative CT AND overall positive RT-PCR

2.7. Statistical analysis

Statistical analyses were performed through STATA/IC11 software. Continuous variables were expressed as means and ranges. Categorical variables were expressed as counts and percentages. Statistical correlation was evaluated by Pearson Chi-square or Bonferroni test.

The diagnostic performance of CT was evaluated by measuring sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and diagnostic accuracy, considering RT-PCR as the reference standard. Diagnostic accuracy was also evaluated in different age groups. The chosen thresholds were 50 years, since Li et al. described an age greater than 50 among the risk factors for severe/critical COVID-19 pneumonia [20], and 60 years, since Ai et al. reported a higher accuracy and PPV of the Chest CT in the patients ≥60 years [8].

3. Results

3.1. Patient population

As shown in Fig. 1 our original population included 1009 patients who underwent chest CT scan in our ED due to suspected COVID-19. We first excluded 185 patients who had not undergone an RT-PCR. Twenty-four of the remaining patients were excluded because of severe motion artifacts, while 27 other patients were excluded because the time-interval between the chest CT and the RT-PCR assay had been longer than 7 days. Thus, a total of 773 patients were available for analysis.

The mean age of the study participants was 62.4 years +/−18.2, and 54.8 % [424/773] of them were males. 206 patients (26.7 %) were younger than 50 years, 134 patients (17.3 %) aged from 50 to 59 years and 433 (56 %) were older than 60 years (Table 1 ). Within the study population, RT-PCR testing showed a positive rate of 59.8 % (462/773) (95 % confidence interval [CI], 56.3 %–63.2 %), while chest CT imaging showed a positive rate of 62.7 % (485/773) (95 % CI, 59.3 %–66.2 %) (Table 1). The median time interval between chest CT exams and RT-PCR tests was 1 day (range: 0−7 days); 74 patients (9.6 %) performed the RT-PCR before the CT scan, while 699 (90.4 %) performed the throat swab after the chest CT. The RT-PCR results were always unavailable at the time of CT reporting.

Table 1.

Demographic Characteristics and main CT Findings of the 773 Chest Scans.

Characteristics	Results
Age (years)
Mean age	62.4 ± 18.2, range 16−100
0−49	206 (26.7)
50−59	134 (17.3)
60	433 (56)
Male	424 (54.8)
Median time-interval between chest CT scan and RT-PCR assay (days)	1, range 0−7
Results of RT-PCR assay
Positive	462 (59.8)
Negative	311 (40.2)

Characteristics	Consistent with COVID-19 Pneumonia	Non-Consistent with COVID-19 Pneumonia
Total No. of Chest CT Scans	485/773 (62.7)	288/773 (37.3)
Findings
1GGO (at least one lobe)	484 /485 (99.9)	73 /288 (25.3)
2Consolidation (at least one lobe)	406 /485 (83.7)	69 /288 (24)
3Bilaterally	432 /485 (89.1)	47 /288 (16.3)
4Crazy paving	164 /485 (33.8)	1 /288 (0.3)
5Reversed halo sign	30 /485 (6.2)	1 /288 (0.3)
6LNs	93 /485 (19.2)	54 /288 (18)
7Pleural effusion	41 /485 (8.4)	56 /288 (19.4)
8Pericardial effusion	16 /485 (3.3)	15 /288 (5.2)

Note: data are patients with percentages in parentheses. Age is mean ± standard deviation. Time-interval is shown as median.

Abbreviation: RT-PCR, reversed transcription polymerase chain reaction; GGO, ground glass opacity; LNs, lymph nodes.

Chest CT findings are reported in Table 1. For patients with CT features consistent with COVID-19 pneumonia (62.7 % [485/773]), the main findings were GGOs, consolidation and bilateral involvement in 99.9 % (484/485), 83.7 % (406/485) and 89.1 % (432/485), respectively (Fig. 2, Fig. 3).

432/485 (89.1 %) patients with CT findings consistent with COVID-19 pneumonia had a “typical appearance” as described by the STR/ACR/RSNA consensus statement, while 53/485 (10.9 %) had an “indeterminate appearance”.

3.2. Radiation exposure

Radiation exposure was retrospectively evaluated in 100 patients, from 200th to 299th. In patients weighting up to 90 kg the mean CT dose index (CTDI) volume was 8.9 +/−1.6 mGy and the mean dose length product (DLP) was 334.2 +/−33.8 mGy*cm. In the subjects weighting more than 90 kg the mean CTDI volume and the mean DLP were 15.1 +/−2.4 mGy and 557.6 +/−62.6 mGy*cm.

3.3. CT diagnostic performance

Using RT-PCR as reference standard (Table 2 ), 419 patients were found to be true positives, while 245 patients resulted true negatives (see Fig. 4 ). Sixty-six patients received a positive chest CT diagnosis and a negative RT-PCR assay (FP), while 43 patients received a negative chest CT diagnosis despite testing positive for SARS-CoV-2 RNA by RT-PCR (FN).

Table 2.

Performance of Chest CT in diagnosing COVID-19 with RT-PCR Results as Standard of Reference.

	Results (n)					Test performance (%)
	TP	TN	FP	FN	Positive RT-PCR	Sensitivity (95 % CI)	p-value	Specificity (95 % CI)	p-value	PPV (95 % CI)	p-value	NPV (95 % CI)	p-value	Accuracy (95 % CI)	p-value
Overall
	419	245	66	43	462 (59.8)	90.7 (87.7−93.2)		78.8 (73.8−83.2)		86.4 (83.6−88.7)		85.1 (81−88.4)		85.9 (83.2−88.3)
Age
<50	81	94	16	15	96 (46.6)	84.4 (75.5−91)	0.017	85.4 (76.5−91.4)	0.033	83.5 (76.1−88.9)	0.354	86.2 (79.6−91)	0.664	85 (79.3−89.5)	0.899
≥50	338	151	50	28	366 (64.5)	90.7 (89.1−94.9)		75.1 (68.5−80.9)		87.1 (84.1−89.6)		84.4 (78.9−88.6)		86.2 (83.1−89)
<60	166	126	29	19	185 (54.4)	89.7 (84.4−93.7)	0.888	81.3 (74.2−87.1)	0.280	85.1 (80.4−88.9)	0.506	86.9 (81.1−91.1)	0.381	85.9 (81.7−89.4)	0.997
≥60	253	119	37	24	277 (63.9)	91.3 (87.4−94.4)		76.3 (68.8−82.7)		87.2 (83.7−90)		83.2 (77−88)		85.9 (82.3−89)
Sex
Male	261	108	33	21	282 (66.6)	92.5 (88.8−95.3)	0.08	76.6 (68.7−83.3)	0.39	88.8 (85.4−91.4)	0.057	83.7 (77.1−88.7)	0.562	87.3 (83.7−90.3)	0.746
Female	158	137	33	22	180 (51.4)	87.8 (82.1−92.2)		80.6 (73.4−86.2)		82.7 (77.8−86.7)		86.2 (80.7−90.3)		84.3 (80−87.9)

Note: for positive RT-PCR, data are patients with percentages in parentheses; the other data in brackets are 95 % confidence interval.

Abbreviation: TP, true positive; TN, true negative; FP, false positive; FN, false negative; PPV, positive predictive value; NPV, negative predictive value; RT-PCR, reversed transcription polymerase chain reaction.

Fig. 4.

Chest CT features non consistent with COVID-19 pneumonia in true negative (TN) patients. Column A) A 32-year-old man with no parenchymal or mediastinal alteration (axial and coronal view). Column B) A 55-year-old woman with “tree-in-bud” alterations in the right upper lobe (axial and coronal view). Column C) A 85-year-old woman with bilateral pleural effusion and CT features consistent with congestive heart failure (CHF) (axial and coronal view).

The CT sensitivity, specificity, PPV, NPV, and accuracy for SARS-CoV-2 infection were 90.7 % [95 % IC, 87.7 %–93.2 %], 78.8 % [95 % IC, 73.8−83.2%], 86.4 % (95 % IC, 76.1 %–88.9 %), 85.1 % (95 % IC, 81.0 %–88.4 %) and 85.9 % [95 % IC 83.2−88.3 %], respectively.

Unsurprisingly the PPV proved to be higher in patients with typical appearance, since 47/432 patients (10.8 %) with typical CT features and 19/53 (35.8 %) patients with indeterminate features proved to be FP; with a resulting PPV of 89.1 % and 64.2 % respectively (p < 0.001).

The performance of chest CT in diagnosing COVID-19 in different age and sex groups is shown in Table 2. Statistically significant differences in terms of sensitivity (<50 years 84.4 % [IC 95 %, 75.5−91,0]; vs. 50 years 90.7 % [IC 95 %, 89.1−94.9]; p = 0.017) and specificity (<50 years 85.4 % [IC 95 %, 76.5–88.9]; vs. 50 years 75.1 % [IC 95 %, 68.5−80.9]; p = 0.033). Noteworthy, in the patients’ group <50 years the positive rate of the RT-PCR was 46.6 %, while in the group ≥50 years the positive rate was 64.5 %. On the other hand, the positive rates in the patients’ groups <60 years and ≥60 years were 57.3 % and 64 % respectively and no statistically significant offset was found between these categories.

3.4. Analysis of FP, FN and TN

Of note, 37.9 % (25/66) (mean age 62.7) patients with positive CT scan and negative RT-PCR assay (Table 3 ) were considered to be affected by SARS-CoV-2 pneumonia by the clinicians based on clinical symptoms, epidemiological features, laboratory test results and chest CT findings. These patients were thus isolated and treated (Fig. 5 ). In particular, 17/25 patients (68 %) received two or more RT-PCR tests, while 8/25 (32 %) performed a single nasopharyngeal swab. No one of these patients had the diagnosis subsequently modified, and they were isolated and treated for COVID-19 until the discharge (23/25) or the demise (2/25).

Table 3.

Analysis of FP, FN and TN.

	Total	Age	p-value	Sex (Male)
FP (n = 66)
Treated for COVID-19 (A)	25 (37.9)	62.7 (17−86)		15 (60)
Not treated for COVID-19 (B)	14 (21.2)	72.1 (35−91)	0.024	4 (28.6)	0.165
Data unknown (C)	27 (40.9)	56.4 (22−96)		12 (44.4)
A x B			0.310*		0.187*
A x C			0.553*		0.789*
B x C			0.020*		1.000*
FN (n = 43)
Completely negative CT scan (D)	20 (46.5)	49.8 (23−100)		7 (35)
Confounding pathologies (E)	16 (37.2)	74.9 (35−88)	0.002	11 (68.7)	0.236
No confounding pathologies (F)	7 (16.3)	59 (42−86)		4 (57.1)
D x E			0.230*		1.000*
D x F			0.001*		0.280*
E x F			0.856*		1.000*
TN (n = 245)
Completely negative CT scan	138 (56.3)	50.1 (16−94)		48 (34.8 %)
Positive CT scan	107 (43.7)	70.1 (17−97)	<0.001	61 (57 %)	0.032
­COPD	11 (4.5)	67.6 (48−89)		9 (81.8)
­CHF	28 (11.4)	79.9 (47−96)		11 (39.3)
­Other interstitial pneumonia	14 (5.7)	66.8 (33−88)		10 (71.4)
­Community-acquired pneumonia	28 (11.4)	66.1 (17−90)		12 (42.8)
­Cancer	13 (5.3)	64.2 (41−87)		10 (76.9)
­Only pleural effusion	9 (3.7)	74.3 (42−97)		7 (77.8)
­Other findings	4 (1.6)	58 (40−73)		3 (75)

Note: data are patients with percentages in parentheses. Age is mean ± standard deviation.

Abbreviation: RT-PCR, reversed transcription polymerase chain reaction; FP, false positive; FN, false negative; COPD, chronic obstructive pulmonary disease; CHF, congestive heart failure.

^*Multivariate analysis conducted with Bonferroni Test.

Fig. 5.

Chest CT images of false positive (FP) patients. Column A) A 79-year-old woman hospitalized but not treated for COVID-19; an alternative diagnosis of bacterial pneumonia by E. Coli was formulated. Chest CT shows diffuse areas of consolidation associated with GGO with a prevalent perihilar distribution (axial and coronal view). Column B) A 57-year-old man hospitalized and treated for COVID-19. Chest CT shows bilateral, peripheral patchy areas of GGO with initial consolidation (axial and coronal view).

Furthermore, 14/66 F P patients (21.2 %), with a mean age of 67.3, were hospitalized but not treated for COVID-19. For two of them it was made a differential diagnosis of bacterial pneumonia by E. coli (Fig. 5a), whereas a diagnosis of pulmonary thromboembolism was established in 1 patients. Finally, an empirical antibiotic treatment for community-acquired pneumonia was performed in the remaining 11 patients.

We were unable to find any clinical data for the other 27/66 patients (40.9 %, mean age 56.4), most likely because they were deferred to home treatment due to the mildness of their condition or because they were admitted to other hospitals. This patients group was significantly younger than the one hospitalized but not treated for COVID-19.

We observed that 46.5 % (20/43) of patients with negative chest CT scan and positive RT-PCR test (mean age 49.8) (Table 3) showed no significant findings in the chest CT, while 16/43 (37.2 %) patients of the same group (mean age 74.9) had CT chest with confounding conditions, that is chronic heart failure (CHF) (8 patients) or chronic obstructive pulmonary disease (COPD) (8 patients), (Fig. 6 ). In the 7/43 remaining patients (16.3 %), with a mean age of 59 years, no confounding conditions were found in the chest CT, and the reviewing radiologists lent toward differential diagnoses. As expected, the patients with completely negative CT scans were significantly younger than those showing confounding conditions.

Fig. 6.

Chest CT images of FN patients with confounding features. Column A) A 29-year-old man with no parenchymal or mediastinal alteration (axial and coronal view). Column B) A 86-year-old with diffuse emphysema (axial and coronal view). Column C) A 75-year-old woman with bilateral pleural effusion, multiple areas of consolidation and CT features consistent with CHF (axial and coronal view).

No significant CT alterations were observed in 138/245 patients of the TN group (56.3 %), while the remaining 107 CT scans showed pathological features which received alternative diagnoses, as shown in Table 3. The patients with negative scans were significantly younger (p < 0.001) and had a higher prevalence of the female sex (p 0.032). Twelve TN patients (4.9 %) were considered affected by COVID-19 by the clinicians and treated accordingly; noteworthy, 4 of these individuals had CT features consistent with CHF.

Assuming that the 25 F P and the 12 TN patients who were hospitalized and treated for COVID-19 were positive for SARS-CoV-2, the number of FP CT scans (i.e., 66) would then decrease to 41, while the number of FN (i.e., 43) would increase to 55. According to these assumptions combined together, the diagnostic accuracy of chest CT imaging in diagnosing COVID-19 pneumonia would slightly improve as follows: sensitivity 89 % (95 % CI 85.9–91.6), specificity 85 % (95 % CI 80.2–89), PPV 91.5 % (95 % CI 89.0–93.5), NPV 80.9 % (76.7–84.5), and accuracy 87.6 % (95 % CI 85–89.8).

3.5. CT findings analysis

An analysis of the CT findings was also conducted by discriminating the CT findings based on the positivity or negativity of the RT-PCR results (Table 4 ). GGO in 3 lobes, consolidation 2 lobes, association of consolidation and GGO, bilateral alterations, crazy paving and reversed halo sign were all significantly correlated with a positive RT-PCR result (Fig. 2, Fig. 3). Absence of consolidation, monolateral alterations and pleural effusion correlated significantly with a negative RT-PCR finding (Fig. 4).

Table 4.

Analysis of the CT findings in the 773 Patients According to RT-PCR results.

Characteristics	Positive RT-PCR	Negative RT-PCR	p-value
Total Chest CT Scans	462	311	<0.001
Overall GGO
0 Lobe	33 (7.1)	183 (58.8)
1 Lobe	24 (5.2)	41 (13.2)
2 Lobe	21 (4.5)	28 (9)
3 Lobe	32 (6.9)	18 (5.8)	<0.001
4 Lobe	56 (12.1)	12 (3.9)
5 Lobe	296 (64.1)	29 (9.3)
≥3 lobes	384 (83.1)	59 (19)	<0.001
Overall consolidation
0 Lobe	88 (19)	210 (67.5)
1 Lobe	49 (10.6)	40 (12.9)	<0.001
2 Lobe	83 (18)	24 (7.7)
3 Lobe	83 (18)	14 (4.5)
4 Lobe	72 (15.6)	15 (4.8)
5 Lobe	87 (18.8)	8 (2.6)
≥2 lobes	325 (70.3)	61 (19.6)	<0.001
GGO + consolidation	365 (79)	82 (26.3)	<0.001
Bilateralism	398 (84.8)	81 (26)	<0.001
Crazy paving	152 (32.9)	13 (4.2)	<0.001
Reversed halo sign	29 (6.3)	2 (0.6)	<0.001
LNs	93 (20.1)	54 (17.4)	0.337
Pleural effusion	42 (9.1)	55 (17.7)	<0.001
Pericardial effusion	18 (3.9)	13 (4.2)	0.844

Note: data are patients with percentages in parentheses.

Abbreviation: RT-PCR, reversed transcription polymerase chain reaction; GGO, ground glass opacity; LNs, lymph nodes.

Noteworthy, the presence of lymphadenopathy or pericardial effusion was not associated with either conditions.

4. Discussion

In our experience the chest CT specificity in diagnosing COVID-19 was significantly higher than previously reported if compared with RT/PCR which is the only reference of standard actually validated.

In their recent meta-analysis, Kim et al. [21] reported that the pooled sensitivity of chest CT in diagnosing COVID-19 is 94 % [95 % CI 91–96]. In comparison, our sensitivity (90.7 %) is slightly lower but still within the 95 % CI. We believe that this minor discrepancy may be due to the fact that in our FN population (n = 43) 20 patients showed no CT alterations, suggesting that these subjects were SARS-CoV-2 positive, as judged by RT-PCR, albeit not displaying SARS-CoV-2-induced pneumonia. It is possible that a percentage of these patients performed the CT scan in the first two days from the symptoms onset, when the CT is commonly judged normal [22] Sixteen of the remaining 23 patients had one or more confounding pathologies.

On the other hand, our specificity was significantly higher than that reported by Kim’s meta-analysis (78.8 % vs. 37 %, respectively). We think that this remarkable offset may be due to three distinct reasons:

• 1Several studies have reported CT scans with various and somewhat inhomogeneous findings to be consistent with COVID-19, while throughout our investigation we have been trying to be very precise and meticulous in defining the radiologic features of COVID-19, as defined by the STR/ACR/RSNA consensus statement [19], and described in the Materials and Methods.
• 2Since the beginning of the worldwide COVID-19 emergency, a number of studies describing the radiologic features of the disease have been published and assimilated by the international radiologic community. While the first studies, especially those from China, were pioneering the subject, we are now starting to incorporate those experiences into our daily clinical practice, which has definitely contributed to improve our diagnostic accuracy.
• 3Notoriously, the accuracy of one test is dependent on pre-test probability, and our investigation was conducted during the Italian COVID-19 epidemic, when the probability of this condition was obviously very high.

Even though Raptis et al. in their recent literature review [23] were skeptical about a study [24] reporting high CT specificity, we believe our findings to be genuinely true and that they reflect the real clinical experience, especially in light of the fact that our population included numerous noninfectious diseases, such as CHF, COPD, and cancer. In addition, as specified above, our readers used objective criteria to define a positive CT examination and developed their experience through direct examination and literature review.

We failed to reproduce the findings by Ai et al. [8], who reported higher PPV and accuracy of chest CT scan in patients younger than 60 years. Conversely, we found a significantly lower sensitivity and a significantly higher specificity in patients younger than 50 years. We explain these results by the presence of a higher number of CT scans without any parenchymal or mediastinal alteration in the younger group, which, conversely, displayed a lower incidence of confounding pathologies such as COPD and heart failure. It is also possible that this results, especially the higher specificity, are due to the lower percentage of positive RT-PCR rate in the patients <50 years, as explained by Leeflang et al. [25].

In our experience, CT findings of COVID-19 pneumonia (GGOs, consolidations, multilobar and bilateral involvement, crazy paving, reversed halo sign) largely overlapped those described in the recent literature [3,19]. However we experienced that CT signs of CHF and pulmonary edema largely overlap with those of COVID-19 pneumonia. In the subgroup of 51 patients with CHF CT sensitivity dropped to 61.9 %. We believe that the radiologist should be extremely careful in the differential diagnosis, possibly stating that Sars-CoV-2 infection cannot be excluded in the most dubious cases.

We trust that, given the supposed FN rate of the RT-PCR, could be correct to consider the CT FP and TN patients treated for COVID-19 as affected by the disease. If our assumptions are true, the CT scan sensitivity for COVID-19 pneumonia drops to 89 %, while the specificity increases to 85 %. While these results seem to support the hypothesis by Radpour et al. [26] that, in highly infected communities chest CT imaging may be a superior diagnostic tool compared to RT-PCR testing, we must consider that the additional benefit of CT scan as a diagnostic tool for COVID-19 in the general population is still to be proven, and that its execution is not exempt from significant risks such as radiation exposure and the risk of infection for other patients or staff in the Radiology department. Moreover, the hazard that a non-COVID-19 patient may end up in a COVID-19 dedicated department because of a FP CT scan and be therefore infected cannot be overlooked.

Even if in our investigation the specificity was considerably higher than previously reported case studies in a general population the RT-PCR, despite the low sensitivity, remains the only gold standard validated globally

As an alternative we believe that developing a “composed gold standard” according to the model utilized in pulmonary thromboembolism [27] could be clinically relevant. The new reference should be based upon clinical adjudication relying on repeat swabs, contact with patients with ascertained COVID-19, clinical and laboratory features, chest radiographs, and CT scans. On the other hand, this could introduce some incorporation bias, since the evaluated tests are part of the reference standard, and this may expand the measured accuracy of these tests, as pointed out by Watson et al. [28]. Further research and consensus along these lines are obviously needed.

Our study has several limitations. The first is that our analysis used a single RT-PCR test as the reference standard in the large majority of patients, while this is considered suboptimal given the fact that repeated RT-PCR are undoubtedly more accurate [28]. A second limitation is that lung involvement has been quantified only via the number of affected lobes: this may fail to assess the actual percentage of affected parenchyma as predictor of outcome. A third limitation is the retrospective nature of the study, during which patients presented autonomously to the ED with the consequent impossibility to collect the time interval between the onset of the clinical signs and the chest CT in the majority of cases. Finally our results could be largely determined by the epidemic context in which pre-test probability of Covid-19 pneumonia is very high.

5. Conclusion

Chest CT scan has a high sensitivity and supposedly a consistently higher specificity than what reported in previous studies, especially if clinical and epidemiological features are taken into consideration. While CT is not to be used as the solely mean to the final diagnosis, it may be an useful addiction to the diagnostic workflow, especially in emergency situations with very high rates of COVID-19 cases and limited resources in terms of personnel, time and equipment for the execution of a large number of RT-PCR tests, as well as an accurate baseline test for the subsequent evaluation of critically ill patients.

Funding

No funding has been released for this research. This study was conducted by voluntary contribution of the authors using technology belonging to Università del Piemonte Orientale and Ospedale Maggiore della Carità-Novara, Italy

Guarantor

The scientific guarantor of this publication is Prof. Alessandro Carriero.

Statistics and biometry

One of the authors, Ferruccio Aquilini, has significant statistical expertise.

Informed consent

Written informed consent was not required for this study because this research was retrospective and did not altered the treatment or the life of the patients in any way. Enrolled patients were assigned a progressive identification number (ID), which was later used to collect their personal data. This procedure guaranteed anonymity, allowing non-disclosure of sensitive data.

Ethical approval

This retrospective study was approved by the Institute Research Medical Ethics Committee at AOU Maggiore della Carità, Novara, Italy; protocol number CE 123/20.

Study subjects or cohorts overlap

This study subjects or cohorts have never been previously reported.

Methodology

Methodology:

• •retrospective
• •diagnostic or prognostic study/observational
• •performed at one institution

CRediT authorship contribution statement

Zeno Falaschi: Conceptualization, Project administration, Writing - original draft, Writing - review & editing. Pietro S.C. Danna: Conceptualization, Project administration, Writing - original draft, Writing - review & editing. Roberto Arioli: Data curation, Formal analysis, Writing - original draft. Alessio Pasché: Investigation, Writing - original draft, Data curation. Domenico Zagaria: Investigation, Writing - original draft, Methodology. Ilaria Percivale: Data curation, Writing - review & editing. Stefano Tricca: Data curation, Formal analysis. Michela Barini: Project administration, Supervision. Ferruccio Aquilini: Data curation, Software, Supervision. Stefano Andreoni: Methodology, Supervision. Alessandro Carriero: Supervision, Project administration, Resources.

Declaration of Competing Interest

The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.