Chest CT versus RT-PCR for the detection of COVID-19: systematic review and meta-analysis of comparative studies

Abstract

Objectives

To compare the performance of chest computed tomography (CT) scan versus reverse transcription polymerase chain reaction (RT-PCR) as the reference standard in the initial diagnostic assessment of coronavirus disease 2019 (COVID-19) patients.

Design

A systematic review and meta-analysis were performed as per the Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines. A search of electronic information was conducted using the following databases: MEDLINE, EMBASE, EMCARE, CINAHL and the Cochrane Central Register of Controlled Trials.

Setting

Studies that compared the diagnostic performance within the same patient cohort of chest CT scan versus RT-PCR in COVID-19 suspected patients.

Participants

Thirteen non-randomised studies enrolling 4092 patients were identified.

Main outcome measures

Sensitivity, specificity and accuracy were primary outcome measures. Secondary outcomes included other test performance characteristics and discrepant findings between both investigations.

Results

Chest CT had a median sensitivity, specificity and accuracy of 0.91 (range 0.82–0.98), 0.775 (0.25–1.00) and 0.87 (0.68–0.99), respectively, with RT-PCR as the reference. Importantly, early small, China-based studies tended to favour chest CT versus later larger, non-China studies.

Conclusions

A relatively high false positive rate can be expected with chest CT. It is possible it may still be useful to provide circumstantial evidence, however, in some patients with a suspicious clinical presentation of COVID-19 and negative initial Severe Acute Respiratory Syndrome Coronavirus 2 RT-PCR tests, but more evidence is required in this context. In acute cardiorespiratory presentations, negative CT scan and RT-PCR tests is likely to be reassuring.

Background

In December 2019, Coronavirus Disease 2019 (COVID-19), caused by Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) emerged from Wuhan city, China.^1,2 SARS-CoV-2 is part of the Coronavirdae family that includes the common cold, Severe Acute Respiratory Syndrome and Middle East Respiratory Syndrome. These viruses can cause Acute Respiratory Distress Syndrome, bilateral pneumonia and pulmonary failure leading to mortality.³ Despite advances in management (e.g. dexamethasone), there is yet to be a highly effective pharmacological therapy identified for COVID-19, further highlighting the importance of early detection and isolation of COVID-19 patients to prevent the spread of the disease.⁴

Reverse-transcription polymerase chain reaction (RT-PCR) tests are currently the gold standard diagnostic tool for COVID-19. RT-PCR assays can be performed on nasopharyngeal and/or oropharyngeal swabs, sputum, blood samples, body fluids, stool samples and bronchoalveolar lavage fluid.⁵ RT-PCR is limited by its lower than ideal sensitivity and associated relatively high number of false negative results that can lead to the erroneous assumption that a patient who actually has COVID-19 does not have it and is not infectious, leading to an undetected transmission risk in either the community or within an institutional setting.⁴

Computed tomography (CT) has become a standard of care in the diagnosis and assessment of a variety of respiratory conditions and optimises the management process.^6,7 Although CT scans are not routinely used to diagnose Acute Respiratory Distress Syndrome, certain complications relating to mechanical ventilation, including pneumonia, pneumothorax and emphysema, are sometimes identified by CT but not by chest radiography.⁸ In addition, CT imaging can be used to identify lung atelectasis due to poor positioning of the endotracheal tube as well as potentially directing ventilation to achieve optimal pressures and air recruitment.⁸ This is important in relation to COVID-19 since patients in intensive care unit require optimisation of ventilatory settings and it is increasingly recognised that prone ventilation appears favourable.⁹ Nevertheless, in the clinical setting, the benefits of routine CT imaging must be weighed against the considerable practicalities and risks, including that of infection transmission associated with transporting a patient from the intensive care unit, or elsewhere in the hospital, to the radiology department.⁸ The Royal College of Radiologists does not recommend the use of CT scanning as a diagnostic tool for COVID-19, as a negative CT result does not rule out COVID-19 infection, except for when CT scanning results can change the management course, such as in those who require emergency surgery.¹⁰ Furthermore, both the American Society of Emergency Radiology and the Society of Thoracic Radiology advise against the use of CT scanning as a diagnostic tool for COVID-19.¹¹

Given the recent observational studies of chest CT and RT-RCR in detecting COVID-19 cases,^4,12–23 there is a need for ongoing systematic reviews and meta-analyses of studies that have directly compared the performance of these tests in the same patient cohorts. This study aimed to compare the performance of chest CT scan versus RT-PCR, using the latter as the gold standard, for the diagnosis of COVID-19 by systematic review and meta-analysis of truly comparative studies.

Methods

A systematic review and meta-analysis were conducted as per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.²⁴

Eligibility criteria

All relevant studies comparing chest CT with RT-PCR (as the reference standard) in the diagnosis of patients with suspected COVID-19 were included.^4,12–23 Chest CT was the intervention group of interest and RT-PCR was the comparator. All patients were included irrespective of age, gender or co-morbidity status. Studies that did not compare the two tests directly within the same patient cohort were excluded. Studies enrolling only confirmed COVID-19 cases or studies performing chest CT for prognostic purposes were also excluded.

Primary outcomes

Sensitivity, specificity and accuracy for the diagnosis of COVID-19 cases were the primary outcomes. Sensitivity (true positive rate) is the proportion of actual COVID-19 cases a test correctly identifies. Specificity (true negative rate) is the proportion of non-COVID-19 cases a test correctly identifies. Accuracy is the proportion (true positives and true negatives) of total cases examined a test correctly identifies.

Secondary outcomes

The secondary outcomes were positive predictive value (in the event of a positive test the probability the patient is truly positive), negative predictive value (in the event of a negative test the probability the patient is truly negative), positive and negative likelihood ratios (respectively, the probability a person who has COVID-19 testing positive divided by the probability of a person who does not have COVID-19 testing positive [positive likelihood ratio] and the probability of a person who has COVID-19 testing negative divided by the probability of a person who does not have COVID-19 testing negative [negative likelihood ratio]) and discrepancy of findings between both investigations.

Search strategy

Three authors independently searched online databases, including MEDLINE, EMBASE, EMCARE, CINAHL and the Cochrane Central Register of Controlled Trials. The last search was run on 23 August 2020. Thesaurus headings, search operators and limits in each of the above databases were adapted accordingly. In addition, the WHO International Clinical Trials Registry, ClinicalTrials.gov, ISRCTN Register and medRxiv were also searched for ongoing and unpublished studies. The search terminologies included ‘COVID-19’, ‘Coronavirus’, ‘SARS-CoV-2’, ‘CT’, ‘computed tomography’, ‘PCR’ and ‘polymerase chain reaction’. The bibliographic lists of relevant articles were also reviewed.

Selection of studies

The title and abstract of articles identified from the literature searches were assessed. The full texts of relevant reports were retrieved and those articles that met the eligibility criteria of our review were selected.

Data extraction

An electronic data extraction spreadsheet was created in line with Cochrane’s data collection form for intervention reviews. Three authors cooperatively collected and recorded the results with any disagreements resolved via discussion.

Data synthesis

Data synthesis was conducted using the Review Manager 5.3 software and Microsoft Excel. The results are reported in forest plots with 95% Confidence Intervals and tables. A summary receiver operator characteristic curve was also generated. Given the heterogeneity of studies across a variety of characteristics, we elected to present medians and ranges across studies rather than provide single pooled statistics.

Subgroup analyses

Additional sub-group comparisons were performed to assess differences in results for China-based versus non-China and large versus small studies.

Methodological quality and risk of bias assessment

Three authors independently assessed the methodological quality and risk of bias of included studies using the Newcastle-Ottawa Scale.²⁵ It offers a maximum score of nine stars across three domains: selection, comparability and exposure. The overall rating of good, fair or poor quality was based on the Agency for Healthcare Research and Quality standards.²⁵

Results

Literature search results

Our search strategy retrieved 1630 studies. Thirteen studies met the eligibility criteria for inclusion following systematic review of abstracts and manuscripts as appropriate (Figure 1).

Figure 1.

PRISMA flow diagram. The PRISMA diagram details the search and selection processes applied during the overview. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Characteristics of included studies

The baseline characteristics of the included studies are summarised in Table 1. All included studies were comparative with CT scans being assessed using RT-PCR as the reference (‘gold’) standard.

Table 1.

Baseline characteristics of the included studies.

Study	Journal or preprint server, country	StudCG y design	Age [mean ± SD, median (range) years]	Sex (M:F)	Total sample
Ai et al.4	Radiology, China	Retrospective study	51 ± 15, NR	547:467	1014
Besutti et al.12	European Radiology, Italy	Prospective study	59 ± 15.8, NR	408:288	696
Caruso et al.13	Radiology, Italy	Prospective study	57 ± 17, NR	83:75	158
Çinkooğlu et al.14	Diagnostic and Interventional Radiology, Turkey	Retrospective study	48.7, NR	87:98	185
Falaschia et al.15	European Journal of Radiology, Italy	Retrospective study	62.4 ± 18.2, NR (16–100)	424:349	773
Fang et al.16	Radiology, China	Retrospective study	NR, 45	29:22	51
Gaia et al.17	La Radiologia Medica, Italy	Retrospective study	59 ± 17, NR	185:129	314
Gietema et al.18	medRxiv, Netherlands	Prospective study	NR, 66 (55–76)	113:80	193
Guillo et al.19	European Journal of Radiology, France	Retrospective study	59 ± 19, NR	119:95	214
He et al.20	Respiratory medicine, China	Retrospective study	COVID-19: NR, 52 (8–74) Non-COVID-19: NR, 37 (1–76)	49:35	82
Long et al.21	European Journal of Radiology, China	Retrospective study	44.8 ± 18.2, NR	46:41	87
Ma et al.22	BMC Medicine, China	Retrospective study	COVID-19: 2.5, NR (0.9–9.8) Non-COVID-19: NR	COVID-19: 54:34 Non-COVID-19: NR	158
Xie et al.23	Radiology, China	Retrospective study	NR	NR	167

NR: not reported.

Primary outcomes

Discriminatory power (sensitivity and specificity)

Sensitivity to detect COVID-19 cases was reported in 13 studies enrolling 4092 patients (Figure 2). The median sensitivity of CT for the identification of COVID-19 cases was 0.91 (range 0.82–0.98) as demonstrated in Table 2. The specificity of CT scanning in identifying COVID-19 was reported in 10 studies enrolling 3689 patients (Figure 2). The median specificity for chest CT was 0.775 (range 0.25–1.00) as shown in Table 2. The discriminatory power of CT scanning in identifying patients with COVID-19 from non-infected patients, based on area under the summary receiver operator characteristic curve, is shown in Figure 3.

Figure 2.

Forest plot of chest CT – sensitivity and specificity for the identification of COVID-19 cases. CT: computed tomography; COVID-19: coronavirus disease 2019.

Table 2.

The performance of CT scan for COVID-19 infection.

Study	TP	FP	FN	TN	Sensitivity [95% CI]	Specificity [95% CI]	Accuracy	PPV	NPV	PLR	NLR
Ai et al.4	580	308	21	105	0.97 [0.95, 0.98]	0.25 [0.21, 0.30]	0.68	0.65	0.83	1.29	0.14
Besutti et al.12	520	61	31	84	0.94 [0.92, 0.96]	0.58 [0.49, 0.66]	0.89	0.89	0.73	2.24	0.09
Caruso et al.13	60	42	2	54	0.97 [0.89, 1.00]	0.56 [0.46, 0.66]	0.72	0.59	0.96	2.21	0.06
Çinkooğlu et al.14	147	0a	38	0	0.79 [0.73, 0.85]	Not estimable	0.79	1.00	0	Not estimable	Not estimable
Falaschia et al.15	419	66	43	245	0.91 [0.88, 0.93]	0.79 [0.74, 0.83]	0.86	0.86	0.85	4.27	0.12
Fang et al.16	50	0a	1	0	0.98 [0.90, 1.00]	Not estimable	0.98	1.00	0	Not estimable	Not estimable
Gaia et al.17	147	24	15	128	0.91 [0.85, 0.95]	0.84 [0.77, 0.90]	0.87	0.86	0.89	5.75	0.11
Gietema et al.18	74	35	9	75	0.89 [0.80, 0.95]	0.68 [0.59, 0.77]	0.77	0.68	0.89	2.80	0.16
Guillo et al.19	113	0	24	77	0.82 [0.75, 0.88]	1.00 [0.95, 1.00]	0.89	1.00	0.76	Not estimable	0.17
He et al.20	26	2	8	46	0.76 [0.59, 0.89]	0.96 [0.86, 0.99]	0.88	0.93	0.85	18.35	0.25
Long et al.21	35	0a	1	51	0.97 [0.85, 1.00]	1.00 [0.93, 1.00]	0.99	1.00	0.98	Not estimable	0.03
Ma et al.22	43	26	7	82	0.86 [0.73, 0.94]	0.76 [0.67, 0.84]	0.79	0.62	0.92	3.57	0.18
Xie et al.23	160	0a	7	0	0.96 [0.92, 0.98]	Not estimable	0.96	1.00	0	Not estimable	Not estimable
Median (range)					0.91 (0.82–0.98)	0.775 (0.25–1.00)	0.87 (0.68–0.99)	0.89 (0.59–1.00)	0.85 (0–0.98)	3.185 (1.29–18.35)	0.13 (0.03–0.25)

TP: true positive; FP: false positive; FN: false negative; TN: true negative; CI: confidence interval; PPV: positive predictive value; NPV: negative predictive value; PLR: positive likelihood ratio; NLR: negative likelihood ratio.

^aAssumes no false positive due to non-reported data.

Figure 3.

Summary receiver operator characteristic (sROC) curves for the performance of chest CT in the initial diagnosis of COVID-19. CT: computed tomography. The x-axis also represents 1−specificity (in which case the figures would be from 0 to 1 reading from left to right).

Diagnostic effectiveness (accuracy)

Accuracy to detect and exclude COVID-19 cases by CT scan was reported in 13 studies enrolling 4092 patients with a median accuracy of 0.87 (range 0.68–0.99) as demonstrated in Table 2.

Secondary outcomes

Other test performance characteristics

As shown in Table 2, CT scanning had median positive and negative predictive values of 0.89 (range 0.59–1.00) and 0.85 (range 0.73–0.98), respectively. Additionally, the median positive and negative likelihood ratios for CT scan were 3.185 (range 1.29–18.35) and 0.13 (range 0.03–0.25) in turn.

Discrepancy of findings

Ai et al.⁴ (n = 1014) reported that 75% (n = 310) of patients with negative RT-PCR results and 97% (n = 583) of those with positive results had positive chest CT findings. Across all studies, 8.2% (207/2529) of PCR positive patients had a negative CT scan and 79.2% (42/53) of patients who initially had a negative PCR but then had a positive PCR later (data were provided from a minority of studies) had a CT scan suggestive of COVID-19 (Supplemental Material 1).

Subgroup analyses

China and non-China based studies

China-based studies had an accuracy range of 0.68 to 0.99 (median 0.92; 50% of studies had an accuracy of > 0.95) versus 0.72 to 0.89 (median 0.86; no studies had an accuracy of > 0.9) in non-China studies in the identification of COVID-19 (Supplemental Material 2).

Larger and smaller studies

There was also a difference between larger (n ≥ 100) and smaller (n < 100) studies in accuracy for the identification COVID-19 (Supplemental Material 3). Larger studies had an accuracy range of 0.68 to 0.96 (median 0.825; 30% of studies had an accuracy ≥ 0.88) versus 0.88 to 0.99 (median 0.98; all studies had an accuracy of ≥ 0.88) for smaller studies.

Methodological quality and risk of bias assessment

Although comparability was low in all the studies included, selection and exposure were of higher quality (Supplemental Material 4). Overall, all studies were of good quality based on the Agency for Healthcare Research and Quality standards.²⁵

Discussion

Our findings suggest that CT scanning performs relatively well compared to RT-PCR as the reference standard in the initial detection of COVID-19 in the clinical context of the studies presented. Relatively few false negatives (approximately 9%) can be expected with CT scanning in the context of a pandemic, although the expected false positive rate is higher at approximately 22.5%. The performance characteristics of all tests is context dependent, however, which means that the performance of CT for identifying COVID-19 will be different outside of a pandemic and in areas of the world with different endemicities. When the causes of bilateral CT changes return to the usual heterogeneous state, with SARS-CoV-2 only one of many potential causes, as was the situation pre-pandemic, the performance of CT will likely deteriorate with an even higher number of false positives, in keeping with the relatively modest median positive likelihood ratio for CT, even within a pandemic setting.⁴ For patients requiring hospitalisation with an acute cardio-respiratory presentation who undergo CT scanning, however, a negative CT scan is reassuring, particularly if RT-PCR is also negative in keeping with the low median negative likelihood ratio for CT. Our results cannot be extrapolated to healthier SARS-CoV-2 patient cohorts who are managed on an outpatient basis or in a community setting.

Diagnostic tests always have inherent strengths and weaknesses. In the case of the COVID-19 pandemic, it would be preferable to have a simple, rapid test with high sensitivity leading to a low number of false negatives thereby allowing accurate decision-making regarding, for example, who may be infectious and requires isolation in the hospital setting or home isolation in the community. Based on those who had a suggestive CT but tested negative with RT-PCR, our findings suggest that CT scan may be able to detect a high proportion of hospitalised COVID-19 cases overall.⁴ There were relatively few patients who had a negative CT scan, but who were initially RT-PCR positive. Of the minority of studies that reported patients who were initially RT-PCR negative, but subsequently became positive, 79.2% of patients (42/53) had a suggestive CT. It is important to note that across the studies, however, only a small proportion who were initially RT-PCR negative were systematically retested and this population is therefore inherently skewed, with the performance of CT scan in this context potentially being misrepresented by these data.

Overall, our results suggest that chest CT scan may be a useful adjunct to RT-PCR in the initial detection of COVID-19 in certain circumstances, but not routinely, which supports published guidance and thus we do not advocate a change in practice. CT scans are subject to their inherent limitations. First, they are best avoided during pregnancy due to the excessive radiation and risk of harming the foetus.²⁶ In addition, over-scanning patients with suspected COVID-19 can increase the risk of incidental imaging findings and an unnecessary financial burden on the healthcare system.²⁷ Furthermore, the routine use of CT scans for suspected cases is not practical since SARS-CoV-2 has a relatively high transmission rate; each CT machine needs to be sterilised after use, which can waste time and resources as well as overworking the radiology department in times of need resulting in delayed reporting of other potentially life-threatening conditions.

Given the above, and based on our results therefore, we suggest that chest CT is limited to a supplementary diagnostic role for COVID-19 in patients with suspicious symptoms, but who have had at least one (or preferably more) negative RT-PCR test results. In such situations, a repeat upper respiratory tract RT-PCR, which is simple to do, rather than a CT scan should be performed first, although in our clinical experience some patients with suspicious presentations are repeatedly negative or only positive on a third or fourth test. It is in such patients that CT scans may be useful to provide circumstantial evidence. Although lower respiratory tract specimens (i.e. sputum or bronchoalveolar lavage) are more likely to be positive in this patient cohort, many hospitalised patients are neither productive nor ventilated.

This review provides a summary of the best available evidence, from the first wave of the global pandemic, using a systematic approach and evaluated the risk of bias of included studies.^4,12-23 Although Kim et al. recently published a meta-analysis on this topic, many of the studies included were not comparative (i.e. the tests were not compared within the same patient cohorts).²⁸ Our study synthesised comparative studies only, and also included additional recently published studies and those in preprint, so our results may more accurately represent how the investigations can be expected to perform and compare to RT-PCR in routine clinical practice and as the pandemic progresses. Additionally, we also performed important sub-group analyses that demonstrate how these tests may compare in different contexts, including the fallacy of basing conclusions about a test’s performance early in a pandemic on small studies in one geographic location.

The reported results should be considered in the context of inherent limitations. Only 13 non-randomised studies were identified enrolling a total of 4092 participants, which may not be enough to draw definite conclusions and suggesting the need for further robust prospective diagnostic research in COVID-19. The performance of molecular tests for SARS-CoV-2 is likely to improve and be closer at the point-of-care with time. Likewise, the use of, for example, machine learning algorithms to interpret chest CT scan appearances may more accurately confirm COVID-19 than human reporting in the future, and thereby decrease the false positive rate of CT scanning that can be expected to increase as the pandemic wanes. Some included studies had incomplete data resulting in difficulty in comparing both interventions completely. Additionally, all but one study included a relatively small number of patients and the majority were of retrospective observational design, with all the associated caveats. Although median values derived for sensitivity and specificity summarise the findings of the studies identified, they do not account for the differing study sizes; readers should interpret the results carefully and reflect on how they apply usefully to the context of their clinical practice. It is likely that further large comparative, and possibly prospective, studies will be reported with further meta-analyses required. The between-study heterogeneity for outcomes was high for sensitivity and accuracy.

Conclusions

Our study synthesised the best available comparative evidence, using a systematic approach, for the identification of COVID-19 using chest CT versus RT-PCR as the reference standard.^4,12–23 Our results suggest that chest CT scan may be a useful adjunct in the initial detection of COVID-19 in certain hospitalised patients in the context of a pandemic. We suggest RT-PCR remains the primary diagnostic tool, however, and that chest CT is only considered if there is a strong clinical suspicion of COVID-19 with repeatedly negative RT-PCR test results, providing infection prevention and control measures can be maintained. A negative CT scan is likely to be reassuring in a RT-PCR negative patient requiring hospitalisation for possible COVID-19.

Highlights

• The median accuracy of chest CT for the diagnosis of coronavirus disease 2019 (COVID-19) is relatively high at 0.87 (range 0.68–0.99).

• Chest CT has relatively low specificity, even within the context of a pandemic, for the diagnosis of COVID-19 with an associated relatively high false positive rate.

• Chest CT scan, however, appears to be able to detect most initially positive and most initially negative/subsequently positive RT-PCR diagnosed cases, although more evidence is required.