How accurate are radiography and computed tomography in the diagnosis of COVID-19?—A Bayesian approach

Abstract

Background

The role of radiology in patients with clinical suspicion of COVID-19 is evolving with scientific evidence, but there are differences in opinion on when and how the technique should be used for clinical diagnosis.

Purpose

To estimate the pre-test and post-test probability that a patient has COVID-19 in the event of a positive and/or negative result from chest X-ray and chest computed tomography (CT) radiological studies, comparing with those of real time polymerase chain reaction (RT-PCR) tests.

Methods

The literature on the sensitivity and specificity of the chest X-ray, chest CT, and RT-PCR was reviewed. Based on these reported data, the likelihood ratios (LR) were estimated and the pre-test probabilities were related to the post-test probabilities after positive or negative results.

Results

The chest X-ray has only a confirmatory value in cases of high suspicion. Chest CT analyses showed that when it is used as a general study, it has almost confirmatory value under high clinical suspicion. A chest CT classified with CO-RADS ≥ 4 has almost a diagnostic certainty of COVID-19 even with moderate or low clinical presumptions, and the CO-RADS 5 classification is almost pathognomonic before any clinical presumption. To rule out COVID-19 completely is only possible in very low clinical assumptions with negative RT-PCR and/or CT.

Conclusions

Chest X-ray and especially CT are fast studies that have the capacity to report high probability of COVID-19, being a real contribution to the concept of “probable case” and allowing support to be installed in an early and timely manner.

Introduction

The world has experimented the most important health challenge in recent times since December 2019, with the COVID-19 pandemic that has already affected more than 600 million people and produced more than 6 million deaths.¹ From the experience of countries with large-scale community transmission, it has been learned that COVID-19 requires an unprecedented mobilization of health systems.^2–5 The demand on health care systems has been stressed, especially intensive care units for large numbers of people who become ill simultaneously. The ability of the health care system to respond will depend on available resources, including hospital beds and intensive care units. It also requires personnel trained in the rapid and timely diagnosis and treatment of COVID-19, where imaging plays a fundamental role.

In the daily experience of radiology, we are faced with the diagnosis of SARS CoV-2 pneumonia with plain radiographs (chest X-ray) or computed tomography (chest CT). Although the Gold Standard for the diagnosis of COVID-19 is the real time polymerase chain reaction (RT-PCR), this may be unknown at the time of image analysis.⁶ The sensitivity of RT-PCR is not very high either, 42–83%,⁷ which makes many patients with a negative result (PCR (−)) for COVID-19 consult. The role of chest CT in patients with clinical suspicion of COVID-19 is evolving with scientific evidence, but there are substantial differences in opinion on when and how the technique should be used for clinical diagnosis.³ While the American College of Radiology only recommends the use of CT to solve problems, the Fleishner society assigns it a role as an important tool to use if symptoms worsen or in a resource-limited setting for RT-PCR.^8,9

The Expert Consensus Statement on Reporting of the Radiological Society of North America (RSNA)¹⁰ has proposed a standardized classification for CT: (a) typical appearance (Cov19Typ): characterized by peripheral, bilateral, ground glass opacities (GGO) with or without consolidation or intralobular lines (crazy paving), multifocal GGO and/or findings of organizing pneumonia; (b) indeterminate appearance (Cov19Ind): characterized by absence of typical pattern but with multifocal, diffuse, perihilar, unilateral or few very small GGO; (c) atypical appearance (Cov19Aty) characterized by signs of other pneumonias; and (d) negative for pneumonia (Cov19Neg). The British Society of Thoracic Imaging (BSTI) has also proposed a classification for Rx that includes the categories: (a) classic/probable, with peripheral GGO, (b) indeterminate due to central/basal consolidation or poor quality film, (c) non-pneumonic findings, and (d) normal.¹¹ The Dutch Radiological Society developed a standardized assessment scheme for pulmonary involvement of COVID-19 in CT that would make it possible to compare data across populations. The authors chose the term CO-RADS, the COVID-19 Reporting and Data System. These authors analyzed 105 randomly selected Chest-CT obtained in a group of consecutive patients who presented to an emergency department, all with RT-PCR results. CT images were extracted from the picture archive and communication system, anonymized and analyzed by eight observers, who assigned CO-RADS scores without clinical or PCR information. They established a very rigorous classification of the levels of suspicion of pulmonary involvement of COVID-19, with the following categories: 0, not interpretable; 1, very low; 2, low; 3, equivocal/unsure; 4, high; 5, very high; and 6, proven by RT-PCR. Each of these categories was defined very precisely, and they related each category to the proportion of cases with RT-PCR positive, which makes this system ideal for use in quantitative/comparative studies.⁸

In clinical practice, including radiology, the diagnostic process follows a Bayesian inference structure, increasing the probability of a test-based diagnosis (i.e., chest X-ray, chest CT, RT-PCR, etc.) and clinical presumptions (pre-test probabilities), which in turn are based on signs and symptoms reported by the patient or observed by the doctor.^12–14 The classical form of the Bayes’s theorem can be presented as follows. If we have a set of events {B_i} (set of possible diagnosis), mutually exclusive, and an event A (a medical image), we can obtain the probability of B_i given A from the following expression

$$P\left(B_i/A\right)=\frac{P\left(A/B_i\right)\cdot P\left(B_i\right)}{P\left(A\right)}$$

Being P ($B_i)$ and $P\left(B_i/A\right)$ a priori and a posteriori probabilities of B_i, P(B_i) corresponds to the probability of B_i without knowing if A has occurred or not, and $P\left(B_i/A\right)$ is the probability of B_i when A has occurred. In other words, knowing the a priori probability P(B_i) and the likelihood function of {B_i} (P(A/B_i)), we can update the probability of B_i when the event A occurred. An usual expression of this theorem in evidence based medicine is $O=LR\hspace{0.17em}{O}_0$, where O and O₀ are a posteriori and a priori odds of B_i and LR is the likelihood ratio.¹²

Although this procedure does not involve those calculations when a diagnosis is proposed in clinical practice, the method is based on estimates of sensitivity, specificity and likelihood ratios that allow a quantitative approximation of the usefulness of the tests.^12,14 It is important to note that the clinical presumption is influenced by the epidemiological context of the patient who consults.¹⁴

Based on current knowledge, when we report a chest CT or a chest X-ray we ask ourselves how likely it is that the patient has COVID-19 when we report a positive result, and conversely, how likely it is that they do not have COVID-19 when we report negative test results. The aim of this paper is using Bayesian approach, to relate the clinical presumptions (pre-test probabilities) to the post-test probabilities of having COVID-19 with different X-ray and CT results, comparing with those of RT-PCR.

Material and methods

Information updated to 2021 was searched in the PubMed, Embase, and Web of Science databases using computed tomography (CT), plain radiograph, COVID-19, SARS CoV-2, CO-RADS, COVID RADS, sensitivity (S) and specificity (Sp) as keywords. Twenty-one studies were selected, among which there were three systematic reviews and meta-analyses. Studies that contributed original numbers which directly or indirectly provided enough information to extract 2 × 2 table information of diagnostic tests with sample sizes greater than 30 were selected. For RT-PCR, 8 studies were selected (see Table 1).Studies with data that did not allow calculations of the parameters required in this study were excluded. There were six studies for chest X-ray, but only the study with the largest sample size¹⁵ met the selection criteria; it was used for the S and Sp values of COVID-19 classic/probable pattern in chest X-ray. Ten studies were used for S and Sp of the CT, including an article using the CO-RADS classification for calculation purposes.⁸ The pooled S and Sp of chest CT and RT-PCR was calculated from these data as the average of the sensitivities reported or calculated directly from the data of each article. The positive and negative likelihood ratios (LR (+) and LR (−)) were then calculated with¹²

$$LR(+)=\frac{S}{1-Sp}\hspace{0.17em}\mathrm{a}\mathrm{n}\mathrm{d}\hspace{0.17em}LR(-)=\frac{1-S}{Sp}$$

Table 1.

Studies included in our analyses, excluding systematic reviews and meta-analyses.

	Number of studies analyzed	Number of studies selected	References
RT-PCR	9	8	(22, 23, 26–31)
Chest computed tomography	10	10	(15, 26–34)
Chest X-ray	6	1	(15)
Total	18	12

For different values of clinical presumption of having COVID-19 (pre-test probabilities, p₀), the a priori odds were calculated by $O_0=\frac{p_0}{1-p_0}$ and then the a posteriori odds (O (+) and O (−)) for a positive and a negative test with $O(+)=LR(+)O_0$ and $O(-)=LR(-)O_0$. With the O (+) we calculated the post-test probabilities $p(+)=\frac{O(+)}{O(+)+1}$ and $p(-)=\frac{O(-)}{O(-)+1}$.¹² A table was obtained for each technique with its respective graph between the clinical assumptions (pre-test probabilities) and the post-test probabilities of having COVID-19 in the event of positive and negative results.

Results

Considering RT-PCR as the Gold Standard, S, Sp, LR (+) and LR (−) were estimated for chest-CT reported with CO-RADS 5 and CO-RADS ≥ 4 (Table 2). The reported chest X-ray sensitivity and specificity with a positive result are low,⁷ helping little in the diagnosis of COVID-19, agreeing with the five other studies analyzed.^16–20 LR (+) = 1.4 was low, indicating that with a positive chest Rx for COVID-19 there is only a small increase in the chance of having it. In contrast, CT without use of CO-RADS score had high S and moderate Sp but high LR (+), increasing by more than four times the chance of having COVID-19 with a positive result. As expected, a high S and the highest Sp were found in RT-PCR, increasing the chance of having COVID-19 with a positive result by more than 32 times and also decreasing 0.16 by times the chance of having COVID-19 with a negative result. It was interesting that a CO-RADS 5 positive result had the highest Sp, increasing the chance of having COVID-19 with a positive result by 203 times, but with low S. A CO-RADS score ≥ 4 had similar S, Sp and LRs to RT-PCR.

Table 2.

Sensitivity (S), Specificity (Sp), positive and negative likelihood ratios (LR (+) and LR (−), respectively) for RT-PCR, chest Rx and chest computed tomography, without using (General) and using the CO-RADS system.

Modality	S	Sp	LR (+)	LR (−)
RT-PCR	0.846	0.974	32.538	0.158
Chest X-Ray*	0.560	0.600	1.400	0.733
Chest computed tomography general	0.905	0.797	4.451	0.119
CO-RADS ≥ 4	0.810	0.981	42.632	0.194
CO-RADS 5	0.610	0.997	203.333	0.391

LR: likelihood ratios.

^afrom reference 7.

A positive result of CT with high clinical presumption of COVID-19 (p₀ ≥ 0.8) had confirmatory value (p (+)> 0.95); the Gold Standard RT-PCR would have confirmatory value even with low clinical presumptions (p₀ ≥ 0.37) (Figure 1(a)). Categorizing a CT with CO-RADS ≥ 4 confirms the diagnosis of COVID-19 (p (+) > 0.95) with clinical presumption p₀ ≥ 0.31 similar to RT-PCR. Interestingly, the typical image for COVID-19 pneumonia (CO-RADS 5) would have confirmatory value for any level of clinical presumption (Figure 1(b)).

Figure 1.

Variation of post-test probabilities of chest X-ray, chest computed tomography and RT PCR (as reference), for several pre-test probabilities to have COVID-19 when the result is positive (a), for CO-RADS ≥ 4 and CO-RADS 5 categories (b), and for a negative result (c). Horizontal green and black lines represent post-test probabilities 0.95 and 0.05, respectively.

In the event of a negative result, chest X-ray does not allow ruling out COVID-19 (i.e., when p (−) <0.05) in any case, and RT-PCR and CT can only do so with very low clinical assumptions, p₀ ≥ 0.24 and 0.33, respectively (Figure 1(c)).

Discussion

All the literature reviewed reported moderate to high sensitivity and high specificity of RT-PCR,^{7,8–11,21–25} which leads to values of LR (+) = 40.0, implying that a positive result increases by 40 times the odds of having COVID-19, so this test has a Gold Standard value. The chest X-ray, on the other hand, has poor S and Sp,^15,26 so it has only a confirmatory value in cases of high suspicion of COVID-19. Although the values presented here for chest X-ray come from a single study, this has a sample size (n = 1198), one order of magnitude larger than the other analyzed studies and also the estimations of S are similar to those studies. For example, S = 0.44,¹⁷ S = 0.591,¹⁸ S = 0.59,²⁰ S = 0.25,¹⁹ and S = 0.67.¹⁶ Some of these studies reported high Sp, but values of Sp = 1¹⁷ and Sp = 0.9¹⁹ may be a consequence of low sample sizes (n = 40–50). A chest X-ray can be useful as a confirmatory test when the clinical symptomatology is very clear and RT-PCR or chest CT are not available.¹⁵ It must also be considered that many hospitals in the world are employing chest X-ray as the first-line method, with faster results than with RT-PCR and chest-CT, especially using portable X-ray units. This test reduces the movement of patients and minimizes the risk of cross-infection considering that these patients need to be isolated.¹⁶

This analysis shows interesting aspects with respect to the chest-CT. When CT is used as a general study without using the CO-RADS classification, it has almost confirmatory value (p (+)> 0.95) only when the clinical presumption is high (i.e., p₀ > 0.8). Consistent with other reports, these results suggest that CT is a real contribution to screening for COVID-19 in patients with clinical and epidemiologic features compatible with COVID-19 infection, particularly in patients without RT-PCR or when results of RT-PCR tests are negative, and thus rapidly initiate treatment.^7,16 The use of CO-RADS classification allows systematizing the diagnosis of COVID-19, agreeing with other studies.⁸ The sensitivity of the CO-RADS classification test decreases for high CO-RADS categories but its specificity increases, being very high for COVID-19 when an image is classified as CO-RADS 5. A chest CT classified with CO-RADS ≥ 4 allows almost certain diagnosis of COVID-19 even with moderate or low clinical presumptions (p₀ ≥ 0.3) and the CO-RADS 5 category is almost pathognomonic before any clinical presumption (Figure 2). None of the analyzed studies (RT-PCR, chest X-ray and chest-CT) can be used to rule out COVID-19 completely, this being possible only at very low clinical assumptions with negative RT-PCR and/or CT.

Figure 2.

Images that can be found in a typical appearance of SARS CoV 2 pneumonia CO-RADS 5. (a) Chest X-ray with multiple peripheral patchy opacities in both lungs; (b) Chest computed tomography showing ground glass opacities mixed with areas of crazy paving; (c) Atoll sign probably indicating organized pneumonia with higher peripheral density and lower central density; (d) peripheral ground glass opacities and atoll sign probably indicating organizing pneumonia; (e) Extensive involvement of both lungs, with predominantly peripheral ground glass parenchymal opacities; (f) bulls-eye sign probably indicating organizing pneumonia; (g and h) Reticular areas of peripheral sub-cortical fibrosis.

This study has some limitations. It is based mainly on average estimates based on previous studies that allowed estimating likelihood ratios and probabilities a posteriori, therefore being dependent on the type and size of the samples of these previous studies. It does not consider intra- or inter-observer variability when analyzing radiological images or the intrinsic variability of the sensitivity and specificity estimates. However, it has the virtue of showing quickly and easily the relevance and contribution that radiological exams are making in clinical practice to the diagnostic support of COVID-19.

In conclusion, Chest X-ray and CT are very fast studies and have the capacity to report high probability of COVID-19, being a real contribution to the concept of “probable case” and allowing support to be installed in an early and timely manner.

ORCID iD

Mauricio Canals https://orcid.org/0000-0001-5256-4439