Accuracy and Reproducibility of Low-Dose Submillisievert Chest CT for the Diagnosis of COVID-19

Abstract

Purpose

To demonstrate the accuracy and reproducibility of low-dose submillisievert chest CT for the diagnosis of coronavirus disease 2019 (COVID-19) infection in patients in the emergency department.

Materials and Methods

This was a Health Insurance Portability and Accountability Act–compliant, institutional review board–approved retrospective study. From March 14 to 24, 2020, 192 patients in the emergency department with symptoms suggestive of COVID-19 infection were studied by using low-dose chest CT and real-time reverse transcription polymerase chain reaction (RT-PCR). Image analysis included the likelihood of COVID-19 infection and the semiquantitative extent of lung involvement. CT images were analyzed by two radiologists blinded to the RT-PCR results. Reproducibility was assessed using the McNemar test and intraclass correlation coefficient. Time between CT acquisition and report was measured.

Results

When compared with RT-PCR, low-dose submillisievert chest CT demonstrated excellent sensitivity, specificity, positive predictive value, negative predictive value, and accuracy for diagnosis of COVID-19 (86.7%, 93.6%, 91.1%, 90.3%, and 90.2%, respectively), in particular in patients with clinical symptoms for more than 48 hours (95.6%, 93.2%, 91.5%, 96.5%, and 94.4%, respectively). In patients with a positive CT result, the likelihood of disease increased from 43.2% (pretest probability) to 91.1% or 91.4% (posttest probability), while in patients with a negative CT result, the likelihood of disease decreased to 9.6% or 3.7% for all patients or those with clinical symptoms for >48 hours. The prevalence of alternative diagnoses based on chest CT in patients without COVID-19 infection was 17.6%. The mean effective radiation dose was 0.56 mSv ± 0.25 (standard deviation). Median time between CT acquisition and report was 25 minutes (interquartile range: 13–49 minutes). Intra- and interreader reproducibility of CT was excellent (all intraclass correlation coefficients ≥ 0.95) without significant bias in the Bland-Altman analysis.

Conclusion

Low-dose submillisievert chest CT allows for rapid, accurate, and reproducible assessment of COVID-19 infection in patients in the emergency department, in particular in patients with symptoms lasting longer than 48 hours. Chest CT has the additional advantage of offering alternative diagnoses in a significant subset of patients.

© RSNA, 2020

Summary

Low-dose submillisievert chest CT allows for rapid, accurate, and reproducible assessment of COVID-19 infection in patients in the emergency room and has the additional advantage of offering alternative diagnoses in a significant subset of patients.

Key Points

• ■ When compared with reverse-transcription polymerase chain reaction, low-dose submillisievert chest CT demonstrated excellent sensitivity, specificity, positive predictive value, negative predictive value, and accuracy for the diagnosis of COVID-19 in patients in the emergency room (86.7%, 93.6%, 91.1%, 90.3%, and 90.2%, respectively), in particular in patients with clinical symptoms for >48 hours (95.6%, 93.2%, 91.5%, 96.5%, and 94.4%, respectively).

• ■ In patients with a positive CT result, the likelihood of disease increased from 43.2% (pretest probability) to 91.1% or 91.4% (posttest probability), while in patients with a negative CT result, the likelihood of disease decreased to 9.6% or 3.7% for all patients or those with clinical symptoms for >48 hours.

• ■ CT results were rapidly available with median time between CT scan acquisition and report of 25 minutes (interquartile range: 13–49 minutes).

Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is a novel enveloped RNA betacoronavirus belonging to the same family of viruses causing severe acute respiratory syndrome and Middle East respiratory syndrome (1). First described in December 2019 in Wuhan, the capital city of Hubei province, China, the disease was named coronavirus disease 2019 (COVID-19) (2). The virus has spread rapidly across the globe, and COVID-19 was declared a pandemic by the World Health Organization on March 11, 2020 (3). Thus far, there are no proven treatments for COVID-19, and current management of the pandemic mainly depends on limiting the transmission of the disease by early detection and isolation of patients. Reverse-transcription polymerase chain reaction (RT-PCR) test represents the standard of reference for the diagnosis of COVID-19 with very high specificity and can be easily obtained by throat swab sampling. The sensitivity of RT-PCR has been reported to be around 70% in initial reports, which is rather low for screening (4). One negative RT-PCR test therefore does not exclude COVID-19, and multiple repeat tests may be required to make the final diagnosis, leading to multiple days of uncertainty for both patients and health care professionals.

Chest CT may represent an additional tool for the initial assessment of patients with possible COVID-19 infection. COVID-19 pneumonia causes bilateral, peripheral, and basal predominant areas of ground-glass opacities, typically evolving to consolidation at later stages of the disease (5,6). A previous study has demonstrated excellent sensitivity (97%) but poor specificity (25%) of chest CT for the diagnosis of COVID-19 (7). These results suggest that a complementary approach combining the high sensitivity of chest CT with the high specificity of RT-PCR may be effective for the early diagnosis of COVID-19. Given the very common and rapidly increasing clinical indication of chest CT for COVID-19 diagnosis, accurate and reproducible assessment with a low-radiation dose (submillisievert) protocol may result in major radiation dose reductions on a population level (8). This study aimed to investigate whether submillisievert chest CT could be used to rapidly, accurately, and reproducibly stratify patients with a possible COVID-19 infection.

Materials and Methods

This retrospective study was compliant with the Health Insurance Portability and Accountability Act and was approved by our institutional review board (Imelda General Hospital, Bonheiden, Belgium). Informed consent was waived. From March 14 to 24, 2020, 192 consecutive patients with possible COVID-19 infection and both RT-PCR test and low-dose chest CT at presentation were included. The RT-PCR results were obtained from the patient electronic medical record. Patients who showed negative results at RT-PCR testing underwent repeat RT-PCR examination the following day. If this second RT-PCR test was positive, the patient was considered to be COVID-19 positive. Two polymerase chain reaction methods were used to detect SARS-CoV-2 in nasopharyngeal swabs (eSwab; Copan Diagnostics, Brescia, Italy), both using the E-gene as the target. Primers and probe sequences for the E-gene were provided by the Belgian National Reference Center (University Hospital Leuven, Leuven, Belgium). The first platform, the ARIES system (Luminex, Austin, Tex), provides an open channel for laboratory-developed tests. It is an all-in-one system with extraction, purification, amplification, and detection in one cassette. Luminex SYNCT software is used to analyze and interpret polymerase chain reaction results. It can run up to 12 samples simultaneously with a hands-on time of 10 minutes and a turnaround time of 1 hour and 45 minutes. The second platform, the Rotor-Gene Q (Qiagen, Hilden, Germany), is a real-time polymerase chain reaction instrument. Extraction is performed on NucliSens easyMAG (BioMérieux, Marcy l’Étoile, France). Rotor-Gene Q Series software is used to interpret the data. It is a higher volume, batch test, with a capacity of up to 72 samples simultaneously, but time from sample collection to results is at least 2 hours and 45 minutes (because of a longer hands-on time and separate extraction process). The sensitivity of the SARS-CoV-2 test on ARIES was analyzed by testing a dilution series of an inactivated culture of SARS-CoV-2 and was similar to the assay of the Belgian National Reference Center. The sensitivity on Rotor-Gene Q was validated in relation to ARIES analyzing samples on both platforms. No cross reactivity has been shown for other human coronaviruses, influenza virus, or respiratory syncytial virus.

CT Scan Protocol

All patients underwent low-dose chest CT by using a SOMATOM Definition AS 64-slice 0.6-mm detector scanner (Siemens Healthineers, Forchheim, Germany). We used the vendor-supplied software (Care Dose 4D; Siemens Healthineers) to calculate size-specific radiation dose estimates for a low-dose chest CT protocol adapted from the protocol used for lung cancer screening (9). Reference values in a so-called average patient were set to 100 kVp and 20 mAs with a pitch of 1.2 and 0.5-second gantry rotation time. A relatively high pitch was used to limit motion artifacts in dyspneic patients with COVID-19. Effective radiation dose was calculated by multiplying the dose-length product by 0.014 mSv/mGy · cm as the constant k-value for thoracic imaging (10). Images were reconstructed at 1-mm slice thickness and 0.7-mm increment with a standard lung-tissue kernel (I50f medium sharp) and at 3-mm slice thickness and 3-mm increment with a standard soft-tissue kernel (I31f medium smooth) using sinogram-affirmed iterative reconstruction strength of 3. All reconstructions were performed with a field of view of 450 mm and a matrix size of 512 × 512 pixels.

CT Image Analysis

All chest CT images were scored as suggestive for or inconsistent with COVID-19 infection based on the presence of findings as presented by Ng et al and Shi et al (11,12). In summary, CT findings suggestive of COVID-19 infection were multiple ground-glass opacities, bilateral/multifocal involvement, peripheral distribution, and, at a later stage, crazy paving, consolidation, and reversed halo sign. CT findings inconsistent with COVID-19 infection were tree-in-bud opacities, centrilobular/peribronchovascular distribution, cavitation, and pleural effusion. A semiquantitative scoring system was used to estimate the extent of pulmonary involvement as reported previously (5,13). Each lobe was scored from 0 to 5 with a total score ranging from 0 to 25: score 0, 0% involvement; score 1, <5% involvement; score 2, 5%–25% involvement; score 3, 26%–50% involvement; score 4, 51%–75% involvement; and score 5, 76%–100% involvement. Two cardiothoracic radiologists (C.G. and R.S., with 8 and 7 years of cardiothoracic imaging experience, respectively) scored the CT scans by consensus and were blinded to the RT-PCR results. One reader (R.S.) re-read a random sample of 50 scans after 1 week to assess intrareader reproducibility. These cases were reread by the other reader (C.G.) to assess interreader reproducibility.

Statistical Analysis

All statistical analyses were performed by using R v.3.5.3. (Foundation for Statistical Computing, Vienna, Austria). By using RT-PCR as standard of reference, the sensitivity, specificity, positive predictive value, negative predictive value, accuracy, positive likelihood ratio, and negative likelihood ratio of low-dose chest CT were calculated with 95% confidence interval (CI) by using the reportROC and UncertainInterval packages. Fagan nomogram (an integration of Bayes theorem) was used to quantify the posttest probability of COVID-19 infection given the CT results and the pretest probability (14). Data were tested for normal distribution with the Shapiro–Wilk test. Summary statistics for all continuous variables are reported as mean ± standard deviation or as median with interquartile range, as appropriate. The Student t test for independent samples and the Mann–Whitney U test were used to compare continuous variables between groups. P < .05 was considered to indicate a statistically significant difference. Intrareader and interreader agreements were assessed by using the McNemar test, intraclass correlation coefficients, and Bland-Altman analysis with 95% limits of agreement (15). Intraclass correlation coefficients of >0.75 and of 0.4–0.75 indicate strong and average agreements, respectively. A difference between intraclass correlation coefficients was considered to be statistically significant when there was no overlap between their respective 95% CI limits. No missing data were present. No patients were excluded from analysis after initial inclusion. No adverse events occurred from chest CT examinations.

Results

The demographic data of the patients and dose parameters of the low-dose CT scan are summarized in Table 1. There were 83 (43.2%) patients positive for COVID-19, and 109 (56.8%) were negative. The mean age of all patients was 62 years ± 18 (standard deviation), reflecting the wide range of ages susceptible to COVID-19 infection. COVID-19–positive patients were significantly older than COVID-19–negative patients (67.4 years ± 16.8 vs 57.5 years ± 18.1, P < .001) and more likely to present with fever (68.7% vs 45.9%, P = .003). Mean dose-length product of all patients was 39.9 mGy · cm ± 17.8, resulting in an effective radiation dose of 0.56 mSv ± 0.25. Median time between CT scan acquisition and report was 25 minutes (interquartile range: 13–49 minutes).

Table 1:

Patient Characteristics and Radiation Dose Parameters

Accuracy and Reproducibility of Chest CT for COVID-19 Diagnosis

When compared with RT-PCR, low-dose submillisievert chest CT demonstrated excellent sensitivity, specificity, positive predictive value, negative predictive value, and accuracy for the diagnosis of COVID-19 in patients in the emergency department (86.7%, 93.6%, 91.1%, 90.3%, and 90.2%, respectively), in particular, in patients with clinical symptoms for more than 48 hours (95.6%, 93.2%, 91.5%, 96.5%, and 94.4%, respectively) (Table 2) (Fig 1). Positive likelihood ratio and negative likelihood ratio were 13.51 (95% CI: 6.57, 27.79) and 0.14 (95% CI: 0.08, 0.25) for all patients and 14.02 (95% CI: 6.47, 30.40) and 0.05 (95% CI: 0.02, 0.14) for patients with clinical symptoms for more than 48 hours, respectively. In patients with a positive CT result, the likelihood of disease increased from 43.2% (pretest probability) to 91.1% or 91.4% (posttest probability) for all patients or those with clinical symptoms for more than 48 hours. In patients with a negative CT result, the likelihood of disease decreased from 43.2% (pretest probability) to 9.6% or 3.7% (posttest probability) for all patients or those with clinical symptoms for more than 48 hours (Fig 2).

Table 2:

Accuracy of Low-Dose Chest CT for Diagnosis of COVID-19 Infection

Figure 1:

Example CT images in two patients with COVID-19. A, Axial and, B, coronal CT images in an 85-year-old woman presenting with dyspnea and fever for 3 days. CT images show typical early COVID-19 findings with bilateral subpleural areas of ground-glass opacities (arrows). Total CT involvement score was 7/25. Effective radiation dose was 0.52 mSv. C, Axial and, D, coronal CT images in a 41-year-old woman presenting with cough and fever for 14 days. CT images show typical late COVID-19 findings with multifocal bilateral subpleural areas of consolidation (arrows). Total CT involvement score was 15/25. Effective radiation dose was 0.53 mSv. Window center, −600 HU; window width, 1600 HU; slice thickness, 1 mm; and increment, 0.7 mm for all images.

Figure 2:

Fagan nomogram for, A, all patients and, B, patients with clinical symptoms for more than 48 hours illustrates pretest and posttest probabilities based on the positive likelihood ratio and negative likelihood ratio of chest CT in patients with possible COVID-19 infection.

Intra- and interreader reproducibility for CT scores of lung involvement was excellent (intraclass correlation coefficient: 0.968 and 0.954; 95% CI: 0.942, 0.983 and 0.918, 0.974, respectively) without significant bias in the Bland-Altman analysis (0.2 and −0.5, 95% limits of agreement: −2.0, 2.5 and −3.0, 1.9, respectively).

Additional Information from Chest CT in Patients with Possible COVID-19 Infection

In patients with CT findings suggestive of COVID-19 infection and positive RT-PCR results (true-positive results, n = 72), CT offered additional information by suggesting the presence of associated bacterial pneumonia in seven patients (9.7%). In patients with findings inconsistent with COVID-19 infection and negative RT-PCR results (true-negative results, n = 102), CT offered additional information by suggesting the presence of an alternative diagnosis in 18 patients (17.6%) (Table 3) (Fig 3). Bacterial pneumonia (n = 10) was confirmed by a combination of sputum cultures, hemocultures, and laboratory tests including RT-PCR. Lung cancer (n = 3) was confirmed with biopsy to be adenocarcinoma of the lung in two patients. No biopsy was performed in the last patient because of comorbidities and patient refusal. Acute Epstein–Barr virus infection (n = 2) was confirmed by serologic studies. Pericarditis (n = 1) was confirmed by the presence of a pericardial effusion in combination with typical electrocardiographic changes (16). Human metapneumovirus (n = 1) was confirmed with RT-PCR. CT-suggested diagnosis of sarcoidosis (n = 1) was not yet confirmed at the time of submission as the patient refused biopsy. In patients with CT findings suggestive of COVID-19 infection but negative RT-PCR results (false-positive results, n = 7), another viral cause of pneumonia was identified in two patients with RT-PCR (human metapneumovirus).

Table 3:

Role of Chest CT in Making Additional or Alternative Diagnoses in True-Positive and True-Negative Patients, Respectively

Figure 3:

Example alternative diagnoses suggested by chest CT in patients with possible COVID-19 infection. A, Axial and, B, coronal CT images in a 59-year-old woman presenting with cough and fever for 7 days. CT images show lobar consolidation of the anterior segments of the left lower lobe with surrounding ground-glass opacities compatible with bacterial lobar pneumonia (arrows). Sputum culture confirmed Streptococcus pneumoniae pneumonia. C, Axial and, D, coronal CT images in a 53-year-old man presenting with cough and dyspnea for 3 weeks. CT images show an irregular mass in the right upper lobe with associated lymphangitic carcinomatosis and multiple hypodense liver lesions compatible with lung cancer with liver metastases (arrows). Biopsy results confirmed adenocarcinoma of the lung with liver metastases. E, F, Axial CT images in an 18-year-old woman presenting with cough and fever for 7 days. CT images show bilateral axillary lymphadenopathy and splenomegaly compatible with Epstein–Barr virus infection, which was confirmed by Epstein–Barr virus immunoglobulin M antibody detection (arrows). A–C, Window center, −600 HU; window width, 1600 HU; slice thickness, 1 mm; and increment, 0.7 mm. D–F, Window center, 40 HU; window width, 400 HU; slice thickness, 3 mm; and increment, 3 mm.

Discussion

Low-dose chest CT may play a complementary role to RT-PCR testing for the early triage of patients with possible COVID-19 infection. When compared with RT-PCR, chest CT achieved high sensitivity, specificity, positive predictive value, negative predictive value, and accuracy (86.7%, 93.6%, 91.1%, 90.3%, and 90.2%, respectively), with further improvement when only patients with symptoms lasting longer than 48 hours were included (95.6%, 93.2%, 91.5%, 96.5%, and 94.4%, respectively). Importantly, these results were obtained by using a low-dose CT protocol with submillisievert effective radiation dose (0.56 mSv ± 0.25) and rapid availability of results (median: 25 minutes, interquartile range: 13–49 minutes).

When evaluating any diagnostic test, it is important to understand how the test result affects the probability of the disease in question being present (17). In this regard, likelihood ratios and posttest probabilities are more informative than sensitivity and specificity analysis. At this moment of the COVID-19 pandemic, the pretest probability of COVID-19 infection in patients presenting at the emergency department with appropriate clinical symptoms was very high (43.2%). However, within a short time (median interval between CT scan and report <30 minutes), chest CT increased this probability to >90% in those patients with CT findings suggestive of COVID-19 infection, while the posttest probability was only 4%–10% for those patients with CT findings inconsistent with COVID-19 infection. This rapid and reliable differentiation with CT can play an important role in effective triage of patients with the possibility of having COVID-19, in particular during times of the emergency department overflowing in a COVID-19 epicenter with a high pretest probability.

When compared with the largest study to date comparing chest CT and RT-PCR by Ai et al (7), our results indicate a significantly higher specificity of CT for the diagnosis of COVID-19. A possible explanation for this discrepancy, already mentioned by the authors in their study limitations, is their use of RT-PCR assays with a low positive rate, likely resulting in an overestimation of CT sensitivity but an underestimation of CT specificity. The use of highly sensitive RT-PCR assays in our study probably explains the higher specificity and overall accuracy in our population. However, it remains important to emphasize that a negative CT scan does not completely rule out COVID-19 infection and a negative CT should not trick caregivers into a false sense of security in a patient with possible COVID-19 infection. In addition, although no exact radiation dose levels are available, reference scan settings in this study were 120 kVp and 30–70 mAs versus 100 kVp and 20 mAs in our study. When comparing these reference values, this leads to an estimated radiation dose reduction by a factor of five. Given the widespread use of chest CT for COVID-19 detection, our results demonstrate the feasibility of using low-dose chest CT to achieve an important reduction in radiation dose on a population level during this pandemic. However, during a public health crisis, it is important to note that radiation dose considerations should not be the determining factor in deciding imaging strategies.

Previous studies have suggested that CT could be used to track COVID-19 lung involvement over time (5,6). However, a key remaining issue has been to formally assess the reproducibility of CT measures of lung involvement that are essential for the validity of interpreting changes in lung involvement over time. Our results show excellent agreement rates of trained readers for determining the likelihood of COVID-19 infection at low-dose chest CT (all agreement rates were ≥ 0.95). These results suggest a potential role for CT for the longitudinal follow-up of patients with COVID-19 in research studies, particularly if this can be achieved with submillisievert scan protocols. However, in clinical practice, CT scans should be performed only when specific clinical indications emerge and CT may help guide patient management (eg, unexplained clinical deterioration) as recommended by the American College of Radiologists (18).

Of particular interest may be those patients with incongruent findings at RT-PCR and chest CT. Eleven patients had a negative chest CT but a positive RT-PCR test (false-negatives). Within this subgroup, eight (72.7%) presented within 48 hours of symptom onset, highlighting the lower sensitivity of chest CT for early disease detection. Seven patients had a positive chest CT but a negative RT-PCR test (false-positives). Within this subgroup, two patients (28.6%) tested positive for human metapneumovirus at RT-PCR, highlighting that differentiation between different types of viral pneumonia is not always possible with chest CT (19). Finally, chest CT may offer alternative diagnoses explaining the clinical presentation suggestive of COVID-19 infection. The prevalence of alternative diagnoses based on chest CT in patients without COVID-19 infection was 17.6%: bacterial pneumonia (n = 10), lung cancer (n = 3), Epstein-Barr infection (n = 2), pericarditis (n = 1), human metapneumovirus (n = 1), and sarcoidosis (n = 1).

This study had several limitations. First, we acknowledge the inherent selection bias of this retrospective cohort study. Second, our study cohort included only patients who presented at the emergency department, so our cohort represents the more severe disease spectrum of COVID-19. Mild or asymptomatic cases with only upper respiratory tract involvement likely have more false-negative findings at chest CT. Third, with the development of more sensitive and rapid test kits for COVID-19 with a turnaround time of <1 hour, the need for COVID-19 screening with CT will reduce. Rapidly expanding knowledge of this disease and its risk factors may lead to the implementation of diagnostic algorithms allowing for a more selective implementation of chest CT in patients with COVID-19 and a lower number of true-negative cases. This will ensure optimal delivery of diagnostic imaging and treatment guidance while minimizing radiation exposure and unnecessary movement of patients within the hospital. Fourth, despite the use of new highly sensitive RT-PCR assays, this technique is not perfect. Some “false-positives” with typical CT and clinical findings but negative RT-PCR result may still be COVID-19 infection (Fig 4). Further studies are warranted to examine whether these patients may have viral infection limited to the lower respiratory tract. Finally, the use of advanced deep learning–based iterative reconstruction algorithms and state-of-the-art hardware may result in better image quality at similar radiation doses.

Figure 4:

Example images from a 60-year-old female patient with clinical and CT findings suggestive of COVID-19 infection but repeated negative RT-PCR analysis. A, Axial and, B, coronal CT images show typical bilateral subpleural areas of ground-glass opacities. The patient was considered to be probably COVID-19 positive and quarantined. Note the incidental finding of moderate thoracic dextroscoliosis. Window center, −600 HU; window width, 1600 HU; slice thickness, 1 mm; and increment, 0.7 mm for all images.

In conclusion, low-dose submillisievert chest CT allows for rapid, accurate, and reproducible assessment of COVID-19 infection in emergency department patients, in particular in patients with symptoms lasting longer than 48 hours. Chest CT may have the additional advantage of offering alternative diagnoses in a significant subset of patients.