Efficacy of Chest CT for COVID-19 Pneumonia in France

Abstract

Background

The role and performance of chest CT in the diagnosis of the coronavirus disease 2019 (COVID-19) pandemic remains under active investigation.

Purpose

To evaluate the French national experience using Chest CT for COVID-19, results of chest CT and RT-PCR were compared together and with the final discharge diagnosis used as reference standard.

Materials and Methods

A structured CT scan survey (NCT04339686) was sent to 26 hospital radiology departments in France between March 2 and April 24 2020. These dates correspond to the peak of the national COVID-19 epidemic. Radiology departments were selected to reflect the estimated geographical prevalence heterogeneities of the epidemic. All symptomatic patients suspected of having a COVID-19 pneumonia who underwent within 48 hours both initial chest CT and at least one RT-PCR testing were included. The final discharge diagnosis, based on multiparametric items, was recorded. Data for each center were prospectively collected and gathered each week. Test efficacy was determined by using Mann-Whitney Test, Student’s t-test, Chi-square test and Pearson’s correlation. A p value <.05 determined statistical significance.

Results

Twenty-six of 26 hospital radiology departments responded to the survey with 7500 patients entered; 2652 did not have RT-PCR results or had unknown or excess delay between RT-PCR and CT. After exclusions, 4824 patients (mean age 64, ± 19 yrs, 2669 males) were included. Using final diagnosis as the reference, 2564 of the 4824 patients were positive for COVID-19 (53%). Sensitivity, specificity, NPV and PPV of chest CT for diagnosing COVID-19 were 2319/2564 (90%, 95% confidence interval [CI]: 89, 91), 2056/2260 (91%, 95%CI: 91, 92%), 2056/2300 (89%, 95%CI; 87, 90%) and 2319/2524 (92%, 95%CI 91, 93%) respectively. There was no significant difference for chest CT efficacy among the 26 geographically separate sites, each with varying amounts of disease prevalence.

Conclusion

Use of chest CT for the initial diagnosis and triage of suspected COVID-19 patients was successful.

Summary

In France, chest CT in combination with reverse transcriptase-polymerase chain reaction (RT-PCR) testing was effective as a diagnostic tool to assess coronavirus disease 2019 (COVID-19) pneumonia in symptomatic patients.

Key Results

• ■ In a national survey of 26 hospitals (N= 4824 subjects), chest CT sensitivity and specificity for diagnosing COVID-19 pneumonia were 90% and 91%, respectively.

• ■ In 103 patients with an initial positive chest CT finding(s) for COVID-19 and a negative initial RT-PCR test, a repeat RT-PCR was positive in 90% (93/103).

• ■ In patients with both negative chest CT and RT-PCR, the negative predictive value regarding final discharge diagnosis for COVID-19 was 99% (2035/2050 patients).

Introduction

To date, over 15 million confirmed coronavirus disease 2019 (COVID-19) cases have been diagnosed and 671,000 people have died. Since its emergence in Asia late last year, the virus has spread to every continent except Antarctica. It is essential to detect this disease at its earliest stage and immediately isolate the infected person to limit its spread. According to several recommendations (1–3), the reference method for diagnosing COVID-19 is the reverse transcription polymerase chain reaction (RT-PCR) assay. However, RT-PCR has some limitations, such as quality of the sample collection and kit performances, which vary by manufacturer. RT-PCR is reported to have high specificity but variable sensitivity ranging from 60 to 70% (4) to 95-97% (5). A recent meta-analysis reported that RT-PCR testing had a pooled sensitivity of 89% (6). As a result, the false negative rate is a practical problem and it is recommended that several negative tests be obtained before being confident about excluding the disease. In the context of this epidemic, the low sensitivity of RT-PCR implies that many patients with COVID-19 may not be identified and consequently may not be isolated from healthy population. These individuals could continue to spread this disease. Chest CT can detect some characteristic features in almost all patients with COVID-19 pneumonia (7–9). These features have also been observed in patients with negative RT-PCR results but with clinical symptoms (10). In a recent meta-analysis, including 5 studies, Kim et al (6) reported pooled sensitivity of 94% (95% CI: 91%, 96%) for chest CT and 89% (95% CI: 81%, 94%; I2=90%) for RT-PCR. Pooled specificity for chest CT was 37% (95% CI: 26%, 50%).

Recent studies have reported good performance of Chest CT for the diagnosis of COVID-19 pneumonia (6, 15). However, chest CT can be normal, especially in the early course of the disease.

In this study, we hypothesized that chest CT has been effective as a primary diagnosis tool in clinical practice given the perceived higher sensitivity of Chest CT compared to the first RT-PCR test during the workup for the first hospital admission. To demonstrate that point, we launched a French national observational survey (11) to determine the efficacy of chest CT for the diagnosis of COVID 19 pneumonia. The final discharge diagnosis based on a multi-parametric item including clinical findings, RT-PCR testing, chest CT imaging, risk level of exposure, local estimated prevalence and biological data, was used as reference standard. Results of chest CT and RT-PCR were compared together and with the final discharge diagnosis.

Materials and Methods

The survey design was approved by the local institutional review board and recorded on the clinicaltrials.gov website (NCT04339686). Written informed consent was waived due to retrospective anonymized data collection.

Survey and Data Collection

A prospective survey was conducted from March 2 - April 24, 2020 corresponding to the French national COVID-19 epidemic peak. The survey was sent to 26 radiology centers, 14 university hospitals and 12 general hospitals, selected to reflect the geographic prevalence of COVID-19.

The level of epidemic prevalence was estimated each week by the French national health care administration and classified for this study in three types: under 20%, between 20 and 30% and between 31 and 40%.

To reflect potentially different management patterns, four university and public hospitals per geographic area were randomly chosen. Two university hospitals from areas with estimated low disease prevalence were also solicited to balance the national mean prevalence.

For each center, a weekly survey was sent to a referent senior radiologist. The survey included the following parameters: clinical patient data (age, sex), results of initial chest CT and initial and/or repeat RT-PCR tests, time intervals between chest CT and RT-PCR, and final discharge summary according to the hospital discharge report. All patients having undergone both chest CT scan and RT-PCR for suspected COVID-19 were eligible for the survey.

All data were retrieved by manual data extraction from electronic hospital medical records by the referent radiologist.

CT Protocol and Image Analysis

CT examinations were established in accordance with the international guidelines and the local references and are given in Appendix E1 along with an enumeration of the RT-PCR test kits used (Appendix E2).

For each center, a first reading of the presenting chest CT was performed by a single on-site senior radiologist with at least 5 years of experience in emergency radiology. In cases of doubt or difficulties, a double reading was performed in consensus with second reader with ≥5 years of experience in thoracic imaging. Each reader was blinded to the RT-PCR result, but aware of suspicion for COVID-19. Years of experience of the readers is provided in Appendix E1.

A dedicated reading grid, the Rad Report issued by the RSNA, translated in French, was used for each reading (12). According to this structured report, typical findings included: Bilateral ground glass opacities with peripheral distribution, bilateral crazy paving appearance with intralobular thickening, reverse halo sign, or other signs compatible with organizing pneumonia. The presence of at least one of these findings was associated with strong COVID-19 suspicion. Normal Chest CT findings and atypical patterns such as mediastinal lymphadenopathy, pleural effusion, multiple tiny pulmonary nodules, tree-in-bud nodules, and cavitation (1, 13, 14) were classified as negative for COVID-19.

RT-PCR Testing

The RT-PCR assay were performed for each patient. Complete description is given in Appendix E2. Qualitative detection of nucleic acid from SARS-CoV-2 was performed using deep oropharyngeal sampling in all 26 centers. If results of the initial RT-PCR test were negative, results of repeat RT-PCR were recorded. We considered that three negative RT-PCR tests within 6 days were indicative of a negative COVID-19 diagnosis. We considered that a positive diagnosis for COVID-19 infection was present when one was found. Patients with more than 48 hours between chest CT and the initial RT-PCR and those for whom the delay between RT-PCR assay and chest CT was not mentioned were excluded from the analysis.

To evaluate the clinical practice, results of chest CT and RT-PCR were compared together and with the final discharge diagnosis used as reference standard. The final discharge diagnosis was based on multiparametric items, risk level of exposure, local estimated prevalence, symptoms (fever, cough, fatigue, dyspnea, anosmia), evolution during hospitalization for inpatient, lymphopenia, low C-reactive Protein, high procalcitonin, Chest CT and initial and repeated RT-PCRs.

Statistical Analysis

Standard data analysis was performed by a data scientist (M.N, 10 years of experience) using a three-step method: (a) automatic data collection using Microsoft Form (Redmond, Washington, USA), (b) data cleaning and indexing upon identification data using Python Data Analysis Library 1.0.3 (AQR Capital Management, Lambda Foundry, Inc.) and (c) manual extraction of data.

The algorithm to assess diagnosis was established considering RT-PCR results and final discharge summary (secondary end point).

Because the cohort in our survey was not derived from random selection, all statistics are deemed descriptive. No imputation was made for missing data. Continuous variables are expressed as medians and simple ranges. A 95% confidence interval (CI) was obtained with the Wilson score method. Categorical variables are summarized as counts and percentages. Diagnostic accuracy, including sensitivity, specificity, PPV, negative predictive value, and accuracy of chest CT imaging, were calculated using final report as the reference standard. Associations were studied using Student t test. All analyses were performed with R software, version 3.6.2 (R Foundation for Statistical Computing, 2010).

Results

Demographic Results

Twenty-six of 26 hospital radiology departments responded to the survey, corresponding to 7500 patients. The study flow chart is given Figure 1. Among the 7500 patients, 2652 were secondarily excluded because either they had no RT-PCR results (n = 57) or because there was an excessive or unknown delay between RT-PCR and CT (n = 2619). Finally, 4824 patients were included. Mean age (±standard deviation) was 63.9 years ± 18.9 [3, 101 years], including 2155 females (45%) and 2669 males (55%). Among them, there were significantly more male than female patients with positive findings at both chest CT and RT-PCR (p = 0.03). The time interval between chest CT and RT-PCR was less than 24 hours for 54.5% (4088 / 4824) of patients, and between 24 and 48 hours for 10% (796/4824) of patients. Table 1 summarizes the demographic and clinical characteristics of the study population. Fifty-four percent of patients were from geographic areas with estimated disease prevalence of less than 20% (2605 / 4824). In 53% of cases (2575/4824), the initial RT-PCR result was negative.

Figure 1.

Flowchart of the study patients.

Table 1:

Estimated prevalence of the disease over the duration of the study is shown in Appendix E2.

The diagnosis algorithm used to assess COVID-19 pneumonia in our survey is provided in Figure E1.

Analysis considering the final diagnosis according to the Hospital discharge report.

By considering the final diagnosis from the hospital discharge report, sensitivity and specificity of chest CT scan were 90% (95%CI; 88, 91; 2320/2564) and 91% (95%CI; 90, 92; 2056/2260) respectively.

With mean estimated prevalence of 20%, the calculated positive predictive value (PPV) was 92% (95%CI; 91, 93; 2320 /2524) and negative predictive value (NPV) was 89% (95%CI; 87, 90; 2056 /2300).

There were no significant differences in the sensitivity of chest CT regardless of geographic disease prevalence (91% in low prevalence area, 86% in intermediate and 89% in high prevalence, p =.14). PPV and sensitivity of chest CT were higher in the male population than in the female population (91% for the male patients, 85% for the female, p = 0.02).

With regard to the final discharge report, 24 RT-PCR samples were false positive (0.005%, 24/4824). The Negative predictive value for RT-PCR was 87% (95% CI; 85, 90; 2236/2575).

According to this survey, 2035 patients had both negative RT-PCR and Chest CT, 202 patients with negative initial RT-PCR and other parameters suggestive of negativity and 6-day follow-up. 10 Patients with at least 2 negative repeated RT-PCR during the 6 days follow When findings for both chest CT and RT-PCR were negative, the negative predictive value regarding final discharge summary was 99% (95% CI: 99, 100, 2035 of 2050 patients).

Table 2 illustrates the performances of Chest CT and RT- PCR performances using the Final discharge summary as the reference standard. Chest Ct performances with regard to geographic prevalence and considering the final discharge summary as reference for each centers are provided in Table E1. Overall chest CT performances with initial RT-PCR as the reference standard and according to age, sex, and geographic prevalence are provided in Appendix E5.

Table 2:

Table 3:

Discussion

This study reports a nationwide survey on the role of Chest CT in initial assessment of COVID-19 pneumonia. We demonstrate that, in clinical practice, RT-PCR and chest CT were used simultaneously for medical triage whatever the hospital's expertise level and estimated prevalence for COVID-19. Twenty-six of 26 hospital radiology departments responded to the survey. 4824 patients were included for this analysis. Using the final discharge report as the reference standard, 2564 of the 4824 patients were positive for COVID-19 (53%). Sensitivity, specificity, NPV and PPV of chest CT for diagnosing COVID-19 were 90% (95% CI; 89, 91), 91% (95% CI; 91, 92), 89% (95% CI; 87, 90) and 92% (95% CI; 91, 93) respectively. There was no significant difference for chest CT efficacy among the 26 geographically separate sites, each with varying amounts of disease prevalence.

For COVID 19, sensitivity and specificity of RT-PCR and Chest CT remains debated; in cases of low disease prevalence (<10%), the positive predictive value of RT-PCR was reported to be ten-fold that of chest CT (16). In cases involving a wide range of prevalence, pooled 94% sensitivity and 37% specificity were reported for RT-PCR in a recent meta-analysis (6).

Thus, the results of this study are in contrast to recommendations for CT use; indeed, for a large majority of them, using CT as a screening tool is actually discouraged (1–3,15) while others recommend it suggest CT as a surrogate diagnostic test (1,13). Whatever the debate, all of them recommend RT-PCR as the reference diagnosis method. In a recent publication dated April 7, 2020 (17), a Fleischner Society consensus stated that imaging is not indicated in cases of suspected COVID-19 with mild clinical symptoms except in cases of disease progression. On the other hand, the Fleischner Society recommends imaging for medical triage in patients suspected of having COVID-19 who present with moderate to severe clinical symptoms and a high pretest probability of disease. This statement was put forward to limit imaging resource over-using, to decrease risk of viral transmission to radiology staff and patients and to consider additional ionizing radiation exposure (15).

The second message of this study is that in clinical practice, final diagnosis of COVID-19 was sometimes made without any positive RT-PCR tests, since in the large majority of COVID patients, only one RT-PCR assay was performed. This is not altogether in compliance with the international recommendations. For these patients, final diagnosis was made from multiparametric criteria; evolution of clinical symptoms, compatible CT findings, and biological ancillary criteria such as lymphopenia, increased prothrombin time, increased lactate dehydrogenase, and/or mild elevations of inflammatory markers (20). Notwithstanding its relative low sensitivity, RT-PCR has the disadvantage of providing delayed results, often in several hours, and its performance could depend on variations in detection rates from different manufacturers, variations due to patient viral load, and/or improper clinical sampling. In addition, Chest CT presents two main interests: the test is available immediately and results are available in fewer than 15 minutes even if imaging features of COVID-19 pneumonia are non-specific, sometimes overlapping with other viral pneumonias (18,19). In a context of spreading epidemic, limits of RT-PCR and advantages of CT, could explain the atypical diagnosis algorithm observed here.

Our survey demonstrates that, whatever the severity of the symptoms, in areas of relative high prevalence, in clinical practice, RT-PCR and chest CT were used simultaneously for medical triage. There are some likely reasons; 1) early data from China suggests relatively poor diagnostic sensitivity of RT-PCR (16) and CT could additionally aid the clinician in patient triage; 2) In a pandemic, the risk of false-negative test results increases with the widespread character and the prevalence of the disease. The sensitivity of CT for COVID-19 pneumonia is debated but was recently estimated higher than RT-PCR by Fang Y et al (16), 91% versus 71% respectively (p<.001) and 90% versus 87% (p = 0.04) in our study. The sensitivity of the RT-PCR affects the timely management of suspected cases (isolation and medical treatment) and furthers the risks of transmission.

In this study, the final diagnosis was based on a combination of parameters such as level of exposure, local prevalence, clinical evolution, compatible CT findings, RT-PCR testing and biological ancillary criteria such as lymphopenia, increased prothrombin time, increased lactate dehydrogenase and or mild elevations of inflammatory markers (20). The reference standard for COVID-19 infection is RT-PCR positivity, but this test does have false negatives.

Our results have limitations: First, the clinical data were limited (e.g., severity status was not precisely recorded). This factor-limited analysis regarding severity, some patients could have been severe, and others moderate to symptomatic. Therefore, it is difficult to state definitively on clinical practice for this criterion since we do not know precisely to whom the study applied. Second, different radiologists read chest CT images without centralized re-reading and reader experience could have introduced bias. Third, the imaging findings used to differentiate typical from atypical and/or normal findings could be debated. Chest CT protocols were not fixed, which could be associated with reading bias. For instance, it has been shown that contrast material injection may influence the interpretation of ground-glass opacity patterns (1). Fourth, approximately one-third of patients were excluded. Fifth, even if CT reading was performed without knowledge of RT-PCR results, chest CT readers were, aware that the patient was suspected for COVID-19. Lastly, disease prevalence evaluated by local French authorities could be not representative. In France, only symptomatic patients and a small proportion of asymptomatic exposed workers (including health workers, childcare workers) were tested for COVID-19 using RT-PCR. Because the whole population was not systematically screened, the disease prevalence used in this study were estimated. This could explain why performance of chest CT was similar regardless of the disease prevalence, which is surprising since prevalence is supposed to have impact on the predictive values according to Bayes’ theorem.

In conclusion, the results of this French national survey shed light on the role of chest CT in the current COVID-19 pandemic as an initial diagnostic tool in areas of relatively high disease prevalence. These data need to be considered during planning for either local hospital or national budget cycle.