Clinical application of the COVID-19 Reporting and Data System (CO-RADS) in patients with suspected SARS-CoV-2 infection: observational study in an emergency department

Introduction

Early detection of patients with SARS-CoV-2 in the emergency department (ED) is essential to limit the spread of the virus and prevent its transmission within hospitals.¹ At present, the reverse-transcription polymerase chain reaction (RT-PCR) test remains the standard for the diagnosis of SARS-CoV-2 infection²; however, the high false-negative rate (25%), the limited availability of reagents, and the time lag (hours or days) before results are available complicates the rapid and safe detection of patients infected with SARS-CoV-2 in the ED.^{3 , 4}

Since the outbreak of the pandemic, ample literature about a possible role for chest computed tomography (CT) in defining SARS-CoV-2-infected patients has been published. The wide availability of CT and its immediate results have led many authors to propose its widespread use in the diagnostic definition of SARS-CoV-2 infection.⁵ Despite the increased radiological risk and the possibility of obtaining many negative results, the Chinese guidelines have recommended chest CT in patients with respiratory symptoms who are suspected of having SARS-CoV-2 infection, whereas more recently, the American Society of Radiology did not recommend chest CT in the diagnostic definition of SARS-CoV-2 infection.6, 7, 8

It is now known that some CT characteristics are pathognomonic for both the presence and the severity of SARS-CoV-2 infection. At the beginning of the pandemic, Prokop et al. proposed the categorical assessment scheme COVID-19 Reporting and Data System (CO-RADS), a five-point scale using ED chest CT to predict COVID-19 in patients with moderate to severe respiratory symptoms. In their observational study of 105 chest CT examinations, CO-RADS had an area under the receiver operating characteristic (ROC) curve (AUC) of 0.91 for positive RT-PCR test and 0.95 for the clinical diagnosis of COVID-19, with substantial agreement among the operators (Fleiss' kappa of 0.47).⁹

In order to avoid the difficulties related to RT-PCR and to quickly obtain a possible diagnostic confirmation of SARS-CoV-2 infection, CT and the application of CO-RADS for all patients with suspected COVID-19 infection has been utilised at the ED of the General Hospital of Merano (Italy; 70,000 visits per year) since 25 March 2020.

This study reports a 1-month retrospective analysis of the clinical application of CO-RADS in the ED in the evaluation of patients with suspected SARS-CoV-2 infection.

Materials and methods

The data from 120 consecutive chest CT examinations in the initial ED assessment for suspected SARS-CoV-2 infection were included retrospectively. Since 25 March 2020, according to the management protocol, on arrival at the ED, patients suspected to have COVID-19 have undergone chest CT with the application of CO-RADS, as described by Prokop et al. ⁹ Patient demographic, medical history, and clinical characteristics were also recorded upon arrival at the ED.

In the CT imaging report, the radiologist provides the CO-RADS score, in addition to describing the thoracic imaging, making it directly available to the ED physician (Table 1 ). A CO-RADS score of 1 was assigned to patients with a normal CT examination, indicating a non-infectious clinical condition; CO-RADS 2 was assigned to patients with CT examination indicating possible infectious disease, with bronchial involvement but without ground-glass opacity (GGO), where the suspicion of COVID-19 was low; CO-RADS 3 was assigned to patients an infectious image, but unsure for COVID-19, where central consolidations and small GGOs may be present, but more suggestive of other infectious or viral diseases; CO-RADS 4 was assigned based on a high, but not complete, level of suspicion, unilateral GGO, multifocal consolidations, but with predominant peribronchovascular involvement; CO-RADS 5 was assigned based on a very high level of suspicion for pulmonary involvement by COVID-19, with multifocal GGO areas, consolidations in lung regions close to visceral–pleural surfaces (including fissures), and a multi-focal, bilateral distribution (Table 1). Both CO-RADS 0 (an uninterpretable CT image) and CO-RADS 6 (proven COVID-19, with a positive RT-PCR test) designations were excluded, as the clinical CT result anticipated the RT-PCR result (24 h required for the result).

Table 1.

Overview of COVID-19 Reporting and Data System (CO-RADS) scores, degree of suspicion, and examples of clinical findings.

	Level of suspicion for pulmonary involvement of COVID-19	Summary	Examples of findings
CO-RADS 0	Not interpretable	Scan technically insufficient for assigning a score	Insufficient image quality due to severe artefacts due to coughing and/or breathing
CO-RADS 1	Very low	Normal or non-infectious	Non-infectious disease such as congestive heart failure, malignancy, etc.
CO-RADS 2	Low	Typical for other infection but not COVID-19	Other infections such as bacterial infection, bronchiolitis, etc.
CO-RADS 3	Equivocal/unsure	Features compatible with COVID-19, but also other diseases	Unsure such as bronchopneumonia, lobar pneumonia, etc.
CO-RADS 4	High	Suspicious for COVID-19	Unilateral ground-glass, multifocal consolidations without other finding, etc.
CO-RADS 5	Very high	Typical for COVID-19	Multifocal ground-glass opacities with or without consolidations, etc.
CO-RADS 6	Proven	RT-PCR positive for SARS-CoV-2	Patient with positive RT-PCR and positive CO-RADS score (4 or 5).

There are no CO-RADS 0 and 6 in the present study. The CO-RADS score was immediately available for the patient's evaluation in ED, obviously not with reverse-transcription polymerase chain reaction (RT-PCR) results (usually available 24 h later) and therefore CO-RADS 6 was not present.

The CO-RADS score was compared primarily with the result of the reference standard RT-PCR test for SARS-CoV-2. Secondly, the discriminative ability of CO-RADS was also evaluated for the final clinical diagnosis of COVID-19. This secondary clinical outcome was obtained by adding patients with a positive RT-PCR test for SARS-CoV-2 to the patients with one or more negative RT-PCR results, but a final clinical diagnosis of COVID-19 according to hospital medical records.⁹ A definite diagnosis of COVID-19 was established according to the case definition of the interim guidance issued by the World Health Organization (WHO).²

The study was conducted in accordance with the Local Ethics Committee (Azienda Sanitaria dell'Alto Adige, Parere nr. 57-2020) and was conducted according to the Declaration of Helsinki regarding the Ethical Principles for Medical Research Involving Human Subjects.

Statistical analysis

The categorical variables were presented as a percentage and number of events out of the total. Univariate comparisons were performed with Fisher's exact test. The continuous variables were presented as median and interquartile range. The discrimination ability of CO-RADS for at least one positive SARS-CoV-2 RT-PCR test and, subsequently, for the clinical final diagnosis of COVID-19 was evaluated using a ROC curve and the AUC. The performance of each CO-RADS score was evaluated for sensitivity, specificity, negative predictive value, positive likelihood ratio (LR+), and negative likelihood ratio (LR-). Moreover, continuous net reclassification improvement (NRI), integrated discrimination improvement (IDI), and the incremental AUC score (IAUC) were used to compare the predictive capacity of RT-PCR alone with the addition of the CO-RADS score to RT-PCR itself.

All testing was two-tailed, with 0.05 as the level of statistical significance. Statistical analyses were performed using STATA 13.0 software (StataCorp, College Station, TX, USA).

Results

The median age of the patients was 68 years (range 60–78 years) and 60.8% (73/120) were male. Fifty-one of the 120 patients (42.5%) had moderate or severe symptoms on ED admission (respiratory rate >22 and/or saturation ≤93%). The median value of the CO-RADS score was 2 (range 1–5). The distribution of the CO-RADS score was 44.2% CO-RADS 1 (very low level of suspicion), 10% CO-RADS 2 or 3 (low level of suspicion, equivocal findings) and 45.8% CO-RADS 4 or 5 (high level of suspicion, very high level of suspicion).

The characteristics of patients consecutively assessed for suspected SARS-CoV-2 infection, grouped by CO-RADS score, are listed in Table 2 . The clinical parameters of patients with higher CO-RADS scores were more altered, but there were no differences in comorbidities between the two groups. Fifty-one patients (42.5%) tested positive RT-PCR, while 53.5% of patients (64/120) had a clinical diagnosis of COVID-19 at hospital discharge after all the clinical and diagnostic procedures.

Table 2.

Characteristics of patients who underwent chest computed tomography (CT) in the emergency department (ED), divided into patients with a low suspicion of COVID-19 infection (COVID-19 Reporting and Data System [CO-RADS] scores 1–3) and those with high suspicion of COVID-19 infection (CO-RADS scores 4–5).

Variables	CO-RADS <4	CO-RADS 4–5	p-Value
Patients, n (%)	65 (54.2)	55 (45.8)
Age, years, median (IQR)	67 (51–78)	70 (62–80)	0.134
Symptoms
Dyspnoea	21 (33.3)	31 (57.1)	0.117
Fever	62 (95.2)	46 (83.7)	0.261
Cough	21 (33.3)	19 (34.7)	1.000
Gastrointestinal symptoms	12 (19.0)	3 (6.1)	0.186
Vital signs (95% CI)
Heart rate (beats/min)	82 (70–94)	80 (75–99)	0.520
Respiratory rate (breaths/min)	18 (16–20)	20 (16–26)	0.001
Systolic blood pressure (mmHg)	127 (116–141)	131 (129–144)	0.040
Oxygen saturation (%)	96 (94–98)	93 (88–96)	<0.001
Temperature (°C)	36.7 (36–37.4)	37.1 (36.4–38)	0.034
Chronic diseases
At least one chronic disease	43 (66.7)	45 (81.6)	0.218
Two or more chronic diseases	31 (47.6)	30 (55.1)	0.609
Positive RT-PCR for SARS-CoV-2	13 (20)	38 (69.1)	<0.001
Clinical diagnosis of COVID-19	14 (21.5)	50 (90.9)	<0.001

RT-PCR, reverse-transcription polymerase chain reaction.

The AUC of CO-RADS for patients who had a positive RT-PCR was 0.790 (95% confidence interval [CI]: 0.704–0.876; Fig 1 ). The sensitivity and specificity for CO-RADS scores ≥4 were 74.5% and 75.4%, respectively. The AUC of CO-RADS for patients who had a clinical final diagnosis of COVID-19 was 0.878 (95% CI: 0.812–0.943; Fig 2 ). The sensitivity and specificity for CO-RADS scores ≥4 were 78.1% and 91.1%, respectively. In patients with moderate to severe symptoms, CO-RADS presented an AUC for final diagnosis of COVID-19 of 0.890 (95% CI: 0.816–0.981), and the sensitivity and specificity of CO-RADS scores ≥4 were 83.8% and 78.6%, respectively (Fig 3 ).

Figure 1.

ROC for the discriminatory ability of CO-RADS for patients who had a positive RT-PCR in the present patient cohort.

Figure 2.

ROC for the discriminatory ability of CO-RADS for patients who had a clinical final diagnosis of COVID-19 in the present patient cohort.

Figure 3.

ROC for the discriminatory ability of CO-RADS in the group of patients with severe and moderate symptoms at ED admission and a final diagnosis of COVID-19.

Fig 4 describes the improvement in discriminatory capacity when CO-RADS was added to RT-PCR for the final clinical diagnosis of COVID-19 (Fig 4). Based on the NRI, CO-RADS improves the risk classification by 65.8% in patients with a medical diagnosis of COVID-19 and by 82.1% in patients not affected by COVID-19 (Table 3 ). The overall discriminatory ability for COVID-19 infection is improved significantly by the application of CO-RADS, with an IAUC increase of 14.8% (p<0.001).

Figure 4.

Plot predictiveness curves in 5,000 resamples using a bootstrap. The plot shows the relationship between estimated risk of COVID-19 versus risk distribution. The implementation of the CO-RADS score improves the ability of risk determination provided by RT-PCR.

Table 3.

Discrimination statistic of the reverse-transcription polymerase chain reaction (RT-PCR) alone and RT-PCR associated with COVID-19 Reporting and Data System (CO-RADS) score and the ability to predict the final diagnosis of COVID-19.

	NRI (95% CI)	NRI events, %	NRI non-events, %	AUC
RT-PCR (reference)				83.9%
CO-RADS application	1.478 (1.394–1.901)	65.6%	82.1%	98.8%

NRI, continuous net reclassification improvement; IDI, integrated discrimination improvement; AUC, area under the receiver operating characteristic curve.

Discussion

The challenging containment of the COVID-19 pandemic in the last few months in Italy has reinforced the idea that the key to control COVID-19 is early discovery and early isolation.¹⁰ The careful application of these rules in the ED has limited the spread of the virus within the hospital and prevented its transmission among hospitalised patients.^{1 , 10 , 11}

The lack of rapid and accurate diagnostic testing has forced EDs to perform major structural and organisational adjustments in order to isolate suspected patients and safely identify those infected; however, this has led to patients suspected of having SARS-CoV-2 infection staying in the ED for longer durations, a high risk of infection for healthcare professionals, and an exponential increase in workload.^{12 , 13}

The role of chest CT in identifying infected patients has been discussed extensively in recent months. Among the authors who hypothesised a possible role of the CT in the management of suspected SARS-CoV-2 patients, Prokop et al. developed a qualitative examination scheme, the CO-RADS, to be applied to the evaluation of chest CT examinations.⁹

The study reported the use of CO-RADS in the clinical evaluation of patients with suspected SARS-CoV-2 infection. Although this was a single-centre study and cases had begun to fall (4 April was the peak of new daily infections in Italy), CO-RADS demonstrated good discriminatory ability for the identification of SARS-CoV-2-infected patients and has provided physicians with immediate information that impacts the entire ED.

The decision to introduce CO-RADS into clinical practice was made due to the need to limit crowding, reduce the duration in the ED, and decrease the possibility of contact between suspected (but not infected) and infected patients. In addition, the different CO-RADS scores allowed the differentiation of the paths of patients waiting for confirmation of diagnosis by the RT-PCR test and the prevention of crowding in shared areas.

CO-RADS provided standardisation of COVID-19 pneumonia imaging and allowed standardised communication between the radiologist and the ED physician, despite its qualitative nature, providing the physician with a reasonable quantification of patient severity.⁹

COVID-19 pneumonia appears to have distinctive CT characteristics that can be used both to diagnose the infection and to estimate its severity.^{10 , 14} Chen et al. suggested that many radiological semantic features can differentiate between COVID-19 and non-COVID-19 pneumonia and that these can be associated with clinical features to create predictive models that demonstrate excellent performance for the diagnosis of COVID-19.¹⁵ Yin et al. confirmed that quantitative CT parameters of lesion involvement (volume and percentage) in COVID-19 patients has higher accuracy (AUC >0.8) than semiquantitative visual scores (AUC 0.716) for determining the clinical severity of SARS-CoV-2 infection.¹⁶

The discriminatory ability of CO-RADS observed in the present study was lower than in the original study by Prokop et al. (using RT-PCR AUC 0.79 versus 0.91, per clinical diagnosis of COVID-19 AUC 0.88 versus 0.95).⁹ This could be caused by the inclusion of patients with minor symptoms in the present cohort; however, the sensitivity of the tool was elevated both in patients with moderate to severe symptoms and in patients with mild symptoms.

CT may have influenced the high discriminative ability of CO-RADS against the final diagnosis of COVID-19. Although this must be considered when evaluating the present results, this secondary clinical outcome may reinforce some considerations applicable to clinical practice. In the ED setting, where infected patients who need emergency treatment and hospitalisation are often present, the availability of the RT-PCR test result is not fast enough. In this setting of false negatives by RT-PCR for SARS-CoV-2, the comparison between CO-RADS and the final diagnosis of COVID-19 demonstrated reduction of admitting infected patients to sections of the hospital not intended for them (without the appropriate precautions).¹⁷ In an earlier stage of the pandemic (when this study was performed), the WHO indicated that patients with a negative RT-PCR test but clinical, laboratory, and radiological findings suggestive of a viral infection, but without other microbiological findings, should be considered positive for COVID-19 based on strong clinical suspicion. In the hospital where this study was conducted, several microbiological tests were performed routinely to exclude the presence of other viral and bacterial infections, in order to meet the WHO guidelines.

One problematic aspect of this tool remains the decision to perform CT on patients with mild respiratory symptoms. There is currently no unanimous consensus around this choice, but several studies report the utility of CT in recognising SARS-CoV-2-infected patients.^{17 , 18}

Although minimal compared to patients who are more symptomatic, patients with mild symptoms may present underlying COVID-19 pneumonia masked by a pulmonary functional reserve that may deteriorate over hours or days.^{14 , 19} As reported by Du et al., only 9.4% of patients who subsequently died in Wuhan Hospitals experienced critical symptoms at the time of arrival at the hospital, most rapidly deteriorated and then died.¹⁹

Although negative CO-RADS cannot exclude the presence of an infection, the system appears safer in patients with respiratory symptoms (even minor ones) than in asymptomatic patients. Therefore, the use of CO-RADS in settings other than an ED, or in patients with flu-like symptoms but without respiratory involvement, would not be recommended. Several RT-PCR tests can be repeated safely in these non-severe patients, who do not need urgent treatment or investigations, or they can be instructed to quarantine at home.

CO-RADS 5 for the clinical diagnosis of COVID-19 correlated less well with RT-PCR than CO-RADS 4. Although a COVID-19-specific radiological pattern has been defined, some conditions can be misdiagnosed. In particular, cases of other viral interstitial pneumonias, aspiration pneumonia, and organising pneumonia with congestive heart failure can be observed as peripheral GGOs, vascular thickening, and subpleural bands, mimicking SARS-CoV2 infection. The need for rapid diagnosis in an emergency setting together with the appearance of a new pathological entity and lack of specific radiological signs could explain the gap between CO-RADS 4 and 5.

Other possible limitations should be reported. The first is bias due to the retrospective nature of the study; however, bias was minimised because all patient information obtained in the ED was recorded precisely during the pandemic period. The second is that the CO-RADS score assigned in the ED may have influenced the final medical diagnosis and thus improved the performance of the tool. Third, conducting the study during a phase of the pandemic characterised by a high incidence of patients with COVID-19-like symptoms, and by a high prevalence of COVID-19 infections, may have increased CO-RADS performance. Therefore, despite good preliminary evidence, prospective studies will be required in the coming months (when the incidence of other diseases will probably increase), to confirm the possible role of CO-RADS in clinical practice.

In conclusion, the rapid and safe determination of SARS-CoV-2-infected patients among suspected patients assessed in the ED remains complicated, with important effects on the ED. The role of chest CT in the initial assessment of patients with suspected SARS-CoV-2 infection has been discussed; however, some CT characteristics are associated with both the presence and severity of the infection. CO-RADS has a good discriminatory ability for the identification of SARS-CoV-2-infected patients and a high specificity for the exclusion of non-infected patients. Despite being preliminary data requiring prospective validation, this first report on the clinical application of CO-RADS suggests that this may be a useful tool in the management of patients with suspected SARS-CoV-2 infection in the ED.

Conflict of interest

The authors declare no conflict of interest.