Radiographers and COVID-19 pneumonia: Diagnostic performance using CO-RADS

Abstract

Introduction

A more structured role of radiographers is advisable to speed up the management of patients with suspected COVID-19. The purpose of our study was to evaluate the diagnostic performance of radiographers in the detection of COVID-19 pneumonia on chest CT using CO-RADS descriptors.

Methods

CT images of patients who underwent RT-PCR and chest CT due to COVID-19 suspicion between March and July 2020 were analysed retrospectively. Six readers, including two radiologists, two highly experienced radiographers and two less experienced radiographers, independently scored each CT using the CO-RADS lexicon. ROC curves were used to investigate diagnostic accuracy, and Fleiss’κ statistics to evaluate inter-rater agreement.

Results

714 patients (419 men; 295 women; mean age: 64 years ±19SD) were evaluated. CO-RADS> 3 was identified as optimal diagnostic threshold. Highly experienced radiographers achieved an average sensitivity of 58.7% (95%CI: 52.5–64.7), an average specificity of 81.8% (95%CI: 77.9–85.2), and a mean AUC of 0.72 (95%CI: 0.68–0.75). Among less experienced radiographers, an average sensitivity of 56.3% (95%CI: 50.1–62.2) and an average specificity of 81.5% (95%CI: 77.6–84.9) were observed, with a mean AUC of 0.71 (95%CI: 0.68–0.74). Consultant radiologists achieved an average sensitivity of 60.0% (95%CI: 53.7–65.8), an average specificity of 81.7% (95%CI: 77.8–85.1), and a mean AUC of 0.73 (95%CI: 0.70–0.77).

Conclusion

Radiographers can adequately recognise the classic appearances of COVID-19 on CT, as described by the CO-RADS assessment scheme, in a way comparable to expert radiologists.

Implications for practice

Radiographers, as the first healthcare professionals to evaluate CT images in patients with suspected SARS-CoV-2 infection, could diagnose COVID-19 pneumonia by means of a categorical reporting scheme at CT in a reliable way, hence playing a primary role in the early management of these patients.

Introduction

Coronavirus Disease-19 (COVID-19), caused by Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), has rapidly expanded to touch every corner of the globe, and it is still causing trouble for health care systems in several countries. Initial manifestations of COVID-19 are highly non-specific, including fever, cough, and dyspnoea.¹ Accordingly, a proper identification of COVID-19 patients is particularly challenging. Moreover, the recognition of interstitial pneumonia, one of the most severe complication of SARS-CoV-2 infection, is of utmost importance. Real-Time Reverse Transcriptase-Polymerase Chain Reaction (RT-PCR) is considered the reference standard test in the diagnosis of COVID-19, even if the risk of eliciting initial false-negative results and its variable turnaround time are relevant issues.^{2 , 3} The role of Computed Tomography (CT) for early diagnosis of COVID-19 pneumonia still remains controversial.⁴ Surely, CT is widely available and easy to perform, allowing to achieve fast diagnosis of lung involvement. Bilateral, ground-glass opacities, often associated with patchy consolidations and/or interstitial changes, with a peripheral distribution, are the most representative CT features of patients with COVID-19.^{5 , 6} These typical CT findings have also been reported in patients with negative RT-PCR results, indicating that CT might have a high sensitivity for diagnosis of COVID-19.⁷ However, many scientific societies have underlined the low specificity of such CT features because of their overlapping patterns with other viral pneumonias, therefore discouraging the use of CT as a primary tool for the detection of COVID-19 and advocating the introduction of a standardized reporting language for CT findings related to COVID-19 pneumonia.^{8 , 9} In purpose, the Dutch Radiological Society developed the COVID-19 Reporting and Data System (CO-RADS), a categorical CT assessment scheme aimed at defining the suspicion for pulmonary involvement of COVID-19 at chest CT images.¹⁰ In the current pandemic, radiographers work on the frontline and play a key role in handling COVID-19, not only producing good quality CT examinations and preventing infection spreading in the CT scanning room, but also accurately vetting and interpreting chest CT findings.¹¹ The latter one represents a fundamental aspect in such a way that patients with high suspicion of COVID-19 pneumonia on chest CT, regardless of RT-PCR test results, could be quickly directed into the most appropriate care pathway. Therefore, radiographers should be familiar with COVID-19 imaging features in any healthcare setting in order to define patient management and avoid the risks of further transmission,¹¹ considering that radiographers are often the first healthcare personnel to look at CT images and their initial evaluation is becoming an important point.¹² With the primary aim to recognise to radiographers a more comprehensive and structured role in the management of COVID-19 patients, the purpose of our study was to evaluate the diagnostic accuracy and inter-observer agreement of radiographers in the detection of COVID-19 pneumonia using CO-RADS in comparison with radiologists.

Material and methods

This was a retrospective, single-center study, and it was approved by our institutional review board, while informed consent was waived. Reporting was done according to the Standards for Reporting of Diagnostic Accuracy (STARD) guidelines.¹³

Study population

The electronic medical records of patients admitted to our hospital who underwent RT-PCR and chest CT scan due to COVID-19 suspicion between March 18, 2020 and July 15, 2020 were reviewed retrospectively. Part of the study sample has been reported in a previously published study.¹⁴

RT-PCR assay and chest CT were performed in all patients suspected of having COVID-19 with at least one of the following symptoms: fever (>37.5 °C), dyspnoea, and cough, with or without risk factors for exposure to SARS-CoV-2, including history of close contact with a confirmed COVID-19 case, recent travel in COVID-19-endemic regions, or close relationship with individuals experiencing signs or symptoms of a respiratory tract infection.

Cases with missing RT-PCR test results or with a period of time between chest CT scan and RT-PCR greater than 7 days, and uninterpretable CT images due to respiratory motion artifacts were excluded from further analysis.

CT image acquisition technique

CT examinations were performed using a 64-slice CT scanner (LightSpeed VCT, GE Healthcare), with a gantry rotation period of 500 msec, during a single breath-hold with the following scanning parameters: tube energy of 100/120 kV, variable tube current with automatic mAs modulation (Smart mA, GE Healthcare), 0.6-mm section thickness, and a pitch of 1.388, using iterative reconstruction (ASiR, GE Healthcare).

Image analysis

For evaluation purposes, all chest CT examinations were retrieved from the Picture Archiving and Communication Systems, anonymized, and displayed on a diagnostic workstation (Advantage Workstation Version 4.5, GE Healthcare) enabling manipulation and processing of CT images. Chest CT images were analysed independently by each reader using the descriptors of the categorical CT assessment scheme CO-RADS.¹⁰ Based on the suspicion for pulmonary involvement of COVID-19 at CT images, each observer chose and recorded one of the assessment categories of CO-RADS, with level of suspicion increasing on a scale from 1 to 5: normal or non-infectious abnormalities (CO-RADS 1); infectious abnormalities not compatible with COVID-19 (CO-RADS 2); equivocal findings for pulmonary involvement of COVID-19 (CO-RADS 3); abnormalities suspicious for COVID-19 (CO-RADS 4); typical COVID-19 (CO-RADS 5).

Each CT examination was interpreted by six different readers, including two board certified consultant radiologists (Radiologist 1 and Radiologist 2, with more than 5 years of experience in thoracic imaging and more than 100 chest CT examinations positive for COVID-19 reported), two highly experienced radiographers (Radiographer 1 and Radiographer 2, with more than 150 chest CT scans positive for COVID-19 performed), and two less experienced radiographers (Radiographer 3 and Radiographer 4, with less than 70 chest CT scans positive for COVID-19 performed). All readers had general familiarity with CO-RADS categories, having adopted this system for chest CT interpretation since its introduction several months before the beginning of this study. All readers were blinded to RT-PCR results and clinical-anamnestic data of patients, just as they were unaware of COVID-19 prevalence in the study population.

Reference standard

RT-PCR assay performed on respiratory specimens collected by nasopharyngeal and oropharyngeal swabs served as reference standard for the diagnosis and exclusion of COVID-19. According to our institution guidelines, one or multiple repeated RT-PCR tests, up to a maximum of three, were performed for each patient within 7 days after CT. Patients with at least one positive RT-PCR were categorised as COVID-19 positive, while they were defined as COVID-19 negative in case of multiple negative RT-PCR results. As for patients with only one initial RT-PCR assay performed and turned out negative, they were clinically monitored for a period of 14 days from the first day of exposure or onset of symptoms, and subsequently defined as non-COVID-19 cases if no clinical worsening or laboratory alterations occurred.

Statistical analysis

Categorical variables were reported as frequencies or percentages. Continuous variables were expressed as means ± standard deviations (SD). Categorical and continuous variables were using the Chi-Squared test and the Mann–Whitney U test, respectively.

Receiver Operating Characteristic Curve (ROC) and the corresponding Area Under the Curve (AUC) were computed for each reader to assess diagnostic accuracy.¹⁵ Mean AUC values obtained from the two radiologists and the four radiographers with their corresponding 95% confidence intervals (95% CI) were calculated, and a pairwise comparison of AUCs was performed.

For each observer, the Youden index (J = sensitivity + specificity – 1) was used to calculate the optimal threshold value to discriminate COVID-19 (+) from COVID-19 (−) patients, and the corresponding sensitivity, specificity, positive predictive value, and negative predictive value were derived.

Fleiss’ Kappa method was used to measure the inter-rater agreement using the following coefficients: κ ≤ 20: slight agreement; κ = 0.21–0.40: fair agreement; κ = 0.41–0.60: moderate agreement; κ = 0.61–80: substantial agreement; κ = 0.81–1.00: almost perfect agreement.¹⁶

In all cases, p-value < 0.05 was considered the threshold for defining statistical significance.

Statistical analyses were conducted using Stata Software version 16.1 (StataCorp, College Station).

Results

Study population

Fig. 1 portrays the flowchart diagram of patient enrolment. A total of 967 consecutive patients were initially evaluated, 253 of which were excluded owing to missing reference standard (n = 36), time interval between chest CT and RT-PCR greater than one week (n = 206), or uninterpretable CT images (n = 11). The final study population thus consisted of 714 patients (mean age: 64 years ±19 [Standard Deviation, SD]; range: 8–96), including 419 men (mean age: 62 years ±20 SD; range: 8–96), and 295 women (mean age: 66 years ±18 SD; range: 18–96).

Figure 1.

Study flow diagram of patient recruitment with associated demographic and clinical data (Note – COVID-19: Coronavirus Disease-19; RT-PCR: Reverse Transcriptase-Polymerase Chain Reaction; CT: Computed Tomography).

Patients’ baseline characteristics are presented in Table 1 .

Table 1.

Demographic data of patients included in the study.

Characteristic	All participants (n = 714)	COVID-19 (+) (n = 263)	COVID-19 (−) (n = 451)	p-value
Sex
Male	419 (59%)	155 (59%)	264 (58%)	0.91
Female	295 (41%)	108 (41%)	187 (42%)
Age, yearsa	64 ± 19 (8–96)	63 ± 18 (12–96)	64 ± 20 (8–96)	0.32
Symptoms
Fever	385 (54%)	147 (56%)	238 (53%)	0.15
Cough	485 (68%)	178 (68%)	307 (68%)
Dyspnoea	335 (47%)	145 (55%)	190 (42%)
RT-PCR test, n
1	391 (55%)	35 (13%)	356 (79%)	<0.001
2	98 (14%)	27 (10%)	71 (16%)
3	225 (31%)	201 (77%)	24 (5%)

Unless otherwise indicated, data are numbers with percentages in parentheses.

COVID-19: Coronavirus Disease-19; RT-PCR: Reverse Transcriptase-Polymerase Chain Reaction.

^aData are means ± standard deviations, with ranges in parentheses.

Reference standard

Among the patients evaluated, 391 (55%) underwent a single RT-PCR test, while 323 (45%) patients underwent multiple repeated RT-PCR assays, 98 (14%) of which were tested twice, and 225 (31%) underwent three RT-PCR tests. The median time period between chest CT and reference standard test was 3 days. Overall, 263 patients had at least one RT-PCR assay positive for SARS-CoV-2, resulting in a disease prevalence of 36.8% in our sample. On the other hand, 451 (63.2%) patients were diagnosed as SARS-CoV-2-negative, including 95 patients (21%) with multiple negative results, and 356 (79%) with initial negative RT-PCR and no clinical and/or laboratory alterations during follow-up.

Diagnostic accuracy

Youden's index in conjunction with ROC analysis identified the score of CO-RADS > 3 as the optimal threshold to discern between COVID-19 (+) and COVID-19 (−) patients for all readers.

As for radiologists, Radiologist 1 and Radiologist 2 achieved an AUC of 0.74 (95% CI: 0.71–0.77) and 0.73 (95% CI: 0.70–0.76), respectively, with a sensitivity of 62.4% (95% CI: 56.2–68.2) and a specificity of 79.8 (95% CI: 75.8–83.4) for Radiologist 1, and a sensitivity of 57.4% (95% CI: 51.2–63.5) and a specificity of 83.6% (95% CI: 79.8–86.9) for Radiologist 2.

With regard to the highly experienced radiographers, the AUC value was 0.72 (95% CI: 0.69–0.76), with a sensitivity of 62.4% (95% CI: 56.2–68.2) and a specificity of 79.6% (95% CI: 75.6–83.2), for Radiographer 1, whereas the diagnostic accuracy of Radiographer 2 resulted in an AUC of 0.71 (95% CI: 0.67–0.74), reaching a sensitivity of 55.1% (95% CI: 48.9–61.2) and a specificity of 84.0% (95% CI: 80.3–87.3).

Considering the group of less experienced radiographers, the AUC values for Radiographer 3 and Radiographer 4 were 0.71 (95% CI: 0.67–0.74) and 0.71 (95% CI: 0.68–0.75), respectively, resulting in a sensitivity of 64.6% (95% CI: 58.5–70.4) and a specificity of 75.6% (95% CI: 71.4–79.5) for Radiographer 3, and a sensitivity of 47.9% (95% CI: 41.7–54.1) and a specificity of 87.4% (95% CI: 83.9–90.3) for Radiographer 4.

The mean AUC estimates were 0.73 (95% CI: 0.70–0.77), 0.72 (95% CI: 0.68–0.75), and 0.71 (95% CI: 0.68–0.74) for radiologists, highly experienced radiographers, and less experienced radiographers, respectively (Fig. 2 and Table 2 ). Results from pairwise comparison of AUCs are supplied in Table 3 . The only statistically significant differences were between Radiologist 1 and Radiographer 2, and Radiologist 1 and the two less experienced radiographers, with Radiologist 1 showing slightly better diagnostic performance.

Figure 2.

The graph shows Receiver Operating Characteristic curves (ROC) for each reader as a measure of accuracy for predicting lung involvement by COVID-19 on chest CT using CO-RADS system.

Table 2.

Diagnostic accuracy of each reader for COVID-19 pneumonia on chest CT in accordance with CO-RADS > 3 as optimal positive threshold.

Reader	TP	TN	FP	FN	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	AUC
Radiologist 1	164	360	91	99	62.4 (164/263) [56.2–68.2]	79.8 (360/451) [75.8–83.4]	64.3 (164/255) [59.5–68.9]	78.4 (360/459) [75.6–81.1]	0.74 [0.71–0.77]
Radiologist 2	151	377	74	112	57.4 (151/263) [51.2–63.5]	83.6 (377/451) [79.8–86.9]	67.1 (151/225) [61.8–72.0]	77.1 (377/489) [74.4–79.6]	0.73 [0.70–0.76]
Radiographer 1	164	359	92	99	62.4 (164/263) [56.2–68.2]	79.6 (359/451) [75.6–83.2]	64.1 (164/256) [59.2–68.6]	78.4 (359/458) [75.5–81.0]	0.72 [0.69–0.76]
Radiographer 2	145	379	72	118	55.1 (145/263) [48.9–61.2]	84.0 (379/451) [80.3–87.3]	66.8 (145/217) [61.3–71.9]	76.3 (379/497) [73.6–78.7]	0.71 [0.67–0.74]
Radiographer 3	170	341	110	93	64.6 (170/263) [58.5–70.4]	75.6 (341/451) [71.4–79.5]	60.7 (170/280) [56.2–65.0]	78.6 (341/434) [75.5–81.3]	0.71 [0.67–0.74]
Radiographer 4	126	394	57	137	47.9 (126/263) [41.7–54.1]	87.4 (394/451) [83.9–90.3]	68.9 (126/183) [62.7–74.4]	74.2 (394/531) [71.8–76.4]	0.71 [0.68–0.75]

Data in parentheses are numerators and denominators, with 95% confidence intervals in square brackets.

COVID-19: Coronavirus Disease-19; CO-RADS: COVID-19 Reporting and Data System; TP: True Positive; TN: True Negative; FP: False Positive; FN: False Negative; PPV: Positive Predictive Value; NPV: Negative Predictive Value; AUC: Area Under ROC Curve.

Table 3.

Pairwise comparison of AUCs considering readers against each other.

AUC	Radiologist 1	Radiologist 2	Radiographer 1	Radiographer 2	Radiographer 3	Radiographer 4
Radiologist 1		0.388	0.255	0.009	0.014	0.033
Radiologist 2	0.388		0.901	0.099	0.139	0.189
Radiographer 1	0.255	0.901		0.127	0.029	0.233
Radiographer 2	0.009	0.099	0.127		0.858	0.791
Radiographer 3	0.014	0.139	0.029	0.858		0.918
Radiographer 4	0.033	0.189	0.233	0.791	0.918

p-values obtained from the comparison of one reader over another are reported in the boxes. Statistically significant p-values are in bold.

AUC: Area Under ROC Curve.

In total, 4284 CO-RADS scores were assigned by six readers and, considering the proportion of COVID-19 (+) patients per CO-RADS category, Fig. 3 demonstrates how the rate of COVID-19 (+) patients increases as the CO-RADS category rises. A total number of 496 false positive chest CT reports was found, 165 of which among radiologists, 164 among highly experienced radiographers, and 167 among less experienced radiographers, while a total number of 658 false negative chest CT reports were observed, 211 of which among radiologists, 217 among highly experienced radiographers, and 230 among less experienced radiographers (Fig. 4 ). Among the 391 patients with a single RT-PCR result, 356 (79%) had an initial negative RT-PCR and no clinical and/or laboratory alterations during follow-up. Considering a subset analysis of these patients, 2136 reads were performed, with a total number of positive CO-RADS scores assigned on CT (i.e., CO-RADS 4 and CO-RADS 5) resulting to be 336 among all readers (15.7%), with only 28/356 (7.9%) identified as COVID-19 (+) by all readers unanimously (CO-RADS 4 and/or CO-RADS 5 considering all readers together). On the other hand, taking into account the 228 COVID-19 (+) patients who underwent multiple repeated RT-PCR assays, 102 of these had an initial negative result, testing positive on their second and/or third consecutive RT-PCR. Among these patients, 612 CO-RADS scores were assigned, with chest CT abnormalities identified as suspicious and typical for COVID-19 (i.e., CO-RADS 4 and CO-RADS 5) in 437 cases (71.4%) and 67/102 (65.7%) patients identified as COVID-19 (+) by all readers unanimously (CO-RADS 4 and/or CO-RADS 5 considering all readers together).

Figure 3.

The graph shows the proportion of patients positive for COVID-19 and negative patients according to each CO-RADS category, with corresponding number of reads assigned in parentheses: the higher the CO-RADS category, the higher the relative proportion of patients positive for COVID-19 (Note – COVID-19: Coronavirus Disease-19; CO-RADS: COVID-19 Reporting and Data System).

Figure 4.

a) Example of a false negative case on chest CT: although RT-PCR was positive for SARS-CoV-2, the CT image shows a single ground-glass opacity (GGO) focus at the level of the left upper lobe anterior segment, and all readers defined the finding as CO-RADS category 3. b) Typical CT appearance of COVID-19 pneumonia, showing multifocal bilateral, peripheral/subpleural GGOs, correctly diagnosed as CO-RADS category 5 by all readers (Note – COVID-19: Coronavirus Disease-19; CT: Computed Tomography; SARS-CoV-2: Severe Acute Respiratory Syndrome Coronavirus 2; RT-PCR: Reverse Transcriptase-Polymerase Chain Reaction; CT: Computed Tomography; CO-RADS: COVID-19 Reporting and Data System).

Inter-rater agreement

The overall agreement level among all readers combining every single CO-RADS rating was found to be moderate (κ = 0.49 [95% CI: 0.48–0.51]).

Considering each CO-RADS category taken individually, there was substantial agreement for CO-RADS 1 category (κ = 0.69 [95% CI: 0.68–0.70]), fair agreement for CO-RADS 2 category (κ = 0.34 [95% CI: 0.33–0.35]), slight agreement for CO-RADS 3 (κ = 0.18 [95% CI: 0.17–0.19), fair agreement for CO-RADS 4 (κ = 0.24 [95% CI: 0.23–0.25]), and substantial agreement for CO-RADS 5 category (κ = 0.64 [95% CI: 0.63–0.65]).

The inter-rater agreement level within each group was as follows: moderate among radiologists (κ = 0.53 [95% CI: 0.45–0.55]; moderate among highly experienced radiographers (κ = 0.52 [95% CI: 0.50–0.52]); fair agreement for the group of less experienced radiographers (κ = 0.40 [95% CI: 0.38–0.44]).

The overall agreement considering all radiographers blended together resulted to be moderate (κ = 0.48 [95% CI: 0.46–0.49]).

Discussion

CT represents a widely used approach for the assessment of lung involvement in patients with suspected COVID-19. Prompt recognition of CT findings related to COVID-19 is fundamental in identifying patients who require comprehensive management, especially when these findings are accidently identified, regardless of the initial reason for the scan: in an Emergency Department setting, when RT-PCR test results are still pending, or in asymptomatic subjects undergoing CT for other reasons. Radiographers, as the first healthcare personnel to evaluate chest CT images, may have a pivotal role in this setting.

Our results show that, using CO-RADS > 3 as optimal diagnostic threshold, both highly experienced and less experienced radiographers were able to identify COVID-19 patients, with a mean AUC of 0.72 (95% CI: 0.68–0,75) and 0.71 (95% CI: 0.68–0.74), respectively, similar to estimates observed among radiologists. In our study, we actually found a lower diagnostic yield of CO-RADS, in particular when compared to that reported in previous studies analysing the accuracy of CO-RADS.^{14 , 17} We believe that this discrepancy could be due to the higher expertise and familiarity with the CO-RADS scoring of the readers involved and to the analysis performed by non-radiologists. Considering the relatively low sensitivity reached among radiographers (average of 57.5%), when managing patients admitted due to COVID-19 suspicion with equivocal chest CT findings, prompt clinical review and image analysis by expert radiologists, as well as thorough cleaning and disinfection of equipment, are necessary. On the other hand, good values for specificity (range: 75.6%–87.4%) and NPV (range: 74.2%–78.6%) were obtained, in line with evidence reported in two recently published meta-analyses determining the diagnostic value of an initial chest CT scan in patients with COVID-19 infection in comparison with RT-PCR.^{18 , 19} Nonetheless, nearly 25% of patients were incorrectly triaged as COVID-19 (+). The latter could represent a major concern, for which reason additional clinical and laboratory tests should be considered in order to exclude other aetiologies, monitoring these patients through a dedicated care pathway, while preventing them from coming into close contact with confirmed COVID-19 cases as far as possible before a final diagnosis is made.

On the other hand, the incremental value of chest CT over a single RT-PCR is clearly reported in our study.

Considering the subset of 102 COVID-19 (+) patients who underwent multiple repeated RT-PCR assays with first initial negative testing, chest CT correctly identified 67/102 as positive for COVID-19 pneumonia. Hence, initial chest CT scan has demonstrated reliable and valid at promptly detecting COVID-19, possibly modifying the management of 66% of patients wrongly identified as negative by the initial RT-PCR test.

CT images for the assessment of lung involvement by COVID-19 were interpreted with an overall moderate inter-reader agreement (Fleiss' κ = 0.49 [95% CI: 0.48–0.51]), just as there was moderate agreement among radiologists (κ = 0.53 [95% CI: 0.45–0.55] and radiographers (κ = 0.48 [95% CI: 0.46–0.49]). These results are consistent with data from Prokop et al., who demonstrated overall moderate agreement among all readers on CO-RADS (Fleiss’ κ = 0.47 [95% CI: 0.45–0.49]).¹⁰

Using CO-RADS, an intuitive and reproducible standardized scheme for logging lung involvement related to SARS-CoV-2, radiographers have demonstrated their effectiveness and reliability in recognising the classic CT appearances of COVID-19. Therefore, our results suggest that radiographers could play a pivotal role in the early assessment of COVID-19 pneumonia on CT by means of a categorical reporting scheme. Indeed, radiographers, besides conducting CT examinations, are often the first healthcare providers to evaluate CT findings, and their recognition of CT appearances consistent with COVID-19, even in asymptomatic subjects, is essential in order to limit the spread of the virus and prevent its transmission within hospitals, directing patients with high suspicion of COVID-19 pneumonia on CT into the most appropriate care pathway. The present study suffers from a number of limitations. First, the retrospective and single-center design. However, the large number of patients enrolled could have mitigated this drawback. Second, our study sample was composed only of symptomatic patients. This aspect might have led to a selection bias, even though patients analysed in our study reflect the population for whom CT imaging is recommended.^{8 , 20} Third, multiple repeated RT-PCR assays were not available for all the study population. In this regard, as per our institution guidelines, patients with only one initial negative RT-PCR test were clinically monitored for a period of 14 days, thus establishing their negative status. However, of these patients, only a small percentage (7.9%) was identified positive for COVID-19 pneumonia on chest CT by all readers unanimously. At the same time, bearing in mind how the COVID-19 pandemic has significantly challenged the entire healthcare system, we think that the possibility of having at our disposal multiple repeated RT-PCR test for 45% of the patients evaluated should be regarded as an important strength of our work, representing a solid reference standard, especially when considering patients with an initial negative RT-PCR later confirmed to be positive in subsequent tests. Moreover, a final diagnosis in patients with respiratory tract infections other than SARS-CoV-2 infection was not assessed, therefore the prevalence of other respiratory tract infections that could show CT patterns similar to those observed in COVID-19 was not available. Further research studies will be needed to determine more accurately and validate the potential role of radiographers in the prompt assessment of COVID-19 pneumonia on chest CT.

In conclusion, the results of the present study indicate that radiographers, applying the CO-RADS assessment system, can accurately and promptly identify typical CT imaging features of lung involvement by COVID-19, demonstrating similar levels of diagnostic accuracy to consultant radiologists, hence playing a primary role in the management and care of patients with suspected COVID-19.

Conflict of interest statement

None.