Chest CT in COVID-19 patients: Structured vs conventional reporting

Abstract

Purpose

To assess clinician satisfaction with structured (SR) and conventional (CR) radiological reports for chest CT exams in coronavirus disease 2019 (COVID-19) patients, objectively comparing both reporting strategies.

Method

We retrospectively included 68 CTs (61 patients) with COVID-19. CRs were collected from the digital database while corresponding SRs were written by an expert radiologist, including a sign checklist, severity score index and final impressions. New CRs were prepared for a random subset (n = 10) of cases, to allow comparisons in reporting time and word count. CRs were analyzed to record severity score and final impressions inclusion. A random subset of 40 paired CRs and SRs was evaluated by two clinicians to assess, using a Likert scale, readability, comprehensiveness, comprehensibility, conciseness, clinical impact, and overall quality.

Results

Overall, 19/68 (28 %) and 9/68 (13 %) of CRs included final impressions and severity score, respectively. SR writing required significantly (p < 0.001) less time (mean = 308 s; SD ± 60 s) compared to CRs (mean = 458 s; SD ± 72 s). On the other hand, word count was not significantly different (p = 0.059, median = 100 and 106, range = 106–139 and 88–131 for SRs and CRs, respectively). Both clinicians expressed significantly (all p < 0.01) higher scores for SRs compared to CRs in all categories.

Conclusions

Our study supports the use of chest CT SRs in COVID-19 patients to improve referring physician satisfaction, optimizing reporting time and provide a greater amount and quality of information within the report.

1. Introduction

Coronavirus disease 2019 (COVID-19) is an ongoing pandemic caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) [[1], [2], [3]]. As of December 2020, more than 68.000.000 of people infected and approximately 1.500.000 deaths related to COVID-19 have been confirmed worldwide. Reverse transcription polymerase chain reaction (RT-PCR) is a highly specific test and the established gold standard for the diagnosis of COVID-19. Several radiological scientific societies have published guidelines on the diagnostic work-up of suspected or confirmed COVID-19 patients, proposing the use of imaging according to clinical findings [4,5]. Although routine use of chest high resolution computed tomography (HRCT) for screening is still debated [[6], [7], [8], [9]], this imaging modality plays a key role in management and follow-up of COVID-19 patients [10].

Since COVID-19 HRCT findings are non-specific [11,12], conventional radiological reports (CRs) dictated in a free-text form often do not provide consistent conclusions. Moreover, there is the risk, with conventional reporting, to omit important key features as well as not to address clinical question [13,14]. Thus, the need of a uniform and standardized reporting scheme and language to describe CT findings of COVID-19 pneumonia has been widely accepted [15,16]. In particular, structured reports (SRs) use a template with standardized headings targeted to provide a complete evaluation of all key features that are relevant to the disease.

Several papers have shown that SRs were easier to understand, improving communication with referring clinicians [17,18]. Therefore, adopting a standardized COVID-19 reporting language could help clinical decision making, potentially influencing patients’ outcome.

The aims of this study were (1) to investigate referring physicians’ satisfaction of SRs and CRs of high-resolution lung computed tomography (HRCT) performed in patient with COVID-19 pneumonia, (2) to compare the two reporting strategies with objective parameters.

2. Materials and methods

This retrospective study received the approval of the local Institution Review Board, and the need for informed consent was waived.

2.1. Patient population

The institutional radiology information system (RIS) was searched to identify consecutive HRCT exams performed in clinical routine to evaluate patients with COVID-19 pneumonia, between September 1st and October 31st, 2020. Only patients with a confirmed diagnosis of COVID-19 at the time of HRCT scan (RT-PCR from nasopharyngeal swab positive for SARS-CoV-2) were included in the study. HRCT images were retrieved from the institutional picture archiving and communication system and exams showing poor quality (e.g., breathing artifacts) were excluded from subsequent analyses. The final population consisted of 61 patients (68 HRCT scans).

2.2. HRCT protocol

All scans were performed on a single CT scanner (Toshiba Astelion 16 Slices, Tokyo, Japan) dedicated to COVID-19 patients. The scanning range was from the apex to lung base. The detailed parameters were the following: tube voltage, 120 kVp, mAs modulation, 80–120 mA s, slice thickness, 1.0 mm, reconstruction interval, 0.8 mm.

2.3. Reporting strategies

CRs generated during clinical practice from various radiologists for every included HRCT were retrieved from the RIS and anonymized. Subsequently, SRs were generated for each study by a single experienced radiologist using a template based on the Radiological Society of North America expert consensus document on reporting COVID-19 lung CT findings [16]. Additionally, SRs also included a final impression (typical or atypical findings for COVID-19 pneumonia) and the severity score index proposed by Chung et al. [19]. An English version of the structured report template is shown in Fig. 1 . Furthermore, for 10 exemplary HRCT exams (randomly selected), the same radiologist also generated CRs and recorded the time needed to report these exams with the two strategies. A wash-out period of 3 weeks between the reporting was used to minimize the risk of recall bias. Finally, for these same 10 HRCT scans, radiology report word count was recorded.

Fig. 1.

Schematic version of the structured template embraced in this study for reporting HRCT scans in patients affected by COVID-19.

2.4. Objective and subjective comparison

A radiology resident was asked to read all CRs to verify whether a final impression and a severity score assessment were included. Additionally, time needed for reporting and word count were compared as objective measures.

A questionnaire was realized to evaluate the preference of referring physicians in terms of reporting strategy (Fig. 2 ). In particular, the survey included 6 statement to be rated with a Likert scale ranging from 1 (absolute disagreement) to 5 (absolute agreement) regarding the satisfaction with radiology reports. Each statement was tailored to explore physicians’ perception of a specific characteristic of the radiology report, namely readability, comprehensiveness, comprehensibility, conciseness, clinical impact and overall quality. Two infectious disease specialists (Reader 1 and Reader 2), both working in a COVID-19 unit for at least 6 months, were asked to fill out the survey after reviewing 80 radiology reports (40 CR and SR pairings, generated from a set of 40 HRCT scans). Each respondent worked independently, in multiple sessions, evaluating the anonymized radiology reports in a randomized order.

Fig. 2.

The complete questionnaire sent to the referring physicians for the subjective evaluation of radiology reports.

2.5. Statistical analyses

The Shapiro-Wilk normality test was used to assess continuous variable distribution. These are presented either as mean and standard deviation (SD) or median and range, as appropriate. Differences in normally distributed continuous variables were assessed with paired t-tests. Paired Wilcoxon signed rank tests were used for the remaining comparisons. A p-value < 0.05 was considered as statistically significant. All analyses were conducted using the “stats” (v3.6.2) R package (v4.0.3) [20].

3. Results

3.1. Objective comparison

Nineteen out of 68 CRs (28 %) reported a final impression of radiological findings at the end of their reports. Only in 9/68 CRs (13 %) a severity score assessment was included.

The time needed to produce SRs (mean = 308 s; SD ± 60 s) was significantly lower (p < 0.001) compared to that of CRs (mean = 458 s; SD ± 72 s). Conversely, word count of SRs (median = 100; range = 106–139) did not statistically differ (p = 0.059) from that of CRs (median = 106; range = 88–131).

3.2. Subjective comparison

Fig. 3 summarizes the findings of the subjective assessment performed by the two referring physicians.

Fig. 3.

Average score (with error bars) assigned by each clinician to the paired conventional and structured reports.

In detail, satisfaction with readability was higher for both with SRs compared to CRs (p < 0.01). The rating ranged from 3 to 5 for SRs and from 1 to 5 for CRs, with Reader 1 and 2 assigning a score of 5 to respectively 98 % and 68 % of SRs.

Regarding comprehensiveness and comprehensibility, a statistically significant difference (p < 0.001) was observed in the ratings for SRs and CRs by both referring physicians. The range of scores varied from 2 to 5 for SRs and from 1 to 5 for CRs. In terms of comprehensiveness, 35 % and 45 % of CRs received a rating of 1 or 2 by Reader 1 and 2 respectively; on the other hand, Reader 1 assigned those scores to the 8 % of SRs while Reader 2 always rated SRs with at least a 3 for comprehensiveness. SRs were also more frequently rated with the highest score in regard of comprehensibility (Reader 1 with 88 % and Reader 2 with 43 %).

Concerning conciseness, SRs were judged reasonably briefer than CRs by both Readers (p < 0.0001). The rating ranged from 2 to 5 for SRs and from 1 to 5 for CRs and 90 %–95 % (Reader 1–2) of SRs received the highest rating.

When evaluating clinical impact of the two reporting strategies, satisfaction of both Readers was higher for SRs compared to CRs (p < 0.001). The range of scores varied from 2 to 5 for SRs and from 1 to 5 for CRs. In detail, a score of 5 was assigned respectively to 73 % (Reader 1) and 65 % (Reader 2) of SRs. Conversely, only 20 % (Reader 1) and 3 % (Reader 2) of CRs were similarly classified.

Finally, concerning overall quality, satisfaction of both physicians was higher for SRs compared to CRs (p < 0.001). In particular, the rating ranged from 3 to 5 for SRs and from 1 to 5 for CRs. Specifically, the need to contact the radiologist for further clarification (scores 1 and 2) was perceived by Reader 1 with 13 % of CRs and by Reader 2 with 65 % of CRs. Conversely, high report quality (score 5) was observed respectively in 88 % and in 50 % of SRs for Reader 1 and 2 respectively.

4. Discussion

Interactions between radiologists and referring clinicians have been definitely reduced with widespread use of picture archiving and communication systems. Moreover, this lack of face-to-face contacts has dramatically worsened due to the current pandemic [21]. Consequently, overall quality of radiologic reports, especially in terms of clarity and completeness of contents, is crucial to guarantee optimal patient care. Several studies have proposed the adoption of structured reporting of HRCT and radiographs in patients with suspected or known COVID-19 pneumonia [15,[22], [23], [24]]. Indeed, due to its standardized and schematic nature, embracing structured reporting could prove particularly advantageous in the setting of novel disease with high volume clinical demand. However, to the best of our knowledge this work is the first one to evaluate the satisfaction of the clinicians as well as to perform a direct comparison between SRs and CRs in this setting.

In our study, both referring physicians expressed significantly greater satisfaction with SRs compared to CRs concerning all items proposed by the questionnaire. Overall, it is interesting to note that SRs were never assigned a score of 1 in any of the examined characteristics by the two infectious disease specialists. On the contrary, CRs scoring was more heterogeneous, with the minimum satisfaction being occasionally reported in all domains. This highlights the higher degree of variability that characterizes CRs in general [25]. Focusing on each individual characteristic, the significantly higher satisfaction with SRs readability indicates that a free text can be harder to read compared to a schematic report based on a checklist. The comprehensiveness of about one third to half of CRs was deemed unsatisfactory according the referring physicians in this study. This confirms that CRs are prone to the omission of relevant imaging findings, an issue that appears to be significantly mitigated by the adoption of a structured template. The perceived clinical utility of SRs was rated significantly higher compared to CRs. We hypothesize that this difference could be mainly related to the higher inclusion rate of final impressions and severity index scores in SRs. While this information could aid referring physicians in their decision making, they could be easily included in CRs too and do not represent a direct advantage of the reporting strategy. Nevertheless, a standardized template ensures that these important details are consistently provided, especially with a departmental adoption of SRs.

Although SRs had on average a slightly but not significantly higher word count, this is probably due to the structured nature of the report template and does not negatively affects the satisfaction of referring physicians with the conciseness of the report, as demonstrated by the results of the subjective analysis. Accordingly, it appears that CRs could have an excessive word count compared to the amount of information provided to the clinician. Furthermore, we demonstrated that SRs take significantly less time to be written compared to CRs despite the similar length. This observation is likely owed to the SRs checklist, which allows radiologists to choose from a limited set of predefined options rather than having to elaborate detailed sentences for each scan. Our results stand with various previous papers stating benefits of SRs over narrative reports [13,17,[26], [27], [28]]. Sabel et al. [13] showed that SRs of CT pulmonary angiography in the setting of acute pulmonary embolism received significantly higher ratings for clarity and content than the conventional ones. Schwartz et al. [17] obtained similar results when evaluating satisfaction of surgical and medical oncologists as well as radiologists towards SRs and CRs of various CT exams. In particular, SRs received higher ratings concerning content and clarity compared to CRs, while the two reporting strategies did not significantly differ in modifying patients’ clinical status. Moreover, rectal MRI structured reporting in patients with primary rectal cancer was proved to facilitate surgical planning and lead to greater satisfaction level of referring surgeons compared to CRs [28]. Objective evaluations for the presence of key features of the two reporting strategies have also been previously performed in different settings [26,27]. Franconeri et al. showed that SRs missed only 1.2 ± 1.5 out of 19 key features, whereas in narrative reports 7.3 ± 2.5 key features were missing for planning of fibroid treatment [27]. Similarly, MRI structured reporting strategy in patients with perianal fistulizing disease was proved to miss fewer key features than the narrative one [26].

This study suffers from some limitations that should be acknowledged. Firstly, CRs were taken from clinical practice and therefore, differently from SRs, they were generated by multiple radiologists, which could have further increased the heterogeneity we found in CRs. Moreover, SRs were produced in a research setting, without the pressure and possible inconveniences that can occur in clinical practice. However, it is fair to assume that the structured reporting strategy would maintain its advantages when translated to the clinical practice, as also observed in other fields [26,29]. Another limitation to address could be represented by the relatively low number of reports included, although previous publications on SRs used similar sample sizes [14,27,28]. The SR template employed in this work was tailored for patients with a confirmed diagnosis of COVID-19 pneumonia at the time of CT scan, and therefore the potential value of chest CT structured reporting in patients with unconfirmed COVID-19 diagnosis was not directly explored. Nevertheless, it is reasonable to expect similar results with a modified SR template specifically designed for COVID-19 suspicion. Finally, we did not explore the effect of a real-life implementation of SRs and we were not able to perform a direct correlation of SRs impact on patient outcome, which is a common limitation of preliminary SRs studies [30].

In conclusion, for HRCT in COVID-19 patients a SR template based on current scientific evidence enables radiologists to produce reports that referring physicians consider more readable, comprehensive and of higher quality compared to CRs, while requiring inferior reporting time, thus contributing to the optimization of productivity in radiology departments.

CRediT authorship contribution statement

Arnaldo Stanzione: Conceptualization, Methodology, Writing - original draft. Andrea Ponsiglione: Conceptualization, Methodology, Writing - original draft. Renato Cuocolo: Formal analysis, Writing - original draft, Writing - review & editing. Mariateresa Rumolo: Investigation. Marika Santarsiere: Investigation. Riccardo Scotto: Investigation. Giulio Viceconte: Investigation. Massimo Imbriaco: Investigation, Supervision, Writing - review & editing. Simone Maurea: Investigation, Supervision, Writing - review & editing. Luigi Camera: Investigation, Supervision, Writing - review & editing. Ivan Gentile: Supervision, Writing - review & editing. Arturo Brunetti: Supervision, Writing - review & editing.

Declaration of Competing Interest

The authors report no declarations of interest.