Reporting radiographers' interpretation and use of the British Society of Thoracic Imaging's coding system when reporting COVID-19 chest x-rays

Abstract

Introduction

The United Kingdom (UK) has experienced one of the worst initial waves of the COVID-19 pandemic. Clinical signs help guide initial diagnosis, though definitive diagnosis is made using the laboratory technique reverse transcription polymerase chain reaction (RT-PCR). The chest x-ray (CXR) is used as the primary imaging investigation in the United Kingdom (UK) for patients with suspected COVID-19. In some hospitals these CXRs may be reported by a radiographer.

Methods

Retrospective review of CXR reports by radiographers for suspected COVID-19 patients attending the Emergency Department (ED) of a hospital in the UK. Interpretation and use of the British Society of Thoracic Imaging (BSTI) coding system was assessed. Report description and code use were cross-checked. Report and code usage were checked against the RT-PCR result to determine accuracy. Report availability was checked against the availability of the RT-PCR result. A confusion matrix was utilised to determine performance. The data were analysed manually using Excel.

Results

Sample size was 320 patients; 54.1% male patients (n = 173), 45.9% female patients (n = 147). The correct code matched report descriptions in 316 of the 320 cases (98.8%). In 299 of the 320 cases (93.4%), the reports were available before the RT-PCR swab result. CXR sensitivity for detecting COVID-19 was 85% compared to 93% for the initial RT-PCR.

Conclusion

Reporting radiographers can adequately utilise and apply the BSTI classification system when reporting COVID-19 CXRs. They can recognise the classic CXR appearances of COVID-19 and those with normal appearances. Future best practice includes checking laboratory results when reporting CXRs with ambiguous appearances.

Implications for practice

Utilisation of reporting radiographers to report CXRs in any future respiratory pandemic should be considered a service-enabling development.

Introduction

The United Kingdom (UK) has experienced one of the worst initial waves of the COVID-19 coronavirus pandemic, which originated in Wuhan, China, in December 2019. At the time of writing, in the UK there have been over 290,000 positive cases and over 41,000 fatalities.¹ The Midlands region has the second highest prevalence of the disease in the UK, second behind London.²

Common clinical signs and symptoms help guide clinicians’ initial diagnosis. The definitive diagnosis is typically made using the laboratory technique reverse transcription polymerase chain reaction (RT-PCR) from a nasal and pharyngeal swab; though, the positivity rates of RT-PCR specimens from these sites is reportedly only 32% and 63%, respectively.³ Consequently, several tests may be undertaken before a positive RT-PCR result is achieved. A recent meta-analysis⁴ describes blood tests that may also manifest as positive indicators of COVID-19, such as lymphopenia, raised C-reactive protein (CRP), and hypoalbuminemia. The blood results are usually interpreted in conjunction with various other tests, including findings from a chest x-ray (CXR).

The CXR is the initial imaging investigation for all patients in which COVID-19 is suspected,⁵ or patients with mild features of COVID-19 at risk for disease progression and patients with worsening respiratory status.⁶ The CXR reportedly has lower sensitivity in early or mild cases when compared with the RT-PCR⁷; with one study reporting 69% sensitivity.⁸ The CXR report may be available to clinicians before the RT-PCR result. Therefore, an accurate chest x-ray interpretation is paramount in helping to guide the early diagnosis and treatment plan. It is suggested that positive CXR findings in patients with high suspicion of COVID-19 can negate the need for a Computed Tomography (CT) scan.⁹ CT scanning was used in China as the primary modality for investigating COVID-19¹⁰. In the UK however, CT scanning is reserved for seriously ill patients.¹¹ Considering the logistical and infection control issues associated with transporting COVID-19 patients to the CT scanner; utilising the CXR appropriately could help maintain uninterrupted inpatient throughput in CT departments.

Departmental workloads may well be reduced during the COVID-19 pandemic due to the postponement of routine examinations from outpatient clinics and general practitioner referrals. Though, it is possible that Radiologists will still be engaged with reporting CT and Magnetic Resonance Imaging (MRI) reporting backlogs rather than be concerned with reporting Emergency Department (ED) COVID-19 CXR images. Subsequently, this work may be completed by Reporting Radiographers. Previous work has highlighted the progression of radiographers reporting CXR images across the West Midlands region,¹² and Reporting Radiographers are proven to be as accurate as Radiologists,13, 14, 15 with CXR reports indistinguishable from expert thoracic Radiologists.¹⁶

In our hospital, a team of three Reporting Radiographers (two Consultant Practitioners and one Advanced Practitioner) hot report the COVID-19 CXR referrals from the ED during 8am to 6pm, Monday to Friday and 8am-12pm Saturday to Sunday. The team utilise the British Society of Thoracic Imaging (BSTI) CXR report proforma¹⁷ to ensure uniformity across the reports provided. The structured proforma for reporting COVID-19 findings on a CXR includes a coding system to allow simple Radiology Information System (RIS) searching (see Table 1 ). The purpose of this article is to review the accuracy of Reporting Radiographers’ interpretation of, and use of, the BSTI coding when reporting COVID-19 CXRs. Secondary aims were to assess availability of the report compared to RT-PCR result availability, as well as evaluate selected blood test results.

Table 1.

The BSTI coding system (REF) for classifying COVID-19 findings on a chest x-ray image.

Code	Classification	Findings
CVCX0	Normal	COVID-19 not excluded. Correlated with RT-PCR
CVCX1	Classic/probable COVID-19	Lower lobe and peripheral predominant multiple opacities that are bilateral (>> unilateral)
CVCX2	Indeterminate for COVID-19	Does not fit Classic or Non-COVID-19 descriptors
CVCX3	Non-COVID-19	Pneumothorax/Lobar pneumonia/Pleural effusion(s)/Pulmonary oedema Other

Methods

This study was a retrospective review of ED chest x-ray reports authored by three Reporting Radiographers in a National Health Service (NHS) district general hospital in the West Midlands region of the UK. The Health Research Authority tool¹⁸ deemed this review to be a service evaluation, so ethical approval was not required. Authorisation to proceed was granted by the local Imaging Quality Team.

The review covered examinations spanning 45 days from 17th March to 30th April 2020. The criteria for inclusion were; adult patients attending the ED with the diagnostic question querying COVID-19, with a BSTI code in the report and an initial RT-PCR swab result. A Radiology Information System (RIS) search was undertaken to include the reports by radiographers and the inclusion criteria. The sample did not represent all patients who were diagnosed with COVID-19 at the Trust during this period, and it did not include any CXRs reported by Radiologists. The decision tree in Fig. 1 illustrates how the sample was determined.

Figure 1.

A decision tree illustrating how the cases in the sample were selected using the described inclusion criteria.

The RT-PCR result was used as a standard to benchmark the reports and codes against. Each report was analysed to primarily assess if the report description matched the BSTI coding system. Each of the codes was then cross-checked with the initial RT-PCR test to confirm accuracy of use. A note was made whether the report was available before the RT-PCR result. The number of subsequent RT-PCR tests was also noted up to confirmation of a positive result, where applicable. Laboratory results were also checked for presence of lymphopenia, raised CRP and hypoalbuminemia.

For any discrepancies between the codes and the RT-PCR results, the team of Reporting Radiographers (two Consultant Practitioners and one Advanced Practitioner) independently and blindly reviewed the CXRs in question, irrespective of who authored the original report, to reach agreement of the CXR findings. In cases where there was not 100% agreement amongst the Reporting Radiographers, a Radiologist was asked to provide their interpretation as the arbiter. The process of reviewing these cases was to state whether they perceived the appearances to be in keeping with COVID-19 or not.

Results

The sample comprised of 320 cases, consisting of 54.1% male patients (n = 173) and 45.9% female patients (n = 147). The mean age of patients was 67.1 years (standard deviation 17.5, minimum 21, maximum 100), the median age was 70 years and the mode was 82 years.

The Reporting Radiographers used the correct code to match their report description in 316 of the 320 cases (98.8%). In 299 of the 320 cases (93.4%), the reports were available for the clinicians before the RT-PCR swab result.

CVCX0 - normal

Of the 47 cases reported with this code, 29 (62%) were correctly reported as normal and 18 (38%) were proven to be positive by an initial RT-PCR swab. The 18 false negative cases were independently and blindly reviewed by the Reporting Radiographers, in five cases the reviews of the Reporting Radiographers were discordant, and these were reviewed by the arbiter Consultant Radiologist. The consensus being that all cases that were reported as normal were correctly identified as having no radiographic features of COVID-19. Of the 18 false negative cases, 15 had raised CRP, 11 had lymphopenia and 3 had decreased albumin; only three cases had a full complement of these features. Eighty-three percent (n = 39/47) of these reports were available before the RT-PCR result.

CVCX1 – classic/probable COVID-19

Of the 160 cases with this code, 123 (77%) identified COVID-19 correctly when correlated with the RT-PCR result, and 37 (23%) were proven to be negative by the initial RT-PCR. However, almost a third of the cases that had a negative initial RT-PCR result went on to have a subsequent positive result (n = 11/37, 30%). Consequently, this increased the number of correct CVCX1 reports to 134 (84%). The mean number of subsequent tests before a positive result was 1.5 (SD 0.82, minimum 1, maximum 3). The mean number of days until a positive subsequent test was 6.6 (SD 4.12, minimum 3, maximum 16). Of the remaining 26 cases with negative RT-PCR, only six had a subsequent RT-PCR (range 2–6 tests) and all were negative. These 26 cases were reviewed by the Reporting Radiographers, including five reviewed by the arbiter Consultant Radiologist; overall consensus was that 21 cases did have CXR features suggestive of COVID-19 and five cases did not. Ninety-five percent (n = 152/160) of these reports were available before the RT-PCR result, and 93% (n = 115/123) of the correct reports were available before the RT-PCR result.

CVCX2 – indeterminate for COVID-19

Of the 82 cases reported with this code, 57 (70%) identified COVID-19 correctly when correlated with the RT-PCR result, and 25 (30%) were proven to be negative on RT-PCR. One of these cases was proven to be positive on a subsequent swab four days later. The remaining 24 cases were reviewed by the Reporting Radiographers, including eight by the arbiter Radiologist; agreement was that 13 cases had appearances suggestive of COVID-19 and that 11 cases had no features of COVID-19. Ninety-five percent (n = 78/82) of these reports were available before the RT-PCR result, and 96% (n = 55/57) of the correct reports were available before the RT-PCR result.

Of the 51 cases, with codes CVCX1 or CVCX2, reported as positive for COVID-19 but with a negative initial RT-PCR, only 20 (40%) had repeat RT-PCR swabs, but 60% of these (n = 12/20) had subsequent positive results.

The agreement of the CXR in reporting positive COVID-19, which is the degree of concordance with the number of positive RT-PCR results, was 85% (n = 191 positive CXR reports/225 positive initial and subsequent RT-PCR results). The agreement of the initial RT-PCR was 94% (n = 211 positive initial RT-PCR results/225 positive initial and subsequent RT-PCR results).

CVCX3 – non-COVID-19

Of the 31 cases with this code, 18 (58%) were correctly reported as non-COVID-19. One of these cases was proven as positive for COVID-19 on a subsequent RT-PCR swab 12 days later. Thirteen (42%) were proven to be positive for COVID-19 on RT-PCR. One of these cases was reported as COVID-19 features but was assigned the wrong code. Two cases exhibited normal appearances and were correctly reported, as agreed on review by the Reporting Radiographers, but assigned the wrong code. Further analyses of the features of the 13 positive cases are shown in Table 2 . Of these 13 cases, all of them had raised CRP, eight had lymphopenia and five had decreased albumin; four of these cases had a full complement of these results. Almost all of these reports (n = 30/31) were available before the RT-PCR result, and all of the correct reports (n = 18/18) were available before the RT-PCR result.

Table 2.

CXR features of the 13 cases reported as non-COVID-19 with positive RT-PCR.

Number of cases	Features
5	Pleural effusions
2	Normal appearances - wrong code
2	COPD, one case with pleural effusion
1	Pulmonary oedema
1	Lobar pneumonia
1	COVID-19 appearances - wrong code
1	Unilateral basal consolidation

The sensitivity and specificity of the reports based on the initial RT-PCR result and prior to any reviews or inclusion of positive subsequent RT-PCR results is shown in the confusion matrix below (Table 3 ).

Table 3.

Sensitivity and Specificity of the CXR in this sample using the initial RT-PCR result as the benchmark prior to any independent reviews of discordant case.

	PCR		Total
	Positive	Negative
CXR
Positive	180	62	242
Negative	31	47	78
			320
Sensitivity	85.3
Specificity	43.1
PPV	74.4
NPV	60.3
Accuracy	70.9

PPV = Positive Predictor value, NPV = Negative Predictor Value.

After applying the adjustments from subsequent positive RT-PCR results and the outcome of the reviews by the Reporting Radiographers and the Radiologist arbiter for the discordant cases, the confusion matrix below (Table 4 ) shows the adjusted performance of the reporting radiographers.

Table 4.

Sensitivity and Specificity of the CXR in this sample after reviews and adjustments.

	PCR		Total
	Positive	Negative
CXR
Positive	234	8	242
Negative	14	64	78
			320
Sensitivity	94.4
Specificity	88.9
PPV	96.7
NPV	82.1
Accuracy	93.1

PPV = Positive Predictor Value, NPV = Negative Predictor Value.

After applying the described adjustments, the sensitivity of the reporting radiographers is increased by almost 10% and the specificity is more than doubled.

Overall, 93% (n = 256/275) of correct reports were available to referrers before the RT-PCR result. A raised CRP level was present in 97.1% (n = 205/211) of all positive cases, lymphopenia was present in 56.8% cases (n = 120/211), and hypoalbuminemia was present in 34.6% cases (n = 73/211). There were 72 fatalities within the sample, and mortality by age distribution in this sample is shown in chart 1 .

Chart 1.

Mortality by age distribution in this sample.

Discussion

The findings from this review suggest that a reporting radiographer-led CXR hot reporting service using a recognised coding system can be effective in a pandemic situation. The hot reporting service offered during normal hours, with reduced service at the weekend, ensured that the large majority of reports were available before the RT-PCR result. The cases with reports correctly describing positive COVID-19 features would likely have aided the referrer in managing the patient appropriately, prior to confirmation by RT-PCR. There was only a small amount of incorrect code use (n = 3/320, 0.9%) or misinterpretation of findings (n = 16, 5%), considered to be within acceptable tolerances as documented in the literature.^{19 , 20}

With the RT-PCR results as a gold standard; the agreement of the initial RT-PCR test and CXR in determining presence of COVID-19 was 94% and 85%, respectively. The sensitivity of the CXR prior to, and following, the described adjustments was 85.3% and 94.4%, respectively, and are superior to previous work by Wong et al.,⁸ which reported 69% CXR sensitivity. In view of this, the CXR can be considered to have been a useful tool in the preliminary screening of patients in this sample.

The fact that the reporting team and Radiologist arbiter reached agreement that the 18 false negative reports with code CVCX0 all had normal appearances, despite returning a positive RT-PCR result, corroborates findings from previous work suggesting that there may be no radiographic features in early or mild disease.^{7 , 10 , 21}

The CXR features in 10 of 13 false negative reports with code CVCX3, categorised these cases as non-COVID-19 as guided by the BSTI proforma. Yet, they had a positive RT-PCR result. It is interesting that the existence of pleural fluid, for example, does not necessarily exclude the presence of COVID-19, having been previously reported as being present in severe cases.²² Whilst the proforma is provided as a guide, it appears in this sample at least that COVID-19 does not always assume the classic appearances. The heterogeneous distribution of the disease⁷ adds to difficulty in reporting CXRs with ambiguous features and is recognised as a limitation of the examination,²³ and the highly nonspecific imaging findings may overlap with other pathological entities.²⁴ Associating the CXR findings with the relevant blood results may help to reduce false negative, and indeed false positive, reporting.

The proportion of false positive reporting was increased for those reports deemed indeterminate for COVID-19 in which the reporters had reduced confidence in the findings (29%, n = 25/82), compared to the classic/probable decisions (17%, n = 27/160). The reasons for this are not clear, but confirmation bias may play a part given the patients’ clinical history, on top of the presumption of a high incidence rate in the pandemic situation. It could also be resultant of a cautious approach and not wanting to dismiss the presence of COVID-19, knowing the impact the disease can have. Sub-optimal image quality effecting the appearance of the CXR images, the majority of which were performed using a Computed Radiography (CR) mobile x-ray machine, also needs to be acknowledged as an exacerbating factor.

It would be prudent to be acquainted with the relevant blood results that could provide indication to presence of the COVID-19. Recent opinion that the CXR should be used as an adjunct to clinical findings⁸ is supported with findings in this sample, especially so in the cases where CXR appearances do not fit the classic COVID-19 description. The raised CRP and lymphopenia in the positive cases in this sample corroborates the suggestion that these are useful indicators for presence of COVID-19⁴, though hypoalbuminemia was only present in just over a third of positive cases. It could be considered to be best practice to check laboratory blood results to correlate with the CXR findings when compiling the report in these cases to aid the timely and accurate treatment of the patient.²⁵

The mean number of days (six) following a negative result in which a subsequent positive RT-PCR result was obtained is comparable to five days in a previous study.²⁶ It has previously been suggested that false negative RT-PCR results can occur due to inadequate technique.³ Additionally, swabs taken from the nasal and oral cavities have the lowest positivity rate in detecting COVID-19, at 32% and 63% respectively, whereas brochoalveolar specimens (93%) and sputum (72%) have the highest positivity rate.³ Ai et al.²⁶ raise an interesting point regarding the low positivity rate of the nasopharyngeal RT-PCR swab. They compared CT reports to RT-PCR results and suggest that some of the “false-positive” cases on CT may be “true-positive” in view of the imperfect reference standard of the RT-PCR. Considering the 60% of cases in this sample with a false positive report that did not have a repeat RT-PCR swab and the subsequent positivity rate for those that did, along with understanding of the issues associated with RT-PCR testing, it is possible that the actual initial sensitivity of the CXR report could be higher than it appears. Furthermore, it is argued that sequential RT-PCR swabs should be undertaken in cases with CXR findings suspicious of COVID-19⁶. It is also possible that the swab collection of these patients occurred at an inappropriate time relative to disease onset considering the positivity rate of nasopharyngeal RT-PCR declines rapidly after one week of symptom onset.³

The distribution of mortality by age in this sample follows the national trend in that the majority of fatalities are seen in the 80+ years old category.² However, the presence of any underlying comorbidities among the fatalities has not been investigated here, but this does provide scope for further investigation.

Limitations

The retrospective nature of this study needs to be appreciated. The potential effects of confirmation bias need to be recognised when interpreting the findings from this review, as previously mentioned. During the process of blindly reviewing the cases that were discordant with the RT-PCR result, the reporting radiographers were not exempt from potentially reviewing a case that they may have previously reported. This introduces the effect of recall bias whereby the individual may remember the image findings and their original report, thus possibly rendering their review to be subjective rather than objective.

Conclusion

The findings from this retrospective review indicate that Reporting Radiographers can adequately utilise and apply the BSTI classification system when reporting COVID-19 CXRs. Reporting Radiographers can accurately recognise the classic CXR appearances of COVID-19, as described by the BSTI, and those with normal appearances. In the cases with ambiguous or non-classic COVID-19 appearances, it would be worthwhile to correlate imaging findings with laboratory results when compiling the report. Utilisation of Reporting Radiographers to report CXRs in any future respiratory pandemics should be considered as a service-enabling development.

Conflict of interest statement

I have no conflict of interest, financial or otherwise, to declare.