A national UK audit for diagnostic accuracy of preoperative CT chest in emergency and elective surgery during COVID-19 pandemic

Abstract

AIM

To report on a snap audit of all departments in the UK as to the value of preoperative thoracic imaging, preferably computed tomography (CT), of patients undergoing any surgery to assess for changes consistent with COVID-19 preoperatively.

MATERIALS AND METHODS

All Imaging departments in the UK were contacted and asked to record the number of preoperative CT examinations performed in patients being considered for both emergency and elective surgical intervention over a 5-day period in May 2020.

RESULTS

Forty-seven percent of departments replied with data provided on >820 patients. Nineteen percent of additional preoperative CT was in patients undergoing elective intervention and 81% in patients presenting with surgical abdominal pain. There was a high rate of false positives in patients who tested negative for COVID-19, producing a sensitivity for thoracic CT of 68.4%.

CONCLUSION

This UK-wide audit demonstrates that a large number of additional thoracic imaging examinations over a 5-day period were performed with a low sensitivity for the identification of COVID-19 in this preoperative group of patients. Given these findings, it is difficult to justify this additional examination in this group of patients.

Highlights

• •Pre operative CT has little role in the management of patients undergoing emergency surgery during the COVID-19 pandemic.
• •Pre operative CT has little role in the management of patients undergoing elective surgery during the COVID-19 pandemic.
• •Preoperative CT represents a considerable additional imaging workload.

Introduction

As the COVID pandemic has developed in the UK, several sources of guidance have been issued indicating that preoperative and potential surgical patients require thoracic imaging, usually computed tomography (CT) of their chest, to assess for changes consistent with COVID pneumonia. The present study is a snapshot survey undertaken over a 5-day period in May 2020, which demonstrates the limited value of these additional CT examinations in the preoperative diagnosis of COVID-19.

On 25 March 2020, the British Society of Thoracic Imaging (BSTI) and the British Society of Gastrointestinal and Abdominal Radiology (BSGAR), produced a statement recommending CT examination of the chest in patients undergoing CT for acute surgical abdomen.¹ This was followed on the 27 March 2020 by a surgical Intercollegiate statement and guidance stating that “any acute patient having a CT abdomen should have a CT chest: all patients requiring emergency or urgent elective surgery should have chest imaging with CT within 24 hours preoperative or failing that a Chest radiograph along with COVID-19 testing”.² This was updated on 15 April 2020 with further surgical intercollegiate guidance confirming “all patients presenting acutely and requiring abdominal CT should have a CT chest; preop CT chest should be undertaken for those having cancer surgery and requiring level II/III critical care postoperatively, particularly those undergoing thoracic and complex abdominal surgery”; however, a subsequent surgical intercollegiate update on 15 May 2020 reflected “emerging evidence suggests a preoperative CT chest does not add to the detection of COVID-19 in those asymptomatic, isolated and tested and so is not recommended for screening per elective cancer surgery”. Given that it has been postulated that there is little evidence to support this additional CT chest examination in asymptomatic patients,³ a survey of all UK imaging departments was conducted. The aim of this prospective audit was to assess the impact of preoperative CT of the chest in identifying COVID-19 in patients being considered for both emergency and elective surgery,³ and so determine whether this imaging is warranted.

Materials and methods

This was a prospective audit. All radiology departments across the UK were asked to prospectively collect CT data over a 5-day period from 4–8 May 2020.

For each patient referred for additional preoperative CT chest imaging, under these guidelines, respondents were asked to advise whether¹: the patient was an emergency presentation or booked in for elective surgery²; in the case of elective surgery, whether the standard preoperative protocol was followed (i.e. the patient had been asymptomatic for seven days, socially isolating for 14 days with shielding and a negative naso/oropharyngeal swab within 48 h of the procedure, according to local infection prevention guidance)³; there was evidence of COVID-19 pneumonia on the scan⁴; the operation went ahead (if known); and⁵ the patient subsequently tested positive for COVID-19 (if known).

For statistical analysis, findings were considered true positive if there were appearances suggestive of COVID-19 on the scan and the patient subsequently tested positive for COVID-19. Results were false negative if there were no appearances suggestive of COVID-19 on the scan and the patient subsequently tested positive for COVID-19. Results were false positive if there were appearances suggestive of COVID-19 on the scan and the patient subsequently tested negative for COVID-19. Results were true negative if there were no appearances suggestive of COVID-19 on the scan and the patient subsequently tested negative for COVID-19. Using these values, disease prevalence, sensitivity, specificity, accuracy, positive likelihood ratio, and negative likelihood ratio, were calculated using MedCalc Statistical Software version 19.3 (MedCalc Software Ltd, Ostend, Belgium; https://www.medcalc.org; 2020).

Results

Of the 174 UK clinical radiology clinical directors who were emailed the survey, 74 responded (43%). Fifty-eight of those responses produced 820 sets of patient data. Eleven clinical directors stated that they had no relevant patients and five said patients were not actively screened for COVID-19 during the data collection period. Having excluded departments without relevant patients and those in which patients were not actively screened, the departmental response rate was 58/158 or 37%.

CT chest screening for COVID-19

Of the 820 patients included in the study, 153 (19%) were booked in for elective surgery and 667 (81%) were emergency presentations (Table 1 ). One hundred and twenty-three out of 153 (80%) elective cases followed the standard preoperative protocol, eight (5%) did not, and in 22 (14%) cases, information was unavailable. For those elective patients who had not followed the preoperative protocol (Table 2 ), the prevalence of findings suggestive of COVID was 1/8 (13%) compared with 10/123 (8%) for the group that had followed the protocol.

Table 1.

Questionnaire responses.

	Elective surgery (n=153)		Emergency presentation (n=667)
	N	%	n	%
If elective surgery, was the standard preoperative protocol followed?
Yes	123	80	-	-
No	8	5	-	-
Do not know	22	14	-	-
Did the scan indicate the presence of otherwise unsuspected COVID-19?
Yes: classical	3	2	11	2
Yes: indeterminate	7	5	59	9
Other	2	1	20	3
No	141	92	577	87
Was the patient's operation delayed/postponed in light of the report?
Yes	7	5	26	4
No	133	87	438	66
Do not know	13	8	164	25
N/A	0	0	39	6
Did the patient subsequently test positive for COVID-19?
Yes	2	1	17	3
No	113	74	511	77
Do not know	38	25	139	21

Percentages may not add up to 100 due to rounding.

Table 2.

Indications for COVID-19 on imaging and COVID-19 test results in elective patients who followed or did not follow a preoperative protocol.

	Preoperative protocol followed (n=123)						Preoperative protocol not followed (n=8)
	Did the patient subsequently test positive for COVID-19?
	Yes (n=1)		No (n=96)		Don't know (n=26)		Yes (n=0)		No (n=8)		Don't know (n=0)
	n	%	n	%	n	%	n	%	n	%	n	%
Did the scan indicate the presence of otherwise unsuspected COVID-19?
Yes: classical	0	0	2	2	1	4	0	0	0	0	0	0
Yes: indeterminate	0	0	4	4	2	8	0	0	0	0	0	0
Other	0	0	1	1	0	0	0	0	1	13	0	0
No	1	100	89	93	23	88	0	0	7	88	0	0

Ninety (13%) of the 667 emergency presentations were positive on imaging (11 classical, 59 indeterminate, 20 other indication) and 577 were negative (Table 1). The option of “other” was included to capture definitions of COVID presence that was neither classical nor indeterminate, but some other localised classification. Of those emergency presentations that tested positive, in 2/17 (12%) cases the examinations indicated the presence of COVID-19 (one classical, one indeterminate) and the patients experienced a delayed or postponed operation (Table 3 ). In another two cases, the CT did not indicate the presence of COVID-19 and the patient experienced a delayed or postponed operation.

Table 3.

Indications for COVID-19 on imaging and whether the operation was delayed or postponed in emergency patients who tested positive for COVID-19.

	Was the patient's operation delayed/postponed in light of the report?
	Yes (n=4)		No (n=9)		Don't know (n=3)		N/A (n=1)
	n	%	n	%	n	%	n	%
Did the scan indicate the presence of otherwise unsuspected COVID-19?
Yes: classical	1	25	3	33	1	33	1	100
Yes: indeterminate	1	25	3	33	1	33	0	0
Other	0	0	0	0	1	33	0	0
No	2	50	3	33	0	0	0	0

In 643 patients for which both imaging reports and COVID-19 test results were available, 19 tested positive for COVID-19 and 624 tested negative (Table 1). There were 13 true positives, six false negatives, 75 false positives, and 549 true negatives. Thus, the sensitivity of CT chest imaging was 68.42% (95% confidence interval [CI]: 43.45%–87.42%) and the specificity was 87.98% (95% CI: 85.17%–90.43%). These results together with positive likelihood ratio, negative likelihood ratio, disease prevalence in the study cohort, and accuracy are shown in Table 4 .

Table 4.

Evaluation of CT chest imaging.

Statistic	Value	95% CI
Sensitivity	68.42%	43.45%–87.42%
Specificity	87.98%	85.17%–90.43%
Positive likelihood ratio	5.69	3.92 to 8.26
Negative likelihood ratio	0.36	0.19 to 0.70
Disease prevalence	2.95%	1.79%–4.58%
Accuracy	87.4%	84.59%–89.87%

Table 2 shows patients imaged prior to elective surgery and whether the standard preoperative protocol was followed, and here outcome data are less conclusive. Of 10 cases in which the protocol was followed and the imaging report suggested the presence of COVID-19, seven (70%) subsequently tested negative for COVID-19 (95% CI: 39.23%–89.67%); in the three remaining cases, test results were unavailable.

Operations were delayed or postponed in 33/604 (5%) of cases for which information on timing of surgery was available (95% CI: 3.90%–7.59%; Table 1). In 23 of these 33 (70%) cases, the imaging report did not indicate the presence of COVID-19 (95% CI: 52.54%–82.75%), implying something other than COVID-19 pneumonia on imaging led to the operative delay. Of the remaining 10/33 (30%) delayed/postponed cases, the imaging report did suggest the presence of COVID-19 (95% CI: 17.25%–47.46%). Two of these 10 (20%) cases subsequently tested positive for COVID-19 (95% CI: 4.59%–52.06%), four (40%) tested negative (95% CI: 16.71%–68.84%), and in four (40%) cases, test results were unavailable (95% CI: 16.71%–68.84%).

Discussion

Over one-third of the imaging departments in the UK submitted data for the audit over a 5-day period following the issued national guidance to undertake preoperative chest CT. This resulted in 820 additional patient CT chest examinations. This is a good response rate considering the challenges for imaging departments during this period, including workforce shortages and staff redeployment. The large number of patient data submitted reflects the impact that the national guidance has had on imaging departments. There were some elements of the data that were incomplete, such as whether there was a known delay to surgery and whether the patient subsequently tested positive for COVID-19 on reverse transcription-polymerase chain reaction (RT-PCR), but this was a fairly small proportion of responses. With responses received from imaging departments from all four UK nations, this provides a useful snapshot view of national departmental practice during this week.

During this pandemic, the use of CT chest examinations in diagnosis of COVID-19 has caused much debate and there has been conflicting advice and research emerging globally.4, 5, 6 The variability of resources, and RT-PCR testing in particular, has led to the publication of a number of different algorithms describing the use of CT both in diagnosis and in management of these patients.

This study demonstrated a sensitivity of CT in identifying COVID-19 of 68.4% and specificity of 88% in this population in whom the CT chest was being performed as a “screening examination”. Several series have reported a sensitivity of CT ranging from between 93–98% with a specificity of between 25–53%,5, 6, 7, 8, 9, 10 but in the these studies the patients were all symptomatic or were RT-PCR, which differs from the patient population studied in this survey and explains the very low sensitivity we demonstrated. Even in the small number of patients undergoing elective surgery where the CT demonstrated features suggestive of COVID-19, 70% of patients were RT-PCR negative. Based on the CT findings alone, this may have led to an unnecessary delay in potentially complex surgery for these patients.

All of the CT examinations in this survey were clinically requested and reported. There is the potential for cognitive bias, with some rather non-specific CT changes being diagnosed as potential COVID-19 pneumonia, particularly those findings in the indeterminate category; however, there have been several publications on the features of COVID-19 pneumonia in the literature,^{5 , 6 , 11} and the radiology community has become familiar with these imaging findings and their classification. Although this may have contributed to false-positive findings, the overall incidence of COVID-19 pneumonia on CT in this survey was low, and so it is unlikely that this will have had a significant impact on the results. It seems unlikely that reporting of imaging findings during the survey period would have been performed any differently from routine clinical practice and so this should be representative of UK practice during that week. There will also have been a variation in CT protocol across UK imaging departments, but this should not have impacted significantly on the ability of the radiologist to identify these CT features. Capturing these data would not have provided additional useful information in respect of the aim of the survey, which was to understand whether preoperative CT of the chest for COVID-19 was a reliable screening tool, and if so, whether it changed patient management during the period of the study.

The data suggest that the diagnostic yield from CT chest used as a screening tool in patients presenting with acute surgical abdominal pain or in routine preoperative practice is very low. In addition, these extra CT examinations provide an extra radiation dose to the patient, and it would seem difficult to justify this added burden. As all clinical services are restored post-pandemic, there will be additional pressure on imaging departments given slower throughput of patients, ongoing workforce shortages, and the need to provide imaging that has been otherwise delayed due to COVID-19. It is clearly important that imaging facilities are used where there is clear patient benefit, and we suggest that national guidance is changed to reflect the results of this survey.

Conflict of interest

The authors declare no conflict of interest.