Performance of ultrasound in the diagnosis of cholecystitis: not so (ultra)sound?

Abstract

Introduction

Ultrasound has long been the radiological investigation of choice for right upper quadrant pain in the detection of gallstones and cholecystitis. However, previously reported sensitivity, specificity and other diagnostic metrics have varied widely and the underlying patient numbers have been small. We present robust and exhaustive diagnostic metrics based on a large series of 795 patients.

Methods

All laparoscopic cholecystectomies at Portsmouth Hospitals University were prospectively logged between 2017 and 2020. Ultrasound findings, Nassar operative difficulty and histopathological findings were all collected in addition to patient biometrics.

Results

In our large patient series, the sensitivity of ultrasound for cholecystitis was lower than previously reported at 75.7% for acute cholecystitis, 34.6% for chronic cholecystitis and 42.7% overall. Moreover, we show that sensitivity degrades with the time between ultrasound and cholecystectomy, falling below 50% at 140 days. Finally, we show that ultrasound strongly predicts the Nassar difficulty grade of cholecystectomy and that its ability to do so is greatest when the interval between ultrasound and cholecystectomy is less than 27 days.

Conclusions

We present robust diagnostic metrics for ultrasound in the diagnosis of cholecystitis. These should caution the clinician that ultrasound may miss a quarter of cases of acute cholecystitis and over half of all cases of cholecystitis. Conversely, the finding of a thickened gallbladder wall on ultrasound can predict a ‘difficult cholecystectomy’ and highlight the need for appropriate expertise and resources. Both this prediction and the diagnostic sensitivity are best if the ultrasound is done less than 27 days before cholecystectomy.

Introduction

Ultrasound has long been the radiological investigation of choice for right upper quadrant pain for the detection of gallstones and cholecystitis.¹ Although ultrasound has slightly reduced accuracy compared with cholescintigraphy, it has much greater clinical availability and familiarity.¹ It is also superior to computed tomography, especially in the detection of gallstones and probably also in the detection of cholecystitis.² Just how sensitive and specific ultrasound is in the detection of cholecystitis, as opposed to just gallstones, remains a question. Many previous studies have had low numbers of total patients examined, ranging from 26 to 186, or incomplete reporting of diagnostic metrics, or did not distinguish acute versus chronic cholecystitis or include the latter in their study.^3–9 Previous studies also used either exclusively fully trained consultants or attending radiologists^3–5 or, in some cases, compared the performance of fully trained radiologists with that of either surgeon-performed^6,7 or emergency department (ED) physician-performed⁸ ultrasound in the diagnosis of gallstones and cholecystitis. The gold standard reference for the calculation of diagnostic metrics has also been confounded in some of the larger studies by the use of an ‘intraoperative’ clinical judgement diagnosis rather than the more objective histopathological diagnosis.⁹ Recent international guidelines (World Society of Emergency Surgery 2020)¹⁰ highlighted that robust metrics for the performance of ultrasound in diagnosing cholecystitis are still lacking.

We present a large series of 795 UK patients who presented to our university hospital during the period January 2017 to January 2020 and underwent ultrasound prior to cholecystectomy. Using the pathological findings of the gallbladder specimen as the gold standard reference, we present full diagnostic metrics for the performance of ultrasound in the diagnosis of both acute and chronic cholecystitis, thereby availing the clinician of robust reliable test performance metrics to inform clinical practice.

We also compare the performance of sonographers, trainee radiologists and fully qualified consultant radiologists with comparative diagnostic metrics, given that in real-world UK practice ultrasounds are often not consultant delivered. We hope these findings will better inform clinicians of the real-world standards of ultrasound in the setting of cholecystitis, provide high-fidelity metrics for audit at other centres and inform the contemporaneous debate regarding surgeon- or ED physician-performed ultrasound.

Previous studies have sought to predict operative difficulty based on ultrasound findings. However, many have been single-surgeon studies¹¹ or had low patient numbers,^12,13 and have employed a wide variety of ‘difficulty’ scoring systems, many of which retrospectively assessed open conversion,^12–14 operative time^13,14 and ‘handling’ difficulties¹² as indicators of a ‘difficult’ cholecystectomy, which may themselves relate to the individual aptitude of the surgeon rather than more objective measures of difficulty. The difficulty grading system for laparoscopic cholecystectomy originally published by Nassar in 1995¹⁵ is an objective measure that has been validated both by Nassar and others in subsequent patient series. In a recent study,¹⁶ ultrasound findings, such as gallbladder wall thickness, were found to be associated with Nassar difficulty. In our study, we sought to provide further independent evidence of the strength of association between ultrasound findings and Nassar difficulty, and hence the ability of ultrasound to predict the ‘difficult cholecystectomy’. Such information could help plan the appropriate skill mix of the surgical team, scheduling of operating time, and hospital beds and perhaps the venue for surgery such as a treatment centre, district general hospital or tertiary centre.

Finally, we examine the relationship between the interval from ultrasound to subsequent cholecystectomy and the diagnostic accuracy of the ultrasound in detecting acute or chronic cholecystitis. We further examine the relationship between this interval and the strength of prediction of the Nassar difficulty grade based on the ultrasound findings. We hypothesise that the less recent the ultrasound compared with the time of the cholecystectomy, the less reliable the ultrasound findings will be in predicting both the histopathological diagnosis and the operative difficulty. More uncertain is the question of how ‘out of date’ an ultrasound can become before its sensitivity for cholecystitis becomes no better than chance, namely 50%, or until its utility in predicting Nassar grade might be impaired. We know of no studies that address this question of the duration of validity of an ultrasound for the diagnosis of cholecystitis and prediction of operative difficulty. Such data might inform guidelines for the preoperative assessment and management of patients undergoing cholecystectomy. This is pertinent for two purposes. First, previous studies from our unit¹⁷ and others¹⁸ have shown the safety and effectiveness of performing cholecystectomies acutely on index presentation for cholecystitis and other emergent gallstone presentations. The argument for performing cholecystectomies acutely rather than in a delayed fashion might be further strengthened by evidence concerning the accuracy of ultrasound and its impact on the prediction of operative difficulty when performed proximately to cholecystectomy. Second, in the current times of the COVID pandemic when routine and even emergency cholecystectomies are deferred and clinicians may be faced with information from an ultrasound performed months or years before, evidence concerning the validity and reliability of ultrasound after some time would inform the clinician as to when repeating an ultrasound might be worthwhile.

Methods

Ten surgeons in the upper gastrointestinal and general surgery departments of Portsmouth Hospitals University NHS Trust prospectively logged all laparoscopic cholecystectomies performed for gallstone disease between January 2017 and January 2020 into a secure Microsoft Excel database. Prospectively collected variables included body mass index (BMI), preoperative diagnosis (eg biliary colic, acute cholecystitis), emergency/elective case, operative time, Nassar difficulty grade and operative date. To these were retrospectively added gender, age, preoperative ultrasound findings and gallbladder histopathology. All retrospectively entered data was found from a search of the hospital electronic patient records, including the clinical documents repository, radiology system and the pathology system, as well as linking to the records of the patient’s general practitioner where necessary (all accessed via Ministrone software). All histology was reported by a consultant histopathologist and all ultrasounds were reported by a sonographer, registrar radiologist or consultant radiologist. In the case of histology, the key terms ‘acute cholecystitis’ and ‘chronic cholecystitis’, if found, were taken to be the key recorded finding. In the case where the term ‘acute on chronic cholecystitis’ was used, this was categorised as a finding of acute cholecystitis. Where acute or chronic cholecystitis was not found, the free text of the other findings were recorded. In the case of ultrasound findings, gallbladder wall thickness, pericholecystic fluid and the sonographic Murphy's sign were sought. Ultrasounds were recorded as performed by a registrar radiologist or sonographer only where the report did not include reference to findings being observed or discussed with a consultant radiologist. Reported diagnostic test metrics were calculated from contingency tables of the data in the usual fashion, as described by Lalkhen and McCluskey.¹⁹

Informed consent for all procedures and use of data was obtained from all participants included in the study.

Results

Demographics and exclusions

In total, 930 cholecystectomies were performed over the 3-year collection period. Of these, no histology could be found for three cases, no ultrasound could be found or a different imaging modality was used for 69, and in 63 cases no comment was made in the ultrasound report as to whether the gallbladder was thick- or thin-walled and these were excluded from further analysis. Only 33 cases were reported as showing pericholecystic fluid and the vast majority of times that pericholecystic fluid was reported, the gallbladder wall was also reported as thickened (30/33). Only 43 cases reported either a positive or negative sonographic Murphy's sign. We therefore decided to focus solely on whether the gallbladder was thick- or thin-walled as the sonographic indication of cholecystitis because this was by far the most reliably reported assessment of gallbladder inflammation. We also prospectively excluded cases of acalculous cholecystitis. This resulted in 795 cholecystectomies for which full data was available.

Within the included data, the patient age varied from 17 to 89 years (median 53, interquartile range (IQR) 27 years). BMI ranged from 17.3 to 61 (median 29.0, IQR 8.3) and the female-to-male ratio was 2.5:1, with 72% of patients female and 28% male. Indications for cholecystectomy included biliary colic (45%), acute cholecystitis (42%), gallstone pancreatitis (7%), choledocholithiasis (5%) and gallbladder polyps (3%). The case mix was 53% elective surgery and 47% emergency surgery. Nassar difficulty case mix was 43% grade I, 24% grade II, 21% grade III, 9% grade IV and 3% grade V. Based on previous studies,¹⁶ we felt our sample was fairly typical of patients undergoing laparoscopic cholecystectomy and specifically those in the UK population.

Ultrasound diagnostic metrics for cholecystitis

Table 1 is a contingency table for cases of histologically confirmed cholecystitis or histologically confirmed normal gallbladder against whether the ultrasound showed thickening of the gallbladder wall. On this basis, we calculated diagnostic metrics for the diagnosis of cholecystitis on ultrasound (Table 2).

Table 1 .

Contingency table of cases

	Ultrasound
Histopathology	Thick walled	Thin walled	Total
Cholecystitis	330	443	773
Acute cholecystitis	115	38	153
Chronic cholecystitis	215	405	620
No cholecystitis on histology	4	18	22
Total	334	461	795

Summary raw data showing the gold standard histopathology versus the key diagnostic metric of gallbladder wall thickness on ultrasound

Table 2 .

Sensitivity, specificity, positive predictive value, negative predictive value, accuracy and likelihood ratios of ultrasound for cholecystitis

n=795	All cholecystitis	Acute cholecystitis	Chronic cholecystitis
Sensitivity (%)	42.7	75.7	34.6
Specificity (%)	81.8	81.8	81.8
PPV (%)	98.8	96.6	98.2
NPV (%)	3.9	32.7	4.2
Accuracy (%)	37.8	76.4	36.2
LR+	2.35	4.13	1.91
LR−	0.70	0.30	0.80

LR = likelihood ratio; NPV = negative predictive value; PPV = positive predictive value

Diagnostic metrics according to clinical staff member performing ultrasound

To investigate the possible variability in diagnostic metrics when different grades of clinical staff assess a patient with ultrasound for the diagnosis of cholecystitis, we computed all metrics based on whether a consultant radiologist, trainee registrar radiologist or sonographer performed the scan (Table 3).

Table 3 .

Diagnostic metrics by clinical staff group

n=795	All cholecystitis			Acute cholecystitis			Chronic cholecystitis
	Consultant radiologist	Registrar	Sonographer	Consultant radiologist	Registrar	Sonographer	Consultant radiologist	Registrar	Sonographer
Sensitivity (%)	37	45	43	71	72	77	35	34	35
Specificity (%)	50	100	89	50	100	89	50	100	89
PPV (%)	95	100	99	71	100	98	95	100	99
NPV (%)	3	0.0	5	50	0	39	3	0	6
Accuracy (%)	38	45	45	64	72	79	36	34	37

NPV = negative predictive value; PPV = positive predictive value

This table shows that diagnostic performance of consultant radiologists, registrars and sonographers was generally similar across the various metrics with no statistically significant differences.

This generally showed that that diagnostic performance was very similar between different grades of clinical staff, giving confidence that our whole data set represented quality equivalent to a consultant radiologist. Contingency table chi-squared testing showed no statistically significant difference in any of the metrics between staff group (χ²=0.64, p=0.953 for sensitivity; χ²=0.0, p=0.982 for specificity; χ²=3.05, p=0.552 for positive predictive value (PPV), χ²=7.58 p=0.116 for negative predictive value (NPV) and χ²=0.71 p=0.951 for accuracy).

Time between ultrasound and cholecystectomy

We further divided cases based on the recency of preoperative ultrasound to the date of cholecystectomy along the arbitrary divisions of within 5 days, within 30 days, and more than 30 days before the cholecystectomy. Using these divisions, we found the diagnostic metrics outlined in Table 4.

Table 4 .

Diagnostic metrics in relation to duration between ultrasound and cholecystectomy

	All cholecystitis			Acute cholecystitis			Chronic cholecystitis
	≤5 days	≤30 days	>30 days	≤5 days	≤30 days	>30 days	≤5 days	≤30 days	>30 days
Sensitivity (%)	60	57	28	79	77	50	47	45	28
Specificity (%)	40	50	94	40	50	94	40	50	94
PPV (%)	98	99	99	97	97	80	97	97	99
NPV (%)	2	2	5	7	8	79	2	2	5
Accuracy (%)	59	56	31	77	76	79	47	45	67

NPV = negative predictive value; PPV = positive predictive value

Comparative diagnostic metrics based on number of days between ultrasound and cholecystectomy. In particular this shows how sensitivity decreases over time for both acute and chronic cholecystitis.

Although these tables show that specificity, PPV and NPV remain relatively similar irrespective of time between ultrasound and cholecystectomy, sensitivity varied widely. To further investigate this relationship we plotted sensitivity and the interval between ultrasound and cholecystectomy on a continuous cumulative basis. As shown in Figure 1, this demonstrated that the sensitivity of an ultrasound to detect gallbladder wall thickening indicative of cholecystitis dropped below 50% after 140 days.

Figure 1 .

Interval between ultrasound and cholecystectomy versus cumulative sensitivity. This graph demonstrates sensitivity calculated on a continuous basis against number of days between the ultrasound and the cholecystectomy. It shows how ultrasound gradually becomes less sensitive and therefore less accurate with increasing time compared with pathological findings at time of cholecystectomy. Leader line A shows the sensitivity at 27 days for comparison with our finding that Nassar difficulty is best predicted with an interval between ultrasound and cholecystectomy of less than 27 days. Leader line B shows the 50% sensitivity point at 140 days for all cholecystitis.

Prediction of Nassar difficulty based on ultrasound gallbladder wall thickness

To investigate the relationship between gallbladder wall thickness on ultrasound and Nassar difficulty at the time of cholecystectomy, we first performed a chi-squared test of independence. Given that gallbladder wall thickness is ultimately a binary variable, we necessarily compacted Nassar difficulty into a binary outcome variable indicating either a Nassar grade 1 or 2 (an ‘easy cholecystectomy’) or Nassar grade 3, 4 or 5 (a ‘difficult’ cholecystectomy). We found a highly significant association between gallbladder wall thickness and Nassar grade (χ²=24.42, p<0.001). We further investigated the strength of this association by calculating the odds that the thinness or thickness of the gallbladder wall was associated with the difficulty of the cholecystectomy (Nassar 1 or 2 vs Nassar ≥3). Overall we found the odds ratio that a thick-walled gallbladder predicted a ‘difficult cholecystectomy’ (Nassar ≥3) was 4.5 (95% confidence interval (CI) 4.1–4.9), showing that the presence of a thickened gallbladder wall on ultrasound was four times more likely to result in a ‘difficult cholecystectomy’. However, further to this, we observed that the individual odds of a ‘easy cholecystectomy’ (Nassar 1 or 2) or ‘difficult cholecystectomy’ (Nassar ≥3) varied depending on the time interval between the ultrasound and the cholecystectomy. With a short interval, the odds of a thick-walled gallbladder predicting a ‘difficult cholecystectomy’ (Nassar grade ≥3) was high but reduced with increasing time. Equally, the odds of a thin-walled gallbladder predicting an ‘easy cholecystectomy’ (Nassar 1 or 2) was low if the interval was short but improved with time. Overall, we found that a period of 27 days was a point of crossover, whereby the odds of a thick-walled gallbladder predicting a ‘difficult cholecystectomy’ and a thin-walled gallbladder predicting an ‘easy cholecystectomy’ were very similar at 2.18 (95% CI 1.75–2.72). This indicates that a finding of a thick-walled gallbladder at 27 days indicates a twofold risk that the cholecystectomy will be difficult. Furthermore, this trade-off of odds indicates that, whereas a thin-walled gallbladder can predict an easy cholecystectomy at an interval greater than 27 days, the more clinically relevant fact of predicting a difficult cholecystectomy requires an ultrasound less than 27 days before surgery.

Discussion

We have presented detailed comprehensive diagnostic metrics for the use of ultrasound in the diagnosis of cholecystitis. These metrics are based on far larger patient numbers than have been reported previously.^3–9 Furthermore, very few studies have provided evidence for the diagnostic metrics of ultrasound in detecting chronic cholecystitis, a condition that arguably can account for as much morbidity and operative difficulty as acute cholecystitis. We show how the sensitivity of ultrasound for cholecystitis, and even specifically acute cholecystitis, is lower than many previous smaller studies have suggested and that ultrasound may fail to identify over half of all cases of cholecystitis and a quarter of all cases of acute cholecystitis. This should serve as cautionary evidence for clinicians relying on ultrasound compared with some of the previously reported diagnostic metrics in small studies that suggested sensitivity of over 90%.⁷ Our findings in a large patient group are supported by the findings by Bingener et al in their smaller cohort.³

We have shown that diagnostic metrics are very similar, if not better, when ultrasound scans are performed by sonographers rather than consultant radiologists. This has significant implications internally for our study in showing that all our data is of the same quality as that performed by fully qualified consultant radiologists, but moreover there is wider applicability when it comes to the debate in the international literature regarding surgeon- and ED physician-performed ultrasound in the diagnosis of cholecystitis. Our data provides real-world performance metrics across the range of staff commonly performing ultrasound in the UK. When previously reported surgeon and ED physician data are compared with our data, it appears to support the premise that many appropriately trained staff groups can perform ultrasound to the same standard for the diagnosis of cholecystitis.^6–8

We present data concerning the ability of ultrasound alone to predict the Nassar difficulty of cholecystectomy and, furthermore, the way in which both the sensitivity and ability of ultrasound to predict the Nassar grade degrade with increased time between the ultrasound and cholecystectomy. We have shown that the sensitivity of ultrasound for the detection of cholecystitis degrades with increasing time between ultrasound and cholecystectomy and is no better than 50% (ie chance) after 140 days. At 30 days it is 56.5% for all cholecystitis and 77% for acute cholecystitis. We have also shown that the presence of gallbladder wall thickening on ultrasound has a highly statistically significant association with a ‘difficult cholecystectomy’ (Nassar grade ≥3). The odds ratio of a thickened gallbladder wall on ultrasound predicting a ‘difficult cholecystectomy’ was 4.5 and the odds were highest when the interval between ultrasound and cholecystectomy was 27 days or fewer. On the basis of these two different metrics, we submit that a new evidence-based recommendation can be given to the clinician that all ultrasounds used for the diagnosis and operative planning of acute cholecystitis should be no more than 27 days old. Use of this recommendation will improve diagnostic accuracy and the preparedness of the surgeon and theatre team. This data further supports the contemporaneous trend towards acute cholecystectomy, something we have long espoused in our unit.^16,17 In addition, this evidence-based recommendation may have particular relevance in the coming years when internationally the COVID pandemic is likely to cause significant delays for cholecystectomies (both elective and deferred emergencies) and the clinician may need the evidence presented in our study to assess the validity of an ultrasound carried out months or even years before.

Conclusions

We have presented robust diagnostic metrics for ultrasound in the diagnosis of cholecystitis. These should caution the clinician that ultrasound may miss a quarter of cases of acute cholecystitis and over half of all cases of cholecystitis. Conversely, the finding of a thickened gallbladder wall on ultrasound can predict a ‘difficult cholecystectomy’ and highlight the need for appropriate expertise and resources. Both this prediction and the diagnostic sensitivity are best if the ultrasound is done less than 27 days before cholecystectomy.