Abstract
Introduction
Clinical ultrasound (CUS) is highly specific for the diagnosis of acute appendicitis but is operator-dependent. The goal of this study was to determine if a heterogeneous group of emergency physicians (EP) could diagnose acute appendicitis on CUS in patients with a moderate to high pre-test probability.
Methods
This was a prospective, observational study of a convenience sample of adult and pediatric patients with suspected appendicitis. Sonographers received a structured, 20-minute CUS training on appendicitis prior to patient enrollment. The presence of a dilated (>6 mm diameter), non-compressible, blind-ending tubular structure was considered a positive study. Non-visualization or indeterminate studies were considered negative. We collected pre-test probability of acute appendicitis based on a 10-point visual analog scale (moderate to high was defined as >3), and confidence in CUS interpretation. The primary objective was measured by comparing CUS findings to surgical pathology and one week follow-up.
Results
We enrolled 105 patients; 76 had moderate to high pre-test probability. Of these, 24 were children. The rate of appendicitis was 36.8% in those with moderate to high pre-test probability. CUS were recorded by 33 different EPs. The sensitivity, specificity, and positive and negative likelihood ratios of EP-performed CUS in patients with moderate to high pre-test probability were 42.8% (95% confidence interval [CI] [25–62.5%]), 97.9% (95% CI [87.5–99.8%]), 20.7 (95% CI [2.8–149.9]) and 0.58 (95% CI [0.42–0.8]), respectively. The 16 false negative scans were all interpreted as indeterminate. There was one false positive CUS diagnosis; however, the sonographer reported low confidence of 2/10.
Conclusion
A heterogeneous group of EP sonographers can safely identify acute appendicitis with high specificity in patients with moderate to high pre-test probability. This data adds support for surgical consultation without further imaging beyond CUS in the appropriate clinical setting.
INTRODUCTION
Acute appendicitis is inflammation of the appendix that can lead to perforation, abscess, other serious infections and death. Over 280,000 appendectomies are performed in the United States annually.1 Although widespread availability of computed tomography (CT) has allowed more accurate diagnosis of acute appendicitis before reaching the operating room, this has come at the price of increased radiation exposure, increased cost and longer emergency department (ED) lengths of stay.2–4
Due to these risks, it is common to perform ultrasound examinations as the initial imaging modality in children to diagnose acute appendicitis.5 Nonetheless, ultrasonography for appendicitis is not available in many EDs, and in most departments the availability of diagnostic ultrasonography is limited by the time of day.6,7 Even when available, the accuracy of formal radiology ultrasound may be much lower in community practice than in academic centers where it is commonly studied.8
Previous studies have demonstrated excellent specificity of point-of-care or clinical ultrasound (CUS) for acute appendicitis among small cohorts of highly trained sonographers,9–15 and incorporation of clinical risk-stratification with sonography has been shown to safely enhance diagnostic accuracy in a variety of settings.16–20 However, the accuracy of ultrasound is highly dependent on the skills of the operator. This may be a barrier to implementation of CUS for appendicitis in new settings. The goal of this study was to determine if a heterogeneous group of emergency physicians (EP) could diagnose acute appendicitis on CUS. We hypothesized that EP sonographers could diagnose acute appendicitis with high specificity using a combination of clinical risk assessment, CUS, and self-assessment of image acquisition and interpretation.
METHODS Study Design
This was a prospective observational study on a convenience sample of adult and pediatric patients presenting to the ED with signs and symptoms concerning for acute appendicitis. Patients were enrolled from three large urban academic EDs between July 2014 and September 2016. The study sites consisted of two adult centers with a combined annual census of approximately 205,000, and one pediatric center with an annual volume >40,000 patient visits. The study protocol was approved by the institutional review board.
We included any patient with suspected acute appendicitis who underwent a diagnostic EP-performed CUS. Children and pregnant women were included. We excluded patients if CUS images were obtained after formal radiology imaging, or if data collection forms had missing information (including patient demographic information, pre-test or post-test probability, or interpretation).
Study Protocol
CUS was performed at the discretion of the treating clinician after history and physical examination. Prior to CUS, the treating physician recorded pre-test probability of appendicitis on a 10-point visual analog scale (VAS) using clinical gestalt. Following ultrasound, the sonographer filled out a standardized data collection form including the ultrasound diagnosis (appendicitis, indeterminate, or no appendicitis) and confidence in ultrasound interpretation on a 10-point VAS.
Sonographers included emergency medicine residents, ultrasound fellows, and board-certified emergency medicine faculty. All sonographers underwent a structured, 20-minute CUS training on appendicitis including didactics and hands-on scanning of one live model. CUS was performed after parenteral analgesics using a linear 5–10 MHz probe (Zonare ZS3 or Z.One Pro, Mindray Zonare, Mountain View, CA). The patient was in a supine position with hips flexed to relax the abdominal musculature. Graded compression was applied over the patient’s maximal site of pain in the right lower quadrant of the abdomen. The presence of a dilated (>6 mm diameter), non-compressible, blind-ending (in long axis) tubular structure was considered a positive study. Secondary signs of appendicitis were not assessed. Non-visualization or indeterminate studies were considered negative.
Outcome
The primary outcome was the diagnostic accuracy of CUS for acute appendicitis in patients with moderate and high pre-test probability. We used unstructured clinical assessment on VAS to determine pre-test probability, as clinical judgment has been shown to outperform clinical decision tools such as the Alvarado score.21 Pre-test probability of appendicitis was grouped into categories of low (1–3), moderate (4–6), and high (7–10). The criterion standard for diagnosis was surgical pathology results for those patients who went to the operating room, and chart review at hospital discharge and one week post-index ED visit for patients who did not go to the operating room. Local and statewide electronic medical records (EMR) were reviewed for repeat ED visits or hospitalizations for missed cases. We defined a missed case of acute appendicitis as a discharge diagnosis or surgical pathology diagnosis of acute appendicitis after the index visit.
Data Analysis
The expected rate of appendicitis was 35%.11–13 We expected specificity to be 85%, based on prior studies demonstrating a specificity ranging from 71 to 91%.17 A sample size of 75 patients with moderate to high pre-test probability of appendicitis was planned to demonstrate specificity within 10% of the expected value. This calculation assumes a power of 0.80 and alpha of 0.05. We calculated sensitivity, specificity, and positive and negative likelihood ratios with 95% confidence intervals (CI). Twenty percent of studies were randomly selected for blinded review by a fellowship-trained expert to calculate observed agreement and inter-rater reliability between EP sonographers’ interpretations using Cohen’s unweighted kappa. The expert reviewer was blinded to sonographer identity, sonographer interpretations, and clinical data.
RESULTS
During the study period 122 patients underwent CUS. Seventeen studies were excluded for missing data on the data collection form, including missing or incorrect patient medical record numbers, missing sonographer interpretation or pre-test probability. Of the remaining 105 patients, 76 (72%) had moderate or high pre-test probability (see Figure). Of these 76 patients, 28 (36.8%) had acute appendicitis (Table 1). There were 27 children, of whom 24 had moderate or high pre-test probability for appendicitis. Two pregnant women underwent CUS and both had a low pre-test probability. At one-week, EMR follow-up there were no missed cases of acute appendicitis.
Figure.
Flow chart of patients with moderate to high risk appendicitis and use of clinical ultrasound. Among patients with moderate to high risk of appendicitis, clinical ultrasound identified 12/28 cases of acute appendicitis. Among positive CUS scans, all tests with high sonographer confidence were true positives.
CUS, clinical ultrasound; PTP, pre-test probability
Table 1.
Patient characteristics with moderate to high pre-test probability.
| Appendicitis (n=28) | No appendicitis (n=48) | |
|---|---|---|
| Age, mean | 23.9 ±13.1 | 28.7 ±16.1 |
| Age <18 | 12 | 12 |
| Sex (M) | 69.6% | 59.5% |
| BMI | 24.5 ±6.6 | 26.3 ±4.7 |
| Symptom duration (d) | 1.1 ±1.1 | 1.5 ±1.6 |
| Fever | 32.1% | 27.1% |
| Vomiting | 50.0% | 41.7% |
| Rebound | 46.4% | 27.1% |
| Migration | 82.1% | 27.1% |
| Anorexia | 78.6% | 56.3% |
| White blood cells | 12.9 ±3.8 | 10.7 ±5 |
| Alvarado score | 4.6 ±1.4 | 2.2 ±1.3 |
| Formal radiology imaging | 90% | 90% |
The sensitivity and specificity of EP-performed CUS in patients with moderate to high pre-test probability were 42.8% (95% CI [25–62.5%]) and 97.9% (95% CI [87.5–99.8%]) (Table 2). The positive and negative likelihood ratios were 20.6 (95% CI [2.8–149.9]) and 0.58 (95% CI [0.42–0.8]). In 31 studies sonographers reported high confidence in image acquisition and interpretation (6 or higher on a 1–10 VAS). Of these studies, the sensitivity and specificity improved to 80% and 100%, respectively. The 16 false negative scans all were interpreted as indeterminate; for all 16, appendicitis was confirmed by CT at the index visit. There was one false positive ultrasound. For this study the sonographer reported low confidence in image interpretation (2 out of 10). This patient had a CT that demonstrated an obstructing ureteral stone at the right ureterovesical junction. Two patients proceeded directly to the operating room for appendectomy based on a positive CUS with no further imaging.
Table 2.
Sensitivity and specificity by level of training.
| Faculty | Fellow | Resident | Total | |
|---|---|---|---|---|
| N | 25 | 11 | 40 | 76 |
| Sensitivity | 50% | 75% | 31% | 43% |
| Specificity | 100% | 100% | 96% | 98% |
Thirty-three different sonographers performed CUS with a range of 1–13 scans per sonographer. Residents performed 40 (52.6%) of the CUS and identified five (41.7%) of the true positives. Inter-rater reliability was high, with 100% agreement and kappa = 1 (95% CI [1–1]).
DISCUSSION
Our study demonstrates that a heterogeneous group of EPs can diagnose acute appendicitis by CUS with high specificity in the appropriate clinical context. In our study, most sonographers had performed an average of 100 prior CUS examinations. Pre-enrollment training was limited to 20 minutes of didactics and hands-on training with a healthy model.
Recent research has shown diagnostic accuracy can be improved by combining clinical assessment with sonography.16–20 Therefore, we identified patients with moderate to high pre-test probability. Since ultrasound accuracy is highly dependent on the skill of the sonographer, we also collected data on sonographer confidence. Higher confidence in the ultrasound diagnosis yielded improved sensitivity and specificity of the results. This suggests that confidence in image acquisition and interpretation is an important predictor of diagnostic accuracy for CUS.
These results are consistent with prior studies demonstrating high specificity and moderate sensitivity of EP-performed CUS for acute appendicitis. In a meta-analysis of 21 studies, Fields et al. showed high specificity of 92% and relatively high sensitivity of 80% for CUS by EPs.15 Our sensitivity is lower than that reported by Fields et al. because the authors excluded non-diagnostic studies from analysis. These results further support the use of CUS as a first-line imaging modality in patients with suspected appendicitis. CUS has potential advantages as compared to CT with reduced time to diagnosis, reduced costs, reduced radiation and contrast dye exposure, and shorter ED stays.2,4
LIMITATIONS
The generalizability of this study is limited by the use of a convenience sample design and small number of subjects. There may be spectrum bias based on the inclusion criteria. EP sonographers were not blinded to patient history or physical exam, which could impact real-time interpretation of the images. However, this reflects pragmatic use of CUS in EDs during the early adoption period. The rates of acute appendicitis in this cohort are consistent with prior studies, suggesting that physicians used CUS in a group of patients similar to those seen in routine clinical practice. Follow-up was limited to one-week chart review, but it is unlikely any patients were missed due to use of a statewide-linked EMR. Although highly specific, the sensitivity of 42.8% does not support the use of CUS to rule out acute appendicitis in moderate to high pre-test probability patients.
CONCLUSION
A heterogeneous group of EPs can safely identify acute appendicitis on CUS with high specificity and a positive likelihood ratio of 20. Clinical risk stratification and appraisal of image quality and interpretation may improve diagnostic accuracy. Surgical consultation without further imaging beyond CUS may be supported in the appropriate clinical setting. This data does not support the use of CUS to rule out appendicitis when there is persistent clinical concern.
Footnotes
Section Editor: Shane Summers, MD
Full text available through open access at http://escholarship.org/uc/uciem_westjem
Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.
REFERENCES
- 1.Centers for Disease Control and Prevention. Ambulatory and inpatient procedures in the United States. 1996;13(139):26. [Google Scholar]
- 2.Krajewski S, Brown J, Phang PT, et al. Impact of computed tomography of the abdomen on clinical outcomes in patients with acute right lower quadrant pain: a meta-analysis. Can J Surg. 2011;54(1):43–53. doi: 10.1503/cjs.023509. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.American Society of Radiologic Technologists. X-ray Radiation Risk Calculator. [Accessed Mar 7 2017]. Available at: http://www.xrayrisk.com/calculator/calculator.php.
- 4.Elikashvili I, Tay ET, Tsung JW. The effect of point-of-care ultrasonography on emergency department length of stay and computed tomography utilization in children with suspected appendicitis. Acad Emerg Med. 2014;21:163–70. doi: 10.1111/acem.12319. [DOI] [PubMed] [Google Scholar]
- 5.Smith MP, et al. ACR Appropriateness Criteria: Right Lower Quadrant Pain - Suspected Appendicitis. American College of Radiology; [Accessed May 2017]. Available at: https://acsearch.acr.org/docs/69357/Narrative/ [Google Scholar]
- 6.Fullerton K, Depinet H, Iyer S, et al. Association of hospital resources and imaging choice for appendicitis in pediatric emergency departments. Acad Emerg Med. 2017;24(4):400–9. doi: 10.1111/acem.13156. [DOI] [PubMed] [Google Scholar]
- 7.Tan VF, Patlas MN, Katz DS. Imaging trends in suspected appendicitis - a Canadian perspective. Emerg Radiol. 2017;24(3):249–54. doi: 10.1007/s10140-016-1472-4. [DOI] [PubMed] [Google Scholar]
- 8.Alter SM, Walsh B, Lenehan PJ, et al. Ultrasound for diagnosis of appendicitis in a community hospital emergency department has a high rate of nondiagnostic studies. J Emerg Med. 2017;52(6):833–8. doi: 10.1016/j.jemermed.2017.01.003. [DOI] [PubMed] [Google Scholar]
- 9.Chen SC, Wang HP, Hsu HY, et al. Accuracy of ED sonography in the diagnosis of acute appendicitis. Am J Emerg Med. 2000;18(4):449–52. doi: 10.1053/ajem.2000.7343. [DOI] [PubMed] [Google Scholar]
- 10.Fox JC, Solley M, Anderson CL, et al. Prospective evaluation of emergency physician performed bedside ultrasound to detect acute appendicitis. Eur J Emerg Med. 2008;15(2):80–5. doi: 10.1097/MEJ.0b013e328270361a. [DOI] [PubMed] [Google Scholar]
- 11.Lam SH, Grippo A, Kerwin C, et al. Bedside Ultrasonography as an adjunct to routine evaluation of acute appendicitis in the emergency department. West J Emerg Med. 2014;15(7):808–15. doi: 10.5811/westjem.2014.9.21491. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Mallin M, Vraven P, Ockerse P, et al. Diagnosis of appendicitis by bedside ultrasound in the ED. Am J Emerg Med. 2015;33(3):430–2. doi: 10.1016/j.ajem.2014.10.004. [DOI] [PubMed] [Google Scholar]
- 13.Gungor F, Kilic T, Akyol KC, et al. Diagnostic value and effect of bedside ultrasound in acute appendicitis in the emergency department. Acad Emerg Med. 2017;24(5):578–86. doi: 10.1111/acem.13169. [DOI] [PubMed] [Google Scholar]
- 14.Benabbas R, Hanna M, Shah J, et al. Diagnostic accuracy of history, physical exam, laboratory tests and point-of-care-ultrasound for pediatric acute appendicitis in the emergency department: a systematic review and meta-analysis. Acad Emerg Med. 2017;24(5):523–51. doi: 10.1111/acem.13181. [DOI] [PubMed] [Google Scholar]
- 15.Fields MJ, Davis J, Alsup C, et al. Accuracy of point-of-care ultrasonography for diagnosing acute appendicitis: a systematic review and meta-analysis. Acad Emerg Med. 2017;24(9):1124–36. doi: 10.1111/acem.13212. [DOI] [PubMed] [Google Scholar]
- 16.Schuh S, Chan K, Langer JC, et al. Properties of serial ultrasound clinical diagnostic pathway in suspected appendicitis and related computed tomography use. Acad Emerg Med. 2015;22(4):406–14. doi: 10.1111/acem.12631. [DOI] [PubMed] [Google Scholar]
- 17.Löfvenberg F, Salö M. Ultrasound for appendicitis: performance and integration with clinical parameters. Biomed Res Int. 2016;2016:5697692. doi: 10.1155/2016/5697692. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Bachur RG, Callahan MJ, Monuteaux MC, et al. Integration of ultrasound findings and a clinical score in the diagnostic evaluation of pediatric appendicitis. J Pediatr. 2015;166(5):1134–9. doi: 10.1016/j.jpeds.2015.01.034. [DOI] [PubMed] [Google Scholar]
- 19.Leeuwenburgh MM, Stockmann HB, Bouma WH, et al. A simple clinical decision rule to rule out appendicitis in patients with nondiagnostic ultrasound results. Acad Emerg Med. 2014;21(5):488–96. doi: 10.1111/acem.12374. [DOI] [PubMed] [Google Scholar]
- 20.Ross MJ, Liu H, Netherton SJ, et al. Outcomes of children with suspected appendicitis and nonvisualized appendix on ultrasound. Acad Emerg Med. 2014;21(5):538–42. doi: 10.1111/acem.12377. [DOI] [PubMed] [Google Scholar]
- 21.Mán E, Simonka Z, Varga A, Rárosi F, Lázár G. Impact of the Alvarado score on diagnosis of acute appendicitis. Surgical Endoscopy. 2014;28(8):2398–405. doi: 10.1007/s00464-014-3488-8. [DOI] [PubMed] [Google Scholar]