Accuracy of ultrasound for the diagnosis of acute appendicitis in the emergency department: A systematic review

Background:

Point-of-Care Ultrasound (POCUS) is a quick, useful, noninvasive, and inexpensive diagnostic tool used for the diagnosis of trauma, abdominal pain, dyspnea, and chest pain in the emergency department (ED). However, the diagnostic accuracy of ultrasound in the ED may be different from those reported in previous studies owing to the setting and time constraints in ED.

Methods:

We conducted our study in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines. A literature search was conducted using databases on US National Library of Medicine’s database of biomedical literature, Ovid MEDLINE, online database of biomedical articles, and the collection of databases of systematic reviews and other evidence. The inclusion criteria were the use of bedside ultrasound as a diagnostic tool for acute appendicitis in the ED and the available data on diagnostic parameters such as sensitivity, specificity, and positive and negative predictive values (NPV). We constructed forest plots and summary receiver operating characteristic curves to evaluate the diagnostic accuracy of bedside ultrasound for acute appendicitis in the ED.

Results:

A total of 21 studies that met the inclusion criteria of this study were included for analysis. The overall pooled sensitivity was 0.81 (95% CI, 0.78–0.83), whereas the pooled specificity was 0.87 (95% CI, 0.85–0.88). However, the I² test showed 91.7% and 90.9% heterogeneity in the sensitivity and specificity values, respectively. The summary receiver operating characteristic curves showed high levels of accuracy, as evidenced by an area under the curve of 0.9249 (standard error: 0.0180).

Conclusions:

The use of ultrasound for the diagnosis of acute appendicitis in the ED showed that ultrasound has high overall sensitivity and specificity for the diagnosis of acute appendicitis. however, high heterogeneity among the included studies was observed.

1. Introduction

Acute appendicitis is a severe inflammation of the appendix that requires emergency surgery.^[1,2] The sensitivity and specificity of ultrasound for the diagnosis of acute appendicitis range from 75% to 90% and 86% to 95%, respectively.^[3,4] Several studies have been conducted to investigate the accuracy of ultrasound for the diagnosis of acute appendicitis; however, the diagnostic accuracies reported in these previous studies vary.^[5,6]

Point-of-Care Ultrasound (POCUS) is a quick, useful, noninvasive, and inexpensive diagnostic tool used for the diagnosis of trauma, abdominal pain, dyspnea, and chest pain in the emergency department (ED).^[7,8] However, the diagnostic accuracy of ultrasound in the ED may be different from those reported in previous studies owing to the setting and time constraints in ED.^[9] Therefore, the aim of this systematic review and meta-analysis was to determine the accuracy of bedside ultrasound for the diagnosis of acute appendicitis in the ED.

2. Materials and methods

2.1. Study design

We systematically reviewed studies on bedside sonography for the diagnosis of acute appendicitis in the ED. This systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines.^[10]

2.2. Data sources and search strategy

The literature used in this study were extracted from the US National Library of Medicine’s database of biomedical literature, Ovid MEDLINE, online database of biomedical articles, collection of databases of systematic reviews and other evidence, and Google Scholar databases. The following keywords were used in the database search: “ultrasound,” “ultrasonography,” “sonography,” “US,” “USG,” “appendicitis,” “appendix,” “emergency,” “Emergency department,” and “ED.” There were no restrictions on the use of keyword combinations or dates in the title or abstract. However, the search was limited to articles published in English. Two researchers independently conducted each step of this study.

2.3. Eligibility criteria and study selection

Studies on the diagnosis of appendicitis in the emergency room were considered eligible for inclusion into this study. The titles and abstracts of extracted articles were read and the full texts of potentially relevant articles were screened. The inclusion criteria were the use of bedside ultrasound as a diagnostic tool for acute appendicitis in the ED and the available data on diagnostic parameters such as sensitivity, specificity, and positive and negative predictive values (NPV). If ultrasound results were positive, surgical pathology was used as a diagnostic reference for acute appendicitis; if negative, clinical follow-up was used as the diagnostic reference standard. The articles included in this study were selected by 2 or more investigators, and any disagreements were resolved through discussion.

2.4. Assessment of methodological quality

We assessed quality and applicability of each included study using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-II).^[11] The QUADAS-II consists of the 4 domains on patient selection, index test, reference standards, and flow and timing, and is designed to be used for the assessment of risk of bias and applicability for each domain by answering 14 questions as “low,” “high,” and “unclear” (Table 1). Two researchers (SKO and SUC) independently assessed each study and resolved discrepancies through discussion. Consensus among the reviewers was quantified using Cohen’s kappa, and discrepancies were resolved through discussion.

Table 1.

The QUADAS-2 tool for the quality assessment of diagnostic accuracy studies.

Item		Yes	No	Unclear
1.	Was the spectrum of patients representative of the patients who will receive the test in practice?
2.	Were selection criteria clearly described?
3.	Is the reference standard likely to correctly classify the target condition?
4.	Is the time period between reference standard and index test short enough to be reasonably sure that the target condition did not change between the 2 tests?
5.	Did the whole sample or a random selection of the sample, receive verification using a reference standard of diagnosis?
6.	Did patients receive the same reference standard regardless of the index test result?
7.	Was the reference standard independent of the index test (i.e., the index test did not form part of the reference standard)?
8.	Was the execution of the index test described in sufficient detail to permit replication of the test?
9.	Was the execution of the reference standard described in sufficient detail to permit its replication?
10.	Were the index test results interpreted without knowledge of the results of the reference standard?
11.	Were the reference standard results interpreted without knowledge of the results of the index test?
12.	Were the same clinical data available when test results were interpreted as would be available when the test is used in practice?
13.	Were uninterpretable/intermediate test results reported?
14.	Were withdrawals from the study explained?

QUADAS-2 = Quality assessment of diagnostic accuracy studies - second edition.

2.5. Data synthesis and analysis

We extracted basic demographic information and data on diagnostic sensitivity, specificity, positive predictive value, and NPV from each selected study. When statistical synthesis was possible, we used random-effects models for analysis of high or moderate heterogeneity and fixed- and random-effects models for evaluation of low heterogeneity. Heterogeneity was quantified using the I² test. In addition, we constructed forest plots and summary receiver operating characteristic curves to evaluate the diagnostic accuracy of bedside ultrasound for acute appendicitis in the ED. When the heterogeneity of the overall pooled sensitivity or pooled specificity was high, subgroup analysis of studies in which ultrasound was performed by an emergency physician was performed. Meta-Disc^[12] and Review Manager (version 5.3; Cochrane Collaboration, The Nordic Cochrane Centre, Copenhagen, The Netherlands) were used for data input and analysis.

3. Results

3.1. Search results

A total of 426 articles were retrieved during the database search. After screening the articles, 65 duplicates were removed, leaving 361 articles. The titles and abstracts of the remaining 361 articles were read and 306 ineligible articles were excluded. The full articles of the remaining 55 articles were reviewed and those that did not provide information on diagnostic parameters, such as sensitivity, specificity, positive predictive value, and NPV, were excluded. In addition, review articles, case reports, letters, comments, or articles on studies that were not performed by emergency physicians or residents were excluded. Finally, 21 studies that met the inclusion criteria of this study were included for analysis (Fig. 1).

Figure 1.

Flowchart of the selection process of included studies.

3.2. Study characteristics and quality assessment

3.2.1. Assessment of methodological quality.

Information on the total number of enrolled patients, first author, journal name, year of publication, sample size, mean age, study design, clinician who performed the ultrasound, and primary data outcomes, including true positives, false positives, true negatives, and false negatives, extracted from all included studies are shown in Table 2.^[13–33]

Table 2.

Characteristics of the included studies.

Yr	Author	Journal	Sample size	Mean age (yr)	Study design	Operator	TP	FP	FN	TN	Sensitivity	Specificity	PPV (%)	NPV (%)
2021	Becker[13]	Acade Emerg Med	256	19	Prospective	EM physician	61	68	11	116	85.0	63.0	47.3	91.3
2018	Doniger[15]	Pediatric Emergency Care	40	9.3 (Ped)	Prospective	EM physician	15	3	1	21	93.8	87.5	83.3	95.5
2018	Shahbazipar[14]	European Journal of Emergency Medicine	121	33.6	Prospective	EM residents	29	1	17	74	63	99	97	81
2018	Sharif[16]	CJEM	90	NA	Retrospective	EM physician or resident	18	6	8	58	69.2	90.6	75.0	87.9
2017	Karimi[17]	Emergency	108	23.91 (all)	Prospective	EM physician	27	12	10	59	80.0	83.1	69.2	85.5
2017	Gungor[18]	Acad Emerg Med	264	30.0 (adult)	Prospective	EM physician	156	4	13	91	92.3	95.8	97.5	87.8
2016	Ünlüer [19]	World J Emerg Med	100	32.9	Prospective	EM physician	34	18	12	36	73.9	66.7	65.4	75.0
2016	Topin[20]	The Journal of Emergency Medicine	100	33.2 (all)	Prospective	EM physician	22	3	3	72	88	96	88	96
2015	Fathi[21]	J Ultrasound	97	34.35 (all)	Prospective	EM physician or resident	19	8	24	46	44.2	85.2	70.4	65.7
2015	Kim[22]	American Journal of Emergency Medicine	115	NA (ped)	Prospective	EM physician or resident	33	8	3	71	91.7	89.9	80.5	95.6
2015	Mallin [23]	Am J Emerg Med	97	28	Prospective	EM physician or resident	23	1	11	62	67.6	98.4	95.8	84.9
2015	Kim[24]	Hong Kong Journal of Emergency Medicine	166	10.6 (ped)	retro	EM physician	40	0	0	126	100	100	100	100
2014	Sivitz[25]	Annals of Emergency Medicine	264	10.2 (Ped)	Prospective	EM physician	72	13	13	166	84.7	92.7	84.7	92.7
2014	Elikashvili[26]	Acade Emerg Med	150	12 (Ped)	Prospective	EM physician	30	5	20	95	60.0	95.0	85.7	82.6
2014	Lam [27]	Western Journal of Emergency Medicine	52	20.2	Prospective	EM physician	33	13	0	6	100	31.6	71.7	100
2013	Lin [28]	Iran J Pediatr	155	6 (Ped)	retro	EM physician	108	10	4	33	96.4	76.7	91.5	89.2
2008	Fox[29]	European Journal of Emergency Medicine	126	NA	Prospective	EM physician	37	7	20	62	64.9	89.9	84.0	75.6
2007	Fox[30]	The California Journal of Emergency Medicine	155	NA	Retrospective	EM physician or resident	27	9	42	77	39.1	90.0	75.0	64.7
2007	Siu [31]	Hong Kong Journal of Emergency Medicine	85	31.6	Prospective	EM physician	18	9	11	47	62.1	83.9	66.7	81.0
2000	Chen[32]	American Journal of Emergency Medicine	147	37.1 (all)	Prospective	EM physician	106	12	4	25	96.3	67.6	90.0	86.2
1998	Oh[33]	J Korean Soc Emerg Med	47	34.8	Prospective	EM resident	40	0	2	5	89.3	100	100	71.4

FN = false negative, FP = false positive, NA = not available, NPV = negative predictive value, PPV = positive predictive value, TN = true negative, TP = true positive.

The mean QUADAS-II score was 9.8 points (range: 6–13). The agreement between the QUADAS-II scores recorded by the 2 investigators was good (k = 0. 61). Figure 2 shows the high, medium, and low ratios of the risk of bias in the included studies for each of the 7 items of the QUADAS-II. The overall risk of bias and applicability were low.

Figure 2.

Summary results of quality assessment using the QUADAS-2 instrument. Risk of bias graph about each risk of bias item is presented as percentages (Green; low risk of bias, red: high risk of bias, yellow: unclear risk of bias).

3.2.2. Comparison of diagnostic accuracy.

Forest plots for the comparison of sensitivity, specificity, and confidence intervals (CIs) in each study are shown in Figure 3. The overall pooled sensitivity was 0.81 (95% CI, 0.78–0.83), whereas the pooled specificity was 0.87 (95% CI, 0.85–0.88). However, the I² test showed 91.7% and 90.9% heterogeneity in the sensitivity and specificity values, respectively. The summary receiver operating characteristic curves showed high levels of accuracy, as evidenced by an area under the curve of 0.9249 (standard error: 0.0180) and a Q-value of 0.8591 (standard error: 0.0211) (Fig. 4).

Figure 3.

Forest plots for sensitivity and specificity of all included studies. Each study is identified by name of first author and year of publication. Horizontal lines represents 95% confidence intervals (CIs).

Figure 4.

Summary receiver operating characteristics curve of sensitivity and specificity of ultrasonography for diagnosis of acute appendicitis in emergency department. AUC: area under curve, SE: standard error, Q*: point at which sensitivity and specificity are equal.

3.2.3. Subgroup analysis.

Subgroup analysis of studies in which an emergency medicine physician performed the ultrasound was conducted. Forest plots of the comparison of the sensitivity, specificity, and CIs reported in each study included in this subgroup analysis are shown in Figures 5 and 6. The results of the subgroup analysis indicated that the pooled sensitivity was 0.86 (95% CI, 0.84–0.88), whereas the pooled specificity was 0.84 (95% CI, 0.82–0.86). However, the I² test showed 88.4% and 93% heterogeneity in the sensitivity and specificity values, respectively.

Figure 5.

Forest plots for sensitivity and specificity for subgroup performed by emergency physicians. Each study is identified by name of first author and year of publication. Horizontal lines represents 95% confidence intervals (CIs).

Figure 6.

Summary receiver operating characteristics curve of sensitivity and specificity for subgroup performed by emergency physicians. AUC: area under curve, SE: standard error, Q*: point at which sensitivity and specificity are equal.

3.2.4. Publication bias.

A funnel plot was constructed to assess potential publication bias (Fig. 7). The funnel plot showed an asymmetric shape, indicating a potential for various types of biases such as search bias, publication bias, and selection bias in ultrasound-based diagnosis of acute appendicitis in the ED.

Figure 7.

Funnel pot analysis on the detection of publication bias. The funnel plot displays an asymmetrical distribution of the studies included.

4. Discussion

It is difficult for emergency physicians to diagnosis acute appendicitis because diagnosis based on clinical evaluation alone has a sensitivity of 39% to 74% and a specificity of 57% to 84%.^[34] The imaging method most commonly used for the diagnosis of acute appendicitis is computed tomography. Computed tomography has been reported to have a high sensitivity of 91% to 98.5% and a specificity of 90% to 98% in the diagnosis of acute appendicitis.^[35,36] However, in crowded ED, time is critical and rapid disposition is crucial; thus, computed tomography may not be performed effectively.

Delayed diagnosis of appendicitis, which can lead to perforation, is the leading cause of morbidity and mortality in patients.^[37] The use of ultrasonography for the diagnosis of appendicitis is safe and cost-effective, and does not present any risk of radiation exposure.^[38] Several studies have shown that the use of bedside ultrasound in the ED is helpful in the diagnosis of acute appendicitis. Since most ED already have ultrasound machines, bedside ultrasound in the ED is fast, noninvasive, and safe. Several reports have shown that ultrasonography has high sensitivity, specificity, and accuracy in the differential diagnosis of acute appendicitis.^[39,40] However, this may be different in a busy ED or acute care setting, especially if the healthcare staff performing the procedure is not a radiologist. This is because a normal appendix is generally filled with air, making it more difficult to view with ultrasound than a pathological appendix. In addition, learning how to perform an ultrasound may be more difficult than learning other procedures, leading to differences in accuracy depending on the examiner.

Several meta-analyses have been conducted to evaluate the use of ultrasound for the diagnosis of acute appendicitis. Terasawa et al reported that the sensitivity and specificity of ultrasound for the diagnosis of acute appendicitis in their study were 86% and 81%, respectively, and that computed tomography was more accurate than ultrasound in the diagnosis of acute appendicitis.^[41] In addition, Yu et al analyzed a Korean study and reported that the sensitivity and specificity of ultrasound for the diagnosis of acute appendicitis in their study were 86.7% and 90.0%, respectively.^[42] Field et al conducted a meta-analysis of 21 studies on the diagnosis of acute appendicitis using ultrasound. The results of their study indicated that the sensitivity and specificity of POCUS for the diagnosis of appendicitis were 91% (95% CI = 83%–96%) and 97% (95% CI = 91%–99%), respectively.^[43] Lee et al conducted a meta-analysis of 17 studies on the use of POCUS for the diagnosis of acute appendicitis, and reported that the pooled sensitivity of POCUS for the diagnosis of acute appendicitis was 84% (95% CI: 72%–92%), whereas the pooled specificity was 91% (95% CI: 85%). In addition, they reported that ultrasound showed significantly better diagnostic performance for pediatric acute appendicitis (sensitivity, 95%; 95% CI, 75%–99%; specificity, 95%; 95% CI, 85%–98%).^[44]

The present study was focused on the use of ultrasound for the diagnosis of acute appendicitis in ED. Subgroup analysis of the included studies in which ultrasound was performed by an emergency physician was conducted. Overall, the pooled sensitivity and pooled specificity values for all the included studies and those in the above-mentioned subgroup were similar; however, high heterogeneity was observed in both results. The high heterogeneity in the specificity and sensitivity values reported in the included studies may be because the accuracy of ultrasonography is highly dependent on the skill of the clinician.^[45]

Formal ultrasound in the radiology department generally shows high accuracy in diagnosing appendicitis, with reported accuracy rates of over 85%. However, results can be uncertain when ultrasound is performed by a nonexpert physician. The POCUS is performed directly by skilled experts and has shown a similar accuracy to traditional radiologist-conducted ultrasound in multiple studies, leading to higher reliability.^[41–45] Ultimately, both POCUS and formal ultrasound in the radiology department have their respective strengths and weaknesses, and either may be preferred depending on the situation. If there is a stand-alone radiology unit in the ED where ultrasounds can be performed directly by radiologists, this may address the issue of delays and compensate for the limitations of POCUS.

This study had several limitations. First, it was difficult to determine the degree of bias in each study using the quality evaluation results because details on the history of the subjects, exclusion criteria, purpose of the screening test, and blinding were not clearly presented. Second, the degree of experience and education of the clinicians who performed ultrasound in the included studies was not investigated. Third, limiting the language of the included studies to English may have introduced some publication bias in the study. In addition, the study selection process was performed by only 1 reviewer, which may have resulted in missing some relevant studies. Fourth, since the funnel plot of the studies included in this analysis showed an asymmetrical distribution, there is a possibility of publication bias. There is also a possibility that studies reporting low sensitivity/specificity were not published, which may have led to an overestimation of the true effect size in the meta-analysis. Finally, many of the studies included did not clearly explain the diagnostic method used to evaluate nonvisualized appendixes or appendixes with uncertain findings. Moreover, the reference standards for the diagnosis of appendicitis varied across the studies, and descriptions of follow-ups and additional tests were ambiguous in some cases, which may have led to lower reported sensitivities and specificities for POCUS in actual practice.

5. Conclusion

This systematic review and meta-analysis of studies on the use of ultrasound for the diagnosis of acute appendicitis in the ED showed that ultrasound has high overall sensitivity and specificity for the diagnosis of acute appendicitis; however, high heterogeneity among the included studies and potential publication bias were observed. Therefore, additional tests are necessary when the POCUS result is positive, to establish a final diagnosis. It is also important to acknowledge the limitations and constraints of POCUS examination, and to interpret test results carefully and professionally.

Author contributions

Conceptualization: Se Kwang Oh.

Data curation: Sung Uk Cho, Se Kwang Oh.

Formal analysis: Sung Uk Cho.

Investigation: Se Kwang Oh.

Methodology: Se Kwang Oh.

Project administration: Sung Uk Cho.

Supervision: Se Kwang Oh.

Visualization: Sung Uk Cho.

Writing – original draft: Sung Uk Cho.

Writing – review & editing: Se Kwang Oh.