Improving Ultrasound Quality to Reduce Computed Tomography Use in Pediatric Appendicitis: The Safe and Sound Campaign

Abstract

Introduction

Safety concerns about the use of radiation-based imaging such as computed tomography (CT) in children have resulted in national recommendations to use ultrasound (US) for diagnosis of appendicitis when possible. We evaluated trends in CT and US use in a statewide sample and the accuracy of these modalities.

Methods

Patients ≤18 years old undergoing appendectomy in Washington State from 2008 to 2013 were evaluated for preoperative US/CT use as well as imaging/pathology concordance using data from the Surgical Care and Outcomes Assessment Program.

Results

Among 3353 children, 98.3% underwent preoperative imaging. There was a significant increase in the use of US first over the study period (p<.001). The use of CT at any time during the evaluation decreased. Despite this, in 2013 over 40% of children still underwent CT imaging. Concordance between US imaging and pathology varied between 40–75% at hospitals performing ≥10 appendectomies in 2013. Over one third (34.9%) of CT scans performed in evaluation of children with appendicitis were performed after an indeterminate US.

Discussion

While use of US as the first imaging modality to diagnose pediatric appendicitis has increased over the past five years, over 40% of children still undergo a CT scan during their preoperative evaluation. Causality for this persistence of CT use is unclear, but could include variability in ultrasound accuracy, lack of training, and lack of awareness of the risks of radiation-based imaging. Developing a campaign to focus on continued reduction in CT and increased use of high-quality ultrasound should be pursued.

Introduction

Appendicitis is a common cause of abdominal pain and emergency department (ED) evaluation in the pediatric population, with approximately 70,000 cases presenting annually in the United States.^1,2 Evaluation for possible appendicitis comprises 5–10% of all pediatric ED visits. The use of diagnostic imaging has become increasingly common since its use has been shown to reduce the rate of negative appendectomy, or finding the appendix to be normal intraoperatively.^3,4,5,6,7. There are increasing safety concerns regarding the use of radiation-based imaging modalities such as computed tomography (CT), given the potential increased lifetime risk of malignancy.^8,9. The American Academy of Pediatrics, the National Cancer Institute and the American Pediatric Surgical Association have recommended the preferential use of non-radiation-based imaging, specifically ultrasound (US), in the evaluation of children with abdominal pain.^10,11,12

Despite these national recommendations and the growing body of literature regarding potential risks of radiation exposure in children, CT scans still constitute a large percentage of imaging studies performed in children with appendicitis.^1,13 Our group found that between 2008 and 2012, over 50% of children who ultimately went on to have an appendectomy received a CT scan as their first study upon presentation to an ED.¹⁴ A number of factors may be driving this use of CT scans, including the perception of decreased accuracy of US relative to CT and the operator-dependent nature of US studies.

In order to better understand current use of CT and ultrasound, we evaluated trends in CT and US use among children undergoing appendectomies in Washington State. The purpose of this study was to explore use of diagnostic imaging modalities over time and the accuracy of these imaging modalities over time and across hospitals.

Materials and Methods

Study Population and Setting

The Surgical Care Outcomes and Assessment Program (SCOAP) is a physician-led quality improvement collaborative that began in 2006 and has subsequently enrolled nearly all hospitals in Washington State. The Comparative Effectiveness Research Translation Network (CERTAIN) is a translational research network composed of thirty-five clinics and twenty-five hospitals in Washington, which uses a unique data-sharing platform to allow investigators and providers to track quality, benchmark best practices, and improve care. SCOAP uses prospective review of clinical records of all patients undergoing specific procedures at participating hospitals and data are collected by trained abstractors. We identified our study population from within the thirty-two hospitals participating in SCOAP and CERTAIN that have provided non-elective appendectomy care to pediatric patients since 2008.

Data Characteristics and Primary Outcome

Demographic information, clinical characteristics, diagnostic imaging use, radiologic interpretations, operative findings and pathology results are abstracted from the clinical record using standardized definitions. Trained abstractors collect the data using a standardized process across all participating sites. In this study, we focused our analysis on diagnostic imaging use, radiologic interpretations and pathology results to understand trends in imaging use and imaging-pathology concordance. We collected data on all consecutive non-elective appendectomies from 2008 to 2013. Interval appendectomies are included in the database. Perforation of the appendix is based on gross evidence of perforation intraoperatively or pathologic findings. Research projects using de-identified SCOAP data are exempted from review by the University of Washington Institutional Review Board.

Concordance between imaging and pathology is determined based on whether the findings of these two modalities agree. The results of each imaging study are based on the final radiologist interpretation and are reported as consistent with appendicitis, not consistent with appendicitis, or indeterminate. From a pathologic standpoint, the appendix is identified as being either positive or negative for appendicitis. The imaging and pathology results are reported as concordant if the imaging is consistent with appendicitis and the pathology is positive or if the imaging is not consistent with appendicitis and the pathology is negative. All other combinations of imaging and pathology are considered non-concordance. In addition, indeterminate imaging findings are considered non-concordant regardless of the pathologic result.

The primary outcome in this study was the imaging modality used first in the patient’s workup – CT or US. Secondary outcomes included the use of CT at any time during the patient’s evaluation and imaging-pathology concordance. Concordance rates by hospital were evaluated at hospitals performing ≥10 appendectomies in 2013.

Analytic Methods

Trends in imaging use over time were evaluated. The first two years of data collection were pilot years (2008–2009) and as such were grouped together for the purpose of evaluating trends in imaging use. Concordance between imaging and pathology were evaluated for CT and US use, both at the state and hospital level. The use of CT at any time during the patient’s work-up was also evaluated over the study period. STATA version 11 was used for all analyses (STATA Corp, College Station, TX). Statistical significance was set at p<0.05.

Results

Cohort Characteristics

During the study period (2008–2013), 3353 children underwent a non-elective appendectomy. Of these, 98.3% (n=3296) underwent preoperative imaging. Of the 3353 patients, the mean age was 11.3 years (SD 4.0 years) and 59.4% were male (table 1). Fifty-eight percent of patients had private insurance, 67.4% were White, and 56.5% were not Hispanic/Latino. Nearly 20% of the patients were overweight (18.4%) and obese (20.2%). Twenty-three percent of patients presented with perforation and 4.2% had a negative appendectomy.

Table 1.

Demographic Characteristics of Study Population

		All
Number of Children (%)		4038
Mean Age (SD)		11.3 (4.0)
Age Group (%)
Age ≤5		331 (9.2)
5<Age≤10		1194 (29.6)
10<Age≤18		2513 (62.2)
Sex (%)
Male		2396 (59.4)
Female		1639 (40.6)
Insurance (%)
Private		2305 (58.0)
Medicaid		1360 (34.2)
Uninsured/Self-Insured		128 (3.2)
Medicare/Tricare/Indian Health Service/VA		183 (4.6)
BMI Group
Normal		1398 (61.4)
Overweight		419 (18.4)
Obese		459 (20.2)
Race, %
White		2571 (67.4)
Black or African American		103 (2.7)
Asian		150 (3.9)
American Indian/Alaska Native		67 (1.8)
Native Hawaiian or Other Pacific Islander		25 (0.66)
Unknown/NA		898 (23.5)
Ethnicity (%)
Hispanic or Latino		930 (24.3)
Not Hispanic or Latino		2160 (56.5)
NA		736 (19.2)
Hospital Type
General		1938 (50.0)
Pediatric Unit in General		552 (14.3)
Free-Standing Pediatric		1384 (35.7)
Transfer from Another Hospital (%)		770 (19.2)
Perforation Rate (%)		927 (23.0)
Negative Appendectomy (%)		168 (4.2)

First Imaging Study

Over the course of the study period, the use of US as the first imaging modality more than doubled (32.6% to 69%, p<0.001; figure 1). Moreover, the use of CT at any time (not just as the first imaging modality) during the patient’s evaluation decreased over the course of the study period (figure 2). In 2013, however, over 40% of children still underwent CT imaging at some point in their work-up for appendicitis. Over one third (34.9%) of these CT scans performed were performed after an indeterminate US. Among patients undergoing a CT scan after an indeterminate US, 30% had perforated appendicitis.

Figure 1.

Use of Ultrasound as First Imaging Study, 2008–2013

Figure 2.

Use of CT at Any Time During Evaluation, 2008–2013

Concordance

US concordance statewide was 68.5% in 2013, although concordance varied between 40–75% by hospital. CT concordance statewide was 89.5% in 2013. Neither CT nor US concordance varied significantly over the study period (figure 3).

Figure 3.

Ultrasound and CT Concordance, 2008–2013

Discussion

Over the course of the past five years, use of US as the first diagnostic imaging study in the evaluation of children with pediatric appendicitis has increased significantly. The use of CT as the first study has decreased, but the use of CT at some point during the evaluation remains high, at over 40% in the most recent year of the study.

Over a third of these CT scans were performed after a non-diagnostic US, suggesting that quality of studies may be a significant driver for the use of radiation-based imaging in children with appendicitis. Overall, there is a significant gap between US and CT accuracy in our state, as well as high levels of variability of US accuracy between hospital sites.

Ultrasound is a highly operator-dependent modality, requiring skill on the part of the ultrasonographer to identify both the appendix as well as secondary signs in patients with appendicitis.^15,16. Additionally, the accuracy of the study has been found to be volume dependent, with lower accuracy among sites that use the modality less frequently.¹⁷ These factors – both the operator-dependent nature of the study as well as the lower accuracy when it is used less frequently – may drive providers to order CT scans for the evaluation of pediatric abdominal pain, especially in hospitals with lower pediatric volumes. Our recent evaluation in Washington State found that non-children’s hospitals had significantly higher odds of CT use, after controlling for population characteristics, and significantly lower rates of US concordance.¹⁴

It is worth noting that in our sample, ultrasound concordance and CT concordance rates are lower than otherwise described in the literature. There may be a number of reasons for this finding, in particular the diverse hospital types found within the SCOAP database, including critical access hospitals, adult community hospitals, and freestanding children’s hospitals. Additionally, because SCOAP is a procedural database, we are not able to measure sensitivity and specificity, but rather concordance between imaging and pathology, which may affect our results as we are not examining the accuracy of either US or CT in patients who do not go on to have an appendectomy. This may also result in lower concordance.

In this study, we found that there has been a significant and documented decrease in CT use in children with appendicitis, perhaps due to documented safety concerns about CT use in children and development of national recommendation. However, the documentation of persistent high rates of CT use in our statewide study is worrisome, suggesting that guidelines alone may not be sufficient to change behavior to reduce use of radiation-based imaging in pediatric patients. Based on these findings, we established a public health campaign, entitled Safe and Sound, in Washington State in early 2014, focused on reducing radiation exposure to children by (1) improving US quality through US training and standardized reporting and (2) promoting an US-first strategy for evaluation for abdominal pain in children with diagnostic protocols highlighting US. Nine of the fifteen hospitals in the state of Washington that perform over 90% of the pediatric appendectomies, including both pediatric and non-pediatric facilities, elected to participate in the campaign. Forthcoming studies will evaluate the effectiveness of this approach.

The results of this study must be interpreted in the context of study design. First of all, the SCOAP databases are procedural databases that collect information on consecutive patients undergoing appendectomy at participating hospitals in Washington State. As such, our study does not evaluate all patients who undergo imaging for the evaluation of acute abdominal pain and may represent a biased sample for that reason. Additionally, because of the procedural nature of the database, we are unable to calculate sensitivity and specificity of US and CT and so have evaluated concordance between imaging and pathology findings. Secondly, the dataset is unable to capture clinical decision-making that may have influenced how patients are allocated to imaging and why a CT may have been chosen over ultrasound.

Conclusions

This study indicates that while ultrasound first use has increased over the past five years, CT use at some point in the evaluation of children with pediatric appendicitis remains high. This use may be driven by the decreased accuracy and operator-dependent nature of ultrasound. Use of a public health campaign focused on training, standardized reporting of studies, and pre-operative protocols that emphasize US first may be effective at improving the use and quality of ultrasound in the diagnosis of pediatric appendicitis.