Use and accuracy of diagnostic imaging in the evaluation of pediatric appendicitis

Abstract

Background

There are safety concerns about the use of radiation-based imaging (computed tomography [CT]) to diagnose appendicitis in children. Factors associated with CT use remain to be determined.

Methods

For patients ≤18 years old undergoing appendectomy, we evaluated diagnostic imaging performed, patient characteristics, hospital type, and imaging/pathology concordance (2008–2012) using data from Washington State’s Surgical Care and Outcomes Assessment Program.

Results

Among 2538 children, 99.7% underwent pre-operative imaging. 52.7% had a CT scan as their first study. After adjustment, age >10 years (OR 2.9 (95% CI 2.2–4.0), Hispanic ethnicity (OR 1.7, 95% CI 1.5–1.9), and being obese (OR 1.7, 95% CI 1.4–2.1) were associated with CT use first. Evaluation at a non-children’s hospital was associated with higher odds of CT use (OR 7.9, 95% CI 7.5–8.4). Ultrasound concordance with pathology was higher for males (72.3 vs. 66.4%, p = .03), in perforated appendicitis (75.9 vs. 67.5%, p = .009), and at children’s hospitals compared to general adult hospitals (77.3 vs. 62.2%, p < .001). CT use has decreased yearly statewide.

Conclusions

Over 50% of children with appendicitis had radiation-based imaging. Understanding factors associated with CT use should allow for more specific QI interventions to reduce radiation exposure. Site of care remains a significant factor in radiation exposure for children.

Appendicitis is the most common surgical condition of childhood, accounting for 5%–10% of all pediatric emergency department visits [1–4]. Timely and accurate diagnosis is critical since symptom duration is associated with perforation, which increases length of stay, complications, and hospital costs [5–7]. Accurate diagnosis is also important to avoid unnecessary surgery where the appendix is found to be normal, commonly known as a negative appendectomy (NA). Diagnostic imaging plays a crucial role in the evaluation of abdominal pain, helping to definitively diagnosis early appendicitis as well as to rule out appendicitis and avoid NA. Over the past two decades, imaging has been shown to reduce NA rates by up to 80%, and its use has become widespread [5,8–10]. However, there remain significant concerns regarding the risk of radiation-induced malignancy in children undergoing radiation-based imaging, such as computed tomography (CT) scans [11,12].

Among diagnostic studies, CT has been shown to be highly sensitive (93%–95%) and specific (95%–98%) in diagnosing appendicitis [13,14]. Since it is widely available, its use for evaluation of pediatric abdominal pain has markedly increased in the past decade [15–17]. In light of increased CT use and concerns regarding risks of radiation-based imaging, the National Cancer Institute, the American Academy of Pediatrics, and the American Pediatric Surgical Association have recommended the use of alternative non-radiation-based imaging such as ultrasound (US) [12,18–24]. A slight increase in US and decrease in CT use for the diagnosis of appendicitis have been documented in freestanding children’s hospitals since 2007 [25]. However, many children with appendicitis are treated at general hospitals where pediatric radiation protocols may be less frequently followed. Single site studies of referrals to pediatric hospitals suggest that community hospitals may be far more likely to use CT for the diagnosis of appendicitis [1,26,27]. Larger studies using administrative databases have also suggested that community hospitals may be more likely to use CT, but these studies are limited in their ability to accurately capture the use of imaging using administrative discharge data, as well as to identify which imaging modality was used first and to test concordance between imaging and pathology [17,28].

To address this evidence gap and continued safety concerns, we evaluated factors associated with CT and US use and the effectiveness of CT and US among children undergoing appendectomies in Washington State. We investigated whether imaging type and accuracy vary by hospital type (e.g. freestanding children’s hospital vs. non-children’s hospital) in children with appendicitis. The purpose of this study was to identify factors associated with the use of CT that may be potential modifiable targets for quality improvement in a large, diverse population of hospitals.

1. Patients and methods

1.1. Study population and setting

The Surgical Care Outcomes and Assessment Program (SCOAP) is a physician-led quality improvement collaborative that began in 2006 and has subsequently enrolled nearly all hospitals in Washington State. The Comparative Effectiveness Research Translation Network (CERTAIN) is a translational research network composed of thirty-five clinics and twenty-five hospitals in Washington, which uses a unique data-sharing platform to allow investigators and providers to track quality, benchmark best practices, and improve care. Unlike administrative datasets in which International Classification of Diseases, Ninth Revision codes are used to obtain information about diagnosis and treatment, SCOAP relies on prospective review of clinical records of all patients undergoing specific procedures, with data collection by trained abstractors. Thirty-two of the hospitals participating in SCOAP provide care to pediatric patients. These hospitals began to collect data on non-elective appendectomies in children in 2008. This study included the pediatric patients (≤18 years old) who underwent a non-elective appendectomy at a SCOAP hospital between 2008 and 2012.

Hospitals were designated as a general adult hospital, a pediatric unit within a general hospital, or a freestanding children’s hospital. Hospitals were determined to have a pediatric unit within a general hospital if they had a pediatric surgeon, a specialized pediatric ward, or a specialized pediatric emergency room.

1.2. Data characteristics and primary outcome

Demographic information, clinical characteristics, diagnostic imaging use, radiologic interpretations, operative findings, and pathology results are abstracted from the clinical record using standardized definitions. The data represent consecutive non-elective appendectomies performed at each participating site. Data collection is standardized across sites and collected by trained abstractors. Inter-rater reliability is verified through twice yearly case review. BMI group (normal: <85th percentile, overweight: 85th–95th percentile, and obese: ≥95th percentile) is determined by age- and sex-standardized BMI percentile calculated from recorded height and weight of each patient. Perforation of the appendix is based on pathologic diagnosis or gross evidence of perforation intra-operatively. Research projects using de-identified SCOAP data are exempted from review by the University of Washington Institutional Review Board.

Data on diagnostic imaging abstracted from the medical record include the type of imaging performed (CT or abdominal ultrasound), the imaging results, and the order in which the imaging occurred. Imaging order is crucial to understanding which study was performed first, as some patients may have more the one imaging study. The results of each imaging study are based on the final radiologist interpretation and are reported as consistent with appendicitis, not consistent with appendicitis, or indeterminate. The imaging and pathology reports are considered concordant if the imaging results are consistent with appendicitis and the pathology is positive, or if imaging results are not consistent with appendicitis and pathology does not show evidence of disease. Indeterminate imaging findings are considered non-concordant. The primary outcome was the type of imaging first used in the diagnostic work-up. First imaging modality used, rather than overall imaging used, was chosen in recognition of the fact that CT use as a second imaging study (after an indeterminate ultrasound) may be appropriate in the evaluation of a child with abdominal pain concerning for appendicitis.

1.3. Analytic methods

1.3.1. Univariate analysis

Demographic and clinical characteristics of patients were compared between those undergoing ultrasound as their first study and those undergoing CT scan as their first study. Characteristics were summarized using frequency distributions for categorical variables and means with standard deviations for continuous variables. Categorical variable comparisons were evaluated for significance using Pearson χ² test (significance set at α = 0.05). Continuous variable comparisons were evaluated for significance using t-tests (α = 0.05).

1.3.2. Concordance

In order to evaluate accuracy, concordance between radiologic interpretation of imaging and pathology was determined for each imaging study performed. Concordance rates were evaluated for US and CT by hospital type.

1.3.3. Multivariate analysis

Using multivariate logistic regression, factors independently associated with use of US or CT as first imaging modality were identified. Patients were excluded from this portion of the analysis if they did not undergo imaging. Covariates were included in the logistic regression model if they were known from existing surgical literature to be associated with differential rates of US and CT use in children or if they were significant in the univariate analysis [1,7,14–17,19,25,28,29]. Using these criteria, a parsimonious logistic regression model was developed that included age group, sex, race, ethnicity, insurance, BMI group, and hospital type as potential factors associated with use of US or CT scan as the first imaging study. Hospital type was included in the model as a binary variable, comparing freestanding children’s hospitals and non-children’s hospitals. The model was adjusted for clustering of patients by institution. STATA version 11 was used for all analyses (STATA Corp, College Station, TX). Statistical significance was set at p < 0.05.

2. Results

2.1. Cohort characteristics

2538 children underwent appendectomy (mean age 11.3 years (4.1), 57.6% male), with 8 (0.3%) undergoing no preoperative imaging prior to appendectomy. These 8 children were excluded from multivariate models identifying factors associated with CT or US use as first imaging study. Of the remaining 2350 patients, the mean age was 11.3 years (4.1), and 57.6% were male (Table 1). Over forty percent of children were overweight or obese. The population was largely white (70%), 25% were Hispanic, and 56.4% had private insurance. The majority (53.1%) were initially seen and evaluated in a general adult hospital, while 27.5% were initially evaluated in a freestanding children’s hospital. The overall perforation rate was 21.7% and the NA rate was 4.6%.

Table 1.

Demographic characteristics based on type of first imaging performed.

	All	CT (52.7%)	Ultrasound (47.3%)	p-value
Number of Children (%)	2530	1332	1198
Mean Age (SD)	11.3 (4.1)	12.3 (3.7)	10.3 (4.1)	<.001
Age Group				<.001
Age ≤5	210 (8.3)	63 (4.7)	147 (12.3)
5 < Age ≤ 10	741 (29.3)	320 (24.0)	421 (35.1)
10 < Age ≤ 18	1579 (62.4)	949 (71.3)	630 (52.6)
Sex (%)				<.001
Male	1457 (57.6)	812 (61.0)	645 (53.9)
Female	1071 (42.4)	519 (39.0)	552 (46.1)
Insurance (%)				.004
Private	1406 (56.4)	697 (53.6)	709 (59.4)	.001
Medicaid	877 (35.2)	500 (38.4)	377 (31.6)	.001
Uninsured/Self-Insured	91 (3.6)	42 (3.2)	49 (4.1)	.20
Medicare/Tricare/Indian Health Service/VA	120 (4.8)	62 (4.8)	58 (4.9)	.29
BMI Group				<.001
Normal	734 (58.8)	441 (55.0)	293 (65.7)
Overweight	232 (18.6)	151 (18.8)	81 (18.2)
Obese	282 (22.6)	210 (26.2)	74 (16.1)
Race, %				<.001
White	1634 (70.0)	815 (70.6)	819 (69.3)
Black or African American	56 (2.4)	14 (1.2)	42 (3.6)
Asian	84 (3.6)	37 (3.2)	47 (4.0)
American Indian/Alaska Native	55 (2.3)	37 (3.2)	18 (1.5)
Native Hawaiian or Other Pacific Islander	19 (0.8)	12 (1.0)	7 (0.6)
Unknown/NA	488 (20.9)	249 (20.8)	248 (21.0)
Ethnicity (%)				<.001
Hispanic or Latino	584 (25.0)	343 (29.7)	241 (20.4)
Not Hispanic or Latino	1296 (55.5)	534 (46.3)	762 (64.5)
NA	455 (19.5)	277 (24.0)	178 (15.1)
Hospital Type				<.001
General	1328 (53.1)	911 (68.8)	417 (35.5)
Pediatric Unit in General	495 (19.4)	326 (24.6)	159 (13.5)
Free-Standing Pediatric	687 (27.5)	88 (6.6)	599 (51.0)
Transfer from Another Hospital (%)	527 (21.0)	279 (21.2)	248 (20.7)	.80
Imaging Performed (%)
CT Scan only	1289 (51.0)	1289 (96.8)	0
Ultrasound only	1004 (39.7)	0	1004 (83.8)
Both	237 (9.4)	43 (3.2)	194 (16.2)	<.001
Perforation Rate (%)	550 (21.7)	264 (19.8)	286 (23.9)	.01
Negative Appendectomy (%)	116 (4.6)	58 (4.4)	58 (4.8)	.56

2.2. First imaging study

2.2.1. Univariate analysis

Over half (52.7%) of children had a CT scan as their first imaging study. Of the 1332 children undergoing a CT scan as their first imaging study, 911 (68.8%) were initially evaluated at a general adult hospital, while just 88 (6.6%) were evaluated at a freestanding children’s hospital. Of the 1198 children undergoing an ultrasound as their first imaging study, 599 (51%) were initially evaluated at a freestanding children’s hospital, compared with 417 (35.5%) at a general adult hospital. Children undergoing an ultrasound as their first imaging study were more likely than children with a CT scan first to have a second imaging study (16.2 vs. 3.2%, p < .001). Children who underwent an ultrasound as their first imaging study were found to have a higher perforation rate than those who underwent CT first (23.9 vs. 19.8%, p < .001). Negative appendectomy rates were not different in the two cohorts (4.6 vs. 4.4%, p = 0.56).

2.2.2. Multivariate analysis

After controlling for potential confounders, older age, male sex, Hispanic ethnicity, and being overweight or obese were associated with increased odds of CT use as the first imaging (Table 2). Initial evaluation at a non-children’s hospital was associated with nearly 8-fold higher odds of undergoing a CT scan as first imaging study (odds ratio 7.9, 95% CI 7.5–8.4) compared to evaluation at a freestanding children’s hospital. Initial evaluation at a non-children’s hospital was associated with 87% lower odds of undergoing an ultrasound as the first diagnostic study (odds ratio 0.13, 95% CI 0.12–0.13; Table 3) compared to evaluation at a freestanding children’s hospital.

Table 2.

Multivariate analysis of factors associated with use of CT as first imaging.

Variable	Univariate Odds Ratio	95% CI	Multivariate Odds Ratio	95% CI
Age Group
Age ≤5	Ref	–	Ref	–
5 < Age ≤ 10	1.8	1.3–2.5	1.6	1.4–1.8
10 < Age ≤ 18	3.5	2.6–4.8	2.9	2.2–4.0
Female Sex	0.8	0.6–0.9	0.7	0.5–0.9
Black or African American	0.3	0.2–0.5	1.0	0.2–5.9
Asian	0.7	0.4–1.1	1.1	0.3–3.4
American Indian/Alaska Native	1.8	1.02–3.2	3.5	0.6–20.9
Native Hawaiian/Other Pacific Islander	1.5	0.6–3.8	2.1	0.1–52.2
Hispanic Ethnicity	1.6	1.3–1.9	1.7	1.5–1.9
Medicaid Insurance	1.3	1.1–1.5	1.1	0.8–1.4
BMI Group
Normal	Ref	–	Ref	–
Overweight	1.2	0.9–1.7	1.1	1.02–1.2
Obese	1.9	1.4–2.6	1.7	1.4–2.1
Non-Pediatric Hospital	14.5	11.4–18.5	7.9	7.5–8.4

2.3. Concordance

Concordance between first imaging study and pathology was compared for CT and US by hospital type. US concordance was higher in imaging studies performed in freestanding children’s hospitals (77.3% compared to general hospitals [62.2%] and pediatric units in a general hospital [57.1%], p < .001). There was no significant difference in CT concordance by hospital type.

3. Discussion

Despite the risk of radiation-induced malignancy and the presence of guidelines from major professional societies, we found that over 50% of children with appendicitis continue to receive a CT scan as their first diagnostic imaging study. Non-children’s hospitals have significantly higher odds of CT use, controlled for population characteristics, and lower rates of concordance. Concordance between imaging and pathology is higher for CT scans than for US, and does not vary significantly by hospital type. Ultrasound concordance with pathology is higher in those imaged at freestanding children’s hospitals. While in some settings CT scans may be a more accurate diagnostic modality, this advantage must be weighed against the risks of radiation, especially given that US correctly diagnoses appendicitis up to 77% of the time. To our knowledge, this is the largest study to evaluate both use and accuracy of imaging for diagnosis of appendicitis in children across a variety of hospital settings. More importantly, many prior studies of imaging relied on administrative codes which are unreliable for outpatient imaging and for imaging that is not distinctly included in the discharge abstract reporting [30].

Female patients may be more likely to undergo US for evaluation of abdominal pain given its benefit in diagnosing gynecologic pathology. In keeping with this theory, we found, in a post-hoc analysis, that use of US as first image was significantly higher in female patients between 10 and 18 years old (odds ratio 1.4, 95% CI 1.1–1.7) compared to younger female patients. The finding of decreased US use in overweight and obese children aligns with previous studies [29,31,32]. Rates of CT scans are higher in obese children, possibly because sensitivity of CT does not vary by BMI, while US is more likely to be non-diagnostic in overweight and obese children [29,31,32]. Given the national epidemic of childhood obesity, decreased US accuracy and increased CT use in overweight and obese children present additional health risks to these children.

The use of diagnostic imaging is widespread, in part because clinical intuition and clinical decision rules leave room for improvement in the diagnosis of appendicitis in children [33–35]. In our sample, over 99% of children had some form of pre-operative imaging. Given that most patients will receive imaging if they present with a history and symptoms consistent with appendicitis, it is imperative that we understand factors that may predispose providers to use CT as the first diagnostic imaging modality. By understanding these factors we may enhance our ability to craft successful interventions to reduce CT use. This study suggests that BMI and sex are patient factors that may influence imaging modality choice. Hospital type also appears to significantly correlate with the type of imaging used. There may be many reasons why this is the case, including the availability of resources, concerns about ultrasound accuracy, reimbursement incentives, and training needs.

In many settings, high-quality US is not viewed as practical for diagnosis at night, given the requirement for an ultrasonographer. In one study, evaluation of US and CT use patterns at a single community hospital found that six times as many ultrasounds as CT scans were performed on children during the day. At night, half as many ultrasounds as CT scans were performed [36]. These findings suggest that resources and availability of US technology and skilled providers may present a real challenge for some hospitals. Additionally, providers may perceive that US has poor diagnostic accuracy and may choose to order a CT scan instead. This may reflect reality at a given hospital, as US accuracy is found to be lower at sites that use it less [37]. This could in part explain why concordance between US findings and pathology in our study was higher at freestanding pediatric hospitals as they more frequently use US. While US has the advantage of avoiding radiation exposure, it is operator dependent [38,39]. The US technologist performing the study, and the radiologist interpreting it, must be skilled to maximize diagnostic accuracy. The lower levels of US use and the decreased US accuracy at non-children’s hospitals may indicate an unmet training need. In order to balance the lower sensitivity of US and the increased risks associated with radiation from CT scans, many have advocated the use of a staged protocol with US as the initial diagnostic modality followed by CT use for patients with a non-diagnostic ultrasound [40–43]. A staged protocol of US followed by CT has been found to have a sensitivity of 98.6% and a specificity of 90.6% for the diagnosis of appendicitis [44].

Previous evaluations of CT and US use in the diagnosis of appendicitis have been limited by the nature of their data sources. Multiple studies have used data from the Pediatric Health Information System (PHIS), which is a comprehensive data source, but only represents freestanding children’s hospitals [19,25]. As such, findings from studies using PHIS data may not accurately reflect trends and outcomes in community settings where many children are evaluated and treated. Others have used national administrative databases, such as the Kid’s Inpatient Database (KID) or the National Hospital Ambulatory Medical Care Survey (NHAMCS) [17,28,45]. While these databases provide a picture of national use, they are limited by the reliability of data entry and coding of secondary procedures, such as diagnostic imaging. Lastly, two recent studies have evaluated patients seen at a freestanding children’s hospital and identified whether CT was used at the children’s hospital or at a community hospital [1,26]. These studies provide a more realistic picture of the patterns of use between different locations, but are hampered by selection bias of just evaluating those children referred to a pediatric center.

The results of this study must be interpreted in the context of study design. Our sample represents consecutive patients undergoing appendectomy at hospitals in Washington State, but does not evaluate patients undergoing imaging for abdominal pain who do not undergo an appendectomy. As such, our findings may represent a biased sample of children who are found to have appendicitis and undergo an operation, when compared to children who have abdominal pain but do not have appendicitis. Additionally, given that our database is a procedural database and not a population-based database, we are unable to calculate sensitivity and specificity of US and CT. Secondly, the data set does not capture clinical decision-making about how patients are allocated to imaging. Although our logistic regression models control for potential confounding by age, sex, BMI, race, ethnicity, and insurance status there may be residual confounding by indication. Third, it is important to note that hospitals included in study as “Pediatric Unit within a General Hospital” may be quite variable in nature. Given the small number of hospitals in this group and their variability, they were included in the “General Adult Hospital” group in our multivariate analysis. While this subset may have different characteristics and patterns of imaging use than the overall genral adult hospital group, we would have expected any difference to bias our findings towards the null. A further potential limitation is the possibility for sampling bias because the SCOAP cohort does not represent a truly random sample of the state’s total pediatric appendectomy volume. However, by the end of 2011, 55 of the 75 hospitals in the state that perform more than twenty appendectomies per year were actively contributing data to SCOAP. All of the hospitals among these 55 hospitals that perform appendectomies on children submit data to this dataset, and these hospitals include general adult hospitals, general hospitals with a pediatric unit, and freestanding children’s hospitals, representing hospital types broadly within the cohort.

This study indicates that while CT scan use has decreased slightly over the past 5 years, its use is still widespread in the evaluation of children with appendicitis. Non-children’s hospitals have significantly higher rates of use of CT scans than freestanding children’s hospitals, and concurrently decreased concordance between US and pathology. These findings present an opportunity to direct quality improvement interventions to reduce the exposure of children to radiation.