Abstract
Background
Patients with Crohn’s disease (CD) are frequently subjected to computed tomography (CT) in the emergency department (ED). This young population is at higher risk of malignancy from radiation exposure.
Objectives
We aimed to validate a decision tool predicting complications (perforation, abscess or other serious finding) on imaging at two sites.
Methods
We conducted a retrospective review of CT outcomes among patients with CD with ED visits at two tertiary care centers. Inclusion criteria were a CT of the abdomen/pelvis with contrast and complete lab data (erythrocyte sedimentation rate (ESR) and C-reactive protein (CRP)) within 24 hours of arrival at the University of Michigan (UM) (2012–2013) and the University of Pittsburgh (UPMC) (2009–2012). Sensitivity, negative predictive value (NPV), miss rate and CT avoidance rate were calculated.
Results
At UPMC (n = 210), the tool had a sensitivity of 88.9% and NPV of 98.0%, potentially saving 47.1% from CT with a miss rate of 1.0%. At UM (n = 248), the tool had a sensitivity of 90.9% and NPV of 96.0%, saving 40.3% from CT with a miss rate of 1.6%.
Conclusion
A decision tool using CRP and ESR predicting CT outcomes among CD patients performed well in an external validation, allowing providers to forgo CT use with a low miss rate.
Keywords: Computed tomography, validation, radiation, over-utilization
Introduction
Patients with Crohn’s disease (CD) are particularly vulnerable to multiple computed tomography (CT) scans over their lifetimes. Because their illness is chronic with frequent but variable flares, these patients often seek care in the emergency department (ED).1 During these ED visits, providers must decide whether CT imaging would be useful in evaluating for clinically important CD complications such as perforation or abscess. However, the benefits of CT must be balanced with downside risks such as cumulative exposure to the ionizing effects of radiation,2 allergic reaction to contrast agents,3 contrast-induced nephropathy,4 and unnecessary additional testing from incidental findings. High cumulative radiation exposure in this young population is particular problematic. Approximately 10% of the IBD population is exposed to more than 50 mSv of radiation, a level associated with an increased risk of malignancy.5,6
In order to help providers better risk-stratify patients who would least benefit from CT imaging, we developed decisions tools that predict the presence of complications in patients with CD presenting to the ED.7 These decision tools use only C-reactive protein (CRP) and erythrocyte sedimentation rate (ESR) in a logistic regression equation to predict the presence of perforation, abscess or another serious finding with good performance characteristics. Using cross-validation, we found that sensitivity to be 93.8%, the negative predictive value (NPV) to be 98.1% and the miss rate to be 0.8%. Using the tool would have saved up to 43% of patients from undergoing CT. An online calculator can be found at www.crohnsctrisk.info. We also created a simple decision tool that can be calculated by hand. While this tool had similar performance characteristics, it would have prevented only 18.5% of the patients from further radiation exposure.
Our aim here is to validate the previously derived clinical decision tools by analyzing their performance characteristics on two additional patient cohorts. The first is an external cohort from the University of Pittsburgh Medical Center (UPMC) while the second is a cohort from the University of Michigan (UM) from later years (2012–2013).
Methods
The electronic medical records systems of both institutions were searched for ED visits for patients ≥18 years of age with a diagnosis of CD based on the International Classification of Diseases, ninth revision (ICD-9) coding (555.x), which included a CT scan of the abdomen and pelvis. At UPMC, visits between January 1, 2009 and December 31, 2012 were analyzed. At UM, records were requested for visits between January 1, 2012 and December 31, 2013. ED and subsequent visits were reviewed to verify a diagnosis of CD and to determine medications, admission status and chief complaint. CT reports from within 24 hours of the ED visit were requested and reviewed to determine findings. CTs were excluded if both oral and intravenous contrast were not administered as contrast would be required to maximize the probability of detecting complications of disease. CT reads of scans performed at external facilities (external to UPMC or UM) or for an indication of trauma were also excluded. Only visits with both ESR and CRP drawn within 24 hours of arrival were included in the analysis. No imputation was performed for missing labs.
The records from UPMC were de-identified prior to analysis. All records were evaluated by one reviewer (SMG) who was blinded to the lab results.
Complications were characterized in the same manner as in the original decision tool derivation.7 Complications consisted of perforations, abscess, or another serious finding that may change management (not including inflammation). Obstruction was not included in the complications model as we found that obstruction is a clinical diagnosis that could not be reliably predicted in the derivation process. The findings that constitute complications are found in Table 1. The enterography protocol is used at UM but not at UPMC. Institutional review boards approved this project at both sites prior to data collection.
Table 1.
CT findings at UM and UPMC
| Derivation set |
Validation set |
||
|---|---|---|---|
| Finding | UM (n = 1095) | UM (n = 315) | UPMC (n = 499) |
| Abscess | 104 (9.5) | 43 (13.7) | 39 (7.8) |
| Obstruction | 108 (10.0) | 39 (12.4) | 42 (8.4) |
| Perirectal abscess | 25 (2.3) | 10 (3.2) | 16 (3.2) |
| Perforation | 15 (1.4) | 4 (1.3) | 10 (2.0) |
| Urolithiasis with complication | 3 (0.3) | 4 (1.3) | 2 (0.4) |
| Diverticulitis | 12 (1.1) | 2 (0.6) | 2 (0.4) |
| Pancreatitis | 9 (0.8) | 2 (0.6) | 4 (0.8) |
| Pyelonephritis | 2 (0.2) | 0 (0) | 2 (0.4) |
| Other serious findinga | 10 (0.9) | 6 (1.9) | 8 (1.6) |
| Inflammation | 597 (54.5) | 192 (60.9) | 248 (49.7) |
| Complications | 184 (16.8) | 67 (21.3) | 80 (16.0) |
UM validation: cholangitis, bowel ischemia, incarcerated umbilical hernia, psoas myositis, osteomyelitis, angioedema; UPMC validation: two instances of pneumonia, pericarditis, pouchitis, IMV thrombosis, intussception, DVT, hydrosalpinx.
CT: computed tomography; UM: University of Michigan; UPMC: University of Pittsburgh Medical Center; IMV: inferior mesenteric vein; DVT: deep vein thrombosis.
The logistic regression tool is e^(0.0953*CRP+0.0189*ESR–3.206)/1+e^(0.0953*CRP+0.0189*ESR–3.206) where CRP is in mg/dl. The cutoff used with this equation is 0.06 with a very low probability of a significant finding with results below this. The simple decision tool is calculated by ESR+5*CRP (mg/dl) with a cutoff of 10 or less.
Statistical methods
Descriptive statistics were calculated comparing each validation cohort with the derivation cohort using the student’s t-test for numerical variables and the chi square test for categorical variables. The sensitivity, NPV, and area under the receiver operating characteristic (AuROC) curve are reported for each site and for each equation (logistic regression and simple formula). AuROC graphs were created using SAS 9.3 (Cary, NC).
Results
UPMC external validation
From 2009 to 2012, there were 802 ED visits among 458 patients with Crohn’s at UPMC. After applying exclusion criteria (see Figure 1), 499 visits representing 308 individuals were reviewed. Laboratory data were complete in 210 cases. Demographic and medication comparisons between the UPMC validation cohort and the UM derivation cohort are presented in Table 2. Most patients had a chief complaint of abdominal pain leading to their ED visit.
Figure 1.
Consolidated Standards of Reporting Trials (CONSORT)-style diagram demonstrating the sample size after applying the exclusion criteria at each institution.
Table 2.
Population characteristics and comparison to derivation population
| Derivation set |
Validation set |
p value |
|||
|---|---|---|---|---|---|
| UM (n = 1095) | UM (n = 315) | UPMC (n = 499) | UM vs. derivation | UPMC vs. derivation | |
| Age | 38.2 (±14.6) | 37.7 (±14.2) | 36.1 (±13.1) | 0.63 | <0.01 |
| Female (%) | 573 (52.3) | 161 (51.1) | 245 (49.1) | 0.7 | 0.23 |
| Race | 0.08 | 0.25 | |||
| White | 940 (85.8) | 263 (83.5) | 442 (88.6) | ||
| Black | 129 (11.8) | 37 (11.8) | 50 (10.0) | ||
| Other | 26 (2.4) | 15 (4.8) | 7 (1.4) | ||
| Steroid use | 306 (28.0) | 73 (23.2) | 78 (15.6) | 0.09 | <0.01 |
| Immunomodulator use | 402 (36.7) | 138 (43.8) | 115 (23.1) | 0.02 | <0.01 |
| Biologic use | 232 (21.2) | 103 (32.7) | 155 (31.1) | <0.01 | <0.01 |
| Ciprofloxacin use | 77 (7.0) | 23 (7.3) | 35 (7.0) | 0.87 | 0.99 |
| Metronidazole use | 85 (7.8) | 23 (7.3) | 38 (7.6) | 0.79 | 0.92 |
| Narcotic use | 328 (30.0) | 98 (31.1) | 175 (35.1) | 0.69 | 0.05 |
| 5-ASA use | 413 (37.7) | 67 (21.3) | 70 (14.1) | <0.01 | <0.01 |
| Admission | 974 (89.0) | 288 (91.4) | 386 (77.4) | 0.21 | <0.01 |
UM: University of Michigan; UPMC: University of Pittsburgh Medical Center; 5-ASA: 5-aminosalicylic acid.
The percentage of patients with complications on CT was 15.6%, but among those with complete lab data, the complication rate was 9.5% (Table 1). Among patients with missing ESR and/or CRP, the complication rate was 20.8%. Inflammation was found in 49.7% of CTs.
The logistic regression decision tool for complications had a sensitivity of 88.9%, NPV of 98.0% and miss rate of two out of 210 (1.0%) while possibly saving 47.1% of the cohort from CT. The simple decision tool for complications had a sensitivity of 88.9%, NPV of 93.1% and miss rate of 1.0%. Using the simple tool would have saved only 13.8% of the patients from radiation. The AuROC for the logistic regression decision tool for complications was 0.78 with a 95% confidence interval (CI) of 0.63–0.92 (Figure 2).
Figure 2.
The receiver operating characteristic (ROC) curves for the derivation logistic regression decision tool and validation logistic regression decision tool are displayed.
The two cases missed by the logistic regression decision tool included an instance of intussusception and urolithiasis. Both patients were managed conservatively without operative intervention.
UM temporal validation
Between 2012 and 2013, there were 405 ED visits among patients with CD. Applying the exclusion criteria, there were 315 eligible visits by 220 individuals during this time period (Figure 1). Demographics and medication comparisons between the validation and derivation cohort are displayed in Table 2. The admission rate among this cohort remained very high, at 91.4%.
The CT findings demonstrated complications in 20.6% and inflammation in 60.9% (specific findings and frequency are shown in Table 1). ESR and CRP were both available for 248 visits and 43.5% were performed with a CT enterography protocol. Patients with a CRP and ESR were less likely to have a complication on CT (17.7%) compared to patients with one or both labs missing (31.3%) (p = 0.01).
Using the logistic regression decision tool to predict complications, the sensitivity of the algorithm was 90.9% and the NPV was 96.0%. Of the 248 patients scanned with complete lab data, four patients were incorrectly predicted as not having a complication for a miss rate of 1.6%. The benefit of this tool is that 100 patients (40.3%) would have been saved from further radiation. The AuROC for the logistic regression model for complications was 0.78 (95% CI: 0.72–0.86) (Figure 2). The simple decision tool had a sensitivity of 97.7%, NPV of 98.0% and a miss rate of 0.4% while saving 20% of the patients from further radiation exposure.
Among the UM validation population, the four missed cases included two instances of abdominal abscess, one case of bowel ischemia, and one perianal abscess. The patient with bowel ischemia was known to have a significant bowel obstruction based on a CT obtained less than 48 hours earlier at another facility and underwent reduction of the parastomal hernia four days after arrival. The patient with perirectal abscess did go to the operating room for examination under anesthesia but was not found to have a drainable fluid collection. One of the patients with an abscess was known to have a prior abscess requiring percutaneous drainage; the patient was taken to the operating room for incision and drainage on this occasion. The other patient with an abscess presented with a fluctuant mass at his ostomy and underwent percutaneous aspiration of the fluid collection.
Discussion
In an attempt to help providers better select patients who could avoid further radiation, we developed equations using easily available labs with a high NPV. In our derivation set, we were able to save 43% of patients from further radiation exposure. Here, we have demonstrated in two separate validation cohorts that our logistic regression equation consisting of ESR and CRP is able to save more than 40% of patients from further radiation exposure with a high NPV. Both of these labs tests (CRP, ESR) are widely available, inexpensive, and have a rapid turnaround time. Further, we have been able to build this multianalyte algorithmic test into our laboratory information system and incorporate the results into the electronic medical records system at UM, with an estimated turnaround time of 90 minutes.
Preventing 40% of the CD population presenting to the ED from undergoing CT has significant implications on health care costs, cancer prevention and the rare risk of contrast-induced nephropathy in this population. The cost of a CT abdomen and pelvis according to the most recent Medicare reimbursement tables is $327.8 Using the assumptions that there are 540,000 patients with CD in the United States9 and that on average they each obtain one CT every decade, application of the logistic algorithm would produce a cost savings of $70 million a decade. Using the same assumptions regarding the prevalence of Crohn’s and the frequency of CT use along with the assumption that for every 910 CTs performed,10 there will be 1 additional cancer, the logarithmic decision tool could prevent 23 additional cancers each year. For patients below the age of 35 who are particularly at risk from repeated radiation exposure,11 this decision tool could help providers use alternative imaging modalities such as magnetic resonance imaging (MRI) or ultrasound for those who are not below the cutoff.
While we believe the NPVs of this decision tool are acceptably high, a potential barrier to their widespread use in the community could be a mindset that any NPV less than 100% is not good enough. For comparison, the NPV of procalcitonin as a marker of infection is 94%–98% at typically used cutoffs,12 and the NPV for CT in the diagnosis of acute appendicitis is 94%–95%.13,14 Much like the use of d-dimer to rule out pulmonary embolism (PE) in low-risk populations, we would recommend the use of these equations in patients with moderate suspicion for a positive finding. Studies at other large tertiary care institutions have identified that patients with a prior history of surgery, abdominal abscess or obstruction, and those that have tachycardia or are underweight (body mass index (BMI) <18.5) are more likely to have significant findings on CT.15,16 We would suggest that a high pretest likelihood to scan should be applied to patients with these characteristics.
Conversely, there could be a paradoxical increase in CT use if our decision tool were to be used too broadly and applied to very low-risk patients whose providers would not have even considered obtaining a CT. In our derivation cohort, we noted that providers were quite good at identifying very low-risk patients who did not need a CT.7 Of those patients who returned to the ED within 30 days after a visit that did not include a CT, 28% underwent a CT on the return visit and none had a complication.
Potential limitations to the use of this decision tool include the inability to identify obstructions as a complication, the frequent use of non-contrast studies, and the possible lack of generalizability to patients seen in the community rather than tertiary centers. For the purpose of this tool, we characterized obstruction as a CT finding consistent with obstruction coupled with the need for nasogastric (NG) tube decompression or a surgical procedure. Reclassifying obstruction as those with consistent imaging requiring a surgical procedure revealed changed the obstruction rate from 10.0% to 2.3%. Among patients who present with signs or symptoms of an obstruction such as nausea or vomiting, it would be reasonable to start with an acute abdominal series as a screening test prior to cross sectional imaging. Non-contrast CT is frequently used in the ED among patients with CD because of the quick turnaround. We did not study non-contrast CT scans since they may not identify outcomes such as abscess. Ideally, we would limit our studies to enterography protocol CTs, but these studies are used infrequently in the ED. In regards to generalizability, the complication rate was higher among patients with missing labs at both centers. It is possible that patients who did not have these labs drawn presented with more obvious signs of a complication (fever, peritoneal signs) and providers did not feel these labs were needed to make a decision regarding imaging. The use of this tool also may be limited by the availability of ESR throughout the day at other centers. Our institution is able to perform this test throughout the day, but other centers internationally may not have this availability. Prospective studies are needed to determine the impact of this algorithm in practice.
In conclusion, we have validated risk stratification algorithms based on CRP and ESR in two distinct cohorts. These algorithms can help providers identify patients with CD who are at very low probability of having a complication on CT and avoid further radiation. Further prospective studies will have to be performed to study the effect of implementation of these algorithms on CT use in CD patients in the ED.
Acknowledgments
The guarantor of this work is Shail M. Govani. Author contributions were as follows. Shail M. Govani: conceptual design, data acquisition, data analysis, drafting and critical review of the manuscript. Akbar K. Waljee: conceptual design, data analysis, critical review of the manuscript. Keith E. Kocher: data analysis and critical review of the manuscript. Jason M. Swoger: data acquisition and critical review of the manuscript. Melissa Saul: data acquisition and critical review of the manuscript. Peter DR Higgins: conceptual design, data analysis, critical review of the manuscript. All authors approved the final version.
Funding
This work was supported by a GI Fellowship Award given by the Inflammatory Bowel Disease Working Group, through an unrestricted grant from UCB Inc to SMG. Akbar K. Waljee's research is funded by a Career Development Award (CDA 11-217) from the United States (U.S.) Department of Veterans Affairs Health Services Research and Development Service. The contents do not represent the views of the U.S. Department of Veterans Affairs or the U.S. Government.
Declaration of conflicting interests
None declared.
References
- 1.Ananthakrishnan AN, McGinley EL, Saeian K, et al. Trends in ambulatory and emergency room visits for inflammatory bowel diseases in the United States: 1994–2005. Am J Gastroenterol 2009; 105: 363–370. [DOI] [PubMed] [Google Scholar]
- 2.Kroeker K, Lam S, Birchall IW, et al. Patients with IBD are exposed to potentially serious excesses of ionizing radiation. Gastroenterology 2009; 136: A–99. [Google Scholar]
- 3.Morcos S, Thomsen H. Adverse reactions to iodinated contrast media. Eur Radiol 2001; 11: 1267–1275. [DOI] [PubMed] [Google Scholar]
- 4.McCullough PA, Sandberg KR. Epidemiology of contrast-induced nephropathy. Rev Cardiovasc Med 2003; 4(Suppl 5): S3–S9. [PubMed] [Google Scholar]
- 5.Chatu S, Subramanian V, Pollok RC. Meta-analysis: Diagnostic medical radiation exposure in inflammatory bowel disease. Aliment Pharmacol Ther 2012; 35: 529–539. [DOI] [PubMed] [Google Scholar]
- 6.Newnham E, Hawkes E, Surender A, et al. Quantifying exposure to diagnostic medical radiation in patients with inflammatory bowel disease: Are we contributing to malignancy? Aliment Pharmacol Ther 2007; 26: 1019–1024. [DOI] [PubMed] [Google Scholar]
- 7.Govani S, Guentner AS, Waljee AK, et al. Risk stratification of emergency department patients with Crohn’s disease could reduce computed tomography use by nearly half. Clin Gastroenterol Hepatol 2014; 12: 1702–1707. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Centers for Medicare and Medicaid Services. Physician fee schedule search, http://www.cms.gov/apps/physician-fee-schedule/search/ (accessed 19 March 2013).
- 9.Loftus CG, Loftus EV, Jr, Harmsen WS, et al. Update on the incidence and prevalence of Crohn’s disease and ulcerative colitis in Olmsted County, Minnesota, 1940–2000. Inflamm Bowel Dis 2006; 13: 254–261. [DOI] [PubMed] [Google Scholar]
- 10.Smith-Bindman R, Lipson J, Marcus R, et al. Radiation dose associated with common computed tomography examinations and the associated lifetime attributable risk of cancer. Arch Intern Med 2009; 169: 2078–2086. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Brenner D, Elliston C, Hall E, et al. Estimated risks of radiation-induced fatal cancer from pediatric CT. AJR Am J Roentgenol 2001; 176: 289–296. [DOI] [PubMed] [Google Scholar]
- 12.Riedel S, Melendez JH, An AT, et al. Procalcitonin as a marker for the detection of bacteremia and sepsis in the emergency department. Am J Clin Pathol 2011; 135: 182–189. [DOI] [PubMed] [Google Scholar]
- 13.Karakas SP, Guelfguat M, Leonidas JC, et al. Acute appendicitis in children: Comparison of clinical diagnosis with ultrasound and CT imaging. Pediatr Radiol 2000; 30: 94–98. [DOI] [PubMed] [Google Scholar]
- 14.Balthazar EJ, Birnbaum BA, Yee J, et al. Acute appendicitis: CT and US correlation in 100 patients. Radiology 1994; 190: 31–35. [DOI] [PubMed] [Google Scholar]
- 15.Kerner C, Carey K, Baillie C, et al. Clinical predictors of urgent findings on abdominopelvic CT in emergency department patients with Crohn’s disease. Inflamm Bowel Dis 2013; 19: 1179–1185. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Yarur AJ, Mandalia AB, Dauer RM, et al. Predictive factors for clinically actionable computed tomography findings in inflammatory bowel disease patients seen in the emergency department with acute gastrointestinal symptoms. J Crohns Colitis 2014; 8: 504–512. [DOI] [PubMed] [Google Scholar]