Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 May 1.
Published in final edited form as: Acad Emerg Med. 2015 Mar 2;22(5):625–631. doi: 10.1111/acem.12640

Optimizing Diagnostic Imaging in the Emergency Department

Angela M Mills 1, Ali S Raja 2, Jennifer R Marin 3
PMCID: PMC4523049  NIHMSID: NIHMS678263  PMID: 25731864

Abstract

While emergency diagnostic imaging use has increased significantly, there is a lack of evidence for corresponding improvements in patient outcomes. Optimizing emergency department (ED) diagnostic imaging has the potential to improve the quality, safety, and outcomes of ED patients, but to date, there have not been any coordinated efforts to further our evidence-based knowledge in this area. The objective of this article is to discuss six aspects of diagnostic imaging in order to provide background information on the underlying framework for the 2015 Academic Emergency Medicine consensus conference, “Diagnostic Imaging in the Emergency Department: A Research Agenda to Optimize Utilization.” The consensus conference aims to generate a high priority research agenda for emergency diagnostic imaging that will inform the design of future investigations. The six components herein will serve as the group topics for the conference: 1) patient-centered outcomes research; 2) clinical decision rules; 3) training, education, and competency; 4) knowledge translation and barriers to image optimization; 5) use of administrative data; and 6) comparative effectiveness research: alternatives to traditional CT use.

INTRODUCTION

Diagnostic imaging has revolutionized the practice of medicine. However, there has been concern about the increasing utilization of diagnostic imaging in the United States, particularly in the emergency department (ED).1-5 The increasing use of diagnostic imaging has led to increased costs6 and ED lengths of stay,7 but up to 50% of diagnostic imaging may be medically “unnecessary,”8 and the increasing use of diagnostic imaging has not led to better identification of pathology nor an improvement in patient-centered outcomes.5,9 Computed tomography (CT) use, specifically, also increases patient exposure to contrast media and the associated risk of contrast-induced nephropathy.10,11 Meanwhile, exposure to ionizing radiation in general has been shown to confer an increased cumulative lifetime risk of cancer.12,13

Several national campaigns, including Image Gently,14 Image Wisely,15 and As Low As Reasonably Achievable (ALARA),16 as well as initiatives by the National Council on Radiation Protection and Measurements17 and the Food and Drug Administration,18 have been established in an effort to reduce inappropriate diagnostic imaging. Likewise, the American College of Radiology and American College of Cardiology have appropriate-use criteria for a variety of diagnostic imaging modalities in order to assist clinicians in their use of diagnostic imaging,19,20 and the American Board of Internal Medicine’s Choosing Wisely campaign21 has focused attention on potentially overused imaging tests. In addition, the Centers for Medicare and Medicaid Services have implemented a project to determine the appropriateness of physician use of diagnostic imaging in relation to specialty guidelines,22 and the National Quality Forum has developed imaging efficiency measures23 in order to enhance the quality of patient care.

However, despite this focus on appropriateness of use, utilization continues to rise and, as of 2009, the National Quality Forum described “a clear need for measurement and research regarding the appropriate and effective use of imaging in the clinical setting.”23 Similarly, the National Heart, Lung and Blood Institute called for “further research into imaging procedures involving radiation for diagnostic or treatment purposes.”24 The ED represents one of the largest venues for the use of diagnostic imaging, with the rate of growth of CT use higher in the ED than in other medical settings.2,25 While we are in urgent need of research on how best to intervene in order to change current practices, to date there has been no coordinated effort to further our evidence-based knowledge on these topics. Our objective is to discuss six different aspects of diagnostic imaging, in order to provide background information on the underlying framework for the 2015 Academic Emergency Medicine consensus conference, “Diagnostic Imaging in the Emergency Department: A Research Agenda to Optimize Utilization.” The overall aim of the consensus conference is to generate a high priority research agenda for emergency diagnostic imaging which will, in turn, inform the design of future investigations in the field.

Prior to the consensus conference, the six topic-specific groups will identify salient studies relevant to their particular topic and gaps in the literature that will serve as a foundation upon which to build the research agenda for each area.

Patient-Centered Outcomes Research

Patient-centered outcomes research as applied to diagnostic imaging includes the engagement of patients in the decision-making process to order imaging, deliver the results to patients and caregivers, and follow up incidental findings from the diagnostic test. One aspect of patient-centered care is the process of shared decision-making, which “allows patients and their providers to make health care decisions together, taking into account the best scientific evidence available, as well as the patient’s values and preferences.”26 Although the risk of radiation from medical imaging has become a topic of discussion for both the public and policy makers, emergency physicians do not routinely inform patients of these risks.27-29 One study found that both patients and physicians agreed these discussions should occur, but barriers existed such as the lack of a structured method to communicate this risk to patients, and physicians’ conflicted perceptions regarding this risk.30 Therefore, the specific goal of this group is to determine potential methods of engaging patients in the decision to obtain imaging, the delivery of diagnostic imaging results, and the follow-up of incidental findings.

Clinical Decision Rules for Emergency Diagnostic Imaging

Clinical decision rules (CDRs) are evidence-based algorithms derived from original research and are used to provide guidance for clinical decision-making. They can either be “directive” (suggesting a course of action) or “assistive” (providing evidence to enhance clinical judgment). Well-validated CDRs can potentially reduce the use of diagnostic tests and empower clinicians with risk assessments for a given constellation of clinical symptoms and signs. They can also serve to reduce inappropriate variation in practice by offering evidence to assist the clinician at the point of care. However, the creation and validation of CDRs requires large sets of prospectively gathered data that show a high degree of inter-observer agreement. For CDRs to be most useful, they must be highly accurate and generalizable to a variety of settings, and data should be available regarding their acceptability, usage, and effects on patient outcomes. While there are still only a few high-quality imaging CDRs, more are being developed and, once validated, these should continue to improve the optimization of ED imaging.

One well known CDR, the Pediatric Emergency Care Applied Research Network (PECARN) head injury rule,31 was developed using a large, multicenter prospective cohort to identify children at very low risk of clinically important traumatic brain injury who could safely avoid CT, and has been both internally and externally validated.32 Two other pediatric head injury CDRs include the Canadian Assessment of Tomography for Childhood Head Injury from Canada,33 and the Children’s Head Injury Algorithm for the Prediction of Important Clinical Events from the United Kingdom.34 While all three rules were developed using robust methodological standards with large multicenter populations, they differ in a number of areas: patient populations, predictor variables, inclusion and exclusion criteria, and outcomes. A recent study prospectively compared these three rules,35 and found that baseline physician judgment and the PECARN rule performed better than the others; however, the study population in the comparison study did not reflect the exact patient population for which each rule was initially developed, and the comparison study was underpowered to detect rare but clinically important traumatic brain injury. Comparisons such as this highlight some of the practical challenges of working with CDRs and need for further evaluation and validation of rules in specific settings and for particular outcomes. The goals of this group are to identify diseases or emergency conditions amenable to the development of new CDRs regarding applicable diagnostic imaging and to define appropriate outcomes for the development of these CDRs.

Training, Education, and Competency

As the practice of emergency medicine advances, so too must our training and education of trainees. New CDRs and algorithms are changing our approach to management, and residents must learn how to incorporate these CDRs into a patient-centered framework. Likewise, emergency physicians are often responsible for both ordering and interpreting imaging studies. These new core competencies and methods for their assessment must be developed. Furthermore, competency-based assessment must be developed for both the ordering and interpreting of common diagnostic imaging tests. One potential solution is simulation. Trainees can acquire both knowledge and skills, while also being assessed for the acquisition of both. Unfortunately, little has been done in the simulation literature regarding ordering and interpreting diagnostic imaging.36,37

Training in point-of-care ultrasonography has been required since 2008, and knowledge and performance assessment38 are often done by varying assessment methods.39 Similar educational programs regarding the appropriate ordering and interpretation of other imaging modalities might be developed following this model. The goal of this group therefore is to define an agenda regarding the training and assessment of emergency physicians in regards to ordering appropriate diagnostic imaging and subsequently interpreting those studies within their scope of practice.

Knowledge Translation and Barriers to Image Optimization

In addition to the overall rise in CT use, there is significant variation in use of imaging across ED patient and provider types, even when adjusting for potential confounders.2,40-45 Once CDRs for diagnostic imaging have been developed and validated, they must then be disseminated, implemented, and adhered to in order to maximize their effect on patient outcomes. For example, there are a number of CDRs for the evaluation of patients with suspected pulmonary embolism (PE), including algorithms using the Wells score coupled with D-dimer testing, and the PE rule-out criteria. While both of these algorithms have been shown to have high negative predictive values in large prospective ED studies,46,47 actual implementation of these decision rules in clinical practice has been inconsistent.48,49

Despite the presence of CDRs to minimize unnecessary use and variation, physicians do not consistently follow them or other evidence-based decision instruments.49-53 One possible barrier to the use of decision rules is that clinicians may feel that clinical gestalt is similar or better. A few studies have compared the use of clinical gestalt in patients with suspected PE to the Wells score, and have demonstrated similar results in assessing pretest probability.54-56 In addition, some have suggested that malpractice fears, discomfort with uncertainty, and risk aversion are responsible for the rise in imaging use.57-63 One study demonstrated that only half of clinicians who were familiar with common CDRs used them, and even then in only half of appropriate patients.48 However, there are no large-scale investigations into the decision-making process or the driving forces (at the patient, clinician, ED, or hospital level) behind the use of emergency diagnostic imaging. The development of effective methods for the dissemination and implementation of CDRs is essential in order to realize their potential improvements on health outcomes.

Through the Health Information Technology for Economic and Clinical Health Act, the US Department of Health and Human Services developed meaningful use criteria for the use of electronic health records. One of the required criteria includes the implementation of clinical decision support tools. Computerized physician order entry64 and the implementation of clinical decision support systems are the most recent tools available to assist clinicians in diagnostic imaging ordering for certain conditions. Although promising as an intervention in the outpatient radiology setting to minimize unnecessary imaging,65 there are few data to date on the effects of clinical decision support embedded in the electronic health record on ED ordering practices and their generalizability across EDs. One such study in an urban quaternary care center examined the effect of clinical decision support on the use and yield of CT for suspected PE in the ED and found both a 20% decrease in CT use and a 69% increase in CT yield post implementation.66 In addition, given that many patients receive multiple CT scans throughout their lifetimes,67-70 decisions regarding imaging may need to take into account an individual’s cumulative exposure to diagnostic radiation over time. In fact, the American College of Cardiology has called for “a surveillance mechanism to identify patients with high cumulative radiation doses due to repeated imaging.”71 The goals of this group are to identify barriers to the dissemination and implementation of evidence-based emergency imaging practices and to formulate a research agenda for studying and overcoming these barriers.

Using Administrative Data for Emergency Imaging Research

Administrative health care datasets may enable investigations of secular trends and large populations of patients but may be limited, because they are often initially collected for other purposes and therefore may not be applicable to the specific research questions being studied. In addition, many datasets rely on billing codes, which may be variably accurate based on the billing and coding strategies employed. One example of a commonly used national administrative database is the National Hospital Ambulatory Medical Care Survey (NHAMCS). A number of studies using NHAMCS have demonstrated trends in diagnostic imaging;1,2,43,72 however, only a few have evaluated outcomes associated with imaging.5 Likewise, the Nationwide Emergency Department Sample and the State Emergency Department Databases are available as part of the Healthcare Cost and Utilization Project and have been used for analyses of imaging practices.45,73 In addition, the Dose Index Registry is a relatively new registry from the American College of Radiology-sponsored National Radiology Data Registry (www.nrdr.acr.org) and collects anonymized data from CT studies completed at participating institutions in the United States.74 Data continue to emerge from this resource and a recent study characterized radiation dose indexes for renal colic protocol CTs performed in the United States.75 Other available administrative data sets for use in imaging research may include multipayer source or single payer commercial products, registry data, and multi- or single-institution sources. The goals for this group are to determine which types of research questions are best answered through the use of administrative data and which data sources are most appropriate for each type of question.

Comparative Effectiveness Research: Alternatives to Traditional CT Use

Efforts to minimize diagnostic radiation exposure include the avoidance of unnecessary testing, minimization of radiation dose, and continuous searches for alternatives to radiation-based imaging modalities. Comparative effectiveness research has been described as a patient-centered method to recognize the most efficient mechanisms to provide the right treatment to the right patient at the right time, and to translate these findings into better health outcomes.76 Although there are studies of non-radiating modalities such as ultrasound and magnetic resonance imaging (MRI), there are few comparative effectiveness studies addressing their use in the ED. In patients presenting to the ED with suspected nephrolithiasis, a recent multicenter comparative effectiveness trial found no significant differences between initial ultrasonography (both point-of-care and radiology) and CT with regards to 30-day high-risk diagnoses with complications, pain scores, serious adverse events, return ED visits, hospitalizations, and diagnostic accuracy.77 In addition, six-month cumulative radiation exposure was found to be significantly lower in the ultrasonography groups than in the CT group.

While ultrasonography for suspected appendicitis has been evaluated in a number of studies, it is highly operator dependent, resulting in a wide sensitivity range.78 When the appendix is visualized, ultrasonography has a sensitivity of 98% with a specificity of 92%79 and leads to avoidance of CT in greater than half of pediatric patients with suspected appendicitis.80 However, sensitivity has been shown to be lower at centers with decreased ultrasound availability and less frequent use.79 Nevertheless, two recent large studies of children with suspected appendicitis demonstrated the utility of a diagnostic pathway using ultrasound and CT following equivocal ultrasound.79,80 Likewise, ultrasonography selectively followed by MRI has been shown to be similar to CT with respect to time to antibiotic administration, time to appendectomy, negative appendectomy rate, perforation rate, and length of stay in pediatric patients with suspected appendicitis.81 MRI has also been shown to be comparable to CT in suspected adult appendicitis and pregnant patients, with similar rates of missed appendicitis and false positives.82,83 Both MRI and CT comparably demonstrate alternative diagnoses, and MRI has been shown to have a 98% negative predictive value in pregnant patients presenting with acute abdominal pain.84 The goal of this group is to identify the diseases or conditions relevant to the study of alternatives to traditional CT and determine the most effective and methodologically rigorous means to study them.

CONCLUSIONS

Emergency diagnostic imaging use has increased at a rapid rate, despite the lack of evidence for corresponding improvement in patient outcomes. Improvements in diagnostic imaging optimization may improve the quality, safety, and outcomes of ED patients. Thus, emergency diagnostic imaging use represents fertile ground for high priority investigations. The 2015 AEM consensus conference (May 12, 2015 in San Diego CA) is a multidisciplinary consensus conference that will formulate the research priorities for emergency diagnostic imaging, initiate a collaborative dialogue between stakeholders, and align a research agenda with that of federal funding agencies. We invite all those with interest in this topic to attend the conference and contribute their expertise (http://www.saem.org/annual-meeting/education/2015-aem-consensus-conference).

Acknowledgments

Dr. Marin is supported by an R13 pertaining to diagnostic imaging from the AHRQ, Dr. Marin teaches for and receives compensation from 3rd Rock Ultrasound, LLC.

Footnotes

Meetings: None

Conflicts of interest: None

REFERENCES

  • 1.Kocher KE, Meurer WJ, Fazel R, Scott PA, Krumholz HM, Nallamothu BK. National trends in use of computed tomography in the emergency department. Ann Emerg Med. 2011;58:452–62. doi: 10.1016/j.annemergmed.2011.05.020. [DOI] [PubMed] [Google Scholar]
  • 2.Larson DB, Johnson LW, Schnell BM, Salisbury SR, Forman HP. National trends in CT use in the emergency department: 1995-2007. Radiology. 2011;258:164–73. doi: 10.1148/radiol.10100640. [DOI] [PubMed] [Google Scholar]
  • 3.Broder J, Warshauer DM. Increasing utilization of computed tomography in the adult emergency department, 2000-2005. Emerg Radiol. 2006;13:25–30. doi: 10.1007/s10140-006-0493-9. [DOI] [PubMed] [Google Scholar]
  • 4.Lee J, Kirschner J, Pawa S, Wiener DE, Newman DH, Shah K. Computed tomography use in the adult emergency department of an academic urban hospital from 2001 to 2007. Ann Emerg Med. 2010;56:591–6. doi: 10.1016/j.annemergmed.2010.05.027. [DOI] [PubMed] [Google Scholar]
  • 5.Korley FK, Pham JC, Kirsch TD. Use of advanced radiology during visits to US emergency departments for injury-related conditions, 1998-2007. JAMA. 2010;304:1465–71. doi: 10.1001/jama.2010.1408. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Raja AS, Ip IK, Sodickson AD, et al. Radiology utilization in the emergency department: trends of the past 2 decades. AJR Am J Roentgenol. 2014;203:355–60. doi: 10.2214/AJR.13.11892. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Yoon P, Steiner I, Reinhardt G. Analysis of factors influencing length of stay in the emergency department. CJEM. 2003;5:155–61. doi: 10.1017/s1481803500006539. [DOI] [PubMed] [Google Scholar]
  • 8.Radiation-Emitting Products: Appropriate Use. Available at: http://www.fda.gov/Radiation-EmittingProducts/RadiationSafety/RadiationDoseReduction/ucm299354.htm. Accessed Feb 3, 2015.
  • 9.Pines JM. Trends in the rates of radiography use and important diagnoses in emergency department patients with abdominal pain. Med Care. 2009;47:782–6. doi: 10.1097/MLR.0b013e31819748e9. [DOI] [PubMed] [Google Scholar]
  • 10.Mitchell AM, Jones AE, Tumlin JA, Kline JA. Immediate complications of intravenous contrast for computed tomography imaging in the outpatient setting are rare. Acad Emerg Med. 2011;18:1005–9. doi: 10.1111/j.1553-2712.2011.01152.x. [DOI] [PubMed] [Google Scholar]
  • 11.Mitchell AM, Jones AE, Tumlin JA, Kline JA. Prospective study of the incidence of contrast-induced nephropathy among patients evaluated for pulmonary embolism by contrast-enhanced computed tomography. Acad Emerg Med. 2012;19:618–25. doi: 10.1111/j.1553-2712.2012.01374.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Brenner DJ, Hall EJ. Computed tomography--an increasing source of radiation exposure. N Engl J Med. 2007;357:2277–84. doi: 10.1056/NEJMra072149. [DOI] [PubMed] [Google Scholar]
  • 13.Pearce MS, Salotti JA, Little MP, et al. Radiation exposure from CT scans in childhood and subsequent risk of leukaemia and brain tumours: a retrospective cohort study. Lancet. 2012;380:499–505. doi: 10.1016/S0140-6736(12)60815-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Goske MJ, Applegate KE, Boylan J, et al. The 'Image Gently' campaign: increasing CT radiation dose awareness through a national education and awareness program. Pediatr Radiol. 2008;38:265–9. doi: 10.1007/s00247-007-0743-3. [DOI] [PubMed] [Google Scholar]
  • 15.Brink JA, Amis ES., Jr. Image Wisely: a campaign to increase awareness about adult radiation protection. Radiology. 2010;257:601–2. doi: 10.1148/radiol.10101335. [DOI] [PubMed] [Google Scholar]
  • 16.Slovis TL. The ALARA concept in pediatric CT: myth or reality? Radiology. 2002;223:5–6. doi: 10.1148/radiol.2231012100. [DOI] [PubMed] [Google Scholar]
  • 17.Linton OW, Mettler FA., Jr. National conference on dose reduction in CT, with an emphasis on pediatric patients. AJR Am J Roentgenol. 2003;181:321–9. doi: 10.2214/ajr.181.2.1810321. [DOI] [PubMed] [Google Scholar]
  • 18.Food and Drug Administration FDA public health notification: reducing radiation risk from computed tomography for pediatric and small adult patients. Pediatr Radiol. 2002;32:314–6. doi: 10.1007/s00247-002-0687-6. [DOI] [PubMed] [Google Scholar]
  • 19.Hendel RC, Patel MR, Kramer CM, et al. ACCF/ACR/SCCT/SCMR/ASNC/NASCI/SCAI/SIR 2006 appropriateness criteria for cardiac computed tomography and cardiac magnetic resonance imaging: a report of the American College of Cardiology Foundation Quality Strategic Directions Committee Appropriateness Criteria Working Group, American College of Radiology, Society of Cardiovascular Computed Tomography, Society for Cardiovascular Magnetic Resonance, American Society of Nuclear Cardiology, North American Society for Cardiac Imaging, Society for Cardiovascular Angiography and Interventions, and Society of Interventional Radiology. J Am Coll Cardiol. 2006;48:1475–97. doi: 10.1016/j.jacc.2006.07.003. [DOI] [PubMed] [Google Scholar]
  • 20.American College of Radiology ACR Appropriateness Criteria®. Available at: http://www.acr.org/Quality-Safety/Appropriateness-Criteria. Accessed Feb 3, 2015.
  • 21.American Board of Internal Medicine Choosing Wisely | An Initiative of the ABIM Foundation. Available at: http://www.choosingwisely.org/. Accessed Feb 2, 2015.
  • 22.Center for Medicare & Medicaid Innovation Medicare Imaging Demonstration. Available at: https://www.cms.gov/Medicare/Demonstration-Projects/DemoProjectsEvalRpts/Medicare-Demonstrations-Items/CMS1222075.html. Accessed Feb 3, 2015.
  • 23.National Quality Forum Efficiency: Imaging Efficiency. Available at: http://www.qualityforum.org/projects/imaging_efficiency.aspx. Accessed Feb 3, 2015.
  • 24.National Heart, Lung, and Blood Institute Media Availability: More Research on Risks and Benefits of Medical Imaging Needed. Available at: http://www.nhlbi.nih.gov/news/press-releases/2009/media-availability-more-research-on-risks-and-benefits-of-medical-imaging-needed. Accessed Feb 3, 2015.
  • 25.Levin DC, Parker L, Sunshine JH, Bushee G, Merritt CR. Role of emergency medicine physicians in US performed in patients in the emergency department: how substantial is their participation? Radiology. 2000;216:265–8. doi: 10.1148/radiology.216.1.r00jl28265. [DOI] [PubMed] [Google Scholar]
  • 26.Informed Medical Decisions Foundation What Is Shared Decision Making? Available at: http://www.informedmedicaldecisions.org/what-is-shared-decision-making/. Accessed Feb 3, 2015.
  • 27.Hartwig HD, Clingenpeel J, Perkins AM, Rose W, Abdullah-Anyiwo J. Parental knowledge of radiation exposure in medical imaging used in the pediatric emergency department. Pediatr Emerg Care. 2013;29:705–9. doi: 10.1097/PEC.0b013e3182949066. [DOI] [PubMed] [Google Scholar]
  • 28.Lee CI, Flaster HV, Haims AH, Monico EP, Forman HP. Diagnostic CT scans: institutional informed consent guidelines and practices at academic medical centers. Am J Roentgenol. 2006;187:282–7. doi: 10.2214/AJR.05.0813. [DOI] [PubMed] [Google Scholar]
  • 29.Zwank MD, Leow M, Anderson CP. Emergency department patient knowledge and physician communication regarding CT scans. Emerg Med J. 2014;31:824–6. doi: 10.1136/emermed-2012-202294. [DOI] [PubMed] [Google Scholar]
  • 30.Robey TE, Edwards K, Murphy MK. Barriers to computed tomography radiation risk communication in the emergency department: a qualitative analysis of patient and physician perspectives. Acad Emerg Med. 2014;21:122–9. doi: 10.1111/acem.12311. [DOI] [PubMed] [Google Scholar]
  • 31.Kuppermann N, Holmes JF, Dayan PS, et al. Identification of children at very low risk of clinically-important brain injuries after head trauma: a prospective cohort study. Lancet. 2009;374:1160–70. doi: 10.1016/S0140-6736(09)61558-0. [DOI] [PubMed] [Google Scholar]
  • 32.Schonfeld D, Bressan S, Da Dalt L, Henien MN, Winnett JA, Nigrovic LE. Pediatric Emergency Care Applied Research Network head injury clinical prediction rules are reliable in practice. Arch Dis Child. 2014;99:427–31. doi: 10.1136/archdischild-2013-305004. [DOI] [PubMed] [Google Scholar]
  • 33.Osmond MH, Klassen TP, Wells GA, et al. CATCH: a clinical decision rule for the use of computed tomography in children with minor head injury. CMAJ. 2010;182:341–8. doi: 10.1503/cmaj.091421. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Dunning J, Daly JP, Lomas JP, Lecky F, Batchelor J, Mackway-Jones K. Derivation of the children's head injury algorithm for the prediction of important clinical events decision rule for head injury in children. Arch Dis Child. 2006;91:885–91. doi: 10.1136/adc.2005.083980. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Easter JS, Bakes K, Dhaliwal J, Miller M, Caruso E, Haukoos JS. Comparison of PECARN, CATCH, and CHALICE rules for children with minor head injury: a prospective cohort study. Ann Emerg Med. 2014;64:145–52. doi: 10.1016/j.annemergmed.2014.01.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Latif RK, Bautista AF, Memon SB, et al. Teaching aseptic technique for central venous access under ultrasound guidance: a randomized trial comparing didactic training alone to didactic plus simulation-based training. Anesth Analg. 2012;114:626–33. doi: 10.1213/ANE.0b013e3182405eb3. [DOI] [PubMed] [Google Scholar]
  • 37.Sites BD, Gallagher JD, Cravero J, Lundberg J, Blike G. The learning curve associated with a simulated ultrasound-guided interventional task by inexperienced anesthesia residents. Reg Anesth Pain Med. 2004;29:544–8. doi: 10.1016/j.rapm.2004.08.014. [DOI] [PubMed] [Google Scholar]
  • 38.Accreditation Council for Graduate Medical Education The Emergency Medicine Milestone Project. Available at: https://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/EmergencyMedicineMilestones.pdf. Accessed Feb 3, 2015.
  • 39.Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65:S63–7. doi: 10.1097/00001888-199009000-00045. [DOI] [PubMed] [Google Scholar]
  • 40.Klassen TP, Reed MH, Stiell IG, et al. Variation in utilization of computed tomography scanning for the investigation of minor head trauma in children: a Canadian experience. Acad Emerg Med. 2000;7:739–44. doi: 10.1111/j.1553-2712.2000.tb02260.x. [DOI] [PubMed] [Google Scholar]
  • 41.Mannix R, Meehan WP, Monuteaux MC, Bachur RG. Computed tomography for minor head injury: variation and trends in major United States pediatric emergency departments. J Pediatr. 2012;160:136–9 e1. doi: 10.1016/j.jpeds.2011.06.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Natale JE, Joseph JG, Rogers AJ, et al. Cranial computed tomography use among children with minor blunt head trauma: association with race/ethnicity. Arch Pediatr Adolesc Med. 2012;166:732–7. doi: 10.1001/archpediatrics.2012.307. [DOI] [PubMed] [Google Scholar]
  • 43.Raja AS, Andruchow J, Zane R, Khorasani R, Schuur JD. Use of neuroimaging in US emergency departments. Arch Intern Med. 2011;171:260–2. doi: 10.1001/archinternmed.2010.520. [DOI] [PubMed] [Google Scholar]
  • 44.Blackwell CD, Gorelick M, Holmes JF, Bandyopadhyay S, Kuppermann N. Pediatric head trauma: changes in use of computed tomography in emergency departments in the United States over time. Ann Emerg Med. 2007;49:320–4. doi: 10.1016/j.annemergmed.2006.09.025. [DOI] [PubMed] [Google Scholar]
  • 45.Marin JR, Weaver MD, Barnato AE, Yabes JG, Yealy DM, Roberts MS. Variation in emergency department head computed tomography use for pediatric head trauma. Acad Emerg Med. 2014;21:987–95. doi: 10.1111/acem.12458. [DOI] [PubMed] [Google Scholar]
  • 46.Kline JA, Courtney DM, Kabrhel C, et al. Prospective multicenter evaluation of the pulmonary embolism rule-out criteria. J Thromb Haemost. 2008;6:772–80. doi: 10.1111/j.1538-7836.2008.02944.x. [DOI] [PubMed] [Google Scholar]
  • 47.Wells PS, Anderson DR, Rodger M, et al. Excluding pulmonary embolism at the bedside without diagnostic imaging: management of patients with suspected pulmonary embolism presenting to the emergency department by using a simple clinical model and d-dimer. Ann Intern Med. 2001;135:98–107. doi: 10.7326/0003-4819-135-2-200107170-00010. [DOI] [PubMed] [Google Scholar]
  • 48.Runyon MS, Richman PB, Kline JA. Emergency medicine practitioner knowledge and use of decision rules for the evaluation of patients with suspected pulmonary embolism: variations by practice setting and training level. Acad Emerg Med. 2007;14:53–7. doi: 10.1197/j.aem.2006.07.032. [DOI] [PubMed] [Google Scholar]
  • 49.Raja AS, Gupta A, Ip IK, Mills AM, Khorasani R. The use of decision support to measure documented adherence to a national imaging quality measure. Acad Radiol. 2014;21:378–83. doi: 10.1016/j.acra.2013.10.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Carpenter CR, Raja AS. Arming the Bayesian physician to rule out pulmonary embolism: using evidence-based diagnostics to combat overtesting. Acad Emerg Med. 2014;21:1036–8. doi: 10.1111/acem.12450. [DOI] [PubMed] [Google Scholar]
  • 51.Cabana MD, Rand CS, Powe NR, et al. Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282:1458–65. doi: 10.1001/jama.282.15.1458. [DOI] [PubMed] [Google Scholar]
  • 52.Garfield FB, Garfield JM. Clinical judgment and clinical practice guidelines. Int J Technol Assess Health Care. 2000;16:1050–60. doi: 10.1017/s0266462300103113. [DOI] [PubMed] [Google Scholar]
  • 53.Tunis SR, Hayward RS, Wilson MC, et al. Internists' attitudes about clinical practice guidelines. Ann Intern Med. 1994;120:956–63. doi: 10.7326/0003-4819-120-11-199406010-00008. [DOI] [PubMed] [Google Scholar]
  • 54.Sanson BJ, Lijmer JG, Mac Gillavry MR, Turkstra F, Prins MH, Buller HR. Comparison of a clinical probability estimate and two clinical models in patients with suspected pulmonary embolism. ANTELOPE-Study Group. Thromb Haemost. 2000;83:199–203. [PubMed] [Google Scholar]
  • 55.Runyon MS, Webb WB, Jones AE, Kline JA. Comparison of the unstructured clinician estimate of pretest probability for pulmonary embolism to the Canadian score and the Charlotte rule: a prospective observational study. Acad Emerg Med. 2005;12:587–93. doi: 10.1197/j.aem.2005.02.010. [DOI] [PubMed] [Google Scholar]
  • 56.Kabrhel C, Courtney DM, Camargo CA, Jr, et al. Potential impact of adjusting the threshold of the quantitative D-dimer based on pretest probability of acute pulmonary embolism. Acad Emerg Med. 2009;16:325–32. doi: 10.1111/j.1553-2712.2009.00368.x. [DOI] [PubMed] [Google Scholar]
  • 57.Andruchow JE, Raja AS, Prevedello LM, Zane RD, Khorasani R. Variation in head computed tomography use for emergency department trauma patients and physician risk tolerance. Arch Intern Med. 2012;172:660–1. doi: 10.1001/archinternmed.2011.2243. [DOI] [PubMed] [Google Scholar]
  • 58.Pines JM, Hollander JE, Isserman JA, et al. The association between physician risk tolerance and imaging use in abdominal pain. Am J Emerg Med. 2009;27:552–7. doi: 10.1016/j.ajem.2008.04.031. [DOI] [PubMed] [Google Scholar]
  • 59.Goold SD, Hofer T, Zimmerman M, Hayward RA. Measuring physician attitudes toward cost, uncertainty, malpractice, and utilization review. J Gen Intern Med. 1994;9:544–9. doi: 10.1007/BF02599278. [DOI] [PubMed] [Google Scholar]
  • 60.Studdert DM, Mello MM, Sage WM, et al. Defensive medicine among high-risk specialist physicians in a volatile malpractice environment. JAMA. 2005;293:2609–17. doi: 10.1001/jama.293.21.2609. [DOI] [PubMed] [Google Scholar]
  • 61.Allison JJ, Kiefe CI, Cook EF, Gerrity MS, Orav EJ, Centor R. The association of physician attitudes about uncertainty and risk taking with resource use in a Medicare HMO. Med Decis Making. 1998;18:320–9. doi: 10.1177/0272989X9801800310. [DOI] [PubMed] [Google Scholar]
  • 62.DeKay ML, Asch DA. Is the defensive use of diagnostic tests good for patients, or bad? Med Decis Making. 1998;18:19–28. doi: 10.1177/0272989X9801800105. [DOI] [PubMed] [Google Scholar]
  • 63.Wong AC, Kowalenko T, Roahen-Harrison S, Smith B, Maio RF, Stanley RM. A survey of emergency physicians' fear of malpractice and its association with the decision to order computed tomography scans for children with minor head trauma. Pediatr Emerg Care. 2011;27:182–5. doi: 10.1097/PEC.0b013e31820d64f7. [DOI] [PubMed] [Google Scholar]
  • 64.Khorasani R. Computerized physician order entry and decision support: improving the quality of care. Radiographics. 2001;21:1015–8. doi: 10.1148/radiographics.21.4.g01jl371015. [DOI] [PubMed] [Google Scholar]
  • 65.Sistrom CL, Dang PA, Weilburg JB, Dreyer KJ, Rosenthal DI, Thrall JH. Effect of computerized order entry with integrated decision support on the growth of outpatient procedure volumes: seven-year time series analysis. Radiology. 2009;251:147–55. doi: 10.1148/radiol.2511081174. [DOI] [PubMed] [Google Scholar]
  • 66.Raja AS, Ip IK, Prevedello LM, et al. Effect of computerized clinical decision support on the use and yield of CT pulmonary angiography in the emergency department. Radiology. 2012;262:468–74. doi: 10.1148/radiol.11110951. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Smith-Bindman R, Miglioretti DL, Larson EB. Rising use of diagnostic medical imaging in a large integrated health system. Health Aff (Millwood) 2008;27:1491–502. doi: 10.1377/hlthaff.27.6.1491. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Sodickson A, Baeyens PF, Andriole KP, et al. Recurrent CT, cumulative radiation exposure, and associated radiation-induced cancer risks from CT of adults. Radiology. 2009;251:175–84. doi: 10.1148/radiol.2511081296. [DOI] [PubMed] [Google Scholar]
  • 69.Griffey RT, Sodickson A. Cumulative radiation exposure and cancer risk estimates in emergency department patients undergoing repeat or multiple CT. AJR Am J Roentgenol. 2009;192:887–92. doi: 10.2214/AJR.08.1351. [DOI] [PubMed] [Google Scholar]
  • 70.Broder J, Bowen J, Lohr J, Babcock A, Yoon J. Cumulative CT exposures in emergency department patients evaluated for suspected renal colic. J Emerg Med. 2007;33:161–8. doi: 10.1016/j.jemermed.2006.12.035. [DOI] [PubMed] [Google Scholar]
  • 71.Amis ES, Jr, Butler PF, Applegate KE, et al. American College of Radiology white paper on radiation dose in medicine. J Am Coll Radiol. 2007;4:272–84. doi: 10.1016/j.jacr.2007.03.002. [DOI] [PubMed] [Google Scholar]
  • 72.Larson DB, Johnson LW, Schnell BM, Goske MJ, Salisbury SR, Forman HP. Rising use of CT in child visits to the emergency department in the United States, 1995-2008. Radiology. 2011;259:793–801. doi: 10.1148/radiol.11101939. [DOI] [PubMed] [Google Scholar]
  • 73.Johnson EK, Graham DA, Chow JS, Nelson CP. Nationwide emergency department imaging practices for pediatric urolithiasis: room for improvement. J Urol. 2014;192(1):200–6. doi: 10.1016/j.juro.2014.01.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74.Morin RL, Coombs LP, Chatfield MB. ACR dose index registry. J Am Coll Radiol. 2011;8:288–91. doi: 10.1016/j.jacr.2010.12.022. [DOI] [PubMed] [Google Scholar]
  • 75.Lukasiewicz A, Bhargavan-Chatfield M, Coombs L, et al. Radiation dose index of renal colic protocol CT studies in the United States: a report from the American College of Radiology National Radiology Data Registry. Radiology. 2014;271:445–51. doi: 10.1148/radiol.14131601. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 76.Conway PH, Clancy C. Comparative-effectiveness research--implications of the Federal Coordinating Council's report. N Engl J Med. 2009;361:328–30. doi: 10.1056/NEJMp0905631. [DOI] [PubMed] [Google Scholar]
  • 77.Smith-Bindman R, Aubin C, Bailitz J, et al. Ultrasonography versus computed tomography for suspected nephrolithiasis. N Engl J Med. 2014;371:1100–10. doi: 10.1056/NEJMoa1404446. [DOI] [PubMed] [Google Scholar]
  • 78.Doria AS, Moineddin R, Kellenberger CJ, et al. US or CT for diagnosis of appendicitis in children and adults? A meta-analysis. Radiology. 2006;241:83–94. doi: 10.1148/radiol.2411050913. [DOI] [PubMed] [Google Scholar]
  • 79.Mittal MK, Dayan PS, Macias CG, et al. Performance of ultrasound in the diagnosis of appendicitis in children in a multicenter cohort. Acad Emerg Med. 2013;20:697–702. doi: 10.1111/acem.12161. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 80.Krishnamoorthi R, Ramarajan N, Wang NE, et al. Effectiveness of a staged US and CT protocol for the diagnosis of pediatric appendicitis: reducing radiation exposure in the age of ALARA. Radiology. 2011;259:231–9. doi: 10.1148/radiol.10100984. [DOI] [PubMed] [Google Scholar]
  • 81.Aspelund G, Fingeret A, Gross E, et al. Ultrasonography/MRI versus CT for diagnosing appendicitis. Pediatrics. 2014;133:586–93. doi: 10.1542/peds.2013-2128. [DOI] [PubMed] [Google Scholar]
  • 82.Leeuwenburgh MM, Wiarda BM, Wiezer MJ, et al. Comparison of imaging strategies with conditional contrast-enhanced CT and unenhanced MR imaging in patients suspected of having appendicitis: a multicenter diagnostic performance study. Radiology. 2013;268:135–43. doi: 10.1148/radiol.13121753. [DOI] [PubMed] [Google Scholar]
  • 83.Basaran A, Basaran M. Diagnosis of acute appendicitis during pregnancy: a systematic review. Obstet Gynecol Surv. 2009;64:481–8. doi: 10.1097/OGX.0b013e3181a714bf. [DOI] [PubMed] [Google Scholar]
  • 84.Oto A, Ernst RD, Ghulmiyyah LM, et al. MR imaging in the triage of pregnant patients with acute abdominal and pelvic pain. Abdom Imaging. 2009;34:243–50. doi: 10.1007/s00261-008-9381-y. [DOI] [PubMed] [Google Scholar]

RESOURCES