Orbital Fracture Clinical Decision Rule Development: Burden of Disease and Use of a Mandatory Electronic Survey Instrument

Abstract

Objectives

In preparation for development of a clinical decision rule (CDR) to promote more efficient use of computed tomography (CT) for diagnosing orbital fractures, the authors sought to estimate the annual incidence of orbital fractures in emergency departments (EDs), and the usage of CT to make these diagnoses. The authors also sought to evaluate a mandatory electronic data collection instrument (EDCI) administered to providers to facilitate CDR data collection.

Methods

National estimates were made by analyzing the 2007 National Hospital Ambulatory Medical Care Survey database, while hospital billing system and coding data were used to make local estimates. An EDCI was integrated into the CT ordering system such that providers had to complete the form in order to perform a CT. Because the EDCI had to be filled out for every CT ordered, data collection efficiency was measured by compliance (counting the number of unrealistic data collection instrument answers) and by timing a convenience sample of providers completing the EDCI.

Results

Out of 116.8 million ED visits in the United States in 2007, 4.1 million patients were treated for injuries of the eye and face. Of those, 820,252 patients underwent CT imaging, with 102,999 patients (12.5%) diagnosed with an orbital fracture. In our local hospital system with 122,500 annual ED visits, 752 CTs of orbits were performed, with 172 (23%) orbital fractures. The EDCI compliance rate was 94.9%, and took less than five minutes to complete.

Conclusions

National and local data demonstrate a low yield for CT imaging in identifying orbital fractures. Data collection using a mandatory electronic data collection instrument linked to computerized provider order entry can provide prospective, consecutive patient data that are needed to develop a CDR for the selective use of CT imaging in orbital trauma. Such a decision rule could increase the efficiency in diagnosing orbital fractures, thereby improving patient care, reducing radiation exposure, and decreasing costs.

INTRODUCTION

In 2006, U. S. emergency departments (EDs) treated approximately 3.79 million patients for eye and face injuries.¹ Computed tomography (CT) is the established imaging method used for assessing orbital trauma, but has certain disadvantages including limited availability, high cost, and exposure to substantial ionizing radiation.^2,3 On the other hand, timely and accurate diagnosis is critical because missed orbital fractures can have permanent consequences, including diplopia and loss of binocular vision.⁴ Weighing the costs and long-term risks of orbital CTs against the consequences of missed fractures likely results in overuse of radiographic imaging. However, to the best of our knowledge, estimates of the prevalence of orbital trauma and the usage of CT imaging for diagnosis of orbital fractures have not been published.

To reduce unnecessary utilization of radiographic testing, clinical decision rules (CDRs) have been derived and validated for ankle films, knee films, cervical spine films, and head CTs for blunt trauma.⁵ Studies have demonstrated decreased resource utilization without compromising patient safety, and CDRs have been widely accepted into the clinical practice of emergency and primary care physicians.^6,7 Developing CDRs requires knowledge of the signs and symptoms associated with the outcome of interest. We did not find any published studies investigating the ability of the clinical exam to predict the presence of orbital fracture. Clinical signs and symptoms of orbital trauma have only been described in association with severity of fractures or their anatomic location.^8,9

Data acquisition for CDR research can be challenging and expensive. It often requires on-site research assistants gathering data as patient care is administered. Traditionally this involves manual coding of medical records, or acquiring data via the collection instrument. This process is costly, and remains susceptible to inaccuracy.^10,11 To our knowledge, there is no published CDR research linking a computerized data collection instrument to an electronic ordering system to accomplish paperless data collection.

Our first hypothesis was that we can successfully estimate the prevalence of orbital fractures and CT imaging using a national database and local hospital system records. Our second hypothesis was that a mandatory electronic data collection instrument (EDCI) can efficiently collect clinical predictor data with minimal noncompliance.

METHODS

Study Design

This study was conducted in three phases. Phase I was an analysis of the latest publicly available National Hospital Ambulatory Medical Care Survey (NHAMCS).¹² Phase II was a review of billing and coding data in our public urban hospital system, North Bronx Healthcare Network (NBHN). Phase III was implementation of a mandatory EDCI conducted as a prospective cohort study of ED patients with blunt orbital trauma. The study received institutional review board approval with waivers of patient and physician consent. Phase III data collection began July 1, 2007 and concluded October 16, 2009.

Study Populations

The NHAMCS is a national probability sample of visits to U.S. hospital emergency and outpatient departments; only the ED data were used in this analysis. NHAMCS uses a four-stage sampling design covering geographic primary sampling units (PSUs), hospitals within PSUs, EDs within hospitals, and patient visits within EDs. Hospitals are selected with a probability proportional to the number of ED visits after being stratified by region, class, and size. Patient visits are systematically selected over a randomly assigned 4-week period, while hospitals are stratified by region, type. and size.

The NBHN’s hospital system consists of a Level I trauma center (75,000 annual ED visits) and a Level II trauma center (47,500 annual ED visits). For Phase III, ED patients (age ≥18 years) suffering blunt orbital trauma undergoing orbital CT imaging were eligible for inclusion and were enrolled in a consecutive fashion through the completion of the mandatory EDCI.

Study Protocol

For Phase I, International Classification of Diseases 9^th Revision (ICD-9) coding identified patients diagnosed with orbital fractures (801.x and 802.6-802.8) using appropriate survey methods in the Stata version 10.1 statistical software package (Stata Corp, College Station, TX). Patient weights in the database were used to obtain estimated national values. To identify all patients with traumatic eye and facial injuries, we used a broader set of ICD-9 coding (802.x, 830.x, 848.x, 870.x, 871.x, 872.x, 873.x, 918.x, 921.x, 940.x, 941.x, and 950.x). To identify CT usage, we used the NHAMCS diagnostic services survey item “CATSCAN.”

For Phase II, ICD-9 coding identified patients diagnosed with orbital fractures in the NBHN network. Our trauma protocols mandate the use of orbital CTs for the diagnosis of orbital fracture, so we used Current Procedural Terminology (CPT) codes (70480, 70481, 70482) to quantify CT usage.

For Phase III, emergency physicians filled out the EDCI as part of the orbital CT order. Completing the EDCI was a mandatory part of ordering the orbital CT, but the specific answers to the questions had no effect on the physician’s ability to order the CT. The data collection instrument included 15 questions about the presence of blunt orbital trauma and exclusion criteria to screen out ineligible cases (Data Supplement 1). A paper version of this form of data capture had been used in this hospital system and in multi-center studies in the past, and been shown to successfully enable consecutive patient enrollment without adversely affecting patient care.^13,14

Potential clinical predictors for the data collection instrument were identified by extensive literature review by the study investigators, followed by a consensus conference with the institutional department heads of the Craniofacial Trauma Service (a multidisciplinary service comprised of otolaryngologists, ophthalmologists, oromaxillofacial surgeons, and plastic surgeons). Prior to implementation, we educated providers to the mandatory data collection instrument through didactic lectures reviewing diagnosis of orbital fractures and provision of review materials in print and via electronic mail.

To measure compliance with the electronic data collection instrument, we looked for patterns of answers that were not likely to occur in real-life: 1) all “yes” - while it is possible that a truly unfortunate trauma patient would have every clinical predictor, it is quite unlikely; and 2) all “unknown” - some of the predictors are based on simple observation of the patient’s face, i.e. periorbital ecchymosis, therefore all “unknown” is unlikely to be a true assessment. Research assistants measured the time it took a convenience sample of providers to complete the electronic data collection instrument.

Data Analysis

National data collection instrument results were reported as estimates with standard errors. Local billing and coding results were reported as counts and percentages. Electronic data collection instrument compliance and completion time were reported as counts and percentages.

RESULTS

Out of 116.8 million ED visits in 2007, the NHAMCS shows 4.1 million patients were treated for injuries of the eye and face (Table 1). Within these, 820,252 patients underwent CT imaging, and of those, 102,999 patients (12.5%) were ultimately diagnosed with orbital fracture.

Table 1.

NHAMCS and NBHN data on eye & face injuries

Source	Condition	Estimate	Standard Error	95% CI
NHAMCS	Eye & Face Injuries	4,106,902	299,243.5	3,516,512 – 4,697,292
NHAMCS	CT Usage	820,252	83,947.4	654,629 – 985,875
NHAMCS	Orbital Fractures	102999	21481.2	60,618 – 145,380
NBHN	CT Usage	748	-	-
NBHN	Orbital Fractures	172	-	-

NHAMCS = National Hospital Ambulatory Medical Care Survey; NBHN = North Bronx Healthcare Network; CT = computed tomography

Based upon CPT codes, we found that 748 patients underwent orbital CT scanning in the NBHN EDs in 2006. According to ICD-9 coding, 172 adult patients were diagnosed with orbital fractures (23% of patients undergoing orbital CT).

Out of 3,123 EDCIs completed, 2,552 (81.7%) were for patients suffering traumatic injury. Only 130 (5.1%) of those were noncompliant as defined above: four all “yes” and 126 all “unknown.” This represents 94.9% compliance with a mandatory electronic data collection instrument. Of 142 observations of providers filling out the data collection instrument, none took more than five minutes to complete.

DISCUSSION

Overuse of CT imaging contributes to rising health care costs. The rate of CT usage in U.S. EDs has quadrupled from 1996 to 2007,¹⁵ while improvements in diagnostic yield have been called into question.¹⁶ From our national and local estimates, almost 80% of CT scans ordered were negative for fractures, indicating an inefficient use of CT imaging. Furthermore, orbital CT scanning exposes the patient to substantial radiation - a dose of 10–130 mGray (10 mGray = 1 rad) is administered to the lens of the eye, while the thyroid gland is exposed to 1–3 mGray.^17,18 While the thyroid gland is widely recognized as radiosensitive, so too is the lens of the eye; as little as 500–2,000 mGray causes detectable opacities, and exposures of >4,000 mGray cause visually impairing cataracts.¹⁹

To identify the need for a CDR, a clearly definable and clinically important condition must be identified where the diagnostic test of choice is used inefficiently, or where there is significant variation in practice. As demonstrated by the national and local data, CT usage for orbital fractures is inefficient. To derive a CDR, clinicians must assess potential predictor variables in patients in a standardized, prospective fashion. We have demonstrated that a mandatory EDCI linked to computerized provider order entry can efficiently enable paperless data collection on consecutive patients. Such a system can streamline the derivation and validation of CDRs at reduced cost.

LIMITATIONS

The NHAMCS sampling may create unreliable estimates. However, with relative standard errors of less than 30%, our estimates are reliable according to the National Center for Health Statistics.²⁰

Without having a discrete category for orbital CT in the NHAMCS database, it is possible that CT usage in orbital trauma is overestimated. The NHAMCS item CATSCAN would have tracked the performance of any CT imaging for the patient, regardless of anatomic location. Furthermore, even in patients with facial trauma, the investigation of other facial injuries may have prompted the ordering of CT imaging. It is likely that CT imaging efficiency for orbital fracture is better reflected by our local hospital system estimates, where we could specifically measure orbital CT usage. However, the generalizability of our usage may be limited because it represents the practice at only one urban hospital system.

Billing and coding data have the inherent limitations of retrospective data, such as being coded properly for data extraction. However, procedural codes are captured from the electronic radiology ordering system, and diagnostic coding is captured from the electronic hospital discharge system, so data fidelity should be high.

Although our approach to evaluating the accuracy of EDCI completion is similar to prior studies,^21,22 there may have been other patterns of noncompliant EDCI completion that we were unable to detect. Other possible patterns of noncompliance, such as alternating “yes” and “unknown” answers (the keyboard keys are next to each other) were not found. Because providers were aware that the EDCI is part of the medical record, it is unlikely that they would have fabricated clinical signs and symptoms simply to skip the data collection instrument. This awareness may explain why noncompliant EDCI results skewed heavily to documenting all “unknown” rather than all “yes.”

CONCLUSIONS

National and local data demonstrate a low yield for computed tomography imaging in identifying orbital fractures. Data collection using a mandatory electronic data collection instrument linked to computerized provider order entry can efficiently provide prospective, consecutive patient data needed to develop a clinical decision rule for the selective use of computed tomography imaging in orbital trauma. Such a decision rule could increase the efficiency in diagnosing orbital fractures, thereby improving patient care, reducing radiation exposure, and decreasing costs.