Abstract
Emergency Department (ED) data are a key component of bioterrorism surveillance systems. Little research has been done to examine differences in ED data capture and entry across hospitals, regions and states. The purpose of this study was to describe the current state of ED data for use in bioterrorism surveillance in 2 regions of the country. We found that chief complaint (CC) data are available electronically in 54% of the North Carolina EDs surveyed, and in 100% of the Seattle area EDs. Over half of all EDs reported that CCs are recorded in free text form. Though all EDs have electronic diagnosis data, less than half report that diagnoses are coded within 24 hours of the ED visit.
INTRODUCTION
The American Medical Informatics Association (AMIA) has advocated a national health information system for tracking and detecting national health threats.1 To support such systems, AMIA recommends adaptation of existing information systems, and rapid development and deployment of new public health surveillance systems as well as computerized health records. Public health officials and health care agencies are partnering to develop surveillance systems for bioterrorism in particular, and research has focused on various aspects of bioterrorism detection and tracking, including privacy, logistics, approaches to detection, and syndromic surveillance.2–4
Emergency department data are a key component of bioterrorism surveillance systems.5 Researchers have demonstrated that ED data can be used to detect disease outbreaks 1–2 weeks earlier than through conventional disease reporting methods.6,7 Specific ED data elements that have been used in bioterrorism surveillance systems include chief complaint (free text or coded with ICD-9), diagnosis (ICD-9), symptom survey, age, gender, visit site, arrival mode/date/time, disposition, ED utilization rates, and geo code or address. 3,5
In order to develop effective bioterrorism surveillance systems using ED data, it is necessary to understand and manage the types of variation in electronic ED data availability, timeliness, sources, and adherence to standards. There are several challenges to using ED data for bioterrorism surveillance. One limitation of ED data for bioterrorism surveillance is the lack of standards. Standardization of ED data has been recommended as a cornerstone for facilitating information management for clinical, research and administrative purposes in Emergency Medicine.8–9 An ongoing national effort to establish standards for ED data led to the release of Data Elements for Emergency Departments (DEEDS) 1.0 in 1997.10–11 156 data elements were included in this first edition of DEEDS, which recommended existing vocabularies for those data elements with such standards. Existing standards included ICD-9-CM for diagnosis, ASTM and HL7 for date of birth and sex, HCFA for ED facility ID, and HL7 for address. No standard existed for chief complaint (CC), arrival mode/date/time, disposition, or symptom survey. For CC, the DEEDS authors recommended that existing terminologies be evaluated for possible use as a standard. A few controlled terminologies for CC have been developed, though no standard has yet to emerge.12–15
Another shortcoming of existing ED data for public health surveillance is that not all data are available in electronic form.16–18 Even when electronic information is available, it may not be documented in a timely manner. For example, at some hospitals, ED diagnosis coding may not be completed until several weeks after an ED visit.19 The Frontlines of Medicine Project has proposed a framework for the reporting of standardized, electronic ED data in a timely manner for public health surveillance.20 The project continues to work on a set of recommendations for data collection systems, data standards and rapid deployment of a public health information infrastructure.
Little research has been done to examine differences in ED data and in the workflow around ED data capture and data entry across hospitals, regions and states. The purpose of this study was to describe the current state of ED data for use in bioterrorism surveillance: availability in electronic form, timeliness, sources, and adherence to standards. We also sought to compare the current state of ED data in two different regions of the country, and to compare temporal variations in electronic ED data capture in North Carolina, contrasting 2003 survey data with data from 1999.
METHODS
In this descriptive study, we surveyed EDs in two geographic areas, each with surveillance systems under development. The two areas were the state of North Carolina (NC) and Seattle -King County, in Washington State. The North Carolina Emergency Department Database (NCEDD) is a statewide public health surveillance system that utilizes ED data exclusively. Currently data from three EDs are collected electronically and sent via secure Internet connection to the State Division of Public Health for aggregation and analysis for public health surveillance. The program is currently in the process of expanding to an additional 10–20 hospitals, with an ultimate goal of including all EDs in NC.
All EDs in North Carolina were included in this study, and data were collected at project inception in 1999, and again in 2003. A structured 41-item paper survey was mailed to all 114 North Carolina EDs in 1999, and a more extensive, structured 52-item web-based survey was conducted during February 2003. In 2003, 111 EDs were included; two had closed in the interim, and one hospital had converted to a long-term care facility. For both data collection periods, those who did not respond to the survey initially were then contacted by phone and the data were collected verbally. In 2003, we also followed up with some respondents by phone or email to clarify questionable responses to the web-based survey. The study was approved by the IRB and ED personnel expressed consent to participate by completing the paper or Internet survey, or verbally at the start of the phone interview. Nurse managers or medical directors were asked to respond to the survey.
The Seattle-King County surveillance system includes selected (4 of 19) EDs and large primary care networks in urban King County, WA. The system is being expanded to all 19 King County providers. The systems utilize data from ED as well as primary care visits. Data collection on ED and primary care data availability and workflow began in January 2001 and is ongoing. For this paper, only ED data are included. Data were collected during semi -structured interviews with CIOs, medical executives, IT managers, technical staff, and clinical staff, and additional information was confirmed by phone. Also, limited data from a 2001 telephone survey of King County sites were included.
RESULTS
In North Carolina, the response rate for the 1999 survey was 84% (96 of the 114 NC EDs) and for 2003 was 86% (96 of the 111 EDs). Most NC respondents were ED nurse manager/directors (60% in 1999, 93% in 2003). Other respondents included medical directors, other nurses, physicians, or administrators, and information systems staff. In Seattle-King County, the response rate for the 2003 survey was 100% (17 of 17 EDs).
Tables 1 and 2 indicate responses about what data are currently captured in electronic form for major functional categories within the ED setting.
Table 1.
ED Data Available in Electronic Form- North Carolina
| Data Description | 1999 N=96 | 2003 N=96 |
|---|---|---|
| Registration (age, sex, zip code) | 87 (91%) | 88 (92%) |
| ED logs (# visits) | 44 (46%) | 65 (68%) |
| Discharge instructions (symptom, diagnosis) | 37 (39%) | 48 (50%) |
| Physician Charting (clinical findings) | 16 (17%) | 22 (23%) |
| Nursing Charting (clinical findings) | 15 (16%) | 16 (17%) |
Table 2.
ED Data Available in Electronic Form- Seattle-King County, Washington
| Data Description | 2001–2003 N=17 |
|---|---|
| Registration (age, sex, zip code) | 17 (100%) |
| ED logs (# visits per day) | 11 (65%) |
| Discharge instructions (symptom, diagnosis) | 13 (76%) |
| Physician Notes (clinical findings) | 5 (29%) |
| Nursing Notes (clinical findings) | 5 (29%) |
Registration data are typically collected by clerical personnel after triage, and include selected data elements that may be used for bioterrorism surveillance: date of birth (age), sex, and address. ED logs are required by the JCAHO and document basic patient information (typically name, time/date of arrival, and sometimes chief complaint) in sequential order of the patients’ arrival at the ED. Automated discharge instruction systems are used to print out post-ED advice to patients, and are typically organized by symptom or diagnosis group (e.g., fever, nausea/vomiting, gastroenteritis). The discharge instruction systems also provide an electronic record stored in the patients’ medical record which documents the instructions given to the patient. Physician and nurse charting systems provide a mechanism for electronic documentation and storage of history and physical assessments, treatments and progress notes.
Respondents also reported on chief complaint (CC) data for the 2003 survey (Table 3).
Table 3.
Chief Complaint Data
| Data Description | NC+ | Seattle* |
|---|---|---|
| How collected? | n=95 | n=17 |
| Electronic | 51 (54%) | 16 (100%) |
| Paper | 44 (46%) | 0 (0%) |
| If electronic, how many fields? | n=41 | n=16 |
| 1 | 29 | 12 |
| 2 | 8 | 4 |
| 3 | 2 | 0 |
| Other | 2 | 0 |
| Data format | n=46 | n=15 |
| Free text only | 24 | 9 |
| Pick list only | 10 | 0 |
| Both | 12 | 6 |
| If CC free text, how big is input field | 12–250 | 25-“paragraphs” |
| If pick list, what is the source of the list? | n=20 | n=4 |
| Locally developed | 5 | 1 |
| Vendor w/local additions | 6 | 0 |
| Vendor | 8 | 1 |
| ICD-9-CM | 1 | 2 |
| Who documents the CC? | n=92 | n=17 |
| Nurse | 78 | 11 |
| Clerk | 10 | 8 |
| Other | 4 | 0 |
| How are nurses taught to document CC? | n=59 | n=15 |
| Patient’s exact words | 24 | 2 |
| Paraphrase pt words | 35 | 14 |
Not all respondents answered all items
Some respondents chose more than one item
Table 4 indicates the responses to questions about the process of electronic documentation of diagnosis and knowledge of DEEDS.
Table 4.
Diagnosis Data, DEEDS
| Data Description | NC+ | Seattle* |
|---|---|---|
| Diagnosis - collected electronically? | n=96 (100%) | n=14 (100%) |
| Diagnosis - who enters? | n=73 | n=14 |
| Coder | 62 | 4 |
| ED physician | 4 | 6 |
| ED clerk | 4 | 4 |
| Other | 4 | 0 |
| When is diagnosis entered in computer? | n=80 | n=13 |
| Within 24 hrs | 28 | 7 |
| Within 1 week | 39 | 2 |
| At discharge | 9 | 4 |
| Within 1 month | 1 | 0 |
| What do you know about DEEDS? | n=68 | |
| Nothing | 50 | Not Assessed |
| Heard of it, know little | 13 | |
| Incorrect information | 3 | |
| Know a lot | 2 |
Not all respondents answered all items
Some respondents chose more than one item
DISCUSSION
Electronic documentation of ED processes, and standardization of ED data, have been encouraged in recent years to improve clinical care and ED operations, reduce errors, and facilitate secondary use for research, quality assurance and public health surveillance.9–11,16,21,22 In particular, the availability of timely, electronic, standardized ED data will greatly enhance bioterrorism surveillance activities. Ideal bioterrorism surveillance systems are population based, and collect data in as near to real-time as possible.1,20
We found that electronic ED data vary in availability, timeliness, formats and adherence to standards. Overall, the Seattle area EDs had more data available in electronic form than the North Carolina sites. This may be due to the relatively rural nature of North Carolina, whereas the Seattle-King County area reflects larger, more urban hospitals that may have more extensive hospital computer systems
Only some of the specific ED data elements that have been previously identified as useful for bioterrorism surveillance are currently available in electronic form. Diagnosis data are collected in electronic form at all sites in Seattle and North Carolina, most likely because diagnoses are required for reimbursement. Most of the EDs in NC and Seattle also have registration data in electronic form, including the data elements age, sex and zip code. Electronic logs that supply the number of daily ED visits, date and time of arrival (and sometimes chief complaint,) are available in 68% of the NC sites and 65% of the Seattle sites. Electronic availability of CC data varied by region, from 100% in Seattle to only 54% in North Carolina.
Computerized discharge instructions systems appear to be gaining in popularity in the ED setting, though use varies by region. We found that discharge instructions are available in electronic form at 76% of the Seattle EDs. In North Carolina, there is a trend toward more use of electronic discharge instructions in the last four years, from 39% to 50%. Discharge instructions are typically symptom and diagnosis oriented, and may represent a new source of clinical information for syndromic surveillance.
Other potential sources of electronic clinically-oriented ED data include physician and nurses notes. Electronic charting systems are not yet widely used, but are becoming more popular. There is an opportunity to implement DEEDS-compliant systems from the start. Text headers and coded identifiers for a basic set of patient history and physical examination categories are included in DEEDS 1.0. Plans for the next DEEDS release call for a more granular set of headers and identifiers for clinical finding categories (personal communication, Daniel Pollock, March 10, 2003).
There is a limited amount of electronic ED data available in a timely fashion. Registration data are entered into ED computer systems at the start of the ED visit, so age, sex and zip code, and arrival time/date are potentially available in real time. Clinically-oriented data are less available. Less than half of the respondents report that diagnosis data are available within the 24-hour “near real time” window commonly used in bioterrorism surveillance.
Though use of ICD-9 as a diagnosis standard is essentially universal, we found a lack of widely available, electronic, standardized CC data in the EDs surveyed. The CC is typically documented at the start of the ED visit, however CC data aren’t available in electronic form in approximately half of the North Carolina EDs. And respondents from both North Carolina and Washington report that CC is entered in free text form at half or more of the sites. Even the sites that document CC using controlled terms use a variety of vendor- or locally-developed lists.
Practices for chief complaint documentation vary across EDs. While nurses document the majority of CC data, clerical staff also record CCs. And though many sites reported that nurses receive instructions on documentation of the CC, there is not a consensus on whether the CC is the “patient’s exact words” or the nurses ’ paraphrasing of the patients words in a more clinically oriented term. It would seem that “productive cough” is a more useful CC for bioterrorism surveillance than “bringing up phlegm”. Perhaps there should be two separate data elements for ED documentation, CC and patient complaint. A controlled terminology for ED CC is needed in order to take advantage of the potential of CC for timely, electronic clinically -oriented information about ED patients. In the absence of a standard for CC, methods are being developed for extracting controlled terms and concepts from free text CC data using key word searches and natural language processing (NLP).4,23
Some of the North Carolina respondents expressed concern about the use of controlled terminologies in the ED clinical arena. For example, they report that the intense time constraints at ED triage, where nurses are expected to perform a focused triage assessment in five minutes or less, make it difficult to require that nurses scroll through long drop-down lists to identify the most appropriate CC term. Many of the responding EDs that use controlled CC pick lists reported that staff often add additional text to a controlled CC, or type in a free text term instead. Effective user interfaces may be the answer to such concerns. Another option may be to allow free text CC entry, with mapping to controlled terms using NLP techniques.
There are limitations to our study methodology. Any survey that attempts to compare data gathered by different investigators, on different populations, without assessing interrater reliability, is subject to bias introduced by the data collection methods. In particular, the North Carolina data were acquired from a more homogeneous set of individuals (typically nurse managers or ED directors) than was the Seattle data. The Seattle interviews opportunistically followed whenever paths were suggested to gain information, such as IT staff and registration clerks. Another potential limitation is the different data collection methods (web-based survey vs. interviews). However, since the questions were factual as opposed to beliefs or opinions, it is unlikely that verbal responses differed significantly from the online responses. Some web survey respondents were also contacted by email or phone to clarify questions about their initial responses.
CONCLUSIONS
This comparison of workflow and information systems penetration in two very different parts of the country shows striking similarity. There is a limited amount of timely, electronic, standardized ED data available for bioterrorism surveillance at present. As more ED functions are computerized, there is an opportunity to implement standards and capture additional data for surveillance purposes.
Acknowledgments
(NC) Funding was provided by NLM training grant LM07071 and the NC Office of Public Health Preparedness and Response in the Bioterrorism Branch of the Epidemiology Section of the Division of Public Health In the NC Department of Health and Human Services. We would like to thank the NCEDD staff (2003) and Kimberly Levin and Samuel Spicer (1999) for data collection & reporting. (WA) This work was done on an implementation subcontract, through Public Health Seattle & King County (PHSKC) Washington State Department of Health, and the Centers for Disease Control and Prevention (U90/CCU017010-02). We appreciate the hard work of Marcus Grandjean for the initial (2001) Seattle data collection, and of Jeremiah Jester and Jane Sebastian (US) for web database development and 2002–3 data collection.
REFERENCES
- 1.Tang PC. AMIA advocates national health information system in fight against national health threats. J Am Med Infor Assoc. 2002;9:123–4. doi: 10.1197/jamia.M1051. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Gesteland PH, Wagner MM, Chapman WW, Espino JU, Tsui FC, Gardner RM, et al. Rapid deployment of an electronic disease surveillance system in the state of Utah for the 2002 Olympic Winter Games. Proc AMIA Symp. 2002:285–9. [PMC free article] [PubMed] [Google Scholar]
- 3.Reis BY, Mandl KD. Defining expected daily emergency department utilization rates for detection of bioterrorist attacks. Proc AMIA Symp. 2002:1137. [Google Scholar]
- 4.Ivanov O, Wagner MM, Chapman WW, Olszewski RT. Accuracy of three classifiers of acute gastrointestinal syndrome for syndromic surveillance. Proc AMIA Symp. 2002:345–9. [PMC free article] [PubMed] [Google Scholar]
- 5.Lober WB, Karras BT, Wagner MM, Overhage JM, Davidson AJ, Fraser H, Trigg LJ, Mandl KD, Espino JU, Tsui F. Roundtable on bioterrorism detection: information system-based surveillance. J Am Med Inform Assoc. 2002;9(2):105–115. doi: 10.1197/jamia.M1052. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Tsui FC, Wagner MM, Dato V, Chang CC. Value of ICD-9-coded chief complaints for detection of epidemics. Proc AMIA Symp. 2001:711–5. [PMC free article] [PubMed] [Google Scholar]
- 7.Teich JM, Wagner MM, Mackenzie CF, Schafer KO. The informatics response in disaster, terrorism, and war. J Am Med Inform Assoc. 2002;9:97–104. doi: 10.1197/jamia.M1055. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Cordell WH, Overhage JM, Waeckerle JF. Strategies for improving information management in emergency medicine to meet clinical, research, and administrative needs. Ann Emerg Med. 1998;31(2):172–7. [PubMed] [Google Scholar]
- 9.Teich JM. Information systems support for emergency medicine. Ann Emerg Med. 1998;31(3):304–7. doi: 10.1016/s0196-0644(98)70339-7. [DOI] [PubMed] [Google Scholar]
- 10.National Center for Injury Prevention and Control (US). Data elements for emergency department systems, release 1.0. Atlanta, GA: Centers for Disease Control; 1997.
- 11.Pollock DA, Adams DL, Bernardo LM, Bradley V, Brandt MD, Davis TE, et al. Data elements for emergency departments systems (DEEDS), release 1.0: A summary report. J Emerg Nursing. 1998;24:35–44. doi: 10.1016/s0099-1767(98)90168-4. [DOI] [PubMed] [Google Scholar]
- 12.DiPasquale JT, Nichols JA, Garvey JL. Chief complaint coding in the emergency department. Acad Emerg Med. 1995;2(5):442. [Google Scholar]
- 13.Grafstein E, Unger B, Bullard M, Innes G. Canadian emergency department information system (CEDIS) presenting complaint list (version 1.0) Can J Emerg Med. 2003;5:27–34. doi: 10.1017/s1481803500008071. [DOI] [PubMed] [Google Scholar]
- 14.Aronsky D, Kendall D, Merkely K, James BC, Haug PJ. A comprehensive set of coded chief complaints for the emergency department. Acad Emerg Med. 2001;8(10):980–989. doi: 10.1111/j.1553-2712.2001.tb01098.x. [DOI] [PubMed] [Google Scholar]
- 15.Fronlines of Medicine: Hierarchical Complaint Categories. Avail. at: http://frontlinesmed.org/(accessed March 12, 2003)
- 16.Garrison HG, Runyan CW, Tintinalli JE, Barber CW, Bordley WC, Hargarten SW, Pollock DA, Weiss HB. Emergency department surveillance: an examination of issues and a proposal for a national strategy. Ann Emerg Med 1994; 24: [DOI] [PubMed]
- 17.Coben JH, Dearwater SR, Garrison HG, Dixon BW. Evaluation of the emergency department logbook for population-based surveillance of firearm-related injury. Injury Prevention. 1996;28(2):188–193. doi: 10.1016/s0196-0644(96)70061-6. [DOI] [PubMed] [Google Scholar]
- 18.Cline S. Morbidity and mortality associated with Hurricane Floyd- North Carolina, September- October 1999. Morbidity and Mortality World Report. 2000;49(17):369–372. [PubMed] [Google Scholar]
- 19.Haas SW, Travers DA, Waller AE, Hilligoss B, Cahill M, Pearce PF. Defining clinical similarity among ICD-9-CM diagnosis codes: Diagnosis cluster schemes. In E. Efthimiadis (Ed.), Advances in Classification Research- Vol. 12: Proceedings of the 12th ASIST SIG/CR Classification Research Workshop. [in press].
- 20.Barthell EN, Cordell WH, Moorhead JC, Handler J, Feied C, Smith M, et al. The Frontlines of Medicine Project: A proposal for the standardized communication of emergency department data for public health uses including syndromic surveillance for biological and chemical terrorism. Ann Emerg Med. 2002;39:422–9. doi: 10.1067/mem.2002.123127. [DOI] [PubMed] [Google Scholar]
- 21.Ingalls McKay J. The emergency department of the future. J Emerg Nurs. 1999;25:480–8. doi: 10.1016/s0099-1767(99)70010-3. [DOI] [PubMed] [Google Scholar]
- 22.Davidson SJ. Errors in emergency medicine: not quite random ruminations of a curmudgeon. Acad Emerg Med. 2000 Nov;7(11):1334. doi: 10.1111/j.1553-2712.2000.tb00485.x. [DOI] [PubMed] [Google Scholar]
- 23.Travers DA, Bodenreider O. Identifying medical concepts in free text chief complaint data. Acad Emerg Med. 2002;9:511. [abstract] [Google Scholar]