Consensus Statement on Advancing Research in Emergency Department Operations and Its Impact on Patient Care

Abstract

The Consensus Conference on “Advancing Research in Emergency Department (ED) Operations and Its Impact on Patient Care,” hosted by The ED Operations Study Group (EDOSG), convened to craft a framework for future investigations in this important but underserved area. The EDOSG is a research consortium dedicated to promoting evidence based clinical practice in Emergency Medicine. The consensus process format was a modified version of the NIH Model for Consensus Conference Development. Recommendations provide an action plan for how to improve ED operations study design, create a facilitating research environment, identify data measures of value for process and outcomes research, and disseminate new knowledge in this area. Specifically, we called for eight key initiatives: 1) the development of universal measures for ED patient care processes; 2) attention to patient outcomes, in addition to process efficiency and best practice compliance; 3) the promotion of multi-site clinical operations studies to create more generalizable knowledge; 4) encouraging the use of mixed methods to understand the social community and human behavior factors that influence ED operations; 5) the creation of robust ED operations research registries to drive stronger evidence based research, 6) prioritizing key clinical questions with the input of patients, clinicians, medical leadership, emergency medicine organizations, payers, and other government stakeholders; 7) more consistently defining the functional components of the ED care system including observation units, fast tracks, waiting rooms, laboratories and radiology sub-units; and 8) maximizing multidisciplinary knowledge dissemination via emergency medicine, public health, general medicine, operations research and nontraditional publications.

INTRODUCTION

Emergency departments (EDs) in the U.S. serve as rapid diagnostic and treatment centers for patients with acute medical needs. To rapidly identify and address emergent conditions, providers in EDs compress a clinical evaluation that may take days to weeks in the outpatient setting into a few hours. Federal mandate requires EDs to conduct medical screening examinations and stabilization any person seeking emergency care without concern for payment, thus offering a public service as the healthcare safety net.^1,2 However, ED services are primarily available for the acutely ill patients. The public preference for unscheduled walk-in care, combined with a reduction in the number of EDs, is increasing the demand for individual EDs and outstripping the supply.^3,4,5 This rapid growth in demand is accompanied by a steady increase in the intensity of care provided to ED patients. This intensity of care progression likely occurs because of the increasing age of the general U.S. population and frequency of comorbid illnesses that complicate care.⁶ Together, these phenomena create the dual ED challenge of managing increasing patient volume and clinical complexity.

We are amidst a tectonic change in U.S. healthcare delivery and financing. Processes and payments are shifting from rewarding providers for the volume of patients they see and amount of care delivered to a model that rewards enhanced quality and value for each patient encounter. Additionally, the cost of care is increasingly more exposed to patients, as third-party payers seek to alter medical consumption and dampen rising expenditures. Medical value is the outcome created relative to cost and other inputs.^7,8 Both cost and resource use reduction and improved clinical processes, that enhance the real and perceived quality of care, are needed to increase value. Targets for ED improvement often focus on: 1) enhanced efficiency of care delivered in the ED, which is commonly seen as expensive, coupled with 2) altering the transitions to subsequent care. After an ED care interval, the decision to discharge, observe, or admit the patient to an in-hospital bed drives the intensity of care and resource use.⁹ Hospital-based observation or admission are often the only pragmatic – although expensive - way to guarantee the continuation of needed services. This reality drives the use of high cost resources, and is not sustainable in the face of reimbursement reform pressures to do less within the hospital. Enhanced primary care, and other urgent or unscheduled visit options, are opportunities to deliver unscheduled care in new settings. However, these are unlikely to be widely available soon or eliminate the need for ED care.¹⁰ Recent observations confirm that despite healthcare reforms, ED visits increased in select settings.¹¹ About half of hospital admissions originate in the ED,⁶ confirming EDs’ central and growing role in U.S. healthcare delivery.¹² Rather than allowing financial pressure alone to alter care, we perceive the need to enhance ED processes with evidence, so that resources and desired outcomes are effectively matched.

Operations research provides methods to understand the performance of any organized unit delivering a product or service. For our purposes, we seek to explore ED operations within health systems, their components, and the relationship between processes and the products they are designed to create. Timely patient diagnosis, treatment, and improved health outcomes are the products that EDs work to generate. A strong base of ED operations research will create a platform to improve the link between ED care and patient outcomes. The unique position of ED within health systems, sitting between the in-hospital and out-patient care environments, offers a perspective on health system successes and failures. ED operations research can improve the national approach to meeting health care demand, while exercising responsible resource utilization and patient-centered medical care. Furthermore, what is learned may guide broader health system improvements.

In March 2014, The ED Operations Study Group (EDOSG) convened a consensus conference to address the strategies needed to advance knowledge development on ED operations and the impact on patient outcomes. This research consortium seeks to facilitate investigation on how process and practice vary within and across institutions, and how this variation affects patient outcomes. In this paper, we share the recommendations from the participants at the conference. We sought to outline an action plan to foster the next steps in research development, improve the approaches in emergency care operations investigations, address the barriers and challenges to high-impact work, and begin to generate and share the knowledge needed to improve the value of ED care.

METHODS

We used a modified version of the NIH Model for Consensus Conference Development.^13,14 The ED Operations Study Group invited key experts on ED operations, operations management, clinical department management, and patient outcomes research. The conference began with a series of presentation by experts recognized for: local ED innovation and analysis, aggregating data across institutions to guide local clinical practice, and knowledge of cross-institutional operations research resources and networks. In addition, leaders from the three organizations leading the development of cross-institutional data sharing attended: The ED Operations Study Group (EDOSG), the Emergency Department Benchmarking Alliance (EDBA), and the Academic of Academic Administrators in Emergency Medicine (AAAEM). Invitees also included representatives from emergency medicine academic journals.

The meeting followed upon a landmark scientific editorial titled, “Key Requirements in a New Era of Emergency Department Operations Research.”¹⁵ The authors specified areas that needed attention to advance the science of ED operations research. These included: 1) enhanced research study designs, 2) a transition to include both outcomes and process measures and, 3) the development of networking and data-sharing across institutions. We structured our conference based upon these areas. We asked participants to assess the interval change since that publication, to identify current gaps in the literature and subsequently outline a research agenda. As listed in Table 1, four work groups addressed each topic and were facilitated by an expert work group leader. Each work group was presented with 1–3 questions to discuss and develop a list of obstacles and opportunities with correlating tasks and actions. We assigned two scribes to each group to ensure that all discussions were captured immediately and to facilitate a structured final output.

Table 1.

Work Group Questions and Task List

Work Group #1	Improved ED Operations Study Design
	Task A: What study designs are best suited to study system process and performance? Task B: What resources are needed within the community of emergency researchers to promote more generalizable studies?
Work Group #2	Environmental Research Requirements
	Task A: What data do we have and need to Understand ED care process and practice? Task B: How should we go about identifying priority disease processes or conditions for ED operations research and care process improvements.
Work Group #3	Measures of Value: Process and Outcomes Task A: What should a comparative operations data set include to help guide evidence based clinical practice and promote consistent data reporting? Task B: What data do we need to know/track to understand and best serve our specialty sub-populations? (trauma, pediatrics, psychiatry, geriatrics, etc.)
Work Group #4	Knowledge Dissemination
	Task A: Where can Emergency Medicine Operations Research be published Task B: Should an EM Operations Research Journal be started? Task C: Which other disciplines have overlapping or complimentary knowledge sets that can help advance EM operations research

Prior to the meeting date, the EDOSG pre-meeting research team created: 1) a summary of currently available national databases with ED operations data, 2) a review of all ED operations manuscript from January 2009– February 2014, and 3) a literature review summarizing varied approaches to identifying target disease processes for operations research and its impact on patient outcomes.

RECOMMENDATIONS

Improving ED Operations Study Design

1. Common Terminology

As EDs work to improve the efficiency and quality of care delivered to patients, we need a standardized language and classification terminology to compare and contrast ED, hospitals, and data sets. We need a taxonomy standardizing chief complaints to allow the development of symptom-driven data. This shifts our focus towards a patient-centered perspective. Currently, most research on ED operations uses the end of visit physician diagnosis.¹⁶ For example, the commonly used NYU Algorithm for Operational Decisions and the National Emergency Department Survey (NEDS) can only sort the clinical description of patient presentations by the end of visit diagnosis.^17,18,19 A symptom-based approach, that uses presenting chief complaints (i.e. - chest pain, shortness of breath, fever, headache) may be relevant to ED studies, as patients do not come to an ED in with a final diagnosis. In many cases, the final diagnosis may not reflect the complexity of the ED encounter. Improving the efficiency and process of reaching that diagnosis is what drives clinical operations research in EM.

Another need is for consistent definitions for event time-stamps. This includes but is not limited to: arrival, time of care initiation, timing-to-testing measures, time to treatment measures, time of disposition decision for a patient, and achievement of the planned disposition destination (home, transfer, observation or admission). Organizations like the EDOSG, the EDBA, the Emergency Nurses Association (ENA), The Joint Commission and Center for Medicare and Medicaid Services (CMS) have existing time stamp definitions. However, there is marked variability in the definition and measurement of these important process metrics. Given that these measures are becoming increasing reportable, consistency within the practice of emergency medicine is essential.

2. Robust Outcomes Measures

The importance of studying operational processes from an administrative perspective is established. However, the clinical relevance of this work is often lacking. In addition, using clinical outcomes as the primary outcome measure in the ED setting is challenging. The relatively short interval of ED care makes it difficult to determine what effect ED care has had on an individual or groups overall health care course, immediately or long term. For lower acuity diseases (e.g., upper respiratory tract infection), relevant clinical outcomes may not be defined. In addition, clinical outcomes may not be known for a long period of time, limiting their utility to the improvement processes. This explains the use of process measures as outcomes, which are more easily measured, but their correlation with clinical outcomes may be low.²⁰ Furthermore, process measures may not provide comprehensive information about the setting they describe. For example, ED length of stay (LOS) is a common surrogate outcome for ED operations analyses tied to quality of care and patient satisfaction. However, it is not sufficient to understand optimal resource use, process implementation or how to enhance patient outcomes.

One contributing factor is that methods for measuring clinical outcomes depend on the study goals. Whether the patients, providers, payers or hospital administrators are the focus may influence which outcome measures are most important. Outcome measures can include measures of: patient experience, short-term mortality after ED discharge, medical errors and adverse events, provider or ED diagnostic accuracy, longer-term mortality or morbidity for time-critical illnesses (including the recurrent need for unscheduled care). While a patient focus will always be a key in defining outcomes, impactful ED operations research should consider multiple stakeholder viewpoints.

3. Multi-Site Studies

A limitation of many ED operations research studies and data sets is that they are either single site/system studies or analyses of aggregated data extrapolated from limited ED samples. An example of the latter is the Hospital Compare database, maintained by CMS,²¹ where many hospitals have too small of a sample to create reliable performance comparisons. More granular and consistently measured data are needed to perform quality research that can inform general practice and account for variability. In the era of electronic health records, this is achievable. However, the networking of diverse information system platforms is a challenge. Real-time data reporting and cross-institutional data sharing are needed to develop databases robust enough to inform clinical care and operations improvement. Developing these systems requires both government and ED information system vendor participation.

4. The Value of Qualitative Methods

Quantitative methods are the standard for clinical research. However, qualitative methods add value by helping us understand the complex social processes driving the demand and utilization of ED services. They provide techniques to understand how the culture within an institution drives variation in ED operation, flow, the quality of care and consistency of practice. Qualitative scientific methods can help explore the social, community and organizational context of healthcare delivery that influence patient engagement, organizational performance, and quality of care delivered at the patient and population level. Given the participant complexity of healthcare, qualitative methods can direct the structure and focus of quantitative research. We recommend increased use of mixed-methods research, which combines qualitative and quantitative data to answer research questions. Within emergency care, mixed methods research has been used to improve the care of patients with ST elevation myocardial infarction (STEMI).²² Further use to elucidate the human behavioral, cultural and social forces driving care efficiency, inefficiency and degree of deviation from best practice is an option.

Data and Environmental Requirements

1. Collecting Data to Drive Impactful Research

There are two types of data needed to study ED clinical care delivery and the relationship to patient outcomes. The first are standardized operations measures - such as arrival time, door-to-doctor time and lab/radiology turn-around-times - to allow process comparisons among different facilities. The second is detailed clinical practice and care episode data for particular disease processes or symptoms. Examples include: the time interval used between serum troponin assay tests to identify myocardial infarction; or the type of testing used to screen for myocardial ischemia (coronary computer tomography angiogram vs. myocardial perfusion imaging vs exercise treadmill testing). Current datasets are often limited in their ability to deliver this needed detail. We list those currently available to operations researchers in Table 2. Few can enable meaningful comparative analyses of clinical operations and clinical practice. Most lack the granularity needed to perform ED operations research to accurately drive clinical practice or systems change. All aggregate measures of performance and practice, typically for a year. This prevents the study of variation in ED practice and process at different times of the day or year or real-time. Reporting of measures of central tendency is easier to collect and database, but this alone cannot inform the study of practice and process at the level of clinical patient care. For clinical research, patient visit data is a requirement. NEDS provides granular data, using visit level billing and coding data. However, it is a limited sample of select EDs and is without process time stamps or detailed testing information. Furthermore, these datasets do not provide enough longitudinal patient-level information, and outcomes such as patient morbidity (other than ones that require hospitalization).

Table 2.

Existing Emergency Department Operations Data Sets

Data Set	Dates	EDs Represented	Availability
Academy of Administrations in Academic Emergency Medicine (AAAEM)25	2009–2013	Academic ED: 85 adult and 20 pediatric	to members only
Emergency Department Benchmarking Alliance (EDBA) database26	1994–2013	1026 academic, community and pediatric	to members only
Emergency Department Operations Study Group (EDOSG)27	2011–2013	26 academically affiliated ED	to members only
ED Benchmarks Collaborative (EDBC)28,29	36 month of data for single site	Undisclosed	for an undisclosed fee
Emergency Excellence (EmEx)30	undisclosed	300	for a fee of $4500
Emergency Department Practice Management Association (EDPMA)31	undisclosed	Undisclosed	for a fee of $1050–$10,500
Hospital Compare32	2009–2013	4,500	free online
The 2011 National ED Inventory-USA (NEDI-USA)33	1992–2013	226	for a fee, per request
The Nationwide Emergency Department Sample (NEDS)34	2006–2011	350	$500 per year
The National Hospital Ambulatory Medical Care Survey (NHAMCS)XXV	1973–1981, 1985, 1989–2010	All U.S. ED, extrapolated from a sample of ~500 EDs	Free (additional variables available for $750/day access)

In designing data sets for operations and practice evaluation, clear questions must be asked. This applies even if the goal is to detect potential areas of research, associations that might define a line of research, or address an intervention to create a cause and effect relationship (or the absence of such) after an intervention. To achieve this we recommend recruiting a group(s) of investigators – drawn broadly from emergency care, hospital care, public heath, human factors engineering, economics experts and patient representatives - to complete four key tasks:

1. Develop a list of potential ED operations questions/topics;

2. Assess what is known for each;

3. Identify the gaps in the question and knowledge;

4. Prioritize the list of research questions and associated research data.

2. Prioritizing our Questions

We posit the importance of including the perspectives of all key emergency medical care and process stakeholders as we further prioritize our greatest need within ED operations research. We have chosen the following four topics that capture the concerns of patients, providers, payers, clinical researchers and ED managers. These include:

• Patient-oriented concerns

• Demand-oriented concerns

• Critical condition and mortality-oriented concerns

• Medico-legal concerns

The ideal processes include an iterative structure and consensus decision-making structure to allow best judgment and a clear agenda to be developed. Key stakeholders in this process could include, but are not limited to: The ED Operations Study Group, The ED Benchmarking Alliance, national and local emergency medicine organizations, private payers and other health care organizations, patient representatives and the Center for Medicare and Medicaid Services (CMS).

3. Research Network Development

Several research networks exist to study specific diseases within the ED care interval. For instance, EMNet (http://www.emnet-usa.org/) is a collaborative of 226 medical centers focused primarily on acute respiratory emergencies. The Pediatric Emergency Care Applied Research Network (PECARN, https://www.pecarn.org/), focuses on pediatric acute illnesses and creating new guidelines for the management of pediatric head injuries and abdominal blunt injuries to date). More recently, the Western Emergency Services Translational Research Network (WESTRN), formed to explore broader research topics (https://www.ctsacentral.org/regional-consortia/western-emergency-services-translational-research-network). This is a group of 11 academic medical centers in the Western United States with National Institute of Health Clinical Translational Science Institutes (CTSIs). They have collaborated and performed projects related with traumatic injury in adults. A similar approach for ED operations studies can facilitate data collection and comparisons across a variety of settings that would transcend existing approaches to operational studies.

Identifying Measures of Value

1. Data to Collect

The EDBA aggregated basic ED operations and process metrics for cross institutional comparison for 20 years to guide administrative decision-making. Their existing data collection offers a starting point for broader data collection.²³ In 2011, the EDOSG introduced the concept of clinical practice comparisons across EDs for cardiovascular disease diagnostic testing pathways. A dataset that draws on the strengths of these two datasets (process metrics and clinical practice metrics), with appropriate representation of the diversity of EDs, is an early step. The next logical step is to obtain patient-level data. This approach maximizes flexibility for analysis because analyses can be customized to the question. For example, seasonal variability and patient sub-populations (e.g., trauma, pediatrics, psychiatry, and geriatrics) can be studied from the same datasets.

2. Defining Common ED Functional Components

Few EDs are exactly the same, though most have similar functional components. These include, but are not limited to: observation units, trauma units, psychiatric EDs, pediatric EDs, fast-tracks, split-flow or mid-tracks, in-ED lab kiosks, and emergency radiology teams. One challenge in aggregating data across EDs is identifying and modeling these components. Defining these provides a better and stable description of the environment and mechanics through which emergency care is provided.

Knowledge Dissemination

1. Publishing Operations Research within Emergency Medicine

While operations research exists in other industries for decades, it is fairly nascent in healthcare, notably in emergency care. With the growing emphasis on system performance, operations research is key to evaluate novel strategies, improve efficiency, and make the healthcare system more responsive to patient needs. Peer-reviewed research publications are relatively few, compared to traditional disease specific studies, but they are growing. Our pre-meeting analysis found that a total of 504 published ED operations research manuscripts between January 2009 and February 2014. Of these, 45% were published in emergency medicine journals. (See Figure 1 and Table 3.) Once ED operations research is more mature, broader dissemination within EM and general medical journals is likely given the importance of emergency care to the entire health care system. A high quality emergency care research framework can inform broader policy and population health needs.

Figure 1.

Distribution of ED Operations Research Manuscript Published by Discipline (January 2009–February 2014)

Table 3.

Top 8 Journals Publishing ED Operations Literature from January 2009 to February 2014

Journal	Percentage of Total Manuscripts
Academic Emergency Medicine	24%
Emergency Medicine Journal	21%
Annals of Emergency Medicine	15%
Pediatric Emergency Care	6%
Journal of Emergency Medicine	5%
Journal of Healthcare Management	5%
Journal of Emergency Nursing	4%
American Journal of Emergency Medicine	4%

2. Publishing Outside of Medical Peer Review Literature

We found that the scientific methodology used in the reported studies correlated with the discipline of the publishing journal. For example, if a mathematical computer model was used to predict ED patient input and throughput, this was most likely to be published in an engineering or informatics journal. Given that the application of operations research to the emergency medicine requires the methodologies of many other disciplines, matching methodology to publication creates an opportunity for interdisciplinary knowledge sharing and development. In addition, cross-publication and review by news services such as Emergency Medicine News, ACEP Emergency Medicine Today and other emergency medicine communications can facilitate the promotion of such work to enhance awareness. Public sources such as US Today, the Wall Street Journal, the New Yorker and internet blogs are methods engage governmental officials and the public, the ultimate consumer of our services. This can augment familiarity with our often silent but necessary role in the public health and health delivery system.

Other strategies for dissemination include public interaction and collaboration with groups focused on system delivery (e.g., Institute for Healthcare Improvement). Conferences allow knowledge dissemination and offer opportunity to network, brainstorm and share ideas across varied groups with expertise. Meetings such as INFORMS Healthcare Conference, the College of Healthcare Operations at the Production and Operations Management Society (POMS), and the Mayo Clinic Conference on Systems Engineering and Operations Research in Healthcare are examples of healthcare operations conferences that encourage inter-disciplinary knowledge dissemination. While the focus is not solely on emergency care, such work is frequently presented and should become a more visible part moving forward.

OBSTACLES TO ADDRESS

With more than 130 million ED visits across 4,500 hospitals in the U.S. annually,²⁴ there is the opportunity and need to study ED operations. The challenge is to do so in a way that improves the health of the public and the individual patient. We have highlighted many opportunities. As we progress, key obstacles that will need to be addressed include the following:

1. Defining an episode of emergency care

   Is this just the ED visit, or does it also include care/illness before and after? How long does it last and with what constraints? How can we access other key data? The electronic medical record, linked with payer activity, holds promise but is highly variable across systems and regions. Existing datasets are often hindered by sampling and abstraction methods that challenge the data granularity or accuracy.

2. Identifying who will and should fund the initial efforts

   Naturally, government and private health care payers are potential resources, but each has constraints in the current economic climate. Traditional research funding strategies seeking foundation, industry, and state or federal sources (e.g., NIH, CDC, PCORI or AHRQ) are other options but have not traditionally had interest in this funding this area of research.

3. Managing data collection and analysis effort

   The options involve outsourcing to those experienced in large data analysis (though likely unfamiliar with acute care issues) versus creating a unified and new team that can drive the analyses. Each has pragmatic benefits – ease, cost, durability, and likelihood of success. The amount and size of data needs depends on the question. Though discrete and common questions may not require national or multi-region data, all will likely need more than what one site or system can provide. Aligning these data needs to create a usable dataset is highly dependent on electronic medical record data sharing across institutions.

4. Facilitating cross-institutional data-sharing

   There are inter-institutional challenges to data sharing. These include obtaining institutional review board approval for large volume site and patient data, and the variable use of data definitions. Unique approaches and joint-review process – akin to that used by many research networks – are required for this to occur. This is particularly important with regard to protected health information (PHI) and local integrity concerns. Simplified datasets will minimize the complexity of various definitions. Blinding to protected health information (PHI) and institution identifiers can facilitate data exchange. Informatics advances, like RedCap and improved data extraction from commonly used ED information systems, can ease the collection and submission of data.

5. Balancing ease versus impact

   Successful research is both feasible and generalizable. Examining both data adequacy and the likelihood of aggregating a sufficient sample are important factors in choosing a project. Data that exist (e.g., administrative data with limited or inconstant clinical information or links) are tempting targets, but they may be limited in their ability to address a relevant research questions. Gaps in the type of data available need to be identified, rather than simply doing what we can with what exists. Data needs to support studies that are generalizable, such that others can expect the same or similar observations and outcomes? If not, the reasons why need to be addressed? Experts in public health and emergency care would be best poised to address this issue.

CONCLUSION

We identified the current status, key opportunities, and likely barriers in advancing the science behind ED operations and the impact on patient outcomes. These recommendations should serve as a template for future cross-disciplinary efforts and governmental support to help deliver high-value, quality acute health care.