Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Jan 1.
Published in final edited form as: Ann Emerg Med. 2022 Oct 15;81(1):47–56. doi: 10.1016/j.annemergmed.2022.08.010

A Quality Framework to Address Racial and Ethnic Disparities in Emergency Department Care

Hazar Khidir 1,2, Rama Salhi 3,4, Amber K Sabbatini 5,6, Nicole Franks 7, Andrea Green 8, Lynne D Richardson 9,10, Aisha Terry 11,12, Nicholas Vasquez 13,14, Pawan Goyal 15, Keith Kocher 4, Arjun K Venkatesh 1,2,16, Michelle P Lin 9,10
PMCID: PMC9780164  NIHMSID: NIHMS1829280  PMID: 36257864

Abstract

The emergency department (ED) serves as a vital source of healthcare for residents in the United States, including as a safety net. However, patients from minoritized racial and ethnic groups have historically experienced disproportionate barriers to accessing healthcare services and lower quality of services compared to White patients. Quality measures and their application to quality improvement initiatives represent a critical opportunity to incentivize healthcare systems to advance health equity and reduce health disparities. Currently, there are no nationally recognized quality measures that track the quality of emergency care delivery by race and ethnicity and no published frameworks to guide the development and prioritization of quality measures to reduce health disparities in emergency care. To address these gaps, American College of Emergency Physicians (ACEP) convened a working group of experts in quality measurement, health disparities, and health equity to develop guidance on the establishment of quality measures to address racial and ethnic disparities in the provision of emergency care. Based on iterative discussion over three working group meetings, we present a summary of existing emergency medicine quality measures that should be adapted to track racial and ethnic disparities as well as a framework for developing new measures that focus on disparities in access to emergency care, care delivery, and transitions of care.

Background

Patients from minoritized racial and ethnic groups have less access to healthcare services and receive lower quality of care compared to White patients.1,2 These differences in care provision contribute to relatively worse health outcomes among minoritized racial and ethnic groups, such as threefold higher maternal mortality amongst Black women compared to White women and twofold higher rates of diabetes-related complications amongst Hispanic patients compared to non-Hispanic White patients.35 The Institute of Medicine’s (IOM) seminal publication Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care defined these differential health outcomes as healthcare disparities or ‘racial or ethnic differences in the quality and provision of healthcare that are not necessarily due to access or clinical needs, preferences, and appropriateness of intervention’.6 The IOM identified disparities as such an important reflection of quality that it named equity as one of the six key pillars of the IOM framework of healthcare quality in 2001.7 Despite this, compared to the other IOM domains of safety, effectiveness, efficiency, timeliness, and patient-centeredness, where significant improvements have been made, less progress has been made in improving healthcare equity.7

The emergency department (ED) serves as a vital source of healthcare for residents in the United States. As a common resource for communities, the ED provides a safety net to all patients, serving a critical public health role. Patients from racial and ethnic minority groups disproportionately use the ED and are more likely to rely on emergency care for both time-sensitive and non-urgent care needs.8 This reality leaves patients from racial and ethnic minority groups vulnerable to existing systemic barriers in access, delivery, and transitions of emergency care. As a result, care decisions in the ED, such as those regarding the initiation of diagnostic and therapeutic services and admission for inpatient monitoring or referral for outpatient care, have significant downstream implications, including limitations in access to inpatient hospitalization, specialist care, and urgent procedural intervention.

Despite the ED’s important role in providing access to healthcare for racial and ethnic minority groups, disparities in emergency care, as in other areas of medicine, are well documented.914 Moreover, although there is no paucity of data documenting disparities in emergency care and increasing calls for action to advance equity and espouse antiracist principles within the practice of emergency medicine,15 there has been little guidance on strategies that health systems and clinicians can employ to address disparities in the emergency care setting.

Quality measures and their application to quality improvement initiatives and value-based payment strategies represent a critical opportunity to incentivize healthcare clinicians and systems to advance health equity. Measuring specific elements of care that are thought to reflect quality can be used to more standardly benchmark progress made through quality improvement initiatives.16 Over the past two decades, adoption of quality measurement has resulted in greater integration of evidenced-based practices, better standardization of care for patients presenting with similar clinical conditions, reduced inefficient resource utilization, and improved health outcomes.17,18

The specialty of emergency medicine has promoted initiatives to develop and implement quality measures nationally for the purpose of quality improvement and value-based payment. For example, in 2021 the Clinical Emergency Data Registry (CEDR) was established as the first emergency medicine qualified clinical data registry to measure acute care quality.19,20 Additionally, programs like the Michigan ED Improvement Collaborative (MEDIC) and the Emergency Quality Network (E-QUAL) were created to collaboratively improve ED care quality.21,22 These and other efforts have led to meaningful improvements in patient outcomes for conditions such as stroke and myocardial infarction23,24 and reductions in unnecessary imaging for minor head trauma and syncope.25

However, the impact of quality measurement and quality improvement on health equity is mixed. In some cases, value-based programs have widened disparities,26,27 in part because the resource-intensive demands of reporting and carrying out quality improvement strategies may divert resources from high-need patients. Therefore, there is increasing recognition that equity and disparities must be explicitly measured and reported. The Centers for Medicare and Medicaid Services (CMS) has announced new measures that will hold clinicians accountable for racial disparities in hospital readmissions and will now require all new models to collect demographic data on healthcare beneficiaries to track whether quality improvement strategies are reducing or exacerbating disparities.28,29 Current ED quality measures, however, do not report performance or outcomes by race or ethnicity, which may mask disparities and limits quality improvement within the domain of equity. In addition, there are no published frameworks to guide the development and prioritization of quality measures to reduce health disparities in emergency care.

To address these gaps, ACEP convened a workgroup of experts in quality measurement and health disparities to develop guidance on the establishment of quality measures to address racial and ethnic disparities in emergency care. Specifically, the objectives of the workgroup were to: (1) identify key principles for applying quality measurement to health disparities (2) evaluate how to adapt existing emergency medicine quality measures to track racial and ethnic disparities in emergency care and (3) develop a framework to guide development of novel measures to address health disparities where substantial gaps exist. We present expert consensus findings related to these objectives.

Workgroup Membership and Process

The ACEP Quality Measurement Development Workgroup was comprised of thirteen practicing emergency physicians who were invited to participate based on their expertise in emergency medicine health disparities; social emergency medicine; emergency medicine quality measurement and/or quality improvement. Workgroup members were past or current leaders of several committees and sections within ACEP, including the Diversity Inclusion & Health Equity Section, Social Emergency Medicine Section, Quality and Patient Safety Committee, and the CEDR Committee.

The workgroup convened over two virtual one-hour sessions and one final in-person two-hour session that occurred in the fall of 2021. Prior to the first meeting, workgroup leaders conducted and reviewed literature searches on existing approaches to disparities quality measurement. During the first meeting, the workgroup reviewed summarized findings on existing approaches and key principles to disparities quality measurement (Objective 1). The workgroup agreed on a strategy to score existing emergency medicine quality measures for their potential to track racial and ethnic disparities (Objective 2) and discussed a preliminary framework for establishing quality measures to reduce health disparities based on theoretical models identified in the literature. During the second meeting, the workgroup refined the proposed framework for developing novel measures to address health disparities in emergency medicine (Objective 3). During the final meeting, members had the opportunity to vote on the prioritization of existing emergency medicine quality measures which demonstrated potential to track racial and ethnic disparities. The workgroup also made final revisions to the framework and collaboratively developed new measure concepts related to key elements of the framework. All disagreements were resolved through group discussion.

Objective 1: Identifying key principles for applying quality measurement to health disparities

We reviewed the landscape of evidence and existing approaches to measuring health disparities across medicine broadly. Overall, we found few existing guidelines3033 and established demonstrations28,34,35 incorporating quality measures related to healthcare disparities, with little mention of emergency medicine. Nonetheless, we presented a summary of ten existing demonstrations outside the field emergency medicine from prior published reports34,36,37 as examples of how health care organizations have approached disparities quality measurement to inform initial workgroup discussions.

Disparities Quality Measures Should Focus on the Emergency Clinician Practice Level

A critical consideration of quality measure development is measure attribution, or the process of selecting a patient population for which a group or entity will be held accountable for providing appropriate health services and achieving adequate health outcomes.38 The population may be defined as the patients who receive care within a particular geographic area, health system, or clinician group. Most existing health disparities quality measurement approaches measure health disparities at the broader geographic level (i.e. state-level outcomes) or on the level of the health plan.34 The reported measure outcomes are thus a reflection of how the entire entity (state, health system) is performing rather than a reflection of the performance of individual clinicians.

Attributing accountability for disparities in care suggests that the group whose performance is being measured has the agency to achieve measurable improvements in those disparities. For health disparities measures related to emergency care, our workgroup focused on quality measurement at the emergency clinician group level, in alignment with existing quality measure sets for emergency care that ACEP supports.19,39 By focusing initially on this level, interventions can be developed, applied, and then measured for change and improvement among this group over time. Group-level measures also mitigate low sample sizes that may not translate into meaningful differences in quality of care at the individual clinician level. However, we acknowledge that measurement of process changes and feedback can be applied more precisely when targeting at a narrower level, such as the individual clinician level. At the same time, we also recognize that health disparities are driven at a broader level, by attributed entities much farther upstream to emergency care, including public health and social service systems.

Disparities Quality Measurement Should Start with Process Measures

Measuring health disparities at the clinician group level has implications for the types of measures that are feasible and valid to implement to improve equity. Quality measures may be designed as structure, process or outcome measures.40 Structural measures assess the capacity to provide quality healthcare. Process measures assess the care delivery activities and resources used to care for patients, and ideally reflect evidenced-based care that is known to be associated with good outcomes. Outcome measures directly measure improvements in health (i.e. mortality, functional status, pain) that result from health care delivery.

In identifying concepts for new measure development, processes of care are actionable targets that are particularly vulnerable to mechanisms of health inequity such as institutional and personally-mediated racism.41,42 At the clinician level, personally mediated racism can operate through implicit and explicit biases that are held by the clinician and influence the quality of patient-clinician communication43 as well as the clinician’s medical decision-making.44 Care delivery in the ED is characterized by high-acuity, time-sensitive conditions, high patient volumes, competing demands and rapid decision-making with incomplete diagnostic information and no pre-existing patient-provider relationship. These features of ED practice yield cognitive stressors that can exacerbate clinician’s use of heuristics such as implicit and explicit biases that can affect processes of care including diagnostic and treatment decisions. Thus, special attention should be paid to processes of care, such as diagnostic, treatment, and disposition decisions, where known disparities can widen.

While, ideally, quality improvement efforts would hold clinicians accountable for improvements in outcomes, valid outcomes measures are often difficult to develop and implement.45,46 This is especially challenging for measures that attempt to capture racial and ethnic disparities in health outcomes as they are influenced by numerous social determinants of health and health system failures outside of the control of the ED clinician. For example, stroke mortality may be a useful measure to assess the quality of care provided by an institution or health plan but would not necessarily be representative of emergency clinician group quality as it reflects not only the care delivered in the ED but also upstream elements related to medications and preventative services like carotid endarterectomy as well as downstream care within the inpatient or post-acute care settings.47 However, a process measure related to the equitable use of computed tomography and provision of tissue plasminogen activator therapy across patient racial and ethnic groups may more specifically target disparities in emergency care quality at the clinician group level.

Disparities measurement should be integrated into existing quality measurement practices.

Efforts should be made to integrate health disparities quality measurement within the existing universe of quality measures. Within the field of emergency medicine, specifically, quality measurement has been applied to multiple quality domains, specifically to improve the timeliness of sepsis, myocardial infarction, and stroke care; the efficiency of imaging use for uncomplicated low back pain; the safety of foley catheter utilization; the effective use of antibiotics for bacterial sinusitis; and to deliver more patient-centered care through faster patient throughput in the ED.19 Isolating disparities measures from other commonly used measures would undermine the goal of integrating equity as an equally important pillar of quality. One strategy to integrate disparities quality measurement within quality improvement initiatives is to stratify existing measures, reporting separate outcomes for each racial and ethnic group. Comparisons of outcomes for patients of minoritized racial and ethnic groups to those of White patients could then be used to assess if gaps in measure outcomes exist. These gaps in outcomes could then be tracked and linked to performance incentives.

Existing measures should be appraised for their potential to detect racial and ethnic disparities and track changes in these disparities over time. In 2012, the NQF published the Disparities Sensitive Measurement Assessment scoring algorithm, which was applied to existing quality measures within ambulatory care.32 The aim of the scoring algorithm is to identify whether an existing quality measure is disparities-sensitive, meaning that it demonstrates potential to detect differences in quality across racial and ethnic groups.

The NQF Disparities Sensitive Measurement scoring consists of three primary criteria: (1) the prevalence of the condition targeted by the measure based on published priority conditions known to disproportionately impact racial and ethnic minority groups; (2) the health impact of the measurement domain or disease on racial and ethnic minority populations, including mortality, quality of life, disability; and (3) the magnitude of the gap in quality between the racial and ethnic minority strata and historically advantaged population. Secondary criteria include consideration of whether the measure domain could be impacted by clinician-patient communication practices. At the time of expert consensus meetings, the NQF Disparities Sensitive Measurement Assessment was the most established strategy for appraising existing quality measures for their potential to track health disparities.

Objective 2: Adapting existing emergency medicine quality measures to track racial and ethnic disparities in emergency care

We applied the NQF Disparities Sensitive Measurement Assessment to 39 existing emergency medicine quality measures supported by CEDR, in addition to non-emergency medicine Quality Payment Program(QPP) measures commonly reported by emergency physicians.19,39 Our aim was to prioritize existing measures to begin stratifying by race and ethnicity and examine disparities in care within and between clinician groups.

There were notable limitations to the NQF Disparities-Sensitive Scoring protocol. Firstly, assessments of impact and quality gap on racial and ethnic minority populations as described above relied on availability of published literature. In some cases, evidenced-based appraisements of prevalence and impact could not be made due to lack of available literature. In addition, some existing emergency medicine quality measures targeted disease conditions such as sepsis—that are particularly significant to the scope of emergency medicine and for which there is literature supporting high prevalence in racial and ethnic minority groups—were not included in the list of priority conditions specified by the algorithm.32 These measures would have scored low for ‘prevalence’ based on NQF guidelines.

After initial review, we identified 13 existing quality measures that were highly disparities-sensitive based on NQF scoring criteria. We also identified 18 measures for which there was limited literature to facilitate accurate scoring. Choosing among the 13 highly disparities-sensitive measures, the workgroup was then asked to select those which should be the highest priority for tracking racial and ethnic disparities. Each workgroup member who attended the final meeting was asked to cast five votes among the measures to identify those with the highest potential to detect racial and ethnic quality gaps. The number of votes for each measure were tallied, and, at the conclusion of workgroup discussions, the 13 highly disparities-sensitive quality measures were ranked by priority based on highest to lowest number of votes (Table 1).

Table 1.

Existing Disparities-Sensitive Emergency Medicine Quality Measures

Measure IDa Measure Title Emergency Medicine Health Disparities Framework Domain Measure Typeb Priority Rank* NQFc Score**
ACEP 56 Follow-Up Care Coordination Documented in Discharge Summary Transitions Process 1 5
ECPR 52 Percentage of Adult Patients with Psychosis or Agitation Who Were Ordered an Oral Antipsychotic Medication in the ED Care Delivery Process 2 9
ACEP 48 Sepsis Management: Septic Shock: Lactate Level Measurement, Antibiotics Ordered, and Fluid Resuscitation Care Delivery Process 3 9
MIPS 87 Stroke & Stroke Rehabilitation: Thrombolytic Therapy for Ischemic CVA Care Delivery Process 4 11
ECPR 51 Percentage of Opioid Poisoning or Overdose Patients Presenting to An Acute Care Facility Who Were Prescribed Naloxone at Discharge Transitions Process 5 10
ECPR 66 Opioid Withdrawal: Initiation of Medication-Assisted Treatment (MAT) and Referral to Outpatient Opioid Treatment Transitions Process 6 10
ACEP 19 Percentage of ED visits for patients aged 18 years and older who presented with a minor blunt head trauma who had a head CT for trauma ordered by an emergency care clinician who have an indication for a head CT Care Delivery Process 7 10
MIPS 317 Preventive Care and Screening: Screening for High Blood Pressure and Follow-Up Documented Transitions Process 8 9
ACEP 22 Appropriate ED Utilization of CT for Pulmonary Embolism Care Delivery Process 9 10
ECPR 40 Initiation of Initial Sepsis Bundle Care Delivery Process 10 9
ACEP 30 Sepsis Management: Septic Shock: Lactate Clearance Rate ≥ 10% Care Delivery Outcome 11 9
ACEP 25 Tobacco Use: Screening and Cessation Intervention for Patients with Asthma and COPD Transitions Process 12 9
ACEP 55 Emergency Medicine: ED Utilization of CT for Minor Blunt Head Trauma for Patients Aged 2 Through 17 Years Care Delivery Process 13 9

ACEP=American College of Emergency Physicians; MIPS= Merit-based Incentive Payment System; ECPR= Emergency-Clinical Performance Registry; NQF= National Quality Forum

*

In descending order of priority for testing and implementation; based on consensus of the ACEP Quality Measurement Development Workgroup

**

NQF score range is 2–14. A score of 9 points or higher met threshold for disparities-sensitivity. Additionally, measures could be deemed disparities-sensitive based on the magnitude of the quality gap between the historically disadvantaged group and the group with the highest quality for that measure.

a

American College of Emergency Physicians. Clinical Emergency Data Registry. 2022. https://www.acep.org/cedr/

b

Donabedian A. The Quality of Care: How Can It Be Assessed? JAMA. 1988;260(12):1743. doi:10.1001/jama.1988.03410120089033

c

National Quality Forum. Healthcare Disparities and Cultural Competency Consensus Standards: Disparities-sensitive Measure Assessment. 2012. Accessed October 2021: https://www.qualityforum.org/Publications/2012/11/Healthcare Disparities and Cultural Competency Consensus Standards__Disparities-Sensitive_Measure_Assessment.aspx

Overall, examining existing emergency quality measures revealed important insights. Most measures were process measures, few were outcome measures. The predominance of process measures was expected based on lessons from published demonstrations that illustrated processes of care as particularly useful targets for health disparities measurement. Processes of care are subject to greater clinician discretion which can be impacted by heuristics engendered within the ED setting.43 In addition, processes of care are generally within the ED’s jurisdiction, creating actionable opportunities for quality improvement and ensuring appropriate measure attribution.

We observed that measures related to the IOM quality pillar of efficiency were less likely to be disparities-sensitive. These efficiency measures typically aimed to increase value by targeting overuse of healthcare resources (e.g., ACEP 58: Measure of antibiotic prescribing for adults diagnosed with an upper respiratory infection).19 Notably, in some cases, the quality gap between the racial and ethnic minority groups and the historically advantaged group (White patients) was inverse (historically advantaged group had lower quality of care compared to racial and ethnic minority groups). This likely reflects patterns that demonstrate that racial and ethnic minority patients are less likely to receive pharmacologic treatments relative to historically advantaged minority groups. This suggests that closer appraisements of the quality gap should be conducted when considering the utility of efficiency measures for disparities quality measurement. Studies should examine whether overuse measures that try to improve value for historically advantaged groups could potentially result in significantly reduced access for racial and ethnic minority groups.

Lastly, though some existing measures will capture incidence of racially disparate care, stratifying existing measures by race is not sufficient to make substantial progress in reducing gaps in healthcare quality. Existing measures did not cover the spectrum of clinical conditions or care processes that house health care disparities. Thus, we sought to identify new measures that specifically targeted domains of emergency care that are vulnerable to mechanisms of institutional and personally-mediated racism and where racial and ethnic disparities are likely to arise.

Objective 3: Developing a framework to guide development of novel measures to address health disparities in emergency department care

To inform our understanding of strengths and weakness of current measures, as well as future measure development, our workgroup developed a conceptual framework drawing on existing literature.32,34,48 A recently published four-tier model,48 for example, provides broad guidance on developing measures for the advancement of equity for both patients and members of healthcare organizations. This framework importantly both categorizes and prioritizes the development of measures for the advancement of equity into four categories: access, transitions, quality of care, and socioeconomic/environmental impact. However, such models are limited in their application to the specific nuances of emergency medicine, underscoring the need for a revised framework.

Here we build upon this existing model for application specifically within emergency medicine. Initial literature review, thematic categorization, and framework drafting was completed by two study group members (HK, RS) with input from the workgroup chair (MPL) prior to the first workgroup meeting. The drafted framework was presented for discussion and feedback iteratively over the course of workgroup meetings. After each meeting, the model was revised and re-presented for further feedback.

The final framework, completed after the third stage of feedback and revisions during the in-person meeting, is presented in Figure 1. Interactions and themes are framed around the patient experience, beginning at the time of the recognition of need to seek emergency care. The final framework can be broadly categorized into: Access to care – including barriers or facilitators that lead one to present to the ED; Care delivery – the provision of care following presentation to the ED; and Transitions – the follow up care and support services that are initiated at the end of an ED visit. Throughout the care continuum, structural racism and bias are identified to highlight the ways in which they mediate and moderate each of the categories. The framework also acknowledges pervasive social determinants of health that influence one’s likelihood of seeking ED care and influence health outcomes during and after the acute care encounter.

Figure 1. Framework to guide development of novel measures to address racial and ethnic disparities in access to emergency department care.*.

Figure 1.

PT=Physical Therapy; OT=Occupational Therapy; SNF=Skilled Nursing Facility; LTCF=Long-term Care Facility

* This framework centers around patients experience receiving emergency care, starting with Access to care – including barriers or facilitators that lead one to present to the ED, as well as prehospital transport; Care delivery – the provision of care following presentation to the ED; and Transitions – the follow up care and support services that are initiated at the end of an ED visit. Structural racism and bias are acknowledged to influence patients experience across the care continuum. Patients also experience structural and environmental factors such as social needs and public health infrastructure that underlie their need and access to ED care

Of the 13 measures identified as disparities sensitive through application of the NQF scoring protocol, five were categorized as transitions, eight were categorized as care delivery, while none of the measures were categorized as access to care (Table 1). Utilizing the same process of drafting and revision based on expert feedback, topics for potential new measures were also compiled (Table 2). Similar to the existing measures, we found that most feasibly measurable benchmarks fell into process and outcomes categories. Further, potential access measures were limited by the ability to confer attribution to the clinician group level.

Table 2.

Potential New Measure Concepts for Health Disparities in Emergency Care

Emergency Medicine Health Disparities Framework Domain Structure Measures Process Measures Outcome Measures
Access 24/7 interpreter services
Availability of on-call specialists at safety-net facilities
Rate of interpreter use for non-English speaking patients
Racial differences in hallway use, vertical space, waiting room care and other alternate sites of care
Racial differences in left without being seen rate
Racial differences in risk-adjusted transfer rate
Care Delivery Universal collection of self-reported race and ethnicity data in EHR
Implementation of clinical protocols for the care of disparities sensitive conditions (ex. substance use disorder, sickle cell pain crisis)
Completeness of self-reported race and ethnicity data
Use of stigmatizing language in medical record
Adherence to clinical guidelines for patients with acute sickle cell pain crisis
Racial differences in use of sedation and physical restraints for acute agitation
Patient experience of justice-involved individuals
Racial differences in left against medical advice
Transitions Access to social work services in ED, including after hours Quality and completeness of handoff to higher level of care
Completion of medically necessary follow-up care for acute illness or injury (ex. fracture)
Risk-adjusted return ED visit
Risk-adjusted inpatient and ICU admission rates
Structural and Environmental Implicit bias training for clinicians
Availability of social risk screening
Referral to community resources for identified social needs Concordance between ED patient population and local community race, ethnicity, and payer mix

Challenges in Implementing Disparities Measurement in the ED

Race and Ethnicity Data Collection

Universal race and ethnicity data collection is a fundamental prerequisite for measuring health disparities. Currently, patient data on race and ethnicity is incomplete and not standardized across most clinical and administrative datasets. Hospital electronic health systems often record patient race as observed or perceived by hospital staff who perform patient intake documentation. Clinician groups should develop a system to universally collect race and ethnicity data using the current gold standard: self-report. In accordance with the IOM, at minimum health systems should include the following racial categories in standardized surveys and questionnaires: Black or African American, White, Asian, American Indian or Alaska Native, and Native Hawaiian or Other Pacific Islander.29 In addition, more granular ethnicity data should also be collected and current data collection standards also recommend capturing English language proficiency. Collection of race, ethnicity, and English language proficiency should incorporate adequate training to ensure data collection is culturally competent and respectful.49,50

Potential for Harm

Well-resourced health systems may have higher capacity to efficiently measure disparities and implement quality improvement. Institutions with limited resources that serve a large volume of historically marginalized patients may benefit from more staffing resources and improved health information technology to facilitate adequate data collection and quality improvement. Caution is necessary to ensure that disparities measurement does not lead to other unintended or adverse consequences such as clinician selection or avoidance of certain patient populations.

Feasibility

Specific measurement challenges may limit the assessment of health disparities. For example, the target of existing disparities-sensitive QPP measure “Preventive Care and Screening: Screening for High Blood Pressure and Follow-Up Documented” is to incentivize clinicians to refer patients who are hypertensive during their ED visit for follow-up evaluation.19 However, to demonstrate compliance, clinicians need only document that they recommended a follow-up plan and not that follow-up care actually occurred. Thus, the measure is more likely a reflection of documentation practices than actual patient outcomes.

With respect to sample size, many hospitals and clinician groups only treat a small number of patients from minoritized racial and ethnic groups. This introduces significant challenges to fair and accurate comparisons of year-to-year performance or measure performance across practice groups. The application of accepted statistical processes in quality measure reporting, such as risk-adjustment for other factors that may influence the measurement outcome such as socioeconomic status, must also be done carefully. Risk adjustment for socioeconomic status may improve attribution by controlling for factors that are beyond the control of clinicians. Risk adjustment can also safeguard against penalizing safety net clinicians who see a case-mix of patients that are vulnerable to multiple intersecting healthcare disparities. However, risk adjustment has the potential to mask nuanced interrelated disparities and hinder incentives to reduce disparities among those of lower socioeconomic status.

Future Areas for Consideration

Numerous populations are affected by healthcare disparities, including but not limited to women, immigrants, the elderly, people with cognitive or physical disabilities, people who live in rural areas, children, and LGBTQ populations. We hope that our framework and initial groundwork on existing measures based on racial and ethnic disparities will serve as a model for developing new measures for other important populations affected by healthcare disparities.

Conclusions

The ED serves a critical role in healthcare access and the provision of acute care to patients from minoritized racial and ethnic groups. Quality measures offer an opportunity to measure and reduce disparities in care quality. Through an iterative process, we prioritized existing disparities-sensitive emergency care quality measures for stratification by race and ethnicity. We also created a framework for the development of new measures focusing on disparities in emergency care access, care delivery, transitions of care from the ED, and structural/environmental considerations. Implementing these concepts for an ED focused quality measurement framework may help decrease racial and ethnic disparities in care delivery in the emergency medicine setting.

Acknowledgement:

The authors acknowledge participants in the American College of Emergency Physician’s Quality Measurement Development Workgroup: Dr. P. Quincy Moore (University of Chicago), Dr. Marilyn Heine (University of Pennsylvania), Dr. Stephen Epstein (Harvard Medical School), Dr. Don Lum (Northfield Hospital), Dr. Cameron Gettel (Yale University School of Medicine), and Dr. Dennis Hsieh (University of California, Los Angeles). The authors would like to thank Dr. Susannah Bernheim (Yale-New Haven Hospital Centers for Outcomes Research and Evaluation) for lending her expertise in quality outcomes measurement and measure development.

Grant:

This publication was made possible by the Yale National Clinician Scholars Program and by CTSA Grant Number TL1 TR001864 from the National Center for Advancing Translational Science (NCATS), a component of the National Institutes of Health (NIH) as well as the National Health Lung Blood Institute under award number K23 HL143042. Its contents are solely the responsibility of the authors and do not necessarily represent the official view of NIH.

Conflicts of Interest.

Dr. Venkatesh reports support from the American Board of Emergency Medicine - National Academy of Medicine Fellowship, support from the Centers for Medicare and Medicaid Services for the development of hospital and health system quality measures and ratings systems, and prior or current support from the National Institute of Drug Abuse, Moore Foundation, the Foundation for Opioid Response Efforts, Genentech Inc., and the American College of Emergency Physicians for the development of emergency care quality measures and work with the ACEP Clinical Emergency Data Registry

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Nong P, Raj M, Creary M, Kardia SLR, Platt JE. Patient-Reported Experiences of Discrimination in the US Health Care System. JAMA Netw Open. 2020;3(12):e2029650. doi: 10.1001/jamanetworkopen.2020.29650 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.US Department of Health and Human Service. Healthy People 2030. Published 2021. https://health.gov/healthypeople
  • 3.Fiscella K, Sanders MR. Racial and Ethnic Disparities in the Quality of Health Care. Annu Rev Public Health. 2016;37(1):375–394. doi: 10.1146/annurev-publhealth-032315-021439 [DOI] [PubMed] [Google Scholar]
  • 4.MacDorman MF, Thoma M, Declcerq E, Howell EA. Racial and Ethnic Disparities in Maternal Mortality in the United States Using Enhanced Vital Records, 2016‒2017. Am J Public Health. 2021;111(9):1673–1681. doi: 10.2105/AJPH.2021.306375 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.2021 National Healthcare Quality and Disparities Report. Published online 2021. https://www.ahrq.gov/research/findings/nhqrdr/nhqdr21/index.html [PubMed]
  • 6.Institute of Medicine (US) Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care. (Smedley BD, Stith AY, Nelson AR, eds.). National Academies Press; (US: ); 2003. Accessed June 19, 2020. http://www.ncbi.nlm.nih.gov/books/NBK220358/ [PubMed] [Google Scholar]
  • 7.Crossing the Quality Chasm: A New Health System for the 21st Century. National Academies Press; 2001:10027. doi: 10.17226/10027 [DOI] [PubMed] [Google Scholar]
  • 8.Gindi RM, Black LI, Cohen RA. Reasons for Emergency Room Use Among U.S. Adults Aged 18–64: National Health Interview Survey, 2013 and 2014. Natl Health Stat Report. 2016;(90):1–16. [PubMed] [Google Scholar]
  • 9.Lee P, Le Saux M, Siegel R, et al. Racial and ethnic disparities in the management of acute pain in US emergency departments: Meta-analysis and systematic review. The American Journal of Emergency Medicine. 2019;37(9):1770–1777. doi: 10.1016/j.ajem.2019.06.014 [DOI] [PubMed] [Google Scholar]
  • 10.Pines JM, Russell Localio A, Hollander JE. Racial Disparities in Emergency Department Length of Stay for Admitted Patients in the United States. Academic Emergency Medicine. 2009;16(5):403–410. doi: 10.1111/j.1553-2712.2009.00381.x [DOI] [PubMed] [Google Scholar]
  • 11.Cone DC. Health Care Disparities in Emergency Medicine. Academic Emergency Medicine. 2003;10(11):1176–1183. doi: 10.1197/S1069-6563(03)00486-X [DOI] [PubMed] [Google Scholar]
  • 12.Schrager JD, Patzer RE, Kim JJ, et al. Racial and Ethnic Differences in Diagnostic Imaging Utilization During Adult Emergency Department Visits in the United States, 2005 to 2014. Journal of the American College of Radiology. 2019;16(8):1036–1045. doi: 10.1016/j.jacr.2019.03.002 [DOI] [PubMed] [Google Scholar]
  • 13.Richardson LD, Norris M. Access to Health and Health Care: How Race and Ethnicity Matter: ACCESS TO HEALTH AND HEALTH CARE. Mt Sinai J Med. 2010;77(2):166177. doi: 10.1002/msj.20174 [DOI] [PubMed] [Google Scholar]
  • 14.Zhang X, Carabello M, Hill T, Bell SA, Stephenson R, Mahajan P. Trends of Racial/Ethnic Differences in Emergency Department Care Outcomes Among Adults in the United States From 2005 to 2016. Front Med. 2020;7:300. doi: 10.3389/fmed.2020.00300 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Franks NM, Gipson K, Kaltiso SA, Osborne A, Heron SL. The Time Is Now: Racism and the Responsibility of Emergency Medicine to Be Antiracist. Annals of Emergency Medicine. 2021;78(5):577–586. doi: 10.1016/j.annemergmed.2021.05.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Burstin H, Leatherman S, Goldmann D. The evolution of healthcare quality measurement in the United States. J Intern Med. 2016;279(2):154–159. doi: 10.1111/joim.12471 [DOI] [PubMed] [Google Scholar]
  • 17.Trivedi AN, Nsa W, Hausmann LRM, et al. Quality and Equity of Care in U.S. Hospitals. N Engl J Med. 2014;371(24):2298–2308. doi: 10.1056/NEJMsa1405003 [DOI] [PubMed] [Google Scholar]
  • 18.Sehgal AR. Impact of Quality Improvement Efforts on Race and Sex Disparities in Hemodialysis. JAMA. 2003;289(8):996. doi: 10.1001/jama.289.8.996 [DOI] [PubMed] [Google Scholar]
  • 19.American College of Emergency Physicians. Clinical Emergency Data Registry. Published 2022. https://www.acep.org/cedr/
  • 20.Epstein SK, Griffey RT, Lin MP, Augustine JJ, Goyal P, Venkatesh AK. Development of a qualified clinical data registry for emergency medicine. JACEP Open. 2021;2(6). doi: 10.1002/emp2.12547 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.MEDIC. Michigan Emergency Department Improvement Collaborative (MEDIC). https://medicqi.org/
  • 22.American College of Emergency Physicians. Emergency Quality Network (E-QUAL). https://www.acep.org/administration/quality/equal/emergency-quality-network-e-qual/
  • 23.Schuur JD, Hsia RY, Burstin H, Schull MJ, Pines JM. Quality Measurement In The Emergency Department: Past And Future. Health Affairs. 2013;32(12):2129–2138. doi: 10.1377/hlthaff.2013.0730 [DOI] [PubMed] [Google Scholar]
  • 24.Kocher KE, Arora R, Bassin BS, et al. Baseline Performance of Real-World Clinical Practice Within a Statewide Emergency Medicine Quality Network: The Michigan Emergency Department Improvement Collaborative (MEDIC). Annals of Emergency Medicine. 2020;75(2):192–205. doi: 10.1016/j.annemergmed.2019.04.033 [DOI] [PubMed] [Google Scholar]
  • 25.Venkatesh AK, Scofi JE, Rothenberg C, et al. Choosing wisely in emergency medicine: Early results and insights from the ACEP emergency quality network (E-QUAL). The American Journal of Emergency Medicine. 2021;39:102–108. doi: 10.1016/j.ajem.2020.01.029 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Chaiyachati KH, Qi M, Werner RM. Changes to Racial Disparities in Readmission Rates After Medicare’s Hospital Readmissions Reduction Program Within Safety-Net and Non–Safety-Net Hospitals. JAMA Netw Open. 2018;1(7):e184154. doi: 10.1001/jamanetworkopen.2018.4154 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Jean-Jacques M, Persell SD, Thompson JA, Hasnain-Wynia R, Baker DW. Changes in Disparities Following the Implementation of a Health Information Technology-Supported Quality Improvement Initiative. J GEN INTERN MED. 2012;27(1):71–77. doi: 10.1007/s11606-011-1842-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Centers for Medicare and Medicaid Services. Disparity Methods Confidential Reporting Overview. Published 2021. https://qualitynet.cms.gov/inpatient/measures/disparity-methods
  • 29.Driving Health System Transformation-A Strategy for the CMS Innovation Center’s Second Decade. Published online 2021. https://innovation.cms.gov/strategic-direction-whitepaper
  • 30.Disparities Solution Center. Commissioned Paper: Healthcare Disparities measurement. Published online 2011. https://5536401f-20a1-4e61-a28e-914fb5dcef51.filesusr.com/ugd/888d39_b399d3b7cffa498b821410dab155b401.pdf
  • 31.National Quality Forum. National Voluntary Consensus Standards for Ambulatory Care—Measuring Healthcare Disparities. Published online 2008. https://www.qualityforum.org/Publications/2008/03/National_Voluntary_Consensus_Standards_for_Ambulatory_Care%E2%80%94Measuring_Healthcare_Disparities.aspx
  • 32.National Quality Forum. Healthcare Disparities and Cultural Competency Consensus Standardds: Disparities-sensitive Measure Assessment. Published online 2012. https://www.qualityforum.org/Publications/2012/11/Healthcare_Disparities_and_Cultural_Competency_Consensus_Standards__Disparities-Sensitive_Measure_Assessment.aspx
  • 33.Anderson AC, O’Rourke E, Chin MH, Ponce NA, Bernheim SM, Burstin H. Promoting Health Equity And Eliminating Disparities Through Performance Measurement And Payment. Health Affairs. 2018;37(3):371–377. doi: 10.1377/hlthaff.2017.1301 [DOI] [PubMed] [Google Scholar]
  • 34.Assistant Secretary for Planning and Evaluation. Developing Health Equity Measures. Published online May 19, 2021. https://aspe.hhs.gov/sites/default/files/migrated_legacy_files//200651/developing-healthequity-measures.pdf
  • 35.Harrington Rachel, Washington Deidre, Paliani Sarah, Thompson Keirsha, Rouse Latoshia, Anderson Andrew. A New Effort To Address Racial And Ethnic Disparities In Care Through Quality Measurement. Health Affairs. doi: 10.1377/forefront.20210907.568444 [DOI] [Google Scholar]
  • 36.Donovan J, Nelson G. Minnesota Healthcare Disparities Report. Published online 2020. https://mncmsecure.org/website/Reports/Community%20Reports/Disparities%20by%20RELC/2020%20Disparities%20by%20RELC%20Chartbook%20-%20FINAL.pdf
  • 37.Measuring Mental Health Disparities: A Roadmap & Recommendations for Implementation of the Mental Health Equity Act. Published online 2018. https://cpehn.org/assets/uploads/archive/resource_files/ab470roadmap0618_5pg_003.pdf
  • 38.National Quality Forum. Attribution: Principles and Approaches. Published online October 7, 2016. https://www.qualityforum.org/Projects/a-b/Attribution_2015-2016/Draft_Report_for_Comment.aspx
  • 39.Gettel CJ, Han CR, Granovsky MA, et al. Emergency clinician participation and performance in the Centers for Medicare & Medicaid Services Merit-based Incentive Payment System. Acad Emerg Med. Published online September 7, 2021:acem.14373. doi: 10.1111/acem.14373 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Donabedian A The Quality of Care: How Can It Be Assessed? JAMA. 1988;260(12):1743. doi: 10.1001/jama.1988.03410120089033 [DOI] [PubMed] [Google Scholar]
  • 41.Jones CP. Levels of racism: a theoretic framework and a gardener’s tale. Am J Public Health. 2000;90(8):1212–1215. doi: 10.2105/ajph.90.8.1212 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Feagin J, Bennefield Z. Systemic racism and U.S. health care. Social Science & Medicine. 2014;103:7–14. doi: 10.1016/j.socscimed.2013.09.006 [DOI] [PubMed] [Google Scholar]
  • 43.Aysola J, Clapp JT, Sullivan P, et al. Understanding Contributors to Racial/Ethnic Disparities in Emergency Department Throughput Times: a Sequential Mixed Methods Analysis. J GEN INTERN MED. Published online August 2, 2021. doi: 10.1007/s11606021-07028-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Dehon E, Weiss N, Jones J, Faulconer W, Hinton E, Sterling S. A Systematic Review of the Impact of Physician Implicit Racial Bias on Clinical Decision Making. Acad Emerg Med. 2017;24(8):895–904. doi: 10.1111/acem.13214 [DOI] [PubMed] [Google Scholar]
  • 45.Rubin H, Pronovost P, Diette G. The Advantage and Disadvantages of Process-Based Measures of Health Care Quality. International journal for quality in health care : journal of the International Society for Quality in Health Care / ISQua. 2002;13:469–474. doi: 10.1093/intqhc/13.6.469 [DOI] [PubMed] [Google Scholar]
  • 46.Hayford T, Maeda J. Issues and challenges in measuring and improving the quality of health care. Published online December 2017. Accessed July 20, 2022. https://www.cbo.gov/system/files/115th-congress-2017-2018/workingpaper/53387workingpaper.pdf
  • 47.Levine DA, Duncan PW, Nguyen-Huynh MN, Ogedegbe OG. Interventions Targeting Racial/Ethnic Disparities in Stroke Prevention and Treatment. Stroke. 2020;51(11):3425–3432. doi: 10.1161/STROKEAHA.120.030427 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Sivashanker K, Duong T, Resnick A, Eappen S. Health Care Equity: From Fragmentation to Transformation. New England Journal of Medicine Catalyst. Published online 2020. https://catalyst.nejm.org/doi/full/10.1056/CAT.20.0414 [Google Scholar]
  • 49.Hasnain-Wynia R, Pierce D, Haque A, Hedges Greising C, Prince V, Reiter J. Health Research and Educational Trust Disparities Toolkit. Published online 2007. doi: 10.13016/NTH0-VT37 [DOI] [Google Scholar]
  • 50.Race, Ethnicity, and Language Data: Standardization for Health Care Quality Improvement. National Academies Press; 2009:12696. doi: 10.17226/12696 [DOI] [PubMed] [Google Scholar]

RESOURCES