Abstract
The growing recognition of differences in health outcomes across populations has led to a slow but increasing shift towards transparent reporting of patient outcomes. In addition, pay-for-equity initiatives, such as those proposed by the Centers for Medicare and Medicaid, will require the reporting of health outcomes across subgroups over time. Dashboards offer one means of visualising data in the health-care context that can highlight essential disparities in clinical outcomes, guide targeted quality-improvement efforts, and ultimately improve health equity. In this Viewpoint, we evaluate all studies that have reported the successful development of a disparity dashboard and share the data collected and unintended consequences reported. We propose a framework for systematic equality improvement through incentivisation of the collecting and reporting of health data and through implementation of reward systems to reduce health disparities.
Introduction
The importance of monitoring for health equity
The COVID-19 pandemic exposed and highlighted health inequities to which racial and ethnic population subgroups are subject, with disproportionately higher rates of infection, hospitalisation, complications, and death faced by these groups.1,2 Recent evidence had identified the flaws and adverse outcomes in race-based medicine, where inequities pervade health care and influence important management decisions and outcomes.3–5 These disparities, however, are not limited to COVID-19, nor are they entirely represented by race and ethnicity, often involving highly connected and intersectional features of social determinants of health.6–8 While some health disparities are overt, others are introduced and perpetuated inadvertently, such as through differences in the performance of health-care devices.3,4,9
Although research has been crucial to understanding the scale and impact of health inequities, there are still significant needs for tracking of day-to-day markers of disparity that can affect patient care, identifying the drivers of evolving trends, and assessing the effectiveness of interventions. Frequent monitoring of health inequities is needed to inspire positive change, from local institutions to national health-care systems.10 Health equity initiatives such as those proposed by the Centers for Medicare and Medicaid Services seek to tie reimbursements to equitable outcomes and reduce disparities.11,12 These initiatives will necessitate the systematic reporting of patient outcomes in specific subgroups, which, in turn, will require the development of infrastructure capable of capturing differences over time and in response to specific interventions.10,13,14
A notable prerequisite for expanding routine monitoring is a larger cultural acceptance and willingness to address health disparities. Changes in mindsets have been aided by the top-down implementation of government operational policies and programmes that require the collection of health equity-focused measures.11 Moreover, aligning payment incentives and hospital ratings with such measures could be an important way of rewarding good practice. Mandatory reporting of surgical outcomes by race or ethnicity and greater transparency among academic institutions about disparities in health care promote greater awareness of health disparities.15,16 These examples reflect the profound shift in attitudes towards medical errors, where the increasing willingness to confront the systemic issue of health inequities requires increasing capabilities to recognise and measure these inequities.14,17,18
Need for continuous evaluation of the impact of artificial intelligence-based solutions on health inequity
Artificial intelligence (AI) can personalise care, improve quality, and reduce adverse events by predicting optimal treatment strategies.19 However, systems that use algorithms to improve decision making can also exacerbate existing biases, including human bias, evidence bias, and those embedded in data trained on electronic health records.20 Furthermore, complete removal of bias is impossible, as no one group can fully comprehend these intrinsic complexities, as highlighted by a study that showed models could learn a patient’s race from medical imaging alone, without any clinical data.21 The potential impact of unchecked bias in AI is profound.22
The US Department of Health and Human Services has acknowledged this potential danger by proposing new legislation prohibiting discrimination against an individual’s race, colour, national origin, sex, age, or disability, through decisions reliant on applying clinical decision support algorithms.23–25 However, there has been little progress in the construction, implementation, and scaling of associated infrastructure necessary to evaluate, validate, and update AI models.26,27 In addition, there is a paucity of systems to evaluate the effects of these algorithms on patient subgroups and the resulting effects on disparities across time. This infrastructure is crucial to safeguard against implementing potentially harmful models and to enforcing such legislation.28
Dashboards to visualise data and guide change
A dashboard is a visual and interactive management tool that uses routinely updated data to display complex trends in summarised and simplified forms. In the clinical setting, the dashboard’s purpose is threefold: (1) to track and monitor key activities, displaying performance relative to goals, sometimes referred to as benchmarks; (2) to facilitate root cause analyses of problems through the interaction of multiple views, levels of detail, and subgroup analysis (disaggregation); and (3) to manage and monitor the effects of policies and processes that aim to improve clinical decisions and outcomes.29
Dashboards offer one means of visualising data in health care that can highlight important patterns, guide quality-improvement efforts, and improve patient outcomes.30 Dashboards can follow a range of process and outcome data spanning individual provider practices to organisation-level outcomes, with previous reviews noting substantial heterogeneity in their composition and use between institutions.31,32 Dashboards can also display regional performance benchmarking data, or nation-level data, and organisations such as the Agency for Healthcare Research and Quality and the Network of Patient Safety Databases have published key safety metrics under the guidance and protection of federal law.33 Although dashboards have significant potential, the substantial complexities of metric identification, data integrity, dashboard design, and information sharing cannot be overstated. Furthermore, deploying these tools into organisational practice in a thoughtful manner is necessary to avoid unintended consequences such as the reinforcement of stereotypes and group labels.17,29
Dashboards have been used to track metrics and performance indicators of disparities in health care. We found neither a previous summary of disparity dashboards nor previously published best practices for the creation and implementation of disparity dashboards. We conducted a scoping review to summarise the available literature describing previously designed disparity dashboards, to identify useful features, and to highlight the challenges and strategies related to implementation. Informed by the findings, we present a framework for systematic equality improvement efforts.
Methods
We searched PubMed to identify published descriptions of dashboards dedicated to monitoring health inequities. This same literature search was done in two parts: the first until July 21, 2022, and the latter to search for all remaining studies until May 13, 2023. Search terms referring to inequalities and demographic features were combined with the terms for dashboards in PubMed (appendix 1 pp 1–2). Papers that focused on dashboard development or validation concerning differences in health outcomes among population subgroups were included, as were clinical dashboards that stated the ability to compare outcomes across subgroups to reduce inequity. Studies that assessed dashboards that do not focus on measuring disparities between subgroups or studies that analysed dashboard data without contributing to that dashboard’s development (eg, studies analysing publicly available dashboards) were excluded. No restriction on language was used. Reviews and perspectives on the topic of health equity dashboards that did not articulate dashboard development were excluded. The full search strategy used for the scoping review and the resulting PRISMA diagram can be found in appendix 1 (pp 1–4).
From the selected articles, the following variables were extracted: region studied (ie, international, national, regional [state, province, county], municipal [city, district], or institutional), type of organisation (ie, international, governmental, academic, hospital, or industry), data source, frequency of updates (ie, static, daily, weekly, monthly, or yearly), and measured outcomes. Demographic features that the dashboard was capable of disaggregating were classified using the REGAL+ system: race, ethnicity, gender and sex, age, language or limited English proficiency, plus other variables.34 Dashboard presentation included the available languages, intended audience, disaggregation level, visualisation options, and availability of time trends. Dashboard features recorded included interactivity, research compatibility, and data extraction capacity. Data were systematically extracted by authors into a predefined spreadsheet. Descriptive statistics were calculated, and full data are available in appendix 2.
Results
We identified 278 articles, of which 256 were excluded. The remaining 22 studies were included in the final analysis. Dashboard focus varied: four were related to COVID-19, five were related to maternal mortality or paediatric health care, and three were based in emergency departments. Other topics included HIV cases, rural health care, and Medicare Health Equity Summary Score outcomes.
A summary of the characteristics of included active dashboards is displayed in table 1. Most published dashboards were completed as part of a national effort (n=8) or focused on regional or statewide differences (n=7). Additionally, four dashboards were internationally focused and the remaining three were centre-specific dashboards. Most originated from North America (n=17), and only two specifically focused on low-income and middle-income countries (LMICs). Few studies showed working data pipelines that received updates at least monthly, and 15 displayed static retrospective data. 21 displayed graphs of collected data, 13 could visualise time trends, and 17 were interactive, but only five allowed extraction of data from the dashboard for analysis.
Table 1:
Overview of disparity dashboards inside and outside the USA
Purpose | Region studied* | Setting† | Update Status‡ | REGAL+ features | Outcomes measured | Data sources | |
---|---|---|---|---|---|---|---|
Dashboards based in the USA | |||||||
Connolly et al (2021)35 | Institution-wide Equity Lens platform to guide quality improvement | City or institution | Hospital inpatients | Live | REGAL + zip code and payor | Hypertension management, hospital readmissions | University of Chicago Medical Center |
Penaia et al (2021),36 NHPI COVID-19 Data Policy Lab Dashboard37 | Disparities in Native Hawaiian and Pacific Islander COVID-19 outcomes | National | Population level | Static | RE | Cases, mortality | Manual scraping of community dashboards |
Tsuchida et al (2021)38 | Dashboard to identify disparities in emergency department outcomes | City or institution | Hospital inpatients | Live | REGAL + sexual orientation, religion weight | Investigation rates, medication rates, time to adverse outcomes, length of stay | University of Michigan institutional warehouse |
Marvel et al (2021),39 COVID-19 Pandemic Vulnerability Index40 | Model vulnerable populations during the COVID-19 pandemic | National | Population level | Live | Health disparity index, insurance status, Social Vulnerability Index, socioeconomic status | Pandemic Vulnerability Index: infection rate, population concentration, intervention measures, health-care vulnerability | CDC’s Social Vulnerability Index, John Hopkins University |
Jonnalagadda et al (2022),41 Ohio Children’s Opportunity Index42 | Display social determinants of paediatric health in new Opportunity Index | State | Population level | Static | REGA + income, insurance status, education, zip code | Opportunity Index: family stability, infant health, children’s health, access, education, housing, environment, criminal justice | US Census-based American Community Survey, Medicaid claims, and Department of Education school report card data |
Bilal et al (2022),43 COVID-19 Health Inequities in Cities44 | Characterise inequities in COVID −19 -related outcomes | National | Population level | Live | RE + neighbourhood | Testing rates, cases, hospitalisations, mortality | Big Cities Health Coalition data |
Sullivan et al (2022),45 America’s HIV Epidemic Analysis Dashboard46 | Display HIV indicators provided by the CDC | National | Population level | Live | REGA | Incidence, knowledge of status, diagnoses, linkage to HIV medical care, viral suppression, pre-exposure prophylaxis coverage | National HIV Surveillance System and CDC data |
Patrick et al (2017)47 | Track trends in HIV testing and care for men who have sex with men, people who inject drugs, and heterosexual people at elevated risk | State | Population level | Static | REG + sexual preference | Number tested for HIV, HIV tested in the past 12 months, HIV-positive and newly positive per time period | CDC National HIV Behavioral Surveillance |
Hester et al (2020)48 | Display baseline data on paediatric health disparities | City or institution | Hospital inpatients | Static | RE | Procedure-related pain control, asthma control, vaccination rates, non-attendance rates | Institutional data warehouse |
Hutchinson-Colas et al (2022),49 New Jersey Maternal Mortality Dashboard50 | Display disparities in maternal outcomes across New Jersey | State | Population level | Static | REA + wealth, insurance status | Maternal mortality rate and ratio | National Vital Statistics System and Pregnancy Mortality Surveillance System |
Martino et al (2023)51 | Track Health Equity Summary Scores for Medicare plan providers | National | Hospital inpatients | Static | R + income | Health Equity Summary Scores | Local warehouses |
Spoer et al (2022),52 City Health Dashboard53 | Track regional risk for SARS-CoV-2 infection and outcomes | National | Population level | Static | Composite score | New composite COVID-19 local risk index | Big Cities Health Coalition data |
Le et al (2022),54 Local Trends in Heart Disease and Stroke Mortality Dashboard55 | Monitor local trends in heart disease and stroke mortality | National | Population level | Static | REGA | Cardiovascular disease-related mortality and associated outcomes | National Vital Statistics System of the National Center for Health Statistics |
Backonja et al (2022),56 Solutions in Health Analytics for Rural Equity across the Northwest57 | Identify health disparities and local social determinants of health | Regional | Population level | Static | REGAL | Insurance types, diabetes, mental health-related outcomes | Multiple publicly available datasets including governmental health agencies and national surveys |
Ising et al (2023)58 | Surveillance of emergency department mental health outcomes | Regional | Hospital inpatients | Static | REGA + insurance | Counts, crude rates, and emergency department visit percentages for five mental health conditions | NC Detect initiative |
Hartford et al (2023)59 | Improving patient equity in a paediatric emergency department | Regional | Hospital inpatients | Active | REAL | Triage level, waiting times, analgesia rates, interpreter use | Seattle Children’s Hospital electronic health records |
Riegels and Mazotti (2023)60 | Visualise programme health equity measures for a residency programme | Regional | Hospital inpatients | Static | R | Care Quality Commission targets on key diseases (eg, diabetes, asthma) | Healthcare Effectiveness Data and Information Set data |
Dashboards based outside the USA | |||||||
Arsenault et al (2017)61 | Equity in vaccination coverage of 45 countries supported by Gavi, The Vaccine Alliance | International | Population level | Static | Income and wealth index, maternal education | Childhood vaccination coverage by country | National vaccination records |
Sanhueza et al (2022),62Todas las Mujeres Todos los Niños (Every Woman Every Child)63 | Visualise trends in social inequalities in health in Latin American and Caribbean countries | International | Population level | Static | Country, urban vs rural, maternal education, wealth | Antenatal care coverage, maternal and newborn health, nutrition, mortality, fertility | National surveys: Demographic and Health Surveys and Multiple Indicator Cluster Survey |
Verhagen et al (2020)64 | Forecast the number of individuals at risk of COVID-19 hospitalisation per region in the UK | National | Population level | Static | R + social deprivation | Hospitalisation, critical care admission, hospital capacity | Census estimates and hospital capacity data from Office for National Statistics and National Health Service |
Goulart et al (2021),65 Measurement of Sexual and Reproductive Health and Rights, Gender Equality, and Women’s Empowerment in Humanitarian Settings66 | Identify health rights and outcomes between genders in humanitarian studies across 45 low-income and middle-income countries | International | Population level | Static | G + country | Maternal health, HIV, perinatal health | Multiple studies |
Allan et al (2022),67 WHO COVID-19 Detailed Surveillance Data Dashboard68 | Display COVID-19 surveillance data across 184 countries | International | Population level | Live | AG | Cases, case fatality ratio | Multiple international sources and registries |
CDC=US Centers for Disease Control and Prevention. REGAL+=race, ethnicity, gender and sex, age, language or limited English proficiency, plus any other feature of interest.
International, national, state, or city or institution.
Hospital inpatients, hospital outpatients, or population level.
Live or static.
All studies that attempted to quantify the impact of the dashboard evaluated the number of users and engagement with the tool itself, which was unanimously positive. However, only six evaluated the use of a disparity dashboard as part of a wider process to identify clinical disparities, hypothesise potential interventions, or track progress to a given goal.35,45,51,54,56,69 Nine of the studies reported communication with local or patient partners during the design phase.35,43,45,48,51,56,58,59,62 Several important lessons were documented in each study, in addition to plans for future developments; we discuss these and our own derived recommendations in table 2.
Table 2:
Important questions for developing clinically useful disparity dashboards
Key questions | Explanation | |
---|---|---|
Clear audience and use case | Who are the intended users? What are their information needs? What language needs to be used? |
For data to be presented appropriately, there must be clarity on the intended use and user of the dashboard. Different user interfaces should be identified for management, governments, physicians, and patients, and multilanguage functionality should be enabled to maximise the engagement of diverse cohorts. |
Focused outcomes | Which outcomes are going to be displayed? What variables are required to encompass the patient’s status? What measures are required by each individual team? |
For dashboards to be useful and accurate, they must collect data addressing the factors that contribute to and explain the root causes of outcomes and disparities, including pre-admission and admission data, social and medical factors, and operational and legal guidelines. Outcomes should be tailored to individual groups on the basis of their goals and incentives. Inclusion of process measures is valuable for tracking intermediate steps and short-term changes, whereas clinical outcomes might lag. |
Interaction and exploration | What analysis can be done within the application? What time periods are available for segmentation? What are the different views of data presentation that can be selected? |
Users will be interested in different catchment areas, so functionality should permit analysis of large populations as well as smaller units, such as clinics within a health-care organisation. Data should be organised intuitively and allow for interactive exploration of findings with different levels of granularity and aggregation or disaggregation. Providing multiple views and being able to explore the data for missingness and hidden biases is imperative and can aid in generating greater understanding than any view can achieve on its own, especially in combination with visual tools such as tables and charts, which can better articulate these biases and differences in outcomes. |
Context-appropriate design | What type and volume of information needs to be displayed? What is the best way to display data and convey uncertainties? What are the various metrics that can be used to explain the data? |
Metrics used are important to the interpretation of results. Absolute and relative values should be presented alongside a measure of uncertainty. These measures should be clarified and articulated using contextual language for each user (eg, 99% survival vs 1% mortality). Appropriate restrictions on the volume of information displayed and the use of visual cues to simplify information and direct attention to important results are necessary. |
Maximum transparency | What data sources are going to be used? What data will be made available to whom? How should the limitations of incomplete data be approached? How are populations and metrics defined? |
Making clear the data sources and methods used to construct indicators is important for building clinical and public trust. Organisations should be as transparent as possible while recognising potential legal and privacy-related issues. Data should be available to researchers who can uncover biases and design tools to improve clinical care. Similarly, patients should know the level of care they are receiving and how their data are being used to help improve care for others. |
Continuous sampling | What sampling frequency is being used? How are changes in clinical practice or health policy marked for evaluation? How often are new questions asked and additional data integrated? |
Public health surveillance through discontinuous episodic research, especially using retrospective study designs, is not sufficient to reduce health inequity. Without continuous monitoring of how disparities change over time and in relation to government policies or other interventions, there can be no accountability for the differential impacts of policies on vulnerable groups. There should be a process for challenging assumptions, collecting new data, and evaluating outcomes. Dashboards should be designed with this flexibility in mind. |
Appropriate disaggregation | Are essential indicators available (eg, REGAL)? Are indicators of underlying social risk factors available (eg, financial strain, food insecurity)? What stratification can be done? What labels have been selected, and how are they defined? |
A commonly cited barrier to reducing health disparities is the ongoing scarcity of data collected in key areas, including but not limited to REGAL. Of the data that are collected, the methods and definitions used can be inconsistent. These factors have resulted in a fragmented data landscape wherein there is a widespread lack of interoperability between institutions and incompatibility of data types, such as social, health, and economic data sources. Moreover, data are often extracted from administrative sources rather than relying on patient self-reports, which risks misclassification. It is important to move beyond demographic criteria to underlying social risk factors, which are root causes of health disparities. Depending on the makeup of the population being tested, standard datapoints of race or ethnicity might not show statistically significant differences; therefore, the creation of a variety of composites, including multiple features, should be allowed to represent that patient best. |
Diversity in design and in use | What is the demographic and specialty composition of dashboard designers? What is the demographic and specialty composition of end users? Have patient partners been consulted in the design process? |
The composition of the group using the dashboard will affect the lens through which the data are perceived. Ensuring a diverse background of end users will safeguard against inaccurate or biased assessments of the information. Understanding the differences of each population and respecting the uniqueness of each patient are crucial in creating this culture. It is equally important to ensure this diverse perspective is sought during the building of the dashboard to reduce incorporation of bias before data collection and analysis. Significant consultation with patient partners and wider stakeholders should become standard practice. |
Process evaluation | What checks will be done after implementation? How will developments be decided upon? How are findings communicated with leadership and front-line staff? How will information be shared with researchers and patients? |
Data integrity checks are crucial to ensure adequate calibration of results and actions in clinical practice. Forecasting and exploratory analysis will allow for relevant evaluation and benchmarking of systems before significant harm has been realised. Subsequently, the distribution of consistent and unbiased findings will allow for honest discourse that can diagnose potential problems and develop relevant solutions. Open-first approaches are those that transparently share data for review, but they also reduce duplication of work and can reduce the risk associated with experimentation. |
Oversight and funding | How will systems be rewarded for reaching objectives? Who will be involved in setting benchmarks? What systems of accountability will be in place for interpreting and acting on findings? |
To identify the impact of dashboards and outcomes, there must be an appropriate benchmark selection and aligned incentives for organisations to strive for. This could include but is not limited to connections to the Joint Commission guidelines and Accreditation Council for Graduate Medical Education funding. There should also be local accountability measures in place that ensure that disparities are actively identified and interventions appropriately deployed. |
Discussion
We identified 14 studies describing dashboards aimed at identifying and measuring health disparities by comparing various outcomes among population subgroups. Although there was substantial heterogeneity in the topic of the dashboards, commonalities were identified, including that most published dashboards were generated by academic medical centres and focused on either institutional or regional catchment areas. Disaggregation by race and ethnicity categories was most common, although some included disaggregation by sex, language, and age. Most disparity dashboards were static and not repeatedly updated after the final versions were released, suggesting that the demands of maintaining disparity dashboards—eg, the infrastructure, personnel, resources, and expertise required—exceeded organisational capacities.
Pertinent absences
There is a dearth of published examples of work on health equity dashboards; we found only 14 developed dashboards on PubMed. Most studies originated from the USA, and few integrated global data. In addition, most were composed of visualisations of static retrospective data without any active data pipeline, severely limiting ongoing clinical utility. The infancy of the technology being used in this field is notable as it is associated with a fragmented data landscape and little interoperability. The lack of collected data and its subsequent compatibility limits large-scale quantification and ongoing monitoring of disparities and the ability to measure the impacts of interventions, including the implementation of health policies.
On a broader scale, data poverty70 in LMICs limits efforts to improve global health outcomes. Inequalities in health care are present in all countries; however, in lower-income countries, there might be causes of inequality that are distinct from those in high-income countries. Without local data across these different regions, these causes cannot be identified and overcome, and whether an intervention transferred from one location would result in a positive outcome in another location cannot be certain.33,69,71
Most disparity dashboards identified in our search were designed to identify differences in outcomes across race and ethnicity, which are powerful predictors of health outcomes, particularly in the USA. However, when evaluated alone, the relationships between race and ethnicity and other social determinants of health are lost, providing an incomplete picture of the drivers of health inequalities. Moreover, inequity is a dynamic process in which the marginalised populations of the future are unlikely to be known at present because of current under-reporting and future global policy changes.72 These differences in outcomes can be created or exacerbated by novel data-driven technologies such as AI. Ultimately, adequately confronting inequities in medicine requires adequate data capture, which, as shown in the identified articles, is not currently occurring with appropriate granularity or scale.
Potential unintended consequences and potential solutions
The process of identifying and quantifying health disparities is not passive, requiring data selection, framing, and visualisation. Therefore, dashboards are not simply a neutral window into the available data and are prone to unintended consequences. They can either encourage the investigation of disparity mediators or suppress differences, depending on the design.
Although the aim is to promote equity and improvement in outcomes, disaggregating data can also lead to stereotypes and labels being reinforced instead of removed—for example, shifting focus to race instead of racism. This is particularly relevant when considering the accuracy of and heterogeneity within labels (eg, “Asian”) and the implications for downstream use of the data.73 It is also important to evaluate the intersectionality of disparities, encompassing access to and use of health care, environmental factors, baseline health status, and differences or biases in the quality of health care received.
Solutions require thoughtful consideration of all factors contributing to disparities, leveraging diverse perspectives. This means using a combination of design thinking and collaborative working sessions with a multidisciplinary team. Different stakeholders or experts would be responsible for identifying priorities, useful measures, and their downstream utility in an ideal scenario. These stakeholders include clinicians, patients and families, researchers, and analysts familiar with dashboard development. The wider team would review alternative perspectives, crowdsource visualisation, and create shared understanding. Moreover, this process must be iterative to ensure ongoing alignment over time.
Important questions for future disparity dashboards
Disparity dashboards build on the strengths of traditional clinical dashboards by not only identifying and monitoring disparities but also by helping to understand the underlying mechanism causing inequality. Moving beyond demographic factors to include underlying social or structural determinants of health is important, as is the inclusion of actionable information. Delivering actionable information to health-care workers and organisations can be achieved by storing past information, presenting new data in real time, and allowing for the forecasting of future outcomes. Several key lessons were noted by authors during the dashboard development process, including the importance of engagement of local stakeholders and the need to clearly identify target users. Table 2 breaks down key questions that need to be considered during development and after implementation of disparity dashboards.
Measuring differences in outcomes within an institution is an important prerequisite for identifying drivers of outcomes as well as potential solutions. However, the ability to make local outcomes interoperable between sites, regions, and countries, which is necessary to scale impact, is often lacking. Practically, it would be ideal if health equity data were standardised to allow for comparative assessment; however, this is a contentious point as the collection and distribution of sensitive data is not unanimously accepted among communities. Furthermore, drivers of inequality that might be of interest are different across regions, as are their definitions. The features themselves thus require local stakeholders and patient advocates to be involved in this decision-making process. It is important also to consider the intersectionality of disparities and the complex interactions of wider determinants of health, insurance status, health-care provider, and REGAL features. In most cases, however, the systematic evaluation of health data and continuous challenging of assumptions are vital.
Health equity standards demand maximum capture at the local level to allow for different contexts and to facilitate implementation. However, national and international standards must be implemented as minimum requirements to evaluate national policy. Improved interoperability helps to maximise the available data for researchers and could provide a transparent comparison of outcomes to patients, improving patient autonomy. An example of the benefits this confers is the regionally focused approach of the social needs index in California, USA.74 Furthermore, a disparity dashboard that goes beyond intermittent outcome measurement could provide a central hub for monitoring commonly used algorithms, safeguarding against the introduction or worsening of biased practices, and providing vital reassurances to the clinician and patients themselves around data sharing and algorithm implementation.
Incentivising disparity dashboard development
The publication and distribution of standardised and disaggregated health outcomes will help to quantify the scale of health inequity. This quantification will, in turn, enable a more appropriate allocation of resources to reduce outcome gaps and identify future research needs that we are currently unaware of. Systematic equality improvement initiatives such as the Medicare Advantage programme will strongly incentivise organisations to develop tools that can track disparities over time.75 Additionally, recent recommendations from the US non-profit organisation The Joint Commission76 have identified several important areas through which to improve health equity, including the uniform collection of demographic data, health-related social needs, stratification of service and technical quality measures, accreditation bodies’ requirements for REGAL outcomes, and the partnering of researchers with technology groups.77 Despite these recommendations, there has been little published on the outcomes of implementing such guidance in clinical care. Where reported, actions have largely focused on measuring and reporting rather than on adapting clinical and population health structures within health systems to be able to accomplish equity goals effectively. Whether due to system pressures, budget constraints, or other wise, the urgency for change is as driven by finance as it is by ethics.
The investment of institutions into the health systems infrastructure required for repeatable measurement and analysis is important, yet the benefit of this investment is the discovery of biases, institutional barriers, and negative outcomes. Therefore, there is a reputational and financial disconnect between the investment required and the associated risk that comes with the morally correct decision. To facilitate the scaling of health equity will require oversight by organisations that regulate and incentivise standardisation of data collection, use, and reporting from the top down. Accreditors, such as The Joint Commission, will also be important in setting standards for care and regulating clinical practice in accordance with nationally set benchmarks.
Limitations
Our study is based on the publication of disparity dashboards in the PubMed database and did not include conference proceedings or other grey literature. In addition, our search strategy contains traditional demographic variables that might not represent features leading to disparities in all areas, particularly LMICs. The focus on inequities related to REGAL primarily in high-income countries might not be generalisable to other settings, where disparities might focus on other identifiers or alternative interpretations of currently included identifiers. Additionally, it is difficult to know whether the publications described here are representative of all disparity dashboards currently in use, and undoubtedly many have been developed but not formally evaluated or published.
Conclusion
Many institutions are beginning to confront the deeper challenge of aligning population health efforts, institutional strategies, and clinical workflows to promote more equitable outcomes. This change is mirrored by increasing proposals that tie funding to reducing disparities in the US context. However, more needs to be reported on the use of dashboards to measure such outcomes continuously.
Disparity dashboards should be developed by a diverse interdisciplinary team that can build continuous and regularly reviewed data streams. Dashboards are vital tools for safeguarding patient outcomes, improving health policies, reducing health inequities, and facilitating safe implementation of AI. Disparity dashboards make it possible for health systems and providers to track, measure, and better understand their ability to deliver equitable care with actionable, granular measures in order to be accountable to themselves and the greater mission of improving health-care equity and quality.
Supplementary Material
Acknowledgments
LAC is funded by the National Institutes of Health through the National Institute of Biomedical Imaging Grant R56EB017205.
Footnotes
Data sharing
Data used in this study are provided in appendix 2.
Declaration of interests
ID-M declares leadership or fiduciary roles and stock or stock options in IBM Watson Health, Merative, and Marti Health. SK declares institutional grants or contracts from IBM Corporation for research related to health equity dashboard design. All other authors declare no competing interests.
References
- 1.Sze S, Pan D, Nevill CR, et al. Ethnicity and clinical outcomes in COVID-19: a systematic review and meta-analysis. eClinicalMedicine 2020; 29: 100630. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Magesh S, John D, Li WT, et al. Disparities in COVID-19 outcomes by race, ethnicity, and socioeconomic status: a systematic-review and meta-analysis. JAMA Netw Open 2021; 4: e2134147. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Gottlieb ER, Ziegler J, Morley K, Rush B, Celi LA. Assessment of racial and ethnic differences in oxygen supplementation among patients in the intensive care unit. JAMA Intern Med 2022; 182: 849–58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Wong AI, Charpignon M, Kim H, et al. Analysis of discrepancies between pulse oximetry and arterial oxygen saturation measurements by race and ethnicity and association with organ dysfunction and mortality. JAMA Netw Open 2021; 4: e2131674. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Sjoding MW, Dickson RP, Iwashyna TJ, Gay SE, Valley TS. Racial bias in pulse oximetry measurement. N Engl J Med 2020; 383: 2477–78. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Cerdeña JP, Plaisime MV, Tsai J. From race-based to race-conscious medicine: how anti-racist uprisings call us to act. Lancet 2020; 396: 1125–28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Baciu A, Negussie Y, Geller A, et al. , eds. The state of health disparities in the United States. In: Communities in action: pathways to health equity. Washington, DC: National Academies Press, 2017. https://www.ncbi.nlm.nih.gov/books/NBK425844/ (accessed Nov 30, 2022). [Google Scholar]
- 8.Agency for Healthcare Research and Quality. 2021 national healthcare quality and disparities report. https://www.ahrq.gov/research/findings/nhqrdr/nhqdr21/index.html (accessed Nov 30, 2022). [PubMed]
- 9.Bhavani SV, Wiley Z, Verhoef PA, Coopersmith CM, Ofotokun I. Racial differences in detection of fever using temporal vs oral temperature measurements in hospitalized patients. JAMA 2022; 328: 885–86. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Hosseinpoor AR, Bergen N, Schlotheuber A, Boerma T. National health inequality monitoring: current challenges and opportunities. Glob Health Action 2018; 11 (suppl 1): 1392216. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Centers for Medicare & Medicaid Services. CMS framework for health equity. https://www.cms.gov/about-cms/agency-information/omh/health-equity-programs/cms-framework-for-health-equity (accessed Jan 9, 2023).
- 12.Becker Epstein & Green. Biden administration prioritizes health equity in proposed reforms to Medicare managed care. https://www.ebglaw.com/insights/biden-administration-prioritizes-health-equity-in-proposed-reforms-to-medicare-managed-care/ (accessed March 26, 2023).
- 13.Hosseinpoor AR, Bergen N, Schlotheuber A. Promoting health equity: WHO health inequality monitoring at global and national levels. Glob Health Action 2015; 8: 29034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Hosseinpoor AR, Bergen N, Magar V. Monitoring inequality: an emerging priority for health post-2015. Bull World Health Organ 2015; 93: 591–591A. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Shahian DM, Torchiana DF, Engelman DT, et al. Mandatory public reporting of cardiac surgery outcomes: the 2003 to 2014 Massachusetts experience. J Thorac Cardiovasc Surg 2019; 158: 110–24.e9. [DOI] [PubMed] [Google Scholar]
- 16.Eberly LA, Richterman A, Beckett AG, et al. Identification of racial inequities in access to specialized inpatient heart failure care at an academic medical center. Circ Heart Fail 2019; 12: e006214. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Stelfox HT, Palmisani S, Scurlock C, Orav EJ, Bates DW. The “To Err is Human” report and the patient safety literature. Qual Saf Health Care 2006; 15: 174–78. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Institute of Medicine, Committee on Quality of Health Care in America. To Err Is Human: building a safer health system. Washington, DC: National Academies Press, 2000. [PubMed] [Google Scholar]
- 19.Bates DW, Levine D, Syrowatka A, et al. The potential of artificial intelligence to improve patient safety: a scoping review. NPJ Digit Med 2021; 4: 54. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Gianfrancesco MA, Tamang S, Yazdany J, Schmajuk G. Potential biases in machine learning algorithms using electronic health record data. JAMA Intern Med 2018; 178: 1544–47. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Gichoya JW, Banerjee I, Bhimireddy AR, et al. AI recognition of patient race in medical imaging: a modelling study. Lancet Digit Health 2022; 4: e406–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Celi LA, Cellini J, Charpignon ML, et al. Sources of bias in artificial intelligence that perpetuate healthcare disparities—a global review. PLoS Digit Health 2022; 1: e0000022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Shachar C, Gerke S. Prevention of bias and discrimination in clinical practice algorithms. JAMA 2023; 329: 283–84. [DOI] [PubMed] [Google Scholar]
- 24.Goodman KE, Morgan DJ, Hoffmann DE. Clinical algorithms, antidiscrimination laws, and medical device regulation. JAMA 2023; 329: 285–86. [DOI] [PubMed] [Google Scholar]
- 25.Centers for Medicare & Medicaid Services. Nondiscrimination in health programs and activities. Fed Regist 2022; 87: 47824–920. [Google Scholar]
- 26.Sheikh A, Anderson M, Albala S, et al. Health information technology and digital innovation for national learning health and care systems. Lancet Digit Health 2021; 3: e383–96. [DOI] [PubMed] [Google Scholar]
- 27.Panch T, Mattie H, Celi LA. The “inconvenient truth” about AI in healthcare. NPJ Digit Med 2019; 2: 77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Ross C. In new guidance, FDA says AI tools to warn of sepsis should be regulated as devices. https://www.statnews.com/2022/09/27/health-fda-artificial-intelligence-guidance-sepsis/ (accessed Nov 30, 2022).
- 29.Randell R, Alvarado N, McVey L, et al. Requirements for a quality dashboard: lessons from National Clinical Audits. AMIA Annu Symp Proc 2020; 2019: 735–44. [PMC free article] [PubMed] [Google Scholar]
- 30.Dowding D, Randell R, Gardner P, et al. Dashboards for improving patient care: review of the literature. Int J Med Inform 2015; 84: 87–100. [DOI] [PubMed] [Google Scholar]
- 31.Kuznetsova M, Frits ML, Dulgarian S, et al. An analysis of the structure and content of dashboards used to monitor patient safety in the inpatient setting. JAMIA Open 2021; 4: ooab096. [DOI] [PMC free article] [PubMed]
- 32.Kroch E, Vaughn T, Koepke M, et al. Hospital boards and quality dashboards. J Patient Safety 2006; 2: 10–19 [Google Scholar]
- 33.Agency for Healthcare Research and Quality. NPSD dashboards. https://www.ahrq.gov/npsd/data/dashboard/index.html (accessed Nov 30, 2022).
- 34.Ramos R, Davis JL, Ross T, Grant CG, Green BL. Measuring health disparities and health inequities: do you have REGAL data? Qual Manag Health Care 2012; 21: 176–87. [DOI] [PubMed] [Google Scholar]
- 35.Connolly M, Selling MK, Cook S, Williams JS Jr, Chin MH, Umscheid CA. Development, implementation, and use of an “equity lens” integrated into an institutional quality scorecard. J Am Med Inform Assoc 2021; 28: 1785–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Penaia CS, Morey BN, Thomas KB, et al. Disparities in Native Hawaiian and Pacific Islander COVID-19 mortality: a community-driven data response. Am J Public Health 2021; 111: S49–52. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Chang R. NHPI COVID-19 Data Policy Lab Dashboard. https://public.tableau.com/views/NHPI_CDPL_Dashboard_with_extract/NHPICDPLDashboard/ (accessed Feb 1, 2023).
- 38.Tsuchida RE, Haggins AN, Perry M, et al. Developing an electronic health record-derived health equity dashboard to improve learner access to data and metrics. AEM Educ Train 2021; 5 (suppl 1): S116–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Marvel SW, House JS, Wheeler M, et al. The COVID-19 Pandemic Vulnerability Index (PVI) Dashboard: monitoring county-level vulnerability using visualization, statistical modeling, and machine learning. Environ Health Persp 2021; 219: EHP8690. [DOI] [PMC free article] [PubMed]
- 40.National Institute of Environmental Health Sciences. COVID-19 Pandemic Vulnerability Index (PVI). https://covid19pvi.niehs.nih.gov/ (accessed Feb 1, 2023).
- 41.Jonnalagadda P, Swoboda C, Singh P, et al. Developing dashboards to address children’s health disparities in Ohio. Appl Clin Inform 2022; 13: 100–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Ohio State University. Ohio Children’s Opportunity Index. https://hpdw.osu.edu/t/COI/views/2020-07-23-4PME_16111908392390/OCOI (accessed Feb 1, 2023).
- 43.Bilal U, McCulley E, Li R, et al. Tracking COVID-19 inequities across jurisdictions represented in the Big Cities Health Coalition (BCHC): the COVID-19 Health Inequities in BCHC Cities Dashboard. Am J Public Health 2022; 112: 904–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Big Cities Health Coalition. COVID-19 Health Inequities in Cities. https://www.covid-inequities.info/ (accessed Feb 1, 2023).
- 45.Sullivan PS, Woodyatt CR, Kouzouian O, et al. America’s HIV epidemic analysis dashboard: protocol for a data resource to support ending the HIV epidemic in the United States. JMIR Public Health Surveill 2022; 8: e33522. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Ending the HIV Epidemic. America’s HIV Epidemic Analysis Dashboard. https://ahead.hiv.gov/ (accessed Feb 1, 2023).
- 47.Patrick R, Greenberg A, Magnus M, Opoku J, Kharfen M, Kuo I. Development of an HIV testing dashboard to complement the HIV care continuum among MSM, PWID, and heterosexuals in Washington, DC, 2007–2015. J Acquir Immune Defic Syndr 2017; 75 (suppl 3): S397–407. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Hester G, Nickel AJ, Griffin KH. Accountability through measurement: using a dashboard to address pediatric health disparities. Pediatrics 2020; 146: e2020024448. [DOI] [PubMed] [Google Scholar]
- 49.Hutchinson-Colas JA, Balica A, Chervenak FA, et al. New Jersey Maternal Mortality Dashboard: an interactive social-determinants-of-health tool. J Perinat Med 2022; 51: 188–96. [DOI] [PubMed] [Google Scholar]
- 50.Johnson & Johnson Women’s Health Group, J&J Advanced Analytics. New Jersey Maternal Mortality Dashboard. https://johnsonandjohnson.shinyapps.io/njmm/ (accessed Feb 1, 2023).
- 51.Martino SC, Mathews M, Beckett MK, et al. Development of a Medicare plan dashboard to promote health equity. Am J Manag Care 2023; 29: e91–95. [DOI] [PubMed] [Google Scholar]
- 52.Spoer BR, McCulley E, Lampe TM, et al. Validation of a neighborhood-level COVID Local Risk Index in 47 large U.S. cities. Health Place 2022; 76: 102814. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.NYU Langone Health. City Health Dashboard. http://www.cityhealthdashboard.com/ (accessed Feb 1, 2023).
- 54.Le P, Casper M, Vaughan AS. A dynamic visualization tool of local trends in heart disease and stroke mortality in the United States. Prev Chronic Dis 2022; 19: E57. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Centers for Disease Control and Prevention. Local Trends in Heart Disease and Stroke Mortality Dashboard. https://www.cdc.gov/dhdsp/maps/hd-stroke-mortality-dashboard.htm (accessed Feb 1, 2023).
- 56.Backonja U, Park S, Kurre A, et al. Supporting rural public health practice to address local-level social determinants of health across Northwest states: development of an interactive visualization dashboard. J Biomed Inform 2022; 129: 104051. [DOI] [PubMed] [Google Scholar]
- 57.SHARE-NW. Solutions in Health Analytics for Rural Equity across the Northwest: data dashboards. https://sharenw.nwcphp.org/ (accessed Feb 1, 2023).
- 58.Ising A, Waller A, Frerichs L. Evaluation of an emergency department visit data mental health dashboard. J Public Health Manag Pract 2023; 29: 369–76. [DOI] [PubMed] [Google Scholar]
- 59.Hartford EA, Thomas AA, Kerwin O, et al. Toward improving patient equity in a pediatric emergency department: a framework for implementation. Ann Emerg Med 2023; 81: 385–92. [DOI] [PubMed] [Google Scholar]
- 60.Riegels NS, Mazotti LA. Quality and health equity dashboards for internal medicine residents: interactive displays to promote systems-based practice and practice-based learning and improvement. Perm J 2023; 27: 139–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Arsenault C, Harper S, Nandi A, Rodríguez JMM, Hansen PM, Johri M. An equity dashboard to monitor vaccination coverage. Bull World Health Organ 2017; 95: 128–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Sanhueza A, Carvajal L, Cueva DA, et al. The Every Woman Every Child initiative: supporting countries in Latin America and the Caribbean to reduce social inequalities in health. Int J Equity Health 2022; 21: 83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Todas las Mujeres Todos los Niños. Country profiles with disaggregated data from EWEC-LAC indicators. https://www.everywomaneverychild-lac.org/medicion-de-desigualdades/perfiles-de-paises-con-datos-desagregados-de-los-indicadores-ewec-lac-2/ (accessed Feb 1, 2023).
- 64.Verhagen MD, Brazel DM, Dowd JB, Kashnitsky I, Mills MC. Forecasting spatial, socioeconomic and demographic variation in COVID-19 health care demand in England and Wales. BMC Med 2020; 18: 203. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Goulart CM, Purewal A, Nakhuda H, et al. Tools for measuring gender equality and women’s empowerment (GEWE) indicators in humanitarian settings. Confl Health 2021; 15: 39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Nakhuda H. Measurement of Sexual and Reproductive Health & Rights, Gender Equality, and Women’s Empowerment in Humanitarian Settings. https://public.tableau.com/app/profile/humairanakhuda/viz/SRHRGEWEScopingReviewStory_Final_Nov16/SRHRGEWEStory (accessed Feb 1, 2023).
- 67.Allan M, Lièvre M, Laurenson-Schafer H, et al. The World Health Organization COVID-19 surveillance database. Int J Equity Health 2022; 21 (suppl 3): 167. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.WHO. WHO COVID-19 Detailed Surveillance Data Dashboard. https://app.powerbi.com/view?r=eyJrIjoiYWRiZWVkNWUtNmM0Ni00MDAwLTljYWMtN2EwNTM3YjQzYmRmIiwidCI6ImY2MTBjMGI3LWJkMjQtNGIzOS04MTBiLTNkYzI4MGFmYjU5MCIsImMiOjh9 (accessed Feb 1, 2023).
- 69.Wimba PM, Bazeboso JA, Katchunga PB, et al. A dashboard for monitoring preventive measures in response to COVID-19 outbreak in the Democratic Republic of Congo. Trop Med Health 2020; 48: 74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Ibrahim H, Liu X, Zariffa N, Morris AD, Denniston AK. Health data poverty: an assailable barrier to equitable digital health care. Lancet Digital Health 2021; 3: e260–65. [DOI] [PubMed] [Google Scholar]
- 71.Orach CG. Health equity: challenges in low income countries. Afr Health Sci 2009; 9 (suppl 2): S49–51. [PMC free article] [PubMed] [Google Scholar]
- 72.Baah FO, Teitelman AM, Riegel B. Marginalization: conceptualizing patient vulnerabilities in the framework of social determinants of health—an integrative review. Nurs Inq 2019; 26: e12268. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 73.Yi SS, Kwon SC, Suss R, et al. The mutually reinforcing cycle of poor data quality and racialized stereotypes that shapes Asian American health. Health Aff 2022; 41: 296–303. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.Senathirajah M, Dankwa-Mullan I, Pickens G, Benevent R, Spurlock B. A hospital social needs index would help hospitals collaborate to address social needs and health equity. Aug 2, 2022. https://www.healthaffairs.org/content/forefront/hospital-social-needs-index-would-help-hospitals-collaborate-address-social-needs-and (accessed Nov 30, 2022).
- 75.Centers for Medicare & Medicaid Services. Medicare Program; contract year 2024 policy and technical changes to the Medicare Advantage program, Medicare Prescription Drug Benefit Program, Medicare Cost Plan Program, Medicare Parts A, B, C, and D overpayment provisions of the Affordable Care Act and Programs of All-Inclusive Care for the Elderly; health information technology standards and implementation specifications. Dec 27, 2022. https://www.federalregister.gov/documents/2022/12/27/2022-26956/medicare-program-contract-year-2024-policy-and-technical-changes-to-the-medicare-advantage-program (accessed March 30, 2023).
- 76.The Joint Commission. The Joint Commission elevates health care equity standard to National Patient Safety Goal. Jan 10, 2023. https://www.jointcommission.org/resources/news-and-multimedia/news/2023/01/the-joint-commission-elevates-health-care-equity-standard-to-national-patient-safety-goal/ (accessed Sept 6, 2023).
- 77.The Joint Commission. Health care equity. https://www.jointcommission.org/our-priorities/health-care-equity/ (accessed Jan 9, 2023).
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.