A scoping review of fair machine learning techniques when using real-world data

Abstract

Objective:

The integration of artificial intelligence (AI) and machine learning (ML) in health care to aid clinical decisions is widespread. However, as AI and ML take important roles in health care, there are concerns about AI and ML associated fairness and bias. That is, an AI tool may have a disparate impact, with its benefits and drawbacks unevenly distributed across societal strata and subpopulations, potentially exacerbating existing health inequities. Thus, the objectives of this scoping review were to summarize existing literature and identify gaps in the topic of tackling algorithmic bias and optimizing fairness in AI/ML models using real-world data (RWD) in health care domains.

Methods:

We conducted a thorough review of techniques for assessing and optimizing AI/ML model fairness in health care when using RWD in health care domains. The focus lies on appraising different quantification metrics for accessing fairness, publically accessible datasets for ML fairness research, and bias mitigation approaches.

Results:

We identified 11 papers that are focused on optimizing model fairness in health care applications. The current research on mitigating bias issues in RWD is limited, both in terms of disease variety and health care applications, as well as the accessibility of public datasets for ML fairness research. Existing studies often indicate positive outcomes when using pre-processing techniques to address algorithmic bias. There remain unresolved questions within the field that require further research, which includes pinpointing the root causes of bias in ML models, broadening fairness research in AI/ML with the use of RWD and exploring its implications in healthcare settings, and evaluating and addressing bias in multi-modal data.

Conclusion:

This paper provides useful reference material and insights to researchers regarding AI/ML fairness in real-world health care data and reveals the gaps in the field. Fair AI/ML in health care is a burgeoning field that requires a heightened research focus to cover diverse applications and different types of RWD.

Introduction

Recent advancements in artificial intelligence (AI) methods, especially machine learning (ML) algorithms, have led to success in integration to healthcare applications, demonstrating promising performance in various problems, such as risk prediction^1-3, disease diagnosis^4-8, and disease subphenotyping^9,10. On the other hand, the rapid adoption of electronic health record (EHR) systems has made large collections of real-world data (RWD)^11,12 available for research. There is an increasing interest in using these RWD, such as EHRs, administrative claims, and billing data, that are collected through routine care, to generate real-world evidence (RWE). Regulatory agencies such as the U.S. Food and Drug Administration (FDA) have started to issue guidance on how they consider RWD and RWE to support their regulatory decision-making¹³ and has indeed approved the new use of transplant drug based on observational RWD.¹⁴ The utilization of AI/ML-based methods, combined with readily available RWD, yields numerous benefits, e.g., more accurate disease and outcome prediction^15,16 and efficient disease risk factors identification^17,18, by effectively learning and analyzing patterns in a large collection of real-RWD¹⁹. However, despite the advantages of AI/ML techniques, there is a growing concern that such techniques may lead to unintended consequences, such as making biased decisions towards socioeconomically disadvantaged groups—leading to algorithmic fairness or bias issues.²⁰

Algorithmic fairness is a relatively new research field in AI/ML but has rapidly gained increasing attention in recent years. A classic example of algorithmic bias comes from the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS), a tool used by courts in the USA to identify the risk of a person recommitting another crime. The COMPAS has higher positive rates for African-American offenders than Caucasian offenders, often falsely predicting them to have a higher risk of recommitting a crime or recidivism.²¹ Similar examples have been found in the health care domain, such as the health care cost predictions are biased in black and white groups²² and the presence of atherosclerotic disease being different across racial-ethnic groups²³, the performance disparity across racial-ethnic and socioeconomic subgroups in ML models that predict heart failure outcomes²⁴, and the ML-models producing unequal predictions in postpartum depression and postpartum mental health service utilization between White and African-American women²⁵. The biased outputs of the ML models often stem from the inherent hidden biases in the data that these algorithms are trained on, reflecting the existing disparities in the society that generated these data²⁶, and leading to exacerbating health equity issues, which negatively affects the utility, trustworthiness, and ethic use of ML models.

In recent years, the AI/ML research community has proposed a series of fairness assessments and subsequent mitigation techniques, which have been quickly adopted in solving various algorithmic bias issues in different applications. In the health and biomedical domain, there also exists some research investigating fairness issues when using AI/ML methods. In 2018, Gianfrancescogian et al²⁷ discussed the potential bias in ML models using EHRs, and pointed out that the sources of bias in EHRs can be from missing data and patients missed by identification algorithms, sample size and potential underestimation, and misclassification or measurement error. There is a larger concern that biases or deficiencies in the EHR data used to train the ML algorithms and subsequently used in clinical decision support systems may contribute to disparities in health care.²⁷ In 2020, Fletcher et al²⁸ presented a few case studies and put forth a few important considerations on bias and fairness related to applying ML models in the context of global health, especially for low- and middle- income countries. In 2021, Mehrabi et al²⁶ summarized how biases in data can skew what is learned by ML methods and comprehensively went over the definitions of fairness and bias that have been proposed in the literature. Moreover, they organized and established a taxonomy (pre-processing, in-processing, and post-processing) detailing the efforts undertaken to tackle or mitigate issues related to ML fairness across different domains. In 2022, Xu et al²⁰ provided a scoping review about fair ML evaluation metrics in computational medicine, while Wan et al (2023)²⁹ focused on the general modeling design for fair ML and went over the state-of-the-art in-processing techniques for improving ML fairness. Nevertheless, there has not been a comprehensive review that systematically summarized existing literature on not only ML fairness assessment but also the bias mitigation strategies when ML techniques are applied to RWD in the health care domain. Therefore, in this paper, we aimed to fill this important gap and provide an overview of the existing literature focused on assessing fairness, identifying potential sources of bias, and bias mitigating techniques on ML models using RWD.

Problem or Issue	There are no comprehensive reviews that systematically summarize existing literature on not only machine learning (ML) fairness assessment but also the fairness optimization techniques when ML models are applied to real-world data (RWD) in the health care domain.
What is Already Known	The use of artificial intelligence (AI) and ML in health care to support clinical decisions is becoming more common. However, as these technologies play a growing role in medical settings, there are rising concerns about their potential bias and fairness. That is, an AI tool may have a disparate impact, with its benefits and drawbacks unevenly distributed across societal strata and subpopulations, potentially exacerbating existing health inequities.
What this Paper Adds	This review aims to explore the current literature and pinpoint areas of need concerning addressing bias and ensuring fairness in AI/ML models that use RWD in health care domains. This article offers valuable perspectives for researchers on the fairness of AI/ML in health care, highlighting areas that need further exploration.

Methods

Search strategy and selection criteria

The review process is divided into two rounds: the evaluation of existing relevant review articles and the assessment of individual studies. Figure 1 summarizes the overall search and screening process. We first identified review articles to study state-of-the-art progress of fairness mitigation techniques, and then we collected and analyzed individual studies that focus on applying fair ML techniques in health care applications. In both phases, we adhered to the same methodology, which encompasses conducting a comprehensive literature search, reviewing abstracts and full texts, and extracting data from the selected articles. To analyze and study the latest cutting-edge fair ML techniques as presented in review articles, we collected peer-reviewed publications from mainstream computer science databases, including IEEE Xplore (n = 92), ACM Digital Library (n = 78), and Web of Science (n = 249), as state-of-the-art ML fairness assessment and bias mitigation strategies are mostly developed by computer scientists, using the search query: ("survey" OR "review" OR "overview") AND ("unfairness" OR "fairness") AND ("machine learning" OR "deep learning" OR "algorithm" OR "clustering" OR "artificial intelligence") with filters limiting to studies published in the last decade. We then excluded the papers that (1) were not review articles, and (2) were not related to fairness in machine learning. All papers underwent a 2-person review process for inclusion in the manuscript.

Figure 1.

The overall search and screening process.

In the second round, we extracted individual studies of fair ML in health care domain focusing on those using RWD, identified through PubMed (n=161) and Embase (n=146). Our initial inclusion criteria were studies that are (1) peer-review original research (excluding pre-prints), (2) having full-text available, (2) published within the last 10 years (encompassing years 2012–2022), (3) published in English, and (4) on the topic of mitigating bias issues in ML models using RWD. We excluded (1) articles that are not in English, (2) commentaries or editorials, (3) studies that did not use RWD, (4) studies that did not use bias mitigation methods, or (5) studies are not in the health care domain. We followed the definition of RWD used by the FDA¹³ but focused on data generated from routine health care including EHRs, claims and billing data, and product and disease registries, but not including data collected in interventional, controlled, experimental clinical research settings. RWD also has other patient-generated health data, such as those generated in home-use care settings and data from mobile devices that can inform health status. We excluded studies using data that are generated from personal devices, such as smartphones and activity trackers, where different ML models are needed for these patient-generated health data.

Results

As depicted in Figure 1, we identified 35 review articles discussing the topic of fair ML along with the fairness assessment and bias mitigation approaches. Table 1 provides an overview of the array of reported algorithmic bias mitigation techniques. The search of individual studies yielded 11 distinct studies centered around the utilization of RWD for health care applications on the topic of fair ML.

Table 1.

Overview of the existing bias mitigation approaches.

Pre-processing	In-processing	Post-processing
Synthetic data generation Sensitive feature removal Sensitive feature replacement Reweighting Resampling Diverse data collection Adjusting label selection Disparate impact remover	Regularization (e.g., Prejudice Remover) Constraint optimization Adversaria learning without generated data Adversarial learning with generated data	Transformation Thresholding Constraint optimization Calibration Varying cut-point selection Reject option-based classification

Fairness assessment

An important first step in fair ML is to quantify the bias generated from various sources that affect the model. In this section, we summarized ten metrics that can be used to evaluate algorithmic fairness in the context of health care. The mathematical notations that are used in the formula of the metrics are listed in Table 2. We used a case study to illustrate these metrics, where the task was to build an ML model for predicting patient hospitalization risk using EHRs. The sensitive attribute was race i.e., Black (the protected group) v.s. White (the reference or privileged group), and the objective was to assess if the ML prediction model for hospitalization risk treated the two subgroups fairly.

Table 2.

Notations and symbols.

Symbol	Description
$G$	Sensitive attribute
$Y$	Actual outcome
$\widehat{Y}$	Predicted outcome

Overall accuracy equality

(a.k.a. delta accuracy) is the evaluation of whether both the protected and privileged groups exhibit an equal level of prediction accuracy. Prediction accuracy, in this context, pertains to the likelihood of correctly assigning individuals from either a positive or negative class to their respective categories. The formula of overall accuracy equality in our case is shown below. ³⁰

$$P(\widehat{Y}=Y,G=White)=P(\widehat{Y}=Y,G=Black)$$

Equality of opportunity

(a.k.a. equal opportunity or false negative rate [FNR] balance or delta true positive rate) assesses whether both groups have equal FNR. The FNR refers to the probability of a subject belonging to the positive class being incorrectly predicted as negative. Notably, a model with equal FNRs will have an equal true positive rate (TPR) as well.³¹

$$P(\widehat{Y}=0\mid Y=1,G=White)=P(\widehat{Y}=0\mid Y=1,G=Black)$$

Predictive parity

is the concept where both the protected and privileged groups have an equal positive predictive value (PPV), which represents the probability of a subject, identified as positive, actually belonging to the positive class. Notably, a model with equal PPVs will have an equal false discovery rate (FDR) as well. ³¹

$$P(\widehat{Y}=1\mid Y=1,G=White)=P(\widehat{Y}=1\mid Y=1,G=Black)$$

Predictive equality

also known as false positive rate (FPR) balance or delta false positive rate, refers to the condition where both the protected and privileged groups have an equal FPR. The FPR is the probability of a subject in a negative class being incorrectly predicted as positive. ^32,33

$$P(\widehat{Y}=1\mid Y=0,G=White)=P(\widehat{Y}=1\mid Y=0,G=Black)$$

Statistical or demographic parity

is one of the earliest definitions of fairness, which is defined as each subgroup having an equal probability to be classified with a positive label. ^30,33

$$P(\widehat{Y}=1\mid G=White)=P(\widehat{Y}=1\mid G=Black)$$

Disparate impact

examines the probability of subjects from different groups being classified with the positive label, which is similar to statistical/demographic parity. ³⁴

$$\frac{P(\widehat{Y}=1\mid G=White)}{P(\widehat{Y}=1\mid G=Black)}$$

Equalized Odds

also known as conditional procedure accuracy equality or disparate mistreatment, is a fairness metric that takes into account both TPR and FPR simultaneously. ³⁰

$$\begin{gathered} (P(Y=1\mid \widehat{Y}=1,G=White)=P(Y=1\mid \widehat{Y}=1,G=Black))\wedge (P(Y=0\mid \widehat{Y} \\ =1,G=White)=P(Y=0\mid \widehat{Y}=1,G=Black)) \end{gathered}$$

Intergroup standard deviation (IGSD)

is setup to understand the tradeoff between different metrics across multiple groups. Given a threshold $i$, metric $M$, and groups $A$, the ${\text{IGSD}}_{Mi}$ is defined as. ³⁵

$$IGSD(M,t_i)=\sqrt{\frac{{\sum }_{a=1}^A{(M_{ia}-\mu M_i)}^2}{A}},where\mu M_i=\frac{{\sum }_{a=1}^A{M}_{ia}}{A}$$

Conjunctive accuracy improvement (${\mathit{CAI}}_{\alpha }$)

is a combination of overall accuracy and accuracy gap (i.e., the difference in accuracy between subgroups) for measuring the success of mitigation algorithms. ³⁶

$$CA{I}_{\alpha }=\alpha ({acc}_{gap}^b-{acc}_{gap}^d)+(1-\alpha )(ac{c}^d-ac{c}^b)$$

where $\alpha$ is the weight coefficient to trade-off the accuracy and fairness metrics, and $ac{c}^b$ and ${acc}_{gap}^b$denote the accuracy and accuracy gap of the baseline model, and $ac{c}^d$ and ${acc}_{gap}^d$ represent the accuracy and gap metrics of the debiased model, respectively. This metric can be generalized to other metrics ($CAUC{I}_{\alpha }$), using AUC as the performance metric.

Generalized entropy index (GEI)

is to measure how unequally the outcomes of an algorithm benefit different individuals. ³⁷

$${ℰ}^{\alpha }(b_1,b_2,\dots ,b_n)=\frac{1}{n\alpha (\alpha -1)}\sum _{i=1}^n[{\left(\frac{b_i}{\mu }\right)}^{\alpha }-1]$$

where $b_i$ is the benefit function of an individual $i$ for presenting the predictive bias, and constant $\alpha \notin \{0,1\}$ is a weight parameter.

Algorithmic bias mitigation

The current review was focused on bias mitigation techniques on supervised ML models within the realm of health care. Readers who are interested in unsupervised learning algorithms can refer to this overview paper.³⁸ As shown in Figure 2, the existing studies covered diverse disease domains and clinical applications, such as attention deficit hyperactivity disorder (ADHD)³⁹, Alzheimer’s disease diagnosis⁴⁰, cardiovascular disease prediction^41,42, acute postoperative pain prediction⁴³, survival analysis and mortality prediction^24,44, postpartum depression (PPD) recognition²⁵, diabetic retinopathy analysis³⁶, and opioid misuse prediction ⁴⁵.

Figure 2.

Fair machine learning in health care.

MIMIC-III (Medical Information Mart for Intensive Care III), iSTAGING (Imaging-based coordinate SysTem for AGing and NeurodeGenerative diseases), GWTG-HF (Get With The Guidelines-Heart Failure), (NPD) National Pupil Database linked with the SLaM CAMHS (South London and Maudsley National Health Service Foundation Trust Child and Adolescent Mental Health Services)

*Non-public dataset

Among these studies, eight conducted experiments using publicly accessible RWD datasets (ten in total), and Figure 3 shows the breakdown of the data types. These datasets include widely used Medical Information Mart for Intensive Care III (MIMIC III)⁴⁶, Clalit Health Services from Israel⁴⁷, Get With The Guidelines-Heart Failure (GWTG-HF) registry dataset⁴⁸, IBM MarketScan Medicaid Database⁴⁹, , EyePACS⁵⁰, MIMIC-CXR⁵¹, CheXper⁵², Chest-Xray8⁵³, Imaging-based coordinate SysTem for AGing and NeurodeGenerative diseases (iSTAGING) consortium⁵⁴, and the National Pupil Database (NPD)⁵⁵ linked with theSouth London and Maudsley National Health Service Foundation Trust Child and Adolescent Mental Health Services (SLaM CAMHS)^56,57.

Figure 3.

Distributions of data types of the real-world datasets used for fair ML research in health care.

Current approaches of bias mitigation strategies for ML ²⁶ can be categorized into three groups (Table 1): Pre-processing, In-processing, and Post-processing. Based on our review, most studies (82%) used pre-processing methods to mitigate identified bias in clinical and biomedical applications.

Pre-processing

We identified nine papers that applied the pre-processing approaches for ML bias mitigation. Davoudi et al⁴³ investigated the issue of prediction bias in CatBoost models designed to forecast acute postoperative pain. They employed a pre-processing technique known as "reweighing" to modify the weight of protected attributes, aiming to diminish the algorithmic bias in the prediction models. However, they found that reweighing the prediction models based on each protected attribute helped reduce the bias (for the predictive equality from 1.75 to less than 1.25) for some cases (e.g., lowest area deprivation index [ADI] tertile vs middle and highest terlites), but it introduced bias in some other cases (English-speaking subgroup vs non-English speaking subgroup) where there was no bias (for the predictive equality from 0.84 to 0.29). Jeanselme et al⁴⁴ demonstrated the impact of missing data in observational datasets on algorithmic fairness in subsequent stages. They conducted experiments using the MIMIC-III dataset to predict short-term survival within 7 days after an observation period. By comparing different imputation strategies (Multiple Imputation by Chained Equations [MICE] and group MICE), they revealed that prevalent imputation practices could negatively affect health equity, disproportionately affecting marginalized groups. Li et al⁴² investigated various predictive models, including Pooled Cohort Risk Equation (PCE), logistic regression (LR), random forest (RF), gradient-boosting trees, and long short-term memory (LSTM), intended for cardiovascular risk assessment based on extensive EHRs from Vanderbilt University Medical Center. They also showcased the potential of pre-processing methods such as eliminating sensitive attributes and resampling to mitigate the algorithmic bias originating from predictive models. Li et al ²⁴ developed ML models to predict the probability of long-term hospitalization and in-hospital mortality for heart failure patients using admission data. They introduced a method to reduce models’ bias by integrating two sources of social determinants of health (SDoH), Social Deprivation Index (SDI)⁵⁸ and ADI⁵⁹, to build ML classifiers. Compared to the baseline (random forest [RF]) model, the model trained by using all SDoHs improves the demographic parity from 0.828 to 0.881 and equalized odds ratio from 0.826 to 0.863. Wang et al. ⁴⁰ used data from several magnetic resonance image (MRI) consortia for diagnosing Alzheimer's disease, schizophrenia, and autism spectrum disorder. The authors evaluated the fairness of their deep neural network across subjects from different genders (female, male), ages (various cutoffs for different diseases), racial groups (various categories for different diseases), and clinical studies (various trials). They found that using multisource data (using demographics, clinical variables, genetic factors, and cognitive scores in addition to MRI features) improves the average AUC and achieves unbiased predictions (e.g., the difference of the equality of opportunity from 0.159 to 0.043 for Alzheimer's disease diagnosis) across different subgroups of models for these three disorders.

Another work by Park et al²⁵ tackled racial bias in predicting PPD and mental health service utilization using ML models trained on observational data. They explored various bias-mitigating techniques, including reweighing, prejudice remover, and removing race from the models. Reweighing was identified as a viable approach, enhancing fairness (e.g., for predicting PPD development, disparate impact Improved from 0.31 to 0.79 and equality of opportunity improved from −0.19 to 0.02) without compromising accuracy (e.g., prediction accuracy from 0.73 to 0.72) across experimental conditions, unlike prejudice remover (prediction accuracy decreased from 0.73 to 0.68). Seyyed-Kalantari et al⁶⁰ evaluated cutting-edge deep neural network models for predicting diagnostic labels from X-ray images collected through routine care. Their experiments suggested that employing multi-source datasets for training could counteract bias during data collection. Ter-Minassian et al³⁹ investigated the prediction of Attention-Deficit Hyperactivity Disorder (ADHD) risk using ML and statistical methods with data from the NPD linked with SLaM CAMHS. To mitigate bias, they employed the pre-processing method of reweighing, which significantly improved disparate impact (from 0.15-0.60 to 1) without compromising the models' ADHD classification performance.

Post-processing

A total of two papers employed the post-processing approaches for ML bias mitigation. Barda et al⁴¹ evaluated two baseline models, the Pooled Cohort Equations (PCE) and the University of Sheffield fracture risk assessment tool (FRAX), on the prediction of primary cardiovascular disease, and found that the post-processing algorithm (i.e., multicalibration⁶¹) improves model fairness (e.g., the difference between the mean observed and the mean predicted risk improved from 1.25 to 1.04 in the PCE and from 1.08 to 1.00 in the FRAX model) in subpopulations defined by 5 protected variables: ethnicity, sex, age group, socioeconomic status, and immigration status. Thompson⁴⁵ assessed the fairness of a previously validated ML opioid misuse classifier based on the EHR data from the Rush University Medical Center. Post-hoc recalibration methods were conducted to eliminate bias in FNR (from 0.32 to 0.24 in the black group) with minimal changes in other subgroup (from 0.17 to 0.21 in the white group) error metrics.

In-processing

In-processing techniques were less commonly used compared to pre-processing and approaches in the existing health care literature, two were identified and only one paper shows the superiority of in-processing techniques. Paul et al³⁶ proposed a novel debiasing approach that uses generative models and adversarial learning, called training and representation alteration (TARA), to improve algorithmic fairness. The proposed TARA approach was tested on several healthcare-related tasks, including evaluation of the state of mental health and diabetic retinopathy analysis, and the results demonstrate the superiority of fairness mitigation of the TARA compared with the competing baselines (e.g., the gap of accuracy between reduced from 10.5 to 0.5 across different race groups on the diabetic retinopathy analysis task).

Discussion

The existing state of research in fair ML utilizing RWD has primarily concentrated on diverse applications and disease domains as demonstration use cases. These studies have certainly illustrated the potential of fair ML in tackling a significant concern in AI-based clinical decision support — AI/ML producing biased prediction and thus exacerbating existing disparities. Our review identified several health care applications in fair ML research using RWD. The existing studies have undoubtedly highlighted the transformative potential of fair ML in different health care applications, but the advancement of this field requires a diverse array of data sources that encompass a wide range of health conditions. However, it is important to acknowledge that the range of applications discovered through our scoping review remains limited. It is crucial to broaden our endeavors to include a wider array of diseases and health-related concerns, thereby maximizing the impact of fair ML in minimizing healthcare inequalities and enhancing disease outcomes.

Our review identified n=9, 2, and 2 studies that employed pre-processing, in-processing, and post-processing techniques for ML bias mitigation. While pre-processing techniques have been widely used, there is a lack of evidence demonstrating their superiority over post-processing or in-processing alternatives. Comparative evaluations of the three different classes of ML mitigation approaches are an important topic but yet to be studied. To advance our understanding of the most effective strategies in addressing algorithmic biases within healthcare domains, there exists a need for comprehensive studies that systematically assess the strengths and weaknesses of pre-processing, post-processing, and in-processing approaches. By comprehensively evaluating the trade-offs and performance nuances of these techniques, researchers can shed light on the most suitable interventions for specific contexts and bolster the foundation for a fairer and more robust ML landscape within health care.

While existing studies have made notable strides in mitigating algorithmic bias within single-modality data like structured EHRs and medical imaging data, the emergence of multi-modality data in health care research presents both opportunities and challenges. The essential task of untangling and mitigating bias across different modalities and understanding the contributions of each modality to potential biases becomes increasingly complex. Furthermore, the underexplored terrain of unstructured data, including clinical narratives and textual data, holds immense potential for comprehensively analyzing patient health status. Unfortunately, the current landscape lacks sufficient methods or studies that effectively evaluate and mitigate algorithmic bias within these advanced models, especially when dealing with complex language models like Large Language Models (LLMs), in the context of clinical notes and other unstructured data. Advancing fair ML into the realm of multi-modality and unstructured data is an important research direction for promoting trustworthy AI in supporting the quality of health care.

Applying bias mitigation methods for improving fair ML within the realm of clinical research using RWD presents a transformative shift towards more ethical, precise, and patient-centered health care practices. Development and innovation of approaches and techniques for improving ML fairness and building trustworthy AI are critically needed for multifaced aspects. First, fair ML can ensure unbiased clinical decision support are unbiased across diverse patient populations and social strata. In addition, fair ML can not only improve the equity of health care processes and outcomes but also foster trust among health care providers and patients, and their families.

Looking ahead, there are several promising future research directions in addressing ML fairness in clinical research using RWD. First, it is critical to develop a research framework and methods that can reveal the underlying causes of bias of an ML algorithm. Moreover, existing research on fair ML is limited to regular ML algorithms and a relatively narrow range of health care applications, expanding the fair ML research into more advanced AI/ML algorithms (e.g., autoML, deep learning) and broader health care applications (e.g., electrocardiogram waveform data). Additionally, given the increasing prominence of multi-modality data in health care research, there is a pressing need for technique innovation that can account for biases raised from interactions among different modalities. ⁶² Similarly, acknowledging the significance of unstructured data, such as clinical narratives, highlights the need for innovative fair ML methods designed specifically for these complex textual resources.

Our study had two major limitations. First, we excluded studies and theses not written in English. Second, we only reviewed the articles on the topic of fair ML using RWD in health care domains. Further investigation on how bias in RWD and analytic methods would affect study results should be systematically studied.

Conclusion

Fair ML can offer equitable, and impactful clinical decision-support tools to improve the quality of care and health outcomes. Our scoping review provides useful reference material and insights to researchers regarding AI/ML fairness in real-world health care data and reveals the gaps in the field. Fair AI/ML in health care is a burgeoning field that requires a heightened research focus to cover diverse applications and different types of RWD.