Adversarial Debiasing for Equitable and Fair Detection of Acute Coronary Syndrome using 12-Lead ECG

Abstract

Objective:

Acute coronary syndrome (ACS) is a life-threatening condition requiring accurate diagnosis for better outcomes. However, variability in signs and symptoms among racial subgroups could cause disparities in diagnostic accuracy. In this study, we use machine learning models to diagnose ACS, focusing on mitigating disparities and ensuring fairness between Black and non-Black populations.

Methods:

We built on a state-of-the-art random forest classifier to compare three mitigation strategies. The first two approaches involved resampling or partitioning the data prior to training, while the third approach proposed an innovative framework called adversarial debiasing. To evaluate our model performance, we used the receiver operating characteristic (ROC) curve and an operating point at 80% specificity for clinical importance.

Results:

After mitigation with adversarial debiasing, the difference in sensitivities between the two subgroups decreased from 9.8% to 1.3%. Specifically, this approach achieved areas under the ROC of 0.810 and 0.817, and sensitivities of 70.1% and 71.4%, respectively for Black and non-Black subgroups.

Conclusion:

The proposed adversarial debiasing model outperformed the other two methods in both diagnostic accuracy and effectiveness in minimizing disparities.

Significance:

We expect this framework to achieve fair diagnostic models across diverse demographic populations globally and be generalizable to other outcomes.

I. Introduction

Acute coronary syndrome (ACS) is a potentially life-threatening condition caused by interruption of blood flow to any branch of the coronary arteries, which are essential blood vessels that supply the heart muscle with oxygen-rich blood[1], [2]. Due to the acute nature of this condition, quick and accurate diagnosis is crucial for tailoring appropriate interventions[3], [4]. A 12-lead electrocardiogram (ECG), generated by electrodes systematically placed at predetermined locations on the surface of the body, records the electrical activity of the heart and is currently used as the initial diagnostic tool for patients with acute chest pain[5].

However, interpretation of ECGs, especially subtle changes, can have a high degree of variability even among clinical experts[6]. Given the complexity of interpreting biological signals, machine learning models can be beneficial in detecting underlying patterns and providing rapid, accurate diagnoses[7]–[9]. These models can be trained on diverse datasets to recognize subtle patterns in patient data that might be overlooked by clinical experts. However, machine learning models may be susceptible to biases present in the collected data. These biases could stem from imbalances in the dataset and underrepresentation of certain demographic groups. Conversely, even if the number of samples representing each demographic group is balanced, the disease prevalence may still be unequal, which could lead to the model being exposed to more positive cases in one demographic subgroup, resulting in skewed outcomes[10].

In addition, differences in symptom presentation across demographic groups, such as sex or race, can lead to mis-interpretation or under-recognition of the disease, hindering diagnosis[11]. Specifically, research has shown that there are disparities in cardiovascular risk factors, health metrics and ECG patterns between non-Hispanic Black patients and other ethnic groups[12], [13]. For instance, healthy Black adults often have a higher QRS complex voltage and a pattern of benign anterior ST segment elevation[14], [15]. Such patterns could falsely suggest ST-elevation myocardial infarction (STEMI), leading to diagnostic disparities. Likewise, geographic bias and socioeconomic variations in access to advanced diagnostic tools and healthcare resources across regions can further exaggerate disparities in diagnostic outcomes[16]. These biases could lead to unfair decision making and diagnosis, and eventually contribute to longer delays to treatment and worse outcomes for patients who are Black[17].

Therefore, in healthcare applications such as ACS diagnosis, the presence of disparities is particularly problematic, as it undermines the goal of providing equitable and effective care to all patients. Ensuring fairness and accuracy in machine learning enhanced diagnostic tools is crucial to avoid perpetuating existing healthcare inequities and to improve outcomes across diverse populations.

In this study, we employ machine learning models to analyze ECG data for ACS diagnosis, while rigorously addressing disparities in the model and outcomes. We utilize a comprehensive ECG dataset with real-world clinical data and apply random forest and multilayer perceptron models to ensure robust and fair model performance across the two racial groups. By incorporating these methods, our aim is to develop an unbiased diagnostic tool to be used in clinical settings that can improve ACS diagnosis for all patients, regardless of their demographic background.

II. Methods

A. Study Design and Dataset

The 12-lead ECG data used in this research were collected from the University of Pittsburgh Medical Center (2013–2019) and the University of North Carolina at Chapel Hill (2019–2022)[18]–[20]. The combined dataset is representative of pre-hospital and in-hospital ECGs collected in the United States. Out of the 7,294 ECGs, each representing an individual patient, 514 were missing race information and were excluded from the dataset. Among the remaining samples, 3,198 (47.2%) patients identified as Black (49.9% Female) and 3,582 (52.8%) patients identified as non-Black (42.0% Female), in which all ethnicities that were not Black were grouped together. The race variable was derived from self-reported race in the electronic health record. The ACS outcome (myocardial infarction or unstable angina) was adjudicated by independent physician reviewers as previously described in detail[7], [19], [21]. ACS was defined as per the 4th University Definition of Myocardial Infarction as having ischemic symptoms (e.g., chest, jaw, or upper extremity discomfort lasting > 20 minutes) and at least one of the three criteria: (1) elevated cardiac troponin I (> 99th percentile), (2) imaging evidence of new myocardial injury, or (3) > 70% coronary artery stenosis on angiography or nuclear imaging. Patients discharged from the emergency department without 30-day cardiac adverse events were classified as no outcome. To make these adjudications, the two independent physician reviewers were provided full access to patients’ complete hospitalization records, including ECGs, clinical notes, laboratory results, imaging studies, and cardiac catheterization reports. Reviewers made their determinations independently of ICD codes. Discrepancies between reviewers were resolved by a third adjudicator. Our adjudication strategy was previously published on ClinicalTrials.Gov (NCT04237688).

ACS was observed in 385 patients who are Black and 679 patients who are non-Black, resulting in an ACS prevalence of 12.0% and 19.0% in the two subgroups, respectively. Table 1 illustrates the patient demographics and ACS prevalence for the whole dataset.

TABLE I:

Comparison of Black and Non-Black Subgroups

	Black subgroup (n = 3198)	Non-Black subgroup (n = 3582)
Age	57 ± 15 (19 – 98)	62 ± 16 (18 – 102)
Sex
Female	1597 (49.9%)	1506 (42.0%)
Male	1601 (50.1%)	2076 (58.0%)
ACS Cases	385 (12.0%)	679 (19.0%)
PMH
Hypertension	2296 (71.8%)	2280 (63.7%)
Diabetes	976 (30.5%)	700 (19.5%)
Past or current smoker	1537 (48.0%)	858 (24.0%)
Dyslipidemia	1223 (38.2%)	1551 (43.4%)
Angina	296 (9.3%)	383 (10.7%)
Known CAD	901 (28.2%)	1262 (35.2%)
Prior MI	770 (24.1%)	966 (27.0%)
Prior PCI	378 (11.8%)	597 (16.7%)
Prior CABG	312 (9.8%)	469 (13.1%)

Values are mean ± s.d. (minimum–maximum) or n (%). PMH: past medical history; CAD: coronary artery disease; MI: myocardial infarction; PCI: percutaneous coronary intervention; CABG, coronary artery bypass graft.

This study complies with all ethical regulations and was approved by the institutional review board of the University of Pittsburgh. The study was registered at https://www.clinicaltrials.gov/ (identifier NCT04237688).

B. ECG Processing and Feature Extraction

The dataset consisted of 10-second 12-lead ECGs recorded at 500 samples per second. Individual leads were isolated from the raw ECG signals through decompression, with noise, artifacts, and ectopic beats removed. A median beat was computed for each lead, from which key temporal features were extracted including the amplitude, duration, and area of important waveforms (e.g., P wave, QRS complex)[21]. Additional measures included spatial angles between waveforms and ratios of principal component analysis eigenvalues across different intervals, resulting in a total of 415 temporal and spatial features[7]. The final set of features were standardized. The same preprocessing techniques were uniformly applied across the entire population, ensuring consistent data quality regardless of demographic differences.

C. Fairness Metrics

Demographic parity and equalized odds are two commonly used fairness metrics for evaluating model fairness. Satisfying these fairness metrics requires the model to make decisions independently of the demographic group associated with the true label[22]. These metrics can be described mathematically with the following equations:

Demographic Parity

$$P(\widehat{Y}=1\mid A=0)=P(\widehat{Y}=1\mid A=1)$$ (1)

Equalized Odds

$$P(\widehat{Y}=1\mid Y=1,A=a)=P(\widehat{Y}=1\mid Y=1,A=b)$$ (2)

$$P(\widehat{Y}=1\mid Y=0,A=a)=P(\widehat{Y}=1\mid Y=0,A=b)$$ (3)

Here, $\widehat{Y}$ is the predicted ACS outcome, $Y$ is the true outcome, and $A$ is the protected attribute of the dataset (e.g., race)[23]. Demographic parity ensures equal selection rates across demographic groups but does not account for fairness in errors. In contrast, equalized odds asserts that both true positive rates and false positive rates should be equal across subgroups. In clinical contexts such as the present study, the equalized odds metric is more appropriate than demographic parity, as the latter is more sensitive to differences in disease prevalence across subgroups. Therefore, we use equalized odds as our primary fairness metric, and report demographic parity results as a secondary reference. The receiver operating characteristic (ROC) curve plots true positive rate against false positive rate of a given model at varying threshold levels. The area under the curve (AUC) is therefore a single performance metric that considers both true and false positive rates. Consequently, aiming for comparable ROC curves and AUCs across subgroups aligns with the goals of enforcing equalized odds[24]. These metrics could be computed separately for the two subgroups, and any differences between them would indicate the presence of disparities. In other words, we could conclude that the model performs similarly across the two subgroups, and equalized odds are achieved only if their ROC curves are comparable.

While comparing AUC values and visually inspecting ROC curves provide useful insights on fairness and disparities, these methods alone were inadequate for a comprehensive evaluation. We thus proposed to use a statistical measure to further assess the difference between the subgroups’ AUCs[25]. Since the AUC is only a single value, and most statistical methods rely on analyzing the distributions of a certain variable, we used bootstrapping to obtain several samples of AUCs, with each value corresponding to a sampled dataset. This was conducted for both demographic subgroups, resulting in two distributions of AUCs to be compared. The Wilcoxon rank-sum test was chosen as the most appropriate statistical measure because it is a non-parametric two-sample test that does not assume normal distributions. This choice was necessary since our aim was to compare two groups of AUCs after bootstrapping, which were independent of each other and might not always follow a normal distribution. The test then computes a p-value, for which any p < 0.05 concludes the existence of racial disparities. Additionally, to ensure that the model was able to make meaningful predictions on both subgroups with clinical diagnostic importance, we compared the sensitivity metric at 80% specificity on the ROC curve between the two subgroups. This metric further enforces equality of opportunity for treatment, and along with the AUC and the results of the Wilcoxon rank sum test, provides meaningful conclusions to the presence and degree of racial disparities.

To assess clinical fairness more quantitatively and comprehensively, we used Fairlearn, an open-source Python package designed for measuring bias in machine learning models. Fairlearn allowed us to evaluate fairness at a specific operating point (80% specificity in this case) using the two fairness metrics aforementioned. The fairness metric was computed as a ratio, where values closer to 1 indicate minimal disparities between subgroups, providing a standardized way to assess model fairness.

Overall, the ROC curves, statistical tests, and the equalized odds and demographic parity scores computed using Fairlearn provide a comprehensive assessment of disparities and the effectiveness of mitigation in our algorithms.

D. Baseline Random Forest Model

We have previously demonstrated that a state-of-the-art random forest classifier can provide high accuracy and precision in diagnosing ACS[7]; we therefore employed a random forest model as our baseline classifier. Prior to training, we conducted feature selection to identify the most relevant features from the full feature-set that contributed to the target variable and were thus retained. Specifically, a combination of statistical tests and feature selection techniques were used to evaluate the predictive power of each feature in the training set, after which the median ranks of the features were calculated to summarize their overall significance[21]. All demographic-related features were also excluded from the training and testing sets for a completely blinded and ethical model. During the training phase, the training subset was further split into ten equal-sized folds for cross validation. This model’s performance acted as a benchmark for any mitigation strategies proposed. The specific parameters used in the model are provided in the Appendix A.

E. Mitigation Strategies

To address potential biases in our baseline classifier, we implemented mitigation strategies designed to enhance fairness and reduce disparities across different demographic groups. These strategies were designed to achieve equalized odds and equality of opportunity among the racial subgroups of the dataset.

The first approach was a statistical and methodological strategy that balances the ACS prevalence among all subgroups, such that the prevalence of ACS in the Black subgroup equals that of the non-Black subgroup. The original dataset had a smaller percentage of ACS cases in the Black subgroup (Table 1). Such balancing was only applied to the training set, as we wanted the testing set to be representative of the general population. The ACS prevalence could be balanced in four different ways, each resulting in a varying number of resampled instances within each subgroup (Fig. 1a). The random forest classifier was then trained on the resampled training set.

Fig. 1:

(a) Four different resampling techniques used to balance ACS prevalence between the two racial subgroups in the training set; (b) The adversarial debiasing framework proposed in this study. The full training set is initially fed into our baseline random forest classifier, and the top contributing features to the ACS outcome were chosen to move forward to the predictor network. The predictor network generates $los{s}_p$ and the adversarial network generates $los{s}_a$.

The second approach we proposed was implementing independent classifiers for the Black and non-Black subgroups[25]. The random forest classifier was trained and validated separately for each subgroup, using 3,198 samples for the Black subgroup and 3,582 for the non-Black subgroup. The performance of the resultant classifiers was evaluated on the two individual subgroups separately as well. One potential drawback of this approach was that the patient’s race must be identified to determine the appropriate classifier of the two to make a diagnosis, leading to unblinded models.

It is crucial to have a mitigation technique that does not alter the existing dataset and preserve blinded models. We thus proposed a neural network architecture that utilizes a technique called adversarial learning. Neural network models have shown promising performance results on ECG data[6], [26], [27]. Such architecture consisted of two multilayer perceptron models, one predictive model and one adversarial model. The predictive model was trained and made predictions on the target variable (ACS status), whereas the adversarial model took the output of the predictive model, as well as the true label, and made predictions on race. Even though race was used as a target variable here for the adversarial model, the performance and results made by such model were kept private within the training phase, keeping the predictive model blinded to demographics.

In theory, if no disparities exist within the predictive model, the adversarial model should not be able to infer the race of the sample based on $\widehat{y}$ and $y$, or based on the accuracy of the model, as expressed by the equations of equalized odds[24], [28]. Therefore, our aim was to train the predictive model with certain precision and accuracy, such that the adversarial model would not be able to infer race from the predictions. This would effectively eliminate any existing association between the performance of the ACS classifier on a certain patient and the racial status of that patient. This was achieved by implementing a custom loss function for the predictive model. We define $los{s}_p$ and $los{s}_a$ representing the binary cross-entropy loss of predicting the ACS status and the racial status, respectively. The predictive model should then be expected to generate results that minimize $los{s}_p$ (for more effective ACS prediction) and maximize $los{s}_a$ (for less effective race prediction). During the early stages of training, the adversarial model may be highly effective at predicting race, as no biases have been mitigated yet. As a result, the combined loss could decrease in a way that reduces both $los{s}_p$ and $los{s}_a$, thus helping the adversarial model. To prevent the predictor from moving in such direction that enhances race prediction, the loss function included a correction term as described by Zhang et al. and Yang et al.[28], [29]. The resultant combined custom loss was designed as follows:

$$los{s}_{combined}=los{s}_p-\beta \frac{los{s}_p}{los{s}_a}-\alpha los{s}_a$$ (4)

In this equation, $\beta \frac{los{s}_p}{los{s}_a}$ is the weighted correction term, and $\alpha los{s}_a$ is the weighted loss of the adversarial model, where the weights $\alpha$ and $\beta$ are tunable hyperparameters. Appendix A provides a detailed description of the hyperparameters used in this network.

Since our random forest classifier has shown promising performance through past research and as a baseline classifier, we used it as a feature selection tool prior to adversarial debiasing. After the baseline random forest classifier was trained, the top features by permutation importance were identified and chosen to move forward as an input to our adversarial debiasing network. The full architecture of the adversarial debiasing framework we proposed is shown in Fig. 1b. Overall, adversarial debiasing enforces equalized odds through the custom loss function, without resampling the original dataset (e.g., removing or duplicating data) or requiring the racial status to be known. The accuracy of such architecture is also promising with the random forest feature selection process.

III. Results

A. Baseline Performance

The ECG dataset was randomly split into 80% training (n = 5424) and 20% testing (n = 1356) for the baseline random forest model. The same training set was used for subsequent mitigation approaches to maintain consistency. The final model was obtained through ten-fold cross validation on the training set, and evaluated with ROC and AUC metrics on the test set (Fig. 2a).

Fig. 2:

Performance of the baseline model and data-driven mitigation approaches: (a) Baseline random forest model; (b) Undersampling method 1 - removing ACS samples in non-Black subgroup; (c) Undersampling method 2 - removing nonACS samples in Black subgroup; (d) Oversampling method 1 - duplicating non-ACS samples in non-Black subgroup; (e) Oversampling method 2 - duplicating ACS samples in the Black subgroup; (f) Independent and unblinded classifiers for the two subgroups.

The points representing sensitivity at 80% specificity are labeled on the curves in black. The AUCs are 0.789 (0.739 – 0.851) and 0.826 (0.785 – 0.863) for the Black and non-Black subgroup respectively, with a statistically significant difference by Wilcoxon rank-sum test (p =4.24×10⁻¹⁶), indicating that the model underperforms for the Black subgroup, a sign of the presence of disparities and an unfair algorithm.

B. Model Performance and Fairness After Mitigation

In the first mitigation technique, the original training set was resampled based on the four strategies described above. None of the mitigation strategies substantially affected the AUC in either subgroup, both by visual inspection of the ROC curves and Wilcoxon rank-sum test (p =4.39×10⁻¹⁵, 4.76×10⁻²¹, 7.44×10⁻²², 2.89×10⁻¹⁸, respectively), mitigation of racial disparities has been unsuccessful with resampling (Fig. 2b–e). The results of training separate classifiers for the two subgroups are shown in Fig. 2f. This approach results in a slight improvement in AUC in the Black subgroup and slight worsening of AUC in the non-Black group. However, there remains a statistically significant difference (p =5.51×10⁻³) between the subgroups’ AUCs, indicating incomplete mitigation.

Lastly, the adversarial debiasing technique was conducted. Here, the full dataset was randomly split into 70% training (n = 4746), 10% validation for hyperparameters tuning (n = 678), and 20% testing (n = 1356). The testing set remained fixed between the baseline classifier and all mitigation strategies to ensure that the results were comparable. The top 103 features, selected by the baseline random forest based on optimal performance and disparity mitigation on the validation set, were used as input to the predictor in the adversarial debiasing architecture. The features used and their associate feature importances are shown in the bar plot in Appendix B. We show the results of adversarial debiasing in Fig. 3a. There is no longer a substantial difference between the AUCs of the two subgroups, indicating this technique has successfully mitigated the disparities in predicting cases of ACS.

Fig. 3:

Performance of the adversarial debiasing network and the baseline multilyaer perceptron model: (a) Adversarial debiasing ; (b) Multilayer perceptron predictor model without the adversarial debiasing architecture.

To assess the clinical significance in mitigation, the sensitivity values at 80% specificity are compiled in Table 2. At this operating point, Fairlearn scores for equalized odds and demographic parity, computed as the ratio between subgroups, are shown in Table 3.

TABLE II:

Sensitivity at 80% Specificity on the ROC Curve

	Baseline random forest	Resampling				Separate classifiers	Adversarial debiasing
		Removing ACS cases in non-Black subgroup	Removing non-ACS cases in Black subgroup	Duplicating non-ACS cases in non-Black subgroup	Duplicating ACS cases in Black subgroup
Black subgroup	62.3% (50.3 – 73.3)	65.0% (49.5 – 76.5)	58.4% (48.4 – 73.2)	64.1% (53.4 – 76.1)	61.0% (49.4 – 73.7)	61.0% (51.9 – 72.2)	70.1% (57.7 – 80.7)
Non-Black subgroup	72.1% (65.8 – 79.0)	69.0% (61.5 – 77.0)	69.1% (61.4 – 77.2)	70.3% (60.5 – 78.5)	71.5% (63.2 – 78.5)	66.7% (58.0 – 72.2)	71.4% (59.5 – 78.9)
Difference	9.8%	4.0%	10.7%	6.2%	10.5%	5.6%	1.3%

TABLE III:

Fairness metrics ratio between subgroups computed with Fairlearn

	Baseline random forest	Resampling				Separate classifiers	Adversarial debiasing
		Removing ACS cases in non-Black subgroup	Removing non-ACS cases in Black subgroup	Duplicating non-ACS cases in non-Black subgroup	Duplicating ACS cases in Black subgroup
Equalized odds	0.864	0.827	0.805	0.871	0.903	0.916	0.960
Demographic parity	0.902	0.998	0.987	0.955	0.913	0.894	0.909

C. Effects of Neural Networks

Adversarial debiasing required the implementation of a custom loss function, which cannot be easily achieved by a random forest classifier, and thus made the shift to neural networks inevitable. It is then necessary to determine whether the change of models resulted in the mitigation of disparities rather than the debiasing architecture. To ensure that neural networks were not the changing factor, we show the results of a baseline multilayer perceptron model, with the same architecture as the predictor model in adversarial debiasing, trained without selecting the top features ranked by our random forest classifier (Fig. 3b). The significant difference between subgroups in the ROC curves of the neural network demonstrates that the disparities still exist (p =2.92×10⁻⁷), indicating that adversarial debiasing is necessary in mitigation.

IV. Discussions

A. Comparison of Mitigation Strategies Proposed

The ROC curves in Fig. 3a have demonstrated the effectiveness of adversarial debiasing in reducing racial disparities in ACS diagnosis. With the two curves almost overlapping one another, we can infer that the subgroups’ true and false positive rates are nearly identical across different thresholds. Statistical tests further confirmed that the adversarial debiasing network outperformed the other two mitigation techniques. From a clinical diagnostic perspective, it also achieved the highest AUC and sensitivity in the Black subgroup among the proposed mitigation strategies.

As demonstrated in Table 2, most of these mitigation techniques have narrowed the differences between the sub-groups, despite consistently observing a lower sensitivity in the Black subgroup. However, this usually came at the cost of significantly decreasing the performance of the model on the non-Black cohort, which is undesired. The adversarial debiasing technique is the only case where the sensitivity of the Black subgroup is considerably improved to match that of the non-Black subgroup, while the performance of the non-Black subgroup stays relatively the same. Thus, adversarial debiasing is the only mitigation strategy that does not hinder the diagnostic performance on the non-Black subgroup while reducing the degree of disparities. The proposed debiasing network mitigated the disparities that existed in the baseline model and achieved equalized odds across subgroups.

The fairness metric scores computed by Fairlearn further supported our findings from the ROC curves and sensitivity analysis. Table 3 presents the ratios of equalized odds and demographic parity across subgroups. In terms of demographic parity, the resampling strategies showed improvements over the baseline score. This is expected, as these strategies aimed to balance disease prevalence across subgroups prior to training and are therefore more aligned with the goal of achieving demographic parity than the other mitigation approaches. Considering the primary fairness metric, equalized odds, most of the proposed mitigation approaches achieved higher scores in comparison to the baseline, with adversarial debiasing approach achieving the highest score, demostrating substantial improvement from the baseline approach. This also aligned with our visual and statistical analysis of the ROC curves.

Overall, the other two techniques were unsuccessful in mitigating the disparities, as measured by equalized odds and ROC curve analysis, likely because the disparities were not caused by unbalanced samples in the dataset.

Furthermore, the resampling and separate classifier techniques modified the existing dataset and thus potentially affected the performance of the classifier. When we balanced the ACS prevalence across subgroups, all four approaches involved removing or duplicating samples in the original training set. This could have hindered the classifier’s performance, as removing samples could lead to elimination of essential details and duplicating samples could lead to overfitting. As shown by the ROC curves in Figure 2f, the difference in sensitivity at 80% specificity in Table II, and the Fairlearn equalized odds score in Table III, implementing separate classifiers outperformed all other ”naive” mitigation approaches. However, this approach inevitably reduces the sample size per subgroup, which can lead to lower overall performance compared to the baseline random forest model. Additionally, current American Heart Association (AHA) guidelines advise against race-based risk stratification tools due to concerns about reinforcing systemic biases and disparities in medicine[30].

Additionally, the dataset used in this study includes approximately 47.2% Black and 52.8% non-Black patients, representing a relatively balanced distribution that makes the separate classifiers approach less affeced by smaller subgroup sizes. However, the non-Black group comprises a heterogeneous mix of racial subgroups (e.g., White, Asian, and Hispanic), some of which have relatively small sample sizes. This limits the feasibility of using the separate classifiers approach for these subgroups in future studies. Adversarial debiasing offers a more scalable solution in such contexts, as it does not require stratifying the dataset. Instead, it tunes the model to match the performance of the minority subgroup to the majority subgroup, so that the race of the sample cannot be inferred from the accuracy of the model. This approach is deemed to be a powerful algorithm that balances high accuracy with fairness, ensuring the model remains unbiased by preserving the original dataset and keeping it blinded from the sensitive attributes. This technique also aligns with AHA’s recommendation for race-insensitive models, and has great potential to serve as a diagnostic tool in clinical settings, without requiring significant interaction from users in handling the input data or extensive technical expertise in machine learning.

B. Fairness and Accuracy Trade-Off

The trade-off between fairness and accuracy in AI models is a critical challenge, as optimizing for one often comes at the expense of the other[23], [31]. Therefore, implementing fair models usually involves both technical decisions and ethical implications. Nevertheless, the proposed debiasing technique overcomes such challenge. The ROC curves in Fig. 3b show that the sensitivities at 80% specificity for the two subgroups prior to adversarial debiasing were 63.6% (60.6 – 77.2) and 68.7% (50.3 – 73.3), respectively. The performance of the model is therefore better with proposed adversarial debiasing and feature selection with random forest, which resulted in sensitivities of 70.1% (57.7 – 80.7) and 71.4% (59.5 – 78.9), respectively for the Black and non-Black subgroups. The use of random forest as a feature selector ensured that only high-quality predictors were provided to our debiasing framework. The sensitivity for the non-Black subgroup after mitigation is also very comparable with that of the baseline random forest model. This reaffirms that adversarial debiasing is a powerful technique that does not hinder model performance, while eliminating the disparities between the subgroups.

C. Alternative Algorithmic Mitigation Strategies and Limitations

Recent contributions in bias mitigation in machine learning models divide debiasing methods into three major categories: pre-processing data, in-processing during model training, and post-processing model predictions. Each method addresses a different phase in model development when enforcing fairness. This study focuses on adversarial debiasing, an in-processing technique that tunes the predictor model’s predictions based on the performance of the adversarial model. Other in-processing methods include propensity score debiasing, contrastive learning, and disentangled representation learning[32], [33]. Contrastive learning aims to learn group-invariant representations by grouping together similar samples and pushing apart dissimilar ones, while disentangled representation learning seeks to separate latent factors related to protected attributes (e.g., race) from those relevant to the prediction task. These methods have shown promise but may be more sensitive to dataset quality, feature representation, and the presence of confounding variables. In contrast, adversarial debiasing can be more flexible across varied datasets, as it directly incorporates fairness constraints during training without requiring explicit modification of feature representations.

Additionally, due to the latent representations involved in these techniques, they tend to operate on hidden layers rather than the prediction layer, which may suffer from less explainability and is less preferred for healthcare applications. Thus, in addition to its demonstrated effectiveness in this study, adversarial debiasing is more appropriate for healthcare data, which are often affected by dataset imbalances and confounding variables, and have high demand for transparency. Other debiasing approaches could also be pre- or post-processing. Pre-processing methods, such as reweighting or resampling, adjust the data to promote fairness, which were briefly explored in this study through under/oversampling and separate classifiers. Post-processing methods, on the other hand, modify model outputs to mitigate disparities after training. Future work could benefit from a more comprehensive comparison of these approaches within clinical contexts, evaluating both predictive and fairness performance, as well as model transparency.

V. Conclusion

In this paper, we explored the application of machine learning in diagnosing ACS and identified significant disparities in the performance of the random forest model between Black and non-Black racial subgroups prior to implementing mitigation strategies. Through a comparison of three mitigation approaches, we found that adversarial debiasing is particularly effective in reducing these disparities, as proven by the performance metrics and statistical analyses conducted in this study. Adversarial debiasing reduced the disparities between the two subgroups, while preserving a blinded model and maintaining model performance. The performance metrics achieved by adversarial debiasing were comparable to the baseline model’s performance. This approach successfully overcomes the common limitation on the trade-off between fairness and accuracy. This research highlights a promising approach for achieving fairness in machine learning models in diagnosing ACS, contributing to the critical development of trustworthy and responsible AI-ECG in healthcare.

APPENDIX

A. Model Parameters

Tables 3 below presents the hyperparameters of the baseline random forest model as well as the layers and hyperparameters of the predictive and adversarial models in the debiasing network.

TABLE IV:

Parameters and Architecture for Random Forest, Predictive, and Adversarial Models

RANDOM FOREST CLASSIFIER
Parameter	Value
n_estimators	300
criterion	’gini’
bootstrap	True
class_weight	’balanced_subsample’
PREDICTIVE MODEL
Layer	Parameter
Dense	256 units, ReLU activation, L2 regularizer (1e-10)
BatchNormalization	—
Dropout	Dropout rate = 0.5
Dense	128 units, ReLU activation
BatchNormalization	—
Dropout	Dropout rate = 0.5
Dense	128 units, ReLU activation
Dense	2 units, Softmax activation
ADVERSARIAL MODEL
Layer	Parameter
Dense	64 units, ReLU activation, L2 regularizer (1e-8)
BatchNormalization	—
Dropout	Dropout rate = 0.5
Dense	128 units, ReLU activation
BatchNormalization	—
Dropout	Dropout rate = 0.5
Dense	64 units, ReLU activation
Dense	2 units, Softmax activation

In addition to the architectural hyperparameters, both models were trained using the Adam optimizer with a learning rate of 0.0001. EarlyStopping with patience = 2 was used to monitor validation loss. In the equation for the combined loss of the debiasing network, the coefficients $\alpha$ and $\beta$ were treated as hyperparameters, initialized at 0.5 and 0.25, respectively, and tuned based on the difference between the AUCs of the subgroups’ ROC curves, which are proportional to the extent of disparities to be mitigated.

B. Top 103 Features and Feature Importance

Fig. 4:

Feature importance of the top 103 features selected by the random forest classifier, which were then used as input to the adversarial debiasing framework.