Abstract
Objective
With the emergence of artificial intelligence (AI)-based health interventions, systemic racism remains a concern as these advancements are frequently developed without race-specific data analysis or validation. To evaluate the potential utility of an AI-based cardiovascular diseases (CVD) screening tool in an under-resourced African-American cohort, we reviewed the AI-enhanced electrocardiogram (ECG) data of participants enrolled in a community-based clinical trial as a proof-of-concept ancillary study for community-based screening.
Methods
Enrollees completed cardiovascular testing including standard 12-lead ECG and a limited echocardiogram (TTE). All ECGs were analyzed using previously published institution-based AI algorithms. AI-ECG predictions were generated for age, sex, and decreased left ventricular ejection fraction (LVEF). Diagnostic accuracy of the AI-ECG for decreased LVEF and sex was quantified using area under the receiver operating characteristic curve (AUC). Correlation between actual age and AI-ECG predicted age was assessed using Pearson correlation coefficients.
Results
Fifty-four participants completed both an ECG and TTE (mean age 55 years [range 31-87 years]; 66.7% female). All participants were in sinus rhythm, and the median LVEF of the cohort was 60-65%. The AI-ECG for decreased LVEF demonstrated excellent performance with an AUC of 0.892 (95% confidence interval [CI] 0.708-1); sensitivity=50% (95% CI 9.5-90.5%; n=1/2) and specificity=96% (95% CI 86.8-98.9%; n=49/51). The AI-ECG for participant sex demonstrated similar performance with AUC of 0.944 (95% CI 0.891-0.998); sensitivity=100% (95% CI 82.4-100.0%; n=18/18) and specificity=77.8% (95% CI 61.9-88.3%; n=28/36). The AI-ECG predicted mean age was 55 years (range 26.9-72.6 years) with a strong correlation to actual age (R=0.769; p<0.001).
Conclusion
Our analyses of previously developed AI-ECG algorithms for prediction of age, sex, and decreased LVEF demonstrated reliable performance in this community-based, African-American cohort. This novel, community-centric delivery of AI could provide valuable screening resources and appropriate referrals for early detection of highly-morbid CVD for under-resourced patient populations.
Keywords: Artificial intelligence, Electrocardiogram, Disparities, Race
Abbreviations: ADI, Area Deprivation Index; AHA, American Heart Association; CBPR, community-based participatory research; CVD, cardiovascular disease; CVH, cardiovascular health; FAITH!, Fostering African-American Improvement in Total Health!; LS7, Life's Simple 7; LVEF, left ventricular ejection fraction; mHealth, mobile health; SDOH, Social determinants of health; TTE, transthoracic echocardiogram
1. Introduction
Minoritized racial and ethnic populations often face significant barriers to prevention and treatment of cardiovascular disease (CVD) [1]. Prior studies have provided evidence demonstrating the impact of adverse social determinants of health (SDOH) on worse CVD outcomes among most of these groups compared to non-Hispanic White individuals [[2], [3], [4]]. These SDOH include limited access to quality health care, economic disempowerment, disenfranchised neighborhoods with unreliable/inaccessible transportation, and destitute housing. Further, there is historic mistrust of the healthcare system, particularly among African-Americans that is attributed to past unethical research and injustices such as the Tuskegee Syphilis Study and the Henrietta Lacks cell line [1,5]. Even with the emergence of mobile health (mHealth), telehealth, and artificial intelligence (AI)-based health interventions as means to address healthcare disparities, systemic racism remains a concern as these advancements are frequently developed and deployed without race-specific data analysis or validation [1,6]. Further, there remains a gap in health technology integration at the community and individual levels which could potentially widen the digital divide and worsen healthcare disparities [1,5].
There have been research efforts to mitigate racial disparities in cardiovascular health (CVH) by prioritizing marginalized populations through authentic community engagement [6,7]. The FAITH! (Fostering African-American Improvement in Total Health!) Trial is a community-based participatory research (CBPR) initiative which aims to bridge the digital divide for African-Americans through co-design and testing of mHealth interventions with community members for CVH promotion. Participants in this trial had significant improvements in their CVH with use of a personalized, smartphone-based, mHealth lifestyle intervention (FAITH! App) promoted among African-American churches [7]. With community partner input (FAITH! Community Steering Committee [CSC]), a community-based FAITH! Trial ancillary study (FAITH! Heart Health+) was launched to provide participants with extended CVH assessment and diagnostic testing including an electrocardiogram (ECG) and limited transthoracic echocardiogram (TTE).
Currently, all ECGs obtained in the Mayo Clinic Health System undergo AI-enhanced algorithmic assessment to identify potential CVDs not readily apparent by manual ECG interpretation alone [8]. To evaluate the effectiveness and potential utility of the AI-based CVD screening tool in an under-resourced African-American cohort, we reviewed the AI-enhanced ECG data of FAITH! participants, as a proof-of-concept ancillary study for community-based AI CVD screening.
2. Methods
2.1. Study design and participants
In partnership with the FAITH! CSC, we conducted a CBPR project, FAITH! Heart Health+, as an ancillary study to the overarching FAITH! Trial which focused on structural racism and CVH. The CSC provided input in all project phases including conceptualization of the need for additional CVD screening, development of a culturally sensitive implementation plan, and dissemination of study results to participants and key stakeholders. Community partners expressed the importance of leveraging innovative technological resources, including AI, for the benefit of the FAITH! Trial participants and to serve as a model for other healthcare systems.
The design, recruitment, and preliminary findings of the overarching FAITH! Trial have been previously described in detail [7]. In brief, members from partnering churches in Rochester and Minneapolis, Minnesota were invited to participate in the study. Participants included in the study were African-American adults with smartphone ownership, basic internet skills (eg, navigation, downloading, etc.) and at least weekly internet and email access. Limited exclusion criteria were impaired ambulatory ability, pregnancy, or mental disability precluding independent mHealth app use. Baseline survey data included CVH assessment (American Heart Association Life's Simple 7 [AHA LS7] metrics for scoring), socio-demographics, relevant medical history, and SDOH influences on CVH. Study participants were randomized to either the FAITH! App mHealth lifestyle intervention or control group to assess impact of the app on overall CVH.
For inclusion in the Heart Health+ study, participants had to be enrolled in FAITH! Trial and provide informed consent. No additional eligibility criteria were applied. Enrolled participants completed a health assessment at a local community-oriented health center by a mobile clinical research unit team. The assessment included extended CVD risk evaluation and SDOH assessment (eg, housing-based socioeconomic status [HOUSES] index; neighborhood deprivation by area deprivation index [ADI]) by electronic survey [9,10]. The study was approved by the Mayo Clinic Institutional Review Board and registered (clinicaltrials.gov NCT03777709).
2.2. Artificial intelligence-enhanced electrocardiography and echocardiography
Enrolled Heart Health+ study participants completed cardiovascular testing including a standard, digital, 10-second 12-lead ECG and a limited TTE using a SonoSite Titan portable ultrasound machine [7]. The ECG date, acquisition location, and rhythm were extracted from the MUSE system (GE Healthcare, Marquette, WI). The ECG rhythm, ECG characteristics and echocardiographic findings including left ventricular ejection fraction (LVEF) were adjudicated by a technologist under cardiologist supervision. Incentives (US$50 cash cards) and parking vouchers were provided to participants for their time and effort at completion of each study assessment: electronic survey, laboratory studies, ECG, and TTE.
All ECGs were analyzed using previously published convolutional neural networks embedded in the Mayo Clinic electronic medical record system via an AI-ECG Dashboard [8]. AI-ECG prediction probabilities were generated for age, sex and decreased LVEF [11,12]. Decreased LVEF in our cohort was defined as a value below <50%, slightly more lenient than the national/international definition of reduced LVEF <40% [13]. LVEF was visually estimated from the TTE images.
2.3. Statistical analysis
Patient demographic variables were summarized as total with percentage for categorical variables and mean with standard deviation for continuous variables. The diagnostic accuracy of the AI-ECG to detect decreased LVEF and predict participant sex was quantified using area under the receiver operating characteristic curve (ROC AUC). Sensitivity and specificity were calculated for decreased LVEF and sex prediction using a cut-off of AI-ECG predicted probability >0.256 and >0.48, respectively. Correlation between actual age and AI-ECG predicted age was assessed using Pearson correlation coefficients. All statistical analyses were performed using R version 4.1.2 (R Foundation for Statistical Computing, Vienna, Austria).
3. Results
3.1. Cohort demographics
There was high enthusiasm and interest in participation in the Heart Health+ study as a total of 63 of 85 FAITH! Trial participants (74%) were successfully enrolled in the Heart Health+ Study of which 54 (86%) completed both an ECG and a TTE between February and April of 2022. The mean age of participants was 55 years (range 31-87 years) and 66.7% were female which is representative of the overall FAITH! Trial [7]. Personalized CVH risk factors and SDOH are reported in Supplemental Table 1. The average CVH (AHA-LS7) score for the cohort was 6.54 (standard deviation [SD] 1.86). Mean state and national ADI rankings were 5.9 (SD 2.6) and 48.6 (SD 20.4) respectively indicating moderate neighborhood deprivation. Participants were also of lower socioeconomic status (mean standardized HOUSES index score -0.93 [SD 2.5]).
3.2. Electrocardiography and echocardiography
All participants were in sinus rhythm (i.e., normal sinus rhythm or sinus bradycardia) at the time of ECG recording, and a TTE was performed on the same day for 35 (64.8%) of participants. Those who underwent TTE at a separate time were evaluated on average within 32.2 days of ECG recording. The median LVEF range of the cohort was 60-65%, and two participants had a reduced LVEF (<50%).
3.3. Artificial intelligence-enhanced electrocardiography
The AI-ECG for the detection of LVEF demonstrated excellent performance with an AUC of 0.892 (95% confidence interval [CI]: 0.708-1) with sensitivity of 50% (n=1/2; 95% CI: 9.5%-90.5%), and specificity of 96% (n=49/51; 95% CI: 86.8%-98.9%). The AI-ECG for prediction of participant sex (probability of male sex being a positive test result) had similarly strong performance with AUC of 0.944 (95% CI: 0.891-0.998), sensitivity of 100% (n=18/18; 95% CI: 82.4%-100.0%), and specificity of 77.8% (n=28/36; 95% CI: 61.9%-88.3%) (Fig.). The AI-ECG predicted mean age was 55 years (range 26.9-72.6 years). The correlation between actual age and AI-ECG predicted age was strong with R=0.769 (p<0.001) (Fig.). On average AI-ECG predicted age was within 0.024 (SD 7.7) years of participants’ actual ages.
Fig.
Successful community-based approach to cardiovascular disease screening by artificial intelligence-enhanced electrocardiograms in an underserved population of African-Americans.
LEFT: Participants were recruited from partnering churches to enroll within the FAITH! Heart Health+ ancillary study. Enrolled participants completed a demographic/health survey, laboratory studies evaluation, electrocardiogram (ECG) and a limited transthoracic echocardiogram. CENTER: ECGs were subsequently processed through our institution-based AI-ECG dashboard. RIGHT: Results from AI-processed ECGs were AI-ECG predicted participant age, participant sex, and decreased left ventricular ejection fraction. AI indicates artificial intelligence; CVD, cardiovascular disease; ECG electrocardiogram; LVEF, left ventricular ejection fraction; TTE, transthoracic echocardiogram.
A single participant with ECG limb lead reversal had a false positive AI-ECG prediction for decreased LVEF, which potentially demonstrates limitations of the AI-ECG in routine clinical practice and highlights ECG leads that likely influence the AI-based probabilities. However, it is noted that this participant also had sex misclassified by the AI-ECG (female participant with AI-ECG estimated a 99% probability of being male; Supplemental Fig. 1) suggesting a potential use of the AI-ECG sex estimation to serve as a “validity check” for AI-ECGs with unanticipated results.
An exploratory analysis was performed to assess potential correlates between AI-ECG scores for decreased LVEF, age, and CVD risk factors including CVH and SDOH survey data (Supplemental Figs. 3-10).
4. Discussion
In this study, we found that previously developed and validated AI-ECG algorithms for LVEF, age, and sex prediction demonstrated reliable performance in this community-based African-American cohort. Participants were able to receive AI-ECG-based screening at a local community venue with subsequent ECG processing via our institution-based AI-ECG dashboard [8]. While only weak correlations existed between CVH, SDOH and AI-ECG results, this study provides a framework for community-based AI-ECG screening in areas with limited healthcare access and/or resources. Based on FAITH! CSC guidance and feedback, all participants will receive the results from their extended CVD screening (including ECG and TTE results) via a lay-friendly, culturally tailored, Heart Healthy card explaining each test in plain language (Supplemental Fig. 2).
This proof-of-concept application of the AI-ECG for community-based CVD screening has far-reaching implications, particularly in communities who may experience barriers to medical care [1,6,7]. This process may also be a means to potentially address healthcare disparities in limited resource medical facilities, including community health centers (eg, Federally Qualified Health Centers). Use of AI-ECG screening for cardiac pathology within these settings could amplify clinical suspicion for patients who would benefit from further diagnostic workup or sub-specialty referral allowing for effective distribution of limited resources in these communities.
Attia et al. demonstrated that having an AI-ECG predicted age greater than chronologic age was associated with low LVEF, hypertension, and coronary artery disease, thus serving as a potential indirect biomarker for overall physiologic CVH [11]. AI-ECG age results may reinforce positive lifestyle changes [14], especially when used alongside a self-monitoring app like the FAITH! App which tracks health behaviors. This hypothesis needs to be tested in a larger cohort.
As with community-based studies, there were several challenges faced by the study team and participants which were addressed with community partner input. Given competing demands of the COVID-19 pandemic and participants’ full-time employment, there were initial scheduling difficulties of participant CVD screenings. The FAITH! CSC offered suggestions of coordination of appointments to accommodate participant work schedules with the opportunity to complete both their ECG and TTE either on the same day or at separate, shorter appointments at their convenience. Further, we were mindful of efficient timing (length) of diagnostic tests to minimize participant burden. This strategy proved useful as most participants (65%) completed all studies at one appointment, and those with separate appointments completed studies within an average of 32 days. Many participants lacked a consistent mode of transportation or had limited financial resources to cover transportation or parking expenses related to health assessments. Thus, the study team offered a convenient, community-based location near the partnering churches and provided parking and ride share vouchers.
4.1. Limitations
Our study is best understood within the context of its limitations. Given the small sample size, there were few participants with LVEF <50% (n=2) though many had multiple risk factors for CVD (eg, hypertension, diabetes, overweight/obesity, etc. [Supplemental Table 1]). We also acknowledge that outlier data from this group, even as few as 1-2 missing inputs, can significantly impact the statistical significance of our findings. As noted, sensitivity (50%) is limited given the very low number of participants with true decreased LVEF (<50%), and the AI-ECG model was originally trained to identify LVEF ≤35% and may be less sensitive for LVEF above this threshold [12]. It is worth noting that AI-ECG probabilities in isolation may be of limited value without clinical correlation or additional diagnostic testing.
5. Conclusions
Our results demonstrate a promising proof-of-concept with significant implications. If appropriately implemented, this community-centric health care initiative could provide valuable screening resources and appropriate referrals for early detection of highly-morbid cardiovascular conditions that are otherwise unavailable or inaccessible to individuals with limited healthcare resources, particularly minoritized racial and ethnic groups. This novel AI initiative also establishes a framework for larger studies to possibly identify socio-demographic risk factors which could have a significant impact on CVD detected by AI.
Contributions
All authors participated intellectually or practically in this work including the conception (Brewer, Harmon, Hayes, Jones, Patten), design (Brewer, Van't Hof, Cooper, Jones, Patten), data analysis (Brewer, Johnson, Harmon, Patten), data interpretation (Brewer, Johnson, Adedinsewo, Harmon, Cooper, Jones, Hayes, Patten) and manuscript drafting and final review (Brewer, Adedinsewo, Van't Hof, Harmon, Attia, Friedman, Lopez, Jones, Hayes, Patten).
Funding
The research reported herein was supported by the National Institutes of Health (NIH)/National Institute on Minority Health and Health Disparities (NIMHD) (Grant No. 1 R21 MD013490-01), the Clinical and Translational Science Awards (CTSA) (Grant No. UL1 TR000135) from the National Center for Advancing Translational Sciences (NCATS) to Mayo Clinic, the Mayo Clinic Center for Health Equity and Community Engagement in Research and the Mayo Clinic Executive Office. Dr. Brewer was supported by the American Heart Association-Amos Medical Faculty Development Program (Grant No. 19AMFDP35040005), NCATS (CTSA Grant No. KL2 TR002379) and the Centers for Disease Control and Prevention (CDC, Grant No. CDC-DP18-1817) during the implementation of this work. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of NCATS, NIH or CDC. The funding bodies had no role in study design; in the collection, analysis, and interpretation of data; writing of the manuscript; and in the decision to submit the manuscript for publication.
Disclosures
Other authors have no relevant funding to disclose.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
The authors thank all study participants and partnering church congregations. We wholeheartedly appreciate the Mayo Clinic Executive Office for financially supporting this ancillary study. Sincerest appreciation also goes to the entire FAITH! study team, Mayo Clinic Clinical Research and Trials Unit, Mayo Clinic Heart Rhythm and Physiologic Monitoring Laboratory, and the University of Minnesota Rasmussen Center for Cardiovascular Disease Prevention (Natalia Florea) for their rigorous data collection. We also thank Mayo Clinic Square Sports Medicine for allowing us to use their facilities for study participant convenience. Finally, we thank our FAITH! Community Steering Committee for their support and guidance throughout the study.
Footnotes
Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.ajpc.2022.100431.
Appendix. Supplementary materials
References
- 1.Johnson AE, Brewer LC, Echols MR, Mazimba S, Shah RU, Breathett K. Utilizing Artificial Intelligence to Enhance Health Equity Among Patients with Heart Failure. Heart Fail Clin. 2022;18(2):259–273. doi: 10.1016/j.hfc.2021.11.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Tsao CW, Aday AW, Almarzooq ZI, et al. Heart Disease and Stroke Statistics-2022 Update: A Report From the American Heart Association [published correction appears in Circulation. 2022 Sep 6;146(10):e141] Circulation. 2022;145(8):e153–e639. doi: 10.1161/CIR.0000000000001052. [DOI] [PubMed] [Google Scholar]
- 3.Lloyd-Jones DM, Ning H, Labarthe D, et al. Status of Cardiovascular Health in US Adults and Children Using the American Heart Association's New "Life's Essential 8" Metrics: Prevalence Estimates From the National Health and Nutrition Examination Survey (NHANES), 2013 Through 2018. Circulation. 2022;146(11):822–835. doi: 10.1161/CIRCULATIONAHA.122.060911. [DOI] [PubMed] [Google Scholar]
- 4.Churchwell K, Elkind MSV, Benjamin RM, et al. Call to Action: Structural Racism as a Fundamental Driver of Health Disparities: A Presidential Advisory From the American Heart Association. Circulation. 2020;142(24):e454–e468. doi: 10.1161/CIR.0000000000000936. [DOI] [PubMed] [Google Scholar]
- 5.Liu P, Astudillo K, Velez D, Kelley L, Cobbs-Lomax D, Spatz ES. Use of Mobile Health Applications in Low-Income Populations: A Prospective Study of Facilitators and Barriers. Circ Cardiovasc Qual Outcomes. 2020;13(9) doi: 10.1161/CIRCOUTCOMES.120.007031. Sep. [DOI] [PubMed] [Google Scholar]
- 6.Noseworthy PA, Attia ZI, Brewer LC, et al. Assessing and Mitigating Bias in Medical Artificial Intelligence: The Effects of Race and Ethnicity on a Deep Learning Model for ECG Analysis. Circ Arrhythm Electrophysiol. 2020;13(3) doi: 10.1161/CIRCEP.119.007988. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Brewer LC, Jenkins S, Hayes SN, et al. Community-Based, Cluster-Randomized Pilot Trial of a Cardiovascular Mobile Health Intervention: Preliminary Findings of the FAITH! Trial. Circulation. 2022;146(3):175–190. doi: 10.1161/CIRCULATIONAHA.122.059046. Jul 19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Siontis KC, Noseworthy PA, Attia ZI, Friedman PA. Artificial intelligence-enhanced electrocardiography in cardiovascular disease management. Nat Rev Cardiol. 2021;18(7):465–478. doi: 10.1038/s41569-020-00503-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Takahashi PY, Ryu E, Hathcock MA, et al. A novel housing-based socioeconomic measure predicts hospitalisation and multiple chronic conditions in a community population. J Epidemiol Community Health. 2016;70(3):286–291. doi: 10.1136/jech-2015-205925. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Johnson AE, Zhu J, Garrard W, et al. Area Deprivation Index and Cardiac Readmissions: Evaluating Risk-Prediction in an Electronic Health Record. J Am Heart Assoc. 2021;10(13) doi: 10.1161/JAHA.120.020466. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Attia ZI, Friedman PA, Noseworthy PA, et al. Age and Sex Estimation Using Artificial Intelligence From Standard 12-Lead ECGs. Circ Arrhythm Electrophysiol. 2019;12(9) doi: 10.1161/CIRCEP.119.007284. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Attia ZI, Kapa S, Lopez-Jimenez F, et al. Screening for cardiac contractile dysfunction using an artificial intelligence-enabled electrocardiogram. Nat Med. 2019;25(1):70–74. doi: 10.1038/s41591-018-0240-2. [DOI] [PubMed] [Google Scholar]
- 13.Yancy CW, Jessup M, Bozkurt B, et al. 2013 ACCF/AHA Guideline for the management of heart failure: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2013;62:e147–e239. doi: 10.1016/j.jacc.2013.05.019. [DOI] [PubMed] [Google Scholar]
- 14.Harmon DM, Lopez-Jimenez F, Friedman PA. Introducing Artificial Intelligence into the Preventive Medicine Visit. Mayo Clin Proc. 2022;97(8):1575–1577. doi: 10.1016/j.mayocp.2022.06.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.