Abstract
Objective
To evaluate the ability of a neural network to identify severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection using point-of-care electrocardiography obtained with a portable device.
Patient and Methods
We enrolled 2827 patients in a prospective observational study, from December 10, 2020, through June 4, 2021, to determine the accuracy of a point-of-care, handheld, smartphone-compatible, artificial intelligence–enabled electrocardiography (ECG) (POC AI-ECG) in detecting asymptomatic SARS-CoV-2 infection using a modified version of an existing deep learning model framework trained on 12-lead ECG data.
Results
Study participants were 48% (n=1067) female, 79% (n=1749) White, and 7% (n=153) endorsed previous COVID-19 infection. We found the POC AI-ECG algorithm was ineffective for detecting asymptomatic SARS-CoV-2 infection (area under curve, 0.56; 95% CI, 0.46-0.66), failing to adequately discriminate between ECGs performed among participants who tested positive compared to those who tested negative.
Conclusion
Contrary to the prior 12-lead ECG study, a POC AI-ECG failed to reliably identify asymptomatic SARS-CoV-2 infection among adults. This study underscores the importance of prospective testing, assuring similar populations, and using similar signals or data when developing AI-ECG tools.
Trial registration
clinicaltrials.gov Identifier: NCT04725097
Graphical abstract
COVID-19 disease, an acute respiratory syndrome caused by respiratory tract infection with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), was declared a pandemic by the World Health Organization on March 11, 2020.1,2 The infectious outbreak, believed to have originated in Wuhan, China, in December 2019, was first reported in the United States in January 2020.3 This disease subsequently spread, with the United States reporting the highest number of cases and deaths in 2020 compared with other countries.4 The US COVID-19 vaccination program started in December 2020, and 64.8% of the total US adult population had been fully vaccinated as of February 23, 2022.5 Despite these efforts, waning immunity and the emergence of new variants (Alpha,6,7 Delta,8 and Omicron9, 10, 11, 12) perpetuated the pandemic. In addition, model estimates have suggested that the burden of asymptomatic or undocumented infections was as high as 78.2% in the United States in 2020,4 and there were concerns that these asymptomatic carriers may be responsible for the continued propagation of the disease. It has been suggested that a universal testing program rather than a symptom-based approach may be key to curbing the spread of infection.13 This strategy was adopted by many countries for international travelers,14, 15, 16 hospitals, health systems before hospital procedures,17,18 and unvaccinated employees.19 However, these procedures involve uncomfortable nasal swabs and potential infection transmission from contact with nasopharyngeal specimens when performed in nonclinical settings, in addition to varying times to obtain results, ranging from 10 minutes for rapid antigen tests to 24 hours or more for polymerase chain reaction (PCR) tests. As such, there is a critical need to develop innovative, scalable, noninvasive mass screening tools to detect SARS-CoV-2 infection. This would facilitate a hassle-free, safe return to routine activities, such as international air travel, working from offices, and participation in large gatherings and events.
Our team has previously demonstrated the utility of artificial intelligence–enabled electrocardiography (AI-ECG) based on 12-lead ECG data for the detection of cardiovascular pathologies that are unrecognizable on routine cardiologist interpretation of the ECG. Detected pathologies include low left ventricular ejection fraction,20 atrial fibrillation while in sinus rhythm,21 cardiac amyloidosis,22 aortic stenosis,23 and SARS-CoV-2 infection.24 Although the standard 12-lead ECG is a ubiquitous test, it is rarely available in nonclinical settings. It also requires some skill to perform and typically involves the placement of electrodes directly on the skin while the patient is lying in a supine position. This limits the scalability of this technology for mass screening purposes in nonclinical environments. Portable smartphone-compatible devices, able to acquire single-lead and multilead ECG recordings and produce reliable clinical measures25 and diagnoses26, 27, 28 have gained US Food and Drug Administration approval, offering a potential alternative. In addition, recently published studies have reported the utility of portable AI-ECG for detecting low ejection fraction29,30 and accurately predicting corrected QT interval values.31 As such, we recognized this as a unique opportunity to acquire ECG recordings in nonclinical settings for mass screening purposes.
On the basis of the known association between ECG changes and myocardial injury in symptomatic COVID-19 disease32, 33, 34 and the documented efficacy of a 12-lead AI-ECG to detect symptomatic COVID-19 disease,24,35 we hypothesized that a neural network can identify the presence of asymptomatic SARS-CoV-2 infection using portable ECG device recordings. To test that hypothesis, we performed a prospective, multicenter, observational study.
Patient and Methods
Study Design
We conducted a prospective, multicenter, observational study to evaluate the accuracy of a point-of-care, handheld, smartphone-compatible, AI-ECG (POC AI-ECG) in detecting asymptomatic COVID-19 infection. This study is registered at ClinicalTrials.gov (NCT04725097).
Study Population
Study locations included 6 US sites (Mayo Clinic, Rochester, Minnesota; Mayo Clinic, Phoenix, Arizona; Mayo Clinic, Jacksonville, Florida; Hackensack University Medical Center, Hackensack, New Jersey; Heart and Vascular Institute, Henry Ford Hospital, Detroit, Michigan; and Louisiana State University Health, Shreveport, Louisiana), and 3 international sites (Sri Jayadeva Institute of Cardiovascular Sciences and Research, Karnataka, India; Royal Brompton and Harefield Hospitals and Imperial College, London, United Kingdom; and Hospital General Universitario Gregorio Marañon, Madrid, Spain). All patients who reported no symptoms and were seen for routine SARS-CoV-2 screening before a planned procedure or visit from December 10, 2020, through June 4, 2021, were approached to participate in the study. Oral consent was obtained from each study participant in accordance with local IRB rules and guidance, and documented evidence of each participant’s acknowledgement of oral consent is housed at each site and can be provided upon request.
Measures
All study participants had a portable 6-lead ECG recorded with the KardiaMobile 6L (AliveCor, Inc) at the same time a nasal swab was performed for SARS-CoV-2 PCR testing. Study participants were considered to have SARS-CoV-2 infection if the PCR test result was positive. At least one 6-lead ECG of 30-second duration using the KardiaMobile 6L device was recorded for all study participants in the sitting position with both thumbs placed on the anterior electrodes while simultaneously placing the posterior electrode on the left ankle or left knee (Figure 1). The study staff who performed portable ECG recordings were not privy to the results of the SARS-CoV-2 PCR test at the time of the ECG recording.
Figure 1.
KardiaMobile 6L (AliveCor, Inc) ECG Recording. A, Portable ECG device is connected by Bluetooth to a smartphone or tablet with installed Kardia application. B, In the sitting position, patients place both thumbs on the anterior electrodes, while simultaneously placing the posterior electrode on the left ankle or left knee. C, Six-lead ECG of 30-second duration was recorded and automatically saved into the KardiaPro (AliveCor, Inc) portal. ECG, electrocardiography.
Demographic data were self-reported by study participants, and clinical information (including SARS-CoV-2 PCR test results) were abstracted from the electronic health records. For cases with positive PCR results, additional medical history data were extracted. We stopped data collection after enrolling at least 25 patients who tested positive for COVID-19, the minimum number of positive cases estimated based on the results of the planned interim analysis. Enrollment in India was brisk, and the total number of patients who tested positive in the entire cohort was 40, exceeding the minimum number required as a result.
Our primary study end point was the detection of SARS-CoV-2 infection using a POC AI-ECG performed at the time of the nasal swab for COVID-19 screening. For diagnostic accuracy assessment, the criterion standard was the SARS-CoV-2 PCR test. Data from all sites were collected and managed using research electronic data capture (REDCap), a secure web-based platform database designed to support data capture for research studies.36 The study was approved by the institutional review boards at each participating site, with Mayo Clinic, Rochester, Minnesota, serving as the coordinating site.
Electrocardiography Data Processing
The 30-second, digital, 6-lead ECG recordings, obtained with the KardiaMobile 6L device, sampled at 300 Hz, were downloaded from the AliveCor servers and preprocessed. The ECG recordings that were flat lines (0 amplitude over the entire 30-second duration) were flagged for exclusion from all analyses owing to ECG acquisition error. The recordings were then analyzed with a modified version of the previously developed AI-ECG algorithm (based on 12-lead ECG data) to predict the presence of symptomatic SARS-CoV-2 infection.24 The model returned a score ranging from 0-1, with higher values suggesting infection, according to the original development work. Some participants had multiple ECGs recorded, and for those, the mean model score was used for the analysis.
To develop the AI algorithm used in this study, we trained 2 additional models using pre-existing retrospective ECG data and SARS-CoV-2 results from multiple sites24: the original model used in the published retrospective study utilized data from all 12 leads24; a second model was trained using data from leads I and II alone, which together represent all 6 limb leads (as leads III, aVR, aVL, and aVF are linear functions of leads I and II), and a third model was trained using data from lead I only. All the models generated prediction probabilities for multiple 2-second windows, and the final prediction for each 10-second ECG recording was the average score for 9 overlapping windows (in seconds: 0-2, 1-3, 2-4, 3-5, 4-6, 5-7, 6-8, 7-9, and 8-10).
For the POC AI-ECG model used in the current study, 6-lead ECGs were recorded with the KardiaMobile 6L device, and the second model described above was used to analyze the data. The ECG recordings were split into 2-second windows in a similar manner as the model derivation steps, but this time the result was the average of 29 overlapping 2-second windows (0-2, 1-3 … 27-29, 28-30). The POC AI-ECG was considered a positive screen if the AI prediction probability crossed the previously determined threshold of 0.44.24
Statistical Analyses
Sample size determination was based on targeting a minimum number of positive PCR tests to provide sufficient estimation precision for sensitivity. To establish the minimum number of COVID-19–positive patients, test sensitivity was assumed to be 90%, and a lower limit for the 95% CI was set at 70%, the minimum clinically relevant amount of sensitivity to warrant further use of the algorithm in this population without model revision. To achieve this degree of precision, at least 25 positive PCR tests were required. The total sample size of the enrolled population necessary to achieve the minimum number of positive PCR tests was estimated to be up to 5000 participants, accounting for an assumed ∼0.5% disease prevalence in an asymptomatic population.
The primary outcome measure for the study was the detection of SARS-CoV-2 infection. Operationally, this was defined as the model discrimination measured by the area under the receiver operating characteristic curve (AUC). Secondary end points included sensitivity for detecting a SARS-CoV-2 positive test result via PCR and specificity. Two-sided 95% CIs were computed for measures of diagnostic performance. For all measures except AUC, an exact binomial CI was used. The AUC CI was computed using the Sun and Xu optimization for the Delong method using the pROC package.37
The primary analysis was conducted among patients with valid and readable ECGs linked to confirmed SARS-CoV-2 test results. Descriptive statistics were used to describe the whole cohort and those who tested positive. A P value of >.05 was considered statistically significant. Statistical analyses were conducted using R version 4.0.3 (The R Foundation).
Results
We enrolled a total of 2827 asymptomatic adults aged 18 or older from 9 US and international sites and had 2725 ECG recordings returned. Participants were excluded if PCR test records were missing from the study database. Flat ECG recordings and recordings that could not be linked to study participant IDs were also excluded. Study participants with more than 1 recorded ECG had AI predictions generated for all available ECGs, and the mean prediction score was used. After the exclusions listed above, the final sample size included 2210 patients with matched ECG and SARS-CoV-2 PCR test results (Figure 2). There were no adverse events reported related to participating in the study.
Figure 2.
Study flow diagram. ECG, electrocardiography; PCR, polymerase chain reaction.
Characteristics of the study population are summarized in Table 1. Forty patients (1.8%) had PCR-confirmed SARS-CoV-2 infection in the study sample. There was no statistical difference in the rate of infection by sex (P=.46). Most participants identified as White, followed by Asian, other race or multiracial, and Black or African American. One hundred and sixty (7.4%) were health care workers, and 153 (6.9%) had a previous SARS-CoV-2 infection; however, neither group was statistically associated with infection rate (P>.05 for both). Additional characteristics of the COVID-19–positive patients are summarized in Supplemental Table 1, available online at https://www.mcpdigitalhealth.org/. Details regarding patient enrollment sites are displayed in Table 2 and Supplemental Figure 1, available online at https://www.mcpdigitalhealth.org/.
Table 1.
| Characteristic | COVID-19–Positive (n=40) | COVID-19–Negative (n=2170) | Overall (N=2210) | P |
|---|---|---|---|---|
| Sex | .46 | |||
| Female | 17 (42.5) | 1050 (48.4) | 1067 (48.3) | |
| Male | 23 (57.5) | 1120 (51.6) | 1143 (51.7) | |
| Race | <.001 | |||
| Asian | 24 (60.0) | 164 (7.6) | 188 (8.5) | |
| Black/African American | 2 (5.0) | 92 (4.2) | 94 (4.3) | |
| White | 11 (27.5) | 1738 (80.1) | 1749 (79.1) | |
| Other | 3 (7.5) | 176 (8.1) | 179 (8.1) | |
| Previous COVID-19c | .89 | |||
| Yes | 3 (7.5) | 150 (6.9) | 153 (6.9) | |
| No | 37 (92.5) | 2017 (93.1) | 2054 (93.1) | |
| Reason for PCR testd | <.001 | |||
| Inpatient procedure | 18 (46.2) | 522 (24.1) | 540 (24.5) | |
| Outpatient procedure | 7 (17.9) | 1254 (57.9) | 1261 (57.2) | |
| Clinic visit | 14 (35.9) | 271 (12.5) | 285 (12.9) | |
| Other | 0 (0.0) | 117 (5.4) | 117 (5.3) |
Abbreviation: PCR, polymerase chain reaction
Data reported as No. (%).
Three COVID-19–negative patients were missing prior COVID-19 histories.
Seven patients were missing reason for PCR test (1 COVID-19–positive, 6 COVID-19–negative). Other reasons for PCR testing included contact with or suspected exposure to COVID-19–infected persons, patient choice, travel requirement, participation in other research studies, dental procedures, and screening before clinical tests or therapies.
Table 2.
Patient Enrollment Sites (N=2210)
| Site | n (%) enrolled |
|---|---|
| Mayo Clinic, Rochester, Minnesota | 864 (39.1) |
| Mayo Clinic, Jacksonville, Florida | 609 (27.6) |
| Mayo Clinic, Phoenix, Arizona | 203 (9.2) |
| Imperial College, London, United Kingdom | 201 (9.1) |
| Sri Jayadeva, Karnataka, India | 192 (8.7) |
| Hospital General Universitario Gregorio Marañon, Madrid, Spain | 140 (6.3) |
| Hackensack University Medical Center, Hackensack, New Jersey | 1 (<0.1) |
| Henry Ford Hospital, Detroit, Michigana | 0 (0.0) |
| Louisiana State University Health, Shreveport, Louisianaa | 0 (0.0) |
Study was stopped before enrollment of the first patient at these participating sites.
Diagnostic Performance of the POC AI-ECG Model
In this study, the POC AI-ECG algorithm was unable to detect asymptomatic SARS-CoV-2 infection (AUC, 0.56; 95% CI, 0.46-0.66), failing to adequately discriminate between ECGs from patients who tested positive for COVID-19 compared with those who tested negative (Figure 3). Sensitivity for detecting the virus was low at 63% (25 of the 40; 95% CI, 46%-77%), which was limited in part by the small number of positive test results in the study. Specificity was lower at 47.2% (1024 of the 2170; 95% CI, 45%-49%) and the negative predictive value was 98.6% (1024 of the 1039; 95% CI, 98%-99%). A confusion matrix is displayed in Supplemental Figure 2, available online at https://www.mcpdigitalhealth.org/.
Figure 3.
Receiver operating characteristic curve for identification of SARS-CoV-2 infection among study participants using point-of-care artificial intelligence–enabled electrocardiography. AUC, area under the curve.
A stratified analysis (Supplemental Figure 3, available online at https://www.mcpdigitalhealth.org/) unpredictably showed acceptable model performance among Black patients (AUC, 0.79), albeit with very wide CIs because of the small sample size (n=94). We also noticed higher AUC values among women (0.63) compared with men (0.51). These may suggest variable performance of the POC AI-ECG model by race and sex; however, owing to the overall poor performance and discrimination of the model (Supplemental Figure 4, available online at https://www.mcpdigitalhealth.org/), the differences noted may be entirely due to chance.
Discussion
In this study, we found that a POC AI-ECG was unable to effectively identify patients with asymptomatic SARS-CoV-2 infection. We hypothesized that the AI-ECG might detect SARS-CoV-2 infection because (1) the virus binds to angiotensin-converting enzyme 2 receptors, which are richly distributed in multiple cardiac cell types38, 39, 40; (2) animal data and human reports indicate the virus may enter myocytes and that ECG changes are associated with coronavirus infection32, 33, 34; (3) troponin levels have been elevated in human infection consistent with myocardial injury38; and (4) a retrospective global study found a signal in the AI-ECG’s ability to detect infection (AUC, 0.78).24 The absence of any predictive power in this study underscores the importance of careful, prospective assessment of AI tools and may have stemmed from (1) the inclusion of ambulatory, often asymptomatic patients as opposed to mostly hospitalized patients in the previous retrospective study; (2) the use of a 6-lead portable ECG rather than the standard clinical 12-lead ECG used in the retrospective study; or (3) other as yet to be determined factors. The planned interim analysis after 25 COVID-19–positive patients were identified showed poor discrimination of the AI-ECG model, and the study was subsequently closed to enrollment.
Because the specific feature signals detected by AI tools are not known, prospectively assessing their generalizability remains an important step, as the performance of these AI tools may vary in different clinical settings.41 Although a prior retrospective study conducted by our team showed an AI-ECG based on 12-lead ECG data was able to detect symptomatic SARS-CoV-2 infection in patients across 4 continents with an AUC of 0.78,24 it remains unknown if the presence of symptoms such as fever may have influenced the performance of the AI-ECG model as ECG changes are also known to occur with fever.42,43 The AI-ECG prediction probabilities in the retrospective study were also higher among patients with moderate to severe symptoms and those requiring hospitalization compared with those with mild symptoms without any activity limitation. Furthermore, it is possible that some positive SARS-CoV-2 PCR test results were false positives or that very low levels of RNA in noninfectious patients may have persisted for months after a previous infection. Thus, asymptomatic patients enrolled in this study may have even lower AI prediction probabilities that may not differ significantly from patients with negative PCR test results. Another important consideration is the signal-to-noise ratio related to the potential variability in ECG changes as it relates to disease severity. The extent to which SARS-CoV-2 infection impacts the ECG may determine the test sensitivity. Another potential hypothesis is that asymptomatic patients diagnosed with COVID-19 may not have the same extent of cardiac involvement (with resultant ECG changes) seen in those with more severe disease, rendering the AI-ECG insensitive to detect a difference. Indeed, in the previous retrospective study, the signal was stronger for inpatients than outpatients.24
The form factor used to collect data for AI analysis may also be important, and AI models would need to specify requisite inputs for expected model performance in terms of sampling rate, dynamic range, and number of leads. The prior AI-ECG models for detection of SARS-CoV-2 infection were based on a 12-lead ECG obtained in a supine or semisupine position, whereas our study used portable 30-second, 6-lead ECG recordings limited only to the limb leads acquired in a sitting position. Although some AI-ECG algorithms appear robust whether using a single lead recorded with a portable device or a standard 12-lead ECG, as with detection of low ejection fraction,30 this cannot be assumed for all AI-ECG algorithms, and the different form factor may have affected performance.
It is also possible that information useful for SARS-CoV-2 detection might rely heavily on the precordial leads, which were not captured in this study, or signals may be slightly altered by patient positioning. Prior studies evaluating the performance of an AI model (originally trained on 12-lead ECG data) using ECG obtained with a portable smart stethoscope to detect low ejection fraction focused on precordial and chest ECG leads (V2 and V5, respectively), with lead V2 reporting the highest AUC in both studies.29,30 Although these studies aimed to identify low ejection fraction, it is possible that the deep learning model architecture gleans more useful diagnostic information from the precordial leads, which are missing from portable 6-lead recordings (limb leads only). In clinical practice, the location of the leads may not matter for certain conditions, such as atrial fibrillation, in which the abnormal ECG findings can be visualized in all leads regardless of position. Alternatively, in cases of myocardial ischemia, ECG changes may be concealed in the limb leads but not in the precordial leads, as seen with anterior ischemia,44 occlusion of the right coronary artery or left circumflex coronary artery, and posterior wall ischemia, where supplemental leads are often required,45,46 or in the presence of a bundle branch block or ventricular pacing.47 Future studies would be needed to evaluate the accuracy of the AI-ECG (using either limb or precordial leads alone or in combination) for the detection of COVID-19 disease and overt COVID-19–related myocarditis, which may also inform its potential utility for detection of other inflammatory or infiltrative cardiomyopathies.
Given the growing field of AI in medicine, particularly for disease detection, there has been an exponential increase in the number of studies evaluating AI as a tool to help curb the COVID-19 pandemic,48,49 for surveillance, screening, drug discovery, and vaccine development.50 Several studies have evaluated different AI models for SARS-CoV-2 detection as potential screening tools, using chest radiography,51,52 computed tomography,53, 54, 55 clinical variables,56, 57, 58 lung ultrasonography,59 saliva,60 breath analysis,61 audio recordings,62, 63, 64 biometric monitoring, and sensor technology.65, 66, 67 Some concerns have been raised about the clinical utility of some imaging-based machine learning models for detecting COVID-19, including methodologic flaws, bias in data sets, and insufficient detail in reporting, limiting the reproducibility of results.68 Many of these methods may also be unsuitable for large-scale screening purposes as imaging modalities such as radiography, computed tomography, or ultrasonography are mostly limited to clinical settings, are expensive (relative to nasal swab testing), and come with the potential risk for ionizing radiation, even in low doses. In addition, the use of clinical variables poses a risk for protected health information data breach, while saliva and breath analysis may pose an infectious risk owing to inadvertent contact with body fluids or respiratory droplets. Other modalities, such as audio recordings and biometric monitoring or sensor technology, are attractive options for mass screening, as is our proposed portable ECG device. What remains unknown is if these methods have been sufficiently validated in prospective studies and whether the infrastructure required to scale these technologies for mass screening is available.
While our study failed to demonstrate the ability of a POC AI-ECG to screen for asymptomatic SARS-CoV-2 infection, potential next steps to further explore its use for screening purposes include retraining the existing 12-lead ECG models to use data from limb leads only, prospective acquisition of a larger number of POC AI-ECG data from patients with confirmed COVID-19 disease, and potential data augmentation with adversarial networks to increase the training examples needed for model development and refinement.
Conclusion
A POC AI-ECG failed to reliably identify SARS-CoV-2 infection among asymptomatic adults undergoing testing before a procedure or other hospital encounter. Our results underscore the need to carefully assess AI tools developed in unique patient populations before generalizing these tools to broad or distinct populations. They also highlight the importance of rigorous prospective validation of AI-ECG tools in diverse (demographic and disease-related) patient cohorts and across institutions and ECG devices (owing to preprogrammed variations in sampling frequencies, signal processing, and filtering procedures) before implementation. Thus, while AI is poised to transform medical practice, further studies on biological plausibility and clinical deployment are needed.
Potential Competing Interests
D.A.A. receives research support from the Mayo Clinic Women’s Health Research Center and the Mayo Clinic Building Interdisciplinary Research Careers in Women’s Health (BIRCWH) program funded by the National Institutes of Health (NIH; grant number K12 HD065987). The content of the article is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. I.Z.A. and P.A.F. are coinventors of several of the AI algorithms described (including screen for low ejection fraction, QT tool, aortic stenosis, and atrial fibrillation detection during normal sinus rhythm). These have been licensed to Anumana, AliveCor, and Eko. Mayo Clinic, I.Z.A., and P.A.F. may receive benefit from their commercialization. P.A.N. receives research funding from the NIH (including the National Heart, Lung, and Blood Institute [R21AG 62580-1, R01HL 131535-4, R01HL 143070-2] the National Institute on Aging [R01AG 062436-1]), Agency for Healthcare Research and Quality (R01HS 25402-3), US Food and Drug Administration (FD 06292), and the American Heart Association (18SFRN34230146). P.A.N. and Mayo Clinic have filed patents related to the application of AI to ECG for diagnosis and risk stratification and have licensed several AI-ECG algorithms to Anumana. P.A.N. and Mayo Clinic are involved in a potential equity/royalty relationship with AliveCor. P.A.N. is a study investigator in an ablation trial sponsored by Medtronic. P.A.N. has served on an expert advisory panel for OptumLabs. E.F.B. receives honorarium from UTD, <$5,000 per year. E.F.B. sat on the Debiopharm advisory board and received <$5,000 in reimbursement in 2021. Given their role as Editorial Board Members, I.Z.A, R.E.C, and P.A.F. had no involvement in the peer-review of this article and had no access to information regarding its peer-review. J.L.D., P.W.J, E.J.D., A.C.M.L., M.A.P., H.H.T., L.T.C., L.R.S., A.M.V., D.P., N.S.P., P.B., M.K., F.F-A., F.A., T.V.G., M.K.L., P.D., S.K., N.L.P., J.C.C., and R.E.C. declare no competing financial or non-financial interests.
Data Availability
The data underlying this article will be shared on reasonable request to the corresponding author. All requests for raw and analyzed data and related materials, excluding programming code, will be reviewed by the Mayo Clinic Legal Department and Mayo Clinic Ventures to verify whether the request is subject to any intellectual property or confidentiality obligations. Requests for patient-related data not included in the paper will not be considered. Any data and materials that can be shared will be released via a material transfer agreement. This study was approved by the Mayo Clinic Institutional Review Boards in Rochester, Minnesota, Jacksonville, Florida, and Phoenix, Arizona, and by the institutional review boards at each participating site. Programming code related to the data preprocessing and Keras model specification will be made available under the GNU General Public License version 3 upon request to I.Z.A. (attia.itzhak@mayo.edu).
Acknowledgments
We thank AliveCor for donation of the KardiaMobile 6L devices used for data collection during this study and ECG data processing and extraction from AliveCor server. We also thank the Departments of Cardiovascular Medicine at Mayo Clinic in Rochester, Minnesota, Jacksonville, Florida, and Phoenix, Arizona; Digital Innovation Laboratory, Mayo Clinic, Jacksonville Florida; and the Department of Community Internal Medicine, Mayo Clinic, Jacksonville, Florida for providing support and resources needed to successfully complete this study. The Scientific Publications staff at Mayo Clinic provided copyediting, proofreading, administrative, and clerical support. P.A.F. and R.E.C. contributed to study conceptualization and design. P.A.F., R.E.C., I.Z.A., P.A.N., and D.A.A. contributed to data analysis and interpretation. All coauthors contributed to the writing and critical revision of the manuscript for intellectual content. All authors approved the decision to submit the final version of the manuscript. P.A.F., R.E.C., I.Z.A., and D.A.A. take responsibility for the integrity of the work, from study inception to the final manuscript. R.E.C. and I.Z.A. had full access to all study data and take responsibility for data integrity and accuracy of the data analysis. All authors reviewed the results presented in the manuscript. All coauthors verified the accuracy of data acquired at each of their individual sites. J.L.D., J.C.C., and I.Z.A. provided administrative, technical, and material support. P.A.F. supervised the study.
Footnotes
Grant Support: Departments of Cardiovascular Medicine at Mayo Clinic in Rochester, Minnesota, Jacksonville, Florida, and Phoenix, Arizona.
Supplemental material can be found online at https://www.mcpdigitalhealth.org/. Supplemental material attached to journal articles has not been edited, and the authors take responsibility for the accuracy of all data.
Supplemental Online Material
Supplemental Figure 1.
Supplemental Figure 2.
Supplemental Figure 3.
Supplemental Figure 4.
References
- 1.Ranney M.L., Griffeth V., Jha A.K. Critical Supply Shortages—The need for ventilators and personal protective equipment during the Covid-19 pandemic. N Engl J Med. 2020;382(18):e41. doi: 10.1056/NEJMp2006141. [DOI] [PubMed] [Google Scholar]
- 2.Coronavirus disease 2019 (COVID-19): situation report, 51. World Health Organization. https://apps.who.int/iris/handle/10665/331475
- 3.Holshue M.L., DeBolt C., Lindquist S., et al. First case of 2019 novel coronavirus in the United States. N Engl J Med. 2020;382(10):929–936. doi: 10.1056/NEJMoa2001191. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Sen P., Yamana T.K., Kandula S., Galanti M., Shaman J. Burden and characteristics of COVID-19 in the United States during 2020. Nature. 2021;598(7880):338–341. doi: 10.1038/s41586-021-03914-4. [DOI] [PubMed] [Google Scholar]
- 5.Covid data tracker weekly review. Center for Disease Control and Prevention. https://www.cdc.gov/coronavirus/2019-ncov/covid-data/covidview/index.html
- 6.Chookajorn T., Kochakarn T., Wilasang C., Kotanan N., Modchang C. Southeast Asia is an emerging hotspot for COVID-19. Nat Med. 2021;27(9):1495–1496. doi: 10.1038/s41591-021-01471-x. [DOI] [PubMed] [Google Scholar]
- 7.Walker A.S., Vihta K.-D., Gethings O., et al. Tracking the emergence of SARS-CoV-2 alpha variant in the United Kingdom. N Engl J Med. 2021;385(27):2582–2585. doi: 10.1056/NEJMc2103227. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Telenti A., Arvin A., Corey L., et al. After the pandemic: perspectives on the future trajectory of COVID-19. Nature. 2021;596(7873):495–504. doi: 10.1038/s41586-021-03792-w. [DOI] [PubMed] [Google Scholar]
- 9.Classification of Omicron (B.1.1.529): SARS-CoV-2 Variant of Concern. World Health Organization. https://www.who.int/news/item/26-11-2021-classification-of-omicron-(b.1.1.529)-sars-cov-2-variant-of-concern
- 10.Callaway E. Heavily mutated Omicron variant puts scientists on alert. Nature. 2021;600(7887):21. doi: 10.1038/d41586-021-03552-w. [DOI] [PubMed] [Google Scholar]
- 11.Karim S.S.A., Karim Q.A. Omicron SARS-CoV-2 variant: a new chapter in the COVID-19 pandemic. Lancet. 2021;398(10317):2126–2128. doi: 10.1016/S0140-6736(21)02758-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Zhang X., Wu S., Wu B., et al. SARS-CoV-2 Omicron strain exhibits potent capabilities for immune evasion and viral entrance. Signal Transduct Target Ther. 2021;6(1):430. doi: 10.1038/s41392-021-00852-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Huff H.V., Singh A. Asymptomatic transmission during the coronavirus disease 2019 pandemic and implications for public health strategies. Clin Infect Dis. 2020;71(10):2752–2756. doi: 10.1093/cid/ciaa654. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Clifford S., Quilty B.J., Russell T.W., et al. Strategies to reduce the risk of SARS-CoV-2 importation from international travellers: modelling estimations for the United Kingdom, July 2020. Euro Surveill. 2021;26(39) doi: 10.2807/1560-7917.ES.2021.26.39.2001440. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Tande A.J., Binnicker M.J., Ting H.H., et al. SARS-CoV-2 Testing before international airline travel, December 2020 to May 2021. Mayo Clin Proc. 2021;96(11):2856–2860. doi: 10.1016/j.mayocp.2021.08.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Kiang M.V., Chin E.T., Huynh B.Q., et al. Routine asymptomatic testing strategies for airline travel during the COVID-19 pandemic: a simulation study. Lancet Infect Dis. 2021;21(7):929–938. doi: 10.1016/S1473-3099(21)00134-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Kibbe M.R. Surgery and COVID-19. JAMA. 2020;324(12):1151–1152. doi: 10.1001/jama.2020.15191. [DOI] [PubMed] [Google Scholar]
- 18.Lu A.C., Schmiesing C.A., Mahoney M., et al. COVID-19 preoperative assessment and testing: from surge to recovery. Ann Surg. 2020;272(3):e230–e235. doi: 10.1097/SLA.0000000000004124. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Jaffe S. Legal challenges threaten Biden's COVID-19 vaccine rule. Lancet. 2021;398(10314):1863–1864. doi: 10.1016/S0140-6736(21)02537-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Attia Z.I., Kapa S., Lopez-Jimenez F., et al. Screening for cardiac contractile dysfunction using an artificial intelligence-enabled electrocardiogram. Nat Med. 2019;25(1):70–74. doi: 10.1038/s41591-018-0240-2. [DOI] [PubMed] [Google Scholar]
- 21.Attia Z.I., Noseworthy P.A., Lopez-Jimenez F., et al. An artificial intelligence-enabled ECG algorithm for the identification of patients with atrial fibrillation during sinus rhythm: a retrospective analysis of outcome prediction. Lancet. 2019;394(10201):861–867. doi: 10.1016/S0140-6736(19)31721-0. [DOI] [PubMed] [Google Scholar]
- 22.Grogan M., Lopez-Jimenez F., Cohen-Shelly M., et al. Artificial intelligence–enhanced electrocardiogram for the early detection of cardiac amyloidosis. Mayo Clin Proc. 2021;96(11):2768–2778. doi: 10.1016/j.mayocp.2021.04.023. [DOI] [PubMed] [Google Scholar]
- 23.Cohen-Shelly M., Attia Z.I., Friedman P.A., et al. Electrocardiogram screening for aortic valve stenosis using artificial intelligence. Eur Heart J. 2021;42(30):2885–2896. doi: 10.1093/eurheartj/ehab153. [DOI] [PubMed] [Google Scholar]
- 24.Attia Z.I., Kapa S., Dugan J., et al. Rapid exclusion of COVID infection with the artificial intelligence electrocardiogram. Mayo Clin Proc. 2021;96(8):2081–2094. doi: 10.1016/j.mayocp.2021.05.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Haberman Z.C., Jahn R.T., Bose R., et al. Wireless smartphone ECG enables large-scale screening in diverse populations. J Cardiovasc Electrophysiol. 2015;26(5):520–526. doi: 10.1111/jce.12634. [DOI] [PubMed] [Google Scholar]
- 26.Perez M.V., Mahaffey K.W., Hedlin H., et al. Large-scale assessment of a smartwatch to identify atrial fibrillation. N Engl J Med. 2019;381(20):1909–1917. doi: 10.1056/NEJMoa1901183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Halcox J.P.J., Wareham K., Cardew A., et al. Assessment of remote heart rhythm sampling using the AliveCor heart monitor to screen for atrial fibrillation: the REHEARSE-AF study. Circulation. 2017;136(19):1784–1794. doi: 10.1161/CIRCULATIONAHA.117.030583. [DOI] [PubMed] [Google Scholar]
- 28.Wegner F.K., Kochhäuser S., Ellermann C., et al. Prospective blinded evaluation of the smartphone-based AliveCor Kardia ECG monitor for atrial fibrillation detection: the PEAK-AF study. Eur J Intern Med. 2020;73:72–75. doi: 10.1016/j.ejim.2019.11.018. [DOI] [PubMed] [Google Scholar]
- 29.Attia Z.I., Dugan J., Maidens J., et al. Prospective analysis of utility of signals from an ECG-enabled stethoscope to automatically detect a low ejection fraction using neural network techniques trained from the standard 12-lead ECG. Circulation. 2019;140 [Google Scholar]
- 30.Bachtiger P., Petri C.F., Scott F.E., et al. Point-of-care screening for heart failure with reduced ejection fraction using artificial intelligence during ECG-enabled stethoscope examination in London, UK: a prospective, observational, multicentre study. Lancet Digit Health. 2022;4(2):e117–e125. doi: 10.1016/S2589-7500(21)00256-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Giudicessi J.R., Schram M., Bos J.M., et al. Artificial Intelligence–enabled assessment of the heart rate corrected QT interval using a mobile electrocardiogram device. Circulation. 2021;143(13):1274–1286. doi: 10.1161/CIRCULATIONAHA.120.050231. [DOI] [PubMed] [Google Scholar]
- 32.Alexander L.K., Keene B.W., Yount B.L., Geratz J.D., Small J.D., Baric R.S. ECG changes after rabbit coronavirus infection. J Electrocardiol. 1999;32(1):21–32. doi: 10.1016/S0022-0736(99)90018-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Basso C., Leone O., Rizzo S., et al. Pathological features of COVID-19-associated myocardial injury: a multicentre cardiovascular pathology study. Eur Heart J. 2020;41(39):3827–3835. doi: 10.1093/eurheartj/ehaa664. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Hendren N.S., Drazner M.H., Bozkurt B., et al. Description and proposed management of the acute COVID-19 cardiovascular syndrome. Circulation. 2020;141(23):1903–1914. doi: 10.1161/CIRCULATIONAHA.120.047349. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Ozdemir M.A., Ozdemir G.D., Guren O. Classification of COVID-19 electrocardiograms by using hexaxial feature mapping and deep learning. BMC Med Inform Decis Mak. 2021;21(1):170. doi: 10.1186/s12911-021-01521-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Harris P.A., Taylor R., Minor B.L., et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. 2019;95 doi: 10.1016/j.jbi.2019.103208. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Robin X., Turck N., Hainard A., et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics. 2011;12:77. doi: 10.1186/1471-2105-12-77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Nishiga M., Wang D.W., Han Y., Lewis D.B., Wu J.C. COVID-19 and cardiovascular disease: from basic mechanisms to clinical perspectives. Nat Rev Cardiol. 2020;17(9):543–558. doi: 10.1038/s41569-020-0413-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.McGonagle D., Plein S., O'Donnell J.S., Sharif K., Bridgewood C. Increased cardiovascular mortality in African Americans with COVID-19. Lancet Respir Med. 2020;8(7):649–651. doi: 10.1016/S2213-2600(20)30244-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Liu H., Gai S., Wang X., et al. Single-cell analysis of SARS-CoV-2 receptor ACE2 and spike protein priming expression of proteases in the human heart. Cardiovasc Res. 2020;116(10):1733–1741. doi: 10.1093/cvr/cvaa191. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Siontis K.C., Noseworthy P.A., Arghami A., et al. Use of artificial intelligence tools across different clinical settings; a cautionary tale. Circ Cardiovasc Qual Outcomes. 2021;14(9) doi: 10.1161/CIRCOUTCOMES.121.008153. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Grune J., Yamazoe M., Nahrendorf M. Electroimmunology and cardiac arrhythmia. Nat Rev Cardiol. 2021;18(8):547–564. doi: 10.1038/s41569-021-00520-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Karjalainen J., Viitasalo M. Fever and cardiac rhythm. Arch Intern Med. 1986;146(6):1169–1171. [PubMed] [Google Scholar]
- 44.Zimetbaum P.J., Josephson M.E. Use of the electrocardiogram in acute myocardial infarction. N Engl J Med. 2003;348(10):933–940. doi: 10.1056/NEJMra022700. [DOI] [PubMed] [Google Scholar]
- 45.Wagner G.S., Macfarlane P., Wellens H., et al. AHA/ACCF/HRS recommendations for the standardization and interpretation of the electrocardiogram: part VI: acute ischemia/infarction: a scientific statement from the American Heart Association Electrocardiography and Arrhythmias Committee, Council on Clinical Cardiology; the American College of Cardiology Foundation; and the Heart Rhythm Society. Endorsed by the International Society for Computerized Electrocardiology. J Am Coll Cardiol. 2009;53(11):1003–1011. doi: 10.1016/j.jacc.2008.12.016. [DOI] [PubMed] [Google Scholar]
- 46.Writing Committee Members. Gulati M., Levy P.D., et al. 2021 AHA/ACC/ASE/CHEST/SAEM/SCCT/SCMR Guideline for the Evaluation and Diagnosis of Chest Pain: a report of the American College of Cardiology/American Heart Association Joint Committee on clinical practice guidelines. J Cardiovasc Comput Tomogr. 2022;16(1):54–122. doi: 10.1016/j.jcct.2021.11.009. [DOI] [PubMed] [Google Scholar]
- 47.Sgarbossa E.B. Recent advances in the electrocardiographic diagnosis of myocardial infarction: left bundle branch block and pacing. Pacing Clin Electrophysiol. 1996;19(9):1370–1379. doi: 10.1111/j.1540-8159.1996.tb04217.x. [DOI] [PubMed] [Google Scholar]
- 48.Alafif T., Tehame A.M., Bajaba S., Barnawi A., Zia S. Machine and deep learning towards COVID-19 diagnosis and treatment: survey, challenges, and future directions. Int J Environ Res Public Health. 2021;18(3) doi: 10.3390/ijerph18031117. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Alimadadi A., Aryal S., Manandhar I., Munroe P.B., Joe B., Cheng X. Artificial intelligence and machine learning to fight COVID-19. Physiol Genomics. 2020;52(4):200–202. doi: 10.1152/physiolgenomics.00029.2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Arora G., Joshi J., Mandal R.S., Shrivastava N., Virmani R., Sethi T. Artificial intelligence in surveillance, diagnosis, drug discovery and vaccine development against COVID-19. Pathogens. 2021;10(8) doi: 10.3390/pathogens10081048. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Al-Ali A., Elharrouss O., Qidwai U., Al-Maaddeed S. ANFIS-Net for automatic detection of COVID-19. Sci Rep. 2021;11(1) doi: 10.1038/s41598-021-96601-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Albadr M.A.A., Tiun S., Ayob M., Al-Dhief F.T., Omar K., Hamzah F.A. Optimised genetic algorithm-extreme learning machine approach for automatic COVID-19 detection. PLoS One. 2020;15(12) doi: 10.1371/journal.pone.0242899. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Chen H., Guo S., Hao Y., et al. Auxiliary diagnosis for COVID-19 with deep transfer learning. J Digit Imaging. 2021;34(2):231–241. doi: 10.1007/s10278-021-00431-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Han Z., Wei B., Hong Y., et al. Accurate screening of COVID-19 using attention-based deep 3D multiple instance learning. IEEE Trans Med Imaging. 2020;39(8):2584–2594. doi: 10.1109/TMI.2020.2996256. [DOI] [PubMed] [Google Scholar]
- 55.Jin C., Chen W., Cao Y., et al. Development and evaluation of an artificial intelligence system for COVID-19 diagnosis. Nat Commun. 2020;11(1):5088. doi: 10.1038/s41467-020-18685-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Li W.T., Ma J., Shende N., et al. Using machine learning of clinical data to diagnose COVID-19: a systematic review and meta-analysis. BMC Med Inform Decis Mak. 2020;20(1):247. doi: 10.1186/s12911-020-01266-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Yan Y., Schaffter T., Bergquist T., et al. A continuously benchmarked and crowdsourced challenge for rapid development and evaluation of models to predict COVID-19 diagnosis and hospitalization. JAMA Netw Open. 2021;4(10) doi: 10.1001/jamanetworkopen.2021.24946. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Zhang J., Jun T., Frank J., Nirenberg S., Kovatch P., Huang K.L. Prediction of individual COVID-19 diagnosis using baseline demographics and lab data. Sci Rep. 2021;11(1) doi: 10.1038/s41598-021-93126-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.La Salvia M., Secco G., Torti E., et al. Deep learning and lung ultrasound for Covid-19 pneumonia detection and severity classification. Comput Biol Med. 2021;136 doi: 10.1016/j.compbiomed.2021.104742. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Heinzel C., Pinilla Y.T., Elsner K., et al. Non-invasive antibody assessment in saliva to determine SARS-CoV-2 exposure in young children. Front Immunol. 2021;12 doi: 10.3389/fimmu.2021.753435. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Wintjens A.G.W.E., Hintzen K.F.H., Engelen S.M.E., et al. Applying the electronic nose for pre-operative SARS-CoV-2 screening. Surg Endosc. 2021;35(12):6671–6678. doi: 10.1007/s00464-020-08169-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Coppock H., Jones L., Kiskin I., Schuller B. COVID-19 detection from audio: seven grains of salt. Lancet Digit Health. 2021;3(9):e537–e538. doi: 10.1016/S2589-7500(21)00141-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Mohammed E.A., Keyhani M., Sanati-Nezhad A., Hejazi S.H., Far B.H. An ensemble learning approach to digital corona virus preliminary screening from cough sounds. Sci Rep. 2021;11(1) doi: 10.1038/s41598-021-95042-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Suppakitjanusant P., Sungkanuparph S., Wongsinin T., et al. Identifying individuals with recent COVID-19 through voice classification using deep learning. Sci Rep. 2021;11(1) doi: 10.1038/s41598-021-98742-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.D’Haese P.F., Finomore V., Lesnik D., et al. Prediction of viral symptoms using wearable technology and artificial intelligence: a pilot study in healthcare workers. PLoS One. 2021;16(10) doi: 10.1371/journal.pone.0257997. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Skibinska J., Burget R., Channa A., Popescu N., Koucheryavy Y. COVID-19 diagnosis at early stage based on smartwatches and machine learning techniques. IEEE Access. 2021;9:119476–119491. [Google Scholar]
- 67.Wong C.K., Ho D.T.Y., Tam A.R., et al. Artificial intelligence mobile health platform for early detection of COVID-19 in quarantine subjects using a wearable biosensor: protocol for a randomised controlled trial. BMJ Open. 2020;10(7) doi: 10.1136/bmjopen-2020-038555. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Roberts M., Driggs D., Thorpe M., et al. Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans. Nat Mach Intell. 2021;3(3):199–217. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The data underlying this article will be shared on reasonable request to the corresponding author. All requests for raw and analyzed data and related materials, excluding programming code, will be reviewed by the Mayo Clinic Legal Department and Mayo Clinic Ventures to verify whether the request is subject to any intellectual property or confidentiality obligations. Requests for patient-related data not included in the paper will not be considered. Any data and materials that can be shared will be released via a material transfer agreement. This study was approved by the Mayo Clinic Institutional Review Boards in Rochester, Minnesota, Jacksonville, Florida, and Phoenix, Arizona, and by the institutional review boards at each participating site. Programming code related to the data preprocessing and Keras model specification will be made available under the GNU General Public License version 3 upon request to I.Z.A. (attia.itzhak@mayo.edu).