Graphical abstract
Summary of the development and validation of the Pulmonary Hypertension (PH) Early Detection Algorithm (PH-EDA) based on analysis of a standard 12-lead ECG. mPAP: mean pulmonary arterial pressure; RHC: right heart catheterisation; TRV: tricuspid regurgitation velocity; VUMC: Vanderbilt University Medical Center; AUC: area under the curve.
Abstract
Background
Early diagnosis of pulmonary hypertension (PH) is critical for effective treatment and management. We aimed to develop and externally validate an artificial intelligence algorithm that could serve as a PH screening tool, based on analysis of a standard 12-lead ECG.
Methods
The PH Early Detection Algorithm (PH-EDA) is a convolutional neural network developed using retrospective ECG voltage–time data, with patients classified as “PH-likely” or “PH-unlikely” (controls) based on right heart catheterisation or echocardiography. In total, 39 823 PH-likely patients and 219 404 control patients from Mayo Clinic were randomly split into training (48%), validation (12%) and test (40%) sets. ECGs taken within 1 month of PH diagnosis (diagnostic dataset) were used to train the PH-EDA at Mayo Clinic. Performance was tested on diagnostic ECGs within the test sets from Mayo Clinic (n=16 175/87 998 PH-likely/controls) and Vanderbilt University Medical Center (VUMC; n=6045/24 256 PH-likely/controls). In addition, performance was tested on ECGs taken 6–18 months (pre-emptive dataset), and up to 5 years prior to a PH diagnosis at both sites.
Results
Performance testing yielded an area under the receiver operating characteristic curve (AUC) of 0.92 and 0.88 in the diagnostic test sets at Mayo Clinic and VUMC, respectively, and 0.86 and 0.81, respectively, in the pre-emptive test sets. The AUC remained a minimum of 0.79 at Mayo Clinic and 0.73 at VUMC up to 5 years before diagnosis.
Conclusion
The PH-EDA can detect PH at diagnosis and 6–18 months prior, demonstrating the potential to accelerate diagnosis and management of this debilitating disease.
Shareable abstract
Early diagnosis of pulmonary hypertension (PH) is critical for effective treatment and management. The PH-EDA is a noninvasive, ECG-based algorithm that has the potential to accelerate the diagnosis and management of patients with PH. https://bit.ly/3KdYF55
Introduction
Pulmonary hypertension (PH) is a progressive pulmonary vascular disease, estimated to affect 1% of the global population [1]. PH encompasses a heterogeneous group of disorders with multiple aetiologies, but all have the common feature of an elevated mean pulmonary arterial pressure (mPAP) [2], and may lead to right ventricular failure and subsequent death [3]. PH diagnosis is defined by a resting mPAP >20 mmHg, measured by right heart catheterisation (RHC) [4, 5]. This diagnostic threshold was recently lowered from ≥25 mmHg [6, 7].
Due to the nonspecific symptoms at presentation, such as dyspnoea [8], and the rarity of PH relative to other conditions with similar symptoms, diagnostic delays are common, often in excess of 2 years [9–12]. Delays are associated with higher risk of morbidity and mortality [13, 14], thus underscoring the importance of early disease recognition [12, 15] and the need for an effective, noninvasive screening tool.
RHC provides a definitive diagnosis of PH [4, 5], but as this is invasive and resource intensive, its use for the initial assessment of PH risk is limited. For patients with suspected PH, standard transthoracic echocardiography is the recommended first-line screening tool [4, 5]. However, this test is not always ordered for patients presenting with initial symptoms of PH, especially in a primary care setting [16], and when it is, right ventricular size and function for evaluation of PH are not always assessed and additional echocardiographic signs of PH not always captured. Conversely, the ECG, one of the most standardised clinical tools in medicine [12], is inexpensive and performed regularly in primary care [4, 5] for patients with unexplained dyspnoea. However, standard interpretation of the ECG generally only detects abnormalities in late-stage PH [17]. Artificial intelligence (AI), specifically deep learning, has the potential to address these diagnostic challenges and delays by enabling sensitive detection of subtle abnormalities in the raw voltage–time data [18–22]. Such models, in particular convolutional neural networks (CNNs) [23], have been applied to ECGs [24] and have performed well in recognising several cardiovascular conditions, including occult and imminent conditions [18, 20, 25].
The purpose of this study was to develop and externally validate a deep learning-based algorithm to aid in the early detection of PH using 12-lead ECG voltage–time data. Importantly, the aim of the algorithm is to raise clinical suspicion of PH, particularly when it is not yet under clinical consideration, and not to diagnose PH. The results from the algorithm may suggest the need for additional testing, including focused echocardiographic evaluation of the right heart, tests to aid in the differential diagnosis of PH and invasive hemodynamic evaluation as clinically indicated. An overview of the analytical plan is shown in figure 1.
FIGURE 1.
Overview of the analytical plan to train and evaluate performance of a deep learning-based ECG algorithm in detection of pulmonary hypertension (PH). It illustrates the steps taken to train and evaluate the algorithm using ECG data from Mayo Clinic. It is a pre-trained algorithm, i.e. when used at other sites, such as Vanderbilt University Medical Center, it is not re-trained on data from those sites. EHR: electronic health record; RHC: right heart catheterisation.
Methods
Data sources and study population
For the development and testing of the PH Early Detection Algorithm (PH-EDA), data were obtained retrospectively from adult patients at all Mayo Clinic locations in the USA between July 2002 and August 2020. External validation was performed using data from Vanderbilt University Medical Center (VUMC) for patients evaluated between January 2000 and September 2020. At both centres, all procedures, codes and measurements used occurred when patients were aged ≥18 years. Patients who denied research authorisation were excluded from the study.
Eligible patients were adults who had undergone RHC or had an echocardiogram with results available in digitised form and voltage–time data from at least one standard 10-s, 12-lead ECG (Mayo Clinic: GE Healthcare, Chicago, IL, USA; VUMC: Philips Healthcare, Andover, MA, USA). All ECG data were down-sampled to 250 Hz and then up-sampled to 500 Hz and ECG amplitudes converted to millivolts.
Patients were retrospectively classified as likely to have PH or unlikely (control). In accordance with the 2022 European Society of Cardiology (ESC)/European Respiratory Society (ERS) guidelines [4, 5], PH-likely patients were defined as those with at least one mPAP >20 mmHg at rest by RHC, and control patients defined as those with mPAP ≤20 mmHg on all available RHCs. Patients who could not be classified as PH based on these definitions due to unavailable RHC data were included in the PH-likely cohort if they had a peak tricuspid regurgitation velocity (TRV) >3.4 m·s−1 at rest recorded at least once by echocardiogram, indicating high probability of PH [4, 5]. Patients without an RHC performed were included in the control cohort if all available echocardiograms had measured TRV ≤2.8 m·s−1, indicating low probability of PH [4, 5].
The diagnosis date for PH-likely patients was defined as the first RHC with mPAP >20 mmHg or first echocardiogram with TRV >3.4 m·s−1 for patients without RHC measurements. RHC and echocardiogram measurements that involved drug or exercise challenge were excluded when defining the PH-likely group, but were used to ensure that challenge-induced PH patients were not included in the control group. The index date was defined as the diagnosis date for PH patients, or the date of the last RHC or echocardiogram for control patients. PH patients with an ECG within 30 days of the index date, and control patients with an ECG on or before the index date were included in the diagnostic dataset. In Mayo Clinic, this dataset was used to train, validate and test the CNN algorithm as described later; at VUMC, this dataset was used to test the algorithm. Additional subgroups within the diagnostic dataset were also used to test the performance of the algorithm (described later).
This study was conducted in accordance with the Declaration of Helsinki and approved by the institutional review boards of the Mayo Clinic (ID: 19-012650) and VUMC (IRB #221350). PH-EDA is an investigational CNN that has not, at the time of this writing, been evaluated by health authorities as a medical device.
Convolutional neural network architecture and training
The PH-EDA was trained to predict the presence of PH. Several structures of the algorithm were evaluated using the Mayo Clinic cohort; the final algorithm used a single branch convolution 1D structure, in which data from all 12 leads were fed as one input branch to the neural network. Details on the algorithm architecture, pre-processing and tuning procedures are provided in supplementary methods SI.
All patients within the diagnostic dataset from Mayo Clinic were randomly divided into three mutually exclusive groups: training (48%), validation (12%) and test (40%) sets. The algorithm was trained on random crops of 2-s windows of the ECG, allowing the algorithm to focus on individual heartbeats. Testing was performed on overlapping 2-s windows (e.g. 0–2 s, 1–3 s, 2–4 s), and an average of the predictions from all fragments was taken.
Performance of the PH-EDA at different time points prior to diagnosis
Performance of the PH-EDA was assessed on ECGs at Mayo Clinic and VUMC within the diagnostic test set (i.e. ECGs within 1 month of PH diagnosis) and on ECGs taken 6–18 months (pre-emptive test set), 18–36 months and 36–60 months before a PH diagnosis.
Relationship between PH severity and the PH-EDA output
The output of the model (PH-EDA score) was categorised according to PH severity as assessed by either mPAP or TRV for the diagnostic test set at Mayo Clinic and visualised using box-and-whisker plots.
Performance of the PH-EDA in patients with dyspnoea
A population within the diagnostic test set enriched for PH-likely patients was evaluated, specifically patients presenting with dyspnoea, identified using International Classification of Disease (ICD) codes (ICD-9: 786.0, 786.05; ICD-10: R06.00, R06.01, R06.02, R06.09) on or before the ECG and RHC/echocardiogram. Performance was also evaluated in dyspnoeic patients with PH defined exclusively using RHC measurements, using both the >20 mmHg and ≥25 mmHg mPAP thresholds of PH diagnosis.
Within the dyspnoea cohort, performance of the PH-EDA was evaluated in subgroups of RHC-defined PH cohorts (mPAP >20 mmHg): pre-capillary PH (Pc-PH), isolated post-capillary PH (Ipc-PH) and combined pre- and post-capillary PH (Cpc-PH). These subgroups were defined based on the 2022 ESC/ERS guidelines, as 1) pulmonary vascular resistance (PVR) >2 Wood Units (WU) and pulmonary capillary wedge pressure (PCWP) ≤15 mmHg; 2) PVR ≤2 WU and PCWP >15 mmHg; and 3) PVR >2 WU and PCWP >15 mmHg, respectively [6].
PH-EDA performance metrics and statistical analyses
Performance of the PH-EDA was evaluated by calculating the area under the receiver operating characteristic curve (AUC), the sensitivity, specificity, diagnostic odds ratio and positive (PPV) and negative predictive values (NPV). PPV and NPV were computed using the prevalence of PH within the PH-likely and control cohorts and using the observed prevalence at each site. A probability threshold was selected using the Youden index (J) method [26]. For all testing, a single ECG was selected per patient. For PH patients, the nearest ECG to diagnosis was selected for the diagnostic window (with the ECG prior to diagnosis prioritised if two ECGs were equidistant), and a random ECG was selected from within the time period for all nondiagnostic windows. For control patients, the closest ECG prior to the index date was selected. To evaluate performance within the dyspnoea population, patients were required to have an ECG on or after the date of a dyspnoea ICD code and within the relevant time window. Confidence intervals were calculated using 50 bootstrapping runs on the patients in the test set of interest.
Comparison of PH-EDA performance to physician interpretation of PH-related ECG findings
To compare the performance of the PH-EDA to physicians manually reviewing an ECG, PH-related ECG abnormalities (listed in supplementary table S1) and direct mentions of PH were extracted from diagnostic text statements within the ECG reports, corresponding to each ECG waveform, and used to label patients within the test set of the existing PH-likely and control cohorts. Out of all waveforms, 99.97% at Mayo Clinic and 100% at VUMC had diagnostic text statements available, and both sites require that all diagnostic text statements are overread by a cardiologist. Natural language processing was used to extract synonyms from the statements. Subsequent pattern matching was used to determine the presence or absence of the abnormality. When presence of a PH-related abnormality was confirmed, the patient associated with the ECG was labelled as PH, while absence/no mention of an abnormality was labelled as control. To compute the sensitivity and specificity of the ECG abnormalities (individually and combined), the labels determined from the diagnostic text were compared to the cohort definitions. Performance was assessed at Mayo Clinic and VUMC for all time windows, in cohorts with and without dyspnoea preceding the ECG and compared to the performance of the PH-EDA.
Results
Patient cohorts
282 927 adult patients (PH-likely n=46 157; control n=236 770) with paired ECGs were retrospectively identified from Mayo Clinic. Among these patients, 259 227 had an ECG within 30 days before or after the date of PH diagnosis, or an ECG on or before last screening, and were included in the diagnostic dataset. The diagnostic dataset included 39 823 PH-likely patients (17 688 patients with dyspnoea code) and 219 404 control patients (40 251 with dyspnoea code) (figure 2). This dataset was subdivided into the training, validation and test sets. Within the diagnostic test set, 7155 PH-likely patients and 16 155 control patients had a dyspnoea ICD code recorded on or prior to an ECG and the diagnostic RHC or echocardiogram. A summary of the number of patients included in the different subgroups and time intervals evaluated is provided in table 1.
FIGURE 2.
Patient flow diagram. Schematic of the dataset creation from Mayo Clinic and Vanderbilt University Medical Center (VUMC). PH: pulmonary hypertension; mPAP: mean pulmonary arterial pressure; RHC: right heart catheterisation; ICD: International Classification of Disease; TRV: tricuspid regurgitation velocity. #: dyspnoea ICD code on or before an eligible ECG and the diagnostic RHC/echocardiogram (for PH-likely patients) or the last screening RHC/echocardiogram (for control patients).
TABLE 1.
Overview of the different test sets evaluated
| Time intervals of ECG before PH diagnosis# | ||||||||
|---|---|---|---|---|---|---|---|---|
| Mayo Clinic test set | VUMC test set | |||||||
| Within 1 month | 6–18 months | 18–36 months | 36–60 months | Within 1 month | 6–18 months | 18–36 months | 36–60 months | |
| Full diagnostic test set | ||||||||
| Control patients | 87 998 | 24 256 | ||||||
| PH patients | 16 175 | 4985 | 4784 | 4135 | 6045 | 1880 | 1602 | 1321 |
| Patients with dyspnoea ICD code ¶ | ||||||||
| Control patients | 16 155 | 9208 | ||||||
| PH patients | 7155 | 2187 | 1811 | 1286 | 3719 | 1147 | 909 | 678 |
| PH patients with RHC diagnosis | ||||||||
| mPAP >20 mmHg | 2737 | 785 | 614 | 436 | 1940 | 564 | 412 | 296 |
| Pc-PH | 507 | 283 | ||||||
| Ipc-PH | 785 | 686 | ||||||
| Cpc-PH | 609 | 442 | ||||||
| mPAP >25 mmHg | 2307 | 670 | 531 | 378 | 1599 | 478 | 363 | 254 |
Data are presented as n. PH: pulmonary hypertension; VUMC: Vanderbilt University Medical Center; ICD: International Classification of Disease; RHC: right heart catheterisation; mPAP: mean pulmonary arterial pressure; Pc-PH: pre-capillary PH; Ipc-PH: isolated post-capillary PH; Cpc-PH: combined pre- and post-capillary PH. #: the numbers sum to greater than the total number of patients included in the study, as patients could belong to more than one time interval, e.g. if they had an ECG within 1 month of RHC and another ECG 36 months before RHC; ¶: dyspnoea ICD code on or before an eligible ECG and the diagnostic RHC/echocardiogram (for PH patients) or the last screening RHC/echocardiogram (for control patients).
In the diagnostic dataset at Mayo Clinic, 87% of patients were Caucasian (table 2). In the PH-likely and control groups, 48.1% and 52.3%, respectively, were female and the mean age was 70.1 years and 61.6 years, respectively, at the index date. In total, 34% of PH-likely patients (n=13 572) had their diagnosis confirmed using RHC, and 66% (n=26 251) were defined as a high probability of PH by TRV.
TABLE 2.
Summary of patient demographics from the Mayo Clinic and Vanderbilt University Medical Center (VUMC) diagnostic datasets
| Mayo Clinic# | VUMC¶ | |||||
|---|---|---|---|---|---|---|
| Overall | PH-likely | Control | Overall | PH-likely | Control | |
| Patients | 259 227 | 39 823 | 219 404 | 30 301 | 6045 | 24 256 |
| Patients with dyspnoea in record | 85 730 (33.1) | 23 916 (60.1) | 61 814 (28.2) | 17 729 (58.5) | 4656 (77.0) | 13 073 (53.9) |
| Patients with dyspnoea prior to diagnostic ECG | 57 939 (22.4) | 17 688 (44.4) | 40 251 (18.3) | 12 927 (42.7) | 3719 (61.5) | 9208 (38.0) |
| Race | ||||||
| White | 225 515 (87.0) | 34 760 (87.3) | 190 755 (86.9) | 24 842 (82.0) | 4762 (78.8) | 20 080 (82.8) |
| Black or African American | 8259 (3.2) | 1624 (4.1) | 6635 (3.0) | 3806 (12.6) | 1036 (17.1) | 2770 (11.4) |
| Asian | 3714 (1.4) | 417 (1.0) | 3297 (1.5) | 333 (1.1) | 46 (0.8) | 287 (1.2) |
| Native American/Pacific Islander | 1912 (0.7) | 351 (0.9) | 1561 (0.7) | 58 (0.2) | 10 (0.2) | 48 (0.2) |
| Other | 4514 (1.7) | 656 (1.6) | 3858 (1.8) | 179 (0.6) | 28 (0.5) | 151 (0.6) |
| Unknown | 15 322 (5.9) | 2017 (5.1) | 13 305 (6.1) | 1178 (3.9) | 188 (3.1) | 990 (4.1) |
| Sex | ||||||
| Female | 133 838 (51.6) | 19 158 (48.1) | 114 680 (52.3) | 15 896 (52.5) | 2920 (48.3) | 12 976 (53.5) |
| Male | 125 387 (48.4) | 20 665 (51.9) | 104 722 (47.7) | 14 405 (47.5) | 3125 (51.7) | 11 280 (46.5) |
| Age at index date, years | ||||||
| Mean±sd | 62.9±17.0 | 70.1±15.1 | 61.6±16.9 | 59.5±17.3 | 64.2±15.8 | 58.3±17.4 |
| Median (range) | 65.0 (18.0–107.0) | 72.0 (18.0–105.0) | 64.0 (18.0–107.0) | 61.9 (18.0–104.3) | 65.8 (18.0–103.4) | 60.8 (18.0–104.3) |
| IQR | 53.0–76.0 | 61.0–81.0 | 51.0–74.0 | 48.5–72.1 | 54.9–75.6 | 46.9–71.2 |
| BMI + , kg·m−2 | ||||||
| Mean±sd | 27.7±5.3 | 28.3±5.5 | 27.6±5.2 | |||
| Median (range) | 27.2 (12.0–53.6) | 27.9 (12.1–40.6) | 27.1 (12.0–53.6) | |||
| IQR | 23.8–31.2 | 24.2–32.2 | 23.7–31.0 | |||
| RHC measurements § | ||||||
| mPAP, mmHg | n=13 572 | n=3126 | n=3128 | n=832 | ||
| Mean±sd | 35.1±11.1 | 16.2±2.9 | 33.6±10.6 | 15.9±3.3 | ||
| Median (range) | 33.0 (21.0–100.0) | 17.0 (5.0–20.0) | 31.0 (21.0–121.0) | 16.0 (4.0–20.0) | ||
| PVR, WU | n=11 740 | n=2341 | n=2827 | |||
| Mean±sd | 4.0±3.6 | 1.5±0.7 | 3.6±10.2 | |||
| Median (range) | 2.9 (0.0–81.7) | 1.4 (0.0–5.5) | 2.6 (-1.0–520.0) | |||
| PCWP, mmHg | n=12 402 | n=2911 | n=3013 | |||
| Mean±sd | 18.2±7.5 | 8.7±3.5 | 18.8±7.6 | |||
| Median (range) | 18.0 (0.0–50.0) | 9.0 (0.0–34.0) | 18.0 (0.0–51.0) | |||
| TRV measurements | n=26 251 | n=216 278 | n=2917 | n=23 424 | ||
| TRVƒ, m·s−1 | ||||||
| Mean±sd | 3.7±0.3 | 2.4±0.2 | 3.8±0.4 | 2.2±0.4 | ||
| Median (range) | 3.6 (3.4–9.0) | 2.4 (0.0–2.8) | 3.6 (3.4–5.8) | 2.3 (0.0–2.8) | ||
Data are presented as n, n (%), unless otherwise stated. PH: pulmonary hypertension; IQR: interquartile range; BMI: body mass index; RHC: right heart catheterisation; mPAP: mean pulmonary arterial pressure; PVR: pulmonary vascular resistance; WU: Wood Units; PCWP: pulmonary capillary wedge pressure; TRV: tricuspid regurgitation velocity. #: training, validation and test sets combined; ¶: diagnostic test set; +: nearest BMI record within 1 year of patient index date within study period; §: values are based on the measurements at index date only; ƒ: values for patients with PH are based on the measurement at diagnosis date only, whereas values for control patients use the average of values per patient on or prior to last procedure.
For external validation of the PH-EDA, a total of 6045 PH-likely patients (3719 with dyspnoea code) and 24 256 control patients (9208 with dyspnoea code) were included in the diagnostic test set from VUMC (table 1 and figure 2). Demographics and disease characteristics were similar to those in the Mayo Clinic diagnostic dataset, apart from a slightly higher proportion of Black/African American patients in the VUMC population (table 1).
A higher proportion of dyspnoea codes was found within patient records at VUMC compared to Mayo Clinic for both the PH-likely (77.0% versus 60.1%) and control (53.9% versus 28.2%) cohorts. A similar finding was found when restricting to dyspnoea codes present prior to an ECG in the diagnostic dataset (PH: 61.5% versus 44.4%, control: 38.0% versus 18.3%). To determine whether dyspnoea was under-coded, a machine learning algorithm for natural language processing termed “augmented curation” (supplementary methods SII) was employed at Mayo Clinic to perform sentiment analysis on the text of the electronic health records. In total, 95.8% of PH-likely patients at Mayo Clinic had confirmed presence of dyspnoea and 85.1% had confirmed presence of dyspnoea prior to an ECG in the diagnostic dataset. Augmented curation was not deployed at VUMC due to data access limitations.
Performance of the PH-EDA in detecting PH
When testing the trained algorithm on ECGs from Mayo Clinic and VUMC, a high degree of discrimination between PH-likely and control patients was achieved for test datasets in all four time windows (table 3). Using the Mayo Clinic diagnostic test set, the PH-EDA achieved an AUC of 0.92, with sensitivity and specificity of 85.5% and 83.8%, respectively (figures 3 and 4).
TABLE 3.
Performance summary of the pulmonary hypertension (PH) early detection algorithm (PH-EDA) in detecting PH in each of the time windows
| Mayo Clinic test sets | VUMC test sets | |||||||
|---|---|---|---|---|---|---|---|---|
| Diagnostic (1 month pre/post-diagnosis) | Pre-emptive (6–18 months pre-diagnosis) | ECGs 18–36 months pre-diagnosis | ECGs 36–60 months pre-diagnosis | Diagnostic (1 month pre-/post-diagnosis) | Pre-emptive (6–18 months pre-diagnosis) | ECGs 18–36 months pre-diagnosis | ECGs 36–60 months pre-diagnosis | |
| All patients | ||||||||
| PH patients# | 16 175 | 4985 | 4784 | 4135 | 6045 | 1880 | 1602 | 1321 |
| Control patients# | 87 998 | 87 998 | 87 998 | 87 998 | 24 256 | 24 256 | 24 256 | 24 256 |
| AUC | 0.92 (0.92–0.92) | 0.86 (0.86–0.86) | 0.82 (0.82–0.83) | 0.79 (0.78–0.79) | 0.88 (0.87–0.88) | 0.81 (0.80–0.82) | 0.78 (0.77–0.79) | 0.73 (0.72–0.75) |
| Sensitivity, % | 85.51 (85.11–85.94) | 70.84 (69.97–71.59) | 61.58 (60.72–62.46) | 53.04 (52.04–54.41) | 82.33 (81.65–83.11) | 68.51 (67.37–69.54) | 62.45 (60.66–64.02) | 56.37 (54.62–58.50) |
| Specificity, % | 83.81 (83.63–83.99) | 83.81 (83.63–83.99) | 83.81 (83.63–83.99) | 83.81 (83.63–83.99) | 76.95 (76.62–77.41) | 76.95 (76.62–77.41) | 76.95 (76.62–77.41) | 76.95 (76.62–77.41) |
| Diagnostic OR | 30.55 (29.49–31.74) | 12.58 (12.01–13.13) | 8.30 (7.99–8.58) | 5.85 (5.59–6.21) | 15.57 (14.70–16.59) | 7.27 (6.88–7.73) | 5.56 (5.14–5.93) | 4.32 (3.97–4.70) |
| PPV¶ | 0.49 (0.49–0.50) | 0.20 (0.19–0.20) | 0.17 (0.17–0.17) | 0.13 (0.13–0.14) | 0.47 (0.47–0.48) | 0.19 (0.18–0.19) | 0.15 (0.15–0.16) | 0.12 (0.11–0.12) |
| NPV¶ | 0.97 (0.97–0.97) | 0.98 (0.98–0.98) | 0.98 (0.98–0.98) | 0.97 (0.97–0.98) | 0.95 (0.94–0.95) | 0.97 (0.97–0.97) | 0.97 (0.97–0.97) | 0.97 (0.97–0.97) |
| PPV¶ (Mayo 2.87/VUMC 3.09% prev.) | 0.13 (0.13–0.14) | 0.11 (0.11–0.12) | 0.10 (0.10–0.10) | 0.09 (0.09–0.09) | 0.10 (0.10–0.10) | 0.09 (0.08–0.09) | 0.08 (0.08–0.08) | 0.07 (0.07–0.07) |
| NPV¶ (Mayo 2.87/VUMC 3.09% prev.) | 0.99 (0.99–1.00) | 0.99 (0.99–0.99) | 0.99 (0.99–0.99) | 0.98 (0.98–0.98) | 0.99 (0.99–0.99) | 0.99 (0.99–0.99) | 0.98 (0.98–0.99) | 0.98 (0.98–0.98) |
| Threshold | 0.308 | 0.308 | 0.308 | 0.308 | 0.308 | 0.308 | 0.308 | 0.308 |
| Dyspnoea subset | ||||||||
| PH patients# | 7155 | 2187 | 1811 | 1286 | 3719 | 1147 | 909 | 678 |
| Control patients# | 16 155 | 16 155 | 16 155 | 16 155 | 9208 | 9208 | 9208 | 9208 |
| AUC | 0.90 (0.90–0.90) | 0.85 (0.84–0.85) | 0.82 (0.81–0.83) | 0.79 (0.79–0.80) | 0.86 (0.85–0.87) | 0.80 (0.79–0.82) | 0.78 (0.76–0.79) | 0.73 (0.72–0.75) |
| Sensitivity, % | 82.78 (81.99–83.52) | 70.08 (68.44–71.62) | 64.46 (62.57–66.67) | 57.48 (55.25–59.40) | 78.71 (77.30–79.74) | 66.90 (65.09–68.74) | 61.72 (59.52–63.30) | 53.64 (50.82–56.11) |
| Specificity, % | 81.58 (81.15–82.01) | 81.58 (81.15–82.01) | 81.58 (81.15–82.01) | 81.58 (81.15–82.01) | 77.80 (77.07–78.41) | 77.80 (77.07–78.41) | 77.80 (77.07–78.41) | 77.80 (77.07–78.41) |
| Diagnostic OR | 21.30 (20.15–22.45) | 10.39 (9.64–11.38) | 8.05 (7.35–8.94) | 6.00 (5.44–6.44) | 12.97 (11.97–14.02) | 7.09 (6.43–7.79) | 5.66 (5.09–6.09) | 4.06 (3.60–4.51) |
| PPV¶ | 0.67 (0.66–0.67) | 0.34 (0.33–0.35) | 0.28 (0.27–0.29) | 0.20 (0.19–0.21) | 0.59 (0.58–0.60) | 0.27 (0.26–0.28) | 0.22 (0.21–0.23) | 0.15 (0.14–0.16) |
| NPV¶ | 0.91 (0.91–0.92) | 0.95 (0.95–0.96) | 0.95 (0.95–0.96) | 0.96 (0.96–0.96) | 0.90 (0.89–0.91) | 0.95 (0.95–0.95) | 0.95 (0.95–0.96) | 0.96 (0.95–0.96) |
| PPV¶ (Mayo 6.12/VUMC 6.23% prev.) | 0.23 (0.22–0.23) | 0.20 (0.19–0.20) | 0.19 (0.18–0.19) | 0.17 (0.16–0.17) | 0.19 (0.19–0.20) | 0.17 (0.16–0.17) | 0.16 (0.15–0.16) | 0.14 (0.13–0.15) |
| NPV¶ (Mayo 6.12/VUMC 6.23% prev.) | 0.99 (0.99–0.99) | 0.98 (0.98–0.98) | 0.97 (0.97–0.97) | 0.97 (0.97–0.97) | 0.98 (0.98–0.98) | 0.97 (0.97–0.97) | 0.97 (0.97–0.97) | 0.96 (0.96–0.96) |
| Threshold | 0.38 | 0.38 | 0.38 | 0.38 | 0.38 | 0.38 | 0.38 | 0.38 |
Data are presented as n, unless otherwise stated; data in parentheses are 95% confidence intervals. VUMC: Vanderbilt University Medical Center; AUC: area under the curve; PPV: positive predictive value; NPV: negative predictive value. #: numbers of patients sum to greater than the total number of patients included in this study, as patients could belong to more than one group, e.g. if they had an ECG within 1 month of right heart catheterisation (RHC) and another ECG 36 months before RHC; ¶: PPV and NPV vary according to the prevalence of PH in the given population: PH prevalence in the Mayo Clinical population, 2.87%, increased to 6.12% among patients with relevant dyspnoea International Classification of Disease (ICD) codes; PH prevalence in the VUMC population, 3.09%, increased to 6.23% among patients with relevant dyspnoea ICD codes.
FIGURE 3.
Receiver operating characteristic curves for the performance of the pulmonary hypertension (PH) early detection algorithm (PH-EDA) to identify patients with PH in ECGs across all four time windows of interest in a, c) all patients and b, d) patients with dyspnoea filter using the test datasets from a, b) Mayo Clinic and c, d) Vanderbilt University Medical Center (VUMC). AUC: area under the curve.
FIGURE 4.
Sensitivity and specificity of the pulmonary hypertension (PH) early detection algorithm (PH-EDA) to detect PH across all four time windows of interest, in a) all patients and b) patients with dyspnoea, using test sets from Mayo Clinic and Vanderbilt University Medical Center (VUMC).
Using the VUMC diagnostic test set, the AUC was 0.88, with sensitivity and specificity 82.3% and 77.0%, respectively. The PPVs for the detection of PH were 13% and 10% using the Mayo Clinic and VUMC diagnostic test sets, respectively (based on PH prevalence of 2.87% and 3.09% in the respective clinic populations); the NPVs were 99%. Performance metrics were highest using ECGs within 1 month of diagnosis, but the AUC remained above 0.80 up to 18 months before the date of diagnosis in both Mayo Clinic and VUMC test sets, with a minimum of 0.73 up to 60 months before diagnosis. The PH-EDA score increased as disease severity increased, as measured by either mPAP or TRV (supplementary figure S1). The performance of the PH-EDA was similar relative to the overall test sets when tested in the dyspnoea cohorts, but with an increase in PPV due to the increased PH prevalence (figures 3 and 4, table 3). Restricting to patients within the dyspnoea cohort who had PH diagnosed by RHC (mPAP >20 mmHg or ≥25 mmHg) did not substantially affect the performance of the algorithm (supplementary table S2). The PH-EDA, which was trained to detect broad PH, also performed similarly well across Pc-PH, Ipc-PH and Cpc-PH subtypes for patients with dyspnoea (supplementary table S3). Performance metrics were comparable using Mayo Clinic and VUMC data.
Comparison of PH-EDA performance to physician interpretation of PH-related ECG findings
The results shown in supplementary tables S4 and S5 demonstrate that the PH-EDA is more predictive than findings of PH-related abnormalities identified by physicians manually reviewing the ECG. Within the diagnostic window, confirmed presence of any of the ECG abnormalities was found in only 29.1% and 30.2% of the PH cohorts at Mayo Clinic and VUMC, respectively, and within only 30.3% and 30.7% of the cohorts with dyspnoea preceding the ECG.
Discussion
Patients with PH are often diagnosed late, and delayed diagnosis is associated with increased mortality [13, 14]. Therefore, earlier detection of PH is critical to improve patients’ quality of life and outcomes. Our study demonstrates the capability of an ECG-based algorithm to help address this unmet need. Here, in two large cohorts of PH patients from different institutions, the PH-EDA was able to accurately identify PH, with robust performance up to 5 years before diagnosis and was more predictive than physicians’ interpretation of ECGs. Notably, the PH-EDA had similar performance within the Pc-PH subgroup, which includes PAH, for which treatments are available.
The ECG has the advantage of being a relatively simple, ubiquitous and inexpensive test. It is often obtained in primary care when a patient presents with nonspecific symptoms, such as dyspnoea, as part of an initial evaluation [4, 5]. The ECG can provide clues to the presence of PH, but in clinical practice, signs are only present in advanced disease [16]. Thus, a clinician cannot exclude PH based on a normal ECG [27, 28]. Once suspicion of PH is raised, echocardiography is recommended to screen for PH [4, 5]. Therefore, the application of echocardiography with a focused evaluation of the right heart sooner in the course of disease represents a promising means to shorten the time to diagnosis. When identification of PH by waveform inspection was compared with the PH-EDA, we found that the algorithm was indeed able to identify PH when waveforms did not show visually detectable abnormalities (table 3, tables S4 and S5). Thus, automated analysis of ECGs, conducted soon after symptom onset to raise suspicion of PH, combined with clinical assessment and further diagnostic evaluation, has the potential to accelerate diagnosis.
The PH-EDA showed overall high performance at both medical centres, with an AUC of 0.92 and 0.88, a sensitivity of 85% and 82% and a specificity of 84% and 77% at Mayo Clinic and VUMC, respectively. When applied to ECGs up to 5 years before a clinical PH diagnosis, the algorithm remained a minimum of 0.79 at Mayo Clinic and 0.73 at VUMC. Importantly, the results suggest that the PH-EDA could detect key subgroups of PH (Pc-PH, Ipc-PH, Cpc-PH) at an earlier stage than the standard of care. The prevalence of PH was ∼3% in the two centres, increasing to ∼6% in the dyspnoea cohorts. Despite the low prevalence compared with other conditions, the PPV was 13% in the overall diagnostic test set and increased to 23% in the dyspnoea cohort at Mayo Clinic, suggesting the potential for a substantial benefit to patients via earlier detection. Notably, the data indicate that there is the opportunity to apply the PH-EDA to patients presenting with dyspnoea, a common early and nonspecific symptom of PH. In addition, we showed that applying natural language processing augments identification of patients with dyspnoea compared with ICD codes alone, suggesting a means to further increase the PPV.
A number of other AI-based predictive algorithms, including deep-learning algorithms, have also been investigated for detecting PH, using technologies such as cardiac magnetic resonance imaging, echocardiograms, healthcare utilisation and chest radiography [29–32]. Altogether, results are promising for AI-enhanced detection of PH.
Some recent studies have also developed deep-learning based ECG algorithms for detecting PH, and they support the feasibility of an ECG algorithm functioning as a PH screening tool [22, 33]. One study was based on 14 039 patients from two hospitals in Korea and reported AUCs of 0.86 and 0.90 in the validation and test sets, respectively [33]. A major limitation to this study is that only echocardiographic assessment of PH was used as the reference method, which is less rigorous than RHC, and the echocardiographic parameters used for labelling were not those recommended by the ESC/ERS guidelines [4–7]. The correct labelling of samples as disease and control is crucial for supervised machine/deep-learning algorithms to perform accurately. As ECG and echocardiography were only performed within a 4-week interval, the authors were unable to investigate the algorithm's ability to predict a later PH diagnosis.
Our work extends the work by Aras et al. [22], who developed and evaluated a deep-learning based ECG algorithm and showed high accuracy in detecting PH up to 2 years before a clinical diagnosis. In our study, the PH-EDA was developed using a larger population and was externally assessed at an independent institution using a different ECG machine original equipment manufacturer. Performance was also tested in differentiating between PH and control patients presenting with dyspnoea and using ECGs up to 5 years before PH diagnosis and compared against physician interpretation of an ECG.
However, there are noteworthy limitations to the study. First, retrospective data were used to develop the model, requiring patients to be labelled as PH-likely or PH-unlikely (control) using measurements from an RHC or echocardiogram, and there is potential selection bias for patients who underwent those diagnostic tests. Additionally, including patients defined by echocardiographic criteria could affect model performance, as these patients have a high or low probability of PH, respectively, rather than a confirmed diagnosis. However, inclusion of only patients who had an RHC performed introduces its own selection bias, as these are higher-risk patients for whom there is already a strong suspicion of PH. This population could significantly differ from patients for whom the PH-EDA would be most impactful, i.e. those without suspicion of PH at the time of their ECG. To address the potential bias in the RHC population, as well as to maximise the sample size for training, echocardiographic PH criteria using TRV were included to supplement the RHC-based definition. While echocardiographic criteria are less rigorous than RHC, it is reassuring that the PH-EDA performed well when restricted to PH patients defined only by RHC measurements. Although the VUMC cohort included a greater proportion of Black/African American patients than Mayo Clinic, it will be important for further research to include more diverse ethnic and racial groups to ensure the generalisability of the algorithm across ethnicities [34]. It will be important to evaluate performance of the algorithm in other subgroups, such as patients with different comorbidities, or different types of PH. The goal of the PH-EDA is to raise suspicion of PH along with appropriate clinical judgement.
Intervals up to 5 years before diagnosis were investigated because studies have reported the mean delay between symptom onset and PAH diagnosis to be between 2.5 and 3.9 years [13, 35, 36]. However, as patients had not undergone RHC during the earlier ECG intervals, it is not known whether patients had PH at these earlier stages or precursors to the disease. The possibility that the algorithm's performance in earlier intervals is due to inappropriate associations cannot be ruled out. Future studies will be aimed at assessment of outcomes in patients who are identified as “PH” or “non-PH'.
In conclusion, the PH-EDA was able to detect PH at the time of clinical diagnosis and 6–18 months prior at two independent medical centres. In addition, the algorithm had notable performance for detection up to 5 years before diagnosis. In a disease for which the diagnosis is often significantly delayed, adding this noninvasive screening tool to those already available, combined with sound clinical decision-making, has the potential to substantially accelerate the diagnosis and management of PH, thereby improving the lives of patients with this debilitating disease.
Supplementary material
Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.
Supplementary material ERJ-00192-2024.Supplement (474.9KB, pdf)
Shareable PDF
Acknowledgements
The authors thank Christel Chehoud (Janssen Research and Development, LLC, Raritan, NJ, USA) for key contributions at study inception. Medical writing and editorial support, under the direction of the authors, was provided by Jessica Beake (Beake Medicom Ltd, Eastbourne, UK) funded by Janssen Research and Development, LLC, a Johnson and Johnson company.
Footnotes
Ethics approval: This study was conducted in accordance with the Declaration of Helsinki and approved by the institutional review boards of the Mayo Clinic (ID: 19-012650) and VUMC (IRB #221350).
This article has an editorial commentary: https://doi.org/10.1183/13993003.01138-2024
Conflict of interest: H.M. DuBrock and R.P. Frantz report personal fees from Janssen Research and Development, LLC, a Johnson and Johnson company, outside the submitted work; and that additionally, Mayo Clinic may benefit financially from the algorithm described. T.E. Wagner, K. Carlson, C.L. Carpenter, S. Awasthi, M. Babu, A. Prasad, U. Yoo, R. Barve and V. Soundararajan are employees of nference, which received financial support from Janssen Research and Development, LLC, a Johnson and Johnson company, for work on this study. T.E. Wagner, S. Awasthi and V. Soundararajan also have patent application numbers 63/091,715 (non-invasive methods for detection of pulmonary hypertension) and 63/126,331 (systems and methods for diagnosing a health condition based on patient time series data) pending. K. Carlson, C.L. Carpenter, M. Babu, A. Prasad and R. Barve also have a patent application number 63/126,331 (systems and methods for diagnosing a health condition based on patient time series data) pending. Z.I. Attia, P.A. Friedman and S. Kapa have nothing to disclose apart from the fact that Mayo Clinic may financially benefit from the algorithm described in future. A.R. Hemnes, J. Annis and E.L. Brittain have no competing interests to report. S.J. Asirvatham reports personal fees from Abiomed, Atricure, Biotronik, Blackwell Futura, Boston Scientific, Medtronic, Medtelligence, Spectranetics, St. Jude, Zoll, Aegis, ATP, Nevro, Sanovas, Sorin Medical and FocusStart, outside the submitted work; and additionally, Mayo Clinic may financially benefit from the algorithm described in future. S.J. Asirvatham also receives royalties for work licensed through Mayo Clinic to a privately held company for contributions related to the use of nerve signal modulation to treat central, autonomic and peripheral nervous system disorders, including pain. Mayo Clinic receives royalties and owns equity in this company. The company does not currently license or manufacture any drug or device in the medical field. S.J. Asirvatham is a co-patent holder for technique to minimise coagulum formation during radiofrequency ablation. Products/techniques related to the above disclosures are not being discussed in this publication. M. Selej, P. Agron, E. Kogan, D. Quinn, P. Dunnmon and N. Khan are employees of Janssen Research and Development, LLC, a Johnson and Johnson company and own shares in the company.
Support statement: This research was funded by Janssen Research and Development, LLC, a Johnson and Johnson company and nference, Inc. The funding bodies were involved in the design of the study, the collection, analysis, and interpretation of data, and in writing/reviewing the manuscript. Funding information for this article has been deposited with the Crossref Funder Registry.
Data availability
Neither the data nor the algorithm can be shared due to legal reasons, as it is patent pending.
References
- 1.Hoeper MM, Humbert M, Souza R, et al. A global view of pulmonary hypertension. Lancet Respir Med 2016; 4: 306–322. doi: 10.1016/S2213-2600(15)00543-3 [DOI] [PubMed] [Google Scholar]
- 2.Rose-Jones LJ, McLaughlin VV. Pulmonary hypertension: types and treatments. Curr Cardiol Rev 2015; 11: 73–79. doi: 10.2174/1573403X09666131117164122 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Vonk Noordegraaf A, Chin KM, Haddad F, et al. Pathophysiology of the right ventricle and of the pulmonary circulation in pulmonary hypertension: an update. Eur Respir J 2019; 53: 1801900. doi: 10.1183/13993003.01900-2018 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Humbert M, Kovacs G, Hoeper MM, et al. 2022 ESC/ERS guidelines for the diagnosis and treatment of pulmonary hypertension. Eur Heart J 2022; 43: 3618–3731. doi: 10.1093/eurheartj/ehac237 [DOI] [PubMed] [Google Scholar]
- 5.Humbert M, Kovacs G, Hoeper MM, et al. 2022 ESC/ERS guidelines for the diagnosis and treatment of pulmonary hypertension. Eur Respir J 2023; 61: 2200879. doi: 10.1183/13993003.00879-2022 [DOI] [PubMed] [Google Scholar]
- 6.Galiè N, Humbert M, Vachiery JL, et al. 2015 ESC/ERS guidelines for the diagnosis and treatment of pulmonary hypertension. Eur Heart J 2016; 37: 67–119. doi: 10.1093/eurheartj/ehv317 [DOI] [PubMed] [Google Scholar]
- 7.Galiè N, Humbert M, Vachiery J-L, et al. 2015 ESC/ERS guidelines for the diagnosis and treatment of pulmonary hypertension. Eur Respir J 2015; 46: 903–975. doi: 10.1183/13993003.01032-2015 [DOI] [PubMed] [Google Scholar]
- 8.Braganza M, Shaw J, Solverson K, et al. A prospective evaluation of the diagnostic accuracy of the physical examination for pulmonary hypertension. Chest 2019; 155: 982–990. doi: 10.1016/j.chest.2019.01.035 [DOI] [PubMed] [Google Scholar]
- 9.Brown LM, Chen H, Halpern S, et al. Delay in recognition of pulmonary arterial hypertension: factors identified from the REVEAL registry. Chest 2011; 140: 19–26. doi: 10.1378/chest.10-1166 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Didden EM, Lee E, Wyckmans J, et al. Time to diagnosis of pulmonary hypertension and diagnostic burden: a retrospective analysis of nationwide US healthcare data. Pulm Circ 2023; 13: e12188. doi: 10.1002/pul2.12188 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Armstrong I, Billings C, Kiely DG, et al. The patient experience of pulmonary hypertension: a large cross-sectional study of UK patients. BMC Pulm Med 2019; 19: 67. doi: 10.1186/s12890-019-0827-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Maron BA, Humbert M. Finding pulmonary arterial hypertension – switching to offense to mitigate disease burden. JAMA Cardiol 2022; 7: 369–370. doi: 10.1001/jamacardio.2022.0011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Khou V, Anderson JJ, Strange G, et al. Diagnostic delay in pulmonary arterial hypertension: insights from the Australian and New Zealand pulmonary hypertension registry. Respirology 2020; 25: 863–871. doi: 10.1111/resp.13768 [DOI] [PubMed] [Google Scholar]
- 14.Klok FA, Barco S, Konstantinides SV, et al. Determinants of diagnostic delay in chronic thromboembolic pulmonary hypertension: results from the European CTEPH Registry. Eur Respir J 2018; 52: 1801687. doi: 10.1183/13993003.01687-2018 [DOI] [PubMed] [Google Scholar]
- 15.Maron BA, Abman SH. Translational advances in the field of pulmonary hypertension. Focusing on developmental origins and disease inception for the prevention of pulmonary hypertension. Am J Respir Crit Care Med 2017; 195: 292–301. doi: 10.1164/rccm.201604-0882PP [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.de Belen E, McConnell JW, Elwing JM, et al. Gaps in the care of pulmonary hypertension: a cross-sectional patient simulation study among practicing cardiologists and pulmonologists. J Am Heart Assoc 2023; 12: e026413. doi: 10.1161/JAHA.122.026413 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Ley L, Höltgen R, Bogossian H, et al. Electrocardiogram in patients with pulmonary hypertension. J Electrocardiol 2023; 79: 24–29. doi: 10.1016/j.jelectrocard.2023.02.007 [DOI] [PubMed] [Google Scholar]
- 18.Ko WY, Siontis KC, Attia ZI, et al. Detection of hypertrophic cardiomyopathy using a convolutional neural network-enabled electrocardiogram. J Am Coll Cardiol 2020; 75: 722–733. doi: 10.1016/j.jacc.2019.12.030 [DOI] [PubMed] [Google Scholar]
- 19.Attia ZI, Kapa S, Lopez-Jimenez F, et al. Screening for cardiac contractile dysfunction using an artificial intelligence-enabled electrocardiogram. Nat Med 2019; 25: 70–74. doi: 10.1038/s41591-018-0240-2 [DOI] [PubMed] [Google Scholar]
- 20.Makimoto H, Höckmann M, Lin T, et al. Performance of a convolutional neural network derived from an ECG database in recognizing myocardial infarction. Sci Rep 2020; 10: 8445. doi: 10.1038/s41598-020-65105-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Reed MJ, Grubb NR, Lang CC, et al. Multi-centre randomised controlled trial of a smartphone-based event recorder alongside standard care versus standard care for patients presenting to the emergency department with palpitations and pre-syncope: the IPED (Investigation of Palpitations in the ED) study. EClinicalMedicine 2019; 8: 37–46. doi: 10.1016/j.eclinm.2019.02.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Aras MA, Abreau S, Mills H, et al. Electrocardiogram detection of pulmonary hypertension using deep learning. J Card Fail 2023; 29: 1017–1028. doi: 10.1016/j.cardfail.2022.12.016 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Shin HC, Roth HR, Gao M, et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 2016; 35: 1285–1298. doi: 10.1109/TMI.2016.2528162 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Feeny AK, Chung MK, Madabhushi A, et al. Artificial intelligence and machine learning in arrhythmias and cardiac electrophysiology. Circ Arrhythm Electrophysiol 2020; 13: e007952. doi: 10.1161/CIRCEP.119.007952 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Attia ZI, Noseworthy PA, Lopez-Jimenez F, et al. An artificial intelligence-enabled ECG algorithm for the identification of patients with atrial fibrillation during sinus rhythm: a retrospective analysis of outcome prediction. Lancet 2019; 394: 861–867. doi: 10.1016/S0140-6736(19)31721-0 [DOI] [PubMed] [Google Scholar]
- 26.Youden WJ. Index for rating diagnostic tests. Cancer 1950; 3: 32–35. doi: [DOI] [PubMed] [Google Scholar]
- 27.Frost A, Badesch D, Gibbs JSR, et al. Diagnosis of pulmonary hypertension. Eur Respir J 2019; 53: 1801904. doi: 10.1183/13993003.01904-2018 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Kovacs G, Avian A, Foris V, et al. Use of ECG and other simple non-invasive tools to assess pulmonary hypertension. PLoS One 2016; 11: e0168706. doi: 10.1371/journal.pone.0168706 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Swift AJ, Lu H, Uthoff J, et al. A machine learning cardiac magnetic resonance approach to extract disease features and automate pulmonary arterial hypertension diagnosis. Eur Heart J Cardiovasc Imaging 2021; 22: 236–245. doi: 10.1093/ehjci/jeaa001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Leha A, Hellenkamp K, Unsöld B, et al. A machine learning approach for the prediction of pulmonary hypertension. PLoS One 2019; 14: e0224453. doi: 10.1371/journal.pone.0224453 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Kiely DG, Doyle O, Drage E, et al. Utilising artificial intelligence to determine patients at risk of a rare disease: idiopathic pulmonary arterial hypertension. Pulm Circ 2019; 9: 2045894019890549. doi: 10.1177/2045894019890549 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Kusunose K, Hirata Y, Tsuji T, et al. Deep learning to predict elevated pulmonary artery pressure in patients with suspected pulmonary hypertension using standard chest X ray. Sci Rep 2020; 10: 19311. doi: 10.1038/s41598-020-76359-w [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Kwon JM, Kim KH, Medina-Inojosa J, et al. Artificial intelligence for early prediction of pulmonary hypertension using electrocardiography. J Heart Lung Transplant 2020; 39: 805–814. doi: 10.1016/j.healun.2020.04.009 [DOI] [PubMed] [Google Scholar]
- 34.Noseworthy PA, Attia ZI, Brewer LC, et al. Assessing and mitigating bias in medical artificial intelligence: the effects of race and ethnicity on a deep learning model for ECG analysis. Circ Arrhythm Electrophysiol 2020; 13: e007988. doi: 10.1161/CIRCEP.119.007988 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.McGoon MD, Miller DP. REVEAL: a contemporary US pulmonary arterial hypertension registry. Eur Respir Rev 2012; 21: 8–18. doi: 10.1183/09059180.00008211 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Strange G, Gabbay E, Kermeen F, et al. Time from symptoms to definitive diagnosis of idiopathic pulmonary arterial hypertension: the delay study. Pulm Circ 2013; 3: 89–94. doi: 10.4103/2045-8932.109919 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.
Supplementary material ERJ-00192-2024.Supplement (474.9KB, pdf)
This one-page PDF can be shared freely online.
Shareable PDF ERJ-00192-2024.Shareable (1.4MB, pdf)
Data Availability Statement
Neither the data nor the algorithm can be shared due to legal reasons, as it is patent pending.