Automatic echocardiographic evaluation of the probability of pulmonary hypertension using machine learning

Abstract

Echocardiography, a simple and noninvasive tool, is the first choice for screening pulmonary hypertension (PH). However, accurate assessment of PH, incorporating both the pulmonary artery pressures and additional signs for PH remained unsatisfied. Thus, this study aimed to develop a machine learning (ML) model that can automatically evaluate the probability of PH. This cohort included data from 346 (275 for training set and internal validation set and 71 for external validation set) patients with suspected PH patients and receiving right heart catheterization. Echocardiographic images on parasternal short axis‐papillary muscle level (PSAX‐PML) view from all patients were collected, labeled, and preprocessed. Local features from each image were extracted and subsequently integrated to build a ML model. By adjusting the parameters of the model, the model with the best prediction effect is finally constructed. We used receiver‐operating characteristic analysis to evaluate model performance and compared the ML model with the traditional methods. The accuracy of the ML model for diagnosis of PH was significantly higher than the traditional method (0.945 vs. 0.892, p = 0.027 [area under the curve [AUC]]). Similar findings were observed in subgroup analysis and validated in the external validation set (AUC = 0.950 [95% CI: 0.897−1.000]). In summary, ML methods could automatically extract features from traditional PSAX‐PML view and automatically assess the probability of PH, which were found to outperform traditional echocardiographic assessments.

INTRODUCTION

Pulmonary hypertension (PH) is a serious disease that causes pulmonary vasoconstriction and remodeling due to various causes, causes right heart failure and even death by increasing pulmonary vascular pressure and resistance. It is a major global chronic disease that endangers human health. Growing evidence suggested that even mild elevations in pulmonary artery pressure estimated with echocardiography were linked to increased mortality. The earlier PH was diagnosed, the greater the variety of treatment options and the greater the benefit to the patient. ¹ The gold standard for pulmonary artery pressure assessment is invasive right heart catheterization (RHC), which requires hospitalization at a specialized pulmonary vascular center, and thus limits its widespread use. ² Moreover, it was unrealistic to perform RHC in every patient. To enable timely diagnosis of PH, widely available noninvasive tools would be ideal. ³

Transthoracic echocardiography had become the main method for early screening of PH because its advantages of noninvasiveness and easy popularization. ⁴ The detection of PH by echocardiography usually relies on tricuspid regurgitation or pulmonary regurgitation using the simplified Bernoulli equation. ⁵ However, there have been a number of earlier studies that indicate that echocardiographic estimates of PA pressure are frequently inaccurate. ⁶ , ⁷ The estimated error ranged from −20 to +25 mmHg compared to data from RHC. ⁸ , ⁹ , ¹⁰ This was mainly contributed to the morphological variation of reflux flow and reflux caliber, and right ventricular volume load and systolic function, intra‐ and interexaminer variability, and other factors. ¹¹ On the other hand, using the estimated pressure alone is not sufficient to diagnose PH. As suggested by the 2022 ESC PH guidelines, a suspected PH includes estimating the systolic pulmonary artery pressures (sPAP) and detecting additional signs suggestive of PH, like signs of RV overload and/or dysfunction. ² Although additional signs could be obtained from traditional methods, it highly relies on experienced investigators and is a time‐consuming process. ¹²

Owing to the development of information technology, machine learning (ML) and deep learning models have been widely used to diagnose diseases. ⁹ As reported by Chih‐Min Liu et al., a deep learning ECG‐trained model was constructed to diagnose PH, having a high accuracy. ¹³ Furthermore, detection and prognostication of PH could be improved using a deep learning echocardiography model. However, these models have to integrate multiple aspects of information, not taking the guideline‐suggested signs for PH into consideration. ¹⁴ In addition, they analyzed patients with certain groups of PH, limiting the generalizability of their findings. Thus, in the current study, we aimed to construct models to automatically assess the probability of PH in a more generalized PH population.

METHOD

Patients

We included patients with suspected PH estimated by echocardiography in Guangdong Provincial People's Hospital and Shanghai Pulmonary Hospital from October 2020 to November 2022. Patients whose images were too poor to train the ML model or whose RHC was performed more than 24 h beyond the echocardiogram were excluded. This retrospective study included echocardiographic data collected anonymously from an echocardiography database of the Guangdong Provincial People's Hospital and Shanghai Pulmonary Hospital. To overcome the issue of generalizability, a separate external validation set of 71 patients was gathered who were performed echocardiography and RHC within 24 h from other two independent hospitals (The First Affiliated Hospital of Guangzhou Medical University and Fuwai Hospital Chinese Academy of Medical Sciences).

Echocardiography acquisition

A commercially available echocardiography system (EPIQ 7C Philips or Aloka 880 HITACHI) and 3.5 MHz transducers were used on all patients in the left lateral decubitus position. To be eligible for inclusion, RHC and echocardiography had to be performed within 24 h. All echocardiographic recordings were performed by experienced operators according to the recommendations of the European Association of Echocardiography. ¹⁵ We acquired images of parasternal short axis‐papillary muscle level (PSAX‐PML) view which required that the contours of the left and right ventricle (RV) were included in three cardiac cycles. Patients with poor image quality and failure to complete RHC within 24 h would be excluded. We measured tricuspid and pulmonary valve regurgitation, then estimated sPAP and mean pulmonary artery pressure (mPAP) according to Bernoulli's equation. ¹⁶ In addition, RV fractional area change (FAC), RV wall thickness (RVWT), tricuspid annular plane systolic excursion (TAPSE), and tricuspid annular peak systolic velocity (S') were also measured.

RHC and clinical assessment

Weight and height were routinely measured and recorded before RHC. RHC was performed in the catheter laboratory under electrocardiographic supervision. By placing a 6‐French vascular sheath through the right femoral vein, a 6‐French MPA 2 catheter (Cordis Inc.) was inserted into the right heart system. Under fluoroscopy, the catheter was manipulated to the correct position and pressures in various parts of the right heart system were measured. At atmospheric pressure, the transducers were calibrated to zero before measuring pressure. Vascular resistance is calculated using the formula of: “(mPAP‐PCWP)/cardiac output.” ¹⁷ , ¹⁸ PH was defined as one in which the mPAP at RHC was more than 20 mmHg. ¹¹

Image preprocessing

Figure 1 shows the steps of ML model construction. Labeling a single section of the left ventricle (LV) and RV and then using findContours function of openCV to detect the contour of the tag to train the model detect the contour of left and RV. Then the drawContours function is used to draw the contour of the left and RV throughout the rest of the cardiac cycle, getting labeled timestamps and unlabeled timestamps. The method is the minimum bounding rectangle method. Labeled timestamps and unlabeled timestamps were selected to form new 3D data set 1 and 3D data set 2, newly formed 3D data set 1 and 3D data set 2 were sliced and then saved as 2D data set 1 and 2D data set 2. A neural network was trained on the 2D data set 1, divided into a training set and a validation set according to the 8:2 ratio. Reproducibility was evaluated for the labeled area in 20 randomly selected subjects. The intraclass correlation coefficient for interobserver reproducibility was 0.993 (95% CI: 0.980−0.998), and the intraclass correlation coefficient for intraobserver reproducibility was 0.995 (95% CI: 0.981−0.999).

Figure 1.

Proposed deep learning workflow. Manual labeling, preprocessing steps for short axis single section images, machining steps visualization of learnt factors and features, and feature maps.

Feature extraction

The long axis and short axis were determined by drawing the outer rectangle of the two labels, and the two‐dimensional data of the two labels were obtained. After that, all timestamps of each patient were traversed. The long axis, short axis, area, and circumference of LV and RV and their ratio were calculated for all time of each patient (Figure 2).

Figure 2.

The process of feature extraction and integration of machine learning model. On the left is the feature extraction network, which is used for feature extraction (obtaining local features and performing image‐level classification) to obtain abstract semantic features. On the right is the feature fusion network, which uses the encoded abstract features to restore the process of the original image size and get feature result.

Feature integration and training model

Features obtained from previous steps were preprocessed and put into the neural network for training. ML performed three methods on the extracted features, such as the ratio of LV to RV maximal area: Linear regression, LightGBM, and CatBoost, resulting in three prediction models.

Model selection and optimization

After comparing the prediction results, ML model by CatBoost was finally selected as the prediction method, setting cross‐validation to 50%. To balance the model efficiency and learning time, we optimized the iterations from 1000 to 100, the learning rate from 0.03 to 0.05, and the depth from 6 to 10.

Statistical analysis

We used t‐test analysis to compare continuous variables between patients with and without PH. Receiver operating characteristic (ROC) of echocardiography indices was used to evaluate their diagnostic power of the traditional method and ML models. ROC curve analysis results are presented as area under the curve (AUC). We use paired comparison method for the comparisons of two ROC curves.

In addition, considering that the presence of a shunt may be an additional contributing factor to the echocardiographic image features of PH patients, a subgroup analysis was performed in patients with and without congenital heart disease (CHD), including atrial septal defect, ventricular septal defect, patent ductus arteriosus.

A p‐value < 0.05 was determined to be statistically significant. All the above‐mentioned statistical were analyzed by using R (http://www.R-project.org) and Empower Stats software (http://www.empower.stats.com, X&Y solutions, Inc.).

RESULTS

Patients

A total of 346 patients with suspected PH were identified as having both echocardiography and RHC within 24 h (see Figure 3). The demographics, echocardiography, RHC data for across patients of training set, and internal validation set are summarized in Table 1. Among 346 patients, 240 patients were with PH and 106 patients were without. In patients with PH, 77 were diagnosed patients were diagnosed with CHD, 63 with PH associated with pulmonary artery obstruction, 58 patients with IPAH, 33 with PH associated with connective tissue disease, and 7 with PH associated with left heart disease (Table 2). Among patients without PH, 73 patients had CHD, 16 had chronic thrombo‐emoblic disease, 8 had connective tissue disease, 2 had left heart disease, and 7 had severe tricuspid regurgitation. Generally, patients with PH had higher mean right atrium pressure, mPAP, sPAP, pulmonary vascular resistance (p < 0.001) than patients without PH. In addition, patients with PH had lower TAPSE, S’, FAC, and higher RVWT (all p < 0.001). The same situation occurred in patients with CHD.

Figure 3.

Patient flow diagram. PH, pulmonary hypertension; RHC, right heart catheterization.

Table 1.

Demographics, right heart catheter, and echocardiography

	All			CHD
	No PH (n = 93)	PH (n = 182)	p	No PH(n = 73)	PH (n = 77)	p
Demographics
Age (years)	40 ± 15	41 ± 15	0.195	40 ± 15	42 ± 15	0.555
Sex female	70 (75%）	138 (76%）	0.919	56 (77%)	56 (73%)	0.575
Echocardiography
LVEF (%)	66 ± 4	67 ± 7	0.169	66 ± 4	66 ± 8	0.632
RVW (mm)	3.8 ± 1.0	6.0 ± 2.2	<0.001	3.8 ± 0.9	6.2 ± 2.4	<0.001
TAPSE (mm)	23.4 ± 6.2	18.2 ± 5.3	<0.001	24.4 ± 5.7	19.8 ± 5.9	<0.001
S' (cm/s)	13.4 ± 3.1	11.2 ± 2.8	<0.001	13.9 ± 3.0	11.9 ± 3.0	<0.001
FAC (%)	42 ± 4	35 ± 9	<0.001	43 ± 4	37 ± 8	<0.001
Echo‐sPAP (mmHg)	35 ± 13	70 ± 29	<0.001	35 ± 14	74 ± 28	<0.001
Echo‐mPAP (mmHg)	18 ± 8	38 ± 18	<0.001	18 ± 8	41 ± 22	<0.001
Right heart catheter
mRAP (mmHg)	3 ± 3	6 ± 4	<0.001	3 ± 3	6 ± 4	<0.001
sPAP (mmHg)	30 ± 7	80 ± 25	<0.001	30 ± 7	81 ± 27	<0.001
mPAP (mmHg)	14 ± 4	45 ± 16	<0.001	14 ± 4	45 ± 17	<0.001
PVR (wood)	1.7 ± 0.8	9.4 ± 6.3	<0.001	1.3 ± 0.7	7.7 ± 5.8	<0.001
PAWP (mmHg)	6 ± 3	9 ± 5	0.001	6 ± 3	8 ± 5	0.041

Note: This table did not contain the external validation set.

Abbreviations: CHD, congenital heart disease; FAC, fractional area change; LVEF, left ventricular ejection fraction; mPAP, echocardiography estimated mean pulmonary arterial pressure; mRAP, mean right atrium pressure; PAWP, pulmonary arterial wedge pressure; PH, pulmonary hypertension; PVR, pulmonary vascular resistance; RAP, mean right atrial pressure; RVWT, right ventricular wall; S’, tricuspid valve annulus peak systolic velocity; sPAP, systolic pulmonary arterial pressure; TAPSE, tricuspid annular plane systolic excursion.

Table 2.

Detailed diagnostic information of the patient.

	All
	No PH (n = 106)	PH (n = 240)
Diagnosis
CHD	73	77
IPAH	‐	58
CTD	8	33
LHD	2	7
CTED	16	63
Othera	7	2

Abbreviations: CHD, congenital heart disease; CTD, connective tissue disease; CTED, chronic thrombo‐embolic disease; IPAH, idiopathic pulmonary arterial hypertension; LHD, left heart disease; PH, pulmonary hypertension.

^a Patients with severe tricuspid regurgitation.

Feasibility

Only 25 patients had insufficient image quality for deep learning, accounting for 5.9% of the studied patients. The excluded patients were mainly due to excessive chest fat or narrow intercostal spaces, causing a poor acoustic window. In general, the model we built is feasible, and it is not overly demanding on the quality of the images.

Learnt features

According to each patient's echocardiographic image, all frames of echocardiographic images of a single patient were analyzed. Features were extracted from the segmented images using ML. The extracted features included the maximum ratio of the long axis to the short axis of LV, the ratio of the minimum area of the LV to the minimum area of the RV. Prediction models were then constructed from the obtained features using different ML methods.

Diagnostic utility

As presented in Figure 4, the deepmachine learning model significantly outperformed traditional echocardiographic measurements (AUC = 0.945, 95% CI: 0.917−0.974 vs. 0.892, 95% CI: 0.852−0.933, p = 0.027). The diagnostic value using ML was similarly excellent in patients with CHD‐PH or non‐CHD‐PH (AUC = 0.929, 95% CI: 0.889−0.969). A separate external validation group of 71 patients included 58 with PH and 13 without PH. The AUC value for external validation of the deep learning model was 0.950 (95% CI: 0.897−1.000) for differentiation of no PH from PH.

Figure 4.

Receiver operating characteristic curve analysis. ML model and TRV in internal validation (a) and external validation (b) for diagnosing pulmonary hypertension. AUC, area under the curve; ML, machine learning.

DISCUSSION

Accurate and timely diagnosis of diseases is the basis for effective clinical treatment. Choosing a diagnostic method with high accuracy is an important guarantee to improve the accuracy of disease diagnosis. Our study demonstrates the performance of a ML model based on PSAX‐PML view in assessing the probability of PH and found that ML models outperformed the traditional echocardiographic assessments. As reported by Michele D'Alto et al., maximal tricuspid regurgitation velocity, LV eccentricity index, pulmonary artery diameter, and so forth can all be used to predict PH, with the highest accuracy reaching 90.9%. ¹⁹ However, one single feature could easily misclassify the patients since the measurements of these parameters were sometimes subjective and sonographer‐dependent. A study about lack of tricuspid regurgitation doppler signal and PH by invasive measurement showed that invasively confirmed PH was present in 47% of patients without a reported TRV versus 68% in those with a reported TRV (p < 0.001). ²⁰ Thus, the newest PH guidelines have proposed that a suspected PH should include the assessment of sPAP and additional signs suggestive of PH.

With the development of artificial intelligence, more and more studies suggested that the accuracy from ML or deep learning model is far superior to conventional echocardiography. The accuracy and stability of LVEF automatically measured by the deep learning model was better than that of junior doctors. ²¹ A deep learning model by Pandey A could integrates multidimensional echocardiographic data to characterize the severity of diastolic dysfunction and identify a specific subgroup of patients with HFpEF, which was hard for conventional echocardiography. ²²

Our study further supported the applications of ML in PH. The ML model learns about the cardiac chamber features after left and RV interactions in patients with PH, then quantifies the features and finally combines multiple learned features. ²³ Although previous studies have suggested an improvement in diagnostic performance using ML models, such as ML algorithms based on Logit Boost can better identify precapillary PH and post‐capillary PH. ²⁴ Studies by Gerhard‐Paul Diller has shown that deep learning can not only accurately diagnose IPAH, but also predict the prognosis of patients. ¹⁴ But they need more echocardiographic information to build the model and were investigated in a more restricted patient groups. On the other hand, our ML model not only learned about the spatial features of the left and RVs in PH patients, but also their temporal features, such as the changes in the size of the two chambers during systole and diastole. Furthermore, subgroup analysis across the presence or absence of CHD did not have much impact on the predictive performance of our ML model. More importantly, analysis from the external validation data set yield similar results. This means that our ML models could be applied to more generalized patient groups with excellent performance in diagnosing PH.

Notably, some of the extracted features have also been validated clinically, including the eccentricity index (the minimum value of the ratio of the long axis to the short axis of the LV), d‐sign, and so forth. ²⁵ However, using only one learned feature could not obtain a satisfactory performance in predicting PH. This further suggested that PH is a complex disease that required multidimensional assessment. By extracting features from traditional echocardiographic images (e.g., PSAX‐PML), the ML model could incorporate the extracted features and automatically assess the probability of PH, with an accuracy of 0.895. This method could to some extent mitigate the subjective assessment of the pulmonary pressures by inexperienced physicians and help enable large‐scale population screening for PH in higher accuracy.

However, we do not intend nor recommend the use of ML‐based models as a replacement for echocardiographic assessment of patients with PH. After all, echocardiography, as a noninvasive test recommended by ESC, in addition to diagnosing PH, its ability to assess the overall function of the patient's heart and the etiology of pulmonary arterial hypertension is beyond the coverage of current deep learning models. ²⁶

Limitations

The number of patients in this study was relatively small, although the diagnostic performance of the trained ML model was very good. The consistency of our model's predictions would be more satisfactory if the sample size were larger. In addition, although we included PH patients with different etiologies, the cases in specific subgroups were insufficient and further subgroup analyses in individual subgroups were unavailable. Finally, the studied patients were all Asian ethnicity, and the applications of this method to other ethnic groups remains unknown and warrants further investigations.

In summary, ML methods could automatically extract features from traditional PSAX‐PML view and automatically assess the probability of PH, which were found to outperform traditional echocardiographic assessments.

AUTHOR CONTRIBUTIONS

Hongwen Fei conceptualized the study. Hongwen Fei and Zuwei Liao designed experiments. Zuwei Liao, Shangwei Ding, Qinhua Zhao, Yong Jiang, and Lan Wang acquired echocardiography images and measured echocardiographic data. Zuwei Liao and Taoran Huang and Lifang Yang labeled echocardiography images and analyzed data. Xiaowei Xu, Kaikai Liu, Erlei Zhang, and Yu Zhang preprocessed echocardiography images which had been labeled and built the machine learning model. Caojin Zhang and Dongling Luo performed right heart catheterization for the patients. Hongwen Fei and Zuwei Liao drafted the manuscript and revised it. All authors approved the final version of the manuscript submitted for publication.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflict of interest.

ETHICS STATEMENT

The study was approved with waiver of informed consent by the Ethics Committee of Guangdong Provincial People's Hospital.

Liao Z, Liu K, Ding S, Zhao Q, Jiang Y, Wang L, Huang T, Yang L, Luo D, Zhang E, Zhang Y, Zhang C, Xu X, Fei H. Automatic echocardiographic evaluation of the probability of pulmonary hypertension using machine learning. Pulm Circ. 2023;13:e12272. 10.1002/pul2.12272