Using Artificial Intelligence for Rheumatic Heart Disease Detection by Echocardiography: Focus on Mitral Regurgitation

Abstract

Background

Identification of children with latent rheumatic heart disease (RHD) by echocardiography, before onset of symptoms, provides an opportunity to initiate secondary prophylaxis and prevent disease progression. There have been limited artificial intelligence studies published assessing the potential of machine learning to detect and analyze mitral regurgitation or to detect the presence of RHD on standard portable echocardiograms.

Methods and Results

We used 511 echocardiograms in children, focusing on color Doppler images of the mitral valve. Echocardiograms were independently reviewed by an expert adjudication panel. Among 511 cases, 229 were normal, and 282 had RHD. Our automated method included harmonization of echocardiograms to localize the left atrium during systole using convolutional neural networks and RHD detection using mitral regurgitation jet analysis and deep learning models with an attention mechanism. We identified the correct view with an average accuracy of 0.99 and the correct systolic frame with an average accuracy of 0.94 (apical) and 0.93 (parasternal long axis). It localized the left atrium with an average Dice coefficient of 0.88 (apical) and 0.9 (parasternal long axis). Maximum mitral regurgitation jet measurements were similar to expert manual measurements (P value=0.83) and a 9‐feature mitral regurgitation analysis showed an area under the receiver operating characteristics curve of 0.93, precision of 0.83, recall of 0.92, and F1 score of 0.87. Our deep learning model showed an area under the receiver operating characteristics curve of 0.84, precision of 0.78, recall of 0.98, and F1 score of 0.87.

Conclusions

Artificial intelligence has the potential to detect RHD as accurately as expert cardiologists and to improve with more data. These innovative approaches hold promise to scale echocardiography screening for RHD.

Nonstandard Abbreviations and Acronyms

AP4

apical 4 chamber

CNNs

convolutional neural networks

MR

mitral regurgitation

PLAX

parasternal long axis

RHD

rheumatic heart disease

SVM

support vector machines

WHF

World Heart Federation

Clinical Perspective.

What Is New?

• The authors developed highly accurate machine and deep learning models for rheumatic heart disease detection based on mitral regurgitation by echocardiography.

What Are the Clinical Implications?

• Echocardiography is the gold standard for diagnosis of early rheumatic heart disease.

• Artificial intelligence could have important implications for scaling rheumatic heart disease detection in endemic regions by nonexpert health care workers.

Rheumatic heart disease (RHD) affects about 40 million people globally, claims nearly 300 000 lives each year, and is the primary cause of morbidity and mortality from heart disease in children and young adults. ¹ RHD is a disease of inequality, primarily affecting those living in low‐ and middle‐income countries. Globally, most patients with RHD are diagnosed late, only once symptoms of advanced cardiac disease or complications develop. However, opportunities exist for earlier detection. Identification of children with latent RHD by echocardiography, before onset of symptoms, provides an opportunity to initiate secondary antibiotic prophylaxis, which prevents additional superficial Streptococcus A infections, the primary driver of RHD. ² Recently, the GOAL (Gwoko Adunu pa Lutino) randomized controlled trial showed that secondary prophylaxis with monthly intramuscular penicillin prevents the progression of RHD among children with screen‐detected RHD, providing further support for the role of early identification in RHD control. ²

Scale‐up of echocardiographic screening for early RHD detection has challenges. Task‐shifting screening to nonphysician health care workers, use of portable devices, and use of simplified screening protocols have been successful in focused research studies, ³ but deploying training and competency at scale has not been attempted. Novel portable imaging technology in conjunction with artificial intelligence (AI) could transform our ability to conduct echocardiographic screening at scale, providing tools of real‐time image interpretation to facilitate screening across all levels of the health care system. In fact, there have been early learnings in the use of machine‐learning and deep learning methods to improve screening and diagnosis of RHD. Peck et al ⁴ demonstrated that AI guidance with color Doppler echocardiography could enable high‐quality image acquisition by nonphysician echocardiogram screeners in an RHD endemic region. Although this is important for increased echocardiogram screening worldwide, it does not address the limited access to those with the expertise to diagnose RHD based on these images acquired. This highlights the importance of our work for automated diagnosis of RHD using AI.

AI is defined as any technique that enables machines to simulate human intelligence. AI is an umbrella term including machine learning algorithms and deep learning processes as subsets. ⁵ Machine learning is a subset of AI that allows machines to operate at the level of human intelligence to replicate information based on set criteria. Deep learning is a more sophisticated subset that can go beyond the human capability to learn and adapt using vast amounts of data.

AI is promising in its early stages of identifying mitral regurgitation (MR), present in 80% to 90% of children with early RHD. ⁶ , ⁷ , ⁸ Here, we first build on this work, using automated methods for RHD detection based on the 2012 World Heart Federation (WHF) criteria for diagnosis of RHD (Table 1). ⁹ Our machine learning methods enable identification of MR and analyze components of the MR jet for RHD detection. Then, the deep learning method goes beyond specific MR measurements to diagnose RHD using holistic image interpretation. Both machine learning methods for MR jet characterization and deep learning methods were evaluated in parallel to establish if one method is superior in RHD detection in the same data cohort.

Table 1.

Adapted From the 2012 World Heart Federation Criteria for Echocardiographic Diagnosis of Rheumatic Heart Disease ⁹

Echocardiographic criteria for patients aged <20 years

Definite RHD (either A, B, C, or D)

A) Pathologic MR and at least 2 morphologic features of RHD of the mitral valve

B) MS mean gradient ≥4 mm Hg

C) Pathologic AR and at least 2 morphological features of RHD of the aortic valve

D) Borderline disease of both the aortic valve and mitral valve

Borderline RHD (either A, B, or C)

A) At least 2 morphological features of RHD of the mitral valve without pathological MR or MS

B) Pathologic MR

C) Pathologic AR

AR indicates aortic regurgitation; MR, mitral regurgitation; MS, mitral stenosis; and RHD, rheumatic heart disease.

Methods

The data that support the findings of this study are available from the corresponding author upon reasonable request; the data containing echocardiogram images will be considered for sharing on a case‐by‐case basis. Echocardiograms included in this study were obtained in 2020 during completion evaluations for the GOAL trial, a 2‐year randomized controlled trial to evaluate the impact of secondary prophylaxis with intramuscular penicillin G benzathine, as compared with no prophylaxis, in Ugandan children and adolescents with latent (borderline and mild definite) RHD. ² The methods of acquisition, interpretation, and adjudication of the echocardiograms in this study have been previously described. ¹⁰ In brief, qualified echocardiographers obtained a standard 13‐view protocol on Vivid Q and Vivid IQ fully functional echocardiography machines (General Electric, Milwaukee, WI) with ECG gating for both the enrollment and final studies. All studies in the GOAL trial were performed using consistent standard color Doppler scales between 60 and 70 cm/s². A 4‐member adjudication panel reviewed enrollment and final echocardiograms to determine a consensus diagnosis that included presence or absence of RHD, RHD subtype category,¹⁰ and length of MR jet (longest present) in either parasternal long axis (PLAX) or apical 4 chamber (AP4). Ethics approval was obtained from Makerere University, Kampala Uganda (REC 2018–048) and Children's National Hospital (P000010408) in Washington, DC. Written informed consent was obtained from a parent or guardian of every child, as well as written assent for all participants over the age of 8 years.

As a significant majority of children diagnosed with RHD using the 2012 WHF criteria ⁹ have pathologic MR and normal valve morphology, ⁸ , ¹¹ our primary goal was to train the machine learning algorithm to detect MR and perform MR jet analysis. Because our methods ado not account for abnormal morphology of the mitral or aortic valve or aortic insufficiency, RHD positive cases only met criteria based that on the presence of aortic insufficiency or abnormal valve morphology alone were removed from testing sets. Additionally, our approach did not include spectral Doppler images for evaluation of the WHF spectral Doppler criteria for pathologic MR (MR peak velocity >3 and pansystolic jet). Based on the 2012 WHF criteria for echocardiographic diagnosis of RHD, this approach would identify patients in the Definite A or Definite D categories as well as Borderline B (Table 1). Our deep learning method aimed to go beyond strict measurements of MR length to diagnose RHD based on holistic image interpretation.

Image Set

View determination analysis used black‐and‐white and color Doppler images and the rest of the analysis used only color Doppler images. As the 2012 WHF criteria for RHD classifies MR jet length of >2 cm as 1 marker of pathologic MR, ⁹ >2 cm was the length used as the threshold for RHD positive cases. Of the 511 echocardiograms, 282 had RHD (limited to Borderline B or Definite A or D cases) and 229 were normal. These cases were then divided to create and test the AI model for RHD detection. Of these cases 20% (100 subjects; 48 normal and 52 RHD cases) were set aside for testing, then the remaining 80% (411 subjects; 181 normal and 230 RHD cases) were divided into training and validation with a ratio of 80:20 using 5‐fold cross‐validation. Both methods were evaluated in parallel to establish if one method is superior in RHD detection.

Image Harmonization

To account for inherent imaging differences such as image timing and spatial resolution, the data were first harmonized to improve consistency in MR detection. To do so, neural networks were used to detect the PLAX and AP4 views, identify systole, and localize the left atrium (Figure 1B). ¹² Each view identification and selection of frames during systole was performed using a ResNet‐50 convolutional neural network. ¹³ Although frame selection for model training was done on echocardiograms with ECG capabilities, the algorithm was expanded to not require ECG gating. This made its use compatible with handheld echocardiogram machines for field deployment. Localization of the left atrium was performed using convolutional neural networks with LinkNet ¹⁴ , ¹⁵ structure.

Figure 1. Overview of the proposed deep learning models for rheumatic heart disease detection.

A, Initial echocardiogram harmonization includes view identification, left atrium localization, and systole detection using convolutional neural networks. Two deep learning models are then combined to create an ensemble model with ultimate outputs of positive screening for rheumatic heart disease based on mitral regurgitation or normal valvular function. B, 3D Convolutional neural networks and Transformer models for deep learning method. A4CC indicates apical 4 chamber with color Doppler; CNNs, convolutional neural networks; FC, fully connected; Gelu, Gaussian error linear unit; Norm, normalization; PLAXC, parasternal long axis with color Doppler; and RHD, rheumatic heart disease. (B) Reproduced from Roshanitabrizi et al ¹² with permission. Copyright ©2022 Springer Nature.

Machine Learning Approach

The first objective of the machine learning approach was to compare the MR length annotated by expert cardiologists to our automated method measurements. To detect RHD using MR jet analysis, we first identified the location of the maximum MR jet by assessing its proximity to the mitral valve and its blue color intensity. The second objective was to use multiple features of the MR jet to predict RHD. From 2 views we extracted 39 morphological features that describe the MR jet's pattern, velocity, duration, and length (Table 2). We used cross‐validation and linear support vector machines (SVM) to rank the 39 computed features according to their importance using the method proposed by Brank et al. ¹⁶ Afterwards, we performed a sequential search to identify the optimal combination of features that resulted in the highest accuracy on the validation data set. Notably, this search yielded 9 features ¹⁷ (highlighted features in Table 2), which collectively provided the maximum accuracy. RHD predictions were done using a SVM ¹⁸ , which is a classic machine learning method for regression and classification tasks with multidimensional data. Before feeding the data into the model, we performed normalization to ensure that all features were in the range [0, 1].

Table 2.

Thirty‐nine Morphological Features Describing the Mitral Regurgitation Jet's Pattern, Velocity, Duration, and Length

Area of the atrium on the A4CC view

Area of the MR jet on the A4CC view

Ratio of the MR jet area to the atrium area on the A4CC view

Area of the atrium on the A4CC view normalized by age

Area of the MR jet on the A4CC view normalized by age

MR jet duration on the A4CC view*

Average of the MR jet velocity on the A4CC view

Maximum of the MR jet velocity on the A4CC view

Minimum of the MR jet velocity on the A4CC view

SD of the MR jet velocity on the A4CC view*

Skewness of the MR jet velocity on the A4CC view*

Kurtosis of the MR jet velocity on the A4CC view

Entropy of the MR jet velocity on the A4CC view

Maximum MR jet length on the A4CC view

Maximum MR jet length on the A4CC view normalized by age

Maximum area of the atrium on the A4CC view

Ratio of the MR jet length to the atrium area on the A4CC view

Duration of ventricle contraction on the A4CC view

Area of the atrium on the PLAXC view*

Area of the MR jet on the PLAXC view

Ratio of the MR jet area to the atrium area on the PLAXC view

Area of the atrium on the PLAXC view normalized by age

Area of the MR jet on the PLAXC view normalized by age

MR jet duration on the PLAXC view

Average of the MR jet velocity on the PLAXC view

Maximum of the MR jet velocity on the PLAXC view

Minimum of the MR jet velocity on the PLAXC view

SD of the MR jet velocity on the PLAXC view

Skewness of the MR jet velocity on the PLAXC view*

Kurtosis of the MR jet velocity on the PLAXC view

Entropy of the MR jet velocity on the PLAXC view

Maximum MR jet length on the PLAXC view*

Maximum MR jet length on the PLAXC view normalized by age

Maximum area of the atrium on the PLAXC view

Ratio of the MR jet length to the atrium area on the PLAXC view*

Duration of ventricle contraction on the PLAXC view

Maximum MR jet length between 2 views*

Maximum area of the atrium between 2 views

Maximum ratio of the MR jet length to the atrium area between 2 views*

A4CC indicates apical 4 chamber with color Doppler; MR, mitral regurgitation; and PLAXC, parasternal long axis with color Doppler.

^*Indicates 9 features used for rheumatic heart disease prediction.

Deep Learning Approach

In the deep learning‐based approach, our aim was to diagnosis RHD based on holistic image interpretation from the 2 standard views described. Following image harmonization (Figure 1A), we combined 2 distinct deep learning models, namely multiview 3‐dimensional (3D) convolutional neural networks (CNNs) ¹⁹ and multiview Transformer ²⁰ (Figure 1B).

The RHD detection model combined spatiotemporal information from 2 views using 3D CNNs (Data S1). Input data, regions of the left atrium during systole, were resampled to 64×64 pixels with 3 color channels and 16 frames. Each data underwent 2 sets of 3×3×3 convolutional kernels with ReLu activation, followed by batch normalization and 2×2×2 max‐pooling (strides of 2). The features obtained from both views were combined and then fed through 2 fully connected layers: the first layer had 256 units with ReLU activation, and the second layer had 2 units with a Softmax activation function to calculate probabilities. Training included a batch size of 64, Adam optimization (learning rate of 0.0001), and 350 epochs to minimize binary cross‐entropy loss. The RHD detection model also used a Transformer to capture frame dependencies during ventricular contraction. It employed pretrained DenseNet121 CNNs ²¹ to extract low‐level features from 2 views of systole frames (resampled to 16 frames). The Transformer learned frame relationships through self‐attention and a feed‐forward neural network, with shortcuts and normalization for efficiency. The encoded features underwent downsizing via global max‐pooling before being fed into a fully connected layer with Softmax activation for RHD prediction. A dropout (keep rate of 0.5) was implemented before the last fully connected layer. Training parameters included batch size of 10, Adam optimization with a learning rate of 0.0001, 350 epochs for binary cross‐entropy loss minimization. Finally, the ensemble model integrated the predictive scores from the 3D CNNs and Transformer using the maximum voting approach.

Both 3D CNNs and Transformers excel in processing sequential data such as videos. 3D CNNs leverage the entire volume data of all frames during ventricular systole to assess RHD, capturing both spatial features and temporal dependencies. On the other hand, Transformers analyze the data frame by frame, allowing them to effectively model intricate temporal relationships between frames. Due to the complementary strengths of these 2 deep learning models, we integrated their predictive scores using an ensemble model and used the maximum voting strategy to improve the overall performance. ¹²

Statistical Analysis

All echocardiograms included in this work contained annotated manual MR jet length measurements as well as a binary diagnosis of RHD positive or negative determined by a panel of expert cardiologists. ² These results were then compared with our automated machine learning and deep learning results. Harmonization results were analyzed using accuracy for view identification and frame selection and a Dice coefficient to determine correct localization of the left atrium. For our machine learning method, we compared manual MR jet length measurements to the automated MR jet length measurements results using means with SDs with MR jet >2 cm considered RHD positive. ⁹ Statistical significance was determined using the Wilcoxon signed‐rank test at a significance level of 0.05 to assess statistical significance within our paired data, ensuring a robust evaluation of MR jet length measurements. To evaluate diagnostic performance, we used DeLong's method ²² and a significance level of 0.05 to compare area under the receiver operating characteristic curves (AUCs). For the compared methods, AUC as well as precision, recall, and F1 score were used to evaluate the performance of each model. Additionally, we reported CIs for each criterion to offer a deeper understanding of our results and provide insights into our model's diagnostic confidence.

Results

Model Training Environment

In our study, the SVM‐based method extracted features using MATLAB, and the SVM classifier was implemented in Python using the Scikit‐learn library. For the AI method, we used Keras (version 2.6.0) and TensorFlow (version 2.6.2) frameworks, and the model training was performed on a desktop computer equipped with a GeForce GTX TITAN X GPU from NVIDIA (Santa Clara, CA), which has 12 GB of memory.

Image Harmonization

We conducted training/validation of image harmonization on a subset of 95 out of 511 subjects, which were annotated by experts, and subsequently tested the remaining 416 out of 511 subjects. During each fold of the cross‐validation, the AI model was trained/validated on an average of 1578/390 frames for view detection, 9806/2450 frames for frame selection, and 2512/628 frames for atrium localization RHD detection was trained on a total of 5108 videos and validated on 1277 videos generated from portable echocardiograms from the GOAL trial, which included different combinations of clips acquired during systole from AP4 and PLAX views. Importantly, it should be highlighted that the number of frames in each video exhibited variation, reflecting the typical diversity encountered in clinical practice. Additionally, the number of frames varied across different tasks within the model. For instance, the first frame was used for view detection, all frames were employed for frame selection, and specifically, frames captured during ventricular systole were employed for atrium localization and RHD detection. Furthermore, subjects often possess multiple videos for each AP4/PLAX view, and some of these videos may contain multiple cardiac cycles. To generate the final predictive score for RHD in our analysis, we integrated all clips obtained during systole from the available AP4 and PLAX views.

The algorithm used for view identification demonstrated an accuracy of 0.99 in both AP4 and PLAX views (Figure 2). The algorithm selected the correct systolic frame with an average accuracy of 0.94 (sensitivity of 0.96/specificity of 0.93) and 0.93 (sensitivity of 0.94/specificity of 0.93%) for the AP4 and PLAX view, respectively. It localized the atrium with an average Dice coefficient of 0.88 and 0.9 for the AP4 and PLAX view, respectively.

Figure 2. Results for the accuracy of harmonization of echocardiogram images.

A4CC indicates apical 4 chamber with color Doppler; and PLAXC, parasternal long axis with color Doppler.

Machine Learning Approach

When examining the results of our machine learning method for RHD detection based on MR jet analysis, we obtained an average length estimation of the MR jet at 2.06±0.93 cm, which closely aligns with manual measurements (2.06±0.77 cm) with a P value of 0.83. The P value of 0.83 suggests that there is no statistically significant difference between our automated measurements and manual measurements. This result indicates robust agreement and supports the validity and reliability of our automated approach for MR jet length estimation, with no statistically significant difference observed. Figure 3 presents the Bland–Altman plot depicting the bias ±1.96 SD of the agreement between the automatic and manual measurements for the test data set, which was determined to be 0.005±1.1 cm. The SVM model's output is a binary classification that precisely distinguishes between RHD and non‐RHD cases. This classification is accomplished using 9 components of MR data, as indicated in Table 2, and employing cross‐validation.

Figure 3. Bland–Altman plot demonstrating the difference between the automatic and manual measurements for the test data set, which was determined to be 0.005±1.1 cm.

This model demonstrated an AUC of 0.93 with a 95% CI of ±0.05 (0.88–0.98). Precision was 0.83 with a 95% CI of ±0.07 (0.75–0.9). Recall was 0.92 with a 95% CI of ±0.05 (0.87–0.97) and a F1 score of 0.87 with a 95% CI of ±0.06 (0.81–0.94). (Table 3).

Table 3.

Quantitative Rheumatic Heart Disease Detection Results Along With 95% CIs Using Different Methods

Method	AUC	Precision	Recall	F1 score
9‐Feature support vector machine model	0.93 (0.88–0.98)	0.83 (0.75–0.9)	0.92 (0.87–0.97)	0.87 (0.81–0.94)
3D‐CNN*	0.76 (0.68–0.84)	0.77 (0.69–0.85)	0.77 (0.69–0.85)	0.77 (0.69–0.85)
Transformer*	0.75 (0.67–0.84)	0.71 (0.62–0.79)	0.92 (0.87–0.97)	0.8 (0.72–0.88)
3D‐CNN and transformer* , †	0.84 (0.77–0.92)	0.78 (0.7–0.86)	0.98 (0.95–1)	0.87 (0.81–0.94)

AUC indicates area under the receiver operating characteristic curve; and CNN, convolutional neural network.

^* P=0.04.

^† P=0.01.

Deep Learning Approach

For our deep learning models, the multiview 3D convolutional neural networks model demonstrated an AUC of 0.76 with a 95% CI of ±0.08 (95% CI, 0.68– 0.84). Precision was 0.77 with a 95% CI of ±0.08 (95% CI, 0.69–0.85). Recall was 0.77 with a 95% CI of ±0.08 (95% CI, 0.69–0.85) and a F1 score of 0.77 with a 95% CI of ±0.08 (95% CI, 0.69–0.85). The multiview Transformer model demonstrated an AUC of 0.75 with a 95% CI of ±0.08 (95% CI, 0.67–0.84). Precision was 0.71 with a 95% CI of ±0.08 (95% CI, 0.62–0.79). Recall was 0.92 with a 95% CI of ±0.05 (95% CI, 0.87–0.97) and a F1 score of 0.8 with a 95% CI of ±0.08 (95% CI, 0.72–0.88). When combined, the ensembled model of multiview 3D convolutional neural networks and multiview Transformer significantly improved the performance compared with each individual application (P values of 0.04 and 0.01, respectively, significance level: 0.05). (Table 3). The ensemble model achieved an AUC of 0.84 (95% CI, 0.77–0.92), precision of 0.78 (95% CI, 0.70–0.86), recall of 0.98 (95% CI, 0.95–1), and F1 score of 0.87 (95% CI, 0.81–0.84).

Discussion

Early detection of RHD has the potential to prevent hundreds of thousands of unnecessary deaths; however, scalability is a significant challenge, in large part due to limited expertise and the lack of a specialized workforce in RHD‐endemic regions. AI holds great promise for widespread echocardiography diagnosis of RHD by nonphysician health care workers but there has been limited progress in this field to date. Our results show that machine learning algorithms for MR jet length and analysis and deep learning algorithms for RHD detection have a high degree of accuracy.

As RHD is predominately endemic in regions where there is less specialized care and limited economic resources, task shifting and the use of handheld echocardiogram devices for diagnosis of RHD are particularly important to increase RHD detection globally. Prior innovative work in resource‐limited settings has demonstrated that task‐shifting of echocardiogram screening by nonphysician health care workers can be both feasible and highly diagnostic in identifying RHD. ³ , ²³ Engelman et al ²⁴ demonstrated that after a 2‐month training program, community nurses in Fiji achieved a diagnostic accuracy of 89% for RHD detection when performing a focused echocardiogram using a portable ultrasound machine. This work has been expanded further in its application to handheld echocardiogram devices to broaden diagnostic potential with limited resources in a study that found that a simplified echocardiogram protocol on handheld devices had a sensitivity of 74% and a specificity of 79% for any RHD, which improved to 90.9% for definite RHD. ³

AI has been successful in its application as a novel and innovative strategy in the realm of cardiac imaging. This work has been primarily focused on adult patients, specifically evaluating left ventricular volumes ²⁵ , ²⁶ and the automated estimation of left ventricular ejection fraction. ²⁷ Narang et al ²⁵ demonstrated that automated machine learning algorithms can quickly measure left‐sided volumes to accurately determine ejection fraction. Further work focusing on the use of automated quantification of 3D transthoracic left‐sided volumes and left ventricular ejection fraction measurements demonstrated that automated measurements were comparable to expert manual measurements as well as cardiac magnetic resonance imaging measurements. ²⁸ This work was then revisited and expanded to evaluate reproducibility in a multicenter setting. This work demonstrated that this automated method was both accurate and reproducible across different echocardiogram laboratories. ²⁶

There has been further work in the development of AI for its specific use in RHD. ⁶ , ⁷ , ²⁹ In conjunction with the implementation of task shifting to nonphysician health care workers, Nascimento et al ²⁹ developed a machine learning algorithm using convolutional neural networks for automatic detection of standard echocardiographic views. This is instrumental in the acquisition of high‐quality images that can then be fed into diagnostic automated algorithms for increased RHD detection.

Our machine learning algorithm is the only clinical work to date that looks at automated measurement of MR length and novel MR jet analysis, combining machine learning and deep learning methods, to diagnose RHD. ¹⁷ Our machine learning approach demonstrated a high degree of accuracy (0.86) that is similar to an expert's manual measurement accuracy (0.92) and is similar to previously published methods for categorization of RHD positive versus negative. ⁶ , ⁷ Our analysis did not take into account WHF spectral Doppler criteria (MR peak velocity >3 and pansystolic jet) for pathologic MR. ⁹ This explains why the expert manual measurement accuracy was less than 1.0 as some patients with MR jet length >2 cm would not meet criteria for RHD if they did not also meet the WHF spectral Doppler criteria. Therefore, the highest accuracy that could be achieved was 0.92.

Although MR jet length is currently used in the 2012 WHF guidelines, ⁹ our analysis highlights that many other features of the MR color Doppler jet may be as or more informative. It is important to note that although MR jet length is the easiest to reproduce manually by experts, the use of AI provides an opportunity to expand our analysis of the many features of MR available within the color Doppler data set. Features such as MR jet duration, skewness, velocity based on color Doppler signal, and jet length/left atrium area may be more specific for RHD than MR jet length (Table 2). This is likely to be of particular importance in younger children who have smaller cardiac dimensions. Diagnosis of RHD in younger children might be better served by using one of the basic principles of pediatric echocardiography, indexing measurements across the spectrum of pediatric size and age (eg, MR jet/left atrium area). ³⁰ This requires further study and could inform future guidelines and practice.

Martins et al ⁷ demonstrated that the use of deep learning methods for RHD detection was feasible in low resource settings with an accuracy of 0.73. Edwards et al ⁶ demonstrated that a machine learning model was capable of identifying mitral regurgitation based on 1 standard echocardiogram view (PLAX) with color Doppler with an accuracy of 0.86 for diagnosis of RHD. This work evaluated MR presence or absence alone and did not evaluate its clinical application. Our work evaluates for pathologic MR to ultimately diagnose RHD based on the WHF criteria ⁹ ; transferring these automated methods from the technical realm to the clinical world. Our deep learning algorithm is also unique in that it goes beyond machine learning approaches that are published using complex convolutional neural network approaches to distinguish RHD positive cases with a high level of sensitivity and specificity. ⁷ Regarding the comparison with a 9‐feature SVM model, although both approaches have similar performance, it is important to recognize that our 2‐arm framework provides additional stability in complex tasks and in anticipation of data from diverse sources and machines.

Our next steps in this work include increasing the number of cases inputted into the current algorithm using the extensive database of RHD positive echocardiograms obtained in Uganda and worldwide over the past 20 years through research collaborations. This will serve to further improve the accuracy of these algorithms. We expect that this will lead to our deep learning method outperforming our machine learning method with more training data. We also plan to adapt our current algorithm to be integrated into handheld echocardiogram devices. This will make field deployment of this technology achievable in limited resource settings. This will, in turn, increase the ability for health care workers in low‐and‐middle income countries to apply this novel technology in RHD endemic regions. As we continue training and testing our current models on handheld echocardiogram images, we will consider incorporating other deep learning approaches, such as signal‐based deep learning models, which could improve accuracy on potentially lower quality images by focusing on color Doppler signal morphology. Our plan would incorporate different levels of use making this technology adaptable to the resources available in each region. This will be achieved by translating our algorithms to a second tablet (separate from the handheld echocardiogram tablet) with and without internet capabilities. Our goal will be to develop software that is versatile and independent of a specific vendor. We believe that the application of these innovative AI technologies with handheld echocardiogram devices are critical for expanding the reach of echocardiogram screening for RHD worldwide. This technology has the potential to drastically reduce the overall global burden of RHD.

This work contains important limitations. As our aim was to focus on MR by color Doppler, which encompasses most patients with RHD, evaluation of other criteria was not included in current machine or deep learning approaches. This highlights that one aspect that will need to be further developed within the AI algorithms is evaluation of mitral valve and aortic valve morphology. In addition to this, we will need to evaluate aortic insufficiency as a small number of patients with RHD have these echocardiogram findings alone without the presence of MR. ⁹ We plan to expand our algorithm to include these additional features of RHD in the future. Also, we did not include spectral Doppler, which could have some impact on accuracy, sensitivity, and specificity. However, field applications are likely to exclude spectral Doppler.

Another potential limitation to our work is that the echocardiograms used so far have been obtained on fully functional portable echocardiography machines. Field implementation will require the use of handheld devices, which have the potential for less diagnostic image quality. This limitation will be addressed with the finetuning of our algorithms on echocardiograms performed on handheld devices from prior RHD screening implementation projects in endemic regions. ³⁰ To ensure that there is true clinical application of AI‐enabled RHD detection, field implementation studies in RHD endemic regions are also needed to evaluate the feasibility of this approach. This will include future prospective validation studies to determine if the results described in this paper can be applied to a broader population of patients with early RHD.

Conclusions

AI, in conjunction with task shifting of echocardiogram screening and use of portable handheld echocardiogram devices, has the potential to aid in significant scalability of RHD detection in low‐ and middle‐income countries where resources and expertise are limited. Our machine learning algorithms for MR jet length and analysis and deep learning algorithms can detect RHD with a similar degree of accuracy as expert cardiologists. Future work needs to be pursued incorporating other features of RHD into our current AI algorithms as well as feasibility testing for field implementation in RHD endemic regions.

Sources of Funding

The original GOAL trial was supported by the Thrasher Research Fund, Gift of Life International, Children's National Hospital Foundation (Zachary Blumenfeld Fund and Race for Every Child [Team Jocelyn]), the Elias–Ginsburg Family, Wiley Rein, Philips Foundation, AT&T Foundation, Heart Healers International, the Karp Family Foundation, Huron Philanthropies, and the Cincinnati Children's Hospital Heart Institute Research Core.

Disclosures

None.

Supporting information

Data S1