Artificial intelligence in fetal echocardiography: Recent advances and future prospects

Abstract

In the past few decades, great progress has been made in prenatal diagnosis of congenital heart disease (CHD). Fetal echocardiography is recognized as the main prenatal screening and diagnostic tool that can accurately detect approximately 85 % of fetal cardiac abnormalities. Evaluation of the fetal heart remains a major challenge in prenatal ultrasound screening and diagnosis due to fetal position, involuntary movement, small and complex fetal cardiac anatomy, maternal abdominal wall conditions, and lack of expertise in fetal echocardiography by some physicians engaged in obstetric ultrasound. Artificial intelligence (AI) can automate and standardize the display of each diagnostic section of the fetal heart and thus contribute to accurate diagnosis, which significantly optimizes the clinical application of fetal echocardiography. In this review, we not only clarify the role of AI but also highlight its significance and future solutions in the field of fetal echocardiography.

1. Introduction

Congenital heart disease (CHD), which accounts for 1 % of all neonates, is the most common birth defect and the main cause of death in neonates with congenital birth defect [1]. Accurate prenatal diagnosis can significantly improve the perioperative treatment effect and surgical success rate of CHD as well as reduce neonatal mortality. Fetal echocardiography has become a major screening and diagnostic tool for fetal CHD because of its noninvasive, nonradiative, real-time, and dynamic advantages [2]. In the past two decades, important advances have been made in the accuracy of prenatal CHD diagnosis. Emerging studies have reported that fetal echocardiography can accurately detect CHD with an accuracy rate of up to 85 % [3], [4].

However, owing to the lack of professional fetal heart screening personnel and imperfect prenatal screening conditions in some areas, the prenatal CHD detection rate is significantly different [5]. Quartermain et al. conducted a large study that showed that only 34 % of CHD were detected before delivery in some communities [6]. In some countries, the prenatal detection rate of CHD is as low as 14 % [6], [7], [8]. In recent years, the rapid development of artificial intelligence (AI) in the field of fetal cardiac ultrasound has enabled automated and standardized display of each diagnostic section of the fetal heart and accurate diagnosis, which is expected to reduce the dependence on operator experience [9]. Therefore, this would improve the difference in CHD detection rates between different regions. In this review, the concept of AI and discussion of the application of AI in fetal echocardiography will also be highlighted.

2. Characteristics of AI

AI is a field of computer science that focuses on the development of algorithms that learn, reason, and self-correct in a human-like manner, which can simulate, extend human intelligence, and perform related tasks [10]. Machine learning (ML) is an important subset of AI that can learn from data, identify images, and make decisions [10]. In the field of machine learning, deep learning (DL), which is the most widely used strategy, is a powerful evolution of ML. In the DL structure, convolutional neural networks (CNN) are commonly used in medical image processing tasks and have great potential for the automatic analysis of ultrasound images [11], [12], [13].

Current DL algorithms of CNN include ResNet, U-Net, DeepLab, and generative adversarial networks. The ResNet network architecture is used in the field of image recognition. U-Net and DeepLab are primarily used in image segmentation, whereas generative adversarial networks are primarily used to generate fake images to overcome the problem of insufficient data.

3. Application of AI in fetal echocardiography

At present, research on AI in fetal echocardiography is more extensive, mainly focusing on image acquisition, image optimization, automatic measurement, recognition of outliers, disease diagnosis, and classification.

3.1. Fetal heart ultrasound image acquisition

The first step in any diagnostic ultrasound process is obtaining an appropriate image. Compared to adult echocardiograms, obtaining images of fetal echocardiograms depends heavily on a number of factors, including fetal position and the operator's experience in acquiring and recognizing the images. The subsequent steps in the diagnostic process are also affected by the quality and ability to obtain images. Automating the process of identifying and storing the optimal fetal heart plane can improve sonographer efficiency, reduce image variability, produce high-quality images, and improve prenatal CHD detection. Fetal intelligent navigation echocardiography (FINE) is a new technology that has developed in recent years. It uses a four-cavity incisal plane as the basic section to collect fetal heart volume data and mark the intracardiac structural targets. Intelligent and standardized fast and automatic generation of nine standard sections is required for CHD screening [14].

In recent years, studies on FINE have mainly focused on the display of standard fetal echocardiography sections and the diagnostic efficacy of fetal CHD. Garcia et al. [15] conducted a prospective clinical study that included 207 normal fetuses, each of whom underwent routine fetal echocardiography and FINE. The results showed that the section rate of standard fetal echocardiography with diagnostic value successfully obtained by FINE was 98–100 %. Our team [16], [17], [18] conducted a series of studies on fetuses with scleroconus arteriae (right ventricular double outlet, complete transposition of the great arteries, and tetralogy of Fallot) diagnosed by echocardiography during middle and late pregnancy using FINE and VIS-Assistance® to obtain a standard diagnostic plane. The display success rates of key diagnostic sections and diagnostic elements of trunk conus artery malformation were obtained by observing the FINE. The results showed that the FINE had a high display rate of diagnostic sections and diagnostic elements for common fetal CHD and had good repeatability and consistency. The results showed that the FINE technique was helpful in the screening and diagnosis of fetal trunk conical artery malformations, which has potential clinical value in remote consultation and teaching. Yeo et al. [19] showed that FINE had a sensitivity of 98 % and specificity of 93 % in detecting CHD; therefore, it was recommended to implement FINE in routine screening to improve the detection rate of prenatal CHD.

Another potential use of AI is to automatically acquire nine standard diagnostic sections from a fetal heart echocardiogram imaging database. Baumgartner et al. [20] first introduced the SonoNet model and used a CNN to detect standard diagnostic sections from a set of ultrasound videos, eliminating the need for sonographers to stop and obtain the necessary images. The SonoNet model automatically detected the four-chamber heart, left ventricular outflow tract, right ventricular outflow tract, and three-vessel section using ultrasound images of approximately 1000 healthy fetuses, with detection rates of 95.00 %, 78.50 %, 73.08 %, and 81.90 %, respectively. Recently, Arnaout et al. [21] developed a CNN model that can automatically identify fetal heart sections. Based on 1326 two-dimensional ultrasound grayscale images, the model distinguished five standard sections of normal and abnormal fetal hearts (abdominal plane, four-cavity heart, left ventricular outflow tract, three-vessel section, and three-vessel trachea section). With an area under curve of 0.99, 95 % sensitivity, 96 % specificity, and 100 % negative predictive value were achieved, while the model sensitivity was comparable to that of clinicians and could significantly improve the detection rate of fetal CHD.

3.2. Fetal heart ultrasound image optimization and quality control

In addition to image acquisition, image quality is critical for accurate diagnosis of fetal CHD. However, owing to the influence of involuntary fetal movement, small and complex heart size, ultrasonic speckle noise and artifacts, and the lack of professional knowledge of fetal echocardiography by obstetrical ultrasound doctors, the image quality of echocardiography is uneven. In recent years, there have been few reports on the automatic quality control of fetal heart ultrasound images. Abdi et al. [22] used the CNN regression model to compare 6919 ultrasound images of the four-chamber incisal surface of the fetal heart with the corresponding quality scores determined by cardiologists, and the results showed that the average absolute error between the training model and the experts was 0.71 ± 0.58, thus the automatic quality assessment of the four-chamber incisal surface of the heart apex was realized.

Abdi et al. [23] proposed a DL model based on recurrent neural networks (RNNs), that could automatically distinguish the five standard sections of fetal echocardiography (four-chamber heart, three-chamber heart, two-chamber heart, major artery brachyaxis, and left ventricular brachyaxis papillary muscle level) and correlated them with the corresponding quality scores determined by cardiologists. The study included 509 cases, including 2450 fetal echocardiograms tested on five standard sectional datasets, and the model achieved an average quality score accuracy of 85 % compared to expert-determined quality scores. Dong et al. [24] proposed a general DL framework for quality control of four-chamber incisions of the fetal heart The framework is composed of three CNN-based networks (B-CNN, D-CNN, and ARVBNet), which are used for rough classification, classification refinement, and anatomical structure detection. Experiments on fetal echocardiography datasets demonstrate the effectiveness of the quality control framework and demonstrate the generalization and adaptability of ARVBNet, which type of algorithm can provide a quantitative assessment of image quality in clinical practice and provide real-time image optimization information for novices.

3.3. Intelligent automatic measurement of fetal echocardiography

The measurement of fetal heart structure, function, and Doppler spectrum is an important part of fetal echocardiography workflow. The advantages of applying AI to workflows include reduced acquisition time, energy savings for standard measurements, and improved repeatability. Automation of real-time measurements also prevents measurement anomalies due to time delays. Sulas et al. [25] developed an algorithm that uses a fully connected neural network to automatically recognize Doppler patterns in fetal echocardiography with an average accuracy of 88 %, which has good practical value for patients with abnormal rhythms or subtle physiological changes related to CHD. In order to predict fetal left ventricular volume more accurately and effectively, Yu et al. [26] proposed a two-dimensional ultrasound single-plane low-pressure volume measurement method based on reverse neural network to calculate fetal left ventricular volume. It was found that the reverse neural network method had the highest consistent correlation coefficient [ICC = 0.9691 (95 % CI: 0.9663–0.9717)] and intra-group correlation coefficient [CCC = 0.9401(95 %CI: 0.9348–0.9449)], and the left ventricular function parameters obtained by this model also had the best consistency with the data obtained by 4D ultrasound. The average deviation of this model in the assessment of fetal stroke volume was 0.03 mL, while the 95 % confidence interval was −0.15 ∼ 0.20 mL.

Bridge et al. [27] used the end-to-end neural network framework for the first time to directly predict the heart cycle from the normal fetal echocardiography video and used the DL model to automatically identify, annotate, and extract features from the fetal echocardiography of four cavities, three vessels, and left ventricular outflow tract. This lays a foundation for the automatic measurement and diagnosis of abnormal fetal heart sections in the future. Cai et al. [28] proposed a fetal heart ultrasound image segmentation network model based on knowledge distillation technology to finely segment the three-vessel sections of 1300 fetal heart ultrasound images, and the results showed that this model obtained more accurate segmentation boundaries than the most commonly used existing segmentation models (U-Net model and DeepLabv3+ model). The average IoU (%), PA (%) and Dice (%) were 68.6 %, 81.4 %, and 81.3 %, respectively. Xu et al. [29] used the DW-Net model to automatically segment multiple anatomical structures of the four-cavity incised surfaces of 895 fetal heart apices. The results showed that DW-NET had a better segmentation effect than other mainstream image segmentation methods. The Dice similarity coefficient was 0.827, the pixel accuracy was 0.933, and the area under the ROC curve was 0.990. The DW-Net model can accurately and automatically segment the four-chamber view of the fetal apex, which is conducive to further extracting useful clinical indicators of early fetal echocardiography and improving the accuracy and efficiency of the prenatal diagnosis of CHDs.

3.4. Identification of fetal CHD

Improving the ability of AI to detect prenatal CHDs is an ongoing research goal. Komatsu et al. [30] developed an SONO model using CNN to detect the anatomical structure of 363 fetal hearts and marked the structural abnormalities in the continuous cross-sectional videos around the four-chamber heart section and the three-vessel section. The areas under the ROC curves for the heart and blood vessels were 0.787 and 0.891, respectively. The automatic detection of each key heart structure in fetal echocardiography is realized, and is suitable for the detection of abnormal fetal heart structures. Zhou et al. [31] proposed a DL model-DGACNN based on a generative adversarial network. In this study, 3596 static pictures of the four-chamber heart section at the end of normal and abnormal systole and 100 dynamic video images were taken as training, test, and verification sets to train, test, and verify the DGACNN model. It was found that when the false-positive rate was in the range of 20 %, the accuracy of the model was 84 %, and the area under the ROC curve was 0.881, which has great potential for prenatal screening. Gong et al. [32] proposed a DGACNN model that used 110,000 video images to enhance the fetal video dataset and used video transfer learning to classify whether the images were likely to have CHD, and the recognition rate of CHD from fetal ultrasound video clips reached 85 %.

Nurmaini et al. [33] used a Mask-RCNN (MRCNN) to process 764 fetal echocardiographic images for the detection and segmentation of fetal septal defects. The model automatically divided the fetal heart into atria, ventricles, and aorta, achieving good detection performance of the heart cavity, with a detection rate of 97.59 % for the right atrium. The detection rates for the left atrium, left ventricle was 86.17 %, right ventricle, and aorta were 99.67 %, 86.17 %, 98.83 %, and 99.97 %, respectively. The mAP of the MRCNN was 99.48 % to identify the location of the atrial and ventricular septal defects. Anda et al. [34] proposed an intelligent decision support system (ISs) based on DL, which could collect and analyze data, communicate with other system frameworks, learn from experience, and adapt accordingly to new cases, and was the first AI for CHD screening in early pregnancy. Arnaout et al. [21] used neural networks on a large fetal ultrasound dataset, which included 1326 fetal echocardiograms of approximately 400 patients with severe CHD, to first classify various cardiac sections using a supervised model and then classify normal or abnormal anatomical structures to identify complex CHDS. In addition, 4, 180 fetuses with 0.9 % CHD were used as a test set, and standard fetal cardiothoracic measurements were calculated using a segmentation model, showing >95 % sensitivity and specificity. These results are very promising, as they bring the AI algorithm closer to expert accuracy.

4. Challenges and future directions

AI research on fetal CHD is still in the preliminary stage, and the generality of AI is limited because of the small datasets used in many studies. To address this issue, it is important to adopt broader and larger datasets; however, the infrastructure to support these studies in the fetal field is relatively scarce. Diller et al. [35] proposed an innovative solution to this problem, using ML to expand the training image pool by generating synthetic frames from existing images. Training the segmentation network on these data produced results comparable to the original database. A similar approach can be applied to AI in fetal echocardiography. Vendor-specific algorithms limit the generalization of models to a wider population, and most models use supervised learning algorithms that require quantitative manual data labeling. In addition, as new technologies emerge, resource-limited regions may use them less, potentially affecting the growing gap between communities, building robust AI-assisted ultrasound models, and creating more robust and effective ultrasound models. Further multicenter, diverse data should be included in future studies, and data quality control standards must be established to ensure the quality of datasets.

AI has significant potential to improve the detection of fetal CHD. This review demonstrates that AI has made important advances in image acquisition, automated measurement, and identification of disease states in fetal echocardiography(Fig. 1). However, the growth of AI in the field has been limited by issues such as lack of data and limited resources to implement new technologies. Nevertheless, AI technology still has great potential and room for improvement in the field of medical imaging,and is expected to become a routine diagnostic tool for fetal heart echocardiography.

Fig. 1.

A schematic diagram of this review AI, artificial intelligence; ML, machine learning; DL, deep learning. Current applications of AI in fetal echocardiography commonly focus on image acquisition (Fig2) [18], image optimization (Fig3) [24], automatic measurement (Fig4) [26] and identification of CHD (Fig5) [21].

5. Conclusion

AI can automate and standardize fetal heart diagnosis, which is beneficial for optimizing the clinical application of fetal echocardiography. Future studies should focus on real-world application and continuous improvement of AI in fetal echocardiography.

CRediT authorship contribution statement

Mingming Ma: Formal analysis, Writing – original draft. Li-Hua Sun: Writing – review & editing. Ran Chen: Writing – review & editing, Funding acquisition. Jiang Zhu: Conceptualization, Funding acquisition. Bowen Zhao: Conceptualization.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.