Multimodal Nomogram for the Prenatal Risk Assessment of Hypoplastic Left Heart Syndrome Using Self‐Supervised Learning

Abstract

Objectives

This retrospective study aims to develop and validate a multimodal nomogram for the prenatal risk assessment of hypoplastic left heart syndrome (HLHS) and to explore significant risk factors for HLHS.

Methods

This retrospective study enrolled 161 normal pregnancies and 52 pregnancies diagnosed with HLHS at the Ultrasound Diagnosis Department of Maternal and Child Health Hospital of Hubei Province in China from September 2019 to September 2023. Experienced sonographers selected standard 4‐chamber cardiac views and delineated the boundaries of the left and right atria and ventricles. A ResNet‐like variational autoencoder (RVAE) was used to extract features from 4‐chamber views in a self‐supervised learning strategy. These features were then converted into an image score. The proposed multimodal nomogram was developed using univariate and multivariate logistic regression analysis, incorporating the image score, demographics, and morphological characteristics of the fetus' heart. To evaluate the nomogram's performance, we constructed a clinical regression model, 4 machine learning (ML) models, and an expert model comprising 3 sonographers with varying years of clinical experience.

Results

The study identified several significant risk factors for assessing the risk of HLHS, including the diameter of the left ventricle, the area ratio between the left and right atrium, the area ratio between the left and right ventricle, and the image score calculated from the 4‐chamber view. A multimodal nomogram was constructed based on these factors, achieving an accuracy of 0.935 and an AUC of 0.991. The performance of the nomogram was better than that of the traditional logistic regression and ML models, and comparable to that of the expert model. Additionally, the multimodal nomogram outperforms sonographers with 3‐ and 6‐year experience, and performs only slightly worse than the sonographer with 10‐year experience. When combined with the heat‐map generated from RVAE, the nomogram can serve as an easy‐to‐use tool to help clinicians better understand the process of computer‐aided diagnosis.

Conclusion

The proposed multimodal nomogram demonstrates its superiority, effectiveness, and interpretability in prenatal risk assessment of HLHS. Additionally, 2 risk factors, ie, the area ratio between the left and right atrium and the area ratio between the ventricle, deserve further investigation during clinical diagnosis of HLHS.

Abbreviations

ACC

accuracy

AI

artificial intelligence

AUC

area under the receiver operating characteristic curve (area under the curve)

CFs

clinical features

CHDs

congenital heart diseases

CI/CIs

confidence interval(s)

CL

confidence limits (eg, 95% CL)

DCA

decision curve analysis

DT

decision tree

DW‐Net

a cascaded convolutional neural network for apical 4‐chamber view segmentation (model name)

EM

expert model

GE

GE (general electric) ultrasound devices

GPU

graphics processing unit

GW

gestational week

HLHS

hypoplastic left heart syndrome

IRB

institutional review board

IS

image score

ISUOG

International Society of Ultrasound in Obstetrics & Gynecology

KL

Kullback–Leibler (divergence)

LA

left atrium

LV

left ventricle

ML

machine learning

MLR

multivariate logistic regression

MRI

magnetic resonance imaging

MSE

mean squared error

OR

odds ratio

RA

right atrium

RF

random forest

RM

(logistic) regression model

ROC

receiver operating characteristic

ROI

region of interest

RV

right ventricle

RVAE

ResNet‐like variational autoencoder

SD

standard deviation

SSL

self‐supervised learning

SVM

support vector machine

ULR

univariate logistic regression

US

ultrasound

VAE

variational autoencoder

VIF/VIFs

variance inflation factor(s)

XGBoost

eXtreme gradient boosting

Hypoplastic left heart syndrome (HLHS) is a serious congenital heart disease characterized by hypoplasia of the left side of the heart, including hypoplasia of the left ventricle, aorta, and aortic arch, aortic and mitral valve atresia or stenosis. ¹ The mortality rate for untreated HLHS fetuses is 100%, and approximately 1 in 3841 newborns is affected by this condition. ² , ³ , ⁴ Fetal echocardiography can be used to diagnose HLHS during pregnancy. However, in China, most parents choose to terminate pregnancies when they receive a diagnosis of HLHS due to concerns about poor prognosis. Although research on the etiology of HLHS has made certain progress in recent years, the only effective treatment for prolonging the life of children with HLHS is still palliative surgery followed by waiting for heart transplantation. This approach is costly, risky, and often produces unsatisfactory results. ⁵ , ⁶ , ⁷ , ⁸ , ⁹ Early detection of HLHS is crucial for prenatal consultation and early clinical management.

Fetal echocardiography is the preferred method for diagnosing fetal heart malformations, such as HLHS. ¹⁰ However, obtaining standard views can be challenging due to factors like fetal heart structure, real‐time motion, fetal position, and maternal abdominal wall fat thickness. ¹¹ Therefore, sonographers must possess professional knowledge and experience. Junior sonographers may struggle with making accurate judgments, which can limit the definitive diagnosis of HLHS. During early pregnancy, detecting any asymmetry between the right and left heart with the naked eye can be difficult. As the fetus develops, the symptoms of HLHS gradually emerge, making timely diagnosis challenging. Furthermore, other conditions such as fetal coarctation of the aorta, complete ectopic pulmonary venous drainage, and restrictive foramen ovale can also result in a small left ventricle, which can be challenging to distinguish from HLHS in the 4‐chamber view in some regions and rural areas with inadequate diagnostic capabilities and unqualified consultation services. ¹² , ¹³ Sonographers must rely solely on their professional expertise to identify and differentiate between these conditions. It is imperative to note that this process requires objectivity and precision in order to circumvent misdiagnosis.

In recent years, there has been rapid development in computer‐aided diagnosis based on artificial intelligence (AI) technology. Machine learning (ML) algorithms show considerable potential in enhancing the effectiveness of cardiovascular and obstetric imaging, with the possibility of minimal additional costs to patients. ¹¹ However, the literature on the combined diagnosis of AI and fetal echocardiography for HLHS is limited, and few studies have utilized AI technology to assess fetal HLHS. In 2020, DW‐Net, a convolutional neural network capable of accurately and automatically segmenting the apical 4‐chamber view, was introduced. ¹⁴ This network could facilitate the extraction of valuable clinical indicators for early fetal echocardiography and improve the accuracy and efficiency of prenatal diagnostics for CHDs. In 2021, a deep learning‐based approach was proposed to explore the segmentation of tricuspid valve leaflets from 3D transthoracic echocardiograms in children with HLHS on a dataset of 133 image‐segmentation pairs with 28 validations. ¹⁵ Arnaout and colleagues discovered that clinical assessment has a low sensitivity of detecting HLHS, as low as 30%. However, they were able to achieve a sensitivity of 89% and a specificity of 92% for the identification of HLHS by using an ensemble of neural networks. ¹⁶ Nevertheless, accurately diagnosing HLHS in fetal ultrasound practice remains a highly challenging task.

To further enhance diagnostic accuracy and minimize the impact of personal biases, this study aims to develop and validate a multimodal nomogram for the prenatal risk assessment of HLHS using cardiac 4‐chamber views and exploring additional potential risk factors. During the construction of the nomogram, the diagnostic significance of demographics and morphological characteristics extracted from the standard 4‐chamber views was investigated, as well as the latent features automatically constructed from the views using a self‐supervised learning (SSL) strategy. ¹⁷ The proposed nomogram was shown to be interpretable and to have the potential to assist in clinical individual diagnosis. The main contributions of this study are as follows:

1. A multimodal nomogram with adequate clinical interpretability was proposed for the accurate diagnosis of HLHS, achieving the performance level of senior sonographers with 10 years of experience.

2. The present study represents a pioneering endeavor through its innovative utilization of a SSL strategy in the creation of the multimodal nomogram, which involves the extraction of latent features and subsequent conversion into a numeric image score (IS).

3. In addition to the diameter of the left ventricle, 2 further potential risk factors were identified for the diagnosis of HLHS, which could be integrated as one of the diagnostic criteria, thus assisting sonographers in making more reliable diagnoses.

Methods

This retrospective study received approval from the Research Ethics Commission of the Maternal and Child Health Hospital of Hubei Province (IRB ID: 2023ICE128), and the requirement for informed consent was waived.

Participants

The study enrolled 161 normal pregnancies and 52 pregnancies diagnosed with HLHS retrospectively from September 2019 to September 2023 at the Ultrasound Diagnosis Department of Maternal and Child Health Hospital of Hubei Province. A final diagnosis of HLHS was made by at least 2 senior physicians if fetal anomalies were suspected during the ultrasound examination. Figure 1 displayed the inclusion and exclusion criteria for the enrolled pregnancies in this study. Specifically, the inclusion criteria encompassed pregnancies who underwent fetal echocardiography examinations at the hospital and possessed 4‐chamber cardiac views. Demographic information, such as the mother's age and gestational week (GW), was obtained from medical records. The exclusion criteria consisted of inadequate ultrasound diagnosis, substandard 4‐chamber view of the heart, and incomplete examination records. After a thorough examination, this study included a total of 160 normal pregnancies and 46 HLHS pregnancies.

Figure 1.

Flow diagram of the study patients.

Acquisition of 4‐Chamber Cardiac View and Morphological Characteristics of Fetus Heart

The 4‐chamber cardiac view of the fetus was obtained in accordance with the practice guidelines of the International Society of Ultrasound in Obstetrics & Gynecology (ISUOG) ¹¹ , ¹⁸ during the diagnosis. Ultrasound devices from GE (29.13%), SAMSUNG (38.35%), Mindray (28.16%), SonoScape (2.91%), and Philips (1.46%) were used to acquire the views (Table S1). Experienced sonographers manually delineated the boundaries of the left and right atria and ventricles using ImageJ (version 1.53) from these views. Each measurement was performed at least 3 times, and the mean value was used. Morphological characteristics, such as diameters, areas, ellipticity, area ratios between the left and right atrium/ventricle, and interventricular septal thickness, ⁴ were estimated based on the delineated boundary of the atria and ventricles and the physical resolutions of the cardiac views extracted from the ultrasound machines (Table 1). For the purpose of model development and validation, pregnancies with standard 4‐chamber views and estimated morphological characteristics, as well as complete clinical records, were randomly divided into 2 datasets with a ratio of 4:1: a training set of 164 pregnancies and a test set of 42 pregnancies (Figure 1).

Table 1.

Summary of Demographics and Morphological Characteristics of Fetal Heart in US

Characteristics (Mean + SD)	Training Set, n = 164 (80%)	Test Set, n = 42 (20%)	p‐Value
Demographic characteristics
Age	30.49 ± 4.21	30.50 ± 4.01	.9933
Gestational week	25.96 ± 4.49	25.46 ± 5.67	.5471
Morphological characteristics—diameter
Left atrium (mm)	7.95 ± 2.95	7.91 ± 3.08	.9406
Right atrium (mm)	9.67 ± 2.83	9.73 ± 2.95	.8981
Left ventricle (mm)	7.42 ± 3.37	7.37 ± 3.65	.9369
Right ventricle (mm)	8.92 ± 2.59	8.41 ± 2.47	.2583
Morphological characteristics—area
Left atrium (mm2)	81.63 ± 56.67	78.57 ± 54.70	.7542
Right atrium (mm2)	95.3 ± 69.71	99.47 ± 79.81	.7385
Left ventricle (mm2)	101.55 ± 70.68	97.83 ± 62.23	.7571
Right ventricle (mm2)	97.91 ± 62.93	92.24 ± 68.94	.6120
Morphological characteristics—area ratio
Left atrium/right atrium	0.83 ± 0.39	0.78 ± 0.34	.4310
Left ventricle/right ventricle	0.99 ± 0.51	1.01 ± 0.48	.7928
Morphological characteristics—ellipticity
Ellipticity of left ventricle	2.14 ± 1.18	2.23 ± 1.35	.6537
Ellipticity of right ventricle	1.60 ± 0.44	1.71 ± 0.62	.1886
Morphological characteristics—others
Interventricular septal thickness (mm)	1.57 ± 0.87	1.65 ± 1.10	.6204
Image score	0.29 ± 0.25	0.30 ± 0.24	.7933

The p‐value represents the difference in each variable between the training and test sets.

Procedures

Figure 2 depicts the study design and workflow for developing and validating of the proposed multimodal nomogram. Following the practice on integrating multimodality data, ¹⁹ , ²⁰ the proposed nomogram uses 4‐chamber US views of the fetal heart and pregnancy demographics at the time of the US diagnosis. Specifically, the 4‐chamber US views were processed using a variational autoencoder ²¹ (VAE) with an SSL strategy ¹⁷ to obtain a vector of feature representation of the original US views. The study utilizes a multivariate logistic regression (MLR) analysis to determine the IS based on the vector of feature representation. Additionally, the morphological characteristics of the fetal heart were extracted from 4‐chamber US views. These characteristics were combined with demographic information, including age and GW, to create a vector of AUC using univariate logistic regression (ULR) analysis. A multimodal nomogram was constructed using the CFs and IS.

Figure 2.

Workflow for developing and validating the multimodal nomogram for predicting HLHS using demographic characteristics, 4‐chamber US views, and morphological characteristics extracted from US. Performance of the multimodal nomogram was compared with that of EM and RM, and its clinical implication was explored.

Concurrently, a logistic regression model (RM) and 4 ML models, including decision tree (DT), random forest (RF), support vector machine (SVM), and XGBoost, were built based on the CFs. Furthermore, an expert model (EM) was developed, which consisted of 3 sonographers with 3, 6, and 10 years of experience, respectively. These sonographers reviewed 4‐chamber views and assessed the risk of HLHS. The performance of these models was compared to that of the multimodal nomogram.

Self‐Supervised Learning and Image Score

SSL is capable of producing generalizable representations of medical images using unlabeled data, thereby greatly mitigating the challenges typically encountered when generating expert annotations in large‐scale datasets. ²² Consequently, SSL can create generalist models whose outputs can be fine‐tuned for a variety of downstream tasks, such as image classification ²³ and reconstructions. ²⁴ It is therefore feasible to integrate the representations learned by SSL into other modalities for the purpose of medical diagnosis. ²⁵ , ²⁶

A Resnet‐like VAE (RVAE) ²¹ was used in this study as a typical SSL approach to learn the latent distribution of the training data and reconstruct the original input US image (Figure 3). The RVAE structure was intuitively standard and simplistic, making the proposed SSL‐based feature representation of US images likely to generalize to more medical image analysis scenarios. The encoder and decoder blocks each consist of 4 stacks of a 3 × 3 convolution layer, a Residual Block, and a pooling layer. The Residual Block is an alternated sequence of Batch Normalization and convolutional layers with unit stride, intertwined with residual connections. All layers utilize a Rectified Linear Unit activation except the final output layer, which uses sigmoid activation. Fully connected layers were used between the encoder and decoder to calculate the mean (μ) and variance (σ) for latent variables, resulting in a 512‐dimensional feature representation vector.

Figure 3.

Network architecture of the RVAE. Light‐green and light‐yellow blocks correspond to the encoder and the decoder networks, respectively. Details on the Residual Block are depicted in the blue box. The feature representation learned by the RVAE is indicated by a purple vector with dimensions of 512.

The VAE loss used in this study was characterized by the reconstruction error and Kullback–Leibler (KL) divergence terms (Eq. 1). A standard mean‐squared error (MSE) loss was employed for the reconstruction error as a general practice.

$$L_{\mathrm{VAE}}=\mathrm{BCE}\left(x,p\left(z\right)\right)-\mathrm{KL}\left(q\left(z|x\right)\Vert p\left(z\right)\right)$$ (1)

where $q$ represents the encoder and $p$ represents the decoder ²⁷ with $x$ denoting the original input image and $z$ represents the latent space.

The RVAE was trained using a 5‐fold cross‐validation on the training set. Each fold consisted of 100 epochs, which were determined by identifying the loss function plateau in some preliminary studies. We used an Adam optimizer with a learning rate of 1.0⁻³ and a weight decay of 1.0⁻². To maximize the usage of the limited US image data, we applied data augmentation techniques on the fly, such as random rotation, flipping, and resizing. The model that achieved the lowest mean loss in cross‐validation was used to extract the feature representation vector from the latent space when given input images.

After training, the VAE's feature representation vector underwent Lasso regression analysis, which selected the most significant variables to calculate the IS using the regression results detailed in Equation ((1), (2)).

$$\text{IS}=b+\sum _{i=1}^n{\omega }_i{X}_i$$ (2)

where $X_i$, ${\omega }_i$, $b$, and n represent the Lasso‐selected feature, the weight coefficient, the offset, and the number of filtered features, respectively. In addition, the feature representation vector can be utilized to generate the grad‐CAM, ²⁸ , ²⁹ which enhances the interpretability of RVAE.

Clinical Features, Regression Model and Machine Learning Models

Thirteen morphological characteristics were extracted from 4‐chamber US views based on the delineated contours of the fetal heart. These characteristics include diameters, areas, ellipticity of both the atrium and ventricle, area ratios between the left and right atriums and between the left and right ventricle, as well as the interventricular septal thickness (Table 1). Concurrently, demographic characteristics, specifically the age and GW of the mothers, were recorded at the time of obtaining 4‐chamber views during US diagnosis. These demographic characteristics were combined with the morphological characteristics to create clinical records (Figure 2). Subsequently, the records underwent processing through a z‐score normalization scheme. A ULR analysis was then performed on the clinical records, yielding a vector of statistically significant CFs. These CFs were used to construct a logistic RM and 4 ML models (ie, DT, RF, SVM, and XGBoost), all of which were designed to predict the risk of HLHS. The aim was to assess the predictive performance of RM and ML models in comparison to that of the proposed multimodal nomogram.

Expert Model

A panel of 3 sonographers, with 3, 6, and 10 years of clinical experience, respectively, was independently asked to diagnose the risk probability of HLHS based on 4‐chamber US views and clinical information such as age and GW. A diagnosis was made only when all 3 sonographers agreed. In cases of discrepancy, the sonographers discussed the case until a final consensus was reached. The consensus of the panel was referred to as the EM.

Development and Validation of the Multimodal Nomogram

A MLR analysis was performed on the IS and CFs within the training set. This analysis aimed to identify variables that exhibited a significant association with HLHS, thereby facilitating the construction of the multimodal nomogram. The predictive efficacy of the nomogram was evaluated on the test set and compared to that of RM and EM through metrics such as accuracy, area under the receiver operating characteristic (ROC) curve (AUC), sensitivity, and specificity.

Statistical Analysis

Both demographic and morphological characteristics were presented using mean and standard deviation (SD) for continuous variables. T‐tests or Mann–Whitney U tests were employed for continuous variables that showed a normal distribution and homogeneity of variance between 2 groups. The Kruskal‐Wallis H‐test was utilized to test other continuous and categorical variables. Univariate and multivariate regression analyses were conducted to predict the risk assessment of HLHS with the significance of variables reported through p values. After multivariate regression, multicollinearity analysis was performed using variance inflation factors (VIFs). A 2‐sided p value of <.05 was considered statistically significant for all analyses.

The AUCs of different models were compared using the DeLong test. Furthermore, 95% confidence intervals (CIs) were calculated through 1000 bootstrapping iterations of ROC analysis. To demonstrate the overall net benefit of the multimodal nomogram compared to alternative methods, we utilized decision curve analysis (DCA). This provides a comprehensive view of the clinical utility and net benefit across a range of threshold probabilities.

The VAE model was implemented using PyTorch 1.10 and ran on an NVIDIA GeForce 3090 GPU with 24 GB of memory. The implementation of Lasso regression and ROC analysis employed the Scikit‐learn package in Python 3.6. All other statistical analyses were carried out using the R statistical software Version 3.4.1. The ULR and MLR analyses were conducted using the “rms” package, while DCA was performed using the “dca” package.

Results

Characteristics of Pregnancy

The mean and SD of the demographic and morphological characteristics of pregnancy were summarized for the training and test sets in Table 1. The IS for the 4‐chamber view of each fetus was calculated using Lasso regression analysis on the feature vector of RVAE after iterative training, as shown in Figure S1. There were no significant differences in the demographic and morphological characteristics of pregnancy between the training and test sets (Table 1). Based on the characteristics observed in the training set, it was found that GW had a strong positive correlation with most morphological characteristics, such as the diameter and area of left atrium (Figure 4). This is expected as these characteristics tend to increase as the fetus grows. Additionally, GW, diameter of left ventricle and atrium were found to have a negative correlation with characteristics such as ellipticity of the ventricles and the IS. It is important to note that the diameter of the left ventricle correlates more strongly with the diameter of the left atrium than with the diameter of the right ventricle, just as it does for the diameter of the right ventricle.

Figure 4.

Cross‐correlation between demographic and morphological characteristics and image scores. LA, left atrium; LV, left ventricle; RA, right atrium; RV, right ventricle.

Development and Validation of the Multimodal Nomogram

The ULR analysis showed that all characteristics, except for age, the diameters and areas of the right atrium and right ventricle, the ellipticity of the right ventricle, and the interventricular septal thickness, were statistically significant predictors (ie, CFs) of HLHS (Table 2). And then, the MLR analysis on CFs and the IS revealed that 4 characteristics are the significant predictors on HLHS: the diameter of the left ventricle, the area ratio between the left and right atrium, the area ratio between left and right ventricle, and IS. The VIFs of these 4 predictors were found to be within acceptable limits, as 2.460, 1.632, 1.981, and 2.802, respectively.

Table 2.

Logistic Regression for Predicting HLHS Based on Clinical and Morphological Characteristics

Predictors	Univariate Logistic Regression		Multivariate Logistic Regression
	OR (95% CL)	p‐Value	OR (95% CL)	p‐Value
Clinical characteristics
Age	0.987 (0.972–1.002)	.0833	N/A
Gestational week	0.955 (0.943–0.967)	<.0001*	1.005 (0.990–1.020)	.5058
Morphological characteristics—diameter
Left atrium	0.915 (0.899–0.930)	<.0001*	0.991 (0.964–1.019)	.5287
Right atrium	0.993 (0.970–1.015)	.5225	N/A
Left ventricle	0.912 (0.901–0.924)	<.0001*	0.967 (0.943–0.992)	.0095*
Right ventricle	0.991 (0.966–1.016)	.4607	N/A
Morphological characteristics—area
Left atrium	0.997 (0.996–0.998)	<.0001*	1.001 (1.000–1.002)	.2267
Right atrium	0.997 (0.996–0.997)	<.0001*	1.000 (0.999–1.001)	.6930
Left ventricle	1.000 (0.999–1.001)	.5766	N/A
Right ventricle	0.999 (0.998–1.000)	.1049	N/A
Morphological characteristics—area ratio
Left atrium/right atrium	0.491 (0.434–0.555)	<.0001*	0.806 (0.724–0.897)	.0001*
Left ventricle/right ventricle	0.540 (0.498–0.587)	<.0001*	0.813 (0.735–0.899)	.0001*
Morphological characteristics—ellipticity
Left ventricle	1.159 (1.103–1.218)	<.0001*	1.018 (0.988–1.049)	.4226
Right ventricle	1.012 (0.875–1.170)	.8735	N/A
Morphological characteristics—others
Interventricular septal thickness (mm)	1.093 (1.017–1.175)	.0162*	1.032 (0.995–1.070)	.0913
Image score	3.794 (3.262–4.414)	<.0001*	1.761 (1.436–2.158)	<.0001*

^* p < .05. The p‐value is a measure of statistical significance that is used to determine the relevance of the predictors in their association with the outputs.

The multimodal nomogram was thus constructed based on these 4 predictors (Figures 1, 2, 5). The predicted risk points by the nomogram were strongly correlated with the HLHS in both training and test sets (Figure 6A). Using a cutoff value of 0.5, the fetuses can be classified into high‐risk and low‐risk groups. The results demonstrate a substantial difference in the pattern of HLHS between the 2 groups (Figure 6B). For instance, the risk probability of HLHS was found to be significantly higher in the high‐risk group compared to the low‐risk group (p < .05).

Figure 5.

The multimodal nomogram for the prenatal risk assessment of HLHS. LA, left atrium; LV, left ventricle; RA, right atrium; RV, right ventricle.

Figure 6.

A, Box plots indicating patterns of correlation between HLHS and the multimodal nomogram predictive risk probability in the training (upper left) and test set (upper right). B, Number of fetuses in low‐risk and high‐risk groups according to the multimodal nomogram predictive risk probability in the training (lower left) and test set (lower right).

Performance of the multimodal nomogram, RM, ML models and EM was assessed on the test set. The results showed that EM achieved the highest accuracy of 0.952, followed by the multimodal nomogram, DT and RF with the accuracy of 0.935, 0.928 and 0.928, respectively. ROC analysis indicated that both the multimodal nomogram and EM outperformed RM and 4 ML models, with AUC values of 0.991 and 0.994, respectively, compared to 0.944, 0.884, 0.975, 0.988, and 0.980 for RM, DT, RF, SVM, and XGBoost, respectively (Figure 7A; Table 3). The Delong test showed that there was no significant difference in the AUC values of the nomogram with the EM and ML models, but significant difference existed between the nomogram and the RM (p < .05). In addition, the DCA indicated improved net benefits of the EM over the multimodal nomogram on the test set (Figure 7B). Regarding the diagnostic performance of individual sonographers, the multimodal nomogram outperformed both junior‐ and intermediate‐level (3‐ and 6‐year experience) sonographers with ACC values of 0.857 and 0.885, but performed slightly worse than senior‐level sonographers (10‐year experience) with ACC value of 0.938.

Figure 7.

ROC (left) and DCA curves (right) of the RM, EM, 4 ML models (DT, RF, SVM, XGBoost), and the multimodal nomogram in the test sets.

Table 3.

Performance Evaluation of RM, 4 ML Models, EM and the Multimodal Nomogram in the Test Set

Models	Accuracy (95% CL)	AUC (95% CL)	F1 (95% CL)	Sensitivity (95% CL)	Specificity (95% CL)	p‐Value
RM	0.881 (0.737–1.000)	0.944 (0.841–1.000)	0.921 (0.801–1.000)	0.906 (0.776–1.000)	0.800 (0.623–0.977)	.0212*
DT	0.928 (0.813–1.000)	0.884 (0.928–1.000)	0.954 (0.861–1.000)	0.969 (0.892–1.000)	0.800 (0.623–0.977)	.1324
RF	0.928 (0.813–1.000)	0.975 (0.905–1.000)	0.954 (0.861–1.000)	0.969 (0.892–1.000)	0.800 (0.623–0.977)	.4810
SVM	0.905 (0.774–1.000)	0.988 (0.940–1.000)	0.938 (0.831–1.000)	0.938 (0.831–1.000)	0.800 (0.623–0.977)	.8456
XGBoost	0.905 (0.774–1.000)	0.980 (0.918–1.000)	0.938 (0.831–1.000)	0.938 (0.831–1.000)	0.800 (0.623–0.977)	.5766
EM	0.952 (0.858–1.000)	0.994 (0.960–1.000)	0.900 (0.766–1.000)	0.969 (0.892–1.000)	0.900 (0.766–1.000)	.8105
Multimodal nomogram	0.935 (0.921–0.949)	0.991 (0.970–1.000)	0.954 (0.901–0.986)	0.969 (0.908–1.000)	0.800 (0.683–0.965)

Better results are shown in bold. * p < 0.05 and the p‐value represents the Delong's test on the significant difference of AUCs between ROCs of the model and the multimodal nomogram.

Interpretable Application of the Multimodal Nomogram

The nomogram can calculate the prenatal risk probability of HLHS for any given fetus with a standard 4‐chamber US view in an interpretable way. For instance, Figure 8A presented a fetus with a left ventricle diameter of 7.2 mm, area ratios of the left and right atrium and of the left and right ventricles as 0.88 and 1.1, an IS of 0.31, and the multimodal nomogram gave an HLHS risk probability of 0.18, which was consistent with the clinical diagnosis. The nomogram's predictive points can be used to compute the percentage contribution of each predictor. In the case of this given fetus, the area ratio of the left and right ventricle made the greatest contribution at 57.7%, followed by the area ratio of the left and right atrium at 29.8%, the diameter of the left ventricle at 8.4%, and the IS at 4.1%, in decreasing order. It was observed that the predictors contributed similarly in Figure 8B, which suggests that the area ratios between the left and right ventricles or left and right atria could be considered as significant factors for assessing the risk of HLHS. Moreover, the grad‐CAM of the corresponding 4‐chamber US view in Figure 8, A and B, indicated that the RVAE correctly focused more on the ventricle during the calculation of the IS for the 4‐chamber US view.

Figure 8.

Interpretable analysis of the multimodal nomogram for predicting HLHS risk. Cases include (A) a 27 GW healthy fetus with diameter of left ventricle of 7.2 mm, area ratio between left and right atrium of 0.88, area ratio between left and right ventricle of 1.1, IS of 0.48, and (B) a 25 GW HLHS fetus with diameter of left ventricle of 5.6 mm, area ratio between left and right atrium of 0.35, area ratio between left and right ventricle of 0.69, IS of 0.67. The percentage contribution of the predictor to the total points is shown in parentheses, which were then converted into the risk probability. In the left of (A) and (B) is the multimodal nomogram with red points assigned to each variable when evaluating the risk probabilities of HLHS. In the right, the 4‐chamber US view with the corresponding grad‐CAM overlaid with color gradient which corresponds to the contribution of a pixel toward the HLHS risk, with darker color indicating the greater contribution. For comparison, a healthy fetus wrongly predicted as HLHS by the nomogram is shown in (C).

For comparison, another case when the multimodal nomogram made the wrong prediction of HLHS risk for a healthy fetus was demonstrated in Figure 8C, where the grad‐CAM of the 4‐chamber view indicated that the RVAE was unable to capture the features from the fetus heart. This suggested that during the deployment of the nomogram, in‐depth analysis from the AI model might help clinicians understand the nomogram in a more cautious way.

Discussion

This retrospective study aims to develop a multimodal nomogram for the prenatal risk assessment of HLHS. The nomogram integrates demographics and morphological characteristics extracted from the standard 4‐chamber US views, as well as the IS calculated based on an SSL strategy. The proposed multimodal nomogram demonstrated high predictive ability and superior performance, achieving an accuracy of 0.935 and an AUC of 0.991. These results suggest that the nomogram has the potential to serve as an easy‐to‐use, first‐of‐its‐kind tool for personalized diagnosis of HLHS by clinicians. The variables utilized in the nomogram suggest that, in addition to the diameter of the left ventricle, the significance of the area ratio between the left and right ventricles and between the left and right atria warrants further investigation in clinical practice. This finding is consistent with a recent study which utilized echocardiographic images to compare healthy and HLHS fetal hearts. ³⁰ The study indicated that the area ratio between the left side and the entire heart was significantly different between healthy and HLHS fetuses. In addition, it is suggested that the ventricular septum should be integrated as one of the diagnostic criteria for HLHS. ⁴ And our study demonstrated the significance of interventricular septal thickness in association with HLHS in a ULR analysis (Table 2).

Compared to the conventional risk models based on clinical variables using logistic regression and ML algorithms, the multimodal nomogram shows a substantial enhancement in performance with regard to ACC and AUC. This improvement is likely due to the integration of the US IS. This finding is consistent with recent studies ³¹ , ³² , ³³ that suggest AI algorithms, including radiomics and deep learning models, can substantially improve the diagnosis compared to conventional approaches. Meanwhile, it is suggested that AI models solely depending on US images might not be good enough to operate independently in diagnosing HLHS, and collaboration between humans and AI will be key for effective clinical diagnosis. ³⁴ Our study demonstrates that with the help of clinicians to delineate the boundaries of ROI in 4‐chamber views, the multimodal nomogram is able to achieve similar performance with human experts, demonstrating the potential of future clinical application.

In contrast to previous studies that utilized supervised learning‐based AI models for the development of the so‐called deep learning radiomics nomogram, ¹⁹ , ³¹ , ³⁵ this study employed a SSL model, specifically RVAE, which demonstrated its effectiveness as an alternative to label‐intensive supervised learning‐based models. This study is among the first to incorporate SSL techniques in the creation of nomograms, which results in a reduced need for human annotations ¹⁷ and allows for pretraining of RVAE on public ultrasound image datasets without labels.

Accurately diagnosing HLHS from 4‐chamber views can be challenging for junior sonographers. Our study shows that the multimodal nomogram achieves similar performance to EM, but significantly outperforms less‐experienced sonographers (ie, those with 6 years or less of experience). Given the urgent need for experienced sonographers in many rural areas and the financial burden associated with uncertainties involved during US diagnoses of HLHS, ³⁴ , ³⁶ the proposed multimodal nomogram offers a reliable alternative for assessing the risk of HLHS during the pregnancy, particularly for junior sonographers or when the medical resources are scarce.

There are some limitations to this study. First, the data used were collected from 2 campuses of our hospital located in central China. Therefore, this dataset may not be representative of pregnancy distribution in other regions. To validate the generalizability of the nomogram, an additional validation cohort from other parts of China or an international cohort should be investigated in further studies. Second, additional human efforts were required to compute morphological characteristics in the US, as the outlines of the ventricle and atrium must be manually delineated. It is recommended that future work might integrate a deep learning‐based segmentation model to greatly reduce manual efforts and create an end‐to‐end diagnosis tool for HLHS risk assessment. Third, magnetic resonance imaging (MRI) examinations are increasingly popular for diagnosing HLHS. ³⁷ , ³⁸ Further studies are needed to determine how the inclusion of MRI images as a complementary imaging modality might impact the development of the multimodal nomogram.

Conclusions

The application of nomograms in clinical diagnosis has gained popularity in recent years due to their simplicity, intuition, and interpretability. ³⁵ , ³⁹ The proposed multimodal nomogram demonstrates its superiority, effectiveness, and interpretability in prenatal risk assessment of HLHS, which could assist in better individual diagnosis and treatment during pregnancy.

Supporting information

Table S1. Statistics of ultrasound examinations and HLHS in various manufacturers of ultrasound devices enrolled in this study.

Figure S1. LASSO regression for RVAE feature selection using iterative fitting with 5‐fold cross‐validation.

Data Availability Statement

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.