Automated placental abruption identification using semantic segmentation, quantitative features, SVM, ensemble and multi-path CNN

Abstract

The placenta is a fundamental organ throughout the pregnancy and the fetus’ health is closely related to its proper function. Because of the importance of the placenta, any suspicious placental conditions require ultrasound image investigation. We propose an automated method for processing fetal ultrasonography images to identify placental abruption using machine learning methods in this paper. The placental imaging characteristics are used as the semantic identifiers of the region of the placenta compared with the amniotic fluid and hard organs. The quantitative feature extraction is applied to the automatically identified placental regions to assign a vector of optical features to each ultrasonographic image. In the first classification step, two methods of kernel-based Support Vector Machine (SVM) and decision tree Ensemble classifier are elaborated and compared for identification of the abruption cases and controls. The Recursive Feature Elimination (RFE) is applied for optimizing the feature vector elements for the best performance of each classifier. In the second step, the deep learning classifiers of multi-path ResNet-50 and Inception-V3 are used in combination with RFE. The resulting performances of the algorithms are compared together to reveal the best classification method for the identification of the abruption status. The best results were achieved for optimized ResNet-50 with an accuracy of 82.88% ± SD 1.42% in the identification of placental abruption on the testing dataset. These results show it is possible to construct an automated analysis method with affordable performance for the detection of placental abruption based on ultrasound images.

1. Introduction

The placenta plays an important role throughout the pregnancy and the health of the mother and fetus is closely related to its proper function. The health of the placenta may be evaluated in multiple sessions during routine prenatal visits. It is essential to assess the details of the placenta in the standard ultrasonography (US) process to evaluate the wellbeing of the mother and her unborn fetus. The radiological images help obstetricians screen for an array of diseases, including identifying abnormal functioning placenta for prompt management decisions. Placental abruption or abruptio placentae is defined as the premature separation, partial or complete, of the normally implanted placenta from the internal lining of the uterus, potentially leading to life-threatening bleeding during the second half of pregnancy [1]. It is a serious complication of pregnancy that contributes to both maternal and infant morbidity and mortality [2,3]. Therefore, a timely diagnosis of placental abruption is vital for preventing adverse perinatal outcomes.

Researchers have recently proposed methods for placenta segmentation to assess abnormal morphological changes. Compared to other imaging approaches including Computerized Tomography (CT), Magnetic Resonance (MRI), and Positron Emission Tomography (PET), the lower resolution of ultrasound waves makes the analysis of ultrasound images more challenging. Wang et al. [4] developed a semi-automatic segmentation method that uses random forest for training the labelled data. They proposed a framework for placenta segmentation that applies individual slices MRI. Conversely, Stevenson et al. [5] used a combination of random walk and virtual organ analysis for extracting volumetric shapes of placenta ultrasound images. Wang et al. [6] introduced the Four-Dimensional Graph Cuts approach that analyses the statistical relation between slice and image connection for segmentation adjustment. A fully automated segmentation mechanism has been proposed by Alansary et al. [7] using the multiscale 3D convolution neural network to identify the areas with a high probability of placenta followed by a conditional field estimator for adjustment of the results of segmentation. A combination of the convolutional neural network and random walk statistical analysis is used by Looney et al. [8] for the extraction of the volumetric shape of the placenta. Miao et al. [9] introduced a method for segmentation and display of the placenta to provide a platform for pathological analysis by the obstetricians. A more advanced convolutional neural network segmentation method for volumetric segmentation of the placenta was proposed by Looney et al. [10] to estimate the first-trimester volume of the placenta in the real-time 3D ultrasound image. Han et al. [11] developed a platform for the detection of the placenta using a U-Net convolutional neural network. Yang et al. [12] proposed a method for real-time segmentation of the fetus, placenta, and gestational sac in 3D ultrasound images.

They developed a 3D convolutional network with recurrent connections for fine adjustment of segmentation results using the placenta-specific semantic information. Wang et al. [13] introduced a method based on interactive learning for improving the segmentation of the convolutional neural network.

Ultrasound imaging is used for routine monitoring and assessment of fetal well-being and growth. However, previous studies were limited to using MRI to assess placenta morphology, and to the best of our knowledge, no method has been utilized ultrasound imaging to assess placenta morphology and its change sequentially. Therefore, the purpose of this study is to develop a semantic segmentation method for extracting placenta images using ultrasound images that help predict potential placental abruption using machine learning methods. This method can be utilized by clinicians to monitor the status of the placenta in real-time during routine prenatal clinical visits before the condition worsens.

2. Materials and methods

The diagram for processing the ultrasound images is shown in Fig. 1 which consists of image pre-processing, placenta segmentation, quantitative feature extraction, and machine learning classifications. The details of these methods are presented in the following subsections.

Fig. 1.

The image processing and classification structure for ultrasonography images.

2.1. Ultrasonography image dataset

This study is focused on the analysis of the ultrasound images produced during prenatal visits at ≥ 24 weeks of gestation. A positive sonographic finding was defined as showing a subchorionic or retroplacental hematoma. Abruption at delivery was defined as a clinical diagnosis of placental abruption made at the time of delivery, whether by concurrent signs and symptoms or by examination of the placenta. A total of 645 images were extracted from 97 sessions with a medical diagnosis of placental abruption. These images belong to 61 pregnancies. A total of 453 images are extracted from the 97 sessions with reported normal pregnancies (controls). These images belong to 66 pregnancies.

The placental abruption is categorized into three grades. The symptoms of grade one include a small amount of bleeding and uterine contractions with no signs of fetal or maternal stress. The symptoms of grade two include mild to moderate amount of bleeding, uterine contractions, and signs of fetal stress. The grade three symptoms include moderate to severe bleeding or concealed bleeding, severe uterine contractions, abdominal pain, and low maternal blood pressure.

The images that are used in this work have the placental remark at the time of diagnosis in the ultrasound procedure notes. The imaging data is prior to the diagnosis with a placental abruption date. The staging is not presented at the time of imaging. All the 194 imaging sessions are randomly selected from the ultrasound prenatal visits in the 15 Kaiser Permanente Southern California (KPSC) medical centers from January 1, 2018, to December 31, 2018. The images are collected from various Kaiser Permanente hospitals and healthcare facilities with the same type of ultrasound machines.

Placental abruption was ascertained by diagnosis codes O45.⋆, according to the International Classification of Diseases, Tenth Revision; Clinical Modification (ICD-10-CM). These structured diagnosis codes have previously been reported to accurately capture placental abruption cases [14]. The images were reviewed and selected manually to carry the placental label from one of the anatomical viewing angles. Eighty percent of the case and control sessions were randomly selected for training and the remaining 20% of the data were used for testing, respectively. The details of the dataset are presented in Table 1. All extracted DICOM images were warped into a standard rectangular shape with 1280 × 960 size from the original conical shape of ultrasound imaging. Gray-scale normalization is applied for balancing the contrast of the images, flattening the distribution of the intensity while distributing the values of the pixels in the range of [0,255] [15]. The pixel values are normalized to serve the consistency of the quantitative feature extraction. Ultrasonography images could have a different number of pixels and the angle of scanning. The training dataset is used for the learning process of placental abruption classifiers while the test dataset is used for performance evaluation. The study is approved by the KPSC Institutional Review Board with a waiver of informed consent.

Table 1.

Number of the imaging sessions, patients and images for testing and training of the classification algorithm. 80% of the studies have been selected for training and 20% of the data have been used for testing.

Imaging Studies		Number of Sessions	Number of Patients	Number of Images
Training	Cases	78	49	510
	Controls	78	51	362
Testing	Cases	19	12	135
	Controls	19	15	91

2.2. Semantic segmentation

The location of the placenta is determined anatomically according to the relative position of the main body from the equator of the uterus. The placenta's relative position could be left or right, anterior, posterior, fundal, or a mixture of them. The attachment position of the placenta is identified by radiologist observations, and the size and shape are achieved by an automated segmentation algorithm. The segmentation of the placenta is achieved through two steps:

• (1)Identification of the surface of the placental attachment: The area that strictly follows the basal decidua between the placenta and uterus is identified as the placental attachment surface. The local basal decidua shows a multidirectional low-intensity contour that is used for delineation of the placenta. The surface between the upper and lower layers of placental attachment is identified using the contrast between the high density of decidua and the lower density of the placenta.
• (2)Identification of surface of the placenta on the uterine side: The region that lies on the border of the placenta and the amniotic fluid is identified as the uterine side of the placenta. The uterine side of the placenta is identified from the contrast between the lower intensity region of amniotic fluid and the higher density region of the placenta.

The wall of the uterus is identified through a curve matching algorithm. The semantic segmentation algorithm finds the most inner harder tissue as the uterus wall. The edge detection algorithm, followed by curve fitting, applies to the images for the identification of the placental attachment surface. The uterine surface is detected using a tissue window that is adapted to the intensity of the images to identify the placenta in the amniotic fluid [16]. Two samples of image pre-processing are shown in Fig. 2. The original ultrasound images of the placental area (left), image warping result to convert the conical sonography image to a standard rectangle shape (middle) and the wavelet filtering using Daubechies transformation (right) are shown in this figure. A guided filter is used to adjust the preliminary placental segmentation to the shape of the placenta accurately. This approach will be presented in the next section.

Fig. 2.

The processing of two sample ultrasonography images. (a) and (d) show the original ultrasound image of placental area for the cases, (b) and (e) show the image warping result to convert the conical sonography images to standard rectangle shape and, (c) and (f) show the Daubechies transformation for two sample ultrasound images.

2.3. Guided filter

The guided filter adjusts the boundary of the preliminarily assigned region to segment the placental pixels accurately. The recursive parameter estimation is used for the fine-tuning transformation function [17]. The output image $F$ is achieved by transformation of the input image $I$:

$$F_{i,j}={T\left(I_{i,j}\right)}_{I_{\left(i,j\right)\in N_p}}$$ (1)

in which, $I_{i,j}$ is the pixel intensity in row $i$ and column j, $T_n(.)$ is the transfer function, $F_{i,j}$ is the intensity of the transformed image and $N_p$ is a cluster of pixels with radius $\mathcal{R}$ centered at the pixel $p$.

The object function $C$ is defined based on the $F_{i,j}$ in Eq. (1):

$$C=\sum _{\forall i,j\in N}{\left(F_{i,j}-I_{i,j}\right)}^2+\epsilon$$ (2)

in which $\epsilon$ is a small positive value. The object function should be minimized based on the semantic regulations and parameter estimation. The semantic transformation ${T\left(I_{i,j}\right)}_{I_{\left(i,j\right)}}$ is adapted to each neighbourhood of $N_p$ pixels to minimize the cost function in Eq. (2) according to the following regulations (Fig. 3(a)):

• 1.The intensity of placental pixels remains in the $\left[{I_L}_{th},I_{H_{th}}\right]$ boundary. The ${I_L}_{th}$ relates to the lowest intensity and $I_{H_{th}}$ relates to the highest intensity of the placental pixels.
• 2.The initial estimation of the placental pixels is based on the histogram distribution of the $N_p$ cluster. The peaks of the histogram are identified. The pixels that rely on the 3-Decibel window of the brighter peak of the histogram are assigned to the placental estimated region.
• 3.The pixels in the $N_p$ cluster between the wall of the uterus and the amniotic fluid is selected using the region growing based on the gradient-guided filter.

Fig. 3.

(a) The boundaries of the segmentation pixels $I_{i,j}$ are recursively transferred by $T\left(I_{i,j}\right)$ to the accurate boundaries based on the semantic regulations in the $N_p$ pixel and (b) the block diagram for recursive adjustment of the segmentation pixels.

The transformation could be expressed in the following multiparameter arrangement:

$$T_n\left(I_{i,j}\right)={\mathrm{\Phi }}_n{I}_{N_p}{+e}_n,i,j\in N_p$$ (3)

in which, $T_n(.)$ is the transfer function on iteration $n$ that maps the pixel intensity to the next iteration, $I_{N_p}$ is the intensity of the pixels in the neighboring $\mathcal{R}$ distance, and $e_n$ is the mismatching error. $e_n$ is identified as the number of pixels with intensities lower than $I_{Pl}$ or higher than $I_{bn}$ that are assigned to the placental region. This term is a measure of the error for the pixels that are assigned to the placenta. The covariance matrix could be defined for ${\mathrm{\Phi }}_n$ in Eq. (3) as:

$$R_n=E\left[{\mathrm{\Phi }}_n{\mathrm{\Phi }}_n^T\right]and{P}_n={\hat{R}}_n^{-1}$$ (4)

The recursive parameter adjustment relation in Eq. (4) could be obtained as [18] (Fig. 3(b)):

$${\hat{\mathrm{\Phi }}}_{n+1}={\hat{\mathrm{\Phi }}}_n+\frac{1-{\Delta }_2}{1-{\Delta }_1}\left({\hat{\mathrm{\Phi }}}_n-{\hat{\mathrm{\Phi }}}_{n-1}\right)-P_n{I}_{N_p}\left\{{\Delta }_1\frac{1-{\Delta }_2}{1-{\Delta }_1}{I}_{N_p}\left({\hat{\mathrm{\Phi }}}_n-{\hat{\mathrm{\Phi }}}_{n-1}\right)\right\}-{\Delta }_2{\Delta }_2{e}_n={\hat{\mathrm{\Phi }}}_n+\Delta {\hat{\mathrm{\Phi }}}_n$$ (5)

$$P_n=\frac{1}{1-{\Delta }_1}{P}_{n-1}-\frac{{\Delta }_1}{1-{\Delta }_1}\frac{P_{n-1}{I}_{N_p}{I}_{N_p}^T{P}_{n-1}}{{\Delta }_1{I}_{N_p}^T{P}_{n-1}{I}_{N_p}-{\Delta }_1}$$

The formulation of Eq. (5) shows that the computational cost of the algorithm is proportional to the dimension of the problem which is mainly related to the radius $\mathcal{R}$. This filtering is essentially a weighted average on the $N_p$ pixels in the cluster. Therefore, the abrupt changes in intensity $I$ can be mostly preserved in $F$ and this averaging also reduces the noise. Motivated by this concept, the guided filter parameters are adjusted to gain more edge information from the original image and restore the spatial semantic segmentation loss caused by noise and a relatively low sampling rate of ultrasound images. The guided filter module also reduces the imaging noise of the raw image. The selection of the number of neighboring pixels is a compromise between the segmentation accuracy and the computation time. The guided filter is optimal for placental image processing because it often smooths away the small structures that are not related to the placenta and eliminates the noise and interferences that are common in ultrasound imaging. The ultrasound wrapped image into the rectangular standard Fig. 4(a), Daubechies wavelet filtered image Fig. 4(b), thresholding for finding hard tissue Fig. 4(c), the result of the heuristic search for finding the uterus wall Fig. 4(d), placental semantic segmentation and the guided filter results Fig. 4(e), and the heatmap of placental pixel intensities Fig. 4(f). The input $I_{N_p}$ of the guided filter module is the raw image and the input $e$ is the number of misclassified segmented pixels according to the semantic regulations. The algorithm is implemented for one channel of data to fit the gray-scale ultrasound images. Color information, including Doppler map or other multi-channel data, is removed from the raw image. The size of images is standardized to 1280 × 960 pixels, and the $\mathcal{R}$ is set to 30. ${I_L}_{th}$ is set to 0.08, $I_{H_{th}}$ is set to 0.91, $I_{Pl}$ is set to 0.12 and $I_{bn}$ is set to 0.86 for this study.

Fig. 4.

(a) The ultrasound wrapped image into the rectangular standard, (b) Daubechies wavelet filtered image, (c) thresholding for finding hard tissue, (d) result of the heuristic search for finding the uterus wall, (e) placental semantic segmentation and the guided filter results, and (f) the heatmap of placental pixel intensities.

The multi-stage semantic segmentation is used in this study to provide robust segmentation of the placental region against the low-resolution of ultrasonography images and the imaging noise. Wavelet filtering reduces the effect of the interference and noise, and the following stage of guided filtering improves the robustness of segmentation by adjustment of the boundary of the placenta based on the regional intensities rather than the individual pixels [19]. The covariance matrix in the guided filter extracts the fundamental area of the placental region, and the recursive adjustment of the boundaries provides a robust segmentation by fine tuning the boundaries using the eigen vectors with lower eigen values. Furthermore, the region of placenta is extracted based on the histogram for each image and the lower and upper thresholds of the intensities are calculated for individual image. The adaptive selection of these parameters also provides robust segmentation algorithm.

2.4. Radiomic feature extraction

A total of 104 quantitative features are extracted from each image (see the detailed list in Appendix). The segmented placental regions are used for extraction of the features in ultrasonography images of both placental abruption cases and normal subjects to be used in the classification algorithm. These features are from six groups: 19 First Order Statistics features, 10 Two-Dimensional Shape features, 24 Gy Level Co-Occurrence Matrix (GLCM) features, 16 Gy Level Run Length Matrix (GLRM) features, 16 Gy Level Size Zone Matrix (GLSZM) features, 5 Neighboring Gray Tone Difference Matrix (NGTDM) features and 14 Gy Level Dependence Matrix (GLDM) features (see Ref. [20] for the details). The features are extracted from the images following the normalization of the intensity of the segmented pixels. The first order statistics and the shape features are normalized using the pixel separation intensity to overcome the unitless relative measures in the ultrasound images.

2.5. Individual image versus session level analysis

Multiple images with different anatomical plans are typically captured within a single imaging session. We performed a session level imaging study compared to the individual image analysis, in which multiple feature vectors within a single imaging session are merged into a single feature vector. The Principal Component Analysis (PCA) is used to combine the features of a single session [21]. The eigenvector of the corresponding larger eigenvalue of the feature vectors is gained. $x_i$ with $i=1,2,\dots ,N_j$ are $D$ dimensional vectors with $N_j$ feature elements in group $j$. The $D\times D$ correlation matrix is calculated as:

$$R=\frac{1}{N_j}\sum _{i=1}^{N_j}\psi \left(x_i\right){\psi \left(x_i\right)}^T$$ (6)

where $\psi \left(x_i\right)$ is the kernel function of $x_i$. The eigenvectors of $R$ in Eq. (6) are calculated as:

$$E_j=\underset{{\lambda }_k}{\mathrm{argmin}}\left\{\left|R-{\lambda }_k{I}_D\right|\right\}$$ (7)

The largest eigenvalue ${\lambda }_j=\mathit{max}\left({\lambda }_k\right),k=1,2,\dots ,D$ for group $j$ and the related eigenvalue $E_j$ is used for the calculation of the representative feature vector of the group $j$. The feature vector $y_j$ for group $j$ is in Eq. (7) are calculated as:

$$y_j={\psi \left(x_k\right)}^T{E}_j$$ (8)

The Gaussian kernel function $\psi \left(x_i\right)$ has been used in Eq. (8) for this study.

2.6. Machine learning classifiers

The performance of several classification methods is evaluated in this study. The classifiers include kernel based SVM, Ensemble Classifier, and Convolutional Neural Network (CNN). Five-fold cross-validation is used for all classifiers in this work which can result in the reduction of the overfitting as much as possible and in obtaining more information from the training feature vectors. This approach also improves the robustness of the classifiers for the training data with a high degree of diversity. The details of classifiers and parameter optimization are described in the next section.

Since the SVM and Ensemble classifiers are not efficient for analysis of two-dimensional inputs like images, the quantitative feature extraction is used in this work to assign a one-dimensional feature vector that represents the information in each image. However, convolutional neural networks can directly process two-dimensional images. Also, the multi-path CNN structures can fuse the extracted features in multiple depth of layers [22]. The multi-path CNN combines the three channels of image, wavelet filtered images and quantitative features in the matrix arrangement. The data fusion in the anchor layers combines the data for further processing in the next layers [23].

2.6.1. Kernel based Support Vector Machine

The SVM classifier [24] is used to categorize the data vectors into normal and placental abruption cases. The training model separates the input data vectors into two different groups based on the labels. The kernelized hyperplane is adjusted by the SVM algorithms to provide maximum separation of data categories. The goal of the SVM algorithm is to optimize the classifier parameters by minimizing the following function:

$$\min C\sum _{i=1}^m\left[y^i{cost}_1\left({\theta }^T{x}^i\right)+\left(1-y^i\right){cost}_0\left({\theta }^T{x}^i\right)\right]+\frac{1}{2}\sum _{i=1}^n{\theta }_j^2$$ (9)

in which $\frac{1}{2}\sum _{i=1}^n{\theta }_j^2$ is the objective function term, $C$ is the factor of a trade-off between the margin size and the correct classification, and ${cost}_1\left({\theta }^T{x}^i\right)$ and ${cost}_0\left({\theta }^T{x}^i\right)$ are the cost function terms related to one and zero values for $y$, respectively. The original data space is mapped into the kernel function $k\left(x,y\right)$ space to overcome the non-linear separation of hyperplanes using high dimensional parameters. The kernel function is selected amongst Gaussian, linear, quadratic, and cubic functions. The optimized hyperplane in the kernel space is identified by the linear combination of the quantitative feature vectors $x_i$ with ${\alpha }_i$. The mapping coefficients for point $x$ are calculated from the following relation to minimize the function in Eq. (9):

$$\sum _{i=0}{\alpha }_{i}k\left(x_i,x\right)=constant$$ (10)

Eq. (10) minimizes the relative distance between the test and data feature vectors and the hyperplane.

2.6.2. Ensemble classifier

Ensemble learning is used as another classification method in this work. $N$ sample data vectors are obtained through the related observations. The weak learners are separately defined for each observation and the strong learners are obtained by combining the weak learners. The adaptive boosting algorithm trains the weak learners to obtain the accuracy of these base learners at the first step. In the second step, the weights of the weak learners are assigned based on their performance on original data vectors. The learners with better performance are assigned with higher weights and the learners with lower weights are assigned with lower weights. The strong learners are trained based on the base learners and their assigned weights to reduce the effect of learners with higher misclassification rates. This process is continued iteratively until a certain error criterion or number of epochs is reached.

2.6.3. Multi-path CNN classifiers

Two standard CNN structures of Resnet-50 [21] and Inception-V3 [22], have been modified to multi-path CNN (MP-CNN) [25] structures to be used as deep classifiers in this work. These structures are evaluated, and their performances are compared with each other. The MP-ResNet-50 network is shown in Fig. 5(a). The parallel bypassing nodes of this network allow for preserving the highly weighted outputs of previous layers by passing them to the next layers directly. The three parallel paths include image input with 1280 × 960 size, wavelet filtered image input with 640 × 480 size and quantitative features input with 104 sizes are applied to the input channels of the network. We separated the three input channels of the standard ResNet-50 for RGB images into the ultrasound embedded, the wavelet image embedded and the feature embedded channels. The 104 components of the feature vector are arranged in a 13 × 8 matrix to match the input dimension of the channel. The network layers are adjusted to match the different sizes of the parallel paths to the anchor layer that combines the three paths into one path of the MP-CNN.

Fig. 5.

The structure of (a) MP-ResNet-50 with 50 convolutional layers and (b) MP-Inception-V3 with 48 convolutional layers.

The structure of the MP-ResNet-50 would keep the high-resolution details of the images by avoiding the convolutional filter and rectifying layers. The higher effective resolution would be beneficial for the detection of anomalies and symptoms. The network output is also adapted to “Case” and “Control” using a fully connected neural network. The MP-Inception-V3 structure is shown in Fig. 5(b). The three parallel paths include image input, wavelet filtered image and quantitative features input similar to the previous network. The network layers are adjusted in to match the different sizes of the parallel paths to the anchor layer that combines the three paths into one path of the MP-CNN.

The SVM and Ensemble classifiers are not efficient for analysis of two-dimensional inputs like images. The quantitative feature extraction is used in this work to extract one dimensional feature vector that represent the information in the image. However, convolutional neural networks can directly process two dimensional images and matrices. The multi-path CNN that is used in this work combines the three channels of image, wavelet filtered images and quantitative features in the matrix arrangement.

The multi-resolution structure of the parallel paths in this network allows for the extraction of high-resolution and low-resolution information at the same time. The output of this network is also adapted to the two categorical states.

2.6.4. Classifier and feature optimization

The hyper-parameters of the classifiers are optimized using the Bayesian optimization method [26] for both SVM and Ensemble methods. The Bayesian optimization in this work is using an acquisition function that provides the next evaluation point in each step. Using the acquisition function leads to balancing the sampling points in the regions with low modelled aim functions and exploring areas that have not yet been properly determined by the model. The loss function is used as the measure of the fitness of the optimization algorithm.

The Recursive Feature Elimination (RFE) method [27] is used to select the most useful predictive features in 104 quantitative features. The basic aim of the RFE is to eliminate the features that have a minimum effect on the performance of the classification in sequential order. The change in loss function is measured when each feature is removed at every iteration of the algorithm. The process will be repeated for the remaining features until the last element of the feature vector. The number of features that provide a performance of at least 95% of the maximum value is selected as the optimal length of the feature vector. The high-ranked quantitative features that survive the elimination process may not directly be related to the classification outcome, however, the subset of the features improves the classifier performance.

In this study, all image processing and machine learning methods are implemented using MATLAB 2021a Image Processing and Deep learning Toolboxes [21,28]. The training of the models is performed using CUDA-capable NVIDIA Quadro K1100 M GPU and Parallel Processing Toolbox to increase the speed. An image warping is applied on the images to transform the ultrasound image pixels into standard rectangular images and distribution flattening is applied to provide consistency for quantitative feature extraction as described in Section = 2.4.

3. Results

The individual image analysis is performed according to the approach described in Section 2. All 104 features are used to tune the SVM classifier within the parameters of kernel function, box constraint level, and kernel scale in the feature space, and the Ensemble classifier within the parameter of ensemble method, maximum number of splits, number of learning cycles and learning rate. The plot of the fitness function for 200 iterations of hyperparameter optimization is shown in Fig. 6(a) for the SVM classifier and Fig. 6(b) for the Ensemble classifier, respectively. The tuning process for SVM achieved the best model at iteration 158 and the Ensemble model reached the optimized fitness at iteration 51.

Fig. 6.

The plot of observed fitness function for 200 iterations of the Bayesian parameter optimization of (a) the SVM classifier with optimum fitness function value of 0.21622 and (b) the Ensemble classifier with optimum fitness function value of 0.16351.

The plots of RFE method for optimization of the feature vectors are shown in Fig. 7(a) for SVM and Fig. 7(b) for Ensemble classifier. The accuracy reached 95% of maximal performance value at 52 features for SVM and 72 for Ensemble, respectively. Similarly, the hyperparameters for these selected features are re-tuned again. The performances of these classifiers on the training dataset are shown in Table 2. The accuracy, sensitivity, specificity, positive predictive value (PPV) and area under curve (AUC) of both SVM and Ensemble classifiers for all the features and selected features are depicted in this table. It could be observed that the decision tree with AdaBoost Ensemble reached the best AUC of 92.24% and the best accuracy of 83.24%.

Fig. 7.

The plots of RFE method for optimization of the feature vectors in image level experiment (a) for SVM classifier that leads to 55 features and (b) for Ensemble classifier that leads to 52 features. The plots of RFE method for optimization of the feature vectors in session level study (c) for SVM classifier that leads to 52 features and (d) for Ensemble classifier that leads to 72 features.

Table 2.

Comparison of the performance of the SVM and Ensemble classifier on the training dataset. The values are shown with all the features and selected features based on the RFE method.

	Accuracy	Sensitivity	Specificity	PPV	AUC
All Features in Image Level
SVM (Gaussian kernel)	78.92	70.37	87.85	85.81	82.38
Ensemble Trees (AdaBoost)	80.81	79.10	82.60	82.60	90.04
Selected Features in Image Level
SVM (Gaussian kernel)	81.22	71.96	90.88	89.18	83.04
Ensemble Trees (AdaBoost)	83.24	82.28	84.25	84.51	92.24
All Features in Session Level
SVM (Gaussian kernel)	78.71	74.03	83.33	81.43	82.83
Ensemble Trees (AdaBoost)	80.65	76.62	84.62	83.10	89.13
Selected Features in Session Level
SVM (Gaussian kernel)	78.06	71.43	84.62	82.09	78.99
Ensemble Trees (AdaBoost)	80.12	75.64	84.61	82.53	88.87

In the sessional level analysis, the quantitative features of each imaging session are grouped as a single feature vector as described in Section II.E. A parameter optimization method similar to the individual image study is used for session level classifiers. The plots of RFE method for optimization of the feature vectors are shown in Fig. 7(c) for SVM and Fig. 7(d) for the Ensemble classifier. The performance of both SVM and Ensemble classifier for all features and selected features on the training dataset are shown in Table 2. The decision tree with AdaBoost Ensemble resulted in an AUC of 88.87% and an accuracy of 80.12% for the full length of the feature vector. It could be observed that the AUC has a reduction of 3.11% and accuracy has a reduction of 2.59% compared to the individual image analysis.

The results for session level analysis shows under performance compared to individual image analysis in SVM and Ensemble classifiers. The reason for these results is that the variance of the input vectors that are applied to classifiers is reduced as the result of feature vector extraction. Extraction of a single vector with the highest variance from all the feature vectors of the patient reduces the diversity of the inputs.

The deep learning methods are only applied for individual image analysis due to the performance reduction of session level in both SVM and Ensemble classifiers compared to individual image analysis. The RFE feature optimization is used for these networks that led to 54 features for MP-ResNet-50 and 27 features for Inception-V3. The observation with 30 repetitions resulted in 85.95% ± SD 1.16% accuracy for MP-ResNet-50 and 87.68% ± SD 0.75% for Inception-V3. The performance of these two CNN classifiers is better than those of SVM and Ensemble classifiers.

The optimized model of each classifier is applied to the test dataset to evaluate their performance. The results for the testing dataset are shown in Table 3 for SVM and Ensemble Classifiers and Table 4 for two CNN classifiers. The MP-ResNet-50 Classifier led to the highest accuracy of 82.88% and PPV of 84.71%.

Table 3.

Comparison of accuracy, sensitivity, specificity, PPV and AUC for SVM classifier and Ensemble classifier on the test dataset for individual image level and session level classification. The feature vector is optimized for all the experiments.

	Accuracy	Sensitivity	Specificity	PPV	AUC
Image level
SVM (Gaussian kernel)	70.68	69.00	72.53	73.40	76.73
Ensemble (AdaBoost)	76.96	79.00	74.73	77.45	81.44
Session level
SVM (Gaussian kernel)	68.42	63.16	73.68	70.59	72.85
Ensemble (AdaBoost)	71.05	73.68	68.42	70.00	72.58

Table 4.

Comparison of accuracy, sensitivity, specificity, PPV and AUC for SVM classifier and Ensemble classifier on the test dataset for individual image level and session level classification. The mean and the standard deviation are achieved from 30 experiments for each method. The feature vector is optimized for all the experiments.

	Accuracy	Sensitivity	Specificity	PPV	AUC
ResNet-50	82.88 ± SD 1.42	82.17 ± SD 2.74	83.66 ± SD 1.99	84.71 ± SD 1.49	88.31 ± SD 1.18
Inception-v3	81.80 ± SD 1.72	81.23 ± SD 3.33	82.42 ± SD 2.53	83.59 ± SD 1.85	87.06 ± SD 1.18

4. Discussion

The identification of the placental abruption through the general ultrasonography images is a challenge. Our study has developed an automatic identification system to effectively segment the placenta, extract radiomic quantitative features and identify placental abruption by applying several machine learning classifiers. The results of the identifications showed that semantic segmentation of the placenta in the ultrasound images followed by quantitative feature extraction and the optimized classifier is capable of identifying the abruption cases. Semantic segmentation is used for finding the edges of the placenta using the recursive guided filter adjustment. This method is using adaptive intensity data for the identification of placental pixels compared to the more intense regions of the uterus and bones and the less intense regions including amniotic fluid and the liquids.

In this study, multiple classification methods including kernel based SVM, decision tree ensemble and CNN classifiers are applied at individual image level and sessional level. The studies show that 1) the performance of optimized models at the individual image level is superior to the optimized models at the session level, 2) the accuracy of the classifiers at the individual image level with selected features on the internal training dataset with cross-validation is in the range of 80% from the highest 87.40% for Inception-V3 to the lowest 81.22% for SVM, and 3) when each optimized model is applied to the test dataset, the performance shows some underperformance regardless of the classifiers. Although this underperformance could be potentially due to overfitting, the performance of the CNN classifiers has a small reduction and overcomes SVM and ensemble classifiers significantly. Both the SVM and the decision tree ensemble are more dependent on the number of features and tunning of the parameters while the CNN classifiers show less dependency. The Highest performance model, the Inception-V3, resulted in the highest accuracy while using 27 features only, however, the SVM and the decision tree ensemble reached lower accuracies with more than twice the number of selected features. The CNN classifiers and specially Inception-V3 can extract the nonlinear relations between the input feature vectors of the network and the desired output of the algorithm. Thus, it can result in the best classification.

The number of feature vector elements is also optimized to improve the performance of the algorithms using RFE method. The number of the parameters and the feature vector components is different for each classifier. It is shown that this effect is common for quantitative features [29]. The combination of the elements determines the final performance of the method rather than individual elements of the feature vector. The components of the feature vector are not independent in general. Therefore, the selection of the feature elements is related to the sequential combination of the components that include some stochastic measures.

5. Conclusions

An automatic identification system is proposed in this study to identify placental abruption cases in routine fetal ultrasonography images. A segmentation method is applied to the sonography images and the identification is performed by extraction of quantitative features of the segments followed by an SVM, Ensemble or CNN classification. The results after optimizing the length of the feature vector and the parameters of the classifiers show that the identification with affordable accuracy is feasible. This monitoring method is performed on general sonography images; therefore, can provide a powerful tool for the identification of abnormalities that could be fatal, through the normal monitoring procedures of the pregnancy. The use of 3D ultrasonography may improve the performance of the algorithm and would be considered in future works.

Author contribution statement

Vahid Asadpour: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Wrote the paper.

Eric J. Puttock: Performed the experiments.

Darios Getahun: Analyzed and interpreted the data.

Michael J. Fassett: Analyzed and interpreted the data.

Fagen Xie: Conceived and designed the experiments; Analyzed and interpreted the data; Wrote the paper.

Funding statement

This study was supported by Kaiser Permanente Direct Community Benefit Funds.

Data availability statement

The data that has been used is confidential.

Declaration of interest's statement

The authors declare no competing interests.

Appendix. Radiomic Features

The various quantitative features that are used in this study are shown in this appendix. These features could be divided into the following classes:

• •First order statistics (19 features)
• •Shape features (10 features)
• •Gray level co-occurrence matrix (24 features)
• •Gray level run length matrix (16 features)
• •Gray level size zone matrix (16 features)
• •Neighboring gray tone difference matrix (5 features)
• •Gray level dependence matrix (14 features)

Calculation of shape features is independence of gray level values. Other features are calculated on pixel values after normalization process. Most features follow feature definitions as described by Ref. [19] (Table A.I). The selected features for classifiers in this study are presented in Table A.II.

Table A.IThe radiomic features that are extracted from the images in this study.

Feature Group	Feature Functions and their abbreviations
First Order Features (19 features)	Total Energy, Energy, Minimum, 10th percentile, 90th percentile, Entropy, Maximum, Mean, Median, Interquartile Range, Range, Mean Absolute Deviation, Robust Mean Absolute Deviation (rMAD), Standard Deviation, Maximum 2D Diameter, Major Axis Length, Minor Axis Length, Elongation, Root Mean Squared (RMS), Skewness, Kurtosis, Variance, Uniformity
Shape Features (10 features)	Spherical Disproportion, Mesh Surface, Perimeter to Surface Ratio, Perimeter, Sphericity, Pixel Surface
Gray Level Co-occurrence Matrix (GLCM) Features (24 features)	Difference Variance, Informational Measure of Correlation (IMC) 2, Inverse Difference Moment (IDM), Joint Energy, Joint Entropy, Informational Measure of Correlation (IMC) 1, Autocorrelation, Joint Average, Cluster Prominence, Cluster Shade, Cluster Tendency, Contrast, Correlation, Difference Average, Difference Entropy, Maximal Correlation Coefficient (MCC), Inverse Difference Moment Normalized (IDMN), Inverse Difference (ID), Inverse Difference Normalized (IDN), Maximum Probability, Sum Average, Inverse Variance, Sum Entropy, Sum of Squares
Gray Level Size Zone Matrix (GLSZM) Features (16 Features)	Gray Level Non-Uniformity Normalized (GLNN), High Gray Level Zone Emphasis (HGLZE), Small Area High Gray Level Emphasis (SAHGLE), Zone Percentage (ZP), Gray Level Variance (GLV), Zone Variance (ZV), Size-Zone Non-Uniformity (SZN), Small Area Low Gray Level Emphasis (SALGLE), Small Area Emphasis (SAE), Large Area Emphasis (LAE), Gray Level Non-Uniformity (GLN), Size-Zone Non-Uniformity Normalized (SZNN), Zone Entropy (ZE), Low Gray Level Zone Emphasis (LGLZE), Large Area Low Gray Level Emphasis (LALGLE), Large Area High Gray Level Emphasis (LAHGLE)
Gray Level Run Length Matrix (GLRLM) Features (16 Features)	Long Run Emphasis (LRE), Run Length Non-Uniformity (RLN), Run Length Non-Uniformity Normalized (RLNN), Short Run High Gray Level Emphasis (SRHGLE), Long Run Low Gray Level Emphasis (LRLGLE), Run Percentage (RP), Gray Level Variance (GLV), Run Variance (RV), Run Entropy (RE), Low Gray Level Run Emphasis (LGLRE), High Gray Level Run Emphasis (HGLRE), Short Run Low Gray Level Emphasis (SRLGLE), Long Run High Gray Level Emphasis (LRHGLE), Gray Level Non-Uniformity (GLN), Gray Level Non-Uniformity Normalized (GLNN), Short Run Emphasis (SRE)
Neighboring Gray Tone Difference Matrix (NGTDM) Features (5 Features)	Complexity, Coarseness, Busyness, Strength, Contrast
Gray Level Dependence Matrix (GLDM) Features (14 Features)	Gray Level Variance (GLV), Dependence Variance (DV), Dependence Entropy (DE), Low Gray Level Emphasis (LGLE), High Gray Level Emphasis (HGLE), Small Dependence Low Gray Level Emphasis (SDLGLE), Large Dependence Low Gray Level Emphasis (LDLGLE), Large Dependence High Gray Level Emphasis (LDHGLE), Small Dependence Emphasis (SDE), Dependence Non-Uniformity (DN), Large Dependence Emphasis (LDE), Gray Level Non-Uniformity (GLN), Small Dependence High Gray Level Emphasis (SDHGLE), Dependence Non-Uniformity Normalized (DNN)

Table A.II.

The optimal radiomic features that are selected using rfe algorithm for each classifier and analysis dataset.

Classifier	Analysis Dataset	Number of selected features	Selected features using RFE
SVM	Image level analysis	52	LDHGLE, LDLGLE, SDHGLE, SDLGLE, HGLE, LGLE, DE, DV, GLVGLDM, DNN, DN, GLNGLDM, LDE, SDE, Strength, Complexity, Busyness, Contrast, NGTDM, Coarseness, LRHGLE, LRLGLE, SRHGE, LRLGE, HGLRE, LGLRE, RE, RV, GLVGLRLM, RP, RLNN, RLN, GLNNGLRLM, GLNGLRLM, LRE, SRE, LAHGLE, LALGLE, SAHGLE, SALGLE, LGLZE, ZE, ZV, GLV, ZP, SZNN, SZN, GLNN, GLN, LAE, SAE, IDM, Maximum 2D diameter
Ensemble	Image level analysis	72	Median, DV, Contrast NGTDM, LRHGLE, SDLGLE, Total Energy, GLV, RP, Variance, MCC, LRE, SDHGLE, Complexity, Joint Entropy, Mean, IDN, SRHGE, Energy, Cluster Prominence, Interquartile Range, LALGLE, Difference Entropy, Joint Energy, Sum of Squares, Elongation 2D, Mesh Surface, Perimeter, LRLGE, LGLZE, HGLZE, LGLE, ZV, Minimum, Joint Average, IDMN, Sphericity 2D, RE, SAHGLE, GLNGLDM, HGLRE, Sum Average, HGLE, SRE, RMS, DE, MAD, IDM, Spherical Disproportion 2D, SAE, Auto Correlation, Sum Entropy, DNN, SZNN, LDE, Standard Deviation, GLNNGLRLM, LGLRE, LAE, RLNN, Busyness, SDE, ZP, SALGLE, LAHGLE, Range, Perimeter to Surfaceratio, GLV, MCC, LRHGLE, SRE, SAHGLE, RP
SVM	Session level analysis	55	Skewness, MAD, Tenth Percentile, Pixel Surface, Cluster Shade, Perimeter to Surface ratio, Mean, LDLGLE, HGLRE, Energy, rMAD, SDHGLE, Joint Energy, Strength, LGLZE, LRLGE, Elongation 2D, ZV, Maximum 2D diameter, LALGLE, LRHGLE, ZP, GLV, Complexity, Uniformity, DN, SAE, RLNN, Variance, Spherical Disproportion 2D, Interquartile Range, GLVGLRLM, Difference Variance, Sum of Squares, Sphericity 2D, Perimeter, Kurtosis, Maximum, GLVGLDM, RV, MCC, Inverse Variance, Correlation, SAHGLE, LRE, Sum Entropy, Entropy, Sum Average, GLNN, Median, Cluster Prominence, Auto Correlation, RMS, Joint Average, Cluster Tendency
Ensemble	Session level analysis	52	Spherical Disproportion 2D, Variance, Standard Deviation, Maximum, Ninetieth Percentile, Tenth Percentile, Minimum, Entropy, Total Energy, Energy, HGLZE, Difference Variance, Sum of Squares, Sum Average, ID, IDMN, Maximum Probability, Uniformity, rMAD, Difference Entropy, IMC2, Joint Entropy, Joint Energy, Sum Entropy, Correlation, Contrast, Inverse Variance, IDN, Difference Average, Cluster Shade, MCC, Joint Average, Cluster Tendency, IMC1, Elongation 2D, Minor Axis Length 2D, Auto Correlation, Kurtosis, RMS, Perimeter, MAD, Perimeter to Surface ratio, Range, Cluster Prominence, Pixel Surface, Mesh Surface, Interquartile Range, Skewness, Major Axis Length 2D, Median, Mean, Sphericity 2D
ResNet-50	Image level analysis	54	LAE, Maximum, GLNNGLRLM, HGLZE, Interquartile Range, Busyness, DNN, Elongation 2D, Sum of Squares, Sphericity 2D, LDE, MAD, Minimum, ZV, LDLGLE, Perimeter, IDM, RE, RP, Sum Entropy, rMAD, Maximum Probability, Mean, ID, Mesh Volume, GLV, RMS, Energy, Pixel Surface, Coarseness, LRHGLE, SZNN, DE, SDHGLE, Spherical Disproportion 2D, SDE, Contrast NGTDM, Contrast, Inverse Variance, ZE, SRE, HGLE, Median, HGLRE, LGLE, SAE, Major Axis Length 2D, Correlation, ZP, Range, Auto Correlation, Difference Variance, Maximum 2D diameter, RLN
Inception-V3	Image level analysis	27	MAD, Maximum 2D diameter, Inverse Variance, GLVGLDM, SDLGLE, Uniformity, DNN, GLNGLDM, Difference Entropy, GLNNGLRLM, Skewness, Auto Correlation, SRHGE, LAHGLE, Coarseness, Sum of Squares, Minimum, GLNGLRLM, LGLRE, Complexity, ZP, SZN, Perimeter, IDN, Kurtosis, Sum Average, Busyness