Cardiac-DeepIED: Automatic Pixel-Level Deep Segmentation for Cardiac Bi-Ventricle Using Improved End-to-End Encoder-Decoder Network

Abstract

Accurate segmentation of cardiac bi-ventricle (CBV) from magnetic resonance (MR) images has a great significance to analyze and evaluate the function of the cardiovascular system. However, the complex structure of CBV image makes fully automatic segmentation as a well-known challenge. In this paper, we propose an improved end-to-end encoder-decoder network for CBV segmentation from the pixel level view (Cardiac-DeepIED). In our framework, we explicitly solve the high variability of complex cardiac structures through an improved encoder-decoder architecture which consists of Fire dilated modules and D-Fire dilated modules. This improved encoder-decoder architecture has the advantages of being capable of obtaining semantic task-aware representation and preserving fine-grained information. In addition, our method can dynamically capture potential spatiotemporal correlations between consecutive cardiac MR images through specially designed convolutional long-term and short-term memory structure; it can simulate spatiotemporal contexts between consecutive frame images. The combination of these modules enables the entire network to get an accurate, robust segmentation result. The proposed method is evaluated on the 145 clinical subjects with leave-one-out cross-validation. The average dice metric (DM) is up to 0.96 (left ventricle), 0.89 (myocardium), and 0.903 (right ventricle). The performance of our method outperforms state-of-the-art methods. These results demonstrate the effectiveness and advantages of our method for CBV regions segmentation at the pixel-level. It also reveals the proposed automated segmentation system can be embedded into the clinical environment to accelerate the quantification of CBV and expanded to volume analysis, regional wall thickness analysis, and three LV dimensions analysis.

The workflow of the Cardiac-DeepIED.

I. Introduction

Cardiovascular diseases cause a great deal of mortality and morbidity globally, which is one of the most common reason of death worldwide [1]. Automatic segmentation of the CBV from MR images is an essential step for the quantitative analysis of the left ventricle (LV), the right ventricle (RV), and it is typically necessary for diagnoses and treatment of cardiovascular diseases [2]. However, the majority of cardiac MR images show that the similar intensity distribution in different regions, thus providing a little edge information. It makes cardiac images segmentation as a very challenging task. Due to the relatively fixed shape of the LV during the complete cardiac cycle, most of the research works only focuses on the segmentation of the LV. In fact, it is more difficult to consider the segmentation of the CBV than the segmentation of the single ventricle. However, when pathologies such as myocardial hypertrophy, myocardial infarction, and ventricular arrhythmias are to be characterized in clinical practice, the CBV segmentation makes more sense. Hence, there is an urgent need for a computational method of LV/RV segmentation simultaneously, namely CBV segmentation.

Currently, many segmentation problems require to employing prior knowledge to improve its accuracy and robustness, as well as in medical image segmentation problems. A large number of segmentation methods based on strong priors have been proposed to segment the LV, RV or CBV, including shape prior [3], active shape and appearance models (AAM/ASM) [4], and atlas-based approaches [5], [6]. Automatic CBV segmentation can benefit from these methods that consist of some assumptions. However, incorrect priori knowledge can lead to unreliable results for the cardiac CBV segmentation methods, and even if the priori information is correct, strong priori information may prevent these CBV segmentation methods from being efficiently used by clinical applications.

To circumvent limitations of these segmentation methods relying on strong priori information, cardiac images segmentation methods with weak or no prior information have grown in popularity. For image-based [7], pixel-based classification methods [8] and deformable models [9], [10], although no methods use strong prior information to segment the CBV, they suffer from a low robustness and accuracy, and require considerable user interaction. These disadvantages will lead to the segmentation accuracy always being at a low level. Therefore, there is still much more room for improvement.

Deep learning has achieved good results in the object semantic segmentation of computer vision in recent years. Inspired by it, many popular end-to-end semantic segmentation techniques of cardiac images such as fully convolutional neural networks (FCN) [11] and U-net [12] are developed. Additionally, it can be found that those approaches [13]–[18] are applied to CBV segmentation field. These methods are all made up of convolutional neural networks (CNN), which can automatically learn complex features and concepts without feature engineering or hard-code a priori knowledge. They are used to classify the pixels in the MR images into the left and right ventricle regions simultaneously. Therefore, these approaches are capable of primly solve the limitation of regression method and traditional segmentation method. However, there has a correlation between the complex shape variables with successive frame images of a cardiac cycle. The temporal dependences between images can extract more accurate and effective features to produce a good segmentation result. Unfortunately, all of the aforementioned methods do not consider the spatiotemporal correlations in the cardiac MR sequences.

In this study, we propose an improved end-to-end Encoder-Decoder network specifically designed to segment the LV, RV and myocardium (MYO) for cardiac MR sequences from pixel-level view, namely Cardiac-DeepIED. This framework is composed of Fire dilated modules, convolution LSTM (Conv-LSTM) and D-Fire dilated modules. It can automatically represent images from the pixel perspective with the spatiotemporal correlations and detect the ventricle region more accurately by leveraging advantages of deep learning to obtain the optimal segmentation result. In summary, our study makes the following contributions:

• 1)It enables an accurate, robust, and automated segmentation of CBV with less assumption. Hence, with our method, an efficient clinical tool is provided in clinical practice for accelerating the diagnosis and treatment of cardiovascular diseases.
• 2)It creatively formulates the segmentation task as a multi-classification problem to makes full use of the advantages of deep learning in holistic fashion for optimizing output while maintaining accuracy, robustness, and efficiency.
• 3)It creatively combines Encoder-Decoder (ED), Conv-LSTM and dilated convolution fusion together in one unified end-to-end framework. The novel integration has super strengths in representation learning and spatiotemporal context learning.

The rest of this paper is structured as follows: Section II presents an experimental framework to demonstrate and evaluate the efficacy of our Cardiac-DeepIED model on cardiac MR images. Section III introduces the dataset and evaluation criteria. Section IV reports the results and analysis of the evaluation. Section V reports discussions and conclusions.

II. Method

The workflow of our proposed method is shown in Fig. 1. As we can see, an ED network and a Conv-LSTM module constitute our Cardiac-DeepIED segmentation network. Moreover, skip connections are designed between the encoder and the decoder. In ED network, we replace the standard convolution layers with Fire dilated and D-Fire dilated modules consisting of dilated convolution. The details of the network will be introduced in the section II.B.

FIGURE 1.

The workflow of the Cardiac-DeepIED.

The goal of our Cardiac-DeepIED method is to segment the LV, the MYO and the RV of a series of MR image . This is done by predicting the label map of each MR image , where n represents the -th MR image. Pixels of the label map contain a label {Background, LV, RV, MYO}. Here, we employ the categorical cross-entropy loss function as the objective function of segmentation network. The formula is as follows:

where labeled data set D is used for segmentation network, denote the input data, denote the ground truth, and is the segmentation function parameterized by .

A. Review of Dilated Convolution

As we know, the Fire dilated and D-Fire dilated module mentioned above is composed of dilated convolution. Before we introduce the details of the proposed method, we will review the principle of dilated convolution. To our knowledge, dilated convolution, also known as convolution with holes or atrous convolution. The dilated convolution is widely used in semantic segmentation [19]–[21], based on the fact that it supports exponential expansion of the receptive field and captures larger context without loss of resolution or coverage.

To better understand the dilated convolution of our network, we first explain the standard 2D convolution layer. Supposed represents a position on the output , represents the spatial coordinate position. denotes the weight, denotes the kernel function and denotes the bias, so the formula of 2D convolution on the input feature map in stride of 1 is as follows:

where represents the kernel-wise nonlinear transformation of Rectified Linear Unit (ReLU). is a convolve operator. The dilated convolution adds a dilated rate to each kernel :

where the is produced by inserting d-1 zeros between two consecutive values of each kernel along each spatial dimension [20]. Standard 2D convolution is a special case of dilated convolution with d = 1. The receptive field of the dilated convolution is a square of exponentially increasing size as shown in Fig. 2. Our Fire dilated modules and D-Fire dilated modules adopt dilation rates d = 2, which allow the Fire dilated modules and D-Fire dilated modules to modify convolution-kernel’s receptive fields rather than stacked down-sampling operators.

FIGURE 2.

Illustration of dilated convolution. Green block indicates the data point of the filter and the red square indicates the receptive field captured by the filter. (a) indicates the 2D image, is generated from through a 1-dilated convolution (also called standard convolution); each element of possess of a receptive field. (b) is generated from through a 2-dilated convolution (dilation rate is 2); each element in possess of a receptive field.

B. Two Advanced Structures of the Segmentation Network

The ED style network [22]–[25] developed in the recent study is used to segment images and obtain good results by the end-to-end fashion in computer vision. Different from above, our model has integrated the ED and Conv-LSTM in an innovative way into one unified end-to-end network architecture. As shown in Fig. 3, the Cardiac-DeepIED network is composed of an ED structure and a Conv-LSTM structure [26], it aims at extracting more effective information at the pixel level and capturing the temporal dependencies to guide cardiac segmentation. The details of the two advanced structures for the segmentation are as follows.

FIGURE 3.

The details of the Cardiac-DeepIED network. Blocks of the same color mean they have the same parameters.

1). The ED Structures

In our model, the ED network takes Fire dilated modules as an encoder and D-Fire dilated modules as a decoder, and three skip connections are added between the encoder and the decoder. Our ED network does not only keep fine-grained information with few parameters and suitable receptive fields but also sufficiently recover spatial dimension and images details information.

The difference between our ED network and existing ED method is that ED network employs Fire dilated modules shown in Fig. 4 (a) as an encoder for efficient images representation and employs D-Fire dilated modules shown in Fig. 4 (b) as a decoder to obtain the full spatial resolution. As illustrated in Fig. 4, the Fire dilated modules and D-Fire dilated modules build a block respectively which is used to encapsulate three dilated convolution layers. The two kinds dilated modules play the role of squeeze network, which capture more image information and enlarge receptive fields more efficiently on limited parameters. Therefore, the ED architecture which consists of Fire dilated modules and D-Fire dilated modules not only definitely solves the high variability of complex cardiac structures from MR images, but also capture more image information and enlarge receptive fields more efficiently without adding extra parameters and computation.

FIGURE 4.

(a) Fire dilated module and (b) D-Fire dilated module. Both Fire dilated module and D-Fire dilated module are composed of dilated convolution (dilation rate is 2). In colored rectangles, represents the kernel size, C/4 and C/2 represent channels of feature maps produced by colored rectangles. Each layer of dilated convolution is followed by a ReLU layer. The input and output of the two modules are the same, and each layer in the modules is the dilated convolution used to increase the receptive field.

Another problem, the resolution of the feature maps and spatial information after three max-pooling layers, need to be solved before our model training, so we add two corresponding up-sampling layer to restore images resolution. However, the lost spatial information is difficult retrieved. To compensate for the loss of resolution caused by the pooling layer, the Cardiac-DeepIED introduces skip connections between its encoder and decoder. Skipping connections helps the decoder path to recover fine-grained information from the encoder path. The higher resolution information is passed by means of standard skip connections between the encoder and the decoder. As a result, it has the advantages of being capable of obtaining semantic task-aware representation and preserving fine-grained information.

In our network, we divided cardiac image input into different semantically interpretable categories. Semantic interpretability, the classification category, makes sense in real images, and semantic segmentation allows us to understand the cardiac image in more detail. In this architecture, the encoder and Conv-LSTM (Introduced in the next section) can effectively capture local semantic information (pixel-based information) in the image, and the pooling layers in the encoder reduces the size of the feature map, making it a low-dimensional image representation, but with rich semantic information. The decoder receives the image representation, recovers the spatial dimension by up-sampling, and finally, the decoder generates a feature map representing the original image label. These feature maps are then input into a standard convolutional layer with a kernel size of 1 (equivalent to the fully connected layer). The convolution operator outputs a score map, which gives the probability of each class at each pixel, as below.

where represents the spatial coordinate position, is the output features of network. denotes the kernel function and denotes the number of channels. is the set of pixel positions. The final score at score map is just summed over all channels of feature maps, as below.

where is the prediction probability and is the output of network, and . As shown in Equation 4 and Equation 5, the final predicted label is the category with highest probability.

2). The CONV-LSTM Structure

As a special RNN structure, LSTM was proposed by Hochreiter & Schmidhuber [27] and it had been proved to be stable and powerful in modeling long-term dependencies in previous works [28]–[30]. Subsequently, LSTM was applied to language modeling [31], image caption [32], [33], scene labeling [34] and so on. In the field of the medical image, LSTM was used to evaluate cardiac function [24] via capturing the temporal dependencies between MR image sequences without considering spatial correlations. Recently Shi et al. proposed Conv-LSTM that integrated convolution into the LSTM by substituting convolutional filters for the weights for precipitation Nowcasting [26]. Later Conv-LSTM was applied to anomaly detection in the video [35] and autonomous driving [36]. The advantage of the convoluted LSTM is that it does not only propagate the characteristics of space but also captures temporal dependences between consecutive frames.

Therefore, Conv-LSTM is more suitable for working with cardiac images than LSTM. The Conv-LSTM structure is explicitly a special LSTM unit and models the potential spatiotemporal correlations between consecutive frame images, which efficiently improve the segmentation accuracy of cardiac dynamic structures with information from consecutive frame images. Therefore, the Conv-LSTM structure is capable of memorizing long-period spatiotemporal correlations from consecutive frame images of cardiac cycles.

The major difference between the proposed method and existing works is that our Cardiac-DeepIED network integrates Conv-LSTM units into ED network for cardiac image analysis. Just because of this, the input and output of all layers are 4-D tensors: the first dimension corresponds to cardiac MR frames, which representing the time step, the two dimensions in the middle represent the length and width of the image, and the fourth indexes different feature channels. When 4D data is fed to Conv-LSTM, it can directly utilize 4D tensors and maintain spatial structure without losing spatial information. In this paper, we use the wrapper Time-Distributed to process 4D data instead of using a 3D network. We adopt the wrapper Time-Distributed to wrap each layer in the network. In fact, we can realize the transition from 2D to 3D through this layer. With the Time-Distributed wrapper, the input of the network is still 4D tensors, while the difference is the size of the first represents the time step. our network parameters and calculations are almost half of 3D networks, which greatly reduces the time cost.

Inner structure of the Conv-LSTM is shown in Fig. 5, given the input, cell outputs and hidden states are represented by , and respectively. Input gates, forget gates and cell outputs gates are , and respectively. The formulation of the Conv-LSTM is as follows:

where * indexes the convolution operator and indexes the Hadamard product. is activation function, index bias vectors. The weighted connections between states are represented by weight matrices , for example, denotes the matrices of weights from forget to the input, , and are diagonal weight matrices for peephole connections.

FIGURE 5.

LSTM and Conv-LSTM network visualization. In short, the principle of Conv-LSTM is that the vector-to-vector multiplication in the standard LSTM replaced by the previously described spatiotemporal convolution. (a) LSTM unit. (b) Inner structure of Conv-LSTM.

The output state controls the amount of information propagated from previous time steps, while the information received by the next time step and layer constitutes the so-called hidden state. The peephole connection not only allows the LSTM unit to access, but also propagates information recorded from the cell state of the previous time step. Just as the convolution filter input to the hidden connection determines the resolution of the feature map created from the input, the convolution filter size hidden to the hidden connection determines the aggregated information received by the Conv-LSTM unit from the previous time step. The state transition between the time steps of the Conv-LSTM unit can then be interpreted as a movement between frames. Larger transition kernels tend to capture faster motions, while smaller transition kernels capture slower motions.

III. Network Implementation

A. Datasets

We employ 2900 CBV images from 145 subjects to evaluate the segmentation performance of our model. These MR images are 2D short-axis cine. The age of subjects ranges from 16 to 97 years old. Each subject includes 20 frames across a cardiac circle. In each frame, the middle slice is selected according to the standard AHA prescriptions [37] for validation of our proposed Cardiac-DeepIED network. The pixel pitch of the MR images within a small range (0.6836-2.0833 mm/pixel) with the mode of 1.5625 mm/pixel. Two landmarks that junctions of the right ventricular wall with the left ventricular are manually labelled for each cardiac image to provide a reference for cardiac ROI cropping. The cropped images are resized to the dimension of . After the preprocessing, the ground truth is labeled manually by two experienced researchers and two experienced cardiac radiologists check the manually obtained bi-ventricle boundaries from all the cardiac MR images.

B. Configurations

We implement all of the codes using Python on a windows10 desktop with Intel Core i5-7400 CPU. The graphics card is an NVIDIA GeForce GTX 1060. The deep learning libraries are implemented with Keras (Tensorflow). We added dropout layer [38] before the last layer to prevent the network from over-fitting. We choose Adaptive Moment Estimation (Adam) as our optimizer and the softmax function is employed to fine-tuning the model to classify each pixel (ventricle area or not), and leave-one-out cross validation is applied to performance evaluation. Due to the experimental limitation on hardware, we set up the mini-batch size as 1 and the number of training epoch as 40.

C. Performance Evaluation Criteria

We evaluate the performance of cardiac segmentation by employing DM, Jaccard coefficient, Accuracy and positive predictive value (PPV). DM measures the overlap between ground truth area and predicted area. The DM value is in the range of 0~1, and a higher DM denotes a better match between manual contour and predicted contour. The formula for DM is as follows:

Similar to the DM, the Jaccard coefficient also is in the range of 0~1, and Jaccard coefficient denotes correspondence with the ground truth. The formula for Jaccard coefficient is as follows:

where and denote the region of manual and automatic contour, respectively.

In addition, we also use accuracy and PPV evaluate the results of pixel classification. They are defined by:

where , , , and represent true positive, true negative, false positive, false positive and false negative of class respectively.

IV. Results

A. The Performance on 145 Subjects

The DM values of our proposed Cardiac-DeepIED method are measured by comparing our segmentation results with the ground truth. Fig. 6 illustrates the segmentation results that the average DM of each subject (20 images). Although the challenges in segmenting CBV from MR images, our method achieves a high average DM of 0.890 (MYO), 0.960 (LV) and 0.903 (RV). Furthermore, the average DM of each subject exceeds 0.9 for the LV and RV, the average DM of each subject is around 0.890 for the MYO. The above results show that the segmentation results of each patient are at a relatively high level, and the segmentation results between patients do not have large fluctuation amplitudes. This also indicates that our method has powerful generalization ability in segmenting images of patients with large differences.

FIGURE 6.

Dice value of 145 subjects based on our proposed network.

For the fact that myocardial segmentation results are lower than the left ventricle and right ventricle, we consider the following factors that may limit the performance of the proposed method. As shown in the following Fig. 7 (a), the myocardial region is very thin, and the inner and outer membranes of the myocardium are almost indistinguishable, which makes the segmentation difficult. As shown in Fig. 7 (b), we can see that some region of the myocardium is missing, which is obviously difficult to segment. As shown in Fig. 7 (c), part of the myocardium is interfered by adjacent tissues, causing the myocardial area to be truncated. As shown in Fig. 7 (d), the myocardial region is blurred and it is difficult to define the myocardial boundary. Fig. 7 (e) shows some of the images in patients 38-42. From the images and the corresponding binary image of the images, we can see that the low intensity contrast between myocardium and adjacent tissues, resulting in little border information. These reasons may cause the MYO segmentation to present consistently lower values for MYO segmentation.

FIGURE 7.

Some cases in the dataset that may affect segmentation results. In the case of (a, d, e), the red dotted rectangle indicates that the myocardial region provides less boundary information and may lead to poor segmentation performance. In the case of (b, c), the red dotted rectangle indicates that the myocardial boundary region is incomplete, resulting in difficulty in segmentation. (a) Thin. (b) Missing. (c) Interfence. (d) Blurry. (e) Dark and low intensity contrast.

B. The Advantages of the Fire Dilated Modules, D-Fire Dilated Modules and Skip Connections

By passing the encoder’s feature maps to the decoder, skip connections help the decoder to have more image detail information, thus recovering a better image. At the same time, the skip connection can solve the problem of gradient disappearance in the case of deep network layers and speed up the training process. If there is no skip connection, the decoder cannot recover many important spatial information lost when restoring the spatial resolution, which will also have a certain impact on the segmentation result. It can be seen from the results shown in the first line of Fig. 8 that the Cardiac-DeepIED without the skip connection, the decoder cannot recover important image details and affect the final segmentation result. In fact, the segmentation accuracy of the cardiac bi-ventricle is 0.961 (left ventricle), 0.916 (myocardium) and 0.945 (right ventricle), respectively. This is far worse than the result of the Cardiac-DeepIED. At the same time, this also proves the importance of skip connections.

FIGURE 8.

Illustration the effectiveness of the fire module and D-Fire dilated module. The first row is the segmentation result of the Cardiac-DeepIED without skip connections. The second row is the segmentation result of replacing fire modules and fire modules with standard convolution layers. The third row is the segmentation result of the proposed method, and the fourth row is ground truth. The fifth row is the cardiac MR image. The last row indicates the corresponding physiological structure of each color region. MYO: Myocardium; LVC: Left ventricular cavity.

To verify the importance of the Fire dilated modules and D-Fire dilated modules, we replaced it with standard convolution layers and the segmentation results are displayed in Table 1. Fig. 8 shows some examples of CBV segmentation results, we obviously observe that our proposed method can effectively make the prediction more consistent. As we have seen, the second row results of the Fig. 8 suffer from background interference, which mainly due to the similar intensity distribution between adjacent tissues and faint cardiac images provide little boundary information. Our Fire dilated modules and D-Fire dilated modules handle the problem effectively. The prediction result is refined by the Fire dilated modules and D-Fire dilated modules shown in the third row of the Fig. 8.

TABLE 1. Comparison Between Our Method and Several Current Segmentation Methods (Mean Value ± Standard Deviation)).

Methods	DM			Jaccard coefficient
	LV	MYO	RV	LV	MYO	RV
Conv-Deconv	0.934 ± 0.003	0.846 ± 0.011	0.881 ± 0.009	0.884 ± 0.064	0.733 ± 0.088	0.778 ± 0.109
Seg-Net	0.937 ± 0.009	0.844 ± 0.013	0.878 ± 0.022	0.885 ± 0.061	0.728 ± 0.088	0.779 ± 0.111
FCN	0.950 ± 0.004	0.873 ± 0.005	0.892 ± 0.025	0.907 ± 0.052	0.778 ± 0.078	0.800 ± 0.130
U-net	0.953 ± 0.016	0.887 ± 0.016	0.897 ± 0.026	0.919 ± 0.049	0.800 ± 0.072	0.827 ± 0.095
Cardiac-DeepED*	0.951 ± 0.008	0.870 ± 0.020	0.899 ± 0.018	0.912 ± 0.048	0.775 ± 0.080	0.818 ± 0.099
Our method	0.960 ± 0.008	0.890 ± 0.018	0.903 ± 0.026	0.923 ± 0.040	0.801 ± 0.068	0.828 ± 0.097
Methods	Accurary			PPV
	LV	MYO	RV	LV	MYO	RV
Conv-Deconv	0.986 ± 0.007	0.965 ± 0.014	0.976 ± 0.010	0.942 ± 0.055	0.840 ± 0.094	0.883 ± 0.111
Seg-Net	0.985 ± 0.007	0.964 ± 0.014	0.976 ± 0.011	0.930 ± 0.065	0.845 ± 0.090	0.883 ± 0.106
FCN	0.989 ± 0.007	0.972 ± 0.011	0.979 ± 0.014	0.951 ± 0.049	0.878 ± 0.073	0.896 ± 0.112
U-net	0.990 ± 0.006	0.975 ± 0.011	0.982 ± 0.010	0.958 ± 0.048	0.887 ± 0.072	0.909 ± 0.097
Cardiac-DeepED*	0.989 ± 0.006	0.972 ± 0.011	0.981 ± 0.010	0.952 ± 0.051	0.871 ± 0.076	0.907 ± 0.096
Our method	0.991 ± 0.005	0.976 ± 0.009	0.982 ± 0.011	0.960 ± 0.040	0.891 ± 0.066	0.913 ± 0.095

Cardiac-DeepED^* represents the our method that replaced the Fire dilated modules and D-Fire dilated modules with standard convolution layers.

C. Quantification Analysis

In order to better demonstrate the effectiveness of our method, we quantify the area of one of the clinical indicators. In Fig. 9 (a), the blue points on the red line indicate that the physiological structure area predicted by our method completely overlaps with the corresponding physiological structure area manually delineation. From the Fig. 9 (a), we can see that the area predicted by the automatic method is very close to the area manually delineation, which demonstrates great correlation between predicted result by automated method and manual delineation. Fig. 9 (b) reports consistency between automated area and the manual delineation area. From the Fig. 9 (b), we can see that most of the points are within the standard line and the mean line is close to 0, which demonstrates the predicted areas by automatic method are highly consist with manual delineation areas.

FIGURE 9.

The comparison between predicted areas by automated method and manual delineation areas. (a) Linear regression analysis. (b) Bland-Altman analysis. The solid line in the middle is the mean line of the difference between the aforementioned two measurement methods, and the upper and lower dashed lines represent ±1.96 standard deviation (SD) of the difference. LVC: Left ventricular cavity; MYO: Myocardium; RV: Right ventricle.

D. Performance Comparison Between our Method and Other Methods

To better understand the segmentation results of the Cardiac-DeepIED network, the predicted maps and results of other methods including Conv-Deconv [39], Seg-Net [22] FCN [13] and U-net [12] are illustrated in Fig. 10. It can be seen that our method enables more accurate and robust segmentation of CBV images. This great robustness of our method due to the seamless combination of ED network and Conv-LSTM unit in a unified end-to-end framework. Therefore, the effectiveness and advantages of our method is beneficial to clinical cardiovascular disease diagnosis.

FIGURE 10.

Visualization of segmentation results.

To evaluate the effectiveness of our method, we compare it with other methods including Conv-Deconv [39], Seg-Net [22], FCN [13] and U-net [12]. From Table 1, we can see that the segmentation performance of all of these compared methods are lower than ours, which demonstrates that our improvements are resulted from adopting the proposed Cardiac-DeepIED method to represent cardiac images at pixel level. In summary, we can conclude that our method is generally outperforms the previous method with higher segmentation capability for segmenting the CBV. We can clearly see that the proposed model is a more effective method to segment CBV compared to existing methods. Therefore, it makes us more convinced that our method is capable of being a useful tool for the diagnosis and treatment of cardiovascular diseases in clinical practice.

V. Conclusion

In previous works, some effective methods of the CBV from cardiac images have proposed. However, these methods have one or more limitations, such strong prior, low robust and accuracy and so on. Because of the limitations, they are no more applicable in real clinical applications. Therefore, we proposed an automatic segmentation network based on deep learning to segment CBV at the pixel-level. Our method combines the advantages of the powerful Fire dilated modules, D-Fire dilated modules with ED for the semantic fine-grained representation, leverages the specialty of the Conv-LSTM structure for modeling spatiotemporal correlations between consecutive frame images. After leave-one-out cross-validation on the 145 subjects, our method achieves an accurate segmentation. Experimental results show that the proposed method performed significantly well in segmenting the CBV. This automatic method of the CBV segmentation has paved a great way for other medical image segmentation, such as spine image segmentation, brain image segmentation and so on.

In our study, we only pay attention to cardiac bi-ventricle segmentation in single magnetic resonance modality. Multi-modalities cardiac magnetic resonance of the cardiac segmentation will be considered for future work. Besides, our method can’t completely get rid of the radiologist, it can only help them to reduce the heavy workload to a certain extent. In further research, we will consider automatically generating a diagnostic report on the multi-modal cardiac magnetic resonance image, which can speed up the initiation of many specific treatments and help save time related to cardiology.

Funding Statement

This work was supported in part by the National Science Foundation of China under Grant 61673020 and the Anhui Provincial Natural Science Foundation under Grant 1708085QF143.