Automatic Detection of Dilated Cardiomyopathy in Cardiac Ultrasound Videos

Abstract

In this paper we address the problem of automatic detection of dilated cardiomyopathy from cardiac ultrasound videos. Specifically, we present a new method of robustly locating the left ventricle by using the key idea that the region closest to the apex in a 4-chamber view is the left ventricular region. For this, we locate a region of interest containing the heart in an echocardiogram image using the bounding lines of the viewing sector to locate the apex of the heart. We then select low intensity regions as candidates, and find the low intensity region closest to the apex as the left ventricle. Finally, we refine the boundary by averaging the detection across the heart cycle using the successive frames of the echocardiographic video sequence. By extracting eigenvalues of the shape to represent the spread of the left ventricle in both length and width and augmenting it with pixel area, we form a small set of robust features to discriminate between normal and dilated left ventricles using a support vector machine classifier. Testing of the method of a collection of 654 patient cases from a dataset used to train echocardiographers has revealed the promise of this automated approach to detecting dilated cardiomyopathy in echocardiography video sequences.

Introduction

Dilated cardiomyopathy is a heart disease in which the left ventricle of the heart becomes very large and loses the ability to pump blood to the rest of the body [1]. This condition can lead to heart failure and death. Cardiomyopathy is diagnosed through an ultrasound recording. In this recording, an ultrasonic probe is moved over the heart region. Sound passes through the blood tissues and gets reflected from different parts of the body. These reflections are recorded as signal and converted to an image. The ultrasound device images the heart from different angles and depicts the chambers of the heart in views such as the apical 4-chamber views, 2 chamber views, etc. A typical cardiac ultrasound image in apical four-chamber view appears as shown in Figure 1b.

Figure 1.

Illustration of an ultrasound image depicting the left ventricle. (a) normal left ventricle, (b) dilated left vetricle.

In Figure 1b, the 4 chambers of the heart are seen, with the largest region being left ventricle. The shape and function of the left ventricle are important in characterizing the heart. Damage to the ventricle’s shape and structure affects the function of the heart and is often seen in several diseases such as aneurysms, cardiomyopathies and infarctions. The left ventricle is often enlarged and oddly-shaped in case of dilated cardiomyopathy [1]. This can be seen by comparing the image of Figure 1b with a normal left ventricle shown in Figure 1a which has a more normal bullet-like shape. The enlargement can be noted not only visually in Figure 1b, but also by mapping the pixels back to cm measurements using the calibration scale in the echocardiogram image. It can be seen that the left ventricle length (from apex to mitral valve) is over 90mm which usually indicates dilated cardiomyopathy in both males and females.

The goal of our work was to develop ways to automatically differentiate between normal and dilated left ventricles in echocardiography images. Such methods could aid in the development of computer-based diagnostic tools to aid clinicians in their decision making.

Automatic detection of dilated cardiomyopathy from cardiac ultrasound videos, however, is a difficult problem. In 4-chamber views, although the left ventricle is more clearly visible, the exact boundary of the left ventricle may be difficult to delineate which alters shape measurements. Using simple thresholding techniques that look for low intensity regions as potential candidate regions for left ventricles may not be sufficient as the low intensities near the apex may cause the left ventricle region to be merged with the background. Although the practice guidelines indicate the single measurement of left ventricle length for diagnosing dilated cardiomyopathy [1], errors in automated left ventricle boundary detection may require more measurements be used to provide robustness.

In this paper, we present a new method of robustly locating the left ventricle by using the key idea that the region closest to the apex in a 4-chamber view is the left ventricular region. For this, we (a) locate a region of interest (ROI) containing the heart in an echocardiogram image using the bounding lines of the viewing sector to locate the apex of the heart, (b) select low intensity regions as candidates, and (c) find the low intensity region closest to the apex as the left ventricle, and finally (d) refine the boundary by averaging the detection across the heart cycle using the successive frames of the echocardiographic video sequence. We then extract the eigenvalues of the shape to represent the spread of the left ventricle in both length and width and augment it with pixel area to form a small set of robust features. A set of 4-chamber view echocardiogram study videos are used for training a support vector machine classifier [3] using disease labels obtained from their corresponding reports. The machine learns the separation between normal and abnormal classes based on the provided features and their labels. New 4-chamber echocardiogram videos are then processed similarly to isolate shape feature vectors, and then classified into dilated left ventricle class or normal class using the learned support vector machine. The method has been tested on a collection of 654 patient cases from a dataset used to train echocardiographers.

Related work

Automatic detection of diseases from echocardiography videos, however, has not been widely addressed. Most of the attention has been paid to valvular diseases from Doppler imaging [13], or measuring hypokinesia (reduced heart motion) and wall thickness in echocardiography videos [10]. Recently, work has been reported on the detection of differences between normal and abnormal left ventricular shapes in echocardiography videos using a modeling approach[12]. Our approach focuses on the detection of dilated cardiomyopathy condition, which to our knowledge has not been investigated earlier by other automated methods.

In medical imaging community, the left ventricular (LV) shape itself has been well-studied primarily for the purpose of segmenting the left ventricle in echocardiography images. A variety of techniques including active shape and appearance models [3,4,5], snakes and active contours [6,8], parametric shape descriptors of endocardial contours[8], deformable models and templates [6], and level set techniques are available. Model-based approaches such as active shape models are difficult to learn from a class of shapes as they need manual marker identification as well as prior registration of shapes during model training. Our experimentation also revealed that many region-based approaches over or under-segment the left ventricle, particularly, in diseased cases, resulting in inaccurate boundaries for shape characterization. We also experimented with an active shape model approach to localize LV as described in [10] but found it could locate the left ventricle accurately in only 35% of the cases of 4-chamber views. As a result, we implemented a new bottom-up approach to LV detection. Our approach consists of 4 major processing stages, namely, (a) ROI identification, (b) LV candidate region generation, (c) LV region selection using apex, (d) LV region refinement using spatio-temporal information. Each of these processing steps are explained below.

Identifying the region of interest and apex

The ultrasound scan sector in the image is usually bounded by dominant lines which can be highlighted using an edge detector, such as the Canny edge detector [14]. Figure 2b shows the edge image generated for the original image of Figure 2a. The popular way to detect strong lines is through the Hough transform [15] which detects the pixels that fall on a line in polar coordinate system by converting lines to points in the Hough space of (r,d) where r is the distance of a point (x,y) from the origin and d is the angle of the vector from origin to the point (x,y).

Figure 2.

(a) Original image. (b) Edge image showing the potential bounding lines of the sector or region of interest. Image source: 123sonography.com

By recording the lines passing through all edge pixels in various orientations, we get the Hough image as shown in Figure 3b. Here the dominant edges seen in the edge image are noted as bright spots in the Hough transform corresponding to the number of pixels that voted for the line, with longer lines getting more votes than short line segments. Of these, the bright spots that are on a horizontal line indicate lines of the same radius and could be potential bounding lines of the sector. Further, if the angle between them as seen by the horizontal separation between the bright spots is within a reasonable angle for a viewing sector (80 to 120 degrees), then they are very likely to be the bounding lines of the sector.

Figure 3.

Illustration of Hough transform. (a) Edge image. (b) Hough transform rendered as an image. The horizontal axis in (b) is the angle, and the vertical axis is the radius in the polar coordinate system.

Our method exploits this observation to analyze the bright spots in the Hough transform image and project them back as bounding lines in the edge image. Verification with the pixels underneath further establishes the bounding lines of the sector. Once the bounding lines are found, the point of intersection can be easily located. Figure 3a shows the point of intersection found in the edge image by a green dot. This is our best estimate of the apex of the heart as well based on the visible content in the sector.

Identifying candidate left ventricle regions

Since the left ventricle in the 4-chamber views is usually a darker region, we analyze the histogram of the image and separate it into intensity levels to pick relatively low intensity regions. Specifically, we use the multi-level Otsu thresholding method [9] to divide the histogram into 5 intensity levels capturing the 5 ranges of intensities typically seen in echocardiogram images. Of these the second lowest intensity level is used to threshold the original image into two classes. Choosing the second lowest ensures that we also capture cases where due to noise in imaging and phase in the heart cycle, the left ventricular region appears brighter than usual. The region within the bounding lines is then retained in the thresholded image to yield the image shown in Figure 4b. As can be seen, the potential merging of left ventricular region with the background is avoided due to the prior detection of region of interest.

Figure 4:

Illustration of candidate region generation within the bounding sector. (a) Original image. (b) Candidate low intensity regions within the region of interest.

Identifying the left ventricle

We then use a connected component grouping algorithm to collect all bright pixels in the thresholded image to form candidate regions. Using the observation that the left ventricle is the closest chamber to the apex in a 4-chamber view image on the right, we obtain the distance of the centroid of each region to the point of intersection previously identified during the region of interest localization, and retain the closest rightmost region as our choice for left ventricle.

Depending on the time in the heart cycle where the echocardiogram image is taken, the left ventricular region may be merged with the left atrium (when the mitral valve is open). This can cause the left ventricular region size to be overestimated. In the next step, we integrate information across successive frames in the video to more precisely localize the boundaries of the left ventricle.

Integrating time-varying information

We process each successive frame of the echocardiography video sequence and extract the left ventricle. The left ventricular shape is best segmented in the end-diastoli position just after the closure of the mitral valve. By tracking the size of the delineated left ventricle through the heart cycle, we pick the end-diastoli frame as the one with the smallest size region indicated for the left ventricle. When multiple heart cycles are present in the echocardiography sequence, we average the detected size of the left ventricle across all such end-diastoli frames. Finally, we remove small holes within the LV region to form smoothly filled left ventricle region as shown in Figure 5. In this figure, Figure 5a shows a left ventricle detected among the regions of Figure 4b. Figure 5b shows the result of averaging the detection in the end-diastoli frames over the heart cycles found in the echocardiogram video sequence.

Figure 5.

Illustration of LV region detection. (a) Raw region from a single image. (b) average region from end-diastoli frames within the echocardiography video sequence.

Extracting shape features

Once the LV region is identified, we extract shape features by using the eigenvalues of the 2D shape. Although the practice guidelines for dilated cardiomyopathy recommend measuring the length of the left ventricle from the apex to the mitral value, we adopted the eigenvalues over the exact extremal point-based length measurements in order to provide robustness to boundary localization errors in the left ventricular region. This can also be seen in Figure 5b where the apex region is included in the left ventricle region identification and would alter shape measurements based on the extremal points. Using the ratio of the two largest eigenvalues of the shape, we get a more robust estimation of the spread of the left ventricle in both length and width. Finally, to avoid overestimation of the area in the case of nonconvex shapes of the left ventricle (which can happen in other diseases such as aneurysms), we add a third shape feature based on the pixel area of the left ventricle.

Extracting scale

The above measurements of shape features must be normalized for zoom effects found in the echocardiography images (when echocardiographers zoom into the regions of interest). The calibration markers found on the region of interest can help identify the scale. The scale difference can be seen in Figures 1a and 1b for the echocardiogram images of two patients from the respective calibration markers shown. In our approach, we detect the number unit markings on the calibration scale using the method described in [12] that is based on an optical character recognition (OCR) algorithm called Tesseract [16]. Using the recognized units, the distance between markers is then converted to pixel coordinates to mm ranges as described in [12]. The resulting shape features are thus normalized and presented for discrimination based on actual physical dimensions rather than the sizes in pixels.

Discriminating between normal and dilated left ventricles

Given a set of labeled training videos depicting normal and dilated left ventricles, we locate the left ventricle as described above and extract the normalized shape feature vectors. We then find the separation between normal and dilated left ventricle using the Support Vector Machine (SVM) framework. An SVM is a classifier that, when used with two classes, tries to find a boundary in the data space such that the two classes are separated by the maximum possible margin. Given a set of training data which is composed of a set of vectors and their labels, SVM tries to find the parameters of this maximum margin boundary. This boundary can be thought of as a high dimensional line, which is characterized by a set of weights (α). Since the objective of this work is to discriminate between normal and dilated LV, we build a single SVM model in which the positive examples (+1 labels) are dilated cases and normal cases are the negative examples (−1 labels).

Given a new test echocardiogram video, we proceed as before to extract the left ventricle region and normalized shape features. The trained SVM model is then used to predict the label for the test case as normal or dilated left ventricle indicating dilated cardiomyopathy.

Results

We evaluated the validity of this approach on an echocardiogram dataset made available from 123sonography.com. This is an educational site for training echocardiographers in the interpretation of echocardiograms. Each video sequence is labeled with the observed condition in the video by their clinical experts. The dataset has over 2000 videos. After automatically analyzing the reports for textual phrases indicating normal or diseased left ventricle, we isolated cases of normal and dilated left ventricles. Figure 6 shows an extract of a report from which the deduction of dilated left ventricles was made and added as a positive example label for the corresponding echocardiography sequence. Raw textual analysis was followed up by manual verification before retaining the labels as ground truth labels. From the 2000 cases, we found about 254 cases of dilated cardiomyopathy and 400 normal cases. We then analyzed all the selected video sequences to locate the left ventricle and selected 124 cases as training data which had accurate detection of the left ventricle and used their shape feature vectors for training data. Of these there were 52 normals and 72 dilated cardiomyopathy cases. The trained support vector machine was then used to classify the rest of the data.

Figure 6.

Extract of an echocardiogram report with the first column indicating the number of the corresponding echocardiography study.

Since all data tested already has a ground truth disease label of dilated cardiomyopathy or normal, the classification accuracy was evaluated as follows. Let F= (f1,f2,…fM) denote the dilated cardiomyopathy videos identified by the classifier. Let G= (g1,g2,…gN) denote the dilated cardiomyopathy cases identified in the ground for the same video. Then the classification accuracy per class is defined as $\frac{\left|F\right|}{\left|G\right|}$. The overall accuracy is averaged over the test samples of the two classes.

The results of the evaluation are shown in Table 1. The average classification accuracy is currently at 77.8% across all the test sequences. The overall classification accuracy is affected by (a) accuracy of region of interest (ROI) localization, (b) Left ventricle detection, (c) left ventricle region size and boundary estimation besides SVM classifier accuracy reported in Table 1. The relative contribution of each of these was measured by visually inspecting the images produced in each step of the process for each of the test video sequences. The results are summarized in Table 2. As can be seen, many of these steps have good accuracy but due to the cumulative errors from their application in sequence, the overall accuracy reduces to 77.8%. As the first approach to attempt this problem, we believe these results are promising and encourage further research in this area.

Table 1.

Illustration of normal versus dilated LV classification accuracy by SVM.

Training normals	Training dilated LV	Test normals	Test dilated LV	Total correct detections	Total false detections	Overall accuracy %
52	72	348	182	412	117	77.8

Table 2.

Illustration of performance of the image processing modules

ROI detection accuracy%	LV region selection accuracy %	LV size estimation accuracy %
92.5	89.3	87.2

Conclusions

In this paper, we have addressed, for the first time, the problem of automatically detecting dilated cardiomyopathy from cardiac ultrasound videos. A robust left ventricle detector was proposed and shape features extracted and fed to machine learning framework based on support vector machines to separate normal from dilated left ventricles. The performance of the classification shows the promise of the method towards developing a reliable decision support tool in future.