Multiple Sclerosis Lesions Segmentation in Magnetic Resonance Imaging using Ensemble Support Vector Machine (ESVM)

Abstract

Background:

Multiple Sclerosis (MS) syndrome is a type of Immune-Mediated disorder in the central nervous system (CNS) which destroys myelin sheaths, and results in plaque (lesion) formation in the brain. From the clinical point of view, investigating and monitoring information such as position, volume, number, and changes of these plaques are integral parts of the controlling process this disease over a period. Visualizing MS lesions in vivo with Magnetic Resonance Imaging (MRI) has a key role in observing the course of the disease.

Material and Methods:

In this analytical study, two different processing methods were present in this study in order to make an effort to detect and localize lesions in the patients’ FLAIR (Fluid-attenuated inversion recovery) images. Segmentation was performed using Ensemble Support Vector Machine (SVM) classification. The trained data was randomly divided into five equal sections, and each section was fed into the computer as an input to one of the SVM classifiers that led to five different SVM structures.

Results:

To evaluate results of segmentation, some criteria have been investigated such as Dice, Jaccard, sensitivity, specificity, PPV and accuracy. Both modes of ESVM, including first and second ones have similar results. Dice criterion was satisfied much better with specialist’s work and it is observed that Dice average has 0.57±.15 and 0.6±.12 values in the first and second approach, respectively.

Conclusion:

An acceptable overlap between those results reported by the neurologist and the ones obtained from the automatic segmentation algorithm was reached using an appropriate pre-processing in the proposed algorithm. Post-processing analysis further reduced false positives using morphological operations and also improved the evaluation criteria, including sensitivity and positive predictive value.

Introduction

Multiple sclerosis is a complex and chronic disorder of the central nervous system (CNS). Nearly, 2.3 million people were suffered globally from Multiple sclerosis in 2013, and there has been currently no cure for this disorder [ 1 , 2 ]. By damaging the myelin sheath, this progressive disease causes various changes in the structure of the central nervous system. Myelin sheath made up fat, raps around the nerve fibers or axons. This sheath helps the neuronal messages to travel faster via neuronal connections. The deterioration of this fatty sheath around axons would lead to an impaired nerve signal transmission, which in turn, causing behavioral, verbal, and cognitive disorders due to its location in the central nervous system [ 3 ]. For the most part, Magnetic Resonance Imaging (MRI) is the most widespread medical imaging technique. Owing to an almost minimally invasive procedure, a high contrast of soft tissue, and low side effects were compared to the other imaging techniques, such as Computer tomography (CT) and that MRI is one of the best imaging modalities in studying the brain tissue. Due to a lack of available treatment, in the clinical field, the focus is primarily on controlling the disease progression rate. Early detection of plaques and provision of clues for the location of the lesion, therefore, are paramount for the patient. MRI allows physicians to identify potential MS patients, monitor their status in various stages of treatment, and detect new lesions which may grow over time. Over the past a few decades, several protocols have been proposed and developed in order to reproduce more vivid and accurate brain tissue images. Moreover, these protocols have had great success in monitoring and treating patients for an extended period of time. Depending on the circumstances, one or several protocols may be used in the process of diagnosing MS. In addition, various MRI protocols may be utilized in order to detect lesions. In addition, research has shown that, in comparison with using a single protocol, the use of T1-weighted, T2-weighted, PD, and Flair imaging protocols can play a vital role in enhancing lesion detection.

Typically, plaques seen in Corpus Callosum, Sulcus, Gyrus (U-fibers), Temporal lobe, Brain Stem, Cerebellum and Spinal Cord, are associated with this disorder [ 4 ]. Lesions appear in high brightness (hyper-signal) in T2 and PD protocols and slow brightness (hypo-signal) in T1 protocol. Due to the suppression of the cerebrospinal fluid effects in the images, Flair Images can demonstrate hyper-intense lesions around the ventricles. Hence, the Flair protocol is the ideal candidate to detect MS plaques [ 5 ].

The manual analysis of brain tissue in MRI images is both challenging and time-consuming and also requires significant expertise. With the growth of computer usage and the advent of machine learning applications in medicine, MS lesions segmentation and localization in brain can be presented to physicians at shorter intervals and with acceptable speed and accuracy. Many fully-automatic and semi-automatic algorithms have been proposed as “Segmentation Techniques” over the past few years. Spatial information of images, brightness threshold determination, edge detection, area growth, and statistical data usage, including probability density function estimation, and lastly a widespread use of artificial neural networks have been employed in these algorithms [ 6 - 12 ].

A few strategies have been explored for the segmentation of brain tissue images using Ensemble of Classifiers. However, generally, these approaches are either not associated with the detection and localization of MS lesions or have simply the lack of precision and accuracy [ 13 , 14 ]. While using two different processing methods based on Ensemble classification, a different approach is pursued in this study which firstly increases the accuracy of lesions segmentation automatically, and secondly seeks a high level of agreement between the automatically obtained results and the physician's opinions.

A new approach is proposed in this paper for MS lesions segmentation using MRI images. Data acquisition and the overall structure of the image processing method are discussed in Section 2 and then results are presented in section 3, after introducing two different methods to detect and segment MS lesions. Finally, in Section 4, the conclusions obtained from this study are given.

Material and Methods

Data gathering

In this analytical study, in order to design and develop the segmentation algorithm, more than 80 actual data with dimensions of 181mm × 217mm × 181mm was collected using the “JHU-MRI DATA” database [ 15 ]. The data was collected as a part of the research study by the Johns Hopkins Multiple Sclerosis Center, using a 3-Tesla Philips MRI scanner. It consists of pathologic MRI images of MP-RAGE type (for T1-weighted), T2-weighted, PD-weighted, and FLAIR from patients associated with MS. For each subject, an approximate interval of one year was considered for data collection. The data was divided into two sections: a) testing data, and b) training data. Initial pre-processing of the features such as brain extraction, noise reduction, background removal, and slice registration were performed on each part of the data using MIPAV (version 7.4.0) software Figure 1.

Figure1.

Shows the preprocessed (a) MPRAGE, (b) T2-w, (c) FLAIR, and (d) PD-w images for axial view.

Manual Segmentation (Gold standard)

Two Neurologists, with 10 and 17 years of experience, manually segmented the T2-w images using MIPAV. These segmented images were then employed applied as a mask containing lesions and non-lesions regions (two binary label masks) to the pre-processed FLAIR images. Two raters, who had no information about the time when the data was collected, used MPRAGE and PD images to ensure consistency [ 16 ].

Performance Evaluation

In order to accurately evaluate the segmentation techniques, an exact comparison between the results obtained from the automatic segmentation (Seg) and the manual segmentation (Ref) was made. Figure 2 illustrates the performance evaluation of the automatic and the manual segmentation. Overlapping of both pixels is considered to be where MS plaques are present (TP as in True Positive). Situations, where pixels are only present in one or none of the segmentation methods, are considered to be without MS plaques such as False Positive (FP), False Negative (FN), and True Negative (TN) pixels. Based on these parameters, the evaluation of criteria such as Dice factor, Jaccard factor, Sensitivity, Specificity, and Positive Predictive Value (PPV) can be measured.

Figure2.

Comparison between the automatic segmentation (Seg) and the manual segmentation (Ref) TP, TN, FP, and FN represent true positives, true negatives, false positives, and false negatives pixels, respectively [ 17 ].

Dice factor

The Dice coefficient or similarity index [ 18 ], a value between 0 and 1, expresses the spatial overlap between two binary images, where 0 is non-matching and 1 is maximum matching. If the mask obtained by the raters is M_R and the Mask from the automatic segmentation is M_A, the Dice coefficient can be calculated from Eq. (1).

$\mathrm{Dice}\left(M_R,M_A\right)=2\frac{|M_R\cap M_A|}{\left|M_R\right|+\left|M_A\right|}=\frac{2\left|\mathrm{TP}\right|}{\left(\right|\mathrm{TP}|+|\mathrm{FN}|+|\mathrm{TP}|+|\mathrm{FP}\left|\right)}$ (1)

Jaccard factor

Jaccard factor [ 19 ] is another common criterion used to assess the level of similarity between the automatic segmentation and the manual segmentation. This factor is defined by Eq. (2).

$J\left(M_R,M_A\right)=\frac{|M_R\cap M_A|}{\left|M_R\right|\cup \left|M_A\right|}=\frac{\left|\mathrm{TP}\right|}{\left(\right|\mathrm{TP}|+|\mathrm{FN}|+|\mathrm{FP}\left|\right)}$ (2)

Sensitivity factor

Sensitivity or True Positive Rate indicates how many lesions are accurately detected (Eq. (3)).

$\mathrm{TPR}\left(M_R,M_A\right)=\frac{|M_R\cap M_A|}{\left|M_R\right|\cap \left|M_A\right|+\left|M_R\right|\cap \left|M_A^c\right|}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$ (3)

Specificity factor

The True Negative Rate (TNR) actually refers to the ability of the segmentation method to remove redundant areas or areas in which plaques are not present (Eq. (4)).

$\mathrm{TNR}=\frac{\mathrm{TN}}{\mathrm{TN}+\mathrm{FP}}$ (4)

The positive predictive value

The Positive Predictive Rate (PPV) is, in fact, the ratio of the correct statistical results and the diagnostic tests, or in other words, real positive results (Eq. (4)).

$\mathrm{PPV}\left(M_R,M_A\right)=\frac{|M_R\cap M_A|}{|M_R\cap M_A|+|M_R^c\cap M_A|}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$ (5)

Method

In this paper, a new approach to MS lesions segmentation was introduced when investigating the axial view of the FLAIR Images was associated with MS Patients. In this method, certain areas of the brain possess a high brightness (hyper-intense). The two methods used in this paper are shown in a block in Figure 3. After classifying the data, the segmentation algorithm is performed directly in approach A and after a secondary processing in approach B.

Figure3.

Pre-Processing, Feature extraction, Feature selection, Classification and Segmentation.

Pre-processing

In order to obtain more accurate results in the lesions segmentation and despite the fact that imaging was performed accurately and also the data is collected only for research, initial pre-processing must be done. This stage is believed to be highly effective in the lesions segmentation. Since the highest levels of intensity in FLAIR images of MS patients indicate the likelihood of a lesion, significant differences were observed in the comparison of the images maximum intensities. A default sample is presented by a 3D image of the brain in Figure 4(a), in which a cross-section of a patient’s brain is obtained. In Figure 4(b), using axial view imaging. In addition, digitizing the image reveals the numeric matrix of the Figure 4(c). The Cij coefficients essentially represent the numerical values of the quantified pixels and the variables i and j represent the number of pixels in the image. The intensity of the image brightness was divided by the total average brightness intensity in order to normalize the images in terms of intensity (Eq. (5) and Eq. (6)).

Figure4.

(a) 3D brain, (b) An axial slice of the brain, (c) Image matrix intensity.

1≤i≤181

1≤j≤217

$A_T=\frac{{\sum }_{\mathrm{i=1}}^{N_i}{\sum }_{\mathrm{j=1}}^{N_j}{C}_{\mathrm{ij}}}{N_i{N}_j}$ (6)

${\stackrel{-}{C}}_{\mathrm{ij}}=\frac{C_{\mathrm{ij}}}{A_T}$ (7)

Feature extraction

In the feature extraction step, in order to find the best possible classification, the most informative portions of data were extracted from the images after applying the preprocessing step. Generally speaking, the brightness intensity was considered as one of the most commonly used features in the segmentation of different regions. Furthermore, textural features are also the basis of the final support vector machine. A total of 11 features (Table 1) were extracted from FLAIR images, which are categorized into three groups of histogram-based features, gradient-based features [ 20 ], and a combination of both. This categorization is most useful when lesions need to be found in areas containing MS plaques.

Table1.

Feature vector.

Features Category	Features
Histogram-based features (1)	Mean Variance Skewness Kurtosis
Gradient-based features (2)	Mean Variance
A new combination of features (3)	InVa IPM VIPM

The histogram expresses a summary of the statistical information in the image. Obviously, the properties obtained from the normalized histogram can also be considered in the category of statistical properties [ 21 , 22 ]. At this stage, the extracted properties were obtained by considering a moving window with dimensions of 3 × 3.

Since gradients are often used in the segmentation to refine and enhance the edges [ 23 ], the absolute gradient is also considered as a separate property and its mean and variance were obtained based on a neighborhood of 5 × 5 for each pixel [ 24 ]. “InVa” (Eq. (9)) is obtained as the product of the brightness intensity with the histogram-based variance.

InVa=Intensity×Varience (8)

To calculate the mean value a 19 × 19, the window was taken into account. Then, combining the local histogram-based mean value with the intensity brightness of each pixel gives out IPM [ 25 ].

$\mathrm{IPM}=\frac{\mathrm{Intensity}}{\mathrm{Mean}(19\times 19)}$ (9)

VIPM is determined by the sum of IPMs and the obtained gradient-base variance (Eq. (10)).

VIPM=GrVarience+IPM (10)

Feature Selection

Selecting the subset of input variables to improve the accuracy and efficiency of the basic information classification and thus, removing weak and inefficient features, is the main purpose of this stage. There are several methods to select each feature. Feature selection helps to achieve the optimal target function for the classification and ultimately segmentation. Sequential Forward Selection (SFS) and the KNN algorithm [ 7 , 17 ] as classifiers in the wrapper algorithm were used in this method to select a feature [ 26 ]. The characteristic block of this method is shown in Figure 5.

Figure5.

The training section block diagram in the suggested SFS Ensemble algorithm.

Classification and Segmentation

During the classification process, images possessing similar features are classified into the same groups. Based on the material, the classification method will vary. The classifier selection step should be dealt with, after choosing an appropriate feature vector. At the end of this procedure, the pixels can be categorized into two different pixels that they are related to the MS plaque and the healthy brain tissue, respectively. Hence, the support vector machine classification, which is used in machine learning as a robust classifier, was considered as the main classifier in the proposed algorithm. SVM belongs to “supervised learning” methods which can be employed for linearly and non-linearly separable data. Radial Basis Functions (RBF) are considered as the Kernel function in the proposed SVM classifier structure. Training the algorithm and SVM structure were created using training data. Bagging (Bootstrap Aggregating) was implemented to increase the training capability of SVM as well as increasing the accuracy of the test data. Therefore, the training dataset was divided into several groups; each new dataset was used to train an SVM classifier, and the Ensemble SVM was achieved when these classifiers were combined.

Randomly, 5 different SVM classifier structures were obtained by dividing the training data into 5 parts and using them as inputs of the SVM classification, in this paper. The block diagram of this method is shown in Figure 6. Two different output states were considered from the result of these five classifiers. In the first approach, the test data was applied to these 5 classifiers, voting was initiated after classifying the data into two sets of MS plaques and healthy brain tissue, and finally, the results were obtained. The proposed scheme of the first approach is demonstrated in Figure 7.

Figure6.

The training section block diagram in the proposed Ensemble SVM algorithm.

Figure7.

The proposed block diagram of the first approach in the Ensemble SVM Algorithm.

In the second approach, the post-processing is applied to the SVM classifier output and then the voting procedure is initiated, and finally, the results were obtained, as shown in the block diagram of Figure 8. Moreover, morphological erosion operation was used after obtaining the SVM classifier outputs from post-processing. While this operation is utilized to thin objects down or shorten them in a binary image [ 27 ], an example of how the morphological erosion operation works on a binary image is shown in Figure 9.

Figure8.

The proposed block diagram of the second approach in the Ensemble SVM Algorithm.

Figure9.

a) A 3 × 3 neighborhood from a binary image. b) Erosion operation, Deleted pixels (Erosion) are displayed brighter [ 28 ].

In both methods of the proposed algorithm, the voting stage determines the outcome of the system. As shown in Figures 7 and 8, based on the highest repeated pattern, the voting block ultimately chooses one matrix out of the 5 input matrixes. The voting rules can be explained as the following: “if the binary-weighted value assigned to a pixel is 3 or more out of the 5 input matrixes in a specified location, that pixel wins the voting procedure, which ultimately, a binary matrix pixel of original image size may fill the proposed system output.”

Results

In this paper, a new approach was proposed for MS lesions segmentation methods using real MRI images. In both the proposed approaches of the Ensemble SVM algorithm, the feature selection module played a vital role in the performance of the system, as is shown in Figure 5. As stated in feature selection Section, out of eleven features, four of them were extracted, including: a) the pixels intensity of the FLAIR images, b) the histogram-based local mean value, c) Variance of the absolute gradient, and d) InVa obtained from SFS method which they were extracted for further analyses. These four features were used in every structure of the SVM classifiers. Considering the described in Section 2.4.4, since the system consists of size-preserving elements, the output of the binary images system will be the same size as the input images. A sample of the system input is illustrated in Figure 10, the MS lesions segmentation results for the first and the second approaches of Ensemble SVM are shown in Figures 11 and 12, respectively.

Figure10.

Shown is the pre-processed FLAIR image for a single view.

Figure11.

Segmentation results, a) The automatic segmentation binary image using the first mode of Ensemble SVM method b) The applied patient’s axial FLAIR image, manual and automatic segmentation using the first mode of Ensemble SVM method.

Figure12.

Segmentation results, a) The automatic segmentation binary image using the second mode of Ensemble SVM method b) The applied patient’s axial FLAIR image, manual and automatic segmentation using the second mode of Ensemble SVM method.

In Figures 11 and 12, white marks are as plaques which are correctly identified by the algorithm (TP). Moreover, green marks represent lesions considered to be healthy brain tissues by the algorithm (FN), and violet marks are in fact healthy brain tissues which mistakenly were detected as lesions and therefore it is considered to be a false positive (FP). In order to measure the performance of the proposed segmentation methods, axial FLAIR images of a real subject were used. Initially, according to the assessment criteria presented in Section 2.3, the results are tabulated in Tables 2 and 3.

Table2.

Evaluation criteria results for MS lesions segmentation using the first mode of Ensemble SVM method in comparison to Gold Standard.

Subjects	Dice	Jaccard	Sensitivity	Specificity	PPV	Accuracy
Subj01	0.48±0.19	0.33±0.15	0.83±0.16	0.97±0.02	0.39±0.20	97.36±2.18
Subj02	0.52±0.18	0.37±0.15	0.90±0.13	0.98±0.01	0.41±0.19	98.46±1.06
Subj03	0.38±0.19	0.25±0.14	0.81±0.15	0.95±0.05	0.30±0.20	94.99±4.00
Subj04	0.38±0.18	0.25±0.14	0.83±0.14	0.95±0.04	0.30±0.22	95.41±4.04
Subj05	0.75±0.13	0.61±0.15	0.93±0.06	0.99±0.00	0.65±0.17	98.99±0.37
Subj06	0.67±0.12	0.51±0.13	0.91±0.06	0.98±0.00	0.54±0.14	98.03±0.96
Subj07	0.69±0.13	0.55±0.15	0.94±0.04	0.98±0.00	0.57±0.16	98.36±0.63
Subj08	0.75±0.12	0.62±0.14	0.98±0.01	0.98±0.00	0.63±0.14	98.78±0.32

Table3.

Evaluation criteria results for MS lesions segmentation using the second mode of Ensemble SVM method in comparison to Gold Standard.

Subjects	Dice	Jaccard	Sensitivity	Specificity	PPV	Accuracy
Subj01	0.50±0.15	0.34±0.13	0.72±0.22	0.98±0.02	0.47±0.21	98.16±0.90
Subj02	0.57±0.14	0.42±0.13	0.81±0.20	0.99±0.00	0.52±0.20	99.02±0.61
Subj03	0.41±0.17	0.27±0.13	0.69±0.20	0.96±0.04	0.38±0.22	96.45±2.27
Subj04	0.41±0.14	0.27±0.10	0.71±0.19	0.97±0.03	0.37±0.22	96.97±2.92
Subj05	0.74±0.09	0.60±0.10	0.78±0.08	0.99±0.00	0.74±0.14	99.12±0.25
Subj06	0.69±0.09	0.54±0.11	0.80±0.07	0.98±0.00	0.63±0.14	98.49±0.60
Subj07	0.73±0.09	0.59±0.11	0.84±0.06	0.99±0.00	0.66±0.14	98.79±0.35
Subj08	0.79±0.09	0.67±0.11	0.91±0.04	0.99±0.00	0.72±0.13	99.10±0.21

Discussion

The manual analysis and segmentation of MS lesions in MRI images are both daunting and time-consuming and also require considerable expertise. In addition, manual noise reduction capabilities are rather limited and almost negligible. On the other hand, experts may not be able to detect small lesions and mistakenly consider them as healthy brain tissue (FN). Therefore, a new algorithm was proposed in this paper for automatic MS plaques segmentation using MRI images. The algorithm aims to help the neurologist in order to improve the diagnosis and monitoring of this disorder. Other advantages of the automatic segmentation method may declare independence from operator and his/her skills. Furthermore, automatic segmentation algorithms are believed to have a significant effect on the detection time of plaques and consequently segmentation time.

In this paper, the feature extraction phase, one of the most important parts in this method, begins precisely after the initial preprocessing on the images. At this stage, efforts were made to impart more novelty to the study, in addition to achieving the desired features from the feature extraction step. The purpose was to combine the image brightness intensity and the histogram-based variance (Equation 8). After extracting the proper feature selection helps to achieve an optimal target function to be used for the classification and ultimately segmentation. Based on the using the SFS method, four informative features were selected out of the 11 initially introduced and the results obtained based on Dice, Sensitivity and PPV parameters have clearly improved, as shown in Figure 13.

Figure13.

The effect of using all features compared to the use of selected features in order to improve the process of segmentation. The criteria are Dice, Sensitivity and PPV parameters.

Selected features in the structure of SVM classifiers were used for the proposed Ensemble SVM algorithm, in both methods. The comparison of the segmentation results in Figures 11 and 12 shows that both methods have relatively acceptable results. In Tables 2 and 3, evaluation criteria results of 8 subjects for MS lesions segmentation using the Ensemble SVM method in comparison to Gold Standard are presented. Based on results achieved by comparing the Dice factors, the similarity level of the automatic and manual segmentation was computed.

Since the similarity criterion (Dice factor) was not able to completely determine the desired result alone; in addition, both PPV and Sensitivity factors roughly determine the sensitivity of the algorithm in lesions detection and also play a significant role in this study. The comparison between two approaches of the Ensemble SVM method, based on these three evaluation parameters, is shown in Figure 14. Although the algorithm possesses a higher sensitivity of correct lesion detection in the first method, its Dice and PPV values are less than the second method. A post-processing stage, before voting initiation in the system, was one of the most important differences between the two methods. The morphology Erosion operation was used in this section, which increased segmentation similarity and improved PPV value in MS lesions detection. Therefore, the results of automatic segmentation in the second approach are quite in agreement with the Neurologist’s opinion.

Figure14.

Evaluation of Dice, Sensitivity and PPV criteria in the two proposed modes of the Ensemble SVM method.

Conclusion

In this article, the authors tried to use the automatic MS lesion segmentation in not only speeding up the detection but also monitoring of MS syndrome. Regarding the advantages of SVM technique, two Ensemble algorithms were proposed and applied to some patients for MS lesion segmentation. The result of these proposed algorithms are nearly similar to each other. One of the algorithms is superior because it is similar to result of specialist’s deed, which the dice criteria approved it. The second algorithm (dice: 0.6(±.12)) has acceptable results by observing the specialist’s point of view (dice: 0.6(±0.178)).