Disease-specific imaging utilizing support vector machine classification of H-scan parameters: assessment of steatosis in a rat model

Abstract

In medical imaging, quantitative measurements have shown promise in identifying diseases by classifying normal vs. pathological parameters from tissues. The support vector machine (SVM) has shown promise as a supervised classification algorithm and has been widely used. However, the classification results typically identify a category of abnormal tissues, but do not necessarily differentiate progressive stages of a disease. Moreover, the classification result is typically provided independently as a supplement to medical images, which contributes to an overload of information sources in the clinic. Hence, we propose a new imaging method utilizing the SVM to integrate classification results into medical images. This framework is called disease-specific imaging (DSI) which produces a color overlaid highlight on B-mode ultrasound images indicating the type, location, and severity of pathology from different conditions. In this paper, the SVM training was performed to construct hyperplanes that can differentiate normal, fibrosis, steatosis, and pancreatic ductal adenocarcinoma (PDAC) metastases in livers based on ultrasound echoes. Also, cluster centroids for specific diseases define unique disease axes, and the inner product between measured features and any disease axis selected by the SVM quantifies the disease progression. The features were measured from 2794 ultrasound frames using the H-scan analysis, attenuation estimation, and B-mode image analysis. The performance of our proposed DSI method was evaluated for a pre-clinical model of steatosis (n = 400 frames). The contribution of each feature was assessed, and the results were compared with ground truth from histology. Moreover, the images generated by our DSI were compared with earlier imaging methods of B-mode, H-scan, and histology. The comparisons demonstrate that DSI images yield higher sensitivity to monitor progressive steatosis than B-mode and H-scan and provide a comparable performance with the histology. For the parameter comparison, DSI and H-scan resulted in similar correlation with histology (r_s=0.83), but higher than attenuation (r_s=0.73) and B-mode (r_s=0.47). Therefore, we conclude that DSI utilizing the SVM applied to steatosis can visually represent the classification results with color highlighting, which can simplify the interpretation of classification compared to the traditional SVM result. We expect the proposed DSI can be used for any medical imaging modality that can estimate multiple quantitative parameters at high resolution.

I. Introduction

In medical ultrasound, a number of quantitative measurements have been developed over the past decades as a means of improving diagnosis and going beyond subjective assessment of B-mode imaging [1–8]. The broad field of tissue characterization has analyzed scattering and propagation parameters of tissues across a variety of conditions [9–14].

As scanning systems develop more capabilities, clinicians can capture more parameters from a region of interest (ROI) interrogated by ultrasound. This leads to multiparametric studies to discriminate between groups, such as normal vs. diseased tissue, and different disease (for example, hepatic steatosis vs fibrosis) [15]. Early work on the statistical classification of multiparametric clusters of ultrasound-related measurements indicated the usefulness of incorporating several quantitative measurements [16, 17]. Today, an emerging set of parameters can be measured using B-mode or optional sequences in clinical scanners. Specifically, the attenuation coefficient estimated using B-mode imaging has the feasibility to stage steatosis [18–20]. Further, there are parameters related to both ultrasound and shear wave propagation in tissues which are linked to the staging of steatosis [21–25] or fibrosis [26, 27]. However, these measurements only show trajectories of the parameters without providing any diagnostic guide, meaning that it is desirable to propose methods that can set thresholds for 1D or boundaries for multi-dimensional space between different disease stages. Moreover, this evolving set of multiparametric measures raises an important next-level question: how to best integrate a plethora of new measurements in a straightforward way that assists the clinical diagnosis of a patient? Recent studies have employed machine learning to classify multi-features [28–32], although deep learning is widely applicable to solve ultrasound problems, such as image reconstruction [33, 34], image enhancement [35], and lesion segmentation [36, 37]. Deep learning is known as more advanced technique than machine learning with the advantage that it directly extracts features from input images, avoiding preprocessing for feature measurement and selection. However, it requires a larger number of training sets and longer processing time, which can be critical disadvantages since (1) in medical imaging, collecting a large number of patient training sets may be challenging, and (2) ultrasound applications aim to be real-time. Hence, tissue classification for multi-parameters utilizes the support vector machine (SVM), which requires less processing time and a smaller number of training datasets compared with other machine learning and deep learning methods. Classification examples using SVM include liver lesions [20, 30, 38–40], liver fibrosis [31], steatosis [41, 42], breast lesions [32], and thyroid nodules [28]. However, these studies extracted features using intensity signals based on general image processing techniques. Recent studies [20, 39, 41, 42] have extracted some features in the frequency domain based on ultrasound physics, including measurements from (1) attenuation caused by ultrasound beam propagation and (2) frequency shifts caused by ultrasound attenuation, which may be not characterized by general image processing. Thus, tissue classification with multiple parameters can take advantage of the SVM and ultrasound physical properties. Although the research demonstrates that multiparametric analysis can classify diseases, routine clinical use of the features will require new integrative and simple-to-use tools to be developed. For example, color-coding utilizing quantitative parameters would be beneficial to clinicians since it directly clarifies feature differences between normal and abnormal tissues [6, 43]. It has been common practice to display each measured parameter independently, however this requires the clinician to integrate information across numerous images. A method combining multi-parameters and showing a representative image may be advantageous. We have addressed this need by formulating a framework called disease-specific imaging (DSI) utilizing machine learning and mathematical approaches. In this framework, a training set is studied using statistical methods to identify clusters of parameters (corresponding to specific disease states) in a multidimensional space comprised of the multiparametric measures [40]. Then, as a new scan is acquired from a patient, these parameters are examined in the multidimensional space and classified using the SVM, which constructs decision planes of the disease clusters [39]. Mathematical measures of closeness to a disease cluster are then applied to the original B-mode image as color overlays, with unique colors assigned to the unique diseases identified within the training set. In this way the complicated mathematical analysis of multiparametric information, which can be overwhelming for an individual to sort out, can be simply displayed in the context of the familiar B-mode image.

In this paper, we apply this DSI framework to in vivo scans from an animal model of simple steatosis of the liver. The DSI framework has been trained on liver scans from three previous studies with four different conditions: liver fibrosis from CCl₄ (carbon tetrachloride) exposure [20], liver metastases from pancreatic ductal adenocarcinoma (PDAC) infiltration [44], liver steatosis from choline-deficient diets [41, 42], and normal liver from the control group for each study. These previous studies all derived multiple quantitative parameters in connection with the H-scan analysis and other metrics, and the clusters for normal vs. pathological liver conditions were identified in a unified multiparametric space, defining corresponding disease axes. The DSI images are generated based on degrees of closeness to the identified disease axes. The results are shown to correspond to histological assessments of fat infiltration in the liver and provide both quantitative information about the progression of steatosis and a simple visual display of the increasing amount of fat.

II. Theory

In this section, we introduce the theoretical background and the motivation of our proposed medical imaging method to visualize SVM classification of diseases. The SVM classification result can be directly imaged and overlaid onto B-mode ultrasound images.

A. Support vector machine

The SVM classifier [45–47] is widely used in medical imaging applications to classify different diseases due to a number of advantages. First, the concept of the SVM is based on maximization of margins between clusters for input data points. While maximizing, the SVM only utilizes the data subsets positioned near the clusters’ boundaries, called support vectors. Thus, the SVM can determine decision planes successfully even if there are not enough data points, but those data points near the boundaries can well describe the data distribution between different classes. Second, SVM is a convex optimization problem. Therefore, the solutions of the SVM are a global optimum, whereas solutions from other classifiers can be local minima. Third, the SVM allows misclassified data points near boundaries, which can provide practical solutions because most real-world data sets are not 100% separable for different classes. The amount of misclassified data points that can be allowed by the SVM is determined by the box constraint parameter. Lastly, the SVM can employ the Gaussian kernel function, which results in non-linear hyperplanes; the smoothness of the hyperplanes is determined by the Gaussian kernel sigma parameter. Therefore, the SVM methods have been applied to the medical imaging field to classify diseases by using multiple parameters extracted from the medical images, of which output is a class; for example, a medical imaging scan can be classified as a normal condition or a specific disease such as inflammation, fibrosis, steatosis, or a tumor type. As illustrated in the ‘Training’ box of Fig. 1, the features extracted from ultrasound signals are used as inputs for the SVM training, and then the SVM outputs decision planes that can be used for the prediction. In the ‘Prediction’ box, Method A in Fig. 1 describes how the decision plane can classify the disease class of a new patient scan. The output is a 1D parameter (‘CLASS’) in:

$$\left\{CLASS|1,2,\dots ,M_{disease}\right\}$$

where M_disease is the number of classes used for the training. For example, when classifying the four liver conditions of normal, steatosis, fibrosis, and tumor, the classification would result in one of the four, and CLASS = 1, 2, 3, and 4 denote normal, steatosis, fibrosis, and tumor, respectively.

Fig. 1.

DSI strategy to visualize SVM classification. DSI is proposed as a supervised learning method, which requires training and prediction procedures. Utilizing a training dataset with ultrasound images and the corresponding ground truth tags, training results in decision planes constructed by the SVM and disease axes obtained from feature clusters. Here, three different prediction methods are provided to introduce DSI as Method C. Method A illustrates traditional SVM classification, and the trained decision plane classifies the new patient scan’s state; the output is the 1D parameter ‘CLASS’. Method B provides further analysis using the inner product in addition to the SVM classification. By utilizing the CLASS obtained from the trained SVM, projection onto the disease axis (i = CLASS) is also calculated where i indicates a class index. Thus, Method B outputs a 2D parameter (CLASS, projection). Method C color-processes the 2D parameter results suggesting that DSI that can visualize the classification results with indication of disease progression level. Method C makes it easier to distinguish disease states compared to Methods A and B.

This classification-based analysis does have limitations. The classification result cannot provide progressive level/disease severity, i.e., it cannot indicate progression from normal or early stages to late stages. The decision planes obtained by the SVM classifier are illustrated in a multiparameter space however these cannot be directly visualized with the using common imaging modalities. Without background knowledge regarding the parameters, users cannot understand the meaning of the decision planes and cluster distributions. Moreover, the decision planes are not visible if there are more than 3 features. Therefore, any visualization method that can directly clarify the 1D classification result is highly desirable.

B. Inner product with disease axis

Because the 1D parameter output of SVM classification does not indicate the level of disease progression, we propose a method that can estimate disease score (the severity of disease/level of disease progression) by using the inner product to calculate a projection. Considering a feature set with M diseases containing progressive disease states and a feature dimension of N (the number of features), the following analysis is performed on the N-dimensional features $\left(\in {ℝ}^N\right)$. We propose the concept of the disease axis, and the feature’s projection (a scalar value $\in {ℝ}^1$) onto the disease axis indicates the disease progression level. The disease axis is defined by the SVM training dataset. The data points in each CLASS form a cluster, whose linear fit line is obtained with the linear equation form:

$$a_1\cdot x_1+a_2\cdot x_2+\dots +a_N\cdot x_N+b=0$$ (1)

where a_i and b are coefficients identifying the linear fit line. We can choose any two different points that the line passes, whose coordinates can be written as $\left(A_1,A_2,\dots ,A_N\right)\equiv \overrightarrow{A}$ and $\left(B_1,B_2,\dots ,B_N\right)\equiv \overrightarrow{B}$. Then, a unit vector for the line can be defined by:

$$\frac{\left(\overrightarrow{A}-\overrightarrow{B}\right)}{{\Vert \overrightarrow{A}-\overrightarrow{B}\Vert }_2}=\frac{\left(\overrightarrow{A}-\overrightarrow{B}\right)}{\sqrt{\left(\overrightarrow{A}-\overrightarrow{B}\right)\cdot \left(\overrightarrow{A}-\overrightarrow{B}\right)}}$$ (2)

where the notation · represents the inner product or dot product, defined by:

$$\overrightarrow{c}\cdot \overrightarrow{d}={\sum }_{i=1}^N{c}_i{d}_i\equiv projection.$$ (3)

$\overrightarrow{c}=\left(c_1,c_2,\dots c_N\right)$ and $\overrightarrow{d}=\left(c_1,c_2,\dots ,c_N\right)$ are any vector in ${ℝ}^N$ space. In (3), the scalar value derived from the inner product is called “projection”. The Fig. 1 training box illustrates an example for N = 3. Each cluster can find its linear fit line indicating the direction of the disease progression, and then we can calculate unit vectors for these lines. The three axes for the unit vectors become disease axes. Thus, the training can result in decision planes and a disease axis set. With the new patient scan and its features, method B in the prediction box in Fig. 1 first classifies the liver states using the decision planes obtained from the SVM and results in CLASS, and then it calculates the inner product between the features and disease axis corresponding to the output CLASS. The projection obtained from the inner product indicates the disease progression level. For example, a feature classified as steatosis has a steatosis progression level determined by calculating the inner product between the steatosis axis and the feature. Therefore, disease classification (CLASS) and its progression level (projection) can be provided as a 2D parameter (CLASS, projection), but these parameters are shown in parameter space, independently, with their corresponding input images.

C. Disease-specific imaging to visualize SVM classification

We propose DSI, which is designed as a supervised learning method to predict disease types and progression levels from input ultrasound scans (or, in principle any imaging modality that can measure multiple features in pixel-wise resolution). DSI first requires training as illustrated in the Fig. 1 Training box, and a training dataset and the corresponding desired output can be ultrasound signals and pathological confirmation for each target of the scans, respectively. Then, the training results are (1) decision planes constructed by using the SVM and (2) disease axes obtained by analyzing the clusters of input features. Based on the desired output provided by pathology, the number of diseases that can be identified by the trained DSI can be specified. For example, in this study, the DSI enables identification of four liver conditions, including normal, steatosis, fibrosis, and PDAC metastasis in liver. The DSI prediction is depicted in the Fig. 1 Prediction box Method C. The trained DSI can output a color image by processing a new patient scan. Features extracted from the new scans are assigned as input to the trained DSI, and then the trained SVM and inner product utilizing its SVM classification result and disease axes output (CLASS, projection). The 2D output corresponds to (disease name, progression level), which is mapped into (Color, Intensity). The color images generated using the assigned color and its intensity are overlaid onto the input B-mode image. The four colors of grayscale, yellow, green, and red indicate normal, steatosis, fibrosis, and PDAC metastasis, respectively. For example, as fat accumulates in a liver, the yellow color becomes brighter. Its measured features are gradually changed, advancing along the steatosis axis. Accordingly, the inner product of the features and the unit vector of the steatosis axis increases. While training the DSI, we set a maximum inner product value indicating the severest steatosis level. Using this maximum, the inner product results (projection onto the steatosis unit vector) are normalized, resulting in a minimum of 0 and a maximum of 1. The normalized values become RGB color components. For steatosis, the normalized value × [1,1,0] becomes the RGB code for each pixel. DSI color can change from dark to bright, indicating disease progression level. Therefore, our proposed method visualizes the SVM classification using a simple color display.

III. Materials and Methods

A. Study overview

The proposed DSI method is shown in Fig. 2. Ultrasound scans were performed, and users manually set a region-of-interest (ROI) box for DSI processing; to monitor disease progression, it is recommended to set the ROI consistently while repeating ultrasound scans over time. Within the ROI, feature extraction is performed. Three features characterizing tissue signatures are obtained from H-scan imaging, attenuation estimation, and B-mode imaging. The features become the input of the DSI processing. The DSI procedure consists of two main steps: (1) training the DSI, resulting in hyperplanes constructed by the SVM and disease axes derived from feature clusters, and (2) predicting liver conditions with the trained DSI to inform disease progression. When predicting liver conditions, DSI finally outputs disease-specific images for the scanned liver.

Fig. 2.

Disease-specific imaging procedure, including feature extraction, training, prediction. More details for the Prediction block are described in Fig.1 Method C.

B. Training and Testing datasets

For the proposed DSI, the training and the testing study were independently performed. As shown in Fig. 2, DSI prediction requires training, and therefore the pre-training was performed [40], which corresponds to the “Training DSI” block in Fig. 2. The trained DSI has applied to steatosis cases in this work.

For the training, we included 4 different classes of liver conditions: normal, steatosis, fibrosis, and PDAC metastasis. The 3 disease conditions of steatosis, fibrosis, and PDAC metastasis were induced in animal livers. The steatosis was induced by a choline-deficient diet in 12 Sprague-Dawley rats (SD, Charles River Laboratories, Wilmington, MA, USA) for 6 weeks. The fibrosis was induced by CCl₄ (carbon tetrachloride) exposure (1mL/kg, three times per week) for 4 to 8 weeks in enrolled 11 rats, of which 7 were SD rats and 4 were TAC NIHRNU (nude, Taconic Biosciences, Inc., Rensselaer, NY, USA). The PDAC metastasis in liver was grown by injecting 4 × 10⁵ luciferase expressing murine pancreatic tumor cells into the spleen of 9 C57BL/6J mice (Jackson Laboratory, Bar Harbor, ME, USA). The progression of PDAC metastasis was monitored from 10 days to 9 weeks after the injection. The baseline animals before diseases induction were considered as the normal groups, and the each of the diseases progressed from early to late stages. Their end time points were investigated by pathologists. The stages became the ground truth for the DSI. The livers were ultrasound-imaged from baseline to late stage of each disease so that the total of 2794 ultrasound frames acquired by three ultrasound scanners: Verasonics ultrasound scanner (Vantage 256, Verasonics, Inc., Kirkland, WA, USA) with an L11–5v probe (10 MHz center frequency) for PDAC metastasis; Vevo 2100 (FUJIFILM VisualSonics, Toronto, ON, Canada) with a MS 250 linear transducer (21 MHz center frequency) for fibrosis; and Vevo 3100 (FUJIFILM VisualSonics, Toronto, ON, Canada) with a MX 201 linear probe (15 MHz center frequency) for steatosis. The center frequencies of transmission were provided above, and this study recommends that the sampling frequencies need to be high enough based on the Nyquist sampling theorem for frequency domain analysis of the H-scan and attenuation estimation. The numbers of scanlines are determined by transducers. The VisualSonics scanner saved radiofrequency (RF) data, and the Verasonics scanner saved in-phase quadrature (IQ) data. The feature extraction of H-scan and attenuation estimation analysis used RF or IQ data, and B-mode intensity was measured from log-compressed data. We involved the three studies with different disease models of liver pathology, which were investigated by three ultrasound scanners utilizing the transmissions with different center frequencies and bandwidths. These experimental settings were designed to verify that our proposed approach, including training for the SVM, is not limited to a specific animal disease model or ultrasound system.

To evaluate the performance of the DSI prediction as a testing study, especially for steatosis, we scanned 400 ultrasound frames independently with the training study. As an in vivo study, ten Sprague–Dawley rats (Charles River Laboratories, Wilmington, MA) were enrolled to monitor progression of nonalcoholic fatty liver disease (NAFLD). The ten animals were randomly divided into two groups: control (n = 4) and diet (n = 6), which were fed by a control or methionine and choline deficient (MCD) diet, respectively. The MCD diet induces fat accumulation in livers. The rat livers were ultrasound-imaged at baseline and week 2. The Vevo 3100 scanner (FUJIFILM VisualSonics Inc., Toronto, Canada) utilizing a 15 MHz center frequency linear probe (MX201) acquired RF data. Each scan acquired 100 frames of RF data, and among them 20 frames were selected at regular intervals, whereby a total of 400 frames were investigated. The animals were sacrificed at week 2 to obtain liver sections for histology of both hematoxylin and eosin (H&E) and Picrosirius red stain; object × 20 and × 40 magnifications were obtained. For all animals in this testing data set, their liver conditions were pathologically confirmed using the histology images, and the confirmation was used as the ground truth. This protocol was approved by the Institutional Animal Care and Use Committee at the University of Texas at Dallas.

C. Feature extraction

We extracted three features from the backscattered ultrasound echoes, utilizing the attenuation estimation, H-scan analysis, and B-mode imaging. Please note that this study demonstrates that the features extracted from ultrasound images acquired from three different ultrasound systems can be unified into the multiparametric space.

1). Attenuation Estimation:

The attenuation coefficient (α, [dB/MHz/cm]) was first measured by estimating the frequency down-shift over depth using the concept of H-scan analysis that can measure frequency components [20]. The processing was performed line by line, whereby each scanline has an estimated attenuation coefficient. An example image of estimated attenuation coefficients is provided in Fig. 2 with the attenuation color bar ranging from 0 to 1 dB/MHz/cm.

2). H-scan analysis:

H-scan is a matched filter analysis used to identify tissue characteristics by measuring frequency shifts from reflected echoes [48]. The H-scan displays colorcoded images by mapping the frequency shifts to red/blue colors. The colormap is provided in Fig. 2; more red colors represent frequency down-shift that may be caused by larger scatterers, whereas more blue colors represent frequency up-shifts caused by smaller scatterers. Prior to measuring the H-scan parameter of percent blue (% blue), based on the estimated attenuation coefficient, attenuation over depth in RF data was first corrected [48]. The attenuation-corrected RF data was assigned as the input of the H-scan analysis, and therefore H-scan color images can result in uniform color distribution over depth but discriminated colors corresponding to scatterer size. The colors from red to blue correspond to relatively smaller to larger scatterer size [48]. The color distributions for smaller to larger scatterers are provided in the Fig. 2H-scan color bar. The H-scan color levels are digitalized with 256 scales, and for quantifying the H-scan analysis we calculate % blue as:

$$\%blue=\frac{\text{Number\hspace{0.17em}of\hspace{0.17em}blue\hspace{0.17em}pixels}}{\text{Number\hspace{0.17em}of\hspace{0.17em}all\hspace{0.17em}pixels\hspace{0.17em}within\hspace{0.17em}a\hspace{0.17em}ROI}}\times 100\%$$ (4)

where the pixels within ROI are divided into red and blue pixels, and these have color levels of 1–128 and 129–256, respectively. The color levels correspond to a matched filter set whose peak frequencies range from 5.3 to 15.7 MHz with an equal interval between the 256 peak frequencies for steatosis. The peak frequencies of the matched filters were determined to have 50% blue for normal livers. Since the transmit frequencies of the three scanners vary, different peak frequencies for matched filters were used.

3). B-mode imaging:

B-mode intensity in the dB scale was measured from log-compressed images. As mentioned in Section III.B, three different ultrasound scanners were used, which have different transmit frequencies, system designs, and post-processing, resulting in different signal levels and spectra. To incorporate features obtained from the different signals into an analysis, feature standardization is required. To be specific, when training DSI, we standardized B-mode intensity obtained from the different scanners. Each scanner has a different intensity baseline, and thus B-mode intensity for normal livers was set to −15 dB, requiring an intensity level offset for each scanner.

D. Testing of DSI applied to steatosis

We evaluated the accuracy of the trained DSI by performing a testing study for steatosis. DSI was applied to the testing dataset of 400 ultrasound frames of the ten animals acquired at baseline and week 2.

1). Parameter Analysis with Feature Trajectories:

For the measured features, statistical analyses were performed. To verify that the parameters are able to monitor liver tissue changes due to disease progression, we investigated trajectories of the measured parameters as a function of severity of the disease states. First, we define a combined parameter $C\left(C\in {ℝ}^1\right)$ by combining the H-scan and attenuation parameters, which can be considered as frequency domain-based analysis. The 1D trajectories were investigated. Parameter trajectories including B-mode intensity were also the investigated in 3D space. For both 1D and 3D trajectories, the pathways of the training and testing studies were compared.

The combined parameter C is defined with H-scan % blue and attenuation coefficient as:

$$C={\omega }_H\cdot \left|x_H\right|+{\omega }_A\cdot \left|x_A\right|$$ (5)

where x_H and x_A are normalized H-scan and attenuation parameters, respectively, obtained by:

$$x_H=\frac{\left(\%\text{blue}\right)-50}{10};x_A=\left(\alpha -0.5\right)\times 10.$$ (6)

These parameters tend to range from −5 to 5 and are 0 when a normal liver is scanned. As any disease progresses in liver, the absolute sum for the normalized H-scan and attenuation coefficient, |x_H|+|x_A|, increases, whereas normal livers have an absolute sum near 0. In addition to the summation of the absolute values which indicate disease severity, there are changing directions determined by positive or negative values of % blue or α. When investigating the liver diseases PDAC metastasis, steatosis, and fibrosis, we find that the weights w_A = 0.6 · sign(x_A) and w_H = 0.4 · sign(x_A) provide the best separation of the three diseases [40].

2). Segmentation for Diseased Area:

DSI performance was evaluated by calculating fat fraction area and comparing it with histology as shown in Fig. 3. The DSI highlights fat fraction in yellow, and the fat fraction was quantified as ‘steatotic %’:

$$\begin{array}{l}\text{steatotic}\%\\ =\frac{\text{Number of pixels classified as steatosis}}{\text{total number of pixels}}\times 100\%\end{array}$$ (7)

where our DSI classifies each pixel’s state between normal, steatosis, fibrosis, and PDAC metastasis. Similarly, ‘fibrotic %’ and ‘tumor %’ were also calculated with the number of pixels classified as fibrosis and PDAC tumor, respectively, by updating the numerator in (7). In this way the percentages for the three diseases were calculated for DSI.

Fig. 3.

Segmentation of diseased area. (a) The method to calculate fat fraction in histology images. (b-e) Fat segmentation in DSI and histology. Examples of (b) control and (c-e) diet group are shown; from (b) to (e), there is gradual fat accumulation. Fat area is segmented using yellow regions in DSI and overlaid on B-mode images within red ROI boxes. Additionally, for the histology images, fat area is segmented using green colors.

To evaluate DSI accuracy by comparing the result with the gold standard, fat fraction in histology images was also calculated using fat-stained areas. We can quantify steatosis progression by measuring the area of fat inclusions within the liver histology images. As shown in Fig. 3 (a) and Fig. 4, fat inclusions appear in brighter than normal tissues. The histology images were first converted to gray scale, and then a threshold for binarization was set. The binarization in Fig. 3 (a) enables the differentiation of fat pixels from normal pixels, then the number of pixels classified as steatosis by binarization was divided by the total number of pixels in histology, and therefore steatotic % in histology images was also calculated using (7). The steatotic % classified by DSI and histology was compared by using Spearman’s correlation coefficient.

Fig. 4.

B-mode, H-scan, and histology images for a normal (top row) and fatty liver (bottom row) case. The red boxes in the B-mode images represent processing ROIs. H-scan colors are shown within the ROIs. H&E and Picrosirius red histology images for the animals are shown. For the normal liver, only normal cells are found; their diameters are approximately 20 µm. For the fatty liver, fat inclusions (white in color) are found within the normal tissues. The inclusions’ diameter, measured for all diet group animals, was 8.6 ± 1.8 µm.

IV. Results

A. Feature analysis

1). H-scan:

Example images of B-mode, H-scan, and histology are displayed in Fig. 4; the upper and lower rows show normal and fatty liver results, respectively. In the fatty liver images, the B-mode images appear brighter, and the H-scan images show as more blue compared to the normal liver. For the histology images, the normal liver only shows normal cells, whereas the fatty liver also shows white fat inclusions. The histology slides indicate that the fat inclusions tend to have smaller diameters than normal cells. The fat diameters were measured for all diet groups’ cases, and the average was 8.6 ± 1.8 µm. This average diameter is smaller than the normal hepatocytes’ diameter of approximately 20 µm. This is plausibly the key mechanism explaining why the fatty livers appear more blue on H-scan compared to normal livers.

2). Feature and Statistics:

The features measured from H-scan, attenuation estimation, and B-mode are presented in Fig. 5. For the control group, the parameter measurements remain unchanged over time, whereas for the diet group they increase from baseline to week 2. The H-scan and B-mode metrics showed significant difference between weeks 0 and 2 for the diet group. While the attenuation coefficient is not significantly different between the control and diet groups, the diet group tends to have higher attenuation than the control; we may need a larger number of livers for a more reliable result. Overall, the measured parameters change as fat accumulates, demonstrating each parameter contributes to the differentiation between normal and fatty livers.

Fig. 5.

Traditional display of individual parameters (a) H-scan % blue, (b) attenuation coefficient, and (c) B-mode intensity. The following statistical notations were employed: ns (no significance) indicates p > 0.05; ^∗ indicates p < 0.05; ^∗∗ indicates p < 0.01; ^∗∗∗ indicates p < 0.001; and ^∗∗∗∗ indicates p < 0.0001.

3). Trajectories:

To track disease progression in addition to a gross classification between normal and abnormal, we combined the parameters and investigated them in multidimensional space, as shown in Fig. 6, which demonstrates that the parameters’ trajectories over time follow the disease progression pathways in combined parameter or multiparametric space; the progression direction is from the origin to the end point of the pathways. Fig. 6 (a) analyzes the 1D combined parameter, showing trajectories for steatosis, fibrosis, and PDAC metastases over time. It displays the three diseases’ progression pathways obtained from the training and testing study, demonstrating that the testing study’s trajectories follow the steatosis pathway obtained from the earlier training set. Moreover, Fig. 6 (a) shows 4 areas representing normal to early stage, steatosis, fibrosis, and PDAC metastasis, where boundaries were decided by SVM with the training dataset (n = 2794 images). The start and end time points were tagged as normal and diseased, respectively, for the training and testing datasets (n = 400 images). The trained SVM was applied to the testing set, and the classification accuracies for the training and testing datasets were 100% and 91.7%, respectively.

Fig. 6.

Parameter changes in (a) 1D and (b) 3D space caused by fat accumulation. (a) The combined parameter changes at 2 weeks follow the steatotic trajectories. (b) In 3D space, the vectors for the second week’s measurements are located near the steatosis axis, instead of the tumor or fibrosis axes. The vectors were projected onto the nearest disease axis, which is the steatotic axis here. The blue line denotes the projection from the original vectors to the steatosis axis. Projected points are shown with yellow dots. The larger the distance between the origin and the projection, the more severe the steatosis.

In 3D vector space, as shown in Fig. 6 (b), the week 2 diet measurements resulted in vectors near the steatosis axis among the three different disease axes. Their projections onto the steatosis axis quantifies components of the steatosis axis. When the projection result is further from the origin, it indicates more progressive steatosis, quantifying fat accumulation level based on the axis obtained from DSI training. The trained axis indicates progression direction and measured parameters from normal to late stage/severe steatosis. Therefore, the multiparametric study depicted in Fig. 6 enables the tracking of parameter changes by employing a joint analysis related to steatosis progression. Hence, we propose DSI as a simple visual display which enables clinicians to visualize how steatosis is progressing based on the imaging, without detailed examination of the features obtained from ultrasound physics.

B. DSI performance

1). Diseased Area Segmentation:

The segmentation results obtained using DSI are illustrated in Fig. 3 with segmented area and color intensity, which is capable of tracking gradual fat increase by utilizing color changes from dark to bright. From Fig. 3 (b) to (e), there is gradual fat accumulation from normal to progressive steatosis. Although Fig. 3 (c) to (e) are all selected from the diet group at week 2, each animal had a different volume percentage of fat, which can be verified by the steatotic % calculated from histology images. Consistent with the fat segmentation in histology, DSI also demonstrates a gradual increase in overlaid yellow area on B-mode images. The yellow area was segmented using DSI; the trained SVM in DSI classified each pixel’s disease type, and then DSI highlights the segmented fat area in yellow. The example images in Fig. 3 only have pixels displaying yellow, meaning all pixels classified as diseased tissues resulted in steatosis. In addition to steatotic %, DSI also measured fibrotic % and tumor %; the average percentages with one standard deviation are listed in Table I. The steatotic % for the MCD diet group obtained from both DSI and histology is higher than the other percentages. The steatotic % for the control group, and the fibrotic % and tumor % for all cases can be considered as false classification results when they are greater than 0; nevertheless, the false classification percentages are less than 1%. The false classification of DSI is comparable to the error of histology, meaning that when investigating control livers using histology figures, the steatotic % also resulted in 0.96 ± 0.95% instead of 0%. Therefore, we conclude that our DSI approach enables the identification of steatotic tissues in liver with a similar error level as measuring fat fraction in histology.

TABLE I.

Disease segmentation percentages obtained by DSI and histology. (Average ±1 standard deviation)

	Diet	Steatotic %	Fibrotic %	Tumor %
Histology	Control	0.96 ± 0.95	-	-
	MCD	20.02 ± 6.90	-	-
DSI	Control	0.77 ± 1.47	0.08 ± 0.36	0.75 ± 3.33
	MCD	31.30 ± 24.97

2). Correlation with Histology:

To further assess DSI performance by testing whether it can detect gradual fat accumulation as well as differentiating between normal liver and steatosis, we compared the performance with histology. The ability to segment the fat fraction was assessed through steatotic % in both DSI and histology, and the percentages were compared. The steatotic % in histology images was calculated for individual livers using both 20× and 40× magnification. The overall correlation between DSI and histology is high, as shown in Fig. 7, with Spearman’s correlation r_s of 0.842 and 0.808 for the 20× and 40× magnification, respectively, although only 10 animals were investigated. Hence, DSI can detect the gradual increase in fat accumulation or progressive steatosis, including early-stage steatosis.

Fig. 7.

The overall correlation between DSI classification of steatosis and histological measurement of fat inclusions was (a) 0.842 for 20× magnification and (b) 0.808 for the 40× magnification.

C. DSI performance comparisons

1). Feature Selection and Contribution for the SVM classification:

We selected three features that can be considered as having independent properties; H-scan and B-mode measure scatterers’ properties which can be estimated in frequency and time-domain, respectively, and attenuation coefficient estimates physical properties derived by ultrasound beam propagation. As described in Section IV.A., we have demonstrated our features and their trajectories can successfully characterize liver states and steatosis progressions through evaluations with statistics and SVM classification. Moreover, as described in Section IV.B., combining the features by the DSI utilizing the SVM shows strong agreements with histology measurements. Therefore, we expect that all the parameters could contribute to classification. To verify this, we investigated their contribution as input features of the SVM classification.

First, we investigated correlation between the features and the ground truth of measurements from histology to estimate each feature’s contribution. As shown in Fig. 8 (a), DSI and H-scan have the highest correlation (r_s = 0.83) with the ground truth. B-mode intensity resulted in r_s = 0.73, and attenuation has the least correlation with r_s = 0.47.

Fig. 8.

Performance evaluation compared to other imaging methods. (a) Correlation between the ground truth and the measured features. (b) SVM hyperplane optimization using 2D features, excluding attenuation coefficient. (c) SVM hyperplane optimization using 3D features. (d) Comparison of the images acquired using different methods. The livers from left to right have fat accumulation from normal to severe steatosis.

Comparing the correlation coefficient, the DSI utilizing attenuation resulted in the comparable performance with H-scan, but the attenuation showed the relatively lower performance than the others. Therefore, to see whether attenuation can actually contribute to the DSI, we performed two different trainings without and with the attenuation parameter. As mentioned in Section IV.A.3., the SVM training for our DSI reached 100% classification accuracy when utilizing 3 parameters, including the attenuation coefficient. Fig. 8 (b) shows SVM training results with just the two input features of H-scan and B-mode, excluding attenuation. The hyperplanes have classification accuracies ranging from 87.8% to 100%, from left to right, according to their parameters. The SVM parameters of box constraint and Gaussian kernel were adjusted to optimize hyperplane shapes, and thus from left to right there is a tendency from under- to over-fitting of results. The implementations were carried out with MATLAB R2018b (The MathWorks, Inc., Natick, MA, USA); the tested classifier is the SVM with Gaussian kernel, and the SVM parameters were adjusted from 1 to 100 for hyperplane optimization. According to Fig. 8 (b), the SVM classification in 2D space excluding the attenuation parameter cannot reach 100% accuracy without overfitting. We can conclude that the classification accuracy obtained from 2D analysis is approximately 90% without overfitting. However, when including the attenuation parameter, the 3D analysis of the SVM in Fig. 8 (c) shows all 100% accuracy. From left to right, the parameters were set to have higher accuracies, but the hyperplanes were getting overfitted. However, the left-most figure provides the decision planes without overfitting. Therefore, we can conclude that the attenuation parameter also contributes to the SVM classification, because without the parameter the SVM classification cannot reach 100% accuracy.

2). Performance Comparisons with Other Methods: Imaging:

To illustrate the advantages of our proposed DSI method in the imaging aspect, we compared imaging results of the DSI and other existing methods providing images in Fig. 8 (d). This figure displays images obtained using B-mode, H-scan, DSI, and histology (Picrosirius red (PSR) and H&E). Each column was imaged using the same animal, but with different imaging modalities. From left to right images, there is fat accumulation; to be specific, the left-most figure is a normal liver, and the others have progressive steatosis from early to late stages. All the imaging methods tend to show gradual changes from left to right. The histology may be the most accurate, but it cannot be acquired without sacrificing the target animals. In the upper row, the traditional B-mode has been used for steatosis diagnosis, and it is known that steatosis produces brighter B-mode echoes than normal livers, but the subtle changes between early to late stages are not easily assessed as demonstrated in the images in the second to fifth columns. In the second row, the H-scan shows increase in blue colors as steatosis progresses, which is possibly more noticeable compared to the subtle changes of B-mode. However, for the use of H-scan for steatosis, clinicians should have a reference value for the blue levels for each stage. Lastly, by incorporating multiple parameters, the DSI method can display the yellow highlighted images with diseased area segmentation. The DSI for the normal liver in the first column does not show any yellow, and the segmented yellow area increases as steatosis progresses. Users will know how much fatty tissue is distributed within the scanned target, similar to the gold standard histology.

Therefore, our proposed DSI method utilizing the SVM (and inner product) enables the visualization of steatosis progression in an easier way and with better performance compared to other imaging methods.

V. Discussion

We note that the general approach utilized within DSI incorporates the well-established classification approach of utilizing a training set and then a testing set. In our case, the training sets were somewhat unusual in that they were acquired from three different institutions, three different preclinical models of liver pathology, and three different ultrasound scanners. However, with proper standardization of the multiparametric measures, these were all incorporated into a common 3D parameter space that is represented by Fig. 2. Once established, all results from new studies within the diseases used in training can be assessed as demonstrated in Fig. 6 with a final visual representation of DSI as shown in Fig. 3 and Fig. 8 (d). It can be noted that the traditional graphing of results, shown in Fig. 5, can be confusing as more parameters are measured. However, the DSI framework incorporates the information within the SVM classification and then can label the B-mode images with color overlays as shown in Fig. 3 and Fig. 8 (d), simplifying interpretation. With this small testing group, the overall correlation between DSI classification of steatosis and histological measurement of vacuoles was over 0.8, indicating a strong agreement. In our recent publications, we demonstrated the high performance of the DSI applied to the three disease models utilizing the ultrasound images acquired by the three ultrasound scanners at three different laboratories [40]. Moreover, with the proper standardization methods, our approach can be applied to other disease models and ultrasound scanners.

This study was performed to assess the proposed DSI approach for steatosis, and therefore assessments for fibrosis, PDAC metastases, and other diseases remain for future study.

For the steatotic animals in this paper, we investigated livers whose fat accumulation ranged from normal to severe steatosis. The average steatosis volume percentage measured in histology was 20.02 ± 6.90%, and minimum and maximum were 10.70% and 36.06%, respectively. However, the performance for earlier stage (less than 10% steatosis) is still unknown.

In human clinical practice, pathological states can present more complicated combinations of diseases. Further research on these complicated conditions is needed to evaluate how the combination of diseases affects feature trajectories between each single disease axis, and how they can be visualized (for example, mixing colors between different diseases). Another question for comparing human scans with animal scans is whether the lower resolution caused by the lower transmission frequencies will yield the same ability to accurately measure the features to yield precise DSI. To demonstrate DSI’s capability in utilizing ultrasound signals acquired at lower transmission frequencies, further studies of human scans are required. However, there have been studies extracting features from human scans, including H-scan and quantitative ultrasound, which show good performance [6, 7, 49]. Therefore, we expect that the DSI method can be applied to human scans.

Our DSI approach was developed utilizing ultrasound echoes and the H-scan analysis, and then we demonstrated that DSI was able to segment steatosis areas with strong agreement with histological measurements. However, the proposed DSI method has the potential to be utilized with higher accuracy and with multiple imaging modalities and features. First, training of DSI was performed using a dataset acquired from different ultrasound scanners as previously mentioned, which can be more challenging than using only one scanner to incorporate different disease models since scanners have different ultrasound beam properties, such as transmit frequency and bandwidth. When building a commercial DSI application for an ultrasound device, training would ideally be performed using signals acquired by a single platform, and then the multiparametric analysis in DSI could be simpler and more reliable since training with a single platform does not require the standardization described in Section III.C.2-3. It can be challenging to standardize normal livers’ brightness and % blue. Second, pixels can be falsely classified by DSI as shown in Table I, although the errors are less than 1%. These errors might be lower if we could train DSI with ultrasound signals obtained from the same scanner thereby excluding the standardization procedures. Lastly, the DSI method is not limited to ultrasound imaging or H-scan analysis. It can be applied to any imaging modality that can estimate parameters examining tissue characteristics. Moreover, it can be applied to ultrasound signals using features other than the H-scan analysis. If features can be extracted at close to pixel-level resolution, the rest of the procedures are identical to this study’s ultrasound and H-scan-based DSI.

Some questions remain for further research. (1) Limitations on the spatial resolution of DSI classification have not been specified; currently the resolution is limited by requirements of the measurement of attenuation. Related to this is the tradeoff between accuracy of parameters and spatial support of the estimator; finer spatial resolution in estimated parameters is gained at the expense of higher variation in the estimated parameters, so an optimal combination remains to be quantified. (2) The upper limit on the number of unique pathological states (diseases) that can be classified with DSI is not known. Generally, the more independent parameters that can be included for analysis, the higher the feature dimension, and then less overlap between different diseases are anticipated. However, this question still entails both mathematics and biology. The analytical side involves all the issues surrounding how many classified regions and SVM boundaries can be reasonably determined with N measured parameters and some nominal variability in measurements. The biological side of the question relates to how different pathologies will influence and separate the N measured parameters, and to how much variability between patients (with the same biopsy grade or score) will be expected in the clinic. Together, these factors will ultimately set a practical limit for DSI classifications. However, the expectation is that as more independent measured parameters are enabled and more additional information is added from patient records, the upper bound may increase as the integration of relevant measures continues. (3) Further work for real-time implementation is desirable only for prediction purposes. The proposed approach requires training and prediction for liver diagnosis using the trained DSI. It is not necessary for the training to be conducted in real-time. Though this study didn’t implement it in real-time, we use the SVM, which is computationally less extensive than other machine learning and deep learning approaches.

VI. Conclusion

We developed the DSI technique to segment diseased tissues and highlight them with colors overlaid on B-mode. Features were first measured based on ultrasound physics, which form unique clusters specific to diseases in multiparametric space. Disease axes can be found from the clusters, and SVM constructs hyperplanes setting boundaries that can differentiate the diseases in the feature spaces. Lastly, a mathematical approach, including the inner product, incorporates the multiparametric and machine learning results, producing a color display of disease severity. The accuracy of DSI applied to a preclinical steatosis model was evaluated, yielding strong agreement with histology for segmentation of fat fraction. Therefore, DSI has the potential to be used to classify areas of steatosis, fibrosis, tumors, or other conditions, enabling better diagnosis. Furthermore, while the current study was performed using ultrasound imaging, the DSI framework has the potential to be applied to other imaging modalities which can provide quantitative multiparametric features.

Biography

Jihye Baek (Graduate Student Member, IEEE) received the bachelor’s degree in Physics from Ewha Womans University, Seoul, Korea, from 2005 to 2008. She received the master’s degree in Medical Science from Ewha Womans University, Seoul, Korea, in 2011, with a focus on medical imaging. She is currently pursuing the Ph.D. degree in electrical and computer engineering from University of Rochester, NY, USA.

From 2011 to 2013, she was a senior researcher at Samsung Medison, Seoul, Korea, with a focus on ultrasound imaging and imaging optimization. Her current research interests include medical imaging, quantitative ultrasound, and machine learning for tissue classification.

Lokesh Basavarajappa (Member, IEEE) received the bachelor’s and master’s degrees from Visvesvaraya Technological University, Belgaum, India, in 2012 and 2014, respectively, and the Ph.D. degree in applied mechanics (Biomedical Engineering Group), Indian Institute of Technology Madras, Chennai, India, in 2018. He is currently a Postdoctoral Researcher with the Department of Bioengineering, University of Texas at Dallas, Richardson, TX, USA. His research interests include biomedical ultrasound, quantitative ultrasound, ultrasound elastography, ultrasound therapy, and photoacoustic imaging.

Kenneth Hoyt (Senior Member, IEEE) received the B.S. degree in electrical engineering from Drexel University, Philadelphia, PA, USA, in 2001, and the M.S. and Ph.D. degrees in biomedical engineering in 2004 and 2005, respectively, from the same institution. He did a postdoctoral fellowship with the Department of Electrical and Computer Engineering, University of Rochester, Rochester, NY, USA. He was a Faculty Member with the Department of Radiology, University of Alabama at Birmingham (UAB), Birmingham, AL, USA, from 2008 to 2015. During this tenure, he also received an M.B.A. degree from the UAB School of Business in 2011. He is currently an Associate Professor with the Department of Bioengineering, University of Texas at Dallas, Richardson, TX, USA. His research focuses on the development of novel ultrasound technologies for improved human disease management (e.g., cancer and diabetes). Dr. Hoyt was an elected fellow of the American Institute of Ultrasound in Medicine (AIUM) in 2014.

Kevin J. Parker (Life Member, IEEE) is the William F. May professor of engineering at the University of Rochester. He earned his graduate degrees from Massachusetts Institute of Technology and served at the University of Rochester as department chair, director of the Rochester Center for Biomedical Ultrasound, and dean of engineering/applied sciences. He holds 26 US and 13 international patents (licensed to 25 companies), is founder of VirtualScopics, and has published 250+ journal articles. He is fellow of the IEEE, AIUM, ASA, AIMBE, and NAI.