A New Multi-Model Machine Learning Framework to Improve Hepatic Fibrosis Grading Using Ultrasound Elastography Systems from Different Vendors

Abstract

The purpose was to show that the diagnostic performance of point and two-dimensional shear wave elastography (pSWE; 2DSWE) using shear wave velocity (SWV) with a new machine learning (ML) technique applied to systems from different vendors is comparable to MR elastography (MRE) in distinguishing non-significant (<F2) from significant (≥F2) fibrosis. We included two patient groups with liver disease: 1) 144 patients undergoing pSWE (Siemens) and MRE; and 2) 60 patients undergoing 2DSWE (Philips) and MRE. Four ML-algorithms using ten SWV measurements as inputs were trained with MRE. Results were validated using two-fold cross validation. The performance of median SWV to binary grade fibrosis was moderate for pSWE (AUC: 0.76) and 2DSWE (0.84); the ML-algorithm support vector machine (SVM) performed particularly well (pSWE: 0.96; 2DSWE: 0.99). The results suggest that the multi-vendor ML-based algorithm SVM can binary grade liver fibrosis using USE with excellent diagnostic performance, comparable to MRE.

INTRODUCTION

Chronic liver disease, caused by hepatic injury from various etiologies, is a crucial global health problem with rising incidence. Precise disease staging is paramount for patient management, treatment recommendations, and accurate prognosis (Ferraioli et al. 2015). Liver biopsy has classically been the gold standard for fibrosis staging; however, non-invasive imaging methods, such as transient elastography (Fibroscan), point shear wave elastography (pSWE), two-dimensional shear wave elastography (2DSWE), and magnetic resonance elastography (MRE), have been shown to be at least as accurate with less complications (Lurie et al. 2015; Afdhal et al. 2015; Zhang et al. 2019). 2DSWE and pSWE provide liver stiffness information using acoustic radiation force impulses (ARFI) (Friedrich-Rust et al. 2012), and MRE uses an external passive driver to generate hepatic shear waves that are imaged by MRE pulse sequences (Trout et al. 2016). MRE has been shown to be highly reproducible and accurate for liver stiffness measurement (Cui et al. 2016), as has ultrasound elastography (Bota et al. 2012; D’Onofrio et al. 2010; Rizzo et al. 2011), although with somewhat lower accuracy (AUC: pSWE 0.81; 2DSWE 0.88 (Sigrist et al. 2017); MRE >0.9 (Shi et al. 2014)). Ultrasound elastography is cheaper than MRE and widely used in clinics; nonetheless, it lacks an ideal sensitivity and specificity in grading liver fibrosis (Sigrist et al. 2017), which can negatively influence patient care. Furthermore, ultrasound elastography cutoff values for grading liver fibrosis based on velocity or stiffness values vary among manufacturers (Sigrist et al. 2017; Ferraioli et al. 2018); thus, results are not interchangeable from one system to another. In addition, studies with the necessary population size to define or improve these cutoff values are becoming harder to conduct due to the lack of gold standard biopsies being performed. There is a critical need for robust cutoff values for standardized hepatic fibrosis grading, which can be applied to all systems and diseases (Dietrich et al. 2017). The literature has raised this as an important unaddressed concern (Sigrist et al. 2017).

In recent years, machine learning (ML) approaches in diagnostic radiology have emerged and gained prominence (Erickson et al. 2017). Prior studies incorporating machine or deep learning algorithms sought to improve liver fibrosis grading with ultrasound elastography (Chen et al. 2017; Fujimoto et al. 2013; Gatos et al. 2017; Stoean et al. 2011). Nonetheless, there is no published literature that assessed an ML technology for characterizing liver fibrosis using ultrasound elastography velocity measurements obtained with pSWE and 2DSWE to train and validate a scoring system that is comparable to MRE for grading liver fibrosis, which can also be applied to systems from different vendors.

Therefore, the purpose of this study was to show that the diagnostic performance of pSWE and 2DSWE for grading liver fibrosis using SWV with a new ML technique is comparable to MRE in distinguishing non-significant (<F2) from significant (≥F2) fibrosis and can be applied to ultrasound systems from different vendors.

MATERIALS AND METHODS

This HIPPA-compliant retrospective study was approved by the Institutional Review Board, and the requirement for written consent was waived for all participating patients. Exclusion criteria were non-diagnostic MRE and unreliable ultrasound elastography with an IQR divided by the median (IQR/Median > 0.3). Figure 1 summarizes the study design.

Figure 1.

Flow diagram of the enrollment process in this retrospective study.

Patient population

Group 1:

From April 2014 to February 2017, 169 ultrasound elastography exams (pSWE) were performed (86 men; mean age: 53.8 years, range: 23–75 years; 80 women; mean age: 56.9 years, range: 22–80 years) in patients who also underwent an MRE examination within 12 months (this timeframe was decided upon based on discussions with hepatologists from our institution as well as evidence in the literature (Pan et al. 2018). 25/169 (14.8%) were excluded due to unreliable exams. All enrolled patients (144/144; 100%) had known chronic liver disease or elevated liver enzymes (Table 1).

Table 1.

Distribution of diseases between the two datasets.

Diagnosis	pSWE + MRE	2DSWE + MRE
Hepatitis B	50	4
Hepatitis C	41	10
Nonalcoholic fatty liver disease or steatohepatitis	19	23
Abnormal liver function studies	13	4
Alcohol abuse and alcoholic cirrhosis	7	7
Primary biliary cholangitis	6	1
Hemochromatosis cirrhosis	3	3
Cryptogenic cirrhosis	2	2
Autoimmune hepatitis		1
Drug induced hepatitis		2
Budd Chiari syndrome	1
Morbus Wilson cirrhosis	1
Cardiac cirrhosis	1
Portal/mesenteric vein thrombosis		1
No known chronic liver disease		2

Group 2:

From February 2016 to October 2017, 63 ultrasound elastography exams (2DSWE) were performed (39 men; mean age: 53.9 years, range: 23–79 years; 24 women; mean age: 55.4 years, range: 22–73 years) in patients who underwent an MRE examination (median interval: 0 days; mean interval: 1 day). 3/63 (4.8%) were excluded due to a non-diagnostic MRE. Chronic liver disease was known in 58/60 (96.7 %) of the enrolled patients (Table 1).

Ultrasound Elastography Image Acquisition

PSWE was performed in patients in Group 1 with the Virtual Touch Tissue Quantification (VTTQ ™) mode on a clinical ultrasound scanner (Acuson S2000™; Siemens Medical Solutions, Mountain View, CA) coupled to a curved array transducer (6C1 HD; Siemens Medical Solutions). Philips 2DSWE (Group 2) was performed using the prototype ElastQ software on an Epiq7 system coupled to a curved array transducer (C5–1; Philips Healthcare, Amsterdam, Netherlands).

Patients were asked to fast for at least 4 hours prior to ultrasound imaging. SWV measurements of the liver were performed in group 1 by one of three sonographers with dedicated training in pSWE and in group 2 by one sonographer with dedicated training in 2DSWE. Patients were placed in the supine position and the right arm was elevated above the shoulder to widen the intercostal space. The regions of interest (ROI; Siemens: 10 × 6 mm; Philips: 0.785 cm²) were placed in liver segment 8 (Figure 2). Ten consecutive SWV measurements (in m/s) were obtained from approximately the same location within 2 cm of Glisson’s capsule and perpendicular to the liver capsule, without including large vessels or dilated bile ducts. Patients were asked to keep breath-holding at a neutral position during measurements.

Figure 2.

Ultrasound elastography images of the liver in segment 8 (transverse plane) obtained in the two different groups. a. Point shear wave elastography (pSWE) on a Siemens scanner in group 1 (51-year old female patient with abnormal liver function studies); b. 2-dimensional shear wave elastography (2DSWE) on a Philips scanner in group 2 (42-year old male patient with chronic hepatitis B).

The results of all 10 measurements were automatically displayed by the systems at the end of the exam and either saved into the clinical picture archiving and communication system (PACS; Centricity; GE; Group 1) or on an external hard drive disk (Group 2).

Magnetic Resonance Elastography Imaging Acquisition

Patients were instructed to fast for 4 hours prior to the MRE examination. All MR elastography examinations were performed on a 3T MR magnet (GE750, GE Healthcare, Waukesha, Wisconsin) using a 32-channel torso phased-array receive coil, with a passive driver placed on the patient’s right upper abdomen to allow the transmission of 60 Hz vibrations into the liver and a 2D phase-sensitive echo-planar MR elastography sequence (MR-Touch, GE Healthcare). The sequence was acquired in a single expiratory breath hold (~20 sec) with the passive driver activated. A direct inversion algorithm automatically created shear wave images and stiffness maps from the acquired data. Radiologists drew an ROI encompassing areas of the right hepatic lobe assessed to have reliable signal, measuring liver stiffness (complex shear modulus) in kilopascals.

Shear Wave Velocity Based Grading and Statistical Analysis

When the 10 SWV measurements demonstrated an IQR divided by the median (IQR/median) >0.3, they were considered unreliable and excluded from the study (Ferraioli et al. 2015): Group 1, 25/169 (14.8%); and group 2, 0/63 (0%). Non-diagnostic MRE were excluded from the study (Group 1, n=0; Group 2, n=3). Liver fibrosis was binary classified as clinically non-significant (<F2) or significant (≥F2) based on stiffness values for MRE with a published cutoff of 3.5 kPa (Venkatesh and Ehman 2014); and for ultrasound elastography based on median SWV using a cutoff value for Siemens of 1.34 m/s (Friedrich-Rust et al. 2012). At the time the current study was performed, Philips has not yet provided a published reference table for the just recently released ElastQ software to grade fibrosis.

For Siemens data, the accuracy of median SWV using the published cut-off value of 1.34 m/s with respect to MRE-based fibrosis grading was calculated. Essentially, median SWV from USE using a cut-off value of 1.34 m/s for Siemens divides the dataset into clinically significant and clinically non-significant fibrosis, while MRE using a cut-off of 3.5 kPa also divides the dataset into clinically significant and clinically non-significant fibrosis, and the accuracy of USE compared to MRE for this determination was compared. However, as Philips does not yet have a published cut-off value for clinically significant fibrosis, for both groups (Siemens and Philips) the performance of median SWV velocity measurements with respect to MRE-based binary fibrosis grades (true labels) was performed using a receiver-operating characteristic (ROC) curve analysis. Hence, for both groups (in a technique that thus does not rely on a published cut-off value) median SWV measurements and MRE-based binary fibrosis grades were input to the Matlab perfcurve function to generate a receiver-operating characteristic curve and calculate area-under-the-curve (AUC).

Machine Learning Based Grading and Statistical Analysis

Figure 3 summarizes the ML approach to binary hepatic fibrosis grading using the two groups: 1) pSWE + MRE and 2) 2DSWE + MRE. Four supervised ML algorithms were applied that are commonly used in literature (Erickson et al. 2017): generalized linear regression model (Dobson 1990), naïve Bayes (Hastie et al. 2009), quadratic discriminant analysis (Guo et al. 2007), and a nonlinear support vector machine (Schölkopf and Smola 2002).

Figure 3.

The proposed multi-model framework for machine learning (ML) based fibrosis staging. This approach will provide a fibrosis staging between 0 to 100 regardless of vendor. In this work we only tested ultrasound elastography shear wave velocity (USE SWV) measurements obtained using Siemens and Philips scanners, with magnetic resonance elastography (MRE) as ground truth. However, in the future this model could be extended to other vendors after additional training and validation on those datasets.

Logistic regression, which falls under the category of generalized linear models, is a commonly used statistical technique that can be used to predict a categorical outcome value, most commonly binary, given a set of predictor values. If the positive event is coded as “1” and the negative event is coded as “0” then binary logistic regression provides the log-odds of the outcome being positive given the predictor values. It uses the logit or sigmoid function, $f\left(t\right)=\frac{1}{1+e^t}$, which represents the log-odds of observing the positive event. In this case, $t={\beta }_0+{\beta }_1{X}_1+\cdots +{\beta }_k{X}_k$, where the values β represent the model parameters to be optimized and the vector X represents the input data (Dobson 1990; Erickson et al. 2017).

A Naïve Bayes classifier uses the idea of prior or previous probabilities, derived from previous outcomes, and applies Bayes theorem, which determines the probability of an event occurring taking advantage of these nown prior probabilities. The classifier takes each outcome, such as “1” versus “0” in the binary case, and selects the one with the highest probability (Hastie et al. 2009; Erickson et al. 2017).

Linear discriminant analysis, in practical terms, seeks to discriminate between two groups. It does this by minimizing the distance between data points from the same class, while maximizing the distance between data points from different classes. Quadratic discriminant analysis is a variation of the aforementioned technique, in which a “pseudo-quadratic” transformation is applied to the data. While linear discriminant analysis naturally allows a linear decision boundary between classes, quadratic discriminant analysis allows quadratic equations to represent that decision boundary. Hence, while quadratic discriminant analysis allows for greater flexibility in the decision boundary, it requires more parameters to be calculated (Guo et al. 2007; Erickson et al. 2017).

A support vector machine seeks to discriminate between classes by mapping each data point into a higher dimensional space and creating an optimal separating hyperplane that maximizes the distance between each data point and that hyperplane, maximizing the differentiation between each class. This mapping into higher dimensional space is accomplished by a kernel, and the particular kernel used in this study was the Gaussian radial basis kernel, which performs well with high-dimensional data (Schölkopf and Smola 2002; Erickson et al. 2017). We also used auto scaling with a Box Constraint of 1.

The ten measurements of shear wave velocity served as inputs to these ML algorithms, and their accuracy for binary hepatic fibrosis grading was assessed. Two-fold cross-validation was performed; i.e half of the data for training and half for testing and vice-versa (Hastie et al. 2009). During each run, the group 1 training dataset was used to train model 1 with MRE (Siemens-Model; Figure 3), and the Matlab predict function applied this model to the validation data and output a score representing the likelihood that the label came from each class, either clinically non-significant or significant fibrosis. The Matlab perfcurve function then used these scores and true class labels (from all data) to generate ROC curves to calculate AUC, sensitivity, specificity, positive and negative predictive value, and accuracy. Next, the group 2 dataset (Philips) was similarly used to train the Philips-Model with MRE.

To show that the improvement in AUC between the ML algorithm and median SWV is statistically significant, we performed the DeLong test.

All statistical analyses were performed in Matlab R2015b (MathWorks, Natick, MA).

RESULTS

Using the current clinically established standard of care (SOC) cutoff value for binary fibrosis grading for Siemens pSWE (group 1), median SWV measurements performed only fair compared to MRE with 60.4% accuracy (Table 2). Note, that SOC versus MRE analysis was not performed in group 2 due to the lack of a published cutoff table for Philips 2DSWE. Next, using the median of 10 consecutive SWV measurements in an analysis employing a ROC curve for both groups, the performance of binary fibrosis grading was moderate for the pSWE (AUC 0.76) and 2DSWE dataset (AUC 0.84) (Table 3 and 4; Figure 4).

Table 2.

Performance of the median shear wave velocity out of 10 measurements in predicting clinically non-significant versus significant fibrosis using a standard of care (SOC) cutoff value of 1.34 m/s for the point shear wave elastography (pSWE) dataset (Friedrich-Rust et al. 2012) compared to the reference standard magnetic resonance elastography (MRE) as well as machine-learning (ML) based staging (MRE-equivalent) for group 1.

pSWE	SOC vs. MRE	SOC vs. ML
Sensitivity	81.6	82.5
Specificity	49.5	47.1
NPV	83.9	87.5
PPV	45.5	37.5
Accuracy	60.4	56.9

Table 3.

Performance of each machine learning algorithm as well as median shear wave velocity in predicting clinically non-significant versus significant fibrosis in the Group 1 dataset (pSWE).

Classifier	Sensitivity	Specificity	NPV	PPV	Accuracy	AUC	p-value
Median SWV	71.4	71.6	82.9	56.5	71.5	0.760	3.36E-07
GLRM	77.1	70.5	85.9	56.9	72.7	0.808	1.87E-09
Bayesian	71.4	76.8	83.9	61.4	75.0	0.776	5.88E-08
QDA	77.1	70.5	85.9	56.9	72.7	0.821	4.16E-10
SVM	81.3	94.7	90.9	88.6	90.2	0.962	1.93E-19

Note that sensitivity and specificity represent different points on the receiver operating characteristic (ROC) curve. P-values were calculated using a Wilcoxon rank-sum test.

NPV = negative predictive value, PPV = positive predictive value, AUC = area-under-the-curve, GLRM = generalized linear regression model, QDA = quadratic discriminant analysis, SVM = support vector machine, SWV = shear wave velocity.

Table 4.

Performance of each machine learning algorithm as well as median shear wave velocity (without cutoff value) in predicting clinically non-significant versus significant fibrosis in Group 2 dataset (2DSWE).

Classifier	Sensitivity	Specificity	NPV	PPV	Accuracy	AUC	p-value
Median SWV	73.7	100.0	89.1	100.0	91.7	0.841	2.54E-05
GLRM	84.2	75.6	91.2	61.5	78.3	0.858	1.16E-05
Bayesian	78.9	80.5	89.2	65.2	80.0	0.886	1.60E-06
QDA	78.9	80.5	89.2	65.2	80.0	0.881	2.55E-06
SVM	89.5	100.0	95.4	100.0	96.7	0.987	1.61E-09

Note that sensitivity and specificity represent different points on the receiver operating characteristic (ROC) curve. P-values were calculated using a Wilcoxon rank-sum test.

NPV = negative predictive value, PPV = positive predictive value, AUC = area-under-the-curve, GLRM = generalized linear regression model, QDA = quadratic discriminant analysis, SVM = support vector machine, SWV = shear wave velocity.

Figure 4.

Receiver operating characteristic (ROC) curves compare the performance of each machine learning (ML) algorithm and the baseline technique using median shear wave velocity to predict clinically non-significant versus significant liver fibrosis, as determined by MRE as gold standard. Support vector machines (blue) had the highest performance of all ML algorithms in both groups.

Next, performance was assessed using the four ML algorithms, with shear wave velocity measurements as inputs and binary fibrosis grading as determined by MRE as the gold standard (Table 3 and 4; Figure 4): support vector machine demonstrated the highest level of performance of the ML algorithms in binary fibrosis grading with an AUC of 0.96 for the pSWE dataset and 0.99 for the 2DSWE dataset. For the 2DSWE dataset, quadratic discriminant analysis achieved an AUC of 0.88. The other ML-based algorithms either reached the same or slightly higher AUC values than median SWV in both datasets.

Most notably, the difference in AUC between median shear wave velocity and SVM was statistically significant for both Siemens and Philips, although the p-value was better for Siemens as it had a larger sample size (Table 5).

Table 5.

Difference in AUC between the ML algorithm SVM and median SWV was statistically significant for both groups.

	p-value	Significantly Different
Siemens - Median SWV vs. SVM	4.95E-05	Yes
Siemens - Median SWV vs. QDA	0.19098	No
Siemens - Median SWV vs. Bayesian	0.46593	No
Siemens - Median SWV vs. GLRM	0.19098	No
Philips - Median SWV vs. SVM	0.036085	Yes
Philips - Median SWV vs. QDA	0.32787	No
Philips - Median SWV vs. Bayesian	0.22957	No
Philips - Median SWV vs. GLRM	0.71877	No

AUC = area-under-the-curve, GLRM = generalized linear regression model, QDA = quadratic discriminant analysis, SVM = support vector machine, SWV = shear wave velocity.

Significantly Different = p-value < 0.05.

Analyzing score separation between non-significant and significant hepatic fibrosis, median SWV demonstrated worse score separation between the two classes (Figure 5). Using the ML-based algorithms, especially support vector machines, there was improved binary score separation for both datasets (Figure 5).

Figure 5.

Scores for non-significant and significant fibrosis separation using median shear wave velocity (SWV) as well as the new machine learning (ML) algorithms in dataset 1 (a. pSWE), and dataset 2 (b. 2DSWE). The different scores reflect the likelihood that the label came from each class (non-significant or significant fibrosis). Boxplots show excellent score separation in both datasets when a support vector machine (SVM) is used to perform classification, compared to worse score separation with median SWV. Note that ML scores differ between systems from different vendors as well as for the different ML algorithms.

MRE = magnetic resonance elastography; GLRM = generalized linear regression model; QDA = quadratic discriminant analysis. The ends of the box are the upper and lower quartiles; the vertical line inside the box represents the median; and the whiskers extend to the highest and lowest values.

DISCUSSION

In our study, the ML algorithm SVM outperformed median SWV in distinguishing between non-significant and significant hepatic fibrosis, with a diagnostic performance similar to MRE-based fibrosis grading. The ML-based algorithm SVM had excellent diagnostic performance in datasets acquired from Siemens and Philips systems; this result is despite the fact that these two vendors used different elastography techniques.

We used ML in elastography by analyzing 10 shear wave velocity measurements obtained with systems from two different vendors as inputs and then training the algorithm with MRE; our study is the first that assessed ML for characterizing liver fibrosis using pSWE and 2DSWE data from different vendors.

A known issue with ultrasound elastography exams is the problem of variability of the 10 measurements that might be due to tissue properties (the higher the liver damage, the higher the variability), operator performance, and/or device precision. Currently, median shear wave velocity is used to grade liver fibrosis (Dietrich et al. 2017). One advantage of ML is that it is able to capture information about the data beyond just the median. Future studies need to be conducted to use ML to analyze data from spatial samples (elasticity maps from 2DSWE) versus temporal samples (10 consecutive measurements, as in our current study). Analyzing spatial data from a single elasticity map would minimize operator dependency, by decreasing the number of maps to be acquired, and reduce scanning time; this would also better account for the heterogeneity of the liver tissue, especially in fibrotic/cirrhotic patients.

In recent years, ML has been further developed and used increasingly for imaging data analysis, including liver elastography. A prior study performed automatic fibrosis staging in hepatitis C patients using multivariate linear regression that characterized texture features derived from color maps from real time elastography (Fujimoto et al. 2013). There is also published literature on ML approaches using elastography in other organs, such as for breast cancer diagnosis (Zhang et al. 2016).

Our study has several limitations. First, the sample size was small and different in both groups – nonetheless, we confirmed that the difference in AUC between the ML algorithm SVM and median SWV was statistically significant; future studies with more patients are warranted. Second, we trained, tested and validated the ML-based algorithm on systems from only two vendors; systems from other vendors need to be addressed in future studies. Third, our ‘study gold standard’ was MRE; ideally our results would be confirmed in a study with a pathology-trained ML-algorithm, although this would be challenging given the low number of patients who undergo liver biopsy at most institutions.

CONCLUSION

In summary, the new machine-learning based algorithm to grade liver fibrosis into clinically non-significant and significant categories with two different ultrasound elastography techniques from two vendors demonstrates excellent diagnostic performance, comparable to magnetic resonance elastography. The machine learning algorithm—support vector machines—outperformed median shear wave velocity. With additional validation in larger studies, this ML-based algorithm, along with a scoring system, might ultimately be included in routine ultrasound screening protocols of the liver for improved liver fibrosis grading, especially in the large patient population with chronic liver disease, without extending the acquisition time. The algorithm, along with a scoring system, could be integrated into the software of clinically established ultrasound elastography systems from different vendors after being trained and validated for each of these vendors. The scoring system would have the same cutoff for differentiating non-significant from significant fibrosis in systems from all vendors and would provide comparable fibrosis staging, thus abrogating the need for establishing and implementing a different reference table for each vendor.