Automated Measurements of Muscle Mass Using Deep Learning can Predict Clinical Outcomes in Patients with Liver Disease

Abstract

Objective:

There is increasing recognition of the central role of muscle mass in predicting clinical outcomes in patients with liver disease. Muscle size can be extracted from computed tomography (CT) scans but clinical implementation will require increased automation. We hypothesize that we can achieve this by using artificial intelligence.

Design:

Using deep convolutional neural networks, we trained an algorithm on the Reference Analytic Morphomics Population (n=5268) and validated the automated methodology in an external cohort of adult kidney donors with a non-contrast CT scan (n=1655). To test the clinical usefulness we examined its ability to predict clinical outcomes in a prospectively followed cohort of patients with clinically diagnosed cirrhosis (n= 254).

Results:

Between the manual and automated methodology, we found excellent inter-rater agreement with an intraclass correlation coefficient of 0.957 (CI 0.953–0.961, p <0.0001) in the adult kidney donor cohort. The calculated dice similarity coefficient was 0.932 ± 0.042, suggesting excellent spatial overlap between manual and automated methodology. To assess the clinical usefulness, we examined its ability to predict clinical outcomes in a cirrhosis cohort and found that automated psoas muscle index was independently associated with mortality after adjusting for age, gender and Child’s classification (p <0.001).

Conclusion:

We demonstrate that deep learning techniques can allow for automation of muscle measurements on clinical CT scans in a diseased cohort. These automated psoas size measurements were predictive of mortality in patients with cirrhosis showing proof of principal that this methodology may allow for wider implementation in the clinical arena.

Introduction

Computed tomography (CT) scans are important diagnostic tools which are used routinely in the care of patients with liver disease. Beyond the scan’s targeted query (e.g. the presence of malignancy or a biliary obstruction), is a multitude of other data. One important example is muscle size and quality. Numerous studies have shown that muscle mass is significantly decreased in patients with chronic liver disease, especially those with endstage liver disease awaiting liver transplantation^1–9. Notably, quantitative measures of muscle mass on CT scans, such as total psoas area (TPA), are strongly predictive of important clinical outcomes such as infection and mortality^10–27.

Measuring muscle mass in CT especially for psoas muscle typically requires delineation by a human operator, who must manually outline individual muscles. Although multiple software packages are available to assist in this task (e.g. sliceOmatic, ImageJ, OsiriX,²⁸ and our group’s Analytic Morphomics^{15, 28}), all involve manual tracing or correction. The effort required limits clinical applicability due to issues with scalability. Furthermore, this method is fraught with inter- observer variability²⁹. Increased automation is needed to facilitate implementation and integration into clinical care and improve the accuracy and reproducibility of muscle measurements. One approach to automation is to utilize deep learning techniques which have revolutionized the field facial recognition³⁰. We hypothesize that by using convolutional neural networks, we can develop an automated method to measure the psoas muscle, which can then be applied to predict clinical outcomes. We trained the algorithm on a set of CT scans performed for trauma indications. We then validated the algorithm by comparing measurements made by automated vs standard manual method using a cohort of healthy outpatients undergoing CT scans as part of the kidney donor preoperative assessment. To assess clinical utility, we tested the automated method in a cohort of cirrhosis patients prospectively followed in an outpatient Hepatology clinic and had an incidental CT performed for clinical indications. We assessed both the accuracy of the automated measurements as well as its ability to predict survival in patients with cirrhosis.

Methods:

The primary aim of the study was to develop an automated methodology for measuring psoas muscle features on incidental CT scans. A secondary and exploratory aim was to assess the clinical associations of psoas muscle measurements and to determine if it could predict survival in patients with cirrhosis. To develop the automated technology, we utilized the manual delineation of the psoas muscle as the ground truth to train an automated model using deep convolutional neural network, a technology routinely used for facial recognition software. After developing the algorithm, we tested it on CT scans from 2 different clinical cohorts (healthy and diseased) which were not included in the original training cohort. In the testing, we compared the performance of the automated model with manual method (ground truth) by assessing the overlap between the two images and the correlation in measurements.

Training Cohort:

For the training of the model, we utilized the previously published Reference Analytic Morphomics Cohort of 5,268 patients, aged 16–91 who received CT scans (chest, abdomen and pelvis) for trauma indications between 04/1998–05/2015 at the University of Michigan³¹. For the training, only the ground truth of manual method was utilized.

Testing Cohort 1- Normal Kidney Donors:

To test the automated algorithm on a cohort of CTs from a healthy population, we utilized a retrospective cohort of 1,655 patients (Aged 18–68) who had CT scans at the University of Michigan as part of evaluation for kidney donation between 1999–2011 who had a CT scan which included L4^{32, 33}.

Testing Cohort 2- Patients with cirrhosis:

Because of concerns that the automated algorithm would not perform well in a cohort of patients with less muscle mass such patients with cirrhosis, we tested the automated algorithm on a cohort of patients followed from 2010–2015 at the University of Michigan Hepatology Clinic for cirrhosis who were enrolled prospectively in chronic disease monitoring system.³⁴ All 254 included patients received an abdominal CT within 365 days of enrollment with manual psoas muscle measurements. Cirrhosis was defined clinically by a board certified hepatologist at the University of Michigan Health System using a combination of imaging, laboratory and/or histological parameters. All demographic and clinical details were abstracted from the chart review with baseline data was obtained within 6 months of the CT date. Death was confirmed with the social security death index³⁵. This study was conducted with the approval of the Institutional Review Board at the University of Michigan Health System.

Manual method for measuring psoas muscle:

For the manual processing of the psoas muscle, we utilized our locally developed image processing system (Analytic Morphomics) as a platform to load the CT scan data and also to choose the slice of interest for psoas measurement. The general methodology of analytic morphomics has been previously described by our group^{15, 36–38}. Briefly, de-identified DICOM (Digital Imaging and Communications in Medicine) files of the CT scans were loaded into the analytic morphomics database. A semi-automated high throughput methodology with algorithms programmed in MATLAB® (MathWorks Inc., Natick, MA) was used to identify the spinal vertebral levels. With this method, we were able to accurately and consistently identify the slice of interest at the bottom of the lumbar 4 (L4) spinal level for further delineation and tracing of the psoas muscle boundaries. Manual outlining of the psoas muscle was facilitated by the use of a “smart string” whereby a trained research analyst selects serial points along the muscle edge and a provisional boundary is generated by our customized software for approval by the analyst. Depending on the condition of the muscle, this can be a simple to tedious process lasting 5 to 20 minutes per study subject scan. All the research analysts undergo a training module and testing prior to initiating any work so that accuracy and consistency of their work is assured. Multiple trained research analysts are used for this initial process and who worked on what cohort was random. The manually generated muscle boundaries are, however, then inspected and adjusted by two quality control analyst to further improve measurement consistency. The same two research analysts provided quality control for all the test cohorts. Once the quality control has been completed, the results are considered our gold standard or ground truth. In internal testing of our quality assurance process, the concordance of results from repeated measures on the same set of scans has been greater than 0.99.

Automated Processing Algorithm Development:

To develop an automated methodology to measure psoas muscle area, we utilized deep convolutional neural networks (CNN) with the MatConvNet package in MATLAB (Mathworks, Inc). Our algorithm consisted of automatically identifying a region adjacent to the spinal canal at L4. The CT slice is then used to create two channels, by creating scaled volumes from −1 to 1 after windowing based on HU thresholds. The first channel takes a window of 0 HU to 100 HU and creates a volume scaled −1 to 1, the second uses a wider window range of −200 HU to 300 HU. The deep learning algorithm takes the two channel input and uses four layers to down sample and build a network. Another four layers are used to up sample back to the original resolution and produce a probability map. Post-processing is employed to improve the segmentation by tracing the edges of the muscle near the boundaries of the output from the neural network.

Statistical Analysis:

To assess the performance of the automated algorithm compared to the manual method, we utilized intraclass correlation (ICC), Spearman’s correlation, Bland-Altman plot, and Passing-Bablok Regression to determine the differences between measurements. The Bland-Altman plot graphs differences between two measures vs average of two measures. Spearman’s correlation and Passing-Bablok Regression were also used as they are based on non-parametric methods, and therefore are more robust to outliers. Intraclass correlation coefficient (ICC) was calculated in the random-effect model framework as defined in Shrout and Fleiss (1979) using ICC(3,1)^{39, 40}. This corresponds to the scenario that two fixed judges rated each target and the absolute agreement was assessed.

To assess for spatial overlap, dice similarity coefficient (DSC) was utilized. Dice similarity coefficient (DSC) was calculated using the formula:

2 (Automated ∩ Manual) / (Automated + Manual), where ∩ is the intersection between the region defined via automated and manual methods, respectively. DSC is a statistical method to analyze the overlap of two images at the pixel level. The values range from 0 to 1 where 0 means there is no overlap and 1 means perfect overlap⁴¹. A DSC of > 0.9 which means that 90% of the pixels overlap is generally considered excellent.

To compare differences of automated muscle measurements among various subgroups, we conducted two-sample Student’s t-test when comparison was for two groups, and analysis of variance (ANOVA) when comparison was for three and more groups. Non-parametric Mann- Whitney U and Kruskal-Wallis tests were also performed for robustness consideration. The Kaplan Meier method was utilized to assess the association of muscle features with survival. Patients were divided into high and low mass mass using the median as the cutoff. In addition, to assess the independent association between muscle measurements and mortality, we utilized a Cox regression controlling for other confounding clinical variables. A priori, we selected the clinical variables which we thought were associated with muscle mass and mortality in this cohort, and adjusted those in the regression model. Considering collinearity among many clinical variables, we presented data controlling for either Child or MELD as they both reflect degree of liver dysfunction and thus could affect muscle size. Other clinical variables considered were etiology, age and gender which had potential to affect muscle mass. Statistical analyses were performed using R 3.6.1 and Stata 12.

Results:

Performance of the automated method on a normal cohort (Test cohort 1)

To test the performance of the learned algorithm, we evaluated it on a cohort of “normal” adult kidney donor candidates who had a non-contrast CT scan (n = 1,655) as part of their clinical evaluation. Visually as noted in Figure 1A, we found that there was significant similarity between the ground truth (manual segmentation) and the CNN generated prediction (automated segmentation). To assess the overall spatial accuracy of the automated method, we calculated the DSC between manual and automated methods and found excellent spatial overlap with a mean DSC of 0.919 ± 0.039 for the total psoas muscle area, TPA (Figure 1B). To test the reliability of the measurements derived from automated vs manual methodology, we examined the ICC for the different measures and found excellent agreement between the manual vs computer measurements (supplementary Table 1). The ICC for TPA was 0.957 (CI 0.953–0.957) with a p < 0.0001 (Figure 1C)

Figure 1.

Normal Kidney Donor Cohort: A) Illustration of psoas muscle in the CT scans of 3 randomly selected patients with the masks of the psoas as noted with the ground truth (manual segmentation) and CNN derived prediction (automated segmentation). B) Density plots of the TPA using the manual (dashed black line) or automated (solid gray line) methods. DSC = 0.919 ± 0.039. C) Correlation of manual vs automated measurements of TPA. ICC = 0.957 (CI 0.953– 0.957, p < 0.0001)

Automated psoas area measurements predict clinical outcomes in patients with liver disease (Test cohort 2)

Recognizing that sarcopenia and quality of muscle decrease in patients with cirrhosis and can affect measurements, we sought to examine the ability of our CNN assisted automated method to quantify psoas muscle measurements in patients with cirrhosis and more importantly, to see if this tool was effective in predicting clinical outcomes. We utilized an outpatient cohort of cirrhosis patients followed in our Hepatology clinic. Descriptions of the cohort are noted in Table 1. The average age was 57 years old with 56% male and 44% female patients. 155 (61%) were decompensated at baseline; 44 (17%), 126 (49.6%), and 73 (28.7%) had history of variceal bleed, ascites, and hepatic encephalopathy at baseline. 121 (47.6%) were Child A and 133 (52.4%) were Child B or C. The average MELD Sodium was 12.7 ± 6.1. Patients were censored at date of last encounter, death or transplantation. The median survival and follow-up time were 5.3 and 5.2 years, respectively. Only 7/254 (2.8%) underwent liver transplantation during the follow-up period. We do not have information about the cause of death but we note that the majority (61%) had history of hepatic decompensation at baseline and 28 % additional patients developed decompensation in the follow-up time. Patients with HCC at baseline were excluded from the cohort but during the follow-up period 25 (9.8%) patients developed HCC.

Table 1:

Description of cirrhosis cohort

Descriptors	Value ± Standard Deviation
Age	57.3 ± 11.3
Gender (M/F)	143 (56.3%)/111 (43.7%)
Body Mass Index (BMI)	30.9 ± 9.6
Liver Disease Diagnosis
HCV	70 (27.6%)
ETOH	52 (20.5%)
NAFLD	87 (33.2%)
Other	45 (17.7%)
AST	64.7 ± 48.0
ALT	49.7 ± 48.4
Platelet	119.5 ± 68.6
Bilirubin	2.3 ± 3.3
Albumin	3.5 ± 0.7
INR	1.26 ± 0.39
Creatinine	1.00 ± 0.85
MELD-Sodium	12.7 ± 6.1
Child A/B/C	121 (47.6%)/ 100 (39.3%)/ 33 (12.9%)
Variceal bleed	44 (17.3%)
Ascites	126 (49.6%)
Hepatic encephalopathy	73 (28.7%)

Within this cirrhosis cohort, we found that our automated methodology performed well with excellent spatial accuracy and reliability. The DSC was 0.917 ± 0.55 (Figure 2A) and ICC was 0.931 (CI: 0.914–0.946, p < 0.0001) (Figure 2B) for TPA when comparing manual with automated methodology. The ICC for all the measurements were more than 0.9 supporting the reliability of using the automated methodology (supplementary Table 1). To further compare automated and manual measurements, we also calculated the psoas muscle index (PMI) which is the TPA adjusted for height (TPA/height squared). Automated and manual PMI were assessed with the Bland-Altman Plot (Supplementary Figure 1a) which was noted to have a bias of −0.34 and upper/lower limit of agreement at 1.46 and −2.14. and Passing-Bablok Regression (Supplementary Figure 1b) which had a slope of 0.99, with 95% confidence interval as (0.95, 1.02). The Spearman’s correlation was calculated at 0.92.

Figure 2:

Cirrhosis Cohort: A) Density plots of the TPA using the manual (dashed black line) or automated (solid gray line) methods. DSC = 0.917 ± 0.55. B) Correlation of manual vs automated measurements of TPA. ICC = 0.931 (CI:0.914–0.946, p < 0.0001)

Using the automated methods, we examined the association of psoas muscle mass with Child classification. We found that muscle mass was lower in patients with more advanced liver disease, with progressively less muscle area in those with Child C cirrhosis compared to Child A or Child B cirrhosis (Table 2). Furthermore, the reduction was mostly in high quality muscle with progressively less normal density muscle and lower overall muscle density in Child C patients. We also noted that patients with history of ascites at baseline had lower PMI compared to those who did not but there was no difference in PMI in those with and without history of hepatic encephalopathy or variceal bleed (Supplemental Table 2). Other variables examined were age, gender, etiology of disease. We found as expected that PMI was significantly different between gender but not etiology of disease (Supplemental Table 2). To ascertain that our automated PMI measures were consistent with those found with another muscle measurement commonly used in the cirrhosis population¹⁰, we measured the total abdominal skeletal muscle area and skeletal muscle index (SMI) at lumbar 3 spinal level (L3) using a manual method which has previously been described^{32, 33}. The associations derived using automated PMI with clinical variables in cirrhosis was similar when manual SMI was used (Supplemental Table 3). There was a decrease in SMI with age, female gender and presence of ascites but not with presence of encephalopathy or variceal bleed at baseline.

Table 2:

Automated psoas muscle measurements and association with Child class

	Child A (n = 121)	Child B (n = 100)	Child C (n = 33)	p
Total Psoas Area TPA (cm2)	24.3 ± 7.9	21.2 ± 7.8	20.9 ± 6.4	0.006
Mean Psoas Density (HU)	49.8 ± 10.0	47.0 ± 8,4	46.5 ± 8.5	0.04
Normal density muscle area (cm2)	18.5 ± 7.3	16.2 ± 6.6	15.5 ± 5.8	0.01
Low Density muscle area (cm2)	3.6 ± 2.0	3.6 ± 1.8	3.7 ± 1.9	0.97
Psoas Muscle Index (PMI- cm2/m2)	8.0 ± 2.2	7.4 ± 2.3	6.9 ± 1.9	0.02

To examine whether baseline PMI was associated with survival, we used Kaplan Meier Survival Curves and Cox regression models. We found those with higher PMI had significantly longer survival than those with lower PMI (Figure 3, p <0.001). These results did not change whether the measurements were made with manual or automated methodology. Given that age, gender and liver function can impact muscle size, we adjusted for these variables in a Cox’s proportional hazard regression model, and found that PMI remained statistically significant, suggesting that it has value in determining prognosis incremental to Child classification (Table 3). Not surprisingly SMI was also associated with survival in cirrhosis and performed equally well as PMI as a single variable to predict mortality in our cohort, but interestingly, in the multivariable model, SMI was no longer significant (Supplemental Table 4). To further ascertain the value of automated muscle mass measurements to predict survival in patients with cirrhosis, we further adjusted for other clinical variables which might impact survival including MELD and etiology of disease and found that automated PMI measurements remained statistically significant (Supplemental Table 5).

Figure 3:

Kaplan Meier Curve of survival probability with 95% confidence intervals of patients with high or low PMI using manual (Left) vs automated (Right) measurements.

Table 3: Univariate and multivariate analysis of predictors of survival.

Input variables for the multivariable model were: age, gender, child classification and PMI.

	Univariate		Multivariate
	Hazard Ratio	P value	Adjusted Hazard Ratio (95%CI)	P value
Age	1.02 (1.00–1.03)	0.09	1.02(1.00–1.04)	0.89
Gender (Male)	0.7 (0.48–1.0)	0.06	1.03 (1.00–1.04)	0.019
Child B*	2.4 (1.6–3.7)	<0.001	2.15 (1.38–3.33)	0.001
Child C*	6.3 (3.7–10.7)	<0.001	5.98 (3.40–10.50)	<0.001
PMI	0.8 (0.73–0.88)	<0.001	0.86 (0.78–0.96)	0.005

^*Hazard ratio were calculated with Child A as reference.

Discussion:

Artificial intelligence (AI) has become increasingly pervasive in our daily lives but only beginning to affect how we care for patients. In this paper, we demonstrate how deep learning techniques learned from facial recognition technology can be applied to radiological images and provide valuable incremental information for prognosis in patients with cirrhosis. Core muscle wasting is a known consequence of cirrhosis and highly predictive of prognosis⁴². The prognostic information available from quantifying muscle bulk remains locked in the countless CT scans obtained for widespread clinical indications. Although many techniques are available to characterize muscle size, a system which requires human manual review would be impossible to implement clinically. Automation is required to assure feasibility and reproducibility. One can envision that if psoas muscle measurements can be automatically calculated when a patient receives a CT for clinical indications, this could have added value for clinicians. In this paper, we show how deep learning techniques learned from facial recognition technology can be applied to radiological images and provide valuable incremental information for prognosis in patients with cirrhosis. Our study demonstrates that we can automate the identification and measurement of psoas muscle area and density to yield clinically meaningful information.

Our automated methodology performed well compared to the manual method in a “normal” kidney donor population. Both muscle quantity and quality decline with progressive liver disease⁴² which can make it more challenging to measure on CT scans compared to normal cohorts as lower muscle quality results in less clear demarcation of muscle definition. Our automated process was, therefore, rigorously tested in a normal as well as diseased cohorts with chronic liver disease. The use of automated algorithms is far faster and more consistent than the manual measurements clinicians have used to date. The process could be rapidly scalable allowing for the ability to integrate into clinical systems and to provide actionable information from clinically obtained CT scans.

Beyond optimized measurements, we demonstrate clinical value by linking automated muscle measures to clinical outcomes. At baseline, automated psoas measurements correlated with liver disease severity. Child C cirrhosis patients had significantly smaller and less dense psoas muscles than Child A and B patients. During follow-up, the psoas muscle index was predictive of survival adjusting for disease severity, adjusted hazard ratio 0.86 95%CI(0.78–0.96). Whereas one prior study used deep learning methods to define muscle features,⁴³ that study neither included patient with cirrhosis nor validated its findings with future clinical outcomes. These findings confirm prior studies showing the central role of muscle mass in the prognosis of patients with chronic liver disease^{9, 21, 23, 27, 44, 45}. In sum, automated measures of psoas size efficiently provide clinically meaningful information from CT scans obtained for other reasons.

These data must be interpreted in the context of the study design. First, our findings are based on method that requires patients to have had CT scans and may not generalize to persons who are not candidates for CTs. Second, we validated our models using robust statistical methods however future studies are needed to validate the performance of our deep learning methods in other settings. Third, although the clinical need for improved prognostic information is in patients with Child A-B cirrhosis, it should be noted that only 1 in 8 patients were Child C and therefore these data may not apply to a larger cohort of Child C patients

In summary, we developed an automated method to identify and measure psoas muscle area and density in CT scans obtained for other clinical indications. This measure was associated with important clinical outcomes. Future studies should examine the practicality and utility of applying such AI-based technology in routine clinical care.