The NAFLD Decompensation Risk Score: External Validation and Comparison to Existing Models to Predict Hepatic Events in a Retrospective Cohort Study

Abstract

Background

The NAFLD decompensation risk score (the Iowa Model) was recently developed to identify patients with nonalcoholic fatty liver disease (NAFLD) at highest risk of developing hepatic events using three variables—age, platelet count, and diabetes.

Aims

We performed an external validation of the Iowa Model and compared it to existing non-invasive models.

Methods

We included 249 patients with NAFLD at Boston Medical Center, Boston, Massachusetts, in the external validation cohort and 949 patients in the combined internal/external validation cohort. The primary outcome was the development of hepatic events (ascites, hepatic encephalopathy, esophageal or gastric varices, or hepatocellular carcinoma). We used Cox proportional hazards to analyze the ability of the Iowa Model to predict hepatic events in the external validation (https://uihc.org/non-alcoholic-fatty-liver-disease-decompensation-risk-score-calculator). We compared the performance of the Iowa Model to the AST-to-platelet ratio index (APRI), NAFLD fibrosis score (NFS), and the FIB-4 index in the combined cohort.

Results

The Iowa Model significantly predicted the development of hepatic events with hazard ratio of 2.5 [95% confidence interval (CI) 1.7–3.9, P < 0.001] and area under the receiver operating characteristic curve (AUROC) of 0.87 (CI 0.83–0.91). The AUROC of the Iowa Model (0.88, CI: 0.85–0.92) was comparable to the FIB-4 index (0.87, CI: 0.83–0.91) and higher than NFS (0.66, CI: 0.63–0.69) and APRI (0.76, CI: 0.73–0.79).

Conclusions

In an urban, racially and ethnically diverse population, the Iowa Model performed well to identify NAFLD patients at higher risk for liver-related complications. The model provides the individual probability of developing hepatic events and identifies patients in need of early intervention.

Graphical abstract

Nonalcoholic fatty liver disease (NAFLD) is rapidly becoming one of the most common liver disorders worldwide, with prevalence in the USA and Europe ranging from 21 to 24% and up to 31–32% in South America and the Middle East.^1,2 While some patients with NAFLD have a good prognosis and do not progress to advanced liver disease^3,4 up to 20% of patients with nonalcoholic steatohepatitis (NASH) experience disease progression and develop cirrhosis with liver-related complications.⁵ NAFLD is now becoming one of the leading indications for liver transplantation in the USA and is projected to overtake chronic hepatitis C infection (HCV) and alcoholic liver disease (ALD).⁶ Recent data trends forecast a further exacerbation of the donor shortage for liver transplantation, and the one-year probability of receiving a liver transplant is lower in patients with NAFLD than those with HCV or ALD, raising the need to identify patients at highest risk of developing hepatic decompensation and preventing disease progression.^6,7

The degree of hepatic fibrosis is one of the most significant predictors of hepatic decompensation, but the current gold standard of liver biopsy is invasive, associated with sampling error, and not routinely performed.^{8, 9, 10} Non-invasive measurement of hepatic steatosis and fibrosis through vibration-controlled transient elastography (VCTE) is not universally accessible in primary care settings.¹¹ Multiple models have been developed to assess for the degree of fibrosis, including the FIB-4 index, NAFLD fibrosis score, and AST-to-platelet ratio index (APRI), but these models were not developed nor validated to predict hepatic decompensation events or hepatic event-free survival in patients with NAFLD.^{12, 13, 14, 15, 16, 17, 18}

The Iowa NAFLD decompensation risk score (hereafter referred to as the Iowa Model) was recently developed and internally validated in a cohort of 700 patients to identify patients with NAFLD without clinically evident cirrhosis who are at higher risk of developing hepatic decompensation and provides an individual patient's probability of development of hepatic events.¹⁹ Our primary objective was to perform an external validation of the Iowa Model in an urban, racially and ethnically diverse cohort of patients with NAFLD and without clinically evident cirrhosis. We also aimed to compare the performance of the Iowa Model with currently existing non-invasive scoring systems in the combined internal and external validation cohort.

Methods

Settings and Participants

For external validation, we included patients who received care in the primary care clinics, hepatology clinics, and inpatient clinical services at the Boston University Medical Center in Boston, Massachusetts, between 2010 and 2019 with ICD-9 and ICD-10 codes of NAFLD, NASH, fatty liver disease, NASH cirrhosis, cryptogenic cirrhosis, or cirrhosis of unknown cause (Appendix Table 1). After creating an initial patient pool from ICD codes, we reviewed individual charts to confirm the presence of hepatic steatosis based on the definition from the American Association for the Study of Liver Disease (AASLD): (1) evidence of hepatic steatosis on imaging, including abdominal ultrasound, computed tomography (CT), and magnetic resonance imaging (MRI) or liver histology and (2) lack of secondary causes of liver fat accumulation, including heavy alcohol use, long-term medication use, or hereditary disorders.²⁰ Patients with controlled attenuation parameter (CAP) score >238 dB/m on FibroScan were also confirmed to have hepatic steatosis.²¹

For the comparison to existing models, we combined 249 patients from the external validation cohort with the 700 patients from the original construction and internal validations cohorts from The University of Iowa Hospitals and Clinics, Iowa City, Iowa, as has previously been described.¹⁹ The combined cohort was used in order to provide adequate sample size for comparison of models.

We excluded patients who had a clinical evidence of cirrhosis (on imaging, or liver biopsy, or elastography) at the time of diagnosis of NAFLD (including METAVIR stages F3 and F4), significant alcohol use (as defined as >7 drinks per week for women, >14 drinks per week for men), use of long-term medications that can cause hepatic steatosis (e.g., corticosteroids and methotrexate), other risk factors for chronic liver disease (including viral hepatitis, drug-induced liver disease, autoimmune disease, or genetic liver diseases, such as Wilson's disease or alpha-1-antitrypsin [A1AT] deficiency), and missing variables to calculate the Iowa Model. We also excluded patients who developed a hepatic event within 12 months of diagnosis of NAFLD as they likely had advanced fibrosis or underlying cirrhosis at the time of diagnosis. The inclusion and exclusion criteria of the patient population in this Boston cohort were similar to the original cohort of 700 patients from which the Iowa Model was developed and validated.

All authors had access to the study data and reviewed and approved the final manuscript.

Data Collection

We manually reviewed the electronical medical records of included patients to obtain baseline demographic, clinical, and biographical data at the time of initial diagnosis of NAFLD. We obtained data on age, sex, race/ethnicity, presence of diabetes mellitus (defined as hemoglobinA1c ≥6.5%, fasting glucose ≥125 mg/dL, or on anti-diabetic medications), body mass index (BMI) in kg/m², serum platelet count, fasting glucose, serum alanine aminotransferase (ALT), serum aspartate aminotransferase (AST), and serum triglyceride at the time of diagnosis. We recorded the time of development of hepatic event (defined as ascites, esophageal or gastric varices, hepatic encephalopathy (HE), or hepatocellular carcinoma in clinical notes, imaging, or procedures). We also recorded the time of liver transplantation or death. The last follow-up date was defined as the most recent clinic hospital encounter, date of development of hepatic events, date of death, and date of transplantation, whichever is earliest.

Charts were reviewed independently by two researchers (HSA and NG). If there was ambiguity in the chart regarding the diagnosis or time of development of hepatic event, both researchers then reviewed the same chart again to come to a consensus.

Non-invasive Scoring Models

We calculated the NAFLD decompensation risk score for each patient with the following formula: NAFLD decompensation risk score = age × 0.06335 + presence of diabetes (yes = 1, no = 0) × 0.92221 - platelet count × 0.01522. This model is available as an online calculator at https://uihc.org/non-alcoholic-fatty-liver-disease-decompensation-risk-score-calculator. We calculated the FIB-4 index, the NAFLD fibrosis score, and the APRI for each patient with the following formulas: (age x AST)/(platelet count xALT^1/2),¹³ NAFLD fibrosis score = −1.675 + (0.037 × age) + (0.094 × BMI) + (1.13 × impaired fasting glucose/diabetes [yes = 1, no = 0]) + (0.99 × AST/ALT) – (0.013 × platelet count) – (0.66 × albumin),¹² and APRI = (AST/ALT)/platelet count x 100,¹⁵ respectively.

Statistical Analysis

We compared the characteristics of patients who developed hepatic events to those who did not develop liver-related events with t-test for continuous variables and chi-squared test for categorical variables. We measured time from the date of diagnosis of NAFLD to the date of development of hepatic event, death, liver transplantation, or last follow-up. We used Kaplan–Meier analysis to calculate the hepatic event-free survival. We used Cox proportional hazards analysis to determine whether the Iowa Model was predictive of development of hepatic events in the external validation cohort. We used the Harrell’s C statistic to calculate the time-dependent receiver operative characteristic (AUROC) curve.

We then calculated and compared the AUROC of the Iowa Model, APRI, NAFLD fibrosis score, and FIB-4 index in the prediction of development of hepatic events in the combined cohort. The comparison was performed in the combined cohort as it provides better discrimination among the scores due to the higher sample size and event rate.

Statistical analysis was performed using the Statistical Analysis Software (SAS), version 9.4, SAS Institute Inc., Cary, NC, USA, and the Stata Statistical Software: Release 13, College Station, TX: StataCorp LP.

The Boston University and the University of Iowa Institutional Review Boards deemed this study exempt from review.

Results

Baseline Characteristics

We reviewed the medical records of 1597 patients to include 249 patients who met our inclusion/exclusion criteria (Figure 1). Baseline characteristics for the entire cohort and in the patients who developed hepatic decompensation are summarized in Table 1. The mean age of the cohort was 52 ± 12 years, and 151 patients (60.6%) of the cohort were women. The cohort was racially and ethnically diverse with 103 (41.4%) patients of Hispanic ethnicity, 73 (29.1%) non-Hispanic white, and 56 (22.4%) Black. The mean BMI was 35.8 ± 16.9 kg/m². Ninety-six patients (39%) had diabetes mellitus. The majority of patients were diagnosed with hepatic steatosis on imaging, with 171 patients diagnosed on ultrasound, 56 patients diagnosed with CT, 2 patients diagnosed with MRI, 6 patients with VCTE, and 12 patients with liver biopsy. The median follow-up for the entire cohort was 78 (43–139) months. Fifteen (6.0%) patients developed hepatic events, three patients with ascites, six patients with varices, one patient with HE, three patients with HCC, one patient with ascites, varices, and HE, and one patient with varices and HE. Ten (4.0%) patients died during follow-up, one with liver-related mortality, one with cardiovascular-related mortality, and eight with unknown cause of death.

Figure 1.

Inclusion and exclusion criteria for eligible patients who received care at Boston Medical Center between 2010 and 2019.

Table 1.

Baseline Characteristics for External Validation Cohort.

Characteristic		Total cohort n = 249	Developed hepatic decompensation n = 15
		Mean (SD) or n (%)	Mean (SD) or n (%)
Age (years)		51.9 ± 12.2	55.4 ± 12.5
Female (%)		151 (60.6)	12 (80.0)
Race/ethnicity (%)	Hispanic	103 (41.3)	6 (40.0)
	White	73 (29.3)	6 (40.0)
	Black	56 (22.5)	3 (20.0)
	Other	17 (6.8)	0 (0.0)
Diabetes (%)		96 (38.6)	8 (53.3)
BMI (kg/m2)		35.8 ± 16.9	34.8 ± 8.2
Platelet count (109/L)		248 ± 71	188 ± 62.9
AST (U/L)		37 ± 23	47 ± 27.7
ALT (U/L)		52 ± 45	60.5 ± 59.2
Albumin (g/dL)		4.2 ± 0.4	3.9 ± 0.7
Triglyceride (mg/dL)		192 ± 128	208 ± 145.5

Note: BMI: body mass index; AST: aspartate aminotransferase; ALT: alanine aminotransferase; NAFLD: nonalcoholic fatty liver disease.

Baseline characteristics with demographic and clinical variables for the 249 patients included in the external validation cohort and the 15 patients who developed hepatic decompensation.

External Validation of the Iowa Model

The univariate analysis confirmed that the variables of the Iowa Model were all significant predictors of hepatic events in our cohort, with age (P = 0.009), platelet count (P = 0.001), and presence of diabetes (0.005). The Iowa Model was significant in predicting the development of hepatic events with a hazard ratio (HR) of 2.5 [95% confidence interval (CI) 1.7–3.9, P < 0.001]. The AUROC was 0.87 (95% CI 0.83–0.91) (Figure 2).

Figure 2.

Observed Kaplan–Meier “time-to-hepatic event” curve versus the predicted cumulative hazard curve. The blue line is the “time-to-hepatic event” curve; solid red line is the predicted cumulative hazard curve; dotted red line is interval bounds.

Comparison of Non-invasive Scoring Models

In the combined Iowa and Boston cohort of 949 patients, 63 (6.6%) patients developed hepatic decompensation at a median follow-up of 72 (33–115) months. All four models—APRI, the NAFLD fibrosis score, the FIB-4 index, and the Iowa Model—were significant in predicting the development of hepatic events (Table 2). The AUROC for APRI was 0.76 (95% CI 0.73–0.79), NAFLD fibrosis score was 0.66 (95% CI 0.63–0.69), FIB-4 index was 0.87 (95% CI 0.83–0.91), and the Iowa model was 0.88 (95% CI 0.85–0.92) (Figure 3). The time-dependent AUROC to predict the development of hepatic events at 5-, 10-, and 12-years was higher for Iowa Model as compared to other models at each of the individual time-points (Table 2).

Table 2.

Comparison of the Iowa Model with Three Other Non-invasive Scoring Systems in the Combined Cohort.

	‘c’ statistic (95% CI)	AUROC at 5 years	AUROC at 10 years	AUROC at 12 years
The Iowa Model	0.88 (0.85–0.92)	0.85	0.83	0.84
APRI	0.76 (0.73–0.79)	0.70	0.69	0.65
NAFLD fibrosis score	0.66 (0.63–0.69)	0.61	0.68	0.71
FIB-4 index	0.87 (0.83–0.91)	0.82	0.77	0.79

Note: AUROC: area under the receiver operator characteristic curve; APRI: AST-to-platelet index; NAFLD: nonalcoholic fatty liver disease.The c-statistic and AUROC of the combined cohort of 949 patients for the Iowa Model, APRI, NAFLD fibrosis score, and FIB-4 index.

Figure 3.

AUROCof the various non-invasive models to predict hepatic decompensation in patients with NAFLD at 5 years. (a) the Iowa Model. (b) AST-to-platelet ratio index (APRI). (c) NAFLD fibrosis score. (d) FIB-4 index.

Percentage of At-risk Patients Based on the Iowa Model

As determined by the Iowa model, the percentage of patients with a >5% predicted probability of developing hepatic events at five years was 23% in the Boston cohort and 20% in the combined cohort. Similarly, the percentage of patients with a >10% predicted probability of hepatic events at five years was 12% in Boston cohort and 10% in the combined cohort.

Discussion

In this study, we have shown that the Iowa Model performs well in an external, racially and ethnically diverse population to identify NAFLD patients at highest risk of developing hepatic events. Additionally, we have shown that in our two-center combined cohort, the Iowa Model performs as well as the FIB-4 index and seems to be better than the APRI and the NAFLD fibrosis score to predict hepatic events.

Several non-invasive models have been developed to predict the presence of steatosis or advanced fibrosis, but few models have been designed specifically to predict outcomes in patients with NAFLD. The Framingham Steatosis Index was developed in 2016 using a cohort of patients in the Framingham Heart Study, using ALT:AST ratio, age, sex, BMI, triglyceride level, hypertension, and diabetes to identify patients with NAFLD on CT imaging,²² but it has not been used in prediction for hepatic decompensation. Additionally, the FIB-4 index, APRI, and NAFLD fibrosis score have all been developed to predict the presence of advanced fibrosis, but they were not specifically designed to predict outcomes in patient with NAFLD. Though studies later tried to evaluate the discriminatory ability of these models to predict the development of hepatic events,²³ the strength of the Iowa Model's predictive capacity lies in the construction of the model based on patients with NAFLD. Unlike in these other models, the construction cohort and validation cohorts specifically excluded patients with advanced fibrosis, making the Iowa Model more likely to represent individuals with higher risk of NASH and progressive disease. The Iowa Model has excellent discrimination in predicting the risk of hepatic events in NAFLD patients, and the AUROC of our model was superior to both APRI and the NAFLD fibrosis score and comparable to the FIB-4 index. Additionally, the Iowa Model provides the probability that an individual patient with NAFLD will develop hepatic events in the future. This will not only allow patients to understand their future risk of liver-related events but also helps clinicians determine when to refer a patient to a hepatologist for early and aggressive interventions.

With the growing incidence and increased recognition of NAFLD, primary care providers will encounter an increasing number of patients with NAFLD, but not all may be feasibly referred to a hepatologist. This raises the important question of how to recognize which patients are at highest need for specialist care. Any patient with clinically obvious cirrhosis or decompensation should be referred to a hepatologist for further evaluation and potential pharmacologic therapy. Patients without clinical cirrhosis, however, should be distinguished into those at low risk and at high risk for the progression of liver disease and the development of hepatic events. We propose that patients with a risk of hepatic events greater than 10% at five years based on the Iowa Model to be considered as higher risk necessitating a referral to a hepatologist. We arrived at the 10% at five years cut-off as it may allow a high-risk NAFLD patient to be evaluated by a hepatologist within a reasonable time frame.

The main strength of our study is the racial and ethnic diversity of the external validation cohort. Although the construction and internal validation of the Iowa Model were completed in a Midwestern, heavily white population, the model still performed well in our more diverse, urban population. Additionally, although ICD codes were used to identify an initial patient pool for review, individual chart review was performed to identify patients with steatosis either on imaging, VCTE, or biopsy, and to exclude patients with other causes of chronic liver disease and those with clinical cirrhosis, per AASLD guidelines.²⁰ Interestingly, the mean platelet count of patients who developed hepatic events was lower than the mean for the total cohort (188 versus 248 10⁹/L), although not within the range of thrombocytopenia associated with portal hypertension at the time of diagnosis. This highlights both the predictive value of platelet count within the Iowa Model and also strengthens the probability that individuals who developed hepatic decompensation did not have cirrhosis or portal hypertension at the time of diagnosis of NAFLD.

There are several limitations to the study. Liver biopsy was not performed in the majority of our patients, so we cannot histologically confirm the presence or absence of NASH nor estimate the degree of fibrosis at the time of study inclusion. However, as mentioned above, each patient chart was individually reviewed to rule out evidence of clinically evident cirrhosis. Additionally, patients who developed hepatic events within one year of diagnosis of NAFLD were excluded as these patients likely had cirrhosis at the time of inclusion into the study. However, it is possible that some patients had at least moderate fibrosis at the time of inclusion in the study. The decision to exclude patients with cirrhosis was made as we were specifically developing a risk stratification model for patients who do not have cirrhosis, as it is already understood that patients with a diagnosis of cirrhosis are at increased risk of hepatic decompensation. However, this exclusion could potentially have introduced a selection bias. The NAFLD fibrosis score, APRI, and FIB-4 index were developed in cohorts that include individuals with advanced fibrosis and cirrhosis—this difference in degree of fibrosis may explain why our model performed better than NAFLD fibrosis score and APRI. Another limitation is the relatively low event rate in the validation cohort for a thorough multivariate analysis of all individual risk factors. Despite the small event rate, the HR of the NAFLD score was relatively narrow and achieved statistical significance.

The Iowa Model performs well in an external racially and ethnically diverse cohort of patients with NAFLD to predict hepatic events. The Iowa Model can especially be used at the primary care level to identify patients who would benefit from specialist referrals and therapeutic interventions.

Ethics approval

This study was performed in accordance with the ethical standards of the institutional committees and the 1964 Declaration of Helsinki and later amendments. The Boston University and the University of Iowa Institutional Review Boards deemed this study exempt from review.

Credit authorship contribution statement

HSA, ARM, and AS were involved in initial conception and design of the study. HSA and NG performed data curation. ARM performed statistical analysis. HSA drafted the manuscript. HSA, ARM, AS, and MTL contributed to major revisions of the manuscript. All authors read and approved the final manuscript.

Conflicts of interest

The authors declare they have no conflicting or competing interests.

Funding

Heidi S. Ahmed is supported in part by NIH 2 T32 DK 7201-42.

Availability of data and materials

The datasets generated and analyzed in this study are not publicly available, but are available from corresponding author on reasonable request.

Appendix

Table 1.

International Classification of Disease (ICD) Codes are Used to Identify Patients for Inclusion in the Study.

ICD version	ICD code	Description in EMR
ICD-10	K75.81	Nonalcoholic steatohepatitis
		Nonalcoholic steatohepatitis (NASH)
		Steatohepatitis
		Steatohepatitis, nonalcoholic
	K76.0	Fatty (change of) liver, not elsewhere classified
		Fatty infiltration of liver
		Fatty liver
		Fatty liver determined by biopsy
		Fatty liver disease, nonalcoholic
		Fatty metamorphosis of liver
		Hepatic steatosis
		Liver fatty degeneration
		NAFL (nonalcoholic fatty liver)
		NAFLD (nonalcoholic fatty liver disease)
		Nonalcoholic fatty liver disease
		Non-alcoholic fatty liver disease
		Nonalcoholic hepatosteatosis
		Steatosis of liver
		Steatosis, liver
	K74.60	Other and unspecified cirrhosis of liver
		Advanced cirrhosis of liver
		Cirrhosis
		Cirrhosis of liver
		Cirrhosis of liver not due to alcohol
		Cirrhosis of liver without ascites
		Cirrhosis of liver without ascites, unspecified hepatic cirrhosis type
		Cirrhosis of liver without mention of alcohol
		Cirrhosis, nonalcoholic
		Cirrhosis, non-alcoholic
		Diffuse nodular cirrhosis of liver
		Hepatic steatosis
		Liver cirrhosis
		Unspecified cirrhosis of liver
ICD-9	571.8	Other chronic nonalcoholic liver diseases
	571.5	Cirrhosisof liver,nonspecific

Note: ICD: International Classification of Disease; EMR: electronic medical record.

Supplementary data

The following is/are the supplementary data to this article: