Models estimating risk of hepatocellular carcinoma in patients with alcohol or NAFLD-related cirrhosis for risk stratification

Abstract

Background and Aims

HCC risk varies dramatically in patients with cirrhosis according to well-described, readily-available predictors. We aimed to develop simple models estimating HCC risk in patients with ALD-cirrhosis or NAFLD-cirrhosis and calculate the net benefit that would be derived by implementing HCC surveillance strategies based on HCC risk as predicted by our models.

Methods

We identified 7068 patients with NAFLD-cirrhosis and 16,175 with ALD-cirrhosis who received care in the Veterans Affairs (VA) healthcare system in 2012 and retrospectively followed them until January 2018 for the development of incident HCC. We used Cox proportional hazards regression to develop and internally validate models predicting HCC risk using baseline characteristics at entry into the cohort in 2012. We plotted decision curves of net benefit against HCC screening threshold.

Results

We identified 1278 incident cases of HCC during a mean follow-up period of 3.7 years. Mean annualized HCC incidence was 1.56% in NAFLD-cirrhosis and 1.44% in ALD-cirrhosis. The final models estimating HCC were developed separately for NAFLD-cirrhosis and ALD-cirrhosis and included 7 predictors: age, gender, diabetes, BMI, platelet count, serum albumin and AST/√ALT ratio. The models exhibited very good measures of discrimination and calibration and area under the ROC curve of 0.75 for NAFLD-cirrhosis and 0.76 for ALD-cirrhosis. Decision curves showed higher standardized net benefit of risk-based screening using our prediction models compared to the screen-all approach.

Conclusions

We developed simple models estimating HCC risk in patients with NAFLD-cirrhosis or ALD-cirrhosis, which are available as web-based tools (www.hccrisk.com). Risk stratification can be used to inform risk-based HCC surveillance strategies in individual patients or healthcare systems or to identify high-risk patients for clinical trials.

LAY SUMMARY

Patients with cirrhosis of the liver are at risk of getting hepatocellular carcinoma (HCC or liver cancer) and therefore it is recommended that they undergo surveillance for HCC. However, the risk of HCC varies dramatically in patients with cirrhosis, which has implications on if and how patients get surveillance, how providers counsel patients about the need for surveillance, and how healthcare systems approach and prioritize surveillance. We used readily available predictors to develop models estimating HCC risk in patients with cirrhosis, which are available as web-based tools at www.hccrisk.com

Introduction

Annual HCC risk varies greatly in patients with cirrhosis ranging from as little as <0.2% to as high as >5%. Although this variability is well recognized, few models exist to estimate HCC risk in patients with cirrhosis and none are commonly used. Liver societies recommend the same screening strategy (abdominal ultrasonography every 6 months with or without concomitant serum AFP) irrespective of HCC risk^1–3. Studies show poor compliance with these screening recommendations^{4, 5}. Stratification of HCC risk in patients with cirrhosis into low (e.g. <1% per year), medium (e.g. 1–3% per year) and high (e.g. >3% per year) would allow optimization and individualization of outreach efforts and screening strategies in patients with cirrhosis. It would also enable identification of high-risk patients for clinical trials of HCC screening.

Many determinants of HCC risk in cirrhotic patients have already been described including routinely available clinical characteristics (e.g. etiology of cirrhosis, age, gender, race, body mass index, diabetes, HCV genotype) and serum laboratory tests (e.g. platelet count, albumin, aspartate aminotransferase, alanine aminotransferase and alpha fetoprotein)⁶. These readily available predictors could potentially be combined into HCC risk estimation models with adequate discrimination and calibration.

The incidence of HCC has been rising dramatically over time^{7, 8}. Therefore, HCC risk prediction models must be based on a recent cohort to avoid underestimating HCC risk. HCC risk is frequently calculated in patients with cirrhosis from the time they were first diagnosed with cirrhosis (i.e. as an “inception cohort”). However, this results in the diagnosis of many prevalent cases as well as incident cases in the early period after the inception of the cohort. For practicing clinicians, a more useful scenario would be to estimate HCC risk in a patient with cirrhosis when they are seen in clinic at some point during their natural history, not necessarily only when they first get diagnosed with cirrhosis. Also, HCC risk prediction models developed specifically to inform screening strategies, need to have a time horizon of approximately 5 years or less. This is counterintuitive since longer follow-up is usually considered better; however, HCCs destined to develop in 5 to 10 years from now are not going to be diagnosed by screening occurring “now” and therefore should not influence the prediction models.

The three most common etiologies of cirrhosis in the United States, which also account for the majority of HCC cases, are hepatitis C virus (HCV), alcoholic liver disease (ALD) and nonalcoholic fatty liver disease (NAFLD). HCC risk prediction models need to be developed separately for different major etiologies of cirrhosis for a number of reasons. Firstly, HCC risk is greater in cirrhotic patients with HCV than those with other underlying liver diseases^{6, 9}. Secondly, certain predictors may be more important for some etiologies than others (e.g. obesity and diabetes may be more important in NAFLD-cirrhosis than in HCV-cirrhosis) while some predictors are unique to some etiologies (e.g. HCV genotype is unique to HCV-cirrhosis). Thirdly, direct antiviral treatments have a dramatic effect on HCC risk in HCV-related cirrhosis, such that HCC risk has to be calculated specifically relative to the time of receipt of antiviral treatment. The prevalence of HCV-related cirrhosis is declining rapidly due to the decreasing prevalence of HCV since the year 2001 and the more recent dramatic increase in HCV cures after the introduction of direct-acting antiviral treatments. In contrast, the prevalence of cirrhosis related to nonalcoholic fatty liver disease (NAFLD) and alcohol-related liver disease (ALD) is increasing^{10, 11}.

For these reasons, we aimed to develop and internally validate models that accurately estimate HCC risk in a recent cohort of patients with cirrhosis, specifically in patients with NAFLD-cirrhosis or ALD-cirrhosis. Furthermore, we wanted to evaluate the net benefit that would be derived by implementing HCC surveillance strategies based on HCC risk estimated by our models. Finally, we wanted to make our HCC risk prediction models available to clinicians as web-based tools so that HCC risk can be readily estimated in clinical practice.

Methods

Study Design

We identified all patients with ALD-cirrhosis or NAFLD-cirrhosis who received care in the Veterans Affairs (VA) healthcare system (VAHS) nationally in calendar year 2012, without a prior history of HCC. We followed them retrospectively from 2012 until 01/01/2018 for the development of incident hepatocellular carcinoma. We used baseline characteristics in 2012 to develop and internally validate HCC risk prediction models separately for ALD-cirrhosis and NAFLD-cirrhosis.

Study Population and Data Source

The VAHS is the largest integrated healthcare system in the United States, with 168 VA Medical Centers and 1053 outpatient clinics throughout the United States serving more than 8.9 million Veterans each year¹², including the largest number of patients with cirrhosis⁷. Nationwide, the VA uses a single comprehensive electronic healthcare information network which integrates all care applications into a single, common database. We derived electronic data on all VA patients with cirrhosis using the VA Corporate Data Warehouse (CDW), a national, continually updated repository of electronic VA data developed specifically to facilitate research¹³. Data extracted included all patient pharmacy prescriptions, demographics, inpatient and outpatient visits, problem lists, procedures, vital signs, diagnostic tests, and laboratory tests. The study was approved by the Institutional Review Board of the Veterans Affairs Puget Sound Healthcare System and a waiver of informed consent was granted.

We identified 62,030 patients who had a diagnosis of cirrhosis first recorded at or before 12/31/2012 and who received VA healthcare in calendar year 2012, defined by having at least 1 inpatient or outpatient visit for any indication during that calendar year. We excluded 2456 patients who had undergone liver transplantation before the beginning of follow-up, 2970 patients who had a diagnosis of HCC first recorded before the beginning of follow-up, and 3933 patients without available laboratory data in 2012. Among the remaining 52,671 patients, 16,175 had ALD-cirrhosis and 7068 had NAFLD-cirrhosis (defined below) and were included in this study, while the remaining 29,428 had other etiologies of cirrhosis (mostly HCV) and were excluded.

Definition of Cirrhosis

The diagnosis of cirrhosis was based on the presence of the ICD-9 codes for cirrhosis or complications of cirrhosis listed in Supplemental Table 1, recorded at least twice in any inpatient or outpatient encounter. This approach has been validated and widely used in VA-based studies by us^{6, 7, 14–20} and others^21–25. The diagnosis of cirrhosis using a single ICD-9 code in VA data has been shown to have a 90% positive predictive value (probability that cirrhosis is present among those with a code) compared to chart extraction²⁶. By requiring the relevant ICD-9 codes to be recorded at least twice we have found that the positive predictive value increased to 97% in a random sample of 250 patients included in the current study.

Among patients with cirrhosis, we defined ALD-cirrhosis and NAFLD-cirrhosis based on our previously published studies⁷ as follows:

ALD-cirrhosis

Patients with ICD-9 codes for alcohol use disorders (Supplemental Table 1) in the absence of serologic or virologic markers of chronic HCV or HBV infection and in the absence if ICD9 codes for hemochromatosis, primary biliary cirrhosis, primary sclerosing cholangitis, and autoimmune hepatitis (Supplemental Table 1).

NAFLD-cirrhosis

was defined in patients with diabetes (ICD-9 code 250– 250.92, recorded at least twice²⁵) or body mass index (BMI) ≥30 kg/m² prior to the diagnosis of cirrhosis, who did not have alcohol use disorders, serologic or virologic markers of chronic HCV or HBV infection or ICD9 codes for hemochromatosis, primary biliary cirrhosis, primary sclerosing cholangitis, and autoimmune hepatitis (Supplemental Table 1). NAFLD-related cirrhosis does not have pathognomonic serological, radiological, or histological features - even hepatic steatosis is frequently absent after cirrhosis develops. Hence we adapted a clinical definition of NAFLD based on previous work^{7, 27} that reflects the diagnostic process used in clinical practice, in which NAFLD is suspected in the presence of risk factors such as obesity and diabetes after exclusion of other etiologies.

Baseline Patient Characteristics

We ascertained the values of all baseline characteristics as of calendar year 2012 (the year of inception of our cohort). We extracted age, sex, race/ethnicity and all the baseline laboratory tests shown in Table 1. Body mass index (BMI) was calculated as the measured weight in kilograms divided by the square of the measured height in meters. For patients with multiple laboratory tests or BMI measurements we recorded the one closest to 01/01/2012. We also determined the presence of complications of cirrhosis (ascites, spontaneous bacterial peritonitis, encephalopathy, gastroesophageal varices with or without bleeding, hepatorenal syndrome, hepatopulmonary syndrome), type 2 diabetes mellitus, alcohol use disorders, substance use disorders and HIV infection based on appropriate ICD-9 codes recorded at least twice prior to the date of entry into the cohort in 2012 (Supplemental Table 1). These ICD9-based definitions of cirrhosis, decompensated cirrhosis and other comorbidities have been widely used and validated in studies using VA medical records^{7, 20–25}. Since 2004, the VA has implemented annual screening for alcohol use disorders using the validated Alcohol Use Disorders Identification Test Consumption (AUDIT-C). We extracted the AUDIT-C score within 12 months prior to entry into the cohort.

Table 1.

Baseline characteristics of our cohort of patients with cirrhosis at the time of cohort inception in 2012, presented according to cirrhosis etiology.

		ETIOLOGY OF CIRRHOSIS:
	ALD or NAFLD N=23,243	ALD N=16,175	NAFLD N=7068	p-value
Age, yrs (mean±SD)	64.2±9.4	62.9±8.9	67.1±9.7	< 0.001
Male (%)	97.2	97.9	95.5	< 0.001
Race/Ethnicity (%)				< 0.001
White, non-Hispanic	73.4	72	76.4
Black, non-Hispanic	10.1	11.1	7.5
Hispanic	8	8.4	7.1
Other	2.2	2.3	2
Declined to answer/missing	6.4	6.1	6.9
BMI, Kg/m2 (mean±SD)	29.9±6.7	28.5±6.3	33.0±6.6	< 0.001
AUDIT-C score† (%)
No alcohol use	57.5	47.9	73.9
Low-level alcohol use	20.1	19.3	19.8
Unhealthy alcohol use	22.2	29.4	3.0	< 0.001
Diabetes (%)	48.4	35.5	77.9	< 0.001
Substance Use Disorder (%)	16.4	22.5	2.6	< 0.001
HIV co-infection (%)	0.5	0.4	0.7	< 0.01
Complications of Cirrhosis (%)	37.7	39.3	34.2	< 0.001
Ascites (%)	19.8	22.2	14.4	< 0.001
Encephalopathy (%)	6.9	8.2	3.8	< 0.001
Gastroesophageal Varices (with bleeding) (%)	5.4	6.1	4	< 0.001
Gastroesophageal Varices (without bleeding) (%)	18.3	18.6	17.7	0.1
Laboratory Results, (mean±SD)
MELD score	11.2 ± 5.3	11.0±5.2	11.7±5.7	< 0.001
Alpha Fetoprotein* (ng/mL)	3.3 ± 2.6	3.4 ± 2.6	3.0 ± 2.4	0.35
Hemoglobin (g/dL)	13.1 ± 2.3	13.2 ± 2.3	13.0 ± 2.2	< 0.001
Platelet Count (k/¼L)	158 ± 83	162 ± 86	150 ± 76	< 0.001
Creatinine (mg/dL)	1.1 ± 0.8	1.0 ± 0.6	1.3 ± 1.0	< 0.001
Bilirubin (g/dL)	1.4 ± 2.2	1.5 ± 2.5	1.0 ± 1.2	< 0.001
INR	1.4 ± 1.2	1.3 ± 1.1	1.4 ± 1.3	< 0.001
Albumin (g/dL)	3.6 ± 0.7	3.6 ± 0.7	3.7 ± 0.6	< 0.001
Alkaline Phosphatase (U/L)	113 ± 79	116 ± 82	105 ± 70	< 0.001
AST/√ALT ratio	7.8 ± 5.1	8.4 ± 5.6	6.5 ± 3.1	< 0.001
FIB-4 score	5.1 ± 23.0	5.3 ± 24.1	4.6 ± 20.4	0.03

^*Median and interquartile range shown for AFP due to substantial left skewing

^†No alcohol use: AUDIT-C score 0. Low-level alcohol use: AUDIT-C 1–3 in men, 1–2 in women. Unhealthy alcohol use: AUDIT-C 4–12 in men, 3–12 in women.

Diagnosis of Hepatocellular Carcinoma

The diagnosis of HCC was based on the presence of ICD-9 code 155.0 and ICD-10 code C22.0 (the VA switched to ICD-10 codes on 10/1/2015) recorded at least twice. The ICD-9 code-based definition of HCC using VA records has been shown to have a positive predictive value of 84–94% compared to chart extraction^{24, 28, 29} and has been widely used by us^{6, 7, 19, 30} and other investigators^31–33

Statistical Analysis

We used multivariable Cox proportional hazards regression to model the risk of HCC using baseline characteristics ascertained in 2012 separately in patients with cirrhosis related to ALD or NAFLD. Follow-up started in 2012 (specifically the date of the latest blood draw during the period 1/1/2011 to 12/31/2012) and continued until 01/01/2018. We did not exclude any period of time following that blood draw from analysis (e.g. 3 or 6 months), even recognizing that there might be a higher likelihood of prevalent cases at the beginning, because we wanted the models to reflect the clinical scenario of a patient being seen by their provider in clinic. Patients who did not develop HCC by 01/01/2018, were censored at the time of death, liver transplantation or the date of last follow-up in the VA. We contemplated reporting a competing risks analysis, accounting for death as a competing risk, but it made little difference to the models and would have unnecessarily complicated the methodology.

Model Building

We considered 25 characteristics listed in Table 1 as potential predictors of HCC for inclusion in our models. We used an iterative process to determine which predictors to include in our final models. First, we estimated measures of discrimination, calibration, and significance when each predictor was added to the base model and identified the top 5 predictors with the greatest improvement in these measures. We chose the predictor that was consistently in the top 5 with a preference for a p-value < 0.10 and an improvement in the main measures of discrimination and calibration, namely Gönen and Heller’s κ, Royston and Sauerbrei’s D-statistic, calibration slope, and integrated Brier score (IBS). We verified graphically that the added predictor improved the observed vs. predicted risk plot thus allowing assessment over the entire time period. A pooled k-fold cross-validation was used to calculate all measures of model performance. A k equal to 10 was chosen to address the bias versus variability in a database with a large sample size, but relatively few events.

We then updated the base model to include the chosen predictor and removed any predictors with a p-value < 0.10; removed predictors were added back into the list of potential predictors. We favored variables that are objectively ascertained (such as laboratory test) and those that have been consistently associated with HCC in previous studies (such as age and sex).

We considered both dummy-categorical as well as continuous (linear or transformed) modeling of laboratory tests. Interaction terms were explored in cases where there were biological indications. The distribution of the model predictions was checked for normality.

Measures of Model Discrimination, Calibration and Accuracy

Discrimination refers to the ability to separate those who will get HCC from those who will not. The measures of discrimination chosen were Harrell’s C-index³⁴ (which measures the degree of concordance between pairs and is sensitive to censoring), Gönen and Heller’s κ³⁵ (which is also a measure of concordance but is robust to censoring), and Royston and Sauerbrei’s D-statistic³⁶ (which can be interpreted as the log hazard ratio of risk between the low and high risk groups dichotomized at their median values). Calibration refers to the degree of agreement between model-derived probabilities and observed probabilities. As a calibration measure, we evaluated the calibration slope³⁷, which is robust to censoring and ideally takes a value of 1. Additionally, we evaluated calibration graphically, by comparing the observed Kaplan-Meier estimates of HCC-free survival and lowess-smoothed model predictions of HCC-free survival after categorizing risk into low, medium, or high groups. Model prediction accuracy was evaluated using the integrated Brier score (IBS)³⁸, which is the mean squared difference between the predicted probability and the actual outcome and the area under the receiver operating characteristic (AUROC) curve, derived from a logistic regression of model predictions and diagnosis of HCC.

Measures of discrimination and calibration were estimated using k-fold cross-validation during model building. In addition, to obtain the most conservative estimates of discrimination and calibration, we split the data in half into derivation and validation datasets balanced on number of events and on VA facility (i.e. patients from approximately half of the VA facilities were included in the derivation dataset and from the other half in the validation dataset)³⁹. Measures of assessment were then calculated for each dataset using model coefficients from the derivation data. We also calculated the performance measures of the “Toronto HCC risk index” in predicting HCC risk in our study population as a comparison⁴⁰. This is a recently published HCC risk prediction tool that uses 4 predictors: age, sex, etiology of liver disease, and platelet count. Additionally, for comparison, we calculated the performance measures of the fibrosis-4 (FIB-4) score⁴¹, which incorporates age, platelet count, and AST/√ALT, and correlates with HCC, and the MELD score, as a measure of liver dysfunction.

Use of Decision Curves to Estimate the Net Benefit of Using our Risk Prediction Models

We used decision curves to estimate the standardized net benefit that would be expected in a population if our models are used to estimate HCC risk and patients are screened only when their estimated risk exceeds a pre-established risk threshold, as compared to the “screen-all” approach. A decision curve is a novel graphical plot of net benefit versus risk threshold that was proposed in 2006 for assessing the potential population impact of adopting a risk prediction instrument⁴². A risk threshold is defined as that probability of HCC above which screening would be favorable over not screening. “Standardized” net benefit is the proportion of total possible net benefit, which would be achieved by a “perfect” risk model that places all the patients with HCC above the risk threshold for screening without any false positives (i.e. without placing patients without HCC above the risk threshold). To avoid over-fitting, decision curves were calculated using repeated 10-fold cross-validation⁴² The cross-validation was repeated 50 times and results were averaged.

RESULTS

Characteristics of Study Population

Among 23,243 patients with ALD-cirrhosis or NAFLD-cirrhosis in VA care in 2012, most were male (97%) and non-Hispanic White (73%) (Table 1). Compared to patients with ALD-cirrhosis, those with NAFLD-cirrhosis were older (67.1 vs. 63.0 years old), had higher BMI (33.0 vs. 28.5 Kg/m²) and were more likely to have diabetes (78% vs. 36%). Substance use disorders were more common in ALD-cirrhosis than NAFLD-cirrhosis (22.5% vs. 2.6%). By definition, alcohol use disorders were universal in ALD-cirrhosis and absent in NAFLD-cirrhosis. Only 3% of patients in the NAFLD-cirrhosis group reported unhealthy alcohol use (AUDIT-C 4–12 in men or 3–12 in women) – which further confirmed our diagnostic definition of NAFLD-cirrhosis – as compared to 29.4% in the ALD-cirrhosis group. Complications of portal hypertension were present in a similar proportion in both groups at baseline (37.7%) and mean MELD score was 11.2%.

HCC Incidence

During a mean follow-up of 3.7 years (range 1 to 6), 1278 out of 23,243 patients with cirrhosis developed HCC (Figure 1). The annualized incidence of HCC was similar in ALD-cirrhosis (1.44%) and NAFLD-cirrhosis (1.56%) and mean follow-up practically identical (3.7 years). The annual incidence was significantly greater in the subset of patients with FIB-4>3.25 (2.68%) than FIB-4 <3.25 (0.68%). However, a substantial proportion of HCCs (354/1278 or 28%) occurred in patients with FIB-4 <3.25, justifying our decision not to exclude patients with a diagnosis of cirrhosis who had a FIB-4<3.25.

Figure 1. Cumulative incidence curves showing the probability of developing HCC in a cohort of patients followed from 2012 to 2018, plotted by.

a. Etiology of cirrhosis (ALD vs. NAFLD) and

b. FIB-4 score >3.25 or ≤3.25.

c. Table showing the incidence of HCC in this population by cirrhosis etiology and FIB-4 score

HCC Predictors

Older age, male sex and low platelet count were strong, independent predictors of HCC, consistent with many published studies (Table 2). In addition, low albumin and high AST/√ALT were strong, independent predictors of HCC. Increasing BMI was a significant predictor in ALD-cirrhosis but not NAFLD-cirrhosis and diabetes was a stronger predictor in ALD-cirrhosis than NAFLD-cirrhosis. However, the associations of BMI and diabetes with HCC in patients with NAFLD-cirrhosis were blunted because presence of diabetes or BMI>30 were used as defining characteristics for NAFLD-cirrhosis.

Table 2. Models developed to estimate HCC risk in patients with ALD-Cirrhosis, NAFLD-cirrhosis or the combined population.

The table shows the adjusted hazard ratios (and their p-values) for each predictor included in the models. The models are available to be executed at www.hccrisk.com. Coefficients and baseline cumulative incidences are reported in Supplemental tables 2 and 3.

PREDICTORS†	ALD-CIRRHOSIS		NAFLD-CIRRHOSIS		ALD or NAFLD CIRRHOSIS
Age, yrs	AHR*	p-value	AHR*	p-value	AHR*	p-value
≤49	1		1		1
>49 – 59	1.91	< 0.01	1.39	0.42	1.83	< 0.01
>59 – 64	2.84	< 0.001	2.01	0.08	2.69	< 0.001
>64 – 69	3.16	< 0.001	2.37	0.03	3.05	< 0.001
>69 – 77	3.76	< 0.001	1.95	0.1	3.15	< 0.001
> 77	3.4	< 0.001	2.09	0.07	3.05	< 0.001
Sex
Male	1		1		1
Female	0.6	0.16	0.24	< 0.01	0.39	< 0.01
BMI, Kg/m2
≤25.2	1		1		1
>25.2 – 29.3	1.32	< 0.01	0.91	0.64	1.25	0.01
>29.3 – 33.8	1.39	< 0.01	0.9	0.6	1.29	< 0.01
>33.8 – 38.5	1.49	< 0.001	0.8	0.28	1.27	0.02
> 38.5	1.31	0.06	0.77	0.22	1.15	0.22
Diabetes
No	1		1		1
Yes	1.46	< 0.001	1.24	0.1	1.39	< 0.001
Platelet Count
> 203	1		1		1
>146 – 203	1.89	< 0.001	1.12	0.61	1.64	< 0.001
>99 – 146	2.64	< 0.001	2.18	< 0.001	2.51	< 0.001
>68 – 99	2.99	< 0.001	3.04	< 0.001	3.04	< 0.001
≤ 68	4.13	< 0.001	3.52	< 0.001	3.92	< 0.001
Albumin, g/dL
> 4.1	1		1		1
>3.7 – 4.1	1.4	< 0.01	1.49	0.03	1.43	< 0.001
>3.3 – 3.7	2.2	< 0.001	2.11	< 0.001	2.18	< 0.001
>2.8 – 3.3	2.6	< 0.001	2.73	< 0.001	2.66	< 0.001
≤ 2.8	3.5	< 0.001	3.41	< 0.001	3.48	< 0.001
AST/√ALT
≤ 5.02	1		1		1
>5.02 – 6.45	1.63	< 0.001	1.27	0.13	1.47	< 0.001
>6.45 – 8.80	2.69	< 0.001	1.99	< 0.001	2.4	< 0.001
>8.80 – 12.83	3.27	< 0.001	2.5	< 0.001	2.95	< 0.001
> 12.83	2.71	< 0.001	4.99	< 0.001	2.67	< 0.001

^*AHR: Adjusted Hazards Ratio

^†The continuous variables (BMI, platelet count, albumin, and AST/√ALT) were categorized according to the 0–25^th, 25–50^th, 50–75^th, 75–90^th, and >90^th percentiles

Development of Models Estimating HCC Risk

Out of the 25 potential predictors that we considered (shown in Table 1), seven were included in the final models that we developed: age, sex, BMI, diabetes, platelet count, serum albumin and serum AST/√ALT ratio (Table 2). The same 7 predictors were selected for the models in NAFLD-cirrhosis and ALD-cirrhosis. However, the coefficients (and hazard ratios) for each predictor were slightly different for each, especially for diabetes and BMI. Of these seven predictors, four predictors (age, platelet count, serum AST/√ALT ratio and albumin) accounted for most of the prediction, i.e. accounted for most of the explained risk⁴³ (93.9% in NAFLD-cirrhosis and 94.0% in ALD-cirrhosis). The coefficients of the models are shown in Supplemental Table 2 and the baseline cumulative incidences at years 1–5 in Supplemental Table 3, providing all the necessary information for other investigators to apply or externally validate the models.

We also developed a model for the combined population of NAFLD-cirrhosis and ALD-cirrhosis also shown in Table 2, which included the same 7 predictors. We considered including the type of cirrhosis as a covariate in the model but it was not a significant predictor and hence was omitted.

Measures of discrimination were generally very good and slightly higher for the ALD-cirrhosis than the NAFLD-cirrhosis model (Table 3). For example, the Harrell’s C index in the validation half of the dataset was 0.74 for ALD-cirrhosis and 0.72 for NAFLD-cirrhosis. Predicted versus observed cumulative incidence curves showed great overlap (Figure 2). The calibration slope was excellent for all models as was the prediction accuracy measured by the Integrated Brier Score. The AUROC in the validation-half of the dataset was ~0.75, which is considered very good.

Table 3.

Measures of discrimination, calibration, and overall model accuracy for the different models we developed to predict HCC. The measures are shown separately for the derivation and validation datasets.

	Discrimination			Calibration	Accuracy
	Gonen and Heller’s κ-statistic	Royston and Sauerbrei’s D-statistic	Harrell’s C	Calibration slope	Integrated Brier score	AUROC
ALD-cirrhosis
k-fold cross-validation	0.737	1.452	0.743	0.947	0.04	0.750
Derivation-half	0.741	1.546	0.755	1	0.034	0.764
Validation-half	0.74	1.431	0.740	0.913	0.047	0.740
NAFLD-cirrhosis
k-fold cross-validation	0.73	1.379	0.720	0.888	0.037	0.739
Derivation-half	0.739	1.695	0.749	1	0.033	0.775
Validation-half	0.736	1.236	0.718	0.74	0.040	0.721
ALD- or NAFLD-cirrhosis
k-fold cross-validation	0.733	1.436	0.739	0.957	0.039	0.749
Derivation-half	0.742	1.58	0.757	1	0.033	0.761
Validation-half	0.742	1.335	0.727	0.845	0.046	0.738
Toronto HCC Risk Score40			0.669			0.679
FIB-4 score			0.701			0.713
MELD score			0.598			0.527

Gonen and Heller’s κ-statistic is a concordance measure and a value of 1 indicates perfect discrimination.

Royston and Sauerbrei’s D-statistic is a hazard ratio and the greater than 1 the greater the discrimination.

Harrell’s C statistic is a concordance measure and a value of 1 indicates perfect discrimination.

A Calibration slope of 1 indicates perfect calibration.

An Integrated Brier score of 0 indicates perfect accuracy. It is the mean squared difference between the predicted probability and the actual outcome.

AUROC: Area under the receiver operating characteristic curve. A value of 1 indicates perfect accuracy

Figure 2. Predicted vs observed cumulative incidence of HCC based on predictive models developed for a. ALDcirrhosis; b. NAFLD-cirrhosis and c. ALD or NAFLD-cirrhosis.

Patients in each subgroup are divided into thirds (low, medium and high) based on the predicted risk. The plots show excellent overlap between observed and predicted cumulative incidence.

Using the published “Toronto HCC Risk Index”⁴⁰ to estimate HCC risk resulted in lower Harrell’s C index (0.67) and AUROC (0.68) than when using our models. The FIB-4 score, which is calculated as the product of 3 measurements (age, inverse platelet count and AST/√ALT) all of which are strong predictors of HCC, performed well with a Harrell’s C (0.70) and AUROC (0.71) that were slightly worse than our models. The MELD score performed poorly (Harrell’s C 0.598, AUROC 0.527).

Standardized Net Benefit of Model-Based HCC Surveillance Ascertained By Decision Curves

Decision curves plotted in Figure 3 show that the risk model-based screening strategy has superior standardized net benefit than the “screen-all” strategy if the screening threshold is >1.5% per year (or >7.5% over 5 years) for ultrasound-based screening, as recommended by AASLD guidelines⁴⁴. The decision curves also show that screening based on risk estimates derived from our model (i.e. screening patients who exceed a given risk threshold) is superior to the screen-all strategy for a wide range of plausible screening thresholds. For more intensive screening strategies, such as those using abbreviated MRI, which are proposed for patients at even higher HCC risk (e.g. >3% per year), the risk-based screening is superior to “screen-all”.

Figure 3. Decision curves comparing the standardized net benefit achieved by screening based on HCC risk predicted by the model (i.e. screening only patients who exceed a certain threshold probability – blue line) to the “screen-all” (green line) or “screen-none” (orange line) strategies.

The vertical axis shows standardized net benefit, which is the proportion of total possible net benefit and would be achieved by a risk model with 100% sensitivity and specificity.

The horizontal axis shows different 5-year HCC risk thresholds that might be used to recommend screening. For example, the AASLD recommends screening when annual HCC risk exceeds 1.5% in patients with cirrhosis, or 5-year risk exceeds 7.5%, marked with a vertical dotted line. The Figures illustrate that the net benefit of screening based on our models (blue line) is greater than the net benefit of the “screen-all” strategy (green line), for both patient groups. The Figures also show that for a wide range of plausible screening thresholds the risk model-based screening has superior standardized net benefit than the screen-all strategy.

Web-Based HCC Risk Estimating Tools (available at www.hccrisk.com)

We implemented the two models shown in Table 2 for ALD-cirrhosis and NAFLD-cirrhosis as web-based tools to allow clinicians to estimate HCC risk in individual patients. The models can be executed at www.hccrisk.com. Table 4 shows 5-year HCC risk estimated by our models using the baseline characteristics shown in selected patients in “low-risk” (annualized risk 0–1%), “medium-risk” (annualized risk >1–3%) and “high-risk” (annualized risk >3%) categories. Table 4 shows that 5-year risk can vary from as low as 0.4% to as high as 30% depending on the values of the 7 simple predictors included in our models. We envision that the web-based HCC risk calculator will be used in the future to risk-stratify patients into low, medium and high-risk categories for the purposes of outreach efforts to improve screening uptake, development of future risk-based screening strategies and selection of patients for clinical trials of screening (Figure 4).

Table 4.

Estimates of 5-year HCC risk calculated by our web-based models for selected patients in the “low-risk”, “medium-risk” and “high-risk” categories.

	LOW-RISK PATIENTS 5-year HCC risk <5% Annual HCC risk <1%				MEDIUM-RISK PATIENTS 5-year HCC risk 5% to 15% Annual HCC risk 1 to 3%				HIGH-RISK PATIENTS 5-year HCC risk >15% Annual HCC risk >3%
Cirrhosis etiology	NAFLD	ALD	NAFLD	ALD	NAFLD	ALD	NAFLD	ALD	NAFLD	ALD	NAFLD	ALD
Age	61	67	62	62	66	60	62	66	63	62	63	62
Sex	F	M	M	M	F	M	M	M	M	M	M	M
Diabetes	No	No	Yes	No	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes
BMI	31	28	31	28	35	23	35	24	37	29	37	29
Albumin	4.2	4.2	3.8	3.9	3.6	3.9	3.5	3.1	3.8	3.4	3.4	3.3
Serum AST	30	30	30	30	45	50	55	40	55	50	55	50
Serum ALT	35	30	35	30	20	20	20	20	25	20	25	20
Platelet Count	225	170	170	120	85	110	115	180	90	110	85	50
Estimated 5-year HCC Risk	0.40%	2.6%	3.5%	4.6%	6.41%	10.0%	12.8%	14.4%	15.5%	20.6%	21.3%	30.2%

Figure 4. HCC risk stratification using prediction models.

Risk estimation models can be used to stratify risk in patients. Outreach efforts to improve screening uptake, future screening strategies and clinical trials can utilize the categorization of patients into low, medium and high-risk categories.

DISCUSSION

We demonstrated that HCC risk (i.e. incidence) varies dramatically in patients with ALD-cirrhosis or NAFLD-cirrhosis, which questions the utility of having a single screening strategy for all patients. We developed and internally validated models that accurately estimate HCC risk in patients with NAFLD-cirrhosis or ALD-cirrhosis according to 7 readily available characteristics: age, sex, BMI, diabetes, platelet count, serum albumin and serum AST/√ALT ratio. HCC surveillance strategies based on HCC risk estimates derived from our models resulted in greater predicted net benefit that the current “one-size-fits-all” strategy. Our models can be used to stratify patients according to estimated annual HCC risk into low-risk (e.g. annual risk <1%), medium-risk (e.g. annual risk 1–3%) and high-risk (e.g. annual risk >3%). Such risk stratification can be used to select patients for clinical trials, to inform screening outreach efforts and to develop risk-based HCC screening strategies, such as more intensive screening strategies (e.g. abbreviated MRI) for high-risk patients. Future research should evaluate specific risk-based strategies, such as screening by abbreviated MRI versus ultrasonography for patients in the high-risk category. Our models are available as web-based tools that can be used to estimate HCC risk in individual patients (www.hccrisk.com).

HCC risk varies widely in cirrhotic patients⁶. In the patients depicted in Table 4, estimated HCC risk varied by more than 30-fold depending on the distribution of readily available HCC risk factors. Despite this, national guidelines recommend a “one-size-fits-all” surveillance strategy, whereby the same screening strategy (biannual ultrasonography ± AFP) is recommended for all patients with cirrhosis¹ irrespective of HCC risk. We believe that great improvements in HCC screening can be achieved if patients are first stratified according to HCC risk and then offered risk-appropriate screening (Figure 4). However, no widely accepted method is available for estimating HCC risk in cirrhotic patients. Accurate estimation of HCC risk by the models that we developed could potentially improve HCC surveillance efforts, increase early detection of HCC, reduce harms related to unnecessary surveillance and facilitate the design of future HCC surveillance trials. Patients at high risk of HCC could be targeted for interventions to improve their uptake of HCC surveillance. It is currently estimated that ≤20% of cirrhotic patients undergo surveillance consistent with guidelines in the United States^{45, 46}. Different surveillance strategies could potentially be proposed for different categories of HCC risk. For example, more effective strategies that are also more expensive or more invasive/harmful, such as screening by MRI, abbreviated MRI or CT, would be more cost-effective if they focus on the higher risk groups⁴⁷. In healthcare systems with limited resources unable to support universal surveillance of all cirrhotic patients, surveillance could be limited to patients with higher HCC risk. Past AASLD guidelines recommended that “for patients with cirrhosis, surveillance should be offered when the risk of HCC is >1.5% per year”⁴⁴. If the HCC risk could be accurately predicted, those with incidence well below 1.5% may not be recommended surveillance thus reducing costs and potential downstream harms such as unnecessary CT scans (radiation and iv contrast), liver biopsies and other procedures (assuming that the risk threshold of >1.5% is indeed valid)⁴⁸. Estimation of HCC risk could enable individualized counseling of patients by their providers leading to improved compliance with surveillance recommendations and engagement in care. Finally, our models could be used to identify high-risk patients for participation in clinical trials of HCC surveillance or HCC risk modifiers.

We hope that the strategy of HCC risk stratification will be utilized more widely in patients with cirrhosis by individual physicians for individual patients or by healthcare systems for patient populations. To that end we created web-based tools to execute our models and estimate HCC risk in patients with NAFLD-cirrhosis or ALD-cirrhosis available at www.hccrisk.com. We also provided all the coefficients and baseline cumulative incidence rates of the models (Supplemental Tables 2 and 3) to allow other investigators to execute or externally validate the models. These new models extend the models we published recently estimating HCC risk in patients with HCV infection⁴⁹, also available on the same website.

Notable efforts have been made recently to develop models estimating HCC risk in patients with cirrhosis using readily available predictors^{40, 50}. The best example is probably the “Toronto HCC risk index”, which combined 4 predictors (age, sex, etiology of liver disease, and platelet count) into an HCC risk prediction model that performed reasonably well and was externally validated⁴⁰. However, the majority of the patients in that study (1279 out of 2079) had viral hepatitis B and C, in whom antiviral treatment dramatically reduces HCC risk, a factor not captured in the prediction model. We believe that distinct models need to be developed for cirrhotic patients with cured HCV⁴⁹ and adequately treated HBV as well as for patients with NAFLD and ALD-cirrhosis, as we have done. Indeed, when we used the “Toronto HCC risk index” in our study population we obtained a lower Harrell’s C index (0.67) than what was reported in their study population (0.76). The “Adress-HCC” model was developed among patients on liver transplant waitlists and used six baseline clinical variables to estimate HCC risk (age, diabetes, race, etiology of cirrhosis, sex, and severity of liver dysfunction)⁵⁰. However, median follow-up was short (1.3 years) and patients were limited to those waitlisted for transplantation. The field of HCC risk calculation is still in its infancy and it is likely the models will continue to evolve, possibly incorporate novel biomarkers, and likely developing different models for distinct subpopulations of patients with cirrhosis.

Decision curves plot the standardized net benefit that would be expected from risk-based screening strategies at different “appropriate” HCC risk thresholds for screening. Figure 3 shows that at a threshold of >1.5% per year (or 7.5% per 5 years), which is commonly recommended in patients with cirrhosis⁵¹, the net benefit is greater with screening based on our models (i.e. screening only patients with estimated HCC risk>1.5% per year) compared with screening all patients. However, if the appropriate risk threshold is much lower (<0.8% per 5 years) then there is no difference between the screen-all and model-based screening strategies. It is important to emphasize that decision curves cannot be used to determine the appropriate HCC risk threshold at which screening is deemed to be beneficial. Instead, this threshold needs to be determined by estimating the harms of missing a case relative to the harms of unnecessarily screening a non-case. Decision analytic theory suggests that if the harms of missing a case are x-times greater than the harms of unnecessarily screening a non-case, then the appropriate threshold for screening is a risk exceeding 1/(x+1)⁵². Therefore, the greater the harms of missing a case (or the greater the benefits of diagnosing a case) the lower the risk threshold at which screening is beneficial. Conversely, the greater the harms of screening the higher the risk threshold. Decision curves have been used before to evaluate the value of predictive models for recommending CT screening for lung cancer⁵³, but not for HCC screening.

A major limitation of our study is that we have not externally validated our models yet in non-VA populations. We suspect that our models may predict well in other similar populations i.e. unselected, predominantly male patients with cirrhosis not limited to transplant centers or tertiary referral centers. However, other models such as the “Toronto HCC risk index” ⁴⁰ or the “Adress-HCC”⁵⁰ model may predict better in patients at tertiary referral or transplant centers similar to the ones used for their model development. It will be critical to externally validate our models in non-VA populations and also ideally in populations undergoing routine HCC surveillance (although the frequency of surveillance does not affect the incidence (or risk) of HCC in a given population). It is possible that ultimately different models might be developed for different subpopulations of patients with cirrhosis, just as we have already developed different models for HCV-infected patients undergoing antiviral treatment. Substantial strengths of the study include the large sample size, large number of incident HCCs, recent time period and appropriate length of follow-up. Baseline characteristics necessary for modeling were available. All patients were derived from a single, national healthcare system with fairly uniform practices and guidelines across its facilities.

In conclusion, we developed and internally validated models estimating HCC risk in patients with ALD-cirrhosis and NAFLD-cirrhosis. These models, which are available as web-based tools, can help stratify patients according to HCC risk and, consequently, help determine an appropriate screening strategy based on a patient’s calculated risk. An intensive screening strategy targeting those who exceed a certain predetermined HCC risk may be more efficacious and cost-effective than the current “screen-all” or “screen-none” strategies which depend solely on cirrhosis status.

• HCC risk varies dramatically in patients with cirrhosis

• We developed models estimating HCC risk in patients with NAFLD-cirrhosis or ALD-cirrhosis

• The models use simple, readily available predictors

• The models are available as web-based tools at www.hccrisk.com

Abbreviations

ALD

Alcoholic Liver Disease

ALT

Alanine aminotransferase

AST

Aspartate aminotransferase

BMI

Body mass index

HCC

Hepatocellular Carcinoma

HCV

Hepatitis C Virus

NAFLD

Nonalcoholic Fatty Liver Disease

VA

Veterans Affair