Development of Models Estimating the Risk of Hepatocellular Carcinoma After Antiviral Treatment for Hepatitis C

Abstract

Background and Aims.

Most patients with hepatitis C virus (HCV) infection will undergo antiviral treatment with direct-acting antivirals (DAA) and achieve sustained virologic response (SVR). We aimed to develop models estimating HCC risk after antiviral treatment.

Methods.

We identified 45,810 patients who initiated antiviral treatment in the Veterans Affairs (VA) national healthcare system from 1/1/2009 to 12/31/2015, including 29,309 (64%) DAA-only regimens and 16,501(36%) interferon ± DAA regimens. We retrospectively followed patients until 6/15/2017 to identify incident cases of HCC. We used Cox proportional hazards regression to develop and internally validate models predicting HCC risk using baseline characteristics at the time of antiviral treatment.

Results.

We identified 1412 incident cases of HCC diagnosed at least 180 days after initiation of antiviral treatment during a mean follow-up of 2.5 years (range 1–7.5 years). Models predicting HCC risk after antiviral treatment were developed and validated separately for four sub-groups of patients: cirrhosis/SVR, cirrhosis/no SVR, no cirrhosis/SVR, no cirrhosis/no SVR. Four predictors (age, platelet count, serum AST/√ALT ratio and albumin) accounted for most of the prediction with smaller contributions from sex, race-ethnicity, HCV genotype, body mass index, hemoglobin and serum alpha fetoprotein. Fitted models were well-calibrated with very good measures of discrimination. Decision curves demonstrated higher net benefit of using model-based HCC risk estimates to determine whether to recommend screening or not compared to the screen-all or screen-none strategies.

Conclusions.

We developed and internally validated models that estimate HCC risk following antiviral treatment. These models are available as web-based tools that can be used to inform risk-based HCC surveillance strategies in individual patients.

Introduction

Most patients with chronic hepatitis C virus (HCV) infection have either already received antiviral treatment or are expected to receive treatment with direct-acting antivirals (DAAs) in the next 3–5 years in the United States. With sustained virologic response (SVR) rates well in excess of 90%, the vast majority of treated patients will achieve HCV eradication. SVR reduces hepatocellular carcinoma (HCC) risk substantially, irrespective of whether it is achieved by interferon (IFN) or DAA-based regimens¹. It follows that HCC risk needs to be estimated specifically for the time period following antiviral treatment incorporating whether SVR was achieved or not, and that previous models predicting HCC risk in untreated HCV-infected patients do not apply to patients who have undergone antiviral treatment.

Current guidelines recommend the same screening strategy for all HCV-infected patients with cirrhosis (ultrasonography every 6 months ± serum alpha fetoprotein [AFP] testing) while no screening is recommended for non-cirrhotic patients, regardless of their HCC risk². This “one-size-fits-all” strategy raises many questions in the DAA era and leaves room for improvements. For example, a patient with cirrhosis may have favorable characteristics that, together with HCV eradication, substantially lower the patient’s HCC risk. Since surveillance is thought to increase survival or become cost-effective in cirrhotic patients only when HCC risk exceeds 1.5% per year^{3, 4}, surveillance may not be warranted in such a patient. Conversely, in cirrhotic patients who fail antiviral treatments and/or have additional adverse characteristics, HCC risk may be so high that more aggressive surveillance strategies like annual MRI, abbreviated MRI⁵ or CT become more efficacious or cost-effective than ultrasonography⁶. Furthermore, patients without established cirrhosis who fail antiviral treatment and have additional adverse characteristics, may have HCC risk sufficiently high to merit screening. However, no method is currently available to estimate HCC risk in these patients.

Central to these considerations is the concept that surveillance confers harms to patients who do not have HCC (or will not develop HCC in the timeframe of interest) as well as benefits to those who have (or will develop) HCC. Such harms include unnecessary anxiety, biopsies, imaging studies or even treatments. Therefore, HCC surveillance should not be recommended for every patient, but instead only for patients whose risk exceeds a predetermined risk threshold. It can be shown that an appropriate risk threshold depends on the ratio of the harms associated with a missed cancer to the harms associated with unnecessary screening^{7, 8}. For example, if surveillance is recommended for an annual HCC risk >2% it means that we consider the harms of missing a cancer to be approximately 50 (or 98/2) times greater than the harms of unnecessary screening. The appropriate risk threshold is likely different in different clinically relevant subgroups of patients such as those with/without cirrhosis and with/without SVR).

We aimed to develop and validate models estimating HCC risk in HCV-infected patients following antiviral treatment separately in the following four clinically relevant sub-groups: cirrhosis/no SVR; cirrhosis/SVR; no cirrhosis/no SVR; no cirrhosis/SVR. Additionally, we used decision curves⁷ to evaluate the net benefit that would be derived by implementing HCC surveillance strategies based on HCC risk as compared to screen-all or screen-none strategies. Finally, we wanted to develop HCC risk prediction models that would be available to clinicians as web-based tools so that HCC risk can be readily estimated in clinical practice.

Methods

Data Source

The Veterans Health Administration (VHA) is the largest integrated healthcare system in the US currently serving more than 8.9 million Veterans at 168 VA Medical Centers and 1053 outpatient clinics throughout the country⁹. The VHA uses a single, nationwide, comprehensive electronic healthcare information network (known as the Veterans Information Systems and Technology Architecture or VistA), which consists of nearly 180 applications of clinical, financial, administrative and infrastructure needs integrated into a single, common database of all Veterans’ health information.

We obtained electronic data on all patients who initiated antiviral treatment in the VA system using the VA Corporate Data Warehouse (CDW), a national, continually updated repository of data from VistA developed specifically to facilitate research¹⁰. Data extracted included all patient pharmacy prescriptions, demographics, inpatient and outpatient visits, problem lists, procedures, vital signs, diagnostic tests, and laboratory tests.

The study was approved by the Institutional Review Board of the VA Puget Sound Healthcare System.

Study Population and Study Period

We identified all HCV antiviral regimens (n=58,936 regimens in 50,257 patients) initiated in the VA during 7 calendar years from 1/1/2009 to 12/31/2015. We excluded 1324 patients who had a diagnosis of HCC (ICD-9 code 155.0 or ICD-10 code C22.0) recorded prior to HCV antiviral treatment. We additionally excluded 625 patients who either died within 180 days from the start of antiviral treatment or had fewer than 180 days of available follow-up, and 276 patients who were diagnosed with HCC within 180 days from the start of antiviral treatment (including 154 who achieved SVR, 82 who did not, and 40 with missing SVR) since these cases were very unlikely to be incident (new) cases. We finally excluded 2222 patients with missing SVR data leaving 45,810 patients in the current analysis, including 1412 who developed HCC at some point from 180 days after the treatment start-date until the end of follow-up on 6/15/2017.

We excluded antiviral treatments prior to 2009 because multiple studies have documented an increase in HCC incidence over time in HCV-infected patients¹¹. Since we aimed to predict the absolute HCC risk in current patients, we chose the most recent possible sample (2009–2015) that provided adequate length of follow-up (maximum follow-up of 8 years, mean follow-up of 2.52 years) to enable robust estimation of HCC incidence extending up to 3 years. We recently demonstrated using the same datasets that HCC risk after antiviral treatment was similar in patients treated with DAAonly regimens from 2014–2015 and in patients treated with interferon-based regimens in 2009–2013¹, thus justifying combining all antiviral treatments for risk modeling. Sufficient time has not yet accrued since the introduction of DAAonly regimens to enable an analysis limited only to these regimens. DAA-only regimens had a mean follow-up of only 1.5 years in our dataset.

Antiviral Treatment Regimens

The regimens were divided into:

1. Interferon only (“IFN-ONLY”) regimens (22.5%): included pegylated interferon (PEG) ± ribavirin but without any DAAs.

2. “DAA+IFN” regimens (13.5%): included any DAA (NS3/4A, NS5A or NS5B inhibitors) with concomitant PEG ± ribavirin. The most common was boceprevir +PEG.

3. “DAA-ONLY” regimens (64%): included only interferon-free, DAA regimens (± ribavirin). The most common was ledipasvir/sofosbuvir.

All VA pharmacy data are included in the CDW; dispensed drugs (rather than just prescribed drugs) were used to define antiviral treatment regimens, as previously described^12–19. Supplemental Table 1 shows the distribution of all regimens included in the study.

Sustained Virologic Response (SVR)

We defined SVR as a serum HCV RNA viral load below the lower limit of detection performed at least 12 weeks after the end of HCV treatment²⁰.

Baseline Patient Characteristics

We collected baseline data including age, sex, body mass index (BMI), HCV genotype, HCV viral load and receipt of prior antiviral treatment. We extracted all laboratory tests shown in Table 1 prior to treatment and recorded the value of each test closest to the treatment starting date within the preceding 6 months (except serum AFP that was recorded within 1 year).

Table 1.

Baseline characteristics of HCV-infected patients who initiated antiviral treatment from 2009–2015, according to cirrhosis and SVR status.

	All Patients (N=45,810)	CIRRHOSIS		NO CIRRHOSIS
		No SVR (n=3074)	SVR (n=7689)	No SVR (n= 8640)	SVR (n= 26,407)
Age, yrs (mean [SD])	59.3 [7.0]	58.9 [5.6]	61.5 [5.5]	56.8 [6.9]	59.6 [7.4]
BMI, Kg/m2 (mean [SD])	28.2 [5.3]	29.2 [5.5]	28.7 [5.4]	28.3 [5.3]	27.9 [5.2]
Male (%)	96.6	97.5	97.2	96.8	96.3
Race/Ethnicity (%)
White, non-Hispanic	54.9	55	56.2	52.3	55.4
Black, non-Hispanic	29.8	24.3	26.3	32.6	30.5
Hispanic	5.7	10.1	7.6	6.1	4.5
Other	1.7	1.9	1.8	1.7	1.6
Declined to answer/missing	7.9	8.7	8.2	7.4	7.9
Antiviral Regimen
IFN ONLY	22.5	38.5	5.4	58.1	14
DAA + IFN	13.5	27.3	10.7	20.4	10.4
DAA ONLY	64	34.2	83.9	21.5	75.6
Treatment experienced	14.7	28	21.5	15.7	10.9
Genotype (%)
Genotype 1	79.2	79.1	84.6	75.6	78.9
Genotype 2	10.7	7.7	7.4	10.7	12
Genotype 3	6.5	9.8	5.5	8.4	5.8
Genotype 4	0.8	0.7	0.7	0.9	0.7
Missing	2.8	2.8	1.8	4.4	2.6
HCV RNA Viral load >6 million IU/mL (%)	19.6	17.9	15.4	23.7	19.7
HIV co-infection	3.8	2.5	3	3.5	4.3
HBV co-infection	1.3	1.3	1.8	1.1	1.1
Decompensated Cirrhosis (%)	6.5	30.9	26.2	N/A	N/A
Liver Transplantation (%)	1.5	3	4.9	0.4	0.7
Diabetes (%)	26.8	35.1	37.7	23.9	23.7
Alcohol Use Disorder (%)	43.7	48.6	47.9	44.4	41.7
Substance Use Disorder (%)	37	34	35.3	39.5	37.1
Laboratory Results (mean [SD])
Alpha Fetoprotein, ng/mL	6.1 [4.2]	8.2 [4.6]	7.4 [4.6]	6.1 [4.2]	5.4 [3.8]
Hemoglobin, g/dL	14.6 [1.6]	14.3 [1.7]	14.1 [1.7]	14.9 [1.5]	14.8 [1.5]
Platelet Count, k/μL	181[70]	127 [59]	134 [64]	193 [65]	197 [64]
Creatinine, mg/dL	1.0 [0.5]	1.0 [0.6]	1.0 [0.5]	1.0 [0.8]	1.0 [0.5]
Bilirubin, g/dL	0.7 [0.5]	1.0 [0.8]	0.9 [0.7]	0.6 [0.4]	0.6 [0.4]
Albumin g/dL	3.9 [0.5]	3.6 [0.6]	3.6 [0.5]	4.0 [0.4]	4.0 [0.4]
INR	1.2 [1.0]	1.3 [1.2]	1.3 [1.2]	1.1 [1.0]	1.1 [0.9]
AST/√ALT	7.5 [3.2]	9.5 [3.9]	8.9 [3.7]	7.3 [3.0]	6.9 [2.8]

We contemplated ascertaining laboratory tests after treatment completion but decided against that because many laboratory tests can change acutely as a result of treatment and, thus, may reflect underlying fibrosis or HCC risk less accurately. Furthermore, laboratory tests are routinely obtained in most patients in clinical practice at the beginning of treatment but not at any specified time point after treatment. Therefore, risk prediction models relying on pretreatment measurements have the greatest potential to be clinically useful.

We defined HBV coinfection by positive HBV surface antigen or viral load. We also determined the presence of cirrhosis, decompensated cirrhosis (ascites, encephalopathy, gastroesophageal varices and hepatorenal syndrome), type 2 diabetes mellitus, alcohol use disorders, substance use disorders, HIV infection and liver transplantation based on appropriate ICD-9 or ICD-10 codes recorded at least twice prior to treatment initiation in any inpatient or outpatient encounter (Supplemental Table 2). These ICD-based definitions of cirrhosis and other comorbidities^{11, 21–25} have been widely used and validated in studies using VA medical records.

Incident Hepatocellular Carcinoma

We identified incident cases of HCC diagnosed for the first time at least 180 days after initiation of antiviral treatment based on ICD-9 code 155.0 or ICD-10 code C22.0 documented at least twice. The ICD-9 code-based definition of HCC using VA records has been shown to have a positive predictive value of 84–94% compared to chart extraction^{24, 26, 27} and has been widely used by us^{11, 16, 28, 29} and other investigators^30–32.

We also identified all serum AFP tests, abdominal ultrasound scans (USS), abdominal computerized tomography (CT) scans with intravenous contrast, and abdominal magnetic resonance imaging (MRI) scans with intravenous contrast performed before and after antiviral treatment to evaluate how frequently screening and diagnostic tests for HCC were being performed.

Statistical Analysis

We developed four different Cox proportional hazards models estimating HCC risk after antiviral treatment in four patient subgroups: cirrhosis/no SVR; cirrhosis/SVR; no cirrhosis/no SVR; no cirrhosis/SVR. Cox proportional hazards models were developed based on the first antiviral treatment that each patient received during the study period. Follow-up time started at 180 days after treatment initiation since cancers diagnosed within 180 days were likely present but undiagnosed at the time of treatment initiation (i.e. not truly “incident” cancers). We considered using the date treatment ended or the date at which SVR was ascertained as starting points for the time-to-event analysis, but decided against that because of the long and variable duration of the treatment and the interval from treatment enddate to ascertainment of SVR, which could introduce significant bias.

Follow-up for HCC incidence extended until 6/15/2017 so that even the patients treated in 2015 (i.e. the most recent in our cohort) would have minimum of 2 years of potential follow-up. Patients without incident HCC were censored at the time of death or last follow-up in the VA. Patients who did not achieve SVR were censored at initiation of a subsequent regimen that led to SVR, if applicable. Analyses were stratified by the VA facility at which the antiviral treatment was administered.

We considered 23 characteristics listed in Table 1 as potential predictors of HCC for inclusion in our models. As expected, serum AFP was missing in a large proportion (40.7%), since it is not recommended to test for AFP in HCVinfected patients without cirrhosis. In addition, serum AFP testing for HCC screening in patients with cirrhosis was either not recommended by EASL and AASLD guidelines³ or optional² during the study period. Therefore, we imputed missing AFP values and developed separate models that included AFP, which we considered exploratory. We estimated the explained relative risk (ERR) contribution of a subset of predictors to the overall model’s predicted risk³³. The ERR was selected because it is robust to censoring.

Model Building

We used an iterative process to determine which predictors to include in our final models. First, we estimated measures of discrimination, calibration, and significance when each predictor was added to the base model and identified the top 5 predictors with the greatest improvement in these measures. We chose predictors that were consistently in the top 5 with preference for p-values < 0.10 and improvement in the Gönen and Heller’s κ-statistic. We verified graphically that the added predictor improved the observed vs. predicted risk plot thus allowing assessment over the entire time period.

We then updated the base model to include the chosen predictors and removed any predictors with a p-value < 0.10; removed predictors were added back into the list of potential predictors. We favored variables for inclusion that were objectively ascertained (e.g. laboratory tests) and those that have been consistently associated with HCC in previous studies (e.g. sex).

The measures that we used to evaluate each predictor were Gönen and Heller’s κ-statistic, Hosmer-Lemeshow’s χ2 goodness-of-fit (GOF), Akaike Information Criterion (AIC) (discrimination and calibration), area under the receiver operating curve (AUROC), Spearman’s correlation (ρ) (raw and categorical), and the p-value. Hosmer-Lemeshow GOF and AUROC measures were derived from a logistic regression of model predictions and a diagnosis of HCC. For discrimination, the AIC was calculated from the Cox proportional hazards model. For calibration, the AIC was estimated from a multivariate logistic regression of Kaplan-Meier survival probability and the predicted risk group. Spearman’s ρ was calculated for Kaplan-Meier survival probability versus the model prediction (raw) or categorized (low, medium, or high) model predictions. A graphical comparison of observed vs. predicted risk scores was generated. A pooled k-fold cross-validation was used to calculate all the above measures and determine inclusion of predictors in the final model. A k of 10 was chosen to address the bias versus variability in a database with a large sample size, but relatively few events.

We considered both dummy-categorical as well as continuous (linear or transformed) modeling of laboratory tests. Interaction terms were explored if there was biological indication. The distribution of model predictions was checked for normality. Once a model was determined, the dataset was split in half into derivation and validation datasets balanced on number of events. Measures of assessment were then calculated for each dataset using model coefficients from the derivation data.

Measures of Model Discrimination and Calibration

We evaluated our models’ discrimination (i.e. ability to separate those who will develop HCC from those who will not), calibration (i.e. degree of agreement between model-derived and observed probabilities), and overall predictive accuracy. The measures of discrimination chosen were Gönen and Heller’s κ-statistic³⁴ (a measure of concordance that is robust to censoring and therefore preferred to the Harrell’s C-index³⁵ for survival data), and Royston and Sauerbrei’s Dstatistic³⁶ (the log hazard ratio of risk between low and high risk groups dichotomized at their median values, which has negligible bias when the distribution of model predictions is normal). For calibration measures, the calibration slope³⁷ and graphical methods were selected. Calibration slope is robust to censoring and ideally takes a value of 1. To evaluate calibration graphically, observed Kaplan-Meier estimates of HCC-free survival and lowess-smoothed model predictions of HCC-free survival were plotted after categorizing risk into low, medium, or high groups. Overall model prediction accuracy was evaluated using the integrated Brier score (IBS)³⁸, which is the mean squared difference between the predicted probability and the actual outcome.

Use of Decision Curves to Estimate the Net Benefit of Using our Risk Prediction Models

We used decision curves to estimate the net benefit that would be expected in a population if our models are used to estimate HCC risk and patients are screened when their estimated risk exceeds an established risk threshold, as compared to the “screen-all” or “screen-none” approaches. A risk threshold is defined as that probability of HCC above which screening would be favorable over not screening. A decision curve is a novel graphical plot of net benefit versus risk threshold that was proposed for assessing the potential population impact of adopting a risk prediction instrument⁸. To avoid over-fitting, decision curves were calculated using repeated 10-fold cross-validation⁸ The cross-validation was repeated 50 times.

Results

Characteristics of Study Population

Among 45,810 patients who initiated HCV antiviral treatment from 1/1/2009 to 12/31/2015, 10,763 (23%) had cirrhosis and 34,096 (74%) achieved SVR (Table 1). Most treatments were DAA-only (64%), followed by IFN-only (22.5%) and DAA+IFN (13.5%). Patients were mostly male (96.6%) and White (55.9%), though other racial/ethnic groups were wellrepresented. Mean age was 55.8 yrs. Diabetes (27%), alcohol use disorders (43.7%) and substance use disorders (37%) were common. Genotype 1 HCV infection predominated (79.2%) followed by genotype 2 (10.7%), 3 (6.5%) and 4 (0.8%).

Compared to patients without cirrhosis, those with cirrhosis had lower platelet count and serum albumin, higher AST/√ALT ratio, bilirubin, INR and AFP levels and were more likely to be diabetic. Patients who achieved SVR more likely to be treated with DAA-only regimens and less likely to be treatment-experienced than the patients who did not achieve SVR.

Screening/diagnostic tests for HCC such as abdominal USS, CT with contrast or MRI with contrast were commonly performed within 1 year prior to antiviral treatment (ranging from 79.7% of cirrhotic patients with SVR to 49.1% in noncirrhotic patients without SVR) as was serum AFP testing (ranging from 71.5% to 47.6%) – Supplemental Table 3.

HCC Incidence by Cirrhosis and SVR Status

During a mean follow-up period of 2.52 years (range 1–7.5 years), 1297 out of 45,810 patients (2.8%) developed HCC (Table 2 and Figure 1). HCC incidence was highest in the cirrhosis/no SVR subgroup (5.0 per 100 patient-years), followed by cirrhosis/SVR (2.2 per 100 patient-years), no cirrhosis/no SVR (1.1 per 100 patient-years), and no cirrhosis/SVR (0.3 per 100 patient-years).

Table 2. Four models developed to predict HCC following antiviral treatment, separately in patients with or without cirrhosis and with or without SVR.

The table shows adjusted hazard ratios (and their p-values) for each predictor included in the models.

CIRRHOSIS			NO CIRRHOSIS
PREDICTORS	No SVR (n=3074)	SVR (n=7689)	PREDICTORS	No SVR (n=8640)	SVR (n=26,407)
Sex			Sex
Male	-	-	Male	1	1
Female	-	-	Female	0.16(0.07)	-
Age, yrs			Age, yrs
≤ 56	1	1	≤ 56	1	1
>56 – 60	0.93(0.57)	1.64(0.02)	>56 – 60	2.32(< 0.001)	1.77(0.02)
>60 – 64	1.28(0.09)	2.01(< 0.001)	>60 – 64	3.34(< 0.001)	2.72(< 0.001)
>64 – 67	1.92(< 0.001)	2.43(< 0.001)	>64 – 67	3.16(< 0.001)	2.47(< 0.001)
> 67	1.63(0.06)	2.59(< 0.001)	> 67	5.36(< 0.001)	2.58(< 0.01)
BMI, Kg/m2			BMI, Kg/m2
< 20	0.57(0.30)	-	< 20	0.70(0.51)	0.81(0.67)
20– 25	1	-	20– 25	1	1
25 – 30	0.89(0.39)	-	25 – 30	1.37(0.02)	0.76(0.14)
30– 35	0.69(0.01)	-	30– 35	0.87(0.42)	1.01(0.98)
> 35	0.76(0.12)	-	> 35	0.82(0.42)	0.39(0.02)
Race/Ethnicity			Race/Ethnicity
White, non-Hispanic	1	1	White, non-Hispanic	-	-
Black, non-Hispanic	0.90(0.49)	0.52(< 0.001)	Black, non-Hispanic	-	-
Hispanic	1.02(0.93)	0.82(0.39)	Hispanic	-	-
Other	2.11(0.02)	0.74(0.49)	Other	-	-
Declined to answer, missing	0.76(0.23)	0.79(0.24)	Declined to answer, missing	-	-
HCV Genotype			HCV Genotype
Non-3	-	-	Non-3	1	1
Genotype 3	-	-	Genotype 3	1.88(< 0.001)	1.81(0.01)
Hemoglobin, g/dL			Hemoglobin, g/dL
> 15.7	-	-	> 15.7	1	-
>14.8 – 15.7	-	-	>14.8 – 15.7	1.01(0.97)	-
>13.7 – 14.8	-	-	>13.7 – 14.8	0.88(0.39)	-
>12.7 – 13.7	-	-	>12.7 – 13.7	0.90(0.59)	-
≤12.7	-	-	≤12.7	0.52(0.03)	-
Platelet count, k/μL			Platelet count, k/μL
> 167	1	1	> 234	1	1
>123 – 167	1.21(0.33)	1.14(0.49)	>192 – 234	0.80(0.32)	0.95(0.86)
>87 – 123	1.40(0.09)	1.37(0.10)	>153 – 192	1.20(0.36)	0.86(0.59)
>61 – 87	2.17(< 0.001)	2.12(< 0.001)	>120 – 153	2.27(< 0.001)	1.96(0.01)
≤61	2.06(< 0.01)	2.44(< 0.001)	≤ 120	2.19(< 0.001)	2.43(< 0.01)
Albumin, g/dL			Albumin, g/dL
> 4	1	1	> 4.3	1	1
>3.7 – 4	1.11(0.59)	1.30(0.20)	>4.0 – 4.3	1.07(0.74)	0.82(0.51)
>3.3 – 3.7	1.64(< 0.01)	1.66(< 0.01)	>3.8 – 4.0	1.13(0.55)	1.25(0.46)
>2.9 – 3.3	2.62(< 0.001)	1.97(< 0.01)	>3.5 – 3.8	1.39(0.10)	1.38(0.27)
≤2.9	2.17(< 0.001)	3.03(< 0.001)	≤3.5	2.01(< 0.01)	2.37(< 0.01)
INR			INR
≤1.0	-	-	≤1.0	-	1
>1.0 – 1.2	-	-	>1.0 – 1.18	-	1.46(0.04)
>1.2 – 1.34	-	-	> 1.18	-	1.15(0.64)
> 1.34	-	-		-	-
AST/√ALT			AST/√ALT
≤6.5	1	1	≤5.2	1	1
(6.5,8.49]	2.03(< 0.001)	1.44(0.05)	>5.2 – 6.31	1.69(0.04)	1.31(0.43)
(8.49,11.01]	2.25(< 0.001)	1.46(0.04)	>6.31 – 8.06	1.99(< 0.01)	2.05(0.03)
(11.01, 13.9]	2.42(< 0.001)	1.47(0.06)	>8.06 – 10.43	3.57(< 0.001)	4.31(< 0.001)
> 13.9	2.07(< 0.01)	1.16(0.53)	> 10.43	3.80(< 0.001)	4.19(< 0.001)

Figure 1.

a. Kaplan-Meier curves showing the development of HCC after antiviral treatment for HCV, by cirrhosis and SVR status

b. Incidence of HCC after antiviral treatment for HCV, by cirrhosis and SVR status.

Screening/diagnostic tests for HCC (abdominal USS, CT, MRI or serum AFP) were being performed commonly during follow-up ranging from 74.5% (in cirrhotic patients with SVR) to 40.7% (in non-cirrhotic patients without SVR) in followup year 0–1, 70.4% to 36.4% in year 1–2, and 62% to 31.7% in year 2–3 (Supplemental Table 4).

Development of Models Predicting HCC

Out of the 23 potential predictors that we considered (Table 1), eleven were included in at least one of the four models that we developed (Table 2). Of these, four predictors (age, platelet count, serum AST/√ALT ratio and albumin) accounted for most of the prediction. The proportion of the relative risk explained by these four predictors (explained relative risk³³) was 95% for the cirrhosis/no SVR model, 98% for the cirrhosis/SVR model, 87% for no cirrhosis/no SVR model, and 98.5% for no cirrhosis/SVR model. The following 6 predictors provided smaller contributions: sex, race/ethnicity, HCV genotype, BMI, hemoglobin, and INR. For most predictors, associations with HCC were stronger among patients without cirrhosis than patients with cirrhosis.

In exploratory models that included serum AFP or imputed AFP, serum AFP level was a significant predictor of HCC, especially in patients without cirrhosis (Supplemental Table 5). Adjusted hazard ratios for other predictors were not significantly affected by the addition of serum AFP into the model.

Predicted versus observed curves of probability free of HCC showed excellent correlation for three of the four models (cirrhosis/SVR; no cirrhosis/no SVR; and no cirrhosis/SVR) and moderate correlation in one model which was based on the highest risk subgroup (cirrhosis/no SVR) (Figure 2).

Figure 2. Predicted vs observed survival free of HCC diagnosis after antiviral treatment for HCV initiated between 2009 and 2015, based on predictive models developed in four subgroups:

a. Cirrhosis and no SVR

b. Cirrhosis and SVR

c. No Cirrhosis and no SVR

d. No Cirrhosis and SVR

Patients in each subgroup are divided into thirds (low, medium and high) based on the predicted risk

Measures of discrimination and calibration were higher for the models developed in patients without cirrhosis than in patients with cirrhosis (Table 3). Gönen and Heller’s κ-statistic was >0.74 in both the derivation and validation datasets in the models developed for non-cirrhotic patients with or without SVR. For models developed in patients with cirrhosis the Gönen and Heller’s κ-statistic was around 0.70 for the derivation and validation datasets. The Integrated Brier Score, a measure of overall accuracy, was remarkably good for all models.

Table 3.

Measures of discrimination, calibration, and overall model accuracy for the four different models we developed to predict HCC. The measures are shown separately for the derivation and validation datasets.

	Discrimination		Calibration	Accuracy
	Gonen and Heller’s κ-statistic	Royston and Sauerbrei’s D-statistic	Calibration slope	Integrated Brier Score
CIRRHOSIS
No SVR
Validation	0.70	1.118	0.8	0.104
Derivation	0.70	1.303	1.0	0.104
SVR
Validation	0.70	0.786	0.63	0.043
Derivation	0.70	1.203	1.0	0.047
NO CIRRHOSIS
No SVR
Validation	0.74	1.866	0.964	0.045
Derivation	0.75	2.000	1.0	0.036
SVR
Validation	0.77	1.299	0.614	0.018
Derivation	0.77	2.074	1.0	0.013

Gonen and Heller’s κ-statistic is a concordance measure and a value of 1 indicates perfect discrimination, while a value of 0.5 indicates no discrimination.

Royston and Sauerbrei’s D-statistic is a hazard ratio and the greater than 1 the greater the discrimination.

A Calibration slope of 1 indicates perfect calibration.

An Integrated Brier score of 0 indicates perfect accuracy.

Net Benefit of Model-Based HCC Surveillance Ascertained By Decision Curves

The decision curves confirm that for any appropriate risk threshold above which screening is recommended, the net benefit of screening is highest in patients with cirrhotics/no SVR (Figure 3a), followed by cirrhosis/SVR (Figure 3b), no cirrhosis/no SVR (Figure 3c) and finally no cirrhosis/SVR (Figure 3d). This is consistent with the progressively lower HCC risk in these groups. The decision curves also confirm that the net benefit in non-cirrhotics who achieve SVR is so low at all risk thresholds that no screening would be recommended.

Figure 3. Decision curves comparing the net benefit achieved by screening based on HCC risk predicted by the model (i.e. screening only patients who exceed a certain threshold probability – blue line) to the “screen-all” (green line) or “screen-none” (orange line) strategies.

The y-axis plots net benefit, which is defined as the proportion of the benefit of screening that would be expected in patients who are destined to develop HCC.

The x-axis shows different 3-year HCC risk thresholds for screening that might be recommended. For example the AASLD recommends screening when annual HCC risk exceeds 1.5% in patients with cirrhosis (or 3-year risk exceeds 4.5%). This threshold is shown as a dotted line in all Figures, which illustrates that the net benefit of screening based on our models shown by the blue line (i.e. screening only patients who have 3-year HCC risk >4.5% as predicted by our models) is greater than the net benefit of the “screen-all” strategy shown by the green line, for all four patient groups. The second dotted line in each panel shows the recommended screening threshold at which the blue and green line diverge i.e. at which screening based on risk estimates for our models should have higher net benefit that the screen-all strategy. For example, among patients with cirrhosis and SVR, as long as screening is recommended at any 3-year risk >2%, screening based on our models (i.e. screening only patients whose predicted 3-year HCC risk exceed 2%) should have greater net benefit than the screen-all strategy.

a. Cirrhosis and no SVR (TOP LEFT)

b. Cirrhosis and SVR (TOP RIGHT)

c. No Cirrhosis and no SVR (BOTTOM LEFT)

d. No Cirrhosis and SVR (BOTTOM RIGHT)

Among cirrhotic patients, the risk model-based screening strategy has superior net benefit than the “screen-all” strategy if the screening threshold above which screening is recommended is >2.5% over 3 years (or ~0.83% per year) for those without SVR, or >2% over 3 years (~0.67% per year) for those with SVR (see dotted lines in Figures 3a and 3b). This result indicates that if the appropriate screening threshold is >1.5% per year, as recommended by AASLD guidelines⁴, risk model-based screening would be superior to the “screen-all” strategy.

Among non-cirrhotic patients without SVR, the risk model-based screening strategy has superior net benefit than the “screen-all” strategy for recommended screening thresholds >0.6% per 3 years (or 0.2% per year). This means that if screening was found to be beneficial in non-cirrhotic patients with annual HCC risk > 0.2%, then risk model-based screening would be superior to a “screen-all” strategy.

Web-Based HCC Risk Estimating Tools

We implemented the four models shown in Table 2 as web-based tools to allow clinicians to estimate HCC risk in individual patients (available at www.hccrisk.com). Table 4 shows 3-year HCC risk estimated using our models in 6 hypothetical patients demonstrating great variability in HCC risk. Patient #1, who has cirrhosis without SVR, has an extremely high predicted 3-year HCC risk of 25.9% - such patients may consider screening by CT or MRI. Patients with cirrhosis who achieve SVR, may have relatively low 3-year risk (e.g. 1.6% in patient #2) or high 3-year risk (e.g. 11.1% in patient #3) depending on the absence/presence of adverse predictors. Patients without cirrhosis (who currently are not recommended screening) who do not achieve SVR, may have sufficiently high HCC risk to merit screening (e.g. 7.0% in patient #4) if they have several adverse predictors.

Table 4.

Estimates of 3-year HCC risk calculated by our web-based models in selected patients, demonstrating great variability in HCC risk depending on baseline characteristics.

Patient #	1	2	3	4	5	6
Cirrhosis	Yes	Yes	Yes	No	No	No
SVR	No	Yes	Yes	No	No	Yes
Age	65	55	66	65	55	65
Albumin	3.3	4.1	3.6	3.8	4.1	4.1
Serum AST	40	25	45	35	35	35
Serum ALT	30	35	30	30	45	45
Platelet Count	110	145	110	145	210	250
3-year HCC risk (%)	25.9	1.6	11.1	7.0	0.6	0.3

Discussion

Most HCV-infected patients in the United States will undergo DAA-based antiviral treatment in the next few years and the vast majority of them will achieve SVR. We developed and internally validated models estimating HCC risk after antiviral treatment in four separate sub-groups: cirrhosis/SVR, cirrhosis/no SVR, no cirrhosis/SVR, and no cirrhosis/no SVR (available at www.hccrisk.com). Categorizing by cirrhosis and SVR was appropriate given that HCC screening is currently recommended only in patients with cirrhosis and that SVR reduces long-term HCC risk. Our models estimate HCC risk based on simple, readily available, objective and reproducible predictors and thus can be utilized easily in clinical practice. We demonstrated that screening strategies based on our models’ HCC risk estimates resulted in superior net benefit compared to “screen-all” or “screen-none” strategies. We hope that our models, which are available as web-based tools, will be externally validated in other populations and used by clinicians to estimate HCC risk after antiviral treatment and guide decisions about the most appropriate HCC surveillance strategy in individual patients.

Current AASLD and EASL HCC guidelines recommend screening only HCV-infected patients who have developed cirrhosis with ultrasound ± AFP testing every 6 months. This “one-size-fits-all” strategy is problematic for many reasons. First, our models show that patients without cirrhosis, in whom screening is currently not recommended, can have a very high risk of HCC especially if they do not achieve SVR. Second, patients with cirrhosis who do not achieve SVR and/or have additional adverse predictors may have alarmingly high HCC risk, such that screening with CT or MRI may be warranted. Finally, our results demonstrate that SVR as well as a number of other patient characteristics dramatically modify HCC risk, such that it does not make sense for “presence of cirrhosis” to be the sole criterion upon which surveillance is based. Instead, we propose that our models can be used to estimate HCC risk and the appropriate surveillance strategy can then be determined based on that risk.

Estimation of HCC risk in individual patients by the models we developed could improve HCC surveillance efforts, increase early detection of HCC and reduce harms related to unnecessary surveillance. First, patients at high risk of HCC could be targeted for interventions to improve their uptake of HCC surveillance. It is currently estimated that ≤20% of cirrhotic patients undergo surveillance consistent with guidelines in the United States³⁹. Second, different surveillance strategies could potentially be proposed for different categories of HCC risk. For example, more effective strategies that are also more expensive or more invasive/harmful, such as annual MRI, abbreviated MRI⁵ or CT, would be more costeffective if they focus on higher risk groups⁶. Third, in healthcare systems with limited resources unable to support universal surveillance of all cirrhotic patients, surveillance could be targeted to patients with higher HCC risk. Fourth, we have demonstrated that screening strategies based on our models’ HCC risk estimates resulted in superior net benefit than “screen-all” or “screen-none” strategies. Therefore, employing our models and limiting surveillance to patients who exceed a certain HCC risk threshold would be expected to reduce the “harms” of unnecessary screening in patients who will not develop HCC (including costs and harms of unnecessary imaging studies, liver biopsies and other procedures⁴⁰) and increase the benefits by targeting patients who are more likely to develop HCC. Finally, estimation of HCC risk enables individualized counseling of patients by their providers potentially leading to improved compliance with surveillance recommendations and engagement in care.

Decision curves plot the net benefit that would be expected at different “appropriate” HCC risk thresholds for screening. Figure 3 shows that at a threshold of >1.5% per year (or 4.5% per 3 years), which is commonly recommended in patients with cirrhosis³, the net benefit is greater with screening based on our models (i.e. screening only patients with estimated HCC risk>1.5% per year) compared with screening all patients. However, if the appropriate risk threshold is much lower (<2.5% per 3 years in cirrhosis/no SVR and <2% per 3 years in cirrhosis/SVR) then there is no difference between the screen-all and model-based screening strategies. It is important to emphasize that decision curves cannot be used to determine the appropriate HCC risk threshold at which screening is deemed to be beneficial. Instead, this threshold needs to be determined by other means, separately for each of the four patient subgroups. Decision analytic theory suggests that if the harms of missing a case are x-times greater than the harms of unnecessarily screening a non-case, then the appropriate threshold for screening is a risk exceeding 1/(x+1)⁴¹. Therefore, the greater the harms of missing a case (or the greater the benefits of diagnosing a case) the lower the risk threshold at which screening is beneficial. Conversely, the greater the harms of screening the higher the risk threshold.

Our study highlights the need to determine appropriate risk thresholds for screening in cirrhotic patients with or without SVR and in non-cirrhotic patients without SVR in the current era. AASLD guidelines recommend HCC surveillance in HCVinfected patients whose (predicted) HCC incidence exceeds 1.5% per year because older studies estimated a survival benefit of HCC surveillance in such patients³. However, these studies did not account for two important developments. First, HCV eradication can lead to long-term survival and, second, HCC treatments have improved dramatically. Both these developments increase the benefits of HCC surveillance and therefore should reduce the risk threshold above which HCC surveillance is warranted. Cirrhotic patients who achieve SVR represent a particularly difficult conundrum for providers: although SVR clearly reduces HCC risk, these patients still have a residual absolute HCC risk and therefore merit surveillance. However, our models show that even among these cirrhotic patients who achieve SVR, there can be dramatic variation in 3-year HCC risk, for example as little as 1.6% in patient #2 and as high as 11.1% in patient #3 in Table 4. The risk threshold above which screening should be recommended in non-cirrhotic patients with HCV is not established. We suggest that appropriate risk thresholds for HCC screening need to be determined for each of the four important subgroups after antiviral treatment.

We specifically used characteristics ascertained at or immediately before the beginning of antiviral treatment in our models to predict incident HCCs occurring at least 6 months after treatment initiation. We believe that this is the most clinically useful scenario since laboratory tests are routinely obtained at the beginning of treatment and since treatment acutely affects many tests. Although it is obviously not known at the beginning of treatment whether a patient will achieve SVR or not, HCC risk can easily be calculated for both SVR and no-SVR possibilities, or calculated after SVR is ascertained using pre-treatment laboratory tests.

Models have been proposed to estimate HCC risk in patients with cirrhosis^{42, 43}, HCV^{44, 45}, or HBV^46–48. Some core predictors are remarkably consistent across these diverse models as well as our model, such as age, platelet count and markers or advanced fibrosis or cirrhosis, corroborating their validity as predictors. We are not aware of other models that estimated HCC risk after antiviral treatment in recent US cohorts that we can directly compare to ours.

Although our study was based on a national cohort of VA patients, we believe our models apply to non-VA patients because the HCC risk that we reported amongst cirrhotic VA patients is very similar to what has been reported in non-VA studies, and because any differences are likely to be due to differences in risk factors included in the model (e.g. older age, male sex) and therefore accounted for in the risk calculation. Although the proportion of women was small, the number of women was high enough to allow modeling of sex as a predictor. It will be critical to externally validate our models in non-VA populations and also ideally in populations undergoing routine HCC surveillance. We combined DAA regimens with the most recent interferon regimens (i.e. those administered after 2009) because we recently showed that the type of antiviral regimen did not influence HCC risk¹. We plan to repeat our analysis in 2 years and update our online models using only DAA regimens. The ICD-10 code for HCC (C22.0) that replaced the ICD-9 code for HCC (155.0) in October 2015 is not yet validated using VA data. However, since a single ICD-10 code directly replaced a single ICD-9 code, it is reasonable to expect a similarly high positive predictive value. The diagnosis of cirrhosis was based on presence of validated ICD-9 and ICD-10 codes recorded by the patients’ providers. Although patients with “occult”, undiagnosed cirrhosis might have been misclassified in the no-cirrhosis group, our models would still be expected to capture their excess HCC risk correctly because they incorporate abnormalities in their platelet count, AST/√ALT ratio, albumin and INR levels. Substantial strengths of the study include the large sample size, large number of incident HCCs and long follow-up time. Baseline characteristics necessary for modeling were available. All patients were derived from a single, national healthcare system with fairly uniform practices and guidelines across its facilities.

In conclusion, we developed and validated models predicting HCC risk in HCV-infected patients categorized by the presence or absence of cirrhosis and SVR. These models, which are available as web-based tools, can help stratify patients according to HCC risk, and consequently, help determine an appropriate screening strategy based on a patient’s calculated risk. A screening strategy targeting those who exceed a certain predetermined HCC risk may be more efficacious and cost-effective than the current “screen-all” or “screen-none” strategies which depend solely on cirrhosis status.

1. We developed models to estimate HCC risk after antiviral treatment for HCV

2. Using these models may improve HCC screening strategies

Abbreviations

DAA

Direct-Acting Antiviral treatment for HCV

HCC

Hepatocellular carcinoma

HCV

Hepatitis C Virus

IFN

Interferon

PEG

Pegylated interferon