Abstract
Background & Aims:
The Hepatocellular carcinoma (HCC) Early detection Screening (HES) algorithm has been proposed to improve the performance of the serum alpha-fetoprotein (AFP) test in surveillance for HCC. The HES algorithm incorporates data on age, level of alanine aminotransferase, platelet count, and rate of AFP change to increase likelihood of earlier detection and thereby reduce HCC-related mortality. We updated the HES algorithm to include etiology of cirrhosis and validated it in a community-based cohort.
Methods:
We collected data from the Veterans Health Administration, from 2010 through 2015, on etiologies for HCC, including hepatitis C, hepatitis B, alcoholic liver disease, and non-alcoholic fatty liver disease. We used these data to update the HES algorithm and tested its accuracy using data from patients with cirrhosis in the Kaiser Permanente Northern California healthcare system (validation cohort)
Results:
Among the 7432 patients with cirrhosis in the validation cohort, 1102 were diagnosed with HCC during a median follow-up time of 3.21 years; 709 patients had early-stage HCC. The HES algorithm identified patients who would receive a diagnosis of early-stage HCC within the next 6 months with 51.20% sensitivity and 90.00% specificity, compared with 46.02% sensitivity for the AFP test alone (5.18% absolute improvement; P=.0015). In HCC screening, a positive result from HES or AFP test leads to follow-up evaluation with more sensitive imaging methods. The number of early-stage HCC cases detected per 1000 imaging analyses were 136.46 with the HES algorithm vs 118.01 with the AFP test alone (P<.0005). The HES algorithm identified 56.00% of patients with HCC in the 6 months before their diagnosis despite no detection of nodules by surveillance ultrasound; the AFP test identified only 50.00% of these patients.
Conclusions:
We validated the HES algorithm using data from a diverse community-based cohort of patients with cirrhosis. The algorithm offers a modest but useful advantage over the AFP test alone in detection of early-stage HCC with virtually no added cost.
Keywords: HCV, HBV, liver cancer, laboratory values, surveillance, AFP, biomarkers
Background
The early detection of hepatocellular carcinoma (HCC) is a key component towards potentially reducing HCC mortality1–3. Most practice guidelines including American, European, Asian Pacific societies and National Comprehensive Cancer Network recommend six-monthly HCC surveillance in high-risk patients using liver ultrasonography with or without serum AFP4. Patients with cirrhosis remain the primary target population for surveillance since 80–90% of HCCs occur in patients with cirrhosis5. Ultrasound surveillance alone is likely to be insufficient since it has reported sensitivity of 32% for detecting early stage HCC in clinical practice6. Serum α-fetoprotein (AFP) is commonly used in HCC screening despite the wide variation in its reported accuracy for cancer detection7,8. The sensitivity of AFP for HCC screening when using a threshold of ≥20ng/ml, for example, varies from 41%−65%, while reported specificity is above 80%9.
We previously developed and refined an AFP-based clinical algorithm that significantly improved upon AFP alone to accurately identify HCC in a large nationwide cohort of veterans with hepatitis C-virus (HCV) related cirrhosis in the Department of Veterans Affairs (VA) healthcare system. Our goal is to develop an algorithm that will replace AFP to be used in combination with ultrasound surveillance and hence we focused on improvements over AFP during the development of the algorithm. The initial algorithm included current AFP, age, platelets, alanine aminotransferase (ALT) and interaction terms (AFP and ALT, and AFP and platelets)10,11, and was updated to include the rate of change from the most recent AFP test performed within one year prior to the current test12. We also evaluated algorithm performance, quantified as the patient-level sensitivity within six months prior to HCC corresponding to 10% screening-level false positive rate (FPR); sensitivity for HCC was 61.37%, an almost 4 percentage point improvement over the sensitivity of AFP alone (57.53%)13. We subsequently validated these results in a more recent national VA cohort (2010 through 2015) that included patients with cirrhosis of any etiology, and again demonstrated superiority of our algorithm compared to AFP alone, regardless of cirrhosis etiology14.
The current study sought to evaluate the performance of an HES algorithm, modified to incorporate etiology of cirrhosis, for early detection of HCC in a diverse, community-based non-Veteran cohort. Additionally, we evaluated the incremental yield of the algorithm beyond concomitant ultrasound alone in a subset of patients that developed HCC.
Methods
Updating the HES algorithm
VA Study Population
The veterans cohort has been previously described14. It included patients with cirrhosis of any etiology seen within the national network of VA hospitals and at least one valid AFP test between 1/2010 and 5/2015. Incident HCC diagnoses were determined using HCC ICD-9 codes and in the VA’s adjudicated Central Cancer Registry. Etiology of cirrhosis was defined based lab tests and diagnostic codes (Supplementary Table 1).
Statistical methods
We randomly split the VA cohort with cirrhosis of any etiology into training and validation cohorts, stratified by HCC development and cirrhosis etiology. We excluded AFP tests where the concurrent ALT>500, AST>500, bilirubin>5.0 or albumin<1.5. In the training cohort, we used a generalized estimating equations approach to build a model for risk of HCC diagnosis in the next six months that included the components of the HES algorithm but expanded to include the etiology of cirrhosis. We defined active HCV (time-dependent), hepatitis B virus (HBV) positive, alcoholic liver disease (ALD), and non-alcoholic fatty liver disease (NAFLD). Models included a main effect for each etiology variable as well as interactions with both AFP and the rate of change in AFP in the past year. The quasi-likelihood under independence model criterion (QICu) was used for backward model selection of etiology interaction terms, where the optimal model minimized the QICu.
In the validation cohort, we compared the discrimination performance of the updated HES algorithm to the original HES algorithm, which was re-estimated in the multi-etiology training cohort. HES includes a rule where a patient has a positive screen if either AFP≥400ng/ml or the predicted HCC probability exceeded a threshold c, which was varied to generate the associated receiver operating characteristic (ROC) curve. The patient-level sensitivity within 6, 12, 24 months, or any time prior to diagnosis was defined as the proportion of patients with at least one positive screen within 6, 12, 24 months or any time prior to clinical HCC diagnosis respectively, among those with testing in this period. The screening-level FPR within 6, 12, 24 months, or any time prior to diagnosis was defined as the proportion of positive screening results in patients not diagnosed with HCC or positive screens that occur earlier than 6, 12 or 24 months prior to the HCC diagnosis, respectively. The 95% bias-corrected bootstrap percentile confidence intervals were estimated for all measures of clinical utility15. The bootstrap P-values were the proportion of bootstrap samples where the improvement in clinical utility was less than or equal to 0.
The parameters of the final selected model were estimated using the full VA cohort prior to external validation of the HES algorithm.
External Validation of the HES algorithm
We performed external validation of the HES algorithm in a non-veterans community-based cohort. We assessed the HES algorithm in terms of detection performance, calibration and the incremental yield of the algorithm over concomitant ultrasound.
Community-Based Study Population
The community-based cohort included patients with cirrhosis of any etiology being cared for within the Kaiser Permanente Northern California (KPNC) healthcare system. The KPNC membership approximates the underlying census race/ethnicity and socioeconomic distributions of the region16 and included patients with cirrhosis who were actively enrolled for at least 12 consecutive months at KPNC between 1/1/1997 and 12/31/2014.
Cirrhosis was defined by the presence of inpatient or outpatient ICD-9 codes 571.2, 571.5 or 571.6 between 1/1/1997 and 12/31/2014. Patients were censored at the date of HCC diagnosis, death, end of membership enrollment, or the study end date (12/31/2014). Patients who were diagnosed with HCC, received a liver transplant or died within six months of their cirrhosis index date were excluded. HCC diagnosis information was extracted from the KPNC cancer registry, which reports to the Surveillance, Epidemiology and End Results (SEER) registry; only HCC diagnoses with staging information were included. Early stage HCC was defined to be those classified as stage 1 (localized) by both the SEER and American Joint Committee on Cancer classifications.
The KPNC validation cohort was restricted to include patients with at least one AFP test that occurred after the cirrhosis index date and prior to either HCC diagnosis or censoring date (in patients without cirrhosis), with ALT and platelet laboratory tests performed within 6 months prior to the AFP test.
Statistical methods
The discrimination performance was primarily evaluated in all those diagnosed with early stage HCC, however the discrimination was also evaluated among all HCC. We also performed a sensitivity analysis excluding patients with any breaks in enrollment during the study period within KPNC.
We evaluated the HES algorithm with respect to positive predictive value (PPV), negative predictive value (NPV), probability of incorrect decision in the high risk group, probability of incorrect decision in the low risk group, the number of incident HCCs detected per 1000 computed tomography (CT) or magnetic resonance imaging (MRI) that were triggered by HES algorithm vs. AFP screen, and the number of CT/MRI needed to detect an incident HCC. In patients who developed HCC, we only included screening occasions up to the first (positive or negative) within six months prior to diagnosis or the first positive screen within one year prior to HCC diagnosis.
Model calibration in the KPNC cohort was assessed graphically by plotting the predicted 6- month risk of HCC from the HES algorithm versus the raw HCC probabilities; the Hosmer-Lemeshow X2 statistic was used to test for statistical significance. We employed a regression-based recalibration approach, where an intercept of 0 and slope 1 implied the predicted probabilities were valid. However, if either the intercept or slope significantly deviated from the ideal case, we used the estimated values to re-calibrate the predicted probabilities for the KPNC cohort.
In a subset of HCC patients with concurrent AFP testing and ultrasound, we estimated the incremental yield of the HES algorithms over ultrasound alone. We extracted data from radiology reports of liver ultrasound from all HCC patients on whether the test was performed for HCC surveillance in the absence of other indication (e.g., biopsy, prior lesion, complaint of pain, previously elevated AFP) and whether a nodule was detected. Each ultrasound was considered concurrent to AFP testing if it occurred within one month prior or six months post AFP test date. A second data extraction of a random subsample of 10% of patients was used to assess inter-rater reliability on measures of indication and nodule detection.
Results
Updating the HES algorithm
The VA cohort consisted of 38,431 patients with cirrhosis, among whom 4,804 patients were diagnosed with incident HCC during 110,936 patient-years of follow-up. The median follow-up duration was 3.12 years. The estimated annual HCC incidence rate was 4.3%. The median age of patients at their first AFP test was 61 years (interquartile range: 56–64 years). Most patients were men (97.1%) and non-Hispanic whites (62.9%); a substantial proportion was black (19.9%). The risk factors for cirrhosis included ALD (72.5%), HCV (51.1%), NAFLD (16.2%), and HBV (8.8%). Among patients with HCV-related cirrhosis, 99.1% had active HCV at every AFP test, < 1% had an AFP test while receiving treatment, and 0.1% had an AFP test after achieving sustained virologic response (SVR). Supplementary Table 2 includes descriptive statistics of the cohort. The VA training subcohort, used to develop an updated HES algorithm that included etiology of cirrhosis, included 18,738 patients among whom 2,208 were diagnosed with HCC. An HES algorithm that incorporated interaction terms between time-dependent active HCV and current AFP and change in AFP over the last year resulted in the smallest QICu (Supplementary Table 3). The effect of treated HCV could not be examined because only a few patients (n=54) received treatment prior to HCC diagnosis in this cohort. Table 1 describes the component variables of the updated HES algorithm.
Table 1:
Component | Units | Time frame |
---|---|---|
Current AFP levels | ng/ml | Current |
Current age | Years | Current |
Rate of change in AFP within the last year | ng/ml per year | Within 1 year prior to current AFP |
ALT | ng/ml | Within 6 months prior to current AFP |
Platelet | count/1000 | Within 6 months prior to current AFP |
Current HCV status | Active vs {On treatment, Treated with SVR, Negative, or Unknown} | Current |
Hepatitis B status | Positive vs {Negative or Unknown} | Ever |
Non-alcoholic fatty liver disease status | Positive vs {Negative or Unknown} | Ever |
Alcoholic liver disease status | Positive vs {Negative or Unknown} | Ever |
The VA validation subcohort included 18,923 patients of whom 2,332 were diagnosed with HCC. At a pre-set false positive rate of 10%, the updated HES algorithm significantly improved patient-level sensitivity by 1.37–1.55 percentage points within 12 month, 24 months, and any time prior to HCC diagnosis (p=0.011, 0.004, and 0.020, respectively) compared to the original HES algorithm (Supplementary Table 4). The updated HES algorithm also improved patient-level sensitivity at 10% screening-level FPR by ~6–7 percentage points at all times priors prior to diagnosis compared to AFP alone.
External Validation of the HES algorithm
The KPNC validation cohort consisted of 7,432 patients with cirrhosis, among whom 1,102 were diagnosed with incident HCC during 28,603 patient-years of follow-up, including 709 with early stage HCC (Supplementary Figure 1). The median follow-up duration was 3.21 years. The estimated annual HCC incidence rate was 3.85%. Patient who did not develop HCC had an average of 2.49 AFP tests (standard deviation=3.50) and 46.22% of these patients had more than one AFP test. Patients diagnosed with HCC had an average of 3.51 (standard deviation=3.49) AFP tests prior to HCC diagnosis, and 70.33% of all those diagnosed with HCC had more than one AFP test. Most patients were men (62.4%) and non-Hispanic whites (52.4%); a substantial proportion was Hispanic (20.1%) or Asian/Pacific Islander (18.1%). The etiological risk factors for cirrhosis included HCV (37.3%), NAFLD (21.6%), ALD (19.1%), and HBV (15.7%). Among patients with HCV-related cirrhosis, 82.7% of patients had active HCV at every AFP test during the study period, 13.9% had an AFP test while receiving treatment and 9.0% had an AFP test after achieving SVR. Supplementary Table 2 includes descriptive statistics of the cohort.
We assessed the discriminatory performance of the updated HES algorithm in the KPNC cohort, with a primary focus on the detection of early stage HCC patients. Figure 1 reports the patient-level sensitivity at the threshold corresponding to 10% screening-level FPR. We evaluated the discrimination performance within 6, 12 and 24 months and any time prior to HCC diagnosis. The updated HES algorithm improved sensitivity for detecting early stage HCC by 5.18–8.58 percentage points compared to using AFP alone within the time-frames considered (p<0.002) (Fig 1). Within six months prior to diagnosis of early stage HCC, the patient-level sensitivity corresponding to 10% screening-level FPR of the updated HES algorithm was 51.20% (95% CI: 46.58–55.34) vs. 46.02% (95% CI: 41.96–50.60) with using AFP alone; a 5.18 percentage point improvement (p=0.0015). There was a small increase in sensitivity of the updated HES algorithm compared to the prior no etiology version but the differences were not statistically significant.
The updated HES algorithm improved sensitivity for detecting all patients with HCC by 5.86–8.22 percentage points compared to using AFP alone within the time-frames considered (p<0.0005) (Fig 1). In a sensitivity analysis that restricted the KPNC cohort to patients with continuous enrollment during the study period (a break in enrollment was defined as non-use for at least 3 months), the improved performance of the HES algorithm over AFP alone remained mostly unchanged with 5.45–7.84 percentage points increased sensitivity over the range of timeframes considered (Supplementary Table 5).
The discrimination performance was also evaluated within non-overlapping intervals (0–6 months, 6–12 months, 12–24 months and >24 months prior to early HCC diagnosis). In patients with early stage HCC, the updated HES algorithm significantly improved sensitivity by 5.18 (p=0.0015), 10.30 (p<0.0005), and 5.59 (p=0.031) percentage points within 0–6 months, 6–12 months and 12–24 months prior to HCC diagnosis, respectively, compared to AFP alone among those with early stage HCC (Supplementary Table 6).
We evaluated the consequences of a decision rule that classifies patients as high- or low- risk based on the updated HES algorithm at a threshold corresponding to a 10% screening-level FPR (Figure 2). We assumed that high-risk (positive screen) patients would undergo testing via CT or MRI, while those classified as low-risk (negative screen) would continue with 6-monthly HCC surveillance. The estimated PPV for early stage HCC increased from 11.80% with using AFP alone to 13.65% with the updated HES algorithm (p<0.0005); the NPV increased from 98.18% to 98.30% (p=0.03) using AFP alone and the updated HES algorithm, respectively. These improvements corresponded to increases in the number of early stage HCC cases detected per 1000 CT/MRI from 118.01 using AFP alone to 136.46 (p<0.0005) using HES, or 18 additional early stage HCC cases (16% increase) that would have been missed if surveillance was solely based on AFP alone. The estimated number of CT/MRI needed to detect an early stage HCC case was 8.47 with AFP alone, while the updated HES algorithm required 7.33 (p-value<0.0005). Therefore, the HES algorithm reduced the number of CT/MRI needed to detect 100 HCC cases by 114 (13% decrease) compared to AFP alone.
We received 1062 ultrasound images on 517 HCC patients. A total of 85 ultrasound scan reports were assessed by two independent raters on whether the ultrasound performed was for HCC screening and whether a nodule was detected. The kappa statistic for the indication of HCC screening was .67 and for the nodule detected was 0.97 corresponding to “substantial” and “nearly perfect” agreement 17. We then identified 242 patients who eventually developed any HCC, with screening ultrasounds that were concurrent with AFP testing. In this subcohort, we were able to assess the added utility of AFP and the updated HES algorithm over results from ultrasound alone, results shown in Table 2, with associated 95% confidence intervals. The patient-level sensitivity of ultrasound alone was 78.43, 67.82 and 61.57% within the 6, 12 and 24 months prior to HCC diagnosis, respectively. Ultrasound with AFP had patient-level sensitivity of 89.54, 80.69 and 76.03%, and ultrasound with the updated HES algorithm had patient-level sensitivity of 90.85, 82.18 and 76.03%, within the 6, 12 and 24 months prior to HCC diagnosis respectively. Based on the 95% confidence intervals, we can conclude that using either AFP, the HES algorithm or the updated HES algorithm in combination with ultrasound significantly improves the patient-level sensitivity over ultrasound alone at 6, 12 and 24 months prior to diagnosis. In Table 3, we examine the results at a screening-test level. There were 50, 142 and 245 AFP tests among 35, 89 and 133 patients that developed HCC in the next 6, 12 and 24 months, respectively, where the concomitant ultrasound examination did not have any suspicious nodules. The updated HES algorithm had a positive screening result in 56.0%, 43.0% and 33.9% of these testing occasions, respectively while AFP alone had a positive screening result in 50.0%, 38.0% and 31%.
Table 2:
Patient sensitivity 6 months prior to diagnosis (153 HCC) |
Patient sensitivity 1 year prior to diagnosis (202 HCC) |
Patient sensitivity 2 years prior to diagnosis (242 HCC) |
|
---|---|---|---|
Ultrasound alone | 78.43 (95% CI: 71.72–84.83) |
67.82 (95% CI: 60.92–74.35) |
61.57 (95% CI: 55.29–67.84) |
AFP + Ultrasound | 89.54 (95% CI: 84.43–94.19) |
80.69 (95% CI: 74.87–86.07) |
76.03 (95% CI: 70.68–81.48) |
HES algorithm + Ultrasound | 91.50 (95% CI: 86.90–95.65) |
82.67 (95% CI: 77.18–87.63) |
76.45 (95% CI: 70.97–81.67) |
Updated HES algorithm + Ultrasound | 90.85 (95% CI: 86.03–95.04) |
82.18 (95% CI: 76.56–87.21) |
76.03 (95% CI: 70.46–81.23) |
Table 3:
Result | 6 months (216 tests in 153 HCC patients) |
1 year (351 tests in 202 HCC patients) |
2 years (485 tests in 242 HCC patients) |
||||
---|---|---|---|---|---|---|---|
Nodule detected | Nodule detected | Nodule detected | |||||
Yes | No | Yes | No | Yes | No | ||
AFP only | Positive | 43 (25.9%) |
25 (50.0%) |
52 (24.9%) |
54 (38.0%) |
57 (23.8%) |
76 (31.0%) |
Negative | 123 (74.1%) |
25 (50.0%) |
157 (75.1%) |
88 (62.0%) |
183 V(76.2%) |
169 (69.0%) |
|
HES algorithm | Positive | 48 (28.9%) |
29 (58.0%) |
58 (27.8%) |
56 (39.4%) |
63 (26.2%) |
82 (33.5%) |
Negative | 118 (71.1%) |
21 (42.0%) |
151 (72.2%) |
86 (60.6%) |
177 (73.8%) |
163 (60.6%) |
|
Updated HES algorithm | Positive | 49 (29.5%) |
28 (56.0%) |
60 (28.7%) |
61 (43.0%) |
65 (27.1%) |
83 (33.9%) |
Negative | 117 (70.5%) |
22 (44.0%) |
149 (71.3%) |
81 (57.0%) |
175 (72.9%) |
162 (66.1%) |
Discussion
This study represents a crucial step in the validation of new HES algorithm for early HCC detection in the general population of patients with cirrhosis, and the use of a more personalized strategy for HCC screening and surveillance. The HES algorithm has been developed and validated to date in cohorts constructed within the VA healthcare system. In this study, we developed an updated HES algorithm that incorporated information on etiology of cirrhosis. We then externally validated the algorithm in a demographically diverse, community-based cirrhosis cohort that was 38% female, 19% Hispanic with 19% of patients with ALD. This was in contrast to the VA cohort that had 3% female, 10% Hispanic and with 75% of patients with ALD.
Our use of large multi-etiology cirrhosis cohorts from two of the largest healthcare systems in the US increases both the precision of the estimates as well as generalizability of findings. The methodical derivation, internal validation and external validation with measurements of calibration and discrimination at every step are major also strengths that are unrivaled in scope by previous studies. The HES algorithm demonstrated clinically meaningful and statistically significant improvements against AFP alone in both early stage HCC patients, as well as in all HCC patients. For example, among early stage HCC, the patient-level sensitivity at 10% screening-level FPR, was 51.20%, an 11% relative increase compared to AFP alone, In addition, the HES algorithm demonstrated clinically significant improvements against presumed usual practice with the estimated number of early HCC cases detected per 1000 CT/MRIs performed increasing from 118.01 to 136.46, or 18 additional early stage HCC cases.
While liver imaging is central to HCC screening and surveillance, our study further confirms the additional value of biomarkers. Among AFP tests occurring within six months prior to HCC diagnosis, where the concurrent ultrasound had no indication of nodules, the HES algorithm indicated a positive screen in 56% of the patients; the AFP test was positive in 50%. A limitation of our study is that the ultrasonography results were extracted manually from free text format within the radiology reports for those diagnosed with HCC only. Future studies to extract imaging information via natural language processing, would be of value and enable a more comprehensive analysis of HCC surveillance using both imaging and AFP-based algorithms.
Our goal in this research is to improve the first step in future surveillance strategies, realizing that this is a necessary but not sufficient to reduce HCC mortality. Optimal application of these tests in the appropriate population, implementing a recall strategy, making timely diagnosis, and providing timely and stage appropriate therapy are required to reduce mortality. Given the retrospective nature of our studies, we can estimate the likelihood of the updated HES without or with ultrasound in detecting an HCC, but we cannot examine effect on mortality.
While HES increases the sensitivity of HCC early detection to approximately 51.2% (at a 10% FPR threshold), a large proportion of cases will be missed. Therefore, additional work is needed to optimize HCC surveillance biomarkers. However, the framework of our HES algorithm, which consists of including demographic and clinical features combined with biomarkers, allows for continuous updating with new or additional non-AFP biomarkers. This gives the approach staying power as new HCC biomarkers are adopted into clinical practice. For example, des-γ carboxy prothrombin (DCP) and lens culinaris agglutinin-reactive alpha-fetoprotein (AFP-L3) have been evaluated in Phase-2 biomarker studies18 and are being evaluated in Phase-3 biomarker studies19. Similar to our score, the GALAD score contains age and AFP but also includes sex, a variable that we did not find improved predictive performance of HES, AFP-L3 and DCP20. In the future, we will update the HES algorithm with these two biomarkers as well other promising biomarkers. In addition, most patients with HCV in the development and validation cohorts had active infection whereas most patients in contemporary clinical practices would have cured HCV. While patients with cirrhosis to whom HES is applicable are likely to retain residual HCC for at least several years after virological cure, the change in HCC risk and unclear surveillance recommendations; therefore the use of HES in this important group will need to be examined.
For now, the added benefits of this HES algorithm, an 11% relative increase in patient-level sensitivity and 16% relative increase in additional early stage HCC cases detected per 1000 CT/MRI, are modest but given that they are associated with virtually no added cost, we believe that it will be a useful tool in clinical practice. A web-based open access version of the HES algorithm is found on www.bcm.edu/hes.
Supplementary Material
Supplemental Figure 2: Evaluating calibration and recalibration of the updated HES algorithm in a multi-etiology external cohort from Kaiser Permanente Northern California.
The graph assesses how closely the predicted probability of HCC in the next six months matched the observed probability. The predicted HCC probability underestimated the observed HCC risk at all except the highest risk decile. The Hosmer-Lemeshow test statistic indicated poor fit of the predicted probability of HCC in the next 6 months to the observed probability (p<0.001). We therefore performed regression-based recalibration and found that =−0.30 (p<0.001) and =0.79 (p<0.001) significantly deviated from the ideal case (α=0 and (β=1). The resulting regression recalibrated predicted probability of HCC greatly improved fit, with the recalibrated linear predictor yielded (p<0.001) and (p<0.001), however there was some remaining evidence of poor fit (Hosmer-Lemeshow p=0.01).
Need to Know.
Background:
The hepatocellular carcinoma (HCC) early detection screening (HES) algorithm, which includes data on age, level of alanine aminotransferase, platelet count, and rate of change in level of alpha-fetoprotein (AFP), was developed to improve surveillance for HCC.
Findings:
In a diverse community-based cohort of patients with cirrhosis, the HES algorithm identified those who would receive a diagnosis of early-stage HCC within the next 6 months with 51.20% sensitivity and 90% specificity, compared with 46.02% sensitivity for the AFP test alone.
Implications for patient care:
The HES algorithm can increase detection of early-stage HCC with virtually no added cost.
Acknowledgments
Grant support:
This research was supported by an NIH NCI Grants (R01CA190776) and NIDDK (P30DK056338) to Dr El-Serag, and by the Houston Veterans Affairs Health Services Research and Development Center of Excellence (HFP90-020)
List of Abbreviations:
- AFP
α-Fetoprotein
- ALT
alanine aminotransferase
- CT
computed tomography
- FPR
false positive rate
- HCC
hepatocellular carcinoma
- HCV
hepatitis C virus
- HES
hepatocellular carcinoma early detection screening
- ICD-9
international classification of diseases, 9th revision
- MRI
magnetic resonance imaging
- NPV
negative predictive value
- PPV
positive predictive value
- ROC curve
receiver operating characteristic curve
- VA
U.S. Department of Veterans Affairs Healthcare System
Footnotes
Disclosures: All authors have no conflicts to disclose
Writing Assistance: None
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Kulik L, El-Serag HB. Epidemiology and Management of Hepatocellular Carcinoma. Gastroenterology. 2019;156(2):477–491.e1. doi: 10.1053/j.gastro.2018.08.065 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Moon AM, Weiss NS, Beste LA, et al. No Association Between Screening for Hepatocellular Carcinoma and Reduced Cancer-Related Mortality in Patients With Cirrhosis. Gastroenterology. 2018;155(4):1128–1139.e6. doi: 10.1053/j.gastro.2018.06.079 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Costentin CE, Layese R, Bourcier V, et al. Compliance With Hepatocellular Carcinoma Surveillance Guidelines Associated With Increased Lead-Time Adjusted Survival of Patients With Compensated Viral Cirrhosis: A Multi-Center Cohort Study. Gastroenterology. 2018;155(2):431–442.e10. doi: 10.1053/j.gastro.2018.04.027 [DOI] [PubMed] [Google Scholar]
- 4.Heimbach JK, Kulik LM, Finn R, et al. Aasld guidelines for the treatment of hepatocellular carcinoma. Hepatology. January 2017. doi: 10.1002/hep.29086 [DOI] [PubMed] [Google Scholar]
- 5.El-Serag HB. Hepatocellular carcinoma. N EnglJ Med. 2011;365(12):m8–1127. doi: 10.1056/NEJMra1001683 [DOI] [PubMed] [Google Scholar]
- 6.Singal AG, Conjeevaram HS, Volk ML, et al. Effectiveness of hepatocellular carcinoma surveillance in patients with cirrhosis. Cancer Epidemiol Biomarkers Prev. 2012;21(5):793–799. doi: 10.1158/1055-9965.EPI-11-1005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Davila JA, Morgan RO, Richardson PA, Du XL, McGlynn KA, El-Serag HB. Use of surveillance for hepatocellular carcinoma among patients with cirrhosis in the United States. Hepatology. 2010;52(1):132–141. doi: 10.1002/hep.23615 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Tzartzeva K, Obi J, Rich NE, et al. Surveillance Imaging and Alpha Fetoprotein for Early Detection of Hepatocellular Carcinoma in Patients With Cirrhosis: A Meta-analysis. Gastroenterology. 2018;154(6):1706–1718.e1. doi: 10.1053/j.gastro.2018.01.064 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Gupta S, Bent S, Kohlwes J. Test characteristics of alpha-fetoprotein for detecting hepatocellular carcinoma in patients with hepatitis C. A systematic review and critical analysis. Ann Intern Med. 2003;139(1):46–50. http://www.ncbi.nlm.nih.gov/pubmed/12834318. [DOI] [PubMed] [Google Scholar]
- 10.Richardson P, Duan Z, Kramer J, Davila JA, Tyson GL, El-Serag HB. Determinants of serum alpha-fetoprotein levels in hepatitis C-infected patients. Clin Gastroenterol Hepatol. 2012;10(4):428–433. doi: 10.1016/j.cgh.2011.11.025 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.El-Serag HB, Kanwal F, Davila JA, Kramer J, Richardson P. A new laboratory-based algorithm to predict development of hepatocellular carcinoma in patients with hepatitis C and cirrhosis. Gastroenterology. 2014;146(5):1249–55.e1. doi: 10.1053/j.gastro.2014.01.045 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.White DL, Richardson P, Tayoub N, Davila JA, Kanwal F, El-Serag HB. The Updated Model: An Adjusted Serum Alpha-Fetoprotein-Based Algorithm for Hepatocellular Carcinoma Detection With Hepatitis C Virus-Related Cirrhosis. Gastroenterology. 2015;149(7):1986–1987. doi: 10.1053/j.gastro.2015.10.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Tayob N, Richardson P, White DL, et al. Evaluating screening approaches for hepatocellular carcinoma in a cohort of HCV related cirrhosis patients from the Veteran’s Affairs Health Care System. BMC Med Res Methodol. 2018;18(1):1. doi: 10.1186/s12874-017-0458-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Tayob N, Christie I, Richardson P, et al. Validation of the Hepatocellular Carcinoma Early detection Screening (HES) algorithm in a Cohort of Veterans with Cirrhosis. Clin Gastroenterol Hepatol. December 2018. doi: 10.1016/j.cgh.2018.12.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Efron B and Tibshirani RJ. An Introduction to the Bootstrap. Taylor & Francis; 1994. [Google Scholar]
- 16.Krieger N Overcoming the absence of socioeconomic data in medical records: validation and application of a census-based methodology. Am J Public Health. 1992;82(5):703–710. doi: 10.2105/ajph.82.5.703 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Viera AJ, Garrett JM. Understanding Interobserver Agreement : The Kappa Statistic. Fam Med. 2005;37(5):360–363. [PubMed] [Google Scholar]
- 18.Marrero JA, Feng Z, Wang Y, et al. Alpha-fetoprotein, des-gamma carboxyprothrombin, and lectin-bound alpha-fetoprotein in early hepatocellular carcinoma. Gastroenterology. 2009;137(1):110–118. doi: 10.1053/j.gastro.2009.04.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Feng Z, Marrero JA, Khaderi S, et al. Design of the Texas Hepatocellular Carcinoma Consortium Cohort Study. Am J Gastroenterol. 2019;114(3). https://journals.lww.com/ajg/Fulltext/2019/03000/Design_of_the_Texas_Hepatocellular_Carcinoma.25.aspx. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Johnson PJ, Pirrie SJ, Cox TF, et al. The detection of hepatocellular carcinoma using a prospectively developed and validated model based on serological biomarkers. Cancer Epidemiol Biomarkers Prev. 2014;23(1):144–153. doi: 10.1158/1055-9965.EPI-13-0870 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental Figure 2: Evaluating calibration and recalibration of the updated HES algorithm in a multi-etiology external cohort from Kaiser Permanente Northern California.
The graph assesses how closely the predicted probability of HCC in the next six months matched the observed probability. The predicted HCC probability underestimated the observed HCC risk at all except the highest risk decile. The Hosmer-Lemeshow test statistic indicated poor fit of the predicted probability of HCC in the next 6 months to the observed probability (p<0.001). We therefore performed regression-based recalibration and found that =−0.30 (p<0.001) and =0.79 (p<0.001) significantly deviated from the ideal case (α=0 and (β=1). The resulting regression recalibrated predicted probability of HCC greatly improved fit, with the recalibrated linear predictor yielded (p<0.001) and (p<0.001), however there was some remaining evidence of poor fit (Hosmer-Lemeshow p=0.01).