An Evaluation of the Potential Cost-Effectiveness of Noninvasive Testing Strategies in the Diagnosis of Significant Liver Fibrosis

Abstract

Objectives

To assess the clinical and economic outcomes of noninvasive testing strategies in the diagnosis of significant liver fibrosis (Metavir score ≥ 2) compared to liver biopsy.

Methods

We developed a decision analytic model of noninvasive testing strategies in a hypothetical patient population with genotype 1 HCV infection, with no contraindications to liver biopsy. The testing strategies included a testing algorithm using the Fibrosure test, a noninvasive measure of fibrosis, followed by liver biopsy for patients with indeterminate results, Fibrospect II, and Fibroscan. The primary outcomes were sensitivity, specificity, diagnostic accuracy (true positive + true negatives / total patients), and costs, evaluated from the healthcare payer perspective.

Results

The testing algorithm using Fibrosure was the most accurate noninvasive strategy with a sensitivity, specificity, and overall accuracy of 84%, 87%, and 86%, respectively. In comparison with liver biopsy alone, there was a cost savings of ~ $770/person with the Fibrosure testing algorithm, but a net decrease in accuracy of 14%. Fibrospect II and Fibroscan had poorer accuracy (decreases of 12% and 4%, respectively) and lower costs (−$138 and −$357, respectively) compared to the Fibrosure algorithm. In uncertainty analyses in which biopsy sampling error was considered, the Fibrosure algorithm remained consistently less accurate (5% to 14% decrease).

Conclusions

The results of our study suggest that compared to liver biopsy, noninvasive testing algorithms can result in short-term cost savings, but the consequences of misdiagnosis in terms of health outcomes and treatment costs may outweigh the short-term gains in cost and convenience.

Introduction

An estimated 2.7 million persons are chronically infected with the hepatitis C virus (HCV) in the United States. HCV can lead to serious liver complications and accounts for 8,000 – 12,000 deaths and $5 billion in medical care costs in the US per year.^{1, 2} Disease progression is often slow and debate persists as to the appropriate treatment especially in certain patient subgroups (those with contraindications, genotype 1 HCV, and patients with mild to no fibrosis).³

Liver biopsy is currently considered the gold standard test to assess severity of hepatic necroinflammation and fibrosis.⁴ However, biopsy is associated with significant costs, requires expertise of a specialist, and carries rare but serious risks of injury and death (mortality: ~ 1/10,000).⁵ Biopsy is also subject to sampling error in staging liver fibrosis, which may range between 15% and 33%.^{6, 7} Therefore, there has been much interest in the development of reliable noninvasive tests for determining the presence and stage of liver fibrosis.⁷ No single marker has yet demonstrated sufficient accuracy, but a few tests using combinations of serum markers have shown encouraging results and have been adopted by some practitioners.^{7, 8} However, the cost-effectiveness and performance characteristics of noninvasive serum markers in comparison to liver biopsy have not been quantitatively evaluated. Accordingly, we developed a decision analytic model to examine the costs and performance of noninvasive testing for hepatic fibrosis using the following commercially available noninvasive serum-based testing strategies: Fibrosure, Fibrospect II, and Fibroscan in comparison with liver biopsy for the diagnosis of liver fibrosis.

Methods

Model Overview

A decision analytic model was developed to assess the performance, costs, and cost-effectiveness of noninvasive testing strategies for evaluating liver fibrosis compared to the reference practice of liver biopsy alone in the diagnosis of significant hepatic fibrosis (Metavir score of ≥ 2).⁹ The analysis was performed on a hypothetical population of 1,000 patients presenting with genotype 1 HCV infection, no clinical signs of cirrhosis, and without contraindications to liver biopsy. The model was developed in consultation with clinical experts to reflect the diagnostic sequence that reflected the manufacturers’ recommended testing algorithms.⁹ Persons with significant fibrosis or cirrhosis identified by either approach are candidates for treatment, while those with no or minimal fibrosis would likely be observed and re-evaluated periodically.¹⁰ The sensitivity and specificity of liver biopsy was assumed to be 100% as it was the reference standard against which the noninvasive testing strategies were developed. In order to assess the potential for sampling error, we varied the sensitivity of liver biopsy in a scenario analysis. The analysis used the healthcare payer perspective and was limited to the initial evaluation phase. Thus, discounting of future costs and outcomes was not employed.¹¹

Test Characteristics

The noninvasive strategies included a step-wise testing algorithm using the Fibrosure test, followed by liver biopsy for patients with indeterminate Fibrosure results (figure 1). In the base case, the following cutoff values were used: Fibrosure score > 0.58 = significant fibrosis or cirrhosis (Metavir score > F2) and a score <0.31 = no or minimal fibrosis (F0 or F1).⁹ The Fibrosure test consists of a panel of six serum biochemical markers (α₂-macroglobulin, haptoglobin, apolipoprotein A₁, bilirubin, gammaglutamyl transpeptidase, and alanine aminotransferase) combined with age and gender in a risk algorithm.⁹ The test characteristics were obtained from a published evaluation of an integrated database of 1,570 patients with liver biopsy as the reference standard. The patient population consisting of 1.270 patients with chronic hepatitis C and 300 healthy blood donors with a prevalence of significant fibrosis or cirrhosis of 31% (table 1).¹²

Figure 1.

Decision algorithm for assessment of fibrosis using liver biopsy (usual care) versus noninvasive testing

Table 1.

Decision Model Parameters

	Base Case	Low	High	Source
Population Distribution (true condition)
No/Mild Fibrosis	69%	67%	71%	Poynard 200412
Moderate/High Fibrosis	31%	33%	29%	Poynard 200412
Fibrosure Test Characteristics
Probabilities for patients with no/mild fibrosis
Test positive	13%	11%	15%	Poynard et al 200412
Test indeterminate	19%	17%	21%	Poynard et al 200412
Test negative	68%	64%	72%	Poynard et al 200412
Probabilities for patients with moderate/severe fibrosis
Test positive	56%	54%	58%	Poynard et al 200412
Test indeterminate	28%	26%	30%	Poynard et al 200412
Test negative	16%	12%	20%	Poynard et al 200412
Fibrospect II Characteristics
Probabilities for patients with no/mild fibrosis
Test positive	29%	21%	36%	Prometheus Labs13
Test indeterminate	0%	-	-
Test negative	71%	67%	75%	Prometheus Labs13
Probabilities for patients with moderate/severe fibrosis
Test positive	81%	60%	100%	Prometheus Labs13
Test indeterminate	0%	-	-
Test negative	19%	15%	23%	Prometheus Labs13
Fibroscan Test Characteristics
Probabilities for patients with no/mild fibrosis
Test positive	11%	8%	14%	Castera et al.14
Test indeterminate	0%	-	-
Test negative	89%	85%	93%	Castera et al.14
Probabilities for patients with moderate/severe fibrosis
Test positive	67%	50%	84%	Castera et al.14
Test indeterminate	0%	-	-
Test negative	33%	29%	37%	Castera et al.14
Costs*
Fibrosure Test	$215	$161	$269	Labcorp personal communication 4/01/06
Liver Biopsy	$1,255	$941	$1,569	Wong et al, 199823
Fibrospect II	$350	$263	$438	Botts et al, 200524
Fibroscan	$131	$98	$164	CEDIT, 200425

^*Adjusted to 2005 US dollars using medical care component of Consumer Price Index

We also evaluated Fibrospect II, a biochemical index using 3 biomarkers (hyaluronic acid, tissue inhibitor of metalloproteinase-1, and α₂-macroglobulin)¹³ and Fibroscan, a device that evaluates hepatic fibrosis by measuring liver stiffness using ultrasound technology¹⁴. The test manufacturers’ website for Fibrospect II recommends a binary cutoff at ≥ 42 with sensitivity and specificity of 80.6% and 71.4%, respectively, based on data from 696 chronic HCV patients with a prevalence of significant fibrosis of 51.7%.¹⁵ For Fibroscan, we used a binary cutoff at 7.1 with sensitivity and specificity of 67% and 89%, respectively, based upon data from 183 HCV infected individuals with a prevalence of significant fibrosis of 74.3%.¹⁴

Costs

Testing and other costs were obtained from published sources, online sources, Centers for Medicare and Medicaid Services fee schedule, and personal communications (Table 1), and were adjusted to 2005 US dollars by using the medical care component of the Consumer Price Index.¹⁶

Analyses

The following outcomes were calculated: test algorithm sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy (True Positives + True Negatives / Total Patients), total evaluation costs per patient, and incremental cost per accurate diagnosis. Uncertainty analyses were performed in which individual inputs were varied one at a time across a range of high and low estimates (See Table 1), to evaluate their influence on the model outcomes. The ranges were derived based on published sources or reasonable ranges from the base case estimate. To assess the possibility that liver biopsy misses a portion of patients with fibrosis or cirrhosis (i.e., sampling error), we set the sensitivity for liver biopsy to 80% in a scenario analysis.^{8, 17, 18} Specificity was assumed to be 100%, based on a study comparing liver biopsy with laparoscopy in the diagnosis of cirrhosis.¹⁹ This assumption is clinically valid since most clinicians would agree that cirrhosis is almost always present if seen on a typical percutaneous biopsy of sufficient length whereas the absence of cirrhosis on a needle biopsy specimen does not exclude the diagnosis because of sampling variability. Since liver biopsy was the reference standard used to derive the test characteristics for the noninvasive testing strategies, it is impossible to calculate how liver biopsy sampling error would affect the noninvasive testing strategies’ characteristics (i.e. sensitivity, specificity, positive and negative predictive values). Therefore, we constructed three possible scenarios: one favorable to the noninvasive tests (all false negatives by biopsy were accurately diagnosed by the noninvasive test), one unfavorable (all false negatives were misdiagnosed), and one scenario in which false negatives were distributed according to base case probabilities.

Results

The Fibrosure testing algorithm had sensitivity and specificity of 84% and 87%, respectively, a PPV and NPV of 74% and 92%, respectively, and an overall 86% accuracy. This resulted in approximately 9% false positives and 5% false negatives, for the entire tested cohort. In the base case, the average expected cost of using the Fibrosure testing algorithm was $488 per person. The cost per accurate diagnosis was $568. In comparison with liver biopsy alone, there was a cost savings of ~$766 per person, but a net decrease in the number of people accurately diagnosed of 14% (9% false positive and 5% false negative).

The Fibrospect II test had sensitivity and specificity of 81% and 71%, respectively, a PPV and NPV of 56% and 89%, respectively, and an overall 74% accuracy. This resulted in approximately 20% false positives and 6% false negatives, for the entire tested cohort. In the base case, the average expected cost of using the Fibrospect II test was $350 per person. The cost per accurate diagnosis was $471. In comparison with liver biopsy alone, there was a cost savings of ~$905 per person, but a net decrease in the number of people accurately diagnosed of 26% (20% false positive and 6% false negative).

The Fibroscan test had sensitivity and specificity of 67% and 89%, respectively, a PPV and NPV of 73% and 86%, respectively, and an overall 82% accuracy. This resulted in approximately 8% false positives and 10% false negatives, for the entire tested cohort. In the base case, the average expected cost of using the Fibrospect II test was $131 per person. The cost per accurate diagnosis was $159. In comparison with liver biopsy alone, there was a cost savings of ~$1,124 per person, but a net decrease in the number of people accurately diagnosed of 18% (8% false positive and 10% false negative).

In comparison with the Fibroscan testing algorithm, Fibrospect II had poorer overall performance across all test characteristics with a net difference in accuracy of 12%. The Fibrsoscan test improved the specificity by 2%, but the remaining test characteristics were lower with a net decrease in accuracy of 4% (Table 3).

Table 3.

Results of scenario analysis (Liver Biopsy Sensitivity of 80%)

	Liver Biopsya	Favorableb	Unfavorablec	Distributedd
Program Sensitivity	80%	92%	64%	78%
Program Specificity	100%	87%	87%	87%
Program PPV	100%	76%	69%	73%
Program NPV	92%	96%	84%	90%
Percent accurate diagnosis	94%	89%	80%	84%

^a: Sensitivity at 80%;

^b: False negatives by biopsy were accurately diagnosed by Fibrosure;

^c: False negatives were misdiagnosed by Fibrosure;

^d: False negatives were distributed according to base case probabilities;

Abbreviations: PPV: positive predictive value; NPV: negative predictive value

The uncertainty analysis indicated that program cost and cost per accurate diagnosis were most influenced by the cost of biopsy, the cost of the noninvasive tests, and the prevalence of significant fibrosis. Our scenario analyis focused on the Fibrosure alogorithm as it was the most accurate non-invasive testing strategy. With liver biopsy sensitivity set at 80%, the accuracy of the Fibrosure testing algorithm ranged from 80% to 89% between the extreme scenarios. The intermediate (distributed) scenario provided an accuracy of 84%--a decrease of 10% compared to liver biopsy alone (Table 4).

Discussion

Using a decision model, we found that the noninvasive testing options were less expensive (~$766 to $1,124 per person), but also less effective, with a 14% to 26% absolute decrease in the number of individuals accurately diagnosed, compared to liver biopsy in the initial evaluation phase. A scenario analysis that evaluated the effect of liver biopsy sampling error supported this finding with a net decrease in accuracy for the Fibrosure algorithm (the most accurate noninvasive testing strategy) ranging from 5% to 14% compared to liver biopsy alone.

Though the cost savings and decreased risk of biopsy-related injury and death achieved with the noninvasive testing options are attractive, the decreased diagnostic accuracy likely would have negative effects for the cohort of individuals undergoing noninvasive testing. For example, with the Fibrosure testing algorithm, approximately 9% of patients would be falsely labeled as having significant fibrosis. Drug spending for persons with false positive results would total approximately $1,700 per patient tested—erasing any cost savings of the Fibrosure algorithm in the pretreatment phase. Furthermore, about 5% of patients would be falsely labeled as not having significant fibrosis/cirrhosis. Presumably, effective treatments might be withheld for this group, as well as recommended monitoring for esophageal varices and hepatocellular carcinoma, which could lead to a decrease in life expectancy and quality of life.

To put these results in context, the long-term cost-effectiveness can be approximated based on previously published analyses. As stated above, there would be an additional 9% false positives and an additional 5% false negatives with the Fibrosure algorithm compared to liver biopsy alone in the base case. Previous studies have estimated the cost of treatment with peginterferon and ribavirin at approximately $19,000,² and gains in quality adjusted life-years (QALYs) for genotype 1 HCV patients of 0.5 for patients with mild fibrosis (gained by the 9% of false positives)² and 2.2 for patients with moderate to severe fibrosis/cirrhosis (lost by the 5% false negatives).²⁰ Because the loss of QALYs is approximately four times greater for false negatives than the gain for false positives, the Fibrosure algorithm would lead to greater per person costs (~ $20) and a decrease in QALYs (~ 0.06) compared to liver biopsy alone.

The findings herein are qualitatively similar to recently published reviews, which conclude that noninvasive tests for liver biopsy are currently insufficiently accurate to replace liver biopsy as standard of care in the pretreatment evaluation of liver fibrosis.^{7, 21} However, noninvasive tests may have a role in monitoring disease progression and/or provide an alternative for persons who decline liver biopsy or have contraindications (e.g. coagulopathy).¹⁰

A number of studies have demonstrated that liver biopsy is subject to sampling error.^{6, 7} The effect of sampling error is difficult to evaluate because the published studies of diagnostic accuracy for the noninvasive testing strategies employ liver biopsy as the reference standard. We attempted to evaluate the potential impact of sampling error in scenario analyses with unfavorable (all false negatives were misdiagnosed by Fibrosure), favorable (false negatives by biopsy were accurately diagnosed by Fibrosure), and distributed (false negatives were distributed according to base case probabilities) scenarios. Errors in the diagnosis of fibrosis using liver biopsy are likely to produce a decrease in sensitivity rather than a decrease in specificity as identifying fibrosis is easier than excluding it. As such, the scenario analysis involved a decrease in sensitivity, which resulted in decreased overall accuracy and increased false negatives for liver biopsy. This decrease in accuracy did not however drop below that of the most accurate noninvasive testing strategy (the Fibrosure algorithm) with a minimum difference in percent accurately diagnosed of 5%. The results of the scenario analyses suggest that the Fibrosure algorithm may be cost-effective if liver biopsy has a high false negative rate and the Fibrosure algorithm correctly classifies a large portion of the false negatives. As such, direct comparisons of the noninvasive testing options and liver biopsy using an alternative reference standard (e.g. laparoscopy) may be warranted.

Our comparison of the Fibrosure algorithm versus Fibrospect II and Fibroscan suggests that the Fibrosure algorithm would likely result in fewer false negatives and false positives as compared to Fibrospect II (Table 2). In the comparison of Fibrosure versus Fibroscan, the Fibrsoscan test improved the specificity by 2%, but the remaining test characteristics were lower with a net decrease in accuracy of 4% (Table 2). In addition, there are limitations on the use of Fibroscan as it does not perform well in patients with a high body mass index.²² Previous studies that have compared Fibrosure to other non-invasive tests have used a single cut-off for Fibrosure, which may bias the results in favor of alternatives. In comparison, we have used Fibrosure with liver biopsy for indeterminate results which takes advantage of the improved performance of Fibrosure in diagnosing at the high and low end of the scoring spectrum, thereby improving the overall test accuracy.

Table 2.

Results

Program Operating Characteristics	Liver Biopsy alone (1)	Fibrosure algorithm (2)	Difference (1–2)	Fibrospect II (3)	Difference (1–3)	Fibroscan (4)	Difference (1–4)
True postive (TP)	31%	26%	−5%	25%	−6%	21%	−10%
True negative (TN)	69%	60%	−9%	49%	−20%	61%	−8%
False postive (FP)	0%	9%	9%	20%	20%	8%	8%
False negative (FN)	0%	5%	5%	6%	6%	10%	10%
Program Sensitivity	100%	84%	−16%	81%	−19%	67%	−33%
Program Specificity	100%	87%	−13%	71%	−29%	89%	−11%
Program PPV	100%	74%	−26%	56%	−44%	73%	−27%
Program NPV	100%	92%	−8%	89%	−11%	86%	−14%
Number accurate diag (TP+TN)	100%	86%	−14%	74%	−26%	82%	−18%
Number incorrect diagnosis (FP+FN)	0%	14%	14%	26%	26%	18%	18%
Percent accurate diagnosis	100%	86%	−14%	74%	−26%	82%	−18%
Outcomes
Program cost per person	$1,255	$488	−$767	$350	−$905	$131	−$1,124
Cost of inappropriate drug spending* (millions)	$0	$1.7	−$1.7	$3.7	−$3.7	$1.4	−$1.4
Cost / accurate diagnosis	$1,255	$568	−$687	$471	−$784	$159	−$1,096

^*1 year treatment of false positive

We note several limitations of this study. In the base case, we did not account for imperfections in liver biopsy.¹⁷ As the noninvasive tests use liver biopsy as a reference standard, it is impossible to ascertain whether decreased accuracy for liver biopsy would increase or decrease the accuracy of the comparator. We explored several possibilities in our sensitivity analysis, but the overall accuracy of liver biopsy remained better than the most favorable of scenarios for the Fibrosure algorithm. Perhaps future studies could use laparoscopy or liver biopsy with 3 samples as reference standards; both of which have shown better accuracy than liver biopsy alone.^{19, 23} A further limitation was the use of intermediate outcomes rather than disease-related endpoints (e.g., liver failure, death or quality adjusted life-years). However, our base case findings suggest that compared to biopsy, noninvasive testing results in higher costs and potentially lower quality-adjusted life-years, and would be considered a dominated strategy from a health economics perspective.

In summary, the results of our analysis suggest that noninvasive tests for the pretreatment assessment of liver fibrosis had an overall accuracy of between 74% and 86% and demonstrated potential cost savings in the short-term compared to liver biopsy. However, the consequences of misdiagnosis of liver fibrosis in terms of both health outcomes and costs for inappropriate drug spending may outweigh the short-term gains in cost and convenience.