Testing for HER2-positive breast cancer: a systematic review and cost-effectiveness analysis

Abstract

Background

Testing to determine HER2 status has come into focus since the approval of trastuzumab (Herceptin) for the treatment of HER2-positive breast cancer. We compared the cost-effectiveness of various strategies used to test HER2 status, an important first step toward evaluating the overall cost-effectiveness of trastuzumab therapy.

Methods

We performed a systematic review of studies that evaluated concordance between immunohistochemistry and fluorescence in situ hybridization testing to determine HER2 status. We performed a meta-analysis to estimate the distribution of immunohistochemistry scores in each category (0, 1+, 2+, 3+) and the probability of receiving a positive result of fluorescence in situ hybridization (which we assumed to be the “gold-standard” test) for each category. We calculated the accuracy and incremental cost per accurate diagnosis for each testing strategy compared with the base strategy (immunohistochemistry testing, followed by confirmation of 2+ scores by fluorescence in situ hybridization).

Results

The median percentage of patients in each category of immunohistochemistry score was: 0, 36.1%; 1+, 35.5%; 2+, 12.0%; and 3+, 16.2%. The median percentage of results of fluorescence in situ hybridization that were positive in each immunohistochemistry category was: 0, 1.6%; 1+, 4.9%; 2+, 29.8%; and 3+, 92.4%. The base strategy was expected to correctly determine the HER2 status of 96% of patients with breast cancer. Confirmation of the HER2 status by fluorescence in situ hybridization in cases that received a score of 3+ reduced the percentage of false-positive results to 0% and increased the percentage of accurately determined HER2 results to 97.6%. Compared with the base strategy, this strategy was associated with a median incremental cost-effectiveness ratio of $6175 per case of accurately determined HER2 status. The strategy of performing fluorescence in situ hybridization testing in all cases of breast cancer was associated with a median incremental cost-effectiveness ratio of $8401 per case of accurately determined HER2 status.

Interpretation

The strategy with the lowest cost-effectiveness ratio involved screening all newly diagnosed cases of breast cancer with immunohistochemistry and confirming scores of 2+ or 3+ with fluorescence in situ hybridization testing.

Overexpression of the human epidermal growth factor receptor-2 (HER2) gene, a breast cancer marker, is associated with rapid tumour growth, increased risk of recurrence after surgery, poor response to conventional chemotherapy and shortened survival.¹ The availability of targeted trastuzumab (Herceptin) therapy for tumours overexpressing HER2 protein (HER2 positive) has brought the need for accurate determination of HER2 status into sharp focus. Trastuzumab therapy improves the survival rate among women with metastatic or localized HER2-positive breast cancer.^2–6 However, trastuzumab has deleterious side effects and a high cost (Can$50 000 per year);^5–9 thus, it is important to accurately determine HER2 status.

There are 2 tests commonly used to determine HER2 status (Box 1): immunohistochemistry (detects overexpression of HER2 protein) and fluorescence in situ hybridization (detects amplification of the HER2 gene). A widely recommended¹¹ testing algorithm (referred to here as the base strategy)¹² is to consider an immunohistochemistry score of 3+ as positive, a score of 0 or 1+ as negative, and a score of 2+ as ambiguous and requiring confirmation with fluorescence in situ hybridization. Trastuzumab therapy is currently approved in Canada for use in women with either an immunohistochemistry score of 3+ or a positive fluorescence in situ hybridization result. Studies that showed fluorescence in situ hybridization to be more sensitive than immunohistochemistry in determining HER2 status^13,14 and a retrospective analysis that showed trastuzumab therapy to be beneficial only in patients who receive a positive result of fluorescence in situ hybridization¹⁵ have raised the issue of whether fluorescence in situ hybridization should be used to determine the HER2 status of all women with breast cancer.^7,16 In some countries (e.g., Belgium), only patients with a positive result of fluorescence in situ hybridization receive trastuzumab therapy.⁸

Box 1.

Immunohistochemistry is easier to perform than fluorescence in situ hybridization and is substantially less expensive (about 20% of the cost).^13,14,17 Immunohistochemistry can be performed in most surgical pathology laboratories. Fluorescence in situ hybridization, on the other hand, is performed only at select sites because specialized equipment and training are required. Despite these arguments in favour of immunohistochemistry testing, there are concerns that it is less accurate, is not standardized and has poor reliability; however, reliability has been improved by the availability of commercial kits. Fluorescence in situ hybridization has generally been shown to have better validity and reliability and is widely considered to be the “gold standard.”^10,17

We estimated the incremental cost-effectiveness of various strategies used to determine HER2 status (based on immunohistochemistry and fluorescence in situ hybridization). This analysis is a necessary precursor to a full cost-effectiveness analysis of trastuzumab therapy for different stages of breast cancer.

Methods

We performed a search of the PUBMED, EMBASE and MEDLINE databases to identify relevant studies published before June 2006. We used the following keywords: (sensitivity OR specificity OR reliability OR reproducibility OR validity OR interobserver OR intraobserver OR accuracy OR predictive value) AND IHC (or immunohistochemistry) AND FISH (or fluorescence in situ hybridization) AND HER2 AND breast. We included studies that met the following criteria: published in English or French in a peer-reviewed journal; included human patients; included consecutive patients or a random sample; reported agreement between results of immunohistochemistry and fluorescence in situ hybridization using standard cutoff scores; and used assays (following manufacturers instructions) licensed by Health Canada to identify patients eligible for trastuzumab therapy. We also searched the bibliographies of the included articles to identify additional relevant studies.

Some of the included studies had a nonrepresentative patient sample, despite including consecutive patients, because patients were recruited at centres where they had been referred for fluorescence in situ hybridization testing, which resulted in an overrepresentation of cases with 2+ or 3+ immunohistochemistry scores. This selection bias (commonly referred to as verification bias) could artificially increase the sensitivity of immunohistochemistry but decrease its specificity.¹⁸ In some studies the presence of the bias was apparent from the methods. If the percentage of cases with scores of 2+ or 3+ exceeded 40%, we treated the study as having selection bias, even if this bias was not evident from the methods. We did not include these studies while estimating the distribution of immunohistochemistry scores, but we did include them while estimating the rate of positive results of fluorescence in situ hybridization.

We performed a Bayesian meta-analysis to estimate the distribution of immunohistochemistry scores (0, 1+, 2+, 3+) and the probability of obtaining a positive result of fluorescence in situ hybridization for each category of immunohistochemistry score. We analyzed the results of the 0 and 1+ cat-egories separately where possible. We expected the lowest rate of positive fluorescence in situ hybridization results to occur in the immunohistochemistry 0 category and the highest rate to occur in the 3+ category. The goal of this analysis was to estimate the percentage of patients whose HER2 status had been accurately determined by immunohistochemistry. We repeated the analysis using the subset of studies that reported results of immunohistochemistry and fluorescence in situ hybridization performed at the same site. This eliminated the studies with high variability in the immunohistochemistry assay that could have been caused by the test being performed at multiple laboratories.^19,20 We also excluded studies from this repeat analysis if we could not determine whether all immunohistochemistry tests had been carried out using a licensed assay.

Under the Bayesian approach, prior knowledge, summarizing our uncertainty about the unknown parameters, is combined with information in the observed data (via the likelihood) to yield a posterior distribution for the unknown parameters.²¹ Descriptive statistics for the unknown parameters, such as the median and 95% credible interval (Bayesian equivalent of a confidence interval), can be determined from the posterior distribution. We selected prior distributions to allow all parameter possibilities to have equal weight so that the observed data would dominate the final result. The analysis was implemented using the WinBUGS software program (version 1.4).

We performed cost-effectiveness analyses of 7 diagnostic strategies (Table 1) from the perspective of the Ministry of Health and Social Services in Quebec. We compared the different strategies based on the combination of immunohistochemistry and fluorescence in situ hybridization tests performed to determine HER2 status. The costs of testing and treatment were obtained from billing information maintained by the provincial health insurance agency in Quebec. The probability of an accurate determination of HER2 status based on each strategy was estimated using the posterior distributions obtained by meta-analysis.

Table 1

All strategies were compared with the “base strategy,” which involves screening all patients who received a new diagnosis of invasive breast cancer with immunohistochemistry followed by confirmatory testing of 2+ scores with fluorescence in situ hybridization. Any strategy that was more expensive but less accurate than another strategy (i.e., not economically attractive) was considered to be “dominated” and was removed from the analysis. Strategies that were less effective and less expensive were also eliminated, because they were unlikely to be acceptable to lower the number of women whose HER2-positive status is accurately diagnosed. We compared the remaining strategies with the base strategy using an incremental cost-effectiveness ratio (defined as the ratio of the difference in costs to the difference in the number of cases with accurately determined HER2 status). We must emphasize that the incremental cost-effectiveness ratio reported here refers only to the cost per diagnosis, not to the cost of trastuzumab therapy. Any strategy that was less accurate but had a higher incremental cost-effectiveness ratio than another strategy was also considered dominated by “extended dominance” (i.e., less attractive in terms of cost per additional accurate diagnosis).

We assessed the robustness of our cost-effectiveness analysis by varying all input variables simultaneously over plausible ranges. The purpose of the sensitivity analysis was to estimate the confidence interval around the incremental cost-effectiveness ratios when allowing for uncertainty in all the variables in our model. The distributions of the immunohistochemistry scores and the rate of positive results of fluorescence in situ hybridization were determined from the meta-analysis. For the distribution of the cost of each test, we used a uniform distribution over the interval defined by +/-20% around the cost.

Results

The study selection process is summarized in Fig. 1. The results from the 17 included studies^{11,13,14,19,20,22–33} are presented in Table 2. Six studies were identified as having selection bias.^{11,19,20,25,32,33} Sixteen of the included studies used the Dako HercepTest for immunohistochemistry; the remaining study used Ventana Medical Systems' Pathway assay.³⁰ All of the included studies used the PathVysion kit from Vysis for fluorescence in situ hybridization.

Fig. 1: Results of literature search to identify studies that compared the use of immunohistochemistry (IHC) and fluorescence in situ hybridization (FISH) to determine HER2 status. Note: some studies were excluded for multiple reasons.

Table 2

The results of the Bayesian meta-analysis are summarized in Table 3. We obtained similar results in the subgroup analysis of 15 studies in which the immunohistochemistry and fluorescence in situ hybridization were performed at the same site (data not shown).^{11,13,14,22–33} The number of breast cancer cases in the immunohistochemistry 0 and 1+ categories accounted for about 70% of all cases, about 50% of whom had an immunohistochemistry score of 1+. The rate of positive results of fluorescence in situ hybridization was very low (median < 5%) among cases with an immunohistochemistry score of 0 or 1+. The rate varied greatly in the 2+ category (95% credible interval 12.0%–56.5%), and as expected, it was highest in the immunohistochemistry 3+ category. The estimated rate of false-positive results in the 3+ category was 3.8%–12.9%, or about 6–23 false-positive results per 1000 breast cancer patients screened. Based on these estimates, the median number of positive results of fluorescence in situ hybridization per 1000 breast cancer patients screened was 211 (95% credible interval 162–268). The estimated percentages of true-and false-positive results that would be expected using each testing strategy are presented in Appendix 1 (available online at www.cmaj.ca/cgi/content/full/176/10/1429/DC1).

Table 3

The results of the cost-effectiveness analysis are shown in Appendix 2 (available online at www.cmaj.ca/cgi/content/full/176/10/1429/DC1) and are presented in terms of costs per 1000 breast cancer cases screened (Canadian dollars, 2006 equivalent). The current cost of immunohistochemistry is $108 per test, and the current cost of fluorescence in situ hybridization is $467 per test³⁴ (includes both technical costs [e.g., reagents and equipment] and cost of the pathologist's time). Under the base strategy (Strategy 1), the median cost of testing per 1000 women was estimated to be $164 000, and the median percentage of accurately identified cases was estimated to be 96%. Of the remaining strategies, 3 were not dominated or eliminated because of lower effectiveness than other studies: strategy 5 (fluorescence in situ hybridization testing in cases with an immunohistochemistry score of 2+ or 3+), strategy 6 (fluorescence in situ hybridization testing in cases with an immunohistochemistry score of 1+, 2+ or 3+) and strategy 7 (fluorescence in situ hybridization for all breast cancer cases). The median incremental cost-effectiveness ratios for these strategies were between $6175 (strategy 5) and $8401 (strategy 7) per accurately determined HER2 status. The increase of 1.6% in the median percentage of accurate diagnoses with strategy 5 compared with the base strategy was due to the reduction of false-positive results to 0%. Thus, strategy 5 would reduce the cost of trastuzumab therapy by about $0.6 million per 1000 women screened. Strategies 6 and 7 were much more expensive because they involved testing the large group of patients who received an immunohistochemistry score of 0 or 1+, among whom the rate of positive results of is low. Strategy 7 had the highest median cost but also had a 4% increase in the number of accurate diagnoses compared with the base strategy. This was expected because we considered fluorescence in situ hybridization to be the gold standard.

The sensitivity analysis allowed us to determine the 95% confidence intervals for the incremental cost-effectiveness ratios (cost per accurate diagnosis) of the 3 competing strategies: strategy 5, $3351–$12 230; strategy 6, $3913–$13 630; strategy 7, $5315–$13 260.

Interpretation

The availability of an efficacious, yet expensive, treatment for HER2-positive breast cancer underlines the need for accurate determination of HER2 status. We performed a systematic review and cost-effectiveness analysis that compared various testing strategies involving 2 widely used tests to determine HER2 status.

Based on our meta-analysis, we estimated that, for every 1000 breast cancer patients whose HER2 status was tested by immunohistochemistry, 362 would receive a score of 0, 355 would receive a score of 1+, 120 would receive a score of 2+, and 162 would receive a score of 3+. We estimated that the number of true HER2-positive cases would be 211. Among the cases that received an immunohistochemistry score of 3+, the rate of false-positive results could be as high as 7.6%. For every 1000 women screened, about 12 would receive a false-positive result and would be exposed to the risks of trastuzumab therapy with virtually no chance of gaining the benefits of this treatment. If the HER2 status of these women were accurately determined, the cost of trastuzumab therapy would be reduced by about $0.6 million per year. About 20 000 women receive a diagnosis of breast cancer every year in Canada. Of these women, an estimated 4218 would truly be HER2 positive and about 240 would receive a false-positive result (immunohistochemistry 3+ score and negative result of fluorescence in situ hybridization). If the HER2 status of these 240 women were correctly determined and no trastuzumab therapy given, the cost of this therapy would be reduced by about $12 million across the country each year.

Although fluorescence in situ hybridization is less than 100% sensitive and specific, the literature suggests that it is superior to immunohistochemistry.^13–15 Therefore, we considered it a reasonable assumption to view fluorescence in situ hybridization as the gold standard. A widely followed practice is to screen all cases of breast cancer with the use of immunohistochemistry and to perform fluorescence in situ hybridization to confirm the result only if the score is 2+. However, it is of interest to evaluate the benefit of fluorescence in situ hybridization testing in cases with scores other than 2+. We found the incremental cost-effectiveness ratio was lowest when cases with immunohistochemistry 2+ and 3+ scores were confirmed by fluorescence in situ hybridization. If all women with newly diagnosed breast cancer were screened with fluorescence in situ hybridization, the cost of testing would be increased by about $303 000 for every 1000 women screened (compared with Strategy 1); however, the percentage of accurate diagnoses would be expected to be 100%.

In previous cost-effectiveness analysis, Elkin and colleagues⁷ reached similar conclusions, although their model included only women with metastatic breast cancer and their analyses adjusted for the efficacy of trastuzumab. We believe that the costs of testing obtained from the Ministry of Health and Social Services in Quebec reflect of the relative costs of immunohistochemistry and fluorescence in situ hybridization testing; thus, our conclusions should be generalizable beyond the province of Quebec.

Besides cost-effectiveness, there are other reasons to be cautious about performing confirmatory testing and prescribing trastuzumab therapy to women who receive an immunohistochemistry score of 1+. In particular, the ramifications of not prescribing trastuzumab therapy in these cases are unknown; cases with 0 and 1+ scores were not included in the HER2 clinical trials.^2–6 One study showed there is no significant difference in survival (in the absence of treatment) among women with an immunohistochemistry score of 0 or 1+.³⁵ Patients with such scores who receive a positive result of fluorescence in situ hybridization generally have been found to have a low level of HER2 amplification.^31,32,35 To date, there is no literature on the relation between trastuzumab efficacy and level of HER2 amplification. Thus, there is no evidence that trastuzumab therapy is beneficial to patients who have an immunohistochemistry score of 0 or 1+. It is therefore hard to justify the increased costs of further testing (by fluorescence in situ hybridization) in this group of patients; this would substantially impact the volume of testing, which would potentially increase the need to purchase more equipment and further increase costs.

Although our analysis has helped to identify a testing strategy with the lowest incremental cost-effectiveness ratio, a major limitation is that our study was cross-sectional and did not account for the longitudinal costs and benefits of trastuzumab treatment. A more complete analysis would account for the efficacy of trastuzumab, the increased life expectancy (based on cancer stage and patient age), the increased risk of cardiac toxicity with trastuzumab therapy and the cost to patients who receive a false-negative result and do not receive treatment. Our study is a necessary first step toward such an analysis.

In conclusion, we have shown that the strategy with the lowest cost-effectiveness ratio for HER2 testing, compared with current practice, is to screen all breast cancer patients with immunohistochemistry and to confirm 2+ and 3+ scores with fluorescence in situ hybridization. In addition to improving specificity, this approach could lead to major cost savings in treatment with trastuzumab.

@ See related article page 1443