The cost-effectiveness of drug-eluting stents: a systematic review

Abstract

Background

Drug-eluting stents have been seen as an attractive alternative to bare-metal stents for percutaneous coronary interventions (PCIs) because of the decreased need for revascularization. However, comparative clinical trials have shown no difference in patient outcomes, and drug-eluting stents are considerably more expensive than their bare-metal counterparts. We conducted a systematic review of all published comparative cost-effectiveness analyses to identify the factors contributing to the heterogeneity of their conclusions.

Methods

We retrieved all articles published between Jan. 1, 2000, and July 31, 2006, in which the cost-effectiveness, from a third-party payer perspective, of drug-eluting stents was compared with that of bare-metal stents for PCI in unrestricted patient populations. Electronic databases, Web sites from health technology assessment groups and references of identified articles were searched. Our outcome variable was whether the study's conclusions favoured widespread use of drug-eluting stents, as assessed by 4 independent reviewers. Study characteristics such as quality, funding source, country and year of publication were extracted. Two-by-2 tables and Fisher's exact test were used to study the association between covariates and the outcome variable. A classification and regression tree (CART) model was used for multivariate analysis.

Results

We identified 19 cost-effectiveness analyses. Ten were in favour of widespread use of drug-eluting stents, and 9 favoured more restrained use. Only 1 of 9 high-quality studies supported widespread use, as compared with 9 of 10 lower quality studies (p < 0.001). All of the 7 sponsored studies argued in favour of widespread use, as compared with 3 of the 12 studies without sponsorship (p = 0.003). Studies from the United States were more likely than those from other countries to endorse unlimited use (p = 0.032). A CART model with 2 covariates — study quality and sponsorship — provided the best fit (error rate 10.5%).

Interpretation

Conclusions drawn by cost-effectiveness analyses of drug-eluting stents for PCI are associated with the study's quality, funding source and country of origin. Vigilance regarding these study characteristics is required when interpreting findings from cost-effectiveness analyses.

Drug-eluting stents became commercially available in 2002 in Europe and Canada and in 2003 in the United States¹ and are now widely used in the treatment of coronary artery disease. Restenosis of the target lesion has been the “Achilles heel” of percutaneous coronary interventions (PCIs), with 15%–40% of patients in randomized controlled trials (RCTs) who received bare-metal stents needing repeat revascularization within 6 months.² Numerous RCTs^3–6 and meta-analyses⁷ have shown that the use of drug-eluting stents reduces this risk by an estimated 83%. Given their comparable short-term safety profiles, drug-eluting stents have been seen as an attractive clinical alternative to bare-metal stents. However, no differences in deaths or myocardial infarctions have been seen between study groups receiving either type of stent.⁷ Furthermore, the absolute reductions in the rate of repeat revascularization observed with drug-eluting stents may be overestimated, owing to protocol-driven angiographic follow-up in RCTs.^8,9 Nevertheless, the clinical benefits of fewer repeat revascularizations with drug-eluting stents and improved quality of life, even if ephemeral, are enticing.

Because drug-eluting stents are considerably more expensive than bare-metal stents, with an estimated world market of $6 billion annually,¹⁰ cost-effectiveness analyses may help decision-makers assess the value of this additional investment. Although many cost-effectiveness studies have been carried out, they have arrived at different conclusions despite using a relatively constant measure of the efficacy of drug-eluting stents. By systematically reviewing all published cost-effectiveness analyses, we attempt to understand the determinants of the different conclusions.

Methods

We performed an extensive literature search for all English-language cost-effectiveness studies published between Jan. 1, 2000, and July 31, 2006, that assessed the impact of costs of drug-eluting stents compared with bare-metal stents on a health care system. We decided not to distinguish between the different models of drug-eluting stents because, although sirolimus-eluting stents appear to be more clinically effective than paclitaxel-eluting ones,¹¹ both are extensively used in clinical practice and because the prices of drug-eluting stents are comparable. Electronic search terms were “stent” and (“sirolimus” or “rapamycin” or “paclitaxel” or “taxus” or “cypher” or “drug eluting”] and [“economic(s)” or “cost(s)”]. We searched MEDLINE, EMBASE and the Cochrane Library as well as the HTA, DARE and NHS EED databases (www.york.ac.uk/inst/crd/crddatabases.htm). In addition, we reviewed 40 Web sites from known health technology assessment agencies in 21 countries (www.inahta.org) and references from identified articles. Studies were first identified based on their title and abstract and subsequently were retained if they met the following inclusion criteria: they were written from a third-party payer perspective; they used bare-metal stents as the comparator in their cost analyses; and they provided an original cost-effectiveness analysis from an unrestricted patient population.

For each study, geographic location, study quality, year of publication, funding source and overall conclusions were extracted. Because a minority of studies reported findings in the standard economic metric of cost per quality-adjusted life-year (QALY) and because we wished to simplify the interpretation of our results, the conclusion of each study was dichotomized as favourable or not favourable toward the widespread use of drug-eluting stents as an alternative to bare-metal stents. To strengthen the reliability of our assessment, this outcome variable was coded by 4 independent reviewers, 2 of whom were blinded to the studies' authors, their affiliations and funding sources. There was perfect initial agreement among the 4 reviewers on 15 of the 19 articles included in the analysis. Differences were discussed for the remaining 4 until universal agreement was reached.

To determine the quality of each study, we used a structured data-collection form created on the basis of previous published guidelines and recommendations (see Appendix 1, available online at www.cmaj.ca/cgi/content/full/176/2/199/DC1).¹² Each of the 16 items in this instrument were accorded 1 point if present, and a total quality score was obtained by summing across the items. Thus, the scale varied from 0 to 16, with higher values indicating better quality. Quality parameters were extracted and assessed independently by 2 of us (S.L. and F.V.) and discrepancies resolved by consensus.

Studies assigned a score above the median (10 points) were classified as high quality and the remaining as low quality. The measurement of quality is thus a relative measurement and gives information about the quality of one study compared with that of the other cost-effectiveness analyses. However, it does not provide information on the validity and appropriateness of the data used in the analyses. To evaluate the validity of this instrument, we performed a sensitivity analysis using the previously validated Quality of Health Economic Studies (QHES) instrument (see Appendix 2, available online at www.cmaj.ca/cgi/content/full/176/2/199/DC1)^13,14 which includes measures of the quality of the different input variables but is therefore also potentially more sensitive to subjectivity. Scores on the QHES scale vary from 0 to 100, with higher scores indicating better quality. Using the median of this scale (62), we again dichotomized studies as being of high quality or low quality, and any disagreements between the 2 extractors were settled by consensus.

Studies were considered to be sponsored if the original publications indicated that funding was provided directly by the manufacturer of a drug-eluting stent.

For statistical analysis, we used 2 × 2 tables to study the association between categorical covariates, and between each covariate and the main outcome measure. Fisher's exact test was used to examine the strength of the associations. We conducted a multivariable analysis, relating the outcome measure to all covariates, using a classification and regression tree (CART) model.^15,16 CART models are fit with the use of nonparametric statistical methods suitable for small data sets. At each stage of the CART model, the sample is split into 2 groups (or nodes) based on a single covariate that is associated with the greatest variability in the outcome measure. The classification process ends when there are 5 or fewer observations in a node.

Results

We identified 19 eligible studies published between Jan. 1, 2000, and July 31, 2006 (Fig. 1). The majority were found through our search of MEDLINE and EMBASE,^9,17–32 with an additional study each identified from the Cochrane Library³³ and the Web sites of known health technology assessment agencies.³⁴ When available, additional information referenced in the original publication was considered to permit a more thorough assessment of study quality.

Fig. 1: Results of literature search for cost-effectiveness analyses comparing drug-eluting stents (DES) and bare-metal stents (BMS). See Methods for search terms. INAHTA = International Network of Agencies for Health Technology Assessment (www.inahta.org).

The characteristics of the studies and whether they favoured widespread use of drug-eluting stents are presented in Table 1. Ten of the 19 studies were classified as being in favour of widespread use. Our main outcome measure was associated with a threshold of $50 000 per QALY in the 9 studies that reported this metric: both of the studies that reported a cost of less than $50 000 per QALY were in favour of widespread use, as compared with 1 of 7 studies that reported a cost of more than $50 000 per QALY (88.9% accuracy, p = 0.083). Exerpts from the original publications supporting our dichotomous classification are provided in Table 2.

Table 1

Table 2

One of the 9 studies with a quality score above the median supported widespread use of drug-eluting stents (Table 3), in contrast to 9 of the 10 studies with a quality score below the median (p < 0.001). All of the 7 sponsored studies argued in favour of widespread use of drug-eluting stents, in contrast to 3 of the 12 studies without direct sponsorship (p = 0.003). All of the 5 studies that performed a cost-effectiveness analysis from a US cost perspective concluded that drug-eluting stents were a favourable economic intervention, as compared with 5 of the 14 studies performed outside the United States (p = 0.0325). Regarding year of publication, no clear time trend was observed, although a larger proportion of early (pre-2005) studies than of studies published in 2005 or later favoured widespread use (75% [6/8] v. 36% [4/11], p = 0.17). One of the 8 studies published in 2005 reached a favourable conclusion (p = 0.041 compared with pre-2005 studies).

Table 3

In the sensitivity analysis in which we used the QHES instrument to validate our quality assessment measure, 3 studies of intermediate quality^24,28,29 changed categories, which resulted in our quality assessment measure having 80% sensitivity and 88.9% specificity with respect to the QHES instrument. When the QHES measure was compared with the outcome, 3 of 10 higher quality and 7 of 9 lower quality studies were found to favour widespread use of drug-eluting stents (p = 0.07). Although not statistically significant, this finding supports the overall robustness both of our original quality scale and of the association between study quality and the outcome measure.

Although the CART model was fit using all variables, the final model comprised only 2 predictors of our outcome measure: lower study quality and direct sponsorship by a manufacturer of a drug-eluting stent (Fig. 2).

Fig. 2: Results of classification and regression tree (CART) analysis showing study characteristics that were predictive of the main outcome measure (study favoured use of drug-eluting stents). Percentages in parentheses indicate the proportion of studies that favoured widespread use. Quality = quality as determined with the use of the instrument in Appendix 1 (available online at www.cmaj.ca/cgi/content/full/176/2/199/DC1); sponsored = study directly sponsored by manufacturer of drug-eluting stent.

Several of the study characteristics were interrelated. For example, lower study quality was associated with sponsorship (p = 0.06) and with the United States being the country of origin (p = 0.032). The United States as country of origin was also correlated with sponsorship (p = 0.038).

Interpretation

Our systematic review of studies evaluating the cost-effectiveness of drug-eluting stents demonstrated differences in the conclusions of the 19 identified studies, with 53% favouring and 47% not favouring widespread use of these stents. Specific study characteristics were found to be associated with the final study conclusion. Studies of lower quality and those that had direct sponsorship from a manufacturer of a drug-eluting stent were strongly associated with a positive endorsement of widespread use of drug-eluting stents. Also, studies performed with a US cost perspective were more likely than those performed in other countries to favour widespread use.

Sponsorship was inversely associated with the overall quality of these economic studies. Although a study of industry-sponsored RCTs did not find them to be of poorer quality,³⁵ RCT quality metrics are much simpler to evaluate than those of economic studies. Moreover, a recent publication of multiple systematic reviews demonstrated the same observation as ours of a relation between sponsorship and study quality.³⁶

Cost-effectiveness studies that favoured widespread use of drug-eluting stents were more likely than those not favouring such use to be published early after the introduction of this technology (Table 1), when encouraging results are most essential to advance dissemination and adoption. Compared with the earlier studies, later studies by health technology assessment agencies were in general more extensive, were of higher quality, had no industry sponsorship and argued that widespread use of drug-eluting stents was economically unattractive and that a more focused approach concentrating on high-risk patients was indicated.^{21,23,24,33,34} Because of their later publication date, their appearance in nonclinical journals and their length and complexity, their impact on clinicians' practices may be lower than that of earlier, more enthusiastic articles published in clinical journals. In 2006, there appears to have been a rebuttal with 3 additional articles favouring widespread use.^29–31 Through their associations both with industry funding and lower quality, our analysis suggests that care should be exercised in the interpretation of these early cost-effectiveness studies.

Finally, our findings suggest that the transferability of cost-effectiveness results from one country to another may be of concern. The difference in study conclusions can perhaps be explained in part by the variation in costs between different geographic locations. In support of this argument, the fact that all of the US studies favoured widespread adoption of drug-eluting stents may, at least in part, be explained by the high revascularization costs in the US health care system that could be substantially reduced with the use of drug-eluting stents. The threshold for accepting an intervention as being cost-effective is also higher in US studies. This suggests that other jurisdictions should carefully examine the conclusions of US cost-effectiveness studies before incorporating them into local decision-making and policy.

Our study has a number of strengths and makes several unique contributions to this burgeoning literature. First, although previous publications^35,37 have suggested an association between sponsorship and study conclusions, the analyzed studies were generally performed before 2000/01, and it was unknown whether the same trends persist currently. Previous studies have also examined multiple interventions, which increases interstudy variability and complicates overall analyses and interpretation. For example, the previous associations between favourable economic recommendations and industry sponsoring observed in a systematic review of 494 cost-effectiveness studies³⁷ could be explained, without evoking any suggestion of bias, if industry had selectively decided early in the life span of a technology not to pursue economically unattractive interventions. Our study is the first to examine exclusively one technology, thereby providing a common denominator for evaluation. This minimizes interstudy variability, enhances transparency and facilitates the attribution of study characteristics to outcomes. We were also able to reliably exclude time-varying efficacy measures as an explanation for the different study conclusions, since these have remained relatively stable over the clinical course of this technology. For example, the first clinical efficacy trials^2,3 showed results similar to those of a later meta-analysis that incorporated several more recent RCTs.⁷ Our outcome measure has also been diligently and transparently recorded (see Table 2) and was not encumbered by the use of differing, arbitrary QALY thresholds. The robustness of our conclusions with different quality scales and our use of relative quality comparisons are other strengths. Finally, this is the first study of sponsorship portending to the device industry.

Our study does have some limitations. First, the small sample limits the number of variables that can be independently assessed in a multivariable analysis. This is especially pertinent because several of our study characteristics were significantly correlated. Second, although we used 2 different scales, there is no definitive method on how to evaluate the quality of cost-effectiveness analyses.^39–41 Moreover, we were unable to completely assess the validity of the input variables for each study. Third, we excluded several studies not published in English, but later work should address this deficiency. Fourth, although our outcome measure — whether studies favoured widespread use of drug-eluting stents — may be considered too broad by some, it nevertheless avoids the ambiguities of more intricate measures. None of the original studies in our review considered the clinical and economic consequences of late stent thrombosis with drug-eluting stents. Finally, our evaluation of sponsorship was based on reported funding in the original publications and does not include any assessment of indirect or unreported funding or conflicts.

Our systematic review suggests that cost-effectiveness analyses can lead to partisan conclusions, at least regarding the evaluation of drug-eluting stents. Specifically, published studies have reached different conclusions despite the use of standardized measures of efficacy. We provide empirical evidence that lower study quality and direct industry sponsorship were associated with positive conclusions favouring widespread use of drug-eluting stents. Moreover, caution should be exercised in drawing conclusions from early cost-effectiveness studies, particularly if funded by industry, and in transferring conclusions from one country to another jurisdiction. Readers, journal editors and policy-makers need to be aware of the impact that these study characteristics may have on the conclusions of cost-effectiveness analyses. Vigilance is therefore required when interpreting findings from cost-effectiveness studies of drug-eluting stents, and possibly other cost effectiveness analyses.

@ See related articles, pages 195 and 199