Do Economic Evaluations of Targeted Therapy Provide Support for Decision Makers?

Economic evaluations of adjuvant trastuzumab were reviewed. Three primary shortcomings were identified including incorporation of local data and estimation and representation (visual) of decision uncertainty.

Abstract

Purpose:

Decision makers must make decisions without complete information. That uncertainty can be decreased when economic evaluations use local data and can be quantified by considering the variability of all model inputs concurrently per international evaluation guidelines. It is unclear how these recommendations have been implemented in evaluations of targeted cancer therapy. By using economic evaluations of adjuvant trastuzumab, we have assessed the extent to which decision support recommendations were adopted.

Study Design:

Systematic review.

Methods:

Published economic evaluations of adjuvant trastuzumab treatment in early-stage breast cancer were examined as an established example of targeted therapy. Canadian, United Kingdom, and US economic evaluation guidelines were reviewed to establish extraction criteria. Extraction characterized the use of effectiveness evidence and local data sources for model parameters, sensitivity analysis methods (scenario, univariate, multivariate, and probabilistic) and uncertainty representation (ie, cost-effectiveness plane, scatterplot, confidence ellipses, tornado diagrams, cost-effectiveness acceptability curve).

Results:

Fifteen economic evaluations of adjuvant trastuzumab were identified in the literature. Local data were used to estimate costs (15 of 15) and utilities rarely (two of 15) but not trastuzumab efficacy. Univariate sensitivity analysis was most common (12 of 15), whereas probabilistic analysis was less frequent (10 of 15). Two thirds of all studies provided visual representation of results and decision uncertainty.

Conclusion:

Authors of adjuvant trastuzumab economic evaluations rarely use local data beyond costs. Quantification of uncertainty and its representation also fell short of guideline recommendations. This review demonstrates that economic evaluations of adjuvant trastuzumab, as an example of targeted cancer therapy, can be improved for decision-making support.

Introduction

Economic evaluation is a tool used by policy and decision makers to address the relationship between clinical effects and costs associated with diagnosis, treatment, adverse effects, supportive health care, and life gained or lost. Payers, providers, and physicians can use economic evaluations to inform drug formulary listing, procedure or device reimbursement, and patient care decisions.^1–8 Decision analytic models have provided valuable support for health policy decisions ever since the Centers for Disease Control first presented such evidence to support vaccine recommendations in the late 1960s.² More recently, these methods are being applied to targeted drug therapies.

Targeted therapies, or personalized medicines, allow physicians to tailor treatment to individual patients. These medicines exert their effect by specifically targeting biologic processes via gene or protein expression⁹ and, though costly, can potentially offer substantial clinical and economic offsets by avoiding ineffectual treatment and minimizing adverse effects. Therefore, decision analytic modeling and economic evaluation of targeted therapies are powerful tools with which clinical efficacy and costs can be weighed against standard care. Nonetheless, care must be taken to ensure that analyses are conducted in a manner that supports informed health care decision making. Many countries have outlined explicit economic evaluation guidelines to encourage appropriate conduct for decision-making purposes. To date, it is unclear how closely researchers have followed guidelines. Understanding how economic evaluations of targeted therapies are designed to inform decision making could enhance the health policy and managed-care environments.

In this article, we examine how economic analyses of targeted therapy were conducted with a focus on informing health care decisions from the payer's perspective. Given its widespread uptake and considerable success in the treatment of breast cancer, trastuzumab (Herceptin; Genentech, South San Francisco, CA) was chosen for assessment. Two decades of clinical study and application have facilitated several economic evaluations of the drug and this systematic review examines those evaluations to understand whether analyses of targeted therapy were reported in a manner that supports informed health care decision making. We used economic evaluation guidelines from Canada, the United Kingdom, and the United States to establish decision support criteria. Our review focuses on recommendations specifically designed to aid the decision-making process by increasing the relevance of the economic evaluation to the decision maker's setting and encouraging quantification and representation of decision uncertainty.

Methods

Systematic Search Strategy and Study Selection

A search strategy was previously developed¹⁰ to identify published, peer-reviewed economic analyses of trastuzumab in the adjuvant treatment of breast cancer. The search encompassed literature published through October 2008 that were indexed in Biosis, Cochrane, the Centre for Reviews and Dissemination, EconLit, EMBASE, the Health Economic Evaluations Database, MEDLINE, and PubMed electronic databases; we updated EMBASE and MEDLINE searches to February 2011. Only English language citations were considered. Economic evaluations were included if they represented original research; considered two or more alternatives in an incremental of cost-effectiveness, cost-utility, cost-benefit, or cost-minimization; and focused on the evaluation of trastuzumab therapy in the adjuvant setting. Abstracts were reviewed independently by two assessors, and relevant articles were obtained in full for additional review. Selection of studies on the basis of reviews of the full articles was conducted by a single reviewer, and a random sample was verified independently.

Data Extraction

We reviewed Canadian,¹¹ United Kingdom¹² and US¹³ national drug or drug and device economic evaluation guidelines to identify recommendations for increasing the relevance of the analysis to the decision maker's setting and quantification and representation of decision uncertainty. The items identified from each guideline were then extracted from included studies. The items selected for abstraction are listed in Table 1 along with the relevant guidance from each country. We excluded the recommendation to model local standard care and practice patterns because of the difficulty in identifying and verifying local patterns across international treatment settings and language barriers.

Table 1.

Summary of Canadian, United Kingdom, and US Guidelines Specific to Efforts to Aid in the Decision-Making Process

Country	Guideline Type	Data Input Recommendation				Methodology Recommendation
		Purpose	Effectiveness	Cost	HRQOL	Sensitivity Analysis	Decision Uncertainty
Canada11	Drug and nondrug technologies	To support the production of “credible and standardized” economic evaluations for use in the Canadian publicly funded decision-making environment	Systematic review; incorporate real-world factors that modify the effect if feasible (ie, patient adherence, diagnostic screening accuracy, physician compliance and skill, entire episode of care for devices)	Identify, measure, and value resources relevant to the decision makers' setting (including use of local unit costs)	Preferred measurement from the general public that funds the health care system	Minimum of deterministic analysis for all inputs but should include additional methods	Quantify the contribution of each parameter to decision uncertainty
				Base resource use estimates on Canadian routine practice	Preferred measurement from the general public that funds the health care system	Probabilistic analysis encouraged to assess parameter uncertainty	Value of information methods suggested
							Tornado diagrams, cost-effectiveness planes, and CEACs recommended as visual representation
United Kingdom12	Drug and nondrug technologies	To provide an overview of appropriate methods for technology appraisal (including systematic review, economic evaluation) for submission to NICE	Evidence of effectiveness preferred over efficacy Systematic review required	Resources should be valued using prices relevant to the NHS	Should be elicited from a representative sample of the United Kingdom population	Distributions should be assigned to characterize parameter uncertainty	Value of information analysis suggested
			Typical patients, routine clinical practice, clinically relevant outcomes should all be included			Probabilistic sensitivity analysis is preferred	Confidence ellipses, scatter plots, and CEACs appropriate for visually representing uncertainty
						Scenario analysis should be used to examine assumptions
United States13	Drug technologies	To improve the “scope, quality, and relevance” of economic evaluations provided to the health care system evaluators and formulary decision makers	Transformation of efficacy estimates into effectiveness is encouraged by incorporating estimates of real-world factors (eg, adherence, comorbidities)	Opportunity costs relevant to decision maker	Not described	One-way deterministic analysis of all variables	One-way sensitivity analysis results should be presented in a figure (eg, tornado diagram)
						Scenario analysis, net benefit, probabilistic analysis, and CEACs recommended	CEAC recommended if probabilistic analysis conducted

Abbreviations: CEAC, cost-effectiveness acceptability curve; HRQOL, health-related quality of life; NHS, National Health Services; NICE, National Institute for Health and Clinical Excellence.

Data was extracted to a single form for data input and decision uncertainty. Here, we use decision uncertainty to represent our understanding of the likelihood that the result predicted by an economic evaluation will occur in practice. To understand how authors made each evaluation relevant to the decision maker's setting, we extracted the source for the following parameters and categorized the source as “local data” or “literature”: human epidermal growth factor receptor 2 (HER2) test properties, trastuzumab efficacy, risk of recurrence or survival, cost, and utility estimates. For an item to be considered local data, the model parameter needed to be derived from actual practice in the jurisdiction of the evaluation or measured from the disease population of that jurisdiction. For example, health state utilities used in an economic evaluation in the United States were considered local if the utilities were measured from a US population of patients with the disease of interest.

Quantification and representation of decision uncertainty was also documented. We extracted parameter type (stochastic [point estimate selected at random from a distribution] or deterministic [single point estimate]) to gain an understanding of the approach used to represent the “best guess” estimate of any variable considered in the evaluation. The methods of assessing uncertainty in those parameters and assumptions (termed sensitivity analysis) were subsequently extracted. Use of univariate, multivariate, scenario, or probabilistic sensitivity analysis was noted, including which parameters were assessed by each method. It was crucial to distinguish the methods of sensitivity analysis as each serves a different purpose. Univariate analysis involves changing a single parameter estimate at a time to understand how that parameter influences results.¹⁴ Multivariate or scenario analysis involves changing multiple variables simultaneously, usually to represent some alternative set of circumstances, to understand the impact on results.¹⁴ Univariate and multivariate analysis most frequently employ deterministic parameters. Finally, probabilistic analysis involves assigning distributions to model parameters (stochastic) and allowing each to vary randomly and concurrently to generate an empirical distribution for the cost-effectiveness ratio.¹⁴ We also documented whether visual representation of results and uncertainty was provided and the type of graphic used to represent that uncertainty (collectively termed decision aids). In this context, provision of decision aids was defined as clear graphic presentation of the cost-effectiveness plane with a scatterplot or confidence ellipses or of univariate sensitivity analysis results or cost-effectiveness acceptability curves (CEACs) with tornado diagrams per the reviewed guidelines. Results presented on the cost-effectiveness plane as a scatterplot or with confidence ellipses give the reader a sense of the distribution of incremental cost-effectiveness ratio (ICER) results. The CEAC shows the probability that a given intervention is more cost-effective than its comparator(s) over a range of willingness-to-pay values, providing the decision maker with an estimate of the likelihood that choosing to adopt the intervention would in fact be the right choice.¹⁵ We also considered value of information (VOI) analysis, because this method was suggested by both Canadian and United Kingdom guidelines.^11,12 Moreover, VOI relates the decision uncertainty of the model or specific parameters to the cost of conducting additional research to decrease that uncertainty¹⁶ and therefore provides information to support decision making.

Results

Search Results

The updated MEDLINE and EMBASE searches returned an additional 385 citations to the 958 citations previously identified. Duplicate citations accounted for 224 of the total, which left 1,119 for review. Abstract review resulted in the exclusion of an additional 694 citations. A total of 15 studies remained after application of the inclusion criteria during full citation review. Figure 1 summarizes the study identification and selection process. The 2006 National Institute for Health and Clinical Excellence (NICE) report on the use of trastuzumab in early-stage breast cancer was included with the additional extraction of data from the related manufacturer's submission, which was available from the NICE Web site.¹⁷ Several conference abstracts were identified but not included in the review because complete peer-reviewed articles were not available.^18–24

Figure 1.

The flow of studies through the review process to identify health economic evaluations of adjuvant trastuzumab in early-stage breast cancer (BC).

Take-Away Points.

Economic evaluations of adjuvant trastuzumab, as an example of targeted therapy, can better support informed decision making through increased use of local data to inform model parameters and quantification and graphic communication of decision uncertainty.

• Data reflecting local practice is rarely used to inform model parameters beyond costs.

• Joint analysis of parameter uncertainty using probabilistic sensitivity analysis was reported in two thirds of reviewed studies.

• Graphics aimed at communicating the results of economic analysis and sensitivity analysis were provided in two thirds of all studies.

• These methods are supported by several international guidelines.

A brief synopsis of economic evaluation methods, settings, and findings of the reviewed articles is presented in 2009 US dollars in Table 2. Overall, trastuzumab therapy was associated with an ICER deemed cost-effective in early-stage breast cancer by the majority study authors.¹⁰ Additional studies identified in this updated review are generally consistent with that finding. However, Skedgel et al²⁷ noted that the cost-effectiveness of adjuvant trastuzumab in Canada could exceed the widely cited $50,000 per quality-adjusted life-year and $100,000 per quality-adjusted life-year thresholds and that this finding was largely dependent on the assumed duration of trastuzumab benefit. Indeed, 10 of 15 studies in the United States and international settings noted sensitivity to the assumed duration of trastuzumab benefit (typically 5 years) or the relative risk reduction associated with therapy. This suggests that follow-up on the long-term benefits of trastuzumab and the relative benefit of 52-week therapy compared with 9-week therapy will be crucial to understanding its cost-effectiveness in the adjuvant setting. Most authors did not consider local willingness-to-pay thresholds when concluding the cost-effectiveness of trastuzumab. The choice of testing strategy significantly impacted that ICER when test properties were modeled in conjunction with treatment. Some analyses suggested that a 9-week trastuzumab regimen²⁸ could result in potential cost savings compared with 52-week therapy²⁹ but that additional long-term data were needed. The results of several studies were sensitive to the cost of trastuzumab therapy.

Table 2.

Updated Summary of Findings From a Previous Review of Adjuvant Trastuzumab Therapy Economic Analyses

Analysis Feature	Updated Summary of Review Findings for Adjuvant Therapy (n = 15)
Perspectives*	Payer (n = 12) Societal (n = 2) Partial societal (n = 2)
International settings*†	Australia, Belgium (n = 2), Canada, China, Finland, Italy, Japan, the Netherlands, Norway, Sweden, Switzerland, United Kingdom, United States (n = 3)
Methods	Analytic approach: cost-effectiveness (n = 3), cost-utility (n = 12) Model type: Markov model (n = 12), mixed model (n = 1), unknown (n = 2) Simulation type: hypothetical cohort (n = 15)
Time horizons*	2 to 5 years (n = 1) 6 to 15 years (n = 3) 16 to 51 years (n = 8) Lifetime (n = 7)
Results (ICER against comparator)‡	ICER estimates varied widely for 52-week therapy ranging from $7,902/QALY in Beijing (lifetime) to $126,580/QALY in Canada (lifetime) and from $24,822/LYS in Japan (lifetime) to $57,544/LYS in Belgium (lifetime)
Most influential parameters	Drug cost (n = 10) Discount rate (n = 8) Duration of survival benefit (n = 8) Trastuzumab relative risk reduction (n = 6) Test sensitivity/specificity (n = 2)
Concluding remarks	Trastuzumab was largely considered cost-effective in the adjuvant setting with 9-week therapy demonstrating lower ICERs than 52-week treatment; hesitation was expressed over the quality of the evidence for 9-week therapy v 52-week The choice of testing strategy can significantly impact trastuzumab cost-effectiveness Drug price and the duration of trastuzumab benefit were seen as key sources of variation in ICER estimates

NOTE. Data adapted.¹⁰

Abbreviations: ICER, incremental cost-effectiveness ratio; LYS, life-year saved; QALY, quality-adjusted life-year.

^*Does not add to 15 because some authors considered more than one international setting, perspective, or time horizon.

^†n = 1 unless otherwise noted.

^‡All currencies converted to 2009 US$ by using annual average exchange rates²⁵ and the medical component of the US consumer price index.²⁶

Relevance to the Decision Maker's Setting

Table 3 lists the data sources reported among analyses of trastuzumab. Cost data were locally derived in all studies. Costing methods often reflected trial protocols or other published cost studies, although two authors used microcosting to reflect local practice patterns.^42,48 All remaining parameter categories were rarely informed by locally derived sources. Measures of treatment efficacy and utility estimates were sourced from the literature in all reports. Two authors used at least some utilities derived from local studies.^42,48

Table 3.

Relevance of the HER2 Testing or Trastuzumab Treatment Economic Evaluation to the Decision Maker's Setting

Author	Source of Data
	Test Accuracy	Trastuzumab Efficacy, Risk of Recurrence, or Survival	Cost	Utility Estimates
Blank30	Literature31,32	Literature29	Local unit costs and literature	Literature; Swedish estimates
Chen33	Not modeled	Literature29	Local unit costs	Literature; United Kingdom estimates
Dedes34	Not modeled	Literature18,28,29	Local unit costs	Not modeled
Essers35	Not modeled	Literature29	Local unit costs but on the basis of United Kingdom resource use	Literature; Swedish estimates
Garrison36	Not modeled	Literature37	Local unit costs and literature	Literature
Kurian38	Not modeled	Literature37	Local unit costs and literature	Literature
Liberato39	Not modeled	Literature21,37	Local unit costs for 2 countries	Literature
Lidgren20	Literature40	Literature19	Local unit costs; future costs from literature	Literature; studies conducted locally
Millar41	Not modeled	Literature42,37	Local unit costs	Literature
Neyt43	Not modeled	Literature28,29	Local unit costs and literature	Not modeled
NICE17,44	Not modeled	Literature29	Manufacturer study and data obtained from Abacus	Manufacturer study and literature
Norum42	Not modeled	Literature37	Local unit costs	Literature; some utilities derived locally
Shiroiwa45	Not modeled	Literature46	Local unit costs	Not modeled
Skedgel27	Not modeled	Literature29,46	Local cost studies at cancer unit	Literature
Van Vlaenderen47	Not modeled	Literature29; underlying progression rates from Belgian registry	Local unit costs and literature	Literature; largely US estimates

NOTE. Data sources were extracted and reviewed for relevance by evaluating whether the source was locally derived from the decision maker's setting.

Abbreviations: HER2, human epidermal growth factor receptor 2; NICE, National Institute for Health and Clinical Excellence.

Quantification of Decision Uncertainty and Decision Aids

Univariate analysis of deterministic parameters was conducted in the majority of the adjuvant trastuzumab studies. Despite the widespread guideline support for probabilistic sensitivity analysis, this technique was used in 10 of 15 studies, particularly in more recent publications. Additionally, multivariate or scenario analyses were conducted in six studies, and two studies used univariate analysis exclusively.^34,42 CEACs were presented in eight of 10 studies that used probabilistic analysis. Beyond the CEAC, the tornado diagram⁴³ and the scatterplot^30,47 were the only other decision aids provided. No trends were noted with respect to international settings and sensitivity analysis conduct. A summary of uncertainty analysis methods is provided in Table 4.

Table 4.

Quantification of Representation of Decision Uncertainty in Published Economic Evaluations of HER2 Testing and Trastuzumab Targeted Therapy

Author	Model Parameter (deterministic v stochastic)	Scenario Analyses	Univariate Sensitivity Analysis (variables)	Multivariate Sensitivity Analysis (variables)	Probabilistic Sensitivity Analysis (variables)	Decision Aids Provided?
Blank30	Deterministic and stochastic	None	Trastuzumab price, HER2 test costs, local/regional and metastatic costs (all ± 30%); discount rate (0, 6%); HER2 prevalence (15, 25%)	None	All	Tornado diagram, scatterplot on cost-effectiveness plane, CEAC
Chen33	Deterministic	None	MBC, local/regional recurrence Y1 costs, and trastuzumab costs (± 20%) Probability of progression to MBC, local/regional recurrence (+50%) Discount rate for life-years (3%), costs (0%, 5%)	None	None	CEAC (unclear how CEAC was produced without stochastic parameters)
Dedes34	Deterministic	All conducted with HERA data only: Clinical benefit of trastuzumab limited to 3 years 20% rate of trastuzumab retreatment in MBC 80% rate of trastuzumab retreatment in MBC Trastuzumab administered in centers saving redundant drug Discount rate, 3%	HERA29 data: trastuzumab price, cost of MBC and local/regional recurrence, efficacy of trastuzumab (all ± 10%, ± 20%, ± 30%); discount rate FinHer28 data: risk of recurrence and MBC (5% and 95% CI)	None	None	No
Essers35	Deterministic and stochastic	None	Unclear, but univariate analyses conducted; model sensitive only to use of trastuzumab at MBC recurrence	None	Yes, although stochastic variables were unclear	CEAC
Garrison49	Deterministic	Payer perspective Societal perspective	Discount rate, trastuzumab, death and HER2 and cardiac diagnostic costs, recurrence rates, utilities, cost of cardiac toxicity, recurrence-associated mortality	All varied to upper or lower bound	None	No
Kurian38	Stochastic	None	Discount rate, trastuzumab, MBC, and cardiac toxicity costs; survival; cardiac toxicity recovery; utility of trastuzumab and cardiac toxicity; cardiac toxicity–associated mortality	None	All	No
Liberato39	Deterministic and stochastic	Assumed characteristics of HERA trial29	None	Conducted as scenario	All	CEAC
Lidgren38	Deterministic and stochastic	None	Costs of trastuzumab, recurrence and AEs, recurrence risks, mortality, and trastuzumab efficacy all varied ± 30%; future costs, duration of trastuzumab effect, HER2 prevalence also varied	IHC test sensitivity and specificity	All	CEAC
Millar41	Deterministic	Age 50 years given 52-week trastuzumab across 95% CI Age 70 years given 52-week trastuzumab across 95% CI	Discount rate, age at diagnosis, cost of treating MBC, relapse, other diseases and cardiac toxicity, duration of trastuzumab effect, risk of distant recurrence after 52-week trastuzumab across 95% CI bounds, risk of distant recurrence after 9-week trastuzumab across 95% CI bounds	Conducted within scenarios	None	No
Neyt43	Stochastic	Subgroup analyses according to age group, stage, and discount rate	None	Conducted as scenario	All but HER2 test, administration, and follow-up costs	Tornado diagram
NICE17,44	Deterministic and stochastic	None	Utilities, discount rate, trastuzumab efficacy, costs, recurrence, and progression rates	None	All	CEAC
Norum42	Deterministic	None	All parameters ± 25%; discount rate	None	None	No
Shiroiwa45	Stochastic	None	Discount rate, recurrence rate, cardiotoxicity costs, palliative care costs	None	All	CEAC
Skedgel27	Stochastic	Assumed 3-year trastuzumab benefit Assumed 5-year trastuzumab benefit	Discount rate, cost of palliative treatment; threshold analyses to determine what parameters could produce ICER estimate below CA$50,000/QALY and CA$100,000/QALY thresholds	None	All	CEAC
Van Vlaenderen47	Stochastic	None	None	None	All	Scatterplot with willingness-to-pay threshold

NOTE. Methods of sensitivity analysis, parameters analyzed, and decision aids, if any, were abstracted.

Abbreviations: AEs, adverse events; CEAC, cost-effectiveness acceptability curve; FinHer, Finland Herceptin; HER2, human epidermal growth factor receptor 2; HERA, Herceptin Adjuvant; ICER, incremental cost-effectiveness ratio; IHC, immunohistochemistry; MBC, metastatic breast cancer; NICE, National Institute for Health and Clinical Excellence; QALY, quality-adjusted life-year.

Discussion

Our findings point to several avenues along which economic analyses of trastuzumab targeted therapy can be improved for decision-making purposes. We noted that local data were rarely incorporated to inform parameters beyond the cost category, thus limiting outcome relevance to the decision maker. Although the inclusion of local costs was unanimous in this review, most authors derived resource use from clinical trial protocols or other published cost analyses. We are unable to comment on the treatment modalities with local practice patterns, given that many authors failed to comment on the similarity of the comparator treatment option to local standard care. Other parameters, such as treatment efficacy, present important challenges in terms of availability of local data and relevance, which often necessitate the use of preexisting trial data for economic evaluations.

However, as Phillips et al^50,51 point out, there are important and unanswered questions about the use of targeted therapies in the real world. Which patients get tested and treated? How accurate is HER2 testing in the clinical setting? What testing and treatment approaches are used to direct targeted HER2 therapy in actual clinical practice? These questions cannot be answered with hypothetical cohort simulations informed primarily by data from trials or early-stage applications. We recognize that not all economic analyses can be informed by local or pragmatic trial data, but the establishment of the Coverage With Evidence Development framework by the Centers for Medicare and Medicaid Services and the recent National Institutes of Health push for comparative effectiveness research⁵² attest to the growing need for this type of evidence in decision making.⁵³

Our work demonstrates that analyses of targeted therapy generally fall short of ensuring relevance through the use of local data. Locally derived, pragmatic evidence requires both the infrastructure and the time to collect long-term follow-up in the early-stage breast cancer setting given the significant improvement in mortality experienced as a result of early detection and adjuvant therapy.⁵⁴ However, Poncet et al⁵⁵ provide an example of how to use local, pragmatic evidence to inform health economic analyses in the metastatic setting. The difficulty in obtaining local clinical evidence is not exclusive to targeted therapy, but the importance of local relevance is heightened because of variability in real-world test performance and the high cost of many (biologic) targeted therapies. For example, a Canadian analysis of testing strategies⁵⁶ suggests that variation in national testing practice significantly impacted cost-effectiveness estimates.

Quantification of decision uncertainty and presentation of decision aids also fell short of supporting informed decision making. Exclusive use of univariate analysis is no longer sufficient for assessing parameter uncertainty⁵⁷ given that the ICER is affected by the shared uncertainty in multiple model inputs. Probabilistic sensitivity analysis is accepted as a much more powerful tool for addressing this. Such information is crucial to decision makers, who must make decisions for highly variable populations (eg, according to age or comorbidity) or for treatment settings that may differ from the studies used to inform the evaluation. Decision aids were provided in two thirds of adjuvant trastuzumab evaluations; Blank et al³⁰ provide an excellent example of the use of scatterplot, tornado, CEAC, and diagrams to represent decision uncertainty graphically. These graphic presentations of base case or sensitivity analysis results are promoted as a tool to communicate aspects of structural, parameter, or assumption-based sensitivity to the reader in a nontechnical manner. This makes the interpretation of the ICER and its variability more accessible to decision makers such as plan managers, who may not have technical expertise in economic evaluation. The use of graphic decision aids was unanimously supported by Canadian, United Kingdom, and US economic evaluation guidelines and is, in fact, promoted by several other jurisdictions.^58–60 CEACs are also a natural extension of probabilistic sensitivity analysis; it was therefore surprising to observe that CEACs were not provided in all studies conducting probabilistic analysis. Conversely, VOI is much more complex to conduct, and the absence of this decision aid was expected.

We recognize that there are some limitations to this review. Several methodologic aspects of cost-effectiveness analysis are challenging to incorporate in publications given current word limits. For example, model calibration was rarely reported in review of cancer simulation models, despite the importance of this method in ensuring that a model accurately predicts outcomes for the population of interest.⁶¹ Calibration was also recommended by Canadian economic evaluation guidelines as a method of ensuring relevance to the decision maker.¹¹ We encourage authors and journals to provide this information via online support materials whenever possible.

The exclusive assessment of trastuzumab as a targeted therapy in this review may limit the generalizability of our findings. We believe that the trends observed herein are not restricted to trastuzumab and that many relate directly to economic evaluations of other targeted therapies, particularly those with prerequisite diagnostic tests. We also acknowledge that some of the citations included in this review may not be intended to inform specific local decisions. Indeed, hypothetical cohort evaluations are designed to predict outcomes in a theoretical population. Conversely, readers of the scientific literature, including physicians and formulary managers, require a more realistic understanding of the potential impacts of treatment or funding decisions. Many review agencies, such as the Canadian Association for Drugs and Technologies in Health and NICE, require a systematic review of economic evaluations within drug reimbursement submissions. This suggests that hypothetical cohort evaluations do factor into reimbursement decisions.

Additionally, the use of Academy of Managed Care Pharmacy (United States), the Canadian Association for Drugs and Technologies in Health (Canada), and NICE (United Kingdom) guidelines as benchmarks for economic evaluation methods is not validated. However, the recommendations included in this review for data sources and decision uncertainty are consistent with Danish,⁵⁸ French,⁵⁹ and German⁶⁰ guidelines and the final recommendations of the International Society for Pharmacoeconomics and Outcomes Research Real World Data⁴⁹ and Good Research Practices–Modeling Studies⁶² task forces. The Canadian, United Kingdom, and US guidelines were applied to strike a balance between current methods and highly regarded guidelines in the absence of empirically validated economic evaluation methods for decision support.

Our findings suggest that economic evaluations of targeted therapy can be improved to support high-quality, informed decision making. Although real-world effectiveness estimates are often unavailable or difficult to generate, several other steps can be taken to ensure relevance to the decision maker's setting, including the incorporation of local utilization patterns to improve costing and behavioral assumptions. Quantification and representation of decision uncertainty can also be improved through the regular conducting of probabilistic sensitivity analysis, the provision of decision aids, and the practical application of VOI methods.

Authors' Disclosures of Potential Conflicts of Interest

The authors indicated no potential conflicts of interest.

Author Contributions

Conception and design: Ilia L. Ferrusi, Natasha B. Leighl, Deborah A. Marshall

Financial support: Deborah A. Marshall

Administrative support: Deborah A. Marshall

Collection and assembly of data: Ilia L. Ferrusi

Data analysis and interpretation: Ilia L. Ferrusi, Natasha B. Leighl, Nathalie A. Kulin, Deborah A. Marshall

Manuscript writing: Ilia L. Ferrusi, Natasha B. Leighl, Deborah A. Marshall

Final approval of manuscript: Ilia L. Ferrusi, Natasha B. Leighl, Nathalie A. Kulin, Deborah A. Marshall