Issues surrounding the health economic evaluation of genomic technologies

Abstract

Aim

Genomic interventions could enable improved disease stratification and individually tailored therapies. However, they have had a limited impact on clinical practice to date due to a lack of evidence, particularly economic evidence. This is partly because health economists are yet to reach consensus on whether existing methods are sufficient to evaluate genomic technologies. As different approaches may produce conflicting adoption decisions, clarification is urgently required. This article summarizes the methodological issues associated with conducting economic evaluations of genomic interventions.

Materials & methods

A structured literature review was conducted to identify references that considered the methodological challenges faced when conducting economic evaluations of genomic interventions.

Results

Methodological challenges related to the analytical approach included the choice of comparator, perspective and timeframe. Challenges in costing centered around the need to collect a broad range of costs, frequently, in a data-limited environment. Measuring outcomes is problematic as standard measures have limited applicability, however, alternative metrics (e.g., personal utility) are underdeveloped and alternative approaches (e.g., cost–benefit analysis) underused. Effectiveness data quality is weak and challenging to incorporate into standard economic analyses, while little is known about patient and clinician behavior in this context. Comprehensive value of information analyses are likely to be helpful.

Conclusion

Economic evaluations of genomic technologies present a particular challenge for health economists. New methods may be required to resolve these issues, but the evidence to justify alternative approaches is yet to be produced. This should be the focus of future work in this field.

The quantity and range of genetic tests available to patients has increased significantly over the last 20 years, and genetic testing is now an established practice in many disease areas (e.g., BRCA1/2 testing in breast cancer [101]). However, the most commonly used tests are not sufficiently reliable or lack the resolution to provide detailed information on multiple genetic changes across the whole genome, which could enable improved disease stratification and individually tailored therapies [1]. Attention is now turning towards genomic interventions with the potential to meet these objectives, such as next-generation sequencing technologies (whole-exome and whole-genome sequencing). These technologies offer genome-wide testing capability, simultaneously scrutinizing all of an individual’s genes and their inter-relationships in order to identify their combined influence, and have a broad-ranging application in medicine, showing promise in allowing disease management to be stratified [2].

Despite some success stories, genomic technologies have had a limited impact on clinical practice to date. Only 1–2% of all the drugs listed by the US FDA in the USA contain pharmacogenomic information on their label [102], and genetic testing is strongly recommended in only four cases [3]. Internationally, adoption rates for genomic technologies vary considerably. Of insured people in the USA, 90% are members of an insurance plan that covers Oncotype Dx (a diagnostic test for breast cancer) [103], but the NICE has only recently approved the use of this test [104].

Differences in the uptake of these technologies have arisen, in part, due to a lack of clinical evidence (on test effectiveness) and ethical concerns regarding the treatment of incidental findings. However, translational research evidence is also scarce, particularly economic evidence [105]. Those economic evaluations that have been conducted have predominantly taken an extra-welfarist approach (e.g., cost–effectiveness analysis), aiming to maximize health outcomes, instead of a welfarist approach (e.g., cost–benefit analysis [CBA]), a broader form of analysis aiming to maximize patient welfare (including both health and nonhealth outcomes). However, health economists have begun to question whether the extra-welfarist approach is appropriate in genomics [4]. There is particular uncertainty surrounding the need to capture information on nonhealth outcomes (e.g., the value of information from test results to patients). Metrics, such as quality-adjusted life-years (QALYs), combine information on additional longevity and quality of life gained from an intervention and are the preferred outcome measure of a number of institutions (e.g., NICE), but such measures may not adequately capture these nonhealth outcomes. Other methodological challenges surround the analytical approach (e.g., should a health service or societal perspective be used?), the heterogeneity of cost data and how to incorporate complex effectiveness data into economic analyses.

Health economists are uncertain as to whether existing economic evaluation methods are sufficient to evaluate genomic technologies. Clarification is required since different methodological approaches may lead to different decisions regarding whether a particular technology should be adopted into routine clinical practice. More advanced (and expensive) genomic tests are starting to emerge (e.g., whole-genome sequencing), providing further challenges for existing methods. If agreement could be reached on the most appropriate methods of economic assessment in this context, health economists would be better placed to contribute to translational research in genomics [5].

This article presents a review of the health economic evaluation literature in genomics and summarizes the methodological issues associated with conducting economic evaluations of genomic interventions. We identify issues where there is consensus over potential solutions and those where there is uncertainty and further research is required, providing guidance for health economists conducting economic evaluations of genomic interventions in the future.

Materials & methods

The aim of the review was to identify references that considered the methodological challenges faced when conducting economic evaluations of genomic interventions (e.g., diagnostic/prognostic tests), and to ascertain whether there was consensus concerning how to resolve these issues. As the terms ‘genetic’ and ‘genomic’ testing are often used interchangeably in the literature, our searches included both [106,107].

Our review included references that fell into one of three categories:

• ▪ Methodological health economics references considering the challenges associated with conducting economic evaluations of genetic or genomic technologies;

• ▪ Applied health economics references that highlight methodological issues when reporting the results of economic evaluations of genetic or genomic technologies;

• ▪ References that consider the translation of genetic or genomic technologies into clinical practice, describing methodological issues that could arise from a health economics perspective.

We anticipated that a number of non health economics references in category three would meet our inclusion criteria. Therefore we designed our review to capture these references by combining multiple keyword variations using the logical operator ‘OR’, maximizing sensitivity and minimizing false negatives. Literature searches were undertaken in Embase, Medline, Econlit, the National Health Service Economic Evaluation Database, UK National Health Service Health Technology Assessments, and the Cochrane Library (Supplementary Material 1 provides search syntax details; see online at www.futuremedicine.com/doi/suppl/10.2217/pgs.13.183). Keywords were identified by examining relevant references in the literature, previous related literature reviews, and the medical subject headings (MeSH) used by Embase and Medline [108]. Searches were limited to full length references published in the English language (to enable in-depth reviews to be carried out), references published from 1990 onwards (to ensure that the results were applicable to the present setting) and studies in humans.

The literature searches were undertaken in April 2013. References that met the inclusion criteria were read by Buchanan, who extracted relevant material and entered this into a proforma under one of five methodological categories: analytical approach, costs and resource use, measuring outcomes, measuring effectiveness and other.

Results

Table 1 summarizes the literature search results. A total of 2772 references were initially identified. Of the 1868 titles remaining after removing duplicates, 30 met our inclusion criteria. References that evaluated preimplantation and prenatal genetic and genomic interventions for family planning purposes were not included: these interventions present a number of very specific challenges for health economists (e.g., should the projected economic benefits from the termination of affected fetuses be included in economic evaluations?), which have been discussed in detail elsewhere [6,7]. Twenty two references were added from other sources (e.g., earlier literature reviews), so 52 references in total were included in the review (see Supplementary Material 2 for details). These references included peer-reviewed journal articles, research papers and reports. The majority (50) were published since 2000 (and 39 since 2008).

Table 1. Literature search results: papers meeting inclusion criteria, by database.

Database	References identified (n)	After removing duplicates (n)	After reviewing titles (n)	After reviewing abstracts (n)	Papers meeting inclusion criteria (n)
Embase	1478	1271	305	106	22
Medline	1002	318	50	14	3
Econlit	49	46	4	2	2
Cochrane Library	228	221	94	65	3
CRD databases	15	12	6	6	0
Other†	–	–	–	–	22
References retained	2772	1868	459	193	52

^†Includes articles identified by reviewing the references of included studies, searching the publication records of key authors in this field and articles that were already known to the authors.

CRD: Centre for Reviews and Dissemination.

The literature search identified articles that discussed methodological issues in all five of the aforementioned categories. These issues are described below and are summarized in Table 2, which also describes some possible solutions to these issues.

Table 2. Summary of methodological issues arising in health economic evaluations of genomic technologies.

Category	Issues	Description of issues	Possible solutions
Analytical approach	1	The correct perspective to use remains uncertain	Future studies should consider multiple analytical perspectives. As costs and effects are likely to be incurred over long time horizons, discount rates should be carefully selected. Analyses should be undertaken in an iterative manner and rerun when new evidence becomes available: this may require extensions in health economic modeling capacity. Study designs must take into account all potential comparators (both genomic and nongenomic) and all subsequent therapeutic decisions. In general, realistic simulations and comprehensive parameter and structural sensitivity analyses should be standard practice
	2	The appropriate analytical timeframe can be many years in length. Studies that focus on the short-term costs and consequences of genomic interventions risk misestimating cumulative costs
	3	The timing of an economic evaluation of a genomic test is critical as standard testing practice evolves continuously
	4	The choice of comparator and the specification of the study design can significantly impact on economic evaluation results
Costs and resource use	1	A large number of cost categories are potentially relevant when conducting an economic evaluation in this context, including the costs of patient recruitment, sample collection and testing, data analysis, communication of test results, and actions taken based on test results	A much broader range of costs must be included in genomic economic evaluations, with these costs collected at more frequent intervals. Analyses must also be increasingly flexible in order to account for temporal and geographical price variations. Costing studies of platform diagnostics with multiple applications should be conducted particularly carefully
	2	There are no national pricing tariffs for genomic tests. Costs vary considerably both within and between countries
	3	The combination of unstable tumor genomes and evolving test filters may require genomic tests to be conducted more frequently
Measuring outcomes	1	Disease-specific and preference-based outcome measures limit comparability and do not capture all relevant dimensions of outcomes	Studies that evaluate outcome measures in this context, which are not disease-specific or preference-based, would be useful. In particular, methods that can be used to value outcomes from genomic interventions within a cost–benefit analysis framework would be particularly valued. As individual outcomes may be more important than population outcomes, subgroup analysis is likely to be crucial
	2	Capturing information on personal utility may be important, but the metrics for measuring personal utility are not well established
	3	Cost–benefit analyses may have a greater role to play in this context than other forms of economic evaluation
	4	Individual outcomes may be more important than population outcomes
Measuring effectiveness	1	Little is known about how patients and clinicians behave when faced with information provided by genomic interventions	Behavioral uncertainty should be incorporated into analyses. The individualistic nature of treatment should also feed through to study design. There is a greater need for post-implementation economic analyses and evaluations, based on pragmatic trials. Finally, studies that consider the use of summary measures, such as polygenic risk scores, within economic evaluations are likely to be valued, as well as studies that better link genomic data with health outcome measures
	2	The quality of effectiveness data is weak
	3	Effectiveness data for genomic interventions, when available, are complex and challenging to incorporate into standard health economic analyses
Other	1	Performance standards for genomic interventions vary considerably between laboratories	Analyses should consider all possible models of service delivery
	2	The pace of innovation in genomics suggests that prioritizing investment in expensive yet informative comparative studies is desirable	Value of information analysis is an important component of economic evaluations of genomic technologies

Analytical approach

Prior to starting any economic evaluation, health economists must make several decisions concerning the appropriate analysis to undertake. A number of methodological issues arise when making these decisions in the context of a genomic intervention.

Perspective

The correct perspective to use in economic evaluations of genomic interventions remains uncertain. Most economic evaluations of nongenomic interventions adopt the perspective of the health service/system, considering the direct impact of an intervention on the health service within which it is used. This approach is usually recommended by decision-makers (e.g., NICE). This is in contrast to a societal perspective, which would also consider indirect impacts, such as the financial costs incurred by families as a result of an intervention. Such a wider perspective may be beneficial in genomics as testing can affect both healthcare and life decisions (e.g., regarding family planning or schooling), but only 50% of all genomic economic evaluations currently adopt this perspective [8,9].

Timeframe of analysis

The information provided by genomic interventions can have long-term implications that are not necessarily observed when nongenomic interventions are evaluated, for example, identifying the presence of mutations linked to specific diseases (e.g., Lynch syndrome) in asymptomatic patients [10]. Studies that focus on the short-term costs and consequences of genomic interventions risk misestimating cumulative costs and effects over longer timeframes [4,6,7,11–13]. Health economists handle costs and effects incurred over long time horizons by applying a discount rate to convert these values to their present worth. Therefore, the choice of discount rate in economic evaluations of genomic technologies is especially important [14,106].

Timing of analysis

Genomic testing practice changes continually [8,15–17,109]. Test sensitivity and specificity can improve over time with experience, and costs, effectiveness and outcomes are particularly poorly defined for newer interventions [18,106]. The cost per megabase of DNA sequence has fallen from approximately US$10,000 in 2001 to below US$0.10 in 2013, for example [110]. Patient categorization can also vary over time: as tests evolve, additional patient subgroups can be identified with implications for individualized treatment (e.g., Type 2 diabetes risk assessment using genotyping) [14,19–21]. Furthermore, many genomic tests (such as the use of microarrays for the genetic analysis of children in the UK) gradually emerge into clinical practice after previously being offered on a research basis (instead of formally entering the healthcare system formally via regulators) [4]. This makes it difficult to compare economic evaluations conducted at different points in time [8,15,17]. Despite these challenges, evidence from economic evaluations is valued by decision-makers at all stages in the translational process, particularly in the early stages of product development when decisions about prioritization of research funding are being made [11].

Analytical context

Genomic tests often have multiple applications in varied contexts and it is rarely possible to synthesize results from economic evaluations undertaken in these differing contexts [13,22]. Furthermore, different applications of a specific genomic technology may not be similarly cost-effective: this depends on the strength of the link between the genomic data and clinical outcomes (e.g., the cost–effectiveness of Oncotype DX varies depending on whether it is being applied in breast or colon cancer) [23,24]. As the performance of a genomic technology can only generally be evaluated in the context of a specific clinical scenario and target population, study design must be carefully considered at the outset of such economic evaluations, with particular attention paid to two issues.

First, standard genomic testing practice rarely exists; genetic test usage varies considerably both between and within different clinical specialties in the UK [25]. It is therefore important to carefully choose the comparators in an economic evaluation of a genomic intervention. It may be appropriate to assess both genomic testing against nongenetic testing, and also genomic testing in addition to nongenetic testing [26,27]: expensive yet specific genomic tests could be more cost-effective if combined with cheaper less-specific screening tests [17,22,28]. For example, testing strategies that utilize preliminary tests, such as immunohistochemistry, before gene sequencing are significantly more cost-effective than strategies that solely utilize sequencing to identify Lynch syndrome among newly diagnosed patients with colorectal cancer [29].

Second, study designs should take into account subsequent therapeutic decisions in determining the clinical impact of a test [15]. For example, genetic testing for Duchenne muscular dystrophy can detect mutations across 79 genetic regions, with 30 different therapies required, each targeted at specific faulty regions. In this scenario, it is insufficient to conduct an economic evaluation of just one test design: results will vary considerably depending on which putative mutations are considered, the prevalence of each mutation and the effectiveness of different therapeutic options [30]. Multiple potential test designs may exist and analysts must carefully consider which designs are appropriate comparators prior to undertaking any health economic analyses.

Costs & resource use

All economic evaluations require health economists to collect information on the costs of an intervention. In genomics, there is uncertainty concerning which costs should be collected and when, and how costs vary between laboratories and countries.

Which costs should be included?

A broad range of costs must be considered in an economic evaluation of a genomic technology (Supplementary Material 3 summarizes potentially relevant cost categories). Several of these cost categories are also important in economic evaluations of nongenomic technologies. However, the wide range of potential costs in genomic economic evaluations requires health economists to carefully plan data collection to ensure that all relevant costs are included.

Genomic economic evaluations should incorporate data on the costs of patient recruitment (e.g., publicity and patient education) [6,8], sample collection, laboratory testing [8], data analysis (e.g., informatics solutions, and data libraries) [31,106] and the reporting of test results to patients [8]. The costs associated with actions taken on the basis of testing results must be estimated, including the costs of treatment and follow-up testing (or treatment and tests avoided), management of adverse drug reactions and genetic counseling [6,8,32–35]. These costs can be significant. A study comparing microarray testing with karyotyping in idiopathic learning disability found that including all of the costs of follow-up testing after karyotyping changed the study conclusions in favor of microarray testing (the more expensive test on a standalone basis) [36].

The time and cost burden of monitoring disease progression and drug response may also be important. If monitoring is cheap and straightforward (e.g., CYP2C9 mutation testing to direct warfarin therapy [35]), genomic tests to direct treatment may not be cost effective. The most cost-effective genomic tests are likely to be those for conditions where monitoring is difficult and expensive [35].

Depending on the analytical perspective chosen, the indirect costs accruing to patients may then need to be considered, for example, time spent seeking treatment or time lost from work due to poor health (‘morbidity costs’ or ‘productivity costs’). For example, if genomic technologies can identify cancer patients who will not benefit from chemotherapy, these patients may be spared the morbidity costs associated with inappropriate treatment, reducing work absenteeism. However, few studies have considered these costs in practice. An exception is a French study that evaluated the cost–effectiveness of the use of Oncotype DX in breast cancer, which found that cost savings were three-times higher if productivity costs were considered [37].

Finally, it may be appropriate to account for the wider infrastructure costs associated with genomic technologies (e.g., staff training costs). However this can be difficult to achieve in practice, as genomic platform diagnostics (such as sequencers) often have multiple applications [106]: a single piece of equipment may initially provide one specific genomic test, but will ultimately develop into a ‘one-stop shop’ for genomic testing in a variety of clinical areas [109]. In this scenario, the appropriate allocation of infrastructure costs is not clear: initial applications of a genomic technology may be burdened by a disproportionate share of overall infrastructure costs, while later applications benefit from economies of scale realized through more intensive use of the technology. These distortions may bias economic evaluation results.

How much do genomic interventions cost?

National guidelines and agreed reimbursement rates (e.g., NHS tariffs or Medicare payment rates) for genomic technologies rarely exist: either the technologies are too new for reimbursement rates to have been defined or they form part of a much bigger cost for an entire care episode and, hence, costs can vary considerably between testing laboratories [4,5,34,38,39,106]. Cost differences between tests developed by accredited clinical laboratories and tests for which commercially marketed kits exist can be particularly acute [9,40]. Testing costs can also vary between countries due to local price differences, variations in testing practice and differences in patient contributions to test costs [41]. These differences make it difficult for health economists to conduct economic evaluations of genomic interventions: many jurisdictions (e.g., NICE) require that nationally agreed prices are used within their technology-appraisal process. This is likely to become increasingly problematic over time as more genomic tests consider multiple genetic changes in a single assay, raising questions regarding the generalizability of results between countries. In addition, a system of flexible pricing based on value (for both drugs and diagnostics) has been proposed for introduction in the UK in 2014. Future genomic diagnostic costing studies will have to be increasingly flexible in order to account for these changing prices [109].

When should cost data be collected?

The frequency of cost data collection depends on the type of genomic intervention being evaluated. Tests that measure germline genetic changes may only need to be performed once [40]. By contrast, multiple genomic tests may be required to identify acquired changes in tumor genomes that are not stable over time (for example, in chronic lymphocytic leukemia [42]). Furthermore, genomic tests are often packaged with an informatics solution that filters data to permit more straightforward interpretation of results. These filters are updated over time to incorporate new research, which means that results may need to be retrieved and re-analyzed multiple times, incurring additional costs [6,43]. As whole-genome sequencing becomes more widespread, it may become cheaper and easier to sequence a patient’s entire genome multiple times, rather than retrieve and re-analyze data for new indications, presenting an additional cost complication.

Measuring outcomes

All full economic evaluations combine information on the costs and outcomes of an intervention. Several methodological issues arise when measuring the outcomes of a genomic intervention, including the type of outcome measure to be used, whether to include information on personal utility, the validity of the CBA approach and the importance of individual outcomes.

Disease-specific & preference-based outcome measures

Disease-specific outcome measures (e.g., European Organization for Research and Treatment of Cancer quality-of-life questionnaires) are designed to collect data on outcomes for specific disorders. These measures are occasionally informative in the context of genomics [32], but in general, they limit comparability and do not capture all relevant dimensions of outcomes. This is a particular problem for genomic interventions that only provide diagnostic information, which can reduce anxiety and help patients to make future plans.

Preference-based outcome measures (e.g., the EuroQol five-dimensional [EQ-5D] questionnaire) improve comparability by collecting data across a broad range of health-related quality-of-life domains. However, QALYs measured using these generic metrics do not capture nonhealth outcomes (e.g., the value to a patient of possessing genomic information) [6,8,12,17,20,32,33,44–48,106] or family spillover effects [46]. Furthermore, these measures cannot reflect all possible health states for genomic interventions (e.g., living with the knowledge that you have a mutation) [28]. As some genomic interventions do not improve health or extend life expectancy (e.g., microarray testing in learning disability), differences in preference-based outcome measures in economic evaluations can tend towards zero, pushing incremental cost–effectiveness ratios (the ratio of cost differences to effects differences) towards infinity [47].

Personal utility

Clinical utility often captures the main benefits of a nongenetic intervention [6], but personal genomic information can lack such utility [49]. However, it may offer other benefits (and harms) that could affect a patient’s wellbeing [4,20,31,50,51]. For some individuals, these effects are more profound than the clinical effects of testing [8]. These effects are captured by the term ‘personal utility’ (those benefits or harms that are primarily manifested outside medical contexts) [48,50].

Factors that can have a positive effect on personal utility include the acquisition of prognostic or diagnostic information that improves patient understanding and widens therapeutic choice (improved certainty of knowing) [52,109]. This information can enhance individuals’ sense of control, improving personal accountability for health-related choices, informing a sense of self-identity and autonomy, and potentially improving uptake and adherence [6,50,109]. Genomic information is also valued for the reassurance and anxiety relief it can provide [11,22,53,54], allowing patients and relatives to plan and make lifestyle modifications (e.g., APOE testing in Alzheimer’s disease) [11,22,49,55], and reducing the frequency of diagnostic odysseys (e.g., idiopathic learning disability [36]). Finally, genomic interventions can provide healthcare process-related benefits (e.g., improvements in waiting time) that are not captured by measures that focus on clinical utility [51].

Genomic interventions can also have a negative impact on personal utility. Anxiety may be increased if a test indicates that a patient will not respond to conventional treatment (but no alternative therapies exist) [17,56], or if false positives, mutations of unknown significance or other incidental findings are reported [6,44,49,50]. Diagnosing a patient with Huntington’s disease does not necessarily provide them with any clinical utility, but may considerably worsen their personal utility. Genomic test results can also disrupt family dynamics and reproductive decisions [6,44,50], and lead to fear of discrimination and stigmatization [13,46]. Negative genetic test results can lead to complacency, encouraging unhealthy behaviors [46].

Although personal utility is important in this context, it is difficult to incorporate into policy decisions – the metrics for measuring personal utility are not well established [15,22,55]. Work is currently underway in the UK to consider how the current definition of value used by NICE could be expanded to consider personal utility and other factors [109], but these efforts have not been replicated in other health technology assessment (HTA) agencies, such as the Pharmaceutical Benefits Advisory Committee (Australia), the Canadian Agency for Drugs and Technologies in Health, the Belgian Health Care Knowledge Centre, College voor zorgverzekeringen (The Netherlands), the Institute for Quality and Efficiency in Healthcare (Germany) and the Pharmaceutical Management Agency (New Zealand). Some studies have attempted to develop outcome measures based on particular concepts of personal utility (e.g., empowerment [49]), but these approaches have a number of limitations [19].

Cost–benefit analysis

Given the challenges associated with frequently used outcome measures and the desire to incorporate information on personal utility within economic evaluations of genomic tests,CBAs may have a greater role to play in this context. CBAs express both the costs and effects of an intervention in monetary terms, evaluating whether social welfare (including both health and nonhealth outcomes) is maximized subject to social budget constraints [8,46]. Several of the instruments that can be used within CBAs to value the effects of an intervention in monetary terms may be well equipped to capture the nonhealth outcomes of genomic testing [6,8,13,20,32,44,46,49,106,109]. The discrete choice experiment (DCE) may be particularly appropriate. DCEs present participants with choices between hypothetical scenarios comprising different attributes and levels for these attributes, relevant to the decision they are making. Respondents reveal their preferences for different combinations of attributes and levels via their choices, and these valuations can feed directly into a CBA, providing information on whether patients value genomic testing [109].

However, DCEs have rarely been used in genomics or to inform CBAs, possibly because people struggle to make hypothetical choices in unfamiliar contexts [8,46,55]. Alternative approaches to CBAs, such as cost–consequence or comparative–effectiveness analyses may also be informative in this context [106], but these have also been applied infrequently. Most HTA agencies (e.g., Canadian Agency for Drugs and Technologies in Health, Pharmaceutical Benefits Advisory Committee, College Voor Zorgverzekeringen and NICE) favor the use of cost–utility analysis to appraise healthcare interventions (including pharmacogenomic tests), with health outcomes measured using a metric, such as QALYs. CBAs are occasionally permitted as supplementary analyses (e.g., Canadian Agency for Drugs and Technologies in Health; and Pharmaceutical Benefits Advisory Committee) but rarely influence HTA decisions in general. One exception is the evaluation of public health programs by NICE in the UK: the use of CBAs is permitted in this context if it has been judged that cost–utility and cost–effectiveness analysis are not suitable [111].

Individual versus population outcomes

There is likely to be significant variation between patients in terms of the value that they place on a genomic intervention, and also how they respond to treatment that may be based on information provided by a genomic intervention [17,50,53,57]. Different patient subgroups may have considerably different opinions on acceptable levels of test sensitivity and specificity, for example [58]. In addition, individual differences in risk perceptions and attitudes are likely, given the complexity of genomic testing and the implications of genomic test information for patients [17]. For example, one study found that the value placed by patients on genetic testing for colorectal cancer varied significantly according to gender, income and education, among other factors [59]. Adopting a narrow ‘population’ approach in light of these issues will yield information on which genomic testing strategy is likely to be cost effective on average, but ignores the fact that such a cost-effective technology is likely to be extremely cost-effective for some patient subgroups, and extremely cost-ineffective for other patient subgroups.

Converting the evidence base from individual tests into an understanding of the value of obtaining the same information in aggregate is, however, difficult [31]. Prescreening activities (e.g., counseling) may be able to identify individuals who are unlikely to value genomic tests or respond to treatment, and should be incorporated into economic evaluations where necessary [50], informing subgroup analysis [26].

Measuring effectiveness

Economic evaluations also incorporate some measure of the effectiveness of the intervention being evaluated. Unpredictable patient and clinician behavior, poor effectiveness data quality and the increasing complexity of genomic effectiveness data make the measurement of effectiveness challenging in this context.

Patient & clinician behavior

The cost–effectiveness of genomic interventions may depend on whether patients and clinicians use these interventions and how they respond to test results (e.g., whether patients undergo appropriate treatment). Ideally, information on behavioral probabilities should feed into economic evaluations of genomic tests [6,11,17,53,54], but little data on behavior is actually available [17]. Those studies that do exist are unclear on whether all patients will want to be tested [60], whether patients will adjust their behavior to fully adhere to genomic testing advice [12,13,38,39] and whether physicians will take information from genomic tests into account when making clinical decisions [17]. Evidence from the USA suggests that if testing is not mandated, universal uptake is unlikely [34].

Effectiveness data quality

The quality of effectiveness data for genomic interventions is often weak [4,15,20]. Many genetic conditions are uncommon, and large randomized controlled trials (RCTs) would be required, run over long time periods, at great expense, to generate sufficient power to detect clinically significant end points. Given the fast pace of genomic technology evolution, such trials may not even provide relevant results [6,12,14,17,22,43,57, 61, 62,106,109]. Data on the effectiveness of laboratory-developed tests is often even more limited due to the ad hoc nature of their development [33,39]. Compounding these issues, tests used in routine clinical practice do not often perform as well as in research programmes [27], and test performance can vary considerably depending on laboratory characteristics (e.g., size [34] and choice of equipment [16]).

Noninferiority trials may be an alternative source of effectiveness data. However, genomic interventions could supplement, rather than replace, existing practice, and noninferiority trials may need to be larger than RCTs designed to show superiority [56]. The personalized nature of genomic interventions means that, to capture their true effects, studies must focus on patients who are outliers [41]. Therefore a more individualistic approach to the measurement of effectiveness may be required (similar to N-of-1 or single-subject clinical trials) [3,106], although it is not easy to conduct prospective well-controlled studies of these patients [41].

In this context, regulators are increasingly open to the use of effectiveness data from alternative sources, including disease registries, observational and cohort studies, data generated in community settings, and expert opinion estimates of efficacy [17,33,38,51,109]. However, these data may suffer from the problem of inadequate numbers [22,27], the fact that such studies cannot account for recent advances in treatment [12], and expert opinion estimates may be biased, and hence thorough sensitivity analyses are required [33]. Practice-based evidence (generated via comprehensive post-implementation economic analyses and evaluations) could instead provide the robust effectiveness data that cannot be generated via RCTs [38,54,56,61,63,109]. However, such evidence may only have a limited impact on coverage decisions, as regulators are often unwilling or unable to sanction disinvestment in technologies that are no longer considered to be sufficiently effective. Bayesian (or adaptive) trial designs may be able to evaluate fast-evolving genomic technologies, but regulators are unconvinced by these new methods [3,15]. For genomic interventions already in clinical practice, large administrative healthcare databases could be used to access routinely collected data, linking this to genomic data to provide evidence for the clinical utility of testing (e.g., Biobank data). Such databases provide information on both infrequent events and routine care [15,61,63,109], but detailed information on specific genomic interventions may be limited.

It is becoming increasingly apparent that the methods that are currently used to generate effectiveness evidence for economic analyses are not necessarily applicable in a genomic context, given the pace of genomic discovery and the costs associated with applying these methods [31,109]. ‘Wait and see’ policy conclusions may become increasingly common in economic evaluations of genomic technologies as decisions are delayed until better evidence emerges [106].

Data complexity

The information provided by genomic technologies is challenging to incorporate into economic evaluations: the evidence base linking complex genomic data to health outcome measures is generally underdeveloped [5,31,47]. For example, there are over 130 documented variations of the CYP2D6 gene (which can indicate tamoxifen resistance), but it is unclear which mutations are most clinically relevant [41]. More generally, genomic testing outcomes are influenced by multiple genes, each genetic variant can influence multiple outcomes, and the influence of a variant on a given outcome can vary across individuals [17]. Association studies performed to detect these variants are commonly underpowered, so the probability of detecting a true signal is very small [64,109]. Consequently, associations between genotypes detected by genomic tests and phenotypes related with disease are frequently weak and overestimated [6].

Further methodological issues that arise when analyzing effectiveness data for genomic interventions include ‘multiple testing’ (how to control the false-positive rate) and the ‘winner’s curse’ (the likelihood that the effect of a newly identified mutation is overestimated in initial studies) [15]. It is also currently unclear how to incorporate information on diagnostic characteristics (e.g., receiver operating curves) into economic analyses. Compounding these methodological issues is the fact that the performance of commercially developed and ‘in-house’ genomic interventions may differ considerably [33,51].

One way in which information on clinical utility can be extrapolated from complex genomic data is to apply multiple cutoff points to identify clinically relevant subgroups with similar genomic profiles [33]. Alternatively, polygenic risk scores can combine all available genotypic data in one measure (e.g., genomic complexity in chronic lymphocytic leukemia) [42], but may require large sample sizes before statistical significance is reached [64]. It is also unlikely that separate studies will converge on identical scoring tools, making it difficult for decision-makers to synthesize evidence from multiple studies [8].

Other

Service delivery

Performance standards for genomic interventions vary considerably between laboratories. The choice of service delivery model could, therefore, be an important driver of economic evaluation results and should be carefully considered by health economists. For some disorders (e.g., HER2 testing in breast cancer) local laboratories with lower testing volumes are more likely to report incorrect findings, suggesting that models of service delivery focused around specialist regional testing hubs may yield more favorable results in an economic evaluation [15,60,61,109].

Research prioritization

Value of information analysis is increasingly used to assist decision makers with research-prioritization decisions in healthcare contexts outside of genomics, and may be an important component of economic evaluations of genomic technologies [13,15,17,33,48,65]. Value of information techniques allow researchers to compare the expected benefits from further research with the expected costs of undertaking this research, and can highlight instances where new evidence would be particularly valued by decision-makers (e.g., uncertain model parameters that drive economic evaluation results). The pace of innovation in genomics suggests that prioritizing investment in expensive yet informative comparative studies is desirable [15]. Furthermore, if evidence of the effectiveness of a genomic intervention is weak, the costs and consequences of collecting additional data to better inform translational research must be carefully considered, particularly if robust methods of data collection, such as RCTs, are not available to analysts [12,66].

Only one study has applied these techniques in this context. This study estimated that population-based genetic screening for hemochromatosis was unlikely to be cost-effective in Germany, and the value of additional research that might overturn this decision was between €500,000–2,200,000. However, when policy-relevant research designs were evaluated, the value associated with these designs was small in comparison with the high cost of conducting these studies [65].

Discussion

Economic evaluations of genomic technologies pose a number of methodological challenges for health economists. The appropriate analytical perspective, timeframe (short-term versus long-term) and timing (when in the translational process to conduct the analysis) are all uncertain, and results can be significantly influenced by the analytical context. The range of costs that could be included is wide, and these costs vary substantially between laboratories. Tests may also need to be repeated an uncertain number of times over an undefined time period.

Commonly used outcome measures may not be appropriate as they may not capture all relevant dimensions of outcomes. Measures that focus on personal utility may be useful, but are not well established. Welfarist approaches to economic evaluation may have a greater role to play, but instruments such as DCEs have rarely been used in genomics or to inform CBAs. Although individual outcomes are important in genomics, most economic analyses focus on average population outcomes.

Little is known about how patients and clinicians behave when faced with information provided by genomic interventions, and this is rarely factored into economic evaluations of these technologies. Effectiveness data is commonly weaker than that existing for nongenetic interventions and large RCTs are rarely feasible. It can be tricky to incorporate complex genomic effectiveness data into economic analyses: the evidence base linking genomic data to health outcome measures is underdeveloped, and possible solutions (e.g., polygenic risk scoring) are still under development. Performance standards also vary considerably between laboratories.

Many of these methodological issues arise in economic evaluations of both genomic and nongenomic technologies. It is always important in economic evaluations to select an appropriate analytical perspective and timeframe, carefully specify the range of costs to be included and choose an appropriate outcome measure, for example. However, the quantity and range of challenges in genomics is such that economic evaluations of these technologies present a particular challenge for health economists: it is the accumulation of methodological challenges in this context that presents a particular problem for the analyst. Furthermore, it could be argued that resolving some of these issues is the responsibility of researchers working in disciplines other than health economics (e.g., developing methods to evaluate the clinical utility of genomic technologies given their rapid evolution over time). However, health economists must still be prepared to adapt their methods to account for these challenges when conducting economic evaluations in this context.

How can these issues be resolved?

It is not feasible for every health economic evaluation of a genomic technology to address all of these methodological challenges due to limitations in both funding and data. However, although the debate surrounding the appropriate methods to use in these economic evaluations is relatively young, a number of solutions to these challenges have already been proposed, which could be incorporated in future economic evaluations in this context (Table 2 summarizes these possible solutions). Some are straightforward (and are not necessarily specific to genomics): realistic simulations and comprehensive parameter and structural sensitivity analyses should be standard practice to allow health economists to fully consider the impact of different analytical perspectives, discount rates and test characteristics [4,8], and to permit analytical frameworks to be more readily applied in different HTA settings. In a genomic context, it may be necessary to widen these sensitivity analyses to consider multiple comparators and different combinations of testing and/or treatment. It is particularly important to evaluate the impact of patient and clinician behavior on economic evaluation results.

Given that health economists will increasingly be faced with both poor-quality effectiveness data and a requirement for more early-stage economic analyses in this context, it is also crucial that health economic modeling capacity is extended. Health economists should be prepared to work in a more iterative manner, rapidly incorporating new evidence into analyses and rerunning simulations regularly [8,14,17,19,20,106,109]. Early-stage prospective economic evaluations will become more common [33,34,47], and genomic effectiveness data will increasingly originate from studies that are not RCTs (e.g., practice-based studies). ‘Wait and see’ or ‘coverage with evidence development’ conclusions are likely to become more common in economic evaluations as a result. Value of information techniques will have an important role to play. Studies that could improve the evidence base for genomic technologies will be expensive, requiring careful prioritization of limited healthcare funds.

Incorporating complex genomic effectiveness data into economic analyses is a challenge that will grow as next-generation sequencing technologies enter clinical practice. Future economic evaluations in genomics will need to present results for multiple scenarios reflecting different cutoff points and thresholds for measures, such as polygenic risk scores. Consequently, studies linking genomic data with health outcome measures will be increasingly valued.

The exact range of costs included in an economic evaluation of a genomic technology will depend on the analytical perspective that is taken. However, health economists should be aware of the issues surrounding costing in this context, including how to cost platform diagnostics with multiple applications. Cost differences between laboratories can be incorporated into economic evaluations by conducting comprehensive sensitivity analyses, but care should be taken when studies span international borders: approaches that combine country-specific resource use and unit costs may be most appropriate here.

The area where there is the most disagreement among health economists is how to measure the health outcomes associated with genomic technologies. Some authors maintain that ‘new genetics’ does not pose new problems for health economics, but it highlights aspects of evaluation that have been neglected in previous economic evaluation research [44]. Others claim that ‘genetic exceptionalism’ exists and new methods are required to evaluate the outcomes arising from genomic technologies [6]. Several authors now advocate a welfarist approach in order to incorporate information on personal utility into economic evaluations [8,46].

Health economists are somewhat constrained in their approach to outcome measurement by the regulatory context within which economic evaluations of genomic technologies are conducted. In the UK, the Diagnostics Assessment Programme assesses diagnostic technologies within NICE’s remit, recognizing that the evaluation of these technologies differs from that of treatments. However, their current HTA methods closely follow those used to evaluate medicines, measuring patient benefits using QALYs [5]. In the USA, most genomic tests are reimbursed using a formulaic cost-based coding system, which leaves little room for regulators to consider factors such as personal utility [51]. Ultimately, incorporating personal utility into economic evaluations will largely depend on the scope of decision-makers. For regulators deciding which tests should be available, it may be reasonable to consider personal utility alongside other dimensions of utility. From a public healthcare provider perspective (with an overall aim of improving health), personal utility may not be relevant [11,46].

Given that decision-makers are yet to be convinced that welfarist approaches are valid in this context, future health economic studies may benefit from considering both welfarist and extra-welfarist approaches simultaneously (depending on the availability of both funding and data) [67]. Applied studies that use real-world data on genomic interventions to assess the impact of the economic evaluation approach on adoption decisions will make a particularly valuable contribution to this methodological debate.

Conclusion

New genomic diagnostic technologies are showing considerable promise in terms of yielding information to allow disease management to be stratified. However, despite some success stories, these technologies have had a limited impact on clinical practice to date. In part, this is due to a lack of evidence on the clinical utility of testing. A lack of high-quality translational research evidence is also a significant obstacle to the more widespread use of these tests. Health economists can contribute to this translational research, but the methodological issues surrounding economic evaluations in this context are wide-ranging. New methods may be required to resolve some of these challenges, but the evidence to justify alternative approaches is yet to be produced by health economists. These challenges should be the focus of future work in this field. Resolving these methodological issues will enable decision-makers to make better informed adoption decisions, which may accelerate the translation of these technologies into widespread clinical practice.

Future perspective

Future work in this area should focus on improving the quality of economic evaluations of genomic interventions by attempting to resolve those methodological challenges that are specific to economic evaluations in this context. These challenges include developing methods to incorporate genomic effectiveness data into economic evaluation frameworks, establishing appropriate methods to cost platform diagnostics with multiple applications and coming to some agreement on how best to measure the health outcomes associated with genomic technologies. In particular, health economic studies that consider both welfarist and extra-welfarist approaches simultaneously will be valuable. If progress is made in these areas over the next 5 years, health economists may be able to contribute to accelerating the translation of genomic technologies into widespread clinical practice.

Executive summary.

• ▪ Genomic interventions have the potential to enable improved disease stratification and individually tailored therapies, but have had a limited impact on clinical practice to date due to a lack of clinical and translational research evidence, particularly economic evidence.

• ▪ Health economists are yet to reach consensus on whether existing economic evaluation methods are sufficient to evaluate genomic technologies, and as different methodological approaches may produce conflicting adoption decisions, clarification is urgently required.

• ▪ The key methodological challenges fall into five categories: analytical approach, costs and resource use, measuring outcomes, measuring effectiveness and other.

• ▪ The challenges related to the analytical approach include the choice of an appropriate comparator, analytical perspective and timeframe, as well as the need to carefully time the economic evaluations of genomic tests.

• ▪ Challenges in costing are centered around the need to collect a much broader range of costs on a more frequent basis, in a data-limited environment.

• ▪ Measuring outcomes can be problematic as disease-specific and preference-based outcomes measures have limited applicability and are focused on population rather than individual outcomes, yet alternative metrics (e.g., personal utility) are underdeveloped and alternative approaches (e.g., cost–benefit analysis) are underused.

• ▪ The quality of effectiveness data is weak and challenging to incorporate into standard health economic analyses, while patient and clinician behavior in this context is unpredictable.

• ▪ It is crucial to consider all possible models of service delivery and undertake a comprehensive value of information analyses.

• ▪ Many of these methodological issues arise in economic evaluations of both genomic and nongenomic technologies; however, the quantity and range of challenges in genomics is such that economic evaluations of these technologies present a particular challenge for health economists.

• ▪ New methods may be required to resolve some of these challenges, but the evidence to justify alternative approaches is yet to be produced by health economists and should be the focus of future work in this field.