Economic evaluation of genomic sequencing in the paediatric population: a critical review

Abstract

Systematic evidence is critical to the formulation of national health policy to provide public funding for the integration of genomic sequencing into routine clinical care. The purpose of this review is to present systematic evidence on the economic evaluation of genomic sequencing conducted for paediatric patients in clinical care, and to identify any gaps in the methodology of economic evaluations. We undertook a critical review of the empirical evidence from economic evaluations of genomic sequencing among paediatric patients searching five electronic databases. Our inclusion criteria were limited to literature published in the English language between 2010 and 2017 in OECD countries. Articles that met our inclusion criteria were assessed using a recognised checklist for a well-designed economic evaluation. We found 11 full-text articles that met our inclusion criteria. Our analysis found that genomic sequencing markedly increased the diagnostic rate to 16–79%, but lowered the cost by 11–64% compared to the standard diagnostic pathway. Only five recent studies in paediatric clinical cohorts met most of the criteria for a well-designed economic evaluation and demonstrated cost-effectiveness of genomic sequencing in paediatric clinical cohorts of patients. Our review identified the need for improvement in the rigour of the methodologies used to provide robust evidence for the formulation of health policy on public funding to integrate genomic sequencing into routine clinical care. Nonetheless, there is emerging evidence of the cost-effectiveness of genomic sequencing over usual care for paediatric patients.

Introduction

Genomic sequencing technologies such as whole-genome sequencing (WGS), whole-exome sequencing (WES) and panel tests have emerged as an important alternative to the traditional standard diagnostic pathway with its long diagnostic odyssey and low diagnostic rates for the diagnosis of complex Mendelian diseases. A major technological advance is that genomic tests perform rapid sequencing of many genes simultaneously. Moreover, the increasing availability and affordability of genomic sequencing technologies means these are becoming an important potential diagnostic tool in clinical practice. Early diagnosis achieved by applying genomic sequencing optimises opportunities to target patient care and clinical management, and may also influence the cost of treatment [1]. To date, there is substantial evidence about the clinical utility of genomic sequencing [2]. However, evidence from economic evaluations of genomic sequencing in a clinical setting are also crucial to policy decision-making to incorporate and publicly fund new technologies such as genomic sequencing into routine patient care in Organization for Economic Co-operation and Development (OECD) countries. There is now an emerging literature on the economic implications of applying genomic sequencing in clinical settings [3–13].

To date there have been four reviews on genomic sequencing and related costs [14–17]. Frank et al. [16] published a systematic review on economic evaluation of sequencing of the human genome where they reported five full-text publications but found quite limited health economic evidence because of poor methodology and questionable reliability and validity of results. Douglas et al. [15] published a scoping review on economic evaluations detecting variants in 56 genes associated with 24 conditions among general and targeted and high-risk populations that are considered clinically actionable by the American College of Medical Genetics and Genomics. They found that none of the studies directly addressed the cost-effectiveness of WGS. Christensen et al. [14] published a commentary review on germline genomic sequencing as part of a randomised clinical trial to understand the costs and cost-effectiveness of integrating WGS into cardiology and primary care settings. They also found limited health economic evidence on genomic sequencing, but indicated increasing integration of genomic sequencing into medicine due to falling costs of sequencing. Recently, Schwarze et al. [17] published a systematic review and summarised the health economic evidence on WGS and WES. The review found that current health economic evidence to support the widespread use of WGS and WES in clinical practice is limited and therefore, emphasised on the need for more carefully conducted cost-effectiveness analyses to support clinical translation. None of these reviews focused on the economic evaluation of genomic sequencing among paediatric patients in clinical settings and their quality assessment in terms of methodological rigour.

To overcome the absence of systematic evidence of economic evaluations of genomic sequencing for paediatric patients, the objective of this paper is to summarise the existing evidence on economic evaluations of genomic sequencing for paediatric patients in clinical settings through a systematic review. Our second objective is to critically appraise the quality of methodologies applied to economic evaluations of genomic sequencing and to identify the gaps in the methods used for economic evaluations of genomic sequencing and where future research and methodological improvement is needed to provide the necessary evidence of efficient resource allocation in health systems.

Overview of economic evaluations

Economic evaluations are used to assess the efficiency of changes in health-related programmes. They assist decision-makers to make an informed choice between alternative programmes, particularly on how limited resources can be used to maximise health outputs. There are four types of economic evaluations:

• Cost minimisation analysis or cost analysis: When outcomes of two interventions are the same and the only difference is in the cost, then the decision between alternative interventions is made based on costs.

• Cost effectiveness analysis: When outcomes of two interventions vary, then a choice is made on the relative cost for alternative treatments for a given cost per additional life year or cost per additional diagnosis (for example). However, this method does not take account of quality of life.

• Cost utility analysis: Cost utility analysis is similar to cost-effectiveness analysis with the exception that it measures outcomes in terms of health-related utility instead of in natural units. Thus, this method does not just consider extension of life or additional diagnoses (for example) as an outcome, but rather additional quality of that life years (such as quality adjusted life years (QALYs)) gained.

• Cost benefit analysis: Cost benefit analysis considers how to maximise all benefits from available resources. It values costs and benefits in a single unit of measure in monetary form.

Materials and methods

To assess the empirical evidence available on economic evaluations of genomic sequencing among paediatric patients, we employed a comprehensive search strategy (Fig. 1) using five electronic databases: PubMed/Medline; EMBASE; EconLit; Science Direct; and Cochrane Library. In these databases we searched a combination of keywords: whole genome sequencing, whole exome sequencing, panel test, cost, cost-minimisation, cost-effectiveness, cost-utility, cost-benefit, child, infant, paediatric, and pediatric. Our inclusion criteria were limited to published literature in English language for paediatric patients in OECD countries. Economic evaluation of genomic sequencing is recent, with earlier literature reviews finding little in the way of evidence of economic evaluation prior to 2010. Thus, we set our inclusion criteria for the studies published during 2010–2017. Published articles that met our inclusion criteria were screened using their titles and abstracts to determine whether they contain any economic evaluation data. Duplicate citations, articles published before 2010 and articles that did not contain economic data were excluded. Full-text articles that were not on paediatric patients, and that did not use genomic sequencing were then excluded. Any ambiguity in the titles and abstracts were clarified through accessing and screening full-text articles, and the senior author took the final decision. We summarised the evidence of economic evaluation of genomic sequencing for paediatric patients in clinical settings from the selected papers. Finally, a Drummond’s checklist for assessing the criteria of economic evaluations was used to evaluate the included articles [18]. The 10-point checklist is mostly objective in nature and used without any modifications to objectively assess to what extent the included articles meet the standard criteria for an economic evaluation.

Fig. 1.

Selection process of economic evaluation of genomic sequencing among children

Results

The database searches identified 1338 references. Figure 1 summarises the search strategy. After careful screening of the titles and abstracts against our inclusion criteria we excluded abstracts with duplications (n = 102), older than 2010 (n = 101), containing no economic data (n = 1095) and containing economic data but no full papers (n = 3). Finally, we accessed and reviewed 37 full-text articles for eligibility. Of these, we found 11 full-text articles that met our inclusion criteria (Appendix Table 1). The other full-text articles were not included as they did not use genomic sequencing for the paediatric patient population.

All 11 studies were conducted in OECD countries and published between 2014 and 2017 [3–13]. Out of these studies, four studies were conducted in the United States, three in the Netherlands, three in Australia and one in Canada. The studies included in this review are heterogeneous in terms of study population, country, health services context, timing of the study, currency of costing, methods and analytical approaches. The WES diagnostic rate ranged 16–79% and our analysis found that genomic sequencing markedly increased the diagnosis rate compared to the standard diagnostic pathway (Table 1). Implementation of genomic sequencing early in the diagnostic pathway substantially reduced the usual long diagnostic trajectory using traditional standard care [9, 10, 13]. Moreover, the cost of the genomic sequencing pathway was 11–64% lower than that of standard care except one study in Canada (Table 1). Finally, genomic sequencing was found to be not only cost-effective compared to the standard diagnostic pathway in three clinical cohorts, but the genomic sequencing pathway produced a cost saving while providing additional diagnoses compared to the traditional standard diagnostic pathway [9, 10, 13].

Table 1.

Summary of the cost estimate of genomic sequencing conducted in the paediatric population

Study	WES diagnosis rate (%)	Cost of standard pathway per patient	Cost of WES pathway per patient	Change of cost (%)	Cost of WES test	Measure of cost-effectiveness
Soden et al. [3]	45	$19 100 per family	$7640 per family	60	$2996 per patient	—
Valencia et al. [4]	47	—	—		—	—
van Nimwegen et al. [5]	—	€12 475 per patient	€10 754 per patient (if WES replaced all genetic tests); €9300 per patient (if replaced by WES + 50% physician visits + all repeated and burdensome tests)	14; 25	€3600 per patient–parent trio	—
Monroe et al. [7]	29.45	$16 409 per patient	$12 437 per diagnosed patient; $14 682 per undiagnosed patient	24; 11	$3972 WES-trio cost	$3547 cost saving per diagnosed patient; $1727 cost saving per undiagnosed patient
Joshi et al. [6]	—	$9015 to $35 483 per patient	—	—	$6100 WES-trio cost	—
Sabatini et al. [8]	20	—	—	—	$1499	US$1.33 to 10.8 million cost saving for 1 million patients
Vissers et al. [12]	29.3	€10 685 per patient	€8356 per patient	22	€3500 WES-trio; €1800 singleton WES	€744 cost saving per patient
Tsiplova et al. [11]	15.8 (WES + CMA); 42.4 (WGS)	CA$744 per sample (CMA)	CA$1655 per sample (WES); CA$2851 to 5519 per sample (WGS)	−122; −(283 to 641)	—	Incremental cost CA$ 25 458 per additional diagnosis (WES + CMA); CA$26 020 to 58 959 per additional diagnosis (WGS)
Stark et al. [10]	63	AU$4734 per patient	AU$3752 per patient	21	AU$2000 singleton WES	Cost saving of AU$2182 per additional diagnosis
Tan et al. [13]	52.3	AU$9901 per patient	AU$5186 per patient first tertiary presentation; AU$7047 per patient first genetics appointment	48; 9	AU$2000 singleton WES	Cost saving of AU$9020 at first tertiary presentation; and AU$5461 at first genetics appointment per additional diagnosis
Schofield et al. [9]	NMD panel 75; WES 79	AU$10 491 per patient	NMD AU$3808 per patient; WES AU$6077 per patient	64; 42	AU$1100 NMD panel; AU$2600 WES	Cost saving AU$23 390 (NMD panel); AU$13 732 (WES) per additional diagnosis

To understand the evidence of economic evaluation of genomic sequencing and to identify evidence gaps, we summarised the findings from the economic evaluations in the selected articles in this review. The first set of studies are mostly analysis of costs and diagnostic yields, and in some cases, cost comparison with hypothetical alternative diagnostic pathways [3, 5–8, 12]. Soden et al. undertook a cost analysis study of exome and genome sequencing for the diagnosis of neurodevelopmental disorders among 119 children in the United States. They found a mean cost of prior negative standard diagnostic tests of $19 100 per family enroled from the ambulatory setting with a diagnostic rate of 40%. They also calculated the break-even point at which WES pathway would cost no more than standard diagnostic care—this was determined to be US$7640 per family, and US$2996 per individual with a molecular diagnostic rate of 45% [3]. Additionally, the study found that if WES or WGS had been performed at symptom onset, a molecular diagnosis might have been made 77 months earlier than occurred in this study. Valencia et al. [4] in a retrospective review of 40 cases to examine the clinical impact and cost-effectiveness of WES in the United States found genetic variants in 30% patients of which 47% of the variants were previously unreported. The study considered WES could be cost-effective as a result of ending the long diagnostic odyssey in positive cases. van Nimwegen et al. [5] in their costing study for paediatric neurological disorders among 50 children in the Netherlands found traditional diagnostic trajectories of complex paediatric neurological disease with a suspected genetic component are lengthy (mean duration 40 months) with low a rate of diagnosis (6% of the patients). They also found replacing all genetic tests by WES would reduce the cost by €1721 per patient and the average cost per patient would be €5321. Another costing study among 17 children with intellectual disability in the Netherlands by Monroe et al. [7] found that WES resulted in a molecular diagnosis for about 30% of patients with the average diagnostic trajectory being 6.6 years for the traditional diagnostic trajectory at a cost of $16409 per patient. They reported that WES sharply reduced the cost to $3972 per patient and resulted in an average cost savings of $3547 per diagnosed patient and $1727 per undiagnosed patient. In the United States, Joshi et al. [6] conducted a scenario cost analysis for the diagnosis of early-onset epileptic encephalopathies among for a small sample four patients, and found the total cost of traditional tests ranged from $9015 to $35 483. They found the cost of a WES-trio was $6100 and the WES diagnostic pathway was cost saving if WES was performed earlier in the diagnostic odyssey. Sabatini et al. [8] in a cost-impact analysis for patients with non-small cell lung cancer, sensorineural hearing loss and neurodevelopmental disorder in the United States found that the WES pathway increased the diagnostic yield from 30 to 40% compared with the traditional diagnostic pathway. They found that the use of chromosomal microarray and testing for fragile X, followed by WES there was a cost saving US$1.33 million for one million patients. A recent study in the Netherlands by Vissers et al. [12] found the diagnostic rate using WES (29.3%) was four times higher than that of standard care (7.3%), and the average cost of the WES diagnostic pathway (€3420 per patient) was three times lower than that of the standard diagnostic pathway (€10 685 per patient). They also revealed WES as a first-line diagnostic test produced a saving of €744 per patient without any incremental analysis.

The second set of studies published in 2017 provided methodologically more solid economic evaluation of genomic sequencing [9–11, 13]. A recent cost-consequence analysis conducted among patients with autism spectrum disorder (ASD) in Canada by Tsiplova et al. [11] estimated the incremental cost per additional ASD patient with a positive genetic diagnosis using WES and chromosomal microarray analysis (CMA) at CA$ 25458, and using WGS, CA$ 26020–58 959 compared to CMA. Stark et al. [10] conducted a cost-effectiveness analysis for a cohort of 40 children (0–2 years of age) with childhood syndromes in Australia where they found WES increased the molecular diagnosis rate from 18 to 63%, with an incremental cost per additional diagnosis of AU$8113, by integrating singleton WES after exhaustive standard investigations. However, their analysis also revealed an incremental cost saving per additional diagnosis of AU$2182 when integrating WES as a first-line test. Another cost-effectiveness study of 44 older children with childhood syndromes in Australia by Tan et al. [13] estimated that WES resulted in a cost saving of AU$9020 (at tertiary presentation) and a cost saving of AU$5461 (at first genetics appointment) per additional diagnosis compared to standard care. Finally, Schofield et al. [9] conducted a cost-effectiveness analysis for 56 children with paediatric muscle diseases in Australia where they found a neuromuscular disease (NMD) panel increased the diagnostic yield from 46 to 75%, and WES to 79%. They further estimated that the NMD panel and WES were both cost saving, AU$23 390 and AU$13 732, respectively, per additional diagnosis compared to the traditional diagnostic approach.

To understand the methodological gap, we systematically examined the methods employed to produce evidence from economic evaluations of genomic sequencing in the included articles in this review using an established checklist for a well-designed economic evaluation (Table 2) [18]. Out of the 11 articles, 1 reported cost of WES but provided no cost comparison with traditional standard diagnostic care [4]. Six studies provided relative cost analyses that compared the use of WES with hypothetical alternative diagnostic pathways through scenario analysis rather than the actual use of WES in a clinical population [3, 5–8, 12]. The cost data collection approach and cost calculation processes were not always clearly described in the majority of these studies. Only 4 studies performed cost-effectiveness analysis in clinical paediatric patient cohorts and met all the criteria for a well-designed cost-effeteness study [9–11, 13].

Table 2.

Assessment of the published economic evaluation of genomic sequencing based on the standard criteria

Serial	Checklist from Drummond et al. [18]	Soden et al. [3]	Valencia et al. [4]	van Nimwegen et al. [5]	Monroe et al. [7]	Joshi et al. [6]	Sabatini et al. [8]	Vissers et al. [12]	Tsiplova et al. [11]	Stark et al. [10]	Tan et al. [13]	Schofield et al. [9]
1	Was a well-defined question posed in answerable form? (Study place a decision-making context; examine both costs and effects; involve alternative comparison)	✓	X	✓	✓	✓	✓	✓	✓	✓	✓	✓
2	Was a comprehensive description of the competing alternatives given?	X	X	?	?	X	✓	✓	✓	✓	✓	✓
3	Was the effectiveness of the programme or services established? (Effectiveness data collected)	✓	X	X	X	✓	✓	✓	✓	✓	✓	✓
4	Were all the important and relevant costs and consequences for each alternative identified? (Capital costs, operating costs included)	X	X	?	X	X	✓	✓	✓	✓	✓	✓
5	Were costs and consequences measured accurately in appropriate physical units?	✓	?	✓	✓	✓	✓	✓	✓	✓	✓	✓
6	Were the cost and consequences valued credibly? (Using market value)	?	?	✓	✓	✓	✓	✓	✓	✓	✓	✓
7	Were costs and consequences adjusted for differential timing? (Discounting and its justification given)	X	X	X	X	X	?	✓	✓	✓	✓	✓
8	Was an incremental analysis of costs and consequences of alternatives performed?	X	X	X	X	X	X	X	✓	✓	✓	✓
9	Was allowance made for uncertainty in the estimates of costs and consequences? (statistical analysis, sensitivity analysis)	X	X	✓	X	X	X	✓	✓	✓	✓	✓
10	Did the presentation and discussion of study results include all issues of concern to users? (Ratio of costs to consequences, generalisability, adoption of preferred programme)	✓	X	X	✓	✓	✓	✓	✓	✓	✓	✓

Note: ✓ = yes; X = no; and ? = unclear

Discussion and conclusions

With the advance of technology and proven efficacy, genomic sequencing is emerging as an effective tool for the diagnosis of rare diseases for routine clinical care in the tertiary healthcare system in OECD countries. However, our review found that while there were a small number of studies, which qualify as cost-effectiveness studies of genomic sequencing for the paediatric patient population in clinical settings, a number of studies claimed to have undertaken a cost-effectiveness analysis but did not meet accepted criteria for provision of evidence for public funding in OECD countries. Further, many of the studies did not actually undertake genomic sequencing in a clinical cohort but used hypothetical data. Others compared only the cost of the diagnostic odyssey in patients who received a diagnosis using genomic sequencing; however, a cost-effectiveness study requires inclusion of patients in the studies for whom genomic sequencing was undertaken and diagnoses were and were not achieved. We do not aim to criticise studies for not undertaking analysis beyond what they claim to offer, but rather to review what evidence is currently available in relation to economic evaluation in the field of genomic sequencing for paediatric patients, to draw attention to the quality of that evidence and where advances are needed.

There are a small number of cost-effectiveness analysis studies in the field of genomic sequencing for paediatric patients included in the current review where cost per additional diagnosis was found to be cost saving (less costly, but more effective). Where the targeted intervention is more costly and more effective, there is no consensus among the decision-makers in regards to the accepted threshold for cost per additional diagnosis. Consensus is made more difficult as genomic sequencing (WES and WGS) is a new technology, there are few economic evaluations and the cost is falling. Furthermore, where there is consensus amongst public funders, it is generally around willingness to pay for an additional QALY gained, rather than an additional diagnosis. (It should be noted that where the proband cannot reliably respond to surveys for themselves, e.g., young children or patients with an intellectual disability, a parent or carer may respond as their proxy.)

In the studies included in the current review there is the potential bias in estimation of costs and effects as the estimates were derived from studies with small sample sizes that are unlikely to be representative of the patient population. These studies conducted in small samples may not be generalisable. Only six studies provided a measure of uncertainty, which is particularly important in studies with a small sample size.

It is important to note that without rigorous economic evaluation and the collection of high-quality data, there is insufficient evidence for public funding for genomic sequencing in routine clinical care. Although there are few methodologically robust economic evaluations of genomic sequencing prior to 2017, our review draws a conclusion that the use of genomic sequencing in diagnosis of rare diseases of genetic origin in some reported paediatric settings is cost-effective, in terms of the diagnostic pathway (implementing genomic sequencing as a first-tier test in paediatric clinical settings was found to be incrementally cost saving per additional diagnosis compared to standard diagnostic pathway). The emergence of well-designed economic studies of the diagnostic pathway in 2017, where genomic sequencing was undertaken within a clinical cohort provide some evidence to inform policy to support funding of genomic sequencing for regular clinical practice.

These cost-effectiveness studies did not measure improvements in health outcomes of the paediatric patient population in terms of QALYs gained as a result of a positive diagnosis and subsequent changes in patient management following both traditional and genomic sequencing diagnostic pathways. Thus, the studies could not report a cost-utility analysis to demonstrate changes in health-related QALYs of the patients as result of implementing genomic sequencing instead of traditional diagnostic approach.

We conclude that there is a need to undertake for the robust economic evaluations in prospective clinical cohorts to provide generalisable evidence of allocative efficiency for genomic sequencing and to provide evidence for the formulation of policy on public funding to integrate genomic sequencing into routine clinical care. There is also a need to include health-related QALYs in the cost-effectiveness analysis so that incremental QALYs gained can be attributed to the genomic sequencing diagnostic pathway.

Funding

The review was funded by Murdoch Children’s Research Institute.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest

Electronic supplementary material

The online version of this article (10.1038/s41431-018-0175-6) contains supplementary material, which is available to authorised users.