Cost Analyses of Genomic Sequencing – Lessons Learned from the MedSeq Project

Abstract

Objective:

To summarize lessons learned while analyzing the costs of integrating whole genome sequencing into the care of cardiology and primary care patients in the MedSeq Project, the first randomized controlled trial of whole genome sequencing in general and specialty medicine.

Methods:

Case study that describes key methodological and data challenges that were encountered or are likely to emerge in future work, describes the pros and cons of approaches considered by the study team, and summarizes the solutions that were implemented.

Results:

Major methodological challenges included defining whole genome sequencing, structuring an appropriate comparator, measuring downstream costs, and examining clinical outcomes. Discussions about solutions addressed conceptual and practical issues that arose due to definitions and analyses around the cost of genomic sequencing in trial-based studies.

Conclusions:

The MedSeq Project provides an instructive example of how to conduct a cost analysis of whole genome sequencing that feasibly incorporates best practices while being sensitive to the varied applications and diversity of results it may produce. Findings provide guidance for researchers to consider when conducting or analyzing economic analyses of whole genome sequencing and other next generation sequencing tests, particularly regarding costs.

Precis:

The MedSeq Project provides lessons about analyzing and interpreting the costs of integrating whole genome sequencing costs into patient care, lessons that are applicable to other tests that use next generation sequencing approaches.

Introduction

Advancements in next generation sequencing (NGS) have made it feasible to integrate whole genome sequencing (WGS) into patient care at a population level, and may streamline the practice of medicine [1]. Currently, genomic testing begins by testing symptomatic patients with panels of genes where mutations are most likely to explain the disorder. If no causal variants are identified, physicians may order additional tests to examine other candidate genes, a process that can continue until options are exhausted. WGS allows all candidate genes to be examined at once, including regulatory domains and genes that are not typically tested. In addition, WGS information can influence medication choices, inform reproductive decisions, facilitate targeted prevention, and more [2, 3]. Moreover, it can be re-queried for diagnostic and treatment purposes as new needs arise. The ability of WGS to provide information with lifelong utility provides a compelling rationale for its use at a population level.

However, many commentators also fear the cost and budgetary implications of integrating WGS into regular medical practice [4–7]. It can be many times more expensive than targeted tests, and typically has lower sensitivity for identifying certain types of variants than other types of genomic tests [8]. WGS also tends to identify more variants of uncertain significance which can require additional clinical workup, and WGS can provide secondary findings that are unrelated to the test indications but may motivate follow-up testing and long-term screening.

To understand the impact of integrating WGS into the everyday care of sick and healthy populations, we conducted the MedSeq Project, the first randomized controlled trial of WGS in cardiology and primary care settings [9]. In addition to describing the molecular yield and clinical impact of disclosure [10, 11], we used microcosting and gross costing methods to report the short-term costs of integrating WGS into clinical practice, including its impact on short-term health care utilization and other health sector costs [12]. Findings showed an incremental cost of $5,000 to integrate WGS into patient care in 2015, no noticeable impact on downstream health care utilization over a six-month time horizon, and less than $200 per patient to disclose secondary findings.

The purpose of this paper is to discuss key methodological challenges that arose in that cost analysis because of the unique characteristics of WGS. The lessons we summarize and the solutions we adopted provide practical guidance and points to consider as researchers and policymakers develop and interpret cost analyses of WGS and NGS tests more broadly.

Methods

The methods of the MedSeq Project have been described in detail previously, including the rationale and design of the study [9], the approach to WGS, variant analysis and reporting [13, 14], and the rationale and design of the cost analyses [15]. Key terms that are used in this case report are summarized in Table 1. Briefly, the MedSeq Project was a set of parallel randomized pilot trials to examine two archetypal scenarios for integrating WGS into clinical care. The first, disease-specific genomic medicine, used WGS to identify molecular causes for disease in patients with family histories or symptoms suggestive of a genetic disorder. To examine this scenario, we enrolled cardiologists and patients with diagnoses of hypertrophic or dilated cardiomyopathy. The second scenario, general genomic medicine, used WGS to screen for genetic disorders, to enhance disease prevention and to improve medical and personal decision making. To examine this scenario, we enrolled primary care physicians and ostensibly healthy patients.

Table 1.

Key terms used in this case report. Definitions were adapted from the NCI Dictionary of Genetics Terms [67] and from published literature [68]. Terms are presented in the order which they appear in the case report.

Whole genome sequencing (WGS)	A laboratory process that is used to determine nearly all of the approximately 3 billion nucleotides of an individual’s complete DNA sequence, including non-coding sequence. Here, we include bioinformatic analyses to identify health-relevant information, and reporting of these findings to health care providers and their patients.
Variant	An alteration in the most common DNA nucleotide sequence. The term variant can be used to describe an alteration that may be benign, pathogenic, or of unknown significance.
Coverage	The number of times a nucleotide is read during sequencing.
Singleton testing	A genetic testing strategy that examines the DNA of a patient alone.
Trio testing	A genetic testing strategy that examines the DNA a patient along with the DNA of parent, usually to identify variants that are present in a sick patient that are absent in healthy parents.
Deletion	A type of genetic change that involves the absence of a segment of DNA. It may be as small as a single base but can vary significantly in size.
Insertion	A type of genetic change that involves the addition of a segment of DNA that can be as small as a single base.
Translocation	A type of chromosomal abnormality in which a chromosome breaks and a portion of it reattaches to a different chromosomal location.
Sanger sequencing	A low-throughput method used to determine a portion of a patient’s nucleotide sequence. This method is well-validated, and has high sensitivity and specificity for identifying variants.
Structural variant	A type of large genetic change (i.e, approximately 1,000 base pairs or larger in size). This change can include an inversion (a segment of a chromosome that breaks off and reattaches in the reverse direction), a translocation, an insertion, or a deletion.
Single nucleotide polymorphism	A type of variant present in at least 1% of the population where a single nucleotide in the genome sequence is altered.

After consenting to the study and completing a baseline survey, patient participants were randomized to meet with their providers and review health information that included or omitted WGS. Participants were then followed for six months. Data relevant to the cost analyses were collected from surveys of providers and patients, and from medical records and administrative data.

Key challenges that are summarized here were identified by consensus of the investigators who led the cost analyses. We focused on decisions that had a large impact on our analyses and would be applicable to future cost analyses of WGS and other NGS tests. We also highlight issues where recent developments may change future analyses.

Results

We identified three key challenges in conducting cost analyses of WGS: defining the test, developing appropriate comparators, and assessing downstream costs. We additionally describe challenges to collecting data about clinical outcomes.

Challenge 1: Defining whole genome sequencing

The first challenge we addressed was to define how we would implement WGS. Decisions about whether to conduct singleton testing or test multiple family members, what sequencing system (“platform”) to use, and the minimum coverage that WGS should achieve can have a large impact on costs and molecular yields [2, 8]. Professional groups such as the American College of Medical Genetics and Genomics (ACMG), the Association for Molecular Pathology, and the College of Medical Pathologists have been developing standards for WGS [16–18], and the optimal approach depends on the purpose of testing, time frame for results, patient characteristics, and more. Nevertheless, many aspects of WGS still vary from setting to setting. Here, we focus here on decisions about conducting WGS that had a significant impact on costs and molecular yields in the MedSeq Project but might be made differently in future work. These decisions are summarized in Table 2.

Table 2.

Clinical and cost implications of different WGS approaches. Only considerations that would affect costs by $200 are presented.

Consideration	MedSeq Approach	Alternatives	Clinical Implications	Costs of Alternatives (relative to MedSeq)
Sequencing approach	Singleton	Trio	Ability to identify de novo variants, interpretation of reported variants	2X [69]
Sequencing platform	Illumina HiSeq 2000	Numerous	Turnaround time and error rates [20, 21]	1/4X to 2X, depending on platform and usage [20, 23]
Mean coverage	30X	100X+	Turnaround time and error rates [16]	4X [22]
Confirmation approach	Sanger sequencing	No confirmation	Error rates	Savings of $250-$625 [12, 70]
Types of WGS findings reported	Monogenic disease risks in 4,600+ genes, carrier status, PGx, cardio-metabolic risks, blood type/antigen predictions	Primary findings only, different combinations of secondary findings, fewer genes	Primary and secondary prevention, medical decision making	Up to $200/patient [12]
Classifications of secondary findings reported	Pathogenic, likely pathogenic, and VUS: favor pathogenic	Pathogenic only, pathogenic + likely pathogenic	Clinical validity [17]	Up to $200/patient [12]

Many decisions about how to conduct WGS in the MedSeq Project were influenced by practical considerations in addition to conceptual ones. We used a singleton testing approach rather than trio testing to maximize the number of different families we could provide WGS, and because identification of de novo variants – which is optimized with trio sequencing – would have limited utility in our cohort of healthy primary care patients. Sequencing was conducted using the Illumina HiSeq 2000 platform because in 2012, when the MedSeq Project launched, it was one of few platforms with well-established quality metrics. Newer sequencing platforms have emerged, however, with varying advantages and disadvantages with regard to speed, accuracy calling specific types of variants (e.g., substitutions vs insertions), and costs [19–23]. We opted for at least 30X mean coverage to conform to ACMG standards for WGS, recognizing that greater coverage would improve our ability to identify mosaicisms (i.e., mutations that are present in only a fraction of cells) and variants such as deletions, insertions, and translocations [16, 24, 25], albeit at greater expense.

Another important consideration that will affect future cost analyses of WGS is whether and how to confirm sequencing findings. Current standards are to confirm variants with additional testing prior to reporting [16–18]. In the MedSeq Project – as is typical in many current clinical testing protocols – we confirmed results using Sanger sequencing, adding over $600 to the average per-patient costs of WGS [12]. However, standards may change in the future given growing evidence about the high accuracy of NGS to detect nucleotide substitutions, although insertions, deletions, and larger structural variants are likely to remain problematic [26].

A final consideration about the conduct of WGS is the scope of information that will be reported. Existing guidelines recommend that laboratories query at least 59 genes for known or expected pathogenic variants in actionable conditions whenever sequencing is initiated, regardless of clinical indication [27, 28]. Many laboratories will also offer to provide other secondary findings in additional genes, carrier status for autosomal recessive conditions or pharmacogenomic results that could influence drug metabolism and side effects. In the MedSeq Project, WGS reports also included findings classified as pathogenic, likely pathogenic, or of uncertain significance where evidence favored pathogenicity in any of over 4,600 genes associated with monogenic diseases. Moreover, in addition to the above, we reported risk predictions for eight cardiometabolic traits [29], and blood group and antigen predictions [30].

To account for alternative approaches to conducting WGS in the MedSeq Project and to provide insight about the future, we conducted sensitivity analyses that considered WGS costs as low as $500 and as high as $10,000 (approximately 10% to 200% of the costs, per our analyses, of $5,225). In addition, we conducted scenario analyses where we examined different reporting criteria, such as omitting specific types of results (e.g., carrier status, risk predictions for cardiometabolic traits) or reporting only secondary findings classified as pathogenic. We were able to conduct these scenario analyses by microcosting laboratory and clinical tasks and by having physicians link the follow-up services they ordered to specific WGS findings.

Challenge 2: Developing an appropriate comparator

At the earliest stages of the MedSeq Project, the study team extensively discussed what intervention to provide to patients randomized to the control arm. Many of these discussions centered on conceptual questions about how to characterize the study’s use of WGS. As a diagnostic tool, we used WGS to identify molecular causes for cardiology patients’ cardiomyopathy diagnoses. At the same time, we used WGS as a screening tool for monogenic disease risks and carrier status, information about cardiometabolic traits, pharmacogenomic information, and red blood cell and platelet antigen might inform targeted prevention. The combination of WGS results with potential benefits for diagnostic, screening, prevention, and decision-making purposes made developing an appropriate comparator challenging.

Some of the options that the study team considered are summarized in Table 3. We briefly considered comparing WGS against no intervention, given the lack of comparators with similar capabilities and study goals that were not focused on specific clinical outcomes. However, a design with an extra clinical encounter to disclosure WGS results had the potential to bias downstream health care utilization and costs in ways that were unrelated to WGS. We also discussed using a wellness visit where physicians would screen for disease and review preventive health recommendations [31], but felt that an intervention with a greater focus on genetic disorders would be a better comparator to WGS. A third strategy we considered was providing a genomics-focused comparator using panel testing or by profiling single nucleotide polymorphisms (SNP). For example, an expansive SNP analysis, similar to those used previously by direct-to-consumer genetic testing companies and studies like the Multiplex Initiative, could provide similar types of information as our WGS analyses, including estimates of disease risk, carrier status, and pharmacogenomic information [32, 33]. However, these approaches have raised questions about the predictive power of the underlying algorithms and the potential that results may misinform medical decisions [34, 35].

Table 3.

Interventons considered for the control arm in the MedSeq Project. Advantages and disadvantages focus on the implications for analyzing costs and clinical benefits.

Comparator	Pros	Cons
No intervention	Control arm would be unbiased by an “artificial” intervention	May inflate costs in the WGS arm in ways that were unrelated to genomics by introducing an extra clinic encounter
Well care visit	Balances number of clinical encounters in randomization arms with interventions focused on screening and prevention	Comparator would lack a genomics focus
Panel-based genomic testing	Provides insight about the incremental benefits and costs of WGS compared to other genomic testing approaches	Comparator would represent standard of care only among symptomatic patients
Family history review	Standard of care, yet frequently neglected; can identify potential genetic disorders	Cardiology patients already had a thorough FH review, FH reporting often biased

Ultimately, we developed a comparator that focused upon reviewing patients’ family histories of disease. Family history review is frequently neglected, despite being standard of care, and it can identify patterns of disease suggestive of an inherited genetic risk factor [36]. Patient participants completed a modified version of the Surgeon General’s “My Family Health Portrait” tool [37] at enrollment, and physicians reviewed findings from these reports with patients in both randomization arms during disclosure sessions. Furthermore, cardiology patients had to have prior or concurrent cardiomyopathy panel tests as a condition of enrollment, and cardiologists reviewed findings from panel testing in both randomization arms during MedSeq Project disclosure sessions. By implementing cohort-specific interventions based on a genomics-focused standard of care, we were able to create comparators and assess the incremental costs of WGS relative to an idealized standard of care, even if it might not mimic typical clinical practice.

Challenge 3: Assessing post-disclosure costs

There are substantial concerns about the potential impact of WGS on downstream health sector costs [38, 39]. Assessing the health care services and associated costs that WGS may generate represented a major challenge in the MedSeq Project cost analyses, given the diversity of conditions and information that we disclosed to patients.

WGS increases the likelihood of identifying variants of uncertain significance that can prompt follow-up testing to verify their clinical importance [38]. Also, additional clinical workup or ongoing screening may be motivated by secondary findings. Only one to three percent of people are thought to have disease-causing variants in any of the 59 genes recommended by the ACMG for secondary findings disclosure [40–42]; but the percentage can be much higher under other reporting criteria. The expansive approach implemented in the MedSeq Project identified monogenic disease risks unrelated to test indications in 16% of cardiology patients and 26% of primary care patients [10, 12]. In addition, nearly all sequenced patients were identified with carrier status for at least one autosomal recessive conditions, and all sequenced patients received pharmacogenomic information, cardiometabolic risk predictions, and blood group/antigen predictions by design [9]. Recommendations for cost analyses in clinical trials are to collect utilization data about “relevant health care services,” regardless of why they were ordered [43, 44], but the diversity of WGS information provided in the MedSeq Project precluded our ability to use disease-specific health care utilization instruments.

Our solution was to implement multiple strategies, as summarized in Table 4. First, we asked physicians to complete checklists after disclosure sessions where they documented recommendations for follow-up care that were prompted by family history or WGS findings. Admittedly, this strategy captured only those services that were initiated by a physician shortly after WGS and/or family history disclosure, and missed services that may have been ordered by specialists after referrals. It may have also missed services initiated by participants rather than physicians. Nevertheless, our approach allowed us to report findings about “immediately attributable” costs with acceptable precision.

Table 4.

Approaches to assessing the downstream impact of family history reviews and WGS on health care utilization in the MedSeq Project.

Strategy	Advantages	Disadvantages	Notes
Document physician recommendations during disclosure sessions	Clearly identifies services that were initiated due to MedSeq Project disclosures.	Only identifies services initiated during disclosure services, cannot account for potential savings.	Implemented, with follow-through confirmed through review of medical records.
Identify services by reviewing medical records	Ensures services occurred, and were often accompanied by billing codes	Time-intensive, difficult to link to study disclosure sessions, misses out-of-system services	Implemented to identify all services, with no attempt to link to disclosure
Survey patients about follow-up health care services	Easy to analyze, identifies care that may have occurred outside the Partners HealthCare system	Subject to reporting biases, challenging for patients to complete	Implemented, but not linked to disclosure
Identify potential follow-up through expert review	Ensures family history and WGS reports are interpreted correctly.	Artificial	Implemented with a focus on monogenic findings.

Second, we documented all medical services that occurred in the six months following disclosure sessions by reviewing participants’ medical records and administrative data, which included billing codes. This time-intensive process missed services that occurred outside the Partners HealthCare system, and introduced great variability into cost estimates by including services that were unlikely to be related to MedSeq Project reports. On the other hand, this encompassing approach established the methodological foundation for follow-up studies where the lack of precision in cost estimates may be overcome by enrolling larger numbers of patients.

Third, we asked patients to report services they received. We administered survey items about medical testing that were adapted from the Behavioral Risk Factor Surveillance System [45] and the Impact of Personal Genomics Study [46], as well as consensus health care utilization measures developed for the Clinical Sequencing Exploratory Research Consortium [47], into surveys that patients completed six weeks and six months following MedSeq Project disclosure sessions. The inclusion of these items provided our study team with insight about services that occurred outside the Partners HealthCare System.

Finally, we included an approach where genetic counselors and medical geneticists identified health care services associated with a comprehensive work-up of monogenic disease risks. These services were based on guidelines provided in repositories such as GeneReviews and Online Mendelian Inheritance in Man (OMIM), as well as a review of published literature [48, 49]. Analyses were included to provide a potential “high side” of costs associated with monogenic conditions, but did not provide insight about services that might be motivated by the information provided in WGS reports about carrier status, drug metabolism, or risk for cardiometabolic conditions.

Challenge 4: Documenting the benefits and harms of disclosure

Our published cost analysis provides crucial insight about the short-term impact of WGS, but we recognize that more important questions remain about whether WGS provided benefits that justify any additional spending. Patients outcomes that we considered examining in the MedSeq Project are summarized in Table 5. We previously published monogenic disease risk findings, carrier status findings, pharmacogenomic findings, and cardiometabolic risk predictions descriptively [10–12]. However, metrics that focused on genomic variants did not have an analogue in the comparison arm.

Table 5.

Clinical outcomes that were considered in the MedSeq Project.

Outcome	Pros	Cons
Molecular yield	Common metric for success of genetic tests	Findings have unclear clinical validity; does not indicate changes to or improvements in care; not relevant to family history analyses
Diagnostic yield (new or revised diagnoses)	Applicable to both randomization arms	Time to diagnosis may be beyond the study time frame
Intermediate (e.g., changes in lab scores) and clinical outcomes (e.g., cardiac events)	Applicable to both randomization arms, indicate a change in clinical care with likely benefits to health	Typically requires a follow-up clinical encounter that is not mandated in the study
General health-related quality of life	Applicable to both randomization arms, facilitates cross-study comparisons, permits estimation of health utility	Insensitive to short-term change

Metrics that will be examined for future reporting may have greater relevance to both randomization arms. Surveys asked patients whether the information they received led to new diagnoses. We are also using approaches developed in the Electronic Medical Records in Genomics (eMERGE) Network to examine intermediate and clinical outcomes associated with the cardiometabolic risk predictions we reported [50]. We will examine whether changes to laboratory scores, such as cholesterol levels, or cardiac events varied by randomization status, although data will only be available on a subset of patients who had clinical encounters and testing after their MedSeq Project disclosure sessions.

Lastly, we included measured health-related quality of life to inform future economic analyses. We administered the SF-12v2 at disclosure and six months after disclosure [51], opting for a generic measure rather than a disease-specific one given the diversity of information and conditions that may be addressed on WGS reports. Moreover, well-established algorithms exist to convert SF-12v2 scores into SF-6D health states and utility values [52]. Six months is likely far too short a time horizon to observe an impact of WGS on quality-adjusted life years, but the foundation for data collection will provide data that can inform long-term follow-up studies.

Discussion

Present applications of WGS tend to focus on prenatal, pediatric, oncology, and rare disease contexts, and a growing number of studies have examined the economic impact of genomic sequencing for diagnostic and treatment purposes. The true potential of WGS may be realized in the everyday practice of medicine, however, where it can be additionally used for screening and prevention [53]. Widespread clinical use of WGS in everyday patient care will only occur if it can provide value. Lessons from the MedSeq Project highlight the unique challenges in assessing the costs of WGS and analyses of other NGS tests.

The lessons we summarize in this report provide practical guidance not only for researchers who conducting their own cost analyses, but also for scientists and policymakers who are interpreting the findings from other work. Aggregated estimates of the costs of sequencing, such as those provided by the National Human Genome Research Institute [54], may be inappropriate if they don’t account for differences in testing choices that may be influenced by the clinical context and patient characteristics, as well as regulatory requirements from oversight organizations such as the FDA [55]. We were able to develop valid estimates of WGS costs in our study by implementing a work-intensive microcosting approach, but more importantly, researchers of the cost impact of NGS tests will need to be careful to incorporate the full variability of approaches that may be appropriate to their clinical contexts and patient populations.

Our work also demonstrates how we were able to create a suitable comparator by developing an intervention focused on procedures that are standard of care, but often neglected: family history review. We identified a number of patients with unaddressed family histories of heart disease, dementia, and cancer using this approach, information that motivated providers to make referrals and initiate clinical follow-up [12]. However, it should be noted that numerous approaches exist for collecting and analyzing family history information, which is often inaccurate [56, 57]. Tools such as MeTree and Family HealthWare have emerged that not only collecting more accurate data, but also generate targeted prevention messages [58, 59]. Also, SNP-based risk predictions have improved since the MedSeq Project launched [60, 61], and the approach may have greater acceptance in clinical settings in the future. In short, determining appropriate comparators for NGS tests will require researchers to be sensitive to developments emerging in other genomics-related tools, not just advances in NGS technologies.

Our solution to measuring the short-term downstream costs of WGS was successful. A major benefit to our approach was its flexibility, allowing for sensitivity and scenario analyses that provided insight about different strategies for reporting unanticipated findings. Yet, our approach was time-intensive, and it is unclear how well we captured patient-reported services that occurred outside of the Partners HealthCare system. These issues will be only more important to address in the future, as larger studies are conducted. Scalable, more accurate solutions to assessing the downstream cost impact of these tests would track participant expenditures across health systems through initiatives such as an All-Payer Claims Database [62].

Finally, the difficulties our research team faced in measuring clinical outcomes demonstrate the challenges of assessing them when the NGS test being analyzed provides varied health-relevant information. Our principal approach, using a generic measure of health-related quality of life, is an appropriate approach but may lack the sensitivity in a small sample size and short time horizon to detect a clinical impact. In addition to enrolling larger samples into future trials, one promising solution may be the use of technology to widen the scope while tightening the precision of measurements. Computer adaptive tests such as PROMIS have demonstrated great potential to detect changes to general health-related quality of life while minimizing participant burdens [63], and ongoing efforts work is underway to convert PROMIS scores to health utilities for economic analyses [64].

Conclusions

As a pilot project, the MedSeq Project cost analyses of WGS provide practical guidance about considering the full costs of NGS tests in patient care. The lessons we learned will be particularly relevant given the launch of large population-based initiatives that include sequencing, such as the Million Veteran Program [65] and the All of Us Research Program [66], and the need to understand their cost impact.

HIGHLIGHTS.

What is already known about the topic?

Improvements in next-generation sequencing are making it feasible to integrate whole genome sequencing into patient care at a population level, raising the need for well-designed studies of its cost and budgetary impact.

What does the paper add to existing knowledge?

We summarize methodological challenges and solutions to analyzing the costs of integrating next generation sequencing into patient care, including accounting for differences in sequencing approaches, creating comparators, measuring downstream costs, and assessing patient outcomes.

What insights does the paper provide for informing health care-related decision making?

The paper provides methodological guidance for investigating or interpreting the costs of whole genome sequencing and other next generation sequencing tests in patient care.