Use of Real-World Evidence in US Payer Coverage Decision-Making for Next-Generation Sequencing–Based Tests: Challenges, Opportunities, and Potential Solutions

Abstract

Objective:

Given the potential of real-world evidence (RWE) to inform understanding of the risk–benefit profile of next-generation sequencing (NGS)–based testing, we undertook a study to describe the current landscape of whether and how payers use RWE as part of their coverage decision making and potential solutions for overcoming barriers.

Methods:

We performed a scoping literature review of existing RWE evidentiary frameworks for evaluating new technologies and identified barriers to clinical integration and evidence gaps for NGS. We synthesized findings as potential solutions for improving the relevance and utility of RWE for payer decision-making.

Results:

Payers require evidence of clinical utility to inform coverage decisions, yet we found a relatively small number of published RWE studies, and these are predominately focused on oncology, pharmacogenomics, and perinatal/pediatric testing. We identified 3 categories of innovation that may help address the current undersupply of RWE studies for NGS: (1) increasing use of RWE to inform outcomes-based contracting for new technologies, (2) precision medicine initiatives that integrate clinical and genomic data and enable data sharing, and (3) Food and Drug Administration reforms to encourage the use of RWE. Potential solutions include development of data and evidence review standards, payer engagement in RWE study design, use of incentives and partnerships to lower the barriers to RWE generation, education of payers and providers concerning the use of RWE and NGS, and frameworks for conducting outcomes-based contracting for NGS.

Conclusions:

We provide numerous suggestions to overcome the data, methodologic, infrastructure, and policy challenges constraining greater integration of RWE in assessments of NGS.

Introduction

Next-generation sequencing (NGS)–based tests (multigene panels, whole-exome sequencing, whole-genome sequencing) have started to transform the clinical approach to prenatal testing, cancer treatment, diagnosis of rare disorders, and predisposition testing for chronic diseases.^1–4 Nevetheless, 2 interrelated factors—inadequate evidence base and lack of coverage by public and private payers—are issues that must be addressed before NGS becomes part of routine care. Payer coverage policies often highlight evidence deficiencies in the clinical validity of the test (result is clinically meaningful) or the clinical utility of the test (result is clinically useful) as reasons for denying coverage. Clinical utility is the evidentiary standard used by most payers when evaluating tests for coverage decision making, a standard that has been affirmed for Medicare by the courts in their determination that the Centers for Medicare and Medicaid services may consider health outcomes and patient management when deciding whether to cover a diagnostic test.⁵

From a regulatory perspective in the United States, NGS tests are governed by the Food and Drug Administration’s (FDA’s) Center for Devices and Radiological Health’s evidence requirements for in vitro diagnostics (IVDs), which focuses on the demonstration of analytic validity (technical efficacy of the test) and clinical validity. The absence of regulatory requirements for premarket evidence of clinical utility and complexity of NGS (tests examine multiple genes and produce multiple results, each with distinct clinical implications) makes conventional randomized controlled trials (RCTs) more challenging and expensive to conduct. Also, the clinical actionability of NGS results can evolve over time as new information regarding the relationship between genetic variants and disease risk or drug response becomes available. Thus, observational data may be particularly useful for informing estimates of the effectiveness of testing in different disorders, patient subgroups, and clinical settings.

Although payers may prefer direct evidence of clinical utility from randomized trials of the impact of NGS tests on provider behavior and patient health outcomes, they frequently must rely on indirect evidence when addressing coverage determinations. In the context of IVDs, indirect evidence is obtained by extrapolating from robust clinical validity studies and treatment outcome studies, constructing a chain of evidence linking test results to patient health outcomes.⁶ Use of real-world data (RWD) from registries, surveys, and observational studies is a practical alternative for building indirect evidence of the clinical impact of NGS tests, also referred to as real-world evidence (RWE; Box 1).

BOX 1.

Real world Data (RWD):

data relating to patient health states and/or delivery of health care routinely collected from a variety of sources, including electronic health records, claims and billing data, product and disease registries and data gathered through personal devices and health applications.⁷

Real world evidence (RWE):

the analysis of RWD in a study designed with a high degree of pragmatism, regardless of study type.⁷

Importantly, the FDA has recognized that RWE can be used as valid scientific evidence to support regulatory claims for new devices. For example, RWE can be used to support expanded indications, conduct postmarketing surveillance, as a control group, and as evidence to identify, demonstrate, or support the clinical validity of a biomarker.⁸ Although decision making for regulatory approval of a new test is distinct from payer coverage determinations regarding the test, improvements in the quality of RWD and usefulness of RWE can be seen as “a rising tide that lifts all boats,” such that there should be a downstream benefit to payer decision making.⁹ One caveat is that the FDA can choose to exercise enforcement discretion for IVDs, allowing laboratory-developed tests to enter the market without FDA approval. Nevertheless, there are ongoing efforts to modernize federal oversight of laboratory-developed tests based on a risk-based approach¹⁰ in addition to recent evidence that the FDA is stepping up its regulation of pharmacogenetic testing (PGx) in particular.¹¹ Our objective was to describe the challenges and opportunities for payers, researchers, and test developers to capitalize on the growing availability and applicability of RWE for coverage decisions and develop potential solutions to support greater use of RWE in the context of NGS.

Previous studies^12–15 focused on pharmaceuticals have outlined the relevance of RWE for payers; however, there is a similar and growing interest regarding how RWE may be used to demonstrate the clinical utility of IVDs, a critical evidentiary threshold for payers. The analytic framework for evaluating the clinical utility of IVDs is well established^16,17 and makes clear that tests must first have demonstrated analytic and clinical validity before clinical utility can be considered. Once the relationship between test use in a defined patient group changes in provider and patient behavior, and health outcomes are established, RWD from observational studies are frequently used to provide indirect evidence of clinical utility. RWD is also essential in the development of decision-analytic models to evaluate the cost-effectiveness of NGS tests,¹⁸ information that is often relevant for private payer coverage determinations. RWD collection may also occur as part of a coverage with evidence development study or as part of outcomes-based contract evaluations.^12,19 The former refers to a situation in which a payer provides provisional coverage of certain items or services conditional on further collection of population-level evidence from a prespecified study²⁰ and the latter to contracts intended to lessen the financial risks to payers for expensive treatments by measuring the actual value delivered to patients.^19,21

Nevetheless, whether and how payers use RWE for NGS coverage decision making is not well understood, and diagnostics present unique methodological challenges for the use of RWE.²² In particular, both evidence developers and decision makers have underscored the need to adapt evidentiary frameworks and the use of RWE for NGS-based tests in clinical areas such as oncology, an area of intensive activity for NGS.^23,24 There is a similar need to understand the current landscape of how RWD is being used to elucidate the risk–benefit profile of NGS in a broader range of clinical contexts and what factors can accelerate greater uptake of RWE by payers. We briefly describe approaches that have been used to support the use of RWE in general by payers, then focus on identifying trends in the literature, policy environment, and data ecosystem that could affect the use of RWE for coverage decision making for NGS specifically. This information fills a critical gap in the RWE literature and forms the basis of potential solutions for enabling the evidence-based use of NGS by payers over time. The development of potential solutions that can be endorsed and applied by NGS stakeholders is the focus of this study.

Methods

Scoping Review

We undertook a scoping review as opposed to a systematic review to rapidly map key concepts underlying this research topic, identify evidence gaps, and synthesize knowledge within policy and practice contexts.^25,26 Scoping reviews are particularly useful for examining the extent of research activity in a particular area and determining the value of undertaking a full systematic review in the future. We limited our review to the United States because of significant variability internationally in how healthcare is financed and new technologies are evaluated for approval and payment. For this review, PubMed was used to search peer-reviewed, scientific literature from January 2013 to November 2019 in an attempt to answer 3 research questions: (1) How has RWE been used to support public and private payer decision making in the United States? (2) What are specific examples of RWE uses related to NGS? and (3) How has RWE been used to support decision making by stakeholders that influence payers? (eg, FDA, clinical guideline developers, health technology assessment). The search strategy and inclusion and exclusion criteria are described in Appendix 1 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2020.02.001.

To locate gray literature, we searched Google and GenomeWeb websites as of November 2019 using the following keywords: “Real World Evidence AND Coverage,” “Real World Evidence AND Reimbursement,” “Real World Evidence AND NGS,” “Real World Evidence AND payer,” and “real world evidence” as exact phrases. We also searched Google for gray literature on oncology data-sharing initiatives referenced in articles (see Appendix 1 in the Supplemental Materials found at https://doi.org/10.1016/j.jval.2020.02.001) using the terms “initiative name” and “coverage or reimbursement,” “clinical utility,” “performance-based risk sharing arrangement,” and “payers.” In addition, we reviewed the press releases on company websites known to be sponsors of RWE initiatives (see Appendix 1 in the Supplemental Materials found at https://doi.org/10.1016/j.jval.2020.02.001) to identify additional gray literature. Data abstraction forms were created in Microsoft Excel and used to summarize information from the included peer-reviewed and gray literature articles (see Appendix 1 in the Supplemental Materials found at https://doi.org/10.1016/j.jval.2020.02.001). Two of the authors (P.A.D. and M.P.D.) extracted data independently from the selected documents; the findings were reviewed jointly with discrepancies resolved by consensus.

Finally, we hand searched the National Human Genome Research Institute’s list of accomplishments in genomic medicine since 2011²⁷ and the peer-reviewed and gray literature reference lists of included articles. Search results from scientific (title and abstract) and gray literature were independently screened by 2 authors according to inclusion and exclusion criteria (see Appendix 1 in the Supplemental Material found at https://doi.org/10.1016/j.jval.2020.02.001). Discrepancies were resolved by consensus and review by a third author. Data from final versions of the abstraction forms were synthesized into themes organized by research question and evidence gaps highlighted.

Results

Literature Review

The PubMed search yielded a total of 426 articles, whereas the various gray literature searches produced 183 hits. From the total of 609 articles or hits, we excluded 529 articles based on the following reasons (see Fig. 1): title/abstract review (509), international studies (22), and background studies (19). Hand searching of manuscript reference lists added an additional 53 articles for a total of 69 peer-reviewed studies and 43 gray literature articles included for analysis.

Figure 1.

Flow diagram of included and excluded articles and sources.

Many articles touted the potential uses of RWE by payers; however, the vast majority of the published literature focused on drugs, not diagnostics.^{12,13,28–30} When researchers critically evaluated how payers used RWE for drug decision-making, RWE was infrequently cited in pharmacy and therapeutics materials, even among therapeutic class reviews, in which RWE is more readily available and studies are of high quality.¹⁴ One reason why RWE has not had a bigger impact is that payers lack confidence in the rigor of study designs and the validity of study conclusions (Table 1). This concern by payers exists despite numerous guidance documents published by researchers and other stakeholders regarding good research practices for conducting and reporting RWE studies.^31–38

Table 1.

RWE limitations.

Payer concerns about RWE studies

Uncertainty regarding internal validity Lack of familiarity with study methods (eg, regression, propensity score matching, etc) Lack of institutionalized methods of obtaining RWD Inaccurate recording of health events Missing data Lack of interoperable data, including genomic data Lack of transparency regarding use of methods and reporting of results Lack of study timeliness for coverage decision making Lack of resources to locate and review observational studies Privacy of patient data/HIPAA compliance Recurring breaches of privacy may discourage individuals from allowing their data to be used for research purposes

HIPAA indicates Health Insurance Portability and Accountability Act; RWD, real-world data; RWE, real-world evidence.

The situation becomes more complicated with respect to NGS because researchers attempt to capture probabilistic information as binary results, false-positives/-negatives are not addressed, testing and reporting standards are in flux, data silos prevent data sharing, reclassification of variants is common,³⁹ and clinical outcomes data are lacking.^40,41 A major hurdle for clinical laboratories is deciding which genes have sufficient evidence to support use in clinical care. A method to evaluate the strength of evidence for a gene’s role in a given disease has been developed by ClinGen,⁴² which oversees the only FDA-recognized public variant database to support clinical validity claims for genetic tests.⁴³

We identified several RWE studies that evaluated the use of NGS to guide oncology therapy in clinical practice⁴⁴ and the impact of NGS tumor profiling on the health outcomes^45,46 and economic outcomes^47–49 of cancer patients. Although tumor profiling using NGS has been gaining coverage if the test has received FDA approval⁵⁰ or there are clinical guidelines supporting test use,⁵¹ there is widespread recognition of the need for RWE to drive new frameworks for payer coverage policy development.⁵² These frameworks include coverage with evidence development and real-world performance-based risk-sharing arrangements (also referred to as “outcomes-based contracts”). There is also a growing body of studies evaluating the clinical utility of NGS for inherited conditions such as hereditary breast and ovarian cancer, familial hypercholesterolemia (FH), and Lynch syndrome (increased risk for colorectal, endometrial, and ovarian cancers), which have resulted in the Centers for Disease Control and Prevention’s assigning these conditions “tier 1” status.⁵³ This designation refers to conditions for which there is some reasonable evidence supporting implementation such as a clinical guideline based on systematic reviews, Centers for Medicare & Medicaid Services (CMS) coverage of testing, or FDA labeling.⁵⁴ Most public and private payers cover testing for hereditary breast and ovarian cancer and Lynch syndrome⁵⁵; however, coverage for FH is highly variable despite recognition by professional associations of the clinical utility of genetic testing. One of the reasons cited for this coverage gap is the lack of cost-effectiveness data supporting FH testing,⁵⁶ which requires RWD to populate economic models.

Pharmacogenomics previously focused on the analysis of variants in a single gene to predict drug response; however, there has recently been much more emphasis on the analysis of multiple pharmacogenes using NGS panels, ideally in a preemptive fashion before the prescription of any targeted drug.⁵⁷ In this scenario, pharmacogenomic results are stored in the electronic health record along with a clinical decision support (CDS) system that alerts the prescriber when an affected drug is prescribed for a patient with variant genetics. The clinical and economic impacts of implementing a preemptive PGx strategy for antiplatelet agents, statins, and warfarin has been demonstrated through modeling and application to a health system cohort.⁵⁸ The improvements were only modest and are consistent with the observation that payers do not reimburse for preemptive PGx panel testing. The strongest evidence from RWE concerns the relationship between variants in CYP2C19 and antiplatelet therapy such as clopidogrel,⁵⁹ HLA B and carbamazepine,⁶⁰ and TMTP genotypes and thiopurines,⁶¹ findings further confirmed by a review of 44 economic evaluations of a PGx-informed strategy.⁶² Nevertheless, lack of reimbursement of PGx testing remains a major barrier for the field and appears less related to lack of RWE and more related to implementation barriers, for example, the inability to represent genomic data in the electronic health record and the need to update CDS recommendations to reflect changes in variant interpretation.⁶³

Some of the most definitive work using RWE has examined the clinical utility and cost-effectiveness of NGS in critically ill infants and pediatric patients with suspected monogenic disorders.^64–68 Correspondingly, a recent qualitative study of payer decision-making revealed that 71% of payers representing 170 million insured lives cover pediatric exome sequencing, primarily because there are available interventions or to end the diagnostic odyssey.⁶⁹ In the specific context of neurodevelopmental disorders in pediatric patients, a study of private payer coverage policies of whole-exome sequencing demonstrated a trend toward more favorable coverage decisions over time, correlated with a larger evidence base, including RWE studies.⁷⁰ Another study demonstrated that payers relied on modeled evidence of clinical utility when affirmatively covering noninvasive prenatal testing.¹ Finally a review of 55 coverage policies for NGS tests compared with coverage policies for other interventions such as drugs or diagnostic imaging revealed that most clinical studies cited as supporting evidence relied on RCTs; however, some policies did cite health technology assessments and cost-effectiveness assessments, which presumably included RWE.⁷¹ Notably, NGS tests had a weaker evidence base than other technologies, confirming the perception that there is an undersupply of clinical utility data for this type of intervention.

There are also studies documenting the growing importance of RWD in the setting of outcomes-based contracting (OBC) for new technologies, primarily concerning drugs.^19,21,72 In addition, there are incentives in bundled care and value-based payment reimbursements mandated by federal legislation⁷³ that are based on outcomes best assessed using RWD. This emphasis on value-based payments is a critical factor reshaping how RWE is used for payer decision-making. The limitation is that the results of OBCs are rarely disclosed publicly. The only publicly available description of an OBC related to NGS is the example of Illumina working with Harvard Pilgrim to offer noninvasive prenatal testing to women at average risk while also committing to third-party evaluation of the impact of risk sharing on clinical outcomes and costs and publication of the results. The investigators demonstrated an increase in noninvasive prenatal testing use, modest increases in total testing and diagnostic expenditures, and a decrease in invasive procedures compared with the baseline year when testing was covered only for high-risk pregnancies.⁷⁴

Numerous data-sharing initiatives have been organized to overcome the data limitations inhibiting the assessment of the real-world impact of NGS. To date, many of these initiatives are specific to oncology, but there are a growing number of population and public health–focused efforts (Table 2).^45,75–96 These information networks have enabled greater use of RWD for various types of NGS evaluations, including studies of clinical and economic impact that have the potential to be useful to payers.^{45,47,48,76,77} Nevertheless, we were unable to find publicly available evidence that these study results have informed positive or negative coverage decisions to date, perhaps because of their relatively recent formation. Similarly, there are no published examples of coverage with evidence development for NGS, although this strategy is frequently described as a valuable option for payers and manufacturers when there is uncertainty regarding clinical utility.

Table 2.

Illustrative US-based RWD networks including NGS.

Cancer related	Population based
ASCO CancerLinq Flatiron Clinico-Genomic Database Foundation Medicine Precision Medicine Exchange Consortium ORIEN M2 Gen Syapse Learning Health Network Information Exchange and Data Transformation (INFORMED) Tempus iKnowMed Cota Healthcare and Horizon BCBS of New Jersey American Association for Cancer Research’s Genomics Evidence Neoplasia Information Exchange (GENIE)	All of Us Research Program* Million Veterans Program MyCode (Geisinger) Healthy Nevada Project (Renown Institute) Electronic Medical Records and Genomics Network (NHGRI) Healthcare Resources and Services Administration (HRSA) Regional Genetics Network Michigan DHHS Public Health Genomics Alabama Genomic Health Initiative (AGHI)

NGS indicates next-generation sequencing; RWD, real-world data.

^*NGS testing currently in planning phase.

A residual data barrier to developing RWE for NGS is the fact that genomic data are not represented in a structured format in the electronic health record (EHR), and test billing codes are too nonspecific to be able to rely solely on claims data.^97,98 To facilitate the use of RWD and overcome the need for manual curation, artificial intelligence–based methods such as natural language processing, machine learning, and deep learning are increasingly being applied to process and analyze unstructured data from the EHR and patient-generated data.⁹⁹ Critical to the success of these methods will be transparency regarding data sources and analytic methods so that payers will understand and trust the results. In addition, systematic analyses of NGS implementation efforts such as the IGNITE (Implementing Genomics in Pratice) network demonstrate that sustainability drivers include infrastructure (EHR, CDS, laboratories, manufacturers, community), evidence of clinical effectiveness, economic measures, workforce and workflow impact, provider and patient education, regulatory/legal updates specific to NGS, and stakeholder engagement in research. After a priority-setting exercise, the top 3 sustainability constructs were provider education, availability of genomic-focused CDS/EHR tools, and reimbursement.¹⁰⁰ Addressing these constructs is necessary to ensure the quality and availability of RWD and RWE.

Finally, there are several policy developments that are also endorsing greater use of RWE by stakeholders that influence payers. In 2017, the Center for Devices and Radiologic Health and the Center for Biologics Evaluation and Research at the FDA issued a guidance document describing the use of RWE to support regulatory decision making for medical devices.⁸ The goal is to incentivize the creation of a system for characterizing, aggregating, and analyzing data from all uses of medical devices so that innovative and accurate tests are made available to patients as efficiently as possible. Related efforts include FDA guidance on the use of public human genetic variant databases to support claims of clinical validity for genomic-based IVD and considerations for the development of evidence of analytic validity for NGS-based IVDs for suspected germline diseases.^9,43,101

The FDA has also helped to establish the Medical Device Innovation Consortium (MDIC) in 2012, a public-private partnership focused solely on advancing medical device regulatory science. The MDIC engages a wide variety of stakeholders, including representatives of the FDA, National Institutes of Health, CMS, industry, nonprofits, and patient organizations to improve medical technology development and review processes. The group recently convened FDA, industry, and payer representatives to build a framework to help IVD manufacturers develop credible evidence of analytical and clinical validity and clinical utility. The section on clinical utility specifically describes the use of RWD as a source of evidence that can be used to support regulatory and reimbursement decision-making.⁶ The FDA has also provided funding to establish the The National Evaluation System for health Technology Coordinating Center (https://nestcc.org/about/faqs/), a coordinating center for a voluntary network of data partners that act as a national evaluation system for devices. This network emphasizes the use of RWE in the evaluation of its test cases, which include an IVD test panel for lung cancer focused on developing evidence of clinical utility.^102,103 Although all of these efforts are primarily focused on improving regulatory and clinical decision making about device access and safety, the relevance to payer decision making is that RWE studies that meet robust methodological standards in the regulatory science arena should also prove useful to payers in as much as they include information about clinical utility.

Patient advocacy groups,¹⁰⁴ professional societies,¹⁰⁵ and nonprofit health and research agencies^106,107 have all become engaged in developing more comprehensive and accurate RWD to support decision makers, including patients and families. The expectation is that over time, patients will become key drivers of use of RWE by payers, particularly as there is greater availability of patient-generated data as part of clinical care.

The scoping review revealed a number of evidence gaps for use of RWE in coverage decision-making for NGS. First, most studies are in oncology, PGx, and diagnosis of suspected genetic disorders in the perinatal or pediatric period, rather than the full spectrum of clinical genomic applications. Second, a wide variety of methods are used to conduct RWE studies, and we could find no examples in which investigators cited adherence to methodologic guidance documents or best practices for RWE studies. Third, published analyses of payer coverage policies do not distinguish RWE from RCT data when evaluating the relationship between clinical studies and coverage determinations. Finally, it is difficult to describe how RWE specifically can inform payer decision making based on publicly available data without an in-depth review of coverage policies and their evolution over time.

Potential Solutions

We developed potential solutions (Table 3) for overcoming the challenges limiting payer uptake of RWE in the setting of NGS based on our team’s experience and findings from the scoping review. The suggestions spanned a spectrum, from the importance of following RWE methodological best practices, to building transparency and relevance of study methods, to collaborating with existing expert groups addressing issues of data standardization, data sharing, and assessment of value.

Table 3.

Potential solutions for advancing use of RWE.

Area of focus	Challenges	Opportunities	Potential solutions
I. Relevance and rigor of RWE for payers	Payers view many RWE studies as lacking relevance and timeliness for coverage or utilization management decisions. RWE study methods not perceived as sufficiently rigorous and transparent by payers.	Payers are willing to engage with test developers and researchers in study planning activities but need to know how to discern clinically valid tests among avalanche of new NGS tests. Multiple publicly available best practices/recommendations for designing, conducting, and reporting RWE studies. Several groups have published RWE evaluation tools specifically for payers.	1Require test developers to show evidence of analytic and clinical validity, as well as clinical flowchart of how new NGS test is hypothesized to affect health outcomes. 2Develop transparent engagement processes to ensure payer information needs are reflected in study design decisions. 3Convene multistakeholder panels to tailor existing best practices and evaluation tools to NGS.
II. Incentives for RWE development	Limited number of studies outside of oncology, PGx, and perinatal/pediatrics examples of RWE directly affecting coverage decisions for NGS. RWE to demonstrate clinical utility of NGS is undersupplied by the market.	Numerous groups building data infrastructure to support RWE studies, particularly in oncology but also in population-based data networks and learning healthcare systems. FDA is supporting the use of RWE to make regulatory decisions for in vitro diagnostics. Federal efforts to require data standardization and interoperability such as HITECH and value-based payments such as MACRA* should also facilitate access to RWD. There are a growing number of learning healthcare systems focused on genomics and publicly subsidized data networks that can reduce research study costs.	4Encourage industry, federal, and nonprofit funding to conduct and publish RWE studies to demonstrate the relative benefits and harms of NGS testing for patient and consumer subgroups in existing data networks. 5Support FDA efforts such as MDIC by providing use cases to demonstrate how RWE can inform regulatory and payer decision making for NGS. 6Include genomic data in meaningful use requirements. 7Conduct RWE studies in networks such as Intermountain, Geisinger, Innovation Health, Sanford, University of Vermont, and PCORnet.
III. Educational needs regarding both RWE and NGS	Some payers lack experience and expertise to use RWE study results in their local context. Some payers lack knowledge on how to evaluate clinical utility of NGS tests. Some clinicians lack knowledge about when to order tests and how to interpret results.	Investments in RWE methods training and professional hiring are growing rapidly in both public and private sectors. Industry, nonprofit, and governmental organizations are investing in genomics literacy and NGS education.	8Provide training in observational study methods and pragmatic clinical trials for payers and other stakeholders on NGS. 9Encourage both just-in-time training based on adult learning principles and in-person training at professional and industry meetings.
IV. Standards for testing and reporting NGS data	Results of NGS tests are complex, and laboratory procedures for test validation and reporting lack transparency. The same test performed by different labs can produce different results.	Federal and private sector groups (eg, professional societies such as CAP AMP, ACMG†) are addressing these problems. Data sharing is required for NIH grants including genomics and promoted by nonprofits such as Global Alliance for Genomics and Health.	10Labs that follow published guidelines or consensus recommendation statements for NGS assay and bioinformatics pipeline validation could receive higher reimbursement rates to incentivize data sharing. 11Support ClinGen efforts to define the clinical relevance of genes and variants for use in precision medicine and research. 12Support federal efforts to standardize procedures for NGS reanalysis and variant reinterpretation and require clinical labs to have policies and protocols in place to support reporting any reclassifications that may affect clinical management.
V. Standards for genomic data representation in EHR	Genomic data integration in the EHR is not a top priority for major vendors, yet solutions require their leadership. Genetic data are not aggregated and stored in a structured manner accessible to researchers and clinicians; EHRs lack comprehensive clinical data and patient-generated data.	Innovative NGS companies and federally funded networks such as eMERGE are developing standardized protocols for genomic data integration across multiple EHR systems. Nonprofits have developed data exchange standards such as FHIR (Fast Healthcare Interoperability Resources) that better support CDS and unify how genomic variant data are accessed. Artificial intelligence (ie, natural language processing, machine learning, deep learning)-based methods reduce the need for manual curation and enable integration with other digital data sources.	13Evaluate clinic-genomic interoperable applications using an implementation science framework. 14Recognize that most payers have limited understanding of these methods and will require educational efforts to trust and apply RWE study results developed with cutting-edge methods.
VI. Standards for NGS evidence review by payers	Payer evidence requirements for NGS coverage decision making not clearly communicated to or understood by test developers.	FDA-CMS parallel review process and MolDx program (CED) is an opportunity to develop necessary evidence for federal payers. Stakeholders worked through nonprofits such as Center for Medical Technology Policy to recommend coverage for NGS tumor profiling based on number of genes.	15Encourage public and private payer support of coverage with evidence development for tests with promising evidence of clinical utility. 16Convene a multistakeholder group including payers to define methodological requirements for demonstration of clinical utility for specific clinical contexts (eg, PGx, rare disease, cancer, etc).
VII. Partnerships	No single stakeholder has the resources to develop clinical utility data independently.	Numerous examples of both public and private NGS implementation and data-sharing partnerships, including learning healthcare systems partnering with industry.	17Engage stakeholders in research question priority setting and study design. 18Conduct clinical utility studies in these settings and publish results.
VIII. Role of RWD in support of OBCs for NGS	OBC implementation is limited by data barriers and lack of outcome measures. Most OBCs are not publicly disclosed, so there is no shared learning.	Payers are demonstrating increased receptivity to, and use of, OBCs for new and/or expensive interventions. Precompetitive collaboration to develop an OBC implementation framework has been identified as a critical unmet need. Companies are beginning to share results of NGS-related OBCs.	19Support risk-based market access and inform evidence claims for NGS by developing an NGS-specific framework for evaluating and implementing OBCs. 20Work with stakeholders to develop NGS-specific outcome measures.
IX. Narrow definition of how value of NGS is measured	RWE necessary for value-based frameworks and payment models, but the patient and other perspectives (personal utility, reduction in uncertainty) are often missing.	Numerous multistakeholder groups have developed approaches for customizing value assessments for payers, patients, and other decision makers. Growing evidence that patients may become key drivers of RWE use by payers.	21Build on work by ISPOR‡ and other non-profits to demonstrate how NGS aligns with expanded definition of value. 22Track developments in value-based payment models as critical factors influencing how RWE is being used by payers. 23Promote inclusion of patient-reported outcomes in value assessments.

CDS indicates clinical decision support; CMS, Centers for Medicare & Medicaid Services; EHR, electronic health record; FDA, US Food and Drug Administration; MDIC, Medical Device Innovation Consortium; NGS, next-generation sequencing; OBC, outcomes-based contracting; PGx, pharmacogenetic testing; RWD, real-world data; RWE, real-world evidence.

^*Health Information Technology for Economic and Clinical Health, Medicare Access and CHIP Reauthorization Act of 2015.

^†College of American Pathology, Association for Molecular Pathology, American College of Genetics and Genomics.

^‡International Society for Pharmacoeconomics and Outcomes Research.

The first 3 solutions address the need to develop processes to promote the relevance and rigor of RWE for payers. Payers must be able to trust study findings and understand how to apply the results in coverage decision-making. Efforts at payer engagement and tailoring existing RWE best practices and evaluation tools to NGS are best pursued as part of multistakeholder initiatives that are already working to advance use of RWE with payers generally. The next 4 potential solutions (4–7) target the lack of incentives for test developers to conduct RWE studies and undersupply of published research. There is an increasing number of curated data networks and integrated healthcare delivery systems focused on genomics that can facilitate robust RWE studies and initiatives supported by the federal government to encourage greater use of RWE in policy decisions, including coverage. Progress could be accelerated if stakeholders advocated for including genomic data in meaningful use requirements. The next 2 solutions (8 and 9) tackle the widespread problem that payers lack the requisite skills to review and apply RWE studies in their local context but through the lens of NGS-related decisions.

There is widespread recognition that a lack of standards is hampering development of clinical utility; these include standards for testing and reporting NGS data, representing clinic-genomic data in the HER and payer requirements for NGS evidence review. There are 7 potential solutions (10–16) that address these interrelated gaps; however, they vary in the level of effort that will be required to advance these tactics. Until these issues are solved in a scalable way, it will be very difficult for researchers and test developers to capitalize on RWD sources. These solutions also address gaps in capturing the necessary clinical, digital, and patient-reported data to conduct RWE studies by focusing on artificial intelligence–based methods to reduce the need for manual curation.

The need for public–private partnerships is addressed by the next 2 solutions (17 and 18), as pooling of infrastructure resources, patient populations, and expertise will be required if the clinical utility of NGS will be demonstrated for genomic conditions, many of which are relatively rare. The next 2 potential solutions (19 and 20) take on the need to overcome barriers to using RWD to support OBCs for NGS, recognizing that although all of the obstacles are not specific to NGS, the remedies should be targeted to how OBCs can be structured and evaluated for NGS specifically. Finally, the last 3 solutions (21–23) focus on the importance of including the patient and other relevant perspectives in value-based frameworks focused on NGS. It is also important to note that for most of these proposed solutions, the policy and data environments are in flux; therefore, each effort will need to be evaluated and recalibrated in response to changing trends.

Discussion

To our knowledge, this is the first article to directly examine payer use of RWE for NGS and what new approaches are needed. Although the topic of RWE as a valuable source of information for clinical, regulatory, and payer decision makers has been addressed by numerous authors and policy makers,^{7,8,12–15,30,40,104,105,108–116} we explored whether and how RWE could play a similar role for NGS specifically in the context of coverage decisions. In the scoping review, we identified numerous examples of how RWE was likely used in oncology, PGx, and pediatric or perinatal settings to bridge the existing evidence gaps for payers, as there is concurrent evidence of increasingly positive coverage decisions for NGS tests in these clinical contexts. These applications included developing empiric evidence of current test and treatment use patterns, assessing real-world cost implications, demonstrating the incremental value of testing additional genes when compared with standard-of-care single-gene tests, supplementing RCT data, creating efficiencies and greater certainty in coverage policy development, and enabling OBC.

Nevertheless, it appears that payers still primarily rely on clinical guidelines and RCTs as opposed to RWE as the type of evidence cited as justification for their decisions. The reasons for this disconnect are multifactorial, including challenges related to RWD quality and comprehensiveness, difficulties representing genomic data in a standardized manner in the EHR, and lack of payer engagement in study development, resulting in results that are not relevant for coverage decision-making. Payers also lack familiarity with RWE study methods and continue to prefer RCT data over observational data. They also have concerns about the lack of transparency regarding data sources and analytic methods, resulting in a lack of trust, particularly regarding studies conducted by industry. Generally, payers are not sure how to use RWE within their current coverage processes and do not fully understand all the relevant questions that RWE could answer. Nevertheless, there is a confluence of policy, technology, and data infrastructure trends that are likely to enable greater use of RWE for NGS, for example, greater receptivity to RWE by the FDA, artificial intelligence–based methods to facilitate data analyses, and a proliferation of data networks built and curated to support RWD studies involving the use of NGS in people with and without known disease.

The data and infrastructure-related barriers to conducting RWE studies for NGS are being addressed by healthcare providers, federal research funders, and private companies. Currently, there is a growing number of learning healthcare systems focused on implementing and evaluating population-based NGS and numerous precision medicine programs focused on oncology.^45,75–82 Each of these organizations has a vested interest in developing the data infrastructure to support collection of high-quality RWD and to publicize the results of RWE studies. By engaging payers in the design of these studies, there is a real opportunity to provide RWE that can inform coverage decision making in an efficient and timely manner. Nevertheless, to ensure that these studies have the desired impact, researchers must follow methodologic best practices for conducting and reporting RWD studies. Fortunately, there are many existing consensus statements from expert groups to guide researcher efforts and reassure payers that the study results are valid. What is missing is agreement about which guidelines are best to follow for NGS. Similarly, existing tools for evaluating the quality and applicability of RWE need to be tailored to NGS for payers to more consistently evaluate these studies. Both efforts will require multistakeholder groups to discuss options, explicitly address conflicts of interest, and agree on a path forward for using RWE to inform payers and other decision makers about the clinical utility of NGS. Although conflicts of interest are ubiquitous, agreeing how these conflicts will be identified and managed is possible, as demonstrated by groups such as the MDIC, a public–private partnership working to promote patient access to innovative medical technologies such as NGS.

Lack of standards is a major stumbling block for using RWE to evaluate the clinical utility of NGS. This refers not only to a lack of standards for analyzing and reporting NGS results, and a lack of standards for representing genomic data in the EHR, but also to a lack of standards for payer assessment of clinical utility data for NGS. Although individual labs or health systems may attempt to reduce variation in their NGS testing practices or NGS-related CDS tools, the definitive solutions require multistakeholder groups to ensure interoperability and efficiency as patients move from one clinical setting to another. Similarly, multistakeholder groups need to define the methodological standards for developing clinical utility evidence for NGS that will address payer requirements for coverage decision-making. Although this has been successfully done for molecular diagnostics in oncology,¹¹⁷ RWE has the potential to address the lack of evidence of clinical utility of NGS, provided that payers agree on how RWE can be used for coverage determinations. Although affirmative coverage decisions are necessary, they are not sufficient to ensure clinical uptake of NGS unless there is simultaneous attention to NGS implementation requirements.

There are also reimbursement-focused enablers that will require access to robust RWD, such as the growing use of OBCs between manufacturers and payers to ensure realization of promised benefits of new interventions. Nevertheless, there are significant barriers to implementing OBCs, and many preliminary discussions fail to result in actual contracts. These challenges include difficulties obtaining accurate data, lack of outcome measures, concerns regarding patient data privacy, and costs of data collection. There is an opportunity for an honest broker to convene payers and manufacturers to develop a framework to assess the suitability/desirability of OBCs for NGS and also guide the negotiation and implementation processes to increase the likelihood of success. Greater transparency regarding the results of OBCs is desirable, but the proprietary nature of contract terms makes this goal unlikely.

Nonprofit groups such as the Personalized Medicine Coalition, the Innovation and Value Initiative, and patient advocacy groups are highlighting the importance of putting the patient perspective at the center of any assessment of the value of new technologies such as NGS. Development of patient-centered outcome measures would advance these efforts and address the lack of standard outcome measures limiting RWE studies and OBCs. Because each group is engaging relevant stakeholders to develop suggestions or consensus statements for a path forward, we recommend following a similar approach for NGS-specific adaptation or evaluating the feasibility of joint efforts.

There are several limitations to this study. There is no Medical Subject Headings term for either RWD or RWE¹¹⁸; therefore, we may have missed relevant articles. We made several efforts to overcome this limitation by searching reference lists and asking experts to share seminal articles. We did not examine specific payer coverage policies, so our conclusions regarding the impact of RWE on payer decision making is likely an underestimate of the true impact. We also limited our evaluation to US payers and data networks. We are aware that there are comparable payer information needs and data initiatives in other countries but determined these to be out of scope for our literature and policy analyses. Future studies could characterize the RWE landscape in other countries and determine whether our recommendations could be generalized.

This study assessed the opportunities and challenges surrounding use of RWE by payers to inform decision making regarding NGS and found a growing number of published RWE studies, particularly in oncology, PGx, and perinatal or pediatric genomic testing, that have played either a direct or supportive role given the state of published coverage analyses. We also identified many examples of policy and data network enablers designed to address the current undersupply of RWE for NGS. For RWE to become a game changer for payers, there must be multifaceted efforts to raise awareness of data and methods advances and publicly available examples of robust studies that are relevant for payer decision making. We present 20 potential solutions that, if pursued collaboratively with stakeholders over time, represent promising steps toward ensuring greater and more effective use of RWE by payers.