Weighing the Evidence: Variant Classification and Interpretation in Precision Oncology, US Food and Drug Administration Public Workshop—Workshop Proceedings

Abstract

PURPOSE

Next-generation sequencing (NGS) oncology panels are becoming integral in hospital and academic settings to guide patient treatment and enrollment in clinical trials. Although NGS technologies have revolutionized decision-making for cancer therapeutics, physicians may face many challenges in parsing and prioritizing NGS-based test results to determine the best course of treatment for individual patients. On January 29, 2018, the US Food and Drug Administration held a public workshop entitled, “Weighing the Evidence: Variant Classification and Interpretation in Precision Oncology.” Here, we discuss the presentations and discussion highlights across the four sessions of the workshop.

METHODS

The goal of the public workshop was to engage stakeholders and solicit input from experts in precision oncology to discuss the integration of complex NGS data into patient management and regulatory innovation within the precision oncology community. The US Food and Drug Administration gathered representatives from academia, industry, patient advocacy, government, and professional organizations for a series of presentations followed by panel discussions. After the workshop, the transcript and speaker presentation slides were reviewed and summarized for manuscript preparation.

RESULTS

Speakers and panelists provided diverse perspectives on the integration of NGS technology into patient care for oncology and on the complexities that surround data interpretation and sharing. Discussions highlighted the challenges with standardization for variant classification while expressing the utility of consensus recommendations among stakeholders in oncology for driving innovation in the era of precision medicine.

CONCLUSION

As precision medicine advances, clear communication within the field of precision oncology will be key to creating an environment that facilitates the generation and sharing of data that have value to patients.

Context

• Key Objective

  To provide an overview of a public workshop hosted by the US Food and Drug Administration that discussed key concepts and considerations for the classification and interpretation of genetic variants in precision oncology.

• Knowledge Generated

  Key highlights and findings from the discussion include the conclusion that as the use of next-generation sequencing oncopanels in the clinical setting increases, it is critical to develop a framework that ensures accurate and consistent interpretation of genetic variants. In addition, transparency in data sourcing and reporting methodologies, as well as frequent updating of variant interpretations, are necessary to advance science and appropriately care for patients.

• Relevance

  As precision medicine advances, clear communication among stakeholders in oncology will be key to creating an environment that facilitates generating and sharing data that have value to patients.

INTRODUCTION

The goal of precision oncology is to use the genetic information—germline or somatic—from a patient with cancer to help determine which drug(s) might be most effective in treating his or her disease. Next-generation sequencing (NGS) is increasingly used in precision oncology because of its ability to identify many mutations simultaneously, thereby maximizing the amount of information obtained from a single test.¹ The global endeavor to identify somatic mutations within a patient’s tumor genome has led to an unprecedented amount of genomic information, with varying degrees of associated clinical evidence. NGS can report a vast number of mutations which may lead to clinician uncertainty in the interpretation and prioritization of mutations with respect to the clinical significance and optimal course of treatment.²

In January 2017, the Association for Molecular Pathology (AMP), ASCO, and the College of American Pathologists (CAP) published a joint consensus recommendation for standards and guidelines for the interpretation and reporting of sequence variants in cancer³; however, stakeholders have not consistently implemented these recommendations. On November 15, 2017, MSK-IMPACT, an NGS laboratory-developed test (LDT), was authorized by the US Food and Drug Administration (FDA) as a tumor profiling assay providing large-scale clinical sequencing for patients at Memorial Sloan Kettering Cancer Center (MSKCC).^4,5 Shortly thereafter, on November 30, 2017, the FDA approved the FoundationOne CDx assay from Foundation Medicine (FMI) as a companion diagnostic test identifying patients with any of five tumor types that may benefit from 15 different FDA-approved targeted treatement options, in addition to providing tumor profiling data.^6,7 With FDA approval and authorization of these two NGS-based pan-tumor oncology panels (oncopanels), the agency was in a unique position to foster a dialogue among key stakeholders about this innovative technology.

On January 29, 2018, the FDA held a public workshop to engage stakeholders and solicit input from experts in precision oncology and patient representatives to discuss how genetic sequencing data could be best implemented in patient management. This was done with the goal of advancing innovative regulatory strategies to support the development of safe and effective precision-based drugs and devices for marketing. The workshop attracted attendees from diverse backgrounds, including academia, patient advocacy groups, industry, the pharmaceutical industry, and government (Fig 1). A brief overview of the workshop is provided, with presentations and panel discussions divided into four sessions as outlined in Table 1.

FIG 1.

Public workshop attendee composition. The chart depicts the self-reported affiliations of the 182 workshop in-person attendees. Data for the 203 online attendees were not available as registration was not required to view the Web cast. IVD, In Vitro Diagnostics.

TABLE 1.

Presentations at the “Weighing the Evidence: Variant Classification and Interpretation in Precision Oncology” Public Workshop

MEETING SUMMARY

The workshop began with an overview of the FDA’s current regulatory strategy for reviewing tumor-profiling NGS tests in oncology and the role of the FDA Oncology Center of Excellence in fostering continuity across the agency to advance the development and regulation of oncology products for patients with cancer. To set the stage for the workshop discussion, Dr Philip outlined the FDA’s new regulatory pathway for NGS tumor-profiling tests, as summarized in the FDA’s Center for Devices and Radiologic Health fact sheet published online on November 15, 2017.⁸ As described by Dr Philip, the agency’s three-tiered approach for reporting biomarkers in tumor-profiling NGS tests includes the following: level 1, companion diagnostic claims; level 2, cancer mutations with evidence of clinical significance; and level 3, cancer mutations with potential clinical significance. Companion diagnostics are tests that provide information essential for the safe and effective use of a corresponding therapeutic product. Tumor-profiling NGS tests may include companion diagnostic claims (level 1) that are prescriptive for a specific therapy and are supported by the clinical validity of the test for each biomarker reported. On the contrary, for level 2 biomarkers, clinical validity is established in professional guidelines but not demonstrated with the test. For level 3 biomarkers, clinical validity has not been demonstrated either in professional guidelines or with the test, but is suggestive on the basis of clinical/biologic evidence.

Session 1: Overview of the State of Science for Sequence Variant Classification in Oncology and Its Practical Use in Treating Patients

Use of an individual’s mutation profile for clinical decision making is driving the ever-expanding number of biomarker-based clinical trials and clinical regimens in precision oncology. The result of this intersection of technology and translational medicine has been a deluge of reported mutations with various levels of supportive clinical evidence. Panelists discussed the importance of offering broad oncopanel clinical sequencing to optimize treatment decisions for patients, particularly those with metastatic and refractory cancer. Oncopanels should be able to detect many different clinically relevant point mutations, copy number variants, and structural rearrangements. Large oncopanels—that is, those that include 300 or more genes—offer the opportunity to analyze genomic features and mutation signatures, such as microsatellite instability and tumor mutational burden. Additional discussion centered around the unique benefits and considerations of sequencing matched tumor tissue and blood specimens from patients. Michael Berger, PhD, noted that sequencing matched tumor and normal DNA allows one to query somatic mutations in a patient's tumor, as well as provide insight abougt inherited germline variants and mutations associated with clonal hematopoiesis; MSKCC currently reports germline variants associated with cancer predisposition, eg, pathogenic and likely pathogenic variants. Vincent Miller, MD, indicated that FMI filters out germline mutations but further noted that when reporting germline mutations, laboratories need to have an operation in place, or at least have an understanding with the clinicians, so that the information can be followed up with genetic counseling.

Comprehensive variant annotation is necessary to inform treatment selection and match patients to clinical trials. The number of patients who are matched to either clinical trials or personalized treatments on the basis of their genomic data were reported by panelists to be between 10% and 37%. The success of matching patients to treatments relies on comprehensive databases and inclusive or germline and/or germline mutations, with meaningful clinical information curated with clinical outcome. Different databases are used to annotate test results and capture the clinical effects and clinical significance of somatic mutations found in patients. Panelists discussed the implementation of comprehensive pan-tumor testing and challenges with the maintenance of a growing, evolving dynamic database. Drs Berger and Miller indicated that it takes a large team of individuals to generate a curated database. Bioinformaticians confirm the variants and evaluate the quality of the data, assessing for potential artifactual effects using metrics and an overlay of algorithms. To compile and interpret the evidence, scientists and pathologists continually gather new information from guidelines, published literature, public databases, clinical trials, FDA-approved therapeutic labeling, and other sources. This comprehensive information is then used to sort biomarkers, either manually or in combination with bioinformatics software algorithms, on the basis of levels of evidence.

Panelists concluded that a key to the success of somatic genotyping will be developing statistical tools and bioinformatics infrastructure to identify clinically meaningful mutations. Dr Berger noted that as more sequencing data accumulates, integrated analyses of big data will allow for the identification of novel hotspot mutations occurring at frequencies greater than would be expected by chance. This approach would allow for these novel mutations to be directly tested in targeted clinical trials to assess drug sensitivity. Panelists noted that the field will continue to evolve and become more challenging with the increase in tissue-agnostic trials in which patients are enrolled on the basis of biomarker status, irrespective of the tumor type, which could, in turn, lead to an increase in tissue-agnostic drug indications. John Deeken, MD, also highlighted some of the unique challenges faced when running an in-house 50-gene oncopanel at a community-based hospital system. These challenges included the cost of developing and supporting an oncopanel test and the lack of reimbursement for the majority of NGS testing performed. Donna Roscoe, PhD, noted that the FDA is striving to ensure that there are validated tests on the market that will allow for dynamic use within clinical settings to enable clinicians to optimize patient decisions in oncology and other therapeutic areas.

Session 2: Levels of Evidence Required for Reporting Variants and Guiding Patient Treatment

Widespread implementation of NGS data in clinical settings has been accompanied by challenges in standardization, interpretation, and reporting of genetic tests. There are several efforts to establish standardized consensus concerning annotation, interpretation, and reporting somatic variants. Shashikant Kulkarni, PhD, provided an overview of the proposed guidelines published by a multidisciplinary working group convened by AMP, ASCO, and CAP for variant interpretation and reporting.³ The four-tiered system to categorize somatic sequence variations on the basis of their levels of evidence are as follows: tier I, variants with strong clinical significance; tier II, variants with potential clinical significance; tier III, variants of unknown clinical significance; and tier IV, variants deemed benign or likely benign.³ Guidelines were developed to provide a general framework, on the basis of an evidence-based variant classification approach, for the standardized interpretation and reporting of somatic variants in cancer. For example, variants with strong clinical evidence include those with level A evidence (FDA-approved therapies or included in professional guidelines) or level B evidence (well-powered studies with consensus from experts in the field). Variants with potential clinical evidence are those with level C evidence (FDA-approved therapies for different tumor types or investigational therapies of multiple small published studies with some consensus) or level D evidence (preclinical trials or found in a few case reports without consensus).

In general, it is widely accepted that tests should be optimized to ensure coverage across targets and high sensitivity for detecting low-frequency mutations; however, as noted in the consensus manuscript,³ the field is split on the use of variant allele frequency (VAF) in reporting and clinical practice. Whereas some panelists acknowledged that the inclusion of VAF in patient reports is useful information, other panelists did not place value on this information for patient management. One perspective, provided from a workshop attendee, was that if frequencies are too low, the detected mutations could represent subclonal events and may not necessarily be appropriate therapeutic targets. On the contrary, Apostolia-Maria Tsimberidou, MD, PhD, noted that, as a clinical investigator, there is utility in having allele fraction present in the patient report as the data could be used to interpret clinical outcomes, particularly because patients have multiple molecular alterations. Similarly, Howard McLeod, PharmD, acknowledged the caveats with using allele fraction in patient management—that there is still much work to be done to determine the best way to implement the allele fraction on identified mutations into patient care. Dr McCleod also highlighted the potential clinical benefit of assessing VAF in liquid biopsy samples in which one can monitor trends in the allele frequency of mutations detected, with the goal of identifying potential emerging drug-resistant clones and use this information to determine when to intervene. A consensus was not reached by workshop panelists or attendees regarding the clinical use or validity of reporting VAF in precision oncology.

Standardizing the annotation and reporting of somatic mutations from NGS data remains a challenge. John Pfeifer, MD, PhD, pointed out that there is a gray zone between where the technical portion of the NGS test ends and the practice of medicine begins. Whereas standardizing interpretation and reporting is important, it should not supersede the practice of medicine. Although many of the speakers and panelists used the AMP/ASCO/CAP guidelines, several groups used a slightly modified tiering system. Panelists generally agreed that treatment options for tier II and below, as noted in the AMP/ASCO/CAP guidelines, should be considered only once tier I treatment options have been exhausted. Moreover, panelists expressed the need to create a database—that is, a knowledge base—that can assist in resolving discordances in variant interpretation and allow for the generation of community standards.

Session 3: Best Practices for Use of Public/Private Databases for Variant Classification and Interpretation in Oncology

To positively affect patient care, mutations detected using NGS technology must be appropriately interpreted and classified. There are multiple public and private databases available to aid in variant classification and interpretation, and each of these has its own advantages and disadvantages. The panel unanimously agreed that a known drawback to using publicly available databases for variant classification is that interpretation is not always consistent. Heidi Rehm, PhD, noted that the quality of the data rests in the hands of those who deposit the information into the database and that amassing sufficient information to provide high-quality classification data requires global sharing. Kenna Shaw, PhD, affirmed that standards and rules are needed to not only interpret but reinterpret variant classification, and she summarized the efforts being conducted at her institution to amass a personalized cancer therapy knowledge base. Dr Shaw provided an overview of AACR Project GENIE (Genomics Evidence Neoplasia Information Exchange), which presently includes 17 institutions across the world that are working together to develop a regulatory-grade registry that aggregates and links cancer genomic data with clinical outcomes from tens of thousands of cancer patients. Project GENIE will allow for sharing of the accumulated data with the global research community through cBioPortal and the Synapse Platform with the noted caveat that the underlying functional evidence int the knowledge base will continue to change over time.

Panelists agreed that databases should be transparent about data sources, methods, and rules used for reporting mutation-associated clinical claims. Transparency in reporting allows users to better understand how classification decisions were made. Karla Bowles, PhD, noted that subjective tools, such as literature reviews, population data, and structural analysis, should be used with caution, as interpretations from these data are often sources of discrepancies. Although continuous updating of variant classifications is challenging, panelists agreed that it is necessary to provide high-quality patient care and advance science. To this end, Ben Park, MD, PhD, encouraged the use of an experienced molecular tumor board and highlighted the utility of expert interpretation and curation to help elucidate actionable targets from NGS data.

Session 4: Future Directions For Data Sharing, Standardization, and Establishing Consistency in Precision Oncology

Many mutations identified by genomic tests for oncology are novel, rare, or otherwise lack conclusive evidence to support a clinical interpretation. The number of these mutations will only increase as additional genes are added to oncopanels and the costs of sequencing decrease. Understanding the clinical utility of these mutations requires addressing barriers that prevent accurate classification. Dane Dickson, MD, cited improved data sharing as a critical need and described the utility of high-quality databases that include both diagnostic results and clinical outcomes.

Dennis Dean, PhD, described the need for interoperability of complex systems and tools to enable analysis of large data sets. Interoperability requires standards and languages to support reproducibility. Dean discussed Rabix, a suite of open-source development tools for creating and running computation workflows, as a development environment that uses Common Workflow Language to promote the analysis of combined large data sets. The ability to aggregate and analyze existing data can lead to new insights, such as the dependency of clinical outcomes on race. Robert Grossman, PhD, emphasized the need for data commons—that is, a unified data system that promotes sharing—such as the National Cancer Institute Genomic Data Commons, that use data models and metadata services to harmonize and share cancer genomic datasets. This will allow for data analysis and reanalysis, as necessary. Grossman discussed the concept of leveraging funders, journal editors, and payers to incentivize the data sharing required to build a data commons.

As a patient advocate, Len Lichtenfeld, MD, discussed the challenges of translating the information generated by large-scale projects to communities that lack the resources and infrastructure to access the latest information necessary to improve care. How to provide access to all patients, regardless of where they live, should therefore become part of the conversation. Panelists agreed that stakeholders should work together to create an environment that facilitates data sharing and the generation of accessible data with value to patients.

In conclusion, precision oncology is a rapidly evolving field. With more oncopanels being developed, and as more genomic and patient outcome data accumulate, physicians are faced with new challenges in interpreting NGS-based test results to determine the best treatment course for each patient. For example, the emergence of novel data and conflicting results can create challenges in interpreting actionable genetic variants. Developing a framework that ensures accurate and consistent interpretation of genetic variants is critical as the use of NGS oncopanels in the clinical setting increases. Published guidelines for somatic variant interpretation can standardize and facilitate reporting data from these panels. In addition, there are multiple public and private databases available that aid in variant classification and interpretation; however, methods used to classify variants and how these databases align remain unclear. Transparency in data sourcing and reporting methodologies as well as frequent updating of variant interpretations are necessary to advance science and appropriately care for patients. Although large-scale sharing of data can be challenging, efforts to create or leverage large databases of genomic and clinical data should be supported and prioritized. To that end, the FDA recently finalized a guidance document entitled, “Use of Public Human Genetic Variant Databases to Support Clinical Validity for Genetic and Genomic-Based In Vitro Diagnostics.” This document describes an approach where test developers may rely on clinical evidence from FDA-recognized public databases to support the clinical validity of genotype–phenotype relationships and help provide assurance of the accurate clinical evaluation of genomic test results.^9,10 As precision medicine advances, clear communication among stakeholders in oncology will be key to creating an environment that facilitates the generation and sharing of data that have value to patients.

A complete transcript and Web cast of the workshop presentations can be viewed at: https://www.fda.gov/MedicalDevices/NewsEvents/WorkshopsConferences/ucm570822.htm.¹¹

AUTHOR CONTRIBUTIONS

Conception and design: Hisani N. Horne, Donna Roscoe, E. David Litwack, Anand Pathak, Jai P. Pandey, Pamela S. Gallagher, Robert N. Schuck, Eunice Y. Lee, Yun-Fu Hu, Julia A. Beaver, Gideon M. Blumenthal, Reena Philip

Collection and assembly of data: Hisani N. Horne, Donna Roscoe, Anand Pathak, Laura M. Koontz, Robert N. Schuck, Yun-Fu Hu, Julia A. Beaver, Gideon M. Blumenthal, Reena Philip

Data analysis and interpretation: Hisani N. Horne, Donna Roscoe, Anand Pathak, Laura M. Koontz, Robert N. Schuck, Yun-Fu Hu, Gideon M. Blumenthal, Reena Philip

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/po/authorcenter.

Hisani N. Horne

Employment: AstraZeneca

Stock and Other Ownership Interests: AstraZeneca

E. David Litwack

Employment: Prevail Therapeutics

Travel, Accommodations, Expenses: Prevail Therapeutics

No other potential conflicts of interest were reported.