Future of Evidence Synthesis in Precision Oncology: Between Systematic Reviews and Biocuration

Precision medicine refers to the tailoring of interventions to patients using approaches that go beyond traditional clinical characteristics (eg, age, sex, disease, symptoms, and medical history) by considering biomarkers consisting of genetic characteristics or molecular profiles.¹ In particular, precision oncology (PO)² includes both the development of novel cancer therapies targeting specific changes that occur in some individuals but not others (eg, inherited mutations or somatic mutations that arise in the carcinogenesis process) and the stratification of individuals according to existing interventions (eg, via screening programs tiered by genetic risk or chemotherapies predicted to work for certain molecular profiles), which is expected to show improved outcomes in the resulting subgroups. Herein, we discuss two approaches for synthesizing evidence for or against the use of specific PO interventions, namely, systematic reviews (SRs) and biocuration, and argue that their engagement with each other could facilitate the timely delivery of appropriate PO interventions.

Cancer is known to be highly heterogeneous, and clinical responses differ among patients. PO thus includes a wide variety of applications, including both the use of targeted therapies against particular changes occurring in individual tumors—such imatinib against the BCR-ABL  fusion in chronic myeloid leukemia,³ vemurafenib⁴ and dabrafenib⁵ against specific BRAF mutations in melanoma, and trastuzumab against HER2-postive breast cancer⁶—and the assignment between existing therapy classes via biomarkers that are not direct drug targets. This can be based either on the activation of certain pathways, as in the case of EGFR inhibitors in KRAS-wild type colon cancer, ^7,8 or on gene expression signatures or other molecular characteristics (ie, Oncotype DX test for breast cancer, used to predict recurrence and likely benefit of chemotherapy).^9,10 These biomarkers are well-established and have been approved by the US Food and Drug Administration for specific cancer types.

The biomarkers discussed in the previous paragraph are classified as having the highest level of evidence for clinical use in society guidelines (eg, those jointly put forward by the Association for Molecular Pathology, ASCO, and the College of American Pathologists).¹¹ To be meaningful for clinical decision making, biomarkers should have high predictive performance (ie, they can be used to stratify patients into treatment groups with differential outcomes). Most research to date, however, has assessed whether biomarkers are prognostic (ie, are associated with a clinical outcome for untreated or standard-of-care patients).¹² It is often the case that the supporting literature includes few if any randomized control trials.¹³ As a result, lower levels of evidence may be assigned, eg, to biomarkers that predict response to a treatment based on well-powered studies evaluated by expert consensus or that predict response in a different tumor type than that being studied.

Many conceptual frameworks have been proposed to facilitate clinical decision making by helping end users (especially patients and clinicians) assess the evidence for or against the use of medical tests, including biomarkers.^14,15 With minor variations, the different frameworks involve a stepwise approach that starts with analytic validity (laboratory reproducibility), proceeds to clinical validity (predictive accuracy, including sensitivity, specificity, and positive and negative predictive values) and clinical utility (use of the biomarker leads directly to improvements in outcomes like survival and quality of life), and eventually assesses the cost effectiveness of the biomarker. Even for biomarkers that successfully reach these milestones, implementation in clinical practice may still be challenging because of additional barriers.¹⁶

As with all tests in clinical practice, tests related to biomarkers are delivered within given health care settings, and currently, little structured evidence exists in regard to how PO can successfully be implemented for particular sets of patients, with different degrees of access to health care, across diverse health care systems. This requires systematic, timely, and accurate collection and evaluation of the evidence base for the benefits and harms of specific PO interventions. As this evidence accumulates from scientific and clinical studies, two approaches dominate its synthesis and assessment. The first is the traditional evidence appraisal that applies the well-established concepts of evidence-based medicine (EBM).^17,18 EBM integrates a physician’s clinical experience with scientific evidence that has undergone an SR, which involves extensive surveying of the literature followed by a synthesis of the primary studies.¹⁹ The second is biocuration, which refers to the distillation and integration of biologic information from scientific literature and large data sets using database- or research field-specific, controlled vocabularies or ontologies; it is a cornerstone of bioinformatics.²⁰ Both disciplines have the same overarching goal of bringing to patients only the interventions proven to be effective by carefully balancing benefits and harms, doing so in different ways; SRs start with a particular question and systematically develop the evidence base around it, whereas biocuration may be either top-down or bottom-up and usually involves more real-time updating. We compare key aspects of these two approaches in Table 1 and summarize important feature of specific curated databases in Appendix Table A1.

Table 1.

Comparison of Key Features of Systematic Reviews and Biocuration

The goal of SRs is to allow a comprehensive and global view of the available evidence base on a particular question of interest by analyzing primary studies that meet prespecified eligibility criteria via explicit, reproducible methodologies that minimize bias. The typical steps involved in an SR include identification of the clinical question, specification of eligibility criteria, systematic search for all studies that meet these criteria, extraction of evidence from eligible studies and assessment of their methodologic rigor, and analysis and qualitative and/or quantitative synthesis (meta-analysis) of the extracted data.¹⁹ An SR is typically structured around the PICOTS elements: the population toward which the findings will be applicable, the intervention, the comparisons being performed, the clinical outcomes, the timing of the eligible studies, and the clinical setting. During the course of decades, a toolbox of methods has been developed in SRs and EBM to evaluate key elements of published studies including systematic biases, statistical precision, applicability to a target clinical setting, and others.^21-23 Evidence for specific interventions on the basis of supporting SRs is generally considered strong. For example, a joint Centers for Disease Control and Prevention/National Cancer Institute framework ranks clinical practice guidelines on the basis of SRs that support the use of a genomic test as tier one (“ready to implement in clinical practice”²⁴) and those that are not based on SRs as tier two (“may be useful in the context of informed clinical decision making”²⁴).

In general, SR teams are multidisciplinary, consisting of clinicians, content experts, methodologists, statisticians, and librarians. Because researchers may often be influenced by their own investment in the field when interpreting the evidence base,²⁵ ideally, parties involved in a SR should have no personal vested interests in the topic of the SR. Although content experts are critical for the clinical interpretation of primary studies and for putting them in context with the broader evidence, their role as authors in SRs has been debated.²⁶ However, since SRs have various degrees of complexity, relevant sponsors often make decisions on a case-by-case basis by thoroughly reviewing the potential conflicts of individual researchers. For example, the Agency for Healthcare and Research Quality Evidence-Based Practice Centers Program has a set of guidelines for how different types of conflicts should be handled and alleviated.²⁷ Given that precision medicine is a relatively new and rapidly evolving field with potentially more complex statistical analyses and contextual questions compared to traditional studies, including and relying on content experts is more critical.

Synthesis of evidence from PO studies encounters a number of specific challenges. In some cases, the biomarker of interest may be examined as predictive for different treatments, but the number of studies considered for each treatment may be small. Consider for instance a Cochrane review of first-line treatments in individuals with EGFR-mutated noncurable stage IIIB to IV nonsquamous non–small-cell lung cancer, which included 19 RCTs.²⁸ The overall conclusion was that the tyrosine kinase inhibitor therapies (erlotinib, gefitinib, and afatinib) led to improved progression-free survival but not to improved overall survival, whereas the monoclonal antibody cetuximab did not show any improved outcomes. While this study included 19 RCTs total, comparing targeted therapies to chemotherapy or best supportive care, the number of RCTs conducted for each therapy was between two and eight and the study designs, outcome measures, and analyses performed and reported varied. These factors led to each meta-analysis considering no more than five RCTs. SRs have also identified existing evidence gaps for other prevision medicine interventions, such as the lack of robust evidence for tailoring smoking cessation interventions on the basis of germline genetic variation.²⁹

Many issues with the SR of evidence for PO interventions will be solved in time, as more studies accumulate, but it is important to note that the desire for new therapies may be at a historical high given the stated promise of precision medicine and PO in particular. However, another challenge that will linger is that the definition of precision medicine will continue to make it difficult to satisfy the PICOTS framework as, for example, more drugs are tested separately in different subsets of patients (eg, if the same biomarker is considered for different tumor types) or molecular tests rapidly evolve to add more biomarkers or change how they are tested (eg, protein expression, gene expression, or DNA amplification).¹³ New RCT designs such as umbrella and basket trials are now being implemented to meet some of these challenges. Although other study designs may provide additional valuable information on specific treatments, the question of clinical utility, in particular, is difficult to answer in the absence of evidence from RCTs.

Biocuration identifies and summarizes biomedical results, including potentially those from SRs, into bioinformatic databases, often by using controlled vocabularies and prespecified standards.²⁰ A text summary of the evidence related to specific scientific questions may also be included, along with links to the original studies or other resources. In particular, medical curation focuses on disease associations and genetic factors and includes efforts like the OMIM (Online Mendelian Inheritance in Man) database³⁰ as well as specialized databases for particular genes, variants, and diseases. ClinVar, a public archive of relationships among medically important germline and somatic variants and human phenotypes, was launched in April 2013.³¹ Biocuration generally provides more immediate updating than formal SRs, but the lack of a unified systematic framework may lead to only partial or inconsistent results. For example, the ClinVar archive contains clinical interpretations of genetic variants submitted by laboratories and expert curators, resulting in a wealth of information but also in issues such as contradictory interpretations and many variants of uncertain significance.³² The ClinGen (Clinical Genome Resource) initiative aims to resolve some of these problems and answer the critical questions of clinical validity, disease causality (pathogenicity), and clinical actionability by standardizing data collection and sharing and implementing an approach for consensus among expert curators.³² Many other cancer-specific databases now exist for synthesizing evidence for PO approaches,^33-37 leading ClinGen to recently develop recommendations and guidelines for defining cancer somatic variants on the basis of their diagnostic, prognostic, and predictive roles, using evidence of their significance and clinical utility.³⁸ In a translational field such as PO, where molecular genomics and clinical practice come together, a particular challenge for biocuration is that each content area has its own detailed terminology (eg, specific diseases and syndromes can be described using ICD-10 codes³⁹ and SNOMED terms⁴⁰). However, mapping these terminologies to specific causal mutations, as well as to biomarkers and tratments, can often be a challenge.

Biocurators come from a wide variety of backgrounds, usually in biology, biochemistry, or medical genetics, and increasingly have interdisciplinary training that includes computer and information sciences, with subject matter experts playing an important role in the field of biocuration.^20,41 A growing number of clinical laboratories in hospitals and community clinics are now also conducting molecular diagnostic testing to identify sequence variants and inform treatment decisions for patients, in which case the curation is often performed by molecular pathologists as well as clinicians. The expert panels used by resources such as ClinGen to resolve conflicting interpretations and curate variants of unknown significance include medical professionals, medical geneticists, clinical laboratory diagnosticians and molecular pathologists that have a long standing scope of work in the disease gene in question.

An example of a curated cancer database is the recently released CIViC (Clinical Interpretation of Variants in Cancer) database,³⁶ which provides associations between drugs and genes or variants in specific cancer types. Like other such repositories, CIViC relies on crowdsourcing as a way of ensuring biocuration support but only accepts expert-reviewed contributions. It provides a large amount of data, including at the time of this commentary results from 1,077 published studies. However, the authors state the presence of a bias toward positive associations with treatment outcomes (91% of records show support for the use of a therapy), which illustrates the absence of a systematic, unbiased framework for identifying the evidence base. Additionally, it does not include quantitative results, such as differences in overall survival between treatment groups. This approach is thus quite different from the traditional SR approach. Given that CIViC considers many different types of studies and that studies of cancer drugs may have many different designs, it is true that it would be extremely challenging to have a fixed list of terms that would be applicable to all studies and easily translatable into PICOTS terminology.

In general, literature searches for SRs are typically more exhaustive and thorough than for curated databases, where eligible studies tend to be identified by experts in a given field. Although currently this may have little direct impact on identifying relevant studies for biocuration, given the small number of clinical PO publications, this will eventually become a major issue in the next years, because the field will need to deal with increasing amounts of published and unpublished evidence. SRs also tend to have more well-defined a priori eligibility criteria that a study has to meet to be considered in the evidence synthesis; in contrast, biocuration approaches show more variability and rely more on individual experts in the field. Although biocuration currently lacks a standardized and widely accepted framework to assess the quality of published evidence, several efforts are under way to change this.^38,42-44

Beyond trying to develop standardized frameworks for biocuration, a number of other approaches for improving curated databases have been considered, including the use of specific incentives. For example, databases can have annotation jamborees⁴⁵ with invited experts, which may result in publications. Some large projects such as UniProt⁴⁶ and ClinVar³¹ employ curators and train them in specific guidelines and processes. While mandatory submissions are successfully used for archival databases or repositories such as the Gene Expression Omnibus,⁴⁷ curated databases generally prefer only high-quality representative publications, meaning that they would not encourage this approach. The emphasis on a systematic survey of the literature which includes non-significant results that is present in SRs is also not an explicit part of the biocuration philosophy. We suggest, that moving forward, it would be extremely helpful for biocurators to be made aware of the SR process and vice-versa. For example, if the EBM community engages the biocuration community, they may find ways of incorporating more elements of the PICOTS framework into existing and future databases.

There is often a tension between systematic, unbiased syntheses and assessments versus timely and accessible curated information, and the highly promising field of PO is a clear example that shows the difficulties in finding a good solution that considers both of these dimensions. In particular, SRs often tend to focus on the first aspect and biocuration on the second. We believe that these two approaches to synthesizing evidence must come together so that their members can discuss their specific philosophies, approaches, and products, instead of continuing to proceed on parallel paths. Expertly curated databases can, for instance, serve as one of the inputs into SRs, because they may provide publications initially missed by the reviewers. Conversely, SRs, including those that appear as detailed whitepapers, should always be included in curated databases. Curated databases will continue to be an initial first stop for many researchers and clinicians, especially because SRs may lag years behind current studies, and decisions often need to be made in cases where SRs are lacking. However, more training is needed for users to understand that, for example, a higher risk of bias exists for finding more significant associations when consulting a curated PO database than when using a SR. Both communities have much to learn from each other, and their collaboration could result in substantial improvements in evidence synthesis in PO, in terms of both quality and speed.

Appendix

Table A1.

Comparison of Key Features of Biocuration Databases

AUTHOR CONTRIBUTIONS

Conception and design: Simina M. Boca, Subha Madhavan

Collection and assembly of data: Simina M. Boca, Shruti Rao

Data analysis and interpretation: Orestis A. Panagiotou, Peter B. McGarvey

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/po/author-center .

Simina M. Boca

Research Funding: Symphogen (Inst)

Orestis A. Panagiotou

No relationship to disclose

Shruti Rao

Research Funding: Symphogen (Inst)

Peter B. McGarvey

No relationship to disclose

Subha Madhavan

Leadership: Perthera

Stock and Other Ownership Interests: Perthera

Consulting or Advisory Role: Perthera