Clinical Use of Precision Oncology Decision Support

Abstract

Purpose

Precision oncology is hindered by the lack of decision support for determining the functional and therapeutic significance of genomic alterations in tumors and relevant clinically available options. To bridge this knowledge gap, we established a Precision Oncology Decision Support team that provides annotations at the alteration level and subsequently determined whether clinical decision making was influenced.

Methods

Genomic alterations were annotated to determine actionability on the basis of a variant’s known or potential functional and/or therapeutic significance. The medical records of a subset of patients annotated in 2015 were manually reviewed to assess trial enrollment. A Web-based survey was implemented to capture the reasons genotype-matched therapies were not pursued.

Results

The Precision Oncology Decision Support team processed 1,669 requests for annotation of 4,084 alterations (2,254 unique) across 49 tumor types for 1,197 patients. A total of 2,444 annotations for 669 patients included an actionable variant call: 32.5% actionable, 9.4% potentially actionable, 29.7% unknown, and 28.4% nonactionable. Sixty-six percent of patients had at least one actionable/potentially actionable alteration, and 20.6% of patients (110 of 535) annotated enrolled in a genotype-matched trial. Trial enrollment was significantly higher for patients with actionable/potentially actionable alterations (92 of 333; 27.6%) than for those with unknown (16 of 136; 11.8%) and nonactionable (2 of 66; 3%) alterations (P < .001). Actionable alterations in PTEN, PIK3CA, and ERBB2 most frequently led to enrollment in genotype-matched trials. Clinicians cited a variety of reasons that patients with actionable alterations did not enroll in trials.

Conclusion

Over half of alterations annotated were of unknown significance or nonactionable. Physicians were more likely to enroll a patient in a genotype-matched trial when an annotation supported actionability. Future studies are needed to demonstrate the impact of decision support on trial enrollment and oncologic outcomes.

INTRODUCTION

In precision oncology, next-generation sequencing is becoming routinely used for diagnostic and therapeutic decision making. However, as the number of genes tested within clinically used platforms increases, so does the rate of detected aberrations; yet, the majority of these are passenger mutations that do not drive tumorigenesis.¹ Importantly, an alteration may occur within an actionable gene, but the alteration itself confers no functional change or imposes a change that does not confer an oncogenic advantage. Therefore, the oncologist needs to assess whether the specific variants are actionable.

A study conducted at a leading cancer center revealed that many oncologists have low confidence in their knowledge of genomics.² Furthermore, earlier studies showed that few patients with potentially actionable alterations were enrolled in genotype-matched trials, indicating low clinical utility of such information.^3-7 Limited resources available to assist with interpretation of genomic reports and to facilitate identification of genomically matched therapies may be contributing factors.^5,7-9 In recent years, teams have recognized this knowledge gap, and publically available resources, such as Personalized Cancer Therapy (http://pct.mdanderson.org), My Cancer Genome (http://www.mycancergenome.org), Targeted Cancer Care (https://targetedcancercare.massgeneral.org), and OncoKB (http://oncokb.org) were developed as decision-making tools. Yet, these resources have potential limitations. First, usability research indicates that clinicians are unlikely to use sources that take longer than 30 seconds to retrieve data¹⁰ and that information presented in graphical summaries enhances interpretation, improving health care quality.^11,12 Second, limited information may be exposed because these Web sites are public facing. Third, novel alterations continue to be discovered, and information is rapidly evolving, generally outpacing the rate of updates. Thus, there is a need for an on-demand, real-time clinical interpretation service that annotates all requested alterations, beyond those available within accessible knowledge bases, to determine their actionability.

The Precision Oncology Decision Support system (PODS) was established at The University of Texas MD Anderson Cancer Center in recognition of these needs.⁹ Herein, we report our experience with large-scale, institution-wide decision support and its initial clinical utility. These data are among the first to suggest that providing alteration-level decision support may have a measurable effect on clinical action.

METHODS

Analysis of Patient Annotation Requests and Reports

Annotation requests originated from CLIA (Clinical Laboratory Improvement Amendments)–validated (90.2%) and research (9.8%) panels. Annotations in all final reports were analyzed. All requestors indicating the same physician as the contact for correspondence were considered a team. Statistical significance was evaluated by a χ² test.

Annotation Process and Return of Reports

PODS scientists receive requests, access our knowledge base of variant annotations, and review and update content as applicable.^9,13 Reports consist of a data summary with references and alteration frequency from external (cBioPortal, COSMIC [Catalogue of Somatic Mutations in Cancer]) and internal databases. In 2015, reports evolved to routinely contain a functional significance, gene, and alteration-level actionability call (Table 1).^7,12 All reports are reviewed by the medical director and returned via e-mail. Starting in August 2015, reports referencing CLIA-validated panels were deposited within MD Anderson’s electronic health record (EHR).

Table 1.

Actionable Variant Call Definitions

Manual Review of Clinical Data

Medical records of 539 patients annotated in 2015 through clinician-initiated requests or for treatment planning conferences were reviewed for the presence of a clinical trial entrance note in the patient’s EHR. Potential matches between the annotated variants and targeted therapy used within the trial were manually recorded.

Clinical Decision Follow-Up Surveys

A Web-based survey was initiated in 2015, and 236 surveys were sent to annotation requestors with a 1-month postannotation delivery date to ascertain whether the patient matched to genomically guided therapy. Responses received by January 2016 were analyzed and manually validated through EHR review. For patients not enrolled to receive therapy on the basis of the annotation provided, the survey inquired as to the reasons why.

RESULTS

Patient Annotation Requests

The PODS team processed 1,669 requests for 1,197 patients, most in 2014 and 2015 (Data Supplement). Multiple requests for the same patient may have been due to additional alterations requested for annotation, different clinicians requesting annotation of the same molecular report, or the same molecular report requested for different venues in which the patient was evaluated (eg, clinic visit v treatment planning conference). Requests were of two main categories: program-initiated (60%) and clinician-initiated (40%). Program-initiated requests were on behalf of genomically informed program leaders requesting annotations for all participating patients and accounted for reports to 58 physicians for 600 patients. Clinician-initiated requests originated from treating physicians and team members for point-of-care decisions on the basis of molecular testing or from clinical trial teams screening for patients to enroll in genotype-matched trials. Fifty-six physician teams initiated requests for 724 patients and ranged from one to 176 per team, with two requests being the median (Data Supplement). Requests were received for patients with 49 different tumor types, of which colorectal (16%) was most common (Fig 1A).

Fig 1.

Requests received and annotations provided. The percentage of requests is shown for (A) each indicated tumor type (top 25) and (B) annotation type (full panel or select alterations). The number of annotations provided within each gene is shown for (C) full panel or (D) select alteration annotations. Genes with 20 or more annotations are shown.

Alterations Requested for Annotation

PODS offered requestors an option to select between full-panel (all reported alterations within an assay) and select-alterations (requestor-specified select alterations within an assay) annotation types. Across all requests, 712 (43%) were for full panel and 957 (57%) were for select alterations (Fig 1B). Clinician-initiated requests were largely (88%) for select alterations, likely derived from clinicians screening patients with alterations in select genes for trial enrollment.

Overall, 4,084 variant-level annotations were provided for 2,254 unique alterations. Multiple annotations may have been provided for a single alteration because of multiple requests to annotate the same or different patients harboring the same alteration. Alterations spanned 356 genes; each gene fusion partner counted individually. Genes most commonly annotated as part of a full panel request included TP53, KRAS, and PIK3CA, representing the three most commonly mutated genes in the genome (Fig 1C). Conversely, requests to annotate select alterations were most frequently for PTEN, EGFR, and SMO (Fig 1D). The range of unique alterations annotated per gene was one to 179, with the largest number in TP53 and PTEN (Data Supplement). Annotated alterations originated from 32 different testing platforms (Data Supplement) and were primarily missense mutations (Data Supplement).

Actionable Genes and Alterations

A gene is actionable if there are clinically available therapies that directly or indirectly targe alterations in the gene and/or there are clinical trials selecting for alterations in the gene. Of the 4,084 annotations, 3,166 (78%) were within a gene defined as actionable at the time of annotation (Fig 2A; Data Supplement). Of the 918 annotations delivered in nonactionable genes, 816 (89%) were part of a full panel annotation. In 2015, annotation reports evolved to contain an actionable variant call describing the alteration’s functional and therapeutic significance, with classification into four broad categories—actionable (further subcategorized by source: literature based, inferred, or functional genomics), potentially actionable, unknown, and nonactionable (Table 1)⁹—which was provided with 2,444 annotations delivered for 669 patients. A total of 794 (32.5%) annotations were actionable, 230 (9.4%) were potentially actionable, 725 (29.7%) were unknown, and 697 (28.4%) were not actionable (Fig 2B). Considering alterations within actionable genes, 40.7% of annotations were actionable, 11.9% were potentially actionable, 36.1% were unknown, and 11.3% were nonactionable (Fig 2C). A total of 648 (97%) of the 669 patients had at least one alteration within an actionable gene, and 66% had at least one actionable/potentially actionable alteration (data not shown).

Fig 2.

Actionable genes and variants reported. (A) The percentage of annotations provided in an actionable or nonactionable gene, as defined at the time of annotation. The percentage of annotations delivered with the indicated actionable variant calls for (B) all or (C) only actionable genes. (D) The number of actionable annotations provided within each specified gene (minimum of 10 actionable annotations per gene). (E) The total number of annotations provided with an actionable variant call (blue bars) and the subset that is actionable (yellow bars) are shown per tumor type for types with at least 10 annotations.

The majority of actionable/potentially actionable annotations were for alterations residing within KRAS, PTEN, and PIK3CA (Fig 2D and Data Supplement, respectively). The total number of annotations in 2015 and the subset that was actionable/potentially actionable are shown per the patient’s tumor type (Fig 2E and Data Supplement, respectively). The actionability of KRAS has been controversial, particularly in colorectal cancer.^14-16 When excluding KRAS as actionable, most actionable annotations were provided for breast cancer patients (n = 90; Data Supplement).

Clinical Utility of Annotations

We then asked whether physicians acted more often on alterations with evidence of actionability (Data Supplement). We manually recorded clinical follow-up data by accessing the EHRs of 539 patients requested for annotation by clinicians or for treatment planning conferences. Four patients were excluded: two had mutations that were actionable only for resistance to therapy, and two because of insufficient follow-up since physician notification. For the remaining 535 patients, variant annotation data were filtered, such that each patient in the data set is represented by a single alteration (Fig 3A). To achieve this, one of two filters was applied: (1) for patients enrolled in genotype-matched therapy, the alteration leading to enrollment was recorded, whereas all other variants were filtered out; or (2) for patients not enrolled in genotype-matched therapy, the alteration with the highest variant call (Yes: Literature Based > Yes: Inferred > Potentially > Unknown > No) was recorded, whereas all other variants were filtered out.

Fig 3.

Clinical decisions made for a subset of patients annotated with actionable variant calls. (A) Patients annotated in 2015 through clinician-initiated requests and for treatment planning conferences. The fraction of patients represented by the highest variant call (yellow column) and the corresponding fraction of patients with that call enrolled in a trial (gray column). (B) Accrual in genomically matched clinical trials on the basis of variant actionability (P < .001). (C) A subset of patients annotated in 2015 through a clinician-initiated request and for whom a survey was returned. The fraction of patients represented by the highest variant call (yellow column) and the corresponding fraction of patients with that call enrolled in a trial (gray column). Trial enrollment varied significantly by variant actionability (P < .001).

Overall, 20.6% of patients (110 of 535) enrolled in a targeted trial. Alterations of 214 patients (40%) were classified into the Yes: Literature Based category; 54 (10.1%) were classified into the Yes: Inferred category; 65 (12.1%) were classified into the Potentially category; 136 (25.4%) were classified into the Unknown category; and 66 (12.3%) were classified into the No (nonactionable) category (Fig 3A, yellow column). Ninety-two of 333 patients (27.6%) with actionable/potentially actionable alterations were enrolled in genomically matched trials compared with 16 of 136 patients (11.8%) with unknown alterations and two of 66 patients (3%) with nonactionable alterations (Fig 3B; P < .001). Patients with actionable alterations in PTEN (n = 20), PIK3CA (n = 11), and ERBB2 (n = 10) most frequently enrolled in a trial (Fig 4), paralleling the data showing a large number of actionable/potentially actionable annotations delivered for PIK3CA and PTEN (Fig 2D; Data Supplement).

Fig 4.

The distribution of genes that led to patient trial enrollment after annotation; 110 of 535 patients annotated by the Precision Oncology Decision Support team in 2015 enrolled in a genotype-matched trial on the basis of one of the 32 genes shown.

To understand why physicians may not have acted on potentially actionable alterations, we introduced a Web-based survey in 2015 to accompany physician-initiated requests. Of 236 surveys sent, 223 were returned (94.5% response rate). Two were excluded because patients enrolled in genotype-matched trials before the annotation (Fig 3C). All reports were classified by actionability as described previously (Fig 3C, yellow column). A combination of survey responses and manual review revealed that 26.8% of annotations (59 of 221) led to a genotype-matched trial (Fig 3C, gray column). Similar to data in Fig 3B, patient enrollment in genotype-matched trials in the survey group was significantly higher on the basis of actionable/potentially actionable versus unknown versus not actionable variants (P < .001).

Of the 90 patients with unknown/not actionable variants, representing 55.9% of total patients not enrolled in a trial, 57 (63.3%) did not enroll because the respondents agreed with the PODS annotation, indicating that the alteration function did not support trial enrollment (Table 2). Another seven patients (7.8%) were responding to current treatment, and physicians acted on another alteration in four patients (4.4%). The latter had another well-characterized oncogenic variant (IDH1 R132C, AKT1 E17K, or FGF3 and FGF4 amplifications) or an alteration clearly inferable as actionable (early truncating PTCH1 mutation), where either decision support was not needed or was previously supplied outside the survey group. Finally, three patients (3.3%) elected to be treated elsewhere.

Table 2.

Reasons Patients Did Not Enroll in Genotype-Matched Trials After Precision Oncology Decision Support Annotations, as Stated by the Survey Respondents

Among 161 patients with variants classified as Yes: Literature Based and Yes: Inferred, there were only 42 patients (26.1%) who did not enroll in a genotype-matched trial (Table 2). Reasons included another alteration pursued (two patients), the patient was responding to current therapy (six patients), and the patient elected treatment elsewhere (nine patients). Other reasons included non–genotype-matched treatment options pursued (five patients) and ineligibility (11 patients). In five instances, physicians indicated that the annotation did not support trial enrollment, which we investigated further. For one patient, although the function of the alteration was clearly activating, the mutation was subclonal. For another, the patient had an activating BRAF mutation; however, the prior treatment precluded treatment with other BRAF inhibitors. For yet another patient, the activating BRAF mutation was found in only one of two biopsies, raising concern about tumor heterogeneity.

DISCUSSION

Meta-analyses comparing precision oncology with nonpersonalized approaches reveal higher response rates, longer survival rates, and fewer toxicity-related deaths in patients treated with targeted therapies.^17-20 However, few biomarkers have an indication for treatment with a Food and Drug Administration–approved drug specific to the patient’s tumor type,^21-27 and a limited number of patients with potentially actionable alterations receive genotype-matched therapies in experimental contexts.^3-7 One contributing factor may be a lack of decision support, as suggested by the Cancer Genome Evaluation Committee, which found that physicians are often overwhelmed by data of uncertain significance and that sound guidelines are essential for determining clinical action.²⁸ Moreover, we previously reported modest differences in trial enrollment between patients with or without potentially actionable alterations.³ Assessing a new population of patients where decision support was provided, we observed that physicians acted more often when the function of the alteration was known.

The necessity for real-time decision support is clear from the large volume of requests received from numerous physicians (Data Supplement) treating patients with diverse tumor types for alterations in a range of genes. However, bias likely exists toward physicians leading targeted therapy trials and for genes targeted by those therapies.

The PODS team is frequently asked to provide an annotation for sequencing reports from commercial vendors that produce end-to-end reports, some already containing alteration-level annotations. Distinguishing factors of PODS reports are a clear call of functional effect⁹ (eg, Activating, Inactivating, Unknown), a range of variant-level actionability categories on the basis of experimental evidence, and inclusion of all MD Anderson genotype-matched clinical trials in current reports.

Previous studies reported high rates of alterations in actionable genes,^4,29-33 and we provided an annotation in at least one actionable gene for 97% of patients in our cohort. Using the TARGET (Tumor Alterations Relevant for Genomics-Driven Therapy) database and PHIAL (Precision Heuristics for Interpreting the Alteration Landscape) algorithm to rank alterations followed by manual annotation for only selected patients, a study showed that 90% of patients have clinically relevant alterations.³⁴ Conversely, we found that 66% of patients annotated have at least one potentially actionable alteration on the basis of manual curation of all aberrations. Potential differences between the studies include (1) the inclusion of potentially actionable diagnostic or prognostic alterations by Van Allen et al³⁴ and not in our study; (2) we provided annotations for only requested alterations, and few annotation requests were obtained for well-established alterations (eg, BRAF V600E); and (3) the difference in the number of patients assessed by manual curation in the two studies.

Among all annotations with an actionable variant call, only 32.5% were classified as actionable, with another 9.4% classified as potentially actionable. Even within genes that are considered actionable, 47.4% of the annotations either had no evidence to support actionability or were not actionable. Moreover, 58% of the genomic alterations annotated and evaluable for frequency have not been reported in the COSMIC database (Data Supplement). These data highlight the need to define actionability at the variant, rather than gene, level.

To determine whether physicians acted according to the evidence presented in the PODS reports, we followed 535 patients. Only trial enrollment was assessed, because few annotation requests were elicited to determine the appropriateness of off- or on-label treatment, as indicated by our survey data. We found that patients with potentially actionable/actionable alterations more often enrolled in genotype-matched trials than did those with unknown/not actionable alterations. Thus, physicians more often act on alterations when sufficient evidence is provided that they may be driver events.

The most often-cited reason that patients did not enroll in genotype-matched trials was that the annotation did not support trial enrollment. This reason was most frequently given for alterations annotated as unknown/not actionable, indicating physicians’ agreement with our annotation. In several cases, physicians acted on well-known, oncogenic variants that did not require an interpretation by PODS, highlighting that, at least at this institution, physicians do not require decision support for the evaluation of most common genomic alterations.

Overall, enrollment in genotype-matched clinical trials was 20.6% (110 of 535), dramatically higher than other independent evaluations.^3-7 The increase we observe from our prior study (11%) may be due to several factors besides delivery of PODS annotations: (1) PODS proactively provided genotype-matched trials in select reports, (2) an expanded portfolio of genotype-relevant trials open at MD Anderson, and (3) e-mail alerts to physicians regarding potential genotype-matched trials for patients with specific alterations.

In conclusion, a decision support system for annotations of patients’ molecular profiles was widely used at a major cancer center. PODS reports provided alteration-level actionability information that translated into more patients enrolled in genotype-matched trials with actionable/potentially actionable alterations than those with unknown/not actionable alterations.

AUTHOR CONTRIBUTIONS

Conception and design: Amber Johnson, Yekaterina B. Khotskaya, Lauren Brusco, Mark Routbort, Kenna R. Mills Shaw, Gordon B. Mills, Funda Meric-Bernstam,

Financial support: Kenna R. Mills Shaw, Funda Meric-Bernstam

Administrative support: Kenna R. Mills Shaw, John Mendelsohn, Funda Meric-Bernstam

Provision of study materials or patients: Sarina Piha-Paul, Vivek Subbiah, David Hong, Kenna R. Mills Shaw, Funda Meric-Bernstam

Collection and assembly of data: Amber Johnson, Yekaterina B. Khotskaya, Lauren Brusco, Jia Zeng, Vijakumar Holla, Beate C. Litzenburger, Nora S. Sánchez, Sarina Piha-Paul, Vivek Subbiah, David Hong, Mark Routbort, Russell Broaddus, Kenna R. Mills Shaw, Funda Meric-Bernstam

Data analysis and interpretation: Amber Johnson, Yekaterina B. Khotskaya, Lauren Brusco, Jia Zeng, Vijakumar Holla, Ann M. Bailey, Md Abu Shufean, Russell Broaddus, Gordon B. Mills, John Mendelsohn, Funda Meric-Bernstam

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or po.ascopubs.org/site/ifc.

Amber Johnson

No relationship to disclose

Yekaterina B. Khotskaya

No relationship to disclose

Lauren Brusco

Employment: Celgene

Jia Zeng

Stock and Other Ownership Interests: Amgen, McKesson, Mylan, Adamas Pharmaceuticals

Vijaykumar Holla

No relationship to disclose

Ann M. Bailey

No relationship to disclose

Beate C. Litzenburger

Employment: Qiagen

Nora S. Sánchez

No relationship to disclose

Md Abu Shufean

No relationship to disclose

Sarina Piha-Paul

Consulting or Advisory Role: Genentech

Research Funding: GlaxoSmithKline, XuanZhu, Puma Biotechnology, Novartis, Merck Sharp & Dohme, Curis, Principa Biopharma, Biomarin, Helix BioPharma, Bayer, Abbvie, Incyte, Five Prime Therapeutics, Cerulean Pharma, MedImmune, Medivation

Vivek Subbiah

Research Funding: Novartis, GlaxoSmithKline, Nanocarrier, NorthWest Biotherapeutics, Roche/Genentech, BergPharma, Bayer, Incyte, FujiFilm, Pharmamar, D3, Pfizer, Amgen, Abbvie, Multivir, Bluprint Medicines

Travel, Accommodations, Expenses: Novartis, Pharmamar, Fujifilm

David Hong

Stock and Other Ownership Interests: MolecularMatch, Oncorena

Honoraria: Adaptimmune, Baxter, Merrimack, Bayer

Consulting or Advisory Role: Baxter, Bayer

Research Funding: Novartis, Genentech, Eisai, AstraZeneca, Pfizer, miRNA Therapeutics, Amgen, Daiichi Sankyo, Merck, Mirati Therapeutics, Eli Lilly

Travel, Accommodations, Expenses: Loxo, miRNA Therapeutics

Other Relationship: Oncorena

Mark Routbort

No relationship to disclose

Russell Broaddus

No relationship to disclose

Kenna R. Mills Shaw

No relationship to disclose

Gordon B. Mills

Stock and Other Ownership Interests: Catena, PTV Sciences, Spindletop Ventures, Myriad Genetics, Immunome

Honoraria: Symphogen, Nuevolution, AstraZeneca, Isis Pharmaceuticals, Eli Lilly, Novartis, Allostery, Immunome

Consulting or Advisory Role: Adventis, AstraZeneca, Catena, Critical Outcome Technologies, Millennium, Nuevolution, Precision Medicine Research Associates, Provista Diagnostics, SignalChem, Symphogen, Allostery, Isis Pharmaceuticals, MedImmune, Eli Lilly, Novartis, Tarveda Therapeutics, Tau Therapeutics

Research Funding: Adelson Medical Research Foundation, AstraZeneca, Critical Outcome Technologies, NanoString Technologies, Komipharm, Breast Cancer Research Foundation, Karus Therapeutics, Illumina, Millennium

Patents, Royalties, Other Intellectual Property: HRD assay to Myriad Genetics

Travel, Accommodations, Expenses: AstraZeneca, Symphogen, Isis Pharmaceuticals, MedImmune, Eli Lilly, Novartis, Immunome, Allostery, Pfizer

John Mendelsohn

Leadership: Merrimack

Stock and Other Ownership Interests: Merrimack

Patents, Royalties, Other Intellectual Property: Royalty payments from University of California, San Diego

Travel, Accommodations, Expenses: Merck - Germany

Funda Meric-Bernstam

Honoraria: Dialecta

Consulting or Advisory Role: Genentech, Inflection Biosciences, Pieris Pharmaceuticals, Clearlight Diagnostics, Darwin Health

Research Funding: Novartis, AstraZeneca, Taiho Pharmaceutical, Genentech, Calithera Biosciences, Debiopharm Group, Bayer, Aileron Therapeutics, PUMA Biotechnology, CytomX Therapeutics, Jounce Therapeutics, Zymeworks, Effective Pharmaceuticals, Curis