Clinical Validation and Implementation of a Targeted Next-Generation Sequencing Assay to Detect Somatic Variants in Non-Small Cell Lung, Melanoma, and Gastrointestinal Malignancies

Abstract

We tested and clinically validated a targeted next-generation sequencing (NGS) mutation panel using 80 formalin-fixed, paraffin-embedded (FFPE) tumor samples. Forty non-small cell lung carcinoma (NSCLC), 30 melanoma, and 30 gastrointestinal (12 colonic, 10 gastric, and 8 pancreatic adenocarcinoma) FFPE samples were selected from laboratory archives. After appropriate specimen and nucleic acid quality control, 80 NGS libraries were prepared using the Illumina TruSight tumor (TST) kit and sequenced on the Illumina MiSeq. Sequence alignment, variant calling, and sequencing quality control were performed using vendor software and laboratory-developed analysis workflows. TST generated ≥500× coverage for 98.4% of the 13,952 targeted bases. Reproducible and accurate variant calling was achieved at ≥5% variant allele frequency with 8 to 12 multiplexed samples per MiSeq flow cell. TST detected 112 variants overall, and confirmed all known single-nucleotide variants (n = 27), deletions (n = 5), insertions (n = 3), and multinucleotide variants (n = 3). TST detected at least one variant in 85.0% (68/80), and two or more variants in 36.2% (29/80), of samples. TP53 was the most frequently mutated gene in NSCLC (13 variants; 13/32 samples), gastrointestinal malignancies (15 variants; 13/25 samples), and overall (30 variants; 28/80 samples). BRAF mutations were most common in melanoma (nine variants; 9/23 samples). Clinically relevant NGS data can be obtained from routine clinical FFPE solid tumor specimens using TST, benchtop instruments, and vendor-supplied bioinformatics pipelines.

In modern oncologic practice, patients with advanced-stage non-small cell lung cancer (NSCLC),1, 2 melanoma,3, 4 and colorectal adenocarcinoma5, 6 are often treated with targeted therapies as standard of care or after enrollment in clinical trials. Molecular mutation analysis is the preferred testing modality to guide therapeutic decision making and/or eligibility for biological studies. Therefore, laboratory-developed mutation assays require robust workflows that produce high-quality sequence information from routine clinical specimens, namely formalin-fixed, paraffin-embedded (FFPE) samples. As molecular testing transitions from an ancillary tool to a seminal requirement for optimal oncologic patient management, multiplex sequencing assays with clearly defined content and bioinformatics workflows are essential for accurate and consistent results, reporting, and patient management.

Published guidelines endorse which genes to test in a particular tumor type and provide timeframes for receipt of actionable results, but they also grant individual laboratories autonomy to perform mutation testing using any suitable validated method.² Historically at our institution, single-gene mutation analysis for clinically relevant genes was performed either in-house or at a Clinical Laboratory improvement Amendment–certified reference laboratory. Depending on the result, reflex testing was performed for additional genes per mutation frequency or designated algorithms. Unfortunately, this approach introduced considerable turn-around time delays and unnecessary cost, particularly when send-out testing was required. Therefore, we sought testing modalities that could analyze multiple clinically relevant mutations simultaneously, accurately, and expeditiously.

Next-generation sequencing (NGS) technologies have revolutionized genomic medicine by allowing high-throughput, parallel sequencing of the human genome.⁷ Currently, however, a large proportion of clinical NGS endeavors are supported by larger academic institutions with shared access to established genomic and bioinformatics research infrastructures, and routine clinical implementation of NGS is complicated by mitigating factors, such as clinical performance, laboratory expertise, lengthy turn-around times, and cost.⁸ Thus, we investigated affordable methods to detect clinically relevant somatic mutations in NSCLC, melanoma, and gastrointestinal (GI) malignancies that generated high-quality sequencing data from FFPE samples, and offered manageable turn-around times. Targeted amplicon-based library preparation methods combined with parallel sequencing offered a practical solution, and recent studies have demonstrated the utility of this approach.9, 10

Reversible terminal dideoxynucleotide sequencing chemistry by Illumina (San Diego, CA) consistently generates accurate and reproducible sequencing data.11, 12 To use this chemistry for clinical testing, we purchased the bench-top NGS sequencer, the Illumina MiSeq, and paired it with the MiSeq-compatible Illumina TruSight tumor (TST) 26-target amplicon-based library preparation kit. TST targets 26 genes and 174 amplicons selected from College of American Pathologists/National Comprehensive Cancer Network guidelines, relevant publications, and late-phase pharmaceutical clinical trials (Supplemental Table S1). TST offered several advantages over other commercially available mutation testing kits, such as bidirectional targeting of the positive and negative DNA strands, full-exon coverage as opposed to hotspot analysis, and robust vendor-supplied bioinformatics techniques optimized for somatic variant detection. More important, TST library preparation is optimized for FFPE samples, multiple safeguards exist to detect FFPE variant artifacts, and deep sequencing of TST libraries consistently yields high depths of coverage of targeted regions.

Somatic mutation testing for many of the TST genes has clinical utility in a wide variety of solid tumors. For example, testing for CTNNB1 exon 3 is performed clinically for diagnostic and prognostic purposes in pediatric desmoid tumors, select PIK3CA hotspot mutations are positive prognostic factors for breast carcinoma, and multiple exons in PDGFRA and KIT are routinely tested in GI stromal tumors to predict response to targeted therapies. More important for our intended validation purposes, all of the clinically relevant genes and regions mutated in NSCLC, melanoma, and colonic adenocarcinoma that were tested in our routine clinical practice were represented. In addition, we could easily incorporate the TST NGS into a 5 business day workflow model, and a cost-analysis demonstrated a reasonable cost per test.

Last, TST NGS data are processed from raw sequence (FASTQ) to called variants with on-board MiSeq Reporter software version 2.3, and variant annotations can be performed with Illumina's VariantStudio software version 2.1 software using standard desktop and laptop computers. The ease of library preparation, sequencing, and data analysis with tools provided by a single vendor best fit our clinical priorities and the resources available at our academic molecular pathology laboratory.

Herein, we present our results from the clinical validation of TST NGS using 80 sequenced samples that were selected from 100 FFPE patient samples (40 NSCLCs, 30 melanomas, and 30 GI malignancies). During our validation, we achieved high depths of coverage for multiple clinically relevant variants when multiplexing 8 to 12 samples on a single MiSeq flow cell. TST NGS consistently demonstrated sensitivities comparable to reference assays, showed 100% concordance with known variants, detected novel variants in many samples, and uncovered variants missed by less-sensitive testing modalities. The TST variant-calling pipeline was robust and showed high concordance when compared with an alternative analysis pipeline, and we used an in-house custom Java program to assess laboratory-defined quality control (QC) metrics and streamline clinical reporting (developed by G.H.S., Emory University, http://github.com/ghsmith/coverageQc). More important, although the results detailed herein represent the experience of a single institution, the data and validation strategies shown herein are broadly applicable to most clinical molecular laboratories interested in offering NGS for NSCLCs, melanomas, and GI malignancies as well as many other solid tumors.

Materials and Methods

Patient Sample Selection

This study was performed with approval from the Institutional Review Board and in concordance with regulatory guidelines regarding clinical assay validation. FFPE tumor blocks from 100 patients were selected for analysis (40 NSCLCs, 30 melanomas, and 12 colonic, 10 gastroesophageal, and 8 pancreatic adenocarcinomas). Samples selected included surgical excisions (n = 57), biopsy specimens (n = 24), cell blocks from fine-needle aspirations (n = 17), and body fluids (n = 2) from primary and metastatic sites. Of the 100 samples selected for TST NGS analysis, 78 were tested for molecular alterations by at least one additional molecular testing methodology. Forty-six samples harbored at least one known oncogenic alteration: 32 single-nucleotide or multinucleotide variants (SNVs or MNVs, respectively), five deletions, four insertions, two gene rearrangements by fluorescence in situ hybridization (FISH), three MET amplifications by FISH (two equivocal and one amplified), and three KIT sequence alterations by high-resolution melting curve analysis. High-resolution melting curve analysis detects sequence alterations, such as SNVs, MNVs, and insertions and deletions (indels), but does not discern the type of variant. One melanoma sample contained both a KIT sequence alteration and an SNV, and one NSCLC sample contained two concomitant SNVs.

Tissue and DNA QC

Sequentially unstained FFPE sections (5 μm thick) were mounted onto glass slides preceded and succeeded by hematoxylin and eosin (H&E)–stained sections. Both H&E slides were examined for viable tumor content and scored for percentage of tumor nuclei by a single pathologist who circled areas of neoplastic cells. Only specimens with estimated tumor nuclei ≥10% (≥5% diploid heterozygous variants) in circled areas were included in the study. A consensus H&E review was performed for all samples with ≤10% tumor nuclei. For all samples when technically feasible, portions of the unstained sections corresponding to the circled tumor fractions were manually macrodissected from nontumoral areas using a scalpel. For samples where tumor cells could not easily be enriched from nontumor cells (eg, cell blocks or small biopsy specimens), whole unstained sections were scraped only if tumor nuclei comprised ≥10% of the total sample nuclei. DNA was extracted from a minimum of four unstained slides using Qiagen FFPE DNA extraction kits (Qiagen Inc., Valencia, CA). Sample DNA and TST QC control DNA specimens were amplified in triplicate using SYBR Green quantitative PCR (qPCR; Fast SYBR Green Master Mix; Life Technologies, Grand Island, NY), and ΔqCT values were calculated as follows: ΔqCT = Mean Sample qCT − Mean Control qCT. A mean ΔqCT ≤6.0 was considered adequate for TST library preparation (see Results).

Multiplexed Allele-Specific PCR (SNaPshot)

Forty NSCLC and two GI samples underwent multiplex PCR, primer extension, and capillary electrophoresis to detect 44 somatic hotspot mutations in 10 genes (AKT1, BRAF, EGFR, IDH1, IDH2, KRAS, MAP2K1, NRAS, PIK3CA, and PTEN), as described.¹³ IDH1 exon 4 and IDH2 exon 4 were amplified using the following primers: 5′-GGTTGAGGAGTTCAAGTTG-3′ (forward) and 5′-AGTTGGAAATTTCTGGGCCAT-3′ (reverse), and 5′-GCTGCAGTGGGACCACTATT-3′ (forward) and 5′-TGTGGCCTTGTACTGCAGAG-3′ (reverse), respectively. Single-base extension primers for IDH1 (5′-AAAAAAAAAAAACTTACTTGATCCCCATAAGCATGA-3′ and 5′-AAAAAAAAAAAAAAAAAAACTTACTTGATCCCCATAAGCATGAC-3′) and IDH2 (5′-AAAAAAAAAAAAAAAAAAAAAAAAAAAAGACCAAGCCCATCACCATTGGCA-3′) interrogated specific IDH1 codon 132 and IDH2 codon 172 mutations. Supplemental Table S1 provides the complete list of tested mutations. The defined limit of detection (LOD) was 10% for diploid heterozygous variants. DNA templates containing the nucleotide variants were spiked in at 10% concentration as positive controls.

EGFR Exons 19 and 20 and ERBB2 (HER2) Fragment-Length Analysis

Forty NSCLC and two GI samples underwent EGFR and ERBB2 testing for insertions and deletions (indels). Briefly, FFPE-extracted DNA templates were amplified using fluorescently labeled reverse primers flanking EGFR exons 19 (5′-GCACCATCTCACAATTGCCAGTTA-3′ and 5′-/6FAM/AAAAGGTGGGCCTGAGGTTCA-3′) and 20 (5′-TCTTCACCTGGAAGGGGTCC-3′ and 5′-/HEX/ACGGTGGAGGTGAGGCAGAT-3′) and ERBB2 exon 20 (5′-ACCGTGCCCGGCCTAATCTT-3′ and 5′-/HEX/ACGGTGGAGGTGAGGCAGAT-3′). Labeled DNA products underwent capillary electrophoresis after formamide treatment on the ABI 3130 (Life Technologies, Grand Island, NY). The output data were interpreted via GeneMapper software version 3.7 (Life Technologies). A shift from the size of wild-type amplicon peaks indicated the presence of an insertion or deletion. Known positive and negative controls were included in each run.

FISH for ALK, ROS1, and RET Gene Rearrangements and MET Amplification

One or more FISH tests were performed in the Emory Cytogenetics Laboratory on 31 NSCLC samples using up to eight DNA probes (break-apart construction) to detect rearrangements of ALK at 2p23 (Abbott Molecular, Inc., Abbott Park, IL), ROS1 at 6q22 (Cytocell, Ltd, Cambridge, UK), RET at 10q11.2 (Abbott Molecular, Inc.), and/or MET at 7q31.2 and CEP7 (Abbott Molecular, Inc.). Each probe set was hybridized to a section of the FFPE tissue. Approximately 200 interphase cells were evaluated with ALK, ROS1, and RET probe sets. Approximately 50 interphase cells were evaluated with a MET probe set. All cells were scored by manual methods.

BRAF AS-PCR

Fifteen melanoma samples were tested by allele-specific PCR (AS-PCR) at a reference laboratory using the US Food and Drug Administration–approved Cobas 4800 BRAF V600 Mutation Test (Clarient Diagnostic Services, Aliso Viejo, CA).

BRAF, KRAS, and NRAS Pyrosequencing

Nine melanoma and six GI samples underwent BRAF pyrosequencing in the Emory Molecular Diagnostic Laboratory (Atlanta, GA), as previously described.¹⁴ Six GI samples underwent KRAS pyrosequencing in the Emory Molecular Diagnostic Laboratory targeting codons 12 and 13, as described.¹⁵ All NRAS pyrosequencing (six melanoma samples) to detect sequence variations in NRAS codons 12, 13, and 61 was performed at a reference laboratory (ARUP Laboratories, Salt Lake City, UT).

KIT High-Resolution Melting Curve Analysis

High-resolution melting curve analysis for three melanoma samples to detect sequence variations in KIT exons 9, 11, 13, and 17 was performed at a reference laboratory (ARUP Laboratories).

PTEN and MET IHC

PTEN and MET immunohistochemistry (IHC) were performed on 32 NSCLC samples using monoclonal antibodies to PTEN (Cell Signaling, Beverly, MA; clone 138G6, 1:200) and MET (Spring Bioscience, Pleasanton, CA; clone SP44, no dilution). IHC epitope retrieval, staining, incubation, and slide preparation were performed as described.¹⁶ PTEN IHC scores were reported as intact (staining present, expected) or lost (staining absent, aberrant). Positive MET IHC scores were determined by >50% of tumor cells expressing 2+/3+ staining intensity on a 0 to 3+ scale. All IHC scoring was performed by a single pathologist (G.L.S.).

TST Library Preparation, Quantification, Normalization, and Sequencing

NGS library preparations were prepared using the TST kit (Illumina), per the manufacturer's protocol. In brief, 20 μL of neat or diluted sample DNA was prepared on the basis of initial ΔqCT values, per manufacturer's recommendations. TST library preparation generates two complementary NGS libraries by targeting the positive and negative DNA strand for each tumor sample with two separate complementary oligo pools (FPA and FPB). FPA and FPB oligonucleotide pools were hybridized to sample DNA overnight, and targets were ligated, size selected via membrane filtration, and PCR amplified with adaptors containing unique index sequences (barcodes) and sequencing adaptors. Each tandem FPA and FPB library contained 174 amplicons covering 85 exonic regions in 26 cancer-related genes (Supplemental Tables S1 and S2). Amplicon libraries were purified using Agencourt AMPure XP magnetic beads (Beckman-Coulter, Indianapolis, IN).

Libraries were quantified in triplicate using KAPA Library Quantification Kits (KAPA Biosystems, Wilmington, MA), per manufacturer's recommendations. KAPA Illumina quantification kits specifically amplify DNA that contains the MiSeq/HiSeq universal primers by using forward and reverse qPCR primers complementary to these known sequences (primer P1, 5′-AATGATACGGCGACCACCGA-3′; primer P2, 5′-CAAGCAGAAGACGGCATACGA-3′). The provided six standards range from 20.0 nmol/L to 0.2 pmol/L and consist of 452-bp linear DNA fragments flanked by the Illumina universal primers. Complementary sample FPA and FPB libraries were diluted to 1:1,000,000 using 10 mmol/L Tris-HCl, pH 8.0 + 0.05% Tween 20 solution to fit the linear range of the standards. Diluted tandem libraries were mixed with the KAPA SYBR FAST qPCR Master Mix containing Primer Premix and run in triplicate on the ABI7500 thermocycler (Life Technologies, Grand Island, NY). The six standards were run in duplicate to generate the standard curves. Average sample concentrations were calculated from the standard curve. All libraries were diluted to 4 nmol/L concentrations using Tris-EDTA buffer for storage before sequencing on the MiSeq benchtop sequencer. TST library preparations were considered successful if at least 4 nmol/L of FPA and FPB library was generated for each sample.

Sequencing was performed per the manufacturer's instructions. Briefly, 4 nmol/L tandem libraries (FPA and FPB) was diluted to 10 pmol/L concentrations, pooled, and spiked with 2% PhiX diversity cluster control (Illumina). Complementary sample FPA and FPB libraries were always sequenced simultaneously on the same flow cell. Ten multiplexed runs of 4, 8, and 12 samples (8, 16, and 24 complementary libraries) were attempted in 1, 4, and 5 individual sequencing runs, respectively. Libraries composed of DNA extracted from leukocytes of two healthy donors were used as negative mutation controls. Positive mutation control libraries were identified prospectively after TST confirmation of the reference standard mutation result.

TST Sequencing QC and Variant Detection

Sequencing runs were considered successful when four PhiX control FASTQ libraries were generated (pool A, two files; pool B, two files), and overall phred scores of Q30 or greater (one misaligned base per 1000 bases) were seen in ≥85% of reads for each run. Sequence alignment, index and primer trimming, and variant calls were performed using on-board MiSeq software. Four FASTQ files, two BAM and BAM index files, and a single genomic and nongenomic variant call file were generated for each sample. A custom Java program (open source software, downloadable at http://github.com/ghsmith/coverageQc, last accessed November 5, 2015) parsed the read depths from the genomic variant call file to generate read depth histograms for each region, where the histogram bins corresponded to read depth intervals (<100, 100 to 499, 500 to 999, and >999 reads). Primer sequences were trimmed by the workflow and not included in the read coverage data. Regions were defined by bed file coordinates. QC criteria for read depth were established and evaluated using these histograms; a region with any base read <500× was considered to have failed sequencing QC.

Genomic variants were annotated using the Illumina VariantStudio 2.1 MiSeq TST Amplicon-DS workflow. All variants with <3.0% variant allele frequency (VAF) were automatically filtered by the workflow before technologist review. The remaining variants were reviewed and filtered to exclude synonymous variants and to include variants affecting coding sequences: missense, frameshifts, stop gained, stop lost, initiator codons, in-frame insertions, and in-frame deletions. The filter also retained splice/intronic variants within the eight flanking base pairs proximal and distal to the coding exons. The variant call file also included strand bias and probe pool bias calls, which were considered during interpretation of variants but were not used for filtering variants before analysis. Strand bias refers to differences between positive and negative strand reads, where a variant allele is disproportionately represented on one strand in paired-end read mode. Pool bias is unique to TST and occurs when a base pair change is present in one strand but not the other (eg, an FFPE deamination artifact). This discordance between the FPA and FPB pool sequences is flagged as pool bias (Illumina reference guide: MiSeq Reporter Amplicon–DS Workflow. San Diego, CA). During analysis of variants, an interpretive variant cutoff of 5% was used for SNVs and insertions/deletions, as described in Results.

Fluidigm Access Array Assay

The use of the Fluidigm 48.48 Access Array Integrated Fluidic Circuit with FFPE tumor specimens has been described in detail elsewhere.¹⁷ We custom designed a lung adenocarcinoma research assay to query the entire coding region of STK11, TP53, PTEN, KRAS, and NRAS, exons 11 and 15 of BRAF, exons 18 to 21 of EGFR, and other clinically relevant coding portions of MET, PIK3CA, and ERBB2. Briefly, DNA aliquots from 13 lung tumor specimens used for the TST assay were made available for the Access Array library preparation and Illumina MiSeq sequencing. Duplicate libraries were prepared for the Access Array, according to the manufacturer's protocol, from each tumor specimen using approximately 4000 functional DNA copies (±1000 copies) estimated using real-time PCR, as described.¹⁷ The libraries were sequenced (2 × 150, 300 cycles) in the same Illumina MiSeq sequencing run using the version 2 reagent kit. Barcoded specimens we demultiplexed and FASTQ reads were processed into annotated variant calls using a custom Galaxy workflow (M.R.R., unpublished data) that used Bowtie-FreeBayes/VarScan-ANNOVAR. Annotated variants and aligned sequences (BAM) were manually inspected and compared with the TST data from the same specimen using the Integrative Genomics Viewer (IGV) version 2.3 (IGV2.3; Broad Institute, Cambridge, MA).

HaloPlex Cancer NGS

One melanoma sample (MEL-9) was concomitantly tested with HaloPlex Cancer NGS (Agilent Technologies, Santa Clara, CA). The sample was prepared per the manufacturer's protocol and sequenced on the Illumina MiSeq. Briefly, 500 ng of sample genomic DNA underwent restriction enzyme digestion, and products were analyzed on a TapeStation bioanalyzer (Agilent Technologies). The restriction fragments were hybridized to the HaloPlex probe capture libraries at the designated hybridization time, and Illumina sequencing motifs, including index sequences, were incorporated into the targeted fragments. Target fragments were PCR amplified and analyzed on the TapeStation to assess library yield. Supplemental Table S1 provides a complete list of targeted regions. The samples were run with provided control samples on the Illumina MiSeq, as recommended. FASTQ files were analyzed and variants were detected using Agilent SureCall version 2.0 analysis software (Agilent Technologies).

Alternative Bioinformatics Analysis Workflow

FASTQ files from 68 samples (27 NSCLCs, 18 melanomas, and 23 GI specimens) that harbored TST-detected variants were analyzed using a custom variant calling pipeline aimed to minimize false-positive calls by removing sequencing artifacts and mismapped reads before variant calling. Overlapping paired-end reads were merged into a single read using ea-utils fastq-join version 1.1.2-686 (https://code.google.com/p/ea-utils/wiki/FastqJoin, last accessed November 5, 2015) retaining the highest-quality score from base calls in overlapping regions. Low-quality bases and adapters from the merged reads were trimmed using Trimmomatic 0.32 (http://www.usadellab.org/cms/?page=trimmomatic, last accessed November 5, 2015). Merged, trimmed reads were mapped to the hg19 human reference genome using the BWA-mem aligner version 0.7.10,¹⁸ and low confidence alignments (mapping quality, <40) were filtered using Samtools.¹⁹ Variant calling was performed using VarScan2²⁰ or GATK2.8²¹ (https://www.broadinstitute.org/gatk, last accessed November 5, 2015), requiring a minimum variant frequency of 1% and minimum coverage of 100×. Variants were annotated using ANNOVAR.²² Next, base pair changes resulting in synonymous amino acid changes, or variants detected in bases or regions with <500× coverage and/or with VAFs <3.0%, were manually excluded to mimic the TST variant-calling criteria. Two individuals (K.E.F. and S.N.) reviewed the remaining variants to assess discordant calls.

LOD Analysis

To assess sensitivity and define the lower LOD, undiluted and diluted libraries were prepared from two NSCLC samples with 40% tumor nuclei: one sample with a KRAS c.35G>T, p.G12V mutation, and a second sample containing both an EGFR c.2264C>A, p.A755D SNV and an EGFR exon 19, 18-bp deletion. Both samples harbored mutations at approximately 20% VAF, as estimated by SNaPshot or fragment-length analysis (FLA) peak height. Tumor and matched nonneoplastic DNA samples were macrodissected from tumor and nontumor areas of the same paraffin slide section; matched nontumor DNA was used as the diluent. The NSCLC sample with the two EGFR mutations was diluted 1:2 and 1:4, resulting in approximately 10% and 5% VAF dilutions, respectively. Similarly, the KRAS mutation sample DNA was diluted in ratios of 1:2.5 and 1:7, resulting in approximately 8% and 3% VAF dilutions, respectively. Libraries for both samples and their respective dilutions (6 samples, 12 libraries) were prepared as described, uniquely indexed, pooled, and multiplexed in a run of 10 samples total: 6 LOD samples with 4 additional controls/samples.

Reproducibility Analysis

To assess reproducibility in the TST assay, library preparations from five previously prepared and tested samples containing heterozygous single-nucleotide polymorphisms, homozygous single-nucleotide polymorphisms, and variants of interest (three SNVs, one deletion, and one insertion, respectively) were repeated in separate multiplexed runs on separate days by two different technologists (four runs total). All samples were sequenced in multiplexed runs containing 10 samples. The first run contained the three SNV samples, and the second run contained the indel samples. The technologists swapped runs so that each completed a SNV and an indel run. VAFs were recorded for each run and compared using relative deviation, calculated as the absolute value of the following: (VAF Run 1 − VAF Run 2)/[(VAF Run 1 + VAF Run 2)/2]. A third run to calculate SD was not feasible because of finite resources.

Results

Specimen Adequacy for TST NGS Library Preparation

The TST assay does not use standard spectrophotometric or fluorometric methods, such as Nanodrop or Qubit, respectively, to determine optimal or minimal starting DNA concentrations. Instead, qPCR is used to predict specimen performance, and the amount of input genomic DNA is determined on the basis of ΔqCT scores that are calculated compared with a non-FFPE reference DNA standard (Illumina white paper, sequencing: Generating Sequencing Libraries from FFPE Samples. San Diego, CA). Library preparations are optimized for samples with ΔqCT values ≤4.0, and dilutions are recommended for samples with ΔqCT values <1.5. Sample dilutions are not recommended when ΔqCT values are ≥1.5.

However, a significant number of samples tested (n = 38) generated ΔqCT values >4.0. To decrease potential ΔqCT failures, we attempted library preparations for all specimens with ΔqCT values ≤6.0 (n = 84). We used standard input volumes (20 μL total of undiluted genomic DNA) for all samples with ΔqCT values ≤6.0 but >4.0. Libraries were successfully prepared for all 62 samples with ΔqCT values ≤4.0, and 76 (90.5%) of 84 samples with ΔqCT values ≤6.0, indicating that a ΔqCT value ≤6.0 is a practical adequacy requirement for TST library preparation. Most of the specimens [13 (65.0%) of 20] that failed library preparation (n = 4) or demonstrated ΔqCT values >6.0 (n = 16) were fine-needle aspiration samples, biopsy specimens, or body fluid cytospin cell blocks, suggesting that scant tissue and/or nonuniform sample processing may have contributed to low ΔqCT ratios and/or poor performance in the assay (Table 1).

Table 1.

Assessment of DNA Adequacy by Quantitative PCR

ΔqCT value	Total passed (%), n = 80	Total failed (%), n = 20	Scant specimens passed (%), n = 29∗	Scant specimens failed (%), n = 13
≤4.0	62/62 (100.0)	0/62 (0.0)	17/17 (100.0)	0/17 (0.0)
4.0 < ΔqCT ≤6.0	18/22 (81.8)	4/22 (18.2)	12/15 (80.0)	3/15 (20.0)
>6.0†	0/16 (0.0)	16/16 (100.0)	0/10 (0.0)	10/10 (100.0)

Formalin-fixed, paraffin-embedded (FFPE) sample qCT values were calculated compared with qCT values from non-FFPE reference DNA (ΔqCT = mean sample qCT − mean control qCT). Library preparations were attempted for all specimens with ΔqCT values ≤6.0.

^∗Fine-needle aspiration cell blocks, biopsy specimens, or body fluid cytospin cell blocks.

^†Library preparations were not attempted for samples with ΔqCT >6.0.

Assessment of Sequencing Depth of Coverage

At least 400 quality reads are necessary to detect a heterozygous variant with 99.9% sensitivity in a cancer specimen containing 10% neoplastic nuclei.²³ Using this estimate as a benchmark, we required that variant bases be sequenced at ≥500× or ≥1000× coverage to detect variant frequencies of ≥10.0% or ≥5.0%, respectively (50 variant calls in ≥500 or ≥1000 total base reads, respectively). We only sequenced tumor DNA composed of at least 10% neoplastic nuclei (light microscopy approximation, see Materials and Methods), mandating a minimum variant fraction detection threshold of approximately 5% for diploid heterozygous mutations. We defined sequencing QC failure as any base in the specified targeted region sequenced at a depth <500×, and these metrics were obtained and analyzed using a Java-based, laboratory-developed program (see Materials and Methods).

In the first pilot experiment using four NSCLCs, the TST-recommended maximum number of multiplexed samples, the MiSeq generated 1.15 GB of sequencing data per sample, sequenced 99.9% (13,952 of 13,964) of the targeted bases with ≥1000× coverage, and generated ≥Q30 phred scores in 94.5% of reads overall. The same 12 bases (0.01%) failed sequencing QC in every sample, the nine initial bases at the 5′ end of STK11 exon 6 and the last three bases at the 3′ end of TP53 exon 4. On closer inspection, these 12 bases were not sequenced (no reads) in any of the four samples in the pilot or in the subsequent 76 validation samples. We confirmed that the provided TST bed file was incorrect and these bases were actually not targeted by the assay. Therefore, we excluded these 12 bases from further QC analysis, leaving a total of 13,952 targeted bases.

Next, we analyzed the sequencing depths of select tumor-specific bases or regions that were routinely tested by non-NGS methods in our clinical practice to ensure these regions were adequately covered by TST (Supplemental Tables S1 and S2). We analyzed the sequencing depths at KRAS c.35 (p.G12, routinely tested in NSCLC and GI malignancies), BRAF c.1799 (p.V600, routinely tested in melanoma and GI malignancies), and all bases in EGFR exon 19 (for exon 19 deletions) and EGFR c.2573 (p.L858), which were routinely tested in NSCLC. The minimum coverage of these four select routinely tested bases or regions in the pilot experiment was 41,909×, 16,426×, 8565×, and 8687× for BRAF c.1799, KRAS c.35, any base in EGFR exon 19, and EGFR c.2573, respectively. The maximum coverage ranged from 16,406× to 55,289× (Supplemental Figure S1A). The pilot data confirmed that sufficient sequencing coverage and target specificity could be obtained using TST for the proposed variant calling thresholds, particularly for common, routinely tested bases or regions.

Effects of Sample Multiplexing

TST had excellent performance when the recommended four multiplexed samples were sequenced simultaneously on a single flow cell. However, given our anticipated test volume of approximately 6 to 10 samples per week, we reasoned that for practical throughput purposes, sequencing of 8 to 12 samples on a single flow cell was wanted. To test whether similar performance metrics could be obtained when multiplexing 8 to 12 samples, we analyzed QC data from 4 multiplexed runs of 8 samples (26 individual samples total plus controls) and 5 multiplexed runs of 12 samples (50 individual samples total plus controls).

In the entire validation data set of 80 patient samples, 35 regions passed QC in every sample, 15 regions failed sequencing QC in >5 samples (Figure 1A and Supplemental Table S2), and only 3 samples failed sequencing QC (<500×) at the 4 select routinely tested regions (Figure 1B). Furthermore, 100% of bases demonstrated >100× coverage in 75 samples, and >99.3% of bases demonstrated >100× coverage in the remaining 5 samples (Supplemental Table S3). However, multiplexing of samples appeared to influence sequencing coverage. The average (0.0 versus 2.4 versus 5.0) (Figure 1C) and total number (0 versus 21 versus 50) (Supplemental Figure S1, B and D) of failed regions per sample increased when we multiplexed more samples, and the number of samples with base failures in routinely tested regions also increased (0 versus 0 versus 3) (Supplemental Figure S1, A, C, and E). All three samples with sequencing failures in currently tested regions (one KRAS c.35 and two EGFR exon 19) were sequenced in multiplexed runs of 12 samples.

Figure 1.

TruSight tumor (TST) generates sufficient depth of coverage for multiplexed samples. A: Histogram of sequencing coverage of all 80 TST samples by the 85 targeted regions. A region failed quality control (QC) if a single base within the region was sequenced <500×. B: Box-and-whiskers plot showing depth of coverage of select routinely tested bases for all samples. C: Histogram showing base coverage by multiplexed run. A total of 13,952 bases (N) in the 85 regions were assessed for QC purposes. Asterisk indicates the 12 bases that were not sequenced in any sample were excluded from analysis (9 bp of STK11 ex6 and 3 bp of TP53 ex4); dagger, Supplemental Table S2 provides a full list of regions. N = 80 (A and B); N = 80 (all), 4 (4), 26 (8) and 50 (12). Avg., average; ex, exon; Max, maximum; Min, minimum; MPx, multiplexed; Reg, regions.

Given the increased number of sequencing QC failures with increased multiplexed samples, we assessed the effects of multiplexing samples on sequencing coverage of all 20 routinely tested bases/regions specific to the tumor type (Supplemental Table S1). Notably, tumor-specific QC sequencing failures were seen only in three NSCLC and three melanoma samples; all GI samples passed sequencing QC at routinely tested loci, and only one NSCLC sample failed sequencing QC at more than one routinely tested region (Table 2). KIT exon 17 (n = 3), ERBB2 exon 20 (n = 2), and AKT1 c.49, p.G17 (n = 2) were the only routinely tested regions to fail sequencing QC in their respective tumor types (Supplemental Table S3). Samples with sequenced in multiplexed runs of 12 with QC failures typically had more failed regions per sample (nine on average) when compared with samples with QC failures that were sequenced in multiplexed runs of eight (four on average) (Supplemental Table S3). The details of all regions that were sequenced <500× and <100× for each sample are shown in Supplemental Table S3.

Table 2.

QC Failures of Previously Tested Regions by Specimen Type

Sample ID	Multiplex	Failed regions (depth)	Total bases read ≥500× (%)
NSCLC-20	12	AKT1 c.49, p.G17 (441×)	91.06
		ERBB2 exon 20 (min 222×)
NSCLC-32	12	ERBB2 exon 20 (min 391×)	97.05
NSCLC-14	8	AKT1 c.49, p.G17 (76×)	98.95
MEL-6	12	KIT exon 17 (min 397×)	82.49
MEL-19	12	KIT exon 17 (min 414×)	86.81
MEL-21	12	KIT exon 17 (min 286×)	92.58

Sequencing QC failures were defined as any base in the specified capture region sequenced at a depth <500×. N = 32 for NSCLC, 23 for melanoma, and 25 for GI. Samples with failed region = 3 (9.4%) for NSCLC, 3 (13.0%) for melanoma, and 0 (0%) for gastrointestinal (GI). Failed routinely tested regions = 4 for NSCLC, 3 for melanoma, and 0 for GI. Routinely tested regions were defined as bases or exons specific to the tumor type tested that were routinely sequenced by non–next-generation sequencing methods in clinical practice (Supplemental Tables S1 and S2 provide a complete list by tumor type).

MEL, melanoma; Min, minimum; NSCLC, non-small cell lung cancer; QC, quality control.

Collectively, we concluded that acquiring high-quality data from 8 to 12 multiplexed TST samples was feasible, but a few important bases/regions may yield insufficient sequencing coverage for clinical interpretation, particularly with higher numbers of multiplexed samples. To ensure comparable clinical testing of NGS compared with previously used methods, we devised a laboratory testing algorithm to handle specimen and sequencing QC failures in various stages of the testing pipeline (Supplemental Figure S2). For clinical testing, we recommended that specimens that fail QC be either repeated or tested with a non-NGS modality and capped all sequencing runs at 10 samples total per flow cell to ensure optimal depth of coverage.

Assessment of Variant Detection Sensitivity and Reproducibility

To test the sensitivity of variant detection near our proposed LOD of 5% VAF for diploid heterozygous mutations, we sequenced serially diluted DNA from two NSCLC samples with approximately 40% tumor and 20% heterozygous VAFs, as determined by FLA or SNaPshot peak heights. One NSCLC harbored both an EGFR exon 19, 18-bp deletion and an EGFR exon 19 SNV missense mutation, and one NSCLC harbored a KRAS exon 2 SNV missense mutation (Table 3). New libraries were prepared and sequenced after dilution in DNA extracted from paired nonneoplastic tissue adjacent to the tumor in multiplexed runs of 10 samples, including controls, as described (see Materials and Methods). The expected serial reductions in VAF were seen, and the lowest detected VAFs were 3.2% for an SNV and 4.7% for a deletion (Table 3).

Table 3.

Limit of Detection Analysis Using Serial Dilutions

Sample	TST gene	TST mutation	Total reads	Variant reads	VAF, %
EGFR Ex19 del18	EGFR	p.Leu747_Pro753delinsSer	5275	898	17.0
EGFR Ex19 del18 1:2	EGFR	p.Leu747_Pro753delinsSer	4406	414	9.4
EGFR Ex19 del18 1:4	EGFR	p.Leu747_Pro753delinsSer	3347	156	4.7
EGFR Ex19 SNV	EGFR	p.Ala755Asp	10,491	1849	17.6
EGFR Ex19 SNV 1:2	EGFR	p.Ala755Asp	8721	862	9.9
EGFR Ex19 SNV 1:4	EGFR	p.Ala755Asp	6596	336	5.1
KRAS Ex2 SNV	KRAS	p.Gly12Cys	2026	394	19.5
KRAS Ex2 SNV 1:2.5	KRAS	p.Gly12Cys	3961	343	8.7
KRAS Ex2 SNV 1:7	KRAS	p.Gly12Cys	2674	85	3.2

Two non-small cell lung cancer samples with 40% tumor and heterozygous mutations (one sample contained both EGFR mutations) were diluted with matched nontumor DNA. The expected variant dilutions were observed. The lowest demonstrable VAFs were 3.2% for an SNV and 4.7% for the deletion.

Del, deletion; Ex, exon; SNV, single-nucleotide variant; TST, TruSight tumor; VAF, variant allele frequency.

Next, we investigated whether TST could detect variants at the approximate LOD of select reference methods (Supplemental Table S4): multiplexed AS-PCR (SNaPshot, ≥10% VAF), BRAF AS-PCR (>5% VAF per reference report), FLA (≥10% VAF), and pyrosequencing (≥5% VAF). TST confirmed both the FLA-detected EGFR insertion and deletion at 8.4% and 13.0% VAF, respectively. TST also confirmed an AS-PCR–positive BRAF p.V600E mutation with a low VAF (9.1% VAF was the lowest in our cohort), a pyrosequencing-positive KRAS p.G12D mutation (4.8% VAF), and a SNaPshot-positive EGFR SNV missense mutation (8.1% VAF). These results indicated that TST offered comparable and, in some instances, slightly improved analytic LODs to the reference methods. We defined our TST analytical LOD as ≥3.2% for SNVs and ≥4.7% for insertions/deletions (indels).

To assess reproducibility of the TST assay, repeat sequencing runs were performed with five samples harboring known SNV missense mutations (two KRAS and one KIT), a known EGFR 15-bp exon 19 deletion, and a known EGFR 3-bp exon 20 insertion. For each sample, the variant of interest and accompanying heterozygous or homozygous single-nucleotide polymorphisms were detected at similar VAFs to the initial run (Table 4), indicating that TST generated reproducible variant detection. However, relative deviations in VAF were more pronounced for SNVs with VAFs <5% (the maximum deviation detected was 29.89%). Assuming a relative deviation of 30%, we calculated that a minimum VAF of 4.4% was required to reproducibly detect variants at >3.0%, the VAF cutoff used for TST Amplicon–DS variant calling. Thus, we defined the clinical LOD for SNVs/MNVs and indels as ≥5% to account for these potential deviations.

Table 4.

Reproducibility Analysis

Sample	TST gene	Type	TST mutation	Run 1			Run 2			Relative deviation (%)
Total reads	Variant reads	VAF, %	Total reads	Variant reads	VAF, %
GI-18	APC	Het SNP	c.4479G>A(p.=)	3018	1481	49.1	10,289	5092	49.5	0.81
	TP53	Hom SNP	p.Pro72Arg	1834	1829	99.7	6277	6255	99.7	0.00
	KRAS	Variant	p.Gly12Arg	2099	92	4.4	7224	271	3.8	14.63
NSCLC-12	KIT	Het SNP	c.2586G>C(p.=)	11,999	5552	46.3	12,357	5876	47.6	2.77
	TP53	Hom SNP	p.Pro72Arg	22,326	22,217	99.5	22,755	22,627	99.4	0.10
	KRAS	Variant	p.Gly12Val	12,871	658	5.1	12,462	626	5.0	1.98
NSCLC-22	TP53	Het SNP	p.Pro72Arg	2380	1249	52.5	1000	490	49.0	6.90
	EGFR	Hom SNP	c.2361G>A(p.=)	2172	2156	99.3	948	943	99.5	0.20
	KIT	Variant	p.Leu859Pro	2716	136	5.0	1231	46	3.7	29.89
EGFR Ex19 DEL	EGFR	Het SNP	c.2361G>A(p.=)	6062	3118	51.4	1961	993	50.6	1.56
	TP53	Hom SNP	p.Pro72Arg	18,297	18,197	99.5	5272	5247	99.5	0.00
	EGFR	Variant	p.Glu746_Ala750del	11,479	1648	14.4	3709	537	14.5	0.69
EGFR Ex20 INS	APC	Het SNP	c.4479G>A(p.=)	28,261	14,077	49.8	10,096	4895	48.5	2.64
	TP53	Hom SNP	p.Pro72Arg	19,738	19,622	99.4	6603	6575	99.6	0.20
	EGFR	Variant	p.Asp770_Asn771insGly	16,863	2931	17.4	5581	998	17.9	2.83

Homozygous and heterozygous SNPs and the accompanying low-level variants were detected in the repeat run at similar VAFs. Relative deviation was calculated as the absolute value of the following: VAF Run 1 − VAF Run 2/(VAF Run 1 + VAF Run 2)/2.

DEL, deletion; Ex, exon; Het, heterozygous; Hom, homozygous; INS, insertion; NSCLC, non-small cell lung cancer; SNP, single-nucleotide polymorphism; TST, TruSight tumor; VAF, variant allele frequency.

TST Detection of Variants

The TST-targeted, amplicon-design method is optimized to detect SNVs, MNVs, and indels, but not chromosomal structural alterations, such as translocations or copy number changes. TST libraries were successfully prepared from 33 archival FFPE samples with 33 known potentially detectable variants: 24 SNVs, 4 insertions, 4 deletions, and 1 MNV. TST confirmed all 33 previously detected alterations (Figure 2, Table 5, and Supplemental Table S5). Two BRAF p.V600E mutations, detected by AS-PCR, sequenced as BRAF p.V600K MNVs by TST NGS; BRAF p.V600K MNVs are known to cross-react in the AS-PCR assay.¹⁶ Five melanoma samples that did not harbor BRAF p.V600E mutations by pyrosequencing or AS-PCR and one RET-rearranged NSCLC by FISH that harbored no mutations by SNaPshot or FLA were wild type at the tested loci by TST (Figure 2, Table 5, and Supplemental Table S5).

Figure 2.

Detectable variants in the TruSight tumor (TST) validation cohort. Data are shown for the 40 non-small cell lung cancer (NSCLC), 30 melanoma, and 30 gastrointestinal (GI) formalin-fixed, paraffin-embedded samples that were selected for TST validation. TST next-generation sequencing (NGS) was performed successfully on 80 samples. TST detected 112 variants in 68 samples, and confirmed all previously detected single-nucleotide variants (SNVs), multinucleotide variants (MNVs), insertions, and deletions. For details of specific alterations, see Supplemental Table S5 and Figure 3. Chr, chromosome; CNA, copy number alteration; FISH, fluorescence in situ hybridization; FLA, fragment-length analysis for EGFR exons 19 and 20 and ERBB2 exon 20; FLDM, Fluidigm Access Array; FNA CB, fine-needle aspiration cell block; HPX, HaloPlex Cancer; IHC, immunohistochemistry; KIT-HRM, KIT exon 9, 11, 13, and 17 mutation analysis by high-resolution melting curve; NPT, no previous testing.

Table 5.

TST Detection of Variants

Variable	Detected by TST and confirmed by second method, n = 49	Detected by TST only and missed by second method, n = 4	Detected by TST only and no additional testing, n = 71
SNVs, N = 89	27	4	58
MNVs, N = 4	3∗	0	1
Deletions, N = 12	5	0	7
Insertions, N = 7	4	0	3
NMD, N = 12	10†		2

All 33 known potentially detectable variants (24 SNVs, 4 insertions, 4 deletions, and 1 MNV) were detected by TST. In total, 39 variants were confirmed by a second method. TST detected 73 novel variants that were not confirmed by a second method, including four KRAS variants undetected by SNaPshot. Figure 2 and Supplemental Table S5 provide a complete list of mutations.

MNV, multinucleotide variant; NMD, no mutation detected; SNV, single-nucleotide variant; TST, TruSight tumor.

^∗Two BRAF p.V600K MNV samples were detected as p.V600E in the allele-specific PCR reference method (cross-reactive).

^†Select wild-type TST loci were wild-type by comparison methods.

To further confirm the specificity of the NGS findings, 14 samples were sequenced by secondary NGS library preparation methods, either Fluidigm Access Array (FLDM; n = 13 NSCLC samples) or HaloPlex Cancer NGS (n = 1 melanoma sample), both of which were sequenced using the Illumina MiSeq. Each orthogonal NGS method used alternative library preparation methods and independent alignment and variant calling pipelines. For FLDM testing, five NSCLC samples with routinely tested SNVs (three KRAS, one EGFR, and one BRAF), four NSCLC samples that harbored no mutations by SNaPshot, FLA, or TST, and four NSCLC samples that harbored mutations only detected by TST (two TP53 SNVs, one AKT1 SNV that was equivocal by SNaPshot testing, and one TP53 12-bp deletion) were sequenced. FLDM NGS confirmed the presence or absence of all comparable mutations in all 13 samples (Figure 2 and Supplemental Tables S1 and S5), including the equivocal AKT1 SNaPshot result. HaloPlex Cancer NGS confirmed both the NRAS p.G60E and PTEN p.A22V SNVs detected by TST (Supplemental Table S5) but failed to detect the KIT exon 11 insertion detected by KIT high-resolution melting curve analysis in the melanoma sample tested (MEL-9).

In addition, TST NGS revealed four potential false-negative KRAS SNaPshot results. In three of four NSCLC samples, the SNV was detected at or lower than the LOD of the SNaPshot assay (Supplemental Table S5). The TST assay demonstrated better sensitivity for detecting low-level variants (eg, <10% VAF), so we attributed these discrepant results to false-negative SNaPshot results. However, for one sample (NSCLC-4), TST detected a KRAS p.G13C variant at a VAF of 30.8%, a VAF readily discernible by SNaPshot. Repeat SNaPshot testing again failed to detect the variant (data not shown), but no residual material was available to perform confirmatory KRAS sequencing. Therefore, we could not determine with certainty whether this sample represented a SNaPshot false-negative or a TST false-positive result. Regardless, TST NGS had 100% concordance for detecting orthogonally confirmed SNVs, MNVs, indels, and wild-type loci in our validation cohort (Table 5).

In all, TST NGS detected 112 variants (89 SNVs, 12 deletions, 7 insertions, and 4 MNVs), and at least 1 variant in 85.0% (68 of 80) and 2 or more variants in 36.2% (29 of 80) of samples. Thirty-nine variants, including the two cross-reactive BRAF mutations (27 SNVs, 5 deletions, 4 insertions, and 3 MNVs) and 10 samples with wild-type loci were confirmed by secondary methods (Supplemental Table S5). TST NGS also detected 73 new variants (62 SNVs, 7 deletions, 3 insertions, and 1 MNV) in 35 samples, including the 3 KRAS SNVs that we determined to be SNaPshot false-negative results and the 1 KRAS SNV that could be either a SNaPshot false-negative or a TST false-positive result. Two GI samples were found to harbor no variants by TST alone (Table 5 and Supplemental Table S5). TP53 variants were the most frequently detected alteration in NSCLCs [13 variants, 13 (40.6%) of 32 samples], GI malignancies [15 variants, 13 (50.0%) of 26 samples], and overall [30 variants, 28 (35.0%) of 80 samples]. BRAF variants were detected most often in melanoma samples (9 variants, 9 (39.1%) of 23 samples). The extent of molecular testing and the TST variants detected in each sample are summarized in Figure 2 and Supplemental Table S5. The variant spectra of the most commonly mutated genes in our cohort, TP53, BRAF, KRAS, and EGFR, are shown in Figure 3.

Figure 3.

Landscape of somatic mutations in TP53, BRAF, KRAS, and EGFR. The figure shows variants in the four most commonly altered genes in the validation cohort. TruSight tumor next-generation sequencing targeted and sequenced all exons of TP53 and KRAS. The schematics are on the basis of the following National Center for Biotechnology Information protein reference sequences: TP53, NP_000537; BRAF, NP_004324; KRAS, NP_203524; and EGFR, NP_005219.

Validation of Bioinformatics Pipeline

The College of American Pathologist guidelines require that laboratories validate their bioinformatics pipelines for clinical NGS.²⁴ For this purpose, we used a parallel module that analyzes raw, untrimmed FASTQ files before alignment and variant calling (see Materials and Methods). FASTQ sequences for all samples with TST variants (n = 68) were aligned using BWA-mem, and variants were determined using two variant calling algorithms (GATK2.8 and VarScan2). Of the 112 variants called by the TST pipeline, 107 (95.5%) were confirmed by the alternative bioinformatics pipeline (Table 6 and Supplemental Table S6). Four SNVs with VAFs <10% (two KRAS, one PIK3CA, and one FBXW7) and one KIT 12-bp exon 11 insertion were not called initially (Table 6). On additional investigation, we discovered that initial analysis missed the SNVs because of mapping quality filters that caused the VAFs to decrease <5%. Modifying the QC parameters for less stringency rescued these misses, and the VAFs for these variants were similar to TST assessments (Supplemental Table S6). TST detected the 12-bp KIT insertion but with strand bias, and the variant was missed by GATK2.8, VarScan2, and the Agilent SureCall pipeline. We determined that during the initial analysis using Samtools version 0.19, the hard-coded coverage limit was insufficient to detect the variant. However, reanalysis with Samtools 1.2 did detect the variant because of a more lenient coverage limit.

Table 6.

Confirmation of Variants by Alternative Bioinformatics Pipeline

Variable	Detected by TST and GATK2.8 or VarScan2, n = 107	Detected by TST alone, n = 5	Not detected by TST, n = 2
SNVs, N = 91	85	4∗	2†
MNVs, N = 4	4	0	0
Deletions, N = 12	12	0	0
Insertions, N = 7	6	1∗	0

The alternative bioinformatics pipeline initially confirmed 95.5% (107/112) TST variants. Slight modifications to the analysis pipeline rescored variant calling for five variants. Therefore, we demonstrated 100.0% concordance between the TST MiSeq Amplicon-DS workflow and the alternative bioinformatics pipeline. Supplemental Table S6 provides a complete list of mutations.

MNV, multinucleotide variant; SNV, single-nucleotide variant; TST, TruSight tumor.

^∗All variants were detected by the alternative bioinformatics pipeline using less stringent quality control (see Results).

^†MAP2K1 p.D67N and KRAS p.V14I mutations were detected by the TST pipeline but excluded from the data set on the basis of interpretive discretion.

We also assessed whether the TST bioinformatics pipeline failed to call any variants in the 68 samples. In two samples, the alternative bioinformatics pipeline confirmed the routinely tested KRAS p.G12V (GI-11) and BRAF p.V600K (MEL-20) variants that were detected by TST. However, the alternate bioinformatics pipeline also identified two unique SNVs that were not in our final TST data set: a MAP2K1 p.D67N SNV (GI-11) and a KRAS p.V14I SNV (MEL-20) (Table 6 and Supplemental Table S6). On review of the raw TST data, both variants were called by VariantStudio 2.1 and seen in IGV2.3. However, the MAP2K1 p.D67N mutation exhibited low depth of coverage (low DP), and the KRAS p.V14I variant demonstrated both low DP and pool bias. Given these flags and the fact that routinely tested, commonly mutated variants were already present in these samples, pathologist discretion was used and the variants were not included in the final TST data set.

Overall, there was 95.5% (107 of 112) initial concordance and 100% (112 of 112) final concordance between the two bioinformatics pipelines. All TST-detected variants, including the one KIT 12-bp insertion, were confirmed by the alternative bioinformatics pipeline, and there was no evidence that TST failed to detect any variants in the tested samples.

Discussion

In this study, we detailed the clinical validation of a targeted NGS assay for the detection of clinically relevant somatic variants in FFPE tumor samples. We tested 100 FFPE patient resection, biopsy, and fine-needle aspiration samples, from which 80 libraries were prepared successfully using Illumina TST, a commercially available kit optimized for FFPE samples, which targets somatic variants in many solid tumors, including NSCLC, melanoma, and GI malignancies. We generated sufficient depths of coverage to confidently detect variants at approximately 5% VAF when 4, 8, or 12 samples were multiplexed on a single MiSeq flow cell. QC metrics revealed excellent coverage of almost all regions and routinely tested variants in each tumor type. Moreover, TST NGS demonstrated concordant variant detection when compared with reference methods, and reliable sensitivity and reproducibility using serial dilution studies and repeat library preparations, respectively. The TST variant-calling pipeline using commercial and laboratory-developed software showed 100% final concordance with the comparison alternative bioinformatics pipeline.

Pre-analytic Tissue QC Was Implemented for Optimal Results

Pre-analytic tissue QC (an H&E assessment of tumor nuclei and assessment of DNA amplifiability by comparative qPCR analysis) was performed on every sample before library preparation. Pathologist review of both preceding and succeeding H&E samples ensured that adequate neoplastic tissue (10% tumor nuclei) was submitted for testing,2, 25 and consensus review was performed for borderline cases because H&E estimations are often inaccurate (K.E.F. and L.Z.).²⁶ We selected a 10% tumor nuclei adequacy requirement, in part because this was standard practice in our laboratory for the pyrosequencing assay, but also to mitigate potential biochemical and analytic artifacts intrinsic to low-level variants.27, 28 Tumor fraction estimation values were also helpful to distinguish true variants from artifacts; expected VAFs for diploid heterozygous mutations could be inferred, whereas discrepant VAFs were scrutinized.

A large proportion of our diagnostic somatic mutation testing is performed on scant tissue specimens, such as biopsy specimens, fine-needle aspiration samples, or body fluid cytospin cell blocks. In our initial assessment using the stringent ΔqCT cutoff of <4.0, 22 samples would have been excluded from library preparation, 15 (68.1%) of which were scant tissue samples (Table 1). We reasoned that simply moving the ΔqCT cutoff from 4.0 to 6.0 would likely accommodate many of these initially excluded specimens. Indeed, for samples with ΔqCT values <6.0 prepared with the recommended undiluted input DNA volumes for samples with ΔqCT values ≥1.5, we successfully prepared libraries for 20.0% and 35.7% more total and scant specimens, respectively. In addition, only 4 samples (18.2%) with ΔqCT <6.0 failed library preparation and/or sequencing. Even though our data suggested that scant specimens contributed to preanalytic QC failure, we currently run QC analysis on all specimens with ≥10% tumor nuclei, regardless of the amount of tissue present, and prepare libraries for all samples with ΔqCT values ≤6.0 to maximize the number of testable clinical samples (Supplemental Figure S2). Collectively, the ability to ensure both adequate tumor content and nucleic acid integrity provided the framework for an optimized NGS workflow.

TST Coverage Depths Were Sufficient to Confidently Call Variants at Low VAF

A principal focus of our clinical NGS validation strategy was to demonstrate that NGS offered comparable variant detection when compared with the assays currently used for clinical testing (Supplemental Table S1). It was necessary that variants be sequenced with sufficient depths of coverage such that unambiguous variant detection could be achieved at comparable LODs. Thus, we established rigorous sequencing QC criteria: we required >500× or >1000× base coverage for variant calling at 10% or 5% VAF, respectively, and defined sequencing QC failure as any base sequenced <500× within a region.

In the initial pilot run using four samples, all regions passed sequencing QC after excluding 12 noncontributory bases, the depth of coverage at select routinely tested bases was 20 to 100 times greater than the proposed minimal requirement of 500× (Supplemental Figure S1), and sufficient coverage depths were obtained even when 12 samples were multiplexed on a single flow cell (Figure 1). However, we noticed a slight decrease in sequencing coverage of other regions because the number of multiplexed samples increased, and inadequate coverage of routinely tested variants was predominantly seen in samples from multiplexed runs of 12 samples (Figure 1, Supplemental Figure S1, and Table 2). Therefore, we decided to cap sequencing runs at a maximum of 10 samples per flow cell for our clinical standard operating procedure. The potential for inadequate coverage of routinely tested variants with a larger number of multiplexed specimens also prompted the development of a repeat testing algorithm. The algorithm ensures that all clinical specimens receive the same quality of testing that was available before NGS implementation (Supplemental Figure S2).

Previous laboratory-established LODs for detecting SNVs/MNVs were 5% using pyrosequencing and 10% using SNaPshot, whereas an LOD of 10% could be achieved using FLA for indel detection. More important, NGS yielded slightly improved analytic LODs for SNVs (3.2%) and indels (4.7%), as determined by dilution analyses (Table 3). We acknowledge that these calculated LODs are the result of data from only two samples containing three variants, and testing additional samples with more replicates likely would have yielded more substantive cutoff values. Given the limited LOD studies performed, we also acknowledge that low-level variants could decrease lower than our 3.2% and 4.7% analytic thresholds for SNVs and indels, respectively. This should be caveat for laboratories that want to apply this assay for detection of low-level sub-clonal populations or circulating tumor DNA, and we strongly advise laboratories to perform more extensive LOD analyses for reporting allele frequencies <5%. Our clinical validation standard operating procedure states that allele frequencies <5% should be interpreted with caution and in the context of the tumor content and other accompanying clinical information. This recommendation is made, in part, to account for VAF fluctuations; there was considerable variation in low-level VAFs in our limited reproducibility studies (Table 4), and additional studies to calculate robust SDs were not feasible given resource restraints. Still, our data suggested negligible deviations in relative VAFs for variants with >5% VAFs. By defining the reportable clinical sensitivity at >5% and ensuring that all samples contained at least 10% neoplastic nuclei, we are confident that true variants can be reliably detected within this defined clinical reportable range. However, repeat NGS or an orthogonal method can always be performed if warranted (Supplemental Figure S2).

Detection of Somatic Mutations in FFPE Tumor Samples

In addition to adequate depth of coverage, it was also imperative we generated sensitive and specific data; NGS can produce erroneous results secondary to chemistry sequencing errors,²⁹ formalin-fixation artifacts,³⁰ or suboptimal coverage, alignment, and/or variant calling.³¹ We collected and tested samples from multiple clinically validated assays to ensure that we generated comparable results with known variants from assays with defined LODs. TST NGS detected all known SNVs, MNVs, indels, and wild-type loci detected by orthogonal methods (n = 49) (Table 5), indicating 100% concordance for known variants.

The excellent concordance was attributable to several features of the TST assay. Sequencing was performed on the Illumina MiSeq, a benchtop sequencer that used high-fidelity terminal dideoxynucleotide sequencing chemistry. The accurate sequencing coupled with high depths of coverage of routinely tested regions allowed for unambiguous variant interpretation. More important, TST targets and amplifies both the forward and reverse strands of each region. Each forward and reverse pool was sequenced individually, and the variants were compared between them (Illumina technical note, sequencing software: Amplicon–DS Somatic Variant Caller. San Diego, CA). This feature assisted in detecting sequencing artifacts, such as hydrolytic deamination C>T/G>A FFPE artifacts, PCR errors that occur during library preparation, or homopolymer misalignments; true variants always manifested similarly in both pools, whereas artifacts were detected only in one pool. The pool bias feature was applied to exclude the KRAS p.V14I variant detected by the alternative bioinformatics pipeline (Table 6). The KRAS variant was likely a false-positive call because of its low VAF, and the fact that concomitant KRAS and BRAF mutations are rare, particularly in melanoma.32, 33

TST also detected 73 variants that were not confirmed by secondary methods. We elected not to Sanger confirm every additional variant despite this approach being the purported gold standard. More than half of our detected variants harbored VAFs <20%, and with published sensitivities of 15% to 20% for variant detection,³⁴ we reasoned Sanger confirmation was an insensitive method for confirmation. Unfortunately, comprehensive confirmatory sequencing was not practical because of cost and paucity of remaining sample material, highlighted by a discrepant KRAS p.G13C mutation that was not detected by SNaPshot analysis (NSCLC-4). Therefore, we acknowledge that some nonconfirmed detected variants may represent false-positive results. However, TST false positives are likely the exception in our cohort given the stringent requirements for variant detection (depth of coverage, absence of strand, or pool bias) and the high concordance of variant calls with an alternative bioinformatics pipeline (Table 6 and Supplemental Table S6). Other NGS validation studies have shown high concordance between novel variants and other confirmatory methods.35, 36

Conversely, the true sensitivity of the TST NGS assay is difficult to define because one or more samples may have harbored masked variants. False-negative results can occur secondary to the capture methods or, more often, the alignment and variant calling algorithms.³⁷ There was no definitive evidence that the TST library preparations failed to target or sequence any particular variant or class of variant, albeit the types of variants tested were clearly skewed toward our predetermined routinely tested variants. However, we encountered an 18-bp EGFR deletion (VAF, approximately 15.0%) in an NSCLC sample that was detected by both FLA and reference laboratory NGS methods during clinical testing. Further investigation revealed a complex mutation involving co-occurring SNVs and indels in EGFR that were not aligned properly by the TST Amplicon–DS workflow.

The TST Amplicon–DS bioinformatics pipeline was robust, and VAFs were highly concordant, although not identical, between the two pipelines. The different alignment methods, thresholds, parameters, and filtering criteria used by the two pipelines account for some of these observed variations. Although the alternative bioinformatics pipeline did not detect additional variants in 68 samples tested, we encountered clear examples of discrepant variant calling during the clinical validation. Five samples harbored variants that were not detected by the alternative analysis pipeline (four SNVs and one insertion). However, all of these discrepant calls were detected in the alternative bioinformatics pipeline when less stringent QC parameters were used (Supplemental Table S6).

Interestingly, the KIT insertion was also missed by the Agilent SureCall 2.0 analysis pipeline. We detected the insertion in the middle of multiple reads in the raw unfiltered SureCall BAM, suggesting a true insertion. However, we also identified multiple false-positive calls at the ends of the indel reads and in other genes throughout the BAM file, suggesting that HaloPlex SureCall performs aggressive filtering to avoid false-positive calls. Given our limited experience with this analysis workflow, we acknowledge that these observations are speculative. Additional studies are required to understand the nuances of the SureCall analysis pipeline and all bioinformatics pipelines in general.

Collectively, these results give credence to the robust nature of the TST bioinformatics pipeline for detecting clinically relevant SNVs, MNVs, and small indels. However, we urge caution in exclusively using the vendor-supplied TST analysis workflow and recommend that all aligned BAM sequences undergo manual inspection using IGV2.3 for indels in EGFR exons 19 and 20 for NSCLC and KIT exons 9, 11, 13, and 17 for melanoma regardless of variant calls. At the pathologist's discretion, repeat testing or retesting with alternative specimens is advised if data inconsistent with the specimen type or clinical history are encountered. Complementary open-source bioinformatics pipelines and multicenter shared clinical databases are also under investigation to unmask false-negative (and false-positive) results. Periodic prospective sample testing by orthogonal NGS and non-NGS methods is included as part of ongoing quality assurance.

Mutation Spectra and Clinical Utility

TST NGS generated somatic mutation spectra comparable to published reports (Figure 2 and Supplemental Table S5).38, 39, 40, 41, 42 TP53 mutations were the most often detected mutations in our cohort, and the mutations mirrored the expected broad tumor suppressor gene distribution. Also, TST NGS robustly detected the most common hotspot mutations in NSCLC, melanoma, and GI malignancies (eg, BRAF, KRAS, and EGFR) (Figure 3). In several cases, TST offered additional clinical mutation data compared with reference methods. For example, KRAS mutations were detected in at least three previously tested KRAS wild-type NSCLCs (Table 5 and Supplemental Table S5). Patients with activating mutations in KRAS do not benefit from targeted therapies and are typically treated with standard chemotherapeutic regimens or enrolled in clinical trials.⁴³ One melanoma sample that was tested for codon V600 mutations harbored a BRAF p.S616F mutation (MEL-11), which is described in melanoma.⁴⁴ Also, all four NSCLC patients with EGFR exon 19 deletions harbored concomitant TP53 mutations, and there are data to suggest that TP53 is involved in resistance mechanisms leading to EGFR tyrosine kinase inhibitor treatment failure.45, 46 Additional studies are needed to investigate the prognostic and predictive effects of the mutations in our validation cohort.

Perhaps the most practical clinical benefit of TST NGS testing was the ability to assess multiple relevant somatic loci simultaneously. We streamlined operations by consolidating multiple assays (SNaPshot, FLA, pyrosequencing, and reference testing), and this generated technologist flexibility, allowed pathologists to interpret one consolidated data set, and incurred significant cost savings. TST NGS also significantly improved clinical mutation testing and reporting. Previously, testing for NRAS and/or KIT mutations in melanoma was only performed in select cases, and usually only in BRAF wild-type melanomas. However, ordering errors, turn-around time delays, and/or insufficient samples often precluded optimal reflex testing. These factors may have contributed to the observation that NRAS or KIT testing was performed on only four of nine BRAF wild-type melanoma validation samples, and only one sample was tested for all three genes (Figure 2).

In two BRAF wild-type melanomas, TST NGS detected one KIT and one NRAS mutation, respectively, suggesting that clinically actionable results were missed because of inefficient testing algorithms. Fortunately, TST NGS consolidates the necessary molecular testing in one assay using one sample, such that patients and providers no longer rely on cumbersome, poorly executed reflex testing algorithms for results, and the laboratory is buffered from inaccurate ordering. Also, providers receive one integrated report that includes all relevant mutation data within 7 to 10 business days in almost all cases that did not require additional testing (eg, repeats or failed runs). As targeted therapeutics evolve and we better understand the impact of genetic mutation testing for prognostic and predictive applications, the routine applicability of NGS will become much more significant. Notably, even in a relatively short time period after implementation, the utility and importance of mutation testing for patients with melanoma was clearly evident.

However, TST was designed to detect the most common mutations in these select cancers, and the TST Amplicon–DS workflow is optimized for detecting common mutations in these tumor types. For example, the used banded Smith-Waterman algorithm performs local sequence alignments using a maximum indel size of 25 bp (Illumina reference guide: MiSeq Reporter Amplicon–DS Workflow. San Diego, CA); the most common clinically relevant indels in EGFR and KIT, for example, are <25 bp. Also, important genes and/or mutations in other malignancies (eg, central nervous system and hematolymphoid) and some relevant regions amenable to NGS testing were not represented in the 26-gene panel (Supplemental Table S1).47, 48 One sample (NSCLC-8) harbored no mutations by SNaPshot, FLA, and TST, but contained an STK11 nonsense mutation in exon 5 by FLDM NGS. STK11 exon 5 was not covered by the TST assay (Supplemental Tables S1 and S5). At our institution, STK11 mutation testing is occasionally requested because NSCLC patients with inactivating mutations in STK11 may benefit from alternative therapies.⁴⁹ Also, testing for gene rearrangements or copy number changes was not considered a priority for our initial NGS workflows, particularly for NSCLC specimens that are currently tested for ALK, ROS1, and RET rearrangements and MET amplification using FISH (data not shown) (Figure 2). We anticipate that this will change in the near future as the availability of higher-throughput and larger-capacity bench sequencing instruments will allow most clinical laboratories to perform more comprehensive DNA- and RNA-based sequencing assays that are currently only available to laboratories with Illumina HiSeq capabilities.

Collectively, TST NGS testing provides valuable clinical information and has replaced inefficient testing algorithms for somatic mutation testing. These data are discussed regularly in molecular tumor board conferences to deliver informed clinical care based for oncologic patients. TST has been well received by our laboratory and clinical colleagues; however, one static-targeted NGS panel does not fit all applications. Thus, every laboratory should ensure adequate coverage of regions of interest before selecting a targeted molecular assay, and assess the ability or lack thereof to detect the intended alterations these regions harbor. Furthermore, strategies to update solid tumor molecular testing on the basis of novel genomic discoveries, clinical trial enrollment criteria, and potential therapeutic targets should also be considered.

Summary

Overall, we achieved excellent results using TST NGS. TST NGS generated reliable, clinically relevant NGS data from FFPE solid tumor samples and was sufficiently modular to meet the testing and workflow needs of our laboratory because of its design, seamless integration into our workflows, and vendor-supplied bioinformatics pipelines. The greatest strengths of the assay were the sequencing fidelity, high depths of coverage, and targeted amplicon design that allowed for high sensitivity and specificity of variant calls at meaningful LODs. Implementation of rigorous preanalytic QC, defining LODs and coverage requirements for variant detection, and recognition that ongoing quality assurance was necessary to address potential erroneous results were essential components of our clinical validation strategy.

Supplemental Data

Supplemental Figure S1

Breakdown of TruSight tumor (TST) depth of coverage for multiplexed samples. Histograms of sequencing coverage of TST samples by the 85 targeted regions for one run of four multiplexed individual samples with no controls, four runs of eight multiplexed individual samples and controls (excluding controls), and five runs of 12 multiplexed individual samples and controls (excluding controls). A region failed quality control (QC) if a single base within the region was sequenced <500×. A box-and-whiskers plot showing depth of coverage of select routinely tested bases for samples multiplexed in a run of 8 or 12 samples. Asterisk indicates the 12 bases that were not sequenced in any sample were excluded from analysis (9 bp of STK11 exon 6 and 3 bp of TP53 exon 4); dagger, Supplemental Table S2 provides a full list of regions. n = 4 (A); n = 26 (B and C); n = 50 (D and E). Max, maximum; Min, minimum; MPx, multiplexed.

Supplemental Figure S2

Clinical specimen workflow for TruSight tumor (TST) next-generation sequencing (NGS). The schematic details a 5 business day workflow for TST NGS testing. Hybridization and sequencing runs on the MiSeq were performed overnight. Repeat NGS testing was only performed if sufficient TST libraries were generated. Routine molecular testing was directed at routinely tested regions for the tested tumor type. The asterisk indicates routinely tested regions/bases were defined as bases or exons specific to the tumor type that were currently sequenced by non-NGS methods in routine clinical practice. Supplemental Table S1 provides a full list of regions by tumor type. ΔqCT, mean sample qCT – mean control qCT in real-time PCR assay.