Abstract
Purpose
DNA analysis by NGS has become important to direct the clinical care of cancer patients. However, NGS is not successful in all cases, and the factors responsible for test failures have not been systematically evaluated.
Materials and methods
A series of 1528 solid and hematolymphoid tumor specimens was tested by an NGS comprehensive cancer panel during 2012–2014. DNA was extracted and 2×101 bp paired‐end sequence reads were generated on cancer‐related genes utilizing Illumina HiSeq and MiSeq platforms.
Results
Testing was unsuccessful in 343 (22.5%) specimens. The failure was due to insufficient tissue (INST) in 223/343 (65%) cases, insufficient DNA (INS‐DNA) in 99/343 (28.9%) cases, and failed library (FL) in 21/343 (6.1%) cases. 87/99 (88%) of the INS‐DNA cases had below 10 ng DNA available for testing. Factors associated with INST and INS‐DNA failures were site of biopsy (SOB) and type of biopsy (TOB) (both p < 0.0001), and clinical setting of biopsy (CSB, initial diagnosis or recurrence) (p < 0.0001). Factors common to INST and FL were age of specimen (p ≤ 0.006) and tumor viability (p ≤ 0.05). Factors common to INS‐DNA and FL were DNA purity and DNA degradation (all p ≤ 0.005). In multivariate analysis, common predictors for INST and INS‐DNA included CSB (p = 0.048 and p < 0.0001) and TOB (both p ≤ 0.003), respectively. SOB (p = 0.004) and number of cores (p = 0.001) were specific for INS‐DNA, whereas TOB and DNA degradation were associated with FL (p = 0.04 and 0.02, respectively).
Conclusions
Pre‐analytical causes (INST and INS‐DNA) accounted for about 90% of all failed cases; independent of test design. Clinical setting; site and type of biopsy; and number of cores used for testing all correlated with failure. Accounting for these factors at the time of tissue biopsy acquisition could improve the analytic success rate.
Keywords: Failed analysis, Molecular oncology testing by NGS, Diagnostic biopsy, Site and type of biopsy, Number of cores, DNA degradation and purity
1. Introduction
Molecular testing by NGS approaches in clinical oncology has become a cornerstone in clinical care of cancer patients with recurrent or metastatic disease. Areas of NGS applications in clinical patient care include disease diagnosis (Doyle et al., 2014; Frederick et al., 2014; Sehn et al., 2014), identification of therapeutic targets in different types of cancer (Cottrell et al., 2014; Frampton et al., 2013; Hagemann et al., 2015; Kris et al., 2014; Pritchard et al., 2014; Tsimberidou et al., 2012), improvement of risk‐stratification which can guide the selection of treatment (Marcucci et al., 2014), and detection of minimal residual disease (Martinez‐Lopez et al., 2014). Achieving such clinical goals requires successful testing, but as with any testing method, the analysis is unsuccessful in a subset cases. However, the factors responsible for NGS test failures in the setting of oncology cases have not been systematically evaluated. Thus, in the current study we evaluated the characteristics of specimens submitted for molecular oncology testing by NGS, and identified major factors associated with failed NGS testing for pre‐analytic and analytic reasons.
2. Materials and methods
2.1. Regulatory approval
This study was approved by the Washington University Human Studies Committee.
2.2. Clinical samples and testing methodology
The cohort of cases in this analysis consisted of consecutive clinical tumor samples submitted to Genomics and Pathology Services at Washington University School of Medicine (GPS@WUSTL) between March 1, 2012 and May 13, 2014 for the Comprehensive Cancer Gene Set testing by NGS. NGS testing was ordered by oncologists to direct clinical patient care (i.e., not on an experimental or investigational basis, or as part of a research protocol). The test uses targeted hybrid‐capture sequencing on an Illumina platform of formalin‐fixed paraffin‐embedded (FFPE) tissue specimens, bone marrow biopsies, or peripheral blood (Cottrell et al., 2014). All samples were reviewed by a board‐certified surgical pathologist to mark tumor areas to be used for testing; pathologist review included estimation of tumor cellularity, heterogeneity, and tumor cell viability within the selected areas. DNA was extracted from FFPE and/or fresh‐frozen tissues using a QIAamp DNeasy blood and tissue kit (Qiagen, Valencia, CA). DNA purity was assessed by measuring DNA absorbance with NanoDrop at both 260/280 nm and 230/260 nm. DNA concentration was measured by Qubit fluorometer (Life Technologies, Carlsbad, CA) and a total of 2 μl of DNA, regardless of concentration, was then electrophoresed on an agarose gel to determine the level of DNA degradation. Library preparation and DNA sequence analysis using Illumina platforms then proceeded as described (Cottrell et al., 2014).
NGS testing was considered successful when high quality DNA sequence results were obtained that met laboratory quality metrics (Cottrell et al., 2014), regardless of findings in the reports issued for the cases (i.e., regardless of whether or not the sequence variants that were detected indicated a specific targeted therapy). Test failure was classified into three categories: insufficient tissue available for testing (INST) when insufficient tumor tissue was available for DNA extraction (generally, tumor sample less than 2 mm in greatest dimension; tissue less than 10% tumor, or tumor less than 10% viability); failure due to insufficient DNA (INS‐DNA) when DNA extraction yielded less than 100 ng DNA; and failure during library preparation (FL) when the libraries generated did not meet specified quality metrics (i.e., pre‐hybridization product size was between 230 and 300 bp by analysis on Agilent Bioanalyzer HS chip and at least 500 ng of the ligated product was available for hybridization). When the pre‐hybridization product was less than 500 ng but there was additional DNA available from the original tissue nuclei acid extraction, library generation was attempted again; if no more DNA was available but there was still tumor tissue available, DNA extraction and library generation were attempted again. Library generation was considered a failure if initial and repeat attempts failed to produce 500 ng of the pre‐hybridization product.
Data were collected on pre‐analytical variables during the pathologist review step and included clinical setting of biopsy (CSB, specifically at the time of primary diagnosis versus at the time of relapse/progression); type of biopsy (TOB); age of biopsy material; site of biopsy (SOB); the lesion type of biopsy (LTOB, specifically primary tumor or metastatic lesion); the tissue processing procedure; the tumor cellularity, heterogeneity and cell viability within the selected tumor area; and the number of cores in the biopsy (where relevant). Biopsies were of six types, excisional biopsy (EX), fine needle aspirate (FNA), core biopsy (CB), laproscopic biopsy (LB), endoscopic biopsy (EB), and cytology specimen. Age of biopsy material was defined as the difference in years between the date the biopsy was obtained and the date the NGS test was performed on the specimen. For SOB, nine sites were considered (gastrointestinal, bone marrow, bone, lung, liver, lymph node, pancreas, kidney, and “other” where all other types of tumors were pooled under this category due to low number of cases in each of these types). Tumor cellularity and tumor cell viability were each defined in 10‐percentile intervals. Tumor heterogeneity (i.e., intra‐tumoral differences in cellularity, grade, architectural pattern, and cytologic atypia) was defined as low or high. Data were also collected on the pre‐analytical variables of DNA purity and DNA degradation as described (9).
3. Statistical analysis
Statistical analyses were performed using SPSS Statistics 22 package. Univariate association analysis of the evaluated variables and failed analysis due to INST, INS‐DNA or FL was performed using the Chi‐square or Fisher's Exact Test for categorical variables and the Mann–Whitney U test for continuous variables, with a two‐tailed significance level of 0.05.
Multivariate logistic regression analyses were performed and included successful cases, and failed cases, due to either INST, INS‐DNA, or FL. Since not all variables were captured on all cases (e.g., number of cores used for testing, DNA purity, and degradation were captured on cases that failed due to INS‐DNA or FL but not due to INST), and since there were some variables that specifically contribute to failure at one step but not the other (e.g., failure due INS‐DNA vs FL), a separate multivariate analysis was run for each failure step. The dependent variable was the analysis outcome (successful or failure due to either INST, INS‐DNA, or FL) and the independent variables were all variables that were significant in the univariate analysis for each step where failure occurred. For the multivariate models, low collinearity of the independent variables was confirmed with all bivariate correlations less than 0.70. Assessment of model fit indicated that the estimates for SOB may be unreliable, and thus we assessed the stability of our conclusions by removing SOB from each multivariate model; the conclusions were similar, thus increasing confidence that the reported model that includes SOB accurately represents the relationship between the independent variables and the likelihood of failure.
4. Results
A total of 1528 cases were submitted for testing during the study period. Of those, 343 (22.45%) cases failed to produce DNA sequence that met laboratory quality metrics. Cases failed due either to INST 223/343 (65%), INS‐DNA 99/343 (28.9%), or FL 21/343 (6.1%). The failure rate due to INST or FL increased in 2014 compared with 2012 (13.0%–20.9% for INST, and 1%–2.3% for FL), while the failure rate due to INS‐DNA remained constant (7.2% and 7.3%). The total failure rate was similar for the years 2012 and 2013 (about 21%) but increased in 2014 (30.5%) as shown in Table 1.
Table 1.
Failure rate by year of testing and by step of failure.
| Year | Insufficient tissues number (%) | Insufficient‐DNA number (%) | Failed library number (%) | Total failure number (%) | Successful cases |
|---|---|---|---|---|---|
| 2012 | 65 (13%) | 36 (7.2%) | 5 (1%) | 106 (21.2%) | 395 (78.8%) |
| 2013 | 112 (13.9%) | 47 (5.8%) | 11 (1.4%) | 170 (21.1%) | 637 (78.9%) |
| 2014 | 46 (20.9%) | 16 (7.3%) | 5 (2.3%) | 67 (30.5%) | 153 (69.5%) |
| Totala | 223 (65%) | 99 (28.9%) | 21 (6.1%) | 343 (100%) | 1185 (100%) |
Total number of cases failed at the specific step with percentage calculated as the failure at the specified step to failure for all reasons.
4.1. Investigation of variables associated with failed analysis
A total of ten variables were investigated for association with failed analysis for NGS molecular oncology testing. These variables are systematically measured as part of the Quality Control/Quality Assurance (QC/QA) program for our clinical molecular oncology testing by NGS. Table 2 lists the results of the univariate analysis comparing these variables between the successful cases and cases that failed due to INST, INS‐DNA or FL. TOB (p < 0.0001), SOB (p < 0.0001), and CSB (p < 0.0001) were highly associated with failures for both INST or INS‐DNA where certain types of specimens and certain anatomic sites were associated with higher failure rates (Table 2), and CSB where higher failure rate was observed in specimens obtained at the time of initial diagnosis (p < 0.0001). Higher tumor heterogeneity, lower tumor viability, and younger specimen age were associated with INST failures (p = 0.04, p = 0.033, and p < 0.0001, respectively). Lower DNA purity (p ≤ 0.005) and higher DNA degradation (p < 0.0001) were associated with failures for both INS‐DNA and FL. The number of tissue cores used for testing was specifically associated with INS‐DNA failures (p < 0.0001). Variables associated with FL included SOB (p = 0.001), lesional type of biopsy (p = 0.022), and lower tumor viability (p = 0.05). Type of biopsy showed a trend towards association (p = 0.08) (Table 2).
Table 2.
Univariate association analysis of NGS molecular oncology analysis outcome and examined variables.
| Variable | Successful cases | INST failures (%) | *P value | Successful cases (%) | INS‐DNA failures (%) | **P value | Successful cases (%) | Failed library (%) | ***P value |
|---|---|---|---|---|---|---|---|---|---|
| a Type of biopsy | <0.0001 | <0.0001 | 0.08 | ||||||
| Core biopsy | 422 (96.6%) | 15 (3.43%) | 422 (88.1%) | 57 (11.9%) | 422 (99.1%) | 4 (0.9%) | |||
| Cytology | 3 (75%) | 1 (25%) | 3 (100%) | 0 (0%) | 3 (100%) | 0 (0%) | |||
| Endoscopic biopsy | 68 (71.6%) | 27 (28.42%) | 68 (77%) | 9 (23%) | 68 (95.8%) | 3 (4.2%) | |||
| Excisional biopsy | 641 (98.3%) | 11 (1.7%) | 641 (99.23%) | 5 (0.77%) | 641 (98.5%) | 10 (1.5%) | |||
| Fine needle aspirate | 34 (48.6%) | 36 (51.4%) | 34 (70.8%) | 14 (29.2%) | 34 (94.4%) | 2 (5.6%) | |||
| Laproscopic biopsy | 6 (75%) | 2 (25%) | 6 (85.7%) | 1 (14.3%) | 6 (100%) | 0 (0%) | |||
| a Site of biopsy | <0.0001 | <0.0001 | 0.001 | ||||||
| Gastrointestinal | 939 (91.2%) | 9 (8.8%) | 93 (97.8%) | 2 (2.2%) | 93 (97.9%) | 2 (2.1%) | |||
| Bone marrow | 38 (92.7%) | 3 (7.3%) | 38 (86.8%) | 5 (13.2%) | 38 (97.4%) | 1 (2.6%) | |||
| Bone | 44 (74.6%) | 15 (25.4%) | 44 (84.1%) | 7 (15.9%) | 44 (89.8%) | 5 (11.2%) | |||
| Lung | 228 (73.8%) | 81 (26.2%) | 228 (84.2%) | 36 (15.8%) | 228 (98.3%) | 4 (1.7%) | |||
| Liver | 140 (81.4%) | 32 (18.6%) | 140 (87.9%) | 17 (12.1%) | 140 (98.6%) | 2 (1.4%) | |||
| Lymph nodes | 154 (87%) | 23 (13%) | 154 (96.1%) | 6 (3.9%) | 154 (100%) | 0 (0%) | |||
| Pancreas | 56 (88.9%) | 7 (11.1%) | 56 (98.2%) | 1 (1.8%) | 56 (98.2%) | 1 (1.8%) | |||
| Kidney | 68 (91.9%) | 6 (8.1%) | 68 (94.4) | 4 (5.6%) | 68 (100%) | 0 (0%) | |||
| Other | 361 (91.6%) | 33 (8.4%) | 361 (95.6%) | 16 (4.4%) | 361 (98.6%) | 5 (1.4%) | |||
| a Clinical setting of biopsy | <0.0001 | <0.0001 | 0.21 | ||||||
| Diagnosis | 504 (82.2%) | 109 (17.8%) | 504 (89.7%) | 58 (10.3%) | 504 (98.1%) | 10 (1.9%) | |||
| Relapse | 664 (91.4%) | 62 (8.5%) | 664 (96.4%) | 25 (3.6%) | 664 (98.8%) | 8 (1.2%) | |||
| a Primary or metastatic lesion | 0.45 | 0.76 | 0.022 | ||||||
| Primary | 597 (86%) | 95 (14%) | 597 (93.4%) | 44 (6.9%) | 597 (97.5%) | 15 (2.5%) | |||
| Metastatic | 519 (85%) | 93 (15%) | 519 (92.2%) | 41 (7.3%) | 519 (99.2%) | 4 (0.8%) | |||
| a Tissue processing | 0.89 | 0.09 | 0.50 | ||||||
| Formalin Fixed | 1046 (85.5%) | 177 (14.5%) | 1046 (93.5%) | 73 (6.5%) | 1046 (98.8%) | 13 (1.2%) | |||
| Formalin fixed ‐ decalcified | 71 (84.5%) | 13 (15.5%) | 71 (87.7%) | 10 (12.3%) | 71 (93.4%) | 5 (6.6%) | |||
| Fresh | 42 (87.5%) | 6 (12.5%) | 42 (89.4%) | 5 (10.6%) | 42 (97.7%) | 1 (2.3%) | |||
| a Heterogeneity | 0.04 | 0.33 | 0.52 | ||||||
| High | 878 (95.7%) | 39 (4.3%) | 878 (93.7%) | 59 (6.3%) | 878 (98.7%) | 12 (1.3%) | |||
| Low | 267 (98.9%) | 4 (1.1) | 267 (92.1%) | 23 (7.9%) | 267 (98.9%) | 3 (1.1%) | |||
| b Age of specimen | |||||||||
| (n = 1183) | (n = 221) | <0.0001 | (n = 1183) | (n = 97) | 0.99 | (n = 1183) | (n = 20) | 0.006 | |
| Mean Rank | 717 | 624 | 640 | 640 | 599 | 787 | |||
| b Percent tumor | 0.38 | 0.26 | 0.28 | ||||||
| (n = 1148) | (n = 43) | (n = 1148) | (n = 85) | (n = 1148) | (n = 15) | ||||
| Mean Rank | 598 | 553 | 620 | 576 | 583 | 491 | |||
| b Tumor viability | 0.033 | 0.39 | 0.05 | ||||||
| (n = 1147) | (N = 43) | (n = 1147) | (n = 86) | (n = 1147) | (n = 15) | ||||
| Mean Rank | 598 | 542 | 615 | 646 | 583 | 428 | |||
| b Number of cores all cases | <0.0001 | 0.17 | |||||||
| N/A | N/A | (n = 969) | (n = 56) | (n = 969) | (n = 14) | ||||
| Mean Rank | 525 | 307 | 493 | 201 | |||||
| b DNA purity | 0<.0001 | 0.005 | |||||||
| N/A | N/A | (n = 1156) | (n = 86) | (n = 1156) | (n = 19) | ||||
| Mean Rank | 632 | 476 | 591 | 378 | |||||
| a DNA degradation | <0.0001 | <0.0001 | |||||||
| Low | N/A | N/A | 1096 (99.4%) | 7 (0.6%) | 1096 (98.9%) | 12 (1.1%) | |||
| High | N/A | N/A | 60 (43.8%) | 77 (56.2%) | 60 (89.6%) | 7 (10.4%) |
*p values assess the differences between successful cases and INST failures.
**p values assess the differences between successful cases and INS‐DNA failures, and.
***p values assess the differences between successful cases and failures due to FL.
N = number of cases, in brackets represents the frequency.
N/A; not applicable (variable was not collected).
P values <0.05 are bolded.
Chi‐square and Fisher's Exact Tests were used to assess differences in the frequency between failed and successful cases for the variables studied.
Mann–Whitney test was used to assess differences in the ranks for the variables studied between failed and successful cases.
Multivariate regression analysis was performed on INST failures, on INS‐DNA failures, and on FL failures, and included significant variables from the univariate analysis (Table 3). Predictors for INST failures included CSB (p = 0.048) and TOB (p = 0.003), whereas SOB showed a trend in association (p = 0.09). Predictors for INS‐DNA failures were CSB (p < 0.0001), TOB (p < 0.0001), SOB (p = 0.004), and number of cores used for testing (p = 0.001). The predictors for FL failures were age of specimen (p = 0.01), TOB (0.04), and DNA degradation (p = 0.02). The predictors for INST failures explained 20.4% of variance, the predictors of INS‐DNA failures explained 27.8% of variance, and the predictors for FL failures explained 33.6% of variance (Nagelkerke R Square) (Table 3).
Table 3.
Multivariate association analysis of NGS molecular oncology analysis outcome and examined variables*.
| Failure due to INST | Failure due to INS‐DNA | Failure due to FL | |
|---|---|---|---|
| Number of samples evaluated (passed vs failed) | 1120 vs 38 | 958 vs 62 | 1057 vs 12 |
| Predictor | P value | P value | P value |
| Age of specimen | 0.96 | N/A | 0.01 |
| Heterogeneity | 0.11 | N/A | N/A |
| Clinical setting of biopsy (biopsy at diagnosis or relapse) | 0.048 | <0.0001 | N/A |
| Site of biopsy | 0.099 | 0.004 | 0.21 |
| Type of biopsy | 0.003 | <0.0001 | 0.04 |
| Tissue processing | N/A | N/A | N/A |
| Primary or metastatic | N/A | N/A | 0.24 |
| Estimated viability | N/A | N/A | 0.21 |
| DNA degradation | N/A | N/A | 0.02 |
| DNA purity | N/A | N/A | 0.76 |
| Number of cores used for testing | N/A | 0.001 | N/A |
| Nagelkerke R Square percent of variance | 0.203 | 0.278 | 0.336 |
*Only significant variables from the univariate analysis were included in the multivariate analysis. N/A, variable was not included because it was not significant in the univariate analysis. P values <0.05 are bolded.
5. Discussion
Our study systematically addresses the factors that impact the analytic success of NGS testing of tissue specimens to direct the clinical care of cancer patients. We evaluated several factors that we systematically measure as part of the QC/QA program for our clinical molecular oncology testing by NGS, including the characteristics of the biopsy (TOB, SOB, CSB, LTOB); the characteristics of the specimen (tumor content, heterogeneity, tumor cell viability in the selected areas used for testing, and age of specimen); number of cores used for testing; type of procedure for tumor processing; and DNA quality. The most important results of our study are that pre‐analytical causes (INST and INS‐DNA) accounted for about 94% of all failed cases, and that overall, 310/343 (90.4%) of failures were due to factors that were not dependent on test design.
In this study of 1528 samples submitted for testing, 343 (22.5%) failed to produce results for pre‐analytic or analytic reasons. Notably, failure in around 94% of cases was due to what are most precisely classified as pre‐analytical factors where the failure was due to INST in 65% of cases, and to INS‐DNA in 28.9% of cases. Only 6.1% of cases failed during the analytical phase itself, namely due to FL (Table 1). It is interesting to note that the failure due to INST was similar in 2012 and 2013 but it increased by 50% during 2014, while the failure rate due to INS‐DNA remained constant (Table 1); at the same time, the percentage of cases for the three sites of biopsy that were most associated with failure (bone, lung, and liver) remained constant from 2012 to 2014 while the contribution of EB and FNA biopsy types increased over the same time span (Table 2). These findings suggest that the increase in INST failure rate in 2014 was most likely due to an increase of some challenging biopsy types such as EB and FNA rather than a change in the contribution of biopsies from anatomic sites that are associated with higher failure rates. This result highlights the fact that while smaller and smaller tissue specimens may still provide sufficient material for definitive histopathologic diagnosis, the same tissue specimens may not be sufficient for NGS. Clinicians need to be aware of these results in order to plan tumor sampling procedures that obtain tissue samples that meet the test regimens for NGS as well as for histopathologic diagnosis.
Univariate analysis revealed three factors that were highly associated with failure whether due to INST or INS‐DNA. These were TOB, SOB, and CSB, all of which were highly statistically correlated (p < 0.0001). In addition, a shorter interval from tissue collection to NGS analysis (p < 0.0001), higher heterogeneity (p = 0.04), and lower tumor viability (p = 0.033) were all associated with INST failures but not INS‐DNA failures. As far as TOB is concerned, when failure due to all reasons was considered, core biopsies and excisional biopsies almost always had a successful outcomes (99.3% and 96.1%, respectively), whereas only 63.6%% and 39.4% of EB and FNA biopsies had successful outcomes, respectively (Table 2). Similar success rates were observed for cytology biopsies and LB, although the number of biopsies in these categories was limited (Table 2). The association between failure due to INST and age of specimen (interval from specimen collection to NGS analysis) was highly significant in the univariate analysis, however, age is no longer predictive of failure in the multivariate after accounting for all other variables in the model indicating that age contributes no unique information about failure above and beyond information that the significant variables in the multivariate model provide, e.g., TOB. This was confirmed by the highly significant association found between TOB and ranks of age (P < 0.000). For example, of the successful cases, only 5.8% and 3.5% of FNA and EB biopsies, respectively were <1 year old compared to 11% and 19.3%, respectively of the failed cases. Taken together, these results clearly demonstrate the importance of biopsy size for a successful NGS test outcome in that the smaller the size of biopsy the higher the failure rate. Recent reports have demonstrated the feasibility of NGS approaches for the analysis of cytology specimens by either a hybrid capture or amplification‐based approach (Kanagal‐Shamanna et al., 2014; Karnes et al., 2014), results which emphasize that it is not the type of specimen itself but rather tumor abundance and nucleic acid quality that are important. The latter point was specifically addressed by one group in a study that assessed the number of passes needed to obtain enough tissue material for molecular genotyping of EGFR and KRAS mutations in non‐small cell lung cancer NSCLC patients using a 21‐gauge needle, which found that four needle passes were able to reliably obtain adequate material for molecular analysis in over 95% of patients (Yarmus et al., 2013).
Interestingly, biopsies obtained at diagnosis were twice as likely to fail NGS testing as those obtained at relapse (26.1% versus 12.5%, p < 0.0001) (Table 2). This association is likely at least partially explained by the fact that, in this case series, biopsies obtained at diagnosis were twice as likely to be EB or FNA versus those obtained at recurrence (17.5% versus 8.4%, respectively) (data not shown). However, it is also possible that tissue specimens obtained at diagnosis may have been depleted for tumor cells due to neoadjuvant therapy, which has recently been shown to confound molecular testing (Dudley et al., 2015). In any event, our result again emphasizes that surgical, endoscopic, cytology, or FNA procedures that provide sufficient tissue for histopathologic diagnosis are not optimized for obtaining samples that meet the specimen requirements of NGS. This explanation also likely accounts for the finding that INST failures had lower tumor cell viability (Mann–Whittney test, p = 0.033) and had higher heterogeneity (p = 0.04) than successful cases, associations that were not observed with INS‐DNA failures.
In multivariate analysis for INST failures that assessed the significant variables from the univariate analysis, TOB remained highly significant (p = 0.003), whereas CSB and SOB had borderline significance (p = 0.048 and p = 0.099, respectively); the same variables remained highly significant in the multivariate analysis for INS‐DNA failures (p < 0.0001, p < 0.0001, and p = 0.004, respectively). Not surprisingly, INS‐DNA failures had fewer number of cores used for testing, lower DNA purity, and higher DNA degradation than successful cases (all p < 0.0001) (Table 3). Since it is expected that using more cores will yield more DNA, the finding that 75.4% of cases with successful outcome had five or more cores used for testing (data not shown) is likewise not surprising. The predictors for FL failures included shorter age of specimen (p = 0.01), TOB (p = 0.04), and higher DNA degradation (p = 0.02) (Table 3). These cases failed due to inability to produce enough pre‐hybridization material by PCR, most likely due to the presence of PCR inhibitor. It is a well‐known phenomenon that the efficiency of PCR is decreased due to protein cross‐linking induced by formalin and the degradation of nucleic acids which increases during storage depending on the pH value of the fixative (Benavides et al., 2006; Greer et al., 1991; Jackson, 1978).
In practice, the results of our study provide some opportunities for improving the likelihood that NGS analysis of a tissue specimen will be successful. By identifying factors associated with test failure, and the rate at which failures occur, our analysis emphasizes the differences between tumor sampling for diagnosis and tumor sampling for NGS testing. In this regard it is interesting to note that a number of studies have demonstrated the rich clonal architecture of malignancies (Gerlinger et al., 2012; Renovanz and Kim, 2014; Yachida et al., 2010), an observation that suggests that larger tumor samples are advantageous not only to increase the probability that NGS analysis will be successful, but also to ensure that the complexity of the tumor is well represented in the test results. The recent demonstration that occult tissue contamination is more likely to impact test results in extremely small tissue samples (Sehn et al. in press) offers yet another reason to obtain larger tumor samples for NGS.
Our results are based on a series of cases in which the NGS testing was performed by a hybrid capture approach (Cottrell et al., 2014). While it is well established that amplification‐based NGS assays have a lower input DNA requirement (Lin et al., 2014; Singh et al., 2013; Simen et al. 2015), it is worth noting that the INST category alone accounted for over 65% of failures of NGS testing, and that these cases (since they arise in the pre‐analytic phase of testing) would be failures regardless of whether a hybrid capture or amplification‐based approach was utilized.
Furthermore, of the 99 INS‐DNA cases in our series, only 12/99 (12%) cases had at least 10 ng of DNA, which suggests that 88% of INS‐DNA cases would have failed NGS testing regardless of the NGS approach (assuming that amplification‐based approaches generally require at least 10 ng input DNA). Although NGS has been described for very low input DNA quantities (Heitzer et al., 2013; Macaulay and Voet, 2014; Nawy, 2014), more cycles of PCR amplification are necessary to generate enough template molecules for sequence production whether by hybrid capture or amplification‐based approaches, which negatively impacts both test sensitivity and specificity (Sims et al., 2014) due to the potential for amplification bias (also known as “jackpotting”) and polymerase sequencing errors during amplification of low abundance templates. In addition, increased cycles of PCR negatively impact the accuracy of test results due to their generation of low complexity DNA libraries despite sufficient DNA quantity (i.e., the information content in 1000 sequence reads derived from one genome is quite different than the information content present in 1000 sequence reads from 1000 different genomes). Thus, changes in NGS test methods to accommodate lower DNA inputs come at a price; the better approach to optimize NGS test utility would be to ensure that better quality specimens (i.e., larger tissue samples with higher tumor viability) are submitted for testing.
Identifying factors associated with successful NGS testing in routine clinical use will hopefully lead to changes in current practice to eliminate aspects of tissue collection that most often contribute to failures. Our results will also guide prioritization of testing by different ancillary techniques; for example, if the available tissue material is limited and there is a need to perform multiple tests (i.e., immunohistochemistry, interphase fluorescence in situ hybridization study, and so on), other tests may be prioritized if the odds for a successful NGS outcome are low.
Acknowledgment
Research reported in this publication was supported in part by the Washington University Institute of Clinical and Translational Sciences grant UL1TR000448 from the National Center for Advancing Translational Sciences (NCATS) of the National Institutes of Health (NIH). The content is solely the responsibility of the authors and does not necessarily represent the official view of the NIH.
Al-Kateb Hussam, Nguyen TuDung T., Steger-May Karen, Pfeifer John D., (2015), Identification of major factors associated with failed clinical molecular oncology testing performed by next generation sequencing (NGS), Molecular Oncology, 9, doi: 10.1016/j.molonc.2015.05.004.
References
- Benavides, J. , García-Pariente, C. , Gelmetti, D. , Fuertes, M. , Ferreras, M.C. , García-Marín, J.F. , Pérez, V. , 2006. Effects of fixative type and fixation time on the detection of Maedi Visna virus by PCR and immunohistochemistry in paraffin-embedded ovine lung samples. J. Virol. Methods. 137, (2) 317–324. [DOI] [PubMed] [Google Scholar]
- Cottrell, C.E. , Al-Kateb, H. , Bredemeyer, A.J. , Duncavage, E.J. , Spencer, D.H. , Abel, H.J. , Lockwood, C.M. , Hagemann, I.S. , O'Guin, S.M. , Burcea, L.C. , 2014. Validation of a next-generation sequencing assay for clinical molecular oncology. J. Mol. Diagn.: JMD. 16, 89–105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Doyle, L.A. , Wong, K.K. , Bueno, R. , Dal Cin, P. , Fletcher, J.A. , Sholl, L.M. , Kuo, F. , 2014. Ewing sarcoma mimicking atypical carcinoid tumor: detection of unexpected genomic alterations demonstrates the use of next generation sequencing as a diagnostic tool. Cancer Genet. 207, 335–339. [DOI] [PubMed] [Google Scholar]
- Dudley, J. , Tseng, L.H. , Rooper, L. , Harris, M. , Haley, L. , Chen, G. , Gocke, C.D. , Eshleman, J.R. , Lin, M.T. , 2015. Challenges posed to pathologists in the detection of KRAS mutations in colorectal cancers. Arch. Pathol. Lab. Med. 139, 211–218. [DOI] [PubMed] [Google Scholar]
- Frampton, G.M. , Fichtenholtz, A. , Otto, G.A. , Wang, K. , Downing, S.R. , He, J. , Schnall-Levin, M. , White, J. , Sanford, E.M. , An, P. , 2013. Development and validation of a clinical cancer genomic profiling test based on massively parallel DNA sequencing. Nat. Biotechnol. 31, 1023–1031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Frederick, L. , Beardell, F. , Viswanatha, D.S. , 2014. Novel BCR-ABL1 fusion identified by targeted next-generation sequencing in a patient with an atypical myeloproliferative neoplasm. Hum. Pathol. 45, 1784–1789. [DOI] [PubMed] [Google Scholar]
- Gerlinger, M. , Rowan, A.J. , Horswell, S. , Larkin, J. , Endesfelder, D. , Gronroos, E. , Martinez, P. , Matthews, N. , Stewart, A. , Tarpey, P. , 2012. Intratumor heterogeneity and branched evolution revealed by multiregion sequencing. New Engl. J. Med. 366, 883–892. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Greer, C.E. , Peterson, S.L. , Kiviat, N.B. , Manos, M.M. , 1991. PCR amplification from paraffin-embedded tissues. Effects of fixative and fixation time. Am. J. Clin. Pathol. 95, (2) 117–124. [DOI] [PubMed] [Google Scholar]
- Hagemann, I.S. , Devarakonda, S. , Lockwood, C.M. , Spencer, D.H. , Guebert, K. , Bredemeyer, A.J. , Al-Kateb, H. , Nguyen, T.T. , Duncavage, E.J. , Cottrell, C.E. , 2015. Clinical next-generation sequencing in patients with non-small cell lung cancer. Cancer. 121, 631–639. [DOI] [PubMed] [Google Scholar]
- Heitzer, E. , Auer, M. , Gasch, C. , Pichler, M. , Ulz, P. , Hoffmann, E.M. , Lax, S. , Waldispuehl-Geigl, J. , Mauermann, O. , Lackner, C. , 2013. Complex tumor genomes inferred from single circulating tumor cells by array-CGH and next-generation sequencing. Cancer Res. 73, 2965–2975. [DOI] [PubMed] [Google Scholar]
- Jackson, V. , 1978. Studies on histone organization in the nucleosome using formaldehyde as a reversible cross-linking agent. Cell. 15, (3) 945–954. [DOI] [PubMed] [Google Scholar]
- Kanagal-Shamanna, R. , Portier, B.P. , Singh, R.R. , Routbort, M.J. , Aldape, K.D. , Handal, B.A. , Rahimi, H. , Reddy, N.G. , Barkoh, B.A. , Mishra, B.M. , 2014. Next-generation sequencing-based multi-gene mutation profiling of solid tumors using fine needle aspiration samples: promises and challenges for routine clinical diagnostics. Modern Patholo.: Official J. United States Can. Acad. Pathol. Inc. 27, 314–327. [DOI] [PubMed] [Google Scholar]
- Karnes, H.E. , Duncavage, E.J. , Bernadt, C.T. , 2014. Targeted next-generation sequencing using fine-needle aspirates from adenocarcinomas of the lung. Cancer Cytopathol. 122, 104–113. [DOI] [PubMed] [Google Scholar]
- Kris, M.G. , Johnson, B.E. , Berry, L.D. , Kwiatkowski, D.J. , Iafrate, A.J. , Wistuba, I.I. , Varella-Garcia, M. , Franklin, W.A. , Aronson, S.L. , Su, P.F. , 2014. Using multiplexed assays of oncogenic drivers in lung cancers to select targeted drugs. JAMA. 311, 1998–2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lin, M.T. , Mosier, S.L. , Thiess, M. , Beierl, K.F. , Debeljak, M. , Tseng, L.H. , Chen, G. , Yegnasubramanian, S. , Ho, H. , Cope, L. , 2014. Clinical validation of KRAS, BRAF, and EGFR mutation detection using next-generation sequencing. Am. J. Clin. Pathol. 141, 856–866. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Macaulay, I.C. , Voet, T. , 2014. Single cell genomics: advances and future perspectives. PLoS Genet. 10, e1004126 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Marcucci, G. , Yan, P. , Maharry, K. , Frankhouser, D. , Nicolet, D. , Metzeler, K.H. , Kohlschmidt, J. , Mrozek, K. , Wu, Y.Z. , Bucci, D. , 2014. Epigenetics meets genetics in acute myeloid leukemia: clinical impact of a novel seven-gene score. J. Clin. Oncol. Official J. Am. Soc. Clin. Oncol. 32, 548–556. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Martinez-Lopez, J. , Lahuerta, J.J. , Pepin, F. , Gonzalez, M. , Barrio, S. , Ayala, R. , Puig, N. , Montalban, M.A. , Paiva, B. , Weng, L. , 2014. Prognostic value of deep sequencing method for minimal residual disease detection in multiple myeloma. Blood. 123, 3073–3079. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nawy, T. , 2014. Single-cell sequencing. Nat. Methods. 11, 18 [DOI] [PubMed] [Google Scholar]
- Pritchard, C.C. , Salipante, S.J. , Koehler, K. , Smith, C. , Scroggins, S. , Wood, B. , Wu, D. , Lee, M.K. , Dintzis, S. , Adey, A. , 2014. Validation and implementation of targeted capture and sequencing for the detection of actionable mutation, copy number variation, and gene rearrangement in clinical cancer specimens. J. Mol. Diagn.: JMD. 16, 56–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Renovanz, M. , Kim, E.L. , 2014. Intratumoral heterogeneity, its contribution to therapy resistance and methodological caveats to assessment. Front. Oncol. 4, 142 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sehn, J.K. , Hagemann, I.S. , Pfeifer, J.D. , Cottrell, C.E. , Lockwood, C.M. , 2014. Diagnostic utility of targeted next-generation sequencing in Problematic cases. Am. J. Surg. Pathol. 38, 534–541. [DOI] [PubMed] [Google Scholar]
- Sehn, J.K. , Spencer, D.H. , Pfeifer, J.D. , Bredemeyer, A.J. , Cottrell, C.E. , Abel, H.J. , Duncavage, E.J. , 2015. Occult specimen contamination in routine clinical NGS testing. Am. J. Clin. Pathol. (in press) [DOI] [PubMed] [Google Scholar]
- Simen, B.B. , Yin, L. , Goswami, C.P. , Davis, K.O. , Bajaj, R. , Gong, J.Z. , Peiper, S.C. , Johnson, E.S. , Wang, Z.X. , 2015. Validation of a next-generation–sequencing cancer panel for use in the clinical laboratory. Arch. Pathol. Lab Med. 4, 508–517. [DOI] [PubMed] [Google Scholar]
- Sims, D. , Sudbery, I. , Ilott, N.E. , Heger, A. , Ponting, C.P. , 2014. Sequencing depth and coverage: key considerations in genomic analyses. Nat. Rev. Genet. 15, 121–132. [DOI] [PubMed] [Google Scholar]
- Singh, R.R. , Patel, K.P. , Routbort, M.J. , Reddy, N.G. , Barkoh, B.A. , Handal, B. , Kanagal-Shamanna, R. , Greaves, W.O. , Medeiros, L.J. , Aldape, K.D. , 2013. Clinical validation of a next-generation sequencing screen for mutational hotspots in 46 cancer-related genes. J. Mol. Diagn.: JMD. 15, 607–622. [DOI] [PubMed] [Google Scholar]
- Tsimberidou, A.M. , Iskander, N.G. , Hong, D.S. , Wheler, J.J. , Falchook, G.S. , Fu, S. , Piha-Paul, S. , Naing, A. , Janku, F. , Luthra, R. , 2012. Personalized medicine in a phase I clinical trials program: the MD Anderson Cancer Center initiative. Clin. Cancer Res.: Official J. Am. Assoc. Cancer Res. 18, 6373–6383. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yachida, S. , Jones, S. , Bozic, I. , Antal, T. , Leary, R. , Fu, B. , Kamiyama, M. , Hruban, R.H. , Eshleman, J.R. , Nowak, M.A. , 2010. Distant metastasis occurs late during the genetic evolution of pancreatic cancer. Nature. 467, 1114–1117. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yarmus, L. , Akulian, J. , Gilbert, C. , Feller-Kopman, D. , Lee, H.J. , Zarogoulidis, P. , Lechtzin, N. , Ali, S.Z. , Sathiyamoorthy, V. , 2013. Optimizing endobronchial ultrasound for molecular analysis. How many passes are needed?. Ann. Am. Thorac. Soc. 10, 636–643. [DOI] [PubMed] [Google Scholar]