Comment on “DNA damage is a pervasive cause of sequencing errors, directly confounding variant identification”

Abstract

Chen et al. (Reports, 17 February 2017, p. 752) highlight an important problem of sequencing artifacts caused by DNA damage at the time of sample processing. However, their manuscript contains several errors that led the authors to incorrect conclusions. Moreover, the same sequencing artifacts were previously described and mitigated in The Cancer Genome Atlas and other published sequencing projects.

The sequencing artifacts discussed in Chen et al. (1) have been described in publications from The Cancer Genome Atlas (TCGA) and other cancer genome projects (2–4). Accordingly, effective mitigation strategies have long been implemented, including software pipelines and improved library preparation methods, as described in Costello et al. (2). Thus, findings described in TCGA publications (5–9) were not impacted by oxidative damage as suggested by Chen et al. While they do raise general awareness of sequencing artifacts, which include machine errors (2), DNA oxidation (2–4), DNA cross-linking (in clinically used formalin-fixed tissues) (10) as well as others, their paper contains several errors that led to incorrect conclusions. The errors affect the following:

(i) Estimation of oxidative damage levels.

While their reported oxidative damage (8-oxoG) metric, GIV_{G_T}, is essentially equivalent to an earlier reported metric, oxoQ (2–4, 11), their software implementation is limited by using reads aligned only to the forward strand and biased by filtering out low quality bases. OxoQ was designed to be interpreted as a (Phred-like) base quality score. As such, oxoQ can be compared to typical levels of sequencing errors, which for most Illumina sequencing protocols is roughly at the level of Q & 30 (i.e., an error rate of 1/1000 bases). We initially noticed a poor agreement between the oxoQ and GIV_{G_T} metrics for damaged samples (oxoQ < 30, Fig. 1a), while scores were consistent for samples with low levels of DNA damage. Examining the methods of Chen et al., we found that the authors’ code with default parameters applied a high base quality threshold (Q > 30) and removed sites where total coverage exceeds 100x, which discarded the bulk of the data for the most damaged samples (Fig. 1c-f). Adjusting the filtering criteria to include bases with Q > 20 and to allow sites with depth above 100x restored concordance between GIV_{G_T} and oxoQ scores (Fig. 1b).

Figure 1.

(A) Picard oxoQ score vs. GIV_{G_T} for &1,900 TCGA tumor exomes. Lack of agreement between oxoQ and GIV_{G_T} scores is apparent in samples with high DNA damage (oxoQ < 30) highlighted in the gray box. GIV_{G_T} is calculated with Chen et al.’s code (estimate_damage.pl) with default parameters; oxoQ was calculated with Picard CollectSequencingArtifactMetrics (11) output. (B) OxoQ vs. GIV_{G_T} corrected (using base quality Q > 20 and coverage depth ≥ 20, comparable to Picard defaults) showing excellent agreement. (C) Histogram of individual base qualities in a typical TCGA 8-oxoG damaged sample (oxoQ = 24.2) compared to a low-damage sample (oxoQ=48.9). Most of the bases in the damaged sample have Q < 30 and hence are ignored by default GIV_{G_T}. This explains both the lack of agreement between the scores seen in (A) and authors missing the sequence context associated with 8-oxoG damage (D vs E) –– the Q > 30 filtering removed bases with the type of damage that is being quantified. “Lego” plot showing the distribution of errors in different 5′ and 3′ sequence contexts (plotted in reverse complement by convention), with Chen et al.’s (D) and our corrected (E) filtering criteria, showing distortion from the characteristic 8-oxoG context pattern arising from data selection criteria. (F) The fraction of bases that remain after applying Chen et al.’s filters show that most bases are removed in highly damaged samples.

(ii) Claim of no context specificity of 8-oxoG damage

The authors’ conclusion that they “did not observe nucleotide context specificity” of 8-oxoG damage (1) is a consequence of their filtering out most of the supporting data, as described above. Sequence context specificity is a key feature of 8-oxoG damage (12), can impact mutational signature analysis, and can be visualized using “Lego” plots (Fig. 1d-e). These plots show the error rate of each type of base substitution in its 3-base sequence contexts (2). The damaged bases reported by Chen et al.’s script (Fig. 1d) have a severely distorted error profile inconsistent with the previously reported (12) 8-oxoG damage pattern (Fig. 1e, C>A errors in the Lego plot with a clear peak at the sequence context CCG>CAG), whereas the corrected GIV_{G_T} script (and the previous implementation in (2)) results in a distribution of errors (Fig. 1e) consistent with 8-oxoG damage. Thus, 8-oxoG damage does in fact have a sequence context.

(iii) Interpretation of 8-oxoG damage levels

The authors state that “73% of the TCGA sequencing runs showed extensive damage, with a GIV_{G_T} > 2”. A GIV_{G_T} metric of 2 indicates that the G>T error rate in the 8-oxoG mode is twice the error rate of the non–8-oxoG mode (background rate for G>T errors). Even if the 8-oxoG artifacts are twice the context-specific background level (i.e., GIV_{G_T}=2), this corresponds to only a 5–10% increase in the overall base-level error rate (summed over all sequence contexts, Fig. 2a-c), which is less than the inter-sample variability of error rates at a fixed oxoQ. A 5% increase in the base-level error rate results in a minor, if any, increase in false-positive mutation calls (Fig. 2e-f), since calling algorithms are designed to handle typical levels of sequencing error. Only at GIV_{G_T} ≳ 5 (equivalent to oxoQ ≲ 35) do the additional errors from 8-oxoG become comparable to the sum of all other errors and adversely impact variant calling. The vast majority of samples in TCGA exhibit only minor 8-oxoG damage that has minimal impact on mutation calling.

Figure 2.

Comparison of 8-oxoG related error rates and other error modes. Samples (A-C) with corrected GIV_{G_T} damage levels below 5 (equivalent to oxoQ & 35) have fewer 8-oxoG–related base errors (red) compared to other sequencing error modes (blue). (A) Count of tumors with a specific corrected GIV_{G_T} score (majority of TCGA samples have very low damage levels). (B) Percentage of additional base mismatches caused by 8-oxoG in relation to GIV_{G_T} score. (C) 8-oxoG–related error rates (red) and other error modes (blue). Other modes dominate until corrected GIV_{G_T} & 5. (D) Copy of Fig 4E from Chen et al. (1). (E) Our estimated FDR vs. corrected GIV_{G_T} for &1,900 TCGA tumors from MuTect (14) before 8-oxoG filtering (GIV_{G_T} < 7.5). (F, G) Estimated FDR vs. corrected GIV_{G_T} for &1,900 TCGA tumors from MuTect before (F) and after (G) 8-oxoG filtering (as described in (2)). Results demonstrate no inflation in number of mutation calls after applying the 8-oxoG software filter.

(iv) Estimation of the false positive rate (FPR) of mutation calling

The authors define an FPR metric, which suggests that mutation calls for many samples in public databases contain >50% falsely detected somatic variants due to sequencing artifacts (Fig. 2d). However, their metric does not reflect the actual somatic mutations used for analyses in (3–10) and other TCGA publications (e.g., a recent reanalysis of TCGA data (13)), but instead represents candidate variants supported by as few as two non-reference reads (not considered as somatic variants by most mutation callers (14)). Moreover, the mathematical definition of their FPR metric is neither a FPR nor a false discovery rate (FDR), which is highlighted by the fact that it can range between −1 and 1 (and not between 0 and 1). Consequently, Chen et al. incorrectly concluded that the FPR exceeds 0.5 in samples that have a low level of DNA damage with nearly no damage-induced false-positive mutation calls (Fig. 2d-g).

(v) Finally, Chen et al. overlook publications that describe 8-oxoG damage (2–10) and that TCGA and other projects have mitigated the 8-oxoG effects with laboratory protocols and software filtering strategies when sequencing library re-generation was impractical (Fig. 2f-g). Moreover, they claim that “recent submissions to TCGA (November to December 2015) displayed similar G-to-T imbalances” but mistakenly interpreted the time that the data repository was last updated as the time of sequencing data generation (repository updates occurred periodically for unrelated issues). The actual dates of generating the sequencing data should be obtained from the sequencing metadata. In fact, most TCGA samples sequenced after October 2012 had low 8-oxoG damage due to improved library preparation (as described in (2)).

In conclusion, we disagree with the assessment of Chen et al. regarding the quality of published cancer genome projects. Raw sequencing data inherently contains errors; therefore, to avoid misinterpreting the data, it is important that researchers use established procedures and carefully curated datasets in downstream analyses.