Expectations and blind spots for structural variation detection from long-read assemblies and short-read genome sequencing technologies

Summary

Virtually all genome sequencing efforts in national biobanks, complex and Mendelian disease programs, and medical genetic initiatives are reliant upon short-read whole-genome sequencing (srWGS), which presents challenges for the detection of structural variants (SVs) relative to emerging long-read WGS (lrWGS) technologies. Given this ubiquity of srWGS in large-scale genomics initiatives, we sought to establish expectations for routine SV detection from this data type by comparison with lrWGS assembly, as well as to quantify the genomic properties and added value of SVs uniquely accessible to each technology. Analyses from the Human Genome Structural Variation Consortium (HGSVC) of three families captured ~11,000 SVs per genome from srWGS and ~25,000 SVs per genome from lrWGS assembly. Detection power and precision for SV discovery varied dramatically by genomic context and variant class: 9.7% of the current GRCh38 reference is defined by segmental duplication (SD) and simple repeat (SR), yet 91.4% of deletions that were specifically discovered by lrWGS localized to these regions. Across the remaining 90.3% of reference sequence, we observed extremely high (93.8%) concordance between technologies for deletions in these datasets. In contrast, lrWGS was superior for detection of insertions across all genomic contexts. Given that non-SD/SR sequences encompass 95.9% of currently annotated disease-associated exons, improved sensitivity from lrWGS to discover novel pathogenic deletions in these currently interpretable genomic regions is likely to be incremental. However, these analyses highlight the considerable added value of assembly-based lrWGS to create new catalogs of insertions and transposable elements, as well as disease-associated repeat expansions in genomic sequences that were previously recalcitrant to routine assessment.

Main text

The field of genomics has seen remarkable advances in the accuracy and efficiency of massively parallel sequencing-by-synthesis technologies that generate pairs of short reads from the ends of small 400–800 base pair (bp) fragments (referred to herein as short-read whole-genome sequencing [srWGS]). This technical leap and derivative approaches such as targeted whole-exome capture sequencing (WES) have catalyzed a deluge of gene discoveries for rare diseases and insights into population genetics and genome biology. Correspondingly, srWGS has been adopted by all major human disease and biobank sequencing initiatives, including the NHGRI Centers for Common Disease Genomics (CCDG)¹ and Centers for Mendelian Genetics (CMG),² the Deciphering Developmental Disorders (DDD) project,³ the Trans-Omics for Precision Medicine (TOPMed),⁴ the All of Us Research Program,⁵ the NICHD Gabriella Miller Kids First (GMKF) initiative, the UK BioBank,⁶ and Genomics England,⁷ to name just a few. As such, a critical step for the field is to establish uniform methods for srWGS data processing and rational benchmarking standards to set expectations for variant detection.

The technical processes of genome alignment and single-nucleotide variant (SNV) detection have been an intensive focus of genomics since the inception of the 1000 Genomes Project⁸ and more recently updated for cross-institute functional equivalence as part of the NHGRI Genome Sequencing Program and variant detection with the Genome Analysis Toolkit (GATK) best practices.^{9, 10, 11} However, no comparable standardized methods have been adopted for detection of structural variants (SVs), defined here as genomic alterations greater than 50 bp, from srWGS, and there are limited gold-standard benchmarking approaches for SV discovery. This lack of uniformity has introduced a barrier to establishing reliable estimates of the SV counts and characteristics per genome. Not surprisingly, as shown in Figure 1A, these estimates have varied considerably across studies. The initial discovery effort from the 1000 Genomes Project^12,13 revealed the landscape of SVs that could be captured from srWGS with just 4–7× coverage (3,431 SVs per genome). More recent population genetic and human disease studies using deeper (30× or higher) srWGS and diverse analytic methods have varied in estimates of SVs that can be captured via srWGS; these estimates vary from 401 to 10,884 per genome. At present, the most sensitive studies have utilized the integration of multiple SV detection methods from the Genome Aggregation Database (gnomAD) and the Human Genome Structural Variation Consortium (HGSVC) projects (Figure 1A).^{1,14,15,13, 16, 17, 18, 19, 20}

Figure 1.

Comparison of SV callsets from srWGS and lrWGS

(A) The substantial increased yield of lrWGS in SV detection is displayed from the HGSVC¹⁴ and the largest Pacific Biosciences (PacBio) lrWGS study published to date¹⁵ by comparison with contemporary srWGS studies. As shown, there is wide variability of SV detection across srWGS studies to date that report SVs detected per individual in more than 100 genomes. Parenthetical numbers next to each study label indicate the number of genomes analyzed, and bold numbers next to each bar represent the number of SVs per genome reported by each study.

(B) Overlap of SVs from the HGSVC srWGS and lrWGS callsets across children of the three trio families, partitioned by SV class.

(C) Distribution of repetitive sequences across the genome, genes, and exons. “Constrained” refers to genes and exons with pLI > 0.9,²¹ and “OMIM genes” includes a curated list of autosomal dominant genes that were defined in both Berg et al.²² and Blekhman et al.²³ Gb, gigabase; Mb, megabase. Percentage listed within each bar is the fraction of each group composed of "unique + RM" sequences.

(D) Distribution of SVs from srWGS and lrWGS split by repetitive sequence context. Formatting conventions are the same as in (C).

(E and F) Concordance of deletions (E) and insertions and duplications (F) between srWGS and lrWGS split by repetitive sequence context.

Emerging long-read WGS (lrWGS) technologies, which involve sequencing thousands to millions of contiguous nucleotides from a single strand of DNA, have significantly increased sensitivity for SV discovery in the human genome. The most widely tested lrWGS technologies include single-molecule real-time (SMRT) sequencing from Pacific Biosciences (PacBio)²⁴ and sequencing by ionic current through a nanopore channel (Oxford Nanopore Technologies [ONT]).²⁵ A key advantage of lrWGS is the abundance of reads that span entire SVs, allowing for direct observation of SVs rather than SV detection by inference as required for srWGS. These unique properties of lrWGS are beginning to revolutionize de novo assembly approaches,^26,27 and methods are already maturing for telomere-to-telomere assembly of individual human chromosomes.^15,28,29 The most recent analyses used the combination of multiple sequencing platforms (e.g., lrWGS, strand-specific sequencing,³⁰ and optical mapping³¹) in relatively small numbers of genomes to generate assembly-based SV callsets,^14,32 which have approximately doubled the number of SVs able to be captured in each genome to ~25,000 as compared with srWGS^14,15 (Figure 1A).

These lrWGS studies have thus opened access to SVs in the genome that were traditionally refractory to discovery by srWGS or interpretation in disease association studies, such as repeat expansions and other alterations within repetitive genomic regions and centromeres.³³ Unfortunately, the current cost of lrWGS is a significant premium over srWGS. For example, as of this writing the cost for generation of PacBio lrWGS over srWGS for equivalent coverage at leading academic platforms from the HGSVC ranges from 5.9-fold increase for continuous long-read technology to 12-fold increase for circular consensus sequencing HiFi technology. Moreover, the comparatively lower throughput of modern lrWGS platforms renders them impractical for adoption in large-scale population studies on the order of tens to hundreds of thousands of individuals. The largest published assembly-based PacBio study to date has analyzed just 15 genomes,¹⁵ and a recent preprint from the HGSVC describes 35 genomes,³⁴ while a published study from Iceland analyzed 3,622 ONT genomes.³⁵ By comparison, millions of genomes have already been sequenced or commissioned via srWGS across international initiatives. Given this predominance of srWGS in the current landscape of genomics research, we present here a series of analyses from the HGSVC to (1) benchmark expectations for the number and class of variants that can be reliably detected from srWGS, (2) predict the genomic features that drive false positive and false negative discoveries for each technology, and (3) establish the scientific and clinical advances offered by state-of-the-art lrWGS assembly as a complementary approach to srWGS.

In this study, we performed a detailed comparison of SVs detected from alignment-based srWGS and assembly-based lrWGS methods on three matched trio families (HG00514, HG00733, and NA19240) from the 1000 Genomes Project, and all results per genome reported here are averages across the three children in these families.¹⁴ For srWGS, this initial study applied a highly sensitive ensemble approach to integrate 13 SV detection algorithms (supplemental material and methods) and discovered an average of 10,884 SVs per genome. The emphasis on sensitivity suggests that approximately 11,000 SVs per genome most likely reflects an upper bound on the total number of SVs that can be routinely captured from srWGS with the alignment-based algorithms applied by the HGSVC, as demonstrated in Figure 1A by comparison with other contemporary studies. However, this sensitivity came at the significant cost of specificity: 685 de novo SVs were observed per genome, over 1,000-fold more than our expectation from srWGS based on family studies, population genetic estimators, and molecular validation, therefore representing many SV predictions that are most likely false positives.¹⁶ The lrWGS-derived SV callset combined whole-genome phasing with two state-of-the-art genome assembly approaches (Phase-SV and MS-PAC^14,26,36) and was supplemented by additional technologies (HiC³⁷ and StrandSeq,³⁸ see Chaisson et al.¹⁴). These methods discovered an average of 24,825 haplotype-resolved SVs per genome, or over 2-fold more than the most sensitive srWGS approaches. Surprisingly, although the srWGS and lrWGS callsets were generated on identical samples, only a limited subset of SVs (66.8% of srWGS and 33.5% of lrWGS) overlapped between technologies. Moreover, the mutational class of SVs dramatically impacted concordance: 60.6% of srWGS and 48.7% of lrWGS deletions demonstrated overlap as compared with 81.7% of srWGS and 24.1% of lrWGS insertions (Figure 1B).

We sought to define and quantify the factors contributing to the poor concordance between SVs derived from each technology to improve SV discovery, filtering, and prioritization from srWGS in future large-scale medical and population genetic initiatives. We first explored the role of genomic features such as repetitive sequences that are enriched for SVs via repeat-mediated mechanisms^39,40 because short-read alignment has well-documented limitations within these genomic regions.^41,42 We annotated all SVs with sequence context based on RepeatMasker⁴³ and segmental duplication⁴⁴ tracks from the UCSC genome browser.^45,46 For simplicity, we consolidated all repetitive sequence annotations into three categories: segmental duplication (SD; 5.1% of the genome), simple repeat (SR; 4.6%), and “repeat masked” (RM; 42.9%), where this RM category referred to all other repetitive sequence not overlapping SD or SR elements. The remaining 47.4% of the genome not overlapping any of these repeat categories was labeled as “unique” sequence, which is a term used for simplicity here, although these regions are not completely devoid of repetitive sequences. The "unique" and RM categories collectively encompass 90.3% of the annotated human reference sequence, 90.9% of all currently annotated protein-coding sequence, 95.8% of all currently annotated coding sequence from evolutionarily constrained genes, and 95.9% of genes currently associated with human disease from the Online Mendelian Inheritance in Man (OMIM; Figure 1C).^{21,47, 48, 49}

As expected, the distribution of SVs was non-uniform and varied by sequence context for each technology (Figure 1D). Most prominently, the enrichment of SV breakpoints in highly repetitive genomic sequences (SD + SR regions) was dramatic and their distribution differed significantly between technologies: despite representing just 9.7% of the reference genome, SD + SR annotated sequences contained at least one breakpoint from 49.8% of all SVs from srWGS and 70.4% of all SVs from lrWGS (p < 2.2e−16 for both technologies, chi-square test, Table S1, see supplemental material and methods for details). This enrichment of SVs in repetitive sequence was also strongly correlated with concordance between srWGS and lrWGS: SVs located in repetitive SD + SR sequences displayed 57.0% concordance among srWGS variants and 22.5% in lrWGS variants, whereas those ratios improved considerably in less repetitive sequences ("unique + RM") to 76.5% in srWGS and 59.9% in lrWGS (Figures 1E and 1F).

Although the divergent distributions and diminished concordance of SV detection by technology aligned with expectations for SD + SR regions, the paucity of overlap between technologies in "unique + RM" regions was unexpected because breakpoints localized to these regions should not suffer from the same technical confounders of SV discovery in highly repetitive sequences. Therefore, we next sought to decouple and quantify the discordance driven by underlying biological features of the genome from technical noise driven by false positive SVs present in the underlying HGSVC callsets that were optimized for sensitivity as described above. We also reasoned that identifying the covariates that have the greatest influence on false positive SV calls would be valuable in guiding the human genetics community toward principled improvements in SV detection and filtering algorithms. To accomplish this, we developed an in silico SV assessment to improve the precision of srWGS and lrWGS callsets in non-repetitive regions. This procedure re-evaluated the following three pieces of orthogonal information from both lrWGS and srWGS for each SV: (1) supporting evidence from raw lrWGS reads in the parent and offspring genomes for the presence of an SV (VaPoR;⁵⁰ Figure 2A); (2) copy states based on srWGS normalized read depth within SVs (Figures 2B and S1); (3) discordant paired-end and split reads information at the breakpoint of each predicted SV (Figures 2C, 2D, and S2, Table S2). We considered the SVs with one or more modes of supporting evidence as “high confidence” and explored their overlap on the basis of repeat context for SV calls from different technologies (see supplemental material and methods for further details).

Figure 2.

Methods to recalibrate SVs in "unique + RM" sequences based on read-level alignment signatures

(A) In silico evaluation results from VaPoR on deletions (pink background), insertions (purple background), and duplications (blue background). Duplications and insertions reported by srWGS were both compared against insertions from lrWGS. “Concordant” represents SVs discovered by both lrWGS and srWGS, and “technology-specific” represents SVs specifically discovered from one technology.

(B) Distribution of normalized read depth of srWGS across deletions (pink background), insertions (purple background), and duplications (blue background) that were supported by VaPoR (red) and the 1 kb genomic regions that flank these SVs (gray).

(C and D) Distribution of aberrant srWGS read pairs (C) and split reads (D) around deletions (pink background), insertions (purple background), and duplications (blue background) that were either homozygous (red), heterozygous (green), or false positives (blue). The homozygous, heterozygous, and likely false positive SV sets were selected with the criteria described in the supplemental material and methods.

(E and F) Concordance of deletions (E) and insertions and duplications (F) in "unique + RM" sequences that were supported by the in silico SV refinement procedure. Percentages represent the fraction of total variants shared between srWGS and lrWGS.

We initially applied this in silico SV refinement procedure to deletions, which represent the most interpretable class of SVs for genomics applications. As expected, the in silico confirmation rate—i.e., the proportion of SVs supported by one or more of the evidence classes described above—was high (93.5%) for deletions concordant between technologies in "unique + RM" regions compared to just 13.5% and 33.1% for those that were only discovered by a single technology for srWGS or lrWGS, respectively (Figure S3). After restricting to high-confidence SVs, we observed a substantial improvement in concordance: 93.5% to 93.8% of deletions were shared between srWGS and lrWGS (Figure 2E). Although mutational processes such as somatic SVs or sub-clonal mutations arising in cell culture can contribute to false positive findings, these results implied that most of the discordance between srWGS and lrWGS for SV discovery in the 90.3% of the genome not encompassed by SD + SR sequence was most likely technical in origin. Importantly, it appeared that most of the discordance wasdriven by false positive SV calls that can be pruned by post hoc heuristic filtering.

We next explored the impact of post hoc filtering on SVs other than deletions. While duplications and insertions were reported as separate SV classes by srWGS, the lrWGS methods applied by the HGSVC treated both classes as insertions. Given this, we considered all srWGS duplications as insertions for subsequent comparisons. In contrast to the strong concordance between srWGS and lrWGS observed for deletions, 45.5% of high-confidence lrWGS insertions in "unique + RM" regions had no matching SV call from srWGS, while the majority (96.0%) of srWGS insertions and duplications were captured by lrWGS (Figures 2F and S4). To investigate the properties of insertions specifically captured by lrWGS in "unique + RM" sequences, we aligned the assembled sequences of high-confidence insertions against a catalog of known repeat elements.⁴³ Most of these insertions aligned to specific types of repeat elements (61.8%, n = 2,485/genome), such as short interspersed nuclear elements (SINEs, n = 1,494/genome), long interspersed nuclear elements (LINEs, n = 312/genome), and long terminal repeat (LTR, n = 139/genome) retrotransposons (Figures 3A and 3B). Notably, a “chimeric” alignment pattern was observed for 31.7% of the insertions specifically discovered by lrWGS where inserted sequences were aligned to multiple different repeat types (Figures 3B and 3C). These results indicate that the complexity of insertion repeat structure is a major determinant of srWGS sensitivity for insertion SVs, as has been previously demonstrated for certain classes of nested insertions.⁵¹ We further observed high variability in the current capabilities of srWGS detection algorithms depending on the type of transposable element insertions when comparing with lrWGS: 74.9% of SINEs, 42.6% of LINEs, and 50.7% of LTRs detected by lrWGS were also discovered by srWGS (Figure 3D). Intriguingly, almost all (95.8%) of the high-confidence lrWGS insertions in "unique + RM" regions that were only discovered by lrWGS nevertheless had some detectable support in the raw srWGS data, indicating that continued development of insertion detection algorithms could substantially improve sensitivity for identification of this variant class from srWGS (Figure 3E). Taken together, these analyses indicate that lrWGS and assembly-based approaches provide substantial improvements over srWGS for insertion discovery, particularly for those events with complex repeat structures.

Figure 3.

Alignment of assembled lrWGS insertion sequences against known repeat elements

(A) Count of lrWGS insertions in "unique + RM" sequences per genome by alignment of inserted sequences to known repeat elements. The number on top of the bar represents the averaged count of high-confidence insertions in "unique + RM" sequences per genome.

(B) Count of lrWGS insertions that are specifically discovered by lrWGS and shared by srWGS, by alignment of inserted sequences to known repeat elements. Formatting conventions are the same as in (A).

(C) An example of an insertion SV assembled by lrWGS, annotated with sequences that align to known repeat element classes. White shading represents sequences not annotated as a known repeat element.

(D) Counts of lrWGS insertions in "unique + RM" sequences per genome by the class of inserted sequence and the proportion that was overlapped by srWGS. “OTH^∗” represents insertions aligned to multiple known repeat elements, such as the example shown in (B). “OTH#” stands for insertions that were not aligned to any repeat elements. Numbers in parentheses represent the proportion of insertions that were overlapped by srWGS.

(E) Count of split reads around the lrWGS high-confidence insertions in histogram.

We also examined SVs in highly repetitive SD + SR regions by using the same in silico evaluation framework (Figures S5A–S5D) as described above with the caveat that the orthogonal evaluation of variants in these regions is more challenging and prone to false positives due to alignment artifacts that do not arise in the less repetitive regions of the genome. Similar to the "unique + RM" regions, insertions were poorly captured by srWGS, and only 17.0% overlapped lrWGS insertions, while 74.0% of srWGS insertions were captured by lrWGS (Figure S5F). The high concordance for deletions in "unique + RM" sequences also dissipated in these more repetitive SD + SR regions, as the concordance was 69.6% and 40.4% of high-confidence deletions from srWGS and lrWGS that were shared by the other technology, respectively (Figure S5E).

Finally, we explored the concordance of SV detection for a class of SVs that is strongly enriched for pathogenic variation and appears to be a significant blind spot for long-read assembly technologies: large CNVs captured by depth-based analyses from srWGS. Our initial analyses suggested that lrWGS assembly methods failed to capture all but one of the small number of large (>5 kb) CNVs that could be detected by srWGS read-depth methods in three probands (average size = 14.7 kb). Recognizing the limitation of read-depth analyses to capture large CNVs in a small number of families, we explored CNV calls from 3,202 individuals from our ongoing analyses of 30× srWGS in the 1000 Genomes Project that included all three families used in this study (see HGSVC preprints for complete details).^34,52 We found an average of 167 large CNVs per genome that were exclusively detected by depth-based methods, 88.2% of which were not detected by lrWGS assembly. These findings highlight an important blind spot in variant detection from lrWGS assembly in the absence of depth-based analyses and have significant implications for human disease studies because large CNVs have a profound deleterious impact on a spectrum of human diseases.

In conclusion, we demonstrate the strong influence of genomic context on expectations for SV detection from srWGS in genomic studies, as well as estimating the anticipated yields of emerging lrWGS technologies. Initial surveys have implied highly variable outcomes and limited overall concordance in SV detection between the two technologies;¹⁴ however, in-depth analyses of these variants emphasize that genome organization, variant type, variant size, and high type I error rates in SV detection from each technology were the predominant features driving discordance. After applying post hoc filters to correct for the relatively high type I error rates for SV detection from this ensemble srWGS approach optimized for sensitivity, and the assembly-based lrWGS approach that was optimized with orthogonal data types, we were able to extrapolate the informative genomic features that influence differences in SV distributions between technologies. The concordance between srWGS and lrWGS was remarkably high for deletions localized to the least-repetitive regions of the genome (93.8%), while almost all lrWGS-specific deletions were localized to repetitive SD + SR regions. We observed poor sensitivity in the detection of large CNVs (>5 kb) via lrWGS assemblies by comparison with srWGS, and this limitation is most likely due to the lack of depth-based lrWGS methods. In contrast, lrWGS showed superior sensitivity for detection of insertions regardless of the genomic context, although most (95.8%) insertions in the least-repetitive genomic regions had detectable alignment signatures in the srWGS data, indicating further improvement in insertion discovery methods for srWGS should continue to bridge this disparity in insertion detection between technologies. Variant types other than deletions and insertions (e.g., inversions, translocations, balanced and complex SVs) were excluded from these analyses because they were not uniformly called by lrWGS assemblies, although we expect future improvement in lrWGS methods to provide novel insights into repeat-mediated mechanisms for these variant classes.

The value added for long-read assembly to discover new disease-associated SVs, or to provide resolution to “unsolved” cases in Mendelian genetics research and clinical diagnostics, is thus a complex calculus. As we note above, srWGS captures virtually all high-quality deletions derived from lrWGS assembly in the regions of the genome that encompass over 95% of currently annotated coding sequence in genes with existing evidence for dominant-acting pathogenic mutations from OMIM. We therefore anticipate that a minority of “unsolved” cases will be explained by novel and readily interpretable deletions that can be captured by lrWGS but remain cryptic to srWGS in known disease-associated genes. However, given that the most highly repetitive regions of the genome have been traditionally inaccessible in human disease studies, it is anticipated that new disease-associated genes and sequences will emerge as functional annotation of these repetitive sequences and duplicated genes continues to improve. Indeed, germline and somatic repeat expansions and contractions are already well established mechanisms of human disease, particularly neurodegenerative disorders.⁵³ As telomere-to-telomere assembly methods continue to mature and eventually reach into centromeres, telomeres, and other highly repetitive regions, the catalog of disease-associated variants will certainly expand beyond what is applied to current clinical interpretation. Moving forward, long-read technologies also offer the opportunity to detect novel transcripts from RNA-seq⁵⁴ and methylation status from technologies such as ONT, which will further expand the list of disease-associated variants.^{54, 55, 56}

Collectively, we estimate from these analyses that future genomic studies and clinical initiatives using srWGS can expect to capture upward of ten to eleven thousand SVs in each human genome, and current large-scale international initiatives are poised to provide exciting new insights into the 90% of the annotated reference genome that encompasses most known genic sequence. Our analyses also confirmed that assembly-based lrWGS methods will access regions of the genome that were previously intractable to conventional technologies and srWGS. We anticipate that advances in lrWGS technologies, and associated analytic approaches, will provide significant long-term value in expanding the catalog of functional variation associated with insertions, mobile elements, and the most challenging sequence features in the human genome.

Declaration of interests

The authors declare no competing interests.

Published: March 30, 2021

Data and code availability

Resource data used in this paper were generated by Chaisson et al. (2019)¹⁴. These data are available under dbVar: nstd152.

Web resources

• HGSV integration pipeline, https://github.com/xuefzhao/HGSV_SV_integration_pipe

• lrWGS data of HGSVC sample, ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/hgsv_sv_discovery/working/20160623_chaisson_pacbio_aligns/

• OMIM, https://omim.org/

• srWGS data of HGSVC sample, ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/hgsv_sv_discovery/data/

• svtk, https://github.com/talkowski-lab/svtk

• VaPoR, https://github.com/mills-lab/vapor