Prioritizing disease-linked variants, genes, and pathways with an interactive whole genome analysis pipeline

Abstract

Whole genome sequencing (WGS) studies are uncovering disease-associated variants in both rare and non-rare diseases. Utilizing the next-generation sequencing for WGS requires a series of computational methods for alignment, variant detection, and annotation, and the accuracy and reproducibility of annotation results are essential for clinical implementation. However, annotating WGS with up to date genomic information is still challenging for biomedical researchers. Here we present one of the fastest and highly scalable annotation, filtering, and analysis pipeline –gNOME – to prioritize phenotype-associated variants while minimizing false positive findings. Intuitive graphical user interface of gNOME facilitates the selection of phenotype associated variants, and the result summaries are provided at variant-, gene-, and genome-levels. Moreover, the enrichment results of specific variants, genes, and gene sets between two groups or compared to population scale WGS datasets that is already integrated in the pipeline can help the interpretation. We found a small number of discordant results between annotation software tools in part due to different reporting strategies for the variants with complex impacts. Using two published whole exome datasets of uveal melanoma and bladder cancer, we demonstrated gNOME's accuracy of variant annotation and the enrichment of loss of function variants in known cancer pathways. gNOME web-server and source codes are freely available to the academic community.

Introduction

The maturation of ultra-high throughput sequencing technology has opened a new era of personal genome sequencing (Ashley et al., 2010; Chang and Wang, 2012; Cirulli and Goldstein, 2010; Drmanac, 2011; Kidd et al., 2012; Meyerson et al., 2010; Tabor et al., 2011), and shifted the researcher's burden from the identification of genetic variants to the interpretation of large numbers of variants in each individual. Although studies using whole genome sequencing (WGS) in a large disease population might still be a few years away, proof-of-concept studies on WGS and whole exome sequencing (WES) have already proven the technology to be useful in identifying disease-causing mutations in rare Mendelian disorders (Bamshad et al., 2011; Hoischen et al., 2010; Klassen et al., 2011; Lalonde et al., 2010; Lupski et al., 2010; Ng et al., 2010a; Ng et al., 2010b; Roach et al., 2010). Moreover, a few studies using the case-control study design also demonstrated the utility of WGS and WES in identifying disease-associated genomic variants for non-rare diseases (Calvo et al., 2010; Holm et al., 2011; Pelak et al., 2010; Rivas et al., 2011).

An individual human genome has 3-4 million variants, or locations that differ from the human reference genome. Because of the large number of variants, it is essential to filter out those weakly associated with the researchers' target phenotype and to reduce these variants down to a manageable number. This can be done by means of various heuristics such as allele frequencies (AFs) and their impacts on protein functions (Cooper and Shendure, 2011; Goldstein et al., 2013). There is a tremendous need in the biomedical research community for a tool that can filter these millions of variants based on the most up-to-date annotations and utilize the growing arsenal of genome analysis methods.

The number of bioinformatics pipelines for analyzing WGS and WES is rapidly increasing. However, a majority of such tools focus on processing raw sequence data to detect high confidence genomic variants rather than focusing on downstream analyses such as annotation-based variant filtering and statistical analysis (Lam, et al., 2012; McKenna, et al., 2010; Pabinger, et al., 2013). Even available downstream analysis tools are limited by their 1) static filtering methods, 2) insufficient annotation, and 3) absence of multi-genome comparison methods (Cingolani et al., 2012; Ge et al., 2011; MacArthur et al., 2012; San Lucas et al., 2012; Wang et al., 2010; Yandell et al., 2011). Moreover, these tools are difficult to use for most researchers and clinicians due to the lack of an intuitive user interface. To overcome these limitations, we developed gNOME, an interactive downstream analysis pipeline that combines comprehensive genomic annotation sources with statistical analysis in an expandable framework. We demonstrated the accuracy of annotation using the validated genomic variants from published WES datasets. The pipeline is written in C++, Perl, and SQL, and all source codes are freely available to the academic community.

Materials and Methods

Overview of gNOME workflow

The goal of most WGS and WES studies is to find variants that are possibly associated with a phenotype of interest. A common approach toward this goal is to prioritize variants that have deleterious impact on protein function and/or are more frequently observed in cases compared to controls and an ethnicity-matched healthy population (Lim, et al., 2013). Following this strategy, gNOME's streamlined analysis workflow is as follows: 1) creating a project and uploading variant files, 2) annotation, 3) filtering variants using an interactive user interface, 4) statistical analysis, and 5) summarizing the results (Figure 1). We use a double colon (::) to indicate a menu::submenu in the gNOME interface and single quotation marks to denote selected values throughout the description.

Figure 1. A schematic overview of gNOME.

The analysis of whole genome and exome dataset starts with creating a project and uploading it according to project type (Step 1). The uploaded files are annotated with 60 annotation tracks (Step 2), and annotation-based variant filtering can be interactively performed (Step 3). gNOME supports variant-, gene-, and gene set-level association tests between two groups: case vs. ethnicity-matched population data from the 1000 Genomes Project or cases vs. controls (Step 4). Filtering and analysis results are dynamically reported on the web-based interface (Step 5). Steps 3 to 5 can be performed iteratively based on different variant-filtering criteria.

The first step is to define a project with a corresponding experimental design (i.e., ‘Case only’ or ‘Case vs. Control’) and to upload variant files (Step 1). The pipeline supports the variant call format (VCF) (Danecek et al., 2011), genome variation format (GVF) (Reese et al., 2010), and Complete Genomics' VAR file format. Each variant file should be assigned to groups (either ‘case’ or ‘control’) in a project with a specific reference genome build (i.e., ‘hg18’ and ‘hg19’). A set of samples in gNOME is distinguished by the project name and group label. The uploaded variant files are placed in the internal queuing system for annotating with 60 different sources of genomic information collected from 17 publicly available databases (Step 2) (see Materials and Methods and Supp. Table S1 for details). This step takes at most 30 minutes for an individual WGS with 4-5 million variants. For efficient handling of dataset with multiple genomes, it is recommended to upload them as multiple sample VCF files such that gNOME can speed-up the annotation step by processing the entire variants – union of variants found in any of genomes in the file – in a single step. Once the annotation is completed, each genome in the multi-individual VCF file is stored individually. For instance, the merged VCF file for 1,097 samples of the 1000 Genomes Project (1KGP) has only 39,706,715 variants, for which gNOME can complete the annotation in 50 minutes (see Results). The resulting annotated variant files are stored in an internal MySQL database. Once the annotation is completed, users will receive a notification email. The summary statistics for all uploaded variant call files are available on 3.Summary::Genome Level (Supp. Figure S1). This overview is available for a single genome at a time or for multiple genomes. When a group of genomes is selected, gNOME displays the average and range of summary statistics for each variant type.

In Step 3, users can select multiple criteria for annotation-based filtering through an interactive web interface. For instance, one can select rare or novel loss of function (LoF) variants at highly conserved loci that are exclusively found among cases but neither in controls nor in an ethnicity-matched population dataset. Also, during this step, possible false positives are reduced by filtering out low quality variants and variants found in repetitive regions. We grouped filtering options into 4 broad categories in the web interface: 1) Allele Frequency, 2) Functional Impact, 3) Knowledge Enrichment, and 4) Others (Figure 2A). The LoF variants at highly conserved loci that are rare or novel in the European population are selected by setting 1) Allele Frequency::Ancestry ‘European’ (Figure 2A-1), 2) Allele Frequency::Allele Frequency ‘≤ 1% (rare)’ (Figure 2A-2), 3) Functional Impact::Gene impact ‘LoF’ (Figure 2A-3), and 4) Knowledge Enrichment::GERP++ score ≥ ‘2’ (Figure 2A-4). We can exclude the variants with low calling quality scores by setting Other::Variant call score (Figure 2A-5). The selected variants or genes are displayed in a table that can be sorted by column, and are available for download as a tab-delimited text file, which can be used as an input for the other protein-protein interaction network based analysis tools such as DAPPLE (Rossin et al., 2011). 3.Summary::Variant Level lists the detailed annotations for all variants that passed the filtering criteria whereas 3.Summary::Gene Level shows the number of variants that met the criteria for each gene.

Figure 2. Discovering somatic mutations in tumor-blood paired whole exome sequences.

(A) A screenshot for comparing variants from tumor tissue (as ‘case’) and blood sample (as ‘control’), both of which come from a single patient (‘MM56’) (see Finding somatic mutations in uveal melanoma and Materials and Methods for detail). From both tumor tissue and blood sample, allele frequencies were estimated with (1) European ancestry, and (2) rare or novel (3) loss of function variants at (4) highly conserved loci were selected. Low-quality variants were excluded by setting (5) ‘Variant call score ≥ 20’. The potential somatic mutations were selected by choosing variants that were present in tumor sample but not in blood sample (6). (B) The result from the comparison shown in (A). The table can be searched for gene symbol or sorted by the columns. A total of 11 genes including BAP1 (displayed) met the criteria. gNOME performs a gene set enrichment analysis for 5 gene set categories with the genes that passed filtering criteria.

Group comparison, Step 4, is one of the unique features that distinguish gNOME from other WGS and WES annotation tools (Cingolani et al., 2012; Ge et al., 2011;Wang et al., 2010). Group comparison helps to identify a set of variants, genes, and gene sets that are significantly enriched in cases as described in Materials and Methods, and is also useful to identify possible false positive incidental findings such as platform-specific sequencing errors and hypervariable genes and gene sets (Kohane, et al., 2012). These genes can be easily identified in gNOME and filtered out for further analysis if desired (Figure 2A-6). 4.Analyze::Variants and 4.Analyze::Genes (Figure 2) identify interesting variants enriched in case genomes and genes with such variants. Additionally, in 4.Analyze::Genes, gNOME can test whether a set of genes with interesting variants are enriched in precompiled gene sets from the Gene Ontology terms, the Kyoto Encyclopedia of Genes and Genomes (KEGG) pathways, and the other disease-gene association databases (Figure 2B). We demonstrated the performance, accuracy, and group comparison features using 3 publicly available WGS and WES datasets.

Ethnicity specific allele frequencies of known variants

To calculate ethnicity specific AFs, we used The NCBI Short Genetic Variations database (dbSNP, http://www.ncbi.nlm.nih.gov/SNP, version 137), the 1KGP (Genomes Project et al., 2012) for European, Asian, and African populations and the Exome Sequencing Project (ESP, http://esp.gs.washington.edu) (Fu, et al., 2013) for European and African populations. The datasets with less than 15 samples in dbSNP were not used due to the inaccuracy in estimating AFs. We categorized AFs into four groups: common (AF ≥ 5%), less common (1% ≤ AF < 5%), rare (AF < 1%), and novel. The numerical codes -1 and -10 are used to represent a reported variant without a known AF and a novel variant, respectively. If AFs from different data sources were inconsistent, the highest value was used to represent the ethnicity specific AF. The same rule was applied for mixed ancestries.

Possible impact on protein function

Predicting the functional impact of amino acid changes resulting from nucleotide changes is an important step for prioritizing disease-associated genes since most known disease-causing variants are in protein-coding regions (Choi, et al., 2009). To provide possible consequences of genomic variants in genic regions, we integrated multiple gene models and prediction algorithms as part of gNOME. The Reference Sequence database (RefSeq, http://www.ncbi.nlm.nih.gov/refseq) (Pruitt et al., 2005), Consensus Coding Sequence (CDS) project (CCDS, http://www.ncbi.nlm.nih.gov/CCDS) (Pruitt, et al., 2009), Ensembl (Hubbard, et al., 2002) (http://www.ensembl.org), and University of California Santa Cruz (UCSC) Known Genes (Hsu et al., 2006) were all implemented in our gene annotation database. The use of multiple transcript models to estimate the functional impact of a variant is essential since possible consequences of a variant can be different across transcript models and an intronic variant in one transcript model can be in the coding region of the other transcript model (see Supp. Figure S2 for an example). Possible impacts of a variant on each transcript are categorized into synonymous, missense, in-frame insertion, in-frame deletion, splice site disruption, nonstop, misstart, frameshift, and nonsense. LoF variants were defined to include splice site disruption, frameshift, and nonsense. A broader category of LoF variants (adding nonstop and misstart) is also provided. We also annotate predicted impacts on protein function using the database for nonsynonymous SNPs' functional predictions (dbNSFP, https://sites.google.com/site/jpopgen/dbNSFP) (Liu et al., 2011) that comprises the predicted impacts estimated using the Sorting Intolerant From Tolerant (SIFT) (Kumar et al., 2009), PolyPhen2 (Adzhubei et al., 2010), MutationTaster (Schwarz et al., 2010), and a likelihood ratio test (LRT) (Chun and Fay, 2009).

Conservation scores, non-coding elements and biomedical knowledge enrichment

Conservation scores according to the Genomic Evolutionary Rate Profiling (GERP++, http://mendel.stanford.org/Sidowlab/downloads/gerp) (Davydov et al., 2010) are used to filter variants per locus, and we used an average score of GERP++ for insertions and deletions (indels). Genotyping errors are more frequently observed in repetitive regions, thus excluding the variants in these regions can reduce false positive findings. We used the RepeatMasker database (http://www.repeatmasker.org) to find any variants in these regions (Smit et al., 1996-2010). The default value for percent overlap with RepeatMasker regions is set to 0 %.

A variant on an important functioning protein domain could have a significant impact on protein function. Known protein domains were collected from the InterPro database (Hunter, et al., 2012) and mapped to the reference genome coordinates in order to facilitate variant annotations. The regulatory regions from the Encyclopedia of DNA Elements (ENCODE) project (Consortium, 2011) (http://encodeproject.org/ENCODE), the conserved transcription factor binding sites from UCSC Table Browser (http://genome.ucsc.edu), and microRNA host genes (Kozomara and Griffiths-Jones, 2011) were included in the annotation database to provide further information for non-coding functioning elements.

The pipeline includes gene sets for diseases, biological processes and canonical pathways. A total of 1,253 disease-associated gene sets were compiled through the gene-to-disease mapping using the literature abstracts annotated with Medical Subject Heading (MeSH) terms and NCBI Genes (Mitchell et al., 2003). The known disease-associated genes and variants from the catalog of genome-wide association studies by National Human Genome Research Institute (Hindorff et al., 2012) (http://www.genome.gov/gwastudies/), Online Mendelian Inheritance in Man (OMIM, http://www.omim.org) (McKusick, 2007), ClinVar (Riggs et al., 2013) (http://www.ncbi.nlm.nih.gov/clinvar), and DisGeNet (http://ibi.imim.es/DisGeNet/web/v02/home) (Bauer-Mehren et al., 2010) were integrated to the annotation database; the Human Gene Mutation Database (HGMD, http://www.hgmd.org) can be used if a user has the license. In addition, we collected 828 biological process gene sets based on the GO terms and 186 KEGG pathways from the Molecular Signatures Database (MSigDB, http://www.broadinstitute.org/gsea/msigdb, c5.bp.v3.0 and c2.cp.v3.0 respectively) (Subramanian, et al., 2005). The original data sources and processing scripts are available on the gNOME website, and the annotation database will be updated with the latest annotation information every 6 months.

Integration of population-scale individual whole genome sequencing data

The purpose of using ethnicity-matched population datasets as a comparison group is twofold. Firstly, the false positive incidental findings can be identified and reduced as previously described (Kohane, et al., 2012). Secondly, the genetic burden due to a set of interesting variants in the ethnicity-matched general population can be estimated and compared with study individuals. The comparison of uploaded data with population-scale data from the 1KGP is one of the unique features of gNOME. Following the categorization of the 1KGP, we included 18 different population categories: 4 by ancestry – European, Asian, African, and admixed American – and 14 by geographical regions.

Statistical comparison at variant-, gene-, and gene set-levels

A set of variants that met a user's annotation filtering criteria can be tested for enrichment in a group. Given two groups of samples (i.e., cases and controls), the interesting variants that are over-represented in cases can be identified as follows. Supposing the existence of N individuals and a total of M LoF variants, we defined M as the number of unique LoF variants across N individuals and set two groups as G₀ (for instance, case group) and G₁ (for instance, non-case group or ethnicity-matched population from the 1KGP). Then, an M-by-N matrix can be expressed as V = {ν_i,j}_{1≤i≤M,1≤j≤N} where ${\nu }_{i,j}=\{\begin{array}{cc}1& i‐\text{th variant found in}j‐\text{th individual}\\ 0& \text{otherwise}\end{array}$. The matrix V is illustrated as the colored grid box with 5 cases and 5 non-cases in Figure 3A. Row-wise hypergeometric tests find the variants that are more frequently found in G₀. Alternatively, the number of individuals with interesting variants can be set for each group as shown in Figure 2A-6. For instance, by setting N_{group A genomes} ≥ 1 and N_{group B genomes} ≤ 0, a set of interesting variants will be further filtered to a smaller set of variants that are exclusively present in Group A.

Figure 3. Association tests for variants, genes, and gene sets between two groups.

The small number of variants that remain after the annotation-based filtering can be associated with a phenotype in three ways. First, we can test whether a specific variant presents more frequently in cases compared to controls or an ethnicity-matched population (A). Second, an association test can be performed at the gene-level when each case individual may have different variants on the same gene (B). Third, we can expand a gene-level aggregation to gene set-level to find the gene set over-represented with interesting variants among cases (C). The rows marked by x (red) denote “hypervariable” variants, genes, or gene sets that frequently have variants in both cases and controls (see Materials and Methods for details).

With the exception of a few Mendelian disorders, the likelihood of finding the same disease-linked variants across the patients is low (McClellan and King, 2010). Instead, multiple rare LoF and missense variants in the same gene or the same pathway could alter disease risks. Burden tests and kernel-based tests compare the cumulative effects of such variants. In burden tests, each variant is weighted differently according to AF, the impact on protein function, and conservation scores (Han and Pan, 2010; Madsen and Browning, 2009; Morris and Zeggini, 2010; Price et al., 2010; Zawistowski et al., 2010). The most burden tests assume that all variants of interest contribute to the phenotype in the same direction while kernel-based tests such as Sequence Kernel Association Test (SKAT) (Wu et al., 2011) and C-alpha (Neale et al., 2011) combine both protective and deleterious effects as well as variant-variant interactions. In our proposed pipeline, we implemented a burden test with the equal weights for all variants selected for specific criteria. The genes with compound heterozygous variants where each variant met user-defined filtering criteria can be prioritized using 4.Analyze::Genes (Figure 2). gNOME aggregates interesting variants by counting the number of variants in each gene, and perform a gene-level association test. Without loss of generality, we can assume that the M variants are linked to the total of P genes and the membership of each variant to genes can be represented as P-by-M matrix G = {g_k,i}_{1≤k≤P,1≤i≤M}, where $g_{k,i}=\{\begin{array}{cc}1& i‐\text{th variant is linked to}k‐\text{th gene}\\ 0& \text{otherwise}\end{array}$. The matrix multiplication, B = G · V = {b_k,j}_{1≤k≤P,1≤j≤N}, gives us the number of variants in a gene for each individual (Figure 3B). The genes with a significantly different number of variants between two groups are ranked by a row-wise two group comparison test of B. If a gene is hypervariable (Kohane, et al., 2012), it would consistently have non-zero value (b_k,j > 0 for all j) and be less significant in group comparison test (marked with a red x in Figure 3B).

A gene-level association test can be further expanded to a set of genes that are functionally related or physically interacting with each other. Even if an individual gene showed weak association in gene-wise test, those genes can collectively contribute to a specific phenotype. Our analysis pipeline provides several options for gene set-level association tests. Firstly, one can perform a gene set enrichment test for the genes with interesting variants in each case (‘Enriched Gene Sets in group A’ function in Figure 2B). Secondly, for each individual in G₀ and G₁, we can prioritize the gene sets that are more frequently observed as significantly enriched among the individuals in G₀. We construct a contingency table T= (t₁₁, t₁₂, t₂₁, t₂₂) for each gene set per individual with the numbers of member genes with the selected variants (t₁₁), member genes without the selected variants (t₁₂), non-member genes with the selected variants (t₂₁), and non-member genes without the selected variants (t₂₂), where the genes in a given gene set are defined as member genes. The relationship between genes and gene sets is defined as Q-by-P matrix S = {S_l,k}_{1≤l≤Q,1≤k≤P} where $s_{l,k}=\{\begin{array}{cc}1& k‐\text{th gene belongs to}l‐\text{th gene set}\\ 0& \text{otherwise}\end{array}$. Then we can collapse the values in matrix B into 0 or 1 to indicate whether an individual has interesting variants in the gene: Ḃ = {ḃ_k,j}_{1≤k≤P,1≤j≤N} where ḃ_k,j = 1 if and only if b_k,j > 0. The four values in T are defined as:

$$\begin{array}{lll}{t}_{11}\hfill & =\hfill & {\sum }_k{s}_{l,k}{\dot{b}}_{k,j}\hfill \\ t_{12}\hfill & =\hfill & {\sum }_k{s}_{l,k}(1-{\dot{b}}_{k,j})\hfill \\ t_{21}\hfill & =\hfill & {\sum }_k(1-s_{l,k}){\dot{b}}_{k,j}\hfill \\ t_{22}\hfill & =\hfill & \mathcal{G}-t_{11}-t_{12}-t_{21}\hfill \end{array}$$

with the total number of genes ( ). The enrichment scores from the contingency tables constitute the Q-by-N matrix C = {c_l,j}_{1≤l≤Q,1≤j≤N} (c_l,j = the enrichment score of l-th gene sets in j-th individual) (Figure 4C). The row-wise two group comparison tests on C are performed either by using a non-parametric test with odds ratios or hypergeometric p-values or by comparing the proportion of individuals that passed a user-defined statistical threshold. Using the latter option, hypervariable gene sets with interesting variants can be identified (marked with a red x in Figure 3C).

Figure 4. Comparison of annotation results from 4 software tools.

For splice site disruption (A), nonsense (B), frameshift insertion and deletion (C), and nonstop (D) variants, we compared the annotation results from 4 different software tools by comparing genomic coordinate, alternative allele, and reported functional impact for each variant. The numbers next to tool names represent the total number of annotated variants in that category, and 4-way Venn diagrams show the concordant and discordant annotation results. Overall, the annotation results are comparable to each other between tools; however, splice site disruption has the most discordant results (A). ANNOVAR reports as frameshift even if such variants are found in canonical splice sites; however, gNOME and SeattleSeq report them as splice site disrupting variants. Supp. Table S3 lists the details on the discordant results.

Evaluation datasets

We used 3 datasets to evaluate gNOME's performance and to compare with other programs. The first dataset consists of 97 patients with transitional cell carcinoma (TCC) (Gui, et al., 2011). For each patient, paired tumor-blood samples were sequenced using WES. We downloaded the raw sequence files from the Sequence Read Archive (SRA, http://www.ncbi.nlm.nih.gov/sra) (Leinonen et al., 2011) (accession number SRA038181), and aligned them to the reference human genome (hg19) using Burrows-Wheeler Aligner (BWA) 0.6.2 (Li and Durbin, 2009). The potential variants were called by the Genome Analysis Toolkit (GATK) 2.3-4 following the guidelines in Best Practice v4 from GATK's website (http://www.broadinstitute.org/gatk). The second WES dataset consisted of tumor-blood pairs from two patients with uveal melanoma (Harbour, et al., 2010). The raw sequences (accession number SRA062369) were processed with the same alignment and variant calling procedure as described in the original paper. This dataset was used to compare the accuracy and performance of gNOME's annotation procedure with the other software tools. Finally, we used all variants from a single individual (NA12889) in 1KGP as well as the entire Phase I integrated call set from the 1KGP to evaluate the scalability with data size.

Comparison with the existing software tools

We compared the performance and annotation results of gNOME with published software tools. The list of software tools included ANNOVAR (http://www.openbioinformatics.org/annovar, latest downloaded on 05/27/2013) (Wang et al., 2010), SnpEff (http://snpeff.sourceforge.net, version 3.2) (Cingolani et al., 2012), and SeattleSeq annotation server (http://snp.gs.washington.edu/SeattleSeqAnnotation137, version 137). We downloaded the latest version and installed on the same workstation if source codes were available, or uploaded the same VCF file to the web server. We used the same RefSeq gene model (downloaded on 05/20/2013 from UCSC Table Browser) for ANNOVAR, SnpEff and gNOME. However, SeattleSeq annotation server was running with a slightly different version of RefSeq model at the time of experiment. We measured the wall clock time to complete annotation for functional consequences of variants using a WGS variant file and 39.7 million variants from 1KGP.

Results

Comparison of performance and annotation results from the other software tools

The annotation speed of gNOME was compared with those of ANNOVAR (Wang et al., 2010) and SnpEff (Cingolani et al., 2012) using variants from a single genome (NA12889) and the integrated variants from 1,092 individuals of the 1000 Genomes Project (1KGP) (Genomes Project et al., 2012). The annotation for functional consequences of variants based on the RefSeq gene model (Pruitt et al., 2005) was performed 10 times for each tool. All 3 tools finished the annotation procedure in a reasonably short amount of time. For a single genome, SnpEff showed the best performance (192.2 ± 4.47 seconds; mean ± standard deviation); however, with 39.7 million variants of 1KGP, gNOME completed the annotation in 484.2 ± 0.92 seconds compared to 3,263.0 ± 65.94 and 1,606.8 ± 43.28 seconds for ANNOVAR and SnpEff (Table 1). The annotation engine of gNOME – gSearch (Song et al., 2012) – was optimized to handle a larger dataset whereas the processing time for ANNOVAR and SnpEff increased linearly with the number of variants. It should also be noted that genomes uploaded to gNOME's web interface will be annotated using 4 popular gene models, taking 4 to 5 times longer than reported in Table 1; at most 30 minutes for a single genome and 50 minutes for 1,092 genomes from 1KGP.

Table 1. Comparison of annotation performance.

We used the variant files from a whole genome sequence and all concatenated variants of 1,092 individuals from the 1000 Genomes Project. For each tool, we repeated the annotation procedure 10 times to calculate the average time required to complete the annotation of variant consequences using RefSeq gene model (the standard deviations are shown in parentheses). All three tools perform reasonably quickly. The annotation time of ANNOVAR and SnpEff linearly increases with the number of variants; however, only 37.8% more processing time is required to complete the annotation of x8.7 larger variant file using gNOME.

Variant file	ANNOVAR	gNOME	SnpEff
A single genomea	395.2 (4.78)	351.4 (3.53)	192.2 (4.47)
1,092 genomesb	3,263.0 (65.94)	484.2 (0.92)	1,606.8 (43.28)

^aAll variants found in one individual (NA12889) from CEPH/Utah family (4,581,619 variants).

^bAll variants in the Phase 1 integrated call set from the 1000 Genomes Project (39,706,715 variants).

For the comparison of annotated functional impacts of variants, we included SeattleSeq, a web-based annotation server, as well as ANNOVAR and SnpEff. For this comparison, we concatenated 4 variant files from 2 tumor-blood pairs in the uveal melanoma dataset using the same gene model –RefSeq gene model (downloaded on 05/20/2013 from UCSC Table Browser) – for most tools to preclude discordant annotations due to different gene models (see Materials and Methods). Since each tool uses a different set of terminology to describe functional impacts, we mapped the various description terminologies used by separate tools, as listed in Supp. Table S2. Overall, the annotation results from different tools were similar. However, there were categories of functional impacts that were not reported by ANNOVAR and SeattleSeq (Table 2). For instance, a loss of start codon –categorized as misstart in gNOME – was not reported in ANNOVAR and SeattleSeq, and in-frame insertion and deletions were not listed in SeattleSeq.

Table 2. Comparison of annotation results from various programs.

The functional consequences for all variants found from 2 tumor-blood pairs of whole exome sequencing of uveal melanoma samples are compared (see Materials and Methods). The functional impacts are based on RefSeq gene definitions, and description terms are compared according to the Supp. Table S2. ‘NA’ implies that the corresponding term is not available for the program.

Category	Variant consequences	ANNOVAR	gNOME	SnpEff	SeattleSeq
Single nucleotide changes in coding sequence	Disrupt	125	137	119	92
	Missense	11,687	11,729	11,825	12,042
	Misstart	NA	22	22	NA
	Nonsense	120	118	119	121
	Nonstop	14	15	28	22
	Synonymous	12,970	13,051	13,105	13,206
Short insertions and deletions in coding sequence	Frameshift	113	126	131	148
	In-frame insertion	78	78	79	NA
	In-frame deletion	127	127	126	NA
Variants outside of coding sequence	5′-UTR	3,677	3,941	3,941	3,879
	3′-UTR	8,659	9,008	9,010	8,931
	Intron	262,678	267,803	267,867	253,821
	Intergenic	255,085	265,926	265,941	264,470

We found differences in the sets of variants for each functional impact category across 4 tools (Figure 4 shows Venn diagrams for splice site disrupting, frameshift, nonsense, and nonstop variants). The differences can be explained partly by the differences in RefSeq versions between SeattleSeq and the other 3 tools (SeattleSeq used genes in Sept. 2012 version from National Center for Biotechnology Information (NCBI), while we used genes in May 2013 version from UCSC Table Browser), by annotation errors in all programs, and by a discrepancy between tools in describing the same variants. Of 92 - 137 splice site disrupting (SSD) variants that were found by 4 tools, 69 were discovered in common. SeattleSeq missed 37 SSD variants that were found by 3 other tools, most of which were suspected from differences in gene models. ANNOVAR and gNOME took different approaches in describing the SSD variants due to insertions and deletions (indels). In gNOME, SSD has priority over other functional consequences if indels were found in canonical splice sites, and vice versa in ANNOVAR. In 21 cases of ambiguous SSD variants, e.g., insertions at exon-intron junctions, gNOME classified all ambiguous cases as SSD, while other programs report only part of them. The proportion of discordant annotations among programs was the smallest for nonsense variants. Among the nonsense variants that were not reported by gNOME, we found 4 erroneous annotations (2 from SnpEff, 1 from ANNOVAR and SeattleSeq each). There were 3 frameshift indels annotated as nonsense by ANNOVAR but as frameshift by the others. The 4 nonsense variants found only by SeattleSeq were due to an outdated gene model. For nonstop variants, 11 out of 15 discordant annotations between gNOME and other tools resulted from the annotation for possible selenocysteine, which was recognized only in gNOME. The other 4 were annotation errors from SnpEff and ANNOVAR. Discordant annotations for frameshift variants were more complex (see Supp. Table S3 for details); however, a majority of discordant annotations was due to the different approaches in classifying functional impacts. The details on discordant annotations between programs are summarized in Supp. Table S3. The functional impact of each variant must be evaluated by experts; however, there is a tremendous need for a standard method to describe variants with complex consequences.

Finding somatic mutations in uveal melanoma

To ensure the annotation accuracy of gNOME, we analyzed a published WES dataset from patients with uveal melanoma (MIM 155720). Harbour and colleagues sequenced two cases of matched tumor and peripheral blood samples using WES to find tumor specific somatic mutations on chromosome 3 (Harbour, et al., 2010). They found an inactivating mutation in each tumor sample on BAP1. One patient (MM56) had a nonsense mutation (p.W196X) whereas the other (MM70) had an 11-bp deletion (p.Q322fsX100) on the same gene. We processed the downloaded data as described in Materials and Methods, and uploaded the variant files from tumor tissues as cases and those from blood as controls to our pipeline and analyzed as depicted in Figure 2. The variant level analysis of gNOME for MM56 revealed 670 possible somatic mutation candidates in protein coding regions including p.W196X in BAP1. After filtering out non-rare (AF >1% in European population, the same ethnic group as the patients) or synonymous variants, 171 candidate variants were found. Of these, four nonsynonymous variants – 3 missense and 1 nonsense (p.W196X in BAP1) – were found on chromosome 3. Similarly, in MM70, the 11bp frameshift deletion in BAP1 was the only high impact tumor-specific nonsynonymous variant on chromosome 3. Interestingly, gene level analysis of gNOME found 11 genes – CEP89, FAM135A, GNAQ, HECTD4, HEXA, KCTD20, RAD17, TAS2R31, THBS3, TTLL1, and WSCD1 – that contain possible somatic mutations in both MM56 and MM70. Of these genes, frequent somatic mutations in GNAQ from patients with uveal melanoma were previously reported (Van Raamsdonk, et al., 2009), and the increased protein expression of HEXA was found in metastatic uveal melanoma (Linge, et al., 2012).

Discovering somatic mutations and enriched pathways in transitional cell carcinoma

Gui and colleagues sequenced 9 tumor-blood pairs from the patients with transitional cell carcinoma (TCC, MIM 109800), and found 465 predicted somatic mutations. Several genes such as ARID1A and CREBBP had different somatic mutations in tumor samples of different patients. Additionally, tumor-blood sample pairs from 88 patients with TCC - 37 non-muscle-invasive (NMI) TCC and 51 muscle-invasive (MI) TCC) were sequenced to find frequently mutated genes in MI-TCC and NMI-TCC. The downloaded data were processed using hg19 (see Materials and Methods). First, all 9 tumors samples were uploaded onto gNOME as cases, and 9 blood samples as controls. We compared the somatic mutation candidates that were identified in gNOME using 4.Analyze::Variants (N_{group A genomes} ≥ 1 and N_{group B genomes} ≤ 0) with the list from the original paper (Supplementary Table 3 in the original paper). Of the 208 somatic substitutions that were validated by genotyping or Sanger sequencing, 195 variants were accurately called and annotated using gNOME, except for 13 variants that were not called by our variant calling approach with hg19. One variant (g.chr1:22186113T>G on hg19) was found in the blood sample of B17 individual, but not in B17's tumor sample. Next, we tested whether the somatic mutation candidates were enriched for any gene sets using 97 tumor-blood pairs. For this analysis, we selected the LoF variants with AF ≤ 1% in Asian population, and p-value threshold ≤ 0.05, N_{group A genomes} ≥ 1, and N_{group B genomes} ≤ 97 in Statistical test parameters in 4.Analyze::Genes. Ninety six genes with the variants that met the criteria were enriched for the cancer related KEGG pathways such as cell cycle (adjusted p-value 0.0014), prostate cancer (adjusted p-value 0.0017), pathways in cancer (adjusted p-value 0.010), arrythmogenic right ventricular cardiomyopathy (adjusted p-value 0.008), and bladder cancer (adjusted p-value 0.013) (Supp. Table S4).

Discussion

gNOME enables users to interactively filter a large number of variants down to a small number of disease/phenotype linked variants dynamically reported at three different levels – variants, genes and gene sets – simultaneously. Additionally, gNOME applies non-parametric statistical tests to variant- and gene-level counts of filtered variants between two groups, as well as to gene set enrichment analysis for biological pathways and known disease-linked genes. With the web interface for interactive annotation-based filtering and statistical tests, we demonstrated our streamlined analysis procedure using two tumor-blood paired datasets – uveal melanoma and TCC. All validated genomic variants in the uveal melanoma dataset and 93% of 208 validated variants in the TCC dataset were accurately identified with gNOME, and new candidate variants and genes were found. Additionally, the cancer related pathways were found to be enriched with LoF variants that were exclusively found in tumor samples.

Precise identification of genomic variants with high accuracy, and the transparent annotation and filtering procedure are essential for clinical sequencing (Gargis, et al., 2012). A combination of the version control system for annotation database and graphical user interface allow gNOME to successfully reproduce results. Furthermore, gNOME can be installed locally as a stand-alone analysis pipeline with a secure storage device behind a firewall since the genome sequence information is confidential. Communications with other servers are not necessary once variant files are transferred. Moreover, enabling the other encryption and security features on MySQL database will make gNOME compatible with the Clinical Laboratory Improvement Amendments.

To further evaluate the annotation accuracy of our proposed pipeline compared to the other genome annotation software tools, we analyzed the same VCF file from the uveal melanoma dataset using ANNOVAR, SnpEff, and SeattleSeq. The results were similar in general, but the number of discordant results varied across functional categories. Aside from cases due to differences in gene models or programming errors, a majority of discordant cases came from variants whose functional consequences can be classified into multiple categories. For instance, the frameshift variants overlapping with canonical splice sites were reported either as frameshift in ANNOVAR or as splice site disrupting in gNOME and SeattleSeq. The ontology for describing the functional consequences of sequence variants and a consensus approach to describe the variants with multiple functional consequences should reduce the number of discordant annotations between tools (Eilbeck et al., 2005).

There are a few limitations of gNOME. First, due to the licensing issue, known disease associated variants in the HGMD were not integrated into gNOME although the integration itself was straightforward. Second, only a limited set of statistical tests were implemented. Adding diverse genetic burden tests such as the methods implemented in PLINK-SEQ (Neale et al., 2011), Efficient and Parallelizable Association Container Toolbox (EPACTS) (Kang et al., 2010), and SKAT (Wu et al., 2011) will improve the flexibility of gNOME. Finally, the scalability of gNOME would be much improved if run on a computing cloud. The current web-based version can process medium-sized datasets of up to 1000 individuals, and a local stand-alone version can be set up to accommodate with a larger dataset or to preserve the confidentiality. However, to analyze a larger dataset (tens of thousands individuals), the gNOME pipeline must be run on a computing cloud. A few WES/WGS tools do support cloud environment as backend: VAT (Habegger et al., 2012) for variant annotation and visualization, Crossbow (Langmead et al., 2009) for variant calling, SIMPLEX (Fischer et al., 2012) for WES analysis, and Galaxy (Goecks et al., 2010) and Taverna (Hull et al., 2006) as general workflow framework. However, these are either difficult to use or do not cover all streamlined process provided by gNOME.

To summarize, we have developed a downstream analysis pipeline for WGS/WES datasets that can perform accurate and reproducible annotation with a graphical user interface for annotation-based filtering. A group comparison is one of unique features that help to reduce possible false positive findings. We have provided a population scale WGS dataset as a part of the pipeline, which enables users to identify variants specific to cases compared to ethnicity-matched generally healthy population. With these strengths, gNOME will be of use to the biomedical research community.