Noncoding Variants Functional Prioritization Methods Based on Predicted Regulatory Factor Binding Sites

Abstract

Backgrounds:

With the advent of the post genomic era, the research for the genetic mechanism of the diseases has found to be increasingly depended on the studies of the genes, the gene-networks and gene-protein interaction networks. To explore gene expression and regulation, the researchers have carried out many studies on transcription factors and their binding sites (TFBSs). Based on the large amount of transcription factor binding sites predicting values in the deep learning models, further computation and analysis have been done to reveal the relationship between the gene mutation and the occurrence of the disease. It has been demonstrated that based on the deep learning methods, the performances of the prediction for the functions of the noncoding variants are outperforming than those of the conventional methods. The research on the prediction for functions of Single Nucleotide Polymorphisms (SNPs) is expected to uncover the mechanism of the gene mutation affection on traits and diseases of human beings.

Results:

We reviewed the conventional TFBSs identification methods from different perspectives. As for the deep learning methods to predict the TFBSs, we discussed the related problems, such as the raw data preprocessing, the structure design of the deep convolution neural network (CNN) and the model performance measure et al. And then we summarized the techniques that usually used in finding out the functional noncoding variants from de novo sequence.

Conclusion:

Along with the rapid development of the high-throughout assays, more and more sample data and chromatin features would be conducive to improve the prediction accuracy of the deep convolution neural network for TFBSs identification. Meanwhile, getting more insights into the deep CNN framework itself has been proved useful for both the promotion on model performance and the development for more suitable design to sample data. Based on the feature values predicted by the deep CNN model, the prioritization model for functional noncoding variants would contribute to reveal the affection of gene mutation on the diseases.

1. INTRODUCTION

In molecular biology and genetics, transcription factors (TFs) refer to a series of proteins that can specifically bind DNA sequences. It is specifically binding that TFs regulate genetic information on the process of transcription from DNA to mRNA [1]. To explore gene expression and regulation, the researchers have carried out many studies on transcription factors and their binding sites [2]. By binding on the DNA sequence in Promotor or Enhancer, TFs can increase or decrease the level of the gene transcription [3]. TFs can regulate gene expression by multiple mechanisms [4]. When acting alone or coordinating with other protein complex, TFs promote or depress the binding of the RNA polymerase to the DNA template [4-7]. Furthermore, TFs are the main regulators of the pluripotency of stem cells. Along with the elaboration to the functions of TFs, stem cell biology gets rapid development [8].

It was proved that a large amount of TF determined the fate of the cell by regulating the expression in specific cell during development process [9, 10]. With the development of the studies on TFBSs and the accumulation of the experimentally validated TFBSs, the data information on transcription regulation has been greatly enriched [11]. The complexities of these new models are increased, but more information on TFBSs is obtained [12].

At the mean time, the researches on multiple genome sequencing, ChIP-seq, the structures of chromatin and the integration of multiple information has promoted the accuracies of the TFBSs motif discovering algorithms [13].

In this paper, firstly we reviewed a few conventional algorithms to identify TFBSs and listed the main models and representations relating to TFBSs. Furthermore, we summarized the newest deep learning methods to predict the TFBSs. Meanwhile, using the values predicted by these deep learning methods, the functional prioritization of the noncoding variants, such as SNPs, can be efficiently calculated. It has been demonstrated that based on the deep learning methods, the performances of the prediction for the function of the noncoding variants are outperforming than those of conventional methods. The research on the prediction for functions of SNPs is expected to uncover the mechanism of the gene mutation affection on traits and diseases of human beings. In the last part of the paper, several models to prioritize the SNVs functions were summed up.

Single Nucleotide Polymorphisms (SNPs) refers to single nucleotide alteration in the DNA sequence which causes the diversity of genomes [14]. SNPs may appear in coding region, noncoding region or intergenic region of genes. The coding region SNPs may influence the amino acid sequence of a protein and then have an impact on the expression of the gene. The noncoding region SNPs may also affect gene expression by gene splicing, binding with transcription factors, mRNA degradation or other mechanisms [15, 16]. SNP is the main formation of DNA sequence mutation of human genomes and is connected with both the variation of organism traits and the production of disease [17]. The difference of SNP distribution characteristic causes the variety on ethnic origin, genetic disease susceptibility, physiological characteristics and appearance of human beings [18-20]. SNP is also the genomic level reflection of the above difference. So, the study on SNP is meaningful to both the health career and the evolution of human beings [21]. The majority of the disease-related SNPs are located in noncoding region whose working mechanisms are unclear. It is speculated that by specifically binding the TF or influencing on gene splicing, SNPs alter the gene expression, since a causal mutation of gene may affect the normal function of gene products [22]. It is discovered that the gene expression pattern mainly depends on regulatory factors, such as promoters and enhancers, which integrate various signal factors and transcription factors to regulate the expression of the genes [23]. Although great progress has been made, little is known about what is the mechanism on how the majority of functional mutations alter the gene expression.

To solve this issue, studies on prioritization of SNVs (Single Nucleotide Variants) have come into being in recent years, which use the machine learning methods to score the SNPs. Based on the pre-processed feature values, the prioritization pipeline can compute the score of a given SNP’s functionality. The method funseq2 [24] even predicts the driver variants of the cancers based on the prioritization. The later prioritization studies have used the features which are the outputs of CNN models. Both the superiority of the performance and the characteristics of just relying on SNP’s sequence information make the latter studies deserve further research in the future.

2. conventional methods to identify TFBS

2.1. Representation for the Pattern of TFBSs

Recognition of transcription factor binding sites (TFBSs) is to search for DNA sequence fragments similar in both functionality and pattern. Since the binding sites of the same transcription factor are highly conserved, the patterns “Consensus” [25] and “PSSM” (position-specific scoring matrix) [26] have been devised to describe the characteristic of the TFBSs. On each position of a “Consensus” sequence there is a nucleotide that occurs most frequently in an alignment. Besides, owing to the specificity of the TFBSs, the sequence fragments that are different from the “Consensus” in individual positions might be recognized as the TFBS which are referred as instances. For instance, it is discovered that there are close frequencies of two or three different nucleotides on the same sequence position. In this case, since the “Consensus” cannot express the conservative of the TFBS, “Degenerate consensuses” [27] has been usually used as the representation of a TFBS. Modeling the TFBS by statistical methods can flexibly and powerfully describe the nucleotides frequencies on each position of the site. The most-often used statistical model is the PFM(Position Frequency Matrix). However, taking into account the nucleotides bias in the constitution of the sequence, the PFM was commonly transformed into the PWM(Position Weight Matrix).

2.2. Methods Based on PWM Motif

The PWM methods went through a series of developments. The earlier methods used various strategies and approaches to find out the PWMs of the binding sites based on the binding sequences for a given transcription factor. But there were some disadvantages of the methods. The first was that they cannot represent the dependencies between the nucleotides in a binding site. The second was that they did not consider the relationship of separated binding sites of the same transcription factor. The later improvement on the PWM methods focused on the first one and a number of methods with dinuleotide model [28, 29] had been put forward. And then, the k-mer methods [30, 31] were developed which had made up for these two disadvantages. The test shows that the k-mer methods were the best ones among all the conventional methods.

The PWM model (Table 1) was proposed as early as 1982 [32]. There were two aspects that need to be specified of a PWM model. The first was the representation of the model. In the earliest PWM model, the weights of the site positions were regarded as the neutrons in a perception [32]; after that, the information contents were used as the measure that analog the frequencies in a PWM model [33]. The energy model emerged to enrich the PWM representations [34]. The other aspect of the PWM model was on how to devise the learning algorithm. EM algorithm (Expectation Maximization algorithm) [35] was a relatively authoritative one in early days. Later, an extended EM algorithm, the MEME (Multiple EM for Motif Elicitation) algorithm [36, 37], was presented to handle the co-regulated sites. Other algorithms using Gibbs sampling [38-42] were also proved to be effective to discover the PWM for a binding site.

Table 1.

Conventional methods for identifying TFBSs.

Training Model Data	EM, Gibbs Simple	Alignment Algorithm	k-mer	HMM	Suffix Tree	Deep Learning
Known binding site	[35-42]	__	__	[125, 126]	[43, 44]	__
ChIP-seq	[55-57]	__	[30, 31]	__	__	[76, 77]
Phylogenetic footprinting	[51, 52]	[53, 54]	__	__	__	__
DNase I & histone-mark	__	__	__	[62-69]	__	[76]
PBM	__	__	__	__	__	[77]

Note: different lines represent methods with different training data; different columns represent methods with different models.

2.3. Various Algorithms and Techniques Applying for TFBSs Prediction

Besides the non-exact PWM model, a class of exact algorithms have been developed to describe the motifs of the binding sites. Both the suffix tree model [43, 44] and projection [45, 46] model are used as data structures to look for high quality motifs. Meanwhile, the GA (Genetic Algorithm) [47, 48] is applied in the PWM model to help it improve performance and Reid et al. [49] have combined the GA algorithm with the suffix tree model to make optimizations and advancements. Based on gradually developed Phylogenetic footprinting [50-54] and ChIP-seq techniques [55-57] lots of related methods for discovering TFBSs have been put forward. Among all the conventional methods, k-mer feature extraction method [30, 31] is acknowledged as the best one in performance. The features in the k-mer method are the frequencies of k-mer subsequences. Both the reduced requirement of training data and the binary feature lag without the threshold values are the superiority of the k-mer to the PWMs methods.

2.4. Integrating Chromatin Features

Now that the recognition effect solely based on the PWM model is often not ideal and it is certified that the bindings of transcription factor have a close relation to histone modification and DNase I hypersensitive site, the methods for identifying TFBSs motif have been enriched by association these chromatin features [58-61]. Many computational methods are proposed which combined with one or more epigenetic data for improving identification performance [62-69].

For instance, Whitington et al.’s study [68] has shown that filtering through histone H3K34Me3 threshold can improve binding sites identification accuracies of many transaction factors. Gusmao et al. [70] show that among many epigenetic data, DNase I sensitivity is the most significant factor to the effect of identification for TFBSs. DNase I splits DNA according to chromatin states. Compared with nucleotides that are not bound, those binding ones would be degraded less frequently. According to this, the possibility of nucleotides being bound can be inferred by the frequency of their degradation.

3. Deep learning methods to predict for TFBS

3.1. The Next-generation Sequencing Technique

Inferring chromatin effects from de novo sequences is a pressing need to decode genetic information to explain traits or disease mechanisms of human beings. High-throughout experimental technologies on chromatin profiling provide huge amount of data, which require algorithms that can leverage and harness such big data. One of the most important next-generation sequencing techniques, Chromatin Immunoprecipitation-sequencing (ChIP-seq) [71-75] is an effective method for detecting DNA segments interaction with either transcription factors or histone modifications in genome-wide. With the development of the next generation sequencing technology represented by ChIP-seq, people can easily and cheaply obtain various information of whole-genome level, which contributes to a substantial proportion of biological research, thus establishing data analysis science. For example, an ordinary ChIP-seq experiment presents tens of thousands sequences which would be used altogether in the following computational analysis. As we all know, deep learning excels at big data processing, and there are several methods to identify protein-binding specificity (or transcription factor motif) based on these ChIP-seq data using deep learning model. Owing to this, there is a variety of the raw experiments data, so various data preparation methods are required to process training data.

3.2. Preprocessing the Sample Dataset

DeepSEA [76] is an outstanding method both to identify TFBSs and to prioritize the noncoding variants based on a deep convolution neural network. In its data preprocessing, DeepSEA [76] separates the whole-genome sequences into tens of millions 200-bp subsequences. For each chromatin feature of each 200-bp subsequences, DeepSEA [76] assigns 1 to the corresponding chromatin feature label if more than half of the 200-bp appear in the peak regions of the corresponding chromatin feature profiling and otherwise 0. For these tens of millions labeled subsequences, DeepSEA screens those which have more than one TF-binding feature positive. Therefore, altogether almost 17 percent of the whole-genome is left for the training, validation, and evaluating data of DeepSEA. It is worth noting that DeepSEA does not deliberately produce negative sample. This is due to the framework of DeepSEA which provides a sharing multitask device that simultaneously predicts multiple chromatin features. The label vector contains many 1 and 0 components, which makes the extra negative samples unnecessary. Nevertheless, another deep learning method DeepBind [77] turns to shuffled negative samples for help. One of the targets of DeepBind is to identify transcription factor binding sites using ChIP-seq data of ENCODE database [78-82]. DeepBind firstly filters and gains 506 TF binding profile datasets after removing some biases. In each profile dataset file, the sequences of the peaks are used as positive samples of DeepBind training, and the dinucleotide shuffles are used as the negative ones.

Thus, by comparing the pre-processing methods of training data using by different deep learning methods, we see that the data preprocessing methods are flexible with a variety even if their raw data are derived from the same ENCODE database. It is notable that DeepBind [77] can harness several different kinds of source data. PBM [83, 84] is another database that is used to verify DeepBind performance. Furthermore, using the specific PBM data, Dream5 [85] project becomes a benchmark platform for measuring the specificity of sequence functional elements and DeepBind [77] and has been proved to be the best method in all the participant teams. PBM gives the intensity of each microarray probe, which represents the specificity of the sequence. That is to say, using the PBM data, the machine learning model needs to predict an intensity value, while a model using ChIP-seq data needs to predict a class label. Because of the characteristic of the sample data, there is no need for the negative samples in the preprocess data of PBM.

3.3. Deep Convolution Neural Network for Identifying the TFBSs

The latest research findings indicate that the technique of deep learning can more efficiently discover the regulatory code of DNA sequence than all previous methods. The deep learning methods used here are those deep convolution neural networks. Compared to those previous motif discovery algorithms, such as PWM model [26, 86-89], dinucleotide model [28, 29, 90-92] and k-mer model [30, 31], the deep convolution neural networks make it possible to extract the long-range dependencies along the sequence.

Convolution-pooling-connection framework is the common character of the convolution neural networks applying in DNA sequence functional motif discovery. Convolution layer is used to identify local sequence code and the deeper is the convolution, the longer-range is the functional code of the sequence, and specifically, the matrixes in the first convolution layer is corresponding to the PWMs of the TFBSs; the rectified function substitutes minus to 0. The variation in the resolution of the sequence mainly relies on the pooling layer which alleviates the local position difference by maximizing operation. Finally, the connection layer refers to full connection which is a general neural setting.

In the different methods, the frameworks of the deep learning have a slight difference. For example, in DeepBind, instead of a single sequence input, a batch of input sequences are processed simultaneously; in Basset [93], the input of a sequence is simultaneously computed in altogether 164 cell-lines backgrounds; in DeepSEA, the multiple predictors are constructed into a framework, which implements the sharing of model sequence code. The comparison of the frameworks of the deep learning methods is shown in (Fig. 1).

Fig. (1).

Comparison framework on deep convolution neural network. It is obvious that the common to all framework is the convolution layer and the max pool layer. The differences are: (a) framework in DeepSEA [76] shares weights between multiple chromatin feature predictors; (b) framework in Basset shares weights between multiple cell type predictors (c) framework in DeepBind shares weights among batch of input sequences; (d) instead of fully connected layer in DeepSEA, framework in DanQ sets Recurrent and Dense layer (e) in framework in DeepMotif, a highway MLP layer takes the place of fully connected layer.

3.4. Performance Measure

In term of the performances of the methods, the authoritative measure is ROC (Receiver Operating Characteristic) curve or the corresponding AUC (Area Under Curve) area. After being trained, for a given input, the deep learning classifier would compute a value and there need a threshold to classify the value into a class. Note that the FP(false positive) and TP(true positive) would change with the threshold changing. The ROC curve is generated by showing the pairs of the true positive rate and the corresponding false positive rate. As for the Dream5 PBM data platform, what is needed to predict is the intensity not the class label, so the threshold value is useless and it is the Pearson correlation not the AUC that is used to measure the performance of the predictor.

3.5. Deep convolution neural network improvement

A modified CNN framework DeepMotif [94] is presented in a recent paper, which makes comparison with the DeepBind in performance. The modification mainly lies in two ways: 1. the highway is substituted for the original conventional full connected-layer; 2. A better visualization motif map is given. There is also another method DanQ [95] that revises the DeepSEA into a hybrid model integrating both the CNN and the RNN. DanQ uses the same training data as DeepSEA and demonstrates that DanQ outperformes DeepSEA.

It should also be noted that the parameters of a CNN would affect the prediction performance. Zeng et al. [96] make experiments with various parameters and draw the conclusion that the improved CNN model excels in the performance by effectively adjusting the parameters, such as the number of the convolution layers, the number of the convolution filters and the length of the convolution filter window et al.

Along with the rapid development of the high-throughout assays, more and more positive sample data would be ready for training. More than this, more and more chromatin features would be conductive to improve CNN model prediction accuracy. What is more, getting more insights into the CNN framework itself has been proved useful for the promotion on model performance [97].

4. Exploring the relationship between the SNVS and the disease

With the advent of the post genomic era, the researches for the genetic mechanism of the diseases have increasingly depended on the studies on the genes, the gene-networks and gene-protein interaction networks [98-100]. Based on the large amount of TFBSs predicting values in the deep learning models, further computation and analysis have been done to reveal the relationship between the gene mutation and the occurrence of the disease, whereas the saturated mutagenesis is a necessary tool by which the influence of a mutation on each single position can be quantitatively measured.

4.1. Saturated Mutagenesis

Mutagenesis employs the physical, chemical or biological factors to induce mutation of the organism to occur, and breeds the new cultivars from the variant offspring based on

breeding objective [101]. Mutagenesis technology can obtain excellent mutant in a short time, which has advantages over hybridization. Quantitative analysis of mutagenesis has been used in large amounts in bioinformatics. The impact of a gene locus on a phenotype is called “effect size”[102], which can be used to measure the effect caused by a single gene mutation on the phenotype. However, it is discovered that the effect size, the extent to which the allelic variation gives rise to the phenotypic change, is often low. Apart from deriving the gene mechanism of the disease phenotype [103, 104], mutagenesis can be applied for quantifying the specificity of each nucleotide in a sequence of DNA [105]. Saturated mutagenesis [106] is a procedure in which all possible nucleotides on a specific position are to be substituted in a given sequence, which makes possible for quantifying the specificity of a certain sequence on a single-nucleotide resolution.

4.2. Prediction Causal SNVs

The destination of the research on chromatin effects is to find the mechanism by which the traits or the diseases are regulated [107]. In order to achieve that, accurate prediction for such chromatin effects is the first step. The focus study of the relationship between the SNP and the diseases is helpful to accomplish the goal. Both DeepSEA and DeepBind have made a lot of work in the establishment of the diseases dependency on the SNPs.

Using the results from the staturated mutagenesis, both DeepSEA [76] and DeepBind [77] demonstrate they have effectively exploited the effects of the SNVs on certain disorders, which have been verified in the previous experiments [108-119].

DeepBind have especially devised a visualized method---mutant map to intuitively display the SNPs’ influence on each position. For example, a mutant map of the staturated mutagenesis in [77] visually shows that the mutant C->D would decrease the transcription factor SP1 binding in the promoter of LDL-R, while the mutant G->C would increase the binding, while the C->G and C->T mutants have been recorded in Human Gene Mutation Database(HGMD) [77, 120].

4.3. Functional SNVs Prioritization

Besides individual cases, both DeepSEA and DeepBind have developed the model to predict the quantitative effect of SNVs caused by mutations of single nucleotide. It is believed that the genetic diseases are the result of DNA mutations, and the SNP is the dominant form of DNA sequence mutation in human genome. For example, the SNP may cause cancers, infectious diseases, autoimmune disorders, neurological illnesses, sickle-cell anaemias, thalassaemias and cystic fibrosis. Studies show that those disease-related SNPs usually locate in noncoding region and the mechanism of the SNP still needs further research. In the recent years, a great progress has been made on the research in the area. Researchers find that the majority of the trait-related DNA mutations change not the gene itself but the regular factors which control the gene expressions. However, surprisingly little is known for us exactly how most of the functional mutations modify the expression of the gene in detail. With the development of DNA sequencing and the decline of the cost of sequencing, researches on the SNP are rapidly expanding, which has done the full preparation for big data analysis. DeepSEA [76] is the first algorithm which large-scaled predicts the functional prioritization of SNPs from de novo sequences. Based on the chromatin feature values predicted by DeepSEA [76] deep learning model, altogether 1842 features are computed, which are generated by reference allele’s 919 features and its corresponding alternative allele’s 919 ones. The label of the sample is binary indicatoe of whether it is a real SNP. The machine learning model is Boosted logistic regression classifier [121]. The model is verified on three different databases. The negative sample is selected from sequences distant from the positive ones. As mentioned above, a classification model needs both positive samples and negative samples, hence it is inevitable to design proper negative samples in order to get better prediction performance and the authors of DeepSEA has testified the different performance with different selection of negative samples.

DeepBind has developed a special modular DeepFind for predicting whether a SNV is causal. In the respect of the classifier, DeepFind adopts a neural network model which takes 1192 features as input which are generated from a wild-type sequence and its corresponding mutant one. Since it is a binary classifier, the positive samples and the negative ones are indispensable. DeepFind mainly carries out the experiment on CADD [122] simulator, using the ‘simulated’ SUVs as positive samples and the ‘observed’ ones as negative.

It is because of the classifier provided by DeepSEA [76] or DeepBind [77] that makes it possible to discriminate whether a noncoding variation (or a SNP) sequence is a functional one (or a disease-related one), which provides the basis for further research on the genetic derivation of the diseases (Table 2).

Table 2.

Lists of methods for SNP functional priorization.

Method	Features Used	Classifier
FunSeq2 [24]	In functional annotations In sensitive regions In ultra-sensitive regions Motif-breaking score (PWM changes) Motif-gaining score (PWM changes) Network centrality score GERP score In ultra-conserved elements In HOT regions In regulatory elements associated with genes Recurrent in multiple samples	Weighted scoring scheme
CADD [123]	conservation metrics; regulatory information; transcript information; protein-level scores (altogether 63 distinct annotations)	SVM model
GWAVA [124]	Open chromatin. Transcription factor binding. Histone modifications. RNA polymerase binding. CpG islands. Genome segmentation. Conservation. Human variation. Genic context. Sequence context.	Modified random forest algorithm
DeepSEA [76]	Evolutionary conservation scores Absolute difference features (919 features) Relative difference features (919 features)	Boosted logistic regression classifier
DeepFind [77]	~600 DeepBind TF predictor values for the wild type and mutant sequences (altogether ~1,200 features)	Neural network

CONFLICT OF INTEREST

This work is supported by the Fundamental Research Funds for the Central Universities of China (Grant No. N120305005 and No. N130305006) and by the National Science Foundation of China (Grant No.31301086).