GimmeMotifs: a de novo motif prediction pipeline for ChIP-sequencing experiments

Abstract

Summary: Accurate prediction of transcription factor binding motifs that are enriched in a collection of sequences remains a computational challenge. Here we report on GimmeMotifs, a pipeline that incorporates an ensemble of computational tools to predict motifs de novo from ChIP-sequencing (ChIP-seq) data. Similar redundant motifs are compared using the weighted information content (WIC) similarity score and clustered using an iterative procedure. A comprehensive output report is generated with several different evaluation metrics to compare and evaluate the results. Benchmarks show that the method performs well on human and mouse ChIP-seq datasets. GimmeMotifs consists of a suite of command-line scripts that can be easily implemented in a ChIP-seq analysis pipeline.

Availability: GimmeMotifs is implemented in Python and runs on Linux. The source code is freely available for download at http://www.ncmls.eu/bioinfo/gimmemotifs/.

Contact: s.vanheeringen@ncmls.ru.nl

Supplementary Information: Supplementary data are available at Bioinformatics online.

1 INTRODUCTION

The spectacular development of sequencing technology has enabled rapid, cost-efficient profiling of DNA binding proteins. Chromatin immunoprecipitation followed by high-throughput deep sequencing (ChIP-seq) delivers high-resolution binding profiles of transcription factors (TFs) (Park, 2009). The elucidation of the binding characteristics of these TFs is one of the obvious follow-up questions. However, the de novo identification of DNA sequence motifs remains a challenging computational task. Although many methods have been developed with varying degrees of success, no single method consistently performs well on real biological eukaryotic data (Tompa et al., 2005). The combination of different algorithmic approaches, each with its own strengths and weaknesses, has been shown to improve prediction accuracy and sensitivity over single methods (Hu et al., 2005).

Here, we report on GimmeMotifs, a motif prediction pipeline using a ensemble of existing computational tools (Supplementary Fig. S1). This pipeline has been specifically developed to predict TF motifs from ChIP-seq data. It uses the wealth of sequences (binding peaks) usually resulting from ChIP-seq experiments to both predict motifs de novo, as well as validate these motifs in an independent fraction of the dataset.

GimmeMotifs incorporates the weighted information content (WIC) similarity metric in an iterative clustering procedure to cluster similar motifs and reduce the redundancy which is the result of combining the output of different tools (see Supplementary Material). It produces an extensive graphical report with several evaluation metrics to enable interpretion of the results (Fig. 1).

Fig. 1.

An example of the GimmeMotifs output for p63 (Kouwenhoven et al., 2010). Shown are the sequence logo of the predicted motif (Schneider and Stephens, 1990), the best matching motif in the JASPAR database (Sandelin et al., 2004), the ROC curve, the positional preference plot and several statistics to evaluate the motif performance. See the Supplementary Material for a complete example.

2 METHODS

2.1 Overview

The input for GimmeMotifs is a file in BED format containing genomic coordinates, e.g. peaks from a ChIP-seq experiment or a FASTA file. This dataset is split: a prediction set contains randomly selected sequences from the input dataset (20% of the sequences by default) and is used for motif prediction with several different computational tools. Predicted motifs are filtered for significance using all remaining sequences (the validation set), clustered using the WIC score as described below, and a list of non-redundant motifs is generated.

2.2 Motif similarity and clustering

The WIC similarity score is based on the information content (IC) and is defined for position i in motif X compared with position j of motif Y as:

(1)

where c is 2.5, and DIC(X_i, Y_j) is the differential IC defined in Equation (3). The IC of a specific motif position is defined as:

(2)

where IC(X_i) is the IC of position i of motif X, f^x_i,n is the frequency of nucleotide n at position i and f_bg is the background frequency (0.25). The differential IC (DIC) of position i in motif X and position j in motif Y is defined as:

(3)

The WIC score of all individual positions in the alignment is summed to determine the total WIC score of two aligned motifs. To calculate the maximum WIC score of two motifs, all possible scores of all alignments are calculated, and the maximum scoring alignment is kept. Similar motifs are clustered using an iterative pair-wise clustering procedure (Supplementary Material).

2.3 Evaluation

The motifs can be evaluated using several different statistics: the absolute enrichment, the hypergeometric P-value, a receiver operator characteristic (ROC) graph, the ROC area under the curve (AUC) and the mean normalized conditional probability (MNCP) (Clarke and Granek, 2003). In addition to these evaluation metrics, GimmeMotifs generates a histogram of the motif position relative to the peak summit, the positional preference plot. Especially in case of high-resolution ChIP-seq data, this gives valuable information on the motif location.

2.4 Implementation

The GimmeMotifs package is implemented in Python, while the similarity metrics are written as a C extension module for performance reasons. It is freely available under the MIT license. Sequence logos are generated using WebLogo (Schneider and Stephens, 1990).

3 BENCHMARK RESULTS

We performed a benchmark study of GimmeMotifs on 18 TF ChIP-seq datasets. The ROC AUC and MNCP of the best performing motif were calculated and compared with the best motif of two other ensemble methods: SCOPE (Carlson et al., 2007) and W-ChipMotifs (Jin et al., 2009) (Supplementary Tables S1 and S2) . The results show that GimmeMotifs consistently produces accurate results (median ROC AUC 0.830). The method also significantly improves on the results of SCOPE (ROC AUC 0.613). The recently developed W-ChIPmotifs shows comparable results to GimmeMotifs (ROC AUC 0.824), although this tool does not cluster similar redundant motifs. In addition, the focus of GimmeMotifs is different. While the web interface of W-ChipMotifs is very useful for casual use, the command-line tools of GimmeMotifs can be integrated in more sophisticated analysis pipelines.

4 CONCLUSION

We present GimmeMotifs, a de novo motif prediction pipeline ideally suited to predict transcription factor binding motifs from ChIP-seq datasets. GimmeMotifs clusters the results of several different tools and produces a comprehensive report to evaluate the predicted motifs. We show that GimmeMotifs performs well on biologically relevant datasets of different TFs and compares favorably to other methods.