Generic eukaryotic core promoter prediction using structural features of DNA

Abstract

Despite many recent efforts, in silico identification of promoter regions is still in its infancy. However, the accurate identification and delineation of promoter regions is important for several reasons, such as improving genome annotation and devising experiments to study and understand transcriptional regulation. Current methods to identify the core region of promoters require large amounts of high-quality training data and often behave like black box models that output predictions that are difficult to interpret. Here, we present a novel approach for predicting promoters in whole-genome sequences by using large-scale structural properties of DNA. Our technique requires no training, is applicable to many eukaryotic genomes, and performs extremely well in comparison with the best available promoter prediction programs. Moreover, it is fast, simple in design, and has no size constraints, and the results are easily interpretable. We compared our approach with 14 current state-of-the-art implementations using human gene and transcription start site data and analyzed the ENCODE region in more detail. We also validated our method on 12 additional eukaryotic genomes, including vertebrates, invertebrates, plants, fungi, and protists.

Eukaryotic genomes are being sequenced at an ever-increasing pace. At the moment, nearly 50 complete genomes of eukaryotes are publicly available, and many more are in the pipeline to be sequenced in the next few years (Liolios et al. 2006). The proliferation of genome sequencing projects has driven the search for fast ways of sequence-based structural annotation, which involves the identification of genes and the modeling of their correct gene structure (Claverie et al. 1997; Mathé et al. 2002; Zhang 2002; Wang et al. 2004). Although great progress has been achieved in gene prediction, for instance by using comparative approaches (Wasserman et al. 2000; Liu et al. 2004; Jin et al. 2006; Wang and Zhang 2006), one of the more difficult tasks in the annotation of whole genomes remains the accurate identification and delineation of promoters (Fickett and Hatzigeorgiou 1997; Ohler 2000, 2001; Bajic et al. 2004, 2006a). Nevertheless, the prediction of the regions that control the transcriptional activation of genes is important for various reasons (Smale 2001; Butler and Kadonaga 2002; Bajic et al. 2004; Sonnenburg et al. 2006). On the one hand, promoter prediction can be used for the discovery of genes that are missed by gene predictors and/or for which experimental support (ESTs, cDNAs, etc.) is not available. On the other hand, the prediction of promoters is important for guiding further in silico searches and experimental work, for instance in narrowing down the regions that play the most important role in transcriptional regulation (Bajic et al. 2004, 2006a; Carninci et al. 2006; Solovyev et al. 2006).

The promoter is commonly referred to as the region upstream of a gene that contains the information permitting the proper activation or repression of the gene that it controls (Pedersen et al. 1999; Smale and Kadonaga 2003). The promoter region itself is typically divided into three parts: (1) the core promoter, which is the region that is responsible for the actual binding of the transcription apparatus and which is typically situated ∼35 bp upstream of the transcription start site (TSS); (2) the proximal promoter, a region containing several regulatory elements, which ranges up to a few hundred base pairs upstream of the TSS; and (3) the distal promoter, which can range several thousands of base pairs upstream of the TSS and contains additional regulatory elements called enhancers and silencers.

It has been known for quite some time that the properties of promoter regions are considerably different from those of other parts in the genome (Pedersen et al. 1998; Aerts et al. 2004; Florquin et al. 2005; Fukue et al. 2005; Tabach et al. 2007). Some features that have proven useful in the detection of promoters in vertebrate genomes are the so-called CpG islands close to the TSS (Delgado et al. 1998; Ioshikhes and Zhang 2000; Hannenhalli and Levy 2001), the presence of typical transcription factor binding sites (Solovyev and Shahmuradov 2003; Choi et al. 2004; Ohler 2006), and statistical properties of the core and proximal promoter (Down and Hubbard 2002; Bajic et al. 2006b; Fitzgerald et al. 2006). The similarities between orthologous promoters (Solovyev and Shahmuradov 2003; Jin et al. 2006) and information from mRNA transcripts (Liu and States 2002) have also been used to identify promoters. The more recent and sophisticated Promoter Prediction Programs (PPPs) look for these promoter-specific characteristics by using machine learning techniques such as discriminant analyses, Hidden Markov Models, and Artificial Neural Networks to predict and delineate promoters (for reviews, see Fickett and Hatzigeorgiou 1997; Rombauts et al. 2003; Bajic et al. 2004, 2006a; Sonnenburg et al. 2006). Programs and tools based on these techniques are difficult to train because they require a large amount of high-quality training data, preferably from an experimental setting (Munch and Krogh 2006). However, for most of the new genome projects there is only a limited amount of such data available. Another problem is that the outcome of programs based on these techniques is often difficult to interpret (Ratsch et al. 2006). Furthermore, all hitherto available programs are species-specific; i.e., they are trained on one species and are able to predict promoters only for that particular species. Another drawback of most PPPs is that they depend on specific motifs expected to be present in the core promoter. Indeed, some programs (Promoter2.0 [Knudsen 1999]; Eponine [Down and Hubbard 2002]; NNPP2.2 [Burden et al. 2004]) are based on the explicit presence of motifs such as the TATA box, which are very common in certain species, such as yeast (Struhl 1989), but much less common in mammals or plants (Suzuki et al. 2001; Butler and Kadonaga 2002; Fukue et al. 2004; Florquin et al. 2005). This hampers the ability of a single program to analyze different species with the same model, and does not facilitate the discovery of different types of promoters. Finally, there is the issue of scalability, as the best performing programs are unable to process large datasets (Ohler et al. 2002; Bajic et al. 2004, 2006a; Sonnenburg et al. 2006).

In light of all these caveats, we propose a simple technique for identifying and delineating (core) promoters that is based on the properties of long stretches of DNA. It has indeed been shown that sequence properties such as GC content and more general chemo-physical properties of the DNA, such as stabilizing energy of Z-DNA (Ho et al. 1990), DNA denaturation values (Blake and Delcourt 1998; Blake et al. 1999), protein-induced deformability (Olson et al. 1998), and duplex-free energy (Sugimoto et al. 1996), among others (for review, see Florquin et al. 2005), can be used to describe (core) promoters, and to discriminate between (core) promoter sequences and non (core) promoter sequences (Ohler et al. 2001; Florquin et al. 2005; Kanhere and Bansal 2005; Uren et al. 2006; Wang and Benham 2006). Because all these properties are calculated from conversion tables using di- or trinucleotides, one may argue that these properties are in fact exactly the same as the nucleotide sequence and do not offer any additional information. However, several studies have shown that this is not the case. Both Liao et al. (2000) and Baldi et al. (1998) have analyzed the correlation between the different properties, and their main conclusion was that the properties are largely independent. Moreover, Florquin et al. (2005) have clustered promoters based on these structural properties. The genes associated with the promoters in each cluster varied greatly for the different properties, which again indicates that the different properties contain complementary information. Bode et al. (2006) have shown that it is very hard to identify scaffold/matrix attachment regions (SMARs) from the sequence, as the important scaffold proteins recognize structural features instead of specific nucleotide sequences. Structural properties are known to have long-range interactions (up to 10 kb), so they can exhibit properties that are not visible in the sequence (Merling et al. 2003; Faiger et al. 2006). The Human Genomic Melting Map (Liu et al. 2007) shows a correlation between GC content and DNA denaturation, but due to the cooperative nature of DNA denaturation, this correlation is weaker on scales <500 bp.

We applied these different properties to the problem of promoter prediction and show that some properties give better performance than others, which again indicates differences in information content. In particular, we present the Easy Promoter Prediction Program (EP3), which uses GC content and large-scale structural features of DNA to identify and delineate promoter regions in whole-genome sequences. EP3 was applied to the human genome and compared with other state-of-the-art PPPs using two different evaluation techniques: One is commonly used for PPPs, while the other is based on a novel scheme also considering intergenic predictions that do not fall in a promoter. We evaluated the different ways of encoding the genomic DNA, and we also applied EP3 to a set of human noncoding RNA genes. Finally, we evaluated EP3 on 12 other eukaryotic genomes.

Results and Discussion

Properties of the core promoter region

Figure 1 shows examples of numerical profiles for human promoter sequences. These profiles were obtained by lining up all the promoter sequences with the TSS at the same position. Next, we converted all sequences using the different structural properties of DNA into numeric sequences and, for each position, plotted the average over all numeric sequences. The X-axis is the position relative to the TSS, and the Y-axis is the normalized value of that property. It is clear that the human core promoter adopts a very specific intrinsic structure that stretches over quite long distances. Either a large peak is visible, representing highly stable regions in the DNA, or a cleft is visible, which represents highly unstable regions in the DNA. For instance, the GC profile shows a peak, while the AT profile shows a cleft. When zoomed in, it can be noticed that, within the broader region of stability, there is a small unstable region that corresponds to the TSS, and a second one that corresponds to the region where TBP binds (see below). The promoter prediction technique we present will identify those higher regions and are, thus, most likely to contain a TSS.

Figure 1.

Examples of numerical profiles representing structural properties of the DNA in human over long-range distances around the transcription start site (TSS) (2000 bp upstream and 2000 bp downstream). The numeric profiles contain the actual average values on each position and are not smoothed using a window. Structural properties can be divided into two broad categories: those for which the numerical profile shows a peak around the TSS (e.g., GC content, bendability, and DNA denaturation), and those for which the profile shows a cleft around the TSS (e.g., AT content, bDNA twist, and Duplex Stability-Free Energy). (X-axis) Position relative to the TSS; (Y-axis) normalized value for each of the different properties. The profiles are based on the properties listed in the first column of Table 2. The coordinates for the TSS were retrieved from Ensembl, and the sequences were extracted around this TSS.

To see how general these properties of promoter sequences are, we created large-scale (2000 bp up- and downstream of the TSS) datasets for 16 different species. For each of these datasets, we analyzed the structural profiles. Over long-range distances, we observe several types of profiles (Fig. 2). The first one applies mostly to protists. In this type, the profile slowly decreases a few hundred base pairs upstream of the TSS, shows a transition of a strong peak and cleft on the TSS, and is flat again downstream of the TSS, which may indicate a relatively small promoter area in these organisms. The second type of profile is observed in mammals and shows a broad elevated region around the TSS, with again a strong local peak and drop on the TSS itself. Between these two extremes, we observe a more gradual transition in fungi and plants. The extremely high and low values that can be seen on the TSS are discussed in more detail in the next section.

Figure 2.

Examples of numerical profiles representing different species using the same property (inverted base stacking value) over long-range distances around the transcription start site (TSS) (2000 bp upstream and 2000 bp downstream). (X-axis) Position relative to the TSS; (Y-axis) normalized value of the property. The profile is calculated with a window size of three.

There seems to be a relationship between the genome size and the size of the peaks or clefts: In prokaryotes (not shown) and protists, these cover ∼100 bp, while they gradually increase to several hundreds of base pairs in fungi and up to ∼1000 bp in plants. In animals, smaller genomes such as that of Drosophila have a peak ranging from a few hundred base pairs on each side of the TSS; this gradually increases up to ∼2000 bp in mammals. In conclusion, the peak is most obvious in mammals, but is also clearly visible in fish, insects, fungi, and plants; it seems considerably smaller in protists.

In humans, as well as in other eukaryotes, three types of RNA polymerases (RNAP) exist that are used to transcribe different types of genes. All three of them manage to identify the TSS in a whole-genome setting in their own way. Here, we focus on humans because this species is among the best documented ones. We retrieved the promoter sequences for several types of genes from Ensembl (see Methods) and constructed three different datasets for each of the promoter types: RNAP I, RNAP II, and RNAP III. For each of these datasets, we calculated a structural profile. Figure 3 shows the inverted base-stacking values over long-range distances around the TSS (2000 bp up- and downstream), as well as a zoomed-in view (top row) of a shorter region around the TSS (ranging from 200 bp upstream of the TSS to 50 bp downstream of the TSS) for sets of promoters recognized by the three different polymerases. Both on a large and a small scale, there are clear and important fluctuations in the profile around the TSS. One of the most striking features, at least for RNAP II promoters, is the presence of two clefts: one at the TSS and one near position −30. The latter one is related to the TATA binding protein that is important for transcription initiation.

Figure 3.

Structural profile of promoters for genes transcribed by different polymerases. (Top row) Numerical profiles over short-range distances around the TSS; (bottom row) profiles over long-range distances. The profile was calculated using the inverse of the base-stacking property. See text for details.

There is, however, a clear difference between RNAP II promoters on the one hand, and RNAP I and III promoters on the other hand. RNAP I and III promoters lack the large-scale stable region of RNAP II promoters and show local destabilization only around the TSS. These local instabilities around the TSS may be caused partly by the structural nature of the transcript originating from these genes. Drawing more definitive conclusions for the RNAP I and III promoters would require much more data than the few hundred sequences that are currently available in Ensembl. The lack of large-scale features is interesting as it may indicate that the region involved in regulation of these gene types is in general quite small. This might be in line with the fact that many genes transcribed by RNAP I and III promoters are produced in very large quantities and need less regulation. For instance, rRNA genes account for 80% of total steady-state cellular RNA and might thus require less sophisticated regulation (Jacob 1995). Another observation is that the regulation of r-protein genes is often post-transcriptional and therefore does not need extensive transcriptional regulation (Amaldi et al. 1989; Presutti et al. 1991). RNAP II promoters have an elevated region of nearly 2000 bp, which indicates a much larger area involved in transcriptional regulation of protein-coding genes.

Relationship between structural profiles and known core promoter elements

To investigate the relationship between the structural profile and the occurrence of known core promoter elements, we analyzed four elements (TATA, INR, BRE, and CpG islands) well known in animal genomes and the relation between the presence or absence of the element in the promoter sequence and the presence or absence of peaks and clefts in the structural profile. Again, we limited ourselves to humans because to detect motifs we require position weight matrices (PWM) or consensus sequences, which are not available for all organisms.

The first element we considered is the TATA box, also known as the Hogness box, which binds the TATA binding protein (TBP) and is often involved in transcription initiation (Smale and Kadonaga 2003). The structural profile for the promoters containing the TATA motif is shown in Figure 4 (top row). As can be noticed, the numerical profile of promoters that do contain the TATA box lies lower than those that do not. Furthermore, the cleft around position −30 is much deeper in TATA-box-containing promoters, implying that the DNA denatures more easily on that position, which conforms to the knowledge that the TBP is involved in the denaturation of DNA. Although it is clear that promoters containing the TATA motif have a much deeper cleft than those without, we must note that the cleft is visible as well. This indicates that the TBP, which is important for transcription to start, requires a region that denatures more easily, whether it contains the motif or not, to allow efficient binding (Comai et al. 1992; Cormack and Struhl 1992; White and Jackson 1992). Therefore, we can conclude that the TATA motif is responsible for the instability of this region, but other combinations of nucleotides also can result in a region that is unstable enough. Also, on a larger scale (Fig. 4, right side), we see that the profile is slightly lower for promoters containing TATA than for non-TATA promoters.

Figure 4.

Structural profiles of RNAP II promoters containing known motifs or elements versus promoters for which the presence of motifs cannot be demonstrated. (Left two columns) Short-range distances around the TSS; (right two columns) long-range distances around the TSS. The profile was calculated using the inverse of the base-stacking property. (Inr) Initiator element, (BR) TFIIB recognition element.

A second well-known core promoter element is the Initiator element (INR) that is found mostly on position 0; i.e., the TSS itself. In Figure 4 (second row), the structural profiles for promoters with and without the INR motif are displayed. Overall, the two profiles are very similar, except for one remarkable difference visible on the small-scale plot: On position −1 there is a significant peak (up to −0.05) upstream of the TSS cleft in the INR-containing promoters, while this peak is missing in the promoters that lack INR. This seems to indicate that this peak, which increases the stability–instability transition, plays a role in the accurate transcription initiation because the INR motif is known to be important for accurate transcription initiation (Lo and Smale 1996). On a larger scale, we do not observe a significant difference between INR-containing promoters and those that do not.

The TFIIB recognition element (BRE) consists of two parts, BRE^u, which is located upstream of the position of the TATA box (Lagrange et al. 1998), and BRE^d, which is located downstream of the TATA motif (Deng and Roberts 2005). The profile of BRE promoters is plotted in the third row of Figure 4. It is clear that the profile of promoters that contain a BRE motif is higher than that of promoters lacking the BRE motif. This is not surprising because the BRE motif consists entirely of G and C nucleotides, which results in higher inverted base-stacking values. The local profile for BRE-containing promoters shows a rather stable area from position −40 to −20. These features fit well with the observation that TFIIB (GTF2B) interacts with the major groove upstream of the TATA box and with the minor groove downstream of the TATA box (Nikolov et al. 1995; Lagrange et al. 1998; Deng and Roberts 2005, 2006). Both interactions leave their tracks on the structural profile. The interaction with the major groove is visible as a peak at position −35, while the interaction with the minor groove is visible as a peak at position −25. These two peaks form a cleft at position −30 that will be used by TBP to bind. The BRE motif is more prevalent among promoters without the TATA motif, and the motif might be a functional substitute for the TATA box (Deng and Roberts 2006). On a larger scale, we see that the profile of promoters lacking the BRE motif has a much lower amplitude than that of promoters containing the BRE motif.

A last well-known characteristic of core promoters is the abundance of CpG islands near protein-coding genes in mammals (Bird 2002). CpG islands are regions that are mainly associated with mammalian promoters, but are also observed in plants (Rombauts et al. 2003). They are related to housekeeping genes (Baek et al. 2007), dynamic usage of TSSs (Kawaji et al. 2006), and bidirectional promoter activity (Trinklein et al. 2004). The CpG dinucleotide is underrepresented in the genome because the cytosine can be converted into thymine after methylation. Only DNA stretches that are under evolutionary stress, such as promoters, are preserved, and these regions are rich in CpG dinucleotides that are unmethylated. The unmethylated CpG-rich regions are often associated with promoters of protein-coding genes (Bird 2002). These islands are typically found upstream of the core promoter, and, when present, these promoters usually lack a TATA box and often have alternative start sites. The local profile does not vary much between promoters having CpG islands and the others, with only an expected shift toward higher stability values due to GC richness. On a larger scale, as with BRE elements, the profile of promoters lacking CpG islands has a much lower amplitude than that of promoters with CpG islands. This may indicate that the existence of either feature is correlated with long-range regulation, which would require further investigation.

Promoter prediction and comparison to the state of the art

We have shown that in eukaryotes the region near the TSS has a very distinct structural profile, while this is absent in the remainder of intergenic sequences and in coding sequences (Fig. 1). The peak in this profile, for instance using base stacking values, is a well-positioned property of the eukaryotic promoter that stretches over several hundreds of base pairs with its top located on the TSS, which can be used to identify promoters. As computation of this profile for individual sequences involves smoothing over a few hundred base pairs, this feature is too coarse to predict the actual TSS; it is nevertheless very well suited to predict the core promoter region. Furthermore, due to the fact that it is such a large-scale property, there are very few false positives when scanning the genome. Using large-scale structural features is new to the field of promoter prediction, because previous studies usually analyzed no more than 200 bp around the TSS (Ohler and Niemann 2001; Fukue et al. 2004, 2005; Florquin et al. 2005; Kanhere and Bansal 2005).

In applying EP3 to the human genome, we observed clear peaks near the start of most genes, which is exemplified in Figure 5 for a small region of the human genome. This figure shows a plot of the structural profile for 2 Mbp on the human chromosome 21 using DNA base-stacking values and a window size of 400 bp (see Methods). To demonstrate the predictive strength of these structural profiles, we have added the gene annotation retrieved from Ensembl to the predictions. This figure clearly shows that most genes do have a peak near the 5′ end, and that there are virtually no peaks outside this region (false positives), indicating that the large-scale structural properties we observe are distinct enough for EP3 to efficiently predict core promoter regions. It should be noted that, while this single property proves to work surprisingly well, future research may focus on combining multiple properties to describe the promoter region even better, as the overlap between the predictions obtained through different properties may vary significantly (Supplemental Table 1). For example, the overlap between Bendability and Duplex Stability-Free energy is only 1%, while the overlap between predictions from Duplex Stability-Free energy and DNA denaturation is ≥80%. The average overlap between a pair of properties is 44% (SD = 29%). Improving the models for the different structural properties may also increase the overall performance.

Figure 5.

Structural profile (blue) of human chromosome 21 between position 32,000,000 bp and 33,000,000 bp. The profile, based on inverted base-stacking values, was made using a window size of 400 bp and using nonoverlapping windows. Experimentally annotated genes from Ensembl are indicated: (yellow bands) positive strand; (blue bands) negative strand. Peaks in the profile that exceed the classification threshold (T = 0.19, horizontal gray line) are predicted as promoter regions. (X-axis) Position in the genomic sequence, (Y-axis) inverse of the value for base-stacking energy. The labels on the plot show the gene names as they appear in Ensembl.

Next, we compared our program with a broad range of programs that are currently available for human promoter predictions (Table 1). We used both the Ensembl gene annotation and the CAGE transcription start data and tested GC content as the simplest feature and compared it with the other structural features. All programs were run using default settings unless indicated otherwise. For those programs that depend on a parameter, we included the best result in the table. For EP3, we included the values for the base-stacking property, as these gave the best results (Table 2). To rank the different programs, we used the F-measure of the program with a maximum allowed distance of 500 bp with the true TSS compared with the CAGE dataset. This is in contrast to other recent reviews and promoter prediction papers that used less strict validation (Bajic et al. 2004, 2006a; Sonnenburg et al. 2006; Xie et al. 2006). We included the less strict mismatch distances (1000 and 2000 bp) for reference with the other papers that allowed a distance of 1000 bp (Bajic et al. 2006a) or 2000 bp (Bajic et al. 2004; Sonnenburg et al. 2006). When using larger values, more distant predictions will also be considered to be true positive (TP). This is not desirable, as predictions should be as close as possible to the TSS. The F-measure clearly decreases when the evaluation becomes stricter, indicating that all programs output predictions that are quite distant from the actual TSS.

Table 1.

The performance of all current promoter prediction programs capable of analyzing the whole human genome

The F-measure of the CAGE dataset with strictest maximum allowed mismatch was used to rank the programs (in bold). The FProm program (Solovyev et al. 2006) was omitted from the analyses because it is not for free for academic use. Prec., precision.

Table 2.

Performance of the different structural features of DNA

^aThe harmonic mean of the F-measures for both the CAGE and Ensembl data. Overall, the base-stacking encoding performs best. Prec., precision.

For NNPP2.2, McPromoter, and Promoter2.0, we considered different parameter settings. We included only the setting that resulted in the highest F-measure. We considered two different techniques to calculate the number of TPs, false positives (FPs), and false negatives (FNs) (see Methods). The first one is based simply on gene annotation and was used before in several studies on promoter predictions (Bajic et al. 2004, 2006a; Sonnenburg et al. 2006; Xie et al. 2006). The second one is an alternative technique we developed that also includes intergenic predictions but requires transcriptional data, such as the CAGE dataset.

For the Ensembl annotation, we see that Dragon Gene Start Finder (GSF) (F = 0.53) and PromoterInspector (F = 0.49) perform better than EP3 (F = 44). However, when using the CAGE annotation, EP3 performs slightly better (F = 0.45) than DragonGSF (F = 0.44) and PromoterInspector (F = 0.43). Although these differences seem small, it still concerns several hundreds of genes or TSSs. For instance, a 1% lower recall means 200 genes in the Ensembl dataset that are missed by that program. One percent less precision means 200 extra false predictions, so even the small differences are significant. In the case of the CAGE dataset, 1% difference corresponds to a difference of ∼1250 TSSs. Combining this into the F-measure means that a program that has 1% lower value misses 1250 TSSs and predicts an additional 1250 wrong ones.

In conclusion, the best performing programs are EP3, DragonGSF, PromoterInspector, Eponine, and FirstEF, which have an F-measure > 0.4. Several programs (CpGProD, PromoterExplorer, N-Scan, and McPromoter) still perform quite well and have an F-measure > 0.25. The rest of the programs (PromFD, ARTS, DragonPF, PromoterScan, NNPP2.2, and Promoter2.0) do not perform very well on the complete human genome (F < 0.25). In all cases, these low F-measures are caused by a very low precision, which is obtained when the PPP outputs many FPs, a problem reminiscent of promoter prediction since the beginning.

The performance of PPPs on the Ensembl dataset is generally higher than that on the CAGE dataset, probably because of the different counting schemes for Ensembl data and the CAGE dataset, whereas the scheme for Ensembl data ignores all intergenic predictions. Some of the other programs may also perform better on the Ensembl data than EP3, because these programs have been trained on promoters of protein-coding genes and are therefore more gene-centric. For the top-performing programs, the balance between the recall (sensitivity) and precision (specificity) is mostly favoring the precision, with the exception of FirstEF, which is perfectly balanced on the CAGE dataset. The other programs often have high recall values, but at the cost of very low precision values.

Performance on the ENCODE region

The ENCODE project aims to carefully annotate all functional elements in a small portion (1%) of the human genome. We used EP3 to make predictions on the 44 regions that cover ∼30 Mb and compared the prediction with three different datasets. First, we compared our predictions to a set of known functional promoters (Cooper et al. 2006). The dataset is only partial for the ENCODE region because Cooper et al. tested only 642 putative promoters of the 921 they predicted. These partial data are insufficient to assess the precision of EP3. Of the 642 promoters tested, 387 were discovered to be functional. EP3 predicts 24% (recall) of the promoters that are marked as functional by Cooper et al. when using a maximum distance of 500 bp. This low recall rate is rather surprising, as the performance on the GENCODE and CAGE data is very good (see below). But when we look in detail, a much higher recall rate (40%) is obtained for the genes expressed in all 16 cell lines, which indicates that EP3 is biased toward broadly expressed genes. Of the 257 nonfunctional promoters, 18 (7%) are predicted by EP3.

Next, we compared the predictions of EP3 on the ENCODE region with the gene annotation from the GENCODE project (Harrow et al. 2006) and with the CAGE data from Riken. The GENCODE annotation was compared with the predictions using the classic method for calculating the performance, as it is a gene annotation similar to the one of Ensembl. We obtained a recall of 0.46, a precision of 0.72, and an F-measure of 0.56. The performance on the CAGE data was calculated with the novel method presented here and has 0.61 recall, 0.87 precision, and 0.72 for the F-measure. Both performances were calculated using a maximum distance of only 500 bp. As expected from previous analyses of the ENCODE region (Bajic et al. 2006a), we see that the performance of our program is better on this region than on the rest of the genome, which indicates that some of the FP in the genome setting are actually missed genes or missed TSSs.

To test this last claim, we retrieved datasets from the ENCODE project for Affymetrix Transcribed Fragments, Yale Transcriptionally Active Regions (TARs), and novel TARs from the DART system (Rozowsky et al. 2007). We combined these three sets with the CAGE data from Riken (single CAGE tags included). This set is called the Evidence for Transcriptional Activity set (EFTA). When comparing the predictions of EP3, we found that of all predictions made by EP3, 87% have a hit with EFTA within 125 bp, 95% have a hit within 500 bp, and 98% have a hit within 2000 bp within EFTA. When excluding the single CAGE tags, the rates drop to 80%, 92%, and 97%, respectively. These numbers indicate that EP3 has a very strong specificity and that many of the so-called FPs discussed above are in fact associated with transcriptionally active regions.

Furthermore, we compared the predictions of EP3 with two sets of DNase hypersensitivity sites (DHSS) retrieved from the ENCODE project. For the first set (encodeNhgriDnaseHsMpssCd4, seven cell types), 50% of the DHSS are near a prediction of EP3, and for the second set (encodeRegulomeDnaseGM06990Sites, one cell type), 28% of the DHSS are near an EP3 prediction. For both sets, the recall rate is lower than that on the CAGE set, but this was to be expected, as the DNase dataset covers only a limited number of cell types.

Performance on different eukaryotic genomes

In the previous sections, we demonstrated the performance of EP3 on the human genome. We have also tested its performance on a wide range of other eukaryotes, including animals (Mus musculus, Tetraodon nigroviridis, Drosophila melanogaster), fungi (Saccharomyces cerevisiae, Schizosaccharomyces pombe), algae (Ostreococcus tauri, Ostreococcus pacifica), higher plants (Arabidopsis thaliana, Oryza sativa, Populus trichocarpa), and a protist (Plasmodium falciparum). Only data for human and mouse are available from the CAGE technique; therefore, we limited the analyses for the other eukaryotes to the data available from Ensembl. Table 3 shows the result when we used EP3 to predict promoter regions in other eukaryotes. The F-measure ranges from 0.17 to 0.71 on the different species, with P. falciparum (F = 0.17) and D. melanogaster (F = 0.19) on the low end of the scale, and O. pacifica (F = 0.71) and O. tauri (F = 0.66) giving the best results. We evaluated the performance of EP3 only; the other programs are not suitable for all other genomes because they are specifically trained for a single species. EP3 obtains a good score for some species, while for other species the score is worse. The F-score is a bit higher for mouse than for human, which indicates that the program performs well for mammals. Within the green lineage (green algae and land plants), there seem to be two groups, based on the genome size. The performance for the two algae (first group) is excellent (F > 0.65), which is probably partly due to the very small genome size and the still large gene space (∼8000 genes). Due to the window approach to assess TPs and FPs, most predictions will be a TP because of the small genome. The second group comprised of rice, Arabidopsis, and poplar has larger genomes, and the performance of EP3 is comparable to that on mammals. The performance for the two yeasts is lower than that on the two Ostreococcus genomes, which have roughly the same genome size. This is probably due to the less obvious profile in yeasts compared with the one in algae (see Fig. 2). The performance of EP3 on Drosophila and Plasmodium is weak. In the case of the fruit fly, this is likely due to its very different structural profile, as observed in Figure 2, while for Plasmodium the low performance is probably caused by the rather flat profile observed in protists that makes it very difficult for EP3 to distinguish promoter regions from other parts of the genome. Therefore, for some species, for example D. melanogaster, specifically trained programs might perform better (Ohler 2006).

Table 3.

Performance of EP3 on different eukaryotic genomes with a maximum allowed mismatch distance of 500 bp

Note that the datasets that have been retrieved from Ensembl are not all of the same quality, even though the organisms presented were selected for having a good annotation.

^aApproximate genome size in megabases (Mb).

Recognizing different promoter types

Besides genes that code for proteins, there are also genes that are transcribed but for which the RNA is not translated into proteins, so-called noncoding RNAs. These genes produce transcripts that function directly as structural, catalytic, or regulatory RNAs. Recent screens for such genes revealed a surprisingly large number of them, with prominent roles such as guiding the post-transcriptional regulation of protein-coding genes (Eddy 2001; Bartel 2004). Previous studies on promoter prediction focused mainly on a single type of promoter, most often the promoter of protein-coding genes, which are transcribed by RNAP II. Using the Ensembl annotation for humans, we show that our approach is also suited to predict other types of promoters. Although the program works best to identify and delineate promoters of protein-coding genes, it can also be used to detect promoters of snRNA, rRNA, miRNAs, snoRNA, and tRNA genes. Other types of noncoding genes such as scRNA and mitochondrial rRNA were not considered because of the lack of data in the Ensembl database. Table 4 shows the different recall (sensitivity) rates for the different types of genes in this study (snRNA, rRNA, miRNA, snoRNA, and tRNA). The precision cannot be calculated for these analyses because the program will always predict all types of promoters and it will never give predictions specific for a single type of promoter. Therefore, we focus only on how many known noncoding RNA promoters we can identify with our approach. Although EP3 can identify non-protein-coding genes, other programs have higher recall rates. However, the programs with high recall rates are also the ones that performed worse when we applied them to humans. From Table 1, we see that the programs that have high recall and low precision have the lowest F-measure, those also being the programs that have the highest recall for the non-protein-coding genes. Although EP3 is thus also capable of predicting the promoters of noncoding genes, its performance is significantly lower, most probably because the peak in the profile is much smaller (see Fig. 2). Nevertheless, compared with the other top-performing programs on the whole genome (DragonGSF and PromoterInspector), EP3 has very similar recall rates. From this analysis, it is clear that general-purpose PPPs such as those listed in Table 1 are best suited for the prediction of protein-coding gene promoters. For miRNA and tRNA promoters, there are probably better approaches using specifically trained tools for identification of these types of promoters (Lowe and Eddy 1997; Zhou et al. 2007). Finally, EP3 is slightly biased toward GC-rich promoters because the structural feature is most outspoken in these promoters. This bias toward GC-rich promoters is also present in all top-performing PPPs from Table 1 (Scherf et al. 2001; Bajic et al. 2004) and indicates that CpG-island-associated housekeeping genes are favored in the predictions.

Table 4.

Recall (sensitivity) for the different gene types for the different programs

For each program, the sensitivity percentages are shown when using a maximum mismatch of 2000 bp.

Conclusion

The evaluation of PPPs in a whole-genome context is crucial to understand the true performance of the program. Evaluation on a small test set, such as the Eukaryotic Promoter Database (Schmid et al. 2006), does not provide sufficient insight into the real performance of the program when used in actual genome annotation projects. The recall and precision values we found in our analysis are lower than those reported in the original papers, where in most cases the evaluation was done on a (much) smaller dataset. Even the evaluation of a complete chromosome is not sufficient, as there are huge differences in nucleotide content and gene density. If possible, it is advisable to use transcription data, such as the CAGE data, to assess the performance. Transcription data are superior to the gene annotation and its associated way of counting TPs; the gene annotation may not give a complete picture on the performance because it completely ignores intergenic predictions. In short, one should assess a promoter predictor on the whole genome, preferably validating with TSS data.

While EP3 does not outperform its peers by much, the program has several additional advantages compared with other PPPs. EP3 requires no training or parameter tuning, unlike other programs that need extensive amounts of experimentally determined data for the training of their model (Ohler et al. 2000; Scherf et al. 2000; Davuluri et al. 2001; Down and Hubbard 2002; Bajic et al. 2003). When working on a genomic scale, speed and memory requirements also are of importance. EP3 is very fast (for instance, it takes <1 h to annotate the complete human genome), requires little memory, and can thus be run on a home computer; in contrast, some programs require a computer cluster of 80 machines for nearly a week to process the human genome (data not shown). Besides performing very well, especially in light of its simplicity, EP3 can handle many eukaryotic genomes without modifications, as was shown here.

Up to now, most PPPs have used biologically driven small-scale features to build a model for promoter recognition (Fickett and Hatzigeorgiou 1997; Hannenhalli and Levy 2001; Werner 2003). These features are different for each species, and thus the current programs need to be retrained for each species. Here, we have presented a simple approach that employs more global structural features of the DNA sequence in promoter and nonpromoter regions. Our technique does not use any complex machine learning algorithms, for which it is often impossible to infer any new knowledge from the model itself. On the contrary, our method requires no training whatsoever and can be applied to any eukaryotic genome. Numerical profiles representing sequence-dependent properties investigated on a large scale and a very large number of genes show a remarkable feature of promoters. To the best of our knowledge, this is the first time that this feature is described for multiple eukaryotes. The feature associated with promoters is an extended region where the DNA is more stable. This large-scale feature is not present in other types of genomic sequence. While it is most outspoken in vertebrate promoters, it is also present in other eukaryotic promoters, so it seems to be universal and largely independent of the presence or absence of binding sites. As a result, EP3 is capable of performing very well on several eukaryotic species, without the need for any training, which is a unique achievement among promoter prediction software, allowing true “ab initio” promoter prediction. Finally, analysis of the well documented ENCODE region shows that most predictions are indeed associated with transcriptionally active regions.

Methods

Datasets

The CAGE datasets for human have been retrieved from the FANTOM3 project (http://fantom.gsc.riken.go.jp/). The database was compiled by Carninci et al. (2006) and was obtained through the CAGE technique, which identifies all possible TSSs with very high accuracy (Shiraki et al. 2003). This dataset covers the whole human genome and can thus be used for validation purposes. Only tag clusters with at least two mapped tags on the same genomic location were considered to be real TSSs, which should filter out most FPs from the CAGE technique. This filtering resulted in 123,400 unique start sites for human. The whole genome sequences for human (hg17) and mouse (mm5) were retrieved from the UCSC Genome Bioinformatics Site (http://genome.ucsc.edu/) (Kuhn et al. 2007).

We retrieved the following genes in human using the BioMart tool at the Ensembl website, release 37 (http://www.ensembl.org): (1) 20,297 protein-coding, (2) 1382 small nuclear RNA (snRNA), (3) 330 ribosomal RNA (rRNA), (4) 326 microRNA (miRNA), and (5) 642 small nucleolar RNA (snoRNA) (Hubbard et al. 2007). The dataset for transfer RNA (tRNA) was downloaded from the tRNAscan-SE homepage (http://lowelab.ucsc.edu/GtRNAdb/Hsapi/) (Lowe and Eddy 1997). The datasets with annotations for different species (Table 3) were retrieved from two different sources: The protein-coding genes for human, mouse, Tetraodon, fruit fly, and yeast were retrieved using BioMart, while the genomic sequences were retrieved from the Ensembl ftp server (ftp://ftp.ensembl.org/pub/). The sequences and annotation for the five plant species were retrieved from in-house data (http://bioinformatics.psb.ugent.be). The genome and annotation for P. falciparum were retrieved from the Sanger database (http://www.sanger.ac.uk/).

The data for the ENCODE analysis have been downloaded from the UCSC ENCODE repository (http://genome.ucsc.edu/ENCODE/). The data for the functional promoters in the encode region were retrieved from Supplemental Table D (Cooper et al. 2006). The data for the novel transcriptionally active regions were retrieved from the DART database (http://dart.gersteinlab.org/ENCODE/GT/).

Calculating structural profiles

Profiles from DNA sequences are calculated as follows: First, the nucleotide sequence is converted into a sequence of numbers (i.e., a numerical profile). This is done by replacing each dinucleotide (or trinucleotide, depending on the physico-chemical feature used) with its corresponding structural value, which is obtained from experimentally validated conversion tables. Florquin et al. (2005) provide references to the protocols to obtain these conversion tables. These contain values, for example, for stabilizing energy of Z-DNA (Ho et al. 1990), DNA denaturation (Blake and Delcourt 1998; Blake et al. 1999), protein-induced deformability (Olson et al. 1998), and duplex-free energy (Sugimoto et al. 1996). Next, we average over several values, and the number of values we use for computing this average is called the window size, in our case 400 bp. The value 400 was chosen because it gave the best performance in a whole-genome context (see Results). If a value in the profile exceeds a certain threshold, this points to a putative promoter region (see below). For properties that show a cleft instead of a peak (see below), we took the inverse value to make sure we also obtained peaks. This was done to be certain we could use the same approach for all structural features.

Prediction algorithm

The algorithm we use (EP3) to identify promoters has two internal parameters, one for the length of the window and one for the deviation from the average. Both have been determined empirically. The optimal length of the window is 400 bp. We determined this by trying values between 50 and 2000 and seeing which one performed best. Smaller windows result in more predictions, but the number of false predictions increases faster than the rate of true predictions. Larger values result in fewer predictions but higher precision. The F-measure indicates that 400 bp is optimal for all organisms. This value appears to be independent of the size of the genome because the organisms we analyzed ranged in size from 12 Mbp to 3 Gbp, and 400 bp was always the optimal value. Because the overall profile of a genome can vary significantly between different species due to different GC content, the threshold should be determined dynamically for each genome. As a basis, we use the average and standard deviation of the values of a property for the whole genome. For bigger genomes (human and mouse), this deviation should be larger than for other species. Empirically, we determined that the optimal threshold is the average plus one standard deviation for all species, except for human and mouse where this is the average plus three standard deviations. Future research with more available genomes, especially ones that fill in the gap between the small genomes (≤300 Mbp) and the large ones (≥2.5 Gbp) may allow us to define a function that can calculate the optimal multiplier for the standard deviation from the genome size. When the window size (400 bp) and the threshold (depends on genome size) are known, the prediction algorithm is straightforward. The program calculates the profile, and each time a value in the profile is above the threshold for this genome, the location is predicted to be a putative promoter region.

Evaluating predictions

A measure for the performance of a PPP is the harmonic mean of the recall (sensitivity) and the precision (specificity), known as the F-measure (van Rijsbergen 1979). The higher this value, the better the program is able to correctly predict promoters. The recall or sensitivity is the number of predicted promoters (TP) divided by the total number of promoters (TP+FN). The precision or specificity is the number of correct predictions (TP) divided by the total number of predictions (TP+FP). To assess the number of TPs, FPs, and FNs, we need to define the maximum allowed mismatch between the prediction and the true TSS. In previous studies, this value was set to 2000 bp (Bajic et al. 2004; Sonnenburg et al. 2006; Xie et al. 2006), although more recently, for the ENCODE project and for two PPPs, a maximum mismatch of 1000 bp also was evaluated (Bajic et al. 2006a). In Tables 1–3, we display the performance of the programs for three different maximum mismatch values: 2000 bp, 1000 bp, and 500 bp. The smaller the mismatch allowed, the stricter the evaluation. We ranked the performance of the different programs according to the smallest window size (500 bp). In the classic way to evaluate a PPP (Bajic et al. 2004), one will start from gene annotations, e.g., as compiled at Ensembl. A TP is then defined as a TSS that has a prediction within the maximum allowed distance from the true TSS as annotated in the database. A FP is a prediction that lies inside the gene but not within the maximum allowed distance from the TSS. A FN is a true TSS from the database that has no prediction within the maximum allowed mismatch. All predictions that fall within an intergenic region and further from a TSS than the maximum allowed mismatch are ignored. Counting a prediction inside a gene as a FP will be incorrect in some cases, as Carninci et al. (2006) showed that transcription can start in the 5′ untranslated region (UTR) of a gene, as well as in an exon or in the 3′ UTR. Therefore, not taking into account any of the intergenic predictions or only considering the TSSs located at the 5′ UTR gives a biased view of a promoter predictor. Using a gene annotation and ignoring the intergenic predictions will not give a real estimate of the performance of the prediction program for two reasons: (1) predictions inside a gene may in fact be true TSSs; TPs might be underestimated, and consequently FPs are then overestimated; (2) intergenic predictions outside the TSS region are ignored because it is impossible to know whether they are real FPs or whether they point to genes missed by the annotation process. This may in turn lead to underestimation or overestimation of FPs. When using gene annotation with all its uncertainties, it is impossible to know the actual performance of the program in a genomic setting, especially in light of the evidence that much more of the genome is transcribed than just the protein-coding regions (Kapranov et al. 2007; Peters et al. 2007; Ponjavic et al. 2007).

To address some of these caveats, we developed an alternative way to assess the performance of a promoter prediction program that does not exhibit these shortcomings. Recently available whole-genome TSS data allow for a new way to count and evaluate. We used a dataset of TSSs for the whole human genome that were characterized experimentally using high-throughput cap-analysis of gene expression (CAGE) (for review, see Carninci 2006). In contrast to previous studies (Bajic et al. 2004, 2006a; Sonnenburg et al. 2006), this dataset allows counting the number of TPs and FPs in an objective way. The dataset contains locations where transcription starts. A TP is a prediction that is within the maximum allowed mismatch from a true TSS, a FN is a true TSS that has no prediction, and a FP is a prediction that is not near a true TSS from the dataset.

Availability

A user-friendly online implementation of our promoter prediction software EP3 is available at http://bioinformatics.psb.ugent.be/. The program allows the user to process FASTA-formatted sequence files. The online implementation is also available as a standalone application.