Modeling multi-species RNA modification through multi-task curriculum learning

Abstract

N⁶-methyladenosine (m⁶A) is the most pervasive modification in eukaryotic mRNAs. Numerous biological processes are regulated by this critical post-transcriptional mark, such as gene expression, RNA stability, RNA structure and translation. Recently, various experimental techniques and computational methods have been developed to characterize the transcriptome-wide landscapes of m⁶A modification for understanding its underlying mechanisms and functions in mRNA regulation. However, the experimental techniques are generally costly and time-consuming, while the existing computational models are usually designed only for m⁶A site prediction in a single-species and have significant limitations in accuracy, interpretability and generalizability. Here, we propose a highly interpretable computational framework, called MASS, based on a multi-task curriculum learning strategy to capture m⁶A features across multiple species simultaneously. Extensive computational experiments demonstrate the superior performances of MASS when compared to the state-of-the-art prediction methods. Furthermore, the contextual sequence features of m⁶A captured by MASS can be explained by the known critical binding motifs of the related RNA-binding proteins, which also help elucidate the similarity and difference among m⁶A features across species. In addition, based on the predicted m⁶A profiles, we further delineate the relationships between m⁶A and various properties of gene regulation, including gene expression, RNA stability, translation, RNA structure and histone modification. In summary, MASS may serve as a useful tool for characterizing m⁶A modification and studying its regulatory code. The source code of MASS can be downloaded from https://github.com/mlcb-thu/MASS.

INTRODUCTION

RNA modification is an evolutionarily conserved modification closely associated with various biological processes and human diseases (1–5). There exist over 160 types of known RNA modifications to date, including the modifications in mRNAs, tRNAs, rRNAs and noncoding RNAs (6). Among them, N⁶-methyladenosine (m⁶A) is the most abundant mRNA modification and has emerged as a critical epitranscriptomic factor in diverse biological processes, such as RNA splicing, translation and RNA stability maintenance, immune response, DNA damage repair and development (7–10). N⁶-methyladenosine is generated under the catalyzation of a methyltransferase complex (also known as an m⁶A writer) composing of METTL3, METTL14, WTAP, RBM15, KIAA1429 and ZC3H13. To participate in various biological processes, m⁶A is recognized and bound by several RNA-binding proteins (also known as m⁶A readers), including YTHDC1, YTHDC2 and YTHDF3. An m⁶A modification can also be removed by several demethylases (also known as m⁶A erasers), including FTO and ALKBH5. A variety of m⁶A writers, readers and erasers determines the diversity of the functions of this modification. To identify transcriptome-wide m⁶A profiles, several experimental techniques combined with next-generation sequencing have been developed, such as MeRIP-seq, m⁶A-seq and miCLIP-seq, which can detect m⁶A sites at single-nucleotide resolution (11–13). Recently, several new methods based on the Oxford nanopore direct RNA sequencing reads, such as Tombo and Epinano, have also been used to detect m⁶A sites (14–17). Significant progress has been made in characterizing m⁶A functions with the aid of these experimental pipelines. However, these experimental techniques are generally too expensive and time-consuming to be applied to large genomes of eukaryotes. In addition, these high-throughput experiments may miss a certain amount of m⁶A sites due to biases caused by read mapping and the employed probing methods. On the other hand, computational methods provide a relatively cheap and effective tool complementary to current high-throughput experimental profiling techniques. These in silico methods are able to not only predict m⁶A sites but also capture the underlying sequence patterns related to this modification.

Recently, machine learning has been applied in solving many computational biology questions, such as the identification of RNA-binding protein (RBP) targets (18), the prediction of translation stalling (19) and the detection of single-cell DNA methylation states (20). A number of computational methods have also been proposed to characterize the sequence features of m⁶A modification. For example, Chen et al. proposed iRNA-Methyl, a machine learning based method that utilized a support vector machine (SVM) framework to identify m⁶A sites (21). Zhou et al. used a random forest based model to identify mammalian m⁶A sites, which achieved an improved performance using novel features extracted from the contextual sequences of the modification sites (22). Wei et al. proposed an ensemble learning based method that trained several SVMs with three different types of feature descriptors, including physico-chemical information, position-specific information and position-specific k-mer nucleotide propensities (23). Zhang et al. applied a convolutional neural network (CNN) model combined with a recurrent neural network (RNN) to capture both spatial and temporal features of RNA sequences (24), which can also provide useful clues into understanding the predicted m⁶A sites via the features captured by the employed CNN. Although these computational methods have been shown capable of identifying a number of novel m⁶A sites, they are quite limited in accuracy, interpretability and generalizability. Moreover, most of the existing prediction frameworks are only designed for single species, which means they cannot be directly applied to investigate the similarity and difference of m⁶A sites among multiple species. Previous studies have shown that the landscapes of m⁶A are highly conserved between human and mouse (12), and the evolution of m⁶A is processed in parallel with that of consensus RNA sequence motifs in primates (2). Therefore, identifying interpretable features among multiple species may provide useful insights into understanding the biological roles of m⁶A modification and discover potential regulatory factors that are shared among different species. In addition, although it has been observed that most of the detected m⁶A sites contain the consensus motif DRACH (where D = A, G or U, R = A or G, H = A, C, or U) (13,25,26), only a small fraction of the DRACH motifs in the transcriptome can be methylated. Hence, capturing the sequence patterns around the core DRACH motifs associated with m⁶A modification may offer novel insights into its regulatory mechanisms.

To overcome the shortcomings of the existing m⁶A prediction methods, further characterize multiple-species m⁶A sites and decipher the biological functions of m⁶A modification, we propose a multi-task curriculum learning based model, called MASS (M⁶A predictor for multiple SpecieS), to predict m⁶A sites across different species. Multi-task learning (27–29) is a learning strategy that trains a group of related tasks simultaneously. Such a training strategy enables the learning model to generalize better and share information among different tasks and has been successfully used in a variety of learning problems (30–32). Naturally, the identification of m⁶A sites across different species can be defined as a multi-task learning problem. However, the variation of genome sizes across different species generally makes it difficult to train multiple models simultaneously (33,34). To address this problem, we trained all the tasks sequentially following the strategy of curriculum learning which is a type of learning starting from relatively easy tasks and gradually increasing task difficulty (35,36). Such a curriculum learning strategy can be used here to alleviate the influences of the variances in genome sizes. Performance evaluation on data from the literature revealed that our model outperforms the state-of-the-art prediction methods, with at least 5% higher performance in terms of the area under the precision-recall curve (AUPRC) score. In addition, the sequence features captured by our model can be well-mapped to the known binding motifs of m⁶A-associated RBPs, which indicates that our model can help characterize the complex mechanisms and functions of m⁶A. Moreover, we screened for important conserved genes enriched with m⁶A sites across different species, which may provide useful molecular clues concerning m⁶A functions. Our downstream analyses also revealed that MASS can be applied to characterize properties of essential biological factors or processes associated with m⁶A, such as gene expression, RNA stability, RNA structure, translation, and histone modification. To our best knowledge, MASS is the first attempt to extract shared sequence features across multiple species in the prediction of m⁶A sites. Our results demonstrated that MASS can serve as a reliable tool to characterize m⁶A sites and offer novel insights into the underlying regulatory mechanisms of m⁶A.

MATERIALS AND METHODS

Datasets

Three benchmark datasets were used in our computational tests. The first one, denoted as sramp17, was a mammalian dataset downloaded from the Ensembl database (http://www.ensembl.org) according to the supplementary data of SRAMP (22). Human and mouse data from this dataset were used to evaluate our model on single species. The second dataset, denoted as nano20, was adopted from the nanopore direct RNA sequencing data collected in (14). Here, the original nanopore sequencing signals were 1D numeric vectors rather than modification marks on raw sequences. To make them suitable for our model, we processed them according to the protocol described in (16) and converted them into RNA modification data in the same format as in sramp17. The third dataset was downloaded from RMBase v2.0 (http://rna.sysu.edu.cn/rmbase/), which contains the m⁶A annotations of seven species, including zebrafish, human, mouse, rhesus, rat, chimpanzee and pig. These data covered a dozen different cell lines or tissues (including HepG2, brain, HEK293T, GM12878, HeLa, U2OS, H1A, H1B, NPC, ESC, LCL, CD8T, MT4, A549, MONO-MAC-6, Huh7, Jurkat, iSLK and PBT003) from the literature. In the above three datasets, only m⁶A sites containing the DRACH motifs were retained as positive samples. Those unmethylated adenosines in DRACH of the whole transcriptome were used as negative samples (non-m⁶A sites). Considering that non-m⁶A sites are much more distributed in the transcriptome than m⁶A sites, we also kept the positive-to-negative ratio as 1:10 as described in (22). To generate input sequences, we expanded each site by 50 nucleotides (nts) upstream and downstream, respectively. For each dataset, we randomly selected 80% of the input sequences as training data to train our model, and 20% as test data to assess our model performance. To further reduce the redundancy in sequence samples, we employed the CD-HIT-EST tool (37) to remove test samples that have a certain similarity (with a similarity score above 80%) to the training samples. The statistics of the two datasets used in our tests can be found in Supplementary Tables S5 and S6. For the downstream analyses, the gene expression data were collected from the GEO dataset with accession number GSM2072352, the mRNA half-lives and ribosome profiles were curated from (9) with GEO accession number GSE49339 and the histone modification (H3K36me3) data were downloaded from the GEO database with accession numbers GSM733685 and GSM733711.

Overview of MASS

Our multi-task curriculum learning based model consists of three main parts, including data organization, feature extractor and splitting classifiers (Figure 1). In data organization, we oversampled positive samples (with replacement) to construct a balanced dataset and packed the positive and negative samples into mini-batches with a ratio of 1:1. The input sequences were then encoded into binary vectors based on a one-hot encoding strategy (Figure 1A and B). The feature extractor designed for capturing features shared among different species is composed of convolution layers, bi-directional Long Short Term Memory (LSTM) units, and multi-head attention layers (i.e., multiple concatenated attention layers), which are used for local feature extraction, global feature extraction and learning of feature importance scores of individual positions in the input sequence, respectively (Figure 1A). A residual architecture that has been widely used in various deep learning tasks (38,39) to attenuate gradient vanishing is also adopted in our convolutional layers to build a deeper and more predictive neural network (40,41). The splitting classifiers are multi-layer fully connected neural networks for predicting the probabilities that the input sites of interest of the corresponding species are methylated. Each classifier of the splitting classifiers corresponds to a particular species and is independent of each other. In the training process, the input sequences are sampled and encoded in the data organization component and the encoded vectors are then fed to the feature extractor following the sequence of species defined according to the phylogenetic tree to generate feature maps. Finally, the m⁶A site prediction scores of different species are computed by the corresponding splitting classifiers based on the feature maps, individually.

Figure 1.

An overview of MASS. (A) Schematic illustration of the pipeline. Samples from multiple-species were first encoded by a one-hot encoder and then fed into our multi-task learning model sequentially. The sequence of species was determined by the evolutionary distances based on the phylogenetic tree. The multi-task learning model is composed of the shared feature extractor and several splitting classifiers, each corresponding to the prediction task of a species. BN: batch normalization layer, BLSTM: bi-directional long short term memory units, Conv: convolutional layer, Dense: fully connected layer. (B) Preparation of the training data. Since there are many more negative samples (unmethylated sites) than positive samples (methylated sites), we constructed the balanced sample set by oversampling positive samples. The balanced sample set was then divided into mini-batches to train the model.

Sequence encoding by dilated convolution

To fully exploit input sequences, we employ a dilated convolution technique that was originally proposed in the computer vision field (42–45) in the feature extractor of our model. The filters of the dilated convolution layers consider the combinations of long-range features, which thus increases the diversities of the higher-level features captured from the input sequences. More specifically, let be a discrete function, and be a discrete filter of size (2r + 1). Then, the 1D dilated convolution operator can be defined as

(1)

where i represents the index of the discrete filter, j represents the index of the given input, p represents the index of the filtered input, *_l stands for an l-dilated convolution and l represents the dilation factor (1-dilated convolution represents the traditional discrete convolution operation). Here, the dilation factors for three concatenated dilated convolutional layers are set to one, three and five, respectively, according to the conventional gapped-k-mer strategy (22,46).

Multi-task curriculum learning with a shared representation

To integrate multi-species m⁶A data to improve the performance of the prediction model, we designed a feature extractor shared across all species following the hard parameter sharing strategy (27) and then trained an independent classifier for each species. For a given training set X consisting of samples for T species and their corresponding label set Y, which can be represented as

(2)

(3)

where X_t stands for the sample set of the t-th species, and Y_t represents its corresponding label set. Then, the loss function for our multi-task learning model can be defined as

(4)

where represents the expectation of the cross entropy for the t-th species, is calculated by

(5)

where represents the shared feature extractor, and represents the splitting classifier of the t-th species. Several issues will arise if we apply multi-task learning directly to the multi-species m⁶A site prediction problem (see Supplementary Note S1). To address these issues, we further apply a curriculum learning strategy based on the known evolutionary relationships of all species in the training process of our multi-task model. More specifically, given a species set S (|S| = T) and their corresponding phylogenetic tree , the similarity between species s_i and s_j can be defined as , where d_ij represents the shortest path from s_i to s_j on . More details on the training process of our multi-task curriculum learning model can be found in Algorithm 1.

Our multi-task curriculum learning model can alleviate potential overfitting issues that are generally prone to occur in single-task models when the training set is not large enough and thus facilitate the training process (47). After training, common features across multiple species are captured by the feature extractor, which in principle should be able to enhance the generalization capability and interpretability of our model (47,48).

Multi-head attention mechanism for site weighting

Generally, not all features along the contextual sequence contribute to the final prediction in classification tasks. To enable our model to capture the importance scores of features from individual input positions in different tasks, we also adopt a special self-attention mechanism named multi-head attention in our deep learning framework (49). More specifically, the self-attention mechanism employed in our model can be represented by a weighted sum over the features of individual positions along the input sequence (L stands for the length of the input sequence, and d stands for the feature dimension of each site), that is

(6)

where stands for the response of the i-th site (d_k stands for a hyperparameter representing the response dimension), stands for the j-th site in the sequence C, stands for the learnable weight matrix, and λ_ij stands for a scalar value that assigns weights to individual sites along the sequence, which can be written as

(7)

where and can be represented as

(8)

where and represent the learnable weight matrices. To provide a more powerful representation of feature subspace at different positions, we concatenate several self-attention layers to construct a so-called multi-head attention layer (49), that is,

(9)

where represents a learnable weight matrix, N stands for the hyperparameter representing the number of heads, stands for the final output of the multi-head attention layer and Concat( · ) stands for the concatenate operation. In this work, N is set to 8, as suggested by the original paper in (49).

The multi-head attention layer in our model captures the connections between distant locations along the input sequence, which thus enhances the learning ability of the prediction model (49–51). In addition, the multi-head attention layer allows our model to capture coexisting sequence patterns associated with m⁶A modification, which thus can further increase the interpretability of the model.

Motif calling and global motif visualization

To interpret our model, we extracted and visualized the sequence motifs captured by MASS. More specifically, we scanned the entire input sequence to collect all subsequences that activate the convolutional filters (50,52,53), and then computed the positional weight matrix (PWM) of each motif based on the collected subsequences for the corresponding filter. For each species, we computed the PWMs by feeding the trained neural network with the sequences containing m⁶A sites from the test set. In particular, given an input sequence of length N, and a sliding window of size K and stride 1, we can obtain in total (N − K + 1) output values for each filter. Suppose that an output value is denoted by o_i, and its corresponding subsequence is represented by , where stands for the input sequence. If o_i is larger than the average , is then collected to generate the PWM, which is also used as the local motif for our downstream analyses.

Although motifs derived from the convolutional filters provide direct visualization of the important sequence patterns, they can only show the independent sequence features, such as protein binding sites. Further analyses of protein cooperation based on the motif co-occurrences, however, cannot be performed in this way. To address this problem, we employed a method called ‘class optimization’ visualization, which was originally proposed in the computer vision field (54,55) and had also been used in sequence analyses (53). More specifically, we maximized the prediction score P₊(S) of an input sequence S (represented as a four-row matrix of scalars) while keeping the neural network model weights unchanged as follows:

(10)

where γ is the regularization parameter. With the above objective function, the sequence S is optimized through a gradient descent optimizer and adjusted at each training step to gain a higher output score. We then converted such a (locally) optimized sequence S into a PWM with a softmax function. In contrast to the local motifs, this optimized sequence depicts all possible sequence patterns that simultaneously determine a specific class (representing the methylated sites here).

Model training and model selection

To train our multi-task learning model, we split our dataset into three independent parts, i.e., training, validation and test sets. The mini-batch gradient descent algorithm was used in the training process. To make the training process more stable, we also applied the Adam algorithm (56) that can determine the learning rate automatically based on the current batch gradient. The ‘save best’ strategy, as described in (57,58), was adopted during the training procedure. To gain a better performance, we applied a line search strategy (59,60) to determine the setting of hyperparameters, including learning rate, the number of units, and the filter size of our model. In particular, we first randomly initialized all of the hyperparameters to train a basic model. Then we searched one hyperparameter with the others fixed in a reasonable range according to a 5-fold cross-validation procedure. This process was repeated until all hyperparameters were determined. The five-fold validation results and the final values of hyperparameters are shown in Supplementary Table S7. After all the hyperparameters were determined, they were used to train MASS again on the whole dataset.

Our implementation of MASS relied on the TensorLayer (61), a TensorFlow-based deep learning library. In addition, the NVIDIA TITAN X was used to speed up the training process.

Single species and multiple species training

To our best knowledge, there exists no multi-species model for m⁶A prediction that can capture both single species and cross-species features simultaneously. Here, we evaluated our model from both single-species and multi-species perspectives to demonstrate that it can predict m⁶A sites precisely. For our multi-task learning task, we considered several test settings with different configurations of species. We first trained individual single species models, and then gradually added a species according to its evolutionary distance to human. In particular, we considered different settings with two, three, five and seven species, respectively. We also compared the performance of MASS to that of the existing methods, including SRAMP and DeepM6ASeq. Since DeepM6ASeq (24) is not particularly designed for multi-species training, we evaluated its performance only on single species. For the SRAMP model (22), as the originated paper did not provide the source code for training, we did not include it in the performance comparison on the nanopore sequencing data and the multi-species prediction task, which would require the retraining of the model.

RESULTS

MASS accurately predicts m⁶A sites

We first assessed the prediction performance of MASS on single-species data from the sramp17 dataset from (22). In particular, we tested MASS on the data concerning human and mouse. We also compared its performance to that of two existing methods: SRAMP (22) and DeepM6ASeq (24). Since SRAMP processes long reads by cutting them into short reads with DRACH motifs and generates a prediction score for each short read, here we used its highest prediction score among all reads as the final score for a given sequence, as described in (24). As shown in Figure 2, MASS trained on single-species data (denoted by MASS-Single) showed the best performance in terms of the two classical metrics, i.e., area under the receiver operating characteristic (AUROC) and area under the precision-recall curve (AUPRC), compared to the two baseline classifiers. In particular, our model on the human data achieved AUROC and AUPRC scores of 0.858 and 0.394, respectively, which were 3.8% and 9.4% higher than those of DeepM6ASeq and 3.1%, and 6.4% higher than those of SRAMP, respectively (Figure 2A and Supplementary Figure S1a). For the mouse dataset, our model achieved AUROC and AUPRC scores of 0.841 and 0.357, respectively, which were 4.2% and 8.2% higher than those of DeepM6ASeq, and 3.2% and 4.4% higher than those of SRAMP, respectively. On average, the AUROC scores of MASS-Single were 4.9% and 3.8% higher than those of DeepM6ASeq and SRAMP, respectively, and AUPRC scores were 29.9% and 16.5% higher than those of DeepM6ASeq and SRAMP, respectively (Figure 2B and Supplementary Figure S1b). Although our model was only slightly better in terms of AUROC, it achieved much higher AUPRC scores than the baselines. Note that AUPRC is more critical and informative than AUROC for an imbalanced dataset (62,63). Thus, the test results on single-species data showed that our model can predict more accurate m⁶A sites than the state-of-the-art baseline methods.

Figure 2.

The performance of different models on single-species data from the sramp17 dataset and multi-species data from the RMBase 2.0. (A andB) The AUPRC (area under the corresponding PR curve) scores of MASS-Single, DeepM6ASeq and SRAMP on the sramp17 single-species data for human (a) and mouse (b), respectively. (C andD) The AUPRC scores of MASS-Single, MASS-2species, MASS-3species, MASS-5species, MASS and DeepM6ASeq on the RMBase 2.0 data in human (c) and chimpanzee (d), respectively. MASS-kspecies represents our model trained using samples from k species. MASS represents our model using the samples from all the species.

To further demonstrate the generalizability of our model to plants and data from other sequencing technologies, such as nanopore direct RNA sequencing, we also compared the performance of our model on a nanopore direct RNA sequencing data from Arabidopsis (called nano20) derived from (14) with DeepM6ASeq. As shown in Supplementary Figure S4, MASS trained on nano20 achieved an AUROC of 0.706 and an AUPRC of 0.442, which were 15% and 27.8% higher than those of DeeepM6ASeq, respectively. This simple but promising comparison result showed that our model can potentially be applied to data from a broad range of species and sequencing technologies to make a quality prediction of m⁶A sites.

Next, we evaluated the prediction performance of our multi-task learning model on multiple-species data derived from RMBase 2.0 (64), also using DeepM6ASeq as a baseline for comparison. When trained all seven species, MASS achieved an AUROC score of 0.911 and an AUPRC score of 0.647 on human, which were 2.2% and 11.9% higher than those of DeepM6ASeq, respectively (Figure 2C and Supplementary Figure S1c). Similar comparison results are given for chimpanzee in Figure 2D and Supplementary Figure S1d. We also assessed the performances of our model in different test settings, including models trained on single-species, two-species, three-species, five-species and seven-species data (Supplementary Table S1, and Supplementary Figures S2 and S3). These tests showed that our multi-task model trained on multi-species data can improve the prediction of m⁶A sites compared to the model trained on single-species data. Overall, the evaluation results demonstrated that integrating multi-species data through the multi-task learning framework employed in our model can improve the m⁶A site prediction results.

MASS depicts the landscapes of m⁶A across multiple species

Once our model was validated, we applied it to predict the m⁶A modification likelihood of each DRACH motif in all mRNA transcripts of different species, particularly human and mouse. The DRACH motifs with predicted probabilities >0.5 were collected as putative m⁶A sites. The densities of putative m⁶A sites along mRNAs (including 5′-UTR, CDS and 3′-UTR regions) were calculated by averaging the results from all mRNAs. We also compared the prediction results to those of MASS-Single and baseline methods, including DeepM6ASeq and SRAMP. We observed that the m⁶A sites predicted by all four models were mainly located in the 3′-UTR regions and significantly enriched near the stop codons and the tails of 3′-UTR regions (Figure 3). Although the distributions of most of the m⁶A sites predicted by the four different models were consistent with each other, there were several distinct regions between the prediction results of our model and the other models. For example, the densities of the m⁶A sites predicted by MASS and MASS-Single were slightly higher than those predicted by DeepM6ASeq and SRAMP in the 5′-UTR and CDS regions, while much lower in the 3′-UTR regions (Figure 3A and B). The overall distributions of predicted m⁶A sites were highly consistent with the previously known trend about the distributions of CLIP-seq peaks of the m⁶A reader YTHDF1 (65). We also applied MASS to illustrate the distributions of m⁶A along mRNAs in all the seven species and observed similar distributions among the six mammals, but different distributions among mammals and zebrafish (Figure 3C). These similarities and differences between the predicted landscapes of m⁶A sites of different species were also consistent with the previous known patterns concerning the distributions of m⁶A sites in the transcriptomes of the species obtained through the m⁶A binding proteins (including YTHDF1, YTHDF2 and WTAP) or m⁶A antibodies, respectively (12,65–69). These results suggested that our multi-species model can depict the landscapes of m⁶A in multiple species in a reasonably accurate manner.

Figure 3.

The distributions of the predicted m⁶A sites along mRNAs. (A andB) The distributions of the m⁶A sites on human mRNAs (a) and mouse mRNAs (b) predicted by different models, including MASS, MASS-Single, SRAMP and DeepM6ASeq. (C) The distributions of the m⁶A sites predicted by MASS for all the seven species, including zebrafish, human, mouse, chimpanzee, rhesus, rat and pig.

MASS provides a protein-binding perspective for understanding m⁶A functions

To interpret our deep learning model, we also visualized and analyzed the sequence features captured by our model from both local segments of the input or the entire input (see ‘Materials and Methods’ section). In particular, we first collected the local sequence features (i.e., within the local segments) captured by 300 convolution filters (each with a length of 18 nts) that were highly active in the first convolution layer of the feature extractor in our model. These sequences features were then overlaid together to generate position weight matrices (PWMs) that were regarded as local motifs. Next, we visualized these local motifs and mapped them to the known binding motifs of RBPs obtained from the CISBP-RNA database. As expected, the consensus motifs of m⁶A (‘GGACU’) and the binding motifs of several m⁶A-associated RBPs repeatedly occurred in our identified local motifs for each species (Figure 4 and Supplementary Figure S4). The analysis results suggested that our model is able to capture the local sequence features of m⁶A sites that can be aligned well with the site recognition patterns of m⁶A related binding proteins. For example, the binding motif of HNRNPA2B1, a mediator of an m⁶A associated process (70), well matches to the local motifs identified by our model in chimpanzee (Figure 4). Overall, an average of 21.94% of the 300 local motifs captured by MASS (see Supplementary Table S2) from seven species can be annotated by TOMTOM (71). Since a number of motifs from several species contained in CISBP-RNA of several species were not derived directly from experiments, in order to further verify that the sequence features captured by our model are valid, we also compared the local sequence patterns captured by our model to those motifs obtained from other databases that are fully verified by experiments, including Dominguez2018 (72), Ray2013 (73) and ATtRACT (74). The results showed that the sequence features captured by our model can also significantly match the motifs in these databases (see Supplementary Figure S6 for the details).

Figure 4.

Local sequence motifs (i.e., within short segments) captured by our model. For each species, we show three examples of the most significant mappings of the local sequence motifs captured by our model to the CIS-BP-RNA motif database. The motifs derived from the CIS-BP-RNA database and our model are shown on the top and bottom, respectively.

Though the local motifs are able to depict certain properties of m⁶A, the long-range co-occurrence motifs around m⁶A sites (i.e., along with the entire input sequences, which were also called the global motifs hereafter) can provide another angle to investigate the underlying mechanisms of m⁶A. Here, we applied a technique called class optimization (54) to capture such global sequence motifs of m⁶A. We found that for all seven species, there existed a ‘GGACA’ motif at the 5′ end of the input sequence (Figure 5A, dashed gray box) and a ‘GGUG’ motif near the center of the input sequence (Figure 5A, solid gray box). Previous studies indicated that the distribution of SRSF1-binding sites is highly correlated with m⁶A sites (75,76). In addition, we found that the ‘GGUG’ motif identified by MASS can be matched to the binding motif of TAF15, a binding partner of the m⁶A reader RBMX (77), which may also help explain the co-occurrence of ‘GGUG’ motif and m⁶A. To test whether the single-species model (i.e., trained using only single-species data) can capture these co-occurrence motifs or not, we also analyzed the optimized input sequences of MASS-Single using the same procedure as in the multi-species model. However, we did not observe any co-occurrence motifs in all the seven species (Figure 5B), which thus demonstrated the advantages of integrating multi-species data in our multi-task learning model to capture the global sequence features of m⁶A. These results suggested that the global motifs derived from our multi-species model can help characterize the complex mechanisms and functions of m⁶A modification.

Figure 5.

Global sequence motifs (i.e., within the entire input sequence) captured by our model. (A) The global sequence motifs of each species derived by the ‘class optimization’ of MASS, which was trained on multiple-species data. The ‘GGACA’ pattern in the dashed gray box can be matched to the binding motif of SRSF1, which has been reported to by closely related to the functions of m⁶A (75,76). The ‘GGUG’ motif in the solid gray boxes can be mapped to the binding motifs of TAF15, which is an interactor of the m⁶A reader RBMX. (B) The co-occurrence features were missed in the global sequence motifs of each species derived by the model trained on single-species data.

MASS detects important conserved genes enriched with m⁶A sites among different species

Based on the distributions of the predicted m⁶A sites in mRNAs, we defined a variable called the m⁶A capacity to assess the overall enrichment of m⁶A in an mRNA transcript (see Supplementary Note S2). We computed the m⁶A capacities of all mRNA transcripts in the seven species based on the prediction results of our model. The mRNAs with m⁶A capacities above the median were regarded as high m⁶A capacity transcripts. We then compared the numbers of high m⁶A capacity transcripts (including their orthologs) shared among different species (from two to seven species) between MASS and MASS-Single, which were trained using multiple-species and single-species data (see ‘Materials and Methods’ section), respectively. Our comparison results showed that MASS was able to identify more conservation across different species (from two to seven) than the single-species model (Figure 6), implying that our multi-species model can capture more common features of m⁶A among species than the single-species one.

Figure 6.

Comparison of the numbers of shared mRNAs with high m⁶A capacities derived by MASS and MASS-Single across different species.

Furthermore, we screened for mRNAs with high m⁶A capacities shared among all the seven species derived by MASS and MASS-Single, respectively. We found that most of the high-m⁶A-capacity mRNAs derived by our model were related to m⁶A functions (Table 1). For example, among all the identified genes, TUBB2A, which encodes a protein participating in mitosis and intracellular transport processes, had been found to interact with the general m⁶A writer METTL3 (78). We also found that there were more mRNAs in the MASS set that have support from the m⁶A literature than in the MASS-Single set (Table 1). For example, CSNK1D is one of the genes in the MASS set and it was reported previously that the deletion of the m⁶A locus in its 3′-UTR may elongate the circadian period of mouse cells (79). These observations indicated that high-m⁶A-capacity transcripts conserved among the species may be highly regulated by m⁶A for important consensus biological functions. These results also suggested that our model derived from multi-species data can provide new insights into understanding the regulatory functions of m⁶A in mRNAs.

Table 1.

The set of the mRNAs with high-m⁶A-capacities derived by MASS (which was trained on multiple-species data) had more support from the literature than that derived by MASS-Single (which was trained on single-species data)

Intersection			MASS (multi-species only)			MASS-Single (single species only)
Gene	Description	Reference	Gene	Description	Reference	Gene	Description	Reference
TUBB2A	METTL3 interactor	BioGRID	CSNK1D	elongate the circadian period in vivo	(79)	ACTC1	KIAA1429 interactor	BioGRID
HLA-A	RA-associated m6A-SNPs	(90)	NME1-NME2	miCLIP targets	(91)	TRIM10
ACTC1	METTL3 and METTL14 interactor	BioGRID	TUBA1A	m6A enriched gene	(92)	HBG1
HLA-C	KIAA1429 interactor	BioGRID	GH2			MIR181A2
DDX39A	METTL3 interactor	BioGRID	CCL3L3
PPAN-P2RY11	m6A related gene	(91)	CSH2
ALDH16A1	METTL14 interactor	(93)	CSH1
DHRS4L2	High level m6A	(94)	NME2
ACTG2			TUBA8
RAB37			CSHL1
HLA-B
ACTA1
HLA-G
TRIM26
HLA-F
HLA-E
RAB26

To further explore the molecular functions of the m⁶A methylated transcripts, we also selected the top 10% mRNAs with the highest m⁶A capacities for the gene ontology (GO) enrichment analysis in each species. GO molecular function terms that were significantly enriched (P < 0.05) in each species were collected (Figure 7). We found that several terms, including structural molecular activity, protein binding, and sequence-specific DNA binding were repeatedly enriched in different species (Supplementary Table S3; also see Supplementary Table S4 for complete results), which implied that m⁶A may play a critical role in protein formation and protein binding processes (80,81). We also analyzed the most significantly enriched molecular function terms in each species, and found that the highly enriched terms were similar among mouse, rat and human (Figure 7), which further verified the conservation of m⁶A profiles between human and mouse (12). Though more experiments are still needed, our prediction results may provide useful molecular hints for further studies of m⁶A regulation.

Figure 7.

The GO (gene ontology) enrichment analysis for the set of top genes with the highest m⁶A capacities (see Supplementary Note S2 for the detailed definition) in each species. Here, we only show GO terms that were among the top five significant ones in each species.

MASS can be applied to characterize the properties of gene regulation

In this section, we show that, based on the prediction results of MASS, we can further analyze the associations between m⁶A modification and the biological processes, such as gene expression regulation, RNA stability, RNA structure, translation and histone modification.

mRNAs with high m⁶A capacities are repressed in expression

It is well known that m⁶A can be widely involved in multiple biological processes including RNA stability, splicing, translation and pre-miRNA processing (9,82–84). Therefore, we speculated that m⁶A can also play an important role in gene expression regulation. To examine the relationship between the predicted m⁶A profiles and gene expression, we compared the expression levels of mRNAs with m⁶A capacities above the average to those with m⁶A capacities below the average. As a result, mRNAs with high m⁶A capacities were significantly down-regulated (Figure 8A, P = 2.78 × 10⁻²⁷, Wilcoxon rank sum test), which can also be supported by the observations reported in previous studies (9,78).

Figure 8.

Downstream analyses of the relationships between m⁶A and other biological processes. (A) Comparison of the gene expressions between mRNAs with m⁶A capacities higher than the average (> Avg.) and those with m⁶A capacities less than the average (< Avg.). mRNAs with higher m⁶A capacities are significantly less expressed (P = 2.78 × 10⁻²⁷, Wilcoxon rank sum test), ‘FPKM’: fragments per kilobase per million mapped fragments. (B) Comparison of the stability between mRNAs with m⁶A capacities higher than the average (> Avg.) and those with m⁶A capacities less than the average (< Avg.). mRNAs with higher m⁶A capacities were significantly more unstable (P = 9.35 × 10⁻⁹⁵, Wilcoxon rank sum test). (C) Comparisons of translation rates (represented by the number of ribosome-bound fragments) between mRNAs with m⁶A capacities higher than the average (> Avg.) and those with m⁶A capacities less than the average (< Avg.), which shows that high m⁶A capacity RNAs translate faster (P = 5.95 × 10⁻³⁰, Wilcoxon rank sum test). ‘RPKM’: reads per kilo base per million mapped reads. (D) Comparisons of m⁶A potentials in different RNA structure types. The predicted m⁶A potentials in single-stranded regions (SS) were higher that those in double-stranded (DS) regions (P = 2.11 × 10⁻¹², Wilcoxon rank sum test) and poised regions (PRs, P = 5.41 × 10⁻⁴, Wilcoxon rank sum test). (E) The scatter plot of average m⁶A potentials normalized by peak lengths and H3K36me3 peak densities transformed by log2. The predicted m⁶A potentials were positively correlated to H3K36me3 densities (Pearson correlation = 0.55, P = 0.0, two-tailed t-test).

mRNAs with higher m⁶A capacities are more unstable

Though we observed a negative impact of m⁶A on gene expression, the regulation mechanisms of m⁶A in determining mRNA stability are not obvious. To further depict the relationship between mRNA stability and m⁶A, we collected mRNA half-lives from (9) and associated them with the corresponding m⁶A capacities derived from the m⁶A prediction results. In particular, we compared the half-lives of mRNAs with m⁶A capacities greater than the average (>Avg.) to those with m⁶A capacities lower than the average (<Avg.), and found that mRNAs with higher m⁶A capacities were generally more unstable (Figure 8B, P = 9.35 × 10⁻⁹⁵, Wilcoxon rank sum test). This observation is also consistent with the previous reports indicating that m⁶A promotes mRNA decay by promoting the binding of m⁶A readers (9). This further verified that the m⁶A capacity derived by our model can represent the overall methylation states of the corresponding mRNAs.

mRNAs with higher m⁶A capacities show higher translation efficiency

We further investigated the association between mRNA translation efficiency and the corresponding m⁶A capacities derived by our model. We first collected two replicated data sets concerning mRNA translation efficiency from the literature (9). Then, we compared the translation rates of mRNAs with m⁶A capacities greater than the average (>Avg.) to those of mRNAs with m⁶A capacities lower than the average (<Avg.), and observed that mRNAs with higher m⁶A capacities were translated in slower rates (Figure 8C, P = 9.78 × 10⁻³¹, Wilcoxon rank sum test). Previous studies had demonstrated the functions of m⁶A in controlling mRNA translation efficiency (83,85,86), and our analysis results were consistent with the conclusions drawn from these studies, which thus also provided another implicit evidence to support the performance of our model.

m⁶A shows a preference for single-stranded regions of RNAs

Since the secondary structures of RNAs are also involved in the regulation of RNAs, we hypothesized that m⁶A sites were non-uniformly distributed in different types of RNA structural regions. To test this hypothesis, we first collected the parallel analysis of RNA structure (PARS) data from (87) and annotated three types of regions, including single-stranded regions (SSs), double-stranded regions (DSs) and poised regions (PRs, i.e., regions that cannot be defined as DSs or SSs) according to the descriptions in the original paper (87). Then we compared the probabilities of the predicted m⁶A sites located in the aforementioned three different types of RNA structural regions (Figure 8D). We found that the potentials of m⁶A sites in single-stranded regions were significantly greater than those in double-stranded regions (P = 2.11 × 10⁻¹², Wilcoxon rank sum test) and poised regions (P = 5.41 × 10⁻⁴, Wilcoxon rank sum test; the effect sizes are given in Supplementary Table S1). The preference for m⁶A single-stranded regions in RNAs suggested that m⁶A may play a certain role in regulating RNA structures, which are also supported by the experimental results in previous studies (80,81).

Predicted m⁶A potentials are positively correlated with H3K36me3 densities

Recently, it has been shown that H3K36me3 can guide m⁶A deposition by recruiting METTL14, which is a component of an m⁶A writer complex (88). Thus, we hypothesized that the m⁶A profiles derived from MASS can be extensively correlated with the enrichment of H3K36me3 at the transcriptome scale. To verify the speculated correlation between H3K36me3 and m⁶A, we collected the H3k36me3 peaks and calculated the average m⁶A potentials in each H3K36me3 peak region according to the predicted results of MASS. We found that the average m⁶A potentials were positively correlated with the densities of H3k36me3 peaks (Figure 8E), which suggested a broad association between H3K36me3 and m⁶A. This result further demonstrated the reliability of our prediction results and implied that our model may be applied to investigate the relationships between m⁶A and other histone modifications.

DISCUSSION

In this paper, we propose a deep learning framework based on multi-task curriculum learning to predict the transcriptome-wide likelihoods of m⁶A sites across multiple species. To our best knowledge, this is the first attempt to model the m⁶A profiles of multiple species simultaneously. In our multi-task learning model, the consensus features of m⁶A shared among multiple species are learned in the shared feature extractor, which can reduce potential species-specific bias in the experimental data. Therefore, this strategy increases the accuracy and generalization capacity of our model in predicting novel m⁶A sites. In addition, characterizing the m⁶A profiles of different species simultaneously can shed light on the similarity and the difference of m⁶A patterns among different species. Though several deep learning based models have already been proposed for m⁶A site prediction in recent years (24,89), most of them are insufficient in model interpretability. Here, our model provides both local and global views of the sequence features, which can help interpret the deep learning model from a protein–RNA binding perspective. Moreover, since our multi-task learning model efficiently and logically integrates m⁶A data from multiple species, it can capture more consensus and specific features of m⁶A in different species than the single-species model or the existing methods.

In the downstream analyses, we further investigated the relationships between m⁶A and gene expression, RNA stability, translation efficiency, RNA structure stability and histone modification. The analyses based on our prediction results were highly consistent with the conclusions drawn in the previous studies, suggesting that the potential applications of our model in solving m⁶A-associated biological problems. Note that the data that we used here to calculate the associations between m⁶A and other variables were mainly collected from different cell lines or tissues. Thus, further experiments to control relevant variables are still needed. In addition, the current version of our multi-species model only considers seven vertebrate species (i.e., plants and prokaryotes are not included), which may prevent our model from uncovering the similarities and differences of m⁶A patterns either between animals and plants or between eukaryotes and prokaryotes. Therefore, integrating data from more species will be regarded as an important direction in our future work.

Contact for reagent and resource sharing

Further information and requests for resource sharing may be directed to and will be fulfilled by the corresponding authors Dr. Jianyang Zeng (zengjy321@tsinghua.edu.cn) and Dr. Tao Jiang (jiang@cs.ucr.edu).

SUPPLEMENTARY DATA

Supplementary Data are available at NAR Online.

FUNDING

National Natural Science Foundation of China [61872216, 81630103, 61772197, 31900862]; Turing AI Institute of Nanjing; National Key Research and Development Program of China [2018YFC0910404]; Zhongguancun Haihua Institute for Frontier Information Technology. Funding for open access charge: National Natural Science Foundation of China.

Conflict of interest statement. None declared.