m5UPred: A Web Server for the Prediction of RNA 5-Methyluridine Sites from Sequences

Abstract

As one of the widely occurring RNA modifications, 5-methyluridine (m⁵U) has recently been shown to play critical roles in various biological functions and disease pathogenesis, such as under stress response and during breast cancer development. Precise identification of m⁵U sites on RNA is vital for the understanding of the regulatory mechanisms of RNA life. We present here m5UPred, the first web server for in silico identification of m⁵U sites from the primary sequences of RNA. Built upon the support vector machine (SVM) algorithm and the biochemical encoding scheme, m5UPred achieved reasonable prediction performance with the area under the receiver operating characteristic curve (AUC) greater than 0.954 by 5-fold cross-validation and independent testing datasets. To critically test and validate the performance of our newly proposed predictor, the experimentally validated m⁵U sites were further separated by high-throughput sequencing techniques (miCLIP-Seq and FICC-Seq) and cell types (HEK293 and HAP1). When tested on cross-technique and cross-cell-type validation using independent datasets, m5UPred achieved an average AUC of 0.922 and 0.926 under mature mRNA mode, respectively, showing reasonable accuracy and reliability. The m5UPred web server is freely accessible now and it should make a useful tool for the researchers who are interested in m⁵U RNA modification.

Graphical Abstract

Jie Jiang et al. present m5UPred, the first web server for identification of m⁵U sites from the primary sequences of RNA by advanced computational method in this study. It can help to accurately predict the m⁵U sites from RNA sequences, which will assist researchers to understand the RNA methylation mechanism and its biological functions.

Introduction

The development of high-resolution transcriptome mapping and quantification technologies has made epigenetic modifications of RNA one of the fastest-growing fields in biological research in the past several years. Over 170 post-transcriptional modifications have been identified, with the majority of them occurred in tRNAs and rRNAs.¹ RNA modifications showed imperative roles in varied biological functions (e.g., embryonic stem cell development,² cancer cell survival, migration,³ and response to environmental exposures⁴). Over 100 RNA modification enzyme mutations have been associated with human diseases.⁵ Besides the well-known role of fine-tuning RNA structures and functions to regulate gene expression and protein synthesis by RNA modifications,⁶ there are many other functions associated with RNA modifications. Some post-transcriptional RNA modifications have shown to be dynamic processes that have regulatory roles similar to post-translational protein modifications in controlling cell-type-specific functions.⁷

However, as one of the most abundant RNA modifications, the identification and functional characterization of 5-methyluridine (m⁵U) remain extremely limited in the literature. As a pyrimidine modification, m⁵U involves methylation at the 5-carbon position of uridine, which may be the first pyrimidine methyltransferases evolved to catalyze the pyrimidine-C5 methylation.⁸ The enzymes catalyzing the modification of m⁵U are TrmA in Escherichia coli,^9,10 Trm2p in Saccharomyces cerevisiae,¹¹ and TRMT2A and TRMT2B in mammals.^12,13 m⁵U has been reported to participate in the development of breast cancer,⁵ systemic lupus erythematosus,¹⁴ and regulation of stress response and development in plants.¹⁵ Accurate identification of m⁵U sites is crucial to understanding fundamental biological processes and functions in all species. Some wet-lab experimental methods, such as miCLIP-Seq, iCLIP, and FICC-seq, have been developed to identify m⁵U sites.¹⁶ However, RNA sequencing could be a high-cost and time-consuming process, and the specificity of antibodies used for immunoprecipitation restricts the delivery of accurate sequencing results. Thus, only very limited data have been generated for m⁵U sites so far. In this study, we would like to propose an in silico technique for the identification of m⁵U sites based on sequence-derived information using a support vector machine (SVM) algorithm.

A number of computational methods have been developed to predict epigenetic modifications of RNA, including m6A site predictors WHISTLE,¹⁷ SRAMP,¹⁸ Gene2vec,¹⁹ iRNA-Methyl,²⁰ and M6AMRFS;²¹ m5C site predictors RNAm5Cfinder,²² iRNA-m5C,²³ and M5C-HPCR;²⁴ pseudouridine site predictors iRNA-PseU,²⁵ PseUI,²⁶ PPUS,²⁷ and PIANO;²⁸ and one-stop platform iMRM for simultaneously identifying various RNA modifications in multiple species.²⁹ However, to the best of our knowledge, there is no m⁵U site predictor available so far. Thus, in this study we would like to propose the first prediction framework, m5UPred, which can be utilized for high-accuracy identification of m⁵U site from RNA sequences. A user-friendly web server has also been developed and made free access publicly at https://www.xjtlu.edu.cn/biologicalsciences/m5u. We anticipate that our newly proposed model, m5UPred, could make the best use of limited experimentally detected data and facilitate the research of m⁵U site modification by providing an alternative computational approach.

Results

Determine the Best Machine Learning Algorithm Used for m⁵U Site Prediction

In order to determine the best classifier for constructing m5UPred, the performances of different classifiers were tested on the independent dataset (see Table 1). SVM achieved area under the receiver operating characteristic curve (AUC) of 0.933 for full transcript mode and 0.951 for mature mRNA mode, which was the best among all classifiers and used to build m5UPred.

Table 1.

Performance Evaluation of m5UPred by Independent Testing Dataset

Mode	Algorithms	Sn (%)	Sp (%)	ACC (%)	MCC	AUC
Full transcript	SVM	86.06	84.72	85.39	0.708	0.933a
	RF	85.18	83.44	84.31	0.687	0.929
	NB	86.06	61.53	73.80	0.491	0.845
	GLM	83.03	80.45	81.74	0.635	0.897
Mature mRNA	SVM	86.99	89.07	88.03	0.761	0.951a
	RF	81.30	97.56	89.43	0.799	0.951
	NB	92.36	46.91	69.63	0.441	0.862
	GLM	85.57	81.54	83.56	0.672	0.915

We randomly selected 80% of experimentally validated m5U sites as training dataset and the performance of predictors was evaluated by the remaining 20% of m5U sites as independent testing data.

^aSVM achieved best performance among all classifier with AUC of 0.933 and 0.951 for full transcript mode and mature mRNA mode.

Performance Evaluation of m5UPred by Benchmark and Independent Testing Dataset

The prediction performance of m5UPred was evaluated by 5-fold cross-validation and an independent testing dataset, respectively (see Table 2). Using SVM as the final classifier, m5UPred was developed and tested using the same datasets that were previously applied for algorithm selection. In addition, the tune length of SVM in caret package was set to 9, with a random grid search for optimization. When evaluated by the independent testing data, m5UPred showed good generalization capability and achieved 0.767 Matthews correlation coefficient (MCC) with 88.35% accuracy and 0.956 AUC for full transcript mode and 0.795 MCC with 89.70% accuracy and 0.954 AUC for mature mRNA mode.

Table 2.

Prediction Performance Using Cross Validation and Independent Testing Dataset

Dataset	Testing Method	Sn (%)	Sp (%)	ACC (%)	MCC	AUC
Full transcript	cross validation	87.59	89.04	88.32	0.767	0.956
	independent test set	87.90	88.80	88.35	0.767	0.956
Mature mRNA	cross validation	88.64	91.18	89.91	0.798	0.956
	independent test set	87.44	91.95	89.70	0.795	0.954

80% of experimentally validated m5U sites were used for training, while its performance was evaluated by the remaining 20% as independent testing data.

Performance Evaluation of m5UPred by Cross-Technique and Cross-Cell-Type Validation

It was shown previously that positive samples of RNA modification captured by different techniques may have different overall patterns.³⁰ To test the performance and robustness of our newly proposed predictor in finding m⁵U sites generated from different techniques and cell types, we further evaluated the performance of m5UPred by cross-technique and cross-cell-type validation (Table 3). When tested by the independent dataset generated from another technique or cell type, m5UPred achieved AUC of 0.882 and 0.922 for cross-technique validation, and AUC of 0.899 and 0.926 for cross-cell-type validation, under full transcript mode and mature mRNA mode, respectively. It shows robustness and reliability of m5UPred.

Table 3.

Cross-Technique and Cross-Cell-Type Validation

Mode	Testing Method	Evaluation Metric	Cross-Technique Validation			Cross-Cell-Type Validation
			miCLIP-Seq	FICC-Seq	Average	HEK293	HAP1	Average
Full transcript	cross validation	Sn (%)	86.70	89.80	88.25	86.26	89.67	87.96
		Sp (%)	86.83	91.37	89.10	87.19	90.48	88.84
		ACC (%)	86.76	90.58	88.67	86.72	80.15	83.44
		MCC	0.735	0.812	0.773	0.735	0.901	0.818
		AUC	0.946	0.966	0.956	0.942	0.969	0.955
	independent dataset	Sn (%)	75.36	56.48	65.92	82.79	57.77	70.28
		Sp (%)	89.23	90.10	89.67	89.62	90.21	89.92
		ACC (%)	82.29	73.29	77.79	86.20	73.99	80.10
		MCC	0.652	0.495	0.574	0.726	0.507	0.617
		AUC	0.910	0.853	0.882	0.941	0.857	0.899
Mature mRNA	cross validation	Sn (%)	88.34	94.14	91.24	89.86	95.32	92.59
		Sp (%)	90.52	98.04	94.28	91.13	96.71	93.92
		ACC (%)	89.43	96.09	92.76	90.50	96.02	93.26
		MCC	0.789	0.922	0.856	0.810	0.920	0.865
		AUC	0.962	0.992	0.977	0.964	0.987	0.975
	independent dataset	Sn (%)	90.07	38.81	64.44	95.41	39.01	67.21
		Sp (%)	90.86	97.51	94.19	88.93	98.12	93.53
		ACC (%)	90.46	68.16	79.31	92.17	68.57	80.37
		MCC	0.809	0.449	0.629	0.845	0.461	0.653
		AUC	0.970	0.873	0.922	0.981	0.871	0.926

As previously shown in Table 2, the experimentally validated m5U sites were further separated by high-throughput sequencing techniques and cell types, under full transcript and mature mRNA mode, respectively. Independent tests by the other technique or cell type are employed to evaluate the performance additional to 5-fold cross validation. When dataset miCLIP_F was used for training, its performance was tested by 5-fold cross-validation and an independent dataset of FICC_F. Similarly, while dataset FICC_F was used for training, miCLIP_F served as an independent test set. The same testing scheme was used for datasets separated by cell types as well. The performance of different sequencing techniques and cell types are then averaged. The performance evaluation of m5UPred by cross-technique and cross-cell-type validation using different machine learning classifiers is listed in Table S1.

Web Implementation

To facilitate the access of our model by experimental researchers, a web server has been developed using Hyper Text Markup Language (HTML), Cascading Style Sheets (CSS), and Hypertext Preprocessor (PHP) and is accessible at https://www.xjtlu.edu.cn/biologicalsciences/m5u. It allows users to submit query RNA sequences in FASTA format with over 41 nt in length and a uridine in the center for analysis. The web server will evaluate the possibility of m⁵U modification in the given sequences, which return all putative m⁵U sites with download function (Figure 1).

Figure 1.

Input and Output of m5UPred Web Server

The input of m5UPred is FASTA sequences. The user can paste the sequences into a text box or provide a FASTA file. m5UPred will predict the m⁵U site possibility using our m5UPred model and returns predicted m⁵U sites. Results can be downloaded in table format.

Discussion

In this study, we extracted m⁵U modification sites from a data source generated by two sequencing methods and two cell types. A high-accuracy predictor was built by using a SVM model to predict the m⁵U modification sites. Satisfactory prediction performance was observed, with an average AUC of 0.956 for full transcript and 0.954 for mature mRNA when it was evaluated by an independent test set with 20% randomly selected sites from the dataset. The model was further tested by cross-technique and cross-cell-type validation. The results show that the positive m⁵U sites generated from different techniques and cell types share some common features, and these features can be captured by our m5UPred by extracting sequence-derived information. To help understand the sequence composition, we analyzed full transcript sequences from different cell types and sequencing methods by DREME³¹ motif discovery tool. The motifs identified are provided in Figure S1 and are consistent with the tRNA T-loop motif of GTTCG/AA proposed by the data source paper.¹⁶ A web server was developed based on our model and made available to the public to assist the prediction of m⁵U sites by other researchers.

It is worth noting that although m⁵U sites were generated under different conditions, they are still from a single source; this may pose an over-fitting problem, and the performance of the model may be over-estimated. We calculated the false discovery rate (FDR) and false omission rate (FOR) at different thresholds and the true positive rate (TPR). The result confirmed our previous statement about over-estimation, and the result is provided in the Supplemental Information. We will keep updating our model when more data are available. When comparing the crosslink peaks from two sequencing methods of the raw data, only around 35% of sites with over 5 crosslink peaks are identified by both methods;¹⁶ therefore, the true-positive m⁵U sites will need to be further confirmed by more experimental data. We noticed that tRNA modifications take the dominant position in the dataset we used for training, which was also confirmed by the data source paper. Considering the biological structures and functional difference of varied RNA molecules, further study will need to be performed to optimize the model using optimized window size, secondary structures, modification motif, and genome information to further improve the model robustness and generalization ability.

Materials and Methods

Training and Testing Datasets

Benchmark and Independent Testing Dataset Used for m⁵U Site Prediction

The positive samples (m⁵U sites) were obtained from the recently published single-nucleotide resolution m⁵U sequencing data,¹⁶ and the sequencing results were generated by FICC-seq and miCLIP-seq technologies on two cell lines, HEK293 and HAP1 (Table 4). Data were downloaded from Gene Expression Omnibus (GEO), with the GEO accession number GEO: GSE109183. Previous studies^32,33 showed RNA sequence 41 nt in length with an RNA modification site in middle provided the most promising prediction result. Thus, we adopt this formula and designed our positive dataset by generating 41 nt sequences with experimental identified U sites in the center. Unmodified uridine sites located on the same transcripts of the positive m⁵U sites were randomly selected, and 10 negative datasets were generated. By combining each of these 10 negative sets with the positive data, 10 separate datasets were constructed with a 1:1 positive-to-negative ratio. Their prediction performances were averaged during the evaluation to reduce batch variance.

Table 4.

Base-Resolution Dataset Used for m⁵U Site Prediction

GEO Accession	Technique	Cell Line	No. of Sites		Source
			Full	Mature
GSE109183	miCLIP-Seq	HEK293	3,696	1,232	15
	FICC-Seq
		HAP1

80% of the data were randomly selected for training, while the remaining 20% were retained as an independent testing dataset for performance evaluation. All positive and negative sequences used in this project can be freely downloaded from https://www.xjtlu.edu.cn/biologicalsciences/m5u. and are provided in the Supplemental Information.

The performance of our predictor was evaluated under two modes, namely full transcript mode and mature mRNA mode. For the full transcript mode, the positive and negative m⁵U sites located in both exonic and intronic regions are all considered to generate the data, whereas, in the mature mRNA mode, only positive and negative m⁵U sites located on mature mRNA transcripts are employed. Positive m5U sites in different RNA families are listed in the Supplemental Information. For performance evaluation, 80% of the dataset was randomly selected as a benchmark training dataset, while the remaining 20% was used as independent testing data.

Dataset Separated by High-Throughput Sequencing Techniques and Cell Types

Besides randomly selected training and testing data from experimentally validated m⁵U sites, we applied two more strictly evaluation methods to validate the performance of the proposed predictor. The experimentally validated m⁵U sites were further divided by high-throughput sequencing technique and cell type (Table 5), from which the cross-technique and cross-cell-type testing were applied for performance evaluation under full transcript and mature mRNA modes, respectively. The performance was also evaluated by 5-fold cross-validation and an independent test set. But rather than randomly selecting 20% of the total data as a test set, we generated test sets by different techniques and cell types. Concretely, for cross-technique validation, when dataset miCLIP_F or miCLIP_M were used for training, their performance was tested by an independent dataset of FICC_F or FICC_M. Then, we switched the train and test set and used dataset FICC_F or FICC_M for training and miCLIP_F or miCLIP_M for independent testing. The validation scheme was applied for the dataset separated by cell types as well.

Table 5.

Dataset Separated by Sequencing Techniques and Cell Types

Separation Method	Mode	Condition	Site No.	Dataset	Total No.
Technique	full	miCLIP-Seq	2,225	miCLIP_F	3,696
		FICC-Seq	1,471	FICC_F
	mature	miCLIP-Seq	823	miCLIP_M	1,232
		FICC-Seq	409	FICC_M
Cell type	full	HEK293	2,467	HEK293_F	3,696
		HAP1	1,229	HAP1_F
	mature	HEK293	868	HEK293_M	1,232
		HAP1	364	HAP1_M

Feature Extraction

High classification accuracy of many machine learning algorithms largely relies on sequence encoding strategies for feature extraction of RNA sequences. To achieve the best performance, the chemical properties of nucleotides and their distribution information are employed for sequence feature extraction in this study. This strategy was adapted from previous work by Bari et al.,³⁴ which was originally used for splicing site prediction from DNA sequence and lately is being widely used in encoding RNA sequences for RNA modification prediction.35, 36, 37

Nucleotide Density

The nucleotide density represents the distribution and frequency information of nucleotides at each position. The density ($d_i$) of a nucleotide $N$at $i^{th}$ position can be calculated by the number ($n$) of $N$ occurred before ${(i+1)}^{th}$ position divided by $i:d_i=n/i$. So, for a sequence “AUAGUCAUAA,” the density of A is 1, 0.67, 0.43, 0.44, and 0.50 at the 1^st, 3^rd, 7^th, 9^th, and 10^th positions, respectively. Similarly, U is 0.50, 0.40, and 0.38 at positions 2, 5, and 8; C is 0.17 at position 6; and G is 0.25 at position 4 (Table 6).

Table 6.

Calculation of Nucleotide Density for a Sample Sequence

Sequence	A	U	A	G	U	C	A	U	A	A
No. of same nucleotide accumulated	1	1	2	1	2	1	3	3	4	5
Position	1	2	3	4	5	6	7	8	9	10
Nucleotide density	1	0.5	0.67	0.25	0.4	0.17	0.43	0.38	0.44	0.5

Nucleotide Chemical Property

The encoding scheme of nucleotide chemical property was designed based on different chemical structures of four RNA nucleotides. Four building blocks of RNA, namely adenosine (A), uridine (U), guanosine (G), and cytosine (C), are categorized into different groups depending on the number of ring structures (two for A and G and one for C and T), existence of an amino group (A and C) or keto group (G and T), and strong (C and G) or weak (A and T) hydrogen bonds. Based on this information, together with the nucleotide density information, nucleotide $N$ at $i^{th}$position from sequence $S$ (with length$l)$ can be represented by the formula $Ni=\{x_i,y_i,z_i,d_i\}\left(i=\mathrm{1,2,3},\dots l\right)$ which satisfies the following equations:

$$x_i=\{\frac{1\text{ if }{\text{s}}_i\in \left\{\text{A},\text{G}\right\}}{\text{0 if }{\text{s}}_i\in \left\{\text{C},U\right\}},y_i=\{\frac{1\text{ if }{\text{s}}_i\in \left\{\text{A},C\right\}}{\text{0 if }{\text{s}}_i\in \left\{\text{G},U\right\}},z_i=\{\frac{1\text{ if }{\text{s}}_i\in \left\{\text{A},U\right\}}{\text{0 if }{\text{s}}_i\in \left\{C,\text{ G}\right\}}$$ (1)

Concretely, A, C, G, and U can be encoded as vectors (1,1,1, $d_i$), (0,1,0,$d_i$), (1,0,0, $d_i$), and (0,0,1, $d_i$), respectively. So, each of the nucleotides in the RNA sequences will be transferred into four numeric values; thus, each of our 41 nt RNA sequences will be encoded into a 164-dimension vector.

Evaluation Matric

Five metrics were employed to evaluate the performance our model, namely Sn (Sensitivity), Sp (Specificity), MCC, ACC (overall accuracy), and AUC, with the equations below:

$$Sn=\frac{TP}{TP+FN}$$ (2)

$$Sp=\frac{TN}{TN+FP}$$ (3)

$$MCC=\frac{TP\times TN-FP\times FN}{\sqrt{(TP+FP)\times (TP+FN)\times (TN+FP)\times (TN+FN)}}$$ (4)

$$ACC=\frac{TP+TN}{TP+TN+FP+FN},$$ (5)

in which TP, TN, FP, and FN represent the number of true positives, true negatives, false positives, and false negatives, respectively. A threshold of 0.5 was applied to calculate Sn and Sp. R package ROCR³⁸ was used to evaluate the performance of the model.

Choice of Machine Learning Classifier

SVM, random forest (RF), generalized linear model (GLM), and Naive Bayes (NB) are the most popular machine learning classifiers in RNA modification prediction and have been widely used for different site predictions.^24,27,35 We evaluated the performance of these algorithms by an independent testing dataset, since the evaluation by cross-validation may over-estimate the performance of models.³⁹ The R package caret was used to construct machine learning models, and all parameters were set by default for primitive evaluation. The results are shown in Table 1. SVM achieved the best results among all four algorithms, which was selected as our final algorithm to build our m⁵U site prediction model.

SVM

SVM is one of the most powerful yet flexible algorithms in bioinformatics research and has been used for many applications (e.g., mammalian microRNA target prediction,⁴⁰ predicting the subcellular location of proteins,⁴¹ and RNA modification site prediction^17,42). The final decision of SVM is determined by a representation of different classes in a hyperplane, which can be used for varied classification and regression tasks. In this study, we used the R package caret to construct our SVM model with a non-linear radial basis function. The prediction performance was evaluated by 5-fold cross-validation and an independent test set. Initially, all parameters were set by default to test the performance of the model, and then tune length was set to 9, with a random grid search for optimization. Random grid search has been proved by a previous study with better efficiency than manually set grid searches.⁴³

Author Contributions

B.S. conceived the idea; K.C. and B.S. collected and processed the data; J.J. and B.S. constructed the predictive model and designed the experiments; Y.T. built the website; and J.J. and B.S. drafted the manuscript. All authors read, critically revised and approved the final manuscript.

Conflicts of Interest

The authors declare no competing interests.