An Interpretable Prediction Model for Identifying N⁷-Methylguanosine Sites Based on XGBoost and SHAP

Abstract

Recent studies have increasingly shown that the chemical modification of mRNA plays an important role in the regulation of gene expression. N⁷-methylguanosine (m7G) is a type of positively-charged mRNA modification that plays an essential role for efficient gene expression and cell viability. However, the research on m7G has received little attention to date. Bioinformatics tools can be applied as auxiliary methods to identify m7G sites in transcriptomes. In this study, we develop a novel interpretable machine learning-based approach termed XG-m7G for the differentiation of m7G sites using the XGBoost algorithm and six different types of sequence-encoding schemes. Both 10-fold and jackknife cross-validation tests indicate that XG-m7G outperforms iRNA-m7G. Moreover, using the powerful SHAP algorithm, this new framework also provides desirable interpretations of the model performance and highlights the most important features for identifying m7G sites. XG-m7G is anticipated to serve as a useful tool and guide for researchers in their future studies of mRNA modification sites.

Graphical Abstract

N⁷-methylguanosine modification of mRNA plays a key role in the regulation of gene expression. In this study, we developed a computational model termed XG-m7G based on XGBoost and SHAP to identify N⁷-methylguanosine sites from sequence information. This will help the discovery of N⁷-methylguansine and its genetic function.

Introduction

Precise regulation of gene expression is vital for the growth and development of organisms in both physiological and pathological processes.¹ Post-translational modification of mRNA was recently found to regulate gene expression. For instance, N⁷-methylguanosine (m7G) is one of the most important mRNA modifications that can be formed during mRNA capping.^2,3 Subsequent experiments have proven that m7G is indispensable for several types of gene processing, including RNA splicing, polyadenylation, and mRNA stability.4, 5, 6, 7, 8 With the development of new experimental technology, such as next-generation sequencing (NGS) and immunoprecipitation sequencing (MeRIP-seq), research data on mRNA translation has rapidly increased.⁹ To date, MODOMICS is a database of RNA modifications that covers more than 160 types of modified ribonucleotides,¹⁰ and these data provide an opportunity to build bioinformatics tools that can identify m7G sites. Using the support vector machine (SVM) classifier, Chen et al.⁸ proposed the first m7G prediction model, iRNA-m7G, by fusing three kinds of features. iRNA-m7G achieved a sensitivity (Sn) of 89.07%, a specificity (Sp) of 90.69%, and a Matthew’s correlation coefficient (MCC) of 0.8 on the jackknife test. In light of the importance of m7G function, we think that it is necessary to further enhance the model performance on m7G sites identification.

Herein, we propose a novel predictor for identifying m7G sites, termed XG-m7G, which applies the extreme gradient boosting (XGBoost) algorithm as the classifier. XG-m7G utilized six types of feature encoders, including binary encoding, composition of k-spaced nucleic acid pairs (CKSNAP), enhanced nucleic acid composition (ENAC), nucleotide chemical property (NCP), nucleotide density (ND), and the series correlation pseudo-dinucleotide composition (SCPseDNC), as its inputs. XGBoost is applied as a classification algorithm to train the model and test its performance. Then, the unified framework SHAP (Shapley additive explanations) is used to interpret predictions,³⁹ rank the feature importance, identify which features are most important, and further select the optimal feature sets. Our benchmarking experiments show that XG-m7G achieved an MCC of 0.825 and 0.839 on 10-fold cross-validation and jackknife tests, respectively, both of which are superior to that of an existing unique model iRNA-m7G.

Results

In this study, we proposed a novel model, XG-m7G, for identifying m7G sites efficiently and accurately from RNA sequences. The performance of XG-m7G was compared with the latest m7G sitesidentification model iRNA-m7G by using both 10-fold cross-validation and jackknife tests. These results indicate that our model outperformed iRNA-m7G in terms of several major metrics, including Sn, Sp, accuracy (Acc), MCC, and area under the receiver operating characteristic (ROC) curve (AUC). Then, we explored the most important features based on a model interpretation method, that is, SHAP, and verified their contribution for identifying m7G sites. In addition, we constructed a web server, which allows interested users to both use our model to identify m7G sites and train their specific models based on their own datasets expediently.

Discussion

XGBoost Outperforms State-of-the-Art Algorithms in m7G Site Prediction

To find the best-performing classification algorithm, four state-of-the-art classifiers, i.e., k-nearest neighbor (KNN),¹¹ SVM,¹² logistic regression (LR),¹³ and random forest (RF),¹⁴ were used to predict m7G sites alongside XGBoost. For each classifier, the important parameters were searched and selected according to the prediction results on 10-fold cross-validation. More specifically, we optimized the neighbors k of KNN; kernel function, C, and gamma of SVM; penalty and solver for LR; and n_estimators of RF. For XGBoost, there were four parameters that need to be considered, which were n_estimators ∈ [10, 100, 1,000], max_depth ∈ [3, 5, 7], learning_rate ∈ [0.1, 0.2, 0.3], and gamma ∈ [0.001, 0.01, 0.1]. The final parameter settings of the machine learning algorithms are provided in Table 1.

Table 1.

Parameter Settings of XGBoost, KNN, SVM, LR, and RF

XGBoost	KNN	SVM	LR	RF
n_estimators = 1,000, max_depth = 3, learning_rate = 0.2, gamma = 0.001	k_neighbors = 4	kernel = “rbf”, C = 10, gamma = 0.02	penalty = “l1”, C = 10, solver = “liblinear”	n_estimators = 10

To minimize the potential effect of randomness on the experimental results, we conducted the 10-fold cross-validation on the five different classifiers by running 100 rounds. Subsequently, the average value of each evaluation metric was calculated and compared. As shown in Figure 1, XGBoost displayed the best performance according to Sp, Acc, MCC, and AUC; however, for the Sn, XGBoost was lower than KNN by about 1.44%. For clarity, we have also listed the win-draw-loss results in Table 2. For each algorithm (KNN, SVM, LR, and RF), “win” means the number of times XGBoost outperformed it, “draw” represents an equivalent performance between the two, and “loss” represents XGBoost being worse. Moreover, the significance of the difference in the prediction results was analyzed by using a Student’s t test. As shown in Table 2, all p values were far less than 0.01, indicating that there was a statistically significant difference between XGBoost and the other four algorithms.

Figure 1.

Average Results of Five Classification Algorithms after 10-Fold Cross Validation Running for 100 Rounds

Table 2.

“Win-Draw-Loss” Results for XGBoost Compared with Other Classifiers

Classifier	KNN	SVM	LR	RF
Sn	3-0-97	96-0-4	100-0-0	100-0-0
p value	5.6103E−34	6.6038E−32	4.0955E−59	1.3241E−80
Sp	100-0-0	100-0-0	100-0-0	100-0-0
p value	3.5946E−95	2.2062E−40	6.5396E−71	7.9065E−58
Acc	100-0-0	100-0-0	100-0-0	100-0-0
p value	8.8436E−72	2.1478E−47	5.2734E−79	4.2097E−84
MCC	100-0-0	100-0-0	100-0-0	100-0-0
p value	8.4909E−71	6.9919E−79	4.2602E−84	2.9426E−47
AUC	100-0-0	100-0-0	100-0-0	100-0-0
p value	1.6011E−131	1.5336E−70	6.9470E−89	1.3920E−112

In addition, for each classifier, the largest AUC achieved in the 100 rounds of 10-fold cross-validation was recorded, and its corresponding ROC curve is shown in Figure 2. Again, XGBoost outperformed the other four algorithms on m7G site prediction, and it achieved the highest AUC of 0.965.

Figure 2.

ROC Curves of the Five Classification Algorithms

The Effect of Feature Encoding on Model Prediction

In this study, six different feature-encoding schemes were used to generate the feature vectors. The performance of each type of feature is listed in Table S1. Afterward, we used the SHAP algorithm to characterize feature importance and assess feature behavior in our samples. For convenience, we named all features as follows: binary-1, binary-2, …, binary-164; CKSNAP-165, CKSNAP-166, …, CKSNAP-212, …; and SCPseDNC-537, SCPseDNC-538, …, SCPseDNC-672. According to Equation 15, SHAP values were calculated and the top 20 features for all samples are plotted in Figure 3.

Figure 3.

Top 20 Features Sorted by SHAP

In Figure 3A, each row represents a feature, and each point is the SHAP value of an instance. Redder sample points indicate that the value of the feature is larger, and bluer sample points indicate that the value of the feature is smaller; the abscissae represent the SHAP values. For clarity, the area in which feature NCP-466 is located has been magnified in Figure 3B. If the SHAP value is positive, this means that the feature drives the predictions toward m7G sites and has a positive effect; if negative, the feature drives the predictions toward non-m7G sites and has a negative effect. We observed that when features such as NCP-432 and NCP-438 take high SHAP values, the model is driven toward positive m7G prediction. Conversely, when these features take low SHAP values, the model is driven toward non-m7G prediction. We also observed that the features SCPseDNC-546, SCPseDNC-543, SCPseDNC-567, and SCPseDNC-579 only promote the prediction of non-m7G sites, while other features, such as SCPseDNC-541, SCPseDNC-538, SCPseDNC-599, and SCPseDNC-549, only promote the prediction of m7G sites by the model.

In order to further explore the contribution of the top 20 features sorted by SHAP to the model performance, we retrained the model without these features and evaluated the performance of the resulting model on jackknife tests. Figure 4 shows the comparison with and without these top 20 features. We also found that selecting a different number of features would influence the model performance. Therefore, to further improve our model performance, we took the mean value of the absolute values of SHAP for each feature to rank the importance of features. Afterward, we designed and implemented a series of experiments. The feature ranking results were exported and are listed in the Table S2. In addition to the k-fold cross-validation test, the jackknife test is also frequently used as a cross-validation method in statistical prediction.¹⁵ For a given benchmark dataset, the jackknife test can always yield a unique outcome.¹⁵ Next, we kept the top 50, 100, 150, 200, 250, 300, 350, 400, 450, 500, 550, and 600 features according to the sorting to find the best feature combination by performing jackknife tests. All of the performance results are listed in Table S3. We found that the model achieved the overall best performance when the feature dimension was reduced to 150. To illustrate the effectiveness of feature selection, we provided the performance comparison results of the models trained using the original features and the optimal features on jackknife tests in Table S4 and Figure 4. The results also showed that the model trained using the selected optimal features by SHAP clearly achieved an improve predictive performance compared with the model trained using all of the original features.

Figure 4.

Comparison of Models Trained Using Different Dimensional Features

In addition, we also conducted the same feature ranking and selection processes using the F-score and minimum redundancy-maximum relevance (mRMR), which has been extensively adopted to reduce the feature dimension in the fields of bioinformatics and computational biology.^16,17 Figure 5 provides the performance comparison of these three feature selection technologies. As can be seen, the SHAP curve was above the F-score curve and the mRMR curve. More specifically, after sorting by the F-score, the combination of the top 400 features reached the best Acc of 91.36%. In comparison, after the ranking by mRMR, the combination of the top 600 features reached the best Acc of 90.35%. These results indicate the effectiveness of SHAP for identifying m7G sites.

Figure 5.

Comparison of SHAP, F-score, and mRMR Dimensionality Reduction Methods on Jackknife Tests

Comparison with iRNA-m7G

To the best of our knowledge, iRNA-m7G is the only model established for searching m7G sites in RNA sequences. Therefore, we compared the performance of XG-m7G with iRNA-m7G by a 10-fold cross-validation test. Table 3 lists the performance comparison of XG-m7G and iRNA-m7G in terms of Sn, Sp, Acc, MCC, and AUC values. Clearly, our proposed XG-m7G method achieved a better performance than iRNA-m7G in terms of four evaluation metrics. Specifically, XG-m7G achieved an improvement of 2.82% and 1.41% for Sn and Acc, respectively. The MCC and AUC of XG-m7G were 0.025 and 0.026, respectively, higher than those of iRNA-m7G. Consequently, a jackknife test was also applied to estimate the performance of XG-m7G. XG-m7G obtained Sn of 92.17%, Sp of 91.77%, Acc of 91.97%, MCC of 0.839, and AUC of 0.972, demonstrating its superior performance to iRNA-m7G. Altogether, these results confirm that our proposed model XG-m7G outperforms iRNA-m7G in identifying m7G sites.

Table 3.

Predictive Performance Comparison of XG-m7G with iRNA-m7G by 10-Fold Cross-Validation and Jackknife Test

Cross-Validation Test	Methods	Sn (%)	Sp (%)	Acc (%)	MCC	AUC
10-Fold	iRNA-m7G	88.66	90.96	89.81	0.800	0.946
	XG-m7G	91.48	90.96	91.22	0.825	0.972
Jackknife	iRNA-m7G	89.07	90.69	89.88	0.800	–
	XG-m7G	92.17	91.77	91.97	0.839	0.972

Implementation of the XG-m7G Web Server

We developed a web server for XG-m7G to perform convenient prediction and analyses of distinctive m7G sites, which is freely available at http://flagship.erc.monash.edu/XG-m7G/. The XG-m7G server was implemented using PHP, HTML, CSS, JavaScript, and Python running Apache2 and configured in the Linux environment on an eight-core server machine with 32 gigabytes (GB) of memory and three hard disks with a total of 1.25 terabytes (TB) of memory. The server requires users to paste the sequences or upload a text file in the FASTA format as the input. Figure 6A illustrates an instance of the prediction steps. As shown, the prediction results are output in probability ranking and they are capable of being viewed in ascending or descending order. Furthermore, we designed a practical function “train model” to allow users to train their own models with their training data as shown in Figure 6B. The computational time needed for the testing task is determined by the number and total length of sequences provided. For 100 sequences with 41 nt residues each, the training task will be accomplished in a few seconds. We hope that this function can provide a theoretical and useful reference for interested researchers.

Figure 6.

Instructions for Using the XG-m7G Web Server

(A) Example of the “prediction” function of the XG-m7G web server. (B) Example of the “train model” function of the XG-m7G web server

Materials and Methods

Overall Framework

The overall framework of XG-m7G is illustrated in Figure 7. As shown, there were five major steps in the development of XG-m7G. First, we collected benchmark datasets for m7G sites. Second, these sequences were transformed into numeric vectors by several feature-encoding methods. Third, we analyzed and interpreted the feature effect with SHAP on XGBoost, and then the most important features were selected and identified. Fourth, we evaluated the model performance and, lastly, we developed a web server for XG-m7G.

Figure 7.

Overall Framework of XG-m7G

Benchmark Datasets

In this study, the benchmark datasets for training and evaluating our XG-m7G model were collected from Chen et al.⁸ Chen et al. obtained 801 m7G site-containing sequences by mapping the 801 base-resolution m7G sites in human HeLa and HepG2 cells that had been detected by Zhang et al.⁹ These sequences, with the m7G sites in the center of the sequences, were extracted to 41 bp, with 20 bp upstream and 20 bp downstream. CD-HIT¹⁸ was then used to remove redundant sequences¹⁹ using a threshold of 80%, after which 741 positive samples containing m7G sites were retained. The negative samples were first collected from those 41-bp-long sequences with the intermediate guanosine that have not been detected as m7G sites (namely non-m7G site) by the MeRIP-seq method. In order to further avoid the potential problem of low Sn caused by the unbalanced data, 741 such sequences containing non-m7G sites with the sequence similarity of less than 80% were selected to constitute the final negative sample dataset.

Feature Extraction

One of the critical steps is to encode each RNA fragment as a numerical vector. In this study, six types of feature encoding schemes were applied to establish the predictor, including binary encoding, CKSNAP, ENAC, NCP, ND, and SCPseDNC. These encoding schemes have been extensively used for identification of pseudouridine sites,²⁰ prediction of citrullination sites,²¹ and prediction of m6A sites^22,23 with demonstrated performance. It is noteworthy that all features used in this study are available and can be calculated using BioSeq-Analysis2.0²⁴ and iLearn.²⁵ A detailed description of these feature-encoding schemes is provided in the following sections.

Binary Encoding

In the binary feature encoding scheme, each nucleotide is represented by a four-dimensional binary vector, e.g., A is coded as (1, 0, 0, 0), C is coded as (0, 1, 0, 0), G is coded as (0, 0, 1, 0), and U is coded as (0, 0, 0, 1). Therefore, each sample has a total of 164 binary features.

CKSNAP

The CKSNAP encoding scheme represents the frequencies of nucleotide pairs separated by k residues. The CKSNAP feature contains 16 values corresponding to pairs of nucleic acids: {AA, AC, AG, ..., UG, UU}. Taking k = 1 as an example, CKSNAP can be given as follows:

$$\mathit{V}\mathit{=}\left[\frac{{\mathit{N}}_{\mathit{A}\mathit{\ast }\mathit{A}}}{{\mathit{N}}_{\mathit{T}\mathit{o}\mathit{t}\mathit{a}\mathit{l}}}\mathit{,}\frac{{\mathit{N}}_{\mathit{A}\mathit{\ast }\mathit{C}}}{{\mathit{N}}_{\mathit{T}\mathit{o}\mathit{t}\mathit{a}\mathit{l}}}\mathit{,}\frac{{\mathit{N}}_{\mathit{A}\mathit{\ast }\mathit{G}}}{{\mathit{N}}_{\mathit{T}\mathit{o}\mathit{t}\mathit{a}\mathit{l}}}\mathit{,}\mathit{\dots }\mathit{,}\frac{{\mathit{N}}_{\mathit{U}\mathit{\ast }\mathit{U}}}{{\mathit{N}}_{\mathit{T}\mathit{o}\mathit{t}\mathit{a}\mathit{l}}}\right]\mathit{,}$$ (1)

where ∗ represents any nucleotide of A, C, G, and U, $N_{X\ast Y}$ represents the number of nucleic acid pairs X∗Y that occur in the sequence, and $N_{Total}$ represents the total number of one-spaced nucleic acid pairs in the sequence. In this study, k = 0, 1, and 2, and the corresponding dimension of CKSNAP features was 48.

ENAC

ENAC encoding calculates the nucleic acid composition based on a fixed-length window, which continuously slides from the 5′ to 3′ terminus of each nucleotide sequence.²⁵ This method is usually applied to encode nucleotide sequences of equal length. ENAC can be calculated as follows:

$$\mathit{V}\mathit{=}\left[\frac{{\mathit{N}}_{\mathit{A}\mathit{,}\mathit{w}\mathit{i}{\mathit{n}}_{\mathit{1}}}}{\mathit{S}}\mathit{,}\frac{{\mathit{N}}_{\mathit{C}\mathit{,}\mathit{w}\mathit{i}{\mathit{n}}_{\mathit{1}}}}{\mathit{S}}\mathit{,}\frac{{\mathit{N}}_{\mathit{G}\mathit{,}\mathit{w}\mathit{i}{\mathit{n}}_{\mathit{1}}}}{\mathit{S}}\mathit{,}\frac{{\mathit{N}}_{\mathit{U}\mathit{,}\mathit{w}\mathit{i}{\mathit{n}}_{\mathit{1}}}}{\mathit{S}}\mathit{,}\frac{{\mathit{N}}_{\mathit{A}\mathit{,}\mathit{w}\mathit{i}{\mathit{n}}_{\mathit{2}}}}{\mathit{S}}\mathit{,}\mathit{\dots }\mathit{,}\frac{{\mathit{N}}_{\mathit{G}\mathit{,}\mathit{w}\mathit{i}{\mathit{n}}_{\mathit{L}\mathit{-}\mathit{S}\mathit{+}\mathit{1}}}}{\mathit{S}}\mathit{,}\frac{{\mathit{N}}_{\mathit{U}\mathit{,}\mathit{w}\mathit{i}{\mathit{n}}_{\mathit{L}\mathit{-}\mathit{S}\mathit{+}\mathit{1}}}}{\mathit{S}}\right]\mathit{,}$$ (2)

where S represents the size of the sliding window, $N_{t,wi{n}_r}$ represents the number of nucleic acids t in the sliding window r,$\mathit{t}\in \left\{\mathit{A},\mathit{C},\mathit{G},\mathit{U}\right\}$, and $r\mathit{=}\mathit{1}\mathit{,}\mathit{2}\mathit{,}\mathit{\dots }\mathit{,}\mathit{L}\mathit{-}\mathit{S}\mathit{+}\mathit{1}$. In this study, the size of the sliding window was 2, and the corresponding feature dimension was 160.

NCP and ND

Each nucleotide of A, C, G, and U has a different chemical structure and chemical binding feature. The NCP encoding scheme incorporates the chemical properties of these four types of nucleotides into the representation. The nucleotides can be classified into three different groups²⁶ as follows:

$$N_i=\left(x_i,y_i,z_i\right),$$ (3)

where

$$x_{\mathit{i}}\mathit{=}\mathit{\{}\begin{array}{c}\mathit{1}\mathit{,}\mathit{i}\mathit{f}{\mathit{N}}_{\mathit{i}}\mathit{\in }\left\{\mathit{A}\mathit{,}\mathit{G}\right\}\\ \mathit{0}\mathit{,}\mathit{i}\mathit{f}{\mathit{N}}_{\mathit{i}}\mathit{\in }\left\{\mathit{C}\mathit{,}\mathit{U}\right\}\end{array}\mathit{;}{y}_{\mathit{i}}\mathit{=}\mathit{\{}\begin{array}{c}\mathit{1}\mathit{,}\mathit{i}\mathit{f}{\mathit{N}}_{\mathit{i}}\mathit{\in }\left\{\mathit{A}\mathit{,}\mathit{C}\right\}\\ \mathit{0}\mathit{,}\mathit{i}\mathit{f}{\mathit{N}}_{\mathit{i}}\mathit{\in }\left\{\mathit{G}\mathit{,}\mathit{U}\right\}\end{array}\mathit{;}{z}_{\mathit{i}}\mathit{=}\mathit{\{}\begin{array}{c}\mathit{1}\mathit{,}\mathit{i}\mathit{f}{\mathit{N}}_{\mathit{i}}\mathit{\in }\left\{\mathit{A}\mathit{,}\mathit{U}\right\}\\ \mathit{0}\mathit{,}\mathit{i}\mathit{f}{\mathit{N}}_{\mathit{i}}\mathit{\in }\left\{\mathit{C}\mathit{,}\mathit{G}\right\}\end{array}.$$ (4)

Based on these three chemical properties, the nucleotide A is mapped to the numeric vector $\left(\mathrm{1,1,1}\right)$, C is mapped to $\left(\mathrm{0,1,0}\right)$, G is mapped to $\left(\mathrm{1,0,0}\right)$, and U is mapped to $\left(\mathrm{0,0,1}\right)$.

ND is also termed accumulated nucleotide frequency (ANF),²⁵ which integrates the nucleotide frequency information and the distribution of each nucleotide in the RNA sequence. The density $d_i$ of any nucleotide $N_j$ at the position i in an RNA sequence can be given by the following formula:

$$d_{\mathit{i}}\mathit{=}\frac{\mathit{1}}{\mathit{\Vert }{\mathit{S}}_{\mathit{i}}\mathit{\Vert }}\mathit{\sum }_{\mathit{j}\mathit{=}\mathit{1}}^{\mathit{l}}f\left({\mathit{N}}_{\mathit{j}}\right)\mathit{,}f\left({\mathit{N}}_{\mathit{j}}\right)\mathit{=}\mathit{\{}\begin{array}{l}\mathit{1}\mathit{i}\mathit{f}{\mathit{N}}_{\mathit{j}}\mathit{i}\mathit{s}\mathit{t}\mathit{h}\mathit{e}\mathit{n}\mathit{u}\mathit{c}\mathit{l}\mathit{e}\mathit{o}\mathit{t}\mathit{i}\mathit{d}\mathit{e}\mathit{c}\mathit{o}\mathit{n}\mathit{c}\mathit{e}\mathit{r}\mathit{n}\mathit{e}\mathit{d}\\ \mathit{0}\mathit{o}\mathit{t}\mathit{h}\mathit{e}\mathit{r}\mathit{w}\mathit{i}\mathit{s}\mathit{e}\end{array}\mathit{,}$$ (5)

where $\Vert S_i\Vert$ is the length of the sliding substring concerned, while l is the corresponding locator’s sequence position.

The NCP and ND encodings are often used together to take into account both chemical property and long-range sequence order information.²⁰ As an example, the sequence fragment ACGCGGAUUA can be represented as {(1, 1, 1, 1), (0, 1, 0, 0.5), (1, 0, 0, 0.33), (0, 1, 0, 0.5), (1, 0, 0, 0.4), (1, 0, 0, 0.5), (1, 1, 1, 0.29), (0, 0, 1, 0.125), (0, 0, 1, 0.22), (1, 1, 1, 0.3)}. Each sample has 164 NCP and ND features.

SCPseDNC

The SCPseDNC encoding²⁷ is defined as follows:

$$\mathit{V}\mathit{=}{\left[{\mathit{d}}_{\mathit{1}}\mathit{,}{\mathit{d}}_{\mathit{2}}\mathit{,}\mathit{\dots }\mathit{,}{\mathit{d}}_{\mathit{16}}\mathit{,}{\mathit{d}}_{\mathit{16}\mathit{+}\mathit{1}}\mathit{,}\mathit{\dots }\mathit{,}{\mathit{d}}_{\mathit{16}\mathit{+}\mathit{\lambda }}\mathit{,}{\mathit{d}}_{\mathit{16}\mathit{+}\mathit{\lambda }\mathit{+}\mathit{1}}\mathit{,}\mathit{\dots }\mathit{,}{\mathit{d}}_{\mathit{16}\mathit{+}\mathit{\lambda }\mathit{\Lambda }}\right]}^{\mathit{T}}\mathit{,}$$ (6)

where

$$d_{\mathit{u}}\mathit{=}\mathit{\{}\begin{array}{l}\frac{{\mathit{f}}_{\mathit{u}}}{{\mathit{\sum }}_{\mathit{i}\mathit{=}\mathit{1}}^{\mathit{16}}{\mathit{f}}_{\mathit{i}}\mathit{+}\mathit{w}{\mathit{\sum }}_{\mathit{j}\mathit{=}\mathit{1}}^{\mathit{\lambda }\mathit{\Lambda }}{\mathit{\theta }}_{\mathit{j}}}\mathit{,}\left(\mathit{1}\mathit{\le }\mathit{u}\mathit{\le }\mathit{16}\right)\\ \frac{\mathit{w}{\mathit{\theta }}_{\mathit{u}\mathit{-}\mathit{16}}}{{\mathit{\sum }}_{\mathit{i}\mathit{=}\mathit{1}}^{\mathit{16}}{\mathit{f}}_{\mathit{i}}\mathit{+}\mathit{w}{\mathit{\sum }}_{\mathit{j}\mathit{=}\mathit{1}}^{\mathit{\lambda }\mathit{\Lambda }}{\mathit{\theta }}_{\mathit{j}}}\mathit{,}\left(\mathit{16}\mathit{\le }\mathit{u}\mathit{\le }\mathit{16}\mathit{+}\mathit{\lambda }\mathit{\Lambda }\right)\end{array}\mathit{,}$$ (7)

where $f_u$ for $u=1,2,\dots ,16$ is the normalized occurrence frequency of the i-th dinucleotide in the sequence, w is the weight factor ranging from 0 to 1, and Λ is the number of physicochemical indices. Six indices (i.e., Rise (RNA), Roll (RNA), Shift (RNA), Slide (RNA), Tilt (RNA), Twist (RNA)) were set as the default indices for RNA sequences. ${\theta }_j$ ($j=1,2,\dots ,\lambda$) is the j-tier correlation factor, defined as:

$$\mathit{\{}\begin{array}{l}{\mathit{\theta }}_{\mathit{1}}\mathit{=}\frac{\mathit{1}}{\mathit{L}\mathit{-}\mathit{3}}\mathit{\sum }_{\mathit{i}\mathit{=}\mathit{1}}^{\mathit{L}\mathit{-}\mathit{3}}{\mathit{J}}_{\mathit{i}\mathit{,}\mathit{i}\mathit{+}\mathit{1}}^{\mathit{1}}\\ {\mathit{\theta }}_{\mathit{2}}\mathit{=}\frac{\mathit{1}}{\mathit{L}\mathit{-}\mathit{3}}\mathit{\sum }_{\mathit{i}\mathit{=}\mathit{1}}^{\mathit{L}\mathit{-}\mathit{3}}{\mathit{J}}_{\mathit{i}\mathit{,}\mathit{i}\mathit{+}\mathit{1}}^{\mathit{2}}\\ \mathit{\dots }\\ {\mathit{\theta }}_{\mathit{\Lambda }}\mathit{=}\frac{\mathit{1}}{\mathit{L}\mathit{-}\mathit{3}}\mathit{\sum }_{\mathit{i}\mathit{=}\mathit{1}}^{\mathit{L}\mathit{-}\mathit{3}}{\mathit{J}}_{\mathit{i}\mathit{,}\mathit{i}\mathit{+}\mathit{1}}^{\mathit{\Lambda }}\\ \mathit{\dots }\\ {\mathit{\theta }}_{\mathit{\lambda \Lambda }\mathit{-}\mathit{1}}\mathit{=}\frac{\mathit{1}}{\mathit{L}\mathit{-}\mathit{\lambda }\mathit{-}\mathit{2}}\mathit{\sum }_{\mathit{i}\mathit{=}\mathit{1}}^{\mathit{L}\mathit{-}\mathit{\lambda }\mathit{-}\mathit{2}}{\mathit{J}}_{\mathit{i}\mathit{,}\mathit{i}\mathit{+}\mathit{\lambda }}^{\mathit{\Lambda }\mathit{-}\mathit{1}}\\ {\mathit{\theta }}_{\mathit{\lambda \Lambda }}\mathit{=}\frac{\mathit{1}}{\mathit{L}\mathit{-}\mathit{\lambda }\mathit{-}\mathit{2}}\mathit{\sum }_{\mathit{i}\mathit{=}\mathit{1}}^{\mathit{L}\mathit{-}\mathit{\lambda }\mathit{-}\mathit{2}}{\mathit{J}}_{\mathit{i}\mathit{,}\mathit{i}\mathit{+}\mathit{\lambda }}^{\mathit{\Lambda }}\end{array}\left(\mathit{\lambda }\mathit{<}\mathit{L}\mathit{-}\mathit{2}\right)\mathit{,}$$ (8)

Where λ represents the highest counted rank (or tier) of the correlation along the nucleotide sequence, and the correlation function is:

$$J_{i,i+m}^u=P_u\left(R_i{R}_{i+1}\right)\cdot P_u\left(R_{i+m}{R}_{i+m+1}\right),$$ (9)

where $u=1,2,\dots ,\mathit{\Lambda };m=1,2,\dots ,\lambda ;i=\mathrm{1,2},\dots ,L-m-2$, and $P_u\left(R_i{R}_{i+1}\right)$ is the numerical value of the u-th (u = 1,2, …,$\mathit{\Lambda }$) physicochemical index of the dinucleotide $R_i{R}_{i+1}$ at position i, and $P_u\left(R_j{R}_{j+1}\right)$ represents the corresponding value of the dinucleotide $R_j{R}_{j+1}$ at position j. By setting $\lambda$ = 20 and w = 0.9, we generated a 136-dimensional vector.

Machine Learning Algorithm

XGBoost^28,29 is a type of optimized distributed gradient boosting algorithm. The principle of the XGBoost algorithm can be summarized as follows:

Assume a training dataset $\mathit{D}=\left\{\left(x_i,y_i\right),i=1,...,n\right\}$ of the size n, where $x_i=(x_{i1},x_{i2},\dots ,x_{im})$ denotes an m-dimensional feature vector with the corresponding (output) category $y_i$:

$$\stackrel{\mathit{ˆ}}{{\mathit{y}}_{\mathit{i}}}\mathit{=}\mathit{\sum }_{\mathit{k}\mathit{=}\mathit{1}}^{\mathit{K}}{f}_{\mathit{k}}\left({\mathit{x}}_{\mathit{i}}\right)\mathit{,}{f}_{\mathit{k}}\mathit{\in }F\mathit{,}$$ (10)

where K represents the number of trees, $f_k\left(x_i\right)$ represents the score that is associated with the model’s k-th tree, and F denotes the space of scoring functions available for all boosting trees.

Differing from another tree-based algorithm, GBDT (gradient boosting decision tree), XGBoost uses the second-order Taylor expansion to approximate the loss function, and it is more efficient in avoiding the over-fitting issue mainly by adding regularization terms to the objective function. For more details about XGBoost, please refer to Chen and Guestrin.²⁵ In recent years, XGBoost has been extensively utilized in bioinformatics and computational biology for addressing a range of challenging tasks, such as pseudouridine site identification,³⁰ on-target activity prediction of single guide RNAs (sgRNAs),³¹ recognition of internal ribosome entry sites,³² and so on. In this study, we applied XGBoost to develop the m7G prediction model. Our results demonstrated that XGBoost achieved a better predictive performance than did other machine learning algorithms for m7G site prediction.

Evaluation Metrics

To evaluate the prediction performance of XG-m7G we used four metrics, that is, Sn, Sp, Acc, and MCC, which have previously been used to assess the performance of predictors in other studies.^33,34 We also used ROC curves,35, 36, 37, 38 which plot the true-positive rate against the false-positive rate, and AUC to further assess the model performance. Sn, Sp, Acc, and MCC are defined as follows:

$$Sn=1-\frac{N_{-}^{+}}{N^{+}},$$ (11)

$$Sp=1-\frac{N_{+}^{-}}{N^{-}},$$ (12)

$$Acc=1-\frac{N_{-}^{+}+N_{+}^{-}}{N^{+}+N^{-}},\mathrm{a}\mathrm{n}\mathrm{d}$$ (13)

$$\text{M}CC=\frac{1-\frac{N_{-}^{+}+N_{+}^{-}}{N^{+}+N^{-}}}{\sqrt{\left(1+\frac{N_{+}^{-}-N_{-}^{+}}{N^{+}}\right)\left(1+\frac{N_{-}^{+}-N_{+}^{-}}{N^{-}}\right)}},$$ (14)

where $N^{+}$ represents the total number of m7G site-containing sequences, $N^{-}$ represents the total number of non-m7G sequences, $N_{-}^{+}$ represents the number of m7G site-containing sequences incorrectly predicted as non-m7G sequences, and $N_{+}^{-}$ represents the number of non-m7G sequences incorrectly predicted as m7G site-containing sequences.

Notably, these metrics are also used in the existing method iRNA-m7G for identification of m7G sites. Therefore, it is convenient to make a fair and credible comparison of XG-m7G and iRNA-m7G.

SHAP

SHAP is a unified framework for interpreting predictions, proposed in 2017 as the only consistent and locally accurate feature attribution method based on expectations.³⁹ This technique can interpret feature importance scores from complex training models, and it provides an interpretable prediction for a test sample. SHAP values have been proposed as a unified measure of feature importance, as they assign an importance value (${\phi }_i$) to each feature representing the effect of including that feature in model prediction. In cooperative game theory, SHAP values can be computed as follows:

$${\phi }_{\mathit{i}}\mathit{=}\mathit{\sum }_{\mathit{S}\mathit{\subseteq }\mathit{F}\mathit{,}\left\{\mathit{i}\right\}}\frac{\mathit{\left|}\mathit{S}\mathit{\right|}\mathit{!}\left(\mathit{\left|}\mathit{F}\mathit{\right|}\mathit{-}\mathit{\left|}\mathit{S}\mathit{\right|}\mathit{-}\mathit{1}\right)\mathit{!}}{\mathit{\left|}\mathit{F}\mathit{\right|}\mathit{!}}\left[{\mathit{f}}_{\mathit{S}\mathit{\cup }\left\{\mathit{i}\right\}}\left({\mathit{x}}_{\mathit{S}\mathit{\cup }\left\{\mathit{i}\right\}}\right)\mathit{-}{\mathit{f}}_{\mathit{S}}\left({\mathit{x}}_{\mathit{S}}\right)\right]$$ (15)

where $F$ represents the set of all features and $S$ represents all feature subsets obtained from $F$ after removing the $i^{th}$ feature. Then, two models, $f_{S\cup \left\{i\right\}}$ and $f_S$, are retrained, and predictions of these two models are compared to the current input $f_{S\cup \left\{i\right\}}\left(x_{S\cup \left\{i\right\}}\right)-f_S\left(x_S\right)$, where $x_S$ represents the values of the input features in the set $S$. To estimate ${\phi }_i$ from $2^{\left|F\right|}$ differences, the SHAP approach approximates the Shapley value by either performing Shapley sampling or Shapley quantitative influence.

Author Contributions

C.J., J.S., and F.L. conceived the initial idea and designed the methodology. F.L. and Y.B. implemented the algorithm, conducted the experiments, and processed the results. F.L., Y.B., D.X., and Z.G. developed the web server. All authors drafted, revised, and approved the final manuscript.

Conflicts of Interest

The authors declare no competing interests.