Selecting Essential MicroRNAs Using a Novel Voting Method

Abstract

Among the large number of known microRNAs (miRNAs), some miRNAs play negligible roles in cell regulation. Therefore, selecting essential miRNAs is an important initial step for a deeper understanding of miRNAs and their functions. In this study, we generated 60 classification models by combining 12 representative feature extraction methods and 5 commonly used classification algorithms. The optimal model for essential miRNA classification that we obtained is based on the Mismatch feature extraction method combined with the random forest algorithm. The F-Measure, area under the curve, and accuracy values of this model were 93.2%, 96.7%, and 93.0%, respectively. We also found that the distribution of the positive and negative examples of the first few features greatly influenced the classification results. The feature extraction methods performed best when the differences between the positive and negative examples were obvious, and this led to better classification of essential miRNAs. Because each classifier’s predictions for the same sample may be different, we employed a novel voting method to improve the accuracy of the classification of essential miRNAs. The performance results showed that the best classification results were obtained when five classification models were used in the voting. The five classification models were constructed based on the Mismatch, pseudo-distance structure status pair composition, Subsequence, Kmer, and Triplet feature extraction methods. The voting result was 95.3%. Our results suggest that the voting method can be an important tool for selecting essential miRNAs.

Introduction

MicroRNAs (miRNAs) are short noncoding RNAs that are found widely in eukaryotes.¹ Their breadth and diversity indicate that they have a very wide variety of biological functions. They are involved in many important biological processes in cells, including regulating the expression of genes that encode proteins involved in biological development,2, 3, 4 cell proliferation,⁵ differentiation,⁶ and apoptosis.⁷ miRNAs are associated with cancer8, 9, 10 and other diseases.11, 12, 13, 14, 15 Drugs that target genes have been developed based on miRNA gene silencing and have been applied to some previously incurable diseases that threaten human health.16, 17, 18, 19, 20 miRNAs also play important roles in cell adaptation to abnormal environments, such as freezing, dehydration, and hypoxia.21, 22, 23 Because of the many biological functions of miRNAs, a lot of attention has been given to miRNA-related problems in bioinformatics.24, 25, 26, 27, 28, 29, 30

Accurate identification of miRNA sequences is one such problem that has achieved good results. For example, in 2013, Wei et al.³¹ constructed a classifier to identify miRNAs using a high-quality negative set and reported a classification accuracy rate of 93%. In 2015, Peace et al.¹ proposed a framework for improving miRNA prediction in non-human genomes using sequence conservation and phylogenetic distance information. Their framework uses accuracy, sensitivity, and specificity parameters to obtain species-specific predictions. In 2016, Jiang et al.⁵ used a backpropagation neural network algorithm to identify miRNAs in Arabidopsis. In their model, the precision and recall rates were 95% and 96%, respectively; however, these results do not make much sense for the in-depth study of miRNAs. The reasons for this failure were likely because of the recent dramatic increase in known miRNAs (e.g., miRBase [Release 22.1: October 2018] contains 38,589 miRNA sequences from 271 species³²) and the proposal that some miRNAs or miRNA families have negligible effects in cell development.³³ Therefore, to efficiently study the biological mechanisms of miRNAs, it is necessary to detect essential miRNAs from among the many other miRNAs.

Two important factors that influence miRNA prediction results are the feature extraction method and classification algorithm selected. A good feature extraction method will fully express the sequence information. The existing methods for RNA feature extraction can be divided into four categories: those based on ribonucleic acid composition, autocorrelation, pseudo ribonucleic acid composition, or predicted structure composition.34, 35 Methods based on RNA sequence composition include basic kmer (Kmer), Mismatch, and Subsequence.31, 36, 37, 38, 39 Kmer³¹ represents RNA sequences as the frequency of occurrence of k adjacent bases and is the simplest of the three methods. Methods based on autocorrelation include dinucleotide-based auto-covariance (DAC),⁴⁰ dinucleotide-based cross-covariance (DCC),⁴¹ dinucleotide-based auto-cross-covariance (DACC; a combination of DAC and DCC),⁴² Moran autocorrelation (MAC),⁴³ Geary autocorrelation (GAC),⁴⁴ and normalized Moreau-Broto autocorrelation (NMBAC).⁴⁵ Methods based on the pseudo-RNA composition46, 47 include general parallel correlation pseudo-dinucleotide composition (PC-PseDNC-General) and its variant general series correlation pseudo-dinucleotide composition (SC-PseDNC-General). The equations used in PC-PseDNC-General and SC-PseDNC-General differ in that they calculate the correlation factors that reflect the sequence or the order correlations, respectively, among all of the consecutive dinucleotides along an RNA sequence.⁴² Methods based on the predicted structure composition include local structure-sequence triplet elements (Triplet),48, 49, 50 pseudo-structure status composition (PseSSC),²⁶ and pseudo-distance structure status pair composition (PseDPC).¹³

The most commonly used classification algorithms are random forest and support vector machine. Random forest51, 52, 53, 54, 55, 56, 57 can be considered an integrated algorithm that reduces the one-sidedness and inaccuracy of a single decision tree by combining multiple different decision trees. Support vector machine13, 48, 58, 59, 60, 61, 62, 63, 64, 65, 66 maximizes the classification of positive and negative examples by constructing a hyperplane. Other machine learning algorithms also have been used for classification and recognition, such as neural networks,5, 67, 68 Naive Bayes,69, 70 evolutionary algorithms,⁷¹ and ensemble learning.72, 73, 74, 75, 76

The aims of this study were: (1) to construct a classification model by combining 12 different feature extraction algorithms and 5 classification algorithms to find the most suitable model for essential miRNA classification; (2) to explore the distribution of positive and negative examples under different feature extraction methods, and to determine the influence of distribution differences between positive and negative examples on classification results; and (3) to further improve the classification accuracy of essential miRNAs through a novel voting method. The performance of the optimal classification model shows the validity of our conclusions and methods.

Results and Discussion

Determine the Parameters for Kmer, Mismatch, and Subsequence

For these three feature extraction algorithms, the parameter k, which has 4^k-dimensional features, has to be set. For k = 1, the extracted features do not represent the complete sequence information. For k > 5, the extracted features will have more than 1,024 dimensions. When the dimensions are very high, the computational time can be very long, and over-fitting phenomenon and dimensionality disaster may occur. To avoid these problems, we set k = 2, 3, and 4. The results for each of the methods on the pre-miRNA dataset are shown in Table 1. Each performance value was taken from the best classification model under each method.

Table 1.

Performances of the Three Feature Extraction Methods with Different k Values

	Kmer			Mismatch			Subsequence
K Values	Sn	Sp	ACC	Sn	Sp	ACC	Sn	Sp	ACC
2	67.0	80.6	73.9	96.4	89.7	93.0	91.7	88.6	90.1
3	76.4	85.2	80.9	94.1	93.1	93.6	96.4	87.5	90.7
4	83.5	90.9	87.2	89.4	92.0	90.7	90.5	88.6	89.5

ACC, accuracy; Sn, sensitivity; Sp, specificity.

In the Kmer-based model, the performance was best for k = 4. In the models that used the Mismatch and Subsequence feature extraction methods, all three k values produced similar results that were better than the classification results obtained with the Kmer-based model (Table 1). Smaller k values will require a shorter computational time, so k = 2 was selected as the best value for the Mismatch- and Subsequence-based models.

Selection of the Best Classification Model for Predicting Essential miRNAs

A total of 12 feature extraction methods were used in this study. The models based on Kmer, Mismatch, and Subsequence involve parameter setting, as described in Determine the Parameters for Kmer, Mismatch, and Subsequence. We combined the 12 feature extraction methods with 5 commonly used classification algorithms to obtain 60 classification models. The accuracy of these models on the pre-miRNA dataset is shown in Figure 1.

Figure 1.

Accuracy of All the Classification Models

The accuracy of each of the classification models varied depending on the feature extraction method that was used (Figure 1). Three classification models had accuracies >90%, namely, Mismatch + random forest, PseDPC + support vector machine, and Subsequence + Logistic. Detailed performance information is shown in Table 2.

Table 2.

Performances of the Three Best Classification Models

Methods	Dimensions	F-Measure (%)	AUC (%)	ACC (%)
Mismatch	16	93.2	96.7	93.0
PseDPC	515	92.6	93.0	93.0
Subsequence	16	90.2	95.1	90.1

ACC, accuracy; AUC, area under the curve.

The Mismatch + random forest model, which has low dimensionality and good performance, was considered the optimal model for predicting essential miRNAs in the dataset (Table 2).

Representation of Important Features of the Classification Models

We explored the distribution of positive and negative examples under the different feature extraction methods. As shown in Figure 1, the accuracy was <70% for all of the classification models when combined with the NMBAC and SC-PseDNC-General feature extraction methods, indicating these methods were not suitable for predicting essential miRNAs. From among the remaining 50 classification models, those with the best classification performance under each feature extraction method were selected. Among the 10 selected models, those based on the Mismatch, Triplet, and PC-PseDNC-General methods all showed better performances when combined with the random forest algorithm. On the basis of the ANOVA of these three feature extraction methods, we took out the first four-dimensional features that had the greatest influence on the classification results. The obtained distribution of positive and negative examples is shown in Figure 2. Clearly, the difference between the positive and negative examples is larger with Mismatch than with the other two methods. This indicates that a feature extraction method that produces a large difference in the distribution of positive and negative samples contributes to a better final classification result.

Figure 2.

Distribution of Positive and Negative Examples of Important Features Obtained Using Different Extraction Methods

Optimal Voting Results

To achieve higher prediction accuracy, we used a novel voting method to predict all samples based on the results from the 10 selected classification models described in Representation of Important Features of the Classification Models. The predictions of these 10 models for all samples are shown in Figure 3.

Figure 3.

Distribution of Mispredicted Samples for Each Classification Model

We obtained four types of voting results from the 10 classification models, and the best results for each type are shown in Table 3. Each voting process eliminates two classification models, and the eliminated models have strong correlations. For example, from the distribution of the erroneously predicted samples shown in Figure 3, the classification models based on the GAC and MAC feature extraction methods had strong correlations, and the correct or mispredicted samples were almost the same, so these methods were eliminated in the second type of voting. Excluding the most relevant classification models was beneficial to the final voting result, as shown in Table 4.

Table 3.

Best Results for Four Types of Voting

Category	Model Included	No. of Samples that Were Mispredicted	Voting Results (%)
1	Mismatch + PseDPC + Subsequence,Kmer + Triplet + PseSSC, DACC,GAC + MAC	14	91.9
2	Mismatch + PseDPC + Subsequence,Kmer + Triplet + PseSSC + DACC	9	94.7
3	Mismatch + PseDPC + Subsequence,Kmer + Triplet	8	95.3
4	Mismatch + PseDPC + Subsequence	9	94.7

Table 4.

Verifying the Impact of Model Relevance on Voting

Reserved Classification Model	No. of Samples that Were Mispredicted	Voting Results (%)
Mismatch + PseDPC + Subsequence	9	94.7
Mismatch + PseDPC + Kmer	14	91.9
Mismatch + PseDPC + Triplet	9	94.7

As shown in Table 3, the results obtained by voting on the classification model based on the Mismatch, PseDPC, Subsequence, Kmer, and Triplet feature extraction methods were the best (accuracy rate of 95.3%). The accuracy of voting was higher than the accuracy of the case alone.

The predictions of the classification model based on the Triplet method were worse than those of the model based on Kmer (Figure 3), but after participating in the voting with the model based on Mismatch and PseDPC, the number of samples that were mispredicted with Triplet was less than the number with Kmer. This is because the model based on Kmer had a stronger correlation with the other two classification models. We chose to vote with the classification model based on Mismatch and PseDPC because these two feature extraction methods performed best in all of the classification models, and the correlation between them was very low.

The results shown in Tables 3 and 4 fully demonstrate that the novel voting method proposed in this study can achieve excellent results for the selection of essential miRNAs.

Conclusions

The aim of this study was to select essential miRNAs from a large number of miRNA sequences, thus making the study of the biological mechanisms of miRNAs more efficient. We used known mouse miRNAs as the dataset. We used different feature extraction methods to represent these data, then combined the extracted features with different classification algorithms to construct classification models. The final classification result was determined by a novel voting method, which gave a final voting result of 95.3%. This result showed that this method was effective in identifying the essential miRNAs in the dataset. In future work, we will focus on detecting new essential miRNAs, analyzing their function, and exploring the relationship between new miRNAs and diseases.27, 77

Materials and Methods

The general pipeline used in this study is shown in Figure 4.

Figure 4.

Flowchart of the Processes Used in This Study

Acquisition of Datasets

Acquisition of essential pre-miRNA sequences: miRNA genes produce primary miRNA (pri-miRNA) sequences that are 300–1,000 nt long. The pri-miRNAs are processed to precursor miRNA (pre-miRNA) sequences that are 60–70 nt long. Mature miRNAs, which are 20–24 nt long, are formed from pre-miRNAs by the action of enzymes.26, 48, 78 The hairpin structure of pre-miRNAs is an important feature that is widely used to identify miRNAs.⁴⁸ In this study, we used pre-miRNA sequences from miRBase (http://www.mirbase.org/). We collected a total of 91 pre-miRNA sequences that are essential in mice from Bartel’s 2018 review of metazoan miRNAs.⁷⁹ The 91 pre-miRNA sequences were from several families.

Acquisition of pre-miRNA sequences of unknown importance: We downloaded all the mice pre-miRNA sequences (a total of 1,234) from miRBase. All the sequences that belonged to the same families as the 91 essential miRNAs were excluded. The remaining pre-miRNAs (1,090) were considered to be non-essential pre-miRNAs.

To shorten the computational time and to ensure better performance results, we removed redundant sequences⁸⁰ and obtained a final dataset that contained 85 essential and 88 non-essential pre-miRNA sequences.

Feature Extraction Methods

Many methods have been used for extracting features of RNA sequences. In this study, we used a number of feature extraction methods on the selected pre-miRNA sequences. We expressed a pre-miRNA sequence as R = r₁, r₂, r₃, r₄, r₅, …, r_L, where r_i ∈{A, U, G, C},³³ and L is the sequence length. These features can be generated easily using the repRNA web server⁸¹ and the BioSeq-Analysis platform.⁸²

Kmer and Mismatch

Kmer is a simple feature extraction method in which k indicates the number of bases in a subsequence. For a given k value, there will be 4^k seed sequences.³¹ For example, for k = 2, there are 16 subsequences, AA, AC, AU, AG, UA, UU, …, CC. Mismatch calculates the number of occurrences of subsequences containing k adjacent bases and uses a parameter m (maximum allowed error match = 0 ≤ m < k).⁸³ For k = 2, m is 0, 1. For example, for the subsequence AC, the A– and –C subsequences are all regarded as AC.

Subsequence

Subsequence also counts the number of occurrences of a subsequence, but, in particular, it takes into account the length of the sequence that is eligible and the influence factor ∂.⁸⁴ This method allows for spacing between the bases in a given subsequence. For example, there are four ways in which a search for AAC in a sequence is eligible: AAC, AAXXXC, AXXXAC, and AXXAC (where X can be U or G). For this example, the number of occurrences of AAC can be expressed as: 1 + ∂6 + ∂6 + ∂5. For AAC, where there is no gap between the bases, the length is considered to be 1.

Triplet

Triplet extracts features based on secondary structure information in the sequence. Two states are considered for each nucleotide: matched (represented by “(” or “)” for A paired with U and G paired with C) and unmatched (represented by “.”). The possible structural forms for any three adjacent nucleotides are: “(((,” “((.,” “(.(,” “.((,” “(..,” “.(., ” “..(,” and “….” For the four bases (A, U, G, C) in a pre-miRNA sequence, there will be 8 × 4 different triplets. Triplet counts the number of occurrences of these 32 triplets in a sequence.⁴⁸

PseDPC

Pre-miRNA sequences can be represented by 10 secondary structure states: A, C, G, U, A–U, U–A, G–C, C–G, G–U, and U–G. Within a given range of distance thresholds, the PseDPC algorithm counts the frequency of occurrence of the two-two combination between the structural states. When the distance d is equal to 0, only the frequencies of each of the 10 states are counted. When d is ≥ 1, there will be 100 combinations of the 10 states, so the frequency of occurrence of the 100 combinations is counted. Therefore, 10 + 100d dimensional features can be extracted. Then the features of the free energy between the two structural states of a combination are extracted, giving φ features, where φ is the highest counted rank of the structural correlation along an RNA chain.¹³

Voting Process

The results produced by each classifier were integrated using a voting process as follows. First, each classification algorithm will correctly or incorrectly predict each of the samples (a sequence is considered a sample). Correctly predicted samples are not marked, the samples that were mispredicted are marked with “+,” and the number of classifiers is represented by n. Second, when n is a singular number ≥3, the number of times each sample is erroneously predicted under n classifiers is counted and recorded as m, where m ≥ n + 1/2; the sample is considered to be a sample that was mispredicted. Third, when n is a double number ≥4, the number of times each sample is mispredicted under the n classifiers is counted, and the sample corresponding to m = n/2 is selected. Suppose that there are z samples that meet the requirements, and the z samples are predicted differently by the different classifiers, the classifier with the highest number of mispredicted samples is eliminated. Then, n is singular and step 2 can be repeated. Fourth, when n satisfies the condition of being a singular number, steps 2 and 3 can count the number of samples considered to be mispredicted in various voting processes, from which the final voting result can be derived.

Author Contributions

X.R. implemented the experiments and drafted the manuscript. P.C., L.L., and Q.Z. initiated the idea, conceived the whole process, and finalized the paper. All authors have read and approved the final manuscript.