Recent Development of Machine Learning Methods in Microbial Phosphorylation Sites

Abstract

A variety of protein post-translational modifications has been identified that control many cellular functions. Phosphorylation studies in mycobacterial organisms have shown critical importance in diverse biological processes, such as intercellular communication and cell division. Recent technical advances in high-precision mass spectrometry have determined a large number of microbial phosphorylated proteins and phosphorylation sites throughout the proteome analysis. Identification of phosphorylated proteins with specific modified residues through experimentation is often labor-intensive, costly and time-consuming. All these limitations could be overcome through the application of machine learning (ML) approaches. However, only a limited number of computational phosphorylation site prediction tools have been developed so far. This work aims to present a complete survey of the existing ML-predictors for microbial phosphorylation. We cover a variety of important aspects for developing a successful predictor, including operating ML algorithms, feature selection methods, window size, and software utility. Initially, we review the currently available phosphorylation site databases of the microbiome, the state-of-the-art ML approaches, working principles, and their performances. Lastly, we discuss the limitations and future directions of the computational ML methods for the prediction of phosphorylation.

1. Introduction

Phosphorylation is an important and most common post-translational modification (PTMs) of proteins, which plays an important role in various aspects of biological processes including cell signaling and gene regulatory functions in both eukaryotes and microbial organisms [1-9], whereas several diseases occur due to abnormal phosphorylation events and different kinase modifications [1, 10, 11]. The phosphorylation events resulting in dysregulation of protein kinases involve a potential signaling mechanism associated with various complex diseases, including cancer development and progression [12]. For example, p53 is a protein critically responsible for tumor suppression, in which multi-site PTMs have been observed, suggesting extensive control of this protein [13]. Due to its pivotal role in various biological cellular processes, the molecular networks of protein phosphorylation in eukaryotes have been studied extensively [14-16]. Phosphorylation studies in mycobacterial organisms have currently demonstrated their critical importance in diverse cellular processes and pathogenesis [17-20]. Since there are numerous unmet clinical needs in bacterial infectious diseases, it is important to study bacterial protein phosphorylation comprehensively [1].

In the last decades, low-throughput experimental techniques were primarily applied to discovering novel phosphorylation sites, but executing these techniques is time-consuming and labor-intensive [21]. Recently, advanced proteome-based high-throughput mass spectrometry methods have greatly accelerated the identification of novel phosphorylation sites [22, 23], which have determined a large number of microbial phosphorylated substrates and PTM sites [15]. With a rapid increase of protein data via high-throughput sequencing techniques, it has been anticipated that the number of potential phosphorylation sites increases. This high-throughput method has several limitations: for a given phosphorylation site with specific modified residues, it is unable to identify the responsible protein kinases for such phosphorylation events [3]; it is difficult to pinpoint the exact phosphorylation sites by handling the existing technical challenges [23]; and experimentation processes mostly require expensive types of equipment and often labor-intensive, which are not available in ordinal laboratories [3, 16, 24, 25]. To identify novel phosphorylation sites by solving these limitations, machine learning (ML)-based approach has become increasingly popular. Hence, current efforts have primarily been made to develop computational methods, particularly ML-based approaches, to precisely identify the bacterial protein phosphorylation sites, but a limited number of computational tools have been presented so far [16, 24-26].

ML algorithms could greatly reduce the costs and labors in detecting potential phosphorylation sites with existing experimental data [16, 27, 28]. This review summarizes the ML-based computational resources with available databases, general or species-specific prediction models, and kinase-specific prediction models for bacterial proteins. It also discusses the benefits and limitations associated with the ML-based approaches. Therefore, this review can assist scientists to select the best predictor of bacterial phosphorylation sites and suggests the future directions of the ML methods.

2. Current phosphorylation databases

In Table 1, we have listed several protein phosphorylation site databases of mycobacteria, namely Phosphorylation Site Database [29], dbPTM 3.0 [30], PHOSIDA [31], UniProt [32], and SysPTM 2.0 [33] containing 1400, 186, 305, 176, and 348 phosphorylation sites, respectively. To date, the dbPSP has been the most updated available phosphorylation site database for microbes, which was constructed by Pan et al. [15] in 2015. It registers 3750 distinct phosphorylated proteins with 7391 phosphorylation sites on different amino acid residues containing arginine (Arg), cysteine (Cys), aspartic acid (Asp), tyrosine (Tyr), serine (Ser), threonine (Thr), and histidine (His) from 96 organisms. This database serves as an extensive data resource for studying bacterial phosphorylation.

3. Homolog redundancy

In PTM analysis, the curated sequences are often affected by homology and redundancy problems. Therefore, sequence redundancy elimination or homology reduction is a prerequisite to decipher the overfitting problem on the datasets. To shrink the homology sequences, most of phosphorylation prediction tools castoff the flanking sequence windows or protein sequences by using the program CD-HIT (http://weizhong-lab.ucsd.edu/cdhit_suite/cgi-bin/index.cgi?cmd= cd-hit) [32] or BLASTCLUST (http://nebc.nox.ac.uk/bioin-formatics/docs/blastclust.html). However, there is no cutoff standard program to filter the high sequence similarity. Therefore, a rigorous investigation on the benchmark dataset is essential to build a precise prediction model.

4. Protein encoding scheme

ML-algorithms are not able to directly handle sequence data, which need to be transformed into numeric feature vectors using different encoding methods [16, 34-59]. Three popular feature encoding schemes, consisting of evolutionary, sequence composition, and structural features, are used for predicting microbial phosphorylation sites (Fig. 1).

4.1. Evolutionary Features

The evolutionary profile is generated from the position-specific scoring matrix (PSSM) by using PSI-BLAST with different constraints including e-value and iteration times [60, 61]. Recently, different potential evolutionary schemes have been generated, including the amino acid composition of PSSM, tri-gram PSSM, dipeptide composition of PSSM, [62-65]. The MPSite predictor has introduced different evolutionary features for bacterial phosphorylation site prediction [16].

4.2. Sequence Composition-based Features

Different types of sequence composition encoding approaches were used, including amino acid frequency composition, amino acid composition (AAC), amino acid index properties (AIP), and binary encoding, for bacterial phosphorylation site prediction (Table 2). Amino acid location encoding is widely used in the field of PTM research [16, 25], where a sequence fragment is encoded into a feature vector by replacing any of the 20 native amino acids with numerical indexes. The dimension of the feature vector depends on the length of the sequence fragment. The composition of k-spaced amino acid pairs (CKSAAP) is widely used in PTMs research [24, 66]. Binary encoding is another common feature [16, 67-69].

4.3. Structure Features

The function of proteins critically depends on their tertiary structures and secondary structures. The α-helix, β-strand, and coil were used to build the native protein structures. It plays an important role in the interaction of the residues inside the proteins [65, 70, 71]. For further investigation of bacterial phosphorylation sites, secondary and tertiary structure information could be integrated [59, 72].

5. Machine learning algorithms

We reviewed existing ML-based bacterial phosphorylation site predictors including our MPSite [16], prkC-PSP [25], cPhosBac [24], and NetPhosBac [26] and compared their key aspects. As mentioned in section 4, most of the developed computational tools (Tables 2 and 3) have been constructed by using the ML algorithms. Based on our survey, the ML classifiers that predict bacterial phosphorylation sites are described below.

5.1. Support Vector Machine

The SVM is a popular supervised classification algorithm and widely used in protein bioinformatics research for classifying biological data. It aims to find the optimal hyperplane with the largest margin to accurately classify samples based on the feature dimensionality of the training dataset [73-75]. For the computational purpose, provided sequences are filtered, converted into the representative fixed-length feature vector and subjected to an objective function (class labels: phosphorylation site: 1 and non-phosphorylation site: 0). The generating mapping formula for SVM learns a function in the form of,

where y stands for the predicted class relative to the input feature vector of x; a_i is the modifiable weight coefficients associated with the sample x_i; b is the bias term which we target to maximize; K represents the suitable kernel function. So once a test dataset is given, features associated with the data are mapped to a high-dimensional space. Their class is predicted based on the sign by applying equation 1, such that if the sign is positive (+) y belongs to class 1; if the sign is negative (-) y belongs to the other class. It is worth mentioning that based on different computational scenarios several kernel functions are available for SVM, including Gaussian radial-basis function (RBF), linear/polynomial functions, and sigmoid functions. SVM is widely used in many bioinformatics studies [43, 76, 77]. The RBF kernel function was commonly employed, but it is important to make a better choice of kernel approaches according to needs and questions of interest [43]. Another important point is to choose the controlling parameters. In the SVM algorithm, two critical parameters are C (the penalty factor that controls the trade-off between the training error and margin) and ᵧ (the parameter that configures the kernel function) [25, 77]. Since variation in parameter configurations could significantly change prediction accuracy, the parameters should be optimized by the cross-validation test using a grid search approach to obtain the best performance.

To date, only four ML-based predictors for bacterial phosphorylation sites have been available. Interestingly, 2 out of 4 methods used SVM [24, 25] (Tables 2 and 3). In 2015, Li et al. [24] retrieved a phosphorylation dataset from NetPhosBac [26], containing 152 of pS/pT sites in 199 substrates (while 90% sequence identity were confirmed by CD-HIT) [78], and proposed a predictor of cPhosbac employing a SVM-based ML algorithm [24]. The method generated 2205 dimensional feature vector based on the CKSAAP methods. They have shown that the cPhosBac achieved high prediction accuracy compared to the NetPhosBac [26]. In 2018, Zhang et al. developed a new online prediction tool

employing the SVM methods, called prkC-PSP [25]. Basically, this is the first kinase (prkC)-specific phosphorylation site predictor in bacterial organisms. Since the kinase-specific dataset for bacterial phosphorylation is not available, they constructed a prkC kinase-related dataset of 36 phosphorylation sites in 14 experimentally validated protein substrates by curating the published literature. The prkC-PSP predictor used the amino acid location feature extraction method for encoding input features and achieved 94.89% accuracy.

5.2. Random Forest

Random forest (RF) [79] is one of the well-known and widely applied ML-algorithms to address various bioinformatics applications [16, 43, 67, 80-90]. RF essentially consists of a large number of N individual decision trees to operate as an ensemble learning algorithm [79]. For a given training sample of size T with Q features, RF builds Q subsets of training datasets by utilizing the bootstrap sampling, and then at each node randomly T features are selected to find the best split according to Gini impurity. Usually, the average prediction performance is reported to avoid overfitting problems. In RF, each decision tree consists of a number of ‘if then rules’ which are fairly simple to provide potential insight and knowledge to biologists. It is worth mentioning that the RF algorithm has three most important parameters: the number of decision trees; the number of variables that are randomly selected in each node partition; and optimization is necessary to minimize the number of samples to split internal nodes.

In 2019 [16], we developed a web-based bacterial phosphorylation site predictor called ‘MPsite’ (Table 2) using the enhanced characteristics of sequence features. To date, this has been only one general or non-species specific method for predicting microbial phosphorylation sites. In the MPsite [16], 2709 pS sites in 1940 proteins, and 2174 pT sites in 1534 proteins were collected from the dbPSP database [15]. From different feature encoding methods, the Wilcoxon rank-sum test was employed to select the key features. Finally, the optimized features were used to train the RF classifier. The MPsite achieved promising performance compared to the existing predictor NetphosBac.

5.3. Artificial Neural Network

Artificial neural network (ANN or NN) is well established and widely used in bioinformatics research [91-95]. ANN is a machine learning approach inspired by biological neural networks (the central nervous system of the animal at brain). Generally, ANN receives multiple input nodes, connect these inputs with their multiple internal states to generate the outputs using an output function. Each connection is assigned a weight coefficient which indicates its relative importance [96]. Generally, there are three important layers of ANN: the input layer; hidden layers; and the output layer [97]. Computationally, ANN is complex in addressing the problem of multiple hidden layers [98, 99]. Among the four reviewed predictors (Table 2) the NetPhosBac applied the ANN algorithm as their core method [26].

Besides the proposed operational framework, it is important to discuss their pros and cons, as noted below: (i) since overestimation is a major issue in ML-based methods and benchmark dataset-based performance is often subject to high risks of overfitting, hence the evaluation of the developed models by using independent dataset other than the benchmark dataset is essential; (ii) in general, web-based predictors are useful to detect putative phosphorylation sites and to develop further advanced tools. In this case, the details of the publicly available tools are listed in Table 3.

6. Current machine learning tools for phosphorylation site prediction

The development and proliferation of ML-based computation approaches to the prediction of phosphorylation PTM have been witnessed in recent decades. ML methods can be selected or designed with respect to training-test datasets, sequence/structural descriptors, targeted phosphorylation types, applied physiochemical properties, feature encoding techniques, etc. In this study, we explored the latest ML-based tools for predicting novel phosphorylation sites in bacterial organisms. In general, the development of the ML-based computation methodology consists of roughly five steps: (i) preparation of high-quality training dataset and independent dataset; (ii) extraction of informative features by suitable encoding schemes; (iii) construction of prediction models using ML-algorithms; (iv) performance evaluation of the models, and (v) web-server development (Fig. 1). This methodology is well established and widely used in computational protein biology and bioinformatics fields [16, 25, 34-43]. In computational biology, identifying phosphorylation sites accurately on a given protein is still a major challenge. From our review, the existing ML-based phosphorylation site predictors can be described in three categories: organism-specific, general, and kinase-specific modes [27].

7. General or organism specific phosphorylation site predictor

Protein phosphorylation analysis in eukaryotes has almost matured over the past few decades [3], but it is still countable in bacterial organisms [16]. To predict non-specific or general phosphorylation sites in microbes, Hasan et al. in 2019 [16] developed the first online-based ML predictor, namely MPSite with a random forest (RF) classifier, which predicts phosphorylated serine (pS) and phosphorylated threonine (pT) residues on the targeted protein sequences (Tables 2 and 3). It is well known that the proteins of each species have a distinct substrate structure for the attachment of different protein kinases (PKs). Thus, the prediction accuracy could be improved by designing the ML-based predictors in an organism-specific manner. In 2008, Miller et al. prepared the training dataset consisting of 103 phosphorylated serine sites (pS) and 37 phosphorylated threonine sites (pT), and developed the first bacterial-specific online predictor of NetPhosBac 1.0 [26] (Tables 2 and 3) by implementing an artificial neural network (ANN) algorithm. Li et al. [24] in 2015 retrieved the same dataset of pS and pT from NetPhosBac, and constructed a predictor of cPhosbac using a support vector machine (SVM) algorithm. The cPhosBac achieved higher prediction performance than the NetPhosBac predictor (Tables 2 and 3).

8. Kinase-specific phosphorylation site predictor

Currently, a number of studies have reported that kinase-specific phosphorylation plays an important role in various cellular activities that are inherently responsible for bacterial pathogenicity [1, 20, 44]. In bacteria, several recent studies have identified that the threonine/serine protein kinase, known as prkC, shows homology in catalytic domains. [45-47]. Further studies found that prkC is often involved in the various cellular process including bacterial resuscitation [48], gliding motility [49], and antimicrobial resistance [47]. Kinase has become one of the largest ‘druggable groups’ in cancer therapeutics in recent years [12]. Although numerous ML-based predictors, such as GPS, Scansite, PKIS, and PPSP, have been proposed to detect the kinase-specific phosphorylation sites in eukaryotes [7, 25, 27, 50-52], predictors for bacteria remain to be developed. In this regard, Zhang et al. recently have constructed a prkC kinase-related dataset of 36 phosphorylation sites in 14 experimentally validated proteins. They developed the first kinase (prkC)-specific web-application, prkC-PSP in 2018 (Table 2), using the SVM algorithm [25].

In recent years, protein kinases have become an important group of ‘druggable’ targets [10, 12]. To connect protein kinases or phosphorylated proteins to drug design and potential biomarker identification, several computational approaches were developed. In particular, unsupervised cluster analysis was used for phosphoproteomics profiling of kinases. The Wilcoxon rank-sum test was used to select important features and a linear kernel-based SVM algorithm was employed to build the final classifier [53, 54]. Recently, Leung et al. [12, 55] developed a command-line-tool called HyperModules to detect clinically and phenotypically related network modules for the discovery of disease mutations biomarkers.

9. Caveats of the state-of-the-art ML 
approaches

Even though great progress has been made in the development of phosphorylation site prediction tools, several challenges and limitations need to be addressed. Firstly, the prediction accuracy evaluated by cross-validation test is difficult to reproduce, unless the ML parameters and source codes regarding feature encodings are provided. On the other hand, the prediction performances will be reproduced on independent datasets, if a developer provides a standalone program or web application. Unfortunately, many published methods neither open their assigned source codes nor datasets, which delays the development of next-generation methods. Therefore, it is highly recommended to provide datasets and source codes while publishing a new methodology [100]. Secondly, most of the publicly available methods used their own independent dataset to evaluate the prediction performance in comparison with existing methods. For fair comparisons, the construction of a unique or independent dataset is essential. While constructing distinct datasets, care should be taken that none of the sequences overlap with the benchmark dataset.

Conclusion and perspectives

Recently, the field of bacterial phosphorylation site detection has made noticeable progress in recruiting the ML-approaches, as mentioned in Tables 2 and 3. Owing to high-throughput sequencing techniques, automated computational approaches are required to enable rapid and accurate prediction phosphorylation sites related to kinases from a large number of candidate proteins. In this regard, several ML-based approaches have been developed in both the sequence-based and structure-based classes; many predictors 
were built in the kinase-specific and organism-specific/ 
general manners by using a variety of training and test dataset resources [3, 27]. In order to develop the next-generation methodology, the following challenges could be explored [27]. First, a reliable, high-quality benchmark dataset is constructed by carefully searching existing phosphorylation site databases and through rigorous literature inquiry. Second, most of the existing feature descriptors are extracted from primary sequences. On the other hand, many functional sites were found based on evolutional and structural information [101-103]. The addition of structure-based and evolutionary information of protein kinases proves valuable to improve the predictors [104-108]. Third, predictors available for a wider variety of organisms are required, because protein kinases are disparate in different organisms [3, 109, 110]. Forth, different ML classifiers are explored to increase prediction performance. It is important not only to integrate different feature encodings [111-115], such as K-nearest neighbors, multivariate information, biochemical properties, and pseudo residues composition, but also to investigate different ML classifiers [116-122], such as an extremely randomized tree, extreme gradient boosting, light gradient boosting, and deep learning. The feature selection technique should remove redundant information to improve performance. In this regard, mRMR [123], ANOVA [124], and MRMD [125, 126] can be considered. Rapid development in structural bioinformatics and sequential bioinformatics have driven the medicinal chemistry undergoing an unprecedented revolution of proteins [127-129], in which the recently proposed encoding methods [130-135] may play an important role in discovering new microbial phosphorylation sites.

Fig. (1).

An overview of the general framework of machine learning (ML) based computational approaches for phosphorylation sites prediction. Generally, the construction of ML-approaches roughly consists of the following 5 steps: (i) dataset preparation; (ii) selection encoding methods; (iii) building prediction models; (iv) performance evaluation; and (v) development of a web-server. (A higher resolution / colour version of this figure is available in the electronic copy of the article).

Table 1.

List of currently available protein phosphorylation site databases in mycobacteria.

Database	Number of Phosphorylation Sites/ Total Proteins	Year	Database URL	References
dbPSP	7391/3750	2015	http://dbpsp.biocuckoo.org/	[15]
SysPTM 2.0	348/213	2014	http://lifecenter.sgst.cn/SysPTM/	[15, 33]
UniProt	176/135	2014	http://www.uniprot.org/	[15]
PHOSIDA	305/382	2010	http://www.phosida.com	[15, 31]
dbPTM 3.0	186/138	2006	http://dbPTM.mbc.nctu.edu.tw/	[15, 30]
Phosphorylation Site Database	1400/960	2004	http://vigen.biochem.vt.edu/xpd/xpd.htm (Not available)	[15, 29]

Table 2.

List of currently available machine-learning tools for bacterial phosphorylation sites prediction.

Tool (Year)	PTM Residue	ML Algorithm	Feature Encoding	Dataset Size (Positive/Negative)		Homolog and Redundancy	Window Size	References
MPSite (2019)	Serine Threonine	RF	Evolutionary, sequence composition, and structure features	Training dataset	Independent dataset		21	[16]
				S: 1704/ 3408 T: 1401/ 2802	S: 341/ 682 T: 254/ 508	30% 30%
prkC-PSP (2018)	Serine Threonine	SVM	Amino acid location	36/512		-	31	[25]
cPhosBac (2015)	Serine Threonine	SVM	CKSAAP	152/5761		-	21	[24]
NetPhosBac (2008)	Serine Threonine	Neural network	sequence composition features	152/841		90%	13	[26]

‘- ‘, means not available.

Table 3.

Detail of the available web server for bacterial phosphorylation sites prediction.

Predictor	Description	Predictor URL	References
MPSite	Web-based machine learning predictor for identifying bacterial phosphorylation sites using the sequence features. This is a non-organism specific or general phosphorylation site predictor.	http://kurata14.bio.kyutech.ac.jp/MPSite/	[16]
				prkC-PSP	Web application for identification of prkC-specific phosphorylation sites in bacteria based on sequence information.	http://free.cancerbio.info/prkc/	[25]
cPhosBac	Web application to predict phosphorylation sites in bacteria. It includes protein and motif length scan to optimize the prediction.	http://netalign.ustc.edu.cn/cphosbac/	[24]
NetPhosBac	First web-based bacterial phosphorylation site predictor based on the sequence information. It is a taxa-specific predictor.	http://www.cbs.dtu.dk/services/NetPhosBac	[26]

Consent for Publication

Not applicable.

Funding

This study was supported by Japan Society for the Promotion of Science by Grant-in-Aid for Scientific Research (B) (19H04208) and by the developing key technologies for discovering and manufacturing pharmaceuticals used for next-generation treatments and diagnoses both from the Ministry of Economy, Trade and Industry, Japan (METI) and from Japan Agency for Medical Research and Development (AMED).

CONFLICT OF INTEREST

The authors declare no conflict of interest, financial or otherwise.