HNetGO: protein function prediction via heterogeneous network transformer

Abstract

Protein function annotation is one of the most important research topics for revealing the essence of life at molecular level in the post-genome era. Current research shows that integrating multisource data can effectively improve the performance of protein function prediction models. However, the heavy reliance on complex feature engineering and model integration methods limits the development of existing methods. Besides, models based on deep learning only use labeled data in a certain dataset to extract sequence features, thus ignoring a large amount of existing unlabeled sequence data. Here, we propose an end-to-end protein function annotation model named HNetGO, which innovatively uses heterogeneous network to integrate protein sequence similarity and protein–protein interaction network information and combines the pretraining model to extract the semantic features of the protein sequence. In addition, we design an attention-based graph neural network model, which can effectively extract node-level features from heterogeneous networks and predict protein function by measuring the similarity between protein nodes and gene ontology term nodes. Comparative experiments on the human dataset show that HNetGO achieves state-of-the-art performance on cellular component and molecular function branches.

INTRODUCTION

As the expression product of genes, protein forms the main material basis of life and plays a key role in life activity and function execution. Functional annotation of proteins is crucial to understanding life activity from the molecular level. Gene ontology (GO) [1] is a systematic method of annotating the properties of genes and gene products, which divides the function of proteins into three different sub-ontology: biological process (BP), cellular component (CC) and molecular function (MF). As shown in Figure 1, for each branch, GO is a direct acyclic graph, where each node has a unique label and refers to a specific term. The nodes with deeper hierarchies refer to a more detailed description of protein function. That means, when a protein is labeled with a specific term, it is also annotated by all ancestor nodes of the term, which is known as true path rule [2–4].

Figure 1.

A subgraph of GO. GO is a direct acyclic graph for each branch, where each node has a unique label and refers to a specific term.

Automatic protein function annotation aims to predict protein function through computational methods, which is more flexible and convenient than experimental methods and has important application prospects. In recent years, shallow machine learning and deep learning have been widely used in the field of bioinformatics, such as biological sequence analysis [5], protein structure prediction [6, 7] and medical image processing [8]. From the perspective of machine learning, protein function prediction is usually regarded as a multi-label classification problem. Methods based on shallow machine learning usually integrate the features extracted from multisource data to measure the similarity between proteins and functional terms and annotate similar functions for similar proteins. The most representative methods are GeneMANIA [9], MS-kNN [10] and NetGO [11]. GeneMANIA [9] is a semi-supervised algorithm based on network label propagation, which fuses heterogeneous networks into a network through linear regression and then makes functional predictions through Gauss label propagation algorithms. The MS-kNN [10] algorithm combines a variety of similarity measures to extract features from sequence similarity, protein–protein interaction (PPI) network and gene expression profile data, which is used by the kNN algorithm to predict protein function. Similarly, the NetGO [11] model predicts protein functions through a ranking framework based on ensemble learning, which comprehensively measures the similarity between proteins and GO terms and uses multiple sub-models to extract features from sequence and PPI network. Compared with MS-kNN only using sequence similarity, NetGO can effectively extract multiple features from the sequence through the sub-model, such as protein family and structural domain information, and thus achieves better performance. However, the reliance on complex feature engineering and model integration methods limits the development of such methods.

Deep learning methods can extract features from large-scale data in an end-to-end manner, leading to their increasing popularity in the field of automatic protein function annotation [12, 13]. Such models usually focus on extracting the features of protein sequence through deep learning networks such as convolution neural network (CNN) and recurrent neural network (RNN), and then integrate sequence similarity, PPI network and other data to improve model performance [14, 15]. In recent years, researchers have made a lot of efforts to develop deep learning models to predict protein function, among which the most representative methods are DeepGO [16] and DeepGOPlus [17], deepNF [18] and DeepMNE-CNN [19], as well as GONET [20] and DeepGOA [21, 22]. DeepGO [16] is one of the earliest models to annotate protein based on deep learning, which combines the sequence and PPI network features to predict protein function and achieve good results. DeepGOPlus [17] model is an improved version of the DeepGO, which does not rely on PPI network data and instead improves prediction performance by integrating sequence similarity information. DeepNF [18] extracts the high-order features of PPI network based on multimodal deep autoencoders, and then predicts protein function via support vector machine. DeepMNE-CNN [19] is an embedding-based function prediction method, which combines semi-supervised autoencoder and CNN to extract complex topological features of multi-networks. GONET [20] predicts protein function by integrating protein sequence and PPI network, which can effectively extract long-range features of protein sequences using RNN and achieves good performance in human and mouse datasets. DeepGOA [21] innovatively utilizes graph convolutional network to model the hierarchy structure of GO term network and annotate proteins by calculating the similarity between protein nodes and term nodes, which achieves good results in corn and human datasets. Based on deep learning algorithms, existing models can effectively extract features from protein sequences, while they still rely on artificially designed model integration or feature integration methods to process sequence similarity information [17, 23] and PPI data [16, 20], which are relatively shallow and inevitably cause information loss, and thus hinders the development of such methods [24].

Besides, even from the perspective of sequence feature extraction, these algorithms must train complex models on specific labeled datasets, resulting in their inability to use large-scale unlabeled sequence data [25–28]. Large-scale pretraining models can effectively alleviate this problem, such as the popular Bert [29] and XLNet [30] in the natural language field, which can train deep learning models through unsupervised learning on large amounts of unlabeled data, and then learn effective semantic representations of sentences. Inspired by such methods, researchers have proposed many pretrained methods [25, 31–34] to model protein sequences, among which the most representative models are ProtVec [31] and SeqVec [25] algorithm. The ProtVec model uses a method similar to the Subword-Embedding in fasttext [35] to obtain the representation of the sequence, which divides the full amino acid sequence into fixed-length substrings and represents each substring as a fixed dimensional vector through the word2vec [36] algorithm. Although the ProtVec model can effectively capture the local features of protein sequence, it ignores the context information of the amino acid sequence due to the context-independence of the word2vec model, and therefore cannot effectively extract the long-range relationship of protein sequence. To tackle this problem, the SeqVec model utilizes the bidirectional LSTM sequence model to capture the long-range association of protein sequence and generates an amino-acid-level embedding vector, which not only contains the semantic information of the amino acid itself, but also the information of its corresponding context [37–41].

In this paper, we propose an end-to-end protein function prediction model HNetGO to solve the problems mentioned above. Firstly, we utilize heterogeneous networks to integrate multisource data in an intuitive and effective way, which avoids the information loss caused by manual designed feature extraction methods to the greatest extent. Secondly, we use a pretraining model to extract protein-level sequence features, which can effectively capture functional-related semantic information within a single protein sequence. In the end, we design a link prediction model based on the attention mechanism to predict protein function. And comparative experiments on the human dataset show that HNetGO achieves state-of-the-art performance on CC and MF branches.

MATERIALS AND METHODS

Datasets acquisition and preprocessing

For our experiment, we downloaded human and mouse protein sequences and their corresponding experimentally verified GO annotation data from the UniProt [42] database, which contained 20 395 human protein sequences and 17 073 mouse protein sequences, respectively. And the PPI network data were downloaded from STRING [43] database (version 11). In addition, we downloaded the latest released GO data (releases/ 01 February 2021) from the official website. After that, we constructed a network of GO terms based on the information extracted from the file. It should be noted that each sub-ontology contains thousands of functional terms, while most of which have not appeared in our dataset. Therefore, we filter out terms with annotated proteins <300 in the BP branch, and the threshold for CC and MF branch is 100, which in turn resulted in unannotated proteins in the dataset. For these proteins, we keep them in the heterogeneous network to improve the connectivity of the network, but do not use them as part of the training set or test set. Table 1 shows some statistics of the dataset.

Table 1.

Dataset statistics

Datasets	Terms (before filter)			Terms (after filter)
	BP	MF	CC	BP	MF	CC
Human	15 658	4803	1995	752	295	293
Mouse	15 838	4772	1985	666	262	291

BP, biological process; CC cellular component; MF molecular function.

Extract protein-level sequence features through pretrained model

The primary structure of protein refers to a one-dimensional sequence composed of 20 kinds of amino acids [44, 45], which can determine the secondary and tertiary structure of the protein, and thus can affect the protein functions [46]. Existing protein function prediction methods usually use one-hot encoding to represent amino acid sequences as a matrix or tensor, which can be used to extract protein-level semantic features through deep learning models. However, due to the inability to use large amount of existing unlabeled sequence data and the semantic independence of one-hot encoding, such methods cannot capture sufficient semantic features of protein sequences, resulting in their poor performance in predicting protein functions.

In this paper, we use the SeqVec, which is inspired by embeddings from language models (ELMO) model [47], to extract protein-level sequence features. ELMO, a powerful pretraining model for natural language sequence processing, can capture the contextual features of words and can generate different embedding vectors of the same word according to different contexts. Similarly, as shown in Figure 2, the SeqVec captures the long-range association of protein sequence and generates an amino-acid-level embedding vector, and then directly obtains an effective protein-level semantic representation through average aggregation. Experiments at the protein level [25, 48, 49] show that methods based on SeqVec features achieve similar results to the state-of-the-art model and have obvious superiority compared with embedding methods such as one-hot and ProtVec.

Figure 2.

Heterogeneous network construction process in HNetGO model. For protein sequence data, we first utilize the results of MSAs to construct a sequence similarity network, and then the pretrained model SeqVec is used to extract amino-acid-level features, which can be used to obtain protein-level features through average aggregation or principal component analysis. Next, a PPI network is built based on the interaction relationship, and a heterogeneous network is used to integrate all the information extracted from the original dataset.

Figure 3.

Framework of HNetGO model. In general, HNetGO model consists of three parts: a node-level mutual attention layer to learn the attention weights of all direct neighbors of the current node; a multi-head messaging layer enables different types of neighbors to deliver messages to the current node based on attention weights and a link prediction layer to predict protein function. In addition, HNetGO uses the cross-entropy loss function to optimize the model.

At the amino acid residue level, experiments [25] show that the performance of this model has some disadvantages compared with models that use protein evolutionary information. In fact, evolutionary information reflects the relationship between proteins, rather than the semantic information within a single protein sequence, so it is no wonder that such model designed to extract single sequence-level features cannot capture this information. To solve this problem, we explicitly integrate the sequence similarity information obtained from the multiple sequence alignments (MSAs) algorithm and the detailed approach is explained in the next section.

In terms of implementation, we use the SeqVec model deployed based on ELMO, which is about 360 MB in size and uses about 33 M sequence data in the UniRef50 [50] database for pretraining. In experiments, with a Tesla P40 graphics card, we can complete the training of 20 395 sequences with an average length of 555.53 in 40 min, which means that, on average, we can get sequence features of a protein in 0.11 s. For each sequence, we get a 1 × 1024-dimensional protein-level feature, and for the input protein that does not contain sequence data, we use a randomly generated vector as its sequence feature.

Modeling evolutionary relationships with MSA

A pretrained model can effectively extract the semantic information contained in a single protein sequence, but theoretically it cannot capture the relationship between proteins. The evolutionary relationships of proteins are exactly a kind of relationship between proteins, which mainly refer to the homology of proteins and encode the information of biodiversity in the process of protein evolution. The amino acids in the protein sequence may mutate during the evolution process, causing the evolutionary tree to split. However, the protein sequence does not directly determine the function of protein, but indirectly affects the protein function through the protein structure, which means that the mutation of many amino acids may not cause the protein function to change [51]. Therefore, homologous proteins with different sequences tend to share similar structures and functions. Considering the significant sequence similarity between homologous proteins, protein sequence similarity, which can be easily obtained through MSAs, is suitable for inferring protein homolog. For example, some classic methods [52, 53] filter the results of MSAs according to a certain threshold and use these data to construct a protein similarity network, which is used by a subsequent clustering algorithm to infer protein homology.

Inspired by such methods, the HNetGO model uses diamond [54, 55], a fast and accurate MSA algorithm, to calculate sequence similarity between proteins, and selects the output with an e-value higher than 0.001 as the final alignment result, which is used to construct protein sequence similarity network. Compared with the manual integration approach used by the DeepGOPlus [17] model, our method can capture a wider range of local features through multi-layer aggregation operations between nodes on the sequence similarity network, whereas the manual integration method only considers the direct similarity between proteins.

P‌PI network

Existing studies [6, 56] have shown that deep learning models can predict the tertiary structure of proteins through sequences, so it is no wonder that pretraining models can extract structure-related local and global features contained in a single protein sequence. However, proteins usually do not perform their functions alone, but achieve specific functions through protein complexes formed by interacting with different proteins. This complex is the quaternary structure of proteins and reflects the association between proteins, which means that the semantic representation of a single protein sequence cannot express such type of relationship. PPI network is an undirected graph with proteins as nodes and protein interactions as edges, which can model the compound relationship between proteins, and thus can reflect the functional connection between proteins.

It should be noted that the protein ID used in the PPI data downloaded from the STRING database is not the same as the protein ID used in the sequence data. Therefore, we use the mapping file obtained from the Uniprot database to perform field alignment and data filtering on the PPI data. Table 2 shows some statistical information of the PPI network after processing.

Table 2.

PPI network statistics

Datasets	Nodes	Edges	Average degree	Network density
Human	18 560	11 098 152	597	0.064
Mouse	16 420	9 730 128	593	0.072

Heterogeneous information network

To avoid the use of manual design methods to converge the network, HNetGO uses a heterogeneous information network to model all the information extracted from the original data. The heterogeneous information network allows different types of edges and nodes to appear in the same network, and thus can model complex entity relationships. As shown in Figure 2, the heterogeneous network constructed in this paper consists of two types of nodes, namely protein and GO terms, and four types of edges between them, which can encode the hierarchical structure between terms, interactions and sequence similarity between proteins, and functional associations between proteins and terms. Formally speaking, we can define the heterogeneous graph as:

(1)

and the corresponding node type mapping relationship is:

(2)

(3)

where represents the collection of nodes, represents the collection of edges and each node and each edge . is the collection of all node types, including protein and term, is a collection of all edge types, including four types of meta-relation, where and reflect the connection between proteins, reflects the hierarchy structure between terms and reflects the association between proteins and terms. Therefore, we can predict protein function by predicting the annotation relationship between protein nodes and term nodes in this heterogeneous network.

Model and implementation

Using heterogeneous network can greatly simplify data preprocessing and preserve more information extracted from original data, while at the same time it inevitably presents a great challenge to the design of prediction model. Inspired by the transformer-based deep learning model [57–59], we utilize a graph neural network based on attention mechanisms to learn embedding vector of nodes in heterogeneous networks, which are fed into a subsequent model for link prediction. As shown in Figure 3, our model is composed of three parts: a node-level mutual attention layer to learn the attention weights of all direct neighbors of the current node; a multi-head messaging layer enables different types of neighbors to deliver messages to the current node based on attention weights; a link prediction layer to predict protein function.

Node-level mutual attention

Similar to the Transformer model, for a given triplet , we map the source node to a key vector, and the target node t to a query vector, and then calculate the contribution weight of the different source nodes to the target node through attention mechanism. If the input feature vector of the target node t is and the vector of the source node s is , the corresponding projected vector can be calculated as follows:

(4)

(5)

where and are the projected vectors of target node and source node, respectively. And, and represent the type of node and . and are type-specific transformation matrices related to node type which map different dimensions or different types of features into the same hidden semantic space and enable the model to calculate similarity score between any node pair. Then, the similarity score is calculated as follows:

(6)

where is the dimension of hidden space and is a weight matrix associated with the type of edge . They capture different semantic relationships formed by the same nodes over different type of edges. For example, two proteins with similar sequences may also interact with each other. By learning different weight matrices based on edge type, HNetGO model is able to extract different features between the same protein pairs. After calculating the similarity score of all neighbors of the target node, the final attention weight can be obtained by normalizing the score through softmax function:

(7)

where contains all neighbor nodes of the target node. If there is a node which is connected to the target node through different edges, we treat it as different neighbors during the calculation of attention weight. At the same time, for the same triple, the attention weight is not symmetrical for the source and target nodes, which means that their contributions to each other are different.

Multi-head attention and message passing

There are large differences between different types of nodes in heterogeneous networks, and even for the same type of nodes, their network characteristics, such as degree distribution, are usually extremely imbalance. To tackle this problem, we design a multi-head attention and message passing layer, which can extract the structure of heterogeneous network from different aspects. In particular, suppose we use attention heads, then for a given triplet (s, e, t), the attention weight vector can be calculated by using:

(8)

Correspondingly, we design a multi-head message approach:

(9)

(10)

where represents the number of attention heads, is the input feature of source node and is a transformation matrix related to node type which can project into a hidden space. is an edge-type-specific matrix that allows the source node to deliver different messages to the destination node based on the edge type. Next, the embedding vector of target node can be obtained by aggregating information from all of its neighbor nodes according to the corresponding attention weight vector:

(11)

Finally, we map the embedding vector back to the target node space and add residual connection to prevent network degradation:

(12)

where is the parameter matrix of linear projection, is the original feature vector of node t, and is the output embedding vector of the target node, refers to sigmoid function.

Protein-term link prediction

For any given protein node , as well as corresponding term node , we use the embedding vector obtained above to calculate a similarity score for this pair of nodes:

(13)

where is the embedding vector of protein node, is the embedding vector of term node is a parameter matrix for training which maps embedding vector of protein node to the term space. is the sigmoid function and converts the output value of the decoder into a probability value , which is between (0,1) and is regarded as the confidence value of the protein function prediction. Specifically, HNetGO regards node pairs with a value >0.4 as positive examples of function prediction.

Finally, we use the cross-entropy loss function to optimize the model:

(14)

here refers to the real annotation relationship between node p and t.

For any protein with unknown function, we use the following three steps to predict its function. First, extract sequence features of the protein through the SeqVec model. Then, use the Diamond algorithm to find the homologous protein node and add the new node to heterogeneous network according to the sequence similarity relationship. Finally, in the prediction stage, the HNetGO model only relies on the neighborhood information of the node, so the subgraph containing this protein node can be extracted through neighborhood sampling to perform function prediction.

Experiment and evaluation criterion

Evaluation criterion

As what was done in other works [20, 21, 60–63], we select area under the ROC curve (AUC), area under the precision-recall curve (AUPR) and Fmax to evaluate the performance of the model from different aspects [64, 65].

AUC reflects the tradeoff between true positive rate (TPR) and false positive rate (FPR) of a model, whereas AUPR pays more attention to the tradeoff between TPR and positive predictive value. Fmax is an official evaluation criterion of the critical assessment of functional annotation [66], which measures the average accuracy and recall rate of the model. For a given threshold , the average precision (), average recall rate () and Fmax on the test set are defined as follows:

(15)

(16)

(17)

where n is the total number of proteins in the dataset and is the number of proteins that annotated with at least one term with the threshold . and refer to the precision and recall rate of the i-th protein with the threshold τ and can be defined by the following formula:

(18)

(19)

where is the ground truth of protein i and refers to the predicted label under specific threshold .

Experiments

To validate the rationality of data integration method and model design, we design different submodels and conduct experiments on mouse and human datasets. In total, we design three variants of HNetGO: HNetGO-PPI removes sequence similarity link from heterogeneous network, whereas HNetGO-SIM removes PPI link and HNetGO-RAN replace features extracted from the pretrained model with randomly generated vectors. In implementation, we take 5-fold cross-validation to reduce the experimental error and make full use of the dataset, which means that in each experiment, the training dataset contains 80% of the data, and the remaining 20% of the data is used as the test set. Specifically, the mouse dataset contains 16 420 protein nodes, of which 13 136 nodes are used as the training set, and the human dataset contains 18 560 protein nodes, of which 14 848 nodes are treated as training set.

As shown in Table 3, removing any part of a heterogeneous network will result in a performance degradation of the model, which precisely indicates that different types of relationships in the network contribute different information and also suggests that our model can effectively extract information encoded by different types of links. In particular, we can easily find that the removal of the PPI network has the greatest impact on the BP branch, which reflects the fact that both BP and PPI network focus on the functional interaction relationship between proteins. Besides, when replacing features extracted by pretrained model with randomly generated vectors, we observe not only a performance loss in the experiment, but also a decrease in convergence speed and stability, which indicates that pretrained models can effectively extract semantic information from a single protein sequence.

Table 3.

Results of ablation experiment

Models		Human			Mouse
		F max	AUC	AUPR	F max	AUC	AUPR
BP	HNetGO	0.561	0.909	0.625	0.543	0.890	0.579
	HNetGO-PPI	0.448	0.879	0.450	0.422	0.838	0.419
	HNetGO-SIM	0.328	0.764	0.285	0.331	0.751	0.297
	HNetGO-RAN	0.404	0.819	0.389	0.393	0.806	0.371
CC	HNetGO	0.748	0.971	0.812	0.742	0.969	0.808
	HNetGO-PPI	0.498	0.891	0.494	0.537	0.893	0.531
	HNetGO-SIM	0.474	0.878	0.458	0.500	0.886	0.492
	HNetGO-RAN	0.441	0.854	0.425	0.432	0.867	0.414
MF	HNetGO	0.697	0.959	0.771	0.674	0.953	0.743
	HNetGO-PPI	0.566	0.892	0.584	0.551	0.888	0.563
	HNetGO-SIM	0.548	0.891	0.567	0.549	0.883	0.556
	HNetGO-RAN	0.589	0.911	0.615	0.573	0.904	0.579

BP, biological process; CC, cellular component; MF, molecular function.

To further verify the performance of the model, we compared our model with several prevailing methods on the human protein dataset, including Naïve and BLAST [67], DeepGO [16] and DeepGOPlus [17], as well as GONET [20] and DeepGOA [21]. The results of comparative experiment are shown in Table 4.

Table 4.

Evaluation on human dataset with other models

Methods	BP			CC			MF
	F max	AUC	AUPR	F max	AUC	AUPR	F max	AUC	AUPR
Naïve	0.344	0.500	0.566	0.551	0.487	0.377	0.326	0.499	0.528
BLAST	0.339	0.577	0.489	0.441	0.563	0.269	0.411	0.623	0.461
GONET	0.612	0.934	0.581	0.718	0.972	0.780	0.646	0.973	0.709
DeepGO	0.327	0.639	0.571	0.589	0.695	0.448	0.404	0.760	0.625
DeepGOPlus	0.362	0.687	0.608	0.628	0.652	0.487	0.468	0.819	0.694
DeepGOA	0.385	0.698	0.622	0.629	0.757	0.500	0.477	0.820	0.710
HNetGO	0.561	0.909	0.625	0.748	0.971	0.812	0.697	0.959	0.771

BP, biological process; CC cellular component; MF molecular function.

Naïve and BLAST are rule-based methods, which can directly annotate protein functions. Naïve is an intuitive method that annotates proteins according to the frequency of GO terms, and thus the algorithm predicts same annotations for all samples in the dataset. BLAST is a classic method based on protein sequence similarity, and as mentioned above, here we use Diamond to calculate sequence similarity between proteins. For other models, we explained in detail in the Introduction section, and it should be noted that GONET actually combines Prot2Vec and a well-designed deep learning model to extract protein sequence features. In detail, GONET first splices the amino acid-level vectors output by Prot2Vec into a matrix (which is a protein-level feature),then reduces the dimensionality of the matrix through a convolutional neural network and finally uses a RNN to extract the long-range connection of protein sequence. Besides, for DeepGO and DeepGOPlus, we predict the function of the protein based on the tools provided by the original author and calculate the prediction performance based on this result.

From Table 3, it is obvious that HNetGO achieves better performance on several evaluation metrics than other models, which indicates that the data integration method and model design of this paper are reasonable. However, despite the state-of-the-art performance achieved in AUPR, HNetGO performs relatively poorly on the AUC criterion, which means that the FPR of the predicted results of our model is slightly higher. This may be partly caused by the incompleteness of protein annotation dataset, which means that there may be some new functions added to a protein in the future dataset, whereas such functions in the prediction results will be regarded as negative examples under the current dataset. Therefore, we cannot determine whether the small decrease in the AUC reflects a decrease in the model’s predictive performance or a better generalization ability. In conclusion, compared with GONET, HNetGO obtains considerable performance, which suggests that it is feasible to replace complex models based on biological prior knowledge with pretrained models.

CONCLUSION AND DISCUSSION

In this paper, we propose an end-to-end, attention-based link prediction model named HNetGO to predict protein function, which can efficiently integrate protein sequence and interaction data through heterogeneous information networks. In particular, HNetGO innovatively utilizes heterogeneous information networks to model the complex relationship between proteins and GO terms and extract distributed embedding features of protein sequences based on the pretrained model. In addition, we introduced a powerful attention-based graph neural network to learn node embedding in heterogeneous networks.

At the same time, in general, amino acids only indirectly affect protein function through the structure of proteins, which determines that protein function prediction is a protein-level task, not an amino acid-level task. Therefore, it is an intuitive and effective choice to build functional prediction models based on protein-level pretrained features. And in this paper, we demonstrate that it is reasonable to replace complex models based on biological prior knowledge with pretrained models.

Finally, it should be noted that the GO database contains a large amount of information about genes and gene products, which means that the relationship between GO terms is very complex and each term has specific functional semantics and is dataset independent. However, due to the limitations of model complexity, we use only part of the association between GO terms, and the input feature of the term nodes is also randomly generated. In future work, we will try to design more complex models to fully mining the information in GO and explore the possibility of using pretrained language models to extract text semantic features of GO terms.

Additional Files

All additional files are available at: https://github.com/BIOGOHITSZ/HNetGO.

AUTHORS’ CONTRIBUTIONS

X.Z., Y.G. and F.Z. performed bioinformatics analysis, X.W., K.W., S.Q., B.L., Y.W., Y.H. and J.L. designed the study, and J.L. drafted the manuscript. All of the authors performed the analysis. J.L. conceived of the study, participated in its design and coordination and drafted the manuscript.

FUNDING

National Key Research Program (2021YFA0910700); Shenzhen Science and Technology University stable support program (GXWD20201230155427003-20200821222112001); National Natural Science Foundation of China (82003553); Guangdong Key Area Research Program (2020B0101380001); Shenzhen Science and Technology Program (JCYJ20200109113201726).