Major Advances in Protein Function Assignment by Remote Homolog Detection with Protein Language Models – a review

Abstract

There is an ever-increasing need for accurate and efficient methods to identify protein homologs. Traditionally, sequence similarity-based methods have dominated protein homolog identification for function identification, but these struggle below ~30% sequence identities. Recently, transformer architecture-based deep learning methods have achieved breakthrough performances in many fields. One type of model that uses transformer architecture is the protein language model (pLM). Here, we describe methods that use pLMs for protein homolog identification intended for function identification and describe their strengths and weaknesses. Several important ideas emerge, such as filtering the substitution matrix generated from embeddings, selecting specific pLM layers for specific purposes, compressing the embeddings, and dividing proteins into domains before searching for homologs that improve remote homolog detection accuracy considerably. All of these approaches produce huge numbers of new homologs that can reliably extend the reach of protein relationships for a deeper understanding of evolution and many other problems.

Introduction

The Remarkable Ability of Protein Language Models to Comprehend Function from Sequences

Language Models are what is behind the popular text and image generative AI, such as ChatGPT, DALL-E and their successors, which rely on neural network architecture called transformers. ChatGPT has scooped up text from the internet to write reasonably sensible paragraphs on many topics. ChatGPT is trained on language by masking text and training to predict the right missing word.

Genes are a language in the four letters of the genetic code, and when translated become the 20+ letters for the names of amino acids. The data input for protein language models (pLMs)are sequences and, in some cases, also structural information. There are two types of protein language models – those trained only on sequences and those trained on both sequences and structures. The protein language models are trained on hundreds of millions and even billions of sequences. They are large because the resulting networks may include even billions of parameters (weights). It is a huge computational project to produce a protein language model because they rely on access to large numbers of GPUs, which is why these usually have been developed at the largest companies (Meta, DeepMind, OpenAI, etc.).

The training procedure leaves out part of the input sequence (or structure data) and trains the language model to predict correctly what the missing letters are. This is properly termed “self-supervised training” since the data used are not labeled in any way, and the learning takes place by blanking out symbols and making guesses, scoring the result, and continuing until scores are good. The approach requires large volumes of data to produce the digested results called embeddings. Transformer networks are effective in keeping track of patterns in the input data, even if the elements are far apart in sequence. Transformer-based large language models gained popularity only in 2019 when GPT-2 was released by OpenAI. Protein language models are more recent; the digested information or embeddings are incredibly valuable and are finding new and exciting applications in the genomics and proteomics world.

How Function Is Identified

Function assignment in the past was routinely performed by sequence matching to find homologous sequences that have known functions, and if the sequence matching score is sufficiently good, with sufficient coverage, then it has often been learned that the query sequence has the same function as its homolog. This same approach continues with many new function prediction methods relying on sequence alignments generated by the protein language model. However, several now do not require the intermediate step of sequence alignment to identify homologs. These use the pLM directly to produce and score a rank-ordered list of homologs. Overall, there are some significant gains from pushing the homolog identification to more remote cases having lower sequence identities, which is extremely important for deep evolutionary studies, and these results often find new relationships for orphan proteins, the proteins that previously were considered to be unique to a specific species or strain and to have no sequence/function homologs.

Traditional Sequence Matching Methods

Traditionally, homologs are found with sequence matching. A scoring system for pairs of amino acid substitutions (substitution matrices) is used to find alignments that have the highest scores and hence most likely to be homologs. These algorithms used align high-scoring pairs by inserting gaps between amino acids and mimic the genetic events such as insertions, and deletions. The two types of sequence alignment that use scoring systems to create alignments are Needleman–Wunsch (NW) for global alignments (Needleman & Wunsch, 1970) and Smith-Waterman (SW) for local alignments (Smith & Waterman, 1981). However, these algorithms require calculations that grow quadratically with the number of input residues and do not scale well. It becomes excessively time-consuming to search a large database of proteins. To solve this scalability issue, methods such as BLAST (Altschul, 1990) and FASTA (Pearson, 1990) perform heuristics and clever methods to reduce the time needed to find sequence alignments. Sequence similarity search with substitution scores performs well when the sequence identity between the query and target protein is > 30%; however, for other pairs that have lower sequence similarity, these methods struggle to distinguish random matches from real homologs. To remedy this problem newer substitution matrices can be used such as (Edgar, 2009) and (Jia and Jernigan, 2021). Not all protein families change their sequence over time at the same rate. For some families, the sequence is conserved for a long time, but for others, it can change quickly (Weisman et al., 2020) This undermines methods that rely only on pair substitution scores. To mitigate this shortcoming, methods such as PSI-BLAST (Altschul, 1997) and CS-BLAST (Biegert & Soding, 2009) were developed to use sequence probabilistic methods that are distilled from multiple sequence alignments. These methods use sequence profiles to find matches.

Earlier Models and Natural Language Processing Influence

Natural language processing (NLP) and protein relationship identification problems are similar in nature. Natural languages such as English also make use of a limited set of symbols, or alphabet, and the ordering of these symbols creates words and sentences, as in proteins, where amino acids and their ordering create functional proteins. Some sentences are similar in their meaning but use different sets of words to convey this meaning. Due to this connection between NLP and protein homolog identification, the bioinformatics field has long been influenced by NLP research. One example is hidden Markov models that are used in speech recognition and later applied to biological sequences. PHMMER (Finn et al., 2011) and HHSEARCH (Soding, 2004) use HMMs to create sequence profiles and align these profiles by state matching and calculating probabilities of alignments to understand which alignment is best. Long-short term memory (LSTM) (Hochreiter & Schmidhuber, 1997) based models have an advantage over HMM based methods in their property of remembering longer contexts. The Hochreiter et al. (2007) work is one example of remote homolog detection that uses LSTM networks. Some other NLP-inspired remote homolog methods applied Latent semantic analysis (Landauer et al., 1998) to PSI-BLAST-derived frequency profiles and used a discriminative support vector machine (SVM) (Vapnik, 1998) to find remote homologs (Dong et al., 2005). Yeh and Chen (2010) used ngrams and latent Dirichlet allocation (Blei et al., 2003) and a SVM to find homologs. For more information on early SVM and LSTM based remote homolog detection methods, one can refer to the Chen et al. (2016) review.

Representing Proteins with Numbers: the Embeddings

One recurrent paradigm that is evident in some of the early methods, such as PSI-BLAST, is the creation of real-valued vector representations of the query protein’s amino acids. PSI-BLAST creates frequency profiles. Another example is the use of Kidera factors (Kidera et al., 1985). This represents amino acids with some selection of their physical properties. Another vector type is a representation of amino acids with a simple 20-dimension vector where all the entries are zero except for the amino acid in that position taken as one. This is called “one hot encoding”. This type of vector representation is generally named embeddings in the NLP field. An early paper that specifically mentions embeddings for proteins is by Melvin et al. (2011) which uses e-values derived from PSI-BLAST to create embeddings for each protein. To understand embeddings better, we can think of embeddings simply as a different dimension. A protein can be defined by its amino acid sequence, or a with a 3-dimensional structure. Embeddings present a way to represent proteins in a different dimension than 1-d sequence or 3-d structure and can be thought of as supportive of these representations, not as a rival.

LSTM-based methods - the Emergence of the First pLMs

Deep learning models that are trained to generate embeddings from the input proteins began to gain attraction due to the distilled and contextual information presented in the embeddings. Schwartz et al. (2018) trained a convolutional neural network (CNN) with 70 million proteins. This model had an autoencoder at the output that distilled the embeddings into a vector of size 256 which allowed finding homologs that were undetectable by traditional methods. Bepler and Berger (2019), Alley et al. (2019), and Heinzinger et al.(2019) trained LSTM-based protein language models with 21.8 million, 24 million, and 33 million sequences, respectively. These language models were shown to learn amino acid physico-chemical properties, secondary structure, and evolutionary information. With the advent of attention networks (Vaswani et al., 2017) transformer based pLMs such as ESM1b (Rives et al., 2021) and ProtTrans (Elnaggar et al., 2021) were developed. These newer pLMs are trained with masked language modeling objectives where a percentage of input amino acids are masked, and the model is trained to recover them. This unsupervised learning objective allows the use of millions of protein sequences for training. Later, the MSA transformer (Rao et al., 2021) used MSA data to generate embeddings which were then used for long-range contact predictions that performed better than Potts models (Rao et al., 2021). Later, it was shown that column attention of the MSA transformer captures the phylogeny signal, and the MSA transformer was able to separate the contact signal from the phylogeny signal while the Potts model struggles to do this (Lupo et al., 2022). Some other pLMs are Ankh (Elnaggar et al., 2023), and xTrimoPGLM with 100 billion parameters (Chen et al., 2024).

Some recent language models incorporated structural information with the help of 3di tokens (Michel van Kempen et al., 2023), which was developed by distilling information from protein structures with an autoencoder generating 1-d sequence representation of structures. These 1-d tokens defined by the structure are used along with amino acid sequences in the masked language modeling. ProstT5 (Heinzinger et al., 2023) and SAProt (Su et al., 2024) are examples of such transformer models. This list of protein language models is not exhaustive but merely examples of the types of methods that have been developed.

Remote Homolog Detection

This paper reviews remote homolog detection methods that use pLM embeddings. We will discuss different types of methods and how they perform remote homology prediction tasks while highlighting their important contributions.

Remote homolog detection methods that are based on pLM embeddings can be classified into two different classes. These are alignment and discriminative methods. Alignment methods align the protein embeddings in a similar fashion as Needleman-Wunsch or Smith-Waterman alignments and generate pairwise alignments, which are then scored to find protein relationships similar to how BLAST reports its results. Discriminative methods generate a measure of similarity, such as distances of the embeddings. The distance measures can be used to cluster protein sequences and nearest neighbors can be used for finding remote homologs.

Discriminative Methods

While it’s not a protein language model, an early example, D-SPACE (Schwartz et al., 2018) used a 100 million parameter CNN to generate protein embedding vectors of size 256 for homology detection. The similarity was calculated using the Euclidean distance between pairs of embeddings. knnProtT5 (Schütze et al., 2022) created per-protein embeddings from ProtT5, searched k-NN using FAISS (Matthijs Douze et al., 2024) vector (embedding) similarity search library from Meta and performed best when combined with MMseqs2 (Steinegger & Söding, 2017). Using contrastive learning, ProtTucker (Heinzinger et al., 2022) trained feed-forward neural networks on ProtT5 embeddings. The model learns to increase the distance between unrelated protein pairs while decreasing the distances between homolog pairs. These transformed embeddings are then used for homolog detection. PROST (Kilinc et al., 2023) optimized ESM1b embeddings by identifying the best-performing layers and compressing the embeddings of the top 2 layers into 475-byte fixed-size per-protein representations and showed that this compression scheme is better than taking the mean of the embedding along the sequence axis. This method outperformed traditional sequence search methods on global homology tasks. ProSSL (Kogay et al., 2023) employed a 10-layer transformer trained on secondary structure prediction of 200,000 bacterial proteins. When fine-tuned for specific tasks like finding terminase or portal proteins, it achieved similar accuracy to PROST, demonstrating the potential of smaller, specialized models. DHR (Hong et al., 2024) fine-tuned ESM1b-based encoders - one for database proteins and another for query encoding using contrastive learning, outperforming Foldseek on challenging cases. DCTdomain (Iovino et al., 2024) improved the PROST method by first splitting proteins into domains and searching through both domain fingerprints and global representations. This approach enhanced performance on local alignment tasks. It is well known that parsing sequences into domains prior to sequence matching is important (Zvelebil and Baum, 2008).

Alignment-Based Methods

There are a large number of these methods. A selection with a description of the method is given next. The ssa-LSTM (Bepler & Berger, 2019) and mt-LSTM (Bepler & Berger, 2021) used bidirectional LSTM networks to create protein embeddings for predicting contact maps and pairwise similarity matrices. The ssa-LSTM method filters the similarity matrix with a soft symmetric alignment algorithm. The mt-LSTM version incorporated masked language modeling and multi-task training, improving performance over its predecessor. These methods create a substitution matrix from embeddings and employ it to find the alignment between pairs of proteins. Both methods perform better than TM-align on a benchmark created from a subset of SCOP. DEDAL (Llinares-López et al., 2022) employed a 6-layer transformer architecture with a differentiable Smith-Waterman module that is trained end-to-end. It can generate alignments purely from sequence input. TM-Vec (Hamamsy et al., 2023) reduced ProtT5 embeddings to fixed vectors for TM-score prediction, while DeepBlast used CNNs to convert embeddings into substitution scores and gap penalty matrices employing a differentiable NW module. This approach showed better accuracy in identifying bacteriocins then a predicted structure and TM-align pipeline. pLM-Blast (Kaminski et al., 2023) generated substitution matrices from ProtT5 embeddings for local alignments without gap penalties and used this to generate alignments. vcMSA (McWhite et al., 2023) first finds related proteins by protein-level embeddings then creates multiple sequence alignments using per-residue embeddings and residue clusters, outperforming traditional MSA methods. Esmologs (Johnson et al., 2024) predicted 3di tokens, hmmer and hhsuite profiles from ESM2 embeddings by using CNN and finetuning of ESM2. The best performance came from 3di token prediction, achieving accuracies similar to Foldseek without requiring structural information. Soft-align (Harrigan et al., 2024) combined protein-level kNN searches with FAISS (Matthijs Douze et al., 2024) and the employed residue-level embeddings to find reciprocal best hits for alignment. PLMSearch (Liu et al., 2024) combined PFAM clan filtering with ESM1b protein-level embeddings for initial homology search, followed by PLMAlign using ProtT5 per-residue embeddings for alignment. This method approached the accuracy of the Foldseek on a SCOP benchmark. EBA (Pantolini et al., 2024) filtered ProtT5 embedding-based similarity matrices using z-scores, achieving performance similar to Foldseek. CLAlign (You et al., 2024) applied contrastive learning with low-rank adaptation (LORA) (Hu et al., 2021) to ProtT5, to calculate a substitution matrix for alignment. This achieved accuracy comparable to structure-based methods on Malisam (Cheng et al., 2007a) and Malidup benchmark (Cheng et al., 2007b). PEbA (Iovino & Ye, 2024) generated substitution matrices from ProtT5 embeddings using scaled cosine similarity. Applied parameters were 11 for gap opening and −1 for gap extension for local alignments. While not suitable for database-level search, it outperformed DEDAL and vcMSA in alignment quality.

Discussion

The use of protein language models in the field of protein homolog detection is highly promising. Embeddings generated by pLMs contain contextual information needed for accurate homolog detection. Here in this section, we will highlight several important contributions among the methods reviewed.

First, embedding matrices can be converted to substitution scores either directly or with the help of a neural network. These substitution matrices along with gap penalties can be used to generate global or local alignments. Filtering these substitution matrices improve the accuracy as in the EBA (Pantolini et al., 2024) and SSA (Bepler & Berger, 2019) cases. Residue level embeddings can be used to cluster residues of the homolog proteins and allow creating MSAs as in the example of vcMSA (McWhite et al., 2023) and soft-align (Harrigan et al., 2024). However, all these alignment-based methods can be computationally expensive. To solve this issue, one can directly use embeddings to make homolog detection without an alignment. We call such methods discriminative methods. The FAISS (Matthijs Douze et al., 2024) vector search tool can be used to search for similar embeddings as it has been utilized in several methods (Schütze et al., 2022) (Harrigan et al., 2024). Pre filtering candidate proteins as in PLMSearch (24Liu) helps increase the accuracy. Contrastive learning can be utilized to differentiate between homologs and non-homologs (Heinzinger et al., 2022) (You et al., 2024). Different fine-tuned encoders can be used for query and database proteins (Hong et al., 2024).

Using a selected layer(s) of the embeddings can lead to gains over using the last layer or all layers. This was important for the gains in PROST (Kilinc et al., 2023) and (Iovino et al., 2024), see Fig. 1 for per-layer pLM performance. Mostly, it has been missed that the last layer of the pLMs is often not the best for remote homolog detection, but this requires empirical exploration to determine which layers perform best. For the ESM family of pLMs the best layer is usually at the third quadrant. By just selecting a different layer, accuracy can be considerably improved. Figure 1 represents the AUC scores for two different pLMs with two different benchmark sets. Moreover, embeddings contain homology, biophysical, biochemical, and structural data along with noise. One can compress by denoising the embeddings and increase the accuracy and efficiency of remote homolog detection as in the PROST example (Kilinc et al., 2023). By first splitting proteins into domains and performing subsequent searches, one could find remote homologs as well as local alignment-based methods without specifically carrying out alignments (Iovino et al., 2024). These types of discriminative methods have extraordinary speed with high accuracy. Additionally, the distance metric for the discriminative methods contains phylogenetic information. Figure 2 shows a phylogeny generated from PROST scores, where yeast phylogeny can be constructed purely by using the embedding distances without any need for MSA and can be similar in accuracy to the traditional MSA-based methods.

Figure 1. Selecting the Layers for Best Performance.

Per layer performance of ESM2 and ESM1b on different benchmarks. A) Domain Based. This panel was adapted from (Iovino et al., 2024) shows per-layer performance of ESM2’s 12 layer and 30 layer versions. The embedding is compressed using the DCT algorithm and the performance tested on SCOP dataset (Andreeva et al., 2020). B) Panel adapted from (Kilinc et al., 2023). It shows ESM1b per-layer performance with DCT compression on PFAM dataset (Saripella et al., 2016). Both panels show that the last layer of a pLM is not the best layer for homolog detection. Instead, the models peak around a third quarter point in the list of layers, following which the performance drops. This finding is also supported by the work of Valeriani et al. (Valeriani et al., 2023).

Figure 2. Comparative Analysis of Fungal Phylogenetic Trees and Genetic Distances Using PROST and Traditional Methods.

A) a phylogenetic tree generated using PROST’s distance metric between species in the BUSCO set of proteins without the need for an MSA. This tree was generated by neighbor-joining method from the PROST distance matrix. B) a phylogenetic tree created using traditional methods, specifically, by concatenating the MSA of the BUSCO set of orthologs and calculating distances with Phylip (Weisman et al., 2020). Both trees display a notable similarity in topology, particularly highlighting the yeast lineage (shaded in yellow), which encompasses Saccharomyces species. C) comparison of the phylogenetic distances obtained by both methods across various yeast species, indicating a strong correlation between the PROST distances (orange line) and those derived from traditional phylogenetic analysis (blue line). The distances parallel each other and are similar, suggesting that PROST can effectively approximate the genetic distances typically obtained through more computationally intensive methods. Here the distance between a pair of organisms is calculated by taking the average PROST distance summed over all BUSCO proteins.

Previously, it was shown that the dynamics of a protein can be calculated purely from sequence data (Jia et al., 2023). This finding implies that the embeddings transformed from sequences can contain protein dynamic information as well, which can also aid in remote homolog detection where no strong sequence similarity is present and no structural similarity can be obtained because of the dynamics. For example, a protein could be in an open state, whereas its remote homolog could be in a closed state so that a structural similarity tool would not detect a high level of similarity, but instead, the embeddings can help find the similarity.

We conclude by highlighting the rich contextual information contained in the embeddings. These embeddings can be thought of as a high dimensional representation of proteins. With the protein language models it becomes routine to represent proteins with the digested embeddings from either their one-dimensional amino acid sequence or their three-dimensional atomic coordinates. Embeddings present another dimension to represent proteins that can help in identifying sequence and structure relationships. Thus, remote homolog detection with not only structure- or sequence-based methods but with embedding-based methods such as have been described here will increase accuracy for homolog detection. Using methods such as MMSeqs2, Foldseek, and PROST together, or combinations of other methods, holds the promise of complete and reliable annotations of nearly all proteins. It should also be kept in mind that there is some error rate in any gene prediction/identification method, but at least some of those proteins that cannot be annotated are because they may not truly be actual genes/proteins (Baxevanis et al., 2020).

Supplementary Material

Supplementary Information

SI-Table 1 provides a list of remote homolog detection methods that utilize protein language models with summaries of their contributions.