Mapping of Antibody Epitopes based on Docking and Homology Modeling

Abstract

Antibodies are key proteins produced by the immune system to target pathogen proteins termed antigens via specific binding to surface regions called epitopes. Given an antigen and the sequence of an antibody the knowledge of the epitope is critical for the discovery and development of antibody based therapeutics. In this work, we present a computational protocol that uses template-based modeling and docking to predict epitope residues. This protocol is implemented in three major steps. First, a template-based modeling approach is used to build the antibody structures. We tested several options, including generation of models using AlphaFold2. Second, each antibody model is docked to the antigen using the FFT based docking program PIPER. Attention is given to optimally selecting the docking energy parameters depending on the input data. In particular, the van der Waals energy terms are reduced for modeled antibodies relative to X-ray structures. Finally, ranking of antigen surface residues is produced. The ranking relies on the docking results, i.e., how often the residue appears in the docking poses’ interface, and also on the energy favorability of the docking pose in question. The method, called PIPER-Map, has been tested on a widely used antibody-antigen docking benchmark. The results show that PIPER-Map improves upon the existing epitope prediction methods. An interesting observation is that epitope prediction accuracy starting from antibody sequence alone does not significantly differ from that of starting from unbound (i.e., separately crystallized) antibody structure.

1. INTRODUCTION

Antibodies are protein assemblies that are found naturally^1,2 or are produced as a response to pathogens as part of the adaptive immune system in vertebrates.³ They bind to solvent-exposed proteins called antigens on pathogen surfaces in order to interfere, immobilize or destabilize pathogen activity.⁴ Specifically designed antibodies can work as drugs due to their diversity and ability to specifically bind to antigens with high affinity. Therefore, understanding and accurately predicting the exact antibody-antigen (Ab-Ag) interface is paramount to exploiting antibodies’ capabilities.⁵ Predicting this interface, also called epitope mapping, is essential in efforts towards developing vaccines,⁶ designing novel antibodies,⁷ and understanding immune responses.⁸

Antibody-based therapeutic discovery process had been traditionally hampered by the inability to obtain large numbers of antibody sequences. However, recent advances in high-throughput sequencing^9,10 have removed this obstacle. Consequently, given a target antigen, the new bottleneck in the discovery process is speedy and accurate prediction of epitopes for the sequenced antibodies. Experimental techniques for epitope prediction are laborious and expensive and cannot be used in a high-throughput manner. Therefore computational sequence specific epitope mapping is an important problem. Several research groups have developed computational tools to predict surface residues that will most likely be in an interface without the knowledge of the partnering antibody.^11–14 Some of these methods were implemented into public servers such as the Spatial Epitope Prediction for Protein Antigens (SEPPA)^12,13 and BEpro.¹⁴ SEPPA utilizes a logistic regression algorithm with features such as antigen residue surface accessibility and propensity of unit-triangle patches (3 residue-groups on the antigen’s surface) to score the surface residues.^11–13 BEpro adds amino-acid propensity scale and side-chain orientations to other features.¹⁴

The antibody-agnostic methods SEPPA and BEpro had some success in epitope mapping. However, it is important to highlight that an epitope is, by definition, a relational entity and that epitope mapping ought to be for a specific antibody-antigen pair. This is supported by several known antigens with different affinities and different epitopes to different antibodies. One well-studied example is hen egg lysozyme (HEL) which is crystallized with four different antibodies in the PDB structures 1BVK, 1DQJ, 2I25, and 1MLC with little overlap of their epitopes.^15–18 Therefore, consideration of both the antibody and the antigen in epitope mapping is not only appropriate but also should serve as an additional information from the antibody’s surface residues (mostly the Complementarity Determining Regions (CDRs) ) that can potentially increase the accuracy⁶. Thus, it follows that computational methods by docking should be a natural approach to epitope mapping. For example, Krawczyk and colleagues used docking in their epitope mapping server called EpiPred.¹⁹ They employed ZDOCK, a protein docking program to generate models which are in turn used to score potential epitope patches determined by geometric fitting.^19,20 More recently, Sikora and colleagues docked the SARS-CoV2 spike protein to monoclonal antibodies to assess the accessibility of potential epitope candidates.²¹ Here we use PIPER, a docking program based on fast Fourier transform (FFT) correlation approach to speedily calculate the energy of billions of possible docking poses.²² Each pose is ranked by an interaction energy which is a linear combination of van der Waals energy terms (repulsive and attractive), electrostatics energy (Coulombic and Born approximations), and a structure-based pairwise statistical potential. A unique pairwise statistical potential was introduced for antibody-antigen complexes in 2012 which improved Ab-Ag docking accuracy significantly.²³ In a recent comparative study PIPER, implemented in the server ClusPro,^24–26 was reported as the leading method for global antibody-antigen docking in terms of accuracy if no a priori information on the interaction site was available.²⁷ Recently, Hua et al. utilized ClusPro’s top 30 models to inform site-directed mutagenesis to localize epitopes, thereby reducing the number of mutations required.²⁸

In this paper we extend the methodology beyond the currently available epitope mapping programs and combine the docking approach with template-based modeling, thereby enabling the prediction based on the antibody sequence and the structure of the antigen. The resulting method will be referred to as PIPER-Map. PIPER-Map builds on the recently published template-based modeling method^29,30 and integrates it with contact prediction from the docking poses in order to rank the likelihood of residues being in a given antigen structure’s epitope from the sequence or structure of the partner antibody. The scores of the antigen surface residues are obtained from thousands of low-energy Ab-Ag docking poses from PIPER. The consensus scores of all the docked models and all the templates are averaged to score the residues. In order to not penalize possible clashes from uncertain modelled structures from the template-based modeling stages, the weight of the van der Waals component of the interaction energy was reduced. Although, there is still more work in regards to accuracy of epitope prediction, we demonstrate that PIPER-Map improves upon established epitope predicting approaches such as SEPPA and EpiPred. Finally, we explore using the recently introduced AlphaFold2^31,32 tool for antibody model building, and demonstrate that it does not provide higher accuracy in predicting epitope residues than PIPER-Map.

As will be shown, epitope mapping is a challenging computational problem. In fact, all current methods have somewhat limited accuracy, and substantial further work will be required to improve the results, particularly if only unbound protein structure or sequence information is available for the antibody. However, based on our results PIPER-Map is slightly better than the other methods currently available. One of the major causes of the limited accuracy is the lack of a sizeable nonredundant set of antibody-antigen (Ab-Ag) complex structures as less than 25% of the Ab-Ag complexes found in the Protein Data Bank (PDB) are unique when 70% sequence identity threshold is used as a cutoff for the antigen.³³ We will also show that the neural net based program AlphaFold2, which generally provides remarkable accuracy in modeling proteins and protein-protein complexes, yields much less accurate results in modeling antibody structures and antibody-antigen complexes. It appears that application of Alpahafold2 to antibodies is hindered by the lack of co-evolutionary information, and that the limited set of antibody-antigen complex structures prevents more specific training of the neural net.

2. METHODS

2.1. Template-Based Modeling of Antibodies

Our method starts with an antibody sequence (Figure 1 - Input A) and a structure of the antigen (Figure 1 - Input B). First, we conduct a BLAST sequence search for homologues of the antibody in the Protein Data Bank (PDB). The search is done for the heavy and the light chains separately and restricted to homologues with sequence identity above 20% and e-value below 1e-40 (when no homologues found, the e-value threshold value is increased to 1e-20). The lower threshold for sequence identity is introduced to avoid candidates adding only noise to the homology modeling. The number, 20%, was chosen because it is generally the sequence identity barrier that gives at least acceptable homology or template-based models and during recent CASP/CAPRI competitions. If homologues are found, then PDB structures that met the sequence constraints for both the heavy and the light chains are retained. From this filtered list, we calculate the CDR3 sequence identity (CDR3SI), defined as the number of identical residues in the CDR3 regions (both light and heavy chains separately), divided by the alignment lengths, and the CDR3 sequence similarity (CDR3SP), defined as the sum of the number of identical residues and the number of conservatively aligned residues in the CDR3 regions, divided by the alignment lengths. For each homologue we compare the CDR3SI values of the heavy and light chains, select the smaller of the two, and based on this value rank all the candidates by descending order. Repeating the process with CDR3SP, the top five candidates from each ranking are moved onto the next stage (note that the two rankings may share none or all of their candidates). At next stage, we construct a single template-based model for each of the homologues (up to 10) selected. Using MODELLER tools,³⁴ the given antibody sequence is aligned to the selected template structures as symbolized in Figure 1. The program models the backbone atoms of the non-aligned residues and all sidechains, while the backbone atoms of the aligned residues are held fixed at the template coordinates. The single best model proposed by MODELLER for each template is retained for the next step in the mapping process.

FIGURE 1.

Outline of the epitope mapping protocol. If inputs are the antigen structure and the antibody sequence, homologues are found for the antibody sequence and the an ensemble of homology-based models are obtained. These are then docked using PIPER. If inputs include the structures of both the antigen and antibody, then the two are docked. In both cases, several conformations (N=1000) are retained and each antigen residue’s frequency of appearance in the interface are calculated and weighted by the PIPER energy.

2.2. Docking Antibody and Antigen

Starting with the model generated in the previous step or with an unbound antibody structure, PIPER-Map performs global antibody-antigen rigid-body docking using the PIPER program²² that directly docks two protein structures. The known antigen structure is docked to each antibody model (if only antibody sequence was known), or to the sole antibody structure (if initially given by user). PIPER employs the fast-Fourier transform (FFT) correlation approach to represent the interaction energy of the complex as a weighted sum of correlations between the fixed antibody and rotationally and translationally mobile antigen grids. As a result, an exhaustive conformational sampling of the 6-dimensional energy landscape becomes computationally feasible. We assume the same standard level of discretization as used in PIPER: 70,000 rotations from the Sukharev quasi-uniform grid sequence²² (approximately 5 degrees by Euler angular step) and translational grid step size of 1Å. The special antibody-antigen asymmetric version of the DARS potential was used for docking the obtained antibodies (or starting antibody structure if given)²⁵.

Following the protocol outlined in detail in a recent paper,³⁵ all antibody residues except for CDRs are masked. If an antibody structure is known and it is not a homology model, the protocol calculates the total antibody-antigen docking potential E as the following linear combination: E = 0.5 E_rep − 0.2 E_attr + 300 E_Coul + 30 E_Born + 0.2 E_DARS (where E_rep and E_attr denotes repulsive and attractive components of the van der Waals energy respectively, E_Coul denotes a Columbic term describing the electrostatic interaction energy, E_Born denotes a generalized Born type polar solvation energy term, and E_DARS denotes another solvation term based on the structure-based statistical potential). This combination of energy terms will be referred to as C003 further in the paper.

For models we use coefficients with reduced van der Waals (vdW) contribution. The first option, called “No vdW”, has the zero coefficients for both van der Waals contributions E = 0.0 E_rep − 0.0 E_attr + 300 E_Coul + 30 E_Born + 0.2 E_DARS, and will be referred to as parameter combination C007. The second option, “Half VdW”, has the weight for the attractive van der Waals term E_attr halved, thus it is defined as E = 0.5 E_rep − 0.1 E_attr + 300 E_Coul + 30 E_Born + 0.2 E_DARS and will be denoted as parameter combination C005. However, it should be noted that the commonly used maximum repulsive and minimum attractive van der Waals thresholds are still in place for all coefficients. As in the ClusPro server, the best-scored pose per rotation is retained, resulting in a total of up to 70000 docked poses for further analysis.

2.3. Ranking of Residues Based on Epitope Likelihood

Once the PIPER docking poses and energies are obtained for the given antibody structure or for each antibody model i in the set of homology models (which can include one to 10 models, depending on the number of suitable templates found) and the given antigen structure, the 1000 lowest energy poses are retained. For each such pose, j, the number of antigen surface atoms that are in contact with the corresponding antibody, l_ij, is counted. Note that any heavy atom on the antigen surface within a 5Å distance from any of the antibody surface heavy atoms is considered to be in contact with the antibody. For each antigen atom on the interface, we calculate a Boltzmann-weighted normalized contact “occurrence” as follows:

$${\upsilon }_{ij}^{atom}=\frac{e^{-\beta ({\epsilon }_{ij}-{\epsilon }_{io})}}{l_{ij}}$$

where ε_ij and ε_io are the jth and the lowest PIPER energy scores (energy of best pose) of the ith antibody structure in the ensemble, and a value of 0.01 was used for β to scale the relative energy scores. After summing the atomic contributions, ${\upsilon }_{ij}^{atom}$, over j (i.e., the 1000 retained docking poses) and averaging over i (i.e., the different antibody models if more than one), an epitope likelihood score that indicates how often the atom participates in the antibody-antigen interface of low-energy models predicted by PIPER is obtained. This likelihood score is shown as the B-factor value in the final output which is a PDB file. It helps to visually highlight different regions with their respective likelihood of being in the epitope.

For epitope prediction accuracy evaluation to compare with peer servers, the atom likelihoods are converted to residue likelihoods by summing up the atomic contributions for each residue. Note that summing the atomic likelihood scores naturally gives better scores to bigger residues with more exposure to solvent. This bias may have to be accounted for by the user.

2.4. Protein Datasets

In order to test the method, the widely used protein-protein docking benchmark version 5.0 (BM5)³⁶ from the Weng lab was selected. Data from this set is shown from Figure 2 through Figure 4. Vreven and colleagues, in order to avoid redundancy, included an antibody-antigen complex only if two conditions are met: the antigen was not in the same SCOP (Structural Classification of Proteins)³⁷ family and it did not share more than 80% of the interface residues with another³⁶. The BM5 set contains 40 antibody-antigen complexes. Twelve of them, referred to as unbound-bound or bound for short, have their antibodies crystallized only in bound form to the antigen in the complex in the PDB. The rest of the cases (N=28), referred to as unbound-unbound or unbound for short, the crystal structures of the antibody alone and in complex with the partnering antigen exist in the PDB. In both subsets, the antigens were independently crystallized and deposited in the PDB.

FIGURE 2.

Fine-tuning the parameters of the method. (A) Impact of loosening the penalty on shape complementarity on the ROC AUC score for the 40 antibody-antigen complexes in the benchmark set BM5. Coefficient 003 is the normal antibody-antigen coefficient set; coefficient 005 has it’s attractive Van der Waal’s potential halved from that of C003; coefficient 007 has weights of zero for both attractive and repulsive Van der Waal’s potentials. (B) Impact of varying the β factor during Boltzman weighting (X-axis) and the number of docking poses (color of lines) to consider for final scoring of each atom on the ROC AUC score.

FIGURE 4.

Examples where an ensemble of homologues was used to predict epitopes as well as or better than when antibody crystal structures were used. The native antibody and antigen binding mode is shown via PyMol where the native antibody is shown in light pink cartoon and the antigen is shown in blue surface. The predicted epitope residues are shown in a spectrum from red to blue where red is highest scoring (most likely to be a true epitope). The antibody inputs were (A) bound X-ray structure of antibody extracted from the complex 2JEL, (B) antibody sequence of 2JEL, (C) unbound X-ray structure of the separately crystallized antibody 2W9D, and (D) sequence of 2W9D.

One of the unbound targets (PDB ID 2I25) – a single-chain shark antigen receptor – did not work using EpiPred due to the fact that EpiPred expects antibody with both heavy and light chains. The performance of PIPER-Map to that of SEPPA and EpiPred was assessed for the 39 other Ab-Ag BM5 cases. Furthermore, to test the fine-tuned parameters, 23 new antibody-antigen complexes newly added to the docking benchmark set (denoted BM5.5)²⁷ were used. Two of the cases were discarded because PIPER-Map was unable to provide antibody models when starting from sequence.

2.5. Performance Metrics

The epitope prediction performance was evaluated using the native bound complexes from which the true epitope residues of the antigen are within 5 Å from the nearest antibody heavy atom.^33,38 The most widely used performance measure among epitope mapping servers is the Area Under the Receiver Operating Characteristic curve (ROC AUC), and hence it was also used to compare the performance of PIPER-MAP with that of other probabilistic servers such as SEPPA. Other studies also use the F score (the harmonic mean of precision and recall).^19,39 The true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN) at the selected cut-off values were considered for each antibody-antigen target and used to calculate the F score:

$$F=\frac{TP}{TP+\frac{1}{2}(FP+FN)}$$

This metric gives a balanced view of recall and precision which are the essential metrics for classifiers. Since no likelihood cutoff value was determined, for comparison, the top 10, 20, 30, 40, and 50 residues were considered and the F scores were obtained for each cutoff value. Indeed, most epitope lengths fall within that range,⁴⁰ and the average epitope length for the antibody-antigen targets in BM5 was 21 residues. For comparing PIPER-Map’s epitope prediction using antibody structure input with SEPPA and EpiPred, mapping jobs were done for all 61 cases mentioned using the public servers. For homology modeling mode, PIPER-Map is only compared with EpiPred since the others are antibody-agnostic. The top antibody model was obtained from the end of the modeling stage in the PIPER-Map protocol and used as the antibody input for EpiPred. The models were generated by setting a maximum of 80% sequence identity threshold for finding homologues. Previously used homology benchmark sets were obtained with more relaxed sequence identity (90%) thresholds,^19,41,42 and thus were not used for the performance evaluation.

3. RESULTS AND DISCUSSION

We present three types of applications of PIPER-Map depending on the initial information available on the antibody that may be given by 1) a crystal structure, 2) a computationally predicted structural model, or 3) only the amino acid sequence. The following section illustrates the fine-tuning of parameters as well as the performance of PIPER-Map for each of those different inputs. Furthermore, the results are compared with peer servers using the BM5 set already discussed^27,36 as well as new Ab-Ag additions to BM5.5 that were not used for fine-tuning the parameters.²⁷ Twenty-eight of the 40 Ab-Ag cases in BM5, which were unbound-unbound, are used mainly to compare the performance of for the three types of inputs. For option 2, starting from a computationally predicted structural model, the neural net based protein structure prediction program AlphaFold2 was used to model the antibodies using the default parameters currently made public.^31,32 In all options, the resulting conformations from the docking stage are analyzed to obtain epitope scores for each residue. As PIPER strives to find the most energetically favorable conformation for each of its pose, it is expected and observed that the more frequently it predicts an atom to be in the epitope, the more likely it is for it be in the true epitope. PIPER performs rigid-body docking in all cases discussed here. The assumption of rigidity is not an accurate representation of the antibody-antigen interface especially given the flexibility of the complementarity determining regions (CDRs) of the antibody. In order to address this flexibility, our procedure generates an ensemble of antibodies and their respective CDR loops to account for the inaccuracy.

3.1. Fine-tuning the Energy Parameters

Whether the starting point of epitope mapping is the antibody sequence, computational structure or crystal structure, the docking stage and the subsequent scoring are an avoidable parts of the protocol. As such, an optimal selection of the parameters that are part of the docking process that are fit for mapping is needed. The major parameters that needed to be fine-tuned are as follows.

A). Coefficient weights for energy potentials

For crystal structure inputs, a previously introduced antibody-specific asymmetric DARS potential was integrated to PIPER and an optimal weight set was identified^23,35. However, for computationally modelled or sequence input (options 2 and 3), the same set was not optimal. Therefore, 11 different coefficient sets were tested especially focused on varying the weight of the van der Waals components of the PIPER energy on template-based modelled antibodies. The resulting ROC AUC scores for all 40 cases in the benchmark set BM5 are shown in Figure S1. The general trend shows that reducing or removing the van der Waals components improves the results to varying degrees when using homology-modelled antibodies. The best performing coefficient sets were C003 (”Standard”), C005 (“Reduced vdW”) and C007(“No vdW”). As described in section 2.2, the set C007 loosens the complementarity constraints set by removing both repulsive and attractive van der Waals potentials while the coefficient set C003 is what is currently used in ClusPro for antibody-antigen docking intended for use with X-ray structures. C005, is another weight set whose attractive van der Waals coefficient is reduced by half. The ROC AUC results for all 40 cases are compared between C003, C005 and C007 in Figure 2A. The average ROC score rises from 0.712, 0.729 and 0.747 from C003 to C005 to C007, respectively, for all these 40 cases. For the 28 unbound cases, the more realistic subset, the ROC score rose from 0.704 to 0.721 to 0.732 respectively. These additional weight sets avoid penalizing possible steric clashes by reducing the weights of van der Waals energy terms and subsequently increase epitope prediction accuracy notably when only the sequence of the antibody or homology-modelled structure is given.

B). Beta (β) and the number of docking models considered

As discussed in section 2.2, the antibodies were treated as the receptors and the non-CDRs were masked automatically and docked to the unbound antigens to obtain 70,000 poses. To score the antigen’s surface residues, two parameters ought to be systematically tested: the number of poses to consider when calculating the frequency of residue in the interface, and the β factor in the Boltzmann weighting of each pose’s score. The effect of varying β and the number of decoys to get the best average ROC AUC score of all 40 cases is shown in Figure 2B. When all 70000 poses are taken, for instance, heavy weighting (represented by β =1/15) is needed for the best ROC AUC. When taking far fewer poses, like N=1000 poses, a smaller weighting (β =1/100) was found to yield the best ROC AUC. This implies that PIPER’s scoring function can discriminate among the poses relatively well. Overall, the best combination was found to be β = 1/100 which conveniently reduced our required number of poses for probability calculation to only 1000 poses.

3.1. Epitope Mapping Starting from Antibody Crystal Structure

Being the simplest option provided by PIPER-Map, this option requires the most prior information about the antibody before mapping. In terms of ROC AUC, PIPER-Map obtained an average score of 0.763 as shown in Table 1. Figure 3 shows the average F scores of 27 of the 28 unbound cases to be 0.304 from when the top 10 up to top 50 residues, ranked by the PIPER-Map score, are considered for each case (EpiPred failed in one case). The average ROC AUC score for just the 12 bound Ab-Ag complexes in BM5 was 0.822 (Table 1), while the F score was 0.297.

TABLE 1:

ROC scores of PIPER-Map performance with X-ray, homology modeling, and AlphaFold2 modeled structures of the antibodies as inputs for all antibody-antigen complexes in BM5.

PDB of complex	X-ray structure†	Homology modeling‡	AF2, no template§	AF2 with template ¶
1AHW	0.920	0.965	0.957	0.955
1BGX	0.271	0.397	0.342	0.361
1BJ1	0.584	0.733	0.382	0.328
1BVK	0.636	0.719	0.704	0.697
1DQJ	0.644	0.709	0.515	0.589
1E6J	0.863	0.754	0.680	0.698
1FSK	0.918	0.736	0.666	0.736
1I9R	0.766	0.648	0.597	0.631
1IQD	0.849	0.923	0.940	0.942
1JPS	0.973	0.973	0.951	0.952
1K4C	0.738	0.678	0.505	0.530
1KXQ	0.939	0.902	0.464	0.471
1MLC	0.477	0.325	0.336	0.348
1NCA	0.847	0.672	0.865	0.843
1NSN	0.752	0.640	0.733	0.676
1QFW	0.916	0.836	0.645	0.656
1VFB	0.858	0.714	0.757	0.760
1WEJ	0.730	0.559	0.556	0.581
2FD6	0.658	0.656	0.814	0.800
2HMI	0.947	0.847	0.892	0.866
2I25	0.802	0.701	0.857	0.784
2JEL	0.893	0.939	0.888	0.924
2VIS	0.684	0.705	0.833	0.855
2VXT	0.796	0.893	0.846	0.844
2W9E	0.881	0.952	0.920	0.934
3EO1	0.841	0.884	0.821	0.841
3EOA	0.566	0.710	0.719	0.708
3G6D	0.919	0.940	0.939	0.927
3HI6	0.501	0.566	0.473	0.475
3HMX	0.851	0.889	0.860	0.840
3L5W	0.883	0.960	0.949	0.953
3MXW	0.935	0.890	0.716	0.725
3RVW	0.848	0.459	0.562	0.592
3V6Z	0.434	0.536	0.380	0.403
4DN4	0.925	0.952	0.949	0.945
4FQI	0.599	0.767	0.823	0.783
4G6J	0.557	0.504	0.450	0.459
4G6M	0.881	0.862	0.838	0.832
4GXU	0.739	0.677	0.795	0.866
9QFW	0.718	0.704	0.760	0.742
Bound	0.822	0.771	0.694	0.695
Unbound	0.738	0.736	0.726	0.732
Total	0.763	0.746	0.716	0.721

^†Using X-ray structure of antibody

^‡Internal homology modeling by PIPER-Map

^§Using AF2 model generated without templates

^¶AF2 model generated with template.

FIGURE 3.

Epitope mapping performance comparing PIPER-Map with SEPPA and EpiPred when tested on 27 unbound-unbound antibody-antigen complexes in the benchmark set BM5 using the F score metric. Their scores at different cut-off thresholds when the antigen residues are ranked by the obtained scores. The metrics are averaged over the 27 complexes that worked for all three methods.

Figure 3 compares the performance by PIPER-Map to that of the epitope prediction servers SEPPA and EpiPred. SEPPA 3.0 was chosen for comparison as it was shown to be the best epitope mapping servers in the recent publication by Zhou and colleagues¹³. As noted previously, it is an antibody agnostic method, meaning the only input required is the antigen structure. Figure 3 shows that PIPER-Map outperforms SEPPA in unbound-unbound cases in BM5 in terms of F scores. The ROC AUC scores obtained were 0.738 and 0.703 for PIPER-Map and SEPPA3.0 respectively (Table S1). Thus, considering both antigen and antibody structures provides PIPER-Map with valuable information on the interface that gives a 4.9% improvement over SEPPA 3.0 despite not being reinforced by machine learning.

The other peer-server chosen for comparison, EpiPred is similar to PIPER-Map in the sense that it requires information on the antibody as well as antigen structure. However, unlike the others and PIPER-Map, EpiPred yields a deterministic prediction of three non-overlapping epitope patches ranked by likelihood. In order to compare with the other methods with F scores, the same arbitrary high scores were given to all residues in the top-ranked epitope patch, followed by an arbitrarily lower score for the second-ranked epitope residues and so on. Figure 3 shows the performances of PIPER-Map, SEPPA 3.0 and EpiPred using F scores for the 27 of the unbound Ab-Ag complexes in BM5 (EpiPred failed to give results for PDB ID 2I25). Taking the top 20 ranked residues, PIPER-Map’s F scores show a 10% and 60% improvement on SEPPA and BEpro respectively while doubling that of EpiPred. For 23 of the 27 cases, PIPER-Map correctly predicted one true epitope residue in its top 20 ranked residues. SEPPA was able to do the same for 24 of the 27 cases. EpiPred, on the other hand, failed to have any true epitope residues in its top 20 residues for 13 of the 27 cases. For detailed look at each case, Table S2 shows the true positives in the top 20 ranked residues for all 3 methods compared and all 40 Ab-Ag complexes in BM5 (with no EpiPred results for 2I25).

3.2. Epitope Mapping Starting from Computationally Predicted Antibody Structure

As noted by Marks and Deane, most antibody modeling programs can generate models within 3Å RMSD of the native structure.⁴³ However, most errors occur in the H3 loop. Therefore, to take into account that models tend to have less reliable H3 loops, PIPER-Map allows for more steric clashes than the X-ray structures during docking, and uses a special PIPER coefficient set that reduces the steric penalty. Especially, the recent introduction of AlphaFold2³² and its availability to the public,^44,45 makes accurate computational prediction of most obligatory complexes and single chain proteins possible⁴⁶. Therefore, PIPER-Map’s performance for mapping epitopes using AlphaFold2 predicted antibodies was tested using the BM5 cases to compare results with antibody X-ray structure and sequence inputs (Table 1). AlphaFold2 was used to model antibodies with and without templates. Adding templates to predict antibody structures did not improve epitope prediction for bound cases, while having a slightly positive impact on that of unbound targets. The ROC score for bound cases (with and without templates) was 0.695, while for unbound cases it increased to 0.726 and 0.732 with and without templates respectively. The latter case is comparable to both crystal structure input (ROC = 0.738) and using PIPER-Map internal homology modeling (ROC = 0.736) that will be further discussed in section 3.3. Table 1 also includes results obtained from PIPER-Map starting from antibody sequences (discussed in the next subsection), demonstrating that AlphaFold2 is not necessarily the best method for modeling antibodies⁴⁶. Furthermore, the table also shows that correct bound X-ray structures of the antibody increase epitope prediction accuracy by 11% and 6.5%, respectively, relative to having just the unbound antibody structures or only homology models. The fact that the use of bound antibody structure outperforms the use of the AlphaFold2 generated models (both with and without templates) confirms that input of an antibody structure closer to its conformation in the complex leads to better epitope prediction accuracy.

3.3. Epitope Mapping Starting from Antibody Sequence

Next we evaluated performance of the method when starting from sequence alone. As described in section 2.1, the homologues considered had no more than 80% and no less than 20% global sequence identity (GSI) to the target antibody. Recent papers on epitope prediction used homologues with up to 90% GSI, which is too close in our view. Table 1, column 2 shows the results obtained from this starting point and compares it with crystal input as well AlphaFold2 inputs. For the 12 bound cases, internal homology had an ROC score of 0.772 compared to 0.822 when using the native bound antibody as input – a mere 6% decrease in performance. For unbound cases, the ROC decreases by less than 0.01% - essentially negligible difference. With a ROC AUC score of 0.736 for the 28 unbound antibody-antigen complexes, the accuracy of the epitope mapping is not that far behind the results obtained for unbound X-ray structures. The results in Table 1 suggest that the use of homology models provides very similar accuracy to that of using separately solved antibody structures, indicating that due to the flexibility of CDRs information on the unbound structure of the antibody does not provide substantial advantage over antibody models.

When using computational models, the placement of the H3 loop is very important for an accurate epitope mapping. To account for the flexibilities from the templates or the inaccurate modeling, PIPER-Map uses an ensemble of models as described. There are some cases where using the internal homology approach performs better than using X-ray structure of the unbound antibody. Two examples of how the ensemble approach is able to compensate for the flexibility of the CDR loops are complexes 2JEL⁴⁷ and 2W9E⁴⁸ in Table 1. For 2JEL the JEL42 Fab fragment was not separately crystallized, and hence its structure is taken from the complex. The PDB structure 1POH was available for the antigen, the of Escherichia coli histidine-containing phosphocarrier protein HPr. Using the bound crystal structure of the Fab fragment, PIPER-Map resulted in the ROC AUC score of 0.893. Somewhat surprisingly, this score was improved to 0.939 when starting from the sequence of the JEL42 Fab fragment and using the internal homology modeling tool of PIPER-Map. The difference is visually depicted in Figure 4A (X-ray structure input) and Figure 4B (sequence input). The use of homology models identifies a more focused region as the predicted epitope, which overlaps well with the true interface. The second example, 2W9E, had its antibody component separately crystallized and deposited in the PDB as 2W9D, which is the monoclonal antibody ICSM 18. In the complex 2W9E the antibody is bound to the human prion protein, whose structure was separately determined by NMR and deposited to PDB as 1QM1.⁴⁹ With this example, while both the homology-based and crystal structure approaches are able to find a region on the antigen overlapping well with the true epitope, the crystal-based approach also predicts an additional region on the other end of the protein as a potential interaction site (Figures 4C and 4D). However, the homology-based approach recognizes regions much closer to the true interface. Moving from X-ray input to homology models of the antibodies in the above two examples increases the ROC AUC scores by 5.2% and 8.1%, respectively, and thus showcases the added benefits of the ensemble approach, which generates and docks a variety of homology models rather than a single X-ray structure.

Comparison with EpiPred was also done on the newly updated docking benchmark set (referred to as BM5.5) which added several Ab-Ag complexes not found in BM5²². PIPER-Map was tested on 21 of the new Ab-Ag cases and compared with EpiPred. Figure 5 shows the F scores of PIPER-Map with the unbound structures as input, just with the sequences as input, and EpiPred with the unbound structures as input. Interestingly, PIPER-Map performs slightly better (0.204 versus 0.196) with the template-based approach than with the unbound crystals when the top 30 ranked residues are considered. When top 40 and top 50 residues are considered, however, PIPER-Map starting from the unbound X-ray structure outperforms PIPER-Map that starts from just the sequence. With the unbound structures as input, PIPER-Map correctly identified one true epitope residues in its top 30 residues for 20 of the 21 complexes, while correctly predicting more than 10 true epitope residues for only one complex. When starting from sequences, the internal homology approach predicted at least one true epitope residue for only 15 of the 21 complexes while correctly predicting more than 10 true epitope residues for the 5 of the 21 targets. This further emphasizes that internal homology modeling by PIPER-Map helps to enhance prediction accuracy when there are good homologues for the antibody. Starting from either unbound crystal structure or just the sequence, PIPER-Map outperforms EpiPred (with the unbound structure). When considering the top 30 residues, PIPER-Map using the X-ray structure and internal homology model, respectively, improves the F score by ~72% and ~78% on EpiPred that uses the X-ray structures.

FIGURE 5.

Comparing PIPER-Map’s performance on X-ray and homology-modeled antibodies as inputs for 21 new antibody-antigen targets in the benchmark set BM5.5 with EpiPred’s predictions based on X-ray structure inputs. The prediction metrics are averaged to obtain the F scores shown

There are limitations of the epitope mapping method presented here that ought to be addressed in future work. Firstly, if a candidate homologue is found for the heavy chain and if that homologue’s light chain is not present or does not pass the template filters, then the candidate is discarded as a template. Such potentially helpful homologues can be utilized if heavy and light chain orientation can be predicted accurately. Secondly, the rigid-body assumptions inherent in PIPER can negatively impact the docking results. Refinement or small-scale molecular dynamics simulation of CDR loops can potentially improve the docking results. Finally, geometry-based clustering of the epitope residues into continuous surface patches can potentially improve the usability and accuracy of the epitope predictions. Despite these limitations, the server presented in this work and the results from it showcase that epitopes can be in many cases predicted fairly well by exploiting ensembles of homologues and docking poses.

Data Availability

All data generated in this work is available in the main text or in Supporting Information.