Performance of Hybrid Strategies combining MDockPP and AlphaFold2 in CAPRI Rounds 47–55

Abstract

CAPRI challenges offer a range of blind tests for biomolecule interaction prediction. This study evaluates the performance of our prediction protocols for the human group Zou and the server group MDockPP in CAPRI rounds 47–55, highlighting the impact of AlphaFold2 (AF2) and the effectiveness of massive sampling approaches. Prior to AlphaFold2’s release, our methods relied on homology modeling and docking-based protocols, achieving limited accuracy due to constraints in structural templates and inherent docking limitations. After AlphaFold2’s public release, which demonstrated breakthrough accuracy in protein structure prediction, we integrated its multimer models and massive sampling techniques into our protocols. This integration significantly improved prediction accuracy, with human predictions increasing from 1 correct interface out of 19 pre-AlphaFold2 to 4 out of 8 post-AlphaFold2. The massive sampling approach further enhanced performance, particularly for targets T231 and T233, yielding medium-quality models that default parameters could not achieve.

1. Introduction

Protein-protein interactions and multi-protein assemblies, including those involving DNA or RNA, are essential for cellular processes¹. Disruptions in these interactions can lead to various diseases^2,3, making it crucial to understand these complexes at both molecular and cellular levels. While atomic 3D structures of these assemblies, typically obtained through techniques like X-ray crystallography⁴ and cryo-electron microscopy (cryo-EM)^5,6, are invaluable, structural data remain limited for many protein complexes identified by modern proteomics. The Critical Assessment of Prediction of Interactions (CAPRI) challenge⁷, initiated in 2001, has played a pivotal role in driving the development and refinement of computational methods to bridge this gap. Over the past two decades, CAPRI has hosted over 200 blind tests of protein-protein interaction predictions, fostering significant improvements in the accuracy and efficiency of computational approaches, such as docking-based algorithms and template-based methods^8–19.

A major breakthrough in the field occurred with the development of AlphaFold2 (AF2)²⁰, a deep learning (DL) algorithm by Google DeepMind, which revolutionized the prediction of protein structures. AlphaFold2 demonstrated unprecedented accuracy in predicting the atomic structures of single protein chains during the CASP14 blind prediction challenge²¹, marking a game-changing moment in structural biology. The public release of AlphaFold2 further accelerated progress in the field, enabling rapid advancements in the modeling of protein complexes.

Before the release of AlphaFold2, during CAPRI rounds 47–55, our group applied our previous protocol²², which involved constructing monomeric structures of query protein sequences through homology modeling or ab-initio approaches. We then used our in-house developed docking program, MDOCKPP with GPU acceleration^22–24, to generate putative binding modes. These binding modes were subsequently scored and ranked using ITScorePP²³, our statistical potential-based protein-protein scoring function. Finally, we incorporated available biological information from our automated literature search server, Rebipp²², to finalize the selection of binding modes.

Following the release of AlphaFold2, we adjusted our approach to fully leverage its capabilities. We used AlphaFold2 multimer²⁵ to directly generate binding modes or employed AlphaFold2 monomer to build structures, which were then processed with MDOCKPP. Further research and development, particularly by Wallner’s group^26,27, showed that enabling dropout during inference combined with extensive sampling significantly enhances the quality of the generated models. By introducing controlled variations in the neural network and generating around 6,000 models per target, compared to the default 25 models from AlphaFold2 multimer, Wallner’s method, using both v1 and v2 multimer network models with and without templates, achieved remarkable results in CASP15-CAPRI Round 54²⁸. Building on these insights, we adopted a similar strategy starting with CAPRI Round 55.

In this paper, we report the performance of our protocols for the human group Zou and the server group MDockPP in CAPRI rounds 47–55, demonstrating that AlphaFold2 significantly improved our prediction accuracy. The use of massive sampling further enhanced performance compared to the default 25 models generated by AlphaFold2 multimer.

2. Methods

In CAPRI Rounds 47–55, excluding the CASP joint Rounds 50 and 54, a total of 11 targets were released. These included one protein-peptide complex (T231), two protein-DNA complexes (T187 and T188), and eight protein-protein complexes. Before the release of AlphaFold2, we employed our established docking-based protocols for prediction. After AlphaFold2’s release, we adopted a new approach based on this tool. Below, we describe these protocols in detail.

2.1. Docking-Based Protocols

Protein-Protein Interactions

Our protocol for predicting protein-protein interactions is detailed in a previous publication²², but a summary is provided here for completeness. When only protein sequences were available for a target, we first searched for homologs in the Protein Data Bank (PDB) using NCBI BLAST^29,30. If high-quality templates were found, we used MODELLER³¹ for homology modeling to build monomeric structures. If no suitable templates were available, we employed ab-initio methods (e.g., I-TASSER^32,33) to model the structures. Molecular docking was then performed using the GPU-enabled MDOCKPP to generate 54,000 putative binding modes. These modes were scored and ranked using ITScorePP²³, our statistical potential-based scoring function for protein-protein docking. Available biological information was used to filter the binding modes. Clustering was performed based on backbone root mean square deviation (b-RMSD), with a default cutoff of 8 Å, to eliminate redundant modes. To ensure that the structure with the highest score was used as the reference, we employed a sequential RMSD-based clustering approach. Briefly, starting with the highest-scoring structure as the initial reference, the pairwise RMSD between this reference and the remaining structures was calculated. Structures with an RMSD below a predefined cutoff were assigned to the same cluster as the reference. The next unassigned structure was selected as the reference for a new cluster. This process was iteratively repeated until all structures were assigned to clusters. The top 10–20 modes were manually inspected, and 10 were selected for human submission (Zou). The server prediction (MDockPP) followed a similar protocol, but without manual inspection and selection.

For the scorer experiment, the same protocol was applied, except that the binding modes were sourced from the predictor experiment submissions redistributed by CAPRI. In the server prediction, ranking was automated without human intervention.

Protein-Peptide Interactions

Protein-peptide interaction prediction followed protocols described in our earlier publication, using the MDockPeP2 docking server³⁴. Briefly, given a peptide sequence and a protein structure, MDockPeP2 performed global docking of the flexible peptide to the protein. The binding modes were ranked by a hybrid scoring function, PepProScore³⁴, which combines a protein-peptide binding score with a conservation score of interacting interfaces. Redundant modes with ligand RMSD below 3 Å were removed. The top 10 models were submitted directly for server prediction, while human prediction involved additional manual inspection.

In the scoring experiment, binding modes provided by CAPRI were scored using our statistical potential-based scoring function, ITScorePeP³⁵, which accounts for both intermolecular and intramolecular interactions. The top modes were manually selected for human prediction, while the server prediction was automated.

Protein-DNA Interactions

For protein-DNA interaction prediction, if the DNA structure was unavailable, it was generated using AmberTools in the AMBER16 suite³⁶, followed by a 5 ns MD simulation with the OL15 force field³⁷. The final frame was used as the DNA structure. The DNA was then docked with the protein using MDockPP, and binding modes were scored with the in-house developed ITScorePD, a knowledge-based scoring function for protein-DNA complexes. Clustering based on b-RMSD was used to eliminate redundancy, and the top modes were manually selected for submission. The server prediction followed the same approach but without manual inspection.

In the scorer experiment, the same scoring function was used as in the predictor experiment. For human prediction, the top modes were manually inspected before submission, while server prediction skipped this step.

2.2. AlphaFold2-Based Protocols

Protein-Protein/Peptide Interactions

After the release of AlphaFold2, and particularly with the introduction of the multimer model, we began using AlphaFold2 to generate complex or monomeric structures for protein-protein and protein-peptide interactions. The detailed protocol used in CAPRI Rounds 47–55 is outlined below.

In the CAPRI server competition, predictions are required within 2 days of target release. Due to time constraints and limited computational resources, we used ColabFold³⁸, which integrates AlphaFold2 with MMseqs2^39,40 to accelerate structure prediction. For server prediction, 25 complex models were generated using default parameters in ColabFold. These models were ranked based on a confidence score (0.5 * predicted TMscore + 0.5 * interchain predicted TMscore) and clustered to remove redundancy. Binding modes were split into monomeric structures for docking with MDOCKPP. Monomeric structures were also directly generated with ColabFold for additional docking. The resulting binding modes were ranked using ITScorePP. If the top complex models generated by ColabFold had a confidence score higher than 0.8, they were treated as biological information and used to filter the binding modes from docking. Clustering was performed to remove redundant binding modes based on b-RMSD. The top 10 binding modes were submitted for evaluation.

For human predictions, we applied settings similar to those described in Wallner’s paper²⁷. The detailed settings are provided in Table 1. For each setting, 1,000 models were generated, totaling 6,000 models per target. These models were ranked by confidence score, clustered by b-RMSD, and manually inspected before submission.

Table 1.

Different settings of ColabFold were used, with all the options controlled by the ColabFold flags.

AlphaFold 2 model version	Templates	Dropout	Recycles
multimer_v2	Yes	Yes	3
multimer_v2	No	Yes	3
multimer_v2	No	Yes	21
multimer_v3	Yes	No	3
multimer_v3	No	Yes	3
multimer_v3	No	Yes	9

In the scorer experiment, binding modes provided by CAPRI were ranked using ITScorePP. If the top complex models generated by ColabFold had a confidence score higher than 0.8, they were treated as biological information and used for filtering. Clustering was then performed to remove redundant binding modes based on b-RMSD. The top binding modes were submitted for evaluation in the server prediction. For human prediction, the top binding modes, along with models submitted by human predictors in the predictor experiment, were manually inspected and selected for submission.

3. Results

We report our performance in CAPRI rounds 47–55, where a total of 11 targets were provided by CAPRI over the past four years for predictor and scorer experiments. These targets included 8 protein-protein interactions, 1 protein-peptide interaction, and 2 protein-DNA interactions. Among these, targets T160, T186, T187, and T188 featured multiple interfaces for prediction, resulting in a total of 27 evaluated interfaces across these rounds. Notably, 19 interfaces from 5 targets were released before the availability of AlphaFold2, while 8 interfaces from 6 targets were released after AlphaFold2 became available. The detailed results from our human and server predictions are presented in Tables 2 and 3, respectively.

Table 2.

The performance of the top 5 models of our human prediction protocols in CAPRI rounds 47–55.

Target. Interface	Complex	Predicting				Scoring
		Fnat	Lrms	Irms	Accuracya	Fnat	Lrms	Irms	Accuracy
T160.1	S-layer protein Sap/nanobody Interfaces	0.00	39.13	14.84	0	0.00	48.79	14.18	0
T160.2		0.00	38.29	13.90	0	0.00	51.31	19.51	0
T160.5		0.00	29.56	15.67	0	0.00	42.97	22.39	0
T160.3	Interfaces between the multiple domains of Sap	0.09	22.45	7.21	0	0.23	24.02	6.24	0
T160.4		0.00	52.63	11.15	0	0.00	52.18	12.19	0
T160.6		0.11	43.21	8.76	0	0.15	42.80	8.23	0
T160.7		0.24	23.78	5.94	0	0.41	23.45	6.39	0
T160.8		0.36	38.01	7.07	0	0.36	28.72	7.54	0
T160.9		0.50	14.71	4.63	0	0.42	19.90	5.24	0
T161	RnlA	0.07	40.48	16.44	0	0.04	25.95	11.65	0
T162	RnlA/RnlB	0.00	60.09	17.79	0	0.00	48.58	6.95	0
T163.1		0.00	34.59	22.63	0	0.00	34.59	22.64	0
T163.2		0.00	35.60	18.26	0	0.00	41.30	22.84	0
T163.3	SYCE2/TEX12	0.00	54.15	12.65	0	0.00	43.06	17.99	0
T163.4		0.00	48.11	13.48	0	0.00	50.11	12.78	0
T163.5		0.00	66.01	14.47	0	0.04	68.26	17.62	0
T186.1	NADH:Ubiquinone oxidoreductase	0.49	5.75	6.53	4*	0.49	5.77	6.53	4*
T186.2		0.13	14.06	9.56	0	0.13	14.06	9.56	0
T186.3		0.17	14.32	8.56	0	0.17	14.32	8.56	0
T187.1	TnpA trasnpsosase/dna	0.00	84.35	31.35	0	0.00	84.28	16.27	0
T187.2		0.00	82.35	7.51	0	0.33	20.83	3.05	2/1**
T188.1	TnpA trasnpsosase/dna	0.00	68.25	14.02	0	0.49	9.77	3.26	1*
T188.2		0.28	7.97	3.56	5*	0.69	16.87	2.95	4*
T231	FLAG-tag/anti-FLAG M2	0.52	1.92	1.32	1/1**	0.52	1.95	1.33	1/1**
T232	Serine/threonineprotein phosphatase/TIPRL	0.85	1.99	1.46	5/5**	0.83	1.97	1.40	3/3**
T233	MHC/antibody	0.86	3.81	1.06	1/1**	0.86	3.81	1.07	1/1**
T234	MHC/antibody	0.34	17.52	5.63	0	0.16	24.31	7.59	0.00

^aThe accuracy is categorized by three parameters following the CAPRI criteria: the percentage of native residue-residue contacts (F_nat), the ligand RMSD (L_rms), and the interface RMSD (I_rms).

^***indicates high accuracy

^**indicates medium accuracy

^*indicates acceptable accuracy, and 0 indicates no correct prediction.

Table 3.

The performance of the top 5 models of our server prediction protocols in CAPRI rounds 47–55.

Target. Interface	Complex	Predicting				Scoring
		Fnat	Lrms	Irms	Accuracya	Fnat	Lrms	Irms	Accuracy
T160.1	S-layer protein Sap/nanobody Interfaces	0.00	33.49	12.47	0	0.14	22.69	8.58	0
T160.2		0.00	44.93	19.28	0	0.00	42.04	17.91	0
T160.5		0.00	42.97	22.39	0	0.00	64.72	22.23	0
T160.3	Interfaces between the multiple domains of Sap	0.09	20.68	6.40	0	0.23	23.86	6.18	0
T160.4		0.00	52.18	12.19	0	0.00	52.12	12.20	0
T160.6		0.11	45.20	8.86	0	0.19	42.79	8.28	0
T160.7		0.24	23.19	6.38	0	0.35	23.23	6.37	0
T160.8		0.27	48.85	9.15	0	0.36	28.67	7.62	0
T160.9		0.42	19.90	5.24	0	0.33	19.99	5.24	0
T161	RnlA	0.00	42.19	15.51	0	0.02	39.56	15.36	0
T162	RnlA/RnlB	0.00	38.84	11.78	0	0.02	42.48	11.81	0
T163.1	SYCE2/TEX12	0.00	40.50	24.05	0	0.00	23.72	10.44	0
T163.2		0.00	37.29	16.76	0	0.00	35.63	18.27	0
T163.3		0.00	56.33	17.99	0	0.00	54.13	12.66	0
T163.4		0.00	50.12	12.78	0	0.00	48.12	13.49	0
T163.5		0.00	65.25	15.29	0	0.00	66.03	14.48	0
T186.1	NADH:Ubiquinone oxidoreductase	0.68	2.55	8.08	5/2**	0.49	5.77	6.53	4*
T186.2		0.02	48.00	21.81	0	0.13	14.06	9.56	0
T186.3		0.00	49.68	26.56	0	0.17	14.32	8.56	0
T187.1	TnpA trasnpsosase/dna	0.00	98.06	32.94	0	0.00	84.29	16.33	0
T187.2		0.00	59.18	7.49	0	0.33	10.32	3.08	3*
T188.1	TnpA trasnpsosase/dna	0.09	41.70	10.27	0	0.49	9.75	3.26	1*
T188.2		0.34	21.28	4.30	0	0.83	10.96	2.45	4*
T231	FLAG-tag/anti-FLAG M2	0.09	10.37	3.28	0	0.48	7.38	2.31	0
T232	Serine/threonineprotein phosphatase/TIPRL	0.85	2.10	1.48	5/5**	0.83	1.97	1.40	5/5**
T233	MHC/antibody	0.00	43.36	18.08	0	0.00	25.18	12.39	0
T234	MHC/antibody	0.00	32.50	9.93	0	0.16	24.31	7.59	0

^aThe accuracy is categorized by three parameters following the CAPRI criteria: the percentage of native residue-residue contacts (F_nat), the ligand RMSD (L_rms), and the interface RMSD (I_rms).

^***indicates high accuracy

^**indicates medium accuracy

^*indicates acceptable accuracy, and 0 indicates no correct prediction.

Before the introduction of AlphaFold2, only one interface was successfully predicted in the human prediction for the predictor experiments, where at least one acceptable binding mode was ranked within the top 5 submissions. Similarly, only one interface prediction was deemed correct in the human prediction for the scorer experiments. For server predictions, the outcomes mirrored the human predictions, with just one correct interface identified in both the predictor and scorer experiments.

After AlphaFold2’s release, our human predictions improved significantly, with 4 interfaces correctly predicted in the predictor experiments and 6 in the scorer experiments. In server predictions, 1 interface was correctly predicted in the predictor experiments, while 4 were correctly identified in the scorer experiments. The detailed analysis for each target is described below.

Target T160

Target T160 involves the protein-protein complex formed by the Bacillus anthracis S-layer protein Sap and two nanobodies, Nb684 and Nb694, available as PDB 6HHU⁴¹. To model the nanobody structures, we used a template-based approach. Based on BLAST results, PDB 6B20⁴² and PDB 6FE4⁴³ were selected as suitable templates. Chain E in 6B20 shared 86.6% sequence identity with Nb684, while Chain F shared 81.1% sequence identity with Nb694. The Sap protein consists of six individual domains arranged in two dimensions. The backbone structure of each Sap domain was provided by the CAPRI organizers, and we added the side chains using MODELLER. We used chain A of PDB 4AQ1⁴⁴ as a template to build the monomer structure of the Sap protein, which is the SbsB S-layer protein from Geobacillus stearothermophilus. The template had a sequence identity of 25.36% and 37% coverage with the query sequence. Given the low coverage and identity, we docked the individual domains one by one and selected the most similar docking structure to the template for subsequent docking steps. Linkers between each domain were added using MODELLER, and the selected structures underwent a 100 ns molecular dynamics (MD) simulation with the Amber ff14SB force field⁴⁵. The representative structures were then used as monomers for docking the two nanobodies.

Upon comparing the released crystal structure of Target 160 with our predicted model, we observed substantial conformational differences in the SAP protein, with an RMSD of 33.41 Å for the Cα atoms. As shown in Figure 1A, none of our submissions were correctly predicted, mainly due to significant disparities between the template we used and the resolved crystal structure of the SAP protein. Although the RMSDs of the Cα atoms between the modeled nanobodies and the experimental structures were relatively small (2.39 Å for Nb694 and 1.77 Å for Nb684), the complementarity-determining region (CDR) showed considerable conformational differences compared to the released structure (Figure 1A). These variations in the CDR regions likely also contributed to the inaccuracies in our prediction for this target.

Figure 1.

(A) Top: predicted (left) and crystal (middle) structures of T160, with superimposed predicted (tan) and crystal (cyan) SAP protein structures (right). The six domains of the SAP protein are highlighted with different colors, while the two nanobodies are shown in cyan. Bottom: superimposed predicted (tan) and crystal (cyan) nanobodies, Nb684 (left) and Nb694 (right). The CDRs, which exhibited significant conformational differences compared to the released structure, are highlighted in red for the crystal and in green for the predicted nanobodies. (B) Predicted (left) and crystal (middle) structures of T161, with superimposed predicted (tan) and crystal (cyan) RnlA monomer structures (right). (C) Predicted (left) and crystal (middle) structures of T163, with superimposed predicted (tan) and crystal (cyan) SYCE2 monomer structures (right). (D) Top: Predicted (left) and crystal (middle) structures of T187, with superimposed predicted (tan) and crystal (cyan) TnpA monomer structures (right); Bottom: Predicted (left) and crystal (middle) structures of T187, with superimposed predicted (tan) and crystal (cyan) DNA structures.

Targets T161 and T162

Both T161 and T162 focus on the toxin-antitoxin complex RnlA-RnlB from E. coli, a heterocomplex comprising a toxin homodimer bound to two antitoxin molecules. The crystal structure is now published as PDB 6Y2P⁴⁶. The challenge for T161 was to predict the bound form of the toxin homodimer, while T162 required predicting the full toxin-antitoxin complex structure. For T161, we used the unbound structure of RnlA (PDB 4I8O⁴⁷) as a template to build the initial monomer structure of the toxin molecule. This initial structure was subjected to MD simulations using the Amber ff14SB force field⁴⁵ to sample the conformational changes in the bound form of the toxin homodimer. However, compared to the released structure, the monomer underwent significant conformational changes in the bound form (Figure 1B), and we were unable to sample these changes in our MD simulations, leading to a failure in predicting the complex conformation for T161. Although an acceptable structure was present in the scoring set of T161, we failed to identify it. Consequently, we could not accurately predict T162, even though the bound structure of RnlB was provided.

Target T163

Target T163 is a heterocomplex of two small helical protein components, SYCE2 and TEX12, from the human synaptonemal complex, featuring a 2:2 heterodimer. This complex is now published as PDB 6R17⁴⁸. At the time T163 was released, the monomer structure of TEX12 was available in PDB 6HK8, but the structure of SYCE2 was unknown. No suitable template was found through sequence similarity searches, but we identified PDB 6H86⁴⁹, a synaptonemal complex protein SYCE3 from Mus musculus. Since PDB 6H86 is in the same family as the query SYCE2 protein, we used it as a template to build the monomer structure of SYCE2 for T163. However, due to the lack of significant similarity between the template and the query sequence, we also used I-TASSER to build the monomer structure of SYCE2. The C2 complexes of SYCE2 and TEX12 were then built using MDOCKPP, and the top C2 complexes were docked together to generate binding modes for the SYCE2/TEX12 complex following our protocol. Unfortunately, our generated monomer structure of SYCE2 was incorrect (Figure 1C), leading to a failed prediction. Although the scoring set contained models with correct interfaces, we were unable to identify them.

Target T186

Target T186 is a component of a stable sub-complex (peripheral arm) of the E. coli respiratory complex NADH:oxidoreductase. The query complex comprises six subunits, each present in a single copy. The structure has been published in PDB 7NYR⁵⁰. The primary challenge for T186 was to predict the conformation of a loop approximately 100 residues in length, positioned on the surface of a large pocket. This loop, which is part of chain G, is well-structured but lacks secondary structure elements and interacts with both chain G and chain D. The CAPRI team conducted three assessments: (1) interactions between chains G and D, (2) interactions between the loop and the rest of chain G, and (3) interactions between the loop and chain D. We identified several templates, PDB 6X89⁵¹, 4HEA⁵², and 6GCS⁵³, that contained homologs of all subunits in T186. Relying on the high sequence similarity and coverage between the query and these templates, we built the complex structure using MODELLER. However, the loop in chain G was missing in these templates. For server predictions, MODELLER automatically added the loop, while in human predictions, additional steps were taken. Specifically, the loop was added to the model using PDB 6X89 as a template with Loopy (https://honig.c2b2.columbia.edu/loopy), followed by a short 5 ns MD simulation with the Amber ff14SB force field⁴⁵ to relax the complex structure.

Our top 5 human predictions achieved acceptable accuracy for the interactions between chains G and D, owing to the high-quality templates used. However, we failed to accurately predict the loop conformation for interactions between the loop and chain G, as well as between the loop and chain D. Similarly, in server predictions, our top five submissions correctly predicted the interactions between chains G and D, while the other two assessments failed. Two medium-accuracy models were observed in our top five server predictions, both based on template PDB 4HEA, which was not used in our human predictions. For both human and server scoring experiments, we achieved acceptable accuracy for the interactions between chains G and D but failed in the other two assessments.

Targets T187 and T188

Target T187 involves the complex of TnpA transposase from the Tn4430 transposon of Bacillus thuringiensis with DNA. Target T188 is the same complex as T187, but with the coordinates of the bound double-stranded DNA provided by the organizer. This structure has now been released as PDB 7QD6⁵⁴. Although AlphaFold2 had been released by the time of T187 and T188, we could not use AlphaFold2 to generate the monomeric or dimeric structures of the transposase due to computational resource limitations. Instead, we used our docking-based protocol described in the Methods section, except the monomer structure of TnpA transposase was generated using the Rosetta web server⁵⁵, and the double-stranded DNA (dsDNA) structure was built using the AmberTools package. The complex structures were generated following our docking protocol, with ITScorePP used to evaluate the dimer interface in TnpA and ITScorePD used to assess the interface between protein and DNA. Unfortunately, the monomeric structures of dsDNA and TnpA showed substantial differences compared to the resolved structure, with RMSDs of 17.46 Å and 10.15 Å, respectively (Figure 1D). Consequently, none of our human or server predictions in the predictor experiments were correct. For the scorer experiments, we identified a medium-quality model for the TnpA dimer interface from the scoring set provided by CAPRI and achieved acceptable accuracy in server predictions.

For T188, we used the dimer structure of transposase from our top model in the human submissions for T187 in the scorer experiment. This dimeric model was selected to dock with the dsDNA structure provided by CAPRI. Our top 5 human predictions achieved acceptable accuracy for the TnpA dimer interface, while the server predictions did not reach the same level of accuracy. In the scoring predictions, both human and server submissions achieved acceptable accuracy for both the dimer structure and the TnpA/DNA interface.

Target T231

Target T231 is the FLAG-tag/anti-FLAG system complex, a widely used biochemical tool for protein detection and purification. The crystal structure is now available as PDB 8RMO⁵⁶. Inspired by the success of massive sampling using AlphaFold in the previous Round 54^18,28, we applied the massive sampling protocol for our human predictions starting with this CAPRI target. Since this target is a peptide/antibody complex, MDOCKPeP2 was used to generate binding modes for server prediction submissions, with the antibody structure was generated using AlphaFold with default parameters. The peptide sequence was used as input, and the total number of peptide conformers for docking was set to 1,000. The peptide was docked to the antibody CDRs by blocking residues distal to the CDRs. Unfortunately, none of our server predictions were correct. The lowest I-RMSD value among the top 5 models was 3.28 Å. However, for human predictions, we successfully identified a medium-quality model (I-RMSD = 1.32 Å) generated by our massive sampling protocol. Interestingly, when using the default parameters to generate 25 AlphaFold models, only acceptable-quality models were produced in the top 5. This result suggests that the massive sampling protocol improved prediction accuracy for this target. For the scorer experiments, we also identified a medium-quality model in the human competition, while the server competition did not yield correct predictions.

Target T232

Target T232 is a complex of serine/threonine-protein phosphatase with TIPRL. For server prediction, models were generated following our AlphaFold2-based server prediction protocol and re-ranked using ITScorePP. These were then clustered based on the b-RMSD, with a cutoff of 1.5 Å. Our submission achieved medium accuracy in the server competition. For human prediction, we applied our massive sampling protocol, ranking the models by the confidence score before clustering them similarly to the server predictions. This also resulted in medium accuracy. Notably, the massive sampling approach did not significantly improve prediction quality for this target.

For the scorer experiment, ITScorePP was used to rank the CAPRI-provided models. Following the scorer experiment protocol after AlphaFold2’s release, these models were filtered based on the top complex structure generated by ColabFold, as it had a confidence score higher than 0.8. The top 10 models, after clustering, were submitted for server scoring predictions, all of which achieved medium accuracy. For human scoring, two models with binding modes significantly different from those generated by ColabFold but with high ITScorePP scores were manually ranked among the top 5 submissions, resulting in three medium-quality models in the top 5 predictions.

Targets T233 and T234

Targets T233 and T234 involved the binding of two different antibodies to a major histocompatibility complex-I (MHC). For T233 server predictions in the predictor experiment, the models were generated following our server prediction protocol described in the methods section. Instead of generating monomer structures for the query sequences, the structures of the antibodies (as dimers) and MHC (as dimers) were generated using ColabFold, and then used as partners in the docking step. The docking score of the MHC-antibody interface, evaluated by ITScorePP, was used to rank the models, which were then clustered with a b-RMSD cutoff of 5 Å for the complex. Unfortunately, none of our submissions achieved acceptable accuracy in the server competition.

For human predictions, we applied the massive sampling protocol, ranking the models based on their confidence scores, followed by clustering, similar to the server predictions. In the T233 human prediction submission, we successfully identified a medium-quality model with an I-RMSD of 1.06 Å. Interestingly, all 25 models generated using the default parameters of AlphaFold were incorrect, whereas the massive sampling approach produced medium-quality models. For the T233 human prediction in the scorer experiment, ITScorePP was used to rank the CAPRI-provided models, which were then clustered using an 8 Å cutoff. The top 10 clusters, along with the top 10 models submitted in the human prediction of the predictor experiment, were manually selected for the human scoring competition. The models with the highest ITScorePP scores in the top 10 clusters were submitted as server scoring predictions. In the scorer experiments, our best model achieved medium accuracy in the human predictions, but no acceptable model was predicted in the server predictions.

The same protocols were applied to the T234 predictor and scorer experiments. Unfortunately, no acceptable model was predicted in either experiment, indicating that the massive sampling protocol did not improve prediction accuracy in this case.

4. Discussions and Conclusion

Our study evaluated the performance of our prediction protocols for the human group Zou and the server group MDockPP in CAPRI rounds 47–55, with particular focus on the impact of AlphaFold2 and the effectiveness of our massive sampling protocol. The results demonstrate a substantial improvement in prediction accuracy following the release of AlphaFold2, for both human and server predictions.

Prior to AlphaFold2’s release, our predictions were constrained by the available structural templates and the inherent limitations of docking-based methods. For example, targets T160 and T163 presented significant challenges due to considerable conformational differences between the predicted and resolved monomeric structures. The lack of templates for these targets hindered our prediction accuracy.

The introduction of AlphaFold2 marked a significant improvement, particularly through its multimer model and massive sampling strategies. Utilizing AlphaFold2’s capabilities led to a notable enhancement in our ability to predict protein-protein and protein-peptide interactions. Specifically, the accuracy of human predictions improved from 1 correct interface out of 19 in the pre-AlphaFold2 rounds to 4 out of 8 in the post-AlphaFold2 rounds. Although we could not directly use AlphaFold2 for targets T187 and T188, the medium-quality protein dimer model available in the scoring set for T187 was beneficial for predicting T188. The improvement could be contributed by AlphaFold2, as the CAPRI assessment noted that AlphaFold2’s monomer structures accurately captured the conformational changes.

Our massive sampling protocol, inspired by Wallner’s methods, allowed us to explore a broader range of possible conformations, leading to improved predictions. This approach was particularly effective for targets T231 and T233 in Round 55, where massive sampling yielded medium-quality models that default parameters could not achieve. Despite these advancements, some targets remained challenging. For example, target T234, similar to T233, did not benefit from the massive sampling protocol. This failure highlights the intrinsic challenges of predicting antibody-antigen complexes. The absence of epitope information makes accurate prediction inherently difficult, primarily due to the conformational variability of the CDRs in antibodies and the presence of multiple potential interaction sites on the antigen. It underscores the need for more specialized approaches or the integration of epitope information to enhance the accuracy of antibody-antigen interaction predictions. Moreover, for T232, the massive sampling approach did not improve in prediction quality compared to the default AlphaFold parameters. Further studies are needed to systematically identify scenarios where massive sampling is unlikely to enhance model quality, especially given its significant computational cost. Although the massive sampling protocol demonstrated significant improvements compared to the pre-AlphaFold2 period, our server predictions still face limitations, as evidenced by the results for targets T231 to T234. The server’s predictions did not consistently match the accuracy of human predictions, particularly in the scorer experiments. This disparity suggests that automated predictions may still lag behind manual inspections, likely due to the inaccuracy of the scoring functions. Future work should focus on improving the integration of AlphaFold2-generated models with advanced scoring functions to refine the selection process.

In summary, the integration of AlphaFold2 represents a significant advancement in the field. Continued innovation and refinement of our methods will be crucial for maintaining and improving prediction accuracy in future CAPRI rounds and beyond.

Data Availability

The data that support the findings of this study are available in the CAPRI official website at https://www.ebi.ac.uk/pdbe/complex-pred/capri/. These data were derived from the following resources available in the public domain: - CAPRI, https://www.ebi.ac.uk/pdbe/complex-pred/capri/