Structure-based TCR-pMHC binding prediction and generalization to unseen peptides

Abstract

The interaction between T-cell receptors (TCRs) with the peptide-bound major histocompatibility complex (MHC) intricately impacts the functional specificity of T-cell-mediated adaptive immune response. Consequently, implication in immunotherapy has contributed to the ever-growing computational methods for TCR recognition, which have recently attracted structure-based approaches due to advancements in protein structure modeling. Despite access to structural information of the predicted binding interface, graph neural network (GNN)-based TCR-pMHC binding specificity classifiers tend to show poor accuracy for samples with unseen peptides. In this work, we comprehensively assess the potential factors that critically impact the generalization performance of classifiers trained with computationally predicted structures. Specifically, our experiments focus on analyzing the sensitivity of such predictors to the interaction features in the TCR-pMHC interface and the structural uncertainty. Building on the analysis, we demonstrate how the design of classifier architecture with auxiliary training objectives can improve the generalization performance to novel peptides not yet seen during model training. Overall, our work highlights the challenges of unseen peptide generalization from different perspectives of the GNN-based classifier paradigm, showcasing the strengths and weaknesses of the current state-of-the-art approaches in the generalization landscape.

Introduction

Insights from CD8⁺ T-cell mediated adaptive immune response, governed by the interaction of T-cell receptors (TCRs) with the peptide and the α1-α2 helices of major histocompatibility complex (MHC-I), are important for diverse immunotherapies against viral pathogens, cancer, and autoimmune diseases. Computational approaches have been proposed to learn the rules for TCR recognition of the MHC-bound peptide utilizing the data from different experimental assays. As the majority of the predictors have relied on the sequence information of peptides, MHCs, and TCRs^1–7, they cannot directly leverage the 3D local structural characteristics of TCRs interacting with the peptide bound in the central binding groove of MHC-I. Since the determination of the first TCR-pMHC complex structure⁸, our knowledge base⁹ regarding the structural aspects of TCR-pMHC interaction has been expanding with the growth in experimentally determined structural data¹⁰. This covers from the MHC restriction with peptide specificity for the TCRs¹¹ to the canonical docking, including structures with exceptions of these characteristics. Leveraging such information through deep learning techniques, such as graph neural networks (GNN), supported by development in computational techniques for protein structure modeling, can offer a promising pathway for predicting TCR-pMHC binding specificity, as observed in other structure-based protein property, protein-protein/ligand/DNA interaction prediction tasks^12–15.

In particular, structure-based efforts^16,17 showed the potential of structural features for predicting the binding interaction between peptide and MHC. Given the cross-reactivity aspects of TCRs, and the limited number of TCR-pMHC samples across peptide-MHC pairs¹⁰, the progress of such structure-based approaches for TCR-pMHC prediction is contingent upon the development of specialized computational tools for TCR-pMHC complex modeling. Consequently, the existing structure-based efforts for TCR-pMHC can be categorized into two branches based on the two intertwined tasks: the TCR-pMHC complex structure prediction and the binding activity prediction. For example, Bradley¹⁸ developed AlphaFold-based TCR-pMHC complex structure prediction pipeline (TCRdock), providing improved structure prediction performance for TCR-pMHC than pretrained AlphaFold-Multimer model¹⁹. Furthermore, the confidence metric score, such as predicted aligned error (PAE) of the predicted TCR-pMHC complex from this specialized tool was shown to be effective in identifying the correct peptide within a pool of decoys. On the other hand, the authors in ref. ²⁰ moved the focus from predicted structure confidence to the docking quality (DockQ score²¹) of the predicted complex by AlphaFold-Multimer. Specifically, they trained a geometric vector perceptron (GVP)-based²² GNN to estimate the DockQ score, which was applied to discriminate binding and non-binding complex structures in a zero-shot manner. Complementary to both model confidence and docking quality, some research works^23–25 explored the application of interaction energy (estimated using tools like Rosetta²⁶, CHARMM²⁷ or contact-map derived pseudo-potential²⁵) as a structure-based scoring function for TCR-pMHC binding. In this group of efforts, the binding activity prediction is considered a zero-shot task, where performance relies on improvements in the structural modeling. The other branch focused on utilizing binding activity data in conjunction with the dataset of predicted complex structures. Slone et al.²⁸ proposed a GNN-based binary classifier for predicting TCR-pMHC binding activity from the residue level graph of TCR-pMHC complex structures predicted from the sequences. This is extended in ref. ²⁹ where the sequence encoding from ESM-2^30,31 was used to augment the node attribute of the TCR-pMHC residue graph, similar to the LM-GVP framework³². In ref. ³³, specific data augmentation strategies, including re-docking and perturbing structures, were adopted to build the dataset of positive and negative samples from experimental TCR-pMHC structures. Such an augmented dataset was used in training a hierarchical GNN classifier from residue- and atomic-level graphs, which showed reduced performance in TCR-pMHC binding classification when structures from computational tools, e.g., AlphaFold3³⁴ and TCRmodel2³⁵, were used instead of experimentally determined complexes.

With the advancement in computational protein complex structure prediction^19,34,36,37, the increasing number of computationally predicted TCR-pMHC complex structures naturally drives the research effort for TCR-pMHC binding prediction toward a structure-based approach like GNN-based classifiers. While the existing works for GNN-based classifiers showed reasonable performance for test samples with unseen TCRs, these models performed very poorly for TCR-pMHC samples with novel peptides, i.e., unseen in the training dataset. One may speculate that such poor generalization performance is simply due to the quality of the training dataset, which can be improved by creating more (and better) samples. Additionally, it is also possible that the current computational dataset may have subtle features that are difficult to pick up for current models during training, and in such cases, increasing the dataset size may be marginally beneficial. As we move on to a larger training dataset scenario, it raises the questions like: “do such computationally predicted datasets have distinct characteristics required for generalization and accessible to our prediction models?” “Can we effectively leverage such predicted TCR-pMHC complex structures for improving the generalization performance for unseen peptides?” Exploring this line of questions is necessary for providing insights toward improved generalization performance for unseen peptides. Our work focused on this aspect of GNN-based classifiers for TCR-pMHC binding, underexplored in contrast to sequence-based approaches^38,39.

Specifically, we assessed the unseen peptide generalization performance of GNN-based classifiers from three perspectives as illustrated in Fig. 1: (a) relative importance of interactions among peptides, MHC and TCRs in the computationally predicted structure in governing the generalized performance, (b) influence of structural uncertainty in predicted TCR-pMHC complex structure on the binding prediction performance, (c) sensitivity of generalization performance to the choice of classifier design. To enable balanced evaluation to explore these directions, we began with strict peptide splitting of the dataset from ref. ²⁸ without compromising the training dataset size, which reduced the bias in the results due to training sample size discrepancy. For assessing the impact of structural uncertainty, we generated two more datasets through different computational tools for the same TCR-pMHC samples, allowing inspection of the performance limitation imposed by the choices of such predicted datasets. Through highlighting the performance sensitivity of such GNN-based classifiers across different test-sets, we demonstrated potential strategies, such as our proposed dual encoder approach⁴⁰ and training with preference data, for moving up the generalization performance further away from the random classifier-like performance. Altogether, our work shed light on the strengths and weaknesses of the structure-based TCR-pMHC binding prediction approach in tackling the challenges of unseen peptides.

Fig. 1. Overarching theme of our study:

Structure-based TCR-pMHC binding prediction pipeline with graph neural network (GNN) and the potential factors (explored in this work) influencing the performance with unseen peptides. The GNN classifies a TCR-pMHC sample as binding or non-binding based on the residue-level graph constructed from the interface of predicted complex structure modeled by computational tools using MHC, peptide, and TCR-αβ sequences. Our study examines three aspects that can impact the performance of GNN-based models on novel peptides: the type of interaction edges, the quality of TCR-pMHC structure prediction, and the design of the GNN-based classifier.

Results

Evaluation with peptides unseen in training

To evaluate the generalization performance of the structure-based approach, we considered the dataset of predicted TCR-pMHC structures from ref. ²⁸, comprising 345 unique peptides over 19282 TCR-pMHC samples. While the large pairwise normalized Levenshtein distance (Fig. 2a) among these peptides makes this dataset suitable for unseen peptide evaluation, the skewed TCR-pMHC sample distribution (Fig. 2b) makes it difficult to make a balanced test-set through random partitioning based on the peptide. Specifically, to fairly evaluate the performance of a prediction model across different test-sets, one needs to keep the corresponding training datasets’ sizes similar. But due to the uneven number of samples per peptide, naive random splitting leads to test-set of different sample sizes as well as peptide counts, and consequently different training dataset sizes. In our work, we constructed 5 different test-set clusters from this dataset such that each contains an equal number (7) of peptide samples, and approximately 1000 TCR-pMHC samples. We distributed 35 peptides (from the shaded region shown in Fig. 2b) among five test-set clusters in such a sequential manner (see “Methods”) that each cluster contains close to 1000 samples. As illustrated in Fig. 2c (in addition to contact map profiles in Fig. S2 of Supplementary Information), the interface geometries of binding TCR-pMHC samples in these clusters are distinct, offering a broad evaluation of the structure-based models.

Fig. 2. Peptide characteristics in dataset from ref. ²⁸ and binding geometry of the holdout test-set clusters.

a Pairwise Levenshtein edit-distance for the 345 unique peptides present in the dataset. b Distribution of TCR-pMHC samples, where the shaded region indicates the test-set peptides. c Docking and incident angles of predicted TCR-pMHC complex structures in test-set clusters.

For evaluation on each of these 5 holdout test-sets, we trained the GNN-based classifiers with the remaining data (removing samples of the holdout test-set from the whole dataset of 19,282 samples) in a 10-fold cross-validation manner, and performed prediction for the holdout test samples with 10 trained models (corresponding to the 10 folds). We considered two types of graph encoder: Equivariant Graph Neural Network (EGNN)⁴¹ and Edge-Variable-Transformer convolution (EVTConv)²⁸ in single and dual encoder-based classifiers⁴⁰ (see “Methods” for details). In our proposed dual encoder approach, both the TCR-pMHC interface residue-level graph and its TCR-MHC subgraph are encoded in such a manner that encourages these two embeddings to be aligned or different depending on the training label (binding or non-binding) through a semantic loss term combined with binary cross-entropy loss. Note the single encoder approach with EVTConv as graph encoder is identical to the classifier proposed in Slone et al.²⁸.

As shown in Table 1 and Fig. 3 (boxplots without the hatches), there is no clear winner among these 4 classifiers that generalizes to all five test-set clusters. Among the graph encoders, the EGNN showed more robustness (higher median) compared to EVTConv. In terms of classifier architecture, the dual approach sometimes outperformed the single encoder counterpart (clusters 1, 2, and 3), and such a trend showed sensitivity to the choice of graph encoder. Specifically, our dual encoder approach showed statistically significant ROC-AUC improvement (by Wilcoxon signed-rank test⁴², Section S9 of Supplementary Information) in cluster 1 for EGNN with p-value of 0.0322 (it was 0.0967 in case of cluster 3) and in cluster 5 for EVTConv with p-value of 0.0420 (0.0801 in cluster 2). Note that this trend of the dual encoder approach is influenced by its semantic loss component (see Section S3 of Supplementary Information). While this improvement in the ROC-AUC metric indicates that models show improved discriminative performance for separating binding samples from non-binding ones, it does not necessarily imply that they also assign a higher probability to the true class than models with lower ROC-AUC for the majority of the test samples (Section S9 of Supplementary Information). For example, there were 7 (out of 10 folds) sets of predictions for cluster 1, where the EGNN (dual) showed a significant success rate over EGNN (single). On the other hand, we had only 2 such significant prediction sets for EVTConv in cluster 5. That being said, this improvement in discriminative performance over the single encoder approach illustrates that the dataset of predicted structures has a certain amount of relevant signal useful for improving the generalization performance, but requires efforts in designing a robust classifier architecture. Interestingly, none of the four classifiers was able to significantly move up from the random classifier baseline (i.e., AUC-ROC of 0.5) for test-set cluster 5, illustrating the need for carefully designed unseen peptide splitting-based evaluation for benchmarking GNN-based approaches in TCR-pMHC binding prediction.

Table 1.

ROC-AUC performance for 5 holdout test clusters. For Boltz-2³⁶ and FoldX-5.1/ Rosetta^26,63,64, the confidence measures and (negative of) interaction energies (between TCR and pMHC), respectively, are used for classification of binding and non-binding samples

Method	Cluster 1	Cluster 2	Cluster 3	Cluster 4	Cluster 5
Boltz-2 iptm	0.5707	0.5574	0.6022	0.6263	0.5636
Boltz-2 confidence score (0.8×pLDDT+0.2×iptm)	0.5742	0.6480	0.5547	0.5997	0.5757
FoldX-5.1 interaction energy	0.4367	0.5255	0.3963	0.5355	0.4858
FoldX-5.1 interaction energy (backbone only)	0.5069	0.4941	0.4790	0.5902	0.5705
Rosetta interaction energy	0.5368	0.5545	0.5707	0.5495	0.5276
EGNN (single)	0.5548 (0.0372)	0.6108 (0.0750)	0.5506 (0.0658)	0.6718 (0.0454)	0.5239 (0.0613)
EVTConv (single)	0.5569 (0.0639)	0.5346 (0.0524)	0.5403 (0.0686)	0.6195 (0.1228)	0.5072 (0.0678)
EGNN (dual)	0.6012 (0.0607)	0.5899 (0.0448)	0.5760 (0.0491)	0.6045 (0.0793)	0.5002 (0.0811)
EVTConv (dual)	0.5453 (0.0688)	0.5899 (0.0802)	0.5532 (0.0902)	0.5976 (0.1007)	0.5322 (0.0751)
NetTCR-2.2 (pan-peptide)	0.5345 (0.0534)	0.4264 (0.0662)	0.4914 (0.0463)	0.5288 (0.0946)	0.5711 (0.0496)
EPACT	0.5781	0.6116	0.4096	0.4238	0.4722

For GNN-based approaches, a 10-fold cross-validation training was first performed on the training dataset, and the resulting 10 models were then applied to the holdout test-sets. We reported the median and interquartile range (IQR) over the 10 ROC-AUC values. For sequence-based models, the performance is evaluated with the published models.

Fig. 3. Performance of 4 different GNN-based classifiers with inference on complete TCR-pMHC interaction graphs and the subgraph with peptide nodes masked.

These four models (two types of graph encoder: EGNN and EVTConv, and two types of classifier architecture: single and dual encoder approach) are trained with complete TCR-pMHC graphs. The horizontal dashed line at 0.5 refers to the performance of a random classifier.

The performance of “zero-shot” baselines (Table 1) further shows the utility of the GNN-based approach with predicted structures. All GNN-based classifiers exceeded the FoldX-estimated interaction-energy-based screening in terms of median performance, except for cluster 5. However, compared to Rosetta-derived energy, the performance of FoldX was highly inconsistent across different test-set clusters. On the contrary, the confidence metric-based approach with Boltz-2 showed more balanced performance, and the training of GNN-based classifiers with TCR-pMHC structures led to better (median) classification performance over Boltz-2 only in clusters 1 and 4.

We also reported performance of two sequence-based models: NetTCR-2.2⁴ and EPACT⁴³ in Table 1. All test-sets except cluster 5 share no peptide sequence with the sequence dataset of NetTCR-2.2, explaining why this approach showed surprisingly better performance in cluster 5. Indeed, when we remove the 159 test samples corresponding to the peptide RPPIFIRRL, which was present in all 5 partitions of NetTCR-2.2’s training dataset, the ROC-AUC drops to 0.4953 (0.0554) for the remaining test samples. On the other hand, the EPACT performed in a similar range to the best-performing GNN-based models for clusters 1 and 2, but significantly fell behind in the remaining clusters. Even in comparison to the zero-shot interaction-energy-based predictors, the performance gap of sequence-based models in clusters 3–4 illustrates the utility of the computationally predicted TCR-pMHC structures. Considering the advantage of pre-training in EPACT, the results with our GNN-based prediction networks have shown more promising potential, albeit with sensitivity to the test-set clusters.

Influence of peptide in structure-based prediction

As discussed in previous section, the performance of GNN-based classifiers was improved for some clusters with the dual encoder approach, which utilizes TCR-MHC and TCR-pMHC interface graphs. Since the peptide nodes are in a small fraction of the total nodes (with a lower fraction of edges involving peptide, as shown in Section S1 of Supplementary Information) in the TCR-pMHC interface graph, we investigated whether they are crucial for the prediction performance or the dual encoder approach merely learns to bypass them with the TCR-MHC subgraph. Specifically, we considered the same 4 classifiers from Table 1 that were trained with the full TCR-pMHC structure dataset, and applied those trained models to the TCR-MHC graph of the test-set clusters. Note that this was inference-time masking of the peptide residue nodes. Fig. 3 revealed three distinct characteristics of the GNN-based prediction. Firstly, the masking of peptides led to a significant drop in performance in clusters 1–3, indicating that the peptides played a role in performance improvement in these clusters by the dual encoder approach. This decrease is also statistically significant in the Wilcoxon signed-rank test. For these 3 clusters, our per-sample success rate comparison (Section S9 of Supplementary Information) between these two variations in inference also indicates that the dual encoder-based models used the peptide nodes of the TCR-pMHC graph more heavily in predicting higher true class probabilities than their single-encoder counterparts.

In cluster 4, we observed a sensitivity to the choice of graph encoder. Interestingly, the performance with EVTConv improved slightly when we performed inference by removing the peptide nodes. This behavior is quite the opposite of EGNN, which showed a diminishing performance (with a statistical significance for the single encoder version). Finally, there is no significant change in performance for cluster 5, implying the prediction of GNNs on this cluster is not heavily impacted by the peptides, which potentially explains why the generalization performance for this cluster significantly differs from that of other test-set clusters.

For a model trained with TCR-pMHC structures, the learned classification rules ideally should utilize the edges involving the peptide nodes in the graph. Intuitively, such a model’s confidence in its prediction for a test sample is expected to decrease when the peptide nodes are removed from the graph. For instance, if a model predicts a test sample’s TCR-pMHC residue graph to have a chance of 80% to be binding, and increases the chance to 90% for the same sample with peptides removed, the earlier prediction of 80% seems less reasonable than it would have been if the prediction decreased toward 50%. This change in predictive entropy due to peptide masking can be used to pick the “best” prediction from 10 models corresponding to 10-fold cross-validation. We discussed such approaches in Section S4 of Supplementary Information.

TCR-pMHC vs TCR-MHC vs TCR-peptide: what matters more?

The variation in performance due to the masking of peptides in test time (Fig. 3) led us to the next question: what interactions among peptide, MHC, and TCRs are crucial for improved generalization performance? We trained classifiers with the TCR-MHC and TCR-peptide interface residue level graphs by masking peptide and MHC nodes, respectively. With two separate GNN encoders, we have four new classifiers to compare (Fig. 4) against the single and dual encoder models which were trained with the full TCR-pMHC structure dataset.

Fig. 4. Performance of models trained (and tested) with TCR-pMHC, TCR-MHC, and TCR-peptide (interface) residue level graphs.

a Except for single/dual encoder-based models, the other two were trained with the sub-structure datasets. b ROC-AUC performance for 5 holdout test clusters.

Let’s first focus on clusters 1–3, where the dual encoder showed performance improvement over the single encoder approach. For clusters 1 and 2, TCR-MHC seemed to be enough, as the models trained with TCR-MHC showed similar performance with the dual-encoder counterpart. However, as seen in Fig. 3, the performance of dual encoder approaches is sensitive to the peptide nodes, which demonstrates that the dual encoder approach is not overlooking the peptides in preference of the TCR-MHC interface graph, even though we have a second encoder that directly encodes the TCR-MHC graph in this approach. This point is further strengthened by the decline in performance in cluster 3 with the models trained with TCR-MHC. Specifically, the generalization performance drops below that of the single and dual encoder approaches, indicating that the TCR-MHC is not as important for this cluster as it is for the first two clusters. This illustrates how the dual encoder approach balances the influence of different interactions during training of the classifier. Now looking back at the result of cluster 4 with peptide masking during inference time (Fig. 3), we saw improvement in generalization performance with EVTConv. This observation of peptides negatively influencing the performance of EVTConv is further supported by the improvement in AUC-ROC of cluster 4 when we trained the models (with EVTConv as graph encoder) using the TCR-MHC sub-structure dataset. Now this also potentially explains why the dual encoder approach performs poorly compared to the single encoder approach, since the former one tries to make the interaction related to peptides more influential in its decision-making.

Models trained with the TCR-peptide graphs performed poorly in cluster 5 compared to the models trained with the complete structure dataset, demonstrating the “usefulness” (albeit for the performance similar to the random classifier) of MHC as a source of information learned by the GNN-classifiers for this cluster. Combining with the observation that generalization slightly improved (EVTConv) or stayed similar (EGNN) for models trained with TCR-MHC, it explains why this cluster showed no significant change in the performance of different GNN-classifiers in the peptides masking experiment of previous section. For cluster 4, the performance of TCR-peptide-only models is significantly worse than the TCR-MHC-only models for EVTConv as the graph encoder, while for the case of EGNN, the scenario is opposite. Furthermore, ignoring MHC during training led to improved generalization performance (compared to the single encoder approach that uses all node types) in clusters 1, 2, and 3, irrespective of the graph encoder type. For example, in the case of cluster 3, the median AUC-ROC of EVTConv classifiers trained with peptide-TCR exceeded 0.6, whereas the corresponding classifiers (single encoder) trained with TCR-pMHC have a median performance of around 0.54. Since the dataset has the least diversity in terms of MHC, it is intuitive to expect that ignoring the information from MHC during training classifiers may not hurt the performance of classifiers. While the results from the first three test-set clusters aligned with this idea, clusters 4–5 presented the case where the signals from MHC could be significant for better generalization performance depending on the types of graph encoders employed in the classifiers.

Overall, the discrepancy in performance across test-set clusters associated with the choice of both graph data and graph encoders highlights the intricate learning rules extracted in the training process. Not all interactions among TCRs, peptides, and MHC may be uniformly present (Section S1 of Supplementary Information) in the test-set, so betting on a particular interaction during training can hurt the performance by losing other information. We speculate that classifiers that allow the learning of classification rules by balancing the importance of these interactions are better suited for generalizable performance. Dual encoder (with EGNN in particular) is along this direction (as demonstrated in next section), but is still biased (by design) to the interactions associated with the peptide.

Impact of varying degree of utilization of TCR-MHC context

During inference for TCR-pMHC samples with an unseen peptide, the TCRs (and MHC) of the test sample can already be present in the training dataset. Having a separate encoder for TCR-MHC can be thought of as a way for encoding this “context” representation for the peptide in the TCR-pMHC sample. With the dual encoder approach, the encoding of the TCR-pMHC is only implicitly influenced by the encoding of TCR-MHC through the combined loss of binary cross-entropy and the semantic loss. In this section, we analyzed what if we allowed the TCR-MHC to have a more dynamic impact on the encoding of the TCR-pMHC graph?

Accordingly, we considered two approaches (see “Methods”) to perform conditional encoding of the TCR-pMHC graph with the subgraph TCR-MHC as a condition. In the first alternative, we fed the graph-level representation of the context (TCR-MHC) as a global node feature to the encoding stage of the TCR-pMHC graph. The second alternative involves injecting the context into the encoding of the TCR-pMHC graph through the parameters of its node embedding MLP layers. Specifically, the parameters of the node embedding MLP layer are predicted by the graph-level representation of the context of each sample. From Fig. 5, we see that the dual encoder approach has a balanced performance between these two levels of utilization for clusters 1–3. On the other hand, the performance deteriorates for clusters 4–5 as we make the dynamic encoding stronger. Interestingly, the dual encoder approach without the semantic loss component (see Section S3 of Supplementary Information for details) also showed a decline in performance similar to that of the first alternative (i.e., context as global node feature)–illustrating the importance of semantic loss formulation. These results re-emphasize the intuition that over-reliance on a particular interaction (TCR-MHC in this case) can weaken the robustness of the classifiers.

Fig. 5. Impact of varying levels of utilization of TCR-MHC context.

a The encoded representation of TCR-MHC residue-level graphs is used either as a global node attribute or as a condition to generate the parameters of node embedding MLP in the encoding of the TCR-pMHC interaction graph. b Unseen peptide performance comparison with single and dual encoder approaches with EGNN as graph encoder.

Influence of structural quality for generalization

Since we are using predicted TCR-pMHC complex structures in the GNN-based classifiers, the generalization performance is inherently impacted by the structural quality of the predicted complex. To analyze the sensitivity to the predicted structures, we constructed two new datasets based on the dataset from ref. ²⁸. The first one (denoted as relaxed struct) consists of Rosetta-relaxed structures where the samples from ref. ²⁸ are used as initial structures for relaxation (Section S5 of Supplementary Information). For the other dataset (denoted as tcrmodel2 struct), we predicted the TCR-pMHC structures from the same sequences using TCRmodel2³⁵. We then repeated the evaluation of Table 1 with these two new datasets, where we kept the same hyperparameters for the classifiers.

Figure 6 depicts the holdout test results with relaxed struct dataset. In terms of generalization performance, we observed a downward shift (either median or 3rd quartile decreased) in AUC-ROC for all classifiers in the case of cluster 5. For other test-set clusters, most of the models showed similar performance (similar to models trained with initial structures) with a couple of exceptions. For instance, both single encoder-based classifiers observed a boost in performance for cluster 3.

Fig. 6. Impact of Rosetta-relaxation of TCR-pMHC structures.

Four different classifiers trained with relaxed structures are compared against the corresponding classifiers trained with initial structures.

Considering the fact that relaxation might have distorted the structural artifact (which the GNN can exploit) in the predicted structures, such variation in test-set performance due to datasets of different structural quality raises the question of which predicted structures to use for training the GNN-based classifiers. Experiments with struct tcrmodel2 showed similar variation in performance (Section S6 of Supplementary Information), including the challenging scenario of cluster 5.

To analyze the prediction models’ sensitivity to the variation in the source of structures, we performed a cross-domain evaluation, where we used test-set samples from one structure source to evaluate the models trained with datasets from a different structure source. Specifically, we considered models trained with init struct dataset, and evaluated them with the test-set from relaxed struct dataset, and vice versa. Sensitivity of the models’ performance to the structures from different sources can potentially indicate whether the learned rules are less dependent on artifacts introduced by computational structure prediction tools.

Figure 7 showed performance variation across different GNN-based predictors when we performed the inference with structures originating from a different source than their training dataset. For example, the left panels (a, c, e, g) compare the ROC-AUC pairs in each holdout test-set by making predictions with structures from init struct, and relaxed struct datasets, respectively, where the models are trained with the init struct dataset. For each set of 10 pairs, we also show the statistical significance (based on the two-sided Wilcoxon signed-rank test) of whether the ROC-AUCs change significantly due to the test-time variation of the structure source. The EGNN (single) models showed more sensitivity (e.g., cluster 2–4) when they were trained with init struct, compared to the same architecture trained with relaxed struct datasets (e.g., only cluster 5). Apart from this case, the other three prediction networks remained stable for the clusters 1–3 while showing different sensitivity for the other two test-sets. This robustness to structural uncertainty has a computational impact, i.e., if one can train robust models with TCR-pMHC structures that undergo computationally heavy processing (e.g., relaxation) for reducing the chance of models exploiting the structural artifacts, they can apply the trained models to the raw predicted structures – leading to faster screening or prioritizing for further processing of the structures.

Fig. 7. Sensitivity of different GNN-based classifiers to test-time variation of structure source.

**(p < 0.01), *(p < 0.05), and n.s. (p ≥ 0.05) are assigned based on p-values from the Wilcoxon signed-rank test with pairs of performance evaluation on the same holdout test-set clusters (1–5) from two different structure sources—one is the same as the training dataset’s source, and the other is not. a, c, e, g show the cases where the training dataset was from init struct, whereas models trained with relaxed struct dataset are in the remaining panels (b, d, f, h). The solid horizontal lines for each set of 10 ROC-AUCs indicate their three quartiles.

Avenues for improving the generalization performance

Since the unique generalization challenges for our test-set clusters persisted across three sources of structure data, we analyzed the clusters in multiple perspectives, ranging from finer (e.g., the interface graph attributes) to coarse (e.g., MHC distribution) level characteristics, to explore what makes these clusters unique in presenting such differing levels of challenge to the GNN-based predictors. One of the findings from the graph attribute-level analysis (Section S1 of the Supplementary Information) showed that clusters 4 and 5 have a higher fraction of the binding samples with more edges between the MHC and peptide than the other test-set clusters. While these two clusters share a similar pattern of docking geometries, the CDR3 sequences of the binding samples showed high inter- and intra-cluster diversity for all five test-sets, indicating finer-level diversity with globally similar docking conformations. Looking at the MHC allele group distribution in each cluster, all of them are highly imbalanced toward the majority allele group in the entire dataset. However, the MHC-allele-wise performance evaluation revealed that the minority allele group of cluster 5 (which is also present in clusters 2 and 4) posed more challenges to the GNN-based predictors than other alleles from the test-sets. The details are presented in Section S2 of the Supplementary Information.

Next, we briefly discuss empirical efforts to improve the generalization performance on the test-set cluster 5 (see Section S7 of Supplementary Information for efforts based on training-data dependency). Previously, we observed that the MHC nodes are important for the different classifiers’ performance in cluster 5. So, we investigated whether explicit incorporation of human leukocyte antigen (HLA) type information of MHC molecules can help in improving the performance. Accordingly, we trained a variant of the dual encoder approach where the one-hot encoding vector of HLA type is concatenated with the encoded graph representations of TCR-pMHC and TCR-MHC before passing to the MLP layers for final binding probability prediction.

Additionally, we also observed (Table 1) that the Rosetta-energy-based scoring mechanism²³ showed somewhat stable performance (across test-set clusters) slightly above the random classifier. We wondered whether the biophysical energy data of the predicted TCR-pMHC complex could be leveraged in a way that complements the typical binary label-based training paradigm. Along this direction, from each fold’s training datasplit ${\mathcal{D}}_{\mathrm{train}}$, we created a paired preference data ${\mathcal{D}}^{\mathrm{p}\mathrm{a}\mathrm{i}\mathrm{r}}=\left\{\right({\mathcal{G}}_i,{\mathcal{G}}_j)\mid y_i=1,y_j=0,\mathrm{\Delta }{E}_{ij}\le {\delta }_{\mathrm{t}\mathrm{h}}\}$, where each paired sample consists of positive and negative samples with same peptide, MHC sequence, but different TCRs. ΔE_ij is the difference between positive and negative samples in terms of TCR-pMHC interaction energy estimated by Rosetta, and we chose to include those samples with a negative energy difference, i.e., lower energy for binding samples. See “Methods” for the details of this construction procedure. In addition to training with binary class labels from ${\mathcal{D}}_{\mathrm{train}}$, we periodically trained the classifiers with ${\mathcal{D}}^{\mathrm{pair}}$ to penalize predicting a lower probability for the positive samples than the negative ones with shared pMHC.

Figure 8 demonstrates the impact of these two approaches for cluster 5. Use of only HLA makes the performance of the dual approach more consistent (lower IQR) with slightly better median AUC-ROC. On the other hand, the periodic training with preference data boosted the performance of both single and dual encoder approaches, i.e., an increase in median. While such improvement is still close to the performance of a random classifier, this upward trend indicates the utility of biophysical energy estimates in enabling the GNN-based classifiers to extract the features that are rarely reached through the binary label-based training. However, this preference loss-based training comes with the disadvantage of introducing sample-specific bias. Specifically, ${\mathcal{D}}^{\mathrm{pair}}$ only includes pairs of positive and negative samples which follow Rosetta-derived energy relation (i.e., positive samples to have lower energy than the negative one in the pair). To reduce the deviation from the classifier trained with binary cross-entropy loss, the periodic training (with a lower learning rate) strategy is employed in this work. As we showed in Section S8 of Supplementary Information, this strategy can, in some cases, fail to find better models than trivial binary classification-based training. Nonetheless, such an approach with interaction energy estimates seems to be one of the potential directions to crack the challenging cases like cluster 5.

Fig. 8. Impact of training with paired preference data.

The single and dual encoder-based classifier is the same as in Table 1, and the third model, denoted as “dual + HLA” has a one-hot encoding vector of HLA type concatenated with encoded graph representations of TCR-pMHC and TCR-MHC. In case of preference data training, the models are trained periodically with a dataset of 3000 preference pairs (from training samples in the corresponding fold) in addition to the usual per-epoch training process.

Model-derived importance score and experimental motif

In this section, we compare the different GNN-based prediction networks in terms of their emphasis on biologically relevant features. Specifically, we considered two TCR-pMHC complexes: 9PBG and 9PBH, where they share the same MHC (HLA-B*27:05) and TCR pairs (AS19.2; AS: ankylosing spondylitis)⁴⁴. Given that these two peptides (the lowest Levenshtein edit-distances from our dataset: 7 and 5, respectively) interact with the TCRs in similar docking conformations, we explored whether the four GNN-based prediction networks emphasize similar peptide residue locations that are significant for binding.

Figure 9 shows the average rank of peptide residues for the predictions with the four GNN-based prediction networks (see “Methods”). For 9PBG, the EGNN (single) and EGNN (dual) models disagree with the EVTConv models on important regions for predictions exceeding the 0.5 threshold. Specifically, segment MLA had an average rank of ≤4 in the former group of models, and it is similar to the experimental binding motif. In contrast, both EVTConv-based models focused on residues VMM for predicting higher binding probability while the region is interacting with the MHC in the experimental structure. This discrepancy highlights that the models can reach similar conclusions by looking at different regions of the same input structure. For the other case (9PBH), all models showed a similar emphasis region (P4, L6, and N7 with average ranks of ≤4) for higher prediction probabilities. When the models failed to assign higher probabilities, we found that the important peptide locations remained largely similar to those for predictions >0.5. However, the relative importance was dispersed, and consequently, nearby residues received increased rank. For example, P8 (L6) were assigned a higher average rank for models with EGNN (EVTConv) in 9PBG. Moreover, the important peptide residue positions for the lower probability split showed higher similarity between the two peptides. Overall, these observations only revealed that GNN-based predictors can differ in their prediction basis even for samples with a similar binding interface. Since both peptides are highly distant from the training dataset of the models that struggled in our benchmark holdout test-sets, it would be unwarranted to over-interpret the presented results.

Fig. 9. Mean rank of peptide residues based on gradient-derived importance score.

For each prediction network architecture, the prediction was performed with TCRmodel2 predicted complex structures using all 50 models trained with tcrmodel2 struct dataset. The mean rank (lower rank value implies higher relative importance) is computed over the gradient-based saliency for the predictions in each partition (formed with a binding probability threshold of 0.5). The highlighted interface regions, visualized with UCSF ChimeraX⁷¹, are from experimental structures.

Discussion

Learning from the structural characteristics of TCR-pMHC interaction is an intuitively promising direction to address the concerning issue of generalization performance of current deep learning predictors in cases of unseen peptides. In this work, we explored the potential of the GNN-based classifiers in improving the scenario by leveraging the dataset of computationally predicted structures of TCR-pMHC complexes. Our experiments with different holdout test-sets revealed the sensitivity of the generalization performance to the choice of graph encoder as well as the relative importance of specific interactions across different test-sets. For example, the same neighborhood context of the peptide, i.e., TCR-MHC interface graph, is as informative as the TCR-pMHC interface in some test clusters, but faces difficulties in identifying binding and non-binding samples in other test clusters. Moreover, in contrast to the sequence-based approaches that often consider only the pairs of peptide and TCRs, our results indicated mixed utility of MHC in structure-based TCR-pMHC binding prediction. Specifically, the large chain of MHC can be beneficial for test samples where key interactions in the TCR-pMHC graph learned by the classifiers are subtle, while in other clusters, it can obscure the peptide-relevant information. Through different levels of TCR-MHC utilization in GNN-based classifiers, we showed the robustness of our proposed dual encoder approach to balance the importance of all interactions in the TCR-pMHC interface, resulting in improvement with EGNN over the single encoder approach, which classifies the sample from only the TCR-pMHC interface graph. However, neither these efforts from the classifier architecture perspective were enough for improved generalization in a particular test-set (cluster 5), nor did the dataset of computationally determined structures of different quality change the scenario. For this case, our periodic finetuning strategy with a pairwise preference dataset based on Rosetta-estimated TCR-pMHC interface energy resulted in a slight improvement, illustrating the need for auxiliary training objectives to enable the classifiers to focus on subtle characteristics of the binding interface that can be overlooked in binding specificity-based training.

In our work, the computational TCR-pMHC complex structure prediction pipelines are decoupled from the binding activity prediction network. While finetuning the structure predictor can be explored¹⁶, a more relevant direction is quantifying the quality of a predicted structure based on not only the structure itself, but how it influences the classifier’s performance. Specifically, the latter approach can be used to prioritize the structural data to reduce the classifiers’ sensitivity to the choice of computational modeling tools. The original structural dataset from²⁸, and the two versions: relaxed struct and tcrmodel2 struct that we generated in our work, are relevant resources for pursuing this direction.

The list of potential factors impacting the generalization performance considered in our work is not exhaustive. In fact, the interaction between TCRs and the pMHC is dynamic in nature, that is difficult for GNN to learn from the static structures of the complexes. For example, the binding does not necessarily lead to functional activation of T-cell response⁴⁵ without the potential roles of other factors, such as the formation of a flexible catch-bond⁴⁶ at the TCR-pMHC interface. While insights from molecular dynamics simulation⁴⁷ can uncover unique characteristics of such complex interactions, integration of such knowledge into the graph encoding network is not straightforward. The development of the large structural datasets of TCR structures^48,49 potentially takes the current deep learning approaches one step closer toward considering the flexibility of complementarity-determining regions (CDR) loops of both α and β chains of TCRs into the binding specificity prediction. Despite advancements in accelerated techniques for protein conformation generation⁵⁰, their incorporation into TCR-pMHC structure modeling warrants detailed investigation, as findings from recent studies⁵¹ are not reassuring for protein-protein docking from the ensemble of conformations.

Our experiments demonstrated that the set of computationally predicted TCR-pMHC complex structures has different interaction characteristics that the GNN can lock onto as a shortcut during training, which do not necessarily lead to more generalized rules for discriminating binding samples with unseen peptides from non-binding ones. Through enforcing the classifier on particular interactions, models trained with the same computational dataset can show improved generalization performance, as illustrated with our proposed dual encoder approach. However, even for a cross-reactive TCR, the binding interfaces do not necessarily conform to the same 3D structural shape for different peptides⁵². Similar to the protein language model for sequence representation, the learned representation of the molecular complex with large-scale pre-training¹³ can potentially help with such structural discrepancies between training and test samples. For instance, a pre-trained graph encoder in ref. ⁵³ was shown to be a reliable zero-shot predictor of binding affinity between TCR and a small group of antigens. Moreover, as the computational approaches^54,55 for TCR-mimicking binder design targeting peptide-MHC complex continue to develop, the interaction profiles between the binder and the pMHC are likely to add new structural insights. While these designed binders may not have the same flexibility as TCRs, the insights can elucidate the local neighborhood preference near the pMHC interface for the binding activity.

GNNs have been widely adopted for capturing the structural characteristics of intermolecular interactions, e.g., protein-protein complexes. Our work assessed the reality of its potential in the challenging task of predicting TCR-pMHC binding specificity for unseen peptides, where the generalization issue is exacerbated by the diverse binding mechanisms with a limited number of experimentally determined structures. We hope that our computational experimental findings with different GNN-based classifiers, as well as the generated datasets, may facilitate further efforts to improve the prediction accuracy of the structure-based approaches with predicted TCR-pMHC structures.

Methods

Dataset and holdout test-set selection

We collected the dataset of Slone et al.²⁸ (denoted as init struct in our work) from https://github.com/KavrakiLab/STAG_public, where 19282 TCR-pMHC structures are reported with the binary class labels of binding and non-binding samples. The authors of ref. ²⁸ curated the positive samples from multiple databases: McPAS-TCR⁵⁶, VDJdb⁵⁷, IEDB⁵⁸ and 10x genomics⁵⁹, while the TCR-swapping strategy was employed to augment the pool of negative samples in addition to samples from 10x genomics. The TCR-pMHC structures were modeled with TCRpMHCmodels⁶⁰ and ImmuneScape⁶¹ based on the sequences of peptide, MHC, and TCRs (α and β chains obtained via Stitchr⁶²). The details of this data curation process can be found in ref. ²⁸.

Building on top of this dataset, we further created two additional datasets: relaxed struct and tcrmodel2 struct. Specifically, relaxed struct comprises Rosetta relaxed structures of all samples from init struct. For the latter case, we predicted the structures with TCRmodel2, which can sometimes predict structures with unrealistic binding or have low prediction confidence. Since TCRmodel2 generates 5 predicted structures for each TCR-pMHC sample, we retained the prediction with the highest model confidence (exceeding 0.49, based on the recommendation in https://tcrmodel.ibbr.umd.edu), which includes inter-chain edges between the peptide and TCRs. Consequently, we had to remove 987 samples (out of 19282), which includes 117 binding samples. Note that this set of removed samples includes one case where the TCRmodel2 program (both local and online server) failed.

Holdout test-set construction

We first rearranged the 345 unique peptides from init struct according to the counts of their corresponding TCR-pMHC samples. Then we picked the shaded region (Fig. 2b) of 35 peptides since they have a more uniform sample distribution, while allowing the test-set size to be around 1000. To minimize the sample discrepancy among test-set clusters, we first assigned the top 5 peptides (within these 35 selected peptides) to clusters 1, 2, 3, 4, and 5, respectively. For the assignment of the next top 5 peptides, we reversed the order, i.e., cluster 5 to cluster 1. This assignment process was repeated until all 35 peptides were distributed to these 5 holdout test-set clusters. Note that this sequential distribution strategy with periodic reversal of assignment order was applied to make sure that GNN-classifiers utilized a similar number of training samples for each holdout scenario, which allowed us to analyze their performance discrepancy across different test-sets.

Zero-shot structure-based binding prediction

For structure confidence-based approach, we predicted the TCR-pMHC complex structures with Boltz-2³⁶ from the sequences, and used the interface predicted template modeling (ipTM) score and the Boltz-2’s overall confidence score (0.8 × pLDDT + 0.2 × ipTM) to discriminate the binding vs non-binding samples. Here, pLDDT is the predicted local distance difference test for the predicted structure.

For interaction energy-based approach, we computed the interaction energy between pMHC and the α and β chains of TCRs in the predicted TCR-pMHC complex structures of init struct dataset. FoldX-5.1⁶³ and Rosetta^26,64 are used to estimate the biophysical energy between these two groups (pMHC and TCRs), preceded by each tool’s recommended pre-processing stages, i.e., repair and relax, respectively. The negative of such estimated energy is applied to rank the test samples, assuming that binding samples have a lower interaction energy than non-binding ones.

TCR-pMHC interface residue graph

Following the residue graph construction process of ref. ²⁸, we first identified the 3D coordinates of TCR-pMHC interface residues (C-α atoms) where the residue group of TCRs and the residue group of pMHC are within 14Å of each other. From the interface residues, the TCR-pMHC interface residue level graph (${\mathcal{G}}_{\mathrm{TCR}\mathrm{-}\mathrm{pMHC}}$) is constructed, where edges exist up to an inter-residue distance of 8Å. For a fair comparison with ref. ²⁸, which is the single encoder with EVTConv in our framework, we used the same node and edge features for the residue level graph from that work. Specifically, the biophysical attributes of amino acids like Atchley⁶⁵ and Kidera⁶⁶, along with one-hot encoding of chain types, were used as the node features (x_feat) of the TCR-pMHC interface graph.

For edge features, the radial basis function of distance (Eq. (1)) with 15 different scales is concatenated with the edge type representing vector (Eq. (2)). Here, τ_i is a one-hot encoding vector to indicate the chain of residue i.

$${\mathit{f}}_{j\to i,r}=\exp \left(-\frac{\mid \mid {\mathit{x}}_j-{\mathit{x}}_i\mid {\mid }^2}{1.5^r}\right)\mathrm{with}r=0,1,\cdots ,14$$ 1

$${\mathit{\tau }}_{j\to i}=\frac{1}{2}\left({\mathit{\tau }}_i+{\mathit{\tau }}_j\right)$$ 2

$${\mathit{e}}_{j\to i}={\mathit{f}}_{j\to i}\oplus {\mathit{\tau }}_{j\to i}$$ 3

Graph encoder

To extract graph level representation, i.e., $\mathit{g}=\varphi \left(\mathcal{G}\right)$, we considered two different GNNs: EGNN⁴¹ and EVTConv²⁸. The message passing steps for EGNN depicted in Eqs. (4) and (5) are different compared to ref. ⁴¹ since we kept the x_i,coords fixed during message propagation across L layers. The node feature x_i,feat is projected to ${\mathit{h}}_i^{\left(0\right)}$ through a MLP block, ϕ_node-MLP. ϕ_e and ϕ_h are MLP blocks for aggregating the messages from neighbors and combining the aggregated message with the current node embedding, respectively.

$${\mathit{m}}_{j\to i}={\varphi }_e\left({\mathit{h}}_i^{\left(l\right)}\oplus {\mathit{h}}_j^{\left(l\right)}\oplus {\mathit{e}}_{j\to i}\right)$$ 4

$${\mathit{h}}_i^{(l+1)}={\mathit{h}}_i^{\left(l\right)}+{\varphi }_h\left(\mathrm{LayerNorm}\left({\mathit{h}}_i^{\left(l\right)}\right)\oplus \sum _{j\in \mathit{N}\left(i\right)}{\mathit{m}}_{j\to i}\right),l=0,1,\cdots ,L-1$$ 5

On the other hand, EVTConv has edge-specific attention score (α_ij) to perform weighted message aggregation with edge-conditional encoding of the upcoming message via W_j→i. In Eqs. (6)–(8), W₀, W_q, W_k are learnable weight matrices, and W_j→i is predicted from edge embedding by a learnable MLP block ϕ_W. Note the edge embedding ${\mathit{h}}_{j\to i}^{\left(l\right)}$ is encoded from ${\mathit{h}}_i^{\left(l\right)}\oplus {\mathit{h}}_j^{\left(l\right)}\oplus {\mathit{e}}_{j\to i}$ through another MLP block (different than the ϕ_e of EGNN).

$${\mathit{h}}_i^{(l+1)}=\mathrm{LayerNorm}\left({\mathit{W}}_0{\mathit{h}}_i^{\left(l\right)}+\sum _{j\in \mathit{N}\left(i\right)}{\alpha }_{\mathit{ij}}{\mathit{W}}_{j\to i}{\mathit{h}}_j^{\left(l\right)}\right),l=0,1,\cdots ,L-1$$ 6

$${\mathit{W}}_{j\to i}={\varphi }_{\mathit{W}}\left({\mathit{h}}_{j\to i}^{\left(l\right)}\right)$$ 7

$${\alpha }_{\mathit{ij}}=\mathrm{softmax}\left(\frac{{\left({\mathit{W}}_q{\mathit{h}}_i^{\left(l\right)}\right)}^{\mathrm{\top }}{\mathit{W}}_k{\mathit{h}}_j^{\left(l\right)}}{\sqrt{d}}\right)$$ 8

Finally, the graph level representation is obtained via mean-pooling (Eq. (9)) of the updated node embeddings after L layers of message passing. We used L = 3 layers of GNN encoder with a message dimension of 32.

$$\mathit{g}=\frac{1}{n_{\mathcal{G}}}\sum _i{\mathit{h}}_i^{\left(L\right)}$$ 9

GNN-based classifiers

Typically, a residue-level graph is encoded by a GNN to produce a graph-level representation, which is further processed via a multilayer perceptron (MLP) block to predict the binding probability of the TCR-pMHC sample. We denote this approach as the single encoder approach of making the prediction. We proposed a dual encoder approach⁴⁰ to enforce the importance of the interaction involving the peptide in the training process of the classifier. As the name suggests, we applied two encoders to process the TCR-pMHC sample. The first encoder (ϕ₁) works the same way as in the single encoder approach. The second encoder (ϕ₂) encodes the subgraph ${\mathcal{G}}_{\mathrm{TCR}\mathrm{-}\mathrm{MHC}}$ where all peptide nodes are removed. The encoded representations from both encoders are concatenated to make the final prediction via the MLP block. In addition to the binary cross-entropy loss ${\mathcal{L}}_{\mathrm{CE}}$, we incorporated a semantic loss ${\mathcal{L}}_{\mathrm{semantic}}$ (Eq. (12)) to make the learned decision rule more biased toward the peptide-related interaction. Specifically, we encourage the embeddings g₁ and g₂ to have higher (lower) cosine distance for binding (non-binding) samples. Intuitively, this loss component can be thought of as a measure against over-exploiting other contexts that are not connected to peptide nodes in the residue-level graph.

$${\mathit{g}}_1={\varphi }_1\left({\mathcal{G}}_{\mathrm{TCR}\mathrm{-}\mathrm{pMHC}}\right){\mathit{g}}_2={\varphi }_2\left({\mathcal{G}}_{\mathrm{TCR}\mathrm{-}\mathrm{MHC}}\right)$$ 10

$${\widehat{y}}^{\mathrm{logit}}={\varphi }_{\mathrm{MLP}}\left({\mathit{g}}_1\oplus {\mathit{g}}_2\right)$$ 11

$${\mathcal{L}}_{\mathrm{semantic}}=\left\{\begin{array}{cc}1-\cos \_\mathrm{sim}\left({\mathit{g}}_1,{\mathit{g}}_2\right),\hfill & y=0\hfill \\ \max \left(0,\cos \_\mathrm{sim}\left({\mathit{g}}_1,{\mathit{g}}_2\right)\right),\hfill & y=1\hfill \end{array}\right)$$ 12

$$\mathcal{L}={\mathcal{L}}_{\mathrm{CE}}+{\alpha }_{\mathrm{sem}}{\mathcal{L}}_{\mathrm{semantic}}$$ 13

For the conditional encoding of the TCR-pMHC graph, we explored two approaches. In first approach, the TCR-MHC embedding ${\mathit{g}}_2={\varphi }_2\left({\mathcal{G}}_{\mathrm{TCR}\mathrm{-}\mathrm{MHC}}\right)$ is used as a global feature in encoding TCR-pMHC graph according to Eq. (15). The other approach utlizes g₂ (Eq. (16)) to estimate the parameters of node-MLP block ϕ_1,node-MLP of ϕ₁ which later encodes ${\mathcal{G}}_{\mathrm{TCR}\mathrm{-}\mathrm{pMHC}}$.

$${\mathit{g}}_1={\varphi }_1\left({\mathcal{G}}_{\mathrm{TCR}\mathrm{-}\mathrm{pMHC}},{\mathit{g}}_2\right)$$ 14

$${\mathit{h}}^{\left(0\right)}={\varphi }_{\mathrm{1,node}\mathrm{-}\mathrm{MLP}}\left({\mathit{x}}_{\mathrm{feat}}\right)+{\varphi }_{\mathrm{1,global}\mathrm{-}\mathrm{MLP}}\left({\mathit{g}}_2\right)$$ 15

$${\mathit{\theta }}_{{\varphi }_{\mathrm{1,node}\mathrm{-}\mathrm{MLP}}}={\varphi }_{\mathrm{1,global}\mathrm{-}\mathrm{to}\mathrm{-}\mathrm{node}\mathrm{-}\mathrm{MLP}}\left({\mathit{g}}_2\right)$$ 16

For each of the 10-fold cross-validation training (with fixed random seed for all experiments), we ran the training process with a learning rate of 0.0001 using the optimizer Adam⁶⁷ for a batch of 256 samples. We set the maximum training epochs to 350 with early stopping (patience of 25 epochs). The best model in each fold was recorded based on the ROC-AUC value on the validation split of the fold, with early stopping enabled after 50 epochs. Unless stated otherwise, the results of dual encoder approaches were from classifiers trained with α_sem = 1.

Training with paired preference data

For each fold’s paired preference data ${\mathcal{D}}^{\mathrm{p}\mathrm{a}\mathrm{i}\mathrm{r}}=\left\{\right({\mathcal{G}}_i,{\mathcal{G}}_j)\mid y_i=1,y_j=0,\mathrm{\Delta }{E}_{ij}\le {\delta }_{\mathrm{t}\mathrm{h}}\}$, we set δ_th = −50 to ignore the noise in the Rosetta-estimated interaction energy. We used a preference dataset of 3000 pairs, but there are more possible pairs that can be constructed from the training split. Picking the 3000 samples based on the lowest ΔE_ij can lead to lowered diversity of the TCR-pMHC samples in ${\mathcal{D}}^{\mathrm{pair}}$. Rather, we first assigned the pairs (A_ij = 1 instead of 0) that minimize Eq. (17) (linear sum assignment problem, solved with SciPy solver^68,69). The remaining required samples were collected from those pairs with an unassigned TCR-pMHC sample.

$$\underset{\mathit{A}}{\min }\sum _i\sum _j\mathrm{\Delta }{E}_{ij}{A}_{ij}$$ 17

$${\mathcal{L}}_{\mathrm{pref}}\left(i,j\right)=\log \left(1+\exp \left(-\left({\widehat{y}}_i^{\mathrm{logit}}-{\widehat{y}}_j^{\mathrm{logit}}\right)+m\right)\right)$$ 18

For the results shown in “Avenues for improving the generalization performance”, the classifiers are periodically (at each 5th epoch starting from 51st epoch) trained to minimize the ${\mathcal{L}}_{\mathrm{pref}}$ loss (Eq. (18) with m = 3). The learning rate for this pairwise preference training was halved from the learning rate for training with binary labels. We performed early stopping after 100 epochs in this case to allow the ${\mathcal{L}}_{\mathrm{pref}}$-based training to have an impact on the selected model.

Sequence-based predictors

For NetTCR-2.2, we reported the median and interquartile range of ROC-AUC for the predictions with 20 pan-peptide models (corresponding to 4-fold cross-validation with 5 partitions in ref. ⁴). For the EPACT⁴³, we used the architecture that utilizes CDR1-3 sequences from both α and β chains of TCRs in combination with MHC and peptide sequences. Following the convention of the original study, we computed the prediction on our test-set clusters by selectively averaging over binding score predictions from 5 trained prediction networks (corresponding to 5-fold cross-validation). Importantly, we excluded predictions (while averaging) from models that contained the test sample’s peptide in their training fold. This allowed us for fairer comparison without reducing the training datasets of these models. However, this did not account for the potential pre-training data leakage in the EPACT, e.g., pre-trained language models for peptides and TCRs.

Peptide residue rank in 9PBG and 9PBH

For each complex, we first predicted the TCR-pMHC structure from the sequences via TCRmodel2, and performed inference on the predicted structure with all 50 models (corresponding to 10-fold cross-validation for five test-set clusters) trained with tcrmodel2 struct dataset. For each prediction, we also computed the relative rank (gradient saliency) of the peptide residue nodes in the TCR-pMHC interface graph based on the magnitude of the gradient of model output logits with respect to the node features. Next, we partitioned these predicted binding scores into two splits based on a 0.5 threshold, and computed the average rank of peptide residues in each split.

Author contributions

A.N.M.N.A. and B.-J.Y. have conceived the study. A.N.M.N.A. designed the methodology, performed experiments, and wrote the first draft of the manuscript. R.S.R., X.Q., B.-J.Y. contributed to the methodology and revised the manuscript.

Data availability

The dataset from Slone et al.²⁸ is found at https://github.com/KavrakiLab/STAG_public. The datasets of Rosetta-relaxed and TCRmodel2 structures are shared in Zenodo (10.5281/zenodo.18344673).

Code availability

We have used the codes from https://github.com/EsamTolba/TCR-CoM for computing docking and crossing angles (Singh et al.⁷⁰) of TCR-pMHC structures. For TCRmodel2, the majority of the structures are predicted with local installation from https://github.com/piercelab/tcrmodel2 (commit 5ffb362), and the rest were done with the web-server at https://tcrmodel.ibbr.umd.edu. The codes for the experiments of this study are deposited at https://github.com/nafizabeer/TCR-pMHC-Struct-generalization.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s44386-026-00064-3.