TCR representation learning with protein language models: a comprehensive review

Abstract

The T-cell receptor (TCR) repertoire is a valuable source of information that reflects an individual’s immune status and infection history. However, due to the exceptional diversity and complexity of the TCR repertoire, predicting its functional properties remains a challenging task. This review summarizes recent advances in protein language models (PLMs), which apply natural language processing techniques to protein sequences, focusing specifically on TCR repertoire analysis. We begin by outlining the biological basis of the TCR repertoire and its current clinical applications. We then describe the methods used for representing TCR data and the training procedures of the corresponding PLMs. PLMs capture context-dependent features from large unlabeled TCR datasets and achieve high generalization performance even with limited labeled data through transfer learning. In this respect, PLMs offer significant advantages over conventional sequence representation methods. We highlight antigen specificity prediction as a key application, comparing supervised deep learning models with PLM-based approaches. While employment of PLMs is promising, TCR repertoire analysis still faces challenges such as data scarcity, bias, and lack of paired-chain information. Addressing these challenges requires rigorous dataset optimization, integration, and augmentation strategies. Future advances will require better interpretation of the representations learned by PLMs and the development of multimodal approaches that integrate structural information. These advances could enable several clinical applications, including disease diagnosis, vaccine development, and personalized immune profiling.

Graphical Abstract

Graphical Abstract.

TCR representation learning with language models

Introduction

The immune system serves as the foundation of biological defense against external threats, with T cells playing a central role by recognizing pathogen-derived peptides through T-cell receptors (TCRs). TCRs form diverse sequence patterns within an individual through gene rearrangement, and the TCR repertoire—the collection of these patterns—reflects the individual’s immune status and infection history (1). Advances in next-generation sequencing technologies have significantly enhanced TCR repertoire analysis, leading to progress in elucidating immune mechanisms and applications for disease diagnosis. However, extracting biologically meaningful information from large-scale TCR datasets and predicting functional characteristics such as antigen specificity remains a significant challenge (2, 3).

Recently, language models originally developed for natural language processing have been applied to protein sequence analysis (4, 5). Protein language models (PLMs) treat amino acid sequences as language, capturing structural and functional features by learning sequential patterns and contextual relationships from large-scale data (6–8). The PLM framework opens new possibilities for analyzing highly diverse TCR repertoires, enabling applications in antigen specificity prediction and disease diagnosis beyond the capabilities of conventional methods.

This review surveys recent research trends in the application of deep learning technologies, particularly PLMs, to TCR repertoire analysis. We begin by discussing the biological foundation of TCR repertoires and the current landscape of their clinical applications. Subsequently, we detail data representation methods for TCRs and the learning process of PLMs, comparing advantages and challenges with conventional approaches. We then focus on antigen specificity prediction as an application of TCR representation learning, presenting a systematic review of supervised deep learning models and PLM-based approaches with specific examples. Furthermore, we explore the current status and challenges of TCR repertoire data, as well as data optimization and augmentation strategies to address them. Finally, we provide a summary and promising directions for future research in this field.

Overview of TCR repertoire

TCR diversity and individual specificity

The TCR, along with the B-cell receptor (BCR), is an essential molecule that constitutes the Adaptive Immune Receptor Repertoire (AIRR). The TCR typically works as a heterodimer consisting of TCRα and TCRβ chains. The variable regions of these chains form three-dimensional structures that recognize peptide antigens presented by major histocompatibility complex (MHC) molecules (9). The diverse collection of TCR clonotypes present in an individual, known as the TCR repertoire, is responsible for monitoring and recognizing a wide range of antigens derived both from its own body cells and various external pathogens. The total number of T cells in the human body is estimated to be approximately 10¹¹ (10). TCR repertoire diversity is generated through somatic recombination of Variable (V), Diversity (D), and Joining (J) gene segments (11). This recombination results in the TCR variable region, comprising four framework regions and three complementarity-determining regions (CDRs). These CDRs form flexible loop structures, known as CDR loops, that extend from the TCR variable domain and directly interact with the peptide-MHC (pMHC) complex. The CDR3 region, one of the three CDRs serving as the major interaction site of antigen recognition, undergoes nucleotide deletions and insertions during gene recombination, further enhancing sequence diversity. T cells expressing unique TCRs generated through somatic recombination proliferate when they encounter their specific antigens in peripheral tissues. This process results in a TCR repertoire that is unique to each individual.

Significance of understanding TCR repertoires

During infections, T cells that bind to specific antigens undergo clonal expansion, forming infection-specific TCR repertoires (12–14). Since the TCR repertoire reflects immune status and infection history, numerous studies have applied TCR analysis to the diagnosis and monitoring of infectious diseases, autoimmune diseases, and cancer. For example, Zaslavsky et al. developed Mal-ID (Machine Learning for Immunological Diagnosis), an ensemble approach that combines multiple machine learning models using TCR and BCR data (15). Their approach integrated three distinct models: (i) repertoire composition analysis using V/J gene segment frequencies, (ii) CDR3 clustering to detect disease-associated sequence motifs shared across patients, and (iii) PLM embeddings to capture functional similarities beyond sequence alignment. This ensemble approach achieved multi-class area under the receiver operating characteristic curve (AUROC) of 0.986 across six disease states, including coronavirus disease 2019 (COVID-19), human immunodeficiency virus, and type 1 diabetes. Furthermore, Ruiz Ortega et al. analyzed repertoire sharing patterns between healthy individuals and COVID-19 patients using probabilistic models (16). This analysis led to the development of a diagnostic tool to predict severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection status. In autoimmune diseases, decreased diversity and the presence of patient-specific T-cell clones have been reported as potential biomarkers (17, 18).

Building on these advances, several TCR repertoire analysis technologies have been translated into clinical practice and continue to progress toward broader medical applications. Notable examples include Adaptive Biotechnologies Corporation’s T-Detect COVID and T-Detect Lyme. T-Detect COVID uses next-generation sequencing to analyze TCR sequences from peripheral blood and identifies COVID-19-specific patterns by employing machine learning models (19). This test received Food and Drug Administration (FDA) Emergency Use Authorization in March 2021. Similarly, T-Detect Lyme assists in the early diagnosis of Lyme disease by analyzing TCRs from blood samples to detect T cells that respond to Lyme disease-associated antigens (20). Beyond diagnostics, TCR-based therapies have made significant advances in cancer immunotherapy. In August 2024, the FDA approved Tecelra, the first TCR therapy for solid tumors, specifically targeting advanced synovial sarcoma (21). This engineered TCR therapy targets the MAGE-A4 protein and achieved a 43% response rate with a 6-month median duration of response in clinical trials. Croce et al. combined phage display with machine learning to identify TCRs targeting a cancer-testis antigen that is highly expressed in tumors but shows restricted expression in normal tissues [area under the curve (AUC) 0.965] (22). For successful clinical application, TCR prediction models must demonstrate robustness across diverse patient populations and provide interpretable outputs that clinicians can evaluate for treatment decisions. These clinical applications demonstrate that specific patterns and sequence features within the TCR repertoire are strongly associated with disease states. Establishing appropriate data representation methods for TCR sequences is essential to effectively capture these patterns and accurately determine their association with diseases.

TCR representation using language models

Conventional TCR representation methods

Because of the extensive diversity and high dimensionality of TCR sequences, numerous studies have explored methods to effectively represent their characteristics. Representation learning is a technique that extracts high-level abstractions and features from raw data for capturing the essential properties of the data. Effective feature representations of TCR sequences could substantially enhance the performance of functional analyses, including sequence similarity evaluation and antigen specificity prediction. However, establishing optimal representation learning methods remains a considerable challenge.

TCR sequences are amino acid chains of variable length, requiring appropriate encoding techniques for subsequent processing with machine learning algorithms. The primary encoding methods proposed to date include: (i) BLOSUM-based encoding (23–25), (ii) k-mer-based encoding (26–28), and (iii) encoding based on physicochemical properties such as charge, hydrophobicity, solubility, and pH (29–31). However, given the significant diversity and variable length of TCR sequences, particularly in the CDR3 region, these conventional methods are limited in capturing context-dependent features and potential higher-order structural information within sequences.

In particular, these limitations in TCR representation directly impact similarity evaluation. In this context, TCR similarity refers not only to sequence similarity but also to functional similarity, such as the ability to recognize the same antigen. Multiple studies demonstrate that TCRs with similar sequences are more likely to recognize the same epitope (32–34). However, sequence similarity does not always directly correspond to functional similarity, as these properties typically exhibit a nonlinear relationship (35). Therefore, simple sequence similarity metrics may not accurately reflect the functional relevance of TCRs, making the acquisition of functionally meaningful similarity metrics essential.

The rapid development of next-generation sequencing technologies has made vast amounts of TCR sequence data available (36). With access to these extensive sequence collections, PLMs have emerged as powerful tools for predicting protein structure and function.

Learning methods of PLMs

The development of PLMs follows a comprehensive pipeline encompassing several critical phases: preparing training data, converting amino acid sequences into tokens, embedding these tokens into vector space, processing through Transformer-based architectures, self-supervised pre-training on large datasets, and task-specific fine-tuning (Fig. 1). This section outlines how PLMs acquire representations related to protein structure and function from sequence information. TCR representation learning has developed in parallel with PLMs that employ self-supervised learning on protein sequences. In self-supervised learning, models acquire meaningful representations by predicting masked amino acids or neighboring residues without external labels. Notable examples include UniRep (37), TAPE (38), ESM2 (7), ProteinBERT (6), and ProGen (8), which are pre-trained on extensive protein databases like UniRef (39) and Pfam (40). These models effectively extract structural and functional protein features through self-supervised learning on large amino acid sequence datasets (41). However, TCRs exhibit unique characteristics as immune receptors, particularly their exceptional diversity centered in the CDR3 region and V(D)J recombination. Because of these distinctive features, numerous language models specifically designed for TCR sequences have been developed in recent years. This paper mainly focuses on such models with TCR amino acid sequences as the input data.

Figure 1.

Computational pipeline for protein sequence representation from protein and TCR sequence databases to downstream applications. The three major approaches (feature-based encoding methods, general-purpose PLMs, and TCR-specific PLMs) extract features based on large-scale sequence information. PLM-based methods employ self-supervised learning to capture contextual patterns from extensive unlabeled data. These extracted features subsequently serve as input for supervised models trained on task-specific labeled data. Representative models for each category are indicated in the figure.

PLMs are primarily based on the Transformer architecture, and their learning process begins with tokenization of the input data. For protein sequences, single amino acid tokenization is the predominant approach. This approach treats each amino acid as a separate token (7, 38, 42). After tokenization, discrete tokens (amino acids) are converted into continuous vector representations through the embedding layer. These high-dimensional embeddings capture the properties of each token, providing the basis for subsequent processing. Additionally, positional encoding is incorporated into the embedding vectors to preserve information about token positions within the sequence. The token embeddings and positional encodings are then integrated to form the final input representation for the model.

Following input data preparation, the language model processes the resulting representations through its neural network architecture. In this paper, we focus on models based on the Transformer architecture (43), which dominate recent PLM development. A Transformer consists of two main components: an encoder that extracts context-dependent features and a decoder that generates new sequences based on the given context. Transformer-based PLMs can be classified into three architectural types: encoder-only, decoder-only, or encoder-decoder models. A prominent example of an encoder-only model is BERT, a natural language processing model developed by Google (4). In the field of PLMs, models such as ProteinBERT and TCR–BERT have been developed based on the BERT architecture (6, 44) and used for protein sequence classification and functional annotation. A notable example of a decoder-only model is GPT, a large language model developed by OpenAI (5). Similarly, protein models such as ProtGPT2 and ProGen have been developed based on the GPT architecture (8, 45) and employed for de novo protein sequence generation and design. Both the encoders and decoders in these architectures incorporate multi-head attention mechanisms, normalization layers, and feed-forward layers. Through these layers, the model learns the relationships between tokens in sequences, whether for contextualizing inputs (encoders) or generating outputs (decoders). For the remainder of this paper, we focus on encoder-only models.

Among the several stages in PLM training, the pre-training stage plays a crucial role. At this stage, the model undergoes self-supervised learning on large datasets of TCR sequences. The two primary pre-training methods are Masked Language Modeling (MLM) and contrastive learning. MLM randomly masks a portion of the input tokens and trains the model to predict these masked tokens on the basis of their surrounding context. This approach enables the model to learn distribution patterns of amino acids within sequences. The MLM method is widely used in BERT-based language models (4).

In contrast, contrastive learning approaches train models to minimize the distance in the embedding space between similar data points while maximizing that between dissimilar ones (46–48). The definition of similarity varies across different methods. For example, ATMTCR generates multiple independent “views” from the same TCR by masking important residues identified by the attention mechanism, which are then treated as similar data points (46). Similarly, in SCEPTR, “views” are generated through data augmentation techniques such as residue deletion, alpha and beta chain removal, and application of dropout noise (47). TouCAN takes a different approach by defining functionally similar TCRs—those that recognize the same antigen despite having different sequences—as similar data points (48). Contrastive learning is particularly effective in situations with limited supervised data. Nagano et al. demonstrated that contrastive learning outperforms MLM alone in antigen specificity prediction when performing transfer learning with limited labeled data (47). These pre-training approaches enable the model to acquire general representations of protein sequences, establishing a robust foundation for subsequent task-specific applications.

After pre-training on large-scale data, models are typically fine-tuned for specific downstream tasks (e.g. antigen specificity prediction). In the fine-tuning stage, the entire model undergoes additional training with labeled task-specific data, employing the representations learned during pre-training. Since the basic structure and many parameters of the model are already optimized during pre-training, fine-tuning enables effective performance with fewer steps and reduced computational resources. Representative downstream tasks in the field of proteomics include predicting protein structure, function, physicochemical properties, mutation effects, and antigen-receptor binding (49). Notably, models pre-trained using MLM on large-scale TCR sequence data, such as TCR–BERT (44) and tcrLM (50), exhibit superior performance in downstream antigen specificity classification.

In summary, the PLM pipeline consists of several key stages: data preparation, tokenization, input embedding, processing by Transformer models, pre-training, and fine-tuning. This systematic approach empowers the model to effectively characterize protein complexity and achieve superior performance across a wide range of protein-related tasks.

Advantages of PLMs for TCR representation

For TCR sequence representation learning, conventional feature-based methods and PLM-based techniques present characteristic advantages, but also entail specific limitations (Fig. 2). In this section, we summarize the key benefits and constraints of using PLMs for TCR representation.

Figure 2.

Comparison of major approaches for TCR sequence representation by contrasting workflows and technical features across three TCR representation methods. The left column depicts feature-based methods utilizing conventional encodings (BLOSUM, k-mer, and physicochemical properties), which offer high interpretability but exhibit limitations in contextual understanding. The middle column illustrates general-purpose PLMs that learn through masked language modeling (MLM) from diverse protein sequences, providing rich contextual representations. The right column highlights TCR-specific PLMs optimized for TCR patterns, either by fine-tuning general-purpose PLMs (upper part) or by training directly on TCR sequences (lower part). The bottom section summarizes the key advantages (marked with +) and limitations (marked with −) of each approach.

Rich information content and representational diversity of PLMs.

Unlike traditional encoding methods based on the BLOSUM, k-mer-based encoding, and physicochemical properties, PLMs can identify higher-order patterns from large-scale sequence data and generate representations that capture the contextual meaning and interrelationships of amino acids more effectively. By visualizing PLM-derived representations, researchers have identified various biological patterns, including structural information, functional properties, and genetic features. For example, Rives et al. demonstrated that secondary and tertiary structure information is encoded in representations learned solely from sequence data (51). Furthermore, several studies have shown that the visualization of TCR sequence embeddings generated by PLMs reveals a range of characteristics, including phylogenetic classification of protein families, antigen specificity, V/J gene-level information, and distinctions between public and private TCRs (44, 49, 52, 53).

Transfer learning capabilities.

As previously discussed, PLMs undergo a two-phase training process: extensive pre-training on large unlabeled datasets, followed by fine-tuning on specific labeled datasets. This approach embodies the core principle of transfer learning, where knowledge gained during pre-training is transferred to downstream tasks during fine-tuning. While PLMs require substantial data and computational resources during pre-training, once trained, they can be adapted to individual tasks through this approach. These pre-trained models optimized for transfer learning are often referred to as foundation models. Transfer learning enables the reuse of general-purpose representations derived from extensive data, making it particularly valuable in biological domains where collecting sufficient task-specific data is often challenging. In contrast, conventional feature representation methods require task-specific model design and training procedures, including learning data embeddings and extracting features from scratch for each task, which increases the burden of both model development and data preparation.

Interpretability of PLM representations.

While PLMs offer efficient and context-aware representations of biological sequences, these models face a fundamental challenge in interpretability: the biological significance of their learned representations is often obscure and difficult to connect with established domain knowledge. Conventional feature representations (such as encodings based on amino acid k-mers or physicochemical properties) are often highly interpretable. For example, the presence or absence of specific k-mers can be treated as discrete features with potential associations to particular biological functions. GLIPH uses k-mer patterns to identify TCRs with shared antigen specificity (33). In contrast, PLMs employ complex model structures with dimensions that do not directly correspond to physical or functional meanings, making biological interpretation difficult. To address this challenge, researchers are working to enhance interpretability by mapping representations learned by PLMs to established biological knowledge. Recent advances in interpretability have been achieved through techniques such as attention mechanism analysis, post hoc interpretation methods, and novel model architectures. For example, the multi-head attention mechanism in Transformer models assigns high weights to important positions in the input sequence, providing insights into which parts of the sequence are most relevant. Wu et al. demonstrated that attention weights were concentrated on residues that physically contact the antigen (44). PISTE effectively identified residue-level contact relationships among TCR, antigen, and human leukocyte antigen (HLA) by incorporating physical insights into conventional attention mechanisms without requiring structural data (54). Post hoc interpretation approaches such as SHapley Additive exPlanations (SHAP) (55) have been successfully applied to TCR models to quantify the contribution of individual features to predictions. For instance, TCR-H employs SHAP analysis to identify the physicochemical properties that drive TCR–epitope binding predictions, providing insights into the underlying molecular mechanisms (56). Furthermore, Leary et al. showed that TCR–VALID enables the quantification of each dimension’s importance in predicting specific generative factors by separately encoding the average physicochemical properties of V genes, J genes, and the insertion regions (57).

In summary, the field has made significant progress in learning latent representations that efficiently capture information from large-scale TCR sequence datasets. However, further research is needed to understand how diverse biological information is encoded in these representations and to improve their biological interpretability.

Antigen specificity prediction using TCR representations

Antigen specificity prediction is one of the most important applications of TCR sequence representation learning using PLMs. Computational methods for this task predict the binding interactions between TCRs and pMHC complexes. Specifically, they either identify the pMHCs that bind to a given TCR or, conversely, identify the TCRs that recognize a particular pMHC. This technology possesses substantial potential for applications in vaccine development and infectious disease diagnostics. In this section, we review the recent advancement to this task by categorizing approaches into two distinct frameworks: supervised deep learning models and language model-based techniques.

Supervised deep learning models

A substantial body of research has focused on developing supervised models that learn from large-scale TCR–epitope binding data to predict the binding affinity of specific epitopes or TCR sequences (2, 24, 58–67). Despite their capabilities, these supervised models face generalization challenges depending on the availability of training data.

It is essential to distinguish between two scenarios: prediction for seen epitopes (those well-represented in the training data) and unseen epitopes (those absent from or scarce in the training data). For seen epitopes, these models produce reliable predictions even when applied to novel TCR sequences. Analysis of existing TCR–epitope binding data reveals that supervised models require at least 50 αβTCR binding examples per pMHC to achieve reasonable prediction accuracy (58). Among the 43 pMHCs in public databases that satisfy this threshold, the MixTCRpred model achieved an average AUC above 0.7 for 40 pMHCs. Montemurro et al. demonstrated that peptides characterized with over 200 TCRs achieved an average AUC of 0.88, while those with fewer than 20 TCRs exhibited significantly reduced performance (AUC 0.38) (25). However, the pMHCs with sufficient amounts of binding data represent only a small fraction of the immunologically relevant epitope space.

For unseen epitopes, prediction is inherently difficult due to the paucity of training examples. While fair benchmarking of models remains challenging due to the lack of dataset standardization and inconsistent strategies for negative sample preparation, the IMMREP23 results clearly demonstrate this challenge: for epitopes that were not present in public databases, most models performed only marginally better than random chance (68). Even the best unbiased model which employed embeddings from PLMs such as ESM2 achieved a maximum AUC of 0.62. This performance gap between seen and unseen epitopes represents a fundamental limitation of existing supervised approaches. TEIM addresses this limitation through a two-stage learning pipeline: pre-training on abundant sequence-level binding data followed by fine-tuning on limited residue-level interaction data. Unlike template-based methods that fail when no structural templates exist in the training set, TEIM maintains robust performance even for these challenging cases (correlation 0.896 between predicted and measured distances) (60).

PLMs for antigen specificity prediction

To address the scarcity of TCR–epitope binding data in antigen specificity prediction, transfer learning for TCR representations has emerged as a notable approach. Transfer learning, as discussed in the section on 'TCR representation using language models’, represents a key application of PLM-based TCR representations. These approaches typically involve models that first learn the sequence space of TCRs by training on large-scale unlabeled TCR sequence data, followed by fine-tuning with TCRs of known antigen specificity. Using this two-stage approach of pre-training and fine-tuning, PLM-based models have demonstrated higher generalization performance with limited supervised data compared to conventional supervised models. In fact, fine-tuned PLMs have shown superior performance relative to non-pre-trained models in predicting peptide binding to MHC class I molecules (62, 69).

Within this transfer learning framework, PLMs applied to TCR binding prediction can be categorized into two approaches based on their pre-training strategies: (i) those utilizing general-purpose PLMs pre-trained on diverse protein sequences, and (ii) those employing TCR-specific pre-trained models (Fig. 2, right column). Both approaches share the fine-tuning stage using TCRs with known antigen specificity, but differ in their pre-training strategies and the nature of the learned representations. The rationale for employing general-purpose PLMs stems from the fact that protein binding, as a physical process, is not specific to TCRs. Therefore, models trained on extensive protein datasets may capture fundamental physicochemical properties relevant to binding predictions, even when applied to TCR–epitope interactions. General-purpose approaches, including TCRconv (70), EPIC-TRACE (71), TCR–ESM (72), and TouCAN (48), adapt models such as ESM (51) and protBERT (73) to the TCR domain. These methods employ PLMs as feature extractors to generate general protein embeddings, which are then input into supervised classifiers trained on labeled TCR–epitope data to predict binding interactions.

In contrast to general-purpose PLMs, TCR-specific pre-training approaches have recently emerged. These models first pre-train on large-scale unlabeled TCR sequences to learn TCR-specific representations, then fine-tune on labeled TCR–epitope binding data for antigen specificity prediction. For example, TCR–BERT (44) achieved superior antigen prediction performance (average AUPRC of 0.91 across 26 antigens) compared to general-purpose PLMs such as ESM (51) by employing TCR-specific pre-training on 88 403 CDR3 sequences followed by fine-tuning on 4365 antigen-annotated sequences. Fang et al. performed large-scale pre-training on over 100 million CDR3 sequences followed by fine-tuning using approximately 110 000 antigen-binding data points. On COVID-19 datasets, tcrLM achieved AUROC and AUPR values of 0.595 and 0.602, respectively, while existing methods showed near-random performance (50). These models focus on TCR-specific datasets during pre-training to develop representations that capture the unique characteristics of immune receptor recognition.

The choice between general-purpose and domain-specific PLMs involves important trade-offs. General-purpose PLMs are trained on diverse protein datasets, potentially capturing fundamental physicochemical properties relevant for binding predictions across protein families. Their larger model capacity [e.g. ESM2-650M (7) contains approximately three times as many parameters as TCR–BERT (44)] enables them to learn more complex patterns. However, domain-specific models are tailored to immune receptor structure and recombination logic, potentially providing advantages in specialized tasks or under data-limited conditions. A recent comparative study demonstrated that general-purpose PLMs like ESM2 can achieve performance comparable to domain-specific models such as TCR–BERT on TCR–antigen specificity prediction tasks, with no statistically significant differences in AUC, accuracy, or F1 score (74). This suggests that the broader protein knowledge encoded in general-purpose PLMs can effectively compensate for their limited TCR-specific domain knowledge.

Many existing models focus on the CDR3 region, which is the most diverse and critical region for epitope binding. However, additional factors play crucial roles in determining epitope specificity, including paired-chain information (TCRα and TCRβ chains) and other CDR loops (CDR1 and CDR2), which collectively form the antigen recognition site. Incorporating such information into the training process enables more accurate modeling of the structural features governing protein folding and binding interactions. A benchmark study comparing 23 binding prediction models has demonstrated that incorporating paired-chain information and CDR loops other than CDR3 improves the accuracy of antigen specificity prediction, with paired-chain models showing average AUC improvements of 0.02–0.07 and CDR1/2/3 models showing improvements of 0.01–0.05 compared to their respective baselines (3). For example, models such as TCR–ESM (72), SC-AIR-BERT (75), and TCRconv (70) have shown that paired-chain information significantly improves performance. The SC-AIR-BERT study demonstrated this improvement across 10 epitopes, with paired-chain models achieving an average AUC of 0.875 compared to 0.782 for single α-chain models and 0.677 for single β-chain models. Zhang et al. showed that considering all CDR loops in binding specificity prediction using VDJdb data improved the average AUC from 0.665 to 0.697 and the average AUPR from 0.382 to 0.443, compared to using only the CDR3 region (76). Despite significant improvements in accuracy achieved through advanced models and additional data sources, predicting binding to unseen epitopes remains a largely unsolved problem. To achieve the precision required for practical applications, substantially larger and more comprehensive datasets will be essential.

TCR repertoire resources for representation learning

The effectiveness of PLMs for TCR repertoire representation is highly dependent on the quality and scale of available training data. The following sections explore the current status of TCR datasets, their inherent limitations, and emerging approaches to enhance these valuable resources.

Current status and limitations of TCR datasets

The success of TCR representation learning depends critically on addressing several key limitations in available data resources. This section discusses the current TCR data landscape, highlighting challenges related to dataset bias, technical scalability constraints, negative data generation, and limited paired-chain information.

TCR data repositories and their applications.

Establishing reliable datasets is crucial for deep learning-based representation learning research. The number of datasets available for TCR repertoire analysis has expanded significantly in recent years. Among these datasets, VDJdb stands out as a comprehensive repository that contains a substantial collection of antigen-specific TCRs (77). The IEDB also serves as a common database for antigen specificity prediction tasks (78). Furthermore, McPAS-TCR aggregates TCR sequences related to pathologies such as infectious diseases, autoimmune diseases, and cancer, providing a valuable resource for training and evaluating models focused on particular domains (79). Additionally, the STCRDab and TCR3d databases provide three-dimensional TCR structural information (80, 81). Structure-based approaches demonstrate superior generalization performance compared to sequence-only methods, particularly when training data are limited (82–86). For instance, DeepAIR, which integrates structural and sequence information, achieved a median AUC of 0.904 in TCR–antigen binding prediction, significantly outperforming both sequence-only methods (AUC of 0.827) and other state-of-the-art approaches (AUC of 0.782−0.845) (84).

Data bias and technical constraints in available TCR–epitope pair data.

Despite the ongoing growth of TCR databases, computational models for predicting TCR–antigen specificity face significant challenges primarily due to inherent biases in available datasets and technical constraints limiting data collection scale. Public TCR–epitope pair datasets exhibit substantial representation imbalances: approximately 97% of antigens known to bind to TCRs are of viral origin, and only 100 antigen types account for 70% of TCR–epitope pairs (87). Additionally, these databases suffer from low data reliability: only 34.7% for YLQ epitope and 63.4% for GLC epitope in VDJdb demonstrated actual functional reactivity when experimentally tested (88). This combination of representation bias and quality issues severely limits the generalizability of prediction models across diverse antigen types. Furthermore, technical constraints limit the number of TCR–epitope pairs that can be detected simultaneously. Over the past decade, high-throughput methods have emerged to identify TCRs specific to multiple antigens. Multiplexed identification of T-cell receptor antigen specificity (MIRA) creates pools with unique combinations of antigens and analyzes T-cell enrichment patterns when these pools are exposed to T-cell samples (89). This method enables more efficient identification of antigens recognized by individual T-cell clones compared to conventional single-antigen testing approaches. Using this technology, Nolan et al. established a database containing more than 160 000 SARS-CoV-2-specific TCRβ sequences, which has become an important resource for COVID-19 research (90). Single-cell pMHC multimer analysis labels antigen-specific T cells using pMHC multimer reagents and identifies TCR sequences with single-cell resolution. Recent advances enable simultaneous detection of T-cell binding specificity across multiple epitopes using DNA-barcoded pMHC multimers (91). For example, research using the 10× Genomics platform has identified approximately 15 000 TCR clones that bind to 44 distinct epitopes in a single experiment (92). Despite these technological advancements, the fundamental issues of dataset bias and technical scalability constraints continue to hinder the development of robust and generalizable TCR–antigen specificity prediction models.

The challenge of reliable negative data generation.

Many existing TCR–antigen specificity prediction models require both TCR–epitope binding pairs (positive data) and non-binding pairs (negative data) for training (2, 66). Ideally, experimentally verified negative data should be used, as implemented in the TCR-H model, which exclusively relies on such data for training (56). Negative data can also be obtained in theory through techniques such as MIRA or single-cell pMHC multimer analysis (89, 92). However, the raw data obtained by these methods is not systematically organized in a standardized, publicly accessible format, which limits the acquisition of datasets of sufficient scale for model training. To circumvent this data shortage, researchers have typically employed alternative methods, such as pairing epitopes from known positive data with random TCRs drawn from a background set (93). However, such randomly generated negative data may contain sequences that actually bind to the target epitopes, potentially undermining model performance (62). To mitigate this risk, several alternative methods for generating negative data have been proposed. Jiang et al. compared four strategies for generating negative data and concluded that the most effective approach involves pairing TCRs with epitopes sampled according to their frequency in positive data sets (62). Myronov et al. generated more reliable non-binding data by applying cluster analysis to reference TCR sequences from healthy donors, thereby reducing the potential for false negatives (94).

Limited availability of alpha-beta chain pairing information.

While important, the paired TCRα and TCRβ chain data are still insufficient and their measurement remains a major challenge (87). Current sequencing technologies face a fundamental trade-off: bulk TCR sequencing captures 10³–10⁶ TCRs per assay but cannot resolve chain pairing, potentially resulting in an underestimation of TCR diversity. In contrast, single-cell sequencing provides paired-chain information but with significantly lower throughput (~65% cell capture efficiency, requiring a minimum of ~1000 cells) and higher cost, which limits its application to small T-cell subpopulations (36). This trade-off between throughput and chain pairing poses a particular challenge for PLM training.

Recent technological innovations are beginning to address this limitation. For example, novel paired-chain sequencing methods now enable cost-effective and high-throughput analysis by employing miniaturized reactions in 384-well plates (95). TIRTL-seq can process 10 million peripheral blood mononuclear cells (PBMCs) at one-tenth the cost of standard 10x Genomics experiments while maintaining comparable accuracy. Furthermore, paired-chain TCR sequence databases such as OTS provide a valuable resource for the research community (96). Although publicly available paired-chain data have not yet reached the scale of TCRβ-chain-only datasets, these technological advances suggest that sufficient data for effective PLM training will become available in the near future.

Enhancing TCR datasets: optimization, integration, and synthetic approaches

Finally, we explore additional approaches to overcome the limitations in current TCR datasets. This section discusses strategies for enhancing dataset utility, standardizing existing resources, and generating synthetic data to address the constraints discussed previously.

Maximizing dataset utility through dataset optimization and incomplete data utilization.

The challenge of limited TCR datasets can be addressed through cost-effective data collection strategies and computational methods that maximize the utility of available data, particularly incomplete datasets. For example, Zhang et al. proposed a framework that uses entropy to quantify model uncertainty about the binding status of each pair in the entire unlabeled data pool (97). This framework achieved a 40% reduction in annotation costs compared to random sampling. Moreover, Montemurro et al. demonstrated that both the reduction of redundant TCRs and negative data selection significantly improved generalization performance (2).

In addition to these data collection optimizations, computational approaches have emerged that can effectively utilize incomplete datasets. For instance, TULIP employs TCR–epitope binding pairs derived from datasets with incomplete information on TCRα chains, TCRβ chains, MHC, and epitopes (98). Despite the incomplete nature of the input data, TULIP achieved superior performance compared to the supervised model NetTCR-2.0 (25), demonstrating higher AUC values across all epitopes with at least 20 TCRs in the training data. This success highlights how advanced computational methods can mitigate the limitations of incomplete experimental data, offering a cost-effective alternative to the generation of complete paired-chain datasets.

Standardization and consolidation of distributed TCR resources.

As the scale of TCR repertoire analysis expands, there is an increasing need for systems capable of searching and analyzing the extensive volume of TCR sequence data distributed across multiple research institutions and databases. iReceptor provides a platform that integrates distributed data into a standardized format (MiAIRR) and enables the comprehensive search and retrieval of sequences (99). The MiAIRR format defines the minimum set of metadata required for interpreting and comparing AIRR-seq experiments and datasets, which is essential for effective data reuse. Similarly, ongoing efforts focus on aggregating diverse TCR sequence data from research groups worldwide, annotating these sequences with relevant information and standardized metadata, and making it publicly available through repositories such as TCRdb and PIRD (100, 101). These integrated databases serve as a foundation for model training and analysis by providing essential contextual metadata.

Computational generation of synthetic TCR data.

Because of the difficulty of obtaining large-scale functionally labeled data experimentally, approaches to augment and complement data through simulation have attracted significant attention. Models such as IGoR, immuneSIM, and LIgO can generate synthetic data resembling experimental data by learning immunological processes from existing datasets, including V(D)J recombination distributions and clonal expansion (102–104). Additionally, researchers have developed approaches to predict missing information from TCR sequences to address experimental data shortages and expand training datasets. Ortega et al. proposed a technique for predicting HLA types from the TCR repertoire and iteratively refined the model using datasets with unknown HLA types (105). The advantages of this simulated data extend beyond addressing the constraints of limited data availability: these approaches enable evaluation of model performance based on well-defined biological principles and provide access to data unaffected by experimental noise.

These diverse approaches offer promising avenues for overcoming current TCR dataset limitations. As they mature, these methods will likely enable more robust PLMs that better capture the complex biological properties of TCR repertoires with greater fidelity and generalizability.

Conclusion

In this paper, we have reviewed PLM-based representations of TCR data, focusing on learning methods, their advantages, and their applications to antigen specificity. Conventional approaches fail to fully capture the extensive diversity of TCRs and the complexity of their sequence–function relationships. The emergence of PLMs has enabled researchers to extract context-dependent features from TCR sequences and predict antigen specificity with improved generalization performance. Notably, the transfer learning approach, which combines pre-training on large unlabeled TCR sequences with fine-tuning on limited labeled data, has demonstrated itself as an effective strategy to overcome the constraints of data scarcity.

Despite these advances, challenges remain in the application of deep learning to TCR repertoire analysis. In particular, the lack of labeled data and limited availability of complete receptor information pose major barriers to enhancing model performance and generalization. In addition to addressing these data-related challenges, significant advances should be achieved in both interpreting the representations learned by PLMs and implementing multimodal approaches that integrate structural information. Breakthroughs in these domains will be essential to practice TCR analysis for a wide range of clinical applications, including disease diagnosis, vaccine development, and personalized immune profiling.

Conflict of interest statement

The authors declare no conflicts of interest.

Funding

This research is supported by JST CREST JPMJCR2011, and JSPS KAKENHI Grant Number 25H01365.