Artificial intelligence for antibody reading comprehension: AntiBERTa

Abstract

Utilizing publicly available antibody big data resources, Leem et al. (2022) developed an antibody-specific language model, AntiBERTa, to understand the “language” of antibodies. Case studies reveal that AntiBERTa can be an extremely useful tool for antibody engineering.

Utilizing publicly available antibody big data resources, Leem et al. (2022) developed an antibody-specific language model, AntiBERTa, to understand the “language” of antibodies. Case studies reveal that AntiBERTa can be an extremely useful tool for antibody engineering.

Main text

There are a large number of players in the immune system to protect biological individuals from harmful foreign substances. Among those, the B cell is a main player in the adaptive immune system. B cells are equipped with receptor molecules (B cell receptor) that can be secreted upon activation. The secreted molecules, antibody, are known to be astronomically diverse (estimated 10¹³–10¹⁵).

The high diversity of the antibody is a two-faced Janus. The immune system can respond to nearly any type of antigen, mainly due to the large diversity of antibodies. According to Antibodypedia,¹ 4.6 million monoclonal antibodies are currently available for 19,000 genes. The diversity also enables antibodies to be highly successful as biotherapeutics. In 2021, FDA approved the 100th therapeutic antibody.² The coronavirus pandemic has been currently boosting the development of therapeutic antibodies for COVID-19, and several new antibodies are waiting to treat SARS-CoV-2-infected patients.

On the other hand, such rich diversity may not be always advantageous. Despite the fact that antibodies have been (perhaps the most) extensively studied and the antibody-related biopharmaceutical industry continues to mature, there seem to be a lot of things to learn about antibodies, as evidenced in the increasing growth of papers with the publication keyword, “antibody.” It is simply practically impossible to experimentally explore the entire antibody repertoire. Thus, computational approaches using artificial intelligence (AI) techniques have become essential for antibody research.

The advancement of AI and big data are not separable. Recent advances in next-generation sequencing technology now enable the construction of a large volume of antibody repertoires. The observed antibody space (OAS) database^3,4 is a compilation of known repertoire studies and databases. Since the release of OAS, many practical applications have been developed including computational antibody humanization using AI.^5,6

The antibody repertoire big data resources also provide an in-depth view and biological insights into antibodies.⁷ Here, Leem et al. present an antibody-specific language model in a timely manner. AntiBERTa (antibody-specific bidirectional encoder representation from transformers) is a 12-layer transformer model pre-trained using the OAS database.⁸ In fact, there has been a language model for general proteins⁹ (ProtBERT), and there have been other antibody-specific language models, such as DeepAb¹⁰ and Sapiens.⁶ Comparing with those existing methods, however, AntiBERTa is more versatile and specific with deeper layers.

It is remarkable that AntiBERTa nicely partitions memory and naive B cells, whereas other models showed relatively less distinct results; i.e. the antibody-specific deep-layered model indeed learns the language of antibodies and finds the origin of B cell. One of the direct applications can be the estimation of antibody humanness and immunogenicity for the development of safer therapeutic antibodies. It is well known that antibodies with high human content tend to be less immunogenic. As demonstrated in the separation of memory and naive B cells, AntiBERTa is shown to be successful in classifying their species origin (murine, chimeric, humanized, and human).

The antibody-specific model generally provides better descriptions of antibodies than the general protein model. The authors found that residue pairs with high self-attention scores give structural insights into long-range interactions, which were not identified by the general protein model. The insight naturally leads to the prediction of paratopes, antigen binding sites. From several case studies, the authors showed that AntiBERTa successfully identifies paratope residues that are not in complementarity determining regions (CDR).

While the authors demonstrated that AntiBERTa outperforms other methods and provided convincing rationales, they also leave something to be desired. As the authors stated in the main manuscript, AntiBERTa can be directly applicable to antibody-structure prediction and humanization (or both at the same time), but the authors left it as potential applications. In the near future, we hope to meet practical application tools based on the AntiBERTa model.