Meta-learning reduces the amount of data needed to build AI models in oncology

Summary

Meta-learning is showing promise in recent genomic studies in oncology. Meta-learning can facilitate transfer learning and reduce the amount of data that is needed in a target domain by transferring knowledge from abundant genomic data in different source domains enabling the use of AI in data scarce scenarios.

Main

Transfer learning has emerged as a powerful technique in machine learning to train neural network models. In principle transfer learning proceeds in two steps. First, a pretraining stage where abundant labelled data from another domain or task is used to train the model, followed by a fine-tuning stage where the model is further trained in the target domain on the task of interest. Transfer learning enables few-shot learning in the target domain, in which a classifier must adapt to distinguish novel classes not seen during training given only a few examples (called shots) of these classes.¹ This has been particularly effective in biomedical data with primarily successful examples in medical imaging.² More recently, several advanced techniques have been proposed including multi-task learning and meta-learning. In multi-task learning, the model is pretrained on not one, but multiple tasks simultaneously—this encourages the model to learn richer internal representations of the data and improves generalisation in the target domain or task. A related approach called meta-learning can be thought of as “learning how to learn”. In “model agnostic meta learning” introduced in ref. ³, for example, the model is first trained on a distribution of tasks with the objective of encouraging model parameters that can be quickly adapted, using gradient descent, to any task drawn from the task distribution. Thus, the model can quickly learn a new task from a small amount of data in the target domain (Fig. 1).⁴

Fig. 1. Meta-learning overview.

A typical meta-learning pipeline consists of two stages: (1) top panel: pre-train on source data which is abundant available and potentially training on different tasks, (2) Bottom panel: fine-tuning of the model on scarce target data.

The use of meta-learning holds great promise to reduce the amount of biomedical data needed to train predictive models in the target domain of interest. In biomedicine this situation—having few data in the target domain—occurs frequently, as new technologies continue to emerge, such as single-cell DNA and RNA sequencing and spatial transcriptomics, while similar abundant source data is often available from different contexts, tissues, cells or generated from older related technologies. Meta-learning provides a solution for this situation by taking advantage of these source data sets from other organisms, institutes or cell types, and first training a model that is poised to learn a new task before touching the data in the target domain.

Meta-learning and few or zero-shot learning has been widely adopted in computer science applications. In particular, several successful applications have been reported using non-medical image data including multi-class classification of pictures given only a small number of examples for new classes,⁵ detection of objects with few examples,⁶ prediction of human motion⁷ and image segmentation.⁸ Despite such success, there have not been many successful use cases in biomedicine so far.

Within genomics, a few examples exist that take advantage of meta-learning. Here, I mention three key early adopters of meta-learning in genomics with diverse uses cases in the area of oncology. First, Brbic et al. showed a successful example of the use of meta-learning to predict existing and novel cell types based on single-cell RNA sequencing data.⁹ The proposed method, MARS, uses deep learning to learn an embedding of labelled cell types, and then uses this in the meta-learning setting to annotate new cell types also across tissues. Next, Ma et al. use meta-learning to model drug response for a singular patient.¹⁰ In this study, the investigators take advantage of the huge amounts of genetic perturbation data that has been generated in cell lines and translate this to benefit individual patients in different contexts with likely much less data available. Ma et al. show that meta-learning works in different contexts, transferring between tissues and different culture systems ranging from cell lines to patient-derived tumour cell cultures in vitro and patient-derived tumour xenografts in vivo, contexts where it is very costly to collect large volumes of data. In a final example, we have used meta-learning on gene expression profiles from RNA sequencing data to predict clinical outcome of cancer patients.⁴ We show that meta-learning outperforms regular transfer learning and direct learning when predicting survival outcome in a target cancer using data from 33 other cancer sites as source data, reducing the amount of data that is needed in the target domain.

The common theme in each of these applications is that meta-learning is successful in drastically reducing the amount of data that is needed in the target domain. Qiu et al. show that one order of magnitude less gene expression profiles is needed to train an optimal model predictive of clinical outcome using meta-learning.⁴ These success stories show highly promising applications particularly in the context of rare diseases. For example, paediatric oncology has lacked behind in terms of new discoveries compared with adult oncology, in most cases due to fact that cancers are far more common in adults than in children. Therefore, large scale efforts such as The Cancer Genome Atlas (TCGA) and The Clinical Proteomic Tumor Analysis Consortium (CPTAC) have focused on adult cancers and paediatric efforts such as the Child Brain Tumor Tissue Consortium (CBTTC) have more difficulty collecting large sets of samples. Meta-learning provides opportunities to take advantage of abundant adult data and transfer knowledge to solve questions in paediatric oncology.

One important area of further research is to determine when meta-learning will be successful. What metrics exist that can tell whether the source domain data is sufficient for the target of interest? Therefore, still much work has to be done beyond these studies to test meta-learning strategies across broader applications areas in genomics in oncology and also beyond.

Author contributions

O.G. performed literature review and wrote the manuscript.

Ethics approval and consent to participate

Not applicable.

Consent to publish

Not applicable.

Data availability

Not applicable.

Competing interests

The author declares no competing interests.

Funding information

O.G. is supported by the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health under Award Number R56 EB020527. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.