The Role of Federated Learning Models in Medical Imaging

See also the article by Luo et al in this issue.

Lily Kwak, BA, is a medical student at the Johns Hopkins University School of Medicine. Her research interests include breast imaging, imaging-related health disparities, and artificial intelligence.

Harrison Bai, MD, MS, is an assistant professor of radiology at Johns Hopkins University in Baltimore, Maryland. His research interests focus on artificial intelligence, machine learning, and computer vision as applied to medical image analysis.

Deep learning continues to have an increasingly important role in radiology. Today, deep learning aids the detection, classification, segmentation, and monitoring of disease in medical imaging. Not only has it automated time-consuming tasks for radiologists, but it has been shown to mitigate interobserver variability in interpretation. Although deep learning models undoubtedly have demonstrated great potential, they require large, diverse datasets in their development. Accessing large datasets from multiple institutions is often challenging because of laws such as the Health Insurance Portability and Accountability Act of 1996, as well as other concerns with patient privacy and data ownership. Maintaining patient privacy while incorporating diverse datasets into a model poses one of the most substantial limitations in deep learning research.

Federated learning offers a solution to the challenge of building multi-institutional models. Unlike a centralized model, in which datasets from separate institutions are combined in a central database where a deep learning model is trained and developed, federated learning trains individual models locally using an institution’s own data (1). The resulting updated model is then returned to the central server, keeping the original data local. By exchanging locally trained models instead of large datasets, patient data remain protected throughout the deep learning process.

Although investigators have studied algorithm performance differences between federated and centralized models, the effect of distribution differences between sites in federated models is unknown and is the focus of Luo et al’s work (2) in the current issue of Radiology: Artificial Intelligence. Luo et al report that a greater difference in data distributions was strongly associated with the decreased performance of federated learning models for the segmentation of liver tumors and brain tumors. Overall, between-site differences in tumor attenuation on liver CT images and tumor intensity on brain MR images significantly affected tumor segmentation. This interesting finding may guide future federated learning studies in imaging.

Luo et al also established a dataset, the Federated Imaging of Liver Tumor Segmentation (FILTS), to train and validate their federated learning model for liver CT using the publicly available Liver Tumor Segmentation (ie, LiTS) multisite dataset plus cases from two additional independent institutions. Although FILTS has limitations, as acknowledged by the authors, it would be beneficial for the data to be shared publicly. Doing so may encourage further understanding, growth, and collaboration in the community of deep learning research. The data used for brain MRI tumor segmentation are publicly available and include the Federated Tumor Segmentation (ie, FeTS) and University of California San Francisco Preoperative Diffuse Glioma MRI (ie, UCSF-PDGM) datasets.

To our knowledge, Luo et al’s study is the first to evaluate the association between differences in data distribution and performance in tumor segmentation in a federated model. Based on their findings, the authors suggest that the training of federated models should involve datasets with small distances. Domain adaptation, a process that can be used to account for differences in data collection or generation between domains in federated learning for functional MRI analysis, has improved classification accuracy in some instances (3). Further studies are needed to better understand the factors that degrade federated model performance and to develop methods to mitigate them.

Several studies have demonstrated the utility of federated deep learning models in radiology. Sheller et al (4) demonstrated comparable performance between federated and centralized models using the 2018 Brain Tumor Segmentation (BraTS) dataset. The BraTS dataset consists of multimodal MR images in patients diagnosed with gliomas at multiple institutions. In this study, a federated model with 10 institutions had 98.7% of the performance ability of a centralized model. Li et al (5) subsequently recognized the need for privacy-preserving methods and implemented differential privacy techniques to reduce the possible risk of reverse engineering of models to reveal patient data.

Recently, a large federated deep learning model involving 71 sites across six continents was used to develop a tumor boundary detector for glioblastoma (6). An increased number of datasets led to better performance when compared with a public model generated using data from 16 sites from the 2020 BraTS challenge dataset. However, the amount of data was not directly related to the model’s performance. Furthermore, the global model was robust to data quality issues such as erroneous annotation on images. As the largest federated deep learning study to date, this model carries great clinical potential and may serve as an example for the development of future federated learning models.

The ultimate goal of high-performance federated learning models is to achieve meaningful clinical impact. Federated learning has been used to predict outcomes using radiologic studies in conjunction with objective clinical data. For instance, the Electronic Medical Record Chest X-ray AI (ie, EXAM) model used vital signs, laboratory data, and chest radiographs to predict whether oxygen would be required in symptomatic patients with COVID-19 (7). This federated learning model, consisting of 20 multinational sites, showed a 16% increase in performance and a 38% improvement in generalizability when compared with locally trained models. A federated deep learning model for COVID-19 chest CT abnormality detection and lesion burden estimation showed good generalizability (8). It is evident that federated deep learning shows great potential for use in patient care.

In addition to the impact that federated deep learning may have in maintaining patient privacy and contributing to clinical practice, it potentially could play a role in addressing health disparities by incorporating more diverse data from underserved populations and in data-limited scenarios (9). Federated learning may help to advance our understanding of rare diseases with its inclusive, yet protected, models. All in all, federated deep learning has demonstrated great potential in radiology and is expected to continue to be a major focus of AI research in the coming years.