A fully open AI foundation model applied to chest radiography

Abstract

Chest radiography frequently serves as baseline imaging for most lung diseases¹. Deep learning has great potential for automating the interpretation of chest radiography². However, existing chest radiographic deep learning models are limited in diagnostic scope, generalizability, adaptability, robustness and extensibility. To overcome these limitations, we have developed Ark⁺, a foundation model applied to chest radiography and pretrained by cyclically accruing and reusing the knowledge from heterogeneous expert labels in numerous datasets. Ark⁺ excels in diagnosing thoracic diseases. It expands the diagnostic scope and addresses potential misdiagnosis. It can adapt to evolving diagnostic needs and respond to novel diseases. It can learn rare conditions from a few samples and transfer to new diagnostic settings without training. It tolerates data biases and long-tailed distributions, and it supports federated learning to preserve privacy. All codes and pretrained models have been released, so that Ark⁺ is open for fine-tuning, local adaptation and improvement. It is extensible to several modalities. Thus, it is a foundation model for medical imaging. The exceptional capabilities of Ark⁺ stem from our insight: aggregating various datasets diversifies the patient populations and accrues knowledge from many experts to yield unprecedented performance while reducing annotation costs³. The development of Ark⁺ reveals that open models trained by accruing and reusing knowledge from heterogeneous expert annotations with a multitude of public (big or small) datasets can surpass the performance of proprietary models trained on large data. We hope that our findings will inspire more researchers to share code and datasets or federate privacy-preserving data to create open foundation models with diverse, global expertise and patient populations, thus accelerating open science and democratizing AI for medicine.

Chest radiography (chest X-ray, CXR) is the most frequently performed radiological examination¹. Rapid and accurate CXR interpretation is essential for patient care². Artificial intelligence (AI) and deep learning have the transformational power to automate CXR interpretation and advance the future of medical diagnostics. However, early deep learning models were limited in accuracy mainly due to data paucity and biased data sources⁴, which restricted their diagnostic scope, generalizability, adaptability, robustness and extensibility.

Foundation models, which are trained on large datasets, have emerged as a transformative approach in deep learning. They offer unparalleled capabilities in various applications, including diagnostic imaging^5–7. Models powered by extensive and diverse training data have matched or surpassed the accuracy of experts in diagnosing abnormalities and have also demonstrated generalizability to clinical situations beyond the scope of their initial training^8,9. Such models can democratize access to expert-level diagnostic capabilities, particularly in underserved or remote areas where radiological expertise is scarce.

However, proprietary high-performance foundation models^10–12 are not fully open¹³ (Extended Data Table 1). This restriction makes it difficult for researchers and developers to build on existing works, which hinders the pace of innovation. We envision a fully open, powerful and robust foundation model that can be trained by aggregating numerous (large or small) public datasets with the option of using federated private data, so that the model is fully accessible by the public. Open foundation models can be continually improved and adapted. With public access and openness to contribution, researchers can iteratively refine and enhance these models, thus ensuring that the models remain up to date with the latest medical knowledge and advances. This continuous development cycle can lead to more accurate and reliable diagnostic tools, thus advancing AI-driven healthcare and ultimately improving patient outcomes.

This article presents a fully open foundation model applied to chest radiography, Ark⁺, which was pretrained by cyclically accruing and reusing the knowledge embedded in the heterogeneous expert labels from six public datasets (Fig. 1a). Ark⁺ builds on previous work, Ark¹⁴, offering several enhancements: a larger backbone model using Swin transformer large (Swin-Large)¹⁵, an increased image input resolution of 768 × 768 and a rearranged data-augmentation approach that feeds the teacher model the resized original image rather than using random cropping. Ark⁺ has three pretrained components: an encoder, a projector and multi-task heads, all of which are openly available for use in various clinical (target) tasks through fine-tuning, linear-probing or zero-shot transfer (Fig. 1b). Fine-tuning involves retraining the entire network, including the backbone encoder and the linear classifier. This process harnesses the full discriminative power of the model to enable it to comprehensively adapt to the target task. Linear-probing, on the other hand, focuses on training only a linear classifier atop the frozen features extracted by the pretrained backbone. This approach efficiently applies the knowledge learned by the model to the target task, effectively testing the quality of the model features. Zero-shot transfer directly uses the model to diagnose conditions in datasets that it has not seen during its pretraining. This method tests the capacity of the model to generalize its learned knowledge to new datasets without the need for further training or exposure to specific samples from these datasets.

Fig. 1 |. Ark⁺ is an open foundation model applied to chest radiography.

a, Ark⁺ has been pretrained in a cyclical manner by accruing and reusing the knowledge embedded in the heterogeneous expert labels from six public datasets: ChestX-ray14 (ref. 19), RSNA Pneumonia⁴³, CheXpert³⁴, VinDr-CXR⁴², Shenzhen CXR⁴⁴ and MIMIC-II⁴⁵ (detailed in Extended Data Fig. 1 and Extended Data Table 7). b, Ark⁺ has three pretrained components: an encoder, a projector and multi-task heads, which can be adopted to accomplish various clinical (target) tasks through fine-tuning, linear-probing or zero-shot transfer. To fine-tune Ark⁺ for a target task, we attach a new randomly initialized linear classifier to the pretrained encoder and train the entire model using the images and their associated labels provided by the target task. For linear-probing, we first use the pretrained encoder and projector to generate the embeddings (information-rich numerical vectors) for all images in the target task and then train a new linear classifier. For zero-shot transfer, we deploy the entire pretrained Ark⁺, including the encoder, projector and multi-task heads, to directly acquire a prediction for each image in the target task, as it requires no further training. This article describes internal evaluations with four datasets and external evaluations with six unseen datasets (Extended Data Table 2), which assessed the capability of Ark⁺ relative to nine large pretrained models (Extended Data Table 1) in diagnosing thoracic diseases across seven different scenarios. Its performance is presented in Figs. 2 to 5 and Extended Data Tables 4 and 5. CXR images in a adapted with permission from ref. 19, IEEE; ref. 34, AAAI; refs. 42,45, Springer Nature Limited; ref. 44, AME Publishing Company.

To assess the performance of Ark⁺ in diagnosing thoracic diseases, we conducted internal evaluations with four datasets and external evaluations with six datasets (Extended Data Table 2). The internal evaluations assessed the model using data like the data in its training domain, specifically using hold-out test data from a ‘seen’ dataset involved in pretraining the model. These evaluations provided insights into the effectiveness of the model within familiar contexts. The external evaluations assessed the model using ‘unseen’ datasets from different sites or hospitals that serve different populations and use different imaging protocols. This type of evaluation is crucial for understanding how well the model can perform in real-world scenarios that may differ substantially from its training environment. Using these ten different datasets, we extensively evaluated Ark⁺ across eight different scenarios: (1) diagnosing common thoracic diseases, (2) adapting to evolving diagnostic needs, (3) learning to diagnose rare conditions from a few samples, (4) handling long-tailed thoracic diseases, (5) adjusting to shifts in the diagnostic setting without training, (6) tolerating sex-related bias, (7) responding to novel thoracic diseases and (8) using private data while preserving patient privacy with federated pretraining.

The experimental results demonstrate the superior generalizability, adaptability, robustness and extensibility of Ark⁺ compared with other foundation models (Extended Data Table 1) across various clinical scenarios. Generalizability (quantitatively assessed in Fig. 2) refers to the ability of a model to perform effectively on new data such that the knowledge learned from the training data can be applied across different but related data or clinical scenarios. Adaptability extends generalizability by emphasizing the capacity of a model to adjust to and excel in diverse conditions, such as new datasets, tasks or clinical settings, as demonstrated in Figs. 3 and 5. Robustness, highlighted in Fig. 4 and Extended Data Table 4, reflects the ability of a model to maintain high performance despite challenges such as long-tailed data distributions or biased training data. Extensibility is the ability to incorporate novel tasks and scenarios, as evidenced in the integration of a (new) COVID-19 diagnostic task in Ark⁺ through incremental learning (Extended Data Table 5). Ark⁺ can be further extended using federated learning to incorporate private data while preserving patient privacy and to distribute pretraining (Extended Data Fig. 2 and Extended Data Table 6). In addition, it can be augmented beyond classification¹⁴ to support localization¹⁶, segmentation¹⁷ and their integration¹⁸. To comprehensively evaluate performance, we adopted several metrics: area under the receiver operating characteristic (ROC) curve (AUC), accuracy, F₁ score, average precision and Matthews correlation coefficient (MCC). Statistical significance between models was assessed using P values from the two-sided independent t-test, and 95% confidence intervals were calculated for zero-shot transfer results based on 1,000 resampled test sets.

Fig. 2 |. Performance on diagnosing common thoracic diseases.

a, Comparison of six large pretrained models in diagnosing 14 thoracic diseases. These were evaluated on the hold-out test data of ChestX-ray14 (ref. 19) by fine-tuning (solid lines) and linear-probing (dashed lines). For each model, the average AUC scores (%) across five runs are plotted on a parallel coordinate to illustrate the relative model performance. Mean AUC scores for all 14 diseases are shown on the second y axis to highlight overall performance. Ark⁺ was the top performer in both set-ups, underscoring its strength as an open foundation model. Ark⁺ and CXR-FM, pretrained in a supervised manner, significantly outperformed the self-supervised models (RAD-DINO, MIM-CXR and CheSS), emphasizing the value of expert knowledge. Fine-tuning consistently surpassed linear-probing, highlighting the importance of openness associated with foundation models for fine-tuning. CXR-FM and ELIXR are closed. They can be linearly probed only through APIs and cannot be fine-tuned, as their pretrained models are not publicly available. b,c, Illustration of the capability of Ark⁺ for expanding the diagnostic capability and correcting possible misdiagnosis by experts. Cases where Ark⁺ disagreed with the ChestX-ray14 expert labels were reviewed by a senior Mayo Clinic radiologist with 30 years of experience. The officially released labels from ChestX-ray14, diagnostic results from Ark⁺ and the doctor’s note are presented on the right. b, The official label indicates no findings for the case; however, Ark⁺ disagreed, diagnosing atelectasis. Additionally, by leveraging training from CheXpert, Ark⁺ expanded the diagnostic scope and successfully identified support devices. This prediction by Ark⁺ was confirmed as correct by the doctor. c, Although the official label indicates ‘edema’, Ark⁺ disagrees, showing no evidence of oedema and instead confirming no finding based on the CheXpert and VinDr-CXR heads. This assessment aligns with the doctor’s opinion, demonstrating that Ark⁺ can tolerate overdiagnosis by the ChestX-ray14 experts. CXR images in b and c adapted with permission from ref. 19, IEEE.

Fig. 3 |. Evaluation of adaptability to evolving diagnostic tasks and the few-shot learning capability for detecting rare conditions.

a–c, Assessing the adaptability of Ark⁺ to evolving diagnostic tasks through linear-probing on the six global labels and 22 local labels from VinDr-CXR⁴². The six global labels were included with VinDr-CXR when pretraining Ark⁺, whereas the 22 local labels were excluded. Ten independent experiments with Ark⁺ and CXR-FM were conducted, and performance was evaluated on hold-out test data. Bar centres represent mean AUC scores, and error bars indicate standard deviations (%). a, Ark⁺ demonstrated statistically significant superiority over CXR-FM across global, local and all labels, as shown by P values from the two-sided independent t-test. b,c, Performance for individual global (b) and local labels (c) indicates Ark⁺ significantly outperformed CXR-FM on all six global labels and the 13 local labels. Note that the local label ‘edema’ is not shown in the figure as there are no samples with this label in the official test data. The results indicate that Ark⁺ can adapt to complex diagnostic tasks with evolving clinical needs, even those for which it was not pretrained. d, Evaluating the adaptability and robustness of Ark⁺ when detecting rare conditions (subcutaneous emphysema, tortuous aorta and pneumoperitoneum) using two-way k-shot learning (no finding versus each rare condition). The results demonstrate the potential of Ark⁺ to adapt by diagnosing rare conditions using only a few labelled samples. Box plots illustrate AUC score distributions across 100 experimental replicas, with medians (central lines), interquartile ranges (box bounds for the 25th and 75th percentiles), whiskers (1.5 times the interquartile range) and outliers as individual points. P values from two-sided independent t-tests are reported in the figure. COPD, chronic obstructive pulmonary disease; ILD, interstitial lung disease; PA, pulmonary artery.

Fig. 5 |. Performance on recognizing common thoracic diseases in new settings without training.

The generalizability and robustness of Ark⁺ were evaluated through zero-shot transfer on four unseen datasets (SIIM-ACR⁴⁷, NODE21 (ref. 48), Mendeley-V2 (ref. 49) and TBX-11K⁵⁰) and four seen diseases (pneumothorax, nodule, pneumonia and tuberculosis). Although the four datasets were not included in its pretraining, Ark⁺ had seen the four diseases in other datasets during its pretraining: pneumothorax in CheXpert, nodule in ChestX-ray14, and pneumonia and tuberculosis in VinDr-CXR. Therefore, we directly acquired the predictions made by Ark⁺ on the four diseases from its pretrained heads without further training, a scenario that we refer to as zero-shot transfer. For comparison, two supervised models (XRV-DenseNet and XRV-ResNet) from TorchXRayVision³⁰ trained with several datasets and two image-text pretrained models (KAD⁹ and ELIXR¹²) were also evaluated under the same conditions. The 95% confidence intervals of the ROC curves are displayed as coloured bands, with the centre of each band representing the mean AUC. Note that although the original CXR-FM cannot perform zero-shot transfer, ELIXR, a more advanced version built on CXR-FM, has this capability. The TorchXRayVision models (XRV-DenseNet and XRV-ResNet) were incapable of predicting tuberculosis because this label was not present during their pretraining. CI, confidence interval.

Fig. 4 |. Performance for long-tailed thoracic diseases.

a–c, We evaluated six pretrained models (CheSS, MIM-CXR, RAD-DINO, ELIXR, CXR-FM and Ark⁺) in diagnosing 19 thoracic diseases with long-tailed distributions using the unseen dataset ChestDR²⁸. Ten independent experiments were conducted per model and set-up. Bar centres represent mean AUC scores and error bars show standard deviations. Performance on ten head classes (a) and nine tail classes (b) using linear-probing. Ark⁺ demonstrated superior performance across 16 diseases, with performance only for aortic calcification and increased lung markings trailing behind CXR-FM and performance for elevated diaphragm lagging behind RAD-DINO. c, Performance of Ark⁺ and CXR-FM across 19 diseases for various fractions of training data. The other four models were excluded for clarity due to their significantly inferior performance (Extended Data Table 3). Ark⁺ exhibited superior label efficiency, outperforming CXR-FM, especially with reduced data. Under limited data (5%, 48 samples; 10%, 97 samples; 25%, 244 samples), Ark⁺ performed better with linear-probing, but as the amount of data was increased, fine-tuning surpassed linear-probing. This is corroborated by Tajbakhsh et al.⁴⁶ and occurred because fine-tuning all the layers in a model requires more data than fine-tuning the last layer (equivalent to linear-probing), indicating the value of openness associated with foundation models, particularly when an adequate amount of data is available for local adaptation through fine-tuning.

Diagnose common thoracic diseases

We conducted an internal evaluation of Ark⁺ using the hold-out test data from ChestX-ray14 (ref. 19), a well-established public dataset commonly used for benchmarking model performance in diagnosing 14 common thoracic diseases (Fig. 2), under both fine-tuning and linear-probing set-ups, relative to the large pretrained models: CXR-FM¹⁰, ELIXR¹², RAD-DINO²⁰, MIM-CXR²¹ and CheSS²². As illustrated in Fig. 2a, Ark⁺ achieved the highest average performance in both set-ups, thus demonstrating its generalizability. The fine-tuning results, depicted as solid lines in Fig. 2a, show that Ark⁺ achieved a mean AUC of 84.43 ± 0.09% over the 14 diseases, surpassing RAD-DINO, MIM-CXR and CheSS by significant margins of 0.89%, 1.35% and 3.97%, respectively. In the linear-probing results, illustrated by dashed lines, Ark⁺ was consistently ranked at the top and CXR-FM ranked second. Both models significantly outperformed the self-supervised models (RAD-DINO, MIM-CXR and CheSS) by a considerable margin, showing the power of learning from expert labels. Moreover, fine-tuning consistently surpassed linear-probing for all models, underscoring the importance of the openness of a foundation model for effective local adaptation through fine-tuning.

Additionally, we showed that Ark⁺ can expand the diagnostic scope and correct possible misdiagnosis by experts through the use of its multi-task heads (Extended Data Fig. 1) and by pretraining on diverse tasks (Fig. 1). The multi-task heads generate diagnostic results from both their specific task and other heads pretrained on different tasks, enabling a broader diagnostic scope and more comprehensive, accurate diagnoses while effectively managing misdiagnosis. To illustrate this capability, two cases are presented in Fig. 2. The officially released labels from ChestX-ray14, diagnostic results from Ark⁺ and the doctor’s note are shown on the right-hand side. The 14 official disease labels for the images were derived from the associated radiological reports written by human experts using natural language processing¹⁹. Diagnostic results for Ark⁺ were obtained from the CheXpert head, the ChestX-ray14 head and the VinDr-CXR head, with thresholds selected using validation data. Cases where Ark⁺ disagreed with the official labels were reviewed by a senior Mayo Clinic cardiopulmonary radiologist with 30 years of experience. In Fig. 2b, the official label indicates no findings for the case, whereas Ark⁺ predicted atelectasis and support devices. The doctor’s note confirms the diagnosis made by Ark⁺ as correct, demonstrating that not only had Ark⁺ detected a previously undiagnosed condition but that it also expanded diagnostic capabilities to identify support devices. In Fig. 2c, the official label indicates ‘edema’. Ark⁺ found no evidence of oedema but instead confirmed no finding based on the CheXpert and VinDr-CXR heads. This diagnosis aligns with the doctor’s opinion, demonstrating the ability of Ark⁺ to correct possible overdiagnosis by ChestX-ray14 experts. Moreover, the prediction by Ark⁺ of support devices further underscores its expanded diagnostic capabilities.

Adapt to evolving diagnostic needs

Annotating medical images is tedious, labour-intensive and time-consuming, and it requires costly specialty-oriented skills that are not easily accessible³. Consequently, medical datasets are often partially labelled so that they incompletely address clinical needs, resulting in incompletely trained models with comparatively restricted adaptability²³. Furthermore, evolving medical practices and clinical applications necessitate updates to datasets (for example, recognizing new conditions) and upgrades or complete retraining of AI models to meet these changing needs. To demonstrate the adaptability of Ark⁺ to evolving diagnostic needs, when pretraining Ark⁺, we consciously included only six image-level global labels from VinDr-CXR and excluded its 22 lesion-level local labels, so that Ark⁺ was not trained on the 22 local diseases. Specifically, during pretraining, even though Ark⁺ ‘saw’ all chest radiographs in the VinDr-CXR training set, including those with any of the 22 local diseases, it did not use these specific disease labels to guide its learning process. As a result, chest radiographs with none of the five global diseases were treated, intentionally albeit incorrectly, as ‘purely normal’ (no finding) when Ark⁺ was pretrained, allowing us to assess the adaptability of Ark⁺ to those 22 local diseases after the pretraining.

We evaluated both Ark⁺ and CXR-FM on their adaptabilities through linear-probing, reporting the average AUC scores and the standard deviations (%) across ten runs for each disease on the VinDr-CXR hold-out test set. As shown in Fig. 3a, Ark⁺ achieved mean AUC scores of 96.19 ± 0.09%, 93.45 ± 0.10% and 94.06 ± 0.09% for the six global labels, the 21 local labels and all 27 labels, respectively. It significantly outperformed CXR-FM, which achieved 95.29 ± 0.07%, 93.06 ± 0.16% and 93.55 ± 0.12%, respectively. Note that although VinDr-CXR is external to CXR-FM, it performed strongly on this task. For global labels (Fig. 3b), Ark⁺ achieved an AUC above 90% for all findings, surpassing CXR-FM in every case. Among the 21 local labels (Fig. 3c), Ark⁺ achieved an AUC above 90% on 17 findings and outperformed CXR-FM on 14 findings. In particular, Ark⁺ excelled on the common thoracic diseases encountered during pretraining, such as atelectasis, cardiomegaly, consolidation, emphysema, infiltration, nodule/mass, pneumothorax and fibrosis. These results highlight the ability of Ark⁺ to adapt to evolving clinical needs without requiring extensive retraining, making it a valuable tool in real-world applications where adaptability and accuracy are essential. Note that although the official training set for VinDr-CXR included samples for all 22 local labels, its test set contained no samples labelled ‘edema’; therefore, no performance is reported in Fig. 3c for oedema.

Learn rare conditions from a few samples

The timely and accurate diagnosis of rare diseases is challenging yet crucial for improving patient outcomes and optimizing treatment strategies²⁴. Because of their low prevalence, the number of available training samples for rare diseases is very limited, which restricts the diagnostic accuracy of the trained model. Therefore, a critical measure of the effectiveness and practical utility of a foundation model in clinical settings is its ability to adapt by detecting rare diseases using only a few labelled samples through few-shot learning²⁵. For this purpose, we conducted two-way k-shot learning (no finding versus a condition present) to assess the few-shot learning ability of Ark⁺, and we compared its performance with CXR-FM. Both models were evaluated under the linear-probing configuration. For this study, we selected three rare conditions: subcutaneous emphysema, tortuous aorta and pneumoperitoneum, and we established a benchmark built on MIMIC-II using the labels from the CXR-LT challenge²⁶. To ensure that Ark⁺ had no previous exposure to these samples during pretraining, we removed the relevant images from the training set, yielding 78 samples for subcutaneous emphysema, 59 samples for tortuous aorta and ten samples for pneumoperitoneum. Each of these three sets was combined with 420 samples labelled no finding to create three rare-condition datasets. For each trial, we randomly selected k samples with the condition and k samples labelled no finding and used these to train a linear classifier. We evaluated its performance with the remaining samples.

The box plots in Fig. 3d depict the distribution of AUC scores for Ark⁺ and CXR-FM across 100 experimental replicas. Ark⁺ consistently outperformed CXR-FM, achieving higher median and maximum AUC scores with a narrower interquartile range in all experiments, underscoring its superior performance. However, both models exhibited outliers with AUC scores below 50%, reflecting the inherent difficulty of detecting rare conditions after training on a very limited number of samples. These findings demonstrate that Ark⁺ can effectively diagnose rare disorders using only a few labelled samples, which highlight its potential for practical clinical applications in scenarios with limited data availability.

Handle long-tailed thoracic diseases

Chest radiographic diagnosis is often a long-tailed problem²⁷. The distribution of clinical findings is skewed towards a few common abnormalities because rare conditions are less frequently encountered, posing challenges when training models owing to the imbalance in class frequencies. Models often prioritize learning from the majority classes, leading to biases towards common findings and, therefore, potentially neglecting rare conditions. Moreover, the scarcity of data for rare conditions can result in overfitting to the limited samples, reducing the ability of a model to generalize effectively to unseen cases. Therefore, robustness and generalizability to long-tailed scenarios are key indicators when evaluating foundation models.

To assess the performance of Ark⁺ in long-tailed scenarios, we used the unseen dataset ChestDR²⁸, which was developed for examining large foundation models in diagnosing 19 thoracic diseases with long-tailed distributions. Ark⁺ was fine-tuned using all training data and 19 disease labels, and its performance was compared with the fine-tuning results of RAD-DINO, MIM-CXR and CheSS, as well as the linear-probing results of CXR-FM and ELIXR. The results, plotted in Fig. 4a,b, show that Ark⁺ outperformed the other models across 16 of the 19 diseases. Consistent with the findings in Fig. 2, supervised models Ark⁺ and CXR-FM demonstrated a significant advantage over self-supervised models RAD-DINO, MIM-CXR and CheSS as well as the image-text model ELIXR built on CXR-FM. For instance, Ark⁺ achieved through fine-tuning a mean AUC of 86.55 ± 0.35% across the 19 diseases, significantly surpassing the 82.73 ± 0.57% for RAD-DINO, 78.03 ± 0.95% for MIM-CXR and 75.86 ± 0.10% for CheSS. Extended Data Table 3 offers a more comprehensive evaluation. It presents the mean MCC, mean average precision and mean F₁ scores for the 19 classes.

To further examine label efficiency, we reduced the training data to 50%, 25%, 10% and 5% of the full dataset and evaluated Ark⁺ under both the fine-tuning and linear-probing set-ups, and then we compared its performance with linear-probing results from CXR-FM. As shown in Fig. 4c, Ark⁺ significantly outperformed CXR-FM in low-data scenarios. For instance, under the same linear-probing set-up, Ark⁺ exceeded the mean AUC for CXR-FM by 11.29%, 9.33% and 4.75% when using 5%, 10% and 25% of the training samples, respectively. Interestingly, in these limited-data scenarios, Ark⁺ achieved better performance with linear-probing than with fine-tuning. However, when the amount of training data was increased, fine-tuning surpassed linear-probing, further boosting performance. This underscores the impact of an open model, particularly when sufficient training data are available.

Transfer to new sites without training

Domain shifts, caused by variations in patient populations, scanner types or imaging protocols across hospitals, can hinder the generalization and accuracy of deep learning models²⁹. Although transfer learning and domain adaptation techniques can address domain shift issues, they rely on access to data and labels from the target domain for retraining or fine-tuning, which limits the scalability and widespread deployment of foundation models.

To evaluate the adaptability of Ark⁺ in generalizing knowledge acquired during pretraining to unseen domains (data from new sites) without further training, we conducted experiments on a diverse range of datasets representing various clinical scenarios. The four datasets were collected from distinct sources and focused on different clinical objectives, namely the detection of pneumothorax, nodule, paediatric pneumonia and tuberculosis. Ark⁺ performed zero-shot transfer to detect these diseases using its pretrained multi-task heads, without any task-specific training. For comparison, we evaluated two supervised models from TorchXRayVision³⁰ trained on several datasets and two image-text pretrained models (KAD⁹ and ELIXR¹²) under the same zero-shot transfer conditions.

The ROC curves with 95% confidence intervals, shown in Fig. 5, illustrate the zero-shot transfer performance for disease detection across the datasets. Ark⁺ achieved impressive AUC scores for detecting pneumothorax (95.79%), paediatric pneumonia (97.60%) and tuberculosis (96.60%). For nodule detection, which excluded the images from ChestX-ray14, Ark⁺ attained an AUC score of 89.15%, significantly outperforming all other models. These results underscore the capacity of Ark⁺ to transfer to unseen domains and diverse clinical contexts without training, highlighting its potential for real-world deployment.

Tolerate sex-related bias

Population imbalance is a prevalent issue in medical datasets, often leading to the development of biased models³¹. These biases result in suboptimal diagnostic performance for minority populations that are under-represented in the training data, raising ethical concerns about equity and inclusivity in healthcare³². A robust foundation model must demonstrate resilience against biased training data, so that it delivers equitable and accurate diagnostics across all population groups.

To evaluate the tolerance of the model to sex-related bias, we followed the methodology of ref. 33 under a linear-probing configuration using sex-exclusive training sets from CheXpert³⁴. We adhered to their training versus testing splits to ensure a balanced number of cases per class across 20 male-only and 20 female-only folds, where the labels no finding and support device were excluded. We trained 40 linear classifiers on the male-only and female-only splits using embeddings from each model and evaluated these classifiers on the corresponding male-only and female-only test splits. Bias was assessed by identifying statistically significant performance declines when the training and test data came from opposite sexes compared with when they came from the same sex. For example, when tested on female-only sets, classifiers trained on female-only embeddings should not significantly outperform those trained on male-only embeddings unless bias exists.

Extended Data Table 4 presents the evaluation results for four large pretrained foundation models after assessing their robustness against sex-related bias. Results without statistically significant performance differences, highlighted in green, indicate robustness when the training data and test data differ in sex. Ark⁺ demonstrated exceptional resilience, with 13 unbiased results, outperforming CheSS with four, MIM-CXR with five and CXR-FM with eight. These findings underscore the superior ability of Ark⁺ to mitigate the effects of imbalanced data and reduce sex-related biases, thus enhancing equity and accuracy in computer-aided diagnosis.

Respond to novel diseases

The COVID-19 pandemic, caused by the SARS-CoV-2 virus outbreak in 2020, profoundly impacted global health. Effective screening of infected individuals became crucial for timely treatment, care and virus containment. Although reverse-transcription polymerase chain reaction testing is a standard diagnostic method, such testing was not available during the initial months of the pandemic, and CXR offers a rapid approach to identifying COVID-19-related lung abnormalities^35,36. However, the surge in patient numbers during the pandemic overwhelmed radiologists, highlighting the need for diagnostic models to assist in the efficient and accurate detection of COVID-19 cases. This global health crisis presented a unique opportunity to evaluate the ability of foundation models to adapt to novel diseases and support timely and accurate diagnosis. Thus, we assessed the adaptability of Ark⁺ in distinguishing COVID-19, pneumonia and normal cases using the COVIDxCXR-3 dataset³⁵, and compared the results with those from CXR-FM, MIM-CXR and RAD-DINO. Additionally, to demonstrate the extensibility of Ark⁺ to novel diseases, we incrementally and continually pretrained Ark⁺ with the COVIDxCXR-3 training data, producing an updated foundation model referred to as Ark⁺_+covid.

As shown in Extended Data Table 5, Ark⁺, despite lacking previous exposure to COVID-19, consistently outperformed MIM-CXR, a self-supervised model pretrained with COVID-19 images, under both the linear-probing and fine-tuning set-ups. The fine-tuned Ark⁺ model achieved a superior accuracy of 98.83 ± 0.26%, surpassing MIM-CXR, RAD-DINO, CXR-FM and the state-of-the-art COVID-Net³⁵ and Medical-MAE³⁷. Although the linear-probing performance of Ark⁺ initially lagged behind that of CXR-FM, incremental pretraining on the COVID-19 diagnostic task enabled Ark⁺_+covid to surpass CXR-FM under identical configurations. To further illustrate the impact of the incremental pretraining, we visualized the t-distributed stochastic neighbour embeddings³⁸ of the hold-out test data in COVIDxCXR-3 generated by Ark⁺ and Ark⁺_+covid (Extended Data Fig. 3a–c). The embeddings for the COVID-19 cases showed further segregation from pneumonia and normal cases after pretraining, highlighting the discriminative prowess of the model in capturing COVID-19-specific features to improve diagnostic accuracy. Furthermore, Extended Data Fig. 3d–g shows the evolution of t-distributed stochastic neighbour embeddings as Ark⁺ was fine-tuned with increasing numbers of samples, demonstrating its efficient adaptation to the novel COVID-19 condition. These results emphasize the adaptability and extensibility of Ark⁺ in addressing previously unseen diseases during pretraining or potentially novel conditions, such as future pandemics. By combining incremental learning with high diagnostic performance, Ark⁺ has demonstrated its value as a reliable tool for rapidly evolving clinical needs.

Protect privacy and distribute pretraining

Ark⁺ adopts centralized pretraining by default owing to the absence of privacy concerns with public datasets. However, the concept underlying Ark⁺ is not restricted to public data, a particular modality or a specific architecture. Given its neutrality in terms of architecture or modality, Ark⁺ can use various backbones to accommodate different modalities (for example, computed tomography, magnetic resonance imaging, videos, audios, tables and text) and multimodal clinical data^39,40. When developing large multimodal AI models for medicine with large clinical data, patient privacy must be safeguarded and pretraining must be distributed across several sites. Thus, we extended Ark⁺ to federated Ark⁺ to support federated learning by deploying a (local) instance of Ark⁺ at each site, as illustrated in Extended Data Fig. 2. Each local Ark⁺ was trained independently on the datasets available at its respective site. The model weights from these local instances were then collected by a central server, which aggregated them using a weight averaging operation. The results were distributed back to the local sites so that they could continue training collaboratively. We simulated a federated learning scenario by distributing one, two or three pretraining datasets across three local sites, each of which hosted an instance of Ark⁺. Although this simulation used only public datasets, the datasets at each local site could be considered ‘private’ and ‘inaccessible’ by other sites. The experimental results, presented in Extended Data Table 6, demonstrate the effectiveness of our federated Ark⁺ in handling heterogeneous annotations across private clients, a new capability for conventional federated learning⁴¹. The table also shows the performance gains due to learning from federating data compared with isolated (local site) training. This approach may empower researchers around the globe to confidently federate private data while ensuring patient privacy and could advance the application of large open foundation models to medicine.

Discussion

This article introduces Ark⁺, an open foundation model applied to chest radiography and pretrained cyclically by accruing and reusing the knowledge embedded in heterogeneous expert labels from six public datasets. Through eight clinical scenarios, we demonstrated the generalizability, adaptability, robustness, extensibility and superior performance of Ark⁺ over nine other foundation models in diagnosing thoracic diseases.

Ark⁺ is generalizable. It surpassed other large pretrained models on the test data ChestX-ray14 (ref. 19), underlining its generalizability in diagnosing common thoracic diseases (Fig. 2a). The supervised models Ark⁺ and CXR-FM consistently outperformed the self-supervised models MIM-CXR, CheSS and RAD-DINO, indicating the power of expert knowledge. Fine-tuning further amplified the advantage of Ark⁺ over CXR-FM, signifying the openness of the foundation model. Moreover, Ark⁺ generated more comprehensive diagnostic predictions through its multi-task heads, a capability that expands its diagnostic scope and addresses potential misdiagnosis (Fig. 2b,c).

Ark⁺ is adaptable. Despite being pretrained on only global labels in VinDr-CXR⁴², it adapted to diagnostic tasks expanded with lesion-level labels (Fig. 3a–c). This adaptability obviates the requirement for extensive retraining for evolving diagnostic needs. Ark⁺ surpassed CXR-FM in detecting rare conditions with only 1–5 samples (Fig. 3d), emphasizing its clinical utility in data-scarce environments. Most importantly, Ark⁺ accommodated shifts in diagnostic setting, achieving high AUC scores in detecting four diseases with unseen datasets through zero-shot transfer (Fig. 5). The ability to transfer to new sites without training highlights its potential for real-world deployment.

Ark⁺ is robust. It surpassed other foundation models on the unseen ChestDR dataset when handling long-tailed disease distributions (Fig. 4). In limited-data scenarios, linear-probing outperformed fine-tuning, underscoring the importance of a large amount of pretraining that yields strong features to overcome overfitting and, thus, making simpler models more effective. With sufficient data, fine-tuning exceeded linear-probing, thus demonstrating another advantage of open and adaptable models. Furthermore, Ark⁺ naturally tolerates sex-biased data without specific designs for bias mitigation (Extended Data Table 4). This capability advances equitable and accurate diagnostics across demographic groups, thus addressing ethical concerns in clinical AI. Future evaluations of model bias should be extended to attributes such as race, age and other demographic factors to ensure that assessments of robustness are more comprehensive and inclusive.

Ark⁺ is extensible. On incorporating a (new) COVID-19 diagnostic task, Ark⁺ responded effectively (Extended Data Table 5) through its incremental learning capability, highlighting its potential to extend to emerging diseases and future pandemics. Through its extension to federated learning (Extended Data Fig. 2), federated Ark⁺ preserves patient privacy and distributes pretraining across clients. This new capability overcomes a limitation with conventional federated learning when handling heterogeneous annotations across clients, empowers privacy-preserving data-sharing for developing large public foundation models, and positions Ark⁺ as a robust, scalable and privacy-aware framework for open medical AI.

Ark⁺ is open, public, light and affordable. Ark⁺ is relatively small, making its training affordable and efficient. All training data and labels are sourced from public datasets, and Ark⁺ helps eliminate the need for the manual consolidation of heterogeneous labels, thus reducing data costs to near zero. Its training takes approximately 700 h using four A100 GPUs. The openness of Ark⁺ not only allows fine-tuning and incremental learning with new data to enhance diagnostic accuracy in various scenarios (Figs. 2 and 4 and Extended Data Table 5) but also enables its extension from classification¹⁴ to localization¹⁶, segmentation¹⁷ and their integration¹⁸. Ark⁺ is pretrained with data sourced predominantly from populations in the USA, Vietnam and China, but we hope its public nature will attract more diverse data sources, especially from under-represented regions and demographic groups, to improve its generalizability. We also believe that its lightness and affordability will encourage quick replication, public evaluation and local adoptions. We anticipate that its full openness¹³ will foster future collaborative developments of Ark⁺ from AI specialists for particular organs (for example, brain and fundus), specialties (for example, pathology and dermatology) and modalities (for example, computed tomography and magnetic resonance imaging) to AI generalists for medicine trained with multimodal clinical data (for example, text, tables, images, videos and audios)^39,40.

In conclusion, Ark⁺ demonstrated impressive performance in diagnosing thoracic diseases across various scenarios. Its generalizability, adaptability, robustness, extensibility, openness, lightness and affordability make it a powerful model fundamental to medical imaging for the public. Its exceptional capabilities are attributable to a simple yet powerful insight: aggregating numerous datasets (public or private and large or small) and using their available annotations incur minimal cost while substantially increasing data size, expanding protocol coverage, diversifying patient populations and accruing expert knowledge from a broad spectrum of global sources. Ark⁺ attests that accruing and reusing knowledge from heterogeneous expert annotations with even only public datasets can surpass the performance of proprietary models trained on unusually large data. Considering the ubiquity of heterogeneous data and labels across various fields such as biology, chemistry, physics and medicine, then thanks to its neutrality in modality and architecture, the concept underlying Ark⁺ is expected to have a far-reaching potential beyond imaging. We hope this research will inspire many more researchers to share code and datasets or federate privacy-preserving data to build fully open, high-performance foundation models, thus accelerating open science and democratizing AI.

Online content

Any methods, additional references, Nature Portfolio reporting summaries, source data, extended data, supplementary information, acknowledgements, peer review information; details of author contributions and competing interests; and statements of data and code availability are available at https://doi.org/10.1038/s41586-025-09079-8.

Methods

Pretraining Ark⁺ using numerous public datasets

Ark⁺ learns superior and robust visual representations from large aggregated medical images by accruing and reusing the expert knowledge embedded in all available heterogeneous labels. Extended Data Fig. 1 depicts the pretraining framework for Ark⁺, which has a teacher model and a student model, each augmented with multi-task heads (each corresponding to one task) and trained through cyclic pretraining. Cyclic pretraining is an iterative process: at each iteration, the student accrues knowledge from every expert annotation through its corresponding task head by scanning all datasets (tasks) individually and sequentially for one epoch. At the end of each task, the knowledge accrued by the student is accumulated into the teacher through exponential moving averages and reused to help the student accrue more knowledge from the expert annotations associated with the next dataset (task). To reinforce the feedback loop between the student and teacher, a projector introduced after their encoders maps the representations to the same feature space through consistency loss. The projected representation also serves as the embedding for linear-probing in our evaluation. After pretraining, the knowledge accumulated by the teacher can be reused and transferred to target tasks. Unlike the previous design in Ark¹⁴, Ark⁺ feeds the teacher model with the resized original image rather than using random cropping. This update to the data augmentation ensures that the teacher provides a consistent and steady supervisory signal for computing the consistency loss, thereby accelerating training and enhancing performance. To emphasize the strengths of the pretraining framework for Ark⁺, we describe the ablation studies used in the multi-task head design, the cyclic training approach and the performance improvements achieved with Ark⁺ in the Supplementary Information.

Pretraining set-up.

Ark⁺ was pretrained with 704,363 chest radiographs from six datasets (Extended Data Table 2) collected from six different institutions around the world and annotated by their experts. The labels originally provided were used, as we did not manually consolidate the heterogeneous labels into a predefined list. To avoid test-image leaks, all validation and test data were excluded from the Ark⁺ pretraining. Ark⁺ leverages the large version of the Swin transformer with an input resolution of 768 × 768 as the backbone. The teacher and student encoders were initialized with the officially released weights trained on ImageNet (https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_large_patch4_window12_384_22kto1k.pth), and the projectors and the multi-task heads were randomly initialized. The student model was trained with both classification and consistency loss. The classification loss (L_cls) was tailored to the labels in each dataset using binary cross-entropy for binary/multi-label tasks and cross-entropy for multi-class tasks. The consistency loss (L_consist) was optimized using the mean-squared error. Training used a stochastic gradient descent optimizer with an initial learning rate of 0.3 and a batch size of 50 across four Nvidia A100 GPUs with 80 GB memory each. At the end of each task, the teacher model was updated using an epoch-wise exponential moving average based on one epoch of learning by the student and using a momentum of 0.9. Image augmentations included random cropping and rotation, as well as changes in brightness, contrast and gamma distribution. The model was pretrained for 50 epochs, iterating through all datasets 50 times. After pretraining, the teacher model, including the encoder, projector and multi-task heads, was deployed for the clinical target tasks through fine-tuning, linear-probing and zero-shot transfer.

Datasets for developing and evaluating Ark⁺

Pretraining datasets.

Extended Data Table 7 summarizes the six datasets used for pretraining Ark⁺. MIMIC-II (MIMIC-CXR v.2.0)⁴⁵ contains 377,110 frontal and lateral chest radiographs and 14 structured labels derived from 227,827 free-text radiology reports using CheXpert³⁴. The radiographic studies were collected at Beth Israel Deaconess Medical Center in Boston, MA, between 2011 and 2016. CheXpert³⁴ consists of 224,316 CXRs of 65,240 patients collected at Stanford Hospital between October 2002 and July 2017, including both frontal and lateral views. Each study was labelled for the presence of 14 observations as positive, negative or uncertain. The National Institutes of Health (NIH) ChestX-ray14 dataset¹⁹ comprises 112,120 frontal-view CXRs collected from 30,805 unique patients, which were obtained from radiological reports spanning from 1992 to 2015. Each image is associated with 14 common disease labels, which overlap with those in CheXpert but are not identical. The RSNA Pneumonia Detection Challenge (RSNA Pneumonia)⁴³, created in collaboration with the Radiological Society of North America (RSNA), comprises 30,000 frontal-view CXRs sourced from the NIH. These images are annotated with image-level labels indicating the presence of lung opacity and any abnormalities. VinDr-CXR⁴² contains 18,000 postero-anterior-view CXRs annotated with the localization of 22 critical findings (local labels) and six thoracic diagnoses (global labels). The images were collected from two of Vietnam’s largest hospitals between 2018 and 2020. The Shenzhen Hospital X-ray set⁴⁴ comprises 662 frontal-view CXRs, including 326 normal X-rays and 336 abnormal X-rays showing various manifestations of tuberculosis. The images were collected by Shenzhen No. 3 Hospital in Shenzhen, Guangdong Province, China, in 2012. If a dataset provided an official data split, we used the training set data for pretraining; otherwise, we randomly allocated 70% of the data to pretraining.

Data with rare diseases.

CXR-LT²⁶ expands on MIMIC-II⁴⁵ by increasing the target classes from 14 to 26, thus incorporating labels for 12 more disease findings derived from parsing radiology reports. Three rare diseases from the newly added labels were used for our few-shot linear-probing benchmark.

Data in long-tailed distributions.

ChestDR²⁸ offers 4,848 frontal chest radiography images (from 4,848 patients) gathered from two regional hospitals in Hubei and Jiangxi provinces, China. The initial disease labels in the dataset were assigned by a radiological resident, supported by previously signed radiology reports, and subsequently verified by a senior radiologist. The dataset exhibits a long-tailed distribution, comprising ten head classes with sample numbers ranging from 429 to 1,300 and nine tail classes with sample numbers ranging from 23 to 305.

Data for zero-shot transfer.

SIIM-ACR⁴⁷, initially introduced for pneumothorax segmentation, includes 12,047 CXRs with 2,669 cases of pneumothorax. TBX-11K⁵⁰ released 8,400 CXRs from the official training and validation sets, including 3,800 healthy cases, 3,800 sick but non-tuberculosis cases and 800 cases with manifestations of tuberculosis. Mendeley-V2 (ref. 49) is a paediatric CXR dataset that includes 4,273 pneumonia images and 1,583 normal images. NODE21 (ref. 48) represents a nodule detection challenge with 4,882 frontal CXRs, of which 1,134 CXRs are annotated with bounding boxes around nodules, whereas the remaining 3,748 images represent the negative class without nodules.

Data for COVID-19.

COVIDxCXR-3 (ref. 35) provides a dataset featuring over 30,000 CXR images sourced from a diverse multinational cohort of more than 16,400 patients. It has 16,490 positive COVID-19 images derived from a cohort of over 2,800 patients. Following Wang et al.³⁵ and Xiao et al.³⁷, we used label version V9A, which was curated for the detection and differentiation of patients without pneumonia and patients with non-COVID-19 pneumonia and COVID-19 pneumonia.

Comparison with large pretrained foundation models

CXR-FM¹⁰ is a proprietary CXR foundation model pretrained on 821,544 labelled and mostly private CXRs from three different sources using supervised contrastive learning⁵¹. ELIXR¹² is an advanced extension of CXR-FM that integrates a language-aligned image encoder with a fixed large language model. It is pretrained on 650,264 private CXRs and 243,324 images from the MIMIC-CXR dataset, along with their associated free-text radiology reports. RAD-DINO²⁰ is a publicly accessible, vision transformer model pretrained on 882,775 CXRs from five public, de-identified datasets using a state-of-the-art self-supervised learning approach (DINOv2 (ref. 52)). MIM-CXR²¹ is a publicly accessible, self-supervised model pretrained on 926,028 CXRs from 13 public, de-identified datasets using masked image modelling (SimMIM⁵³). CheSS²² is a publicly accessible model pretrained on a 4.8 million in-house CXR dataset using self-supervised contrastive learning (MoCo v.2)⁵⁴. KAD⁹ is an image-text model pretrained using the knowledge-enhanced auto diagnosis approach, which leveraged a medical knowledge base to guide vision-language pretraining with paired chest radiographs and radiology reports from MIMIC-CXR. XRV-ResNet-50 and DenseNet-121 (ref. 30) are two official models from TorchXRayVision pretrained using over 200,000 unique chest radiographs from four public datasets, each filtered to include only one antero-posterior or postero-anterior view per patient⁵⁵.

Evaluation metrics and statistical analysis

To offer a comprehensive assessment of the classification performance of a model, we adopted the following metrics, which focus on different aspects, such as discrimination ability, overall accuracy, balance between precision and recall, and handling of class imbalances.

AUC evaluates the performance on binary classification tasks. A ROC curve is a plot of the true positive rate (sensitivity) against the false positive rate (1 - specificity) at various threshold values. A higher AUC value (closer to 1) indicates better positive and negative class model discriminatory ability. Accuracy measures the proportion of correctly classified instances (both true positives and true negatives) relative to the total number of instances. It is commonly used for evaluating classification models and is calculated as the ratio of correct predictions to total predictions.

The F₁ score is the harmonic mean of both precision and recall. It provides a balanced assessment of the classification performance of a model, especially in cases of class imbalance.

The average precision evaluates the performance of models using precision–recall curves, particularly for binary classification tasks. This measure calculates the area under the precision versus recall curve to provide an indication of how well the model balances precision and recall across different thresholds.

MCC measures the quality of binary classifications by considering true positives, true negatives, false positives and false negatives. Values range from −1 to 1, where 1 indicates perfect prediction, 0 indicates a random prediction and −1 indicates total disagreement between predictions and actual labels. MCC is particularly useful for imbalanced datasets as it considers all four elements of the confusion matrix.

The thresholds for computing accuracy, F₁ and MCC were determined with the Youden index⁵⁶, which involved selecting a threshold that maximized the difference between the true positive rate and the false positive rate. In other words, we determined the threshold that optimally balances the ability to detect true positives while minimizing false positives.

A statistical analysis incorporating an independent two-sample t-test and 95% confidence intervals was used to determine the significance among model performances and draw reliable conclusions. In experiments involving fine-tuning and linear-probing, as the linear classifier was randomly initialized, each run produced different results. Hence, we evaluated each model at least ten times and reported the mean and standard deviation for each performance metric. We performed a statistical analysis using the two-sided independent t-test. For experiments using zero-shot transfer, the model produced consistent results, as all components were frozen. To obtain reliable estimates, we derived 95% confidence intervals through bootstrapping techniques on the test set to create 1,000 resampled datasets. Within each resampled dataset, we computed the AUC score and constructed a confidence interval to encapsulate its variability. This iterative approach, repeated 1,000 times, yielded a distribution of statistics and corresponding confidence intervals and offered valuable insights into the variability and uncertainty inherent in our estimates.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Extended Data

Extended Data Fig. 1 |. Illustration of training pipeline of Ark⁺.

Ark⁺ is built on a teacher-student framework augmented with multi-task heads, each corresponding to a specific task, and employs cyclic pretraining to iteratively accrue and reuse knowledge. At each iteration, the student model sequentially scans datasets (tasks) one by one for one epoch, learning from expert annotations through the task-specific head. The knowledge accrued by the student is accumulated into the teacher via exponential moving averages (EMA), enabling the teacher to guide the student in subsequent tasks. To reinforce the feedback loop between the student and teacher, after their encoders, a projector is introduced to map the representations to the same feature space via the consistency loss. The projected representation also serves as the embedding for linear-probing in our evaluation. After pretraining, the accumulated knowledge in the teacher can be reused and transferred to target tasks. Differing from the previous design in Ark¹⁴, Ark⁺ feeds the teacher with the resized original image instead of random cropping. This update in data augmentation ensures the teacher provides a consistent and steady supervisory signal for computing the consistency loss, thereby accelerating training and enhancing performance. Images adapted with permission from ref. 19, IEEE.

Extended Data Fig. 2 |. Illustrating a federated learning scenario by distributing Ark⁺’s training across three local sites.

Ark⁺ can be federated by deploying a (local) Ark⁺ at each site to protect privacy and distribute training. In this setup, each local site trains its own Ark⁺ with all its available data, employing the same cyclic training strategy to train the student and the same epoch-wise EMA to update the teacher. After completing a round of local training, all sites send their student weights to a central server, where weights are averaged to aggregate these local models into a “master” model, consolidating knowledge from all sites. This master model is then distributed back to the local sites, allowing iterative learning and continuous improvement of the local teacher model. For simplicity, the projectors and multi-task heads are omitted from the illustration.

Extended Data Fig. 3 |. Illustration of how the embeddings for COVID-19, Pneumonia and Normal evolve in t-SNE³⁸.

From Ark⁺ (a) to Ark⁺_+covid (b), an upgraded Ark⁺ model is created by incrementally and continually pretraining Ark⁺ with the COVID-19 diagnostic task. Ark⁺_+covid has more distinct embeddings for the three conditions, revealing its newly-acquired capacity for capturing the features specific to COVID-19. This capability can be further enhanced through fine-tuning (c). d–g illustrate how the embeddings for COVID-19, Pneumonia and Normal evolve in t-SNE from the pretrained Ark⁺ (d) to fine-tuning Ark⁺ with increasing numbers of samples continually (e–g). Ark⁺ obtains distinguishable embeddings when the training data reach 3,000, representing 10% of the full training set (f). This highlights Ark⁺’s ability to efficiently develop distinct feature representations, markedly enhancing its diagnostic accuracy and adaptability to new information.

Extended Data Table 1 |.

Comparative overview of Ark⁺ with nine large-scale pretrained models

Model	Learning type	Model backbone	Parameter size	Input resolution	Infrastructure	Training data size	Data accessibility	Full Openness
Ark+	Supervised	Swin-Large	197M	768×768	A100 GPU	704K	public	Yes
CXR-FM	Supervised	EfficientNet-L2	480M	1024×1024	TPUv3	821K	public+private	No
RAD-DINO	Self-supervised	ViT-Base	86M	518×518	A100 GPU	882K	public	No
MIM-CXR	Self-supervised	Swin-Base	88M	224×224	V100 GPU	926K	public	No
CheSS	Self-supervised	ResNet-50	25.6M	512×512	V100 GPU	4.8M	private	No
KAD	Image-text	ResNet-50	25.6M	512×512	Not reported	377K	public	Yes
ELIXR	Image-text	ELIXR-C + Q-Former	Not reported	1280×1280	TPUv3	893K	public+private	No
XRV-ResNet XRV-DenseNet	Supervised	ResNet-50 DenseNet-121	25.6M 7.98M	512×512 224×224	Not reported	201K	public	Yes

According to Basdevant et al.¹³, Ark⁺, KAD, XRV are considered “fully open”: open source, open model, open weights, and open code and data for pretraining and evaluation. By contrast, no pretraining code was released for CXR-FM, RAD-DINO, MIM-CXR, and ELIXR; no pretraining data were released for CheSS.

Extended Data Table 2 |.

Overview of datasets utilized for evaluating Ark⁺

Dataset	Clinical conditions	Configuration	#Train	#Test
Internal
ChestX-ray14	14 thoracic diseases	fine-tuning, linear-probing	75,312	25,596
VinDr-CXR	6 image-level and 22 lesion-level thoracic findings	linear-probing	15,000	3,000
CXR-LT* (rare condition)	Subcutaneous Emphysema Tortuous Aorta Pneumoperitoneum	k-shot learning (linear-probing)	2–10 2–10 2–10	496–488 477–469 428–420
CheXpert	12 thoracic diseases	sex bias study (linear-probing)	following Larrazabal et al.
External
ChestDR	19 thoracic diseases	fine-tuning, linear-probing	979	3,869
SIIM-ACR†	Pneumothorax	zero-shot transfer	0	12,046
NODE21‡	Nodule	zero-shot transfer	0	4,265
TBX-11K	Tuberculosis	zero-shot transfer	0	8,400
Mendeley-V2	Pediatric pneumonia	zero-shot transfer	0	5,856
COVIDxCXR-3	COVID-19 and Pneumonia	fine-tuning, linear-probing	29,634	400

This article evaluates Ark⁺ using 10 datasets—four seen datasets for an internal evaluation and six unseen datasets for an external evaluation—by following the official training/testing data splits provided by each dataset, except for CXR-LT (rare condition) in few-shot learning, where we randomly selected 1 to 5 samples with the disease and an equal number of samples labeled No Finding for training, using all remaining samples for testing.

^*CXR-LT expands on MIMIC-II with 12 new classes, among which three rare diseases are selected for our few-shot learning. To ensure no samples are seen by Ark⁺ during its pretraining, only images from the hold-out validation and test sets of MIMIC-II were utilized.

^†SIIM-ACR, originally for pneumothorax segmentation, is converted into a classification task for zero-shot pneumothorax detection.

^‡617 images shared with the ChestX-ray14 dataset have been excluded to ensure an external evaluation for Ark⁺.

Extended Data Table 3 |.

Performance on diagnosing 19 thoracic diseases with long-tailed distributions

Configuration	Model	meanAUC(%)	meanMCC(%)	meanAP(%)	meanF1(%)
	CheSS	73.20±0.02	21.34±0.08	24.53±0.06	26.91±0.13
linear-probing	MIM-CXR	72.14±0.03	20.45±0.09	23.24±0.05	26.11±0.12
	RAD-DINO	79.18±0.06	27.53±0.14	33.49±0.07	31.15±0.16
	ELIXR	81.97±0.24	29.95±0.46	34.28±0.47	32.60±0.47
	CXR-FM	84.93±0.10	33.29±0.29	40.02±0.15	32.68±1.26
	Ark+	85.92±0.06	35.70±0.21	42.96±0.10	34.75±0.43
	CheSS	75.86±0.10	23.31±0.29	25.15±0.26	27.63±0.31
fine-tuning	MIM-CXR	78.03±0.95	25.25±0.92	28.06±1.01	28.99±0.58
	RAD-DINO	82.73±0.57	31.10±1.16	37.24±1.90	33.89±1.11
	Ark+	86.55±0.35	36.90±0.55	45.69±0.84	38.81±0.55

Each model was trained using all 979 available samples and evaluated on the 3,869 hold-out test samples. The table reports the mean and standard deviation (mean ± s.d.) of four metrics across 10 independent trials.

Extended Data Table 4 |.

Comparing four large-scale pretrained foundation models’ ability to tolerate sex-related bias

AUC(%)		CheSS		MIM-CXR		CXR-FM		Ark+
Dz*	Test data	Training data		Training data		Training data		Training data
		Female	Male	Female	Male	Female	Male	Female	Male
EC	Female	63.92±1.4	63.13±1.4	61.82±1.3	61.47±1.6	69.53±1.4	67.26±1.5	70.41±1.4	68.34±1.5
	Male	66.11±1.6	68.92±1.6	64.79±1.4	68.04±1.5	70.46±1.5	71.63±1.5	69.99±1.5	72.59±1.5
CM	Female	79.37±0.8	77.99±0.9	72.99±1.1	72.28±1.3	84.15±0.7	83.82±0.6	86.64±0.7	86.32±0.8
	Male	80.00±0.8	81.47±0.9	73.84±1.0	75.91±0.8	85.11±0.8	85.59±0.9	87.16±0.8	87.52±0.8
LO	Female	67.74±0.7	67.16±0.7	68.96±0.7	68.11±0.7	73.38±0.7	73.00±0.6	74.84±0.4	74.72±0.6
	Male	72.27±0.5	73.32±0.5	73.44±0.8	74.10±0.6	77.24±0.4	77.67±0.4	78.75±0.4	79.04±0.4
LL	Female	63.03±1.4	62.79±1.8	66.71±1.9	65.57±2.2	75.60±1.5	75.57±1.3	79.53±1.3	79.12±1.3
	Male	64.81±1.7	66.69±1.6	67.50±1.3	69.36±1.3	77.29±1.2	77.49±1.2	79.80±1.2	80.25±1.1
ED	Female	78.60±0.7	78.32±0.8	78.33±0.7	77.52±0.8	83.35±0.7	82.99±0.7	84.35±0.7	84.02±0.7
	Male	81.95±0.6	82.29±0.6	81.86±0.6	82.43±0.6	86.04±0.4	86.01±0.4	86.95±0.5	87.07±0.5
CS	Female	67.72±1.3	67.23±1.5	67.27±1.5	66.93±1.5	75.22±1.1	74.63±1.4	77.76±1.1	77.72±1.2
	Male	67.45±1.1	68.18±1.3	66.62±1.6	67.01±1.5	75.27±1.0	75.88±1.0	77.67±1.2	78.33±1.2
PN	Female	62.05±2.2	62.25±2.7	63.85±1.5	64.24±1.4	73.46±1.8	73.45±2.1	75.58±2.0	75.11±1.9
	Male	65.48±2.2	66.81±1.7	65.13±2.0	67.50±2.2	78.38±1.7	79.31±1.4	79.67±1.7	80.11±1.6
AT	Female	63.89±0.8	63.04±0.7	64.77±1.1	64.36±1.2	70.10±0.8	69.31±0.7	72.46±0.8	71.32±0.8
	Male	67.05±0.9	67.81±0.8	67.78±0.8	68.74±0.5	72.51±0.5	73.67±0.7	75.02±0.7	76.05±0.5
PX	Female	72.95±1.1	71.36±1.2	77.69±1.2	76.14±1.3	84.74±1.0	84.33±1.0	87.34±1.0	86.89±1.0
	Male	67.10±2.0	68.46±1.5	74.58±1.4	76.22±1.7	83.02±0.9	83.64±1.2	86.14±1.3	86.54±1.3
PE	Female	79.73±0.6	79.04±0.5	81.15±0.5	80.02±0.6	88.70±0.5	88.46±0.4	89.61±0.5	89.34±0.5
	Male	81.64±0.6	82.38±0.6	83.61±0.6	84.26±0.5	89.53±0.5	89.75±0.5	90.60±0.4	90.80±0.3
PO	Female	70.45±2.8	67.89±3.5	73.05±2.6	71.47±3.0	78.59±2.5	76.55±2.4	80.11±2.1	78.53±2.1
	Male	65.42±2.0	69.50±2.9	69.58±3.0	72.08±2.6	77.12±2.2	79.86±2.4	80.78±2.5	81.83±2.6
FR	Female	70.57±1.5	65.70±2.0	73.88±1.3	70.29±1.7	80.17±1.2	76.98±1.3	80.08±0.9	77.92±1.2
	Male	61.24±1.6	66.04±1.5	68.69±2.1	72.68±1.3	76.28±1.4	79.34±1.1	78.57±1.3	80.94±1.3
# Unbiased Results ↑		4		5		8		13

All models are evaluated on CheXpert³⁴ via linear-probing using sex-exclusive training/test splits by following Larrazabal et al.³³. Ark⁺ has 13 unbiased results (bolded), demonstrating the greatest resilience to sex-imbalanced data compared with the other models. Sex bias is characterized by a significant drop in performance when training and test data are of the opposite sex compared to when they are of the same sex. A robust model should have more unbiased results that do not show a statistically significant performance difference (at p = 0.05) between datasets exclusively composed of male or female data. All unbiased results are highlighted in green.

^*Abbreviation: Dz: Disease, EC: Enlarged Cardiomediastinum, CM: Cardiomegaly, LO: Lung Opacity, LL: Lung Lesion, ED: Edema, CS: Consolidation, PN: Pneumonia, AT: Atelectasis, PX: Pneumothorax, PE: Pleural Effusion, PO: Pleural Other, FR: Fracture.

^↑Greater number of unbiased results indicate that the model is more effective in tolerating sex-related biases.

Extended Data Table 5 |.

Performance comparison on COVID-19 diagnostic task

Model	ACC(%)	meanAUC(%)	meanMCC(%)	meanAP(%)	meanF1(%)
fine-tuning *
MIM-CXR	95.88±1.06	99.11±0.47	93.88±2.02	97.10±1.65	95.67±1.37
RAD-DINO	97.05±0.27	99.64±0.14	95.58±0.81	99.08±0.30	96.81±0.56
Ark+	98.83±0.26	99.87±0.14	97.35±0.32	99.60±0.19	98.59±0.47
Ark++covid	99.08±0.26	99.96±0.01	97.72±0.17	99.89±0.03	98.33±0.03
linear-probing
CXR-FM	98.60±0.46	99.80±0.04	97.07±0.32	99.45±0.09	98.14±0.27
MIM-CXR	95.50±0.16	99.16±0.14	91.87±0.49	97.60±0.26	94.57±0.38
RAD-DINO	95.85±0.14	99.07±0.09	92.28±0.33	98.08±0.16	94.58±0.25
Ark+	95.87±0.20	99.09±0.10	93.04±0.28	97.80±0.33	95.49±0.19
Ark++covid	99.10±0.12	99.90±0.06	97.92±0.21	99.61±0.36	98.97±0.16

Evaluating Ark⁺’s transferability to a novel disease, COVID-19, that Ark⁺ has never encountered in its pretraining. Via linear-probing, Ark⁺ performs significantly better than MIM-CXR (p-value: 1.98E-04), on a par with RAD-DINO (p-value: 1.38E-01), and significantly inferior to CXR-FM (p-value: 3.44E-06) in Accuracy (ACC), while Ark⁺ outperforms MIM-CXR and RAD-DINO via fine-tuning and overtakes CXR-FM’s linear-probing performance. However, once Ark⁺ has been incrementally and continually pretrained with the COVID-19 diagnostic task, the resultant upgraded model, named Ark⁺_+covid, surpasses CXR-FM via linear-probing. The findings underscore the importance of openness in foundation models for continual pretraining, which endows Ark⁺ with the adaptability and scalability necessary to address novel diseases and potential future pandemics.

^*Medical-MAE³⁷ achieved an accuracy of 97.3% and COVIDNet³⁵ holds an official accuracy score of 98.3% with their best-performing models.

Extended Data Table 6 |.

Performance comparison among centralized, isolated (local site), and federated Ark⁺

Pretraining task	Metrics	3-site	Isolated (local site) training	Federated training	Centralized training
MIMIC-II	AUC	Site #1	79.67	79.41	80.37
CheXpert	AUC	Site #2	87.60	88.53	89.60
RSNA Pneumonia	Accuracy		75.45	75.78	75.56
ChestX-ray14	AUC		81.61	83.16	84.15
VinDr-CXR	AUC	Site #3	94.63	96.17	96.63
Shenzhen	AUC		98.66	99.29	99.29
Average performance (%)			86.27	87.06	87.60

Training federated Ark⁺ with data distributed across multiple clients maintains high performance on each task, underscoring the effectiveness and viability of distributed training for Ark⁺ . The performance of federated Ark⁺ was evaluated for each task using their corresponding local teacher models, with predictions generated from the multi-task heads immediately after distributed training, without any additional fine-tuning. For a comprehensive comparison, we also include isolated (local site) training, where a local Ark⁺ is trained with all datasets available at a particular site but no datasets are federated across sites. For Sites #2 and #3, federated training achieves superior performance compared with isolated (local site) training, highlighting the advantages of federated training and performance gains from federating privacy-preserving data over isolated (local site) training. While federated Ark⁺ can offer competitive performance in scenarios requiring privacy protection and distributed computing across multiple sites, centralized training is preferred when feasible, as it eliminates the communication overhead and allows the model to achieve superior overall performance^58,59.

Extended Data Table 7 |.

Overview of datasets utilized for pretraining Ark⁺

Datasets	Classification Task	Country	Collection Institute	# Pretrain
MIMIC-II	Multi-label	USA	BIDMC	368,879
CheXpert	Multi-label	USA	Stanford Hospital	223,414
ChestX-rayl4	Multi-label	USA	NIH	75,312
RSNA Pneumonia	Multi-class	USA	RSNA	21,295
VinDr-CXR	Multi-label	Vietnam	VinBigData	15,000
Shenzhen	Binary	China	Shenzhen No.3 Hospital	463

Ark⁺ is pretrained on 704,363 CXR images sourced from six public datasets collected from six different institutions worldwide, utilizing readily-accessible but heterogeneous expert labels.

Supplementary Material

The online version contains supplementary material available at https://doi.org/10.1038/s41586-025-09079-8.

Data availability

All data are publicly available as follows: MIMIC-II (https://physionet.org/content/mimic-cxr-jpg/2.0.0/); CheXpert (https://stanfordmlgroup.github.io/competitions/chexpert/); NIH ChestX-ray14 (https://nihcc.app.box.com/v/ChestXray-NIHCC/folder/36938765345); RSNA Pneumonia Detection Challenge (www.kaggle.com/c/rsna-pneumonia-detection-challenge); VinDr-CXR (https://vindr.ai/datasets/cxr); Shenzhen Hospital X-ray set (https://data.lhncbc.nlm.nih.gov/public/Tuberculosis-Chest-X-ray-Datasets/Shenzhen-Hospital-CXR-Set/index.html); CXR-LT (https://physionet.org/content/cxr-lt-iccv-workshop-cvamd/1.1.0/); ChestDR (https://springernature.figshare.com/articles/dataset/ChestDR_Thoracic_Diseases_Screening_in_Chest_Radiography/22302775); SIIM-ACR (www.kaggle.com/c/siim-acr-pneumothorax-segmentation); TBX-11K (www.kaggle.com/datasets/usmanshams/tbx-11); Mendeley-V2 (www.kaggle.com/datasets/andrewmvd/pediatric-pneumonia-chest-xray); NODE21 (https://node21.grand-challenge.org/) and COVIDxCXR-3 (https://github.com/lindawangg/COVID-Net).

Code availability

The pretrained Ark⁺ model and source code are publicly available via GitHub at https://github.com/jlianglab/Ark.