External Validation of Deep Learning Algorithms for Radiologic Diagnosis: A Systematic Review

Abstract

Purpose

To assess generalizability of published deep learning (DL) algorithms for radiologic diagnosis.

Materials and Methods

In this systematic review, the PubMed database was searched for peer-reviewed studies of DL algorithms for image-based radiologic diagnosis that included external validation, published from January 1, 2015, through April 1, 2021. Studies using nonimaging features or incorporating non-DL methods for feature extraction or classification were excluded. Two reviewers independently evaluated studies for inclusion, and any discrepancies were resolved by consensus. Internal and external performance measures and pertinent study characteristics were extracted, and relationships among these data were examined using nonparametric statistics.

Results

Eighty-three studies reporting 86 algorithms were included. The vast majority (70 of 86, 81%) reported at least some decrease in external performance compared with internal performance, with nearly half (42 of 86, 49%) reporting at least a modest decrease (≥0.05 on the unit scale) and nearly a quarter (21 of 86, 24%) reporting a substantial decrease (≥0.10 on the unit scale). No study characteristics were found to be associated with the difference between internal and external performance.

Conclusion

Among published external validation studies of DL algorithms for image-based radiologic diagnosis, the vast majority demonstrated diminished algorithm performance on the external dataset, with some reporting a substantial performance decrease.

Keywords: Meta-Analysis, Computer Applications–Detection/Diagnosis, Neural Networks, Computer Applications–General (Informatics), Epidemiology, Technology Assessment, Diagnosis, Informatics

Supplemental material is available for this article.

© RSNA, 2022

Summary

Published external validation studies of deep learning for radiologic diagnosis are infrequent, with the vast majority reporting diminished performance in the external dataset compared with the dataset used for algorithm development.

Key Points

• ■ Studies of deep learning algorithms for radiologic diagnosis infrequently include an external dataset, with our systematic review identifying 83 published studies that performed external validation over a 6-year period.

• ■ Nearly half of studies that performed external validation reported at least a modest decrease in external performance, with nearly a quarter reporting a substantial decrease.

Introduction

Deep learning (DL) algorithms, predominantly employing convolutional neural networks, have been associated with high diagnostic accuracy in a growing number of classification tasks in medical imaging (1–3). Compared with other machine learning methods, DL algorithms have several advantages. DL algorithms have performed with similar, if not higher, accuracy for classifying large imaging datasets (4–6) compared with conventional machine learning methods, such as support vector machines. Furthermore, DL algorithms do not require labor-intensive feature identification and extraction for data reduction. Reported accuracies of DL algorithms are beginning to match or even exceed those of radiologists (1,7,8).

As we consider the potential applications of DL algorithms to radiology practice, we must consider whether these research results are applicable to the general population. Clinical imaging research is particularly challenging to interpret because of selection bias and the reliance on retrospective data sources (9,10). In both clinical and data science research, data represent characteristics that have a certain distribution in the research population. Selection bias occurs when the distribution in the research population is unknowingly different from that of the general population.

In machine learning research, the issue of selection bias has been recognized under alternate names, such as “dataset shift” (11). Diagnostic imaging applications of machine learning algorithms are even more susceptible to this problem because their performance is entirely dependent on the original development data. If an algorithm's high diagnostic accuracy depends on hidden peculiarities in the development data with respect to patient population, clinical setting, imaging equipment, and distribution of imaging findings, then the algorithm may not perform well in a general, more diverse population (12,13).

Therefore, to assess real-world clinical efficacy, it is essential to know an algorithm's performance on an external dataset, one derived from a source that is different than the development data and not used in the algorithm’s training. While the importance of considering external validation in artificial intelligence research is increasingly recognized (14,15), it has been performed in relatively few published studies (16).

To gain a better estimation of the generalizability of DL algorithms for image-based radiologic diagnosis, we conducted a systematic review of studies of DL algorithms that employed an external dataset to perform external validation. We sought to obtain an estimate of the magnitude of performance differences on external datasets and to investigate whether basic study characteristics affect external validation results.

Materials and Methods

Literature Search

This study was a systematic review and was therefore exempt from review by our institutional review board. On May 1, 2021, we searched PubMed for studies published in the English language from January 1, 2015, through April 1, 2021, on DL algorithms for radiologic diagnosis from medical images, using the search phrase shown in Figure 1. We also reviewed the reference lists of relevant articles for eligible studies. We chose a starting date that was 2 years prior to the release of the National Institutes of Health ChestX-ray14 dataset (17) and the conclusion of the Radiological Society of North America Pneumonia Challenge (18). We assumed that studies published prior to these major events were highly unlikely to meet inclusion criteria.

Figure 1:

Diagram summarizing literature search and article selection.

Study Selection

We considered all studies that evaluated DL algorithms for performing diagnostic classification using radiologic images as direct input. We selected only studies that included external validation of the final algorithm using an external data source from a facility or institution different from that used to develop the algorithm.

Our review focused on the task of diagnostic classification to limit heterogeneity of the included studies. Therefore, we excluded studies that involved tasks other than patient-level diagnostic classification (for example, image segmentation, worklist triage). For a similar reason, we also excluded studies that involved nonimaging clinical features (for example, age, biomarkers, genomic data), methods other than DL for either feature extraction or classification (for example, support vector machines), and feature extraction requiring an expert reader (for example, radiomic data). We excluded animal or phantom studies, review articles, and clinical applications outside of radiology.

Three physicians with 19 (J.E.), 4 (B.M.), and 1 (A.C.Y.) years of experience in conducting systematic reviews in radiology independently assessed titles and abstracts to identify potentially relevant articles for inclusion. The full text of potentially relevant articles was reviewed to identify those meeting inclusion criteria, if necessary. Discrepancies between the reviewers were resolved by consensus.

Data Extraction

For each eligible study, one investigator extracted pertinent information from the full text, including classification task characteristics, labeling method, DL architecture, use of validation, dataset characteristics, performance results, and publication characteristics (Table 1). A second investigator reviewed the extracted data for accuracy, and discrepancies were resolved by consensus. A primary performance measure was identified for each study; in order of preference, we looked for area under the receiver operating characteristic curve (AUC), sensitivity and specificity together, or overall accuracy (proportion of cases correctly classified).

Table 1:

Main Extracted Data for Each Eligible Study

For each study, a representative performance difference was defined as the difference between the primary performance measures of the development and external data sources. Clinically conservative choices were made for studies that reported multiple performance measures, such as those involving multiple institutions. For these studies, the greatest absolute difference between development and external sources was chosen as a representative difference for purposes of categorization and statistical analysis. For studies that reported both sensitivity and specificity differences, the more negative difference was chosen as the representative difference for purposes of analysis. For studies involving multiple institutions for either the development or external datasets, size and disease prevalence were averaged for the purposes of analysis.

On the basis of our experience with receiver operating characteristic analysis, the performance differences between the development and external data sources were grouped for convenience. Performance differences were considered “substantial” if the difference was 0.10 or greater on a positive or negative unit scale, “modest” if less than 0.10 but greater than or equal to 0.05, or “little change” if less than 0.05.

Classification task difficulty was captured in two variables: conspicuity of image findings and composition of nondiseased cases. Conspicuity was classified as “major” (can be confidently diagnosed by imaging alone), “subtle” (diagnosis associated with uncertainty, usually requiring tissue sampling), or “imperceptible” (imaging not usually involved in diagnosis). The second variable, composition of nondiseased cases, indicated whether the “negative” cases were all normal or contained diagnoses other than the index diagnosis. As an indicator of reporting quality, we recorded whether each eligible study stated compliance with any published guideline, such as the Checklist for Artificial Intelligence in Medical Imaging (ie, CLAIM) (15).

Statistical Analysis

The main dependent variable was the representative performance difference between the development and external data sources, computed as external performance minus development performance. Relationships between the dependent variable and pertinent study characteristics were evaluated using various statistical tests, depending on variable type. Relationships between the dependent variable and binary categorical covariates, such as CT versus radiography, were explored with the Wilcoxon rank sum (Mann-Whitney U ) test. Relationships with dataset size or disease prevalence were explored with the Spearman rank correlation coefficient. The Wilcoxon signed rank test was used to compare paired covariates, such as development versus external dataset sizes. Statistical analysis was performed with the Stata package (version 17; StataCorp). A two-sided P value less than .05 was considered statistically significant.

Results

Search Results

A total of 6018 articles were screened, yielding 83 published articles that met inclusion criteria (2,19–100) (Fig 1). Three of the articles (2,31,68) each reported two major classification tasks being performed by separate algorithms. These three additional tasks were treated as separate studies in the subsequent analysis, resulting in a total of 86 studies.

Study Characteristics

Characteristics of the included studies are shown in Tables 2 and 3. The full table of extracted data is included in Table E1 (supplement). The chest was by far the most common body part imaged for algorithm categorization (41 of 86 studies, 48%), followed by the brain (14 of 86, 16%), bone (10 of 86, 12%), abdomen (seven of 86, 8%), breast (five of 86, 6%), and others (Table E2 [supplement]). Almost three-quarters of studies involved either radiography or CT as the imaging modality. Only three studies implemented prospective data collection for either the development or external dataset, with two of these studies involving diagnosis of COVID-19 (56,94) and one involving thyroid cancer diagnosis (19). The dataset size and disease prevalence varied widely (Table 3). The sizes of the external datasets were statistically significantly smaller than those of the development datasets (P < .001, signed rank test). Multiple convolutional neural network architecture types were represented in the included studies, with ResNet being the most common.

Table 2:

Characteristics of Included Studies

Table 3:

Comparison of Development and External Data Sources in Included Studies

DL Algorithm External Validation

The median representative performance difference between development and external data sources was –0.046, with a range of –0.60 to 0.13, and 81% (70 of 86) of studies reporting a negative difference (Fig 2). Forty-nine percent of studies (42 of 86) demonstrated at least modestly lower external performance, and 24% of studies (21 of 86) demonstrated substantially lower external performance (Table 4). A few studies reported higher performance with the external dataset than the one for development, including one study showing the AUC increased from 0.84 with the development test set to 0.97 with an external dataset (42).

Figure 2:

Plot of representative diagnostic performance difference between external and development datasets. The three most common imaging modalities and body parts are indicated. AUC = area under the receiver operating characteristic curve, BO = bone, BR = brain, CH = chest, XR = radiography.

Table 4:

Algorithm Performance in External Dataset Relative to Development Dataset

We found no evidence of relationships between the results of external validation and the study characteristics we examined, using the representative performance difference between the development and external data sources as the measure of external validation. The study characteristics included body part, modality, conspicuity of imaging findings (major vs subtle), composition of negative cases (normal vs other diagnoses), labeling method (direct vs natural language processing), institutional diversity (single vs multiple institutions), population size, disease prevalence, and presence of a validation step during algorithm development.

Study Quality

Only a small number of studies (11 of 86, 13%) stated adherence to a reporting quality guideline. Six used the Nature Research Reporting Summary, a nonspecific guideline for research (101); four used the Standards for Reporting of Diagnostic Accuracy Studies (ie, STARD) (102); and one used the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (ie, TRIPOD) (103).

Discussion

In this systematic review of external validation of DL for radiologic diagnosis, we found that 81% of studies demonstrated at least some diminished performance in external datasets, with nearly half (49%) of studies reporting at least a modest diminution and nearly a quarter (24%) showing a substantial diminution. Balancing accuracy in a study population with that in the general population is a challenge not unique to machine learning research. This issue, also known as generalizability, has long been recognized and studied in clinical trials (104). In clinical trials, assessing generalizability can be done through a number of methods, such as comparing study and target population characteristics and statistical modeling (105–108). However, applying such methods to DL studies is problematic for two main reasons. First, DL studies seldom provide enough demographic or clinical information about the development dataset to allow assessment for potential selection or other bias. Second, the “black box” nature of DL algorithms means that the most important diagnostic features are usually unknown, making it difficult to assess whether these features could be subject to selection or other bias (12).

Among many hundreds of published DL algorithms for radiologic diagnosis, our systematic review identified 83 published articles that reported algorithm performance on an external dataset. This finding corroborates a systematic review performed by Kim et al (16) that found that only 6% of artificial intelligence publications in medical imaging included external validation. Similarly, Yao et al (109) found that only 16 of 155 studies (10%) in their systematic review of DL applications in neuroradiology included external validation, and Nguyen et al (110) found that one in eight studies (13%) in their systematic review of machine learning algorithms distinguishing glioblastoma multiforme from primary central nervous system lymphoma were tested in an external dataset. Potential reasons for the limited number of external validation studies include the difficulty in obtaining an appropriate external dataset of medical images and lack of awareness of external validation's importance in establishing clinical value. These challenges may diminish as large public datasets become increasingly available and major journals begin supporting guidelines that highlight the importance of performing external validation (15,111,112).

DL algorithms derived from large datasets are expected to have greater generalizability, as larger datasets are more likely to include a broader feature distribution than smaller datasets. Indeed, prior studies of DL algorithms for nonclassification tasks in medical imaging found that larger, multi-institutional development datasets led to improved generalizability (113,114). In contrast, we did not find the size or number of institutions in the development dataset to have a statistically significant impact on external performance, suggesting that other factors may be involved.

An unexpected finding was that a few studies reported higher performance with the external dataset than the one for development. Such a result might be naively interpreted as evidence that some algorithms are highly generalizable, but such a conclusion should be questioned. Because a machine learning algorithm's “knowledge” is exclusively drawn from the development dataset, a generalizable algorithm is expected to have similar, if not slightly lower, external performance compared with internal development performance. Two potential causes of misleadingly high external performance should be considered. First, the external dataset might contain only images with heavily weighted features responsible for correct classification and not be representative of a realistic target population. Second, the image data might contain information about the diagnosis that is unrelated to the disease process, such as a radiography marker or “burned-in” text in the images. In machine learning, this unintentional information is known as data leakage (115) and is analogous to the epidemiologic concept of a confounding variable. Interpretability techniques such as image embedding and activation maps can help identify data leakage. In the study with the most dramatic external performance increase (42) in our review, the authors found that the external dataset, which was a publicly available breast US dataset, contained very straightforward examples and possibly only contained heavily weighted features.

Limitations

Our systematic review had several limitations. First and most evident was the heterogeneity of the reviewed studies, especially with respect to body part, imaging modality, disease of interest, diagnostic complexity, and performance measures. Heterogeneity in performance measures includes their inherent sources of variation, such as the dependence of sensitivity and specificity on the reader's interpretation threshold. It is reasonable to suspect additional, potentially substantial heterogeneity with respect to imaging equipment, technique, and protocols, as these details were almost always missing from the reviewed studies. Consequently, the overall heterogeneity of included studies precluded quantitative pooling of study results and limited the statistical power of any subgroup comparisons. It is also possible, however, that population and task heterogeneity among medical imaging applications of DL may not be as important as we envision, as many commonly used DL algorithms already originated from tasks outside of medical imaging.

Second, to limit heterogeneity, we focused on a specific type of machine learning and classification task, excluding major areas such as support vector machines, random forests, image segmentation, feature analysis, and image reconstruction. Therefore, our results do not necessarily apply to these other important areas of machine learning. Future systematic reviews should be dedicated to external validation of these algorithm types and radiologic applications.

Third, most of the reviewed studies were focused on technical development and provided little methodological information or clinical description about the datasets and participant populations that were involved, as evidenced by the infrequent use of reporting quality guidelines. Because of this serious limitation in the literature, we were unable to perform a systematic, meaningful assessment of the quality of the reviewed studies and their risk of bias using standardized reporting guidelines (15). The limited methodological and clinical information also reduces the chance of detecting confounding variables associated with dataset and population characteristics. Quality assessment tools like the widely used Quality Assessment of Diagnostic Accuracy Studies 2 (ie, QUADAS-2) (116) may be limited because they are validated for study results derived from a single population, unlike the population comparisons sought in our review. Last, we recognize that our systematic review was subject to potentially large publication bias, likely leading us to overestimate the summary performance of algorithms in external validation studies meeting our selection criteria.

Future Directions

The specific causes of diminished DL algorithm performance on external datasets are largely unknown. Questions remain about what features are actually important for correct diagnosis by machine learning algorithms (117–119), how these features may be biased in datasets, and how external validation is affected. A better understanding of these questions will be necessary before diagnostic machine learning systems achieve routine clinical radiology practice.

We found that substantial improvement is needed in published descriptions of populations from which DL datasets are derived. These improvements are necessary to allow meaningful assessment of study quality and generalizability.

Conclusion

In conclusion, our systematic review found that the vast majority of external validation studies demonstrated diminished algorithm performance on an external dataset, some reporting a substantial performance decrease. Our findings stress the importance of including an external dataset to evaluate the generalizability of DL algorithms, which would improve the quality of future DL studies.