Key concepts, common pitfalls, and best practices in artificial intelligence and machine learning: focus on radiomics

Abstract

Artificial intelligence (AI) and machine learning (ML) are increasingly used in radiology research to deal with large and complex imaging data sets. Nowadays, ML tools have become easily accessible to anyone. Such a low threshold to accessibility might lead to inappropriate usage and misinterpretation, without a clear intention. Therefore, ensuring methodological rigor is of paramount importance. Getting closer to the real-world clinical implementation of AI, a basic understanding of the main concepts should be a must for every radiology professional. In this respect, simplified explanations of the key concepts along with pitfalls and recommendations would be helpful for general radiology community to develop and improve their AI mindset. In this work, 22 key issues are reviewed within 3 categories: pre-modeling, modeling, and post-modeling. Firstly, the concept is shortly defined for each issue. Then, related common pitfalls and best practices are provided. Specifically, the issues included in this article are validity of the scientific question, unrepresentative samples, sample size, missing data, quality of reference standard, batch effect, reliability of features, feature scaling, multi-collinearity, class imbalance, data and target leakage, high-dimensional data, optimization, overfitting, generalization, performance metrics, clinical utility, comparison with conventional statistical and clinical methods, interpretability and explainability, randomness, transparent reporting, and sharing data.

Main points

• Medical images are complex and include huge amounts of minable data, leading to the rise of a new field called radiomics.

• High-dimensional data introduced by radiomics exceed not only human comprehension but also the capabilities of conventional statistical tools. In this respect, artificial intelligence (AI) seems to be an attractive alternative to handle such complex data sets.

• Low threshold of accessibility to AI and machine learning (ML) tools might lead to inappropriate usage and misinterpretation, without a clear intention to do so.

• Considering the recent advances of AI in the field of radiology, a basic understanding of the key issues of AI and ML should be a must for every radiology professional.

• To develop and improve the AI mindset of the radiology community, simplified explanations of the key concepts along with pitfalls and recommendations would be helpful.

Medical images are complex and include huge amounts of minable data.¹ This led to the rise of a new research field in medical imaging, namely, radiomics.^2,3 Radiomics simply aims to extract high-dimensional data from clinical images, to find clinically meaningful correlations and models.^2,3 However, complexity and high dimensionality introduced by radiomics exceed not only human comprehension but also the capabilities of traditional statistical tools. Artificial intelligence (AI) is now regarded as one of the attractive ways to analyze and make predictions on large and heterogeneous data sets as commonly seen with radiomic approaches.

As a subfield of AI, machine learning (ML) learns to automatically extract patterns from complex data to generate predictions on previously unseen instances, even without a theoretical model.⁴ AI has been increasingly used in radiology research over the years, as evidenced by the exponential increase of publications (Figure 1). Typical ML tasks in radiology can be like predicting whether a pathology or genomic feature is present or not,^5-7 determining treatment response status,^8,9 segmentation,^10,11 image quality enhancement,^12,13 contrast medium dose reduction,¹⁴ and radiation dose reduction.^15,16

Figure 1.

Count of publications on artificial intelligence and machine learning in the field of radiology, between 2000 and 2021. Search syntax was (“artificial intelligence” OR “machine learning”) AND (“radiology” OR “computed tomography” OR “CT” OR “magnetic resonance imaging” OR “MRI” OR “ultrasound” OR “radiography” OR “X-ray”) on PubMed/MEDLINE. The last search date was the 12th of December 2021.

Nowadays, ML tools have become so easily accessible to anyone that ensuring methodological rigor is of paramount importance, especially in medical applications. Such a low threshold to accessibility might lead to inappropriate usage and misinterpretation, without a clear intention to do so. Getting closer and closer than ever to the clinical implementation of AI, a basic understanding of the key methodological concepts should be a must for every professional in the field of radiology.

This article aims to provide a simplified methodological perspective to radiologists about the key concepts, common related pitfalls, and best practices of AI and ML. To do so, 22 important issues are reviewed within 3 consecutive stages: pre-modeling, modeling, and post-modeling (Table 1).

Table 1.

Key concepts covered in this review

Stages	Key concepts
Pre-modeling	Validity of scientific question
	Unrepresentative samples
	Sample size
	Missing data
	Quality of reference standard
	Batch effect
	Reliability of features
	Feature scaling
	Multi-collinearity
	Class imbalance
	Data and target leakage
High-dimensional data
Modeling	Optimization
	Overfitting
Generalization
Post-modeling	Performance metrics
	Clinical utility
	Comparison with conventional statistical and clinical methods
	Interpretability and explainability
	Randomness
	Transparent reporting
Sharing data

Issues related to pre-modeling stage

Validity of scientific question

Concept

As with any scientific work, the true value of AI largely depends on whether a valid scientific clinical question has been formulated.

Pitfalls

Disregarding the real-world representation as a study goal is a very common pitfall, ultimately, resulting in the total failure in clinical utility.

Best practices

Before beginning an AI project, the study’s objective and how it would solve the clinical or scientific questions at hand must be defined very clearly.¹⁷ Having too many eligibility criteria generally signals unrealistic projects that are far away from clinical practice and production at scale. An AI project must be clinically meaningful and actionable, that is, its predictions should induce some action by the physicians or patients.¹⁸ For instance, a project tries to predict whether a solitary brain mass belongs to lung or breast metastasis. All too often, the model developed works with excellent predictive performance, even with the independent validation data from another institution. Despite being attractive at first glance, the model created seems problematic from a clinical perspective. In contrast, to use such a model in real life, one must question whether the lesion is primary in the first place. Furthermore, one should also question why other types of metastatic lesions are not included. Undoubtedly, the project might have been much more valid and useful if the primary purpose was to discriminate between primary and secondary neoplasms and then the multi-class prediction of metastatic subtypes like breast versus lung versus others.

Unrepresentative samples

Concept

The actual potential of an AI model is not only dependent on the valid scientific question but also on whether one has proper data to answer that question. Data used for AI modeling and future predictions must be representative of real-life scenarios in terms of distribution (Figure 2). The concept applies to both training and test data.

Figure 2.

Simplified illustration of representativeness of data. The unrepresentative sample includes different distributions compared to the target population.

Pitfalls

Focusing on the most common types of instances (e.g., lesions, tumors) and disregarding the uncommon ones that might constitute a considerable proportion is a widespread modeling pitfall. This can also be defined as spectrum bias.^19,20 Such distribution differences or shifts are highly likely to cause significant performance problems in future predictions.

Best practices

Uncommon categories should not be ignored. Those can be grouped as a different category (e.g., others) to improve the representativeness of the data used. This issue is rather significant to detect particularly in the test dataset since main conclusions are usually based on the performance of the test data. Stratified sampling can be applied to split the data to preserve the distribution both in the training and test sets. In addition to simple summary statistics of clinical and demographic data, a comparison of the error patterns using the learning curve approach would simplify the identification of the unrepresentative sample problem.

Sample size

Concept

Data size problems affect medical imaging more than any other AI field. This is largely due to (i) the high-dimensional feature space of the medical images in that each pixel or voxel can represent a single feature and (ii) a very high number of hand-crafted radiomic features that can be extracted from original and derivative images.¹ Such high dimensionality necessitates enormous data sets for proper training to achieve a particular level of stable performance and very low generalizability error.^21-24

Pitfalls

Training a model with an extremely small data set is a common pitfall, leading to overfitting, noise, and outliers (Figure 3).

Figure 3.

Simplified illustration of the influence of data size on the performance of machine learning models.

Best practices

The optimal sample size for a particular AI algorithm applied to medical imaging data is often unknown. Nearly all methods for determining the optimal data size are empirical. A systematic literature search revealed the scarcity of research on data size calculation in ML applied to medical imaging (not only limited to radiologic and nuclear medicine images).²⁵ In this recent analysis, such determination processes relied on (i) model-based considerations^26,27 or (ii) generating predictive functions of model strength based on empirical testing at selected sample sizes, in other words, the learning curve-fitting approach.^28,29 For tabular data, another general recommendation is to have a data size that is more than 10 times the number of features^1,30 or that is with at least 50 samples.²⁵ Unfortunately, gathering large quantities of high-quality medical image data is difficult.^31,32 However, there are other ways to mitigate this problem: data augmentation (i.e., transforming original data like rotation, scaling, or generating images with generative adversarial networks, etc.),^33,34 transfer learning (i.e., fine-tuning or training previously pre-trained models),³⁵ and federated learning (i.e., developing models across institutions while protecting data privacy).^36-38 No matter which technique is used, complex methods like deep learning still require a significant amount of data. To solve this data size problem, some data-efficient neural network techniques are on the horizon such as capsule systems adding viewpoint variance³⁹ and vision transformers having self-attention mechanisms focusing on the most valuable components of the layers.^40-42

Missing data

Concept

Most data sets in the real world contain missing data. Missing data can be anything from images with artifacts, missing sequence or phase in multi-parametric evaluations, incomplete features, and lacking clinical information. Missing values are rarely encountered in radiomics because features are computed from images and patients without imaging studies are usually excluded.

Pitfalls

Excluding all samples with the missing data may cause bias in the results.⁴³

Best practices

Many ML algorithms expect complete data and fail to work when the data are incomplete. Having missing values is not necessarily a drawback, needing careful transformations. In addition to deletion methods, the missing data may require statistical imputation that can be simply defined as the replacement of missing data with new values. To impute, various methods have been developed such as mean/mode/median imputation, regression imputation, expectation–maximization, multiple imputation, and so on.^43,44 Another alternative would be to use ML algorithms that accept missing values or even better algorithms that automatically detect and deal with missing data. Some algorithms can use the missing value as a unique and different value when building predictive models, such as classification and regression trees. For instance, a well-known and very powerful algorithm called XGBoost takes into consideration of any missing data.⁴⁵

Quality of reference standard

Concept

In radiology, the reference standard (i.e., label) is generally an accepted test or a group of accepted criteria used in clinical practice. The reference standard may require subjective decisions, resulting in intra-rater and inter-rater inconsistencies. Quality and stability of reference standard should be ensured to achieve a precise evaluation of the proposed ML models and comparison with the other traditional tools.⁴⁶

Pitfalls

Mislabeling and selecting an unstable reference standard are major problems that need to be avoided in developing AI models.

Best practices

The reference standard that is more insensitive to varying conditions (e.g., the dependency of experience) should be selected if options exist. In this context, intra-rater and inter-rater stability can be studied. Nonetheless, it is still very common to encounter reliability concerns. To alleviate this issue, consensus evaluations (i.e., combining interpretations by more than one evaluator) and voting (i.e., selection based on majority of decisions) should be considered.

Batch effect

Concept

The batch effect can be defined as technical variations that confound the determination of valuable features from data (Figure 4).⁴⁷ In radiomics, this problem may be caused by the use of different scanners, varying acquisition parameters, and image processing factors.⁴⁸ Thus, some subgroups might be under-represented, while others are unnecessarily over-represented. This is highly likely to be a problem for multi-center projects. Nonetheless, it can be seen in single-center studies.

Figure 4.

Simplified illustration of the batch effect. Tumor X and tumor Y from 2 different centers are classified according to 2 features as principal components. Please note the change of feature values following batch effect removal. PC, principal component

Pitfalls

Disregarding possible batch effects might inflate cross-validation accuracy which is the standard protocol for classifier evaluation in the absence of independent validation data sets.

Best practices

In practice, the batch effect is inevitable and needs to be dealt with in certain data preparation steps. The voxel size, slice thickness, and convolution kernel are relevant sources of radiomics variability. Data wrangling may include strategies that aim to reduce batch effects. Various techniques have been investigated in radiomics,^48-52 of which, the ComBat strategy seems promising.^48,49 In simple terms, this method creates a batch-specific transformation to represent the whole radiomic data in a common space lacking center or batch effects caused by scanner models, acquisition protocols, and/or image processing settings.

Reliability of features

Concept

The use of reliable features is important for the reproducibility of models. In radiomics, such concerns can be encountered in image acquisition and reconstruction level (e.g., vendor-based or technical differences),^53,54 segmentation level (e.g., intra- and inter-rater segmentation differences, slice selection),⁵⁵ and computation level (e.g., different mathematical formulas of feature extraction tools, different techniques for normalization, discretization, and resampling) (Figure 5).⁵⁶

Figure 5.

Segmentation and labeling differences. (a) A low-grade glioma segmented by 2 different radiologists. (b) Image cropping or labeling for the same tumor with different sizes of boxes. Even slight differences in segmentation areas can result in significant changes in feature values or representations.

Pitfalls

Common pitfalls are lack of feature reliability assessment, use of whole data in reliability assessment, building AI models on non-robust features, and not being clear on the computational parameters of the features.

Best practices

Considering the variations in image acquisition and processing in radiomic features, it appears to be a difficult task to obtain constant and stable results. Therefore, a great effort should be exerted to minimize variations. The reliability of features can be assessed using several approaches such as intra- and inter-rater agreement analysis for the detection of segmentation differences,⁵⁷ repeatability analysis for automatic methods,⁵⁸ and reproducibility analysis with different image acquisition settings or even using phantom or simulation measurements.^54,59 Automated techniques, particularly deep learning-based methods such as U-Net,⁶⁰ might be a better option for segmentation purposes because it almost completely avoids rater’s variability. However, the generalizability of trained algorithms is currently a major limitation, and applying those algorithms to a different data set might result in complete failure. Regardless of the methodology, such reliability assessments should be performed on the training set without data leakage. For computational reliability, preprocessing steps (e.g., discretization, resampling, standardization, registration, filtering) must be transparently presented.

Feature scaling

Concept

Radiomic features come in widely different value ranges, influencing the performance of some AI algorithms. Feature scaling is the transformation of feature values to a standard range.

Pitfalls

Ignoring feature scaling may lead to overrepresentation or underrepresentation of some features. Another important pitfall is scaling features on the whole data set.

Best practices

Feature scaling matters for many of the algorithms such as support vector machines, k-nearest neighbors, and artificial neural networks. Most of these algorithms are based on the calculation of distance between data points. Feature scaling is also a key part of deep learning-based architectures.⁶¹ On the other hand, some other algorithms do not require feature scaling. Most of these are tree-based algorithms such as random forest and XGBoost. The procedure can be done in a couple of ways, for instance, standardization (i.e., z-score normalization), normalization (i.e., min-max normalization), or logarithmic transformation (Figure 6).⁶² AI practitioners should be cautious while performing feature scaling to avoid data leakage by separately performing the task for training and testing data.

Figure 6.

Simplified illustration of common feature scaling techniques: standardization and normalization.

Multi-collinearity

Concept

Multi-collinearity (i.e., collinearity) is a common statistical problem in which at least 2 variables are dependent upon each other such that one can be linearly predicted from the other with high accuracy.

Pitfalls

Creating models with highly correlated and clustered variables is a common problem in modeling. It is much more problematic in linear modeling.

Best practices

Since many radiomic features can be repetitively extracted using filtered and transformed images, leading to high correlation and clusters, radiomics-based modeling is likely to suffer from multi-collinearity.⁶³ It inflates the variance of the coefficients, particularly in linear models. Nevertheless, non-linear algorithms can also be affected.⁶⁴ Multi-collinearity might or might not influence the predictive performance of the ML models, rather it may affect the individual dependence on the features and interpretation of the final model. The most common method for detecting collinearity is bivariate correlation. However, bivariate correlation method disregards the collinearity of multiple variables together. In such circumstances, variance inflation factor and tolerance are well-known methods for removing multi-collinear features before modeling.⁶⁴

Class imbalance

Concept

When the number of instances in one target class is higher than the other class or classes, there exists a class imbalance (Figure 7).⁶⁵ Some ML algorithms are tended to vote for the class that constitutes the majority if a severe class imbalance is present.

Figure 7.

Simplified illustration of modeling with class balance and imbalance. Please note balancing the classes results in a new hyperplane and a different model. w, with.

Pitfalls

Class imbalance can lead to false impression of predictive performance due to bias toward one class. In medicine, another important pitfall is the balancing test set.

Best practices

In ideal conditions, ML algorithms perform better when the classes are equal in the training set.⁶⁶ However, this is not common in the medical context. To fix the class imbalance problem, data sets can be rebalanced by resampling strategies such as under-sampling or over-sampling.⁶⁷ Such sampling strategies are recommended only for the training data sets. An alternative approach might be optimization for detecting true-positive or false-negative instances depending on the clinical need, for instance, screening rather than diagnosis.⁶⁸

Data and target leakage

Concept

Data leakage corresponds to the transfer of data among training, validation, and testing data sets, due to incorrect split of the data (Figure 8). This concept can also be referred to as data snooping bias. Target leakage is a special type of data leakage, and it happens if a model is trained with a feature that will not be available in real unseen data (Figure 9).

Figure 8.

Simplified machine learning pipeline highlighting the recommended and biased ways of dimension reduction along with possible data leakage scenarios. It is usually better to make the data split at the beginning. When needing calculations to determine the optimal parameters for a preprocessing step (e.g., optimal bin-width for radiomic feature extraction), such decisions should be made on training data only and transferred to the other splits. The recommended way for dimension reduction is to perform it within cross-validation folds.

Figure 9.

Simplified illustration of target leakage. Using data unavailable at the prediction time results in inflated and in turn misleading results. w, with; ROC, receiver operating characteristic; TPR, true-positive rate; FPR, false-positive rate.

Pitfalls

Leakage is a simple but major pitfall that must be avoided.⁶⁹ Otherwise, it might lead to abnormally high performance of the model and significant generalizability issues.

Best practices

Data leakage can occur as early as the first steps of the pipeline. Therefore, the data split must be done at the beginning, before any transformation or processing of the images or data. Data snooping bias occurs when the test set directly or indirectly influences the training process. It should be double-checked whether there are no data from the same patient placed in training, validation, and test sets at the same time. If a model is extremely accurate, one should be suspicious of the results and further evaluate the pipeline for possible data and target leakage. For target leakage, features having a high correlation with the target can be determined with traditional statistical analysis.

High-dimensional data

Concept

High-dimensional data refers to having numerous features or very wide feature space. It forces the algorithms to learn several small feature details, causing overfitting and generalizability problems, particularly in situations in which the number of features is far greater than the number of instances,^70-73 which is a common scenario in radiomics.

Pitfalls

Reducing the dimension of whole data is a common pitfall, which needs to be done only in the training set (Figure 8). Also, reducing the dimension before the cross-validation is an incorrect application, leading to bias.⁷⁴

Best practices

Reducing the dimension of the data called feature selection or dimension reduction is a key step for producing valid and generalizable results. Several rules exist for defining optimal feature numbers for given sample size, but no true evidence exists in the literature. Several methods can be used for dimension reduction.⁷⁵ Algorithm-based feature selection methods, intra-reader and inter-reader reliability analysis, multi-collinearity analysis, clustering, principal component analysis, and independent component analysis are the most common dimension reduction techniques. Another very good option would be using certain ML algorithms that have inherent feature selection capabilities, for instance, XGBoost. All dimension reduction techniques should be performed in the training phase, without data leakage. It is also best to select features within cross-validation folds, not before cross-validation.⁷⁴

Issues related to modeling stage

Optimization

Concept

ML algorithms have hyperparameters that can be externally configured to optimize the model to the given data. The process of such configuration is called hyperparameter tuning or optimization. Optimization is a key step in modeling and has a significant impact on the model’s performance.

Pitfalls

The most common pitfall is not performing the optimization at all. Other common problems are the selection of wrong performance metrics to optimize and overfitting.

Best practices

Optimization of the hyperparameters is a key part of any modern ML pipeline. Avoiding this step might result in significant performance problems on predictions of unseen data. Relying on the default configuration of the hyperparameters might be inappropriate for the classification or regression problem at hand. Optimization can be done to increase the performance of certain performance metrics such as sensitivity, specificity, accuracy, or area under the receiver operating characteristic curve depending on the purpose of the study, for instance, the screening of disease. Overfitting in optimization can be avoided by cross-validation or regularization techniques. Random search, grid search, and Bayesian optimization are popular methods that might result in different results (Figure 10).^76,77

Figure 10.

Over-simplified illustration of different hyperparameter tuning or optimization techniques. In grid search, all the hyperparameters are searched within a designated range. In random search, randomly selected hyperparameter values are experimented with, which can be limited to a maximum number of experiments. The random search takes less time than the grid search. Both grid search and random search methods are completely uninformed by the previous tuning experiments, which means that previous tuning experiments are not considered in the current or future tuning experiments. Bayesian optimization is a completely different and model-based approach, taking into account the previous experiments to find the best set of hyperparameters.

Overfitting

Concept

Overfitting occurs when an algorithm learns the patterns of limited data too well and is unable to make accurate predictions on unseen and new examples (Figure 11a).⁷⁸ In other words, it memorizes certain individual patterns and noise related to each instance of the data set upon which the model is developed, making it inflexible to make predictions on real data. Most often, the algorithm is overfitted to the training data set, but overfitting can also occur on validation or test set in case one performs so many experiments on these data sets to find high-performing models. To better understand the concept of overfitting, one should be familiar with the concepts of bias and variance trade-off (Figure 11b).^79-80

Figure 11.

Illustration of the concepts related to overfitting and bias-variance trade-off. (a) Learning and memorizing nearly every detail of the data results in overfitting, leading to failure to fit future data points. (b) Bias and variance are 2 sources of error in modeling. Bias is the error that is evident when a complex problem is represented by a simple model due to erroneous assumptions. Variance is the error that appears when a model is sensitive to very small alterations or noise in the data, producing unrealistic patterns. There should be a trade-off between bias and variance to achieve a better generalization beyond the training data. When the variance is high and bias is low, overfitting becomes inevitable.

Pitfalls

Overfitting itself is a common pitfall and needs to be dealt with appropriately. The main reason for high overfitting in medical models is their failure to represent the real-world situations in the data sets due to high degree of heterogeneity of medical data, varying demographic and biologic features even in the same disease state, differences in disease prevalence, vendor-based variations, and so on.^81-84

Best practices

Overfitting is a multi-factorial issue and can be amended by following actions including using more training data to eliminate statistical bias, preventing information and target leakage, using resampling techniques, and regularization.⁸⁵ Among these, reducing the complexity by regularization is probably the most important step in data science. Before introducing the sophisticated techniques for dealing with model complexity, it is noteworthy to mention that starting with simple models such as linear regression, least absolute shrinkage and selection operator (LASSO), and decision trees is generally regarded as the best practice in data science.⁸⁶ Complex algorithms should only be spared only if the additional performance gain is required or relevant. Model complexity can be changed by adjusting the loss function (L1-L2 regularization, entropy regularization), sampling methods (data augmentation and cross-validation), and adjusting the training phase (dimension reduction or drop-out, adding noise, hyperparameter optimization, etc.). Of the regularization techniques, the most powerful strategies are loss function modifications and dimension reduction (feature selection and drop-out, etc.).

Generalization

Concept

Generalization in ML refers to the ability of the model to adapt to previously unseen and new data. Generalizability can be evaluated using internal or external test sets (Figure 12).

Figure 12.

Simplified illustration of optimal data split and generalizability assessment. Hyperparameter tuning needs a different split, namely, validation set (i.e., tuning set). Please note that it is much more valuable to perform testing both internally and externally in terms of generalizability assessment.

Pitfalls

Using only internal validation is highly likely to result in over-estimated generalization performance. Unrepresentativeness of the test data in terms of prevalence and distribution is a common pitfall.^19,87 Inappropriate sampling strategies and data leakage are other common problems. Tuning the hyperparameters on the training data without using a tuning or validation set is another source of generalizability issue.

Best practices

In proper AI-based modeling, the data should include 3 partitions: training set, validation set, and test set. These data partitions are partly or wholly obtained and used with different sampling and validation strategies such as k-fold cross-validation, leave-one-out, hold-out, nested cross-validation, and so on. For the simplified schematic explanations of these methods, the readers are kindly referred to the following references.^3,88,89 It is crucial to clearly understand the terms “validation” and “test.” A validation set is used to tune the hyperparameters of the algorithm. A test set is used to measure the generalization performance. The test set can be internal or external. AI models’ performance in the clinical environment might be different than that of the experiments done during development. Therefore, it is important and best to conduct a generalizability assessment with external data obtained directly in the target clinical environment.^{19,78,81,90,91} To further improve the generalizability of an AI model, additional training rounds and continuous learning capabilities might be incorporated into the model development stage using data from target hospitals and specific clinical environments where the end product will be used. For a true generalizability assessment, an independent test set must correctly represent the actual population of interest, for instance, in terms of disease prevalence and demographics.

Issues related to post-modeling stage

Performance metrics

Concept

Regardless of being a classification or a regression problem, the metrics are essential parts of performance evaluation in every ML pipeline. Appropriate selection of these metrics is important for proper performance evaluation. Due to several reasons such as randomness, data size, data perturbations, and class imbalance, the value of performance metrics is subject to considerable variability.

Pitfalls

Inappropriate selection of metrics and disregarding variability of metrics (i.e., single-point evaluation) are common pitfalls.

Best practices

Performance metrics should be selected based on the characteristics of the data in hand (e.g., class imbalance).^92-94 Particularly, in classification tasks, it is important to evaluate confusion matrices that might provide broader insights about the capabilities of the models both in terms of overall performance and class-wise performance. In case of class imbalance, metrics like accuracy might be misleading. On the other hand, there are better metrics that can be used if class imbalance exists such as balanced accuracy, no information rate, the Matthews correlation coefficient, F1 measure, area under the receiver operating characteristic curve, and area under the precision-recall curve. Particularly, if the positive class constitutes the minority class, precision and recall can be used. Otherwise, if the negative class is the majority or there is balance, then, area under the receiver operating characteristic curve might be a much more appropriate metric due to lack of bias to the classes. Performance assessments should represent the variability limits of the study. The confidence interval, standard deviation, and standard error are common indicators of performance variability and should be included in the evaluation process.

Clinical utility

Concept

High predictive performance does not necessarily indicate that an AI model can improve clinical outcomes or does have high clinical utility. It is critical to directly assess the efficacy and utility of AI on clinical outcomes, in addition to its performance.⁹⁵

Pitfalls

Assessment for clinical utility is often disregarded in ML-based classification tasks.

Best practices

The 2 most common tools to assess clinical utility are calibration statistics^96,97 and decision curve analysis.⁹⁸ Calibration statistics is the process of determining whether the predicted probability scores match the actual probability scores. Well-calibrated models are often much more useful than high-performing binary classification models, giving more insights to clinicians and patients about the probabilities (Figure 13). Decision curve analysis provides evidence about the net benefits of the models, considering both discriminatory predictive performance and calibration.⁹⁹

Figure 13.

Clinical utility assessment with calibration statistics. (a) Calibration curves: poorly calibrated (i) and well-calibrated (ii) models. (b) Example use case. From the clinical perspective, it is much more useful to get well-calibrated probability scores (e.g., predicted probability of having a renal mass is 91.2%) rather than binary or multi-class predictions (e.g., the patient probably has a renal mass).

Comparison with conventional statistical and clinical methods

Concept

Due to media hype, AI might lead to unrealistic expectations. It should be known that traditional methods can outperform ML.¹⁰⁰

Pitfalls

Evaluation of ML modeling without considering the traditional methods would not reflect the actual impact on real-world clinical practice. Disregarding the negative results is another problem while comparing the methods.

Best practices

Comparisons can be made with traditional statistical models such as logistic regression or widely accepted clinical tools like expert readings. Potential gains should be meticulously evaluated in terms of discriminatory performance and clinical utility. It is noteworthy to mention that negative results are also as valuable as the positive ones.¹⁰¹

Interpretability and explainability

Concept

ML algorithms are usually perceived as black boxes, that is, lacking interpretability and explainability for the process that takes place from input to output.¹⁰² Interpretability is the ability to understand the workings of the ML method. Explainability is a kind of transparency in the processes when the ML model decides. Adding interpretability and explainability to the ML might provide trust and enhanced control (Figure 14).

Figure 14.

Example of interpretability in a patient with renal cell cancer. (a), Input image. (b) The input image is improved with activation mapping, adding some degree of interpretability to the model. It might also be useful in getting the attention of human readers. (c) Shapley additive explanations (SHAP) plot example for explainability. It simply summarizes how low or high feature values impact the model performance.

Pitfalls

Particularly in healthcare, failure in achieving improved interpretability and explainability will limit the potential impact of the model created.

Best practices

Some ML algorithms have the inherent characteristics of interpretability and explainability, for instance, linear regression, logistic regression, decision trees, and generalized linear models. However, the use of these algorithms may not be the best solution to capture the actual complexity of real-world problems. As ML models grow in complexity, generating interpretable and explainable models becomes increasingly difficult. Neural networks or ensemble models are complex models and need further tools to achieve interpretability and explainability. These goals in ML can be approached in 2 broad ways: local (i.e., explaining based on individual features such as local interpretable model-agnostic explanations (LIME)) and global (i.e., explaining based on a certain collection of features such as partial dependence plots (PDP)). Common methods for interpretability and explainability are permutation feature importance,¹⁰³ PDP,¹⁰⁴ LIME,¹⁰⁵ Shapley additive explanations (SHAP),¹⁰⁶ and activation atlases.¹⁰⁷ Still, significant effort goes into creating new approaches to deal with this problem.

Randomness

Concept

In data science, the results are largely dependent on some random components such as the order of the instances, selection of samples or subsamples, initial weights that initialize the algorithms, and so on.

Pitfalls

If the randomness is not considered very well, it is highly likely to get different results even with the same data and settings, leading to uncertainty of the results (Figure 15).¹⁰⁸ Not considering the possible effects of the uncertainty brought by randomness in pre-modeling and modeling is a major problem in terms of the reproducibility of ML models.

Figure 15.

Over-simplified illustration for the influence of randomness in machine learning modeling. Unstable (a) and stable (b) performances using different random states. Exp, experiment; w, with.

Best practices

Fixing the random number generator’s seed before model development is a key step to achieving reproducible results. It should be a default part of each experiment from sampling to modeling. Because of the dependency of randomness, there will be a range of several possible models, not a single model. Moreover, simply changing a seed value to a different one can drastically change the performance of the models created.¹⁰⁹ Therefore, the performance should be evaluated within a range, not a single point value.

Transparent reporting

Concept

AI-based modeling is subject to considerable reproducibility and replicability issues.¹⁰⁸ Considering the heavy parameter burden in AI models, slight changes in values can result in complete failure. The quality and replicability of the studies can be improved and maintained by systematic and transparent reporting strategies.

Pitfalls

Incomplete or non-systematic reporting of study parameters is the most common problem. Commonly missed parts are preprocessing details (e.g., image resizing, cropping, standardization, feature extraction parameters, etc.), model training details, and demographic and clinical characteristics.¹¹⁰

Best practices

Using checklists or frameworks is the most important best practice in transparent reporting. Some well-known frameworks are Radiomics Quality Score,² checklist for artificial intelligence in medical imaging (CLAIM),¹⁷ transparent reporting of a multivariable prediction model for individual prognosis or diagnosis,¹¹¹ and prediction model risk of bias assessment tool.¹¹² Among them, CLAIM is specifically designed for medical imaging-related AI.¹⁷ There are also other detailed frameworks that can be used to improve the methodologic transparency of AI and ML-related works.^20,88

Sharing data

Concept

AI research might be almost useless unless it is reusable, reproducible, and replicable.¹⁰⁸ Therefore, sharing data in AI is important. Data sharing provides a basis for communication with a wider scientific community that can efficiently build upon the proposed project.

Pitfalls

No attempt to share data is a commonly occurring problem in AI studies.

Best practices

Data in AI can simply be grouped into pre-modeling (raw images, processed images, feature data, etc.) and post-modeling data (scripts, model files, etc.). There are several ways to share pre-modeling data, for instance, data pooling with anonymization and federated learning. AI teams should at least consider sharing post-modeling data such as scripts and resultant model files and using online repositories. These are important for better understanding, external validation, and improvement of the methodology.

Final Thoughts

AI and ML in medical imaging have several different methodological aspects, with their challenges and pitfalls. Even though it is not possible to cover all key concepts, common pitfalls, and best practices in a review article, the issues covered here will improve the AI mindset of the radiology community. As the community gains more awareness and experience on key methodological issues and related pitfalls, it will be easier to evaluate, compare, and select the most reasonable solutions for the radiologic problems in hand: ML versus traditional methods versus combination of techniques.