Radiomics Beyond the Hype: A Critical Evaluation Toward Oncologic Clinical Use

Abstract

Radiomics is a promising and fast-developing field within oncology that involves the mining of quantitative high-dimensional data from medical images. Radiomics has the potential to transform cancer management, whereby radiomics data can be used to aid early tumor characterization, prognosis, risk stratification, treatment planning, treatment response assessment, and surveillance. Nevertheless, certain challenges have delayed the clinical adoption and acceptability of radiomics in routine clinical practice. The objectives of this report are to (a) provide a perspective on the translational potential and potential impact of radiomics in oncology; (b) explore frequent challenges and mistakes in its derivation, encompassing study design, technical requirements, standardization, model reproducibility, transparency, data sharing, privacy concerns, quality control, as well as the complexity of multistep processes resulting in less radiologist-friendly interfaces; (c) discuss strategies to overcome these challenges and mistakes; and (d) propose measures to increase the clinical use and acceptability of radiomics, taking into account the different perspectives of patients, health care workers, and health care systems.

Keywords: Radiomics, Oncology, Cancer Management, Artificial Intelligence

© RSNA, 2024

Summary

To fully unlock the potential of radiomics as a clinical tool, improvements must be made to address multiple challenges, including challenges in data standardization, infrastructure support, and radiomics signature reproducibility, transparency, validation, usability, and trustworthiness.

Key Points

• ■ Radiomics can improve cancer imaging assessment by extracting quantitative data from routinely used medical images in the management of oncologic patients.

• ■ Unaddressed challenges in radiomics hinder its clinical utility, leading skeptics to doubt its usefulness due to pipeline limitations.

• ■ To maximize radiomics’ clinical potential, it is essential to enhance multidisciplinary integration and address concerns regarding data standardization, infrastructure support, reproducibility, transparency, validation, usability, and trustworthiness.

Introduction

Radiomics is a promising and fast-developing field in oncology that leverages advanced computational techniques to mine high-dimensional quantitative data from medical images. The premise of radiomics is that medical images hold valuable information regarding tumor biology, behavior, and pathophysiology that are not readily discernible through conventional radiologic visual assessment (1).

Given the ability of radiomics to provide high-dimensional quantitative data from medical images, it has the potential to transform cancer management, whereby radiomics data can be used to aid early tumor characterization, prognosis, risk stratification, treatment planning, response assessment, and surveillance (1). Correspondingly, since its introduction in the field of oncology not more than a decade ago, there has been an exponential growth in radiomics studies within oncology. Nevertheless, despite the substantial body of radiomics research that has been established in the literature to date, certain challenges have delayed the translation and acceptability of radiomics in routine clinical practice. To fully unlock the potential of radiomics as a clinical tool, improvements must yet be made to address these challenges, including data standardization, infrastructure support, and radiomics signature reproducibility, transparency, validation, usability, and trustworthiness (2,3).

In this report, our objectives are to (a) provide our perspective on the translational potential and potential impact of radiomics in oncology; (b) explore frequent challenges and mistakes in its derivation, encompassing study design, technical requirements, standardization, model reproducibility, transparency, data sharing, privacy concerns, quality control, as well as the complexity of multistep processes resulting in less radiologist-friendly interfaces; (c) discuss strategies to overcome these challenges and mistakes; and (d) propose measures to increase the clinical use and acceptability of radiomics, taking into account the different perspectives of patients, health care workers, and health care systems.

Radiomics and Oncology

Imaging remains integral in patients with cancer, providing critical information to oncologists, surgeons, and other members of the multidisciplinary patient management team who must discern the most appropriate treatment road map for each patient. However, there is a continuing shortage and unequal distribution of radiologists and other members of the oncology workforce throughout the world (4), even as the global cancer burden is projected to surge by 47% between 2020 and 2040, surpassing 28 million cases worldwide (5). This highlights a compelling need for innovative approaches not only to maximize the information provided by imaging but also to address the clinical and economic challenges posed by the growing cancer burden worldwide.

Radiomic features can be divided into two broad categories: (a) engineered or handcrafted features and (b) deep learning–derived features. The former refers to intensity, shape, and texture-related information that can be captured after applying predefined mathematical transformations at a pixel or voxel level, constituting traditional, manually crafted elements. On the other hand, the latter refers to advanced computational techniques and features that are learned in an end-to-end manner, mostly using convolutional neural networks. Throughout this report, we focus on the engineered features. Radiomics holds the potential to augment what is currently provided by imaging as well as to provide a critical advantage in the setting of specialist shortages and rising cancer burden. The clinical translation potential of radiomics in oncology spans the entire patient journey, including early tumor detection (6), tumor characterization and subtyping (7), genetic profiling (8), prognostic assessment (9), risk stratification (10), assessment of response to treatment (11), assessment of treatment toxicity (12), surveillance (13), and assessment of survival outcomes (14). Fundamentally, radiomics can provide diagnostic or prognostic markers of clinically relevant oncologic outcomes, either alone or in combination with clinical data, laboratory tests, semantic radiologic features, and histologic and/or genetic findings. Ultimately, its goal is to function as a clinical decision support tool, providing the multidisciplinary teams with novel information for personalized and cost-effective treatment decisions.

Since radiomics was introduced to oncology in 2012, there has been almost 8000 studies evaluating the use of radiomics in oncology that have been indexed by PubMed up to December 6, 2023, as determined through a search employing the search terms: (“radiomics”) AND (“cancer” OR “tumor” OR “oncology”). Nevertheless, only a limited number of radiomics signatures have undergone rigorous external testing (15–25), and none have gained routine adoption in clinical practice. Clinical trials are essential for advancing medical knowledge, testing innovations, and improving patient outcomes through rigorous methodology and standardized data collection, ultimately leading to better treatment strategies. However, only a small number of radiomics studies have been conducted within a clinical trial setting, with fewer than 20 published studies incorporating clinical trial data (26–41). Notably, none of the published radiomics studies have prospectively implemented radiomics as a clinical decision support tool.

Radiomics Workflow, Challenges, Common Mistakes, and Strategies to Overcome Them

Radiomics is a multistep process which involves several domains of expertise and elements. Given its interdisciplinary nature, the integration of these diverse elements can be prone to challenges and mistakes, which can be grouped into three main layers: (a) clinical relevance, (b) radiomics pipeline, and (c) publication bias. Figure 1 summarizes an ideal radiomics workflow from conception to clinical implementation.

Figure 1:

The figure summarizes an ideal radiomics workflow from conception to clinical implementation. IRB = institutional review board.

Clinical Relevance

Rationale: Similar to any other clinical research, the primary goal of radiomics research is to develop a model or signature to address unmet clinical needs and positively impact patient management. Defining the clinical relevance of a research study is the foundation not only in radiomics research, but also in all domains of medical research to ensure meaningful, impactful, and ethical research.

Mistakes: Based on our research experience, we believe that a frequent error in delineating the clinical relevance of a research study is the absence of integration and communication among various members of the multidisciplinary team during the initial stages of the study. Operating in isolation without a defined end point for clinical relevance and application, in addition to lacking potential benefits for patients and the health care community, stands out as a common and consequential mistake.

Strategies to overcome mistakes: To overcome this challenge, multidisciplinary stakeholders must be engaged at the project’s outset and throughout the research process. This ensures a comprehensive understanding of clinical needs and real-world applications. Additionally, defining clear clinical utility (intended use) and potential benefits for patients and the community helps to align and prioritize research efforts. Finally, regular meetings and discussion among the team members helps to keep the group focused and facilitate dynamic exchanges and problem solving.

Another factor worth reflecting on is the likelihood of success. Although radiomics is a data-driven exercise, linking the radiomics findings across the spatial scale to a cellular or molecular event may be more challenging than providing radiomics quantification to a radiologic observation that can be discerned by the human eye or that is underpinned by a known biologic or pathologic link between the imaging and the outcome. Indeed, although radiomics signatures have been developed for molecular or genomic expression, none have been translated into a clinical tool or used in prospective studies. This reflects the challenges of filtering the relevant imaging signal from noise in the data when crossing spatial scales, which would impact the stability, reliability, and reproducibility of the observations.

Radiomics Pipeline

Pre-execution.—Rationale: This step is required before conducting a study, primarily for study planning, assessing feasibility (sometimes necessitating pilot data gathering to determine the sample size), and evaluating the availability of high-quality data for model development and validation, including external testing. Typically, this preliminary phase involves establishing the research team, designing the study, formulating hypotheses, setting study objectives, specifying inclusion and exclusion criteria, and estimating the expected sample size. It also entails deciding on the collection of relevant nonradiomics data (such as clinical, laboratory, and semantic quantitative data) and selecting the imaging modality to be evaluated. Moreover, it includes determining whether one or multiple timelines (eg, delta radiomics) will be incorporated (1,42). Finally, it includes seeking approval from the local ethics board, and if a multicentric study is planned, establishing data sharing agreements.

It is also crucial to carefully design the strategy for radiomics research. This involves constructing a causal framework by creating a direct acyclic graph and considering the development of the radiomics signature within this framework. Since radiomics relies solely on the collection of historical observational data that may be subject to biases, the modeling strategy plays a vital role in mitigating these biases and ensuring validity of the results.

Mistakes: Once again, the lack of cohesion among stakeholders can lead to misalignment in the overall study design and objectives. For instance, the absence of a comparison or an association between the radiomics signature and established imaging or clinical assessment tools can diminish its added clinical value. Additionally, the inclusion of a small sample or low-quality data can substantially weaken the reliability and quality of the radiomics signature being developed. As with other types of studies, data quality is intricately linked to study outcomes, such that the data integrated into the model affects the accuracy of the reference standard end point. Finally, it is a crucial mistake to overlook potential input bias which may be associated with sample selection, data gaps, imbalanced classes, socioeconomic disparities, imaging protocols and scanners as it can introduce bias into the output model and reduce generalizability of the results.

Strategies to overcome mistakes: It is a complex task to overcome the possible mistakes in this first step of the radiomics pipeline. This may require not only effort from the research team but also the institution and potentially societies and professional organizations.

From the research team, the following strategies should be considered: (a) adhere to radiomics specific checklists, such as CheckList for EvaluAtion of Radiomics research (CLEAR) (43), radiomics quality score (RQS) (44), and METhodological RadiomICs Score (METRICS), which was recently developed by the European Society of Medical Imaging Informatics with a large international expert panel aiming at a transparent methodology from handcrafted radiomics to deep learning workflows (45) from the conceptualization phase to ensure comprehensive completion of the checklist; (b) increase integration at the inception of the research; (c) consider sample size and the need for multicentric collaborations; (d) implement data quality control to ensure high-quality input; and (e) actively identify and mitigate possible sources of bias. Regarding an initial sample size estimation, it is known that when performing feature reduction in radiomics studies, ensuring an adequate sample size for model validation is paramount. Typically, it is recommended to have around one-third of the training sample size for validation purposes. However, this requirement can vary depending on the balance in the dataset (eg, the number of “positive” or “negative” events in the dataset). In cases where the data are highly unbalanced, a larger sample size may be necessary to maintain model robustness and reliability. For instance, consider a scenario where a 10-feature radiomics model is being developed. To adhere to the one-third validation criteria, a minimum of 133 samples would be required, with 100 allocated for training and 33 for validation. Adequate sample sizes and careful consideration of data balance are crucial steps in ensuring the accuracy and potential generalizability of radiomics models (1,46).

In scenarios where dealing with imbalanced data are unavoidable, imbalanced data represent a potential source of bias. Acknowledging this challenge, the research team can employ various strategies to address it effectively. For instance, data preprocessing techniques, such as oversampling the minority class or undersampling the majority class, can be applied. Additionally, the extraction of relevant radiomics features that provide information for both classes is crucial. The choice of algorithms can be also helpful, and opting for those robust to imbalanced data, such as ensemble methods like random forest or those with cost-sensitive learning capabilities, is recommended. Use of cross-validation strategies, including k-fold, becomes essential to maintain the unbalanced distribution across different folds during both training and testing phases. Finally, continuous monitoring is also emphasized to ensure ongoing awareness and mitigation of any potential degradation in model performance.

Facilitating external testing across institutions and societies requires a thoughtful approach. Several key strategies can be considered to foster collaboration and ensure the reliability of research findings. First, promoting collaborative expertise among experts and encouraging the formation of multidisciplinary and multicentric teams can enhance the collective knowledge base. Second, streamlining research processes and ethical approvals, alongside implementing secure data-sharing protocols that prioritize privacy measures, proves instrumental in facilitating external testing and supporting multicentric collaboration. Last, strengthening data management infrastructure is paramount. Allocating dedicated resources ensures the efficient handling of large datasets while also promoting seamless integration with existing clinical workflows and tools. These strategies collectively contribute to the establishment of a robust framework for external testing, fostering trust and collaboration.

Data collection.—Rationale: High-quality imaging and clinical data can effectively reflect cancer behavior and heterogeneity. This forms the basis for robust radiomics research, as the quality of the collected data directly influences the reliability and relevance of subsequent analyses.

Mistakes: Mistakes often stem from the absence of a streamlined data collection workflow, from data curation performed by individuals not specialized in the field and from heterogeneous imaging acquisition parameters. Image acquisition parameters greatly influence various stages in the radiomics workflow, from feature extraction to model building, thereby impacting the robustness and reproducibility of radiomic analyses. It is essential to consider these parameters during the data collection step to ensure that imaging parameters will be collected to be taken into consideration during model building and interpretability of the results. Conversely, depending on the radiomics signature and sample size, heterogeneity in the collected data can also present opportunities to enhance the generalizability of the radiomics signature being developed.

Strategies to overcome mistakes: Well-defined and optimized workflows that outline clear data collection processes, standardized clinical and radiologic reports, and standardized imaging protocols may be implemented to overcome mistakes pertaining to data collection. Notably, standardization across the radiomics pipeline has been recommended in different published guidelines, but there is no universal agreement as to how this should be achieved.

In addition, most radiomics signatures in published studies have been developed based on a “minimum” or limited dataset, which makes real-world clinical translation challenging when exposed to previously unseen data. Hence, data quantity and data diversity need to be addressed toward the creation of meaningful radiomics signatures that can be more widely generalized. The need for multicentric research and data curation at scale as well as the creation of well-curated imaging databases cannot be overemphasized. Large-scale data curation projects (eg, European Cancer Imaging Initiative) and existing cancer imaging databases (eg, The Cancer Imaging Archive) will play an increasingly important role in hosting relevant imaging data to drive radiomics research and clinical translation. Finally, engaging all stakeholders in data collection and instituting robust quality control programs are integral to the data collection process.

Imaging segmentation and preprocessing.—Rationale: The goal of image segmentation is to include a representative two-dimensional area (ie, region of interest) or three-dimensional volume (ie, volume of interest) of the tumor, peritumoral region, or a tumor subregion, also known as a tumor habitat. Image segmentation can be performed using manual, semiautomatic, or automatic methods. Meanwhile, image preprocessing (such as filtering, normalization, and resampling) helps to enhance image quality and may simultaneously improve the standardization and reproducibility of radiomics features across scanners and imaging techniques. Accurate segmentation is extremely important before performing radiomics analysis to include both reproducible and representative areas of interest.

Combat batch effect adjustment (ComBat) is a statistical method initially designed to remove batch effects from microarray data. However, it has also been applied in radiomics research for image normalization and harmonization. ComBat uses an empirical Bayes model to adjust for batch effects by modeling the relationship between batches and gene expression levels. In the context of radiomics, ComBat can be used to adjust for acquisition parameters that may introduce systematic differences between datasets, allowing for more accurate comparisons and integrations of data from multiple sources or scanners. The application of ComBat in the radiomics workflow typically occurs during preprocessing or normalization stages before feature extraction and selection. By removing batch effects early on, subsequent steps in the pipeline, such as feature selection and machine learning model development, can benefit from more accurate and comparable data (47,48). It is essential to note that harmonization techniques like ComBat should be applied judiciously and only when necessary as they may introduce additional computational complexity and potential biases if not properly validated or used appropriately. Finally, it is our opinion that ComBat is not universally accepted due to its limitations with unseen data representations.

Mistakes: The most frequent shortcomings with image segmentation are related to segmentation errors, inter- and intrareader variability, and insufficient integration with the clinical workflow or lack of functionality of the software used for image segmentation. Segmentation often requires analyzing multiple scans and employing techniques like comparing current scans with baseline images and using multiplanar reconstructions for precise delineation. When the segmentation software does not have full functionality like the picture archiving and communication system or when segmentation is performed without the availability of other image series on picture archiving and communication system workstations, it can hamper accuracy. Meanwhile, image preprocessing, although often necessary to harmonize data from different sources, can mask relevant radiomics signatures. Image preprocessing encompasses various steps, including spatial resampling and intensity discretization which, when applied without careful consideration, may inadvertently alter the inherent characteristics of the radiomics signatures. Spatial resampling, for instance, involves altering the voxel size, potentially leading to information loss or distortion. Similarly, intensity discretization, the process of categorizing pixel values into discrete bins, might oversimplify the intricate patterns present in the data. It is crucial to recognize these nuances and exercise caution during preprocessing to avoid unintended consequences that could compromise the authenticity and interpretability of radiomics signatures.

Strategies to overcome mistakes: To avoid these mistakes, presegmentation training sessions should be conducted to familiarize operators with the process and avoid pitfalls. The development and incorporation of automatic segmentation algorithms, when possible, can substantially reduce human fatigue and errors. Additionally, the implementation of stringent quality control measures as an integral part of the workflow can help ensure data reliability. In the context of heterogeneous tumors characterized by diverse components, different studies take different approaches to segment tumors. Some studies opt to exclude internal structures such as necrosis, calcification, and vessels from the region of interest or volume of interest, while others encompass these elements within the tumor segmentation. This variability underscores the need for a more standardized approach to tissue segmentation in every radiomics project. It is crucial to recognize that even the choice of segmentation tools can substantially impact the accuracy and reproducibility of radiomics results. For instance, a study by Owens et al (49) compared two segmentation approaches and revealed that radiomics features extracted from lesion sizing toolkit contours exhibited greater reliability across and among observers. The study further demonstrated that semiautomatic segmentation tools enabled observers without formal clinical training to achieve comparable results to physicians in evaluating tumor segmentation (49).

Regarding possible obscuration of relevant radiomics features during preprocessing, a meticulous and well-documented approach is essential. The following strategies can be considered: (a) clearly define and document each preprocessing step, (b) rigorously validate these techniques on representative datasets, (c) regularly review preprocessed data against original images, and (d) engage domain experts in decision-making.

Feature extraction, feature selection, and model building.—Rationale: The goal of this step is to define the radiomics features that are both substantial and reproducible, with a focus on prioritizing those that are informative, stable, and nonredundant while also assessing their associations with biologic aspects of the tumor region or with outcome variables and to build a clinically meaningful and robust model.

Mistakes: Several mistakes can occur in this step, the most common ones being the following: (a) a lack of adherence to standardization, as recommended by the recently updated Image Biomarker Standardization Initiative (50), which can lead to inconsistencies in feature engineering and hinder comparison across studies and datasets; (b) feature extraction errors, such as selecting inappropriate algorithms, which can incur bias and diminish the quality of data representation; (c) cherry-picking features, which can lead to overfitting of the model; (d) multiple testing without proper correction, which can hinder the identification of potential false patterns that are due to chance rather than tumor characteristics; (e) lack of transparency in documenting the model-building process, including data preprocessing and algorithm selection, which can hinder model interpretability and reproducibility; and (f) insufficient small sample sizes, which can lead to overoptimistic model performance and limited generalizability.

Strategies to overcome mistakes: Well-suited feature extraction algorithms and robust statistical methods to strike the right balance between information retention and noise reduction should be employed. Feature selection methods should be implemented in a way that they do not expose information from the hold-out test set to the model and consequently lead to overoptimistic performance estimates. Rigorous and systematic approaches to model building include having a proper sample size, employing rigorous feature selection, implementing statistical correction for multiple tests, ensuring careful data handling to prevent leakage, and maintaining transparency throughout the process to ensure the model’s reliability and usability.

In the context of small datasets, using nested cross-validation methods for feature selection can help mitigate bias and select features in an unbiased manner. Nested cross-validation involves an outer loop to evaluate the model’s performance and an inner loop to select features within each outer fold. This approach can provide more reliable estimates of model performance and help prevent overfitting.

Notably, traditional wisdom in feature selection prioritizes features based on the level of statistical significance in their association with the outcome variable. This can result in the inclusion of features that lack explainability. Shape-based first-order features, and to some extent texture features (such as gray-level co-occurrence matrix [GLCM], gray-level run-length matrix [GLRLM], gray-level size-zone matrix [GLSZM], and neighboring gray-tone difference matrix [NGTDM]), can be more explainable compared with higher-order features (such as fractal dimension and Laplace of Gaussian) and may add confidence to the deployment and use of radiomics. While understanding the extraction processes and interpretability may not be essential for ensuring clinically useful and accurate models, prioritizing explainability in predictive modeling can substantially enhance trust, reliability, and ultimately the effectiveness of clinical translation and engagement. Moreover, legal considerations further underscore the importance of explainability in clinical contexts.

In the setting of large numbers of features, applying corrections for multiple testing, such as Bonferroni correction or false discovery rate correction, becomes crucial to mitigate the risk of spurious findings or false positives. While it may seem intuitive to perform corrections before feature selection to address this issue comprehensively, practical considerations may limit this approach’s feasibility. The challenge lies in striking a balance between minimizing false positives and maintaining the power to detect relevant features. Therefore, a judicious application of correction methods, considering the specific stage of analysis and its computational demands, is essential for robust and reliable radiomics investigations.

Additionally, when working with a large number of texture features extracted from medical images, it is crucial to reduce their dimensionality to enhance model performance and prevent overfitting. To select the most important features, criteria such as relevance, redundancy, and generalization ability are among the most important to be considered. Relevance refers to how well each feature correlates with the target variable one aims to predict. Redundancy helps eliminate duplicate features that provide similar information. For example, strongly correlated features are likely redundant. Although analysis of variance (ANOVA) is a commonly used feature selection method, its application in radiomics comes with major limitations due to the high correlation between input variables. This issue relates to the fact that ANOVA is insensitive to such correlations, leading to suboptimal results. Therefore, it is recommended to consider using alternative methods like least absolute shrinkage and selection operator (LASSO), minimum redundancy maximum relevance (mRMR), recursive feature elimination (RFE), and Boruta for feature selection before applying ANOVA. These methods are specifically designed to handle highly correlated radiomics features effectively and may provide more robust radiomics signatures. Additionally, they can be used in combination with ANOVA or other statistical methods to improve the overall performance of feature selection and ensure the most accurate and effective results for clinical applications. Features with consistent relevance across datasets have greater generalization ability. Regularized regression techniques like LASSO that penalize less useful features are commonly used in radiomics to automate feature selection based on relevance and redundancy criteria. LASSO is among the most popular automated methods. Ensemble approaches that aggregate rankings from multiple selection algorithms also enhance robustness. The dimensionality reduction goals guide the process, whether optimizing for model accuracy, efficiency, or other aims. There is no definitive rule for deciding the ideal number of features, but based on empirical evidence, it is generally advisable to have one feature for every 10 to 15 training samples (51).

Finally, in case one class is far more prevalent than the others, a common mistake is to use accuracy as an evaluation metric of the model’s performance. Accuracy is defined as the proportion of correctly classified samples divided by the total number of samples. In an imbalanced dataset, a dummy classifier can achieve high accuracy by simply predicting the majority class. In such situations, the F1 score is a better choice. It is defined as the harmonic mean of precision and recall, where precision is the ratio of true positives to the sum of true positives and false positives, and recall is the ratio of true positives to the sum of true positives and false negatives. The F1 score takes into consideration both false positives and false negatives, making it less affected by class imbalance. It is a more robust measure of a model’s capacity to classify instances in an imbalanced dataset correctly (52).

Validation.—Rationale: The goal of this step to validate generalizable radiomics signatures to predict the desired end points.

Mistakes: Improper data management, such as data leakage of information from the test set, influences the training set. Model validation should not be static given the various drifts that may happen at the image, target, or concept level.

Strategies to overcome mistakes: External test sets should be implemented to validate built models as they will reduce bias and lead to potentially more generalizable radiomics signatures. Assessing the temporal performance of a model through use of consecutive prospective patients for a period of time can give indications of model’s robustness to external changes (ie, before and after a “silent” update in the scanner software). Ideally, models should be deployed in a machine learning operations (MLOps) platform to be able to monitor their performance and check for input data drifts that may trigger retraining sessions of the model to keep it up to date. Model aging is a known effect that may result in gradual increases in model errors and variability, posing a potential risk for harming patients through erroneous predictions if left unattended.

One important limitation in the field of radiomics, as highlighted in recent studies (53), is the variability of radiomics features across different datasets. These features often demonstrate high performance only when applied to data similar to that on which they were originally trained. This limitation impacts the generalizability and reproducibility of radiomics signatures, making universal clinical application challenging. Beyond patient demographics, variations in data representations can stem from less apparent factors, including patient socioeconomic status, medical imaging vendors, model types, software releases, acquisition protocols, and specific local guidelines for patient selection for medical imaging examinations. This complexity underscores the need for standardization with image acquisition and feature extraction processes. One approach to mitigate these challenges is to retrain or fine-tune existing models using local data, creating model versions tailored to each hospital setting. This approach, however, poses substantial regulatory challenges. Additionally, the development of collaborative models trained on multi-institutional datasets may promote the creation of more robust and universally applicable radiomics signatures.

Publication Bias

Publication bias favors the publication of studies with positive outcomes over those reporting negative results, contributing to an overly optimistic view of radiomics that can have a detrimental impact on its reproducibility and hinder clinical acceptance. Moreover, numerous published studies have neglected to provide comprehensive details regarding imaging preprocessing and feature extraction methods, including aspects like voxel size resampling or bin width settings. This lack of transparency can impede the reproducibility of results. Addressing publication bias demands not only the encouragement of publishing negative or inconclusive findings but also advocating for the use of radiomics-specific checklists and quality scores to enhance the overall transparency and rigor of research methodologies.

Unmet Needs in Radiomics Research and Implementation

Despite more than 7000 published articles in radiomics research, there has been poor clinical translation of radiomics into patient care. This has led some skeptics to proclaim that radiomics is unlikely to be clinically useful due to the current limitations encountered across the entire radiomics pipeline. Optimists and supporters, however, hold firm that more needs to be done to create radiomics signatures that are more stable, repeatable, reproducible, interpretable, explainable, and generalizable, which can then provide unique insights into meaningful clinical outcomes. Perhaps the middle ground is to explore and address the current unmet needs in radiomics development that are hindering the progress of radiomics in clinical translation.

Radiomics is data-driven research, and the paucity of high-quality well-annotated data are limiting the extent and depth of data exploration. Most radiomics signatures are built on limited datasets of under 1000 data points, which is a small number compared with the hundreds of radiomics features that are typically generated as part of the workflow. Despite our best efforts using current standardization and cross-validation techniques, there remains uncertainty in the feature selection process. Hence, collating larger data volume by the creation or linkage of image databases and repositories will be an important step toward the availability of data for radiomics research. Furthermore, as discussed, radiomics features often lack explainability, which can also hinder their clinical adoption. While interpretability may not be critical for ensuring clinically useful and accurate models, prioritizing explainable models can greatly enhance trust, reliability, and, ultimately, the effectiveness of clinical translation and engagement.

Part of the reason for limited data availability is because image segmentation and annotation is a time-consuming task that is reliant on trained radiologists and/or technicians to undertake. As the radiomics pipeline is not within the mainstream clinical workflow, these tasks are usually performed retrospectively. With the current workforce crisis in radiology, the ability to create data annotations prospectively is limited. Hence, artificial intelligence–assisted image annotation and segmentation holds the key to the automatic prospective creation of useful large datasets that can be fed into the radiomics workflow.

Although much has been discussed regarding standardization of the radiomics workflow, there is no universal agreement between published guidelines as to how this should be achieved. Due to the wide heterogeneity of protocols across vendor system and hospitals, standardization of scanning parameters is only pragmatic within a trial setting or across a few centers. However, the majority of our imaging data remain from the real world and are retrospective, which cannot be subject to such rigorous standardization. Hence, large datasets remain paramount in the search for meaningful radiomics signatures in heterogeneous datasets across different scanner systems. Recognizing the anticipated diversity in these datasets, the adoption of postextraction techniques, such as harmonization, emerges as a valuable strategy to improve model performance in the context of real-world cohorts. Even if we allow for the creation of a meaningful radiomics signature from clinical trial data, the validity of such a signature over time is uncertain as machine upgrades and performance drifts may affect the reliability of the signature.

The relevance of the radiomics use case cannot be overstated, as many published studies are of low-to-modest quality and may not address useful clinical questions. In the search for a relevant and widely generalizable radiomics signature, initial efforts may focus on an explainable signature where the outcome is more tightly linked to the imaging phenotype rather than across spatial scales (eg, gene expression). The latter may suffer from poorer signal-to-noise in the imaging data and potentially lead to spurious results without also considering all the potential causal inferences.

Undertaking the full radiomics pipeline and data analyses requires skills that are beyond the training of most radiologists. Hence, radiologists should be prepared and educated in those elements of the workflow that go beyond defining the research question, identifying the study population, performing image segmentation and annotation, and contributing to the interpretation of the results. The radiology department should recognize the investments needed to be ready for artificial intelligence and imaging data science. In particular, radiologists should be aware of the skill sets and new personnel required for future radiology departments, as well as the informatics infrastructure needed to operationalize a radiomics and artificial intelligence workflow. Figure 2 summarizes the most common mistakes within the radiomics workflow and suggests strategies to overcome them.

Figure 2:

Summary of the most common mistakes within the radiomics workflow and suggested strategies to overcome them. PACS = picture archiving and communication system.

Radiomics: From Promises to Clinical Application

Navigating the journey of radiomics is not solely reliant on technological advancements and scientific innovations. It also relies on the ability to convince end users of its value. In the setting of radiomics, we should consider the different perspectives of patients; health care providers, including radiologists and nonradiologists; and health care systems. Beyond the well-known importance of constructing an accurate, reproducible, and robust radiomics model, it is paramount to also comprehend the needs, trust, preferences, and behaviors of users in the radiomics clinical translation process (Fig 3).

Figure 3:

Diagram shows different perspectives to be considered before radiomics is translated to clinical practice.

Patients’ Perspective

The traditional paternalistic approach of medicine is gradually evolving toward a more patient-centered paradigm, wherein patients are assuming an increasingly important role in shaping their health care decisions (54). As radiomics advances, it is essential to align its clinical application with patients’ preference, needs, and values. A framework that includes education, communication, compliance with legal and regulatory standards, and the safeguarding of patient privacy may be considered in light of its proven benefits in other innovation initiatives. Informed consent and shared decision-making should be taken into consideration in the clinical translation process of radiomics (54,55). Although some technical definitions are difficult to understand, it is important for medical doctors to be able to explain the clinical significance to patients as much as possible. Patients should have access to information about how radiomics can contribute to their diagnosis and treatment, and they should feel empowered to ask questions, express their concerns, and actively participate in decisions about its use in their health care.

Health Care Providers’ Perspective

Health care providers, including radiologists, oncologists, and other specialists, will play a central role in employing the radiomics tools in the patient care workflow. The radiomics journey should include various stakeholders at the inception to build trust regarding tool reliability, reproducibility, and accuracy, since skepticism can be a barrier to adoption. Health care providers must receive comprehensive education to understand radiomics’ capabilities and limitations, critically assess its evidence base, and seamlessly integrate it into their clinical workflow. Simplicity, automatization, and easy integration between different systems are crucial factors. Health care providers already grapple with burnout, and the introduction of tools that increase the complexity of their daily work may exacerbate anxiety and contribute to increasing burnout (56), ultimately adversely affecting patient outcomes (57). The goal of radiomics solutions should be to align with the preferences and workflows of health care professionals to facilitate their integration into clinical practice. Furthermore, health care providers must be able to assess radiomics’ clinical utility in specific patient populations, engage in shared decision-making with patients regarding its use, and collaborate across disciplines to maximize its impact. Programs of continuous learning are essential in this evolving field. Ultimately, radiomics should also align with patient-centered care, enhancing diagnostic accuracy and treatment planning while maintaining patient privacy and improving health care outcomes.

Health Care Systems’ Perspective

Despite the inherent complexity of health care systems, potential priorities can be summarized as follows: (a) patient outcomes and cost optimization (radiomics tools should prioritize the demonstration of clinical utility and cost-effectiveness with clear end points that align with the health care system’s priorities to a given disease scenario), (b) feasibility (compatibility with existing systems and data sharing strategies is critical for seamless integration), and (c) regulatory and ethical considerations (close and upfront adherence to established standards and guidelines with a strong emphasis on patient privacy and data security are necessary to address health care system concerns). Therefore, involving health care system stakeholders and being aware of their priorities are important in shaping future radiomics tools and ensuring that they align with overall system goals and needs (58).

Conclusion

In conclusion, this report provides a critical evaluation of the potential use of radiomics in oncology. While radiomics has the potential to add value to cancer management through the mining of quantitative high-dimensional data from medical images and much has been published on radiomics, important challenges remain unaddressed that hinder its clinical translation. To bridge the gap between the potential of radiomics and its clinical translation, this report highlights the current unmet needs, strategies that can address important mistakes in the radiomics pipeline, and the importance of including the perspective of multiple stakeholders to enhance the acceptability and widespread clinical use of radiomics in the oncologic setting.