Table 3.

Potential problems at each step of the radiomics workflow along with possible solutions offered by the literature. Each workflow step with potential problems and solutions identified by the literature is labelled with a letter A-H to reference in-text. Note: Modelling does not have a letter associated with since there is no consensus on the best statistical modelling strategies.

Problem area		Potential problems	Potential solutions
Image acquisition	A	Different scanners and acquisition protocols affect feature reproducibility [[79], [80], [81], [82], [83], [84], [85], [86], [87], [88], [89], [90], [91]]	Image phantoms on different scanners to provide baseline [79], establish credibility of scanners and protocols [84], catalogue reproducible features [86,90], model a correction algorithm [89], harmonize data [91].
	B	Patient motion affects feature reproducibility [80,92,93]	Set motion tolerances, reduce ROI boundaries [80], use single phase from 4D images [92], find robust features using 4DCT data [93].
Image acquisition and reconstruction	C	Image resolution parameters (voxel size, slice thickness) affect feature values [79,88,[94], [95], [96], [97], [98]] model performance [99].	Control resolution [79] parameters in prospective studies, resample to common resolution and voxel depth [[94], [95], [96],98], apply smoothing image filters [95], apply deep learning methods [100].
Image reconstruction	D	Image reconstruction algorithm and reconstruction parameters (kernel) affects features [97,101,102]	Pre-processing image correction [101] and harmonization of acquisition techniques [97,102].
Segmentation	E	Delineation variability [90,[103], [104], [105], [106], [107]] affects features and is time consuming [106,107]. Results from one disease site are not necessarily transferrable to another [108].	Expert ROI definition [103], multiple observers [103,104,108], identification of stable features with respect to delineation [90,104,105], automated segmentation [106,107], image filtering [108]
Pre-processing	F	Number of grey levels used to discretize histogram and texture features affects feature values [96,98,109], as does bin width [94].	Texture features can be normalized to reduce dependency on the number of grey levels [98], number of grey levels used for discretization should be recorded with feature formula. 128 grey levels may be optimal for texture features, along with thresholding [109]
Feature extraction		No studies found in the literature search.
Feature correlation	G	Strong correlations between tumor volume and radiomic features exist [98,[110], [111], [112]]	Normalization of features to volume [98], bit depth resampling [110], feature redesign [110], more robust statistics to check added value of radiomics signatures [111].
Test re-test	H	Radiomic features may not be repeatable over multiple measurements [[113], [114], [115]], repeatable features are not generalizable to other disease sites [116].	Test-retest data acquisition [113,116], use of multiple 4D phases [113,115], use of simulated retest by image perturbation [114].
Modelling clinical outcome		Different modelling strategies affect model performance [[117], [118], [119], [120]]	Sample sizes above 50 give better predictive performance [118], as does normalizing features [117]. No consensus on best modelling strategies to use.