Table 1.

Overview of radiomics quality score items and mode of the respective scores in the reviewed studies

RQS checkpoint	RQS item number and name	Description and (points)	Mode
First	Item 1: image protocol quality	Well-documented protocol (+1) AND/OR publicly available protocol (+1)	1
Second	Item 2: multiple segmentation	Testing feature robustness to segmentation variability, e.g., different physicians/algorithms/software (+1)	0
	Item 3: phantom study	Testing feature robustness to scanner variability, e.g., different vendors/scanners (+1)	0
	Item 4: multiple time points	Testing feature robustness to temporal variability, e.g., organ movement/expansion/shrinkage (+1)	0
Third	Item 5: feature reduction	Either feature reduction OR adjustment for multiple testing is implemented (+ 3); otherwise, (−3)	3
	Item 6: multivariable analysis	Non-radiomic feature are included in/considered for model building (+1)	0
	Item 7: Biological correlates	Detecting and discussing correlation of biology and radiomic features (+1)	0
	Item 8: cut-off analysis	Determining risk groups by either median, pre-defined cut-off, or continuous risk variable (+1)	0
	Item 9: discrimination statistics	Discrimination statistic and its statistical significance are reported (+ 1); a resampling technique is also applied (+1)	2
	Item 10: calibration statistics	Calibration statistic and its statistical significance are reported (+ 1); a resampling technique is also applied (+1)	0
	Item 11: prospective design	Prospective validation of a radiomics signature in an appropriate trial (+7)	0
	Item 12: validation	Validation is missing (−5) OR internal validation (+2) OR external validation on single dataset from one institute (+3) OR external validation on two datasets from two distinct institutes (+4) OR validation of a previously published signature (+4) validation is based on three or more datasets from distinct institutes (+5)	2
	Item 13: comparison to “gold standard”	Evaluating model’s agreement with/superiority to the current “gold standard” (+2)	0
	Item 14: potential clinical application	Discussing model applicability in a clinical setting (+2).	2
	Item 15: cost-effectiveness analysis	Performing a cost-effectiveness of the clinical application (+1)	0
	Item 16: open science and data	Open source scans (+1) AND/OR open source segmentations (+1) AND/OR open source code (+1) AND/OR open source representative features and segmentations (+1)	0

RQS radiomics quality score