Table 1.
RQS and average score for the included studies.
| Criteria | Points | Average Score | |
|---|---|---|---|
| 1 | Image protocol quality | +1 if protocols are well-documented +1 if public protocol is used |
0.65 |
| 2 | Multiple segmentations | +1 if multiple segmentations are carried out (i.e., different physicians/algorithms/software) | 0.53 |
| 3 | Phantom study | +1 if phantom study is used on all scanners | 0.00 |
| 4 | Multiple time points | +1 if images are collected at additional time points | 0.07 |
| 5 | Feature reduction or adjustment for multiple testing | −3 if neither measure is implemented +3 if either measure is implemented |
0.69 |
| 6 | Multivariable analysis with non-radiomics features | +1 if multivariable analysis with non-radiomics features is carried out | 0.12 |
| 7 | Biological correlates | +1 if phenotypic differences are demonstrated | 0.96 |
| 8 | Cut-of-analyses | +1 if risk groups are determined by either the median, a previously published cut-off or if a continuous risk variable is reported | 0.09 |
| 9 | Discrimination statistics | +1 if a discrimination statistic and its statistical significance is reported (i.e., ROC curve, AUC) +1 if a resampling method technique is also applied (i.e., bootstrapping, cross-validation) |
1.21 |
| 10 | Calibration statistics | +1 if a calibration statistic and its statistical significance is reported (i.e. Calibration-in-the-large/slope) +1 if a resampling method technique is also applied (i.e., bootstrapping, cross-validation) |
0.02 |
| 11 | Prospective | +7 for prospective validation of a radiomics signature in an appropriate trial | 0.46 |
| 12 | Validation | −5 if validation is missing +2 if validation is based on a dataset from the same institute +3 if validation is based on a dataset from another institute +4 if validation is based on two datasets from two institutes +4 if study validates a previously published signature +5 of validation is based in three or more datasets from distinct institutes |
−3.88 |
| 13 | Gold standard | +2 if comparison to the current gold standard is carried out | 1.91 |
| 14 | Potential clinical utility | +2 if a potential application in a clinical setting is reported | 2.00 |
| 15 | Cost-effectiveness analysis | +1 if the cost-effectiveness of the clinical application is reported | 0.00 |
| 16 | Open science and data | +1 if scans are open source +1 if region of interest (ROI) segmentations are open source +1 if code is open source +1 if representative segmentations and features are open source |
0.16 |