Table 1.
Description of RQS items divided per domain, with scoring ranges and criteria
| Domain 1—Protocol quality and stability in image and segmentation (0–5 points) | |
| a) | Image protocol quality (item 1) |
| + 1 (if protocols are well-documented) | |
| + 1 (if public protocol is used) | |
| b) | Imaging at multiple time points (item 4) |
| c) | Phantom study on all scanners—detect inter-scanner differences and vendor-dependent features (item 3) |
| d) | Multiple segmentations (item 2) |
| + 1 for each 1b, 1c, and 1d | |
| Domain 2—Feature selection and validation (−8 to 8 points) | |
| a) | Feature reduction or adjustment for multiple testing (item 5) |
| − 3 (if neither measure is implemented) + 3 (if either measure is implemented) | |
| b) | Validation (item 12) |
| − 5 (if validation is missing) | |
| + 2 (if validation is based on a dataset from the same institute) | |
| + 3 (if validation is based on a dataset from another institute) | |
| + 4 (if validation is based on two datasets from two distinct institutes) | |
| + 4 (if the study validates a previously published signature) | |
| + 5 (if validation is based on three or more datasets from distinct institutes) | |
| Domain 3—Biologic/clinical validation and utility (0–6 points) | |
| a) | Multivariable analysis with non-radiomics features (item 6) |
| b) | Detect and discuss biological correlates (item 7) |
| + 1 for each 3a and 3b | |
| c) | Comparison to “gold standard” (item 13) |
| d) | Potential clinical utility (item 14) |
| + 2 for each 3a and 3b | |
| Domain 4—Model performance index (0–5 points) | |
| a) | Cut-off analyses (item 8) |
| + 1 if cut-off analyses are conducted | |
| b) | Discrimination statistics (item 9) |
| + 1 (if a discrimination statistic and its statistical significance are reported) | |
| + 1 (if a resampling method technique is also applied) | |
| c) | Calibration statistics (item 10) |
| + 1 (if a calibration statistic and its statistical significance are reported) | |
| + 1 (if a resampling method technique is also applied) | |
| Domain 5—High level of evidence (0–8 points) | |
| a) | Prospective study registered in a trial database (item 11) |
| + 7 (for prospective validation of a radiomics signature in an appropriate trial) | |
| b) | Cost-effectiveness analysis (item 15) |
| + 1 if cost-effectiveness analyses are conducted | |
| Domain 6—Open science and data (0–4 points) | |
| a) | Open science and data (item 16) |
| + 1 (if scans are open source) | |
| + 1 (if region of interest segmentations are open source) | |
| + 1 (if code is open source) | |
| + 1 (if radiomics features are calculated on a set of representative ROIs and the calculated features and representative ROIs are open source) | |
RQS Radiomics Quality Score