Table 2.

Inter-rater agreement in the assessment of the RQS

RQS scoring item	S* [95% CI]
Image Protocol	0.45 [0.20–0.67]
Multiple Segmentations	0.93 [0.82–1.00]
Phantom Study	1.00 [1.00–1.00]
Multiple Time Points	0.93 [0.82–1.00]
Feature Reduction	0.93 [0.82–1.00]
Non Radiomics	0.67 [0.49–0.85]
Biological Correlates	0.93 [0.82–1.00]
Cut-off	0.93 [0.82–1.00]
Discrimination and Resampling	0.82 [0.71–0.92]
Calibration	0.96 [0.89–1.00]
Prospective	1.00 [1.00–1.00]
Validation	1.00 [1.00–1.00]
Gold Standard	0.76 [0.63–0.88]
Clinical Utility	0.60 [0.38–0.82]
Cost-effectiveness	1.00 [1.00–1.00]
Open Science	1.00 [1.00–1.00]

CI: confidence interval, RQS: Radiomics Quality Score