Table 4.
Criteria for good measurement properties & certainty of evidence according to the modified GRADE method
| Tool or dimension | Content validity | Internal consistency | Reliability | Measurement error | Criterion validity | Construct validity |
|---|---|---|---|---|---|---|
| “Applicability”-dimension of LEGEND [77] | ||||||
| CGMP | (?) | |||||
| GRADE | Low | |||||
| “Applicability”-dimension of Carr´s evidence-grading scheme [63] | ||||||
| CGMP | (?) | |||||
| GRADE | Very Low | |||||
| Bornhöft´s checklist [78] | ||||||
| CGMP | (?) | |||||
| GRADE | Very Low | |||||
| Cleggs´s external validity assessment [64] | ||||||
| CGMP | (?) | |||||
| GRADE | Very Low | |||||
| Clinical Applicability [66] | ||||||
| CGMP | (?) | |||||
| GRADE | Very Low | |||||
| Clinical Relevance Instrument [79, 80] | ||||||
| CGMP | (?) | (-) | ||||
| GRADE | Moderate | Moderate | ||||
| Clinical Relevance according to the CCBRG [81] | ||||||
| CGMP | (?) | |||||
| GRADE | Moderate | |||||
| Clinical relevance scores [68] | ||||||
| CGMP | (?) | |||||
| GRADE | Very Low | |||||
| Estrada´s applicability assessment criteria [82] | ||||||
| CGMP | (?) | |||||
| GRADE | Very Low | |||||
| External Validity Assessment Tool (EVAT) [83] | ||||||
| CGMP | (?) | |||||
| GRADE | Low | |||||
| “External validity”-dimension of the Downs & Black Checklist [22, 84] | ||||||
| CGMP | (?) | (?) | (±)a | (?) | (-) | |
| GRADE | Moderate | Very Low | Moderate | Very Low | Very Low | |
| “External validity”-dimension of Foy´s quality checklist [65] | ||||||
| CGMP | (?) | |||||
| GRADE | Very Low | |||||
| “External validity”-dimension of Liberati´s quality assessment criteria [69] | ||||||
| CGMP | (?) | |||||
| GRADE | Very Low | |||||
| “External validity”-dimension of Sorg´s checklist [71] | ||||||
| CGMP | (?) | |||||
| GRADE | Very Low | |||||
| “External validity”-criteria of the USPSTF manual [73, 84] | ||||||
| CGMP | (?) | (+) | ||||
| GRADE | Very Low | Very Low | ||||
| Feasibility, Appropriateness, Meaningfulness and Effectiveness (FAME) scale [70] | ||||||
| CGMP | (?) | |||||
| GRADE | Very Low | |||||
| Generalizability, Applicability and Predictability (GAP) checklist [76] | ||||||
| CGMP | (?) | |||||
| GRADE | Very Low | |||||
| Gartlehner´s tool [86, 87] | ||||||
| CGMP | (?) | (-) | (?) | (+) | ||
| GRADE | Very Low | Moderate | Very Low | Very Low | ||
| Green & Glasgow´s external validity quality rating criteria [88, 90, 91] | ||||||
| CGMP | (?) | (+) | (-) | |||
| GRADE | Very Low | Very Low | Very Low | |||
| “Indirecntess”-dimension from the GRADE Handbook [48, 92, 93] | ||||||
| CGMP | (?) | (-) | ||||
| GRADE | Moderate | Very Low | ||||
| Loyka´s external validity framework [75] | ||||||
| CGMP | (?) | (?) | ||||
| GRADE | Very Low | Low | ||||
| modified “Indirectness” of the Checklist for GRADE [94, 95] | ||||||
| CGMP | (?) | (-) | ||||
| GRADE | Low | Very Low | ||||
| External validity checklist of the National Health & Medical Research Council (NHMRC) Handbook [74] | ||||||
| CGMP | (?) | |||||
| GRADE | Very Low | |||||
| revised Graphical Appraisal Tool for Epidemiological Studies (GATE) [72] | ||||||
| CGMP | (?) | |||||
| GRADE | Very Low | |||||
| Rating of Included Trials on the Efficacy-Effectiveness Spectrum (RITES) [47, 97] | ||||||
| CGMP | (+) | (+) | (+) | |||
| GRADE | Moderate | Very Low | Low | |||
| “Selection Bias”-dimension (Section A) of EPHPP [98, 99] | ||||||
| CGMP | (?) | (-) | (+) | |||
| GRADE | Moderate | Low | Very Low | |||
| Section C of the CASP checklist for RCTs [100] | ||||||
| CGMP | (?) | |||||
| GRADE | Very Low | |||||
| Whole Systems research considerations´checklist [67] | ||||||
| CGMP | (?) | |||||
| GRADE | Very Low | |||||
Abbreviations: CCBRG Cochrane Collaboration Back Review Group, CGMP criteria for good measurement properties, EPHPP Effective Public Health Practice Project, GRADE Grading of Recommendations Assessment, Development and Evaluation, LEGEND Let Evidence Guide Every New Decision, NICE National Institute for Health and Care Excellence, USPSTF U.S. Preventive Services Task Force;
Criteria for good measurement properties: (+) = sufficient; (?) = indeterminate; (-) = insufficient, (±) or inconsistent
Level of evidence according to the modified GRADE approach: high, moderate, low, or very low evidence.Note: the measurement properties “structural validity” and “cross-cultural validity” are not presented in this table, since they were not assessed in any of the included studies
Fields left blank indicate that those measurement properties were not assessed by the study authors
a please refer to Table S4 for more information on reliability of the “external validity”-dimension of the Downs & Black checklist