Table 3.
Property | Rating | Quality Criteria |
---|---|---|
Reliability | ||
Internal consistency | + | (Sub)scale unidimensional AND Cronbach's alpha(s) ≥ 0.70 |
? | Dimensionality not known OR Cronbach's alpha not determined | |
- | (Sub)scale not unidimensional OR Cronbach's alpha(s) < 0.70 | |
Measurement error | + | MIC > SDC OR MIC outside the LOA |
? | MIC not defined | |
- | MIC ≤ SDC OR MIC equals or inside LOA | |
Reliability | + | ICC/weighted Kappa ≥ 0.70 OR Pearson's r ≥ 0.80 |
? | Neither ICC/weighted Kappa, nor Pearson's r determined | |
- | ICC/weighted Kappa < 0.70 OR Pearson's r < 0.80 | |
Validity | ||
Content validity | + | The target population considers all items in the questionnaire to be relevant |
AND considers the questionnaire to be complete | ||
? | No target population involvement | |
- | The target population considers items in the questionnaire to be irrelevant | |
OR considers the questionnaire to be incomplete | ||
Construct validity | ||
- Cross-cultural validity | + | Original factor structure confirmed OR no important DIF |
? | Confirmation original factor structure AND DIF not mentioned | |
- | Original factor structure not confirmed OR important DIF | |
- Structural validity | + | Factors should explain at least 50% of the variance |
? | Explained variance not mentioned | |
- | Factors explain < 50% of the variance | |
- Hypothesis testing | + | (Correlation with an instrument measuring the same construct ≥ 0.50 OR |
at least 75% of the results are in accordance with the hypotheses) AND | ||
correlation with related constructs is higher than with unrelated constructs | ||
? | Solely correlations determined with unrelated constructs | |
- | Correlation with an instrument measuring the same construct < 0.50 OR | |
< 75% of the results are in accordance with the hypotheses OR | ||
correlation with related constructs is lower than with unrelated constructs | ||
Responsiveness | ||
Responsiveness | + | (Correlation with an instrument measuring the same construct ≥ 0.50 |
OR at least 75% of the results are in accordance with the hypotheses | ||
OR AUC ≥ 0.70) AND correlation with related constructs is higher | ||
than with unrelated constructs | ||
? | Solely correlations determined with unrelated constructs | |
- | Correlation with an instrument measuring the same construct < 0.50 OR | |
< 75% of the results are in accordance with the hypotheses OR AUC < 0.70 | ||
OR correlation with related constructs is lower than with unrelated constructs |
[..] = reference number, MIC = minimal important change, SDC = smallest detectable change, LOA = limits of agreement, ICC = intraclass correlation coefficient, DIF = differential item functioning, AUC = area under the curve
† + = positive rating, ? = indeterminate rating, - = negative rating