Table 3.

Criteria for measuring quality of findings for studies examining measurement properties based on Terwee et al. (2007) and Schellingerhout et al. (2011).

COSMIN measurement property	Rating	Quality Criteria
Internal consistency	+	Subtests one-dimensional (determined through factor analysis with adequate sample size) and Cronbach alpha between 0.70 and 0.95
	?	Dimensionality of subtests unknown (no factor analysis) or Cronbach's alpha not calculated
	−	Subtests uni-dimensional (determined through factor analysis with adequate sample size) and Cronbach's alpha < 0.7 or > 0.95
	±	Conflicting results
	NR	No information found on internal consistency
	NE	Not evaluated due to “poor” methodology rating on COSMIN
Reliability	+	ICC/weighted Kappa equal to or > than 0.70
	?	Neither ICC/weighted Kappa calculated or doubtful design or method (e.g., time interval not appropriate)
	−	ICC/weighted Kappa < 0.70 with adequate methodology
	±	Conflicting results
	NR	No information found on reliability
	NE	Not evaluated due to “poor” methodology on COSMIN
Measurement error	+	MIC > SDC or MIC equals or inside LOA
	?	MIC not defined or doubtful design or method
	−	MIC < SDC or MIC equals or inside LOA with adequate methodology
	+	Conflicting results
	NR	No information found on measurement error
	NE	Not evaluated due to “poor” methodology on COSMIN
Content validity	+	Good methodology (i.e., an overall rating of “Good” or above on COSMIN criteria for content validity) and experts examined all items for content and cultural bias during development of assessment
	?	Questionable methodology or experts only employed to examine one aspect (e.g., cultural bias)
	−	No expert reviewer involvement
	±	Conflicting results
	NR	No information found on content validity
	NE	Not evaluated due to “poor” methodology
Structural validity	+	Factor analysis performed with adequate sample size. Factors explain at least 50% of variance
	?	No factor analysis or inadequate sample size. Explained variance not mentioned
	−	Factors explain < 50% of variance despite adequate methodology
	±	Conflicting results
	NR	No information found on structural validity
	NE	Not evaluated due to “poor” methodology
Hypothesis testing	+	Convergent validity: Correlation with assessments measuring similar constructs equal to or >0.5 and correlation is consistent with hypothesis Discriminant validity: findings consistent with hypotheses using appropriate statistical analysis (e.g., t-test p < 0.05 or Cohen's d effect size > 0.5)
	?	Questionable methodology e.g., only correlated with assessments that are not deemed similar
	−	Discriminant validity: findings inconsistent with hypotheses (e.g., no significant difference identified from appropriate statistical analysis) Convergent validity: Correlation with assessments measuring similar constructs equal to or < 0.5 or correlation is inconsistent with hypothesis
	±	Conflicting results
	NR	No information found on hypothesis testing
	NE	Not evaluated due to “poor” methodology

+, Positive result; −, Negative result; ?, Indeterminate result due to methodological shortcomings; ±, Conflicting results within the same study (e.g., high correlations for some results but not on others); NR, Not reported; NE, Not evaluated; MIC, minimal important change; SDC, smallest detectable change; LOA, limits of agreement; ICC, Intra-class correlation; SD, standard deviation.