Chart 3. Methodological quality assessment of sample validity studies using the Quality Appraisal tool for Validity Studies, Uberaba, Minas Gerais, Brazil, 2022.
| Items | Studies | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 9 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | |
| 1. Was the study design reported? | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| 2. Did the study provide an accurate description of the type of validity tested? | Y | Y | Y | Y | Y | Y | Y | Y | NR | Y | Y | Y | Y | Y |
| 3. Was the study setting and time frame of participant recruitment clearly outlined and described? | Y | Y | Y | Y | Y | Y | Y | Y | N | Y | Y | Y | Y | Y |
| 4. Were the criteria for participant selection clearly described? | Y | Y | Y | Y | Y | Y | Y | Y | N | Y | Y | Y | Y | Y |
| 5. Were the participants in the study representative of the sample population from which they were recruited? | Y | Y | Y | Y | Y | Y | Y | Y | NR | Y | Y | Y | Y | Y |
| 6. Did the study clearly describe the outcome measures to be validated? | Y | Y | Y | Y | Y | Y | Y | Y | N | Y | Y | Y | Y | Y |
| 7. Did the study provide a clear description of the procedures for testing validity? | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | N |
| 8. Was the testing procedure standardized for all participants? | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | NR |
| 9. Was a priori sample size calculation performed to ensure that the study had sufficient power? | N | N | N | N | N | N | N | N | N | Y | N | N | N | N |
| 10. Did the study describe and justify any attrition that may have occurred? | NR | NR | NR | NR | NR | NR | NR | NR | NR | NR | NR | NR | NR | NR |
| 11. Were the statistical analyses used to test validity appropriate for the study? | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | NR |
| 12. When multiple comparisons were performed, were appropriate statistical adjustments used to control for the likelihood of a type 1 error? | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 13. Did the study identify potential confounding variables and if so, were measures taken to adjust for these confounders? | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 14. Were the primary findings of the study clearly described? | Y | Y | Y | Y | Y | Y | Y | N | Y | Y | Y | N | Y | N |
| 15. Were validity coefficients reported for primary outcomes? | Y | N | N | N | Y | Y | Y | Y | N | Y | Y | Y | Y | N |
| 16. For primary outcomes, did the study report the standard deviation or confidence intervals for normally distributed data? Or, if non-normally distributed data, did the study report the inter-quartile range for the main outcomes? | N | N | N | N | N | N | N | N | Y | Y | N | N | N | N |
| 17. Was the process of selecting expert panel and their qualifications described? | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| 18. Did the study provide a rationale for the selection of the reference standard? | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 19. When the index test was assessed by more than one rater, were the raters blinded to the findings of the other raters? | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 20. When the index test was assessed by more than one rater, was the inter-rater reliability between raters established and reported? | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 21. Was the time interval used between administration of reference standard and the test measure appropriate? | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 22. Were subjects in different groups homogenous at baseline or if they weren’t homogenous at baseline, were differences between groups accounted for during the analysis? | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 23. Did the measures used for convergent validity represent a similar construct as the outcome measure of interest? | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 24. Did the measures used for discriminant validity represent a construct different from the outcome measure of interest? | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
NA - not applicable; NR - not reported; Y - yes; N - no.