Table 1.

Data extracted from methods studies evaluating tools for assessing risk of bias in SRs

Study design Category	Data extracted
Primary methods studies
Study characteristics	First author, year
	Title
	Primary objective
Description of primary methods studies	Name of the included tools or measures
	Type of assessment (e.g. assessment of reliability, content validity)
	Content validity—methods of item generation
	Content validity—comprehensiveness
	Reliability—description of reliability testing
	Tests of validity description of correlation coefficient testing
	Other assessment (feasibility, acceptability, piloting)
Risk of bias criteria	Existence of a protocol
	Method to select the sample of SRs to which the tool/measure was applied
	Process for selecting the raters/assessors who applied the tool/measure
	Pre-specified hypotheses for testing of validity
Systematic reviews of methods studies
Study characteristics	First author, year
	Title
Description of SRs of methods studies	Primary objective
	Number of included tools
	Number of studies reporting on the included tools
	Name of the included tools or measures (unnamed tools are identified by first author name and year of publication)
	Content validity—reported method of development (e.g. item generation, expert assessment of content)
	Reliability—description of reliability testing
	Construct validity—description of any hypothesis testing. For example, how assessments from two or more tools relate, whether assessments relate to other factors (e.g. effect estimates or findings)
	Other assessment (feasibility, acceptability, piloting)
Risk of bias criteria (using three domains from the ROBIS tool [15])	Domain 1—study eligibility criteria: concerns regarding specification of eligibility criteria (low, high or unclear concern)
	Domain 2—identification and selection of studies: concerns regarding methods used to identify and/or select studies (low, high or unclear concern)
	Domain 3—data collection and study appraisal: concerns regarding methods used to collect data and appraise studies (low, high or unclear concern)
	Overall judgment: Interpretation addresses all concerns identified in Domains 1–3, relevance of studies was appropriately considered, reviewers avoided emphasising results based on statistical significance.