TABLE 1.

Characteristics of included studies

Author, year	Search strategy (year last searched; no. databases; supplementary searches)	SR design (type of review; no. of studies included)	Topic; subject area	SR objectives	SR authors’ comments on study quality
Crumley, 2005 19	2004; Seven databases; four journals handsearched, reference lists and contacting authors	SR; n = 64	RCTs and CCTs; not specified	To identify and quantitatively review studies comparing two or more different resources (e.g., databases, Internet, handsearching) used to identify RCTs and CCTs for systematic reviews.	Most of the studies adequately described reproducible search methods, expected search yield. Poor quality in studies was mainly due to lack of rigor in reporting selection methodology. Majority of the studies did not indicate the number of people involved in independently screening the searches or applying eligibility criteria to identify potentially relevant studies.
Hopewell, 2007 20	2002; eight databases; selected journals and published abstracts handsearched, and contacting authors	SR and MA; n = 34 (34 in quantitative analysis)	RCTs; health care	To review systematically empirical studies, which have compared the results of handsearching with the results of searching one or more electronic databases to identify reports of randomized trials.	The electronic search was designed and carried out appropriately in majority of the studies, while the appropriateness of handsearching was unclear in half the studies because of limited information. The screening studies methods used in both groups were comparable in most of the studies.
Hopewell, 2007 21	2005; two databases; selected journals and published abstracts handsearched, reference lists, citations and contacting authors	SR and MA; n = 5 (5 in quantitative analysis)	RCTs; health care	To review systematically research studies, which have investigated the impact of gray literature in meta‐analyses of randomized trials of health care interventions.	In majority of the studies, electronic searches were designed and conducted appropriately, and the selection of studies for eligibility was similar for handsearching and database searching. Insufficient data for most studies to assess the appropriateness of handsearching and investigator agreeability on the eligibility of the trial reports.
Horsley, 2011 22	2008; three databases; reference lists, citations and contacting authors	SR; n = 12	Any topic or study area	To investigate the effectiveness of checking reference lists for the identification of additional, relevant studies for systematic reviews. Effectiveness is defined as the proportion of relevant studies identified by review authors solely by checking reference lists.	Interpretability and generalizability of included studies was difficult. Extensive heterogeneity among the studies in the number and type of databases used. Lack of control in majority of the studies related to the quality and comprehensiveness of searching.
Morrison, 2012 24	2011; six databases and gray literature	SR; n = 5	RCTs; conventional medicine	To examine the impact of English language restriction on systematic review‐based meta‐analyses	The included studies were assessed to have good reporting quality and validity of results. Methodological issues were mainly noted in the areas of sample power calculation and distribution of confounders.
Robson, 2019 14	2016; three databases; reference lists and contacting authors	SR; n = 37	N/R	To identify and summarize studies assessing methodologies for study selection, data abstraction, or quality appraisal in systematic reviews.	The quality of the included studies was generally low. Only one study was assessed as having low RoB across all four domains. Majority of the studies were assessed to having unclear RoB across one or more domains.
Schmucker, 2017 26	2016; four databases; reference lists	SR; n = 10	Study data; medicine	To assess whether the inclusion of data that were not published at all and/or published only in the gray literature influences pooled effect estimates in meta‐analyses and leads to different interpretation.	Majority of the included studies could not be judged on the adequacy of matching or adjusting for confounders of the gray/unpublished data in comparison to published data.
					Also, generalizability of results was low or unclear in four research projects
Morissette, 2011 23	2009; five databases; reference lists and contacting authors	SR and MA; n = 6 (5 included in quantitative analysis)	N/R	To determine whether blinded versus unblinded assessments of risk of bias result in similar or systematically different assessments in studies included in a systematic review.	Four studies had unclear risk of bias, while two studies had high risk of bias.
O'Mara‐Eves, 2015 25	2013; 14 databases and gray literature	SR; n = 44	N/R	To gather and present the available research evidence on existing methods for text mining related to the title and abstract screening stage in a systematic review, including the performance metrics used to evaluate these technologies.	Quality appraised based on two criteria‐sampling of test cases and adequacy of methods description for replication. No study was excluded based on the quality (author contact).

SR = systematic review; MA = meta‐analysis; RCT = randomized controlled trial; CCT = controlled clinical trial; N/R = not reported.