Table 1.

Methodological features of exemplar overviews

	Pollock [19]	McClurg [21]	Estcourt [20]	Hunt [22]	Brunton [23]
Type of overview	Cochrane overview of reviews	Cochrane overview of reviews	Cochrane overview of reviews	Overview of reviews of diagnostic test accuracy	Mixed method overview of reviews
Aim/question of overview	To synthesise systematic reviews of interventions to improve upper limb function after stroke	To synthesise Cochrane reviews of conservative interventions for the prevention or treatment of female urinary incontinence	To summarise the evidence in Cochrane reviews of the effectiveness and safety of red cell transfusions for treatment or prevention of complications experienced by people with sickle cell disease	To summarise evidence from studies of the accuracy of diagnostic tests of brief cognitive assessments for identifying dementia in primary care	Overview of workplace health promotion interventions
Inclusion criteria: participants, interventions, outcomes	Participants: stroke	Participants: female adults with urinary incontinence	Participants: people with sickle cell disease	Participants: all adults aged 18 years or over recruited from a primary care or general practice population	Participants: any
					Interventions: any interventions delivered in a workplace setting
	Interventions: any	Interventions: any conservative intervention (definition provided)	Interventions: red cells transfusions
				Index test: brief cognitive assessments
	Outcomes: upper limb function, impairment or activity of daily living	Outcomes: any	Outcomes: mortality, sickle-cell related complications, adverse events, quality of life, red cell transfusion requirements, and hospital admissions	Outcomes: diagnostic test result (positive/negative)	Outcomes: healthcare or wellbeing outcomes
				Other: diagnostic test accuracy information, other reported test outcomes
Inclusion criteria: type of study	Systematic reviews including randomised controlled trials	Cochrane systematic reviews	Cochrane systematic reviews	Systematic reviews of diagnostic test accuracy	Systematic reviews in which the search strategy, inclusion criteria and quality assessment methods are described
					Published from 1995 to present; in English;
Search strategy	CDSR, DARE, PROSPERO	CDSR	CDSR	CDSR, EMBASE, MEDLINE, PsycInfo	MEDLINE, DARE, Cochrane Library, PsycInfo, DOPHER, ASSIA, ABI Inform, Scopus, Business Source Premier
	Reported line by line for CDSR and DARE	Reported line by line	Reported line by line	Pubmed search reported line by line
					Details of the search strategy are available from the reviewers
Updating searches of out of date reviews	–	–	Searches of some included reviews were updated, in order to identify additional primary research studies, and update the original review to include these. The authors determined which reviews to update by using a process of judgement which systematically considered the search dates, presence of included or ongoing trials and relevance of the review topic	–	–
Selection of reviews	Two authors independently apply inclusion criteria	Two authors independently apply inclusion criteria	Two authors independently apply inclusion criteria	Two authors independently apply inclusion criteria	Two reviewers independently apply inclusion criteria
Data extraction	Characteristics of reviews; results of meta-analyses	Characteristics of reviews; results of meta-analyses	Characteristics of reviews; results of meta-analyses	Characteristics of reviews; results of meta analyses; diagnostic test accuracy information	Characteristics of reviews; results of meta-analyses; data from process evaluations
Assessment of quality of reviews	Modified AMSTAR: each of the original 11 AMSTAR questions/criteria were broken down into a series of dichotomous questions, and the answers (yes, no or unclear) to these dichotomous questions were used to determine the answer to the original AMSTAR questions	ROBIS	AMSTAR	AMSTAR and ROBIS	AMSTAR
			(Additional comments were added to each of the AMSTAR criteria to clarify the decision making process the authors had made)	(Note: employed both tools as there was an absence of guidance relating to the optimal tool for assessing reviews of DTA and these were the available options. Both tools were reported to provide comparable assessment results, but the ROBIS tool was considered to be easier to apply, and arguably more relevant, as this tool could be specifically tailored to reviews of DTA)	(Note: the AMSTAR scores were categorised into low (score 0–3), medium (score 4–7) or high (8–11))
	(Note: this assessment was completed prior to publication and availability of the ROBIS tool)
Assessment of quality of evidence within reviews	GRADE approach, with use of algorithm (details presented and discussed elsewhere [10, 37])	GRADE approach, with use of algorithm, supplemented with input from key stakeholders (clinicians) and statisticians	GRADE approach, applied based on assessments provided in the included reviews, with two independent overview authors applying GRADE levels of evidence and any differences resolved through discussion or third party adjudication	No quality assessment was undertaken. Quality assessment measures from included systematic reviews reported narratively. Results showed that 2/13 reviews made no reference to quality assessment; 2/13 reviews referred to STARD criteria [38] as quality assessment; 2/13 did not report any details on methods or tool used; 6/13 used QUADAS-2 [39] or QUADAS; 1/13 review used a series of tools, including the United States Preventative Services Task Force’s design-specific quality criteria [40], National Institute for Health and Care Excellence (previously National Institute for Health and Clinical Excellence) methodology checklist [41], AMSTAR [42] and the Newcastle-Ottowa Scale for observational studies [43]	No quality assessment was undertaken: the objective of this overview was to provide a ‘systematic map’ of relevant reviews which aimed to identify characteristics associated with effectiveness
Data synthesis	Grouped according to GRADE, for intervention versus control, for each outcome explored. Highlighting evidence of beneficial effect, harm or no effect	Data grouped and presented according to type of urinary incontinence. Tabulated summary of systematic review evidence relating to all conservative interventions for urinary incontinence, signposting which systematic reviews address which interventions, with summary of details of the population of participants, comparisons, volume and quality of evidence	Reported using the GRADE approach, with presentation of statistical outcome data as determined by data available in the reviews	Synthesis of results across included studies using sensitivity and specificity measures, presented narratively and within tables	Framework synthesis to synthesise evidence from systematic reviews with other types of evidence (research of stakeholders’ experiences and perspectives; key policy documents). Data relating to moderator analyses to identify ‘successful’ intervention characteristics were tabulated
Statistical analysis	No statistical analysis was planned or completed	1. Network maps, created using Stata software, to illustrate the number of trial and participants within trials	No statistical analysis was planned or completed	No statistical analysis was completed	No statistical analysis was planned or completed
				(Note: additional analysis is ongoing relating to the assessment of test administration times, analysis of DTA across the most widely used brief cognitive assessments, and comparing diagnostic test performance across all systematic reviews)
	A visual plot of the effect size relating to the effect of interventions on the primary outcome was produced (reproducing data from the included reviews)
		2. Visual plot of effect size for comparisons of conservative intervention versus control, placebo or standard care for outcomes of condition-specific quality of life and symptomatic cure or improvement
Summary of key findings	Two key summary tables were produced; each of these summarised the available evidence relating to 19 identified different interventions, highlighting the GRADE of the evidence, the direction of the effect (where GRADE was high or moderate), and key implications for clinical practice and research (see Additional file 1)	A ‘Summary of results’ table was planned, clearly indicating where there is evidence of an effect of conservative interventions, for each relevant intervention comparison and for both primary and secondary outcome of interest (see Additional file 2)	‘Overview of reviews’ table in a format similar to the ‘Summary of findings’ table as recommended in chapter 22 of the Cochrane Handbook for Systematic Reviews of Interventions [33]. These will include the primary outcomes of this overview for each intervention (see Additional file 2)	A summary of findings table was produced displaying review reference, component study reference, reference standard, index test, threshold, time taken, sample size and key demographic data, target condition, condition prevalence, accuracy data reported and notes. Additional summary tables were produced for comparisons at review level, non-DTA data reported, and individual brief cognitive assessments	Structured summary
					Executive summary
					Table of ‘key characteristics identified from effectiveness reviews, views studies and policy documents’
					A summary table in which information about each of the included reviews was provided, including the AMSTAR rating of the review. The direction of effect for different intervention types was also summarised, categorising evidence according to whether there was a statistically significant beneficial effect, non-statistically significant beneficial effect, no difference between control and intervention, non-significant detrimental effect or statistically significant detrimental effect (see Additional file 3)
Publication status	Published (Cochrane Library) [19]	Ongoing (protocol published in Cochrane Library) [21]	Ongoing (protocol published in Cochrane Library) [20]	Ongoing (protocol published on PROSPERO ref. CRD42015022078) [22]	Published (EPPI-Centre website) [23]