Kirkham 2010b.

Methods	Investigated discrepancies between systematic review protocols and published systematic reviews.
Data	N = 288 systematic reviews Inclusion criteria: New reviews published in the Cochrane Database of Systematic Reviews between Issue 4, 2006 and Issue 2, 2007, with an available protocol (either published or provided by the authors on request). Exclusion criteria: Reviews from the Cochrane Methodology Review Group; Reviews without an available protocol. Year(s) of publication of the systematic reviews: 2006 to 2007 Areas of health care addressed by the systematic reviews: Various (at least one review from each of 49 Cochrane Review Groups was included) Methodological quality of systematic reviews: Not assessed Number (proportion) of systematic reviews that only included RCTs: Not formally assessed (38 reviews did not identify any RCTs, and the remaining reviews included at least one RCT, though the proportion of reviews that included RCTs and non‐RCTs is unclear) Number (proportion) of systematic reviews that are Cochrane Reviews: 288 (100%) Extent of overlap of RCTs included in the systematic reviews in the empirical study: Not assessed, though unlikely to have impacted on the results
Comparisons	Systematic review protocols versus published systematic reviews
Outcomes	Prevalence of systematic reviews with: any discrepancy in at least one outcome measure (primary, secondary, or unlabelled) from protocol to publication; addition of at least one new primary outcome in the review that was not specified at all (i.e. as either a primary or secondary outcome) in the protocol; omission of at least one primary outcome in the review that was listed as a primary outcome measure in the protocol; an upgraded outcome (if a secondary protocol outcome was promoted to a review primary outcome; an upgrade also occurred if the review specified primary outcome(s) but the same outcomes listed in the protocol were not listed with any order of importance, i.e. primary or secondary); a downgraded outcome (if a protocol primary outcome was demoted to a review secondary outcome; a downgrade also occurred if the protocol specified primary outcome(s) but the same outcomes listed in the review had no order of importance). Risk of obtaining a statistically significant result (P < 0.05) in the meta‐analysis if the discrepancy between protocol and published review was: either an "addition" or "upgrade" compared to if there was no discrepancy*; a "downgrade" compared to if there was no discrepancy*.
Notes	Statistical significance defined as P < 0.05. *The analysis of this outcome is based on 245 meta‐analysis results from 148 systematic reviews (140 of the 288 included systematic reviews were excluded from the analysis because they did not report a meta‐analysis for the primary outcome). Rather than analysing one meta‐analysis result per systematic review, the authors analysed all meta‐analyses of the primary review comparison in each review (as many reviews had more than one primary outcome). Study published as a journal article and conference abstract. Additional unpublished data retrieved from the authors.
Risk of bias
Item	Authors' judgement	Description
Is the empirical study at low risk of selection bias?	Yes	Quote: "Eight percent (24/297) of reviews did not have a protocol sourced next to the review under the "Protocol and previous versions" section on the Cochrane Library. The reason was not provided by two lead systematic reviewers. Seven (2% of 295 reviews) did not have a protocol: five reviewers went straight from registered title to review and two reviews were published by an alternative source and were later updated and developed into a Cochrane Review using Cochrane guidelines. For the remaining 15 reviews, the reviewer authors sent a copy of the protocol. These protocols were missing from the "Protocol and previous versions" section of the Cochrane Library because a) the review was split into a number of separate reviews and only one protocol was registered (9 reviews), b) a draft protocol was accepted by the Cochrane Review Group (CRG) but was not registered on the Cochrane Library as it was never formally published (4 reviews) and c) the reviewer thought the protocol was registered on the Cochrane Library but its source location could not be found (2 reviews). For this last category, the CRG was contacted and the protocols had been withdrawn from the Library on the advice of the Collaboration because they were seen to be out of date. Thus 288 protocol‐review pairs were available for study." Comment: This empical study included all systematic reviews with an available protocol, which were published during a specified time period.
Is the empirical study at low risk of selective reporting bias?	Yes	Comment: Data for all outcomes and analyses specified in an unpublished study protocol (provided by the authors on request) were either reported in the publications or provided by the authors.