Table 1.

Main reasons for disagreements in cases of a different interpretation of the same information

Risk of bias Item	Main reasons for disagreements	N (%)*	Examples of support for judgement from the review†
Random sequence generation	Consider differently incomplete or unclear description	73 (83)	‘States ‘cluster randomisation by computer’’; Low risk of bias ‘Cluster randomisation by computer. No further information provided’; Unclear risk of bias
	Confusion with allocation concealment	9 (10)	‘Allocation was done using sealed envelopes containing name of one of the two groups.’; Low risk of bias
Allocation concealment	Consider differently incomplete or unclear description	49 (33)	‘Not specified.’; High risk of bias ‘Method of concealment not described.’; Unclear risk of bias
	Consider differently envelopes description	26 (17)	‘Sequentially numbered sealed envelopes’. Does not state if opaque envelopes.’; Unclear risk of bias ‘Sequentially numbered sealed envelopes.’; Low risk of bias
	Random sequence generated by computer or external centre considered enough for low risk	21 (14)	‘Treatment was allocated based on the computer-generated no list.’; Low risk of bias
	Confusion in the definition of the item	19 (13)	‘Researchers attempted to contact all patients seen by physicians during 1 month’; High risk of bias
	Confusion with blinding	15 (10)	‘Participants were told to which compound they had been allocated.’; High risk of bias
	Confusion with random sequence generation	6 (4)	‘Computer-generated randomised lists.’; Low risk of bias
Blinding of participants and personnel	Assess risk differently if blinding was not feasible because of the type of intervention	20 (36)	‘Not possible to blind participants’; Low risk of bias ‘Participants were not blinded for provided treatment. This is inherent to study design’; High risk of bias
	Outcome considered not influenced by blinding	12 (21)	‘No information given about whether patients were blind to physician allocation but treatment outcomes judged unlikely to be affected by lack of blinding’; Low risk of bias
	Consider differently information of ‘double blind’	9 (16)	‘Quote: ‘ … patients were randomised in double-blind conditions … ‘Comment: probably done’; Low risk of bias ‘Quote: ‘double blind conditions’. No further details.’; Unclear risk of bias
	Consider differently incomplete or unclear description	7 (12)	‘Researchers were blind until after the baseline assessment. participants were not blinded.’; Unclear risk of bias ‘Not possible to blind participants to intervention. Insufficient information to make a judgement about blinding of therapists’; High risk of bias
	Confusion in the definition of the item	5 (9)	‘Described as an ‘open-label’ pilot study.’; Low risk of bias
Blinding of outcome assessment	Consider differently incomplete or unclear description	24 (34)	‘Not explicitly discussed in the publish study, it was assumed to be open label’; High risk of bias ‘Not described in published study’; Unclear risk of bias
	Outcome considered not influenced by blinding	16 (23)	‘Not stated, but it was unlikely that the outcome was influenced by lack of blinding’; Low risk of bias
	Consider differently patient-reported outcomes when patients are blinded or not to the intervention	9 (13)	‘Comment: depression assessed by patient self-report’; High risk of bias ‘Insufficient information available to assess’; Low risk of bias
	Consider differently information of ‘double blind’	9 (13)	‘Quote: ‘ … double blind’ Comment: probably done’; Low risk of bias ‘Quote: ‘double blind conditions’. No further details.’; Unclear risk of bias
	Assess risk differently if blinding was not feasible because of the type of intervention	6 (9)	‘Blinding not possible due to intervention’; High risk of bias ‘Unclear blinding of outcome assessment’; Low risk of bias
Incomplete outcome data	Use different cut-off for the rate of missing data	57 (26)	‘11 withdrawals (10%).’; Low risk of bias ‘Comment: there were post-randomisation drop-outs’; High risk of bias
	Focus on no versus reasons/precise report of missing data	28 (13)	‘20 drop-outs (27.2%) with 4 deaths (3 males, 1 female) from cardiovascular events’; High risk of bias ‘Numbers and reasons for drop-outs and withdrawals in all intervention groups were described.’; Low risk of bias
	Consider differently incomplete or unclear description	27 (12)	‘Women who were untraceable or unsuitable for follow-up were excluded, other losses included as smokers’; Low risk of bias ‘167/1287 (12.9%) (C=83, I=84) excluded from analysis due to moving away, being untraceable or deemed unsuitable for follow-up (eg, miscarriage). 1120 in sample. 51/1287 non-responders were included as continuing smokers.’ High risk of bias
	Consider differently intention-to-treat (ITT) analysis	25 (11)	‘147 randomised; r4 in the letrozole group and 3 in the laparoscopic ovarian drilling dropped out of the trial, all for non-compliance. However, ITT analysis was not conducted.’; Unclear risk of bias ‘7 women lost to follow-up, but similar (3 vs 4) in both groups; losses due to non-compliance’; Low risk of bias
	Consider differently report of ‘no missing data’	22 (10)	‘Did not report number of withdrawals. Comment: all patients who were randomised were included in the final analysis. ITT analysis was conducted.’; Unclear risk of bias ‘It does not appear that there were any withdrawals or drop-outs’ Low risk of bias
	Consider differently imputation of missing data	20 (9)	‘Imputation method not described’; Unclear risk of bias ‘Drop-out rate was not significant’; Low risk of bias
	Use different cut-off for difference in the rate missing data between different arms/comparisons	13 (6)	‘Drop-out higher in placebo group (35% vs 25% in budesonide group). ITT used.’; High risk of bias ‘Similar rates of withdrawal between arms. Withdrawals: 36 BUD, 51 placebo’; Low risk of bias

*Number of RCTs disagreeing for this reason; percentage over the total of disagreements for different interpretation.

†When two extracts are reported, they refer to the same study.

RCT, randomised controlled trial.