Table 3.
Study | Sample (search strategy) | Comparison | Results | Direction of bias |
---|---|---|---|---|
Chalmers 198318 | 145 controlled trials of treatment for acute myocardial infarction (systematic) | Studies with different allocation schemes (non-random, non-concealed random, and concealed random allocation) on maldistribution of prognostic variables, frequency of significant outcomes, and case fatality rates | In non-RCTs, non-concealed RCTs, and RCTs with concealed allocation, the maldistribution of prognostic factors was 34%, 7%, and 3.5% respectively, frequency of significant outcomes was 25%, 11%, and 5% respectively, average relative risk reduction for mortality was 33%, 23%, and 3% respectively. Case fatality rate for control groups was 32%, 23%, and 16% and for treatment groups was 21%, 18%, and 16% respectively | Overestimation of effect |
Schulz 199519 | 250 RCTs from 33 meta-analyses (Cochrane Pregnancy and Childbirth Database) | Association between methodological features of controlled trials (allocation concealment, double blinding, and follow up), and treatment effect (odds ratio) | Treatment effect overestimated by 41% in RCTs with inadequate concealment and by 30% in RCTs with unclear adequacy of concealment compared with those with adequate concealment (P⩽0.001) after adjustment for other methodological features. Studies with no double blinding overestimated treatment effect by 17% compared with double blinded studies (P =0.01). Lack of complete follow up had no influence on treatment effect (7%, P=0.32) | Overestimation of effect |
RCT=Randomised controlled trial.