Table 2.

Randomised controlled trials (RCTs) compared with non-randomised controlled trials (non-RCTs) across different interventions

Study	Sample (search strategy)	Comparison	Results	Direction of bias
Colditz 198915	113 studies published in 1980 comparing new interventions with old, identified in leading cardiology, neurology, psychiatry, and respiratory journals (systematic)	36 parallel RCTs, 29 randomised COTs, 46 non-randomised COTs, 3 CCTs, 5 ECTs, 9 observational studies compared for “treatment gain” (Mann-Whitney statistic), and relation between quality score and “treatment gain” assessed	All but one design achieved similar “treatment gains” (0.56-0.65). Overall, 89% of new treatments were rated as improvements, but only non-randomised COTs detected a significantly higher “treatment gain” from the new treatment compared with RCTs (P=0.004). Within RCTs, there was no correlation between quality score and “treatment gain” (P=0.18)	Inconclusive
Miller 198916	188 studies comparing new surgical interventions with old, published in 1983 and identified in leading surgical journals (systematic)	81 RCTs, 15 CCTs, 27 HCTs, 91 observational studies, 7 BASs compared on “treatment gain” (Mann-Whitney), and association between treatment success and study design and the relation between quality score and treatment gains assessed	Non-significant trend towards larger “treatment gains” for new treatments on the principal disease in non-RCTs (0.56 to 0.78) than in RCTs (0.56). For treatment of complications the “treatment gain” was similar across all study designs (0.54 to 0.55) except in BASs (0.90). Within RCTs, there was no correlation between quality scores and treatment gains (P=0.7)	Inconclusive
Ottenbacher 199217	Sample of 30 RCTs and 30 trials with non-random process of allocation, eg matching or HCTs (systematic search of N Engl J Med and JAMA across several medical specialties)	RCTs and non-RCTs on treatment effects as measured by standardised mean differences	No difference in treatment effect found between non-RCTs (0.23) and RCTs (0.21)	Similar effects

COT=Crossover trial; CCT=concurrently controlled trial; ECT=external control study; BAS=before and after study; HCT=historically controlled trial.