Table 1.

Randomised controlled trials (RCTs) compared with non-randomised controlled trials (non-RCTs) of the same intervention

Study	Sample (search strategy)	Comparison	Results	Direction of bias
Chalmers 19777	32 controlled studies of anticoagulation in acute myocardial infarction (systematic)	RCTs with CCTs and HCTs on case fatality rate, rate of thromboembolism, and haemorrhages	Relative risk reduction for mortality overestimated by 35% in HCTs and 6% in CCTs compared with RCTs. Case fatality rate highest in HCTs (38.3%) compared with RCTs (19.6%) and CCTs (29.2%). Similar pattern for thromboembolism	Overestimation of effect
Sacks 19828	Sample of 50 RCTs and 56 HCTs, assessing 6 interventions (treatment of oesophageal varices, coronary artery surgery, anticoagulation in myocardial infarction, chemotherapy for colon cancer and melanoma, and diethylstilboestrol for recurrent miscarriage) (at hand)	RCTs with HCTs on frequency of detecting statistically significant results (P⩽0.05) of primary outcome and reduction of mortality	20% of the RCTs found a statistically significant benefit from the new treatment compared with 79% of the HCTs. Relative risk reduction of mortality in HCTs v RCTs was 0.49/0.27 (1.8) for cirrhosis, 0.68/0.26 (2.6) for coronary artery surgery at 3 years, 0.49/0.22 (2.2) for anticoagulation in myocardial infarction, and 0.67/−0.02 for diethylstilboestrol in recurrent miscarriage. Outcomes in treatment groups were similar in both designs, but outcomes in control groups were worse among historical controls	Overestimation of effect
Diehl 19869	19 RCTs and 17 HCTs for 6 types of cancer (breast, colon, stomach, lung cancer, melanoma, soft tissue sarcoma) (reference lists of two textbooks)	Matching of randomised and historical controls for disease, stage, and follow up, and comparison on survival and relapse free survival	18 of 43 matched control groups (42%) varied by >10% (absolute difference in either outcome), 9 (21%) by >20%, and 2 (5%) by >30%. Survival or relapse free survival was better in RCTs compared with HCTs in 17/18 matches	Overestimation of effect
Reimold 199210	6 RCTs and 6 CCTs of chinidine in atrial fibrillation (systematic)	RCTs and CCTs on maintenance of sinus rhythm 3, 6, and 12 months after cardioversion	At 3 months, beneficial effect of maintaining sinus rhythm with chinidine was 54% less in non-RCTs compared with RCTs, and was 76% less at 12 months	Underestimation of effect
Recurrent Miscarriage Immunotherapy Trialists Group 199411	9 RCTs and 6 CCTs (with self selected treatment) of allogenic leucocyte immunotherapy for recurrent miscarriage (systematic)	RCTs and CCTs on live birth rate	Beneficial effect of immunotherapy on birth rate among pregnant women was 9% larger in CCTs compared with RCTs, but was 63% lower in CCTs when all women were considered	Underestimation of effect when all women considered, similar effect for pregnant women
Watson 199412	4 RCTs and 6 CCTs/HCTs of oil soluble contrast media during hysterosalpingography in infertile couples (systematic)	RCTs and CCTs/HCTs on pregnancy rate	RCTs and CCTs/HCTs detected similar increases in pregnancy rates: odds ratio for RCTs 1.92 (95% CI, 1.33 to 2.68) and for CCTs/HCTs 1.92 (1.55 to 2.38)	Similar effect
Pyörälä 199513	11 RCTs and 22 (not further specified) non-RCTs on hormonal therapy in cryptorchidism (systematic)	RCTs and non-RCTs on the descent of testes after therapy with luteinising hormone releasing hormone or human chorionic gonadotrophin	Success rate of descent of testes after therapy with luteinising hormone releasing hormone was 2.3 times larger in non-RCTs than in RCTs and 1.7 times larger after therapy with human chorionic gonadotrophin	Overestimation of effect
Carroll 199614	17 RCTs and 19 non-RCTs (including HCTs or trials with inadequate randomisation procedures) on transcutaneous electrical nerve stimulation (systematic)	RCTs and non-RCTs on control of postoperative pain	Transcutaneous electrical nerve stimulation judged ineffective at improving postoperative pain in 85% of RCTs, while 89% of non-RCTs concluded that it did improve postoperative pain	Overestimation of effect

CCT=concurrently controlled trial; HCT=historically controlled trial.