RCTs are here to stay

Dear Sir,

The article on randomized controlled trials (RCTs) I refer to originates from an invited debate at the EOS meeting in Reykjavik (1). With some concern I read the second part of the article arguing against the use of RCTs in orthodontics. It is right to point out that RCTs have some inherent limitations. Of course not every RCT comes close to clinical reality, and of course we cannot randomize everything. Nonetheless, I think that the criticism in the second half of the article goes too far. In order to give an example of the Hawthorne effect, a study from pain therapy is cited. But pain perception is not a merely physiologic parameter, instead it is a sensation that is influenced by multiple social, cultural, and psychological factors. All this does not hold true for bone cell activity, which is responsible for tooth movement, and which will hardly be affected by such factors. In orthodontics, the only parameter that potentially could be affected by the Hawthorne effect would be the wear time of removable appliances. However, RCTs assessing class-II-therapy (2) resulted in smaller therapy effects than those of earlier retrospective studies (3). If the Hawthorne effect had been of importance in these RCTs, treatment effects should have surpassed those of retrospective studies. Furthermore, there is no indication that compliance rates in RCTs on class-II-therapy exceeded normal levels (4–6). Thus, we can conclude that the impressive skeletal effects of many retrospective studies on class-II-therapy (7–9) reflect nothing else than the more or less marked selection bias of their authors, while the RCTs on the same subject seem to come close to the—disappointingly small—‘true’ effects.

Interestingly, in medical research the same dependence of effect size from study quality could be found. In a meta-analysis, it was found that non-randomized studies yielded 0.15, studies without blinding 0.11 better results than randomized and blinded studies (Mann–Whitney statistics). Both would mean a significant difference not only statistically, but also clinically (10). In a similar meta-analysis, the authors found an average treatment benefit 52% for low-quality trials and 29% for high-quality trials (11). Thus, the strong interrelation of study design and effect size can be accepted as a well established fact in medical research. With this in mind there is no reason to declare orthodontics to be a special area of medicine with no need for RCTs. Meikle’s contention that RCTs had contributed nothing more than older retrospective studies and animal research (12), cited even two times in the second half of the article, could not be more faulty. Orthodontic RCTs provided the most sound and trustworthy answers to many clinical questions and will continue to do so. There is no reason to deviate from that way!