Antidepressant trials generally have methodological defects

Editor—The study reported by Woelk tried to show the antidepressant properties of St John's wort,¹ but its methodology was subsequently much criticised.² We put the four major criticisms together to form a short “methodological quality” checklist and added one item regarding the integrity of blindness: the checklist assessed use of remission criteria; use of a three arm design; use of an active placebo; individual determination of the dose of the reference compound; and evaluation of the integrity of blindness.

We have stated that blindness is a major defect in antidepressant trials and that it has often jeopardised their validity.³ In the study reported by Woelk, because of the tolerance profile of imipramine it was probably easy for the patients (and evaluators) to guess whether they were receiving St John's wort or imipramine. The expectations of both patients and evaluators may thus have biased the results.

To check whether the criticisms about the study reported by Woelk were addressed in “regular” antidepressant trials we performed a Medline search for the year 2000. We identified 19 randomised controlled trials that evaluated the short term efficacy of regular antidepressants in major depression and reviewed them and rated them according to the above checklist. Two of the studies addressed three requirements in the checklist, five addressed two, and 12 addressed one or none. Interestingly, these 12 studies did not elicit any critical correspondence in the three issues of the journals that followed publication. In conclusion, many trials that evaluated the efficacy of regular antidepressants bore the limitations observed in the study reported by Woelk.

If the criticisms against the study reported by Woelk are correct then St John's wort still needs to prove itself. But this may also hold for some regular antidepressants. The general methodological defects of antidepressant trials smooth out the differences in the efficacy of antidepressants and lead to all antidepressants being lumped into a single category regardless of their efficacy. But the compounds that have shown their efficacy in high quality trials (for example, those with a three arm design) should not be lumped with those that have not.

Only better designs and methods could enable discrimination between compounds that were and were not efficacious; this holds for new and alternative compounds as well as for classic ones.