How much of orthodox medicine is evidence based?

Scientific heavyweights deplore the NHS money wasted on unproved and disproved treatments used by practitioners of complementary and alternative medicine (CAM),^{1 2} but Lewith, a CAM proponent [see previous letter], is cited elsewhere as saying that the BMJ reckons that 50% of the treatments used in general practice aren't proved, and 5% are pretty harmful but still being used.³

His data were taken from the BMJ Clinical Evidence website (http://clinicalevidence.com/ceweb/about/knowledge/jsp, viewed 6 May 2007). A pie chart indicates that, of about 2500 treatments supported by good evidence, only 15% of treatments were rated as beneficial, 22% as likely to be beneficial, 7% part beneficial and part harmful, 5% unlikely to be beneficial, 4% likely to be ineffective or harmful, and in the remaining 47% the effect of the treatment was “unknown.” The text says, “The figures suggest that the research community has a large task ahead and that most decisions about treatments still rest on the individual judgements of clinicians and patients.” On 9 October 2007 the situation had changed—but not for the better. Treatments rated “beneficial” had decreased from 15% to 13%. The associated text is unchanged.

Acute low back pain is a common and well investigated condition. BMJ Best Treatments reports that back pain affects 70-85% of all adults, and each year almost half of us get back pain that lasts at least a day (http://besttreatments.bmj.com/btuk/conditions/1559.html). There are 18 treatments for acute low back pain which have been tested by randomised controlled trials (RCTs), of which two (11%) were graded “beneficial” and 13 (72%) “unknown.” The accompanying table shows all of the 18 treatments for acute low back pain and their rated effect. According to this table, a condition that is extremely common, and for which many treatments have been intensively researched, has an even higher than average proportion of treatments that are labelled “unknown” efficacy, or in other places “need further study.” There must be some mistake.

The solution to the mystery is that the label “unknown” does not mean, “We have no knowledge of the effect of this treatment because it has not been tested in an RCT.” Astonishingly, it means, “We have tested this treatment in several RCTs, but on balance there is currently no convincing evidence that it is effective for this condition.” So really the efficacy of these 13 treatments for acute back pain is not “unknown” but “not demonstrated.”

Lewith's interpretation of the pie chart is highly misleading. The research community has been commendably diligent, but of course RCTs often fail to find that certain treatments are effective. Euphemisms such as “unknown” or “needs more study” for the inefficacy of such treatments may soothe the feelings of proponents of those treatments that have so far failed to show efficacy, but it does an injustice to the researchers who obtained these data, and misleads both practitioners and patients about the extent to which orthodox medicine is evidence based. It is particularly ironic that CAM therapies are over-represented in the ‘‘not shown to be effective” category, so if anyone should be concerned about lack of evidence it should be CAM practitioners rather than conventional medics.