Challenging Received Wisdom: Antidepressants and the Placebo Effect

Abstract

This article explores the reaction when an article challenging received wisdom is published and covered extensively by the media (1). The article in question was a meta-analysis of antidepressant clinical trials indicating that for most patients, difference between drug and placebo was not clinically significant. Reactions ranged from denial that the effects of antidepressants are so small to criticisms of the clinical trials that were analyzed. Each of these reactions is explored and countered.

“Doctors and patients know what works and what does not.”

(C. Vargas, February 27, 2008, PLoS Medicine)

“Clinical practice plus millions of content patients can't be that wrong.”

(R. Werner, February 27, 2008, PLoS Medicine)

On February 26, 2008, PLoS Medicine published a meta-analysis that my colleagues and I had conducted on antidepressant medication (1). Most meta-analyses suffer from publication bias, which can happen when pharmaceutical companies withhold unsuccessful trials from publication (2, 3). To circumvent this, we used the Freedom of Information act in the U.S. to obtain the data on all clinical trials submitted to the Food and Drug Administration (FDA) for the licensing of the four new-generation antidepressants.

The results of our meta-analysis showed that people got better on medication, but they also got better on placebo, and the difference between the two was small. In fact, it was below the criterion for clinical significance established by the National Institute for Health and Clinical Excellence (NICE), which sets treatment guidelines for the National Health Service in the UK. Clinical significance was found only in a few relatively small studies conducted on patients with extremely severe levels of depression.

This study was the subject of widespread media attention, especially in the UK, where it was front page news in most of the national daily newspapers (4). Not surprisingly, it stirred considerable controversy. In this article, I examine some of the reactions to the meta-analysis.

“PLACEBOS COULD NOT PRODUCE EFFECTS LIKE THESE”

Placebo effects can be surprisingly strong. Placebos can reverse the effects of powerful medications. They can affect the body as well as the mind. They produce side effects as well as beneficial effects. They can produce symptoms and alleviate them. In this section, I look at the power of belief to produce profound changes in people’s experience.

One of the earliest reports on the power of placebo was the seminal work of Stewart Wolf, who demonstrated the ability of placebos to block the effect of potent drugs (5). A pioneer in the investigation of placebo effects, Wolf reported three successful experimental attempts at reversing the effects of active medications that typically induce abdominal discomfort. In each case, the reversal was brought about by misinforming the subject about the nature of the drug being administered, and in each case the subjective changes were verified by physiological assessment. One of Wolf's subjects was a 28-year-old pregnant woman who was suffering from nausea and vomiting. Wolf gave her ipecac, a drug that interrupts normal gastric contractions, thereby inducing vomiting and nausea. Although ipecac is commonly used to induce vomiting when toxic substances have been swallowed, Wolf misinformed his patient that it was a medicine which would alleviate her nausea. Prior to taking ipecac, the patient displayed an absence of gastric contractions. Within 20 minutes of ingesting the drug, normal gastric contractions resumed and the nausea ended.

Placebos have been reported to produce some rather startling effects on skin conditions. The most impressive of these reports involves the suggestion-related production and inhibition of contact dermatitis (6). Contact dermatitis is a skin condition produced by chemical substances to which people have become sensitized. In the study reported by Ikemi and Nakagawa, 13 students were touched on one arm with leaves from a harmless tree, but were told that the leaves were from a lacquer or wax tree (Japanese trees that produce effects similar to poison ivy and to which the boys had reported being hypersensitive). On the other arm, the subjects were touched with poisonous leaves, which they were led to believe were from a harmless tree. All 13 subjects displayed a skin reaction to the harmless leaves (the placebo), but only two reacted to the poisonous leaves.

Although a meta-analysis published in the New England Journal of Medicine concluded that the placebo effect is not very powerful (7), Wampold and his colleagues have reanalyzed those data and calculated the number needed to treat (NNT) for placebo compared to no treatment at all (8). NNT is the number of patients that need to be treated to achieve one success by means of a particular treatment. So the smaller the NNT, the larger the effect. Compared to no-treatment, the NNT for placebo is 7. Although this is not a large effect, it is instructive to compare it to the NNT for various accepted medical treatments, as published in a growing database of published studies provided online (http://www.cebm.utoronto.ca/glossary/nntsPrint.htm) by the University of Toronto’s Centre for Evidence-Based Medicine. The NNT for radiotherapy for breast cancer, for example, is shown on that database to be 8, that for beta-blockers for chronic heart failure is 24, the flu vaccine has an NNT of 12, and aspirin as a prophylactic for myocardial infarction has an NNT of 208.

The NNT of 7 for placebo treatment was calculated across studies of many different clinical conditions. But there is good reason to believe that the placebo effect should be even greater on depression. This is because hopelessness is a core feature of depression, and one of the presumed effects of a placebo is to instill hope (9). If you ask depressed patients what the worst thing in their life is, many will tell you that it is their depression. They feel stuck in an intolerable condition and they are hopeless about the possibility of getting better (10). So it stands to reason that a treatment promising relief would bring some relief, merely on the basis of hope-instilling promise. Indeed, a meta-analysis of the published antidepressant literature indicates that the placebo effect (placebo – no-treatment) is twice as large as the drug effect (drug – placebo) (11).

“ANTIDEPRESSANTS WORK IN CLINICAL PRACTICE”

Clinical experience shows that antidepressant drugs work, in the sense that patients get better when given medication. So do our meta-analyses. Patients given antidepressants in the clinical trials showed substantial and clinically significant improvement, as did those given placebo. Physicians do not generally prescribe placebos to their patients. Hence they have no way of comparing the effects of the drugs they prescribe to placebos. When they prescribe a treatment and it works, quite naturally they ascribe the cure to the treatment. But the history of medicine is replete with cures that were “known” to work by doctors and their patients. These apparently effective treatments, that we now consider to have been placebos, include dolphin’s genitalia, lizard’s blood, crocodile dung, pig's teeth, putrid meat, frog’s sperm, powdered stone, human sweat, worms, and spiders. That is why placebo-controlled trials are required in order to demonstrate drug efficacy. When the administration of a drug is followed by improvement, the improvement might not be due to the drug’s chemical composition. Placebo-controlled trials are used to separate the drug effect from such factors as the placebo effect, spontaneous remission, and regression towards the mean.

“YOU HAVEN’T PROVEN THAT ANTIDEPRESSANTS DON’T WORK”

This is absolutely true. In fact, our data show a small advantage for drug over placebo that is statistically significant, but not clinically significant. But I will extend this criticism. Not only haven’t we proven that antidepressants don’t work, but we also haven’t proven that their effect is below the threshold of clinical significance. What we have shown is that the data upon which drug approval was based does not show clinical significance. But it is always possible that future studies, perhaps with better experimental methods or measures of depression, will show a greater effect.

Possibility is a long way from fact, however. The onus should not be on critics to demonstrate that a treatment is ineffective, but rather for proponents to demonstrate that it is. If all that is needed is an absence of proof that a treatment does not work, then perhaps we ought to resume using treatments crocodile dung and dolphin genitalia until well-enough designed clinical trials prove conclusively that they are not more effective than placebo.

“ANTIDEPRESSANTS DRUGS MIGHT WORK FOR SOME PEOPLE BUT NOT OTHERS”

This is indeed possible. In fact, we found evidence of greater drug effectiveness in a small subset of studies involving patients with exceptionally high levels of initial depression, although it seemed that they were less responsive to the placebo, rather than more responsive to the drug. We also found that the drug-placebo difference was zero for people who were moderately depressed. For this rather large group of sufferers, anti-depressants seemed to have no drug effect at all.

It is feasible that there are other subgroups of patients for which antidepressants are more effective, but simply asserting this possibility is not enough. One must identify those subgroups and demonstrate the clinically-significant benefit they obtain from active medication over placebo. Some of the data for doing so have already been collected. For example, gender is most certainly identified in the data sets of most, if not all, clinical trials. It would be a simple matter, for example, to reanalyze these data to test whether women are in fact more responsive to SSRIs and men to tricylic medication, as has been suggested (12).

Note that if there are some groups of patients that respond better than the overall mean, then there must also be some that respond worse. Given how small the advantage is over placebo overall, this should be of considerable concern. If some are responding substantially better, then others must not be responding at all—or even being made worse by the active medication, compared to how they would have fared on placebo. If this is the case, it would be important to know it so that antidepressant medication could be prescribed more selectively.

Understandably, pharmaceutical companies might be reluctant to carry out analyses of this sort, as they have the potential to cut into sales substantially. One way around this is to require that raw data for all approved medications be available for researchers to reanalyze. This could easily be done with full protection of the anonymity of those who participated as subjects. Greater transparency and availability of the data would be in the best interests of patients, doctors, third-party payers, health researchers, and government agencies.

“THE CLINICAL TRIALS ARE FLAWED”

Defenders of antidepressants have noted a number of flaws in the clinical trials used to evaluate them. One is that the patients in these trials were not depressed enough. In fact, using the American Psychiatric Association classification scheme, the mean baseline severity was in the very severe range for all but one of the trials we analyzed. The one exception was a clinical trial involving moderately depressed patients, in which the response to drug was virtually identical to the response to placebo.

“Very severe” is the most severe category in the classification scheme. So how can it be asserted that the patients were not depressed enough? One possibility is that clinical trial researchers distort the data. According to a spokesperson for the FDA, patients may be rated as more severely depressed than they actually are so that they will qualify for the trial (4). Now if this is true, then the response of treatment is even less than the clinical trials indicate, unless of course the researchers also inflate scores at the end of the trial. Equally troubling is the idea that researchers are intentionally distorting the data in any way. These trials are the basis for drug approval. If the data have been distorted, then perhaps the drugs should not have been approved in the first place.

Whereas some critics have complained that the patients in the clinical trials we assessed were not depressed enough, others have argued that they were too depressed. An editorial in Nature Reviews Drug Discovery, for example, complained that “All but one trial analyzed involved groups with mean initial depression scores in the ‘very severe’ range, limiting the strength of extrapolations” (13). Of the two contradictory criticisms, this is the more cogent. The drug companies did not conduct any trials on patients between the very severely depressed and moderately depressed categories. But here too, the evidence should be on the compani s to demonstrate efficacy for the severely—but not very severely—depressed patients.

There may be many flaws in clinical trials, including relatively short durations, unrepresentativeness of the sample, and the breaking of blind by patients and doctors on the basis of side effects (14). But these trials are the data on the basis of which the drugs were approved. If they are flawed, then we have no evidence of drug effectiveness, and the drugs should not have been approved in the first place.

“DON’T ASK; DON’T TELL”

Finally, some have argued that even if the drugs don’t work, it was wrong of my colleagues and me to publish our studies. We shouldn’t tell patients that the drugs don’t work because it will undermine their faith in treatment. I disagree. Without accurate knowledge, patients and physicians cannot make informed treatment decisions, researchers will be asking the wrong questions, and policymakers will be implementing misinformed policies. If the antidepressant effect is largely a placebo effect, it is important that we know this. It means that improvement can be obtained without reliance on addictive drugs with potentially serious side effects (15, 16).

Biography

Irving Kirsch is professor of psychology the University of Hull. He has published eight books and more than 200 scientific journal articles and book chapters on placebo effects, antidepressant medication, hypnosis, and suggestion. His recent meta-analysis on the efficacy of antidepressants has been covered extensively in the international media, and his previous analyses influenced the current guidelines on the treatment of depression in the United Kingdom.