Publication Bias and the Pharmaceutical Industry: The Case of Lamotrigine in Bipolar Disorder

Abstract

Publication bias, especially the lack of publication of negative treatment studies, is known to be a major problem in the medical literature. In particular, it appears that the pharmaceutical industry is not routinely making data from negative studies available through the published scientific literature. In this paper, we review the case of studies with lamotrigine in bipolar disorder, describing evidence of lack of efficacy in multiple mood states outside of the primary area of efficacy (prophylaxis of mood episodes). In particular, the drug has very limited, if any, efficacy in acute bipolar depression and rapid-cycling bipolar disorder, areas in which practicing clinicians, as well as some academic leaders, have supported its use. The negative unpublished data now made available on lamotrigine provide an important context for clinical practice and research, and also raise important scientific and public policy concerns about having access to studies showing inefficacy with psychotropic medications.

Introduction

Publication bias is known to be a major problem in the medical literature.[1–5] In general, studies with positive outcomes are more likely to be published than studies with negative outcomes.[3] This finding also extends to the psychiatric literature.[6] Negative outcomes are here defined as randomized clinical trials (RCTs) which fail to find statistically significant benefit with an active treatment as opposed to a control group in an adequate sample size.[7] The nonpublication of negative outcomes in RCTs has been especially reported to occur in pharmaceutical industry-funded studies, as opposed to studies funded by governmental sources.[6,8–12] Recent controversies in varied medical disciplines have highlighted the importance of having access to such unpublished data; previously underappreciated unpublished side effect data have led to removal of certain agents, such as the anti-inflammatory agent rofecoxib, from the US market.[13,14] Marketing of other agents, such as gabapentin, for unproven treatment indications such as bipolar disorder despite initially unpublished studies showing absence of efficacy in that condition also has led to controversies and even malpractice lawsuits.[15,16] A recent review of antidepressant RCTs in unipolar depression in the Food and Drug Administration (FDA) database found that almost all negative studies were unpublished, leading to a false impression in the published literature that 93% of antidepressant RCTs had positive results; when unpublished studies were included, 51% of all RCTs were positive and 49% were negative.[17] Thus, unpublished studies raise questions of concerns regarding both underreported risks and underreported limitations in efficacy.

In response to some of the above controversies, in 2002, the US lobbying group for the pharmaceutical industry (Pharmaceutical Research and Manufacturers Association; PhRMA) published its “Principles on the Conduct of Clinical Trials and Communication of Clinical Trial Results” (http://www.phrma.org/publications/policy_papers/phrma_clinical_trial_registry_proposal/) and committed to voluntarily registering all clinical trials at initiation, as well as providing access to the results of all study outcomes, positive or negative, via company-based (specific Web sites or www.clinicalstudyresults.org) or government-based Web sites (www.clinicaltrials.gov).

In this paper, we access those Web sites of companies with agents approved for the treatment of bipolar disorder, to examine the availability of study results with negative data, and then we focus on one of those agents with availability of negative data, lamotrigine, to examine the impact of those results on clinical interpretation of the utility of that agent in bipolar disorder.

Methods

We searched the Web sites of the following companies with FDA-approved drugs for the treatment of bipolar disorder patients: Abbott Laboratories (Depakote brand of divalproex sodium), GlaxoSmithKline (Lamictal brand of lamotrigine), Pfizer Inc. (Geodon brand of ziprasidone), Eli Lilly and Co. (Zyprexa, Symbiax brands of olanzapine and olanzapine-fluoxetine combination), Janssen Pharmaceutica (Risperdal and Risperdal Consta brand of risperidone), Bristol Myers Squibb (Abilify brand of aripiprazole), and AstraZeneca (Seroquel brand of quetiapine). We supplemented searches of the primary company Web sites with searches of the Web sites for each drug. If no studies were identified, or if links were provided, we accessed the Web site established by the Pharmaceutical Research Manufacturers Association (PHRMA) for clinical trial results (www.clinicaltrialresults.org) and that of the National Institutes of Health (NIH) for current clinical trials (www.clinicaltrials.gov). Table 1 describes the studies found on their Web sites, whether they were positive or negative, and whether they were previously published. Published studies of the same drugs were assessed by a computerized MEDLINE search of the National Library of Medicine (NLM) database (1966–2006).

Table 1.

Studies of Drugs on Bipolar Disorder on Pharmaceutical Industry Web Sites

Company	Drug	Web site visited	Ease of navigation	Results	If limited data on company Web site, findings on www.clinicalstudyresults.org	Type of information provided	# of studies on company Web site or PHRMA Web site	# of previously published studies	# of previously published negative studies	# of previously unpublished negative studies
Abbott	Depakote (divalproex sodium)	www.abbott.com	Link to “Depakote”; link to “Professionals”; link to “Proven Efficacy.” No further data.	No data	1 unpublished positive study for Depakote ER in acute mania[38]; no data provided. No reference to 2 previously published positive studies in acute mania,[39,40] and a failed maintenance study.[41]	Only title of study	1	0	0	0
AstraZeneca	Seroquel (quetiapine)	www.astrazeneca.com	Link to “AstraZeneca-US”; Link to “Products”; Link to “Seroquel”; Link to “Healthcare professionals.” No further data.	4 in acute mania reported, 3 positive have been published,[42–44] and 1 negative unpublished study in acute mania;[45] 1 positive published study in acute bipolar depression[46]	NA	Full protocol with detailed summary of results. Mean ± SD length 8.6 ± 1.1 pages (range 7–10)	6	5	0	1
Bristol-Myers Squibb	Abilify (aripiprazole)	www.bms.com	Link to “Clinical trials disclosure”; Link to “Clinical trials registry”; Link to “Therapeutic areas”; Link to “Psychiatric disorders”;	5 studies in acute mania, bipolar depression, or mania cited by titles and listed as “Recruiting” with no further information; 3 studies cited as “no longer recruiting” and 1 as “completed.” No citations provided.	No titles or data. No reference to 2 previously published positive studies.[47,48]	Titles of studies only	5	2	0	1
Eli Lilly	Zyprexa (olanzapine)	www.elililly.com	Link to “Products”; link to “Lilly clinical trials”; link to “Trials results by product”; link to “Zyprexa”	6 studies of acute mania, 3 studies of relapse prevention, all positive published studies[49–57]	NA	Full protocol with detailed summary of results. Mean ± SD length 15.6 ± 8.0 pages (range 6–28)	10	9	1	0
Eli Lilly	Symbiax (olanzapine-fluoxetine combination)	www.elililly.com	Link to “Products”; link to “Lilly clinical trials”; link to “Trials results by product”; link to “Symbiax”	1 positive study in acute bipolar depression[58]	NA	Full protocol with detailed summary of results. Length 16 pages	1	1	0	0
GSK	Lamictal (lamotrigine)	www.gsk.com	Link to “Clinical trial register”; link to “Lamotrigine”	See Table 2	NA	See Table 2	9	2	2	5
Janssen	Risperdal (risperidone)	www.janssen.com	Link to “Our products”; link to “Risperdal”; link to “Risperdal for bipolar mania”	No data	No titles or data. No reference to four previously published positive studies in acute mania,[59–62] or one previously published negative but underpowered study in acute bipolar depression[63]	No titles or data	0	4	1	0
Pfizer	Geodon (ziprasidone)	www.pfizer.com	Link to “Medicines and products”; link to “Geodon”; link to “For healthcare professionals”; link to “Clinical trial data”; link to “Geodon for bipolar mania and mixed episodes”	One efficacy graph from 1 published study provided[64]	2 positive studies in acute mania, 1 published[64] and 1 unpublished.[65] No reference to another positive published acute mania trial[66]	Full protocol with detailed summary of results. Mean ± SD length 7.0 ± 2.0 pages (range 5–9)	2	2	0	0

Negative vs Failed Trials

Our use of the term “negative” to describe the outcomes of studies is defined as follows: In the strictest definition, this refers to a study in which the experimental drug is the same as placebo, and in which a proven active control is more effective than placebo.[18] If an active control fails to separate from placebo, as well as the experimental drug, then this study would be seen as a “failed” study, meaning that no conclusion could be drawn for or against efficacy, since an already proven treatment also failed to show benefit. Usually in that case, peculiar characteristics of the sample (such as a low severity of illness leading to high placebo response) can obscure a real treatment effect.[18] In the absence of an active control group, which is often the case for FDA regulation-oriented pharmaceutically sponsored studies, one cannot be definitively certain that equivalence of drug to placebo represents a “negative” outcome.[18] However, if the study is adequately powered to demonstrate a modest effect size, then one can at least conclude that the study is most suggestive of lack of moderate or greater benefit.[7] In that sense, we will define studies as negative in this report if drug is equivalent to placebo in the absence of an active control group. Further, all definitions of difference between drug and placebo will be limited to the primary outcome measures, since secondary outcome measures are associated with inflated chance of statistically significant findings (P value below .05).[19] This limitation does not mean that secondary outcomes are always false or unhelpful, but rather one should look for multiple measures showing robust effects, rather than isolated borderline P value-based analyses.[20]

Where the comparison is of a drug to an active control without a placebo group, usually the purpose of the study is to demonstrate “noninferiority” or “equivalence” to a proven treatment.[7] If the experimental drug is equivalent to or better than the proven treatment, the study is considered positive. If the experimental drug is less effective then the proven treatment, the study is considered negative.[7]

Results

Two companies providing the largest number of studies were Eli Lilly and GlaxoSmithKline (GSK); an overall summary of our results is shown in Table 1. We then focused on the results provided for the agent with the largest number of negative studies available, lamotrigine.

Examining the Lamotrigine Data

Of major US pharmaceutical companies, so far only GSK has provided data on unpublished negative studies with results that were unfavorable to their product lamotrigine (Lamictal) (Table 2).

Table 2.

Published and Unpublished Randomized Clinical Trials of Lamotrigine in Bipolar Disorder

Study	Diagnosis	Design	N	Duration (wks)	Scale	Outcome	Result	Published
SCAA20 SCAA2008	Mania	LTG vs Li vs Pla	216	3	MRS	LTG −9.3 ± 10.9	Failed*	No
						Li −10.7 ± 11.6
						Pla −9.5 ± 10.5
SCAA2009	Mania	LTG vs Li vs Pla	229	6	MRS	LTG −11.6 ± 14.0	Negative**	No
						Li −15.6 ± 13.0
						Pla −11.4 ± 12.3
SCAA2010	Bipolar depression	LTG vs Pla	206	10	HDRS	LTG −10.6 ± 8.3	Negative***	No
						Pla −10.2 ± 8.1
SCA40910	Bipolar depression	LTG vs Pla	257	8	MADRS	LTG −12.2	Negative***	No
						Pla −11.2
						SDs not provided
SCAB2001	Bipolar depression	LTG vs Pla	195	7	HDRS	LTG (200 mg/d) −10.5 ± 8.1 (P = .08)	Negative***	Yes[25]
						LTG (50 mg/d) −9.3 ± 8.9 (P= .24)
						Pla −7.8 ± 7.9
SCAB2005	Rapid cycling	LTG vs Pla	137	32	TIME	Median survival time	Negative***	No
						LTG 142 days
						Pla 133 days
						P = .73
SCAA2012	Rapid cycling	LTG vs Pla	182	34	TIME	Median survival time	Negative***	Yes[24]
						LTG 126 days
						Pla 79 days
						P = .18
SCAB2003	Prophylaxis	LTG vs Li vs Pla	463	72	TIME	Median survival time	Positive	Yes[22]
						LTG 200 days
						Li 170 days
						Pla 93 days
						P = .029 for both Li and LTG
SCAB2006	Prophylaxis	LTG vs Li vs Pla	175	72	TIME	Median survival time	Positive	Yes[23]
						LTG 141days
						Li 292 days
						Pla 85 days
						P = .02 for LTG
						P = .003 for Li

Li=lithium; LTG=lamotrigine; Pla = placebo; TIME = time to Intervention with psychotropic medications due to full relapse or initiation of relapse into mood episodes; MRS = Mania Rating Scale from SADS-C; HDRS = Hamilton Depression Rating Scale; MADRS = Montgomery Asberg Depression Rating Scale

^*Lamotrigine and lithium (active control) were equivalent to placebo, thus no information for or against lamotrigine efficacy can be concluded.

^**Lamotrigine was equivalent to placebo but lithium was more effective than placebo.

^***Lamotrigine was equivalent to placebo in an adequately powered study to demonstrate modest effect sizes.

This was not a voluntary act but rather due to legal judgment brought by the state of New York after a lawsuit about paroxetine use in children.[21] Of the 9 studies with lamotrigine in bipolar disorder provided by the GSK Web site, 2 were positive and published, and supported the company's success in securing an FDA-approved indication for lamotrigine for delay of relapse in the long-term treatment of bipolar disorder patients.[22,23] Two negative studies have been published, 1 in rapid-cycling[24] and another in acute bipolar depression,[25] but both published versions emphasize positive secondary outcomes as opposed to the negative primary outcomes. A negative study in rapid cycling has not been published in detail (GW611), nor have 2 negative randomized studies in acute bipolar depression (GW 40910 and GW 603), or 2 negative randomized trials in acute mania (GW609 and GW 610). However, all of these negative studies are reported at the company Web site, which also refers to a publication[26] that briefly summarizes results from the above 5 negative studies. However, that publication provides little actual data from these negative studies; the results are much more fully provided on their Web site.

Recently, using meta-analysis, academic investigators have published the results of 5 negative studies with lamotrigine in acute bipolar depression in more detail, confirming lack of benefit for primary outcomes.[27] When pooling those 5 studies to produce a large sample of over 2000 subjects, a small effect size of depressive symptom benefit was statistically significant. However, more benefit was seen in a subgroup analysis of those with high severity of depressive symptoms.[27]

Discussion

Given the above analysis, it still appears that negative outcomes, instead of being seen as important results in their own right, are treated by the pharmaceutical industry and many academic authors as if they represent signets, putting a seal on all further discussion, thus needing to be suppressed or explained away. Even though GSK has now made the studies available, the information has not been widely appreciated by the medical community.

The clinical relevance of the specific results with lamotrigine that are available through the GSK Web site is not insignificant. Two non-industry-funded studies found some benefit with lamotrigine in acute bipolar depression (1 published[28] and another recent one not yet published).[29] Yet the unpublished negative studies described here are needed to put such positive studies in context. Many clinicians and academics widely view lamotrigine as an effective treatment for, colloquially, “bipolar depression,”[30] even though the entire literature is less consistent on this point than many clinicians and academics appear to assume. This impression has been furthered by the selective publishing of one of the RCTs[25] with a positive secondary outcome on Montgomery Asberg Depression Rating Scale without emphasizing its negative outcome on the study's primary outcome on Hamilton Depression Rating Scale. Further, numerous review papers of lamotrigine in bipolar disorder cite the single published study as evidence of benefit in acute bipolar depression, but make no mention of the unpublished studies.[30–32] This problem takes nothing away from the evidence of lamotrigine's benefit in prophylaxis of bipolar disorder, particularly in prevention of depressive episodes, although it should be noted that the benefits seen in those prophylaxis studies are limited to patients who initially tolerated and appeared to respond to lamotrigine for acute mood episodes. But preventive benefit does not necessarily translate to acute efficacy, or vice versa.[33] Clinicians must learn to distinguish treatment of acute bipolar depression from its prevention. The lamotrigine data, positive and negative, could have been an important means of providing empirical evidence of the importance of this distinction.

The importance of academic access to such unpublished data can hardly be underestimated. For instance, the published study of lamotrigine in rapid cycling bipolar disorder is conservatively reported: the negative result in the primary outcome is acknowledged in the abstract, and the benefit in a secondary outcome in type II bipolar disorder may be clinically plausible and important. However, despite this acceptable presentation of this study, the nonpublication of the second RCT, where there was no indication at all of any benefit in type II rapid cycling bipolar disorder, leaves a mistaken impression in the published literature that lamotrigine is likely effective in that population. One out of 1 positive RCTs provides a different context than 1/2 positive RCTs. Instead, post-hoc exploratory analyses of the unpublished data are published, which, after extensive data analysis, find some possible benefit (somewhat more euthymia in lamotrigine-treated patients vs placebo utilizing a secondary outcome measure, the Life Chart Methodology).[34] While these exploratory analyses might be relevant, and even correct, the nonpublication of the straightforward findings of the original RCT leads to a distorted context whereby the psychiatric literature overestimates the benefit of the drug.

Further, it is well known that systematic reviews, especially meta-analyses, are vulnerable to major biases if negative or failed trials showing statistical nonseparation of a drug vs placebo or an active comparator are not available for analysis.[4,35] Further, narrative reviews of agents for bipolar disorder will provide the false impression that certain agents, such as lamotrigine, have been consistently effective whenever they were studied, since positive studies are evidently more likely to be reported at Web sites or in published articles. The practice of burying negative data in infrequently read review papers seems to be one approach to “publishing” such data, as exemplified by the single GSK-sponsored review paper which briefly summarized 5 negative studies with limited detail[26]; yet this approach is not likely to provide adequate information for clinicians and researchers. Even when the negative studies are made available in more detail, as with a recent meta-analysis of 5 negative acute bipolar depression studies with lamotrigine,[27] that kind of analysis should not replace earlier availability of each specific study. Further, since meta-analysis is an observational data analytic exercise, it too is not definitive, and is open to differing interpretations.[36] Too often, it can take the form of “statistical alchemy,”[37] whereby huge samples provide statistical significance to clinically meaningless differences. There is some danger of this kind of interpretation of the recent lamotrigine meta-analysis.[27]

Attention to publishing negative RCTs in a straightforward manner might be less necessary if clinicians were more critical readers of the research literature. However, many clinicians appear unable to exercise methodological critique in their evaluation of papers, thus not sufficiently realizing, for instance, that the 2 main published studies of lamotrigine in acute bipolar depression needed to be interpreted with much caution (one used a crossover design,[28] and the other emphasized a positive secondary outcome[25]). Thus, publishing negative data, though an improvement on the status quo, will not be a panacea; clinicians will still need more training in critically interpreting research.

The clinical relevance of the lamotrigine studies is notable: Taking the negative outcomes into account, as of now, one might say that this agent is reasonably effective in maintenance treatment of bipolar disorder, particularly in prevention of depression, among patients who initially tolerate and may benefit from acute lamotrigine treatment. It is proven ineffective in acute mania, rapid cycling, and acute bipolar depression. Exploratory analyses suggest that a small effect size of benefit might occur in some patients with bipolar depression,[27] perhaps among those more severely ill,[27] and the hypothesis of possible benefit in type II rapid cycling bipolar disorder was raised by one study but disconfirmed by another. This context of where the drug is effective, and where it is not, is vital for scientifically valid and ethically honest clinical practice and research.

Conclusions

The pharmaceutical industry is not routinely making data from negative studies available through the published scientific literature or in available Internet sources. In the case of bipolar disorder, the negative unpublished data made available on lamotrigine provide an important context for clinical practice and research.