Reporting Bias in Industry-Supported Medication Trials Presented at the American Psychiatric Association Meeting

To the Editors: Although most evidence suggests that inexpensive, off-patent antidepressant and antipsychotic medications are of comparable effectiveness to more expensive patented medications, the off-patent antidepressant and antipsychotics are prescribed at a lower rate than the patented medications (1-5). One possible explanation for this disconnect could be a bias in the subset of the overall evidence that is made available to the prescribing clinicians (4). In 2008, Turner and colleagues reported on publication bias in the results of pharmaceutical industry supported antidepressant trials. They found that while only 51% of the trials registered with the FDA were positive, among the subset of studies that were published, 94% were reported as positive (6). Here, we sought to assess whether a similar reporting bias may be present among research presented at the largest annual psychiatry meeting in the United States, the American Psychiatric Association (APA) Annual Meeting.

Two raters (SS and MP) identified all comparison medication studies presented as new research at the 2009 and 2010 APA Meetings. Each abstract was independently rated as positive, negative or mixed to the medication in question, using a modified version of a validated rating system (Table 1) (7). Raters were blind to industry sponsorship at the time of assessment. There were two disagreements between the raters that were resolved with discussion. An independent sample T-test was utilized to assess whether there was a significant rating score difference between the set of industry-supported and the set of non industry-supported studies.

Table 1.

Scale Used to Rate Abstract Conclusions

Positive (1)	The “experimental” medication was superior to the studied alternative for at least one outcome measure (efficacy, side effect profile or cost) and not inferior to the studied alternative on any outcome measure.
Mixed (2)	The “experimental” medication was either no different from the studied alternative or the “experimental” medication was superior to the studied alternative for one or more outcome measures but inferior for other outcome measures.
Negative (3)	The “experimental” medication under study was inferior to studied alternative for one or more outcome measure and not superior to the studied alternative on any outcome measure.

In total, 278 medication trial abstracts were identified (195 industry supported; 83 non-industry supported). 97.4% of the industry-supported studies reported results positive towards medication in question, 2.6% reported mixed results and none reported negative results. In contrast, 68.7% of the non industry-supported studies reported results positive with regards the medication studied, while 24.1% reported mixed results and 7.2% reported negative results. Overall, industry-supported abstracts were significantly more positive than non-industry supported abstracts (T=7.55;p<0.0001).

Our finding that 97% of industry-supported studies reported positive results is remarkably similar to the finding of Turner and colleagues; it suggests that there is bias in the results reported at the APA meeting. Unlike the antidepressant trial study, we do not know how many relevant comparable industry-supported trials were performed but not reported at the APA meeting. As a result, one possible explanation for our results is that the medications being reported on are highly effective and safe and that there are few unreported negative studies. However, this hypothesis is not supported by our observation that the non-industry supported studies reported a far lower rate of positive results. A more likely explanation for our findings is that the positive reporting bias among industry-supported studies previously identified in the literature is also present at the APA meeting. The likely selective reporting present does not imply an intent to deceive or mislead; investigators and industry may simply feel that positive results would be of most interest to meeting attendees.

It is important to note that the APA meeting is a primary source of continuing medical education for many of the 15,000 clinicians in attendance and a formative meeting for a substantial number of psychiatrists-in-training. For these attendees, the selective reporting of medication studies could create an impression that the newer, more expensive psychiatric medications are more effective and safer than justified by an unbiased assessment of the evidence.