‘They would say that, wouldn't they?’ A reader's guide to author and sponsor biases in clinical research

Commercial sponsorship of clinical research, especially for drugs, is ubiquitous. One of the solutions to some of the dilemmas arising from this is full disclosure of authors' financial interests and relationships with sponsors. But is disclosure enough, and what should we as readers make of the fine print at the end of journal articles? Is sponsorship bias the only bias we should watch out for?

The arguments for and against are familiar. In one corner, people accuse researchers of having sold out; no one should believe anything that Professor XYZ now says. From the other corner, people energetically defend academics' integrity and argue that we can pretty much trust them no matter where their money comes from. And anyway, the journal can be trusted not to let anything too dubious get through, can't it?

For readers, wherever our views might stand along the spectrum between these polarized positions, questions remain. Should we just gloss over the disclosure of interest fine print and somehow be reassured just because it is there, or should we be scrutinizing it? Is there a point at which this information should signal the warning lector emptor (let the reader beware), and if so, can we tell?

Clinical research does not just serve clinical interests. It also plays a major economic role. And research provides the building blocks of academic careers. That means that, inevitably, authors have interests of their own, although the weight of these no doubt varies greatly among individuals. Those interests do not always conflict with the interests of patients and clinicians, but sometimes they might. Journals have, to a greater or lesser degree, shifted the responsibility to the reader: only a minority of journals might actually decide against publication purely on conflict of interest grounds.^1,2 Most journals and all readers will not know if the amounts of declared financial interest involved are trivial or major. It is an honour system, and it is in the disclosure to readers that journals see their ethical obligations largely filled. There is no routine or random investigation, and generally no policy on sanctions—picture major international sport before random doping tests and the barring of offending players from future competition.

FINANCIAL BIAS

Fortunately research bias itself is the subject of research. Articles on this subject, too, come from polarized positions, but over time, some fairly clear messages have emerged. The question most studied here is whether or not drug companies have an influence on the research they sponsor on their products. One-third of trials are sponsored by drug companies.³ Somewhere between one-third to three-quarters of the trials in the five biggest general medical journals are industry-financed (Annals of Internal Medicine, BMJ, JAMA, Lancet and New England Journal of Medicine).⁴ But does it influence the results?

The most recent review of studies that compared trials sponsored by drug companies with those that were not concluded that sponsored trials were more likely to say that the sponsor's product was effective.⁵ Recent studies of systematic reviews came to the same conclusion.^6,7 For trials, this was not because of problems within the conduct of the research. If anything, the quality of these trials was better than that of non-sponsored trials. Simply not publishing studies that fail to show a benefit might be a factor here, so-called publication bias.⁸ In the end, though, it has been suggested that sponsors are by and large getting the results they want, not by suppressing unpalatable studies or ‘fiddling’ the results, ‘but rather by asking the ‘right’ questions’,⁹ for example, by comparing the drug to a smaller dose of a competitor's drug. Biased reporting of outcomes has also been shown to affect the outcomes of trials.¹⁰ In many fully legitimate ways, trials can be done in the most ideal circumstances, giving a treatment a chance to perform at its personal best.

All of which suggests the best strategy for readers is to maintain some healthy scepticism, especially with early positive results. A promising treatment is often in fact merely the larval stage of a disappointing one. At least a third of influential trials suggesting benefit may either ultimately be contradicted or turn out to have exaggerated effectiveness.¹¹

Critical readers need to be conscious of two other aspects that are characteristic of industry-conducted or sponsored trials, and which could be influencing them. The first is the extent to which the overall clinical research agenda is dominated by research into drugs because of the availability of this sponsorship. And the second is the full impact of marketing, which begins with the article itself. The proportion of trials where industry employees are named co-authors appears to be increasing.³

Less obvious may be the influence of professional medical writers paid for by industry funds. This might at times be exerting an influence on framing and even content. It might often lead to an improvement in the quality of writing. A professionally written article might quite simply be a more compelling one, and that effect may be boosted by post-publication marketing. Articles that are discussed in the general media may end up being more influential in the scientific literature, too.¹² At the time of writing, out of the five big general medical journals, only the BMJ and Lancet explicitly asked authors questions about the involvement of professional writers who may or may not be listed as co-authors. JAMA recently updated its policy,¹³ but did not add this question.

FINANCIAL ENTANGLEMENTS

Research is not as helpful in relation to the other parts of the financial declaration made by our fictional Professor XYZ at the head of this article. Nor is there a consensus of opinion of what these financial connections mean. There is a truism that says most people avoid biting the hand that feeds them—and in this case, it is a hand that pours champagne, too. That suggests that it is self-censorship and a relationship of dependence and privileges that might be important. Social interaction and the collegiality that emerges from working together might blunt the willingness to be too critical. But it is also argued that as long as people are taking money from multiple commercial sources, then they have hedged their bets and are consequently more independent. In the end, on this matter, as readers we are left to our own judgment. There is no evidence to suggest this strategy protects researchers (or anyone else) from bias.

While the requirements for adequate author independence in relation to data are fairly similar between major journals, the same cannot be said about other aspects of authors' financial interests. Table 1 shows the types of interests the five major medical journals specifically ask of their authors (although all ask general questions). There are few issues in common. That might help explain why these declarations vary so much between journals, with some fairly short and others often quite extensive. Journals vary in other matters too: for most, the information about conflict of interest is buried in fine print at the end of the article, but some have it prominently at the front.

Table 1.

Specific questions asked by the major five medical journals on authors' financial interests in July 2006

	Ann Int Med	BMJ	JAMA	Lancet	NEJM
Fee from other than the journal for writing or being named on the paper				•
Paid professional writer		•		•
Employment	•	•	•	•
Consultancy	•	•	•	•	•
Holding stocks, shares or equity	•	•	•	•	•
Patents/patent applications			•	•	•
Company board position				•
Expert testimony	•	•	•	•
Funds for grants and/or research and/or staff and/or equipment	•	•	•	•	•
Travel grants		•		•
Speaking fee		•			•
Honoraria	•			•
Royalties	•		•

Ann Int Med: past five years and pending (including grants); BMJ: last five years; JAMA: last five years and foreseeable future; Lancet: within three years of submitted work, presently or applying for; NEJM: time not specified

University standards do not always provide much of a safeguard either. A study of academic medical centres in the US found some major differences in what was deemed acceptable in contracts between industry and clinical trialists.¹⁴ For example, 50% of the universities allowed sponsors to draft the manuscript, and almost 15% found it acceptable to allow sponsors the authority to revise manuscripts or decide whether they could be published.

Interesting for readers, too, would be the medical journals' policies in relation to their own editorial staff, but their internal practices are not as readily available as their requirements of authors. The Lancet has published its quite strict standards. Editors at the Lancet may not accept offers for travel, accommodation, hospitality or gifts.¹⁵ Such policies exist at other journals, too, but the policies are not generally publicly available. Conflict of interest statements for editorial staff of the BMJ are published online.¹⁶

What about research participants? A study of potential clinical research participants in the US found that people with chronic illness believed they should be told about financial entanglements of researchers before they are asked to participate in a trial.¹⁷ Most would still participate, but depending on the specific scenario, from 2% to 32% would not participate if they knew of financial benefits to the researchers. The scenarios that attracted the highest frequency of concern were those involving personal income for the researchers.

What do researchers and readers think? If the debates on this are anything to go by, many researchers believe their entanglements with industry have no influence, and others believe those researchers are deluding themselves.¹⁸ Recent editorials in The Wall Street Journal¹⁹ and New York Times²⁰ suggest the medical journals have a real credibility problem in the wider public sphere on this issue.

Readers are likely to also have divergent views. We definitely have unequal knowledge. We cannot really expect to understand the extent of an academic's dependence on industry from the way the data are generally presented to us. Insiders can often tell you that an academic is now wholly dependent on large sums of industry money, but the average reader cannot know if financial entanglements are in the trivial or mind-boggling range. It would be helpful for many of us as readers if there was a way of classifying this. Establishing a random audit system is worth considering, along with negative consequences for the future if a researcher transgresses. But perhaps most urgently of all, readers might be helped by a better lector emptor: serious conflicts of interest should be up at the level of the abstract—ideally within the abstract—so that few readers will miss it.

SELF-PROMOTION BIAS

Inevitably, researchers and many other knowledge workers potentially have an interest in promoting themselves, or at least in peddling their views, theories and hobby horses. Research grants and publications are both major sources of life-blood and currency for academic careers. Publications, particularly in the more important journals like the one you are reading now, carry influence because of their impact on you, the reader. But they are also necessary to amass credit for the progress of academic careers. The article being cited in other articles in turn accrues the academic version of frequent flyer points in the Science Citation Index.

Authors will often be building on their own body of work, which means they will inevitably be at least a bit self-referential. But one study has found some evidence that a high proportion of self-citation within an article might indicate a lower quality of article.²⁰ Another has found no association between self-citation in the diabetes literature with methodological rigour.²¹ Still, ‘Even if I say so myself’ is not a very high level of evidence. Of course a certain amount of self-citation is legitimate and essential. On the other hand, self-citation also ‘perpetuates one's interpretations or opinions’ and it might ‘falsely validate the conclusions of an author or group’.²¹ Self-citation has been calculated as an average of 20% of all citations in one study,²¹ and 36% in another.²⁰

‘EUREKA!’ BIAS

Archimedes' famous cry of discovery and intellectual excitement brings us to a key bias that is possibly implicated in all author and sponsor biases—and it is one that readers can share, too. We can all believe too quickly that which we most earnestly hope to be true. One of the key reasons blinding is such a critical part of trial methodology is because we know that bias is human, and blinding helps protect us from investigator bias. While this kind of bias does not always mean that the benefits of a treatment end up being exaggerated, sometimes it does.^22,23 While some people are nihilistic, most of us want to find something that helps. The research suggests, though, that when we see a claim of benefit, the strength of the claim is weakened when the research has been underwritten by the product's manufacturer.

Key questions

From journal to journal, the answers to questions about financial entanglement may be sprinkled through different parts of a research article, including: authorship affiliations and contributorship, the methods section, declaration of interests section, acknowledgements and funders' section. Check all the fine print if you want to assess potential for author and sponsor bias. You need to see the full article—the abstract will not include such information.

• How much independence from the funders did the researchers have, and did they control the data, its interpretation and the publication?

• Is there a systematic review on the topic the research is addressing, and if so, how do the results of this study fit into the other evidence? If there is no systematic review, are there trials?

• What do accompanying editorials, letters and analyses in secondary evaluation publications (such as ACP Journal Club and Evidence Based Medicine) say?

• Is the range of opinion on the issue shown? Do they tell you what the other schools of thought on this issue are?

• Who says so? Is there a high proportion of self-citation? Are there many unsupported claims?

• If you disagree with the author, what evidence do you have to support your position?

It will not help, perhaps, if we lose all optimism and hope for progress, but healthy scepticism is essential too. In the end, it is not surprising when a manufacturer publishes a study that says its product is better than another, or someone says their own pet theory is the best idea since sliced bread: well, they would say that, wouldn't they?