Taking Financial Relationships into Account When Assessing Research

Abstract

Many scientific journals, government agencies, and universities require disclosure of sources of funding and financial interests related to research, such as stock ownership, consulting arrangements with companies, and patents. Although disclosure has become one of the central approaches for responding to financial conflicts of interest (COIs) in research, critics contend that information about financial COIs does not serve as a reliable indicator of research credibility, and therefore, studies should be evaluated solely based on their scientific merits. We argue that, while it is indeed important to evaluate studies on their scientific merits, it is often difficult to detect significant influences of financial relationships that affect research credibility. Moreover, at least five factors can be examined to determine whether financial relationships are likely to enhance, undermine, or have no impact on the credibility of research. These include as follows: whether sponsors, institutions, or researchers have a significant financial stake in the outcome of a study; whether the financial interests of the sponsors, institutions, or researchers coincide with the goal of conducting research that is objective and reliable; whether the sponsor, institution, or researchers have a history of biasing research in order to promote their financial goals; how easy it is to manipulate the research in order to achieve financial goals; and whether oversight mechanisms are in place which are designed to minimize bias. Since these factors vary from case to case, evaluating the impact of financial relationships depends on the circumstances. In some situations, onemay decide that the financial relationships significantly undermine the study’s credibility; in others, one may decide that they have no impact on credibility or even enhance it.

INTRODUCTION

Many scientific journals, government agencies, and universities require disclosure of sources of funding and financial interests related to research, such as stock ownership, consulting arrangements with companies, and patents (Cho et al., 2000; McCary et al., 2000; Krimsky and Rothenberg, 2001.; Cooper et al., 2006; Blum et al., 2009; Department of Health and Human Services, 2011; National Science Foundation, 2011). There are at least two rationales for disclosure policies: to minimize bias and to promote public trust.¹ Disclosure may help to minimize biases related to research because scientists who become aware of financial relationships pertaining to a study may take these into account when assessing the research (Krimsky, 2003; Resnik, 2007). Additionally, requiring investigators to disclose financial interests may encourage them to refrain from engaging in relationships that would be considered financial conflicts of interests (COIs) or at least to try to prevent financial COIs from influencing their judgment. Disclosure of COIs in published research may help to promote trust by assuring the public that there are no hidden financial interests affecting the research, since the interests will be out in the open, for all to see (De Angelis, 2000). Public trust can be negatively impacted if financial interests are not disclosed when an article is published but come to the surface later. When this occurs, people may feel that the researchers are not trustworthy because they were trying to hide important information.

Despite the widespread acceptance of financial disclosure policies, many have questioned their usefulness. Criticism comes from opposite ends of the spectrum. Some oppose financial disclosure policies on the grounds that they are ad hominem attacks on researchers that distract the readers’ attention from an objective evaluation of the science. Readers should focus on data, methods, and results, not on the authors’ financial relationships (Rothman, 1993; Rothman and Evans, 2005; Borgert, 2007; Society of Toxicology, 2008). Disclosing financial COIs does little good, because experts on the topic under investigation can evaluate the quality of the science on its merits, without taking COIs into account, while novices cannot respond effectively to information about financial COIs, because they cannot evaluate the quality of the science itself. They may only reach crude and unjustified conclusions, such as that all financially conflicted research is biased.

Even those who generally support disclosure policies have raised some other concerns. Some argue that disclosure policies are little more than a Band-Aid that covers up larger problems because they do nothing to discourage problematic relationships that create bias (Krimsky, 2003; Elliott, 2008; Cosgrove and Krimsky, 2012). Some argue that disclosure policies may, paradoxically, encourage bias because authors who disclose their COIs may decide that this gives them a moral license to manipulate data or results (Loewenstein et al., 2012). Others argue that journal disclosure policies are easy to evade because they lack an enforcement mechanism, and that authors often have financial relationships that are not fully disclosed (Resnik, 2009). Finally, some argue that financial disclosure policies are unfair because authors often have interests other than financial ones, such as political or institutional interests, that can bias their work, and it is unfair to focus only on financial interests (Resnik, 2007).

We will not question the general wisdom of financial disclosure policies in this article (for discussion, see Resnik, 2007; Elliott, 2008). Instead, we will examine the contentious notion that one of the reasons for disclosing financial interests is so that the reader may take this information into account when evaluating the research (Krimsky, 2003; Resnik, 2007). We will argue that it is important to take information about financial COIs into account when assessing research and that there are at least five factors that one should consider when making judgments on how financial relationships affect the credibility of research. We will conclude that how one should take financial relationships into account varies from case to case, depending on these five factors. In some situations, there is little need to be concerned about financial relationships because they are not likely to influence the research results, while in other cases considerable concern is warranted.

HOW FINANCIAL RELATIONSHIPS CAN IMPACT RESEARCH

Disclosure policies typically require researchers to provide information about two types of financial relationships: sources of funding, such as contracts or grants from private companies or government agencies, and financial COIs, such as stock ownership or consulting agreements. Institutional affiliations, which constitute a type of financial relationship, are also disclosed during the submission process.

A COI (for an individual) is a situation in which the individual has financial, personal, professional, or political interests that are likely to compromise his or her judgment or decision-making related to the performance of his or her ethical, legal, or professional obligations (Resnik, 2007).² For example, suppose that an investigator is collaborating on a phase IV (post-marketing) study, sponsored by a pharmaceutical company, which compares the company’s hypertension drug to competing medications. Suppose the investigator also owns a significant amount of stock in the company. In this situation, the investigator might be tempted to make inappropriate judgments or decisions that tend to favor the company’s drug in order to increase the value of his stock.

Organizations or institutions can also have COIs. For example, if a university owns a significant amount of stock in a company that is sponsoring research on campus concerning one of the company’s products, the university would have a COI in this situation, because some of the decisions made by the university would likely be impacted by this financial interest (Resnik and Shamoo, 2002). Although most of the discussion of COIs focuses on financial interests, it is important to note that personal, professional, and political interests can also impact judgment and behavior. For example, if an individual reviews a grant submitted by one of his or her former students, the reviewer would have a COI in this situation because his or her judgments and decisions concerning the grant would likely be affected by his or her relationship to the applicant. Indeed, most granting agencies prohibit reviewers from evaluating applications submitted by colleagues, mentors, students, and former students until a sufficient amount of time has elapsed (Shamoo and Resnik, 2009). We will not examine these nonfinancial COIs in this paper, but our approach to addressing financial relationships may provide a model for handling these other sorts of conflicts.

A growing body of evidence supports the hypothesis that financial relationships are statistically associated with research outcomes. For example, one study found that 96% of authors publishing studies reporting outcomes that favored the use of calcium channel blockers had financial relationships with the companies sponsoring the research, whereas only 37% of the authors who published studies questioning the use of calcium channel blockers had such relationships (Stelfox et al., 1998). Another study found that only 5% of articles with industry funding reported negative results for cancer treatments, as opposed to 38% of articles not sponsored by industry (Friedberg et al., 1999). A study of cardiovascular clinical trials found that industry-sponsored studies were more likely to report positive findings than studies not funded by industry (Ridker and Torres, 2006). A systematic review of 30 studies examining the relationship between pharmaceutical industry sponsorship and research outcomes found that studies with industry funding were more likely to report results that favored the company’s products than studies with independent sources of funding (Lexchin et al., 2003). Another review of 11 previous studies found that industry-funded biomedical research was almost four times more likely to yield results favorable to industry than independently funded research (Bekelman et al., 2003). A more recent systematic review of 19 articles on examining the relationship between industry funding and research outcomes in clinical research found that 17 articles reported a strong positive correlation whereas only two did not (Sismondo, 2008).³

There are some possible explanations of the relationship between financial relationships and research outcomes that do not impute dishonesty or malfeasance to private sponsors or investigators (Dickersin, 1990; Easterbrook et al., 1991). First, journals tend to be more interested in publishing studies with lower p-values (i.e., p < .05) than higher ones. Private companies have the financial resources to conduct clinical trials with large sample sizes (> 500) with increased statistical power. In larger studies one is more likely to detect a difference between treatments (i.e., a positive result) than in smaller ones. Since government-funded researchers generally have less money to spend, they often cannot afford to conduct trials with larger sample sizes, and they are therefore less likely to obtain positive results (Hochster, 2008). Thus, the amount of money available for research may tend to distort the publication record in favor of private industry. Second, if private funders have preliminary data indicating that one of their products is particularly effective, they are more likely to fund additional studies on that product. Companies could also decide to expand the sample size of a clinical trial based on preliminary data of potential effectiveness of a product. A company could also decide not to fund a study based on preliminary evidence of lack of effectiveness (Krimsky, 2012). Such decisions could create correlations between financial relationships and research outcomes that have little to do with dishonesty or malfeasance.

While these explanations of the relationship between financial interests and study outcomes are cogent and merit additional investigation, they do not tell the whole story. Financial relationships can influence research outcomes in a variety of ways (Resnik, 2007; Michaels, 2008; Elliott, 2011; Lexchin, 2012; Krimsky, 2012) (see Table 1). First, financial relationships affect the selection of research problems or questions, because companies tend to fund research that has the potential to promote their financial interests. For example, if a company is trying to obtain regulatory approval for a new drug, it is required to sponsor studies designed to obtain data on the drug’s safety and efficacy. Once a drug is approved, a company may conduct post-marketing studies that can provide additional data concerning safety and efficacy and can help publicize the drug. It is important to note that even though the influences of funding on the choice of research questions does not by itself lead to biased results, the overall published research record can be impacted if a sponsor only publishes studies that are favorable to its products or only studies questions that are likely to reveal favorable information (Elliott and McKaughan, 2009; Elliott, 2013). Thus, if a company sponsors ten studies comparing its drug to alternative medications, and it publishes six studies with favorable results but does not publish four studies with unfavorable results, then the overall published research record will reflect this bias (Goozner, 2004a; Michaels, 2008; Lexchin, 2012).

Table 1.

How Financial Conflicts of Interest can Influence Research

Type of Influence	Example
Problem selection	Funding research projects that are likely to reflect positively on a company’s products
Study design	Under-powering a study to show no statistically significant evidence of an adverse effect
Data collection	Fabricating or falsifying data
Data analysis	Choosing a method of data analysis most likely to support one’s hypothesis
Data interpretation	Overstating the significance of data
Publication	Not publishing data or studies that undermine one’s hypothesis

Second, financial relationships may impact study design if an individual or an organization with a financial stake in the outcome makes methodological choices that affect how the data are analyzed (Lexchin, 2012). For example, a pesticide company might conduct a study with a small sample size that shows no evidence of adverse effects of exposing human subjects to low doses of one of its products (Michaels, 2008; Shrader-Frechette, 2007). Because the study is underpowered, it is not likely to demonstrate a statistically significant adverse effect. If the study had a large enough sample size, it might demonstrate such an effect (Resnik, 2007). This is assuming, of course, that reviewers will not recognize that the sample size is too small and that the study is therefore susceptible to a Type II error. (Problems with peer review will be discussed below.) Another way that financial relationships can impact research design concerns the selection of variables that the study will measure. A researcher or organization with a stake in the outcome might fail to collect data relevant to human health in order to avoid producing data that shows its productmay have adverse impacts. For example, suppose that animal studies have shown that a pesticide can affect endocrine system functioning and that a company conducts low-dose human studies but does not collect data related to the endocrine system. The published report will not include such data even though it might be relevant to assessing the safety of the chemical (Elliott and McKaughan, 2009). Again, this is assuming that peer reviewers will not recognize that the study has failed to gather data related to an important health outcome.

Third, financial interests may impact data collection if a researcher falsifies or fabricates data in order to support a particular outcome (Resnik, 2007). Although data fabrication and falsification are thought to be rare in scientific research (Fanelli, 2009), when these transgressions do occur, there are often financial interests at stake, such as the need to produce results to obtain approval of a drug, to demonstrate that an invention works as designed, or to continue receiving funding (Shamoo and Resnik, 2009). Since the need to produce results is a pressure that virtually all researchers face, this financial factor can impact not only industry-funded research but also government-funded research (Resnik, 2007). Although information about the financial interests related to a study need not constitute a reason to suspect that data have been fabricated or falsified, it can be a relevant consideration to take into account when a researcher associated with a publication is accused of misconduct (Shamoo and Resnik, 2009).

Fourth, financial interests can affect data analysis, as there is often more than one way to analyze the data. A researcher or organization with a stake in the outcome of a study may choose a method of analyzing the data that is more likely to yield favorable results. For example, in clinical trials, data can be analyzed using an intent-to-treat approach or an on-study approach. In the intent-to-treat approach, all data from subjects who enroll in the trial are analyzed, including data from subjects who are withdrawn from the study or drop out. The rationale for using this approach is to ensure that all adverse effects of a treatment are taken into account in evaluating the treatment. Using the on-study approach, only data from those subjects who complete the study are reported; data from those who are withdrawn or drop out are not analyzed. The rationale for using the on-study approach is that it provides a better understanding of the efficacy of the treatment, since to obtain maximum efficacy it is usually necessary to receive the treatment for the appropriate interval, and subjects who are withdrawn or drop out may not achieve maximum benefit. A study sponsor who is interested in demonstrating efficacy of a treatment and wants to minimize the reporting of adverse outcomes may choose the on-study approach instead of the intent-to-treat approach, and this decision may have a significant impact on the evaluation of the treatment because it may underreport adverse effects (Gupta, 2011).

Fifth, financial interests can impact the interpretation of data. An investigator or organization with a financial stake in the outcome of the research may overstate the significance of data to ensure that the research has the desired impact on science, medicine, and society (Goozner, 2004a; Resnik, 2007; Michaels, 2008). For example, suppose that a company sponsors a clinical trial comparing its cholesterol-lowering medication to competing drugs which shows that patients taking its drug had total blood cholesterol levels that were 7% lower than patients taking competing drugs. However, patients taking the company’s drug did not have a lower mortality ormorbidity as compared to patients taking competing drugs, and they had some other adverse effects, such as muscle weakness and cognitive impairment. Nevertheless, the company concludes that its drug is superior to alternative treatments because it is better at lowering total cholesterol. However, a balanced assessment of the evidence may not support this interpretation, since the drug did not reduce mortality and morbidity and had adverse effects that patients might want to avoid.

Sixth, financial interests can impact publication because individuals or organizations with a stake in the outcome of research may fail to publish unfavorable data or results. One of the most blatant examples of this occurred when Merck suppressed publication of data showing that its drug Vioxx (rofecoxib) was associated with significantly higher cardiovascular risks as compared to competing drugs. Shortly after receiving Food and Drug Administration (FDA) approval to market Vioxx, Merck sponsored a clinical trial, known as Vioxx Gastrointestinal Outcomes Research (VIGOR), which compared its drug to another non-steroidal anti-inflammatory drug (NSAID), naproxen. The published version of the VIGOR study indicated that Vioxx had lower gastrointestinal risks as compared to naproxen and that it did not significantly increase the risk of heart attack or stroke (Bombardier et al., 2000). Although the cardiovascular risks of Vioxx were higher than Naproxen, the investigators explained this difference as due to the protective effects of naproxen (Bombardier et al., 2000). However, Merck did not publish all of the cardiovascular data collected for the study. If the data had been published, it would have shown that Vioxx significantly increases cardiovascular risks as compared to naproxen. However, the company did report the data to the FDA. In 2002, the FDA issued a black box warning stating that Vioxx increases the risks of a heart attack or stroke. Merck withdrew Vioxx from the market in 2004, after another study showed that it had twice the cardiovascular risks compared to other NSAIDs. Thousands of people sued Merck claiming that they had had a heart or stroke as a result of taking Vioxx (James et al., 2007; Resnik, 2007).

As mentioned above, a critic of disclosure policies could acknowledge that financial relationships can bias scientific research, yet argue that examining financial relationships does nothing to combat this problem. From this perspective, the best way to minimize bias is to ensure that publications report everything one needs to evaluate the scientific merits of a study, including methods, data, assumptions, and results, and that all supporting data relevant to a study are available to the reviewers and readers (McCarty et al., 2012). If reviewers and readers have this information, they can assess the study’s scientific validity and significance. Thus, the critic could insist that, at best, disclosure of financial interests adds nothing of consequence to peer review and scientific debate, and at worst it leads to the unjustifiable dismissal of high-quality research (Borgert, 2007; Conrad and Becker, 2011). If key details of a study are included in the publication, then reviewers and readers should be able to determine whether aims and objectives are well-described, whether relevant variables are measured, whether statistical methods are appropriate, and whether the data support the conclusions. Reviewers and readers can also arrive at their own interpretations of the data, and do not need to be swayed by the investigator’s views. Others may also try to replicate the results. Eventually, science’s process of self-correction will weed out the hypotheses and theories that do not stand up to rigorous testing and criticism.

While we recognize the importance of rigorously evaluating the methods, assumptions, data, and results reported in scientific publications, it is important to recognize a number of significant limitations associated with this process. Most importantly, the influence of financial relationships will usually not be evident in a published paper, because many important decisions that affect the research will have been made prior to publication and are not stated in print or available to readers and reviewers (Ziman, 2000). Researchers may not state in a publication that they did not report or analyze all the data for the study, that they decided not to collect some types of data, that they considered several different strategies for analyzing the data and chose the one that provides the best support for their hypothesis, that the sponsor required them to change the design of their study, or that they manipulated data or digital images. Because money operates behind the scenes, it is important for reviewers and readers to have information about financial relationships involved in a study, so they can make fully informed decisions about it.

A second problem with focusing only on the scientific merits of a study and depending on the self-correcting nature of the scientific process is that peer reviewers for journal articles are not perfect and often miss errors and other problems, as noted earlier (Elliott, 2011; Smith, 2005, 2006). Even if higher-impact journals use reviewers who are more likely to spot problems with a paper, lower impact journals may use reviewers who are less likely to spot such problems, or they may have lower publication standards. A study of poor methodological quality that is not published in a high-impact journal may still be published in a lower-impact journal. Moreover, even high-impact journals have published studies that have been manipulated in various ways because of financial considerations, and their editors have expressed a lack of confidence in the peer-review process (De Angelis and Fontanarosa, 2008; Healy and Cattell, 2003; Smith, 2005). Therefore, the published record is likely to include numerous papers with methodological weaknesses and other problems which may be due to financial relationships. Even when these problems can be discerned in the published versions of the papers, it often requires a great deal of specialized expertise and careful scrutiny to find them. Therefore, understaffed regulatory agencies and concerned citizens are unlikely to catch these problems.

Third, science is not always self-correcting. Replication is rare in science because journals and sponsors are mostly interested in original research. Important experimentsmay be replicated, but others may not be (Ziman, 2000). Although the truth may win out in the long-run, in the short-run biased or erroneous research may have adverse effects on public health and safety or public policy. For example, although scientists eventually were able to prove that Vioxx significantly increased cardiovascular risks, Merck sold over several billion dollars’ worth of the drug and patients suffered adverse effects before the drug was taken off the market.

Fourth, financial relationships can influence judgment and decision-making at a subconscious level, so that researchers may not even be aware of how they have been affected by money. Psychological research has shown that people can be influenced by money even when they think they are being unbiased (Cialdini, 1993; Katz et al., 2003). Thus, researchers may be affected by financial relationships even when they do not deliberately manipulate methods, data, or results in order to promote their financial interests. The subtlety of these psychological effects exacerbates all the other problems associated with evaluating studies in terms of their scientific merits.

The foregoing considerations support the widely held view that it can be useful for reviewers and readers of a study to have information concerning sources of funding and conflicts of interest related to a study. That way they can take these relationships into account and not focus solely on the self-correcting nature of science and their ability to evaluate studies on their scientific merits. But the worry remains that financial relationships provide very unreliable guidance for determining whether studies are credible. What does it mean to take financial relationships into account in a responsible fashion? We will now address that question.

TAKING FINANCIAL RELATIONSHIPS INTO ACCOUNT WHEN EVALUATING RESEARCH

As mentioned earlier, there is a broad consensus that one of the main reasons for disclosing sources of funding and financial COIs is so that reviewers and readers can take these into account when evaluating research. Evaluating research involves making an overall assessment of the scientific merits of the research: Are the results of the study reliable and objective? Can they be trusted? Normally, reviewers and readers make this assessment by carefully examining the methods, data, and results reported in the publication, but we have seen that there can be significant limitations to this process. Therefore, we contend that if the authors have disclosed financial relationships that have some bearing on the research, these may also impact the overall assessment.

Indeed, empirical studies have shown that scientists take financial relationships into account when assessing research. One study sent copies of the same fictitious paper to 290 British Medical Journal (BMJ) readers, randomized into two groups. Group 1 received a paper stating that the authors were employed by a private company sponsoring the research and had stock in the company. Group 2 were sent the same paper stating instead that the authors were employed by an ambulatory care center and had no competing interests. Readers were asked to rate the paper for importance, relevance, validity, and believability, using a 5-point scale. Eighty-six readers from Group 1 participated in the study and 84 from Group 2 (58.6% response rate). Readers that received the paper with no private funding and no competing interests declared scored the paper significantly higher on all four metrics than readers who received the paper with private funding and competing interests declared (Chaudhry et al., 2002). The researchers conducted another study with a larger group of BMJ readers, using the same ratings. Readers received copies of a fictitious paper with a declaration of no competing interests (Group 1), funding by a private company and ownership of stock in the company (Group 2), and a student scholarship (Group 3). Five-hundred twenty-two out of 882 recipients (response rate 59.2%) returned the survey. Readers that received a paper with financial relationships to a private company rated the paper lower than groups that received a paper with no competing interests declared or support from a scholarship declared (Schroter et al., 2004). Another study by a different group of researchers sent a survey to 515 physicians that presented them with variations of a fictitious scenario involving a clinical trial of a drug. Two-hundred fifty-three returned the survey (response rate 49.1%). Sixty-nine percent of respondents said they would lower their opinion of the study when the scenario stated that the researchers had COIs. However, the presence of financial relationships in the scenario did not affect their likelihood of prescribing a drug unless situations of conflict and no-conflict were directly compared, in which case the participants were more likely to prescribe based on a study without a conflict (Silverman et al., 2010). Another recent study found that physicians were half as willing to prescribe drugs studied in industry-funded trials as compared with National Institutes of Health (NIH)–funded trials, and disclosures of industry funding also caused the physicians to downgrade their assessment of study rigor and their confidence in the results (Kesselheim et al., 2012).

Given that empirical evidence shows that scientists take financial relationships into account when evaluating research, the normative question arises as to whether this is justified and how they should do this. In thinking about the impact of financial relationships, one should consider how they should affect a study’s credibility. By “credibility” we mean the justifiability of accepting the results of the study for purposes of future research (such as relying on the results for one’s own research) or for practical or policy implications (such as using information from the study to regulate products or prescribe medications) (see also Conrad and Becker, 2011). Financial relationships could enhance credibility to varying degrees, undermine credibility to varying degrees, or have no impact on credibility.

What factors should one take into account when determining whether financial relationships impact a study’s credibility? Based on our assessment of studies on the psychological, economic, and social aspects of scientific research, we propose that at least five factors are particularly relevant when assessing the impact of financial relationships on research (Greenberg, 2001; Resnik, 2007; Elliott, 2011; Krimsky, 2012; Lexchin, 2012) (see Table 2). The first factor to consider is whether sponsors, institutions, or researchers have a significant financial stake in the outcome of a study, which is often the case when companies sponsor clinical trials of their products or studies likely to affect the regulation of their products (Bekelman et al., 2003). Researchers may have a significant financial stake in the outcome of the study if they are employed by the company or serve as paid consultants. Researchers and institutions can have a significant stake in the outcome of a study if they own patents related to the research or they have stock in a company sponsoring the research. One reason why numerous authors have recently called for more studies of pharmaceuticals and industrial chemicals through government funding agencies such as the NIH and the National Science Foundation (NSF) is that these agencies are not seen as having a significant financial stake in obtaining specific outcomes (American Public Health Association, 2003; Volz and Elliott, 2012; Shrader-Frechette, 2007). Thus, while researchers funded by these agencies still face numerous pressures that can affect their judgment, financial stakes for the study sponsors are not perceived to be one of those influences.

Table 2.

Factors to Consider when Assessing the Impact of Financial Relationships on Research

Do sponsors, institutions, or researchers have a significant financial stake in the outcome of a study? Do the financial interests of the sponsor(s), institution(s), or researcher(s) coincide with the goal of conducting research that is objective and reliable? Do the sponsor(s), institution(s), or researcher(s) have a history of biasing research in order to promote their financial goals? How easy is it to manipulate the research in order to achieve financial goals? Are there oversight mechanisms in place that minimize bias and make it more difficult to manipulate research successfully?

A second factor to consider is whether the financial interests of the sponsors, institutions, or researchers coincide with the goal of conducting research that is objective and reliable.⁴ Very often, financial interests coincide with conducting objective and reliable research. For example, if a company is developing a new product which will not bring any profit to the company unless it works as designed, then the company has a strong incentive to produce reliable data concerning the product’s efficacy. In such a case, profit maximization and good research go hand-in-hand. It is likely to be the case that financial interests and good research will coincide for many products, including (under some circumstances) pharmaceuticals. However, if a company can realize a profit or increase the value of its stock without conducting objective and reliable research, then its financial interests and good research practices may diverge. For example, Merck was able to sell billions of dollars’ worth of Vioxx and increase the value of its stock, despite significant flaws in its published research. Although problems with this product eventually came to the surface, Merck was able to realize significant profits from the product before the truth came to light. One might argue that situations like the Vioxx debacle are rare in science, and that companies will usually have a strong incentive to conduct objective and reliable research in order to avoid legal liability for harms caused by their products. However, legal scholars have argued that the current tort system does not consistently provide adequate incentives for companies to avoid putting harmful products on the market (Cranor, 2008; McGarity and Wagner, 2008).

Divergence can also occur when companies are conducting research in order to influence the development of regulations or public policy. For example, if a pesticide company is conducting animal studies on the safety of one its products in order to submit data to the Environmental Protection Agency (EPA), then the company will have an interest in obtaining results that favorably impact the EPA’s regulatory decision-making. Company leaders may sometimes sacrifice scientific quality in order to produce data that could lead to favorable regulatory decisions.

A third factor to consider is whether the sponsor, institution, or researchers have a history of biasing research in order to promote their financial goals. If a sponsor, institution, or researcher has a history of producing biased research in the past, then we would have reasons to believe they are likely to do so again under similar circumstances. For example, secret industry documents disclosed as a result of litigation show that tobacco companies suppressed and manipulated data concerning the harmful effects of smoking on smokers and on those exposed to second-hand smoke. Tobacco companies also have publicly denied that smoking is addictive while conducting secret research on how to enhance nicotine’s addictive properties (Glantz et al., 1998; Michaels, 2008). One could argue that there are good reasons to doubt the reliability and objectivity of research funded by tobacco companies, given this checkered history (Bero, 2005). Similarly, one of the reasons that many citizens, scientists, and regulators have become suspicious of industry-funded research on pharmaceuticals and industrial chemicals is because of evidence of questionable research practices, although it is difficult to discern exactly how prevalent these practices are (Angell, 2005; McGarity and Wagner, 2008; Michaels, 2008; Smith, 2005).

A fourth factor to consider is how easy it is to manipulate the research in order to achieve financial goals. Complex studies involving diverse populations and many different health-related outcomes, such as some types of clinical trials, may be easier to manipulate than studies which are more straightforward, such as pharmacokinetic studies of exposure to a single dose of a chemical and measurement of its excretion in the urine. Complex studies can be easier to manipulate than straightforward ones because they involve more decisions concerning the population studied, choice of variables, measurement tools, methods, and data analysis plans. When there are more decisions to be made concerning the design and execution of a study, there are more opportunities to bias the results, deliberately or subconsciously.

Related to this fourth factor, a fifth factor to consider is whether oversight mechanisms are in place that minimize bias and make it more difficult to manipulate research successfully. Most biomedical research journals now require prior registration of clinical trials in a public database as a condition of publication (De Angelis et al., 2005). To register a clinical trial, the sponsor must provide basic information concerning the study, such as the aims and objectives, endpoints, methods, research sites, and investigators. Registration can help reduce bias related to selective publication of clinical trial outcomes because key information about the study is available to the research community. Biomedical researchers can learn about the studies that are being conducted even if they cannot obtain access to data and results.

The Journal of the American Medical Association (JAMA) requires that data analysis for industry-funded studies be conducted by an independent statistician at an academic institution, rather than an employee of the company or contract research organization. The journal also requires that at least one author who is independent of a commercial sponsor have full access to all of the data and takes responsibility for the accuracy of the data and data analysis (Journal of the American Medical Association, 2012). One could argue that industry-funded research conducted according to these policies tends to be more credible than industry-funded research in the absence of such policies, because it is more difficult for those with a significant financial stake in the outcome to manipulate the data interpretation.

Some writers have argued for oversight mechanisms stronger than JAMA’s policy and clinical trial registration. Krimsky (2003) and Angell (2005) have argued that companies seeking approval of biomedical products should contribute funds to an independent, scientific organization (such as a government agency) that would design and conduct clinical trials (Krimsky, 2003; Angell, 2005). The organization would report its data and results to the company and the regulatory agencies, and publish them in the open scientific literature. The company would have very little influence over research design, data collection, analysis and interpretation, or publication, and the investigators would have no financial relationships to the company. Volz and Elliott (2012) have developed a similar proposal for industrial chemicals, arguing that safety testing could be coordinated by an organization such as the OECD that would receive industry funding but that would insulate study designs and interpretation from industry control. Since private sponsors usually do not want to relinquish control over how their funds are spent, this situation is rare in scientific research, but it does occur from time to time. For example, North Carolina established the Golden Leaf Foundation in 1999 to administer funds it received as part of a legal settlement with tobacco companies. The foundation sponsors economic development, education, and scientific research. The companies have no control over how the money is spent (Golden Leaf Foundation, 2012). Someone who learned that an investigator received funding from the Golden Leaf Foundation to study the health of agricultural workers would have no reason to believe that the research was influenced by tobacco companies.

To evaluate the impact of financial relationships, one must take the preceding factors (and possibly others) into account. Since these factors may vary from case to case, the evaluation of the impact of financial relationships should also reflect the circumstances. In some situations, one may decide that the financial relationships significantly undermine the study’s credibility. For example, some critics of the tobacco industry hold that the influence of financial relationships is so pervasive that industry funded research cannot be trusted (Bero, 2005; Michaels, 2008). In 2010, the editors of PLoS Medicine stated that they would no longer publish any studies supported in whole or in part by tobacco industry funding (PLoS Medicine Editors, 2010). This is a very extreme position and can only be justified in special circumstances. It may be the case that the tobacco industry’s campaign to distort and manipulate research concerning the public health impacts of smoking and tobacco use is so all-encompassing and devious that tobacco industry influence automatically negates the scientific validity of any study funded by the industry (Michaels, 2008). However, we are reluctant to take such a categorical position concerning a particular source of funding, since it is conceivable that tobacco industry money could be used to fund legitimate research related to smoking cessation programs (Eisenberg et al., 2010), medical uses of nicotine (Quik et al., 2009), or genetic engineering of the tobacco plant to produce biopharmaceuticals (Shabir et al., 2010).

In other situations, one may decide that the financial relationships only minimally undermine the credibility of the research. For example, suppose that a company sponsors a clinical trial that compares its drug to competing medications. The investigators work at academic medical centers and have no additional financial relationships to the company, such as consulting arrangements or stock. The institutions in which the research takes place also have no financial relationships to the company. The data analysis is conducted by an academic statistician who is independent of the company. One could argue that the financial relationships in this case have only a minimal impact on the credibility of the research, because the only significant relationship involves the sponsorship of the research, and there are processes in place to reduce the sponsor’s influence.

In some situations financial relationships may be viewed as having no impact on the credibility of the research. For example, suppose a government agency, such as the NIH, sponsors research on the genetics of Type II diabetes conducted by researchers at academic institutions. Neither the researchers nor the institutions own any stock related to the research. In this case, one could argue that the financial relationships have no impact on the credibility of the research, because neither the sponsor nor the researchers nor the institutions have any financial stake in the outcome. Of course, there may be other situations in which a government agency does have a significant financial stake in the outcome of a study, such as when the Department of Energy or the Department of Defense examines whether its installations pose health risks to nearby populations (Michaels, 2008). It is crucial to examine the financial stakes in specific cases to evaluate their impact on study credibility.

Finally, in some situations financial relationships might be regarded as enhancing the credibility of a study. For example, suppose that the sponsors and researchers have a financial stake in the outcome of a study and that their financial goals coincide with the goal of conducting objective and reliable research. A situation like this might occur if they are developing a product, such as a solar cell, that cannot be marketed unless potential buyers believe that it works according to specifications. The researchers cannot sell the product or increase the value of their stock unless they can prove that it functions as designed. In this situation, one might decide that the financial relationships enhance the credibility of the research, since the investigators and sponsors would have a strong financial interest in producing objective and reliable data and results.

While the five factors that we have identified can go a long way toward helping scientists, policy makers, and concerned citizens to evaluate the credibility of research, we would emphasize once again that these factors should not replace the careful evaluation of studies in terms of their scientific merits. The five factors that we have elucidated indicate when significant incentives are present for financial relationships to influence research, when there are ample opportunities for those influences to occur, and when those influences have occurred in the past. Nevertheless, this does not guarantee that particular sponsors, investigators, or studies will succumb to those influences. Thus, our five factors must still be regarded as “indirect” indications of credibility. It is still preferable to evaluate studies directly, by having experts evaluate their methodological quality in detail. Our contention is that, because there are so many limits to carrying out effective evaluations of this sort (due to constraints of time, money, and access to the unpublished methodological choices associated with scientific studies), it is crucial to supplement these scientific evaluations with thoughtful assessments of study credibility based on financial relationships.

CONCLUSION

We have argued that scientists should take financial relationships into account when evaluating research because financial relationships can impact study design, data analysis and interpretation, and other aspects of scientific investigation. Although assessment of the impact of financial relationships should not replace the careful evaluation of the scientific merits of a study, we have shown that these efforts to evaluate studies on their merits face significant limitations. Therefore, attention to financial relationships can help give scientists a better understanding of the credibility of the research. A major concern with considering financial relationships is that it can result in crude, unreliable assessments of research credibility. We have elucidated five factors that can assist scientists, policy makers, and citizens in making more sophisticated judgments of credibility. These include: whether sponsors, institutions, or researchers have a significant financial stake in the outcome of a study; whether the financial interests of the sponsors, institutions, or researchers coincide with the goal of conducting research that is objective and reliable; whether the sponsor, institution, or researchers have a history of biasing research in order to promote their financial goals; how easy it is to manipulate the research in order to achieve financial goals; and whether effective oversight mechanisms are in place that are designed to minimize bias. Since these factors may vary from case to case, the evaluation of the impact of financial relationships depends on the circumstances. In some situations, one may decide that the financial relationships significantly undermine the study’s credibility; in others, one may decide that they have no impact on credibility or even enhance it. Assessing credibility based on financial relationships is admittedly a less “direct” approach than assessing credibility based on a study’s scientific merits. Nevertheless, given the difficulties associated with performing purely scientific evaluations, financial considerations can play a valuable supplementary role in assessing study credibility.