Evidence for stratified conflicts of interest policies in research contexts: a methodological review

Abstract

Objectives

The purpose of this study was to conduct a methodological review of research on the effects of conflicts of interest (COIs) in research contexts.

Design

Methodological review.

Data sources

Ovid.

Eligibility criteria

Studies published between 1986 and 2021 conducting quantitative assessments of relationships between industry funding or COI and four target outcomes: positive study results, methodological biases, reporting quality and results–conclusions concordance.

Data extraction and synthesis

We assessed key facets of study design: our primary analysis identified whether studies stratified industry funding or COI variables by magnitude (ie, number of COI or disbursement amount), type (employment, travel fees, speaking fees) or if they assessed dichotomous variables (ie, conflict present or absent). Secondary analyses focused on target outcomes and available effects measures.

Results

Of the 167 articles included in this study, a substantial majority (98.2%) evaluated the effects of industry sponsorship. None evaluated associations between funding magnitude and outcomes of interest. Seven studies (4.3%) stratified industry funding based on the mechanism of disbursement or funder relationship to product (manufacturer or competitor). A fifth of the articles (19.8%) assessed the effects of author COI on target outcomes. None evaluated COI magnitude, and three studies (9.1%) stratified COI by disbursement type and/or reporting practices. Participation of an industry-employed author showed the most consistent effect on favourability of results across studies.

Conclusions

Substantial evidence demonstrates that industry funding and COI can bias biomedical research. Evidence-based policies are essential for mitigating the risks associated with COI. Although most policies stratify guidelines for managing COI, differentiating COIs based on the type of relationship or monetary value, this review shows that the available research has generally not been designed to assess the differential risks of COI types or magnitudes. Targeted research is necessary to establish an evidence base that can effectively inform policy to manage COI.

Strengths and limitations of this study.

• We considered a broad range of available research on the effects of industry funding and conflict of interest (COI) on biomedical research.

• This methodological review evaluates research designs assessing the relationships between industry funding or author COI and biomedical research.

• We achieved high inter-rater reliability for article screening.

• This review does not address studies of the relationships between industry funding or COI and guidelines development, regulatory decision making or clinical practice.

Background

Substantial evidence indicates that industry funding of biomedical research and author financial conflicts of interest (COIs) arising from financial relationships with medically related industry can bias research results.^1–7 Associations between industry funding or COI and positive outcomes, such as results favourable to the sponsor, are the most well documented.^{2–5 7} Available evidence indicates that industry-funded trials can be up to 5.4 times more likely to return positive results than trials not sponsored by industry,⁸ and trials with author COI may be as much as 8.4 times more likely to return favourable results when compared with those without author COI.⁶ Additional research has demonstrated that industry funding and COI may be associated with reduced drug and device safety^{6 9} and can have adverse effects on the methodological quality of clinical trials.^10–12 Recent studies also suggests that industry sponsorship may be associated with premature trial termination and non-reporting of trial results.^{13 14} Calls for more evidence documenting that industry funding and COI can measurably bias biomedical research persist even though these findings have been repeatedly replicated.¹⁵

Recognising the risks of bias, many organisations involved in biomedical research have adopted specific policies designed to address industry funding and COI. These include both policies designed to manage the risks associated with individual researcher COIs and guidelines addressing potential institutional COI resulting from industry gifts and research sponsorship. The need for such policies is clear, which in turn raises important questions about the form those policies should take. Differentiation among COI types and magnitudes is a common feature of the policies adopted by universities, academic medical centres (AMCs), government laboratories and similar research institutions. COI policy guidelines published in the literature and by professional medical organisations also routinely differentiate among different COI types and magnitudes. That is, COI policies and guidelines routinely make distinctions based on the method of remuneration (industry employment, consultancy relationships, honoraria, travel fees, etc), the nature of the funder (eg, industry, nonprofit, government agency), the recipient of remuneration (eg, self, partner, family and collaborator) and the magnitude or monetary value of the disbursement. Table 1 describes explicit recommendations by the American Medical Student Association (AMSA),¹⁶ the Association of Academic Medical Centers (AAMC),¹⁷ the British Medical Association (BMA)¹⁸ and Brennen et al.¹⁹

Table 1.

Illustrative recommendations for strata-specific COI policies

COI	AMSA	AAMC	BMA	Brennen et al
Attendance at unaccredited industry-sponsored events	Prohibit			Prohibit
Consulting	Restrict
Donations			Disclose
Ghostwriting	Prohibit	Prohibit		Prohibit
Gifts	Prohibit	Prohibit	Prohibit	Prohibit
Grants			Disclose
Industry access: device representatives	Restrict	Restrict		Restrict
Industry access: pharmaceutical representatives	Prohibit	Restrict	Restrict	Prohibit
Industry sponsored continuing medical education	Restrict	Restrict		Restrict
Industry sponsored scholarships		Restrict
Meals	Prohibit			Prohibit
Pharmaceutical samples		Restrict
Research contracts			Disclose
Speakers bureaus	Prohibit			Prohibit
Travel funds		Restrict
Travel for industry sponsored meetings		Prohibit
Travel funds for trainees	Prohibit		Prohibit	Prohibit
Treatment inducements	Prohibit

This table shows AMSA,¹⁶ AAMC,¹⁷ BMA¹⁸ and Brennen et al’s¹⁹ recommendations for whether AMC COI policies should prohibit, restrict or require disclosure of specific COI strata. Where entries are blank, the guidance provided no specific recommendations for that type of relationship.

AAMC, Association of Academic Medical Centers; AMC, academic medical centre; AMSA, American Medical Student Association; BMA, British Medical Association; COI, conflict of interest.

These COI policies and guidelines suggest that some types of COI should be prohibited outright, others should be subjected to specific restrictions and some should merely require disclosure. However, different policies and guidelines do not agree on the risk presented by different types or magnitudes of COI. The recommendations typically advise a total prohibition on gifts from industry and ghostwriting but recommendations about other COI types vary widely. For example, AMSA recommends restrictions on consulting fees, but the AAMC, BMA and Brennen et al do not address consultancies outside general recommendations for transparency via COI disclosure. All four guidelines disagree if industry representative access to research spaces should be restricted or prohibited outright.

Various policies also make distinctions about the magnitude or monetary value of COI to set disclosure thresholds. However, recommended thresholds vary widely within and between organisations. For example, since 1995, the US Department of Health and Human Services has required AMCs and other entities that receive federal research funding to adopt policies that require disclosure of COI over a certain threshold.²⁰ This value was lowered from $10 000 to $5000 in 2011.²¹ The BMA sets the declaration threshold for gifts at £500 and for equity holdings at greater than 1% of the value of the company or greater than £25 000.¹⁸

The establishment of approaches to COI management that differentiate by type and magnitude indicate that common guidance assumes that different COI types and magnitudes carry different degrees of risk for biomedical research and require different responses. This assumption even drives much of the available research on COI policies at AMCs and similar institutions. The AMSA scorecard, for example, is a well-established framework for COI policy evaluation.^{16 22} It has been used to assess the extent to which COI policies at AMCs in the USA,¹⁶ France²³ and Germany²⁴ follow AMSA recommendations for COI policy construction and stratification.

Despite the significant investments in developing and evaluating stratified COI policies, it is not clear that different types of COI do, in fact, carry different risks or levels of risk for biomedical research. If one were to assess the efficacy of COI policies (ie, determine if COI policies have any effects on the quality of research), one must first assess whether policies stratified by COI types are grounded in evidence about the differential risks of different COI types. This study sought to assess the extent to which orthodox research designs for assessing the effects of COI on biomedical research have been designed to generate evidence relevant to the stratification of COI policies. Demonstrating the existence of differential risk profiles for different COI types would require, at minimum, research designs that stratify COI variables prior to analysis. They should further disaggregate industry research sponsorship generally from specific forms of author COI. Therefore, the goal of this methodological review is to evaluate the extent to which study designs in available industry funding and COI research can support COI policies or that policy recommendations should assume differential risk profiles for different types of COI and/or different monetary values. Put another way, the evidence for the need for mitigating the risks imposed by COI is strong, but the state of the research that can guide how to manage that risk is unclear. This study reviews methodological designs for: (1) industry funding variable stratification and disaggregation, (2) COI variable stratification and disaggregation and (3) diversity of outcomes assessments.

Methods

Methodological reviews are designed to provide information on the prevalence of available study designs in a body of literature. They have facilitated advances in a wide variety of health and health policy contexts and can be used to identify and prioritise new pathways for research.^25–28 A methodological review is the ideal approach for this study, which requires identifying if research on the effects of industry funding and COI has been conducted in ways that could support current COI policy stratifications. Our review proceeded in three phases. First, we replicated the search strategy and article screening protocol for a previously published Cochrane systematic review of the effects of industry funding on biomedical research.² The prior Cochrane review evaluated the overall strength of the evidence base regarding the association of industry funding with results favourable to the sponsor, risks of bias associated with the methodological design and the quality of reporting of the concordance between results and conclusions, but it did not document the methodological design elements in focus in this study.² While the meta-analysis did not expressly evaluate author COI as an isolated variable ‘conflicts of interest’ was a key term in the search strategy, and many articles included in the Cochrane review used COI as proxy for industry funding. Our study adopted the search strategy and screening protocol of the original review, and the second phase of this review involved conducting a novel assessment of the methodological features of included articles, with particular focus on how industry funding and COI variables were operationalised in statistical analyses. Finally, we used these data to synthesise the evidence for evaluating different types of industry funding or author COI on target outcomes in biomedical research.

Search strategy and study selection

We began by replicating the search strategy in a previously published Cochrane review. The strategy was designed to identify relevant articles indexed in the Ovid database. (See the online supplemental materials for complete details.) The original review and screening protocol identified 75 studies of interest published between 1986 and 2016. We retrieved each of the original 75 studies, and in June 2021, we repeated the search strategy to collect additional relevant articles published since 2016. We also replicated the study inclusion protocol from the previous Cochrane review. Specifically, eligible studies provided a quantitative assessment of the extent to which industry funding or author COI were associated with target outcomes of interest (ie, results favourable to industry, methodological biases, reporting quality and results–conclusions concordance) within research on drug and device products. All collected studies evaluated one of these outcomes on a dataset of clinical trials. Clinical trials data may come from published articles, clinical trials registries or both. Studies of the effects of industry funding and/or COI in research areas related to smoking, nutrition, physical therapy, psychotherapy and surgery were excluded except in cases where analyses were performed on separate identifiable drug or device data. Additionally, studies that evaluated the effects of industry funding or COI on clinical practices, guidelines development, patient organisations and regulatory policy were excluded.

Three evaluators screened titles and abstracts. After initial norming, a random sample of 255 titles and abstracts were selected by all three raters to assess reliability across screeners. A sample size of 255 was chosen to achieve 90% assurance using the intraclass correlation coefficient (ICC).²⁹ Overall agreement between the three raters was 94.9% with an ICC=0.801. A secondary analysis of the random sample indicated that the abstracts for all articles selected for further screening included at least one of the following terms: ‘funding’, ‘funded’, ‘COI’, ‘fCOI’, ‘conflict’ or ‘sponsor’, which allowed us to develop an automated screening tool based on those terms. Articles selected for full-text review passed both automated and manual screening. The full article text of the remaining articles was evaluated by three raters.

Data extraction and synthesis

The current methodological review was designed to collect data on the underlying analytic designs in selected articles. Specifically, the investigators collected data on which independent and dependent variables had been operationalised and defined. That is, each industry funding and COI independent variable was categorised as ‘stratified’, ‘unstratified’ or ‘magnitude’. Here, ‘stratified’, refers to what is often called categorical or nominal variables. For example, a study that stratified industry funding variables might assess if funding provided by a drug manufacturer or a competing pharmaceuticals company has differential impacts on target outcomes. Similarly, a study that stratified a COI variable might evaluate the relative impact of different disclosed COI types such as ‘industry employed author’, ‘receipt of consulting fees’ or ‘receipt of travel fees’. We classified independent variables as ‘magnitude’ if they assessed industry funding or COI as a continuous or ordinal variable. This might mean assessing industry funding in terms of disbursed amounts (eg, $5000 or £20 000) or the total number of COI per article. Relevant variables were identified as ‘unstratified’ when they were assessed as simply present or absent (eg, industry funded vs non-industry funded or reported COI vs no reported COI). We also noted if variables had been dichotomised prior to analysis. This occurs when articles present stratified variable data as part of descriptive statistics, but then perform statistical analyses on simplified, unstratified, dichotomous industry funding or COI variables.

Our analysis also assessed whether author COI was used as a proxy for industry funding. This research design choice would indicate that the article in question did not fully disaggregate general industry sponsorship from specific types of author COI. Each outcome variable was also categorised according to the primary domain of interest, including outcome favourability to sponsor, drug or device safety; quality of study design or reporting; and if results were reported at all. Finally, for all articles with stratified independent variables for industry funding or author COI, we identified clinical areas of interest, sample sizes used, each assessed stratum, outcomes against which the stratum were assessed, significance of the results and any reported effect sizes. A complete description of the criteria is available in online supplemental table 1.

Patients and public involvement

No patients or public were involved in the study.

Results

Our replication of the previously published search strategy retrieved 3884 unique records for articles published in 2016 and later. Automated screening removed 2671 articles from consideration. Subsequent manual screening of titles and abstracts excluded another 926 articles. The remaining 287 articles were selected for full-text review, and 92 studies were ultimately selected for inclusion. An additional 75 articles were included from the pre-existing systematic review for a dataset of 167 articles (see figure 1).

Figure 1.

PRISMA flow diagram. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Industry funding variable assessment

Of the 167 articles included in this study, a substantial majority (n=164, 98.2%) evaluated the effects of industry sponsorship (see online supplemental table 2). In most cases, industry funding was determined based on an article’s acknowledgements or sponsorship declaration. However, some studies collected data from clinical trials registries like clinicaltrials.gov, which index sponsorship. Notably, 35 studies (21.3%) assessing industry funding used author employment in industry or other author COI as part of the inclusion criteria for a variable identified as ‘industry funding’ or ‘industry sponsorship’. Studies also used industry provision of drugs or devices as a criterion for industry funding. Others treated provision of supplies as its own isolated variable.

Among the articles that assessed industry funding in some form, none evaluated associations between funding magnitude and outcomes of interest. Ten studies (6.1%) collected stratified data on industry funding but dichotomised the variable prior to statistical analysis. Only seven studies (4.3%) stratified industry funding for analysis in any way. Evaluated strata included details about the nature of the sponsor (evaluated drug manufacturer vs competitor company) or the nature of the sponsorship (full study sponsorship, collaborative sponsorship with other funders or provision of medications). Three of the seven studies included assessed differences in favourable outcomes based on funder relationship to the product evaluated (eg, manufacturer vs competitor company).^30–32 Only one study found significant results³⁰: this review of 542 psychiatry studies found that a greater percentage of studies sponsored by the drug manufacturer have positive outcomes than those not sponsored by a pharmaceutical company (78% vs 48%) and that research sponsored by a competitor had the lowest rate of favourable findings (28%). Pairwise comparisons between manufacturer-funded or competitor-funded and non-industry-funded studies were significantly different, but the study reported no indicators of effect size. Three studies evaluated strata related to the mode of industry involvement.^33–35 These studies assessed the relationship between favourable outcomes and industry provision of medication, report of findings in an industry publication venue and other (unspecified) industry involvement. One study found significant results and reported that ‘other’ industry involvement associates with favourable outcomes for industry.³⁵ See table 2 for further details. In sum, a substantial proportion of the research that might provide insight into COI policy design assesses only industry sponsorship generally. Nearly a quarter of the assessed studies conflate industry funding and COI variables making it impossible for results to shed light on potentially useful policy differences. And, finally, studies of industry funding that do stratify variables primarily provide insight on different sponsorship modalities and not on issues related to author COI.

Table 2.

Industry funding and COI strata assessed and associated results

Article	Area	Samp.	Outcome	Strata	Sig.	Effect measure	Effect
Ahmer 200533	Psychiatry	306	Outcome favourability	Industry provided medications	0.053	–	–
				Author is industry employee	0.01*	OR	8.33 (1.64–50.0)
Bartels 201237	Spine research	51	Outcome favourability	Disclosed COI	<0.05*	OR	16.5 (4.7–58.1)
				Statement of no COI	–	–	–
				Disclosure not required by journal	–	–	–
Bond 201234	Asthma	91	Outcome favourability	Industry sponsorship	0.546	–	–
				Industry publication venue	0.191		–
				Other industry involvement	NR	–	–
				Author is industry employee	0.003*	Risk ratio	1.42 (1.10–1.82)
Jinapriya 201131	latanoprost	44	Outcome favourability	Sponsorship by parent company	0.53	–	–
				Sponsorship by competing company	0.53	–	–
Kelly 200630	Psychiatry	542	Outcome favourability	Sponsorship by manufacturer	0.001*	–	–
				Sponsorship by competing company	0.001*	–	–
Rattinger 200932	Thiazolidinediones	61	Outcome favourability	Sponsorship by manufacturer	0.7778	–	–
				Sponsorship by competing company	0.037*	OR	0 (0,0.886)
				No funding disclosure	0.4153	–	–
				Corresponding author COI	0.3939	–	–
				Corresponding author is sponsor employee	0.5714	–	–
				Corresponding author no disclosure	0.4388	–	–
				Corresponding author COI with sponsor	0.049*	OR	4.125 (1.048;19.525
				First author COI	0.1667	–	–
				First author is sponsor employee	–	–	–
				First author no disclosure	–	–	–
				First author COI with sponsor	0.4588	–	–
Vlad 200735	Osteoarthritis	15	Outcome favourability	Industry sponsorship	0.05	–	–
				Other industry involvement	0.02*	Random effects	0.55 (0.29–0.81)
				Author COI	0.04*	Random effects	0.55 (0.27–0.84)
Cherla 201838	Multiple	590	Outcome favourability	Full disclosure	0.001*	OR	8.65 (2.46–30.44)
				Incomplete industry disclosure	0.003*	OR	3.61 (1.53–8.51)
				Incomplete self-disclosure (partial)	0.004*	OR	4.14 (1.58–10.82)
				Incomplete self-disclosure (none)	0.002*	OR	0.14 (0.37–1.15)
Saa 201836	Probiotics	66	Outcome favourability	Industry sponsorship	0.491	–	–
				Non-disclosure of Sponsorship	0.491	–	–
			Methodological or reporting quality	Industry sponsorship	0.491	–	–
				Non-disclosure of sponsorship	0.491	–	–

This table describes the clinical area, methodological design (sample, outcome, variable strata) and results of analysis presented in articles that evaluated identifiable industry funding and COI strata.

COI, conflict of interest.

COI variable assessment

Of the 167 articles evaluated, only 33 (19.8%) assessed COI as a discrete variable. Attention to COI began considerably later in the dataset, not appearing until 2005. Most studies that evaluated author COI relied on the data in the published disclosure statement. A handful of studies used the author’s institutional affiliation as an indicator of industry employment, and a few studies also compared disclosure statements to data available in the Open Payments Database. Of the articles that evaluated author COI, none assessed COI magnitude, and only three studies (9.1%) stratified COI for analysis. Four studies (12.1%) collected stratified COI data but dichotomised it prior to analysis. The few studies that assessed COI strata independently tended to evaluate disclosure practices as opposed to COI types.^36–38 These articles report on evaluations of the relationship between favourable outcomes or methodological quality and COI disclosure, lack of funding disclosure, incomplete disclosure, lack of disclosure requirements by journal or affirmative statements of no author COI. Disclosure of COI and ‘full’ disclosure of COI were most strongly associated with results favourable to industry.^{37 38} Here ‘full’ disclosure meant that all payments reported to the Open Payments Database were reflected in published disclosure statements. Assessments of these different disclosure practices returned non-significant results or smaller effect sizes. Two studies evaluated the relationship between participation of industry-employed authors and results favourable to industry.^{33 34} An evaluation of 215 psychiatric studies published between 1998 and 2003 found that participation of industry authors was significantly associated with favourable outcomes.³³ Similarly, an assessment of 91 asthma product studies found that favourable outcomes were more likely for studies with industry-employed authors³⁴ (see table 2).

Target outcomes evaluation

Most studies in the dataset (n=108, 64.7%) evaluated the relationship between industry funding or COI and outcomes favourability for sponsors. Sixty-six (39.5%) evaluated methodological or reporting quality. Nineteen (11.4%) assessed reporting of results, and 15 (9.0%) evaluated drug or device safety. Attention to specific outcomes appears to have changed over time. Industry favourability of study outcomes had long been the dominant focus of research on industry funding and COI. Quality, safety and reporting grew increasingly prevalent (figure 2). Importantly, however, studies that stratified industry funding or COI variables were less diverse in their target outcomes. Of the 10 studies that stratified relevant variables, outcomes favourability to industry was assessed in all cases. One study also assessed the relationship between disclosure practices and methodological or reporting quality.³⁶

Figure 2.

Variable types by year number (1986–2021). Figure includes number of studies per year by dependent variable (DV) type (A), number of studies by independent variable (IV) type for studies assessing industry funding (B) and number of studies by IV type for studies evaluating COI (C). COI, conflicts of interest.

Discussion

For COI policies to make effective distinctions based on nature of relationships or amount of remuneration, these distinctions must be grounded in research that assesses differential risk profiles of COI types and magnitudes. However, a substantial majority of research assessing the effects of industry funding and author COI on biomedical research does not stratify relevant variables. Remarkably, zero studies included in this review conducted any assessments of the magnitude of either industry funding or author COI. Additionally, the available literature’s ability to support evidence-based stratifications in COI policies is further compromised by regular conflation of industry sponsorship and author COI variables as well as the practice of dichotomising variables prior to conducting statistical analyses. The few studies that did stratify COI variables tended to focus on disclosure practices rather than COI types, and most studies assess only if COI types associated with results favourable to industry and not if they associated with other target outcomes of interest. These findings point to limitations in current disclosure practices that allow authors a great deal of latitude in reporting and describing COI. The variability of disclosure statements limits the extent to which research on COI can evaluate differential effects. Nevertheless, the results of this methodological review indicate that the available research on industry funding and COI has generally not been designed to guide COI policy stratifications or the establishment of disclosure thresholds.

Appropriate and evidence-based COI policies are essential for safeguarding the integrity of the biomedical research enterprise. Therefore, it is critical that research can meaningfully inform continued policy refinement. Clearly, guiding the design of COI policy requires additional research designed to assess the differential risks associated with various COI types and magnitudes.

Furthermore, research in this area could also be better supported by the development of standardised taxonomies of industry funding and/or author COI. Since the literature variously defines ‘industry funding’ as sponsorship, employment, provision of medications or any author COI, it is quite difficult to compare and aggregate findings across studies. Likewise, competing understandings of author COI based in different disclosure practices and type definitions also indicate the strong need for robust taxonomies that can guide future research. Empirically validated taxonomies could also support more consistent disclosure practices, which would aid future research evaluating the differential effects of COIs by type or magnitude.

These taxonomies combined with evidence about the magnitude of COIs would allow for computation and aggregation of COIs essential for supporting research that could effectively guide COI policy refinement. New research on the risks of COI would also benefit from continued diversification of outcomes assessment. Recent years have seen a steady expansion of outcomes of interest (eg, outcomes favourability giving way to more assessments of quality, safety and reporting practices), but favourability of results is still the overwhelmingly dominant target outcome.

Finally, the results of this review also suggest that researchers and policymakers would benefit from considering COI risks beyond those manageable at the individual researcher level. It is notable that common COI policies and guidelines tend to be strict with respect to relationships of modest economic benefit to individuals (eg, meals and travel), whereas relationships with well-documented risks but considerable economic benefit to institutions (eg, industry grants and collaborations) are largely left out of COI policy recommendations. Furthermore, the strongest evidence relates to author employment in industry, although specific instructions about disclosing employment have been removed from the latest ICMJE disclosure guidance. Given that collaborations with industry are a common form of institutional COI, and one not addressed by individualised COI policies, these findings support recent calls for greater attention to institutional COI at institutions that conduct biomedical research.^39–42 Research conducted primarily at universities, AMCs and other research institutions may be more prone to bias when it is supported by industry funding or industry collaboration. COI policies that focus on individual researchers alone will not mitigate against these risks.

This study has several limitations that should inform the reading of the findings. Our review evaluates the methodological design and approaches to variable stratification for studies of the relationships between industry funding or author COI and four specified outcomes of interest in biomedical research. Although we are aware of studies that evaluate COI magnitude, for example, they were not returned by our search strategy either because they treat COI magnitude in the aggregate⁴³ or because they assess non-target outcomes such as associations with commercial publishing practices.⁴⁴ Additionally, AMC guidelines are designed to respond to COI risks in multiple domains including research, clinical practice and medical education. We assume that COI strata related to industry-funded continuing medical education or pharmaceutical representative access to AMCs are designed primarily to address risks of bias associated with medical education and clinical practice. Additional research not covered by this review is available that evaluates the relationships of industry funding and COI with prescription practices, guidelines development and policy decision making.

These limitations notwithstanding, the results suggest that policies designed to address COI risks associated with clinical practice may not effectively safeguard the integrity of biomedical research across institutional contexts because of the gap between policy and available COI research. Furthermore, it is possible that a one-size-fits-all COI policy may not be appropriate. Additional efforts should be made to ensure that COI policies are responsive to risks associated with bias in biomedical research. For example, AMCs should potentially consider differential policies based on institutional roles. Future research might, therefore, investigate the utility of separate COI policies for clinical, educational and research staff as well as staff holding multiple roles. In such cases, it might be appropriate to require staff to adhere to the most restrictive policy. COI policies should be developed based on an understanding of the differential effects of distinct strata and magnitudes of COI on outcomes across the multiple domains.

Conclusion

Current COI policies in research contexts devote considerable attention to distinguishing between different types and magnitudes of COI. Although substantial evidence exists that industry funding and COI have adverse effects on biomedical research, the current evidence cannot guide policy stratification by type or magnitude. Given the broad adoption of policies that distinguish between COI types and set disclosure thresholds, the shortcomings identified here are weaknesses of current research that must be addressed. Importantly, however, we are not calling for a suspension of COI policies while this research is conducted. Inaccurate claims to insufficient evidence have long served to limit the scope of COI policies and to delay adoption.¹⁵ A precautionary approach would involve adopting more restrictive unstratified policies until such time that certain COI types are demonstrated to be of lower risk. Furthermore, our findings also suggests that these problematic claims may have adversely affected COI research itself. Unspecified calls for ‘more research’ might partially explain why, despite the clear findings of the 2017 meta-study,² so many studies continue to assess if COI has an effect rather than which COI have what effects and why. Instead of suggesting the need for more COI research broadly, the current methodological review points towards targeted research needs about COI types and magnitudes. If stratified policies at research institutions are to mitigate the risks of COI, they must be based on comparative assessments of differential risks.

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

Not applicable.