Discordance of Conflict-of-Interest Self-Disclosure and the Centers of Medicare and Medicaid Services

Abstract

BACKGROUND

The Open Payments Database (OPD) discloses financial transactions between manufacturers and physicians. The concordance of OPD- versus self-reported COI is unknown.

MATERIALS AND METHODS

Our objectives were to compare 1) industry and self-disclosed COI in clinical literature, 2) payments within each disclosure level, and 3) industry- and self-disclosed COI and payments by specialty. This was an observational study. PubMed was searched for clinical studies accepted for publication from 1/2014 to 6/2016. Author and OPD-disclosed COI were compared. Articles and authors were divided into Full Disclosure; Incomplete Industry Disclosure; Incomplete Self-Disclosure; and No COI. Primary outcome (differences in reported COI per manuscript) was assessed using McNemar’s test. Payment differences were compared using Kruskal-Wallis Test.

RESULTS

OPD- and self-disclosed COI differed (65.0% discordance rate by manuscript, p<0.001). Percentages of authors within each disclosure category differed between specialties (p<0.001). Hematology manuscripts exhibited the highest discordance rate (79.0%) and received the highest median payment for Incomplete Self-Disclosure ($30,812).

CONCLUSIONS

Significant discordance exists between self- and OPD-reported COI. Additional research is needed to determine reasons for these differences.

INTRODUCTION

Financial relationships between healthcare workers and industry are prevalent in the United States. Self-disclosed conflicts-of-interest (COI) have been associated with the publication of favorable articles, defined as articles conveying a positive impression of a product produced by a company having an affiliation with at least 1 author. This bias influences the opinions and decisions of healthcare providers and patients^1,2,3,4,5.

Until 2010, COI among healthcare providers depended entirely upon self-disclosure as no reliable method existed to verify conflicts. The Physician Payments Sunshine Act was recently enacted to increase the transparency of relationships between clinicians and industry and led the Centers for Medicare and Medicaid Services to establish the Open Payments Database (OPD) website⁶. The OPD reports financial transactions between manufacturers, group purchasing organizations, and physicians and hospitals. As a result of the establishment of the OPD, annual payments to physicians per specialty have recently become available.

Compliance with the Sunshine Act costs $180 million annually. This burden will inevitably be transferred to patients and providers⁷. Despite this expense and effort, it remains to be determined what OPD adds to the current system of self-disclosure. Few objective methods currently exist to evaluate the accuracy of the OPD. Our aims were 1) to compare industry-reported COI (e.g. OPD) with self-disclosed COI by published authors in the medical and surgical literature and identify if there are differences between the two sources of COI disclosure, 2) to compare financial payments received within each level of disclosure and assess for any associations with amounts of payments, and 3) to compare industry-reported COI, self-disclosed COI, and financial payments by specialty to identify any fields disproportionately affected by COI.

MATERIALS AND METHODS

Search Strategy

PubMed was searched for studies in the medical and surgical literature accepted for publication between January 2014 and June 2016. These publication dates were chosen because they represent time points during which at least 6 months of OPD information is available for each author. Five specialties or subspecialties listed on the Accreditation Council for General Medical Education⁸ were randomly chosen using a computer generated random number generator. The search terms used for the five specialties were “pulmonology,” “hematology,” “orthopedics,” “cardiac surgery,” and “otorhinolaryngology”. Articles were retrieved in reverse chronological order. All peer-reviewed research studies published by senior authors from United States-affiliated institutions who had National Provider Identifier (NPI) numbers were included. United State-affiliated authors were chosen because these are the only individuals included in the OPD. Exclusion criteria were review articles, editorials, replies, technical papers, and senior authors who did not have NPI numbers. Approval by an institutional review board was not necessary.

Data Extraction

All authors from included articles were recorded along with the authors’ self-disclosed COI listed in their published manuscripts. Each author was also searched on the OPD for any industry-disclosed COI. COI were defined as payments received as honoraria; consulting fees; compensation for serving as faculty/speaker; research-funding payments; or company ownerships/partnerships within the year preceding publication of an article; and food and travel reimbursements equivalent to or exceeding $5,000 provided entirely by a single source. Authors with no information recorded on the OPD were assumed to have no industry-reported COI per the OPD database. COI disclosed in the published manuscripts were compared to the financial relationships reported by industry listed in the OPD. Articles and authors were subsequently divided into 4 categories: 1) full disclosure, 2) incomplete industry disclosure, 3) incomplete self-disclosure, and 4) no COI. The authors of manuscripts in the Incomplete Disclosure category self-disclosed COI that did not perfectly match with those COI listed in the OPD. Incomplete industry disclosure refers to manuscripts in which industry listed some but not all COI listed by the authors. Incomplete self-disclosure refers to manuscripts in which authors did not list all COI listed by industry (please refer to Supplemental Table 1 for examples). Individual payments satisfying the COI criteria were summated for a total COI payment to each author. Payments for authors within the Incomplete Industry Disclosure and the No COI categories could not be calculated due to the lack of availability of complete funding information. Multiple reviewers rechecked the listed COI at different time points in anticipation that modifications would be made to the industry-disclosed COI following author disputes.

Statistical Analysis

Power calculations were based upon a prior study of ventral hernia literature⁹ in which the estimated difference between self-reported and OPD-reported COI was 45%. Assuming an alpha of 0.05, beta of 0.20, and a 50% reduction in effect size, we estimated a sample size of 100 manuscripts (per specialty) would be needed. The primary outcome was differences in reported COI per manuscript and the secondary outcome was differences per author. Both were assessed using McNemar’s test. Differences in payments were compared using the Kruskal-Wallis Rank-Sum Test. Statistical analysis was performed with STATA 14.1® software. The overall discordance rate of industry-reported and self-disclosed COI was determined to be the summation of the Incomplete Self-Disclosure and Incomplete Industry Disclosure groups.

RESULTS

Over 1200 articles were screened, chosen in chronological order from most recent to oldest per a Pubmed search by the designated search term, to identify 500 manuscripts (100 per category) that met the inclusion criteria. Exclusions included articles with authors primarily affiliated with institutions outside the United States (n=588), letters/commentaries (n=143), and inability to access full text (n=17). Among 500 manuscripts, 333 (66.6%) manuscripts met criteria for COI. Among all authors (n=2,898), 855 (29.5%) met the criteria for a COI (self-disclosed or OPD-disclosed). The overall discordance rate of industry-reported and self-disclosed COI was 65.0% (325/500) for manuscripts and 28.4% (824/2,898) for all authors (Tables 1A and 1B). Between the medical and surgical published literature, the discordance rate for manuscripts differed significantly (71.5% vs 60.7%, p=0.01). Financial payments received by authors did not significantly differ between the Full Disclosure and the Incomplete Self-Disclosure groups (p=0.80) (Table 2). The percentages of authors within each COI disclosure category differed significantly between specialties (p<0.001) (Table 3A).

Table 1A.

Distribution of Manuscripts into Each COI Disclosure Category (n=500)

		Industry Disclosure (OPD)
		Yes	No
Self-Disclosure	Yes	Full Disclosure 8 (1.6%)	Incomplete Industry Disclosure 33 (6.6%)
	No	Incomplete Self-Disclosure 292 (58.4%)	No COI 167 (33.4%)

OPD=Open Payments Database; COI=Conflicts-of-interest; (Number of authors, percentage of total authors)

Table 1B.

Distribution of Authors into Each COI Disclosure Category (n=2,898)

		Industry Disclosure (OPD)
		Yes	No
Self-Disclosure	Yes	Full Disclosure 31 (1.1%)	Incomplete Industry Disclosure 181 (6.2%)
	No	Incomplete Self-Disclosure 643 (22.2%)	No COI 2,043 (70.5%)

OPD=Open Payments Database; COI=Conflicts-of-interest; (Number of authors, percentage of total authors)

Table 2.

Financial Payments for Authors Falling into Each COI Disclosure Category [USD; Median (IQR)]

		Industry Disclosure (OPD)
		Yes	No
Self-Disclosure	Yes	Full Disclosure $17,900 (9,093–67,233)	Incomplete Industry Disclosure NA*
	No	Incomplete Self-Disclosure $17,007 (3,718–95,784)	No COI NA*

COI=Conflicts-of-interest; IQR=Interquartile range USD=United States Dollars

^*Denotes fields that could not be calculated due to the inability to find the relevant financial information within the OPD.

Table 3A.

Number and Percentage of Manuscripts Falling into Each COI Disclosure Category per Specialty

	Surgical			Medical
	Orthopedics (n=100)	Cardiac Surgery (n=100)	Otorhino-laryngology (n=100)	Hematology (n=100)	Pulmonology (n=100)	p-value
Full Disclosure	5 (5.0%)	1 (1.0%)	2 (2.0%)	0 (0.0%)	0 (0.0%)	<0.001
Incomplete Self-Disclosure	54 (54.0%)	67 (67.0%)	42 (42.0%)	75 (75.0%)	54 (54.0%)
Incomplete Industry Disclosure	15 (15.0%)	2 (2.0%)	2 (2.0%)	4 (4.0%)	10 (10.0%)
No COI	26 (26.0%)	30 (30.0%)	54 (54.0%)	21 (21.0%)	36 (36.0%)

COI=Conflicts-of-interest; IQR=Interquartile range

^*Denotes fields that could not be calculated due to the inability to find the relevant financial information within the OPD.

Hematology manuscripts exhibited the highest discordance rate (79.0%) while Otorhinolaryngology manuscripts showed the lowest discordance rate (44.0%). The amount of financial payments received between the individual specialties significantly differed for the Incomplete Self-Disclosure category (p<0.001) but did not significantly differ for the Full Disclosure (p=0.34) category (Table 3B). Hematology received the highest median payment for the Incomplete Self-Disclosure category ($30,812) while Otorhinolaryngology received the lowest ($3,000).

Table 3B.

Financial Payments for Authors Falling into each COI Disclosure Category per Specialty [USD; Median (IQR)]

	Surgical			Medical
	Orthopedics (n=496)	Cardiac Surgery (n=548)	Otorhino-laryngology (n=509)	Hematology (n=737)	Pulmonology (n=608)	p-value
Full Disclosure	$10,500 (9,093–25,875)	$67,233 (13,120–181,643)	$8,152 (500–15,805)	$106,183 (2,000–200,000)	$21,044 (7,950–30,928)	0.34
Incomplete Self-Disclosure	$7,387 (1,920–38,283)	$15,965 (4,229–104,598)	$3,000 (446–14,240)	$30,812 (7,122–145,087)	$24,277 (4,260–119,917)	<0.001
Incomplete Industry Disclosure	NA*	NA*	NA*	NA*	NA*	NA*
No COI	NA*	NA*	NA*	NA*	NA*	NA*

COI=Conflicts-of-interest; IQR=Interquartile range USD=United States Dollars

^*Denotes fields that could not be calculated due to the inability to find the relevant financial information within the OPD.

DISCUSSION

Financial relationships are common between medical practitioners and industry^{10,11,12,13,14}. In this study, approximately two-thirds of published manuscripts and one-fourth of the published authors across the medical and surgical literature had a monetary interaction that qualified as a COI. Our data demonstrate that the majority of authors with a COI had self-disclosed COI that differed from industry-reported COI listed in the OPD. These results support the suspicion that discordant reporting is pervasive throughout the medical and surgical scientific literature. The specific amount received as payment did not, however, appear to impact disclosure status (full disclosure versus discordant disclosure).

Multiple potential explanations exist for this high discordance rate, including accuracy of the data source, disclosure criteria, and human error. The OPD obtains its information from industry reports of financial transactions. Although medical practitioners are given the opportunity to review and edit the posted results, the frequency of such a review process remains unknown. Another potential reason for the discordance is differing COI disclosure criteria by academic journals¹⁵. Some COI may not be perceived as relevant to a manuscript by the reporting author, and therefore may not be reported. A valid COI that is not reported may also simply be due to an oversight by the author(s) or journal.

Regardless of the disclosure status (disclosed or undisclosed), COI by itself may have a negative impact on clinical evidence, public opinion and decision-making as it can inadvertently introduce bias to a study¹⁶. Some published reports have demonstrated that studies with COI as compared to studies with no COI are more likely to report favorable outcomes. Researchers with conflicts were more likely to selectively choose parameters that produce better results and to report larger magnitudes of differences versus researchers with no COI^5,17,18. However, the effects of unreported COI on study results are currently unknown.

Our data draws attention to many issues in present COI disclosure policies and in current legislative handling of COI reporting policies. Establishment of the OPD by the Centers for Medicare and Medicaid now allows for a new non-self-disclosed or OPD-disclosed COI category for authors who have COI that they do not self-report but that are contained on the OPD database. All previous literature has only evaluated the impact of self-disclosed COI. Further studies are needed to determine the effects of non-self-disclosed COI on study outcomes and if these non-self-disclosed COI are important to recognize. Additional policies are needed to standardize the COI disclosure process due to the high discordance rate. The COI-reporting process needs improvement. This reform may be achieved by simply referring all manuscript readers and conference viewers to the OPD site (thereby eliminating issues with author self-disclosure); encouraging all authors to disclose all COI within the broad time period during which the manuscript was written and not simply those COI that they consider relevant to their research; and scientific journals and academic oversight bodies more harshly penalizing authors if authors are discovered to conceal COI. Evidence-based guidelines must be developed to standardize disclosure processes among journals and meetings. Improving the management of COI may improve the quality of published research literature, much of which is biased despite having considerable influence on practice recommendations^19,20. Further study should also determine if additional factors, such as career position and role in authorship, are associated with or affect differing levels of COI disclosure. These factors are outside the scope of the present study.

There are a number of study limitations. First, the accuracy of the OPD cannot be validated. A previous retrospective observational study determined that only 62% of individuals listed as neurosurgeons within the OPD in 2013 were correctly identified for their specialty, casting doubt on the accuracy of additional information such as payment data²¹. We recognized this limitation with the OPD and therefore, instead of considering the OPD as a gold standard, decided to assess correlation or discordance with author self-disclosure. Secondly, the OPD can only track healthcare providers with a NPI number. Not all authors may have an NPI number including statisticians, medical students, and research associates/assistants. However, exclusion of these authors could only increase the proportion of eligible authors who have a COI or have discordant COI.

The authors do seek to emphasize that we do not discourage a partnership between industry and science. Collaboration is vital for advancement. If COI are present, authors must take steps to minimize bias by utilizing blinded procedures/data collectors, randomized study designs, research outcomes decided by third party organizations, and oversight by academic bodies familiar with the research area that may review study protocols to evaluate bias prior to protocol implementation. Published literature has already alluded to similar steps¹⁶. Researchers must also commit to posting all results, both positive and negative for the industry partner, in an open access format. This is currently pursued through registries like clinicaltrials.gov²².

CONCLUSIONS

The overall discordance rate among manuscripts for author self-disclosure versus industry COI disclosure on the OPD among the sampled literature was 65.0%. Issues with COI disclosure were found to affect all of the medical and surgical specialties examined. Further research is needed to determine the impact of this discordance on research outcomes and if standardized, evidence-based guidelines for COI disclosure across scientific journals are needed.