Reporting of conflicts of interest by authors of primary studies on health policy and systems research: a cross-sectional survey

Abstract

Objectives

The objective of this study was to assess the frequency and types of conflict of interest (COI) disclosed by authors of primary studies of health policy and systems research (HPSR).

Design

We conducted a cross-sectional survey using standard systematic review methodology for study selection and data extraction. We conducted descriptive analyses.

Setting

We collected data from papers published in 2016 in ‘health policy and service journals’ category in Web of Science database.

Participants

We included primary studies (eg, randomised controlled trials, cohort studies, qualitative studies) of HPSR published in English in 2016 peer-reviewed health policy and services journals.

Outcome measures

Reported COI disclosures including whether authors reported COI or not, form in which COI disclosures were provided, number of authors per paper who report any type of COI, number of authors per paper who report specific types and subtypes of COI.

Results

We included 200 eligible primary studies of which 132 (66%) included COI disclosure statements of authors. Of the 132 studies, 19 (14%) had at least one author reporting at least one type of COI and the most frequently reported type was individual financial COI (n=15, 11%). None of the authors reported individual intellectual COIs or personal COIs. Financial and individual COIs were reported more frequently compared with non-financial and institutional COIs.

Conclusion

A low percentage of HPSR primary studies included authors reporting COI. Non-financial or institutional COIs were the least reported types of COI.

Strengths and limitations of this study.

• This is the first study to assess the frequency and types of conflict of interest (COI) disclosed by authors of primary studies of health policy and systems research.

• The study used a rigorous methodology that included a search strategy specific to health policy and services journals and duplicate study selection and data abstraction processes.

• We used a comprehensive framework for the classification of COI.

• The study focused on reported COI, thus these statements depend on journals’ COI policy requirements, and whether authors’ disclosures are accurate or complete remains uncertain.

Background

Evidence-informed health policymaking aims to ensure that policymaking is well informed by the best available evidence.¹ Evidence from health policy and systems research (HPSR) can inform health system policy decisions including who delivers health services and where, and how these services are financed and organised.^{2 3} Furthermore. policymakers are increasingly recognising the importance of the use of research evidence in improving health, reducing health inequities and contributing to economic development.^{4 5} However, conflict of interest (COI) of researchers may influence the conduct and reporting of HPSR.

COI is defined as ‘a financial or intellectual relationship that may impact an individual’s ability to approach a scientific question with an open mind’.⁶ For instance, one study assessing the frequency and influence of financial COI on economic analyses in oncology found that the studies disclosing financial COI directly or indirectly consistently supported the sponsor’s product.⁷ Additionally, Forsyth et al found that opinion articles sceptical of the use of systematic reviews for policymaking were more likely to have industry ties than articles supportive of their use.⁸

Reporting of COI in HPSR is important given its potential influence on public policy and decision-making. We previously assessed the reporting of COI in HPSR systematic reviews.⁹ We found that 20% of those reviews did not include a COI disclosure statement, and only 15% of disclosure statements reported the existence of any COI. Furthermore, the reporting of COI in primary studies is important for both policymakers, relying on their findings for decision-making, as well as for authors of systematic reviews assessing the potential bias associated with the COI of study investigators.¹⁰ Therefore, this study aims to assess the types and frequency of COI disclosed by authors of primary studies of HPSR.

Methods

Design overview and definitions

We conducted a cross-sectional survey using standard systematic review methodology for study selection and data extraction. We defined COI disclosure as the reporting of whether a COI exists or not. We classified the types of disclosed COIs as shown in figure 1 and detailed in online supplementary appendix S1. Our classification of COIs relies on a framework informed by a literature review, the findings of recent studies assessing COIs reported by authors of clinical systematic reviews, HPSR systematic reviews and randomised controlled trials^{9 11 12} and the International Committee of Medical Journal Editors (ICMJE) COI disclosure form.¹³ We used the word ‘loogly’ to label ‘any additional statement in the COI disclosure that attempts to downplay a disclosed relationship by suggesting that it is unrelated to COI’ (eg, ‘this relationship did not influence the content of the manuscript’).¹¹

Figure 1.

Classification of conflicts of interest.

Eligibility criteria

We included articles meeting the following eligibility criteria:

• Type of study: primary studies (eg, randomised controlled trials, cohort studies, qualitative studies). We excluded systematic and literature reviews, case studies, technical reports, conference reports, proceedings, editorials and opinion pieces; type of field: HPSR; we used the taxonomy of health systems topics used to code Health Systems Evidence database of McMaster Health Forum to assess eligibility: governance, financial, delivery arrangements and implementation strategies.^{14 15} Governance arrangements cover five topics: policy authority, organisational authority, commercial authority, professional authority, and consumer and stakeholder involvement. Financial arrangements include topics on financing systems, funding organisations, remuneration providers, purchasing products and services, and incentivising consumers. Delivery arrangements cover topics related to how care is designed to meet consumers’ needs, by whom care is provided, where care is provided and with what supports is care provided. Implementation strategies comprise topics on consumer-targeted strategy, provider-targeted strategy and organisation-targeted strategy.

• Articles published in English in 2016.

Search strategy

We searched for papers published in peer-reviewed health policy and services journals. We ran the search in the Web of Science database limiting to ‘Health Policy and Services’ journal category, ‘article’ document type, English language and to the year 2016. Online supplementary appendix S2 presents the detailed search strategy.

Selection process

We drew a random sample of 200 papers from the set of citations retrieved by the search to undergo the selection process using an online random sequence generator (www.random.org/sequences). This sample of 200 primary studies is a subset of our previously published study on the reporting of funding in HPSR.¹⁶

Citations were exported to EndNote X7.5 software (Thomson Reuters, Philadelphia, PA, USA). Reviewers completed calibration exercises before starting the selection process. Two reviewers screened title and abstracts for eligibility in duplicate and independently using EndNote. We ensured that papers retrieved by our search were effectively on HPSR. We retrieved the full text of citations judged as potentially eligible by at least one of the two reviewers. The two reviewers screened the full texts in duplicate and independently. The reviewers resolved their disagreements by discussion, and consulted a third reviewer when consensus could not be reached. We used a standardised and pilot-tested full-text screening form. We recorded reasons for exclusion and summarised the selection process results in a Preferred Reporting Items for Systematic Reviews and Meta-Analyses study flow diagram.¹⁷

Data extraction process

We developed and pilot-tested a standardised data extraction form with detailed instructions (see online supplementary appendix S3). Two teams of eight reviewers completed calibration exercises and extracted data in duplicate and independently. Reviewers extracted study data using Research Electronic Data Capture tool, a secure, web-based application designed to support data capture for research studies.¹⁸ The reviewers compared results and resolved disagreements through discussion, or with the help of a third person when consensus could not be reached.

Extracted data

We extracted the following general characteristics of each article:

• Number of authors.

• Reported affiliation(s) of first and last authors (private or public academic institution, government, not-for-profit organisation, private for profit, intergovernmental).

• Country of affiliation of the first author and its classification (as per World Bank list of economies issued in September 2016).

• Health systems arrangement of the paper (governance, financial, delivery arrangements and implementation strategies).

We extracted the following characteristics of the reported COI disclosures (as defined above):

• Whether authors reported COI or not.

• Form in which COI disclosures were provided (a narrative statement, an online document, available on request).

• Number of authors per paper who report any type of COI.

• Number of authors per paper who report each specific type of COI, and when applicable, the different subtypes of COI.

• Whether the paper reports relevant characteristics of the COI (source, monetary value, duration).

• Whether individuals other than the authors provided COI disclosures (eg, editors, peer reviewers, external writers, others).

We extracted the following information on the characteristics of the journal:

• Impact factor.

• Existence of a COI disclosure policy.

Data analysis

For eligible articles, we conducted descriptive analyses, focusing on the reported COI disclosures. For continuous variables, we present summary data as medians and quartiles since the application of the Kolmogorov-Smirnov test did not demonstrate normality. We presented the results for categorical variables as frequencies and percentages, and analysed them using the χ² test or, if the expected event number proved less than 5, the Fisher’s exact test. We considered a p value of <0.05 as statistically significant. We performed the analysis using SPSS V.21.0 for Windows (SPSS).

Results

Out of the 2648 citations identified, we included 200 eligible primary studies that were published in 55 ‘Health Policy & Services’ journals. Figure 2 shows the study flow diagram.

Figure 2.

Study flow diagram. HPSR, health policy and systems research.

General characteristics of the included primary studies

Table 1 presents the general characteristics of the included primary studies. The median number of authors per study was 4. The majority of studies were conducted by authors affiliated with institutions located in high-income countries (92%) where most articles were conducted in the USA (54%) followed by UK (8%). Most articles addressed the topic of delivery arrangements (72%). Most first authors and last authors were affiliated with public academic institutions (68% and 65%, respectively).

Table 1.

General characteristics of the included primary studies (n=200)

	Overall
	n (%)
Number of authors; median (IQR)	4 (3–6)
Classification of the country of the institution to which the first author is affiliated
High income	183 (92)
USA	107 (54)
UK	16 (8)
Australia	13 (7)
Canada	9 (5)
The Netherlands	7 (4)
Other high-income countries	31 (16)
Upper middle income	10 (5)
China	3 (2)
South Africa	3 (2)
Other upper middle-income countries	4 (2)
Lower middle income	4 (2)
Kenya	1 (0.5)
Philippines	1 (0.5)
Bangladesh	1 (0.5)
Vietnam	1 (0.5)
Low income	3 (2)
Uganda	3 (2)
Affiliation of first author*
Public academic institution	135 (68)
Private academic institution	46 (23)
Government	18 (9)
Not-for-profit organisation	23 (12)
Private for profit	2 (1)
Intergovernmental	1 (1)
Affiliation of last author*
Public academic institution	129 (65)
Private academic institution	51 (26)
Government	21 (11)
Not-for-profit organisation	20 (10)
Private for profit	3 (2)
Intergovernmental	0 (0)
Type of health systems arrangement*
Delivery arrangement	143 (72)
Implementation strategies	25 (13)
Governance arrangement	23 (12)
Financial arrangement	67 (34)

*Studies may have more than one option that applies.

Characteristics of the reported COI disclosures

Of the 200 primary studies, 66% (132/200) included COI disclosure statements of authors. All but one study provided COI disclosures narratively in the main document; the single study provided them in an online form that was not accessible. None of the included studies reported COI by individuals other than the authors (eg, editors or peer reviewers).

Table 2 presents the reporting of the different types of COI in the 132 studies that included COI disclosure statements. Of these 132 studies that included COI disclosure statements, 19 (14%) had at least one author reporting at least one type of COI while 113 (86%) studies had their authors reporting that they had no COI. The most frequently reported type was individual financial COI (n=15, 11%), with the median percentage of authors reporting this type of COI being 25%. None of the authors reported individual intellectual COIs or personal COIs. Of the 132 primary studies that provided COI disclosure statements, more had at least one author reporting financial COIs compared with non-financial COIs (n=16, 12% vs n=3, 2%; p=0.04). More studies had at least one author reporting individual COIs compared with institutional COIs (n=15, 11% vs n=5, 4%; p=0.01).

Table 2.

Reporting by primary study authors of the different types of conflict of interest (COI) (n=132)

	Studies with at least one author reporting a specific type of COI* n (%)	Distribution of the percentage of authors per study reporting that type of COI† Median (IQR)
At least one type	19 (14)	25 (17–50)
Individual financial (direct benefit)	15 (11)	25 (15–50)
Individual financial (benefit through professional status)	0 (0)	N/A
Individual intellectual	0 (0)	N/A
Individual personal	0 (0)	N/A
Institutional financial	2 (2)	‡
Institutional intellectual	3 (2)	§
Institutional cultural	0 (0)	N/A
‘Other types’¶	4 (3)	30 (18–85)
Provided a ‘loogly statement’	3 (2)	**

*One study can have authors reporting more than one type of COI.

†Calculated using the number of papers with at least one author reporting the specific type of COI (ie, papers counted in the preceding column) as the denominator.

‡Authors of only two studies reported institutional financial COI, with the percentages being 20% and 100%.

§Authors of only three studies reported institutional intellectual COI, with the percentages being 20%, 25% and 33%.

¶‘Other types’ of COIs included: ‘implementing national clinical audit’ (n=1), ‘non-compensated affiliations’ (n=1), ‘attended meetings’ (n=1) and relationship with a publishing entity (n=1). We consider these as individual and non-financial types of COI.

**Authors of only three studies provided a ‘loogly statement’, with the percentages being 10%, 25% and 100%.

N/A, not applicable.

Individual financial COI

Table 3 presents the reporting of the different subtypes of individual financial COI in the 15 primary studies with at least one author reporting individual financial COI. The two most frequently reported subtypes were ‘personal fees’ (n=9, 60%) and ‘grant’ (n=6, 40%). The median percentages of authors reporting these two subtypes were 20% and 18%, respectively.

Table 3.

Reporting of primary study authors of different subtypes of individual financial conflict of interest (COI) (n=15)

	Studies with at least one author reporting the subtype of individual financial COI* n (%)	Distributions of the percentage of authors per study reporting that subtype of COI† Median (IQR)
Grant	6 (40)	18 (9–27)
Employment	2 (13)	‡
Personal fees (other than employment)	9 (60)	20 (12–38)
Non-monetary support	1 (7)	§
Study supplies/services	0 (0)	N/A
Patent(s)	0 (0)	N/A
Stocks, bonds, stock options, other securities	3 (20)	¶
’Other subtypes’	0 (0)	N/A

*One study can have authors reporting more than one type of COI.

†Calculated using the number of papers with at least one author reporting the specific type of COI (ie, papers counted in the preceding column) as the denominator.

‡Authors of only two studies reported ‘Employment’, with the percentages being 50% and 100%.

§Authors of only one study reported ‘Non-monetary support’, with the percentage being 17%.

¶Authors of only three studies reported ‘Stocks, bonds, stock options, other securities’, with the percentages being 20%, 25% and 33%.

N/A, not applicable.

Of the 15 studies with at least one author reporting individual financial COI, 14 reported the source of financial COI. Only two of these 14 studies specified the relationship of the source to the field under study; in both cases, the sources produced a product not the subject of the study but under the same field. Only one of the 15 studies reported on the timing of the conflicted relationship relative to the conduct of the study; in that case, the relationship occurred during the conduct of the study. None of the studies reported on the monetary value of the financial COI.

Characteristics of the journals

The median impact factor of the 55 journals that published the included primary studies was 1.66 (IQR=1.36–2.41). Ninety-six per cent (53/55) of the journals had a COI disclosure policy requiring authors to report their COI. Of the 68 papers that did not include a COI statement, 90% (61/68) were published in journals that did have a COI disclosure policy. The percentage of papers that included a COI statement was 68.2% in journals with a COI disclosure policy and 12.5% in journals without a COI disclosure policy (p=0.012). We provided the list of the 55 journals that published the included primary studies in online supplementary appendix S4.

Discussion

Summary of findings

In summary, 66% of 200 HPSR primary studies included COI disclosure statements of authors, with only one using an inaccessible online disclosure form. Of these studies, 14% had at least one author reporting at least one type of COI. Most frequently, the authors reported individual financial COI. Very few studies reported non-financial or institutional COIs. The two most frequently reported subtypes of individual financial COI were ‘personal fees’ and ‘grant’. None of the studies reported on the monetary value of the financial COI, or provided disclosure by individuals other than the authors such as editors or reviewers.

Strengths and limitations

This is the first study to assess the frequency and types of COI disclosed by authors of primary studies of HPSR. We have used a rigorous methodology that included a search strategy specific to health policy and services journals and duplicate study selection and data abstraction processes. We used a comprehensive framework for the classification of COI used in previous studies.^{9 11 12} Our study focused on reported COI, thus these statements depend on journals’ COI policy requirements, and whether authors’ disclosures are accurate or complete remains uncertain.

Comparison to other studies

Our findings, in relation to similar studies, demonstrate that COI disclosure statements are less frequently included in HPSR primary studies (66%) compared with HPSR systematic reviews (80%), clinical randomised controlled trials (94%) and clinical systematic reviews (97%) (figure 3).^{9 11 12} Factors that may be contributing to these differences include the less rigorous COI policies in HPSR journals compared with core clinical journals, and potentially a less strict implementation: 93% of HPSR journals (including the 55 journals that published the primary studies included in this study) have a COI disclosure policy compared with 99% for core clinical journals.^{19 20}

Figure 3.

Chart comparing the reporting of financial and non-financial COIs in different types of publications. The denominator for the reporting of the different types of COI is the number of studies that included a COI disclosure statement. COI, conflict of interest.

The percentage of authors reporting any type of COI in HPSR primary studies (14%) was comparable to that of HPSR systematic reviews (15%). However, that percentage is much lower compared with that of clinical systematic reviews (41%) and clinical trials (57%).^{9 11 12} ‘Possible explanations for this low rate of disclosure could be either an actual low prevalence of COI in this field, or an underreporting by HPSR authors of their COIs’. Indeed, an increasing number of studies are using resources such as the Open Payment database to verify the accuracy of the COI disclosures of health researchers.^21–24 They are consistently showing that researchers tend to under-report their COIs (up to 81% in one study²⁵).

Reporting of financial COI was higher than non-financial COI in HPSR primary studies. This is consistent with the findings of previous studies that focused on COI reporting in HPSR systematic reviews, clinical systematic reviews and randomised controlled trials.^{9 11 12} Although this might reflect how frequently these types of COI exist, it might also be that authors are less aware of the concept of non-financial COI, or of what exactly qualifies as a non-financial COI. Another explanation could be related to the extent of use of standard COI disclosure forms: we found that only one study used a standardised form to report COI, compared with 12% in clinical trials.¹²

Implications for practice and research

As HPSR may be used to inform policy decisions, COI of HPSR authors may bias their research output and subsequently lead to misguided public policies and decisions.^{26 27} For example, Bes-Rastrollo et al found that financial COI may bias findings of systematic reviews of the effects of sugar-sweetened beverages consumption on weight gain and obesity.²⁸ In turn, such biased conclusions might adversely influence policymaking related to regulation of sugar-sweetened beverages. Consequently, the appropriate disclosure and management of COIs are essential for the credibility and trust in HPSR, and hence might increase its uptake in policymaking. For that reason, HPSR journals strengthen their COI disclosure policies, and the implementation of existing policies. One approach to help authors better recognise and disclose their COIs would be to develop a standardised COI disclosure form similar to that of the ICMJE but more specific to HPSR. Journals publishing HPSR should also consider collecting and publishing the COIs of editors and peer reviewers. Future research should investigate the reasons behind the higher reporting of financial COI compared with non-financial COI in HPSR primary studies. Investigation of the accuracy and completeness of reporting of COI may also provide insight into the low rates of disclosed COI.