Abstract
Objective
To determine if observational studies examining the association of wholegrain foods with cardiovascular disease (CVD) with food industry sponsorship and/or authors with conflicts of interest (COI) with the food industry are more likely to have results and/or conclusions that are favourable to industry than those with no industry ties, and to determine whether studies with industry ties differ in their risk of bias compared with studies with no industry ties.
Design
Systematic review and meta-analysis of observational studies.
Data sources
We searched eight databases from 1997 to 2017 and hand searched the reference lists of included studies.
Eligibility criteria for selecting studies
Cohort and case–control studies that quantitatively examined the association of wholegrains or wholegrain foods with CVD outcomes in healthy adults or children.
Results
21 of the 22 studies had a serious or critical risk of bias. Studies with industry ties more often had favourable results compared with those with no industry ties, but the Confidence Interval (CI) was wide, Risk Ratio (RR)=1.44 (95% CI 0.88 to 2.35). The same association was found for study conclusions. We did not find a difference in effect size (magnitude of RRs) between studies with industry ties, RR=0.77 (95% CI 0.58 to 1.01) and studies with no industry ties, RR=0.85 (95% CI 0.73 to 1.00) (p=0.50) I2 0%. These results were comparable for studies that measured the magnitude using Hazard Ratios (HR); industry ties HR=0.82 (95% CI 0.76 to 0.88) versus no industry ties HR=0.86 (95% CI 0.81 to 0.91) (p=0.34) I2 0%.
Conclusions
We did not establish that the presence of food industry sponsorship or authors with a COI with the food industry was associated with results or conclusions that favour industry sponsors. The association of food industry sponsorship or authors with a COI with the food industry and favourable results or conclusions is uncertain. However, our analysis was hindered by the low level of COI disclosure in the included studies. Our findings support international reforms to improve the disclosure and management of COI in nutrition research. Without such disclosures, it will not be possible to determine if the results of nutrition research are free of food industry influences and potential biases.
PROSPERO registration number
CRD42017055841.
Keywords: nutrition, industry sponsorship, conflict of interest, bias, food industry
Strengths and limitations of this study.
This is the first systematic review and meta-analysis to evaluate the association of industry sponsorship and author conflicts of interest (COI) with the results, conclusions and risk of bias of primary nutrition studies examining the effect of wholegrain foods on cardiovascular disease outcomes.
We conducted a comprehensive search and followed explicit and well-defined inclusion and exclusion criteria for the included studies.
Although our sample was small, we searched several databases and reference lists of included studies.
We did not attempt to contact the authors of studies lacking a COI disclosure statement; thus, we may be underestimating the number of articles that had authors with COI.
Our assessment of risk of bias in the included studies was based on a tool that is under development, but changes to the tool are unlikely to affect the risk of bias ratings.
Background
Dietary guidelines are designed to promote well-being and reduce the risk of non-communicable diseases. Recent evaluations of the development of dietary guidelines have identified concerns with the methods of the systematic reviews and how evidence from these reviews is synthesised into final recommendations.1–3 Several countries including the UK, USA and Australia have dietary guidelines offering recommendations around the consumption of wholegrain foods.4–6 The guidelines conclude that there is a probable association between wholegrain consumption and a reduced risk of cardiovascular disease (CVD).4–6 These recommendations are supported by recent systematic reviews and meta-analyses of prospective cohort studies, which have found a consistent, inverse relationship between wholegrain intake and CVD risk and mortality.7–9 However, the beneficial effects of wholegrains on CVD when assessed in randomised controlled trials (RCTs) are uncertain.10
Wholegrain products can be defined in various ways, including by the species (eg, wheat, oats), components (eg, endosperm, bran, germ) and percentages (eg, 25%–100%). While some food regulators use a definition of 100% retention of wholegrain content, the epidemiological literature typically uses 25% or more retained content. In the development of the Australian Dietary Guidelines, the most common definition for wholegrain foods was those containing 25% or more of wholegrains.11
Dietary guidelines use a variety of methods to assess bias in primary research studies, but these do not assess one potential source of bias—financial conflicts of interest (COI).12 Across a variety of research areas, industry sponsorship and author COI have been found to be associated with outcomes that favour the study sponsor.13–15 Even when controlling for methodological biases, industry-sponsored studies are more likely to have results that favour the sponsor’s product than those studies with no or other sources of sponsorship.13 Industry sponsors may bias research via the questions they ask (research agenda), how they design and conduct a study, the selection of results they report and through ‘spin’ on conclusions.16–19
A systematic review of methodological studies that compared food industry-sponsored studies with those that had no or other sources of sponsorship found that food industry-sponsored studies were more likely to have favourable conclusions than non-industry-sponsored studies.20 However, there were insufficient data to quantitatively assess the association of sponsorship with study results. Only one methodological study examined the association of author COI and conclusions, and found a statistically significant association between them.21
Funding sources and author COI may be a risk of bias in studies of wholegrain consumption as these studies could test formulated or processed wholegrain products, such as breakfast cereals. Industry sponsors may gain financially from finding that these types of products have health benefits that can be used to market their products. There has been no assessment of the association of food industry sponsorship and author COI with the food industry and the statistical significance of results, effect sizes, conclusions and risk of bias of observational studies examining the cardiovascular health benefits of wholegrain consumption. The primary objective of this review is to determine whether:
Primary studies examining the association of wholegrain foods with CVD with food industry sponsorship and/or authors with COI with the food industry are more likely to have results and/or conclusions that are favourable to industry than those with no industry ties.
This review also examines whether any differences between industry and non-industry-sponsored studies could be related to their methods or interpretation of results.
The secondary objectives of this review are to determine whether:
Studies with food industry sponsorship and/or authors with COI with the food industry differ in their risk of bias compared with studies with no industry ties.
Studies with food industry sponsorship and/or authors with COI with the food industry have a higher level of discordance between study results and conclusions, with the conclusions more likely to be favourable compared with the results.
Methods
We conducted a systematic review of observational studies examining the association of wholegrain consumption with CVD.
Literature search strategy
The search was based on the Process Manual used in the development of the 2013 Australian Dietary Guidelines22 and the advice of an information specialist. We searched the following databases from January 1997 to October 2017: MEDLINE; CINAHL; PubMed; PreMEDLINE; Cochrane Library; PsycINFO; Science Direct; and ERIC. The search strategy we used for Ovid MEDLINE is shown in online supplementary file 1. We adapted this strategy for the other databases. We also hand searched the references lists of identified studies and reviews. The search also included terms for RCTs to identify relevant trials for a future systematic review.
bmjopen-2018-022912supp001.pdf (71.5KB, pdf)
Eligibility criteria
The RCTs identified in our search were included in another review currently under development. We selected observational studies for this review. This review included primary nutrition studies of cohort or case–control designs that quantitatively examined the benefits or harms of wholegrain consumption related to CVD outcomes in healthy children and/or adults.
We included studies that defined wholegrains in any way, as defined by the author of the included study. If total wholegrain consumption had been assessed in the study, we included this as our only exposure. If total wholegrain consumption as an exposure was not available, we included any type of wholegrain consumption (ie, wholegrain cereal, breakfast cereal, bread, rice, etc) as our exposure. We included studies that compared wholegrain food to other foods or compared various levels of wholegrain consumption. We included the result representing the effect of the highest level of wholegrain consumption compared with the lowest level of wholegrain consumption (eg, ‘yes’ to wholegrain consumption vs ‘no’ to wholegrain consumption, tertile 3 vs tertile 1, quartile 4 vs quartile 1, quintile 5 vs quintile 1). If our prespecified rules for selection did not uniquely identify one exposure for inclusion in the meta-analysis, we randomly selected one result.
We included studies that had a clinical outcome measure related to CVD, defined as mortality related to specific cardiovascular events, and/or cardiovascular events (eg, first myocardial infarction, total stroke, etc). If ‘cardiovascular disease mortality/death/s’ (verbatim) had been assessed, we included this as our only outcome. If not, we included any type of CVD mortality (eg, coronary heart disease mortality, stroke mortality, etc) as our outcome. If there were no mortality outcomes assessed in the study, we included any CVD event as our outcome. If a study assessed subgroups of CVD deaths and events (eg, intracerebral haemorrhages, ischaemic stroke) and also assessed them collectively (eg, cerebrovascular diseases), we took the result that had assessed them collectively. If our prespecified rules for selection did not uniquely identify one outcome for inclusion in the meta-analysis, we randomly selected one result.
We excluded conferences presentations, opinion pieces and letters to the editor. We had no language restrictions.
Types of outcome measures
Primary outcomes
We hypothesised that studies with food industry sponsorship and/or authors with a COI with the food industry would be more likely to have favourable findings than those with no industry ties. We assessed three primary outcomes.
Statistical significance of results favourable to the sponsor
Favourable results were defined as results that were favourable to the sponsor’s product(s), either indicating greater health benefits or less harm than the comparator. Specifically, for studies of health benefits of wholegrains, favourable results were defined as those that were statistically significant at the 0.05 level (two tailed). For studies of harms of wholegrains, favourable results were defined as those where harms were not statistically significant at the 0.05 level or there were a statistically significant higher number of harms in the comparator group. Otherwise, results were classified as unfavourable.
Effect size of results
Effect size was defined as the risk ratio (RR), hazard ratio (HR) or odds ratio (OR) of the association between wholegrains and a clinical outcome of CVD. We compared the magnitude of the pooled effect estimates in studies with food industry sponsorship and/or authors with a COI compared with studies with no industry ties.
Conclusions
Conclusions that suggested that the wholegrain intervention being studied was beneficial to health and/or safe were considered favourable to the study sponsor. Otherwise, the conclusions were considered unfavourable.
Secondary outcomes
We assessed two secondary outcomes.
The risk of bias of the included studies
We hypothesised that studies with industry sponsorship and/or authors with a COI with the food industry would have the same overall risk of bias as those with no industry ties.
Concordance between study results and conclusions
We hypothesised that studies with industry sponsorship and/or authors with a COI would be more likely to have discordant results and conclusions, with results not favouring the sponsor and conclusions favouring the sponsor, than those with no industry ties.
Selection of studies
Three investigators (NC, SM and JT, working in pairs) independently screened the titles and abstracts of all retrieved records for obvious exclusions. Full text of potentially eligible studies was then retrieved, and three investigators (NC, SM and JT) assessed these against our inclusion criteria. Agreement was reached by consensus.
Data collection and analysis
Three assessors (NC, SM and JT) independently extracted the following data from each included study. Discrepancies in data extraction were resolved by consensus. If agreement could not be reached, a fourth assessor (LB) adjudicated the outcome.
From each study, we extracted:
Year of publication.
Study design (cohort or case–control).
Sample size of study.
Age of participants.
Exposure duration or observation period.
How the study defined wholegrain (verbatim).
Level of wholegrain content in wholegrain foods.
Disclosure of funding source (no disclosure, yes and there is a sponsor, the authors state they received no funding for their work).
Name of the funders of the study (verbatim).
Role of the funders (role of the sponsor not mentioned, sponsor not involved in study design and analyses, sponsor involved, N/A).
Disclosure of author COI (no disclosure, yes, the authors state they had no COI to declare).
Authors COI statement (verbatim).
Outcomes assessed in the study (any CVD death and/or event).
The numerical results of the study (eg, OR, HR).
We stored all extracted data from the included studies in REDcap, a secure web-based application for the collection and management of data.23
Classification of industry sponsorship and author COI
Sponsorship was categorised as (1) industry or (2) non-industry. We defined industry-sponsored studies as those declaring any sponsorship from the food industry, including if the study received ‘mixed funding’ from the food industry, non-profit organisations or other industries (ie, pharmaceutical). Any study with an author with any disclosed financial tie to the food industry was classified as having a COI. Author COI were categorised as (1) presence of a COI with the food industry or (2) no COI. Any studies that did not contain an author COI disclosure statement were classified as no COI. We contacted the authors of one paper24 for clarification on their disclosure of funding source.
Assessment of risk of bias in included studies
We used an adapted version of the Cochrane Collaboration’s ‘Risk of Bias in Non-Randomised Studies-of Interventions’25 tool to measure the risk of bias of included observational studies. The tool assesses bias across seven domains. Each domain is assessed at a low, moderate, serious or critical risk of bias or no information. The domain rating with the highest risk of bias determines the overall risk of bias rating for the study. For example, if a study is rated as being at a serious risk of bias in one domain, the overall risk of bias rating is ‘serious.’
Analysis
We report frequencies and percentages of study characteristics across all studies and, separately, by funding source. We visually depict the overall risk of bias rating and the ratings for each domain by study.
We calculated RRs or HRs (and 95% CIs) to quantify the association between food industry sponsorship and/or authors with COI with the food industry and favourable results, favourable conclusions and the overall study risk of bias rating. For the risk of bias rating analysis, we dichotomised the overall risk of bias ratings as low (low or moderate) or high (serious or critical). We had planned to calculate an RR for level of concordance; however since in all studies there was concordance between the results and conclusions, we did not undertake this analysis.
We used meta-analysis to examine whether food industry sponsorship and/or authors with COI with the food industry modified the magnitude of association between wholegrains and CVD outcomes. Specifically, we undertook a subgroup analysis within a random effects meta-analysis model that compared the pooled associations across subgroups defined by industry sponsorship. The associations were pooled using inverse variance weighting and DerSimonian and Laird’s method of moments estimator was used to estimate between study heterogeneity. Separate meta-analyses were fitted for studies that had measured the association using HRs and those that had used either RRs or ORs. Given cardiovascular events were rare, the OR approximated risk ratios. We quantified heterogeneity for subgroup differences using the I2 statistic26 and tested for heterogeneity using the χ2 test. Review Manager V.5.3 was used to analyse the data.27
Protocol registration
The protocol is published in PROSPERO28 ID CRD42017055841 (see online supplementary file 2).
bmjopen-2018-022912supp002.pdf (100.4KB, pdf)
Patient involvement
No patients were involved in the completion of this review.
Results
Search results
We identified 6818 references for screening, from which 22 studies met the inclusion criteria (figure 1). See online supplementary file 3 for ‘List of excluded Studies’ and reasons for exclusion.
Figure 1.
Study flow diagram.
bmjopen-2018-022912supp003.pdf (137.6KB, pdf)
Characteristics of included studies
All studies were published between 1998 and 2015. Three of the studies were case–control and 19 were cohort design. All studies contained a sponsorship disclosure. Five studies disclosed food industry sponsorship, but only one of these had a statement describing the role of the sponsor. Five studies contained an author with a COI with the food industry. Ten studies did not contain an author COI disclosure statement. Nine studies contained either food industry sponsorship or had an author with a COI.
A greater proportion of industry-sponsored studies (67%) than non-industry sponsored studies (31%) used a definition of wholegrain as greater than 25%, and most of these examined breakfast cereals (table 1). Industry sponsored studies were also more likely than non-industry studies to focus on a specific food (44%) than total wholegrain intake (23%) (table 1). Industry-sponsored studies were less likely (56%) to have a serious or critical risk of bias in classification of exposures than non-industry-sponsored studies (85%). Other characteristics were similarly distributed across industry-sponsored versus non-industry-sponsored studies. Details of each individual study are in online supplementary file 4.
Table 1.
Characteristics of the included studies by sponsorship and author COI funding source, n (%)*
| Characteristic | Category | Total n=22 |
Industry/COI n=9 | Non-industry/no COI n=13 |
| Sex | Male | 4 (18) | 4 (44) | 0 (0) |
| Female | 6 (27) | 1 (11) | 5 (38) | |
| Both | 12 (55) | 4 (44) | 8 (62) | |
| Sample size, quartiles | <5000 | 6 (27) | 2 (22) | 4 (31) |
| 5000–50 000 | 9 (41) | 4 (44) | 5 (38) | |
| >50 000 | 7 (32) | 3 (33) | 4 (31) | |
| Length of follow-up | N/A† | 3 (14) | 1 (11) | 2 (15) |
| <10 years | 1 (5) | 1 (11) | 0 (0) | |
| 10–15 years | 12 (55) | 4 (44) | 8 (62) | |
| >15 | 6 (27) | 3 (33) | 3 (23) | |
| Percent wholegrain | Not defined | 12 (55) | 3 (33) | 9 (69) |
| >25%‡ | 10 (45) | 6 (67) | 4 (31) | |
| Type of wholegrain | Only wholegrain intake | 15 (68) | 5 (56) | 10 (77) |
| Individual wholegrain food§ | 7 (32) | 4 (44) | 3 (23) | |
| Primary outcome | Favourable to wholegrains | 16 (73) | 8 (89) | 8 (62) |
| Unfavourable to Wholegrains | 6 (27) | 1 (11) | 5 (38) | |
| Conclusions | Favourable to wholegrains | 16 (73) | 8 (89) | 8 (62) |
| Unfavourable to wholegrains | 6 (27) | 1 (11) | 5 (38) | |
| Risk of bias assessment | ||||
| Serious/critical bias due to confounding | 21 (95) | 9 (100) | 12 (92) | |
| Serious/critical bias in selection of participants into the study | 3 (14) | 1 (11) | 2 (15) | |
| Serious/critical bias in classification of exposures | 16 (73) | 5 (56) | 11 (85) | |
| Serious/critical bias due to deviations from exposures | 7 (32) | 3 (33) | 4 (31) | |
| Serious/critical bias due to missing data | 0 (0) | 0 (0) | 0 (0) | |
| Serious/critical bias in measurement of outcomes | 1 (5) | 1 (11) | 0 (0) | |
| Serious/critical bias in selection of reported results | 0 (0) | 0 (0) | 0 (0) | |
| Serious/critical overall risk of bias | 21 (95) | 9 (100) | 12 (92) |
*Percentages may not add to 100 due to rounding.
†Case–control studies were not followed up.
‡Any wholegrain foods defined as >25%.
§Individual foods included wholegrain cereal, breakfast cereal, bread and brown rice.
COI, conflict of interest.
bmjopen-2018-022912supp004.pdf (549.8KB, pdf)
Risk of bias in included studies
One study29 was assessed as having an overall moderate risk of bias, four as having a serious risk of bias and seventeen as having a critical risk of bias (figure 2). The majority of studies had a critical risk of bias in the confounding domain. For example, a confounder was fruit and vegetable intake. If this was not appropriately controlled for when assessing the effect of wholegrain intake on a CVD outcome, the study was rated as having a risk of bias for confounding. All but one study was assessed at a low risk of bias on the outcome measurement domain. For all domains, except classification of exposure, the risk of bias ratings were similarly distributed across industry-sponsored versus non-industry-sponsored studies (table 1).
Figure 2.
Risk of bias of included studies.
Favourable results: statistical significance: industry ties versus no industry ties
The risk of reporting favourable outcomes was 44% higher in studies with industry sponsorship and/or authors with a COI with the food industry RR=1.44 (95% CI 0.88 to 2.35). However, the CI was wide and included differences in risks that were unimportant or operating in the opposite direction as plausible estimates. When we compared only industry-sponsored (n=5) and non-industry-sponsored studies (n=17), the risk was smaller RR=1.13 (95% CI 0.66 to 1.94).
Favourable results: effect size: industry ties versus no industry ties
There was no difference in the magnitude of RRs (measuring the association between wholegrains and CVD outcomes) between studies with industry sponsorship and/or authors with a COI with the food industry RR=0.77 (95% CI 0.58 to 1.01) and those studies with no industry sponsorship or author COI RR=0.85 (95% CI 0.73 to 1.00) (subgroup test p=0.50, I2=0%) (figure 3). For studies that had measured the association using HRs there was also no difference found in the magnitude of HRs between studies with industry sponsorship and/or authors with a COI with the food industry HR=0.82 (95% CI 0.76 to 0.88) and studies with no industry sponsorship or author COI HR=0.86 (95% CI 0.81 to 0.91) (subgroup test p=0.34, I2=0%) (figure 4).
Figure 3.
Effect size: industry sponsored and/OR author conflict of interest (COI) versus non-industry-sponsored and no author COI studies, risk ratio.
Figure 4.
Effect size: industry-sponsored and/OR author conflict of interest (COI) versus non-industry-sponsored and no author COI studies, hazard ratio.
Our analysis comparing studies with industry sponsorship RR 0.63 (95% CI 0.28 to 1.39) and those with no industry sponsorship RR 0.85 (95% CI 0.74 to 0.97) (subgroup test p=0.46, I2=0%) showed no important difference in the magnitude of RRs. This was again comparable between industry-sponsored HR 0.82 (95% CI 0.77 to 0.87) and non-industry-sponsored studies HR 0.85 (95% CI 0.81 to 0.90) (subgroup test p=0.29), I2=12.2%) that measured the association using HRs.
Favourable conclusions: industry ties versus no industry ties
As there was concordance between the results and conclusions of every included study, the same associations were found for conclusions as for the statistical significance of results. Studies with industry sponsorship and/or authors with a COI with the food industry were more likely to have favourable conclusions compared with those with no industry sponsorship or author COI RR=1.44 (95% CI 0.88 to 2.35); however, the CI was wide. When studies were compared only by industry sponsorship, the risk was again smaller RR=1.13 (95% CI 0.66 to 1.94).
Risk of bias assessment by industry ties
Studies with industry sponsorship and/or authors with a COI with the food industry were less likely (0/9) to have an overall low risk of bias rating compared with those studies with no industry sponsorship or author COI (1/13), RR=0.47 (95% CI 0.02 to 10.32); however, there was large uncertainty in the association.
Discussion
Observational studies examining the effect of wholegrain consumption on CVD outcomes that were sponsored by the food industry and/or had authors with a COI with the food industry more often had favourable results than research not tied to the food industry. However, this finding was inconclusive with respect to the association between industry ties and favourable results, as the relative risk could be as high as 2.35 or as low as 0.88. We found no evidence of a difference in the magnitude of effect between industry-sponsored and non-industry-sponsored studies. It is difficult to detect differences in effect size by sponsorship as many study design features, such as dose and duration of exposures, and specific CVD outcomes, vary across studies and may influence the effect size. In previous assessments of drug studies that have demonstrated that industry-funded studies are more likely to have results that favour the study sponsors, there was no statistically significant difference found in effect sizes between industry-sponsored and non-industry-sponsored studies.13
Although all the included studies had a sponsorship disclosure, almost half were missing disclosures about author COI. Non-disclosed COIs in nutrition research are a concern.30 Larger samples of industry-funded studies and studies with disclosed author COI could make it possible to establish the association of sponsorship with research outcomes.
Studies that were sponsored by the food industry and/or had authors with a COI with the food industry more often had favourable conclusions than studies with no industry ties, although there was uncertainty in this relationship. There was absence of spin in the included studies as all the results agreed with the conclusions.
The overall risk of bias in every study, other than one non-industry sponsored study,29 was classified as high (meaning either serious or critical). The overall risk of bias rating was based on the domain with the highest risk of bias rating within each study, and most of the studies had a risk of bias related to confounding. Across each domain, we found little difference in the risk of bias between industry-sponsored and non-industry-sponsored studies.
Strengths and limitations of this review
Our review was registered in PROSPERO.28 We conducted a comprehensive search and followed explicit and well-defined inclusion and exclusion criteria for the included studies. Although our sample was small, we searched several databases and reference lists of included studies. Authors of the studies for which we required clarification on funding source were also contacted, but we did not attempt to contact the authors of studies lacking a COI disclosure statement. Thus, we may be underestimating the number of articles that had authors with COI. Our assessment of risk of bias in the included studies was based on a tool that is under development, but changes to the tool are unlikely to affect the risk of bias ratings.25
Agreements and disagreements with other studies or reviews
The relationship that we identified between food industry sponsorship and authors with a COI and favourable study outcomes towards the study sponsor has been previously demonstrated in an assessment of a broad range of nutrition research.20 Only one study has reported an association of food industry funding with effect sizes.31 Of studies examining the association between soft drink consumption and adverse health outcomes, food industry-sponsored studies reported significantly smaller effects than non-food industry-sponsored studies. Compared with our study, this study examined studies with a homogeneous population of industry funders, sugar-sweetened beverage companies, which may have a more consistent influence on study outcomes than the diverse pool of food industry sponsors in our study.
There was also no difference in the level of risk of bias between industry-sponsored and non-industry-sponsored studies. This is consistent with previous assessments of pharmaceutical, tobacco and nutrition research that has shown industry-sponsored studies are of equal or better quality than non–industry-sponsored studies.13 20 32–34
Implications for clinicians, policy makers and future research
The recent critiques to reform the methods used in the development of dietary guidelines have proposed steps to improve the transparency of how evidence is evaluated and synthesised into recommendations.1 2 However, until the influence of industry sponsorship in primary nutrition studies has been further explored and measured with larger samples of industry-sponsored studies, or studies that have author disclosure statements, this bias may still be unaccounted for in dietary guidelines. Although there was uncertainty around the differences in the results and conclusions that we observed between industry and non-industry studies, the differences are unlikely to be explained by methodological risks of bias in these studies.
There are ways that study sponsorship can influence outcomes other than through the design of research. Bias may also be introduced in the way industry-sponsored studies code events and analyse data,35 36 through the selective reporting of study outcomes and through publication bias.37 It has been demonstrated in other areas of medical research that there is a greater propensity to publish studies with statistically significant results.38 Therefore, selective publication of study results or studies in their entirety may limit the availability of all relevant nutrition data and can skew results that are used in dietary guideline development.39 Publication bias could be minimised with the introduction of study registries for nutrition research, as has been established in pharmaceutical research.40 The association of food industry sponsorship with the reporting of nutrition research still needs to be assessed.
Almost half of the studies included in this review had authors that did not disclose if they had a COI with the food industry or not. Compliance with COI disclosure policies is now well documented across many domains of research.41–46 Recent examinations of the levels of disclosure in research assessing the effects of artificially sweetened beverages on weight outcomes found similarly poor disclosure rates.32 Several solutions have been proposed to increase transparency and disclosure rates, including the use of different databases and additional resources to identify conflicted authors, and the introduction of mandatory disclosure requirements in all journals, with the use of penalties for those who do not adhere to the stated policies.20 32
Conclusion
We did not establish that the presence of food industry sponsorship or authors with a COI with the food industry was associated with results or conclusions that favour industry sponsors. The association of food industry sponsorship or authors with a COI with the food industry and favourable results or conclusions is uncertain. However, our analysis was hindered by the low level of COI disclosure in the included studies. This research further strengthens calls for stricter policies relating to the disclosure and management of COI in nutrition research. Without such disclosures, it will not be possible to determine if the results of nutrition research are free of food industry influences and potential biases.
Supplementary Material
Acknowledgments
We thank Matthew Page, Monash University for answering specific statistical questions.
Footnotes
Contributors: NC, AF, SM, MA-F and LB designed and wrote the review protocol. NC wrote the search strategy and undertook the literature search. NC, SM and JT conducted the title and abstract screening and full article screening for final study inclusion. NC, SM and JT conducted data collection and cleaning, LB supervised. NC and JM undertook all data analysis. LB advised on methods, statistical analyses and interpretation of findings. All authors contributed to the final manuscript. NC and LB are guarantors.
Funding: This work was funded by National Health and Medical Research Council Grant #APP1139997. Nicholas Chartres is a recipient of the James Millner PhD Scholarship in Pharmacy from the University of Sydney. Sally McDonald is a scholarship recipient (Charles Perkins Centre summer scholarship) from the University of Sydney. Jessica Turton is a scholarship recipient (Charles Perkins Centre summer scholarship) from the University of Sydney.
Competing interests: None declared.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data sharing statement: Available from The University of Sydney data repository. DOI to be determined.
Patient consent for publication: Not required.
References
- 1. National Academies of Sciences, Engineering, and Medicine. Redesigning the process for establishing the dietary guidelines for Americans. 2017. https://www.nap.edu/catalog/24883/redesigning-the-process-for-establishing-the-dietary-guidelines-for-americans (accessed Nov 2017). [PubMed]
- 2. National Academies of Sciences, Engineering, and Medicine. Optimizing the process for establishing the Dietary Guidelines for Americans: the selection process. 2017. https://www.nap.edu/catalog/24637/optimizing-the-process-for-establishing-the-dietary-guidelines-for-americans (accessed Nov 2017). [PubMed]
- 3. Bero L. Developing reliable dietary guidelines. BMJ 2017;359:j4845 10.1136/bmj.j4845 [DOI] [PubMed] [Google Scholar]
- 4. U.S. Department of Health and Human Services and U.S. Department of Agriculture. 2015 – 2020 Dietary Guidelines for Americans. 8th edn, 2015. Available: https://health.gov/dietaryguidelines/2015/guidelines/ (accessed Dec 2017).
- 5. National Health and Medical Research Council. Department of Health and Ageing. Australian Dietary Guidelines. Canberra, Commonwealth of Australia: NHMRC, 2013. [Google Scholar]
- 6. Public Health England. The Eatwell Guide 2017, 2017. https://www.gov.uk/government/publications/the-eatwell-guide
- 7. Aune D, Keum N, Giovannucci E, et al. . Whole grain consumption and risk of cardiovascular disease, cancer, and all cause and cause specific mortality: systematic review and dose-response meta-analysis of prospective studies. BMJ 2016;353:i2716 10.1136/bmj.i2716 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Benisi-Kohansal S, Saneei P, Salehi-Marzijarani M, et al. . Whole-Grain Intake and Mortality from All Causes, Cardiovascular Disease, and Cancer: A Systematic Review and Dose-Response Meta-Analysis of Prospective Cohort Studies. Adv Nutr 2016;7:1052–65. 10.3945/an.115.011635 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Zhang B, Zhao Q, Guo W, et al. . Association of whole grain intake with all-cause, cardiovascular, and cancer mortality: a systematic review and dose-response meta-analysis from prospective cohort studies. Eur J Clin Nutr 2018;72 10.1038/ejcn.2017.149 [DOI] [PubMed] [Google Scholar]
- 10. Kelly SA, Hartley L, Loveman E, et al. . Whole grain cereals for the primary or secondary prevention of cardiovascular disease. Cochrane Database Syst Rev 2017;8:CD005051 10.1002/14651858.CD005051.pub3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Jacobs DR, Meyer KA, Kushi LH, et al. . Whole-grain intake may reduce the risk of ischemic heart disease death in postmenopausal women: the Iowa Women’s Health Study. Am J Clin Nutr 1998;68:248–57. 10.1093/ajcn/68.2.248 [DOI] [PubMed] [Google Scholar]
- 12. Blake P, Durão S, Naude CE, et al. . An analysis of methods used to synthesize evidence and grade recommendations in food-based dietary guidelines. Nutr Rev 2018;76:290–300. 10.1093/nutrit/nux074 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Lundh A, Lexchin J, Mintzes B, et al. . Industry sponsorship and research outcome. Cochrane Database Syst Rev 2017;2:Mr000033 10.1002/14651858.MR000033.pub3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Huss A, Egger M, Egger M, et al. . Source of funding and results of studies of health effects of mobile phone use: systematic review of experimental studies. Cien Saude Colet 2008;13:1005–12. 10.1590/S1413-81232008000300022 [DOI] [PubMed] [Google Scholar]
- 15. Barnes DE, Bero LA. Why review articles on the health effects of passive smoking reach different conclusions. JAMA 1998;279:1566–70. 10.1001/jama.279.19.1566 [DOI] [PubMed] [Google Scholar]
- 16. Lexchin J. Those who have the gold make the evidence: how the pharmaceutical industry biases the outcomes of clinical trials of medications. Sci Eng Ethics 2012;18:247–61. 10.1007/s11948-011-9265-3 [DOI] [PubMed] [Google Scholar]
- 17. Sismondo S. How pharmaceutical industry funding affects trial outcomes: causal structures and responses. Soc Sci Med 2008;66:1909–14. 10.1016/j.socscimed.2008.01.010 [DOI] [PubMed] [Google Scholar]
- 18. Boutron I, Dutton S, Ravaud P, et al. . Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA 2010;303:2058–64. 10.1001/jama.2010.651 [DOI] [PubMed] [Google Scholar]
- 19. Odierna DH, Forsyth SR, White J, et al. . The cycle of bias in health research: a framework and toolbox for critical appraisal training. Account Res 2013;20:127–41. 10.1080/08989621.2013.768931 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Chartres N, Fabbri A, Bero LA. Association of industry sponsorship with outcomes of nutrition studies: a systematic review and meta-analysis. JAMA Intern Med 2016;176:1769–77. 10.1001/jamainternmed.2016.6721 [DOI] [PubMed] [Google Scholar]
- 21. Diels J, Cunha M, Manaia C, et al. . Association of financial or professional conflict of interest to research outcomes on health risks or nutritional assessment studies of genetically modified products. Food Policy 2011;36:197–203. 10.1016/j.foodpol.2010.11.016 [DOI] [Google Scholar]
- 22. Dietitians Association of Australia. A review of the evidence to address targeted questions to inform the revision of the Australian dietary guidelines 2009: Process Manual. 2011.
- 23. Harris PA, Taylor R, Thielke R, et al. . Research electronic data capture (REDCap)- A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377–81. 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Holmberg S, Thelin A, Stiernström EL. Food choices and coronary heart disease: a population based cohort study of rural Swedish men with 12 years of follow-up. Int J Environ Res Public Health 2009;6:2626–38. 10.3390/ijerph6102626 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Sterne JA, Hernán MA, Reeves BC, et al. . ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016;355:i4919 10.1136/bmj.i4919 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Higgins JP, Thompson SG, Deeks JJ, et al. . Measuring inconsistency in meta-analyses. BMJ 2003;327:557–60. 10.1136/bmj.327.7414.557 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Version 5.3 [program]. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014. [Google Scholar]
- 28. National Institute for Health Research. International Prospective Register for Sytematic Reviews [Internet]. 2015. http://www.crd.york.ac.uk/PROSPERO/ (accessed 11 Mar 2015).
- 29. Wu H, Flint AJ, Qi Q, et al. . Association between dietary whole grain intake and risk of mortality: two large prospective studies in US men and women. JAMA Intern Med 2015;175:373–84. 10.1001/jamainternmed.2014.6283 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Nestle M. Corporate Funding of Food and Nutrition Research: Science or Marketing? JAMA Intern Med 2016;176:13–14. 10.1001/jamainternmed.2015.6667 [DOI] [PubMed] [Google Scholar]
- 31. Vartanian LR, Schwartz MB, Brownell KD. Effects of soft drink consumption on nutrition and health: a systematic review and meta-analysis. Am J Public Health 2007;97:667–75. 10.2105/AJPH.2005.083782 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Mandrioli D, Kearns CE, Bero LA. Relationship between Research Outcomes and Risk of Bias, Study Sponsorship, and Author Financial Conflicts of Interest in Reviews of the Effects of Artificially Sweetened Beverages on Weight Outcomes: A Systematic Review of Reviews. PLoS One 2016;11:e0162198 10.1371/journal.pone.0162198 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Barnes DE, Bero LA. Industry-funded research and conflict of interest: an analysis of research sponsored by the tobacco industry through the Center for Indoor Air Research. J Health Polit Policy Law 1996;21:515–42. 10.1215/03616878-21-3-515 [DOI] [PubMed] [Google Scholar]
- 34. Cho MK, Bero LA. The quality of drug studies published in symposium proceedings. Ann Intern Med 1996;124:485–9. 10.7326/0003-4819-124-5-199603010-00004 [DOI] [PubMed] [Google Scholar]
- 35. Psaty BM, Prentice RL. Minimizing bias in randomized trials: the importance of blinding. JAMA 2010;304:793–4. 10.1001/jama.2010.1161 [DOI] [PubMed] [Google Scholar]
- 36. Psaty BM, Kronmal RA. Reporting mortality findings in trials of rofecoxib for Alzheimer disease or cognitive impairment: a case study based on documents from rofecoxib litigation. JAMA 2008;299:1813–7. 10.1001/jama.299.15.1813 [DOI] [PubMed] [Google Scholar]
- 37. Hart B, Lundh A, Bero L. Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. BMJ 2012;344:d7202 10.1136/bmj.d7202 [DOI] [PubMed] [Google Scholar]
- 38. Hopewell S, Loudon K, Clarke MJ, et al. . Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev 2009;46:Mr000006 10.1002/14651858.MR000006.pub3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Misakian AL, Bero LA. Publication bias and research on passive smoking: comparison of published and unpublished studies. JAMA 1998;280:250–3. [DOI] [PubMed] [Google Scholar]
- 40. Dickersin K, Rennie D. The evolution of trial registries and their use to assess the clinical trial enterprise. JAMA 2012;307:1861–4. 10.1001/jama.2012.4230 [DOI] [PubMed] [Google Scholar]
- 41. Forsyth SR, Odierna DH, Krauth D, et al. . Conflicts of interest and critiques of the use of systematic reviews in policymaking: an analysis of opinion articles. Syst Rev 2014;3:122 10.1186/2046-4053-3-122 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Ruff K. Scientific journals and conflict of interest disclosure: what progress has been made? Environ Health 2015;14:45 10.1186/s12940-015-0035-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Resnik DB. Science and money: problems and solutions. J Microbiol Biol Educ 2014;15:159–61. 10.1128/jmbe.v15i2.792 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44. Roseman M, Milette K, Bero LA, et al. . Reporting of conflicts of interest in meta-analyses of trials of pharmacological treatments. JAMA 2011;305:1008–17. 10.1001/jama.2011.257 [DOI] [PubMed] [Google Scholar]
- 45. Bero L. Addressing bias and conflict of interest among biomedical researchers. JAMA 2017;317:1723–4. 10.1001/jama.2017.3854 [DOI] [PubMed] [Google Scholar]
- 46. Grundy Q, Dunn AG, Bourgeois FT, et al. . Prevalence of disclosed conflicts of interest in biomedical research and associations with journal impact factors and altmetric scores. JAMA 2018;319:408–9. 10.1001/jama.2017.20738 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2018-022912supp001.pdf (71.5KB, pdf)
bmjopen-2018-022912supp002.pdf (100.4KB, pdf)
bmjopen-2018-022912supp003.pdf (137.6KB, pdf)
bmjopen-2018-022912supp004.pdf (549.8KB, pdf)