Determinants of evidence use in public health policy making: Results from a study across six EU countries

Highlights

• •Media attitude towards underpinning policy with evidence influences policy decision makers.
• •Individual skills, attitudes, values of policy makers impact the extent evidence use.
• •A solid research infrastructure is facilitating but not sufficient for evidence use.
• •Factors that impact evidence use in policy making differ by country and policy context.
• •Interventions connecting policy makers and researchers in the policy context seem most promising.

Abstract

The knowledge-practice gap in public health is widely known. The importance of using different types of evidence for the development of effective health promotion has also been emphasized.

Nevertheless, in practice, intervention decisions are often based on perceived short-term opportunities, lacking the most effective approaches, thus limiting the impact of health promotion strategies. This article focuses on facilitators and barriers in the use of evidence in developing health enhancing physical activity policies.

Data was collected in 2012 by interviewing 86 key stakeholders from six EU countries (FI, DK, UK, NL, IT, RO) using a common topic guide. Content analysis and concept mapping was used to construct a map of facilitators and barriers.

Barriers and facilitators experienced by most stakeholders and policy context in each country are analysed. A lack of locally useful and concrete evidence, evidence on costs, and a lack of joint understanding were specific hindrances. Also users’ characteristics and the role media play were identified as factors of influence.

Attention for individual and social factors within the policy context might provide the key to enhance more sustainable evidence use. Developing and evaluating tailored approaches impacting on networking, personal relationships, collaboration and evidence coproduction is recommended.

1. Introduction

It is well acknowledged that the use of robust evidence to inform public health policy is likely to ensure the greatest and most equitable population health gains [1], [2]. Increasing focus on evidence-informed public health in which next to different sources of research evidence contextual factors also play a substantial role in the decision making process [3] and has numerous direct and indirect benefits. Among these are access to more and higher quality information on what works, a higher likelihood of successful programs and policies being implemented, greater workforce productivity, and more efficient use of resources [4]. Nevertheless, in practice, intervention decisions are often based on perceived short-term opportunities, lacking systematic planning and review of the best evidence regarding effective approaches [4] thus resulting in slow uptake of research evidence in practice. It has been estimated that it takes an average of 17 years for 14% of research to translate into practice [5]. More recent results show that even in clinical practice which is supposed to be more evidence oriented, the uptake of evidence has not changed substantially since then, indicating that the gap between evidence and practice has not diminished substantially [6]. Generally in public health policy making the use of research evidence is less than anticipated when considering the extensive availability of research evidence. While research evidence on effective health enhancing physical activity (HEPA) policies and interventions is available, it appears not to be optimally used to inform health related policy development [7], [8], [9], [10], [11]. A multitude of factors that impede (or facilitate) evidence-informed policy making exists resulting in below optimal health outcomes when implemented. Literature shows that specific contexts and traditions, political priorities, individual beliefs and preferences, social values, and available resources all play a major role [12], [13]. Among these factors three main categories can be distinguished. Firstly, easy access to relevant and useful research [14] also entailing timely access to good quality and relevant research evidence [15]. Secondly, frequent opportunities to interact with researchers [7] including collaboration and networking with policymakers [15]. Thirdly, working in research receptive organizations [16], [17] facilitates evidence-informed policy making.

The updated systematic review on barriers and facilitators of evidence use in policy making by Oliver et al. in 2014 [15] concluded that over the past 10 years these have basically remained the same and that it is difficult to find new perspectives. Some recent research however points in the direction of personal relationships and policy makers’ networks as well as differences in contextual factors to be of utmost importance in relation to improving the uptake of evidence in policy. Policy makers appear to have a need for and also use a much wider range of information sources than research evidence and they access most of these through personal contacts [18]. In addition, policy makers’ relationships within networks and characteristics of the organizational context such as the much neglected role of managers in policy decision making appears to be of great influence in evidence use [18], [19], [20], [21], [22]. Also studies with empirical data on interactions between stakeholders in policymaking report that the use of evidence in the policy process was difficult to trace or that the process itself appears to be rather closed [23], [24], [25], [26], [27].

Furthermore the literature shows that policymakers with respect to use of evidence need to pay attention to larger entities and multi-dimensional factors such as communities, municipalities, resources, politics and other factors as compared to for instance clinicians [21], [28]. This makes use of evidence by policymakers much more complicated and may be the reason that the extent of evidence use by them is lower in comparison to clinicians who focus on one specific issue only, i.e. the physical condition of the individual patient.

Despite several decades of work on evidence informed policy, the goals to improve evidence uptake and promote greater use of evidence within policy making are still elusive. Recent literature warrants more research on evidence use by policy makers through interaction and personal contacts, relationships within networks and the complexity and varied context of policy making.

In 2011 the European Commission (EC) funded the Research into Policy to enhance Physical Activity (REPOPA) project. One of its aims was to study the extent to which EU member states use research evidence and other kinds of evidence in HEPA policies and what promotes or hinders the uptake of research evidence in the policy-making process of HEPA policies [25]. The general aim of the project was to facilitate the integration of research evidence to stimulate more evidence-informed physical activity policies. The aim, design, methods and preliminary baseline results of the overall REPOPA—(www.repopa.eu) project are described by Aro et al. [25]. Preliminary results show that supportive institutional resources, access to applicable context-relevant research evidence, media attention, good personal relationships and networks, joint language and collaboration between researchers and policy makers were found to facilitate the use of research evidence. Barriers identified were related to non-supportive institutional management, lack of easy access to best available evidence, limited contacts between administrative personnel, experts and researchers [25].

The aim of this article is to further explore barriers and facilitators in the use of research and other evidence in developing HEPA policies from six EU countries using semi-structured interviews conducted with key stakeholders as part of the REPOPA project. More specifically it focuses on aspects that (local, regional or national) policymakers and other stakeholders in different European contexts experience as most influencing in the uptake of evidence in real-life policymaking processes.

2. Methods and design

In the REPOPA project 21 HEPA policies were identified across six European countries (Finland, Italy, Romania, UK, The Netherlands and Denmark) (for details see Ref. [24]). They varied significantly across countries-. The policies were almost always part of a broader (public) health care or sports policy. In each country a national and a regional/local level policy was selected where available (not all countries had policies at both level). Semi-structured interviews were undertaken using a standard topic guide across all six participating countries. The interviews focused on perceptions and experiences on how research and other types of evidence, comprising a broad variety of information sources were used by policymakers [18], [29] and what factors facilitated or hindered their use in the policy making process. The interviews were conducted by researchers from the respective countries in their native language. Each interview took about 1.5-h was tape-recorded and transcribed (tape-recording was only done when participants provided consent), otherwise hand-written notes were made [26].

By purposeful sampling a total of 86 stakeholders who were directly involved in the policy making process of the selected policies and who could inform about the use of research or other evidence in this specific policy making process were identified and included. The stakeholders included (national, regional or local) policy makers, researchers, public sector officers or other influential stakeholders. All interviewees were contacted by email or phone by the research team in the country with background information on the project and consent forms in the local language [26]. On the (rare) occasion of an identified stakeholder was not available for interview, a deputy was contacted. The interviews were specifically designed to collect in-depth information on stakeholders experience in terms of facilitators and barriers in the uptake of evidence in the everyday world of policy making.

The data collected were analysed by each country team using a common guideline for qualitative content analysis (analysis were carried out manually or using software packages for qualitative research such as NVivo, MaxQdata). Coding was done by two researchers independently from each other in the country teams [26]. Each country produced a report presenting its findings which were integrated into a single (internal) project report in English.

From the country reports, lists of facilitators and barriers identified for the use of research evidence were grouped and categorized using an online concept map tool [30]. The concept map tool helped find associations and pull together similar items and separate differing items mentioned in all the interviews. By using the concept map tool the qualitative data from interviews formed a pattern of facilitators and barriers for the use of research evidence in HEPA policy making. Since the focus was not on finding differences in views of different stakeholder groups, overall views were taken together in the categorization and no weighting of data was carried out.

REPOPA developed an Ethics Road Map and Ethics Guidance Document to coordinate varying national ethics clearance procedures in partner countries. Ethics clearance was done in each country according to country-specific regulations and procedures (for details see Ref. [31]: Edwards et al.); however, irrespective of each country’s requirements, informed consent was obtained from all participants. Ethical Committees for each country included were as follows: Ethics Committee of the Region of South Denmark and the National Data protection Agency (DK); Central Committee on Research Involving Human Subjects (NL); Research Ethical Committee of the National Health and Welfare Institute (FI); Ethics Committee of the University of Babes-Bolyai (RO); National Research Council Research Ethics and Bioethics Advisory Committee (IT); Health and Social Sciences Review Board of University of Ottawa (CA); Ethics approval by the Research Councils UK (UK).

The ethical clearance papers of all countries were approved by the EC before start of the project. The EC had oversight of the ethics of the overall project.

3. Results

Fig. 1 presents an overview of the facilitators and barriers for the use of evidence in the policy development process highlighted by the interviewees. Although facilitators and barriers were to some extent in itself complementary, the results presented here show those factors that interviewees recognized as being mainly facilitating or mainly hindering in their real world policy making experience. Factors found to facilitate or hinder the use of evidence can be categorized into three main domains:

• •Domain 1-organizations, systems and infrastructure;
• •Domain 2-access and availability of relevant evidence;
• •Domain 3-networking and collaboration between researchers and policy makers.

Fig. 1.

Overview of facilitators and barriers for the use of evidence in HEPA policy making as mentioned by interviewees.

In addition factors that appeared specific within the country or policy context are described.

3.1. Domain 1: organizations, systems and infrastructure

Within the domain of organizations, systems and infrastructure the following facilitating factors were most frequently mentioned. Support of administration, which included the organizations’ structure, resources, systems, staff and their skills (Fig. 1 upper left), positive attitudes from managers, setting clear cut criteria for how the policy process should evolve, close monitoring of the process and training of personnel in the use of evidence in policy making.

“My experience from local level is that the management is concerned with a high level of professionalism…… hence the fact that we have the (professional) capacity as well as a management concerned with evidence based policy making (Local policymaker, Denmark).”

In addition some interviewees mentioned the role of media. Evidence from traditional media sources such as television and newspaper articles, but also relatively new sources such as social media (blogs, internet and twitter) were felt to have a substantial impact through framing the problem and its potential solutions by providing exposure for decision makers.

“Yes, I have the idea that it [priorities] also comes from the citizens, sometimes because of what is mentioned in the newspapers and sometimes it is also what the minister of state has experienced him- or herself (National policymaker, Netherlands)”

Barriers which were most frequently mentioned were the lack of simultaneity between research and policy making (Fig. 1 upper right), lack of resources (Fig. 1 lower right), financial and time constraints, and the fact that research evidence hardly ever is directly applicable in policy within the tight political timeframe. In addition, lack of staff competences and factors related to ‘relevant research achievable’ were mentioned (Fig. 1 middle left). A regularly updated, well-functioning research infrastructure was mentioned as a basic precondition.

3.2. Domain 2: access to and availability of relevant evidence

In the second domain often mentioned hindering factors were inapplicability of evidence because of its complexity, extensiveness or being too theoretical (Fig. 1 upper right). Also the lack of direct relevance was mentioned frequently because often policy makers face situations where research evidence does not comprise information on (economic) impact and costs on concrete policy measures, or on the specific local problem or policy context.

“Evaluating national policy in a systematic way leaves room for improvement so to speak. …. Of course the Court of Auditors prioritizes this continuously. They put this lack of evaluative information on national policy expenditures on the agenda continuously… (National policy advisor, Netherlands).”

“If we should work evidence based we should not be doing anything as there is no evidence we can translate …directly to municipal context. (Local policy maker, Denmark).”

“Starting with 2001 there is no research conducted [in HEPA domain] because of lack of funding. Instead, we use annual reports from county level representatives, observations, and population participation rates and expressed preferences [for guiding our work]… The efficacy indicators from the National Program cannot be evaluated due to lack of data. (Local policy advisor, Romania).”

Some interviewees stressed the importance of the availability of tools and methodologies in order to get timely, concrete and practically applicable evidence, namely tailored research evidence. However, funds especially at local level appeared limited.

“I can no longer pay for a scientific advice. It is not only a funding problem, but also a normative problem: the law does not allow me to ask for advice. (Local policy maker, Italy).”

3.3. Domain 3: networking and collaboration between researchers and policy makers

The third domain is about factors related to networking and collaboration between researchers and policy makers, which were often mentioned as potential facilitators for evidence use (Fig. 1 lower left). More specifically, creating cross sector cooperation, seminars and conferences where researchers and policymakers could meet in order to facilitate building a joint language and reference framework.

“..that one can participate in a half- or full-day conference and yes, listen to results and conclusions being presented while I also get the opportunity to discuss my own experience, that's how it becomes a platform you can indeed stand up on (Local policy maker, Denmark).”

“Academics do not accept not to be protagonists. Vice versa, policy makers believe the world is only what they see. A neutral place would weaken the sense of supremacy of the two parts. There must exist a neutral place in which each one feels less self-referential and accept peer discussion (Local policy maker, Italy).”

Similarly, direct and frequent face-to-face interactions between researchers and policy makers and working together as early as possible in policy-related research projects were mentioned as facilitators for improving mutual understanding.

“If there would be clear and applicable research studies, that would allow decision makers to understand the ways using the results could be beneficial, then decision makers would be open [to use these studies]. Otherwise, if the studies are encrypted in a scientific language; they will only remain on paper (National policy maker, Romania).”

“We based our policy on facts, not opinions. We used international research results, but also data from the research projects on the national level and information produced by research institutions in various research projects. In addition, we benefited from individual researchers and their knowledge on specific research results in Finland or elsewhere (National policy maker, Finland).”

Interviewees also mentioned personal liaisons or ‘linking pins’ between knowledge institutes and policy making organizations as facilitators. Working closely together in various ways facilitates balancing the uptake of evidence stemming from research and other types of evidence, like ‘best practices’ and ‘common sense’ or ‘what works by intuition’ type of knowledge. This also relates to how the political relevance of (research) evidence is valued (Fig. 1 upper right). In the case that political interests (because of pursuit of voters, political values) and scientific evidence do not complement each other, it may lead to compromises to the extent to which (research) evidence uptake occurs.

“I don’t know if I really want to say this, but there wasn’t any evidence to say that we could do it. It was actually an aspiration and an absolute belief by the bid team in particular about ’We can’ (National policy maker, UK).”

This seems especially relevant because personal characteristics, interests and values of all stakeholders (policy makers, researchers and politicians) were mentioned by some as important as to whether individuals actually show ‘a will to cooperate’ and are open to discuss the use of evidence in policymaking at all.

“In the very beginning it was very difficult to get involved stakeholders to come together to prepare a joint paper on nutrition and physical activity. The fear was that the significance, the amount of resources and entity of physical activity as its own issue would be decreased if nutrition is in the same policy e.g. at the same sandbox sharing the sand cakes with physical activity’ (National policy maker, Finland).”

3.4. Country and context specific factors

The analysis also revealed some findings as to how barriers and facilitators appeared specific to the country context. Political relevance of the issue at stake and media sensitivity is mentioned in UK, Denmark and Dutch cases as being quite influential. Media can influence political prioritizing and agenda setting. It also helps create research friendly trends by making policy decisions more transparent to the public.

The example of the sports policy linked to the Olympic Games in London 2012 illustrated that regardless of ample research evidence, a contextual factor such as political will in large part determined whether and how this evidence was used. The conclusion from the interviewees in this respect was: politics decided whether or not a certain program or policy is launched, independent from what research evidence suggested [32], [33]. Politicians simply felt that they had to try (because they had a hunch that it might work) and that London 2012 Olympic Games could inspire a generation to take up sport. This idea had already been accepted despite the fact that research evidence from other countries where this already had been tried before [26], [27] clearly showed a lack of success in motivating the general public by an Olympic Games policy initiative.

The Finnish policy cases also showed that there was no lack of available evidence as such, on the contrary, there was plenty of evidence available, however, evidence uptake in policymaking appeared to depend mainly on personal enthusiasm, willingness, competences and perceived needs of policy professionals. Despite the abundant availability of evidence on HEPA policy issues, restraints in time, resources and competencies withhold policy officers from evaluating and implementing all this information. When evidence was used it mostly stemmed from direct (personal) relationships between researchers and policymakers, that by the way in Finland were present quite often.

The Dutch and Danish cases were similar to the extent that HEPA policies are mainly locally initiated and implemented. The Danish cases focused on the local and regional level only (no national HEPA policy) while the Dutch case had a general national umbrella policy, however the Dutch HEPA policymaking was carried out at municipality level. Both cases showed that locally relevant and directly usable evidence appeared to be what stakeholders were lacking mostly. Despite a well-functioning national research infrastructure in both countries, municipality policy officers appeared to lack practical information on for example cost-effectiveness and concrete interventions or policy measures fitting the actual local context and problems. In the Netherlands the distance (created by law) between national level policy ambitions and the implementation of local policy was mentioned to hinder uptake of evidence. Trust based (personal) relations between researchers, policy advisors and policymakers at local level were mentioned in both cases as facilitating the use of evidence.

The lack of concreteness and relevance of evidence to the local context was mentioned by Italian interviewees as an important barrier for use of evidence in policy. The Italian case also showed incomprehensibility and difficulty to understand evidence as a key barrier. Translation and synthesis of the evidence and organizing meetings between researchers, policymakers and other stakeholders so that (personal) relationships and networks could be established, were seen as main facilitators.

Finally the Romanian cases showed how a country’s national and local administrative context is of specific influence on the uptake of evidence in policy. It showed that in Romania a conjunction of empirical, experience grounded evidence and available resources and opportunities at local level were leading in HEPA policymaking, as research evidence appeared not available or not of interest to policymakers as it was judged to be policy irrelevant. Romania differed from the other five countries in that there was little information available on how the policy was being developed, which stakeholders were involved and to what extent. Interviewees were also difficult to reach or were not willing to participate and the literacy level regarding evidence-informed policy making and related terminology of the interviewees was rather low. The stakeholders that participated mentioned that in general, research evidence was lacking and that the evidence that was available was mostly not relevant to the specific policy context and therefore not very useful for implementation. However, one interviewee mentioned that the research evidence that was available such that from PhD theses lacked visibility or political interest and was therefore not used. In addition policymakers not being used to including evidence in their work and the lack of financial resources at local level were mentioned as specific barriers. To compensate this lack of concrete (research) evidence, other information such as direct (field) observations (e.g. field visits), experiences from previously implemented programs and local level practitioners’ feedback was used.

4. Discussion

This article presents findings from semi-structured interviews with 86 stakeholders involved in the policy development of 21 HEPA policies from national, regional or local levels in six EU countries. Factors hindering or facilitating use of evidence in real-life policy making processes were explored and analysed. Barriers and facilitators mentioned most frequently were categorized into three general domains: organizations, systems and infrastructure; access and availability of relevant evidence; and networking and collaboration between researchers and policy makers. Further specifics of how barriers and facilitators appeared to interact with country or policy contexts were described. This study confirms that most of what is already known from recent literature on key hindrances in uptake of evidence in policy making in mostly English speaking countries, also holds true for other European countries. Besides this finding which is based on a substantial number of interviews in 6 European countries, we also found contextual differences in the country cases. Firstly, some cases showed that the attitude of media towards underpinning policy with evidence (as done in television debates, newspapers, social media) as well as the political context in which policymaking takes place influences evidence uptake in policy. Policy decision makers (e.g. politicians) may feel that the media’s attitude has a large impact on voters and therefore act accordingly. Reviews in the literature on barriers and facilitators in the use of evidence to develop health policy mainly focus on factors influencing researchers, practitioners and policy officers as main stakeholders. Media appear not to play a substantial role among these factors [34], [15], [35], [36], [10]. Orton et al. [10] do mention aspects such as political viability and degree of community support as non-evidentiary factors of influence on policy making. Media engagement can of course strongly affect these aspects. The fact that we found stakeholders explicitly mention that media has an impact on decision makers’ opinions on the use of evidence adds new insight to this knowledge. In addition, the rapid development of social media makes its influence on the uptake of evidence hard to find in recent literature which makes it even more relevant. These findings indicate that the tactical or political model of evidence utilization in public health policy resembles reality more closely than the problem-solving or knowledge-driven model [16], [37], [38], [39].

Secondly, personal characteristics of both users and providers of evidence as well as their relationships were highlighted as being important. Skills, attitudes and values of individual policy makers (national and local level) and their ‘will to cooperate’ have an impact on the extent to which they will access and use available evidence. Oliver et al. stated that improved skills [15] and building personal relationships with researchers [18] are important facilitators for evidence uptake. Other research showed that decision makers’ own experiential knowledge impedes the direct influence of research evidence in a study of evidence use among UK policy makers [40]. The importance of ‘individual factors’ is also known in the research literature to play a crucial role in the knowledge conversion process [41], [16], [10]. Our results also seem to be consistent with the ‘interaction explanation’ according to which ‘knowledge utilization depends on disorderly interactions between researchers and users’ and interactions across the interface between policy makers and researchers which are important to the transfer of evidence to policy [42], [34]. Decision makers’ e.g. politicians’ personal beliefs and perceptions as well as cultural circumstances and traditions in appreciating evidence or more system oriented limitations will always interact with the mere availability or transferring of research evidence [39], [43].

In addition, structural partnerships with all stakeholders generating evidence emerging from active two way partnerships between researchers, policy decision makers, funders and other stakeholders and therefore contextually embedded from the outset, in which evidence is made synthesized or translated into ‘easier to understand and applicable evidence’ may offer potential benefit [49], [50].

Thirdly, our study showed that actual use of evidence requires both optimal policy relevance, which in some countries needs to be locally embedded. A more personal and trust based interaction between researchers and (local) policy makers is also required. Having evidence at hand on when and where it fits the phase and context of policymaking, in particular evidence on costs and consequences of concrete policy measures and interventions, was shown to support use of evidence in policy making. Although a lot of valuable work is being done in this respect by national and international institutions [56], [57], policymakers still appeared to experience a lack of evidence especially on cost effectiveness and implementation of interventions to the local or regional context. This is also acknowledged in literature where timely access to good quality and relevant research evidence and interactions across the interface of research and policy-making are reported to be the most important factors in influencing use of evidence [15], [34], [12], [39].

4.1. Strengths and limitations

In this study HEPA policies from different levels in six EU member states were included. This can be considered both a strength and a limitation. A strength since a large variety of countries and stakeholders involved increased the possibilities for gathering and exploring the whole continuum of potential barriers and facilitators that play a role throughout different contexts and on different levels of policymaking. Selecting policies from different policy contexts offered a possibility to learn from different policy making systems across EU member states. Although the data stem from 2012, findings are relevant as policy processes are long (4–10 years) and dependant on elections and administrative underpinnings that are still in force. Our findings therefore can potentially inform next policymaking cycles. In addition, having different stakeholder groups involved, with researchers being a minority, can be seen as a strength [15]. This study on the one hand offers a vast amount of rich data in its variety and representation. It spans across different political and social contexts that warrants analytic generalization [38], [44]. While on the other hand, it leaves little room for strict comparison, which can be seen as a limitation. This is illustrated by the participation of Romania for instance. Romania is a highly centralized country compared to other EU member states that participated in this research project [39]. Therefore the development of HEPA policies in Romania mainly takes place at national level and within a single sector (i.e. the sport sector) [45]. This national political and administrative context mainly resulted in less articulated information on barriers and facilitators in cross sector policy making in Romania. In general, the barriers and facilitators mentioned in Romania reflect similar aspects such as lack of relevant evidence for policy, although focusing more on implementation of programs than on HEPA policy development processes. A further strength is that the interviewees were chosen for being actually involved in the process of development of selected HEPA policies. This made their responses real life based rather than theoretical policy process. This is considered a major strength because our findings add to the existing literature with new information on the barriers and facilitators that stakeholders experience in the everyday policy making processes.

4.2. Recommendations

For all the barriers and facilitators identified in this study, recommendations can be formulated. With respect to organizations, systems and infrastructure, structural factors within the policy makers’ organization create a strong potential for accelerating research evidence uptake. However, to enhance evidence uptake more attention needs to be paid to social and personal factors. The role of managers [18], relationships and networks in which researchers and decision makers act [20], [19] as well as improved skills of policy makers can make a clear difference in the extent of evidence uptake in the policy process [28], [10], [58]. Also both researchers and policy makers should be more aware of how media impact research utilization by influencing politicians and how to make better use of its impact. ‘Framing’ of a problem is one of the ways policy makers and politicians try to reduce ambiguity and uncertainty of a specific problem. Researchers can support policymakers with providing evidence to support this process of framing [59]. In addition, more timely interaction, collaboration and promoting trust based interactions across the interface between policymakers and researchers could facilitate use of research evidence by bringing about a joint language, more mutual understanding of each other’s norms, values and everyday life realities [15], [34], [46]. And finally the contextual relevance is a main aspect of how the applicability of evidence is valued. Therefore, researchers need to be able to convince policymakers ‘that this works in this policy context’ with the evidence they provide. They need to do it such that attitudes and ideologies of decision makers, their skills and personal values are known and taken into account; otherwise research evidence will not be taken into consideration.

Furthermore, research should focus on interventions that bring researchers and policy makers closer together. It can start with interventions intensifying contact, collaboration and understanding between different stakeholders such as stewardship-based intervention [47] and policy games [48]. One should be aware of the importance to keep off researchers’ independence and academic freedom [51], [52], [53], [11]. In that respect, the role of ‘pivotal’ people who can inspire and trigger the policy making process and an enhanced functioning of knowledge brokers may be studied further [54]. Knowledge brokers were shown to have a positive effect in organizations that are not very evidence oriented [55]. Furthermore, it seems equally important that researchers, policy makers, politicians and media-officers learn about each other’s realities and the different obstacles they are facing. This will help increase evidence to be experienced as ‘relevant’ among different stakeholders and consequently will enhance evidence being used in policy, thus ultimately resulting in more effective public health policy. Our study results therefore urge for interventions that support more structural and personal interaction between all stakeholder groups throughout the entire policy making process in generating, interpreting and discussing the evidence needed to optimally inform public health policy in different contexts.

Conflict of interest

Authors declared no conflict of interest.