Accessing evidence to inform public health policy: A study to enhance advocacy

Abstract

Objectives

Improving population health often involves policy changes that are the result of complex advocacy efforts. Information exchanges among researchers, advocates, and policymakers is paramount to policy interventions to improve health outcomes. This information may include evidence on what works well for whom and cost effective strategies to improve outcomes of interest. However, this information is not always readily available or easily communicated. The purposes of this paper are to describe ways advocates seek information for health policy advocacy and to compare advocate demographics.

Study design

Cross-sectional telephone survey

Methods

Seventy-seven state-level advocates were asked about the desirable characteristics of policy-relevant information including methods of obtaining information, what makes it useful, and what sources make evidence most reliable/trustworthy. Responses were explored for the full sample and variety of subsamples (i.e., gender, age, and position on social and fiscal issues). Differences between groups were tested using t-tests and one-way analysis of variance.

Results

On average, advocates rated frequency of seeking research information as 4.3 out of five. Overall, advocates rated the Internet as the top source, rated unbiased research and research with relevancy to their organization as the most important characteristics, and considered information from their organization as most reliable/believable. When ratings were examined by subgroup, the two characteristics most important for each question in the total sample (listed above) emerged as most important for nearly all subgroups.

Conclusions

Advocates are a resource to policymakers on health topics in the policy process. This study, among the first of its kind, found that advocates seek research information, but have a need for evidence that is unbiased and relevant to their organizations and report that university-based information is reliable. Researchers and advocates should partner so research is useful in advocating for evidence-based policy change.

INTRODUCTION

Improving population health often involves promoting and implementing policy changes.^1–7 These policy changes result from a very complex policy process.⁸ Policymakers should consider a number of issues, priorities, and stakeholders in their decision-making. These issues may include constituents’ needs or opinions, evidence of acceptability or feasibility, health impact, personal interest, local leaders, socio-political considerations, the political dynamics affecting the process, evidence of scientific effectiveness, and efforts of advocacy groups.^9–11 Even though the role of research evidence is one of many influences in this complex policy process,¹² use of such evidence is important because its use can inform policy decisions that will improve public health.

Advocates can play an important role in bringing evidence into the policy process.^{13, 14} For example, the Susan G. Komen for the Cure, is an organization, which raised considerable awareness about breast cancer, changed the national conversation around the disease, and raised billions of dollars for research.¹⁵ A number of different groups can act as advocates including special interest organizations, corporations and their associations (business interest), academics, professional associations, unions, think tank, and foundations.¹⁶ Advocates use a set of skills to create a shift in public opinion and public policy to mobilize the necessary resources and focus to support and change policy.¹⁴ Simply put, policy advocacy can be defined as intentional activities initiated to influence the policy making process.¹⁷ Information exchange between advocates and policymakers is critical for influencing the policy process and mobilizing these changes. Advocates utilize a number of methods to influence the policy change process in the United States, which include garnering public support, building relationships with decision makers, collaborating with other organizations, and serving as an important resource for policy topics. Through these mechanisms, advocacy groups have been able to influence policy.¹⁶ In these efforts to provide information to decision makers and the public to build support for important policy issues, advocates often look to research to help support their position.^{11, 18} While it would be ideal for policymakers to independently seek out research evidence in an unbiased way, they are very busy, often working on a number of issues, and therefore rely on advocates to provide information.^{5, 11, 13, 18, 19} Further, policymakers can use the information provided in a number of ways. The evidence can specifically lead to action, relate to a change in thinking or understanding, or justify a position or action already held or taken.^{11, 20} Because of the potential impact on policy, it is important that advocates provide accurate, research-based evidence.

Researchers can also play an important role in advocacy, by providing advocates with credible and understandable scientific information on health topics that can then be passed along to policymakers.^{5, 11, 13, 18, 19, 21, 22}. Information may include evidence on what works well for whom and cost effective strategies to improve outcomes of interest.²³ However, this information is not always readily available or easily communicated.⁸ Both researchers and advocates often find the lack of use of research evidence in policy decision making frustrating, even though policymakers are under increasing scrutiny for their use of evidence.¹¹ More attention is needed on how researchers can provide advocates with evidence and how advocates can influence evidence use.¹¹ This is particularly true for state legislators, as they are central players in making decisions that affect health programs within their state.^24–26

There are many barriers to providing advocates and policymakers with timely access to useful and interpretable scientific findings.^25–28 Although barriers, such as lack of timeliness^{18, 29} and use of appropriate formats^{18, 30–32} have been suggested, there is scarce information about these barriers and how to overcome them. Given the importance of advocates in the policy process, a better understanding of how to provide them with useful information is essential.⁸ Therefore, it is the purpose of this paper to describe the ways in which advocates seek health information for policy advocacy and to compare and contrast advocate demographics in relation to the way they seek information. This paper is part of a larger study to investigate research dissemination to advocates and policymakers for cancer and other health-related issues.^{10, 19, 33, 34}

METHODS

Sample

To populate the sample of state-level advocates, the research team identified advocacy groups through a Google search using the keywords “advocacy, policy, obesity, physical activity, cancer, nutrition + STATE name (e.g., Alabama).” State level advocates were included because in the United States, states retain much of the power to make decisions about healthcare expenditures and because many public health policy efforts are more effective at the state or local level. To ensure variability in the sample, conservative advocacy groups were specifically sought through websites such as policyexperts.org,³⁵ an online guide to public policy experts and organizations and heritage.org, a web-based resource.³⁶ A broad range of advocacy group types was sampled; the sample included national organizations, such as the American Cancer Society and American Heart Association as well as state organizations, such as Partnership for a Healthy Mississippi. Using the only health-related search terms listed on policyexperts.org and heritage.org, searches were filtered for health-specific advocates using the terms “health and welfare general, Medicaid, Medicare, government health programs, and health care reform.” Employees of the organizations who worked with government agencies or were responsible for public policy efforts of the organization were identified as the contact person. The first list contained 290 contacts. An attempt to reach the contact person by email or phone was made to ensure accuracy of contact information. If the original contact was no longer with the organization, an attempt was made to connect with a new person. The final sample list included 213 valid contacts.

Survey Development

The research team developed survey questions in accordance with the project aims: namely, to better understand how state-level advocates use research information and what makes the information useful to advocates when working with policymakers. Several preexisting questions were used with permission from others working in this area.³⁷ The survey underwent cognitive testing with three advocates, who were representative of the survey sample. These advocates participated in the survey via telephone with research staff. As part of survey administration, these participants were asked additional questions about the clarity and perceived intent of survey questions. The survey was revised for clarity, based on information and suggestions from these interviews.

This study focused on four closed-response questions. The exact survey items are available from the lead author on request and are described in more detail below. Each item was assessed on a five-point scale. The first question asked advocates to rate how often (never to always) they used different sources when working on policy (e.g., talk with colleagues, popular media). In the second question, advocates rated whether proposed characteristics of research information (e.g., understandably written, conducted in my region) made the information useful (low priority to high priority). The third question asked about the reliability and believability (very unreliable to very reliable) of research information based on the source (e.g., government source, the media). Finally, advocates were asked how often they seek out research information (never to always). Advocates also reported on demographic factors as well as their position on fiscal and social issues (liberal, conservative). In United States politics, liberal generally means seeking to expand the scope of government responsibility, and conservative refers to seeking to contract the scope of government responsibility.¹⁶

Survey Administration

Trained interviewers conducted surveys by telephone between February and April 2013; interviews took, on average, 14 minutes. Valid telephone numbers were returned to queue and routinely dialed until the end of the data collection period. Advocate responses were recorded into an SPSS v. 20.0.0 database. The interviews were digitally recorded. In addition to the closed-response questions listed below, advocates answered open-ended, qualitative questions (not used in this analysis), which were professionally transcribed.

Data Analysis

For this analysis, mean scores for each of the four factors were calculated for the full sample, as well as for a variety of subsamples (i.e., gender, age, and position on social and fiscal issues - means by position on fiscal issues are not displayed as there were few differences by position). Age and gender were explored, because these factors, are often related to political positions in the United States.^38–40 Differences between groups were tested using t-tests and one-way analysis of variance.

RESULTS

In total, 77 advocates completed surveys by telephone resulting in a 36% response rate (representing 46 organizations). The sample demographics are summarized in Table 1. Forty-six percent of advocates were aged 50 years or older, and 39.5% were male. Over half of respondents (62%) indicated that they had a postgraduate degree. Sixty-three percent of advocates had worked in health policy and government communication for at least 10 years. In terms of fiscal and social positions, 46% of advocates indicated that they considered themselves fiscally liberal, 17% reported being socially conservative, and 57% reported holding liberal social positions. Forty-seven percent of advocates reported that they always actively seek out research information when working on policy; 1.3% reported never seeking out such information. The mean score for frequency of seeking research information by advocates was 4.3 out of five. There were no significant differences in the frequency with which advocates reported seeking out research information based on gender, age, position on social issues, or position on fiscal issues.

Table 1.

Demographic Characteristics of Advocates Participating in the Survey

Category	n	%
Gender
Male	30	39.5
Female	46	60.5
Age
<50 years	41	53.9
50+ years	35	46.1
Position on Social Issues
Liberal	43	56.6
Moderate	9	11.8
Conservative	13	17.1
Independent/other	11	14.5
Position on Fiscal Issues
Liberal	35	46.1
Moderate	14	18.4
Conservative	21	27.6
Independent/other	6	7.9
Highest level of education completed
Trade, technical, or vocational education beyond high school	1	1.3
College degree	28	36.8
Postgraduate degree	47	61.8
Years in health policy and government communication
0–9	28	36.8
10–19	28	36.8
≥20	20	26.3

Frequency of source use

When advocates were asked to rate how often they used different sources when working on policy, the top-rated factor was the Internet (Table 2). The second highest-rated source was using research to justify a decision s/he made. Advocates under age 50 were significantly less likely to attend seminars or presentations where research is discussed than those over 50 (rating of 3.0 compared to 3.8). This was also the case for taking the results of a relevant scientific study into account and for talking with colleagues about research on issues, where those over age 50 reported greater frequency for these sources. There were no significant differences by gender, position on social issues, or position on fiscal issues.

Table 2.

Mean Ratings for Sources, Characteristics, and Other Factors By Advocate Characteristics

Mean Variable	Total	Gender		Age (years)		Social Issues
Category		Male	Female	<50	50+	Conservative	Moderate	Liberal	Independent
Frequency of source usea
The internet	4.8	4.8	4.8	4.7	4.9	4.5	4.8	4.8	4.9
Research to justify a decision	4.6	4.6	4.7	4.6	4.7	4.4	4.8	4.7	4.7
Talk with colleagues	4.4	4.5	4.3	4.2*	4.6*	4.2	4.7	4.3	4.4
A scientific study	4.3	4.3	4.3	4.0*	4.5*	4.2	4.6	4.2	4.5
Other states	4.1	3.9	4.2	4.1	4.1	4.2	4.1	4.0	4.4
Scientific research reports	3.9	3.9	4.0	3.9	4.0	3.5	4.4	3.9	4.2
Popular media	3.8	3.8	3.8	3.7	3.9	3.8	3.7	3.9	3.5
Research seminars	3.4	3.4	3.3	3.0*	3.8*	3.3	3.6	3.3	3.6
Contact scientific researchers	3.1	3.4	3.0	2.9	3.4	3.2	3.9	2.9	3.2
Research bureaus	2.9	2.6	3.1	2.7	3.1	2.5	2.9	3.0	2.9
Characteristics of research information b
Information is unbiased	4.6	4.3*	4.8*	4.7	4.5	4.5	4.9	4.5	4.9
Relevant to my organization	4.6	4.6	4.7	4.6	4.6	4.5	4.7	4.6	4.6
Is understandably written	4.5	4.1*	4.7*	4.5	4.4	4.2	4.7	4.5	4.5
Brief, concise way	4.4	4.2	4.5	4.4	4.4	4.2	4.6	4.3	4.7
Available when decisions being made	4.4	4.2*	4.6*	4.5	4.3	4.0	4.7	4.5	4.5
Policy cost effectiveness	4.4	4.0*	4.6*	4.4	4.3	4.3	4.4	4.3	4.5
Story of health issue affects my constituents	4.3	4.2	4.4	4.1	4.5	4.2	4.4	4.3	4.2
High priority issue	4.3	4.3	4.2	4.0*	4.6*	4.2	4.3	4.2	4.4
Provides policy options	4.2	4.1	4.2	4.0	4.4	4.0	4.1	4.2	4.4
Delivered by someone I trust	3.9	3.9	3.8	3.6*	4.2*	4.4	3.8	3.7	4.0
Conducted in my region	3.8	3.7	3.8	3.8	3.7	3.8	3.9	3.7	3.8
Supports position I hold	3.6	3.5	3.7	3.5	3.7	3.8	3.3	3.6	3.5
Implications politically feasible	3.5	3.0*	3.8*	3.4	3.6	3.4	3.9	3.6	2.9
Reliable and believablec
My organization	4.6	4.5	4.7	4.5	4.7	4.4	4.7	4.6	4.6
University	4.2	4.1	4.3	4.2	4.3	3.7*	4.7*	4.3	4.1
Government source	4.0	3.7*	4.1*	4.0	4.0	3.5*	4.6*	4.1	3.6
Advocacy groups	3.2	3.1	3.2	3.3	3.1	3.2	3.1	3.3	2.9
Other advocates	3.0	3.0	3.0	3.1	2.9	2.9	2.9	3.1	2.9
Industry	2.6	2.6	2.6	2.7	2.5	3.0	2.7	2.4	2.7
The media	2.5	2.6	2.4	2.5	2.5	2.7	2.1	2.4	2.7
Lobbyists	2.3	2.1	2.4	2.4	2.3	2.4	2.1	2.3	2.6
Actively seek out research informationa
	4.3	4.3	4.3	4.2	4.4	4.2	4.4	4.2	4.6

^*p<.05;

^a1 = never to 5 = always;

^b1 = low priority to 5 = high priority;

^c1 = very unreliable to 5 = very reliable;

^d1 = not at all useful to 5 = highly useful

Characteristics of research information

Characteristics of research information that made it most useful to advocates (Table 2) were “research information is unbiased” and “research information is relevant to my organization”, with the second-most useful characteristic being “research information is understandably written.” Female advocates were significantly more likely to report the importance of research information being unbiased, being understandably written, providing data on the cost effectiveness of a policy, having politically feasible implications, and being available at the time decisions are being made. Younger advocates were less likely to report the importance of information being delivered by someone they know or trust and dealing with an issue they feel is a high priority for state legislative policy action. There were no significant differences by position on social or fiscal issues.

Reliable and believable

Research information from their own organization was rated as the most reliable and believable to advocates, followed by research from a university (Table 2). There were a number of differences in the reliability and believability of research information from a government source based on advocate characteristics. Male advocates were significantly less likely than female advocates to find research information from a government source reliable and believable. Advocates identifying themselves as moderate on social issues rated information from government sources most highly in terms of reliability, followed by liberals, independent/others, and conservatives. Further, those considering themselves liberal on fiscal issues rated this significantly higher (mean 4.2) than those rating themselves as conservative (mean 3.5). Social and fiscal positions were also found to be important in the reliability and believability of research information from a university source. Advocates rating themselves as fiscally independent/other finding information from a university source to be more reliable and believable (mean 5 out of 5) than those rating themselves as fiscally conservative (mean 3.8) and those rating themselves as socially conservative having significantly lower ratings for universities than moderates. There were no significant differences by age.

Overall, when ratings were examined by subgroups, the two factors for each question that were most important in the total population emerged as most important for every group (e.g., men, women, all ages, fiscally conservative, fiscally liberal), with few exceptions. Notably, those rating themselves as socially conservative rated research being delivered by someone they know and trust as the second highest priory characteristic, not research being understandably written, which was the second highest response in the remainder of the advocates surveyed.

DISCUSSION

This study describes the ways in which advocates seek health research information for policy advocacy and compares advocate demographics in relation to the way they seek this information. While we found that advocates seek out information often, less than half of the advocates surveyed reported that they always actively seek out research information when working on policy. As advocates have specialized knowledge and skills, they play an important role in ensuring that research evidence is used in the policy process.^{17, 41} It is important that advocates have access to the scientific evidence necessary to inform an evidence-based policy agenda and that this information have the characteristics advocates find most important and that it comes from sources they feel are reliable/trustworthy.

Overall, advocates in this sample rated the Internet as the most frequently used source to research information when working on a policy issue. This was true across age, gender, fiscal and social positions, indicating that all types of advocates are utilizing the Internet to access research information when they are working on an issue. This has important implications for the way in which researchers can more effectively disseminate their findings. To enhance the use of the research evidence they produce, researchers, research centers and government bodies (e.g., United States Centers for Disease Control and Prevention, National Institute for Health and Care Excellence United Kingdom, United States National Institutes of Health, Medical Research Council United Kingdom, state health departments) should create briefs or summaries for completed studies and make them easily available on websites advocates frequently use. For state-level advocates, this may be at the level of a national society or other national-level sources such as National Conference of State Legislatures, which is a non-partisan group providing states support, ideas, and connections.⁴² Further exploration is needed to identify specific sites most often used by advocates, so researchers can target these for reporting results.

Advocates reported that the reliability and believability of research information varies depending on the source. Universities and their own organization were the most important for most subgroups of advocates, however there were important differences based on advocate characteristics including gender, position on social issues, and position on social issues. For example, advocates identifying themselves as independent/other on fiscal issues rated the reliability of research information from universities, on average, as 5 on a scale of 1 to 5, while those considering themselves conservative on such issues rated information from the same sources as 4. Thus, it may be important to tailor the source of research information to advocates, perhaps by partnering with organizations the advocates trust. This may legitimize research information in their eyes, encouraging its use in the policy process.

Advocates in this study identified several characteristics that make research information more useful to them in their work. They reported information that is unbiased, relevant to their organization, and understandably written made the evidence most useful. While the characteristics of what makes research information useful to advocates has not been well studied, there is a literature on making research information available to policymakers,^{9, 18, 29, 31, 43} and findings have been similar to those of the current study. Therefore, techniques found to make research information more useful for advocates may also, in turn, help effectively inform policymakers. Examples of techniques that may cross over these two groups include providing local examples so the information is more pertinent to constituents and providing information that is concise, understandable, and relevant to current debates.^{29, 31, 44} If researchers would like to see their findings effectively inform policies that promote health, they should ensure that the information is tailored to meet advocates’ needs.³¹

The current study has limitations that warrant mention. First, the sample of advocates interviewed was not generated from a comprehensive list. The research team conducted a thorough web-based search, but it is not known how current online sources may have been. Second, we do not know if there are inherent differences in those advocates on the list who could not be reached or refused to respond compared with those that participated. Third, we had a low response rate, so we cannot assume universal generalizability. Even though participants in this study were limited to the United States, the concepts of evidence-based policy and advocacy are likely to be relevant in other Democratic countries as reported in several other studies. Fourth, a small number of items were used to collect this information. It is likely that additional probing might allow for a deeper understanding of advocate preferences. Finally, lobbyists were rated as the least reliable/believable source for information; there is the potential that this and other findings were due to social desirability bias. In spite of these limitations, this study is unique and addresses a knowledge gap about how best to make research accessible to inform the policy process via advocates.

Policy efforts are an important way to improve public health.^1–7 Advocates play an important part in the policy process by being a resource to policymakers on health topics. This study found that advocates seek out research information in their work, but find evidence which is unbiased and relevant to their organizations and which comes from a university source is more useful and reliable/trustworthy. Therefore, to provide advocates evidence-based resources they will trust and use in their work, researchers and others who produce and disseminate research should partner with universities and advocacy organizations, the most trusted sources for unbiased information, to ensure that the research information they create is useful for those advocating for evidence-based policy change.³⁴

Highlights.

• We describe ways advocates seek information for health policy advocacy

• Advocates seek unbiased and relevant research information

• Advocates report that university-based information is reliable

• Researchers and advocates should partner so research is useful