Uses of Research Evidence in the State Legislative Process to Promote Active Environments in Minnesota

Abstract

Purpose

To describe how research evidence and non–research-based information are used in testimony and other legislative documents used in arguments for and against physical activity–related bills in Minnesota.

Design

Content analysis

Setting

Documents and oral testimony archived by the Minnesota State Legislature from 2007 to 2011.

Subjects

Not applicable.

Measures

A coding instrument was developed to measure descriptive features of materials (e.g., length, document type) and the presence or absence of certain types of research evidence and non–research-based information.

Analysis

Frequencies of variables and measures of associations using Pearson χ² tests.

Results

Over a third (36%) of the sample contained research evidence, and 88% of the sample contained non–research-based information. Compared to materials related to physical activity, materials related to built environment were significantly less likely to reference research evidence.

Conclusion

Despite an abundance of evidence, research evidence was present in only about one-third of the sample. There may be opportunities during legislative discussions on the built environment for obesity-related data to help make the case for sound policies.

Purpose

Preventing obesity is an urgent public health challenge that demands innovative policy approaches, yet the pace of translating evidence-based approaches into policy and practice is slow.¹ Researchers have described numerous challenges in translating research into the policy arena.^2–5 Previous work lacks a comprehensive assessment of the frequency with which evidence has been used in the legislative process over multiple obesity-related policy issues. Such an assessment is an important first step toward designing a more efficient system for translating evidence from research to policy.

This study convened a faculty-community study team to analyze the use of evidence in obesity policy making in Minnesota. The team included health policy and nutrition researchers, a health advocate, a public health lawyer, and a university-policy liaison. This report describes the extent to which research evidence and persuasive information were used in policy-relevant materials (e.g., testimony, fact sheets) presented during legislative discussions on physical activity and the built environment in Minnesota between 2007 and 2011.

Methods

Design

To identify materials for the sample, we relied on study team experience and key informant interviews to develop a timeline of obesity-relevant legislation introduced between 2007 and 2011 in Minnesota. We identified 13 bills that were diverse in issue focus and legislative outcomes. Of these, six had a primary focus on changing nutrition behaviors and were not included in this analysis. Seven bills were relevant to active living because of their potential to affect physical activity–related behaviors at the individual or community levels (e.g., Complete Streets, physical education standards, school siting, and school-community shared use agreements). We further characterized active living materials as having a major focus on “physical activity” or “built environment” policies. Materials that discussed individual-level physical activity and physical education were considered “physical activity” materials. Those that discussed community-level health promotion (urban planning, shared use of school property) were considered “built environment” materials.

Sample

We used the Minnesota Legislative Library and State Legislature archives to identify and collect all committee meeting materials related to each legislative event, including bills, fact-sheets, reports, and audio- and video-recorded testimony. The searches resulted in a sampling frame of 200 items (130 testimony, 70 documents). Out of feasibility concerns (e.g., coding staff time), we selected a 50% random sample of testimony and included all other documents except for bills. Bills in the Minnesota Legislature rarely include a preamble or introductory language. The study team therefore excluded bills because they did not include language eligible for analysis and would misrepresent the types of rhetoric used in Minnesota policy making. These restrictions yielded a total of 69 materials for analysis.

Measures

We developed a structured coding instrument with 66 items to collect data on descriptive information about each material (e.g., length, author) and to characterize the types of research evidence (e.g., data on children, data on impact of programs and policies) and non–research-based information (e.g., use of anecdotes, appeals to political principles) present. (The coding instrument is available from the authors upon request.) We also developed a definition book to assist in consistency between coders. Only statements with clear predefined signals of research evidence, such as a citation; mention of “study,” “data,” or “researchers”; or use of words like “rates,” “odds,” or “significantly” were coded as research evidence. Non-research information was defined based on literature on the policy making process,⁶ and included information cited as influential in policy making: public opinion, expert beliefs, cost information, stories/narratives, emotional appeals, and media influence. Documents often contained both research evidence and non–research-based information, often multiple times, and are thus not mutually exclusive. The presence or absence of all types of information was recorded as dichotomous variables. Two coders coded all materials, with 20% double coded to ascertain interrater reliability (for all variables reported, κ ≥ .70). Video and audio testimony were coded in their original forms, not transcribed.

Analysis

We calculated frequencies of variables and statistical measures of associations using Pearson χ² tests in STATA 11, with significance set at α ≤ .05.

Results

The sample consisted of testimony (n = 44; 64%), fact sheets (n = 7; 10%), policy briefs (n = 1; 1.5%), and other policy-relevant documents (n = 17; 25%), e.g., letters to stakeholders. Thirteen materials were introduced in committee meetings in 2007, 0 in 2008, 35 in 2009, 14 in 2010, and 7 in 2011. Testimony length ranged from less than 1 minute to 27 minutes, with 80% of testimony lasting 6 minutes or less. Paper documents ranged from one to five pages, with 72% of documents between one and two pages.

Over a third (36%) of the materials in the sample included at least one reference to research evidence. Evidence describing the health- and non–health-related consequences of obesity (68%), the impact of policies or programs (64%), and the prevalence of obesity (48%) were mentioned the most frequently. In contrast, 88% of the materials included non–research-based information, with presentation of expert beliefs (51%), public or constituent opinion (39%), cost or financial impact (33%), appeals to political principles (e.g., unfunded mandates) (31%), or anecdotes/stories about communities (21%) particularly common.

Compared to the materials concerning the built environment (N = 39), materials related to physical activity legislation (N = 30) were significantly more likely to mention at least one type of obesity-related research evidence (50.0% vs. 25.6%; p = .037; see Table).

Table. Use of Research Evidence and Non–Research-Based Information in Documents Related to Active Living Legislation.

	All Active Living Documents (n = 69) No. (%)	Physical Activity Documents (n = 30) No. (%)	Built Environment Documents (n = 39) No. (%)
Use of research evidence	25 (36.2)	15 (50.0)	10 (25.6)*
Use of non–research-based information	61 (88.4)	27 (90.0)	34 (87.2)

^*Indicates significant difference between built environment and physical activity from χ² test, p = 0.037.

Of the 25 materials that cited research evidence, the most common pattern of research citation was not to cite a source (n = 14; 61%) or to refer to research generically (n = 11; 49%) by saying that “research shows” or “a recent study” without referencing a journal name or academic institution. There were no significant differences in how materials related to physical activity and the built environment referenced evidence sources.

Discussion

Summary

Despite an abundance of research evidence on community-level approaches to physical activity,² only one-third of sample materials cited any such evidence in legislative discussions concerning active living–promoting bills under consideration in Minnesota from 2007 to 2011. Non–research-based information was much more prevalent, appearing in almost 9 in 10 materials.

When physical activity and built environment legislative materials were compared, materials connected to physical activity legislation contained more research evidence on obesity prevention. This matches the study team's impressions from listening to testimony. Advocates and legislators who favored community design legislation tended to reference the policies' impacts on the environment and economic vitality, but the connection between the built environment and health was mentioned only in passing. Although we do not have data to test this directly, this difference across policy type may be because advocates testifying on many of the built environment bills were concerned mostly with climate change and environmental issues, whereas advocates promoting physical education were often from health-related organizations. These findings suggest that there may be opportunities during legislative discussions on the built environment for obesity-related data to help make the case for sound policies. Bringing the health perspective to the table may personalize built environment issues and elicit support from elected officials who are less concerned with other impacts.

Limitations

First, the sample only included publically available materials. Discussions between legislators and advocates that occur out of the public eye can be influential in determining whether legislators support a bill, but the research team was not privy to those conversations. Second, the study did not account for the quality of the research evidence presented. It is well known that not all research evidence is equally sound.³ Finally, our results are specific to one state's experience, so the findings may not be generalizable to other state contexts.

Significance

Health promotion policies designed based on the available evidence may be more likely to be effective, increasing the likelihood of practical and political success.⁷ As an initial step to improve the translation of evidence from research to practice, it is essential to understand how research evidence is currently used in arguments for or against legislation related to creating active environments.

SO WHAT? Implications for Health Promotion Practitioners and Researchers.

What is already known on this topic?

Despite the abundance of research evidence on obesity, that evidence is slow to reach policy makers. Previous studies have examined evidence translation in health departments and among other decision makers and described barriers to using evidence, such as timing and political factors. It is unclear whether certain types of obesity-related evidence are more common in policy discussions.

What does this article add?

This study contributes results on the use of research evidence in legislative discussions about physical activity and built environment bills at the Minnesota State Legislature, showing use of obesity-related evidence is more common in physical activity–related discussions.

What are the implications for health promotion practice or research?

The results suggest an opportunity for obesity-related data to help make the case for built environment–related policies that can promote health. This study takes a first step toward a better understanding of how research evidence is used in legislative settings, potentially leading to improved translation of research into health promotion policies and practice.