Pathways to the Use of Health Services Research in Policy

Abstract

Objective

To apply social science theory so as to define more explicitly the pathways that influence policy makers' use of health services research.

Methods

The analysis builds on a literature review and the author's observations. It identifies important social science concepts relevant to use of research in policy and organizational decision making. It integrates and expands upon existing frameworks to differentiate and analyze 10 pathways that can lead to the use of health services research by policy makers.

Principal Findings

The process through which research is applied involves many factors, only some of which are amenable to influence by researchers. Within these constraints, multiple pathways can drive research use; no one of these is likely to perform better in all circumstances. Successful uptake is more likely when these pathways cause findings to be converted into messages meaningful to policy makers. Various intermediaries play an important role in creating effective pathways, while users also can influence them.

Conclusions

The pathways open up what too often is an unexplored “black box” that mediates between health services research and its use by policy makers. Such pathways can help stakeholders to bridge different perspectives in ways that strengthen the possibility that effective research will be supported and used.

PURPOSE

Clinical research seeks to identify effective treatments that clinicians can use for individuals, while health services research focuses on the performance of the entire system. Health services research has value in itself, but often its relevance rests in its application (Buxton and Hanney 1996; Gray 2003;).1 Many studies examine how economic, organizational, policy, and market variables influence the way patients and clinicians interact to shape outcomes—such as cost, access, and quality—at the societal or systems levels. The complexity of issues addressed by health services research, and the role of knowledge, values, and ideology in decision making, can challenge and limit use of such research (Hanney et al. 2003), yet the key link between research findings and their applications often goes unexamined.

In this paper, we seek to focus more explicitly on the potential pathways that link research to policy makers' decision-making applications. We review relevant concepts from research on policy formation and organizational behavior, factors influencing research use, and knowledge transfer and exchange strategies. With this base, we provide a framework to highlight processes mediating research and its use and with it identify and discuss 10 diverse pathways that may be relevant in different circumstances. We end by discussing the implications for researchers, users, and those bridging both worlds who want to enhance the use of health services research. The assumption is that researchers and other stakeholders can play a more constructive role if they better understand the links between research and its application.

BACKGROUND AND RELEVANT RESEARCH

Interest in whether research is used in policy making—and how to promote such use—is not new; it has been a concern in social policy research for almost as long as such research has been funded (Davis and Salasin 1975; Weiss 1979;). Researchers who wish their work to be relevant can benefit from past efforts to address these concerns, some of which may challenge deeply held assumptions. When researchers still are surprised that rational frameworks of “knowledge transfer and exchange” do not always mesh with the underlying politics of health policy making (Mitton et al. 2007) something is amiss in how social science findings are conveyed to researchers in various disciplines.

How Policy Is Made and Organizations Behave

For years, policy incrementalism, and the seeming randomness or ambiguities with which some problems and their potential solutions are treated in decision making, have been issues in political science and organizational theory (Lindblom 1959; Kingdon 1984; Cohen, March, and Olsen 1972;). For example, stakeholder-preferred policy solutions may be linked to particular events or problems of current relevance (“windows of opportunity”) to take advantage of opportunities, even when the solutions may not be particularly applicable to those problems. Research findings may be used to define problems that can be solved by or support preferred solutions, regardless of their policy relevance. Research with insights not currently relevant, or in conflict with preferred solutions, may be ignored. Ongoing research may be used as ammunition to promote the status quo by delaying action or deflecting criticism (Weiss 1979).

Because research is not valuable at a particular time, however, does not mean that it will be valueless in the future. Such research can be drawn upon if circumstances or needs change (Cohen, March, and Olsen 1972). Whitehead et al. (2004) term this the “sleeper” effect, reflecting what Hanney et al. (2003) describe as the “research reservoir.”

Typically, there is no single decision maker or audience for research findings; rather, there are many players in a political or organizational process. Each may have different goals and values, and some at least will have predetermined positions, which research is used to support (Allison and Zelikow 1999). This use can sometimes make researchers uncomfortable (Spitz and Abramson 2005), but it may be legitimate, especially if the findings are presented appropriately (Weiss 1979).

The nature of the political process raises issues for researchers seeking to link their work to policy formation. For example, in examining U.S. policy on equity, Gamble and Stone (2006) argue for rejecting the classical model of research use, which assumes that disparities in health care are addressed only if they are first documented as research findings and then disseminated. Instead, the authors argue, health disparities are addressed only if they are converted into political issues that are then defined as something government can address with the tools it has available. Viewed in this way, effective research must frame the problems it reveals as “bad situations and moral wrongs that government can and should fix.” Regardless of whether one agrees with researchers framing the policy program, this function may be essential to the incorporation of research into policy making. The real issue is who should do the framing.

Factors That Influence the Use of Research

Health services research can support the policy process by generating many kinds of knowledge, including the following: evidence that a problem exists, examples of the impacts of policies on people and organizations, controlled evaluations of policy initiatives, feedback from natural experiments with variation, and historical evidence (Whitehead et al. 2004). Barriers to using this evidence also exist, however, because of the inherent ambiguity of issues and the policy process, institutional constraints that lead to fragmentation of focus, and competing interests (Waddell et al. 2005). Further, there may be persistent entrenched ideas (e.g., managed care is inherently bad because of its constraints on professional behavior) and conflicts between competing philosophies (Waddell et al. 2005). Also, social networks influence the communication of ideas, and both the diffusion and eventual acceptance of research findings and other forms of innovation take place in stages (Gray 1973; Rogers 1983;).

A recent review of research on factors that influence take-up of new ideas (“knowledge”) in health services organizations provides numerous insights into ways that research use can be enhanced or diminished (Greenlaugh et al. 2004; see Table 1).2 For example, findings are more likely to be acted upon if they generate ideas that are unambiguous and easy to apply. Action itself is less likely to be an event than an evolving process of decision making and implementation in which research can play a continuing role. In organizations, take-up of ideas often occurs through informal processes. Some organizations may be more “permeable” to outside knowledge because of their institutional philosophies, the positioning of influential staff members, their financial strength, or the “organizational slack” necessary to identify new knowledge. Although not always the sources of the research, individuals who share common attributes with those they seek to reach may be more effective translators.

Table 1.

Factors That Influence the Uptake of Innovation or Knowledge

Categories of Relevant Factors	Relevant Variables	Potential Applicability to Health Services Research Transfer
The innovation	Relative advantage, compatibility, low complexity, plausibility, observability, potential for reinvention, fuzzy boundaries, risk, task issues, knowledge of knowledge required, technical support.	Take-up of change or knowledge is easier when the ideas are unambiguous and simple to apply, viewed as useful and consistent with values, yield visible benefits, allow flexible application, and have risks in line with benefits.
The individual	Needs, motivation, values and goals, skills, learning style, social networks.	Individuals differ in ways that influence their reactions to and use of knowledge. By nature they may be easier to reach (early adopters), have a different intellectual capacity or tolerance for ambiguous findings, associate different meanings with the same facts, and make decisions differently (e.g., authoritative versus collective). Regardless, they need to know about findings, have adequate access to information to explore them, and understand the consequences of action based on findings (change).
The user system	Structure (size/maturity, formalization, differentiation, decentralization, slack resources), absorptive capacity for new knowledge (preexisting knowledge and skills, ability to integrate new knowledge, ability to share knowledge across networks), receptive context for change (leadership and vision, good management relations, risk-taking climate, clear goals and priorities, high-quality data capture).	The nature of the “user” system matters, and influences the way take-up occurs, and its likelihood. Take-up is easier in mature and differentiated systems, in which units can make decisions relatively autonomously, when take-up addresses real needs, when resources or slack capacity exist to pursue and absorb knowledge, when external links help to infuse knowledge, and when leadership supports the spread of knowledge and pursuit of new ideas.
The knowledge purveyor or change agent (diffusion and dissemination)	Communication and influence may be diffused (informal/unplanned) or disseminated (formal/planned). Relevant variables include social networks, similarities, peer opinions, marketing, expert opinions, champions, boundary spanners, and change agents.	Interpersonal influence through social networks exerts a great influence over the uptake of knowledge. Uptake is more likely when messenger and audience are similar, opinion leaders support it, a few key individuals champion it, the boundaries between user and outside purveyor are permeable, and when formal programs take account of user needs and perspectives via tailored strategies.
System readiness for innovation	Tension for change, innovation-system fit, power balances (supporters/opponents), assessment of implications, dedicated time/resources, monitoring and feedback.	Uptake (change) is more likely if the status quo is viewed as intolerable, the solution fits the values of the organization, supporters are well placed, and there is time and capacity to evaluate and act on information.
Linkage	Design Stage: shared meanings and mission, effective knowledge transfer, user involvement in specification, capture of user-led innovation. Implementation stage: communication and information, user orientation, product augmentation (e.g., technical help), project management support.	When there is organizational linkage between knowledge generator (research center) and user, uptake is more likely, especially if relations are good. External purveyors of information are more effective when they are similar to users, have good interpersonal skills, can translate user needs to knowledge producers, and enable potential users to make decisions.
Implementation process	Decision making devolved to frontline teams; hands-on approach by leaders and managers; human resources issues, especially training, dedicated resources, internal communication, external collaboration, reinvention/development, feedback on progress.	Follow-through and the implementation process will have a great impact on the ultimate effectiveness of knowledge transfer in influencing the path between knowledge uptake and change.
Outside context	Social and/or political climate, incentives and mandates, interorganizational norm-setting and networks, environmental stability.	External influences on users' influence uptake of information. Formal knowledge transfer initiatives are sometimes, but not always, effective. Congruence with external policy requirements makes uptake more likely, although it does not influence the capacity of the organization to implement.

Source: Author's analysis of Greenlaugh et al. (2004).

Columns 1 and 2 come largely from Figure 3 (p. 595). Column 3 is a composite of textual findings and their translation by the author to the health services research context.

The literature reinforces the importance of viewing the translation of research into policy use as a complex, time-consuming process involving the convergence of a number of factors. Researchers can both control parts of the process and influence others by how they conduct and disseminate their research. They also are likely to be more effective when the system infrastructure supports the dissemination process. Such an infrastructure is particularly useful in facilitating the informal transfer of information through the networks it creates and the support it provides for tailoring the research message to needs of policy makers.

Yet a supporting infrastructure, while important, is not sufficient. Some findings will reach a more receptive audience than others, regardless of the supporting infrastructure. Some users and their organizations will be more receptive to information than others. When the demands of a user's environment converge with research findings, uptake may be more likely, so it is valuable to examine not only what is used, but why or why not.

Knowledge-Transfer and Exchange Strategies

While politics and organizational processes may promote or inhibit the use of research in policy, there are techniques to build bridges between researchers and potential users (Mitton et al. 2007). Lavis et al. (2003; see Table 2) are valuable in identifying five relevant questions:

Table 2.

Framework for Knowledge Transfer and Exchange

Question	Evidence on Best Practices	Caveats and Clarification
What is to be transferred to decision makers (the message)?	Transfer actionable message from a body of research, not simply a single research report or study. Message can place study in context. Research shows that ideas, rather than data, influence decision making.	Not all research can or should have an impact. (This excuse can be overused, however—decision makers need to learn about potential solutions.) Accountability mechanisms regarding the appropriateness of the message need to be in place.
To whom should research be transferred (the target audience)?	Clearly identify target audiences and fine-tune strategies for their decisions. Multiple audience-specific messages are needed.	Learning about decision-making environments requires substantial investment. Research does not show how to identify the audience. One could ask (1) who can act; (2) who can influence those who can act; and (3) which audience is likely to be most receptive, and which messages are relevant to them.
By whom should research knowledge be transferred (the messenger)?	The credibility of the messenger delivering the message is important to knowledge transfer.	Building credibility as a messenger is time consuming. One-size-fits-all is unlikely to work. Researchers with communication skills may be ideal. Trusted intermediaries (knowledge brokers) can substitute.
How should research knowledge be transferred (process and supporting communication)?	Interactive engagement appears most effective, regardless of the audience. (Passive processes are not effective.)	Interaction can occur at many stages of the research process. Over time, two-way exchanges can produce cultural shifts. Individualized feedback can be effective. Websites and newsletters, etc. can augment interaction, especially if it is targeted or searchable.
With what effect should research knowledge be transferred (evaluation)?	Performance measures for knowledge transfer should be appropriate to the target audience.	While behavior change consistent with evidence may be appropriate for clinicians, informing debate may be more appropriate for public policy. Measures can capture process (e.g., presentations made), intermediate outcomes (e.g., awareness, knowledge), or ultimate outcomes (decisions). Research can be used instrumentally (specific and direct ways), conceptually (enlightenment), and symbolically (to justify an action).

Source: Author's summary of Lavis et al. (2003).

• What should be transferred (the message)?

• To whom should knowledge be transferred (the target audience)?

• By whom should knowledge be transferred (the messenger)?

• How should knowledge be transferred (process and supportive communications)?

• With what effect should knowledge be transferred (evaluation)?

Lavis' analysis suggests that research-transfer strategies are effective only if they incorporate:

• “Actionable” findings, usually from a body of research, which can be translated into a “message” in ways that go beyond numbers, facts, or single studies. Whitehead et al. (2004) refer to this as “assembling the evidence jigsaw puzzle.”

• Audience-specific messages that are part of a well-conceived strategy for reaching those who can act and are reachable.

• A credible “messenger” that the target audience respects and trusts.

• Continued interaction between user and messenger over time.

• Realistic performance measures for a given type of research (e.g., informing debate is likely to be an appropriate goal for health services research, while behavior change may be appropriate for clinical research).

The successful pursuit of transfer strategies requires a substantial investment in understanding the audience and its needs, building credibility, and continuing the exchange of ideas (Lavis et al. 2002), but there are substantial barriers to achieving this goal.

AcademyHealth (2006) assessed users' perspectives on the need for health services research; their work highlights perceived defects in researchers' training as skilled communicators and leaders. Intermediaries may be valuable in addressing this shortcoming, including the following:(1) credible “policy entrepreneurs,” who can translate research into state policy implications in relevant and well-received ways (Coburn 1998); (2) “knowledge brokers,” who can translate research into support for actions, leading—for example—to better care for children (Simpson 2004); and (3) “consultants” funded by users to identify and translate relevant knowledge for specific policy or management questions (Jacobson et al. 2005). Mitton et al. (2007) find that personal contacts and trust established through long-term quality relationships are essential to exchange of information. Hence, effective and credible links between research and its users are important.

PATHWAYS LINKING RESEARCH AND USE

To help researchers think more concretely about how translation activities influence whether research findings are used, we have drawn on the literature and our own experience to develop a framework showing the factors that mediate the translation of research findings into messages communicated to and used by policy makers (Figure 1). By “used,” we mean that, at a minimum, policy makers or those seeking to influence them are aware of ideas generated through the process of research, and that those ideas have some influence, either on the debate over policy decisions or on the actual decisions.

Figure 1.

Factors, Processes, and Actors That Shape Pathways between Research and Its Use Source: Author's analysis; the concept of a research reservior builds on works by Hanney et al. (2003). Note: Numbers (in circles) refer to pathways in Table 3.

Table 3.

Potential Pathways to Research Translation by Key Driver and Limits/Barriers

Potential Pathways	Brief Description and Key Elements	Example	Limits and Barriers to Effectiveness and Risks to Effective Use of Research
Research findings drive use
(1) “Big Bang”	Results of a single study, often published in an influential journal and cited in the major media, frame the debate in new ways, or drive initiatives.	Original National Medical Care Expenditure Survey finding on number of uninsured. RAND study showing that a large percentage of care is inappropriate, with overuse, misuse, and underuse.	Window of opportunity for interest in the findings is not always there; most studies by themselves will not have the scope, robustness, or focus for a “big bang.” Uptake is more likely if the message is clear.
(2) Gradual accumulation and diffusion	Researchers follow others in studying given questions, with gradual decentralized build-up of knowledge. No formal mechanisms of diffusion, but policy makers or their aides or staff note the evidence, or interest groups introduce it to the debate.	It becomes accepted that insurance coverage makes a difference in access to health care and outcomes. Administration cites concern over “crowd out” as one reason for opposing state child health insurance program expansion.	No one will make the links to develop the message; older findings are forgotten or hard to find because they were not published in retrievable sources, or were catalogued by different indexes and search engines. No scientific vetting of what users take away as the central findings and messages.
(3) Gradual accumulation and formal synthesis	The research enterprise obtains support that allows it to conduct structured syntheses that summarize research; the syntheses are published or reported to highlight critical findings and the level of evidence behind them.	Former User Liaison syntheses funded through NCHSR; interest by journals such as Medical Care Research and Review in publishing syntheses; MCRR syntheses (e.g., organizational behavior). H.S. Luft book on HMO Performance.	Findings may be synthesized around questions of interest to researchers rather than policy makers. Syntheses could yield a main message that “more research is needed.” Funding to support syntheses is limited, with funders favoring new work. Researchers receive less credit for synthesis than for original research.
Effective intermediaries convert research to policy information
(4) Researcher as messenger—users consult “Experts”	Researchers interact and build relationships with policy makers, which leads to trust and sharing of information. Policy makers or staff become aware that a researcher has studied an area, and call on such an expert for advice.	Certain health services researchers become well-known political pundits and experts, either publicly or behind the scenes. Foundations support congressional fellowships for researchers to work with Congress. Meetings such as AHRQ's Building Bridges link researchers and users.	Pathway is reinforced by researcher visibility and willingness to participate. Effective use requires skills in communication and effective judgment on what findings say about policy questions not addressed directly. Research authority could be used to give credibility to “false” findings. Researchers may not be rewarded professionally for these activities.
(5) Formal intermediary-brokered translation	Agencies are set up to analyze existing research and new data to answer specific kinds of policy questions within a timeframe set by policy makers. Outside public sector groups establish “neutral meeting grounds” to discuss research.	CBO, MedPAC, CRS, and other agencies are established to advise Congress. Various executive branch agencies involved, as are the National Health Policy Forum and Alliance for Health Reform, Kaiser Commission on the Uninsured, Commonwealth Commission on Value-Based Health System, and state health policy centers within or outside of government. Research and analysis activity are within interest groups and surrounding public interests.	Areas of interest determined by the entity's mandated focus, with less attention to research findings of interest to the private sector. Ability to attract staff and gain credibility are enhanced where there is perceived independence. The number of qualified people who can perform this function is limited, and their skills may not be rewarded fully in either the research or policy world. Credibility depends on perceived objectivity of the messenger.
(6) Press publicizes and may generate own research findings	Major media coverage of findings generates policy interest. Some media outlets collaborate to generate their own polls and primary analyses on policy-relevant topics, which they then publicize and use to frame issues.	Weekly pick-up of new findings reported in New England Journal of Medicine, the Journal of the American Medical Association, and Health Affairs web exclusives, along with PBS/KFF/Harvard Polls and others. Investigative journalism in the business sector that sometimes includes primary data collection, or analysis generating new information.	Ability to assess quality of research may be impeded to the extent methods are proprietary or not documented. Important caveats or qualifications could be lost in the interest of enhancing communication of key message. Embargoes may delay public availability of important information.
User seeks to influence or enhance value of research
(7) User defines topics for synthesis of accumulated research	Researchers and policy makers collaborate on identifying policy questions that can be answered by research. Researchers synthesize the answers. Policy makers review analysis to enhance its utility.	RWJF Policy Synthesis Project. Canadian Center for Health Services Research efforts. IOM is commissioned to study and develop evidence-based recommendations on how to address a given issue.	Policy problems selected for synthesis may be too generic to address the specificity with which some policy decisions are framed. Available research often is insufficient to answer the question. The window of opportunity passes by the time a synthesis is commissioned and completed.
(8) User participates in peer review of research proposals	Government establishes a vehicle whereby funding is made available to support a joint researcher/user interactive process to create and use research relevant to policy and practice.	Canadian Foundation for Health Services Research requires investigator-initiated grants to be reviewed by users as well as researchers. Only applicants meeting threshold criteria on quality and relevance are funded. AHRQ's IDSRN ACTION funds projects requiring collaboration between researchers and health delivery organizations or other user groups.	Research that fulfills both user and research criteria may be limited, especially if data are lacking on key questions. Some fear “worst of all worlds”—research neither of good quality nor useful.
(9) Users contract for particular studies	A user (often a government agency) contracts with an organization or individual to conduct a specific study that will answer a pressing policy issue.	HCFA (now CMS) commissions evaluation of whether Medicare HMOs save money. ASPE commissions work on the elasticity of use to price of services (copayments). Congress mandates particular demonstrations or studies.	User direction may lead to very narrowly defined studies not done in time and ultimately not very useful. Funder interests in certain results may compromise the independence of the research.
(10) Researcher as user—when policy makers have research skills	A person with research experience is appointed to a major policy position in the public or private sector and uses that post to encourage translation.	Mark McClellan appointed to head CMS and encourages public reporting and effectiveness research. Karen Davis heads ASPE's health reform initiative and integrates research into analysis. Bill Roper works for Aetna and encourages evidence-based health promotion.	Such appointments are rare and limited to researchers with unique skills, political access, and interests. Demands on the policy maker may lead to loss of credibility or effectiveness, as conflicts between the world views of policy makers and researchers prove hard to bridge.

Source: Author's analysis.

Note: The numbers refer to pathways noted in Figure 1.

Within the framework, we identify nodes that distinguish 10 pathways through which research may get applied in policy making, emphasizing different ways in which messages get framed and communicated, who is involved, and what drives or constrains movement along the pathway. The framework in which the pathways are nested assumes they all operate in a political and professional environment, and that there is a reservoir of knowledge to which researchers contribute and policy makers can use (Hanney et al. 2003). The reservoir's richness and contents are influenced over time by the research funded and its topics. Only research that is completed, documented, and made public contributes to the reservoir; proprietary restrictions, release policies, and publication lags all limit flow into it (AcademyHealth 2004, 2007). Richer, better quality findings enhance potential contributions (Soumerai et al. 1993), so funding streams and their allocation are critical.

Some pathways reflect traditional emphasis on research as “knowledge development” and “enlightenment.” These generally assume that meritorious research findings will find an appropriate audience without much emphasis on the mediating process. Other pathways emphasize the role of intermediary, as well as how users more actively influence the way research is conducted and used. The goal is not to rank pathways, but to create a better understanding of each, and their possible roles in enhancing the use of research.

Traditional Pathways: Meritorious Findings Drive Use

Traditionally, researchers aim for peer-reviewed publication of their work in journals or other vehicles. Such findings influence policy use directly through at least three pathways.

“Big Bang.”

Some studies produce blockbuster findings that immediately inform or change the policy debate and provide value over time (RAND Health 2006). For example, the National Medical Expenditure Survey's estimates on the uninsured influenced congressional consideration of legislation to address the effects of unemployment on health coverage (Wilensky 1987). McGlynn et al.'s (2003) work on the extent of inappropriate care has been used to enhance policy support for quality improvement. Publication in a prestigious journal can generate press coverage that propels the communication and uptake of key messages from research. This pathway also may be used less visibly; an example is the work Ashe and her colleagues conducted on risk-based predictive modeling, which was honored with AcademyHealth's (2008) HSR Impact Award (AcademyHealth 2008).

“Big bang” research carries a great deal of weight and professional satisfaction. Sometimes, however, important findings may not match current policy interest. Also, publication in prestigious journals may delay the use of important research results because of restrictions on prior public disclosure. Realistically, most research will not have sufficient saliency for a “big bang.”

Gradual Accumulation and Diffusion

Researchers may strive for a “big bang,” but it is more common for research to contribute to a knowledge reservoir that accumulates over time and is available for policy use. In some cases, this accumulation of findings is converted into “common knowledge” through what has been termed the “enlightenment/percolation/limestone model” (Buxton and Hanney 1996). A good example involves the gradual build-up of understanding that health insurance coverage influences access to care and outcomes. The absence of formal processes to synthesize and identify “messages” from the pool of research means, however, that there is no quality control (Weiss 1979). (Formal synthesis could enhance “scientific vetting,” as in the Institute of Medicine's (IOM) (2001) synthesis of generally accepted findings on the impact of lack of insurance, which added to the credibility of the research.) Use of accumulated knowledge also is impeded when research is difficult to retrieve because findings are not published in archival sources or indexed in appropriate databases; web-based search engines may mitigate this. Older research also may be forgotten or viewed as dated, which can occur when support for specific research topics waxes and wanes, as with interest in the limit between Medicaid payment rates and physician access (Gold et al. 2006).

Formal Synthesis

Researchers can build on existing knowledge through formal synthesis of existing evidence in a particular area, such as knowledge of performance by health maintenance organizations (Luft 1981), or from two decades of health care research on organizational change (Bazzoli et al. 2004). Although exceptions exist,3 support for such synthesis tends to be limited in health services research, with funders preferring to support new work. Researcher-initiated syntheses run the risk of addressing questions of interest to other researchers, not policy makers, with exceptions (e.g., Soumerai et al. 1993). Syntheses pointing out where more research is needed can be important, but those drawing concrete conclusions, even if constrained, are likely to be valued more by policy makers.

Intermediaries Help to Communicate Research Messages

A key barrier to the use of research is the potential users' lack of awareness of a study or body of work and why it may be relevant. Intermediaries or processes can support better connections between the policy needs of users and findings from researchers.

Researchers as Communicators and Experts Consulted by Users

Through this pathway, researchers take steps to make it more likely that policy makers will know about and make use of research. Simple steps can enhance relevance, such as thinking carefully about the conclusions drawn from a study's findings and developing a clear message. Embedding the message in press advisories and other concise documents shared with potential intermediaries and users make it more likely that messages will reach key users. Such actions also allow researchers to help shape the “take away.” Some refer to this as satisfying the “elevator test,” using messages communicated between floors.

Such actions make policy makers aware of researchers' expertise. Ongoing relationships might develop, particularly when researchers actively cultivate them by being responsive or promote them through actions such as writing in mass media vehicles or speaking and spending time at user events. For example, in 2004, Uwe Reinhardt and Stuart Altman received AcademyHealth's Distinguished Investigator Award as recognized leaders in this form of translation (AcademyHealth 2008). This pathway draws on knowledge of a reservoir of research and knowledge reaching beyond particular studies. Effective performance as an active and credible research translator takes skills and judgment, however, particularly when policy questions do not neatly match a given body of research. Researchers often are not trained in such skills, and the same interests drawing them to research may limit their involvement in communication endeavors.

Formal Intermediary-Brokered Translation

In some instances, formal organizations exist to bridge the gap between the different needs and orientations of the research and policy worlds. At the federal level, examples are the Congressional Budget Office (CBO), Congressional Research Service, Government Accountability Office (GAO), and such specialized entities as the Medicare Payment Advisory Commission. Some, such as GAO, emphasize original research in direct response to congressional requests. Others build on the existing reservoir of research. For example, CBO is required to provide a fiscal estimate of the costs of each piece of legislation; it also analyzes potential legislative options and provides more general guidance that often determines which legislative policy options are feasible. With a staff dominated by economists, CBO draws heavily from existing research to execute its mission. Staff evaluate and synthesize relevant research findings and use research to support the development of cost models. Some states also have established independent policy centers, sometimes affiliated with universities or foundations, to carry out similar functions.

To policy makers, such intermediaries have advantages, notably their responsibility to provide rigorous analysis consistent with policy needs and timeframes. Led by researchers with national reputations, such organizations have the capability and independence to attract and use highly qualified researchers. The value of this pathway is constrained by agency mandates—these organizations exist to serve particular users and their policy concerns. Because reports are framed in policy terms, policy makers also may not appreciate how much these agencies depend on the reservoir of research. Such organizations also depend on people with skills that may differ substantially from common professional training. Researchers able to bridge this gap may be discouraged if their efforts are not appreciated or rewarded by peers.

Formal authority can enhance credibility, as can, less formally, good work that gains recognition. In the executive branch, there are many examples of intramural research or policy offices staffed by “honest brokers”; achieving credibility depends on the caliber of staff and openness of agency leadership to supporting rigorous analysis. Outside of government, groups such as the Kaiser Family Foundations' (KFF) Commission on Medicaid and the Uninsured, the Robert Wood Johnson Foundation's (RWJ) Council on Health Care Economics and Policy, and the Commonwealth Fund's Commission on a High Performance Health System perform similar functions. Credibility may be harder to achieve when such entities are affiliated with interest groups or private sector organizations.

The Mass Media as Intermediary

The mass media reaches opinion leaders and helps policy makers to become aware of research, which researchers can use to their advantage in publicizing research findings. Recently, some major media outlets arguably have become knowledge generators, as well as intermediaries, in the communication of research findings. Such media outlets now regularly sponsor surveys or other research analyses that address questions viewed as important to their readers, especially opinion leaders, and then report the findings publicly. For example, the Public Broadcasting System and ABC News have teamed up with the KFF/Harvard School of Public Health to study public opinion. Nationally focused papers, such as the New York Times or Wall Street Journal, sometimes develop long-term investigative reports, including primary analysis of such data as financial records, to increase public visibility of an issue. Media professionals know what is newsworthy and can effectively frame and communicate messages. Such outlets may not necessarily be committed to the standards typical of traditional research, however.

Users Seek to Enhance Value of Research

While the pathways already mentioned assume a fixed and generally researcher-determined reservoir of research for policy makers, there also are ways for users to influence the value of research.

Commissioning Synthesis around Policy Problems or Questions

This pathway involves active solicitation by users of research syntheses addressing important policy topics. Users define questions and timeframes; they may even specify the format for presentation and discussion of results to make these products relevant to policy makers. Work by the Canadian Foundation for Health Services Research and the RWJ Synthesis Project falls into this category (Clancy, Bilheimer, and Gagnon 2006), as does some work commissioned from the IOM to support expert review of a particular question. The value of user-commissioned work varies with how the question is framed, the underlying quality of available knowledge, and the timeliness and policy relevance of the synthesis.

User-Commissioned Studies

Organizations responsible for managing programs and executing policy, such as the Centers for Medicare and Medicaid Services (CMS), directly commission research on questions of interest. Buxton and Hanney (1996) term this the “problem solving/engineering model,” although it can be much more. Such work often is likely to be “work for hire” or conducted under contract. The researchers involved vary in their independence in defining methods, presenting findings, and drawing conclusions; how such issues are handled will influence the credibility of the work. Commissioned work is responsible for some of the most well-regarded research studies (e.g., the RAND national health insurance experiment) (Newhouse and the Insurance Experiment Group 1993) or the evaluation of Medicare HMOs (Brown et al. 1993). This pathway also can lead to concentration on studies narrowly defined by context- and time-specific questions. Findings may not always be released publicly, and publication may not be valued, leading to research findings that do not contribute as much to the cumulative base of knowledge available to support policy making and managerial decision making.

Users Provide Input into New Research

Some groups, such as the Canadian Foundation for Health Services Research (Lomas 2000), aim for a middle ground, modifying traditional assumptions of grant review while maintaining peer review. Both users and researchers review grant applications, with awards dependent on requiring minimum thresholds for both scientific merit and practical relevance. Ongoing relationships may be fostered between research organizations and users, as in AHRQ's ACTION program. Processes often are structured so that the researcher and user agree on the criteria for a useful study. Because of differences in culture and priorities between researchers and users, identifying research that satisfies both audiences may be challenging, particularly when data are limited. One concern is that collaboration will lead to the worst of all worlds, that is, research that is neither of good quality nor useful.

Researcher as User

On rare occasions, trained and experienced researchers are appointed to key policy or management positions and become users. Familiar with both research and policy worlds, they may initiate efforts that apply research to policy objectives and build supporting analytic capacity. For example, Mark McClellan's appointment to head CMS in the George W. Bush administration resulted in CMS providing more support to performance measures, public reporting, and effectiveness research. When she headed HHS's health reform initiative under President Carter, Karen Davis structured that process to incorporate the use of research. However, most researchers do not have the skills, interests, or political access to obtain such appointments. Once appointed, they also may face demands that generate conflicts between their roles as researcher and policy maker, leading to a loss of professional credibility or policy effectiveness.

The Relevance of Multiple Pathways

In the real world, pathways often do not exist solely in the forms characterized here; rather, a composite of pathways contributes to research use. A good example involves Congress' authorization of a change in Medicare physician payments allowing for resource use; this came about through a mix of one researcher's initiative (Dr. William Hsiao), contracted work to make original findings operational, and expert review by an intermediary (the Physician Payment Review Commission). An infrastructure that recognizes and better supports multiple pathways is likely to best enhance the uptake of health services research.

CONCLUSIONS

The process through which health services research is used in policy making is complex and multidimensional, including a fair amount of uncertainty not controlled by researchers. Perhaps because of its complexity, and their competing demands, researchers may treat this process as a “black box” that will absorb their work and make it relevant to policy. Such perspectives work in some circumstances, but not others. In this paper, we have sought to make more explicit the “black box” of activities and considerations that mediate between completion of research and its use. The analysis suggests a number of ways in which the utility of existing health services research can be enhanced.

Implications for Individual Researchers

Publication of research contributes to its ultimate use, but additional efforts by researchers can increase its potential. This analysis reinforces the distinction between research findings and message. Researchers can enhance the utility of their work by thinking as carefully about the meaning of the findings (the “message”) as they do about their methods. The analysis also notes the relevance of communication and use of intermediaries in reaching end users. Press advisories or other notices targeting key audiences may increase the reach and uptake of research substantially. Key audiences include intermediaries who influence what end users hear, as well as the end users themselves. Networking and trust make a difference in knowledge transfer, so researchers willing to engage in communications that go beyond publication are likely to find a greater payoff; expectations, however, should be modest, due to the variety of factors influencing policy decision making.

Implications for Research Users

This analysis highlights the fact that research may be relevant and valuable, even if not immediately useful. It also illustrates how uses of research may be important, but not always visible. The analysis of pathways between research and its use could help those commissioning research to think more carefully about the trade-offs between funding very focused research and permitting flexibility, within constraints, to encourage research that could be valuable in the future. Like researchers, policy makers also can benefit from understanding the distinction between research findings and policy messages. Building an infrastructure that supports better mining and synthesis of research can make the findings more relevant for applications unlikely to be addressed by single studies or that entail some ambiguity. To address this problem, one idea is a publicly accessible repository of research findings that will overcome delays or barriers to publication and make it easier for users to access available studies. But Internet search engines already exist and the potential for information overload is high. Thus, building bridges between individual studies and the knowledge they generate cumulatively about relevant policy concerns may be even more important than a repository in allocating resources. Existing intermediaries can make such links but often are pressed for time and resources. Enhancing available synthesis would help intermediaries take better advantage of cumulative knowledge. It also might expand the audience for research findings and develop linkages not well addressed in the current intermediary structure.

Implications for Those Seeking to Bridge Both Worlds

Analyzing pathways that link research to its use shows that the world does not divide itself neatly into “researchers,” who generate knowledge, and “users,” who apply it. Instead, there is a spectrum of often unexplored and intermediate activities and feedback loops that determine what messages, if any, develop from a study or body of work, and how these are communicated and used in the policy process. Organizations that play a leadership role in bridging both worlds, such as AcademyHealth (2006), can help researchers and users alike to better understand each other and create bridges to meet the needs of both audiences. Such organizations can help users to understand why delaying or limiting public disclosure may hurt their interests in the long term (AcademyHealth 2004); transparency and the public reporting of results is essential to a free flow of knowledge. Leadership also can help to address issues of mutual concern, such as balancing safeguards on research quality and independence with policy makers' strong interest in making research more relevant and timely. Leaders also might consider whether the current imbalance between clinical and health services research funding makes sense in an environment where rising health care costs raise continuing issues of how best to think about and encourage value (AcademyHealth 2006).

Putting U.S. Efforts in Context

The work cited in this paper suggests that some nations, such as Canada and the United Kingdom, have been more forthright and aggressive in building bridges between research and policy use than has been the case in the United States. The reasons are unclear, but the gap likely reflects the greater emphasis U.S. policy places on “muddling along” with limited government control over the free market of ideas, and limited public investment in an infrastructure to support more deliberative policy making. This paper may indicate ways to do better.

Supporting Information

Additional supporting information may be found in the online version of this article:

Appendix SA1: Author Matrix.

Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.