Evaluating Interventions Aimed at Promoting Information Utilization in Organizations and Systems

Abstract

This paper presents a theoretical and practical framework for evaluating interventions aimed at promoting information utilization in organizational decision-making. The framework integrates three distinct yet complementary theoretical perspectives on information utilization in politico-administrative systems: (1) the literature on evaluation utilization, (2) current thinking on knowledge transfer (KT) in the health policy field and (3) approaches derived from the analysis of lobbying in political science. Our analysis calls into question the relevance of effect analysis in evaluating organizational or policy-level KT initiatives, but also suggests dimensions that should be explored when evaluating KT initiatives. We conclude that there are probably trade-offs to be made between the effectiveness of KT and the scientific robustness and validity of the information transmitted.

The relationship between policy making and scientific knowledge has long been the subject of debate. Recent years have seen a resurgence of this debate in healthcare, expressed as a growing preoccupation with ensuring that scientific knowledge is taken into account in the development and implementation of health policy. Nevertheless, efforts to integrate knowledge into organizational decision-making and policy development also present a particular challenge for evaluation – specifically, identifying operational and conceptual tools that can be used to assess a particular intervention aimed at integrating knowledge into organizational or political decision-making.

Ferlie and Shortell (2001: 283) identify four levels as targets for system change: “the individual, the group or team, the overall organization, and the larger system or environment in which individual organizations are embedded.” At the individual and group levels, there are well-developed strands of work designed as evidence-based practice, quality improvement and implementation research (e.g., Doumit et al. 2007; Bonetti et al. 2005; Eccles et al. 2003, 2005; Grimshaw et al. 2006; Rycroft-Malone et al. 2002; Kitson et al. 1998; MacIntosh-Murray and Choo 2005). This literature focuses on the development and evaluation of interventions to ensure that professional practice aligns with best practices and evidence. Most of this work rests on statistical analysis, since it can rely on repeated quantitative measures of a well-defined output (the more or less easily measured change in clinicians' repeated clinical decisions), although qualitative approaches are also used (MacIntosh-Murray and Choo 2005), and psychological and social factors are taken into account.

Rather than cover the same ground, this paper targets broad organizational (management) and systemwide (governance) decision-making levels. In other words, our concern is the incorporation of evidence, not into individual or group practice, but rather into governance and management. At this level, these concerns have been explored from different angles. In the field of evaluation, there is a vast literature on evaluation utilization that has not only modified our ideas on utilization, but also contributed to the development of evaluation methods that promote utilization. Current thinking on knowledge transfer (KT) in the health policy field (e.g., CHSRF 2003, 2005; Lavis et al. 2003; Lomas 2005; Denis and Lomas 2003) has also identified approaches that promote knowledge utilization. Finally, a third approach, derived from the analysis of lobbying in political science, examines information utilization within the larger process of policy development. Each of these approaches sheds its own light on the phenomenon, but these literatures are relatively independent; to our knowledge, there have been few efforts at theoretical and conceptual integration. All three approaches focus on knowledge transfer interventions in a much murkier context than those targeted by implementation research, since the former target KT that can, at best, hope to influence a few one-shot decisions to an imprecise degree in an indeterminate time frame (Weiss 1977; Knott and Wildavsky 1980; Patton et al. 1977). This limitation prevents the use of statistical methods (unless one relies on self-declaration of perceived use, which we feel is inappropriate; see Knott and Wildavsky 1980). Moreover, the nature of the evidence itself will usually be much softer than the randomized controlled trials behind evidence-based practice.

The objective of this paper is to develop a framework that will enable the evaluation of interventions aimed at promoting information utilization in organizational and systemic decision-making. We propose a conceptual model of information utilization whose originality is based on the integration and articulation of the three theoretical perspectives on information utilization in politico-administrative systems described above. The integrative framework we propose is organized around three main themes. First, we analyze the concept of information by comparing different types of information utilization. Second, we examine information as it is perceived in the political arena. Third, we look at the process of information transmission. We consider this model to be original in that it demonstrates, at the levels we focus on, the weakness of the causal link between any given knowledge-dissemination intervention and actual knowledge utilization, calling into question the relevance of effect analysis in evaluating many KT initiatives. On the other hand, our model demonstrates the importance of the intervention process. Finally, we discuss the conceptual model's implications for the types of evaluation possible and the dimensions of utilization that should be considered when evaluating information utilization.

Types of Information

We posit that every decision is based on information, whether internalized or externalized. It is equally credible to suggest that not all information is of the same type, nor of equal value. One possible typology would distinguish information according to its scientific credibility or strength (Rycroft-Malone et al. 2002; Kitson et al. 1998). However, as we will argue, no convincing evidence exists that shows a link between scientific validity and utilization. As Knott and Wildavsky (1980: 545) point out, “excellent analysis is often ignored. Information is one, but only one, input into the bargaining process that yields policy decisions.” We thus rely rather on Peterson's (1995) typology of the kinds of information used in health policy development.

Peterson's model implicitly conceives the decision-making process as teleological and instrumental, in that decision-makers strive to maximize the positive and minimize the negative consequences of their decisions. Thus, for the decision-maker, the decision process implies an attempt to foresee the consequences of decisions, generally in a context of high uncertainty and ambiguity. According to Peterson, decision-makers are sensitive to two types of consequences or sources of uncertainty. On one side are “programmatic” consequences, related to the objectives or impacts of the policy or decision (Will the proposed means make it possible to achieve the objectives? What are the secondary impacts of the decision?). On the other side are the “political” consequences, those related to the impacts of the decision on the balance of political power, in the narrow sense of the term (Will this decision raise or reduce satisfaction levels among the electorate? Does it change the power relationships among political adversaries?).

For each of these two types of uncertainty, decision-makers will use three sources or types of information. First, there is the decision-maker's own experience, which includes interaction with his or her immediate circle, exposure to content circulated by the media, daily activities and past decision-making. The second is what Peterson calls distributional information, which comes from different organized groups or actors potentially affected or concerned by the decision. Finally, the third source of information – termed “analytical” – aims to be neutral and free of bias or subjectivity by basing itself on scientific methods. From this model we can create a matrix of six cells in which each of the three types of information is used to respond to each of the two types of uncertainty. We believe it is useful to consider how this typology might be combined with classic categories in the field of evaluation: conceptual, instrumental and symbolic utilization (Table 1).

TABLE 1.

Matrix of information utilization in decision-making

	Everyday experience	Distributional information	Analytical information
Programmatic uncertainty	Conceptual utilization		Instrumental utilization
Political uncertainty			Symbolic utilization

Ever since its earliest days as an academic specialty, the field of evaluation has explored the meaning and definitions of utilizing the information and knowledge that arise from an evaluation project (Weiss 1988). On observing that evaluation results seemed to be less utilized than might have been expected, evaluators questioned what would be adequate means of promoting evaluation utilization. This debate led to a broader reflection on evaluation utilization and resulted in the identification of different types of utilization (Champagne et al. 2004; Beyer and Trice 1982). The best-known categories – instrumental, symbolic and conceptual – have been taken up in the literature on knowledge transfer. The less familiar category – process utilization – is seldom used in KT literature but appears relevant to our focus here.

Instrumental utilization refers to the use of programmatic analytical information to make, influence or change a decision, policy or program (Patton 1988). Symbolic utilization occurs when analytical information produced by evaluations or other scientific sources also serves to legitimize predetermined positions. Conceptual utilization refers to the cognitive process of enlightenment, through which analytical information received by decision-makers is gradually integrated and assimilated, becoming part of their everyday experience base (Weiss 1977). Process utilization refers to the use of evaluation strategies in themselves as a tool to enhance the use of information (something we address in the next section of this paper).

Finally, as mentioned earlier, distributional information is characterized by the fact that it emanates from groups personally concerned by the information they are transmitting. It should be stressed that it is not the validity of the information in itself, but the presumed neutrality of the source that distinguishes analytical information from distributional information. Because the typology based on utilization of evaluation results does not address any competition that may exist among sources of information, it follows that this typology would not mention distributional information.

Information in the Political Arena

Another way of approaching utilization of scientific information in policy development is to conceive of it as arising at the junction of two fields with distinct properties. On one side is the field of science, the main source of information in this case, and on the other side is the politico-organizational arena, considered broadly, that is likely to put this information into practice. We will return later to the hypothesis that the information used may in fact be co-produced by the intersection of the fields, but first we believe it is useful to examine the specificities of each and the logic underlying their behaviours.

The meeting of the political and scientific arenas

Hannah Arendt defended the idea that the scientific and political arenas were based on fundamentally different principles. Whereas science functions using the ideal of truth, even if only for heuristic ends, in politics there are only opinions. “What is annoying is that factual truth, like all other truth, demands to be recognized and refuses to be discussed, whereas discussion is the essence itself of political life” (Arendt 1972: 307, authors' translation). A fundamental difference between truth and opinion is that truth, by its nature, is unique and unquestionable, while the nature of opinion is to be free and in perpetual evolution.

Yet another difference, which for Arendt is more fundamental, is that the forms of communication that deal with truth are never, in themselves, about action: they do not aim to bring about truth, but rather to describe it. In contrast, discourse dealing with opinion is, by nature, action-oriented and aimed at changing the course of events. According to Arendt, it is not possible to move from one discursive universe to the other without changing the nature of the content itself. “If the person speaking factual truth wishes to play a political role, and thus be persuasive, he will almost always take considerable detours to explain why his own truth best serves the interests of any group” (Arendt 1972: 318, authors' translation). When presented summarily, this proposition may seem radical, but it simply formalizes observations around which there is consensus in the field of political analysis.

This process poses a challenge for health specialists: groups pushing their own interests will stand up and oppose even the most unambiguous scientific findings. … From this perspective, health science constantly wrestles with self-interested politics. Even robust findings are only as good as the policy coalition that assembles around them. (Morone 2005: 13)

Setting the agenda

One obvious characteristic of information, on the other hand, is the conditional nature of the criteria for relevance. No information is relevant or interesting in itself. Information is relevant only in relation to a given context (Knott and Wildavsky 1980), whether this is conceptualized as a research question or as an administrative reform. Thus, administratively and politically, only information related to a subject on the agenda is relevant, regardless of any consideration of its value or scientific interest. In this context, often the most important factor in moving a dossier along, and likewise for promoting utilization of scientific knowledge, is for it to be on the agenda. To get a dossier on the agenda, it is obviously not only possible, but even desirable, to use as arguments specific information drawn from the scientific arena. Nevertheless, the nature of this information by no means determines the success in getting the dossier on the agenda. There are certain facts, extremely well documented and scientifically considered to be true, that have no impact in the political arena (e.g., that pollution due to automobiles is a major factor in urban morbidity and mortality). Conversely, other scientific facts become central and formative in the political arena, as demonstrated by the issue of tobacco control.

The Process of Information Transmission

Knowledge transfer and co-production

The view presented above on the interaction between the political and scientific fields is not always taken into account in decision-making circles and among granting agencies. The dominant view is what we will call here the school of knowledge transfer.

If the vision derived from Arendt's works is eminently political and structuralist, the KT perspective is functionalist and founded on an iterative empirical approach, rather than on any specific theoretical basis. According to this approach, the issue of the utilization of scientific knowledge in policy development and implementation is first and foremost an issue of communication between a source and a receiver (Ko et al. 2005). At the risk of overly simplifying, this means transmitting to the right person, at the right time, the right information presented in the right way. Given all the proper conditions, and if the information supplied is scientifically valid, sound and reliable, then utilization will occur and the ultimate outcome will be better policies. We will not present this model in detail, as it is likely already familiar to the reader. On the other hand, there is room for some interesting cross-analysis between the politico-structuralist and the KT schools.

One of the significant contributions of the KT movement has been the suggestion that the most relevant information, and therefore the most useful and usable, is a product of neither the politico-administrative nor the scientific arena, but rather a co-production made possible by the interaction of individuals in both. Involving the decision-maker at various stages of the knowledge production process creates a two-way interaction that promotes a co-building of knowledge based on teamwork and partnership (Lavis et al. 2003). This interaction would support the utilization of research or evaluation results (Gibbons et al. 1994; Denis and Lomas 2003; Lavis et al. 2003) and tallies with what is known about process utilization.

Likewise, the KT school proposes that for analytical information to be legitimized, it should be subject to deliberation. Such deliberation is justified on the basis that although scientific information may be reliable, it is also uncertain, dynamic, complex, disputable and rarely complete (CHSRF 2005). These characteristics would allow scientific information to be complemented by non-scientific data (expertise, viewpoints and realities of other actors) and for knowledge to be adapted to the context in which it would be used (Pirkis et al. 2006; Lavis et al. 2005; CHSRF 2005; Abelson et al. 2003). Still, for best results, the deliberation process should include balanced representation of scientists, users and groups involved (CHSRF 2005; Pirkis et al. 2006).

Lobbying

One possible approach to bringing the KT and the politico-structuralist schools closer together is by means of conclusions drawn from political science on the influence of lobbying on policy development.

We begin by presenting briefly one of the first analytical views of lobbying proposed by Milbrath in 1960 (Milbrath 1960, 1963; Koeppl 2001; Terry 2001). This model is of interest to our discussion insofar as it considers lobbying as essentially a process of communicating information (de Figueiredo 2002). This concept of lobbying allows us, in practice, to apply the model more broadly to understand how information and its communication are integrated into decision-making in a political context.

Anyone wishing to influence the decision of a governmental official, then, must be concerned not only with getting the information to him but also with the problem of presenting it so that the decision maker will be receptive. The only effective communications are those which get through the perceptual screen. In fact there is no other way to influence governmental decisions short of remaking the personalities of decision makers or replacing them with other persons. The lobbying process, then, is essentially a communication process, and the task of the lobbyist is to figure out how he can handle communications most effectively in order to get through to decision makers. (Milbrath 1960: 35)

This description of the nature of lobbying postulates that for information to be used, what is most important is not how valid or how sound it is, but rather its capacity to be communicated to the decision-maker. This is one point of compatibility between the politico-structuralist and KT schools. For Arendt (1972), the political and scientific arenas are too interdependent on many levels to be separated from each other. It is therefore both inevitable and desirable that there be cross-fertilization between the two. However, she insists it is analytically fundamental to understand that their logics differ, as do the nature and position accorded to information (Boaz and Pawson 2005). In the political arena, there are only, on the one hand, producers of performative discourse who are thereby political by nature, and on the other hand, information that, simply by existing in that arena, cannot be neutral. We could therefore say that to intervene in the political arena is to accept its rules and to accept being a lobbyist among others, in line with Milbrath (1960). Moreover, Quebec law provides a definition of lobbying that encompasses, at least potentially, the activities of knowledge transfer:

Any oral or written communication with a public office holder in an attempt to influence or that may reasonably be considered by the initiator of the communication as capable of influencing a decision … . (Quebec 2006)

It is worth noting here that Milbrath's proposed description of lobbying is not incompatible with the position of the KT school. In fact, we can conceive of a political system that works according to the Milbrath model and within which researchers would constitute a particular type of producer and disseminator of analytical information, along the lines of Peterson's model. In this process, the determinant of information utilization would still not be its scientific validity, but rather the effectiveness of the information-dissemination strategies. If Milbrath's model does not entirely reconcile the politico-structuralist vision with that of KT, it is nevertheless compatible with both and at least permits a common conceptualization of the phenomenon that would allow us to build hypotheses to check the robustness of each model against empirical practice.

Lobbying and power

At a second level, the communication process that is characteristic of lobbying is not simply aimed at transmitting neutral information. One of its central functions is to enable the communication of power relationships or, in other words, to sensitize decision-makers to the possible or probable consequences of their decisions. In the vocabulary of Clark and Wilson (1961), we could say that lobbying consists of convincing an individual that his or her action (or decision) is associated with positive or negative incentives. Thus, even if lobbying is limited to a communication process, this process is integrated into broader power relationships that must be considered in the analysis.

We therefore believe that there is an important distinction to be made in the lobbyist's level of control – or more generally, that of the organization being represented – over the consequences that are emphasized. On the one hand, a “performative” discourse around consequences is sometimes possible, in the sense that given a certain decision, the group affected can to some extent control the unfolding or the nature of the consequences. On the other hand, groups often must be satisfied with a “fatalistic” discourse on the consequences, recognizing that they do not control them but can only describe a logical sequence that is outside their control. A simple but clear illustration of this distinction can be found in the two sentences: “If you continue, I will push you and you will fall,” and “If you balance on the edge, you will ultimately fall.” In both cases, the speaker is trying to modify the behaviour of another person – an action that corresponds to the classic definition of the exercise of power (Crozier and Friedberg 1977) – by communicating to the other information on the consequences of his actions. However, while in the first formulation the speaker asserts his ability to control the consequences, in the second he presents the same consequences as being outside his control.

From this perspective, the results of relevant scientific studies, if brought to the decision-maker's attention, are identical in nature to lobbying efforts based on a fatalistic sensitization to consequences. Similarly, if we refer back to Peterson's typology, all scientific knowledge transfer, however neutral, involves communicating specific analytical information to the decision-maker and therefore implies a desire to influence the decision-making process by changing the weight attributed to a particular type of consequence. While the objective may be as well-intentioned and programmatic as possible, the process is nevertheless identical in nature to that of lobbying, with the effectiveness of the message dependent upon implementing strategies that will influence the decision-maker. The effectiveness of lobbying depends upon sensitizing the decision-maker to a specific and partisan group of consequences, while endowing these particular consequences with more importance than they actually have (Slovic 1966; Kahneman et al. 1982; Brunsson 1982), all within an integrated set of communication strategies.

In terms of desirability, the KT school is particularly compatible with lobbying theories and analyses. Actually, a fundamental characteristic of the lobbyist–decision-maker relationship is that of mutual benefit. Lobbyists are an important source of information for decision-makers, while decision-makers hold the decisional power coveted by lobbyists. Decision-makers benefit from lobbyists' ability to supply them rapidly and freely with important information. In exchange, the lobbyists, who are in a position to develop relationships of trust with decision-makers and to be regarded as dependable and predictable, gain an invaluable channel of communication. Many empirical studies have documented this symbiotic relationship and the structuring of exchanges it implies in terms of information circulation (Berry 1997; Heinz et al. 1993; Birnbaum 1993; Jordan 1991; Ainsworth and Sened 1993; Sousa 1998).

Similarly, the KT movement emphasizes that the development of interpersonal relationships among researchers and actors within decision-making circles offers advantages to both parties. On the one hand, it makes it possible to take into account the needs of the potential users of research and evaluation results and the specificities of their decisional contexts that could influence their choices and decisions (Cousins 2006). On the other hand, it allows researchers to deepen the utilization context and to prepare the way for development (Hanney 2003). Finally, collaborative research helps stimulate utilization of evaluation results via the proximity created between knowledge producers and users (Cousins 2006; Pirkis et al. 2006; Hanney 2003).

An effective communication process

The literature on lobbying has always stressed that the effectiveness of communication with the decision-maker is much more important in influencing the decision than is the validity of the information being communicated. This fundamental lesson – which is the A-B-C of lobbying – is interesting in that it corresponds exactly to the advice currently offered to researchers by the KT movement for increasing knowledge utilization. One version of this trend is to entrust to specialists – knowledge brokers (CHSRF 2003; Clark and Kelly 2005) – the role of interface between information producers and decision-makers. The structural position and role of these brokers makes them indisputably lobbyists according to the majority of current definitions, as well as under most laws governing the practice of lobbying (Quebec 2006). However, this specific view of KT is not very compatible with the notion of co-production of knowledge.

A second version of KT seeks to maximize the effectiveness of the communication process by offering advice and formulas to producers of scientific information on how to improve communication (Lavis et al. 2003; Dickinson 2004; CHSRF 2002). This perspective thus implicitly recognizes, as suggested by the analysis of lobbying, that the determinant of utilization is not necessarily the nature of the message but rather the process of communication. Thus, identifying a target for the message (Lavis et al. 2003), adapting the message to the selected decisional environment (Dickinson 2004) and formulating it in such a way as to reduce the cultural gap between the parties (Gülich 2003) are all integral components of the tool set that will make it possible, in Milbrath's terms, to pass through the decision-maker's perceptual filter. Likewise, the main messages of a study should be specified and presented synthetically and concisely (Lavis et al. 2003), and the message content should include relevance, interests, needs, objectives, concerns, contextual information and consequences of implementing the knowledge (CHSRF 2002, 2005; Dickinson 2004; Entwistle et al. 1998). The message should therefore be established and formulated in line with the needs of the decision-maker, something every good lobbyist has always known.

In addition, both the KT school and the literature on lobbying consider that certain personal traits of the lobbyist, or of the actor disseminating the knowledge, will influence the effectiveness of the communication process. The logical link is that these personal traits will affect the perceived legitimacy of the carrier and thereby also the perceived legitimacy of the current information, which ultimately will influence any eventual utilization. Thus, a long-term relationship with the decision-maker, a reputation for reliability, a certain level of celebrity and a recognized mastery of the issue are examples of traits currently considered to be generally positively associated with effectiveness (Heinz et al. 1993).

Implications for Applied Evaluation of Organizational KT

The starting point for the development of the framework presented here was our being invited to evaluate an innovative experiment in knowledge building and dissemination: the Research Collective on the Organization of Primary Care Services in Quebec (Pineault et al. 2006, 2007). Our first thought was to elaborate and measure a set indicators of effects and impacts. However, we soon found ourselves theoretically stranded by the logical complexity and uncertainty of the link between KT activities and their intended effects. We then decided to conduct a logic analysis of the Research Collective. Logic analysis is a way to analyze the theory of the intervention by comparing the program theory or the implemented intervention with scientific knowledge (Contandriopoulos et al. 2000; Brousselle et al. 2006, 2007). At that point, we started working on the integration of three fields that approach information utilization from different perspectives and use different concepts to understand the complexity of a phenomenon. Our unambiguous conclusion was that because of this complexity, there could definitely be cases where scientific information of good quality is transmitted according to the rules without any subsequent utilization. Information utilization depends not only on the transmission process, but also, and primarily, on the contextual dynamics of the political arena, over which the researcher has no control. In addition, it appears that neither the content, the nature nor the scientific quality of the information will have anything but a marginal impact on its utilization. Thus, there is only a weak causal link between information transmission and its utilization in the decision-making process.

This conclusion calls directly into question the relevance of effect analysis. Most effect analyses of KT at the macro level rely on questionnaire sampling of users' perceived use (e.g., Landry et al. 2003; Amara et al. 2004). This practice allows for large samples and quantitative data. However, the divergence between self-report and actual use can be significant. By way of illustration, consider physicians' opinion that their prescribing behaviour is unaffected by the pharmaceutical industry's marketing, notwithstanding abundant evidence to the contrary (Morgan et al. 2006; Steinman et al. 2001). In fact, our framework suggests there is only a very tenuous and questionable link between perceived and actual utilization of information, leading us to set perceived use aside as an unreliable and invalid indicator of utilization.

While our framework prompted us to reject effect analysis as a method to evaluate KT of the kind we dealt with, we nevertheless retained the idea of developing a practical and valid evaluation framework, in the firm conviction that it is possible to evaluate an intervention's potential for utilization. Integration of the three approaches presented here demonstrates that there are different dimensions supporting information utilization. We believe it is possible to evaluate information utilization by assessing dimensions that have an impact on the actual utilization of information, according to the theoretical model developed above.

Turning again to the evaluation of the utilization potential of specific information-transmission initiatives: if optimal conditions are brought together, we can conclude that the potential for information utilization is strong, but given the complexity involved, we cannot draw the conclusion that there is any real utilization. Evaluation helps in understanding whether the conditions were favourable for information utilization and whether the process was optimal or could have been planned differently to improve the potential for utilization. Paradoxically, we could legitimately suggest that potential is in no way a guarantee of real utilization. We could easily imagine an initiative with strong utilization potential but no actual utilization at all and, conversely, an initiative with weak utilization potential that, for whatever reason, results in an important utilization of information.

At the practical level, we evaluated the Research Collective's KT activities by analyzing the characteristics of the context, the traits of the information carrier, the characteristics of the process of information transfer and the externalities related to the utilization. For each of these dimensions, we identified specific characteristics and synthesized the theoretical effects that each of the strands of literature studied here predicted. Sometimes the predicted effects are convergent in all the literature and sometimes they are not, prompting some discussion. However, this approach allowed us to establish, with a strong theoretical basis, the potential information utilization of the Research Collective. The detailed evaluation is currently submitted for publication elsewhere (Brousselle et al. 2008).

Conclusion

The logic analysis we carried out here extends beyond the specific case of the Research Collective and leads to significant consequences: first, for understanding the role of research, and of the researcher, in the decision-making process; and second, for developing an evaluation project on information utilization.

Concerning the role of research and of researchers themselves in the decision-making process, we conclude, first, that information emanating from the research community is probably used only to limit the programmatic uncertainty faced by the decision-maker. However, this is not the only level of uncertainty, and the decision-maker will probably also take political factors into account. Then, in the political and organizational arenas, information coming out of research will always compete with other sources of information (regular, distributional and from other analytical sources). Scientific information will probably not be considered significant a priori. This conclusion has consequences at different levels.

First, scientific criteria for information quality (validity criteria) are concepts belonging to the scientific field, and their ability to be exported to the field of political decision-making is questionable. Thus, if it is important that there be mechanisms to maximize the chances that information communicated by the research community meets scientific standards of validity, these mechanisms belong to the field of science. Moreover, and we believe this is an important conclusion, it is questionable whether these criteria have an impact on information utilization.

Second, three major dimensions seem to influence utilization: the context of the political arena, the traits of the information carrier and the characteristics of the information transmission process. The only element over which the researcher has a certain amount of control is the transmission process. Our analysis demonstrates its importance for information utilization. Convergences among the literature on lobbying and knowledge transfer support the relevance and validity of these factors. Still, even if it is possible to ensure that the information transmission process is carefully thought out to maximize utilization, there are no guarantees the information will be used. The quality of the process is in some way a necessary, but not sufficient, condition for information utilization.

Third, the convergence between political theory and knowledge transfer means that increasing the consideration of scientific information in decision-making requires changing the knowledge dissemination process to make it similar to more general lobbying efforts. Our line of argument shows, in fact, that the nature of knowledge transfer is related to a specific type of lobbying. Thus, if there are good reasons to believe that incorporating knowledge more fully into decision-making is desirable, then researchers should be able to orient their actions more broadly in such a way as to influence the decision-making process – which inevitably draws the researcher out of the scientific arena and more deeply into the dynamics of the political arena, where opinions must be asserted.

Finally, it is important to note the potential for tension between conditions that support effective communication processes in the political arena and demands for validity in the scientific arena. Thus, concessions may be required on the scientific validity front for the sake of political effectiveness. In this context, researchers may need to find a compromise between effective information transmission and adherence to scientific criteria. However, we cannot postulate a priori that compromises in scientific validity have a negative impact on the quality of the decision or of the political process.