Informing Policy Making and Management in Healthcare: The Place for Synthesis

Abstract

Research synthesis has an important role supporting the transfer of knowledge between researchers and healthcare decision-makers. But if our goal is to make knowledge more useable and context specific, then extending the scope of systematic reviews or producing syntheses with policy makers and managers may be insufficient. Dialogues, partnerships and reinterpretations of evidence in context will help us achieve this goal.

Lomas (2005) and Lavis et al. (2005) provide a timely contribution to debates about how to make research evidence available and useful to healthcare managers and policy makers. Both papers argue that the relatively well-developed methods for systematic reviewing – used, for example, within the Cochrane Collaboration – do not address the “broader contextual factors of the managers’ and policy makers’ world” (Lomas 2005: 59). Lavis et al. suggest that managers and policy makers ask more complex questions that go beyond “What works?” (i.e., questions of effectiveness) and include questions about relationships, mechanisms and meaning. Lavis et al. call for a better repository of knowledge, in essence adapting and extending the Cochrane Library’s systematic reviews, to make it more suited to the needs of policy makers and managers. Lomas adds a further twist to this argument by suggesting that the synthesis of research knowledge for policy makers and managers requires different kinds of input (notably, from interpretive social science) and a closer partnership between researchers and managers/policy makers.

So, What’s Wrong with Systematic Reviews?

Before we rush into a program of synthesis, it is worth pointing out that many synthesis methods are emerging – the techniques (how to do it) and definitions (what it is) are being developed (Mays et al. 2005; Dixon-Woods et al. 2005). Lomas differentiates summative and interpretive approaches to synthesis. He argues that summative approaches centre on questions of effectiveness, while interpretive approaches are more closely allied to the needs of policy makers and managers. Unfortunately, this implies that policy makers and managers do not need summative accounts. We would suggest that effectiveness reviews may be a necessary, but not sufficient, aid to policy makers and management (they still need to know “what works”). Moreover, there are examples of Cochrane-style systematic reviews aimed at exactly the broad, complex policy-type questions that both Lomas and Lavis et al. identify (Garcia et al. 2002; Harden et al. 2004). There is a danger, in overstating the distinction between summative and interpretive approaches, that we ignore the usefulness of particular types of review (i.e., systematic reviews of effectiveness) and the presence of interpretive work within existing systematic reviews.

For us a more important distinction, inspired by the pioneering work of Jack Dowie (2001, 2002), is the difference between reviews or syntheses providing knowledge support and those providing decision support. Lavis et al. focus on the problem of providing knowledge support. They argue that methods used by the Cochrane Collaboration need to be extended to make reviews more generalizable in order to answer the kinds of questions that policy makers and managers ask. We would further argue that a range of types of review and interpretive and summative syntheses (such as narrative synthesis, meta-ethnography and cross-case analysis, described in Mays et al. 2005) have the potential to provide the kinds of knowledge support that Lavis et al. recommend.

The decision-support approach is distinct from knowledge support because it seeks to go beyond research synthesis and to take on some of the tasks entailed in the decision-making process, for example, incorporating weightings that represent values or judgments. Summative and interpretive approaches might have a place early in this process, as in a literature review of qualitative and quantitative studies to make the inferences that inform a Bayesian analysis. In an example of this approach, findings from qualitative research about parents’ reasons for having their children immunized (or not) were used to inform a statistical analysis of the factors that influenced this behaviour (Roberts et al. 2002). For policy makers and managers, this approach provided a more comprehensive picture of the potentially important factors than would have been available if only effectiveness data had been included.

Partnerships

Both Lomas (2005) and Lavis et al. (2005) make compelling arguments for moving away from researcher-driven knowledge translation towards co-production of knowledge and a partnership between managers/policy makers and researchers. At present, policy makers and managers are accused of failing to utilize research evidence. This charge gives rise to an impression that they exist in an evidence-free vacuum. Clearly, managers and policy makers do use evidence. What they don’t do so often is use the particularized kinds of evidence that some researchers recognize (e.g., systematic reviews). As Burns points out (2005: 53), these decision-makers have little tradition of using the library. But they utilize intelligence from a variety of sources, including formal research evidence, albeit in an adapted form. One of the reasons for this is that researchers often do not provide evidence that is timely and accessible to policy makers and managers (Popay in press).

Some of the current difficulties in getting knowledge into policy and management practice relate to presentation. At a very basic level, there is a strong case for “jargon-busting” – avoiding discipline- or profession-specific terminology and acronyms, and encouraging the use of “plain English” or French (e.g., the Plain English Campaign) to get our messages across. Lavis et al. champion the 1:3:25 report format pioneered by the Canadian Health Services Research Foundation (CHSRF). This format has proved useful, but there is a danger in becoming over-prescriptive and assuming that standardized formats are a quick fix for knowledge translation. At the heart of Lomas’s paper is a plea for closer relationships between policy making and management. Reports in 1:3:25 format, or a larger Cochrane Library with policy-relevant add-on reviews, will not deliver this. We need to think about making the dialogue between researchers and policy makers/managers work (Elliott and Popay 2000). Inevitably, this dialogue will consider making and re-making partnerships in local contexts. One way might be the process adopted by the CHSRF Policy Synthesis Program (CHSRF 2000) whereby researchers and policy makers/managers meet to discuss the content and format of reviews and syntheses. These kinds of critical conversations help to establish what it is that both sides want from the partnership.

There are emerging methods for synthesis that can contribute to the dialogue between research and policy making and management. Synthesis can promote knowledge transfer, but it is not simply an advance on other kinds of literature reviewing; rather, it is a key aspect of this broader activity. Some synthesis approaches allow the inclusion of forms of evidence, such as qualitative research, which have previously been considered too small-scale or too contextualized to inform policy making or management. Others have the potential to become mixed-method approaches, enabling the inclusion of evidence from qualitative and quantitative research and from non-research sources.

Decision support requires a different kind of engagement. It is likely that any synthesis or review would require serious adaptation to meet the demands of decision-makers. It may not be possible to use existing reviews or syntheses for this purpose. In many ways, decision support requires an even closer partnership between research and policy making or management.

Challenges

There are issues that neither Lomas (2005) nor Lavis et al. (2005) address about who should engage in this business of knowledge translation. We need to recognize the different skills required for different approaches – summative or interpretive, knowledge or decision support. The development of transparent, formalized methods for systematic reviewing has enabled researchers from a variety of backgrounds (clinical/non-clinical, research/informatics) to undertake such reviews. Contemporary work developing methods for synthesis suggests that these approaches may require discipline- or methodologically specific expertise (e.g., work on meta-ethnography has highlighted the need for expertise in qualitative methods).

Decision support is quite different from reviewing or synthesis and, again, requires appropriate skills. Engagement with decision-making processes is likely to require input from a team, extending beyond a partnership between researchers and policy makers/managers to include other types of decision-makers, stakeholders and experts.

The Way Ahead

At the heart of the debate about informing policy making and management in healthcare is a paradox: the more we attempt to make knowledge useable and context specific, the more difficult it becomes to draw on a repertoire of reviews or a stock of knowledge. Both Lomas (2005) and Lavis et al. (2005) emphasize that, in the business of policy making and management, context matters. What they are both arguing for in their different ways – Lavis et al. with their extended version of the Cochrane Collaboration and Lomas with his call for co-produced research synthesis – is for knowledge that is relevant. Ultimately, there may well be a place for new forms of research synthesis, as well as for systematic reviews, in informing management and policy, but local partnerships, critical dialogues and reinterpretation in context will be what make a difference in the world of healthcare management and policy making.