The Research Collective: A Model for Developing Timely, Contextually Relevant and Dynamic Approaches to Research Synthesis?

Abstract

In this issue, Pineault and colleagues (2006) report on a Quebec-based research collective, a process aimed at rapidly synthesizing results from ongoing or recently completed research and releasing them to decision-makers.

The process outlined differs markedly from Cochrane-based formal systematic reviews, and the authors claim a number of benefits that make the results more relevant to decision-makers, including enhanced timeliness of results, increased triangulation among emerging research findings, and more in-depth understanding of the impact of contextual environments on research results.

This approach may offer opportunities to both advance and enrich existing synthesis tools. The research community should learn more from this process.

Following on a series of articles and commentaries that have featured prominently in these pages about the role and nature of research synthesis, this issue of Healthcare Policy features an article by Pineault and colleagues describing a dynamic and interactive approach to the synthesis of ongoing and recently completed research. The authors report on a Quebec-based research collective in the area of primary healthcare (PHC) in which a lead team of investigators worked with researchers to synthesize 30 ongoing or recently completed studies to produce “timely, context-linked research syntheses” within a summary report. The synthesis process focused on determining how, and to what extent, different modes of organizing primary care services can affect service delivery, and on understanding change processes that would facilitate the implementation of effective primary care practices in different contexts. The process involved active participation by researchers, development of a conceptual/analytic model, consultation with decision-makers and development of a report that highlighted seven key messages targeted at the decision-making community (Pineault et al. 2005).

What was different about this process? The focus was on synthesizing results from locally conducted research that was either underway or had recently been completed but was unpublished, and the aim was to release results rapidly. The process involved active and ongoing participation of local researchers; it also involved participation of decision-makers, although this aspect is less well described. Finally, the process sought to link research to the context in which it was conducted, to involve researchers in considering their findings in light of other research, and to work collectively to develop a more comprehensive and contextually based understanding of their individual work within a broader framework.

This process stands in stark contrast to the Cochrane approach of developing formal, systematic reviews of published research (see, for example, Lavis et al. 2006). First, there was no specific question, no explicit definition of a search strategy and no explicit statement about the types of research evidence that were to be included and excluded. Instead, the focus was broadly defined in terms of content area alone: researchers (presumably within Quebec) whose work focused on the organization of healthcare services were identified and invited to submit information about research projects that were related to the organization of PHC services. More than 90 projects were identified; of these, 30 projects satisfied a set of criteria and were selected for the synthesis. Based on review of the eligible studies, investigators developed an analytic framework and worked with researchers and decision-makers to synthesize results. The resulting synthesis included qualitative and quantitative studies and supported consideration of a broad range of questions and issues.

Another significant contrast with the Cochrane review process related to the lack of formal assessment of scientific quality of the studies. While a process was developed to seek researchers’ assessment of the validity of their own studies (and many were fairly self-critical), there was no validation of their interpretations. Finally, in contrast to Cochrane processes, there was not a transparent process of interpretation of the findings of studies in the review. Instead, investigators and researchers worked collaboratively to develop the project description form; projects were presented at a seminar attended by decision-makers; and revision of project descriptions was undertaken by researchers in response to feedback and understanding gained from familiarity with other projects. From this point, the lead investigators classified projects, produced a first draft of a synthesis report, and sent it to researchers for validation. Researchers’ suggestions and corrections were incorporated into the final version of the report. The process, therefore, was unique in a number of ways: it focused on reviewing unpublished research and producing timely dissemination of results; it adapted the synthesis to maximize local applicability; it involved active partnerships with researchers, and to some extent with decision-makers; and it used a flexible and evolving approach both to frame the questions and content areas to be addressed, as well as to conduct the overall synthesis. The result is a textured set of messages targeted at decision-makers.

Pineault et al. suggest a number of benefits to the process they undertook, including timeliness of results; the relatively low cost of producing synthesis results relative to the cost of producing the primary research studies; opportunities for exchanges among researchers, which led to familiarity with other projects and greater perspective on theoretical, methodological and applied aspects of the work; and support for more in-depth analysis of the contextual environment in which studies were conducted. The authors claim that these aspects make the synthesis more relevant to decision-makers. They also identify a number of limitations of the approach, including potential for bias related to the interaction between researchers and investigators, the risk of disseminating preliminary and invalid findings and the limited generalizability of results.

What are the lessons from this process, and what more do we need to know about it? Lomas (2005) has suggested that summing up research evidence is more than a checklist exercise and requires interpretation, largely by researchers; he cites this research collective as an example of a richer process. On review, this process appears to be investigator-driven, with substantial input from the research community. The research collective model may therefore provide opportunities for more intensive and collaborative involvement of researchers in developing knowledge translation skills and products. In evaluation feedback, researchers involved in the collaborative indicated that they “appreciated the exercise” and valued publication of their work in the synthesis report as well as their interaction with decision-makers. But questions remain about how this process served participating researchers. Was the research collective simply a one-time event, or has it resulted in richer and longer-lasting collaborations? How might the process be applied to other issues and communities of researchers? Is there a will among the research community to extend the research collective process? Could it serve as a model to develop communities of practice (Wenger 1998; Wenger et al. 2002) more clearly oriented towards influencing decision-making? To answer these questions, we will need to understand the long-term effects of this process on the research community.

Other questions relate to involvement of the research collective with the decision-making community. The authors indicate that the process produced results more relevant to decision-makers, but they also suggest that in the future, decision-maker participation could be enhanced. Lavis and colleagues (2006) argue that healthcare managers and policy makers need answers to different questions than those typically addressed in systematic reviews – how and why interventions work, and how to fit interventions into complex healthcare systems, for example. Others make compelling arguments “for moving away from researcher-driven knowledge translation towards co-production of knowledge and a partnership between managers/policy makers and researchers,” and argue that this dialogue should be based on critical conversations to establish the content and format of reviews and syntheses in local contexts (Pope et al. 2006). They suggest that “there may well be a place for new forms of research synthesis, as well as for systematic reviews, in informing management and policy, but local partnerships, critical dialogues and reinterpretation in context will be what make a difference in the world of healthcare management and policy making.” The research collective advanced such interactions, and may provide a model for making the synthesis process more relevant to decision-makers. However, the description of the process suggests that even though decision-makers were involved, their involvement may have been somewhat limited. At this point we do not know if and how the decision-making community has used the synthesis results and whether there has been continued involvement, partnership or evolution towards a partnered research collective.

What have we learned, and what more do we need to know about this process? Many have argued for advancing and improving synthesis approaches, but there are debates about how best to do this (Lomas 2005, 2006; Greenhalgh and Russell 2006; Lavis 2006; Lavis et al. 2006; Pope et al. 2006; Roger 2006). Funding agencies interested in knowledge translation should take note of the research collective and encourage the research community to learn more from this and other innovative examples.