Abstract
Background
Public health interventions are complex by nature, and their evaluation requires unpacking their intervention logic and their interactions with open social systems. By focusing on the interrelationships between context, mechanism, and outcome, Pawson and Tilley’s realist approach appears a promising innovation for public health-related evaluation works. However, and as expected of any methodological innovation, this approach is being constructed gradually by answering the multiple challenges to its operationalization that fall in its path. One of these challenges, users of this approach agree on, is the necessity of clarifying its key concept of mechanism.
Method
We first collected the definitions of mechanism from published works of Pawson and colleagues. Secondly, a scoping review was conducted to identify the ones quoted by users of the realist approach for evaluating public health interventions (1997–2012). We then appraised the clarity and precision of this concept against the three dimensions defined by Daigneault and Jacobs “term, sense and referent.”
Results
Of the 2344 documents identified in the scoping review, 49 documents were included. Term: Users of the realist approach use adjectives qualifying the term mechanism that were not specifically endorsed by Pawson and colleagues. Sense: None of the attributes stated by Pawson and colleagues has been listed in all of the documents analyzed, and some contributions clarified its attributes. Referent: The concept of mechanism within a realist approach can be ascribed to theory-based evaluation, complex social interventions, and critical realism.
Conclusion
This review led us to reconsider the concept of mechanism within the realist approach by confronting the theoretical stance of its proponents to the practical one of its users. This resulted in a clearer, more precise definition of the concept of mechanism which may in turn trigger further improvements in the way the realist approach is applied in evaluative practice in public health and potentially beyond. A mechanism is hidden but real, is an element of reasoning and reactions of agents in regard to the resources available in a given context to bring about changes through the implementation of an intervention, and evolves within an open space-time and social system of relationships.
Electronic supplementary material
The online version of this article (doi:10.1186/s13012-015-0345-7) contains supplementary material, which is available to authorized users.
Keywords: Mechanism, Realist approach, Evaluation, Realism, Conceptual analysis, Public health intervention, Complexity, Context
Background
Researchers working in the field of public health interventions evaluation suggest embracing their complex nature [1, 2] and their dynamic within an open system [3]. It is particularly important to understand how the intervention relates to both the individuals involved and the context in which it is implemented with the overarching aim of improving population health and reducing social inequalities in health [4]. Since the 1980s, different approaches arising out of the theory-based evaluation perspective have suggested theorizing the logic of intervention and taking into account the mechanisms through which the intervention produces its outcomes [5–8]. The realist approach, proposed in 1997 by the sociologists Pawson and Tilley [9], is increasingly attracting attention from researchers in evaluation [10, 11]. Evaluators are called to do more than just demonstrate whether the intervention works, to produce evidence on the way it works (or fails to work) in its reality, on how it produces outcomes, among whom, in what circumstances [12]. Acknowledging the richness of the context, the center of gravity for evaluation research is thus shifted away from causal intervention-outcome interplays to Context-Mechanism-Outcome configuration (CMOc) [13].
If this new approach to evaluation may at first seem innovative in public health, no consensus has emerged yet among its users with regard to the concept of mechanism, even though this is at the heart of its use [14]. Astbury and Leeuw confer three fundamental aspects upon the concept of mechanism “in line with the ‘realist’ principles: Mechanisms (1) are usually hidden, (2) are sensitive to variations in context, and (3) generate outcomes” [15] (p. 368). However, these attributes do not seem sufficiently precise for a unanimous definition of the concept within the Pawson and Tilley’s realist approach. Moreover, two systematic reviews have highlighted all the ambiguity of the concept of mechanism in its comprehension and in its use for evaluating public health interventions [16, 17]. According to these reviews, some researchers encountered difficulties in differentiating mechanisms from contextual factors, activities, or specific resources to the intervention, thus raising the issue of structuring the CMO configurations. Through what Marchal and colleagues call the CMO dilemma [18], the question of clarity of the concept of mechanism also arises. A recent article outlines the delicate exercise of the clarification of this concept in realistic evaluation using a social science illustration [19]. Through our scoping review, we have tried to do the same exercise from the practices of the realist evaluation in the field of public health. Our work has three main objectives to propose a clearer and more precise definition of the concept of mechanism: (1) trace how the concept of mechanism is defined in the writings of Pawson and his colleagues, (2) describe how this concept is defined and operationalized in the contribution of the users of the realist approach in the public health field, and (3) explain what are the differences between mechanism, intervention, and context.
Methods
To define a concept, Daigneault and Jacobs’ conceptual framework confers upon it three dimensions [20] (i.e., Fig. 1): (1) the concept is designated by a term (e.g., the concept of capital). A term may have several meanings (homonymy, e.g., capital as a city or as wealth) and several terms may have the same meaning (synonymy, e.g., capital and asset as resources); (2) the concept carries sense, which is to say that it expresses an idea (e.g., capital refers to financial resources available for use). The sense of the concept relates to the set of necessary and sufficient attributes included in this concept, an attribute being one characteristic of the concept; (3) the concept refers to a class of objects in the real world, which is designated by the referent. Therefore, to ensure the clarity of a concept, a term must only have a single sense within a given referent. Moreover, a concept is that much more precise when the empirical phenomena to which it applies are clearly distinct from those to which it does not apply (e.g., the concept of capital from the Marx’s economic theory or from the theory of human capital).
Research strategies
We first collected the definitions of mechanism from the published works of Pawson and colleagues, from 1997 the year in which Pawson and Tilley’s Realistic Evaluation was published to 2012. This allowed us to unearth the terms and fundamental attributes characterizing this concept as well as the referents used.
Secondly, the definitions of mechanism by users of the realist approach were collected using the scoping review method. This exploratory strategy was preferred as it allows for a systematic examination of all documentation available on the concept of mechanism (i.e., scientific and gray literature) and the identification of the main gaps in the existing documentation [21, 22]. The PRISMA diagram was used to guide the scoping review process [23].
To be selected, documents had to (1) address the concept of mechanism by referring to the realist approach in evaluation, (2) deal with one (or more) public health intervention(s), (3) be written in English or French, and (4) be published between January 1997 and June 2012.
Bibliographical databases were also searched (i.e., Medline, Academic Search Complete, Eric, SAGE Journals Online, BDSP, Cairn info, and ScienceDirect). The choice of key words was adjusted according to the different thesaurus (“realist* review” OR “realist* synthes*” OR “realist* approach” OR “realist* evaluation” OR “realist* case stud*” AND “mechanism*”). We also searched the reference lists of all the articles meeting our inclusion criteria looking for other key documents (especially in the gray literature) that may have eluded our general search strategy.
The data analysis entailed (1) demonstrating how the writings collected contribute to the three dimensions of the concept of mechanism (“term, sense and referent”) as defined by Pawson and his colleagues and (2) identifying the differences in understanding and use of this concept among users of the realist approach to shed light on the conceptual gaps in the definition.
Results
For the first step of our work, 7 references of Pawson and colleagues have been included [9, 12, 24–28] (i.e., Additional file 1). Secondly, the search for papers within the scoping review yielded 2344 references; 1460 of these were selected on the basis of their titles and abstracts. 96 documents were fully searched. 49 met the criteria for inclusion and were selected for analysis (i.e., Fig. 2).
In light of the papers collected [18, 29-76], Pawson and Tilley’s realist approach appears to be thriving. This is regardless of the level of intervention addressed (mainly that of program but also practice or policy). The approach was also applied to most areas in public health, from health system management [29, 30] to prevention and health promotion [31, 32], healthcare services [33, 34], and participative research [35, 36]. In what follows, we look at the three dimensions of the concept of mechanism (i.e., Additional file 2).
The term of the concept of mechanism
Pawson and colleagues use the term mechanism attached to its object, “programme” or accompanied by qualifying adjectives such as “underlying,” “explanatory,” “causal,” “change,” “generative,” “intended,” “dominant,” “particular,” and “social” [9, 12, 24, 27]. Synonyms of the term mechanism are also used by the authors such as “power,” “theory,” and “force” and “trigger”.
Contrary to the synonyms of the term mechanism (e.g., “theory” [37], “force” [38], “triggers” [39]), users of the realist approach resort to adjectives not specifically endorsed by Pawson and colleagues. It should be noted that in addition to the adjectives used by Pawson and his colleagues, those chosen by users bring us back to the position of a mechanism through the different levels of the intervention contexts (e.g., “individual,” “collective,” “organizational,” “external” [40]), the nature of a mechanism (e.g., “behavioral” [40], “cognitive” [40], “emotional,” “structural” [41], “social” [42]), varying among the subjects (e.g., “participants,” “staff,” or “policymakers” mechanisms) and the relationship between them (e.g., “mechanism of,” “mechanism between”), the abstraction level of a mechanism (e.g., “causal,” “generative,” or “observed” mechanism), its emergence (e.g., “emerging” [43]), its behavior for change (e.g., “enabling,” “disabling” [44], “positive” [45], “negative” [46]), its interactivity (e.g., “interactive,” “interacting”[43]), its categorization (e.g., “primary,” “secondary,” “submechanism”), its purpose (e.g., “for change” [47] or “of change” [48]) as well as to the different components (e.g., “action,” “project,” or “program” mechanisms) and steps of the intervention to which it is attached (e.g., “implementation mechanism” [35] or “mechanism of implementability”).
The sense of the concept of mechanism
In 1997, Pawson and Tilley conferred three fundamental attributes to the mechanism of a program [9]. It (1) reflects the embeddedness of the program within the stratified nature of social reality, (2) takes the form of propositions which will provide an account of how both macro and micro processes constitute the program, and (3) demonstrates how program outputs follow from the stakeholders’ choices (reasoning) and their capacity (resources) to put these into practice (p. 66). According to them, a mechanism is not a variable of causality stricto sensu but rather a theory in the sense of a logic “which spells out the potential of human resources and reasoning” (p. 68). In 2002, Pawson refined a mechanism as the subjects’ interpretation of the intervention stratagem [24]. It is not the intervention that works but the resources they offer to enable their subjects to make them work (p. 342). In this way, the intervention will work “if those subjects are persuaded to accept, install, maintain and act upon it” (p. 344). In 2004, Pawson and Tilley no longer spoke of intervention stratagem but rather of the logic of an intervention, declaring that these are the mechanisms which explain by retracing the destiny of a program theory. They also complete their definition by specifying that a mechanism is usually hidden, sensitive to variations in context, and produces effects [12]. In similar or distinct contexts, a mechanism can produce outcomes that are identical or different. In 2006, Pawson [25] added that mechanisms describe “the powers inherent in a system, be those (…) agents or structures” (p. 23), namely the “choices under the inducement of programme resources” (p. 24) to make change happen. Pawson and colleagues (2011) specified that interventions work through the reasoning and reactions (i.e., mechanisms) of its subjects [27]. In other words, mechanisms “capture the many different ways in which the resources on offer may impinge on the stakeholders’ reasoning” [28] (p. 187).
Each of these attributes has been picked up by one (or more) user(s) of the realist approach, but none of these has been listed in all of the documents analyzed. Moreover, some attributes have been deepened in the writings of users of the realist approach.
Firstly, as Pawson and colleagues, they define mechanisms (1) as reasoning (e.g., ideas) of (a) human agent(s) and their choices about how the change will be achieved through an intervention [48] and (2) as individual or collective reactions of agent(s) to the resources provided by the intervention that trigger change [49]. For instance, Robert focuses on the conviction or reticence of the staff and the satisfaction of users (i.e., mechanisms) when the exemption programs of direct care payments have been implemented in different African countries [40].
Secondly, mechanisms (i.e., reasoning and reactions of human agents) can evolve in a range of circumstances at different times [44] and according to problematic situations on which to intervene [41]. During the intervention process, including implementation, some existing strategies can die or change direction and new ones can emerge [50]. Furthermore, some users specify also that a mechanism is latent and reveals itself in the implementation of the intervention [51]. It illustrates thus the history of an intervention before its implementation. So, when an intervention is designed and implemented in a given context, mechanisms may not be triggered intentionally by the intervention designers [47].
Thirdly, even though according to Pawson and Tilley a mechanism is generally hidden, users insist on specifying that, though not directly observable, the mechanism is real [52] and exists activated or not. Although they cannot be measured directly “because they happen in people’s heads” [53] (p. 92), mechanisms, once activated in a specific context, can be identified and measured through their undesired or desired outcomes [34], making explicit one (or more) CMO configuration(s).
Finally, some users have emphasized that mechanisms are interactive with one another, with the context [54], or with the outcomes they produce [45]. These interactions can lead to positive or negative feedback loops (e.g., after negotiation between stakeholders, interference with other interventions), which may or may not lead to the success of the intervention resulting in a change or not [45, 54].
Mechanisms described by users of the realist approach for evaluating interventions in public health are mainly linked to the participation, collaboration, partnership, or management processes of/between subjects involved in the implemented intervention (e.g., stakeholders, policymakers, health workers, patients) in order to improve (or change) behaviors, practices, programs, policies, or performance of an organization (e.g., the access to quality care) in a given context. The mechanisms (i.e., reasoning and reactions of human agents) are diverse but recurring, such as “promoting reflection” [53], “taking control” [43], “increasing motivation, interest and satisfaction” [32, 53, 55], “building and increasing confidence” [43, 53, 55], “promoting mutual support” [53], “gaining acceptance of new information” [47], “creating a sense of belonging and respect” [47], or “subjects empowerment” [32].
The referent of the concept of mechanism
The realist approach can be ascribed to the theory-based evaluation, complex social interventions, and critical realism.
Theory-based evaluation
Firstly, the works of Pawson and colleagues arise from the proponents of theory-based evaluation approach such as Chen and Rossi [6, 7] and Weiss [5] in particular. This approach stresses the importance of making explicit the logic of the intervention (or program theory) and to clearly distinguish it from the logic of the intervention implementation. The program theory (or logic of intervention) is defined as the set of hypotheses that explains how and why the intervention is expected to produce its effects [17]. With the realist approach, a special attention is paid to the collective or individual reasoning and reactions of human agents (i.e., mechanisms) depending on the resources available which allow or hamper the change in a specific context [41]. Declined as CMO configurations constructed as part of a realist synthesis or realist evaluation, the program theory identifies and describes how mechanisms (e.g., actors’ ideas or choices) led to the outcomes for change in a given context [30, 48]. For evaluators, understanding of how the actors generate the outcomes for change in regard to the available resources is essential for a proper understanding of mechanisms searched for by realist evaluation or realist synthesis [41].
Complex social interventions
Some users of the realist approach in evaluation [17, 33, 40] have described the several characteristics of a complex social intervention stated by Pawson [26, 27]. These interventions are long sequences of theories (i.e., a long chain of decision-making processes involving sequences of mechanisms as reasoning and reactions of human agents) in the sense of logics of intervention. As illustrated by Pawson [25], intervention theories “begin in the heads of policy architects, pass into the hands of practitioners and, sometimes, into the hearts and minds of subjects” (p. 28). As the product of the several layers of its context and involving the participation of numerous stakeholders (that are rooted in different localities, institutions, cultures, and histories [27]), these interventions are embedded in multiple social systems (i.e., systems of social relationships). They are also learning systems and prone to be borrowed, for instance to improve the delivery of interventions. According to Pawson, social complex interventions are nonlinear and sometimes go into reverse because of feedback negative or positive loops in interventions implementation. Finally, these interventions are open and dynamic systems in space and time and depend on its history and its past, being the offspring of previous interventions.
Critical realism
The critical realism is one of the important pillars on which rests the realist approach of Pawson and Tilley [9]. The latter recognize the stratified nature of the social world and identify the generative mechanisms underlying a stratified social reality. Wilson and McCormack refer to three specific domains presented by the critical realist Bhaskar: the domain of the real (the causal mechanisms), the domain of the actual (the intervention itself), and the empirical domain (the change that is observable in reality) [56]. The evaluation of an intervention allows for the uncovering and analysis of causal mechanisms operating at the level of the real. “The prerequisite is to look beneath the surface in order to inspect how they work” ([25], p. 24). Another principle to which certain users refer like Pawson and colleagues is the generative causality. In similar or different contexts, individuals can make similar choices in such a way that reoccurring models can emerge, known as demi-regularities [33, 57]. Byng and colleagues underline that the realist approach in evaluation does not explicitly discuss the importance of the interactions between mechanisms or feedback loops, whereas in the original realist writings of Bhaskar, they are seen as fundamental to emergence [45]. Furthermore, as Connelly recalls [3, 58], the importance of time is recognized in critical realism. Some users also stress the interplay between agency and structure. Social structures provide resources that enable agents to act, and agents are therefore able to transform social structures by responding creatively to the circumstances in which they find themselves [34].
Clarity and precision of the concept mechanism
According to the Daigneault and Jacobs’ conceptual framework, a concept is clear when its term has just one sense within a given referent and it is that much more precise when the empirical phenomena to which it applies are clearly distinct from those to which it does not apply [20].
Firstly, some of the terms employed by users are an integral part of the lexicon of the realist approach as “power” and “force.” Wilson and McCormack confer upon the mechanism the power to generate outcomes [56], while Jagosh and colleagues see in it a generative force which leads to these outcomes [38]. However, the use of some terms reveals difficulties in a clear and precise comprehension of this concept with regard to the attributes presented earlier. For instance, few users assimilate mechanisms to intervention strategies or activities implemented [31, 58–60]. However, as recalled by Jagosh and colleagues [36, 38], mechanisms (i.e., reasoning and reactions) are linked to, but not synonymous with, the intervention strategies which are, according to these authors, intentional measures or a rational plan taken by program implementers.
Secondly, the concept of mechanism will become more precise and clearer once the frontier between the concepts of context and intervention has been clearly settled.
Mechanism versus context
According to users, these two concepts can be confusing, the boundaries between them being blurred [55, 61]. Pawson and Tilley [9] associate the context with the “spatial and institutional locations of social situations together, crucially, with the norms, values, and interrelationships found in them” (p. 216). Wong and colleagues also support a similar view referring the context as the prevailing beliefs, social and cultural norms, regulations, and economic factors [53]. Pawson and Tilley [12] honed their definition describing context as being the characteristics of the conditions in which interventions are introduced (and this even prior to its implementation [47]). Corresponding to the “backdrop” of interventions [38], the concept of context is also useful to describe the pre-existing characteristics of the individuals, localities, situations, or systems of interpersonal and social relationships in which an intervention is being set up. As Robert and colleagues have reiterated, context is social, cultural, historical or institutional [33]. In a nutshell, quoting Pawson and colleagues [25, 26], Macfarlane and colleagues use the four layers of contextual factors that shape the implementation of the social programs: (1) the individual capabilities of the key actors to take the intervention forward (e.g., values, roles, knowledge, purpose); (2) the interpersonal relationships supporting the intervention (e.g., communication, collaboration, network, influences); (3) the institutional settings (e.g., informal rules, organizational culture, leadership, resource allocation, local priorities); and (4) the infra-structural system (e.g., political support) [62]. These contextual layers can thus be at micro-level (e.g., individual actors), meso-level (e.g., departments and teams), or macro-level (e.g., organization) [35]. So, describing context and its effects through constraining or enabling factors [56] is the in-depth examination of all these elements which could prove relevant for our understanding of mechanisms [45, 60, 63, 64]. So, context, by interacting with mechanisms through its constraining and enabling factors, determines the direction of outcomes and change [39, 52, 65]. That is why the relationship between these mechanisms and the effects of the context in which they exist need to be understood [56]. Acknowledging context and mechanisms can be used to modify program theory (i.e., logic of the intervention), can help to explain why the intervention worked or not in a certain context, can help to identify where the intervention is likely to be most effective [58], and can strengthen the implementation of an intervention to other contexts (similar or not).
Mechanism versus intervention
Pawson and Tilley [12] remind us that the concept of mechanism sometimes becomes conflated with the one of intervention. In the realist approach, the attributes of the concept of mechanism are attached to the attributes of an intervention that is by nature complex and dynamic. As Pawson and Tilley said, the term mechanism is not used to distinguish the components of an intervention, each one of which will work through its own underlying processes (i.e., sequences of mechanisms) [12]. Whereas mechanisms (i.e., reasoning and reactions of human agents) refer to “the ways in which any one of the components of the intervention or any set of them, or any step of series of steps (e.g. decision-making steps) brings about change” [8] (p. 7), interventions can be seen as the “opportunities that an agent, situated inside structures and organizations, can choose to take” in a given context to bring about changes ([8], p. 62). As the scoping review reveals, the concepts of mechanism and intervention are located at different levels of abstraction. The attributes of the concept of intervention arise out of a more all-encompassing approach such as strategies and implemented activities whereas the attributes of the concept of mechanism seek to be more centered on the elements of individual or collective reasoning or reactions of agents in regard of the available resources allotted to the intervention implementation [25, 27]. Interventions can be regrouped around the mechanisms out of which they are built [24, 45, 53]. The realist approach focuses more on families of mechanisms rather than on families of interventions [53].
The operationalization of the concept “mechanism”
With the scoping review, we found that users have tried to categorize mechanisms in order to unpack, define, and prioritize them. This resulting typology takes different forms, dependent on the way in which the logic of the intervention is understood. For instance, Marchal and colleagues differentiate vision (i.e., what the team wants), from discourse (i.e., what the team says) and the actual practice of the intervention (i.e., what the team does) [18]. As a result, their typology of mechanisms is structured according to the different levels of interpretation and analysis suggested: i.e., targeted causal mechanisms (which are, according to these authors, close to the level of vision), theorized causal mechanisms (discourse) and observed mechanisms (arising out of practice). Similarly, Ridde and Guichard have highlighted different types of mechanisms: theoretical mechanisms (i.e., mechanisms proposed prior to the study), candidate mechanisms (i.e., mechanisms empirically collected during the study), and confirmed mechanisms (i.e., mechanisms empirically confirmed by the study) [31].
Discussion
Our scoping study of the concept of mechanism, assessed against the three dimensions of a concept identified by Daigneault and Jacobs [20], has yielded a rich picture of the evolving conceptual perspectives of both the proponents of the realist approach and of the users trying to enrich or operationalize the approach.
In the scoping study, only those writings featuring a definition of the concept of mechanism grounded in the realist approach were selected. The object of this study being this specific concept, we did not attempt at assessing the quality of the studies. In the same way, the analyses of the attributes of the concepts of context and intervention, presented above, are not exhaustive, given that these were not targeted by our inclusion criteria. These could, therefore, be the subject of a closer, more detailed study with a view to offering an exhaustive list of the attributes of these two concepts, both intimately related to that of the mechanism in the realist approach. However, given that initially, all the writings cited as references by Pawson and his colleagues have been read and analyzed through these different concepts; we can confidently say that the evidence collected remains solid and representative of what these concepts stand for in the realist approach.
Understanding the relationships between mechanism, intervention, and context: focus on the realist ontology
Our work pinpoints various difficulties that evaluators will need to address in order to make it useable in their practice. First of all, the identification and categorization of mechanisms proves to be a real challenge. Some mechanisms, as they pre-date the intervention, will elude the designers of the intervention. Moreover, public health interventions being complex, they encompass several concomitant mechanisms which can operate in parallel. On top of that, one needs to take into account the temporality of mechanisms and their relations with context, intervention, and other mechanisms to produce outcomes. These elements appear essential in order to categorize them before and during implementation of the intervention and its evaluation.
Next, the realist ontology seems relevant for understanding the complex and dynamic nature of public health intervention within a realist evaluation. Pawson reminds us that “realism is a general research strategy rather than a strict technical procedure” [77] (p. 14). For Bhaskar [78], a critical realist quoted by several users [34, 45, 49, 56, 57], the reality is stratified into three levels: the real (including the causal mechanisms and intransitive structures, which pre-exist beyond the consciousness of—individual or collective—human agents), the actual (including the events produced when the causal mechanisms are activated), and the empirical (what reflexive agents experiment to understand the phenomena). “Structures and mechanisms are real and distinct from the patterns of events that they generate; just as events are real and distinct from the experiences in which they are apprehended” [78] (p. 56). A public health intervention stands therefore as an event in a system [79]. This intervention can be described by its fixed functions and its forms that can vary in different contexts [80]. In other words, the functions can be seen as essential mechanisms (e.g., the mechanisms for reducing the social inequalities of health could be one of them), and the forms as strategies and implemented activities resulting from the interaction between functions and context. Pre-existing context (or pre-intervention context) and context of action could be distinguished [79]. The three levels of reality (i.e., the real, the actual, and the empirical) are also useful to interpret and explore the interrelationships between structures, human agents, and mechanisms. Archer, another influential theorist in the critical realism movement, illustrates the interplay between structure and agency over time within the morphogenetic approach [81]. Even though human agents depend on structures and their resources, they can also transform these structures through their actions. The causal mechanisms emanate both from the individuals and the social relations and structures which they form. It is important to bear in mind that both structures and agents are elements of context. As put by Poland and colleagues [82], context can be seen as “the local mix of conditions and events, social agents, objects and interactions which characterize social systems, and whose unique confluence in time and space selectively activates, triggers, blocks or modifies causal powers and mechanisms in a chain of reactions that may result in very different outcomes depending on the dynamic interplay of conditions and mechanisms in time and space” (p. 309). So, mechanisms would be for Pawson and Tilley the way-of-reasoning and reactions of human (individual or collective) agent(s) to bring about changes through the implementation of an intervention according to the resources available in a given context.
One strategy to improve the understanding of the diversity of these mechanisms could pass by the identification of the elements pertaining to the domain of the following: the real (i.e., causal mechanisms as functions of the intervention), the actual (i.e., intervention strategies as forms of the intervention), and the empirical (i.e., data collected during the intervention evaluation). Applied throughout the different steps of the evaluation process (theory building and theory testing), this strategy would make more explicit the multiple interrelationships between the mechanism of interest and the contextual factors (e.g., characteristics of individuals or collective agents—stakeholders, practitioners, or participants—and structures—pre-existing resources and/or new resources implemented by the intervention) and the various elements of the intervention, such as time (programming or implementation processes), space (micro- or macro-level), and form (strategies and implemented activities).
Definition of mechanism
Based on our thorough analysis of the literature on mechanism in a realist approach perspective, we can now suggest our own definition of this central concept that builds on Astbury and Leeuw’s [15].
A mechanism is hidden but real
To gain a better grip, and clarify the ontological focus chosen, the mechanism of interest should be characterized according to how it fits in the intervention and within the different strata of the social reality (i.e., the real, the actual, and the empirical). Existing prior to the intervention, but latent, a causal mechanism reveals itself during implementation of the intervention within a given context. Sensitive to the variations of context be they at the micro-, meso-, and macro-levels, it produces expected or unexpected outcomes that may or may not be favorable to a change in the problematic situation.
A mechanism is an element of reasoning and reactions of (an) individual or collective agent(s) in regard of the resources available in a given context to bring about changes through the implementation of an intervention
A mechanism results in the interaction between human agents, intervention, and structures. It reflects the logic of intervention of the various actors involved directly (e.g., stakeholders) or indirectly (e.g., populations) in the intervention.
A mechanism evolves within an open space-time and social system of relationships
A mechanism is dynamic and it may be interacting with other mechanisms (family of mechanisms), which may or may not be parallel, with the same process or another, with elements of context or with the effects it has itself produced (i.e., positive or negative feedback). The multiplicity and the temporality of mechanisms are thus important elements to be taken into account during the implementation and evaluation of an intervention. Indeed, interventions are subjected to the influence of other interventions (and thus to external mechanisms and contextual factors), have history of their own (sometimes that goes back well before their implementation), may also arise out of earlier interventions, and grow and evolve beyond their planned duration.
As one can readily see, this definition constitutes a refinement of the different dimensions of the key concept of mechanism and clarifies its attributes in order to facilitate its operationalization in public health program evaluation.
Conclusion
Like any methodological innovation, the realist approach in evaluation is gradually built through the challenges arising during its operationalization. It therefore came as no surprise to see the definition of the concept of mechanism evolving [19], just like the concepts of context and intervention that are so intimately related to it. This study has been useful on two fronts: (1) the analysis we carried out has yielded a clearer, more precise definition of the concept of mechanism, drawing on what the attributes of the concept are, going beyond Astbury and Leeuw’s three attributes that are said to be realist [15] and (2) this review has also underlined the importance of specifying the various levels of intervention, through the dynamic of mechanisms in its multiform context, to bring about changes. As a complement of the recent articles published to better operationalize the realist approach [19, 83, 84], the results of this study may in turn trigger further improvements in the way the realist approach is applied in evaluative practice in public health and potentially beyond.
Acknowledgements
This work was conducted as part of the task 1 of the AAPRISS (Apprendre et Agir Pour Réduire les Inégalités Sociales de Santé) program and thanks to a grant from the Institut National du Cancer (INCa), research convention No.2011-09 and from the Agence Nationale de la Recherche, grant No. ANR 11 INEG 003 01. We would like to thank Prof Thierry Lang and the AAPRISS team for their support in the conduct of this research as well as the many colleagues who contributed by their comments and questions to the writing of this paper.
Additional files
Footnotes
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
AL, EB, AG, and VR contributed equally to this manuscript. All authors read and authorized the final version.
Authors’ information
AL is a fellow of the Brittany Region Doctoral Research Allocation (ARED) and the Population Health Intervention Research Network (PHIRNET), which is funded by the CIHR. VR holds a CIHR-funded Research Chair in Applied Public Health (CPP-137901). EB is the holder of the INPES Health Promotion Chair at the EHESP French School of Public Health. The INPES Health Promotion Chair at the EHESP is funded by the French National Institute for Prevention and Health Education. AG is a professor, researcher in public health, at the Laval University, Quebec, Canada.
References
- 1.McQueen DV, Kickbusch I, Potvin L, Pelikan JM, Balbo L, Abel Th. Health and modernity: the role of theory in health promotion. New York: Springer; 2007
- 2.Petticrew M. When are complex interventions “complex”? When are simple interventions “simple”? Eur J Public Health. 2011;21(4):397–8. doi: 10.1093/eurpub/ckr084. [DOI] [PubMed] [Google Scholar]
- 3.Connelly JB. Evaluating complex public health interventions: theory, methods and scope of realist enquiry. J Eval Clin Pract. 2007;13(6):935–41. doi: 10.1111/j.1365-2753.2006.00790.x. [DOI] [PubMed] [Google Scholar]
- 4.Ridde V. Reducing social inequalities in health: public health, community health or health promotion? Promot Educ. 2007;14(2):63–7. doi: 10.1177/10253823070140021401. [DOI] [PubMed] [Google Scholar]
- 5.Weiss CH. How can theory-based evaluation make greater headway? Eval Rev. 1997;21(4):501–24. doi: 10.1177/0193841X9702100405. [DOI] [Google Scholar]
- 6.Chen HT, Rossi PH. The multi-goal, theory-driven approach to evaluation: a model linking basic and applied social science. Soc Forces. 1980;59(1):106–22. doi: 10.1093/sf/59.1.106. [DOI] [Google Scholar]
- 7.Chen HT, Rossi PH. The theory-driven approach to validity. Eval Program Plann. 1987;10(1):95–103. doi: 10.1016/0149-7189(87)90025-5. [DOI] [Google Scholar]
- 8.Stame N. Theory-based evaluation and types of complexity. Evaluation. 2004;10(1):58–76. doi: 10.1177/1356389004043135. [DOI] [Google Scholar]
- 9.Pawson R, Tilley N. Realistic evaluation. London: Sage publications; 1997. [Google Scholar]
- 10.Rogers P. Using programme theory to evaluate complicated and complex aspects of interventions. Evaluation. 2008;14(1):29–48. doi: 10.1177/1356389007084674. [DOI] [Google Scholar]
- 11.Sridharan S, Nakaima A. Towards an evidence base of theory-driven evaluations: some questions for proponents of theory-driven evaluation. Evaluation. 2012;18(3):378–95. doi: 10.1177/1356389012453289. [DOI] [Google Scholar]
- 12.Pawson R, Tilley N. Realist evaluation. London: Cabinet Office; 2004. [Google Scholar]
- 13.Blaise P. Culture qualité et organisation bureaucratique, le défi du changement dans les systèmes publics de santé. Une évaluation réaliste de projets qualité en Afrique [thèse] Bruxelles: Université Libre de Bruxelles; 2004. p. 202. [Google Scholar]
- 14.Blaise P, Marchal B, Lefèvre P, Kegels G. Au-delà des méthodes expérimentales : l’approche réaliste en évaluation. In: Potvin L, Moquet MJ, Jones C, editors. Réduire les inégalités sociales en santé. Saint-Denis: INPES coll. Santé en action; 2010. pp. 285–96. [Google Scholar]
- 15.Astbury B, Leeuw FL. Unpacking black boxes: mechanisms and theory building in evaluation. Am J Eval. 2010;31(3):363–81. doi: 10.1177/1098214010371972. [DOI] [Google Scholar]
- 16.Marchal B, Van Belle S, Van Olmen J, Hoerée T, Kegels G. Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research. Evaluation. 2012;18(2):192–212. doi: 10.1177/1356389012442444. [DOI] [Google Scholar]
- 17.Ridde V, Robert E, Guichard A, Blaise P, Van Olmen J. L’approche Realist à l’épreuve du réel de l’évaluation des programmes. Revue Canadienne d’évaluation de Programme. 2012;26(3):14. [Google Scholar]
- 18.Marchal B, Dedzo M, Kegels G. A realist evaluation of the management of a well-performing regional hospital in Ghana. BMC Health Serv Res. 2010;10:24. doi: 10.1186/1472-6963-10-24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Dalkin SM, Greenhalgh J, Jones D, Cunningham B, Lhussier M. What’s in a mechanism? Development of a key concept in realist evaluation. Implement Sci. 2015;10:49. doi: 10.1186/s13012-015-0237-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Daigneault PM, Jacob S. Les concepts souffrent-ils de négligence bénigne en sciences sociales? Éléments d’analyse conceptuelle et examen exploratoire de la littérature francophone à caractère méthodologique. Soc Sci Inf. 2012;51(2):188–204. doi: 10.1177/0539018412437107. [DOI] [Google Scholar]
- 21.Mays N, Roberts E, Popay J. Synthesising research evidence. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the organization and delivery of health services: research methods. London: Routledge; 2001. pp. 188–220. [Google Scholar]
- 22.Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32. doi: 10.1080/1364557032000119616. [DOI] [Google Scholar]
- 23.Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:6. doi: 10.1371/journal.pmed.1000097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Pawson R. Evidence-based policy: the promise of “realist synthesis”. Evaluation. 2002;8(3):340–58. doi: 10.1177/135638902401462448. [DOI] [Google Scholar]
- 25.Pawson R. Evidence-based policy: a realist perspective. Thousand Oaks: Sage; 2006. [Google Scholar]
- 26.Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10:21–34. doi: 10.1258/1355819054308530. [DOI] [PubMed] [Google Scholar]
- 27.Pawson R, Wong G, Owen L. Known knowns, known unknowns, unknown unknowns: the predicament of evidence-based policy. Am J Eval. 2011;32(4):518–46. doi: 10.1177/1098214011403831. [DOI] [Google Scholar]
- 28.Pawson R, Manzano-Santaella A. A realist diagnostic workshop. Evaluation. 2012;18(2):176–91. doi: 10.1177/1356389012440912. [DOI] [Google Scholar]
- 29.Blaise P, Kegels G. A realistic approach to the evaluation of the quality management movement in health care systems: a comparison between European and African contexts based on Mintzberg’s organizational models. Int J Health Plann Manage. 2004;19(4):337–64. doi: 10.1002/hpm.769. [DOI] [PubMed] [Google Scholar]
- 30.Marchal B, Dedzo M, Kegels G. Turning around an ailing district hospital: a realist evaluation of strategic changes at Ho Municipal Hospital (Ghana) BMC Public Health. 2010;10:787. doi: 10.1186/1471-2458-10-787. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Ridde V, Guichard A. Perception De Quelques Mécanismes Favorables À La Réduction Des Inégalités Sociales De Santé En France. Glob Health Promot. 2011;18(3):47–60. doi: 10.1177/1757975911412400. [DOI] [PubMed] [Google Scholar]
- 32.Pommier J, Guével MR, Jourdan D. Evaluation of health promotion in schools: a realistic evaluation approach using mixed methods. BMC Public Health. 2010;10:1–12. doi: 10.1186/1471-2458-10-43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Robert E, Ridde V, Marchal B, Fournier P. Protocol: a realist review of user fee exemption policies for health services in Africa. BMJ Open. 2012;2(1):e000706. doi: 10.1136/bmjopen-2011-000706. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Wand T, White K, Patching J. Applying a realist(ic) framework to the evaluation of a new model of emergency department based mental health nursing practice. Nurs Inq. 2010;17(3):231–9. doi: 10.1111/j.1440-1800.2010.00488.x. [DOI] [PubMed] [Google Scholar]
- 35.Rycroft-Malone J, Wilkinson JE, Burton CR, Andrews G, Ariss S, Baker R, et al. Implementing health research through academic and clinical partnerships: a realistic evaluation of the Collaborations for Leadership in Applied Health Research and Care (CLAHRC) Implement Sci. 2011;6(1):74–85. doi: 10.1186/1748-5908-6-74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Jagosh J, Pluye P, Macaulay AC, Salsberg J, Henderson J, Sirett E, et al. Assessing the outcomes of participatory research: protocol for identifying, selecting, appraising and synthesizing the literature for realist review. Implement Sci. 2011;6(1):24–31. doi: 10.1186/1748-5908-6-24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Kastner M, Estey E, Perrier L, Graham ID, Grimshaw J, Straus SE, et al. Understanding the relationship between the perceived characteristics of clinical practice guidelines and their uptake: protocol for a realist review. Implement Sci. 2011;6(1):69–77. doi: 10.1186/1748-5908-6-69. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J, et al. Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Q. 2012;90:2. doi: 10.1111/j.1468-0009.2012.00665.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Byng R, Norman I, Redfern S, Jones R. Exposing the key functions of a complex intervention for shared care in mental health: case study of a process evaluation. BMC Health Serv Res. 2008;8:274. doi: 10.1186/1472-6963-8-274. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Robert E. Etat des lieux des connaissances disponibles sur les politiques et les programmes d’exemption des paiements directs des soins en Afrique de l’Ouest (Bénin, Burkina Faso, Ghana, Guinée, Mali, Niger et Sénégal). Projet de capitalisation d’expériences d’exemption du paiement des soins en faveur des populations vulnérables du Sahel en Afrique de l’Ouest (CRCHUM, ONG HELP, ECHO); 2011.
- 41.Holma K, Kontinen T. Realistic evaluation as an avenue to learning for development NGOs. Evaluation. 2011;17(2):181–92. doi: 10.1177/1356389011401613. [DOI] [Google Scholar]
- 42.Clark AM, Whelan HK, Barbour R, MacIntyre PD. A realist study of the mechanisms of cardiac rehabilitation. J Adv Nurs. 2005;52(4):362–71. doi: 10.1111/j.1365-2648.2005.03601.x. [DOI] [PubMed] [Google Scholar]
- 43.Pittam G, Boyce M, Secker J, Lockett H, Samele C. Employment advice in primary care: a realistic evaluation. Health Soc Care Community. 2010;18(6):598–606. doi: 10.1111/j.1365-2524.2010.00929.x. [DOI] [PubMed] [Google Scholar]
- 44.Ranmuthugala G, Cunningham FC, Plumb JJ, Long J, Georgiou A, Westbrook JI, et al. A realist evaluation of the role of communities of practice in changing healthcare practice. Implement Sci. 2011;6(1):49–54. doi: 10.1186/1748-5908-6-49. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Byng R, Norman I, Redfern S. Using realistic evaluation to evaluate a practice-level intervention to improve primary healthcare for patients with long-term mental illness. Evaluation. 2005;11(1):69–93. doi: 10.1177/1356389005053198. [DOI] [Google Scholar]
- 46.Kane SS, Gerretsen B, Scherpbier R, Dal Poz M, Dieleman M. A realist synthesis of randomised control trials involving use of community health workers for delivering child health interventions in low and middle income countries. BMC Health Serv Res. 2010;10:286–92. doi: 10.1186/1472-6963-10-286. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Dieleman M, Gerretsen B, van der Wilt GJ. Human resource management interventions to improve health workers’ performance in low and middle income countries: a realist review. Health Res Policy Syst. 2009;7:1–13. doi: 10.1186/1478-4505-7-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Maluka S, Kamuzora P, SanSebastián M, Byskov J, Ndawi B, Olsen ØE, et al. Implementing accountability for reasonableness framework at district level in Tanzania: a realist evaluation. Implement Sci. 2011;6:11. doi: 10.1186/1748-5908-6-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Wand T, White K, Patching J. Realistic evaluation of an emergency department-based mental health nurse practitioner outpatient service in Australia. Nurs Health Sci. 2011;13(2):199–206. doi: 10.1111/j.1442-2018.2011.00601.x. [DOI] [PubMed] [Google Scholar]
- 50.Greenhalgh T, Humphrey C, Hughes J, Macfarlane F, Butler C, Pawson R. How do you modernize a health service? A realist evaluation of whole-scale transformation in London. Milbank Q. 2009;87(2):391–416. doi: 10.1111/j.1468-0009.2009.00562.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Steven A, Dickinson C, Pearson P. Practice-based interprofessional education: looking into the black box. J Interprof Care. 2007;21(3):251–64. doi: 10.1080/13561820701243664. [DOI] [PubMed] [Google Scholar]
- 52.Hoddinott P, Britten J, Pill R. Why do interventions work in some places and not others: a breastfeeding support group trial. Soc Sci Med. 2010;70:769–78. doi: 10.1016/j.socscimed.2009.10.067. [DOI] [PubMed] [Google Scholar]
- 53.Wong G, Greenhalgh T, Westhorp G, Pawson R. Realist methods in medical education research: what are they and what can they contribute? Med Educ. 2012;46(1):89–96. doi: 10.1111/j.1365-2923.2011.04045.x. [DOI] [PubMed] [Google Scholar]
- 54.Tolson D, McIntosh J, Loftus L, Cormie P. Developing a managed clinical network in palliative care: a realistic evaluation. Int J Nurs Stud. 2007;44(2):183–95. doi: 10.1016/j.ijnurstu.2005.11.027. [DOI] [PubMed] [Google Scholar]
- 55.Pedersen LM, Nielsen KJ, Kines P. Realistic evaluation as a new way to design and evaluate occupational safety interventions. Saf Sci. 2012;50(1):48–54. doi: 10.1016/j.ssci.2011.06.010. [DOI] [Google Scholar]
- 56.Wilson V, McCormack B. Critical realism as emancipatory action: the case for realistic evaluation in practice development. Nurs Philos. 2006;7(1):45–7. doi: 10.1111/j.1466-769X.2006.00248.x. [DOI] [PubMed] [Google Scholar]
- 57.Rycroft-Malone J, McCormack B, Hutchinson AM, DeCorby K, Bucknall TK, Kent B, et al. Realist synthesis: illustrating the method for implementation research. Implement Sci. 2012;7(1):33. doi: 10.1186/1748-5908-7-33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Connelly JB, Duaso MJ, Butler G. A systematic review of controlled trials of interventions to prevent childhood obesity and overweight: a realistic synthesis of the evidence. Public Health (Elsevier) 2007;121(7):510–7. doi: 10.1016/j.puhe.2006.11.015. [DOI] [PubMed] [Google Scholar]
- 59.Lhussier M, Bangash S, Dykes F, Zaman M, Lowe NM. Development and implementation of a nutrition intervention programme in North West Pakistan: a realist framework. Health Promotion International. 2011; doi:10.1093/heapro/dar067. [DOI] [PubMed]
- 60.Jackson SF, Kolla G. A new realistic evaluation analysis method: linked coding of context, mechanism, and outcome relationships. Am J Eval. 2012;33(3):339–49. doi: 10.1177/1098214012440030. [DOI] [Google Scholar]
- 61.Rycroft-Malone J, Fontenla M, Bick D, Seers K. A realistic evaluation: the case of protocol-based care. Implement Sci. 2010;5:38. doi: 10.1186/1748-5908-5-38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Macfarlane F, Greenhalgh T, Humphrey C, Hughes J, Butler C, Pawson R. A new workforce in the making? A case study of strategic human resource management in a whole-system change effort in healthcare. J Health Organ Manag. 2011;25(1):55–72. doi: 10.1108/14777261111116824. [DOI] [PubMed] [Google Scholar]
- 63.Ogrinc G, Batalden P. Realist evaluation as a framework for the assessment of teaching about the improvement of care. J Nurs Educ. 2009;48(12):661–7. doi: 10.3928/01484834-20091113-08. [DOI] [PubMed] [Google Scholar]
- 64.Oroviogoicoechea C, Watson R. A quantitative analysis of the impact of a computerised information system on nurses’ clinical practice using a realistic evaluation framework. Int J Med Inform. 2009;78(12):839–49. doi: 10.1016/j.ijmedinf.2009.08.008. [DOI] [PubMed] [Google Scholar]
- 65.Macaulay AC, Jagosh J, Seller R, Henderson J, Cargo M, Greenhalgh T, et al. Assessing the benefits of participatory research: a rationale for a realist review. Glob Health Promot. 2011;18(2):45–8. doi: 10.1177/1757975910383936. [DOI] [PubMed] [Google Scholar]
- 66.Porter S, O’Halloran P. The use and limitation of realistic evaluation as a tool for evidence-based practice: a critical realist perspective. Nurs Inq. 2012;19(1):18–28. doi: 10.1111/j.1440-1800.2011.00551.x. [DOI] [PubMed] [Google Scholar]
- 67.Rycroft-Malone J, Fontenla M, Bick D, Seers K. Protocol-based care: impact on roles and service delivery. J Eval Clin Pract. 2008;14(5):867–73. doi: 10.1111/j.1365-2753.2008.01015.x. [DOI] [PubMed] [Google Scholar]
- 68.Evans D, Killoran A. Tackling health inequalities through partnership working: learning from a realistic evaluation. Crit Public Health. 2000;10(2):125–40. doi: 10.1080/09581590050075899. [DOI] [Google Scholar]
- 69.Ogilvie D, Bull F, Powell J, Cooper AR, Brand C, Mutrie N, et al. An applied ecological framework for evaluating infrastructure to promote walking and cycling: the iConnect study. Am J Public Health. 2011;101(3):473–81. doi: 10.2105/AJPH.2010.198002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Leone L. Realistic evaluation of an illicit drug deterrence programme: analysis of a case study. Evaluation. 2008;14(1):9–28. doi: 10.1177/1356389007084673. [DOI] [Google Scholar]
- 71.Keller C, Gare K, Edenius M, Lindblad S. Innovations in health care: design theory and realist evaluation combined. Sprouts: Working Papers on Information Systems. 2010;10:66. [Google Scholar]
- 72.Kœnig G. Realistic Evaluation and Case Studies Stretching the Potential. Evaluation. 2009;15(1):9–30. doi: 10.1177/1356389008097869. [DOI] [Google Scholar]
- 73.Wong G, Pawson R, Owen L. Policy guidance on threats to legislative interventions in public health: a realist synthesis. BMC Public Health. 2011;11(1):222–32. doi: 10.1186/1471-2458-11-222. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.McCormack B, Wright J, Dewar B, Harvey G, Ballantine K. A realist synthesis of evidence relating to practice development: methodology and methods. Pract Dev Health Care. 2007;6(1):5–24. doi: 10.1002/pdh.210. [DOI] [Google Scholar]
- 75.Wong G, Greenhalgh T, Pawson R. Internet-based medical education: a realist review of what works, for whom and in what circumstances. BMC Med Educ. 2010;10:12. doi: 10.1186/1472-6920-10-12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 76.Michielsen J, Criel B, Devadasan N, Soors W, Wouters E, Meulemans H. Can health insurance improve access to quality care for the Indian poor? Int J Qual Health Care. 2011;23(4):471–86. doi: 10.1093/intqhc/mzr025. [DOI] [PubMed] [Google Scholar]
- 77.Pawson R. The science of evaluation: a realist manifesto. London: SAGE Publications; 2013. [Google Scholar]
- 78.Bhaskar R. A realist theory of science. Revisedth ed. London: Routledge; 2008.
- 79.Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol. 2009;43:267–76. doi: 10.1007/s10464-009-9229-9. [DOI] [PubMed] [Google Scholar]
- 80.Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? BMJ. 2004;328:1561. doi: 10.1136/bmj.328.7455.1561. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 81.Archer M. Realist social theory: the morphogenetic approach. Cambridge: Cambridge University Press; 1995. [Google Scholar]
- 82.Poland B, Frohlich KL, Cargo M. Context as a fundamental dimension of Health Promotion Program Evaluation. In: Potvin L, McQueen D, editors. Health promotion evaluation practices in the Americas: values and research. New York: Springer; 2008. pp. 299–317. [Google Scholar]
- 83.Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: realist syntheses. BMC Med. 2013;11:21. doi: 10.1186/1741-7015-11-21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 84.Greenhalgh T, Wong G, Jagosh J, Greenhalgh J, Manzano A, Westhorp G, et al. Protocol—the RAMESES II study: developing guidance and reporting standards for realist evaluation. BMJ Open. 2015;5:8. doi: 10.1136/bmjopen-2015-008567. [DOI] [PMC free article] [PubMed] [Google Scholar]