Abstract
Introduction
Scaling-up is essential to ensure universal access of effective health interventions. Scaling-up is a complex process, which occurs across diverse systems and contexts with no one-size-fits-all approach. To date, little attention has been paid to the process of scaling-up in how to make adaptations for local fit. The aim of this research is to develop theory on what actions can be used to make adaptations to health interventions for local fit when scaling-up across diverse contexts that will have practical application for implementers involved in scaling-up.
Methods and analysis
Given the complexity of this subject, a realist review methodology was selected. Specifically, realist review emphasises an iterative, non-linear process, whereby the review is refined as it progresses. The identification of how the context may activate mechanisms to achieve outcomes is used to generate theories on what works for whom in what circumstances. This protocol will describe the first completed stage of development of an initial programme theory framework, which identified potential actions, contexts, mechanisms and outcomes that could be used to make adaptations when scaling-up. It will then outline the methods for future stages of the review which will focus on identifying case examples of scale-up and adaptation in practice. This realist review consists of six stages: (i) clarifying scope and development of a theoretical framework, (ii) developing a search strategy, (iii) selection and appraisal, (iv) data extraction, (v) data synthesis and analysis and (vi) further theory refinement with stakeholders.
Ethics and dissemination
This review will develop theory on how adaptations can be made when scaling-up. Findings will be disseminated in a peer-reviewed journal and through stakeholder engagement as part of the research process. Ethical approval has been received through Health Policy and Management/Centre for Global Health Research Ethics Committee of Trinity College Dublin.
Keywords: scale up, scaling up, adaptation, fidelity, realist review, realist synthesis
Strengths and limitations of this study.
The use of a realist review approach will allow for the exploration of the complexity of scale-up and adaptation in practice.
We present an initial programme theory framework which identifies potential actions, contexts, mechanisms and outcomes that may influence scale-up and adaptation which is based on peer-reviewed literature and frameworks in the fields of fidelity, adaptation and scale-up.
This protocol provides a detailed account of proposed methods for a realist review, including the supplemental files of a research logbook, coding and synthesis procedures, which may assist future researchers in options for approaches that can be taken and for addressing the issue of decision making and transparency for realist reviews.
This study will use the inclusion of stakeholders for theory refinement in the later stages of the review, ensuring practicality of findings and dissemination through the review process.
The scope of this review is ambitious within the time-frame, however in keeping with realist reviews this may be further refined throughout the stages in light of findings from the literature or by stakeholder consultation.
Introduction
The process of scaling-up an effective health intervention is complex and occurs across diverse systems and contexts.1 2 It is estimated that only 14% of healthcare research makes it into real-world settings.3 4 Therefore, many existing health problems could be addressed through scaling-up of interventions already known to be effective. For example, it is estimated that 85% of childhood deaths could be avoided in low-income and middle-income countries through scale-up of existing health interventions like zinc and oral rehydration therapy treatment.5 Currently, scale-up has been estimated to take 15 years from pilot to national scale.6 Scale-up is time consuming and challenging due to the complexity of implementing across diverse contexts where the population,7 finances, resources and capacity8 may differ. The result is a growing discussion on the need to provide more evidence for how to address this important research-to-practice gap.
Scale-up, adaptation and fidelity
Scale-up can be defined as a purposeful expansion of a health intervention to a wider population.1 9 This could involve expanding geographically, or to a wider population within the same setting. Adaptations can be defined as deliberate and/or unintended changes to the intervention content, context or training and delivery.10 As per the international classification of health interventions,11 a health intervention can be defined as ‘an act performed for, with or on behalf of a person or population to assess, improve, maintain, promote or modify health, functioning or health conditions’ (paragraph 1). When scaling-up it can be necessary to adapt for local contexts as needs and resources may differ between scale-up sites.7 12 Adaptation has been seen as an essential process to match community needs, organisation resources and to gain trust and ownership by community.13 By addressing and adapting for local fit, it can assist in successful implementation and sustainability of an intervention.12 However, with adaptations there is also a need to ensure fidelity to the intervention theory and essential components to ensure the effectiveness of an intervention is not reduced or lost.7
Fidelity has been described by Castro et al 14 as delivering the programme as intended and tested, however they also noted there may often be a need to adapt to the target population.14 Fidelity can often be seen as a top-down (researcher, intervention developer) driven approach and adaptation has been viewed as more of a community-driven bottom-up approach (frontline service providers, communities, individuals).15 However, when looking at fidelity it is impossible to ignore potential necessary adaptations for local needs, and when looking at adaptation to ignore how to maintain fidelity to the original intervention. With both proposed as necessary when scaling-up across diverse populations and delivery systems.7 In considering fidelity it is suggested that any adaptations retain the underlying intervention theory and that the essential components or active ingredients remain intact, with any changes made to match the unique features of the setting.13 15 16 This opinion was shared by Aarons et al 7 in relation to scale-up, and Chambers et al 12 in relation to sustainability, where identification of theory and essential elements of the intervention can facilitate adaptation outside of these, and assist in avoiding a ‘voltage drop’,12 or the tendency for effectiveness to taper with ongoing implementation.
The need for adaptation when scaling-up
Within complex systems, such as healthcare, applying a single approach in all settings is unlikely to be effective, as it does not take into account the complex contextual environment within which the intervention takes place.17 Therefore, adaptations are important in terms of ensuring that the intervention content, context and/or delivery strategy fits with local needs across scale-up sites.10 A trade-off may need to occur between increasing scale and adapting to maintain local values, local relevance, quality and sustainability.18 Additionally, and given that contexts are continually changing over time, allowing for adaptations with contextual changes is needed to ensure sustainability of interventions.12 Specifically, adaptation has the potential to enable implementers to match the needs of a more heterogeneous population; to simplify a complex intervention; to focus on a specific problem or to expand to address multiple problems; to increase ownership of an intervention; to adjust to a lack of available resources or requirements made by agencies or funders; allow for additional applications of an intervention and/or address a lack of knowledge of the intervention.13 It is important to note that adaptations may be intended or unintended,10 19 and may be positive or negative.20 With positive adaptations supporting implementation and achieving desired clinical outcomes, negative adaptations could potentially hinder or reduce these. Holliday et al 20 put forward in the design and testing of an educational intervention, that adaptations can be on a spectrum from acceptable to unacceptable, and avoidable to unavoidable. Thus, some adaptations may be unavoidable for local fit, however are acceptable as they maintain the intervention theory and essential components. While others may be unacceptable as they change the underlying theory and essential components, or avoidable in that they may not necessarily need to be adapted within that setting.
Although adaptation has been highlighted as a key contributing factor in addressing feasibility and/or acceptability for local settings when scaling-up,7 12 adaptation is rarely documented as part of the scale-up process.1 21 22 Often efforts to achieve scale-up can focus on the replication of the originally tested pilot or feasibility study. Replication however, does not account for the diverse social, political and cultural contexts across scale-up sites. This results in a need for more tailored approaches.23 There is a need to understand how an intervention may work in a given context to allow for selection of approaches that are most likely to be effective in that setting, thus avoiding interventions being deemed potentially ineffective and not achieving scale-up.24 The need to adapt interventions for local settings has been put forward across the health spectrum from maternal and child health,25 malaria prevention,26 HIV,27–29 to mental health.30 While some recent frameworks have supported the local development of adaptations when scaling-up, and suggested actions such as use of quality improvement methodology.2 12 Unfortunately, there still remains minimal guidance on how to complete local adaptation, adding to the difficulty in achieving and reporting of scale-up of health interventions with local fit.1 31
Adaptations in practice
Despite its absence of documentation, adaptation has been discovered as naturally and commonly occurring in the practice of scaling-up.32 For example, within 44 preventive interventions in a Substance Abuse and Mental Health Services (SAMHSA) national database in the USA, over half of these had been adapted, suggesting that adaptation is more common than not.14 A study by Moore et al 32 looked at the adaptation of evidence-based programmes in Pennsylvania for reducing delinquency and violence. Out of 10 evidence-based programmes with over 200 replications across the state, 44% reported making adaptations. It has also been recognised that informal on-the-job adaptations are often made by professionals working in, and deeply embedded in the context and who therefore may understand the nuances.33 However, this informal on-the-job approach to adaptation is rarely discussed, documented or evaluated, resulting in a dearth of information on how adaptation may impact the intervention in the long term.
Guidance for adaptation and scale-up
Within the implementation and scale-up literature there are many models and frameworks mentioning the need for adaptations, for example, the AIDED and ExpandNet process frameworks.1 34 Despite growing recognition of the importance of adapting across diverse local settings, along with evidence that adaptations are occurring in practice,14 32 33 there is minimal guidance on what and how specific actions (eg, transferring decision making to local level, generation and use of local data or engagement of the community) can be used to achieve adaptations when scaling-up, and even less guidance on how, why and when to choose one method over another across different contexts. Moreover, while there is some guidance available for implementers on adaptations, these guidelines are not specific to scale-up, and most existing guidance on adaptation are based in the field of substance abuse prevention and HIV behaviour change interventions.13 19 35–38 These were largely designed for high-income country contexts and some of this guidance requires highly skilled, and resource heavy processes. Additionally, some guidance suggests involving the original intervention developers, and at times promoting redesign and testing of the intervention,37 which may not be feasible at multiple diverse sites when scaling-up. Previous scale-up and sustainability frameworks have promoted adaptation for local fit,7 12 however, there is a need for more guidance on how to achieve this. A previous review explored the process of scale-up of complex interventions,39 however did not specifically address adaptations for local fit when scaling-up. Therefore, while acknowledging the importance of adaptation for local fit there is minimal guidance for implementers on what actions can be used to achieve adaptations when scaling-up, by what mechanisms these may work and how the context may influence this. Therefore, there is a need to build on current knowledge of scale-up.
Research questions
What are the actions that can be used to guide adaptations when scaling-up healthcare interventions?
How do these actions work (ie, by what mechanisms, and in what contexts)?
Aims and objectives
The aim of this research is to develop theory on what actions can be used to make adaptations to health interventions for local fit when scaling-up across diverse contexts that will have practical application for implementers involved in scaling-up.
Objectives:
Identify what adaptations are being made in practice when scaling-up health interventions for local fit.
Identify what actions are used to achieve adaptations when scaling-up health interventions for local fit.
Discover how these actions work by uncovering what mechanisms are triggered, in what contexts, to achieve adaptations when scaling-up health interventions for local fit.
To put forward theories on what actions can be used, and how these actions may work to achieve adaptations when scaling-up health interventions for local fit, by identifying demi-regularities within the uncovered contexts and mechanisms.
Methods and analysis
Realist review methodology
Realist review is a methodology for evidence synthesis that uses a theory-driven interpretive approach to explain findings.40 41 It aims to provide an explanation of what works, for whom and why, in what circumstances.42 It allows for exploration of the complexity of a topic with a focus on theory generation that may be applicable in the setting under study, and applicable in wider settings through development of theory of ‘middle-range’.40 43 Realist review methodology allows for inclusion of a wide body of evidence including grey literature sources.44 It supports stakeholder involvement throughout the stages of the review to inform the scope of the review, to develop and refine theory41 and/or assist in dissemination45 of findings.
Realist review focuses on causation, with identification of where an intervention or action under certain contextual conditions (C), may trigger a mechanism (M), to achieve a given outcome (O).40 46 It completes this through development of context-mechanism-outcome configurations (CMOCs),42 which are central to the analysis and theory building process with mechanisms often seen as the integral link between the context and the outcome.43 They can uncover the ‘why’ a given outcome may have occurred. Dalkin et al 46 conceptualised mechanisms as either resources or reasoning. They put forward that a mechanism can be a resource which can be introduced in a context, which can trigger a mechanism in the form of response or reasoning, resulting in an outcome. However, mechanisms may only activate in specific contextual conditions with the context as acting like a dimmer switch.46 Within optimal contextual conditions mechanisms are triggered or ‘fire’, and with suboptimal conditions mechanisms may fire to a lesser degree or not at all.46 It is also acknowledged that actions may influence and change the context, which in turn may influence whether and how a mechanism fires.
In the current research, we view actions that were carried out to achieve adaptations when scaling-up (eg, generation of evidence or participation of stakeholders), as a mechanism in the form of a resource. These actions, under the optimal contextual conditions, may trigger a mechanism in the form or reasoning or response (eg, awareness or commitment), which in turn may generate outcomes. For this research outcomes can be proximal or distal. With distal outcomes relating to the overall aim, for example, adaptations with local fit or sustainability as reported by the evidence, and proximal outcomes relating to those that may occur prior to this, for example, local ownership of the intervention or consensus for adaptations.
Demi-regularities (semi-predictable patterns occurring in the CMOCs) can further assist in explanation of the findings. Abductive reasoning can be used, which Jagosh et al 47 described as the ‘iterative process of examining evidence and developing hunches or ideas about the causal factors linked to that evidence’ (p. 5). Abductive reasoning could be discussed as explaining a finding from both the seen and unseen, and drawing from theoretical perspectives to provide possible explanations for an outcome.48 49 This can involve recontextualising or redescribing explanations based on interpretations.48 Retroduction can also then be used to situate the findings and put forward what causal pathways and conditions may need to be present for the phenomenon of interest to occur.48
A realist review methodology was chosen as appropriate to address the study objectives for four reasons. First, within scale-up research, realist review methodology allows for in-depth consideration of how actions can be influenced by contextual factors (eg, resource availability, level of perceived need for intervention in a local setting, etc) to trigger mechanisms (eg, trust, commitment, awareness) to generate desired outcomes (eg, local ownership, feasible and acceptable adaptations of a health intervention) leading to successful scale-up and local fit. Second, realist review methodology was chosen as it recognises the use of multiple evidence sources, which was considered particularly important for scaling-up. While not prioritised on a traditional hierarchy of evidence, grey literature reports may contain valuable information on the scale-up process. Third, stakeholder involvement can also assist in validation and refinement of theory50 and it has been put forward by Brennan et al 41 that involvement of stakeholders can provide a ‘reality check’ as to whether the findings are consistent with experience and knowledge from practice. Involvement of stakeholders with experience in adaptation and scale-up through research and practice, may assist in ensuring the findings are practical and of utility to implementers in the field. Finally, realist review methodology has been previously successfully used to explore the process of scaling-up complex healthcare interventions.39 This allowed for an in-depth analysis of how complex health interventions were scaled-up across three case studies, identifying active mechanisms that were needed to achieve scale-up, and suggesting how the context may have influenced the scale-up across these cases.
Stages of the realist review
This protocol is based on the five stages of realist review by Pawson et al,40 with the addition of a further stage of stakeholder involvement for theory refinement, which has been put forward by previous reviews41 51 (see figure 1, adapted from Molnar et al 52and Groot et al 53). These stages are not necessarily carried out in a linear process as the stages are iterative and may overlap and inform each other as learning on the topic progresses and theory refinement takes place.
Stage one of this review has been completed to clarify the scope of the review and develop the initial theoretical framework. This protocol paper will briefly describe this first stage process and how findings were used to develop the protocol for the following stages 2–6, which are to be carried out from June 2018 to March 2019.
Stage 1: clarifying the scope of the review and developing a theoretical framework
According to Pawson et al,40 a realist review begins with clarifying the scope of the review and the elicitation of initial rough theories in the form of an initial programme theory (IPT). The IPT can provide a map of the areas to be investigated and gives a structure for data synthesis.39 These can be further refined, tested and added to as the synthesis progresses.43
Developing the IPT framework
For this research, an IPT framework was developed which was a theoretical framework to guide the review. This will be refined as the review progresses in future stages. The methodology and format of the IPT framework to guide this review was informed by the Willis et al 39 realist review, which focused on the process of scale-up of complex interventions, identifying in their initial IPT framework actions, contexts and outcomes. After analysis and synthesis of three case studies, they further identified what mechanisms were triggered to achieve scale-up of complex interventions and what contexts influenced this. Therefore, the realist review by Willis et al 39 provided an appropriate guide to inform the methodology for the IPT development for this review. In light of this method, this study developed an IPT framework focusing on what potential actions, contexts, mechanisms, distal outcomes and proximal outcomes may be of relevance to scale-up and adaptation. A particular focus of this IPT framework was the identification of potential actions (eg, definition of roles, use of feedback loops, etc), and how these can achieve the outcome of adaptation for local fit. This IPT framework will provide a theoretical map for further exploration in the following stages of the review.
Purposeful and iterative searching was undertaken for this first stage of the realist review to inform the content of this IPT framework.39 An initial scoping search was undertaken for scale-up and adaptation in healthcare to get an overview of the available literature in the field. There was a large volume returned with many articles discussing the need to adapt for local fit, however without giving guidance for how, why or when to complete this when scaling-up. As a result, a decision was made to particularly focus on guidance and frameworks relating to scale-up, adaptation and fidelity to prioritise identification of what actions could be taken (eg, create opportunities for learning, giving guidance to sites, etc) to make adaptations when scaling-up. Guidance and frameworks were identified from the initial scoping search results, in addition to use of reference lists, in particular of recent reviews in the field of implementation and scale-up by Milat et al,54 Subramanian et al,55 Nilsen56 and also use of the ExpandNet bibliography. This was complimented by input from the review team (with backgrounds in global health and health systems), and two further experts (in the fields of fidelity and of implementation research) to highlight and direct to any further relevant literature (figure 2). The frameworks included can be seen in online supplementary file 1.
bmjopen-2018-022084supp001.pdf (522.4KB, pdf)
A challenge of developing an IPT framework in a realist review is finding a level of abstraction that allows the recognition of demi-regularities among the detail and variation in the evidence, while being specific enough to answer the review question.39 The IPT framework went through revisions aiming to keep the actions, contexts, mechanisms and outcomes that were deemed most relevant to adaptation and scale-up, rather than those relating to scale-up in general. Decision making was recorded in the research logbook (see online supplementary file 2 for an example from the research logbook). Causation between the potential actions, contexts, mechanisms and outcomes were not made at this stage and will be added iteratively as the review progresses and scope is refined. The contexts were placed under headings adapted from the socioecological model57 to aid organisation. The IPT framework can be seen in figure 3 (see online supplementary file 1 for the guidance and frameworks identified which informed this).
bmjopen-2018-022084supp002.pdf (524.3KB, pdf)
The IPT framework will assist in (i) initial coding of actions, contexts, mechanisms and outcomes for data extraction in stage 4, and will inform a codebook for reviewers (while also allowing for new actions, contexts, mechanisms and outcomes to emerge), (ii) providing an initial framework for the synthesis to assist in organisation of the CMOCs and demi-regularities in stage 5 (figure 1). As mentioned, coding and synthesis of findings will be guided by this IPT framework; however, new actions, contexts, mechanisms and outcomes will be identified from the data and added to this as they emerge. Thus, this review will add to and refine the framework as the stages progress.
Clarifying the scope of the review
As learning progressed it was noted that much guidance and frameworks in the implementation and scale-up literature in healthcare may be untested and largely theoretical in nature. Therefore, the scope of the review was refined to include specific examples of scale-up and adaptation in practice. Decision making while clarifying the scope of the review was documented in the research logbook (see example in online supplementary file 2). The scope of this review may be further refined in an iterative process as the review progresses, as per realist guidance,43 and will be documented in the research logbook for transparency. This iterative focusing of the review may be carried out based on findings from the examples of scale-up in practice and in consultation with stakeholders.
The remaining section presents the protocol for stages 2–6 detailing the methods that will be used throughout the remainder of the realist review which will be carried out from June 2018 to March 2019. An overview of the stages and details can also be seen in figure 4.
Stage 2: search strategy
Stage 2 will involve a search of scale-up and adaptation in practice. Scoping and pilot searches were completed throughout stage 1 and a librarian was consulted to help inform the selection of databases and concept headings for use in stage 2. A systematic search will be completed using the concept headings of scale-up, context (contextualise, adapt, tailor, redesign, etc) and healthcare. Search terms will be adapted for each database. Search databases will include: PubMed, CINAHL, Global Indicus Medicus (WHO library including both academic and grey literature), SCOPUS, EMBASE and PsychINFO. For further grey literature searching, Social Care Institute for excellence (SCIE), Open Grey and Greylit will be used. Searching of reference lists from identified papers will be carried out along with forward citation searching using Google Scholar. Additionally, the corresponding author from the articles selected will be contacted to identify other articles on their scale-up example that could be relevant to answering the research question.
Further rounds of searches may be completed in later stages of the review in keeping with the iterative nature of realist reviews.50 This may be to search for further evidence or wider theories that may explain findings and assist in theory refinement. The need for searches, search terms and strategies will be identified as the review progresses. These will be documented in the research logbook, as they occur.
Stage 3: study selection, criteria and procedures
Inclusion criteria
All articles and sources obtained from stage 2 will subsequently undergo further review for inclusion based on three criteria. To be retained for further review sources must describe (i) an example of scale-up of a healthcare intervention(s) in practice, (ii) adaptations that were made for health intervention(s) to fit local settings and (iii) discuss in detail actions for adapting health intervention(s) at scale. Both scale-up at national and subnational levels will be included once the intervention was being purposefully expanded to a new wider population group in practice. For inclusion in this study, adaptations will need to have occurred during scale-up to adapt for local contexts, and actions used (eg, local decision making) have been documented in detail. Articles discussing the adaptations without describing what actions were used to adapt the intervention will also be excluded. Studies where the adaptations occurred during the randomised controlled trial or pilot stage, and the same intervention was rolled out nationally (or subnationally) without further adaptations to the content, context or delivery, will be excluded. Both positive and negative adaptations may be included (see online supplementary file 3 for more details).
bmjopen-2018-022084supp003.docx (117.7KB, docx)
As scale-up occurs over a long-time period, with an estimated 15 years to reach national scale, no time limit will be placed on evidence.6 Keeping the time period open allows for documents published at the beginning of scale-up projects to also be captured in the search. Searches will be carried out in English. Languages will be limited to those spoken by the review team: English, Spanish, Portuguese and French.
Study appraisal: relevance and rigour
As realist philosophy does not exclude evidence based on type of study, Realist And Meta-narrative Evidence Syntheses: Evolving Standards (RAMESES) guidelines will be applied to assess the relevance and rigour of the studies.43 As discussed by Pawson,44 useful information can arise from studies which may not be prioritised on a traditional hierarchy of evidence. Each retained evidence or document will be assessed in terms of its relevance to the research question and whether it is rigorous enough to hold value to theory building, testing or refinement. Any exclusions based on these criteria will be documented in the research logbook.
Procedures
Title and abstract screening will be completed. Following this two reviewers will complete full-text screening independently, with a third reviewer available to resolve any conflicts should they arise. Depending on the number of documents with examples of scale-up returned, further refinement of the scope of the review may be decided by the review team. This will be documented in the research logbook. For further searches as they arise in an iterative fashion, selection criteria will be decided by the review team and will be based on the ability of studies to further refine theory.
Stage 4: data extraction
The following data will be extracted from the scale-up examples identified in stage 3: (i) what adaptations were made, (ii) what actions were used to make these adaptations and (iii) the contexts, mechanisms and outcomes that relate to these actions (in the form of CMOCs). A data extraction form and codebook will be developed from the IPT framework to guide data extraction for each scale-up example (see online supplementary file 4 and 5 for draft versions, noting that these will be refined as the review progresses). Where multiple documents relate to the same example of scale-up, these will be combined into one data extraction form for that case example, and the supporting quotes referenced as to which document it originated from.
Adaptations
It has previously been identified that adaptations are often poorly reported in research.10 33 58 59 While some adaptations may reflect small, surface-level changes, others may reflect large deep structural adaptations. To systematically capture the type of adaptations made, Stirman et al 10 taxonomy of modifications will be used to assist categorisation of adaptations, including what type of adaptations were made, who made them and at what level.
Actions
A description of what action(s) was/were carried out to achieve the adaptation(s) will be extracted from the examples to the data extraction form. These may relate to the potential actions identified in the IPT framework and resultant codebook, or may reflect new actions emerging from the data. Any new actions will be categorised and added to the codebook as they emerge.
CMOCs relating to the actions
CMOCs relating to the actions and adaptations will be extracted from each case example. Note that a case example may contain multiple sources (ie, a peer-reviewed article and a national report on the same scale-up) or just one source. While some CMOCs may relate to the potential actions, contexts, mechanisms and outcomes identified in the IPT framework and resultant codebook, others may reflect new CMOCs emerging from the data. Quotes and descriptions will be taken from the text to support these CMOCs. Abductive reasoning will also be applied for any inferred contexts or mechanisms and the reasoning stated on the data extraction form.39 47
Once the above steps are completed for each case example, an example will be presented to members of the Irish Realist Researcher Group for feedback on the coding procedures to inform refinements of the methods and codebook as needed (online supplementary file 4 and 5). This is a group of 8–10 researchers with experience in realist methods. Following this, the completed data extraction forms for each case will be reviewed again by the first reviewer using the updated coding procedures. A second reviewer will then take a random sample of 10% of the scale-up case examples for extraction following the same coding procedures using a data extraction form for these case examples. The reviewers will then discuss and compare the CMOCs extracted and reach agreement, if differences occur. Following this, the remaining data extraction forms for all case examples will be reviewed by the second reviewer and agreement reached between reviewers on the CMOCs, including any inferred contexts or mechanisms and reasoning for the same. A third reviewer will be available for input or to resolve any discrepancies between the first and second reviewer.
Stage 5: data synthesis
The findings from each example of scale-up will then be synthesised across cases. The data extraction forms from each case example will be uploaded and coded in NVivo, using the IPT framework and resultant codebook to guide initial codes. New codes will be added or refined as they emerge, thus adding to the theoretical framework as the synthesis progresses.
The type of adaptations made will be synthesised across cases to give a picture of what adaptations are happening in practice. Then the actions used to achieve adaptations will be synthesised. To explore how these actions achieved adaptations, the CMOCs identified from each case example will be coded in NVivo to look for demi-regularities occurring across the different case examples. Therefore some, but not all, of the CMOCs from each case example may be identified based on whether demi-regularities were seen and if they add value to theory building and refinement at this stage. These results will be synthesised to make further sense of the findings and refine the theory. The following conceptual tools may be used as needed to assist in this theory refinement43 60: (i) juxtaposing, where evidence from one setting may aid explanation of outcomes from another, (ii) reconciliation, where differences are identified to explain findings which may contradict each other, (iii) adjudication between studies, (iv) consolidation, for example, by building multiple explanations, and (iv) situating by identifying what may happen in one setting compared with another. Reasoning will be documented in a research logbook.
Numerous theories may emerge from the literature. Therefore, further focusing of the review in an iterative fashion may be required. Focusing on particular theories may be guided by demi-regularities occurring across examples. However, it is acknowledged that frequency of occurrence may not necessarily correlate to importance in practice. Therefore, if certain areas are highlighted as particularly critical for successful adaptation by the literature this may also assist focusing of the review. This will be further be guided by stakeholder involvement to give a ‘reality check’,41 aiming to ensure the review will focus on what is of relevance and importance to those in practice. Decision making for this process will be recorded in the research logbook for transparency.
Abductive reasoning47 and retroduction48 will be used to guide the review to interpret and explain the findings and put forward contextual conditions that may need to be present for the outcomes to occur. As part of this process, wider substantive theory will be searched for to assist in explanation of the findings and for further theory refinement.43 This will lead to the development of theory relating to how adaptations can be made when scaling-up.
Stage 6: stakeholder involvement
Finally, stakeholders with experience of adaptation and scale-up in both research and practice internationally will be sought and contacted to assist in theory refinement. Initial stakeholders will be identified and contacted by the research team. Stakeholders will also be asked to identify further persons in their field of expertise. Initial review findings will be presented to stakeholders and their opinions sought, based on their practical knowledge and expertise.41 The resultant theories may be further refined and the review focused based on learning from this process.61 This involvement of stakeholders with experience in adaptation and scale-up through research and practice, will assist in focusing the review and its findings, and thus may ensure the findings are useful in practice for implementers in the field. The involvement of stakeholders will allow for initial dissemination of the research findings. This stage may inform further searches through focusing of the review as needed in an iterative fashion.
A summary of the stages and proposed actions for this realist review can be found in figure 4. The above stages will be carried out as per realist review methodology in an iterative fashion, allowing for refinement of theory and the scope of the review and subsequent searches as learning progresses. Any iterations to the above protocol will be captured in the research logbook and reported in the final dissemination of the research. The above stages set out to achieve the study’s objectives of discovering what actions can be used to achieve adaptations when scaling-up health interventions for local fit, by what mechanisms do these actions work and what contextual factors may influence this. It is hoped this approach will provide practical and useful findings for implementers in the field of scale-up.
Patient and public involvement
Members of the public and patients were not involved in the development of this protocol.
Ethics and dissemination
The dissemination of the findings of this review will follow the RAMESES reporting guidelines.62 The results of this review will be used to put forward theory to explain what and how actions can be used to influence and achieve adaptations when scaling-up for local fit. Use of a realist review methodology, with the stages outlined above, allows for an exploration of the complexity of the process of scale-up across diverse contexts, and the identification of the contextual factors that may influence actions, and by what mechanisms these may work. By including stakeholders with experience in the field of adaptation and scale-up, it is hoped this will add to theory development and refinement, and will help ensure that findings have practical utility for implementers. The findings of this study will be published in a peer-reviewed journal, through conference presentations and dissemination through stakeholder involvement in theory refinement. The review will also be published as a PhD thesis, available through Trinity College Dublin library.
bmjopen-2018-022084supp004.pdf (323.3KB, pdf)
bmjopen-2018-022084supp005.pdf (573.4KB, pdf)
Supplementary Material
Acknowledgments
The authors would like to thank Dr Ayat Abu-Agla from the Centre for Global Health, Trinity College Dublin for lending her expertise to stage one of this review.
Footnotes
Patient consent for publication: Not required.
Contributors: JP is lead reviewer completing this study as part of a PhD. JP developed the general study concept and outline. JP completed the literature review and development of the IPT framework and led the iterations. JP developed the search strategy, and methods for selection, appraisal, extraction, synthesis and stakeholder involvement. BG contributed to IPT framework refinement, the methods for data extraction and coding processes using NVivo and the methods for synthesising across cases. FV contributed to the general study concept and design, clarifying the scope of the review and refinement of the IPT framework. ET contributed as an expert in fidelity in stage one and assisted in refinement of the IPT framework. ET contributed to the methods of the search strategy and methods of the data extraction and synthesis stages, including the role of second reviewer. HM contributed to conceptual study design, and clarifying the scope of the review by refinement of research question and IPT framework. EM contributed to conceptual study design, refinement of research question and IPT framework, the search strategy, refinement of the data extraction form and contributed to the method of involvement of stakeholders for theory refinement. JP wrote the first draft of the paper, which was reviewed and contributed by all other authors. All authors approved the final version.
Funding: This research is supported by the Health Research Board Trials Methodology Research Network (HRB TMRN) of Ireland and Trinity College Dublin through support of a PhD studentship and training.
Competing interests: None declared.
Ethics approval: Ethical approval for this study was received from the Health Policy and Management and Centre for Global Health Research Ethics Committee of Trinity College Dublin, Ireland in March 2017.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
- 1. Simmons R, Fajans P, Ghiron L. Scaling up health service delivery: from pilot innovations to policies and programmes. Geneva: ExpandNet, World Health Organisation 2007:1–208. [Google Scholar]
- 2. Barker PM, Reid A, Schall MW. A framework for scaling up health interventions: lessons from large-scale improvement initiatives in Africa. Implement Sci 2016;11:12 10.1186/s13012-016-0374-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Gitlin LN. Introducing a new intervention: an overview of research phases and common challenges. Am J Occup Ther 2013;67:177–84. 10.5014/ajot.2013.006742 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Green LW, Ottoson JM, García C, et al. . Diffusion theory and knowledge dissemination, utilization, and integration in public health. Annu Rev Public Health 2009;30:151–74. 10.1146/annurev.publhealth.031308.100049 [DOI] [PubMed] [Google Scholar]
- 5. Kruk ME, Yamey G, Angell SY, et al. . Transforming global health by improving the science of scale-up. PLoS Biol 2016;14:e1002360 10.1371/journal.pbio.1002360 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Cooley L, Ved R. Scaling up–from vision to large‐scale change: A management framework for practitioners. Washington DC: Management Systems International, 2012:1–54. [Google Scholar]
- 7. Aarons GA, Sklar M, Mustanski B, et al. . “Scaling-out” evidence-based interventions to new populations or new health care delivery systems. Implement Sci 2017;12:111 10.1186/s13012-017-0640-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Yamey G. What are the barriers to scaling up health interventions in low and middle income countries? A qualitative study of academic leaders in implementation science. Global Health 2012;8:11 10.1186/1744-8603-8-11 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Mangham LJ, Hanson K. Scaling up in international health: what are the key issues? Health Policy Plan 2010;25:85–96. 10.1093/heapol/czp066 [DOI] [PubMed] [Google Scholar]
- 10. Stirman SW, Miller CJ, Toder K, et al. . Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implement Sci 2013;8:65 10.1186/1748-5908-8-65 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. World Health Organisation. International classification of health interventions. 2016. http://www.who.int/classifications/ichi/en/ (accessed 19th Dec 2017).
- 12. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci 2013;8:117 10.1186/1748-5908-8-117 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. McKleroy VS, Galbraith JS, Cummings B, et al. . Adapting evidence-based behavioral interventions for new settings and target populations. AIDS Educ Prev 2006;18:59–73. 10.1521/aeap.2006.18.supp.59 [DOI] [PubMed] [Google Scholar]
- 14. Castro FG, Barrera M, Martinez CR. The cultural adaptation of prevention interventions: resolving tensions between fidelity and fit. Prev Sci 2004;5:41–5. 10.1023/B:PREV.0000013980.12412.cd [DOI] [PubMed] [Google Scholar]
- 15. Pérez D, Van der Stuyft P, Zabala MC, et al. . A modified theoretical framework to assess implementation fidelity of adaptive public health interventions. Implement Sci 2016;11:91 10.1186/s13012-016-0457-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Bauman LJ, Stein RE, Ireys HT. Reinventing fidelity: the transfer of social technology among settings. Am J Community Psychol 1991;19:619–39. 10.1007/BF00937995 [DOI] [PubMed] [Google Scholar]
- 17. Robert R, Fulop N. The role of context in successful improvement. Perspectives on context A selection of essays considering the role of context in successful quality improvement. London: The Health Foundation, 2014:31–58. [Google Scholar]
- 18. Skibiak JP, Mijere P, Zama M. Expanding contraceptive choice and improving quality of care in Zambia’s Copperbelt: Moving from pilot projects to regional programmes In: Simmons R, Fajans P, Ghiron L, eds Scaling Up Health Service Delivery From Pilot Innovations to Policies and Programmes. Geneva: Expandnet, World Health Organisation, 2007. [Google Scholar]
- 19. Centre for Substance Abuse Prevention USA, Substance Abuse and Mental Health Services Administration, US Department of Health and Human Services. Finding the Balance: Program Fidelity and Adaptation in Substance Abuse Prevention. 2002. A State-of-the-art review https://www.csun.edu/sites/default/files/FindingBalance1.pdf (accessed 6th Nov 2017).
- 20. Holliday J, Audrey S, Moore L, et al. . High fidelity? How should we consider variations in the delivery of school-based health promotion interventions? Health Educ J 2009;68:44–62. 10.1177/0017896908100448 [DOI] [Google Scholar]
- 21. Peters DH, Tran NT, Adam T. Implementation Research in Health. A Practical Guide. Geneva: Alliance for Health Policy and Systems Research, World Health Organisation, 2013. [Google Scholar]
- 22. Lara M, Bryant-Stephens T, Damitz M, et al. . Balancing “fidelity” and community context in the adaptation of asthma evidence-based interventions in the “real world”. Health Promot Pract 2011;12:63S–72. 10.1177/1524839911414888 [DOI] [PubMed] [Google Scholar]
- 23. Paina L, Peters DH. Understanding pathways for scaling up health services through the lens of complex adaptive systems. Health Policy Plan 2012;27:365–73. 10.1093/heapol/czr054 [DOI] [PubMed] [Google Scholar]
- 24. Parry GJ, Carson-Stevens A, Luff DF, et al. . Recommendations for evaluation of health care improvement initiatives. Acad Pediatr 2013;13:S23–S30. 10.1016/j.acap.2013.04.007 [DOI] [PubMed] [Google Scholar]
- 25. Byrne A, Morgan A, Soto EJ, et al. . Context-specific, evidence-based planning for scale-up of family planning services to increase progress to MDG 5: health systems research. Reprod Health 2012;9:27 10.1186/1742-4755-9-27 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Dunn CE, Le Mare A, Makungu C. Malaria risk behaviours, socio-cultural practices and rural livelihoods in southern Tanzania: implications for bednet usage. Soc Sci Med 2011;72:408–17. 10.1016/j.socscimed.2010.11.009 [DOI] [PubMed] [Google Scholar]
- 27. Aral SO, Cates W. Coverage, context and targeted prevention: optimising our impact. Sex Transm Infect 2013;89:336–40. 10.1136/sextrans-2012-050707 [DOI] [PubMed] [Google Scholar]
- 28. Audureau E, Kahn JG, Besson MH, et al. . Scaling up prevention of mother-to-child HIV transmission programs in sub-Saharan African countries: a multilevel assessment of site-, program- and country-level determinants of performance. BMC Public Health 2013;13:286 10.1186/1471-2458-13-286 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Rasschaert F, Koole O, Zachariah R, et al. . Short and long term retention in antiretroviral care in health facilities in rural Malawi and Zimbabwe. BMC Health Serv Res 2012;12:444 10.1186/1472-6963-12-444 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Sorsdahl K, Stein DJ, Lund C. Mental health services in South Africa: scaling up and future directions. Afr J Psychiatry 2012;15:168–71. 10.4314/ajpsy.v15i3.21 [DOI] [PubMed] [Google Scholar]
- 31. World Health Organisation, ExpandNet. Beginning with the end in mind: planning pilot projects and other programmatic research for successful scaling up. Geneva: Expandnet, World Health Organisation, 2011:1–18. [Google Scholar]
- 32. Moore JE, Bumbarger BK, Cooper BR. Examining adaptations of evidence-based programs in natural contexts. J Prim Prev 2013;34:147–61. 10.1007/s10935-013-0303-6 [DOI] [PubMed] [Google Scholar]
- 33. Sampson M, Torres LR. What tension between fidelity and cultural adaptation? A reaction to marsiglia and booth. Res Soc Work Pract 2015;25:828–31. 10.1177/1049731515599067 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Bradley EH, Curry LA, Taylor LA, et al. . A model for scale up of family health innovations in low-income and middle-income settings: a mixed methods study. BMJ Open 2012;2:e000987 10.1136/bmjopen-2012-000987 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Solomon J, Card JJ, Malow RM. Adapting efficacious interventions: advancing translational research in HIV prevention. Eval Health Prof 2006;29:162–94. 10.1177/0163278706287344 [DOI] [PubMed] [Google Scholar]
- 36. Tortolero SR, Markham CM, Parcel GS, et al. . Using intervention mapping to adapt an effective HIV, sexually transmitted disease, and pregnancy prevention program for high-risk minority youth. Health Promot Pract 2005;6:286–98. 10.1177/1524839904266472 [DOI] [PubMed] [Google Scholar]
- 37. Wingood GM, DiClemente RJ. The ADAPT-ITT model: a novel method of adapting evidence-based HIV Interventions. J Acquir Immune Defic Syndr 2008;47 Suppl 1:40–6. 10.1097/QAI.0b013e3181605df1 [DOI] [PubMed] [Google Scholar]
- 38. Kilbourne AM, Neumann MS, Pincus HA, et al. . Implementing evidence-based interventions in health care: application of the replicating effective programs framework. Implement Sci 2007;2:42 10.1186/1748-5908-2-42 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Willis CD, Riley BL, Stockton L, et al. . Scaling up complex interventions: insights from a realist synthesis. Health Res Policy Syst 2016;14:88 10.1186/s12961-016-0158-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Pawson R, Greenhalgh T, Harvey G, et al. . Realist review - a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy 2005;10:21–34. 10.1258/1355819054308530 [DOI] [PubMed] [Google Scholar]
- 41. Brennan N, Bryce M, Pearson M, et al. . Understanding how appraisal of doctors produces its effects: a realist review protocol. BMJ Open 2014;4:e005466 10.1136/bmjopen-2014-005466 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Pawson R, Tilley N. Realistic Evaluation. London: Sage, 1997. [Google Scholar]
- 43. Wong G, Greenhalgh T, Westhorp G, et al. . Realist Synthesis. 2013. RAMESES Training Materials http://www.ramesesproject.org/ (accessed 1 Mar 2017).
- 44. Pawson R. Digging for Nuggets: How ‘Bad’ Research Can Yield ‘Good’ Evidence. Int J Soc Res Methodol 2006;9:127–42. 10.1080/13645570600595314 [DOI] [Google Scholar]
- 45. Wong G, Greenhalgh T, Pawson R. Internet-based medical education: a realist review of what works, for whom and in what circumstances. BMC Med Educ 2010;10:12 10.1186/1472-6920-10-12 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46. Dalkin SM, Greenhalgh J, Jones D, et al. . What’s in a mechanism? Development of a key concept in realist evaluation. Implement Sci 2015;10:49 10.1186/s13012-015-0237-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47. Jagosh J, Pluye P, Wong G, et al. . Critical reflections on realist review: insights from customizing the methodology to the needs of participatory research assessment. Res Synth Methods 2014;5:131–41. 10.1002/jrsm.1099 [DOI] [PubMed] [Google Scholar]
- 48. Danermark B, Ekstrom M, Jakobsen L, et al. . Generalization, scientific inference and models for an explanatory social science : Danermark B, Ekstrom M, Jakobsen L, Explaining Society: An Introduction to Critical Realism in the Social Sciences. London and. New York: Routledge, 2002:73–114. [Google Scholar]
- 49. Eastwood JG, Jalaludin BB, Kemp LA. Realist explanatory theory building method for social epidemiology: a protocol for a mixed method multilevel study of neighbourhood context and postnatal depression. Springerplus 2014;3:12 10.1186/2193-1801-3-12 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50. Pawson R, Greenhalgh T, Harvey G, et al. . Realist synthesis: an introduction. RMP Methods Paper 2/2004. Manchester, UK: ESRC Research Methods Programme, University of Manchester, 2004:1–42. [Google Scholar]
- 51. Weetman K, Wong G, Scott E, et al. . Improving best practise for patients receiving hospital discharge letters: a realist review protocol. BMJ Open 2017;7:e018353 10.1136/bmjopen-2017-018353 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52. Molnar A, O’Campo P, Ng E, et al. . Protocol: realist synthesis of the impact of unemployment insurance policies on poverty and health. Eval Program Plann 2015;48:1–9. 10.1016/j.evalprogplan.2014.09.002 [DOI] [PubMed] [Google Scholar]
- 53. Groot G, Waldron T, Carr T, et al. . Development of a program theory for shared decision-making: a realist review protocol. Syst Rev 2017;6:e018353 10.1186/s13643-017-0508-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54. Milat AJ, Bauman A, Redman S. Narrative review of models and success factors for scaling up public health interventions. Implement Sci 2015;10:113 10.1186/s13012-015-0301-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55. Subramanian S, Naimoli J, Matsubayashi T, et al. . Do we have the right models for scaling up health services to achieve the Millennium Development Goals? BMC Health Serv Res 2011;11:336 10.1186/1472-6963-11-336 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci 2015;10:53 10.1186/s13012-015-0242-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57. Bronfenbrenner U. Toward an experimental ecology of human development. Am Psychol 1977;32:513–31. 10.1037/0003-066X.32.7.513 [DOI] [Google Scholar]
- 58. Pinnock H, Epiphaniou E, Sheikh A, et al. . Developing standards for reporting implementation studies of complex interventions (StaRI): a systematic review and e-Delphi. Implement Sci 2015;10:42 10.1186/s13012-015-0235-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59. Sundell K, Beelmann A, Hasson H, et al. . Novel programs, international adoptions, or contextual adaptations? Meta-analytical results from german and swedish intervention research. J Clin Child Adolesc Psychol 2016;45:784–96. 10.1080/15374416.2015.1020540 [DOI] [PubMed] [Google Scholar]
- 60. Pawson R, Evidence-based Policy. A Realist Perspective. London: Sage, 2006:1–208. [Google Scholar]
- 61. Saul JE, Willis CD, Bitz J, et al. . A time-responsive tool for informing policy making: rapid realist review. Implement Sci 2013;8:103 10.1186/1748-5908-8-103 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62. Wong G, Greenhalgh T, Westhorp G, et al. . RAMESES publication standards: realist syntheses. BMC Med 2013;11:21 10.1186/1741-7015-11-21 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2018-022084supp001.pdf (522.4KB, pdf)
bmjopen-2018-022084supp002.pdf (524.3KB, pdf)
bmjopen-2018-022084supp003.docx (117.7KB, docx)
bmjopen-2018-022084supp004.pdf (323.3KB, pdf)
bmjopen-2018-022084supp005.pdf (573.4KB, pdf)