Abstract
Effectiveness of health interventions can be substantially impaired by implementation failure. Context-driven implementation strategies are critical for successful implementation. However, there is no practical, evidence-based guidance on how to map the context in order to design context-driven strategies. Therefore, this practice paper describes the development and validation of a systematic context-mapping tool. The tool was cocreated with local end-users through a multistage approach. As proof of concept, the tool was used to map beliefs and behaviour related to chronic respiratory disease within the FRESH AIR project in Uganda, Kyrgyzstan, Vietnam and Greece. Feasibility and acceptability were evaluated using the modified Conceptual Framework for Implementation Fidelity. Effectiveness was assessed by the degree to which context-driven adjustments were made to implementation strategies of FRESH AIR health interventions. The resulting Setting-Exploration-Treasure-Trail-to-Inform-implementatioN-strateGies (SETTING-tool) consisted of six steps: (1) Coset study priorities with local stakeholders, (2) Combine a qualitative rapid assessment with a quantitative survey (a mixed-method design), (3) Use context-sensitive materials, (4) Collect data involving community researchers, (5) Analyse pragmatically and/or in-depth to ensure timely communication of findings and (6) Continuously disseminate findings to relevant stakeholders. Use of the tool proved highly feasible, acceptable and effective in each setting. To conclude, the SETTING-tool is validated to systematically map local contexts for (lung) health interventions in diverse low-resource settings. It can support policy-makers, non-governmental organisations and health workers in the design of context-driven implementation strategies. This can reduce the risk of implementation failure and the waste of resource potential. Ultimately, this could improve health outcomes.
Keywords: diagnostics and tools, health policies and all other topics, public health, asthma, chronic obstructive pulmonary disease
Summary box.
Implementation failure can seriously impair the effectiveness of health interventions, thereby draining resource potential and undermining health outcomes.
Context-driven implementation strategies can help to avoid failure. To design such strategies, practical guidance is needed on how to map local contexts.
We developed and validated the Setting-Exploration-Treasure-Trail-to-Inform-ImplementatioN-strateGies (SETTING-tool) for mapping local health beliefs and behaviours.
The SETTING-tool proved highly feasible, acceptable and effective in four diverse countries across the globe, demonstrating its wide applicability.
Introduction
Evidence-based interventions consistently fail to be implemented into routine practice, resulting in a substantial waste of resource potential and undermining health outcomes.1 2 Implementation—the act of carrying an intervention into effect3—is a typically effortful, complex process.4 Context-driven implementation strategies help to avoid implementation failure.2 3 5–10 However, there is no evidence-based, practical guide on how to first understand and map contexts in order to design such strategies.
Context can be defined as a set of unique characteristics and circumstances surrounding the implementation effort.11 A critical dimension of context pertains to local attitudes and knowledge regarding an intervention.8 Health beliefs and risk perceptions shape behaviour, ranging from perceived causes of a disease to perceived barriers to change disease-related behaviour.12–14 For example, it would be unfeasible to motivate a community to instal kitchen chimneys to reduce indoor smoke, when local beliefs entail that ghosts enter homes via the chimneys. However, implementers are often unaware of such beliefs.15
Beliefs and behaviours are mostly not adequately mapped as it is resource-consuming and time-consuming.15 Moreover, there are no standardised practical methodologies available to guide context-mapping efforts.6 16 17 The identification, development and testing of practical implementation methodologies are now considered top priorities in implementation science.16 Especially clinical researchers without implementation science expertise would benefit from systematic guidance to enable them to reproduce evidence-based methodologies in practice.18
Paradoxically, implementation evidence is particularly limited for low-resource settings, where the global burden of disease is highest and resources are scarcest.9 19 Therefore, we aimed to develop and validate a practical, systematic context-mapping tool for assessing local beliefs and behaviours in low-resource settings. To validate the tool, it was tested during a research project on chronic respiratory diseases (CRDs). CRDs are a major burden to low-resource settings; 90% of chronic obstructive pulmonary disease (COPD)-related deaths and 80% of asthma-related deaths occur in low-income and middle-income countries (LMICs).20In this paper, we describe the development and validation of the Setting-Exploration-Treasure-Trail-to-Inform-implementatioN-strateGies (SETTING-tool) in practice, and report on its feasibility, acceptability and effectiveness.
Development and validation of the context-mapping tool
With a multidisciplinary expert panel including professionals from four diverse low-resource countries, we employed a multistage and multimethod approach to develop a context-mapping tool (see online supplemental appendix 1). The approach was similar to the approach for the development of the ImpRes tool.18 We first generated components for the tool based on a pragmatic literature search and methodological principles considered state-of-the art in our fields (see online supplemental appendix 1). A principle would be, for example, that mixed-method designs generally provide better insights than qualitative or quantitative research alone). Together with the expert panel, we then detailed the components (eg, use a ‘Rapid Assessment Process (RAP) for qualitative data collection’) based on expected feasibility, acceptability and effectiveness. We discussed these components until consensus was reached, and consolidated these into a comprehensive step-by-step tool. The resulting context-mapping tool had a stepwise approach, and was named the SETTING-tool (figure 1). Next, a research team sequentially validated the SETTING-tool prospectively in six diverse low-resource settings across four countries (see online supplemental appendix 2) within the Free Respiratory Evaluation and Smoke-exposure reduction by primary Health cAre Integrated gRoups (FRESH AIR) project (Box 1). Feasibility and acceptability of the tool were evaluated using the modified Conceptual Framework for Implementation Fidelity.21Effectiveness of the tool was assessed by determining the degree to which its application resulted in context-driven adjustments in the implementation strategies of subsequent FRESH AIR lung health interventions. Results guided iterative fine-tuning of the tool for the next setting. Online supplemental appendix 1 provides a detailed description of the full methodology for the development and evaluation of the tool.
Box 1. The Horizon 2020 FRESH AIR study.
The FRESH AIR project (2015–2018) studied the implementation of evidence-based chronic respiratory disease-related interventions in low-resource settings across the globe: in Uganda, Kyrgyzstan, Vietnam and Greece. The project included preventive interventions (reducing exposure to tobacco smoke and biomass smoke), diagnostic (eg, diagnosing asthma in young children, online spirometry training) and treatment interventions (smoking cessation, pulmonary rehabilitation). The design of the implementation strategies of these interventions was context-driven, and informed by the application of the Setting-Exploration-Treasure-Trail-to-Inform-implementatioN-strateGies-tool.
https://www.ipcrg.org/projects/research/fresh-air (Trial ID NTR5759; trialregister.nl/trial/5644).
bmjgh-2020-003221supp001.pdf (484.5KB, pdf)
Application of the SETTING-tool in the FRESH AIR project
The tool is discussed step by step, including the rationale for each step and its application in the FRESH AIR project (see also table 1).
Table 1.
STEP | Content | Rationale |
1: Coset study priorities | Define end-users of the study data, identify other stakeholders, and actively engage all. Then explore the needs and coformulate the exact aim and scope of the context assessment—guided by a theoretical framework, the setting and other priorities. | Identifying and engaging stakeholders from the beginning enables that the study meets the stakeholders’ needs, which can be crucial in fostering uptake of the findings into practice later.3 7 22–25
A theoretical framework helps to guide the specification of the aim and methodological orientation.26–29 Implementation research generally aims to produce knowledge applicable across various settings.24 |
2: Cocreate a mixed-method study design: combine a rapid assessment with a survey | Cocreate the mixed-method study design. Include (1) a qualitative rapid assessment process (RAP) in which data are collected by multiple methods, sources and researchers, and 2) a quantitative survey. Triangulate all findings. A RAP: The key principles of a RAP are (1) a system (community) perspective, (2) rapid, in-depth and iterative data collection and analysis, and (3) triangulation of data. The RAP research team immerses into a community for a short period and collects data simultaneously in subgroups using multiple methods. |
Whereas the researchers can provide input on the use of evidence-based methods and a theoretical framework,26 27 the end-users and other stakeholders can provide input on the compatibility of these methods with the local context. This collaboration furthermore stimulates the stakeholders’ engagement.3 7
Mixed-method designs generally provide a better understanding of situations than qualitative or quantitative research alone.3 32 Triangulation of findings optimises validity of research results.34 With a RAP, data collection is highly flexible and driven by local needs.33 35 |
3: Create context-sensitive study materials with high validity | Use evidence-based components where available throughout every substep, in our case: use (1) a theoretical framework, (2) a syndromic approach, (3) a vignette, (4) validated questionnaires, (5) a careful translation process (including translation, back-translation and according adjustments) and (6) pilot-testing of the materials. | The framework helps to theoretically underpin material development.26–29
A syndromic (symptom-based) approach helps the participants to identify with the phenomenon studied, when a low awareness about it is expected. Use of a vignette can help to discuss topics more openly, while avoiding stigmatisation or while addressing sensitive topics.39 |
4: Collect data with a trained, diverse team including community researchers | Set up a diverse team, with local (community) researchers with an insiders’ perspective combined with researchers with an external perspective. Train the team members by an experienced researcher. |
Diversity in the team helps to enlighten the research topic from multiple perspectives and enrich the data.35
The right community researchers are trusted by their community and have a thorough understanding of local networks and local health beliefs and behaviours. The combination with the ‘outsiders’ perspectives’ helps to point out remarkable, typical local themes taken for granted by the community team members. Training secures ethical standards during data collection and enhances uniformity and high-quality data collection.26 |
5: Analyse data pragmatically and/or in-depth | Decide to analyse the data pragmatically and/or in-depth, depending on the objective. | Ensure to timely inform the implementation design for related health interventions, which may have to be supplemented by more in-depth analyses for scientific purposes. For feasibility, findings from pragmatic analyses via a ‘targeted approach’ can also be effective.27 |
6: Disseminate findings and promote data use (continuous step) | Continuously communicate relevant findings to the end-users and other stakeholders using a tailored message and delivery strategy. | Frequent communication with end-users and other stakeholders promotes sustained engagement and alignment, and uptake of the findings.7 23 47 48
A systematic, practical tool designed to effectively communicate research findings can guide in facilitating knowledge uptake by policy-makers.23 A tailored message and delivery strategy benefit information use.23 49 |
SETTING-tool, Setting-Exploration-Treasure-Trail-to-Inform-implementatioN-strateGies-tool.
STEP 1: Coset study priorities (figure 2)
As a first step, we (the context-mapping researchers) collaboratively set the study priorities with our end-users and other influential stakeholders. We defined end-users of our study findings as (1) our FRESH AIR consortium colleagues, who would use the data to design context-driven implementation strategies for their planned interventions and (2) future implementers in similar settings, such as policy-makers, non-governmental organisations (NGOs) and health workers. The other influential stakeholders we identified, ranged from the Minister of Health to local community members (CMs). There were diverse in terms of age, sex and educational background. They were engaged in Stakeholder Engagement Groups (see online supplemental appendix 3 table E1). Stakeholders received no financial compensation for participation. We engaged stakeholders from the beginning to ensure the study would closely meet their needs; this can be crucial in fostering uptake of the findings into practice later.3 7 22–25
Together with the FRESH AIR consortium colleagues and the Stakeholder Engagement Groups, we collaboratively set the context-mapping study’s priorities. We formulated the exact study aim: ‘To explore CRD-related beliefs and behaviours in diverse low-resource settings.’ We used a theoretical framework (a combination of three well-stablished health-behaviour frameworks,12–14 online supplemental appendix 4) to further guide the specification of the aim and our methodological orientation.26–29 This framework consisted of factors leading to (intention to conduct) behaviour, such as the locally perceived identity of CRDs, perceived causes, and perceived barriers to change CRD-related behaviour. CRD-related behaviour included risk behaviour (tobacco smoking, cooking/heating in a way that generates household air pollution), help-seeking behaviour by CMs, and helping behaviour by healthcare professionals.
We also determined the study settings. Implementation research generally aims to produce knowledge applicable across various settings,24 and rural settings have a high prevalence of CRD and risk factors to CRD.30 31 Therefore, six rural settings across four countries were selected purposively to represent diversity, and conveniently based on established relationships with the settings (online supplemental appendix 2 further details the settings and the definition of rural). These settings were in Uganda, Kyrgyzstan (a highland and lowland setting), Vietnam and Greece (a Roma camp and a rural Greek setting).
STEP 2: A mixed-method study design: Combine a rapid assessment with a survey (figure 3)
Second, we codesigned the context-mapping study with the end-users (the FRESH AIR consortium colleagues) and other stakeholders. Our researchers’-perspective enabled us to underpin the study with evidence-based methods, whereas the stakeholders ensured high compatibility of the selected methods with the local context. Codesigning the methods furthermore benefited the stakeholders’ engagement.3 7
We cocreated a population-based, cross-sectional, observational design. Mixed-method designs generally provide a better understanding of situations than qualitative or quantitative research alone.3 32 Therefore, we adopted a mixed-method approach enabling to (1) explore and describe qualitatively how people perceive CRD, and what their related behaviour is like, and then 2) quantify the findings. We selected an RAP for the qualitative data collection,33 as it allowed us to timely inform our FRESH AIR colleagues on the implementation strategies of their interventions. We chose to collect the quantitative data using a survey. This enabled us to triangulate our data (to compare data collected through multiple methods, sources, theories and/or researchers) to optimise the validity of our findings (figure 3).34
The qualitative RAP
The key principles of an RAP are (1) a system (community) perspective, (2) rapid, in-depth and iterative data collection and analysis and (3) triangulation of data. We immersed into the rural communities for a short period and collected data simultaneously in subgroups using multiple methods.35 We included semistructured interviews, focus groups, documents and observations of healthcare professionals (respiratory consultations) and CMs (smoking, cooking and heating behaviour). Online supplemental appendix 2 details our planning and field methods.
Informants were selected based on opportunity and diversity (in sex, age, background, etc), to obtain rich data.26 We distinguished three participant groups: healthcare workers (HPs), CMs and key informants (stakeholders with either in-depth knowledge or an overall overview, such as religious leaders) (see online supplemental appendix 3 for inclusion and exclusion criteria). The sample size relied on when data saturation was achieved (when additional research activities would no longer result in additional insights). Preliminary findings were collated in daily debriefings with the entire team, guided by the theoretical framework. New informants were identified, and emerging themes and data saturation discussed. Iterative adaptations to the data collection materials, strategy and planning were also made. Accordingly, data collection was highly flexible and driven by local needs.33
The quantitative survey
We conducted one survey for CMs (200 per setting) and one for HPs (40 per setting). The CMs were randomly selected using a three-stage sampling approach based on the WHO ‘Expanded programme for Immunisation’, because it was especially suitable for rural settings.36 We then invited all eligible HPs from the nearest healthcare facilities.
STEP 3: Create context-sensitive study materials with high validity (figure 4)
Step 3 concerns the development of context-sensitive research materials with high validity. For the field methods selected in step 2, we needed corresponding materials: topic lists for the interviews and focus groups, semistructured forms for the observations, and structured questionnaires for the survey. Our theoretical framework guided the development of the research materials (eg, the construct ‘perceived cause of CRD’) (step 3A, figure 4). We anticipated a low awareness of COPD, asthma and the implications.30 37 38 Hence, to enable informants to relate to the topic we adopted a syndromic (symptom-based) approach (step 3B). As informants could fear stigmatisation for tuberculosis they could be hesitant to speak freely about respiratory symptoms. Therefore, we introduced the topic using a vignette (3C) which detailed a story about someone with slowly progressive chronic respiratory symptoms typical for COPD.39 We then probed using questions such as: ‘Are there people with a similar situation living in your community?’ and ‘What, according to you, is the cause of the situation?’
The questionnaires for CMs and HPs were composed of validated questionnaires (3D). They included demographical questions, the revised version of the brief-Illness Perception Questionnaire,40 CM risk behaviour (solid fuel use for cooking/heating and tobacco use)41–43 and HP treatment behaviour respectively.44 Context-specific adjustments were made (eg, rainy/dry season instead of winter/summer).
The research materials were translated into each local language, back-translated and adjusted accordingly (3E). The qualitative materials were tested internally during our training sessions and iteratively improved throughout the study.27 The quantitative materials were pilot tested on feasibility and content validity among 10% or our targeted sample in each setting (3F). Resulting adaptations were, for example, replacing the Likert-scales (0–10) by multiple choice options, as scales turned out to be too abstract for local participants to comprehend. See online supplemental appendix 5 and online supplemental file 2 for the resulting research materials.
bmjgh-2020-003221supp002.zip (1.7MB, zip)
STEP 4: Collect data with a trained, diverse team including community researchers (figure 5)
In step 4, we set up a well-trained and diverse team to collect the data. It included researchers from the communities, such as local nurses. Their insiders’ perspectives offered deep understanding of local networks and prevailing themes,35 and they facilitated access to and openness of the participants. The combination with the ‘outsiders’ perspectives’ (team members from the country’s universities and Dutch researchers) helped to point out remarkable, typical local themes that were taken for granted by the community team members. Additional diversity in terms of background, age and gender helped to further enlighten the research topic from multiple perspectives and enrich the data.35 Notably, debriefing sessions generated additional data, for example, when community researchers and international researchers discussed explanations for certain findings from their own perspective. All team members received a thorough 1-day training from an experienced researcher, securing ethical standards during data collection and enhancing uniformity and high-quality data collection (online supplemental appendix 3).26 Further information on the data collection and training can be found in online supplemental appendix 3.
STEP 5: Analyse pragmatically and/or in-depth (figure 6)
Step 5 emphasises the important difference between more pragmatic and in-depth analyses. The overall FRESH AIR project was bound to timelines that did not allow for time-consuming in-depth qualitative analyses of the context-assessment before the subsequent lung health interventions had to be implemented. Therefore, we pragmatically used our preliminary qualitative analyses of our debriefing sessions, which we communicated immediately after the RAP. We continued to update partners on the findings during the subsequent in-depth analyses.
For the in-depth analyses of qualitative data, we first transcribed the audio recordings verbatim.26 45 We coded the data both deductively and inductively, using thematic analyses to identify themes.46 Two researchers independently applied The Framework Method for data structuring and reduction (online supplemental appendix 6), compared results and discussed these until consensus was reached.45 We used appropriate qualitative software to facilitate data storing, analyses and sharing (online supplemental appendix 6). Next, data from the observations were charted into a table. These data together with field notes were used to complement and triangulate the data from the interviews and focus groups.
Our quantitative data were analysed using associative analyses and descriptive statistics, in which we calculated frequencies of prevalence.
STEP 6: Disseminate findings and promote data use (continuous step) (figure 7)
As a continuous process, we disseminated the findings using a tailored message and delivery strategy, to optimise information use among the FRESH AIR colleagues and other stakeholders.7 23 23 47–49 We communicated our preliminary findings immediately after the RAP. Further dissemination of our in-depth findings occurred during international scientific conferences. Policy-makers, NGOs and health workers were reached via international and local health conferences in each country.
Validation outcomes
Testing of the SETTING-tool in the six settings demonstrated that feasibility of the tool was high. Generally, all steps were implemented as intended with reasonable effort, and within our budget and time planning (online supplemental appendix 7,8). However, in the Roma setting, we needed to invest a substantial amount of time for building trust with the community. Therefore, we could only test our methodology, but did not achieve data saturation. Acceptability of the tool was consistently high; both the users of the tool and to whom it was applied considered it appropriate, based on their collaborative attitude (online supplemental appendix 7). Lastly, effectiveness of the tool was considered high; the application resulted in findings that led to multiple major context-driven changes to the planning and implementation strategies of the subsequent FRESH AIR interventions (table 2).
Table 2.
Theoretical factors* | Contextual input | Context-driven adaptations |
Perceived identity of CRD | Awareness on CRDs and their implications was considerably lower than anticipated among rural communities and their healthcare professionals in Uganda, Kyrgyzstan and Vietnam. | For the Online Spirometry Trainings to improve knowledge and skills successfully, either the video content had to be adapted fundamentally to be compatible with a more basic level of understanding of CRD, or the trainings needed to be implemented in areas where the level of understanding was higher. In agreement with the stakeholders, we chose the latter strategy. |
Perceived causes/ susceptibility/ cue to action | Awareness on the risk of household air pollution was low in Uganda, Kyrgyzstan and Vietnam; communities did not perceive their traditional cooking habits to be a risk for CRD, and therefore perceived no need for cleaner cooking measures. | The Awareness Programme on CRDs was expanded: the delivery strategy turned into a cascading train-the-trainer programme with a larger reach.56 Household air pollution as a risk factor to lung health was covered extensively, creating a locally perceived need for cleaner cooking interventions. This contributed substantially to a high acceptability of the interventions (Cleaner Cooking and a Midwife-led Smoke-reduction Programme).57 58 |
Perceived causes/ benefits/ norms | A (rural) Kyrgyz norm is that ‘a real man smokes’, while smoking women are despised. The risk of smoke exposure during pregnancy is fairly unknown, and the man’s position in the family does not allow to question his smoking behaviour. Meanwhile, the youngest son in the family is responsible for taking care of his parents later in life (the families’ pension). | In the Kyrgyz Awareness Programme, we emphasised even more on the risk of smoke exposure to (unborn) children, and provided solutions to secondhand smoke exposure (smoking cessation, smoking outdoors).56 This promoted openness to the Awareness Programme and uptake of the Smoking Cessation Counselling intervention (very brief advice).59 |
Perceived causes/ susceptibility | The Vietnamese word for ‘overwork’ (Lao Lu’c) resembled the word for ‘tuberculosis’ (Lao), and so communities and their healthcare professionals often associated exercise, including pulmonary rehabilitation, with lung impairment. | To overcome the hesitance of patients to participate in the Pulmonary Rehabilitation Programme,60 we had to introduce a component in the programme that emphasised the benefits of exercise for respiratory symptoms. |
Norms/ barriers | We had planned to also address the Roma population in Greece with our interventions, yet our RAP provided us the insight that this population was extremely hard-to-reach; working with the Roma would require years of trust, which was beyond the scope of our 3 year research and funding period. | To use our resources more effectively in Greece, we decided to prioritise solely on the rural, traditional Greek population (low-resource, rural populations) instead. |
Norms/ cue to action | In rural Greece, brotherhood (filotimo) was a prevailing norm: connecting with and helping those around you was highly valued. | We embraced this norm as a motivator in the Pulmonary Rehabilitation Programme.60 We capitalised on the positive social pressure to exercise together as a group and help each other to complete the programme by not missing a single training. The programme in Greece had almost no drop-outs. |
*In many examples, multiple factors of our composed theoretical framework would apply to the contextual input. The factors with highest applicability are reported.
CRD, chronic respiratory disease; RAP, rapid assessment process.;
Conclusion
We have developed and validated a systematic tool for context-mapping, enabling to design context-driven implementation strategies for (lung) health interventions in low-resource settings. Its application to the FRESH AIR project demonstrated that the SETTING-tool is highly feasible, acceptable, and effective in facilitating context-driven adaptations to implementation strategies in six diverse settings across the globe.
The SETTING-tool was cocreated by an international expert panel and local end-users. Development was both theory driven and practice driven. Concepts from a rich diversity of relevant fields were consolidated, ranging from implementation science to global health. Additionally, the methodology was prospectively validated in six diverse settings in four countries on three continents: from deprived Ugandan slums, to rapidly developing Vietnamese villages, to austerity-impacted rural Greece and highly risk-burdened Kyrgyzstan. Feasibility, acceptability and effectiveness of the tool were consistently high. This demonstrates its global applicability.
However, it is difficult to assess whether better alternatives would have been available to the components included in our tool. In our pragmatic literature searches, there were no studies comparing the effectiveness of different components and certainly not across contexts. In this practice paper, we also did not compare our context-mapping outcomes against those of a control group. Lastly, although application of the SETTING-tool was feasible, the overall feasibility of working with certain populations should remain of consideration. For example, the tool’s feasibility was also high in the Roma camp, yet working with the Roma population generally required more time than working in other settings.
Existing studies, frameworks and toolkits emphasised the need for context-driven implementation strategies and repetitive calls highlighted the need for evidence on how to design those.5 6 9 16 17 19 The SETTING-tool addresses these calls by guiding the researcher step-by-step through the context-mapping process. Other than existing approaches such as Community-Based Participatory Research,50 this tool is developed as a practical guide in the field. It covers the entire context-mapping process, including for example developing the research materials and promoting uptake of the findings. Each step is easily reproducible. Key elements for successful completion of the steps are highlighted. This structured guidance can particularly serve clinical researchers without implementation expertise.18
The SETTING-tool is particularly relevant as it was developed for low-resource settings, where the burden of disease is highest, while the means to combat disease and evidence on how to do so are scarcest.5 9 19 22 51 52 Furthermore, the tool addresses two important issues that contribute to the avoidable loss of 85% of investments in health and biomedical research (US$200 billion lost in 2010 alone): (1) failure to establish priorities based on stakeholders’ needs and (2) poorly designed research methods.53 54
The SETTING-tool can be used flexibly; it can be adapted according to local needs, as long as the core elements (the six steps) remain intact and the tool remains simple to use.7 55 Researchers should also ensure to continuously collaborate closely with the stakeholders (box 2). Flexible aspects are, for example, that not every health topic requires the use of a vignette to avoid stigmatisation. Use of the tool across diverse contexts, for other health topics and by researchers that were not involved in its development, could provide further insight on the generalisability of the tool. This would also allow for assessing the importance of individuals components depending on the setting or targeted disease. Although validated in LMICs, the tool might also be transferable to high-income settings.
Box 2. Main lesson learnt.
Continuous collaboration between the researchers and the stakeholders was vital for successful completion of every step of the Setting-Exploration-Treasure-Trail-to-Inform-implementatioN-strateGies-tool. The close collaboration helped to align the study aim with the actual needs, to promote compatibility of methods and materials with the local reality, to identify the right community researchers to join the team and to promote uptake of the findings. Additional lessons learnt are detailed in online supplemental appendix 9.
Notably, the tool focuses on local beliefs and behaviours, and it is important to remain aware of other elements that shape context.8 Systematically assessing a context on multiple elements (beliefs and behaviour, physical environment, organisational structures, etc) would require a substantial amount of resources. We would then recommend to conduct the tool’s steps more pragmatically; particularly the in-depth analyses in step 5 as they are most time-consuming.
To conclude, the SETTING-tool can support researchers, policy-makers, health workers, NGOs and other implementers to apply evidence-based methods in context-mapping. This can facilitate them to design context-driven implementation strategies to increase implementation success of their health interventions, potentially avoiding the widescale waste of scarcely available resources. Ultimately, this could improve health outcomes.
Acknowledgments
We acknowledge the International Primary Care Respiratory Group (IPCRG) for introducing us to the primary care networks involved in this study. We thank Sian Williams and Lisa Cragg (IPRG) for supporting the countries’ teams in the formation of the Stakeholder Engagement Groups. We thank the country team members: Prossie Aliwebwa, Nathan Bagagire, Shamim Buteme, Joshua Kitimbo, Robinah Komuhendo, Jackie Mwendeze, Ronald Tenywa, Bruce Kirenga (Uganda); Pham Duong Uyen Binh, Nguyễn Thuy Thuc Doan, Le Huynh Thi Cam Hong, Tran Thanh Duy Linh, Kim Xuan Loan, Nguyen Minh Nhan, Lê Thị Oanh (Vietnam); Ilyas Dautov, Kumbat, Maamed Mademilov, Akak Sadykova, (Kyrgyzstan); Marilena Anastasaki, Vasiliki Chatzea, Andreas Karelis, Dimitra Sifaki-Pistolla (Greece). We also thank the community researchers not mentioned by name (to guarantee their anonymity, as they were also included as stakeholders or key informants in the study). We thank Asel Arykbaeva, Egid van Bree, Xaviera Cárdenas, Mariann Gódor, Birgit Hasenack, Anna Jansen, Iris van Loo, Sanne Molenaar, Debbie Vermond, Sylvia de Vreede and Annemijn Walsweer (Leiden University Medical Centre) for their important contribution to the conduct and/or analysis of this study. Lastly, we thank all stakeholders and participants for their vital contribution, with a special mention of the workers of the local Support Centre for Roma and Minority Groups, without whom we would not have been able to include Roma perspectives.
Footnotes
Handling editor: Seye Abimbola
Collaborators: On behalf of the FRESH AIR collaborators: Pham Le An, Marilena Anastasaki, Azamat Akylbekov, Andy Barton, Antonios Bertsias, Pham Duong Uyen Binh, Job FM van Boven, Evelyn A Brakema, Dennis Burges, Lucy Cartwright, Vasiliki E Chatzea, Niels H Chavannes, Liza Cragg, Tran Ngoc Dang, Ilyas Dautov, Berik Emilov, Irene Ferarrio, Frederik A van Gemert, Ben Hedrick, Le Huynh Thi Cam Hong, Nick Hopkinson, Elvira Isaeva, Rupert Jones, Corina de Jong, Sanne van Kampen, Winceslaus Katagira, Bruce Kirenga, Jesper Kjærgaard, Rianne MJJ van der Kleij, Janwillem Kocks, Le Thi Tuyet Lan, Tran Thanh Duv Linh, Christos Lionis, Kim Xuan Loan, Maamed Mademilov, Andy McEwen, Patrick Musinguzi, Rebecca Nantanda, Grace Ndeezi, Sophia Papadakis, Hilary Pinnock, Jillian Pooler, Charlotte C Poot, Maarten J Postma, Anja Poulsen, Pippa Powell, Nguyen Nhat Quynh, Susanne Reventlow, Dimitra Sifaki-Pistolla, Sally Singh, Talant Sooronbaev, Jaime Correia de Sousa, James Stout, Marianne Stubbe Østergaard, Aizhamal Tabyshova, Ioanna Tsiligianni, Tran Diep Tuan, James Tumwine, Le Thanh Van, Nguyen Nhu Vinh, Simon Walusimbi, Louise Warren, Sian Williams.
Contributors: EB led the design, conduct and analyses of the research. EB, CP and RvdK performed the literature searches for the theoretical underpinning of the methodology. Together with all coauthors, the identified methodological components were discussed and selected for the tool. The country experts—IT, SW, PLA and TS—specifically focused on local acceptability and feasibility. RR trained the team and supervised application of the methodology on the ground in the first setting. Analyses were performed by all researchers. EB wrote the manuscript and RvdK provided feedback on each version. All authors provided input and approved the final manuscript.
Funding: His study was funded by the EU Research and Innovation program Horizon2020 (Health, Medical research and the challenge of ageing) under grant agreement no. 680997.
Competing interests: None declared.
Patient and public involvement statement: This methodology was cocreated with local stakeholders, among others with district health officers, health workers and community members. We closely collaborated through face-to-face and digital meetings.
Patient consent for publication: Not required.
Ethics approval: The study was approved by the coordinating centre and by each local research ethical review board: the Medical Ethical Committee of the Leiden University Medical Centre (P16.063;04/15/2016), the Mulago Research and Ethics Committee (933;03/31/2016), the Ho Chi Minh City University of Medicine and Pharmacy (188/DHYD-HD;06/27/2016), the National Centre of Cardiology and Internal Medicine in Bishkek Ethics Committee (5;03/03/2016), and the 7th Health Region of Crete (protocol 6951;05/27/2016).
Provenance and peer review: Not commissioned; externally peer reviewed.
Data availability statement: All data and meta-data which are not already included in the supplementary materials will be available upon reasonable request within a reasonable time frame.
Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
Contributor Information
FRESH AIR collaborators:
Pham Le An, Marilena Anastasaki, Azamat Akylbekov, Andy Barton, Antonios Bertsias, Pham Duong Uyen Binh, Job FM van Boven, Evelyn A Brakema, Dennis Burges, Lucy Cartwright, Vasiliki E Chatzea, Niels H Chavannes, Liza Cragg, Tran Ngoc Dang, Ilyas Dautov, Berik Emilov, Irene Ferarrio, Frederik A van Gemert, Ben Hedrick, Le Huynh Thi Cam Hong, Nick Hopkinson, Elvira Isaeva, Rupert Jones, Corina de Jong, Sanne van Kampen, Winceslaus Katagira, Bruce Kirenga, Jesper Kjærgaard, Rianne MJJ van der Kleij, Janwillem Kocks, Le Thi Tuyet Lan, Tran Thanh Duv Linh, Christos Lionis, Kim Xuan Loan, Maamed Mademilov, Andy McEwen, Patrick Musinguzi, Rebecca Nantanda, Grace Ndeezi, Sophia Papadakis, Hilary Pinnock, Jillian Pooler, Charlotte C Poot, Maarten J Postma, Anja Poulsen, Pippa Powell, Nguyen Nhat Quynh, Susanne Reventlow, Dimitra Sifaki-Pistolla, Sally Singh, Talant Sooronbaev, Jaime Correia de Sousa, James Stout, Marianne Stubbe Østergaard, Aizhamal Tabyshova, Ioanna Tsiligianni, Tran Diep Tuan, James Tumwine, Le Thanh Van, Nguyen Nhu Vinh, Simon Walusimbi, Louise Warren, and Sian Williams
References
- 1. Grimshaw JM, Eccles MP, Lavis JN, et al. . Knowledge translation of research findings. Implement Sci 2012;7:50. 10.1186/1748-5908-7-50 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol 2008;41:327–50. 10.1007/s10464-008-9165-0 [DOI] [PubMed] [Google Scholar]
- 3. Peters DH, Adam T, Alonge O, et al. . Implementation research: what it is and how to do it. BMJ 2013;347:f6753. 10.1136/bmj.f6753 [DOI] [PubMed] [Google Scholar]
- 4. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med 2011;104:510–20. 10.1258/jrsm.2011.110180 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Brakema EA, Vermond D, Pinnock H, et al. . Implementing lung health interventions in low- and middle-income countries: a fresh air systematic review and meta-synthesis. Eur Respir J 2020;56. 10.1183/13993003.00127-2020. [Epub ahead of print: 23 07 2020]. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Daivadanam M, Ingram M, Sidney Annerstedt K, et al. . The role of context in implementation research for non-communicable diseases: Answering the 'how-to' dilemma. PLoS One 2019;14:e0214454. 10.1371/journal.pone.0214454 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Damschroder LJ, Aron DC, Keith RE, et al. . Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50. 10.1186/1748-5908-4-50 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Nilsen P, Bernhardsson S. Context matters in implementation science: a scoping review of determinant frameworks that describe contextual determinants for implementation outcomes. BMC Health Serv Res 2019;19:189. 10.1186/s12913-019-4015-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Ridde V. Need for more and better implementation science in global health. BMJ Glob Health 2016;1:e000115. 10.1136/bmjgh-2016-000115 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Allanson ER, Tunçalp Özge, Vogel JP, et al. . Implementation of effective practices in health facilities: a systematic review of cluster randomised trials. BMJ Glob Health 2017;2:e000266. 10.1136/bmjgh-2016-000266 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Pfadenhauer LM, Mozygemba K, Gerhardus A, et al. . Context and implementation: a concept analysis towards conceptual maturity. Z Evid Fortbild Qual Gesundhwes 2015;109:103–14. 10.1016/j.zefq.2015.01.004 [DOI] [PubMed] [Google Scholar]
- 12. Rosenstock IM, Strecher VJ, Becker MH. Social learning theory and the health belief model. Health Educ Q 1988;15:175–83. 10.1177/109019818801500203 [DOI] [PubMed] [Google Scholar]
- 13. Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process 1991;50:179–211. 10.1016/0749-5978(91)90020-T [DOI] [Google Scholar]
- 14. Kleinman A. Patients and healers in the context of culture: an exploration of the borderland between anthropology, medicine, and psychiatry. Berkeley: University of California Press, 1980. [Google Scholar]
- 15. Hansen S, Kanning M, Lauer R, et al. . MAP-IT: a practical tool for planning complex behavior modification interventions. Health Promot Pract 2017;18:696–705. 10.1177/1524839917710454 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Waltz TJ, Powell BJ, Fernández ME, et al. . Choosing implementation strategies to address contextual barriers: diversity in recommendations and future directions. Implement Sci 2019;14:42. 10.1186/s13012-019-0892-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Evans RE, Craig P, Hoddinott P, et al. . When and how do 'effective' interventions need to be adapted and/or re-evaluated in new contexts? The need for guidance. J Epidemiol Community Health 2019;73:481–2. 10.1136/jech-2018-210840 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Hull L, Goulding L, Khadjesari Z, et al. . Designing high-quality implementation research: development, application, feasibility and preliminary evaluation of the implementation science research development (ImpRes) tool and guide. Implement Sci 2019;14:80. 10.1186/s13012-019-0897-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Siddiqi K, Newell J, Robinson M. Getting evidence into practice: what works in developing countries? Int J Qual Health Care 2005;17:447–54. 10.1093/intqhc/mzi051 [DOI] [PubMed] [Google Scholar]
- 20. World Health Organization Factsheet chronic obstructive pulmonary disease. Available: https://www.who.int/respiratory/copd/en/ [Accessed 26 Jun 2019].
- 21. Hasson H. Systematic evaluation of implementation fidelity of complex interventions in health and social care. Implement Sci 2010;5:67. 10.1186/1748-5908-5-67 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Martin K, Mullan Z, Horton R. Overcoming the research to policy gap. Lancet Glob Health 2019;7:S1–2. 10.1016/S2214-109X(19)30082-8 [DOI] [PubMed] [Google Scholar]
- 23. Poot CC, van der Kleij RM, Brakema EA, et al. . From research to evidence-informed decision making: a systematic approach. J Public Health 2018;40:i3–12. 10.1093/pubmed/fdx153 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. World Health Organization - The Special Programme for Research and Training in Tropical Diseases Implementation research toolkit., 2019. Available: http://adphealth.org/irtoolkit/ [Accessed 22 Nov 2019].
- 25. Doherty T, Lewin S, Kinney M, et al. . Addressing the tensions and complexities involved in commissioning and undertaking implementation research in low- and middle-income countries. BMJ Glob Health 2018;3:e000741. 10.1136/bmjgh-2018-000741 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007;19:349–57. 10.1093/intqhc/mzm042 [DOI] [PubMed] [Google Scholar]
- 27. Hamilton AB, Finley EP. Qualitative methods in implementation research: an introduction. Psychiatry Res 2019;280:112516. 10.1016/j.psychres.2019.112516 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Ridde V, Pérez D, Robert E. Using implementation science theories and frameworks in global health. BMJ Glob Health 2020;5:e002269. 10.1136/bmjgh-2019-002269 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Van Belle S, van de Pas R, Marchal B. Towards an agenda for implementation science in global health: there is nothing more practical than good (social science) theories. BMJ Glob Health 2017;2:e000181. 10.1136/bmjgh-2016-000181 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Brakema EA, Tabyshova A, Kasteleyn MJ, et al. . High COPD prevalence at high altitude: does household air pollution play a role? Eur Respir J 2019;53:1801193–3. 10.1183/13993003.01193-2018 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. van Gemert F, Kirenga B, Chavannes N, et al. . Prevalence of chronic obstructive pulmonary disease and associated risk factors in Uganda (fresh air Uganda): a prospective cross-sectional observational study. Lancet Glob Health 2015;3:e44–51. 10.1016/S2214-109X(14)70337-7 [DOI] [PubMed] [Google Scholar]
- 32. Johnson RB, Onwuegbuzie AJ. Mixed methods research: a research paradigm whose time has come. Educational Researcher 2004;33:14–26. 10.3102/0013189X033007014 [DOI] [Google Scholar]
- 33. B J. The SAGE encyclopedia of qualitative research methods; rapid assessment process. Thousand Oaks: SAGE Publications, Inc, 2008. [Google Scholar]
- 34. Farmer T, Robinson K, Elliott SJ, et al. . Developing and implementing a triangulation protocol for qualitative health research. Qual Health Res 2006;16:377–94. 10.1177/1049732305285708 [DOI] [PubMed] [Google Scholar]
- 35. Beebe J. Basic concepts and techniques of rapid appraisal. Hum Organ 1995;54:42–51. 10.17730/humo.54.1.k84tv883mr2756l3 [DOI] [Google Scholar]
- 36. World Health Organization The epi coverage survey, training for mid-level managers, 2008. Available: http://www.who.int/immunization/documents/MLM_module7.pdf [Accessed 22 Dec 2019].
- 37. Nguyen VN, Huynh TTH, Chavannes NH. Knowledge on self-management and levels of asthma control among adult patients in Ho Chi Minh City, Vietnam. Int J Gen Med 2018;11:81–9. 10.2147/IJGM.S157050 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. van Gemert F, Chavannes N, Nabadda N, et al. . Impact of chronic respiratory symptoms in a rural area of sub-Saharan Africa: an in-depth qualitative study in the Masindi district of Uganda. Prim Care Respir J 2013;22:300–5. 10.4104/pcrj.2013.00064 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Gourlay A, Mshana G, Birdthistle I, et al. . Using vignettes in qualitative research to explore barriers and facilitating factors to the uptake of prevention of mother-to-child transmission services in rural Tanzania: a critical analysis. BMC Med Res Methodol 2014;14:21. 10.1186/1471-2288-14-21 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Broadbent E, Petrie KJ, Main J, et al. . The brief illness perception questionnaire. J Psychosom Res 2006;60:631–7. 10.1016/j.jpsychores.2005.10.020 [DOI] [PubMed] [Google Scholar]
- 41. Global Adult Tobacco Survey Collaborative Group Tobacco questions for surveys: a subset of key questions from the global adult tobacco survey (GATS). Atlanta, GA: Centers for Disease Control and Prevention, 2011. [Google Scholar]
- 42. World Health Organization World health survey questions on cooking and heating practices, 2017. Available: http://www.who.int/indoorair/health_impacts/cooking/en/ [Accessed 17 Jan 2016].
- 43. World Health Organization WHO STEPS instrument Question-by-Question Guide - the WHO STEPwise approach to noncommunicable disease risk factor surveillance (STEPS) 2015.
- 44. Yeh K-W, Chen S-H, Chiang L-C, et al. . Survey of asthma care in Taiwan: a comparison of asthma specialists and general practitioners. Ann Allergy Asthma Immunol 2006;96:593–9. 10.1016/S1081-1206(10)63555-7 [DOI] [PubMed] [Google Scholar]
- 45. Gale NK, Heath G, Cameron E, et al. . Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 2013;13:117. 10.1186/1471-2288-13-117 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46. Dixon-Woods M, Agarwal S, Young B, et al. . Integrative approaches to qualitative and quantitative evidence: NHS health development agency 2004.
- 47. Nagler EM, Pednekar MS, Viswanath K, et al. . Designing in the social context: using the social contextual model of health behavior change to develop a tobacco control intervention for teachers in India. Health Educ Res 2013;28:113–29. 10.1093/her/cys060 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48. Tricco AC, Zarin W, Rios P, et al. . Engaging policy-makers, health system managers, and policy analysts in the knowledge synthesis process: a scoping review. Implement Sci 2018;13:31. 10.1186/s13012-018-0717-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49. Siron S, Dagenais C, Ridde V. What research tells us about knowledge transfer strategies to improve public health in low-income countries: a scoping review. Int J Public Health 2015;60:849–63. 10.1007/s00038-015-0716-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50. Israel BA, Schulz AJ, Parker EA, et al. . Review of community-based research: assessing partnership approaches to improve public health. Annu Rev Public Health 1998;19:173–202. 10.1146/annurev.publhealth.19.1.173 [DOI] [PubMed] [Google Scholar]
- 51. Mendis S, Al Bashir I, Dissanayake L, et al. . Gaps in capacity in primary care in low-resource settings for implementation of essential noncommunicable disease interventions. Int J Hypertens 2012;2012:1–7. 10.1155/2012/584041 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52. Brakema EA, van der Kleij RM, Vermond D, et al. . Let's stop dumping cookstoves in local communities. it's time to get implementation right. NPJ Prim Care Respir Med 2020;30:3. 10.1038/s41533-019-0160-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet 2009;374:86–9. 10.1016/S0140-6736(09)60329-9 [DOI] [PubMed] [Google Scholar]
- 54. Macleod MR, Michie S, Roberts I, et al. . Biomedical research: increasing value, reducing waste. Lancet 2014;383:101–4. 10.1016/S0140-6736(13)62329-6 [DOI] [PubMed] [Google Scholar]
- 55. Greenhalgh T, Robert G, Macfarlane F, et al. . Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004;82:581–629. 10.1111/j.0887-378X.2004.00325.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56. Brakema EA, van Gemert FA, Williams S, et al. . Publisher correction: implementing a context-driven awareness programme addressing household air pollution and tobacco: a fresh air study. NPJ Prim Care Respir Med 2020;30:49. 10.1038/s41533-020-00208-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57. van Gemert F, de Jong C, Kirenga B, et al. . Effects and acceptability of implementing improved cookstoves and heaters to reduce household air pollution: a fresh air study. NPJ Prim Care Respir Med 2019;29:32. 10.1038/s41533-019-0144-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58. Nantanda R, Buteme S, van Kampen S, et al. . Feasibility and acceptability of a midwife-led health education strategy to reduce exposure to biomass smoke among pregnant women in Uganda, a fresh air project. Glob Public Health 2019;14:1770–83. 10.1080/17441692.2019.1642931 [DOI] [PubMed] [Google Scholar]
- 59. McEwen A, Pooler J, Lionis C, et al. . Adapting very brief advice (VBA) on smoking for use in low-resource settings: experience from the fresh air project. J Smok Cessat 2019;14:190–4. 10.1017/jsc.2019.4 [DOI] [Google Scholar]
- 60. Anastasaki M, Trigoni M, Pantouvaki A, et al. . Establishing a pulmonary rehabilitation programme in primary care in Greece: a fresh air implementation study. Chron Respir Dis 2019;16:1479973119882939. 10.1177/1479973119882939 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjgh-2020-003221supp001.pdf (484.5KB, pdf)
bmjgh-2020-003221supp002.zip (1.7MB, zip)