Factors critical to implementation success of cleaner cooking interventions in low-income and middle-income countries: protocol for an umbrella review

Abstract

Introduction

Over a third of the world’s population relies on solid fuels as their primary energy source. These fuels have damaging effects on health, air quality and forest resources. Interventions to promote access to cleaner solid fuel cookstoves and clean fuels have existed for decades. However, the adoption by local communities has largely failed, which led to a waste of resources and suboptimal outcomes. Therefore, the objective of this umbrella review is to identify factors that determine implementation success for cleaner cooking interventions in low-resource settings and weigh their level of confidence in the evidence.

Methods and analysis

We identified systematic and narrative reviews examining factors that influence the acquisition, initial adoption or sustained use of cleaner solid fuel cookstoves and clean fuels at any scale by a literature search in PubMed, Embase, Global Health Database, Cochrane, PsycINFO, Emcare, Web of Science and CINAHL, without date or language restrictions. The search was conducted on 23 October 2017 and updated on 10 July 2019. Reviews based on qualitative, quantitative or mixed-methods studies were included and will be appraised using the Meta Quality Appraisal Tool combined with the Assessment of Multiple Systematic Reviews. Data will be extracted and factors affecting implementation will be coded using the Consolidated Framework for Implementation Research. The Grading of Recommendations Assessment, Development and Evaluation-Confidence in the Evidence from Reviews of Qualitative Research tool will be used to determine the level of confidence in the coded factors. Two researchers will independently conduct these steps.

Ethics and dissemination

This umbrella review does not require the approval of an ethical review board. Study results will be published in an international peer-reviewed journal. The outcomes will be converted into two practical tools: one for cleaner solid fuel cookstoves and one for clean fuels. These tools can guide the development of evidence-based implementation strategies for cleaner cooking interventions in low-income and middle-income countries to improve implementation success. These tools should be pilot-tested and promoted among regional and global initiatives.

PROSPERO registration number

CRD42018088687.

Strengths and limitations of this study.

• This umbrella review will aggregate up-to-date evidence on factors critical to implementation success for cleaner cooking interventions in low-resource settings and weigh the level of confidence in the evidence.

• We will comprehensively search eight databases without date or language restrictions, and two researchers will independently apply validated tools throughout each step of the process.

• Narrative reviews will also be included in the study, as only a fraction of the literature has been reported in systematic reviews; the lower methodological rigour of narrative reviews (compared with systematic reviews) will be considered during appraisal of the methodological quality of the reviews.

• The identified implementation factors will be integrated into two hands-on implementation tools to facilitate the design of evidence-based implementation strategies for cleaner cooking interventions and hence stimulate the translation of evidence into practice.

Introduction

The use of solid fuels for cooking is a deep-rooted practice for approximately 2.8 billion people, especially in low-income and middle-income countries (LMICs).¹ Solid fuels include biomass fuels (such as wood, dung, crop residues and charcoal) and coal. Combustion of solid fuels leads to emission of greenhouse gases and black carbon, aggravating climate change. It also leads to high levels of household air pollution affecting health.² Furthermore, collecting fuels leads to forest degradation and deforestation and is time consuming for women and children involved in fuel gathering.³ Interventions that promote the replacement of traditional cookstoves (that use solid fuels or kerosene as fuel) by cleaner solid fuel cookstoves (later referred to as cleaner cookstoves) or clean fuels seem to be promising solutions due to higher combustion efficiency and lower emissions. Clean fuels include liquefied petroleum gas (LPG), natural gas, alcohol fuels, biogas, solar cookers and electricity.⁴ Kerosene stoves are relatively efficient but are discouraged due to significant health risks.⁴ Cleaner cookstoves came on the market in the 1950s and were later supported by promising laboratory-based experiments with different models, trying to improve combustion efficiency.^{5 6} Subsequently, many non-governmental organisations and governments promoted the large-scale rollout of these stoves. The push for a transition to clean fuels has come into force since World Health Organisation (WHO) published guidelines for indoor air quality and household fuel combustion.⁷ However, real-world outcomes of interventions that promoted the switch to cleaner cookstoves or clean fuels have often not been fruitful.^{5 6} Many of the programmes ran into hurdles and failed⁸; adoption rates are often not reported in the literature, and if they are, rates may vary between 4% and 10%.^9–13 Moreover, even if cleaner stoves and clean fuels are used in local communities, they are often combined with traditional ones.¹⁴ Amidst scarce resources, implementation efforts have thus largely been a waste of time, money, energy and human resource. Progress in large-scale implementation has been inexplicably slow.^{3 15}

Implementation, the act of carrying an intervention into widespread use, is a complex process.¹⁶ Although the failure to implement cleaner cooking interventions has been reported already in the 1980s and 1990s,8 13 17–19 implementation strategies and adoption rates have generally not changed accordingly. Numerous additional studies have since then examined barriers and facilitators for the implementation of cleaner cooking interventions, including more than a dozen systematic and narrative reviews. Although evidence has been consolidated in a study by Puzzolo et al,²⁰ an up-to-date overview of the evidence, including more recently published reviews, is missing. Furthermore, evidence of factors that influence the implementation of clean fuels was limited in this study. Therefore, we will conduct an umbrella review or in other words a review of systematic and narrative reviews. We aim to systematically identify critical factors for implementation success of cleaner cooking interventions in LMICs and to weigh their level of evidence. We define implementation success as sustained predominant use of cleaner cookstoves and clean fuels.²¹ As we expect limited evidence on sustained use (defined as use for 1–2 years after acquisition or use for more than 2 years²⁰), we will also include factors that relate to acquisition (defined as purchase or installation) and initial adoption (defined as use for less than 1 year from acquisition²⁰) of cleaner cooking interventions. Findings will be consolidated in two easy-to-use implementation tools: one for cleaner cookstoves and one for clean fuels. These tools can practically guide the development of evidence-based implementation strategies. Use of these tools could ensure more effective use of scarce resources and, ultimately, alleviate climate change and improve health outcomes.

Methods and analysis

This study is part of a larger study conducted by Brakema et al.²² This latter study focused on critical implementation factors for interventions targeting chronic lung disease in LMICs, as part of the Horizon 2020 Free Respiratory Evaluation and Smoke-exposure reduction by primary Health cAre Integrated gRoups (FRESH AIR) project.²³ Because the search resulted in a high number of articles that focus on the implementation of cleaner cooking interventions, it was decided to split the review into two parts. The review of Brakema et al includes articles on implementation of all interventions other than cleaner cooking interventions (such as tobacco cessation or the implementation of guidelines), while the current study is an umbrella review that includes reviews focusing on cleaner cooking interventions. The study is registered at PROSPERO (CRD42018088687). Because the methodology for this umbrella review differs from the methodology used in the review by Brakema et al,²² and because an umbrella review of systematic and narrative reviews is not common,^{24 25} it was decided to record the protocol for this study to increase transparency.

Search strategy and inclusion criteria

The initial search was part of the study conducted by Brakema et al.²² The search strategy was developed together with a certified medical librarian. Search terms included (synonyms of) implementation, LMICs and interventions targeting chronic lung health, including cleaner cooking interventions. No names of fuels or specific cooking interventions, such as LPG or biogas, were included in the search strategy. This is a limitation of the study. Cleaner cooking interventions were included regardless whether they were defined in terms of health benefits. The search terms are displayed in online supplemental material S1. Searches were conducted using the databases PubMed, Embase, Global Health Database, Cochrane, PsycINFO, Emcare, Web of Science and CINAHL, with no date or language restrictions. The search was conducted on 23 October 2017 and updated on 10 July 2019. The full search strategy and the operationalisation is described elsewhere.²² Furthermore, experts in the field of cleaner cooking interventions will be consulted to identify relevant publications from July 2019. All reviews regarding the implementation of cleaner cookstoves and clean fuels in LMICs were extracted from the results and will be used in the current study. Given the expected large number of cookstove studies covered in reviews other than systematic reviews (narrative reviews), it was decided to be inclusive and not limit the search to systematic reviews only. Reviews based on quantitative, qualitative or mixed-methods studies will be considered relevant.¹⁶ Articles will be excluded if they focus on legislation at a national governmental level or on hypothetical interventions. Besides, articles will be excluded if no factors that influence implementation are reported or if the full text is not available after contacting the authors.

The steps that will be conducted, and the tools used to conduct these steps, are displayed in figure 1. Methodological quality assessment, data extraction and data analysis will be conducted in the same manner for systematic and narrative reviews. The risk of biased recommendations in narrative reviews will be accounted for during appraisal of the methodological quality of the reviews.

Figure 1.

Tools that will be used in each phase. Adapted from Brakema et al.²² *Tools to assess the quality of narrative reviews are not in the list of recommended design-specific companion tools to augment the MetaQAT tool.²⁶ Therefore, it was decided to use the AMSTAR-tool for systematic and narrative reviews. AMSTAR, Assessment of Multiple Systematic Reviews; CFIR, Consolidated Framework for Implementation Research; GRADE-CERQual, Grading of Recommendations Assessment, Development and Evaluation-Confidence in the Evidence from Reviews of Qualitative research; MetaQat, Meta Quality Appraisal Tool.

Methodological quality assessment

The Meta Quality Appraisal Tool (MetaQAT)²⁶ combined with the Assessment of Multiple Systematic Reviews (AMSTAR)²⁷ will be used to assess the methodological quality of included studies. The MetaQAT assesses the relevancy, reliability, validity and applicability of the included studies.²⁶ The AMSTAR will be embedded into the MetaQAT, as recommended by the MetaQAT authors.²⁶ The results will be used as input for the assessment of confidence in the evidence.

Data extraction

Data will be extracted using standardised pilot-tested extraction forms.²⁸ We will extract data on descriptive characteristics (author; year; narrative/systematic review; whether the review included quantitative, qualitative or mixed-method studies; country; setting/population; fuel used before intervention; intervention targeting cleaner cookstoves vs clean fuels (type); outcome; data sources; funding source of the review; role of funding organisation) and implementation factors. Factors will not be extracted if they are speculations.

Data analysis

Content analysis will be used for data analysis.²⁹ Content analysis is a research technique for making replicable and valid inferences from data to their context.³⁰ It can be used to synthesise qualitative and quantitative data.²⁹ First, to prepare the data for content analysis, the implementation factors will be coded using the Consolidated Framework for Implementation Research (CFIR).³¹ The CFIR is validated for identifying implementation factors at multilevel for complex processes³² and helps to understand what works where and why across multiple contexts (interventions) and settings.^{31 33} Implementation factors are categorised in five domains: (1) intervention characteristics, (2) outer setting, (3) inner setting, (4) characteristics of individuals involved and (5) the implementation process.³¹ Each CFIR domain includes several constructs. Several constructs were added to the CFIR by Brakema et al,²² such as the extent to which language affects the implementation process. If relevant, more constructs will be added inductively to the framework. Second, the Grading of Recommendations Assessment, Development and Evaluation-Confidence in the Evidence from Reviews of Qualitative research (GRADE-CERQual) will be used to assess the confidence in the included factors.³⁴ The GRADE-CERQual consists of four domains: methodological limitations, relevance, coherence and adequacy. However, coherence will not be included as this will be taken into account in the end by counting the number of studies that mention the factor. For each of the other three domains, a score will be assigned per study in which the factor appeared. The score for adequacy will be determined based on the richness of the data. Relevance and methodological limitations will be scored on the aforementioned MetaQAT-AMSTAR tool (relevance and reliability+validity, respectively). This score ranges from one point (substantial concerns) to four points (no to very minor concerns). Third, an overall level of confidence in the factor will be determined by totalling the scores of the three domains. This total score will thus be higher if the studies’ adequacy, relevance or quality of the methodology was appraised as high or if the factor appeared in more studies (the principle of content analysis). Guidance to preclude the double-counting that overstates the evidence due to overlap among reviews is limited and inconsistent.³⁵ To gain insight into the frequency of double-counting, a matrix of the primary studies included in each review will be presented.³⁶

At least two authors will independently conduct the screening and selection process, the data extraction and the quality appraisal of the included studies. A third researcher will solve any discrepancies. Endnote will be used to manage records and data throughout the review. In case of deviations from the protocol, these amendments will be described in the final paper.

Patient and public involvement

Patients and the public will not be involved in this research.

Ethics and dissemination

This umbrella review does not require the approval of an ethical review board. Study results will be published in an international peer-reviewed journal. The outcomes will be converted in two practical tools, one for cleaner cookstoves and one for clean fuels, that can guide the development of evidence-based implementation strategies for cleaner cooking interventions in LMICs. These tools should be pilot-tested in future cookstove implementation projects and adapted accordingly. Furthermore, the tools should be presented to brokers in large networks, including the Clean Cooking Implementation Science Network, the Clean Cooking Alliance, and WHO. We have already reached out to several of these organisations. Ideally, these organisations should then promote the use of the tool to make it well known and easily available.

Discussion

With this umbrella review, we aim to consolidate all available evidence on critical factors for successful implementation of cleaner cookstoves and clean fuels in LMICs and weigh the level of confidence in the evidence. Implementation methodologies are considered as main priorities in implementation science, especially in low-resource settings.^37–39 This umbrella review uses a comprehensive implementation science framework, includes up-to-date data from both systematic and narrative reviews and will result in two practical tools that may aid in the development of evidence-based implementation strategies. An example of such a tool can be found in the study of Brakema et al.²² These tools should help to develop evidence-based implementation strategies for cleaner cooking interventions in the field.

With the increasing number of scientific reviews comes a need for third-level analyses, combining evidence of multiple reviews.⁴⁰ Various terms are given to these overviews of reviews, including meta-reviews, review of systematic reviews, reviews of reviews or umbrella reviews.⁴¹ A precise definition of the different terms is unclear. However, the common denominator is that they often only assess systematic reviews. Umbrella reviews are usually more inclusive of different types of systematic reviews compared with the other overview of reviews.⁴¹ We decided to also include narrative reviews in this umbrella review given the nature of the sector. However, narrative reviews may increase the risk of biased recommendations as they have less stringent criteria for information selection and appraisal than systematic reviews.⁴² We will account for this risk of bias as the level of confidence in the evidence is based on the quality of the methodology of the review in which the factor is reported. Excluding narrative reviews would exclude a considerable amount of available evidence. We decided not to distinguish between factors influencing adoption and factors influencing sustained use, as we expect limited evidence on sustained use only. Furthermore, in cleaner cooking literature, adoption and sustained use are often not clearly distinguished.⁴³ Ideally, studies would investigate the outcomes of cleaner cooking interventions over the course of several years, as the outcomes are known to differ over time.⁴⁴ Notably, we will weigh confidence in the factors that influence the implementation of cleaner cookstoves and clean fuels, based on the frequency of occurrence in the studies, and the quality of the study in which the factors are reported. This should not be interpreted as weighing or prioritising the importance of the factors.^{45 46} We also decided not to separately report on barriers and facilitators, as reversed barriers will most likely serve as facilitators and vice versa.^{22 46} We decided to include both cleaner cookstoves and clean fuels in this review, although the first are often not classified as ‘clean’ according to the guidelines of WHO due to high exposure levels.^{4 7} However, cleaner cookstoves can play an important role in the transition to clean fuels and are therefore included in this review. Furthermore, newer cleaner cookstoves, such as advanced biomass cookstoves, will be included in this umbrella review, but it is likely that these will not be covered by the identified reviews. However, findings of the other cleaner cookstoves are probably largely applicable to the newer cleaner cookstoves.²⁰

This study will result in two tools with an overview of implementation factors, confidence in the evidence of the factors and practical examples for implementers to promote large-scale uptake of cleaner cookstoves and clean fuels, respectively. These tools can therefore complement WHO Clean Household Energy Solutions Toolkit.⁴⁷ Successful implementation, including sustained use, of cleaner cooking interventions could ultimately alleviate climate change and improve health outcomes.⁴⁸