Process evaluation of complex interventions in non-communicable and neglected tropical diseases in low- and middle-income countries: a scoping review

Abstract

Objectives

The aim of this review is to map out the use of process evaluation (PE) in complex interventions that address non-communicable diseases (NCDs) and neglected tropical diseases (NTDs) to identify gaps in the design and conduct, as well as strengths, limitations and implications, of this type of research in low- and middle-income countries (LMICs).

Design

Scoping review of PE studies of complex interventions implemented in LMICs. Six databases were searched focused on studies published since 2008.

Data sources

Embase, PubMed, EbscoHost, Web of Science (WOS), Virtual Health Library (VHL) Regional Portal and Global Index Medicus: Regional Indexes AIM (AFRO), LILACS (AMRO/PAHO), IMEMR (EMRO), IMSEAR (SEARO), WPRIM (WPRO) Global Index Regional Indexes, MEDLINE, SciELO.

Eligibility criteria

Studies conducted in LMICs on PEs of randomised controlled trials (RCTs) and non-RCTs published between January 2008 and January 2020. Other criteria were studies of interventions for people at risk or having physical and mental NCDs, and/or NTDs, and/or their healthcare providers and/or others related to achieve better health for these two disease groups. Studies were excluded if they were not reported in English or Spanish or Portuguese or French, not peer-reviewed articles, not empirical research and not human research.

Data extraction and synthesis

Data extracted to be evaluated were: available evidence in the utilisation of PE in the areas of NCDs and NTDs, including frameworks and theories used; methods applied to conduct PEs; and in a subsample, the barriers and facilitators to implement complex interventions identified through the PE. Variables were extracted and categorised. The information was synthesised through quantitative analysis by reporting frequencies and percentages. Qualitative analysis was also performed to understand facilitators and barriers presented in these studies. The implications for PEs, and how the information from the PE was used by researchers or other stakeholders were also assessed in this approach.

Results

303 studies were identified, 79% were for NCDs, 12% used the label ‘PE’, 27% described a theory or framework for the PE, and 42% used mixed methods to analyse their findings. Acceptability, barriers and facilitators to implement the interventions, experiences and perceptions, and feasibility were the outcomes most frequently evaluated as part of the PEs. Barriers and facilitators themes identified were contextual factors, health system factors, human resources, attitudes and policy factors.

Conclusions

PEs in NCDs and NTDs are used in LMICs with a wide variety of methods. This review identified many PEs that were not labelled by the authors as such, as well as a limited application of PE-related theories and frameworks, and heterogeneous reporting of this type of study.

Strengths and limitations of this study.

• Six databases focused on process evaluation of complex interventions implemented in low- and middle-income countries in the area of non-communicable diseases and neglected tropical diseases were searched.

• The search strategy was designed with an experienced librarian and it was built based on the systematic review of Liu et al.

• All languages were not included due to limited human resources but we expect that the four major selected languages covered the majority of relevant literature.

• Data extraction was not duplicated because of the high number of studies; this was mitigated by previous specific training of the two reviewers and frequent meetings to clear doubts.

• Only a subset (n=50) of studies were included in the subsample analysis of barriers and facilitators, which may have limited the number of identified barriers or facilitators.

Introduction

In low- and middle-income countries (LMICs), the double burden of chronic long-term non-communicable (NCDs) and neglected tropical diseases (NTDs) represents a major challenge to vulnerable populations and their health systems.^{1 2} NCDs kill 41 million people each year and one-third of them are between 30 and 69 years old.³ This group of diseases is having an increasing impact in LMICs with variability in terms of burden and mortality within and between these countries.^4–6 NTDs affect more than 1 billion people worldwide and they are the most vulnerable populations with poor access to healthcare services.^{7 8} NCDs and NTDs impact individuals,^{9 10} communities and health system responses.^{4 11 12} Complex interventions are needed to address these complex and multifaceted health challenges^{13 14} in order to tackle global health problems,^{15 16} and therefore achieve the Sustainable Development Goal 3 to ‘Ensure healthy lives and promote well-being for all at all ages’.¹⁷ However, interventions with multiple components face diverse challenges when these interventions are implemented on the ground,¹⁸ especially in LMICs, where the context and health system barriers add additional complexity.¹⁹ Given this, process evaluation (PE) is a type of study that can be used to understand how complex interventions are implemented, clarify causal mechanisms and identify contextual factors associated with variation in outcomes rather than simply measuring effectiveness outcomes.²⁰ PEs are also useful to close translation gaps and reduce research waste,²¹ because they can inform future interventions, policy and practice.^{20 22}

Two recently published systematic reviews explored the use of PEs in the literature. One study focused on primary healthcare interventions addressing NCDs in randomised controlled trials (RCTs) worldwide.^{23 24} Another systematic review was conducted on PE studies of knowledge translation interventions without restriction of health conditions.²⁵ These reviews found that PEs used both quantitative and qualitative methods of data collection, and that they can be conducted at different moments during a project such as feasibility testing phases, alongside evaluations of effectiveness or alongside scale-up studies. These reviews were restricted to publications in English language only, and found limited LMIC representation. Liu et al^{23 24} in their study that included PEs conducted in primary care settings found only three RCTs for NCDs in LMICs (one study in India, Malaysia, and Zambia, respectively). Scott et al²⁵ did not report the regions of the implementation of the studies included in their review. Neither of these reviews explored the main barriers and facilitators of implementing complex interventions on NCDs and NTDs in LMICs.

This scoping review aims to identify gaps in the design and conduct of PE of complex interventions for NCDs and NTDs in LMICs. These findings expand the knowledge of previous systematic reviews by focusing on LMICs, where barriers to implement complex interventions are frequent and challenging. The objectives of this scoping review are: (1) determining the proportion of studies labelled as PE and identifying other terms used instead of PE in the literature; (2) identifying theories and frameworks used in PEs of complex interventions for NCDs and NTDs in LMICs; (3) examining the design and conduct of PE of complex interventions for NCDs and NTDs in LMICs; and (4) summarising barriers and facilitators identified in a subsample of studies of PEs of complex interventions for NCDs and NTDs in LMICs.

Methods

The full protocol for this scoping review was published elsewhere.²⁶ Results are reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for scoping reviews.²⁷

Eligibility Criteria

Studies on PEs of RCTs and non-RCTs (feasibility studies, observational studies, quasi-experimental studies) were included. Given that findings in previous systematic reviews^23–25 showed that PEs are often not explicitly labelled as such, we included studies with data that: tried to understand how complex interventions were implemented in the field; measured PE outcomes; clarified causal mechanisms; and/or identified contextual factors beyond measuring effectiveness outcomes. Studies conducted in LMICs according to the World Bank definition in 2019²⁸ and published between January 2008 and January 2020 were included. This date was chosen as it was when the UK’s Medical Research Council (MRC) updated its guidance for developing and evaluating complex interventions.^{29 30} If a study was conducted in high-income countries and LMICs, only the data from the LMICs were included.

Studies considered for this scoping review had interventions targeted at people at risk or having physical and mental NCDs (type 2 diabetes mellitus, type 1 diabetes mellitus, cardiovascular disease, depression, chronic obstructive pulmonary disease and chronic kidney disease),³¹ and/or NTDs (online supplemental material 1),³² and/or healthcare providers (physicians, nurses, technicians) and/or others related to achieve better health (reforms in universal coverage, public policy, service delivery and leadership) for these two disease groups.

The types of interventions included were complex interventions according to the MRC guideline that considered ‘intervention with several interacting components’ but also could include other dimensions of complexity such as: ‘number and difficulty of behaviours required by those delivering or receiving the intervention’; ‘number of groups or organisational levels targeted by the intervention’; and ‘degree of flexibility or tailoring of the intervention permitted’.^{29 30} In addition, we included studies implemented in diverse settings such as community, health system or at policy level. The comparators were treatment as usual, active control or placebo control, as well as absent control condition (eg, pre/post-evaluations). In addition, studies with multiple active interventions were included.

Studies were excluded if they were not reported in English or Spanish or Portuguese or French, not peer-reviewed articles, not empirical research and not human research.

Information sources

Embase, PubMed, EbscoHost, Web of Science (WOS), Virtual Health Library (VHL) Regional Portal and Global Index Medicus: Regional Indexes AIM (AFRO), LILACS (AMRO/PAHO), IMEMR (EMRO), IMSEAR (SEARO), WPRIM (WPRO) Global Index Regional Indexes, MEDLINE, SciELO.

Search strategy

A different search strategy (online supplemental material 2) was designed for each database and with an experienced librarian.

Study records

Data management

Covidence (Veritas Health Innovation, Melbourne, Australia)³³ was used in the screening process. Titles and abstracts were screened after the importation of citations from the databases, and manuscripts were selected to proceed to data extraction.

Selection process

Five reviewers (ML-P, HL, MO, XY, AM) in pairs considered the potential eligibility of all abstracts and titles identified from executing the search strategy. Disagreements were solved by a third reviewer. Agreement on the screening selection among reviewers was between 90% and 92%. Also, teams of two independent and blind reviewers evaluated full texts of all potential eligible studies. First, the reviewers calibrated their judgements using a smaller set of articles to ensure the same understanding of inclusion and exclusion criteria. Subsequently, they reviewed the complete set of articles. Any disagreements were resolved by group discussion or arbitration until reaching consensus. Agreement on the screening selection among reviewers was between 70% and 81%. Additionally, we included studies: (1) if the search strategy identified protocols with a published PE article and (2) if a PE study was identified during the full-text screening through further reference checking. Also, these potential eligible articles were reviewed by duplicated too.

Data collection process and data items

Data extraction included:

1. Full description of the study and complex intervention.

2. Details about the PE such as clear labelling as a PE, purpose, prespecified protocol, stage when the PE was applied, specific framework and/or theory of the PE18 20 34–38 and how the framework and/or theory of the PE was used (informed by, applied theory or tested theory),³⁹ methods used, leading team of the evaluation and type of analysis (quantitative, qualitative, mixed or multimethod), as well as PE outcomes defined by the literature that were measured (online supplemental material 3).⁴⁰

3. Lessons learnt about the PE: barriers and facilitators for implementation of complex interventions and implications for PE in LMICs (how the information of the PE was used). Data extraction on lessons learnt about the PE was literally extracted. The tools and detailed information for the data collection can be found in the protocol.²⁶

Reporting of results

Variables were extracted and categorised (eg, data related to the methods of the PE and details of the intervention). The information was synthesised through quantitative analysis by reporting frequencies and percentages.

Qualitative analysis was also performed to understand facilitators and barriers to implement the complex interventions in the field as well as the implications for PEs in LMICs (how the information from the PE was used by researchers or other stakeholders). Fifty studies were randomly chosen from the total sample of included articles for the qualitative analysis using the web platform Research Randomizer (https://www.randomizer.org) to guarantee saturation. Based on the information extracted from these studies, codes were generated and used to synthesise and analyse the data.

Results are presented in the following four sections:

1. General characteristics of the interventions.

2. Available evidence in PE in the areas of NCDs and NTDs, including frameworks and theories used.

3. Methods applied to conduct PE.

4. Findings from the PEs related to barriers and facilitators of implementation of the complex interventions, as well as findings about implications of the PE.

Patient and public involvement

There was no patient nor public involvement in this study.

Results

Our search strategy identified 11 297 articles, 9977 were screened after removing 1320 duplicates, and 285 studies were included. The reasons for exclusions are available in figure 1 and the main reason was that they were not PE studies. Additional 18 articles were included, 11 studies that the search strategy identified as protocols but with published PE results and 7 articles which were found during the full-text screening through further reference checking (figure 1). Thus, a total of 303 studies were included in this review (online supplemental material 4) and 18 of them were articles published in Spanish or Portuguese.

Figure 1.

Flow chart. LMIC, low- and middle-income country; PE, process evaluation.

General characteristics of the interventions

Disease and region

Of the 303 studies, 239 of 303 (79%) focused on NCDs, 62 of 303 (20%) on NTDs and 2 of 303 (1%) included both groups of diseases or had a health system perspective. Among the most common NCDs, studies focusing on cardiovascular diseases were 86 of 239 (36%) and type 2 diabetes with 50 of 239 (21%) studies. Regarding NTDs, dengue was the most common condition with 19 of 62 (31%) studies.

In regard to the geographical regions where the studies were conducted, 83 of 303 (27%) were from sub-Saharan Africa, 65 of 303 (21.5%) from Latin America and 16 of 303 (5%) from countries in the Middle East and North Africa. More than 50% of the studies were conducted in upper middle-income countries (table 1).

Table 1.

General characteristics of the studies

	NCD (N=239)	NTD (N=62)	Total (N=303)*
Region
East Asia and Pacific	53 (22.2%)	7 (11.3%)	61 (20.1%)
Latin American countries	41 (17.1%)	24 (38.7%)	65 (21.5%)
Middle East and North Africa	15 (6.3%)	1 (1.6%)	16 (5.3%)
South Asia	40 (16.7%)	12 (19.3%)	52 (17.1%)
Sub-Saharan Africa	68 (28.5%)	14 (22.6%)	83 (27.4%)
Multicountry	23 (9.2%)	4 (6.5%)	26 (8.6%)
Income*
Low-income countries	13 (6.0%)	7 (12.1%)	21 (7.6%)
Lower-middle-income countries	84 (38.7%)	28 (48.3%)	112 (40.4%)
Upper-middle-income countries	120 (55.3%)	23 (39.6%)	144 (52.0%)
Integration with other diseases
Yes	74 (31.1%)	8 (12.9%)	84 (27.8%)
No	165 (68.9%)	53 (87.1%)	218 (72.2%)
Study designs
RCT or cRCT	66 (27.6%)	9 (14.5%)	75 (24.8%)
Pre/post or time series	36 (15.1%)	4 (6.5%)	41 (13.5%)
Observational	49 (20.5%)	16 (25.8%)	66 (21.8%)
National programmes	22 (9.2%)	18 (29.0%)	40 (13.2%)
Qualitative	17 (7.1%)	2 (3.2%)	19 (6.3%)
Mixed/multimethods studies	10 (4.2%)	1 (1.6%)	11 (3.6%)
Pilot/feasibility studies	17 (7.1%)	3 (4.9%)	20 (6.6%)
Others	22 (9.2%)	9 (14.5%)	31 (10.2%)
Intervened setting†
Community	61 (25.6%)	33 (53.3%)	95 (31.5%)
Primary healthcare	124 (54.2%)	14 (23.3%)	139 (47.9%)
Hospital	58 (24.4%)	5 (8.1%)	63 (20.9%)
National programmes or policies	12 (5.0%)	11 (17.7%)	23 (7.6%)
Intervened group†
Healthcare workers	136 (57.1%)	30 (48.4%)	167 (55.3%)
Patients	121 (50.8%)	13 (21.0%)	134 (44.5%)
Lay people	24 (10.1%)	7 (11.3%)	31 (10.3%)
Community	48 (20.2%)	42 (67.7%)	90 (29.9%)
Health workers and patients	55 (23.1%)	7 (11.3%)	62 (20.6%)
Intervention approach†
Collaborative care models	25 (10.6%)	4 (6.6%)	29 (9.7%)
Task-shifting	46 (19.4%)	8 (13.1%)	55 (18.3%)
Clinical practice guidelines or protocols	48 (20.3%)	7 (11.5%)	56 (18.7%)
Patient self-management	48 (20.3%)	1 (1.6%)	50 (16.7%)
Use of technology	38 (16.0%)	4 (6.6%)	43 (14.0%)
Organisational change	33 (13.9%)	10 (16.4%)	44 (14.7%)
Physical activity	28 (11.8%)	–	26 (9.3%)
Health education	77 (32.5%)	15 (24.5%)	92 (30.7%)
MDA or polypill	1 (0.4%)	15 (24.6%)	16 (5.3%)
Communication	16 (6.8%)	12 (19.7%)	28 (9.3%)
Training	23 (9.7%)	8 (13.1%)	31 (10.3%)
Outcome of the study
Positive	149 (62.3%)	30 (48.4%)	180 (59.4%)
Negative	15 (6.3%)	7 (11.3%)	22 (7.3%)
Equivalent	18 (7.5%)	5 (8.0%)	23 (7.6%)
Not specified	46 (19.3%)	14 (22.6%)	60 (19.8%)
Not applied	11 (4.6%)	6 (9.7%)	18 (5.9%)

*The total column includes those studies that were conducted in NCD and NTD area.

†More than one label could be included in one study.

cRCT, cluster RCT; MDA, Mass Drug Administration; NCD, non-communicable disease; NTD, neglected tropical disease; RCT, randomised controlled trial.

Study design and context

Seventy-five studies (75 of 303, 25%) were RCTs or cluster RCTs (cRCTs), 19 of 303 (6%) were qualitative studies and 11 of 303 (3%) were mixed/multimethod studies (table 1). Furthermore, 139 of 303 (48%) studies were implemented in primary healthcare setting, and 85 of 303 (28%) studies were conducted in urban areas (31% NCDs and 18% NTDs).

Characteristics of the evaluated interventions and study results

Most of the interventions aimed at management (139 of 298, 47%) or prevention (99 of 298, 33%) of specific conditions, with few studies focusing on rehabilitation (13 of 298, 4%) or including more than two types of interventions (eg, prevention and management, 14%) (figure 2). Additionally, the most common interventions were: health education (92 of 300, 31%); clinical practice guidelines (56 of 300, 18%); task-shifting (55 of 300, 17%); self-management (50 of 300, 17%) and use of technology (42 of 300, 14%). Health education was frequently reported with other interventions, such as self-management (21 of 300), task-shifting (20 of 300) and training of health personnel (17 of 300). Conversely, the least common interventions were peer support, vector control, screening and epidemiological surveillance. In terms of target population, 62 of 300 (21%) of the studies’ interventions focused on health workers and patients; and other groups were intervened such as community members or lay people. Most of the studies reported positive results of the efficacy of their studies (180 of 303, 59%), whereas only 22 of 303 (7%) reported negative results. More information is shown in table 1.

Figure 2.

Type of the interventions in the included studies.

Frameworks and theories

Only 35 of 303 (12%) studies defined their study design as ‘PE’ (13% for NCDs and 6% for NTDs). Other terms used by researchers included ‘mixed methods study’, ‘qualitative study’, ‘feasibility study’, ‘pilot study’ or terms related to PE outcomes such as ‘fidelity’, ‘acceptability’, ‘evaluation of interventions/programmes’ or ‘implementation/integration of interventions’.

Eighty-one of 303 (27%) studies adopted a theory or framework in the PE. Ten studies used classic qualitative approaches (ie, ground theory, phenomenology, content analysis, etc), eight studies used the MRC framework for PE, seven studies used the RE-AIM framework, four used the Donobedian model and three used the Consolidated Framework for Implementation Research (CFIR). Also, the Theoretical Domains Framework, the Andersen healthcare utilisation model and PRECEDE-PROCEED model were used by two studies each. The other 42 studies used other theories or frameworks such as the normalisation process theory, Saunders frameworks for cRCT, among others. When exploring how the theories or frameworks were used, 53 of 81 (66%) studies applied the given theory or framework; 15 (19%) studies were informed by the theory or framework; 11 of 81 (14%) studies tested the theory or framework; and 2 studies used more than one, for example, applied and tested the theory. Some examples of the use of theories and frameworks are Morrison et al⁴¹ who used the MRC framework to describe the implementation and fidelity of a Participatory Learning Action in an RCT in diabetes in Bangladesh; Gyamfi et al⁴² who developed their interview guide using the CFIR focusing on resources needed to implement a task-shifting intervention in Ghana; and Hashimoto et al⁴³ who analysed the process and effects of implementing a vector surveillance system for Chagas diseases in the health centres from Honduras using the PRECEDE-PROCEED model (online supplemental material 5).

Two hundred and sixty-six of 301 (89%) studies stated the purpose of the PE and 177 of 299 (59%) mentioned the causal assumption of how the intervention will work. Also, 53 of 303 (19%) studies had a prespecified protocol.

Methods applied to conduct PE and PE outcomes

Most of the PE study designs were qualitative studies (97 of 303, 32.3%) or observational studies (92 of 303, 30.3%) (table 2). Ninety-seven studies of the 300 (32%) exclusively used qualitative methods and 64 of 300 (21%) used mixed methods according to the study design of the PE (table 2).

Table 2.

Methods applied in process evaluation

	NCD (N=239)	NTD (N=62)	Total (N=303)*
Study design
Observational studies	72 (30.4%)	18 (29.5%)	92 (30.3%)
Feasibility	41 (17.3%)	7 (11.5%)	48 (16.0%)
Qualitative study	73 (30.8%)	23 (37.7%)	97 (32.3%)
Mixed or multimethods	51 (21.5%)	13 (21.3%)	64 (21.4%)
Methods
Quantitative	60 (25.1%)	17 (27.4%)	78 (25.8%)
Mixed or multimethods	97 (40.6%)	30 (48.4%)	127 (41.9%)
Qualitative	82 (34.3%)	15 (24.2%)	98 (32.3%)
Qualitative methods†
Individual interviews	134 (76.1%)	31 (68.9%)	166 (74.8%)
Focus groups	60 (35.9%)	22 (48.9%)	83 (39.0%)
Observations	34 (20.2%)	13 (28.9%)	47 (22.0%)
Others	11 (6.6%)	5 (11.1%)	16 (7.5%)
Quantitative methods†
Routine monitoring data	60 (38.5%)	20 (43.5%)	80 (39.4%)
Descriptive analysis	104 (66.7%)	29 (63.0%)	133 (65.5%)
Advanced quantitative methods	31 (19.8%)	8 (17.4%)	39 (19.2%)
Factorial analysis	4 (2.6%)	–	5 (2.5%)
Integration of process evaluation and main study outcome
Yes	169 (70.7%)	42 (67.7%)	212 (70.0%)
No	70 (29.3%)	20 (32.3%)	91 (30.0%)
Team leading the process evaluation
Completely independent	35 (14.6%)	17 (27.4%)	53 (17.5%)
Not independent	145 (60.7%)	31 (50.0%)	176 (58.1%)
Not specified	59 (24.7%)	14 (22.6%)	74 (24.4%)

*The total column includes those studies that were conducted in NCD and NTD area.

†More than one label could be included in one study.

NCD, non-communicable disease; NTD, neglected tropical disease.

Between the qualitative methods applied, interviews were the most common with 166 of 222 (75.2%), followed by focus groups with 83 of 213 (39.2%). In the case of the quantitative approaches, descriptive analysis with 133 of 203 (66%) and routine monitoring data with 80 of 203 (39%) were the most common (table 2). Data collection for the PE was recorded only post-intervention in 92 of 301 (31%) studies, whereas it was recorded only during intervention in 75 of 301 (25%) studies, and in 31 of 301 (10%) studies it was not clear at which moment these data were collected (figure 3). Also, 212 of 303 (70%) studies integrated their main outcome results with the results of the PE and 53 of 303 (17%) studies had a completely independent team leading the PE.

Figure 3.

Time of data collection for the process evaluation.

As an example, one study evaluated the diabetes programme of the Mexican Institute of Social Security using only qualitative methods.⁴⁴ This included interviews, focus groups, observations and case studies with different stakeholders. Another study in Colombia evaluated the Onchocerciasis Elimination Program for the Americas using routine data for its PE to confirm that the country had achieved the elimination of onchocerciasis.⁴⁵ Another case was a self-management education programme in patients with chronic obstructive pulmonary diseases, where the authors used descriptive analysis, advanced statistical analysis and qualitative methods through focus groups.⁴⁶

The most common measured outcomes in these PEs were acceptability, barriers and facilitators for the implementation of interventions, experiences and perceptions, and feasibility (table 3). Other outcomes, measured in less than 5% of the studies, were adaptation, penetration and mechanism of impact.

Table 3.

Process evaluation outcomes

Outcomes*	NCD (N=239)	NTD (N=62)	Total (N=303)†
Acceptability	77 (32.2%)	22 (35.5%)	99 (32.7%)
Adherence	42 (17.6%)	8 (12.9%)	50 (16.5%)
Fidelity	34 (14.2%)	7 (11.3%)	41 (13.5%)
Attitudes	27 (11.3%)	11 (17.7%)	39 (12.9%)
Barriers and facilitators	77 (32.2%)	23 (37.1%)	101 (33.3%)
Contextual factors	43 (18.0%)	15 (24.2%)	59 (19.5%)
Experiences and perceptions	71 (29.7%)	22 (35.5%)	93 (30.7%)
Feasibility	85 (35.6%)	17 (27.4%)	102 (33.7%)
Feedback	32 (13.4%)	9 (14.5%)	41 (13.5%)
Satisfaction	30 (12.6%)	4 (6.5%)	35 (11.6%)
Sustainability	20 (8.4%)	6 (9.7%)	26 (8.6%)
Reach	22 (9.2%)	8 (12.9%)	30 (9.9%)
Adoption	30 (12.6%)	11 (17.7%)	41 (13.5%)

*More than one label could be included in one study.

†The total column includes those studies that were conducted in NCD and NTD area.

NCD, non-communicable disease; NTD, neglected tropical disease.

Barriers and facilitators for intervention implementation

The review of a subsample of 50 randomly selected articles enabled the identification of some barriers and facilitators to implement complex interventions in LMICs. The main themes identified were contextual factors including: health systems, human resources, medicine, equipment, attitudes, policy environment and others. For example, Anand et al⁴⁷ mentioned the following barriers: ‘Waiting time and out of pocket expenditure for the blood glucose testing emerged as major issues. The general reaction of the participants to disclosure of high-risk status was that of distress and denial’. Nicholls et al⁴⁵ highlighted as facilitator ‘the health education component also addressed improving the inhabitants’ living conditions and health literacy’. Details of barriers and facilitators are provided in table 4.

Table 4.

Barriers and facilitators to implement complex interventions identified by the process evaluation

	Barriers	Facilitators
Contextual factors	Contextual factors (14 studies): Sociocultural environment Natural disasters or seasonal trends Lack of financial resources Poor physical environment of the intervened setting (school) Parent and socioeconomic-related issues Social environment does not endorse healthy behaviours to prevent and control NCDs New responsibilities for participants as caregiver for parents/children Work schedule No money to pay transport Relocation Poverty	Contextual factors (one study): Access to the healthcare services was not a limitation
Health system	Health systems (17 studies): Low funding and poor organisation of the health system Poor infrastructure (periods without electricity) Limited access to physicians Limited health insurance Low evidence of based practices Poor follow-up of patients and lack of quality in medical records Absence of protocol for disease management Low contrareference of patients Poor integration of health workers in the intervention team Poor infrastructure in the health facility for conducted intervention Delay in receiving the results and the quality of the examinations Weak monitoring systems Need for context-adapted guidelines Low availability of medicines (four studies) Low availability of equipment’s and supplies (six studies)	Health system (seven studies): Strong primary healthcare structure for NCD and good funding Effective referral systems and availability of resources Network of healthcare centres Organisation between health services Good availability of resources Good availability of drugs (two studies): Facilities were adequately equipped and had sufficient reagents (one study)
Human resources	Human resources (14 studies): Lack of health personnel and few health specialists Low availability of trained health professionals High workload of healthcare workers due to the intervention Staff turnover Poor training of health providers	Health resources (two studies): Physician recognised that health providers have more skills and the prescription is better Health providers receive strong support from their respective departments
Attitude	Attitudes (five studies): Low trust in the intervention or poor knowledge about the intervention Cultural attitudes Fears and myths Poor motivation of local leaders Poor involvement due to low-risk perception to get sick or to have severe disease (six studies) Low trust in the project (three studies)	Buy-in from the health workers and other intervened groups (four studies): ‘Health providers consider the integrated screening relevant’ Physician’s duties were delegated to the nurse and patients have become motivated to participate in the classes, ‘They developed a sense of belonging to the project…’ Community engagement or social support (10 studies): Support from neighbours Peer leaders value the emergence of emotional support Group and individual contacts provided opportunities for encouragement and attention to emotional and motivational issues ‘Improve the patients' self-esteem and to overcome internal stigma which in turn reduced social isolation’ Good attitude of the staff and good communication skills (four studies)
Policy and organisation factors	Policy factors (five studies): Deficiencies in the regulatory sector between legal norms and health policies Absence of a primary healthcare-level policy Policies without guidance or a proper implementation Change of local government Policies without local evidence and high turnover of decision makers Poor coordination between stakeholders (five studies)	Political support or support from decision-makers (eight studies): Medicines were included in the list of essential drugs Health centres are supported by district health authorities and their development partners Political and technical stakeholders were very positive towards the programme
Others	Other specific barriers related to each project: Poor interaction between participants in the support group, differences between HIV and NCD services Stigma	Positive impact in process evaluation outcomes (seven studies): Positive experiences of patients Patients accept the intervention The technological tool was acceptable to community and doctors Rigour in implementation and good reach Providers felt that they had observed changes in patient behaviour Positive characteristics of some intervention components (six studies)

NCDs, non-communicable diseases.

Implications for PE in LMICs

Of the 50 articles, some mentioned how the PE data would be used to inform research, decision-makers or other stakeholders’ next steps. For example: (1) to improve the intervention without mentioning the next step or to improve the intervention for a future RCT (17 studies); (2) future implementation of the study in the health system (7 studies); (3) scale up the intervention (5 studies); (4) further research to expand the intervention to other settings or other diseases (4 studies); and (5) identification of PE outcomes relevant for future studies (2 studies). Some examples include Venables et al⁴⁸ who mentioned possible next steps: ‘We could also consider expanding antiretroviral treatment and NCD refills into community venues, as has been done in other contexts, although this would depend upon the strength of the group dynamic’ and Perez et al⁴⁹ who declared intentions to scale up the intervention: ‘The process-oriented fidelity research adopted offers insights for both strategy developers and Aedes aegypti Control Program managers to design implementation plans for further scaling-up and institutionalization of the participatory strategy.’

Discussion

Main findings

In this scoping review, 303 studies were identified, most of them focused more on NCDs than on NTDs. More than 50% of the studies were conducted in upper middle-income countries. There was also a wide variety of methods used to conduct the PEs, particularly in terms of data collection approaches and analysis, as well as use of theories and frameworks. The most common interventions described were for health education with focus on management of the diseases. Findings show that 1 of 10 studies used other labels for PE studies and about one-third of the studies used a theory or framework for their PE. Additionally, 42% used mixed methods to report their findings. The most frequent outcomes of PEs were acceptability, barriers and facilitators, experiences and perceptions, and feasibility. In a subsample of 50 studies, the most common barriers for implementation of the interventions were contextual factors including: the social determinants of health, health system factors, related to infrastructure, organisation and human resources, for example, the scarcity of health providers or limited medical skills. In contrast, facilitators mentioned were support from decision-makers, community engagement and social support.

Comparison with other studies

Recently, two systematic reviews explored the use of PEs.^23–25 The study by Liu et al^{23 24} included 69 studies and only 22 (31%) were labelled as PE, which is higher in comparison with the current scoping review where only 12% used this label. It is noteworthy that many studies are using other terms for the same methodology in LMICs. This review found only 27% of studies adopted a theory or framework in the PE, which is close to the figures reported by Liu et al^{23 24} and Scott et al,²⁵ 31% and 38%, respectively. It is possible that this is related to the scarce use of the label ‘PE’, because if a study is not recognised as a PE, it will not use a theory or framework for PE. Additionally, it could be related to the fact that some researchers do not think that the current theories or frameworks fit their design or findings. Also, other potential reasons are poor knowledge about the existence of these theories and/or frameworks, poor matching between them and data collected, difficulties to select one between the huge number of available theories and/or frameworks or it could be related to the fact that most of them were developed in high-income countries. For example, a previous systematic review published in 2018⁵⁰ found 108 feasibility studies or PE studies that used the normalisation process theory, but were conducted only in high-income countries. The critiques identified in 10% of the included studies were the complexity of the technical vocabulary and constant overlap between the definitions or interpretations of the theory domains. One study in the review⁵¹ complained that the focus is in the participants and the mechanism of action instead of the interaction of the participants with the context, and another study⁵² mentioned that the theory focused too much in the professional participants and not in the patient.⁵⁰

Concerning the design of PEs, our scoping review found a higher proportion of mixed or multimethod analysis (42%) in comparison with previous systematic reviews.^{24 25}

Our scoping review found that 177 (59%) studies mentioned the causal assumption of how the intervention will work; this is similar to Liu et al²⁴ where 46 (67%) studies have a clear description of the intervention and/or clarification of the causal assumption. This feature is important because complex interventions have many components, therefore, it is relevant to understand the intervention and how it will achieve the expected change. Also, only 53 (19%) articles in this review included a prespecified protocol, which is a lower percentage than Liu et al²⁴ found, where 30 (48%) studies had a prespecified protocol. This could be related to the fact that Liu et al²⁴ only included RCTs and it is more possible that the PE protocols would be published independently or alongside the intervention protocol.

PE outcomes reported in the current scoping review were acceptability, barriers and facilitators, experiences and perceptions, and feasibility. This is different to Scott et al²⁵ who focused on PE studies in different diseases areas, where barriers and/or facilitators to implementation were by far the most reported outcomes in 72% studies; and experiences and perceptions and adherence and fidelity were the next more commonly reported outcomes. On the other hand, Liu et al²⁴ identified that 20% of studies explored the influence of contextual factors in the intervention and they highlighted as a strength that they included studies that triangulated qualitative and quantitative results of acceptability; another strength was the determination of the intervention fidelity, for example, use of routine data or e-health indicators, among others, to determine this outcome. There is a huge variety of PE outcomes and there is certain overlap between definitions, for example, adoption and adherence are outcomes that in practical terms are the same. Also, the measure of these outcomes could be made through diverse methods (qualitative, quantitative or both) without knowing exactly which method is more appropriate to measure a specific outcome. Additionally, the use of strategies to incorporate the measurement of PE outcomes into routine is a good option, however the quality of the data and the overload of work for health professionals could be problematic.

In terms of barriers and facilitators for implementation of interventions identified in our study, we found a variety of factors related to the context, health system, human resources and policy. The most common barriers related to health systems were low availability of equipment, supplies and drugs as well as the organisation of the health centres or hospitals. This is similar to other studies, where one systematic review of studies that integrate the management of HIV and chronic services found that the main barriers in the health system were: (1) the need of effective collaboration and coordination between health teams, health personnel and patients; (2) limited trained health workers with the adequate skills and incentives; and (3) scarce institutional support and resources.⁵³ Acknowledging and identifying the main barriers could allow the identification of modifiable barriers which can be addressed, and barriers which are not modifiable and need to be monitored to understand the impact in the success or failure of the implementation of complex interventions. Although this is key for the future implementation of complex interventions, few studies look at contextual factors and health system indicators.

Implications for research and practice

It has been described that it can take 17 years between innovation to the actual application in practice, and only 14% of the evidence is implemented.^{54 55} In order to alleviate this, PEs can serve as a useful approach to identify what needs adjustment and/or changes required before implementation of an intervention to avoid wasting opportunities to improve the translation of knowledge into daily practice. In addition, it is important to consider that the health system is complex and dynamic and measuring changes through PE is relevant in order to understand the interaction between the intervention and the health system.

This review has allowed the identification of several weaknesses in the literature in PE and proposes the following four recommendations. First, as for other methodologies or study designs such as systematic reviews or clinical trials, there is a need for standard protocols such as PRISMA⁵⁶ or Consolidated Standards of Reporting Trials⁵⁷ for PEs to guide researchers in the presentation of their work with clear aims, design, data collection tools and analysis. Next, there is need to explore and understand the reasons for the scarce use of theories and/or frameworks. Third, another element is the importance of conducting and publishing PE of studies with ‘negative’ efficacy results to identify if some components of the intervention were positive or if the problem was the implementation of the intervention or the inherent lack of efficiency of the intervention. For example, a study with the aim of comparing thermometry and mHealth versus thermometry alone to prevent diabetic foot ulcers was negative.⁵⁸ However, the PE identified that the adherence to using thermometry was high in both groups showing that some aspects of the overall intervention were positive even though the overall efficacy result was negative. Another area that needs to be addressed is the importance of capacity building in research teams to strengthen their skills in qualitative and mixed methods. Also, additional resources needed in order to collect and analyse diverse types of data are necessary to foster the use of PEs. Another recommendation is the development of a guideline for the reporting of PEs, including PE outcomes in order to make these studies more homogeneous and comparable, which would be very valuable. Moreover, this tool could guide researchers on how to select theories and/or frameworks. Finally, another finding in this scoping review was the small number of PE studies in the area of NTDs, even when this group includes 20 conditions and represents a high burden for many LMICs. There is need for this type of research to close gaps to achieve better prevention, diagnosis, management or even elimination of NTDs.

Given that PE is key for different stakeholders (funders, researchers institutions, policymakers and communities), it is imperative for these stakeholders to take on board the recommendations presented above to support the improvement in the design of this methodology in order to achieve better results.^{24 25} Different stakeholders can take different actions according to their roles: funders can provide financial resources to conduct PE and request a proper design; researchers can provide protocols and reporting guidelines in order to improve their quality and to strengthen the capacity building of their teams in PE, qualitative research and mixed methods; policymakers can get involved in the PE and commit to apply the results for future implementation of the intervention; and finally communities to be active in providing information and critical thinking to improve the diverse components of the complex intervention.

Strength and weaknesses

Some limitations and strengths of our study need to be reported: (1) our search strategy was not peer reviewed,⁵⁹ but it was designed with an experienced librarian and it was built based on the systematic review of Liu et al²⁴; (2) we did not include all languages due to limited human resources but we expect that the four major selected languages covered the majority of relevant literature; (3) we are using the definition of LMIC from the World Bank 2019; however, it is possible that since 2008 there have been some changes in the categories or that the data collection was conducted when the country was not classified as an LMIC; (4) grey literature was not included but six databases from different regions ensure that most of the literature in the topic has been included; (5) the data extraction was not duplicated because of the high number of studies; this was mitigated by previous specific training of the two reviewers and frequent meetings to clear doubts; (6) only a subset (n=50) of studies were included in the subsample analysis of barriers and facilitators, which may have limited the number of identified barriers or facilitators, (7) due to the limited use of the term PE (many studies used other labels), it is possible that some articles were not identified by our search strategy despite our use of a broad terminology. However, in the current scoping review, only 12% of the articles were labelled as PE, so we were able to identify a good proportion of studies without this label. Other strengths of this study are that it explores PE of RCTs and non-RCTs, also the number of articles included allowed us to have a proper overview of the research in the topic.

Conclusions

PE is a methodology that allows researchers, practitioners and decision-makers to understand how a complex intervention is implemented. This information is particularly relevant in LMICs because of the challenges faced during the implementation of complex interventions due to their context and health systems.

Multiple gaps were identified such as scarce use of the labelled PE, poor use of theories and frameworks, and heterogeneous reporting of these studies. The mentioned findings from the current scoping review reinforce the findings from the two previous systematic reviews.^{24 25} Guidelines are needed in order to standardise the reporting of PE studies, which should improve the identification of barriers and facilitators that will inform future research or scale up of complex interventions in NCDs and NTDs.

Data availability statement

Data are available upon reasonable request. The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Ethics statements

Patient consent for publication

Not required.

Ethics approval

Not applicable.