Support and performance improvement for primary health care workers in low- and middle-income countries: a scoping review of intervention design and methods

Abstract

Primary health care workers (HCWs) in low- and middle-income settings (LMIC) often work in challenging conditions in remote, rural areas, in isolation from the rest of the health system and particularly specialist care. Much attention has been given to implementation of interventions to support quality and performance improvement for workers in such settings. However, little is known about the design of such initiatives and which approaches predominate, let alone those that are most effective. We aimed for a broad understanding of what distinguishes different approaches to primary HCW support and performance improvement and to clarify the existing evidence as well as gaps in evidence in order to inform decision-making and design of programs intended to support and improve the performance of health workers in these settings. We systematically searched the literature for articles addressing this topic, and undertook a comparative review to document the principal approaches to performance and quality improvement for primary HCWs in LMIC settings. We identified 40 eligible papers reporting on interventions that we categorized into five different approaches: (1) supervision and supportive supervision; (2) mentoring; (3) tools and aids; (4) quality improvement methods, and (5) coaching. The variety of study designs and quality/performance indicators precluded a formal quantitative data synthesis. The most extensive literature was on supervision, but there was little clarity on what defines the most effective approach to the supervision activities themselves, let alone the design and implementation of supervision programs. The mentoring literature was limited, and largely focused on clinical skills building and educational strategies. Further research on how best to incorporate mentorship into pre-service clinical training, while maintaining its function within the routine health system, is needed. There is insufficient evidence to draw conclusions about coaching in this setting, however a review of the corporate and the business school literature is warranted to identify transferrable approaches. A substantial literature exists on tools, but significant variation in approaches makes comparison challenging. We found examples of effective individual projects and designs in specific settings, but there was a lack of comparative research on tools across approaches or across settings, and no systematic analysis within specific approaches to provide evidence with clear generalizability. Future research should prioritize comparative intervention trials to establish clear global standards for performance and quality improvement initiatives. Such standards will be critical to creating and sustaining a well-functioning health workforce and for global initiatives such as universal health coverage.

Background

Key Messages

• Comparative examination of a range of approaches to performance and quality improvement programmes targeted at primary health care workers in low- and middle-income countries.

Primary health care workers (HCWs) in LMICs typically work in challenging environments. They are frequently stationed in remote, rural areas and charged with working on the front lines of their country’s health system as the first point of patient contact. They have often received inadequate or incomplete education and training, and have limited resources and tools at their disposal. And yet, the role of primary HCWs remains critical to health care delivery in LMICs, especially as complex, longitudinal care interventions are implemented and scaled up. Such interventions, including antiretroviral therapy for HIV/AIDS, have increasingly employed a decentralized health systems approach to coverage (Suthar et al. 2014), relying on task-shifting to non-physician cadres of health workers to achieve access at scale (Lazarus et al. 2014). This approach is a direct result of attempts to cover a large population within the constraints of significant health workforce shortages. Such shortages are notorious barriers to the delivery and quality of health care in LMICs (Haines et al. 2007). Primary HCWs are thus required to do more and to provide greater complexity of care. Robust support mechanisms to maintain high standards of quality and performance are therefore required.

Traditionally, poor performance of health workers was considered a result of poor education and a lack of knowledge and skills, and that as long as providers knew the correct care and decisions to make, they would implement them (Brugha and Zwi 1998). The majority of interventions to improve health worker performance have, therefore, focused on education, training, and dissemination of evidence-based guidelines through didacticism. But this singular approach has had mixed, if not disappointing, long-term results (Oxman et al. 1995). A review of health worker performance in low-resource settings, found that dissemination of written materials and guidelines alone – often through in-service training courses – without additional post-training support interventions, was usually ineffective in improving performance and quality (Rowe et al. 2005b). It was noted that supervision with audit-and-feedback techniques usually proved an effective complement to training, and that multi-pronged support interventions were more effective than single interventions. This suggested that a broader approach to health worker support was needed. Fritzen (2007) and Bach (2001) have noted that typical training of health personnel emphasizes factual and specialized medical knowledge, but in order to perform their job adequately, they have to identify and analyse problems in real-time, supervise and audit of other workers and processes, as well as coordination across multiple levels of the health system, including communities.

Interventions to support and improve health worker performance and quality of care have taken on myriad forms, functions, and structures, and have been known by various names, including ‘supervision’, ‘mentoring’, and ‘quality improvement.’ There is currently no clear consensus on what distinguishes these approaches from one another, nor agreement on which approach is more effective. In an attempt to provide some clarity on the design and implementation of programs aimed at supporting primary HCWs in LMICs, we conducted a systematic literature search and a comprehensive narrative review of the literature identified. Our objective was to generate a comprehensive description of the current range of support and performance improvement interventions for primary HCWs in LMICs, and to identify generalizable themes and explanations as to how these approaches are designed, and what distinguishes them from one another. In this review, performance was not restricted to a single definition, but rather left broadly to include a range of metrics commonly used to assess health worker practice, knowledge, skills, and behaviours against pre-defined standards set out by a specific program, as well as additional dimensions of quality including patient and provider satisfaction and motivation. We did not formally include clinical outcomes, cost or cost-effectiveness, or health care utilization as performance endpoints as we focused more specifically on HCWs themselves, though these are important dimensions of quality in their own right. Primary health care workers in this review were defined as all non-physician front-line health care providers. This includes facility-based nurses, clinical officers, medical and physician assistants, etc., as well as community health workers. While we recognize that in some LMIC settings physicians are also a part of front-line care teams, we elected to restrict our definition to non-physicians as this reflects the bulk of our direct experience working in sub-Saharan Africa and South and Southeast Asia, regions that are home to the majority of countries classified as LMICs as well as the majority of countries that fall within the lowest two tiers of physician density (<0.5 per 1000 and <1 per 1000) based on most recent WHO data (WHO 2016). Finally, we also aimed to establish a basic understanding of some general and unifying principles of support interventions intended to improve quality and performance of primary HCWs in LMICs.

Methods

Search strategy

Using a pre-defined search strategy, we systematically searched the literature for relevant articles. Databases searched included MEDLINE, OVID, Web of Science, EMBASE, EBSCOHOST, and PubMed, restricted to publications in English. To identify relevant unpublished grey literature, we also searched Google and Google Scholar, as well as websites of the World Health Organization (WHO), Department of Human Resources for Health, The Global Health Workforce Alliance, Human Resources for Health (HRH), Global Resource Center from the United States Agency for International Development (USAID), and the International Training & Education Center for Health (I-TECH) at the University of Washington. With assistance from a research librarian, a comprehensive search string was created to systematically identify publications reporting on interventions to support health workers in low and middle-income nations (Table 1). Terms were truncated to alternative spellings or tenses and no methodological or study design filters were applied given the wide range of observational, experimental and quasi-experimental study designs used in research of large-scale health interventions, especially in LMICs. After reviewing results, additional targeted searches in PubMed were conducted by frequently published authors or institutions, by reviewing the bibliographies of included studies and by studies known a priori to the authors.

Table 1.

Search String

Search String

"developing world" OR "developing countries" OR "rural" OR "low middle income" AND ("health worker" OR "health care worker" OR "nurse" OR "nursing" OR "community health worker" OR "physician" OR "human resources" OR "personnel" OR "worker") AND ("mentor" OR "support" OR "train" OR "supervision" OR "advise") AND ("tools" OR "checklist" OR "curriculum" OR "guideline") AND ("evaluation" OR "appraisal" OR "validation") AND ("delivery of health care" OR "quality of health care" OR "quality improvement" OR "quality assurance")

Study selection

The primary search was conducted by the first author with the approval of the search criteria and screening methodology from the remaining co-authors. The results of the electronic database search were first screened by title and abstract for relevance. Studies from high-income settings were excluded, along with studies that contained some of the search terms but did not explicitly address the question at hand, which was to propose, describe, implement or evaluate a performance support or improvement program for frontline non-physician HCWs in LMICs. Letters to the Editor, Response to Authors, short editorial articles and short commentary pieces that did not report data were also excluded, although narrative reviews were included. Otherwise, no particular preference for inclusion was given to any specific study design or writing format.

The second stage of screening involved a detailed reading and review of the remaining articles and an attempt to group the articles by topic. Inclusion criteria included studies that explicitly addressed topics of supervision, supportive supervision, performance and/or quality improvement, mentoring and clinical mentoring, or coaching. Studies that did not explicitly address these issues, or that included one or more of these techniques packaged within a larger intervention, were excluded. Primary health care was defined as care provided at the first level of the health system, whether in a facility- or community-based setting, and focused primarily on ambulatory care. Exclusion criteria included articles that focused on hospital-based, inpatient management, articles that addressed performance and quality in procedural, laboratory, or radiology-based skills, as well as articles that targeted pre-service or in-service training as the primary intervention without adequate description of a program of sustained follow-up, support or re-training, and discussion or analysis of its impact. Such studies do not address the question of ongoing, sustainable in-service practice support and performance improvement of trained primary health care workers. Articles simply describing the level of quality of care, without attention given to health worker support and performance improvement, were also excluded.

Categorization and evidence review

The articles included in the review were grouped into general topic areas. Three primary categories of literature emerged: (1) Supervision and supportive supervision; (2) Mentoring and clinical mentoring; (3) Tools and Aids. Two secondary categories, with more limited literature, were also identified: (4) Quality and performance improvement, and (5) Coaching and peer-review strategies. In cases where an article had obvious overlap between more than one category and where articles did not distinctly fit in one category, the authors decided by consensus review on the best fit. While we attempted to restrict this review to studies solely depicting HCW support programs, initial results were limited and thus relevant multi-component interventions that included a well-defined and detailed description of HCW support components, were also included in order to strengthen and broaden the evidence base.

Results

The overall results of the literature search and study selection are shown in Fig. 1. In total 347 articles were identified in our search, 114 were included for full-text review, and 40 were included in the review and categorization exercise. Of these 40 papers, 13 papers used cross-sectional survey study designs based on pre-/post-intervention plausibility design. Eight papers involved qualitative research using a variety of methods (realist evaluation, focus group discussion, key informant interviews). Seven papers described randomized controlled trials (RCTs) or studies nested within RCTs, and another seven papers were Reviews, three of which were systematic reviews with or without meta-analyses. Other study designs included were four program evaluations (using routinely collected data not under research conditions), and one time-use study. Twenty-three of the 40 studies included in the review (57.5%) were categorized under ‘Supervision or Supportive Supervision,’ ten studies (25%) fell under ‘Tools, and Aids,’ three studies (7.5%) were included under ‘Mentoring or Clinical Mentoring’ and three under ‘Quality Improvement,’ and finally only one study was found for ‘Coaching and Peer Review Strategies’ (Table 2).

Figure 1.

Results of systematic search protocol .

Table 2.

Summary of intervention studies for three principal article categories under review

Study Category	Authors	Type of study	Country	Setting	Intervention	Relevant Measured Outcomes	Impact/Relevant Findings
Supervision and supportive supervision	Hoque et al. 2014	Randomized Control Trial	Bangladesh	First-level primary health centres	IMCI training + monthly supervision w/case observation	HCW performance IMCI assessment Case management	>30% gain in correct assessment of children using IMCI Results sustained at 3 and 5 years from baseline Equivalent performance of HCWs w/18 months vs. 4 years of pre-service education
	Loevinsohn et al. 1995	Randomized Control Trial	Philippines	Primary care health posts and health centres	Enhanced monthly supervision, supported by checklist	Prenatal care Immunization rates Midwife knowledge on vitamin A	>75% increase in correct ANC record-keeping at 6 mos >30% increase in > 3 prenatal visits at 6 mos >87% increase in midwife knowledge of vitamin A dose/schedule No change in infant immunization
	Okwen et al. 2014	Randomized Control Trial	Tanzania	Primary health care facilities/dispensaries	Routine vs. Enhanced supervision for eye care after training	HCW knowledge Vision testing performance Total primary eye care (PEC)	80% improvement in PEC knowledge in intervention group vs. 59% improvement in control Vision testing score higher (1.8 vs. 0.88, p = 0.03) Total PEC score not significantly different
	Kayemba Nalwadda et al. 2013	Randomized Control Trial	Uganda	CHWs	Enhanced supervision system after training (monthly, then quarterly visits, case observation, feedback)	CHW knowledge of identification and referral of sick newborns	68% CHWs passed the knowledge exam 74% able to identify the five major newborn danger signs 98% recognized sick vs. not sick in case vignettes 96% completed referral forms correctly 63% passed the caregiver communication skills exam
	Minh et al. 2013	Pre/post intervention plausibility design	Vietnam	Front line pharmacists	Quarterly supervision after initial training	Pharmacist knowledge and management of childhood diarrhoea and emergency contraceptive pills (ECP)	Knew ≥ 3 danger signs of diarrhoea (OR 15.9, CI 9.1-29.1) after 18 mos Knew ≥ 3 dehydration symptoms (OR 32.4, CI 19.7-53.7) after 18 mos Knew when to use ECP (OR 8.4, CI 5.5-12.9)
	Mogasale et al. 2010	Pre/post intervention plausibility design	India	State STI clinics	Quarterly supervision supported by checklist/tool	STI service coverage Quality of care, prevention support, effective drugs Referral into care Community involvement	3- to 7-fold increase in all indicators over 45 month observation period at 292 STI clinics in 7 states
	Osterholt et al. 2006	Pre/post intervention plausibility design	Malawi	Government and private non-profit front line health facilities	Routine and enhanced supervision for malaria	Effective treatment with anti-malarial drugs	Supervision type (w/or w/o clinical observation) or frequency was not significantly associated with improved quality of malaria treatment
	Djibuti et al. 2009	Pre-/post-intervention plausibility design	Georgia	Primary health care providers and managers of immunization programs	Continuous, structured supportive supervision	Immunization service delivery coverage	Significant increase in DPT-3, Polio, Hep B coverage after 1 year (p = 0.000, 0.000 and 0.002, respectively) Significant reduction in vaccine wastage for DPT, OPV, Hep B (p = 0.016, 0.029, and 0.022, respectively)
	Zurovac et al. 2004.	Pre-/post-intervention plausibility design	Kenya	Front line dispensaries, health centres, and small hospitals	Routine government supervision for IMCI and malaria	Quality of care of uncomplicated malaria (error rates)	Children treated by HCWs supervised 4-10 times in past 6 months significantly less likely to receive inappropriate treatment (major errors) vs. zero visits (OR 0.28, 0.12-0.66)
	Frimpong et al. 2011	Time-use study	Ghana	Primary health care facilities	Routine government supervision	HCW productivity	Supervision within last month associated with higher proportion of time spent on direct patient care (OR = 1.57, CI 1.26-1.96)
	Hernandez et al. 2014	Qualitative; Realist evaluation	Guatemala	Primary health posts	Routine government supervision	HCW perceptions of supervision	Primary focus on managerial control Move toward approach of humanized support focused on HCW as integral actors, attentive and actively engaged and supportive supervision Leads to improved HCW recognition, initiative, and shared vision
	Suri et al. 2007	Qualitative survey and focus group	South Africa	Community health workers	Routine government supervision	CHW perceptions of supervision and administrative support	Poor monitoring and accountability mechanisms Lack of support or supervision from community health facilitators Lack of emotional and moral support from supervisors
	Manongi et al. 2006	Qualitative; focus group discussion	Tanzania	Primary health care facilities	Routine government supervision	HCW perceptions of supervision, feedback, and training	More negative comments received than positive about supervision Little supervision, the irregular supervision from DHMT not supportive No written or oral feedback Issue of poor central planning for supervision
	McAuliffe et al. 2013	Quantitative & qualitative survey	Multi-country (Malawi, Tanzania, Mozambique)	Front-line HCWs providing obstetric care	Routine government supervision	HCW perception and survey of supervision	28.7% HCWs in Malawi had no supervision at all; 21.4% in Tanzania, 9.6% in Mozambique Only 35% HCW in Malawi have formal obstetrics supervision < 10% of HCWs report supervision is available on request ∼20% had only negative feedback to report on supervision Lack of supervision significantly correlated with intention to leave job p < 0.01
	Mugala et al. 2010	Quantitative & qualitative survey	Zambia	Front line facility HCWs trained in IMCI	Supervision for HIV guidelines in IMCI	Use of HIV guidelines in IMCI algorithm	90% respondents received at least 1 supervision visit for IMCI All respondents reported supervision visits were helpful in better understanding the HIV guideline
	Mohan et al. 2011	Program evaluation using routine data	India	Primary health care	Supervision for IMCI (IMNCI) in 7 states	Implementation quality of supervision	6 of 7 districts assessed to have poor supervision
	Rowe et al. 2010	Mixed methods (records review, FGD, key informant interviews, cross-sectional survey	Benin	Public and private outpatient front-line facilities	IMCI supervision, supported by job aids and non-financial incentives	Frequency and quality of supervision within RCT of IMCI	Supervision took at least a quarter to kick off after training despite small HCW number After 5 year follow-up, rate of supervision (at least 1 visit in past 6 mos) almost twice in intervention district vs. control (88.1% vs. 47.8%) 97.2% of supervision checklists had most important section (observation and feedback) completed
	Steinmann and Bosh-Capblanch 2011	Systematic review and meta-analysis (Cochrane)	Global (LMICs)	Primary health care workers	Managerial supervision	Effect of managerial supervision on quality of primary health care, including: Adherence to guidelines Service coverage Provider knowledge and satisfaction	Only nine studies met inclusion criteria GRADE quality of evidence was "low" or "very low" Mixed results; authors are uncertain about the effect of supervision on PHC service quality
	Nguyen et al. 2013	Systematic review & meta-analysis	Global (LMICs)	Primary health care centres	IMCI training and supervision	Effect of IMCI supervision visits on: IMCI classification Medication administration Vaccination Nutrition assessment Caregiver counseling	IMCI trained HCWs w/≥ 1 supervisory visit in last 6 months vs. < 1 visit, associated with improved (versus untrained HCWs): IMCI classification (RR 2.09 vs. RR 1.88) Medication administration (RR 1.91 vs. RR 1.73) Nutrition assessment (RR 5.97 vs. RR 2.64) Caregiver instructions (RR 3.18 vs. RR 1.79) Worsening in vaccination rates (RR 1.11 vs. RR 1.81* (not significant)
	Bosch-Capblanch et al. 2008	Systematic review	Global (LMICs)	Primary health care workers	Supervision (general review)	N/A	Supervision focused mainly on checklists and administration Usually focused at district level visiting peripheral HCWs Some evidence of benefit on performance, but evidence is limited
	Clements et al. 2007	NarrativeReview	Global (LMICs)	Primary health care	N/A	N/A	Call for new model of supportive supervision based on problem-based learning and continuous feedback Need to institutionalize supervision
	Haines et al. 2007	Narrative review	Global (LMICs)	Community health workers	N/A	N/A	Supervision as a performance improvement intervention for CHWs deserves special attention Important factors: two-way flow of information between supervisor and CHW, supervisor acts as role model Challenge is scaling up successful supervision programs to national programs Supervisors may be the only formal link of CHWs to health system in rural areas Important potential of peer support as well
	Marquez and Kean 2002	Narrative review	Global (LMICs)	N/A	General review of supervision	N/A	Shift from traditional supervision emphasizing "inspection and control" to supportive supervision Supportive supervision shifts responsibility and activity from one supervisor to the wider health workforce Requires continuous implementation of supervision for sustained performance improvement
Mentoring and Clinical mentoring	Fatti et al. 2013	Pre-/post- intervention plausibility design	South Africa	Front-line primary health facilities (ANC and birthing sites)	Implementation of Quality Nurse Mentor (QNM)	Improvement in outcomes and processes for PMTCT including: % of HIV negative women retested at 32 weeks % new diagnosed women receiving CD4 testing % of HIV positive women, not on ART, receiving antenatal ZDV uptake of infant HIV testing at 6 weeks, 18 mos	Increased HIV re-testing at 32 weeks from 38.5% to 46.4% (RR = 1.2, p<.0001) ZDV uptake for eligible women increased from 80.9% to 88.1% (RR = 1.09, p<.0001) Infant testing at 6 weeks increased from 68.5% to 76.7% (RR 1.12, p<.0001) Infant testing at 18 months increased from 12.4% to 22.9% (RR 1.84, p<.0001)
	Workneh et al. 2013	Program evaluation; retrospective chart review	Botswana	Rural HIV clinics	Outreach clinical mentoring	Completion of chart for key indicators of quality pediatric HIV care, including: Weight, height, growth curve Pill count and adherence assessment Recent CD4 and viral load Use of co-trimoxazole prophylaxis Correct ART dosing	Significant improvements in all indicators at 1 of 4 sites, after one year of mentoring (p < 0.0001) Other site with improvement only in CD4, VL and pill counts (p < 0.0001)
	Anatole et al. 2013	Descriptive report/program evaluation	Rwanda	Rural primary health clinics/nurses	Enhanced district-based nurse-led mentoring	Interim program progress	Significant increases in mean proportion of assessments completed per consultation for IMCI, IMAI, and ANC after 1 year (p < 0.0001) Significant increase in correctly classified patients for IMCI after 1 year (53.3% vs. 34.6%, p < 0.0001) and IMAI (53.5% vs. 40.5%, p < 0.0001)
Tools and technology	Sodhi et al. 2011	RCT; qualitative survey	Malawi	Rural primary health clinics/HCWs	Job aid (w/training) integrating guidelines for HIV, TB and other condition into single tool	Effect of PALM PLUS tool on staff satisfaction and quality	HCW worry that tool could slow down patient consultations Time pressures dissuaded routine use of tool Not used as checklist for in-consultation decision-making No difference b/w intervention and control in job satisfaction or likelihood to quit in next 12 months, not all non-significant effects
	Zurovac et al. 2011	RCT	Kenya	Rural health facilities	Text messages (SMS) for malaria case management	Correct treatment of malaria with arthemeter-lufemantrine Dispensing and counseling	Correct treatment improved in the intervention arm immediately by 23.7% (p = 0.004) and by 24.5% at 6 months (p-0.003) Sig improvement in counseling on side effects (vomiting) immediately (p = 0.0017) and at 6 mos (p < 0.0001)
	Peters et al. 2006	RCT	India	Primary health care facilities	Computer-assisted decision-support tool for patient screening	Increase in patient throughput Global patient assessment of care HCW perceptions	Sig difference of differences of 430 patient visit in intervention vs. control sites (p = 0.005), with 18% increase in intervention vs. 5% increase in control Sig increase in difference of differences in patient assessment of care (mean 7.9, p < 0.001) Did not affect HCW attitudes or perceptions of their work
	Rowe et al. 2012	Cross sectional survey	Benin	Primary health facilities	Simulated Client (SC) vs. Conspicuous Observation (CO) for IMCI	HCW performance per IMCI protocol	HCW performance measured by CO was moderately and significantly higher than by SC (median 16.4 point differential)
	Aung et al. 2012	Cross sectional survey	Myanmar	Primary health facilities	Observed simulated patient (OSP) vs. direct observation	HCW performance in treatment of pediatric malaria	Agreement >90% in all areas of HCW performance between direct observation and OSP approach, exp for history taking on past anti-malarial use
	Rowe et al. 2000	Cross sectional survey	Central African Republic	Outpatient health facilities	Fever treatment chart	Predictors of high quality treatment of children with fever	Inverse association of use of fever chart and correct treatment (OR = 0.19, CI 0.01-0.91)
	Zeitz et al. 1993	Chart review and cross- sectional survey, pre/post	Nigeria	Rural primary health clinics	Quality assurance tools (flow charts, decision trees, checklist)	HCW performance in history-taking, exam, disease classification, treatment, and counseling	Significant improvements seen 6 of 9 history questions, 2 of 4 exam skills Sig improvement in disease classification and treatment of childhood diarrhoea
	Spector et al. 2012	Pre-/post-intervention	India	Sub-district birth centre	WHO Safe Childbirth Checklist	Impact of checklist on HCW’s frequency of performing essential birth practices	Significant improvement in 28 of the 29 measured essential birth practices Overall improvement in proportion of essential birth practices performed from 10/29 (9.4-10.1) to 25/29 (24.6-25.3) p < 0.001
	Derenzi et al. 2011	Narrative review	Global (LMIC)	CHWs	Review of mobile technologies to support CHW performance	N/A	Identified 6 main health system functions impacted by mobile technology and relevant to CHW Data collection Training and access to reference material Communication amongst HCWs Job aids and decision support Supervision Promoting healthy population behaviours
	Songstad et al. 2012	Qualitative survey and focus groups	Tanzania	Rural HCWs	Open performance assessment tool (OPRAS) and pay-for-performance scheme	HCW motivation and performance self-appraisal	Recognition of good performance important for HCW motivation Lack of regular feedback on their work Concern about OPRAS tool and its relevance, impact on performance OPRAS not linked to actual feedback Encouraged by potential impact of P4P scheme

Supervision and supportive supervision

Of all of the strategies to support primary HCWs in LMICs, supervision is the most studied and referenced, but also has the greatest diversity of definitions and implementation. Terms commonly used to describe supervisory activity in the health sector include ‘supervision,’ ‘supportive supervision,’ and ‘managerial supervision’ or ‘clinical supervision,’ each with distinctions in approach and aims. Nonetheless, these terms are generally used to describe a range of activities where a more senior professional, or a supervisor from a higher level in the health system, audits and/or directly observes the work of a primary HCW to ensure that the correct activities are being performed, that they are done effectively. In addition, appropriate guidance and support is given, aimed at helping staff to become more competent, knowledgeable and effective in their work.

Clements et al. (2007) describe traditional supervision as focused on inspection and oversight of behaviour and practices for the purposes of finding fault, with little guidance on improvement. They regard this, in part, as a vestige of colonialism, where foreign supervisory structures were designed in a hierarchical and often punitive manner aimed at local staff, and where a worker higher-up on the administrative chain was responsible for oversight of a lower-level worker and charged with ensuring that the lower level worker executed their duties appropriately. Marquez and Kean (2002) have similarly noted that the goal of supervision was geared towards ‘inspection and control’ by external actors, based on the premise that front-line workers require strong controls to induce satisfactory performance, as they tended to be unmotivated and lacking incentive for high performance.

In its primary oversight function, supervision has largely focused on administrative tasks such as facility inspection, use of resources, supply logistics, review of records, and communication of information and directives from higher to lower levels of the health system (Simmons 1987). Problem solving within this type of system tended to be reactive and episodic, with little attention to empowering front-line staff to identify, report, and solve problems proactively. In turn, supervisors were expected to deal with a broad range of problems at the facility, yet they often lacked the necessary skills and capacity. Thus, their function was largely in monitoring alone, with little emphasis placed on training and support of front-line providers, and teamwork or communication improvement. The typical mode of implementation of such a supervisory structure involved the ‘site visit’ as the primary episode of contact, where an external supervisor would make a fleeting visit to a facility, largely to complete forms and checklists.

Previous attempts to modify supervisory structures have focused largely on increasing the frequency and/or duration of these site visits and introducing tools, including supervisory guidelines and checklists, in order to highly structure the site visit. Effectiveness of supervision is generally measured by changes in the numbers or frequency of activities performed during the site visit or based on records review, but is rarely linked to actual HCW behaviour, practice improvement, or health outcomes themselves (Center For Human Services 1987, 1990).

In the post-colonial era, particularly in sub-Saharan Africa, supervision has shifted from its discrete oversight function towards a broader concept of supportive supervision. It has been described as ‘a process that promotes quality at all levels of the health system by strengthening relationships within the system, focusing on the identification and resolution of problems, and helping to optimize the allocation of resources promoting high standards, teamwork, and better two-way communication’ (Marquez & Kean 2002: p 12). Others have described supportive supervision as involving direct, personal contact, on a regular basis to guide, support, and assist designated staff to become more competent in their work (Djibuti et al. 2009). Marquez and Kean (2002) note that supportive supervision incorporates self-assessment and peer-assessment, as well as community input, in the process of performance improvement. In doing so, they argue, supportive supervision shifts the locus of supervision from a single official to the broader workforce as a whole. Multiple actors, including officially designated supervisors, informal supervisors, peers, and health providers themselves, implement it. The multiple lines of responsibility and accountability has the potential to promote quality throughout the system through stronger communication, better problem-solving, facilitation of teamwork, and the necessary leadership and support to empower primary HCWs to monitor and improve their own performance as well as of those around them (Marquez and Kean 2002). In general, this suggests that supportive supervision moves a step beyond supervision alone by focusing on broader performance improvement (Children's Vaccine Program at Path 2003).

Another core paradigm shift from supervision to supportive supervision is in changing the focus of improvement from tasks to performance, and from activities to individuals. Stinson and colleagues (1998) describe supportive supervision as emphasizing joint problem-solving, mentoring, and two-way communication between supervisors and those being supervised, which is usually based on a sustained, longitudinal relationship between supervisor and supervisee. They also note, for example, that while supportive supervision may include some element of the audit and performance review, it refers to an ongoing relationship between providers and supervisors, a characteristic that is usually not captured in traditional supervision programs based on single site visits (ibid:. 1998). Working through this dyadic relationship between supervisor and HCW, some of the major functions are to set clear expectations, to monitor and assess performance, to identify problems and opportunities, and to take action on these issues as needed. Additionally, there is a role for setting individual HCW performance objectives and for monitoring and managing performance problems as they arise, in addition to motivation, feedback and guidance in problem solving, on-site training, and assistance with managing resources and logistics (ibid:. 1998).

This model of supervision, based on a sustained relationship between supervisor and supervisee, also serves an important human function by connecting often remote and isolated facilities and providers to the wider health system. Valdez and colleagues (1990) emphasize the importance of sustained supportive supervision within the context of increasing health service decentralization and strengthening of rural care. In light of increasing decentralization, they argue that the lower and more remote levels of the health system need more capacity building and connection with ongoing supervisory and management structures as they are burdened with a heavier workload and with the need for improved quality (Mills et al. 2001). Senunn and colleagues (2006) also emphasize the role of supervision in connecting rural and urban levels of the health system, emphasizing the role of urban supervisors in connecting peripheral facilities and ensuring adequate managerial oversight, but also in facilitating communication and improving quality at the front lines of health systems.

There is extensive agreement that supervision forms an important part of human resources management for the delivery of health services (Simmons 1987). This was corroborated, for example, in a survey of sixteen field-based organizations by USAID, who agreed on the importance of supervisory structures in ensuring quality care (Marquez and Kean 2002). But what remains in question is whether effective supervision in itself can definitively and generalizably improve the quality of health care and patient outcomes (Kilminster and Jolly 2000). Perhaps the most conclusive review of supervision was a Cochrane review of managerial supervision – defined as any effort at supervision linking higher levels to lower, more peripheral levels of the health system – in order to improve primary health care in LMICs (Bosch-Capblanch and Garner 2008, Kilminster and Jolly 2000). This review included randomized controlled trials, pre-/post-intervention studies, and interrupted time series studies, with only nine papers meeting criteria, highlighting the paucity of high quality evidence. Amongst the papers included in the review, the evidence for an effect of supervision on quality was equivocal (Table 2).

Despite this uncertain result from a comprehensive and stringent review, multiple individual studies suggest that supervision can improve quality of care and primary HCW performance. Randomized trial evidence has shown positive effects of supervision on care for children under-5 using IMCI (Hoque et al. 2014, Pariyo et al. 2005b, Kayemba Nalwadda et al. 2013, Amaral et al. 2004) for general primary care (including vaccination, vitamin A distribution, family planning, antenatal care, and costs per capita) (Loevinsohn et al. 1995) and primary eye care (Okwen et al. 2014). Plausibility trial evidence also supports the association of increased or enhanced supervision with quality of immunization services (Djibuti et al. 2009), STI services (Mugala et al. 2010), malaria care and treatment (Zurovac et al. 2004), management of childhood diarrhoea (Pham DM et al. 2013), and correct use of pharmaceuticals from front-line dispensaries (Ross-Degnan D et al. 2007). Additionally, a time use study from Ghana showed that increased supervision led to increased HCW ‘productivity’, defined as time spent directly on patient care activities as a measure of quality (Frimpong et al. 2011), while a study from Mexico found that supervision increased health service efficiency, equity, and cost-effectiveness (Kroeger and Hernandez 2003). Only one study showed an overall non-significant result for increased supervision on the quality of malaria care in Malawi (Osterholt et al. 2006).

Similarly, qualitative research has shown that supervision is associated with improved HCW knowledge, perception of their work and professional satisfaction and motivation, across a number of settings including Tanzania (Manongi et al. 2006), Zambia (Mugala et al. 2010), Guatemala (Hernández et al. 2014), and Uganda (Kaye et al. 2011). The positive associations of supervision on HCW motivation have also been repeated in multi-country studies (McAuliffe et al. 2013, Willis-Shattuck et al. 2008, Ahmed et al. 1993). Lack of supervision or poor quality supervision has been shown in India (Mohan et al. 2011) and South Africa (Suri et al. 2007) to be associated with poorer performance, inferior fidelity to protocols, and even treatment failure and poor patient outcomes, respectively. These country-specific findings echo policy statements that have also emphasized the need for supervision, in addition to training and other investments in primary HCW performance and quality of care, to not only improve motivation and performance but also to achieve broader global health targets (Gouws et al. 2005, Pariyo et al. 2005a, Haines A et al. 2007, Rowe et al. 2005a, Teasdale et al. 2001).

With respect to patient outcomes, a study from Rwanda showed that over a two year period, a system of task-shifting to nurses to deliver antiretroviral therapy (ART) at rural health centres, combined with weekly supervision visits guided by supervision checklists, led to > 90% patient retention, significant increases in mean CD4 counts, and patient weight over the follow-up period, which compares favourably to other ART cohorts in sub-Saharan Africa that do not employ such a model (Shumbusho et al. 2009). The challenge of this study, along with many others, is in separating out the effect of supervision in relation to the intervention as a whole, which included training, new clinical protocols, improved infrastructure, and systems of patient monitoring and reporting. Though found in our systematic search, this study was not included in our formal review, because although the supervision system was described clearly, it was part of a system-wide intervention, and thus attribution would be impossible. We found no other studies exploring the impact of supervision on patient or clinical outcomes, let alone morbidity and mortality.

While supervision is thought to have a positive impact on performance and care, there is little empirical agreement as to the optimal amount or timing of supervision. Some studies argue that more supervision generally improves performance, provided that supervision time is used on productive, high-yield activities (Loevinsohn et al. 1995). A systematic review and meta-analysis of Integrated Management of Childhood Illness (IMCI) effect on health worker performance observed a stronger association across all domains of quality within studies that reported more supervisory visits (RR = 1.11, CI 103-1.20) (Nguyen et al. 2013). Rowe and colleagues (2010) in Benin documented the rise and fall of supervision visits for IMCI using record review, focus groups, interviews, and surveys, and found that only 29% of needed supervision visits actually occurred overall, blaming poor coordination and lack of integration of supervision systems as possible explanations for these poor results; but overall quality improved with supervision, despite sub-optimal implementation of the supervision system. While a controlled trial in Brazil found that reducing the frequency of supervision for community-based contraceptive program reduced costs significantly, it had no impact on quality or primary HCW performance (Foreit and Foreit 2015). In general, more supervision is considered better, but the evidence is weak to support this and there is certainly no guidance on optimal ‘dose’ of supervision.

Mentoring and clinical mentoring

Few studies were found that explicitly addressed mentoring or clinical mentoring as the principal intervention to improve health worker performance, though it appears that mentorship – like other categories found in this review – can encompass a range of often overlapping activities to support health worker performance and delivery of care. The International Training & Education Center for Health (I-TECH: p3) based at the University of Washington, defines clinical mentoring as a ‘sustained, collaborative relationship in which a highly experienced health care provider guides improvement in the quality of care delivered by other providers and the health care systems in which they work.’ The World Health Organization (WHO) (2005: p3) defines clinical mentoring as ‘…a system of practical training and consultation that fosters ongoing professional development to yield sustainable high-quality clinical care outcomes.’ The WHO regards clinical mentors as highly experienced clinicians who provide mentoring to less-experience health workers in the form of review of clinical cases, feedback, and direct assistance in managing complex cases. Andrews and Wallis (1999) and Marquez and Kean (2002) have suggested that clinical mentoring is founded on collaboration and focuses on clinical teaching in the setting of direct patient care. The goal of clinical mentoring is to enhance knowledge, build confidence, and maintain adherence to protocols in less-experienced providers of care. Usually this is through direct, side-by-side case observation, targeted specifically at improving the clinical skills of the less experienced health care worker. This is achieved through the provision of individualized feedback and in response to provider-driven queries, mostly related to clinical reasoning, diagnosis and management, and physical exam skills, though this seems to vary widely. Clinical mentoring can be thought of as a follow-on activity to initial didactic training, involving both on-site mentoring and distance-based consultation and communication utilizing technology (Department of Health Republic of South Africa 2011).

While these definitions may seem broad, there is agreement that mentoring improves skills through direct, on-site observation of case management and the provision of targeted and individualized feedback to the provider. While clinical mentoring is most commonly part of a multi-faceted performance improvement and support intervention, there are examples of small observational studies that have examined the effectiveness of mentoring alone, or as the distinct and primary intervention, on the performance of health care workers and on program outcomes (Anatole et al. 2013, Magge et al. 2014, Workneh et al. 2013, Fatti et al. 2013) and on HCW motivation (Songstad et al. 2012).

Tools and aids

While much of the literature focuses on interpersonal or educational strategies to support health worker performance in LMICs, as described above, a separate, but important body of work focuses on the implementation of specific tools and aids that can support HCWs. These tools generally complement one of the educational approaches described above, or are implemented using teaching and educational practices, but have also been studied alone. These tools can take the form of checklists that HCWs and/or supervisors or mentors can use to monitor performance; job aids, guidelines, or protocols to provide real-time decision-support and guidance during the patient consultation; or mobile technology or e-Health tools to facilitate improved record-keeping and increased communication between less experienced or rural providers and supervisors, consultation with specialists, etc.

Checklists can target primary HCWs directly and can aid them with the provision of high-quality care through decision support during patient care, or through a retrospective review of cases. Checklists can also be aimed at mentors and supervisors to measure HCW performance and to identify gaps in care, thus serving as a basis for ongoing quality improvement efforts. It is this latter approach that has been increasingly studied, for example in Nigeria for childhood diarrhoea management (Zeitz et al. 1993), in Uganda for CHW diagnosis and treatment of paediatric malaria and pneumonia (Mukanga et al. 2011), for delivery of essential birth practices in India (Spector et al. 2012), and surgical safety procedures in Moldova (Kwok et al. 2013). Each of these studies showed that the implementation of a checklist led to improvements in quality of services and HCW performance of key activities.

Job aids, guidelines, protocols, and/or charts that provide real-time decision support to providers during the patient encounter have also been widely studied. These tools describe in detail, often in a prescriptive and a standardized manner, the steps to implement a particular standard of care for a disease, set of diseases, or a particular population group during the consultation. For example, multiple studies have shown that using decision support tools and job aids to structure supervisory visits of HCWs leads to faster reactions by supervisors to changes in health worker behaviour (Armstrong Schellenberg et al. 2004, Derenzi et al. 2011). Clinical job aids also lead to improved HCW satisfaction (Sodhi et al. 2011). Wall charts have been used successfully to improve clinical decision-making in multiple settings (Zurovac et al. 2004, Nicholas 1991). Nevertheless, one older study showed an unexpected inverse correlation between the presence of a fever wall chart and correct treatment in children with fever in Central African Republic, 25% of HCWs in this study identified lack of supervision as a barrier to providing correct treatment, rather than the wall chart itself (Rowe et al. 2000).

Technology, in the form of mHealth or eHealth tools targeted at primary health care workers, is also being used more widely in performance improvement and quality assurance for primary HCWs, though it has been little studied. Short message service (SMS) reminders have been shown to improve paediatric malaria case management with artemisinin combination therapy (ACTs) for instance (Zurovac et al. 2011), while a study from India showed that computerized decision support system and job aid for primary health care workers was associated with significant improvement in patient satisfaction and perceived quality of care (Peters et al. 2006).

Simulated patient cases are another tool that has been used to evaluate and improve health worker performance. This usually involves the presentation of a case scenario to a health worker either through a trained human actor or mannequin aimed at mimicking a patient case scenario, or through clinical vignettes presented in a classroom setting. Simulated cases are a potentially important tool for performance improvement because they reduce the potential of likely positive bias that occurs during conspicuous observation by a supervisor or mentor, otherwise known as the ‘Hawthorne Effect’ (Leonard and Masatu 2006). Few studies were found that specifically tested these approaches in the primary care setting in LMICs, but they have been used effectively in Myanmar (Aung et al. 2012). In Benin, direct case observation of IMCI was associated with higher HCW performance than a simulated client method which the authors attributed to the Hawthorne Effect (Rowe et al. 2012). This effect was similar to other studies comparing conspicuous observation to patient interviews and to patient registers (Rowe et al. 2006)

Quality improvement/quality of care

While quality improvement (QI) itself is often described as the underlying objective of interventions to support primary health worker performance, it has also been studied as an intervention or set of interventions unto itself, often combining multiple aspects of the approaches and tools described above. Despite an extensive literature on quality improvement in LMICs, the heterogeneity of intervention types makes it challenging to separate out which interventions were specifically targeted at primary HCW performance, versus larger systems improvement that include HCW performance as one dimension of quality. Quality improvement is often framed as a sequence of steps to assist health workers and managers to identify and solve problems of poor or inadequate health worker performance. Rowe and colleagues (2005) have described the process of QI as similar to that implemented by clinicians in the longitudinal care of chronic illness, including periodic assessments, identification and diagnosis of new problems, treatment of those new problems, follow-up of the problem to see if the recommended treatment had the intended effect, and if not, attempting another therapy and repeating the cycle. They have also highlighted how QI can demonstrate how particular interventions like supervision and job aids – amongst others – fit together into the larger process of managing a health system and the health workers within it. Rowe (2009) has also described the use of integrated continuous surveys and quality management techniques to support scale-up of health interventions, which revolved around a continuously-implemented quality survey similar to DHS (Demographic and Health Surveys) in its implementation (Eriksen et al. 2007). Other than these, however, no additional studies meeting the inclusion criteria that looked specifically at quality improvement as the intervention itself were found. Rather, most studies that made reference to ‘quality improvement’ initiatives either were not specifically targeted at improving HCW performance, or actually tested one or more of the groups of interventions described above and thus were included in those categories.

Coaching and peer-review strategies

Finally, the least known and studied approach to health worker performance and quality improvement is coaching. Only one paper was found, a narrative review from the grey literature, that dealt specifically with coaching as a strategy for performance improvement in the settings we studied (Steinmann and Bosch-Capblanch 2011). Coaching was defined as a one-on-one activity where a coach attempts to induce change in the trainees to boost performance in a particular sector, which shares some specific attributes with mentoring but remains distinctly different in approach. Whereas a coach facilitates learning by enhancing the client’s behavioural change through self-awareness and/or by achieving higher levels of skill performance, the authors argue that mentoring is rather based on a stable, longitudinal, and dyadic relationship where an experienced person fosters a junior protégé using his/her superior professional and social experience, knowledge, and connections to advance the overall development of the mentee. Whereas a coach does not necessarily come from the same technical or professional background as the trainee, but rather focuses on general professional issues, a mentor is generally from higher up on the professional hierarchy within the same field of work, if not the same organization. The authors argue that the evidence for the effectiveness of coaching is strong within the business and private sector, but in general, the available studies are of low quality and often biased, and they argue for the need for higher-quality evidence. They also suggest that what little evidence exists from the health sector suggests that coaching is minimally effective, and they conclude that there is no strong evidence for coaching overall or clear guidance on optimal design.

Discussion

Our objective in this review was to generate a comprehensive description of the current range of support and performance improvement interventions for primary HCWs in LMICs, and to identify generalizable themes and explanations as to how these approaches are designed, and what distinguishes them from one another. During this literature review, we identified several major themes concerning methods and potential uses of performance and quality improvement interventions to support primary HCWs in LMIC settings:

1. Lack of comparative evidence: Despite decades of implementation of performance improvement initiatives for primary HCWs in LMICs, there remains insufficient evidence regarding which specific approaches are most effective for improving quality. We were unable to find any clear comparative trials. As such, those responsible for program design and planning remain without clear guidance on how best to design interventions for post-training HCW support, to what extent they should rely on supervision, mentoring, or other strategies, or what combination of approaches would be beneficial.

2. Ambiguous approaches to supervision: supervision was the approach for which the published literature is currently most extensive. Based on results from the individual studies reviewed here, supervision is likely to be a beneficial intervention for HCW performance and quality improvement in primary care settings in LMICs. What remains unclear, however, is what type(s) of supervision are optimal for performance improvement (e.g. clinical, managerial, etc.) and what is the optimal ‘dose’ and frequency of supervision to achieve and to sustain durable gains in quality and HCW performance. What also remains ambiguous is whether the optimal supervision strategy would differ based on disease or based on covariates such as previous health worker experience, education level, or the local epidemiology and case mix.

3. Repositioning the role of mentoring in the health care delivery system: Mentoring has been inadequately tested and must be evaluated with more breadth and rigour. If, as the literature suggests, mentoring is highly focused on a particular educational approach to clinical quality improvement, in addition to its current role within routine health care administration and public health systems, it could benefit from better integration with the formal health educational system, such as nursing, medical, and health sciences schools. This would be similar to preceptorship or mentorship strategies employed by medical and nursing training programs in high-income settings, where individual mentorship and improvement of clinical skills is the responsibility of degree-granting training programs, post-training certificates and continuing medical education programs. More study is needed in this regard, as well as ways to better describe how clinical mentoring initiatives dovetail with pre-service education and training provided by medical and nursing education institutions and post-training continuing education.

4. Unclear role of coaching: Insufficient evidence exists to draw conclusions on coaching as a performance improvement strategy; because of the dearth of evidence, it is not clear that coaching is indeed an applicable strategy for primary HCWs in LMICs, and thus its inclusion may not be necessary in any future analyses of strategies to improve HCW performance. However, more review of the corporate and business school literature from health care is probably warranted to closely examine health workforce performance improvement initiatives, and whether there are particular operational principles of these programs that could be applied or adapted to primary HCWs working in LMICs, such as team-based approaches and cooperation, communication strategies, and performance reviews.

5. Further investment in tools and technology: Tools and technology are an important area of ongoing and future investment and research to support HCW performance. This is especially true for primary health care workers who work in remote areas, often poorly supported with inadequate infrastructure, connectivity and communication with the rest of the health system, and with urban areas where specialized care and consultation exists. Preliminary evidence is mixed, although singular successes do exist, as described above in multiple settings (Kwok et al. 2013, Eriksen et al. 2007, Mukanga et al. 2011, Spector et al. 2012). We note that despite a growing literature on eHealth interventions, most did not meet inclusion criteria because they did not address HCW performance improvement specifically, or were not targeted at primary health care delivery. More data and research are required to outline exactly which tools should be used as in-service job aids by primary HCWs, and which tools can be used by supervisors and mentors. As well, more research is needed on the design and impact of eHealth interventions on performance and quality improvement metrics. Specifically, the impact of eHealth tools on the fidelity of treatment or decision protocols, HCW practice behaviour and self-perception of care, as well as patient perception of quality, would be important to systematically study and review.

6. Further study on operational approaches: Finally, little is known about the optimal approach to delivering any of the interventions discussed in this review. As noted, the traditional model of delivery of any supervision, mentoring, QI or other performance improvement intervention has typically involved a short site visit. This model is typical because supervisors are often based at higher levels of the system or come from external agencies outside of the public sector due to donor structures and financing. This structure is also a reaction to evidence that suggests that workshops and didactic training sessions that pull HCWs out of practice are both costly and often ineffective. However, one can assume that the short site visit may not be the optimal model for achieving durable gains in quality and performance, and rather alternative models such as week-long intensive visits, longitudinal embedded supervision, or HCW learning exchanges between facilities could be explored. Dedicated study of these alternative models is thus warranted.

The approach to this review had a number of strengths and limitations. The principal strength of this approach was in the comprehensive scope of the review. Most systematic and narrative reviews of performance improvement initiatives have typically addressed one particular intervention (e.g. supervision, quality improvement, etc.) using restrictive definitions. In this review, a comprehensive search strategy was employed to simultaneously identify a wide range of interventions across the five different categories studied. The goal was to provide some comparative evidence across these categories and to establish some clarity as to what distinguishes these approaches from one another in theory and in practice. The limitations of this review were in the restrictions we placed on the setting of the study. Our search terms, while expansive across different categories and approaches to performance and quality improvement, were restrictive with respect to countries studied and the particular level of the health system targeted. Therefore a number of studies may not have been captured in our search that describe performance improvement initiatives at district, regional, or national hospitals in LMICs, or that targeted primary HCWs in high income countries, that probably could have been instructive in our analysis, but were beyond the geographic scope of this review. Additionally, and despite a systematic search, the limitations of the data and differences in outcomes and study design precluded any form of quantitative synthesis within or across categories, given our expansive inclusion criteria and definitions of specific performance improvement approaches.

Conclusion

This review is one of the first of its kind to engage in a comparative examination of a range of approaches to performance and quality improvement programs targeted at primary health care workers in low- and middle-income countries. Primary HCWs in these settings are often the most in need of accompaniment and support due to their geographic isolation and lack of ancillary support, and thus it is important to especially focus quality improvement efforts on these workers and on the front lines of health systems in the developing world. Whether referred to as supervision, mentoring, quality improvement, or otherwise, it is critical that policy-makers and planners are equipped with strong evidence as to which approaches are most effective at achieving significant and sustained gains in front-line health worker performance and quality of care. Further comparative analysis trials are warranted to rigourously examine which specific approaches are more effective than others.