Abstract
Background
A key function of health systems is implementing interventions to improve health, but coverage of essential health interventions remains low in low‐income countries. Implementing interventions can be challenging, particularly if it entails complex changes in clinical routines; in collaborative patterns among different healthcare providers and disciplines; in the behaviour of providers, patients or other stakeholders; or in the organisation of care. Decision‐makers may use a range of strategies to implement health interventions, and these choices should be based on evidence of the strategies' effectiveness.
Objectives
To provide an overview of the available evidence from up‐to‐date systematic reviews about the effects of implementation strategies for health systems in low‐income countries. Secondary objectives include identifying needs and priorities for future evaluations and systematic reviews on alternative implementation strategies and informing refinements of the framework for implementation strategies presented in the overview.
Methods
We searched Health Systems Evidence in November 2010 and PDQ‐Evidence up to December 2016 for systematic reviews. We did not apply any date, language or publication status limitations in the searches. We included well‐conducted systematic reviews of studies that assessed the effects of implementation strategies on professional practice and patient outcomes and that were published after April 2005. We excluded reviews with limitations important enough to compromise the reliability of the review findings. Two overview authors independently screened reviews, extracted data and assessed the certainty of evidence using GRADE. We prepared SUPPORT Summaries for eligible reviews, including key messages, 'Summary of findings' tables (using GRADE to assess the certainty of the evidence) and assessments of the relevance of findings to low‐income countries.
Main results
We identified 7272 systematic reviews and included 39 of them in this overview. An additional four reviews provided supplementary information. Of the 39 reviews, 32 had only minor limitations and 7 had important methodological limitations. Most studies in the reviews were from high‐income countries. There were no studies from low‐income countries in eight reviews.
Implementation strategies addressed in the reviews were grouped into four categories – strategies targeting:
1. healthcare organisations (e.g. strategies to change organisational culture; 1 review);
2. healthcare workers by type of intervention (e.g. printed educational materials; 14 reviews);
3. healthcare workers to address a specific problem (e.g. unnecessary antibiotic prescription; 9 reviews);
4. healthcare recipients (e.g. medication adherence; 15 reviews).
Overall, we found the following interventions to have desirable effects on at least one outcome with moderate‐ or high‐certainty evidence and no moderate‐ or high‐certainty evidence of undesirable effects.
1.Strategies targeted at healthcare workers: educational meetings, nutrition training of health workers, educational outreach, practice facilitation, local opinion leaders, audit and feedback, and tailored interventions.
2.Strategies targeted at healthcare workers for specific types of problems: training healthcare workers to be more patient‐centred in clinical consultations, use of birth kits, strategies such as clinician education and patient education to reduce antibiotic prescribing in ambulatory care settings, and in‐service neonatal emergency care training.
3. Strategies targeted at healthcare recipients: mass media interventions to increase uptake of HIV testing; intensive self‐management and adherence, intensive disease management programmes to improve health literacy; behavioural interventions and mobile phone text messages for adherence to antiretroviral therapy; a one time incentive to start or continue tuberculosis prophylaxis; default reminders for patients being treated for active tuberculosis; use of sectioned polythene bags for adherence to malaria medication; community‐based health education, and reminders and recall strategies to increase vaccination uptake; interventions to increase uptake of cervical screening (invitations, education, counselling, access to health promotion nurse and intensive recruitment); health insurance information and application support.
Authors' conclusions
Reliable systematic reviews have evaluated a wide range of strategies for implementing evidence‐based interventions in low‐income countries. Most of the available evidence is focused on strategies targeted at healthcare workers and healthcare recipients and relates to process‐based outcomes. Evidence of the effects of strategies targeting healthcare organisations is scarce.
Plain language summary
Implementation strategies for health systems in low‐income countries
What is the aim of this overview?
The aim of this Cochrane Overview is to provide a broad summary of what is known about the effects of strategies for implementing interventions to improve health in low‐income countries.
This overview is based on 39 relevant systematic reviews. Each of these reviews searched for studies that evaluated the different types of implementation strategies within the scope of the question addressed by the review. The reviews included a total of 1332 studies.
This overview is one of a series of four Cochrane Overviews that evaluate different health system arrangements.
What was studied in the overview?
A key function of health systems is implementing interventions to improve health. Coverage of essential health interventions remains low in low‐income countries. Decision‐makers may use a range of strategies to implement health interventions, and these choices should be based on evidence of the strategies' effectiveness.
What are the main results of the overview?
The following implementation strategies had desirable effects on at least one outcome with moderate‐ or high‐certainty evidence and no moderate‐ or high‐certainty evidence of undesirable effects.
Strategies targeted at healthcare workers
‐ Educational meetings.
‐ Nutrition training of health workers.
‐ Educational outreach (vs. no intervention).
‐ Practice facilitation.
‐ Local opinion leaders.
‐ Audit and feedback.
‐ Tailored interventions (vs. no intervention).
Strategies targeted at healthcare workers for specific types of problems
‐ Training healthcare workers to be more patient‐centred in clinical consultations.
‐ Use of birth kits.
‐ Clinician education and patient education to reduce antibiotic prescribing in ambulatory care settings.
‐ In‐service neonatal emergency care training.
Strategies targeted at healthcare recipients
‐ Mass media interventions to increase immediate uptake of HIV testing (leaflets and gain‐framed videos).
‐ Intensive self‐management and adherence, intensive disease management to improve health literacy.
‐ Behavioural interventions and mobile phone text messages for adherence to antiretroviral therapy.
‐ A one‐time incentive to start or continue tuberculosis prophylaxis.
‐ Default reminders for patients being treated for active tuberculosis.
‐ Use of sectioned polythene bags for adherence to malaria medication.
‐ Community‐based health education, and reminders and recall strategies for vaccination uptake.
‐ Providing free insecticide‐treated bednets.
‐ Interventions to improve uptake of cervical screening (invitations, education, counselling, access to health promotion nurse, and intensive recruitment).
‐ Health insurance information and application support.
The following implementation strategies had low‐ or very low‐certainty evidence (or no studies available) for all the outcomes that were considered.
Strategies targeted at healthcare organisations
‐ Strategies to improve organisational culture.
Strategies targeted at healthcare workers
‐ Printed educational materials.
‐ Internet‐based learning.
‐ Interprofessional education.
‐ Teaching critical appraisal.
‐ Educational outreach (vs. another intervention).
‐ Pharmacist‐provided services.
‐ Safety checklists for use by medical care teams in acute hospital settings.
‐ Tailored interventions (vs. non‐tailored interventions, and interventions targeted at organisational and individual barriers vs. interventions targeted at individual barriers only).
‐ Interventions to encourage the use of systematic reviews in clinical decision‐making.
Strategies targeted at healthcare workers for specific types of problems
‐ Interventions to improve handwashing.
‐ Interventions to reduce unnecessary caesarean section rates.
‐ Training of traditional birth attendants.
‐ Skilled birth attendance.
‐ Training of traditional healers about STD and HIV medicine.
Strategies targeted at healthcare recipients
‐ Providing information/education for promoting HIV testing (multimedia).
‐ Providing written medicine information.
‐ Single interventions to improve health literacy.
‐ Interventions to improve medication adherence.
‐ Adherence – TB (immediate versus deferred incentives; cash vs. non‐cash incentive; different levels of cash incentives; incentives vs. other interventions).
‐ Adherence – malarial medication (blister packed tablets and capsules compared to tablets and capsules in paper envelopes; tablets in sectioned polythene bags compared to bottled syrup).
‐ Training of healthcare workers, home visits, and monetary incentives to improve immunisation coverage.
‐ Risk factor assessment to improve the uptake of cervical cancer screening.
How up to date is this overview?
The overview authors searched for systematic reviews that had been published up to 17 December 2016.
Background
This is one of four overviews of systematic reviews of strategies for improving health systems in low‐income countries (Ciapponi 2014; Herrera 2014; Wiysonge 2014). The aim is to provide broad overviews of the evidence about the effects of alternative delivery, financial and governance arrangements, and implementation strategies based on systematic reviews. This overview addresses implementation strategies.
The scope of each of the four overviews is summarised below.
Delivery arrangements include changes in who receives care and when, who provides care, the working conditions of those who provide care, coordination of care amongst different providers, where care is provided, the use of information and communication technology to deliver care, and the quality and safety systems in place (Ciapponi 2014).
Financial arrangements include changes in how funds are collected and services purchased, different insurance schemes, and the use of targeted financial incentives or disincentives (Wiysonge 2014).
Governance arrangements include changes in rules or processes that determine authority and accountability for health policies, organisations, commercial products and health professionals, and the involvement of stakeholders in decision‐making (Herrera 2014).
Implementation strategies include interventions designed to bring about changes in healthcare organisations, the behaviour of healthcare professionals or the use of health services by healthcare recipients.
Healthcare systems worldwide are faced with the challenge of improving the quality and safety of care they deliver in order to improve health outcomes. However, in many cases they fail to use the best available evidence to inform decisions about the implementation of specific healthcare interventions, resulting in suboptimal outcomes and inefficiencies (McGlynn 2003). Even when there is consensus around a clear evidence‐informed course of action, its implementation can be difficult, particularly if it requires complex changes in clinical routines, better collaboration among disciplines, changes in patients' behaviour, or changes in the organisation of care (Grol 2007). Effective strategies targeted at multiple levels of the healthcare system are therefore needed to implement improvements in clinical care and the organisation of health services. Outcomes that can potentially be affected by implementation strategies include healthcare recipients' outcomes (health and health behaviours), the quality or utilisation of healthcare services, resource use, healthcare provider outcomes (such as sick leave), and social outcomes (such as poverty or employment) (EPOC 2017). Impacts on these outcomes can be intended and desirable or unintended and undesirable. In addition, the effects of implementation strategies on these outcomes can either reduce or increase inequities.
Health systems in low‐income countries differ from those in high‐income countries in terms of the availability of resources and access to services. Thus, some problems in high‐income countries are not relevant to low‐income countries, such as how best to implement the delivery of expensive technologies that are not available in low‐income countries. Similarly, some problems in low‐income countries are not relevant to high‐income countries, such as how to implement the delivery of services that are already widely available or not needed in high‐income countries. Our focus in this overview was specifically on implementation strategies in low‐income countries. By low‐income countries we mean countries that the World Bank classifies as low or lower‐middle‐income (World Bank 2016). Because upper‐middle‐income countries often have a mixture of health systems with problems similar to both those in low‐income countries and high‐income countries, our focus is relevant to middle‐income countries but excludes consideration of conditions that are not relevant in low‐income countries and are relevant in middle‐income countries.
Description of the interventions
Health system administrators can use a wide range of implementation strategies to improve health systems. Different authors have used a number of approaches to classify these (Abraham 2008; Bero 1998; Dolan 2010; Grimshaw 2001; Grol 1997; Grol 2003; Michie 2011). For this overview we have used a pragmatic approach based on the level of the healthcare system targeted by the intervention: healthcare organisations, healthcare workers, and healthcare recipients (Table 1; EPOC 2017). This approach allows an intuitive matching of the barriers identified for the implementation of specific courses of action and the strategies proposed to address them, as illustrated in Table 2. We also have included reviews of alternative interventions targeted at specific types of problems that are common in low‐income countries, including problems with different types of healthcare worker practice and with the utilisation of health services by healthcare recipients (Table 1; EPOC 2017).
1. Types of implementation strategies.
| Implementation strategy | Definition |
| Targeted at healthcare organisations | |
| Change of organisational culture | Strategies to change organisational culture |
| Continuous quality improvement | Improvement of quality through improvement of organisational processes, use of structured problem‐solving processes incorporating statistical methods and measurement to diagnose problems and monitor progress, use of teams including employees from multiple departments and from different organisational levels, empowering employees to identify quality problems and improvement opportunities and to take action on these, and an explicit focus on 'customers' |
| Targeted at healthcare worker practice – types of interventions | |
| Printed educational materials | Distribution of printed recommendations for clinical care, including clinical practice guidelines. The materials can be delivered personally or through mass mailings. |
| Internet‐based or computerised educational materials | Distribution of electronic recommendations for clinical care, including clinical practice guidelines. The materials are usually delivered through mass mailings and/or published in web pages. |
| Educational meetings | Courses, workshops or other educational meetings |
| Local consensus processes | Formal or informal local consensus processes aimed at promoting implementation of guidelines |
| Educational outreach visits | Personal visits by a trained person to healthcare workers in their own setting |
| Local opinion leaders | The identification and use of local opinion leaders to promote implementation of guidelines |
| Audit and feedback | A summary of healthcare workers' performance over a specified period of time, given to them in a written, electronic or verbal format |
| Patient‐mediated interventions | The use of patients to change professional practice |
| Reminders | Manual or computerised interventions that prompt healthcare workers to perform some action |
| Tailored interventions | Interventions to change practice that are selected based on an assessment of barriers to change |
| Multifaceted interventions | Combinations of two or more strategies to change practice |
| Multidisciplinary teams | Strategies to promote inter or multidisciplinary team work |
| Targeted at healthcare worker practice – types of problems | |
| Communication with patients | Strategies for improving communication with patients |
| Handwashing | Strategies for improving hand hygiene compliance in patient care |
| Obstetrics | Strategies for improving obstetric care |
| Healthcare‐associated infections | Strategies for decreasing healthcare associated infections |
| Patient‐centred approach | Strategies for promoting a patient‐centred approach in clinical consultations |
| Prescribing | Strategies for improving prescribing |
| Prescribing antibiotics | Strategies for improving the use of antibiotics |
| Seriously ill newborn care | Strategies for improving the care of seriously ill newborns |
| Traditional birth attendants | Strategies for improving care delivered by traditional birth attendants |
| Traditional healers | Strategies for improving care delivered by traditional healers |
| Targeted at healthcare recipient use of health services – types of interventions | |
| Information and education provision | Strategies to enable consumers to know about programmes/policies to be implemented (e.g. mass media campaigns) |
| Skill and competencies acquisition | Strategies focused on the acquisition of skills relevant to the use of health services (e.g. lay healthcare workers) |
| Facilitation of communication and decision‐making | Strategies to involve healthcare recipients in decision‐making about programmes/policies to be implemented (e.g. communication skills training for patients) |
| Behaviour change support | Strategies focusing on the adoption or promotion of health and treatment behaviours such as adherence to medicines |
| Targeted at healthcare recipient use of health services – types of problem | |
| Adherence – antiretroviral drugs for human immunodeficiency virus infection/acquired immunodeficiency syndrome (HIV/AIDS) | Strategies for improving adherence to antiretroviral drugs for HIV/AIDS |
| Adherence – medication | Strategies for improving medication adherence |
| Adherence – tuberculosis | Strategies for improving tuberculosis medication adherence |
| Facility‐based deliveries | Strategies for promoting facility‐based deliveries |
| Immunisation coverage | Strategies for improving immunisation coverage |
| Access to healthcare services | Strategies for increasing use of healthcare services by specific populations |
2. Examples of implementation strategies to address different types of barriersa.
| Level | Determinants | Examples of implementation strategies | |
| Healthcare organisations | Inadequate internal communication |
Necessary communication between different levels of the health system may be lacking | Structured referral sheets, involvement of consultants in primary care educational activities |
| Inadequate processes |
Processes for outreach and receiving, referring and transferring patients may not be adequate to implement the option or the types of effective care at which the option is targeted | Redesign of processes to facilitate appropriate and efficient utilisation of services (continuous quality improvement) | |
| Inadequate leadership |
There may be insufficient leadership to implement the option or the types of effective care at which the option is targeted | Identification of effective leaders; engagement of opinion leaders; establishment of leadership systems | |
| Healthcare workers | Knowledge | Healthcare workers may be unaware of the likely impacts of the option or of the types of effective care at which the option is targeted | Dissemination of educational materials |
| Competency | Healthcare workers may not feel competent or may lack competency | Educational meetings or outreach visits | |
| Attitudes | Healthcare workers may not agree that implementing the option or the types of effective care at which the option is targeted is important | Disseminate information regarding the size of the problem, including relevant comparisons; use opinion leaders | |
| Motivation to change | Healthcare workers may not be motivated to change their practices | Dissemination of information that is designed to motivate healthcare workers to change their practice; financial or other incentives; reduce the burden of changing practices | |
| Healthcare recipients | Knowledge | People may be unaware of the likely impacts of the option or of the types of effective care at which the option is targeted | Disseminate information that is reliable and accessible, e.g. using the mass media or community healthcare workers |
| Competency (skill) | People may not have the necessary skills to use the types of effective care at which the option is targeted | Provide training and support | |
| Attitudes | People may not agree that implementing the option or the types of effective care at which the option is targeted is important | Disseminate information regarding the size of the problem, including relevant comparisons | |
| Access to care | People may not have access to the types of effective care at which the option is targeted due to financial constraints or lack of transportation | Reduce financial or physical barriers to care | |
| Motivation to change | People may not be motivated to change their behaviours, for example by seeking types of effective care at which the option is targeted | Dissemination of information that is designed to motivate people to, for example, seek care; use financial or material incentives | |
aAdapted from Grol 2005.
How the intervention might work
Different interventions might work through different mechanisms. There is a plethora of contending theories from the social and behavioural sciences that attempt to explain behaviour change. Many of these have been applied to healthcare professionals and organisations in attempts to explain how different strategies to implement improvements might work (Grol 2007; Michie 2008; Wensing 2005). Michie 2005, for example, identified 33 psychological theories relevant to the implementation of evidence‐based practice. These theories contained 128 constructs (components of the theories) that they categorised into 12 domains. In another review of a broader range of theories relevant to quality improvement, Wensing 2005 included 29 theories that focused on individuals, social context, organisations and structures. There was limited evidence to support the theories, particularly regarding change of professional practice or organisation of care. Like Michie 2005, they found substantial overlap in the factors described by the various theories, which they reduced to a list of 30 factors. More recently, Michie 2011 reviewed 19 frameworks of behaviour change interventions that can be used to characterise interventions and explain how they might work. Based on a synthesis of these frameworks, they developed a new framework called the 'behavioural change wheel' which includes three essential conditions for change and nine intervention functions that address these. The three conditions are capability, opportunity and motivation. The nine intervention functions aimed at addressing deficits in one or more of these conditions are education, persuasion, incentivisation, coercion, training, enablement, modelling, environmental restructuring and restrictions.
Logically, implementation strategies should address determinants of practice, that is, factors that prevent or enable improvements. However, our understanding of how to identify determinants and match appropriate implementation strategies to identified determinants is limited (Baker 2015; Wensing 2011), although there are several frameworks or checklists designed to facilitate this (Flottorp 2013; Krause 2014). Table 2 shows examples of how different implementation strategies might work by addressing different determinants of organisational change, healthcare worker practice, and utilisation of health services by healthcare recipients.
Why it is important to do this overview
Although there are an increasing number of studies and systematic reviews about the effects of different implementation strategies (e.g. Arnold 2005; Baker 2015; Davey 2013; Flodgren 2011; Forsetlund 2009; Giguère 2012; Gould 2010; Ivers 2012; Murthy 2012; O'Brien 2007; Opiyo 2015; Oyo‐Ita 2016; Parmelli 2011; Reeves 2013; Rosenbaum 2011; Sibley 2012; Sorsdahl 2009), much of this literature is not easily accessible to policymakers and other stakeholders making decisions about how to implement improvements in their health systems. Our aim is to facilitate access to this information by providing a broad overview of the evidence from systematic reviews about the effects of alternative implementation strategies in low‐income countries. Such a broad overview can help policymakers and other stakeholders to identify strategies for implementing improvements in their health systems. This overview also can help to identify needs and priorities for evaluating alternative implementation strategies, as well as priorities for systematic reviews of the effects of implementation strategies. The overview can also help to refine the framework outlined in Table 1 for considering alternative implementation strategies.
Changes in health systems are complex. They may be difficult to evaluate, the applicability of the findings of evaluations from one setting to another may be uncertain, and synthesising the findings of evaluations may be difficult. However, the alternative to well‐designed evaluations is poorly designed evaluations, the alternative to systematic reviews is non‐systematic reviews, and the alternative to using the findings of systematic reviews to inform decisions is making decisions without the support of this rigorous evidence. Other types of information, including context‐specific information and judgments (including judgments about the applicability of the findings of systematic reviews in a specific context) are still needed. Nevertheless, this overview can help people make decisions about implementation strategies by summarising the findings of available systematic reviews, including estimates of the effects of implementing specific strategies and the certainty of those estimates. The overview can also help identify important uncertainties identified by those systematic reviews as well as where new or updated systematic reviews are needed. Finally, the overview can help to inform judgments about the relevance of the available evidence in a specific context (Rosenbaum 2011).
Objectives
To provide an overview of the available evidence from up‐to‐date systematic reviews about the effects of implementation strategies for health systems in low‐income countries. Secondary objectives include identifying needs and priorities for future evaluations and systematic reviews on alternative implementation strategies and informing refinements of the framework for implementation strategies presented in the overview (Table 1).
Methods
We used the methods described below in all four overviews of health system arrangements and implementation strategies in low‐income countries (Ciapponi 2014; Herrera 2014; Wiysonge 2014).
Criteria for considering reviews for inclusion
We included reviews that:
assessed the effects of implementation strategies (as defined in Background) for health systems improvement;
had a Methods section with explicit selection criteria;
reported at least one of the following types of outcomes: patient outcomes (health and health behaviours), the quality or utilisation of healthcare services, resource use, healthcare provider outcomes (such as sick leave) or social outcomes (such as poverty or employment);
were relevant to low‐income countries as classified by the World Bank (World Bank 2016); and
were published after April 2005.
Judgments about relevance to low‐income countries are sometimes difficult to make, and we are aware that evidence from high‐income countries is not directly applicable to low‐income countries. We based these judgments on an assessment of the likelihood that the implementation strategies considered in a review address a problem that is important in low‐income countries, would be feasible, and would be of interest to decision‐makers in low‐income countries, regardless of where the included studies took place. So, for example, we excluded strategies that require technology that is not widely available in low‐income countries. At least two of the overview authors made judgments about the relevance to low‐income countries and discussed with the other review authors whenever there was uncertainty.
We excluded reviews that only searched for and included studies from a single high‐income country due to concerns about the wider applicability of the findings of such reviews. However, we included reviews that only included studies from high‐income countries if the interventions were relevant for low‐income countries.
We excluded reviews published before April 2005 as these were highly unlikely to be up‐to‐date. We excluded reviews with methodological limitations important enough to compromise the reliability of the review findings (Appendix 1).
Search methods for identification of reviews
We searched Health Systems Evidence in November 2010 using the following filters:
health system topics = implementation arrangements;
type of synthesis = systematic review or Cochrane Review;
type of question = effectiveness; and
publication date range = 2000 to 2010.
We conducted subsequent searches using PDQ ('pretty darn quick')‐Evidence, which was launched in 2012. We searched PDQ up to 17 December 2016, using the filter 'Systematic Reviews' with no other restrictions. We updated that search, excluding records that were entered into PDQ‐Evidence prior to the date of the previous search.
PDQ‐Evidence is a database of evidence for decisions about health systems, which is derived from the Epistemonikos database of systematic reviews (Rada 2013). It includes systematic reviews, overviews of reviews (including evidence‐based policy briefs) and studies included in systematic reviews. Epistemonikos and PDQ‐Evidence search the following databases with no language or publication status restrictions.
Cochrane Database of Systematic Reviews (CDSR).
PubMed.
Embase.
Database of Abstracts of Reviews of Effectiveness (DARE).
Health Technology Assessment Database.
CINAHL.
LILACS.
PsycINFO.
Evidence for Policy and Practice Information and Co‐ordinating Centre (EPPI‐Centre) Evidence Library.
3ie Systematic Reviews and Policy Briefs.
World Health Organization (WHO) Database.
Campbell Library.
Supporting the Use of Research Evidence (SURE) Guides for Preparing and Using Evidence‐Based Policy Briefs.
European Observatory on Health Systems and Policies.
UK Department for International Development (DFID).
National Institute for Health and Care Excellence (NICE) public health guidelines and systematic reviews.
Guide to Community Preventive Services.
Canadian Agency for Drugs and Technologies in Health (CADTH) Rx for Change.
McMaster Plus KT+.
McMaster Health Forum Evidence Briefs.
Appendix 2 presents the detailed search strategies for PubMed, Embase, LILACS, CINAHL and PsycINFO. We screened all records in the other databases. PDQ staff and volunteers update these searches weekly for Pubmed and monthly for the other databases, screening records continually and adding new reviews to the database daily.
In addition, we screened all of the Cochrane Effective Practice and Organisation of Care (EPOC) Group systematic reviews in Archie (i.e. the Cochrane central server for managing documents) and the reference lists of relevant policy briefs and overviews of reviews.
Data collection and analysis
Selection of reviews
Two of the overview authors (NO and TP) independently screened the titles and abstracts found in PDQ‐Evidence to identify reviews that appeared to meet the inclusion criteria. Two other authors (AO and SL) screened all of the titles and abstracts that could not be confidently included or excluded after the first screening to identify any additional eligible reviews. One of the overview authors screened the reference lists.
One of the overview authors (NO or TP) applied the selection criteria to the full text of potentially eligible reviews and assessed the reliability of reviews that met all of the other selection criteria (Appendix 1). Two other authors (AO or SL) independently checked these judgments.
Data extraction and management
We summarised each included review using the approach developed by the SUPPORT Collaboration (Rosenbaum 2011). We used standardised forms to extract data on the background of the review; the interventions, participants, settings and outcomes; the key findings; and considerations of applicability, equity, economics, and monitoring and evaluation. We assessed the certainty of the evidence for the main comparisons using the GRADE approach (EPOC 2017; Guyatt 2008; Schünemann 2011a; Schünemann 2011b).
Each completed SUPPORT Summary was peer‐reviewed and published on an open access website, where there are details about how the summaries were prepared, including how we assessed the applicability of the findings, impacts on equity, economic considerations, and the need for monitoring and evaluation. We describe our rationale for the criteria that we used for these assessments in the SUPPORT Tools for evidence‐informed health Policymaking (Fretheim 2009; Lavis 2009; Oxman 2009a; Oxman 2009b). As noted there, "a local applicability assessment must be done by individuals with a very good understanding of on‐the‐ground realities and constraints, health system arrangements, and the baseline conditions in the specific setting" (Lavis 2009). In this overview we have made broad assessments of the applicability of findings from studies in high‐income countries to low‐income countries using the criteria described in the SUPPORT Summaries database with input from people with relevant experience and expertise in low‐income countries.
Assessment of methodological quality of included reviews
We assessed the reliability of systematic reviews that met the inclusion criteria using criteria developed by the SUPPORT and SURE collaborations (Appendix 1). Based on these criteria, we categorised each review as having:
only minor limitations;
limitations that were important enough that it would be worthwhile to search for another systematic review and to interpret the results of this review cautiously, if a better review could not be found;
limitations important enough to compromise the reliability of the review findings. We did not include these reviews in the overview.
Data synthesis
We describe the methods used to prepare a SUPPORT Summary of each review in detail on the SUPPORT Summaries website. Briefly, for each included systematic review we prepared a table summarising what the review authors searched for and what they found along with 'Summary of findings' tables for each main comparison, and we assessed the relevance of the findings for low‐income countries. The SUPPORT Summaries include key messages, important background information, a summary of the findings of the review, and structured assessments of the relevance of the review for low‐income countries. We subjected the SUPPORT Summaries to review by the lead author of each review, at least one content area expert, people with practical experience in low‐income settings, and a Cochrane EPOC Group editor (AO or SL). The authors of the SUPPORT Summaries responded to each comment and made appropriate revisions, and the summaries underwent copy‐editing. The editor determined whether the summary authors had adequately addressed the comments and whether the summary was ready for publication on the SUPPORT Summary website.
We organised the review using a modification of the taxonomy for health systems arrangements used by Health Systems Evidence (Lavis 2015). We adjusted the framework iteratively to ensure that we appropriately categorised all of the included reviews and that we included and logically organised all relevant health system arrangements and implementation strategies. We prepared a table listing the included reviews as well as the types of implementation strategies for which we were not able to identify a reliable, up‐to‐date review (Table 3). We also prepared a table of excluded reviews (Table 4), detailing reviews that addressed a question for which another (more up‐to‐date or reliable) review was included, reviews that were published before April 2005 (for which a SUPPORT Summary had previously been prepared), reviews with results that we did not consider transferable to low‐income countries, and reviews with limitations important enough to compromise the reliability of the review findings.
3. Included reviews.
| Implementation strategy | Included reviews |
| Strategies targeted at healthcareorganisations | |
| Organisational culture | The effectiveness of strategies to change organisational culture to improve healthcare performance (Parmelli 2011) |
| Continuous quality improvement | No eligible systematic review identified |
| Strategies targeted at healthcareworkers by type of intervention | |
| Educational materials | Printed educational materials: effects on professional practice and healthcare outcomes (Giguère 2012) |
| Internet based learning | Internet‐based learning in the health professions: a meta‐analysis (Cook 2008) |
| Educational meetings | Continuing education meetings and workshops: effects on professional practice and healthcare outcomes (Forsetlund 2009) |
| Educational meetings | Interprofessional education: effects on professional practice and healthcare outcomes (Reeves 2013) |
| Educational meetings | Teaching critical appraisal skills in healthcare settings (Horsley 2011) |
| Educational meetings | Effectiveness of nutrition training of health workers toward improving caregivers' feeding practices for children aged six months to two years: a systematic review (Sunguya 2013) |
| Local consensus processes | No eligible systematic review identified |
| Educational outreach | Educational outreach visits: effects on professional practice and healthcare outcomes (O'Brien 2007) |
| Educational outreach | Systematic review and meta‐analysis of practice facilitation within primary care settings (Baskerville 2012) |
| Educational outreach | Effectiveness of pharmacist provided services on patient outcomes, health‐service utilisation and costs in low‐ and middle‐income countries (Pande 2013) |
| Local opinion leaders | Local opinion leaders: effects on professional practice and healthcare outcomes (Flodgren 2011) |
| Patient‐mediated interventions | No eligible systematic review identified |
| Audit and feedback | Audit and feedback: effects on professional practice and healthcare outcomes (Ivers 2012) |
| Reminders | Systematic review of safety checklists for use by medical care teams in acute hospital settings—limited evidence of effectiveness (Ko 2011) |
| Tailored interventions | Tailored interventions to overcome identified barriers to change: effects on professional practice and healthcare outcomes (Baker 2015) |
| Multifaceted interventions | Interventions encouraging the use of systematic reviews in clinical decision‐making: a systematic review (Perrier 2011) |
| Strategies targeted at healthcareworkers by type of problem | |
| Communication with patients | Training healthcare providers to be more 'patient‐centred' in clinical consultations (Dwamena 2012) |
| Handwashing | Interventions to improve hand hygiene compliance in patient care (Gould 2010) |
| Obstetrics | Non‐clinical interventions for reducing unnecessary caesarean section (Khunpradit 2011) |
| Obstetrics | Traditional birth attendant training for improving health behaviours and pregnancy outcomes (Sibley 2012) |
| Obstetrics | The effect of providing skilled birth attendance and emergency obstetric care in preventing stillbirths (Yakoob 2011) |
| Obstetrics | The effects of birth kits to on newborn and maternal outcomes (Hundley 2012) |
| Prescribing antibiotics | Interventions to reduce unnecessary antibiotic prescribing: a systematic review and quantitative analysis (Ranji 2008) |
| Seriously ill newborn care | In‐service training for health professionals to improve care of the seriously ill newborn or child in low and middle‐income countries (Opiyo 2015) |
| Quality of care for STD and HIV | Interventions for educating traditional healers about STD and HIV medicine (Sorsdahl 2009) |
| Strategies targeted at healthcarerecipients | |
| Providing information/education | Mass media interventions for promoting HIV testing. (Vidanapathirana 2005) |
| Providing information/education | Written information about individual medicines for consumers (Nicolson 2009) |
| Providing information/education | Health literacy interventions and outcomes: an updated systematic review (Berkman 2011) |
| Adherence ‐ medication |
Interventions for enhancing medication adherence (Haynes 2008) |
| Adherence – ART for HIV/AIDS |
Efficacy of interventions in improving highly active antiretroviral therapy adherence and HIV‐1 RNA viral load. A meta‐analytic review of randomized controlled trials (Simoni 2006) Antiretroviral adherence interventions: translating research findings to the real world clinic (Simoni 2010) |
| Adherence – ART for HIV/AIDS |
Mobile pone text messages for improving adherence to antiretroviral therapy (ART): an individual patient data meta‐analysis of randomised trials (Mbuagbaw 2013) |
| Adherence – TB |
Incentives and enablers to improve adherence in tuberculosis (Lutge 2015) |
| Adherence – TB |
Reminder systems and late patient tracers in the diagnosis and management of tuberculosis (Liu 2014) |
| Adherence – malaria medication |
Effects of unit‐dose packaged treatment on treatment failure and treatment adherence in people with uncomplicated malaria (Orton 2005) |
| Facility‐based deliveries | No eligible systematic review identified |
| Immunisation coverage | Interventions for improving coverage of child immunization in low‐income and middle‐income countries. (Oyo‐Ita 2016) Patient reminder and patient recall systems for improving immunization rates.(Jacobson Vann 2005) |
| Malaria | Strategies to increase the ownership and use of insecticide‐treated bednets to prevent malaria (Augustincic Polec 2015) |
| Access to healthcare services | Interventions targeted at women to encourage the uptake of cervical screening (Everett 2011) |
| Access to healthcare services | Outreach strategies for increasing health insurance coverage for vulnerable populations (Jia 2014) |
ART: antiretroviral therapy.
4. Excluded reviews.
| Review ID | Title | Reasons for exclusion |
| Arditi 2012 | Computer‐generated reminders delivered on paper to healthcare professionals; effects on professional practice and healthcare outcomes | Limited relevance to low‐income countries |
| Beilby 1997 | Trials of providing costing information to general practitioners: a systematic review | Search out‐of‐date |
| Bordley 2000 | The effect of audit and feedback on immunization delivery: a systematic review | Search out‐of‐date |
| Chaillet 2006 | Evidence‐based strategies for implementing guidelines in obstetrics: a systematic review | Major methodological limitations |
| Fudickar 2012 | The effect of the WHO surgical safety checklist on complication rate and communication | Major limitations |
| Grilli 2002 | Mass media interventions: effects on health services utilisation | Search out‐of‐date |
| Grimshaw 2004 | Effectiveness and efficiency of guideline dissemination and implementation strategies | Search out‐of‐date |
| Horvath 2012 | Mobile phone text messaging for promoting adherence to antiretroviral therapy in patients with HIV infection | Addressed by Mbuagbaw 2013 |
| Hulscher 2001 | Interventions to implement prevention in primary care | Search out‐of‐date |
| Ioannidis 2001 | Evidence on interventions to reduce medical errors: an overview and recommendations for future research | Search out‐of‐date |
| Jayaraman 2010 | Advanced trauma life support training for ambulance crews | Addressed by Forsetlund 2009 |
| Jayaraman 2009 | Advanced trauma life support training for hospital staff | Addressed by Forsetlund 2009 |
| Kendrick 2000 | The effect of home visiting programmes on uptake of childhood immunization: a systematic review and meta‐analysis | Search out‐of‐date |
| Lam‐Antoniades 2009 | Electronic continuing education in the health professions: an update on evidence from RCTs | Addressed by Cook 2008 |
| Lee 2009 | Linking families and facilities for care at birth: What works to avert intrapartum‐related deaths? | Important limitations |
| Legare 2010 | Interventions for improving the adoption of shared decision making by healthcare professionals | Limited relevance to low‐income countries. |
| Minkman 2007 | Performance improvement based on integrated quality management models: what evidence do we have? A systematic literature review | Limited relevance to low‐income countries |
| Naikoba 2001 | The effectiveness of interventions aimed at increasing handwashing in healthcare workers‐ a systematic review | Search out‐of‐date |
| Nglazi 2013 | Mobile phone text messaging for promoting adherence to anti‐tuberculosis treatment: a systematic review | Addressed by Liu 2014 |
| Pattinson 2005 | Critical incident audit and feedback to improve perinatal and maternal mortality and morbidity | Addressed by Ivers 2012 |
| Pearce 2012 | The most effective way of delivering a train‐the‐trainers program: a systematic review | Addressed by Forsetlund 2009 |
| Rowe 2002 | Improving communication between health professionals and women in maternity care: a structured review | Search out‐of‐date |
| Rueda 2006 | Patient support and education for promoting adherence to highly active antiretroviral therapy for HIV/AIDS | Major methodological limitations |
| Safdar 2008 | Educational interventions for prevention of healthcare‐associated infection: a systematic review | Addressed by Forsetlund 2009 |
| Shea 2009 | Increasing the demand for childhood vaccination in developing countries: a systematic review | Addressed by Oyo‐Ita 2016 |
| Shojania 2009 | The effects of on‐screen, point of care computer reminders on processes and outcomes of care | Limited relevance to low‐income countries |
| Smith 2009 | Provider practice and user behavior interventions to improve prompt and effective treatment of malaria: do we know what works? | Major limitations |
| Smits 2002 | Problem based learning in continuing medical education: a review of controlled evaluation studies | Search out‐of‐date |
| Sowden 2000 | Mass media interventions for preventing smoking in young people | Search out‐of‐date |
| Turner 2005 | What is the evidence for effectiveness of WHO guidelines for the care of children in hospitals in developing countries? | Major methodological limitations |
| Wafula 2010 | Are interventions for improving the quality of services provided by specialized drug shops effective in sub‐Saharan Africa? A systematic review | Addressed by Forsetlund 2009 and O'Brien 2007 |
| Zedler 2011 | Does packaging with a calendar feature improve adherence to self‐administered medication for long‐term use? A systematic review | Addressed by Haynes 2008 |
We described the characteristics of the included reviews in Appendix 3 that includes the date of the last search, any important limitations, what the review authors searched for and what they found. We summarise our detailed assessments of the reliability of the included reviews in a separate table (Table 5) showing whether individual reviews met each criterion in Appendix 1.
5. Reliability of included reviews.
| Review | A. Identification, selection and critical appraisal of studiesa | B. Analysisb | C. Overallc | |||||||||||
| 1. Selection criteria | 2. Search | 3. Up‐to‐date | 4. Study selection | 5. Risk of bias | 6. Overall | 1. Study characteristics | 2. Analytic methods | 3. Heterogeneity | 4. Appropriate synthesis | 5. Exploratory factors | 6. Overall | 1. Other considerations | 2. Reliability of the review | |
| Baker 2015 | + | + | + | + | + | + | + | + | + | + | + | + | + | + |
| Baskerville 2012 | + | ? | + | ? | + | ? | ? | + | + | + | + | + | + | ‐ |
| Berkman 2011 | + | ? | + | + | + | + | + | + | + | + | + | + | + | + |
| Cook 2008 | + | ? | + | + | + | + | + | + | + | + | + | + | + | + |
| Dwamena 2012 | + | + | + | + | + | + | + | + | + | + | + | + | + | + |
| Everett 2011 | + | + | + | + | + | + | + | + | + | + | + | + | + | + |
| Flodgren 2011 | + | ? | + | + | + | + | + | + | + | + | + | + | + | + |
| Forsetlund 2009 | + | ? | + | + | + | + | + | + | + | + | + | + | + | + |
| Giguère 2012 | + | ? | + | + | + | + | + | + | + | + | + | + | + | + |
| Gould 2010 | + | + | + | + | + | + | + | + | + | + | ? | + | + | + |
| Haynes 2008 | + | + | + | + | ? | + | + | + | ? | ? | NA | ‐ | + | ‐ |
| Horsley 2011 | + | + | + | + | + | + | + | + | + | + | + | + | + | + |
| Hundley 2012 | + | + | + | ? | ? | + | + | + | ? | + | ? | + | + | + |
| Ivers 2012 | + | ? | + | + | + | + | + | + | + | + | + | + | + | + |
| Jacobson Vann 2005 | + | ? | + | ? | + | ‐ | + | + | + | + | + | + | + | ‐ |
| Jia 2014 | + | ? | + | + | + | + | + | + | + | + | NA | + | + | + |
| Khunpradit 2011 | + | + | + | + | + | + | + | + | + | + | + | + | + | + |
| Ko 2011 | + | ? | + | ? | + | ‐ | + | ? | + | ‐ | + | ‐ | + | ‐ |
| Liu 2014 | + | + | + | + | + | + | + | + | + | + | + | + | + | + |
| Lutge 2015 | + | + | + | + | + | + | + | + | + | + | + | + | + | + |
| Mbuagbaw 2013 | + | ? | + | ? | ? | + | ? | + | + | + | + | + | + | ‐ |
| Nicolson 2009 | + | + | ? | + | ? | ‐ | + | + | ? | ? | ‐ | ‐ | + | ‐ |
| O'Brien 2007 | + | ? | + | + | + | + | + | + | + | + | + | + | + | + |
| Opiyo 2015 | + | + | + | + | + | + | + | + | + | + | + | + | + | + |
| Orton 2005 | + | + | + | + | + | + | + | + | + | + | NA | + | + | + |
| Oyo‐Ita 2016 | + | ? | + | + | ? | + | + | + | + | + | NA | + | + | + |
| Pande 2013 | + | ? | + | + | + | + | + | + | + | ? | ? | + | + | + |
| Parmelli 2011d | + | + | + | + | + | + | NA | NA | NA | NA | NA | NA | + | + |
| Perrier 2011 | + | + | + | + | + | + | + | + | ? | ? | + | + | + | + |
| Augustincic Polec 2015 | + | + | + | + | + | + | + | + | + | + | + | + | + | |
| Ranji 2008 | + | ? | + | ? | + | ‐ | + | + | ? | + | ‐ | ‐ | + | ‐ |
| Reeves 2013 | + | + | + | + | + | + | + | + | + | + | + | + | + | + |
| Sibley 2012 | + | ? | + | + | + | + | + | + | + | + | + | + | + | + |
| Simoni 2006 | + | ? | + | ? | ? | ‐ | + | + | + | + | + | + | + | + |
| Simoni 2010 | + | ? | + | ? | ? | ‐ | + | + | + | + | + | + | + | + |
| Sorsdahl 2009 | + | + | + | + | + | + | + | + | + | + | + | + | + | + |
| Sunguya 2013 | + | + | + | + | + | + | + | + | + | + | + | + | + | + |
| Vidanapathirana 2005 | + | + | + | + | + | + | + | + | + | + | + | + | + | + |
| Yakoob 2011 | + | ? | + | + | + | + | + | + | + | + | + | + | + | + |
aA. Identification, selection and critical appraisal of studies 1. Selection criteria: were the criteria used for deciding which studies to include in the review reported? (+ yes; ? can't tell/partially; − no) 2. Search: was the search for evidence reasonably comprehensive? (+ yes; ? can't tell/partially; − no) 3. Up‐to‐date: is the review reasonably up‐to‐date? (+ yes; ? can't tell/partially; − no) 4. Study selection: was bias in the selection of articles avoided? (+ yes; ? can't tell/partially; − no) 5. Risk of bias: did the authors use appropriate criteria to assess the risk for bias in analysing the studies that are included? (+ yes; ? can't tell/partially; − no) 6. Overall: how would you rate the methods used to identify, include and critically appraise studies? (+ only minor limitations, − important limitations)
bB. Analysis 1. Study characteristics: were the characteristics and results of the included studies reliably reported? (+ yes; ? can't tell/partially; − no; NA: not applicable, i.e. no studies or data) 2. Analytic methods: were the methods used by the review authors to analyse the findings of the included studies reported? (+ yes; ? can't tell/partially; − no; NA: not applicable, i.e. no studies or data) 3. Heterogeneity: did the review describe the extent of heterogeneity? (+ yes; ? can't tell/partially; − no; NA: not applicable, i.e. no studies or data) 4. Appropriate synthesis: were the findings of the relevant studies combined (or not combined) appropriately relative to the primary question the review addresses and the available data? (+ yes; ? can't tell/partially; − no; NA: not applicable, i.e. no studies or data) 5. Exploratory factors: did the review examine the extent to which specific factors might explain differences in the results of the included studies? (+ yes; ? can't tell/partially; − no; NA: not applicable, i.e. no studies or data) 6. Overall: how would you rate the methods used to analyse the findings relative to the primary question addressed in the review? (+ only minor limitations, − important limitations; NA: not applicable, i.e. no studies or data)
cC. Overall 1. Other considerations: are there any other aspects of the review not mentioned before which lead you to question the results? (+ yes; ? can't tell/partially; − no) 2. Reliability of the review: based on the above assessments of the methods how would you rate the reliability of the review? (+ only minor limitations, − important limitations)
dThis was an empty review.
Our structured synthesis of the findings of the overview was based on two tables. We summarised the main findings of each review in a table that included the key messages from each SUPPORT Summary (Table 6). In a second table (Table 7), we reported the direction of the results and the certainty of the evidence for each of the following type of outcomes: health and other patient outcomes; access, coverage or utilisation; quality of care; resource use; social outcomes; impacts on equity; healthcare provider outcomes; adverse effects (not captured by undesirable effects on any of the preceding types of outcomes), and any other important outcome (that did not fit into any of the preceding types of outcomes) (EPOC 2017). We categorised the direction of results as: a desirable effect, little or no effect, an uncertain effect (very low certainty evidence), no included studies, an undesirable effect, not reported (i.e. not specified as a type of outcome that the review authors considered by ), or not relevant (i.e. no plausible mechanism by which the type of health system arrangement could affect the type of outcomes).
6. Key messages of included reviews.
| Implementation strategy | Key messages |
| Strategies targeted at healthcareorganisations | |
|
Organisational culture Effectiveness of strategies to change organisational culture to improve healthcare performance (Parmelli 2011) |
➡ Strategies to improve organisational culture include:
➡ It is uncertain whether any of these strategies to improve organisational culture are effective in changing healthcare performance, as no studies met the review's inclusion criteria. ➡ The implementation of strategies to improve organisational culture should include well‐designed evaluations. |
| Strategies targeted at healthcareworkers by type of intervention | |
|
Educational materials Printed educational materials: effects on professional practice and healthcare outcomes (Giguère 2012) |
➡ When used alone, printed educational materials may slightly improve practice outcomes among healthcare providers, compared to no intervention. ➡ The effects of printed educational materials on patient outcomes are uncertain. ➡ Of the 45 studies included in the review, 44 were from high‐income countries. Rigorous studies from low‐income countries are needed to assess the impacts of printed educational materials on professional practice in these settings. |
|
Internet‐based learning Internet‐based learning in the health professions: a meta‐analysis (Cook 2008) |
➡ Internet‐based learning methods compared with no intervention may improve health professionals' knowledge, but it is uncertain whether they improve skills and behaviours of health professionals, or if they lead to beneficial effects on patients.
➡ It is uncertain whether Internet‐based learning by health professions improves knowledge or other outcomes when compared to other forms of teaching and learning. |
|
Educational meetings Continuing education meetings and workshops: effects on professional practice and healthcare outcomes (Forsetlund 2009) |
➡ Educational meetings alone or combined with other interventions probably improve professional practice and healthcare outcomes for patients. ➡ Educational meetings may be more effective with higher attendance at the educational meetings; mixed interactive plus didactic educational meetings may be more effective compared to only interactive or only didactic educational meetings. ➡ Educational meetings may not be effective for complex behaviours, and they may be less effective for less serious outcomes. |
|
Educational meetings Interprofessional education: effects on professional practice and healthcare outcomes (Reeves 2013) |
➡ Interprofessional education may lead to improved outcomes for patients and greater patient satisfaction. ➡ Interprofessional education may improve professionals' adherence to guidelines or standards. ➡ It is uncertain whether interprofessional education improves collaborative behaviours among professionals, the competencies of professionals to work together in delivering care or clinical processes. ➡ None of the included studies were conducted in low‐income countries. The extent to which these findings are applicable to these settings is uncertain. |
|
Educational meetings Teaching critical appraisal skills in healthcare settings (Horsley 2011) |
➡ Teaching critical appraisal skills to health professionals may improve their knowledge of how to critically appraise research papers. ➡ It is uncertain whether teaching critical appraisal skills to health professionals leads to actual changes in their critical appraisal skills. ➡ We did not find any studies that evaluated the impact of teaching critical appraisal skills on processes of care or patient outcomes. ➡ None of the included studies were from low‐income countries. |
|
Educational meetings Effectiveness of nutrition training of health workers toward improving caregivers' feeding practices for children aged six months to two years: a systematic review (Sunguya 2013) |
➡ Nutrition training of health workers increases daily energy intake of children aged 6 months to 2 years. ➡ Nutrition training of health workers increases feeding frequency of children under 2 years of age. ➡ Children whose caregivers are counselled by trained health workers have a higher consumption of targeted food items compared to their counterparts. |
|
Educational outreach Educational outreach visits: effects on professional practice and healthcare outcomes (O'Brien 2007) |
➡ The quality of care delivered to patients:
➡ For prescribing, the effects are relatively consistent and small, but potentially important. ➡ For other types of professional performance, the effects vary more widely. ➡ Educational outreach visits may not be effective in low‐income countries if resources are not available to provide clinical and managerial support. |
|
Educational outreach Practice facilitation within primary care settings (Baskerville 2012) |
➡ The use of practice facilitation as a multifaceted approach probably improves the adoption of evidence‐based guidelines in primary care settings. ➡ All studies of the effects of practice facilitation took place in high‐income countries. Further research is needed to determine the effectiveness and cost implications of practice facilitation in low‐income countries. |
|
Educational outreach Pharmacist‐provided non‐dispensing services: effects on patient outcomes, health service utilisation and costs (Pande 2013) |
➡ The provision of additional services by pharmacists targeted at patients, such as patient health education and follow‐up, may lead to:
➡ The provision of additional services by pharmacists targeted at healthcare professionals, such as educational outreach visits, may improve patient outcomes. ➡ The applicability of the findings to low‐income countries may be limited by pharmacist numbers, patients and physicians' attitudes to pharmacists, pharmacists' training, and laws governing pharmaceutical practice. |
|
Local opinion leaders Local opinion leaders: effects on professional practice and healthcare outcomes (Flodgren 2011) |
➡ Opinion leaders probably influence the behaviour of healthcare professionals. ➡ Patient outcome data were not reported by studies included in the review. ➡ Most of the studies included in this review took place in high‐income countries. ➡ Rigorous studies from low‐income countries are needed to fully understand the applicability of these findings to low‐income country healthcare settings. |
|
Audit and feedback Audit and feedback: effects on professional practice and healthcare outcomes (Ivers 2012) |
➡ Interventions that include audit and feedback (alone or as a core component of a multifaceted intervention) probably improve professionals' adherence to desired practice compared with usual care. ➡ Audit and feedback may be more effective when baseline professional performance is low; when the source of the feedback is a supervisor or senior colleague; when the feedback is delivered at least monthly; when it is provided both verbally and in a written format; and when it includes both explicit targets and an action plan. ➡ The effects on patient outcomes of interventions that include audit and feedback may range from little if any effect to some improvement, compared with usual care. ➡ We found few randomised trials of audit and feedback in low‐income countries. Audit and feedback is difficult to implement if reliable, routinely collected data are not readily available. |
|
Reminders Safety checklists for use by medical care teams in acute hospital settings (Ko 2011) |
➡ Surgical safety checklists may improve death rates and major complications within 30 days after surgery. ➡ It is uncertain whether safety checklists improve adherence to guidelines or patient safety in intensive care units, emergency departments or acute care settings. ➡ Randomised trials are needed to inform decisions about the use of safety checklists in acute hospital settings. |
|
Tailored interventions Tailored interventions to overcome identified barriers to change: effects on professional practice and healthcare outcomes (Baker 2015) |
➡ Interventions tailored to address identified barriers are probably more likely to improve professional practice than no intervention or the dissemination of guidelines alone. ➡ It is uncertain whether tailored interventions are more likely to improve professional practice than non‐tailored interventions. ➡ Little is known about how best to identify barriers to improving professional practice and how to tailor interventions to address these barriers. |
|
Multifaceted interventions Interventions encouraging the use of systematic reviews in clinical decision‐making (Perrier 2011) |
➡ It is uncertain whether targeted multifaceted or single interventions (such as training) improve informed decision‐making by practitioners. ➡ Multifaceted interventions may improve awareness and use of evidence‐based resources, such as searching for systematic reviews using online libraries. ➡ None of the included studies took place in a low‐income country. |
| Strategies targeted at healthcareworkers by type of problem | |
|
Communication with patients Training healthcare providers to be more 'patient‐centred' in clinical consultations (Dwamena 2012) |
➡ Patient‐centred training for providers (with or without co‐interventions):
➡ This review identified no studies from low‐ and middle‐income countries. |
|
Handwashing Interventions to improve hand hygiene compliance in patient care (Gould 2010) |
➡ Educational interventions may increase hand hygiene guidance compliance. ➡ Multifaceted marketing campaigns may increase the use of hand hygiene products. ➡ It is uncertain whether marketing campaigns decrease healthcare‐associated infections. ➡ Rigorous evaluation of interventions to increase hand hygiene compliance are needed. |
|
Obstetrics Non‐clinical interventions for reducing unnecessary caesarean section (Khunpradit 2011) |
➡ Interventions that may reduce unnecessary caesarean sections include: nurse‐led relaxation training, birth preparation classes, education of local opinion leaders, and review of each delivery that does not meet guideline criteria plus a 24‐hour in‐house coverage system. ➡ A mandatory second opinion and post‐caesarean section presentation of cases may reduce repeat caesarean section rates. ➡ Interventions that may have little or no overall effect on caesarean section rates include: a prenatal education support programme for vaginal birth after caesarean sections, intensive group therapy for women with fear of childbirth, decision aids, a mandatory second opinion and post‐caesarean section presentation of cases, audit and feedback, childbirth education classes for primary care nurses, changes in fees for vaginal deliveries or caesarean sections, and mandatory peer review. ➡ To the extent that reducing unnecessary caesarean sections is a priority, interventions to achieve this goal should be evaluated in randomised trials or interrupted time series studies and the cost‐effectiveness of effective interventions should be evaluated. |
|
Obstetrics Traditional birth attendant training for improving health behaviours and pregnancy outcomes (Sibley 2012) |
➡ Initial training of TBAs may:
➡ Additional TBA training may:
➡ Most of the included studies took place in resource‐limited settings in low‐income countries. |
|
Obstetrics The effect of providing skilled birth attendance and emergency obstetric care in preventing stillbirths (Yakoob 2011) |
➡ Skilled birth attendance may reduce stillbirths and perinatal mortality. ➡ It is uncertain what the effects of alternative ways of providing emergency obstetric care are on stillbirths or perinatal mortality. |
|
Obstetrics Birth kits (Hundley 2012) |
➡ The use of birth kits (together with education and/or a topical antimicrobial) compared with no intervention:
➡ Most of the included studies took place in low‐income countries. |
|
Prescribing antibiotics Interventions to reduce unnecessary antibiotic prescribing: a systematic review and quantitative analysis (Ranji 2008) |
➡ Strategies such as clinician education and patient education alone or combined with audit and feedback probably reduce antibiotic prescribing in ambulatory care settings. ➡ The effects of the interventions on the proportion of patients treated with appropriate antibiotics and on clinical outcomes were not reported. ➡ Most of the studies took place in high‐income countries. |
|
Seriously ill newborn care In‐service training for health professionals to improve care of the seriously ill newborn or child in low‐ and middle‐income countries (Opiyo 2015) |
➡ In‐service neonatal emergency care training of health professionals probably:
➡ In‐service neonatal emergency care training of health professionals may reduce mortality in newborns requiring resuscitation. ➡ We found no studies that evaluated the effects of in‐service neonatal emergency care training on long‐term outcomes or the effects of in‐service emergency care training for older children. |
|
Quality of care for STD and HIV Interventions for educating traditional healers about STD and HIV medicine (Sorsdahl 2009) |
➡ Training traditional healers may increase their general knowledge about HIV/AIDS, as well as their knowledge about HIV/AIDS signs, symptoms and prevention. ➡ Training traditional healers may improve their HIV/STD patient management practices. ➡ Training traditional healers may lead to little or no difference in the incidence of HIV/AIDS risk behaviours among healers or in their referral practices. ➡ Traditional healers who have received training may refer patients to allopathic health more frequently if their traditional treatment fails. |
| Strategies targeted at healthcarerecipients | |
|
Providing information/education Mass media interventions for promoting HIV testing (Vidanapathirana 2005) |
➡ Mass media interventions lead to an increase in immediate uptake of HIV testing. ➡ These initial increases in uptake of HIV testing following mass media interventions may not be sustained in the long term. ➡ Mass media interventions may lead to an increase in the number of infected people diagnosed through voluntary counselling and testing. ➡ These findings come from studies conducted in high‐income non‐endemic countries. Factors that may affect the transferability of these findings to low‐income countries include access to television, radio, and print media; availability of (and user‐fees for) HIV voluntary counselling and testing; the level of stigma and discrimination against people living with HIV in the community; and the maturity of the HIV epidemic. |
|
Providing information/education Written information about individual medicines for consumers (Nicolson 2009) |
➡ Written medicine information may slightly improve knowledge and attitudes about medicines compared with no written information. ➡ Written medicine information may lead to little or no difference in adherence to instructions compared with no written information. ➡ The effect of written medicine information on health outcomes is uncertain. The review did not find studies that evaluated this. ➡ Written medicine information delivered in an 'easy‐to‐read' format compared with a standard manufacturer's format may lead to little or no difference in knowledge about and behaviours related to medicines, but it may slightly improve attitudes towards the information presented. ➡ Written numerical information about the risks of medicines may slightly improve knowledge and attitudes about medicines compared with the same information as text. ➡ The effects of written medicine information are mediated by the ability to read the information presented. Low literacy levels in low‐income countries could make these findings less applicable. |
|
Providing information/education Health literacy interventions (Berkman 2011) |
➡ Some single strategies may improve comprehension for people with low health literacy, such as presenting essential information by itself, using the same denominators to present baseline risk and treatment benefit information, and adding icon arrays to numerical presentations of treatment benefit information. ➡ It is uncertain whether single strategies improve the use of healthcare services, health outcomes, resource use or disparities in the use of healthcare services. ➡ Some mixed strategies such as intensive self‐management and adherence interventions probably improve the use of healthcare across health literacy levels. ➡ Some mixed strategies such as intensive disease management programmes probably reduce disease prevalence across health literacy levels. ➡ It is uncertain whether mixed strategies improve resource use or disparities in the use of healthcare services. ➡ Only one of the included studies took place in a low‐income country. |
|
Adherence – medication Interventions for enhancing medication adherence (Haynes 2008) |
➡ It is uncertain whether interventions to increase adherence to short‐term treatments improve adherence or patient outcomes. ➡ Interventions aimed at increasing adherence to long‐term treatments may improve adherence, but it is uncertain whether they improve patient outcomes. ➡ Most of the included studies assessed complex interventions with multiple components in high‐income countries. Adherence interventions may be difficult to implement in low‐income countries where health systems face greater challenges. |
|
Adherence – ART for HIV/AIDS Efficacy of interventions in improving highly active antiretroviral therapy adherence and HIV‐1 RNA viral load (Simoni 2006; Simoni 2010) |
➡ Behavioural interventions probably lead to slightly better adherence to HAART. ➡ Behavioural interventions may slightly improve the number of patients with undetectable viral load (a laboratory measure of successful HAART). ➡ We did not find any studies measuring patient outcomes such as morbidity and mortality. ➡ Only one included study took place in a low‐income country. |
|
Adherence – ART for HIV/AIDS Mobile phone text messages for improving adherence to antiretroviral therapy (ART) (Mbuagbaw 2013) |
➡ Mobile phone text messages compared to standard care improve adherence to ART for up to 12 months. ➡ Mobile phone text messages compared to standard care may lead to little or no difference in mortality or loss to follow‐up after up to 12 months. ➡ Weekly text messages probably improve adherence compared to daily text messages, and interactive text messages probably improve adherence compared to non‐interactive text messages. ➡ All studies took place in low‐income countries in Africa. |
|
Adherence – TB Material incentives and enablers in the management of tuberculosis (Lutge 2015) |
➡ Sustained material incentives may lead to little or no difference in cure or completion of treatment for active TB, compared to no incentive. ➡ It is not clear if sustained material incentives improve completion of TB prophylaxis, compared to no incentive, because findings varied across studies. ➡ A single, once only incentive may increase the number of people who return to a clinic for reading of their tuberculin skin test, compared to no incentive. ➡ A single, once only incentive probably increases the number of people who return to a clinic to start or continue TB prophylaxis, compared to no incentive. ➡ Compared to a non‐cash incentive, cash incentives may slightly increase the number of people who return to a clinic for reading of their tuberculin skin test and may increase the number of people who complete TB prophylaxis. ➡ Compared to counselling or education interventions, material incentives may increase the number of people who return to a clinic for reading of their tuberculin skin test. ➡ Compared to counselling or education interventions, material incentives may lead to little or no difference in the number of people who return to a clinic to start or continue TB prophylaxis or in the number of people who complete TB prophylaxis. ➡ Higher cash incentives may slightly improve the number of people who return to a clinic for reading of their tuberculin skin test, compared to lower cash incentives. |
|
Adherence – TB Reminder systems to improve patient adherence to tuberculosis clinic appointments for diagnosis and treatment (Liu 2014) |
➡ For patients being treated for active TB:
➡ For people on TB prophylaxis, pre‐appointment reminders may increase clinic attendance. ➡ For people undergoing screening for TB, pre‐appointment reminders may have little or no effect on the number of people who return to clinic for the result of their skin test. ➡ Due to the low‐certainty evidence, more well‐designed trials are needed to establish whether reminder systems are effective in different settings, and the best way of delivering reminders, especially in low‐income countries. |
|
Adherence – malaria medication Unit‐dose packaged drugs for treating malaria (Orton 2005) |
➡ No studies measured treatment failure on or by day 28 after initiation of treatment, which was the primary outcome in this review. ➡ The use of blister packs compared to paper envelopes for antimalarial drugs may improve adherence to treatment and may slightly improve clinical outcomes. No studies reported adverse events. ➡ The use of sectioned polythene bags compared with bottled syrup may improve adherence to treatment in children under 5 years who have malaria, but may increase vomiting. It is uncertain whether there is a difference in clinical outcomes. ➡ The use of sectioned polythene bags compared to paper envelopes for antimalarial drugs probably improves adherence to treatment and may slightly improve clinical outcomes in children over 7 years and adults with malaria. Their use may lead to little if any difference in adverse events. ➡ It is uncertain whether the use of sectioned compared to unsectioned polythene bags leads to a difference in adherence, clinical outcomes or adverse events. |
|
Immunisation coverage Interventions for improving coverage of child immunization in low‐income and middle‐income countries (Oyo‐Ita 2016) Patient reminder and patient recall systems for improving immunization rates (Jacobson Vann 2005) |
➡ Community‐based health education probably improves coverage of three doses of diphtheria‐tetanus‐pertussis vaccine (DTP3). However, the impacts of facility‐based health education on coverage of DPT3 may vary from little or no effect to potentially important benefits. ➡ Health education combined with reminders may increase DTP3 coverage. ➡ Training vaccination managers to provide supportive supervision for healthcare provider may have little or no effect on coverage of DTP, oral polio vaccine (OPV) and hepatitis B virus (HBV) vaccine. ➡ Integrating vaccination with other healthcare services may increase DTP3 and measles vaccine coverage and may have little or no effect on BCG coverage. ➡ Household monetary incentives may have little or no effect on achieving full vaccination coverage. ➡ Home visits may improve OPV3 and measles coverage. ➡ Reminders and recall strategies probably increase routine childhood vaccination uptake. |
|
Malaria Strategies to increase the ownership and use of insecticide‐treated bednets to prevent malaria (Augustincic Polec 2015) |
➡ Providing free insecticide‐treated bednets compared to providing subsidised or full market price bednets probably increases the number of pregnant women, adults and children who possess insecticide‐treated bednets, but probably leads to little or no difference in appropriate use of bednets. ➡ Education about appropriate use of insecticide‐treated bednets may increase the number of adults and children under 5 sleeping under bednets. ➡ Providing incentives to encourage the use of insecticide‐treated bednets may lead to little or no difference in use. ➡ The included studies took place in rural communities in Africa, India and Iran. |
|
Access to healthcare services Interventions targeted at women to encourage the uptake of cervical screening (Everett 2011) |
➡ Education, counselling, access to health promotion nurse, invitations to attend cervical screening programmes and intensive recruitment probably increase the uptake of cervical screening. ➡ Enhanced risk factor assessment may lead to little or no difference in the uptake of screening. ➡ Photo‐comic book and message framing probably lead to little or no difference on the uptake of screening. ➡ Most of the included studies took place in high‐income countries. |
|
Access to healthcare services Outreach strategies for increasing health insurance coverage for vulnerable populations (Jia 2014) |
➡ Health insurance information and application support probably:
➡ Handing out application forms in the emergency department of hospitals probably increases the enrolment of children in health insurance schemes. ➡ Only 2 studies conducted in high‐income countries were included in the review.
|
ART: antiretroviral therapy; HAART: highly active antiretroviral therapy; TB: tuberculosis.
7. Intervention‐outcome matrix.
| Implementation strategy | Direction of effects and certainty of the evidencea | ||||||||
| Patient outcomes | Access, coverage, utilisation | Quality of care | Resource use | Social outcomes | Impacts on equity | Healthcareprovider outcomes | Adverse effects b | Other | |
| Strategies targeted at healthcare organisations | |||||||||
|
Organisational culture (Parmelli 2011) |
NS | NS | NS | NS | NS | NS | NS | NS | NS |
| Strategies targeted at healthcare workers by type of intervention | |||||||||
|
Printed educational materials (Giguère 2012) |
?⊕ooo | NR | ✓⊕⊕oo1 | NS | NR | NR | NS | NR | NR |
|
Internet‐based learning in health professions Vs no intervention (Cook 2008) |
NR | NR | ✓⊕⊕oo2 | NR | NR | NR | NR | NR | NR |
| Vs non‐Internet‐based learning (Cook 2008) |
NR | NR | ∅⊕⊕oo2 | NR | NR | NR | NR | NR | NR |
|
Educational meetings Continuing education meetings (Forsetlund 2009) |
✓⊕⊕⊕o | NR | ✓⊕⊕⊕o1 | NR | NR | NR | NR | NR | NR |
| Interprofessional education (Reeves 2013) |
✓⊕⊕oo | NR | ✓⊕⊕oo1 | NR | NR | NR | ?⊕ooo3 | NR | NR |
| Teaching critical appraisal (Horsley 2011) |
NS | NR | NS | NR | NR | NR | NR | NR | ✓⊕⊕oo4 ?⊕ooo5 |
| Nutrition training of health workers Sunguya 2013 |
NR | NR | NR | NR | NR | NR | NR | NR | ✓⊕⊕⊕⊕6 |
|
Educational outreach Vs no intervention O'Brien 2007 |
NR | NR | ✓⊕⊕⊕⊕7 ✓⊕⊕⊕o8 |
NR | NR | NR | NR | NR | NR |
| Vs another intervention O'Brien 2007 |
NR | NR | ✓⊕⊕oo9 | NR | NR | NR | NR | NR | NR |
| Practice facilitation Baskerville 2012 |
NR | NR | ✓⊕⊕⊕o1 | NR | NR | NR | NR | NR | NR |
|
Pharmacist‐provided services targeted at patients, such as patient health education and follow‐up Pande 2013 |
✓⊕⊕oo10 | ✓⊕⊕oo11 | NR | NS | NS | NR | NR | NR | ✓⊕⊕oo12 |
|
Pharmacist‐provided services targeted at healthcare professionals, such as educational outreach visits Pande 2013 |
✓⊕⊕oo | NR | NS | NS | NS | NR | NS | NS | NR |
|
Local opinion leaders (Flodgren 2011) |
NR | NR | ✓⊕⊕⊕o1 | NR | NR | NR | NR | NR | NR |
|
Audit and feedback – (with or without other interventions) vs usual care (Ivers 2012) |
∅⊕⊕⊕o | NR | ✓⊕⊕⊕o1 | NR | NR | NR | NR | NR | NR |
|
Audit and feedback – compared with other interventions (Ivers 2012) |
∅⊕⊕⊕o | NR | ∅⊕⊕⊕o1 | NR | NR | NR | NR | NR | NR |
|
Reminders – intensive care unit, emergency department and acute care settings (Ko 2011) |
NR | NR | ?⊕ooo1 | NR | NR | NR | NR | NR | NR |
|
Reminders – surgery setting (Ko 2011) |
✓⊕⊕oo13 | NR | NR | NR | NR | NR | NR | NR | NR |
|
Tailored interventions ‐ Vs no intervention (Baker 2015) |
NR | NR | ✓⊕⊕⊕o1 | NR | NR | NR | NR | NR | NR |
| ‐ Vs non‐tailored interventions (Baker 2015) |
NR | NR | ?⊕ooo1 | NR | NR | NR | NR | NR | NR |
| ‐ Interventions targeted at organisational and individual barriers vs interventions targeted at individual barriers only (Baker 2015) |
NR | NR | ?⊕ooo1 | NR | NR | NR | NR | NR | NR |
|
Multifaceted interventions ‐ Interventions to encourage the use of systematic reviews in clinical decision‐making (Perrier 2011) |
NS | NR | ?⊕ooo14 | NR | NR | NR | NS | NR | NR |
| Strategies targeted at healthcare workers by type of problem | |||||||||
|
Communication with patients – interventions to promote patient‐centred care Dwamena 2012 |
✓⊕⊕⊕o15 | NR | ✓⊕⊕⊕o16 | NR | NR | NR | NR | NR | NR |
|
Handwashing – educational interventions and marketing campaigns Gould 2010 |
?⊕ooo17 | NR | ✓⊕⊕oo18 | NR | NR | NR | NR | NR | ✓⊕⊕oo19 |
|
Obstetrics – non‐clinical interventions to reduce unnecessary cesarean section rates Guidelines + mandatory second opinion Khunpradit 2011 |
?⊕⊕oo | NR | ✓⊕⊕oo | NR | NR | NR | NR | NR | NR |
| Nurse training/insurance reform/legislative changes/audit & feedback/external peer review Khunpradit 2011 | ?⊕⊕oo | NR | ?⊕⊕oo | NR | NR | NR | NR | NR | NR |
|
Obstetrics – traditional birth attendants (TBA) Trained TBAs versus untrained TBA Sibley 2012 |
?⊕⊕oo | NR | ?⊕⊕oo20 | NR | NR | NR | NR | NR | NR |
| Additional TBAs training vs no additional TBA training Sibley 2012 | ✓⊕⊕oo21 ✓⊕⊕oo22 ∅⊕⊕oo23 |
NR | ∅⊕⊕oo24 | NR | NR | NR | NR | NR | NR |
| Obstetrics – skilled births attendance vs usual care Yakoob 2011 | ✓⊕⊕oo25 | NR | NR | NR | NR | NR | NR | NR | NR |
|
Obstetrics – birth kits Hundley 2012 |
✓⊕⊕⊕⊕26 ✓⊕⊕⊕o27 ✓⊕⊕oo28 |
NR | NR | NR | NR | NR | NR | NR | NR |
|
Prescribing antibiotics Clinician education with/without other intervention vs no intervention Ranji 2008 |
NR | NR | ✓⊕⊕⊕o29 | NR | NR | NR | NR | NR | NR |
|
Seriously ill newborn care – in‐service neonatal emergency care training Opiyo 2015 |
∅⊕⊕oo30 | NR | ✓⊕⊕⊕o31 | NR | NR | NR | NR | NR | NR |
|
Traditional healers – training about STD and HIV medicine Sorsdahl 2009 |
NR | NR | ✓⊕⊕oo32 ∅⊕⊕oo33 |
NR | NR | NR | NR | NR | ✓⊕⊕oo34 |
| Strategies targeted at healthcare recipients | |||||||||
|
Providing information/education – multimedia for promoting HIV testing Vidanapathirana 2005 |
∅⊕⊕oo35 | ✓⊕⊕oo36 | NR | NR | NR | NR | NR | NR | NR |
|
Providing information/education – leaflets for promoting HIV testing Vidanapathirana 2005 |
NR | ✓⊕⊕⊕⊕37 | NR | NR | NR | NR | NR | NR | NR |
|
Providing information/education – gain‐framed versus loss‐framed video tapes for promoting HIV testing Vidanapathirana 2005 |
NR | ✓⊕⊕⊕⊕38 | NR | NR | NR | NR | NR | NR | NR |
|
Providing information: written medicine information Nicolson 2009 |
NS | NR | NR | NR | NR | NR | NR | NR | ✓⊕⊕oo39 ∅⊕⊕oo40 |
|
Single interventions to improve health literacy (e.g. presenting essential information first, presenting information so that the higher numbers indicate better quality) Berkman 2011 |
?⊕ooo | ?⊕ooo | NR | NS | NR | ?⊕ooo | NR | NR | ✓⊕⊕oo41 |
|
Mixed interventions to improve health literacy (intensive self‐management and adherence and intensive disease management) Berkman 2011 |
✓⊕⊕⊕o42 | ✓⊕⊕⊕o43 | NR | ?⊕ooo | NR | NS | NR | NR | NR |
|
Adherence – medication: interventions to improve adherence to short‐term treatments Haynes 200844 |
?⊕⊕oo45 | NR | NR | NR | NR | NR | NR | NR | NR |
|
Adherence – medication: interventions to improve adherence to long‐term treatments (more than 6 months) Haynes 200840 |
?⊕⊕oo46 ✓⊕⊕oo47 |
NR | NR | NR | NR | NR | NR | NR | NR |
|
Adherence – ART for HIV/AIDS: behavioural interventions Simoni 2006 |
✓⊕⊕⊕o35 | NR | ✓⊕⊕oo48 | NR | NR | NR | NR | NR | NR |
|
Adherence ‐ ART for HIV/AIDS Mbuagbaw 2013 |
✓⊕⊕⊕⊕49 ∅⊕⊕oo51 |
NR | NR | NR | NR | NR | NR | NR | ∅⊕⊕oo52 |
|
Adherence – TB: incentives vs routine care Lutge 2015 |
NR | ✓⊕⊕oo50 ✓⊕⊕⊕o53 ?⊕ooo54 ∅⊕⊕oo55 |
NR | NR | NR | NR | NR | NR | NR |
|
Adherence – TB: immediate vs deferred incentives Lutge 2015 |
∅⊕⊕oo56 | NR | NR | NR | NR | NR | NR | NR | NR |
|
Adherence – TB: cash vs non‐cash incentive Lutge 2015 |
NR | ✓⊕⊕oo57 | NR | NR | NR | NR | NR | NR | NR |
|
Adherence – TB: different levels of cash incentives Lutge 2015 |
NR | ✓⊕⊕oo58 | NR | NR | NR | NR | NR | NR | NR |
|
Adherence – TB: incentives vs other interventions Lutge 2015 |
✓⊕⊕oo59 | ✓⊕⊕oo60 | NR | NR | NR | NR | NR | NR | NR |
|
Adherence – TB: mobile phone messages reminders Liu 2014 |
NR | ✓⊕⊕oo61 ✓⊕⊕⊕o62 |
NR | NR | NR | NR | NR | NR | |
|
Adherence – malaria medication: blister packed tablets and capsules vs tablets and capsules in paper envelopes Orton 2005 |
✓⊕⊕oo63 | NR | NR | NR | NR | NR | NR | NR | NR |
|
Adherence – malaria medication: tablets in sectioned polythene bags vs bottled syrup Orton 2005 |
✓⊕⊕oo64 ?⊕ooo65 |
NR | NR | NR | NR | NR | NR | ?⊕⊕oo66 | NR |
|
Adherence – malaria medication:tablets in sectioned polythene bags vs tablets and capsules in paper envelopes Orton 2005 |
✓⊕⊕⊕o67 ✓⊕⊕oo68 |
NR | NR | NR | NR | NR | NR | ∅⊕⊕oo68 | NR |
|
Adherence – malaria medication: tablets in sectioned polythene bags vs polythene bags (unsectioned) Orton 2005 |
?⊕ooo70 | NR | NR | NR | NR | NR | NR | NR | NR |
|
Immunisation coverage: health education Oyo‐Ita 2016 |
NR | ✓⊕⊕⊕o71 | NR | NR | NR | NR | NR | NR | NR |
|
Immunisation coverage: healthcare providers training Oyo‐Ita 2016 |
NR | ∅⊕⊕oo71 | NR | NR | NR | NR | NR | NR | NR |
|
Immunisation coverage: home visits Oyo‐Ita 2016 |
NR | ✓⊕⊕oo71 | NR | NR | NR | NR | NR | NR | NR |
|
Immunisation coverage: monetary incentives (withdrawing of monetary vouchers) Oyo‐Ita 2016 |
NR | ∅⊕⊕oo71 | NR | NR | NR | NR | NR | NR | NR |
|
Immunisation coverage:multifaceted interventions (monetary incentives + quality assurance + provision of equipment, drugs and materials) Oyo‐Ita 2016 |
NR | ∅⊕⊕oo71 | NR | NR | NR | NR | NR | NR | NR |
| Immunisation coverage: reminders and recall systems Jacobson Vann 2005 | NR | ✓⊕⊕⊕o71 | NR | NR | NR | NR | NR | NR | NR |
|
Malaria: Providing free ITNs vs subsidised ITNs Augustincic Polec 2015 |
NS | ✓⊕⊕⊕o72 ∅⊕⊕⊕o73 |
NS | NS | NS | NS | NS | NS | NR |
|
Malaria: Education about appropriate ITN use vs no education Augustincic Polec 2015 |
NS | ✓⊕⊕oo74 | NS | NS | NS | NS | NS | NS | NR |
|
Access to healthcare services: Interventions to improve the uptake of cervical cancer screening Invitations to attend screening (face‐to‐face, letter, telephone) Everett 2011 |
NR | ✓⊕⊕⊕o75 | NR | NR | NR | NR | NR | NR | NR |
| Educational interventions for women Everett 2011 |
NR | ✓⊕⊕⊕o75 | NR | NR | NR | NR | NR | NR | NR |
| Counselling Everett 2011 |
NR | ✓⊕⊕⊕o75 | NR | NR | NR | NR | NR | NR | NR |
| Risk factor assessment Everett 2011 |
NR | ∅⊕⊕oo75 | NR | NR | NR | NR | NR | NR | NR |
| Other interventions (access to a health promotion nurse, intensive recruitment) Everett 2011 |
NR | ✓⊕⊕⊕o75 | NR | NR | NR | NR | NR | NR | NR |
| Other interventions (photo‐comic book, message framing) Everett 2011 |
NR | ∅⊕⊕⊕o75 | NR | NR | NR | NR | NR | NR | NR |
|
Access to healthcare services – outreach strategies for increasing health insurance coverage for vulnerable populations Health insurance information and application support Jia 2014 |
NR | ✓⊕⊕⊕o76 | ✓⊕⊕⊕o77 | NR | NR | NR | NR | NR | NR |
| Handing out application forms in emergency department of hospitals Jia 2014 |
NR | ✓⊕⊕⊕o76 | NR | NR | NR | NR | NR | NR | NR |
| Ratings | Implications | ||||||||
| ⊕⊕⊕⊕ High |
This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different is low. | This evidence provides a very good basis for making a decision about whether to implement the intervention. Impact evaluation and monitoring of the impact are unlikely to be needed if it is implemented. | |||||||
| ⊕⊕⊕o Moderate |
This research provides a good indication of the likely effect. The likelihood that the effect will be substantially different is moderate. | This evidence provides a good basis for making a decision about whether to implement the intervention. Monitoring of the impact is likely to be needed and impact evaluation may be warranted if it is implemented. | |||||||
| ⊕⊕oo Low |
This research provides some indication of the likely effect. However, the likelihood that it will be substantially different is high. | This evidence provides some basis for making a decision about whether to implement the intervention. Impact evaluation is likely to be warranted if it is implemented. | |||||||
| ⊕ooo Very low |
This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different is very high. | This evidence does not provide a good basis for making a decision about whether to implement the intervention. Impact evaluation is very likely to be warranted if it is implemented. | |||||||
a✓: a desirable effect; ∅: little or no effect; ?: an uncertain effect; ✕: an undesirable effect; NS: no studies were included; NR: not reported; NA: no plausible mechanism by which the type of implementation strategy might be expected to have an effect on the type of outcome. bOther than adverse effects on any of the outcomes in the previous columns. 1Adherence to recommended practice (guidelines recommendations). 2Clinical behaviour end effects on patient care. 3Collaborative behaviour and practitioner competencies. 4Provider's knowledge. 5Provider's skills. 6Daily energy intake, feeding frequency, consumption of targeted food items. 7Prescribing. 8Management of patients at increased cardiovascular risk, with asthma or diabetes; or delivery of preventive services, including counselling for smoking cessation. 9Better documentation of care, preventive cardiovascular care or prostate specific antigen testing in primary care. 10Hospitalisation, general practice visits and emergency room visits. 11Patients' medication costs. 12Clinical outcomes (for diabetic and hypertensive patients such as reductions in fasting plasma glucose levels or systolic and diastolic blood pressure). 13Death rate and major complications. 14Changes in physician performance. 15Patient health status (included physiological measures, clinical assessments, patient self‐reports of symptom resolution or quality of life; and patient self‐esteem). 16Patient satisfaction. 17Healthcare‐associated infections. 18Compliance with hand hygiene guidance. 19Use of hand hygiene products. 20Frequency of referral to emergency obstetrical care. 21Maternal mortality. 22Maternal morbidity: haemorrhage, infections, obstructed labour and referral to emergency. 23Neonatal deaths and stillbirths. 24Breastfeeding and appropriate advice about immediate feeding of colostrum. 25Stillbirths and perinatal mortality. 26Neonatal tetanus‐related mortality and puerperal sepsis. 27Neonatal mortality, maternal mortality, haemorrhage. 28Neonatal sepsis. 29Number of patient visits at which an antibiotic is prescribed. 30Mortality in all resuscitation episodes. 31Proportion of adequate initial resuscitation steps, Inappropriate and potentially harmful practices per resuscitation. 32HIV/STI patient management practices. 33Incidence of HIV/AIDS risk behaviours and referral practices (self‐reported). 34Traditional healers' knowledge about STDs and HIV. 35Number of people who were positive for HIV antibody testing. 36Number of people tested for HIV. 37Number of pregnant women taking HIV tests. 38Proportion of low‐income ethnic minority women being HIV tested. 39Knowledge and attitudes about medicines. 40Compliance with medicine's instructions. 41Comprehension 42Disease prevalence 43Use of health services. 44See Nieuwlaat 2014 in the main text for updated findings. 45Adherence to medications and various clinical outcomes. 46Treatment outcomes. 47Adherence to medications. 48Undetectable viral load. 49Adherence to ART. 50Return for tuberculin skin test reading. 51Morality. 52Loss to follow‐up. 53Return for start or continuation of treatment. 54Completion of treatment for active TB. 55Completion of TB prophylaxis. 56Adherence to anti‐tuberculosis treatment. 57Patient return for tuberculin skin test reading and completion of TB prophylaxis. 58Patient return for tuberculin skin test reading. 59Adherence to anti‐tuberculosis prophylaxis. 60Return to clinic for completion of treatment and prophylaxis for latent TB. 61Clinic attendance. 62Completion of TB treatment. 63Adherence to malaria treatment and treatment failure. 64Adherence to malaria treatment. 65Treatment failure. 66Number of minor adverse events in malaria treatment. 67Adherence to malaria treatment. 68Treatment failure. 69Incidence of itching, dizziness and other adverse events. 70Treatment adherence and treatment failure in malaria. 71Routine childhood immunisations. 72 ITN possession among pregnant women, adults and children. 73Appropriate use of ITNs. 74ITN use by adults, ITN use by children under 5 years old. 75Uptake of screening. 76Enrolment into insurance. 77Parental satisfaction.
We took into account other relevant considerations besides the findings of the included reviews when drawing conclusions about implications for practice (EPOC 2017). This included considerations related to the applicability of the findings and likely impacts on equity. Our conclusions about implications for systematic reviews were based on types of implementation strategies for which we were unable to find a reliable, up‐to‐date review and limitations identified in the included reviews. Our conclusions about implications for future evaluations were based on the findings of the included reviews (EPOC 2017).
Results
We identified 7272 systematic reviews of health systems arrangements and implementation strategies. We excluded 6926 reviews from this overview following a review of titles and abstracts. We retrieved the full texts of 82 reviews for further detailed assessment. Of these, we excluded 32 reviews: 6 because they had important methodological limitations, 11 that were out‐of‐date, 11 that focused on an area already covered by one of the included reviews and 4 that were of limited relevance to low‐income countries (Table 4). We included 39 systematic reviews published between 2005 and 2016 in this overview. In addition, four reviews provided supplementary numerical data or methodological information used in a SUPPORT Summary (Figure 1; Appendix 4). Seven related reviews were similar to one of the primary reviews. We did not prepare SUPPORT Summaries for the related reviews, and they did not contribute data to the summaries or the overview because of substantial overlap with one of the primary reviews. Following the screening of the subsequent searches of PDQ‐Evidence, we identified five ongoing reviews of implementation strategies (Brennan 2009; Dudley 2009; Fønhus 2016; Pantoja 2014b; Rowe 2015); six additional systematic reviews of implementation strategies that are awaiting assessment (Baldwin 2011; Mauger Rothenberg 2012; Mundell 2013; Oluoch 2012; Rolfe 2014; Tannenbaum 2013; Appendix 5); and a number of other reviews that are awaiting classification and also need to be checked for relevance to this overview (Appendix 5).
1.
Flow diagram
Description of included reviews
Of the 39 included systematic reviews, 27 were Cochrane Reviews and 12 were non‐Cochrane reviews. Fifteen of the reviews were updates of earlier reviews (Baker 2015; Berkman 2011; Everett 2011; Forsetlund 2009; Giguère 2012; Gould 2010; Haynes 2008; Horsley 2011; Ivers 2012; Liu 2014; Lutge 2015; O'Brien 2007; Oyo‐Ita 2016; Reeves 2013; Sibley 2012).
The reviews reported results from 1332 studies (Appendix 3). Study designs included: 710 randomised trials; 26 non‐randomised trials; 34 controlled before‐after studies; 50 interrupted time series; 69 repeated measures studies; and 243 other designs (cross‐sectional, cohort, historical or ecological, and quasi‐experimental studies). The number of studies included in each review ranged from 0 in Parmelli 2011 to 201 in Cook 2008. Dates of most recent searches in the reviews ranged from April 2004 to May 2016.
Most studies in the reviews were from the USA, the UK, Canada, the Netherlands and Australia. There were no studies from low‐income countries in eight reviews (Baskerville 2012; Dwamena 2012; Horsley 2011; Jia 2014; Ranji 2008; Reeves 2013; Simoni 2006; Vidanapathirana 2005). Study settings varied and included outpatient and inpatient settings in hospitals, health centres, families, workplaces and community settings (Appendix 3). The health professionals included in the reviews were physicians, nurses, pharmacists, psychologists, dentists, social workers and traditional healers. The participants included in the reviews were children, adults and pregnant mothers. Outcomes examined included healthcare provider performance, quality of care, patient outcomes, access to care, coverage, utilisation of health services, resource use, impacts on equity and adverse effects (Table 7, Appendix 3).
Implementation strategies addressed in the reviews were grouped into four categories based on the level of the healthcare system targeted by the intervention (Table 1; Table 3).
Strategies targeting healthcare organisations (e.g. strategies to change organisational culture) (1 review).
Strategies targeting healthcare workers (e.g. printed educational materials) (14 reviews).
Strategies targeting healthcare workers to address a specific problem (e.g. unnecessary use of antibiotics) (9 reviews).
Strategies targeting healthcare recipients (e.g. medication adherence) (15 reviews).
Methodological quality of included reviews
We have summarised our assessment of the methodological quality (reliability) of the included reviews in Table 5. We judged 32 reviews to have only minor limitations. We judged the other seven reviews to have methodological limitations that were important enough to make it worthwhile to search for another systematic review or to interpret the review results cautiously, if no better review were available (Baskerville 2012; Haynes 2008; Ko 2011; Nicolson 2009; Ranji 2008).
We assessed the included reviews as being well conducted in relation to the identification, selection and critical appraisal of the included studies. However, we assessed the comprehensiveness of the search for evidence as only partially achieved in 19 reviews. We also assessed the included reviews as being well conducted in relation to the analysis of the available evidence, with only a few of them presenting limitations related to descriptions of the extent of heterogeneity (Haynes 2008; Nicolson 2009; Perrier 2011; Ranji 2008), the methods used to synthesise the evidence (Haynes 2008; Ko 2011; Nicolson 2009; Pande 2013; Perrier 2011), and the examination of factors that might explain differences in the results of included studies (Gould 2010; Nicolson 2009; Pande 2013; Ranji 2008).
Effect of interventions
We used a pragmatic approach to group the interventions assessed in the overview based on the level of the healthcare system targeted by the intervention: healthcare organisations, healthcare workers, and healthcare recipients. Additionally, for interventions targeted at healthcare workers, we distinguished reviews evaluating a specific type of intervention from those evaluating interventions for a specific problem that we considered especially relevant to low‐income countries (Table 3).
We report here the main findings using plain language statements based on GRADE 'Summary of findings' tables that we prepared for each included review (EPOC 2017). The 'Summary of findings' tables are available in the SUPPORT Summaries database.
Strategies targeted at healthcare organisations
Organisational culture
One review assessed the effects of strategies to change organisational culture to improve healthcare performance but did not find any eligible studies (Parmelli 2011). It was therefore not possible to draw any conclusions about the impacts of this type of organisational intervention.
Continuous quality improvement
Continuous quality improvement (CQI) comprises improvement of organisational processes, use of structured problem‐solving processes incorporating statistical methods and measurement to diagnose problems and monitor progress, use of teams including employees from multiple departments and from different organisational levels, empowerment of employees to identify quality problems and create opportunities to correct them, and an explicit focus on 'customers'. None of the included reviews assessed the effects of CQI strategies. One Cochrane Review on this topic is in progress (Brennan 2009).
Strategies targeted at healthcare workers by type of intervention
In order to give a general comparative picture of the range of effects found for this group of interventions we summarised the findings in Appendix 6. Additionally we present the specific findings for each strategy below.
Educational materials
One review assessed the effects of printed educational materials on professional practice. The review found 45 studies done in a variety of settings (e.g. outpatient, inpatient, community) and involving different healthcare professionals (e.g. physicians, nurses, allied health professionals in the field of community health) (Giguère 2012). Printed educational materials may slightly improve practice outcomes (e.g. diagnosis, prescribing, referral practices) among healthcare providers, when used alone and compared to no intervention (low‐certainty evidence). There were no studies assessing the effects of these materials on patient outcomes.
Internet‐based learning
Internet‐based learning refers to any educational intervention delivered to healthcare workers through the Internet. This approach is intended to allow learners to participate at a time and place convenient to them and to facilitate innovation in instructional methods. It also potentially allows instruction to be tailored to the individual's needs.
One review assessing the effects of Internet‐based learning in health professions found 201 studies addressing a wide range of topics and using a range of modalities for teaching and learning (Cook 2008). Compared with no intervention, Internet‐based learning may improve health workers' knowledge (low‐certainty evidence), but it is unclear whether it improves health professionals' skills and behaviours, or if it leads to beneficial effects on patients (very low‐certainty evidence). When compared to other forms of teaching and learning, Internet‐based learning may improve knowledge, but may not improve satisfaction, skills, behaviour and patient outcomes (low‐certainty evidence).
Educational meetings and workshops
Four reviews were included in this category (Forsetlund 2009; Reeves 2013; Horsley 2011; Sunguya 2013).
The first review assessed the effects of educational meetings and workshops on professional practice and healthcare outcomes (Forsetlund 2009). The review identified 81 studies. Findings showed that educational meetings alone or combined with other interventions probably improve professional practice and healthcare outcomes for patients (moderate‐certainty evidence). Combined interactive and didactic (lecture‐based) educational meetings may be slightly more effective than didactic educational meetings alone (low‐certainty evidence).
The second review assessed the effects of interprofessional education (IPE) on professional practice and healthcare outcomes (Reeves 2013). The review identified 15 studies. Compared with separate, profession‐specific educational interventions or no education intervention, IPE may lead to improvements in outcomes for patients, adherence to clinical guidelines, and clinical processes (e.g. shared decisions on surgical incisions) (low‐certainty evidence).
The third review assessed the effects of teaching health professionals critical appraisal skills on their knowledge (Horsley 2011). The review identified three studies. Teaching critical appraisal skills, compared to usual practice, may improve health professionals' knowledge on how to critically appraise research papers (low‐certainty evidence). Effects on critical appraisal skills were uncertain. None of the studies evaluated process of care or patient‐related outcomes
The final review assessed the effect of nutrition training of health workers on caregivers’ feeding practices for children aged six months to two years (Sunguya 2013). The review identified ten studies. Nutrition training of health workers, compared to usual care, increases daily energy intake, feeding frequency, and consumption of targeted food items. The certainty of evidence was high. None of the included studies assessed cost or health outcomes (such the proportion of undernourished children or children with adverse health outcomes).
Local consensus processes
Consensus development processes are decision‐making processes that aim to help a group of people reach agreement about a given issue. Healthcare workers have used local consensus processes to achieve agreement on clinical policies and guidelines. None of the included reviews assessed the effects of local consensus processes.
Educational outreach
Three reviews assessed the effects of educational outreach interventions on professional practice and healthcare outcomes (Baskerville 2012; O'Brien 2007; Pande 2013). Educational outreach visits entail the use of a trained person from outside the practice setting to meet with healthcare professionals in their practice in order to provide information with the intent of improving practice, for example feedback about health worker performance. This type of face‐to‐face visit is also called academic detailing and educational visiting.
A review assessing the effects of educational outreach visits on professional practice and practice outcomes found 69 studies (O'Brien 2007). Educational outreach visits alone or combined with other interventions probably improve the quality of care delivered to patients (moderate‐certainty evidence). For prescribing, the effects are relatively consistent (high‐certainty evidence); for other types of professional performance, the effects vary widely from small to modest improvements (moderate‐certainty evidence). The effects on patient outcomes were uncertain (very low‐certainty evidence).
Practice facilitation is a multifaceted approach whereby skilled individuals, either internal or external to a primary care setting, promote the adoption and use of evidence‐based guidelines. A review assessing the effects of practice facilitation on evidence‐based practice behaviours identified 23 studies (Baskerville 2012). All of them took place in high‐income countries. The use of practice facilitation probably improves the adoption of evidence‐based guidelines (moderate‐certainty evidence).
Another review assessing the effect of additional pharmacist‐provided services included one study of strategies targeted at healthcare professionals, comparing educational outreach to usual care (Pande 2013). The findings showed that when pharmacists provide additional services targeted at health professionals, such as educational outreach visits, patient outcomes may improve (low‐certainty evidence). Effects on healthcare cost were uncertain (no studies were found).
Local opinion leaders
Opinion leaders are individuals in a community or organisation who have a substantial influence on what the rest of the community or organisation does. Because of their influence, opinion leaders may be able to persuade healthcare providers to use the best available evidence when managing patients. A review assessing the effects of local opinion leaders on healthcare professional behaviour and patient outcomes found 18 studies conducted in both hospital and primary care settings (Flodgren 2011). Local opinion leaders, acting alone or in concert with other interventions, probably improve healthcare workers' adherence to desired practice (moderate‐certainty evidence). None of the included studies assessed patient outcomes.
Patient‐mediated interventions
Patient‐mediated interventions include any intervention aimed at changing the performance of healthcare professionals through interactions with patients or information provided by or to patients. A Cochrane Review of patient‐mediated interventions is in progress (Fønhus 2016).
Audit and feedback
A review assessing the effects of audit and feedback on the practice of healthcare professionals and patient outcomes identified 140 studies using a wide range of interventions with respect to their content, format, timing and source (Ivers 2012). Overall they found that interventions that include audit and feedback (alone or as a core component of a multifaceted intervention), compared with usual care, probably improve adherence to desired practice (moderate‐certainty evidence) and probably lead to little difference in patient outcomes (moderate‐certainty evidence). Compared with a number of educational interventions (e.g. reminders, educational outreach) and organisational, financial or patient‐mediated interventions, audit and feedback probably leads to little or no difference in compliance with desired practice or patient outcomes (moderate‐certainty evidence).
Reminders
One review assessed if using safety checklists improved patient safety in acute hospital settings compared to not using them. The review included nine studies that evaluated a wide variety of checklist designs as well as training on use of the checklists (Ko 2011). The findings showed that surgical safety checklists may reduce death rates and major complications within 30 days after surgery (low‐certainty evidence). It was uncertain whether safety checklists improve adherence to guidelines or patient safety in intensive care units, emergency departments or acute care settings (very low‐certainty evidence).
One Cochrane Review on the effects of manual paper reminders on professional practice outcomes is in progress (Pantoja 2014b). We excluded two Cochrane Reviews assessing the effects of computer‐generated or onscreen reminders because of their limited relevance to low‐income countries (Arditi 2012; Shojania 2009).
Tailored interventions
A range of barriers may impede changes to health professional behaviour. Change may be more likely if implementation strategies address specific barriers. A review assessing the effects of interventions tailored to address specific barriers to change on professional practice and healthcare outcomes identified 26 studies. The review found that these interventions were probably more likely to improve professional practice than no intervention or dissemination of guidelines alone (moderate‐certainty evidence). However, it is uncertain whether tailored interventions are more likely to improve professional practice than non‐tailored interventions, or whether tailored interventions targeted at organisational and individual barriers are more likely to improve professional practice than tailored interventions targeted only at individual barriers (very low‐certainty evidence). A recent update of this review identified six additional trials but without any change to the conclusions (Baker 2015).
Multifaceted interventions
Multifaceted interventions are those including more than one of the interventions described above. Four reviews examined the effects of multifaceted interventions on professional practice and/or patient outcomes.
One review assessed the effectiveness of interventions for seeking, appraising and applying evidence from systematic reviews in clinical decisions. It included five trials in middle‐ and high‐income countries evaluating multifaceted interventions (Perrier 2011). It is uncertain whether multifaceted interventions, such as training and workshops, improve informed decision‐making by healthcare workers in low‐income countries (very low‐certainty evidence).
Three reviews compared the effects of multifaceted interventions to audit and feedback (Ivers 2012), educational meetings (Forsetlund 2009), and outreach visits (O'Brien 2007). In the case of audit and feedback and outreach visits, the authors found that combining the core intervention with other interventions led to a larger effect size than using the core intervention alone. However, the results were inconsistent or based on indirect comparisons. On the other hand, Forsetlund 2009 did not find relevant differences between the effects of multifaceted interventions and educational meetings alone.
Strategies targeted at healthcare workers by type of problem
In order to have a general comparative picture of the range of effects found for this group of interventions, we have summarised the findings in Appendix 7. Additionally we present specific findings for each strategy below.
Communication with patients
One review assessed the effects of interventions for health providers that aim to promote patient‐centred care (PCC) in clinical consultations (Dwamena 2012). The review included 45 studies of training related to a variety of PCC skills (e.g. disease‐specific training for providers and patients). Compared to no intervention, there was moderate‐certainty evidence that patient‐centred training probably improves patient health status (e.g. clinical outcomes), and there was low‐certainty evidence that it may improve the patient‐provider consultation process (e.g. the communication of treatment options) and may slightly improve patient satisfaction with the consultation and patient behaviour (e.g. attendance at follow‐up consultation).
Handwashing
A review of interventions to improve hand hygiene adherence in patient care and reduce healthcare‐associated infections identified four hospital‐based studies (Gould 2010). This review found that compared to usual care, educational interventions may increase hand hygiene compliance (low‐certainty evidence). Compared to usual care, multifaceted marketing campaigns may increase the use of hand hygiene products (low‐certainty evidence), but the effects on healthcare‐associated infections are uncertain (very low‐certainty evidence).
Obstetric care
Four reviews examined strategies for improving obstetric care in pregnant women. The first review assessed the effects of non‐clinical interventions for reducing unnecessary caesarean section rates, based on 16 studies in community and hospital settings (Khunpradit 2011). Compared to standard prenatal care, the following non‐clinical interventions may decrease caesarean section rates: nurse‐led relaxation, birth preparation classes for mothers, implementation of guidelines with mandatory second opinion and with support from local opinion leaders, and audit and feedback given to individual care providers (low‐certainty evidence). Prenatal education and support programmes, computerised patient decision aids, decision‐aid booklets and intensive group therapy may have little or no overall effect on caesarean section rates (low‐certainty evidence).
The second review included studies of training traditional birth attendants (TBAs) and included six studies from rural communities (Sibley 2012). The review found that the training of untrained TBAs may reduce neonatal deaths and stillbirths, pregnancy‐related haemorrhage, and puerperal sepsis, and increase referral of pregnant women with obstetric complications (low‐certainty evidence). However, such training may increase the number of pregnant women with obstructed labour (low‐certainty evidence). The effect of providing training to untrained TBAs on maternal mortality was uncertain (very low‐certainty evidence). Also, the effect of providing additional training (on newborn resuscitation and breastfeeding) to TBAs who already have some formal training on maternal mortality, morbidity, stillbirths, neonatal deaths, exclusive breastfeeding, and advice about immediate feeding of colostrum was uncertain (very low‐certainty evidence).
The third review (21 studies) assessed the effects of skilled birth attendance and emergency obstetric care on stillbirths and perinatal mortality (Yakoob 2011). Participants in the studies included 'village midwives', professional midwives, and trained traditional birth attendants. Compared to usual care, skilled birth attendance may reduce stillbirths and perinatal mortality (low‐certainty evidence). The effect of alternative ways of providing emergency care on stillbirths was uncertain (very low‐certainty evidence).
The fourth review included nine studies and assessed the effects of birth kits on newborn and maternal outcomes (Hundley 2012). A birth kit was defined as any disposable kit intended for routine use in the intrapartum period, specifically at the delivery of the baby. Compared with no intervention, the use of birth kits (alongside with education or topical antimicrobial): reduces puerperal sepsis and neonatal tetanus‐related mortality (high‐certainty evidence), probably reduces maternal mortality, haemorrhage and neonatal mortality (moderate‐certainty evidence), and may reduce neonatal sepsis (low‐certainty evidence).
Prescribing antibiotics
One review assessed the effects of interventions to improve antibiotic prescribing in ambulatory settings. The review focused on the effectiveness of strategies to reduce antibiotic prescribing for acute outpatient illnesses for which antibiotics are often prescribed inappropriately, and included 43 studies (Ranji 2008). Most of the studies in the review focused on prescribing for acute respiratory tract infections. Interventions assessed included: clinician education (distribution of materials, educational meetings, educational outreach, educational workshops with or without guideline distribution); patient education (written educational materials, educational meetings); and clinician education plus other interventions (e.g. audit and feedback, patient education). Strategies such as clinician education and patient education alone or combined with audit and feedback probably reduce antibiotic prescribing in ambulatory care settings (moderate‐certainty evidence). The effects of the interventions on the proportion of patients treated with appropriate antibiotics and on clinical outcomes were not reported.
Seriously ill newborn care
A review of the effects of in‐service training of health professionals on emergency care of seriously ill newborns and children identified two hospital‐based studies (Opiyo 2015). In‐service neonatal emergency care training probably improves provider practices (i.e. increases adequate resuscitation and preparedness for resuscitation, probably decreases the frequency of inappropriate and potentially harmful resuscitation practices) (moderate‐certainty evidence) and may reduce mortality in newborns requiring resuscitation (low‐certainty evidence).
Quality of care for sexually transmitted diseases and HIV
One review examined the effectiveness of interventions for educating traditional healers about sexually transmitted diseases (STDs) and HIV and identified two studies (Sorsdahl 2009). The studies assessed the impact of short training courses on STDs, HIV and other related health issues (e.g. family planning). Training of traditional healers may increase their knowledge of STDs and HIV (signs and symptoms, prevention), patient management practices, and referral practice (low‐ to moderate‐certainty evidence). Training may lead to little or no difference in the incidence of HIV/AIDS risk behaviour and traditional healers' self‐reported referral practices (low‐certainty evidence).
Strategies targeted at healthcare recipients
Providing information or education
Four reviews examined the effects of providing information or education on health issues to healthcare recipients. None of the studies included in these reviews was from a low‐income country.
The first review assessed the effect of mass media interventions for promoting HIV testing and included 14 studies in diverse populations (Vidanapathirana 2005). Despite substantial heterogeneity in the populations studied, media used, duration and frequency of interventions, and study designs, each study showed that mass media increased initial uptake of HIV testing (high‐certainty evidence). However, the initial increase in uptake of HIV tests may not be sustained in the long‐term (low‐certainty evidence). Mass media interventions may lead to an increase in the number of infected people diagnosed through voluntary counselling and testing.
The second review (25 studies) addressed the effects of written information about prescribed and over‐the‐counter medicines (Nicolson 2009). Written medicine information may lead to little or no difference in adherence to instructions compared with no written information (low‐certainty evidence). None of the included studies assessed health outcomes.
The third review (128 studies) focused on the effects of health literacy interventions (e.g. simplifying the presentation of information) (Berkman 2011). Some mixed strategies such as intensive self‐management and adherence interventions probably improve the use of healthcare across health literacy levels (moderate‐certainty evidence). It is uncertain whether single strategies improve the use of healthcare services, health outcomes, resource use, or disparities in the use of healthcare services (very low‐certainty evidence).
The fourth review (11 studies) assessed the effect of additional pharmacist‐provided services targeted at patients versus usual care (Pande 2013). The intervention comprised patient education, counselling, complete pharmaceutical care follow‐up, and bespoke educational booklets explaining disease, medication and lifestyle modifications. The findings showed that pharmacist‐provided services targeted at patients may reduce the use of specific health services (e.g. hospital admissions, general practitioner visits), reduce patients' medication costs, and improve some clinical outcomes (low‐certainty evidence).
Medication adherence
One review examined the effects of interventions to improve patient adherence to medication (Haynes 2008). The review identified 78 studies conducted in many different settings, most of which were in high‐income countries. Nine studies evaluated 10 different interventions to increase short‐term adherence in very diverse conditions. The interventions evaluated were: the provision of more detailed instructions to patients (4 studies), the use of dose‐dispensing units of medication (1 study), counselling about the target disease of the patients (3 studies), the use of different medication formulations (1 study) and augmented pharmacy services (1 study). It is uncertain whether interventions to increase adherence to short‐term treatments improve adherence or patient outcomes (very low‐certainty evidence).
Seventy‐one studies evaluated 81 different interventions to increase adherence in diverse chronic conditions, including: asthma and chronic obstructive pulmonary disease (12 studies), hypertension (12 studies), diabetes (6 studies), HIV (12 studies), rheumatoid arthritis (2 studies), dyslipidemia (5 studies), mental health conditions (14 studies), epilepsy (1 study), heart failure (1 study) and ischaemic heart disease (1 study). Some studies focused on specific medications, such as oral anticoagulant therapy (1 study) and contraception (1 study). Two studies evaluated interventions to increase adherence to complex regimens in the elderly. Interventions aimed at increasing adherence to long‐term treatments may improve the adherence to medications (low‐certainty evidence), but it is uncertain whether interventions to increase adherence to long‐term treatments improve patient outcomes (very low‐certainty evidence).
The update of this review, which identified 109 additional studies, reported similar effects on adherence (Nieuwlaat 2014). However, it was not possible to prepare 'Summary of findings' tables for the update of Haynes 2008. The update found five randomised trials from the previous review to be ineligible and excluded them. Notably, review authors found 17 of the 182 trials to be at 'low risk of bias', and "generally involved complex interventions with multiple components, trying to overcome barriers to adherence by means of tailored ongoing support from allied health professionals such as pharmacists, who often delivered intense education, counselling (including motivational interviewing or cognitive behavioral therapy by professionals) or daily treatment support (or both), and sometimes additional support from family or peers. Only five of these trials reported improvements in both adherence and clinical outcomes, and no common intervention characteristics were apparent. Even the most effective interventions did not lead to large improvements in adherence or clinical outcomes". We did not assess the certainty of evidence for these outcomes as this would not have been reliable given the synthesis approach used in the updated review.
Adherence to antiretroviral therapy
Three reviews addressed strategies for improving adherence to antiretroviral therapy (ART) in patients with HIV/AIDs.
Simoni 2006 and Simoni 2010 examined the effects of patient support strategies and education for improving ART adherence. Simoni 2006 included 19 studies of the effects of a range of behavioural interventions (didactic information on ART, interactive discussions addressing cognitions, motivations, and expectations, cue dosing or cognitive‐behaviour therapy, reminders such as pagers). Simoni 2010 identified 10 additional studies. These reviews found that behavioural interventions probably lead to slightly better adherence to highly active antiretroviral therapy (HAART) (moderate‐certainty evidence), and they may slightly improve the number of patients with undetectable viral load (a laboratory measure of successful HAART) (low‐certainty evidence). Authors identified no studies measuring patient outcomes such as morbidity and mortality, and only one included study took place in a low‐income country.
Mbuagbaw 2013 focused on mobile text messaging for promoting adherence to ART. The review included three randomised trials. It found that mobile phone text messages compared to standard care improve adherence to ART for up to 12 months (high‐certainty evidence) but may lead to little or no difference in mortality or loss to follow‐up after up to 12 months (low‐certainty evidence). Weekly text messages probably improve adherence compared to daily text messages, and interactive text messages probably improve adherence compared to non‐interactive text messages (moderate‐certainty evidence).
Adherence to malaria treatment
A review of the effects of unit‐dose packaged treatment on treatment failure and treatment adherence in people with uncomplicated malaria identified five studies (Orton 2005). Use of blister packs, compared to tablets and capsules in paper envelopes, may improve adherence to treatment for malaria and may lead to slightly fewer treatment failures (low‐certainty evidence). No studies reported adverse effects. Use of sectioned polythene bags, compared to bottled syrup, may improve adherence to treatment in children under 5 years with malaria but may increase vomiting (low‐certainty evidence). It is uncertain whether there is a difference in clinical outcomes (very low‐certainty evidence). Compared to paper envelopes, use of sectioned polythene bags probably improves adherence to treatment (moderate‐certainty evidence), may slightly decrease treatment failures in children aged over seven years and adults with malaria (low‐certainty evidence), and may lead to little if any difference in adverse events (low‐certainty evidence). It is uncertain whether the use of sectioned polythene bags, compared with unsectioned bags, impacts on treatment adherence or patient outcomes (very low‐certainty evidence), or on adverse events (not reported).
Adherence to anti‐tuberculosis therapy
Two reviews examined strategies for improving adherence to drugs to prevent or cure tuberculosis (TB).
The first review included 11 studies and assessed the effects of material incentives given to patients undergoing diagnostic testing for TB or receiving drug therapy to prevent or cure TB (Lutge 2015). Most of the studies took place in high‐income countries. Compared to routine care, cash‐and non‐cash incentives probably increase health service utilisation (return visits for tuberculin skin test reading, start or continuation of treatment) (low‐ to moderate‐certainty evidence). They may not improve completion of TB prophylaxis (low‐certainty evidence), and it is uncertain if they improve completion of treatment for active TB (very low‐certainty evidence). Cash incentives may slightly improve patient return for tuberculin skin test reading and completion of TB prophylaxis compared to non‐cash incentives (low‐certainty evidence). Immediate (compared to deferred) incentives may not improve adherence to anti‐tuberculosis treatment (low‐certainty evidence).
The second review assessed the effects of reminder systems aimed at reminding patients to take their TB medication or attend appointments (pre‐appointment reminders) or to contact patients who have missed an appointment with default reminders (Liu 2014). The review included six trials of pre‐appointment reminders and three trials of default reminders. Five studies took place in the USA, two in India and one each in Spain and Irak. For patients being treated for active TB, pre‐appointment reminders may increase clinic attendance and the number of patients completing treatment (low‐certainty evidence), and default reminders probably increase the number of patients completing treatment (moderate‐certainty evidence) and may increase clinic attendance (low‐certainty evidence). For people on TB prophylaxis, pre‐appointment reminders may increase clinic attendance (low‐certainty evidence). For people undergoing screening for TB, pre‐appointment reminders may have little or no effect on the number of people who return to clinic for the result of their skin test (low‐certainty evidence).
Insecticide‐treated bednets for malaria
A review of the effects of strategies to increase ownership and use of insecticide‐treated bednets (ITNs) to prevent malaria identified 10 studies (Augustincic Polec 2015). Providing free ITNs, compared to subsidised ITNs, probably increases ITN ownership among households and women attending prenatal clinics (moderate certainty evidence). Providing free ITNs also probably results in increased ITN ownership compared to households paying full price or receiving a loan (moderate certainty evidence). The review also found that providing education on the proper use of ITNs may increase the number of people and children under five years sleeping under bednets (low‐certainty evidence).
Immunisation coverage
One review of intervention strategies to improve immunisation coverage identified six studies (Oyo‐Ita 2016). It found that health education (evidence‐based discussions, distribution of posters and leaflets, information campaigns) compared to usual care, may increase the uptake of routine childhood vaccination (low‐ to moderate‐certainty evidence). However, the authors noted that intervention costs should be carefully considered before implementation. Another review evaluated the effects of patient reminder and recall systems to improve immunisation coverage (Jacobson Vann 2005), finding that reminders and recall strategies probably increase routine childhood vaccination uptake (moderate‐certainty evidence).
Access to healthcare services
One review of interventions to increase the uptake of cervical cancer screening identified 36 studies in diverse settings (community clinics, primary care, health maintenance organisations) (Everett 2011). Compared to usual care or no intervention, invitations to attend cervical screening programmes, education, counselling, access to health prevention nurses, and intensive recruitment probably increase uptake of cervical cancer screening (moderate‐certainty evidence). Risk factor assessments, photo‐comic books, and message framing may lead to little or no difference on the uptake of screening (low‐ to moderate‐certainty evidence).
A second review assessed the effects of outreach strategies to increase health insurance coverage for vulnerable populations (Jia 2014). The review included one study that compared handing out application forms in the emergency department of hospitals to routine care. The findings showed that handing out application forms in the hospital emergency department probably increases the enrolment of children in health insurance schemes (moderate‐certainty evidence). The other study assessed the effects of health insurance information and application support compared with routine care. It found that health insurance information and application support probably increases the enrolment of children in health insurance schemes, leads to continuous enrolment of children in insurance schemes, decreases the mean time taken to obtain insurance for children, and leads to parental satisfaction with the process of enrolment (moderate‐certainty evidence).
Discussion
Summary of main results
The available evidence from 39 systematic reviews of implementation strategies relevant to health systems in low‐income countries covers a range of strategies (targeted at health professionals, patients, and organisations) in diverse settings (geographical and clinical environments), disease conditions, and populations (of both health professionals and patients). The estimates of effects and certainty of the estimates for the different strategies varied. Table 8 provides a summary of the main findings, organised into the following categories.
8. Summary of effects of interventions and certainty of evidence.
| Interventions found to have desirable effects on at least one outcome with moderate‐ or high‐certainty evidenceand no moderate‐ or high‐certainty evidence of undesirable effects |
Strategies targeted at healthcare workers by type of intervention
|
Strategies targeted at healthcare workers by type of problem
|
Strategies targeted at healthcare recipients
|
| Interventions found to have at least one outcome with little or no effect with moderate‐ or high‐certainty evidenceand no moderate‐ or high‐certainty evidence of desirable or undesirable effects |
Strategies targeted at healthcare workers by type of intervention
|
| Interventions for which the certainty of the evidence was low or very low (or no studies were found) for all outcomes examined |
Strategies targeted at healthcare organisations
|
Strategies targeted at healthcare workers by type of intervention
|
Strategies targeted at healthcare workers by type of problem
|
Strategies targeted at healthcare recipients
|
Interventions found to have desirable effects on at least one outcome with moderate‐ or high‐certainty evidence and no moderate‐ or high‐certainty evidence of undesirable effects.
Interventions found to have at least one outcome with little or no effect with moderate‐ or high‐certainty evidence and no moderate‐ or high‐certainty evidence of desirable or undesirable effects.
Interventions for which the certainty of the evidence was low or very low (or no studies were found) for all outcomes examined.
Overall completeness and applicability of evidence
There was extensive evidence for the effects of strategies targeting healthcare workers and healthcare recipients. However, we identified only one systematic review on strategies targeting healthcare organisations (Parmelli 2011), and that review did not find any eligible studies. Evidence of the effects of some interventions is not included in this overview because we did not find an eligible systematic review, even though there may be primary studies evaluating those interventions. Table 7 summarises the outcomes examined in the individual reviews. Few studies in the reviews considered equity outcomes (differential effects of interventions for disadvantaged populations, such as pregnant women, children aged under five, rural poor) (Augustincic Polec 2015; Berkman 2011; Ko 2011; Forsetlund 2009; Oyo‐Ita 2016). Similarly, few of the reviews included studies that examined the costs and cost‐effectiveness of interventions (Augustincic Polec 2015; Ivers 2012; Oyo‐Ita 2016; O'Brien 2007; Pande 2013). The sparse economic and equity data (in comparison to effectiveness data) limit assessment of the efficiency of the interventions that we examined.
We assessed the applicability of summarised evidence in the 'relevance' section of the SUPPORT Summaries, incorporating these judgments into our assessments of the certainty of the evidence. In general, it was difficult to draw generalisable conclusions regarding the applicability of findings to low‐income countries given that in most of the cases the evidence came from studies conducted in high‐income countries (USA, UK, Canada, Netherlands, Australia) with very different on‐the‐ground realities and important differences in health system arrangements compared to low‐income countries. This was particularly so for interventions that require substantial resources, advanced technology or specialised skills for their delivery (e.g. Internet‐based learning (Cook 2008), mass media interventions (Vidanapathirana 2005), health literacy interventions (Berkman 2011), and educational outreach visits (O'Brien 2007; Pande 2013)), and for interventions that are complex and require substantial changes in the organisation of care.
However, there are a limited number of interventions in the included reviews where most of the evidence is from low‐ and middle‐income countries and are likely to be applicable to low‐income countries. This may be the case for free insecticide‐treated bednets for malaria prevention (Augustincic Polec 2015), skilled birth attendance (Yakoob 2011), mobile phone messaging to improve adherence to ART (Mbuagbaw 2013), anti‐malarial drugs packed in unit doses (Orton 2005), and hand‐hygiene interventions (Gould 2010).
Certainty of the evidence
Some of the included reviews had specific methodological limitations related to the identification, selection and critical appraisal of the included studies, and the analysis of the available evidence. However, the included reviews were for the most part well conducted (Table 5). The certainty of the effect estimates for the different strategies varied, ranging from very low, for example for the effect of single interventions to improve health literacy, to high, indicating that the research provides a very good indication of the likely effect (for example, for educational outreach interventions to improve prescribing by health professionals) (Table 7).
Potential biases in the overview process
Although the searches used for PDQ‐Evidence are relatively comprehensive, it is possible that we did not identify some relevant reviews. We also excluded reviews that were published prior to April 2005. It is possible that some of those reviews provide information that is still useful and that might supplement information provided by the included reviews. Although this cut‐off was arbitrary, it is unlikely that we excluded a substantial amount of useful information. Fourteen of the included reviews were published before 2010, and it is possible that more recent research would change their conclusions.
Classifying the interventions in the included reviews was sometimes uncertain and required judgment. In particular, the distinction between 'delivery arrangements' (covered in Ciapponi 2014) and 'implementation strategies' was not always clear. Likewise, classifying interventions in reviews that addressed a problem (such as improving referrals from primary to secondary care) rather than a category of interventions (such as audit and feedback) was not always straightforward. There also are other ways of categorising implementation strategies (French 2012; Michie 2011). Although these judgments and differences in approaches to characterising implementation interventions are unlikely to have introduced bias into this overview, they might result in some confusion, since there is no universally agreed upon classification system for implementation interventions. Moreover, any system for categorising health system interventions is, to some extent, arbitrary. A unified taxonomy for classifying health system interventions – such as the one recently published by Lavis 2015 – could facilitate explicit and systematic synthesis and interpretation of the existing body of evidence on health systems interventions.
Judgments about the relevance of some interventions to low‐income countries were sometimes difficult to make, as were judgments about the applicability of some of the findings of the included reviews (Lavis 2009). While these judgments might have been biased, this seems unlikely. At least two of the authors of this overview assessed the eligibility of systematic reviews for inclusion, and judgments about the applicability of the findings of included reviews were made by the authors of each SUPPORT Summary, externally peer reviewed, and then reviewed by at least two authors of this overview. Our general approach has been inclusive rather than exclusive so that readers can assess for themselves the relevance of the included reviews. Similarly, our approach has been to assume that findings are applicable to low‐income countries, unless there are relevant explicit differences between the settings where the studies took place and settings in low‐income countries, or specific factors that would likely modify the effects in low‐income countries.
An important limitation of our overview is that it depends on the findings of systematic reviews that have not used consistent methods. Although there are potential advantages of a broad review that would include all relevant implementation strategies (e.g. Grimshaw 2004; Rowe 2015), such broad reviews require a substantial investment of time and resources, are difficult to keep up‐to‐date, and risk duplicating efforts when there are many reliable reviews that are more focused. An optimal approach that might be possible in the future would be to have a number of focused reviews using standard methods, such as EPOC reviews, together with an overview of those reviews.
Agreements and disagreements with other studies or reviews
We identified 10 related overviews of reviews published in the last 10 years (Althabe 2008; Bloom 2005; Boaz 2011; Chan 2017; Cheung 2012; Hall 2015; Lewin 2008; Mostofian 2015; Squires 2014; Wensing 2006). These overviews addressed a range of implementation strategies, disease conditions and behaviours in diverse settings and populations. Similarly to our overview, most of the studies included in those overviews were from high‐income countries, and data on patient outcomes, equity, costs and cost‐effectiveness were scarce. We describe the findings of the seven overviews below.
Bloom 2005 included 26 reviews and assessed the effectiveness of continuing medical education (CME) tools and techniques for changing physician clinical practices and improving patient health outcomes. The findings showed that interactive techniques (audit/feedback, academic detailing/outreach and reminders) were most effective for simultaneously changing physician care and patient outcomes. Clinical practice guidelines and opinion leaders were found to be less effective. Didactic presentations and distribution of printed information had little or no beneficial effect for changing physician practice. These findings mostly agree with ours regarding the effects of strategies targeted at healthcare workers by type of intervention (with the exception of the possible effects of printed materials found in our overview).
Wensing 2006 included 36 reviews and assessed organisational strategies (defined as planned re‐arrangements of one or more aspects of the organisation of patient care) to implement improvements in patient care. The findings showed that revision of professional roles and computer systems for knowledge management generally improved professional performance. Multidisciplinary teams, integrated care services and computer systems generally improved patient outcomes. Integrated care services led to cost savings. The benefits of quality management remained uncertain. Most of these findings agree with those from our overview.
Lewin 2008 examined the effectiveness of health systems arrangements and implementation strategies (with a particular focus on evidence relevant to primary healthcare). Five included reviews assessed strategies to change professional behaviours or performance. The strategies assessed were guideline dissemination, audit and feedback, educational outreach visits, and educational meetings. These interventions resulted in small to moderate (but important) improvements in professional performance and health outcomes. These findings are mostly similar to those reported in our overview as our findings are based on the same or more recent versions of the same systematic reviews (Baker 2015; Forsetlund 2009; Ivers 2012; O'Brien 2007). The only difference was our exclusion of the review on guideline dissemination strategies because it was published more than 10 years ago (Grimshaw 2004).
Althabe 2008 identified 23 reviews and assessed strategies for improving the quality of healthcare in maternal and child health in low‐ and middle‐income countries. Seventeen of the reviews focused on continuing education and quality improvement, two addressed financial and reimbursement strategies, and four examined organisation of care strategies. Some of these reviews were included in this overview. The overview found that interactive workshops, reminders, multifaceted interventions, audit and feedback, and mass media interventions had small to moderate positive effects on professional practice. Educational outreach visits improved prescribing but had variable effects on other behaviours. Multifaceted interventions were not more effective than single interventions. There was little evidence on the use of financial, regulatory or organisational interventions. Although these authors used a slightly different classification to the one used in this overview, the findings were relatively similar, with only minor changes related to updated versions of some reviews.
Boaz 2011 assessed the effectiveness of interventions to increase the use of research in clinical practice. It identified 13 reviews containing 313 primary studies. The overview found that multifaceted interventions are more likely to improve practice (small to moderate effects) than single interventions (audit and feedback, computerised decision support, opinion leaders) (small effects). These findings disagree with what we found (see results for multifaceted interventions) and with another recent overview that did not find evidence supporting the effectiveness of multifaceted interventions over single interventions (Squires 2014). Disagreements in the conclusions of these overviews could be due to differences in the evidence base examined, that is, differences in the reliability of included reviews and the included interventions.
Cheung 2012 assessed the effectiveness of reminders in changing professional behaviour in clinical settings. It included 35 systematic reviews and found that moderate improvements in provider behaviour can occur with the use of reminders. Most of these reviews assessed computerised reminders, which were excluded from our overview because we considered them not to be highly relevant to low‐income country settings.
Squires 2014 evaluated the effectiveness of multifaceted interventions versus single‐component interventions in changing healthcare professionals' behaviour in clinical settings. Twenty‐five reviews were included: five reviews found no evidence of a relationship between the number of intervention components and the effect size. Eight reviews reported direct (non‐statistical) comparisons of multifaceted to single‐component interventions. Multifaceted interventions were found to be effective in four reviews, and the remaining four reviews found that multifaceted interventions had either mixed effects or were generally ineffective compared to single interventions. Twenty‐three reviews indirectly compared the effectiveness of multifaceted to single interventions, and most (15 reviews) showed similar effectiveness for multifaceted and single interventions when compared to controls. Of the remaining eight reviews, six found single interventions to be generally effective while multifaceted interventions had mixed effectiveness. The authors concluded that the overview provides no evidence that multifaceted interventions are more effective than single‐component interventions. As noted above, these findings agree with what we found in our overview for at least three strategies targeted at healthcare workers (audit and feedback, educational meetings, and outreach visits).
Mostofian 2015 assessed the effectiveness of interventions aimed at changing physician practice patterns through the implementation of clinical research findings and clinical guidelines in surgical settings and general practice. They identified 14 reviews covering a wide range of interventions: audit and feedback, computerised decision support systems, continuing medical education, financial incentives, local opinion leaders, marketing, passive dissemination of information, patient‐mediated interventions, reminders, and multifaceted interventions. Active approaches, such as academic detailing, led to greater effects than traditional passive approaches, such as distribution of educational materials. These findings mostly agree with our results about the effects of strategies targeted at healthcare workers by type of intervention (with the exception of the possible effects of printed materials found in our overview) and are in agreement with the findings of Bloom 2005. The differences regarding 'passive approaches' could be related to the methods used to analyse the results, which were based to some extent on vote counting and on descriptions by the authors of the reviews (Mostofian 2015).
Hall 2015 evaluated the effectiveness of text‐messaging interventions (TMI) on health outcomes and behaviour change in community settings. They identified 15 reviews including 228 studies with diverse study designs (randomised trials, quasi‐experimental designs and observational studies) and assessing interventions for a variety of health‐behaviour topics related to health promotion, disease prevention and chronic disease self‐management. Six of the included reviews conducted or included meta‐analyses. According to the authors almost all of the 15 reviews, extremely diverse intervention characteristics showed positive effects on diverse health behaviours. Likewise, all of the meta‐analyses assessed concluded that TMIs had a statistically significant positive effect on health outcomes or health behaviours. These findings are in agreement with ours regarding TMI for adherence to antiretroviral therapy and anti‐tuberculosis treatments (Liu 2014; Mbuagbaw 2013). However, readers should interpret the findings of the overview by Hall and colleagues cautiously because the included studies were at high risk of bias and had small sample sizes and short intervention durations.
Chan 2017 assessed the effectiveness of strategies to enhance the adoption and implementation of clinical practice guidelines focusing on four interventions: reminders, educational outreach visits, audit and feedback, and provider incentives. They included 55 studies, 39 systematic reviews, and 16 overviews of reviews. Using vote counting, the authors found that audit and feedback and educational outreach visits were generally effective in improving both process of care and clinical outcomes; provider incentives showed mixed effectiveness for improving both processes of care and clinical outcomes; and reminders showed mixed effectiveness for improving process of care outcomes and were generally ineffective for clinical outcomes. Despite differences in the analytical approaches, those findings are similar to what we found in our overview regarding the effectiveness of audit and feedback and educational outreach visits (Ivers 2012; O'Brien 2007). On the other hand, our findings regarding the effects of reminders on patient outcomes are more positive (Ko 2011), possibly due to differences in the settings where reminders were evaluated.
Authors' conclusions
Investigators have evaluated a wide range of strategies for implementing evidence‐based interventions in low‐income countries using sound systematic review methods. These strategies have been targeted at different levels in the health systems and have addressed a range of outcomes. Most of the available evidence is focused on strategies targeted at healthcare workers and recipients, and assessment of process of care outcomes. Strategies targeting healthcare organisations are scarce. Limitations of the available evidence include wide variations in settings, targeted behaviours, estimates of effects, and certainty of the evidence for the implementation strategies examined.
Implications for practice.
We found moderate‐ or high‐certainty evidence that a number of interventions had desirable effects on at least one outcome, with no moderate‐ or high‐certainty evidence of undesirable effects (Table 8). On the other hand, the certainty of the evidence was low or very low (or no studies were found) for all outcomes examined for a number of interventions. These findings may help to distinguish interventions for which there is clear evidence of impact on at least one outcome from interventions for which there is important uncertainty about their potential effects. Additionally we identified a "middle ground" with interventions that showed little or no effect for at least one outcome with moderate or high‐certainty evidence and no moderate‐ or high‐certainty evidence of other desirable or undesirable effects. This implies that those interventions could not have a relevant impact on the outcomes assessed and they should not be considered a priority for implementation.
Implications for research.
Based on the included reviews, we have identified gaps in primary research because of uncertainty about the applicability of the evidence to low‐income countries (Table 9), low‐certainty evidence, or a lack of studies (Table 8). In 18 out of the 39 included reviews, all or most of the studies took place in high‐income settings; and in 25 of the 39 reviews there was at least one comparison where the certainty about the effects was low or there were no studies to inform an estimate of the intervention's effects. Furthermore, the included reviews rarely evaluated social outcomes, resource use, impacts on equity, and adverse (undesirable or unintended) effects (Table 7). All of the included reviews found that primary research is needed for at least one of these four types of outcomes (Table 7). This highlights the need for conducting studies assessing specific comparisons (Table 9) in low‐income settings and including all relevant outcomes.
9. Priorities for primary research.
| Implementation strategy | Systematic review | Applicability limitations | ||
| Findings | Interpretation | |||
| Strategies targeted at healthcare workers by type of intervention | ||||
| Educational materials | Giguère 2012 | The studies reviewed were mostly from high‐income countries. Only one study out of the 45 included was from a middle‐income country | In low‐income countries (LICs) where health systems may be weaker, it may be difficult to ensure that up‐to‐date printed educational materials reach the appropriate providers promptly. It is therefore unclear whether similar effects would be expected in LICs | |
| Interprofessional education (IPE) | Reeves 2013 | Included studies were done primarily in the USA and UK in varied settings (hospital emergency departments, health maintenance organisations, community mental health provider organisations, primary care practices) | The impact of IPE interventions in low‐income settings is uncertain (differences in the health system contexts, gender relationships and comparable social status of different health professions may influence the effectiveness of IPE in different settings) | |
| Teaching critical appraisal skills in healthcare settings | Horsley 2011 | None of the included studies took place in a low‐income country. Interventions assessed included journal club supported by a half‐day workshop, critical appraisal materials, listserv discussions and articles and a half‐day Critical Appraisal Skills Programme (CASP) workshop | The impact of critical appraisal educational interventions (workshops, materials) in low‐income countries is uncertain | |
| Educational outreach | Baskerville 2012 | The review did not include any studies conducted in low‐income countries that evaluated the use of practice facilitation to promote adoption of evidence‐based guidelines | Practice facilitation might be difficult to implement in low‐resource settings, particularly the audit and feedback component, and it might be more difficult to make necessary organisational changes | |
| Local opinion leaders | Flodgren 2011 | The included trials were from primary care in the USA (6 studies) and Canada (1 study) | Rigorous studies from low‐income countries are needed to fully assess applicability of the findings to all healthcare settings | |
| Patient‐mediated interventions (i.e. new clinical information collected directly from patients and given to the provider) | No updated review identified | — | — | |
| Reminders | No updated review identified | — | — | |
| Tailored interventions | Baker 2015 | The studies were undertaken in the USA, the UK, Belgium , Canada, Indonesia , Norway, the Netherlands and Portugal | The impact of tailored interventions in low‐income countries is uncertain | |
| Interventions encouraging use of systematic reviews in clinical decision‐making | Perrier 2011 | All studies included in the review took place in middle and high income countries. | The applicability of these findings to low‐income countries is uncertain. Important issues in adopting of such interventions include acceptance by the end user and integration into the healthcare system | |
| Strategies targeted at healthcare workers by type of problem | ||||
| Communication with patients – Training healthcare providers to be more 'patient‐centred' in clinical consultations | Dwamena 2012 | This review did not find any study conducted in low‐ and‐middle‐income countries, all 43 studies included were from high‐income countries | Although patient‐centredness may be an objective of care in many settings, it is not possible to be confident about the applicability of the reported interventions to low‐income countries (LICs) and to settings other than primary care, as almost all studies took place in the USA and Europe. Human resource constrains in some health systems and low motivation to deliver patient‐centred care may limit the feasibility and potential of this approach for improving professional practice and health outcomes. |
|
| Obstetrics – Non‐clinical interventions for reducing unnecessary caesarean section | Khunpradit 2011 | The included studies took place in high‐income and middle‐income countries | The evidence for the effectiveness of these interventions in low‐income settings is limited; observed effects may be different in low‐income settings given differences in contextual factors (e.g. socioeconomic factors)‐ | |
| Strategies targeted at healthcare recipients | ||||
|
Providing information/education – Mass media interventions for promoting HIV testing |
Vidanapathirana 2005 | All the studies took place in high‐income countries | The degree of diffusion and importance of multimedia (television, radio, etc.) in low‐income countries should be considered in order to reach more population with the messages. | |
| Providing information/education – Written information about individual medicines for consumers | Nicolson 2009 | All the trials – except one conducted in Turkey – were carried out in high‐income countries | Implementation of written medicine information (WMI) depends on the health systems' regulatory context. For instance in high‐income countries there are specific laws that already governed the use of WMI. Hence its implementation and impact in low‐income countries could be different to the findings from this review. The effects of WMI are mediated by the ability to read the information presented. Hence low literacy levels in a country could make these findings less applicable. |
|
| Providing information/education –Health literacy interventions and outcomes | Berkman 2011 | Most studies took place in high‐income countries. | There is insufficient evidence of the effectiveness of these interventions in low‐income countries. The effects observed in these studies were limited to clinical environments and a narrow geographical area which may be different in low‐income settings given that they have different literacy levels and other contextual factors (e.g. sociodemographic factors). | |
| Adherence – medication | Haynes 2008 | Most studies took place in high‐income countries. | Almost all the interventions that were effective were complex and included combinations of interventions. Even the most effective interventions did not lead to large improvements in treatment outcomes. The findings indicate that interventions to improve medication adherence should be used with caution given that there is a high degree of uncertainty about both their effects and costs. |
|
| Adherence – ART for HIV/AIDS | Simoni 2006 | All studies took place in high‐income countries. The update of the review includes 3 studies in Brazil, China and Mozambique, with mixed results. We are very uncertain if there is a different effect in those settings. | Some studies use combinations of interventions, so it is difficult to know which component is leading to the observed effects. There are countless aspects where interventions to improve adherence in HIC and LMIC may differ. |
|
|
Adherence – TB – Material incentives and enablers in the management of tuberculosis |
Lutge 2015 | Most studies took place in the USA; most studies were conducted in population subgroups |
The findings need to be applied with caution in low‐ and middle‐income countries considering the structural and qualitative differences in health systems, healthcare provision, resources and healthcare‐seeking behaviour. The included studies focus on specific subgroups of patients, such as injection drug users. The findings may therefore not be applicable in the general population. |
|
| Access to healthcare services – Interventions targeted at women to encourage the uptake of cervical screening | Everett 2011 | All except one of the studies included in the systematic review took place in high‐income countries. One study took place in a middle‐income country | When assessing the transferability of these findings to low‐income countries the following factors should be considered.
|
|
We only have included priorities for research on the effects of implementation strategies based on the findings of the included systematic reviews.
We identified five topics for which we could not identify an up‐to‐date systematic review but did find a review in progress (Table 10). We also identified one topic for which we did not find a up‐to‐date systematic review or one in progress: strategies other than handwashing targeted at healthcare‐associated infections.
10. Priorities for systematic reviews.
| Implementation strategy | What we found |
| Strategies targeted at healthcare organisations | |
| Continuous quality improvement | Review in progress (Brennan 2009) |
| Strategies targeted at healthcare workers by type of intervention | |
| Patient‐mediated interventions | Review in progress (Fønhus 2016) |
| Reminders | Review in progress (Pantoja 2014b) |
| Strategies targeted at healthcare workers by type of problem | |
| Healthcare‐associated infections | No up‐to‐date review identified |
| Strategies targeted at healthcare recipients | |
| Facility‐based deliveries | Review in progress (Dudley 2009) |
Acknowledgements
We would like to thank the following editors and peer referees who provided comments to improve the overview: Sasha Shepperd (editor), Alex Rowe, Shaun Treweek, Michel Wensing and Meggan Harris for copy‐editing the overview.
We would also like to acknowledge the following colleagues who helped to produce the SUPPORT Summaries upon which this overview is based: Peter Steinmann, Andrea Darzi, Nour Hemadi, Michael Gathu, Mercy Mulaku, Dimelza Osorio, Jesse Uneke, Signe Flottorp and Jimmy Volmink.
We would also like to thank Susan Munabi‐Babigumira, Atle Fretheim, Simon Goudie and Hanna Bergman for editing some of the SUPPORT Summaries, as well as the review authors and others who have provided feedback on the SUPPORT Summaries.
Charles S Wiysonge's work is supported by the South African Medical Research Council and the National Research Foundation of South Africa (Grant Numbers: 106035 and 108571).
The Norwegian Satellite of the Effective Practice and Organisation of Care (EPOC) Group receives funding from the Norwegian Agency for Development Co‐operation (Norad), via the Norwegian Institute of Public Health to support review authors in the production of their reviews.
This overview is a product of the Effective Health Care Research Consortium, which provided funding to make this overview open access. The Consortium is funded by UK aid from the UK Government for the benefit of developing countries (Grant: 5242). The views expressed in this overview do not necessarily reflect UK government policy.
Appendices
Appendix 1. SUPPORT Summaries checklist for making judgments about how much confidence to place in a systematic review
| Review: | |
| Assessed by: | |
| Date: | |
| Section A: Methods used to identify, include and critically appraise studies | |
|
A.1 Were the criteria used for deciding which studies to include in the review reported? Did the authors specify: _ Types of studies _ Participants _ Intervention(s) _ Outcome(s) Coding guide ‐ check the answers above YES:All four should be yes |
_ Yes _ Can't tell/partially _ No |
| Comments (note important limitations or uncertainty) | |
|
A.2 Was the search for evidence reasonably comprehensive? Were the following done: _ Language bias avoided (no restriction of inclusion based on language) _ No restriction of inclusion based on publication status _ Relevant databases searched (including Medline + Cochrane Library) _ Reference lists in included articles checked _ Authors/experts contacted Coding guide ‐ check the answers above: YES:All five should be yes PARTIALLY:Relevant databases and reference lists are both ticked off |
_ Yes _ Can't tell/partially _ No |
| Comments (note important limitations or uncertainty) | |
|
A.3 Is the review reasonably up‐to‐date? Were the searches done recently enough that more recent research is unlikely to be found or to change the results of the review? Coding guide – consider how many years since the last search (e.g. if more than 10 years the review is unlikely to be up‐to‐date) and whether there is ongoing research |
_ Yes _ Can't tell/not sure _ No |
| Comments (note important limitations or uncertainty) | |
|
A.4 Was bias in the selection of articles avoided? Did the authors specify: _ Explicit selection criteria _ Independent screening of full text by at least 2 reviewers _ List of included studies provided _ List of excluded studies provided Coding guide ‐ check the above YES:All four should be yes |
_ Yes _ Can't tell/partially _ No |
| Comments (note important limitations or uncertainty) | |
|
A.5 Did the authors use appropriate criteria to assess the risk for bias in analysing the studies that are included?†( See Appendix for an example of criteria ‐ Assessing Risk of Bias Criteria for EPOC Reviews) _ The criteria used for assessing the risk of bias were reported _ A table or summary of the assessment of each included study for each criterion was reported _ Sensible criteria were used that focus on the risk of bias (and not other qualities of the studies, such as precision or applicability) Coding guide ‐ check the above YES:All four should be yes |
_ Yes _ Can't tell/partially _ No |
| Comments (note important limitations or uncertainty) | |
|
A.6 Overall – how would you rate the methods used to identify, include and critically appraise studies? Summary assessment score A relates to the 5 questions above. If the "No"or "Partial"option is used for any of the questions above, the review is likely to have important limitations. Examples of major limitations might include not reporting explicit selection criteria, not providing a list of included studies or not assessing the risk of bias in included studies. |
_ Major limitations (limitations that are important enough that the results of the review are not reliable and they should not be used in the policy brief) _ Important limitations (limitations that are important enough that it would be worthwhile to search for another systematic review and to interpret the results of this review cautiously, if a better review cannot be found) _ Reliable (only minor limitations) |
| Comments (note any major limitations or important limitations). | |
| Section B: Methods used to analyse the findings | |
|
B.1 Were the characteristics and results of the included studies reliably reported? Was there: _ Independent data extraction by at least 2 reviewers _ A table or summary of the characteristics of the participants, interventions and outcomes for the included studies _ A table or summary of the results of the included studies. Coding guide ‐ check the answers above YES:All three should be yes |
_ Yes _ Partially _ No _ Not applicable (e.g. no included studies) |
| Comments (note important limitations or uncertainty) | |
| B.2 Were the methods used by the review authors to analyse the findings of the included studies reported? | _ Yes _ Partially _ No _ Not applicable (e.g. no studies or no data) |
| Comments (note important limitations or uncertainty) | |
|
B.3 Did the review describe the extent of heterogeneity? _ Did the review ensure that included studies were similar enough that it made sense to combine them, sensibly divide the included studies into homogeneous groups, or sensibly conclude that it did not make sense to combine or group the included studies? _ Did the review discuss the extent to which there were important differences in the results of the included studies? _ If a meta‐analysis was done, was the I2, chi square test for heterogeneity or other appropriate statistic reported? |
_ Yes _ Can't tell/partially _ No _ Not applicable (e.g. no studies or no data) |
| Comments (note important limitations or uncertainty) | |
|
B.4 Were the findings of the relevant studies combined (or not combined) appropriately relative to the primary question the review addresses and the available data? How was the data analysis done? _ Descriptive only _ Vote counting based on direction of effect _ Vote counting based on statistical significance _ Description of range of effect sizes _ Meta‐analysis _ Meta‐regression _ Other: specify _ Not applicable (e.g. no studies or no data) How were the studies weighted in the analysis? _ Equal weights (this is what is done when vote counting is used) _ By quality or study design (this is rarely done) _ Inverse variance (this is what is typically done in a meta‐analysis) _ Number of participants _ Other, specify: _ Not clear _ Not applicable (e.g. no studies or no data) Did the review address unit of analysis errors? _ Yes ‐ took clustering into account in the analysis (e.g. used intra‐cluster correlation coefficient) _ No, but acknowledged problem of unit of analysis errors _ No mention of issue _ Not applicable ‐ no clustered trials or studies included Coding guide ‐ check the answers above If narrative OR vote counting (where quantitative analyses would have been possible) OR inappropriate table, graph or meta‐analyses OR unit of analyses errors not addressed (and should have been) the answer is likely NO. If appropriate table, graph or meta‐analysis AND appropriate weights AND the extent of heterogeneity was taken into account, the answer is likely YES. If no studies/no data:NOT APPLICABLE If unsure:CAN'T TELL/PARTIALLY |
_ Yes _ Can't tell/partially _ No _ Not applicable (e.g. no studies or no data) |
| Comments (note important limitations or uncertainty) | |
|
B.5 Did the review examine the extent to which specific factors might explain differences in the results of the included studies? _ Were factors that the review authors considered as likely explanatory factors clearly described? _ Was a sensible method used to explore the extent to which key factors explained heterogeneity? _ Descriptive/textual _ Graphical _ Meta‐regression _ Other |
_ Yes _ Can't tell/partially _ No _ Not applicable (e.g. too few studies, no important differences in the results of the included studies, or the included studies were so dissimilar that it would not make sense to explore heterogeneity of the results) |
| Comments (note important limitations or uncertainty) | |
|
B.6 Overall ‐ how would you rate the methods used to analyse the findings relative to the primary question addressed in the review? Summary assessment score B relates to the 5 questions in this section, regarding the analysis. If the "No"or "Partial"option is used for any of the 5 preceding questions, the review is likely to have important limitations. Examples of major limitations might include not reporting critical characteristics of the included studies or not reporting the results of the included studies. |
_ Major limitations (limitations that are important enough that the results of the review are not reliable and they should not be used in the policy brief) _ Important limitations (limitations that are important enough that it would be worthwhile to search for another systematic review and to interpret the results of this review cautiously, if a better review cannot be found) _ Reliable (only minor limitations) |
| Use comments to specify if relevant, to flag uncertainty or need for discussion | |
| Section C: Overall assessment of the reliability of the review | |
| C.1 Are there any other aspects of the review not mentioned before which lead you to question the results? | _ Additional methodological concerns _ Robustness _ Interpretation _ Conflicts of interest (of the review authors or for included studies) _ Other _ No other quality issues identified |
|
C.2 Based on the above assessments of the methods how would you rate the reliability of the review? _ Major limitations (exclude); briefly (and politely) state the reasons for excluding the review by completing the following sentence: This review was not included in this policy brief for the following reasons: Comments (briefly summarise any key messages or useful information that can be drawn from the review for policy makers or managers): _ Important limitations ; briefly (and politely) state the most important limitations by editing the following sentence, if needed, and specifying what the important limitations are: This review has important limitations. _ Reliable ; briefly note any comments that should be noted regarding the reliability of this review by editing the following sentence, if needed: This is a good quality systematic review with only minor limitations. | |
Appendix 2. Search strategies
PubMed
From 2000 to present. Update:weekly
#1. MEDLINE[Title/Abstract]
#2. (systematic[Title/Abstract] AND review[Title/Abstract])
#3. meta analysis[Publication Type]
#4. #1 OR #2 OR #3 (Methods filter for systematic reviews–Clinical Queries–Max Specificity)
#5. overview[Title] AND (reviews[Title] OR systematic[Title]
#6. meta‐review[Title]
#7. review of reviews[Title]
#8. review[Title] AND systematic reviews[Title]
#9. umbrella[Title] AND (review[Title] OR reviews[Title] OR systematic[Title])
#10. policy[Title] AND (brief[Title] OR evidence[Title])
#11. #5 OR #6 OR #7 OR #8 OR #9 OR #10 (Methods filter for overviews)
#12. #4 OR #11 (Methods filter for systematic reviews and for overviews)
LILACS
From 2000 to present. Update: monthly
(TW:"revision sistematica" OR TW:"revisao sistematica" OR TW:"systematic review" OR MH:"review literature as topic" OR MH:"meta‐analysis as topic" OR PT:"meta‐analysis")
OR
(PT:revision AND (TW:metaanal$ OR TW:"meta‐analysis" OR TW:"metaanalise" OR TW:"meta‐analisis" OR TI:overview$ OR TW:"estudio sistematico" OR TW:"systematic study" OR TW:"estudo sistematico" OR TI:review OR TI:revisao OR TI:revision OR TI:systematic OR TI:sistematico))
OR
((TW:overview OR TW:"estudio sistematico" OR TW:"systematic study" OR TW:"estudo sistematico") AND (TI:review OR TI:revisao OR TI:revision OR TI:systematic OR TI:sistematico))
CINAHL (EBSCO)
From 2000 to present. Update: monthly
((TI meta analys* or AB meta analys*) or (TI systematic review or AB systematic review))
PsycINFO (EBSCO)
From 2000 to present. Update: monthly
meta‐analysis OR search*
EMBASE (Ovid)
From 2000 to present. Update: monthly
meta‐analysis.tw. OR systematic review.tw
Appendix 3. Characteristics of included reviews
| Strategies targeted at healthcareorganisations | ||
| Organisational culture | ||
| Parmelli 2011 | ||
| Review objective: to determine the effectiveness of strategies to change organisational culture in order to improve healthcare performance and considering – when possible – different patterns of organisational culture. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Any strategy intended to change organisational culture in order to improve healthcare performance. The comparator could be normal care or any other active intervention. The following study designs were considered: randomised trials, non‐randomised trials, controlled before‐after studies and interrupted time series studies | No available studies eligible for inclusion in this review |
| Participants | Healthcare organisations applying strategies to change organisational culture | No available studies eligible for inclusion in this review |
| Settings | Any type of healthcare organisation in any country | No available studies eligible for inclusion in this review |
| Outcomes | Professional performance such as prescription rates, evidence‐based practice, quality of care, and efficiency. Patient outcomes such as mortality, condition‐specific measures of outcomes, quality of life, functional health status, and patient satisfaction. Organisational performance indicators such as wait times, inpatient hospital stay times, and staff turnover rates | No available studies eligible for inclusion in this review |
| Date of most recent search: October 2009 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. However, it did not identify any studies that met the inclusion criteria. | ||
| Strategies targeted at healthcare workers by type of interventions | ||
| Educational materials | ||
| Giguère 2012 | ||
| Review objective: to determine the effects of printed educational materials (PEMs) in improving professional practice and patient outcomes | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials, non‐randomised trials, controlled before‐after studies, and interrupted time series studies assessing the effects of printed educational materials, such as clinical practice guidelines, journals, and monographs, delivered personally, through mass mailing or passively via wider channels such as the Internet or mass media | 45 studies: 8 cluster‐randomised trials, 6 randomised trials, and 31 interrupted time series studies. Most studies (36/45) evaluated a single PEM. 2 studies evaluated simultaneously several PEMs (respectively 12 and 11 distinct PEMs) that presented similar characteristics; and 3 interrupted time series studies assessed more than 2 PEMs with very similar characteristics. The 45 studies included the following PEMS: 23 journal publications, 16 evidence‐based guidelines, 6 newsletters, 3 summaries of clinical guidelines and 1 clinical article reprint |
| Participants | Any type of healthcare professionals | Physicians, psychologists, psychiatrists, nurses, critical care fellows, Masters‐level therapists, and allied health professionals in the field of community health |
| Settings | Studies originating from any setting | Country: Canada (12 studies), USASA (11 studies), UK (11 studies), Spain (1 study), Belgium (1 study), The Netherlands (2 studies), Finland (1 study), Ireland (1 study), Germany (1 study), Italy (1 study), Japan (2 studies), Brazil (1 study), USA and Canada (1 study) Healthcare setting: general family or community‐based practice (10 studies), outpatient (ambulatory) settings (9 studies), hospitals (6 studies), mixed settings (3 studies), municipal health centre (1 study), managed behavioural healthcare organisation (1 study), clinical setting unclear (15 studies) |
| Outcomes | Any objective measure either of professional practice (e.g. the number of tests ordered, prescriptions for a particular drug) or of patient health outcomes (e.g. blood pressure, complications after surgery) | Prescribing/treatment (39 studies); financial (resource use) (2 studies); general management of a problem (8 studies); diagnosis (4 studies); procedures (7 studies); referrals (4 studies); test ordering (5 studies); surgery (5 studies); patient education/advice (4 studies); clinical prevention service (3 studies); screening (2 studies); reporting (1 study); discharge planning (2 studies); patient health outcome (4 studies) |
| Date of most recent search: June 2011 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Internet‐based education | ||
| Cook 2008 | ||
| Review objective: to assess the effects of internet‐based learning for health professionals. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Internet‐based learning for health professionals at any stage of training or practice | 201 studies (including observational and experimental designs) of Internet‐based learning for health professionals, addressing a wide range of topics, and using a range of modalities for teaching and learning |
| Participants | Health profession learners (including students and practising physicians, nurses, dentists, pharmacists and others) | Health profession learners |
| Settings | All settings and languages | All settings |
| Outcomes | Satisfaction; learning (knowledge, attitudes, skills); behaviours or effects on patients | Knowledge, skills, behaviours and effects on patients, satisfaction |
| Date of most recent search: January 2008 | ||
| Limitations: the review is from 2008 and the studies up to 2007. New information is likely to be available. | ||
| Educational meetings | ||
| Forsetlund 2009 | ||
| Review objective: to address the following questions: Do educational meetings and workshops improve professional practice and healthcare outcomes? What are the effects of educational meetings compared with the effects of other interventions? and Can changes in how educational meetings are done increase the effects? | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | The following types of educational meetings: conferences, lectures, workshops, seminars, symposia and courses. Only randomised trials were included | 81 trials (74 cluster‐randomised trials, 7 randomised by providers). Targeted behaviours were preventive care (11 studies), test ordering (3 studies), screening (6 studies), prescribing (13 studies), general management of a wide array of problems (41) and other (7 studies). The interventions were multifaceted in 32 studies. |
| Participants | Studies involving qualified health professionals or health professionals in post‐graduate training were included. Studies involving only undergraduate students were excluded. | The health professionals were physicians in most trials, nurses (2 studies), pharmacists (3 studies), prescribers (1 study), or mixed providers (18 studies). |
| Settings | All healthcare settings (primary care and hospital care) | General practice (43 studies), community‐based care (16 studies), hospital‐based care (17 studies) and 'other type of settings' (5 studies). Studies were from USA (28 studies), UK (14 studies), Netherlands (10 studies), Canada (4 studies), Australia (3 studies), Norway (3 studies), France (2 studies), Indonesia (2 studies), South‐Africa (2 studies); Sweden, Denmark, Belgium, Spain, Scotland, Mali, Thailand, Peru, Mexico, Zambia, Sri Lanka, New Zealand and Brazil (1 study each) |
| Outcomes | All objectively measured health professional practice behaviours or patient outcomes | There was wide variation in the outcome measures and number of outcomes measured. Median follow‐up was 6 months (range 14 days to 2 years) |
| Date of most recent search: March 2006 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Educational meetings | ||
| Reeves 2013 | ||
| Review objective: to assess the effects of interprofessional education on professional practice and healthcare outcomes | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials, controlled before‐after studies and interrupted time series studies of interprofessional education interventions. These included all types of educational, training, learning or teaching initiatives involving more than one profession in joint, interactive learning with the explicit purpose of improving interprofessional collaboration or the health and well‐being of patients | 15 studies: randomised trials (8 studies), controlled before‐after studies (5 studies) and interrupted time series studies (2 studies). The interprofessional education interventions assessed were varied, and included (among others): communication skills training, teamwork and team planning interventions, and behaviour change training (interactive workshops). |
| Participants | Health and social care professionals | A range of health and social care professionals including (among others): physicians, nurses, nutritionists, optometrists, social workers, physician assistants, psychiatrists, mental health workers, medicine residents, pharmacy students, obstetricians and anaesthetists |
| Settings | Any health or social care setting | Countries: USA (12 studies), UK (2 studies), Mexico (1 study) Healthcare settings: hospital emergency departments, community mental health provider organisations, primary care clinics, and a health maintenance organisation |
| Outcomes | Objectively measured or self‐reported patient/client outcomes, healthcare process outcomes | Patient outcomes, guideline adherence rates, patient satisfaction, clinical process outcomes, collaborative behaviour, medical error rates, professionals competencies |
| Date of most recent search: August 2011 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Educational meetings | ||
| Horsley 2011 | ||
| Review objective: to assess the effects of teaching critical appraisal skills to health professionals on the process of care, patient outcomes and knowledge of health professionals. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials, non‐randomised trials, controlled before‐after studies, and interrupted time series studies that examined the effectiveness of educational interventions teaching critical appraisal to health professionals | 3 randomised trials of: journal club supported by a half‐day workshop (1 study); critical appraisal materials ( a package including papers with methodological reviews), list‐serve discussions and articles (1 study); and a half‐day workshop based on a Critical Appraisal Skills Programme (CASP) (1 study) |
| Participants | Any qualified healthcare professionals (including managers and purchasers) with direct patient care. Studies involving students were excluded | Interns (1 study) and physicians (2 studies) |
| Settings | Any clinical setting | USA, UK and Canada |
| Outcomes | Process of care, patient mortality, morbidity, quality of life and satisfaction | Knowledge (2 studies) and critical appraisal skills (3 studies) |
| Date of most recent search: June 2011 | ||
| Limitations: this was generally a well‐conducted systematic review with only minor limitations. | ||
| Educational meetings | ||
| Sunguya 2013 | ||
| Review objective: to examine the effectiveness of nutrition training for health workers on child feeding practices among children aged 6 months to 2 years. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials and cluster‐randomised trials of interventions in which health workers received nutrition training targeting caregivers' feeding practices | 10 randomised trials and cluster‐randomised trials of interventions in which health workers received nutrition training targeting caregivers' feeding practices |
| Participants | Health workers including doctors; nurses and nurse midwives; midlevel providers including assistant medical officers, clinical officers, assistant nurses, or assistant physicians; community health workers including village health workers | Nutritionists (2 studies); doctors (1 study); health workers (6 studies); community health workers (2 studies); auxiliary nurse midwives (1 study) |
| Settings | Health facilities | Health facilities in low‐ and middle‐income countries: India (4 studies), Brazil (1 study), Peru (1 study), Pakistan (1 study), China (1 study), Bangladesh (1 study), Vietnam (1 study) |
| Outcomes | Feeding frequency measured in the number of times the child was fed in the previous 24 hours; energy intake in kilojoules (kJ) per day; and dietary diversity, defined as the variety of food items that was fed to a child | Feeding frequency (5 studies); energy intake (6 studies); dietary diversity: consumption of targeted food items (7 studies) |
| October 2012 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Educational outreach | ||
| O'Brien 2007 | ||
| Review objective: to assess the effects of educational outreach on health professional practice and patient outcomes | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials of educational outreach to healthcare professionals by trained people that may be from the same organisation, but not from the same practice site. The information given may include feedback about their performance. | 69 trials found |
| Participants | Healthcare professionals responsible for patient care | Primary care physicians or teams practising in community settings (53 studies), physicians in hospital settings (6 studies), nurses and nursing assistants (4 studies), pharmacists/owners and counter attendants (2 studies), dentists (1 study) |
| Settings | Any practice setting | Mostly primary and community healthcare settings. The studies were from the USA (23 studies), the UK (22 studies), Europe (14 studies), Australia (8 studies), Indonesia (2 studies) and Thailand (1 study) |
| Outcomes | Objectively measured professional performance in a healthcare setting or healthcare outcomes. Studies that only measured knowledge or performance in a test situation were excluded. | Most studies reported multiple effect measures and many did not specify a primary outcome. 28 studies (34 comparisons) contributed to the calculation of the median for the main comparison of professional performance. Educational outreach was compared to another type of intervention, usually audit and feedback, in 8 trials (12 comparisons). |
| Date of most recent search: March 2007 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Educational outreach | ||
| Baskerville 2012 | ||
| Review objective: to undertake a quantitative synthesis of the effect of practice facilitation on evidence‐based practice behaviour. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised and non‐randomised trials and prospective studies of individual practice facilitation | 23 studies of practice facilitation interventions (17 randomised trials, 3 cluster randomised trials, and 3 non‐randomised studies) |
| Participants | All healthcare providers in primary care practices | Studies included 1398 practices (697 allocated to facilitation intervention and 701 in the control group) |
| Settings | Primary care settings | Primary care practices in the USA (12 studies), the Netherlands (5 studies), Canada (3 studies), the UK (2 studies) and Australia (1 study) |
| Outcomes | Change in evidence‐based practice behaviour | Studies reported this outcome in varied ways, such as increased screening or management of different conditions and improvements in care provided |
| Date of most recent search: December 2010 | ||
| Limitations: this is a well‐conducted systematic review. However, the literature searches were restricted to English‐language studies. | ||
| Educational outreach – Pharmacist‐provided services | ||
| Pande 2013 | ||
| Review objective: to examine the effectiveness of services provided by pharmacists on patient outcomes and health service utilisation and costs in low‐ and middle‐income countries. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Any health or drug‐related, patient‐targeted service delivered by pharmacists (other than drug compounding and dispensing, and excluding other services such as the selling of cosmetics or other non‐pharmaceutical products) evaluated in a randomised trial, non‐randomised trial, controlled before‐after study, or interrupted time series study. | 12 randomised trials in middle‐income countries were included. 11 examined pharmacist interventions targeted at patients, and 1 evaluated a pharmacist intervention targeted at healthcare professionals. All the included studies compared pharmacist‐provided services with usual care. |
| Participants | Pharmacists (or pharmacies) delivering services in outpatient settings other than, or in addition to, drug compounding and dispensing | Practising pharmacists and research pharmacists |
| Settings | Outpatient settings | Sudan (1 study), India (2 studies), Egypt (1 study), Paraguay (1 study), Thailand (2 studies), Chile (2 studies), Bulgaria (2 studies), and South Africa (1 study) |
| Outcomes | Objective measurement of patient outcomes and process outcomes such as health service utilisation and costs | Process outcomes (4 studies), rate of hospitalisation (2 studies), number of visits to private clinics or outpatient clinics and emergency rooms in hospitals (1 study), medication costs of patients with chronic obstructive pulmonary disease and asthma (1 study), the number of visits to general practitioners (2 studies), clinical and humanistic outcomes (11 studies), patient outcomes (7 studies), asthma score (1 study) |
| Date of most recent search: March 2010 | ||
| Limitations: this is a well‐conducted systematic review with minor limitations. There were few evaluations of impact that allowed robust conclusions to be drawn, particularly as many of the studies did not take all the costs involved into account. | ||
| Local opinion leaders | ||
| Flodgren 2011 | ||
| Review objective: to assess the effectiveness of local opinion leaders in improving the behaviour of healthcare professionals and patient outcomes | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials | 18 randomised trials in which opinion leaders delivered educational initiatives to members of their own healthcare profession |
| Participants | Healthcare professionals in charge of patient care | Physicians (14 studies); nurses (2 studies); physicians, nurses and midwives (2 studies) |
| Settings | Any healthcare setting | Hospitals (14 studies), primary care practice (1 study), both primary and secondary care (1 study), and undefined healthcare settings (2 studies); in the USA of America (10 studies), Canada (6 studies), China (1 study), and Argentina and Uruguay (1 study) |
| Outcomes | Objective measures of professional performance and/or patient outcomes | General management of a clinical problem (all 18 randomised trials) |
| Date of most recent search: May 2009 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Audit and feedback | ||
| Ivers 2012 | ||
| Review objective: to assess the effects of audit and feedback on the practice of healthcare professionals and on patient outcomes. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials assessing the effects of audit and feedback. Interventions were only included if audit and feedback was a core or essential element. | 140 randomised trials were included. The interventions used were highly heterogeneous with respect to their content, format, timing and source. Targeted behaviours were prescribing (39 trials), managing patients with diabetes or cardiovascular diseases (34 studies), and test ordering (31 studies). The remaining trials varied widely in terms of health conditions and targeted behaviours. |
| Participants | Healthcare professionals responsible for patient care | In most of the trials the healthcare professionals were physicians. Other targeted providers included dentists (1 trial), nurses (3 studies), pharmacists (2 studies), and a mix of providers (14 studies). |
| Settings | Healthcare settings | USA (69 trials), Canada (11 studies), UK or Ireland (21 studies), Australia or New Zealand (10 studies), and elsewhere (29 studies). Only 5 studies took place in low‐ and middle income countries: Sudan (2 studies), Thailand (1 study), Laos (1 study), Argentina and Uruguay (1 study). 94 trials were in outpatient settings, 36 in inpatient settings, and the clinical setting was unclear in 10 trials. |
| Outcomes | Objectively measured provider performance or healthcare outcomes | There was large variation in outcome measures, and many trials reported multiple primary outcomes. Most trials measured professional practice, with some also reporting patient outcomes. |
| Date of most recent search: December 2010 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Reminders | ||
| Ko 2011 | ||
| Review objective: to assess if the use of safety checklists, compared to not using checklists, improves patient safety in acute hospital settings | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Comparative studies of paper‐based checklists, applied to hospitalised patients by medical care teams, compared to controls (care provided without checklists) | Before‐after studies (9 studies) that evaluated a wide variety of checklist designs and training on use of the checklists. |
| Participants | Medical care teams (a medical clinician or surgeon had to be included) | Medical teams |
| Settings | Acute hospital settings | Intensive care units (5 studies), emergency departments (2 studies), surgical units (1 study) and multidepartmental acute care settings (1 study) |
| Outcomes | Any patient‐relevant clinical outcome | Length of stay (3 studies), percentage of ventilator days on which patients received recommended care (1 study), time from admission until prescription of medical deep venous thrombosis prophylaxis (1 study), appropriate indications for use of an indwelling urinary tract catheter (1 study), complications during the postoperative period (1 study), patients receiving antibiotics within 8 hours of a diagnosis of pneumonia (1 study) |
| Date of most recent search: September 2009 | ||
| Limitations: only articles in English were included and the results of included studies were not described or analysed systematically. | ||
| Tailored interventions | ||
| Baker 2015 | ||
| Review objective: to assess the effectiveness of interventions tailored to address identified barriers to change on professional practice or patient outcomes. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials of interventions tailored to address prospectively identified barriers to change. Studies had to involve a comparison group that did not receive a tailored intervention or a comparison between an intervention that was targeted at both individual and social or organisational barriers, compared with an intervention targeted at only individual barriers. |
32 randomised trials. Interventions assessed were varied and included (among others): printed materials; educational outreach; clinical guidelines; audit and feedback; interactive workshops; teaching sessions/discussions of patients; facilitation/practice meetings; and individual/group academic detailing. |
| Participants | Healthcare professionals responsible for patient care | Primarily physicians (14 studies), mixed professional groups (8 studies), nurses (4 studies); pharmacists (2 studies), geriatric teams (1 study), gynaecology teams (1 study), and physicians (1 study) |
| Settings | Any setting | Primary care or community settings (17 studies), hospital settings (7 studies), nursing homes (3 studies), and 1 each in child health clinics, community pharmacies, a regional health system, and a Medicaid programme. The studies took place in the USA of America (USA) (12 studies), the Netherlands (5 studies), the UK (UK) (4 studies), Belgium (2 studies), Indonesia (2 studies), Norway (2 studies), South Africa (2 studies), and Canada (1 study), Ireland (1 study), and Portugal (1 study) |
| Outcomes | Objectively measured professional performance (excluding self‐reporting) or patient outcomes in a healthcare setting or both. | Change in prescribing behaviour (12 studies), management of a disease (including diagnosis, assessment and treatment) (11 studies), preventive care (6 studies), influenza vaccination (2 studies), reporting adverse drug reactions (1 study) |
| Date of most recent search: December 2014 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Multifaceted interventions | ||
| Perrier 2011 | ||
| Review objective: to assess the effectiveness of interventions for seeking, appraising, and applying evidence from systematic reviews in clinical decisions. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Quantitative studies using any intervention to encourage use of systematic reviews in clinical decision‐making | 5 randomised trials that examined strategies ranging from multifaceted to simple interventions |
| Participants | Healthcare practitioners of any specialty involved in providing patient care | Physicians (4 studies) – 1 each in Canada, Thailand, UK, and Uruguay; midwives (3 studies) – 1 each in Thailand, UK, and Uruguay; residents (1 study ‐ Uruguay); interns (1 study ‐ Thailand); students (1 study ‐ Thailand); dentists (1 study ‐ Scotland) |
| Settings | All settings | Primary care (1 study), hospitals (3 studies), dental practice (1 study) |
| Outcomes | Change in professional performance (prescribing patterns, use of diagnostic tests), health outcomes for patients (return visits, adverse events, length of stay, decrease in admissions), and measures of healthcare provider satisfaction, knowledge, or attitude | All 5 studies provided objective performance measures. Patient health outcome measures and measures of healthcare provider satisfaction were not reported in any study. |
| Date of most recent search: July 2009 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Strategies targeted at healthcareworkers by type of problem | ||
| Communication with patients | ||
| Dwamena 2012 | ||
| Review objective: to assess the effects of interventions for healthcare providers that aim to promote patient‐centred care (PCC) approaches in clinical consultations. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials of interventions for healthcare providers that promote PCC in clinical consultations | 43 randomised trials. All studies assessed interventions that included training related to a variety of PCC skills, using diverse teaching techniques and lengths of training. 20 of the 43 studies included additional interventions.
|
| Participants | Any types of healthcare providers, including those training to qualify as healthcare providers | Most of the studies included primary care physicians or nurses practicing in community or hospital outpatient settings. |
| Settings | Clinical consultations of any type | Community or outpatient settings in the USA (16 studies), UK (10 studies), Germany (3 studies), Switzerland (2 studies), Netherlands (2 studies), Spain (2 studies), Australia (2 studies), Canada (1 study), France (1 study), Holland (1 study), Norway (1 study), Israel (1 study) and Taiwan (1 study). There were no studies from low‐ and middle‐income countries. |
| Outcomes | Consultation processes, including the extent to which patient‐centred care was judged to be achieved in practice Patient satisfaction with care Patient healthcare behaviours, e.g. concordance with care plans and service utilisation Patient health status and well‐being: physiological measures (e.g. blood pressure); clinical assessments (e.g. wound healing); patient self‐reports of symptom resolution or quality of life; and patient self‐esteem |
Most of the studies assessed the impacts on consultation processes and many also evaluated the impact on patient satisfaction. Patient health behaviours were less frequently assessed and patient health status was evaluated quite frequently. |
| Date of most recent search: June 2010 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Handwashing | ||
| Gould 2010 | ||
| Review objective: to assess the effectiveness of strategies to improve hand hygiene compliance in patient care and their subsequent effects on healthcare‐associated infections. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Any single or multifaceted intervention intended to improve compliance with hand hygiene using aqueous solutions or alcohol‐based products | 1 randomised clinical trial assessing education about hand hygiene and universal precautions 2 interrupted time series studies of social marketing campaigns, 1 of which also analysed a campaign for substituting types of alcohol‐based hand rub either for another type or for soaps 1 controlled before‐after study that used a single teaching session |
| Participants | Healthcare workers (except operating theatre staff) | Healthcare workers |
| Settings | Any hospital or community setting | 4 studies: UK (general surgical wards), China (hospital), Switzerland (acute hospital) and Australia (3 acute units) |
| Outcomes | Rates of observed hand hygiene compliance (or proxies for compliance), and reduction in healthcare‐associated infection or colonisation rates | Frequency of hand washes, percentage of nurses washing hands, and use of hand hygiene products |
| Date of most recent search: November 2009 | ||
| Limitations: this is a well‐conducted systematic review. | ||
| Obstetrics | ||
| Khunpradit 2011 | ||
| Review objective: to determine the effectiveness and safety of non‐clinical interventions for reducing unnecessary caesarean section rates | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials, quasi‐experimental studies, non‐randomised trials, controlled before‐after studies, and interrupted time series studies that evaluated interventions targeting patients, interventions targeting healthcare providers; financial, organisational and regulatory interventions | 16 studies, including cluster‐randomised trials (5 studies), patient randomised trials (6 studies), and interrupted time series studies (5 studies) targetting patients (6 studies) and healthcare providers (10 studies), of which 2 were financial interventions and 3 were regulatory interventions. |
| Participants | Pregnant women and their families, healthcare providers who work with expectant mothers, communities and advocacy groups | Pregnant women (6 studies), physicians/obstetricians (6 studies), public health nurses (1 study), hospitals or departments (3 studies) |
| Settings | Healthcare settings in low‐, middle‐ and high‐income countries | North America (6 studies), Latin America (1 study), Taiwan (2 studies), Iran (2 studies), UK (1 study), Netherlands (1 study), Australia (1 study), Finland (2 studies) |
| Outcomes |
Primary outcomes: the rate of caesarean sections and the rate of unnecessary caesarean sections Other outcomes: maternal, fetal or neonatal complications, cost and financial benefits, patient and provider satisfaction |
Caesarean section rates (16 studies) and complications (11 studies) |
| Date of most recent search: March 2010 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations, but the last search for studies took place in 2010. | ||
| Obstetrics | ||
| Sibley 2012 | ||
| Review objective: to assess the effects of initial training or additional training for traditional birth attendants (TBAs) on TBA and maternal behaviours thought to mediate positive pregnancy outcomes, as well as on maternal, perinatal, and newborn mortality and morbidity | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised and quasi‐randomised trials (including cluster‐randomised trials) | 4 cluster‐randomised trials and 2 randomised trials |
| Participants | TBAs: a person who assists the mother during childbirth and who initially acquired her skills by delivering babies herself or through an apprenticeship to other TBAs. Mothers and neonates cared for by trained and untrained TBAs or those who are living in areas where such TBAs attend most births. |
The TBAs were poorly described in the included studies. They were mostly between 40 and 50 years of age, and had low levels of education. Marital and socioeconomic status was generally not reported. |
| Settings | Rural communities | Studies from rural communities in Bangladesh (2 studies), Guatemala (1 study), Malawi (1 study), Pakistan (1 study), and Zambia (1 study). 1 study took place in 5 countries (Democratic Republic of Congo, Guatemala, India, Pakistan, and Zambia). |
| Outcomes | TBA or maternal behaviours thought to mediate positive pregnancy outcomes; maternal mortality; perinatal and neonatal mortality | Maternal mortality, maternal morbidity, haemorrhage (antepartum, intrapartum, postpartum combined), puerperal sepsis, frequency of obstructed labour, referral to emergency obstetrical care, neonatal mortality, advice about immediate feeding of colostrum, exclusive breastfeeding |
| Date of most recent search: June 2012 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Obstetrics | ||
| Yakoob 2011 | ||
| Review objective: to determine the effect of provision of skilled birth attendance as well as basic and emergency obstetric care on stillbirths | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised and non‐randomised trials as well as observational studies evaluating the provision of skilled birth attendance and emergency obstetric care. | 21 studies: 13 for skilled birth attendance (10 before‐after or non‐randomised studies and 3 observational studies) and 9 historical or ecological studies for emergency obstetric care |
| Participants | Pregnant women and newborns | Most women were from rural areas, but some were also from suburbs and mixed areas. |
| Settings | Community based settings in any country | Most skilled birth attendance studies were from low‐ and middle‐income countries (Bangladesh, Bolivia, China, Guatemala, Indonesia, Malawi, Mexico, Mozambique, Nigeria, Papua New Guinea, Sudan, and Tanzania). 3 studies were from high‐income countries (Netherlands, Norway, and Sweden). |
| Outcomes | Stillbirths and perinatal mortality | 2 (uncontrolled) before‐after studies reported stillbirths and 4 reported perinatal mortality and were included in the primary analysis. |
| Date of most recent search: March 2010 | ||
| Limitations: this is reasonably well‐conducted systematic review with only minor limitations such as the incomplete reporting of included studies' characteristics. | ||
| Obstetrics | ||
| Hundley 2012 | ||
| Review objective: to assess the effects of birth kits on newborn and maternal outcomes. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | All available evidence, irrespective of study design. For the purpose of this review, a birth kit was defined as any disposable kit intended for routine use in the intrapartum period, specifically at the delivery of the baby. | 9 included studies reporting effects of intervention packages including births kits: randomised trial (1 study), non‐randomised trial (1 study), before‐after studies (2 studies) and cross‐sectional studies (5 studies) |
| Participants | Pregnant women in the intrapartum period | Pregnant women (median delivery at home 87%) |
| Settings | Home or health facility | Mostly rural areas from Nepal (2 studies), Egypt (2 studies), Pakistan (1 study), Kenya and Tanzania (1 study), Papua New Guinea (1 study), India (1 study), Tanzania (1 study) |
| Outcomes | Primary outcomes: newborn outcomes and maternal outcomes Secondary outcomes: process measures—clean birth practices |
Newborn outcomes (perinatal mortality, neonatal tetanus, neonatal sepsis, and omphalitis) and maternal outcomes (maternal mortality, puerperal sepsis) Process measures—clean birth practices (clean hands, birth surface, cord cutting, cord tie) |
| Date of most recent search: September 2009 | ||
| Limitations: this is well‐conducted systematic review with only minor limitations. | ||
| Prescribing antibiotics | ||
| Ranji 2008 | ||
| Review objective: to evaluate strategies to reduce unnecessary antibiotic prescribing in outpatient practice and to compare the effect of strategies targeting clinicians, patients and/or healthcare systems | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials, controlled before‐after studies, and interrupted time series studies that evaluate interventions to reduce unnecessary prescription of antibiotics for acute nonbacterial illnesses using one of the following interventions: clinician and patient education, audit and feedback, clinician reminders and decision support systems, financial and regulatory incentives and provision of delayed prescriptions | A total of 43 studies were included, reporting 55 trials: Randomised trials (22 studies); quasi‐randomised trials (3 studies), controlled before‐after studies (19 studies). 24 trials from 23 studies tested an intervention using at least 2 distinct quality improvement (QI) strategies such as clinician education combined with patient education. The remaining trials used a single QI strategy, most commonly clinician education or patient education alone. |
| Participants | Clinicians, patients, healthcare systems | Patients were adults and children with acute respiratory infection. Clinicians were mostly from primary care settings. |
| Settings | Outpatient settings | USA (17 studies), Canada (2 studies), Europe (12 studies), Australia and New Zealand (4 studies), Israel (1 study). 6 studies took place in low‐ or middle‐income countries (Cuba, Indonesia, Mexico, South Africa, Sri Lanka and Zambia). |
| Outcomes | Proportion of patients' visits at which an antibiotic was prescribed | Most of the studies reported changes in the proportion of visits at which patients were prescribed antibiotics. |
| Date of most recent search: March 2007 | ||
| Limitations: this is a well‐conducted systematic review, although there was no detailed report of risk of bias assessment for the included studies and the search was in 2007 | ||
| Seriously ill newborn care | ||
| Opiyo 2015 | ||
| Review objective: to investigate the effectiveness of in‐service training of health professionals on their management and care of seriously ill neonates or children in low‐income settings | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials, cluster‐randomised trials, non‐randomised trials, controlled before‐after studies, and interrupted time series studies of neonatal life support courses, paediatric life support courses, life support elements within the Integrated Management of Pregnancy and Childbirth, and other in‐service newborn and child health training courses aimed at the recognition and management of seriously ill children | 2 randomised trials: a 1‐day Newborn Resuscitation Training course and a 4‐day Essential Newborn Care Training course |
| Participants | Qualified healthcare professionals | Qualified healthcare professionals: doctors, nurses, and midwives |
| Settings | Healthcare delivery sites in low‐income countries | Delivery rooms in Kenya and Sri Lanka |
| Outcomes | Health professional performance outcomes (e.g. clinical assessment/diagnosis, recognition and management/referral of seriously ill newborn/child, prescribing practices) Participant outcomes (e.g. mortality, morbidity) Health resource utilisation (e.g. drug use, laboratory tests) Health services utilisation (e.g. length of hospital stay) Other markers of clinical performance (e.g. simulated health worker performance in practice settings) Training/implementation costs Impact on equity Adverse effects |
Proportion of adequate initial resuscitation steps Inappropriate and potentially harmful practices per resuscitation Mortality in all resuscitation episodes Preparedness for resuscitation |
| Date of most recent search: February 2015 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Quality of care for STD and HIV | ||
| Sorsdahl 2009 | ||
| Review objective: to evaluate the efficacy of interventions for educating traditional healers in the fundamentals of STDs and HIV medicine | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials, non‐randomised trials, and controlled before‐after studies of interventions aimed at educating traditional healers in STDs and HIV medicine. | 1 controlled before‐after study of a 3.5‐day training course for traditional healers focusing on HIV, STDs, TB, nutrition, family planning, motivation for risk behaviour reduction 1 non‐randomised trial of a 7‐day training course for traditional healers focusing on malnutrition, acute respiratory infection, diarrhoea, night blindness and HIV |
| Participants | Traditional healers in any setting | Traditional healers |
| Settings | Any setting in which traditional healers are practising | South Africa (1 study in 2 urban and 2 rural communities in KwaZulu‐Natal Province) and Nepal (I study in 10 village development committee areas in a remote, rural region). |
| Outcomes | Increase in knowledge about STDs and HIV medicine (HIV transmission, prevention, antiretroviral therapy) Changes in behaviour (HIV risk practices, referral for testing, condom distribution, safe sex advice and counselling, referral to allopathic medical services) |
Increase in knowledge about STDs and HIV (signs and symptoms, prevention) (2 studies) Change in behaviour (patient management practices, HIV/AIDS risk behaviours, referral practices) (2 studies) |
| Date of most recent search: November 2008 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Strategies targeted at healthcare recipients | ||
| Providing information/education ‐ mass media campaigns | ||
| Vidanapathirana 2005 | ||
| Review objective: to assess the effect of mass media interventions on the uptake of HIV testing | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials, non‐randomised trials, and interrupted time series studies assessing multimedia interventions or interventions using one type of media | 2 randomised trials, 3 non‐randomised trials, and 9 interrupted time series studies. Interventions included multimedia (9 studies), video (1 study), television (1 study), group education (1 study), and leaflets plus discussion with participants (2 studies). No study compared different types of media. |
| Participants | The public or specific target groups (such as sex workers or drug users), excluding healthcare providers | The studies targeted the public (8 studies), pregnant women (2 studies), men who have sex with men (1 study), blood transfusion recipients (1 study), and women (2 studies) |
| Settings | Not specified | Studies from the UK (7 studies), USA (3 studies), Australia (2 studies), Canada (1 study) and Israel (1 study) |
| Outcomes | Primary: rate of people tested for HIV Secondary: improvement in detecting HIV seropositivity |
All studies reported on uptake of HIV testing and 3 reported on HIV seropositivity |
| Date of most recent search: April 2004 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Providing information/education | ||
| Nicolson 2009 | ||
| Review objective: to assess the effects of providing written information about prescribed and over‐the‐counter medicines on patient outcomes | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials, non‐randomised trials, controlled before‐after and interrupted time series studies in which the effects of written information were compared with a control group or alternative intervention | 25 randomised trials |
| Participants | Patients of any age receiving written information about a prescribed or over‐the‐counter medicine in any setting (hospital in‐ and outpatients and primary care) | 4788 participants were enrolled in the included trials. 19 studies involved patients with chronic conditions (using non‐steroidal anti‐inflammatory drugs (NSAIDs) or cardiovascular medicines), 5 trials were focused on patients with acute conditions, and 1 on both. |
| Settings | Any setting | The trials took place in 9 countries: USA (8 trials), UK (8 studies), Belgium (2 studies), Canada (2 studies), Finland (1 study), France (1 study), Hong Kong (1 study), Switzerland (1 study) and Turkey (1 study). |
| Outcomes | Patient knowledge about the medicine, patients' attitudes towards taking the medicine, patients' medicine‐taking behaviour, and patients' health outcomes | Patients' knowledge: recall of information about the medicine, recall of side effects; patients' attitudes towards taking medicines; and patients' medicine‐taking behaviour |
| Date of most recent search: March 2007 | ||
| Limitations: this is a review with important limitations related to the assessment of the risk of bias of individual studies and the analysis of the sources of heterogeneity. Additionally it has not been updated since 2007. | ||
| Providing information/education | ||
| Berkman 2011 | ||
| Review objective: to assess the effects of health literacy interventions on health services utilisation and health outcomes. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials, cluster‐randomised trials, quasi‐experimental studies, cohort studies, before‐after studies, cross‐sectional studies All interventions specifically designed to mitigate the effects of low health literacy by improving the use of healthcare services or health outcomes in low health literacy or low numeracy individuals |
42 intervention studies, including randomised trials (27 studies), cluster‐randomised trials (2 studies), quasi‐experimental pre‐post studies (10 studies), and quasi experimental post only studies (3 studies) The interventions included: alternative document design (2 studies), alternative numerical presentations (3 studies), additive or alternative pictorial representations (8 studies), alternative media (4 studies), combinations of alternative readability and document design (7 studies), physician notification on patients' literacy status (1 study), intensive self‐management (3 studies), educational interventions (1 study), and intensive disease management programmes (2 studies). |
| Participants | People of all ages, including different ethnicities and cultural groups | People of all ages, whites, blacks, Hispanics, different ethnicities and cultural groups |
| Settings | All settings | Inpatient or outpatient settings in healthcare systems and institutions, various community‐based settings or homes. Only 1 of the studies took place in a low‐income country. |
| Outcomes | Use of healthcare services such as emergency room visits, office visits, hospitalisations and prevention Health outcomes such as disease, disease severity, quality of life and death. |
For single strategies (21 studies): physician use of effective communication (1 study), comprehension (14 studies), knowledge (3 studies), accuracy (3 studies), self‐efficacy (1 study), intent (1 study), choices (3 studies), adherence (1 study), health outcomes (1 study) For mixed strategies (21 studies): comprehension (1 study), knowledge (10 studies), self‐efficacy (8 studies), adherence (5 studies), use of healthcare (6 studies), and health outcomes (10 studies) |
| Date of most recent search: May 2010 | ||
| Limitations: this was a well‐conducted systematic review with only minor limitations. | ||
| Adherence ‐ medication | ||
| Haynes 2008 (the updated review Niewlaat 2014 identified 109 additional studies – further details reported in the main text of the overview) | ||
| Review objective: to summarise the effects of interventions to help patients follow prescriptions for medications. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials evaluating interventions to improve adherence with prescribed, self‐administered medications | 78 trials evaluating 93 diverse interventions |
| Participants | Patients who were prescribed medication for a medical disorder (including psychiatric), but not for addictions | Patients with several different chronic conditions including hypertension (12 studies), schizophrenia or acute psychosis (10 studies), asthma or chronic obstructive pulmonary disease (COPD) (11 studies), rheumatoid arthritis (2 studies), hyperlipidaemia (3 studies), depression (4 studies) and HIV (12 studies) |
| Settings | Any setting | Many different settings and venues were included. Trials took place in the USA (30 studies), UK (14 studies), Spain (5 studies), Canada (8 studies), Australia (3 studies), the Netherlands (3 studies), China (3 studies), France (2 studies), Mexico (1 study), Norway (1 study), Italy (1 study), Sweden (1 study), Ghana (1 study), Denmark (1 study), Republic of Ireland (1 study), United Arab Emirates (1 study), Switzerland (1 study) and Malaysia (1 study). |
| Outcomes | Medication adherence and patient outcomes | 9 studies on short‐term and 71 on long‐term treatments measuring adherence and patient outcomes |
| Date of most recent search: February 2007 | ||
| Limitations: this is a systematic review with moderate limitations related to how the results were synthesised. | ||
| Adherence – ART for HIV/AIDS | ||
| Simoni 2006; Simoni 2010 | ||
| Review objective: to summarise literature on the effects of patient support strategies and education for improving adherence to highly active antiretroviral therapy (HAART) in people living with HIV/AIDS. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials evaluating behavioural interventions to improve adherence to HAART | 19 randomised trials of diverse behavioural interventions: 1‐on‐1 counselling (10 trials), and group format (3 studies). Components of the intervention were: didactic information on HAART (15 studies); interactive discussions addressing cognitions, motivations, and expectations (15 studies); Behavioural strategies (16 studies), such as cue dosing or cognitive‐behaviour therapy; external reminders such as pagers (5 studies). Many interventions included more than one of these components. |
| Participants | Adults infected with HIV and receiving HAART | 7 studies restricted inclusion to patients exhibiting some marker of risk for non‐adherence, such as poor baseline adherence or detectable viral load. Participants in the USA studies were mostly racial/ethnic minorities. |
| Settings | Any setting | Most studies (16 studies) studies took place in outpatient HIV primary care clinics in high‐income countries: USA (14 studies), Spain (2 studies), France (2 studies) and Switzerland (1 study). |
| Outcomes | Any measure of adherence or HIV‐1 RNA viral load (a measure of successful HAART) | Adherence was measured in 18 studies. Data on undetectable viral load were available from 14 studies. |
| Date of most recent search: September 2005 in Simoni 2006, updated October 2009 in Simoni 2010 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Adherence – ART for HIV/AIDS | ||
| Mbuagbaw 2013 | ||
| Review objective: to determine whether mobile phone text‐messaging is efficacious in enhancing adherence to ART in people with HIV infection. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials in which patients receiving ART or their caregivers (for children) were provided with mobile phone text messages to promote adherence to ART | 3 randomised trials comparing text messaging to a control condition. In 2 studies, weekly text messages reminders were compared to standard care. In the other study, short or long text messages, either daily or weekly, were compared to the provision of a cell phone, but without study‐related communication. |
| Participants | Adults or children receiving ART | The studies included adults only |
| Settings | Any setting | Kenya (2 studies) and Cameroon (1 study) |
| Outcomes | The primary outcomes were adherence to ART and viral load suppression. The secondary outcomes were quality of life, mortality, losses to follow‐up, transfers and withdrawals | All studies reported adherence to ART at 48 or 52 weeks and viral load suppression at 52 weeks. 1 study reported mortality, losses to follow‐up, transfers and withdrawals. 1 study reported quality of life. |
| Date of most recent search: this review included 3 studies, which were the only published studies of which the authors were aware that met their selection criteria up until September 2013. | ||
| Limitations: this is a well‐conducted review that analysed individual patient data from 3 randomised trials. However, a systematic search for other relevant studies was not undertaken. | ||
| Adherence ‐ TB | ||
| Lutge 2015 | ||
| Review objective: to evaluate the effects of material incentives and enablers given to people undergoing diagnostic testing for TB, or receiving drug therapy to prevent or cure TB | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials of any form of material inducement to return for TB test results, or adhere to or complete anti‐TB preventive or curative treatment | 12 randomised trials were included, assessing incentives for adherence to different stages of TB management: returning for reading of tuberculin skin test results (2 studies); clinic attendance for initiation of preventive therapy (1 study); clinic attendance for continuation of preventive therapy (2 studies); adherence to preventive treatment (5 studies); adherence to treatment for active TB (2 studies). The incentives used included cash, vouchers that could be redeemed for various products and food. |
| Participants | Patients receiving curative treatment for TB Patients receiving preventive therapy for TB Patients suspected of TB who are undergoing, and collecting results of, diagnostic tests |
Adolescents aged 11‐19 years (1 study); injection drug or cocaine users (4 studies); homeless or marginally housed adults (3 studies); prisoners (2 studies); and studies on the general adult population (2 studies) |
| Settings | No restrictions | South Africa (1 study), Timor Leste (1 study), USA (10 studies) |
| Outcomes |
For treatment of active TB: cure and/or completion of treatment and/or successful treatment For prophylaxis: cases of active TB; completion of prophylactic treatment For diagnostics: number returning to collect test results Also adverse events and costs |
Return for tuberculin skin test reading Completion of TB prophylaxis Return to clinic for continuation of treatment Successful TB treatment and / or completion of treatment Time needed to track participants who missed appointments |
| Date of most recent search: June 2015 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Adherence ‐ TB | ||
| Liu 2014 | ||
| Review objective: to assess the effects of reminder systems and 'late patient tracers' on the completion of diagnostics, the commencement of treatment in people referred for curative or prophylactic treatment of tuberculosis, the completion of treatment in people starting curative or prophylactic treatment for tuberculosis, and cure rates in people being treated for active tuberculosis. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials, non‐randomised trials or controlled before‐after studies of any actions taken to remind patients to take their TB medication or attend appointments (pre‐appointment reminders) or to contact patients who have missed an appointment (default reminders) | 6 trials of pre‐appointment reminders and 3 trials of default reminders |
| Participants | Children and adults requiring TB treatment, TB prophylaxis, or referred for TB diagnostics or screening | People on treatment for active TB (4 studies), prophylaxis for latent TB (1 study), undergoing TB screening using skin tests (3 studies), and undergoing TB diagnosis, chemoprophylaxis, or treatment (1 study) |
| Settings | Any setting | Pre‐appointment reminders: USA (4 studies), Spain (1 study), and Thailand (1 study); default reminders: India (2 studies) and Iraq (1 study) |
| Outcomes | Completion of TB diagnostics; completion of screening process; commencement of prophylactic treatment; commencement of curative treatment; completion of prophylactic treatment; completion of curative treatment; cure; incidence of active tuberculosis | The main outcomes assessed were the number of patients who adhered to a scheduled appointment and cure for pre‐appointment reminders (6 studies) and the number of patients who completed treatment for default reminders (3 studies) |
| Date of most recent search: August 2014 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations | ||
| Adherence ‐ antimalarials | ||
| Orton 2005 | ||
| Review objective: to summarise the effects of unit‐dose packaged treatment on treatment failure and treatment adherence in people with uncomplicated malaria | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials and quasi‐randomised trials evaluating programmes that include unit‐dose packaging of antimalarial drugs | 1 randomised trial, 1 cluster‐randomised trial, and 3 quasi‐randomised trials evaluating labelled and boxed blister packs of chloroquine and primaquine tablets and capsules (2 studies) and simple, labelled and sectioned polythene bags of chloroquine tablets (3 studies) |
| Participants | People diagnosed with uncomplicated malaria infection | People with uncomplicated malaria confirmed clinically (2 studies), microscopically (2 studies), or using both methods (1 study) |
| Settings | Any setting | Outpatient health centres in China (2 studies), Ghana (2 studies) and Papua New Guinea (1 study) |
| Outcomes | Treatment failure on or by day 28 after initiation of treatment (primary outcome), other clinical measures, treatment adherence and adverse events | None of the trials reported on treatment failure but all reported on some of the following: parasitaemia, clinical symptoms, wellness of the child, cure according to medical notes and the perception of participants, and the recrudescence of infection. All 5 trials reported on treatment adherence. Adverse events were measured in 2 studies |
| Date of most recent search: February 2009 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations. | ||
| Immunisation coverage | ||
| Oyo‐Ita 2016; Jacobson Vann 2005 | ||
| Review objective: to assess the effectiveness of intervention strategies to improve immunisation coverage in LMICs. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials, non‐randomised trials, controlled before‐after studies and interrupted time series studies that evaluate patient oriented (health education or incentives), provider oriented (audit and feedback, reminders) or health system oriented (outreach programmes, interventions oriented to improve quality) interventions to increase immunisation coverage | 14 studies were included: 10 cluster randomised trials and 4 individually randomised trials. Interventions included health education (6 studies), monetary incentives (4 studies), health education plus parent reminders (2 studies), provider oriented interventions (1 study), home visits (1 study), integration of immunisation services with intermittent preventive treatment of malaria in infants (1 study), regular immunisation outreach sessions (1 study) and a combination of provider training and quality assurance (1 study). Several studies evaluated more than 1 intervention |
| Participants | Healthcare personnel who deliver immunisation. Children under 5 years who receive immunisation or their caregivers | Children birth to 4 years (10 studies), primary healthcare workers (1 study), general adult population (1 study), and pregnant and postpartum women (2 studies) |
| Settings | Low‐ and middle‐income countries | Ambulatory care settings in: Georgia (1 study), Ghana (1 study), Honduras (1 study), India (2 studies), Mali (1 study), Mexico (1 study), Nepal (1 study), Nicaragua (1 study), Pakistan (4 studies) and Zimbabwe (1 study) |
| Outcomes |
Primary outcomes: proportion of children who received DTP3 by 1 year; proportion of children who received all recommended vaccinations by 2 years of age Secondary outcomes: occurrence of vaccine preventable diseases, number of under‐fives immunised, costs, attitudes of caregivers and clients to vaccination, adverse events |
DTPs coverage (6 studies), proportion of the target population that was fully immunised (11 studies), percentage change in immunisation coverage over time (2 studies). Other outcomes reported were coverage for specific vaccines (3 studies), costs (1 study), received at least one vaccine (1 study), completion of schedule (1 study). None of the studies provided data on the attitudes of caregivers and clients to vaccination |
| Date of most recent search: May 2016 for most databases | ||
| Limitations: this is well‐conducted systematic review with only minor limitations. | ||
| Insecticide‐treated bednets for malaria | ||
| Augustincic Polec 2015 | ||
| Review objective: to assess the evidence on the effectiveness and equity of strategies to increase ownership and proper use of insecticide‐treated bednets (ITNs) | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials, non‐randomised trials, controlled before‐after studies and interrupted time series studies evaluating interventions to increase ITN ownership and use | 10 randomised trials: 4 studies used a combination of strategies focusing on ITN delivery to increase ITN ownership and appropriate ITN use; 2 studies focused on ITN delivery strategies only; and 7 studies examined appropriate use strategies |
| Participants | Individuals (children and adults) in malaria endemic areas |
Adults, children under 5 years, pregnant women, mothers of children under 5 years, rural cotton farmers |
| Settings | Not specified | Rural communities in Africa, India, and Iran |
| Outcomes | ITN ownership, ITN use and a range of secondary outcomes including (among others) equity ratio of household ITN ownership and adverse effects | ITN ownership, ITN use, and malaria morbidity |
| Date of most recent search: February 2013 | ||
| Limitations: this was a well‐conducted systematic review with only minor limitations. | ||
| Access to healthcare services | ||
| Everett 2011 | ||
| Review objective: to assess the effectiveness of interventions aimed at women, to increase the uptake of cervical cancer screening.. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials assessing universal, selective or opportunistic cervical cancer screening. | 38 randomised trials, including 6 cluster randomised trials assessed invitations (17 studies), education (6 studies), message framing (1 study), counselling (2 studies), risk factor assessment (2 studies), procedures (1 study), use of a photo comic book (1 study) and intensive recruitment (1 study). |
| Participants | Women eligible to participate in cervical cancer screening. | Women receiving care in community clinics, primary care practices and Health Maintenance Organisations, mostly located in urban areas. |
| Settings | Community, workplace, health centre and hospital settings. | USA (16 studies), Australia (9 studies), UK (7 studies), Canada (2 studies), Sweden (2 studies), South Africa (1 study) and Italy (1 study) |
| Outcomes | Primary outcomes: uptake of cervical screening as recorded by health service records and via self‐report Secondary outcomes: booking of appointments; reported intentions to attend screening; satisfaction with screening service, attitudes and knowledge about screening; costs of the interventions |
All primary outcomes, booking of appointments (1 study), acceptability of the intervention (1 study) and costs of the interventions (5 studies) |
| Date of most recent search: March 2009 | ||
| Limitations: this is well‐conducted systematic review with only minor limitations. | ||
| Access to healthcare services | ||
| Jia 2014 | ||
| Review objective: to assess the effects of outreach strategies for expanding insurance coverage of vulnerable populations. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and Interventions | Randomised trials, non‐randomised trials, controlled before‐after studies, and interrupted time series studies | 1 randomised trial and 1 non‐randomised trial |
| Participants | Children, the elderly, women, low‐income individuals, rural population, racial or ethnic minorities, immigrants, informal sector workers, and population with disability or chronic diseases | 674 children aged 18 years or younger recruited from 2 minority communities (1 study) or the emergency departments of 4 inner‐city hospitals (1 study) |
| Settings | Not pre‐specified | USA (2 studies) |
| Outcomes | Primary outcomes: enrolment into health insurance programmes Secondary outcomes: health service utilisation, health status, satisfaction, costs, adverse effects |
Enrolment of children into health insurance (2 studies), maintaining enrolment of children in insurance schemes (1 study), mean time to obtain insurance (1 study), parental satisfaction with process of enrolment (1 study) |
| Date of most recent search: November 2012 | ||
| Limitations: this is a well‐conducted systematic review with only minor limitations | ||
Appendix 4. Supplementary/related reviews
Educational meetings
In situ simulation in continuing education for the healthcare professions: a systematic review (Rosen 2012)
Reminders
The relationship between nursing leadership and patient outcomes: a systematic review (Wong 2013)
Multifaceted interventions
Systematic review of knowledge translation strategies in the allied health professions (Scott 2012)
What are the effects of interventions to improve the uptake of evidence from health research into policy in low‐ and middle‐income countries? (Graham 2011)
Prescribing antibiotics
Improving antibiotic selection: a systematic review and quantitative analysis of quality improvement strategies (Steinman 2006)
Adherence ‐ medication
Financial reinforcers for improving medication adherence: findings from a meta‐analysis (Petry 2012)
Reminder packaging for improving adherence to self‐administered long‐term medications (Mahtani 2011)
Interventions for enhancing medication adherence (Nieuwlaat 2014)
Immunisation coverage
Too little but not too late: results of a literature review to improve routine immunization programs in developing countries (Ryman 2008)
Appendix 5. Reviews awaiting classification
Likely included reviews
Bowman LR, Donegan S, McCall PJ. Is dengue vector control deficient in effectiveness or evidence?: Systematic review and meta‐analysis. PLOS Neglected Tropical Diseases 2016;10(3):e0004551.
Abdel‐Aleem H, El‐Gibaly OMH, El‐Gazzar AFS, Al‐Attar GST. Mobile clinics for women's and children's health. Cochrane Database of Systematic Reviews. 2016;8:CD009677.
Akl EA, El‐Jardali F, Bou Karroum L, El‐Eid J, Brax H, Akik C, et al. Effectiveness of Mechanisms and Models of Coordination between Organizations, Agencies and Bodies Providing or Financing Health Services in Humanitarian Crises: A Systematic Review. PloS one. 2015;10(9):e0137159.
Algie CM, Mahar RK, Wasiak J, Batty L, Gruen RL, Mahar PD. Interventions for reducing wrong‐site surgery and invasive clinical procedures. The Cochrane database of systematic reviews. 2015;3(3):CD009404.
Ambia J, Mandala J. A systematic review of interventions to improve prevention of mother‐to‐child HIV transmission service delivery and promote retention. Journal of the International AIDS Society. 2016;19(1):20309.
Baldwin K, Namdari S, Donegan D, Kamath AF, Mehta S. Early effects of resident work‐hour restrictions on patient safety: a systematic review and plea for improved studies. Journal of Bone and Joint Surgery 2011;93:e5.
Barnard S, Kim C, Park MH, Ngo TD. Doctors or mid‐level providers for abortion. The Cochrane database of systematic reviews. 2015;7(7):CD011242.
Basu S, Andrews J, Kishore S, Panjabi R, Stuckler D. Comparative performance of private and public healthcare systems in low‐ and middle‐income countries: a systematic review. PLoS medicine. 2012;9(6):e1001244.
Blacklock C, Gonçalves Bradley DC, Mickan S, Willcox M, Roberts N, Bergström A, et al. Impact of Contextual Factors on the Effect of Interventions to Improve Health Worker Performance in Sub‐Saharan Africa: Review of Randomised Clinical Trials. PloS one. 2016;11(1):e0145206.
Byrne A, Hodge A, Jimenez‐Soto E, Morgan A. What works? Strategies to increase reproductive, maternal and child health in difficult to access mountainous locations: a systematic literature review. PloS one. 2014;9(2):e87683.
Coast E, Jones E, Lattof SR, Portela A. Effectiveness of interventions to provide culturally appropriate maternity care in increasing uptake of skilled maternity care: a systematic review. Health policy and planning. 2016;31(10):1479‐91.
Cornish F, Priego‐Hernandez J, Campbell C, Mburu G, McLean S. The impact of Community Mobilisation on HIV Prevention in Middle and Low Income Countries: A Systematic Review and Critique. AIDS and behavior. 2014;18(11):2110‐34.
Dawson A, Tran NT, Westley E, Mangiaterra V, Festin M. Improving access to emergency contraception pills through strengthening service delivery and demand generation: a systematic review of current evidence in low and middle‐income countries. PloS one. 2014;9(10):e109315.
de Jongh TE, Gurol‐Urganci I, Allen E, Zhu NJ, Atun R. Integration of antenatal care services with health programmes in low‐ and middle‐income countries: systematic review. Journal of global health. 2016;6(1):010403.
Dyer TA, Brocklehurst P, Glenny AM, Davies L, Tickle M, Issac A, et al. Dental auxiliaries for dental care traditionally provided by dentists. The Cochrane database of systematic reviews. 2014;8(8):CD010076.
Ehiri JE, Gunn JK, Center KE, Li Y, Rouhani M, Ezeanolue EE. Training and deployment of lay refugee/internally displaced persons to provide basic health services in camps: a systematic review. Global health action. 2014;7:23902.
Emdin CA, Chong NJ, Millson PE. Non‐physician clinician provided HIV treatment results in equivalent outcomes as physician‐provided care: a meta‐analysis. Journal of the International AIDS Society. 2013;16(no pagination):18445.
Fernandez Turienzo C, Sandall J, Peacock JL. Models of antenatal care to reduce and prevent preterm birth: a systematic review and meta‐analysis. BMJ open. 2016;6(1):e009044.
Feyissa GT, Lockwood C, Munn Z. The effectiveness of home‐based HIV counseling and testing in reducing stigma and risky sexual behavior among adults and adolescents: a systematic review and meta‐analysis. JBI Database of Systematic Reviews and Implementation Reports. 2015;13(6):318‐72.
Fiander M, McGowan J, Grad R, Pluye P, Hannes K, Labrecque M, et al. Interventions to increase the use of electronic health information by healthcare practitioners to improve clinical practice and patient outcomes. The Cochrane database of systematic reviews. 2015;3(3):CD004749.
Flodgren G, Rachas A, Farmer AJ, Inzitari M, Shepperd S. Interactive telemedicine: effects on professional practice and health care outcomes. The Cochrane database of systematic reviews. 2015;9(9):CD002098.
Gaitonde R, Oxman AD, Okebukola PO, Rada G. Interventions to reduce corruption in the health sector. Cochrane Database of Systematic Reviews. 2016;8:CD008856.
George AS, Branchini C, Portela A. Do Interventions that Promote Awareness of Rights Increase Use of Maternity Care Services? A Systematic Review. PloS one. 2015;10(10):e0138116.
Ghada Abou El S, Therese D, Hatem AM. Planned home versus hospital care for preterm prelabour rupture of the membranes (PPROM) prior to 37 weeks' gestation. Cochrane Database of Systematic Reviews. 2014;4(4):CD008053.
Giedion U, Alfonso EA, Diaz Y. The Impact of Universal Coverage Schemes in the Developing World: A Review of the Existing Evidence. Universal Health Coverage (UNICO) studies series; no. 25. Washington D.C.: The Worldbank. 2013.
Handford CD, Tynan AM, Agha A, Rzeznikiewiz D, Glazier RH. Organization of care for persons with HIV‐infection: a systematic review. AIDS care. 2016:1‐10.
Health Quality Ontario. Interventions to Improve Access to Primary Care for People Who Are Homeless: A Systematic Review. Ontario health technology assessment series. 2016;16(9):1‐50.
Hensen B, Taoka S, Lewis JJ, Weiss HA, Hargreaves J. Systematic review of strategies to increase men's HIV‐testing in sub‐Saharan Africa. AIDS (London, England). 2014;28(14):2133‐45.
Hernández AV, Pasupuleti V, Benites‐Zapata V, Velásquez‐Hurtado E, Loyola‐Romaní J, Rodríguez‐Calviño Y, et al. [Systematic review of the efectiveness of community‐based interventions to decrease neonatal mortality]. Revista peruana de medicina experimental y salud pública. 2015;32(3):532‐45.
Hesselink G, Berben S, Beune T, Schoonhoven L. Improving the governance of patient safety in emergency care: a systematic review of interventions. BMJ open. 2016;6(1):e009837.
Hopkins U, Itty AS, Nazario H, Pinon M, Slyer J, Singleton J. The effectiveness of delegation interventions by the registered nurse to the unlicensed assistive personnel and their impact on quality of care, patient satisfaction, and RN staff satisfaction: a systematic review. JBI Library of Systematic Reviews. 2012;10(15):895‐934.
Hoyler M, Hagander L, Gillies R, Riviello R, Chu K, Bergström S, et al. Surgical care by non‐surgeons in low‐income and middle‐income countries: a systematic review. Lancet (London, England). 2015;385 Suppl 2:S42.
Joshi R, Alim M, Kengne AP, Jan S, Maulik PK, Peiris D, et al. Task shifting for non‐communicable disease management in low and middle income countries‐‐a systematic review. PloS one. 2014;9(8):e103754.
Kien C, Reichenpfader U, Nußbaumer B, Rohleder S, Punz P, Christof C, et al. [Comparative effectiveness and safety of screening and counselling interventions conducted by non‐physicians and physicians: A systematic review]. Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen. 2015;109(1):18‐27.
Kilpatrick K, Kaasalainen S, Donald F, Reid K, Carter N, Bryant‐Lukosius D, et al. The effectiveness and cost‐effectiveness of clinical nurse specialists in outpatient roles: a systematic review. Journal of evaluation in clinical practice. 2014;20(6):1106‐23.
Kim K, Choi JS, Choi E, Nieman CL, Joo JH, Lin FR, et al. Effects of Community‐Based Health Worker Interventions to Improve Chronic Disease Management and Care Among Vulnerable Populations: A Systematic Review. American journal of public health. 2016;106(4):e1‐e26.
Kredo T, Adeniyi FGB, Bateganya M, Pienaar ED. Task shifting from doctors to non‐doctors for initiation and maintenance of antiretroviral therapy. Cochrane Database of Systematic Reviews. 2014;7(7):CD007331.
Lassi ZS, Musavi NB, Maliqi B, Mansoor N, de Francisco A, Toure K, et al. Systematic review on human resources for health interventions to improve maternal health outcomes: evidence from low‐ and middle‐income countries. Human resources for health. 2016;14(1):10.
Lavender T, Richens Y, Milan SJ, Smyth RMD, Dowswell T. Telephone support for women during pregnancy and the first six weeks postpartum. Cochrane Database of Systematic Reviews. 2013;7(7):CD009338.
Lawrence D, Fedorowicz Z, van Zuuren EJ. Day care versus in‐patient surgery for age‐related cataract. The Cochrane database of systematic reviews. 2015;11(11):CD004242.
Liu G, Jack H, Piette A, Mangezi W, Machando D, Rwafa C, et al. Mental health training for health workers in Africa: a systematic review. The lancet Psychiatry. 2016;3(1):65‐76.
MacPherson P, Munthali C, Ferguson J, Armstrong A, Kranzer K, Ferrand RA, et al. Service delivery interventions to improve adolescents' linkage, retention and adherence to antiretroviral therapy and HIV care. Tropical medicine & international health : TM & IH. 2015;20(8):1015‐32.
Mauger Rothenberg B, Marbella A, Pines E, Chopra R, Black ER, Aronson N. Prevention of healthcare‐associated infections. Closing the quality gap: revisiting the state of the science. Rockville, MD, USA: Agency for Healthcare Research and Quality. Evidence Report/Technology Assessment; 208. 2012.
Mbuagbaw L, Medley N, Darzi AJ, Richardson M, Habiba Garga K, Ongolo‐Zogo P. Health system and community level interventions for improving antenatal care coverage and health outcomes. The Cochrane database of systematic reviews. 2015;12(12):CD010994.
Mdege ND, Chindove S. Bringing antiretroviral therapy (ART) closer to the end‐user through mobile clinics and home‐based ART: systematic review shows more evidence on the effectiveness and cost effectiveness is needed. The International journal of health planning and management. 2013;29(1):e31‐47.
Moraros J, Lemstra M, Nwankwo C. Lean interventions in healthcare: do they actually work? A systematic literature review. International journal for quality in health care : journal of the International Society for Quality in Health Care / ISQua. 2016;28(2):150‐65.
Mundell WC, Kennedy CS, Szostek JH, Cook DA. Simulation technology for resuscitation training: a systematic review and meta‐analysis. Resuscitation 2013;84:1174‐83.
Nunan M, Duke T. Effectiveness of pharmacy interventions in improving availability of essential medicines at the primary healthcare level. Tropical medicine & international health : TM & IH. 2011;16(5):647‐58.
Oluoch T, Santas X, Kwaro D, Were M, Biondich P, Bailey C, et al. The effect of electronic medical record‐based clinical decision support on HIV care in resource‐constrained settings: a systematic review. International journal of medical informatics. 2012;81(10):e83‐92.
Palmer KS, Agoritsas T, Martin D, Scott T, Mulla SM, Miller AP, et al. Activity‐based funding of hospitals and its impact on mortality, readmission, discharge destination, severity of illness, and volume of care: a systematic review and meta‐analysis. PloS one. 2014;9(10):e109975.
Pega F, Liu SY, Walter S, Lhachimi SK. Unconditional cash transfers for assistance in humanitarian disasters: effect on use of health services and health outcomes in low‐ and middle‐income countries. The Cochrane database of systematic reviews. 2015;9(9):CD011247.
Penazzato M, Davies MA, Apollo T, Negussie E, Ford N. Task shifting for the delivery of pediatric antiretroviral treatment: a systematic review. Journal of acquired immune deficiency syndromes (1999). 2014;65(4):414‐22.
Pollaris G, Sabbe M. Reverse triage: more than just another method. European journal of emergency medicine : official journal of the European Society for Emergency Medicine. 2015;23(4):240‐7.
Polus S, Lewin S, Glenton C, Lerberg PM, Rehfuess E, Gülmezoglu AM. Optimizing the delivery of contraceptives in low‐ and middle‐income countries through task shifting: a systematic review of effectiveness and safety. Reproductive health. 2015;12(1):27.
Rashidian A, Omidvari AH, Vali Y, Sturm H, Oxman AD. Pharmaceutical policies: effects of financial incentives for prescribers. The Cochrane database of systematic reviews. 2015;8(8):CD006731.
Reichow B, Servili C, Yasamy MT, Barbui C, Saxena S. Non‐specialist psychosocial interventions for children and adolescents with intellectual disability or lower‐functioning autism spectrum disorders: a systematic review. PLoS medicine. 2013;10(12):e1001572.
Reisman J, Arlington L, Jensen L, Louis H, Suarez‐Rebling D, Nelson BD. Newborn Resuscitation Training in Resource‐Limited Settings: A Systematic Literature Review. Pediatrics. 2016;138(2):1‐16.
Robyn PJ, Sauerborn R, Bärnighausen T. Provider payment in community‐based health insurance schemes in developing countries: a systematic review. Health policy and planning. 2013;28(2):111‐22.
Rolfe A, Cash‐Gibson L, Car J, Sheikh A, McKinstry B. Interventions for improving patients' trust in doctors and groups of doctors. Cochrane Database of Systematic Reviews 2014, Issue 3. Art. No.: CD004134. DOI: 10.1002/14651858.CD004134.pub3.
Salam RA, Das JK, Lassi ZS, Bhutta ZA. Impact of community‐based interventions for the prevention and control of malaria on intervention coverage and health outcomes for the prevention and control of malaria. Infectious diseases of poverty. 2014;3(1):25.
Schmidt E, Goldhaber‐Fiebert SN, Ho LA, McDonald KM. Simulation exercises as a patient safety strategy: a systematic review. Annals of internal medicine. 2013;158(5 Pt 2):426‐32.
Sharon RL, Amanda N, Andrew FS, Phil A. Physician anaesthetists versus non‐physician providers of anaesthesia for surgical patients. Cochrane Database of Systematic Reviews. 2014;7(7):CD010357.
Sondaal SF, Browne JL, Amoakoh‐Coleman M, Borgstein A, Miltenburg AS, Verwijs M, et al. Assessing the Effect of mHealth Interventions in Improving Maternal and Neonatal Care in Low‐ and Middle‐Income Countries: A Systematic Review. PloS one. 2016;11(5):e0154664.
Spaan E, Mathijssen J, Tromp N, McBain F, ten Have A, Baltussen R. The impact of health insurance in Africa and Asia: a systematic review. Bulletin of the World Health Organization. 2012;90(9):685‐92.
Sunguya BF, Poudel KC, Mlunde LB, Urassa DP, Yasuoka J, Jimba M. Nutrition training improves health workers' nutrition knowledge and competence to manage child undernutrition: a systematic review. Frontiers in public health. 2013;1:37.
Susan FM, Benjamin MH, Ramila B, Tim E, Debra B. Demand‐side financing measures to increase maternal health service utilisation and improve health outcomes: a systematic review of evidence from low‐ and middle‐income countries. JBI Library of Systematic Reviews. 2012;10(58):4165‐567.
Tannenbaum SI, Cerasoli CP. Do team and individual debriefs enhance performance? a meta‐analysis. Human Factors 2013;55(1):231‐45.
Thomas SM, Jeyaraman M, Hodge WG, Hutnik C, Costella J, Malvankar‐Mehta MS. The effectiveness of teleglaucoma versus in‐patient examination for glaucoma screening: a systematic review and meta‐analysis. PloS one. 2014;9(12):e113779.
Tibingana‐Ahimbisibwe B, Katabira C, Mpalampa L, Harrison RA. The effectiveness of adolescent‐specific prenatal interventions on improving attendance and reducing harm during and after birth: a systematic review. International journal of adolescent medicine and health. 2016.
Till SR, Everetts D, Haas DM. Incentives for increasing prenatal care use by women in order to improve maternal and neonatal outcomes. Cochrane Database of Systematic Reviews. 2015;12(12):CD009916.
Watterson JL, Walsh J, Madeka I. Using mHealth to Improve Usage of Antenatal Care, Postnatal Care, and Immunization: A Systematic Review of the Literature. BioMed research international. 2015;2015(no pagination):153402.
Weeks G, George J, Maclure K, Stewart D. Non‐medical prescribing versus medical prescribing for acute and chronic disease management in primary and secondary care. The Cochrane database of systematic reviews. 2016;11:CD011227.
Wiysonge CS, Abdullahi LH, Ndze VN, Hussey GD. Public stewardship of private for‐profit healthcare providers in low‐ and middle‐income countries. Cochrane Database of Systematic Reviews. 2016;8(8):CD009855.
Wong WC, Luk CW, Kidd MR. Is there a role for primary care clinicians in providing shared care in HIV treatment? A systematic literature review. Sexually transmitted infections. 2012;88(2):125‐31.
Likely excluded reviews
Bassili A, Fitzpatrick C, Qadeer E, Fatima R, Floyd K, Jaramillo E. A systematic review of the effectiveness of hospital‐ and ambulatory‐based management of multidrug‐resistant tuberculosis. The American journal of tropical medicine and hygiene. 2013;89(2):271‐80.
Bhageerathy R, Nair S, Bhaskaran U. A systematic review of community‐based health insurance programs in South Asia. The International journal of health planning and management. 2016.
Blaya JA, Fraser HS, Holt B. E‐health technologies show promise in developing countries. Health affairs (Project Hope). 2010;29(2):244‐51.
Callese TE, Richards CT, Shaw P, Schuetz SJ, Paladino L, Issa N, et al. Trauma system development in low‐ and middle‐income countries: a review. The Journal of surgical research. 2015;193(1):300‐7.
Davy C, Bleasel J, Liu H, Tchan M, Ponniah S, Brown A. Effectiveness of chronic care models: opportunities for improving healthcare practice and health outcomes: a systematic review. BMC health services research. 2015;15(1):194.
Dawson AZ, Walker RJ, Campbell JA, Egede LE. Effective Strategies for Global Health Training Programs A Systematic Review of Training Outcomes in Low and Middle Income Countries. Global journal of health science. 2016;8(11):56719.
Higgs ES, Goldberg AB, Labrique AB, Cook SH, Schmid C, Cole CF, et al. Understanding the role of mHealth and other media interventions for behavior change to enhance child survival and development in low‐ and middle‐income countries: an evidence review. Journal of health communication. 2014;19 Suppl 1:164‐89.
Hubert GJ, Müller‐Barna P, Audebert HJ. Recent advances in TeleStroke: a systematic review on applications in prehospital management and Stroke Unit treatment or TeleStroke networking in developing countries. International journal of stroke : official journal of the International Stroke Society. 2014;9(8):968‐73.
Margaret Elizabeth K, Denis P, Peter CR, Wim Van L. The contribution of primary care to health and health systems in low‐ and middle‐income countries: A critical review of major primary care initiatives. 2010.
Pannick S, Davis R, Ashrafian H, Byrne BE, Beveridge I, Athanasiou T, et al. Effects of Interdisciplinary Team Care Interventions on General Medical Wards: A Systematic Review. JAMA internal medicine. 2015;175(8):1288‐98.
Schiavo R, May Leung M, Brown M. Communicating risk and promoting disease mitigation measures in epidemics and emerging disease settings. Pathogens and global health. 2014;108(2):76‐94.
Zulfiqar AB, Zohra SL, Nadia M. Systematic review on human resources for health interventions to improve maternal health outcomes: Evidence from developing countries. 2010.
Uncertain reviews
Acheampong F, Anto BP, Koffuor GA. Medication safety strategies in hospitals‐‐a systematic review. The International journal of risk & safety in medicine. 2014;26(3):117‐31.
Alkhenizan A, Shaw C. Impact of accreditation on the quality of healthcare services: a systematic review of the literature. Annals of Saudi medicine. 2011;31(4):407‐16.
Al‐Mallah MH, Farah I, Al‐Madani W, Bdeir B, Al Habib S, Bigelow ML, et al. The Impact of Nurse‐Led Clinics on the Mortality and Morbidity of Patients with Cardiovascular Diseases: A Systematic Review and Meta‐analysis. The Journal of cardiovascular nursing. 2015;31(1):89‐95.
Bakitas MA, Elk R, Astin M, Ceronsky L, Clifford KN, Dionne‐Odom JN, et al. Systematic Review of Palliative Care in the Rural Setting. Cancer control : journal of the Moffitt Cancer Center. 2015;22(4):450‐64.
Balfour J, Abdulcadir J, Say L, Hindin MJ. Interventions for healthcare providers to improve treatment and prevention of female genital mutilation: a systematic review. BMC health services research. 2016;16(1):409.
Bannan DF, Tully MP. Bundle interventions used to reduce prescribing and administration errors in hospitalized children: a systematic review. Journal of clinical pharmacy and therapeutics. 2016;41(3):246‐55.
Bashshur RL, Howell JD, Krupinski EA, Harms KM, Bashshur N, Doarn CR. The Empirical Foundations of Telemedicine Interventions in Primary Care. Telemedicine journal and e‐health : the official journal of the American Telemedicine Association. 2016;22(5):342‐75.
Boccia D, Hargreaves J, Lönnroth K, Jaramillo E, Weiss J, Uplekar M, et al. Cash transfer and microfinance interventions for tuberculosis control: review of the impact evidence and policy implications. The international journal of tuberculosis and lung disease : the official journal of the International Union against Tuberculosis and Lung Disease. 2011;15 Suppl 2:S37‐49.
Booth A, Cantrell A, Preston L, Chambers D, Goyder E. What is the evidence for the effectiveness, appropriateness and feasibility of group clinics for patients with chronic conditions? A systematic review2015 2015/12/None.
Brocklehurst P, Mertz B, Jerković‐Ćosić K, Littlewood A, Tickle M. Direct access to midlevel dental providers: an evidence synthesis. Journal of public health dentistry. 2014;74(4):326‐35.
Candy B, France R, Low J, Sampson L. Does involving volunteers in the provision of palliative care make a difference to patient and family wellbeing? A systematic review of quantitative and qualitative evidence. International journal of nursing studies. 2014;52(3):756‐68.
Chapman SM, Wray J, Oulton K, Peters MJ. Systematic review of paediatric track and trigger systems for hospitalised children. Resuscitation. 2016;109:87‐109.
Coxeter P, Del Mar CB, McGregor L, Beller EM, Hoffmann TC. Interventions to facilitate shared decision making to address antibiotic use for acute respiratory infections in primary care. The Cochrane database of systematic reviews. 2015;11(11):CD010907.
Damiani G, Pinnarelli L, Sommella L, Vena V, Magrini P, Ricciardi W. The Short Stay Unit as a new option for hospitals: a review of the scientific literature. Medical science monitor : international medical journal of experimental and clinical research. 2011;17(6):SR15‐9.
Gentry S, van Velthoven MHMMT, Tudor Car L, Car J. Telephone delivered interventions for reducing morbidity and mortality in people with HIV infection. Cochrane Database of Systematic Reviews. 2013;5(5):CD009189.
Harding R, Albertyn R, Sherr L, Gwyther L. Pediatric palliative care in sub‐saharan Africa: a systematic review of the evidence for care models, interventions, and outcomes. Journal of pain and symptom management. 2014;47(3):642‐51.
Hastings SE, Armitage GD, Mallinson S, Jackson K, Suter E. Exploring the relationship between governance mechanisms in healthcare and health workforce outcomes: a systematic review. BMC health services research. 2014;14(1):479.
Hines S, Munday J, Kynoch K. Effectiveness of nurse‐led preoperative assessment services for elective surgery: a systematic review update. JBI database of systematic reviews and implementation reports. 2015;13(6):279‐317.
Hotchkiss DR, Diana ML, Foreit KG. How can routine health information systems improve health systems functioning in low‐ and middle‐income countries? Assessing the evidence base. Advances in health care management. 2012;12:25‐58.
Housden L, Wong ST, Dawes M. Effectiveness of group medical visits for improving diabetes care: a systematic review and meta‐analysis. CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne. 2013;185(13):E635‐44.
Ireland S, Kent B. Telephone pre‐operative assessment for adults: a comprehensive systematic review. JBI Library of Systematic Reviews. 2012;10(25):1452‐503.
Kågesten A, Parekh J, Tunçalp O, Turke S, Blum RW. Comprehensive adolescent health programs that include sexual and reproductive health services: a systematic review. American journal of public health. 2014;104(12):e1‐e14.
Lazarus JV, Safreed‐Harmon K, Nicholson J, Jaffar S. Health service delivery models for the provision of antiretroviral therapy in sub‐Saharan Africa: a systematic review. Tropical medicine & international health : TM & IH. 2014;19(10):1198‐215.
Leidy Johanna Rueda D, Diná Lopes Monteiro da C. The efficacy of telephone use to assist and improve the wellbeing of family caregivers of persons with chronic diseases: a systematic review. JBI Library of Systematic Reviews. 2015;12(12):106‐40.
McCormack L, Sheridan S, Lewis M, Boudewyns V, Melvin CL, Kistler C, et al. Communication and dissemination strategies to facilitate the use of health‐related evidence. Evidence report/technology assessment. 2013(213):1‐520.
Meid AD, Lampert A, Burnett A, Seidling HM, Haefeli WE. The impact of pharmaceutical care interventions for medication underuse in older people: a systematic review and meta‐analysis. British journal of clinical pharmacology. 2015;80(4):768‐76.
Mengistu TA, Tafere TE. Effect of antenatal care on institutional delivery in developing countries: a systematic review. JBI Library of Systematic Reviews. 2011;9(35):1447‐70.
Mitchell GK, Burridge L, Zhang J, Donald M, Scott IA, Dart J, et al. Systematic review of integrated models of health care delivered at the primary?secondary interface: how effective is it and what determines effectiveness? Australian journal of primary health. 2015;21(4):391‐408.
Palmas W, March D, Darakjy S, Findley SE, Teresi J, Carrasquillo O, et al. Community Health Worker Interventions to Improve Glycemic Control in People with Diabetes: A Systematic Review and Meta‐Analysis. Journal of General Internal Medicine. 2015;30:1004‐12.
Pérez‐Escamilla R, Martinez JL, Segura‐Pérez S. Impact of the Baby‐friendly Hospital Initiative on breastfeeding and child health outcomes: a systematic review. Maternal & child nutrition. 2016;12(3):402‐17.
Rinke ML, Bundy DG, Velasquez CA, Rao S, Zerhouni Y, Lobner K, et al. Interventions to Reduce Pediatric Medication Errors: A Systematic Review. Pediatrics. 2014;134(2):338‐60.
Rudge MV, Lima SA, El Dib RP, Marini G, Magalhães C, Calderon Ide M. Effect of ambulatory versus hospital treatment for gestational diabetes or hyperglycemia on infant mortality rates: a systematic review. São Paulo medical journal = Revista paulista de medicina. 2013;131(5):331‐7.
Sabater‐Hernández D, Sabater‐Galindo M, Fernandez‐Llimos F, Rotta I, Hossain LN, Durks D, et al. A Systematic Review of Evidence‐Based Community Pharmacy Services Aimed at the Prevention of Cardiovascular Disease. Journal of managed care & specialty pharmacy. 2016;22(6):699‐713.
Salmoiraghi A, Hussain S. A Systematic Review of the Use of Telepsychiatry in Acute Settings. Journal of psychiatric practice. 2015;21(5):389‐93.
Santos MT, Moura SC, Gomes LM, Lima AH, Moreira RS, Silva CD, et al. Telehealth application on the rehabilitation of children and adolescents. Revista paulista de pediatria : orgão oficial da Sociedade de Pediatria de São Paulo. 2014;32(1):136‐43.
Saxon RL, Gray MA, Oprescu FI. Extended roles for allied health professionals: an updated systematic review of the evidence. Journal of multidisciplinary healthcare. 2014;7((Saxon R.L., robyn.saxon@health.qld.gov.au; Gray M.A.; Ioprescu F.) School of Health and Sports Sciences, University of the Sunshine Coast, Sippy Downs, Australia):479‐88.
Stokes J, Panagioti M, Alam R, Checkland K, Cheraghi‐Sohi S, Bower P. Effectiveness of Case Management for 'At Risk' Patients in Primary Care: A Systematic Review and Meta‐Analysis. PloS one. 2015;10(7):e0132340.
Suksomboon N, Poolsup N, Nge YL. Impact of phone call intervention on glycemic control in diabetes patients: a systematic review and meta‐analysis of randomized, controlled trials. PloS one. 2014;9(2):e89207.
Tao D, Xie L, Wang T, Wang T. A meta‐analysis of the use of electronic reminders for patient adherence to medication in chronic disease care. Journal of Telemedicine and Telecare. 2015;21(1).
Tricco AC, Antony J, Ivers NM, Ashoor HM, Khan PA, Blondal E, et al. Effectiveness of quality improvement strategies for coordination of care to reduce use of health care services: a systematic review and meta‐analysis. CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne. 2014;186(15):E568‐78.
Pitt V, Lowe D, Hill S, Prictor M, Hetrick SE, Ryan R, et al. Consumer‐providers of care for adult clients of statutory mental health services. Cochrane Database of Systematic Reviews. 2013;3(3):CD004807.
Weaver MS, Lönnroth K, Howard SC, Roter DL, Lam CG. Interventions to improve adherence to treatment for paediatric tuberculosis in low‐ and middle‐income countries: a systematic review and meta‐analysis. Bulletin of the World Health Organization. 2015;93(10):700‐11B.
Wekesah FM, Mbada CE, Muula AS, Kabiru CW, Muthuri SK, Izugbara CO. Effective non‐drug interventions for improving outcomes and quality of maternal health care in sub‐Saharan Africa: a systematic review. Systematic reviews. 2016;5(1):137.
Willey B, Smith Paintain L, Mangham‐Jefferies L, Car J, Armstrong Schellenberg J. Effectiveness of interventions to strengthen national health service delivery on coverage, access, quality and equity in the use of health services in low and lower middle income countries. 2013 2013.
World Health Organization, University of California SF. Task shifting ‐ physicians (doctors) versus non‐physicians (nurses or clinical officers) for initiation and maintenance of antiretroviral therapy. World Health Organization. 2013.
Yasmin F, Banu B, Zakir SM, Sauerborn R, Ali L, Souares A. Positive influence of short message service and voice call interventions on adherence and health outcomes in case of chronic disease care: a systematic review. BMC medical informatics and decision making. 2016;16:46.
Zhai YK, Zhu WJ, Hou HL, Sun DX, Zhao J. Efficacy of telemedicine for thrombolytic therapy in acute ischemic stroke: a meta‐analysis. Journal of telemedicine and telecare. 2015;21(3):123‐30.
Zhou K, Fitzpatrick T, Walsh N, Kim JY, Chou R, Lackey M, et al. Interventions to optimise the care continuum for chronic viral hepatitis: a systematic review and meta‐analyses. The Lancet Infectious diseases. 2016.
Zwanikken PA, Dieleman M, Samaranayake D, Akwataghibe N, Scherpbier A. A systematic review of outcome and impact of master's in health and health care. BMC medical education. 2013;13:18.
Likely supplemental reviews
Abdulwahid MA, Booth A, Kuczawski M, Mason SM. The impact of senior doctor assessment at triage on emergency department performance measures: systematic review and meta‐analysis of comparative studies. Emergency medicine journal : EMJ. 2015;33(7):504‐13.
Adebayo EF, Uthman OA, Wiysonge CS, Stern EA, Lamont KT, Ataguba JE. A systematic review of factors that affect uptake of community‐based health insurance in low‐income and middle‐income countries. BMC health services research. 2015;15(1):543.
Alghamdi M, Gashgari H, Househ M. A Systematic Review of Mobile Health Technology Use in Developing Countries. Studies in health technology and informatics. 2015;213:223‐6.
Alkhaled L, Kahale L, Nass H, Brax H, Fadlallah R, Badr K, et al. Legislative, educational, policy and other interventions targeting physicians' interaction with pharmaceutical companies: a systematic review. BMJ open. 2014;4(7):e004880.
Altowaijri A, Phillips CJ, Fitzsimmons D. A systematic review of the clinical and economic effectiveness of clinical pharmacist intervention in secondary prevention of cardiovascular disease. Journal of managed care pharmacy : JMCP. 2013;19(5):408‐16.
Amouzou A, Morris S, Moulton LH, Mukanga D. Assessing the impact of integrated community case management (iCCM) programs on child mortality: Review of early results and lessons learned in sub‐Saharan Africa. Journal of global health. 2014;4(2):020411.
Aziz H, Hatah E, Makmor Bakry M, Islahudin F. How payment scheme affects patients' adherence to medications? A systematic review. Patient preference and adherence. 2016;10:837‐50.
Bailey C, Blake C, Schriver M, Cubaka VK, Thomas T, Martin Hilber A. A systematic review of supportive supervision as a strategy to improve primary healthcare services in Sub‐Saharan Africa. International journal of gynaecology and obstetrics: the official organ of the International Federation of Gynaecology and Obstetrics. 2015;132(1):117‐25.
Baxter PE, Hewko SJ, Pfaff KA, Cleghorn L, Cunningham BJ, Elston D, et al. Leaders' experiences and perceptions implementing activity‐based funding and pay‐for‐performance hospital funding models: A systematic review. Health policy (Amsterdam, Netherlands). 2015;119(8):1096‐110.
Bbosa GS, Wong G, Kyegombe DB, Ogwal‐Okeng J. Effects of intervention measures on irrational antibiotics/antibacterial drug use in developing countries: A systematic review. Health. 2014;6.
Bellows B, Bulaya C, Inambwae S, Lissner CL, Ali M, Bajracharya A. Family Planning Vouchers in Low and Middle Income Countries: A Systematic Review. Studies in family planning. 2016;47(4):357‐70.
Bellows BW, Conlon CM, Higgs ES, Townsend JW, Nahed MG, Cavanaugh K, et al. A taxonomy and results from a comprehensive review of 28 maternal health voucher programmes. Journal of health, population, and nutrition. 2013;31(4 Suppl 2):106‐28.
Benishek LA, Dugosh KL, Kirby KC, Matejkowski J, Clements NT, Seymour BL, et al. Prize‐based contingency management for the treatment of substance abusers: a meta‐analysis. Addiction (Abingdon, England). 2014;109(9):1426‐36.
Beratarrechea A, Lee AG, Willner JM, Jahangir E, Ciapponi A, Rubinstein A. The impact of mobile health interventions on chronic disease outcomes in developing countries: a systematic review. Telemedicine journal and e‐health : the official journal of the American Telemedicine Association. 2014;20(1):75‐82.
Blank L, Baxter S, Woods HB, Goyder E, Lee A, Payne N, et al. What is the evidence on interventions to manage referral from primary to specialist non‐emergency care? A systematic review and logic model synthesis. Health services and delivery research. 2015.
Bloomfield GS, Vedanthan R, Vasudevan L, Kithei A, Were M, Velazquez EJ. Mobile health for non‐communicable diseases in Sub‐Saharan Africa: a systematic review of the literature and strategic framework for research. Globalization and health. 2014;10(1):49.
Boksmati N, Butler‐Henderson K, Anderson K, Sahama T. The Effectiveness of SMS Reminders on Appointment Attendance: a Meta‐Analysis. Journal of medical systems. 2016;40(4):90.
Borchard A, Schwappach DL, Barbir A, Bezzola P. A systematic review of the effectiveness, compliance, and critical factors for implementation of safety checklists in surgery. Annals of surgery. 2012;256(6):925‐33.
Braet A, Weltens C, Sermeus W. Effectiveness of discharge interventions from hospital to home on hospital readmissions: a systematic review. JBI database of systematic reviews and implementation reports. 2016;14(2):106‐73.
Brata C, Gudka S, Schneider CR, Clifford RM. A review of the provision of appropriate advice by pharmacy staff for self‐medication in developing countries. Research in social & administrative pharmacy: RSAP. 2014;11(2):136‐53.
Byrne A, Morgan A. How the integration of traditional birth attendants with formal health systems can increase skilled birth attendance. International journal of gynaecology and obstetrics: the official organ of the International Federation of Gynaecology and Obstetrics. 2011;115(2):127‐34.
Campanella P, Vukovic V, Parente P, Sulejmani A, Ricciardi W, Specchia ML. The impact of Public Reporting on clinical outcomes: a systematic review and meta‐analysis. BMC health services research. 2016;16(1):296.
Carter EB, Temming LA, Akin J, Fowler S, Macones GA, Colditz GA, et al. Group Prenatal Care Compared With Traditional Prenatal Care: A Systematic Review and Meta‐analysis. Obstetrics and gynecology. 2016;128(3):551‐61.
Chhina HK, Bhole VM, Goldsmith C, Hall W, Kaczorowski J, Lacaille D. Effectiveness of academic detailing to optimize medication prescribing behaviour of family physicians. Journal of pharmacy & pharmaceutical sciences : a publication of the Canadian Society for Pharmaceutical Sciences, Société canadienne des sciences pharmaceutiques. 2013;16(4):511‐29.
Chin WY, Lam CL, Lo SV. Quality of care of nurse‐led and allied health personnel‐led primary care clinics. Hong Kong medical journal = Xianggang yi xue za zhi / Hong Kong Academy of Medicine. 2011;17(3):217‐30.
Chishinga N, Godfrey‐Faussett P, Fielding K, Ayles H. Effect of home‐based interventions on virologic outcomes in adults receiving antiretroviral therapy in Africa: a meta‐analysis. BMC public health. 2014;14(1):239.
Clark CE, Smith LF, Taylor RS, Campbell JL. Nurse led interventions to improve control of blood pressure in people with hypertension: systematic review and meta‐analysis. BMJ (Clinical research ed). 2010;341(7771):c3995.
Cobos Muñoz D, Merino Amador P, Monzon Llamas L, Martinez Hernandez D, Santos Sancho JM. Decentralization of health systems in low and middle income countries: a systematic review. International journal of public health. 2016.
Conn VS, Ruppar TM, Enriquez M, Cooper PS, Chan KC. Healthcare provider targeted interventions to improve medication adherence: systematic review and meta‐analysis. International journal of clinical practice. 2015;69(8):889‐99.
Davis R, Parand A, Pinto A, Buetow S. Systematic review of the effectiveness of strategies to encourage patients to remind healthcare professionals about their hand hygiene. The Journal of hospital infection. 2014;89(3):141‐62.
Davy C, Bleasel J, Liu H, Tchan M, Ponniah S, Brown A. Factors influencing the implementation of chronic care models: A systematic literature review. BMC family practice. 2015;16:102.
Decroo T, Rasschaert F, Telfer B, Remartinez D, Laga M, Ford N. Community‐based antiretroviral therapy programs can overcome barriers to retention of patients and decongest health services in sub‐Saharan Africa: a systematic review. International health. 2013;5(3):169‐79.
Dempsey E, Pammi M, Ryan AC, Barrington KJ. Standardised formal resuscitation training programmes for reducing mortality and morbidity in newborn infants. The Cochrane database of systematic reviews. 2015;9(9):CD009106.
Devi BR, Syed‐Abdul S, Kumar A, Iqbal U, Nguyen PA, Li YC, et al. mHealth: An updated systematic review with a focus on HIV/AIDS and tuberculosis long term management using mobile phones. Computer methods and programs in biomedicine. 2015;122(2):257‐65.
do Amaral JJF, Victora CG. The effect of training in Integrated Management of Childhood Illness (IMCI) on the performance and healthcare quality of pediatric healthcare workers: a systematic review. Revista Brasileira de Saúde Materno Infantil. 2008;8(2):151‐62.
Druetz T, Siekmans K, Goossens S, Ridde V, Haddad S. The community case management of pneumonia in Africa: a review of the evidence. Health policy and planning. 2013;30(2):253‐66.
Dzakpasu S, Powell‐Jackson T, Campbell OM. Impact of user fees on maternal health service utilization and related health outcomes: a systematic review. Health policy and planning. 2014;29(2):137‐50.
Eichler R, Agarwal K, Askew I, Iriarte E, Morgan L, Watson J. Performance‐based incentives to improve health status of mothers and newborns: what does the evidence show? Journal of health, population, and nutrition. 2013;31(4 Suppl 2):36‐47.
Elder E, Johnston AN, Crilly J. Review article: Systematic review of three key strategies designed to improve patient flow through the emergency department. Emergency medicine Australasia : EMA. 2015;27(5):394‐404.
Evans BA, Porter A, Gammon B, Mayes RH, Poulden M, Rees N, et al. A systematic review of rapid access models of care and their effects on delays in emergency departments. Emergency medicine journal : EMJ. 2015;32(6):e15‐6.
Free C, Phillips G, Galli L, Watson L, Felix L, Edwards P, et al. The effectiveness of mobile‐health technology‐based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS medicine. 2013;10(1):e1001362.
Gielen SC, Dekker J, Francke AL, Mistiaen P, Kroezen M. The effects of nurse prescribing: A systematic review. International journal of nursing studies. 2013;51(7):1048‐61.
Gillespie BM, Chaboyer W, Thalib L, John M, Fairweather N, Slater K. Effect of Using a Safety Checklist on Patient Complications after Surgery: A Systematic Review and Meta‐analysis. Anesthesiology. 2014;120(6):1380‐9.
Gilmore B, McAuliffe E. Effectiveness of community health workers delivering preventive interventions for maternal and child health in low‐ and middle‐income countries: a systematic review. BMC public health. 2013;13(1):847.
Gogia S, Sachdev HP. Home‐based neonatal care by community health workers for preventing mortality in neonates in low‐ and middle‐income countries: a systematic review. Journal of perinatology : official journal of the California Perinatal Association. 2016;36 Suppl 1(S1):S55‐73.
Gogia S, Sachdev HS. Home visits by community health workers to prevent neonatal deaths in developing countries: a systematic review. Bulletin of the World Health Organization. 2010;88(9):658‐66B.
Govindasamy D, Meghij J, Kebede Negussi E, Clare Baggaley R, Ford N, Kranzer K. Interventions to improve or facilitate linkage to or retention in pre‐ART (HIV) care and initiation of ART in low‐ and middle‐income settings ‐ a systematic review. Journal of the International AIDS Society. 2014;17(1):19032.
Hamine S, Gerth‐Guyette E, Faulx D, Green BB, Ginsburg AS. Impact of mHealth Chronic Disease Management on Treatment Adherence and Patient Outcomes: A Systematic Review. Journal of medical Internet research. 2015;17(2):e52.
Hecht L, Buhse S, Meyer G. Effectiveness of training in evidence‐based medicine skills for healthcare professionals: a systematic review. BMC medical education. 2016;16(1):103.
Hurt K, Walker RJ, Campbell JA, Egede LE. mHealth Interventions in Low and Middle‐Income Countries: A Systematic Review. Global journal of health science. 2016;8(9):54429.
Jia L, Meng Q, Yuan B, Fang L. Effects of drug cost sharing policy on the drug use, financial risks and moral hazard for the health insurance beneficiaries. Value in Health. 2014;17(7):A795.
Kamarudin G, Penm J, Chaar B, Moles R. Educational interventions to improve prescribing competency: a systematic review. BMJ open. 2013;3(8):e003291.
Kanters S, Park JJ, Chan K, Socias ME, Ford N, Forrest JI, et al. Interventions to improve adherence to antiretroviral therapy: a systematic review and network meta‐analysis. The lancet HIV. 2016.
Ke KM, Blazeby JM, Strong S, Carroll FE, Ness AR, Hollingworth W. Are multidisciplinary teams in secondary care cost‐effective? A systematic review of the literature. Cost effectiveness and resource allocation : C/E. 2013;11(1):7.
Khanal S, Burgon J, Leonard S, Griffiths M, Eddowes LA. Recommendations for the Improved Effectiveness and Reporting of Telemedicine Programs in Developing Countries: Results of a Systematic Literature Review. Telemedicine journal and e‐health : the official journal of the American Telemedicine Association. 2015;21(11):903‐15.
Kok MC, Dieleman M, Taegtmeyer M, Broerse JE, Kane SS, Ormel H, et al. Which intervention design factors influence performance of community health workers in low‐ and middle‐income countries? A systematic review. Health policy and planning. 2014;30(9):1207‐27.
Kondo KK, Damberg CL, Mendelson A, Motu'apuaka M, Freeman M, O'Neil M, et al. Implementation Processes and Pay for Performance in Healthcare: A Systematic Review. Journal of general internal medicine. 2016;31 Suppl 1:61‐9.
Korachais C, Macouillard E, Meessen B. How User Fees Influence Contraception in Low and Middle Income Countries: A Systematic Review. Studies in family planning. 2016;47(4):341‐56.
Körner M, Bütof S, Müller C, Zimmermann L, Becker S, Bengel J. Interprofessional teamwork and team interventions in chronic care: A systematic review. Journal of interprofessional care. 2015;30(1):1‐14.
Kurtzman ET, Greene J. Effective presentation of health care performance information for consumer decision making: A systematic review. Patient education and counseling. 2015;99(1):36‐43.
Lee IH, Bloor K, Hewitt C, Maynard A. International experience in controlling pharmaceutical expenditure: influencing patients and providers and regulating industry ‐ a systematic review. Journal of health services research & policy. 2014;20(1):52‐9.
Lee SH, Nurmatov UB, Nwaru BI, Mukherjee M, Grant L, Pagliari C. Effectiveness of mHealth interventions for maternal, newborn and child health in low‐ and middle‐income countries: Systematic review and meta‐analysis. Journal of global health. 2016;6(1):010401.
Lehnbom EC, Stewart MJ, Manias E, Westbrook JI. Impact of medication reconciliation and review on clinical outcomes. The Annals of pharmacotherapy. 2014;48(10):1298‐312.
L'Engle KL, Mangone ER, Parcesepe AM, Agarwal S, Ippoliti NB. Mobile Phone Interventions for Adolescent Sexual and Reproductive Health: A Systematic Review. Pediatrics. 2016;138(3):1‐16.
Lin Y, Yin S, Huang J, Du L. Impact of Pay for performance on Behavior of Primary Care Physicians and Patient Outcomes. Journal of evidence‐based medicine. 2015;9(1):8‐23.
Liu X, Dou L, Zhang H, Sun Y, Yuan B. Analysis of context factors in compulsory and incentive strategies for improving attraction and retention of health workers in rural and remote areas: a systematic review. Human resources for health. 2015;13:61.
Liu X, Hotchkiss DR, Bose S. The effectiveness of contracting‐out primary health care services in developing countries: A review of the evidence. Health Policy and Planning. 2007; 23(1): 1‐13.
Luangasanatip N, Hongsuwan M, Limmathurotsakul D, Lubell Y, Lee AS, Harbarth S, et al. Comparative efficacy of interventions to promote hand hygiene in hospital: systematic review and network meta‐analysis. BMJ (Clinical research ed). 2015;351:h3728.
Mann BS, Barnieh L, Tang K, Campbell DJ, Clement F, Hemmelgarn B, et al. Association between drug insurance cost sharing strategies and outcomes in patients with chronic diseases: a systematic review. PloS one. 2014;9(3):e89168.
Martínez‐González NA, Djalali S, Tandjung R, Huber‐Geismann F, Markun S, Wensing M, et al. Substitution of physicians by nurses in primary care: a systematic review and meta‐analysis. BMC health services research. 2014;14:214.
Martínez‐González NA, Rosemann T, Tandjung R, Djalali S. The effect of physician‐nurse substitution in primary care in chronic diseases: a systematic review. Swiss medical weekly. 2015;145(no pagination):w14031.
Martínez‐González NA, Tandjung R, Djalali S, Rosemann T. The impact of physician‐nurse task shifting in primary care on the course of disease: a systematic review. Human resources for health. 2015;13:55.
Mbuagbaw L, Sivaramalingam B, Navarro T, Hobson N, Keepanasseril A, Wilczynski NJ, et al. Interventions for Enhancing Adherence to Antiretroviral Therapy (ART): A Systematic Review of High Quality Studies. AIDS patient care and STDs. 2015;29(5):248‐66.
McCollum R, Gomez W, Theobald S, Taegtmeyer M. How equitable are community health worker programmes and which programme features influence equity of community health worker services? A systematic review. BMC public health. 2016;16(1):419.
McCulloch P, Rathbone J, Catchpole K. Interventions to improve teamwork and communications among healthcare staff. The British journal of surgery. 2011;98(4):469‐79.
McGrady ME, Ryan JL, Gutiérrez‐Colina AM, Fredericks EM, Towner EK, Pai AL. The impact of effective paediatric adherence promotion interventions: systematic review and meta‐analysis. Child: care, health and development. 2015;41(6):789‐802.
McMillan SS, Kendall E, Sav A, King MA, Whitty JA, Kelly F, et al. Patient‐centered approaches to health care: a systematic review of randomized controlled trials. Medical care research and review : MCRR. 2013;70(6):567‐96.
Mijovic H, McKnight J, English M. What does the literature tell us about health workers' experiences of task‐shifting projects in sub‐Saharan Africa? A systematic, qualitative review. Journal of clinical nursing. 2016;25(15‐16):2083‐100.
Montagu D, Goodman C, Berman P, Penn A, Visconti A. Recent trends in working with the private sector to improve basic healthcare: a review of evidence and interventions. Health policy and planning. 2016;31(8):1117‐32.
Musa BM, Iliyasu Z, Yusuf SM, Uloko AE. Systematic review and metanalysis on community based interventions in tuberculosis care in developing countries. Nigerian journal of medicine: journal of the National Association of Resident Doctors of Nigeria. 2014;23(2):103‐17.
Mwai GW, Mburu G, Torpey K, Frost P, Ford N, Seeley J. Role and outcomes of community health workers in HIV care in sub‐Saharan Africa: a systematic review. Journal of the International AIDS Society. 2013;16(1):18586.
Nazar H, Nazar Z, Portlock J, Todd A, Slight SP. A systematic review of the role of community pharmacies in improving the transition from secondary to primary care. British journal of clinical pharmacology. 2015;80(5):936‐48.
Nguyen DT, Leung KK, McIntyre L, Ghali WA, Sauve R. Does integrated management of childhood illness (IMCI) training improve the skills of health workers? A systematic review and meta‐analysis. PloS one. 2013;8(6):e66030.
Nijmeijer KJ, Fabbricotti IN, Huijsman R. Is franchising in health care valuable? A systematic review. Health policy and planning. 2014;29(2):164‐76.
Nilsson C, Lundgren I, Smith V, Vehvilainen‐Julkunen K, Nicoletti J, Devane D, et al. Women‐centred interventions to increase vaginal birth after caesarean section (VBAC): A systematic review. Midwifery. 2015;31(7):657‐63.
Ofek Shlomai N, Rao S, Patole S. Efficacy of interventions to improve hand hygiene compliance in neonatal units: a systematic review and meta‐analysis. European journal of clinical microbiology & infectious diseases : official publication of the European Society of Clinical Microbiology. 2015;34:887‐97.
Ogbechie OA, Hsu J. Systematic review of benefit designs with differential cost sharing for prescription drugs. The American journal of managed care. 2015;21(5):e338‐48.
Olisemeke B, Chen YF, Hemming K, Girling A. The Effectiveness of Service Delivery Initiatives at Improving Patients' Waiting Times in Clinical Radiology Departments: A Systematic Review. Journal of digital imaging. 2014;27(6):751‐78.
Owusu‐Addo E, Cross R. The impact of conditional cash transfers on child health in low‐ and middle‐income countries: a systematic review. International journal of public health. 2014;59(4):609‐18.
Pallas SW, Minhas D, Pérez‐Escamilla R, Taylor L, Curry L, Bradley EH. Community Health Workers in Low‐ and Middle‐Income Countries: What Do We Know About Scaling Up and Sustainability? American journal of public health. 2013;103(7):e74‐82.
Patel J, Ahmed K, Guru KA, Khan F, Marsh H, Shamim Khan M, et al. An overview of the use and implementation of checklists in surgical specialities ‐ a systematic review. International journal of surgery (London, England). 2014;12(12):1317‐23.
Rashidian A, Omidvari AH, Vali Y, Mortaz S, Yousefi‐Nooraie R, Jafari M, et al. The effectiveness of regionalization of perinatal care services‐‐a systematic review. Public health. 2014;128(10):872‐85.
Robinson DJ. An integrative review: triage protocols and the effect on ED length of stay. Journal of emergency nursing: JEN : official publication of the Emergency Department Nurses Association. 2013;39(4):398‐408.
Roque MD, Herdeiro MT, Soares SI, Teixeira Rodrigues A, Granadeiro LA, Gusman AF. Educational interventions to improve prescription and dispensing of antibiotics: a systematic review. BMC public health. 2014;14(1):1276.
Ruizendaal E, Dierickx S, Peeters Grietens K, Schallig HD, Pagnoni F, Mens PF. Success or failure of critical steps in community case management of malaria with rapid diagnostic tests: a systematic review. Malaria journal. 2014;13(1):229.
Russ S, Rout S, Sevdalis N, Moorthy K, Darzi A, Vincent C. Do safety checklists improve teamwork and communication in the operating room? A systematic review. Annals of surgery. 2013;258(6):856‐71.
Ruth Lv, Francke AL, Mistiaen P. Effects of nurse prescribing of medication: a systematic review. The Internet Journal of Healthcare Administration. 2008;5(2).
Sacks GD, Shannon EM, Dawes AJ, Rollo JC, Nguyen DK, Russell MM, et al. Teamwork, communication and safety climate: a systematic review of interventions to improve surgical culture. BMJ quality & safety. 2015;24(7):458‐67.
Santschi V, Chiolero A, Colosimo AL, Platt RW, Taffé P, Burnier M, et al. Improving blood pressure control through pharmacist interventions: a meta‐analysis of randomized controlled trials. Journal of the American Heart Association. 2014;3(2):e000718.
Schepman S, Hansen J, de Putter ID, Batenburg RS, de Bakker DH. The common characteristics and outcomes of multidisciplinary collaboration in primary health care: a systematic literature review. International journal of integrated care. 2015;15:e027.
Schmutz J, Manser T. Do team processes really have an effect on clinical performance? A systematic literature review. British journal of anaesthesia. 2013;110(4):529‐44.
Schweizer ML, Reisinger HS, Ohl M, Formanek MB, Blevins A, Ward MA, et al. Searching for an optimal hand hygiene bundle: a meta‐analysis. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America. 2014;58(2):248‐59.
Suwannakeeree W, Picheansathian W. Strategies to Promote Adherence to Treatment by Pulmonary Tuberculosis Patients: A systematic review. JBI Database of Systematic Reviews and Implementation Reports. 2012;10(11):615.
Sweeney S, Obure CD, Maier CB, Greener R, Dehne K, Vassall A. Costs and efficiency of integrating HIV/AIDS services with other health services: a systematic review of evidence and experience. Sexually transmitted infections. 2012;88(2):85‐99.
Thakkar J, Kurup R, Laba TL, Santo K, Thiagalingam A, Rodgers A, et al. Mobile Telephone Text Messaging for Medication Adherence in Chronic Disease: A Meta‐analysis. JAMA internal medicine. 2016;176(3):340‐9.
Trehan A, Maruthappu M, Barnett‐Vanes A, Carty M, McCulloch P. Does feedback of surgical outcome data improve surgical performance? A systematic review. Journal of the American College of Surgeons. 2014;219(4):e148.
Tripathi A, Kabra SK, Sachdev HP, Lodha R. Home visits by community health workers to improve identification of serious illness and care seeking in newborns and young infants from low‐ and middle‐income countries. Journal of perinatology : official journal of the California Perinatal Association. 2016;36 Suppl 1(S1):S74‐82.
Tshiananga JK, Kocher S, Weber C, Erny‐Albrecht K, Berndt K, Neeser K. The effect of nurse‐led diabetes self‐management education on glycosylated hemoglobin and cardiovascular risk factors: a meta‐analysis. The Diabetes educator. 2011;38(1):108‐23.
Tsiachristas A, Wallenburg I, Bond CM, Elliot RF, Busse R, van Exel J, et al. Costs and effects of new professional roles: Evidence from a literature review. Health policy (Amsterdam, Netherlands). 2015;119(9):1176‐87.
Turcotte‐Tremblay AM, Spagnolo J, De Allegri M, Ridde V. Does performance‐based financing increase value for money in low‐ and middle‐ income countries? A systematic review. Health economics review. 2015;6(1):30.
Uyei J, Coetzee D, Macinko J, Guttmacher S. Integrated delivery of HIV and tuberculosis services in Sub‐Saharan Africa: A systematic review. International Initiative for Impact Evaluation (3ie). 2011.
Van Camp YP, Van Rompaey B, Elseviers MM. Nurse‐led interventions to enhance adherence to chronic medication: systematic review and meta‐analysis of randomised controlled trials. European journal of clinical pharmacology. 2013;69(4):761‐70.
van Velthoven MHMMT, Tudor Car L, Gentry S, Car J. Telephone delivered interventions for preventing HIV infection in HIV‐negative persons. Cochrane Database of Systematic Reviews. 2013;5(5):CD009190.
Wagner B, Filice GA, Drekonja D, Greer N, MacDonald R, Rutks I, et al. Antimicrobial stewardship programs in inpatient hospital settings: a systematic review. Infection control and hospital epidemiology : the official journal of the Society of Hospital Epidemiologists of America. 2014;35(10):1209‐28.
Wald DS, Butt S, Bestwick JP. One‐way versus two‐way text messaging on improving medication adherence: meta‐analysis of randomized trials. The American journal of medicine. 2015;128(10):1139.e1‐5.
Wallace AS, Ryman TK, Dietz V. Experiences integrating delivery of maternal and child health services with childhood immunization programs: systematic review update. The Journal of infectious diseases. 2012;205 Suppl 1:S6‐19.
Wallace J, Byrne C, Clarke M. Improving the uptake of systematic reviews: a systematic review of intervention effectiveness and relevance. BMJ open. 2014;4(10):e005834.
Watson SJ, Aldus CF, Bond C, Bhattacharya D. Systematic review of the health and societal effects of medication organisation devices. BMC health services research. 2016;16(1):202.
Wilcher R, Hoke T, Adamchak SE, Cates W. Integration of family planning into HIV services: a synthesis of recent evidence. AIDS (London, England). 2013;27 Suppl 1:S65‐75.
World Health Organization, University of California SF. Electronic reminders for promoting adherence to ART among people living with HIV. World Health Organization. 2013.
World Health Organziation, University of California SF. Integration of HIV and TB services. World Health Organization. 2013.
Yamada J, Shorkey A, Barwick M, Widger K, Stevens BJ. The effectiveness of toolkits as knowledge translation strategies for integrating evidence into clinical care: a systematic review. BMJ open. 2015;5(4):e006808.
Zulu JM, Kinsman J, Michelo C, Hurtig AK. Integrating national community‐based health worker programmes into health systems: a systematic review identifying lessons learned from low‐and middle‐income countries. BMC public health. 2014;14(1):987.
Appendix 6. Summary of findings for interventions targeted at healthcare workers by type of interventiona
| Intervention | Definition | Effect on practice | Costs | Pros | Cons |
| Printed educational materials (PEMs) (Giguère 2012) | Distribution of printed recommendations for clinical care, including clinical practice guidelines. The materials can be delivered personally or through mass mailings | Printed educational materials may slightly improve practice. Absolute risk difference across various outcomes: 0.02 higher (range from −0.06 to 0.29). (ARD = 0.02 indicates an absolute improvement in adherence to the targeted behaviours of 2%) Standardised mean difference across various outcomes: 0.13 higher (range from −0.16 to 1.96). |
The costs of producing and distributing PEMs is likely to be highly variable and should be estimated in the context in which these interventions are being considered. | Printed educational materials provide familiar, accessible and convenient strategy to distribute evidence and recommendations for clinical care. | In low‐income countries (LICs) where health systems are weaker it may be difficult to obtain resources to produce up‐to‐date PEMs, and ensure that PEMs reach the appropriate providers in a timely way. "Printed educational materials frequently lead to small or no improvement in professional practice." |
| Internet‐based learning or computerised educational materials (Cook 2008) | Distribution of electronic recommendations for clinical care, including clinical practice guidelines. The materials are usually delivered through mass mailings and/or published on web pages | Internet‐based learning compared to no intervention may improve health worker knowledge, skills and behaviours. Knowledge (SMD 1.00, 95% CI 0.90 to 1.10); Skills (SMD 0.85, 95% CI 0.49 to 1.20); Behaviour (SMD 0.82, 95% CI 0.63 to 1.02). Internet‐based learning compared to alternative instructional media, including face‐to‐face teaching and learning may slightly improve knowledge, but has little or no effect on satisfaction, skills and behaviour. Satisfaction (SMD 0.10, 95% CI −0.12 to 0.32); Knowledge (SMD 0.12, 95% CI 0.003 to 0.24); Behaviour and effects on patient care (SMD 0.51, 95% CI −0.24 to 1.25) |
The costs of Internet‐based learning (e.g. reliable information technology infrastructure) may be high, limiting scale‐up and sustainability in low‐income countries. | Internet‐based learning permits learners to participate at a time and place convenient to them, facilitates innovation in instructional methods, and potentially allows instruction to be tailored to the individual's needs. Internet learning might reduce inequities when face‐to‐face learning is not an option for disadvantaged health professions and they have access to Internet‐based learning resources. |
Internet learning requires advanced information technology and knowledge which may be lacking in low‐income settings. |
| Educational meetings (Forsetlund 2009) | Lectures, workshops and courses | Educational meetings probably improve professional practice (median absolute improvement 6%, IQR 1.8% to 15.9%). | The cost of educational meetings is likely to be highly variable and must be estimated based on specific local conditions. | Integration of educational meetings in other quality improvement interventions (e.g. audit and feedback, educational outreach) may improve efficiency in resource use, cost savings and patient care. A mix of interactive small group work and didactic presentations may be more effective (compared to either alone). |
Resources needed for educational meetings may be less available in disadvantaged settings. Thus, additional resources may be needed to deliver effective educational meetings in disadvantaged settings to reduce inequities. Educational meetings may be less effective with lower attendance. Educational meetings may not be effective for complex behaviours. |
| Educational outreach (O'Brien 2007) | Personal visits by a trained person to healthcare workers in their own setting in order to provide information with the intent of improving practice. The information provided may include feedback about health worker performance. | Educational outreach improves appropriate prescribing and probably improves other types of practice. Appropriate prescribing: median improvement 4.8% (IQR 3.0% to 6.5%); Non‐prescribing practices: median improvement 6.0% (IQR 3.6% to 16.0%). |
The cost of educational outreach visits may limit scale‐up, although at least 1 study in a low‐resource setting in South Africa found that educational outreach visits for improving the quality of asthma care would be worthwhile and affordable. | It might be possible to reach professionals who would not attend educational meetings. | Educational outreach might be more challenging to organise than educational meetings. Educational outreach visits may require additional resources to provide clinical and managerial support. Some co‐interventions such as feedback about healthcare professionals' performance or reminders might require information systems that are not available in low‐resource settings. |
| Educational outreach ‐ practice facilitation (Baskerville 2012) | Personal visits by a trained person to healthcare workers in their own setting in order to provide information with the intent of improving practice. The information provided may include feedback about health worker performance. | Practice facilitation probably improves the adoption of evidence‐based guidelines. Moderate adherenceb : Difference: 21 more patients receiving recommended practice per 100 patient encounters (Margin of error: 17 to 24 more). Low adherencec : Difference: 21 more patients receiving recommended practice per 100 patient encounters (Margin of error: 15 to 26 more) |
The cost of practice facilitation is likely to be highly variable and must be estimated based on specific local conditions. | Poor adherence to evidence‐based guidelines often impacts more on disadvantaged populations. Practice facilitation as a strategy to improve evidence‐based guideline adoption could help these populations achieve the benefits of better quality service. | Practice facilitation might be difficult to implement in low‐resource settings, particularly the audit and feedback component, and it might be more difficult to make necessary organisational changes such as implementation of quality improvement tools. |
| Educational outreach ‐ Pharmacist provided services (Pande 2013) | Personal visits by a trained person to healthcare workers in their own setting in order to provide information with the intent of improving practice. The information provided may include feedback about health worker performance. | Provision of additional services by pharmacists and targeted at health professionals (such as educational outreach visits) can probably improve patient outcomes Effects on healthcare cost were uncertain (no studies were found) |
Pharmacist‐provided outreach services are likely to be expensive (e.g. high cost of training pharmacists) | Expanding the role of pharmacists could reduce inequalities if, for example, help from pharmacists is available when access to other healthcare professionals is limited | Expanding the role of pharmacists is dependent on having a workforce able to supply sufficient numbers of competent pharmacists, pharmacy technicians or assistants Regulatory frameworks are needed to allow pharmacists to extend their activities beyond their traditional professional responsibilities There may be too few pharmacists in low‐income countries and who may lack sufficient training or support to assume additional roles and responsibilities |
| Local opinion leaders (Flodgren 2011) | The identification and use of local opinion leaders to promote implementation of guidelines. | Local opinion leaders probably improve healthcare workers compliance with desired practice (median improvement 12%, IQR 6% to 14.5%). | Effective implementation of opinion leaders may be expensive due to high costs associated with supplementary interventions required for implementation (e.g. reminders, audit and feedback). | Use of opinion leaders may improve implementation of clinical guidelines as they are perceived by their peers as credible and trustworthy. | In most included studies opinion leaders were supplemented by other interventions such as reminders, faxed evidence summaries, and audit and feedback which rely on technologies that may not always be available in low‐income settings. Implementation of opinion leader interventions in low‐income settings, utilising such inaccessible technologies, may exacerbate health inequities or fail to address them adequately. |
| Audit and feedback (Ivers 2012) | A summary of healthcare workers' performance over a specified period of time given to them in written, electronic or verbal format. | Audit and feedback probably improves slightly professional practice: Median absolute increase in desired practice (dichotomous outcomes): 4.3% (IQR 0.5% to 16.0%) Median percentage increase in desired practice (continuous outcomes): 1.3% (IQR 1.3% to 28.9%) |
The cost of audit and feedback is likely to be highly variable and must be estimated based on specific local conditions, including the availability of reliable routinely collected data and personnel costs. | Audit and feedback may be more effective when baseline performance is low, when the source is a supervisor or senior colleague, when it is provided more than once, when it is provided both verbally and written, and when it includes both measurable targets and an action plan. | The feasibility of using audit and feedback may be limited by the scarcity of health professionals, availability of reliable data, the costs of collecting data, or low staff morale and a lack of motivation among health professionals. Resources needed for audit and feedback may be less easily available in disadvantaged populations. |
| Reminders (Ko 2011) | Manual or computerised interventions that prompt healthcare workers to perform some action. | Surgical safety checklists may reduce death rates and major complications within 30 days after surgery. It was uncertain whether safety checklists improve adherence to guidelines or patient safety in intensive care units, emergency departments or acute care settings |
There may be some additional costs involved in training and educating staff on how to use checklists, as well as the time taken to use checklists. | The benefit/cost ratio of checklists may be high where health provider compliance is adequate. | It is possible that resource levels, staff workloads, staff training and other factors could influence the effectiveness of patient safety checklists, and that they might be less effective in disadvantaged settings. |
| Tailored interventions (Baker 2015) | Interventions to change practice that are selected based on an assessment of barriers to change. | Tailored interventions probably improve adherence to guideline recommendationsd | Low‐cost methods can be used to tailor interventions, but some interventions to address identified barriers may be costly, limiting their use in low‐income countries | It is logical to tailor interventions to address barriers to change. | Little is known about how best to identify barriers to improving practice and how to tailor interventions to address these barriers. |
ARD: absolute risk difference; CI: confidence interval; IQR: interquartile range; LIC: low‐income country; LMIC: low‐ and middle‐income countries; RR: risk ratio; SMD: standardised mean difference. aThe effects of implementation strategies on patient outcomes are smaller than the effects on professional practice. This is because both implementation strategies and clinical interventions typically have moderate effects (improvements in practice), and the effect of implementation strategies on patient outcomes is the product of the effect on practice × the effect of that practice on patient outcomes. For example, consider an implementation strategy that improved practice by 10% (i.e. 10 more health professionals per 100 delivered an effective clinical intervention). That effect is larger than the median effect for most implementation strategies. Now assume that the clinical intervention improved patient outcomes by 10% (i.e. for every 10 patients treated, 1 additional patient benefited; a number needed to treat of 10). That is more effective than many, if not most, clinical interventions at which implementation strategies are targeted. The effect of the implementation strategy on patient outcomes then would be 10% x 10%, or 1 more patient per 100 in the intervention group having a desirable outcome compared to patients in the control group. Because of this, we have focused in this table on effects on practice rather than effects on patient outcomes. Improvements in practice are surrogates for improvements in patient outcomes. However, provided the certainty of the evidence for a clinical intervention being implemented is high, it is reasonable to assume that appropriate implementation of a recommendation to use that intervention will result in improved patient outcomes, albeit the improvements in patient outcomes will be smaller than the improvements in practice. bModerate adherence was assumed at 60% of desired practice. cLow adherence was assumed at 20% of desired practice. dThe odds ratio ranged from 1.08 to 12.25. The combined (average) odds ratio for 12 of the studies was 1.54 (95% CI 1.16 to 2.01), in favour of tailored interventions. In a situation where adherence with recommended practice was initially 60% this would correspond to an improvement to 70%. In a situation where adherence was initially 20% this would correspond to an improvement to 28%.
Appendix 7. Summary of findings for interventions targeted at healthcare workers by type of problema
| Intervention | Definition | Effect on practice | Costs | Pros | Cons |
| Communication with patients | Interventions that aim to promote patient‐centred care (PCC) in clinical consultations (Dwamena 2012) | Patient‐centred training probably improves the patient‐provider consultation process (provider consultation skills and behaviour): SMD 0.7, 95% CI 0.57 to 0.82. | The cost of promoting patient‐centred care is likely to be highly variable and must be estimated based on specific local conditions outside of research settings. | If interventions to promote patient centred care result in improved healthcare behaviours and outcomes, such as improved adherence to treatment, then these programmes may result in cost savings. | Human resource constraints in some health systems and low motivation to deliver patient‐centred care, may limit the feasibility and potential of this approach for improving professional practice and health outcomes. The additional resources needed to provide patient‐centred training and materials for providers and patients may be less easily available in disadvantaged settings, particularly where absolute access to providers is poor. Low literacy levels in some settings may also limit the applicability of written materials for patients. |
| Handwashing | Single or multifaceted intervention intended to improve compliance with hand hygiene (Gould 2010) | Educational interventions may increase hand hygiene compliance: 65% (25% to 120%) increase in hand hygiene compliance before patient contact, and 29% (6% to 56%) increase after patient contact. Multifaceted marketing campaigns may increase the use of hand hygiene products (range of increase 48% to 56%). |
The costs of most interventions are likely to be low except in instances in which washing facilities do not exist and need to be provided. The provision of hand hygiene facilities may not be expensive if locally sustainable solutions (such as the use of rainwater) are implemented. |
Healthcare units can create their own low‐cost interventions using the formula for alcohol‐based products. | Variable availability of functioning washing facilities or alcohol‐based products for health‐care workers could limit the applicability of the results in LICs. |
| Obstetric care | Non‐clinical interventions to reduce unnecessary cesarean section rates (nurse led relaxation, birth preparation classes for mothers, implementation of guidelines with mandatory second opinion and with support from local opinion leaders, and audit and feedback given to individual care providers) (Khunpradit 2011) | Non‐clinical interventions may decrease caesarean section rates. The effect of prenatal education and support programmes, computer patient decision aids, decision‐aid booklets and intensive group therapy on caesarean section rates is uncertain. |
Given that some interventions will involve changes in the guidelines such as having mandatory secondary opinion, training healthcare professionals this might result into considerable cost implications. Hence costs and potentials benefits need to be considered before adapting the intervention. | Interventions such as nurse led relaxation and birth preparation classes for mothers can be integrated in the existing mother to child health clinics. | Scarcity of health professionals may limit the feasibility and potential of interventions to reduce unnecessary caesarean sections in LICs (e.g. mandatory second opinion, birth preparation classes for mothers). |
| Training traditional birth attendants (TBAs) (Sibley 2012) | Training of TBAs may increase referral of pregnant women with obstetric complication (RR 1.45 (1.17 to 1.19). | In general, TBA training may be associated with high costs (e.g. requirement for suitably trained facilitators and locally adapted training materials). | TBAs are found widely in low‐ and middle‐income countries, and it is estimated that they may assist at up to 25% of all births in these settings. TBA training may have potential to reduce inequalities in health experienced by disadvantaged populations through facilitating timely referral of pregnant women where improved health services are available. However, further evidence on these impacts is required. |
TBA training may limit promotion of health facility deliveries. Implementation of TBA training requires an existing network of TBAs that can be targeted for further training and provided with support and acceptance of non‐professional providers within the formal health system. |
|
| Skilled birth attendance and emergency obstetric care (Yakoob 2011) | Professional practice outcomes not reported. Skilled birth attendance may reduce stillbirths (by 23%, 95% CI 15 to 31) and perinatal mortality (by 12%, 95% CI 5 to 18). |
Scaling up skilled birth attendance and access to emergency obstetric care will require considerable resources, particularly in more rural settings. However, the benefits of this may be substantial. | The beneficial effects of interventions are expected to be larger for underserved populations, therefore reducing inequalities. | Good access to these services are needed to optimise their benefits. | |
| Are birth kits a good idea? A systematic review of the evidence (Hundley 2012) | Professional practice outcomes not reported. The use of birth kits (alongside with education or topical antimicrobial) compared with no intervention: reduces neonatal tetanus related mortality, neonatal tetanus related mortality; probably reduces neonatal mortality rate, maternal mortality, haemorrhage; and may reduce neonatal sepsis. |
Scaling up birth kits introduction will require resources that should to be considered. The cost of items purchased separately, could be higher than as a kit. Re‐use of items could reduce costs | Birth kits may decrease inequity; however, resources needed for birth kits introduction may be less available in even more disadvantaged settings. | The availability acceptability and costs of the birth kits could limit feasibility among disadvantaged populations. | |
| Prescribing antibiotics (Ranji 2008) | Interventions to improve antibiotic prescribing in ambulatory settings – clinician education (e.g. educational meetings, audit and feedback) | Clinician education may reduce the number of patient visits at which an antibiotic is prescribed (by 9%, 95% CI 8.2 to 13). It is uncertain if clinician educational interventions improve the percentage of patients treated with adequate antibiotics. |
The costs of some interventions to promote appropriate antibiotic prescribing may be high, limiting feasibility in low‐income countries (e.g. educational meetings, audit and feedback). | These interventions could save costs arising from unnecessary medication and treatment of drug resistant strains of bacteria. | Scarcity of resources (health workers, educational materials) may limit feasibility of interventions to improve appropriate antibiotic prescribing. |
| Seriously ill newborn care (Opiyo 2015) | In‐service training of health professionals | In‐service neonatal emergency care training probably improves professional practices: Proportion of adequate initial resuscitation: difference 39 more adequate steps per 100 initial resuscitation steps (95% CI 21 to 66 more) Inappropriate and potentially harmful practices per resuscitation: mean difference 0.4 (95% CI 0.13 to 0.66) Preparedness for resuscitation: mean percentage improvement 8.83% (95% CI 6.41 to 11.25) |
In general, in‐service training is associated with considerable financial costs and may potentially disrupt the standard functioning of healthcare services. | In‐service training may result in improvements in quality of care and may be worthwhile if provided at a low cost. | It is possible that courses are offered predominantly to staff in large, central healthcare facilities. These facilities tend to be relatively better equipped and often benefit the better‐off disproportionately. This could therefore negatively affect the poor who often live in rural areas or are unable to access such healthcare facilities due to prohibitive fees. Feasibility of in‐service training in low‐income settings may be limited by the need to have adequately skilled instructors and appropriate, locally adapted training materials. |
| Quality of care for sexually transmitted diseases (STDs) and HIV (Sorsdahl 2009) | Educating traditional healers about STDs and HIV | Training of traditional healers may increase patient management practices, knowledge of STDs and HIV (signs and symptoms, prevention) and referral practice. Training may lead to little or no difference in the incidence of HIV/AIDS risk behaviour and traditional healers' self‐reported referral practices. |
Training is generally expensive (remuneration for trainers and facilitators, per diems, training materials). | Traditional healers are most common in traditional societies which tend to be rural and poor, and usually have poor access to other forms of healthcare. Improving the skills of traditional healers might therefore benefit these otherwise marginalised populations. | The nature of the practices employed by traditional healers and the degree to which healers are integrated into healthcare systems differs widely between societies. |
CI: confidence interval; LIC: low‐income country; RR: risk ratio; SMD: standardised mean difference; TBA: traditional birth attendant. aThe effects of implementation strategies on patient outcomes are smaller than the effects on professional practice. This is because both implementation strategies and clinical interventions typically have moderate effects (improvements in practice) and the effect of implementation strategies on patient outcomes is the product of the effect on practice × the effect of that practice on patient outcomes. For example, consider an implementation strategy that improved practice by 10% (i.e. 10 more health professionals per 100 delivered an effective clinical intervention). That effect is larger than the median effect for most implementation strategies. Now assume that the clinical intervention improved patient outcomes by 10% (i.e. for every 10 patients treated 1 additional patient benefited; a number needed to treat of 10). That is more effective than many, if not most, clinical interventions at which implementation strategies are targeted. The effect of the implementation strategy on patient outcomes then would be 10% x 10%, or 1 more patient per 100 in the intervention group having a desirable outcome compared to patients in the control group. Because of this, we have focused in this table on effects on practice rather than effects on patient outcomes. Improvements in practice are surrogates for improvements in patient outcomes. However, provided the certainty of the evidence for a clinical intervention being implemented is high, it is reasonable to assume that appropriate implementation of a recommendation to use that intervention will result in improved patient outcomes, albeit the improvements in patient outcomes will be smaller than the improvements in practice.
Contributions of authors
All of the authors contributed to drafting and revising the overview. All of the authors contributed important intellectual input to the overview.
Sources of support
Internal sources
Department of Family Medicine, School of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile.
Institute for Clinical Effectiveness and Health Policy, Buenos Aires, Argentina.
Norwegian Knowledge Centre for the Health Services, Oslo, Norway.
University of Cape Town, Cape Town, South Africa.
South African Medical Research Council, South Africa.
External sources
Norwegian Agency for Development Cooperation (Norad), Oslo, Norway.
The Effective Health Care Research Consortium which is funded by UK aid from the UK Government for the benefit of developing countries, UK.
Declarations of interest
Tomas Pantoja, Newton Opiyo, Simon Lewin, Elizabeth Paulsen, Cristian A Herrera, Gabriel Rada, and Andrew D Oxman are editors of the Cochrane Effective Practice and Organisation of Care (EPOC) Group. Newton Opiyo, Simon Lewin, Andrew D Oxman, and Charles S Wiysonge are authors on some of the included reviews. Agustín Ciapponi, Blanca Peñaloza, Lilian Dudley, Marie‐Pierre Gagnon, and Sebastian Garcia Marti have no relevant conflicts to declare.
New
References
References to included reviews
Augustincic Polec 2015
- Augustincic Polec L, O'Neill J, Welch V, Ueffing E, Tanjong Ghogomu E, Pardo Pardo J, et al. Strategies to increase the ownership and use of insecticide‐treated bednets to prevent malaria. Cochrane Database of Systematic Reviews 2015, Issue 3. [DOI: 10.1002/14651858.CD009186.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Baker 2015
- Baker R, Camosso‐Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to address determinants of practice. Cochrane Database of Systematic Reviews 2015, Issue 4. [DOI: 10.1002/14651858.CD005470.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]
Baskerville 2012
- Baskerville NB, Liddy C, Hogg W. Systematic review and meta‐analysis of practice facilitation within primary care settings. Annals of Family Medicine 2012;10(1):63‐74. [DOI] [PMC free article] [PubMed] [Google Scholar]
Berkman 2011
- Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Viera A, Crotty K, et al. Health literacy interventions and outcomes: an updated systematic review. Rockville (MD): Agency for Healthcare Research and Quality (prepared by RTI International–University of North Carolina Evidence‐based Practice Center under contract No. 290‐2007‐10056‐I); 2011 March. AHRQ Publication Number 11‐ E006.
Cook 2008
- Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet‐based learning in the health professions: a meta‐analysis. JAMA 2008;300(10):1181‐96. [DOI] [PubMed] [Google Scholar]
Dwamena 2012
- Jorgenson S, Sadigh G, Sikorskii A, Lewin S, Smith RC, Coffey J, Olomu A. Interventions for providers to promote a patient‐centred approach in clinical consultations. Cochrane Database of Systematic Reviews 2012, Issue 12. [DOI: 10.1002/14651858.CD003267.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Everett 2011
- Everett T, Bryant A, Griffin MF, Martin‐Hirsch PPL, Forbes CA, Jepson RG. Interventions targeted at women to encourage the uptake of cervical screening. Cochrane Database of Systematic Reviews 2011, Issue 5. [DOI: 10.1002/14651858.CD002834.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Flodgren 2011
- Flodgren G, Parmelli E, Doumit G, Gattellari M, O'Brien MA, Grimshaw J, et al. Local opinion leaders: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2011, Issue 8. [DOI: 10.1002/14651858.CD000125.pub4] [DOI] [PMC free article] [PubMed] [Google Scholar]
Forsetlund 2009
- Forsetlund L, Bjørndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf F, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2009, Issue 2. [DOI: 10.1002/14651858.CD003030.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Giguère 2012
- Giguère A, Légaré F, Grimshaw J, Turcotte S, Fiander M, Grudniewicz A, et al. Printed educational materials: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews 2012, Issue 10. [DOI: 10.1002/14651858.CD004398.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]
Gould 2010
- Gould DJ, Moralejo D, Drey N, Chudleigh JH. Interventions to improve hand hygiene compliance in patient care. Cochrane Database of Systematic Reviews 2010, Issue 9. [DOI: 10.1002/14651858.CD005186.pub3] [DOI] [PubMed] [Google Scholar]
Haynes 2008
- Haynes RB, Ackloo E, Sahota N, McDonald HP, Yao X. Interventions for enhancing medication adherence. Cochrane Database of Systematic Reviews 2008, Issue 2. [DOI: 10.1002/14651858.CD000011.pub3] [DOI] [PubMed] [Google Scholar]
Horsley 2011
- Horsley T, Hyde C, Santesso N, Parkes J, Milne R, Stewart R. Teaching critical appraisal skills in healthcare settings. Cochrane Database of Systematic Reviews 2011, Issue 11. [DOI: 10.1002/14651858.CD001270.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Hundley 2012
- Hundley VA, Avan BI, Braunholtz D, Graham WJ. Are birth kits a good idea? A systematic review of the evidence. Midwifery 2012;28(2):204‐15. [DOI] [PubMed] [Google Scholar]
Ivers 2012
- Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard‐Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews 2012, Issue 6. [DOI: 10.1002/14651858.CD000259.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]
Jacobson Vann 2005
- Jacobson Vann JC, Szilagyi P. Patient reminder and patient recall systems for improving immunization rates. Cochrane Database of Systematic Reviews 2005, Issue 3. [DOI: 10.1002/14651858.CD003941.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Jia 2014
- Jia L, Yuan B, Lu Y, Garner P, Meng Q, Huang F. Strategies for expanding health insurance coverage in vulnerable populations. Cochrane Database of Systematic Reviews 2014, Issue 11. [DOI: 10.1002/14651858.CD008194.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Khunpradit 2011
- Khunpradit S, Tavender E, Lumbiganon P, Laopaiboon M, Wasiak J, Gruen RL. Non‐clinical interventions for reducing unnecessary caesarean section. Cochrane Database of Systematic Reviews 2011, Issue 6. [DOI: 10.1002/14651858.CD005528.pub2] [DOI] [PubMed] [Google Scholar]
Ko 2011
- Ko HC, Turner TJ, Finnigan MA. Systematic review of safety checklists for use by medical care teams in acute hospital settings‐limited evidence of effectiveness. BMC Health Serv Res 2011;11:211. [DOI] [PMC free article] [PubMed] [Google Scholar]
Liu 2014
- Liu Q, Abba K, Alejandria MM, Sinclair D, Balanag VM, Lansang MAD. Reminder systems to improve patient adherence to tuberculosis clinic appointments for diagnosis and treatment. Cochrane Database of Systematic Reviews 2014, Issue 11. [DOI: 10.1002/14651858.CD006594.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]
Lutge 2015
- Lutge EE, Wiysonge CS, Knight SE, Sinclair D, Volmink J. Incentives and enablers to improve adherence in tuberculosis. Cochrane Database of Systematic Reviews 2015, Issue 9. [DOI: 10.1002/14651858.CD007952.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]
Mbuagbaw 2013
- Mbuagbaw L, Kop ML, Lester RT, Thirumurthy H, Pop‐Eleches C, Ye C, et al. Mobile phone text messages for improving adherence to antiretroviral therapy (ART): an individual patient data meta‐analysis of randomised trials. BMJ Open 2013;3:e003950. [DOI: 10.1136/bmjopen-2013-003950] [DOI] [PMC free article] [PubMed] [Google Scholar]
Nicolson 2009
- Nicolson DJ, Knapp P, Raynor DK, Spoor P. Written information about individual medicines for consumers. Cochrane Database of Systematic Reviews 2009, Issue 2. [DOI: 10.1002/14651858.CD002104.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]
O'Brien 2007
- O'Brien MA, Rogers S, Jamtvedt G, Oxman AD, Odgaard‐Jensen J, Kristoffersen DT, et al. Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2007, Issue 4. [DOI: 10.1002/14651858.CD000409.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Opiyo 2015
- Opiyo N, English M. Opiyo N, English M. In‐service training for health professionals to improve care of seriously ill newborns and children in low‐income countries. Cochrane Database of Systematic Reviews 2015, Issue 5. [DOI: 10.1002/14651858.CD007071.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]
Orton 2005
- Orton LC, Barnish G. Unit‐dose packaged drugs for treating malaria. Cochrane Database of Systematic Reviews 2005, Issue 2. [DOI: 10.1002/14651858.CD004614.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Oyo‐Ita 2016
- Oyo‐Ita A, Wiysonge C, Oringanje C, Nwachukwu CE, Oduwole O, Meremikwu MM. Interventions for improving coverage of child immunisation in low‐ and middle‐income countries. Cochrane Database of Systematic Reviews 2016, Issue 7. [DOI: 10.1002/14651858.CD008145.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]
Pande 2013
- Pande S, Hiller JE, Nkansah N, Bero L. The effect of pharmacist‐provided non‐dispensing services on patient outcomes, health service utilisation and costs in low‐ and middle‐income countries. Cochrane Database of Systematic Reviews 2013, Issue 2. [DOI: 10.1002/14651858.CD010398] [DOI] [PMC free article] [PubMed] [Google Scholar]
Parmelli 2011
- Parmelli E, Flodgren G, Schaafsma ME, Baillie N, Beyer FR, Eccles MP. The effectiveness of strategies to change organisational culture to improve healthcare performance. Cochrane Database of Systematic Reviews 2011, Issue 1. [DOI: 10.1002/14651858.CD008315.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Perrier 2011
- Perrier L, Mrklas K, Shepperd S, Dobbins M, McKibbon KA, Straus SE. Interventions encouraging the use of systematic reviews in clinical decision‐making: a systematic review. Journal of General Internal Medicine 2011;26(4):419‐26. [DOI] [PMC free article] [PubMed] [Google Scholar]
Ranji 2008
- Ranji SR, Steinman M, Shojania K, Gonzalez R. Interventions to reduce unnecessary antibiotic prescribing. A systematic review and quantitative analysis. Medical Care 2008;46(8):847‐62. [DOI] [PubMed] [Google Scholar]
Reeves 2013
- Reeves S, Perrier L, Goldman J, Freeth D, Zwarenstein M. Interprofessional education: effects on professional practice and healthcare outcomes (update). Cochrane Database of Systematic Reviews 2013, Issue 3. [DOI: 10.1002/14651858.CD002213.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]
Sibley 2012
- Sibley LM, Sipe TA, Barry D. Traditional birth attendant training for improving health behaviours and pregnancy outcomes. Cochrane Database of Systematic Reviews 2012, Issue 8. [DOI: 10.1002/14651858.CD005460.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]
Simoni 2006
- Simoni JM, Pearson CR, Pantalone DW, Marks G, Crepaz N. Efficacy of interventions in improving highly active antiretroviral therapy adherence and HIV‐1 RNA viral load. A meta‐analytic review of randomized controlled trials. Journal of Acquired Immune Deficiency Syndrome 2006;43(Suppl 1):S23‐35. [DOI] [PMC free article] [PubMed] [Google Scholar]
Simoni 2010
- Simoni JM, Amico KR, Smith L, Nelson K. Antiretroviral adherence interventions: translating research findings to the real world clinic. Current HIV/AIDS Reports 2010;7(1):44‐51. [DOI] [PMC free article] [PubMed] [Google Scholar]
Sorsdahl 2009
- Sorsdahl K, Ipser JC, Stein DJ. Interventions for educating traditional healers about STD and HIV medicine. Cochrane Database of Systematic Reviews 2009, Issue 4. [DOI: 10.1002/14651858.CD007190.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Sunguya 2013
- Sunguya BF, Poudel KC, Mlunde LB, Shakya P, Urassa DP, Jimba M, et al. Effectiveness of nutrition training of health workers toward improving caregivers' feeding practices for children aged six months to two years: a systematic review. Nutrition Journal 2013 May 20;12:66. [DOI] [PMC free article] [PubMed] [Google Scholar]
Vidanapathirana 2005
- Vidanapathirana J, Abramson MJ, Forbes A, Fairley C. Mass media interventions for promoting HIV testing. Cochrane Database of Systematic Reviews 2005, Issue 3. [DOI: 10.1002/14651858.CD004775.pub2] [DOI] [PubMed] [Google Scholar]
Yakoob 2011
- Yakoob MY, Ali MA, Ali MU, Imdad A, Lawn JE, Broek N, Bhutta ZA. The effect of providing skilled birth attendance and emergency obstetric care in preventing stillbirths. BMC Public Health 2011;11(Suppl 3):S7. [DOI] [PMC free article] [PubMed] [Google Scholar]
References to excluded reviews
Arditi 2012
- Arditi C, Rège‐Walther M, Wyatt JC, Durieux P, Burnand B. Computer‐generated reminders delivered on paper to healthcare professionals; effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2012, Issue 12. [DOI: 10.1002/14651858.CD001175.pub3] [DOI] [PubMed] [Google Scholar]
Beilby 1997
- Beilby JJ, Silagy CA. Trials of providing costing information to general practitioners: a systematic review. Medical Journal of Australia 1997;167(2):89‐92. [DOI] [PubMed] [Google Scholar]
Bordley 2000
- Bordley WC, Chelminski A, Margolis PA, Kraus R, Szilagyi PG, Vann JJ. The effect of audit and feedback on immunization delivery: a systematic review. American Journal of Preventive Medicine 2000;18(4):343‐50. [DOI] [PubMed] [Google Scholar]
Chaillet 2006
- Chaillet N, Dubé E, Dugas M, Audibert F, Tourigny C, Fraser WD, Dumont A. Evidence‐based strategies for implementing guidelines in obstetrics: a systematic review. Obstetrics & Gynecology 2006;108(5):1234‐45. [DOI] [PubMed] [Google Scholar]
Fudickar 2012
- Fudickar A, Hörle K, Wiltfang J, Bein B. The effect of the WHO surgical safety checklist on complication rate and communication. Dtsch Arztebl Int 2012;109(42):695‐701. [DOI] [PMC free article] [PubMed] [Google Scholar]
Grilli 2002
- Grilli R, Ramsay C, Minozzi S. Mass media interventions: effects on health services utilisation. Cochrane Database of Systematic Reviews 2002, Issue 1. [DOI: 10.1002/14651858.CD000389] [DOI] [PubMed] [Google Scholar]
Grimshaw 2004
- Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assessment 2004;8(6):1‐72. [DOI] [PubMed] [Google Scholar]
Horvath 2012
- Horvath T, Azman H, Kennedy GE, Rutherford GW. Mobile phone text messaging for promoting adherence to antiretroviral therapy in patients with HIV infection. Cochrane Database of Systematic Reviews 2012, Issue 3. [DOI: 10.1002/14651858.CD009756] [DOI] [PMC free article] [PubMed] [Google Scholar]
Hulscher 2001
- Hulscher ME, Wensing M, Weijden T, Grol R. Interventions to implement prevention in primary care. Cochrane Database of Systematic Reviews 2001, Issue 1. [DOI: 10.1002/14651858.CD000362] [DOI] [PubMed] [Google Scholar]
Ioannidis 2001
- Ioannidis JP, Lau J. Evidence on interventions to reduce medical errors: an overview and recommendations for future research. Journal of General Internal Medicine 2001;16(5):325‐34. [DOI] [PMC free article] [PubMed] [Google Scholar]
Jayaraman 2009
- Jayaraman S, Sethi D. Advanced trauma life support training for hospital staff. Cochrane Database of Systematic Reviews 2009, Issue 2. [DOI: 10.1002/14651858.CD004173.pub3] [DOI] [PubMed] [Google Scholar]
Jayaraman 2010
- Jayaraman S, Sethi D. Advanced trauma life support training for ambulance crews. Cochrane Database of Systematic Reviews 2010, Issue 1. [DOI: 10.1002/14651858.CD003109.pub2] [DOI] [PubMed] [Google Scholar]
Kendrick 2000
- Kendrick D, Hewitt M, Dewey M, Elkan R, Blair M, Robinson J, et al. The effect of home visiting programmes on uptake of childhood immunization: a systematic review and meta‐analysis. Journal of Public Health Medicine 2000;22(1):90‐8. [DOI] [PubMed] [Google Scholar]
Lam‐Antoniades 2009
- Lam‐Antoniades M, Ratnapalan S, Tait G. Electronic continuing education in the health professions: an update on evidence from RCTs. Journal of Continuing Education in the Health Professions 2009;29(1):44‐51. [DOI] [PubMed] [Google Scholar]
Lee 2009
- Lee ACC, Lawn JE, Cousens S, Kumar V, Osrin D, Bhutta ZA, et al. Linking families and facilities for care at birth: What works to avert intrapartum‐related deaths?. International Journal of Gynecology & Obstetrics 2009;107:65‐88. [DOI] [PMC free article] [PubMed] [Google Scholar]
Legare 2010
- Legare F, Ratte S, Stacey D, Kryworuchko J, Gravel K, Graham ID, Turcotte S. Interventions for improving the adoption of shared decision making by healthcare professionals. Cochrane Database of Systematic Reviews 2010, Issue 5. [DOI: 10.1002/14651858.CD006732.pub2] [DOI] [PubMed] [Google Scholar]
Minkman 2007
- Minkman M, Ahaus K, Huijsman R. Performance improvement based on integrated quality management models: what evidence do we have? A systematic literature review. International Journal for Quality in Health Care 2007;19(2):90‐104. [DOI] [PubMed] [Google Scholar]
Naikoba 2001
- Naikoba S, Hayward A. The effectiveness of interventions aimed at increasing handwashing in healthcare workers ‐ a systematic review. Journal of Hospital Infections 2001;47(3):173‐80. [DOI] [PubMed] [Google Scholar]
Nglazi 2013
- Nglazi MD, Bekker LG, Wood R, Hussey GD, Wiysonge CS. Mobile phone text messaging for promoting adherence to anti‐tuberculosis treatment: a systematic review. BMC Infectious Diseases 2013;13:566. [DOI: 10.1186/1471-2334-13-566.] [DOI] [PMC free article] [PubMed] [Google Scholar]
Pattinson 2005
- Pattinson RC, Say L, Makin JD, Bastos MH. Critical incident audit and feedback to improve perinatal and maternal mortality and morbidity. Cochrane Database of Systematic Reviews 2005, Issue 4. [DOI: 10.1002/14651858.CD002961.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Pearce 2012
- Pearce J, Mann MK, Jones C, Buschbach S, Olff M, Bisson JI. The most effective way of delivering a train‐the‐trainers program: a systematic review. Journal of Continuing Education in the Health Professions 2012;32(3):215‐26. [DOI] [PubMed] [Google Scholar]
Rowe 2002
- Rowe RE, Garcia J, Macfarlane AJ, Davidson LL. Improving communication between health professionals and women in maternity care: a structured review. Health Expectations 2002;5(1):63‐83. [DOI] [PMC free article] [PubMed] [Google Scholar]
Rueda 2006
- Rueda S, Park‐Wyllie LY, Bayoumi AM, Tynan AM, Antoniou TA, Rourke SB, et al. Patient support and education for promoting adherence to highly active antiretroviral therapy for HIV/AIDS. Cochrane Database of Systematic Reviews 2006, Issue 3. [DOI: 10.1002/14651858.CD001442.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Safdar 2008
- Safdar N, Abad C. Educational interventions for prevention of healthcare‐associated infection: a systematic review. Critical Care Medicine 2008;36(3):933‐40. [DOI] [PubMed] [Google Scholar]
Shea 2009
- Shea B, Andersson N, Henry D. Increasing the demand for childhood vaccination in developing countries: a systematic review. BMC International Health and Human Rights 2009;9(Suppl 1):S5. [DOI] [PMC free article] [PubMed] [Google Scholar]
Shojania 2009
- Shojania KG, Jennings A, Mayhew A, Ramsay CR, EcclesMP, Grimshaw J. The effects of on‐screen, point of care computer reminders on processes and outcomes of care. Cochrane Database of Systematic Reviews 2009, Issue 3. [DOI: 10.1002/14651858.CD001096.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Smith 2009
- Smith LA, Jones C, Meek S, Webster J. Provider practice and user behavior interventions to improve prompt and effective treatment of malaria: do we know what works?. American Journal of Tropical Medicine and Hygiene 2009;80(3):326‐35. [PubMed] [Google Scholar]
Smits 2002
- Smits PB, Verbeek JH, Buisonjé CD. Problem based learning in continuing medical education: a review of controlled evaluation studies. BMJ 2002;324:153‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]
Sowden 2000
- Sowden AJ. Mass media interventions for preventing smoking in young people. Cochrane Database of Systematic Reviews 2000, Issue 2. [DOI: 10.1002/14651858.CD001006] [DOI] [PubMed] [Google Scholar]
Turner 2005
- Turner T, Harris C, Molyneux E. What is the evidence for effectiveness of WHO guidelines for the care of children in hospitals in developing countries?. International Child Health Review Collaboration 2005.
Wafula 2010
- Wafula FN, Goodman CA. Are interventions for improving the quality of services provided by specialized drug shops effective in sub‐Saharan Africa? A systematic review. International Journal for Quality in Health Care 2010;22(4):316‐23. [DOI] [PMC free article] [PubMed] [Google Scholar]
Zedler 2011
- Zedler BK, Kakad P, Colilla S, Murrelle L, Shah NR. Does packaging with a calendar feature improve adherence to self‐administered medication for long‐term use? A systematic review. Clinical Therapy 2011;33(1):62‐73. [DOI: 10.1016/j.clinthera.2011.02.003] [DOI] [PubMed] [Google Scholar]
Additional references
Abraham 2008
- Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychology 2008;27(3):379‐87. [DOI] [PubMed] [Google Scholar]
Althabe 2008
- Althabe F, Bergel E, Cafferata ML, Gibbons L, Ciapponi A, Alemán A, et al. Strategies for improving the quality of health care in maternal and child health in low‐ and middle‐income countries: an overview of systematic reviews. Paediatric Perinatal Epidemiology 2008;22(Suppl 1):42‐60. [DOI] [PubMed] [Google Scholar]
Arnold 2005
- Arnold SR, Straus SE. Interventions to improve antibiotic prescribing practices in ambulatory care. Cochrane Database of Systematic Reviews 2005, Issue 4. [DOI: 10.1002/14651858.CD003539.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Baldwin 2011
- Baldwin K, Namdari S, Donegan D, Kamath AF, Mehta S. Early effects of resident work‐hour restrictions on patient safety: a systematic review and plea for improved studies. Journal of Bone and Joint Surgery 2011;93(2):e5. [DOI] [PubMed] [Google Scholar]
Bero 1998
- Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA. Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. BMJ 1998;317(7156):465‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]
Bloom 2005
- Bloom BS. Effects of continuing medical education on improving physician clinical care and patient health: A review of systematic reviews. International Journal of Technology Assessment in Health Care 2005;21(3):380‐5. [DOI] [PubMed] [Google Scholar]
Boaz 2011
- Boaz A, Baeza J, Fraser A, European Implementation Score Collaborative Group (EIS). Effective implementation of research into practice: an overview of systematic reviews of the health literature. BMC Research Notes 2011;4:212. [DOI] [PMC free article] [PubMed] [Google Scholar]
Brennan 2009
- Brennan S, McKenzie J, Whitty P, Buchan H, Green S. Continuous quality improvement: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews 2009, Issue 4. [DOI: 10.1002/14651858.CD003319.pub2] [DOI] [Google Scholar]
Chan 2017
- Chan WV, Pearson TA, Bennett GC, Castillo G, Cushman WC, Gaziano TA, et al. ACC/AHA Special Report: Clinical Practice Guideline Implementation Strategies: A Summary of Systematic Reviews by the NHLBI Implementation Science Work Group: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines [1076‐1092]. Journal of the American College of Cardiology 2017;69(8):1076‐1092. [DOI] [PubMed] [Google Scholar]
Cheung 2012
- Cheung A, Weir M, Mayhew A, Kozloff N, Brown K, Grimshaw J. Overview of systematic reviews of the effectiveness of reminders in improving healthcare professional behavior. Systematic Reviews 2012;1:36. [DOI] [PMC free article] [PubMed] [Google Scholar]
Ciapponi 2014
- Ciapponi A, Lewin S, Bastías G, Dudley L, Flottorp S, Gagnon MP, et al. Delivery arrangements for health systems in low‐income countries: an overview of systematic reviews. Cochrane Database of Systematic Reviews 2014, Issue 5. [DOI: 10.1002/14651858.CD011083] [DOI] [PMC free article] [PubMed] [Google Scholar]
Davey 2013
- Davey P, Brown E, Charani E, Fenelon L, Gould IM, Holmes A, et al. Interventions to improve antibiotic prescribing practices for hospital inpatients. Cochrane Database of Systematic Reviews 2013, Issue 4. [DOI: 10.1002/14651858.CD003543.pub3] [DOI] [PubMed] [Google Scholar]
Dolan 2010
- Dolan P, Hallsworth M, Halpern D, King D, Vlaev I. MINDSPACE: Influencing behaviour through public policy. UK Cabinet Office & Institute for Government. 2010. Available from www.instituteforgovernment.org.uk/publications/mindspace.
Dudley 2009
- Dudley L, Hviding K, Paulsen E. The effectiveness of policies promoting facility‐based deliveries in reducing maternal and infant morbidity and mortality in low‐ and middle‐income countries. Cochrane Database of Systematic Reviews 2009, Issue 3. [DOI: 10.1002/14651858.CD007918] [DOI] [Google Scholar]
EPOC 2017
- Cochrane Effective Practice, Organisation of Care (EPOC). EPOC resources for review authors, 2017. Available at epoc.cochrane.org/epoc‐resources‐review‐authors.
Flottorp 2013
- Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki‐Cwirko M, et al. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implementation Science 2013;8:35. [DOI] [PMC free article] [PubMed] [Google Scholar]
French 2012
- French SD, Green SE, O'Connor DA, McKenzie JE, Francis JJ, Michie S, et al. Developing theory‐informed behaviour change interventions to implement evidence into practice: a systematic approach using theTheoretical Domains Framework. Implementation Science 2012;7:38. [DOI] [PMC free article] [PubMed] [Google Scholar]
Fretheim 2009
- Fretheim A, Oxman AD, Lavis JN, Lewin S. SUPPORT Tools for evidence‐informed health Policymaking (STP). 18. Planning monitoring and evaluation of policies. Health Research Policy and Systems 2009;7(Suppl 1):S18. [DOI] [PMC free article] [PubMed] [Google Scholar]
Fønhus 2016
- Fønhus MS, Dalsbø TK, Johansen M, Fretheim A, Skirbekk H, Flottorp S. Patient‐mediated interventions to improve professional practice. Cochrane Database of Systematic Reviews 2016, Issue 12. [DOI: 10.1002/14651858.CD012472] [DOI] [PMC free article] [PubMed] [Google Scholar]
Graham 2011
- Graham W, Davidson L, Campbell S, Clar C. What are the effects of interventions to improve the uptake of evidence from health research into policy in low‐ and middle‐income countries?. Aberdeen (UK): University of Aberdeen; 2011 January. DFID systematic review report Nº 40032846.
Grimshaw 2001
- Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior: an overview of systematic reviews of interventions. Medical Care 2001;39(8 Suppl 2):II2‐45. [PubMed] [Google Scholar]
Grol 1997
- Grol R. Beliefs and evidence in changing clinical practice. BMJ 1997;315(7105):418‐21. [DOI] [PMC free article] [PubMed] [Google Scholar]
Grol 2003
- Grol R, Grimshaw JM. From best evidence to best practice: effective implementation of change in patients' care. Lancet 2003;362(9391):1225‐30. [DOI] [PubMed] [Google Scholar]
Grol 2005
- Grol R, Wensing M. Chapter 8: Selection of strategies. In: Grol R, Wensing M, Eccles M editor(s). Improving Patient Care. The Implementation of Change in Clinical Practice. 1st Edition. Edinburgh: Elsevier Butterworth Heinemann, 2005:122‐134. [Google Scholar]
Grol 2007
- Grol RP, Bosch MC, Hulscher ME, Eccles MP, Wensing M. Planning and studying improvement in patient care: the use of theoretical perspectives. Milbank Quarterly 2007;85(1):93‐138. [DOI] [PMC free article] [PubMed] [Google Scholar]
Guyatt 2008
- Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck‐Ytter Y, Schunemann HJ, et al. What is "quality of evidence" and why is it important to clinicians?. BMJ 2008;336(7651):995‐98. [DOI] [PMC free article] [PubMed] [Google Scholar]
Hall 2015
- Hall AK, Cole‐Lewis H, Bernhardt JM. Mobile text messaging for health: a systematic review of reviews. Annual Review of Public Health 2015;36:393‐415. [DOI] [PMC free article] [PubMed] [Google Scholar]
Herrera 2014
- Herrera CA, Ciapponi A, Bastías G, Lewin S, Garcia Marti S, Okwundu CI, et al. Governance arrangements for health systems in low‐income countries: an overview of systematic reviews. Cochrane Database of Systematic Reviews 2014, Issue 4. [DOI: 10.1002/14651858.CD011085] [DOI] [PMC free article] [PubMed] [Google Scholar]
Krause 2014
- Krause J, Lieshout J, Klomp R, Huntink E, Aakhus E, Flottorp S, et al. Identifying determinants of care for tailoring implementation in chronic diseases: an evaluation of different methods. Implementation Science 2014;9:102. [DOI: ; PUBMED: 25112492] [DOI] [PMC free article] [PubMed] [Google Scholar]
Lavis 2009
- Lavis JN, Oxman AD, Souza NM, Lewin S, Gruen RL, Fretheim A. SUPPORT Tools for evidence‐informed health policymaking (STP) 9: assessing the applicability of the findings of asystematic review. Health Research Policy and Systems 2009;7 Suppl 1:S9. [DOI] [PMC free article] [PubMed] [Google Scholar]
Lavis 2015
- Lavis JN, Wilson MG, Moat KA, Hammill AC, Boyko JA, Grimshaw JM, et al. Developing and refining the methods for a 'one‐stop shop' for research evidence about health systems. Health Research Policy and Systems 2015;13(10):1‐10. [DOI] [PMC free article] [PubMed] [Google Scholar]
Lewin 2008
- Lewin S, Lavis JN, Oxman AD, Bastías G, Chopra M, Ciapponi A, et al. Supporting the delivery of cost‐effective interventions in primary health‐care systems in low‐income and middle‐income countries: an overview of systematic reviews. Lancet 2008;372(9642):928‐39. [DOI] [PubMed] [Google Scholar]
Mahtani 2011
- Mahtani KR, Heneghan CJ, Glasziou PP, Perera R. Reminder packaging for improving adherence to self‐administered long‐term medications. Cochrane Database of Systematic Reviews 2011, Issue 9. [DOI: 10.1002/14651858.CD005025.pub3] [DOI] [PubMed] [Google Scholar]
Mauger Rothenberg 2012
- Mauger Rothenberg B, Marbella A, Pines E, Chopra R, Black ER, Aronson N. Prevention of healthcare‐associated infections. Closing the quality gap: revisiting the state of the science. AHRQ Publication No. 12(13)‐E012‐EF 2012. [PMC free article] [PubMed]
McGlynn 2003
- McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, et al. The quality of health care delivered to adults in the United States. New England Journal of Medicine 2003;348:2635‐45. [DOI] [PubMed] [Google Scholar]
Michie 2005
- Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A. Making psychological theory useful for implementing evidence based practice: a consensus approach. Quality & Safety in Health Care 2005;14(1):26‐33. [DOI] [PMC free article] [PubMed] [Google Scholar]
Michie 2008
- Michie S. Designing and implementing behaviour change interventions to improve population health. Journal of Health Services Research & Policy 2008;13(Suppl 3):64‐9. [DOI] [PubMed] [Google Scholar]
Michie 2011
- Michie S, Stralen MM, West R. The Behaviour Change Wheel: a new method for characterising and designing behaviour change interventions. Implementation Science 2011;6:42. [DOI] [PMC free article] [PubMed] [Google Scholar]
Mostofian 2015
- Mostofian F, Ruban C, Simunovic N, Bhandari M. Changing physician behavior: what works?. American Journal of Managed Care 2015;21(1):75‐84. [PubMed] [Google Scholar]
Mundell 2013
- Mundell WC, Kennedy CS, Szostek JH, Cook DA. Simulation technology for resuscitation training: a systematic review and meta‐analysis. Resuscitation 2013;84:1174‐83. [DOI] [PubMed] [Google Scholar]
Murthy 2012
- Murthy L, Shepperd S, Clarke MJ, Garner SE, Lavis JN, Perrier L, et al. Interventions to improve the use of systematic reviews in decision‐making by health system managers, policy makers and clinicians. Cochrane Database of Systematic Reviews 2012, Issue 9. [DOI: 10.1002/14651858.CD009401.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Nieuwlaat 2014
- Nieuwlaat R, Wilczynski N, Navarro T, Hobson N, Jeffery R, Keepanasseril A, et al. Interventions for enhancing medication adherence. Cochrane Database of Systematic Reviews 2014, Issue 11. [DOI: 10.1002/14651858.CD000011.pub4] [DOI] [PMC free article] [PubMed] [Google Scholar]
Oluoch 2012
- Oluoch T, Santas X, Kwaro D, Were M, Biondich P, Bailey C, Abu‐Hanna A, Keizer N. The effect of electronic medical record‐based clinical decision support on HIV care in resource‐constrained settings: a systematic review. International Journal of Medical Informatics 2012;81:e83‐e92. [DOI] [PubMed] [Google Scholar]
Oxman 2009a
- Oxman AD, Lavis JN, Lewin S, Fretheim A. SUPPORT Tools for evidence‐informed health Policymaking (STP). 10. Taking equity into consideration when assessing the findings of a systematic review. Health Research Policy and Systems 2009;7(Suppl 1):S13. [DOI] [PMC free article] [PubMed] [Google Scholar]
Oxman 2009b
- Oxman AD, Fretheim A, Lavis JN, Lewin S. SUPPORT Tools for evidence‐informed health Policymaking (STP). 12. Finding and using research evidence about resource use and costs. Health Research Policy and Systems 2009;7(Suppl 1):S15. [DOI] [PMC free article] [PubMed] [Google Scholar]
Pantoja 2014b
- Pantoja T, Green ME, Grimshaw J, Denig P, Durieux P, Gill P, et al. Manual paper reminders: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2014, Issue 9. [DOI: 10.1002/14651858.CD001174.pub3] [DOI] [Google Scholar]
Petry 2012
- Petry NM, Rash CJ, Byrne S, Ashraf S, White WB. Financial reinforcers for improving medication adherence: findings from a meta‐analysis. American Journal of Medicine 2012;125(9):888‐96. [DOI] [PMC free article] [PubMed] [Google Scholar]
Rada 2013
- Rada G, Pérez D, Capurro D. Epistemonikos: a free, relational, collaborative, multilingual database of health evidence. Studies in Health Technology and Informatics 2013;192:486‐90. [PubMed] [Google Scholar]
Rolfe 2014
- Rolfe A, Cash‐Gibson L, Car J, Sheikh A, McKinstry B. Interventions for improving patients' trust in doctors and groups of doctors. Cochrane Database of Systematic Reviews 2014, Issue 3. [DOI: 10.1002/14651858.CD004134.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]
Rosen 2012
- Rosen MA, Hunt EA, Pronovost PJ, Federowicz MA, Weaver SJ. In situ simulation in continuing education for the health care professions: a systematic review. Journal of Continuing Education of the Health Professions 2012;32(4):243‐54. [DOI] [PubMed] [Google Scholar]
Rosenbaum 2011
- Rosenbaum SE, Glenton C, Wiysonge CS, Abalos E, Mignini L, Young T, et al. Evidence summaries tailored to health policy‐makers in low‐ and middle‐income countries. Bulletin of the World Health Organization 2011;89:54‐61. [DOI] [PMC free article] [PubMed] [Google Scholar]
Rowe 2015
- Rowe AK, SY Rowe, Peters KA, Holloway JC, Ross‐Degnan D. The health care provider performance review: a systematic review of the effectiveness of strategies to improve health care provider performance in low‐ and middle‐income countries. (presentation of preliminary results, as supplied prior to 10 July 2017). Provided by Dr. Samantha Rowe, US Centers for Disease Control and Prevention, Atlanta, GA.
Ryman 2008
- Ryman TK, Dietz V, Cairns KL. Too little but not too late: results of a literature review to improve routine immunization programs in developing countries. BMC Health Services Research 2008;8:134. [DOI] [PMC free article] [PubMed] [Google Scholar]
Schünemann 2011a
- Schünemann HJ, Oxman AD, Higgins JPT, Vist GE, Glasziou P, Guyatt GH. Chapter 11: Presenting results and 'Summary of findings' tables. In: Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from www.cochrane‐handbook.org.
Schünemann 2011b
- Schünemann HJ, Oxman AD, Vist GE, Higgins JPT, Deeks JJ, Glasziou P, et al. Chapter 12: Interpreting results and drawing conclusions. In: Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from www.cochrane‐handbook.org.
Scott 2012
- Scott SD, Albretch L, O'Leary K, Ball GD, Hartling L, Hofmeyer A, et al. Systematic review of knowledge translation strategies in the allied health professions. Implementation Science 2012;7:70. [DOI] [PMC free article] [PubMed] [Google Scholar]
Squires 2014
- Squires JE, Sullivan K, Eccles MP, Worswick J, Grimshaw JM. Are multifaceted interventions more effective than single‐component interventions in changing health‐care professionals’ behaviours? An overview of systematic reviews. Implementation Science 2014;9:152. [DOI] [PMC free article] [PubMed] [Google Scholar]
Steinman 2006
- Steinman MA, Ranji SR, Shojania KG, Gonzales R. Improving antibiotic selection: a systematic review and quantitative analysis of quality improvement strategies. Medical Care 2006;44(7):617‐28. [DOI] [PubMed] [Google Scholar]
Tannenbaum 2013
- Tannenbaum SI, Cerasoli CP. Do team and individual debriefs enhance performance? A meta‐analysis. Human Factors 2013;55(1):231‐45. [DOI] [PubMed] [Google Scholar]
Wensing 2005
- Wensing M, Bosch M, Foy R, Weijden T, Eccles M, Grol R. Factors in theories on behaviour change to guide implementation and quality improvement in health care. Centre for Quality of Care Research (WOK) 2005.
Wensing 2006
- Wensing M, Wollersheim H, Grol R. Organizational interventions to implement improvements inpatient care: a structured review of reviews. Implementation Science 2006;1:2. [DOI] [PMC free article] [PubMed] [Google Scholar]
Wensing 2011
- Wensing M, Oxman AD, Baker R, Godycki‐Cwirko M, Flottorp S, Szecsenyi J, et al. Tailored implementation for chronic diseases (TICD): a project protocol. Implementation Science 2011;6:103. [DOI] [PMC free article] [PubMed] [Google Scholar]
Wiysonge 2014
- Wiysonge CS, Herrera CA, Ciapponi A, Lewin S, Garcia Marti S, Opiyo N, et al. Financial arrangements for health systems in low‐income countries: an overview of systematic reviews. Cochrane Database of Systematic Reviews 2014, Issue 4. [DOI: 10.1002/14651858.CD011084] [DOI] [PMC free article] [PubMed] [Google Scholar]
Wong 2013
- Wong CA, Cummings GG, Ducharme L. The relationship between nursing leadership and patient outcomes: a systematic review. Journal of Nursing Management 2013;21(5):709‐24. [DOI] [PubMed] [Google Scholar]
World Bank 2016
- The World Bank Group. Data. Countries and Economies, 2016. Available fromdata.worldbank.org/country/ (accessed prior to 7 July 2017).
References to other published versions of this review
Pantoja 2014a
- Pantoja T, Opiyo N, Ciapponi A, Dudley L, Gagnon MP, Herrera CA, et al. Implementation strategies for health systems in low‐income countries: an overview of systematic reviews. Cochrane Database of Systematic Reviews 2014, Issue 5. [DOI: 10.1002/14651858.CD011086] [DOI] [PMC free article] [PubMed] [Google Scholar]