Abstract
Background
Governance arrangements include changes in rules or processes that determine authority and accountability for health policies, organisations, commercial products and health professionals, as well as the involvement of stakeholders in decision‐making. Changes in governance arrangements can affect health and related goals in numerous ways, generally through changes in authority, accountability, openness, participation and coherence. A broad overview of the findings of systematic reviews can help policymakers, their technical support staff and other stakeholders to identify strategies for addressing problems and improving the governance of their health systems.
Objectives
To provide an overview of the available evidence from up‐to‐date systematic reviews about the effects of governance arrangements for health systems in low‐income countries. Secondary objectives include identifying needs and priorities for future evaluations and systematic reviews on governance arrangements and informing refinements of the framework for governance arrangements outlined in the overview.
Methods
We searched Health Systems Evidence in November 2010 and PDQ Evidence up to 17 December 2016 for systematic reviews. We did not apply any date, language or publication status limitations in the searches. We included well‐conducted systematic reviews of studies that assessed the effects of governance arrangements on patient outcomes (health and health behaviours), the quality or utilisation of healthcare services, resource use (health expenditures, healthcare provider costs, out‐of‐pocket payments, cost‐effectiveness), healthcare provider outcomes (such as sick leave), or social outcomes (such as poverty, employment) and that were published after April 2005. We excluded reviews with limitations that were important enough to compromise the reliability of the findings of the review. Two overview authors independently screened reviews, extracted data and assessed the certainty of evidence using GRADE. We prepared SUPPORT Summaries for eligible reviews, including key messages, 'Summary of findings' tables (using GRADE to assess the certainty of the evidence) and assessments of the relevance of findings to low‐income countries.
Main results
We identified 7272 systematic reviews and included 21 of them in this overview (19 primary reviews and 2 supplementary reviews). We focus here on the results of the 19 primary reviews, one of which had important methodological limitations. The other 18 were reliable (with only minor limitations).
We grouped the governance arrangements addressed in the reviews into five categories: authority and accountability for health policies (three reviews); authority and accountability for organisations (two reviews); authority and accountability for commercial products (three reviews); authority and accountability for health professionals (seven reviews); and stakeholder involvement (four reviews).
Overall, we found desirable effects for the following interventions on at least one outcome, with moderate‐ or high‐certainty evidence and no moderate‐ or high‐certainty evidence of undesirable effects.
Decision‐making about what is covered by health insurance
‐ Placing restrictions on the medicines reimbursed by health insurance systems probably decreases the use of and spending on these medicines (moderate‐certainty evidence).
Stakeholder participation in policy and organisational decisions
‐ Participatory learning and action groups for women probably improve newborn survival (moderate‐certainty evidence). ‐ Consumer involvement in preparing patient information probably improves the quality of the information and patient knowledge (moderate‐certainty evidence).
Disclosing performance information to patients and the public
‐ Disclosing performance data on hospital quality to the public probably encourages hospitals to implement quality improvement activities (moderate‐certainty evidence). ‐ Disclosing performance data on individual healthcare providers to the public probably leads people to select providers that have better quality ratings (moderate‐certainty evidence).
Authors' conclusions
Investigators have evaluated a wide range of governance arrangements that are relevant for low‐income countries using sound systematic review methods. These strategies have been targeted at different levels in health systems, and studies have assessed a range of outcomes. Moderate‐certainty evidence shows desirable effects (with no undesirable effects) for some interventions. However, there are important gaps in the availability of systematic reviews and primary studies for the all of the main categories of governance arrangements.
Plain language summary
Effects of governance arrangements for health systems in low‐income countries
What is the aim of this overview?
The aim of this Cochrane Overview is to provide a broad summary of what is known about the effects of different governance arrangements for health systems in low‐income countries.
This overview is based on 19 relevant systematic reviews. These systematic reviews searched for studies that evaluated different types of governance arrangements. The reviews included a total of 172 studies.
This overview is one of a series of four Cochrane Overviews that evaluate health system arrangements.
Main results
What are the effects of different ways of organising authority and accountability for health policies?
Three reviews were included and the key findings are that:
‐ collaboration between local health agencies and other local government agencies may lead to little or no difference in physical health or quality of life (low‐certainty evidence);
‐ placing restrictions on the medicines reimbursed by health insurance systems probably decreases the use of and spending on these medicines (moderate‐certainty evidence);
‐ it is uncertain if fraud prevention, detection and response interventions reduce healthcare fraud and related spending (very low‐certainty evidence).
What are the effects of different ways of organising authority and accountability for organisations?
Two reviews were included and the key findings are that:
‐ Contracting non‐state, not‐for‐profit providers to deliver health services may increase access to and use of these services, improve people's health outcomes and reduce household spending on health (low‐certainty evidence). No evidence was available on whether contracting out was more effective than using these funds in the state sector.
What are the effects of different ways of organising authority and accountability for commercial products such as medicines and technologies?
Three reviews were included and the key findings are that:
‐ systems in which the World Health Organization (WHO) certifies medicine manufacturers (prequalification) and medicines registration (in which medicine regulatory authorities assess medicine manufacturers to ensure they meet international standards) may decrease the proportion of medicines that are substandard or counterfeit (low‐certainty evidence);
‐ establishing a maximum reimbursement for pharmacies dispensing similar medicines covered by insurance may increase the use of generic medicines and may reduce the use of brand‐name medicines (low‐certainty evidence), but it is uncertain whether this approach affects the overall amount spent on medicines (very low‐certainty evidence);
‐ direct‐to‐consumer advertising increases people's requests for medicines and the numbers of prescriptions given (high‐certainty evidence).
What are the effects of different ways of organising authority and accountability for healthcare providers?
Seven reviews were included and the key findings are that:
‐ training programmes for district health system managers may increase their knowledge of planning processes and their monitoring and evaluation skills (low‐certainty evidence);
‐ reducing immigration restrictions in high‐income countries probably increases the migration of nurses from low‐ and middle‐income to these countries (moderate‐certainty evidence);
‐ it is uncertain whether inspection by an external body of healthcare organisation adherence to quality standards improves adherence, quality of care or health‐acquired infection rates in hospitals (very low‐certainty evidence).
What are the effects of different ways of organising stakeholder involvement in governing health services?
Four reviews were included and the key findings are that:
‐ participatory learning and action groups for women probably improve newborn survival (moderate‐certainty evidence) and may improve maternal survival (low‐certainty evidence);
‐ disclosing performance data on health insurance scheme quality to the public may lead people to select health plans that have better quality ratings or to avoid those with worse ratings and may lead to slight improvements in clinical outcomes for health insurance schemes (low‐certainty evidence);
‐ disclosing performance data on hospital quality to the public may lead to little or no difference in people's selection of hospitals (low‐certainty evidence), probably encourages hospitals to implement quality improvement activities (moderate‐certainty evidence) and may lead to slight improvements in hospital clinical outcomes (low‐certainty evidence);
‐ disclosing performance on individual healthcare providers to the public probably leads people to select providers that have better quality ratings (moderate‐certainty evidence).
No studies evaluated the effects of stakeholder participation in policy and organisational decisions.
How up‐to‐date is this overview?
The overview authors searched for systematic reviews that had been published up to 17 December 2016.
Background
This is one of four overviews of systematic reviews of strategies for improving health systems in low‐income countries (Ciapponi 2014; Pantoja 2014; Wiysonge 2014). The aim is to provide broad overviews of the evidence about the effects of delivery, financial and governance arrangements, and implementation strategies. This overview addresses governance arrangements.
We summarise the scope of each of the four overviews below.
Delivery arrangements include changes in who receives care and when, who provides care, the working conditions of those who provide care, coordination of care amongst different providers, where care is provided, the use of information and communication technology to deliver care, and quality and safety systems (Ciapponi 2014).
Financial arrangements include changes in how funds are collected, insurance schemes, how services are purchased, and the use of targeted financial incentives or disincentives (Wiysonge 2014).
Governance arrangements include changes in rules or processes that determine authority and accountability for health policies, organisations, commercial products and health professionals, and the involvement of stakeholders in decision‐making.
Implementation strategies include interventions designed to bring about changes in healthcare organisations, the behaviour of healthcare professionals or the use of health services by healthcare recipients (Pantoja 2014).
The term 'governance' has been defined in several ways, as illustrated in Table 1. Although these definitions overlap, they may create confusion. We have defined governance here as rules or processes that affect the way in which powers are exercised, particularly with regard to authority, accountability, openness, participation, and coherence. Governance includes processes and institutions through which individuals and groups "articulate their interests, mediate their differences and exercise their legal rights and obligations" (Siddiqi 2009). Our focus accordingly is on the effects of governance arrangements to achieve health and related goals, such as efficiency, equity, human rights, responsiveness and fairness (Murray 2000). Attributes such as accountability, openness and participation can also be goals in and of themselves. For example, the World Health Organization (WHO)'s Declaration of Alma‐Ata states that "The people have a right and duty to participate individually and collectively in the planning and implementation of their health care" (WHO 1978). Governance arrangements can potentially affect patient outcomes (health and health behaviours), the quality or utilisation of healthcare services, resource use, healthcare provider outcomes (such as sick leave) and social outcomes (such as poverty or employment) (EPOC 2017). Impacts on these outcomes can be intended and desirable, or unintended and undesirable. In addition, the effects of delivery arrangements on these outcomes can either reduce or increase inequities. Health systems in low‐income countries differ from those in high‐income countries in terms of the availability of resources and access to services. Thus, some problems in high‐income countries are not relevant to low‐income countries, such as governance arrangements that rely on expensive technologies that are not available in low‐income countries. Similarly, some problems in low‐income countries are not relevant to high‐income countries, such as policies that regulate emigration of health workers. Our focus in this overview is specifically on governance arrangements in low‐income countries, by which we mean countries that the World Bank classifies as low‐ or lower‐middle‐income (World Bank Group 2016). Because upper‐middle‐income countries often have a mixture of health systems with problems similar to both those in low‐income countries and high‐income countries, our focus is relevant to middle‐income countries but excludes consideration of conditions that are not relevant in low‐income countries and are relevant in middle‐income countries.
1. Definitions of governance and of stewardship.
| Governance: definitions |
|
| Stewardhip: definitions and features distinguishing it from governance |
Stewardship is similar to the concept of public governance but, as envisaged by the WHO, is more specifically focused on the state's role in taking responsibility for the health and well‐being of the population, and guiding the health system as a whole (Travis 2003). Stewardship has been described as one of the four basic functions of health system organisations (Murray 2000). The other three functions in this model are financing, provision, and resource generation. Definitions of stewardship include the following.
|
Description of the interventions
It is possible to categorise alternative governance arrangements in a number of ways. For example, Health Systems Evidence (Lavis 2015) uses the following categories: policy authority, organisational authority, commercial authority, professional authority, and consumer and stakeholder involvement. Frenk 2013 and Murray 2000, as noted in Table 1, have described six sub‐functions of stewardship (a particular type of governance): overall system design, performance assessment, priority setting, intersectoral advocacy, regulation and consumer protection. Furthermore, WHO has identified three basic tasks of stewardship (WHO 2000): formulating health policy (defining the vision and direction), exerting influence (approaches to regulation), and collecting and using intelligence. The types of interventions that we include in this overview are listed in Table 2 using a structure derived from the taxonomy developed by Lavis 2015. We used this framework as our starting point because it is not limited to stewardship, and it is comprehensive and detailed. We adapted the framework in order to clarify the classification of interventions where this was ambiguous.
2. Types of governance arrangements.
| Governance arrangement | Definition |
| Authority and accountability for health policies | |
| Interagency collaboration | Collaboration and partnerships for health and social development between the health sector and other different sectors |
| Centralisation and decentralisation | Policies to regulate the degree of which managerial responsibilities are transferred to regional or local authorities in contrast to having them at the central level |
| District management | Policies that regulate the management of district health systems |
| Decision‐making about what or who is covered by health insurance | Processes for deciding what is reimbursed and who is covered by health insurance |
| Policies to reduce corruption | Policies for reducing corruption in the health sector |
| Policies to manage absenteeism | Regulations for managing absenteeism of health professionals |
| Requirements for monitoring or evaluation | Policies that regulate programme monitoring and evaluation |
| Authority and accountability for organisations | |
| Ownership | Policies that regulate who can own health service organisations |
| Stewardship of private health services | Policies that regulate health services provided by the private sector |
| Insurance | Policies that regulate the provision of insurance (e.g. who can provide insurance, mandatory open enrolment, coverage of essential drugs) |
| Accreditation | Processes for accrediting healthcare providers |
| Multi‐institutional arrangements | Policies for how multiple organisations work together |
| Authority and accountability for commercial products | |
| Registration | Procedures for registering or licensing commercial products (e.g. drugs) |
| Patents and profits | Policies that regulate patents and profits |
| Pricing and purchasing policies | Policies that determine the price that is paid or how commercial products are purchased |
| Marketing regulations | Policies that regulate marketing of commercial products |
| Sales and dispensing | Policies that regulate the sale and dispensing of drugs or other healthcare products |
| Liability for commercial products | Policies that regulate liability for commercial products |
| Authority and accountability for health professionals | |
| Training and licensing | Policies that regulate training and licensure requirements for health professionals |
| Scope of practice | Policies that regulate what health professionals can do |
| Recruitment and retention strategies | Policies that regulate where health professionals work (e.g. restrictions on where they can work or requirements to work in rural areas) |
| Emigration and immigration policies | Policies that regulate emigration and immigration of health professionals |
| Dual practice | Policies that regulate dual practice, in which health workers hold two or more jobs, for example in both the public or private sectors |
| Quality of practice | Policies or systems for assuring quality of care |
| Professional competence | Policies or procedures for assuring professional competence |
| Policies to manage absenteeism | Policies for managing absenteeism of health professionals |
| Professional liability | Policies that regulate liability for health professionals |
| Stakeholder involvement | |
| Stakeholder participation in policy and organisational decisions | Policies and procedures for involving stakeholders in decision‐making |
| Community mobilisation | Processes that enable people to organise themselves |
| Community monitoring | Monitoring of health services by individuals or community organisations |
| Patient information | Policies that regulate what information is provided to patients |
| Patients' rights | Policies that regulate patients' rights, including access to care and information |
How the intervention might work
Changes in governance arrangements can affect health and related goals in multiple ways. Generally, this is likely to occur through changes in authority, accountability, openness, participation, and coherence (promotion of mutually reinforcing policy actions). Table 3 presents examples of how changes in different types of governance arrangements might lead to better healthcare outcomes.
3. Examples of how changes in governance arrangements might work.
| Governance arrangement | Definition |
| Authority and accountability for health policies | |
| Interagency collaboration | Policies to facilitate interagency collaboration, for instance, between local government and local health authorities in order to address social determinants of health, can contribute to improve health of the population. |
| Decentralisation and centralisation | Shifting authority closer to those who are affected might improve accountability, openness and participation, which might in turn lead to more appropriate priorities, more efficiency and less corruption, and in turn better health outcomes. |
| District management | Regulations that lead to improvements in the management of district health systems can improve access to and the quality of care, and in turn better health outcomes. |
| Decision‐making about what or who is covered by health insurance | Changes in processes used to decide what is reimbursed or who is covered by health insurance might improve access to cost‐effective interventions, and in turn lead to better health outcomes. |
| Policies to reduce corruption | Regulations that reduce corruption can increase the availability of resources for care, and in turn improve health outcomes. |
| Requirements for monitoring or evaluation | Policies that improve decisions about when and how healthcare programmes are monitored or evaluated can lead to better‐informed decisions, and in turn better health outcomes. |
| Authority and accountability for organisations | |
| Ownership | For‐profit health services might limit access for people who cannot afford to pay or divert funds from care to profits and taxes, which might result in poorer quality care and worse health outcomes. |
| Stewardship of private health services | Regulations that increase the accountability of the private sector might improve the quality of care, and in turn lead to better health outcomes. |
| Insurance | Changes in regulations that determine who can provide insurance, who receives it, who pays for it, and who makes decisions about reimbursement might affect coverage and access to care, and in turn health outcomes. |
| Accreditation | Changes in provider accreditation might improve the quality of care, and in turn health outcomes. |
| Multi‐institutional arrangements | Changes in how donors and governments work together might result in more effective and efficient use of resources, and in turn lead to better health outcomes. |
| Authority and accountability for commercial products | |
| Registration | Changes in how drugs or other health technologies are licensed might improve safety, and in turn health outcomes, |
| Patents and profits | Changes in patent regulations might affect the development and availability of drugs or other health technologies, and in turn health outcomes. |
| Pricing and purchasing policies | Regulations that reduce the price that is paid or how drugs or services are purchased might improve access to care, and in turn health outcomes. |
| Marketing regulations | Regulations that limit inappropriate marketing of drugs, other technologies or services might reduce inappropriate use and increase the availability of resources for cost‐effective care, and in turn improve health outcomes. |
| Sales and dispensing | Changes in who can sell drugs or other healthcare products might improve access or improve quality, and in turn health outcomes. |
| Liability for commercial products | Changes in liability for drugs, other technologies or services might improve safety, and in turn health outcomes. |
| Authority and accountability for health professionals | |
| Training and licensing | Regulations that improve training or licensure of health professionals might improve the safety and quality of care, and in turn health outcomes. |
| Scope of practice | Regulations that determine what health professionals can do might improve access to care or safety, and in turn health outcomes. |
| Recruitment and retention strategies | Regulations that determine where health professionals can work might improve access to care, and in turn health outcomes. |
| Emigration and immigration policies | Regulations that determine emigration or immigration of health professionals might improve access to care, and in turn health outcomes. |
| Dual practice | Regulations that affect the extent of dual practice might improve access to care, and in turn health outcomes. |
| Quality of practice | Policies or systems for assuring quality of care might improve the quality of care, and in turn health outcomes. |
| Professional competence | Policies or procedures for assuring professional competence might improve the safety and quality of care, and in turn health outcomes. |
| Policies to manage absenteeism | Regulations that reduce absenteeism can improve access to care, and in turn health outcomes. |
| Professional liability | Changes in liability for health professionals might improve safety or remove impediments to evidence‐based care, and in turn improve health outcomes. |
| Stakeholder involvement | |
| Stakeholder participation in policy and organisational decisions | Involving stakeholders in decision‐making might improve the overall decision‐making about how to use resources and organise care, and in turn lead to better health outcomes. |
| Community mobilisation | Processes that enable people to organise themselves might raise awareness, change behaviours and lead to improvements in access and utilisation of health services, and in turn improve health outcomes. |
| Community monitoring | Monitoring of health services by individuals or community organisations might help to ensure quality, improve access to care, and reduce corruption, and in turn improve health outcomes. |
| Patient information | Regulations that improve the extent to which patients are well‐informed might lead to better informed decisions, and in turn improve health outcomes. |
| Patients' rights | Policies that regulate patients' rights, such as access to care and information, might improve access and utilisation of health services and improve the quality of health services, and in turn improve health outcomes. |
Why it is important to do this overview
Our objective is to provide a broad overview of current evidence from systematic reviews evaluating the effects of alternative governance arrangements for health systems in low‐income countries. We recognise that there is a paucity of research that has evaluated the effects of governance arrangements (Bennington 2010; Frenk 2013). Nonetheless, a broad overview of the findings of systematic reviews can help policymakers, their technical support staff and other stakeholders to identify strategies for addressing problems with the governance of their health systems. It can also help to identify needs and priorities for evaluations of governance arrangements, as well as priorities for systematic reviews of the effects of governance arrangements. The overview also helps to refine the framework outlined in Table 2 for considering alternative health system arrangements for allocating authority and ensuring accountability, openness, participation and coherence.
Our focus is specifically on low‐income countries in this overview because there are structural differences in health systems and country contexts compared to middle‐ and high‐income countries. These differences make it difficult to select, analyse and summarise the evidence for low‐, middle‐ and high‐income countries in a single overview. By focusing on low‐income countries, we were able to exclude reviews that are not relevant to those countries and to consistently address the relevance of the evidence in included reviews for those countries. This makes the overview more helpful for people making decisions about governance arrangements in low‐income countries.
Changes in health systems are complex. They may be difficult to evaluate, the applicability of the findings of evaluations from one setting to another may be uncertain, and synthesising the findings of evaluations may be difficult. However, the alternative to well‐designed evaluations is poorly designed evaluations; the alternative to systematic reviews is non‐systematic reviews; and the alternative to using the findings of systematic reviews to inform decisions is making decisions without the support of this rigorous evidence. Policymakers still need other types of information, including context specific information and judgments (e.g. judgments about the applicability of the findings of systematic reviews in a specific context) when making decisions about governance arrangements.
This overview can help people making decisions about governance arrangements by summarising the findings of available systematic reviews, including estimates of the effects of changes in governance arrangements and the certainty of those estimates, by identifying important uncertainties identified by those systematic reviews and by identifying where new or updated systematic reviews are needed. The overview can also help to inform judgments about the relevance of the available evidence in a specific context (Rosenbaum 2011).
Objectives
To provide an overview of the available evidence from up‐to‐date systematic reviews about the effects of governance arrangements for health systems in low‐income countries. Secondary objectives include identifying needs and priorities for future evaluations and systematic reviews on governance arrangements and informing refinements of the framework for governance arrangements outlined in the overview (Table 2).
Methods
We used the methods described below in all four overviews of health system arrangements and implementation strategies in low‐income countries (Ciapponi 2014; Pantoja 2014; Wiysonge 2014).
Criteria for considering reviews for inclusion
We included systematic reviews that:
assessed the effects of governance arrangements (as defined in the Background);
had a Methods section with explicit selection criteria;
reported at least one of the following types of outcomes: patient outcomes (health and health behaviors), the quality or utilisation of healthcare services, resource use (health expenditures, healthcare provider costs, out‐of‐pocket payments, cost‐effectiveness), healthcare provider outcomes (such as sick leave, burnout), or social outcomes (such as poverty, employment);
were relevant to low‐income countries as classified by the World Bank (World Bank Group 2016);
were published after April 2005.
Judgments about relevance to low‐income countries are sometimes difficult to make, and we are aware that evidence from high‐income countries is not directly generalisable to low‐income countries. We based our judgments on an assessment of the likelihood that the governance arrangements considered in a review address a problem that is important in low‐income countries, would be feasible, and would be of interest to decision‐makers in low‐income countries, regardless of where the included studies took place. So, for example, we excluded arrangements that require technology that is not widely available in low‐income countries. At least two of the overview authors made judgments about the relevance to low‐income countries and discussed with the other authors whenever there was uncertainty. Reviews that only included studies from a single high‐income country were not eligible due to concerns about the wider applicability of the findings of such reviews. However, we did consider reviews that only included studies from high‐income countries if the interventions were relevant for low‐income countries.
We excluded reviews published before April 2005 as these were highly unlikely to be up‐to‐date. We also excluded reviews that had methodological limitations that were important enough to compromise the reliability of the review findings (Appendix 1).
Search methods for identification of reviews
We searched Health Systems Evidence in November 2010 using the following filters.
Health system topics = governance arrangements.
Type of synthesis = systematic review or Cochrane Review.
Type of question = effectiveness.
Publication date range = 2000 to 2010.
We conducted subsequent searches using PDQ ('pretty darn quick')‐Evidence, which was launched in 2012. We searched PDQ up to 17 December 2016, using the filter 'Systematic Reviews' with no other restrictions. We updated that search, excluding records that were entered into PDQ‐Evidence prior to the date of the last previous search.
PDQ‐Evidence is a database of evidence for decisions about health systems, which is derived from the Epistemonikos database of systematic reviews (Rada 2013). It includes systematic reviews, overviews of reviews (including evidence‐based policy briefs) and studies included in systematic reviews. Epistemonikos and PDQ‐Evidence incorporate searches from the following databases with no language or publication status restrictions.
Cochrane Database of Systematic Reviews (CDSR).
PubMed.
Embase.
Database of Abstracts of Reviews of Effectiveness (DARE).
Health Technology Assessment Database.
CINAHL.
LILACS.
PsycINFO.
Evidence for Policy and Practice Information and Co‐ordinating Centre (EPPI‐Centre) Evidence Library.
3ie Systematic Reviews and Policy Briefs.
World Health Organization (WHO) Database.
Campbell Library.
Supporting the Use of Research Evidence (SURE) Guides for Preparing and Using Evidence‐Based Policy Briefs.
European Observatory on Health Systems and Policies.
UK Department for International Development (DFID).
National Institute for Health and Care Excellence (NICE) public health guidelines and systematic reviews.
Guide to Community Preventive Services.
Canadian Agency for Drugs and Technologies in Health (CADTH) Rx for Change.
McMaster Plus KT+.
McMaster Health Forum Evidence Briefs.
Appendix 2 presents the detailed search strategies for PubMed, LILACS, Embase, CINAHL and PsycINFO. We screened all records in the other databases. PDQ staff and volunteers update these searches weekly for Pubmed and monthly for the other databases, screening records continually, and adding new reviews to the database daily.
In addition, we screened all of the Cochrane Effective Practice and Organisation of Care (EPOC) Group reviews in Archie (i.e. Cochrane's central server for managing documents) and the reference lists of relevant policy briefs and overviews of reviews.
Data collection and analysis
Selection of reviews
Two of the overview authors (CH and SL) independently screened the titles and abstracts found in PDQ‐Evidence to identify reviews that appeared to meet the inclusion criteria. Two other authors (AO and SL) screened all of the titles and abstracts that we could not confidently include or exclude after the first screening to identify any additional eligible reviews. One of the overview authors screened the reference lists (CH).
One of the overview authors applied the selection criteria to the full text of potentially eligible reviews and assessed the reliability of reviews that met all of the other selection criteria (CH) (Appendix 1). Two other authors (AO or SL) independently checked these judgments.
Data extraction and management
We summarised each included review using the approach developed by the SUPPORT collaboration (Rosenbaum 2011). We used standardised data extraction forms to extract data on the background of the review: interventions, participants, settings and outcomes; key findings; and considerations of applicability, equity, economic considerations, and monitoring and evaluation. We assessed the certainty of the evidence for the main comparisons using the GRADE approach (Guyatt 2008; Schünemann 2011a; Schünemann 2011b; EPOC 2016).
Each completed SUPPORT Summary underwent peer‐review and was published on the SUPPORT Summaries website, where we provide details about how we prepared the summaries and how we assessed the applicability of the findings, impacts on equity, economic considerations, and the need for monitoring and evaluation. We describe the rationale for the criteria that we used for these assessments in the SUPPORT Tools for evidence‐informed health policymaking (Fretheim 2009; Lavis 2009; Oxman 2009a; Oxman 2009b). As noted there, "a local applicability assessment must be done by individuals with a very good understanding of on‐the‐ground realities and constraints, health system arrangements, and the baseline conditions in the specific setting" (Lavis 2009). In this overview we have made broad assessments of the applicability of findings from studies in high‐income countries to low‐income countries using the criteria described in the SUPPORT summaries database, with input from people with relevant experience and expertise in low‐income countries.
Assessment of methodological quality of included reviews
We assessed the reliability of systematic reviews that met our inclusion criteria using criteria developed by the SUPPORT and SURE collaborations (Appendix 2; SUPPORT 2009, SURE 2011). Based on these criteria, we categorised each review as having:
only minor limitations;
limitations that are important enough that it would be worthwhile to search for another systematic review and to interpret the results of this review cautiously, if no better review is available;
limitations that are important enough to compromise the reliability of the review and prompt its exclusion from the overview.
Data synthesis
We describe the methods used to prepare a SUPPORT Summary of each review in detail on the SUPPORT Summaries website. Briefly, for each included systematic review, we prepared a table summarising what the review authors searched for and what they found (Appendix 3), we prepared 'Summary of findings' tables for each main comparison, and we assessed the relevance of the findings for low‐income countries. The SUPPORT Summaries include key messages, important background information, a summary of the findings of the review and structured assessments of the relevance of the review for low‐income countries. We subjected the SUPPORT Summaries to review by the lead author of each review, at least one content area expert, people with practical experience in low‐income settings, and a Cochrane EPOC Group editor (AO or SL). The authors of the SUPPORT Summaries responded to each comment and made appropriate revisions, and the summaries underwent copy‐editing. The editor determined whether the comments had been adequately addressed and whether the summary was ready for publication on the SUPPORT Summary website.
We organised the review by modifying the taxonomy for health systems arrangements used by Health Systems Evidence (Lavis 2015), adjusting this framework iteratively to ensure that we appropriately categorised all of the included reviews and that we included and logically organised all relevant health system governance arrangements. We prepared a table listing the included reviews as well as the types of governance arrangements for which we were not able to identify a reliable, up‐to‐date review (Table 4). We also prepared a table of excluded reviews (Table 5), describing reviews that addressed a question for which another (more up‐to‐date or reliable) review was available, reviews that were published before April 2005 (for which a SUPPORT Summary was available), reviews with results that we did not consider transferable to low‐income countries, and reviews with limitations that were important enough to compromise the reliability of the review findings.
4. Included reviews.
| Governance arrangement | Included reviews |
| Authority and accountability for health policies | |
| Interagency collaboration | Collaboration between local health and local government agencies for health improvement (Hayes 2012) |
| Decentralisation and centralisation | No eligible systematic review found |
| District management | No eligible systematic review found |
| Decision‐making about what or who is covered by health insurance | |
| Policies that regulate what drugs are reimbursed | No eligible systematic review found |
| Policies that regulate what services are reimbursed | No eligible systematic review found |
| Restrictions on drug reimbursement | Pharmaceutical policies: effects of restrictions on reimbursement (Green 2010) |
| Restrictions on reimbursement for health insurance | No eligible systematic review found |
| Strategies for expanding health insurance coverage | No eligible systematic review found |
| Policies to reduce corruption | No evidence of the effect of the interventions to combat health care fraud and abuse: a systematic review of literature (Rashidian 2012) |
| Policies to manage absenteeism | No eligible systematic review found |
| Requirements for monitoring or evaluation | No eligible systematic review found |
| Authority and accountability for organisations | |
| Ownership | No eligible systematic review found |
| Stewardship of private health services | No eligible systematic review found |
| Contracting out | The impact of contracting out on health outcomes and use of health services in low and middle income countries (Lagarde 2009) |
| Accreditation | No eligible systematic review found |
| Regulation of insurance provision | |
| Provision of drug insurance | No eligible systematic review found |
| Provision of health insurance | No eligible systematic review found |
| Multi‐institutional arrangements | |
| Policies that regulate interactions between donors and governments | No eligible systematic review found |
| Social franchising | The effect of social franchising on access to and quality of health services in low‐ and middle‐income countries (Koehlmoos 2009) |
| Governance arrangements for coordinating care across multiple providers | No eligible systematic review found |
| Mergers | No eligible systematic review found |
| Authority and accountability for commercial products | |
| Registration | |
| Drugs | Interventions to combat or prevent drug counterfeiting: a systematic review (El‐Jardali 2015) |
| Health technology | No eligible systematic review found |
| Patents and profits | |
| Drugs | No eligible systematic review found |
| Health technology | No eligible systematic review found |
| Pricing and purchasing policies | |
| Drugs | Pharmaceutical policies: effects of reference pricing, other pricing, and purchasing policies (Acosta 2014) |
| Health technology and services | No eligible systematic review found |
| Marketing regulations | |
| Drugs | Benefits and harms of direct to consumer advertising: a systematic review (Gilbody 2005) |
| Health technology and services | No eligible systematic review found |
| Sales and dispensing | |
| Drugs | No eligible systematic review found |
| Health technology | No eligible systematic review found |
| Liability for commercial products | No eligible systematic review found |
| Authority and accountability for health professionals | |
| Training and licensing | |
| Pre‐licensure education | Effects of changes in the pre‐licensure education of health workers on health‐worker supply (Pariyo 2009) |
| Training district health system managers | Interventions for hiring, retaining and training district health system managers in low‐ and middle‐income countries (Rockers 2013) |
| Licensure | No eligible systematic review found |
| Specialty certification | No eligible systematic review found |
| Scope of practice | No eligible systematic review found |
| Recruitment and retention strategies | Interventions for increasing the proportion of health professionals practising in underserved communities (Grobler 2015) |
| Recruitment and retention strategies | Interventions for hiring, retaining and training district health system managers in low‐ and middle‐income countries (Rockers 2013) |
| Movement of health workers between public and private organisations | Financial interventions and movement restrictions for managing the movement of health workers between public and private organizations in low‐ and middle‐income countries (Rutebemberwa 2014) |
| Emigration and immigration policies | Interventions for controlling emigration of health professionals from low‐ and middle‐income countries (Peñaloza 2011) |
| Dual practice | Interventions to manage dual practice among health workers (Kiwanuka 2011) |
| Authority and accountability for quality of practice | |
| Authority and accountability for quality of outpatient care | External inspection versus external standards for improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes (Flodgren 2011) |
| Authority and accountability for quality assurance of hospital (inpatient) care | External inspection versus external standards for improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes (Flodgren 2011) |
| Professional competence | No eligible systematic review found |
| Professional liability | No eligible systematic review found |
| Stakeholder involvement | |
| Stakeholder participation in policy and organisational decisions | Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material (Nilsen 2010) |
| Community mobilisation | Women's groups practicing participatory learning and action to improve maternal and newborn health in low‐resource settings: a systematic review and meta‐analysis (Prost 2013) |
| What do community‐based dengue control programmes achieve? A systematic review of published evaluations (Heintze 2007) | |
| Community monitoring | No eligible systematic review found |
| Patient information | |
| Drug information | No eligible systematic review found |
| Public disclosure of performance data | Systematic review: the evidence that publishing patient care performance data improves quality of care (Fung 2008) |
| Patients’ rights | No eligible systematic review found |
5. Excluded reviews.
| Review ID | Excluded reviews | Reasons for exclusion |
| Bärnighausen 2009 | Financial incentives for return of service in underserved areas: a systematic review | Addressed by Grobler 2015 |
| Berendes 2011 | Quality of private and public ambulatory health care in low and middle income countries: systematic review of comparative studies | Addressed by upcoming Herrera 2013 |
| Boote 2002 | Consumer involvement in health research: a review and research agenda | More than 10 years out of date |
| Comondore 2009 | Quality of care in for‐profit and not‐for‐profit nursing homes: systematic review and meta‐analysis | Not transferable to low‐income countries |
| Crawford 2002 | Systematic review of involving patients in the planning and development of health care | Addressed by Nilsen 2010 |
| Devereaux 2002a | A systematic review and meta‐analysis of studies comparing mortality rates of private for‐profit and private not‐for‐profit hospitals. | More than 10 years out of date |
| Devereaux 2002b | Comparison of mortality between private for‐profit and private not‐for‐profit hemodialysis centers | More than 10 years out of date |
| Devereaux 2004 | Payments for care at private for‐profit and private not‐for‐profit hospitals: a systematic review and meta‐analysis | Not transferable to low‐income countries |
| Ekman 2004 | Community‐based health insurance in low‐income countries: a systematic review of the evidence | Addressed by Meng 2010 |
| Faber 2009 | Public reporting in health care: how do consumers use quality‐of‐care information? A systematic review | Addressed by Fung 2008 |
| Faden 2011 | Active pharmaceutical management strategies of health insurance systems to improve cost‐effective use of medicines in low‐ and middle‐income countries: a systematic review of current evidence. | Major limitations |
| Greenfield 2008 | Health sector accreditation research: a systematic review | Major limitations |
| Greenfield 2012 | The standard of healthcare accreditation standards: a review of empirical research underpinning their development and impact | Major limitations |
| Griffiths 2007 | Effectiveness of intermediate care in nursing‐led in‐patient units | Not transferable to low‐income countries |
| Henderson 2010 | Provision of a surgeon’s performance data for people considering elective surgery | Addressed by Fung 2008 |
| Jia 2014 | Strategies for expanding health insurance coverage in vulnerable populations | Scope of the Implementation overview |
| Lagarde 2006 | Evidence from systematic reviews to inform decision making regarding financing mechanisms that improve access to health services for poor people. A policy brief prepared for the International Dialogue on Evidence‐Informed Action to Achieve Health Goals in Developing Countries IDEAHealth | Addressed by Lagarde 2009 |
| Lee 2009 | Linking families and facilities for care at birth: what works to avert intrapartum‐related deaths? | Major limitations |
| Lehmann 2008 | Staffing remote rural areas in middle‐ and low‐income countries: a literature review of attraction and retention | Addressed by Grobler 2015 |
| Liu 2008 | The effectiveness of contracting‐out primary health care services in developing countries: a review of the evidence | Addressed by Lagarde 2009 |
| Loevinsohn 2004 | Contracting for the delivery of community health services: a review of global experience | Addressed by Lagarde 2009 |
| Marshall 2000 | The public release of performance data: what do we expect to gain? A review of the evidence | More than 10 years out of date |
| Meng 2010 | Expanding health insurance coverage in vulnerable groups: a systematic review of options | Addressed by Jia 2014 |
| Molyneux 2012 | Community accountability at peripheral health facilities: a review of the empirical literature and development of a conceptual framework | Major limitations |
| Montagu 2011 | Private versus public strategies for health service provision for improving health outcomes in resource‐limited settings | Major limitations |
| Morgan 2009 | Comparison of tiered formularies and reference pricing policies: a systematic review | Addressed by Acosta 2014 |
| Ossai 2012 | Rural retention of human resources for health | Addressed by Grobler 2015 |
| Patouillard 2007 | Can working with the private for‐profit sector improve utilization of quality health services by the poor? A systematic review of the literature | Major limitations |
| Patterson 2010 | Systematic review of the links between human resource management practices and performance | Major limitations |
| Peters 2004 | Strategies for engaging the private sector in sexual and reproductive health: how effective are they? | More than 10 years out of date |
| Phillips 2010 | Can clinical governance deliver quality improvement in Australian general practice and primary care? A systematic review of the evidence | Addressed by Flodgren 2011 |
| Preston 2010 | Community participation in rural primary health care: intervention or approach? | Addressed by Nilsen 2010 |
| Puig‐Junoy 2007 | Impact of pharmaceutical prior authorisation policies: a systematic review of the literature | Addressed by Green 2010 |
| Ranji 2007 | Effects of rapid response systems on clinical outcomes: systematic review and meta‐analysis | Scope of the Delivery overview |
| Schadewaldt 2011 | Nurse‐led clinics as an effective service for cardiac patients: results from a systematic review | Major limitations |
| Shah 2011 | Can interventions improve health services from informal private providers in low and middle‐income countries? A comprehensive review of the literature | Major limitations |
| Sharp 2002 | Specialty board certification and clinical outcomes: the missing link | More than 10 years out of date |
| Shen 2007 | Hospital ownership and financial performance: a quantitative research review | Not transferable to low‐income countries |
| Socha 2011 | Physician dual practice: a review of literature | Addressed by Kiwanuka 2011 |
| Steinman 2006 | Improving antibiotic selection: a systematic review and quantitative analysis of quality improvement strategies | Addressed by New Reference |
| Tait 2004 | Clinical governance in primary care: a literature review | Addressed by Phillips 2010 |
| Wafula 2012 | Examining characteristics, knowledge and regulatory practices of specialised drug shops in Sub‐Saharan Africa: a systematic review of the literature | Not a review of effects of interventions |
| Waters 2003 | Working with the private sector for child health | More than 10 years out of date |
| Willis‐Shattuck 2008 | Motivation and retention of health workers in developing countries: a systematic review | Not a review of effects of interventions |
| Wilson 2009 | A critical review of interventions to redress the inequitable distribution of healthcare professionals to rural and remote areas | Addressed by Grobler 2015 |
We described the characteristics of the included reviews in a table that included the date of the last search, any important limitations, what the review authors searched for and what they found (Appendix 3). We summarised our detailed assessments of the reliability of the included reviews in a separate table (Table 6) showing whether individual reviews met each criterion in Appendix 2.
6. Reliability of included reviews.
| Review | A. Identification, selection and critical appraisal of studiesa | B. Analysisb | C. Overallc | |||||||||||
| 1. Selection criteria | 2. Search | 3. Up‐to‐date | 4. Study selection | 5. Risk of bias | 6. Overall | 1. Study characteristics | 2. Analytic methods | 3. Heterogeneity | 4. Appropriate synthesis | 5. Exploratory factors | 6. Overall | 1. Other considerations | 2. Reliability of the review | |
| Acosta 2014 | + | + | + | + | + | + | + | + | + | + | + | + | No | + |
| El‐Jardali 2015 | + | + | + | + | + | + | + | + | + | + | + | + | No | + |
| Flodgren 2011 | + | + | + | + | + | + | + | + | NA | + | NA | + | No | + |
| Fung 2008 | + | ? | + | + | + | + | + | + | + | + | + | + | No | + |
| Gilbody 2005 | + | + | − | + | + | + | + | + | + | + | + | + | No | + |
| Green 2010 | + | + | + | + | + | + | + | + | + | + | + | + | No | + |
| Grobler 2015 | + | + | + | + | + | + | + | + | + | + | + | + | No | + |
| Hayes 2012 | + | + | + | + | + | + | + | + | + | + | + | + | No | + |
| Heintze 2007 | + | ? | + | ? | + | + | + | + | + | + | ? | + | No | + |
| Kiwanuka 2011 | + | + | + | + | + | + | NA | NA | NA | NA | NA | NA | No | + |
| Koehlmoos 2009 | + | + | + | + | + | + | NA | NA | NA | NA | NA | NA | No | + |
| Lagarde 2009 | + | + | + | + | + | + | + | + | + | + | + | + | No | + |
| Nilsen 2010 | + | + | + | + | + | + | + | + | + | + | + | + | No | + |
| Pariyo 2009 | + | + | + | + | + | + | + | + | + | + | + | + | No | + |
| Peñaloza 2011 | + | + | + | + | + | + | + | + | + | + | + | + | No | + |
| Prost 2013 | + | + | + | ? | + | + | + | + | + | + | + | + | No | |
| Rashidian 2012 | ? | − | + | ? | + | − | ? | + | ? | + | NA | + | No | − |
| Rockers 2013 | + | ? | + | + | + | + | + | + | + | + | NA | + | No | + |
| Rutebemberwa 2014 | + | ? | + | + | + | + | NA | NA | NA | NA | NA | NA | No | + |
aIdentification, selection and critical appraisal of studies ‐ details of assessment criteria
1. Selection criteria: were the criteria used for deciding which studies to include in the review reported? (+ yes; ? can't tell/partially; − no)
2. Search: was the search for evidence reasonably comprehensive? (+ yes; ? can't tell/partially; − no)
3. Up‐to‐date: is the review reasonably up‐to‐date? (+ yes; ? can't tell/partially; − no)
4. Study selection: was bias in the selection of articles avoided? (+ yes; ? can't tell/partially; − no)
5. Risk of bias: did the authors use appropriate criteria to assess the risk for bias in analysing the studies that are included? (+ yes; ? can't tell/partially; − no)
6. Overall: how would you rate the methods used to identify, include and critically appraise studies? (+ only minor limitations, − important limitations)
bAnalysis ‐ details of assessment criteria
1. Study characteristics: were the characteristics and results of the included studies reliably reported? (+ yes; ? can't tell/partially; − no, NA: not applicable; e.g. no studies or data)
2. Analytic methods: were the methods used by the review authors to analyse the findings of the included studies reported? (+ yes; ? can't tell/partially; − no, NA: not applicable; e.g. no studies or data)
3. Heterogeneity: did the review describe the extent of heterogeneity? (+ yes; ? can't tell/partially; − no, NA: not applicable; e.g. no studies or data)
4. Appropriate synthesis: were the findings of the relevant studies combined (or not combined) appropriately relative to the primary question the review addresses and the available data? (+ yes; ? can't tell/partially; − no, NA: not applicable; e.g. no studies or data)
5. Exploratory factors: did the review examine the extent to which specific factors might explain differences in the results of the included studies? (+ yes; ? can't tell/partially; − no, NA: not applicable; e.g. no studies or data)
6. Overall: how would you rate the methods used to analyse the findings relative to the primary question addressed in the review? (+ only minor limitations, − important limitations)
cOverall ‐ details of assessment criteria
1. Other considerations: are there any other aspects of the review not mentioned before which lead you to question the results?
2. Reliability of the review: based on the above assessments of the methods how would you rate the reliability of the review? (+ only minor limitations, − important limitations)
We based our structured synthesis of the findings of our overview on two tables (Table 7; Table 8). We summarised the main findings of each review in a table that included the key messages from each SUPPORT Summary (Table 7). In a second table (Table 8), we reported the direction of the results and the certainty of the evidence for each of the following type of outcomes: health and other patient outcomes; access, coverage or utilisation; quality of care; resource use; social outcomes; impacts on equity; healthcare provider outcomes; adverse effects (not captured by undesirable effects on any of the preceding types of outcomes); and any other important outcomes (that did not fit into any of the preceding types of outcomes) (EPOC 2016). We categorised the direction of results as: a desirable effect, little or no effect, an uncertain effect (very low‐certainty evidence), no included studies, an undesirable effect, not reported (i.e. not specified as a type of outcome that was considered by the review authors), or not relevant (i.e. no plausible mechanism by which the type of health system arrangement could affect the type of outcomes).
7. Key messages of included reviews.
| Governance arrangement | Key messages |
| Authority and accountability for health policies | |
|
Interagency collaboration Hayes 2012 |
➡ Local interagency collaborative interventions may lead to little or no difference in physical health and quality of life compared with standard care. ➡ It is uncertain whether local interagency collaborative interventions decrease mortality or mental health symptoms. ➡ This review did not include any evidence from low‐income countries. |
|
Decision‐making about what or who is covered by health insurance ‐ Restrictions on drug reimbursement Green 2010 |
➡ Restrictions on reimbursement in health insurance systems with substantial coverage for medicines probably decreases targeted drug use and expenditures on targeted drugs or drug classes. ➡ The effects of restrictions on reimbursement vary by drug and drug class, and by how the restrictions are implemented and enforced. ➡ The impacts of restrictions on health outcomes and health service utilisation are uncertain. ➡ All the studies were done in high‐income countries and participants were mainly senior citizens or low‐income adult populations whose medications were being paid for in whole or part through publicly funded drug benefit plans. ➡ There are no studies on the effect of reimbursement restrictions on equity. |
|
Policies to reduce corruption Rashidian 2012 |
➡ It is uncertain if prevention, detection or response interventions reduce healthcare fraud and abuse and related expenditures. ➡ None of the included studies took place in a low‐income country. |
| Authority and accountability for organisations | |
|
Contracting out Lagarde 2009 |
➡ Contracting out services to non‐state not‐for‐profit providers may increase access to and utilisation of health services. ➡ Patient outcomes may be improved and household health expenditures reduced by contracting out. ➡ None of the included studies presented evidence on whether contracting out was more effective than making a similar investment in the public sector. We are therefore uncertain of the effects of investing in contracting out compared to an equivalent investment in public sector health services. |
|
Multi‐institutional arrangements ‐ Social franchising Koehlmoos 2009 |
➡ We found no evidence regarding the effects of social franchising on access to or the quality of health services in low‐ and middle‐income countries. We are therefore uncertain of the effects of social franchising. ➡ There is a need for well‐designed experimental studies that are informed by the theoretical and empirical literature. |
| Authority and accountability for commercial products | |
|
Registration ‐ Drugs El‐Jardali 2015 |
➡ Certain regulatory measures, specifically drug registration, may decrease the prevalence of counterfeit and substandard drugs. It is uncertain whether licensing of drug outlets reduces the prevalence of counterfeit drugs or the failure rates of drugs undergoing quality testing. ➡ WHO prequalification of drugs may lead to a reduction in the failure rates of drugs undergoing quality testing. ➡ Multifaceted interventions (including a mix of regulations, training of inspectors, public‐private collaborations and legal actions against counterfeiters) may be effective in decreasing the prevalence of counterfeit and substandard drugs. ➡ All studies identified took place in low‐ and middle‐income countries. ➡ The transferability of the findings may be influenced by a country's existing pharmaceutical supply chain and infrastructure, the availability of routine data on drug quality, qualified and skilled personnel, and financial resources. |
|
Pricing and purchasing policies ‐ Drugs Acosta 2014 |
➡ Reference pricing may reduce insurers' cumulative drug expenditures by shifting drug use from cost‐share drugs to reference drugs. ➡ Index pricing may increase the use of generic drugs, reduce the use of brand‐name drugs, slightly reduce the price of generic drugs, and have little or no effect on the price of brand‐name drugs. ➡ It is uncertain whether maximum pricing affects drug expenditures. ➡ The effects of these policies on healthcare utilisation or health outcomes is uncertain. ➡ None of the included studies took place in a low‐income country. ➡ The effects of other pharmaceutical pricing and purchasing policies are uncertain. |
|
Marketing regulations ‐ Drugs Gilbody 2005 |
➡ Direct‐to‐consumer advertising increases patient demand for advertised medicines and the number of related prescriptions by doctors. ➡ We found no studies that reported on the impact of direct‐to‐consumer advertising on health outcomes. We are therefore uncertain of their effects. ➡ In light of the lack of evidence of the benefits, potential harms, and costs of direct‐to‐consumer advertising: ‐ the value of policies that allow for the increased use of direct to consumer advertising is uncertain at best; and ‐ rigorous monitoring and evaluation are warranted if such policies are implemented. |
| Authority and accountability for health professionals | |
|
Training and licensing ‐ Pre‐licensure education Pariyo 2009 |
➡ There is little evidence of the effects of interventions to increase the capacity of health professional training institutions, reduce student dropout rates or increase the number of students recruited from other countries into health professional training institutions. ➡ Academic advising programmes for minority groups may: ‐ increase the number of minority students enrolled in health sciences; ‐ slightly increase retention through to graduation; ‐ decrease differences in retention levels through to graduation between minority and non‐minority students in the health sciences. ➡ We found no studies of the effects of other pre‐licensure measures to increase health worker supply. |
|
Training and licensing ‐ Training district health system managers Rockers 2013 |
➡ Private contracting ("contracting in") of district health managers compared to direct employment by the Ministry of Health may improve access and utilisation of healthcare. It is uncertain whether contracting in improves health outcomes. ➡ Intermittent training programmes may increase knowledge of planning processes and monitoring and evaluation skills of district managers. ➡ The effects of other interventions are uncertain, including changes in how district managers are hired, strategies for retaining district managers such as making the positions more attractive, and other training programmes such as in‐service workshops with onsite support. |
|
Recruitment and retention strategies Grobler 2015 |
➡ It is uncertain whether any of the following types of interventions to recruit or retain health professionals increase the number of health professionals practising in in underserved areas ‐ Educational interventions (e.g. student selection criteria, undergraduate and postgraduate teaching curricula, exposure to rural and urban underserved areas) ‐ Financial interventions (e.g. undergraduate and postgraduate bursaries or scholarships linked to future practice location, rural allowances, increased public sector salaries) ‐ Regulatory strategies (e.g. compulsory community service, relaxing work regulations imposed on foreign medical graduates who are willing to work in rural or urban underserved areas) ‐ Personal and professional support strategies (e.g. providing adequate professional support and attending to the needs of the practitioners family) |
| Recruitment and retention strategiesRockers 2013 | ➡ Private contracting ("contracting in") of district health managers compared to direct employment by the Ministry of Health may improve access and utilisation of healthcare. It is uncertain whether contracting in improves health outcomes. ➡ Intermittent training programmes may increase knowledge of planning processes and monitoring and evaluation skills of district managers. ➡ The effects of other interventions are uncertain, including changes in how district managers are hired, strategies for retaining district managers such as making the positions more attractive, and other training programmes such as in‐service workshops with onsite support. |
|
Movement of health workers between public and private organisations Rutebemberwa 2014 |
➡ No rigorous studies have evaluated the effects of interventions to manage the movement of health workers between public and private organisations. ➡ There is a need for well‐designed studies to evaluate the impact of interventions that attempt to regulate health worker movement between public and private organisations in low‐income countries. |
|
Emigration and immigration policies Peñaloza 2011 |
➡ Lowering immigration restrictions in high‐income countries probably increases the migration of nurses from low‐ and middle‐income countries to high‐income countries. The effectiveness of interventions implemented in low‐ and middle‐income countries to decrease the emigration of health professionals is uncertain. No studies were found that evaluated such interventions. ➡ Low‐ and middle‐income countries should monitor changes in high‐income countrie immigration legislation, model the impact of proposed migration changes on their own retention of domestic health professionals, and lobby for immigration laws in high‐income countries that consider the health system needs of source countries. ➡ Rigorous studies are needed of the effectiveness of interventions designed to decrease the emigration of health professionals, particularly the effectiveness of interventions in low‐ and middle‐income countries. |
|
Dual practice Kiwanuka 2011 |
➡ No studies met the inclusion criteria for the review, as no rigorous studies have evaluated the effects of interventions to manage dual practice. ➡ There is a need for well‐designed studies to evaluate the impact of interventions that attempt to regulate health worker dual practice in low‐income countries. |
|
Authority and accountability for quality of practice ‐ Authority and accountability for quality of outpatient care ‐ Authority and accountability for quality assurance of hospital (inpatient) care Flodgren 2011 |
➡ It is uncertain whether external inspection results in improved compliance with accreditation standards, improved quality of care or decreased healthcare‐acquired infection (i.e. methicillin‐resistant Staphylococcus aureus) rates in hospitals. ➡ This review found no direct evidence on the effectiveness of external inspections of compliance with standard in ambulatory settings. We are therefore uncertain of the effects in this setting. ➡ This review found no direct evidence on the effectiveness of external inspections of compliance with standards in low‐income countries. |
| Stakeholder involvement | |
|
Stakeholder participation in policy and organisational decisions Nilsen 2010 |
➡ Consumer consultations in developing patient information probably: ‐ facilitate the development of material that is more relevant, readable and understandable to patients; ‐ improve patient knowledge; ‐ make little or no difference in decreasing the anxieties that patients may associate with clinical procedures. ➡ Consumer interviewers may lead to small differences in the results of satisfaction surveys compared to healthcare professional interviewers. ➡ It is uncertain whether telephone discussions compared with face‐to‐face meetings change consumer priorities for community health goals. ➡ Consumer consultation in the development of consent documents may have little or no impact on self‐reported participant understanding of the trial described in the consent document, satisfaction with study participation, adherence to the protocol or refusal to participate ➡ There are good arguments for introducing consumer involvement in low‐income countries. To accomplish this: ‐ strategies to overcome barriers such as low baseline levels of social participation, organisation and education should be explored; ‐ efforts to include consumers or families of disadvantaged groups should be considered in order to achieve inclusive representation; ‐ evaluations are needed of the effects of consumer involvement on healthcare decisions and how to achieve more effective consumer involvement. |
|
Community mobilisation Prost 2013 |
➡ Women's groups practising participatory learning and action probably improve newborn survival, may improve maternal survival, may slightly reduce stillbirths, and may be a cost‐effective strategy in rural areas in low‐ and middle‐income countries. ➡ The effectiveness of women's groups may depend on participation of a substantial proportion of pregnant women, adequate supervision and support, home visits, access to care, improving the quality of care, and adequate resources. |
|
Community mobilisation Heintze 2007 |
➡ Multi‐component community‐based dengue control programmes may reduce mosquito larval indices. ➡ Multi‐component community‐based dengue control programmes combined with chemical larvicides may reduce mosquito larval indices. ➡ Multi‐component community‐based dengue control programmes combined with fish and chemical larvicides may reduce mosquito larval indices. ➡ Multi‐component community‐based dengue control programmes combined with the use of crustaceans that eat mosquito larvae (Mesocyclops copepods) may reduce mosquito larval indices. ➡ It is uncertain whether multi‐component community‐based dengue control programmes combined with the use of crustaceans that eat mosquito larvae (Mesocyclops copepods) reduce dengue incidence. ➡ Most studies took place in low‐ and middle‐income countries. |
|
Patient information ‐ Public disclosure of performance data Fung 2008 |
➡ Public disclosure of performance for health plans: ‐ may lead to patients selecting health plans that have better quality ratings; ‐ has uncertain effects on quality improvement activities; ‐ may slightly improve health outcomes. ➡ Public disclosure of performance for hospitals: ‐ may lead to little or no difference in patient selection of hospitals; ‐ probably stimulates quality improvement activities; ‐ may improve health outcomes. ➡ Public disclosure of performance for individual healthcare providers: ‐ probably leads to patients selecting providers that have better quality ratings; ‐ has uncertain effects on quality improvement activities; ‐ may improve health outcomes. ➡ All of the included studies took place in high‐income countries. ‐ Public disclosure of performance may be difficult to implement in low‐income countries because of limitations of the ability of health facilities and providers to produce accurate data, the capacity to disseminate the data, the ability of patients to interpret the data and, in some places, the lack of choice available in terms of facilities or providers. |
8. Intervention‐outcome matrix for included reviews.
| Direction of effects and certainty of the evidencea | |||||||||
| Governance arrangement | Patient outcomes | Access, coverage, utilisation | Quality of care | Resource use | Social outcomes | Impacts on equity | Health care provider outcomes | Adverse effects b | Other |
| Authority and accountability for health policies | |||||||||
|
Interagency collaboration Hayes 2012 |
? ⊕⊖⊖⊖ ∅⊕⊕⊖⊖c |
NR | NR | NR | ∅⊕⊕⊖⊖d | NR | NR | NR | NR |
|
Decision‐making about what is covered by health insurance – restrictions on drug reimbursement Green 2010 |
? ⊕⊖⊖⊖e | ✔⊕⊕⊕⊖f | NR | ✔⊕⊕⊕⊖f | NR | NR | NR | NR | NR |
|
Policies to reduce corruption – fraud detection and response actions Rashidian 2012 |
NR | NR | NR | ?⊕⊖⊖⊖g | NR | NR | NR | NR | NR |
| Authority and accountability for organisations | |||||||||
|
Contracting out – to non‐state not‐for–profit providers Lagarde 2009 |
✔⊕⊕⊖⊖h | ✔⊕⊕⊖⊖i | NR | NR | NR | NR | NR | NR | NR |
|
Multi‐institutional arrangements – social franchising Koehlmoos 2009 |
NS | NS | NS | NS | NS | NS | NS | NS | NS |
| Authority and accountability for commercial products | |||||||||
|
Registration – drugs El‐Jardali 2015 |
NR | NR | NR | NR | NR | NR | NR | NR | ✔⊕⊕⊖⊖j |
|
Pricing and purchasing policies – medicines – reference pricing Acosta 2014 |
NR | ✔⊕⊕⊖⊖k | NR | ✔⊕⊕⊖⊖k | NR | NR | NR | NR | NR |
|
Pricing and purchasing policies – medicines – index pricing Acosta 2014 |
NR | ✔⊕⊕⊖⊖l | NR | ∅⊕⊕⊖⊖m | NR | NR | NR | NR | NR |
|
Marketing regulations – medicines direct‐to‐consumer advertising Gilbody 2005 |
NS | ✔✕⊕⊕⊕⊕n | NR | NS | NR | NR | NR | NR | NR |
| Authority and accountability for health professionals | |||||||||
|
Training and licensing – pre‐licensure education – minority academic advising programme Pariyo 2009 |
NR | NR | NR | NR | NR | ✔⊕⊕⊖⊖o | NR | NR | NR |
|
Training and licensing – manager training programme versus no training Rockers 2013 |
NR | NR | NR | NR | NR | NR | NR | NR | ✔⊕⊕⊖⊖p |
|
Recruitment and retention strategies – health professionals in underserved areas Grobler 2015 |
NS | NS | NS | NS | NS | ?⊕⊖⊖⊖q | NS | NS | NS |
|
Recruitment and retention strategies – private versus public contracts of district health managers Rockers 2013 |
?⊕⊖⊖⊖r | ✔⊕⊕⊖⊖r | NR | NR | NR | NR | NR | NR | NR |
|
Movement of health workers between public and private organisations Rutebemberwa 2014 |
NS | NS | NS | NS | NS | NS | NS | NS | NS |
|
Emigration and immigration policies – reducing immigration restrictions Peñaloza 2011 |
NS | ✕⊕⊕⊕⊖s | NS | NS | NR | NR | NR | NR | NR |
|
Dual practice Kiwanuka 2011 |
NR | NS | NS | NS | NS | NS | NS | NR | NR |
|
Authority and accountability for quality of practice – authority and accountability for quality of outpatient care – external inspection Flodgren 2011 |
NS | NS | ? ⊕⊖⊖⊖t | NR | NR | NR | NS | NR | NR |
| Stakeholder involvement | |||||||||
|
Stakeholder participation in policy and organisational decisions – communication forums Nilsen 2010 |
NS | NS | NS | NS | NS | NS | NS | NS | ? ⊕⊖⊖⊖u |
|
Stakeholder participation in policy and organisational decisions – consumer involvement in research Nilsen 2010 |
NS | NS | ✔⊕⊕⊖⊖v | NS | NS | NS | NS | NS | ∅⊕⊕⊖⊖w |
|
Stakeholder participation in policy and organisational decisions – consumer involvement in preparing patient information Nilsen 2010 |
∅⊕⊕⊕⊖x | NS | NS | NS | NS | NS | NS | NS | ✔⊕⊕⊕⊖y |
|
Community mobilisation – women's groups practising participatory learning and action Prost 2013 |
✔⊕⊕⊖⊖z ✔⊕⊕⊕⊖aa |
NS | NS | NS | NS | NS | NS | NS | NS |
|
Community mobilisation – community‐based dengue control Heintze 2007 |
✔⊕⊕⊖⊖bb ✔⊕⊕⊖⊖cc ? ⊕⊖⊖⊖dd |
NS | NS | NS | NS | NS | NS | NS | NS |
|
Patient information – public disclosure of performance data – health plans Fung 2008 |
∅⊕⊕⊖⊖ee | ✔⊕⊕⊖⊖ff | NS | NS | NS | NS | NS | NS | NS |
|
Patient information – public disclosure of performance data – hospitals Fung 2008 |
✔⊕⊕⊖⊖gg | ✔⊕⊕⊖⊖hh | ✔⊕⊕⊕⊖ii | NS | NS | NS | NS | NS | NS |
|
Patient information – public disclosure of performance data –individual healthcare providers Fung 2008 |
✔⊕⊕⊖⊖jj | ✔⊕⊕⊕⊖kk | NS | NS | NS | NS | NS | NS | NS |
| The certainty of the evidence is an assessment of how good an indication the research provides of the likely effect; i.e. the likelihood that the effect will be substantially different from what the research found. By 'substantially different' we mean a large enough difference that it might affect a decision. These judgements are made using the GRADE system and the following definitions. | |||||||||
| Ratings | Definitions | Implications | |||||||
| ⊕⊕⊕⊕ High |
This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different is low. | This evidence provides a very good basis for making a decision about whether to implement the intervention. Impact evaluation and monitoring of the impact are unlikely to be needed if it is implemented. | |||||||
| ⊕⊕⊕⊖ Moderate |
This research provides a good indication of the likely effect. The likelihood that the effect will be substantially different is moderate. | This evidence provides a good basis for making a decision about whether to implement the intervention. Monitoring of the impact is likely to be needed and impact evaluation may be warranted if it is implemented. | |||||||
| ⊕⊕⊖⊖ Low |
This research provides some indication of the likely effect. However, the likelihood that it will be substantially different is high. | This evidence provides some basis for making a decision about whether to implement the intervention. Impact evaluation is likely to be warranted if it is implemented. | |||||||
| ⊕⊖⊖⊖ Very low |
This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different is very high. | This evidence does not provide a good basis for making a decision about whether to implement the intervention. Impact evaluation is very likely to be warranted if it is implemented. | |||||||
a✓: a desirable effect; ∅: little or no effect; ?: an uncertain effect; ✕: an undesirable effect; NS: no studies found by this review that reported this outcome; NR: outcome not reported by this review. bOther than adverse effects on any of the outcomes in the previous columns. cLocal interagency collaborative interventions may lead to little or no difference in physical health and may slightly improve functional level in patients with psychiatric disorders, compared with standard care. It is uncertain whether local interagency collaborative interventions decrease mortality and mental health symptoms. dLocal interagency collaborative interventions may lead to little or no difference in quality of life. eIt is uncertain whether pharmaceutical policies that restrict reimbursements change health outcomes. fRestrictions to pharmaceutical reimbursement probably decrease targeted drug use in the short and long term and reduce expenditures on target drug or drug class. gIt is uncertain if prevention, detection and response interventions reduce healthcare fraud and abuse and related expenditures. hPatient outcomes (auto‐reporting of being sick in the past month, diarrhoea incidence) may be improved and household health expenditures reduced by contracting out. iContracting out services to non‐state not‐for‐profit providers may increase access to and utilisation of health services. jMedicine registration and multifaceted interventions (including a mix of regulations, training of inspectors, public‐private collaborations and legal actions against counterfeiters) may decrease the prevalence of counterfeit and substandard medicines; WHO prequalification of medicines may lead to a decrease in the failure rates of medicines undergoing quality testing. kReference pricing (a system in which a reference price is established within a country as the maximum level of reimbursement for a group of medicines) may reduce insurers' cumulative medicine expenditures; may increase the use of reference medicines; and may reduce the use of cost‐share medicines. lIndex pricing (a maximum refundable price to pharmacies for medicines within an index group of therapeutically interchangeable medicines) may increase the use of generic medicines and reduce the use of brand‐name medicines. mIndex pricing may slightly reduce the price of generic medicines and may have little or no effect on the price of brand‐name medicines. nDirect‐to‐consumer advertising increases people's requests for advertised medicines as well as prescription volumes for advertised medicine. The direction of the effect depends on the medicine. For instance, for essential medicines this may be a desirable effect but for non‐essential medicines this may be a harmful effect. oMinority academic advising programmes may increase the number of black health sciences students enrolled and slighlty increase retention to graduation. pManager training programmes may increase knowledge of planning processes and monitoring and evaluation skills. qIt is uncertain whether educational or financial interventions or regulatory or personal and professional support strategies to recruit or retain health professionals increase the number of health professionals practising in in underserved areas. rHiring district health managers to work within the Ministry of Health system through private contracts may improve access to health care and service use, but it is uncertain if this improves population health outcomes. sReducing immigration restrictions in high‐income countries probably increases the migration of nurses from low‐ and middle‐income countries to high‐income countries. tIt is uncertain whether external inspection adherence to accreditation standards improves quality of care. uIt is uncertain whether telephone discussions compared with face‐to‐face meetings change consumer priorities for community health goals. vConsumer interviewers may slightly improve responses regarding patient satisfaction, compared to staff interviewers. wConsumer consultation in the development of consent documents may have little or no impact on self‐reported participant understanding of the trial described in the consent document, satisfaction with study participation, adherence to the protocol or refusal to participate. xPatients probably experience little or no difference in their levels of worry or anxiety associated with procedures when they receive information material that has been developed following consumer consultation. yConsumer consultation in developing patient information material probably results in material that is more relevant, readable and understandable to patients, and probably improves the knowledge of patients who read the material. zWomen's groups practising participatory learning and action cycles may improve survival in mothers and may slightly reduce stillbirths. . aaWomen's groups practising participatory learning and action cycles probably improve survival in newborn babies. bbMulti‐component community‐based dengue control programmes may reduce mosquito larval indices, and such programmes combined with fish and chemical larvicides may reduce mosquito larval indices. ccMulti‐component community‐based dengue control programmes combined with the use of crustaceans that eat mosquito larvae may reduce mosquito larval indices. ddIt is uncertain whether multi‐component community‐based dengue control programmes combined with the use of crustaceans that eat mosquito larvae reduce dengue incidence. eePublic disclosure may lead to slight improvements in clinical outcomes for health plans. ffPublic disclosure may lead patients to select health plans with better quality ratings or to avoid those with worse ratings. ggMay lead to slight improvements in hospital clinical outcomes. hhMay lead to little or no difference in patient selection of hospitals. iiProbably stimulates hospitals to undertake quality improvement activities. jjPublic disclosure of performance data may improve clinical outcomes (risk‐adjusted mortality rates for surgeons) among individual providers. kkPublic disclosure probably influences users of health care services to select providers with better quality ratings or to avoid those with worse ratings.
We took into account other relevant considerations besides the findings of the included reviews when drawing conclusions about implications for practice (EPOC 2017). This includes considerations related to the applicability of the findings and likely impacts on equity. Our conclusions about implications for systematic reviews were based on types of governance arrangements for which we were unable to find a reliable, up‐to‐date review and on the limitations identified in the included reviews. This includes considerations related to the applicability of the findings and likely impacts on equity. Our conclusions about implications for future evaluations are based on the findings of the included reviews (EPOC 2017).
Results
We identified 7272 systematic reviews of health systems arrangements and implementation strategies. We excluded 6953 reviews from this overview following a review of titles and abstracts. We retrieved the full texts of 66 reviews for further detailed assessment, excluding 43 for the following reasons (Table 5): they had important methodological limitations (10 reviews), were out‐of‐date (7 reviews), focused on an area already covered by one of the included reviews (20 reviews), did not focus on the effects of interventions (2 reviews), or were of limited relevance to low‐income countries (4 reviews) (Figure 1). We considered two other reviews for inclusion but, after discussion, agreed that they were part of the scope of another of the overviews (Jia 2014; Maharaj 2015).We considered Ketelaar 2011 and WHO 2010 to be supplementary in that they contributed information about interventions for which other reviews were the main source of information (because those reviews, Fung 2008 and Grobler 2015, were more reliable, included more studies, or were more up‐to‐date). Appendix 5 lists the reviews still awaiting classification.
1.
Review flow diagram.
Description of included reviews
We included 19 systematic reviews published between 2005 and 2015 in this overview (Table 4). Of these, 13 were Cochrane Reviews and 6 non‐Cochrane reviews.
The reviews reported results from 172 studies and included the following study designs .
28 randomised trials (16.3%).
6 non‐randomised trials (3.5%).
15 controlled before‐after studies (8.7%).
62 interrupted time series studies (36.0%).
1 repeated measures study (0.6%).
56 observational study designs (32.6%).
3 studies used more than one design (1.7%).
1 before‐after study, reanalysed as an interrupted time series study (0.6%).
The number of studies included in each review ranged from zero (Koehlmoos 2009; Kiwanuka 2011; Rutebemberwa 2014) to 45 (Fung 2008). The dates of the most recent searches in the reviews ranged from October 2004 in Gilbody 2005 to April 2014 in Grobler 2015.
Nine reviews did not include any studies from low‐ or middle‐income countries (Gilbody 2005; Fung 2008; Pariyo 2009; Green 2010; Nilsen 2010; Hayes 2012; Rashidian 2012; Acosta 2014; Grobler 2015), and four reviews only included studies conducted in low‐ or middle‐income countries (Lagarde 2009; Prost 2013; Rockers 2013; El‐Jardali 2015). Overall, 74% of the studies from the included reviews took place in high‐income countries. Study settings varied and included primary care; home, workplace and community settings; and outpatient and inpatient settings in hospitals and non‐primary level health centres (Appendix 3). Health workers who participated in the studies included in the reviews included: physicians, nurses, pharmacists, psychologists, dentists, social workers and traditional healers. Recipients of care participating in studies included in the reviews included children, adults and pregnant mothers (Appendix 3). Outcomes examined by the reviews included: healthcare provider performance, patient outcomes, access to care, coverage, utilisation of health services, social outcomes, impacts on equity and adverse effects (Table 8).
We grouped the governance arrangements addressed in the reviews into five categories.
Authority and accountability for health policies: 3 reviews.
Authority and accountability for organisations: 2 reviews.
Authority and accountability for commercial products: 3 reviews.
Authority and accountability for health professionals: 7 reviews.
Stakeholder involvement: 4 reviews.
Methodological quality of included reviews
We present the methodological quality (reliability) of the included reviews in Table 6. One of the 19 included reviews, Rashidian 2012, had important methodological limitations, but we retained it in the overview because no better review was available. We judged the other 18 reviews to have only minor limitations.
We found a number of problems with respect to the identification, selection and critical appraisal of the included studies in reviews. Five reviews had some limitations in relation to the comprehensiveness of the search, and three reviews had some limitations in relation to study selection. We found few problems with respect to the analysis of the available evidence. Two reviews had limitations related to either the description of the extent of heterogeneity or the examination of factors that might explain differences in the results of included studies (Rashidian 2012 and Heintze 2007, respectively).
Effect of interventions
Table 7 summarises the key messages from the included reviews, and Table 8 presents the key findings of the different governance interventions considered by each of the included reviews as well as the certainty of this evidence by outcome. Table 9 summarises the effects and certainty of the evidence from the included reviews according to whether the interventions had desirable effects, little or no effect, undesirable effects, or uncertain effects. In the following text, we report the main findings of the included comparisons.
9. Summary of effects of interventions and certainty of evidence.
| Interventions found to have desirable effects on at least one outcome with moderate‐ or high‐certainty evidence and no moderate‐ or high‐certainty evidence of undesirable effects |
| Authority and accountability for health policies |
Decision‐making about what is covered by health insurance
|
| Authority and accountability for commercial products |
Marketing regulations
|
| Stakeholder participation in policy and organisational decisions |
Community mobilisation
|
|
Patient information |
|
| Interventions for which the certainty of the evidence was low or very low (or no studies were found) for all outcomes examined |
| Authority and accountability for health policies |
|
| Authority and accountability for organisations |
|
| Authority and accountability for commercial products |
|
| Authority and accountability for health professionals |
|
| Stakeholder participation in policy and organisational decisions |
|
aFor this intervention, the direction of the effect depends on the medicine. For instance, for essential medicines this may be a desirable effect (and is therefore listed as such above) but for non‐essential medicines this may be a undesirable effect.
Authority and accountability for health policies
Three reviews considered interventions related to authority and accountability for health policies (Green 2010; Hayes 2012; Rashidian 2012).
Interagency collaboration
Hayes 2012 examined the effects of interagency collaboration between local health and other local government agencies and services, comparing it with standard practice or no intervention. The review included 16 studies, all conducted in high‐income countries. The findings suggested that it is uncertain whether local interagency collaborative interventions decrease mortality or mental health symptoms (very low‐certainty evidence). The studies also suggest that these interventions may lead to little or no difference in physical health and quality of life but may slightly improve functional levels among people with psychiatric disorders, compared with standard ways of delivering services (low‐certainty evidence).
Decision‐making about what is covered by health insurance – restrictions on medicines reimbursement
Green 2010 included 29 studies in high‐income countries and assessed the effects of placing restrictions on the medicines reimbursed by health insurance systems. The review found that restrictions on reimbursement probably decrease the use of the targeted medicines as well as expenditures on targeted medicines or medicine classes (moderate‐certainty evidence). The impacts of such restrictions on health outcomes and health service utilisation were uncertain (very low‐certainty evidence). Review authors could not assess the impacts of such restrictions on equity measures, as none of the included studies reported this outcome.
Policies to reduce corruption
Rashidian 2012 studied the effects of interventions to reduce healthcare fraud. It included four studies from high‐income countries. The review found that it is uncertain if prevention, detection and response interventions reduce healthcare fraud and related expenditures (very low‐certainty evidence).
Authority and accountability for organisations
Two reviews considered interventions related to authority and accountability for organisations (Koehlmoos 2009; Lagarde 2009). The review addressing the effects of social franchising, Koehlmoos 2009, did not identify any eligible studies, so we do not discuss it further below.
Contracting out
Lagarde 2009 examined the effects of contracting out (sometimes called sub‐contracting) and included three studies conducted in middle‐income countries. The review found that contracting out services to non‐state, not‐for‐profit providers may increase access to and utilisation of health services (low‐certainty evidence). In addition, patient outcomes may be improved and household health expenditures reduced (low‐certainty evidence). None of the included studies presented evidence on whether contracting out was more effective than making a similar investment in the public sector. We are therefore uncertain of the effects of investing in contracting out compared to an equivalent investment in public sector health services.
Authority and accountability for commercial products
Three reviews considered interventions related to authority and accountability for commercial products (Gilbody 2005; Acosta 2014; El‐Jardali 2015).
Registration of medicines
El‐Jardali 2015 explored the effect of interventions for combating or preventing medicine counterfeiting (e.g. medicines with the wrong ingredients, without active ingredients, with insufficient active ingredients or with fake packaging). The review included 21 studies conducted in low‐ and middle‐income countries and found that it is uncertain whether the licensing of drug or medicines outlets reduces the prevalence of counterfeit medicines or the failure rates of medicines undergoing quality testing (very low‐certainty evidence). The review also found that medicine registration may decrease the prevalence of counterfeit and substandard medicines (low‐certainty evidence) and that the prequalification of medicines by WHO (in which manufacturers receive WHO‐approved certificates of good manufacturing practices) may lead to a decrease in the failure rates of medicines undergoing quality testing (low‐certainty evidence). Finally, multifaceted interventions (that include a mix of regulations, training of inspectors, public‐private collaborations and legal actions against counterfeiters) may be effective in decreasing the prevalence of counterfeit and substandard medicines (low‐certainty evidence).
Pricing and purchasing policies for pharmaceuticals
Acosta 2014 evaluated the effects of reference pricing (a system that establishes a benchmark or reference price within a country as the maximum level of reimbursement for a group of drugs or medicines), maximum pricing (a fixed, maximum price that a medicine can have within a health system) and index pricing (maximum refundable price to pharmacies for medicines within an index group of therapeutically interchangeable medicines). The 18 included studies took place in high‐income countries. Reference pricing may reduce insurers' cumulative medicine expenditures by shifting medicine use from cost‐share medicines (more expensive medicines in the same group as the reference medicines, for which patients have to pay the difference between the reference price and the price of the medicine purchased) to reference medicines; and may increase the use of reference medicines and reduce the use of cost‐share medicines (low‐certainty evidence). Index pricing may increase the use of generic medicines and may reduce the use of brand‐name medicines; may slightly reduce the price of generic medicines; and may have little or no effect on the price of brand‐name medicines (low‐certainty evidence). It is uncertain whether maximum pricing affects medicine expenditures (very low‐certainty evidence). The effects of reference pricing, maximum pricing and index pricing on healthcare utilisation or health outcomes is uncertain, as the included studies did not assess these outcomes.
Marketing regulations
Gilbody 2005 explored the effects of direct‐to‐consumer advertising of prescription‐only medicines. The review included four studies performed in high‐income countries and found that direct‐to‐consumer advertising increases people's requests for advertised medicines as well as the number of related prescriptions by doctors (high‐certainty evidence). The direction of the effect depends on the medicine. For instance, for essential medicines this may be a desirable effect but for non‐essential medicines this may be a undesirable effect. The review did not identify any studies that evaluated the impact of direct‐to‐consumer advertising on health outcomes or the cost‐effectiveness of such advertising.
Authority and accountability for health professionals
Seven reviews considered interventions related to authority and accountability for health professionals (Pariyo 2009; Flodgren 2011; Kiwanuka 2011; Peñaloza 2011; Rockers 2013; Rutebemberwa 2014; Grobler 2015). Kiwanuka 2011 examined the effects of interventions to improve the management of dual practice, in which healthcare providers hold more than one job, but did not identify any eligible studies. Likewise, Rutebemberwa 2014 assessed interventions to manage the movement of health workers between public and private organisations but did not include any studies. Therefore, we do not discuss either of these empty reviews below.
Training and licensing – pre‐licensure education
Pariyo 2009 examined the effects of changes in pre‐licensure education (the training of health professional students prior to their registration as professionals) on the supply of health workers. The review included two studies that addressed the effects of an academic advising programme for minority groups, in which training institutions in a high‐income country provide additional support for minority group students. The review found that such programmes may increase the number of minority group health sciences students enrolled, slightly increase retention to graduation and decrease the difference in retention levels to graduation between a minority group and those in other population groups (low‐certainty evidence). The review did not find any studies of the effects on the supply of health workers of other changes in pre‐licensure education.
Rockers 2013 examined the effects of interventions to hire, retain and train district health systems managers and included two studies conducted in four middle‐income countries. The review found that manager training programmes may increase knowledge of planning processes as well as managers' monitoring and evaluation skills, compared with no training (low‐certainty evidence).
Recruitment and retention strategies
Grobler 2015 examined strategies for the recruitment and retention of health workers practising in underserved and rural areas. The review included one study from a high‐income country (Taiwan), but it is uncertain whether educational or financial interventions, or regulatory, personal and professional support strategies to recruit or retain health professionals increase the number of health professionals practising in underserved areas, as the review did not identify any studies that evaluated such interventions.
Rockers 2013 examined the effects of interventions to hire, retain and train district health systems managers and included two studies conducted in four middle‐income countries. The review found that hiring district health managers to work within the Ministry of Health system through private contracts ('contracting in') may improve access to health care (health facilities open 24 hours and supplies and equipment available) and may increase use of antenatal care and other publicly funded services, compared to hiring managers through public sector contracts (low‐certainty evidence). However, it is uncertain whether this approach improves population health outcomes (very low‐certainty evidence).
Emigration and immigration policies
Peñaloza 2011 examined the effects of interventions for controlling the emigration of health professionals from low‐ and middle‐income countries. It included one study that evaluated the effect of a change to immigration legislation in the USA on the migration of nurses from the Philippines to the USA. It found that reducing immigration restrictions in high‐income countries probably increases the migration of nurses from low‐ and middle‐income to high‐income countries (moderate‐certainty evidence). The review did not identify any studies that evaluated the effectiveness of interventions implemented in low‐income countries to decrease the emigration of health professionals.
Authority and accountability for quality of care
Flodgren 2011 examined the effects on healthcare organisation behaviour, healthcare professional behaviour and patient outcomes of external inspection systems to improve adherence to external quality standards in organisations delivering health care. The review included one study each from a middle‐ and a high‐income country. The review found that it is uncertain whether external inspection of adherence to standards improves adherence and quality of care or decreases health‐acquired infection rates in hospitals (very low‐certainty evidence). This review did not find any studies of the effectiveness of external inspections of adherence to standards in ambulatory (outpatient) settings.
Stakeholder involvement
Four reviews considered interventions related to stakeholder involvement (Heintze 2007; Fung 2008; Nilsen 2010; Prost 2013).
Stakeholder participation in policy and organisational decisions
Nilsen 2010 examined the effects of interventions to involve consumers in developing healthcare policies and research, clinical practice guidelines and patient information material. The review included six randomised trials, all conducted in high‐income countries. One of these studies evaluated consumer involvement in policy development and found that it is uncertain whether telephone discussions change consumer priorities for community health goals compared with face‐to‐face meetings (very low‐certainty evidence). None of the other included studies assessed stakeholder participation in policy and organisational decisions, but rather assessed consumer involvement in developing patient information, delivering satisfaction with care interviews and developing informed consent forms for research.
Community mobilisation
Two reviews examined the effects of community mobilisation – strategies to empower people to organise themselves to address an issue of common concern, and to identify and employ available resources to change a given situation. Prost 2013 included seven cluster‐randomised trials from low‐ and middle‐income countries. The review found that women's groups practising participatory learning and action cycles may improve maternal survival and may slightly reduce stillbirths (low‐certainty evidence), and these interventions probably improve survival in newborn babies (moderate‐certainty evidence). Heintze 2007 included 11 studies of community‐based interventions for dengue control: 9 from middle‐income countries and 2 from high‐income countries. The review found that community‐based dengue control programmes that include some form of mobilisation may reduce mosquito larval indices (low‐certainty evidence).
Patient information – public disclosure of performance data
Fung 2008 examined the effects of public disclosure of performance data on health plans (including health insurance schemes, health maintenance organisations, private health insurance, etc.) as well as on hospitals and healthcare professionals, and included 45 studies from high‐income countries. The review found that public disclosure of performance data on health insurance scheme quality may lead people to select health plans with better quality ratings or to avoid those with worse ratings and may lead to slight improvements in clinical outcomes for health insurance schemes (low‐certainty evidence). Public disclosure of performance data on hospital quality may lead to little or no difference in patient selection of hospitals (low‐certainty evidence), probably stimulates hospitals to undertake quality improvement activities (moderate‐certainty evidence), and may lead to slight improvements in hospital clinical outcomes (low‐certainty evidence). Public disclosure of performance for individual healthcare providers probably leads to patients selecting providers that have better quality ratings (moderate‐certainty evidence) and may improve clinical outcomes among individual providers (low‐certainty evidence).
Discussion
Summary of main results
The evidence from the 19 included systematic reviews of governance arrangements for health systems in low‐income countries covers a range of strategies (e.g. at policy, organisational, commercial, health professional and stakeholder levels), involving diverse settings (geographical, health system level) and populations (managers, health professionals, patients). Of the 24 outcomes for which an intervention had a desirable effect, 7 were supported by evidence of moderate certainty and 17 by evidence of low certainty. The one outcome on which an intervention had an undesirable effect was supported by evidence of moderate certainty. For eight outcomes reported in the included reviews, we assessed the effects as uncertain (very low‐certainty evidence). We found high or moderate‐certainty evidence that interventions in the areas of restrictions on medicine reimbursement, community mobilisation, public disclosure of provider's performance data and patient involvement in decision‐making had desirable effects, with no undesirable effects.
Overall completeness and applicability of evidence
We identified reviews for 19 of 48 types of the governance arrangements. However, three of these reviews did not identify any eligible studies (Koehlmoos 2009; Kiwanuka 2011; Rutebemberwa 2014). We found only three reviews of strategies addressing authority and accountability for commercial products (Gilbody 2005; Acosta 2014; El‐Jardali 2015). Table 8 summarises the outcomes examined in the individual reviews. Only two reviews in the overview reported on the impacts of governance interventions on equity (Pariyo 2009; Grobler 2015). Three reviews reported outcomes related to resource use (Green 2010; Rashidian 2012; Acosta 2014), with none addressing cost‐effectiveness of the interventions. The sparse economic and equity data (in comparison to effectiveness data) limit assessment of the cost‐effectiveness and equity impacts of the interventions examined.
We incorporated our judgments about the applicability of summarised evidence (particularly, indirectness in relation to settings, populations and outcomes) into the GRADE assessments of its certainty, and we reported these applicability judgments in each of the SUPPORT Summaries. In general, it is difficult to draw firm conclusions regarding the applicability of the overview findings to low‐income countries. For many of the comparisons and outcomes, the evidence comes from studies conducted in high‐income countries (mainly the USA, UK, Canada and Australia) with very different on‐the‐ground realities and health systems arrangements. These differences are particularly important in relation to interventions that require substantial resources for design and implementation or that may require advanced technology or specialised skills for delivery, for instance systems for reimbursement and reference pricing for medicines (Green 2010; Acosta 2014), for fraud detection and response actions (Rashidian 2012), and for public disclosure of performance data (Fung 2008). These differences may also affect the applicability of interventions that are complex and may require substantial changes to the organisation of care – for example, improved collaboration between local health and local government agencies (Hayes 2012). It is therefore uncertain whether similar effects are likely if the interventions assessed in these reviews are implemented in low‐income countries.
Certainty of the evidence
Although some of the included reviews had methodological limitations, they were, for the most part, relatively well conducted (Table 6). The certainty of the evidence for the effect estimates for the interventions considered in these reviews ranged from very low to high (Table 8). Of the 39 outcomes considered by at least one study, the certainty of the evidence was high for 1 (3%), moderate for 8 (22%), low for 22 (56%) and very low for 8 (21%) (Table 10).
Potential biases in the overview process
Although our searches were relatively comprehensive, it is possible that we missed some relevant reviews. We also excluded reviews that were published before April 2005. It is possible that some of those reviews provide information that is still useful and that might supplement information provided by the included reviews. Although this cut‐off was arbitrary, it is unlikely that we excluded a substantial amount of useful information. However, 6 of the 19 included reviews were published before 2010, and it is possible that more recent evidence has been published since then that would change the review conclusions. None of these considerations would likely bias the results of this overview, but they might limit its comprehensiveness.
Classifying the interventions in the included reviews was sometimes uncertain and required judgment. For example, Jia 2014 assessed strategies for expanding health insurance coverage in vulnerable populations, and we decided to include it in the implementation strategies overview (Pantoja 2014). Another review evaluated the effects of rapid response systems on clinical outcomes (Maharaj 2015), and we included that one in the delivery overview (Ciapponi 2014). On the other hand, Fung 2008 related to the public disclosure of information directed to patients, and we included it in this overview instead of the implementation strategies overview. Although these judgments and differences in approaches to characterising governance interventions are unlikely to have introduced bias into this overview, they might result in some confusion, since there is no universally agreed upon classification system for governance arrangements. Moreover, any system for categorising health system interventions is, to some extent, arbitrary. A unified taxonomy for classifying health system interventions could facilitate explicit and systematic synthesis and interpretation of the existing body of evidence on health systems interventions across studies.
Judgments about the relevance of some interventions to low‐income countries (applicability, equity, economic considerations, and monitoring and evaluation) were sometimes difficult to make. While these judgments might have led to systematic errors, it seems unlikely. At least two overview authors made all of these judgments on the basis of the SUPPORT Summaries, which undergo peer review by the contact author of the summarised review and by individuals from low‐ and middle‐income countries.
Our general approach towards including reviews of studies from high‐income countries was inclusive rather than exclusive to enable readers to assess for themselves the relevance of the review findings. Similarly, our approach was to assume that findings are applicable to low‐income countries unless we identified differences between the study settings and settings in low‐income countries or factors that would likely modify the effects in low‐income countries.
Agreements and disagreements with other studies or reviews
We identified three related overviews of reviews published in the last 10 years (Lewin 2008; Scott 2009; Brunton 2015). These overviews addressed a range of governance arrangements in diverse settings and populations. As with our overview, most of the studies included in those overviews were from high‐income countries, and data on patient outcomes, equity, costs and cost‐effectiveness were scarce. We describe the findings of the three overviews briefly below.
Brunton 2015 aimed to understand the components of community engagement and the contribution of active content to health and social outcomes. The overview included three reviews, which found that more extensive community engagement (i.e. where community members design, deliver and evaluate health interventions) was associated with improved behavioural outcomes. More extensive engagement across design, delivery and evaluation was noted in studies where community engagement processes included bidirectional communication, collective decision‐making and intervention delivery training support to community members.
Lewin 2008 reviewed the effects of governance, financial and delivery arrangements, and implementation strategies that have the potential to improve the delivery of cost‐effective interventions in primary health care in low‐ and middle‐income countries. It included 21 systematic reviews, one of which addressed governance strategies for working with the private for‐profit sector – including franchising, regulation and accreditation – to improve the use of quality health services by people in low‐income settings (Patouillard 2007). We excluded this particular review in the present overview and did not identify any other eligible reviews that addressed governance strategies for working with the private for‐profit sector. Lewin 2008 did not find any systematic reviews that addressed other questions about governance arrangements for primary health care, including decentralisation of decision‐making, the regulation of training, or the control of corruption.
Scott 2009 included 23 reviews and assessed public scorecards and performance reports, external accreditation and clinical governance arrangements. Review authors found that studies have not adequately evaluated these interventions. These quality improvement strategies are heterogeneous, and methodological flaws in much of the evaluative literature limit the validity and generalisability of results. The authors assert that, based on current best available evidence, clinician/patient‐driven quality improvement strategies appear to be more effective than manager/policymaker driven ones. Some of the included reviews would have been excluded from our overview as they are more than 10 years old; some are covered in the delivery and implementation overviews; and some reviews address interventions that we did not consider to be highly relevant to low‐income countries.
Authors' conclusions
Well‐conducted, systematic Cochrane Reviews and non‐Cochrane reviews have evaluated a wide range of governance arrangements relevant to health systems in low‐income countries. The interventions assessed have targeted different levels of the health system and report a range of outcomes. However, in all the main categories of our taxonomy of governance arrangements for health systems there are important evidence gaps where primary studies and/or rigorous reviews are needed.
Implications for practice.
We found the following governance arrangements to be effective (moderate or high‐certainty evidence of desirable effects on at least one outcome and no moderate or high‐certainty evidence of undesirable effects).
Restrictions on medicine reimbursement for prescription medicines (Green 2010).
Public disclosure of hospitals' and individual healthcare providers' performance data (Fung 2008).
Consumer involvement in developing patient information materials (Nilsen 2010).
Women's groups practising participatory learning and action, in relation to newborn survival (Prost 2013).
The following governance arrangements have undesirable effects (moderate or high certainty evidence of at least one outcome with an undesirable effect, and no moderate or high certainty evidence of desirable effects).
Reducing immigration restrictions in high income countries for health workers from other settings (Peñaloza 2011).
The effects of the following governance arrangements are uncertain (low‐ or very‐low certainty evidence (or no studies were found) for all outcomes examined).
Interagency collaborative interventions (Hayes 2012).
Prevention, detection, and response interventions to reduce healthcare fraud and abuse and related expenditures (Rashidian 2012).
Contracting out service delivery to non‐state, not‐for‐profit providers (Lagarde 2009).
Social franchising within health services (Koehlmoos 2009).
Regulatory measures and multifaceted interventions to decrease the prevalence of counterfeit and substandard medicines, and WHO prequalification of medicines to reduce medicine quality testing failure rates (El‐Jardali 2015).
Index pricing and reference pricing for prescription medicines (Acosta 2014).
Pre‐licensure academic advising programmes for minority groups (Pariyo 2009).
Recruitment strategies for health professionals in underserved areas (Grobler 2015).
Movement of health workers between public and private organisations (Rutebemberwa 2014).
District manager training programmes, in relation to managers' knowledge of planning processes and monitoring and evaluation skills (Rockers 2013).
Private contracting ("contracting in") of district health managers compared to direct employment by the Ministry of Health (Rockers 2013).
Dual practice among health professionals (Kiwanuka 2011).
External inspection for adherence to accreditation standards in hospitals (Flodgren 2011).
Different communication forums (face‐to‐face, telephone discussions, mail surveys, etc.) for consumer involvement in healthcare policy (Nilsen 2010).
Community mobilisation for dengue control (Heintze 2007).
Public disclosure of data on the performance of health plans (Fung 2008).
Because the effects of these arrangements are uncertain, their health system impacts need to be monitored and evaluated if they are implemented.
Implications for research.
Based on the included reviews, we have identified gaps in primary research because of uncertainty about the applicability of the evidence to low‐income countries (Table 10) and low‐certainty evidence or a lack of studies (Table 11). It is notable that in 9 out of the 19 included reviews, all of the studies took place in high‐income countries, and in 15 of the 19 reviews there was at least one comparison where the certainty of the evidence on effects was low, or no studies were included. Further studies evaluating the effects of these interventions are needed, particularly in low‐income countries.
10. Priorities for primary research based on the applicability limitations to low‐income countries of the governance arrangements identifieda.
| Governance arrangement | Applicability limitations | ||
| Findings | Interpretation | ||
| Authority and accountability for health policies | |||
|
Interagency collaboration Hayes 2012 |
All studies included in this review took place in high income countries. | The reality of local agencies in low‐income countries is probably very different to that in high‐income countries so results reported in this review should be applied with caution in low‐income countries settings. | |
|
Decision‐making about what is covered by health insurance – restrictions on drug reimbursement Green 2010 |
All of the included studies took place in high‐income countries. Thus there is uncertainty regarding the transferability of the results to low‐ and middle‐income country settings. Participants were mainly senior citizens or low‐income adult populations in publicly subsidised or administered pharmaceutical benefit plans. Only two of the studies included in this review reported health outcome data, precluding any conclusions about the impact of the policies on patient outcomes. |
Applicability of these interventions to low‐income country settings depends on there being: – a regulatory framework; – an administrative and managerial system which support the implementation of the policy; –an insurance system with relatively broad medicines benefit; – efficient, timely access to patient‐specific information; – availability of preferred products incentivised by the re‐imbursement policy; – product quality assessments and prescriber and patient trust in the quality of preferred products. |
|
|
Policies to reduce corruption – fraud detection and response actions Rashidian 2012 |
There is no study from low income‐countries and only two from middle‐income countries. | Low‐income countries might be more prone and vulnerable to health care fraud and its consequences. When assessing the transferability of these findings to low‐income countries the following factors should be considered. – The availability of human and technical resources to combat fraud. – The acceptability and costs of the interventions. |
|
| Authority and accountability for organisations | |||
|
Subcontracting to non‐state not‐for‐profit providers Lagarde 2009 |
All of the studies took place in low‐ and middle‐income countries In the three included studies, the contracts were carried out with non‐governmental organisations (NGOs); no studies were found that evaluated contracts with private for‐profit providers. The studies provided very little description of the actual measures implemented by the contractor (management, organisation, salaries, and incentives) to achieve the goals established in the contract. |
Differences in health systems; patient and physician attitudes to NGOs; and legal restrictions may limit applicability of the findings. Subcontracting can be a potentially effective strategy in particular settings but it may be difficult for governments to re‐deploy public funds to private providers when available funds are already committed to public services. Factors that need to be considered to asses whether the intervention effects are likely to be transferable include: – the availability of not‐for‐profit organisations to carry out the contracts; – the capacity within the public sector for set up and monitor the contracts. |
|
|
Multi‐institutional arrangements –social franchising Koehlmoos 2009 |
The review did not find any studies conducted in low‐ and middle‐income countries that met its inclusion criteria. | Although social franchising is currently used and advocated in low‐ and middle‐income countries, no rigorous evaluations of its impacts (both positive and negative) are available. | |
| Authority and accountability for commercial products | |||
|
Registration – drugs El‐Jardali 2015 |
The studies were all undertaken in low‐ and middle‐income countries. The results suggest that drug registration, WHO prequalification of drugs, and multi‐faceted interventions may be effective in reducing the prevalence of counterfeit drugs. |
The findings are applicable to low‐ and middle‐ income settings. However, a country's existing pharmaceutical supply chain and infrastructure, availability of routine data on quality of drugs, qualified and skilled personnel, and financial resources may facilitate the transferability of the findings. While registration may be effective, it should probably encompass both domestic manufacturers and importers and be complemented with routine postmarketing surveillance to sustain the quality of drugs circulating in the market. Countries that rely heavily on imported drugs may consider opting for drugs that are WHO‐prequalified. However, even among WHO‐prequalified products, the quality may vary depending on the country of export. The success of multifaceted interventions requires collaborations with drug regulatory bodies, skilled human resources, and technical capacity for routine drug inspections. |
|
|
Reference pricing Acosta 2014 |
All of the 18 studies included were in high‐income countries. | The effectiveness of reference pricing policy in low‐income countries may depend on factors such as: – health systems structure and settings as copayments, reimbursment and cost share; – access to prices data sources; – availability of adequate incentives for healthcare providers, patients, physicians, pharmacists and pharmaceutical companies to comply with the reference pricing policy; – significant price differences between the drugs in the intervention group before reference pricing is introduced; – clear information for managers, clinicians and patients; – availability and access to drugs in the reference group; – a regulatory framework that allows generic substitution or prescribing by international non‐proprietary name (INN); – appropriate exemptions (exemptions that are too limited could lead to higher co‐payments for appropriate use of more expensive drugs and incentives to use a less effective drug. Exemptions that are too broad could reduce savings by not shifting drug use towards appropriate use of less expensive drugs.). |
|
|
Marketing regulations – Drugs direct‐to‐consumer advertising Gilbody 2005 |
The studies, all conducted in high‐income countries, show that direct‐to‐consumer advertising alters prescribing behaviour and volume, but no studies examined the impact of such advertising on health outcomes | Given the absence of any evidence of improvement in health outcomes from direct‐to‐consumer advertising, its benefits are uncertain in any setting. | |
| Authority and accountability for health professionals | |||
|
Pre‐licensure education Pariyo 2009 |
All included studies took place in high‐income countries. | The challenges faced in health care worker education in high‐ and low‐income countries are qualitatively and quantitatively different (e.g. the availability of funds, laws regarding equity and awareness of these, job prospects including remuneration, and curricula). Appropriate interventions could be expected to have a comparatively higher impact in low‐income countries, where alternatives and opportunities are generally more limited than in high‐income countries. However, there is no evidence regarding the effects of such interventions. | |
|
Recruitment and retention strategies Grobler 2015 |
Some observational studies, mostly from high‐income countries, suggest that some interventions, such as selecting students from rural areas, exposing students to clinical rotations in rural areas, or financial incentive programmes might increase the number of health professionals in underserved areas. However, the certainty of this evidence is very low. | Economic and cultural differences, differences between health system structures, and differences in state and educational institutional capacity to regulate and manage various types of interventions may limit the applicability of findings from high‐ to low‐income countries. | |
|
Training/recruitment and retention strategies Rockers 2013 |
The two included studies took place in low and middle income countries. Tested in a low income country, there is uncertainty about the impact of having private contracts (contract‐in districts) compared to public contracts of district health managers. |
The capacity and strength of the government to oversee and supervise districts with private contracts could be an important issue to consider when it comes to assure the attainment of public regulations and goals. The level of power decentralisation in the districts might change the impact of policies related with health managers. The higher the degree of decentralisation, the higher the impact they might have. |
|
|
Movement of health workers between public and private organisations Rutebemberwa 2014 |
No studies met the inclusion criteria for the review. | Health worker availability remains one of the key barriers to strengthening health systems in low‐income countries. Effective interventions to manage the movement of health professionals could help to address this and need to be evaluated rigorously. | |
|
Emigration and immigration policies – reducing immigration restrictions Peñaloza 2011 |
The available evidence is based on an intervention made in a high‐income country. | Policies in high‐income countries may have an effect on the number of health workers migrating from low‐ and middle‐income countries. Low‐ and middle‐income countries have little direct influence on high‐income country policies, including immigration policies. However, low‐ and middle‐income countries may attempt to influence such policies by means of diplomacy, lobbying, or public relations before they are enacted. |
|
|
Dual practice Kiwanuka 2011 |
No studies met the inclusion criteria for the review. | Dual practice may be more of a problem in low‐income countries, due to low wages in the public sector, and interventions to manage it may have different effects, e.g. the risk of health professionals migrating is likely to be greater in low‐income countries compared to high‐income countries. | |
|
Authority and accountability for quality of practice – authority and accountability for quality of outpatient care – external inspection Flodgren 2011 |
Neither of the two studies included in this review took place in a low‐income country: one was done in South Africa and the other in England. Both studies assessed the effect of external inspection of compliance of different standards on quality of hospital services. According to the findings in this review, it is uncertain whether external inspection contributes or not to improve quality of health services in hospital setting. |
External inspection of compliance standards may have varying acceptability and impact across different healthcare and cultural settings; may involve different components from training to organisational restructuring; and may impact in different ways on consumer and provider satisfaction across different settings Although quality of care is an objective of care in all health systems, it is not possible to be confident about the applicability of the reported interventions to low income countries and to settings other than hospital care |
|
| Stakeholder involvement | |||
|
Stakeholder participation in policy and organisational decisions – consumer involvement in preparing patient information Nilsen 2010 |
All the studies took place in high‐income countries. Some interventions used technologies such as telephones and email. Baseline levels of consumers involvement were not reported. |
Strategies to overcome barriers such as low baseline levels of social participation and education should be explored when considering consumer involvement in low‐income countries. Training and support may be essential. The attitudes and the perspectives of health professionals and policymakers regarding consumer involvement should also be considered. As the availability of communication technologies may be a problem, face‐to‐face involvement may be most appropriate. |
|
|
Community mobilisation – women's groups practising participatory learning and action Prost 2013 |
All 7 studies took place in low‐and middle‐income countries, including Bangladesh, Malawi, India and Nepal. | The use of women's groups practicing participatory learning and action probably decreases newborn mortality and may reduce maternal mortality in rural areas in low‐income countries. However, its effectiveness may depend on participation of a substantial proportion of pregnant women. It might also depend on adequate supervision and support, home visits, access to care, improving the quality of care, and adequate resources. The intervention might be less effective in urban areas if there is less community cohesion and interaction among women included in women's groups, and higher baseline use of health services. |
|
|
Community mobilisation – community‐based dengue control Heintze 2007 |
10 out of 11 studies included in the systematic review took place in low‐ and middle‐income countries. | These findings are applicable to low‐income countries; however, the availability acceptability and costs of the interventions should be considered. | |
|
Patient information – public disclosure of performance data Fung 2008 |
The studies, all conducted in high‐income countries, provided mixed evidence for using the public disclosure of performance data to improve the quality of care. | There is no evidence to date that the public disclosure of performance data affects the quality of care. Even if public disclosure were effective in improving quality of care in high‐income countries, the results would not be directly transferable to low‐income country settings because of differences in health infrastructure, the ability of health facilities and providers to produce accurate data, the capacity to disseminate the data, and the ability of consumers to interpret the data. There is a need for high‐quality studies of public disclosure of performance data in high‐, middle‐ and low‐income countries. |
|
aPriorities for primary research are based on the applicability limitations to low‐income countries of the governance arrangement interventions identified by the included reviews. We did not search for additional primary studies.
11. Priorities for primary research based on insufficient evidence for important outcomesa,b.
| Governance arrangement | Included review | No studies | Very low certainty of evidence | Low certainty of evidence |
| Authority and accountability for health policies | ||||
| Interagency collaboration | Hayes 2012 | PO, ACU, QoC, RU | PO | PO |
| Decision‐making about what is covered by health insurance – Restrictions on drug reimbursement | Green 2010 | QoC | PO | — |
| Policies to reduce corruption – fraud detection and response actions | Rashidian 2012 | PO, ACU, QoC | RU | — |
| Authority and accountability for organisations | ||||
| Subcontracting to non‐state not‐for‐profit providers | Lagarde 2009 | QoC, RU | — | PO, ACU |
|
Multi‐institutional arrangements Social franchising |
Koehlmoos 2009 | PO, ACU, QoC, RU | — | — |
| Authority and accountability for commercial products | ||||
| Registration – drugs | El‐Jardali 2015 | — | — | PO, ACU, QoC, RU |
| Reference pricing – reference and index price | Acosta 2014 | PO, QoC | — | ACU, RU |
| Marketing regulations – drugs direct to consumer advertising | Gilbody 2005 | PO, QoC, RU | — | — |
| Authority and accountability for health professionals | — | — | — | — |
|
Training and licensing Pre‐licensure education – minority academic advising programme |
Pariyo 2009 | PO, ACU, QoC, RU | — | — |
| Recruitment and retention strategies | Grobler 2015 | PO, ACU, QoC, RU | — | |
| Training and licensing/recruitment and retention strategies | Rockers 2013 | QoC, RU | PO, ACU | — |
| Movement of health workers between public and private organisations | Rutebemberwa 2014 | PO, ACU, QoC, RU | — | — |
| Emigration and immigration policies – reducing immigration restrictions | Peñaloza 2011 | PO, QoC, RU | — | — |
| Dual practice | Kiwanuka 2011 | PO, ACU, QoC, RU | — | — |
|
Authority and accountability for quality of practice Authority and accountability for quality of outpatient care – external inspection |
Flodgren 2011 | PO, ACU, RU | QoC | — |
| Stakeholder involvement | ||||
| Stakeholder participation in policy and organisational decisions ‐ consumer involvement in preparing patient information | Nilsen 2010 | PO, ACU, RU | — | QoC |
| Community mobilisation – women's groups practising participatory learning and action | Prost 2013 | ACU, QoC, RU | — | — |
| Community mobilisation – community‐based dengue control | Heintze 2007 | ACU, QoC, RU | — | — |
|
Patient information Public disclosure of performance data |
Fung 2008 | QoC, RU | — | PO, ACU |
ACU: access, coverage and utilisation outcomes; PO: patient outcomes; QoC: quality of care outcomes; RU: resource use outcomes. aWe have included here only priorities for research on the effects of governance arrangements based on the included reviews for each category of the health systems taxonomy. Since we did not search for primary studies we cannot discard primary evidence outside this review‐based approach.
The included reviews rarely reported social outcomes, resource use, impacts on equity or adverse (undesirable or unintended) effects (Table 8). Systematic reviews and updates of reviews should include all outcomes that are relevant to decision‐makers and those groups affected by governance arrangements. In addition, there is a wide range of interventions for which we did not find a reliable up‐to‐date systematic review (Table 12), including the effects of governance arrangements affecting what or who is covered by health insurance; policies to manage absenteeism; requirements for monitoring or evaluation; organisational policies for accrediting healthcare providers; regulation of insurance provision; multi‐institutional arrangements for coordinating care; regulation of registration, patents, profits and liability for commercial products; regulation of professional competence and liability; and regulation of patients' rights.
12. Priorities for new systematic reviews on governance arrangements in low‐income countries.
| Governance arrangement | What we found |
| Authority and accountability for health policies | |
| Decentralised versus centralised authority for health services | No reviews identified |
| Policies that regulate what drugs are reimbursed | No reviews identified |
| Policies that regulate what services are reimbursed | No reviews identified |
| Restrictions on reimbursement for health insurance | No reviews identified |
| Strategies for expanding health insurance coverage | No reviews identified |
| Policies to manage absenteeism | No reviews identified |
| Requirements for monitoring or evaluation | No reviews identified |
| Authority and accountability for organisations | |
| Ownership | Review in progress (Herrera 2013) |
| Stewardship of private health services | No reviews identified |
| Accreditation | No reviews identified |
| Provision of drug insurance | Review in progress (Pantoja 2015) |
| Provision of health insurance | No reviews identified |
| Policies that regulate interactions between donors and governments | No reviews identified |
| Governance arrangements for coordinating care across multiple providers | No reviews identified |
| Mergers | No reviews identified |
| Authority and accountability for commercial products | |
| Registration of health technology | No reviews identified |
| Patents and profits of drugs | No reviews identified |
| Patents and profits of health technology | No reviews identified |
| Pricing and purchasing policies of health technology and services | No reviews identified |
| Marketing regulations for health technology and services | No reviews identified |
| Sales and dispensing policies for drugs | Review in progress (Peñaloza 2015) |
| Liability for commercial products | No reviews identified |
| Authority and accountability for health professionals | |
| Licensure of health professionals | No reviews identified |
| Specialty certification | No reviews identified |
| Scope of practice | No reviews identified |
| Authority and accountability for quality assurance of hospital care | No reviews identified |
| Professional competence | No reviews identified |
| Professional liability | No reviews identified |
| Stakeholder involvement | |
| Community monitoring | No reviews identified |
| Patient information about drugs | No reviews identified |
| Patients' rights | No reviews identified |
Acknowledgements
We would like to thank the following editors and peer referees who provided comments to improve the overview: Sasha Shepperd (editor), Kaelan Moat, Rhona Mijumbi‐Deve and to Meggan Harris for copy‐editing the overview.
We would also like to acknowledge the following colleagues who helped to produce the SUPPORT Summaries upon which this overview is based: Racha Fadlallah, Fadi El‐Jardali, Elie Akl, Taryn Young, Peter Steinmann, Primus Che Chi and Yasser Sami Amer.
Additionally, we thank Susan Munabi‐Babigumira, Atle Fretheim, Simon Goudie and Hanna Bergman for editing some of the SUPPORT Summaries as well as the review authors and others who provided feedback on them.
Charles S Wiysonge's work is supported by the South African Medical Research Council and the National Research Foundation of South Africa (Grant Numbers: 106035 and 108571).
The Norwegian Satellite of the Effective Practice and Organisation of Care (EPOC) Group receives funding from the Norwegian Agency for Development Co‐operation (Norad), via the Norwegian Institute of Public Health to support review authors in the production of their reviews.
This overview is a product of the Effective Health Care Research Consortium, which provided funding to make this overview open access. The Consortium is funded by UK aid from the UK Government for the benefit of developing countries (Grant: 5242). The views expressed in this overview do not necessarily reflect UK government policy.
Appendices
Appendix 1. PubMed and LILACS search strategies
PubMed
From 2000 to present. Update: weekly
#1. MEDLINE[Title/Abstract]
#2. (systematic[Title/Abstract] AND review[Title/Abstract])
#3. meta analysis[Publication Type]
#4. #1 OR #2 OR #3 (Methods filter for systematic reviews – Clinical Queries – Max Specificity)
#5. overview[Title] AND (reviews[Title] OR systematic[Title]
#6. meta‐review[Title]
#7. review of reviews[Title]
#8. review[Title] AND systematic reviews[Title]
#9. umbrella[Title] AND (review[Title] OR reviews[Title] OR systematic[Title])
#10. policy[Title] AND (brief[Title] OR evidence[Title])
#11. #5 OR #6 OR #7 OR #8 OR #9 OR #10 (Methods filter for overviews)
#12. #4 OR #11 (Methods filter for systematic reviews and for overviews)
LILACS
From 2000 to present. Update: monthly
(TW:"revision sistematica" OR TW:"revisao sistematica" OR TW:"systematic review" OR MH:"review literature as topic" OR MH:"meta‐analysis as topic" OR PT:"meta‐analysis")
OR
(PT:revision AND (TW:metaanal$ OR TW:"meta‐analysis" OR TW:"metaanalise" OR TW:"meta‐analisis" OR TI:overview$ OR TW:"estudio sistematico" OR TW:"systematic study" OR TW:"estudo sistematico" OR TI:review OR TI:revisao OR TI:revision OR TI:systematic OR TI:sistematico))
OR
((TW:overview OR TW:"estudio sistematico" OR TW:"systematic study" OR TW:"estudo sistematico") AND (TI:review OR TI:revisao OR TI:revision OR TI:systematic OR TI:sistematico))
CINAHL (EBSCO)
From 2000 to present. Update: monthly
((TI meta analys* or AB meta analys*) or (TI systematic review or AB systematic review))
PsycINFO (EBSCO)
From 2000 to present. Update: monthly
meta‐analysis OR search*
EMBASE (Ovid)
From 2000 to present. Update: monthly
meta‐analysis.tw. OR systematic review.tw
Appendix 2. SUPPORT Summaries checklist for making judgments about how much confidence to place in a systematic review
| Review: | |
| Assessed by: | |
| Date: | |
| Section A:Methods used to identify, include and critically appraise studies | |
|
A.1 Were the criteria used for deciding which studies to include in the review reported? Did the authors specify: _ Types of studies _ Participants _ Intervention(s) _ Outcome(s) Coding guide ‐ check the answers above YES: All four should be yes |
_ Yes _ Can't tell/partially _ No |
| Comments (note important limitations or uncertainty) | |
|
A.2 Was the search for evidence reasonably comprehensive? Were the following done: _ Language bias avoided (no restriction of inclusion based on language) _ No restriction of inclusion based on publication status _ Relevant databases searched (including Medline + Cochrane Library) _ Reference lists in included articles checked _ Authors/experts contacted Coding guide ‐ check the answers above: YES: All five should be yes PARTIALLY: Relevant databases and reference lists are both ticked off |
_ Yes _ Can't tell/partially _ No |
| Comments (note important limitations or uncertainty) | |
|
A.3 Is the review reasonably up‐to‐date? Were the searches done recently enough that more recent research is unlikely to be found or to change the results of the review? Coding guide – consider how many years since the last search (e.g. if more than 10 years the review is unlikely to be up‐to‐date) and whether there is ongoing research |
_ Yes _ Can't tell/not sure _ No |
| Comments (note important limitations or uncertainty) | |
|
A.4 Was bias in the selection of articles avoided? Did the authors specify: _ Explicit selection criteria _ Independent screening of full text by at least 2 reviewers _ List of included studies provided _ List of excluded studies provided Coding guide ‐ check the above YES: All four should be yes |
_ Yes _ Can't tell/partially _ No |
| Comments (note important limitations or uncertainty) | |
|
A.5 Did the authors use appropriate criteria to assess the risk for bias in analysing the studies that are included?†( See Appendix for an example of criteria ‐ Assessing Risk of Bias Criteria for EPOC Reviews) _ The criteria used for assessing the risk of bias were reported _ A table or summary of the assessment of each included study for each criterion was reported _ Sensible criteria were used that focus on the risk of bias (and not other qualities of the studies, such as precision or applicability) Coding guide ‐ check the above YES: All four should be yes |
_ Yes _ Can't tell/partially _ No |
| Comments (note important limitations or uncertainty) | |
|
A.6 Overall – how would you rate the methods used to identify, include and critically appraise studies? Summary assessment score A relates to the 5 questions above. If the "No"or "Partial"option is used for any of the questions above, the review is likely to have important limitations. Examples of major limitations might include not reporting explicit selection criteria, not providing a list of included studies or not assessing the risk of bias in included studies. |
_ Major limitations (limitations that are important enough that the results of the review are not reliable and they should not be used in the policy brief) _ Important limitations (limitations that are important enough that it would be worthwhile to search for another systematic review and to interpret the results of this review cautiously, if a better review cannot be found) _ Reliable (only minor limitations) |
| Comments (note any major limitations or important limitations). | |
| Section B:Methods used to analyse the findings | |
|
B.1 Were the characteristics and results of the included studies reliably reported? Was there: _ Independent data extraction by at least 2 reviewers _ A table or summary of the characteristics of the participants, interventions and outcomes for the included studies _ A table or summary of the results of the included studies. Coding guide ‐ check the answers above YES: All three should be yes |
_ Yes _ Partially _ No _ Not applicable (e.g. no included studies) |
| Comments (note important limitations or uncertainty) | |
| B.2 Were the methods used by the review authors to analyse the findings of the included studies reported? | _ Yes _ Partially _ No _ Not applicable (e.g. no studies or no data) |
| Comments (note important limitations or uncertainty) | |
|
B.3 Did the review describe the extent of heterogeneity? _ Did the review ensure that included studies were similar enough that it made sense to combine them, sensibly divide the included studies into homogeneous groups, or sensibly conclude that it did not make sense to combine or group the included studies? _ Did the review discuss the extent to which there were important differences in the results of the included studies? _ If a meta‐analysis was done, was the I2, chi square test for heterogeneity or other appropriate statistic reported? |
_ Yes _ Can't tell/partially _ No _ Not applicable (e.g. no studies or no data) |
| Comments (note important limitations or uncertainty) | |
|
B.4 Were the findings of the relevant studies combined (or not combined) appropriately relative to the primary question the review addresses and the available data? How was the data analysis done? _ Descriptive only _ Vote counting based on direction of effect _ Vote counting based on statistical significance _ Description of range of effect sizes _ Meta‐analysis _ Meta‐regression _ Other: specify _ Not applicable (e.g. no studies or no data) How were the studies weighted in the analysis? _ Equal weights (this is what is done when vote counting is used) _ By quality or study design (this is rarely done) _ Inverse variance (this is what is typically done in a meta‐analysis) _ Number of participants _ Other, specify: _ Not clear _ Not applicable (e.g. no studies or no data) Did the review address unit of analysis errors? _ Yes ‐ took clustering into account in the analysis (e.g. used intra‐cluster correlation coefficient) _ No, but acknowledged problem of unit of analysis errors _ No mention of issue _ Not applicable ‐ no clustered trials or studies included Coding guide ‐ check the answers above If narrative OR vote counting (where quantitative analyses would have been possible) OR inappropriate table, graph or meta‐analyses OR unit of analyses errors not addressed (and should have been) the answer is likely NO. If appropriate table, graph or meta‐analysis AND appropriate weights AND the extent of heterogeneity was taken into account, the answer is likely YES. If no studies/no data: NOT APPLICABLE If unsure: CAN'T TELL/PARTIALLY |
_ Yes _ Can't tell/partially _ No _ Not applicable (e.g. no studies or no data) |
| Comments (note important limitations or uncertainty) | |
|
B.5 Did the review examine the extent to which specific factors might explain differences in the results of the included studies? _ Were factors that the review authors considered as likely explanatory factors clearly described? _ Was a sensible method used to explore the extent to which key factors explained heterogeneity? _ Descriptive/textual _ Graphical _ Meta‐regression _ Other |
_ Yes _ Can't tell/partially _ No _ Not applicable (e.g. too few studies, no important differences in the results of the included studies, or the included studies were so dissimilar that it would not make sense to explore heterogeneity of the results) |
| Comments (note important limitations or uncertainty) | |
|
B.6 Overall ‐ how would you rate the methods used to analyse the findings relative to the primary question addressed in the review? Summary assessment score B relates to the 5 questions in this section, regarding the analysis. If the "No” or "Partial” option is used for any of the 5 preceding questions, the review is likely to have important limitations. Examples of major limitations might include not reporting critical characteristics of the included studies or not reporting the results of the included studies. |
_ Major limitations (limitations that are important enough that the results of the review are not reliable and they should not be used in the policy brief) _ Important limitations (limitations that are important enough that it would be worthwhile to search for another systematic review and to interpret the results of this review cautiously, if a better review cannot be found) _ Reliable (only minor limitations) |
| Use comments to specify if relevant, to flag uncertainty or need for discussion | |
| Section C:Overall assessment of the reliability of the review | |
| C.1 Are there any other aspects of the review not mentioned before which lead you to question the results? | _ Additional methodological concerns _ Robustness _ Interpretation _ Conflicts of interest (of the review authors or for included studies) _ Other _ No other quality issues identified |
|
C.2 Based on the above assessments of the methods how would you rate the reliability of the review? _ Major limitations (exclude); briefly (and politely) state the reasons for excluding the review by completing the following sentence: This review was not included in this policy brief for the following reasons: Comments (briefly summarise any key messages or useful information that can be drawn from the review for policy makers or managers): _ Important limitations ; briefly (and politely) state the most important limitations by editing the following sentence, if needed, and specifying what the important limitations are: This review has important limitations. _ Reliable ; briefly note any comments that should be noted regarding the reliability of this review by editing the following sentence, if needed: This is a good quality systematic review with only minor limitations. | |
Appendix 3. Characteristics of included reviews
| Authority and accountability for health policies | ||
| Interagency collaboration | ||
| Hayes 2012 | ||
| Review objective: to evaluate the effects of interagency collaboration between local health and local government agencies on health outcomes in any population or age group. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised trials , non‐randomised trials, controlled before‐after studies and interrupted time series studies that assess any interventions of interagency collaboration and partnership and local government agencies | This review included 16 studies: 7 randomised trials(7 studies), 4 non‐randomised trials(4 studies), 4controlled before‐after studies(4 studies) and 1 interrupted time series study. 11 studies were included in the meta‐analysis. 7 studies reported on interventions to improve the care or treatment of patients and 9 studies about health education, health promotion or disease prevention |
| Participants | All population types and all age groups were included | Studies were delivered through community and primary care services (8 studies), in schools (5 studies), and in the wider community (3 studies). |
| Settings | Any local or national setting | Studies took place in the UK (7 studies), Denmark (1 study), Sweden (1 study), Norway and Sweden (1 study), the Netherlands (1 study), the USA (2 studies), Canada (1 study), Israel (1 study), and Australia (1 study). |
| Outcomes | Mortality, morbidity and behavioural change | A variety of outcomes were reported, including behavioural changes, morbidity and healthcare process |
| Date of most recent search: December 2011 | ||
| Limitations: This is a well‐conducted systematic review with only minor limitations. | ||
| Decision‐making about what is covered by health insurance | ||
| Green 2010 | ||
| Review objective: to determine the effects of a pharmaceutical policy restricting the reimbursement of selected medications on medicine use, healthcare utilisation, health outcomes and costs (expenditures). | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised and non‐randomised trials, interrupted time series studies including repeated measures studies, and controlled before‐after studies assessing prescribing policies – introduction of restriction to reimbursement, relaxation of previously instituted restrictions to reimbursement, or exemption from restrictive policies for targeted cost‐effective medicines | 24 studies evaluating restrictions to reimbursement policies. Most interventions were prior authorization. 5 studies evaluated policies of releasing or relaxing past restrictions to reimbursement. All of the studies were interrupted time series. |
| Participants | Healthcare consumers and providers within a large jurisdiction or system of care (regional, national or international) | Participants were predominantly the beneficiaries of publicly subsidised or administered pharmaceutical insurance plans – most often senior citizens aged 65 years or over and low‐income adult populations. |
| Settings | All settings | Health insurance systems with substantial coverage of medicines in the USA (14 studies), Canada (11 studies), Norway (2 studies) and Denmark (2 studies) |
| Outcomes | Primary outcomes: medicine use (prescribed, dispensed or actually used), healthcare utilisation, health outcomes, costs (expenditures). Secondary outcomes: changes in equity of access to medicines, changes in access to medically necessary medicines by disadvantaged groups, changes in the distribution of financial burden | Medicine use and medicine expenditures (24 studies), health outcome data (2 studies), healthcare utilisation (9 studies) |
| Date of most recent search: MEDLINE (2005 to January 2009) and other databases (2005 to October 2008) | ||
| Limitations: This is a well‐conducted systematic review with only minor limitations; however, the most recent searches were in January 2009. | ||
| Policies to reduce corruption | ||
| Rashidian 2012 | ||
| Review objective: to assess the effectiveness of interventions to combat healthcare fraud and abuse. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Interventional studies with or without a concurrent control group assessing any intervention to combat healthcare fraud (including prevention, detection, and response interventions) | 4 studies were included: 3 assessing detection actions and 2 response actions. The study designs were: longitudinal with concurrent control group (1 study), data mining (2 studies) and before‐after study (1 study). |
| Participants | Providers, patients or insured people, insurers (third party payers) | Taiwan's National Health Insurance, Medicare and Medicaid (in USA) |
| Settings | Public and private health sectors | Taiwan (2 studies) and the USA (2 studies) |
| Outcomes | Prevention, detection, and response related outcomes | Detection of fraudulent claims, amount of anti‐fraud expenditure, occurrence of healthcare fraud and abuse, fraudulent activities in diagnostic laboratories |
| Date of most recent search: December 2010 | ||
| Limitations: This is a well‐conducted systematic review with only minor limitations. | ||
| Authority and accountability for organisations | ||
| Contracting out | ||
| Lagarde 2009 | ||
| Review objective: to assess the effects of contracting out healthcare services in health services utilisation, equity of access, health expenditure and health outcomes. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised trials, controlled before‐after studies and interrupted time series studies of contracting out of healthcare services via a formal contractual relationship between government and non‐state providers | 1 controlled before‐after study, 1 interrupted time series study, and 1 cluster randomised trial |
| Participants | Populations that would potentially access health services (users and non‐users) as well as health facilities in low‐ and middle‐income countries | ‐ Bolivia: a neighbourhood in the capital city of la Paz ‐ Pakistan: the population of the rural district of Rahimyar Khan ‐ Cambodia: 6 districts of the country (2 contracted out and 4 run by the government). It also evaluated a non‐reported number of districts contracted in |
| Settings | Not limited to any level of healthcare delivery | 2 studies (Pakistan, Cambodia) evaluated a contracting out motivated by weaknesses or absence of public system. Both took place in mostly rural areas. 1 study (Bolivia) included a programme based in an urban setting consisting of a network of 8 health centres and 1 hospital |
| Outcomes | Objective measures of health services utilisation, access to care, healthcare expenditure, health outcomes or changes in equity | Health services utilisation and access to care (3 studies), health expenditure (1 study) and health outcomes (1 study). No studies were found that measured changes in equity of access |
| Date of most recent search: April 2006 | ||
| Limitations: This is a well‐conducted systematic review with only minor limitations, but the last search for studies was in 2006. | ||
| Multi‐institutional arrangements | ||
| Koehlmoos 2009 | ||
| Review objective: to assess the effects of the social franchising of health service delivery on access to and the quality of services and health outcomes in low‐ and middle‐income countries. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised trials, non‐randomised trials, interrupted time series studies, and controlled before‐after studies reporting on social franchises delivering health services, driven by seeking social benefits | No studies meeting the inclusion criteria were identified. |
| Participants | All levels of healthcare delivery, all types of patients and healthcare providers | — |
| Settings | Low‐ and middle‐income countries | — |
| Outcomes | Healthcare access, quality of care, health outcomes, adverse effects, equitable access or utilization, cost/service, patient satisfaction | — |
| Date of most recent search: October 2007 to March 2008 | ||
| Limitations: This is a well‐conducted systematic review with only minor limitations, but the last search for studies was done in 2008. | ||
| Authority and accountability for commercial products | ||
| Registration ‐ medicines | ||
| El‐Jardali 2015 | ||
| Review objective: to assess the evidence on the effectiveness of interventions implemented to combat or prevent medicine counterfeiting, particularly in low‐ and middle‐income countries. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised trials; non‐randomised studies (e.g. cohort studies, retrospective studies, cross‐sectional studies, before‐after studies); and non‐comparative studies Any intervention at the health system level to combat or prevent medicine counterfeiting. The review excluded studies that focused on internet/online medicine counterfeiting, analytical techniques and medication errors. Studies that also considered substandard medicines were included only when they did not differentiate between substandard and counterfeit medicines, or where it was unclear if the poor quality medicine was counterfeit or substandard |
Designs: 21 studies with 25 comparisons: cross‐sectional (17 studies); before‐after (5 studies); retrospective (1 study); non‐comparative (1 study); randomised trial (1 study) Interventions: medicine registration (5 comparisons); WHO prequalification of medicines (3 studies); licensing of drug or medicine outlets (8 studies); multi‐faceted interventions (6 studies); deployment of handheld spectrometers at the point of sale (1 study); a public awareness campaign (1 study); an international model of collaboration (1 study) |
| Participants | "Counterfeit/spurious/falsely‐labeled/falsified/medicines", as defined by WHO as medicines with the wrong ingredients, without active ingredients, with insufficient active ingredients or with fake packaging | Most of the studies did not distinguish between counterfeit and substandard medicines |
| Settings | Any setting | Studies from low‐ and middle‐income countries |
| Outcomes | Changes in failure rates of tested medicines; changes in the prevalence of counterfeit medicines; changes in quality of medicine; changes in consumer behaviour; seizures of counterfeit medicines; and closures of illegal outlets/warehouses | Changes in failure rates of medicines (19 comparisons); changes in prevalence of counterfeit medicines (4 studies); changes in purchasing behaviour of consumers (1 study); confiscation of counterfeit medicines (2 studies); closure of illegal outlet(2 studies) Some studies reported more than one outcome. |
| Date of most recent search: April 2014 | ||
| Limitations: This was a well‐conducted systematic review with only minor limitations. However, the included studies used largely observational designs. | ||
| Pricing and purchasing policies | ||
| Acosta 2014 | ||
| Review objective: to determine the effects of pharmaceutical pricing and purchasing policies on medicine use, healthcare utilisation, health outcomes and costs (expenditures). | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised trials, non‐randomised trials, controlled repeated measures studies, interrupted time series studies and controlled before‐after studies of pharmaceutical pricing and purchasing policies | 18 studies were included. Some used more than one design: 14 interrupted time series, 1 interrupted time series/controlled before‐after/controlled repeated measures, 1 controlled repeated measures/repeated measures and 2 controlled before‐after/repeated measures studies. 17 studies evaluated reference pricing, one of which also assessed maximum prices, and 1 study evaluated index pricing. |
| Participants | Healthcare users and providers | In 8 Canadian studies, the patients were Pharmacare beneficiaries in British Columbia: senior citizens aged 65 years and older. The other studies included all beneficiaries of national medicine insurance plans, including vulnerable groups of people from all ages. 1 German and 1 Spanish study did not provide information about the participants. |
| Settings | Large jurisdictions or systems of care. Jurisdictions could be regional, national or international. Studies within organisations, such as health maintenance organisations, were included if the organisation was multi‐sited and served a large population. | Canada (8 studies), USA (2 studies), Spain (2 studies), Germany (2 studies), Norway (2 studies), Australia (1 study) and Sweden (1 study) |
| Outcomes | Medicine use, healthcare utilisation, health outcomes, costs (expenditures), including medicine costs and prices, other healthcare costs and administration costs | Medicine use (10 studies), third party (insurance) medicine expenditures (9 studies), medicine prices (4 studies), medicine expenditures savings (5 studies), and patient costs |
| Date of most recent search: December 2012 | ||
| Limitations: This is well‐conducted systematic review with only minor limitations. | ||
| Marketing regulations | ||
| Gilbody 2005 | ||
| Review objective: to examine the benefits and harms of direct‐to‐consumer advertising of prescription‐only medicines | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised trials, controlled clinical trials, controlled before‐after studies, interrupted time series studies, and cross‐sectional studies with a control group | 3 interrupted time series studies and 1 comparative cross‐sectional survey were found |
| Participants | Not pre‐specified | Patients and physicians in primary care |
| Settings | Not pre‐specified | USA (2 studies), USA and Canada (1 study), Netherlands (1 study) |
| Outcomes | Health‐seeking behaviours of patients at the point of access to care; requests for prescription‐only medicines; patient‐doctor communication and satisfaction with care; prescribing patterns; costs | Requests for prescription only medicines (4 studies); prescription volume (4 studies); patient‐doctor communication and satisfaction with care (1 study) |
| Date of most recent search: October 2004 | ||
| Limitations: This is a well‐conducted systematic review with only minor limitations | ||
| Authority and accountability for health professionals | ||
| Training and licensing | ||
| Pariyo 2009 | ||
| Review objective: to assess the effect of changes in the pre‐licensure education of health professionals on health‐worker supply. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised trials, controlled before‐after studies and interrupted time series studies of interventions that could increase the capacity of health professional training institutions; reduce the loss of students (and increase the likelihood that students will graduate); or increase the recruitment of students from other countries into health professional training institutions | 2 controlled before‐after studies of minority academic advising programmes consisting of academic, personal, financial and vocational advice, skills building, mentorships, supplementary training and annual evaluations |
| Participants | Health professional students prior to licensure | 2 studies among minority groups and general health professional students |
| Settings | No restrictions | 2 studies from the USA |
| Outcomes | Increased numbers of health workers ultimately available for recruitment into the health workforce, improved population‐to‐health professional ratios | 2 studies of the numbers of health workers ultimately available for recruitment into the health workforce |
| Date of most recent search: February 2008 | ||
| Limitations: This is a well‐conducted systematic review with only minor limitations | ||
| Training and licensing | ||
| Rockers 2013 | ||
| Review objective: to assess the effectiveness of interventions to hire, retain and train district health systems managers in low‐ and middle‐income countries. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised trials, quasi‐randomised trials, controlled before‐after studies, interrupted time series studies Interventions related to hiring, retaining and training managers |
One randomised trial: district managers were hired through private contracts to work within the Ministry of Health system. One controlled before‐after study: 18‐month manager training programme. |
| Participants | District health systems managers in low‐ and middle‐income countries | District health systems managers |
| Settings | Districts in low‐ and middle‐income countries | Cambodia (1 study); Mexico, Colombia, El Salvador (1 study) |
| Outcomes | Health systems: population health outcomes; access; utilization; quality; efficiency; equity. Operational: job‐posting vacancy rates, skills | Health facility staffing and supervision, maternal and child health service use (e.g. immunisation, antenatal care), and population health outcomes (e.g. diarrhea incidence). Managers' competencies |
| Date of most recent search: December 2011 | ||
| Limitations: This is a well‐conducted systematic review with only minor limitations. | ||
| Rectruitment and retention strategies | ||
| Grobler 2015 | ||
| Review objective: to assess the effectiveness of interventions to increase the proportion of healthcare professionals working in rural and other underserved areas | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised trials, non‐randomised trials, controlled before‐after studies and interrupted time series studies of any intervention to increase the recruitment or retention of health professionals in underserved areas | 1 interrupted time series study from Taiwan of the effects of National Health Insurance on the equality of distribution of healthcare professionals |
| Participants | Qualified healthcare professionals of any cadre or specialty | Physicians, doctors of Chinese medicine and dentists |
| Settings | All settings | Taiwan |
| Outcomes | Recruitment of health professionals: the proportion of health professionals who initially choose to work in rural or urban underserved communities as a result of being exposed to the intervention. Retention: the proportion of healthcare professionals who continue to work in rural or urban underserved communities as a consequence of the intervention | Equality of geographic distribution of healthcare professionals measured using the Gini coefficient |
| Date of most recent search: April 2014 | ||
| Limitations: This is a well‐conducted systematic review with only minor limitations. | ||
| Rectruitment and retention strategies | ||
| Rockers 2013 | ||
| See characteristics above under 'Training and licensing' | ||
| Movement of health workers between public and private organisations | ||
| Rutebemberwa 2014 | ||
| Review objective: to assess the effects of financial incentives and movement restriction interventions to manage the movement of health workers between public and private organisations in low‐ and middle‐income countries. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised trials and non‐randomised trials; controlled before‐after studies; controlled interrupted time series and interrupted time series studies without controls | No studies were found eligible for inclusion in the review. 9 surveys, 1 review of government reports, 1 study of speeches in the national assembly, and 1 policy analysis paper were found. |
| Participants | All health professionals | No studies were found eligible for inclusion in the review. |
| Settings | Any public or private sector organisations | No studies were found eligible for inclusion in the review. |
| Outcomes | Change in the numbers or proportion of health workers entering or leaving the public or private sectors; duration of stay in a particular sector |
No studies were found eligible for inclusion in the review. |
| Date of most recent search: November 2012 | ||
| Limitations: This is a well‐conducted systematic review with only minor limitations. | ||
| Emigration and immigration policies | ||
| Peñaloza 2011 | ||
| Review objective: to assess the effects of policy interventions to control the emigration of health professionals from low‐ and middle‐income countries to high‐income countries | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised trials, non‐randomised trials, controlled before‐after studies, or interrupted studies of any interventions in source or recipient countries (or both) as well as international agreements that could have an impact on the outcomes | 1 interrupted time series study of the effects of a modification to USA immigration laws (The American Act of October, 1965, which decreased barriers to emigration from countries outside the Americas to the USA) |
| Participants | Health professional nationals of a low‐ and middle‐income country whose graduate training was in a low‐ and middle‐income country | Nurses |
| Settings | Not restricted | USA and the Philippines |
| Outcomes | Proportion (or other measure of change in number) of health professionals that emigrate from a low‐ and middle‐income country to a high‐income countrie | Annual number of nurses migrating from the Philippines to the USA |
| Date of most recent search: March 2011 | ||
| Limitations: This is a well‐conducted systematic review with only minor limitations. | ||
| Dual practice | ||
| Kiwanuka 2011 | ||
| Review objective: to assess the effects of interventions implemented to manage dual practice. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised trials, non‐randomised trials, controlled before‐after studies, interrupted time series studies | No studies were found eligible for inclusion in the review |
| Participants | All health professionals | No studies were found eligible for inclusion in the review |
| Settings | Not specified | No studies were found eligible for inclusion in the review |
| Outcomes | Increased working hours by health workers in public facilities, reduced patient waiting times, reduced absenteeism, reduction in number of private sector licenses issued, reduction in private earning, reduced job satisfaction | No studies were found eligible for inclusion in the review |
| Date of most recent search: May 2011 | ||
| Limitations: This is a well‐conducted systematic review with minor limitations, but no studies were found that met the inclusion criteria. | ||
| Quality of practice | ||
| Flodgren 2011 | ||
| Review objective: to evaluate the effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour and patient outcomes. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised trials, non‐randomised trials, interrupted time series studies and controlled before‐after studies evaluating the effect of external inspection against external standards on healthcare organisation change, healthcare professional behaviour or patient outcomes | 1 cluster‐randomised trial conducted in South Africa and 1 before‐after study reanalysed as an interrupted time series study, conducted in England. The study in South Africa assessed the effects of external inspection on compliance with hospital accreditation standards. The study conducted in England assessed the effects of the Healthcare Commissions Infection Inspection programme on compliance with standards related to healthcare‐acquired infections. |
| Participants | Hospitals, primary healthcare organisations and other community‐based healthcare organisations containing health professionals | 20 public hospitals in Kwa Zulu province of South Africa, and all acute hospital trusts in England |
| Settings | Any health system | 1 study was conducted in South Africa and 1 in England |
| Outcomes | Measures of healthcare organisational change (e.g. organisational performance, waiting list times, inpatient hospital stay time); measures of healthcare professional behaviour (e.g. referral rate, prescribing rate); measure of patient outcomes (e.g. mortality and condition‐specific measures) | Outcomes assessed in 1 study were related to adherence to standards in: medical records, patient outcomes such as satisfaction and patient education, and outcomes related with health processes. The other study assessed the rate of hospital‐acquired infections. |
| Date of most recent search: May 2011 | ||
| Limitations: This is a well‐conducted systematic review with only minor limitations. | ||
| Stakeholder involvement | ||
| Stakeholder participation in policy and organizational decisions | ||
| Nilsen 2010 | ||
| Review objective: to assess the effects of consumer involvement and to compare different methods of involvement in developing healthcare policy and research, clinical practice guidelines, and patient information material. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised trials of ways to involve consumers and enable them to inform and participate in decisions about healthcare policy and research, clinical practice guidelines or patient information material | 6 randomised trials of involvement compared with no involvement in developing patient information, satisfaction interviews conducted by patients compared with staff, informed consent forms developed by consumers versus investigators, and methods of consulting consumers regarding priorities for improving community health |
| Participants | Healthcare consumers or professionals involved in decisions about healthcare at the population level, or evaluating the effects of consumer involvement | Involvement in research (3 studies), developing patient information (2 studies) and healthcare policy (1 study) |
| Settings | No specific settings | Canada (2 studies), USA (2 studies), Norway (1 study) and the UK (1 study) |
| Outcomes | Participation or response rates of consumers; consumer views elicited; consumer influence on decisions, healthcare outcomes or resource utilisation; consumer or professional satisfaction with the involvement process or resulting products; impact on participating consumers; costs | Levels of patient satisfaction with different health services, self‐reported participant understanding, satisfaction with study participation, adherence to the protocol and refusal to participate; knowledge and anxiety with a specific medical procedure; impact on prioritising health concerns and determinants |
| Date of most recent search: October 2009 | ||
| Limitations: This is a well‐conducted systematic review with only minor limitations | ||
| Community mobilisation | ||
| Prost 2013 | ||
| Review objective: to assess the impact of women's groups practising participatory learning and action cycles on birth outcomes in low‐ and middle‐income countries. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised trials of participatory women's groups in low‐ and middle‐income countries | 7 cluster‐randomised trials of participatory women's groups in low‐ and middle‐income countries |
| Participants | Women's groups in which most of the participants are of reproductive age (15–49 years) | 7 studies that included a total of 111 women's groups and 119,428 births |
| Settings | Low‐ and middle‐income countries | Rural areas in Bangladesh (2 studies), India (2 studies), Malawi (2 studies), and Nepal (1 study) |
| Outcomes | Maternal mortality, neonatal mortality and stillbirths | Maternal mortality (7 studies), neonatal mortality (7 studies), and stillbirths (7 studies) |
| Date of most recent search: October 2012 | ||
| Limitations: This is a well‐conducted systematic review with only minor limitations. | ||
| Community mobilisation | ||
| Heintze 2007 | ||
| Review objective: to assess the effectiveness of community‐based interventions in reducing vector populations for dengue control. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Randomised trials, non‐randomised trials, controlled before‐after studies and interrupted time series studies of community‐based interventions aimed at reducing vector populations for dengue control. | 11 included studies: 2 randomised trials, 6 controlled before‐after studies and 3 interrupted time series studies assessing community‐based dengue control interventions alone (5 studies); combined with chemical larvicides (2 studies); combined with fish and chemical larvicides (2 studies); and combined with larvae‐eating crustaceans (Mesocyclops copepods) (2 studies) Studies used educational materials (7 studies); educational meetings such as workshops (9 studies); and educational outreach visits (8 studies). Studies described the involvement of local opinion leaders (6 studies) and national institutions (5 studies), or the use of mass media (5 studies). |
| Participants | Community people and professionals serving the community. | Household inhabitants (mostly housewives), the elderly, children, health committees, healthcare personnel, government officers, teachers and community organisations |
| Settings | Community | 5 studies took place in the Americas: Honduras (3 studies), Mexico (1 study), and Cuba (1 study). 6 studies were carried out in Asia: Vietnam (2 studies), Thailand (1 study), Taiwan (1 study), French Polynesia (1 study), Fiji Islands (1 study). |
| Outcomes | Incidence of dengue disease or infestation of the community with Aedes mosquitoes | Classical entomological/larval indices such as the House Index (HI), the Container Index (CI) and the Breteau Index (BI) – all measures of larvae infestation in the home or in water containers; seroconversion or incidence of dengue disease |
| Date of most recent search: March 2005 | ||
| Limitations: This is a well‐conducted systematic review with only minor limitations. | ||
| Patient information | ||
| Fung 2008 | ||
| Review objective: to synthesise the evidence for using public disclosure of performance data to improve healthcare quality. | ||
| Types of | What the review authors searched for | What the review authors found |
| Study designs and interventions | Peer‐reviewed articles published between 1986 and 2006. Type of studies not pre‐specified | 2 randomised trials, 2 non‐randomised trials, 1 controlled before‐after study, 9 interrupted time series studies, and 31 other observational studies |
| Participants | Not pre‐specified | Hospitals, patients, and hospital staff (45 studies) |
| Settings | Not pre‐specified | USA (43 studies), United Kingdom (1 study), Canada (1 study) |
| Outcomes | Selection of health plans, hospitals, and individual providers, quality improvement activity, clinical outcomes, unintended consequences | Selection of health plans (8 studies), selection of hospitals (9 studies), selection of individual providers (7 studies), quality improvement activity (11 studies), clinical outcomes (11 studies), unintended consequences (13 studies) |
| Date of most recent search: March 2006 | ||
| Limitations: Only peer‐reviewed, English‐language articles were included. | ||
Appendix 4. Supplementary and additional related reviews
Recruitment and retention strategies
Increasing access to health workers in remote and rural areas through improved retention (WHO 2010)
Public disclosure of performance data
Public release of performance data in changing the behaviour of healthcare consumers, professionals or organisations (Ketelaar 2011) (Supplementary review)
Appendix 5. Reviews awaiting classification
Likely included reviews
Bowman LR, Donegan S, McCall PJ. Is dengue vector control deficient in effectiveness or evidence?: Systematic review and meta‐analysis. PLOS Neglected Tropical Diseases 2016;10(3):e0004551.
Abdel‐Aleem H, El‐Gibaly OMH, El‐Gazzar AFS, Al‐Attar GST. Mobile clinics for women's and children's health. Cochrane Database of Systematic Reviews. 2016;8:CD009677.
Akl EA, El‐Jardali F, Bou Karroum L, El‐Eid J, Brax H, Akik C, et al. Effectiveness of Mechanisms and Models of Coordination between Organizations, Agencies and Bodies Providing or Financing Health Services in Humanitarian Crises: A Systematic Review. PloS one. 2015;10(9):e0137159.
Algie CM, Mahar RK, Wasiak J, Batty L, Gruen RL, Mahar PD. Interventions for reducing wrong‐site surgery and invasive clinical procedures. The Cochrane database of systematic reviews. 2015;3(3):CD009404.
Ambia J, Mandala J. A systematic review of interventions to improve prevention of mother‐to‐child HIV transmission service delivery and promote retention. Journal of the International AIDS Society. 2016;19(1):20309.
Barnard S, Kim C, Park MH, Ngo TD. Doctors or mid‐level providers for abortion. The Cochrane database of systematic reviews. 2015;7(7):CD011242.
Basu S, Andrews J, Kishore S, Panjabi R, Stuckler D. Comparative performance of private and public healthcare systems in low‐ and middle‐income countries: a systematic review. PLoS medicine. 2012;9(6):e1001244.
Blacklock C, Gonçalves Bradley DC, Mickan S, Willcox M, Roberts N, Bergström A, et al. Impact of Contextual Factors on the Effect of Interventions to Improve Health Worker Performance in Sub‐Saharan Africa: Review of Randomised Clinical Trials. PloS one. 2016;11(1):e0145206.
Byrne A, Hodge A, Jimenez‐Soto E, Morgan A. What works? Strategies to increase reproductive, maternal and child health in difficult to access mountainous locations: a systematic literature review. PloS one. 2014;9(2):e87683.
Coast E, Jones E, Lattof SR, Portela A. Effectiveness of interventions to provide culturally appropriate maternity care in increasing uptake of skilled maternity care: a systematic review. Health policy and planning. 2016;31(10):1479‐91.
Cornish F, Priego‐Hernandez J, Campbell C, Mburu G, McLean S. The impact of Community Mobilisation on HIV Prevention in Middle and Low Income Countries: A Systematic Review and Critique. AIDS and behavior. 2014;18(11):2110‐34.
Dawson A, Tran NT, Westley E, Mangiaterra V, Festin M. Improving access to emergency contraception pills through strengthening service delivery and demand generation: a systematic review of current evidence in low and middle‐income countries. PloS one. 2014;9(10):e109315.
de Jongh TE, Gurol‐Urganci I, Allen E, Zhu NJ, Atun R. Integration of antenatal care services with health programmes in low‐ and middle‐income countries: systematic review. Journal of global health. 2016;6(1):010403.
Dyer TA, Brocklehurst P, Glenny AM, Davies L, Tickle M, Issac A, et al. Dental auxiliaries for dental care traditionally provided by dentists. The Cochrane database of systematic reviews. 2014;8(8):CD010076.
Ehiri JE, Gunn JK, Center KE, Li Y, Rouhani M, Ezeanolue EE. Training and deployment of lay refugee/internally displaced persons to provide basic health services in camps: a systematic review. Global health action. 2014;7:23902.
Emdin CA, Chong NJ, Millson PE. Non‐physician clinician provided HIV treatment results in equivalent outcomes as physician‐provided care: a meta‐analysis. Journal of the International AIDS Society. 2013;16(no pagination):18445.
Fernandez Turienzo C, Sandall J, Peacock JL. Models of antenatal care to reduce and prevent preterm birth: a systematic review and meta‐analysis. BMJ open. 2016;6(1):e009044.
Feyissa GT, Lockwood C, Munn Z. The effectiveness of home‐based HIV counseling and testing in reducing stigma and risky sexual behavior among adults and adolescents: a systematic review and meta‐analysis. JBI Database of Systematic Reviews and Implementation Reports. 2015;13(6):318‐72.
Fiander M, McGowan J, Grad R, Pluye P, Hannes K, Labrecque M, et al. Interventions to increase the use of electronic health information by healthcare practitioners to improve clinical practice and patient outcomes. The Cochrane database of systematic reviews. 2015;3(3):CD004749.
Flodgren G, Rachas A, Farmer AJ, Inzitari M, Shepperd S. Interactive telemedicine: effects on professional practice and health care outcomes. The Cochrane database of systematic reviews. 2015;9(9):CD002098.
Gaitonde R, Oxman AD, Okebukola PO, Rada G. Interventions to reduce corruption in the health sector. Cochrane Database of Systematic Reviews. 2016;8:CD008856.
George AS, Branchini C, Portela A. Do Interventions that Promote Awareness of Rights Increase Use of Maternity Care Services? A Systematic Review. PloS one. 2015;10(10):e0138116.
Ghada Abou El S, Therese D, Hatem AM. Planned home versus hospital care for preterm prelabour rupture of the membranes (PPROM) prior to 37 weeks' gestation. Cochrane Database of Systematic Reviews. 2014;4(4):CD008053.
Giedion U, Alfonso EA, Diaz Y. The Impact of Universal Coverage Schemes in the Developing World: A Review of the Existing Evidence. Universal Health Coverage (UNICO) studies series; no. 25. Washington D.C.: The Worldbank. 2013.
Handford CD, Tynan AM, Agha A, Rzeznikiewiz D, Glazier RH. Organization of care for persons with HIV‐infection: a systematic review. AIDS care. 2016:1‐10.
Health Quality Ontario. Interventions to Improve Access to Primary Care for People Who Are Homeless: A Systematic Review. Ontario health technology assessment series. 2016;16(9):1‐50.
Hensen B, Taoka S, Lewis JJ, Weiss HA, Hargreaves J. Systematic review of strategies to increase men's HIV‐testing in sub‐Saharan Africa. AIDS (London, England). 2014;28(14):2133‐45.
Hernández AV, Pasupuleti V, Benites‐Zapata V, Velásquez‐Hurtado E, Loyola‐Romaní J, Rodríguez‐Calviño Y, et al. [Systematic review of the efectiveness of community‐based interventions to decrease neonatal mortality]. Revista peruana de medicina experimental y salud pública. 2015;32(3):532‐45.
Hesselink G, Berben S, Beune T, Schoonhoven L. Improving the governance of patient safety in emergency care: a systematic review of interventions. BMJ open. 2016;6(1):e009837.
Hopkins U, Itty AS, Nazario H, Pinon M, Slyer J, Singleton J. The effectiveness of delegation interventions by the registered nurse to the unlicensed assistive personnel and their impact on quality of care, patient satisfaction, and RN staff satisfaction: a systematic review. JBI Library of Systematic Reviews. 2012;10(15):895‐934.
Hoyler M, Hagander L, Gillies R, Riviello R, Chu K, Bergström S, et al. Surgical care by non‐surgeons in low‐income and middle‐income countries: a systematic review. Lancet (London, England). 2015;385 Suppl 2:S42.
Joshi R, Alim M, Kengne AP, Jan S, Maulik PK, Peiris D, et al. Task shifting for non‐communicable disease management in low and middle income countries‐‐a systematic review. PloS one. 2014;9(8):e103754.
Kien C, Reichenpfader U, Nußbaumer B, Rohleder S, Punz P, Christof C, et al. [Comparative effectiveness and safety of screening and counselling interventions conducted by non‐physicians and physicians: A systematic review]. Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen. 2015;109(1):18‐27.
Kilpatrick K, Kaasalainen S, Donald F, Reid K, Carter N, Bryant‐Lukosius D, et al. The effectiveness and cost‐effectiveness of clinical nurse specialists in outpatient roles: a systematic review. Journal of evaluation in clinical practice. 2014;20(6):1106‐23.
Kim K, Choi JS, Choi E, Nieman CL, Joo JH, Lin FR, et al. Effects of Community‐Based Health Worker Interventions to Improve Chronic Disease Management and Care Among Vulnerable Populations: A Systematic Review. American journal of public health. 2016;106(4):e1‐e26.
Kredo T, Adeniyi FGB, Bateganya M, Pienaar ED. Task shifting from doctors to non‐doctors for initiation and maintenance of antiretroviral therapy. Cochrane Database of Systematic Reviews. 2014;7(7):CD007331.
Lassi ZS, Musavi NB, Maliqi B, Mansoor N, de Francisco A, Toure K, et al. Systematic review on human resources for health interventions to improve maternal health outcomes: evidence from low‐ and middle‐income countries. Human resources for health. 2016;14(1):10.
Lavender T, Richens Y, Milan SJ, Smyth RMD, Dowswell T. Telephone support for women during pregnancy and the first six weeks postpartum. Cochrane Database of Systematic Reviews. 2013;7(7):CD009338.
Lawrence D, Fedorowicz Z, van Zuuren EJ. Day care versus in‐patient surgery for age‐related cataract. The Cochrane database of systematic reviews. 2015;11(11):CD004242.
Liu G, Jack H, Piette A, Mangezi W, Machando D, Rwafa C, et al. Mental health training for health workers in Africa: a systematic review. The lancet Psychiatry. 2016;3(1):65‐76.
MacPherson P, Munthali C, Ferguson J, Armstrong A, Kranzer K, Ferrand RA, et al. Service delivery interventions to improve adolescents' linkage, retention and adherence to antiretroviral therapy and HIV care. Tropical medicine & international health : TM & IH. 2015;20(8):1015‐32.
Mbuagbaw L, Medley N, Darzi AJ, Richardson M, Habiba Garga K, Ongolo‐Zogo P. Health system and community level interventions for improving antenatal care coverage and health outcomes. The Cochrane database of systematic reviews. 2015;12(12):CD010994.
Mdege ND, Chindove S. Bringing antiretroviral therapy (ART) closer to the end‐user through mobile clinics and home‐based ART: systematic review shows more evidence on the effectiveness and cost effectiveness is needed. The International journal of health planning and management. 2013;29(1):e31‐47.
Moraros J, Lemstra M, Nwankwo C. Lean interventions in healthcare: do they actually work? A systematic literature review. International journal for quality in health care : journal of the International Society for Quality in Health Care / ISQua. 2016;28(2):150‐65.
Nunan M, Duke T. Effectiveness of pharmacy interventions in improving availability of essential medicines at the primary healthcare level. Tropical medicine & international health : TM & IH. 2011;16(5):647‐58.
Oluoch T, Santas X, Kwaro D, Were M, Biondich P, Bailey C, et al. The effect of electronic medical record‐based clinical decision support on HIV care in resource‐constrained settings: a systematic review. International journal of medical informatics. 2012;81(10):e83‐92.
Palmer KS, Agoritsas T, Martin D, Scott T, Mulla SM, Miller AP, et al. Activity‐based funding of hospitals and its impact on mortality, readmission, discharge destination, severity of illness, and volume of care: a systematic review and meta‐analysis. PloS one. 2014;9(10):e109975.
Pega F, Liu SY, Walter S, Lhachimi SK. Unconditional cash transfers for assistance in humanitarian disasters: effect on use of health services and health outcomes in low‐ and middle‐income countries. The Cochrane database of systematic reviews. 2015;9(9):CD011247.
Penazzato M, Davies MA, Apollo T, Negussie E, Ford N. Task shifting for the delivery of pediatric antiretroviral treatment: a systematic review. Journal of acquired immune deficiency syndromes (1999). 2014;65(4):414‐22.
Pollaris G, Sabbe M. Reverse triage: more than just another method. European journal of emergency medicine : official journal of the European Society for Emergency Medicine. 2015;23(4):240‐7.
Polus S, Lewin S, Glenton C, Lerberg PM, Rehfuess E, Gülmezoglu AM. Optimizing the delivery of contraceptives in low‐ and middle‐income countries through task shifting: a systematic review of effectiveness and safety. Reproductive health. 2015;12(1):27.
Rashidian A, Omidvari AH, Vali Y, Sturm H, Oxman AD. Pharmaceutical policies: effects of financial incentives for prescribers. The Cochrane database of systematic reviews. 2015;8(8):CD006731.
Reichow B, Servili C, Yasamy MT, Barbui C, Saxena S. Non‐specialist psychosocial interventions for children and adolescents with intellectual disability or lower‐functioning autism spectrum disorders: a systematic review. PLoS medicine. 2013;10(12):e1001572.
Reisman J, Arlington L, Jensen L, Louis H, Suarez‐Rebling D, Nelson BD. Newborn Resuscitation Training in Resource‐Limited Settings: A Systematic Literature Review. Pediatrics. 2016;138(2):1‐16.
Robyn PJ, Sauerborn R, Bärnighausen T. Provider payment in community‐based health insurance schemes in developing countries: a systematic review. Health policy and planning. 2013;28(2):111‐22.
Salam RA, Das JK, Lassi ZS, Bhutta ZA. Impact of community‐based interventions for the prevention and control of malaria on intervention coverage and health outcomes for the prevention and control of malaria. Infectious diseases of poverty. 2014;3(1):25.
Schmidt E, Goldhaber‐Fiebert SN, Ho LA, McDonald KM. Simulation exercises as a patient safety strategy: a systematic review. Annals of internal medicine. 2013;158(5 Pt 2):426‐32.
Sharon RL, Amanda N, Andrew FS, Phil A. Physician anaesthetists versus non‐physician providers of anaesthesia for surgical patients. Cochrane Database of Systematic Reviews. 2014;7(7):CD010357.
Sondaal SF, Browne JL, Amoakoh‐Coleman M, Borgstein A, Miltenburg AS, Verwijs M, et al. Assessing the Effect of mHealth Interventions in Improving Maternal and Neonatal Care in Low‐ and Middle‐Income Countries: A Systematic Review. PloS one. 2016;11(5):e0154664.
Spaan E, Mathijssen J, Tromp N, McBain F, ten Have A, Baltussen R. The impact of health insurance in Africa and Asia: a systematic review. Bulletin of the World Health Organization. 2012;90(9):685‐92.
Sunguya BF, Poudel KC, Mlunde LB, Urassa DP, Yasuoka J, Jimba M. Nutrition training improves health workers' nutrition knowledge and competence to manage child undernutrition: a systematic review. Frontiers in public health. 2013;1:37.
Susan FM, Benjamin MH, Ramila B, Tim E, Debra B. Demand‐side financing measures to increase maternal health service utilisation and improve health outcomes: a systematic review of evidence from low‐ and middle‐income countries. JBI Library of Systematic Reviews. 2012;10(58):4165‐567.
Thomas SM, Jeyaraman M, Hodge WG, Hutnik C, Costella J, Malvankar‐Mehta MS. The effectiveness of teleglaucoma versus in‐patient examination for glaucoma screening: a systematic review and meta‐analysis. PloS one. 2014;9(12):e113779.
Tibingana‐Ahimbisibwe B, Katabira C, Mpalampa L, Harrison RA. The effectiveness of adolescent‐specific prenatal interventions on improving attendance and reducing harm during and after birth: a systematic review. International journal of adolescent medicine and health. 2016.
Till SR, Everetts D, Haas DM. Incentives for increasing prenatal care use by women in order to improve maternal and neonatal outcomes. Cochrane Database of Systematic Reviews. 2015;12(12):CD009916.
Watterson JL, Walsh J, Madeka I. Using mHealth to Improve Usage of Antenatal Care, Postnatal Care, and Immunization: A Systematic Review of the Literature. BioMed research international. 2015;2015(no pagination):153402.
Weeks G, George J, Maclure K, Stewart D. Non‐medical prescribing versus medical prescribing for acute and chronic disease management in primary and secondary care. The Cochrane database of systematic reviews. 2016;11:CD011227.
Wiysonge CS, Abdullahi LH, Ndze VN, Hussey GD. Public stewardship of private for‐profit healthcare providers in low‐ and middle‐income countries. Cochrane Database of Systematic Reviews. 2016;8(8):CD009855.
Wong WC, Luk CW, Kidd MR. Is there a role for primary care clinicians in providing shared care in HIV treatment? A systematic literature review. Sexually transmitted infections. 2012;88(2):125‐31.
Likely excluded reviews
Bassili A, Fitzpatrick C, Qadeer E, Fatima R, Floyd K, Jaramillo E. A systematic review of the effectiveness of hospital‐ and ambulatory‐based management of multidrug‐resistant tuberculosis. The American journal of tropical medicine and hygiene. 2013;89(2):271‐80.
Bhageerathy R, Nair S, Bhaskaran U. A systematic review of community‐based health insurance programs in South Asia. The International journal of health planning and management. 2016.
Blaya JA, Fraser HS, Holt B. E‐health technologies show promise in developing countries. Health affairs (Project Hope). 2010;29(2):244‐51.
Callese TE, Richards CT, Shaw P, Schuetz SJ, Paladino L, Issa N, et al. Trauma system development in low‐ and middle‐income countries: a review. The Journal of surgical research. 2015;193(1):300‐7.
Davy C, Bleasel J, Liu H, Tchan M, Ponniah S, Brown A. Effectiveness of chronic care models: opportunities for improving healthcare practice and health outcomes: a systematic review. BMC health services research. 2015;15(1):194.
Dawson AZ, Walker RJ, Campbell JA, Egede LE. Effective Strategies for Global Health Training Programs A Systematic Review of Training Outcomes in Low and Middle Income Countries. Global journal of health science. 2016;8(11):56719.
Higgs ES, Goldberg AB, Labrique AB, Cook SH, Schmid C, Cole CF, et al. Understanding the role of mHealth and other media interventions for behavior change to enhance child survival and development in low‐ and middle‐income countries: an evidence review. Journal of health communication. 2014;19 Suppl 1:164‐89.
Hubert GJ, Müller‐Barna P, Audebert HJ. Recent advances in TeleStroke: a systematic review on applications in prehospital management and Stroke Unit treatment or TeleStroke networking in developing countries. International journal of stroke : official journal of the International Stroke Society. 2014;9(8):968‐73.
Margaret Elizabeth K, Denis P, Peter CR, Wim Van L. The contribution of primary care to health and health systems in low‐ and middle‐income countries: A critical review of major primary care initiatives. 2010.
Pannick S, Davis R, Ashrafian H, Byrne BE, Beveridge I, Athanasiou T, et al. Effects of Interdisciplinary Team Care Interventions on General Medical Wards: A Systematic Review. JAMA internal medicine. 2015;175(8):1288‐98.
Schiavo R, May Leung M, Brown M. Communicating risk and promoting disease mitigation measures in epidemics and emerging disease settings. Pathogens and global health. 2014;108(2):76‐94.
Zulfiqar AB, Zohra SL, Nadia M. Systematic review on human resources for health interventions to improve maternal health outcomes: Evidence from developing countries. 2010.
Uncertain reviews
Acheampong F, Anto BP, Koffuor GA. Medication safety strategies in hospitals‐‐a systematic review. The International journal of risk & safety in medicine. 2014;26(3):117‐31.
Alkhenizan A, Shaw C. Impact of accreditation on the quality of healthcare services: a systematic review of the literature. Annals of Saudi medicine. 2011;31(4):407‐16.
Al‐Mallah MH, Farah I, Al‐Madani W, Bdeir B, Al Habib S, Bigelow ML, et al. The Impact of Nurse‐Led Clinics on the Mortality and Morbidity of Patients with Cardiovascular Diseases: A Systematic Review and Meta‐analysis. The Journal of cardiovascular nursing. 2015;31(1):89‐95.
Bakitas MA, Elk R, Astin M, Ceronsky L, Clifford KN, Dionne‐Odom JN, et al. Systematic Review of Palliative Care in the Rural Setting. Cancer control : journal of the Moffitt Cancer Center. 2015;22(4):450‐64.
Balfour J, Abdulcadir J, Say L, Hindin MJ. Interventions for healthcare providers to improve treatment and prevention of female genital mutilation: a systematic review. BMC health services research. 2016;16(1):409.
Bannan DF, Tully MP. Bundle interventions used to reduce prescribing and administration errors in hospitalized children: a systematic review. Journal of clinical pharmacy and therapeutics. 2016;41(3):246‐55.
Bashshur RL, Howell JD, Krupinski EA, Harms KM, Bashshur N, Doarn CR. The Empirical Foundations of Telemedicine Interventions in Primary Care. Telemedicine journal and e‐health : the official journal of the American Telemedicine Association. 2016;22(5):342‐75.
Boccia D, Hargreaves J, Lönnroth K, Jaramillo E, Weiss J, Uplekar M, et al. Cash transfer and microfinance interventions for tuberculosis control: review of the impact evidence and policy implications. The international journal of tuberculosis and lung disease : the official journal of the International Union against Tuberculosis and Lung Disease. 2011;15 Suppl 2:S37‐49.
Booth A, Cantrell A, Preston L, Chambers D, Goyder E. What is the evidence for the effectiveness, appropriateness and feasibility of group clinics for patients with chronic conditions? A systematic review2015 2015/12/None.
Brocklehurst P, Mertz B, Jerković‐Ćosić K, Littlewood A, Tickle M. Direct access to midlevel dental providers: an evidence synthesis. Journal of public health dentistry. 2014;74(4):326‐35.
Candy B, France R, Low J, Sampson L. Does involving volunteers in the provision of palliative care make a difference to patient and family wellbeing? A systematic review of quantitative and qualitative evidence. International journal of nursing studies. 2014;52(3):756‐68.
Chapman SM, Wray J, Oulton K, Peters MJ. Systematic review of paediatric track and trigger systems for hospitalised children. Resuscitation. 2016;109:87‐109.
Coxeter P, Del Mar CB, McGregor L, Beller EM, Hoffmann TC. Interventions to facilitate shared decision making to address antibiotic use for acute respiratory infections in primary care. The Cochrane database of systematic reviews. 2015;11(11):CD010907.
Damiani G, Pinnarelli L, Sommella L, Vena V, Magrini P, Ricciardi W. The Short Stay Unit as a new option for hospitals: a review of the scientific literature. Medical science monitor : international medical journal of experimental and clinical research. 2011;17(6):SR15‐9.
Gentry S, van Velthoven MHMMT, Tudor Car L, Car J. Telephone delivered interventions for reducing morbidity and mortality in people with HIV infection. Cochrane Database of Systematic Reviews. 2013;5(5):CD009189.
Harding R, Albertyn R, Sherr L, Gwyther L. Pediatric palliative care in sub‐saharan Africa: a systematic review of the evidence for care models, interventions, and outcomes. Journal of pain and symptom management. 2014;47(3):642‐51.
Hastings SE, Armitage GD, Mallinson S, Jackson K, Suter E. Exploring the relationship between governance mechanisms in healthcare and health workforce outcomes: a systematic review. BMC health services research. 2014;14(1):479.
Hines S, Munday J, Kynoch K. Effectiveness of nurse‐led preoperative assessment services for elective surgery: a systematic review update. JBI database of systematic reviews and implementation reports. 2015;13(6):279‐317.
Hotchkiss DR, Diana ML, Foreit KG. How can routine health information systems improve health systems functioning in low‐ and middle‐income countries? Assessing the evidence base. Advances in health care management. 2012;12:25‐58.
Housden L, Wong ST, Dawes M. Effectiveness of group medical visits for improving diabetes care: a systematic review and meta‐analysis. CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne. 2013;185(13):E635‐44.
Ireland S, Kent B. Telephone pre‐operative assessment for adults: a comprehensive systematic review. JBI Library of Systematic Reviews. 2012;10(25):1452‐503.
Kågesten A, Parekh J, Tunçalp O, Turke S, Blum RW. Comprehensive adolescent health programs that include sexual and reproductive health services: a systematic review. American journal of public health. 2014;104(12):e1‐e14.
Lazarus JV, Safreed‐Harmon K, Nicholson J, Jaffar S. Health service delivery models for the provision of antiretroviral therapy in sub‐Saharan Africa: a systematic review. Tropical medicine & international health : TM & IH. 2014;19(10):1198‐215.
Leidy Johanna Rueda D, Diná Lopes Monteiro da C. The efficacy of telephone use to assist and improve the wellbeing of family caregivers of persons with chronic diseases: a systematic review. JBI Library of Systematic Reviews. 2015;12(12):106‐40.
McCormack L, Sheridan S, Lewis M, Boudewyns V, Melvin CL, Kistler C, et al. Communication and dissemination strategies to facilitate the use of health‐related evidence. Evidence report/technology assessment. 2013(213):1‐520.
Meid AD, Lampert A, Burnett A, Seidling HM, Haefeli WE. The impact of pharmaceutical care interventions for medication underuse in older people: a systematic review and meta‐analysis. British journal of clinical pharmacology. 2015;80(4):768‐76.
Mengistu TA, Tafere TE. Effect of antenatal care on institutional delivery in developing countries: a systematic review. JBI Library of Systematic Reviews. 2011;9(35):1447‐70.
Mitchell GK, Burridge L, Zhang J, Donald M, Scott IA, Dart J, et al. Systematic review of integrated models of health care delivered at the primary?secondary interface: how effective is it and what determines effectiveness? Australian journal of primary health. 2015;21(4):391‐408.
Palmas W, March D, Darakjy S, Findley SE, Teresi J, Carrasquillo O, et al. Community Health Worker Interventions to Improve Glycemic Control in People with Diabetes: A Systematic Review and Meta‐Analysis. Journal of General Internal Medicine. 2015;30:1004‐12.
Pérez‐Escamilla R, Martinez JL, Segura‐Pérez S. Impact of the Baby‐friendly Hospital Initiative on breastfeeding and child health outcomes: a systematic review. Maternal & child nutrition. 2016;12(3):402‐17.
Rinke ML, Bundy DG, Velasquez CA, Rao S, Zerhouni Y, Lobner K, et al. Interventions to Reduce Pediatric Medication Errors: A Systematic Review. Pediatrics. 2014;134(2):338‐60.
Rudge MV, Lima SA, El Dib RP, Marini G, Magalhães C, Calderon Ide M. Effect of ambulatory versus hospital treatment for gestational diabetes or hyperglycemia on infant mortality rates: a systematic review. São Paulo medical journal = Revista paulista de medicina. 2013;131(5):331‐7.
Sabater‐Hernández D, Sabater‐Galindo M, Fernandez‐Llimos F, Rotta I, Hossain LN, Durks D, et al. A Systematic Review of Evidence‐Based Community Pharmacy Services Aimed at the Prevention of Cardiovascular Disease. Journal of managed care & specialty pharmacy. 2016;22(6):699‐713.
Salmoiraghi A, Hussain S. A Systematic Review of the Use of Telepsychiatry in Acute Settings. Journal of psychiatric practice. 2015;21(5):389‐93.
Santos MT, Moura SC, Gomes LM, Lima AH, Moreira RS, Silva CD, et al. Telehealth application on the rehabilitation of children and adolescents. Revista paulista de pediatria : orgão oficial da Sociedade de Pediatria de São Paulo. 2014;32(1):136‐43.
Saxon RL, Gray MA, Oprescu FI. Extended roles for allied health professionals: an updated systematic review of the evidence. Journal of multidisciplinary healthcare. 2014;7((Saxon R.L., robyn.saxon@health.qld.gov.au; Gray M.A.; Ioprescu F.) School of Health and Sports Sciences, University of the Sunshine Coast, Sippy Downs, Australia):479‐88.
Stokes J, Panagioti M, Alam R, Checkland K, Cheraghi‐Sohi S, Bower P. Effectiveness of Case Management for 'At Risk' Patients in Primary Care: A Systematic Review and Meta‐Analysis. PloS one. 2015;10(7):e0132340.
Suksomboon N, Poolsup N, Nge YL. Impact of phone call intervention on glycemic control in diabetes patients: a systematic review and meta‐analysis of randomized, controlled trials. PloS one. 2014;9(2):e89207.
Tao D, Xie L, Wang T, Wang T. A meta‐analysis of the use of electronic reminders for patient adherence to medication in chronic disease care. Journal of Telemedicine and Telecare. 2015;21(1).
Tricco AC, Antony J, Ivers NM, Ashoor HM, Khan PA, Blondal E, et al. Effectiveness of quality improvement strategies for coordination of care to reduce use of health care services: a systematic review and meta‐analysis. CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne. 2014;186(15):E568‐78.
Pitt V, Lowe D, Hill S, Prictor M, Hetrick SE, Ryan R, et al. Consumer‐providers of care for adult clients of statutory mental health services. Cochrane Database of Systematic Reviews. 2013;3(3):CD004807.
Weaver MS, Lönnroth K, Howard SC, Roter DL, Lam CG. Interventions to improve adherence to treatment for paediatric tuberculosis in low‐ and middle‐income countries: a systematic review and meta‐analysis. Bulletin of the World Health Organization. 2015;93(10):700‐11B.
Wekesah FM, Mbada CE, Muula AS, Kabiru CW, Muthuri SK, Izugbara CO. Effective non‐drug interventions for improving outcomes and quality of maternal health care in sub‐Saharan Africa: a systematic review. Systematic reviews. 2016;5(1):137.
Willey B, Smith Paintain L, Mangham‐Jefferies L, Car J, Armstrong Schellenberg J. Effectiveness of interventions to strengthen national health service delivery on coverage, access, quality and equity in the use of health services in low and lower middle income countries. 2013 2013.
World Health Organization, University of California SF. Task shifting ‐ physicians (doctors) versus non‐physicians (nurses or clinical officers) for initiation and maintenance of antiretroviral therapy. World Health Organization. 2013.
Yasmin F, Banu B, Zakir SM, Sauerborn R, Ali L, Souares A. Positive influence of short message service and voice call interventions on adherence and health outcomes in case of chronic disease care: a systematic review. BMC medical informatics and decision making. 2016;16:46.
Zhai YK, Zhu WJ, Hou HL, Sun DX, Zhao J. Efficacy of telemedicine for thrombolytic therapy in acute ischemic stroke: a meta‐analysis. Journal of telemedicine and telecare. 2015;21(3):123‐30.
Zhou K, Fitzpatrick T, Walsh N, Kim JY, Chou R, Lackey M, et al. Interventions to optimise the care continuum for chronic viral hepatitis: a systematic review and meta‐analyses. The Lancet Infectious diseases. 2016.
Zwanikken PA, Dieleman M, Samaranayake D, Akwataghibe N, Scherpbier A. A systematic review of outcome and impact of master's in health and health care. BMC medical education. 2013;13:18.
Likely supplemental reviews
Abdulwahid MA, Booth A, Kuczawski M, Mason SM. The impact of senior doctor assessment at triage on emergency department performance measures: systematic review and meta‐analysis of comparative studies. Emergency medicine journal : EMJ. 2015;33(7):504‐13.
Adebayo EF, Uthman OA, Wiysonge CS, Stern EA, Lamont KT, Ataguba JE. A systematic review of factors that affect uptake of community‐based health insurance in low‐income and middle‐income countries. BMC health services research. 2015;15(1):543.
Alghamdi M, Gashgari H, Househ M. A Systematic Review of Mobile Health Technology Use in Developing Countries. Studies in health technology and informatics. 2015;213:223‐6.
Alkhaled L, Kahale L, Nass H, Brax H, Fadlallah R, Badr K, et al. Legislative, educational, policy and other interventions targeting physicians' interaction with pharmaceutical companies: a systematic review. BMJ open. 2014;4(7):e004880.
Altowaijri A, Phillips CJ, Fitzsimmons D. A systematic review of the clinical and economic effectiveness of clinical pharmacist intervention in secondary prevention of cardiovascular disease. Journal of managed care pharmacy : JMCP. 2013;19(5):408‐16.
Amouzou A, Morris S, Moulton LH, Mukanga D. Assessing the impact of integrated community case management (iCCM) programs on child mortality: Review of early results and lessons learned in sub‐Saharan Africa. Journal of global health. 2014;4(2):020411.
Aziz H, Hatah E, Makmor Bakry M, Islahudin F. How payment scheme affects patients' adherence to medications? A systematic review. Patient preference and adherence. 2016;10:837‐50.
Bailey C, Blake C, Schriver M, Cubaka VK, Thomas T, Martin Hilber A. A systematic review of supportive supervision as a strategy to improve primary healthcare services in Sub‐Saharan Africa. International journal of gynaecology and obstetrics: the official organ of the International Federation of Gynaecology and Obstetrics. 2015;132(1):117‐25.
Baxter PE, Hewko SJ, Pfaff KA, Cleghorn L, Cunningham BJ, Elston D, et al. Leaders' experiences and perceptions implementing activity‐based funding and pay‐for‐performance hospital funding models: A systematic review. Health policy (Amsterdam, Netherlands). 2015;119(8):1096‐110.
Bbosa GS, Wong G, Kyegombe DB, Ogwal‐Okeng J. Effects of intervention measures on irrational antibiotics/antibacterial drug use in developing countries: A systematic review. Health. 2014;6.
Bellows B, Bulaya C, Inambwae S, Lissner CL, Ali M, Bajracharya A. Family Planning Vouchers in Low and Middle Income Countries: A Systematic Review. Studies in family planning. 2016;47(4):357‐70.
Bellows BW, Conlon CM, Higgs ES, Townsend JW, Nahed MG, Cavanaugh K, et al. A taxonomy and results from a comprehensive review of 28 maternal health voucher programmes. Journal of health, population, and nutrition. 2013;31(4 Suppl 2):106‐28.
Benishek LA, Dugosh KL, Kirby KC, Matejkowski J, Clements NT, Seymour BL, et al. Prize‐based contingency management for the treatment of substance abusers: a meta‐analysis. Addiction (Abingdon, England). 2014;109(9):1426‐36.
Beratarrechea A, Lee AG, Willner JM, Jahangir E, Ciapponi A, Rubinstein A. The impact of mobile health interventions on chronic disease outcomes in developing countries: a systematic review. Telemedicine journal and e‐health : the official journal of the American Telemedicine Association. 2014;20(1):75‐82.
Blank L, Baxter S, Woods HB, Goyder E, Lee A, Payne N, et al. What is the evidence on interventions to manage referral from primary to specialist non‐emergency care? A systematic review and logic model synthesis. Health services and delivery research. 2015.
Bloomfield GS, Vedanthan R, Vasudevan L, Kithei A, Were M, Velazquez EJ. Mobile health for non‐communicable diseases in Sub‐Saharan Africa: a systematic review of the literature and strategic framework for research. Globalization and health. 2014;10(1):49.
Boksmati N, Butler‐Henderson K, Anderson K, Sahama T. The Effectiveness of SMS Reminders on Appointment Attendance: a Meta‐Analysis. Journal of medical systems. 2016;40(4):90.
Borchard A, Schwappach DL, Barbir A, Bezzola P. A systematic review of the effectiveness, compliance, and critical factors for implementation of safety checklists in surgery. Annals of surgery. 2012;256(6):925‐33.
Braet A, Weltens C, Sermeus W. Effectiveness of discharge interventions from hospital to home on hospital readmissions: a systematic review. JBI database of systematic reviews and implementation reports. 2016;14(2):106‐73.
Brata C, Gudka S, Schneider CR, Clifford RM. A review of the provision of appropriate advice by pharmacy staff for self‐medication in developing countries. Research in social & administrative pharmacy: RSAP. 2014;11(2):136‐53.
Byrne A, Morgan A. How the integration of traditional birth attendants with formal health systems can increase skilled birth attendance. International journal of gynaecology and obstetrics: the official organ of the International Federation of Gynaecology and Obstetrics. 2011;115(2):127‐34.
Campanella P, Vukovic V, Parente P, Sulejmani A, Ricciardi W, Specchia ML. The impact of Public Reporting on clinical outcomes: a systematic review and meta‐analysis. BMC health services research. 2016;16(1):296.
Carter EB, Temming LA, Akin J, Fowler S, Macones GA, Colditz GA, et al. Group Prenatal Care Compared With Traditional Prenatal Care: A Systematic Review and Meta‐analysis. Obstetrics and gynecology. 2016;128(3):551‐61.
Chhina HK, Bhole VM, Goldsmith C, Hall W, Kaczorowski J, Lacaille D. Effectiveness of academic detailing to optimize medication prescribing behaviour of family physicians. Journal of pharmacy & pharmaceutical sciences : a publication of the Canadian Society for Pharmaceutical Sciences, Société canadienne des sciences pharmaceutiques. 2013;16(4):511‐29.
Chin WY, Lam CL, Lo SV. Quality of care of nurse‐led and allied health personnel‐led primary care clinics. Hong Kong medical journal = Xianggang yi xue za zhi / Hong Kong Academy of Medicine. 2011;17(3):217‐30.
Chishinga N, Godfrey‐Faussett P, Fielding K, Ayles H. Effect of home‐based interventions on virologic outcomes in adults receiving antiretroviral therapy in Africa: a meta‐analysis. BMC public health. 2014;14(1):239.
Clark CE, Smith LF, Taylor RS, Campbell JL. Nurse led interventions to improve control of blood pressure in people with hypertension: systematic review and meta‐analysis. BMJ (Clinical research ed). 2010;341(7771):c3995.
Cobos Muñoz D, Merino Amador P, Monzon Llamas L, Martinez Hernandez D, Santos Sancho JM. Decentralization of health systems in low and middle income countries: a systematic review. International journal of public health. 2016.
Conn VS, Ruppar TM, Enriquez M, Cooper PS, Chan KC. Healthcare provider targeted interventions to improve medication adherence: systematic review and meta‐analysis. International journal of clinical practice. 2015;69(8):889‐99.
Davis R, Parand A, Pinto A, Buetow S. Systematic review of the effectiveness of strategies to encourage patients to remind healthcare professionals about their hand hygiene. The Journal of hospital infection. 2014;89(3):141‐62.
Davy C, Bleasel J, Liu H, Tchan M, Ponniah S, Brown A. Factors influencing the implementation of chronic care models: A systematic literature review. BMC family practice. 2015;16:102.
Decroo T, Rasschaert F, Telfer B, Remartinez D, Laga M, Ford N. Community‐based antiretroviral therapy programs can overcome barriers to retention of patients and decongest health services in sub‐Saharan Africa: a systematic review. International health. 2013;5(3):169‐79.
Dempsey E, Pammi M, Ryan AC, Barrington KJ. Standardised formal resuscitation training programmes for reducing mortality and morbidity in newborn infants. The Cochrane database of systematic reviews. 2015;9(9):CD009106.
Devi BR, Syed‐Abdul S, Kumar A, Iqbal U, Nguyen PA, Li YC, et al. mHealth: An updated systematic review with a focus on HIV/AIDS and tuberculosis long term management using mobile phones. Computer methods and programs in biomedicine. 2015;122(2):257‐65.
do Amaral JJF, Victora CG. The effect of training in Integrated Management of Childhood Illness (IMCI) on the performance and healthcare quality of pediatric healthcare workers: a systematic review. Revista Brasileira de Saúde Materno Infantil. 2008;8(2):151‐62.
Druetz T, Siekmans K, Goossens S, Ridde V, Haddad S. The community case management of pneumonia in Africa: a review of the evidence. Health policy and planning. 2013;30(2):253‐66.
Dzakpasu S, Powell‐Jackson T, Campbell OM. Impact of user fees on maternal health service utilization and related health outcomes: a systematic review. Health policy and planning. 2014;29(2):137‐50.
Eichler R, Agarwal K, Askew I, Iriarte E, Morgan L, Watson J. Performance‐based incentives to improve health status of mothers and newborns: what does the evidence show? Journal of health, population, and nutrition. 2013;31(4 Suppl 2):36‐47.
Elder E, Johnston AN, Crilly J. Review article: Systematic review of three key strategies designed to improve patient flow through the emergency department. Emergency medicine Australasia : EMA. 2015;27(5):394‐404.
Evans BA, Porter A, Gammon B, Mayes RH, Poulden M, Rees N, et al. A systematic review of rapid access models of care and their effects on delays in emergency departments. Emergency medicine journal : EMJ. 2015;32(6):e15‐6.
Free C, Phillips G, Galli L, Watson L, Felix L, Edwards P, et al. The effectiveness of mobile‐health technology‐based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS medicine. 2013;10(1):e1001362.
Gielen SC, Dekker J, Francke AL, Mistiaen P, Kroezen M. The effects of nurse prescribing: A systematic review. International journal of nursing studies. 2013;51(7):1048‐61.
Gillespie BM, Chaboyer W, Thalib L, John M, Fairweather N, Slater K. Effect of Using a Safety Checklist on Patient Complications after Surgery: A Systematic Review and Meta‐analysis. Anesthesiology. 2014;120(6):1380‐9.
Gilmore B, McAuliffe E. Effectiveness of community health workers delivering preventive interventions for maternal and child health in low‐ and middle‐income countries: a systematic review. BMC public health. 2013;13(1):847.
Gogia S, Sachdev HP. Home‐based neonatal care by community health workers for preventing mortality in neonates in low‐ and middle‐income countries: a systematic review. Journal of perinatology : official journal of the California Perinatal Association. 2016;36 Suppl 1(S1):S55‐73.
Gogia S, Sachdev HS. Home visits by community health workers to prevent neonatal deaths in developing countries: a systematic review. Bulletin of the World Health Organization. 2010;88(9):658‐66B.
Govindasamy D, Meghij J, Kebede Negussi E, Clare Baggaley R, Ford N, Kranzer K. Interventions to improve or facilitate linkage to or retention in pre‐ART (HIV) care and initiation of ART in low‐ and middle‐income settings ‐ a systematic review. Journal of the International AIDS Society. 2014;17(1):19032.
Hamine S, Gerth‐Guyette E, Faulx D, Green BB, Ginsburg AS. Impact of mHealth Chronic Disease Management on Treatment Adherence and Patient Outcomes: A Systematic Review. Journal of medical Internet research. 2015;17(2):e52.
Hecht L, Buhse S, Meyer G. Effectiveness of training in evidence‐based medicine skills for healthcare professionals: a systematic review. BMC medical education. 2016;16(1):103.
Hurt K, Walker RJ, Campbell JA, Egede LE. mHealth Interventions in Low and Middle‐Income Countries: A Systematic Review. Global journal of health science. 2016;8(9):54429.
Jia L, Meng Q, Yuan B, Fang L. Effects of drug cost sharing policy on the drug use, financial risks and moral hazard for the health insurance beneficiaries. Value in Health. 2014;17(7):A795.
Kamarudin G, Penm J, Chaar B, Moles R. Educational interventions to improve prescribing competency: a systematic review. BMJ open. 2013;3(8):e003291.
Kanters S, Park JJ, Chan K, Socias ME, Ford N, Forrest JI, et al. Interventions to improve adherence to antiretroviral therapy: a systematic review and network meta‐analysis. The lancet HIV. 2016.
Ke KM, Blazeby JM, Strong S, Carroll FE, Ness AR, Hollingworth W. Are multidisciplinary teams in secondary care cost‐effective? A systematic review of the literature. Cost effectiveness and resource allocation : C/E. 2013;11(1):7.
Khanal S, Burgon J, Leonard S, Griffiths M, Eddowes LA. Recommendations for the Improved Effectiveness and Reporting of Telemedicine Programs in Developing Countries: Results of a Systematic Literature Review. Telemedicine journal and e‐health : the official journal of the American Telemedicine Association. 2015;21(11):903‐15.
Kok MC, Dieleman M, Taegtmeyer M, Broerse JE, Kane SS, Ormel H, et al. Which intervention design factors influence performance of community health workers in low‐ and middle‐income countries? A systematic review. Health policy and planning. 2014;30(9):1207‐27.
Kondo KK, Damberg CL, Mendelson A, Motu'apuaka M, Freeman M, O'Neil M, et al. Implementation Processes and Pay for Performance in Healthcare: A Systematic Review. Journal of general internal medicine. 2016;31 Suppl 1:61‐9.
Korachais C, Macouillard E, Meessen B. How User Fees Influence Contraception in Low and Middle Income Countries: A Systematic Review. Studies in family planning. 2016;47(4):341‐56.
Körner M, Bütof S, Müller C, Zimmermann L, Becker S, Bengel J. Interprofessional teamwork and team interventions in chronic care: A systematic review. Journal of interprofessional care. 2015;30(1):1‐14.
Kurtzman ET, Greene J. Effective presentation of health care performance information for consumer decision making: A systematic review. Patient education and counseling. 2015;99(1):36‐43.
Lee IH, Bloor K, Hewitt C, Maynard A. International experience in controlling pharmaceutical expenditure: influencing patients and providers and regulating industry ‐ a systematic review. Journal of health services research & policy. 2014;20(1):52‐9.
Lee SH, Nurmatov UB, Nwaru BI, Mukherjee M, Grant L, Pagliari C. Effectiveness of mHealth interventions for maternal, newborn and child health in low‐ and middle‐income countries: Systematic review and meta‐analysis. Journal of global health. 2016;6(1):010401.
Lehnbom EC, Stewart MJ, Manias E, Westbrook JI. Impact of medication reconciliation and review on clinical outcomes. The Annals of pharmacotherapy. 2014;48(10):1298‐312.
L'Engle KL, Mangone ER, Parcesepe AM, Agarwal S, Ippoliti NB. Mobile Phone Interventions for Adolescent Sexual and Reproductive Health: A Systematic Review. Pediatrics. 2016;138(3):1‐16.
Lin Y, Yin S, Huang J, Du L. Impact of Pay for performance on Behavior of Primary Care Physicians and Patient Outcomes. Journal of evidence‐based medicine. 2015;9(1):8‐23.
Liu X, Dou L, Zhang H, Sun Y, Yuan B. Analysis of context factors in compulsory and incentive strategies for improving attraction and retention of health workers in rural and remote areas: a systematic review. Human resources for health. 2015;13:61.
Liu X, Hotchkiss DR, Bose S. The effectiveness of contracting‐out primary health care services in developing countries: A review of the evidence. Health Policy and Planning. 2007; 23(1): 1‐13.
Luangasanatip N, Hongsuwan M, Limmathurotsakul D, Lubell Y, Lee AS, Harbarth S, et al. Comparative efficacy of interventions to promote hand hygiene in hospital: systematic review and network meta‐analysis. BMJ (Clinical research ed). 2015;351:h3728.
Mann BS, Barnieh L, Tang K, Campbell DJ, Clement F, Hemmelgarn B, et al. Association between drug insurance cost sharing strategies and outcomes in patients with chronic diseases: a systematic review. PloS one. 2014;9(3):e89168.
Martínez‐González NA, Djalali S, Tandjung R, Huber‐Geismann F, Markun S, Wensing M, et al. Substitution of physicians by nurses in primary care: a systematic review and meta‐analysis. BMC health services research. 2014;14:214.
Martínez‐González NA, Rosemann T, Tandjung R, Djalali S. The effect of physician‐nurse substitution in primary care in chronic diseases: a systematic review. Swiss medical weekly. 2015;145(no pagination):w14031.
Martínez‐González NA, Tandjung R, Djalali S, Rosemann T. The impact of physician‐nurse task shifting in primary care on the course of disease: a systematic review. Human resources for health. 2015;13:55.
Mbuagbaw L, Sivaramalingam B, Navarro T, Hobson N, Keepanasseril A, Wilczynski NJ, et al. Interventions for Enhancing Adherence to Antiretroviral Therapy (ART): A Systematic Review of High Quality Studies. AIDS patient care and STDs. 2015;29(5):248‐66.
McCollum R, Gomez W, Theobald S, Taegtmeyer M. How equitable are community health worker programmes and which programme features influence equity of community health worker services? A systematic review. BMC public health. 2016;16(1):419.
McCulloch P, Rathbone J, Catchpole K. Interventions to improve teamwork and communications among healthcare staff. The British journal of surgery. 2011;98(4):469‐79.
McGrady ME, Ryan JL, Gutiérrez‐Colina AM, Fredericks EM, Towner EK, Pai AL. The impact of effective paediatric adherence promotion interventions: systematic review and meta‐analysis. Child: care, health and development. 2015;41(6):789‐802.
McMillan SS, Kendall E, Sav A, King MA, Whitty JA, Kelly F, et al. Patient‐centered approaches to health care: a systematic review of randomized controlled trials. Medical care research and review : MCRR. 2013;70(6):567‐96.
Mijovic H, McKnight J, English M. What does the literature tell us about health workers' experiences of task‐shifting projects in sub‐Saharan Africa? A systematic, qualitative review. Journal of clinical nursing. 2016;25(15‐16):2083‐100.
Montagu D, Goodman C, Berman P, Penn A, Visconti A. Recent trends in working with the private sector to improve basic healthcare: a review of evidence and interventions. Health policy and planning. 2016;31(8):1117‐32.
Musa BM, Iliyasu Z, Yusuf SM, Uloko AE. Systematic review and metanalysis on community based interventions in tuberculosis care in developing countries. Nigerian journal of medicine: journal of the National Association of Resident Doctors of Nigeria. 2014;23(2):103‐17.
Mwai GW, Mburu G, Torpey K, Frost P, Ford N, Seeley J. Role and outcomes of community health workers in HIV care in sub‐Saharan Africa: a systematic review. Journal of the International AIDS Society. 2013;16(1):18586.
Nazar H, Nazar Z, Portlock J, Todd A, Slight SP. A systematic review of the role of community pharmacies in improving the transition from secondary to primary care. British journal of clinical pharmacology. 2015;80(5):936‐48.
Nguyen DT, Leung KK, McIntyre L, Ghali WA, Sauve R. Does integrated management of childhood illness (IMCI) training improve the skills of health workers? A systematic review and meta‐analysis. PloS one. 2013;8(6):e66030.
Nijmeijer KJ, Fabbricotti IN, Huijsman R. Is franchising in health care valuable? A systematic review. Health policy and planning. 2014;29(2):164‐76.
Nilsson C, Lundgren I, Smith V, Vehvilainen‐Julkunen K, Nicoletti J, Devane D, et al. Women‐centred interventions to increase vaginal birth after caesarean section (VBAC): A systematic review. Midwifery. 2015;31(7):657‐63.
Ofek Shlomai N, Rao S, Patole S. Efficacy of interventions to improve hand hygiene compliance in neonatal units: a systematic review and meta‐analysis. European journal of clinical microbiology & infectious diseases : official publication of the European Society of Clinical Microbiology. 2015;34:887‐97.
Ogbechie OA, Hsu J. Systematic review of benefit designs with differential cost sharing for prescription drugs. The American journal of managed care. 2015;21(5):e338‐48.
Olisemeke B, Chen YF, Hemming K, Girling A. The Effectiveness of Service Delivery Initiatives at Improving Patients' Waiting Times in Clinical Radiology Departments: A Systematic Review. Journal of digital imaging. 2014;27(6):751‐78.
Owusu‐Addo E, Cross R. The impact of conditional cash transfers on child health in low‐ and middle‐income countries: a systematic review. International journal of public health. 2014;59(4):609‐18.
Pallas SW, Minhas D, Pérez‐Escamilla R, Taylor L, Curry L, Bradley EH. Community Health Workers in Low‐ and Middle‐Income Countries: What Do We Know About Scaling Up and Sustainability? American journal of public health. 2013;103(7):e74‐82.
Patel J, Ahmed K, Guru KA, Khan F, Marsh H, Shamim Khan M, et al. An overview of the use and implementation of checklists in surgical specialities ‐ a systematic review. International journal of surgery (London, England). 2014;12(12):1317‐23.
Rashidian A, Omidvari AH, Vali Y, Mortaz S, Yousefi‐Nooraie R, Jafari M, et al. The effectiveness of regionalization of perinatal care services‐‐a systematic review. Public health. 2014;128(10):872‐85.
Robinson DJ. An integrative review: triage protocols and the effect on ED length of stay. Journal of emergency nursing: JEN : official publication of the Emergency Department Nurses Association. 2013;39(4):398‐408.
Roque MD, Herdeiro MT, Soares SI, Teixeira Rodrigues A, Granadeiro LA, Gusman AF. Educational interventions to improve prescription and dispensing of antibiotics: a systematic review. BMC public health. 2014;14(1):1276.
Ruizendaal E, Dierickx S, Peeters Grietens K, Schallig HD, Pagnoni F, Mens PF. Success or failure of critical steps in community case management of malaria with rapid diagnostic tests: a systematic review. Malaria journal. 2014;13(1):229.
Russ S, Rout S, Sevdalis N, Moorthy K, Darzi A, Vincent C. Do safety checklists improve teamwork and communication in the operating room? A systematic review. Annals of surgery. 2013;258(6):856‐71.
Ruth Lv, Francke AL, Mistiaen P. Effects of nurse prescribing of medication: a systematic review. The Internet Journal of Healthcare Administration. 2008;5(2).
Sacks GD, Shannon EM, Dawes AJ, Rollo JC, Nguyen DK, Russell MM, et al. Teamwork, communication and safety climate: a systematic review of interventions to improve surgical culture. BMJ quality & safety. 2015;24(7):458‐67.
Santschi V, Chiolero A, Colosimo AL, Platt RW, Taffé P, Burnier M, et al. Improving blood pressure control through pharmacist interventions: a meta‐analysis of randomized controlled trials. Journal of the American Heart Association. 2014;3(2):e000718.
Schepman S, Hansen J, de Putter ID, Batenburg RS, de Bakker DH. The common characteristics and outcomes of multidisciplinary collaboration in primary health care: a systematic literature review. International journal of integrated care. 2015;15:e027.
Schmutz J, Manser T. Do team processes really have an effect on clinical performance? A systematic literature review. British journal of anaesthesia. 2013;110(4):529‐44.
Schweizer ML, Reisinger HS, Ohl M, Formanek MB, Blevins A, Ward MA, et al. Searching for an optimal hand hygiene bundle: a meta‐analysis. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America. 2014;58(2):248‐59.
Suwannakeeree W, Picheansathian W. Strategies to Promote Adherence to Treatment by Pulmonary Tuberculosis Patients: A systematic review. JBI Database of Systematic Reviews and Implementation Reports. 2012;10(11):615.
Sweeney S, Obure CD, Maier CB, Greener R, Dehne K, Vassall A. Costs and efficiency of integrating HIV/AIDS services with other health services: a systematic review of evidence and experience. Sexually transmitted infections. 2012;88(2):85‐99.
Thakkar J, Kurup R, Laba TL, Santo K, Thiagalingam A, Rodgers A, et al. Mobile Telephone Text Messaging for Medication Adherence in Chronic Disease: A Meta‐analysis. JAMA internal medicine. 2016;176(3):340‐9.
Trehan A, Maruthappu M, Barnett‐Vanes A, Carty M, McCulloch P. Does feedback of surgical outcome data improve surgical performance? A systematic review. Journal of the American College of Surgeons. 2014;219(4):e148.
Tripathi A, Kabra SK, Sachdev HP, Lodha R. Home visits by community health workers to improve identification of serious illness and care seeking in newborns and young infants from low‐ and middle‐income countries. Journal of perinatology : official journal of the California Perinatal Association. 2016;36 Suppl 1(S1):S74‐82.
Tshiananga JK, Kocher S, Weber C, Erny‐Albrecht K, Berndt K, Neeser K. The effect of nurse‐led diabetes self‐management education on glycosylated hemoglobin and cardiovascular risk factors: a meta‐analysis. The Diabetes educator. 2011;38(1):108‐23.
Tsiachristas A, Wallenburg I, Bond CM, Elliot RF, Busse R, van Exel J, et al. Costs and effects of new professional roles: Evidence from a literature review. Health policy (Amsterdam, Netherlands). 2015;119(9):1176‐87.
Turcotte‐Tremblay AM, Spagnolo J, De Allegri M, Ridde V. Does performance‐based financing increase value for money in low‐ and middle‐ income countries? A systematic review. Health economics review. 2015;6(1):30.
Uyei J, Coetzee D, Macinko J, Guttmacher S. Integrated delivery of HIV and tuberculosis services in Sub‐Saharan Africa: A systematic review. International Initiative for Impact Evaluation (3ie). 2011.
Van Camp YP, Van Rompaey B, Elseviers MM. Nurse‐led interventions to enhance adherence to chronic medication: systematic review and meta‐analysis of randomised controlled trials. European journal of clinical pharmacology. 2013;69(4):761‐70.
van Velthoven MHMMT, Tudor Car L, Gentry S, Car J. Telephone delivered interventions for preventing HIV infection in HIV‐negative persons. Cochrane Database of Systematic Reviews. 2013;5(5):CD009190.
Wagner B, Filice GA, Drekonja D, Greer N, MacDonald R, Rutks I, et al. Antimicrobial stewardship programs in inpatient hospital settings: a systematic review. Infection control and hospital epidemiology : the official journal of the Society of Hospital Epidemiologists of America. 2014;35(10):1209‐28.
Wald DS, Butt S, Bestwick JP. One‐way versus two‐way text messaging on improving medication adherence: meta‐analysis of randomized trials. The American journal of medicine. 2015;128(10):1139.e1‐5.
Wallace AS, Ryman TK, Dietz V. Experiences integrating delivery of maternal and child health services with childhood immunization programs: systematic review update. The Journal of infectious diseases. 2012;205 Suppl 1:S6‐19.
Wallace J, Byrne C, Clarke M. Improving the uptake of systematic reviews: a systematic review of intervention effectiveness and relevance. BMJ open. 2014;4(10):e005834.
Watson SJ, Aldus CF, Bond C, Bhattacharya D. Systematic review of the health and societal effects of medication organisation devices. BMC health services research. 2016;16(1):202.
Wilcher R, Hoke T, Adamchak SE, Cates W. Integration of family planning into HIV services: a synthesis of recent evidence. AIDS (London, England). 2013;27 Suppl 1:S65‐75.
World Health Organization, University of California SF. Electronic reminders for promoting adherence to ART among people living with HIV. World Health Organization. 2013.
World Health Organziation, University of California SF. Integration of HIV and TB services. World Health Organization. 2013.
Yamada J, Shorkey A, Barwick M, Widger K, Stevens BJ. The effectiveness of toolkits as knowledge translation strategies for integrating evidence into clinical care: a systematic review. BMJ open. 2015;5(4):e006808.
Zulu JM, Kinsman J, Michelo C, Hurtig AK. Integrating national community‐based health worker programmes into health systems: a systematic review identifying lessons learned from low‐and middle‐income countries. BMC public health. 2014;14(1):987.
Contributions of authors
All of the authors contributed to drafting and revising the overview. All of the authors contributed important intellectual input to the overview.
Sources of support
Internal sources
Department of Family Medicine, School of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile.
Institute for Clinical Effectiveness and Health Policy, Buenos Aires, Argentina.
Norwegian Knowledge Centre for the Health Services, Oslo, Norway.
Department of Public Health, School of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile.
South African Medical Research Council, Cape Town, South Africa.
External sources
Norwegian Agency for Development Cooperation (Norad), Oslo, Norway.
National Research Foundation (CSW), South Africa.
The Effective Health Care Research Consortium which is funded by UK aid from the UK Government for the benefit of developing countries, UK.
Declarations of interest
Cristian A Herrera, Simon Lewin, Elizabeth Paulsen, Newton Opiyo, Tomas Pantoja, Gabriel Rada, and Andrew D Oxman are editors of the Cochrane Effective Practice and Organisation of Care (EPOC) Group. Agustín Ciapponi, Tomas Pantoja, Gabriel Rada, Cristian A Herrera, Andrew D Oxman, and Blanca Peñaloza are authors on some of the included reviews. Charles S Wiysonge, Gabriel Bastías, Sebastian Garcia Marti, and Charles I Okwundu have no relevant conflicts to declare.
New
References
References to included reviews
Acosta 2014
- Acosta A, Ciapponi A, Aaserud M, Vietto V, Austvoll‐Dahlgren A, Kösters JP, et al. Pharmaceutical policies: effects of reference pricing, other pricing, and purchasing policies. Cochrane Database of Systematic Reviews 2014, Issue 10. [DOI: 10.1002/14651858.CD005979.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
El‐Jardali 2015
- El‐Jardali F, Akl EA, Fadlallah R, Oliver S, Saleh N, El‐Bawab L, et al. Interventions to combat or prevent drug counterfeiting: a systematic review. BMJ Open 2015;5(3):e006290. [DOI: 10.1136/bmjopen-2014-006290] [DOI] [PMC free article] [PubMed] [Google Scholar]
Flodgren 2011
- Flodgren G, Pomey MP, Taber SA, Eccles MP. Effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes. Cochrane Database of Systematic Reviews 2011, Issue 11. [DOI: 10.1002/14651858.CD008992.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Fung 2008
- Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that publishing patient care performance data improves quality of care. Annals of Internal Medicine 2008;148(2):111‐23. [DOI] [PubMed] [Google Scholar]
Gilbody 2005
- Gilbody S, Wilson P, Watt I. Benefits and harms of direct to consumer advertising: a systematic review. Quality & Safety in Health Care 2005;14(4):246‐50. [DOI] [PMC free article] [PubMed] [Google Scholar]
Green 2010
- Green CJ1, Maclure M, Fortin PM, Ramsay CR, Aaserud M, Bardal S. Pharmaceutical policies: effects of restrictions on reimbursement. Cochrane Database of Systematic Reviews 2010, Issue 8. [DOI: 10.1002/14651858.CD008654] [DOI] [PMC free article] [PubMed] [Google Scholar]
Grobler 2015
- Grobler LA, Marais BJ, Mabunda S. Interventions for increasing the proportion of health professionals practising in rural and other underserved areas. Cochrane Database of Systematic Reviews 2015, Issue 6. [DOI: 10.1002/14651858.CD005314.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]
Hayes 2012
- Hayes SL, Mann MK, Morgan FM, Kelly MJ, Weightman AL. Collaboration between local health and local government agencies for health improvement. Cochrane Database of Systematic Reviews 2012, Issue 10. [DOI: 10.1002/14651858.CD007825.pub6] [DOI] [PMC free article] [PubMed] [Google Scholar]
Heintze 2007
- Heintze C, Velasco Garrido M, Kroeger A. What do community‐based dengue control programmes achieve? A systematic review of published evaluations. Transactions of the Royal Society of Tropical Medicine & Hygiene 2007;101(4):317‐25. [DOI] [PubMed] [Google Scholar]
Kiwanuka 2011
- Kiwanuka SN, Rutebemberwa E, Nalwadda C, Okui O, Ssengooba F, Kinengyere AA, et al. Interventions to manage dual practice among health workers. Cochrane Database of Systematic Reviews 2011, Issue 7. [DOI: 10.1002/14651858.CD008405.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Koehlmoos 2009
- Koehlmoos TP, Gazi R, Hossain SS, Zaman K. The effect of social franchising on access to and quality of health services in low‐ and middle‐income countries. Cochrane Database of Systematic Reviews 2009, Issue 1. [DOI: 10.1002/14651858.CD007136.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Lagarde 2009
- Lagarde M, Palmer N. The impact of contracting out on health outcomes and use of health services in low and middle‐income countries. Cochrane Database of Systematic Reviews 2009, Issue 4. [DOI: 10.1002/14651858.CD008133] [DOI] [PubMed] [Google Scholar]
Nilsen 2010
- Nilsen ES, Myrhaug HT, Johansen M, Oliver S, Oxman AD. Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material. Cochrane Database of Systematic Reviews 2006, Issue 3. [DOI: 10.1002/14651858.CD004563.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Pariyo 2009
- Pariyo GW, Kiwanuka SN, Rutebemberwa E, Okui O, Ssengooba. Effects of changes in the pre‐licensure education of health workers on health‐worker supply. Cochrane Database of Systematic Reviews 2009, Issue 2. [DOI: 10.1002/14651858.CD007018.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Peñaloza 2011
- Peñaloza B, Pantoja T, Bastías G, Herrera C, Rada G. Interventions to reduce emigration of health care professionals from low‐ and middle‐income countries. Cochrane Database of Systematic Reviews 2011, Issue 9. [DOI: 10.1002/14651858.CD007673.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Prost 2013
- Prost A, Colbourn T, Seward N, et al. Women's groups practicing participatory learning and action to improve maternal and newborn health in low‐resource settings: a systematic review and meta‐analysis. Lancet 2013;381(9879):1736‐46. [DOI] [PMC free article] [PubMed] [Google Scholar]
Rashidian 2012
- Rashidian A, Joudaki H, Vian T. No evidence of the effect of the interventions to combat health care fraud and abuse: a systematic review of literature. PLOS ONE 2012;7(8):e41988. [DOI: 10.1371/journal.pone.0041988] [DOI] [PMC free article] [PubMed] [Google Scholar]
Rockers 2013
- Rockers PC, Bärnighausen T. Interventions for hiring, retaining and training district health systems managers in low‐ and middle‐income countries. Cochrane Database of Systematic Reviews 2013, Issue 4. [DOI: 10.1002/14651858.CD009035.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Rutebemberwa 2014
- Rutebemberwa E, Kinengyere AA, Ssengooba F, Pariyo GW, Kiwanuka SN. Financial interventions and movement restrictions for managing the movement of health workers between public and private organizations in low‐ and middle‐income countries. Cochrane Database of Systematic Reviews 2014, Issue 2. [DOI: 10.1002/14651858.CD009845.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
References to excluded reviews
Berendes 2011
- Berendes S, Heywood P, Oliver S, Garner P. Quality of private and public ambulatory health care in low and middle income countries: systematic review of comparative studies. PLOS Medicine 2011;8(4):e1000433. [DOI: 10.1371/journal.pmed.1000433] [DOI] [PMC free article] [PubMed] [Google Scholar]
Boote 2002
- Boote J, Telford R, Cooper C. Consumer involvement in health research: a review and research agenda. Health Policy 2002;61(2):213‐36. [DOI] [PubMed] [Google Scholar]
Bärnighausen 2009
- Bärnighausen T, Bloom DE. Financial incentives for return of service in underserved areas: A systematic review. BMC Health Services Research 2009;29(9):86. [DOI] [PMC free article] [PubMed] [Google Scholar]
Comondore 2009
- Comondore VR, Devereaux PJ, Zhou Q, Stone SB, Busse JW, Ravindran NC, et al. Quality of care in for‐profit and not‐for‐profit nursing homes: systematic review and meta‐analysis. BMJ 2009;339:b2732. [DOI: 10.1136/bmj.b2732] [DOI] [PMC free article] [PubMed] [Google Scholar]
Crawford 2002
- Crawford MJ, Rutter D, Manley C, Weaver T, Bhui K, Fulop N, Tyrer P. Systematic review of involving patients in the planning and development of health care. BMJ 2002;325(7375):1263. [DOI] [PMC free article] [PubMed] [Google Scholar]
Devereaux 2002a
- Devereaux PJ, Choi PT, Lacchetti C, Weaver B, Schünemann HJ, Haines T, et al. A systematic review and meta‐analysis of studies comparing mortality rates of private for‐profit and private not‐for‐profit hospitals. CMAJ 2002;166(11):1399‐406. [PMC free article] [PubMed] [Google Scholar]
Devereaux 2002b
- Devereaux PJ, Schünemann HJ, Ravindran N, Bhandari M, Garg AX, Choi PT, et al. Comparison of mortality between private for‐profit and private not‐for‐profit hemodialysis centers. JAMA 2002;288(19):2449‐57. [DOI] [PubMed] [Google Scholar]
Devereaux 2004
- Devereaux PJ, Heels‐Ansdell D, Lacchetti C, Haines T, Burns KE, Cook DJ, et al. Payments for care at private for‐profit and private not‐for‐profit hospitals: a systematic review and meta‐analysis. CMAJ 2004;170(12):1817‐24. [DOI] [PMC free article] [PubMed] [Google Scholar]
Ekman 2004
- Ekman B. Community‐based health insurance in low‐income countries: a systematic review of the evidence. Health Policy and Planning 2004;19(5):249‐70. [DOI] [PubMed] [Google Scholar]
Faber 2009
- Faber M, Bosch M, Wollersheim H, Leatherman S, Grol R. Public reporting in health care: how do consumers use quality‐of‐care information? A systematic review. Medical Care 2009;47(1):1‐8. [DOI: 10.1097/MLR.0b013e3181808bb5] [DOI] [PubMed] [Google Scholar]
Faden 2011
- Faden L, Vialle‐Valentin C, Ross‐Degnan D, Wagner A. Active pharmaceutical management strategies of health insurance systems to improve cost‐effective use of medicines in low‐ and middle‐income countries: a systematic review of current evidence. Health Policy 2011;100(2‐3):134‐43. [DOI: 10.1016/j.healthpol.2010.10.020] [DOI] [PubMed] [Google Scholar]
Greenfield 2008
- Greenfield D, Braithwaite J. Health sector accreditation research: a systematic review. International Journal for Quality in Health Care 2008;20(3):172‐83. [DOI: 10.1093/intqhc/mzn005] [DOI] [PubMed] [Google Scholar]
Greenfield 2012
- Greenfield D, Pawsey M, Hinchcliff R, Moldovan M, Braithwaite J. The standard of healthcare accreditation standards: a review of empirical research underpinning their development and impact. BMC Health Services Research 2012;12:329. [DOI: 10.1186/1472-6963-12-329] [DOI] [PMC free article] [PubMed] [Google Scholar]
Griffiths 2007
- Griffiths PD1, Edwards MH, Forbes A, Harris RL, Ritchie G. Effectiveness of intermediate care in nursing‐led in‐patient units. Cochrane Database of Systematic Reviews 2007, Issue 2. [DOI: 10.1002/14651858.CD002214.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]
Henderson 2010
- Henderson A, Henderson S. Provision of a surgeon's performance data for people considering elective surgery. Cochrane Database of Systematic Reviews 2010, Issue 11. [DOI: 10.1002/14651858.CD006327.pub2] [DOI] [PubMed] [Google Scholar]
Jia 2014
- Jia L, Yuan B, Huang F, Lu Y, Garner P, Meng Q. Strategies for expanding health insurance coverage in vulnerable populations. Cochrane Database of Systematic Reviews 2014, Issue 11. [DOI: 10.1002/14651858.CD008194.pub3] [DOI] [PMC free article] [PubMed] [Google Scholar]
Lagarde 2006
- Lagarde M, Palmer N. Evidence from systematic reviews to inform decision making regarding financing mechanisms that improve access to health services for poor people. A policy brief prepared for the International Dialogue on Evidence‐Informed Action to Achieve Health Goals in Developing Countries. Khon Kaen, Thailand: Alliance for Health Policy and Systems Research; 2006 December. Available from www.who.int/rpc/meetings/HealthFinancingBrief.pdf.
Lee 2009
- Lee AC, Lawn JE, Cousens S, Kumar V, Osrin D, Bhutta ZA, et al. Linking families and facilities for care at birth: What works to avert intrapartum‐related deaths?. International Journal of Gynaecology and Obstetrics 2009;107(Suppl 1):S65‐85, S86‐8. [DOI: 10.1016/j.ijgo.2009.07.012] [DOI] [PMC free article] [PubMed] [Google Scholar]
Lehmann 2008
- Lehmann U, Dieleman M, Martineau T. Staffing remote rural areas in middle‐ and low‐income countries: a literature review of attraction and retention. BMC Health Services Research 2008;8:19. [DOI: 10.1186/1472-6963-8-19] [DOI] [PMC free article] [PubMed] [Google Scholar]
Liu 2008
- Liu X, Hotchkiss DR, Bose S. The effectiveness of contracting‐out primary health care services in developing countries: a review of the evidence. Health Policy and Planning 2008;23(1):1‐13. [DOI] [PubMed] [Google Scholar]
Loevinsohn 2004
- Loevinsohn B, Harding A. Contracting for the delivery of community health services: a review of global experience. Washington, DC: World Bank: 2004. Available at: openknowledge.worldbank.org/handle/10986/13621.
Maharaj 2015
- Maharaj R, Raffaele I, Wendon J. Rapid response systems: a systematic review and meta‐analysis. Critical Care 2015;19:254. [DOI] [PMC free article] [PubMed] [Google Scholar]
Marshall 2000
- Marshall MN, Shekelle PG, Leatherman S, Brook RH. The public release of performance data: what do we expect to gain? A review of the evidence. JAMA 2000;283(14):1866‐74. [DOI] [PubMed] [Google Scholar]
Meng 2010
- Meng Q, Yuan B, Jia L, Wang J, Yu B, Gao J, Garner P. Expanding health insurance coverage in vulnerable groups: a systematic review of options. Health Policy and Planning 2011;26(2):93‐104. [DOI: 10.1093/heapol/czq038] [DOI] [PubMed] [Google Scholar]
Molyneux 2012
- Molyneux S, Atela M, Angwenyi V, Goodman C. Community accountability at peripheral health facilities: a review of the empirical literature and development of a conceptual framework. Health Policy and Planning 2012;27(7):541‐54. [DOI: 10.1093/heapol/czr083] [DOI] [PMC free article] [PubMed] [Google Scholar]
Montagu 2011
- Montagu DD, Anglemyer A, Tiwari M, Drasser K, Rutherford GW. Private versus public strategies for health service provision for improving health outcomes in resource‐limited settings. San Francisco, CA: Global Health Sciences, University of California, San Francisco. 2011.
Morgan 2009
- Morgan S, Hanley G, Greyson D. Comparison of tiered formularies and reference pricing policies: a systematic review. Open Medicine 2009;3(3):e131‐9. [PMC free article] [PubMed] [Google Scholar]
Ossai 2012
- Ossai EN, Ibiok NC, Chukwuogo O, Umeobieri AK, Aniwada EC, Ugwunna NC. Rural retention of human resources for health. Nigerian Journal of Medicine 2012;21(2):138‐45. [PubMed] [Google Scholar]
Patouillard 2007
- Patouillard E, Goodman CA, Hanson KG, Mills AJ. Can working with the private for‐profit sector improve utilization of quality health services by the poor? A systematic review of the literature. International Journal for Equity in Health 2007;6:17. [DOI] [PMC free article] [PubMed] [Google Scholar]
Patterson 2010
- Patterson M, Rick J, Wood S, Carroll C, Balain S, Booth A. Systematic review of the links between human resource management practices and performance. Health Technology Assessment 2010;14(51):1‐334. [DOI: 10.3310/hta14510] [DOI] [PubMed] [Google Scholar]
Peters 2004
- Peters DH, Mirchandani GG, Hansen PM. Strategies for engaging the private sector in sexual and reproductive health: how effective are they?. Health Policy and Planning 2004;19(Suppl 1):i5‐i21. [DOI] [PubMed] [Google Scholar]
Phillips 2010
- Phillips CB, Pearce CM, Hall S, Travaglia J, Lusignan S, Love T, Kljakovic M. Can clinical governance deliver quality improvement in Australian general practice and primary care? A systematic review of the evidence. Medical Journal of Australia 2010;193(10):602‐7. [DOI] [PubMed] [Google Scholar]
Preston 2010
- Preston R, Waugh H, Larkins S, Taylor J. Community participation in rural primary health care: intervention or approach?. Australian Journal of Primary Health 2010;16(1):4‐16. [DOI] [PubMed] [Google Scholar]
Puig‐Junoy 2007
- Puig‐Junoy J, Moreno‐Torres I. Impact of pharmaceutical prior authorisation policies: a systematic review of the literature. Pharmacoeconomics 2007;25(8):637‐48. [DOI] [PubMed] [Google Scholar]
Schadewaldt 2011
- Schadewaldt V, Schultz T. Nurse‐led clinics as an effective service for cardiac patients: results from a systematic review. International Journal of Evidence Based Healthcare 2011;9(3):199‐214. [DOI: 10.1111/j.1744-1609.2011.00217.x] [DOI] [PubMed] [Google Scholar]
Shah 2011
- Shah NM, Brieger WR, Peters DH. Can interventions improve health services from informal private providers in low and middle‐income countries? A comprehensive review of the literature. Health Policy and Planning 2011;26(4):275‐87. [DOI: 10.1093/heapol/czq074] [DOI] [PubMed] [Google Scholar]
Sharp 2002
- Sharp LK, Bashook PG, Lipsky MS, Horowitz SD, Miller SH. Specialty board certification and clinical outcomes: the missing link. Academic Medicine 2002;77(6):534‐42. [DOI] [PubMed] [Google Scholar]
Shen 2007
- Shen YC, Eggleston K, Lau J, Schmid CH. Hospital ownership and financial performance: a quantitative research review. Inquiry 2007;44(1):41‐68. [DOI] [PubMed] [Google Scholar]
Socha 2011
- Socha KZ, Bech M. Physician dual practice: a review of literature. Health Policy 2011;102(1):1‐7. [DOI: 10.1016/j.healthpol.2010.10.017] [DOI] [PubMed] [Google Scholar]
Steinman 2006
- Steinman MA, Ranji SR, Shojania KG, Gonzales R. Improving antibiotic selection: a systematic review and quantitative analysis of quality improvement strategies. Medical Care 2006;44(7):617‐28. [DOI] [PubMed] [Google Scholar]
Tait 2004
- Tait AR. Clinical governance in primary care: a literature review. Journal of Clinical Nursing 2004;13(6):723‐30. [DOI] [PubMed] [Google Scholar]
Wafula 2012
- Wafula FN, Miriti EM, Goodman CA. Examining characteristics, knowledge and regulatory practices of specialized drug shops in Sub‐Saharan Africa: a systematic review of the literature. BMC Health Services Research 2012;27(12):223. [DOI: 10.1186/1472-6963-12-223] [DOI] [PMC free article] [PubMed] [Google Scholar]
Waters 2003
- Waters H, Hatt L, Peters D. Working with the private sector for child health. Health Policy and Planning 2003;18(2):127‐37. [DOI] [PubMed] [Google Scholar]
Willis‐Shattuck 2008
- Willis‐Shattuck M, Bidwell P, Thomas S, Wyness L, Blaauw D, Ditlopo P. Motivation and retention of health workers in developing countries: a systematic review. BMC Health Services Research 2008;8:247. [DOI: 10.1186/1472-6963-8-247] [DOI] [PMC free article] [PubMed] [Google Scholar]
Wilson 2009
- Wilson NW, Couper ID, Vries E, Reid S, Fish T, Marais BJ. A critical review of interventions to redress the inequitable distribution of healthcare professionals to rural and remote areas. Rural Remote Health 2009;9(2):1060. [PubMed] [Google Scholar]
Additional references
Bennington 2010
- Bennington L. Review of the corporate and healthcare governance literature. Journal of Management & Organization 2010;16(2):314‐33. [Google Scholar]
Brunton 2015
- Brunton G, Caird J, Stokes G, Stansfield C, Kneale D, Richardson M. Community engagement for health via coalitions, collaborations and partnerships. London: EPPI‐Centre, Social Science Research Unit, Institute of Education, University of London; 2015.
Ciapponi 2014
- Ciapponi A, Lewin S, Bastías G, Dudley L, Flottorp S, Gagnon M, et al. Delivery arrangements for health systems in low‐income countries: an overview of systematic reviews. Cochrane Database of Systematic Reviews 2014, Issue 5. [DOI: 10.1002/14651858.CD011083] [DOI] [PMC free article] [PubMed] [Google Scholar]
Dogson 2002
- Dodgson R, Lee K, Drager N. Global Health Governance: A Conceptual Review. Geneva: World Health Organization, Department of Health and Development, 2002. [Google Scholar]
EPOC 2017
- Cochrane Effective Practice, Organisation of Care (EPOC). EPOC Resources for review authors, 2017. Available at: epoc.cochrane.org/epoc‐resources‐review‐authors.
Forest 1999
- Forest PG, Gagnon D, Abelson J, Turgeon J, Lamarche P. Ottawa: Issues in the governance of integrated health systems. Policy synthesis. Canadian Health Services Research Foundation; 1999. Available from www.chsrf.ca/final_research/commissioned_research/policy_synthesis/pdf/pses‐governihs_e.pdf.
Frenk 2013
- Frenk J, Moon S. Governance challenges in global health. New England Journal of Medicine 2013;368:936‐42. [DOI] [PubMed] [Google Scholar]
Guyatt 2008
- Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck‐Ytter Y, Schunemann HJ, et al. What is "quality of evidence" and why is it important to clinicians?. BMJ 2008;336(7651):995‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]
Herrera 2013
- Herrera CA, Rada G, Barrios X, Kuhn‐Barrientos L, Torres R, Pantoja T. Comparing public, private not‐for‐profit and private for‐profit performance of health care organizations in low and middle income countries: a systematic review. PROSPERO 2013: CRD42013005716 Available from www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42013005716.
Hufty 2006
- Hufty M, Báscolo E, Bazzani R. Governance in health: a conceptual and analytical approach to research [Gobernanza en salud: un aporte conceptual y analítico para la investigación]. Cadernos de Saúde Pública 2006;22 Suppl:S35‐45. [DOI] [PubMed] [Google Scholar]
Ketelaar 2011
- Ketelaar NA, Faber MJ, Flottorp S, Rygh LH, Deane KH, Eccles MP. Public release of performance data in changing the behaviour of healthcare consumers, professionals or organisations. Cochrane Database of Systematic Reviews 2011, Issue 9. [DOI: 10.1002/14651858.CD004538.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Lavis 2015
- Lavis JN, Wilson MG, Moat KA, Hammill AC, Boyko JA, Grimshaw JM, Flottorp S. Developing and refining the methods for a 'one‐stop shop' for research evidence about health systems. Health Research Policy and Systems / BioMed Central 2015;13:10. [DOI: 10.1186/1478-4505-13-10] [DOI] [PMC free article] [PubMed] [Google Scholar]
Lewin 2008
- Lewin S, Lavis JN, Oxman AD, Bastías G, Chopra M, Ciapponi A, et al. Supporting the delivery of cost‐effective interventions in primary health‐care systems in low‐income and middle‐income countries: an overview of systematic reviews. Lancet 2008;372(9642):928‐39. [DOI: 10.1016/S0140-6736(08)61403-8] [DOI] [PubMed] [Google Scholar]
Murray 2000
- Murray CJL, Frenk J. A framework for assessing the performance of health systems. Bulletin of the World Health Organization 2000;78(6):717‐31. [PMC free article] [PubMed] [Google Scholar]
Pantoja 2014
- Pantoja T, Opiyo N, Ciapponi A, Dudley L, Gagnon MP, Herrera CA, et al. Implementation strategies for health systems in low‐income countries: an overview of systematic reviews. Cochrane Database of Systematic Reviews 2014, Issue 5. [DOI: 10.1002/14651858.CD011086] [DOI] [PMC free article] [PubMed] [Google Scholar]
Pantoja 2015
- Pantoja T, Peñaloza B, Cid C, Herrera CA, Bigdeli M. Pharmaceutical policies: effects of policies regulating drug insurance schemes. Cochrane Database of Systematic Reviews. Ongoing review, 2015; Vol. Pantoja T, Peñaloza B, Cid C, Herrera CA, Bigdeli M. Pharmaceutical policies: effects of policies regulating drug insurance schemes (Protocol). Cochrane Database of Systematic Reviews 2015, Issue 5. Art. No.: CD011703. DOI: 10.1002/14651858.CD011703.. [DOI] [PMC free article] [PubMed]
Peñaloza 2015
- Peñaloza B, Pantoja T, Bigdeli M, Herrera CA. Pharmaceutical policies: effects of sales and dispensing policies. Cochrane Database of Systematic Reviews 2015, Issue 8. [DOI: 10.1002/14651858.CD011830] [DOI] [PMC free article] [PubMed] [Google Scholar]
Rada 2013
- Rada G, Pérez D, Capurro D. Epistemonikos: a free, relational, collaborative, multilingual database of health evidence. Studies in Health Technology and Informatics 2013;192:486‐90. [PubMed] [Google Scholar]
Rosenau 1995
- Rosenau JN. Governance in the twenty‐first century. Global Governance 1995;1(1):13‐43. [Google Scholar]
Rosenbaum 2011
- Rosenbaum SE, Glenton C, Wiysonge CS, Abalos E, Mignini L, Young T, et al. Evidence summaries tailored to health policy‐makers in low‐ and middle‐income countries. Bulletin of the World Health Organization 2011;89:54‐61. [DOI] [PMC free article] [PubMed] [Google Scholar]
Saltman 2000
- Saltman RB, Ferroussier‐Davis O. The concept of stewardship in health policy. Bulletin of the World Health Organization 2000;78(6):732‐9. [PMC free article] [PubMed] [Google Scholar]
Savedoff 2011
- Savedoff WD. Governance in the health sector: a strategy for measuring determinants and performance; 2011 May. Policy Research working paper WPS5655. Available from www‐wds.worldbank.org/external/default/WDSContentServer/IW3P/IB/2011/05/09/000158349_20110509125737/Rendered/PDF/WPS5655.pdf.
Schünemann 2011a
- Schünemann HJ, Oxman AD, Higgins JPT, Vist GE, Glasziou P, Guyatt GH. Chapter 11: Presenting results and ‘Summary of findings' tables. In: Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from www.cochrane‐handbook.org.
Schünemann 2011b
- Schünemann HJ, Oxman AD, Vist GE, Higgins JPT, Deeks JJ, Glasziou P, et al. Chapter 12: Interpreting results and drawing conclusions. In: Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from www.cochrane‐handbook.org.
Scott 2009
- Scott I. What are the most effective strategies for improving quality and safety of health care?. Internal Medicine Journal 2009;39(6):389‐400. [DOI: 10.1111/j.1445-5994.2008.01798.x] [DOI] [PubMed] [Google Scholar]
Siddiqi 2009
- Siddiqi S, Masud TI, Nishtar S, Peters DH, Sabri B, Bile KM, et al. Framework for assessing governance of the health system in developing countries: Gateway to good governance. Health Policy 2009;90(1):13‐25. [DOI] [PubMed] [Google Scholar]
Stoker 1998
- Stoker G. Governance as theory: five propositions. International Social Science Journal 1998;50(155):17‐30. [Google Scholar]
SUPPORT 2009
- Lewin S, Oxman AD, Lavis JN, Fretheim A. SUPPORT Tools for evidence‐informed health Policymaking (STP) 8: Deciding how much confidence to place in a systematic review. Health Research Policy and Systems 2009;7(Suppl 1):S8. [DOI] [PMC free article] [PubMed] [Google Scholar]
SURE 2011
- SURE Collaboration. Chapter 4. Deciding on and describing policy options. In: SURE Guides for Preparing and Using Evidence‐Based Policy Briefs. Version 2.1 (updated November 2011). The SURE Collaboration, 2011. Available from epoc.cochrane.org/sure‐guides.
Travis 2003
- Travis P, Egger D, Davies P, Mechbal A. Chapter 25: Towards better stewardship: concepts and critical issues. Murray CJL, Evans DB (editors), Health Systems Performance Assessment: Debates, Methods and Empiricism. Geneva: World Health Organization, 2003. [Google Scholar]
Van Olmen 2010
- Olmen J, Criel B, Damme W, Marchal B, Belle S, Dormael M, et al. Analysing Health Systems to Make Them Stronger. Antwerpen: Institute of Tropical Medicine, 2010. (Van Lerberghe W, Kegels G, and De Brouwere V, editors. Studies in health services organisation & policy). Available from www.strengtheninghealthsystems.be/doc/SHSO&P27_HS%20ANALYSIS_FINAL.pdf. [Google Scholar]
WHO 1978
- World Health Organization. WHO called to return to the Declaration of Alma‐Ata. International conference on primary health care. www.who.int/social_determinants/tools/multimedia/alma_ata/en/ (accessed prior to 8 August 2017).
WHO 2000
- World Health Organization. The World Health Report 2000 ‐ Health Systems: Improving Performance. Geneva: World Health Organization, 2010. Available from http://www.who.int/whr/2000/en/. [Google Scholar]
WHO 2007
- World Health Organization. Everybody's Business: Strengthening Health Systems to Improve Health Outcomes: WHO's Framework for Action. Geneva: World Health Organization, 2007. Available from www.who.int/healthsystems/strategy/everybodys_business.pdf. [Google Scholar]
WHO 2010
- World Health Organization. Increasing Access to Health Workers in Remote and Rural Areas through Improved Retention: Global Policy Recommendations. Geneva: World Health Organization, 2010. [PubMed] [Google Scholar]
Wikipedia 2011
- Wikipedia. Governance; 2011 January. en.wikipedia.org/w/index.php?title=Governance&oldid=406646940 (accessed prior to 8 August 2017).
Wiysonge 2014
- Wiysonge CS, Herrera CA, Ciapponi A, Lewin S, Garcia Marti S, Opiyo N, et al. Financial arrangements for health systems in low‐income countries: an overview of systematic reviews. Cochrane Database of Systematic Reviews 2014, Issue 4. [DOI: 10.1002/14651858.CD011084] [DOI] [PMC free article] [PubMed] [Google Scholar]
World Bank Group 2013
- World Bank Group. Worldwide Governance Indicators (updated 25 Sep 2013). Available from databank.worldbank.org/data/views/variableselection/selectvariables.aspx?source=worldwide‐governance‐indicators (accessed prior to 8 August 2017).
World Bank Group 2016
- World Bank Group. Data. Countries and Economies, 2016. Available from data.worldbank.org/country (accessed prior to 8 August 2017).
References to other published versions of this review
Herrera 2014
- Herrera CA, Ciapponi A, Bastías G, Lewin S, Garcia Marti S, Okwundu CI, et al. Governance arrangements for health systems in low‐income countries: an overview of systematic reviews. Cochrane Database of Systematic Reviews 2014, Issue 4. [DOI: 10.1002/14651858.CD011085] [DOI] [PMC free article] [PubMed] [Google Scholar]