Abstract
Background
Governments use different approaches to ensure that private for‐profit healthcare services meet certain quality standards. Such government guidance, referred to as public stewardship, encompasses government policies, regulatory mechanisms, and implementation strategies for ensuring accountability in the delivery of services. However, the effectiveness of these strategies in low‐ and middle‐income countries (LMICs) have not been the subject of a systematic review.
Objectives
To assess the effects of public sector regulation, training, or co‐ordination of the private for‐profit health sector in low‐ and middle‐income countries.
Search methods
For related systematic reviews, we searched the Cochrane Database of Systematic Reviews (CDSR) 2015, Issue 4; Database of Abstracts of Reviews of Effectiveness (DARE) 2015, Issue 1; Health Technology Assessment Database (HTA) 2015, Issue 1; all part of The Cochrane Library, and searched 28 April 2015. For primary studies, we searched MEDLINE, Epub Ahead of Print, In‐Process & Other Non‐Indexed Citations, MEDLINE Daily and MEDLINE 1946 to Present, OvidSP (searched 16 June 2016); Science Citation Index and Social Sciences Citation Index 1987 to present, and Emerging Sources Citation Index 2015 to present, ISI Web of Science (searched 3 May 2016 for papers citing included studies); Cochrane Central Register of Controlled Trials (CENTRAL), 2015, Issue 3, part of The Cochrane Library (including the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register) (searched 28 April 2015); Embase 1980 to 2015 Week 17, OvidSP (searched 28 April 2015); Global Health 1973 to 2015 Week 16, OvidSP (searched 30 April 2015); WHOLIS, WHO (searched 30 April 2015); Science Citation Index and Social Sciences Citation Index 1975 to present, ISI Web of Science (searched 30 April 2015); Health Management, ProQuest (searched 22 November 2013). In addition, in April 2016, we searched the reference lists of relevant articles, WHO International Clinical Trials Registry Platform, Clinicaltrials.gov, and various electronic databases of grey literature.
Selection criteria
Randomised trials, non‐randomised trials, interrupted time series studies, or controlled before‐after studies.
Data collection and analysis
Two authors independently assessed study eligibility and extracted data, comparing their results and resolving discrepancies by consensus. We expressed study results as risk ratios (RR) or mean differences (MD) with 95% confidence intervals (CI), where appropriate, and assessed the certainty of the evidence using Grades of Recommendation, Assessment, Development and Evaluation (GRADE). We did not conduct meta‐analysis because of heterogeneity of interventions and study designs.
Main results
We identified 20,177 records, 50 of them potentially eligible. We excluded 39 potentially eligible studies because they did not involve a rigorous evaluation of training, regulation, or co‐ordination of private for‐profit healthcare providers in LMICs; five studies identified after the review was submitted are awaiting assessment; and six studies met our inclusion criteria. Two included studies assessed training alone; one assessed regulation alone; three assessed a multifaceted intervention involving training and regulation; and none assessed co‐ordination. All six included studies targeted private for‐profit pharmacy workers in Africa and Asia.
Three studies found that training probably increases sale of oral rehydration solution (one trial in Kenya, 106 pharmacies: RR 3.04, 95% CI 1.37 to 6.75; and one trial in Indonesia, 87 pharmacies: RR 1.41, 95% CI 1.03 to 1.93) and dispensing of anti‐malarial drugs (one trial in Kenya, 293 pharmacies: RR 8.76, 95% CI 0.94 to 81.81); moderate‐certainty evidence.
One study conducted in the Lao People's Democratic Republic shows that regulation of the distribution and sale of registered pharmaceutical products may improve composite pharmacy indicators (one trial, 115 pharmacies: improvements in four of six pharmacy indicators; low‐certainty evidence).
The outcome in three multifaceted intervention studies was the quality of pharmacy practice; including the ability to ask questions, give advice, and provide appropriate treatment. The trials applied regulation, training, and peer influence in sequence; and the study design does not permit separation of the effects of the different interventions. Two trials conducted among 136 pharmacies in Vietnam found that the multifaceted intervention may improve the quality of pharmacy practice; but the third study, involving 146 pharmacies in Vietnam and Thailand, found that the intervention may have little or no effects on the quality of pharmacy practice (low‐certainty evidence).
Only two studies (both conducted in Vietnam) reported cost data, with no rigorous assessment of the economic implications of implementing the interventions in resource‐constrained settings. No study reported data on equity, mortality, morbidity, adverse effects, satisfaction, or attitudes.
Authors' conclusions
Training probably improves quality of care (i.e. adherence to recommended practice), regulation may improve quality of care, and we are uncertain about the effects of co‐ordination on quality of private for‐profit healthcare services in LMICs. The likelihood that further research will find the effect of training to be substantially different from the results of this review is moderate; implying that monitoring of the impact is likely to be needed if training is implemented. The low certainty of the evidence for regulation implies that the likelihood of further research finding the effect of regulation to be substantially different from the results of this review is high. Therefore, an impact evaluation is warranted if government regulation of private for‐profit providers is implemented in LMICs. Rigorous evaluations of these interventions should also assess other outcomes such as impacts on equity, cost implications, mortality, morbidity, and adverse effects.
Plain language summary
Government regulation, training, or co‐ordination of private for‐profit health care in low‐ and middle‐income countries
What is the aim of this review?
The aim of this Cochrane review was to evaluate the effect of government regulation, training, or co‐ordination of private for‐profit health care in low‐ and middle‐income countries.
We collected and analysed all relevant studies to answer this question and included six studies in the review.
Why do governments regulate, train or co‐ordinate private healthcare providers?
In many low‐ and middle‐income countries, the public sector is not able to provide high quality healthcare services to all citizens, and private healthcare providers therefore play a major role. However, there is concern that health care provided by the private sector is not always of high quality and that recommended practices and guidelines are not always followed. Governments therefore use different approaches to ensure that private for‐profit healthcare services meet certain quality standards. This type of government guidance is referred to as 'public stewardship' and can for instance involve training and education for private for‐profit healthcare providers; introduction of regulations where quality standards are set and enforced; and co‐ordination between private for‐profit and public sector healthcare providers, for instance, creating referral systems between the private for‐profit and public sectors.
What happens when governments regulate, train or co‐ordinate private, for‐profit health care providers?
Training In two studies in Kenya and Indonesia, the Ministry of Health offered private drug sellers short training sessions on prescribing and dispensing drugs. These sellers were compared to drug sellers who were not offered training. The studies suggested that training probably improves the quality of healthcare services.
Regulation In one study in the Lao People’s Democratic Republic, the Ministry of Health supervised private pharmacy services in certain districts over a three‐month period, applied sanctions when rules were broken, and offered information about areas needing improvement. These districts were compared to districts without this enhanced supervision. The study suggested that this enhanced regulation may make little or no difference to quality of care.
Training and regulation In three studies in Vietnam and Thailand, private pharmacies in some districts received educational visits as well as visits from pharmacy inspectors to enforce regulations. These districts were compared to districts that did not receive any visits. The studies suggested that these types of visits may improve quality of care.
Co‐ordination The review did not find any eligible study that assessed the effects of co‐ordination on quality of care.
How up‐to‐date is this review?
The review authors searched for studies that had been published up to June 2016.
Summary of findings
Summary of findings for the main comparison. Government training of private for‐profit healthcare providers.
| Training compared to no training for improving quality of care | |||
| Population: Private for‐profit providers of healthcare services Settings: Kenya and Indonesia (1 study) and Kenya (1 study) Intervention: Training Comparison: No training | |||
| Outcomes | Impacts | No of Participants (studies) | Certainty of the evidence (GRADE) |
| Quality of care | Both studies show that training probably improves the quality of healthcare services | 486 pharmacies (2 studies) | ⊕⊕⊕⊝ Moderate* |
| * We downgraded the certainty of evidence by 1 point, because of a moderate risk of selection bias in included studies. GRADE Working Group grades of evidence High certainty: Further research is very unlikely to change our confidence in the estimate of effect. Moderate certainty: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate. Low certainty: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. Very low certainty: We are very uncertain about the estimate. | |||
Summary of findings 2. Government regulation of private for‐profit healthcare providers.
| Regulation compared to no regulation for improving quality of care | |||
| Population: Private for‐profit providers of healthcare services Settings: Lao People's Democratic Republic Intervention: Regulation Comparison: No regulation | |||
| Outcomes | Impacts | No of Participants (studies) | Certainty of the evidence (GRADE) |
| Quality of care | Regulation may improve quality of care. The study observed an increase of 34% in the availability of essential materials for dispensing and an increase of 19% in mean orderliness (including the presence of advertisements, and storage of drugs in their original packaging away from sunlight) in the intervention pharmacies compared to the control pharmacies | 115 pharmacies (1 study) | ⊕⊕⊝⊝ Low* |
| * We downgraded the certainty of the evidence by 2 points, because of a high risk of attrition bias and wide confidence intervals around the effect estimate, ranging from a large benefit to important harm GRADE Working Group grades of evidence High certainty: Further research is very unlikely to change our confidence in the estimate of effect. Moderate certainty: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate. Low certainty: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. Very low certainty: We are very uncertain about the estimate | |||
Summary of findings 3. Government training and regulation of private for‐profit healthcare providers.
| Training and regulation compared to no intervention for improving quality of care | |||
| Population: Private for‐profit providers of healthcare services Settings: Thailand and Vietnam Intervention: Training, regulation, and peer influence Comparison: No intervention | |||
| Outcomes | Impact | No of Participants (studies) | Certainty of the evidence (GRADE) |
| Quality of care | Training and regulation may improve quality of care | 379 pharmacies (3 studies) | ⊕⊕⊝⊝ Low* |
| * We downgraded the certainty of evidence by 2 points because of a high risk of attrition bias and heterogeneity of intervention effects. Two studies found that training, regulation, and peer influence may improve quality of care while the third study found little or no effects in the quality of care with the intervention GRADE Working Group grades of evidence High certainty: Further research is very unlikely to change our confidence in the estimate of effect. Moderate certainty: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate. Low certainty: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. Very low certainty: We are very uncertain about the estimate. | |||
Background
The public health sector in low‐ and middle‐income countries is not always sufficiently well‐equipped and financed to provide high quality health care that is accessible to all citizens (Basu 2012; Lagomarsino 2009). The consequence of this public sector failure has been a proliferation in private providers of healthcare services in most of the countries (Forsberg 2011; Levin 2011; Scott 2011). Governments have a responsibility to ensure the quality of healthcare services delivered by private providers, to expand the coverage of existing private providers, and to rationalise this coverage with that of public sector providers (Waters 2003). Such government guidance is referred to as public stewardship. However, there is a paucity of high quality research evidence on the effects of public stewardship on the quality and accessibility of private for‐profit health care in low‐ and middle‐income countries (Patouillard 2007; Waters 2003); thus the need for this review.
Description of the condition
The private health sector is not homogeneous, but consists of not‐for‐profit and for‐profit as well as formal and informal providers of healthcare services (Basu 2012; Sulzbach 2011). Private not‐for‐profit healthcare providers refer to healthcare organisations that use any surplus revenues to achieve their goals, rather than distributing them as dividends. On the other hand, the private for‐profit sector refers to the part of the economy that is run by individuals and companies for profit and is not state‐controlled. The consequence of the expansion in the private health sector in LMICs is that (poor) communities spend outsized amounts of money for private healthcare services; at times when cheaper public sector alternatives are available (Forsberg 2011; Patouillard 2007). However, the quality of the services provided by the private for‐profit healthcare sector in LMICs is increasingly being questioned (Berendes 2011; Waters 2003).
Description of the intervention
The growing concern regarding the technical failures of health care provided by the private for‐profit sector has led to the development of interventions aimed at addressing these limitations, which simultaneously take advantage of the potential for involving the private for‐profit sector to achieve public health goals. This review assessed the public stewardship of private for‐profit healthcare providers in LMICs. Stewardship can be defined as a function of governments responsible for the welfare of their populations (Veillard 2011). It involves policy guidance to the whole health system, co‐ordination between actors and regulation of different functions, levels and actors in the system, an optimal allocation of resources and accountability towards all stakeholders. Although many actors have an influence on stewardship, there is a central role for the government in ensuring equity, efficiency and sustainability of the health system (Van Olmen 2010). Therefore, stewardship entails oversight and guidance of the whole system; ensuring strategic policy frameworks exist and are combined with effective oversight, coalition‐building, regulation, attention to system‐design and accountability. The stewardship function involves the role of the government in health and its relation to other actors whose activities impact on health. Public stewardship encompasses government policies, regulatory mechanisms, and implementation strategies for ensuring guidance and accountability in which healthcare services are delivered; in order to protect the public interest (WHO 2007). While ultimately it is the responsibility of government, this does not mean all stewardship functions have to be carried out by central ministries of health (WHO 2009).
Various strategies have been proposed for improving the functioning of the private for‐profit health sector in order to increase the quality, availability, and affordability of health care for poor people in LMICs (Lagomarsino 2009; Levin 2011; Patouillard 2007; Waters 2003). These strategies include regulation, contracting‐out, social marketing, franchising, use of vouchers, training, pay for performance, accreditation, and co‐ordination. The strategies use various markers of success which are analysed by their association with differences in performance of intermediate goals or outcomes (Travis 2002). We focused on three types of strategic interventions, namely, regulation, training, and co‐ordination of private for‐profit providers. In the context of this review, regulation refers to the setting and enforcing of standards for the private for‐profit sector; training involves educating and supporting private for‐profit service providers; and co‐ordination entails organising and creating alliances between private for‐profit and public sector healthcare providers. We excluded public stewardship interventions which are already covered by other Cochrane reviews; including social marketing and franchising (Koehlmoos 2009), contracting (Lagarde 2009), and pay for performance (Witter 2012).
How the intervention might work
Regulatory interventions take the form of rules, enforcement systems and sanction mechanisms, and can be applied at the levels of the healthcare provider, organisation, or facility. At the provider level, regulation may include requirements for pre‐service training, continuing education, licensing, and certification. At the organisational or facility level, regulation may aim to control the location of facilities, their registration, prices, and minimum complement of staff or facilities. For example, pharmaceutical market regulation aims to limit the availability of harmful drugs and unregistered products, minimise drug misuse, control the sale of specific drugs through prescriptions, and control drug manufacture and importation. Training interventions may involve formal educational sessions (educational meetings and workshops), vendor‐to‐vendor education, distribution of guidelines, printed educational materials, educational outreach i.e. a personal visit by a trained government official to private for‐profit healthcare providers in their own settings, or audit and feedback i.e. a summary of the performance of private for‐profit providers over a specified period of time given in a verbal or written format; alone or in combination (Forsetlund 2009; Jamtvedt 2006; O'Brien 2007; Patouillard 2007). A wide variety of private for‐profit healthcare sector components could be targeted for training, including physicians, pharmacists, midwives, nurses, and traditional healers. Finally, government co‐ordination of private for‐profit health care ensures harmonised minimum standards for health service delivery across geographic areas and social groups (Waters 2003). For instance, the creation of referral systems between the private for‐profit and public sector and ensuring that health professionals in different health sectors understand their roles in disease management. The ultimate aim of government regulation, training, and co‐ordination of the private for‐profit health sector is to promote equity, better health outcomes, and financial protection (Lagomarsino 2009).
Why it is important to do this review
A systematic review published in 2007 found “evidence that effective public‐private partnerships can increase access, improve equity, and raise quality of health services" (Patouillard 2007). However, using the GRADE approach (Balshem 2011; Guyatt 2008), this evidence on the effectiveness of interventions for working with the private for‐profit sector to improve the utilisation and quality of health services for the poor in low‐ and middle‐income countries was found to be of low certainty (Wiysonge 2008). The implication of the low certainty of the evidence is that further research on this topic is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. It is possible that additional primary studies may have been conducted on this topic. Therefore, we reviewed the currently available evidence on public sector efforts to work with private for‐profit health service providers to improve the quality of existing healthcare services as well as expand and rationalise their coverage (Waters 2003).
Objectives
To assess the effects of public sector regulation, training, or co‐ordination of the private for‐profit health sector in low‐ and middle‐income countries.
Methods
Criteria for considering studies for this review
Types of studies
The studies eligible for inclusion in the review were:
randomised trials, including individually‐randomised and cluster‐randomised trials;
non‐randomised controlled trials i.e. experimental studies in which people are allocated to different interventions using methods that are not random;
interrupted time series studies with at least three measurements before and after introducing the intervention; and
controlled before‐after studies with at least two intervention groups and at least two comparable control groups, with simultaneous data collection (EPOC 2013).
Types of participants
Studies taking place in low‐ and middle‐income countries as defined by the World Bank. All types of health services provided by for‐profit providers were eligible for inclusion in this review.
Types of interventions
Regulation, training, and co‐ordination of any intensity or duration, implemented by the public sector. The control group received no intervention or an alternative intervention.
Types of outcome measures
The outcomes of interest were as follows.
Primary outcomes
Quality of care (defined as adherence to recommended practice or guidelines).
Secondary outcomes
Equity
Mortality or morbidity
Adverse effects (e.g. undesirable impacts on existing public or private services, inappropriate use of services, and distortion in the provision of services)
Satisfaction
Attitudes
Costs of implementing the interventions
Search methods for identification of studies
We developed a sensitive and previously validated search strategy for randomised trials, non‐randomised trials, controlled before‐after studies, and interrupted time series studies combined with relevant medical subject headings (MeSH) and free‐text terms relating to health regulation, training and co‐ordination literature for low‐ and middle‐income countries. We placed no language or date restrictions on the search strategy. We translated the MEDLINE (Ovid) search strategy into the other databases using the appropriate controlled vocabulary.
Electronic searches
We searched the following databases for systematic reviews:
Cochrane Database of Systematic Reviews (CDSR) 2015, Issue 4, part of The Cochrane Library. www.cochranelibrary.com (searched 28 April 2015)
Database of Abstracts of Reviews of Effectiveness (DARE) 2015, Issue 1, part of The Cochrane Library. www.cochranelibrary.com (searched 28 April 2015)
Health Technology Assessment Database (HTA) 2015, Issue 1, part of The Cochrane Library.www.cochranelibrary.com (searched 28 April 2015).
We searched the following databases, with no language or date restrictions, for primary studies:
MEDLINE, Epub Ahead of Print, In‐Process & Other Non‐Indexed Citations, MEDLINE Daily and MEDLINE 1946 to Present, OvidSP (searched 16 June 2016)
Science Citation Index and Social Sciences Citation Index 1987 to present, and Emerging Sources Citation Index 2015 to present, ISI Web of Science (searched 3 May 2016 for papers citing included studies)
Cochrane Central Register of Controlled Trials (CENTRAL), 2015, Issue 3, part of the Cochrane Library. www.cochranelibrary.com (including the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register) (searched 28 April 2015)
Embase 1980 to 2015 Week 17, OvidSP (searched 28 April 2015)
Global Health 1973 to 2015 Week 16, OvidSP (searched 30 April 2015)
WHOLIS, WHO (searched 30 April 2015)
Science Citation Index and Social Sciences Citation Index 1975 to present, ISI Web of Science (searched 30 April 2015)
Health Management, ProQuest (searched 22 November 2013).
Searching other resources
In April 2016 we searched the following databases and websites for eligible studies:
OpenGrey (opengrey.eu)
Grey Literature Report (greylit.org)
World Bank e‐Library (elibrary.worldbank.org)
US National Institutes of Health (NIH) (nih.gov)
United Nations Children's Fund (UNICEF) (unicef.org)
Alliance for Health Policy and Systems Research (who.int/alliance‐hpsr/en)
United States Agency for International Development (USAID) (usaid.gov)
Gavi, The Vaccine Alliance (gavi.org)
Private Healthcare in Developing Countries (ps4h.org)
Population Services International (PSI) (psi.org)
Shops (shopsproject.org)
United Kingdom Department for International Development (gov.uk/government/organisations/department‐for‐international‐development)
Center For Health Market Innovations (healthmarketinnovations.org
World Bank (worldbank.org)
Trial Registries
International Clinical Trials Registry Platform (ICTRP), Word Health Organization (WHO) (who.int/ictrp/en) (searched April 2016)
ClinicalTrials.gov (clinicaltrials.gov) (searched April 2016)
We checked the reference lists of identified reviews (Basu 2012; Berendes 2011; Forsberg 2011; Forsetlund 2009; Levin 2011; Patouillard 2007; Sulzbach 2011; Waters 2003) as well as reference lists of full‐text articles reviewed for inclusion in this review.
Data collection and analysis
Selection of studies
Two authors (Leila Abdullahi and Valantine Ndze, Leila Abdullahi and Charles Wiysonge, or Valantine Ndze and Charles Wiysonge) screened the titles and abstracts of outputs from the searches using a pre‐designed screening guide to identify potentially eligible studies. We retrieved the full text of all publications deemed potentially eligible by at least one of the two authors. The two authors then independently examined each of these for eligibility. Each author compiled a list of studies which he or she believed met the inclusion criteria. Both authors compared the lists and resolved discrepancies by discussion and consensus.
Data extraction and management
The two authors independently extracted descriptive and outcome data from each included study using a pre‐designed data collection form. Both authors compared extracted data, resolving any discrepancies by discussion and consensus, failing which a third author would have arbitrated. One of two authors (Leila Abdullahi and Charles Wiysonge) entered the data into Review Manager (RevMan) 5.3 (RevMan 2014) and the other author performed double checks to ensure that there were no errors in the data entered.
Assessment of risk of bias in included studies
We assessed the risk of bias based on six standard domains:
Sequence generation
Concealment of allocation
Blinded or objective assessment of primary outcome(s)
Incomplete outcome data
Selective outcome reporting; and
Other sources of bias (Higgins 2011a).
We also used three additional criteria specified by the Cochrane Effective Practice and Organisation of Care Group (EPOC) (EPOC 2015):
Similar baseline characteristics
Similar baseline outcome measures
Reliable primary outcome measures; and
Adequate protection against contamination.
For each included study, two authors independently reported their assessment of the risk of bias for each domain (i.e. low, high, or unclear) together with a descriptive summary of the information that influenced their judgment. The two authors compared the results of their independent assessments of the risk of bias and resolved any discrepancies by discussion and consensus. Had the two authors failed to reach an agreement, a third author would have arbitrated.
Measures of treatment effect
We grouped measures of treatment effect based on outcome variables and study designs. We recorded and used estimates of effect from the primary analysis reported by the investigators.
We anticipated that there would be important baseline differences between intervention and control groups and planned to base our primary analyses for trials and controlled before‐after studies on estimates of effect that were adjusted for baseline differences. For dichotomous outcomes we planned to calculate the adjusted risk difference as the difference in adherence after the intervention minus the difference before the intervention. A positive risk difference would indicate that compliance with the recommended practice improved more in the intervention group than in the control group (e.g. an adjusted risk difference of 0.11 would indicate an absolute improvement in compliance with targeted behaviours of 11%). For continuous outcomes we planned to calculate the adjusted change relative to the control group as the post‐intervention difference in means minus the baseline difference in means divided by the baseline control group mean. As with the adjusted risk difference, a positive change would indicate that compliance improved more in the intervention group than in the control group. This is a relative effect rather than an absolute effect; the effect size reflects the baseline performance as well as the change in performance and it is not bound between ‐100% and +100%.
We planned to analyse interrupted time series studies using either a regression analysis with time trends before and after the intervention, which adjusts for auto‐correlation and any periodic changes; or any other technique that adjusts for auto‐correlation and periodic changes. We would present results for the outcomes as changes along two dimensions: change in level and change in slope. Change in level is the immediate effect of the policy and change in slope is the change in the trend from pre‐ to post‐intervention. It reflects the long‐term effect of the intervention.
For all measures we planned to calculate 95% confidence intervals (CI).
Unit of analysis issues
We planned that if investigators reported cluster‐randomised trial data as if the randomisation was performed on the individuals rather than the clusters, we would request the intra‐cluster correlation coefficient from the study authors; failing which we would obtain external estimates of the intra‐cluster correlation coefficient from similar studies or available resources. Once established, we would use the intra‐cluster correlation coefficient to re‐analyse the trial data to obtain approximate correct analyses; as described in the Cochrane Handbook for Systematic Reviews of Interventions (Deeks 2011). We planned to combine the effect estimates and their corrected standard errors from cluster‐randomised trials with those from parallel group designs using the generic inverse variance method (Deeks 2011). If insufficient information was available to control for clustering in this way, we would enter data into RevMan using individuals as the unit of analysis. We would then perform sensitivity analyses to assess the potential bias that may have occurred as a result of the inadequately controlled cluster‐randomised trials. We planned that we would also perform sensitivity analyses if the intra‐cluster correlation coefficients were obtained from external sources to assess the potential biasing effects of inadequately controlled cluster‐randomised trials.
Three included studies were cluster‐randomised trials based on matched pairs of clusters (Chalker 2002; Chuc 2002; Stenson 2001). We did not re‐analyse these data as matching cannot be taken into account in re‐analyses in such studies unless the raw data are available. The studies, however, conducted appropriate analyses of the data, and we have provided the results as reported in the studies. We have re‐analysed the data for the fourth cluster‐randomised trial (Abuya 2009). We did not conduct a meta‐analysis.
Dealing with missing data
We planned that where necessary, we would contact the corresponding authors of included studies to supply any unreported data. If the corresponding author did not respond within one week of our request, we planned to contact other authors (copying the corresponding author). If a study reported outcomes only for participants completing the trial or only for participants who followed the protocol, we planned to contact the authors and ask them to provide additional information to permit us to conduct meta‐analyses by intention‐to‐treat. We would describe missing data and dropouts for each included study in the risk of bias table, and discuss the extent to which the missing data could alter our results. We planned to conduct sensitivity analyses to assess the effect of missing data on our primary meta‐analyses. If we had at least 10 studies in a meta‐analysis, we would have explored the impact of including trials with high levels of missing data in the overall assessment of intervention effects by using sensitivity analyses.
For the current version of the review, we did not contact the primary study authors for missing data. We identified levels of attrition for included trials and performed analyses for reported outcomes. All participants were analysed in the group to which they were allocated, regardless of whether or not they received the allocated intervention. We assumed that missing participants did have the outcome of interest, and did not conduct sensitivity analyses imputing values for the outcome status of missing participants.
Assessment of heterogeneity
Given the variation found across studies in relation to the interventions, study design and outcome measures, we have not conducted a meta‐analysis of study results. A statistical assessment of heterogeneity of results was therefore not done.
If we found homogeneous studies of similar interventions that reported similar outcomes, we would have conducted a meta‐analysis, examined statistical heterogeneity between study results using the Chi2 test of homogeneity (with significance defined at the alpha‐level of 10%) (Deeks 2011), and quantified any statistical heterogeneity between study results using the I2 statistic (Higgins 2003).
Assessment of reporting biases
We employed strategies to search for and include relevant unpublished studies in order to reduce possible publication bias. These strategies included searching the grey literature and prospective trial registration databases to overcome time‐lag bias.
Data synthesis
Due to important heterogeneity between studies, a pooled statistical analysis of the results was not possible. Therefore we did a qualitative analysis based on intervention and outcome measures. If we had identified two or more clinically homogenous studies with similar interventions and comparison groups that reported similar outcome measures, we would have used meta‐analysis to estimate the overall effect across those studies. We would have calculated all overall effects, if applicable, using inverse variance methods.
We used the GRADE approach to summarise the certainty of the evidence for each outcome (Guyatt 2008).The GRADE approach results in an assessment of the certainty of a body of evidence as high, moderate, low, or very low. High certainty evidence implies that "further research is very unlikely to change our confidence in the estimate of effect". Moderate certainty evidence means that "further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate". Evidence is considered of low certainty if "further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate", and very low quality if "we have very little confidence in the effect estimate" (Balshem 2011).
Subgroup analysis and investigation of heterogeneity
We stratified analysis by type of intervention. We did not find any studies that were similar enough to combine in a meta‐analysis and, therefore, we did not conduct any subgroup analyses.
Sensitivity analysis
If we had found 10 or more studies that were similar enough that it would be sensible to combine them in a meta‐analysis, we would have conducted sensitivity analyses to investigate the robustness of the results to risk of bias (i.e. omitting any studies with high risk of bias) and method of meta‐analysis (i.e. random‐effects versus fixed‐effect).
Results
Description of studies
Results of the search
Our search yielded 20,177 titles and abstracts. After removing 1,419 duplicates , we screened 18,758 records; 18,708 of which were not relevant. We reviewed the 50 potential eligible articles for inclusion. Six of these studies met our inclusion criteria (Abuya 2009; Chalker 2002; Chalker 2005; Chuc 2002; Ross‐Degnan 1996; Stenson 2001), and we excluded 39 for reasons given in the Characteristics of excluded studies. Five studies were identified after the review was submitted and are awaiting assessment (see Characteristics of studies awaiting classification). We present the search and selection of studies for this review in Figure 1.
1.
Study flow diagram
Included studies
We included six randomised trials on regulation and training of private for‐profit healthcare providers in low‐ and middle‐income countries (Abuya 2009; Chalker 2002; Chalker 2005; Chuc 2002; Ross‐Degnan 1996; Stenson 2001). One study (Ross‐Degnan 1996) had two components; one being a randomised trial, and the other a non‐randomised trial. Five studies assessed training (Abuya 2009; Chalker 2002; Chalker 2005; Chuc 2002; Ross‐Degnan 1996), four studies assessed regulation (Chalker 2002; Chalker 2005; Chuc 2002; Stenson 2001), and no study assessed co‐ordination.
Description of interventions
Two studies (Abuya 2009; Ross‐Degnan 1996) assessed only training interventions (N = 486 pharmacies), one study (Stenson 2001) assessed regulation only (N = 115 pharmacies), and three studies (Chalker 2002; Chalker 2005; Chuc 2002) had a multifaceted intervention which combined training, regulation, and peer influence (N = 379 pharmacies). All six studies targeted private pharmacy workers or drug retailers.
Training
The intervention in the two 'training‐only' studies consisted of short‐duration training sessions of one or two days in Kenya (Abuya 2009; Ross‐Degnan 1996) and Indonesia (Ross‐Degnan 1996). In both studies drug sellers were trained on prescription and dispensing of drugs, and surrogate patients (i.e. simulated clients) were used to assess the effects of the intervention on the quality of care provided by the trained retailers. The training was provided by the Ministry of Health in each country.
Regulation
One study (Stenson 2001) assessed regulation only (N = 115 pharmacies). The regulatory intervention involved three‐month intensive supervision of pharmacy services in the Lao People's Democratic Republic, applying sanctions when rules were violated and providing up‐to‐date regulatory documents and information about particular areas needing improvements (Stenson 2001). The study compared districts with active regulation to districts with only "regular supervision". The 'regular supervision' intervention package was implemented in the way and speed that would have taken place in the absence of the study. The aim was to let the control districts follow their natural course. The intervention was provided by the Ministry of Health with assistance from the United Nations Children's Fund (Stenson 2001).
Multifaceted intervention
Three studies (Chalker 2002; Chalker 2005; Chuc 2002) had a multifaceted intervention which combined training, regulation, and peer influence (N = 379 pharmacies). Each intervention lasted three months, with a gap of four months before the next intervention. The quality of practice after the intervention was assessed through simulated clients. Two studies (Chalker 2002; Chuc 2002) were performed in Hanoi (Vietnam) with the intervention delivered by the Hanoi Health Bureau and the Hanoi Pharmacy Association. One study (Chalker 2005) was performed in both Hanoi (Vietnam) and Bangkok (Thailand); with the intervention delivered by the Hanoi Health Bureau and the Hanoi Pharmacy Association in Vietnam and the Bangkok Health System Research Institute and the Community Pharmacy Association in Thailand. The studies compared pharmacies with the multifaceted intervention to pharmacies without any intervention. All pharmacists who received the multifaceted intervention received all three interventions as a set. Enforcement regulation was performed by pharmacy inspectors while the educational intervention consisted of educational visits by senior researchers. Peer influence involved a number of group leaders and representatives of the pharmacy associations, who attended seminars where the research group informed them about the peer influence strategy and reviewed management.
Co‐ordination
We did not identify any study that assessed the effects of government co‐ordination of private for‐profit healthcare providers, such as the creation of referral systems between the private for‐profit and public sectors.
Description of outcomes
All six included studies reported on change in quality of care. The latter was measured using different dimensions in the different studies. One study (Stenson 2001) that assessed only regulation measured quality of care through change in the quality of private pharmacy practices (using "pharmacy indicators"). The two studies (Abuya 2009; Ross‐Degnan 1996) that assessed only training measured quality of care through correct management of childhood malaria or diarrhoea respectively. The three studies with multifaceted interventions (Chalker 2002; Chalker 2005; Chuc 2002) measured quality of care through change in the correct management of tracer conditions, antibiotic dispensing practices, and correct symptomatic treatment of sexually transmitted infections respectively. Two studies (Chuc 2002; Chalker 2002) reported the cost of implementing the interventions.
No studies reported data on equity, mortality, morbidity, adverse effects, satisfaction, or attitudes.
Excluded studies
We excluded 34 studies (Akoria 2008; Akol 2014; Ali 2011; Ali 2012; Andrianasolo 2012; Bhat 1996; Bojalil 1999; Chakraborty 2000; Contiades 2007; Dholakia 2013; Farsi 1999; Fernandes 2009; Goodman 2007; Grundy 2010; Guiscafre 2001; Harrison 2000; Hongoro 2000; Kangwana 2011; Khan 2006; Kumaranayake 2000; Maiga 2010; Marsh 2004; Minh 2013; Murugesan 2009; Nsimba 2007; Obua 2004; Okonofua 2003; Osterholt 2009; Rutta 2011; Stenson 2001b; Syhakhang 2001; Tavrow 2003; Tumwikirize 2004; Willey 2014) for reasons given in the table of Characteristics of excluded studies. The most common reason for exclusion was an ineligible study design.
Risk of bias in included studies
We have summarised our judgements about the risk of bias in each included study in Figure 2 and Figure 3.
2.
Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.
3.
Risk of bias summary: review authors' judgements about each risk of bias item for each included study.
Allocation
The methods for generation of the randomisation sequence and allocation concealment were unclear in all six studies (Abuya 2009; Chalker 2002; Chalker 2005; Chuc 2002; Ross‐Degnan 1996; Stenson 2001).
Blinding
Outcome assessors were blinded in two studies (Abuya 2009; Ross‐Degnan 1996), but there was no description of blinding in the rest (Chalker 2002; Chalker 2005; Chuc 2002; Stenson 2001).
Incomplete outcome data
Loss to follow up was moderate to high in the six studies.
Selective reporting
Selective reporting was categorised as unclear since we had no access to the study protocols.
Other potential sources of bias
All studies reported similar baseline characteristics among the intervention and control groups. Two studies (Chalker 2002; Chalker 2005) reported small differences in the outcome measures at baseline while no description was provided on baseline outcome measures in the rest of the studies. In one cluster‐randomised controlled trial there was some degree of contamination in a cluster that was meant to be a control cluster (Abuya 2009), but none of the other studies reported contamination of control clusters with the interventions assessed. We did not have any evidence that other biases were introduced into the remaining studies, over and above the ones reported above.
Effects of interventions
See: Table 1; Table 2; Table 3
Primary outcome
All six included studies reported effects on quality of care; although quality of care was measured using different indicators. Three studies focused on improving treatment of childhood illnesses such as acute respiratory infection, malaria, or diarrhoea (Abuya 2009; Chalker 2002; Ross‐Degnan 1996); one assessed the quality of treatment of sexually transmitted infections (Chuc 2002); and two assessed antibiotic dispensing practices (Chalker 2005; Chuc 2002).
Training
Each of the two studies that assessed training alone (Abuya 2009; Ross‐Degnan 1996) observed improvements in quality of care. The study conducted in Kenya and Indonesia (Ross‐Degnan 1996) showed an overall improvement in the management of diarrhoea among counter attendants in the intervention pharmacies compared to the controls. The sale of oral rehydration solution in the intervention pharmacies increased by 204% in Kenya (1 trial, 106 pharmacies; RR 3.04, 95%CI 1.37 to 6.75: Analysis 1.1) and 41% in Indonesia (1 trial, 87 pharmacies; RR 1.41, 95%CI 1.03 to 1.93: Analysis 1.1); compared to control pharmacies. In Kenya (Abuya 2009), correct prescription and dispensing of anti‐malarial drugs improved substantially (1 trial, 293 pharmacies; RR 8.76, 95% CI 0.94 to 81.81: Analysis 1.2). Using the GRADE approach (Balshem 2011) we judged the certainty of evidence on the effects of training on quality of care as moderate (Table 1). Although the findings were consistent across the studies, we downgraded the evidence because of a moderate risk of bias in the included studies (Figure 3).
1.1. Analysis.
Comparison 1 Training, Outcome 1 Quality of care.
1.2. Analysis.
Comparison 1 Training, Outcome 2 Quality of care.
Regulation
In the 'regulation only' study, conducted in the Lao People's Democratic Republic, the distribution and selling of registered pharmaceutical products was regulated in order to protect consumers against unfair practices (Stenson 2001). The study observed an increase of 34% in the availability of essential materials for dispensing and an increase of 19% in mean orderliness (including the presence of advertisements, and storage of drugs in their original packaging away from sunlight) in the intervention pharmacies compared to the control pharmacies. "Information given to customers increased from 35% to 51% and the mixing of different drugs in the same package went down from 17% to 9%. The pharmacies in the active intervention districts showed greater improvements for four of the six pharmacy indicators" (Stenson 2001) . Using the GRADE approach (Balshem 2011), we judged the certainty of the evidence on the effects of the regulatory interventions on quality of care as low (Table 2). Our main concern with the evidence was the imprecision of the effect estimate.
Multifaceted intervention
In the multi‐faceted intervention studies (Chalker 2002; Chalker 2005; Chuc 2002), the interventions (regulation, training, and peer influence) were applied in a sequence, and the study design does not permit separation of the effects of the different interventions. The three studies provided inconsistent results regarding the effect of the multiple interventions on quality of pharmacy practice; including the ability to ask questions, give advice, and provide appropriate treatment for four tracer conditions (acute respiratory conditions, malaria, diarrhoea, and sexually transmitted infections). In one study conducted in Vietnam (Chalker 2002), knowledge and reported practice among drug sellers improved for three of the four tracer conditions in intervention pharmacies compared to control pharmacies. The second study conducted in Vietnam, (Chuc 2002), found that the intervention pharmacies improved substantially in all tracer conditions compared to the control pharmacies. Chalker 2005 was conducted in both Vietnam and Thailand and had mixed results. Improvements were observed in Vietnam in the dispensers’ behaviour for all tracer conditions, but in Thailand improvements occurred in only one of the tracer conditions. We judged the certainty of the evidence on the effects of the multifaceted intervention as low (Table 3), because of concerns regarding inconsistency of findings and high risk of bias in the included studies.
Secondary outcomes
Two multifaceted intervention studies reported the cost of the interventions (Chalker 2002; Chuc 2002). Chalker 2002 reported the cost incurred for the three interventions in 30 pharmacies to be USD 5700. The Chuc 2002 study reported that the costs of treating four tracer conditions increased for both intervention and control pharmacies.
No study reported data on equity, mortality, morbidity, adverse effects, satisfaction, or attitudes.
Discussion
Summary of main results
Our comprehensive search of the literature identified 20,177 records, from which six randomised controlled trials fulfilled our inclusion criteria. In two studies in Kenya and Indonesia, the Ministry of Health offered private drug sellers short training sessions on prescribing and dispensing drugs. These sellers were compared to drug sellers who were not offered training. The studies suggest that training probably improves the quality of healthcare services. In one study in the Lao People’s Democratic Republic, the Ministry of Health supervised private pharmacy services in certain districts over a three‐month period, applied sanctions when rules were broken, and offered information about areas needing improvement. These districts were compared to districts without this enhanced supervision. The study suggests that this enhanced regulation may improve quality of care. In three studies in Vietnam and Thailand, private pharmacies in some districts received educational visits as well as visits from pharmacy inspectors to enforce regulations. These districts were compared to districts that did not receive any visits. The studies suggest that these types of visits may provide mixed results. The review did not find any eligible study that assessed the effects of co‐ordination on quality of care.
Overall completeness and applicability of evidence
Despite the large number of records obtained in our literature search, only six studies with moderate to high risk of bias met our inclusion criteria. All studies were conducted in Africa and Asia; and the results may be applicable to low‐ and middle‐income countries in other continents.
Health worker availability in the public sector is a key barrier to strengthening health systems in LMICs. Effective government interventions to expand the coverage of private for‐profit healthcare providers and rationalise access to their services with that of public sector providers could strengthen health systems in LMICs. However, countries considering the implementation of public stewardship interventions need to assess the human and financial resource capacity of the public sector to properly supervise private providers.
All the studies covered only services of private for‐profit pharmacists. The findings may not be directly transferable to other cadres of private healthcare providers. Therefore, there is a need for studies on other private sector components such as private hospitals, physicians, midwives, nurses, and traditional healers.
There were no studies that reported data on equity, mortality, morbidity, adverse effects, satisfaction, or attitudes. If training or regulatory interventions are directed at providers that serve disadvantaged populations they could help to decrease inequity. However, intervention effects could vary across settings, for example between rural and urban areas, because of the distribution of private providers in these different areas. Expanding the coverage of private for‐profit providers could reduce inequity if, for example, access to the private sector is available where access to the public sector is limited. However, if private for‐profit providers are unavailable in underserved areas, expanding access to private providers may increase inequity between urban and rural areas.
There was no rigorous evaluation of the cost implications of implementing the interventions, thus this review does not provide evidence on investment in private for‐profit providers on quality of care in low‐ and middle‐ income countries. The structure and specific tasks associated with a particular public stewardship function will determine the costs. Given these uncertainties, implementation of public sector regulation, training or co‐ordination of private for‐profit healthcare providers should be accompanied by a robust framework for monitoring the costs and impacts of the interventions.
Much of the currently available literature on training and regulation of private for‐profit providers in low‐ and middle‐income countries is descriptive rather than evaluative, detailing experiences that may have great potential without rigorously testing their effectiveness (Berendes 2011; Patouillard 2007; Waters 2003; Wiysonge 2008). Well‐designed studies evaluating public stewardship functions are therefore needed before these are implemented on a large scale in low‐income countries.
Certainty of the evidence
Using the GRADE approach, we judged the certainty of evidence on the effects of training interventions on quality of care as moderate; which implies that “further research is likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate”. We rated the certainty of evidence on regulation and the multifaceted intervention as low, which means that “further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate” (Balshem 2011). Our main concerns with the evidence were limitations of the included studies, wide confidence intervals around the effect estimates, and heterogeneity of intervention effects.
Potential biases in the review process
We minimised potential biases in the review process by adhering to Cochrane guidelines (Higgins 2011). We conducted comprehensive searches without limiting the searches to a specific language. Two authors independently assessed study eligibility, extracted data, and assessed the risk of bias in each included study.
Agreements and disagreements with other studies or reviews
Despite the widespread availability of private healthcare services in low‐ and middle‐income countries, there is a shortage of systematic reviews that have assessed interventions showing how governments have worked with the private for‐profit providers to achieve public health goals (Brugha 1998; Levin 2011; Patouillard 2007; Peters 2004; Waters 2003). To the best of our knowledge, our review is the most comprehensive and up‐to‐date assessment of the evidence on the effects of training, regulation, and co‐ordination of private‐for‐profit health care in low‐ and middle‐income countries.
In 2003 Hugh Waters and colleagues published a review that assessed the evidence available concerning public sector efforts to work with private health service providers and other components of the private sector, in order to both improve the quality of their services and to rationalise and expand their coverage (Waters 2003). The review focused on interventions aimed at regulating, contracting, financing, social marketing, training, co‐ordinating, and informing private providers in low‐ and middle‐income countries. The review authors searched Pubmed and Popline databases, and reference lists of included articles, for both published and unpublished literature from 1980 onwards. They included 42 studies including six ‘controlled’ trials comparing results in two or more groups; 10 studies with a pre‐post evaluative component, but no comparison group; four cross‐sectional studies; and 22 descriptive case studies. Waters 2003 found that although governments are gaining experience in using the tools of contracting, regulating, financial incentives, training, co‐ordinating, and informing to influence the private sector, the evidence on the effectiveness of these interventions remains weak.
In 2007 Edith Patouillard and co‐workers published a related review, which assessed the effectiveness of interventions on working with the private for‐profit sector to improve utilisation of quality health services by the poor in low‐ and middle‐income countries (Patouillard 2007). Interventions of interest to the review authors included social marketing, use of vouchers, pre‐packaging of drugs, franchising, training, regulation, accreditation, and contracting‐out. They conducted a comprehensive search of peer‐reviewed and grey literature for eligible studies; focusing on studies which evaluated the impact of interventions on utilisation or quality of services, or both, and which provided information on the socioeconomic status of the beneficiary populations. The review authors identified 52 eligible studies; five provided data on the average socioeconomic status of recipient communities and five provided data on the distribution of benefits across socioeconomic groups. Patouillard 2007 concluded that it is not possible to prove from the available literature that private sector interventions benefit the poor and improve equity. However, they argue that the fact that many such interventions have operated successfully in relatively poor settings indicates that the interventions do benefit the poor. The authors went on to recommend that better evidence of the equity impact of interventions working with the private sector is needed for more robust conclusions to be drawn.
There are marked differences between these two previous reviews (Patouillard 2007; Waters 2003) and the current systematic review. Although the authors of the two previous reviews conducted literature searches that were relatively comprehensive, they do not explicitly say whether they undertook duplicate study selection and data extraction and do not report reliable criteria for assessing the risk of bias in included studies. They do not provide appropriate description of the characteristics of included studies and do not seem to synthesise data from included studies using reliable methods. In addition, much of the data reported in the reviews is descriptive rather than evaluative, detailing experiences that may have great potential without rigorously testing their effectiveness. Most of the studies included in the reviews were not set up as research projects (Patouillard 2007; Waters 2003). Furthermore, the two reviews included studies in which the stewardship functions were carried out by the public sector as well as those in which the functions were carried out by non‐governmental organisations (NGO). We acknowledge that while ultimately public stewardship is the responsibility of government, this does not mean all stewardship functions have to be carried out by the public sector. While we recognise the importance of private sector interventions implemented by NGOs, their characteristics and incentives are likely to differ from those implemented by government. We included one study in our review (Abuya 2009), which was published after publication of the two previous reviews (Patouillard 2007; Waters 2003).
To the best of our knowledge, our review is the most comprehensive and up‐to‐date assessment of the evidence on the effects of training, regulation, and co‐ordination of private for‐profit healthcare providers in low‐ and middle‐income countries. Some recent Cochrane reviews have assessed public stewardship interventions not covered in our review; such as social marketing and franchising (Koehlmoos 2009), contracting (Lagarde 2009), and pay for performance (Witter 2012). Koehlmoos 2009 did not find any eligible studies that assessed the effects of social franchising on access to, and the quality of, health services in low‐ and middle‐income countries. We are therefore uncertain about the effects of social franchising as a public stewardship function. Lagarde 2009 examined the effects of contracting out and included three studies, all conducted in low‐ and middle‐income countries. The review found that contracting out services to non‐state not‐for‐profit providers may increase access to and utilisation of health services, improve patient outcomes, and reduce household health expenditures. None of the three included studies presented evidence on whether contracting out was more effective than making a similar investment in the public sector. Witter 2012 found nine studies that assessed the effects of pay‐for‐performance schemes on the provision of healthcare and health outcomes in low‐ and middle‐income countries. The review found that it is uncertain whether pay‐for‐performance improves provider performance, the utilisation of services, patient outcomes or resource use in low‐ and middle‐income countries. Unintended effects of pay‐for‐performance schemes might include adverse selection (for example, excluding high‐risk people from care in order to obtain better performance), deception (i.e. inaccurate or false reporting), and distortion (i.e. ignoring important tasks that are not rewarded with incentives).
Authors' conclusions
Implications for practice.
This review provides different levels of strength for the currently available evidence on the effectiveness of the three public stewardship interventions. The finding that training probably improves quality of care implies that monitoring of the impact is likely to be needed and an impact evaluation may be warranted if government training of private for‐profit providers is implemented in low‐ and middle‐income countries. The low certainty of the evidence for regulation implies that an impact evaluation is warranted if government regulation of private for‐profit providers is implemented in low‐ and middle‐income countries. We found no studies on the effects of government co‐ordination of private providers.
Implications for research.
Rigorous evaluations of the interventions assessed in this review (as well as other public stewardship interventions) should assess cost implications, patient outcomes, and impacts on equity; in addition to quality of care. Given that there was no evidence on the impact of the interventions on equity, the challenge for the future is to design evaluations and report results in ways that can assess equity clearly, and indicate how equity can be enhanced.
Acknowledgements
We are grateful to the editorial base of the Norwegian Satellite of the Cochrane Effective Practice and Organisation of Care Group for assistance in the preparation of the review. In particular, we gratefully acknowledge Ms Marit Johansen for assistance in developing and implementing the search strategy, and Drs Andy Oxman, Susan Munabi‐Babigumira and Gabriel Rada for advice and support at various stages of the preparation of the review.
We would also like to thank the following editors, peer referees and others who provided comments to improve the review: Cristian Herrera, John Chalker, Newton Opiyo, Jan Odgaard‐Jensen; Claire Glenton for preparing the plain language summary, and Denise Mitchell for copy‐editing the review.
We are very grateful to Dr Hassan Mahomed who contributed to the writing of the protocol of the review (Abdullahi 2012).
Charles S Wiysonge's work is partly supported by Stellenbosch University, the South African Medical Research Council, the National Research Foundation of South Africa, and the Effective Health Care Research Consortium (EHCRC). The EHCRC is funded by UK aid from the UK Government for the benefit of developing countries (Grant: 5242). The views expressed in this publication do not necessarily reflect UK government policy.
The Norwegian Satellite of the Effective Practice and Organisation of Care (EPOC) Group receives funding from the Norwegian Agency for Development Cooperation (Norad), via the Norwegian Institute of Public Health to support review authors in the production of their reviews.
Appendices
Appendix 1. Search strategies
CDSR and CENTRAL, Cochrane Library
| ID | Search | Hits |
| #1 | MeSH descriptor: [Public‐Private Sector Partnerships] this term only | 8 |
| #2 | MeSH descriptor: [Private Sector] this term only | 41 |
| #3 | MeSH descriptor: [Private Practice] this term only | 84 |
| #4 | MeSH descriptor: [Hospitals, Private] this term only | 16 |
| #5 | MeSH descriptor: [Privatization] this term only | 2 |
| #6 | privat*:ti,ab | 2190 |
| #7 | #2 or #3 or #4 or #5 or #6 | 2245 |
| #8 | MeSH descriptor: [Public Sector] this term only | 51 |
| #9 | MeSH descriptor: [Public Policy] this term only | 53 |
| #10 | MeSH descriptor: [Health Policy] this term only | 338 |
| #11 | [mh ^"state medicine"] | 460 |
| #12 | MeSH descriptor: [State Dentistry] this term only | 7 |
| #13 | MeSH descriptor: [Health Care Reform] this term only | 34 |
| #14 | MeSH descriptor: [Health Planning] this term only | 64 |
| #15 | MeSH descriptor: [Social Control, Formal] this term only | 30 |
| #16 | MeSH descriptor: [Law Enforcement] this term only | 32 |
| #17 | MeSH descriptor: [Government] 2 tree(s) exploded | 813 |
| #18 | MeSH descriptor: [Government Programs] this term only | 30 |
| #19 | MeSH descriptor: [Government Regulation] this term only | 14 |
| #20 | MeSH descriptor: [Facility Regulation and Control] this term only | 3 |
| #21 | MeSH descriptor: [Policy Making] this term only | 47 |
| #22 | MeSH descriptor: [Jurisprudence] this term only | 75 |
| #23 | MeSH descriptor: [Mandatory Reporting] this term only | 10 |
| #24 | MeSH descriptor: [Politics] this term only | 40 |
| #25 | MeSH descriptor: [Legislation as Topic] this term only | 8 |
| #26 | MeSH descriptor: [Legislation, Hospital] this term only | 0 |
| #27 | MeSH descriptor: [Legislation, Medical] this term only | 5 |
| #28 | MeSH descriptor: [Legislation, Nursing] this term only | 0 |
| #29 | MeSH descriptor: [Legislation, Pharmacy] this term only | 5 |
| #30 | MeSH descriptor: [Legislation, Drug] this term only | 15 |
| #31 | [mh ^"legislation, dental"] | 0 |
| #32 | (public* or stewardship* or governance or governing or coordinat* or co next ordinat* or legislat* or regulat* or government* or law or laws or act or acts or policy or policies or politics or reform* or control* or supervis* or monitor*):ti,ab | 356001 |
| #33 | #8 or #9 or #10 or #11 or #12 or #13 or #14 or #15 or #16 or #17 or #18 or #19 or #20 or #21 or #22 or #23 or #24 or #25 or #26 or #27 or #28 or #29 or #30 or #31 or #32 | 357053 |
| #34 | MeSH descriptor: [Physician's Practice Patterns] this term only | 1104 |
| #35 | MeSH descriptor: [Nurse's Practice Patterns] this term only | 62 |
| #36 | MeSH descriptor: [Dentist's Practice Patterns] this term only | 20 |
| #37 | MeSH descriptor: [Health Knowledge, Attitudes, Practice] this term only | 3833 |
| #38 | MeSH descriptor: [Malpractice] this term only | 13 |
| #39 | MeSH descriptor: [Professional Impairment] this term only | 0 |
| #40 | MeSH descriptor: [Physician Impairment] this term only | 4 |
| #41 | MeSH descriptor: [Medical Errors] this term only | 119 |
| #42 | MeSH descriptor: [Diagnostic Errors] this term only | 258 |
| #43 | MeSH descriptor: [Professional Competence] this term only | 211 |
| #44 | [mh ^"medication errors"] | 228 |
| #45 | [mh ^"clinical competence"] | 1999 |
| #46 | (competence or practice next pattern* or malpractice or mal next practice or error*):ti,ab | 9019 |
| #47 | #34 or #35 or #36 or #37 or #38 or #39 or #40 or #41 or #42 or #43 or #44 or #45 or #46 | 15853 |
| #48 | MeSH descriptor: [Education] this term only | 498 |
| #49 | MeSH descriptor: [Competency‐Based Education] this term only | 67 |
| #50 | MeSH descriptor: [Education, Public Health Professional] this term only | 3 |
| #51 | MeSH descriptor: [Education, Medical] this term only | 300 |
| #52 | MeSH descriptor: [Education, Medical, Continuing] this term only | 638 |
| #53 | MeSH descriptor: [Education, Nursing] this term only | 117 |
| #54 | MeSH descriptor: [Education, Nursing, Continuing] this term only | 249 |
| #55 | MeSH descriptor: [Education, Dental] this term only | 110 |
| #56 | MeSH descriptor: [Education, Dental, Continuing] this term only | 17 |
| #57 | MeSH descriptor: [Education, Pharmacy] this term only | 36 |
| #58 | [mh ^"education, pharmacy, continuing"] | 27 |
| #59 | (educat* or train or training or trained or colloquium* or conference* or course* or lecture* or meeting* or seminar* or support* or symposi* or workshop*):ti,ab | 144494 |
| #60 | #48 or #49 or #50 or #51 or #52 or #53 or #54 or #55 or #56 or #57 or #58 or #59 | 144875 |
| #61 | MeSH descriptor: [Delivery of Health Care] this term only | 751 |
| #62 | MeSH descriptor: [Quality of Health Care] this term only | 933 |
| #63 | MeSH descriptor: [Quality Assurance, Health Care] this term only | 741 |
| #64 | MeSH descriptor: [Quality Improvement] this term only | 216 |
| #65 | MeSH descriptor: [Total Quality Management] this term only | 172 |
| #66 | MeSH descriptor: [Outcome and Process Assessment (Health Care)] this term only | 2034 |
| #67 | MeSH descriptor: [Outcome Assessment (Health Care)] this term only | 5357 |
| #68 | MeSH descriptor: [Process Assessment (Health Care)] this term only | 129 |
| #69 | MeSH descriptor: [Guideline Adherence] this term only | 748 |
| #70 | MeSH descriptor: [Benchmarking] this term only | 100 |
| #71 | MeSH descriptor: [Standard of Care] this term only | 91 |
| #72 | [mh ^"reference standards"] | 357 |
| #73 | (best next practice or quality or standard* or benchmark* or adherence or requirement*):ti,ab | 129505 |
| #74 | #61 or #62 or #63 or #64 or #65 or #66 or #67 or #68 or #69 or #70 or #71 or #72 or #73 | 136459 |
| #75 | (Africa or Asia or Caribbean or "West Indies" or "South America" or "Latin America" or "Central America"):ti,ab,kw | 5159 |
| #76 | (Afghanistan or Albania or Algeria or Angola or Antigua or Barbuda or Argentina or Armenia or Armenian or Aruba or Azerbaijan or Bahrain or Bangladesh or Barbados or Benin or Byelarus or Byelorussian or Belarus or Belorussian or Belorussia or Belize or Bhutan or Bolivia or Bosnia or Herzegovina or Hercegovina or Botswana or Brasil or Brazil or Bulgaria or "Burkina Faso" or "Burkina Fasso" or "Upper Volta" or Burundi or Urundi or Cambodia or "Khmer Republic" or Kampuchea or Cameroon or Cameroons or Cameron or Camerons or "Cape Verde" or "Central African Republic" or Chad or Chile or China or Colombia or Comoros or "Comoro Islands" or Comores or Mayotte or Congo or Zaire or "Costa Rica" or "Cote d'Ivoire" or "Ivory Coast" or Croatia or Cuba or Cyprus or Czechoslovakia or "Czech Republic" or Slovakia or "Slovak Republic"):ti,ab,kw | 10175 |
| #77 | (Djibouti or "French Somaliland" or Dominica or "Dominican Republic" or "East Timor" or "East Timur" or "Timor Leste" or Ecuador or Egypt or "United Arab Republic" or "El Salvador" or Eritrea or Estonia or Ethiopia or Fiji or Gabon or "Gabonese Republic" or Gambia or Gaza or Georgia or Georgian or Ghana or "Gold Coast" or Greece or Grenada or Guatemala or Guinea or Guam or Guiana or Guyana or Haiti or Honduras or Hungary or India or Maldives or Indonesia or Iran or Iraq or "Isle of Man" or Jamaica or Jordan or Kazakhstan or Kazakh or Kenya or Kiribati or Korea or Kosovo or Kyrgyzstan or Kirghizia or "Kyrgyz Republic" or Kirghiz or Kirgizstan or "Lao PDR" or Laos or Latvia or Lebanon or Lesotho or Basutoland or Liberia or Libya or Lithuania):ti,ab,kw | 12314 |
| #78 | (Macedonia or Madagascar or "Malagasy Republic" or Malaysia or Malaya or Malay or Sabah or Sarawak or Malawi or Nyasaland or Mali or Malta or "Marshall Islands" or Mauritania or Mauritius or "Agalega Islands" or Mexico or Micronesia or "Middle East" or Moldova or Moldovia or Moldovian or Mongolia or Montenegro or Morocco or Ifni or Mozambique or Myanmar or Myanma or Burma or Namibia or Nepal or "Netherlands Antilles" or "New Caledonia" or Nicaragua or Niger or Nigeria or "Northern Mariana Islands" or Oman or Muscat or Pakistan or Palau or Palestine or Panama or Paraguay or Peru or Philippines or Philipines or Phillipines or Phillippines or Poland or Portugal or "Puerto Rico"):ti,ab,kw | 5924 |
| #79 | (Romania or Rumania or Roumania or Russia or Russian or Rwanda or Ruanda or "Saint Kitts" or "St Kitts" or Nevis or "Saint Lucia" or "St Lucia" or "Saint Vincent" or "St Vincent" or Grenadines or Samoa or "Samoan Islands" or "Navigator Island" or "Navigator Islands" or "Sao Tome" or "Saudi Arabia" or Senegal or Serbia or Montenegro or Seychelles or "Sierra Leone" or Slovenia or "Sri Lanka" or Ceylon or "Solomon Islands" or Somalia or Sudan or Suriname or Surinam or Swaziland or Syria or Tajikistan or Tadzhikistan or Tadjikistan or Tadzhik or Tanzania or Thailand or Togo or "Togolese Republic" or Tonga or Trinidad or Tobago or Tunisia or Turkey or Turkmenistan or Turkmen or Uganda or Ukraine or Uruguay or USSR or "Soviet Union" or "Union of Soviet Socialist Republics" or Uzbekistan or Uzbek or Vanuatu or "New Hebrides" or Venezuela or Vietnam or "Viet Nam" or "West Bank" or Yemen or Yugoslavia or Zambia or Zimbabwe or Rhodesia):ti,ab,kw | 7359 |
| #80 | (developing or less* next developed or "under developed" or underdeveloped or "middle income" or low* next income or underserved or "under served" or deprived or poor*) next (countr* or nation* or population* or world):ti,ab,kw | 3266 |
| #81 | (developing or less* next developed or "under developed" or underdeveloped or "middle income" or low* next income) next (economy or economies):ti,ab,kw | 20 |
| #82 | low* next (gdp or gnp or "gross domestic" or "gross national"):ti,ab,kw | 31 |
| #83 | (low near/3 middle near/3 countr*):ti,ab,kw | 293 |
| #84 | (lmic or lmics or "third world" or "lami country" or "lami countries"):ti,ab,kw | 75 |
| #85 | ("transitional country" or "transitional countries"):ti,ab,kw | 2 |
| #86 | #75 or #76 or #77 or #78 or #79 or #80 or #81 or #82 or #83 or #84 or #85 | 38271 |
| #87 | #1 and #86 | 3 |
| #88 | #7 and #33 and #86 | 190 |
| #89 | #7 and #47 and #86 | 20 |
| #90 | #7 and #60 and #86 | 108 |
| #91 | #7 and #74 and #86 | 104 |
| #92 | #87 or #88 or #89 or #90 or #91 in Cochrane Reviews (Reviews and Protocols) | 23 |
| #93 | #87 or #88 or #89 or #90 or #91 in Trials | 164 |
DARE and HTA, Cochrane Library
| ID | Search | Hits |
| #1 | MeSH descriptor: [Public‐Private Sector Partnerships] this term only | 8 |
| #2 | MeSH descriptor: [Private Sector] this term only | 41 |
| #3 | MeSH descriptor: [Private Practice] this term only | 84 |
| #4 | MeSH descriptor: [Hospitals, Private] this term only | 16 |
| #5 | MeSH descriptor: [Privatization] this term only | 2 |
| #6 | privat* | 4370 |
| #7 | #2 or #3 or #4 or #5 or #6 | 4370 |
| #8 | MeSH descriptor: [Public Sector] this term only | 51 |
| #9 | MeSH descriptor: [Public Policy] this term only | 53 |
| #10 | MeSH descriptor: [Health Policy] this term only | 338 |
| #11 | [mh ^"state medicine"] | 460 |
| #12 | MeSH descriptor: [State Dentistry] this term only | 7 |
| #13 | MeSH descriptor: [Health Care Reform] this term only | 34 |
| #14 | MeSH descriptor: [Health Planning] this term only | 64 |
| #15 | MeSH descriptor: [Social Control, Formal] this term only | 30 |
| #16 | MeSH descriptor: [Law Enforcement] this term only | 32 |
| #17 | MeSH descriptor: [Government] 2 tree(s) exploded | 813 |
| #18 | MeSH descriptor: [Government Programs] this term only | 30 |
| #19 | MeSH descriptor: [Government Regulation] this term only | 14 |
| #20 | MeSH descriptor: [Facility Regulation and Control] this term only | 3 |
| #21 | MeSH descriptor: [Policy Making] this term only | 47 |
| #22 | MeSH descriptor: [Jurisprudence] this term only | 75 |
| #23 | MeSH descriptor: [Mandatory Reporting] this term only | 10 |
| #24 | MeSH descriptor: [Politics] this term only | 40 |
| #25 | MeSH descriptor: [Legislation as Topic] this term only | 8 |
| #26 | MeSH descriptor: [Legislation, Hospital] this term only | 0 |
| #27 | MeSH descriptor: [Legislation, Medical] this term only | 5 |
| #28 | MeSH descriptor: [Legislation, Nursing] this term only | 0 |
| #29 | MeSH descriptor: [Legislation, Pharmacy] this term only | 5 |
| #30 | MeSH descriptor: [Legislation, Drug] this term only | 15 |
| #31 | [mh ^"legislation, dental"] | 0 |
| #32 | (public* or stewardship* or governance or governing or coordinat* or co next ordinat* or legislat* or regulat* or government* or law or laws or act or acts or policy or policies or politics or reform* or control* or supervis* or monitor*) | 911896 |
| #33 | #8 or #9 or #10 or #11 or #12 or #13 or #14 or #15 or #16 or #17 or #18 or #19 or #20 or #21 or #22 or #23 or #24 or #25 or #26 or #27 or #28 or #29 or #30 or #31 or #32 | 912008 |
| #34 | MeSH descriptor: [Physician's Practice Patterns] this term only | 1104 |
| #35 | MeSH descriptor: [Nurse's Practice Patterns] this term only | 62 |
| #36 | MeSH descriptor: [Dentist's Practice Patterns] this term only | 20 |
| #37 | MeSH descriptor: [Health Knowledge, Attitudes, Practice] this term only | 3833 |
| #38 | MeSH descriptor: [Malpractice] this term only | 13 |
| #39 | MeSH descriptor: [Professional Impairment] this term only | 0 |
| #40 | MeSH descriptor: [Physician Impairment] this term only | 4 |
| #41 | MeSH descriptor: [Medical Errors] this term only | 119 |
| #42 | MeSH descriptor: [Diagnostic Errors] this term only | 258 |
| #43 | MeSH descriptor: [Professional Competence] this term only | 211 |
| #44 | [mh ^"medication errors"] | 228 |
| #45 | [mh ^"clinical competence"] | 1999 |
| #46 | (competence or practice next pattern* or malpractice or mal next practice or error*) | 27575 |
| #47 | #34 or #35 or #36 or #37 or #38 or #39 or #40 or #41 or #42 or #43 or #44 or #45 or #46 | 31029 |
| #48 | MeSH descriptor: [Education] this term only | 498 |
| #49 | MeSH descriptor: [Competency‐Based Education] this term only | 67 |
| #50 | MeSH descriptor: [Education, Public Health Professional] this term only | 3 |
| #51 | MeSH descriptor: [Education, Medical] this term only | 300 |
| #52 | MeSH descriptor: [Education, Medical, Continuing] this term only | 638 |
| #53 | MeSH descriptor: [Education, Nursing] this term only | 117 |
| #54 | MeSH descriptor: [Education, Nursing, Continuing] this term only | 249 |
| #55 | MeSH descriptor: [Education, Dental] this term only | 110 |
| #56 | MeSH descriptor: [Education, Dental, Continuing] this term only | 17 |
| #57 | MeSH descriptor: [Education, Pharmacy] this term only | 36 |
| #58 | [mh ^"education, pharmacy, continuing"] | 27 |
| #59 | (educat* or train or training or trained or colloquium* or conference* or course* or lecture* or meeting* or seminar* or support* or symposi* or workshop*) | 383851 |
| #60 | #48 or #49 or #50 or #51 or #52 or #53 or #54 or #55 or #56 or #57 or #58 or #59 | 383851 |
| #61 | MeSH descriptor: [Delivery of Health Care] this term only | 751 |
| #62 | MeSH descriptor: [Quality of Health Care] this term only | 933 |
| #63 | MeSH descriptor: [Quality Assurance, Health Care] this term only | 741 |
| #64 | MeSH descriptor: [Quality Improvement] this term only | 216 |
| #65 | MeSH descriptor: [Total Quality Management] this term only | 172 |
| #66 | MeSH descriptor: [Outcome and Process Assessment (Health Care)] this term only | 2034 |
| #67 | MeSH descriptor: [Outcome Assessment (Health Care)] this term only | 5357 |
| #68 | MeSH descriptor: [Process Assessment (Health Care)] this term only | 129 |
| #69 | MeSH descriptor: [Guideline Adherence] this term only | 748 |
| #70 | MeSH descriptor: [Benchmarking] this term only | 100 |
| #71 | MeSH descriptor: [Standard of Care] this term only | 91 |
| #72 | [mh ^"reference standards"] | 357 |
| #73 | (best next practice or quality or standard* or benchmark* or adherence or requirement*) | 180929 |
| #74 | #61 or #62 or #63 or #64 or #65 or #66 or #67 or #68 or #69 or #70 or #71 or #72 or #73 | 184786 |
| #75 | (Africa or Asia or Caribbean or "West Indies" or "South America" or "Latin America" or "Central America") | 9614 |
| #76 | (Afghanistan or Albania or Algeria or Angola or Antigua or Barbuda or Argentina or Armenia or Armenian or Aruba or Azerbaijan or Bahrain or Bangladesh or Barbados or Benin or Byelarus or Byelorussian or Belarus or Belorussian or Belorussia or Belize or Bhutan or Bolivia or Bosnia or Herzegovina or Hercegovina or Botswana or Brasil or Brazil or Bulgaria or "Burkina Faso" or "Burkina Fasso" or "Upper Volta" or Burundi or Urundi or Cambodia or "Khmer Republic" or Kampuchea or Cameroon or Cameroons or Cameron or Camerons or "Cape Verde" or "Central African Republic" or Chad or Chile or China or Colombia or Comoros or "Comoro Islands" or Comores or Mayotte or Congo or Zaire or "Costa Rica" or "Cote d'Ivoire" or "Ivory Coast" or Croatia or Cuba or Cyprus or Czechoslovakia or "Czech Republic" or Slovakia or "Slovak Republic") | 42820 |
| #77 | (Djibouti or "French Somaliland" or Dominica or "Dominican Republic" or "East Timor" or "East Timur" or "Timor Leste" or Ecuador or Egypt or "United Arab Republic" or "El Salvador" or Eritrea or Estonia or Ethiopia or Fiji or Gabon or "Gabonese Republic" or Gambia or Gaza or Georgia or Georgian or Ghana or "Gold Coast" or Greece or Grenada or Guatemala or Guinea or Guam or Guiana or Guyana or Haiti or Honduras or Hungary or India or Maldives or Indonesia or Iran or Iraq or "Isle of Man" or Jamaica or Jordan or Kazakhstan or Kazakh or Kenya or Kiribati or Korea or Kosovo or Kyrgyzstan or Kirghizia or "Kyrgyz Republic" or Kirghiz or Kirgizstan or "Lao PDR" or Laos or Latvia or Lebanon or Lesotho or Basutoland or Liberia or Libya or Lithuania) | 40756 |
| #78 | (Macedonia or Madagascar or "Malagasy Republic" or Malaysia or Malaya or Malay or Sabah or Sarawak or Malawi or Nyasaland or Mali or Malta or "Marshall Islands" or Mauritania or Mauritius or "Agalega Islands" or Mexico or Micronesia or "Middle East" or Moldova or Moldovia or Moldovian or Mongolia or Montenegro or Morocco or Ifni or Mozambique or Myanmar or Myanma or Burma or Namibia or Nepal or "Netherlands Antilles" or "New Caledonia" or Nicaragua or Niger or Nigeria or "Northern Mariana Islands" or Oman or Muscat or Pakistan or Palau or Palestine or Panama or Paraguay or Peru or Philippines or Philipines or Phillipines or Phillippines or Poland or Portugal or "Puerto Rico") | 17933 |
| #79 | (Romania or Rumania or Roumania or Russia or Russian or Rwanda or Ruanda or "Saint Kitts" or "St Kitts" or Nevis or "Saint Lucia" or "St Lucia" or "Saint Vincent" or "St Vincent" or Grenadines or Samoa or "Samoan Islands" or "Navigator Island" or "Navigator Islands" or "Sao Tome" or "Saudi Arabia" or Senegal or Serbia or Montenegro or Seychelles or "Sierra Leone" or Slovenia or "Sri Lanka" or Ceylon or "Solomon Islands" or Somalia or Sudan or Suriname or Surinam or Swaziland or Syria or Tajikistan or Tadzhikistan or Tadjikistan or Tadzhik or Tanzania or Thailand or Togo or "Togolese Republic" or Tonga or Trinidad or Tobago or Tunisia or Turkey or Turkmenistan or Turkmen or Uganda or Ukraine or Uruguay or USSR or "Soviet Union" or "Union of Soviet Socialist Republics" or Uzbekistan or Uzbek or Vanuatu or "New Hebrides" or Venezuela or Vietnam or "Viet Nam" or "West Bank" or Yemen or Yugoslavia or Zambia or Zimbabwe or Rhodesia) | 23854 |
| #80 | (developing or less* next developed or "under developed" or underdeveloped or "middle income" or low* next income or underserved or "under served" or deprived or poor*) next (countr* or nation* or population* or world) | 4639 |
| #81 | (developing or less* next developed or "under developed" or underdeveloped or "middle income" or low* next income) next (economy or economies) | 31 |
| #82 | low* next (gdp or gnp or "gross domestic" or "gross national") | 34 |
| #83 | (low near/3 middle near/3 countr*) | 697 |
| #84 | (lmic or lmics or "third world" or "lami country" or "lami countries") | 165 |
| #85 | ("transitional country" or "transitional countries") | 11 |
| #86 | #75 or #76 or #77 or #78 or #79 or #80 or #81 or #82 or #83 or #84 or #85 | 121587 |
| #87 | #1 and #86 | 3 |
| #88 | #7 and #33 and #86 | 1386 |
| #89 | #7 and #47 and #86 | 497 |
| #90 | #7 and #60 and #86 | 1000 |
| #91 | #7 and #74 and #86 | 922 |
| #92 | #87 or #88 or #89 or #90 or #91 in Other Reviews (DARE) | 15 |
| #93 | #87 or #88 or #89 or #90 or #91 in Technology Assessments (HTA) | 2 |
MEDLINE: Epub Ahead of Print, In‐Process & Other Non‐Indexed Citations, MEDLINE Daily and MEDLINE 1946 to Present, OvidSP
| # | Searches | Results |
| 1 | Public‐Private Sector Partnerships/ | 1464 |
| 2 | Private Sector/ | 8019 |
| 3 | Private Practice/ | 7811 |
| 4 | Hospitals, Private/ | 2492 |
| 5 | Privatization/ | 2002 |
| 6 | privat*.ti,ab. | 71034 |
| 7 | or/2‐6 | 80562 |
| 8 | Public Sector/ | 5420 |
| 9 | Public Policy/ | 29034 |
| 10 | Health Policy/ | 55474 |
| 11 | State Medicine/ | 52723 |
| 12 | State Dentistry/ | 2224 |
| 13 | Health Care Reform/ | 29873 |
| 14 | Health Planning/ | 20949 |
| 15 | Social Control, Formal/ | 11298 |
| 16 | Law Enforcement/ | 2937 |
| 17 | exp Government/ | 132337 |
| 18 | Government Programs/ | 4139 |
| 19 | Government Regulation/ | 19057 |
| 20 | "Facility Regulation and Control"/ | 3144 |
| 21 | Policy Making/ | 13572 |
| 22 | Jurisprudence/ | 29305 |
| 23 | Mandatory Reporting/ | 2784 |
| 24 | Politics/ | 43289 |
| 25 | Legislation as Topic/ | 15676 |
| 26 | Legislation, Hospital/ | 2379 |
| 27 | Legislation, Medical/ | 16269 |
| 28 | Legislation, Nursing/ | 3047 |
| 29 | Legislation, Pharmacy/ | 1186 |
| 30 | Legislation, Drug/ | 9217 |
| 31 | Legislation, Dental/ | 1902 |
| 32 | (public* or stewardship* or governance or governing or coordinat* or co ordinat* or legislat* or regulat* or government* or law or laws or act or acts or policy or policies or politics or reform* or control* or supervis* or monitor*).ti,ab. | 5512906 |
| 33 | or/8‐32 | 5761440 |
| 34 | Physician's Practice Patterns/ | 47211 |
| 35 | Nurse's Practice Patterns/ | 1503 |
| 36 | Dentist's Practice Patterns/ | 1837 |
| 37 | Health Knowledge, Attitudes, Practice/ | 84689 |
| 38 | Malpractice/ | 26511 |
| 39 | Professional Impairment/ | 1181 |
| 40 | Physician Impairment/ | 2163 |
| 41 | Medical Errors/ | 14062 |
| 42 | Diagnostic Errors/ | 33683 |
| 43 | Medication Errors/ | 11292 |
| 44 | Professional Competence/ | 21816 |
| 45 | Clinical Competence/ | 73827 |
| 46 | (competence or practice pattern* or malpractice or mal practice or error*).ti,ab. | 283157 |
| 47 | or/34‐46 | 547625 |
| 48 | Education/ | 19169 |
| 49 | Competency‐Based Education/ | 3164 |
| 50 | Education, Public Health Professional/ | 686 |
| 51 | Education, Medical/ | 50988 |
| 52 | Education, Medical, Continuing/ | 22759 |
| 53 | Education, Nursing/ | 29751 |
| 54 | Education, Nursing, Continuing/ | 21849 |
| 55 | Education, Dental/ | 13502 |
| 56 | Education, Dental, Continuing/ | 3274 |
| 57 | Education, Pharmacy/ | 4548 |
| 58 | Education, Pharmacy, Continuing/ | 800 |
| 59 | (educat* or train or training or trained or colloquium? or conference? or course? or lecture? or meeting? or seminar? or support* or symposi* or workshop?).ti,ab. | 2455704 |
| 60 | or/48‐59 | 2532583 |
| 61 | "Delivery of Health Care"/ | 72524 |
| 62 | "Quality of Health Care"/ | 62196 |
| 63 | Quality Assurance, Health Care/ | 52042 |
| 64 | Quality Improvement/ | 10943 |
| 65 | Total Quality Management/ | 12099 |
| 66 | "Outcome and Process Assessment (health care)"/ | 23242 |
| 67 | "Outcome Assessment (health care)"/ | 55982 |
| 68 | "Process Assessment (health care)"/ | 3515 |
| 69 | Guideline Adherence/ | 24837 |
| 70 | Benchmarking/ | 11195 |
| 71 | "Standard of Care"/ | 1366 |
| 72 | Reference Standards/ | 38051 |
| 73 | (best practice or quality or standard* or benchmark* or adherence or requirement*).ti,ab. | 1843970 |
| 74 | or/61‐73 | 2052050 |
| 75 | Developing Countries.sh,kf. | 76998 |
| 76 | (Africa or Asia or Caribbean or West Indies or South America or Latin America or Central America).hw,kf,ti,ab,cp. | 213538 |
| 77 | (Afghanistan or Albania or Algeria or Angola or Antigua or Barbuda or Argentina or Armenia or Armenian or Aruba or Azerbaijan or Bahrain or Bangladesh or Barbados or Benin or Byelarus or Byelorussian or Belarus or Belorussian or Belorussia or Belize or Bhutan or Bolivia or Bosnia or Herzegovina or Hercegovina or Botswana or Brasil or Brazil or Bulgaria or Burkina Faso or Burkina Fasso or Upper Volta or Burundi or Urundi or Cambodia or Khmer Republic or Kampuchea or Cameroon or Cameroons or Cameron or Camerons or Cape Verde or Central African Republic or Chad or Chile or China or Colombia or Comoros or Comoro Islands or Comores or Mayotte or Congo or Zaire or Costa Rica or Cote d'Ivoire or Ivory Coast or Croatia or Cuba or Cyprus or Czechoslovakia or Czech Republic or Slovakia or Slovak Republic or Djibouti or French Somaliland or Dominica or Dominican Republic or East Timor or East Timur or Timor Leste or Ecuador or Egypt or United Arab Republic or El Salvador or Eritrea or Estonia or Ethiopia or Fiji or Gabon or Gabonese Republic or Gambia or Gaza or Georgia Republic or Georgian Republic or Ghana or Gold Coast or Greece or Grenada or Guatemala or Guinea or Guam or Guiana or Guyana or Haiti or Honduras or Hungary or India or Maldives or Indonesia or Iran or Iraq or Isle of Man or Jamaica or Jordan or Kazakhstan or Kazakh or Kenya or Kiribati or Korea or Kosovo or Kyrgyzstan or Kirghizia or Kyrgyz Republic or Kirghiz or Kirgizstan or Lao PDR or Laos or Latvia or Lebanon or Lesotho or Basutoland or Liberia or Libya or Lithuania or Macedonia or Madagascar or Malagasy Republic or Malaysia or Malaya or Malay or Sabah or Sarawak or Malawi or Nyasaland or Mali or Malta or Marshall Islands or Mauritania or Mauritius or Agalega Islands or Mexico or Micronesia or Middle East or Moldova or Moldovia or Moldovian or Mongolia or Montenegro or Morocco or Ifni or Mozambique or Myanmar or Myanma or Burma or Namibia or Nepal or Netherlands Antilles or New Caledonia or Nicaragua or Niger or Nigeria or Northern Mariana Islands or Oman or Muscat or Pakistan or Palau or Palestine or Panama or Paraguay or Peru or Philippines or Philipines or Phillipines or Phillippines or Poland or Portugal or Puerto Rico or Romania or Rumania or Roumania or Russia or Russian or Rwanda or Ruanda or Saint Kitts or St Kitts or Nevis or Saint Lucia or St Lucia or Saint Vincent or St Vincent or Grenadines or Samoa or Samoan Islands or Navigator Island or Navigator Islands or Sao Tome or Saudi Arabia or Senegal or Serbia or Montenegro or Seychelles or Sierra Leone or Slovenia or Sri Lanka or Ceylon or Solomon Islands or Somalia or Sudan or Suriname or Surinam or Swaziland or Syria or Tajikistan or Tadzhikistan or Tadjikistan or Tadzhik or Tanzania or Thailand or Togo or Togolese Republic or Tonga or Trinidad or Tobago or Tunisia or Turkey or Turkmenistan or Turkmen or Uganda or Ukraine or Uruguay or USSR or Soviet Union or Union of Soviet Socialist Republics or Uzbekistan or Uzbek or Vanuatu or New Hebrides or Venezuela or Vietnam or Viet Nam or West Bank or Yemen or Yugoslavia or Zambia or Zimbabwe or Rhodesia).hw,kf,ti,ab,cp. | 3191736 |
| 78 | ((developing or less* developed or under developed or underdeveloped or middle income or low* income or underserved or under served or deprived or poor*) adj (countr* or nation? or population? or world)).ti,ab. | 71001 |
| 79 | ((developing or less* developed or under developed or underdeveloped or middle income or low* income) adj (economy or economies)).ti,ab. | 358 |
| 80 | (low* adj (gdp or gnp or gross domestic or gross national)).ti,ab. | 187 |
| 81 | (low adj3 middle adj3 countr*).ti,ab. | 6548 |
| 82 | (lmic or lmics or third world or lami countr*).ti,ab. | 4266 |
| 83 | transitional countr*.ti,ab. | 127 |
| 84 | or/75‐83 | 3349766 |
| 85 | randomized controlled trial.pt. | 421325 |
| 86 | controlled clinical trial.pt. | 91035 |
| 87 | pragmatic clinical trial.pt. | 342 |
| 88 | multicenter study.pt. | 204840 |
| 89 | non‐randomized controlled trials as topic/ | 66 |
| 90 | interrupted time series analysis/ | 163 |
| 91 | controlled before‐after studies/ | 146 |
| 92 | (randomis* or randomiz* or randomly or random allocat*).ti,ab. | 676468 |
| 93 | groups.ab. | 1601971 |
| 94 | (trial or multicenter or multi center or multicentre or multi centre).ti. | 184051 |
| 95 | (intervention* or controlled or control group or compare or compared or (before adj5 after) or (pre adj5 post) or pretest or pre test or posttest or post test or quasiexperiment* or quasi experiment* or evaluat* or effect or impact or time series or time point? or repeated measur*).ti,ab. | 7859186 |
| 96 | or/85‐95 | 8596750 |
| 97 | Animals/ | 5899594 |
| 98 | Humans/ | 16088033 |
| 99 | 97 not (97 and 98) | 4232417 |
| 100 | comment.pt. | 673136 |
| 101 | editorial.pt. | 409272 |
| 102 | cochrane database of systematic reviews.jn. | 12472 |
| 103 | comment on.cm. | 673135 |
| 104 | review.pt. | 2144868 |
| 105 | review.ti. | 325750 |
| 106 | or/99‐105 | 7253662 |
| 107 | 96 not 106 | 6192962 |
| 108 | 1 and 84 and 107 | 116 |
| 109 | 7 and 33 and 84 and 107 | 6355 |
| 110 | 7 and 47 and 84 and 107 | 932 |
| 111 | 7 and 60 and 84 and 107 | 3376 |
| 112 | 7 and 74 and 84 and 107 | 2768 |
| 113 | or/108‐112 | 7596 |
Embase, OvidSP
| # | Searches | Results |
| 1 | (privat* adj6 (public* or stewardship* or governance or governing or coordinat* or co ordinat* or legislat* or regulat* or government* or law or laws or act or acts or policy or policies or politics or reform* or control* or supervis* or monitor*)).ti,ab. | 23079 |
| 2 | (privat* adj6 (competence or practice pattern* or malpractice or mal practice or error*)).ti,ab. | 116 |
| 3 | (privat* adj6 (educat* or train or training or trained or colloquium? or conference? or course? or lecture? or meeting? or seminar? or support* or symposi* or workshop?)).ti,ab. | 2817 |
| 4 | (privat* adj6 (best practice or quality or standard* or benchmark* or adherence or requirement*)).ti,ab. | 1329 |
| 5 | or/1‐4 | 25716 |
| 6 | Developing Country.sh. | 78600 |
| 7 | (Africa or Asia or Caribbean or West Indies or South America or Latin America or Central America).hw,ti,ab,cp. | 242519 |
| 8 | (Afghanistan or Albania or Algeria or Angola or Antigua or Barbuda or Argentina or Armenia or Armenian or Aruba or Azerbaijan or Bahrain or Bangladesh or Barbados or Benin or Byelarus or Byelorussian or Belarus or Belorussian or Belorussia or Belize or Bhutan or Bolivia or Bosnia or Herzegovina or Hercegovina or Botswana or Brasil or Brazil or Bulgaria or Burkina Faso or Burkina Fasso or Upper Volta or Burundi or Urundi or Cambodia or Khmer Republic or Kampuchea or Cameroon or Cameroons or Cameron or Camerons or Cape Verde or Central African Republic or Chad or Chile or China or Colombia or Comoros or Comoro Islands or Comores or Mayotte or Congo or Zaire or Costa Rica or Cote d'Ivoire or Ivory Coast or Croatia or Cuba or Cyprus or Czechoslovakia or Czech Republic or Slovakia or Slovak Republic or Djibouti or French Somaliland or Dominica or Dominican Republic or East Timor or East Timur or Timor Leste or Ecuador or Egypt or United Arab Republic or El Salvador or Eritrea or Estonia or Ethiopia or Fiji or Gabon or Gabonese Republic or Gambia or Gaza or Georgia Republic or Georgian Republic or Ghana or Gold Coast or Greece or Grenada or Guatemala or Guinea or Guam or Guiana or Guyana or Haiti or Honduras or Hungary or India or Maldives or Indonesia or Iran or Iraq or Isle of Man or Jamaica or Jordan or Kazakhstan or Kazakh or Kenya or Kiribati or Korea or Kosovo or Kyrgyzstan or Kirghizia or Kyrgyz Republic or Kirghiz or Kirgizstan or Lao PDR or Laos or Latvia or Lebanon or Lesotho or Basutoland or Liberia or Libya or Lithuania or Macedonia or Madagascar or Malagasy Republic or Malaysia or Malaya or Malay or Sabah or Sarawak or Malawi or Nyasaland or Mali or Malta or Marshall Islands or Mauritania or Mauritius or Agalega Islands or Mexico or Micronesia or Middle East or Moldova or Moldovia or Moldovian or Mongolia or Montenegro or Morocco or Ifni or Mozambique or Myanmar or Myanma or Burma or Namibia or Nepal or Netherlands Antilles or New Caledonia or Nicaragua or Niger or Nigeria or Northern Mariana Islands or Oman or Muscat or Pakistan or Palau or Palestine or Panama or Paraguay or Peru or Philippines or Philipines or Phillipines or Phillippines or Poland or Portugal or Puerto Rico or Romania or Rumania or Roumania or Russia or Russian or Rwanda or Ruanda or Saint Kitts or St Kitts or Nevis or Saint Lucia or St Lucia or Saint Vincent or St Vincent or Grenadines or Samoa or Samoan Islands or Navigator Island or Navigator Islands or Sao Tome or Saudi Arabia or Senegal or Serbia or Montenegro or Seychelles or Sierra Leone or Slovenia or Sri Lanka or Ceylon or Solomon Islands or Somalia or Sudan or Suriname or Surinam or Swaziland or Syria or Tajikistan or Tadzhikistan or Tadjikistan or Tadzhik or Tanzania or Thailand or Togo or Togolese Republic or Tonga or Trinidad or Tobago or Tunisia or Turkey or Turkmenistan or Turkmen or Uganda or Ukraine or Uruguay or USSR or Soviet Union or Union of Soviet Socialist Republics or Uzbekistan or Uzbek or Vanuatu or New Hebrides or Venezuela or Vietnam or Viet Nam or West Bank or Yemen or Yugoslavia or Zambia or Zimbabwe or Rhodesia).hw,ti,ab,cp. | 2972104 |
| 9 | ((developing or less* developed or under developed or underdeveloped or middle income or low* income or underserved or under served or deprived or poor*) adj (countr* or nation? or population? or world)).ti,ab. | 73774 |
| 10 | ((developing or less* developed or under developed or underdeveloped or middle income or low* income) adj (economy or economies)).ti,ab. | 382 |
| 11 | (low* adj (gdp or gnp or gross domestic or gross national)).ti,ab. | 207 |
| 12 | (low adj3 middle adj3 countr*).ti,ab. | 5147 |
| 13 | (lmic or lmics or third world or lami countr*).ti,ab. | 4078 |
| 14 | transitional countr*.ti,ab. | 157 |
| 15 | or/6‐14 | 3164473 |
| 16 | Randomized Controlled Trial/ | 368416 |
| 17 | Controlled Clinical Trial/ | 390467 |
| 18 | Quasi Experimental Study/ | 2348 |
| 19 | Pretest Posttest Control Group Design/ | 226 |
| 20 | Time Series Analysis/ | 15208 |
| 21 | Experimental Design/ | 10990 |
| 22 | Multicenter Study/ | 119790 |
| 23 | (randomis* or randomiz* or randomly or random allocat*).ti,ab. | 784492 |
| 24 | groups.ab. | 1818653 |
| 25 | (trial or multicentre or multicenter or multi centre or multi center).ti. | 209182 |
| 26 | (intervention* or controlled or control group or compare or compared or (before adj5 after) or (pre adj5 post) or pretest or pre test or posttest or post test or quasiexperiment* or quasi experiment* or evaluat* or effect or impact or time series or time point? or repeated measur*).ti,ab. | 8562609 |
| 27 | or/16‐26 | 9336489 |
| 28 | review.ti. | 320850 |
| 29 | "cochrane database of systematic reviews".jn. | 3771 |
| 30 | Nonhuman/ | 4494366 |
| 31 | or/28‐30 | 4795875 |
| 32 | 27 not 31 | 7358995 |
| 33 | 5 and 15 and 32 | 4227 |
| 34 | limit 33 to embase | 2929 |
Science Citation Index; Social Sciences Citation Index, ISI Web of Knowledge
TOPIC: ("public stewardship" or "public partnership") OR TOPIC: ((privat*) AND (stewardship* or governance or governing or policy or policies or politics or coordinat* or legislat* or regulat* or supervis* or monitor*) AND (health* or medical* or pharmac* or drug or drugs or doctor* or physician* or nurse or nurses or hospital*) AND (((developing or "less developed" or "lesser developed" or underdeveloped or "under developed" or "middle income" or "low income" or "lower income" or transitional) AND (countr* or nation* or population* or world)) or (lmic or lmics)) AND (randomis* or randomiz* or impact or effect or evaluat* or control* or intervention* or "time series" or "time point" or "time points" or "repeated measure" or "repeated measures" or quasiexperiment* or "quasi experiment")) OR TOPIC: ((privat*) AND (public*) AND (partnership* or engagement* or collaborat*) AND (health* or medical* or pharmac* or drug or drugs or doctor* or physician* or nurse or nurses or hospital*) AND (((developing or "less developed" or "lesser developed" or underdeveloped or "under developed" or "middle income" or "low income" or "lower income" or transitional) AND (countr* or nation* or population* or world)) or (lmic or lmics)) AND (randomis* or randomiz* or impact or effect or evaluat* or control* or intervention* or "time series" or "time point" or "time points" or "repeated measure" or "repeated measures" or quasiexperiment* or "quasi experiment"))
Global Health, OvidSP
| # | Searches | Results |
| 1 | (public stewardship or public partnership*).mp. | 43 |
| 2 | (private and public and (stewardship or governance or governing or policy or policies or politics or coordinate or coordination or co ordinate or co ordination or legislate or legislation or regulat* or supervise or supervision or monitor or monitoring or partnership or partnerships or engagement or collaborate or collaboration or collaborating) and (randomised or randomized or random allocation or randomly allocated or impact or impacts or effect or effects or evaluate or control group or control groups or controlled or intervention or time series or time point or time points or repeated measur* or quasiexperiment* or quasi experiment*)).mp. | 1127 |
| 3 | 1 or 2 | 1157 |
WHOLIS, WHO
words or phrase: "public stewardship$ or public partnership$"
Health Management, ProQuest
(Two strategies)
1. ALL("public stewardship" or "public partnership")
2. ALL(privat*) and ALL(public* or stewardship* or (national NEAR/3 program*) or governance) and ALL(coordinat* or (co PRE/0 ordinat*) or educat* or train or training or regulat*) and ALL((Africa or Asia or Caribbean or "West Indies" or "South America" or "Latin America" or "Central America" or Afghanistan or Albania or Algeria or Angola or Antigua or Barbuda or Argentina or Armenia or Armenian or Aruba or Azerbaijan or Bahrain or Bangladesh or Barbados or Benin or Byelarus or Byelorussian or Belarus or Belorussian or Belorussia or Belize or Bhutan or Bolivia or Bosnia or Herzegovina or Hercegovina or Botswana or Brasil or Brazil or Bulgaria or "Burkina Faso" or "Burkina Fasso" or "Upper Volta" or Burundi or Urundi or Cambodia or "Khmer Republic" or Kampuchea or Cameroon or Cameroons or Cameron or Camerons or "Cape Verde" or "Central African Republic" or Chad or Chile or China or Colombia or Comoros or "Comoro Islands" or Comores or Mayotte or Congo or Zaire or "Costa Rica" or "Cote d'Ivoire" or "Ivory Coast" or Croatia or Cuba or Cyprus or Czechoslovakia or "Czech Republic" or Slovakia or "Slovak Republic" or Djibouti or "French Somaliland" or Dominica or "Dominican Republic" or "East Timor" or "East Timur" or "Timor Leste" or Ecuador or Egypt or "United Arab Republic" or "El Salvador" or Eritrea or Estonia or Ethiopia or Fiji or Gabon or "Gabonese Republic" or Gambia or Gaza or "Georgia Republic" or "Georgian Republic" or Ghana or "Gold Coast" or Greece or Grenada or Guatemala or Guinea or Guam or Guiana or Guyana or Haiti or Honduras or Hungary or India or Maldives or Indonesia or Iran or Iraq or "Isle of Man" or Jamaica or Jordan or Kazakhstan or Kazakh or Kenya or Kiribati or Korea or Kosovo or Kyrgyzstan or Kirghizia or "Kyrgyz Republic" or Kirghiz or Kirgizstan or "Lao PDR" or Laos or Latvia or Lebanon or Lesotho or Basutoland or Liberia or Libya or Lithuania or Macedonia or Madagascar or "Malagasy Republic" or Malaysia or Malaya or Malay or Sabah or Sarawak or Malawi or Nyasaland or Mali or Malta or "Marshall Islands" or Mauritania or Mauritius or "Agalega Islands" or Mexico or Micronesia or "Middle East" or Moldova or Moldovia or Moldovian or Mongolia or Montenegro or Morocco or Ifni or Mozambique or Myanmar or Myanma or Burma or Namibia or Nepal or "Netherlands Antilles" or "New Caledonia" or Nicaragua or Niger or Nigeria or "Northern Mariana Islands" or Oman or Muscat or Pakistan or Palau or Palestine or Panama or Paraguay or Peru or Philippines or Philipines or Phillipines or Phillippines or Poland or Portugal or Puerto Rico or Romania or Rumania or Roumania or Russia or Russian or Rwanda or Ruanda or "Saint Kitts" or "St Kitts" or Nevis or "Saint Lucia" or "St Lucia" or "Saint Vincent" or "St Vincent" or Grenadines or Samoa or "Samoan Islands" or "Navigator Island" or "Navigator Islands" or "Sao Tome" or "Saudi Arabia" or Senegal or Serbia or Montenegro or Seychelles or "Sierra Leone" or Slovenia or "Sri Lanka" or Ceylon or "Solomon Islands" or Somalia or Sudan or "South Africa" or Suriname or Surinam or Swaziland or Syria or Tajikistan or Tadzhikistan or Tadjikistan or Tadzhik or Tanzania or Thailand or Togo or "Togolese Republic" or Tonga or Trinidad or Tobago or Tunisia or Turkey or Turkmenistan or Turkmen or Uganda or Ukraine or Uruguay or USSR or "Soviet Union" or "Union of Soviet Socialist Republics" or Uzbekistan or Uzbek or Vanuatu or "New Hebrides" or Venezuela or Vietnam or" Viet Nam" or "West Bank" or Yemen or Yugoslavia or Zambia or Zimbabwe or Rhodesia or lmic or lmics or "third world" or "lami country" or "lami countries" or "transitional country" or "transitional countries") or (developing or "less developed" or "lesser developed" or "under developed" or underdeveloped or "middle income" or "low income" or "lower income" or underserved or "under served" or deprived or poor*) PRE/0 (countr* or nation* or population* or world or economy or economies)) and ALL(randomised or randomized or "randomly allocated" or "random allocation" or trial or intervention or interventions or controlled or "control group" or "before and after" or "pre and post" or pretest or "pre test" or posttest or "post test" or quasiexperiment or quasiexperimental or evaluate or effect or impact or "time series" or "time point" or "time points" or "repeated measure" or "repeated measures" or "repeated measurement" or "repeated measurements")
OpenGrey
((stewardship* OR partnership*) OR (privat* AND public*)) AND (05T Health services, health administration, community care services)
The Grey Literature Report
1. Keywords:public stewardship and tick off for Grey Literature under: Collection codes
2. Title:public stewardship and tick off for Grey Literature under: Collection codes
3. Keywords:private stewardship and tick off for Grey Literature under: Collection codes
4. Title:private stewardship and tick off for Grey Literature under: Collection codes
5. Keywords:public partnership and tick off for Grey Literature under: Collection codes
6. Title:public partnership and tick off for Grey Literature under: Collection codes
7. Keywords:private partnership and tick off for Grey Literature under: Collection codes
8. Title:private partnership and tick off for Grey Literature under: Collection codes
World Bank e‐Library
[Title: stewardship* OR partnership* OR (privat* AND public*)] AND [Keyword: stewardship* OR partnership* OR (privat* AND public*)] AND [Abstract: stewardship* OR partnership* OR (privat* AND public*)]
International Clinical Trials Registry Platform
1. private AND public (in title) and set Recruitment status to ALL
2. private AND public (in intervention) and set Recruitment status to ALL
3. stewardship OR stewardships OR partnership OR partnerships (in title) and set Recruitment status to ALL
4. stewardship OR stewardships OR partnership OR partnerships (in intervention) and set Recruitment status to ALL
ClinicalTrials.gov
In Search Terms
(private OR public) AND (stewardship OR stewardships OR partnership OR partnerships)
US National Institutes of Health
In Search Terms
(private OR public) AND (stewardship OR stewardships OR partnership OR partnerships)
United Nations Children's Fund
In Search Terms
(private OR public) AND (stewardship OR stewardships OR partnership OR partnerships)
Alliance for Health Policy and Systems Research
In Search Terms
(private OR public) AND (stewardship OR stewardships OR partnership OR partnerships)
United States Agency for International Development
In Search Terms
(private OR public) AND (stewardship OR stewardships OR partnership OR partnerships)
Gavi, The Vaccine Alliance
In Search Terms
(private OR public) AND (stewardship OR stewardships OR partnership OR partnerships)
Private Healthcare in Developing Countries
In Search Terms
(private OR public) AND (stewardship OR stewardships OR partnership OR partnerships)
Population Services International
In Search Terms
(private OR public) AND (stewardship OR stewardships OR partnership OR partnerships)
Shops
In Search Terms
(private OR public) AND (stewardship OR stewardships OR partnership OR partnerships)
Department for International Development
In Search Terms
(private OR public) AND (stewardship OR stewardships OR partnership OR partnerships)
Center For Health Market Innovations
In Search Terms
(private OR public) AND (stewardship OR stewardships OR partnership OR partnerships)
World Bank
In Search Terms
(private OR public) AND (stewardship OR stewardships OR partnership OR partnerships)
Data and analyses
Comparison 1. Training.
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
|---|---|---|---|---|
| 1 Quality of care | 1 | Risk Ratio (M‐H, Fixed, 95% CI) | Totals not selected | |
| 1.1 RCT studies | 1 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.2 Non RCT studies | 1 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.0 [0.0, 0.0] | |
| 2 Quality of care | 1 | Odds Ratio (Fixed, 95% CI) | Totals not selected |
Characteristics of studies
Characteristics of included studies [ordered by study ID]
Abuya 2009.
| Methods | Cluster‐randomised trial conducted in 3 districts in Kenya from 2002‐2005 | |
| Participants | Number enrolled: 10 clusters (sub‐districts) with a total of 140 functional anti‐malarial medicine 'shops' Study population: 10 divisions (clusters) in 3 rural districts in Kenya (i.e. Kwale, Makueni and Busia ) with divisions as unit of randomisation in Kenya Inclusion criteria: the main sellers in outlets stocking anti‐malarial medicines. The eligible outlets had to be relatively stable (on the basis of local knowledge) Exclusion criteria: non‐functional anti‐malarial medicine outlets Withdrawals and exclusions: in Kwale district, 24% of trainees changed business or closed their outlets. In Makueni and Busia districts, 4% and 5% of trainees respectively closed their outlets within 6–8 months after training |
|
| Interventions | Intervention: 5 divisions (clusters) with 60 functional medicine outlets were allocated to the intervention group. In total, 122 private medical retailers were trained in Kwale, 79 in Busia, and 247 in Makueni districts Duration: the intervention consisted of 2‐day training workshops at local venues Who delivered the intervention: Kenya's Ministry of Health Description: the training covered signs of simple and severe malaria; malaria treatment and prevention; drug resistance; referral practices; storage and expiry of medicines; and communication skills Comparison: no training was conducted in 5 'control' divisions (clusters) with 80 outlets |
|
| Outcomes | Retailers’ knowledge on the treatment of childhood fevers; recommended amodiaquine adequately Surrogate clients were used to pose as patients and retail audits were used to collect information on the outlets and retailers |
|
| Notes | The study was approved by the Kenya National Scientific Steering and Ethical Research Committees and the WHO Secretariat Committee on Research Involving Human Subjects (SCRIHS). A cost‐effectiveness study of the intervention in Kenya suggested that, although the initial cost of setting such programmes is high, the annual running costs could be contained within a typical district budget. In total, USD 5202.00 and USD 5882.00 were released to Kwale and Makueni districts respectively, for the implementation of the programmes in the setup year. The programme in Busia district was implemented with funds from UNICEF, with a total of USD5838 allocated |
|
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | No description |
| Allocation concealment (selection bias) | Unclear risk | No description |
| Blinding of participants and personnel (performance bias) All outcomes | Low risk | Surrogate clients, fieldworkers posed as clients seeking care from a provider who was unaware of the clients' identity |
| Blinding of outcome assessment (detection bias) All outcomes | Low risk | Surrogate clients, fieldworkers posed as clients seeking care from a provider who was unaware of their identity |
| Incomplete outcome data (attrition bias) All outcomes | High risk | 4% and 24% of trained outlets closed within 6–8 months of training |
| Selective reporting (reporting bias) | Unclear risk | No description |
| Other bias | Low risk | No evidence of other sources of bias |
| Similar baseline characteristics | Low risk | The District Health Management Team in each district identified divisions they considered similar in characteristics |
| Similar baseline outcome measures | Unclear risk | No description |
| Reliable primary outcome measures | Unclear risk | No description |
| Adequate protection against contamination | High risk | It is possible that there was some degree of contamination in Busia district, a suggestion supported by the District Health Management Team report that the district had experienced almost complete coverage with PMR programmes at the time of the evaluation, leaving only one division as the control |
Chalker 2002.
| Methods | Cluster‐randomised trial conducted in Hanoi, Vietnam, conducted from May 1998‐September 1999 | |
| Participants | Number enrolled: 34 paired pharmacies in urban Hanoi Study population: private pharmacies registered in the urban area of Hanoi Inclusion criteria: private registered pharmacies in urban Hanoi Exclusion criteria: pharmacies outside hospitals and not mainly wholesalers Withdrawal and exclusions: in the course of the study, 4 pharmacies closed and 1 refused to take part in the third intervention. No post‐intervention interviews were taken from 4 intervention and 4 control pharmacies as they were closed early before the Vietnamese New Year holidays. These gave 22 matched pairs |
|
| Interventions | Intervention: regulatory enforcement, face‐to‐face education, and peer influence. The three interventions were implemented sequentially in 22 pharmacies Duration: each intervention lasted 3 months, with a gap of 4 months before the next intervention Who delivered the intervention: the Hanoi Health Bureau and the Hanoi Pharmacy Association Description:
Comparison: 22 pharmacies with no intervention |
|
| Outcomes | Knowledge and reported change in practice for correct management of tracer conditions | |
| Notes | The cost for the 3 interventions was approximately USD 5700.00 for 30 pharmacies | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | No description |
| Allocation concealment (selection bias) | Unclear risk | No description |
| Blinding of participants and personnel (performance bias) All outcomes | Unclear risk | No description |
| Blinding of outcome assessment (detection bias) All outcomes | Unclear risk | No description |
| Incomplete outcome data (attrition bias) All outcomes | Unclear risk | No description |
| Selective reporting (reporting bias) | Unclear risk | No description |
| Other bias | Low risk | Failure to interview people from 4 intervention and 4 control pharmacies after the interventions may have caused a bias in results, but this is unlikely because of the equivalent numbers between intervention and control |
| Similar baseline characteristics | Low risk | There was matching of pharmacies to ensure similar baseline characteristics |
| Similar baseline outcome measures | Low risk | There was a slight difference in the outcome measures at baseline level which was not statistically significant |
| Reliable primary outcome measures | Unclear risk | No description |
| Adequate protection against contamination | Unclear risk | No description |
Chalker 2005.
| Methods | Randomised trial conducted in Hanoi (Vietnam) and Bangkok (Thailand) from June 1998‐September 1999 | |
| Participants | Number enrolled: 68 pharmacies in Hanoi and 78 pharmacies in Bangkok Study population: private registered pharmacies in Hanoi and Bangkok Inclusion criteria: private pharmacies in Hanoi and Bangkok Withdrawals and exclusions: in Bangkok 1 intervention pharmacy and 2 control pharmacies closed down and 3 of each were missed in at least one of the simulated clients' round of visits. In Hanoi 1 intervention and 3 control pharmacies closed, 1 intervention pharmacy refused to continue, and 4 intervention and 4 control pharmacies had incomplete data for different rounds of client visits |
|
| Interventions | Intervention: enforcement of regulations, education, and peer review in 34 pharmacies in Hanoi and 39 pharmacies in Bangkok Duration: each intervention lasted 3 months, with a gap of 4 months before the next intervention Who delivered the intervention: the Hanoi Provincial Health Bureau, Hanoi Pharmacy Association, Hanoi Medical University, and the Hanoi College of Pharmacy. In Bangkok, the interventions were delivered by the Health System Research Institute, Chulalongkorn University, and the Community Pharmacy Association Description
Comparison: no intervention on 34 control pharmacies in Hanoi and 39 control pharmacies in Bangkok |
|
| Outcomes | Change in the dispensing practices of antibiotics in Hanoi and Bangkok | |
| Notes | The study had ethical approval from the Karolinska Institute as well as the Ministries of Health in Vietnam and Thailand | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | No description |
| Allocation concealment (selection bias) | Unclear risk | No description |
| Blinding of participants and personnel (performance bias) All outcomes | Unclear risk | No description |
| Blinding of outcome assessment (detection bias) All outcomes | Unclear risk | No description |
| Incomplete outcome data (attrition bias) All outcomes | High risk | In Bangkok 1 intervention pharmacy and 2 control pharmacies closed down, and 3 of each were missed in at least one of the simulated clients' visits. In Hanoi 1 intervention and 3 control pharmacies closed, 1 intervention pharmacy refused to continue and 4 intervention and 4 control pharmacies had incomplete data for different rounds of client visits |
| Selective reporting (reporting bias) | Unclear risk | No description |
| Other bias | Low risk | The study appears to be free of other biases |
| Similar baseline characteristics | Low risk | Some basic characteristics (i.e. turnover, staff numbers and qualifications) were not possible to judge from the simulated clients' visit. However, the random selection ensured comparability between intervention and control pharmacies |
| Similar baseline outcome measures | Low risk | There were no significant differences in the outcome measures at baseline at intra‐city level but there was a difference in the baseline knowledge at inter‐city levels |
| Reliable primary outcome measures | Unclear risk | No description |
| Adequate protection against contamination | Unclear risk | No description |
Chuc 2002.
| Methods | Cluster‐randomised controlled trial conducted from October 1997‐January 2000 in Hanoi, Vietnam | |
| Participants | Number enrolled: 34 pairs of private pharmacies Study population: 34 pairs of private pharmacies in Hanoi randomly selected according to the following matching criteria:
Inclusion criteria: private pharmacies in the study area Exclusion criteria: pharmacies located inside a hospital or that were mainly wholesalers were excluded Withdrawals and exclusions: 4 pharmacies were excluded because of irregular opening hours. As a consequence, the other pharmacy in each pair was also excluded as the results were analysed in pairs |
|
| Interventions | Intervention: regulatory enforcement, educational intervention, and peer review Duration: each intervention lasted 3 months, with a gap of 4 months before the next intervention Who delivered the intervention: Hanoi Bureau Health and Hanoi Pharmacy Association Description
Comparison: no intervention |
|
| Outcomes | Effect of multi‐component intervention using tracer conditions i.e. correct symptomatic treatment of sexually transmitted disease | |
| Notes | The estimated total cost of the interventions was approximately USD 5700.00 | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | No description |
| Allocation concealment (selection bias) | Unclear risk | No description |
| Blinding of participants and personnel (performance bias) All outcomes | Unclear risk | No description |
| Blinding of outcome assessment (detection bias) All outcomes | Unclear risk | No description |
| Incomplete outcome data (attrition bias) All outcomes | High risk | 4 pharmacies were excluded because of irregular opening hours. As a consequence, the other pharmacy in each pair was also excluded as the results were analysed in pairs |
| Selective reporting (reporting bias) | Unclear risk | No description |
| Other bias | Low risk | The study appears to be free of other sources of bias |
| Similar baseline characteristics | Low risk | The pharmacies paired to ensure a similar composition for intervention and control groups |
| Similar baseline outcome measures | Unclear risk | No description |
| Reliable primary outcome measures | Unclear risk | No description |
| Adequate protection against contamination | Low risk | To prevent the client’s individual behaviour from affecting the pharmacy staff, all clients were trained to act in a reproducible way, and they were not informed about which pharmacies belonged to the intervention or control groups |
Ross‐Degnan 1996.
| Methods | Non‐randomised controlled trial conducted in Kenya Randomised controlled trial conducted in IndonesiaDate not stated |
|
| Participants | Number enrolled: 112 pharmacies in Kenya and 87 pharmacies in Indonesia Study population
Inclusion criteria: private pharmacy in the study area Withdrawals and exclusions: Four pharmacies in Kenya and five pharmacies in Indonesia were excluded because the analyses were limited to the outcomes that were measured both before and after the intervention |
|
| Interventions | Intervention: training in 82 pharmacies in Kenya and 43 pharmacies in Indonesia Duration: 1 day in Kenya and 2 days in Indonesia Who delivered the intervention: Control of Diarrhoea Diseases (CDD) programmes in the Ministry of Health Description: in both countries, the intervention began with brief one‐on‐one meetings with pharmacists/pharmacy owners discussing key training messages and ways to deal with perceived barriers to practice recommendations, followed by training of all counter attendants in group sessions of 5‐10 attendees organised close to their pharmacies. To measure changes in treatment practice, all pharmacies included in the study in both countries were visited by surrogate patients. Comparison: the no‐training control group consisted of 25 pharmacies in Kenya and 44 pharmacies in Indonesia |
|
| Outcomes | Proportion of cases who received oral rehydration salts (ORS) | |
| Notes | ||
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | No description |
| Allocation concealment (selection bias) | Unclear risk | No description |
| Blinding of participants and personnel (performance bias) All outcomes | Low risk | Trainees and pharmacy owners were not informed that the sales practices in their pharmacies would be evaluated |
| Blinding of outcome assessment (detection bias) All outcomes | Low risk | Outcomes were assessed by surrogate patients who were blind to the purpose of the study, and to the study or control status of the pharmacies |
| Incomplete outcome data (attrition bias) All outcomes | Unclear risk | No description |
| Selective reporting (reporting bias) | Unclear risk | No description |
| Other bias | Low risk | The study appears to be free of other sources of bias |
| Similar baseline characteristics | Low risk | There were similar baseline characteristics |
| Similar baseline outcome measures | Unclear risk | No description |
| Reliable primary outcome measures | Unclear risk | No description |
| Adequate protection against contamination | Unclear risk | No description |
Stenson 2001.
| Methods | Randomised controlled trial conducted from June 1997 to March 1999 in the Lao People's Democratic Republic (Lao P.D.R.) | |
| Participants | Number enrolled: 115 pharmacies Study population: the study was based on licensed private pharmacies in Savannakhet province, Lao P.D.R. All pharmacies except 2 belonged to class 3, the lowest class of licensed pharmacies, where the licensee does not have to be a pharmacist or an assistant pharmacist. Most of them had a nursing background Inclusion criteria: operating pharmacy within the province Exclusion criteria: the pharmacies that were missed in the (post‐intervention analysis) were excluded from the analysis Withdrawals and exclusions: 14 pharmacies were missed during study and the reasons for missing pharmacies were mortality among drug sellers or that the pharmacy had moved or was closed for family reasons |
|
| Interventions | Intervention: active regulation in 46 pharmacies Duration: One and half years Who delivered the intervention: Ministry of Health with assistance from UNICEF Description: the regulatory intervention involved three month's intensive supervision of the quality of pharmacy services, applying sanctions when rules were violated, and providing up‐to‐date regulatory documents and information about particular areas needing improvements. The study compared districts with active regulation to districts with regular care Comparison: 92 control pharmacies received regular care |
|
| Outcomes | Change of the quality of private pharmacy practice (pharmacy indicators) | |
| Notes | ||
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | No description |
| Allocation concealment (selection bias) | Unclear risk | No description |
| Blinding of participants and personnel (performance bias) All outcomes | Unclear risk | It was not possible to make the study blinded with regard to the main intervention vehicle, the research assistant. It could not be established to what extent the drug sellers had any active knowledge of the study objectives |
| Blinding of outcome assessment (detection bias) All outcomes | Unclear risk | The district pharmacists who were active partners in the research team and participated in the sampling procedures and assessing the outcome were also aware of the scope of the intervention |
| Incomplete outcome data (attrition bias) All outcomes | High risk | 14 pharmacies were missed during the study and the reasons for missing pharmacies were mortality among drug sellers or that the pharmacy had moved or was closed for family reasons |
| Selective reporting (reporting bias) | Unclear risk | No description |
| Other bias | Low risk | The study appears to be free of other sources of bias |
| Similar baseline characteristics | Low risk | There were similar baseline characteristics i.e. income levels and literacy |
| Similar baseline outcome measures | Unclear risk | No description |
| Reliable primary outcome measures | Unclear risk | No description |
| Adequate protection against contamination | Unclear risk | No description |
UNICEF: The United Nations Children's Fund WHO: World Health Organization
Characteristics of excluded studies [ordered by study ID]
| Study | Reason for exclusion |
|---|---|
| Akol 2014 | This was a cross‐sectional study |
| Akoria 2008 | This was a controlled before‐after study but the participants were not private‐for‐profit providers |
| Ali 2011 | This was a cross‐sectional study |
| Ali 2012 | This was a cross‐sectional survey |
| Andrianasolo 2012 | This was a prospective descriptive study |
| Bhat 1996 | This was a cross‐sectional study |
| Bin 2013 | A controlled before‐after study with single unit for intervention and control arms |
| Bojalil 1999 | This was a controlled before‐after study and the participants were described as private providers, but it is not clear if these were for‐profit or not‐for‐profit healthcare providers |
| Chakraborty 2000 | This was an uncontrolled before‐after study design |
| Contiades 2007 | This was a descriptive study |
| Dholakia 2013 | This was a descriptive study of an intervention administered through a non‐governmental organisation |
| Farsi 1999 | This was a randomised trial but its not clear who delivered the intervention |
| Fernandes 2009 | This was a before‐after study and the intervention was undertaken among school children and not healthcare providers |
| Goodman 2007 | This was a cross‐sectional study |
| Grundy 2010 | This was a cross‐sectional study |
| Guiscafre 2001 | This was a cross‐sectional study |
| Harrison 2000 | This was a randomised trial; but the intervention was not implemented by the public sector, and participants were not private‐for‐profit providers |
| Hongoro 2000 | This was a cross‐sectional study |
| Hulda 2015 | A controlled before‐after study with single unit for intervention and control arms |
| Kangwana 2011 | This was a randomised trial; but the intervention was not implemented by the public sector, and participants were not private‐for‐profit providers |
| Khan 2006 | This was a cross‐sectional study |
| Kumaranayake 2000 | This was a cross‐sectional study |
| Maiga 2010 | This was a cross‐sectional study |
| Marsh 2004 | This was a cross‐sectional study |
| Minh 2013 | This was a cross‐sectional study |
| Morris 2013 | This was an uncontrolled before‐after study |
| Murugesan 2009 | This was a cross‐sectional study |
| Nsimba 2007 | This was a cross‐sectional study |
| Obua 2004 | This was a controlled before‐after study, but it is not clear whether the participants were private not‐for‐profit or private for‐profit providers |
| Okonofua 2003 | This was a randomised trial, but the intervention was offered by the private sector rather than the public sector |
| Osterholt 2009 | This was a time series study, with one measure before and two measures after introduction of the intervention. This does not meet our criteria for eligible interrupted time series studies i.e. at least three measurements before and after introducing the intervention |
| Rutta 2011 | This was a descriptive pilot study |
| Rutta 2015 | This was a prospective descriptive study |
| Stenson 2001b | This was a cross‐sectional study |
| Syhakhang 2001 | This was a cross‐sectional study |
| Tavrow 2003 | This was a cross‐sectional study |
| Tumwikirize 2004 | This was a non‐randomised trial but it is not clear who offered the intervention |
| Ward 2013 | This was an uncontrolled before‐after study |
| Willey 2014 | The intervention was not administered on participants |
Characteristics of studies awaiting assessment [ordered by study ID]
Ansah 2015.
| Methods | Cluster‐randomised trial in Dangme West, a poor rural district of Ghana, from August 2011‐January 2013 |
| Participants | Drug sellers in registered chemical shops within 24 community clusters |
| Interventions | Intervention: training on malaria rapid diagnostic tests and appropriate malaria case management Comparison: standard care |
| Outcomes | Change of practice in malaria diagnosis and antimalarial prescription |
| Notes |
El‐Khoury 2015.
| Methods | Randomised trial in two regions in Jordan, between January 2012 and January 2013 |
| Participants | 267 private doctors who provide family planning services in Amman and Zarqa regions in Jordan |
| Interventions | Intervention: evidence‐based medicine seminar on depot medroxy progesterone acetate and two educational visits to reinforce the messages from the seminar Comparison: invited to the seminar, but were not offered the educational visits |
| Outcomes | Providers’ knowledge, attitudes, practices regarding depot medroxy progesterone acetate |
| Notes |
Kafle 2013.
| Methods | Randomised trial in three geographical regions of Nepal |
| Participants | 342 drug sellers in private pharmacies in 12 districts in Nepal |
| Interventions | Intervention 1: mailed printed educational materials followed by mailed feedback Intervention 2: small group training followed by feedback Intervention 3: small group training only Comparison group: no intervention |
| Outcomes | Quality of management of acute respiratory infections and diarrhoea in children, and anaemia in pregnancy |
| Notes |
Mbonye 2015.
| Methods | Cluster‐randomised trial in Mukono district, central Uganda, from October 2010‐July 2012 |
| Participants | Drug sellers in 20 geographical clusters of registered drug shops |
| Interventions | Intervention: training to perform malaria rapid diagnostic tests to inform malaria case management Comparison: current practice of presumptive clinical diagnosis and treatment of fever |
| Outcomes | Proportion of patients receiving appropriate treatment with artemisinin‐based combination therapy |
| Notes |
Santa‐Ana‐Tellez 2016.
| Methods | Interrupted time series among private sectors in Mexico and Brazil from the first quarter of 2007 to the first quarter of 2013 |
| Participants | Described as private providers |
| Interventions | Over‐the‐counter antibiotic sales restrictions |
| Outcomes | Changes in the use of non‐steroidal anti‐inflammatory drugs, non‐opioid analgesics, and cough and cold medicines and their relation with the use of antibiotics |
| Notes |
Differences between protocol and review
None
Contributions of authors
Charles Wiysonge and Leila Abdullahi conceived the review, were involved in all stages of the study, and share joint first authorship. All authors participated in the preparation and final approval of the review for publication.
Sources of support
Internal sources
University of Cape Town, South Africa.
External sources
-
Norwegian Knowledge Centre for the Health Sciences, Norway.
Leila Abdullahi received a travel grant to spend one week and finalise the review, at the Norwegian Knowledge Centre for the Health Sciences.
Declarations of interest
Charles S Wiysonge: none known
Leila H Abdullahi: none known
Valantine N Ndze: none known
Gregory D Hussey: none known
Edited (no change to conclusions)
References
References to studies included in this review
Abuya 2009 {published data only}
- Abuya T, Fegan G, Rowa Y, Karisa B, Ochola S, Mutemi W, et al. Impact of ministry of health interventions on private medicine retailer knowledge and practices on anti‐malarial treatment in Kenya. American Journal of Tropical Medicine and Hygiene 2009;80(6):905‐13. [PubMed] [Google Scholar]
Chalker 2002 {published data only}
- Chalker J, Chuc NTK, Falkenberg T, Tomson G. Private pharmacies in Hanoi, Vietnam: a randomized trial of a 2‐year multi‐component intervention on knowledge and stated practice regarding ARI, STD and antibiotic/steroid requests. Tropical Medicine & International Health 2002;7(9):803‐10. [DOI] [PubMed] [Google Scholar]
Chalker 2005 {published data only}
- Chalker J, Ratanawijitrasin S, Chuc NTK, Petzold M, Tomson G. Effectiveness of a multi‐component intervention on dispensing practices at private pharmacies in Vietnam and Thailand ‐ a randomized controlled trial. Social Science & Medicine 2005;60(1):131‐41. [DOI] [PubMed] [Google Scholar]
Chuc 2002 {published data only}
- Chuc NTK, Larsson M, Do NT, Diwan VK, Tomson GB, Falkenberg T. Improving private pharmacy practice: a multi‐intervention experiment in Hanoi, Vietnam. Journal of Clinical Epidemiology 2002;55(11):1148‐55. [DOI] [PubMed] [Google Scholar]
Ross‐Degnan 1996 {published data only}
- Ross‐Degnan D, Soumerai SB, Goel PK, Bates J, Makhulo J, Dondi N, et al. The impact of face‐to‐face educational outreach on diarrhoea treatment in pharmacies. Health Policy and Planning 1996;11(3):308‐18. [DOI] [PubMed] [Google Scholar]
Stenson 2001 {published data only}
- Stenson B, Syhakhang L, Eriksson B, Tomson G. Real world pharmacy: assessing the quality of private pharmacy practice in the Lao People's Democratic Republic. Social Science & Medicine 2001;52(3):393‐404. [DOI] [PubMed] [Google Scholar]
- Stenson B, Syhakhang L, Lundborg CS, Eriksson B, Tomson G. Private pharmacy practice and regulation. International Journal of Technology Assessment in Health Care 2001;17(4):579‐89. [PubMed] [Google Scholar]
References to studies excluded from this review
Akol 2014 {published data only}
- Akol A, Chin‐Quee D, Wamala‐Mucheri P, Namwebya JH, Mercer SJ, Stanback J. Getting closer to people: family planning provision by drug shops in Uganda. Global Health Science & Practice 2014;2(4):472‐81. [DOI] [PMC free article] [PubMed] [Google Scholar]
Akoria 2008 {published data only}
- Akoria OA, Isah AO. Prescription writing in public and private hospitals in Benin City, Nigeria: the effects of an educational intervention. Canadian Journal of Clinical Pharmacology 2008;15(2):e295‐e305. [PubMed] [Google Scholar]
Ali 2011 {published data only}
- Ali GKM, Omer AM. The impact of the pharmaceutical regulations on the quality of medicines on the Sudanese market: Importers' perspective. Journal of Physiology and Pharmacology Advances 2012;2(1):26‐40. [Google Scholar]
Ali 2012 {published data only}
- Ali GKM, Omer AM. Pharmaceuticals in Sudan: development in regulations, governance and implementation of national drug policies. African Journal of Pharmacy and Pharmacology 2012;6(1):1‐12. [Google Scholar]
Andrianasolo 2012 {published data only}
- Andrianasolo RL, Rakotoson J, Rakotoarivelo RA, Andriamahenina R, Randria Mamy JD, Rapelanoro RF. Adhesion of physicians to the national malaria policy: situation in Antananarivo (Madagascar), 5 years after the policy revision. Medecine et Sante Tropicales 2012;22(2):177‐81. [DOI] [PubMed] [Google Scholar]
Bhat 1996 {published data only}
- Bhat R. Regulating the private health care sector: the case of the Indian Consumer Protection Act. Health Policy and Planning 1996;11(3):265‐79. [DOI] [PubMed] [Google Scholar]
Bin 2013 {published data only}
- Bin Ghouth AS. Availability and prescription practice of anti‐malaria drugs in the private health sector in Yemen. Journal of Infection in Developing Countries 2013;7(5):2013. [DOI] [PubMed] [Google Scholar]
Bojalil 1999 {published data only}
- Bojalil R, Guiscafre H, Espinosa P, Viniegra L, Martinez H, Palafox M, et al. A clinical training unit for diarrhoea and acute respiratory infections: an intervention for primary health care physicians in Mexico. Bulletin of the World Health Organization 1999;77(11):936‐45. [PMC free article] [PubMed] [Google Scholar]
Chakraborty 2000 {published data only}
- Chakraborty S, D'Souza SA, Northrup RS. Improving private practitioner care of sick children: testing new approaches in rural Bihar. Health Policy and Planning 2000;15(4):400‐7. [DOI] [PubMed] [Google Scholar]
Contiades 2007 {published data only}
- Contiades X, Golna C, Souliotis K. Pharmaceutical regulation in Greece at the crossroad of change: economic, political and constitutional considerations for a new regulatory paradigm. Health Policy 2007;82(1):116‐29. [DOI] [PubMed] [Google Scholar]
Dholakia 2013 {published data only}
- Dholakia YN. TB/ HIV coordination through public private partnership: lessons from the field. Indian Journal of Tuberculosis 2013;60(1):23‐7. [PubMed] [Google Scholar]
Farsi 1999 {published data only}
- Farsi NM. The effect of education upon dentists' knowledge and attitude toward fissure sealants. Odonto‐stomatologie tropicale 1999;22(86):27‐32. [PubMed] [Google Scholar]
Fernandes 2009 {published data only}
- Fernandes PS, Bernardo Cd, Campos RMMB, Vasconcelos FAG. Evaluating the effect of nutritional education on the prevalence of overweight/obesity and on foods eaten at primary schools. Jornal de Pediatria 2009;85(4):315‐21. [DOI] [PubMed] [Google Scholar]
Goodman 2007 {published data only}
- Goodman C, Kachur SP, Abdulla S, Bloland P, Mills A. Drug shop regulation and malaria treatment in Tanzania ‐‐ why do shops break the rules, and does it matter?. Health Policy and Planning 2007;22(6):393‐403. [DOI] [PMC free article] [PubMed] [Google Scholar]
Grundy 2010 {published data only}
- Grundy J. Country‐level governance of global health initiatives: an evaluation of immunization coordination mechanisms in five countries of Asia. Health Policy and Planning 2010;25(3):186‐96. [DOI] [PubMed] [Google Scholar]
Guiscafre 2001 {published data only}
- Guiscafre H, Martinez H, Palafox M, Villa S, Espinosa P, Bojalil R, et al. The impact of a clinical training unit on integrated child health care in Mexico. Bulletin of the World Health Organization 2001;79(5):434‐41. [PMC free article] [PubMed] [Google Scholar]
Harrison 2000 {published data only}
- Harrison A, Karim SA, Floyd K, Lombard C, Lurie M, Ntuli N, et al. Syndrome packets and health worker training improve sexually transmitted disease case management in rural South Africa: randomized controlled trial. AIDS 2000;14(17):2769‐79. [DOI] [PubMed] [Google Scholar]
Hongoro 2000 {published data only}
- Hongoro C, Kumaranayake L. Do they work? Regulating for‐profit providers in Zimbabwe. Health Policy and Planning 2000;15(4):368‐77. [DOI] [PubMed] [Google Scholar]
Hulda 2015 {published data only}
- Huda FA, Ahmed A, Ford ER, Johnston HB. Strengthening health systems capacity to monitor and evaluate programmes targeted at reducing abortion‐related maternal mortality in Jessore district, Bangladesh. BMC Health Services Research 2015;15:426. [DOI] [PMC free article] [PubMed] [Google Scholar]
Kangwana 2011 {published data only}
- Kangwana BP, Kedenge SV, Noor AM, Alegana VA, Nyandigisi AJ, Pandit J, et al. The impact of retail‐sector delivery of artemether‐lumefantrine on malaria treatment of children under five in Kenya: A cluster randomized controlled trial. PLoS Medicine 2011;8(5):e1000437. [DOI] [PMC free article] [PubMed] [Google Scholar]
Khan 2006 {published data only}
- Khan MMH, Wolter S, Mori M. Post‐training quality of syndromic management of sexually transmitted infections by chemists and druggists in Pokhara, Nepal: is it satisfactory?. International Journal for Quality in Health Care 2006;18(1):66‐72. [DOI] [PubMed] [Google Scholar]
Kumaranayake 2000 {published data only}
- Kumaranayake I, Lake S, Mujinja P, Hongoro C, Mpembeni R. How do countries regulate the health sector? Evidence from Tanzania and Zimbabwe. Health Policy and Planning 2000;15(4):357‐67. [DOI] [PubMed] [Google Scholar]
Maiga 2010 {published data only}
- Maiga D, Williams‐Jones B. Assessment of the impact of market regulation in Mali on the price of essential medicines provided through the private sector. Health Policy 2010;97(2‐3):130‐5. [DOI] [PubMed] [Google Scholar]
Marsh 2004 {published data only}
- Marsh VM, Mutemi WM, Willetts A, Bayah K, Were S, Ross A, et al. Improving malaria home treatment by training drug retailers in rural Kenya. Tropical Medicine & International Health 2004;9(4):451‐60. [DOI] [PubMed] [Google Scholar]
Minh 2013 {published data only}
- Minh PD, Huong DT, Byrkit R, Murray M. Strengthening pharmacy practice in Vietnam: findings of a training intervention study. Tropical Medicine & International Health 2013;18(4):426‐34. [DOI] [PubMed] [Google Scholar]
Morris 2013 {published data only}
- Morris A, Ali A, Moonen B, Msellem M, Al‐Mafazy AW, Mulokozi A, et al. Testing market‐based and regulatory strategies to minimize over treatment with anti‐malarial drugs in Zanzibar. American Journal of Tropical Medicine and Hygiene 2013;1:347. [Google Scholar]
Murugesan 2009 {published data only}
- Murugesan N, Shobana R, Snehalatha C, Kapur A, Ramachandran A. Immediate impact of a diabetes training programme for primary care physicians ‐ an endeavour for national capacity building for diabetes management in India. Diabetes Research and Clinical Practice 2009;83(1):140‐44. [DOI] [PubMed] [Google Scholar]
Nsimba 2007 {published data only}
- Nsimba SED. Assessing the impact of educational intervention for improving management of malaria and other childhood illnesses in Kibaha District‐Tanzania. East African Journal of Public Health 2007;4(1):5‐11. [PubMed] [Google Scholar]
Obua 2004 {published data only}
- Obua C, Ogwal‐Okeng JW, Waako P, Aupont O, Ross‐Degnan D. Impact of an educational intervention to improve prescribing by private physicians in Uganda. East African Medical Journal 2004;Suppl:S17‐S24. [PubMed] [Google Scholar]
Okonofua 2003 {published data only}
- Okonofua FE, Coplan P, Collins S, Oronsaye F, Ogunsakin D, Ogonor JT, et al. Impact of an intervention to improve treatment‐seeking behavior and prevent sexually transmitted diseases among Nigerian youths. International Journal of Infectious Diseases 2003;7(1):61‐73. [DOI] [PubMed] [Google Scholar]
Osterholt 2009 {published data only}
- Osterholt DM, Onikpo F, Lama M, Deming MS, Rowe AK. Improving pneumonia case‐management in Benin: a randomized trial of a multi‐faceted intervention to support health worker adherence to Integrated Management of Childhood Illness guidelines. Human Resources for Health 2009;7:77. [DOI] [PMC free article] [PubMed] [Google Scholar]
Rutta 2011 {published data only}
- Rutta E, Kibassa B, McKinnon B, Liana J, Mbwasi R, Mlaki W, et al. Increasing access to subsidized artemisinin‐based combination therapy through accredited drug dispensing outlets in Tanzania. Health Research Policy and Systems 2011;9(9):22. [DOI] [PMC free article] [PubMed] [Google Scholar]
Rutta 2015 {published data only}
- Rutta E, Liana J, Embrey M, Johnson K, Kimatta S, Valimba R, et al. Accrediting retail drug shops to strengthen Tanzania's public health system: an ADDO case study. Journal of Pharmaceutical Policy & Practice 2015;8:23. [DOI] [PMC free article] [PubMed] [Google Scholar]
Stenson 2001b {published data only}
- Stenson B, Syhakhang L, Eriksson B, Tomson G. Real world pharmacy: assessing the quality of private pharmacy practice in the Lao People's Democratic Republic. Social Science & Medicine 2001;52(3):393‐404. [DOI] [PubMed] [Google Scholar]
Syhakhang 2001 {published data only}
- Syhakhang L, Stenson B, Wahlstrom R, Tomson G. The quality of public and private pharmacy practices. European Journal of Clinical Pharmacology 2001;57(3):221‐7. [DOI] [PubMed] [Google Scholar]
Tavrow 2003 {published data only}
- Tavrow P, Shabahang J, Makama S. Vendor‐to‐vendor education to improve malaria treatment by private drug outlets in Bungoma District, Kenya. Malaria Journal 2003;2:10. [DOI] [PMC free article] [PubMed] [Google Scholar]
Tumwikirize 2004 {published data only}
- Tumwikirize WA, Ekwaru PJ, Mohammed K, Ogwal‐Okeng JW, Aupont O. Impact of a face‐to‐face educational intervention on improving the management of acute respiratory infections in private pharmacies and drug shops in Uganda. East African Medical Journal 2004;Suppl:S25‐S32. [PubMed] [Google Scholar]
Ward 2013 {published data only}
- Ward A, Morris A, Lesage PL, Miller A, Lam F, Cohen J. Low‐cost educational training causes sustained improvement in prescribing and stocking of artemisinin‐based drugs in Madagascar. American Journal of Tropical Medicine and Hygiene 2013;1:347. [Google Scholar]
Willey 2014 {published data only}
- Willey BA, Tougher S, Ye Y, Act watch Group, Mann AG, Thomson R, et al. Communicating the AMFm message: exploring the effect of communication and training interventions on private for‐profit provider awareness and knowledge related to a multi‐country anti‐malarial subsidy intervention. Malaria Journal 2014;13:46. [DOI] [PMC free article] [PubMed] [Google Scholar]
References to studies awaiting assessment
Ansah 2015 {published data only}
- Ansah EK, Narh‐Bana S, Affran‐Bonful H, Bart‐Plange C, Cundill B, Gyapong M, et al. The impact of providing rapid diagnostic malaria tests on fever management in the private retail sector in Ghana: a cluster randomized trial. British Medical Journal 2015;350:h1019. [DOI] [PMC free article] [PubMed] [Google Scholar]
El‐Khoury 2015 {published data only}
- El‐Khoury M, Thornton R, Chatterji M, Choi SK. Effectiveness of evidence‐based medicine on knowledge, attitudes, and practices of family planning providers: a randomized experiment in Jordan. BMC Health Services Research 2015;15:449. [DOI] [PMC free article] [PubMed] [Google Scholar]
Kafle 2013 {published data only}
- Kafle KK, Karkee SB, Shrestha N, Prasad RR, Bhuju GB, Das PL, et al. Improving private drug sellers' practices for managing common health problems in Nepal. Journal of Nepal Health Research Council 2013;11(24):198‐204. [PubMed] [Google Scholar]
Mbonye 2015 {published data only}
- Mbonye AK, Clarke SE, Lal S, Chandler CI, Hutchinson E, Hansen KS, et al. Introducing rapid diagnostic tests for malaria into registered drug shops in Uganda: lessons learned and policy implications. Malaria Journal 2015;14(1):448. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mbonye AK, Magnussen P, Lal S, Hansen SK, Cundil B, Chandler C, et al. A cluster randomised trial introducing rapid diagnostic tests into registered drug shops in Uganda: impact on appropriate treatment of malaria. PLoS One 2015;10(7):e0129545. [DOI] [PMC free article] [PubMed] [Google Scholar]
Santa‐Ana‐Tellez 2016 {published data only}
- Santa‐Ana‐Tellez Y, Mantel‐Teeuwisse AK, Leufkens HG, Wirtz VJ. Effects of over‐the‐counter sales restriction of antibiotics on substitution with medicines for symptoms relief of cold in Mexico and Brazil: time series analysis. Health Policy and Planning 2016;26:26. [DOI] [PubMed] [Google Scholar]
Additional references
Balshem 2011
- Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines 3: rating the quality of evidence ‐ introduction. Journal of Clinical Epidemiology 2011;64:401‐6. [DOI] [PubMed] [Google Scholar]
Basu 2012
- Basu S, Andrews J, Kishore S, Panjabi R, Stuckler D. Comparative performance of private and public healthcare systems in low‐ and middle‐income countries: a systematic review. PLoS Medicine 2012;9(6):e1001244. [DOI: ] [DOI] [PMC free article] [PubMed] [Google Scholar]
Berendes 2011
- Berendes S, Heywood P, Oliver S, Garner P. Quality of private and public ambulatory health care in low and middle income countries: systematic review of comparative studies. PLoS Medicine 2011;8(4):e1000433. [DOI] [PMC free article] [PubMed] [Google Scholar]
Brugha 1998
- Brugha R, Zwi A. Improving the quality of private sector delivery of public health services: challenges and strategies. Health Policy and Planning 1998;13:107‐20. [DOI] [PubMed] [Google Scholar]
Deeks 2011
- Deeks JJ, Higgins JPT, Altman DG (editors). Chapter 9: Analysing data and undertaking meta‐analyses. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). Cochrane, 2011. Available from handbook.cochrane.org.
EPOC 2013
- Cochrane Effective Practice, Organization of Care (EPOC). What study designs should be included in an EPOC review?. EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2013. Available at: http://epoc.cochrane.org/epoc‐specific‐resources‐review‐authors.
EPOC 2015
- Effective Practice, Organisation of Care (EPOC). Suggested risk of bias criteria for EPOC reviews. EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services. Available at: http://epoc.cochrane.org/epoc‐specific‐resources‐review‐authors.
Forsberg 2011
- Forsberg BC, Montagu D, Sundewall J. Moving towards in‐depth knowledge on the private health sector in low and middle‐income countries. Health Policy and Planning 2011;26:i1‐i3. [DOI] [PubMed] [Google Scholar]
Forsetlund 2009
- Forsetlund L, Bjørndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf F, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2009, Issue 2. [DOI: 10.1002/14651858.CD003030.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Guyatt 2008
- Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck‐Ytter Y, Alonso‐Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. British Medical Journal 2008;336:924‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]
Higgins 2003
- Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta‐analyses. British Medical Journal 2003;327(7414):557‐60. [DOI] [PMC free article] [PubMed] [Google Scholar]
Higgins 2011
- Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. Cochrane, 2011. Available from handbook.cochrane.org.
Higgins 2011a
- Higgins JPT, Altman DG, Sterne JAC (editors). Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). Cochrane, 2011. Available from handbook.cochrane.org.
Jamtvedt 2006
- Jamtvedt G, Young JM, Kristoffersen DT, O'Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2010, Issue 2. [DOI: 10.1002/14651858.CD000259.pub2] [DOI] [Google Scholar]
Koehlmoos 2009
- Koehlmoos TP, Gazi R, Hossain SS, Zaman K. The effect of social franchising on access to and quality of health services in low‐ and middle‐income countries. Cochrane Database of Systematic Reviews 2009, Issue 1. [DOI: 10.1002/14651858.CD007136.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Lagarde 2009
- Lagarde M, Palmer N. The impact of contracting out on health outcomes and use of health services in low and middle‐income countries. Cochrane Database of Systematic Reviews 2009, Issue 4. [DOI: 10.1002/14651858.CD008133] [DOI] [PubMed] [Google Scholar]
Lagomarsino 2009
- Lagomarsino G, Ferranti D, Pablos‐Mendez A, Nachuk S, Nishtar S, Wibulpolprasert S. Public stewardship of mixed health systems. Lancet 2009;374:1577‐8. [DOI] [PubMed] [Google Scholar]
Levin 2011
- Levin A, Kaddar M. Role of the private sector in the provision of immunization services in low‐ and middle‐income countries. Health Policy and Planning 2011;26:i4‐i12. [DOI] [PubMed] [Google Scholar]
O'Brien 2007
- O'Brien MA, Rogers S, Jamtvedt G, Oxman AD, Odgaard‐Jensen J, Kristoffersen DT, et al. Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2007, Issue 4. [DOI: 10.1002/14651858.CD000409.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
Patouillard 2007
- Patouillard E, Goodman CA, Hanson KG, Mills AJ. Can working with the private for‐profit sector improve utilization of quality health services by the poor? A systematic review of the literature. International Journal for Equity in Health 2007;6:17. [DOI: 10.1186/1475-9276-6-17] [DOI] [PMC free article] [PubMed] [Google Scholar]
Peters 2004
- Peters DH, Mirchandani GG, Hansen PM. Strategies for engaging the private sector in sexual and re‐productive health: how effective are they?. Health Policy and Planning 2004;19(Suppl 1):i5‐i21. [DOI] [PubMed] [Google Scholar]
RevMan 2014 [Computer program]
- The Nordic Cochrane Centre, The Cochrane Collaboration. Review Manager (RevMan). Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014.
Scott 2011
- Scott A, Sivey P, Ait Ouakrim D, Willenberg L, Naccarella L, Furler J, et al. The effect of financial incentives on the quality of health care provided by primary care physicians. Cochrane Database of Systematic Reviews 2011, Issue 9. [DOI: 10.1002/14651858.CD008451] [DOI] [PubMed] [Google Scholar]
Sulzbach 2011
- Sulzbach S, De S, Wang W. The private sector role in HIV/AIDS in the context of an expanded global response: expenditure trends in five sub‐Saharan African countries. Health Policy and Planning 2011;26:i72‐i84. [DOI] [PubMed] [Google Scholar]
Travis 2002
- Travis P, Egger D, Davies P, Mechbal A. Towards better stewardship: concepts and critical issues. Evidence and Information for Policy. Geneva: World Health Organization, 2002 (Document WHO/EIP/DP/02.48). who.int/healthinfo/paper48.pdf (accessed 27 July 2016).
Van Olmen 2010
- Olmen J, Criel B, Damme W, Marchal B, Belle S, Dormael M, et al. Analysing Health Systems to Make Them Stronger. Studies in Health Services Organisation & Policy, 27, 2010. Antwerp: ITG Press.
Veillard 2011
- Veillard JHM, Brown AD, Baris E, Permanand G, Klazinga NS. Health system stewardship of National Health Ministries in the WHO European region: concepts, functions and assessment framework. Health Policy 2011;103(2‐3):191‐9. [DOI] [PubMed] [Google Scholar]
Waters 2003
- Waters H, Hatt L, Peters D. Working with the private sector for child health. Health Policy and Planning 2003;18:127‐37. [DOI] [PubMed] [Google Scholar]
WHO 2007
- World Health Organization. Everybody’s business ‐ strengthening health systems to improve health outcomes: WHO’s framework for action. Geneva: World Health Organization, 2007. who.int/healthsystems/strategy/everybodys_business.pdf (accessed 27 July 2016).
WHO 2009
- WHO report 2009. The European health report; health and health system. euro.who.int (accessed 16 November 2011).
Witter 2012
- Witter S, Fretheim A, Kessy FL, Lindahl AK. Paying for performance to improve the delivery of health interventions in low‐ and middle‐income countries. Cochrane Database of Systematic Reviews 2012, Issue 2. [DOI: 10.1002/14651858.CD007899] [DOI] [PubMed] [Google Scholar]
Wiysonge 2008
- Wiysonge CS. Can working with private for‐profit providers improve utilization and quality of health services for the poor? A SUPPORT Summary of a systematic review, August 2008. supportsummaries.org (accessed 24 May 2013).
References to other published versions of this review
Abdullahi 2012
- Abdullahi LH, Hussey GD, Mahomed H, Wiysonge CS. Public stewardship of private for‐profit health care in low‐ and middle‐income countries. Cochrane Database of Systematic Reviews 2012, Issue 5. [DOI: 10.1002/14651858.CD009855] [DOI] [PMC free article] [PubMed] [Google Scholar]