Interventions for increasing the proportion of health professionals practising in rural and other underserved areas

Abstract

Background

The inequitable distribution of health professionals, within countries, poses an important obstacle to the optimal functioning of health services.

Objectives

To assess the effectiveness of interventions aimed at increasing the proportion of health professionals working in rural and other underserved areas.

Search methods

We searched the Cochrane Central Register of Controlled Trials (CENTRAL, including specialised register of the Cochrane Effective Practice and Organisation of Care Group; March 2014), MEDLINE (1966 to March 2014), EMBASE (1988 to March 2014), CINAHL (1982 to March 2014), LILACS (February 2014), Science Citation Index and Social Sciences Citation Index (up to April 2014), Global Health (March 2014) and the World Health Organization (WHO) International Clinical Trials Registry Platform (ICTRP) (June 2013). We also searched reference lists of all papers and relevant reviews identified, and contacted authors of relevant papers regarding any further published or unpublished work.

Selection criteria

Randomised trials, non‐randomised trials, controlled before‐and‐after studies and interrupted time series studies evaluating the effects of various interventions (e.g. educational, financial, regulatory or support strategies) on the recruitment or retention, or both, of health professionals in underserved areas.

Data collection and analysis

Two review authors independently screened titles and abstracts and assessed full texts of potentially relevant studies for eligibility. Two review authors independently extracted data from eligible studies.

Main results

For this first update of the original review, we screened 8945 records for eligibility. We retrieved and assessed the full text of 125 studies. Only one study met the inclusion criteria of the review. This interrupted time series study, conducted in Taiwan, found that the implementation of a National Health Insurance scheme in 1995 was associated with improved equity in the geographic distribution of physicians and dentists. We judged the certainty of the evidence provided by this one study very low.

Authors' conclusions

There is currently limited reliable evidence regarding the effects of interventions aimed at addressing the inequitable distribution of health professionals. Well‐designed studies are needed to confirm or refute findings of observational studies of educational, financial, regulatory and supportive interventions that might influence healthcare professionals' decisions to practice in underserved areas. Governments and medical schools should ensure that when interventions are implemented, their impacts are evaluated using scientifically rigorous methods to establish the true effects of these measures on healthcare professional recruitment and retention in rural and other underserved settings.

Plain language summary

Interventions for increasing the proportion of health professionals practicing in rural and underserved areas

What was the aim of this review?

The aim of this Cochrane review was to find out whether strategies to increase the number of health professionals working in rural and underserved areas are effective. Researchers in The Cochrane Collaboration searched for all new studies that could answer this question, given that the previous review done in 2009 did not identify any high certainty evidence.

What did the review conclude?

We were only able to identify one new study with very low certainty of evidence. Therefore, we remain uncertain about the effects of these types of strategies. Well conducted interventional studies with appropriate control or comparison groups are needed to assess critically the impact of strategies to increase the number of health professionals working in underserved areas.

What was studied in the review?

All countries have areas that are relatively underserved by health professionals; mostly among rural communities and the urban poor. This problem is seen across the world, but is particularly serious in low‐ and middle‐income countries.

While some health professionals choose to work in underserved areas, we need strategies to persuade more of them to do so. Different strategies have been used. For example, some medical schools try to recruit more students with a rural background or arrange for students to practice in rural settings in the hope that graduates will return to those areas to practice. Scholarships, grants, loan repayment schemes and higher salaries have been offered to attract health professionals to underserved areas, while countries also utilise mandatory redeployment strategies. Although some strategies have shown promise, there is a lack of well‐designed studies to measure the short‐term and long‐term effectiveness of these strategies in a rigorous fashion.

What are the main results of the review?

This review found one new study that evaluated the effect of a National Health Insurance scheme on the distribution of health professionals in Taiwan. Before the implementation of National Health Insurance, people living in rural areas were less able to afford medical costs than their urban counterparts. The implementation of a National Health Insurance scheme made medical care more affordable for all Taiwanese citizens in both urban and rural areas. This may have led to better geographical distribution of health professionals. However, we assessed the certainty of the evidence to be very low. Therefore, we are uncertain about the effects of National Health Insurance schemes on the distribution of health professionals.

How up‐to‐date is this review?

The review authors searched for studies that had been published up to April 2014.

A summary of this review for policy‐makers is availablehere

Summary of findings

Summary of findings for the main comparison. Implementation of National Health Insurance scheme on equity of physician distribution.

Implementation of National Health Insurance scheme on equity of physician distribution
Patient or population: Western medicine physicians, Chinese medicine physicians and dentists Settings: Taiwan Intervention: National Health Insurance scheme Comparison: no National Health Insurance scheme
Outcomes	Impact1	No of participants (studies)2	Certainty of the evidence (GRADE)3
Equity of distribution of Western medicine physicians post implementation (Gini coefficient)	At 1 year post implementation; estimate, standard error: ‐0.008, 0.004, P value = 0.071 At 4 years post implementation; estimate, standard error: ‐0.018, 0.003, P value = 0.00 At 8 years post implementation; estimate, standard error: ‐0.032, 0.005, P value = 0.00 Although these results suggest a reduction in the inequitable distribution of these health professionals, these changes were small	42.68‐136.40 per 100,000 population (1 study)	⊕⊖⊖⊖ Very low
Equity of distribution of Chinese medicine physicians post implementation (Gini coefficient)	At 1 year post implementation; estimate, standard error: ‐0.007, 0.013, P value = 0.589 At 4 years post implementation; estimate, standard error: ‐0.016, 0.011, P value = 0.172 At 8 years post implementation; estimate, standard error: ‐0.025, 0.016, P value = 0.143 Although these results suggest a reduction in the inequitable distribution of these health professionals, these changes were small	9.81‐17.76 per 100,000 population (1 study)	⊕⊖⊖⊖ Very low
Equity of distribution of dentists post implementation (Gini coefficient)	At 1 year post implementation; estimate, standard error: ‐0.013, 0.017, P value = 0.444) At 4 years post implementation; estimate, standard error: ‐0.039, 0.03, P value = 0.196 At 8 years post implementation; estimate, standard error: ‐0.073, 0.056, P value = 0.201 Although these results suggest a reduction in the inequitable distribution of these health professionals, these changes were small	6.09‐39.92 per 100,000 population (1 study)	⊕⊖⊖⊖ Very low
GRADE Working Group grades of evidence High certainty: This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different* is low. Moderate certainty: This research provides a good indication of the likely effect. The likelihood that the effect will be substantially different* is moderate. Low certainty: This research provides some indication of the likely effect. However, the likelihood that it will be substantially different* is high. Very low certainty: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different* is very high. *Substantially different = a large enough difference that it might affect a decision.

1. The Gini coefficient provides a measure for the extent of distribution equity. Its value lies between 0 (no inequality) and 1 (complete inequality). A decrease in Gini coefficient towards 0 signifies an improvement in the equality of distribution.

2. Population: range in number of physicians (Western medicine, Chinese medicine and dentists) per 100,000 population in Taiwan from 1971‐2001.

3. GRADE: interrupted time series analysis study, therefore, certainty of the evidence started at low. Certainty of the evidence further downgraded for indirectness as the study was conducted in one country and the outcomes may not be generalisable or transferrable to other settings.

Background

Description of the condition

Inequitable distribution of healthcare professionals within countries contributes to poorly functioning health services in many parts of the world (WHO 2006; WHO 2010). Globally, 50% of the population live in rural areas and are served by 23% of the global health workforce, 38% of the total nursing workforce and 24% of the total physician workforce (Scheil‐Adlung 2015; WHO 2006; WHO 2010). In Canadian metropolitan areas there are approximately 2.6 physicians per 1000 people, whereas in Canadian rural areas there are 0.9 physicians per 1000 people (Viscomi 2013). Similarly in Australia there are 4.0 doctors per 1000 people in the major cities and 2.4 doctors per 1000 people in the remote/very remote areas of the country (National Rural Health Alliance 2013). Statistics from 2010 from China show a similar rural/urban disparity in the distribution of health‐care professionals with 3.04 health technicians per 1000 people in rural areas and 7.62 health technicians per 1000 people in urban areas (Chen 2014).

In low‐ and middle‐income countries (LMIC), inequities in healthcare service provision are even greater. For instance, in South Africa, 44% of the population live in rural areas and are served by only 12% of doctors and 19% of nurses (Hamilton 2004; SA HRH strategy 2013; Statistics SA 1996). In Ghana, 87% of general physicians worked in urban areas serving only 44% of the population (Dussault 2006). Furthermore, in Senegal, 60% of the country's physicians work in Dakar where only 23% of the population live (Zurn 2010).

The inequitable distribution of healthcare professionals between countries resulting from a combination of under‐production of healthcare professionals in poorer countries and net migration to the economically more developed parts of the world is also a cause for concern (Mullan 2005). Thirty‐seven per cent of the world's health workers work in the World Health Organization (WHO) region of the Americas, which accounts for only 10% of the global burden of disease. By contrast, the WHO African region, with 24% of the global burden of disease, is served by only 3% of the world's health workers. Furthermore, the African continent, with the highest global disease burden, has the lowest density of healthcare professionals (2.3 healthcare professionals per 1000 population) and spends the lowest proportion (29.5%) of government health expenditure on health worker salaries (WHO 2006). The emigration of skilled professionals from LMICs to high‐income countries (HICs), while alleviating the shortage of healthcare professionals in HICs, contributes to human resource shortages in the health systems of poorer countries. One Cochrane review examined the effect of interventions to reduce the emigration of healthcare professionals from LMICs to HICs (Penaloza 2011), therefore we will not consider these interventions in our review.

The shortage of healthcare professionals compounds the problem of inequitable distribution. Based on a threshold of 41.1 health workers per 10,000 population, there is a global deficit of 10.3 million health workers, seven million of whom are needed in rural areas compared to the three million needed in urban areas (Scheil‐Adlung 2015). In Africa, 77% of the rural population, compared to 50% of the urban population, lacks access to health care due to the shortage of skilled health workers (Scheil‐Adlung 2015). It has been estimated that by 2015, the target date of the United Nations Millennium Development Goals, there will be a shortage of 800,000 healthcare professionals (doctors, nurses and midwives) in 31 African countries (Scheffler 2009).

Possible factors contributing to the reduced number of healthcare professionals in rural and urban underserved areas include demanding working conditions, substandard medical equipment and facilities, inadequate financial remuneration, inadequate opportunities for personal and professional development, safety concerns, and lack of job opportunities for spouse and educational opportunities for children (Lehmann 2008). It has been suggested that a country's ability to recruit and retain healthcare professionals in underserved areas ultimately depends upon the provision of a stable, rewarding and fulfilling personal and professional environment (Hart 2002), yet the provision of such an environment continues to elude most countries, independent of their income status.

In resource‐constrained countries, rural and other underserved areas are generally worst afflicted by problems such as unemployment, poverty, malnutrition, lack of clean drinking water and poor sanitation (Boulle 1997). These factors often lie at the heart of the poor health experienced by people living in these areas (Strasser 2003).

The spatial maldistribution of health workers means that those who have the greatest need have the poorest services; thus fulfilling Hart's "inverse care law" (Hart 1971). Literature from sub‐Saharan Africa shows that treatment of people with human immunodeficiency virus (HIV)/acquired immunodeficiency syndrome (AIDS) has been severely hampered not merely by the lack of financial resources or medical supplies but by the shortage of healthcare personnel needed to deliver the treatment (WHO 2003). In Uganda, lack of skilled staff at the primary care level is one of the reasons why pregnant women do not utilise community maternity services, but resort to traditional birthing practices, which result in higher rates of maternal mortality (Kyomuhendo 2003). The current Ebola crisis in West Africa is a stark reminder of the importance of high‐quality health systems, of which sufficient and appropriately trained staff are an integral part. Before the Ebola outbreak, the entire population of Liberia (4.3 million people) was served by 51 physicians (Boozary 2014). In Sierra Leone, there are two physicians for every 100,000 people, compared to 245 physicians per 100,000 people in the USA (Rid 2014; WHO 2006). Such excessively overburdened health systems cannot deal with the everyday healthcare needs of the population, let alone the outbreak of a deadly virus such as Ebola.

Description of the intervention

While some healthcare professionals choose to work in underserved areas, they remain a small minority and additional strategies are required to persuade more healthcare professionals to do so. These strategies can generally be grouped into educational, financial, regulatory and supportive strategies.

Educational strategies

Targeted admission policies for medical schools

Numerous observational studies show that healthcare professionals originating from rural areas are more likely to practise in a rural healthcare environment (Rabinowitz 1993; Rabinowitz 1998; Stearns 2000; Wilson 2009). Based on these findings the selection criteria of some medical schools and universities have been modified to ensure the recruitment of greater numbers of individuals from rural areas.

Location of medical schools and family medicine residency programmes

Observational studies have reported increases in rural health workforce following the establishment of medical schools or medical school campuses in rural areas (Crump 2013; MacDowell 2013; Wilkinson 2004).

Undergraduate and postgraduate training exposure

Some medical school curricula specifically involve greater exposure of undergraduate students to underserved areas (Moores 1998), and concentrate on the development of healthcare skills relevant to these areas (Rabinowitz 1999). Financial, cultural and academic support is provided during their medical training in the hope that once qualified they will return to work in rural areas (Crandall 1990; Rabinowitz 2000).

Financial incentives

Financial mechanisms that have been implemented to reduce inequitable distribution patterns include scholarships and loan repayment schemes, which are linked to service obligations in underserved areas (Pathman 2000; Scammon 1994). In addition, higher salaries for individuals working in the public health sector, rural allowances (Ditlopo 2011; Reid 2001), and retention grants (Humphreys 2001) have been introduced in some countries.

Regulatory strategies

A number of countries have instituted a mandatory period of service in underserved areas in an attempt to distribute healthcare professionals in a more equitable manner (Cavender 1998; Ezenwa 1986; Fadayomi 1984; Frehywot 2010; Reid 2001). Many wealthy countries attempt to address the problem by simply recruiting foreign healthcare professionals from poorer countries to work in underserved areas (Hamilton 2004; Kiesouw 1996; Reid 2001), ignoring the negative impact on health services in their country of origin.

Personal and professional support

Various interventions have been implemented in an attempt to provide support to healthcare professionals working in rural and remote areas. Support strategies might include a functional work environment (i.e. hospital or clinic infrastructure and medical equipment in working order), effective management, supervision and mentoring, improved professional development (e.g. opportunities for both career development and continuing education), provision of locum relief, support from colleagues and networks of practitioners, and adequate housing and school facilities. One review summarised and appraised the effectiveness of such interventions on change in practitioner behaviour, service outcome or patient outcomes (Moran 2014).

How the intervention might work

Below we include a brief narrative overview of the strategies employed and how they might work, with further expansion and comment in Table 2.

1. Examples of current strategies to address the maldistribution of health professionals.

Hypothesis	Comments
Educational strategies
Students with a rural origin are more likely to practice in a rural setting.	That students with a rural origin are more likely to practice in a rural setting is a consistent finding in several observational studies. It appears to be the single factor most strongly associated with rural practice. Some studies found that the spouse's background is as important as that of the health professional as a predictor of practice location.
Students from 'underserved' populations are more likely to practice in these areas.	Students from 'underserved' populations are more likely to practice in these areas subsequently has been documented in 1 study in an inner‐city population in the US.
Men are more likely to practice rural medicine than women.	Men are more likely to practice rural medicine than women is a consistent finding in several observational studies. This finding may change if more accommodating conditions are created for women in these areas. It is important to have female rural doctors in culturally sensitive settings.
Students whose intent at study entry is to practice rural medicine are more likely to do so.	Students who expressed an intention at study entry to practice rural medicine found independently predict rural practice in the Physician Shortage Areas Program, but 60% of rural doctors in the US reported no such career intention initially.
Students who report involvement in volunteer activities are more likely to practice rural medicine.	In 1 study conducted at the University of North Carolina it was observed that the students who report involvement in volunteer activities were more likely to become generalists; however, this did not necessarily translate to them practicing in rural areas.
Emphasising the importance of rural health issues influences medical students to consider rural practice.	Exposing undergraduate students to health care in underserved areas is important to their overall training and development; it would an added bonus if it is found to encourage them to work in an underserved area subsequently.
Clinical rotation in a rural setting influences medical students to consider rural practice.	Observational studies have shown that actual clinical exposure (immersion) may be important, although the perceived impact of rural rotations may be biased by self selection (i.e. those students who do rural rotations would have chosen to do so prior to the intervention).
The availability of generalist fellowships encourages more doctors to enter rural practice. Undergraduate students from medical schools that offer generalist fellowships are more likely to become rural doctors.	Rural health specialists and family physicians may be more likely to enter rural practice, but there is no evidence that the creation/availability of these specialities actually reduces the rural urban maldistribution.
Students from medical schools located in rural areas are more likely to practice in a rural setting.	There is limited but consistent evidence that rural medical schools produce more rural doctors. However, rural placement may be a surrogate of various other factors, such as rural origin or likelihood to marry someone of rural origin. The establishment of more rural medical schools or satellite campuses warrant further evaluation.
Financial strategies
Providing scholarships with enforceable rural service agreements encourages rural practice.	This particular intervention has shown variable success in different countries. Furthermore, these strategies are dependent on the availability of funds to provide scholarships and rural allowances to health‐care professionals working in underserved areas. The long‐term benefit and cost effectiveness of these strategies requires further investigation.
Providing direct financial incentives encourages rural practice.	In Canada, allowing higher fees in rural areas is believed to have had a positive influence on general practitioner distribution. Reports from low‐ and middle‐income countries have also been positive. The sustainability of these strategies is dependent on the availability of funds to provide scholarships and rural allowances to health‐care professionals working in underserved areas and the long‐term benefit and cost effectiveness of these strategies requires further investigation.
Regulatory strategies
Requiring that recently qualified doctors perform 'community service' in underserved areas will reduce maldistribution.	Forced 'community service' addresses short‐term recruitment, but there is concern that it may alienate people from long‐term rural practice. This strategy should be adopted with great caution as newly graduated medical students are often ill equipped to deal with the lack of professional and personal support in underserved areas. Some may be able to cope but in the absence of adequate guidance and support many have negative experiences that alienate them from future rural practice.
Requiring that doctors spend a minimum number of years in an underserved area before allowing them to specialise will reduce maldistribution.	This intervention has been applied in a number of low‐ and middle‐income countries. It has been criticised in Indonesia for attracting the "wrong type" of doctor to rural areas and for reducing the return on investment placed in specialised training.
Recruiting foreign doctors and limiting them to rural practice will address staff shortages in these areas.	Recruitment of foreign doctors is widely practiced. It does offer a short‐term solution to importing countries, but may worsen global health professional distribution patterns. Long‐term retention of foreign doctors in underserved areas remains poor. However, there is a need to respect the right of healthcare professionals to enter the global labour market like any other professional; failure to do so may have a negative impact on those considering health care as a profession.
Professional and personal support strategies
Providing sufficient professional support encourages rural practice.	There is some evidence from questionnaire‐based surveys supporting this hypothesis. Healthcare professionals need continuous professional stimulation. Attendance at conferences and supporting local academic activities may assist them to keep up to date with new medical developments.
Providing sufficient personal support encourages rural practice.	There is some evidence from questionnaire‐based surveys which supports this intervention. It is self evident that providing personal support such as accommodation, meeting the educational needs of children etc. is important.

Educational strategies

Targeted admission policies for medical schools

Based on evidence from retrospective studies, it is believed that male students with a rural background who intend to practice rural medicine upon entry to medical school are more likely to be recruited to and retained in a rural practice (de Vries 2003; Rabinowitz 1999; Rabinowitz 2000; Rabinowitz 2001; Western 2000; Woloschuk 2002; Woloschuk 2004). The 2010 WHO technical report found evidence of moderate certainty that medical students from rural backgrounds were more likely to work in rural areas upon graduation (WHO 2010). This finding was based on the findings of a review (Laven 2003a; Laven 2003b), and several other studies (de Vries 2003; Rabinowitz 2005; Woloschuk 2004). However, the issues of bias and confounding, common to all observational studies, may limit the validity of these findings.

Furthermore, special primary care and rural health programmes have been implemented at a number of medical schools. These programmes involve special admission for students from rural backgrounds who express an interest in practicing family medicine in a rural area, as well as special curricular components and financial aid. A number of observational studies have shown an association between these programmes and an increase in the number of physicians entering rural practice (Brooks 2002; Moores 1998; Rabinowitz 1999).

Location of medical schools, medical school campuses and family medicine residency programmes

A number of countries have established medical schools, specific medical school campuses family medicine programmes, or a combination of these in rural or underserved areas in an attempt to entice graduates of these programmes to work in these areas upon graduation (Curran 2004). The Australian Rural Clinical School initiative established in 2000/2001 aims to increase recruitment and retention of healthcare professionals in rural areas through rurally based undergraduate clinical training experience (Eley 2012; Wilkinson 2004). One time series analysis evaluated the impact of clinical rotations through rural clinical schools during the third and fourth years of medical training at the University of Queensland. Following the establishment of the rural clinical schools, the number of medical students selecting rural regional hospitals as their preferred site for internship increased; whereas the number of students requesting rural intern employment at institutions without a rural clinical school remained the same during this time period (Wilkinson 2004). It is not known to what extent choice of a rural internship predicts future practice in a rural location.

One study conducted in the Democratic Republic of the Congo found that 81% of the graduates from the rural medical school were employed in rural areas in the province where they trained, with only 26% of the graduates from the urban medical school employed in rural areas in the province in which they were trained (Longombe 2009).

Undergraduate and postgraduate training exposure and location

Studies investigating the effect of rural clerkships/rotations and preceptorships in underserved areas often have "change in student attitudes towards rural practice" or "intention to practice rural" as their outcome measures. While these outcomes may be seen as proxies of recruitment to rural practice, such outcomes do not reflect actual practice location upon graduation. Such outcomes may also be subject to information bias in that the participants may tell the interviewer what the participant thinks the interviewer wants to hear.

A further proxy for practice in an underserved area used in some studies is choice of a primary care or family practice speciality. The impact of a third year family practice clerkship on medical students' choice of family practice as a speciality was investigated in a controlled, longitudinal study (Campos‐Outcalt 1999). Graduates' family practice speciality choice was noted for three years before and after the introduction of the clerkship. Family practice speciality choice of graduates from schools that did not have a third year family practice clerkship was used as the control data. There was a significant increase in the number of graduates who chose family practice as a speciality in the three years following the introduction of the clerkship (Campos‐Outcalt 1999). In four quasi‐randomised trials, medical students who were exposed to an undergraduate primary care clerkship or curriculum track were more likely to choose family practice residencies (Erney 1991; Harris 1982; Herold 1993; Rosenblatt 1979). These studies had the limitation that choice of a primary care or family practice speciality may not equate with practice in rural or underserved areas in all settings.

Between 1979 and 1993 the University of New Mexico School of Medicine offered two different curricular programmes ‐ a conventional programme and a primary care curriculum (a community‐oriented, problem‐based programme) ‐ during the first two years of medical school. A survey investigating the effect of the primary care curriculum on graduates practice patterns reported that primary care curriculum graduates were significantly more likely to work in medically underserved areas than the conventional program graduates (Mennin 1996). As medical students in this study self selected their programmes it is not possible to disentangle the effect of the curricular exposure from that of students' prior characteristics.

Evidence from retrospective cohort studies suggests that faculty role models influenced physicians' choice to specialise in family medicine (Campos‐Outcalt 1995). However, two prospective controlled observational studies did not show an effect of instruction by family physician faculty on the students' decision to follow a family practice residency (Allen 1987; Beasley 1993).

Financial incentives

Government bodies attempt to encourage healthcare professionals to practice in underserved areas through various financial support‐for‐service programmes (Pathman 1994; Pathman 2004). Evidence regarding the impact of these programmes on the recruitment and retention of health‐care professionals in underserved areas is based on descriptive, cross‐sectional surveys (Jackson 2003; Pathman 2004) and retrospective cohort studies (Pathman 1992). Bärnighausen 2009 systematically reviewed financial incentives for return of service in underserved areas. The systematic review presented findings from 43 studies (34 from the USA and the remaining studies from Canada, Japan, New Zealand and South Africa). All of the studies were observational studies and considered various types of loans and scholarship schemes. The authors reported promising results with regards to retention of healthcare professionals in underserved areas following the initial obligation period. An earlier systematic review of 10 studies investigating the effectiveness of financial incentives in exchange for rural and underserved area return‐of‐service commitments reported that financial return of service programmes appear to be effective in the short term; their long‐term effectiveness was unknown (Sempowski 2004). One prospective cohort study included in this review found that physicians who voluntarily chose to practice in rural areas were more likely to stay in rural areas compared to physicians working in rural areas due to return‐of‐service commitments (Sempowski 2004).

Some governments have introduced additional rural allowances in an effort to attract and retain healthcare professionals in these underserved areas. The effect of rural allowances on the distribution of healthcare professionals is being monitored and evaluated by questionnaire‐based surveys (Reid 2004).

Regulatory strategies

Coercive strategies can address shortages in the short term, but forced redeployment of poorly prepared or motivated staff may defeat the purpose. A number of coercive strategies, such as compulsory community service and restricting recruited foreign doctors to rural practice, have been adopted by governments in an attempt to address the shortage of healthcare professionals in medically underserved areas (Cavender 1998; Reid 2001). Frehywot 2010 reviewed the various compulsory service programmes instituted in countries around the globe. The paper describes the various programmes and attempts to provide some insight into the effectiveness of the programmes. Compulsory service programmes have been adopted by more than 70 countries including Russia, Mexico, Norway, Ecuador, Bolivia, Cuba, Dominican Republic, South Africa, Nigeria and Ghana (Frehywot 2010). While most of the countries have not measured the effectiveness of the compulsory service programmes, those that have found mixed results with respect to retention of healthcare professionals in underserved areas (Frehywot 2010) and there are concerns that a negative experience may adversely affect future career decisions to work in an underserved area (Marais 2007). Although widely practiced, the effect of recruiting foreign doctors to work in underserved areas on the distribution of healthcare professionals, and consequent provision of health services, has not been thoroughly assessed. Coercive strategies may provide short‐term solutions; however, there is concern that these strategies will alienate people from the profession and underserved areas in particular and thus prove counterproductive.

Professional and personal support

Questionnaire‐based surveys suggest that professional and personal support may also influence healthcare professionals' choice to work in underserved areas. Professional development, ongoing training and style of health service management were important factors influencing retention of healthcare professionals in underserved areas (Kotzee 2006). Poor living conditions and inadequate health care and education facilities were unanimously declared to be barriers to working in rural and remote areas (Lehmann 2008). The WHO technical report found very little information on the effectiveness of interventions aimed at professional development of healthcare professionals working in rural and underserved areas (WHO 2010). Continued education and training and career development opportunities were reported to be important motivational issues with respect to retaining healthcare professionals in rural and remote areas (Willis‐Shattuck 2008).

Why it is important to do this review

Evidence from cross‐country ecological analyses show an increased probability of survival and better health outcomes with an increase in the number of healthcare workers per population (WHO 2010). There are also studies showing that lower nurse‐to‐patient ratios lead to more complications and poorer patient outcomes, at hospital level (Aiken 1996; Aiken 2001). It is important to note that staff shortages, in general, impact negatively on the motivation of the existing staff ‐ as they have to deal with increased workload ‐ and may contribute to staff attrition. Given the wide array of strategies that have been adopted to influence locality of practice, educationalists and policy makers need to know which of these strategies are supported by scientific evidence of effectiveness and impact.

Objectives

To assess the effectiveness of interventions aimed at increasing the proportion of health professionals working in rural and other underserved areas.

Methods

Criteria for considering studies for this review

Types of studies

Randomised controlled trials (RCTs), non‐randomised trials, controlled before‐and‐after (CBA) studies and interrupted time series (ITS) studies that have evaluated the effects of various interventions on at least one of the outcomes listed below. We did not consider studies with historical controls and studies that did not include a control or comparison group.

We used the Cochrane Effective Practice and Organisation of Care (EPOC) Group's definition of study designs to consider studies for inclusion (EPOC 2013a). We only considered cluster randomised trials, non‐randomised cluster trials, and CBA studies with at least two intervention sites and two control sites and ITS studies with a clearly defined point in time when the intervention occurred and at least three data points before and three after the intervention.

Types of participants

All qualified healthcare professionals, for example, doctors (general practitioners and specialists), nurses, occupational therapists, physiotherapists, speech and hearing therapists, pharmacists, dieticians, clinical psychologists and dentists. We excluded studies investigating non‐professional healthcare workers such as lay health workers.

Types of interventions

• Educational interventions (e.g. student selection criteria, undergraduate and postgraduate teaching curricula, exposure to rural and urban underserved areas).

• Financial interventions (e.g. undergraduate and postgraduate bursaries/scholarships linked to future practice location, rural allowances, increased public sector salaries).

• Regulatory strategies (e.g. compulsory community service, relaxing work regulations imposed on foreign medical graduates who are willing to work in rural or urban underserved areas).

• Personal and professional support strategies (e.g. providing adequate professional support and attending to the needs of the practitioners family).

We planned to include studies that compared one of the above strategies with either no intervention or an alternative strategy as a control.

Types of outcome measures

Primary outcomes

• The proportion of healthcare professionals who initially choose to work in rural or urban underserved areas as a consequence of being exposed to the intervention (recruitment).

• The proportion of healthcare professionals who continue to work in rural or urban underserved areas as a consequence of the intervention (retention).

There are no internationally agreed definitions for what constitutes "rural underserved" and "urban underserved" areas. These terms tend to be relative and their meaning will vary from country to country. Thus, for each study, we accepted the definitions as provided by the authors, and record their particular use in the Characteristics of included studies table.

Secondary outcomes

• Patient satisfaction with care provided.

• Impact on health status of participants.

Search methods for identification of studies

Electronic searches

We developed search strategies for electronic databases using the methodological component of the EPOC search strategy combined with selected MeSH terms and free‐text terms. We revised and updated the search strategy in this version of the review. The detailed search strategies used for each of the databases are presented in Appendix 1.

We searched the following databases:

• Cochrane Central Register of Controlled Trials (CENTRAL), 2014, Issue 3, part of The Cochrane Library. www.thecochranelibrary.com (including the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register) (searched 10/04/2014)

• MEDLINE In‐Process & Other Non‐Indexed Citations, MEDLINE Daily, MEDLINE and OLDMEDLINE 1946 to Present, Ovid (searched 11/04/2014)

• Embase 1980 to 2014 Week 14, Ovid (searched 11/04/2014)

• CINAHL 1980 to present, EbscoHost (searched 11/04/2014)

• LILACS (VHL) http://pesquisa.bvsalud.org/portal/advanced/?lang=en (searched 10/02/2014)

• Science Citation Index and Social Sciences Citation Index (Web of Knowledge) (searched 22/06/2015)

• Global Health (CAB Direct) (searched 14/04/2014)

Searching other resources

Trial registries:

• International Clinical Trials Registry Platform (ICTRP), Word Health Organization (WHO) http://www.who.int/ictrp/en/

We also:

• Searched reference lists of all relevant publications and reviews that were identified

• Contactedauthors of relevant papers regarding any further published or unpublished work.

• Conducted cited reference searches for all included studies in ISI Web of Knowledge

Data collection and analysis

Selection of studies

Two review authors (LG and SM) screened the titles and abstracts of all articles obtained from the search. They independently applied the selection criteria to determine which studies should be included or excluded. We retrieved full copies of all reports deemed eligible by either of the review authors for closer inspection. We listed studies that appeared to meet the inclusion criteria on the first screening but were later deemed unsuitable for inclusion are listed in the Characteristics of excluded studies table, together with the reasons for their exclusion. One study met the inclusion criteria of the review (see Characteristics of included studies table).

Data extraction and management

One review author (LG) extracted the data from the eligible study and the three remaining review authors (JV, BM and SM) independently cross‐checked and confirmed the process. We used a data extraction form based on those used by the EPOC review group, but modified for this review. We extracted data regarding study design; description of the intervention (including process); details about participants (including number in each group), length of intervention and follow‐up; and the proportion of healthcare professionals who chose to work in rural or urban under‐served areas.

Assessment of risk of bias in included studies

We used the EPOC group's risk of bias checklists for RCTs, non‐randomised trials, ITS studies and CBA studies to assess all eligible studies. These checklists account for study design, method of randomisation, characteristics of control groups, method of data collection, confounding factors, appropriate statistical methods, selection of outcome variables and risk of bias. LG assessed the risk of bias of the eligible ITS study using the seven standard criteria set out by the Cochrane EPOC group for assessing the risk of bias of ITS study (EPOC 2015). For details of the risk of bias criteria for the ITS study see Table 3. The review authors (JV, BM and SM) checked the risk of bias assessment.

2. Risk of bias criteria for interrupted time series studies.

Item	High risk of bias	Low risk of bias	Unclear risk of bias
Was the intervention independent of other changes?	The intervention was not independent of other changes in time.	If there are compelling arguments that the intervention occurred independently of other changes over time and the outcome was not influenced by other confounding variables/historic events during the study period. If events/variables identified, note what they are.	Insufficient information to permit judgement of low or high risk.
Was the shape of the intervention effect pre‐specified?	It is clear that the shape of the intervention was not pre‐specified.	The point of analysis is the point of intervention OR a rational explanation for the shape of intervention effect was given by the author(s). Where appropriate, this should include an explanation if the point of analysis is NOT the point of intervention.	Insufficient information to permit judgement of low or high risk.
Was the intervention unlikely to affect data collection?	The intervention itself was likely to affect data collection (e.g. any change in source or method of data collection reported).	The intervention itself was unlikely to affect data collection (e.g. sources and methods of data collection were the same before and after the intervention).	Insufficient information to permit judgement of low or high risk.
Was knowledge of the allocated interventions adequately prevented during the study?	The outcomes were not assessed blindly.	The authors stated explicitly that the primary outcome variables were assessed blindly, or the outcomes are objective, e.g. length of hospital stay. Primary outcomes were those variables that corresponded to the primary hypothesis or question as defined by the authors.	Insufficient information to permit judgement of low or high risk.
Were incomplete outcome data adequately addressed?	Missing outcome data were likely to bias the results.	Missing outcome measures were unlikely to bias the results (e.g. the proportion of missing data was similar in the pre‐ and post‐intervention periods or the proportion of missing data was less than the effect size, i.e. unlikely to overturn the study result).	Insufficient information to permit judgement of low or high risk. If not specified in the paper (do not assume 100% follow up unless stated explicitly).
Was the study free from selective outcome reporting?	Some important outcomes were subsequently omitted from the results.	There was no evidence that outcomes were selectively reported (e.g. all relevant outcomes in the methods section are reported in the results section).	Insufficient information to permit judgement of low or high risk
Was the study free from other risks of bias?	There was evidence that other risks of bias exist, such as seasonality.	There was no evidence of other risk of biases (e.g. should consider if seasonality was an issue (i.e. if January to June comprises the pre‐intervention period and July to December the post, could the 'seasons' have caused a spurious effect).	Insufficient information to permit judgement of low or high risk

Measures of treatment effect

We conducted data analysis using Review Manager 5 (RevMan 2014). For RCT data, we planned to calculate outcome measures for dichotomous data (e.g. the proportion of decreasing consumption) as risk ratios (RR) with 95% confidence intervals (CIs). For continuous data (e.g. mean age of initiation) we calculated the mean differences (MDs) and standard deviations (SDs) where means were reported. For cohort and other study design data, if the data had been analysed correctly, we preferentially reported on the adjusted analysis using the estimate of effect reported in the study. If the incorrect analyses had been used, we planned to re‐analyse the data appropriately using the crude data. Where only crude data were presented, where appropriate, we calculated the crude RRs and 95% CIs for dichotomous data and MDs and SDs for continuous data where means were reported. If data were skewed, we planned to report medians.

We reported the findings of Yang 2013 in narrative form as reported by the authors. However, Yang 2013 analysed their data using spline regression, which is correct when there are independent outcomes. However, in this study one cannot assume that the observations are independent. The Gini coefficient in any one year will be dependent on the Gini coefficient from the previous year as neither the population nor the number of doctors fluctuates randomly. By using spline regression the authors do not take the auto‐correlated structure of the data into account. In order to re‐analyse the data provided in Yang 2013, we followed the instructions set out in the EPOC manual (EPOC 2013b). We extracted the crude data for each of the relevant time points presented in Figure 1 in the paper using PlotDigitizer (PlotDigitizer). We arranged the crude data appropriately and then imported them into SPSS statistical software (IBM SPSS Statistics). The statistical analyses were carried out as per instructions in the EPOC manual (EPOC 2013b).

Data synthesis

We analysed the studies separately depending on whether the evaluated intervention was aimed at the initial recruitment or subsequent retention of healthcare professionals.

Randomised controlled trials, controlled clinical trials and controlled before‐and‐after studies

For these specific study designs, we planned to report the pre‐intervention and post‐intervention proportions for both study and control groups. We planned to calculate the absolute change from baseline for each group. We planned to analyse data using Review Manager 5 (RevMan 2014). We planned to pool estimates of effect using RRs for binary data and MDs for continuous data. For continuous outcome data that is expressed in different units, we planned to calculate standardised mean differences (SMDs).

Interrupted time series

We planned to report on the adjusted analysis using the estimate of effect reported in the study. Where the adjusted estimate of effect was reported with 95% CIs, we planned to calculate the standard error (SE) in order to enter the data into Review Manager 5 (RevMan 2014), using the following formulae for ratio measures:

• lower limit = ln(lower confidence limit given for hazard ratio (HR));

• upper limit = ln(upper confidence limit given for HR);

• intervention effect estimate = lnHR;

• SE = (upper limit ‐ lower limit)/3.92.

We planned to calculate the relative and absolute MD in before and after values. Whenever possible, we used time series regression analyses to calculate mean change in level and mean change in slope.

Meta‐analysis

We planned to conduct a meta‐analysis on the combined results of similar studies (with similar comparisons and outcome measures) using a random‐effects model. However, only one study was eligible for this review.

Subgroup analysis and investigation of heterogeneity

We anticipated significant heterogeneity in study findings related to differences in the intervention type, type of health professional, study setting (rural versus urban underserved), study design and methodological quality.

We planned to conduct subgroup analyses based upon:

• type of intervention (educational, financial, regulatory);

• type of health professional (doctor, dentist, nurse, allied health professional);

• country income level (high, middle, low).

However, only one study met the inclusion criteria of the review.

Sensitivity analysis

To determine how robust and consistent the results are, we planned to conduct sensitivity analyses based upon study design (RCT versus other) or risk of bias in study (high, medium, low ‐ according to the EPOC group's 'Risk of bias' checklists). However, since only one study met the inclusion criteria of the review, we were unable to conduct these analyses.

Results

Description of studies

Results of the search

We screened 8945 records for eligibility (see Figure 1 for details of the search results). We retrieved and assessed the full text of 125 studies for eligibility. Only one study met all of the inclusion criteria of the review (Yang 2013).

1.

Study flow diagram.

Included studies

Yang 2013 used interrupted trend analysis with time series observations to assess the impact of the implementation of National Health Insurance (NHI) in Taiwan on the geographic distribution of three different health providers, Western medicine physicians, Chinese medicine physicians and dentists over 32 years (1971 to 2002). The equality of the geographic distribution of the health providers was measured using Gini coefficients (Gini coefficient of 0 = no inequality; Gini coefficient of 1 = complete inequality) for each of the 21 prefectures/cities in Taiwan. The investigators examined the distribution of health providers 24 years prior to NHI and eight years after the implementation of NHI (in 1995; Yang 2013). Taiwan's NHI is administered by a government‐run insurer. Enrolment in the NHI is mandatory for all citizens. It entails a single‐payer system and provides comprehensive benefits including dental care and Chinese medicine. The authors reported that by 1995 the NHI had enrolled 96% of the Taiwanese population.

Excluded studies

We excluded almost all the studies investigating recruitment and retention of healthcare professionals because their study design did not meet the inclusion criteria of the review. We excluded other studies because the outcomes measured were surrogate markers of recruitment and retention, for example, medical school student's intention to practice in a rural setting. Four quasi‐randomised trials, one controlled trial and one ITS study were brought to our attention by experts in the field. The reasons for excluding these studies from the review are stipulated in the Characteristics of excluded studies table. Details of other potentially eligible studies that were later deemed unsuitable for inclusion are also provided in the table.

Risk of bias in included studies

We assessed the risk of bias of the eligible study using the EPOC suggested risk of bias criteria for ITS studies (Table 3). We provide a full description of the risk of bias for the included study in the text below. The risk of bias of the study is summarised in Figure 2 and Figure 3.

2.

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.

3.

Risk of bias summary: review authors' judgements about each risk of bias item for each included study.

Was the intervention independent of other changes?

High risk of bias: we cannot be sure that the intervention was independent of other changes. However, in an attempt to account for other variables (socioeconomic, political) that may have influenced the geographic distribution of health providers, the study authors included a year‐specific time trend variable (T_t) in their analysis model. The authors limited the post NHI observations to 2002 in order to exclude the possible effect of other health insurance measures adopted by the Taiwanese Government after the introduction of the NHI (e.g. pay‐for performance in 2001, global budgeting in 2002 and regressive payment to service volume for assuring quality of care in 2004) on health provider distribution.

Was the shape of the intervention effect pre‐specified?

Unclear risk of bias: the authors did not specifically state how they felt the implementation of the NHI would affect the outcome of interest. However, the authors did state that any changes occurring as a result of the intervention would be gradual and this was accounted for in the analyses and the variables included in the analyses.

Was the intervention unlikely to affect data collection?

Low risk of bias: sources and methods of data collection were similar before and after the intervention.

Was knowledge of the allocated interventions adequately prevented during the study?

Low risk of bias: even though the health providers would have been aware of the implementation of the NHI, the outcome was objective and was obtained at a national level from published government records.

Were incomplete outcome data adequately addressed?

Low risk of bias: although the authors did not report the extent of missing data, the outcome was obtained from routine government records for the entire period of analysis, without any change in practice. Therefore, any potential flaws inherent to the data are likely to have been present throughout the time period

Was the study free from selective outcome reporting?

Low risk of bias: while we do not have access to the study protocol, all relevant outcomes in the methods section are reported in the results section.

Was the study free from other risks of bias?

Unclear risk of bias: while the authors declared that they had no competing interests, the funding source of the study was not reported. In addition, the historical nature of the data reduces the risk of political expediency.

Effects of interventions

See: Table 1

Yang 2013 assessed the effect of NHI on the distribution of Western medicine physicians, Chinese medicine physicians and dentists over 32 years (1971 to 2002) in Taiwan. The authors presented data on the equality of the distribution of the health providers for 24 years prior to NHI and eight years after the implementation of NHI in 1995 (Yang 2013). The equality of the geographic distribution of the health providers was measured by Gini coefficients. A Gini coefficient of 0 represented 'no inequality' and a Gini coefficient of 1 represented 'complete inequality'.

At one year post implementation of the NHI scheme, inequality of geographic distribution of Western medicine physicians (estimate, SE: ‐0.008, 0.004, P value = 0.071), Chinese medicine physicians (estimate, SE: ‐0.007, 0.013, P value = 0.589) and dentists (estimate, SE: ‐0.013, 0.017, P value = 0.444) improved when compared to the expected level based on the pre‐intervention trend for each discipline.

At four years after the implementation of NHI scheme, the geographic distribution of Western medicine physicians (estimate, SE: ‐0.018, 0.003, P value = 0.00), Chinese medicine physicians (estimate, SE: ‐0.016, 0.011, P value = 0.172) and dentists (estimate, SE: ‐0.039, 0.03, P value = 0.196) was more equitable compared to the expected level based on the pre‐intervention trend.

At eight years post implementation of NHI scheme, there was a similar trend. The inequality of geographic distribution of Western medicine physicians (estimate, SE: ‐0.032, 0.005, P value = 0.00), Chinese medicine physicians (estimate, SE: ‐0.025, 0.016, P value = 0.143) and dentists (estimate, SE: ‐0.073, 0.056, P value = 0.201) improved compared to the expected level based on the pre‐intervention trend.

Prior to the implementation of the NHI, Western medicine physicians were more equally distributed across all Taiwanese prefectures compared to Chinese medicine physicians and dentists (Yang 2013). This improved further after implementation of the NHI scheme compared to the expected level based on the pre‐intervention trend. Although these results suggest a reduction in the inequitable distribution of these health professionals, the changes were small.

Discussion

Summary of main results

The main finding of this review is that there is a dearth of well‐designed studies in which bias and confounding have been minimised to guide the implementation of interventions to address the shortage of healthcare professionals practicing in underserved areas. Only one ITS study conducted in Taiwan met the stipulated inclusion criteria of the review. The study found that the implementation of a NHI scheme reduced inequalities in the distribution of Western medicine physicians, Chinese medicine physicians and dentists in Taiwan (Yang 2013). Prior to the implementation of the NHI, people living in urban areas were more likely to be able to afford higher medical costs. Therefore, it was more financially rewarding for health providers to work in urban areas compared to the poorer rural areas. The study authors reasoned that universal NHI coverage made medical care more affordable for all Taiwanese citizens, irrespective of their financial means or geographic location. Health providers practicing in both urban and rural areas received the same payment for the provision of medical care, reducing previous financial disincentives to work in underserved areas where people were less able to afford medical care.

Certainty of the evidence

The main reasons for downgrading the certainty of the evidence was serious concerns regarding risk of bias and serious concerns regarding the indirectness of the data from one study conducted in one country. The study results may not be generalisable or transferable to other settings.

Agreements and disagreements with other studies or reviews

A number of reviews have investigated the effectiveness of interventions to improve recruitment and retention of healthcare professionals in rural and underserved areas. The main difference between these reviews and our review is that the other reviews have included research findings from studies with all types of study designs, including uncontrolled observational study designs. Dolea 2010 and Wilson 2009 provided comprehensive overviews of the available evidence, but emphasised the urgent need for more rigorous assessment of study interventions. While the findings of these reviews are useful, one must bear in mind the certainty of the evidence upon which the conclusions of the reviews were based. All of the review authors agree that more robust, good‐quality research needs to be conducted. Viscomi 2013 reviewed the findings of 86 studies investigating interventions to recruit and retain general practitioners in rural Canada and Australia (Viscomi 2013). Factors such as rural background (of medical student or partner, or both), male gender, interest in living in a rural area and meaningful rural elective exposure during medical training were some of the important indicators related to rural practice (Viscomi 2013).

The latest technical report prepared by the WHO found evidence of moderate certainty that medical students from rural backgrounds were more likely to work in rural areas upon graduation (WHO 2010). The report found very little information on the effectiveness of interventions aimed at professional development of healthcare professionals working in rural and underserved areas (WHO 2010).

Another relatively recent review focusing on effective retention incentives found that financial incentives and healthcare worker obligation were the most commonly utilised retention strategies (Buykx 2010). Both of these strategies appeared to be effective in the short‐term but their effectiveness in retaining healthcare professionals in rural areas was not clear (Buykx 2010). Additional high‐quality research investigating personal and work‐related factors is needed (Buykx 2010).

Wilson 2009 conducted a critical review of interventions to redress the inequitable distribution of healthcare professionals in rural and remote areas, irrespective of meeting pre‐defined study criteria. This comprehensive overview utilised a simple grading system to rate the certainty of the evidence for specific interventions (selection criteria; pre‐ and post‐vocational training; medical school location; coercive, incentives and support strategies). Strong evidence supported selection criteria such as rural background, intention to practice in a rural setting and male gender, as well as post‐vocational training in generalist specialities such as rural health and family medicine (with major bias from self selection; Wilson 2009).

Barnighausen 2009 systematically reviewed financial incentives for return of service in underserved areas. The systematic review presented findings from 43 studies (34 from the USA and the remaining studies from Canada, Japan, New Zealand and South Africa). All studies included in the review were observational studies and considered various types of loans and scholarship schemes. The authors reported promising results with regards to retention of healthcare professionals in underserved areas following the initial obligation period (Barnighausen 2009).

One review looking at incentives, motivation and subsequent retention of health workers in developing countries found that the importance of the various motivational factors varied between countries. However, financial incentives, career development opportunities and sound management were similarly important across the developing countries investigated in the reviewed studies (Willis‐Shattuck 2008). Financial incentives alone were not enough to motivate and retain health workers. Health worker morale was affected by recognition, adequate medical resources and equipment, and appropriate infrastructure (Willis‐Shattuck 2008).

Lehmann 2008 provided a conceptual framework of the international, national, work, local (home and social) and individual environmental factors that impact on the attraction and retention of health workers in LMICs. The review authors recommended that the attraction and retention factors specific to country, area and healthcare facility be assessed first and then combinations of various strategies be implemented to address these factors (Lehmann 2008).

Authors' conclusions

Implications for practice.

There is currently limited reliable evidence regarding the effects of interventions aimed at addressing the inequitable distribution of health professionals. Evidence from one study suggested that providing universal coverage through a National Health Insurance policy and standardising the cost of health care on a national level is associated with a more equal distribution of health providers. Although these results suggest a reduction in the inequitable distribution of these health professionals, these changes were small. We judged the certainty of the evidence provided by this one study and the generalisability of the findings to be very low. Despite the dearth of reliable evidence, governments and medical schools have implemented numerous strategies aimed at addressing the shortage of healthcare professionals practicing in underserved areas. Rigorous evaluation of the benefits and risks of various strategies is required to determine the true impact of these interventions and to inform future policy better. The inequitable distribution of healthcare professionals in underserved areas reflects multiple factors with complex interactions that may vary across different settings. More rigorous evidence is required to guide tailored interventions and identify generalisable principles.

Implications for research.

Observational studies suggest that a number of factors related to student selection criteria, undergraduate and postgraduate training, financial incentives, regulatory measures, and personal and professional support may influence healthcare professionals' choice to practice in underserved areas. Strategies that have shown promise include selection of students with a rural background, the establishment of university departments or teaching clinics (or both) in rural areas, rural and scarce skills allowances, and enhanced professional and personal support. However, these findings await confirmation or refutation by means of well‐designed studies in which bias and confounding are minimised. Future studies should also take into account the different contexts and settings in which the inequitable distribution of healthcare professionals occur, for example, the type of healthcare professional, the presiding cultural expectations and social pressures, the type of intervention, the location of the underserved community (i.e. rural versus urban) and the income or development status of the country.

What's new

Date	Event	Description
14 October 2019	Amended	A link to a summary for policy‐makers was added to the plain language summary

History

Protocol first published: Issue 2, 2005
 Review first published: Issue 1, 2009

Date	Event	Description
25 May 2015	Amended	Jimmy Volmink asked to be removed as author of updated version of the review. He does not have the necessary time at this stage to commit to the process.
25 May 2015	Feedback has been incorporated	plain language summary updated and added.
26 March 2015	New citation required and conclusions have changed	The review is no longer an empty review. The findings of one eligible study were included in this updated version of the review. Although the overarching conclusions of the review remain the same, the review conclusions have changed somewhat following the incorporation of the findings of the eligible study.
25 February 2015	New search has been performed	This is the first update of the original review. One eligible study was added to the review.
17 February 2009	Amended	Plain language summary modified.
12 November 2008	Amended	RevMan5 conversion

Appendices

Appendix 1. Search strategies

CENTRAL, Cochrane Library

#1	MeSH descriptor: [Rural Health Services] this term only and with qualifier(s): [Manpower ‐ MA]	6
#2	MeSH descriptor: [Rural Health] this term only and with qualifier(s): [Manpower ‐ MA]	0
#3	MeSH descriptor: [Suburban Health] this term only and with qualifier(s): [Manpower ‐ MA]	0
#4	MeSH descriptor: [Medically Underserved Area] this term only	86
#5	MeSH descriptor: [Resource Allocation] this term only and with qualifier(s): [Manpower ‐ MA]	1
#6	MeSH descriptor: [Hospitals, Rural] this term only and with qualifier(s): [Manpower ‐ MA]	1
#7	MeSH descriptor: [Rural Nursing] this term only and with qualifier(s): [Manpower ‐ MA]	0
#8	"human resources for health"	12
#9	(#1 or #2 or #3 or #4 or #5 or #6 or #7 or #8)	106
#10	MeSH descriptor: [Health Personnel] explode all trees	5654
#11	MeSH descriptor: [Health Manpower] this term only	12
#12	#10 or #11	5663
#13	MeSH descriptor: [Personnel Management] explode all trees	1651
#14	MeSH descriptor: [Resource Allocation] this term only	57
#15	#13 or #14	1704
#16	#12 and #15	270
#17	(recruit* or retain* or retention or scale next up or scaling next up) near/6 (nurse or nurses or midwife or midwives or physician* or clinician* or doctor* or practitioner* or dentist* or dental next staff or pharmacist* or dietician* or nutritionist* or psychologist* or occupational next therapist* or physiotherapist* or physical next therapist* or speech next therapist* or language next therapist* or logopaedist* or logopedist* or speech next pathologist* or language next pathologist* or audiologist* or internist* or paediatrician* or pediatrician* or ophthalmologist* or surgeon* or radiographer* or radiologist* or optometrist* or personnel or worker* or staff or professional* or provider* or manpower or man next power or human next resources or workforce*)	1557
#18	(density or turnover or turn next over or brain drain* or shortage or understaffed or under next staffed or inequit* near/2 distribut* or inequal* near/2 distribut* or maldistribut* or mal next distribut* or management or allocation) near/2 (nurse or nurses or midwife or midwives or physician* or clinician* or doctor* or practitioner* or dentist* or dental next staff or pharmacist* or dietician* or nutritionist* or psychologist* or occupational next therapist* or physiotherapist* or physical next therapist* or speech next therapist* or language next therapist* or logopaedist* or logopedist* or speech next pathologist* or language next pathologist* or audiologist* or internist* or paediatrician* or pediatrician* or ophthalmologist* or surgeon* or radiographer* or radiologist* or optometrist* or personnel or worker* or staff or professional* or provider* or manpower or man next power or human next resources or workforce*)	1356
#19	(increas* or expand* or enlarge* or extend or enhanc*) near/3 (proportion* or number* or amount*) near/3 (nurse or nurses or midwife or midwives or physician* or clinician* or doctor* or practitioner* or dentist* or dental next staff or pharmacist* or dietician* or nutritionist* or psychologist* or occupational next therapist* or physiotherapist* or physical next therapist* or speech next therapist* or language next therapist* or logopaedist* or logopedist* or speech next pathologist* or language next pathologist* or audiologist* or internist* or paediatrician* or pediatrician* or ophthalmologist* or surgeon* or radiographer* or radiologist* or optometrist* or personnel or worker* or staff or professional* or provider* or manpower or man next power or human next resources or workforce*)	76
#20	((client near/1 staff) or (patient near/1 staff) or (patient near/1 nurse)) next (ratio or ratios)	52
#21	#16 or #17 or #18 or #19 or #20	3072
#22	MeSH descriptor: [Rural Health Services] this term only	251
#23	MeSH descriptor: [Rural Health] this term only	437
#24	MeSH descriptor: [Suburban Health] this term only	11
#25	MeSH descriptor: [Medically Underserved Area] this term only	86
#26	MeSH descriptor: [Hospitals, Rural] this term only	38
#27	MeSH descriptor: [Rural Population] this term only	872
#28	MeSH descriptor: [Rural Nursing] this term only	0
#29	(rural next health* or rural next medical next care or rural next medical next service* or rural next medicine)	896
#30	(rural or remote or nonmetropolitan or non next metropolitan or suburb* or developing or less* next developed or under next developed or underdeveloped or middle next income or low* next income or underserved or under next served or deprived or poor) near/3 (communit* or area or areas or village* or region or regions or province* or setting*)	3262
#31	(rural or remote or nonmetropolitan or non next metropolitan or suburb* or middle next income or low* next income or underserved or under next served or deprived or poor or village*) near/3 population*	1734
#32	(rural or remote or nonmetropolitan or non next metropolitan or suburb* or village*) near/3 (clinic or clinics or hospital or hospitals or facility or facilities or health* next center* or health* next centre* or health next care next center* or health next care next centre* or medical next center* or medical next centre*)	464
#33	(shortage or understaffed or under next staffed) near/3 (area or areas)	5
#34	(Africa or Asia or Caribbean or "West Indies" or "South America" or "Latin America" or "Central America")	8042
#35	(Afghanistan or Albania or Algeria or Angola or Antigua or Barbuda or Argentina or Armenia or Armenian or Aruba or Azerbaijan or Bahrain or Bangladesh or Barbados or Benin or Byelarus or Byelorussian or Belarus or Belorussian or Belorussia or Belize or Bhutan or Bolivia or Bosnia or Herzegovina or Hercegovina or Botswana or Brazil or Brasil or Bulgaria or "Burkina Faso" or "Burkina Fasso" or "Upper Volta" or Burundi or Urundi or Cambodia or "Khmer Republic" or Kampuchea or Cameroon or Cameroons or Cameron or Camerons or "Cape Verde" or "Central African Republic" or Chad or Chile or China or Colombia or Comoros or "Comoro Islands" or Comores or Mayotte or Congo or Zaire or "Costa Rica" or "Cote d'Ivoire" or "Ivory Coast" or Croatia or Cuba or Cyprus or Czechoslovakia or "Czech Republic" or Slovakia or "Slovak Republic")	35790
#36	(Djibouti or "French Somaliland" or Dominica or "Dominican Republic" or "East Timor" or "East Timur" or "Timor Leste" or Ecuador or Egypt or "United Arab Republic" or "El Salvador" or Eritrea or Estonia or Ethiopia or Fiji or Gabon or "Gabonese Republic" or Gambia or Gaza or Georgia or Georgian or Ghana or "Gold Coast" or Greece or Grenada or Guatemala or Guinea or Guam or Guiana or Guyana or Haiti or Honduras or Hungary or India or Maldives or Indonesia or Iran or Iraq or "Isle of Man" or Jamaica or Jordan or Kazakhstan or Kazakh or Kenya or Kiribati or Korea or Kosovo or Kyrgyzstan or Kirghizia or "Kyrgyz Republic" or Kirghiz or Kirgizstan or "Lao PDR" or Laos or Latvia or Lebanon or Lesotho or Basutoland or Liberia or Libya or Lithuania)	33854
#37	(Macedonia or Madagascar or "Malagasy Republic" or Malaysia or Malaya or Malay or Sabah or Sarawak or Malawi or Nyasaland or Mali or Malta or "Marshall Islands" or Mauritania or Mauritius or "Agalega Islands" or Mexico or Micronesia or "Middle East" or Moldova or Moldovia or Moldovian or Mongolia or Montenegro or Morocco or Ifni or Mozambique or Myanmar or Myanma or Burma or Namibia or Nepal or "Netherlands Antilles" or "New Caledonia" or Nicaragua or Niger or Nigeria or "Northern Mariana Islands" or Oman or Muscat or Pakistan or Palau or Palestine or Panama or Paraguay or Peru or Philippines or Philipines or Phillipines or Phillippines or Poland or Portugal or "Puerto Rico")	15344
#38	(Romania or Rumania or Roumania or Russia or Russian or Rwanda or Ruanda or "Saint Kitts" or "St Kitts" or Nevis or "Saint Lucia" or "St Lucia" or "Saint Vincent" or "St Vincent" or Grenadines or Samoa or "Samoan Islands" or "Navigator Island" or "Navigator Islands" or "Sao Tome" or "Saudi Arabia" or Senegal or Serbia or Montenegro or Seychelles or "Sierra Leone" or Slovenia or "Sri Lanka" or Ceylon or "Solomon Islands" or Somalia or Sudan or Suriname or Surinam or Swaziland or Syria or Tajikistan or Tadzhikistan or Tadjikistan or Tadzhik or Tanzania or Thailand or Togo or "Togolese Republic" or Tonga or Trinidad or Tobago or Tunisia or Turkey or Turkmenistan or Turkmen or Uganda or Ukraine or Uruguay or USSR or "Soviet Union" or "Union of Soviet Socialist Republics" or Uzbekistan or Uzbek or Vanuatu or "New Hebrides" or Venezuela or Vietnam or "Viet Nam" or "West Bank" or Yemen or Yugoslavia or Zambia or Zimbabwe or Rhodesia)	20399
#39	(developing or less* next developed or "under developed" or underdeveloped or "middle income" or low* next income or underserved or "under served" or deprived or poor*) next (countr* or nation* or population* or world or area or areas)	5057
#40	(developing or less* next developed or "under developed" or underdeveloped or "middle income" or low* next income) next (economy or economies)	33
#41	low* next (gdp or gnp or "gross domestic" or "gross national")	33
#42	(low near/3 middle near/3 countr*)	578
#43	(lmic or lmics or "third world" or "lami country" or "lami countries")	142
#44	("transitional country" or "transitional countries")	11
#45	(#22 or #23 or #24 or #25 or #26 or #27 or #28 or #29 or #30 or #31 or #32 or #33 or #34 or #35 or #36 or #37 or #38 or #39 or #40 or #41 or #42 or #43 or #44)	105507
#46	#21 and #45	893
#47	#9 or #46 in Trials	294

MEDLINE, Ovid

#	Searches	Results
1	Rural Health Services/ma [Manpower]	1382
2	Rural Nursing/ma [Manpower]	0
3	Rural Health/ma [Manpower]	250
4	Medically Underserved Area/ma [Manpower]	5
5	Resource Allocation/ma [Manpower]	4
6	Hospitals, Rural/ma [Manpower]	284
7	human resources for health.ti,ab.	398
8	or/1‐7	2279
9	exp Health Personnel/	367692
10	Health Manpower/	11202
11	manpower.fs.	56093
12	or/9‐11	409921
13	exp Personnel Management/	126526
14	Resource Allocation/	7359
15	or/13‐14	133712
16	12 and 15	55813
17	((recruit* or retain* or retention or scale up or scaling up) adj6 (nurse or nurses or midwife or midwives or physician* or clinician? or doctor* or practitioner* or dentist* or dental staff or pharmacist* or dietician* or nutritionist* or psychologist* or occupational therapist* or physiotherapist* or physical therapist* or speech therapist* or language therapist* or logopaedist* or logopedist* or speech pathologist* or language pathologist* or audiologist* or internist* or paediatrician* or pediatrician* or ophthalmologist* or surgeon* or radiographer* or radiologist* or optometrist* or personnel or worker? or staff or professional? or provider? or manpower or man power or human resources or workforce?)).ti,ab.	11075
18	((density or turnover or turn over or brain drain* or shortage or understaffed or under staffed or (inequit* adj2 distribut*) or (inequal* adj2 distribut*) or maldistribut* or mal distribut* or management or allocation) adj2 (nurse or nurses or midwife or midwives or physician* or clinician? or doctor* or practitioner* or dentist* or dental staff or pharmacist* or dietician* or nutritionist* or psychologist* or occupational therapist* or physiotherapist* or physical therapist* or speech therapist* or language therapist* or logopaedist* or logopedist* or speech pathologist* or language pathologist* or audiologist* or internist* or paediatrician* or pediatrician* or ophthalmologist* or surgeon* or radiographer* or radiologist* or optometrist* or personnel or worker? or staff or professional? or provider? or manpower or man power or human resources or workforce?)).ti,ab.	10943
19	((increas* or expand* or enlarge* or extend or enhanc*) adj3 (proportion* or number? or amount*) adj3 (nurse or nurses or midwife or midwives or physician* or clinician? or doctor* or practitioner* or dentist* or dental staff or pharmacist* or dietician* or nutritionist* or psychologist* or occupational therapist* or physiotherapist* or physical therapist* or speech therapist* or language therapist* or logopaedist* or logopedist* or speech pathologist* or language pathologist* or audiologist* or internist* or paediatrician* or pediatrician* or ophthalmologist* or surgeon* or radiographer* or radiologist* or optometrist* or personnel or worker? or staff or professional? or provider? or manpower or man power or human resources or workforce?)).ti,ab.	1766
20	(((client adj1 staff) or (patient adj1 staff) or (patient adj1 nurse)) adj ratio?).ti,ab.	233
21	or/17‐20	23270
22	16 or 21	74849
23	Rural Health Services/	8884
24	Rural Health/	21364
25	Rural Nursing/	10
26	Suburban Health/	388
27	Medically Underserved Area/	5656
28	Hospitals, Rural/	4014
29	Rural Population/	39243
30	(rural adj (health* or medical care or medical service?)).ti,ab.	4107
31	rural medicine.ti,ab.	250
32	((rural or remote or nonmetropolitan or non metropolitan or suburb* or developing or less* developed or under developed or underdeveloped or middle income or low* income or underserved or under served or deprived or poor) adj3 (communit* or area? or village* or region? or province? or setting*)).ti,ab.	64771
33	((rural or remote or nonmetropolitan or non metropolitan or suburb* or middle income or low* income or underserved or under served or deprived or poor or village*) adj3 population*).ti,ab.	14736
34	((rural or remote or nonmetropolitan or non metropolitan or suburb* or village*) adj3 (clinic? or hospital? or facility or facilities or health* center? or health* centre? or health care center? or health care centre? or medical center? or medical centre?)).ti,ab.	8482
35	((shortage or understaffed or under staffed) adj3 area?).ti,ab.	337
36	or/23‐35	124714
37	Developing Countries.sh,kf.	72050
38	(Africa or Asia or Caribbean or West Indies or South America or Latin America or Central America).hw,kf,ti,ab,cp.	193137
39	(Afghanistan or Albania or Algeria or Angola or Antigua or Barbuda or Argentina or Armenia or Armenian or Aruba or Azerbaijan or Bahrain or Bangladesh or Barbados or Benin or Byelarus or Byelorussian or Belarus or Belorussian or Belorussia or Belize or Bhutan or Bolivia or Bosnia or Herzegovina or Hercegovina or Botswana or Brazil or Brasil or Bulgaria or Burkina Faso or Burkina Fasso or Upper Volta or Burundi or Urundi or Cambodia or Khmer Republic or Kampuchea or Cameroon or Cameroons or Cameron or Camerons or Cape Verde or Central African Republic or Chad or Chile or China or Colombia or Comoros or Comoro Islands or Comores or Mayotte or Congo or Zaire or Costa Rica or Cote d'Ivoire or Ivory Coast or Croatia or Cuba or Cyprus or Czechoslovakia or Czech Republic or Slovakia or Slovak Republic or Djibouti or French Somaliland or Dominica or Dominican Republic or East Timor or East Timur or Timor Leste or Ecuador or Egypt or United Arab Republic or El Salvador or Eritrea or Estonia or Ethiopia or Fiji or Gabon or Gabonese Republic or Gambia or Gaza or Georgia Republic or Georgian Republic or Ghana or Gold Coast or Greece or Grenada or Guatemala or Guinea or Guam or Guiana or Guyana or Haiti or Honduras or Hungary or India or Maldives or Indonesia or Iran or Iraq or Isle of Man or Jamaica or Jordan or Kazakhstan or Kazakh or Kenya or Kiribati or Korea or Kosovo or Kyrgyzstan or Kirghizia or Kyrgyz Republic or Kirghiz or Kirgizstan or Lao PDR or Laos or Latvia or Lebanon or Lesotho or Basutoland or Liberia or Libya or Lithuania or Macedonia or Madagascar or Malagasy Republic or Malaysia or Malaya or Malay or Sabah or Sarawak or Malawi or Nyasaland or Mali or Malta or Marshall Islands or Mauritania or Mauritius or Agalega Islands or Mexico or Micronesia or Middle East or Moldova or Moldovia or Moldovian or Mongolia or Montenegro or Morocco or Ifni or Mozambique or Myanmar or Myanma or Burma or Namibia or Nepal or Netherlands Antilles or New Caledonia or Nicaragua or Niger or Nigeria or Northern Mariana Islands or Oman or Muscat or Pakistan or Palau or Palestine or Panama or Paraguay or Peru or Philippines or Philipines or Phillipines or Phillippines or Poland or Portugal or Puerto Rico or Romania or Rumania or Roumania or Russia or Russian or Rwanda or Ruanda or Saint Kitts or St Kitts or Nevis or Saint Lucia or St Lucia or Saint Vincent or St Vincent or Grenadines or Samoa or Samoan Islands or Navigator Island or Navigator Islands or Sao Tome or Saudi Arabia or Senegal or Serbia or Montenegro or Seychelles or Sierra Leone or Slovenia or Sri Lanka or Ceylon or Solomon Islands or Somalia or Sudan or Suriname or Surinam or Swaziland or Syria or Tajikistan or Tadzhikistan or Tadjikistan or Tadzhik or Tanzania or Thailand or Togo or Togolese Republic or Tonga or Trinidad or Tobago or Tunisia or Turkey or Turkmenistan or Turkmen or Uganda or Ukraine or Uruguay or USSR or Soviet Union or Union of Soviet Socialist Republics or Uzbekistan or Uzbek or Vanuatu or New Hebrides or Venezuela or Vietnam or Viet Nam or West Bank or Yemen or Yugoslavia or Zambia or Zimbabwe or Rhodesia).hw,kf,ti,ab,cp.	2813968
40	((developing or less* developed or under developed or underdeveloped or middle income or low* income or underserved or under served or deprived or poor*) adj (countr* or nation? or population? or world or area?)).ti,ab.	60338
41	((developing or less* developed or under developed or underdeveloped or middle income or low* income) adj (economy or economies)).ti,ab.	270
42	(low* adj (gdp or gnp or gross domestic or gross national)).ti,ab.	147
43	(low adj3 middle adj3 countr*).ti,ab.	3365
44	(lmic or lmics or third world or lami countr*).ti,ab.	3320
45	transitional countr*.ti,ab.	105
46	or/37‐45	2962745
47	36 or 46	3020234
48	randomized controlled trial.pt.	370644
49	controlled clinical trial.pt.	88147
50	multicenter study.pt.	169970
51	(randomis* or randomiz* or randomly allocat* or random allocat*).ti,ab.	390359
52	groups.ab.	1342101
53	(trial or multicenter or multi center or multicentre or multi centre).ti.	146695
54	(intervention* or controlled or control group or compare or compared or (before adj5 after) or (pre adj5 post) or pretest or pre test or posttest or post test or quasiexperiment* or quasi experiment* or evaluat* or effect or impact or time series or time point? or repeated measur*).ti,ab.	6634159
55	or/48‐54	7246452
56	exp Animals/	17281097
57	Humans/	13355173
58	56 not (56 and 57)	3925924
59	review.pt.	1862998
60	meta analysis.pt.	46977
61	news.pt.	160916
62	comment.pt.	578468
63	editorial.pt.	352003
64	cochrane database of systematic reviews.jn.	10582
65	comment on.cm.	578467
66	(systematic review or literature review).ti.	49049
67	or/58‐66	6599177
68	55 not 67	5126050
69	8 and 68	678
70	22 and 47 and 68	3297
71	69 or 70	3672

Embase, Ovid

#	Searches	Results
1	Health Care Manpower/	9691
2	exp Health Care Personnel/	880657
3	Psychologist/	6696
4	or/1‐3	883612
5	Manpower Planning/	827
6	Health Care Personnel Management/	900
7	Hospital Personnel Management/	341
8	Resource Allocation/	15104
9	Nurse Patient Ratio/	185
10	Nursing Shortage/	180
11	Personnel Shortage/	940
12	or/5‐11	18172
13	4 and 12	4679
14	((recruit* or retain* or retention or scale up or scaling up) adj6 (nurse or nurses or midwife or midwives or physician* or clinician? or doctor* or practitioner* or dentist* or dental staff or pharmacist* or dietician* or nutritionist* or psychologist* or occupational therapist* or physiotherapist* or physical therapist* or speech therapist* or language therapist* or logopaedist* or logopedist* or speech pathologist* or language pathologist* or audiologist* or internist* or paediatrician* or pediatrician* or ophthalmologist* or surgeon* or radiographer* or radiologist* or optometrist* or personnel or worker? or staff or professional? or provider? or manpower or man power or human resources or workforce?)).ti,ab.	12828
15	((density or turnover or turn over or brain drain* or shortage or understaffed or under staffed or (inequit* adj2 distribut*) or (inequal* adj2 distribut*) or maldistribut* or mal distribut* or management or allocation) adj2 (nurse or nurses or midwife or midwives or physician* or clinician? or doctor* or practitioner* or dentist* or dental staff or pharmacist* or dietician* or nutritionist* or psychologist* or occupational therapist* or physiotherapist* or physical therapist* or speech therapist* or language therapist* or logopaedist* or logopedist* or speech pathologist* or language pathologist* or audiologist* or internist* or paediatrician* or pediatrician* or ophthalmologist* or surgeon* or radiographer* or radiologist* or optometrist* or personnel or worker? or staff or professional? or provider? or manpower or man power or human resources or workforce?)).ti,ab.	12864
16	((increas* or expand* or enlarge* or extend or enhanc*) adj3 (proportion* or number? or amount*) adj3 (nurse or nurses or midwife or midwives or physician* or clinician? or doctor* or practitioner* or dentist* or dental staff or pharmacist* or dietician* or nutritionist* or psychologist* or occupational therapist* or physiotherapist* or physical therapist* or speech therapist* or language therapist* or logopaedist* or logopedist* or speech pathologist* or language pathologist* or audiologist* or internist* or paediatrician* or pediatrician* or ophthalmologist* or surgeon* or radiographer* or radiologist* or optometrist* or personnel or worker? or staff or professional? or provider? or manpower or man power or human resources or workforce?)).ti,ab.	2040
17	(((client adj1 staff) or (patient adj1 staff) or (patient adj1 nurse)) adj ratio?).ti,ab.	261
18	or/14‐17	27185
19	13 or 18	31431
20	Rural Health Care/	9644
21	Rural Area/	28714
22	Rural Population/	28987
23	Medically Underserved/	301
24	(rural adj (health* or medical care or medical service?)).ti,ab.	4429
25	rural medicine.ti,ab.	248
26	((rural or remote or nonmetropolitan or non metropolitan or suburb* or developing or less* developed or under developed or underdeveloped or middle income or low* income or underserved or under served or deprived or poor) adj3 (communit* or area? or village* or region? or province? or setting*)).ti,ab.	71346
27	((rural or remote or nonmetropolitan or non metropolitan or suburb* or middle income or low* income or underserved or under served or deprived or poor or village*) adj3 population*).ti,ab.	16048
28	((rural or remote or nonmetropolitan or non metropolitan or suburb* or village*) adj3 (clinic? or hospital? or facility or facilities or health* center? or health* centre? or health care center? or health care centre? or medical center? or medical centre?)).ti,ab.	9467
29	((shortage or understaffed or under staffed) adj3 area?).ti,ab.	378
30	or/20‐29	123519
31	Developing Country.sh.	74448
32	(Africa or Asia or Caribbean or West Indies or South America or Latin America or Central America).hw,ti,ab,cp.	218967
33	(Afghanistan or Albania or Algeria or Angola or Antigua or Barbuda or Argentina or Armenia or Armenian or Aruba or Azerbaijan or Bahrain or Bangladesh or Barbados or Benin or Byelarus or Byelorussian or Belarus or Belorussian or Belorussia or Belize or Bhutan or Bolivia or Bosnia or Herzegovina or Hercegovina or Botswana or Brazil or Brasil or Bulgaria or Burkina Faso or Burkina Fasso or Upper Volta or Burundi or Urundi or Cambodia or Khmer Republic or Kampuchea or Cameroon or Cameroons or Cameron or Camerons or Cape Verde or Central African Republic or Chad or Chile or China or Colombia or Comoros or Comoro Islands or Comores or Mayotte or Congo or Zaire or Costa Rica or Cote d'Ivoire or Ivory Coast or Croatia or Cuba or Cyprus or Czechoslovakia or Czech Republic or Slovakia or Slovak Republic or Djibouti or French Somaliland or Dominica or Dominican Republic or East Timor or East Timur or Timor Leste or Ecuador or Egypt or United Arab Republic or El Salvador or Eritrea or Estonia or Ethiopia or Fiji or Gabon or Gabonese Republic or Gambia or Gaza or Georgia Republic or Georgian Republic or Ghana or Gold Coast or Greece or Grenada or Guatemala or Guinea or Guam or Guiana or Guyana or Haiti or Honduras or Hungary or India or Maldives or Indonesia or Iran or Iraq or Isle of Man or Jamaica or Jordan or Kazakhstan or Kazakh or Kenya or Kiribati or Korea or Kosovo or Kyrgyzstan or Kirghizia or Kyrgyz Republic or Kirghiz or Kirgizstan or Lao PDR or Laos or Latvia or Lebanon or Lesotho or Basutoland or Liberia or Libya or Lithuania or Macedonia or Madagascar or Malagasy Republic or Malaysia or Malaya or Malay or Sabah or Sarawak or Malawi or Nyasaland or Mali or Malta or Marshall Islands or Mauritania or Mauritius or Agalega Islands or Mexico or Micronesia or Middle East or Moldova or Moldovia or Moldovian or Mongolia or Montenegro or Morocco or Ifni or Mozambique or Myanmar or Myanma or Burma or Namibia or Nepal or Netherlands Antilles or New Caledonia or Nicaragua or Niger or Nigeria or Northern Mariana Islands or Oman or Muscat or Pakistan or Palau or Palestine or Panama or Paraguay or Peru or Philippines or Philipines or Phillipines or Phillippines or Poland or Portugal or Puerto Rico or Romania or Rumania or Roumania or Russia or Russian or Rwanda or Ruanda or Saint Kitts or St Kitts or Nevis or Saint Lucia or St Lucia or Saint Vincent or St Vincent or Grenadines or Samoa or Samoan Islands or Navigator Island or Navigator Islands or Sao Tome or Saudi Arabia or Senegal or Serbia or Montenegro or Seychelles or Sierra Leone or Slovenia or Sri Lanka or Ceylon or Solomon Islands or Somalia or Sudan or Suriname or Surinam or Swaziland or Syria or Tajikistan or Tadzhikistan or Tadjikistan or Tadzhik or Tanzania or Thailand or Togo or Togolese Republic or Tonga or Trinidad or Tobago or Tunisia or Turkey or Turkmenistan or Turkmen or Uganda or Ukraine or Uruguay or USSR or Soviet Union or Union of Soviet Socialist Republics or Uzbekistan or Uzbek or Vanuatu or New Hebrides or Venezuela or Vietnam or Viet Nam or West Bank or Yemen or Yugoslavia or Zambia or Zimbabwe or Rhodesia).hw,ti,ab,cp.	2771032
34	((developing or less* developed or under developed or underdeveloped or middle income or low* income or underserved or under served or deprived or poor*) adj (countr* or nation? or population? or world or area?)).ti,ab.	69052
35	((developing or less* developed or under developed or underdeveloped or middle income or low* income) adj (economy or economies)).ti,ab.	333
36	(low* adj (gdp or gnp or gross domestic or gross national)).ti,ab.	176
37	(low adj3 middle adj3 countr*).ti,ab.	3668
38	(lmic or lmics or third world or lami countr*).ti,ab.	3608
39	transitional countr*.ti,ab.	134
40	or/31‐39	2948110
41	30 or 40	3006527
42	human resources for health.ti,ab.	332
43	Randomized Controlled Trial/	338773
44	Controlled Clinical Trial/	383742
45	Quasi Experimental Study/	1827
46	Pretest Posttest Control Group Design/	199
47	Time Series Analysis/	13620
48	Experimental Design/	9399
49	Multicenter Study/	105390
50	(randomis* or randomiz* or randomly or random allocat*).ti,ab.	695471
51	groups.ab.	1629944
52	(trial or multicentre or multicenter or multi centre or multi center).ti.	182986
53	(intervention* or controlled or control group or compare or compared or (before adj5 after) or (pre adj5 post) or pretest or pre test or posttest or post test or quasiexperiment* or quasi experiment* or evaluat* or effect or impact or time series or time point? or repeated measur*).ti,ab.	7735141
54	or/43‐53	8453577
55	(systematic review or literature review).ti.	57730
56	"cochrane database of systematic reviews".jn.	3777
57	Nonhuman/	4268363
58	or/55‐57	4328296
59	54 not 58	6662154
60	19 and 41 and 59	2825
61	42 and 59	136
62	60 or 61	2916
63	limit 62 to embase	1904

CINAHL, EbscoHost

#	Query	Results
S78	S76 AND S77	514
S77	EM 201202‐ Exclude MEDLINE records	197,281
S76	S73 or S74 Exclude MEDLINE records	1,910
S75	S73 or S74	5,608
S74	S24 and (S40 or S58) and S72	5,368
S73	S12 and S72	417
S72	S59 or S60 or S61 or S62 or S63 or S64 or S65 or S66 or S67 or S68 or S69 or S70 or S71	1,076,806
S71	TI ( (intervention* or controlled or control W0 group* or compare or compared or before N5 after or pre N5 post or pretest or "pre test" or posttest or "post test" or quasiexperiment* or quasi W0 experiment* or evaluat* or effect or impact or "time series" or time W0 point* or repeated W0 measur*) ) OR AB ( (intervention* or controlled or control W0 group* or compare or compared or before N5 after or pre N5 post or pretest or "pre test" or posttest or "post test" or quasiexperiment* or quasi W0 experiment* or evaluat* or effect or impact or "time series" or time W0 point* or repeated W0 measur*) )	600,150
S70	TI ( randomis* or randomiz* or random* W0 allocat* ) OR AB ( randomis* or randomiz* or random* W0 allocat*)	75,701
S69	(MH "Health Services Research")	6,625
S68	(MH "Multicenter Studies")	7,887
S67	(MH "Quasi‐Experimental Studies+")	7,262
S66	(MH "Pretest‐Posttest Design+")	22,840
S65	(MH "Experimental Studies")	13,192
S64	(MH "Nonrandomized Trials")	143
S63	(MH "Intervention Trials")	5,180
S62	(MH "Clinical Trials")	79,824
S61	(MH "Randomized Controlled Trials")	18,517
S60	PT research	872,561
S59	PT clinical trial	51,255
S58	S41 or S42 or S43 or S44 or S45 or S46 or S47 or S48 or S49 or S50 or S51 or S52 or S53 or S54 or S55 or S56 or S57	181,609
S57	TI transitional W0 countr* OR AB transitional W0 countr*	29
S56	TI ( lmic or lmics or third W0 world or lami W0 countr* ) OR AB ( lmic or lmics or third W0 world or lami W0 countr*)	400
S55	TI low N3 middle N3 countr* OR AB low N3 middle N3 countr*	794
S54	TI ( low* W0 (gdp or gnp or gross W0 domestic or gross W0 national) ) OR AB ( low* W0 (gdp or gnp or gross W0 domestic or gross W0 national) )	14
S53	TI ( (developing or less* W0 developed or under W0 developed or underdeveloped or middle W0 income or low* W0 income) W0 (economy or economies) ) OR AB ( (developing or less* W0 developed or under W0 developed or underdeveloped or middle W0 income or low* W0 income) W0 (economy or economies) )	38
S52	TI ( (developing or less* W0 developed or under W0 developed or underdeveloped or middle W0 income or low* W0 income or underserved or under W0 served or deprived or poor*) W0 (countr* or nation or nations or population* or world or area or areas) ) OR AB ( (developing or less* W0 developed or under W0 developed or underdeveloped or middle W0 income or low* W0 income or underserved or under W0 served or deprived or poor*) W0 (countr* or nation or nations or population* or world or area or areas) )	9,522
S51	MW (Afghanistan or Bangladesh or Benin or "Burkina Faso" or Burundi or Cambodia or "Central African Republic" or Chad or Comoros or Congo or "Cote d'Ivoire" or Eritrea or Ethiopia or Gambia or Ghana or Guinea or Haiti or India or Kenya or Korea or Kyrgyz or Kyrgyzstan or Lao or Laos or Liberia or Madagascar or Malawi or Mali or Mauritania or Melanesia or Mongolia or Mozambique or Burma or Myanmar or Nepal or Niger or Nigeria or Pakistan or Rwanda or "Salomon Islands" or "Sao Tome" or Senegal or "Sierra Leone" or Somalia or Sudan or Tajikistan or Tanzania or Timor or Togo or Uganda or Uzbekistan or Vietnam or "Viet Nam" or Yemen or Zambia or Zimbabwe ) or TI ( Afghanistan or Bangladesh or Benin or "Burkina Faso" or Burundi or Cambodia or "Central African Republic" or Chad or Comoros or Congo or "Cote d'Ivoire" or Eritrea or Ethiopia or Gambia or Ghana or Guinea or Haiti or India or Kenya or Korea or Kyrgyz or Kyrgyzstan or Lao or Laos or Liberia or Madagascar or Malawi or Mali or Mauritania or Melanesia or Mongolia or Mozambique or Burma or Myanmar or Nepal or Niger or Nigeria or Pakistan or Rwanda or "Salomon Islands" or "Sao Tome" or Senegal or "Sierra Leone" or Somalia or Sudan or Tajikistan or Tanzania or Timor or Togo or Uganda or Uzbekistan or Vietnam or "Viet Nam" or Yemen or Zambia or Zimbabwe ) or AB ( Afghanistan or Bangladesh or Benin or "Burkina Faso" or Burundi or Cambodia or "Central African Republic" or Chad or Comoros or Congo or "Cote d'Ivoire" or Eritrea or Ethiopia or Gambia or Ghana or Guinea or Haiti or India or Kenya or Korea or Kyrgyz or Kyrgyzstan or Lao or Laos or Liberia or Madagascar or Malawi or Mali or Mauritania or Melanesia or Mongolia or Mozambique or Burma or Myanmar or Nepal or Niger or Nigeria or Pakistan or Rwanda or "Salomon Islands" or "Sao Tome" or Senegal or "Sierra Leone" or Somalia or Sudan or Tajikistan or Tanzania or Timor or Togo or Uganda or Uzbekistan or Vietnam or "Viet Nam" or Yemen or Zambia or Zimbabwe)	39,830
S50	MW (Albania or Algeria or Angola or Armenia or Azerbaijan or Belarus or Bhutan or Bolivia or Bosnia or Herzegovina or "Cape Verde" or Cameroon or China or Colombia or Congo or Cuba or Djibouti or "Dominican Republic" or Ecuador or Egypt or "El Salvador" or Fiji or Gaza or Georgia or Guam or Guatemala or Guyana or Honduras or "Indian Ocean Islands" or Indonesia or Iran or Iraq or Jamaica or Jordan or Kiribati or Lesotho or Macedonia or Maldives or "Marshall Islands" or Micronesia or "Middle East" or Moldova or Morocco or Namibia or Nicaragua or Palestin* or Paraguay or Peru or Philippines or Samoa or "Sri Lanka" or Suriname or Swaziland or Syria or "Syrian Arab Republic" or Thailand or Tonga or Tunisia or Turkmenistan or Ukraine or Vanuatu or "West Bank" ) or TI ( Albania or Algeria or Angola or Armenia or Azerbaijan or Belarus or Bhutan or Bolivia or Bosnia or Herzegovina or "Cape Verde" or Cameroon or China or Colombia or Congo or Cuba or Djibouti or "Dominican Republic" or Ecuador or Egypt or "El Salvador" or Fiji or Gaza or Georgia or Guam or Guatemala or Guyana or Honduras or "Indian Ocean Islands" or Indonesia or Iran or Iraq or Jamaica or Jordan or Kiribati or Lesotho or Macedonia or Maldives or "Marshall Islands" or Micronesia or "Middle East" or Moldova or Morocco or Namibia or Nicaragua or Palestin* or Paraguay or Peru or Philippines or Samoa or "Sri Lanka" or Suriname or Swaziland or Syria or "Syrian Arab Republic" or Thailand or Tonga or Tunisia or Turkmenistan or Ukraine or Vanuatu or "West Bank" ) or AB ( Albania or Algeria or Angola or Armenia or Azerbaijan or Belarus or Bhutan or Bolivia or Bosnia or Herzegovina or "Cape Verde" or Cameroon or China or Colombia or Congo or Cuba or Djibouti or "Dominican Republic" or Ecuador or Egypt or "El Salvador" or Fiji or Gaza or Georgia or Guam or Guatemala or Guyana or Honduras or "Indian Ocean Islands" or Indonesia or Iran or Iraq or Jamaica or Jordan or Kiribati or Lesotho or Macedonia or Maldives or "Marshall Islands" or Micronesia or "Middle East" or Moldova or Morocco or Namibia or Nicaragua or Palestin* or Paraguay or Peru or Philippines or Samoa or "Sri Lanka" or Suriname or Swaziland or Syria or "Syrian Arab Republic" or Thailand or Tonga or Tunisia or Turkmenistan or Ukraine or Vanuatu or "West Bank")	44,109
S49	MW ("American Samoa" or Argentina or Belize or Botswana or Brazil or Brasil or Bulgaria or Chile or Comoros or "Costa Rica" or Croatia or Dominica or Guinea or Gabon or Grenada or Grenadines or Hungary or Kazakhstan or Latvia or Lebanon or Libia or libyan or Libya or Lithuania or Malaysia or Mauritius or Mayotte or Mexico or Micronesia or Montenegro or Nevis or "Northern Mariana Islands" or Oman or Palau or Panama or Poland or Romania or Russia or "Russian Federation" or Samoa or "Saint Lucia" or "St Lucia" or "Saint Kitts" or "St Kitts" or "Saint Vincent" or "St Vincent" or Serbia or Seychelles or Slovakia or "Slovak Republic" or "South Africa" or Turkey or Uruguay or Venezuela or Yugoslavia ) or TI ( "American Samoa" or Argentina or Belize or Botswana or Brazil or Bulgaria or Chile or Comoros or "Costa Rica" or Croatia or Dominica or Guinea or Gabon or Grenada or Grenadines or Hungary or Kazakhstan or Latvia or Lebanon or Libia or libyan or Libya or Lithuania or Malaysia or Mauritius or Mayotte or Mexico or Micronesia or Montenegro or Nevis or "Northern Mariana Islands" or Oman or Palau or Panama or Poland or Romania or Russia or "Russian Federation" or Samoa or "Saint Lucia" or "St Lucia" or "Saint Kitts" or "St Kitts" or "Saint Vincent" or "St Vincent" or Serbia or Seychelles or Slovakia or "Slovak Republic" or "South Africa" or Turkey or Uruguay or Venezuela or Yugoslavia ) or AB ( "American Samoa" or Argentina or Belize or Botswana or Brazil or Bulgaria or Chile or Comoros or "Costa Rica" or Croatia or Dominica or Guinea or Gabon or Grenada or Grenadines or Hungary or Kazakhstan or Latvia or Lebanon or Libia or libyan or Libya or Lithuania or Malaysia or Mauritius or Mayotte or Mexico or Micronesia or Montenegro or Nevis or "Northern Mariana Islands" or Oman or Palau or Panama or Poland or Romania or Russia or "Russian Federation" or Samoa or "Saint Lucia" or "St Lucia" or "Saint Kitts" or "St Kitts" or "Saint Vincent" or "St Vincent" or Serbia or Seychelles or Slovakia or "Slovak Republic" or "South Africa" or Turkey or Uruguay or Venezuela or Yugoslavia)	53,913
S48	TI ( (Africa or Asia or "South America" or "Latin America" or "Central America") ) OR AB ( (Africa or Asia or "South America" or "Latin America" or "Central America") )	11,841
S47	(MH "Asia+")	84,896
S46	(MH "West Indies+")	4,675
S45	(MH "South America+")	22,262
S44	(MH "Latin America")	1,116
S43	(MH "Central America+")	1,942
S42	(MH "Africa+")	27,922
S41	(MH "Developing Countries")	8,157
S40	S25 or S26 or S27 or S28 or S29 or S30 or S31 or S32 or S33 or S34 or S35 or S36 or S37 or S38 or S39	27,535
S39	TI ( (shortage or understaffed or under W0 staffed) N3 (area or areas) ) OR AB ( (shortage or understaffed or under W0 staffed) N3 (area or areas) )	119
S38	TI ( (rural or remote or nonmetropolitan or non W0 metropolitan or suburb* or village*) N3 (clinic or clinics or hospital or hospitals or facility or facilities or health* W0 center or health* W0 centers or health* W0 centre or health* W0 centres or health care center or health care centers or health care centre or health care centres or medical center or medical centers or medical centre or medical centres) ) OR AB ( (rural or remote or nonmetropolitan or non W0 metropolitan or suburb* or village*) N3 (clinic or clinics or hospital or hospitals or facility or facilities or health* W0 center or health* W0 centers or health* W0 centre or health* W0 centres or health care center or health care centers or health care centre or health care centres or medical center or medical centers or medical centre or medical centres) )	2,537
S37	TI ( (rural or remote or nonmetropolitan or non W0 metropolitan or suburb* or middle W0 income or low* W0 income or underserved or under W0 served or deprived or poor or village*) N3 population* ) OR AB ( (rural or remote or nonmetropolitan or non W0 metropolitan or suburb* or middle W0 income or low* W0 income or underserved or under W0 served or deprived or poor or village*) N3 population* )	3,146
S36	TI ( (rural or remote or nonmetropolitan or non W0 metropolitan or suburb* or developing or less* W0 developed or under W0 developed or underdeveloped or middle W0 income or low* W0 income or underserved or under W0 served or deprived or poor) N3 (communit* or area or areas or village* or region or regions or province* or setting*) ) OR AB ( (rural or remote or nonmetropolitan or non W0 metropolitan or suburb* or developing or less* W0 developed or under W0 developed or underdeveloped or middle W0 income or low* W0 income or underserved or under W0 served or deprived or poor) N3 (communit* or area or areas or village* or region or regions or province* or setting*) )	14,046
S35	TI ( rural W0 (health* or medical care or medical service*) ) OR AB ( rural W0 (health* or medical care or medical service*) )	1,384
S34	(MH "Medically Underserved")	1,768
S33	(MH "Rural Population")	1,863
S32	(MH "Medically Underserved Area")	867
S31	(MH "Suburban Health")	9
S30	(MH "Frontier Nursing Service")	554
S29	(MH "Rural Health Nursing")	1,584
S28	(MH "Hospitals, Rural")	1,628
S27	(MH "Rural Health Centers")	114
S26	(MH "Rural Health Services")	3,650
S25	(MH "Rural Health")	3,309
S24	S19 or S20 or S21 or S22 or S23	92,259
S23	TI ( ((client N1 staff) or (patient N1 staff) or (patient N1 nurse) W0 (ratio or ratios)) ) OR AB ( ((client N1 staff) or (patient N1 staff) or (patient N1 nurse) W0 (ratio or ratios)) )	1,731
S22	TI ( (increas* or expand* or enlarge* or extend or enhanc*) N3 (nurse or nurses or midwife or midwives or physician* or clinician* or doctor* or practitioner* or dentist* or "dental staff" or pharmacist* or dietician* or nutritionist* or psychologist* or occupational W0 therapist* or physiotherapist* or physical W0 therapist* or speech W0 therapist* or language W0 therapist* or logopaedist* or logopedist* or speech W0 pathologist* or language W0 pathologist* or audiologist* or internist* or paediatrician* or pediatrician* or ophthalmologist* or surgeon* or radiographer* or radiologist* or optometrist* or personnel or worker or workers or staff or professional* or provider* or manpower or "man power" or "human resources" or workforce*) ) OR AB ( (increas* or expand* or enlarge* or extend or enhanc*) N3 (nurse or nurses or midwife or midwives or physician* or clinician* or doctor* or practitioner* or dentist* or "dental staff" or pharmacist* or dietician* or nutritionist* or psychologist* or occupational W0 therapist* or physiotherapist* or physical W0 therapist* or speech W0 therapist* or language W0 therapist* or logopaedist* or logopedist* or speech W0 pathologist* or language W0 pathologist* or audiologist* or internist* or paediatrician* or pediatrician* or ophthalmologist* or surgeon* or radiographer* or radiologist* or optometrist* or personnel or worker or workers or staff or professional* or provider or manpower or "man power" or "human resources" or workforce*) )	14,948
S21	TI ( (density or turnover or "turn over" or brain W0 drain* or shortage or understaffed or "under staffed" or (inequit* N2 distribut*) or (inequal* N2 distribut*) or maldistribut* or mal W0 distribut* or management or allocation) N2 (nurse or nurses or midwife or midwives or physician* or clinician* or doctor* or practitioner* or dentist* or "dental staff" or pharmacist* or dietician* or nutritionist* or psychologist* or occupational W0 therapist* or physiotherapist* or physical W0 therapist* or speech W0 therapist* or language W0 therapist* or logopaedist* or logopedist* or speech W0 pathologist* or language W0 pathologist* or audiologist* or internist* or paediatrician* or pediatrician* or ophthalmologist* or surgeon* or radiographer* or radiologist* or optometrist* or personnel or worker or workers or staff or professional* or provider* or manpower or "man power" or "human resources" or workforce*) ) OR AB ( (density or turnover or "turn over" or brain W0 drain* or shortage or understaffed or "under staffed" or (inequit* N2 distribut*) or (inequal* N2 distribut*) or maldistribut* or mal W0 distribut* or management or allocation) N2 (nurse or nurses or midwife or midwives or physician* or clinician* or doctor* or practitioner* or dentist* or "dental staff" or pharmacist* or dietician* or nutritionist* or psychologist* or occupational W0 therapist* or physiotherapist* or physical W0 therapist* or speech W0 therapist* or language W0 therapist* or logopaedist* or logopedist* or speech W0 pathologist* or language W0 pathologist* or audiologist* or internist* or paediatrician* or pediatrician* or ophthalmologist* or surgeon* or radiographer* or radiologist* or optometrist* or personnel or worker or workers or staff or professional* or provider or manpower or "man power" or "human resources" or workforce*) )	8,972
S20	TI ( recruit* or retain* or retention or "scale up" or "scaling up") N6 (nurse or nurses or midwife or midwives or physician* or clinician* or doctor* or practitioner* or dentist* or "dental staff" or pharmacist* or dietician* or nutritionist* or psychologist* or occupational W0 therapist* or physiotherapist* or physical W0 therapist* or speech W0 therapist* or language W0 therapist* or logopaedist* or logopedist* or speech W0 pathologist* or language W0 pathologist* or audiologist* or internist* or paediatrician* or pediatrician* or ophthalmologist* or surgeon* or radiographer* or radiologist* or optometrist* or personnel or worker or workers or staff or professional* or provider or manpower or "man power" or "human resources" or workforce* ) OR AB ( recruit* or retain* or retention or "scale up" or "scaling up") N6 (nurse or nurses or midwife or midwives or physician* or clinician* or doctor* or practitioner* or dentist* or "dental staff" or pharmacist* or dietician* or nutritionist* or psychologist* or occupational W0 therapist* or physiotherapist* or physical W0 therapist* or speech W0 therapist* or language W0 therapist* or logopaedist* or logopedist* or speech W0 pathologist* or language W0 pathologist* or audiologist* or internist* or paediatrician* or pediatrician* or ophthalmologist* or surgeon* or radiographer* or radiologist* or optometrist* or personnel or worker or workers or staff or professional* or provider* or manpower or "man power" or "human resources" or workforce* )	7,025
S19	S15 and S18	67,442
S18	S16 or S17	167,282
S17	(MH "Health Resource Allocation")	5,754
S16	(MH "Personnel Management+")	162,039
S15	S13 or S14	328,995
S14	(MH "Health Manpower+")	312,565
S13	(MH "Health Personnel+")	306,976
S12	S1 or S2 or S3 or S4 or S5 or S6 or S7 or S8 or S9 or S10 or S11	800
S11	TI "human resources for health" OR AB "human resources for health"	98
S10	(MH "Resource Allocation/MA")	2
S9	(MH "Health Resource Allocation/MA")	1
S8	(MH "Medically Underserved Area/MA")	2
S7	(MH "Frontier Nursing Service/MA")	3
S6	(MH "Services for Australian Rural and Remote Allied Health")	2
S5	(MH "Rural Health Nursing/MA")	26
S4	(MH "Hospitals, Rural/MA")	32
S3	(MH "Rural Health Centers/MA")	1
S2	(MH "Rural Health Personnel")	424
S1	(MH "Rural Health Services/MA")	227

LILACS (VHL)

(nurse or nurses or midwife or midwives or physician or physicians or clinician or clinicians or doctor or doctors or practitioner or practitioners or dentist or dentists or pharmacist or pharmacists or "health care staff" or "healthcare staff" or "medical staff" or "health personnel" or "health care personnel" or "healthcare personnel" or "medical personnel" or "health worker" or "health workers" or "health care worker" or "health care workers" or "healthcare worker" or "healthcare workers" or "medical worker" or "medical workers" or "health professional" or "health professionals" or "health care professional" or "health care professionals" or "healthcare professional" or "healthcare professionals" or "medical professional" or "medical professionals" or "health provider" or "health providers" or "health care provider" or "health care providers" or "healthcare provider" or "healthcare providers" or "medical provider" or "medical providers" or "health workforce" or "health care workforce" or "healthcare workforce" or "medical workforce" or manpower or "man power" or "human resources" or enfermer* or médico* or odontólogo* or farmacéutico* or partera* or "equipo medico" or "equipo sanitario" or "trabajadores de salud" or "trabajadores de la salud" or "recursos humanos" or enfermeir* or médico* or odontólogo* or farmacêutico* or parteira* or "profissionais de saúde") AND (recruit* or retain* or retention or distribut* or "scale up" or "scaling up" or turnover or "turn over" or "brain drain" or maldistribution or "mal distribution" or distribucion or retencion or distribuição or fixação or retenção) AND ("Medically Underserved Area" or "Área sin Atención Médica" or "Área Carente de Assistência Médica" or "Equity in Access" or "Equidad en el Acceso" or "Equidade no Acesso" or "Health Services Accessibility" or "Accesibilidad a los Servicios de Salud" or "Acesso aos Serviços de Saúde" or "Rural Areas" or "Zonas Rurales" or "Zonas Rurais" or "Rural Health Services" or "Servicios Rurales de Salud" or "Serviços Rurais de Saúde" or "Rural Population" or "Población Rural" or "População Rural" or "Suburban Health Services" or "Servicios Suburbanos de Salud" or "Serviços Suburbanos de Saúde" or desatendid* or carente* or "mal servedas" or subatendida* or shortage or escasez or escassez or rural* or rurais) AND (randomised or randomized or "random allocation" or "randomly allocated" or "controlled trial" or "control group" or evaluat* or effect* or impact or intervention* or "multicenter study" or "multi center study" or "multicentre study" or "multi centre study" or " before and after" or "pre and post" or pretest or "pre test" or posttest or "post test" or quasiexperiment* or "quasi experiment" or evaluat* or effect or impact or "time series" or "time point" or "time points" or "repeated measure" or "repeated measures" or "repeated measurement" or "repeated measurements" or intervencion* or azar or acaso or intervencao* or evaluar or evaluacion or avaliacao or efecto or efectos or efeito or efeitos or impacto or impactos or "serie temporal" or "series temporal" or "serie temporales" or "series temporales" or "series temporais" or "puntos de tiempo" or "puntos temporales" or "pontos temporais" or "medida repetida" or "medida repetidas" or "medidas repetida" or "medidas repetidas" or "medicion repetida" or "medicion repetidas" or "mediciones repetida" or "mediciones repetidas" or "ensayo clinico controlado aleatorio" or "ensayo clinico controlado" or aleatorios or evalua* or "series de tiempo" or "pre test" or "prueba previa" or "despues de la prueba" or effect* or experiment* or "ensaio clinico controlado aleatorio" or "ensaio clinico controlado" or avalia* or "series temporais" or "pre teste" or "pos teste")

Science Citation Index and Social Sciences Citation Index (Web of Knowledge)

# 13	271	#12 AND #11 Indexes=SCI‐EXPANDED, SSCI Timespan=All years
# 12	3,895,370	YEAR PUBLISHED: (2012‐2014) Indexes=SCI‐EXPANDED, SSCI Timespan=All years
# 11	893	#10 AND #9 AND #1 Indexes=SCI‐EXPANDED, SSCI Timespan=All years
# 10	12,754,907	TOPIC: ((randomised or randomized or "random allocation" or "randomly allocated" or trial or multicenter or "multi center" or multicentre or "multi centre" or intervention or interventions or controlled or "control group" or compare or compared or "before and after" or "pre and post" or ((pretest or "pre test") AND (posttest or "post test")) or quasiexperiment or quasiexperiments or experiment or experiments or evaluat* or effect or impact or "time series" or "time point" or "time points" or "repeated measure" or "repeated measures" or "repeated measurement" or "repeated measurements")) Indexes=SCI‐EXPANDED, SSCI Timespan=All years
# 9	223,272	#8 OR #7 OR #6 OR #5 OR #4 OR #3 OR #2 Indexes=SCI‐EXPANDED, SSCI Timespan=All years
# 8	2,617	TOPIC: ((developing or "less developed" or "lesser developed" or "under developed" or underdeveloped or "middle income" or "low income" or "lower income") near/0 (economy or economies)) Indexes=SCI‐EXPANDED, SSCI Timespan=All years
# 7	8,526	TOPIC: ((lmic or lmics or "third world" or "lami country" or "lami countries")) Indexes=SCI‐EXPANDED, SSCI Timespan=All years
# 6	106,786	TOPIC: ((developing or "less developed" or "lesser developed" or "under developed" or "underdeveloped" or "middle income" or "low income" or "lower income" or underserved or "under served" or deprived or poor) NEAR/0 (country or countries or nation or nations or population or populations or world or area or areas)) Indexes=SCI‐EXPANDED, SSCI Timespan=All years
# 5	170	TOPIC: (("shortage area" or "shortage areas" or "understaffed area" or "understaffed areas" or "under staffed area" or "under staffed areas")) Indexes=SCI‐EXPANDED, SSCI Timespan=All years
# 4	7,588	TOPIC: ((rural or remote or nonmetropolitan or "non metropolitan" or village or villages) NEAR/3 (clinic or clinics or hospital or hospitals or facility or facilities or center or centers or centre or centres)) Indexes=SCI‐EXPANDED, SSCI Timespan=All years
# 3	16,052	TOPIC: ((rural or remote or nonmetropolitan or "non metropolitan" or "middle income" or "low income" or "lower income" or underserved or "under served" or deprived or poor or village or villages) NEAR/3 (population or populations)) Indexes=SCI‐EXPANDED, SSCI Timespan=All years
# 2	107,542	TOPIC: ((rural or remote or nonmetropolitan or "non metropolitan" or developing or "less developed" or "lesser developed" or "under developed" or underdeveloped or "middle income" or "low income" or "lower income" or underserved or "under served" or deprived or poor) NEAR/3 (community or communities or area or areas or village or villages or region or regions or province or provinces or setting or settings)) Indexes=SCI‐EXPANDED, SSCI Timespan=All years
# 1	15,904	TOPIC: ((nurse or nurses or midwife or midwives or physician* or clinician* or doctor or doctors or practitioner* or dentist or dentists or staff or pharmacist* or dietician* or nutritionist* or psychologist* or therapist* or physiotherapist* or logopaedist* or logopedist* or pathologist* or audiologist* or internist* or paediatrician* or pediatrician* or ophthalmologist* or surgeon* or radiographer* or radiologist* or optometrist* or personnel or worker or workers or staff or professional or professionals or provider* or manpower or "man power" or "human resources" or workforce) near/6 (recruit* or retain* or retention or distribution or "inadequately distributed" or "unequally distributed" or misdistribution or misdistributed or maldistribution or "mal distribution" or "scale up" or "scaling up" or turnover or "turn over" or "brain drain")) Indexes=SCI‐EXPANDED, SSCI Timespan=All years

Global Health, CAB Direct

("health professionals" OR "healthcare professionals" OR "health care professionals" OR "health personnel" OR "healthcare personnel" OR "health care personnel" OR "health manpower" OR "healthcare manpower" OR "health care manpower" OR "health workforce" OR "healthcare workforce" OR "health care workforce" OR "human resources" OR "doctor" OR "doctors" or "physician" OR "physicians" OR "midwife" OR "midwifes") (All fields)

AND

(("increase" AND "proportion") OR ("increasing" AND "proportion") OR ("increase" AND "distribution") OR ("increasing" AND "distribution") OR ("recruit" AND strategy") OR ("recruiting" AND strategy") OR ("recruitment" and "strategy") OR ("recruit" AND strategies") OR ("recruiting" AND strategies") OR ("recruitment" and "strategies") OR "retain" OR "retention" OR "retaining" OR "scale up" OR "scaling up" OR "brain drain") (All fields)

AND

("low income country" OR "low income countries" OR "middle income country" OR "middle income countries" OR "low income economy" OR "low income economies" OR "middle income economy" OR "middle income economies" OR "LMIC" OR "LMICS" OR "developing country" OR "developing countries" OR "underserved area" OR "underserved areas" OR "rural area" OR "rural areas" OR "shortage area" OR "shortage areas") (All fields)

AND

("randomised" OR "randomized" OR "randomly allocated" OR "random allocation" OR "controlled trial" OR "control group" OR "control groups" OR "quasiexperiment" OR "quasiexperiments" OR "quasiexperimental" OR "quasi experiment" OR "quasi experiments" OR "quasi experimental" OR "evaluate" OR "evaluating" OR "evaluation" OR "time series" OR "time point" OR "time points" OR "repeated measure" OR "repeated measures" OR "repeated measurement" OR "repeated measurements" OR "before and after" OR "pre and post" OR (("pretest" OR "pre test") AND ("posttest" OR "post test")) OR "trial" OR "effect" OR "effects" OR "impact" OR "impacts" OR "intervention" OR "interventions" OR "multicenter" OR "multi center" OR "multicentre" OR "multi centre") (All fields)

AND

sc:"X9" OR sc:"HE" (All fields)

Appendix 2. Results of re‐analysis of data from Yang 2013

Western Medicine Physicians

GET DATA /TYPE=XLSX

/FILE='C:\Users\TonyaE\Documents\Stellenbosch\Consulting\Liesel Nicol\Copy of Yang 2013 raw data.xlsx'

/SHEET=name 'Western meds'

/CELLRANGE=full

/READNAMES=on

/ASSUMEDSTRWIDTH=32767.

EXECUTE.

DATASET NAME DataSet3 WINDOW=FRONT.

SAVE OUTFILE='C:\Users\TonyaE\Documents\Stellenbosch\Consulting\Liesel Nicol\western meds.sav'

/COMPRESSED.

PREDICT THRU END.

* Time Series Modeler.

TSMODEL

/MODELSUMMARY PRINT=[MODELFIT]

/MODELSTATISTICS DISPLAY=YES MODELFIT=[ SRSQUARE]

/MODELDETAILS PRINT=[ PARAMETERS]

/SERIESPLOT OBSERVED FORECAST

/OUTPUTFILTER DISPLAY=ALLMODELS

/SAVE PREDICTED(Predicted)

/AUXILIARY CILEVEL=95 MAXACFLAGS=24

/MISSING USERMISSING=EXCLUDE

/MODEL DEPENDENT=Outcomes INDEPENDENT=timeperiod phase interact

PREFIX='Model'

/ARIMA AR=[1] DIFF=0 MA=[0]

TRANSFORM=NONE CONSTANT=YES

/AUTOOUTLIER DETECT=OFF.

Time Series Modeler

[DataSet3] C:\Users\TonyaE\Documents\Stellenbosch\Consulting\Liesel Nicol\western meds.sav

Model Description
			Model Type
Model ID	Outcomes	Model_1	ARIMA(1,0,0)

Model Summary

Model Fit
Fit Statistic	Mean	SE	Minimum	Maximum	Percentile
					5	10	25	50	75	90	95
Stationary R‐squared	.599	.	.599	.599	.599	.599	.599	.599	.599	.599	.599
R‐squared	.599	.	.599	.599	.599	.599	.599	.599	.599	.599	.599
RMSE	.006	.	.006	.006	.006	.006	.006	.006	.006	.006	.006
MAPE	1.837	.	1.837	1.837	1.837	1.837	1.837	1.837	1.837	1.837	1.837
MaxAPE	5.101	.	5.101	5.101	5.101	5.101	5.101	5.101	5.101	5.101	5.101
MAE	.005	.	.005	.005	.005	.005	.005	.005	.005	.005	.005
MaxAE	.012	.	.012	.012	.012	.012	.012	.012	.012	.012	.012
Normalized BIC	‐9.641	.	‐9.641	‐9.641	‐9.641	‐9.641	‐9.641	‐9.641	‐9.641	‐9.641	‐9.641

Model Statistics
Model	Number of Predictors	Model Fit statistics	Ljung‐Box Q(18)			Number of Outliers
		Stationary R‐squared	Statistics	DF	Sig.
Outcomes‐Model_1	3	.599	6.907	17	.985	0

ARIMA Model Parameters
					Estimate	SE	t	Sig.
Outcomes‐Model_1	Outcomes	No Transformation	Constant		.242	.002	111.214	.000
			AR	Lag 1	‐.213	.189	‐1.125	.271
	time period	No Transformation	Numerator	Lag 0	.001	.000	6.577	.000
	phase	No Transformation	Numerator	Lag 0	.082	.024	3.404	.002
	interact	No Transformation	Numerator	Lag 0	‐.004	.001	‐4.186	.000

PREDICT THRU END.

* Time Series Modeler.

TSMODEL

/MODELSUMMARY PRINT=[MODELFIT]

/MODELSTATISTICS DISPLAY=YES MODELFIT=[ SRSQUARE]

/MODELDETAILS PRINT=[ PARAMETERS]

/SERIESPLOT OBSERVED FORECAST

/OUTPUTFILTER DISPLAY=ALLMODELS

/SAVE PREDICTED(Predicted)

/AUXILIARY CILEVEL=95 MAXACFLAGS=24

/MISSING USERMISSING=EXCLUDE

/MODEL DEPENDENT=Outcomes INDEPENDENT=phase pre4 post4

PREFIX='Model'

/ARIMA AR=[1] DIFF=0 MA=[0]

TRANSFORM=NONE CONSTANT=YES

/AUTOOUTLIER DETECT=OFF.

Time Series Modeler

Model Description
			Model Type
Model ID	Outcomes	Model_1	ARIMA(1,0,0)

Model Summary

Model Fit
Fit Statistic	Mean	SE	Minimum	Maximum	Percentile
					5	10	25	50	75	90	95
Stationary R‐squared	.599	.	.599	.599	.599	.599	.599	.599	.599	.599	.599
R‐squared	.599	.	.599	.599	.599	.599	.599	.599	.599	.599	.599
RMSE	.006	.	.006	.006	.006	.006	.006	.006	.006	.006	.006
MAPE	1.837	.	1.837	1.837	1.837	1.837	1.837	1.837	1.837	1.837	1.837
MaxAPE	5.102	.	5.102	5.102	5.102	5.102	5.102	5.102	5.102	5.102	5.102
MAE	.005	.	.005	.005	.005	.005	.005	.005	.005	.005	.005
MaxAE	.012	.	.012	.012	.012	.012	.012	.012	.012	.012	.012
Normalized BIC	‐9.641	.	‐9.641	‐9.641	‐9.641	‐9.641	‐9.641	‐9.641	‐9.641	‐9.641	‐9.641

Model Statistics
Model	Number of Predictors	Model Fit statistics	Ljung‐Box Q(18)			Number of Outliers
		Stationary R‐squared	Statistics	DF	Sig.
Outcomes‐Model_1	3	.599	6.907	17	.985	0

ARIMA Model Parameters
					Estimate	SE	t	Sig.
Outcomes‐Model_1	Outcomes	No Transformation	Constant		.242	.002	111.213	.000
			AR	Lag 1	‐.213	.189	‐1.125	.271
	phase	No Transformation	Numerator	Lag 0	‐.018	.003	‐5.426	.000
	pre4	No Transformation	Numerator	Lag 0	.001	.000	6.577	.000
	post4	No Transformation	Numerator	Lag 0	‐.003	.001	‐3.031	.005

PREDICT THRU END.

* Time Series Modeler.

TSMODEL

/MODELSUMMARY PRINT=[MODELFIT]

/MODELSTATISTICS DISPLAY=YES MODELFIT=[ SRSQUARE]

/MODELDETAILS PRINT=[ PARAMETERS]

/SERIESPLOT OBSERVED FORECAST

/OUTPUTFILTER DISPLAY=ALLMODELS

/SAVE PREDICTED(Predicted)

/AUXILIARY CILEVEL=95 MAXACFLAGS=24

/MISSING USERMISSING=EXCLUDE

/MODEL DEPENDENT=Outcomes INDEPENDENT=phase pre8 post8

PREFIX='Model'

/ARIMA AR=[1] DIFF=0 MA=[0]

TRANSFORM=NONE CONSTANT=YES

/AUTOOUTLIER DETECT=OFF.

Time Series Modeler

Model Description
			Model Type
Model ID	Outcomes	Model_1	ARIMA(1,0,0)

Model Summary

Model Fit
Fit Statistic	Mean	SE	Minimum	Maximum	Percentile
					5	10	25	50	75	90	95
Stationary R‐squared	.599	.	.599	.599	.599	.599	.599	.599	.599	.599	.599
R‐squared	.599	.	.599	.599	.599	.599	.599	.599	.599	.599	.599
RMSE	.006	.	.006	.006	.006	.006	.006	.006	.006	.006	.006
MAPE	1.837	.	1.837	1.837	1.837	1.837	1.837	1.837	1.837	1.837	1.837
MaxAPE	5.102	.	5.102	5.102	5.102	5.102	5.102	5.102	5.102	5.102	5.102
MAE	.005	.	.005	.005	.005	.005	.005	.005	.005	.005	.005
MaxAE	.012	.	.012	.012	.012	.012	.012	.012	.012	.012	.012
Normalized BIC	‐9.641	.	‐9.641	‐9.641	‐9.641	‐9.641	‐9.641	‐9.641	‐9.641	‐9.641	‐9.641

Model Statistics
Model	Number of Predictors	Model Fit statistics	Ljung‐Box Q(18)			Number of Outliers
		Stationary R‐squared	Statistics	DF	Sig.
Outcomes‐Model_1	3	.599	6.907	17	.985	0

ARIMA Model Parameters
					Estimate	SE	t	Sig.
Outcomes‐Model_1	Outcomes	No Transformation	Constant		.242	.002	111.213	.000
			AR	Lag 1	‐.213	.189	‐1.125	.271
	phase	No Transformation	Numerator	Lag 0	‐.032	.005	‐6.842	.000
	pre8	No Transformation	Numerator	Lag 0	.001	.000	6.577	.000
	post8	No Transformation	Numerator	Lag 0	‐.003	.001	‐3.031	.005

Chinese Medicine Physicians

DATASET ACTIVATE DataSet1.

SAVE OUTFILE='C:\Users\TonyaE\Documents\Stellenbosch\Consulting\Liesel Nicol\dentist.sav'

/COMPRESSED.

DATASET ACTIVATE DataSet1.

SAVE OUTFILE='C:\Users\TonyaE\Documents\Stellenbosch\Consulting\Liesel Nicol\dentist.sav'

/COMPRESSED.

GET DATA /TYPE=XLSX

/FILE='C:\Users\TonyaE\Documents\Stellenbosch\Consulting\Liesel Nicol\Copy of Yang 2013 raw data.xlsx'

/SHEET=name 'chinese meds'

/CELLRANGE=full

/READNAMES=on

/ASSUMEDSTRWIDTH=32767.

EXECUTE.

DATASET NAME DataSet2 WINDOW=FRONT.

SAVE OUTFILE='C:\Users\TonyaE\Documents\Stellenbosch\Consulting\Liesel Nicol\chinese meds.sav'

/COMPRESSED.

DATASET ACTIVATE DataSet2.

SAVE OUTFILE='C:\Users\TonyaE\Documents\Stellenbosch\Consulting\Liesel Nicol\chinese meds.sav'

/COMPRESSED.

PREDICT THRU END.

* Time Series Modeler.

TSMODEL

/MODELSUMMARY PRINT=[MODELFIT]

/MODELSTATISTICS DISPLAY=YES MODELFIT=[ SRSQUARE]

/MODELDETAILS PRINT=[ PARAMETERS]

/SERIESPLOT OBSERVED FORECAST

/OUTPUTFILTER DISPLAY=ALLMODELS

/SAVE PREDICTED(Predicted)

/AUXILIARY CILEVEL=95 MAXACFLAGS=24

/MISSING USERMISSING=EXCLUDE

/MODEL DEPENDENT=Outcomes INDEPENDENT=timeperiod phase interact

PREFIX='Model'

/ARIMA AR=[1] DIFF=0 MA=[0]

TRANSFORM=NONE CONSTANT=YES

/AUTOOUTLIER DETECT=OFF.

Time Series Modeler

[DataSet2] C:\Users\TonyaE\Documents\Stellenbosch\Consulting\Liesel Nicol\chinese meds.sav

Model Description
			Model Type
Model ID	Outcomes	Model_1	ARIMA(1,0,0)

Model Summary

Model Fit
Fit Statistic	Mean	SE	Minimum	Maximum	Percentile
					5	10	25	50	75	90	95
Stationary R‐squared	.764	.	.764	.764	.764	.764	.764	.764	.764	.764	.764
R‐squared	.764	.	.764	.764	.764	.764	.764	.764	.764	.764	.764
RMSE	.013	.	.013	.013	.013	.013	.013	.013	.013	.013	.013
MAPE	3.771	.	3.771	3.771	3.771	3.771	3.771	3.771	3.771	3.771	3.771
MaxAPE	12.798	.	12.798	12.798	12.798	12.798	12.798	12.798	12.798	12.798	12.798
MAE	.009	.	.009	.009	.009	.009	.009	.009	.009	.009	.009
MaxAE	.032	.	.032	.032	.032	.032	.032	.032	.032	.032	.032
Normalized BIC	‐8.085	.	‐8.085	‐8.085	‐8.085	‐8.085	‐8.085	‐8.085	‐8.085	‐8.085	‐8.085

Model Statistics
Model	Number of Predictors	Model Fit statistics	Ljung‐Box Q(18)			Number of Outliers
		Stationary R‐squared	Statistics	DF	Sig.
Outcomes‐Model_1	3	.764	13.930	17	.672	0

ARIMA Model Parameters
					Estimate	SE	t	Sig.
Outcomes‐Model_1	Outcomes	No Transformation	Constant		.264	.008	33.314	.000
			AR	Lag 1	.322	.187	1.725	.096
	time period	No Transformation	Numerator	Lag 0	‐.002	.001	‐2.990	.006
	phase	No Transformation	Numerator	Lag 0	.047	.074	.638	.529
	interact	No Transformation	Numerator	Lag 0	‐.002	.003	‐.842	.407

PREDICT THRU END.

* Time Series Modeler.

TSMODEL

/MODELSUMMARY PRINT=[MODELFIT]

/MODELSTATISTICS DISPLAY=YES MODELFIT=[ SRSQUARE]

/MODELDETAILS PRINT=[ PARAMETERS]

/SERIESPLOT OBSERVED FORECAST

/OUTPUTFILTER DISPLAY=ALLMODELS

/SAVE PREDICTED(Predicted)

/AUXILIARY CILEVEL=95 MAXACFLAGS=24

/MISSING USERMISSING=EXCLUDE

/MODEL DEPENDENT=Outcomes INDEPENDENT=phase pre4 post4

PREFIX='Model'

/ARIMA AR=[1] DIFF=0 MA=[0]

TRANSFORM=NONE CONSTANT=YES

/AUTOOUTLIER DETECT=OFF.

Time Series Modeler

Model Description
			Model Type
Model ID	Outcomes	Model_1	ARIMA(1,0,0)

Model Summary

Model Fit
Fit Statistic	Mean	SE	Minimum	Maximum	Percentile
					5	10	25	50	75	90	95
Stationary R‐squared	.764	.	.764	.764	.764	.764	.764	.764	.764	.764	.764
R‐squared	.764	.	.764	.764	.764	.764	.764	.764	.764	.764	.764
RMSE	.013	.	.013	.013	.013	.013	.013	.013	.013	.013	.013
MAPE	3.771	.	3.771	3.771	3.771	3.771	3.771	3.771	3.771	3.771	3.771
MaxAPE	12.798	.	12.798	12.798	12.798	12.798	12.798	12.798	12.798	12.798	12.798
MAE	.009	.	.009	.009	.009	.009	.009	.009	.009	.009	.009
MaxAE	.032	.	.032	.032	.032	.032	.032	.032	.032	.032	.032
Normalized BIC	‐8.085	.	‐8.085	‐8.085	‐8.085	‐8.085	‐8.085	‐8.085	‐8.085	‐8.085	‐8.085

Model Statistics
Model	Number of Predictors	Model Fit statistics	Ljung‐Box Q(18)			Number of Outliers
		Stationary R‐squared	Statistics	DF	Sig.
Outcomes‐Model_1	3	.764	13.931	17	.672	0

ARIMA Model Parameters
					Estimate	SE	t	Sig.
Outcomes‐Model_1	Outcomes	No Transformation	Constant		.264	.008	33.313	.000
			AR	Lag 1	.322	.187	1.725	.096
	phase	No Transformation	Numerator	Lag 0	‐.016	.011	‐1.404	.172
	pre4	No Transformation	Numerator	Lag 0	‐.002	.001	‐2.990	.006
	post4	No Transformation	Numerator	Lag 0	‐.004	.003	‐1.517	.141

PREDICT THRU END.

* Time Series Modeler.

TSMODEL

/MODELSUMMARY PRINT=[MODELFIT]

/MODELSTATISTICS DISPLAY=YES MODELFIT=[ SRSQUARE]

/MODELDETAILS PRINT=[ PARAMETERS]

/SERIESPLOT OBSERVED FORECAST

/OUTPUTFILTER DISPLAY=ALLMODELS

/SAVE PREDICTED(Predicted)

/AUXILIARY CILEVEL=95 MAXACFLAGS=24

/MISSING USERMISSING=EXCLUDE

/MODEL DEPENDENT=Outcomes INDEPENDENT=phase pre8 post8

PREFIX='Model'

/ARIMA AR=[1] DIFF=0 MA=[0]

TRANSFORM=NONE CONSTANT=YES

/AUTOOUTLIER DETECT=OFF.

Time Series Modeler

Model Description
			Model Type
Model ID	Outcomes	Model_1	ARIMA(1,0,0)

Model Summary

Model Fit
Fit Statistic	Mean	SE	Minimum	Maximum	Percentile
					5	10	25	50	75	90	95
Stationary R‐squared	.764	.	.764	.764	.764	.764	.764	.764	.764	.764	.764
R‐squared	.764	.	.764	.764	.764	.764	.764	.764	.764	.764	.764
RMSE	.013	.	.013	.013	.013	.013	.013	.013	.013	.013	.013
MAPE	3.771	.	3.771	3.771	3.771	3.771	3.771	3.771	3.771	3.771	3.771
MaxAPE	12.798	.	12.798	12.798	12.798	12.798	12.798	12.798	12.798	12.798	12.798
MAE	.009	.	.009	.009	.009	.009	.009	.009	.009	.009	.009
MaxAE	.032	.	.032	.032	.032	.032	.032	.032	.032	.032	.032
Normalized BIC	‐8.085	.	‐8.085	‐8.085	‐8.085	‐8.085	‐8.085	‐8.085	‐8.085	‐8.085	‐8.085

Model Statistics
Model	Number of Predictors	Model Fit statistics	Ljung‐Box Q(18)			Number of Outliers
		Stationary R‐squared	Statistics	DF	Sig.
Outcomes‐Model_1	3	.764	13.931	17	.672	0

ARIMA Model Parameters
					Estimate	SE	t	Sig.
Outcomes‐Model_1	Outcomes	No Transformation	Constant		.264	.008	33.313	.000
			AR	Lag 1	.322	.187	1.725	.096
	phase	No Transformation	Numerator	Lag 0	‐.025	.016	‐1.508	.143
	pre8	No Transformation	Numerator	Lag 0	‐.002	.001	‐2.990	.006
	post8	No Transformation	Numerator	Lag 0	‐.004	.003	‐1.517	.141

Dentists

GET DATA /TYPE=XLSX

/FILE='C:\Users\TonyaE\Documents\Stellenbosch\Consulting\Liesel Nicol\Copy of Yang 2013 raw data.xlsx'

/SHEET=name 'dentist'

/CELLRANGE=full

/READNAMES=on

/ASSUMEDSTRWIDTH=32767.

EXECUTE.

DATASET NAME DataSet1 WINDOW=FRONT.

SAVE OUTFILE='C:\Users\TonyaE\Documents\Stellenbosch\Consulting\Liesel Nicol\dentist.sav'

/COMPRESSED.

DATASET ACTIVATE DataSet1.

SAVE OUTFILE='C:\Users\TonyaE\Documents\Stellenbosch\Consulting\Liesel Nicol\dentist.sav'

/COMPRESSED.

PREDICT THRU END.

* Time Series Modeler.

TSMODEL

/MODELSUMMARY PRINT=[MODELFIT]

/MODELSTATISTICS DISPLAY=YES MODELFIT=[ SRSQUARE]

/MODELDETAILS PRINT=[ PARAMETERS]

/SERIESPLOT OBSERVED FORECAST

/OUTPUTFILTER DISPLAY=ALLMODELS

/SAVE PREDICTED(Predicted)

/AUXILIARY CILEVEL=95 MAXACFLAGS=24

/MISSING USERMISSING=EXCLUDE

/MODEL DEPENDENT=Outcomes INDEPENDENT=timeperiod phase interact

PREFIX='Model'

/ARIMA AR=[1] DIFF=0 MA=[0]

TRANSFORM=NONE CONSTANT=YES

/AUTOOUTLIER DETECT=OFF.

Time Series Modeler

[DataSet1] C:\Users\TonyaE\Documents\Stellenbosch\Consulting\Liesel Nicol\dentist.sav

Model Description
			Model Type
Model ID	Outcomes	Model_1	ARIMA(1,0,0)

Model Summary

Model Fit
Fit Statistic	Mean	SE	Minimum	Maximum	Percentile
					5	10	25	50	75	90	95
Stationary R‐squared	.916	.	.916	.916	.916	.916	.916	.916	.916	.916	.916
R‐squared	.916	.	.916	.916	.916	.916	.916	.916	.916	.916	.916
RMSE	.015	.	.015	.015	.015	.015	.015	.015	.015	.015	.015
MAPE	2.916	.	2.916	2.916	2.916	2.916	2.916	2.916	2.916	2.916	2.916
MaxAPE	9.490	.	9.490	9.490	9.490	9.490	9.490	9.490	9.490	9.490	9.490
MAE	.010	.	.010	.010	.010	.010	.010	.010	.010	.010	.010
MaxAE	.037	.	.037	.037	.037	.037	.037	.037	.037	.037	.037
Normalized BIC	‐7.869	.	‐7.869	‐7.869	‐7.869	‐7.869	‐7.869	‐7.869	‐7.869	‐7.869	‐7.869

Model Statistics
Model	Number of Predictors	Model Fit statistics	Ljung‐Box Q(18)			Number of Outliers
		Stationary R‐squared	Statistics	DF	Sig.
Outcomes‐Model_1	3	.916	18.943	17	.332	0

ARIMA Model Parameters
					Estimate	SE	t	Sig.
Outcomes‐Model_1	Outcomes	No Transformation	Constant		.351	.035	10.047	.000
			AR	Lag 1	.904	.100	9.048	.000
	time period	No Transformation	Numerator	Lag 0	.000	.002	‐.149	.883
	phase	No Transformation	Numerator	Lag 0	.199	.174	1.142	.264
	interact	No Transformation	Numerator	Lag 0	‐.009	.007	‐1.196	.242

PREDICT THRU END.

* Time Series Modeler.

TSMODEL

/MODELSUMMARY PRINT=[MODELFIT]

/MODELSTATISTICS DISPLAY=YES MODELFIT=[ SRSQUARE]

/MODELDETAILS PRINT=[ PARAMETERS]

/SERIESPLOT OBSERVED FORECAST

/OUTPUTFILTER DISPLAY=ALLMODELS

/SAVE PREDICTED(Predicted)

/AUXILIARY CILEVEL=95 MAXACFLAGS=24

/MISSING USERMISSING=EXCLUDE

/MODEL DEPENDENT=Outcomes INDEPENDENT=phase pre4 post4

PREFIX='Model'

/ARIMA AR=[1] DIFF=0 MA=[0]

TRANSFORM=NONE CONSTANT=YES

/AUTOOUTLIER DETECT=OFF.

Time Series Modeler

Model Description
			Model Type
Model ID	Outcomes	Model_1	ARIMA(1,0,0)

Model Summary

Model Fit
Fit Statistic	Mean	SE	Minimum	Maximum	Percentile
					5	10	25	50	75	90	95
Stationary R‐squared	.916	.	.916	.916	.916	.916	.916	.916	.916	.916	.916
R‐squared	.916	.	.916	.916	.916	.916	.916	.916	.916	.916	.916
RMSE	.015	.	.015	.015	.015	.015	.015	.015	.015	.015	.015
MAPE	2.914	.	2.914	2.914	2.914	2.914	2.914	2.914	2.914	2.914	2.914
MaxAPE	9.492	.	9.492	9.492	9.492	9.492	9.492	9.492	9.492	9.492	9.492
MAE	.010	.	.010	.010	.010	.010	.010	.010	.010	.010	.010
MaxAE	.037	.	.037	.037	.037	.037	.037	.037	.037	.037	.037
Normalized BIC	‐7.872	.	‐7.872	‐7.872	‐7.872	‐7.872	‐7.872	‐7.872	‐7.872	‐7.872	‐7.872

Model Statistics
Model	Number of Predictors	Model Fit statistics	Ljung‐Box Q(18)			Number of Outliers
		Stationary R‐squared	Statistics	DF	Sig.
Outcomes‐Model_1	3	.916	18.981	17	.330	0

ARIMA Model Parameters
					Estimate	SE	t	Sig.
Outcomes‐Model_1	Outcomes	No Transformation	Constant		.351	.035	10.117	.000
			AR	Lag 1	.904	.100	9.056	.000
	phase	No Transformation	Numerator	Lag 0	‐.039	.030	‐1.324	.196
	pre4	No Transformation	Numerator	Lag 0	.000	.002	‐.138	.891
	post4	No Transformation	Numerator	Lag 0	‐.009	.006	‐1.607	.120

PREDICT THRU END.

* Time Series Modeler.

TSMODEL

/MODELSUMMARY PRINT=[MODELFIT]

/MODELSTATISTICS DISPLAY=YES MODELFIT=[ SRSQUARE]

/MODELDETAILS PRINT=[ PARAMETERS]

/SERIESPLOT OBSERVED FORECAST

/OUTPUTFILTER DISPLAY=ALLMODELS

/SAVE PREDICTED(Predicted)

/AUXILIARY CILEVEL=95 MAXACFLAGS=24

/MISSING USERMISSING=EXCLUDE

/MODEL DEPENDENT=Outcomes INDEPENDENT=phase pre8 post8

PREFIX='Model'

/ARIMA AR=[1] DIFF=0 MA=[0]

TRANSFORM=NONE CONSTANT=YES

/AUTOOUTLIER DETECT=OFF.

Time Series Modeler

Model Description
			Model Type
Model ID	Outcomes	Model_1	ARIMA(1,0,0)

Model Summary

Model Fit
Fit Statistic	Mean	SE	Minimum	Maximum	Percentile
					5	10	25	50	75	90	95
Stationary R‐squared	.916	.	.916	.916	.916	.916	.916	.916	.916	.916	.916
R‐squared	.916	.	.916	.916	.916	.916	.916	.916	.916	.916	.916
RMSE	.015	.	.015	.015	.015	.015	.015	.015	.015	.015	.015
MAPE	2.916	.	2.916	2.916	2.916	2.916	2.916	2.916	2.916	2.916	2.916
MaxAPE	9.480	.	9.480	9.480	9.480	9.480	9.480	9.480	9.480	9.480	9.480
MAE	.010	.	.010	.010	.010	.010	.010	.010	.010	.010	.010
MaxAE	.037	.	.037	.037	.037	.037	.037	.037	.037	.037	.037
Normalized BIC	‐7.870	.	‐7.870	‐7.870	‐7.870	‐7.870	‐7.870	‐7.870	‐7.870	‐7.870	‐7.870

Model Statistics
Model	Number of Predictors	Model Fit statistics	Ljung‐Box Q(18)			Number of Outliers
		Stationary R‐squared	Statistics	DF	Sig.
Outcomes‐Model_1	3	.916	18.958	17	.331	0

ARIMA Model Parameters
					Estimate	SE	t	Sig.
Outcomes‐Model_1	Outcomes	No Transformation	Constant		.351	.035	10.131	.000
			AR	Lag 1	.903	.100	9.041	.000
	phase	No Transformation	Numerator	Lag 0	‐.073	.056	‐1.311	.201
	pre8	No Transformation	Numerator	Lag 0	.000	.002	‐.147	.884
	post8	No Transformation	Numerator	Lag 0	‐.009	.006	‐1.605	.120

Appendix 3. Evidence profile

Implementation of National Health Insurance on equity of distribution of physicians (Western medicine physicians, Chinese medicine physicians and dentists) in Taiwan.

Comparison: pre and post implementation of National Health Insurance

No of studies	Design	Risk of bias	Inconsistency	Indirectness[1]	Imprecision	Other[2]	Certainty (overall score)[3]	Importance
Outcome: Equity of distribution of Western medicine physicians post implementation of National Health Insurance
1	ITS study 1	Serious risk of bias 2	No serious inconsistency	Very serious indirectness 3	No serious imprecision	none	Very Low 4	Important
Outcome: Equity of distribution of Chinese medicine physicians post implementation of National Health Insurance
1	ITS study 1	Serious risk of bias 2	No serious inconsistency	Very serious indirectness 3	No serious imprecision	none	Very Low 4	Important
Outcome: Equity of distribution of dentists post implementation of National Health Insurance
1	ITS study 1	Serious risk of bias 2	No serious inconsistency	Very serious indirectness 3	No serious imprecision	none	Very Low 4	Important

Footnotes:

1. Interrupted time series study

2. Apart from the risk of bias inherent in interrupted time series studies (high risk of selection bias and whether or not the effect is due to the intervention), it was a well‐conducted study.

3. Data from one study conducted in one country. Results may not be generalizable or transferable to other settings.

4. Very low: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different (a large enough difference that it might affect a decision) is very high.

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Yang 2013.

Methods	Interrupted trend analysis with time series observations over 32 years (24 years before intervention and 8 years post intervention). The equality of geographic distributions for each of the 3 types of health providers is measured by Gini coefficients derived from Lorenz curve based on 21 prefecture/city assigned by the Ministry of Interior. A Lorenz curve of cumulative wealth or income vs. population, for instance, plots cumulative population in ascending order on the X‐axis against the cumulative wealth or income in those areas on the Y‐axis. The Gini coefficient is derived from the Lorenz curve that provides an alternative numerical measure for the extent of equal distribution in wealth or income. Its value lies between 0 (no inequality) and 1 (complete inequality). The dependent variable Gini coefficient in this study was a summarised result of the accumulation the number of physicians against the accumulation of population of all the 21 geographic areas (in the order of physician concentration in a prefecture/city) compared to perfect distribution of physicians (i.e. for a certain percentage of population it has the same percentage of physicians. For example, 40% of population has 40% of physicians, 70% of population has 70% of physicians, accumulated from the lowest to the highest ratio of physicians to population in an area).
Participants	Western physicians, Chinese physicians and dentists in Taiwan.
Interventions	Introduction of NHI in 1995 in Taiwan.
Outcomes	The equality of geographic distributions for each of the 3 types of health providers is measured by Gini coefficients derived from Lorenz curve based on 21 prefecture/city assigned by the Ministry of Interior.
Notes	Ethics: not reported. Funding source: not reported. Conflict of interest: authors declared no competing interests.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Not an RCT.
Allocation concealment (selection bias)	High risk	Not an RCT.
Intervention independent	High risk	However, in an attempt to account for other variables (socioeconomic, political) that may have influenced the geographic distribution of the health providers, the study authors included a year‐specific time trend variable (Tt) in their analysis model. The authors limited the post NHI observations to 2002 in order to exclude the possible effect of other health insurance measures adopted by the Taiwanese Government after the introduction of the NHI (e.g. pay‐for performance in 2001, global budgeting in 2002 and regressive payment to service volume for assuring quality of care in 2004) on health provider distribution.
Analysed appropriately	Low risk	Data were analysed using spline regression, which is absolutely correct when you have independent outcomes but assumes that the regression is not linear across all the scales. In this instance, one cannot assume that the observations are independent. The Gini coefficient one year will be dependent on the Gini coefficient from the previous year (the population does not fluctuate randomly and neither does the number of doctors). By using spline regression, the authors do not take the auto correlated structure of the data into account. Based on this, a high risk of bias should be allocated to this domain. However, following re‐analysis (based on figure 1) is performed that take autocorrelation into account the risk of bias assessment can remain as low. A spline‐regression analysis was used to detect a structural change in the Gini coefficients due to external programme intervention over time, such as policy changes. Spline regression uses the polynomial term for capturing non‐linear changes along with the introducing of the policy. Since NHI was implemented to cover the entire healthcare system in Taiwan, all the relevant parties within the healthcare system needed a substantial period of time to adjust into the NHI scheme. In the spline regression analysis in this study, they used 0 for the variable NHIt before 1995, and 1‐8 after 1995 to capture potential non‐linear structural change after the implementation of NHI. A year‐specific time trend variable denoted by the term Tt was included in the model to account for other confounding socioeconomic factors in the wider society over the 3‐decade time period.
Shape of effect pre‐specified	Unclear risk	This refers to a priori hypotheses about the direction of the intercept (immediate level change) and slope (trend) effects at the time of the intervention/policy and the nature of slope parameter (e.g. linear, polynomial, etc.). That is, the investigators should state explicitly that the intervention was intended to produce (or not) an immediate change in the outcome of interest (positive or negative) or that the effect would increase/decrease over time, or both. The authors did not specifically state how they felt the implementation of the NHI would affect the outcome of interest. However, the authors did state that any changes occurring as a result of the intervention would be gradual and this was accounted for in the analyses and the variables included in the analyses. The criterion regarding the "point of analysis" is critical. Usually the inflection point ("interruption") is specified when the intervention begins; however, investigators often allow for a "ramp up" period where the observations around the intervention are censored in order to measure the effect of the intervention when it reaches "full strength". If a ramp up period is used in the analysis then this should be carefully explained and justified.
Was the intervention unlikely to affect data collection	Low risk	Sources and methods of data collection were likely to be the same before and after the intervention.
Blinding	Low risk	Even though the health providers would have been aware of the implementation of the NHI the outcome measure was objective and was obtained from published government records.
Incomplete outcome data addressed	Low risk	Although the authors did not report on the extent of missing data, the outcome was obtained from published government records for the entire period of analysis so any flaws inherent in this type of government data were likely to have been present throughout the time period.
Free of selective reporting	Low risk	While we do not have access to the study protocol, all relevant outcomes in the methods section were reported in the results section.
Free of other bias	Unclear risk	While the authors declared that they had no competing interests, the funding source of the study was not reported. Seasonality: not applicable.

NHI: National Health Insurance; RCT: randomised controlled trial.

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Adzei 2012	Incorrect study design ‐ observational study.
Ageyi‐Baffour 2013	Incorrect study design ‐ observational study (discreet choice experiment).
Antwi 2013	Incorrect study design ‐ observational study (discreet choice experiment).
Awofeso 2010	Incorrect study design ‐ personal perspective.
Barnighausen 2009	Review article.
Baumann 2006	Incorrect study design ‐ descriptive study.
Bazemore 2011	Incorrect study design ‐ descriptive study.
Bigbee 2014	Incorrect study design ‐ retrospective study.
Buchan 2013	Incorrect study design ‐ report of the effect of implementing World Health Organization recommendations.
Burch 2011	Incorrect study design ‐ observational cross‐sectional study.
Buykx 2010	Review article.
Clark 2013	Incorrect study design ‐ longitudinal follow up of 3 cohorts (2005‐2008; 2006‐2009; 2007‐2010) at 1 medical school.
Codjia 2010	Incorrect study design ‐ longitudinal cohort study (programme evaluation).
Crump 2013	10 years of data from Kentucky rural school; last 2 years of medical school at rural campus. Compared outcomes from rural campus graduates and Louisville campus graduates.
de Vries 2003	Incorrect study design ‐ descriptive study.
de Vries 2010	Incorrect study design ‐ retrospective descriptive study.
Deutschlander 2014	Scoping review on interprofessional practice; clerkship; clinical rotations; etc.
Deveney 2013	Incorrect study design ‐ retrospective descriptive study.
Devine 2013	Incorrect study design ‐ descriptive study.
Ditlopo 2011	Incorrect study design ‐ descriptive study.
Dolea 2010	Review.
Dunbabin 2006	Incorrect study design ‐ descriptive study.
Efendi 2012	Incorrect study design ‐ descriptive study.
Eley 2006	Incorrect study design ‐ descriptive cohort study.
Eley 2012	Report.
Erney 1991	Inappropriate outcome: "choice of family practice residency" is not a direct indication of practice location.
Eyal 2012	Review ‐ overview of conditional scholarships and their effectiveness.
Frehywot 2010	Review.
George 2012	Incorrect study design ‐ descriptive study.
Gorsche 2012	Incorrect study design ‐ cohort controlled study.
Gow 2013	Incorrect study design ‐ programme evaluation.
Haq 2013	Incorrect study design ‐ programme evaluation.
Harris 1982	Inappropriate outcome: "choice of family practice residency" was not a direct indication of practice location.
Henderson 2008	Review.
Henry 2009	Incorrect study design ‐ descriptive study.
Herold 1993	Inappropriate outcome: "choice of family practice residency" was not a direct indication of practice location.
Huicho 2012	Incorrect study design ‐ descriptive study.
Inoue 2007	Incorrect study design ‐ retrospective descriptive study.
Inoue 2009	Incorrect study design ‐ descriptive study.
Jackson 2003	Incorrect study design ‐ cohort controlled study.
Jamieson 2013	Incorrect study design ‐ cohort controlled study.
Kane 2013	Incorrect study design ‐ programme evaluation.
Lehmann 2008	Review.
Longombe 2009	Incorrect study design ‐ descriptive study.
Lupton 2012	Incorrect study design ‐ retrospective case controlled study.
MacDowell 2013	Retrospective cohort comparing Rockford Rural Medical Education and non‐Rockford Rural Medical Education and practice location.
Matsumoto 2008a	Incorrect study design ‐ retrospective cohort comparing Jichi Medical University and non‐Jichi Medical University and practice location.
Matsumoto 2008b	Incorrect study design ‐ retrospective cohort comparing Jichi Medical University and non‐Jichi Medical University and practice location.
Matsumoto 2010a	Review.
Matsumoto 2010b	Review/programme evaluation.
Matsumoto 2010c	Incorrect study design ‐ descriptive study.
Matsumoto 2010d	Incorrect study design ‐ descriptive study.
Matsumoto 2012	Incorrect study design ‐ descriptive study.
Mbemba 2013	Umbrella review.
Mennin 1996	Students self selected the intervention, thus firm conclusions regarding the influence of the intervention, per se, could not be made.
Miranda 2012	Incorrect study design ‐ discreet choice experiment.
Morell 2014	Incorrect study design ‐ programme evaluation.
Munga 2014	Incorrect study design ‐ contingent valuation method/discreet choice type intervention.
Orzanco 2011	Incorrect study design ‐ retrospective descriptive study.
Ozegowski 2013	Incorrect study design ‐ qualitative comparative analysis.
Pena 2010	Incorrect study design ‐ programme evaluation.
Playford 2014	Incorrect study design ‐ cohort controlled study.
Rabinowitz 1993	Incorrect study design.
Rabinowitz 2001	Review.
Rabinowitz 2005	Incorrect study design.
Rabinowitz 2011	Incorrect study design.
Rabinowitz 2012a	Incorrect study design ‐ cohort controlled study, no intervention.
Rabinowitz 2012b	Incorrect study design ‐ cohort controlled study, no intervention.
Rabinowitz 2012c	Incorrect study design ‐ cohort controlled study, no intervention.
Rabinowitz 2013	Incorrect study design ‐ cohort controlled study, no intervention.
Rockers 2012	Incorrect study design ‐ discreet choice experiment.
Rockers 2013a	Incorrect study design ‐ discreet choice experiment.
Rockers 2013b	Review ‐ Cochrane.
Rosenblatt 1979	Inappropriate outcome: "choice of family practice residency" is not a direct indication of practice location.
Royston 2012	Incorrect study design ‐ cross‐sectional survey.
Russell 2012a	Review.
Russell 2012b	Incorrect study design ‐ descriptive study.
Russell 2013a	Incorrect study design ‐ descriptive study.
Russell 2013b	Incorrect study design ‐ descriptive study.
Rutebemberwa 2014	Review.
Salafsky 2005	Incorrect study design ‐ programme evaluation.
Sasaki 2013	Incorrect study design ‐ descriptive study.
Scott 2013	Incorrect study design ‐ discreet choice experiment.
Somers 2007	Incorrect study design ‐ descriptive study.
Somers 2012	Incorrect outcome ‐ rural career choice likelihood.
Sousa 2012	Incorrect study design ‐ descriptive study.
Straume 2010	Incorrect comparison ‐ Longitudinal cohort with comparison county.
Sundararaman 2011	Incorrect study design ‐ descriptive study.
Suter 2012	Review.
Tani 2004	Incorrect study design ‐ descriptive study.
Toussaint 2010	Incorrect study design ‐ programme evaluation.
Walker 2012	Incorrect study design ‐ descriptive study.
Wang 2013	Incorrect study design ‐ descriptive study.
Wang 2014	Incorrect study design ‐ descriptive study.
Wilkinson 2004	Inappropriate outcome: medical student's choice of internship site. Choice of rural internship does not necessarily predict rural practice after graduation.
Williamson 2012	Cohort with controls/comparison. Questionnaire comparing the effect of a 7‐week rural rotation on practice location 8 years post graduation. Compared 3 New Zealand medical schools.
Zurn 2010	Incorrect study design ‐ descriptive study: effect of implementation of Project Cobra flexible contracting system in Senegal.

Differences between protocol and review

The review followed the protocol as it was published, with no differences in methodology. The authorship of the review protocol is not the same as the authorship of the published or the currently updated review. Helmuth Reuter and Philemon Marindi were both authors of the review protocol and the first published version of the review. However, neither of them were involved in the update of the current version of the review. Both of them requested that they not be included in the authorship of the updated review as they felt that they did not have the time required to remain involved that would warrant authorship. Jimmy Volmink edited the first draft of the updated review but was not able to commit any additional time to the review as so asked to be removed as an author of the review.

Contributions of authors

Phillemon Marindi, Sikhumbuzo Mabunda, Helmuth Reuter and Jimmy Volmink assisted Liesl Grobler in preparing the original review protocol and in searching for background information on the topic. Liesl Grobler, together with Phillemon Marindi and Sikhumbuzo Mabunda screened search results for relevant articles and Jimmy Volmink was consulted in the case of disagreements arising from this process. Input from Helmuth Reuter, Ben Marais and Jimmy Volmink was integral in completing the initial review. Liesl Grobler and Sikhumbuzo Mabunda screened the updated search results for eligible studies. Liesl Grobler prepared a draft of the updated review and Sikhumbuzo Mabunda, Jimmy Volmink and Ben Marais provided input into the final version of the updated review.

Sources of support

Internal sources

• University of Cape Town, South Africa.

• University of Stellenbosch, South Africa.

• Medical Research Council, South Africa.

External sources

• Cochrane Public Health and Health Professionals Field, South Africa.

• Collaboration for Health Equity through Education and Research (CHEER), South Africa.

Declarations of interest

None known.

Edited (no change to conclusions)