Nurses as substitutes for doctors in primary care

Abstract

Background

Current and expected problems such as ageing, increased prevalence of chronic conditions and multi‐morbidity, increased emphasis on healthy lifestyle and prevention, and substitution for care from hospitals by care provided in the community encourage countries worldwide to develop new models of primary care delivery. Owing to the fact that many tasks do not necessarily require the knowledge and skills of a doctor, interest in using nurses to expand the capacity of the primary care workforce is increasing. Substitution of nurses for doctors is one strategy used to improve access, efficiency, and quality of care. This is the first update of the Cochrane review published in 2005.

Objectives

Our aim was to investigate the impact of nurses working as substitutes for primary care doctors on:

• patient outcomes;

• processes of care; and

• utilisation, including volume and cost.

Search methods

We searched the Cochrane Central Register of Controlled Trials (CENTRAL), part of the Cochrane Library (www.cochranelibrary.com), as well as MEDLINE, Ovid, and the Cumulative Index to Nursing and Allied Health Literature (CINAHL) and EbscoHost (searched 20.01.2015). We searched for grey literature in the Grey Literature Report and OpenGrey (21.02.2017), and we searched the International Clinical Trials Registry Platform (ICTRP) and ClinicalTrials.gov trial registries (21.02.2017). We did a cited reference search for relevant studies (searched 27.01 2015) and checked reference lists of all included studies. We reran slightly revised strategies, limited to publication years between 2015 and 2017, for CENTRAL, MEDLINE, and CINAHL, in March 2017, and we have added one trial to ‘Studies awaiting classification’.

Selection criteria

Randomised trials evaluating the outcomes of nurses working as substitutes for doctors. The review is limited to primary healthcare services that provide first contact and ongoing care for patients with all types of health problems, excluding mental health problems. Studies which evaluated nurses supplementing the work of primary care doctors were excluded.

Data collection and analysis

Two review authors independently carried out data extraction and assessment of risk of bias of included studies. When feasible, we combined study results and determined an overall estimate of the effect. We evaluated other outcomes by completing a structured synthesis.

Main results

For this review, we identified 18 randomised trials evaluating the impact of nurses working as substitutes for doctors. One study was conducted in a middle‐income country, and all other studies in high‐income countries. The nursing level was often unclear or varied between and even within studies. The studies looked at nurses involved in first contact care (including urgent care), ongoing care for physical complaints, and follow‐up of patients with a particular chronic conditions such as diabetes. In many of the studies, nurses could get additional support or advice from a doctor. Nurse‐doctor substitution for preventive services and health education in primary care has been less well studied.

Study findings suggest that care delivered by nurses, compared to care delivered by doctors, probably generates similar or better health outcomes for a broad range of patient conditions (low‐ or moderate‐certainty evidence):

• Nurse‐led primary care may lead to slightly fewer deaths among certain groups of patients, compared to doctor‐led care. However, the results vary and it is possible that nurse‐led primary care makes little or no difference to the number of deaths (low‐certainty evidence).

• Blood pressure outcomes are probably slightly improved in nurse‐led primary care. Other clinical or health status outcomes are probably similar (moderate‐certainty evidence).

• Patient satisfaction is probably slightly higher in nurse‐led primary care (moderate‐certainty evidence). Quality of life may be slightly higher (low‐certainty evidence).

We are uncertain of the effects of nurse‐led care on process of care because the certainty of this evidence was assessed as very low.

The effect of nurse‐led care on utilisation of care is mixed and depends on the type of outcome. Consultations are probably longer in nurse‐led primary care (moderate‐certainty evidence), and numbers of attended return visits are slightly higher for nurses than for doctors (high‐certainty evidence). We found little or no difference between nurses and doctors in the number of prescriptions and attendance at accident and emergency units (high‐certainty evidence). There may be little or no difference in the number of tests and investigations, hospital referrals and hospital admissions between nurses and doctors (low‐certainty evidence).

We are uncertain of the effects of nurse‐led care on the costs of care because the certainty of this evidence was assessed as very low.

Authors' conclusions

This review shows that for some ongoing and urgent physical complaints and for chronic conditions, trained nurses, such as nurse practitioners, practice nurses, and registered nurses, probably provide equal or possibly even better quality of care compared to primary care doctors, and probably achieve equal or better health outcomes for patients. Nurses probably achieve higher levels of patient satisfaction, compared to primary care doctors. Furthermore, consultation length is probably longer when nurses deliver care and the frequency of attended return visits is probably slightly higher for nurses, compared to doctors. Other utilisation outcomes are probably the same. The effects of nurse‐led care on process of care and the costs of care are uncertain, and we also cannot ascertain what level of nursing education leads to the best outcomes when nurses are substituted for doctors.

Plain language summary

Nurses as substitutes for doctors in primary care

What is the aim of this review?

The aim of this Cochrane Review was to find out what happens when primary healthcare services are delivered by nurses instead of doctors. We collected and analysed all relevant studies to answer this question and found 18 studies for inclusion in the review.

What are the key messages of this review?

Delivery of primary healthcare services by nurses instead of doctors probably leads to similar or better patient health and higher patient satisfaction. Nurses probably also have longer consultations with patients. Using nurses instead of doctors makes little or no difference in the numbers of prescriptions and tests ordered. However, the impacts on the amount of information offered to patients, on the extent to which guidelines are followed and on healthcare costs are uncertain.

What was studied in this review?

In most countries, the population is growing older and more people have chronic disease. This means that the services that primary healthcare workers need to deliver are changing. At the same time, many countries lack doctors and other healthcare workers, or people struggle to pay for healthcare services. By using nurses instead of doctors, countries hope to deliver care of the same quality for less money.

In this review, we searched for studies that compared nurses to doctors for delivery of primary care services. We looked at whether this made any difference in patients’ health, satisfaction, and use of services. We also looked at whether this made any difference in how services were delivered and in how much they cost.

What are the main results of this review?

We included in this review 18 studies, mainly from high‐income countries. In some studies, nurses were responsible for all patients who came to the clinic or for all patients who needed urgent consultation. In some studies, nurses were responsible for patients with particular chronic diseases, or were responsible for providing healthcare education or preventive services to certain groups of patients. Included studies compared these nurses to doctors carrying out the same tasks.

Our review shows that nurse‐led primary care may lead to slightly fewer deaths among certain groups of patients, compared to doctor‐led care. However, the results vary and it is possible that nurse‐led primary care makes little or no difference to the number of deaths. In addition, patients probably have similar or better results in areas of health such as heart disease, diabetes, rheumatism, and high blood pressure. Patients also are probably slightly more satisfied with their care and may have a slightly better quality of life when treated by nurses.

This review also shows that, compared to doctors, nurses probably have longer consultations, and their patients are slightly more likely to keep follow‐up appointments. Studies found little or no difference in the number of prescriptions and there may be little or no difference in the numbers of tests and investigations ordered, or in patients’ use of other services. The effects of nurse‐led primary care on the amount of advice and information given to patients, and on whether guidelines are followed, are uncertain as the certainty of these findings is very low.

Our review suggests that the impacts on the costs of care of using nurses instead of doctors to deliver primary care are uncertain. We assessed the certainty of this finding as very low.

How up‐to‐date is this review?

We searched for studies that had been published up to March 2017.

Summary of findings

for the main comparison.

Nurse‐led primary care compared with doctor‐led primary care for patient outcomes, process of care and utilisation
Patient or population: all presenting patients in primary care Settings: UK (n = 6), Netherlands (n = 3), USA (n = 3), Canada (n = 3), Sweden (n = 1), Spain (n = 1), South Africa (n = 1) Intervention: substitution of doctors with nurses for primary care Comparison: routine doctor‐led primary care
Outcomes	Impact				Number of participants (studies)	Certainty of the evidence (GRADE)
	Illustrative comparative risks* (95% CI)		Effect estimate (95% CI)	Results in words
	Assumed risk	Corresponding risk
	Doctor‐led primary care	Nurse‐led primary care
Mortality follow‐up: 0.5 to 48 months Mean = 21 (SD 19) months	6.29 per 1000	4.84 per 1000 (4 to 6)	RR 0.77 (0.57 to 1.03)	Nurse‐led primary care may lead to slightly fewer deaths among certain groups of patients, compared to doctor‐led care. However, the results vary and it is possible that nurse‐led primary care makes little or no difference to the number of deaths.	36,529 (8)1	⊕⊕⊝⊝ a Low
Patient health status follow‐up: 0.2 to 47 months Mean = 14 (SD 12) months	Compared to doctor‐led care, nurse‐led primary care probably slightly improves blood pressure control; probably leads to similar outcomes for diabetes indicators and measures of disease activity and pain in people with rheumatological disorders; may lead to similar outcomes for physical functioning; and leads to similar outcomes for cholesterol				Clinical outcomes (3) Self‐reported measurements (13)2	⊕⊕⊕⊝ b Moderate
Satisfaction and preferences follow‐up: 0.5 to 25 months Mean = 12 (SD 10) months	Patient satisfaction is probably slightly higher in nurse‐led primary care compared to doctor‐led primary care.				16,993 (7)3	⊕⊕⊕⊝ c Moderate
Quality of life follow‐up: 6 to 25 months Mean = 15 (SD 9) months	Quality of life may be slightly higher in nurse‐led primary care compared to doctor‐led primary care.				16,002 (6)4	⊕⊕⊝⊝d Low
Process of care follow‐up: 0.5 to 48 months Mean = 17 (SD 15) months	We are uncertain of the effects of nurse‐led care on process of care because the certainty of this evidence was assessed as very low.				(10)5	⊕⊝⊝⊝e Very low
Utilisation (consultations, prescriptions, tests, investigations, and services) follow‐up: 0.2 to 48 months Mean = 14 (SD 13) months	Consultations: Compared to doctor‐led primary care, consultation length is probably longer in nurse‐led primary care; there may be little or no difference in scheduled return visits; and the number of return visits attended is slightly higher for nurses. Prescriptions, tests and investigations: There is little or no difference between nurses and doctors in the number of prescriptions and may be little or no difference in the number of tests and investigations ordered. Use of other services: There may be little or no difference between nurses and doctors in the likelihood of hospital referrals and hospital admissions; little or no difference in attendance at accident and emergency units.				(16)6	⊕⊕⊕⊝ f Moderate
Costs follow‐up: 0.2 to 48 months Mean = 14 (SD 14) months	We are uncertain of the effects of nurse‐led care on the cost of care because the certainty of this evidence was assessed as very low.				(9)7	⊕⊝⊝⊝ g Very low
*The basis for the assumed risk is the mean control group risk across studies for pooled results. The corresponding risk is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: confidence interval; RR: risk ratio; SD: standard deviation. aDowngraded by 1 for imprecision owing to a wide confidence interval that includes no effect and downgraded by 1 for clinical heterogeneity as the trials contributing to this estimate are quite varied (some focus on people with specific health issues and others on more generalist primary care attenders). bDowngraded by 1. Outcomes were downgraded by 1 for inconsistency, imprecision, indirectness or high risk of bias. The certainty of the evidence is moderate for all outcomes listed, apart for physical functioning for which the certainty of evidence was low and cholesterol for which the certainty of evidence was high. cDowngraded by 1 for inconsistency. dDowngraded by 1 for imprecision, due to a wide confidence interval that touches on the null, and 1 for inconsistency eNon‐comparable results and therefore downgraded to very low. fDowngraded by 1. Outcomes were downgraded by 1 for inconsistency, imprecision or high risk of bias. gNon‐comparable results (the types of costs assessed varied widely and a range of different approaches were used to value resources and calculate costs) and therefore downgraded to very low. 1Campbell 2014; Hemani 1999; Lattimer 1998; Ndosi 2013; Sanne 2010; Shum 2000; Spitzer 1973; Voogdt‐Pruis 2010. 2Campbell 2014; Chambers 1978; Chan 2009; Dierick‐van Daele 2009; Houweling 2011; Iglesias 2013; Larsson 2014; Lattimer 1998; Lewis 1967; Moher 2001; Mundinger 2000; Sanne 2010; Shum 2000; Spitzer 1973; Venning 2000; Voogdt‐Pruis 2010. 3Campbell 2014; Dierick‐van Daele 2009; Iglesias 2013; Larsson 2014; Mundinger 2000; Shum 2000; Venning 2000. 4Campbell 2014; Chan 2009; Dierick‐van Daele 2009; Houweling 2011; Mundinger 2000; Ndosi 2013. 5Campbell 2014; Dierick‐van Daele 2009; Houweling 2011; Moher 2001; Mundinger 2000; Ndosi 2013; Shum 2000; Spitzer 1973; Venning 2000; Voogdt‐Pruis 2010. 6Campbell 2014; Chan 2009; Dierick‐van Daele 2009; Hemani 1999; Houweling 2011; Iglesias 2013; Larsson 2014; Lattimer 1998; Lewis 1967; Moher 2001; Mundinger 2000; Ndosi 2013; Shum 2000; Spitzer 1973; Venning 2000; Voogdt‐Pruis 2010. 7Campbell 2014; Chambers 1978; Chan 2009; Dierick‐van Daele 2009; Lattimer 1998; Lewis 1967; Ndosi 2013; Spitzer 1973; Venning 2000. *there may be additional data in the Campbell 2014 articles that have not been extracted
GRADE Working Group grades of evidence. High certainty: This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different† is low. Moderate certainty: This research provides a good indication of the likely effect. The likelihood that the effect will be substantially different† is moderate. Low certainty: This research provides some indication of the likely effect. However, the likelihood that it will be substantially different† is high. Very low certainty: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different† is very high †Substantially different = a large enough difference that it might affect a decision.

Background

Description of the condition

A range of existing and anticipated issues, including ageing populations, increased prevalence of chronic conditions and multi‐morbidity, increased emphasis on healthy lifestyle and prevention, and movement of healthcare services from hospitals to communities, have encouraged countries worldwide to develop new models of primary care delivery (Freund 2015; Roland 2014). As many tasks do not necessarily require the knowledge and skills of a doctor, using nurses to expand the capacity of the primary care workforce is a topic of increasing interest (Kooienga 2015; Maier 2016b). Substitution of nurses for doctors is one strategy for improving access to and efficiency and quality of care (NHS 2016; Perloff 2016), and advanced nursing practice roles are common in high‐, middle‐ and low‐income countries (Kooienga 2015; Maier 2016b). However, variation in primary care practice compositions is strong, and the same cadre might have different roles and authority in relation to practice depending on legislation, the healthcare system, and local practices (Freund 2015; Groenewegen 2015; Laurant 2009). Furthermore, reforms currently being implemented in many countries regarding nurses’ regulatory barriers or expansion of nurses’ scope of practice (e.g. in relation to prescribing medicines) suggest a shift in the boundaries between medicine and nursing (Maier 2016b).

Description of the intervention

Nurses in primary care may undertake many tasks traditionally performed by doctors. Tasks can be supplementary to those performed by doctors or can be substituted for doctors' tasks. The current review focusses on tasks in which nurses substitute for doctors, meaning that they provide the same services as doctors (Laurant 2009; Rashidian 2013), and is limited to care delivery for patients presenting with a physical complaint. These tasks may include diagnostics, treatment, referral to other services, health promotion, management of chronic diseases, or management of acute problems needing same‐day consultations. Contact with patients may take place in a primary health facility or in the home of the patient. Because people's understanding of what constitutes a nurse, as well as the educational levels of nurses, differs across countries (Kooienga 2015; Maier 2016b), we have included in this review all registered nurses who provide care as substitutes for doctors. When available, we have provided information on the educational levels of nurses in the included studies, based on the European Qualification Framework (EQF 2016). Moreover, the review aimed to include studies from high‐, middle‐, and low‐income countries, and we have described the impact of this approach on heterogeneity.

How the intervention might work

The expectation is that nurses substituted for doctors can do the following (e.g. Freund 2015; Kooienga 2015; Newhouse 2011; Rashidian 2013).

• Enhance the quality of services provided in primary care.

• Increase access to primary care services, as capacity increases.

• Reduce doctors' workload and thus free up time for doctors to take up more complex tasks.

• Reduce costs of care through lower salary costs for nurses and limited educational training, which is provided more quickly and is less expensive.

This last point was not, however, confirmed by a previous systematic review on this intervention (Martínez‐González 2015c). Gains in service efficiency may be achieved if doctors no longer provide the services they have delegated to nurses. This enables doctors to focus on complexity in their caseload and on utilising their advanced training and experience (Contandriopoulos 2015; Richardson 1999).

Why it is important to do this review

Advanced nursing practice roles have been developed worldwide, including in low‐, middle‐, and high‐income countries (Freund 2015; Kooienga 2015; Maier 2016b). The first advanced nursing roles were developed in the USA and Canada in the late 1960s/70s, in the UK in the 1980s, and in other high‐income countries in the 1990s and onwards (Laurant 2009). From the outset, nurses have been utilised to deliver primary care, traditionally in underserved areas and to vulnerable populations. Nowadays, their role has been extended to include other types of services in primary care (Poghosyan 2012), and this change has been implemented in a range of countries around the globe (Freund 2015; Kooienga 2015; Maier 2016b).

Nurses in advanced roles represent a substantial source of human capital to increase quality of care, access to (primary) care, and, as it is sometimes argued, efficiency of care, although recent reviews have not confirmed improvements in efficiency (Martínez‐González 2014a; Martínez‐González 2014b; Martínez‐González 2015a; Martínez‐González 2015b; Martínez‐González 2015c). It is believed that inclusion of nurses in advanced roles can ensure that the demand for healthcare services to address patient needs is properly met. Both practitioners and policy makers believe that to meet the challenges faced by primary care, a more robust healthcare workforce, including both doctors and nurses in advanced nursing roles, is needed (NHS 2016).

Although interest in expanding nursing roles and employing nurses as substitutes for doctors is increasing globally, underlying reasons for these initiatives differ depending on local context and circumstances (Savrin 2009). For example, the Health Resources and Services Administration in the USA, anticipating a shortage of doctors, has increased the amount of money available not only to train doctors but also to prepare nurse practitioners and physician assistants to support the primary care workforce (Petterson 2012). In addition, organisations such as the World Health Organization (WHO) have made several recommendations regarding ways to expand the role of nurses (WHO 2012). To enable policy makers to make informed decisions about healthcare delivery models, we need rigorous evidence on the quality of care, as well as on access and costs, associated with care provided by nurses compared with care provided by doctors.

Since this review was first published in 2005 (Laurant 2005), a large number of comparative studies have produced a stronger evidence base with regard to the effectiveness of nurse‐doctor substitution. Many new studies on nurses in primary care show increasing international interest in task shifting and in shifting of boundaries between medicine and nursing. Moreover, regulatory and educational reforms internationally support the trend towards advanced nursing roles in healthcare delivery and task shifting. The growth rate of the nursing workforce is now three times that of the workforce for doctors (nine times that for nurse practitioners), which provides an important opportunity to meet increasing demand within primary care (Maier 2016b).

This updated review adds value to recently published systematic reviews on this topic by excluding studies that do not focus solely on substitution, resulting in more accurate findings regarding the effectiveness of nurse‐doctor substitution specifically; and by using rigorous Cochrane methods. Additional insights provided by this update are important because results reported by some other reviews have been inconclusive (e.g. on costs), and because the ways in which primary healthcare services are organised have changed since our original review was published.

Objectives

Our aim was to investigate the impact of nurses working as substitutes for primary care doctors on:

• patient outcomes;

• processes of care; and

• utilisation, including volume and costs.

Methods

Criteria for considering studies for this review

Types of studies

Randomised trials (i.e. trials with random allocation of participants to intervention and control groups). We included controlled before‐after studies and non‐randomised trials in our previous Cochrane review (Laurant 2005). The number of available randomised trials has increased since that time; therefore, we decided to exclude controlled before‐after studies (n = 3) and non‐randomised trials (n = 3) from this update. Randomised trials provide more robust evidence on effectiveness, and including other study designs is unlikely to be worthwhile in light of the many randomised trials now available.

Types of participants

• Doctors: any kinds of doctors working in a primary care setting, including general practitioners, family doctors, paediatricians, general internists, and geriatricians. In this review, we use 'doctor' as the generic term for this cadre.

• Nurses: any qualified registered nurses working as substitutes for doctors in primary care. The definition of a qualified nurse is “a graduate who has been legally authorised (registered) to practice after examination by a state board of nurse examiners or similar regulatory authority” (WHO 2012). Included are nurse practitioners, clinical nurse specialists, advanced practice nurses, practice nurses, health visitors, etc. As the job title, education, and experience of nurses vary considerably among and within countries, we did not select nurses by virtue of their job title. We excluded only mental health nurses as this kind of substitution is addressed by the EPOC review on non‐specialist health workers for mental health (Ginneken 2013). We also excluded trainee nurses as they do not work to their full potential as a consequence of their traineeship.

• Patients: any persons presenting in primary care with a physical complaint.

This review is limited to primary healthcare services that provide first contact and ongoing care for patients with all types of physical health problems. It includes family practice, general practice, out‐patient care, and ambulatory primary care settings but excludes accident and emergency departments in hospitals. Patients presenting to accident and emergency departments in hospitals are not considered to be comparable to patients presenting for primary care services. These hospital departments generally deal with genuine life‐threatening emergencies and therefore are not considered an alternative to an appointment with a doctor in primary care.

Types of interventions

This review focusses on nurses working as substitutes for primary care doctors. Substitution refers to the situation wherein task(s) formerly performed by one type of professional (i.e. a doctor) are transferred to a different type of professional (i.e. a nurse), usually with the intention of reducing cost or addressing workforce shortages (Freund 2015; Laurant 2009). Substitution studies typically examine cases in which a nurse is responsible for providing the same health care as a doctor and compare the performance of these two practitioners. For example, study authors may compare a nurse‐led clinic for a particular disease or condition versus a doctor‐led clinic for that same disease or condition.

We excluded studies which evaluated nurses supplementing the work of primary care doctors. Supplementation refers to the situation wherein a nurse supplements or extends the care provided by a doctor by providing a new primary care service. Generally, the aim is to improve the quality of care rather than reduce cost or address workforce shortages. Supplementation studies typically compare usual care provided by a doctor versus an innovative service provided by a nurse working alongside a doctor. For example, researchers may compare a family practice with a nurse‐led diabetes clinic versus a family practice without such a clinic. This type of study risks confounding two aspects of care provision: type of service (specialised clinic vs routine consultation), and who provides that service (doctor or nurse).

Types of outcome measures

We considered three types of outcomes for inclusion in this review: patient outcomes; process of care outcomes; and utilisation outcomes, including both volume and costs.

Primary outcomes

Patient outcomes

• Mortality

• Health status (clinical outcomes and self‐reported outcomes)

• Satisfaction

• Quality of life

• Other (compliance, knowledge, preference for doctor or nurse)

Secondary outcomes

Process of care outcomes

• Practitioner adherence to clinical guidelines

• Practitioner healthcare activity (examinations, provision of advice)

Utilisation outcomes

Volume

• Frequency and length of consultations

• Number of return visits

• Number of prescriptions

• Numbers of tests and investigations

• Number of referrals to or frequency of use of other services

Costs

• Direct health service costs related to volume

• Indirect (societal) costs

Search methods for identification of studies

Electronic searches

We searched the following databases.

• Cochrane Central Register of Controlled Trials (CENTRAL; 2014, Issue 12), part of the Cochrane Library (www.cochranelibrary.com (searched 20.01.2015).

• MEDLINE In‐Process & Other Non‐Indexed Citations, MEDLINE Daily, MEDLINE and Ovid OLDMEDLINE 1946 to present, Ovid (searched 20.01.2015).

• Cumulative Index to Nursing and Allied Health Literature (CINAHL) 1981 to present, EbscoHost (searched 20.01.2015).

We performed an updated search in CENTRAL, MEDLINE, and CINAHL in March 2017. We have added one study to ‘Studies awaiting classification’ and will incorporate this study into the review at the next update.

Searching other resources

Grey literature databases

• The Grey Literature Report (http://www.nyam.org/library/online‐resources/grey‐literature‐report/) (searched 21.02.2017)

• OpenGrey (http://www.opengrey.eu/) (searched 21.02.2017)

Trial registries

• International Clinical Trials Registry Platform (ICTRP), World Health Organization (WHO) (http://www.who.int/ictrp/en/) (searched 21.02.2017)

• ClinicalTrials.gov, US National Institutes of Health (NIH) (http://clinicaltrials.gov/) (searched 21.02.2017)

We also searched the Science Citation Index and the Social Sciences Citation Index 1975 to present, for articles citing relevant studies, as well as Web of Knowledge (Thomson Reuters) (searched 27.01.2015) and the reference lists of all included papers and identified relevant reviews.

Please see Appendix 1 for strategies used and the PRISMA flow chart (Figure 1) for records retrieved, excluded, and included.

1.

Study flow diagram.

Data collection and analysis

Selection of studies

At least two review authors (among ML, NW, KW, EK, and AVV) independently screened search results at three levels: titles; abstracts to assess which studies potentially satisfied the inclusion criteria; and full‐text copies of papers that were potentially relevant. If we could not assess the paper for eligibility based on title or abstract, we obtained the full text. Where data was published in duplicate, we included these data only once in the review.

Data extraction and management

For this review, we designed a data extraction form that was based on the previously used standard form of the Cochrane Effective Practice and Organisation of Care Group (EPOC). At least two review authors (of ML, MB, NW, KW, EK, and AVV) independently abstracted data from each study and resolved differences by discussion.

If a single publication reported two or more separate studies, we extracted each study separately. If findings of a single study were spread across two or more publications, we extracted data from these publications as one. We extracted outcomes measured at different time points and presented in different publications, for example, at six months and two years after the intervention. We used the longest follow‐up in meta‐analyses. For each study with more than one control or comparison group for the nurse intervention, we reported only results for the control condition in which doctors provided the same intervention as the nurse.

Assessment of risk of bias in included studies

At least two review authors (among ML, MB, NW, KW, EK, and AVV) independently assessed risk of bias of each included study using the criteria suggested by EPOC (EPOC 2017). We assessed randomised trials for generation of allocation sequence, concealment of allocation, similar baseline outcome measurements, similar baseline characteristics, incomplete outcome data, blinding of participants, blinding of outcome assessors, protection against contamination, selective outcome reporting, and bias due to lack of power. We scored each study for risk of bias as follows: ’low’ if all key domains were scored as ’low risk’; ’unclear’ if one or two key domains were scored as ’unclear risk’; and ’high’ if more than two key domains were scored ’unclear risk’ or ’high risk’. When no information was available, we scored 'unclear risk'. For similar baseline characteristics and outcome measurements, we scored 'low risk' when baseline values were equal, or when analysis included a correction for differences in baseline values. We scored incomplete outcome data as low risk when follow‐up was ≥ 80% or when follow‐up was < 80%, with equal results attained by intention‐to‐treat (ITT) and per‐protocol (PP) analyses. With respect to blinding, we used the following approach. When investigators reported no blinding of patients and personnel, we scored 'unclear risk', because we do not know whether lack of blinding influenced study results. For some objective outcomes (e.g. mortality), blinding does not influence risk of bias, but for other outcomes in the same study (e.g. satisfaction), non‐blinding may influence outcomes.

We did not split the different outcomes for assessment of risk of bias within a study because the judgement of risk of bias was generally equal for all outcomes within a study. If the risk of bias judgement for a particular outcome was divergent, we commented on that.

We have shown assessments of risk of bias for included studies in the Characteristics of included studies table and have summarised this information in Figure 1 and Figure 2. We did not use risk of bias assessments in deciding which studies should be included in the meta‐analyses. However, we conducted sensitivity analyses by excluding studies with high risk of bias (see Sensitivity analysis). Furthermore, we used these assessments in interpreting study results and, particularly, in assessing the certainty of evidence for nurse‐doctor substitution.

2.

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.

Measures of treatment effect

The measure of effect size for continuous outcomes (patient satisfaction, disease activity score, pain, and length of consultation) was the standardised mean difference (SMD). The SMD is more appropriate than the mean difference (MD) in situations where the measurement instrument (i.e. the patient satisfaction scale) differs between studies. (See the Cochrane Handbook for Systematic Reviews of Interventions version 5.1.0.) We used MDs for patient outcomes including blood pressure, cholesterol, and glycated haemoglobin (HbA1c).

The measure of effect size used for dichotomous outcomes was the risk ratio (RR). (See the Cochrane Handbook for Systematic Reviews of Interventions version 5.1.0.) We converted odds ratios (ORs) to RRs using the built‐in calculator in RevMan 5.3 (RevMan 2014).

Unit of analysis issues

We included in the meta‐analyses three cluster‐randomised trials along with fifteen individually randomised trials. We included one cluster‐randomised trial that accounted for clustering in their analyses (Campbell 2014). Two of the included cluster‐randomised trials did not correct adequately for clustering (Chambers 1978; Spitzer 1973). We explored the impacts of these trials by performing a Sensitivity analysis.

Dealing with missing data

For missing or unclear information, we contacted study investigators to request clarification or additional information. For studies that reported continuous data but did not report standard deviations, we calculated these values from other available data such as standard errors, or imputed them using the methods suggested in Higgins 2011.

We extracted data from the ITT analysis when possible. If ITT data were not present, we excluded the study from meta‐analyses by performing a Sensitivity analysis.

Assessment of heterogeneity

Clinical settings, country contexts, and methodological diversity

We first made a qualitative assessment of the extent to which studies assessing a particular comparison were similar to one another. This included assessment of clinical settings, country contexts, and types of measurement scales to determine whether meta‐analysis was appropriate.

Statistical heterogeneity

We obtained an initial visual overview of statistical heterogeneity by scrutinising forest plots, looking at the overlap between confidence intervals around the estimate for each included study. In addition, we used the I² statistic and confidence intervals to estimate and quantify heterogeneity.

Assessment of reporting biases

To reduce possible publication bias, we employed strategies to search for and identify relevant unpublished studies for inclusion. These strategies included searching the grey literature and prospective trial registration databases to overcome time‐lag bias.

We used funnel plots for outcomes reported by more than four studies to visualise whether data showed asymmetry. No plots showed asymmetry. However, we identified too few studies for reliable assessment of funnel plot asymmetry ‐ visually or quantitatively.

Data synthesis

To summarise the effectiveness of the nurse‐doctor substitution, we performed several meta‐analyses. We conducted statistical meta‐analyses using the RevMan 5.3 software distributed by Cochrane (RevMan 2014). For studies in which quantitative data were absent or were insufficient for calculation, we reproduced the data as presented in the additional tables, undertook a structured synthesis, and reported the findings narratively. We performed a meta‐analysis if the nature of the outcome and other key aspects of studies were similar. We used adjusted RRs if available in the article. When not available, we calculated RRs from events. For categorical outcomes, we calculated log RRs and standard errors (SEs) of log RRs for both individual and cluster‐randomised trials. We analysed together log RRs for individual randomised trials and adjusted log RRs for cluster‐randomised trials. We preferred RRs over ORs because interpretation is intuitive. When no cluster‐randomised trials were included in the meta analysis, we used RRs instead of log RRs.

We used a random‐effects meta‐analysis, which is known to be more conservative and more suitable in the presence of any heterogeneity (Kontopantelis 2012). Although we expected substantial heterogeneity in some cases, which could be attributed to differences among populations, interventions, comparators, outcomes, and settings, we are aware that detection of existing heterogeneity can be problematic for meta‐analysis that includes a small number of studies (Kontopantelis 2013). Therefore, for greater transparency, we reported 95% confidence intervals of the I² statistic, obtained under an inverse variance DerSimonian‐Laird random‐effects model for continuous outcomes, and a Mantel‐Haenszel/DerSimonian‐Laird random‐effects model hybrid for dichotomous outcomes.

'Summary of findings'

We used the GRADE approach to assess the certainty of evidence related to each of the key outcomes (Schünemann 2009). We used the GRADE profiler to import data from Review Manager 5.3 and create Table 1 (RevMan 2014; GRADEpro GDT 2015).

For assessments of the overall certainty of evidence for each outcome, we downgraded the evidence from 'high certainty' by one level for serious (or by two levels for very serious) study limitations (risk of bias), indirectness of evidence, serious inconsistency, imprecision of effect estimates, or potential publication bias. We provided justification for decisions to downgrade or upgrade ratings by inserting footnotes into the table and made comments to aid readers' understanding of the review when necessary. We used plain language statements to report these findings in the review.

We used these assessments, along with evidence for absolute benefit or harm of the interventions and the sum of available data on all critical and important outcomes from each study included for each comparison, to draw conclusions about the effectiveness of nurse‐led primary care. Table 1 consists of critically important clinical and functional outcomes identified in the selected trials.

When judging the importance of SMDs, we acknowledged that 0.2 represents a slight effect, 0.5 a moderate effect, and 0.8 a significant effect (Guyatt 2008; Higgins 2011).

Subgroup analysis and investigation of heterogeneity

During the review process, we identified several factors that might explain heterogeneity in review findings, including type of nurse (i.e. nurse, registered nurse, nurse practitioner, specialised nurse); characteristics of the intervention and the comparator (i.e. total substitution, partial substitution); study size; duration of follow‐up; type of care (i.e. single contact, series, urgent care); range of patient complaints (i.e. all patients or particular patient groups); and setting. We undertook these as exploratory, hypothesis‐generating analyses because these factors were not identified a priori and several potentially explanatory factors were considered. We considered undertaking a subgroup analysis based on nurse title as described in the included studies, as has been done in other systematic reviews (Martínez‐González 2014a). However, we have little information about exact role definitions and educational levels of nurses in the different trials, and we know that job titles differ among countries; therefore, we decided it was not possible to create clear and valid subgroups for subgroup analyses.

Sensitivity analysis

We performed sensitivity analyses by excluding trials assessed as having high risk of bias (overall) (Chambers 1978; Lewis 1967; Hemani 1999; Mundinger 2000), cluster‐randomised trials (Chambers 1978; Spitzer 1973), trials presenting per‐protocol (PP) rather than intention‐to‐treat (ITT) data when follow‐up was < 80% (Chambers 1978; Mundinger 2000; Venning 2000), trials from low‐income countries (Sanne 2010), and trials in which investigators had calculated the RR from an OR (Iglesias 2013). We performed all sensitivity analyses on all outcomes.

Results

Description of studies

Results of the search

We identified a total of 4831 articles from electronic and supplementary searches. We excluded 4741 articles following a review of titles and abstracts and retrieved and assessed the full text of 90 articles. We excluded 78 full‐text articles that investigated the role of nurses working as supplements to primary care doctors and excluded two additional studies that involved a mix of primary and hospital care. Nine randomised trials met the inclusion criteria, and we included them in this update. We performed an updated search in CENTRAL, MEDLINE, and CINAHL in March 2017. We have added one study to ‘Studies awaiting classification’ and will incorporate this study into the review at the next update. We have presented the study flow diagram in Figure 1.

Included studies

We included nine new randomised trials in this update (Campbell 2014; Chan 2009; Dierick‐van Daele 2009; Houweling 2011; Iglesias 2013; Larsson 2014; Ndosi 2013; Sanne 2010; Voogdt‐Pruis 2010). The review now includes 18 randomised trials in which nurses worked as substitutes for doctors. Four of them were cluster‐randomised trials (Campbell 2014; Chambers 1978; Moher 2001; Spitzer 1973) that were randomised by practice (Campbell 2014; Moher 2001) or by family (Chambers 1978; Spitzer 1973). We described the findings of the included studies below and summarised them in the Characteristics of included studies table.

Setting

Six studies were conducted in the UK (Campbell 2014; Chan 2009; Lattimer 1998; Moher 2001; Ndosi 2013; Shum 2000), three in the Netherlands (Dierick‐van Daele 2009; Houweling 2011; Voogdt‐Pruis 2010), three in the USA (Hemani 1999; Lewis 1967; Mundinger 2000), three in Canada (Chambers 1978; Spitzer 1973; Venning 2000), one in Sweden (Larsson 2014), one in Spain (Iglesias 2013), and one in South Africa (Sanne 2010).

Nurses substituted for doctors in a range of care settings. Interventions were carried out in general practices/family practices (Campbell 2014; Chambers 1978; Dierick‐van Daele 2009; Houweling 2011; Iglesias 2013; Lattimer 1998; Moher 2001; Mundinger 2000; Sanne 2010; Shum 2000; Spitzer 1973; Venning 2000; Voogdt‐Pruis 2010),(out‐patient) nurse clinics (Chan 2009; Lewis 1967; Larsson 2014; Ndosi 2013). and specialised practices (Hemani 1999).

The study period ranged from 2 weeks in Venning 2000 to 48 months (Ndosi 2013) with a mean of 14 months (standard deviation (SD) 12 months). For one study, the study period remains unknown (Houweling 2011).

Role of the nurse

Included studies were of nurse‐doctor substitution in primary care for provision of first contact care (including urgent care), ongoing care for all presenting physical complaints, and follow‐up of patients with a particular chronic condition. Nurse‐doctor substitution for preventive services and health education in primary care has been less well studied.

• In five studies, the nurse assumed responsibility for first contact and ongoing care for all presenting patients (Chambers 1978; Hemani 1999; Iglesias 2013; Mundinger 2000; Spitzer 1973).

• In five studies, the nurse assumed responsibility for first contact care for patients wanting (urgent) consultations during routine practice hours ‐ Campbell 2014; Dierick‐van Daele 2009; Shum 2000; Venning 2000 ‐ or out‐of‐hours ‐ Lattimer 1998.

• In seven studies, the nurse had responsibility for ongoing treatment or follow‐up of patients with a particular chronic disease (Chan 2009; Houweling 2011; Larsson 2014; Lewis 1967; Moher 2001; Ndosi 2013; Sanne 2010).

• In one study, the nurse provided mainly health education or preventive services to a specific group of patients (Voogdt‐Pruis 2010).

Excluded studies

We excluded almost all excluded full‐text articles because they investigated the role of nurses working as supplements to primary care doctors. We excluded seven studies from this update that had been included in the previous version of the review (Laurant 2005): one study focussed on mental health problems (McIntosh 1997); three controlled before‐after studies ‐ Chambers 1977; Gordon 1974; Myers 1997; and three non‐randomised studies ‐ Flynn 1974; Kinnersley 2000; Stein 1974 . In addition, we excluded two studies that involved a mix of primary care and hospital care (Kuethe 2011; Irewall 2015). We listed these nine studies in the Characteristics of excluded studies tables.

Risk of bias in included studies

We prepared an assessment of risk of bias for each trial and illustrated final judgements for the ten criteria in Figure 2 and Figure 3. All studies had some methodological shortcomings, in most instances related to unclear risk of bias for different criteria. We judged only one study to be at high risk of bias for more than one criterion (Mundinger 2000). The criteria most commonly assessed as having unclear risk of bias were blinding of personnel, outcome assessment, and selective reporting. The criterion most commonly assessed as having high risk of bias was contamination (Lewis 1967; Mundinger 2000; Spitzer 1973; Voogdt‐Pruis 2010).

3.

Risk of bias summary: review authors' judgements about each risk of bias item for each included study.

Allocation

Most studies stated that participants or practices (in case of cluster randomisation (Campbell 2014)) were assigned randomly, according to a computerised randomisation scheme. Twelve of the included studies met the ‘low risk of bias’ criteria for random sequence generation. Most of these studies used a computer (Chan 2009; Hemani 1999; Voogdt‐Pruis 2010), and some used envelopes for this purpose (Dierick‐van Daele 2009; Houweling 2011; Larsson 2014; Shum 2000). For six studies, the risk of bias for random sequence generation was unclear owing to poor reporting. Campbell 2014 used a random component in the sequence process, and 10 of 15 practices withdrew after randomisation, which made the risk of bias unclear.

Baseline values

Most studies provided similar outcome measurements between the two study arms at baseline or corrected for differences in baseline values. Hemani 1999 did not assess baseline characteristics, and study arms differed for one or two outcome measures in Larsson 2014.

Blinding

Risk of performance bias was low in two studies (Hemani 1999; Voogdt‐Pruis 2010). In Hemani 1999, personnel did not know which patients were included in the study, and Voogdt‐Pruis 2010 collected data retrospectively and asked patients for their consent after one year. For all other studies, we judged the risk of performance bias as unclear because no information was available. We expect that patients and personnel were not blinded in these studies because the care provider constitutes the intervention. Whether this lack of blinding influences outcomes is unclear. Three studies satisfied the criteria for blinding of outcome assessors (Chan 2009; Iglesias 2013; Ndosi 2013). These studies provided independent researchers who where blind to group assignment when measuring outcomes. Most studies did not provide sufficient information on blinding of outcome assessment; we therefore assessed them as having unclear risk of detection bias.

Incomplete outcome data

Three studies reported follow‐up less than 80% (Chambers 1978; Mundinger 2000; Venning 2000); we therefore judged these studies to have high risk of bias for incomplete outcome data. In most studies, 80% or more of the initial participants completed the study. Risk of bias due to incomplete outcome data was unclear in Hemani 1999 because of limited reporting about follow‐up. Ndosi 2013 reported follow‐up of less than 80%. However, investigators performed both ITT and PP analyses and reported the same results (Ndosi 2013).

Selective reporting

We judged two studies to have low risk for selective outcome reporting bias (Campbell 2014; Ndosi 2013). A protocol was available for each study, and these papers reported predefined outcome measures. Absence of study protocols to confirm reporting of all intended outcomes led to the unclear judgement in all other studies.

Other potential sources of bias

Risk of bias due to contamination was high in four studies (Lewis 1967; Mundinger 2000; Spitzer 1973; Voogdt‐Pruis 2010). These studies reported an increased likelihood of cross‐over of patients or personnel between groups. Contamination was not likely in six studies (Campbell 2014; Chan 2009; Dierick‐van Daele 2009; Moher 2001; Ndosi 2013; Sanne 2010). Lack of information or insufficient details in the paper led to judgement of unclear risk in the other studies.

Effects of interventions

See: Table 1

Patient outcomes

A total of 18 trials investigated patient outcomes (Campbell 2014; Chambers 1978; Chan 2009; Dierick‐van Daele 2009; Hemani 1999; Houweling 2011; Iglesias 2013; Larsson 2014; Lattimer 1998; Lewis 1967; Moher 2001; Mundinger 2000; Ndosi 2013; Sanne 2010; Shum 2000; Spitzer 1973; Venning 2000; Voogdt‐Pruis 2010) (Table 2).

1. Patient outcome: health status.

Study	Various health status outcomes
Chambers 1978	Health status: ‐ Emotional function: no differencea ‐ Social function: no differencea
Chan 2009	Health status: ‐ Severity of symptoms: Nurse group had greatest improvement. Difference adjusted for baseline 2.3 (95% CI 1.4 to 3.1), P < 0.001
Dierick‐van Daele 2009	Health status: ‐ Burden of illness: nurse vs doctor (MD 0.27, P = 0.16) ‐ Concerns about illness: nurse vs doctor (MD 0.11, P = 0.20) ‐ Absence of work: both nurse and doctor 1.11 daysa,b ‐ Ability to perform daily activities: nurse mean 2.53, doctor mean 2.69a,b
Houweling 2011	Objective measures of patient health (MD (95% CI)): ‐ BMI (kg/m2): nurse ‐0.2 (‐0.5; 0.1), doctor ‐0.3 (‐0.6; ‐0.1), P = 0.377 ‐ Cholesterol/HDL: nurse ‐0.03 (‐0.1; 0.2), doctor ‐0.07 (‐0.1; ‐0.2), P = 0.321 Health status: ‐ Diabetes symptom score: no differencea,b ‐ Fatigue: no differencea,b ‐ Cognitive distress: no differencea,b
Larsson 2014	Health status: ‐ DAS28‐CRP: nurse vs doctor 0.05 (95% CI ‐0.28 to 0.19, P = 0.70) ‐ ESR (mm/h): nurse vs doctor ‐1.05 (95% CI ‐3.97 to 1.86, P = 0.47) ‐ CRP (mg/L): nurse vs doctor ‐1.07 (95% CI ‐2.02 to ‐0.12, P = 0.03) ‐ Swollen joints (28): nurse vs doctor 0.13 (95% CI ‐2.18 to 0.61, P = 0.60) ‐ Tender joints (28): nurse vs doctor 0.33 (95% CI ‐0.47 to 1.13, P = 0.42) ‐ VAS global health (mm): nurse vs doctor 4.29 (95% CI ‐2.58 to 11.16, P = 0.22)
Lewis 1967	Health status: ‐ Resolution of symptoms in nurse group from 16.33 to 18.39 (possible range 6 to 24; higher scores mean fewer reductions in complaints). Doctors no change. P < 0.02
Moher 2001	Health status (lifestyle factor): ‐ Smoking: no differencea ‐ Blood pressure (mmHg) systolic: nurse 148 (142 to 153), GP 147 (135 to 153), P = 0.82a ‐ Blood pressure (mmHg) diastolic: nurse 80 (74 to 87), GP 81 (75 to 83), P = 0.82a ‐ Cholesterol (mmol/L) total: nurse 5.4 (5.2 to 5.5), GP 5.5 (5.0 to 5.9), P = 0.61a ‐ Cholesterol (mmol/L) high‐density lipoprotein: nurse 1.2 (1.1 to 1.3), GP 1.2 (1.2 to 1.3), P = 0.83a
Mundinger 2000	Health status (10 dimensions): no differenceb Objective measures of patient health: ‐ Asthma ‐ peak flow: NP 292.82 (94.2), GP 319.90 (136.56), P = 0.365
Ndosi 2013	Health status: ‐ Fatigue ITT: nurse < doctor; mean (95% CI) 3.38 (‐2.01 to 8.76), P = 0.0171 ‐ Stiffness ITT: nurse < doctor; mean (95% CI) 8.91 (‐2.66 to 20.5), P = 0.0113 ‐ RAQoL ITT: nurse < doctor; mean (95% CI) ‐0.14 (‐1.77 to 1.49), P = 0.0001 ‐ HAQ ITT: nurse > doctor; mean (95% CI) ‐0.07 (‐0.21 to 0.07), P < 0.0001 ‐ HAD‐Anxiety ITT: nurse < doctor; mean (95% CI) 0.54 (‐0.36 to 1.43), P = 0.0179 ‐ HAD‐Depression ITT: nurse < doctor; mean (95% CI) 0.12 (‐0.65 to 0.89), P = 0.0004 ‐ ASES ITT: nurse > doctor; mean (95% CI) ‐0.92 (‐4.96 to 3.12), P = 0.0019
Sanne 2010	Health status: ‐ Cumulative failure: nurse 48%, doctor 44% HR (95% CI) 1.09 (0.89 to 1.33) ‐ All virological failure: nurse 11%, doctor 10% HR (95% CI) 1.15 (0.75 to 1.76) ‐ Toxicity failure: nurse 17%, doctor 16% HR (95% CI) 1.04 (0.74 to 1.45) ‐ Death: nurse 3%, doctor 3% HR (95% CI) 0.92 (0.39 to 2.17)
Spitzer 1973	Health status: ‐ Physical function (3 indicators): nurses 86%, doctors 88%b ‐ Emotional function: nurses 58%, doctors 58%b ‐ Social function: nurses 84%, doctors 83%b
Venning 2000	Health status: no differencea
Voogdt‐Pruis 2010	Objective measures of patient health: ‐ LDL cholesterol: nurse 2.9, doctors 3.0, P = 0.07 ‐ BMI: nurse 27.2, doctor 27.2, P = 0.87 Health status (lifestyle factor): ‐ Smoking: 4% of smokers in the GP group (4/102) and 6% in the practice nurse group (4/67)b Subgroup: at‐risk patients ‐ Systolic blood pressure: nurse 144.0, doctor 147.6, P = 0.1 ‐ Total cholesterol: nurse 5.2, doctor 5.6, P = 0.006 ‐ LDL cholesterol: nurse 3.1, doctor 3.3, P = 0.16 ‐ BMI: nurse 28.6, doctor 28.6, P = 0.78

^a Authors reported no effect size or reported effect sizes in graphs (no exact effect sizes extracted).

^b No p‐value reported.

ASES: Standardized Shoulder Assessment Form.

BMI: body mass index.

CI: confidence interval.

CRP: C‐reactive protein.

DAS28: disease activity score 28.

ESR: erythrocyte sedimentation rate.

GP: general practitioner.

HAD: Hospital and Anxiety Depression Scale.

HAQ: Health Assessment Questionnaire.

HDL: high‐density lipoprotein.

HR: heart rate.

ITT: intention‐to‐treat.

LDL: low‐density lipoprotein.

MD: mean difference.

NP: nurse practitioner.

RAQoL: Rheumatoid Arthritis Quality of Life Questionnaire.

VAS: visual analogue scale.

*there may be additional data in the Campbell 2014 articles that have not been extracted

We have grouped patient outcomes into the following categories: mortality, health status outcomes, satisfaction and preferences, quality of life, and other patient outcomes.

Mortality

Eight trials evaluated mortality (Campbell 2014; Hemani 1999; Lattimer 1998; Ndosi 2013; Sanne 2010; Shum 2000; Spitzer 1973; Voogdt‐Pruis 2010). Meta‐analysis of data from these trials suggests that nurse‐led primary care may lead to slightly fewer deaths among certain groups of patients, compared to doctor‐led care. Among those people who received doctor‐led care, 6 per 1000 people died. Among those people who received nurse‐led care, between 4 and 6 people per 1000 died (RR 0.77, 95% CI 0.57 to 1.03, low certainty evidence). Data show no evidence of statistical heterogeneity (I² = 0%, 95% CI 0 to 68; Analysis 1.1). The evidence is of low certainty owing to a wide confidence interval that includes no effect (imprecision) and clinical heterogeneity, as the trials contributing to this estimate are quite varied (some focus on people with specific health issues and others on more generalist primary care attenders). Excluding from the meta‐analysis a trial assessed as cluster‐randomised did not greatly change the result (RR 0.56, 95% CI 0.33 to 0.95) (Lattimer 1998). Results did not differ considerably in the other sensitivity analyses.

1.1. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 1 Mortality.

Other health status outcomes

We grouped health status outcomes into clinical outcomes (e.g. blood pressure, cholesterol, glycated haemoglobin (HbA1c)) and self‐reported measurements of health status, including measures related to physical functioning (e.g. pain, Disease Activity Score (DAS)) and lifestyle factors (e.g. smoking, alcohol consumption, exercise).

Clinical outcomes

Three trials focussing on patients with cardiovascular disease or diabetes evaluated clinical outcomes (Houweling 2011; Mundinger 2000; Voogdt‐Pruis 2010). Meta‐analyses for blood pressure levels suggest that, compared to doctor‐led care, nurse‐led primary care probably slightly improves blood pressure outcomes for both systolic blood pressure (MD ‐3.73, 95% CI ‐6.02 to ‐1.44, moderate‐certainty evidence; Analysis 1.5) and diastolic blood pressure (MD ‐2.54, 95% CI ‐4.57 to ‐0.52, moderate‐certainty evidence; Analysis 1.6). For both outcomes, data show no evidence of statistical heterogeneity (systolic blood pressure: I² = 0%, 95% CI 0 to 90; diastolic blood pressure: I² = 0%). Assessment of moderate‐certainty evidence is due to high risk of bias in one of the included studies (Mundinger 2000). Results did not change considerably under Sensitivity analysis.

1.5. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 5 Systolic blood pressure.

1.6. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 6 Diastolic blood pressure.

A meta‐analysis for HbA1c suggest that nurse‐led primary care probably leads to similar outcomes as doctor‐led care and a meta‐analysis for cholesterol suggest that nurse‐led primary care leads to similar outcomes as doctor‐led care for patients with heart failure or diabetes (HbA1c levels: MD 0.08, 95% CI ‐0.25 to 0.41, moderate‐certainty evidence; Analysis 1.8; total cholesterol: MD ‐0.15, 95% CI ‐0.32 to 0.02, high‐certainty evidence; Analysis 1.7). For both outcomes, data show no evidence of statistical heterogeneity (cholesterol: I² = 0%, 95% CI 0 to 90; HbA1c: I² = 0%). The assessment of moderate‐certainty evidence for HbA1c evidence is due to high risk of bias in one of the included studies (Mundinger 2000). Results did not change considerably under Sensitivity analysis.

1.8. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 8 HbA1c.

1.7. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 7 Total cholesterol.

Self‐reported measurements of health status

Twelve trials provided self‐reported measurements of health status (Chambers 1978; Chan 2009; Dierick‐van Daele 2009; Houweling 2011; Larsson 2014; Lewis 1967; Moher 2001; Ndosi 2013; Sanne 2010; Spitzer 1973; Venning 2000; Voogdt‐Pruis 2010). Two trials among patients with rheumatological diseases (Larsson 2014; Ndosi 2013) assessed the outcomes disease activity in rheumatoid arthritis and pain. Meta‐analyses for DAS and pain suggest that nurse‐led primary care, compared to doctor‐led care, for patients with rheumatological disease probably leads to similar outcomes for DAS and pain (DAS: MD 0.04, 95% CI ‐0.17 to 0.24, moderate‐certainty evidence; Analysis 1.9; pain: MD 0.76, 95% CI ‐3.85 to 5.38, moderate‐certainty evidence; Analysis 1.3). For both outcomes, there was no evidence of statistical heterogeneity (DAS: I² = 1%; pain: I² = 0%). The evidence is of moderate certainty owing to indirectness, as only patients with rheumatoid arthritis were included. Results did not change considerably under Sensitivity analysis.

1.9. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 9 Disease Activity Score.

1.3. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 3 Pain.

We included in a meta‐analysis three studies assessing physical functioning. Results suggest that, compared to doctor‐led care, nurse‐led primary care may lead to little or no difference in physical functioning (RR 1.03, 95% CI 0.98 to 1.09, low‐certainty evidence; Analysis 1.2). Results showed statistical heterogeneity (I² = 62%, 95% CI 0 to 87, P = 0.07). The evidence is of low certainty owing to inconsistency and high risk of bias. Results did not change considerably when a trial assessed as having high risk of bias (Chambers 1978) was excluded under Sensitivity analysis.

1.2. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 2 Physical function (better vs not better).

In addition, studies measured a large number of other outcomes related to health status and lifestyle. It was not possible to pool these results because of the wide range of outcomes assessed, but results suggest that care provided by nurses was at least as good as care provided by doctors. We have summarised the details in Table 2.

Satisfaction and preferences

Ten trials measured satisfaction with care (Campbell 2014; Dierick‐van Daele 2009; Iglesias 2013; Larsson 2014; Lewis 1967; Mundinger 2000; Ndosi 2013; Shum 2000; Spitzer 1973; Venning 2000). This outcome was assessed in many different ways across trials; therefore we could include only seven trials in a meta‐analysis (Campbell 2014; Dierick‐van Daele 2009; Iglesias 2013; Larsson 2014; Mundinger 2000; Shum 2000; Venning 2000). This showed that patient satisfaction is probably slightly higher in nurse‐led primary care than in doctor‐led primary care (SMD 0.08, 95% CI 0.01 to 0.15, moderate‐certainty evidence; Analysis 1.10). The evidence is of moderate certainty owing to inconsistency (I² = 56%, 95% CI 23 to 74), suggesting that the extent to which nurse‐led care increased patient satisfaction varied considerably with the context of care. Results did not change considerably under Sensitivity analysis. Findings of trials not included in this meta‐analysis also suggest that patients are probably at least as satisfied with nurse‐led care as with doctor‐led care. Table 3 summarises the data for all trials that assessed this outcome.

1.10. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 10 Patient satisfaction.

2. Patient outcome: satisfaction and preference.

Study	Satisfaction, preference
Campbell 2014	Overall satisfaction: nurse triage vs GP triage MD 2.60 (95% CI 0.58 to 4.63)a
Dierick‐van Daele 2009	Overall satisfaction: nurse vs doctor (0 to 10), MD ‐0.015, P = 0.83 Communication/attitude (1 to 6) ‐ Understanding: nurse vs doctor, MD ‐0.015, P = 0.41 ‐ Telling the plan: nurse vs doctor, MD ‐0.02, P = 0.74 ‐ Explaination goals and treatment: nurse vs doctor, MD ‐0.01, P = 0.76 ‐ Importance advice: nurse vs doctor, MD ‐0.07, P = 0.17 ‐ Appropriate attention: nurse vs doctor, MD 0.01, P = 0.78 Provision of information (1 to 6) ‐ Cause of problems: nurse vs doctor, MD ‐0.08, P = 0.21 ‐ Relief of symptoms: nurse vs doctor, MD ‐0.04, P = 0.47 ‐ Duration of illness: nurse vs doctor, MD ‐0.09, P = 0.25 ‐ Change of recurrence: nurse vs doctor, MD ‐0.15, P = 0.08 ‐ What to do: nurse vs doctor, MD ‐0.06, P = 0.45 Subgroup at least 1 chronic condition Satisfaction: NP 8.35 (1.07) vs GP 8.11 (1.32), P = 0.02 Judgement seeing the right professional: P = 0.35b Attending same provider in future: P = 0.67 Recommendation to others: P = 0.41
Iglesias 2013	Satisfaction: ‐ Satisfaction with duration of the visit (0 to 10): doctor 8.1, nurse 8.4; MD (95% CI%) 0.256 (0.016 to 0.496)a ‐ Satisfaction with personal attention (0 to 10): doctor 8.1, nurse 8.4, MD (95% CI%) 0.240 (0.003 to 0.476)a ‐ Satisfaction with explanations and information received in the visit (0 to 10): doctor 8.3, nurse 8.5, MD (95% CI%) 0.240 (0.015 to 0.495)a Provider preference: More than 40% of patients in each group expressed indifference. In the control group, 13.9% of patients would prefer to be seen by a nurse, as opposed to 20.9% in the intervention group.a
Larsson 2014	Confidence: ‐ NRS confidence: nurse vs doctor: 0.20 (95% CI ‐0.29 to 0.69), P = 0.42
Lewis 1967	Provider preference: doctor 5.72 vs nurse 9.80, P < 0.001. Possible range 0 to 20; higher scores indicate a more positive view of the provider.
Mundinger 2000	Satisfaction (9 items): no difference in overall satisfaction, or on any of the 9 subscalesa Would recommend provider to others: no differencea
Ndosi 2013	Leeds Satisfaction Questionnaire ‐ LSQ Week 26 ‐ LSQ‐General: nurse vs doctor effect size: 0.17, P = 0.036 ‐ LSQ‐Information: nurse vs doctor effect size: 0.08, P = 0.327 ‐ LSQ‐Empathy: nurse vs doctor effect size: 0.05, P = 0.557 ‐ LSQ‐Technical: nurse vs doctor effect size: 0.08, P = 0.293 ‐ LSQ‐Attitude: nurse vs doctor effect size: 0.14, P = 0.082 ‐ LSQ‐Access: nurse vs doctor effect size: 0.01, P = 0.936 Week 52 ‐ LSQ‐General: nurse vs doctor effect size: 0.12, P = 0.183 ‐ LSQ‐Information: nurse vs doctor effect size: 0.09, P = 0.301 ‐ LSQ‐Empathy: nurse vs doctor effect size: 0.05, P = 0.578 ‐ LSQ‐Technical: nurse vs doctor effect size: 0.08, P = 0.369 ‐ LSQ‐Attitude: nurse vs doctor effect size: 0.08, P = 0.375 ‐ LSQ‐Access: nurse vs doctor effect size: 0.10, P = 0.248
Shum 2000	Satisfaction: ‐ Professional care: nurse 79.2 (13.4) vs GP 76.7 (15.1), possible range 0 to 100, P = 0.002 ‐ Relationship to provider: nurse 64.3 (15.7) vs GP 64.2 (16.9), possible range 0 to 100, P = 0.945 ‐ Adequacy of time: nurse 73.3 (16.9) vs GP 67.7 (19.3), possible range 0 to 100, P < 0.001 ‐ Explanation helpful: nurse 88.8% vs GP 87.3%, P = 0.359 ‐ Advice helpful: nurse 86.9% vs GP 83.9%, P = 0.060 Provider preference: GP group: 47.5% prefer GP, 2.0% nurse, 50.5% no preference. Nurse group: 31.5% prefer GP, 7.5% nurse, 61% no preference; P < 0.001
Spitzer 1973	Satisfaction: nurses 96%, doctors 97%a
Venning 2000	Satisfaction: Adults ‐ Communication: NP 4.35 (0.54) vs GP 4.21 (0.60), P = 0.001 ‐ Distress relief: NP 4.43 (0.47) vs GP 4.26 (0.57), P = 0.001 ‐ Professional care: NP 4.44 (0.49) vs GP 4.22 (0.57), P < 0.001 Children ‐ General: NP 4.39 (0.46) vs GP 4.17 (0.57), P < 0.001 ‐ Communication with parent: no difference ‐ Communication with child: NP 4.16 (0.63) vs GP 3.67 (0.77), P < 0.001 ‐ Distress relief: NP 4.41 (0.53) vs GP 4.21 (0.64), P = 0.002 ‐ Adherence intent: no difference

^ano p‐value reported

^b authors reported no effect size or reported effect sizes in graphs (no exact effect sizes extracted)

CI: confidence interval.

GP: general practitioner.

LSQ: Leeds Satisfaction Questionnaire.

MD: mean difference.

NP: nurse practitioner.

NRS: Numeric Rating Scale

*there may be additional data in the Campbell 2014 articles that have not been extracted

In addition, investigators measured a large number of other outcomes related to patient satisfaction and preferences. It was not possible to pool these results, but findings suggest that patients are at least as satisfied with nurses as with doctors. We have summarised details in Table 3.

Quality of life

Six trials evaluated quality of life (Campbell 2014; Chan 2009; Dierick‐van Daele 2009; Houweling 2011; Mundinger 2000; Ndosi 2013). Meta‐analysis of data from these trials suggests that quality of life may be slightly higher for people receiving nurse‐led primary care, compared to doctor‐led primary care (SMD 0.16, 95% CI 0.00 to 0.31, low‐certainty evidence; Analysis 1.4). The evidence is of low certainty owing to inconsistency (I² = 85%, 95% CI 69 to 93) and to imprecision, as the confidence interval touches on the null. The heterogeneity was caused by one trial (Chan 2009), which included a specific patient group (i.e. people who had experienced dyspepsia after direct access gastroscopy). After we excluded this trial, we found that there may be little or no difference in quality of life among patients receiving nurse‐led primary care, compared to doctor‐led primary care (SMD 0.02, 95% CI ‐0.01 to 0.05). The results also did not show evidence of heterogeneity or change considerably under other Sensitivity analysis.

1.4. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 4 Quality of life.

Other patient outcomes

Investigators measured a large number of other patient outcomes, including patient knowledge (understanding the health issue) and patient enablement (coping with his or her health issues). It was not possible to pool these results, but findings suggest that care provided by nurses was probably at least as good as care provided by doctors. We have summarised the details in Table 4.

3. Patient outcome: compliance and other.

Study	Compliance	Other
Mundinger 2000		Rating information (5 items): no differencea,b
Venning 2000		Enablement: nurse vs GP, MD = 0.65 (CI ‐1.50 to 0.19), P = 0.13
Voogdt‐Pruis 2010	Patient adherence to medical treatment after 1 year of follow‐up nurse vs doctor (95% CI) Medication blood pressure: 92.2 vs 84.9 (1.06 to 3.73; P = 0.03) Forgetting to take medication: group difference 1.32 (0.88 to 1.97; P = 0.18) ‐ Never: 52.6 vs 61.0 ‐ Sometimes: 46.8 vs 39.0 Patient lifestyle after 1 year of follow‐up nurse vs doctor (95% CI) ‐ Exercise: 28.6 vs 27.3 (0.73 to 1.67; P = 0.79) ‐ Alcohol 5 days per week at most: 78.6 vs 75.5 (0.79 to 2.01; P = 0.33) ‐ Alcohol 2 for woman, 3 for man at most: 79.1 vs 80.6 (0.53 to 1.56; P = 0.73) ‐ Fat intake: 6.5 vs 7.2 (0.02 to 1.28; P = 0.04)

^aTrial authors reported only the direction of the outcome; it is unknown if the difference is statistically significant.

^bTrial authors reported no effect size or reported effect sizes on graphs (no exact effect sizes extracted).

CI: confidence interval.

GP: general practitioner.

MD: mean difference.

Process of care outcomes

Ten trials investigated process of care outcomes (Campbell 2014; Dierick‐van Daele 2009; Houweling 2011; Moher 2001; Mundinger 2000; Ndosi 2013; Shum 2000; Spitzer 1973; Venning 2000; Voogdt‐Pruis 2010). We have summarised the data in Table 5. Owing to the large variety of approaches used in measuring the process of care, we did not judge it appropriate to pool these data in a meta‐analysis. The individual trial results show some differences between nurses and primary care doctors in process of care measures. For example, investigators reported that nurses gave more advice/information to patients and adhered to guidelines more frequently. However, the quality of patient examinations appeared to be similar between nurses and doctors. Overall, we assessed this evidence to be of very low certainty as the results were non‐comparable and we could not calculate an overall effect size.

4. Process of care outcomes.

Study	Provider care
Campbell 2014	Difficulty with (nurse triage vs GP triage, MD (95% CI): Phone access: 6.49 (–1.26 to 14.25)a Receiving prompt care: 6.63 (3.23 to 10.03)a Seeing a doctor or nurse: 3.67 (–0.37 to 7.71)a Getting medical help: 5.09 (2.69 to 7.50)a Convenience of care 3.68 (1.13 to 6.24)a Problem resolution: nurse triage vs GP triage: 0.41 (–1.86 to 2.67)a Process indicators: ‐ Number of contacts per person: nurse vs GP triage: 1·04 (1·01 to 1·08)a ‐ 23% in the GP‐triage group and 12% in the nurse‐triage group had just 1 contact after their initial consultation requestb
Dierick‐van Daele 2009	Adherence to guidelines: nurse 79.8%, doctor 76.2%a,c
Houweling 2011	Process indicators: ‐ Patients with last retina control > 24 months ago (n = 64) referred to an ophthalmologist: nurse 24/34 (70.6) vs GP 11/30 (36.7), P = 0.007 ‐ Patients with feet at‐risk (n = 109) for whom measures were taken: nurse 34/60 (56.7) vs GP 13/49 (26.5), P = 0.001 ‐ Patients referred to an internist to start insulin therapy: nurse 10/102 (9.8) vs GP 2/104 (1.9), P = 0.015 ‐ Patients with HbA1c ≥ 7 at baseline (n = 120), for whom glucose‐lowering therapy was intensified: nurse 53/64 (82.8) vs GP 28/56 (50.0), P = 0.001 ‐ Patients with BP > 140/90 at baseline (n = 170) for whom blood pressure‐lowering therapy was intensified: nurse 42/85 (49.4) vs GP 24/85 (28.2), P = 0.005 ‐ Patients not meeting target values for lipid profile at baseline (n = 55), for whom lipid‐lowering therapy was intensified: nurse 13/29 (44.8) vs GP 13/26 (50.0), P = 0.147
Moher 2001	Adequate assessment: ‐ Clinical assessment: nurse vs GP: 9% (95% CI ‐3 to 22), P = 0.13 ‐ Blood pressure: no differencec ‐ Cholesterol: no differencec ‐ Smoking status: no differencec
Mundinger 2000	Documentation of provider behaviour diabetes care: ‐ Education (8 items): overall ‘any education’: nurse 84,9% vs medical doctor 42.4% (P < 0.001). With regard to specific items, nurse more education: 4 out of 7 topics: nutrition, weight, exercise, and medication (P < 0.01) ‐ History taken (5 items): no difference ‐ Monitoring (9 items): nurse ordered/carried out more laboratory tests, such as urinalysis (nurse 80.2%, medical doctor 55.9%, P < 0.01) and glycosylated haemoglobin (A1C value) (nurse 81.4, medical doctor 66.1, P < 0.05); nurse reported more frequently height of patients (nurse 91.9%, medical doctor 71.2%, P < 0.01). On other 6 items, no difference ‐ Referral (1 item): no differences
Ndosi 2013	Interventions: ‐ Giving patient education: nurse > doctor; RR (95% CI) 1.76 (1.15 to 2.69), P = 0.009 ‐ Giving psychosocial support: nurse > doctor; RR (95% CI) 3.29 (2.55 to 4.24), P < 0.0001
Shum 2000	Provision of information: ‐ Self‐medication: nurse 22.2% vs GP 13.7%, P < 0.001 ‐ Self‐management: nurse 81.7% vs GP 57.6%, P < 0.001
Spitzer 1973	Adequate treatment: ‐ Drug treatment: nurses 71%, doctors 75%a ‐ Management of episodes: nurses 69%, doctors 66%a
Voogdt‐Pruis 2010	Lifestyle and medical intervention nurse vs doctor: ‐ Smoking behaviour 8.2% vs 3.2%a ‐ Blood pressure 35.4% vs 26.6% (1.01 to 2.24; P = 0.04) ‐ Lipids 47.1 vs 22.3 (1.98 to 4.43; P < 0.01) ‐ Weight 36.9 vs 7.6 (4.26 to 12.52; P < 0.01) ‐ Exercise 19.4 vs 3.2a ‐ Food intake 14.6 vs 3.2a ‐ Medication 22.3 vs 14.7 (0.99 to 2.59; P = 0.05) ‐ None 22.8 vs 43.2 (1.69 to 3.86; P < 0.01) Asked about the use of medication: nurse vs doctor Group difference 2.12 (1.38 to 3.26; P < 0.01) ‐ Never 57.4 vs 75.4 ‐ Sometimes 20.0 vs 14.4 ‐ Often 22.1 vs 9.7
Venning 2000	Examinations: nurse vs GP: MD 0.19 (95% CI ‐0.03 to 0.71), P = 0.072

^aNo P value reported.

^bTrial authors reported only the direction of the outcome; it remains unknown whether the difference is statistically significant.

^cTrial authors reported no effect size or reported effect sizes in graphs (no exact effect sizes extracted).

CI: confidence interval.

GP: general practitioner.

MD: mean difference.

RR: risk ratio.

*there may be additional data in the Campbell 2014 articles that have not been extracted

Utilisation outcomes

Sixteen trials measured utilisation and costs (Campbell 2014; Chan 2009; Dierick‐van Daele 2009; Hemani 1999; Houweling 2011; Iglesias 2013; Larsson 2014; Lattimer 1998; Lewis 1967; Moher 2001; Mundinger 2000; Ndosi 2013; Shum 2000; Spitzer 1973; Venning 2000; Voogdt‐Pruis 2010). The range of outcomes varied across trials and can be grouped into four categories: length and frequency of consultations; numbers of prescriptions, tests, and investigations ordered; use of other healthcare services, such as hospital admissions or referral to other professionals (Table 6); and costs (Table 7). Findings for each of these categories are presented below.

5. Utilisation outcomes.

Study	Number, length, and frequency of consultations	Numbers of prescriptions, tests, and investigations	Use of other services
Dierick‐van Daele 2009			Referrals: nurse 12%, doctor 14.2%, P = 0.24a
Hemani 1999	Compared to qualified doctors Consultation rate: Nurses 3.52 vs qualified doctors 4.03 (P > 0.05) Compared to residents (trainee doctors) Consultation rate: Nurses 3.52 vs residents 2.95 (P < 0.05)	Mean utilisation rate: Compared to qualified doctors Tests & investigations: Lab tests: NP 32.67, doctor 29.46, P > 0.05 Radiological tests (total): NP 1.68. doctor 1.37, P > 0.05 ‐ CT/MRI: NP 0.32, doctor 0.13, P < 0.05 ‐ Ultrasound: NP 0.16, doctor 0.07, P < 0.05 Compared to residents (trainee doctors) Tests & investigations: Lab tests: NP 32.67, doctor 28.26, P > 0.05 ‐ Urinalysis: NP 1.31, doctor 0.99, P < 0.05 ‐ Thyroid function: NP 0.37, doctor 0.19, P < 0.05 Radiological tests: NP 1.68, doctor 1.48, P > 0.05	Mean utilisation rate: Compared to qualified doctors Hospital admission: NP 0.43, doctor 0.33, P > 0.05 Emergency room visits: NP 1.22, doctor 1.23, P > 0.05 Specialty visits: NP 5.35, doctor 4.26, P > 0.05 Compared to residents (trainee doctors) Hospital admission: NP 0.43, doctor 0.31, P > 0.05 Emergency department visits: NP 1.22, doctor 1.05, P > 0.05 Specialty visits: NP 5.35, doctor 4.21, P > 0.05
Houweling 2011	Mean number of visits: nurse 6.1, GP 2.8 (P < 0.0001) Total duration of visits: significantly higher in nurse groupa Consultation of nurses' patients with GP: Median number of these consultations per patient was 1.4 (25 to 75 quartiles: 0.0 to 2.0) with median time of 1.0 (25 to 75 quartiles: 0.0 to 3.3) minute
Iglesias 2013	Level of resolution by nurses: Nurses led 86.3% (95% CI 83.6 to 88.7) of consultations without referral to GP (referrals according to protocol indication not included)
Larsson 2014		Proportion nurse‐led vs doctor‐led: ‐ Cortisone injections in addition to regular rheumatologist monitoring visits (1:0.7; P = 0.463) ‐ Blood tests (1:3.9; P = 0.014) ‐ Radiography (1:1.6; P = 0.162) ‐ Pharmacological therapy (1:1.1; P = 0.029)	Proportion nurse‐led vs doctor‐led: ‐ Additional telephone calls to a rheumatology nurse (1:1.8; P = 0.060) ‐ Additional telephone calls to a rheumatologist (1:1.9; P =0.287) ‐ Additional rheumatologist visits (1:2.4; P = 0.077) ‐ Team rehabilitation in in‐patient settings (0:79; P = 0.086) ‐ Team rehabilitation in out‐patient settings (15:0; P = 0.135) ‐ Occupational therapist treatments (0:3.0; P = 0.162) ‐ Psychosocial treatments (0:1.0; P = 0.152) ‐ Specialist consultations (1:1.0; P = 0.949)
Lattimer 1998	Impact on GP workload: ‐ Telephone advice from GP: fewer with nurse‐led care, 35% reductionb ‐ Surgery visits: 10% fewer with nurse‐led careb ‐ Home visits: 6% fewer home visits during intervention periodb		Hospital admission within 24 hours: nurse 2%, GP 6.5%, RR 0.31 (95% CI 0.07 to 1.42) Hospital admission within 3 days: nurse 5%, GP 6.5%, RR 0.77 (95% CI 0.26 to 2.28) Emergency department visit: nurse 3%, GP 2%, RR 1.84 (95% CI 0.31 to 10.82)
Lewis 1967	Consultation length: doctor 15 minutes, nurse 30 minutesc Consultation rate: doctor 150 visits, nurse 345 visitsc		Days in hospital: doctor 68 days, nurse 45 daysc
Moher 2001		Prescriptions: ‐ Antihypertensives: no difference, P = 0.35a ‐ Lipid lowering: no difference, P = 0.63a ‐ Antiplatelet: nurse 8% (95% CI 1% to 9%) more than GP (P = 0.031)
Mundinger 2000	Consultation rate: Doctor patients had higher primary care utilisation than nurse practitioner patients (2.50 vs 1.76 visits, P = 0.05)		Speciality visits: no differenced, P = 0.61
Ndosi 2013	Consultation length: Mean total consultation time: nurse 111 min, doctor 71 mina,b Consultation rate: Patients attending all 5 sessions: nurse 92%, doctor 85%a,b	‐ Change in medicines: nurse < doctor; RR (95% CI) 0.58 (0.43 to 0.79), P = 0.0006 ‐ Dosage changes: nurse < doctor; RR (95% CI) 0.52 (0.34 to 0.79), P = 0.0020 ‐ Intra‐articular injections: nurse < doctor; RR (95% CI) 0.82 (0.50 to 1.35), P = 0.4400 ‐ Intramusclar injections: nurse < doctor; RR (95% CI) 0.73 (0.45 to 1.19), P = 0.2100 ‐ Non‐protocol bloods: nurse < doctor; RR (95% CI) 1.02 (0.74 to 1.40), P = 0.9100	‐ Referral to physiotherapy: nurse < doctor; RR (95% CI) 1.21 (0.62 to 2.39), P = 0.5800 ‐ Referral to occupational therapy: nurse < doctor; RR (95% CI) 1.74 (0.76 to 3.96), P = 0.1900 ‐ Referral to podiatry: nurse < doctor; RR (95% CI) 0.89 (0.37 to 2.14), P = 0.8000 ‐ Conferrals: nurse < doctor; RR (95% CI) 2.92 (1.77 to 4.83), P < 0.0001 ‐ Referral to other consultants: nurse < doctor; RR (95% CI) 0.58 (0.11 to 3.11), P = 0.5200
Shum 2000			Out‐of‐hours calls: nurse 0.9% vs GP 1.8%, P = 0.218
Venning 2000		Physical examinations: nurse vs GP; MD 0.19 (95% CI ‐0.03 to 0.71), P = 0.072
Voogdt‐Pruis 2010			Referred to professional nurse vs doctor: ‐ Dietician 17.0 vs 8.9b ‐ Physiotherapist 3.1 vs 1.9b ‐ Cardiovascular specialist 1.9 vs 6.3b ‐ Visited a cardiovascular specialist 46.3 vs 45.3 (0.84 to 1.79; P = 0.30) ‐ Admission into hospital because of CVD 10.4 vs 13.4 (0.43 to 1.38; P = 0.38)

^aTrial authors reported no effect size or reported effect sizes in graphs (no exact effect sizes extracted).

^bNo P value reported.

^cTrial authors reported only the direction of the outcome; it remains unknown whether the difference is statistically significant.

^dToo many numbers to report.

CI: confidence interval.

CT: computed tomography.

CVD: cardiovascular disease.

GP: general practitioner.

MRI: magnetic resonance imaging.

NP: nurse practitioner.

RR: risk ratio.

*there may be additional data in the Campbell 2014 articles that have not been extracted

6. Utilisation; cost outcomes.

Study	Costs based on	Cost outcomes
Campbell 2014	Staff training Setup of the interventions Cost of computer decision support software in nurse triage Clinician triage time Patient‐level quantities of resource use on other primary care contacts	Total costs: Mean 28‐day cost estimates for primary outcome contacts: Nurses ‐ £75·68 (63·09) GPs ‐ £75·21 (65·45)
Chan 2009	Medication use	Costs of medication use: Nurses – mean £35.5 (SD £48.8) Doctors – mean £71.7 (SD £ 63.1) Mean difference (adjusted baseline level): £39.6 (95% CI 24.2 to 55.1); P < 0.001
Dierick‐van Daele 2009	Direct healthcare costs Prescriptions Diagnostic procedures Referrals (in the 2 weeks after consultation) Follow‐up consultation Length of consultations Salary costs Costs outside the healthcare sector Sick leave days	Total direct healthcare costs: Nurses: €31.94 Doctors: €40.15 Mean difference (95% CI): €8.21 (3.56 to 12.85); P = 0.001 Total direct healthcare costs and productivity: Nurses: €140.40 Doctors: €145.87 Mean difference (95% CI): €1.48 (‐4.94 to 7.90); P = 0.65 Subgroup younger than 65 years: Total direct healthcare costs and productivity: Nurses: €161.57 Doctors: €170.75 Mean difference (95% CI): €9.18 (4.84 to 13.88); P < 0.001
Larsson 2014	Fixed monitoring (monitoring visit at 6 months to a rheumatology nurse, a rheumatologist; for both groups, a monitoring visit at 12 months to a rheumatologist and monitoring blood tests) Variable monitoring (additional telephone calls to a rheumatology nurse, additional telephone calls to a rheumatologist (additional rheumatologist visits, cortisone injections in addition to regular rheumatologist monitoring visits, and additional blood tests)) Rehabilitation (team rehabilitation days of care in in‐patient and out‐patient settings, individual physiotherapy treatments, occupational therapist treatments, and psychosocial treatments) Specialist consultations (orthopaedic surgeon, hand surgeon, dermatologist, and orthotist) Radiography (standard x‐ray and dual energy x‐ray absorptiometry (DEXA) scanning) Pharmacological therapy	Total annual rheumatology care per patient: Nurse‐led: €14107,70 Doctor‐led: €16274,90 Mean difference (95% CI): −2167.2 (−3757.3 to −641.7) P = 0.004
Lattimer 1998	Costs for nurse telephone consultation Recruitment Nurse salaries Indemnity insurance Co‐operative management Education programme 1 H grade – 0.25 whole time equivalent 10 days lecturer B Technical support Computers Decision support software Furniture Telephones Digital tape recorder Savings Emergency hospital admission Home visits by general practitioner Surgery attendance within 3 days	Annual direct cost:Nurse‐led service: ‐ £81,237 more than doctor‐led service Savings: Generated in reduced hospital and primary care utilisation £94,422 Net reduction in costs:with nurse‐led service £3,728 to £123,824 (determined by sensitivity analysis)
Lewis 1967	Cost per hour of the time of doctors and nurses Length of visits Total number of visits Total days of in‐patient care Unknown other costs	Total direct cost per year: Nurses ‐ $3,251 Doctors ‐ $4,199 Average cost per patient per year: Nurses ‐ $98.51 Doctors ‐ $127.24
Ndosi 2013	Resource use Healthcare professional consultations (primary and secondary care) Hospital admissions (day care, in‐patient stays, A&E visits) Investigations and treatments including over‐the‐counter medications Private out‐of‐pocket expenditures Healthcare service use Travel Medication Aids Special dietary requirements Productivity losses	NHS resources plus out‐of‐pocket expenditures: Nurses ‐ mean £1276 Doctors ‐ mean £2286 (95% CI ‐352 to 1773) P = 0.1872
Spitzer 1973a	Doctors Nurses (including nurse practitioners) Hospital and extended care Dentists Optometrists/Opticians Chiropractors Podiatrists Laboratory Diagnostic radiography Direct cash expenditures	Average cost per patient per year: Nurses ‐ $297.01 Doctors – $285.67
Venning 2000	Basic salary costs of each healthcare professional Prescriptions Tests Referrals Return consultations in the following 2 weeks	Total direct cost per consultation: Nurses – mean £18.11 (SD £33.43; range £0.66 to £297.1) Doctors – mean £20.70 (SD £33.43; range £0.78 to £300.6) Mean difference (adjusted age, sex): £2.33 (95% CI 1.62 to 6.28); P = 0.247

^aSpitzer reported an overall reduction in practice costs following the introduction of nurse practitioners, but this finding was based on observational before‐and‐after data. Data obtained from the related randomised controlled trial (reported above) did not support this finding.

A&E: accident and emergency.

CI: confidence interval.

DEXA: dual energy x‐ray absorptiometry.

GP: general practitioner.

*there may be additional data in the Campbell 2014 articles that have not been extracted

Consultations

Seven trials investigated consultation length (Dierick‐van Daele 2009; Houweling 2011; Iglesias 2013; Lewis 1967; Ndosi 2013; Shum 2000; Venning 2000). Four trials provided sufficient data for a meta‐analysis on consultation length (Dierick‐van Daele 2009; Iglesias 2013; Shum 2000; Venning 2000). This analysis suggests that nurses probably have longer consultations than doctors (SMD 0.38, 95% CI 0.22 to 0.54, moderate‐certainty evidence; Analysis 1.11). The evidence is of moderate certainty owing to serious inconsistency (I² = 90%, 95% CI 80 to 95). The extent of heterogeneity suggests that differences in consultation length varied considerably with the context of care. On average, consultations with nurses were 39% (95% CI 30% to 52%) longer than those with doctors. Results did not change considerably under Sensitivity analysis. Findings of the trials not included in the meta‐analysis also suggest that consultations in nurse‐led care were probably longer than those in doctor‐led care.

1.11. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 11 Length of consultation.

Nine trials investigated consultation rates in primary care (including overall consultation rates, return visits for whatever reason, and home visits) (Dierick‐van Daele 2009; Hemani 1999; Houweling 2011; Iglesias 2013; Lewis 1967; Mundinger 2000; Ndosi 2013; Shum 2000; Venning 2000). Three trials provided sufficient data for a meta‐analysis of scheduled return visits (Dierick‐van Daele 2009; Shum 2000; Venning 2000), and four trials on attended return visits (Dierick‐van Daele 2009; Iglesias 2013; Shum 2000; Venning 2000). Data show that there may be little or no difference in scheduled return visits (RR 1.31, 95% CI 0.89 to 1.94, low‐certainty evidence; Analysis 1.12). The number of attended return visits is higher in nurse‐led primary care than in doctor‐led primary care (RR 1.19, 95% CI 1.07 to 1.33, high‐certainty evidence; Analysis 1.13). For scheduled return visits, the evidence is of low certainty owing to serious inconsistency (I² = 86%, 95% CI 54 to 92) and imprecision (wide confidence interval). Results did not change considerably under Sensitivity analysis.

1.12. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 12 Scheduled return visits.

1.13. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 13 Attended return visit.

Findings of the trials not included in the meta‐analysis were congruent with those reported above . Furthermore, the workload of doctors was probably slightly less where care was led by nurses, compared to where it was led by doctors, as were waiting times for patients in the waiting room. Table 6 summarises the data for all trials that assessed this outcome.

Numbers of prescriptions, tests, and investigations

Seven trials evaluated rates of prescriptions, tests, and investigations (Dierick‐van Daele 2009; Hemani 1999; Iglesias 2013; Moher 2001; Shum 2000; Venning 2000; Ndosi 2013). Four trials provided sufficient data for a meta‐analysis on the number of prescriptions given (Dierick‐van Daele 2009; Iglesias 2013; Shum 2000; Venning 2000), and four trials on the number of tests and investigations (Dierick‐van Daele 2009; Hemani 1999; Venning 2000; Ndosi 2013). Meta‐analyses of data from these trials suggest little or no difference between nurse‐led care and doctor‐led care in the number of prescriptions given (RR 0.99, 95% CI 0.95 to 1.03, high‐certainty evidence; Analysis 1.14).

1.14. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 14 Prescription ordered.

The findings also show that there may be little or no difference in the number of tests/investigations (RR 0.95, 95% CI 0.59 to 1.51, low‐certainty evidence; Analysis 1.15). The evidence is of low certainty owing to serious inconsistency (I² = 76%, 95% CI 23 to 86) and a wide confidence interval, suggesting that the number of ordering tests/investigations varied between nurse‐led care and doctor‐led care according to the context of care.

1.15. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 15 Investigations.

The findings of trials not included in the meta‐analyses also suggest little or no difference between nurse‐led and doctor‐led care in numbers of prescriptions and investigations/tests. Table 6 summarises the data for all trials that assessed this outcome.

Use of other services

Thirteen trials investigated people's use of services, including referrals, specialty visits, and hospital admissions (Campbell 2014; Dierick‐van Daele 2009; Hemani 1999; Houweling 2011; Iglesias 2013; Larsson 2014; Lattimer 1998; Lewis 1967; Mundinger 2000; Ndosi 2013; Shum 2000; Venning 2000; Voogdt‐Pruis 2010). Of these trials, four provided sufficient data for a meta‐analysis on hospital referral (Houweling 2011; Lattimer 1998; Mundinger 2000; Venning 2000), five for a meta‐analysis on attendance at accident and emergency units (Campbell 2014; Iglesias 2013; Lattimer 1998; Mundinger 2000; Shum 2000), and three for a meta‐analysis on hospital admission (Lattimer 1998; Mundinger 2000; Ndosi 2013). These meta‐analyses suggest that there may be little or no difference between nurse‐led care and doctor‐led care in the likelihood of hospital referrals (RR 0.90, 95% CI 0.54 to 1.49, low‐certainty evidence; Analysis 1.16), as well as little or no difference in attendance at accident and emergency units (RR 1.00, 95% CI 0.91 to 1.09, high‐certainty evidence; Analysis 1.17). In addition, there may be little or no difference in hospital admissions (RR 1.04, 95% CI 0.78 to 1.39, low‐certainty evidence; Analysis 1.18). For referrals, evidence is of low certainty owing to inconsistency (I² = 50%, 95% CI 0 to 86) and a wide confidence interval, suggesting that the extent to which the frequency of referrals differs between nurse‐led care and doctor‐led care varied with the context of care. For hospital admissions, the evidence is of low certainty owing to risk of bias in one of the included trials (Mundinger 2000) and a wide confidence interval. Results did not change considerably in the Sensitivity analysis.

1.16. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 16 Hospital referral.

1.17. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 17 Attendance at accident and emergency.

1.18. Analysis.

Comparison 1 Doctor‐nurse substitution study results, Outcome 18 Hospital admission.

Findings of the trials not included in the meta‐analyses also suggest little or no difference between nurse‐led and doctor‐led care on use of other services. Table 6 summarises the data for all trials that assessed this outcome.

Costs

Nine trials investigated the cost of care (Campbell 2014; Chambers 1978; Chan 2009; Dierick‐van Daele 2009; Lattimer 1998; Lewis 1967; Ndosi 2013; Spitzer 1973; Venning 2000) (Table 7). Three trials estimated cost of care for nurses providing first contact care (Dierick‐van Daele 2009; Lattimer 1998; Venning 2000), two trials for nurses providing first contact and ongoing care (Chambers 1978; Spitzer 1973), and three trials for nurses providing ongoing care for patients with chronic disease (Chan 2009; Lewis 1967; Ndosi 2013). Individually, the trials appear to suggest little or no difference in cost of care between nurse‐led care and doctor‐led care. However, owing to the large variety of approaches used to value resources and calculate costs, we judged the results to be non‐comparable and did not pool these in a meta‐analysis. We therefore assessed the certainty of the evidence as very low.

Discussion

Summary of main results

This review identified 18 randomised trials evaluating the impact of nurses working as doctors’ substitutes. One study was from a middle‐income country, and all of the other studies were from high‐income countries. The type of nursing cadres involved in the studies was often unclear or varied between and even within studies.

Findings suggest that care delivered by nurses, compared to care delivered by doctors, probably leads to similar or better health outcomes for a broad range of patient conditions (moderate‐certainty evidence).

• Nurse‐led primary care may lead to slightly fewer deaths among certain groups of patients, compared to doctor‐led care. However, the results vary and it is possible that nurse‐led primary care makes little or no difference to the number of deaths.

• Blood pressure outcomes are probably slightly improved in nurse‐led primary care. Other clinical or health status outcomes are probably similar.

• Patient satisfaction is probably slightly higher in nurse‐led primary care. Quality of life may be slightly higher.

We are uncertain of the effects of nurse‐led care, compared to doctor‐led care, on processes of care such as patient education and adherence to guidelines. The effect of nurse‐led care on utilisation is mixed and depends on the type of outcome. Consultations are probably longer in nurse‐led primary care, and numbers of attended return visits are slightly higher for nurses than for doctors. We found little or no difference between nurses and doctors in the number of prescriptions and attendance at accident and emergency units. There may also be little or no difference in the number of tests and investigations, hospital referrals and hospital admissions between nurses and doctors. We are uncertain of the effects of nurse‐led care on the cost of care because the certainty of this evidence was assessed as very low.

An overview can be found in Table 1.

Overall completeness and applicability of evidence

Several issues need to be considered when one is making judgements about the applicability of these findings in primary care systems. First, we were able to identify a large number of studies published up to March 2017, which were sufficient to address all objectives of the review. These studies are highly varied in terms of types of nurses (with regard to both educational level and nurses' roles), healthcare systems, and geographical settings, and they examine care provided to general patient populations as well as to specific groups of patients, such as people with cardiovascular disease, diabetes and rheumatological diseases. Next, we found a large variation in outcome measures. For a number of outcomes there were only a few contributing studies whereas for some other outcomes a relative large number of studies contributed to the evidence. Furthermore, often details (such as nursing education level) were missing from study reports. Therefore we were not able to conduct planned subgroup analyses. As a result, it is not possible to draw conclusions on the influence of nurse type on outcomes. In addition, all but one of the included studies were conducted in high‐income country settings. Second, in some studies, interventions in nurse‐led and doctor‐led groups were somewhat different. For example, nurses had protocols or were offered a computerised decision tool, and doctors were not (Campbell 2014; Houweling 2011; Iglesias 2013; Lattimer 1998). In other studies, nurses’ patients were given an appointment but doctors’ patients were only advised to see their doctor (Chan 2009), or nurse‐led care included a longer time slot for consultations (Ndosi 2013). These differences in the interventions provided might have influenced study outcomes. Last, over the ten years since our previous review was published, primary care services have changed considerably in many settings. However, we did not identify a trend in types of nurse substitutions for doctors or in changes in outcomes assessed that might reflect changes in primary care services. The reasons for this are not clear.

Quality of the evidence

This review included studies from a wide range of nursing levels, patient groups, and countries. We were able to identify evidence on many different outcomes, but certainty of this evidence varies. All studies had some methodological shortcomings, such as contamination and lack of blinding both patients and personnel, which sometimes led to downgrading of the evidence owing to risk of bias. Although lack of blinding is considered a shortcoming, blinding is often not possible for organizational interventions, such as the substitution of one kind of health care provider with another. While the impact of this on outcomes is unclear, we believe the impact on patient satisfaction and experiences with care is likely to be limited. For example, a Dutch study which evaluated the impact of nurse practitioners in acute primary care settings showed that patients often do not know the profession of their care provider, even when the care provider had introduced themselves at the start of the consultation (only 18% of the patients treated by nurse practitioners remembered this) (Wijers 2013). The study suggests that patients do not judge the practitioner on the basis of their profession but rather on their competencies. Not blinding personnel may affect collaboration, as acceptance of a new professional is one of the main factors influencing skill mix changes (Laurant 2009). This influence could go either way: when a new professional is first introduced, other professionals and patients may be uncertain about the competencies of the new professional. However, over time the new professional may be accepted by both patients and other professionals. This has been shown in a recent study in which patients were more satisfied with the care provided by nurses over time (Wijers 2013). It is unclear whether this effect might also impact on patient outcomes, process of care outcomes and resource utilization.

For many meta‐analyses, measures of statistical heterogeneity were high indicating inconsistency across the included trials. Even where statistical heterogeneity was not detected, clinical heterogeneity may be present due to the range of types of nurses, health issues and settings included in the review (Kontopantelis 2013). Wide confidence intervals (imprecision) were another common reason to downgrade.

For some studies and outcomes, we were not able to conduct meta‐analyses owing to the diversity of the outcomes assessed. For these studies, we could only describe the results narratively, which made drawing overall conclusions difficult.

Potential biases in the review process

Our search strategy was designed to maximise sensitivity (detecting relevant research) at the expense of specificity (excluding irrelevant research). Even so, relevant research proved difficult to identify, and some studies may have been missed.

We conducted this review according to Cochrane standards. Therefore, we are confident in the quality of the review itself. Although publication bias cannot be ruled out in this area (Egger 1997), it seems unlikely that this bias could be substantial, as the clinical and research communities are equally interested in whether nurses perform as well as or better than doctors, or the reverse.

Agreements and disagreements with other studies or reviews

Results of this update are similar to those of the original review (Laurant 2005) in terms of health outcomes for patients, process of care, and resource utilisation.

Several other published reviews have examined nurses in primary care (Bonsall 2008; Hollinghurst 2006; Horrocks 2002; Martin‐Misener 2015; Martínez‐González 2014a; Martínez‐González 2014b; Martínez‐González 2015a; Martínez‐González 2015b; Martínez‐González 2015c; Naylor 2010; Newhouse 2011; Swan 2015). Although the findings of our current review are generally consistent with those of other reviews, differences in findings might be explained by differences in review methods. Our review is most closely related to the reviews of Martínez‐González et al. (Martínez‐González 2014a; Martínez‐González 2014b; Martínez‐González 2015a; Martínez‐González 2015b; Martínez‐González 2015c). Although those review authors used inclusion criteria similar to ours, we noted differences in the included studies. There are several possible explanations for these differences. Firstly, there may be differences in the way the type of nurse role revision was labelled, and specifically whether this change was assessed as substitution or supplementation. Other systematic reviews included some studies that we assessed as involving nurses supplementing care provided by doctors rather than taking over tasks of doctors through substitution (Andryukhin 2010; Denver 2003; Du Moulin 2007; Fairall 2012; Hesselink 2004; Hiss 2007; Jarman 2002; Kernick 2000; Kernick 2002; Kuethe 2011; Winter 1981). In these supplementation studies, the intervention nurses provided an intervention that was complementary to usual care or both nurses and physicians were both involved in patient care, providing care as a team rather than providing care separately . Secondly, we included several studies that were not included in the reviews of the Martínez‐González team (Chambers 1978; Lattimer 1998; Moher 2001; Sanne 2010; Spitzer 1973). Only one of these five studies (Spitzer 1973) was listed in the table presenting the reasons for exclusions. Martinez‐Gonzalez team argued this study was not real substitution due to the contamination that occurred during the trial, with 30% of patients also treated by physicians at the end of the study. We judged contamination as high risk in this study, but not as a reason to exclude the study. The other four studies (Chambers 1978; Lattimer 1998; Moher 2001; Sanne 2010) were not included in the full text screening of papers by Martinez‐Gonsalez. It is therefore unclear whether these papers were not identified due to differences in search strategies or whether these papers were excluded at the title and abstract screening stage of the Martinez‐Gonsalez reviews. We have presented in Table 8 the key methodological differences between our review and the other reviews mentioned here.

7. Methodological differences with published reviews on care delivered by nurses compared to doctors in primary care.

Focus of other reviews	Differences from our review
	Does not include meta‐analyses	Includes nurses working as supplements according to our definition	Includes non‐randomised studies	Focusses on particular countries	Has a particular focus on cost outcomes
Bonsall 2008 This literature review assesses the impact of advanced primary care nursing roles, particularly first contact nursing roles, for patients, nurses themselves, and their colleagues.	x	x	x
Hollinghurst 2006 This study used the literature search Horrocks 2002 and aims to estimate resource use for a typical same‐day primary care consultation and the cost difference of employing an extra salaried GP or nurse practitioner.	x			UK	x
Horrocks 2002 This systematic review compares effects of nurse practitioners and doctors providing care at first point on patient satisfaction, health status, process measures, and quality of care.		x	x	Developed countries
Martínez‐González 2014a; Martínez‐González 2014b; Martínez‐González 2015a; Martínez‐González 2015b; Martínez‐González 2015c Several systematic reviews investigating effects of nurses working as substitutes for doctors in primary care on clinical effectiveness, course of disease, process care, resource utilisation, and costs.		x
Martin‐Misener 2015 This systematic review determines the cost‐effectiveness of nurse practitioners delivering primary and specialised ambulatory care.		x			x
Naylor 2010 This structured literature review investigates the value of advance practice nurses in delivering primary care, with a particular emphasis on the contributions of nurse practitioners.	x	x	x
Newhouse 2011 This systematic reviews compares patient outcomes of care by advanced practice registered nurses (APRNs) to care by other providers (doctors or teams without APRNs).	x	x	x	USA
Swan 2015 This systematic review includes 10 studies evaluating the cost and quality of care provided by APRNs in primary care.	x

APRN: advanced practice registered nurse.

Several reviews found similar results to ours in terms of reductions in mortality in nurse‐led primary care compared to doctor‐led primary care, in particular in ongoing care and non‐urgent care provided by nurse practitioners (Martínez‐González 2014a; Swan 2015). However, one review (Newhouse 2011) showed that mortality rates were similar across these cadres, possibly owing to differences in review inclusion criteria. All other reviews described results similar to ours in terms of equal or higher health status for patients who received care from nurses compared to doctors (Martínez‐González 2015a; Newhouse 2011; Swan 2015).

Other reviews also found that nurse‐led care probably leads to higher patient satisfaction (Horrocks 2002; Martin‐Misener 2015; Martínez‐González 2014a; Swan 2015 ); slightly higher quality of life (Martínez‐González 2014a); longer consultation length; and higher rate of return visits (Hollinghurst 2006; Horrocks 2002; Martin‐Misener 2015; Martínez‐González 2015b), compared to doctor‐led care. Our finding that there is little or no difference between nurses and doctors in frequency of prescriptions, tests, and investigations, and in patients’ use of other services is similar to that of the Martínez‐González 2015b review. However, Horrocks 2002 found that nurses‐led care was associated with more investigations but an equal number of prescriptions. It is likely, though, that the findings of the Horrocks 2002 review are now out of date.

While we were uncertain of the effects of nurse‐led care on the cost of care most other reviews (Hollinghurst 2006; Martin‐Misener 2015; Martínez‐González 2015b; Swan 2015)reported that there may be little or no difference in costs of care between nurse‐led care and doctor‐led care. Naylor 2010 and Newhouse 2011 indicated that nurse care was associated with lower costs. This difference might be explained by a focus on the USA only (Newhouse 2011), by the inclusion of non‐randomised trials, and by a focus on advanced nurses and nurse practitioners (Naylor 2010; Newhouse 2011). Authors of all reviews agree that evidence of effects of nurse‐led care on costs of care is of low quality.

Authors' conclusions

Implications for practice.

Overall, nurse‐doctor substitution in primary care for provision of first contact care (including urgent care), ongoing care for all presenting physical complaints, and follow‐up of patients with a particular chronic condition has been relatively well evaluated. Nurse‐doctor substitution for preventive services and health education in primary care has been less well studied.

This review shows that trained nurses, such as nurse practitioners, practice nurses, and registered nurses, probably provide care that is equal to or of better quality than that provided by primary care doctors, and probably achieves equal or better health outcomes for patients. Nurses probably provide more health advice to patients, and probably achieve slightly higher levels of patient satisfaction, compared to primary care doctors (Table 1).

From this review, we cannot conclude whether it is better to deploy nurses providing care for a broad range of health issues or nurses who target groups of patients. Both approaches seem possible, with at least equal quality of care. Futhermore, the authors of this review cannot draw conclusions on the level of nursing education that leads to the best outcomes when nurses are substituted for doctors. In our review, the educational level of nurses was often unknown. In addition, studies often included a range of nurse roles and types, so we were not able to explore within our review whether evidence shows differences by type of nurse or by nursing role (Maier 2016b).

Whether nurse‐doctor substitution leads to substantial savings or whether nurse‐doctor substitution is cost‐effective remains unclear. Savings on nurse salaries may be offset by nurses' longer consultations and nurse rates as compared to doctor rates (Table 1). On the other hand, nurses probably adhere better to guideline recommendations, and their patients are probably more likely to attend return visits, which may positively affect health outcomes and reduce costs over the medium to long term.

Our review focussed on differences in outcomes between care provided by nurses and care provided by doctors. Although the included studies show effects of an independent practice role for nurses, it is likely that the quality of patient care overall is determined by overall functioning of the primary care team, including nurses, doctors, and other healthcare providers. Only three studies in our review assessed the impact of nurses on doctor behaviour. Policy makers should be aware that implementing nurse substitution in primary care teams may have an influence on the functioning and quality of care delivered by the entire care team.

Implications for research.

Although this review includes a large number of studies, several important research questions remain.

The methodological quality of recent included studies is still variable. Future studies should seek to maximise the numbers of included healthcare providers, rather than the numbers of patients, to reduce the effect of any individual provider on outcomes. Moreover, studies with longer follow‐up periods are needed to provide better insights into impact on health status. For a full understanding of the impact of nurses in primary healthcare teams, we need deeper insights into the functioning of the entire team. Qualitative studies may be useful and could explore questions such as how nurses and doctors work as a team, how they interact, how their roles and responsibilities are defined, and how these agreements on roles and responsibilities affect nurse and doctor behaviours (Rashidian 2013). Further research is needed to enhance understanding of the limits of substitution, and to explore optimal models of collaboration and deployment of doctors and nurses as part of primary healthcare teams.

All studies except one were conducted in high‐income countries, and it is not clear whether results from this single study can be generalised to populations and health systems in middle‐ or low‐income countries. More research in middle‐ and low‐income countries is needed. Moreover, the influence of nursing education level on effects of nurse‐doctor substitution is poorly understood. Reasons include lack of international educational standardisation and insufficient reporting of nursing levels in research papers.

Since mortality is very important outcome, and the results of this review show important impacts, with mortality probably being decreased in nurse‐led primary care, this outcome should be assessed in future studies.

Costs, particularly societal costs, have not been well investigated, despite the widely held view that nurse‐led care will generate savings. Most studies have major limitations in cost evaluation. Future studies of nurse‐doctor substitution should give more attention to its financial aspects, for example, by performing cost‐effectiveness analyses. Related to this is the question of what impact changes in nurses' work have on the behaviours of doctors and on their workload. Only three of the included studies evaluated this, despite the widely held view that nurses can ‘save’ doctors’ time.

Authors of future reviews about nurse‐doctor substitution must take into account that healthcare services change extensively over time, and that new treatments and innovations may affect healthcare delivery. Organisational changes such as nurse‐doctor substitution are complex and should be treated in a way that leads to a well‐informed understanding of mechanisms and how these may impact outcomes (Salter and Kothari 2014).

Feedback

Comments submitted via Cochrane Library, 17 July 2018

Summary

Comment #1: I have never witnessed so many "probablies" in the conclusion of a cochrane review...These are probably conclusions. There is probably some bias. Cochrane, you need to do better.

Comment #2: It was probably written by nurses... I see an OBgyn.. not sure of their validity in this..

Reply

Response to Comment #1: We are aware of the many ‘probablies’ in the conclusion of our review. This is the result of GRADE assessments of the certainty of the evidence for the outcomes included in the review. For example, we had to downgrade the certainty of the evidence in some instances due to flaws in how the included studies were conducted (i.e., risk of bias). In line with Cochrane EPOC guidance on reporting the effects of an intervention (see: https://epoc.cochrane.org/sites/epoc.cochrane.org/files/public/uploads/Resources‐for‐authors2017/how_to_report_the_effects_of_an_intervention.pdf), we used standardized statements to indicate the certainty of the evidence to readers. In this approach, ‘probably’ is used to flag moderate certainty evidence of effect while ‘may’ is used to indicate low certainty evidence of effect. As the evidence for a number of outcomes in this review was graded as moderate certainty, the term ‘probably’ was used frequently.

Response to Comment #2: The review is written by health research scientists, with different specializations in epidemiology and health promotion sciences. Four authors (ML, MvdB, NW, AvV) have their roots in nursing (bachelor nursing), but have not been in practice for a long time. One author is a medical doctor (KW) and one author (EK) is a statistician.

Contributors

Comments submitted by: Sandra Allison

Response to comments from: Miranda Laurant and Anneke van Vught on behalf of all authors

Comments submitted via Cochrane Library, 27 July 2018

Summary

Comment #1:

I have issue with the inclusion of several articles in this review. Specifically, multiple articles reference specific populations (e.g. those receiving anti‐retrovirals, those who have rheumatologic conditions). I do not think the experience of these patients can be generalized to the whole primary care population

Comment #2:

Multiple studies are more than 18 years old ‐ some are 45 years old. I do not think the care and experience of care from 45 yrs ago compares to the standards of care today.

Comment #3:

Some studies reference nurses and some reference nurse practitioners, who have very different scopes of practice and training.

Comment #4:

I have respect and admiration for my nursing and nurse practitioner colleagues and am heartened to hear that patients feel well taken care of and are well taken care of in primary care settings. I'm concerned that this article could lead health policy specialists and governments to believe that a nurse is equal to or better than a physician at providing all kinds of primary care.

Family physician training (in Canada) has emphasized patient oriented, full person care in a biopsychosocial model. The old school model of the "general practitioner" has been replaced by a minimum two year post‐graduate training degree in "Family Medicine". I'm concerned that this review doesn't reflect the intricacies and value of my expertise as a generalist Family Physician, as it lumps the care of specialized populations and/or populations from a different era with the population presenting for primary care today

Reply

Response to Comment #1:

The criteria for considering studies for this review focused on primary healthcare service settings that provide first contact and ongoing care for patients with all types of physical health problems. Whether or not the patient population in a particular trial was representative of the whole primary care population was not a selection criteria.

In response to your comments, we once again critically examined whether all of our included studies met our inclusion criteria. We have now decided to exclude two studies that on closer inspection do not appear to have been conducted in primary care settings. These studies focus on care for patients with rheumatologic conditions (i.e. Ndosi 2013 and Larsson 2014). The review will be amended by excluding these two studies and a minor update published in the second quarter of 2019.

The other studies that focus on specific populations of service users are clearly studies undertaken in primary care. In the next update of this review, we will consider in the analyses and interpretation whether the nurses in the included studies provided more generalist or more specific types of care, and try to explore the applicability of the results to these different types of care.

Response to Comment #2:

Year of study/publication was not an exclusion criteria for this review, although we have considered this and will do so again when the next update of the review is undertaken. We decided not to exclude studies on the basis of year of study/publication as primary healthcare is organized very differently worldwide and even though some studies were conducted some time ago, their results may still be relevant to some primary healthcare settings practices today. It is also worth noting that most of the included studies were published from 2010 onwards. In future updates, we will consider the age of the studies when discussing the interpretation of the results

Response to Comment #3:

We agree that the training of nurses, registered nurses, practice nurses and nurse practitioners is different worldwide and, in addition, the terms used to describe nurses with different types of training are not consistent from setting to setting. Indeed, there is considerable variations between, and sometimes within, countries regarding the training and role of nurses who use the title ‘Nurse Practitioners’ (see, amongst others, Maier 2015¹ and Maier et al, 2016²). Further, it is often difficult to assess the training received by nurses in the studies included in the review.

In this review, however, we were interested in situations in which task(s) formerly performed by physicians were transferred to nurses (i.e. substitution of care), regardless of the training or scope of practice of these nurses. We planned a subgroup analyses based on the level of nursing education to explore the issue that you raise. However, we were not able to perform this due to inadequate data, so it remains uncertain how level of nursing education impacts on outcomes when nurses are substituted for doctors in primary care.

¹Maier CB. The role of governance in implementing task‐shifting from physicians to nurses in advanced roles in Europe, U.S., Canada, New Zealand and Australia. Health Policy. 2015; 119: (12), 1627‐1635.

²Maier CB, Barnes H, Aiken LH, Busse R. Descriptive, cross‐country analysis of nurse practitioners in size countries: size, growth, physician substitution potential. BMJ Open. 2016; 6 (9): e011901.

Response to Comment #4:

We understand your concerns and, like all Cochrane reviews, this review does not make any recommendations regarding the replacement of primary healthcare doctors with nurses. Such decisions need to be taken within a specific context, and will involve a range of other evidence and information in addition to evidence of the effectiveness of nurses as substitutes for doctors in primary care. Overall, the review shows that ‘trained nurses, such as nurse practitioners, practice nurses, and registered nurses, probably provide care that is equal to or of better quality than that provided by primary care doctors, and probably achieves equal or better health outcomes for patients.’ However, we also note that we cannot conclude whether it is better to deploy nurses providing care for a broad range of health issues or nurses who target groups of patients, and that we cannot draw conclusions on the level of nursing education that leads to the best outcomes when nurses are substituted for doctors. In addition, the certainty of the available evidence varies – the likelihood that the effect will be substantially different from that found in the review is moderate or high for some findings. Nonetheless, the outcomes all pointed in the same direction, towards at least equal care and patient outcomes when nurses are substituted for doctors.

We would suggest that any new models of primary healthcare, including nurses in independent practice roles or substituting for physicians in other ways, should be considered by policy makers in dialogue with doctors, nurses, and other professionals, patients and the public to ensure the quality of, and access to, primary healthcare and to ensure that these models are acceptable and feasible.

Contributors

Comments submitted by: Oona Hayes

Response to comments from: Miranda Laurant and Anneke van Vught on behalf of all authors

What's new

Date	Event	Description
1 February 2019	Amended	Response to comments submitted via Cochrane Library.

History

Protocol first published: Issue 4, 1998
 Review first published: Issue 2, 2005

Date	Event	Description
26 April 2017	New citation required but conclusions have not changed	This update includes nine new studies. The total number of included studies in the review is now 18. New review authors have contributed to this update. We have excluded from this update seven studies previously included in the review (3 controlled before‐after studies, 3 non‐randomised trials, and 1 study focussing on mental health problems).
26 April 2017	New search has been performed	This is the first update of the Cochrane review published in 2005. We have conducted a new search and have updated other content. We updated the search in March 2017 and added one trial report to ‘Studies awaiting classification’.
16 July 2004	New citation required and conclusions have changed	We have made substantive amendments.

Notes

This is an update of the review “Substitution of doctors by nurses in primary care”, which was first published in 2005 for the Cochrane Library (Laurant 2005).

Appendices

Appendix 1. Search strategies 2015

CENTRAL, the Cochrane Library (searched 2017)

ID	Search	Hits
#1	MeSH descriptor: [Nurses] explode all trees	1141
#2	MeSH descriptor: [Midwifery] this term only	322
#3	(nurse or nurses or midwife or midwives):ti,ab	12380
#4	#1 or #2 or #3	12750
#5	MeSH descriptor: [Physicians] this term only	781
#6	MeSH descriptor: [General Practitioners] explode all trees	176
#7	MeSH descriptor: [Physicians, Family] this term only	479
#8	MeSH descriptor: [Physicians, Primary Care] this term only	127
#9	(physician* or doctor or doctors or general next practitioner* or GP or GPs or family next practitioner* or "conventional care" or "usual care" or treatment near/1 usual):ti,ab	37835
#10	#5 or #6 or #7 or #8 or #9	38075
#11	MeSH descriptor: [Primary Health Care] this term only	3964
#12	MeSH descriptor: [Family Practice] explode all trees	2190
#13	MeSH descriptor: [Ambulatory Care] this term only	3294
#14	MeSH descriptor: [Ambulatory Care Facilities] explode all trees	1873
#15	MeSH descriptor: [Community Health Services] this term only	1056
#16	MeSH descriptor: [Community Medicine] this term only	46
#17	MeSH descriptor: [Home Care Services] this term only	1831
#18	("primary care" or "primary healthcare" or "primary health care" or primary next practice* or general next practice* or family next practice* or outpatient* or "ambulatory care" or "community care" or community next health* or "community medicine" or "home care"):ti,ab	38393
#19	#11 or #12 or #13 or #14 or #15 or #16 or #17 or #18	43654
#20	MeSH descriptor: [Community Health Nursing] 2 tree(s) exploded	390
#21	MeSH descriptor: [Delegation, Professional] this term only	3
#22	[mh Nurses/UT]	36
#23	MeSH descriptor: [Midwifery] explode all trees and with qualifier(s): [Manpower ‐ MA]	3
#24	MeSH descriptor: [Nurse's Role] this term only	350
#25	(substitut* or delegat* or task* near/2 shift* or change* near/2 role* or expand* near/2 role* or extend* near/2 role* or expand* near/2 responsabilit* or extend* near/2 responsabilit* or expand* near/2 task* or extend* near/2 task*):ti,ab and (nurse or nurses or midwife or midwives):ti,ab	167
#26	("nurse led" or "nurse managed" or "nurse run"):ti,ab	994
#27	#21 or #22 or #23 or #24 or #25 or #26	1461
#28	MeSH descriptor: [Professional Role] this term only	186
#29	MeSH descriptor: [Professional Autonomy] this term only	36
#30	MeSH descriptor: [Professional Competence] this term only	244
#31	MeSH descriptor: [Clinical Competence] this term only	2609
#32	MeSH descriptor: [Task Performance and Analysis] this term only	2138
#33	MeSH descriptor: [Outcome Assessment (Health Care)] this term only	6564
#34	MeSH descriptor: [Delivery of Health Care] this term only	911
#35	(role or competence or performance or skill or skills) near/3 (nurse or nurses or midwife or midwives):ti,ab	296
#36	#28 or #29 or #30 or #31 or #32 or #33 or #34 or #35	12559
#37	#4 and #10 and #19	1703
#38	#10 and #20	66
#39	#19 and #27	554
#40	#4 and #19 and #36	345
#41	#37 or #38 or #39 or #40 in Trials	1972
#42	MeSH descriptor: [Nurse Practitioners] explode all trees and with qualifier(s): [Organization & administration ‐ OG, Standards ‐ ST, Utilization ‐ UT]	98
#43	nurse next (led or managed or management or run or delivered):ti	587
#44	#42 or #43 in Trials	594
#45	#41 or #44 Publication Year from 2015 to 2017, in Trials	366

CENTRAL, the Cochrane Library (searched 2015)

#1	MeSH descriptor: [Nurses] explode all trees	987
#2	MeSH descriptor: [Midwifery] this term only	254
#3	(nurse or nurses or midwife or midwives):ti,ab	9484
#4	#1 or #2 or #3	9809
#5	MeSH descriptor: [Physicians] this term only	613
#6	MeSH descriptor: [General Practitioners] explode all trees	86
#7	MeSH descriptor: [Physicians, Family] this term only	465
#8	MeSH descriptor: [Physicians, Primary Care] this term only	62
#9	(physician* or doctor or doctors or general next practitioner* or GP or GPs or family next practitioner* or "conventional care" or "usual care" or treatment near/1 usual):ti,ab	27535
#10	#5 or #6 or #7 or #8 or #9	27728
#11	MeSH descriptor: [Primary Health Care] this term only	3089
#12	MeSH descriptor: [Family Practice] explode all trees	2130
#13	MeSH descriptor: [Ambulatory Care] this term only	3034
#14	MeSH descriptor: [Ambulatory Care Facilities] explode all trees	1642
#15	MeSH descriptor: [Community Health Services] this term only	854
#16	MeSH descriptor: [Community Medicine] this term only	39
#17	MeSH descriptor: [Home Care Services] this term only	1563
#18	("primary care" or "primary healthcare" or "primary health care" or primary next practice* or general next practice* or family next practice* or outpatient* or "ambulatory care" or "community care" or community next health* or "community medicine" or "home care"):ti,ab	30438
#19	#11 or #12 or #13 or #14 or #15 or #16 or #17 or #18	35098
#20	MeSH descriptor: [Community Health Nursing] 2 tree(s) exploded	375
#21	MeSH descriptor: [Delegation, Professional] this term only	1
#22	[mh Nurses/UT]	36
#23	MeSH descriptor: [Midwifery] explode all trees and with qualifier(s): [Manpower ‐ MA]	2
#24	MeSH descriptor: [Nurse's Role] this term only	310
#25	(substitut* or delegat* or task* near/2 shift* or change* near/2 role* or expand* near/2 role* or extend* near/2 role* or expand* near/2 responsabilit* or extend* near/2 responsabilit* or expand* near/2 task* or extend* near/2 task*):ti,ab and (nurse or nurses or midwife or midwives):ti,ab	123
#26	("nurse led" or "nurse managed" or "nurse run"):ti,ab	695
#27	#21 or #22 or #23 or #24 or #25 or #26	1097
#28	MeSH descriptor: [Professional Role] this term only	141
#29	MeSH descriptor: [Professional Autonomy] this term only	32
#30	MeSH descriptor: [Professional Competence] this term only	210
#31	MeSH descriptor: [Clinical Competence] this term only	1984
#32	MeSH descriptor: [Task Performance and Analysis] this term only	1810
#33	MeSH descriptor: [Outcome Assessment (Health Care)] this term only	5316
#34	MeSH descriptor: [Delivery of Health Care] this term only	729
#35	(role or competence or performance or skill or skills) near/3 (nurse or nurses or midwife or midwives):ti,ab	230
#36	#28 or #29 or #30 or #31 or #32 or #33 or #34 or #35	10124
#37	#4 and #10 and #19	1315
#38	#10 and #20	64
#39	#19 and #27	424
#40	#4 and #19 and #36	300
#41	#37 or #38 or #39 or #40 in Trials	1520

MEDLINE, Ovid (searched 2017)

#	Searches	Results
1	exp Nurse Practitioners/og, st, ut [Organization & Administration, Standards, Utilization]	4459
2	(nurse adj (led or managed or management or run or delivered)).ti.	1732
3	or/1‐2	6127
4	exp Nurses/	80220
5	Midwifery/	17512
6	(nurse or nurses or midwife or midwives).ti,ab.	238820
7	or/4‐6	287084
8	Physicians/	78793
9	General Practitioners/	5342
10	Physicians, Family/	15802
11	Physicians, Primary Care/	2199
12	(physician* or doctor or doctors or general practitioner* or GP* or family practitioner? or conventional care or usual care or treatment as usual).ti,ab.	601268
13	or/8‐12	639057
14	Primary Health Care/	64700
15	Family Practice/	63955
16	Ambulatory Care/	39608
17	exp Ambulatory Care Facilities/	50994
18	Community Health Services/	29745
19	Community Medicine/	1966
20	Home Care Services/	31082
21	(primary care or primary healthcare or primary health care or primary practice? or general practice? or family practice? or outpatient? or ambulatory care or community care or community health* or community medicine or home care).ti,ab.	317006
22	or/14‐21	464085
23	Community Health Nursing/	19226
24	Delegation, Professional/	533
25	exp Nurses/ma, ut [Manpower, Utilization]	1944
26	Midwifery/ma, ut [Manpower, Utilization]	354
27	Nurse's Role/	37906
28	(substitut* or delegat* or (task? adj2 shift*) or (change* adj2 role?) or (expand* adj2 role?) or (extend* adj2 role?) or (expand* adj2 responsabilit*) or (extend* adj2 responsabilit*) or (expand* adj2 task?) or (extend* adj2 task?)).ti,ab. and (nurse or nurses or midwife or midwives).mp.	3511
29	(nurse led or nurse managed or nurse run).ti,ab.	3356
30	or/24‐29	46163
31	Professional Role/	10722
32	Professional Autonomy/	9162
33	Professional Competence/	22804
34	Clinical Competence/	78620
35	"Task Performance and Analysis"/	28279
36	"Outcome Assessment (Health Care)"/	60323
37	Delivery of Health Care/	76184
38	Health Resources/ma [Manpower]	1
39	((role or competence or performance or skill?) adj3 (nurse or nurses or midwife or midwives)).ti,ab.	13857
40	or/31‐39	286993
41	randomized controlled trial.pt.	456235
42	pragmatic clinical trial.pt.	530
43	controlled clinical trial.pt.	93311
44	multicenter study.pt.	222585
45	(randomis* or randomiz* or randomly allocat* or random allocat*).ti,ab.	530083
46	(trial or multicenter or multi center or multicentre or multi centre).ti.	208025
47	or/41‐46 [Modified version of CHSSS Max Sensitivity/Precision 2008]	984560
48	exp Animals/	21008246
49	Humans/	16648674
50	48 not (48 and 49)	4359572
51	review.pt.	2256553
52	meta analysis.pt.	76540
53	news.pt.	181319
54	comment.pt.	685589
55	editorial.pt.	432663
56	cochrane database of systematic reviews.jn.	13061
57	comment on.cm.	685588
58	(systematic review or literature review).ti.	92433
59	or/50‐58	7614173
60	47 not 59	830997
61	3 and 60	487
62	7 and 13 and 22 and 60	1798
63	13 and 23 and 60	91
64	22 and 30 and 60	587
65	7 and 22 and 40 and 60	496
66	or/61‐65	2532
67	remove duplicates from 66	2360
68	limit 67 to yr="2015 ‐ 2017"	350

MEDLINE, Ovid (searched 2015)

#	Searches	Results
1	exp Nurses/	71138
2	Midwifery/	15065
3	(nurse or nurses or midwife or midwives).ti,ab.	199597
4	or/1‐3	244175
5	Physicians/	61940
6	General Practitioners/	2235
7	Physicians, Family/	14696
8	Physicians, Primary Care/	1355
9	(physician* or doctor or doctors or general practitioner* or GP* or family practitioner? or conventional care or usual care or treatment as usual).ti,ab.	508208
10	or/5‐9	540257
11	Primary Health Care/	55075
12	Family Practice/	59999
13	Ambulatory Care/	36174
14	exp Ambulatory Care Facilities/	44114
15	Community Health Services/	27027
16	Community Medicine/	1890
17	Home Care Services/	27898
18	(primary care or primary healthcare or primary health care or primary practice? or general practice? or family practice? or outpatient? or ambulatory care or community care or community health* or community medicine or home care).ti,ab.	263380
19	or/11‐18	397575
20	Community Health Nursing/	18468
21	Delegation, Professional/	431
22	exp Nurses/ma, ut [Manpower, Utilization]	1863
23	Midwifery/ma, ut [Manpower, Utilization]	305
24	Nurse's Role/	33510
25	(substitut* or delegat* or (task? adj2 shift*) or (change* adj2 role?) or (expand* adj2 role?) or (extend* adj2 role?) or (expand* adj2 responsabilit*) or (extend* adj2 responsabilit*) or (expand* adj2 task?) or (extend* adj2 task?)).ti,ab. and (nurse or nurses or midwife or midwives).mp.	2931
26	(nurse led or nurse managed or nurse run).ti,ab.	2509
27	or/21‐26	40314
28	Professional Role/	8704
29	Professional Autonomy/	8537
30	Professional Competence/	20500
31	Clinical Competence/	66723
32	"Task Performance and Analysis"/	24353
33	"Outcome Assessment (Health Care)"/	50016
34	Delivery of Health Care/	66159
35	Health Resources/ma [Manpower]	1
36	((role or competence or performance or skill?) adj3 (nurse or nurses or midwife or midwives)).ti,ab.	11774
37	or/28‐36	245604
38	randomized controlled trial.pt.	382060
39	pragmatic clinical trial.pt.	92
40	controlled clinical trial.pt.	88475
41	multicenter study.pt.	177243
42	(randomis* or randomiz* or randomly allocat* or random allocat*).ti,ab.	410331
43	(trial or multicenter or multi center or multicentre or multi centre).ti.	154503
44	or/38‐43 [Modified version of CHSSS Max Sensitivity/Precision 2008]	800967
45	exp Animals/	17606521
46	Humans/	13630323
47	45 not (45 and 46)	3976198
48	review.pt.	1925848
49	meta analysis.pt.	52132
50	news.pt.	165705
51	comment.pt.	606507
52	editorial.pt.	366834
53	cochrane database of systematic reviews.jn.	10839
54	comment on.cm.	606507
55	(systematic review or literature review).ti.	55879
56	or/47‐55	6755068
57	44 not 56	679799
58	4 and 10 and 19 and 57	1440
59	10 and 20 and 57	87
60	19 and 27 and 57	428
61	4 and 19 and 37 and 57	400
62	or/58‐61	1809

CINAHL, EbscoHost (searched 2017)

#	Query	Results
S61	S59 AND S60	62
S60	PY 2015 OR PY 2016 OR PY 2017	453,305
S59	S49 AND S57 [Exclude MEDLINE records]	328
S58	S49 AND S57	1,354
S57	S50 OR S51 OR S52 OR S53 OR S54 OR S55 OR S56	218,043
S56	TI trial or multicenter or "multi center" or multicentre or "multi centre"	46,982
S55	TI ( randomis* or randomiz* or randomly) OR AB ( randomis* or randomiz* or randomly)	120,964
S54	(MH "Multicenter Studies")	22,224
S53	(MH "Clinical Trials")	87,754
S52	(MH "Randomized Controlled Trials")	30,526
S51	PT clinical trial	52,908
S50	PT randomized controlled trial	30,877
S49	S44 OR S45 OR S46 OR S47 OR S48	30,520
S48	S4 AND S17 AND S39	2,019
S47	S17 AND S27	5,416
S46	S8 AND S18	801
S45	S4 AND S8 AND S17	4,523
S44	S41 OR S42 OR S43	22,587
S43	TI ( nurse W0 (led or managed or management or run or delivered) )	2,045
S42	(MH "Clinical Nurse Specialists")	5,529
S41	(MH "Nurse Practitioners+")	15,944
S40	S28 OR S29 OR S30 OR S31 OR S32 OR S33 OR S34 OR S35 OR S36 OR S37 OR S38 OR S39	158,594
S39	TI ( (role or competence or performance or skill or skills) N3 (nurse or nurses or midwife or midwives) ) OR AB ( (role or competence or performance or skill or skills) N3 (nurse or nurses or midwife or midwives) )	19,444
S38	(MH "Health Resource Utilization/MA")	1,017
S37	(MH "Health Care Delivery")	26,197
S36	(MH "Outcome Assessment")	19,978
S35	(MH "Task Performance and Analysis")	6,102
S34	(MH "Professional Competence")	9,330
S33	(MH "Nursing Skills")	3,090
S32	(MH "Clinical Competence")	21,138
S31	(MH "Professional Autonomy")	3,383
S30	(MH "Physician's Role")	6,094
S29	(MH "Nursing Role")	40,396
S28	(MH "Professional Role")	21,193
S27	S19 or S20 or S21 or S22 or S25 or S26	49,630
S26	TI ( "nurse led" or "nurse managed" or "nurse run" ) OR AB ( "nurse led" or "nurse managed" or "nurse run" )	3,147
S25	S23 AND S24	4,157
S24	TX ( nurse or nurses or midwife or midwives ) OR TX ( nurse or nurses or midwife or midwives )	543,661
S23	TI ( substitut* or delegat* or (task* N2 shift*) or (change* N2 role*) or (expand* N2 role*) or (extend* N2 role*) or (expand* N2 responsabilit*) or (extend* N2 responsabilit*) or (expand* N2 task*) or (extend* N2 task*) ) OR AB ( substitut* or delegat* or (task* N2 shift*) or (change* N2 role*) or (expand* N2 role*) or (extend* N2 role*) or (expand* N2 responsabilit*) or (extend* N2 responsabilit*) or (expand* N2 task*) or (extend* N2 task*) )	13,864
S22	(MH "Nursing Role")	40,396
S21	(MH "Midwives+/MA/UT")	210
S20	(MH "Nurses+/MA/UT")	2,301
S19	(MH "Delegation of Authority")	1,623
S18	(MH "Community Health Nursing+")	23,411
S17	S9 OR S10 OR S11 OR S12 OR S13 OR S14 OR S15 OR S16	141,570
S16	TI ( "primary care" or "primary healthcare" or "primary health care" or primary W0 practice* or general W0 practice* or family W0 practice* or outpatient* or "ambulatory care" or "community care" or community W0 health* or "community medicine" or "home care" ) OR AB ( "primary care" or "primary healthcare" or "primary health care" or primary W0 practice* or general W0 practice* or family W0 practice* or outpatient* or "ambulatory care" or "community care" or community W0 health* or "community medicine" or "home care" )	89,176
S15	(MH "Home Health Care")	15,668
S14	(MH "Community Medicine")	46
S13	(MH "Community Health Services")	12,590
S12	(MH "Ambulatory Care Facilities+")	9,408
S11	(MH "Ambulatory Care")	6,683
S10	(MH "Family Practice")	12,121
S9	(MH "Primary Health Care")	34,178
S8	S5 OR S6 OR S7	130,746
S7	TI ( physician* or doctor or doctors or (general W0 practitioner*) or GP or GPs or (family W0 practitioner*) or "conventional care" or "usual care" or "treatment as usual" ) OR AB ( physician* or doctor or doctors or (general W0 practitioner*) or GP or GPs or (family W0 practitioner*) or "conventional care" or "usual care" or "treatment as usual" )	105,241
S6	(MH "Physicians, Family")	9,248
S5	(MH "Physicians")	35,499
S4	S1 OR S2 OR S3	327,119
S3	TI ( nurse or nurses or midwife or midwives ) OR AB ( nurse or nurses or midwife or midwives )	233,516
S2	(MH "Midwives+")	9,480
S1	(MH "Nurses+")	165,004

CINAHL, EbscoHost (searched 2015)

#	Query	Results
S54	S49 OR S50 OR S51 OR S52 Exclude MEDLINE records	259
S53	S49 OR S50 OR S51 OR S52	1,199
S52	S4 AND S17 AND S40 AND S48	1,065
S51	S17 AND S27 AND S48	245
S50	S8 AND S18 AND S48	48
S49	S4 AND S8 AND S17 AND S48	537
S48	S41 OR S42 OR S43 OR S44 OR S45 OR S46 OR S47	177,524
S47	TI trial or multicenter or "multi center" or multicentre or "multi centre"	35,740
S46	TI ( randomis* or randomiz* or randomly) OR AB ( randomis* or randomiz* or randomly)	99,382
S45	(MH "Multicenter Studies")	8,673
S44	(MH "Clinical Trials")	80,747
S43	(MH "Randomized Controlled Trials")	20,909
S42	PT clinical trial	51,624
S41	PT randomized controlled trial	24,980
S40	S28 OR S29 OR S30 OR S31 OR S32 OR S33 OR S34 OR S35 OR S36 OR S37 OR S38 OR S39	956,442
S39	TI ( (role or competence or performance or skill or skills) N3 (nurse or nurses or midwife or midwives) ) OR AB ( (role or competence or performance or skill or skills) N3 (nurse or nurses or midwife or midwives) )	16,049
S38	(MH "Health Resource Utilization/MA")	890,003
S37	(MH "Health Care Delivery")	23,166
S36	(MH "Outcome Assessment")	17,332
S35	(MH "Task Performance and Analysis")	5,148
S34	(MH "Professional Competence")	8,480
S33	(MH "Nursing Skills")	2,663
S32	(MH "Clinical Competence")	18,541
S31	(MH "Professional Autonomy")	3,105
S30	(MH "Physician's Role")	5,524
S29	(MH "Nursing Role")	37,124
S28	(MH "Professional Role")	18,725
S27	S19 or S20 or S21 or S22 or S25 or S26	45,412
S26	TI ( "nurse led" or "nurse managed" or "nurse run" ) OR AB ( "nurse led" or "nurse managed" or "nurse run" )	2,769
S25	S23 AND S24	3,814
S24	TX ( nurse or nurses or midwife or midwives ) OR TX ( nurse or nurses or midwife or midwives )	450,292
S23	TI ( substitut* or delegat* or (task* N2 shift*) or (change* N2 role*) or (expand* N2 role*) or (extend* N2 role*) or (expand* N2 responsabilit*) or (extend* N2 responsabilit*) or (expand* N2 task*) or (extend* N2 task*) ) OR AB ( substitut* or delegat* or (task* N2 shift*) or (change* N2 role*) or (expand* N2 role*) or (extend* N2 role*) or (expand* N2 responsabilit*) or (extend* N2 responsabilit*) or (expand* N2 task*) or (extend* N2 task*) )	12,387
S22	(MH "Nursing Role")	37,124
S21	(MH "Midwives+/MA/UT")	179
S20	(MH "Nurses+/MA/UT")	2,019
S19	(MH "Delegation of Authority")	1,524
S18	(MH "Community Health Nursing+")	21,668
S17	S9 OR S10 OR S11 OR S12 OR S13 OR S14 OR S15 OR S16	125,143
S16	TI ( "primary care" or "primary healthcare" or "primary health care" or primary W0 practice* or general W0 practice* or family W0 practice* or outpatient* or "ambulatory care" or "community care" or community W0 health* or "community medicine" or "home care" ) OR AB ( "primary care" or "primary healthcare" or "primary health care" or primary W0 practice* or general W0 practice* or family W0 practice* or outpatient* or "ambulatory care" or "community care" or community W0 health* or "community medicine" or "home care" )	78,393
S15	(MH "Home Health Care")	14,528
S14	(MH "Community Medicine")	36
S13	(MH "Community Health Services")	11,175
S12	(MH "Ambulatory Care Facilities+")	8,574
S11	(MH "Ambulatory Care")	6,117
S10	(MH "Family Practice")	10,420
S9	(MH "Primary Health Care")	30,073
S8	S5 OR S6 OR S7	114,891
S7	TI ( physician* or doctor or doctors or (general W0 practitioner*) or GP or GPs or (family W0 practitioner*) or "conventional care" or "usual care" or "treatment as usual" ) OR AB ( physician* or doctor or doctors or (general W0 practitioner*) or GP or GPs or (family W0 practitioner*) or "conventional care" or "usual care" or "treatment as usual" )	91,989
S6	(MH "Physicians, Family")	8,240
S5	(MH "Physicians")	31,958
S4	S1 OR S2 OR S3	301,180
S3	TI ( nurse or nurses or midwife or midwives ) OR AB ( nurse or nurses or midwife or midwives )	216,144
S2	(MH "Midwives+")	8,346
S1	(MH "Nurses+")	151,395

Open Grey = 21 hits (27.02.2017)

Grey Literature Report = 97 hits (27.02.2017)

The Grey Literature Report was searched on 27 February 2017 using different search terms:

1. "nurse practitioner" = 14 hits

2. "nurse clinician" = 1 hit

3. "nurse led" = 6 hits

4. "nurse managed" = 65 hits

6. "nurse delivered" = 11 hits

7. substitut ‐ with Additional Keywords: doctor = 0 hits

8. substitut ‐ with Additional Keywords: physician = 0 hits

9. substitut ‐ with Additional Keywords: "general practitioner" = 0 hits

International Clinical Trials Registry Platform (ICTRP), Word Health Organization (WHO): www.who.int/ictrp/en/= 71 hits (21.02.2017)

1. Advanced search: nurse led OR nurse managed OR nurse run OR nurse delivered [in Title + Recruitment status: All]

OR

nurse led OR nurse managed OR nurse run OR nurse delivered [in Intervention + Recruitment status: All]

2. Advanced search: substitut* AND nurse* AND (doctor* OR physician* OR general practitioner OR general practitioners) [in Title + Recruitment status: All]

OR

substitut* AND nurse* AND (doctor* OR physician* OR general practitioner OR general practitioners) [in Intervention + Recruitment status: All]

ClinicalTrials.gov, US National Institutes of Health (NIH):clinicaltrials.gov/= 172 hits (21.02.2017)

1. Search Terms: "nurse led" OR "nurse managed" OR "nurse run" OR "nurse delivered"

2. Search Terms: (substitute OR substitution OR substituting) AND (nurse OR nurses) AND (doctor OR doctors OR physician OR physicians OR "general practitioner" OR "general practitioners")

Science Citation Index and the Social Sciences Citation Index, Web of Science, Thomson Reuters = 41 hits (2015)

Citation search for the following studies: Campbell 2014; Houweling 2011; Iglesias 2013; Larsson 2014; Ndosi 2014

Appendix 2. GRADE profiles

Assessing the certainty [1]of evidence across studies for an outcome

Comparison nurse ‐ doctor substitution in primary care

Certainty assessment of evidence for each outcome

No of studies	Design	Risk of bias		Inconsistency		Indirectness [2]		Imprecision	Other [3]	Certainty (overall score) [4]
Outcome: mortality
8	Randomised trials	No serious risk of bias		The trials contributing to this estimate are quite varied (some focus on people with specific health issues and others on more generalist primary care attenders (‐1)		No serious indirectness		Wide CI that includes no effect (‐1)	None	Low (3)
Outcome: patient health status
Clinical outcomes (3) Self‐reported measurements (13)	Randomised trials	No serious risk of bias		Some studies: effect varies between trials (‐0.5)		No serious indirectness		Some studies: wide CI (‐0.5)	None	Moderate (3)
Outcome: physical function
3	Randomised trials	No serious risk of bias		Effect varies between trials. (‐1)		No serious indirectness		No serious imprecision	None	Moderrate (3)
Outcome: pain
2	Randomised trials	No serious risk of bias		No serious inconsistency		Only patients with rheumatoid arthritis were included (‐1).		No serious imprecision	None	Moderate (3)
Outcome: systolic blood pressure
3	Randomised trials	High risk of bias in 1 out of 3 studies (‐1)		No serious inconsistency		No serious indirectness		No serious imprecision	None	Moderate (3)
Outcome: diastolic blood pressure
2	Randomised trials	High risk of bias in 1 out of 2 studies (‐1)		No serious inconsistency		No serious indirectness		No serious imprecision	None	Moderate (3)
Outcome: total cholesterol
2	Randomised trials	No serious risk of bias		No serious inconsistency		No serious indirectness		No serious imprecision	None	High (4)
Outcome: HbA1C
2	Randomised trials	High risk of bias in 1 out of 2 studies (‐1)		No serious inconsistency		No serious indirectness		No serious imprecision	None	Moderate (3)
Outcome: DAS
2	Randomised trials	No serious risk of bias		No serious inconsistency		Only patients with rheumatoid arthritis were included. (‐1)		No serious imprecision	None	Moderate (3)
Outcome: satisfaction and preference
7	Randomised trials	No serious risk of bias		Important heterogeneity (‐1)		No serious indirectness		No serious imprecision	None	Moderate (3)
Outcome: quality of life
6	Randomised trials	No serious risk of bias	Important heterogeneity (‐1)		No serious indirectness		Wide CI that includes no effect (‐1)		None	Low (2)
Outcome: process of care ‐ no GRADE due to no pooled analyses and a wide range of outcomes
Outcome: utilisation
19	Randomised trials	No serious risk of bias		Some outcomes: important heterogeneity and effects that vary between trials (‐0.5)		No serious indirectness		Some outcomes: Wide CI (‐0.5)	None	Moderate (3)
Outcome: length of consultation
4	Randomised trials	No serious risk of bias		Important heterogeneity (‐1)		No serious indirectness		No serious imprecision	None	Moderate (3)
Outcome scheduled: return visits
3	Randomised trials	No serious risk of bias		Important heterogeneity (‐1)		No serious indirectness		Wide CI (‐1)	None	Low (2)
Outcome: attended return visits
4	Randomised trials	No serious risk of bias		No serious inconsistency		No serious indirectness		No serious imprecision	None	High (4)
Outcome: prescriptions ordered
4	Randomised trials	No serious risk of bias		No serious inconsistency		No serious indirectness		No serious imprecision	None	High (4)
Outcome: investigations
4	Randomised trials	No serious risk of bias		Important heterogeneity (‐1)		No serious indirectness		Wide CI (‐1)	None	Low (2)
Outcome: hospital referral
5	Randomised trials	No serious risk of bias		Important heterogeneity (‐1)		No serious indirectness		Wide CI (‐1)	None	Low (2)
Outcome: attendance at A&E
6	Randomised trials	No serious risk of bias		No serious inconsistency		No serious indirectness		No serious imprecision	None	High (4)
Outcome: hospital admission
3	Randomised trials	High risk of bias in 1 out of 3 studies (‐1).		No serious inconsistency		No serious indirectness		Wide CI (‐1)	None	Low (2)
Outcome: costs ‐ no grade since the types of costs assessed varied widely and a range of different approaches were used to value resources and calculate costs

[1] This can also be referred to as ‘quality of the evidence’ or ‘confidence in the estimate’. The “certainty of the evidence” is an assessment of how good an indication the research provides of the likely effect; i.e. the likelihood that the effect will be substantially different from what the research found. By “substantially different”, we mean a large enough difference that it might affect a decision.

[2] Indirectness includes consideration of:

· Indirect (between‐study) comparisons

· Indirect (surrogate) outcomes

· Applicability (study populations, interventions, or comparisons that are different from those of interest).

[3] Other considerations for downgrading include publication bias. Other considerations for upgrading include a strong association with no plausible confounders, a dose response relationship, and if all plausible confounders or biases would decrease the size of the effect (if there is evidence of an effect), or increase it if there is evidence of no harmful effect (safety).

[4]

4 High = This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different** is low.

3 Moderate = This research provides a good indication of the likely effect. The likelihood that the effect will be substantially different** is moderate.

2 Low = This research provides some indication of the likely effect. However, the likelihood that it will be substantially different** is high.

1 Very low = This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different** is very high.

** Substantially different = a large enough difference that it might affect a decision.

Data and analyses

Comparison 1. Doctor‐nurse substitution study results.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Mortality	8	36529	Risk Ratio (IV, Random, 95% CI)	0.77 [0.57, 1.03]
2 Physical function (better vs not better)	3	3549	Risk Ratio (Random, 95% CI)	1.03 [0.98, 1.09]
3 Pain	2		Mean Difference (Random, 95% CI)	0.76 [‐3.85, 5.38]
4 Quality of life	6	16002	Std. Mean Difference (Random, 95% CI)	0.16 [0.00, 0.31]
5 Systolic blood pressure	3	1023	Mean Difference (IV, Random, 95% CI)	‐3.73 [‐6.02, ‐1.44]
6 Diastolic blood pressure	2	562	Mean Difference (IV, Random, 95% CI)	‐2.54 [‐4.57, ‐0.52]
7 Total cholesterol	2	702	Mean Difference (IV, Random, 95% CI)	‐0.15 [‐0.32, 0.02]
8 HbA1c	2	310	Mean Difference (IV, Random, 95% CI)	0.08 [‐0.25, 0.41]
9 Disease Activity Score	2		Mean Difference (Random, 95% CI)	0.04 [‐0.17, 0.24]
10 Patient satisfaction	7	16993	Std. Mean Difference (Random, 95% CI)	0.08 [0.01, 0.15]
11 Length of consultation	4	5848	Std. Mean Difference (Random, 95% CI)	0.38 [0.22, 0.54]
12 Scheduled return visits	3	3934	Risk Ratio (Random, 95% CI)	1.31 [0.89, 1.94]
13 Attended return visit	4	5064	Risk Ratio (Random, 95% CI)	1.19 [1.07, 1.33]
14 Prescription ordered	4	5702	Risk Ratio (Random, 95% CI)	0.99 [0.95, 1.03]
15 Investigations	4	3654	Risk Ratio (Random, 95% CI)	0.95 [0.59, 1.51]
16 Hospital referral	4	17299	Risk Ratio (Random, 95% CI)	0.90 [0.54, 1.49]
17 Attendance at accident and emergency	6	29905	Risk Ratio (Random, 95% CI)	1.00 [0.91, 1.09]
18 Hospital admission	3	16466	Risk Ratio (Random, 95% CI)	1.04 [0.78, 1.39]

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Campbell 2014.

Methods	Randomised trial
Participants	13,707 patients (total group); mean age in intervention group 41.5 (25.2), in control group 44.7 (25.0); 40% male in intervention group, 41% male in control group 42 practices at 4 centres
Interventions	Intervention: nurse‐led computer‐supported telephone triage Control: GP‐led telephone triage Detailed description of the intervention: Compared 3 groups delivering telephone triage: GP‐led triage, nurse‐led computer‐based triage, and usual care triage GP‐led: Components of the Stour Access System were used. Once the receptionist had established that the patient was requesting a same‐day appointment, the patient was asked to leave a contact number with the receptionist and was advised that the GP would call the patient back within around 1 to 2 hours. This time scale (for both GP‐led and nurse‐led arms) was flexible, so as to optimise prioritisation. The GP discussed the complaint with the patient and triaged the patient to the most appropriate person, such as a nurse, or booked a face‐to‐face appointment with the GP, or provided advice on the telephone Nurse‐led: The Plain Healthcare Odyssey Patient Access System was used for patients registered at the practice. A computerised clinical decision support (CCDS) system was used to assist nurses at the practice in assessing and making decisions about the clinical needs of patients who have called the practice requesting a same‐day appointment Usual care: Standard consultation management system practices were used (differed between practices) Supervision, oversight: unknown
Outcomes	Patient outcomes: Patients' experience of care after the same‐day request Problem resolution Overall satisfaction with care provided on the day of the consultation request Health status (EQ‐5D) Deaths associated with trial processes Process of care measures: Primary care workload (total numbers of primary care contacts taking place in the 28 days after the patient’s index appointment request) Occurrences of each of the 20 individual consultation types contributing to the primary outcome Profile of patient contacts and their distribution by healthcare professionals Patient safety (i.e. the occurrence and duration of any emergency hospital admissions within 7 days of the index request, and the number of patients with any attendances at accident and emergency departments within 28 days) Resource utilisation: Costs: costs over 28 days with regard to primary outcome contacts
Notes	Country: UK Study period: 25 months Nurse role: nurse in charge of computer‐supported telephone triage for patients requesting a same‐day appointment Nurse title: nurse (nurse practitioners and practice nurses) Nurse educational background: EQF level unknown Nurse years of experience: unknown Nurse additional training: training in the use of Odyssey Patient Access and in telephone consultation skills. Following this was a pretrial period of 1 month, during which nurses were expected to practise using the decision support in their daily work; towards the end of this period, their use of the system was assessed
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	The sequence process included a random component. "Participating practices were randomly assigned (1:1:1), via a computer‐generated randomisation sequence, to GP triage, nurse triage with computer decision support, or usual care. The randomisation sequence [was] minimized for research centre, deprivation (deprived [below average Index of Multiple Deprivation 2010, based on practice postcode] or not‐deprived [average or above]) and list size (small [< 3500 patients], medium [3500–8000 patients], or large [> 8000 patients]) of the trial team". However, 10 of the 15 practices allocated to NP triage withdrew. "To maintain balance between groups, any practices that withdrew after randomisation were replaced with a waiting‐list practice that was from the same location, and of similar size and deprivation when possible. Because of the small numbers of waiting‐list practices, replacements were purposively allocated according to minimisation criteria".
Allocation concealment (selection bias)	Low risk	Patients and investigators enrolling patients could not foresee assignment. "A stochastic element within the minimisation algorithm maintained concealment. Allocation was done by a statistician independent form". "Allocations were concealed from practices until after they had agreed to participate; this concealment also applied to practices replacing practices that had withdrawn from the study for whatever reason".
Baseline characteristics	Low risk	Characteristics of patients were similar in both groups. "Practice and patient characteristics were well balanced between groups".
Baseline outcome measurement	Low risk	Baseline primary outcome measurement was not relevant. Baseline secondary outcome health status was not measured, and differences in baseline health status could bias the outcome ‘health status’.
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	Practitioners and patients were not blinded. It is unclear whether the outcome was influenced by lack of blinding of patients and care providers. "Patients, clinicians, and researchers were not masked to allocation, but practice assignment was concealed from the trial statistician".
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Researchers were not blinded; however, the trial statistician was blinded. "Patients, clinicians, and researchers were not masked to allocation, but practice assignment was concealed from the trial statistician".
Incomplete outcome data (attrition bias) All outcomes	Low risk	Follow‐up of patients < 80%; however per‐protocol and intention‐to‐treat analyses showed similar results. "Findings from our per‐protocol analysis showed intensification of the noted effects of both GP triage and nurse triage (data not shown)".
Selective reporting (reporting bias)	Low risk	Protocol was available. Predefined outcomes measurements were reported.
Contamination	Low risk	Not likely, because allocation was by practice
Bias due to lack of power	Low risk	"7046 patients per group would need to be recruited from 42 practices". In the GP triage, 6781 patients were eligible for intervention (6697 + 84). This is a relatively small difference with the calculated power. Trial authors commented: "The trial was fully powered and we exceeded our recruitment target in gaining access to the primary outcome data, partly because of a process of obtaining initial verbal consent to participate".

Chambers 1978.

Methods	Randomised trial
Participants	868 patients (total group), all ages, 34% male 1 nurse 1 doctor
Interventions	Intervention: families allocated to nurse‐led primary care Control: families allocated to doctor‐led primary care Detailed description of the intervention: Compared 2 groups providing family care: A conventional group, assigned to continuing primary clinical services from a family doctor A family practice nurse group whose first contact primary clinical services were to be provided by the family practice nurse Supervision, oversight: The family practice nurse was delegated the responsibility of choosing between three possible courses of action: providing specific treatment; providing reassurance alone, without specific treatment; or referring the patient to the associated family doctor, to another clinician, or to an appropriate service agency.
Outcomes	Patient outcomes: Health status
Notes	Country: Canada Study period: 12 months Nurse role: first contact and ongoing primary care Nurse title: practice nurse Nurse educational background: EQF level unknown Nurse years of experience: The nurse already worked for 4 years in the family practice. Nurse additional training: The nurse attended a special 9‐month education programme for family practice nurses including skills such as decision‐making, clinical judgement, social history taking, physical examinations, and the ability to distinguish between abnormal and normal patient symptoms and signs.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	No clear method of randomisation was reported. "Random allocation in a ratio of 2:1" (family doctor:family practice nurse)
Allocation concealment (selection bias)	Unclear risk	No information
Baseline characteristics	Low risk	Baseline characteristics were reported and were similar for both groups. "The groups are highly similar and none of the observed differences are statistically significant".
Baseline outcome measurement	Low risk	Baseline outcomes were reported and were similar for both groups.
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	Families/patients and care providers were not blinded. It is unclear whether the outcome was influenced by lack of blinding of patients and care providers.
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	No information
Incomplete outcome data (attrition bias) All outcomes	High risk	Follow‐up of patients < 80% ("65.5%")
Selective reporting (reporting bias)	Unclear risk	The protocol was not available.
Contamination	Unclear risk	No information
Bias due to lack of power	Unclear risk	No power calculation performed

Chan 2009.

Methods	Randomised trial
Participants	175 patients (total group), mean age 48.4 years (control), 50.2 years (intervention); 49% male (in total) 1 nurse and unknown number of doctors
Interventions	Intervention: patient care after gastric endoscopy allocated to nurse Control: patient care after gastric endoscopy allocated to doctor Detailed description of the intervention: Compared 2 groups providing follow‐up for patients with dyspepsia after direct access gastroscopy. Systematic GNP‐led follow‐up in an out‐patient clinic: The 'GNP' group was given 1 out‐patient appointment; a full medical history was taken. Usual care by GPs: The 'GP' cohort was discharged and was advised to see their GP. Patients included were those with mild gastroesophageal reflux disease (GORD – non‐erosive or grade A and B oesophagitis, hiatus hernia), those with non‐ulcer dyspepsia (NUD) (mild and moderate gastritis or duodenitis), and those with normal findings. After gastroscopy, endoscopists maintained their routine practice in giving verbal and written advice to patients and documented treatment recommendations to GPs in a formal report. Clinical management was structured, based on national and local guidelines, with reference to each patient's predominant symptoms. Patients were given counselling and lifestyle advice, supplemented with relevant locally devised leaflets (i.e. reflux, non‐ulcer dyspepsia, weight control), and an individualised treatment plan was agreed upon. Further investigation such as the urea breath test, motility studies, and barium meal were initiated, if required, as per routine clinical practice. To ensure practice consistency and reproducibility, 'history taking' and 'lifestyle advice' proformas were devised and used. Supervision, oversight: Studied interventional patients were seen in the nurse‐led clinic within secondary care, without direct supervision from any consultant gastroenterologists. However, cases could be discussed with a doctor, if deemed necessary.
Outcomes	Patient outcomes: Gladys, health status short form (SF‐12)
Notes	Country: UK Study period: 6 months Nurse role: ongoing care (follow‐up) after gastroendoscopy Nurse title: gastrointestinal nurse practitioner Nurse educational background: EQF level 8 Nurse years of experience: The nurse had been qualified as a State Registered Nurse for 20 years and specialised in gastro nursing for 4 years and 2 months. Nurse additional training: Clinic consultation skill was developed with the help of a named GI consultant. Initially, the nurse sat in that clinic (2 months) as an observer. The next stage was to see patients who had been filtered by the consultant from that clinic on that day. The nurses’ consulting room was next to the GI consultants’ room to effect direct supervision, as each patient case was presented to the GI and treatment identified (6 months). Finally, a nurse‐led clinic was established and was formally running alongside the GI clinics, with pre‐identified patients advanced from all GI consultants. Some 18 months later, the nurse was authorised to discuss selective cases with the patient’s named consultant, if required. Three monthly reviews were performed initially; this was reduced to yearly and was incorporated in the annual appraisal.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	The sequence process included a random component. "Patients eligible for entry after endoscopy were randomised into intervention (GNP) and control (GP) groups, with a password protected, computer generated random number table”.
Allocation concealment (selection bias)	Low risk	Participants and investigators enrolling participants could not foresee assignment. "with a password protected, computer generated random number table"
Baseline characteristics	Low risk	Baseline characteristics were reported and were similar for both groups. "The baseline Gladys scores (high scores equal higher burden of disease and symptoms) were similar (10.0 vs 9.9) but the SF12 scores (672.0 vs 627.7) were higher (high scores equal better health) in the GP group (see Table 1). The cost of UHD used, 6 months prior to the investigation, was lower in the GP group (£52.4 vs £59.5)". But, "The two groups were compared by the change from baseline to month 6 in the key outcome variables – Gladys score, SF12 and overall UHDs cost, adjusted for baseline values by including the baseline levels of the outcome in the ANOVA as a covariate; p < 0.05 was taken as being significant".
Baseline outcome measurement	Low risk	Baseline outcome measurements were reported and adjusted analyses performed. "Adjusted for baseline level using ANOVA"
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	No information
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Blinding of assessment was performed. "A researcher blinded to the patients' study status and diagnosis interviewed all participants by telephone, at a prearranged time suitable to the patient, six months after randomisation".
Incomplete outcome data (attrition bias) All outcomes	Low risk	Follow‐up of patients > 80% "199 unselected patients were approached and 196 (98.5%) were recruited. One hundred and seventy five (89.3%) patients were eligible after investigation. Of the 21 ineligible patients, 16 did not meet the criteria (Barrett's oesophagus: 6, oesophagitis grade C: 2, oesophageal stricture: 1, peptic ulcer disease: 3, possible cancer: 1). Three cases were deemed unsuitable by the endoscopist, as symptoms were attributed to other conditions (rhinitis 1, angina 2)". Two did not have the procedure (failed intubation 1, food in stomach 1). "Early withdrawals (GP n = 3, GNP n = 4) after randomisation were experienced in both groups (Figure 1). Three in the 'GP' group decided not to see their GP. The four in the GNP group were due to work commitments (2), leaving the area (1) and after own GP consultation (1)".
Selective reporting (reporting bias)	Unclear risk	The protocol was not available.
Contamination	Low risk	Patients in the intervention group went to a nurse‐led clinic, and controls went to their doctor. Therefore, it is unlikely that both groups were contaminated.
Bias due to lack of power	Low risk	Number of included patients was approximately similar to results of the sample size calculation.

Dierick‐van Daele 2009.

Methods	Randomised trial
Participants	1501 patients (total group); mean age in intervention group 46.1, in control group 42.8; 38.2% male in intervention group, 40% male in control group 50 GPs 12 NPs
Interventions	Intervention: patients allocated to nurse practitioners Control: patients allocated to GPs Detailed description of the intervention: Compared 2 groups providing care to patients with common complaints as first point of contact The NP saw patients with respiratory and throat problems, ear and nose problems, musculoskeletal problems and injuries, skin injuries, urinary problems, gynaecological problems, and geriatric problems. The role of the NP involved assessing symptoms including physical examinations when appropriate and diagnosing and making decisions about further treatment, including writing prescriptions and referrals to primary or secondary services and clinical investigations. Supervision, oversight: The NP did not have full authority to prescribe medications, and so the GP was always available for consultation and for validation of prescriptions and referrals.
Outcomes	Patient outcomes: Satisfaction Burden of illness Quality of life Process of care measures: Adherence to clinical guidelines Appropriate medication prescribed Resource utilisation: Prescriptions Investigations Return visits Costs: direct healthcare costs, including and excluding productivity
Notes	Country: Netherlands Study period: 6 months Nurse role: first contact and ongoing care Nurse title: nurse practitioners Nurse educational background: EQF level 7 Nurse years of experience: 0 years as an NP, at least 2 years of experience as a registered nurse Nurse additional training: unknown
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	The sequence process included a random component. "Sequentially‐numbered sealed envelopes containing randomised assignments to the two groups were provided by an independent person. The codes were generated from random number tables".
Allocation concealment (selection bias)	Low risk	Patients and investigators enrolling patients could not foresee assignment. "Sequentially‐numbered sealed envelopes containing randomised assignments to the two groups were provided by an independent person. The codes were generated from random number tables"
Baseline characteristics	Low risk	Baseline outcomes were reported and were similar for both groups. Only age was different. "Patients who returned all questionnaires were statistically significantly older (mean = 48Æ74, SD = 16Æ8) than those who did not (mean = 42Æ75, SD = 16Æ4; p < 0Æ001). There were no statistically significant differences in gender and type of diagnosis between patients with or without complete data. No statistically significant differences were noted between patients in two groups in terms of other (chronic) diseases".
Baseline outcome measurement	Unclear risk	Primary outcomes could not be assessed before the intervention.
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	Patients and care providers were not blind to the intervention. It is unclear whether the outcome was influenced by lack of blinding of patients and care providers.
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	No information
Incomplete outcome data (attrition bias) All outcomes	Low risk	Follow‐up of patients > 80% "499 met one or more exclusion criteria, declined to participate, had no interest or were too ill". "58 patients who were allocated to the NP intervention group and 47 patients in the reference group did not attend the appointment they had booked or refused to participate because of being too ill or not having an interest".
Selective reporting (reporting bias)	Unclear risk	The protocol was not available.
Contamination	Low risk	It is unlikely that patients who visited the GP consulted the NP for the same question/problem, or vice versa. "Patients in the intervention group who did see the GP were excluded from analysis (n = 43)".
Bias due to lack of power	Unclear risk	No power calculation performed

Hemani 1999.

Methods	Randomised trial
Participants	450 patients (total group), mean age 61 years, 98% male 9 nurses 45 doctors
Interventions	Intervention: patients allocated to nurse‐led primary care Control 1: patients allocated to trainee doctors (2nd and 3rd year residents) Control 2: patients allocated to fully trained doctors (attending doctors) Detailed description of the intervention: not available Supervision, oversight: First‐year residents and newly graduated nurse practitioners were required to present every patient to the attending doctors during the first 6 months of their appointment, whereas the remainder of residents and nurse practitioners presented cases only when they believed it to be necessary.
Outcomes	Resource utilisation: Consultation rate Tests Use of other services ‐ hospital admissions, emergency department visits, specialty visits
Notes	Country: USA Study period: 12 months Nurse role: first contact and ongoing primary care Nurse title: nurse practitioners Nurse educational background: EQF level unknown Nurse years of experience: unknown Nurse additional training: unknown
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	The sequence process included a random component. "Most patients were assigned to any available provider and these patients were then scheduled by a clerk on a computerized system for the net available appointment, regardless of the type of provider" "Our study sample makes use of this quasi random assignment".
Allocation concealment (selection bias)	Low risk	Participants or investigators enrolling participants could not foresee assignments. A computerised system was used.
Baseline characteristics	Unclear risk	No information
Baseline outcome measurement	Unclear risk	Primary outcomes were not assessed before the intervention.
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Practitioners low risk, patients no information (unclear risk) "The practitioners at the Baltimore VAMC were aware that a study of utilization patterns was being conducted, but did not know which patients were included".
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	No information
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	No information available about follow‐up
Selective reporting (reporting bias)	Unclear risk	The protocol was not available.
Contamination	Unclear risk	No information. However, the quote "For the purposes of this study, patients remained in the group to which they were initially assigned, even if their type of primary care provider changed after the first visit" suggests that contamination has occurred.
Bias due to lack of power	Unclear risk	No power calculation performed

Houweling 2011.

Methods	Randomised trial
Participants	239 patients (total group); mean age in intervention group 67.1 (11.0), in control group 69.5 (10.6); 52.9% male in intervention group, 42.3% male in control group 5 doctors (GPs) 2 nurses
Interventions	Intervention: patients with T2DM allocated to nurse practitioners Control: patients with T2DM allocated to GPs Detailed description of the intervention: Compared 2 groups providing diabetes care: Treatment primarily by PNs Standard care from a GP Eligible patients were selected via the GPs’ patient information system and the local pharmacy. Initial selection included patients with a diagnosis of diabetes, patients who were on medication for diabetes, and patients whose glycated haemoglobin (HbA1c) levels had been measured within the past 3 years. Exclusion criteria were (1) no diagnosis of diabetes, (2) type 1 diabetes, (3) diabetes not treated in the primary healthcare setting, (4) inability to participate in the study because of old age or comorbidity, in the opinion of the GP, and (5) not willing to return for follow‐up. PNs were permitted to prescribe 14 different medications and to adjust dosages for a further 30. They were also allowed to order laboratory tests. PNs specifically were not permitted to prescribe insulin but were able to adjust the dosage. Supervision, oversight: PNs worked with a protocol published in "protocollaire diabeteszorg". The protocol indicated when the PN had to consult the GP. In case the patient showed specific complaints during consultation, the patient would be referred to the GP.
Outcomes	Patient outcomes: HbA1c, BP, chol, chol/hdl, glycaemic control Blood pressure Lipid profile HRQOL Diabetes‐related symptoms Patients’ satisfaction Process of care measures: Referred to an ophthalmologist after not having visited one for the past 2 years, by whom measures were taken for feet at‐risk Referred to an internist for starting insulin therapy, after diabetic, antihypertensive, and/or lipid‐lowering drugs had been intensified Resource utilisation: Health care consumption (number of patient visits, number of contacts between PNs and GP)
Notes	Country: Netherlands Study period: unknown Nurse role: ongoing care for patients with diabetes type 2 in a primary care setting Nurse title: practice nurse Nurse educational background: EQF level 5 Nurse years of experience: 2 PNs, experienced in working as a nurse; however no prior experience working in general practice Nurse additional training: At the beginning of the trial, PNs received 1 week of training on a detailed treatment and management protocol aimed at optimising glucose, blood pressure, and lipid profile regulation and eye and foot care in patients with diabetes. Training aimed to educate PNs to a level comparable to the level of a GP, so they would be able to provide diabetes care without supervision.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	A random sequence was used; sequence generation was by odd/even number of closed envelopes. "Patients willing to participate were then randomised by two independent medical investigators (STH and NK)…... Subjects with even numbers were assigned to the intervention group, and those with odd numbers were assigned to the control group".
Allocation concealment (selection bias)	Low risk	Allocation was concealed using sequentially numbered closed envelopes "The patient population was randomised using non‐transparent, closed envelopes containing sequential numbers".
Baseline characteristics	Low risk	Characteristics of patients were similar in both groups. "The groups were comparable with respect to age, gender, T2DM duration, body mass index (BMI), blood pressure, HbA1c and lipid profile".
Baseline outcome measurement	Low risk	Baseline outcomes were reported and were similar for both groups, except feet at‐risk. One of the secondary outcomes was measures to prevent development of diabetic foot symptoms. The percentage of feet at‐risk cases was calculated. Therefore, we do not expect bias due to unsimilarity in baseline feet at‐risk. "The groups were comparable with respect to age, gender, T2DM duration, body mass index (BMI), blood pressure, HbA1c and lipid profile. However, more patients in the PN group had feet at‐risk compared to the GP group".
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	It is unclear whether the outcome was influenced by lack of blinding of patients and care providers.
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Not performed. It is unclear whether the outcome was influenced by lack of blinding of the outcome assessment, because outcomes could not be easily influenced. "The outcome assessors of the clinical variables (such as blood pressure) were not blinded to the intervention".
Incomplete outcome data (attrition bias) All outcomes	Low risk	Follow‐up of patients > 80%
Selective reporting (reporting bias)	Unclear risk	The protocol was not available.
Contamination	Unclear risk	Allocation was by patient. Only 1 practice was involved. Not reported whether trial authors protect against contamination
Bias due to lack of power	Unclear risk	Lack of power, according to the power calculation. If this really was biased, the outcome was unclear. However, trial authors discussed the following: "the required sample size to detect a 0Æ5%‐point difference in HbA1c was a total of 216 patients. Unfortunately, we only have a complete follow‐up of 206 patients. However, the difference in HbA1c ( confidence interval) between groups after 14 months was 0Æ042% (0Æ207;0Æ265). As the confidence interval does not include the possibility of a 0Æ5%‐point difference in HbA1c between groups, we are able to make the conclusions as hypothesised".

Iglesias 2013.

Methods	Randomised trial
Participants	1461 patients (total group), 708 control, 753 intervention; mean age in intervention group 39.0 (15.1), in control group 38.6 (14.5); 39.0% male in intervention group, 38.8% male in control group 142 GPs 155 nurses
Interventions	Intervention: care delivered by nurses to patients asking same‐day appointment Control: usual care delivered by GPs to patients asking same‐day appointment Detailed description of the intervention: Compared effectiveness of care delivered by nurses vs usual care delivered by GPs, in adult patients asking to be seen on the same day in primary care practices. Patients assigned to the intervention group were seen by trained nurses, who followed guidelines developed during the study's preparation phase. Nurses had access to an electronic application, which included the guidelines, designed as a decision‐making support tool. Patients assigned to the control group were seen by the GP, who followed the usual procedures established in the practice and did not have access to any kind of decision‐making support tools. Supervision, oversight: unknown
Outcomes	Patient outcomes: Resolution of symptoms Patient satisfaction Patient perception of the quality of information and care received Patient preference Process of care measures: Resolution by nurse Duration of the visit Resource utilisation: Drug prescriptions Sick leave Re‐visit in primary care for the same reason during the following 2 weeks Admission to hospital for the same reason
Notes	Country: Spain Study period: 5 months Nurse role: nurses trained to respond to low‐complexity, acute pathologies Nurse title: nurse Nurse educational background: EQF level unknown Nurse years of experience: unknown Nurse additional training: unknown
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	The sequence process included a random component. "Participants were randomly assigned following simple randomisation procedures to intervention or control using an automatic probabilistic function which assigns one group or another using a probability of 0.5. Patients were recruited consecutively until the necessary number of subjects was obtained, ensuring a balanced allocation of groups".
Allocation concealment (selection bias)	Low risk	Patients and investigators enrolling patients could not foresee assignment. "The application was used to implement the random allocation sequence. The sequence was concealed until groups were assigned because the application generated the sequence just after the patient gave oral and written consent to participate in the study".
Baseline characteristics	Low risk	Characteristics of patients were similar in both groups.
Baseline outcome measurement	Unclear risk	Outcome patient preference was not assessed before the intervention.
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	It is unclear whether the outcome was influenced by lack of blinding of patients and care providers. "Participants, nurses and GPs where not blinded to group assignment".
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Outcome assessors were blind. "The administrative staff member, who phoned the patients 15 days later to the first visit, where blinded to group assignment".
Incomplete outcome data (attrition bias) All outcomes	Low risk	Follow‐up of patients > 80% "Of the 1,461 randomised patients, 1,351 (92.5%) completed the study".
Selective reporting (reporting bias)	Unclear risk	The protocol was not available.
Contamination	Unclear risk	One of the outcomes was ‘level of resolution by nurses’. It is unclear what happened in the analysis with patients seen by both groups, owing to non‐resolution by nurses.
Bias due to lack of power	Low risk	Sufficient power "…obtaining a final sample size of 1,340 patients (670 per group)"

Larsson 2014.

Methods	Randomised trial
Participants	107 patients (total group). Mean age in intervention group 55.0 (12.3), in control group 55.8 (13.2); 45.0% male in intervention group, 44% male in control group 5 nurses Unknown number of rheumatologists
Interventions	Intervention: patients monitored by a nurse, later monitored by a rheumatologist Control: patients monitored by a rheumatologist Detailed description of the intervention: Compared and evaluated treatment outcomes at a nurse‐led rheumatology clinic and a rheumatologist‐led clinic in patients with low disease activity or in remission undergoing biological therapy. The intention was to replace one of the 2 annual rheumatologist monitoring visits by a nurse‐led rheumatology monitoring visit in patients undergoing biological therapy. Rheumatologist‐led clinic: Patients with CIA undergoing biological therapy were monitored by a rheumatologist every 6 months for 30 minutes to evaluate effects of the medication and to measure disease activity. The rheumatologist assessed disease activity by examining tender and swollen joints based on a 28‐joint count in addition to evaluating the results of laboratory tests. Nurse‐led rheumatology clinic: Patients were monitored for 30 minutes by a rheumatology nurse after 6 months, then for 30 minutes by a rheumatologist after 12 months. The nurse assessed patients' disease activity by examining tender and swollen joints based on the 28‐joint count in addition to evaluating results of laboratory tests in the same way as a rheumatologist. Drug treatment was discussed in terms of administration, adherence, side effects, and laboratory tests, as well as patients' global health. Supervision, oversight: If necessary, the nurse could contact the rheumatologist to ask for advice or to obtain a prescription.
Outcomes	Patient outcomes: Disease activity Perceived global health the previous week Physical difficulties in performing activities of daily living Pain Satisfaction with and confidence in obtaining rheumatology care Medication record Employment status Adverse events Resouce utilisation: Cortisone injections in addition to regular rheumatologist monitoring visits Blood tests Radiography Pharmacological therapy Additional telephone calls to a rheumatology nurse Additional telephone calls to a rheumatologist Additional rheumatologist visits Team rehabilitation in in‐patient settings Team rehabilitation in out‐patient settings Occupational therapist treatments Psychosocial treatments Specialist consultations Costs: total annual rheumatology care per patient
Notes	Country: Sweden Study period: 22 months Nurse role: nurse‐led rheumatology monitoring visit for patients undergoing biological therapy Nurse title: registered nurse Nurse educational background: EQF level 6 Nurse years of experience: 22 to 39 years' professional experience and 9 to 20 years' experience managing rheumatic diseases in both in‐patient and out‐patient rheumatology care Nurse additional training: Nurses had undergone special training provided by a rheumatologist and RA instructors to assess swollen and tender joints based on the 28‐joint count to make an evidence‐based assessment of disease activity. • Theoretical lecture about anatomy of the joint with pictures and about joint examination techniques – inspection, palpation, assessing range of motion and function for a half hour • Practical examination of the hand and wrist. Nurses were trained in groups of 2 to 3 by RA instructors (patient partners) who had RA themselves and were well educated. Time: 1.5 hours • Same procedure, but now foot and ankle. Time: 1.5 hours • All nurses also got a booklet about hand and wrist examination, and another about foot and ankle examination, for self‐study and training. • One week later, another 1.5‐hour lecture to repeat both hand and foot examinations in the same groups • Rheumatologist met the whole group of nurses and gave a lecture on how to examine the big joints ‐ shoulder, elbow, knee, and hip. Nurses examined an RA patient and then practiced on each other. Time: 1.5 hours • Time to ask the rheumatologist questions afterwards if needed, and to watch the rheumatologist examining other patients in the practice
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	The sequence process included a random component. "Randomization took the form of sealed envelopes containing assignment to one of the two groups. The envelopes were mixed and when a patient met the inclusion criteria, an envelope was randomly picked".
Allocation concealment (selection bias)	Low risk	Patients and investigators enrolling patients could not foresee assignment. "Randomization took the form of sealed envelopes containing assignment to one of the two groups. The envelopes were mixed and when a patient met the inclusion criteria, an envelope was randomly picked".
Baseline characteristics	Unclear risk	Characteristics of patients were similar in both groups, except in those with rheumatic disease. It is unclear whether this biased trial results.
Baseline outcome measurement	Low risk	Primary outcomes were assessed before the intervention. Mean differences were used as an outcome.
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	No information available It is unclear whether the outcome was influenced by possible lack of blinding of patients and care providers.
Blinding of outcome assessment (detection bias) All outcomes	High risk	Not performed It is unclear whether the outcome was influenced by lack of blinding of the outcome assessment. Assessment of disease activity may have been influenced by lack of blinding of the outcome assessor. "The monitoring by the rheumatology nurse (intervention group) and the rheumatologist (control group) included an assessment of the number of swollen and tender joints based on the DAS28". and "All patients were monitored by the rheumatologist at baseline and after 12 months".
Incomplete outcome data (attrition bias) All outcomes	Low risk	Follow‐up of patients > 80% "In total, 47 patients (89%) in the intervention group and 50 patients (93%) in the control group completed the 12‐month trial".
Selective reporting (reporting bias)	Unclear risk	The protocol was not available.
Contamination	Unclear risk	Not reported whether trial authors protect against contamination
Bias due to lack of power	Low risk	Sufficient power "Based on a change of 0.6 in the DAS28 score and a SD of 1.0, the power analysis demonstrated that 95 patients would be a sufficient number to detect a clinically moderate difference between groups at a 5% significance level with at least 90% power. It was decided to include 107 patients to allow for the predicted 10% dropout. The primary outcome measure was change in the DAS28 over a 12‐month period".

Lattimer 1998.

Methods	Randomised trial
Participants	10134 patients (total group), all ages, 48% male 6 nurses 55 doctors
Interventions	Intervention: nurse call management during out‐of‐hours Control: GP call management during out‐of‐hours Detailed description of the intervention: Compared 2 groups answering incoming phone calls for patients during out‐of‐hours Nurse telephone consultation: In the intervention arm of the trial, all calls were passed straight to the nurse, except in the case of immediate referral to the ambulance service by the receptionist. The nurse then undertook a systematic assessment of the caller's problem and recommended an appropriate course of action. The nurse was aided by TAS (telephone advice system), a computer‐based primary care call management system. Triage nurses were able to complete calls without onward referral. Call management options for nurses included: Telephone advice: on home management of the problem to see the patient’s own GP the next day to attend the Accident and Emergency Department Referral of the patient to the GP on duty: inviting the patient to attend the primary care centre advising the caller that the GP would contact them by telephone contacting the 999 ambulance service plus referral to the GP on duty referring to another agency (e.g. on call Community Psychiatric Nurse) plus referring to the GP on duty At the time of the study, triage nurses were seen to be acting as ‘competent agents’ of the GP. They had personal professional responsibility to ensure that they had been adequately prepared for the role and were accountable for their own actions. The GP could delegate care, but not accountability for that care. Doctor telephone consultation: Incoming phone calls were answered by a receptionist, who passed the message to a doctor. Call management options for the GP were: Telephone advice: on home management of the problem to see the patient’s own GP the next day to bring the patient to be examined at the primary care centre to take the patient to the accident and emergency department Examination of the patient at home or in the primary care centre with: advice on home management advice to see the patient's own GP the next day treatment admission to hospital Supervision, oversight: Nurses would refer calls to a GP if in doubt about how best to manage a situation, or would discuss the situation with the patient (in person at the centre or over the telephone). Before the end of every shift, triage nurses contacted the general practitioners on duty to report back on all calls they had managed. Formal, monthly professional supervision was provided by the trial project nurse.
Outcomes	Patient outcomes: Mortality Resource utilisation: Doctor workload Hospital referral and admission Emergency department visits Direct costs
Notes	Country: UK Study period: 3 to 7 days Nurse role: first contact care for patients with urgent problems out‐of‐hours Nurse title: not clear Nurse educational background: EQF level 6 Nurse years of experience: Nurses were required to have a minimum of 5 years of post registration experience, including experience in primary health care. Nurse additional training: 6‐week educational programme to prepare nurses for a 3‐month probationary period of supervised telephone triage practice. The taught component covered clinical skills (management of adult and child health problems and related pharmacology); telephone consultation (including professional and medicolegal aspects, communication, and interpersonal skills at different phases of the telephone encounter); assessment and decision‐making skills in telephone triage; approaches to managing a variety of situations on the telephone including ‘difficult’ calls using scenarios; skills in using the TAS system; and patient perspectives. Programme contributors were largely drawn from clinical GPs involved with the trial and academic staff. The programme comprised approximately 40 hours in total, with 20 hours taught over 6 weeks and 20 hours of individual practical work and assessment.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	The sequence process included a random component. "The trial year was divided into 26 blocks of two weeks. Within each block, one of each pair of matching out of hours periods ‐ for example, Tuesday evenings ‐ was randomly allocated to receive the intervention, the other being allocated to the normal service, by means of a random number generator on a Hewlett Packard 21S pocket calculator".
Allocation concealment (selection bias)	Low risk	Patients and investigators enrolling patients could not foresee assignment. "random number generator on a Hewlett Packard 21S pocket calculator"
Baseline characteristics	Low risk	Baseline characteristics were reported and were similar for both groups. "There were no substantial differences between the two trial groups".
Baseline outcome measurement	Unclear risk	Primary outcomes were not assessed before the intervention.
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	Personnel (low risk): "The complete pattern of intervention periods was known in advance only to the lead investigators and the trial coordinator. Nurses providing the intervention knew their shifts only after the duty roster for general practitioners providing out of hours care had been fixed. General practitioners were therefore blind to the intervention at the point at which they were able to choose or swap duty periods". Patients (unclear risk): no blinding; however it is unclear whether the outcome was influenced by lack of blinding of patients
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	No information
Incomplete outcome data (attrition bias) All outcomes	Low risk	Follow‐up of patients > 80%
Selective reporting (reporting bias)	Unclear risk	The protocol was not available.
Contamination	Unclear risk	No information
Bias due to lack of power	Low risk	Sufficient power. "..., we calculated that 5455 patients would be required in each arm of the trial using the formula described by Jones et al".

Lewis 1967.

Methods	Randomised trial
Participants	66 patients (total group), 16+ years, 12% male Unknown numbers of nurses and doctors
Interventions	Intervention: patients allocated to nurse‐led care Control: patients allocated to doctor‐led care Detailed description of the intervention: Compared 2 groups delivering care to patients with chronic illnesses: Nurse clinic: nurses as the primary source of care for adults with chronic illnesses (i.e. hypertensive cardiovascular disease; arteriosclerotic heart disease; exogenous obesity; psychophysiological reactions; and arthritis) Control: medicine clinic Supervision, oversight: unknown
Outcomes	Patient outcomes: Health status Provider preference Compliance with follow‐up attendance Resource utilisation: Direct costs
Notes	Country: USA Study period: 12 months Nurse role: ongoing primary care for patients with stable chronic disease Nurse title: not clear Nurse educational background: EQF level unknown Nurse years of experience: unknown Nurse additional training: unknown
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	No information
Allocation concealment (selection bias)	Unclear risk	No information
Baseline characteristics	Low risk	Baseline characteristics were reported and were similar for both groups. "There were no differences among the scores of the two groups on initial testing".
Baseline outcome measurement	Low risk	Baseline outcomes were reported and were similar for both groups.
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	Not performed It is unclear whether the outcome was influenced by lack of blinding of patients and care providers.
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	No information
Incomplete outcome data (attrition bias) All outcomes	Low risk	Follow‐up of patients > 80%
Selective reporting (reporting bias)	Unclear risk	The protocol was not available.
Contamination	High risk	Potential contamination represented by cross‐over of patients/clinicians between groups "On 95% of these occasions, patients were seen by the nurse alone".
Bias due to lack of power	Unclear risk	No power calculation performed

Moher 2001.

Methods	Randomised trial
Participants	1347 patients (total group), mean age 66 years, 69% male Unknown numbers of nurses and doctors in 21 practices
Interventions	Intervention: patients with coronary heart disease allocated to nurse‐led follow‐up Control: patients with coronary heart disease allocated to doctor‐led follow‐up Detailed description of the intervention: Compared 3 different interventions for improving secondary preventive care of patients with coronary heart disease delivered at the level of general practice: audit and feedback; recall to a general practitioner; and recall to a nurse clinic Audit and feedback (audit group) ‐ Practices were given summary audit results at a practice meeting (1 practice requested written material only). Results presented were numbers of patients with myocardial infarction, angina, and revascularisation; prevalence of identified coronary heart disease in the practice; and proportions of patients with “adequate assessment” and treatment with antiplatelet drugs, hypotensive agents, and lipid‐lowering drugs. Anonymised data from other practices in the study were given for comparison. Practices were asked to provide usual care and were given no further support during the trial. Recall to general practitioner (GP recall group) ‐ Practices were given the same patient information as was given to the audit group but were also given the names of patients identified as having coronary heart disease. Guidelines for secondary prevention were discussed and agreed upon with practice doctors and provided ongoing support in setting up a register and recall system for regular review of patients with coronary heart disease by their general practitioner. Recall to nurse clinic (nurse recall group) ‐ Practices were given the same patient information as was given to the GP recall group. The trial's nurse facilitator gave ongoing support to the practices in setting up a register and recall system for systematic review of patients with coronary heart disease in a nurse‐led clinic. After discussion of and agreement on guidelines for secondary prevention, practice doctors and nurses agreed on the clinical protocol, and nurses were taught how to implement it. Supervision, oversight: unknown
Outcomes	Patient outcomes: Cardiovascular risk factors Process of care: Adherence to guidelines Resource utilisation: Prescriptions
Notes	Country: UK Study period: 18 months Nurse role: ongoing primary care for patients with coronary heart disease Nurse title: practice nurse Nurse educational background: EQF level unknown Nurse years of experience: unknown Nurse additional training: Nurses received education on how to implement the clinical protocol.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	No information
Allocation concealment (selection bias)	Unclear risk	No information
Baseline characteristics	Low risk	Baseline characteristics were reported and were similar for both groups. "Characteristics of the patients were similar in the three trial groups". "At baseline about 30% of patients were adequately assessed overall".
Baseline outcome measurement	Low risk	Baseline outcomes were reported and were similar for both groups. Effect sizes were adjusted for baseline.
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	Not performed It is unclear whether the outcome was influenced by lack of blinding of patients and care providers.
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	No information
Incomplete outcome data (attrition bias) All outcomes	Low risk	Follow‐up of patients > 80% "Only patients were included who were alive and registered with the practice at follow up".
Selective reporting (reporting bias)	Unclear risk	The protocol was not available.
Contamination	Low risk	Allocation was by practice. "at the level of general practice"
Bias due to lack of power	Unclear risk	No power calculation performed

Mundinger 2000.

Methods	Randomised trial
Participants	1316 patients (total group), mean age 44.5 years, 25.5% male 7 nurses 17 doctors
Interventions	Intervention: patients allocated to nurse‐led care Control: patients allocated to doctor‐led care Detailed description of the intervention: Compared NPs and doctors as primary care providers within a conventional medical care framework at the same medical centre, where all other elements of care were identical. NPs provided all ambulatory primary care, including 24‐hour call, and made independent decisions for referrals to specialists and hospitalisations. NPs and doctors had the same authority to prescribe, consult, refer, and admit patients. Furthermore, they used the same pool of specialists, in‐patient units, and emergency departments. Supervision, oversight: MD supervision of NPs was consistent with New York State and hospital regulations: In New York State, NPs have a written agreement with an MD that states the MD will meet with the NP once or twice a year to review any practice issues, or to discuss certain cases. No on‐site or regular "supervision" is provided. In terms of hospitals in New York State, and an MD must sign off on every hospital admission within 24 hours of admission, but this still allows an NP with privileges to independently admit and care for a patient.
Outcomes	Patient outcomes: Health status Satisfaction Process of care: Care given by providers Resource utilisation: Consultation rate Use of other services ‐ hospital admissions, emergency department visits, specialty visits
Notes	Country: USA Study period: 2 years Nurse role: first contact and ongoing primary care Nurse title: nurse practitioners Nurse educational background: EQF level 7 Nurse years of experience: average of 8 to 10 years of experience for NPs in the study Nurse additional training: Additional training was received from MDs in hospital‐based activities, including how to admit and bring necessary resources to the patient (specialists, radiology, lab work, etc); training was also provided in interpreting tests and conducting emergency department evaluations.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	No method of randomisation was clearly reported. "Those who provided informed consent were randomly and blindly assigned to either the nurse practitioner or 1 of the physician practices. Different assignment ratios were used during the recruitment period. Initially the ratio was 2:1, with more patients assigned to the nurse practitioner practice, because it opened after the physician practices and was able to accept more new patients. Subsequently, the ratio was changed to 1:1 as the nurse practitioner practice’s patient panel increased".
Allocation concealment (selection bias)	Unclear risk	Method of concealment was not described in sufficient detail.
Baseline characteristics	Low risk	Most baseline characteristics were reported and were similar for both groups. "With regard to demographic characteristics, groups are similar with exception: Significant more patients Medicaid enrolled in physician group (95.7%) versus 87.4% nurse group; p = 0.004".
Baseline outcome measurement	Low risk	Baseline outcomes were reported and were similar for both groups.
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	It is unclear whether the outcome was influenced by lack of blinding of patients and care providers. "Those who provided informed consent were randomly and blindly assigned to either the nurse practitioner or 1 of the physician practices". "Patients were told which provider group they were assigned to after randomisation, and the type of provider could not be masked during the course of care".
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	No information.
Incomplete outcome data (attrition bias) All outcomes	High risk	Follow‐up of patients < 80% (Figure 1, page 64) 1316 were enrolled, 1040 completed 6‐month interview (79%). "Only 406 of the original eligible patients are included, as these patients were the only ones who still received care from original provider". "The number varied per measure from 77 to 119". 77/145 = 53.1%; 119/145 = 82.06%
Selective reporting (reporting bias)	Unclear risk	The protocol was not available.
Contamination	High risk	Contamination could have occurred. "The 159 patients (12.1%) who, after the first visit, either went to a clinic other than the one assigned or to multiple primary care clinics were maintained in the initially assigned group for the analyses, consistent with an intent‐to‐treat analysis. All analyses were repeated without these 159 patients, and the results were the same". Art. Lenz et al: "The present analysis is limited to the 406 patients who received primary care only from the assigned practice and made at least one follow‐up visit to that practice during the 2 years following the initial visit. This subsample was the only one that received the treatment as assigned and in which the effect of the treatment could be isolated".
Bias due to lack of power	Low risk	Sufficient power

Ndosi 2013.

Methods	Randomised trial
Participants	181 patients (total group), 91 intervention group, 90 control group; mean age in intervention group 60.2 (11.3), in control group 57.3 (12.2); 26.5% male in intervention group, 25.7% male in control group 9 nurses 10 doctors (rheumatologists)
Interventions	Intervention: RA patients allocated to nurse‐led care Control: RA patients allocated to rheumatologist care Detailed description of the intervention: Compared 2 groups providing care to patients with a positive diagnosis of RA Nurse‐led care: included allocated 30‐minute time slots in which the nurse took history, performed physical examination, provided pain control, prescribed or recommended medication and dosage changes, administered intra‐articular or intramuscular steroid injections, provided patient education and psychosocial support, and ordered blood tests or x‐rays. Referrals for ward admission, to the rheumatologist or to other healthcare professionals, were carried out as appropriate. Rheumatologist care: The usual RLC is similar to the above, except that it usually involves an allocated 15‐minute time slot. Supervision, oversight: Rheumatology nurse‐led clinics were autonomous but were conducted alongside rheumatologist‐led clinics; therefore, a rheumatologist was available on‐site and could be consulted.
Outcomes	Patient outcomes: DAS28 Pain Fatigue Duration of morning stiffness Quality of life Disability Hospital anxiety Depression Arthritis self‐efficacy Satisfaction Resource utilisation: Costs: EQ5D, costs applied to units of resource use
Notes	Country: UK Study period: 4 years Nurse role: ongoing care for patients with rheumatological arthritis Nurse title: clinical nurse specialist Nurse educational background: EQF level 7 Nurse years of experience: The nurse had a median experience of 10 years in their current post and had experience in running nurse‐led clinics. Nurse additional training: none
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	The sequence process included a random component. "Randomisation was on a 1 : 1 basis to either NLC (experimental group) or rheumatologist‐led care (RLC) (control group), by random permuted blocks, using the stratification factors, centre and DAS28 (low disease activity DAS28 ≤ 3.2, or moderate to high disease activity DAS28 > 3.2)".
Allocation concealment (selection bias)	Low risk	Patients or investigators enrolling patients could not foresee assignments, because a random permuted block method was used.
Baseline characteristics	Low risk	Characteristics of patients were similar in both groups, except DMARD. In the analyses, trial authors corrected for DMARD. "The demographics and baseline characteristics of patients under NLC (n = 91) were comparable to those under RLC (n = 90) except in the proportion of patients receiving biological disease‐modifying antirheumatic drugs (DMARD)". "The baseline difference in the proportion of patients receiving biological DMARD was a result of chance (not systematic). In the follow‐up period, the proportion of patients receiving biological agents in NLC remained more or less constant while that in RLC doubled. Assuming that change onto biological agents would significantly improve DAS28, this was likely to favour RLC. Predictably, additional adjustment for baseline biological agents increased the effects on NLC".
Baseline outcome measurement	Low risk	Primary outcomes were assessed before the intervention.
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	It is unclear whether the outcome was influenced by lack of blinding of patients and care providers.
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Outcome assessor was blind. "The independent assessors, performing the joint counts for DAS28, were masked".
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Follow‐up of patients < 80% However, intention‐to‐treat and per‐protocol analyses were performed. Differences in outcomes were reported. "Of the 622 patients who were assessed for eligibility, 181 were eventually randomly assigned and 133 (73.5%) had complete DAS28 data for all the five visits (PP analysis)".
Selective reporting (reporting bias)	Low risk	The protocol was available.
Contamination	Low risk	One patient crossed over. It seems that the patient crossing over was registered; therefore no further contamination took place.
Bias due to lack of power	Low risk	Sufficient power "Allowing for a 10% participant dropout rate, a total sample size of 180 participants (90 per treatment arm) was needed on the basis of a repeated‐measures analysis of between‐group differences averaged over four equidistant follow‐up time points given 90% power and one‐sided statistical testing with 2.5% significance level (with anticipated SD of 1.5, intraclass correlation coefficient of 0.5)".

Sanne 2010.

Methods	Randomised trial
Participants	812 patients (total group), gender unknown 4 nurses 4 medical officers
Interventions	Intervention: patients with HIV allocated to nurses Control: patients with HIV allocated to medical officers Detailed description of the intervention: Compared nurse‐ vs doctor‐monitored HIV care. All patients were managed under South African National Guidelines for HIV treatment and were given standard ART regimens. Supervision, oversight: unknown
Outcomes	Patient outcomes: Mortality Failure (virological failure, toxicity failure, study losses) Satisfaction
Notes	Country: South Africa Time period: 47 months Nurse role: primary healthcare nurses Nurse type: primary healthcare nurses Nurse educational background: EQF level unknown Nurse years of experience: unknown Nurse additional training: unknown
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	The sequence process included a random component. "Participants were randomly assigned in a ratio of 1:1 within sites. Randomisation lists were generated centrally with a stratified permuted block randomisation (with blocks of six). The strata corresponded to the different study sites".
Allocation concealment (selection bias)	Low risk	Patients and investigators enrolling patients could not foresee assignment. "The allocation codes for a particular site were sealed in sequentially numbered envelopes, reflecting their order on the randomisation list, and distributed to the site. At randomisation, the site pharmacist unsealed the sequential envelope to reveal the randomisation code and participant randomisation number".
Baseline characteristics	Low risk	Characteristics of patients were similar in both groups.
Baseline outcome measurement	Low risk	Baseline outcome measurement was not relevant. "The primary study outcome was a composite endpoint of possible treatment‐limiting events that could occur on first‐line ART".
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	Not performed It is unclear whether the outcome was influenced by lack of blinding of patients and care providers. "Neither the participant nor those analysing the data were masked to the assignment".
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Not performed It is unclear whether the outcome was influenced by lack of blinding of the outcome assessment. "Neither the participant nor those analysing the data were masked to the assignment".
Incomplete outcome data (attrition bias) All outcomes	Low risk	Follow‐up of patients > 80% "917 patients were assessed for eligibility, 105 excluded. Of excluded patients, 16 refused to participate and 89 did not meet inclusion criteria. There were 10 lost to follow‐up in the nurse group and 14 lost to follow‐up in the doctor group". Trial authors did not mention the reason for loss to follow‐up, but all patients were included in primary outcome analysis.
Selective reporting (reporting bias)	Unclear risk	The protocol was not available.
Contamination	Low risk	It is unlikely that both groups were contaminated. "To limit contamination between randomised groups, work activity and monitoring schedules were separated with routine visits scheduled on different days of the week, although at least one clinician was available to undertake unscheduled visits in the other group of the study".
Bias due to lack of power	Low risk	Sufficient power "The sample size was calculated based on an 18‐month accrual and 96 weeks’ follow‐up with 80% power and α of 0·05. Because we did not record significant household clustering, enrolment was able to be discontinued after 812 patients with no compromise of pre‐established study power".

Shum 2000.

Methods	Randomised trial
Participants	1815 patients (total group), mean age 27.5 years, 40% male 5 nurses 19 doctors
Interventions	Intervention: patients allocated to nurse Control: patients allocated to doctor Detailed description of the intervention: Compared acceptability and effectiveness of a practice‐based minor illness service led by nurses versus routine care offered by general practitioners. Nurses managed patient care and took the history, performed a physical examination, offered advice and treatment, issued prescriptions (which required a doctor's signature), and referred the patient to the doctor when appropriate. Supervision, oversight: Patients seen by a nurse were referred to a general practitioner when appropriate.
Outcomes	Patient outcomes: Health status Satisfaction Provider preference Process of care: Provision of information Resource utilisation: Length of consultation Return visits Prescriptions Emergency department visits Use of out‐of‐hour services
Notes	Country: UK Study period: 2 weeks Nurse role: first contact care for patients with urgent problems Nurse title: practice nurse Nurse educational background: EQF level unknown Nurse years of experience: average of 8.4 (3.8) years of experience in practice nursing Nurse additional training: 3‐month academically accredited degree level course on managing minor illnesses. Nurses attended one half‐day a week of formal group teaching by a nurse practitioner and were taught twice a week by general practitioners during routine surgeries in the practice where the nurses worked.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Sequence was generated by non‐resealable opaque envelopes. "Allocation to being seen by a doctor or nurse was determined using random permuted blocks of four with sequentially numbered, non‐resealable opaque envelopes".
Allocation concealment (selection bias)	Low risk	Allocation was concealed by sequentially numbered, non‐resealable opaque envelopes. "Allocation to being seen by a doctor or nurse was determined using random permuted blocks of four with sequentially numbered, non‐resealable opaque envelopes".
Baseline characteristics	Low risk	Baseline characteristics were reported and were similar for both groups. "The two groups of patients were comparable in terms of age, sex, the number who usually preferred to see a female doctor rather than a male, and their reported rates of consultation in the previous 12 months (table 1)". Baseline outcome measures were not relevant.
Baseline outcome measurement	Unclear risk	Primary outcomes were not assessed before the intervention.
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	It is unclear whether the outcome was influenced by lack of blinding of patients and care providers.
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	No information
Incomplete outcome data (attrition bias) All outcomes	Low risk	For most outcomes, follow‐up was > 80%. Follow‐up for satisfaction questionnaire was > 75%, for mailed questionnaire 76%.
Selective reporting (reporting bias)	Unclear risk	The protocol was not available.
Contamination	Unclear risk	No information
Bias due to lack of power	Low risk	Sufficient power "It was calculated that 1060 valid responses would be sufficient to detect an effect size of 0.2 SD at the 95% confidence level with a power of 90% using two tailed tests".

Spitzer 1973.

Methods	Randomised trial
Participants	4325 patients (total group), all ages, 42.5% male 2 nurses 2 doctors
Interventions	Intervention: families allocated to nurse Control: families allocated to doctor Detailed description of the intervention: not available Supervision, oversight: unknown
Outcomes	Patient outcomes: Health status Satisfaction Provider preference Process of care: Standards of care Resource utilisation: Direct costs
Notes	Country: Canada Study period: 12 months Nurse role: first contact and ongoing primary care Nurse title: nurse practitioners Nurse educational background: EQF level unknown Nurse years of experience: unknown Nurse additional training: unknown
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	No information
Allocation concealment (selection bias)	Unclear risk	Method of concealment was not described in sufficient detail. "Because a case load half that of a family physician's was considered manageable for nurse practitioner, the eligible families were stratified by practice of origin, and randomly allocated in a ration of 2:1. They formed a randomized conventional group, assigned to continuing primary clinical services from a family physician and a conventional nurse, and a randomized nurse‐practitioner group, whose first‐contact primary clinical services were to be provided by a nurse practitioner".
Baseline characteristics	Low risk	Baseline characteristics were reported and were similar for both groups. "As determined in the 1971 household survey, the patients in the conventional and nurse‐practitioner groups had highly similar values for physical function, ability to carry out usual daily activities and freedom from bed disability the baseline health status of the two groups of patients showed only minor differences that were not statistically significant (at an alpha level of 0.05)". Figure 1: Baseline outcome variable was measured. "Physical status of patients in surveys during baseline and comparison periods"
Baseline outcome measurement	Low risk	Baseline outcomes were reported and were similar for both groups.
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	It is unclear whether the outcome was influenced by lack of blinding of patients and care providers.
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	No information
Incomplete outcome data (attrition bias) All outcomes	Low risk	Follow‐up of patients > 80% "The resulting cohort that was successfully interviewed in both years included 817 patients, with 296 in the experimental group and 21 in the conventional control group. The referral rates in the surveys were 11% in 1971 and 5% in 1972".
Selective reporting (reporting bias)	Unclear risk	The protocol was not available.
Contamination	High risk	It is likely that both groups were contaminated, because randomisation was by families within a practice. Not reported whether they protect against contamination
Bias due to lack of power	Unclear risk	No power calculation performed

Venning 2000.

Methods	Randomised trial
Participants	1316 patients (total group), all ages, 42% male 20 nurses Unknown number of doctors
Interventions	Intervention: patients allocated to nurse Control: patients allocated to doctor Detailed description of the intervention: Compared care given by general practitioners and nurse practitioners for patients requesting a same‐day appointment Supervision, oversight: unknown
Outcomes	Patient outcomes: Health status Satisfaction Compliance with follow‐up attendance Enablement Process of care: Examinations Resource utilisation: Length of consultation Return visits Prescriptions Investigations Use of other services ‐ hospital referral Direct costs
Notes	Country: UK Study period: 2 weeks Nurse role: first contact care for patients with urgent problems Nurse title: nurse practitioners Nurse educational background: EQF levels 5, 6, and 7 Nurse years of experience: The median length of time nurses had been qualified as nurse practitioners was 3 (range 1 to 5) years, and the median time as registered nurses was 22 (9 to 35) years. Each nurse practitioner had been seeing patients as first point of contact for at least 2 years. Nurse additional training: unknown
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	The sequence process included a random component. "In each practice, experimental sessions were booked when both the nurse practitioner and a general practitioner had appointments available for patients who asked to be seen on the same day. Patients were eligible for entry to the study if they requested an appointment the same day and were able to come to the experimental session. If these conditions were satisfied, the receptionist then asked patients whether they would agree to be randomised to see either a[n] NP or a GP. A method of coded block randomisation was developed which meant that neither the receptionist nor the patient could determine the group to which a patient had been allocated at the time of booking. The coded blocks were generated from random number tables. The randomisation code was broken by one of the researchers at the start of each experimental session, at which point it became apparent which patient would see which practitioner. Randomization continued until a minimum of 60 patients in each practice had been allocated to the clinician groups".
Allocation concealment (selection bias)	Low risk	Patients and investigators enrolling patients could not foresee assignment. "Method of coded block randomisation was developed which meant that neither the receptionist nor the patient could determine the group to which a patient had been allocated at the time of booking. The coded blocks were generated from random number tables. The randomisation code was broken by one of the researchers at the start of each experimental session, at which point it became apparent which patient would see which practitioner".
Baseline characteristics	Low risk	Baseline characteristics were reported and were similar for both groups.
Baseline outcome measurement	Unclear risk	No baseline outcome measurement was performed.
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	GPs and NPs were not blinded. Patients were not blinded. It is unclear whether the outcome was influenced by lack of blinding of patients and care providers.
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	No information
Incomplete outcome data (attrition bias) All outcomes	High risk	Follow‐up of patients < 80%
Selective reporting (reporting bias)	Unclear risk	The protocol was not available.
Contamination	Unclear risk	No information Allocation on patient level
Bias due to lack of power	Unclear risk	No power calculation performed

Voogdt‐Pruis 2010.

Methods	Randomised trial
Participants	1626 patients (1626 randomised, 701 trial population); 64% male 6 practice nurses 25 GPs
Interventions	Intervention: patients at cardiovascular risk allocated to practice nurses Control: patients at cardiovascular risk allocated to GPs Detailed description of the intervention: Compared 2 groups following the Dutch guideline for cardiovascular risk management. Patients in the practice nurse group had a consultation with the practice nurse for assessment of other risk factors, and a 3‐monthly monitoring schedule was set up for patients but was adjusted individually according to the risk profile, (co)morbidity, and patient preferences. Patients could be referred to other professionals, such as a dietician. Substitution involved the following tasks: Risk assessment Interventions needed: advice on lifestyle, referral to dietician or other professional, adjustment of medical therapy Supervision, oversight: unknown
Outcomes	Patient outcomes: Blood pressure Cholesterol BMI Smoking Satisfaction Patient adherence to medical treatment after 1 year of follow‐up Patient lifestyle after 1 year of follow‐up Process of care: Lifestyle and medical interventions Asking about the use of medication Resource use: Referral to professionals Visiting a cardiovascular specialist Admission into hospital because of CVD
Notes	Country: Netherlands Study period: not clear (1 measurement at 1 year with an unclear total period of the study) Nurse role: health education: secondary prevention consultation for patients with cardiovascular disease Nurse title: practice nurse Nurse educational background: EQF level 5 Nurse years of experience: unknown Nurse additional training: All nurses received a 1‐day course on motivational interviewing and shared decision‐making.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk
Allocation concealment (selection bias)	Low risk
Baseline characteristics	Low risk
Baseline outcome measurement	Low risk	Baseline outcomes were reported. "The marginal mean is controlled for health care centre, baseline risk factors, and other confounders".
Blinding of participants and personnel (performance bias) All outcomes	Low risk
Incomplete outcome data (attrition bias) All outcomes	Low risk
Contamination	High risk
Bias due to lack of power	Low risk

Outcomes: If we noted a difference in incomplete outcome data or baseline outcome measurement for different outcomes in the studies, we described these as support for judgement.

ART: antiretroviral therapy.

BP: blood pressure.

CCDS: computerised clinical decision support.

Chol: cholesterol.

CVD: cardiovascular disease.

DAS28: disease activity score 28.

DMARD: disease‐modifying antirheumatic drug.

EQ‐5D: EuroQoL Group Quality of Life Questionnaire.

EQF: European Qualifications Framework.

GI: gastrointestinal.

GNP: geriatric nurse practitioner.

GORD: gastroesophageal reflux disease.

GP: general practitioner.

HbA1c: glycated haemoglobin.

Hdl: high‐density lipoprotein.

HIV: human immunodeficiency virus.

HRQOL: health‐related quality of life.

NLC: nurse‐led care.

NP: nurse practitioner.

NUD: non‐ulcer dyspepsia.

PN: practical nurse.

RA: rheumatoid arthritis.

RLC: rheumatologist‐led care.

SF‐12: Short Form questionnaire.

T2DM: type 2 diabetes mellitus.

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Chambers 1977	CBA design
Flynn 1974	non‐randomised study
Gordon 1974	CBA design
Irewall 2015	Setting: mixed primary healthcare and hospital care
Kinnersley 2000	non‐randomised study
Kuethe 2011	Setting: mixed primary care and hospital medicine
McIntosh 1997	Aimed at mental health problems (alcohol abuse and addiction)
Myers 1997	CBA design
Stein 1974	non‐randomised study

CBA: controlled before‐after study.

Characteristics of studies awaiting assessment [ordered by study ID]

Lewis 2016.

Methods	Cluster randomised trial
Participants	40 patients in intervention group. Mean age: 40 (8.4). 65 patients in control group. Mean age: 42 (8,5) 80% male in intervention group, 74% male in control group
Interventions	Intervention: patients allocated to nurse‐initiated antiviral therapy Control: patients allocated to doctor‐initiated antiviral therapy Detailed description of the intervention: Patients without contraindications to nurse‐led therapy were offered immediate antiviral therapy administered by their Blood Borne Virus Team nurse in their outreach clinic without physician assessment. Patients who did not fulfil the safety criteria for the ‘nurse led’ treatment arm were referred to one of the specialist addiction units for treatment, that is, were managed according to current standard of care. Supervision, oversight: unknown
Outcomes	Proportion of participants initiating treatment during follow‐up Adherence Side effects of the treatment Adverse events
Notes	Country: UK Study period: 24‐48 weeks Nurse role: administration of antiviral therapy Nurse title: Blood Borne Virus nurses Nurse educational background: unknown Nurse years of experience: unknown Nurse additional training: unknown

Differences between protocol and review

Because of the large number of available randomised trials, we excluded non‐randomised trials from this review (Laurant 2000).

In the next update, we will consider again (1) including subgroup analyses on differently trained nurses related to the level of training of participating nurses; and (2) pooling cost data from studies that reported costs.

We performed no statistical testing for funnel plot asymmetry, as none of the pooled outcomes included more than 10 studies. If more than 10 studies would be included, we will follow the recommendations provided in the Cochrane Handbook for Systematic Reviews of Interventions (Chapter 10.4.3.1.).

Change in authorship: We have added Anneke van Vught, Mieken van der Biezen, Nancy Wijers, Kanokwaroon Watananirun, and Evangelos Kontopantelis to the review author list. We have removed D. Reeves, R. Hermes, J. Braspenning, R. Grol, and B. Sibbald from the list of review authors (see Acknowledgements).

Contributions of authors

This review update was led by AvV and ML. ML, MvdB, NW, and AvV assessed studies for inclusion. ML, MvdB, NW, and AvV participated in data extraction and contributed to data analysis. AvV undertook the meta‐analysis with assistance from the other review authors. ML, MvdB, and AvV drafted the review, drawing on contributions from several review authors, and all review authors commented on this draft.

Sources of support

Internal sources

• Centre for Quality of Care Research (WOK), University of Nijmegen, Netherlands.

• National Primary Care Research and Development Centre (NPCRDC), University of Manchester, UK.

External sources

• Ministry of Health, Welfare and Sports, Netherlands.

• Department of Health, UK.

• The Effective Health Care Research Consortium which is funded by UK aid from the UK Government for the benefit of developing countries, UK.

Declarations of interest

Miranda Laurant: none known.

Mieke van der Biezen: none known.

Nancy Wijers: none known.

Kanokwaroon Watananirun: none known.

Evangelos Kontopantelis: none known.

Anneke JAH van Vught: none known.

Edited (no change to conclusions)