Hospital nurse‐staffing models and patient‐ and staff‐related outcomes

Abstract

Background

Nurses comprise the largest component of the health workforce worldwide and numerous models of workforce allocation and profile have been implemented. These include changes in skill mix, grade mix or qualification mix, staff‐allocation models, staffing levels, nursing shifts, or nurses’ work patterns. This is the first update of our review published in 2011.

Objectives

The purpose of this review was to explore the effect of hospital nurse‐staffing models on patient and staff‐related outcomes in the hospital setting, specifically to identify which staffing model(s) are associated with: 1) better outcomes for patients, 2) better staff‐related outcomes, and, 3) the impact of staffing model(s) on cost outcomes.

Search methods

CENTRAL, MEDLINE, Embase, two other databases and two trials registers were searched on 22 March 2018 together with reference checking, citation searching and contact with study authors to identify additional studies.

Selection criteria

We included randomised trials, non‐randomised trials, controlled before‐after studies and interrupted‐time‐series or repeated‐measures studies of interventions relating to hospital nurse‐staffing models. Participants were patients and nursing staff working in hospital settings. We included any objective reported measure of patient‐, staff‐related, or economic outcome. The most important outcomes included in this review were: nursing‐staff turnover, patient mortality, patient readmissions, patient attendances at the emergency department (ED), length of stay, patients with pressure ulcers, and costs.

Data collection and analysis

We worked independently in pairs to extract data from each potentially relevant study and to assess risk of bias and the certainty of the evidence.

Main results

We included 19 studies, 17 of which were included in the analysis and eight of which we identified for this update. We identified four types of interventions relating to hospital nurse‐staffing models:

‐ introduction of advanced or specialist nurses to the nursing workforce;

‐ introduction of nursing assistive personnel to the hospital workforce;

‐ primary nursing; and

‐ staffing models.

The studies were conducted in the USA, the Netherlands, UK, Australia, and Canada and included patients with cancer, asthma, diabetes and chronic illness, on medical, acute care, intensive care and long‐stay psychiatric units. The risk of bias across studies was high, with limitations mainly related to blinding of patients and personnel, allocation concealment, sequence generation, and blinding of outcome assessment.

The addition of advanced or specialist nurses to hospital nurse staffing may lead to little or no difference in patient mortality (3 studies, 1358 participants). It is uncertain whether this intervention reduces patient readmissions (7 studies, 2995 participants), patient attendances at the ED (6 studies, 2274 participants), length of stay (3 studies, 907 participants), number of patients with pressure ulcers (1 study, 753 participants), or costs (3 studies, 617 participants), as we assessed the evidence for these outcomes as being of very low certainty. It is uncertain whether adding nursing assistive personnel to the hospital workforce reduces costs (1 study, 6769 participants), as we assessed the evidence for this outcome to be of very low certainty. It is uncertain whether primary nursing (3 studies, > 464 participants) or staffing models (1 study, 647 participants) reduces nursing‐staff turnover, or if primary nursing (2 studies, > 138 participants) reduces costs, as we assessed the evidence for these outcomes to be of very low certainty.

Authors' conclusions

The findings of this review should be treated with caution due to the limited amount and quality of the published research that was included. We have most confidence in our finding that the introduction of advanced or specialist nurses may lead to little or no difference in one patient outcome (i.e. mortality) with greater uncertainty about other patient outcomes (i.e. readmissions, ED attendance, length of stay and pressure ulcer rates). The evidence is of insufficient certainty to draw conclusions about the effectiveness of other types of interventions, including new nurse‐staffing models and introduction of nursing assistive personnel, on patient, staff and cost outcomes. Although it has been seven years since the original review was published, the certainty of the evidence about hospital nurse staffing still remains very low.

Plain language summary

What do we know about the impact of hospital nurse staffing on patients, staff and the costs of care?

What is the aim of this review?

The aim of this Cochrane Review was to find out if changes made to nurse staffing in hospitals improve outcomes for patients or nurses, or have an impact on the cost of health care. Nurse staffing can refer to the number of nurses per patient, the mix of different types of nurses in a hospital unit, or models used to allocate nurses to patients in a hospital unit.

Key messages

The research relating to hospital nurse staffing is very limited and the findings should be treated with caution.

It is unlikely that adding nurses with advanced nursing skills (Nurse Practitioners (NPs)) or with expertise in a particular area of practice (Clinical Nurse Specialists (CNSs)) to hospital nurse staffing makes any difference to patient death rates. We cannot be sure what other effect it might have on patients, for example, if it reduces the time patients spend in hospital or the costs of patient care. We cannot be sure if changes to the way in which nurses are allocated to patient care reduces the numbers of nurses resigning, or if introducing unqualified nurses to the nursing workforce reduces costs, as the research here is very limited too.

What was studied in the review?

We found studies that looked at the effects of four main strategies or models of nurse staffing: adding advanced or specialist nurses to the nursing workforce, introducing less‐qualified nursing personnel to the nursing workforce, changing the way in which nurses are allocated within a hospital unit to provide patient care, and changing the way hospital units schedule nursing shifts. We were most interested in the impact of these interventions on seven main outcomes: nursing‐staff resignations (turnover), patient deaths, patients being readmitted following discharge from the hospital, patients attending the Emergency Department (ED) for care following discharge, the number of days patients stayed in the hospital, the number of patients with pressure sores, and the costs of care.

What are the main results of the review?

We found 11 studies where advanced or specialist nurses were added to the nursing workforce. None of the studies reported the impact of this intervention on nursing‐staff resignations; three studies found that it may make little or no difference to patient deaths. We cannot be sure whether this intervention has an effect on reducing the number of patients being readmitted following discharge from hospital or attending an ED for care after discharge because the research is very limited. As well, we are uncertain about its effect on reducing the number of days patients stayed in the hospital, the number of patients with pressure sores, or healthcare costs, again because the research is very limited.

We found one relevant study that looked at adding nursing assistants to the nursing workforce, which was aimed at reducing costs. We cannot be sure about the effect on costs as the research is very limited.

We found five studies of primary nursing (where one nurse is responsible for the total care of a number of patients 24 hours a day, seven days a week) and two studies of nurse‐staffing models. One nurse‐staffing model study tested hospital units scheduling their own nursing shifts (self‐staffing), and the other study compared different ways to schedule nursing shifts. We cannot be sure about the impact of primary nursing or nurse‐staffing models on nurse resignations or costs because the research is very limited.

How up‐to‐date is this review?

The review authors searched for studies that had been published up to March 2018.

Summary of findings

Summary of findings for the main comparison. The introduction of advanced or specialist nurses to the nursing workforce versus usual staffing.

The introduction of advanced or specialist nurses to the nursing workforce versus usual staffing
Patient or population: medical patients and patients with cancer, asthma, diabetes, heart failure and chronic illness Setting: hospitals in the USA, UK and Australia Intervention: adding advanced or specialist nurses to nursing staff Comparison: usual nurse staffing
Outcomes	Impact	№ of participants (studies)	Certainty of the evidence (GRADE)
Nursing‐staff turnover	No studies reported this outcome.	‐	‐
Patient mortality	May make little or no difference to inpatient mortality or mortality within 30 days of discharge or to mean survival rates for patients receiving palliative care.	1358 (3 randomised trials)	⊕⊕⊝⊝ Low a, b
Patient readmissions	Two studies reported a reduction in total readmissions and in disease‐specific readmissions. Three studies found little or no difference between groups for readmission. When the data were combined from two studies, patients receiving the intervention were more likely to be readmitted within 30 days of discharge (OR 1.52, 95% CI 1.04 to 2.21). We are uncertain whether this intervention reduces readmissions because the certainty of the evidence is very low.	2995 (5 randomised trials, 1 non‐randomised trial, 1 CBA study)	⊕⊝⊝⊝ Very low b, c, d
Patient attendances at the ED	All studies reported little or no difference, but when data from two studies were combined, patients in the intervention group had a higher risk of attending the ED within 30 days of discharge (OR 1.20, 95% CI 0.82 to 1.76). However, we are uncertain whether this intervention increases or reduces patient attendances at the ED because the certainty of the evidence is very low.	2274 (5 RCTs, 1 non‐randomised trial)	⊕⊝⊝⊝ Very lowb, d, e
Length of stay	May have no impact on length of stay in the ED or when admitted to a ward. However, we are uncertain whether this intervention increases or reduces patient length of stay because the certainty of the evidence is very low.	907 (3 randomised trials)	⊕⊝⊝⊝ Very low b, e
Patients with pressure ulcers	Greater reduction in number of patients with pressure ulcers at 12 and 24 months in the intervention group. However, we are uncertain whether this intervention reduces the number of patients with pressure ulcers because the certainty of the evidence is very low.	753 (1 CBA study)	⊕⊝⊝⊝ Very low f
Costs	In two studies total health care costs were lower in the intervention group but in one study there was no impact on overall costs. We are uncertain whether this interventions reduces or increases cost because the certainty of the evidence is very low.	617 (3 randomised trials)	⊕⊝⊝⊝ Very low b, d, g
*The risk in the intervention group (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CBA: controlled before‐after study; CI: confidence interval; ED: emergency department; OR: odds ratio
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect Moderate certainty: we are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different Low certainty: our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect Very low certainty: we have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect

^aWe downgraded by one level due to moderate risk of bias. It was unclear if allocation was concealed and personnel, participants, and assessors were not blinded in one randomised trial: in the other two randomised trials, differences in baseline characteristics were not adequately analysed.

^bWe downgraded by one level due to imprecision (the width of the confidence interval is consistent with both a reduction and an increase in the outcome).

^cWe downgraded by two levels due to serious risk of bias. Sequence generation or concealment was not reported in one randomised trial, in five trials either personnel/participants or assessors were not blinded, in one trial differences in baseline characteristics were not adequately analysed, and in one trial other sources of bias included missing data, differences between sites and possible confounding.

^dThe certainty of evidence was downgraded by one level due to serious inconsistency between studies in measures used, incomplete reporting of data, and poor study design.

^eWe downgraded by two levels due to serious risk of bias due to issues with sequence generation or concealment, blinding of personnel/participants or assessors, and/or differences in baseline characteristics not being adequately analysed.

^fWe downgraded by two levels due to serious risk of bias due to personnel/participants not being blinded, incomplete outcome reporting, and confounding.

^gWe downgraded by three levels due to serious risk of bias (two levels) and indirectness (one level). In all three randomised trials sequence generation or concealment was not reported and personnel and participants were not blinded; in one randomised trial other biases were present. One study measured costs indirectly.

Summary of findings 2. The introduction of nursing assistive personnel (NAP) to the hospital workforce versus usual staffing.

The introduction of nursing assistive personnel (NAP) to the hospital workforce versus usual staffing
Patient or population: patients admitted to a cardiovascular surgery/urology/ophthalmology unit, a kidney transplant/plastic surgery unit, an oncology unit, or an orthopaedic surgery unit Setting: four units in a 560‐bed hospital in the USA Intervention: the introduction of NAP to the hospital workforce Comparison: usual nurse staffing
Outcomes	Impact	№ of participants (studies)	Certainty of the evidence (GRADE)
Nursing‐staff turnover	No studies reported this outcome.	‐	‐
Patient mortality	No studies reported this outcome.	‐	‐
Patient readmissions	No studies reported this outcome.	‐	‐
Patient attendances at the ED	No studies reported this outcome.	‐	‐
Length of stay	No studies reported this outcome.	‐	‐
Patients with pressure ulcers	No studies reported this outcome.	‐	‐
Costs	Personnel costs were higher in the intervention group. It is uncertain whether this intervention reduces costs because the certainty of the evidence is very low.	6769 (1 CBA study)	⊕⊝⊝⊝ Very low a
CBA: controlled before‐after study; ED: emergency department
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect Moderate certainty: we are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different Low certainty: our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect Very low certainty: we have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect

^aWe downgraded by two levels because outcome assessors were not blinded, incomplete data were reported, baseline assessment was not conducted and control units appear different, and the intervention changed during the study.

Summary of findings 3. Primary nursing compared to usual/team/functional nursing.

Primary nursing compared to usual/team/functional nursing
Patient or population: nurses working on medical or long‐term psychiatric units Setting: hospital psychiatric units, the Netherlands; hospital in‐patient medical units, USA Intervention: primary nursing where a named nurse is responsible for co‐ordinating care for the entirety of a patient’s admission Comparison: usual/team/functional nursing/modular nursing
Outcomes	Impact	№ of participants (studies)	Certainty of the evidence (GRADE)
Nursing‐staff turnover	In two studies, nursing turnover was lower in the intervention group with a risk of 23 per 100 (95% CI 16 to 32), compared with 37 per 100 in the control group (OR 0.51, 95% CI 0.32 to 0.81). In another study, turnover was lower in primary nursing than in a new modular model, but higher in primary nursing than in the existing team nursing. We are uncertain whether this intervention reduces staff turnover because the certainty of the evidence is very low.	> 464 (1 non‐randomised trial and 2 CBA studies)	⊕⊝⊝⊝ Very low a, b
Patient mortality	No studies reported this outcome.	‐	‐
Patient readmissions	No studies reported this outcome.	‐	‐
Patient attendances at the ED	No studies reported this outcome.	‐	‐
Length of stay	No studies reported this outcome.	‐	‐
Patients with pressure ulcers	No studies reported this outcome.	‐	‐
Costs	In one study, an all‐RN primary‐nursing model was slightly more expensive than team or modular nursing models. In another study, costs per patient per day were lower in the intervention group. We are uncertain whether this intervention reduces or increases costs because the certainty of the evidence is very low.	>138 (1 non‐randomised trial and 1 CBA study)	⊕⊝⊝⊝ Very low c
*The risk in the intervention group (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CBA: controlled before‐after study; CI: confidence interval; ED: emergency department; OR: odds ratio; RN: registered nurse
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect Moderate certainty: we are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different Low certainty: our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect Very low certainty: we have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect

^aWe downgraded by two levels for risk of bias because one study did not report blinding and reported changes in the setting during the study period, while the other had missing data, had considerable differences in response rates between the intervention and control groups, and reported contamination on the control units. Both studies had no or limited discussion of baseline characteristics,

^bWe downgraded by one level due to serious imprecision in one study.

^cWe downgraded by two levels due to no reports of blinding, limited information on baseline characteristics and changes in the study setting during the study period.

Summary of findings 4. Self‐staffing versus usual staffing.

Self‐staffing versus usual staffing
Patient or population: nurses working on acute care, intensive care or medical care units Setting: private, not‐for‐profit hospital in a Mid‐Western city, USA Intervention: self‐staffing, where nursing units have full responsibility for staffing, using only their own nursing staff to fill staffing gaps Comparison: usual staffing
Outcomes	Impact	№ of participants (studies)	Certainty of the evidence (GRADE)
Nursing‐staff turnover	Authors reported a reduction in nursing‐staff turnover on intervention units that was sustained, in comparison to higher and fluctuating nursing‐staff turnover on other units. We are uncertain whether this intervention reduces staff turnover because the certainty of the evidence is very low.	674 (1 CBA study)	⊕⊝⊝⊝ Very low a
Patient mortality	No studies reported this outcome.	‐	‐
Patient readmissions	No studies reported this outcome.	‐	‐
Patient attendances at the ED	No studies reported this outcome.	‐	‐
Length of stay	No studies reported this outcome.	‐	‐
Patients with pressure ulcers	No studies reported this outcome.	‐	‐
Costs	No studies reported this outcome.	‐	‐
*The risk in the intervention group (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CBA: controlled before‐after study; ED: emergency department
GRADE Working Group grades of evidence High certainty: We are very confident that the true effect lies close to that of the estimate of the effect Moderate certainty: We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different Low certainty: Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect Very low certainty: We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect

^aWe downgraded by two levels for risk of bias because there was no blinding of participants or personnel, baseline characteristics were not provided, and the study used a multiple probe design (interventions introduced in units at different times) and it was unclear what impact this might have on results.

Background

It is generally understood that nurse staffing is closely related to the quality of the nursing practice environment, the care provided for patients, and, subsequently, to patient outcomes (Griffiths 2014; Leiter 2006; Squires 2015). The availability of nurses with the appropriate expertise and skills within and across countries has been identified as a key factor in the achievement of the Sustainable Development Goals (SDGs) (UN 2015; White 2015; WHO 2016a). Currently, there is a shortage of nurses across many countries and a related 'global health personnel crisis' (OECD 2010). This is likely to continue for the foreseeable future, for example, it is estimated that there will be a global shortage of about nine million nurses/midwives by 2030 (WHO 2016b). At the same time, hospital patients have become more dependent, requiring additional or more complex nursing care, due to factors such as advances in technology, ageing populations, increases in patient co‐morbidities, and advances in community‐based care (Buchan 2015; Kim 2013).

Changes have been introduced to the ways in which hospitals organise nursing staff. On the one hand, new roles have been introduced for advanced practice and specialist nurses ‐ aimed at supporting more holistic and responsive patient care and addressing shortages of junior doctors (Cowley 2016), and making better use of the existing healthcare workforce through more efficient skill‐mixing (OECD 2010). On the other hand, unregistered staff (e.g. nurse extenders, nursing assistants, health care assistants (HCAs) have been added to the hospital workforce to support nursing care and to improve the cost‐effectiveness of nurse staffing (Griffiths 2014). The allocation of nursing resources across hospital units and the structure of nursing shifts continue to evolve. In some jurisdictions, minimum nurse‐to‐patient ratios have been introduced (e.g. California and Australia) (Gerdtz 2007; SEIU 2018; Serratt 2013), and in others 'safe staffing' initiatives have been introduced (UK and Ireland). In other jurisdictions comprehensive strategies have been developed such as the Health Workforce Australia (HWA) initiative (Buchan 2015).

Nurse‐staffing models are used to determine the optimal allocation of nursing resources (number of nurses and mix of nursing staff) to meet the care needs of patients. The focus of this review is on hospital nurse‐staffing models that include changes to nurse‐staffing levels, nursing skill mix, grade mix, and education mix. This is the first update of the original review published in 2011 (Butler 2011).

Description of the condition

Nursing shortages are reported across many developed countries, including the USA, Canada, the UK, Ireland, Australia, and in many low‐ and middle‐income countries in South America, Africa, and Asia. There are continuing concerns about nurses from low‐ and middle‐income countries being recruited to countries which can offer better pay and conditions (Alittus 2014; Kohn 2003). Difficulty recruiting and retaining nurses is linked to difficult working conditions, unsafe nurse‐to‐patient ratios, stress, and poor pay (Alittus 2014; Butterfly 2017; NMC 2017). A number of studies have identified that the youngest generation of nurses is the most likely to leave the profession and that this is largely due to highly demanding work, burnout, and dissatisfaction with salary levels (Flinkman 2013). In some countries (e.g. the USA), the shortage of nurses is compounded by an ageing workforce and a sharp increase in nurses coming close to retirement (ANA 2015; Buchan 2015).

The International Council of Nurses reported that "a common challenge facing HR [human resources] managers is determining the most effective mix of staff and skills needed to deliver quality and cost‐effective patient care" in the light of "rising demand for health services, cost containment and shortages of nurses and other health workers" (ICN 2006). The mix of nursing staff providing hospital care (often referred to as skill mix) involves the differentiation of roles between the 'professional' nurse and unregistered healthcare staff, variously referred to as nurse extenders, nurse or nursing assistants, or as HCAs.

Description of the intervention

Nurse‐staffing models are used to identify and allocate the numbers and mix of nurses required to meet the care needs of hospital patients. There are two approaches to deciding on the numbers and mix of nurses required in a hospital unit: firstly, top‐down approaches that involve comparisons between similar hospitals, and secondly, bottom‐up approaches aimed at matching staff to patient dependency workload (Hurst 2006). As a top‐down approach, the number of nurses available in a hospital or hospital unit can be quantified in relation to the number of patients in that hospital or hospital unit (nurse‐to‐patient ratio). By comparison, an example of a bottom‐up approach is the safe staffing initiative (Fenton 2015), which was introduced in the UK in the wake of the Francis Report on the failings at the Mid Staffordshire Foundation Trust, and recommendations from the Berwick Report (Berwick 2013). This bottom‐up approach is gathering momentum and a number of projects are underway in the UK and Ireland to implement safe staffing initiatives.

Numbers of nurses can also be quantified in terms of hours of nursing care and nurse full‐time equivalents (FTE) or whole‐time equivalents (WTE). Currently one WTE/FTE is equivalent to 37.5 hours per week in Australia and Canada, and 39 hours per week in Ireland. Mandatory nurse‐to‐patient ratios have been introduced in California, USA and in a number of Australian states in response to concerns about staffing levels. The State of California mandates specific ratios of nurses to patients for different types of nursing units. For example, one nurse to five patients on a medical/surgical ward, a ratio of one‐to‐one for emergency room trauma, and one‐to‐two for critical care/intensive care unit (ICU) (SEIU 2018). The current mandatory nurse‐to‐patient ratio in Victoria, Australia, is five nurses to 20 patients (the 5‐20 model) (Serratt 2013). Serratt reported that this ratio was set to accommodate nursing requirements per ward rather than per patient, which supports the team basis of nursing work. Changes have also been made to nursing shifts or nurses' work patterns (e.g. moving to 12‐hour shifts, while some hospitals are reverting to eight‐hour shifts due to concerns about the quality and safety of care (National Nursing Research Unit 2013)), and there is a greater reliance on the use of overtime and agency staff to cover nursing shifts (Rogers 2004).

The mix of nurses can be quantified in terms of skill mix, grade mix or qualification mix. Skill mix may refer to the mix of 'licensed' and 'unlicensed' nurses in the USA (Kane 2007), or 'registered' and 'unregistered' staff in the Irish, Australian and UK workforces. Skill mix has also been defined as "the proportion of different nursing grades, and levels of qualification, expertise and experience" (Ayre 2007).

Skill mix may also refer to enhancing the nursing workforce by adding or creating new roles for Advanced Practice Nurses (APNs). APNs (also referred to as Nurse Practitioners (NPs) or Clinical Nurse Specialists (CNSs)) have been deployed in over 70 ‐ primarily high‐income ‐ countries. However, a growing need for APNs in low‐ and middle‐income countries has been identified by Bryant‐Lukosius 2015. They report that CNSs are usually introduced to provide highly complex and specialised care, to support the development of nursing practice, to support nurses at the point‐of‐care, and to lead quality improvement and evidence‐based practice initiatives in response to research advances in treatment and technology. The role of NPs usually involves an expanded scope of practice with additional autonomy and the authority to order diagnostic tests, diagnose conditions, and prescribe treatments and medications. Bryant‐Lukosius reports that APN roles have been introduced more recently to support healthcare reform, to improve the quality of health care, and to provide more sustainable models of healthcare delivery.

Grade mix refers to the proportion of nursing grades in the nursing workforce. These are occupational grades rather than individuals that are assigned to posts, and the grading models vary within and across countries. Grade may be used as a proxy for skill (Carr‐Hill 1995), but skill mix is more than grade mix ‐ it relates to qualifications, experience and competencies. Qualification mix refers to the proportion of different nursing qualifications in the workforce.

Skill mix, grade mix, or qualification mix may refer to the mix of nurses in a hospital, in a hospital unit or on a hospital ward. Changes in the mix of nurses with different educational qualifications may also result in a change in skill mix in relation to the proportion of nurses with or without additional or more advanced skills and knowledge. Concurrently, the education and training of nurses has rapidly evolved to attempt to address issues of shortage of supply, increased demand, and expansion of their role. Examples include the introduction of a shorter programme (often of two years' duration instead of three), the introduction of degree programmes, and the introduction of post‐registration education programmes.

New models of nurse staffing have also been introduced in different countries that relate to how patients are assigned to nurses working on a hospital ward or unit. One example of this is seen in primary nursing, where one nurse (the primary nurse) is responsible for the total care of a number of patients 24 hours a day, seven days a week, aimed at providing "comprehensive, individualised and consistent care" (Kozier 2008). Acting as a co‐ordinator, the primary nurse assesses and prioritises each patient's needs, and plans and evaluates their care as their "first line manager ... with all its inherent accountabilities and responsibilities". However, other nursing staff may also be involved in the patient's care (Kozier 2008).

How the intervention might work

It has long been argued that nurse staffing and nursing skill mix are "directly linked" to quality of care and patient outcomes (Currie 2005). More recently, the focus of concern has been on the cost‐effectiveness and safety of nurse staffing (Griffiths 2014). In the UK, NICE 2014 identified nine indicators of safe nurse staffing. Four of these indicators relate to patient outcomes: falls, pressure ulcers, medication administration errors, and the adequacy of meeting patients’ nursing care needs. Two indicators relate to nursing staff: missed breaks and compliance with any mandatory training; and three indicators relate to staffing outcomes: nursing overtime; planned, required and available nurses for each shift; and high levels or ongoing reliance on temporary nursing staff, or both. It is reported that having an adequate number of registered nurses decreases patient deaths, injury and permanent damage; reduces rates of falls, missed care, and pressure ulcers; and is associated with the prevention of healthcare‐acquired infections and associated costs (Aiken 2008; Kane 2007; Lankshear 2005). Furthermore, nursing care that is cost‐effective, accessible and of high quality, results in good clinical outcomes and patient satisfaction; highly educated nurses lead to lower patient mortality, complication rates, and shorter hospital stays (Griffiths 2016; Shekelle 2014; Squires 2015).

It is suggested that APNs can contribute significantly to SDGs and improve key patient outcomes. In relation to hospital care, it is suggested that the deployment of APNs can:

• improve access to supportive care;

• improve quality of life, increase survival rates, lower complication rates, and improve physical, functional, and psychological well‐being of patients with acute or chronic conditions;

• improve health promotion practices;

• improve recruitment and retention of nurses at the front‐line of care; and

• reduce waiting times in emergency departments (EDs), lengths of hospital stay and use of unnecessary diagnostic tests (Bryant‐Lukosius 2015).

Although the introduction of unregistered healthcare staff has been used to increase the numbers of staff available to provide patient care, the reduction in the proportion of registered nurses may impact on patient outcomes in other ways. A review of unregistered healthcare staff identified that HCAs accounted for about a third of the caring workforce in UK hospitals (Cavendish 2013). The authors reported that HCAs spent more time at the bedside than nurses, and they identified a lack of any compulsory or consistent training and "a profusion of job titles". Routine tasks generally expected of HCAs include: making beds; helping patients to eat and bathe; monitoring and recording patients' glucose levels, temperature, pulse, respiration and weight; carrying out simple dressing changes; and escorting patients to the operating theatre. However, Cavendish 2013 also identified that some HCAs are doing jobs that used to be carried out by registered nurses and doctors, including: female catheterisation; insertion of intravenous drips; taking blood; applying complex dressings; monitoring diagnostic machines; setting up infusion feeds; giving injections; preparing medication and administering it to patients; making electrocardiogram tracings; liaising with medical staff; relating medical information to relatives; and developing and updating care plans. It is suggested that this is because registered nurses are spending more time on organisational tasks. Cavendish 2013 also examined the selection and training of HCAs in considerable detail and reported that although they found some "pockets of excellence" in relation to the selection of recruits and rigorous training and development, often there were no minimum educational requirements for the selection of HCAs and overall training was "neither sufficiently consistent, nor sufficiently well supervised, to guarantee the safety of patients and users of health care ...".

A systematic review of the effects of shift length on the quality of patient care and health‐provider outcomes reported equivocal results (Estabrooks 2009).

Why it is important to do this review

The arguments made in our original 2011 review about the lack of good evidence relating to the impact of nurse staffing on patient‐ and staff‐related outcomes still stand (Butler 2011). In our original review, we argued that although the effects of changes to nurse staffing have important implications for healthcare provision, the bulk of the public policy driving these changes is not evidence based because of "an insufficient body of credible evidence linking changes in the hospital nurse work force to potentially adverse effects on patient outcomes" (Buerhaus 2000). Furthermore, it has been suggested that the "considerable research" capable of informing the debate about the relationship between the nursing workforce and patient outcomes is often "selectively quoted to support arguments" (Lankshear 2005). Concerns remain that the evidence that is available is not being used to inform effective policies (Buchan 2015). Research on this topic continues, and although a number of systematic reviews have been conducted since our original review in 2011, differences in scope, review methods and inclusion criteria limit the generalisability of their findings. There is a clear need for a Cochrane systematic review that is truly comprehensive in terms of the range of interventions relating to nurse‐staffing models, and that is inclusive of all eligible studies conducted in all jurisdictions and in all languages.

Several systematic reviews of nurse staffing and patient outcomes have been conducted previously, but focused selectively on specific aspects of this review. For example, Mattila 2013 investigated primary nursing models; of the nine studies included in this review, four were of midwifery care and the remaining five related to three studies that were included in our review (Boumans 1999; Gardner 1991; Melchoir 1996). Shekelle 2014 focused specifically on nurse‐patient staffing ratios. Other reviews were comprehensive in nature, but the scope of the search was limited. For example, the Lang 2004 systematic review of the effects of nurse staffing on patient, employee, and hospital outcomes was limited to studies conducted in the USA and published between 1980 and 2003; the Mattila 2013 search was limited to studies published in English from 1990; and the Lankshear 2005 systematic review of nurse staffing and healthcare outcomes was limited to studies published between 1990 and 2004.

Other reviews have included studies that are outside of the scope of this review in relation to study design or outcomes. For example, all nine studies included by Numata 2006 were observational and did not include interventions; none of the 28 studies included by Shekelle 2014 were experimental studies; and the Kane 2007 systematic review of nurse staffing and the quality of patient care was limited to observational studies. Three systematic reviews of hospital nurse staffing were conducted in the UK in 2014 to inform the development of the National Institute for Health and Care Excellence (NICE) guidelines on safe staffing (Drennan 2014; Griffiths 2014; Simon 2014). Taken together they provided a very comprehensive overview of hospital nurse staffing but included mostly observational and cross‐sectional designs. This review aims to address the limitations identified in previous related studies through an inclusive systematic review of the current research evidence related to the effect of hospital nurse‐staffing models on patient‐ and staff‐related outcomes.

Objectives

The purpose of this review was to explore the effect of hospital nurse‐staffing models on patient and staff‐related outcomes in the hospital setting, specifically to identify which staffing model(s) are associated with: 1) better outcomes for patients, 2) better staff‐related outcomes, and, 3) the impact of staffing model(s) on cost outcomes.

Methods

Criteria for considering studies for this review

Types of studies

We sought all relevant published and unpublished randomised trials, non‐randomised trials, controlled before‐after studies, interrupted‐time‐series studies and repeated‐measures studies that met the Cochrane Effective Practice and Organisation of Care (EPOC) Group eligibility criteria (EPOC 2018a). We included these four types of designs because few randomised trials of hospital nurse staffing have been conducted and we wanted to assess what additional evidence is available from non‐randomised designs. We imposed no restrictions regarding time period, jurisdiction, or language. We excluded any relevant studies that did not use one of the previously mentioned designs. We assessed the risk of bias of all included studies using the EPOC criteria (EPOC 2018b).

Types of participants

Participants were hospital nursing staff and hospital patients. Hospitals included acute and non‐acute, small, medium, and large, teaching and non‐teaching, and public and private hospitals. Staff were registered nurses or their international equivalents (e.g. registered general nurse, staff nurse, professional nurse), licensed practical nurses or their international equivalents (e.g. licensed vocational nurse, enrolled nurse), and unlicensed assistive personnel or their international equivalents (e.g. nurses' aide, auxiliary nurse, nursing assistant, HCA). We excluded studies of nurse staffing outside hospitals (e.g. community, nursing homes), as staffing models in residential‐ or nursing‐homes, or extended‐care settings are the focus of a separate Cochrane Review (Hodgkinson 2011).

Types of interventions

We searched for studies of all types of hospital nurse‐staffing model interventions. These included interventions of staffing models, staffing levels, skill mix, grade mix, or qualification mix. Staffing models are models used to identify and allocate nursing staff, shift patterns, use of overtime, or use of non‐core staff. Staffing levels include nurse‐to‐patient ratios, hours of nursing care, use of full‐ or part‐time staff, or both. Skill mix refers to the proportion of total hours of nursing care provided by registered nurses, the number of registered nurse hours per day, the proportion of registered nurses in the work force, or the proportion of APNs. Grade mix refers to the proportion of nursing grades in the work force. Qualification mix refers to the proportion of graduate nurses in the nursing work force, the proportion of nurses with a post‐registration qualification (obtained following registration as a nurse), or the proportion of nurses with a post‐graduate qualification. For all interventions, we compared the nurse staffing intervention with usual or previous nurse staffing. For example, primary nursing was compared with team and functional nurse‐staffing models.

We excluded studies of the substitution of doctors by nurses. Such substitution is the focus of a separate Cochrane Review (Laurant 2018). Studies of ratios between nurses and other professionals were also beyond the scope of this review.

Types of outcome measures

The primary outcomes of interest to this review were any objective measures of staff‐related outcomes, patient outcomes, or economic outcomes (using the methodological inclusion criteria for an EPOC review (EPOC 2018a)). These included nursing‐staff turnover rates, staff sick‐leave rates, patient mortality, risk‐adjusted patient mortality, in‐hospital death, and patient length of stay. We also included nursing‐sensitive patient outcomes, which are of particular interest in studies of nurse staffing. These are defined as "variable patient or family caregiver states, behaviours, or perceptions at a low level of abstraction that are responsive to nursing interventions and used for determining a patient outcome" (Gordon 1998). Doran 2003 defined nursing‐sensitive outcomes as ''those that are relevant, based on nurses' scope and domain of practice, and for which there is empirical evidence linking nursing inputs and interventions to the outcomes.'' Several measures of nurse‐sensitive or nursing‐sensitive patient outcomes can be found in the literature (Doran 2006; Kane 2007). Examples of objective nursing‐sensitive outcomes include infections, falls, pressure/decubitus ulcers, complications, or medication errors. The review also included any objective measure of economic outcome included in studies e.g. incremental resource use, incremental costs, incremental cost‐effectiveness such as cost/life year saved, cost/quality‐adjusted life year (QALY), and cost/disability‐adjusted life year (DALY).

We identified the following as the most important outcomes in this review:

• nursing‐staff turnover;

• patient mortality;

• patient readmissions;

• patient attendances at the ED;

• length of stay;

• patients with pressure ulcers;

• costs.

Selection of these outcomes was based on consideration of which outcomes are most likely to be important to people making decisions about nurse staffing. We did not specify the smallest important difference for outcomes in our protocol for this review. We assessed the importance of effects and the precision of the estimates based on how likely it seemed to us that some people would make different decisions if the true effect was near one end or the other of the 95% confidence interval (EPOC 2018d).

Following the original protocol, we excluded studies that focused only on outcomes that were not considered to be objective from this review. Examples of non‐objective outcomes found in studies of nurse staffing included patient satisfaction, staff satisfaction, quality of life, disease impact, staff stress, and staff burnout. Revised EPOC guidelines allow for the inclusion of wider measures such as quality of life, surrogate physiological measures, and psychological well‐being (EPOC 2018c). These should be included in the next update, but will require protocol revisions.

Search methods for identification of studies

Electronic searches

We searched the following databases from 2009 (last date searched in the previous version of this review (Butler 2011)) to 22 March 2018:

• Cochrane Central Register of Controlled Trials (CENTRAL; 2018, Issue 2) in the Cochrane Library;

• MEDLINE Ovid (including Epub Ahead of Print, In‐Process & Other Non‐Indexed Citations and Versions);

• Embase Ovid;

• NHS Economic Evaluation Database (NHSEED; 2015, Issue 2) in the Cochrane Library;

• CINAHL EBSCO (Cumulative Index to Nursing and Allied Health Literature).

The EPOC Cochrane Information Specialist (CIS) developed the search strategies in consultation with the authors. Search strategies were comprised of keywords and controlled vocabulary terms. We applied no language limits.

Searching other resources

Trial registries

• International Clinical Trials Registry Platform (ICTRP), Word Health Organization (WHO) www.who.int/ictrp/en (searched 22 March 2018)

• ClinicalTrials.gov, US National Institutes of Health (NIH) clinicaltrials.gov (searched 22 March 2018)

In addition, we used the following to identify primary studies:

• handsearches of high yield journals and conference proceedings not already handsearched on behalf of Cochrane;

• searches of reference lists of all papers and relevant reviews identified;

• contact with authors of relevant papers and other related reviews to seek information on any further published or unpublished work;

• searches the ISI Web of Science for papers which cite studies included in the review.

All search strategies used are provided in Appendix 1.

Data collection and analysis

We worked in pairs to screen studies, extract data and to assess the risk of bias of all eligible studies independently. We resolved any disagreement by discussion between authors, and with referral to a third author where necessary. We used Covidence software to manage screening and data extraction (Covidence).

Selection of studies

We worked in pairs to examine all potential studies independently using pre‐established inclusion criteria. We examined all titles and abstracts identified in the search and downloaded full text copies of studies that appeared relevant. We excluded studies if they were not of the appropriate design (i.e. randomised trial, non‐randomised trial, controlled before‐after studies with at least two control and two intervention sites, interrupted‐time‐series or repeated‐measures studies with at least three data points pre‐ and post‐intervention), did not relate to hospital staff or hospital patients, did not relate to one of the interventions specified (i.e. staffing models, staffing levels, skill mix, grade mix or qualification mix), or included only secondary outcomes or outcomes that were not considered to be objective. We catalogued all excluded studies along with their reasons for exclusion.

Data extraction and management

We extracted the following study characteristics from the included studies using Covidence software:

• study identification: authors, study title, institution, contact details;

• methods: study design, study setting, date of study, follow‐up;

• participants: inclusion criteria, exclusion criteria, group differences;

• interventions: intervention components, comparison;

• outcomes: main and other outcomes specified and collected, time points reported;

• findings: results reported for all relevant outcomes;

• notes: sponsorship source, notable conflicts of interest of trial authors.

Assessment of risk of bias in included studies

Working in pairs, we assessed the risk of bias of each study independently, using the suggested 'Risk of bias' criteria for EPOC reviews (EPOC 2018b):

• random sequence generation;

• allocation concealment;

• blinding of participants and personnel;

• blinding of outcome assessment;

• incomplete outcome data;

• selective outcome reporting;

• baseline characteristics similar for intervention group and control;

• other bias.

Measures of treatment effect

We estimated the effects of interventions by measuring changes in absolute numbers or mean values and calculating odds ratio, mean differences and confidence intervals for some outcomes. However, the small number of eligible studies identified for each intervention limited our analysis.

Where possible, results from controlled before‐after studies are presented in terms of:

• absolute post‐intervention difference (mean or proportion in intervention group minus control);

• relative percentage difference (absolute difference divided by post‐intervention score in the control group);

• absolute change from baseline (pre to post changes in both groups); and

• difference in absolute change from baseline.

Unit of analysis issues

In all studies, participants were allocated either to the intervention or the control unit using a parallel design. Some data were collected at the hospital unit level (e.g. number of nurse resignations in the unit/group), rather than for each individual participant.

Dealing with missing data

We contacted authors by email and sent follow‐up requests where we identified missing data in eligible studies. In some cases (11 studies), we were unable to consider studies for inclusion because we could not contact authors or authors did not respond to our requests.

Assessment of heterogeneity

We assessed the comparability of different studies in relation to: setting, population, intervention type, outcomes, and measurement of outcome. We conducted meta‐analysis for two different outcomes (readmission within 30 days and patients attending the ED within 30 days of discharge) where the studies (n = 2 for both outcomes) were similar. However, for other outcomes the analysis indicated that studies were too different from each other to combine in a valid meta‐analysis, therefore, we did not explore the data further for quantitative measures of heterogeneity such as I².

Assessment of reporting biases

We were unable to assess reporting bias by creating a funnel plot due to the small number of eligible studies and heterogeneity across studies.

Data synthesis

We used a narrative synthesis to describe results in cases in which only one study was included, or when heterogeneity between studies (e.g. type of intervention, outcome or population) precluded meta‐analysis and subgroup analysis.

We included reported hospital cost data as indirect costs, as full costing approaches (direct and indirect costs), and hospital charges. There were insufficient reported data to synthesise full economic evaluations. We added the cost/charges effects of nurse‐staffing models (cost/charges analysis), but not the cost‐effectiveness, for all studies that reported on cost measures. Cost/charges data is presented in USD for the common price year 2016 by using the 'CCEMG‐EPPI‐Centre Cost Converter' (Version 1.5), a web‐based tool that can be used to adjust an estimate of cost expressed in one currency and price year to a target currency and, or price year, or both (Shemilt 2008; Shemilt 2010). We adjusted costs/charges for inflation by applying Gross Domestic Product deflators (GDPD values) (Drummond 1996). Additionally, we have provided the adjusted cost outcomes and the undiscounted cost data to allow readers to recalculate the results using any discount rate (Appendix 2; Appendix 3; Appendix 4; Appendix 5).

Summary of findings

We summarised the findings for each intervention and graded the certainty of the evidence for each of the following most important outcomes in 'Summary of findings' tables:

• nursing‐staff turnover;

• patient mortality;

• patient readmissions;

• patient attendances at the ED;

• length of stay;

• patients with pressure ulcers; and

• costs.

We used the GRADE approach to conduct an assessment of the certainty of evidence for each outcome using the 'EPOC Worksheets for preparing a Summary of Findings (SoF) table using GRADE' (EPOC 2018e; Guyatt 2008). We assessed the certainty of evidence (high, moderate, low, and very low) for each outcome using the five GRADE criteria for up‐ or downgrading the certainty of the evidence (risk of bias, consistency of effect, imprecision, indirectness, and publication bias) (GRADEpro). We recorded the main reasons for up‐ or downgrading the certainty of the evidence in footnotes to the 'Summary of findings' tables and in the full evidence profiles Appendix 6.

Subgroup analysis and investigation of heterogeneity

We were unable to conduct subgroup analysis due to insufficient numbers of studies with similar outcomes.

Sensitivity analysis

We were unable to conduct sensitivity analysis due to insufficient numbers of studies with similar outcomes.

Results

Description of studies

We included 19 studies (20 records) that examined the effects hospital nurse‐staffing models on patient and staff‐related outcomes. Results from 17 of these studies were included in our analysis. See: Characteristics of included studies.

Results of the search

Our search yielded a total of 14,458 titles. We screened all titles and abstracts, and identified 336 potentially eligible studies for inclusion. Following detailed eligibility assessment of the full text articles of these studies, we excluded 326 studies, identified two ongoing studies (see Characteristics of ongoing studies), and included eight new studies in the review (Bakitas 2009; Castro 2003; Choi 1986; Gardner 1991; McPhail 1990; Plant 2015; Shukla 1983; Sisk 2006). This review now includes 19 studies (Figure 1).

1.

Review flow diagram.

Included studies

Trial design, country of conduct, and funding

Eleven of the 19 studies included were randomised controlled trials (Bakitas 2009; Castro 2003; Choi 1986; Davies 2001; Forster 2005; McPhail 1990; Plant 2015; Pozen 1977; Ritz 2000; Sisk 2006; Talley 1990), two were non‐randomised trials (Einstadter 1996; Shukla 1983), and six were controlled before‐after studies (Boumans 1999; Forbes 2006; Gardner 1991; Melchoir 1996; Neidlinger 1993; O'Connor 1992).

Twelve studies were conducted in the USA, two in the Netherlands, two in the UK, one in Australia, and two in Canada. One hospital was a Veterans' Affairs (VA) medical centre, one study involved five psychiatric hospitals, one involved a group of four large, medium and small private and municipal hospitals, one involved a group of six specialist hospital units, and one involved an integrated healthcare centre. Four studies described the setting as a university or teaching hospital, two as a tertiary hospital, one as a major medical centre, and seven as a general or city hospital.

Six studies were funded by a research grant, three by a research group, one by a health department, one by local health services, and two by a charitable trust. In six studies, there was no mention of funding sources.

Interventions

Twelve of the 19 studies related to nursing skill mix. We identified two types of nursing skill mix interventions:

• the introduction of advanced or specialist nurses to the nursing workforce versus usual hospital staffing (11 studies) (Bakitas 2009; Castro 2003; Davies 2001; Einstadter 1996; Forbes 2006; Forster 2005; Plant 2015; Pozen 1977; Ritz 2000; Sisk 2006; Talley 1990), and

• the introduction of nursing assistive personnel (NAP) to the hospital workforce versus usual staffing (one study) (Neidlinger 1993).

In addition, five studies were of primary nursing (Boumans 1999; Gardner 1991; Melchoir 1996; McPhail 1990; Shukla 1983), and two were of staffing models: one of self‐staffing, where units organised their own staffing (O'Connor 1992), and one of different nursing‐shift models (Choi 1986).

1. The introduction of advanced or specialist nurses to the nursing workforce versus usual staffing

Eleven studies examined the introduction of advanced or specialist nurses to the nursing workforce versus usual staffing. Six studies examined the impact of care provided by an NP or CNS on patient outcomes and costs for patients with specific conditions: Bakitas 2009 (advanced cancer, USA), Castro 2003 (asthma, USA), Davies 2001 (diabetes, UK), Forbes 2006 (multiple sclerosis (MS), UK), Ritz 2000 (breast cancer, USA), and Sisk 2006 (heart failure, USA). Five studies examined the impact of specialist nursing roles on patient outcomes and costs: Talley 1990 (liaison psychiatric nurse (LPN), USA); Pozen 1977 (a critical care unit‐based nurse rehabilitator, USA); Einstadter 1996 (a NP and nurse case manager, USA); Forster 2005 (CNS as a nurse team co‐ordinator, Canada); and Plant 2015 (a case manager/care co‐ordinator/care navigator, Australia). The majority of these studies were randomised trials, except for Einstadter 1996, which was a non‐randomised trial, and Forbes 2006, which was a controlled before‐after study.

2. The introduction of nursing assistive personnel (NAP) to the hospital workforce versus usual staffing

One study conducted in the USA examined the introduction of NAP into a nursing professional‐practice model of nursing in four acute hospital units (Neidlinger 1993). Each NAP was assigned to work with two to three registered nurses, assisting in the care of 12 to 18 patients.

3. Primary nursing compared to usual/functional/team nursing

Five studies examined the impact of introducing primary nursing on staff‐related outcomes and costs (Boumans 1999; Gardner 1991; McPhail 1990; Melchoir 1996; Shukla 1983). The Boumans 1999 and Melchoir 1996 studies were conducted in the Netherlands, the McPhail 1990 study in Canada, and Gardner 1991 and Shukla 1983 in the USA. Primary nursing refers to the practice of a named nurse being responsible for co‐ordinating care for the entirety of a patient’s admission Manthey 2002. One study was a randomised (cross‐over) trial (McPhail 1990), one study was a non‐randomised trial (Shukla 1983), and three studies were controlled before‐after designs (Boumans 1999, Gardner 1991; Melchoir 1996). Boumans 1999 and Melchoir 1996 both reported problems with contamination or imitation in the control groups. Shukla 1983 reported some slight variations between the planned and actual staffing, due to scheduling difficulties. We did not have sufficient information from the results to include McPhail 1990 in the analysis.

4. Staffing models

One study (conducted in the USA) used a controlled before‐after design to examine the impact of nursing self‐staffing on nursing‐staff turnover/retention (O'Connor 1992). In this model, units had full responsibility for staffing, would use only their own nursing staff, and staff from other units could not be moved around to fill staffing gaps. One study (conducted in the USA) (Choi 1986) used a randomised trial to compare three different shift models:

• straight shifts;

• computer‐assisted scheduling (called "compflex");

• a staff‐developed schedule (called "select‐a‐plan").

They examined the impact of these shift models on nurse retention. We did not have sufficient information from the results to include the Choi 1986 study in the analysis.

5. Other hospital nurse‐staffing interventions

We did not identify eligible studies of any other nurse‐staffing interventions such as education mix or grade mix, or nurse‐staffing levels (e.g. nurse to patient ratios).

Outcomes

We found a range of different patient‐ and staff‐related outcomes reported across studies. We found staff‐related outcomes relating to absenteeism, nursing‐staff retention and nursing‐staff turnover. Patient outcomes included patient mortality, length of stay, hospital days, patient readmissions, attendance at the ED within 30 days of discharge, and other clinical outcomes (see Table 5). Studies also reported outcomes related to costs.

1. Outcomes reported across studies.

Study	Mortality	Length of stay	Patient readmissions	Patients attendances at the ED	Patients with pressure ulcers	Other clinical	Costs	Staff absence	Staff turnover/ retention
Adding advanced or specialist nurses to nursing staff compared to usual nurse staffing
Bakitas 2009	x	x		x		x
Castro 2003			x			x	x
Forster 2005	x	x		x		x
Davies 2001		x	x
Forbes 2006			x		x	x	x
Einstadter 1996		x	x	x
Pozen 1977						x
Plant 2015		x	x	x
Ritz 2000							x
Sisk 2006	x		x	x
Talley 1990		x
The introduction of nursing assistive personnel to the hospital workforce versus usual staffing
Neidlinger 1993							x
Primary nursing compared to usual/team/functional nursing
Boumans 1999								x
Melchoir 1996									x
Gardner 1991							x		x
Shukla 1983						x	x		x
McPhail 1990								x
Staffing models
O'Connor 1992							x		x
Choi 1986									x

Abbreviation

ED: emergency department

Excluded studies

In total we identified 336 studies of hospital nurse staffing. We excluded most of these because the design criteria did not meet the types identified for inclusion in this review (randomised trial, non‐randomised trial, controlled before‐after, interrupted‐time‐series, or repeated‐measures study). We excluded 25 studies because they were of nurse/physician substitution (one of our exclusion criteria). We could not include a further 51 studies that used an eligible design because they were not conducted to the standard required for EPOC reviews (i.e. they used controlled before‐after design, but without at least two intervention and two control sites (n = 26), or they used an interrupted‐time‐series or repeated‐measures study and did not have sufficient data points to meet the standard for inclusion (n = 25)).

We excluded four studies in this update that were included in the original review. Biro 2000 was a study of team midwifery and we decided that midwifery is not the same as nursing. We excluded Duncan 2006 because this was a study of dietary assistants and we judged these staff to be dietetic staff, not nursing staff. In the Feddersen 1994 study, we deemed the intervention to be an educational intervention facilitated by a nurse rather than a nurse‐staffing intervention. Finally, in the Dawes 2007 study, we deemed the intervention to be early discharge, and although facilitated by a nurse, we did not consider it to be a nurse‐staffing intervention.

Risk of bias in included studies

We assessed the risk of bias of all studies using EPOC criteria (EPOC 2018b). Overall, the risk of bias in studies was high, with limitations mostly related to blinding of participants and personnel, allocation concealment, sequence generation, and blinding of outcome assessment. See the overview in Figure 2.

2.

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.

We assessed three of the 11 randomised trials to be at low risk of bias (Forster 2005; Plant 2015; Sisk 2006). Three trials were high risk of bias (Choi 1986; McPhail 1990; Ritz 2000), and the remaining five randomised trials were at moderate risk of bias. We assessed the two non‐randomised trials to be at moderate to high risk of bias (Einstadter 1996; Shukla 1983, respectively). Most of the six controlled before‐after studies had a higher risk of bias than the randomised trials, primarily due to the limitations of controlled before‐after designs. All six of these studies fulfilled the criteria for prespecification of the features to be assessed, adequate recording of what happened in the study, and prospective collection of data pre‐ and post‐intervention. Although we identified a small number of interrupted‐time‐series in our search, none met the criteria for inclusion in the review. The risk of bias of included studies is summarised in Figure 3.

3.

Risk of bias summary: review authors' judgements about each risk of bias item for each included study.

Allocation

We identified a high risk of selection bias in six randomised trials (all older studies) (Choi 1986; Davies 2001; McPhail 1990; Pozen 1977; Ritz 2000; Talley 1990). Although the authors reported randomisation, the method of sequence generation was not discussed and there was no discussion of allocation concealment. Selection bias was present in all controlled before‐after studies and non‐randomised trials.

Blinding

We identified a high risk of performance bias in five randomised trials (Bakitas 2009; Castro 2003; Davies 2001; Ritz 2000; Talley 1990), because neither participants, clinicians or outcome assessors were blinded. Choi 1986; McPhail 1990; Plant 2015, Pozen 1977 and Sisk 2006 did not blind participants or clinicians, but collected outcome data through hospital records, patient questionnaires or blinded research assistants. Participants, clinicians and outcome assessment were not blinded in the two non‐randomised trials (Einstadter 1996; Shukla 1983). None of the controlled before‐after studies blinded participants/clinicians, however, outcome assessment was blinded in three studies (Forbes 2006; Melchoir 1996; O'Connor 1992).

Incomplete outcome data

Data were incomplete in four controlled before‐after studies (Boumans 1999; Forbes 2006; Melchoir 1996; Neidlinger 1993), and two randomised trials (McPhail 1990; Ritz 2000).

Selective reporting

There was no evidence of selective reporting in the majority of studies, but this was unclear in four studies (Choi 1986; Forster 2005; Gardner 1991; Ritz 2000).

Baseline characteristics similar for intervention group and control

All randomised trials conducted a baseline assessment. In six trials the control groups appeared to be similar, but we noted some differences between groups for two trials (Davies 2001; Forster 2005). For two trials it was reported that baseline measures were taken, but the findings were not reported fully (Plant 2015, Sisk 2006). A baseline assessment was not conducted in one non‐randomised trial (Shukla 1983), but control variables were measured during the trial to monitor the implementation of the interventions. Two controlled before‐after studies did not report baseline characteristics for the intervention and control groups (Neidlinger 1993; O'Connor 1992); three controlled before‐after studies reported that baseline data had been collected, but the findings were not reported fully (Boumans 1999; Gardner 1991; Melchoir 1996).

Other potential sources of bias

There were other potential sources of bias identified in nine studies: confounding (Forbes 2006); contamination and response rate differences (McPhail 1990; Melchoir 1996); changes to the intervention (Neidlinger 1993), processes (Ritz 2000), or setting during the study (Gardner 1991); study design (O'Connor 1992); and multiple potential sources of bias (Boumans 1999; Shukla 1983).

Effects of interventions

See: Table 1; Table 2; Table 3; Table 4

Although all included studies examined patient and/or staff‐related outcomes, there was variation between studies in the range of outcomes reported (see Table 5), which impeded the potential for meta‐analysis. In addition, we could not use all data for further analysis as studies used different assessment measures (i.e. mean and median), or reported means without reporting standard deviations. Therefore, we used a narrative approach to describe the outcomes reported by the authors, and where possible, conducted further analysis.

1. The introduction of advanced or specialist nurses to the nursing workforce versus usual staffing

Eleven studies examined the impact of care provided by an NP or CNS (Bakitas 2009; Castro 2003; Davies 2001; Einstadter 1996; Forbes 2006; Forster 2005; Plant 2015; Pozen 1977; Sisk 2006; Ritz 2000; Talley 1990).

Nursing‐staff turnover

No studies included nursing‐staff turnover.

Patient mortality

Three studies reported mortality (1358 participants). Bakitas 2009 reported little or no difference in survival between the intervention (care from an APN with specialist palliative care training) and control group. Median survival for the intervention group was 14 months (95% confidence interval (CI) 10.6 to 18.4 months) and 8.5 months (95% CI 7.0 to 11.1 months) for the usual care group (P = 0.14). Sisk 2006 reported little or no difference in mortality at 12 months (odds ratio (OR) 1.00, 95% CI 0.53 to 1.87) and 18 months (OR 0.87, 95% CI 0.48 to 1.58) in patients with heart failure who received nurse‐managed care versus those receiving usual care. Forster 2005 examined the impact of adding a CNS to physician teams as a nurse team co‐ordinator whose role included retrieving preadmission information, arranging in‐hospital consultations and investigations, as well as organising post‐discharge follow‐up visits and checking on patients post‐discharge with a telephone call. They found little or no difference between the intervention and control groups in relation to rates of in‐hospital or post‐discharge death. The three studies were downgraded because of a serious risk of bias and serious imprecision. The certainty of evidence was low for this outcome and the intervention may lead to little or no difference in patient mortality.

Patient readmissions

Seven studies reported patient readmissions (2995 participants; Castro 2003; Davies 2001; Einstadter 1996; Forbes 2006; Forster 2005; Plant 2015; Sisk 2006).

Two studies reported a reduction in total readmissions/hospitalisations with specialist nurses (Castro 2003; Sisk 2006). For Sisk 2006, these were found at 12 months (OR 0.63, 95% CI 0.46 to 0.86) and at 12 to 18 months (OR 0.57, 95% CI 0.35 to 0.94). Castro 2003 reported a 60% reduction in total readmissions at 12 months (OR 0.19, 95% CI 0.10 to 0.35). These two studies found reductions in disease‐specific readmissions in the intervention group at 12 months: Castro 2003 reported fewer readmissions due to asthma (mean difference (MD) ‐0.50, 95% CI ‐1.00 to 0.00; OR 0.25, 95% CI 0.12 to 0.52) and Sisk 2006 reported fewer hospitalisations for heart failure (OR 0.39, 95% CI 0.17 to 0.89).

Davies 2001 (care from a Diabetes Nurse Specialist), Forbes 2006 (care from an MS Specialist Nurse), and Plant 2015 (Nursing Care Navigator for patients with chronic illness) found little or no difference between groups for readmission. Einstadter 1996 (NP/Nurse Case Manager for medical patients) and Forster 2005 (CNS/Nurse Team Co‐ordinator for medical patients) reported little or no difference between the groups in terms of readmissions within 30 days.

When we combined the readmission data from Forster 2005 and Einstadter 1996 (the only two studies that we could combine for further analysis of this outcome), we found that patients in the intervention group were more likely to be readmitted within 30 days (OR 1.52, 95% CI 1.04 to 2.21). However, we are uncertain whether this intervention reduces or increases patient readmissions, as we assessed the evidence as being of very low certainty for this outcome. The evidence was downgraded due to very serious risk of bias, serious inconsistency, and serious imprecision.

Patient attendances at the ED

Six studies reported on patient attendance at the ED (2274 participants). Castro 2003, Bakitas 2009, Einstadter 1996, Forster 2005, Plant 2015,and Sisk 2006 reported little or no difference between the groups in terms of number of attendances at the ED. We were only able to combine data from two studies for further analysis (Einstadter 1996; Forster 2005), and we found patients in the intervention group had a higher risk of attending the ED within 30 days of discharge (OR 1.20, 95% CI 0.82 to 1.76). However, it is uncertain if this intervention reduces or increases patient attendances at the ED, as we assessed the evidence as being of very low certainty for this outcome. The evidence was downgraded due to very serious risk of bias, serious inconsistency, and serious imprecision.

Length of stay

Three studies reported length of stay (907 participants). Davies 2001 reported a shorter median length of stay (8 days versus 11 days) for diabetes patients receiving care from a diabetes specialist nurse. Talley 1990 reported little or no difference between the intervention and control groups for length of stay (consultation with a Psychiatric Liaison Nurse Specialist for patients assigned a sitter). The Plant 2015 study was the only study that provided data that we could use, and suggested that the intervention probably led to little or no difference in length of stay in the ED or when admitted to a ward. However, it is uncertain if this intervention reduces or increases length of stay, as we assessed the evidence as being of very low certainty for this outcome. The evidence was downgraded due to very serious risk of bias and serious imprecision.

Number of patients with pressure ulcers

One study reported the number of patients with pressure ulcers (753 participants). Forbes 2006 examined a range of complications associated with MS and the only impact identified related to the number of patients with pressure ulcers. Here the intervention group had a marked reduction in the number of patients with pressure ulcers, with a significant group*time effect (Chi² = 12:7, degrees of freedom = 2, P = 0.001). Further analysis of the data confirmed a greater reduction in number of patients with pressure ulcers in the group receiving care from an MS Nurse Specialist at 12 months (OR 4.77, 95% CI 2.14 to 10.65) and at 24 months (OR 9.38, 95% CI 3.24 to 27.14). However, it is uncertain whether this intervention reduces pressure ulcers, as we assessed the evidence as being of very low certainty for this outcome. We downgraded the evidence due to very serious risk of bias.

Costs

Three studies reported costs (617 participants). Studies described reductions in costs associated with length of stay (Davies 2001), and reductions in hospital days (a combination of readmissions and length of stay) (Castro 2003). Castro 2003 reported on direct and indirect cost (total cost) in USD, and we adjusted the reported cost effects to USD 2016. The authors found the intervention reduced the number of readmissions by 60%, which was primarily responsible for a reduction of 69% hospital days per patient and a subsequent reduction in total healthcare costs, reported as MD of USD 8946.61 between intervention and control group. Castro 2003 also reported a reduction of indirect costs in the intervention group, resulting in cost savings of USD 3073.58 per patient. This was mostly related to a reduction in lost workdays and non‐professional/caregiver costs. Conversely, Ritz 2000 reported on charges, as well as reimbursement collected from hospital and clinic billing systems for the two‐year study period. Clinic reimbursement was estimated by multiplying charges with the net revenue received from the insurance divided by the gross charges assessed to this insurance. Not all provider fees were included in the cost analysis (e.g. ED physician fees, and oncologist fees). Also, it remains unclear whether cost outcomes included direct or indirect costs, or both. The adjusted (USD 2016) mean difference between experimental and control group was USD 2458.41 (P = 0.128). The investigators concluded that there was little or no difference between women with breast cancer who received care from an APN and the control group in relation to charges or reimbursement. It is uncertain if this intervention reduces costs, as we assessed the evidence as being of very low certainty for this outcome. The evidence was downgraded due to serious risk of bias, serious inconsistency, serious indirectness, and serious imprecision.

Other outcomes

We identified other objective outcomes in two studies, but they were not included in the seven most important outcomes. Both studies were of adding advanced or specialist nurses to the workforce and the certainty of the evidence in both is very low.

Einstadter 1996 examined the impact of a nurse case manager who provided discharge planning for general medical patients. They found more patients in the intervention group had a scheduled outpatient appointment at the time of discharge, particularly if they were discharged at the weekend, and more patients turned up for their outpatient appointment.

Pozen 1977 examined the impact of care from a nurse rehabilitator on patients with myocardial infarction. They found patients in the intervention group returned to work earlier than those in the control group and more patients quit smoking. However, the intervention had no impact on weight reduction or anxiety scores.

2. The introduction of nursing assistive personnel (NAP) to the hospital workforce versus usual staffing

One study with 6769 participants examined the introduction of NAP into a nursing professional practice model of nursing (Neidlinger 1993). Costs were the only reported outcome.

Costs

The Neidlinger 1993 study examined the impact on personnel costs of adding NAP to the nursing workforce in two acute care hospital units. The trialists found that personnel costs increased by USD 19.28 (USD 2016) per patient day (PPD) in the intervention units “for undetermined reasons”. (see Table 6). It is uncertain whether this intervention reduces or increases costs because the certainty of the evidence is very low. The evidence was downgraded due to very serious risk of bias.

2. Outcomes for addition of nursing assistive personnel to usual nurse staffing.

Neidlinger 1993	Addition of nursing assistive personnel
Personnel costs (mean USD PPD)
	Study	Control
Pre	185	205
Post	212	220
Difference	27	15
Pre‐test mean	185 vs 205
Post‐test mean	212 vs 220
Absolute change (post)	‐8
Relative percentage change (post)	‐3.64
Absolute change from baseline	27 vs 15
Difference in absolute change from baseline	12
Registry (Bank) costs (mean USD PPD)
	Study	Control
Pre	33.21	24.15
Post	8.83	9.32
Difference	‐24.38	‐14.83
Pre‐test mean	33.21 vs 24.15
Post‐test mean	8.83 vs 9.32
Absolute change (post)	‐0.49
Relative percentage change (post)	‐5.26
Absolute change from baseline	‐24.38 vs ‐14.83
Difference in absolute change from baseline	‐9.55

3. Primary nursing versus usual/functional/team nursing

Four studies examined the impact of introducing primary nursing on staff‐related outcomes and costs (Boumans 1999; Gardner 1991; Melchoir 1996; Shukla 1983).

Nursing‐staff turnover

Three studies reported nursing‐staff turnover (> 630 participants). The Melchoir 1996 study found lower turnover rates in nurses in the intervention group. The findings favour the intervention group, but the CI crosses the line of no effect (OR 0.57, 95% CI 0.32 to 1.02). Gardner 1991 examined the impact of primary nursing on nurse retention (the inverse of nursing‐staff turnover), and costs. They identified higher retention rates of nurses in the intervention group over three years (OR 2.33, 95% CI 1.12 to 4.87), particularly in relation to nurses with bachelor's degrees or above. We converted retention rates into turnover rates to combine data from the Melchoir and Gardner studies. This analysis provided an overall result that favoured the intervention (OR 0.51, 95% CI 0.32 to 0.81). In the third study (Shukla 1983), nursing turnover over 12 months was lower in all‐registered nurse (RN) primary nursing (20%), compared to a new modular model of nursing (29%), but higher when compared to the existing team nursing (16%). However, it is uncertain whether this intervention reduces nursing‐staff turnover because the certainty of the evidence is very low. The evidence was downgraded due to very serious risk of bias and serious imprecision.

Costs

Two studies reported costs (> 138 participants). In one study (Shukla 1983), an all‐RN primary nursing model was more expensive (total cost USD 45.78 PPD) than team nursing (USD 35.33 PPD) and a new modular model (USD 44.68 PPD) (USD 2016). Direct personnel costs PPD were also slightly higher in primary nursing than in the other two models. In the second study (Gardner 1991), costs PPD were lower in primary nursing (USD 95.63) than in the usual team nursing (USD 98.5) (USD 2016). The trialists attributed savings to higher patient‐to‐nurse ratios and less use of agency nurses and administrative staff. It is uncertain whether this intervention reduces costs because the certainty of the evidence is very low. The evidence was downgraded due to very serious risk of bias.

Other outcomes

One study examined other objective outcomes that we did not include in the seven most important outcomes in this review (Boumans 1999). These were frequency and duration of staff absence, for which little or no difference between the intervention group (primary nursing) and the control group (functional nursing) was reported.

One study examined infection rates in primary nursing compared with team nursing and modular nursing, and reported little or no difference between the groups (Shukla 1983).

4. Staffing models

One study examined the impact of nursing self‐staffing on nursing‐staff turnover/retention (O'Connor 1992). No other outcomes were reported.

Nursing‐staff turnover

O'Connor 1992 (647 participants) identified a reduction in nursing‐staff turnover that was sustained on units with self‐staffing, in comparison to higher and fluctuating nursing‐staff turnover on other units (see Table 7). It is uncertain whether this intervention reduces staff turnover because the certainty of the evidence is very low. The evidence was downgraded due to very serious risk of bias.

3. Outcomes for self‐staffing versus usual staffing models.

O'Connor 1992	Self‐staffing
	Group A vs control (Group D)
Nursing‐staff turnover (%)	Post 1		Post 2
	Study	Control	Study	Control
Pre	10	28	10	28
Post	11	7	10	29
Difference	1	‐21	0	1
Pre‐test mean	10 vs 28		10 vs 28
Post‐test mean	11 vs 7		10 vs 29
Absolute change (post)	4		‐19
Relative percentage change (post)	57.14		‐65.52
Absolute change from baseline:	1 vs ‐21		0 vs 1
Difference in absolute change from baseline	22		‐1
	Group B vs control (Group D)
Nursing‐staff turnover (%)	Post 1
	Study	Control
Pre	32	28
Post	10	7
Difference	‐22	‐21
Pre‐test mean	32 vs 28
Post‐test mean	10 vs 7
Absolute change (post)	3
Relative percentage change (post)	42.86
Absolute change from baseline	‐21
Difference in absolute change from baseline	‐1
Turnover (%)	Group A vs control (Group C ‐ pre‐tests)
	Post 1
	Study	Control
Pre	10	26
Post	11	24
Difference	1	‐2
Pre‐test mean	10 vs 26
Post‐test mean	11 vs 24
Absolute change (post)	‐13
Relative percentage change (post)	‐54.17
Absolute change from baseline	1 vs ‐2
Difference in absolute change from baseline	3
Nursing‐staff turnover (%)	Group B vs control (Group C ‐ pre‐tests)
	Post 1
	Study	Control
Pre	10	24
Post	11	24
Difference	1	0
Pre‐test mean	10 vs 26
Post‐test mean	11 vs 7
Absolute change (post)	‐13
Relative percentage change (post)	‐54.17
Absolute change from baseline	1 vs ‐21
Difference in absolute change from baseline	1

Discussion

In this review we set out to identify which, if any, nurse‐staffing models in the hospital setting are associated with improved outcomes for patient‐, staff‐related, and economic outcomes. The scope of the review was broad and included a wide range of possible interventions. We sought to identify relevant studies conducted across all jurisdictions and in all languages. Despite the initial identification of 336 studies with eligible interventions, finally we included only 19 studies, primarily due to design and reporting limitations. We could only include 17 studies in our analysis as the other two studies did not provide sufficient information in their results to be included.

Summary of main results

This review included 11 randomised trials, two non‐randomised trials, and six observational (controlled before‐after) studies of four nurse‐staffing interventions. We identified seven outcomes as important: nursing‐staff turnover, patient mortality, patient readmissions, patient attendances at the ED, length of stay, number of patients with pressure ulcers, and costs. The certainty of evidence for one outcome was low, and the findings suggest that adding advanced or specialist nurses to nurse staffing may lead to little or no difference in patient mortality. The certainty of evidence for the remaining six outcomes examined for this intervention is very low and it is uncertain if adding advanced or specialist nurses to nurse staffing reduces any of them. The certainty of evidence for the introduction of NAP to the nursing workforce is very low and it is uncertain if this intervention reduces costs. The certainty of evidence for primary nursing and staffing models is also very low and it is uncertain if they reduce nursing‐staff turnover or costs.

Overall completeness and applicability of evidence

We failed to identify any studies of interventions relating to nurse‐staffing levels, education mix, or grade mix that met our inclusion criteria. This was despite the range of changes that have occurred across countries since the 1980s in relation to nurse education and the introduction of mandatory staffing levels in some states in the USA and Australia.

The coverage of the seven most important outcomes was patchy across the 19 included studies, with no more than seven studies reporting each of the outcomes, and data being available for a maximum of four studies, due to the range of ways in which outcomes were measured or analysed. The scope of the review did not include outcomes that were not considered to be objective measures of patient‐ or staff‐related outcomes. As such, the large volume of published studies that focus on outcomes such as nurse or patient satisfaction, quality of life, burnout, or staff stress were not included.

There was considerable discussion at the beginning of this update process about whether the study protocol should be extended to include nurse/physician substitution. Although a review has been conducted of nurse/physician substitution (Laurant 2018), this does not include hospital‐based nurses. The final decision was to adhere to the original protocol. It is recommended that a separate study be conducted of hospital nurse/physician substitution.

Certainty of the evidence

We identified a large number of papers relating to hospital nurse staffing. However, many papers were commentaries or literature reviews. A large number of studies were of nurse staffing with relevant interventions and outcomes, but were excluded on the basis of inappropriate design. Most of these studies were observational studies and used secondary or administrative data. Despite the shortcomings of such designs, often some of these studies are cited as evidence that the skill mix, grade mix, or educational mix of nursing staff makes a difference to patient outcomes.

The evidence regarding the impact of hospital nurse staffing provided by the final set of studies included in this review is weak and the findings should be treated with caution. Although the use of strict inclusion criteria reduced the amount of evidence available for review, systematic reviews can be very useful in identifying areas where there is insufficient high quality evidence and where further research is required (Egger 2001). The small number of eligible studies and considerable heterogeneity between studies limited the potential for more detailed analyses (e.g. an overall meta‐analysis, subgroup analysis). However, the findings can inform further research on this topic. In particular, the current evidence highlights topics around which findings are limited and where future priorities may lie (e.g. the introduction of minimum nurse‐to‐patient ratios; the impact of nurse education interventions on patient outcomes), or where knowledge is developing and can be enhanced further through research (e.g. the impact of specialist nurse roles on patient outcomes). It also highlights the lack of any consistency between studies in the types of outcome measures used in studies of nurse staffing, and how the measures that are used are operationalised consistently. The limited nature of the evidence to date relating to hospital nurse staffing is also highlighted by Griffiths 2016, which encouraged those considering future studies to consider randomised trials, despite the challenges involved in implementing such studies. This article also encourage researchers to consider the direction of causality and whether nurse staffing precedes outcomes, whether there are other factors besides nurse staffing influencing the outcomes assessed, and other sources of bias in the study design used.

Potential biases in the review process

Most members of the study team are nursing academics, and great care was taken to ensure that the review adhered to Cochrane methodology and EPOC guidance to minimise any potential biases.

Agreements and disagreements with other studies or reviews

We have already identified the limitations of the systematic and literature reviews of hospital nurse staffing conducted previously. Only one review included randomised trials (Carter 2007). Several reviews used secondary or administrative data. Although our review only included randomised trials, non‐randomised trials, controlled before‐after studies, interrupted‐time‐series or repeated‐measure studies, all the existing reviews that we identified also included observational studies and some qualitative studies.

The impact of advanced or specialist nursing roles is also explored in other reviews. Carter 2007 looked at the impact of NPs in the ED and included qualitative and observational studies in the analysis in addition to the three randomised trials identified. The review concluded that NPs could reduce waiting times in the ED and had similar or better outcomes to medical residents in relation to the accuracy of X‐ray examinations, physical examinations, appropriateness of urgent referrals, and patient satisfaction. The De Broe 2001 rapid systematic review failed to find support ‐ other than that based on expert opinion and anecdotal evidence ‐ for the benefits of specialist nurses for patients with MS, diabetes and epilepsy. Our review identified 11 eligible studies of the impact of the specialist nurse roles on patient outcomes and concluded that it may lead to little or no difference in patient mortality and that the effects on other patient outcomes and costs are uncertain due to the low certainty of the evidence in the studies identified.

We found five studies of primary nursing and concluded that the impact on nursing‐staff turnover, nurse absences, and costs is uncertain due to the low certainty of the evidence in the studies identified. The Simon 2014 review identified a range of nurse‐staffing models, none of which would have met our inclusion criteria for study design or study quality. These included two studies of the introduction of a new supervisory post, which was associated with reductions in the number of falls and patients with pressure ulcers, and improved patient satisfaction and nurse job satisfaction. Simon 2014 also identified two studies of the introduction of a total‐patient‐care model versus team nursing, which was not associated with any differences in patient‐ or staff‐related outcomes. Two studies also examined the move from a total‐patient‐care model to team nursing, one of which found little or no difference in relation to patient‐ or staff‐related outcomes, though the other reported significantly higher levels of job satisfaction in the team‐based approach. They found one study that identified that patients had a lower risk of medication administration errors, falls, pneumonia, urinary tract infections, unjustified restraints, and pressure ulcers in clinical areas with professional models of care (higher nurse skills and staffing levels) compared with clinical areas with functional models. A review of non‐traditional staffing models did not draw conclusions overall (Lookinland 2005). The Fernandez 2012 review included studies comparing team nursing, primary care, functional nursing, and case management models. Fernandez 2012 found changes to some models were associated with lower patient pain scores, medication errors, patient care quality, restraint use and seclusion, but little or no difference in relation to length of stay or patient satisfaction. For staff‐related outcomes, they found no evidence of an association between nursing models and satisfaction, absenteeism and role clarity/confusion. Simon 2014 included one study of the use of a nurse‐staffing model based on nursing hours per patient day, which was associated with improved patient outcomes (reduction in patient complications and mortality). We also found one study of staffing models that the authors associated with an improvement in nursing‐staff turnover that was sustained. However, we concluded that the effect on nursing‐staff turnover is uncertain due to the low certainty of evidence. Simon 2014 also included reviews of Magnet versus non‐Magnet hospitals and patient outcomes to infer the impact of hospital organisation on patient and staff‐related outcomes, but these go beyond the scope of nurse staffing.

With regard to replacing the proportion of registered nurses with licensed practical nurses, licensed vocational nurses, or nursing assistants, some authors suggest there is no or little evidence to suggest that it compromises the quality of patient care (Crossan 2005; Currie 2005). Lankshear 2005 found one study that associated higher levels of licenced practical nurses/licensed vocational nurses with higher rates of patient complications. Spilsbury 2001 suggested the evidence showed RNs do make a difference, but the research failed to offer guidance regarding the most effective skill mix to provide "best" patient care. We only identified one eligible study that related to the impact of replacing RNs with unqualified support staff, and could not be certain about the impact on costs due to the low certainty of the evidence. Griffiths 2014 identified 22 studies of HCA staffing levels or nursing skill mix. Studies varied in their quality and the reviewers concluded that there was "no evidence to support a positive role of HCAs in patient safety outcomes. Some evidence points to a negative effect". The one study included in our review reported higher costs, although this type of skill‐mixing is often introduced as a cost‐saving measure.

Several reviews have supported the association between higher nurse‐staffing levels and better patient outcomes (Crossan 2005; Currie 2005; Kane 2007; Kravitz 2002; Lankshear 2005), better staff‐related outcomes (Currie 2005), and between a higher proportion of RNs and better patient outcomes (Currie 2005). However, Lankshear 2005 identified one study that did not support an association between staffing levels and patient outcomes. Lang 2004 suggested that the literature offers minimal support for specific minimum nurse‐to‐patient ratios in the acute hospital setting, but there are other factors involved in the quality of care that should be considered in addition to nurse‐staffing ratios. A more recent review of nurse staffing on acute adult inpatient wards concluded that there was good evidence that higher nurse‐staffing levels were associated with lower rates of mortality, failure to rescue (defined elsewhere as death
 among patients with treatable complications (Griffiths 2008)), length of stay, and readmissions (Griffiths 2014). Griffiths report that three high‐quality studies associated lower levels of nurse staffing with higher rates of drug administration errors (although this was disputed in another low‐quality study) and missed nursing care. They reported weak or mixed evidence of the impact of staffing levels on hospital‐acquired infections, falls, pressure ulcers, and costs of care. They reported no association with rates of venous thromboembolism, patient satisfaction, and staff‐related outcomes. The Drennan 2014 review of staffing levels in the ED identified conflicting results of studies on the introduction of mandatory nurse‐to‐patient ratios. In one study, a mandatory ratio was associated with a significant increase in waiting times and admission times, in another it was associated with a 16% reduction in waiting times. Drennan attributed these contradictory results to differences in how the studies were conducted. They found a weak association between staffing levels and number of patients leaving without being seen, emergency care time, medication errors, time to antibiotic administration for patients with pneumonia, and nurse absenteeism. With regard to the evidence to support the impact of staffing levels on patient or staff‐related outcomes, our review failed to identify any eligible studies of staffing levels.

With regard to nursing shifts, Estabrooks 2009 stated that there was insufficient evidence to suggest that shifts affect patient or provider outcomes. Although we did identify studies of nursing shifts, only one was eligible for inclusion and did not provide report sufficient information on results to be included in our analysis.

We found no eligible studies relating to education mix. Kane 2007, which drew again on observational studies, identified a significant negative correlation between the proportion of Bachelor Degree (BSN) nurses in nursing staff and the incidence of patient deaths.

We highlight the lack of evidence about the impact of hospital nurse‐staffing models on patient and staff‐related outcomes, despite the number of studies that have been conducted. This lack of evidence is also highlighted by several review authors (e.g. Drennan 2014; Griffiths 2014; Simon 2014), who documented the limitations in the evidence base due to the small number of studies conducted, and an overall lack of rigour due to design issues such as sample size, methodology and means of measurement. In addition, Spilsbury 2001 identified a tendency for researchers to measure grade mix rather than skill mix, and a lack of coherence in definitions of roles and in the tools used in studies, which makes it difficult to compare research studies. Our review supports this finding regarding the quality of evidence. Furthermore, the restriction of our review to only those study designs that provide the highest level of evidence to support the impact of interventions on patient and staff‐related outcomes (randomised trials, non‐randomised trials, controlled before‐after studies, interrupted‐time‐series and repeated‐measures studies) helps to demonstrate the lack of high quality evidence around this broad topic and the need for more robust research.

Authors' conclusions

Implications for practice.

It is difficult to identify a form of best practice from this review, despite the number of studies that have been conducted on hospital nurse staffing. It is clear that more robust study designs are required in the future to generate good evidence of the impact of different nurse‐staffing models on patient and staff‐related outcomes.

We found low quality evidence to suggest there may be no relationship between nurse staffing and patient mortality. The impact of nurse staffing on other patient outcomes, on nurse‐staffing turnover, and on costs is unclear due to the very low certainty of the evidence.

Implications for research.

This review highlights the limited nature of the research conducted on this topic. More specifically, it highlights the large number of studies conducted in the area that were not of an appropriate design, and so cannot be considered as an adequate source of evidence on the impact of nurse‐staffing models on patient‐, staff‐related, or economic outcomes.

The limitations of the included studies highlight the need for larger studies, preferably using the following designs: randomised or non‐randomised trials, controlled before‐after studies, interrupted‐time‐series and repeated‐measures studies (with several data points pre‐ and post‐intervention). It is important that researchers publish their results fully to facilitate further analysis of their findings and use appropriate frameworks to enhance the quality of their reports such as CONSORT (CONSORT).

This review also highlighted a diverse range of patient outcomes used to measure the impact of hospital nurse staffing and differences in how these outcomes are captured. This suggests there may be merit in developing a set of core standardised outcomes to be used in studies of nurse staffing, such as those developed for other healthcare areas by the COMET initiative (COMET).

While this review highlights the inadequacies of research conducted across nurse‐staffing interventions generally, it particularly highlights the need for research in relation to educational, grade mix and staffing level interventions.

What's new

Date	Event	Description
2 November 2018	New citation required and conclusions have changed	Changes in findings: based on the application of GRADE, this update provides less confidence in demonstrable effects of changes to nurse staffing on patient, staff and cost outcomes. We excluded four studies from the original review and included eight new studies. The total number of included studies in the review is 19.
2 November 2018	New search has been performed	This is the first update of the Cochrane Review last published in 2011. We conducted a new search, added cost outcomes and included eight new studies. Changes in authorship include the addition of four new authors.

History

Protocol first published: Issue 1, 2008
 Review first published: Issue 7, 2011

Date	Event	Description
8 March 2018	Amended	Changes in authorship: The following changes were made in authorship since the original protocol was published: 1. Rita Collins, Eileen Vilis, and Donal O'Mathuna resigned from the review team. 2. Leigh Kinsman, Thomas Rotter, Robyn Kelly, and Jonathan Beaumier joined the review team.
7 March 2018	Amended	Protocol revisions: 1) Objective added: 3. To identify the impact of staffing model/s on economic outcomes. 2) Outcomes added: Economic outcomes:We will consider any objective measure of economic outcome e.g. incremental resource use, incremental costs, incremental cost‐effectiveness such as cost/life year saved, cost/QALY, cost/DALY. 3) Databases added to search: Economic databases: NHS EED, CEA Registry.

Appendices

Appendix 1. Search Strategies

Medline (OVID)

including In‐Process & Other Non‐Indexed Citations and Versions

o.	Search terms
1	*nurse clinicians/
2	advanced practice nursing/
3	(nurs* adj1 (clinician? or specialist? or expert?)).ti,ab.
4	(advance? practice adj1 nurs*).ti,ab.
5	((nurse or nurses or nursing) adj1 (assistant? or assistive personnel)).ti,ab.
6	((usual or conventional) adj4 nursing).ti,ab.
7	((nurse or nursing) adj1 (consultant? or advisor?)).ti,ab.
8	((nurse or nurses or nursing) adj2 (roster? or rostering)).ti,ab.
9	(fewer adj2 ("rn" or "rns" or nurses or nurse or registered nurse?)).ti,ab.
10	("nurse? patient? ratio?" or "patient? nurse? ratio?").ti,ab.
11	(nurs* and (mix or skillmix)).ti,ab.
12	nursing service, hospital/og
13	nursing staff, hospital/og
14	nursing staff, hospital/sd
15	or/1‐14
16	hospital?.ti,ab,hw.
17	nursing team/
18	nurse practitioners/
19	(nurs* adj3 (staffing or delivery or model?)).ti,ab.
20	or/17‐19
21	16 and 20
22	15 or 21
23	randomized controlled trial.pt.
24	controlled clinical trial.pt.
25	multicenter study.pt.
26	pragmatic clinical trial.pt.
27	(randomis* or randomiz* or randomly).ti,ab.
28	groups.ab.
29	(trial or multicenter or multi center or multicentre or multi centre).ti.
30	(intervention? or effect? or impact? or controlled or control group? or (before adj5 after) or (pre adj5 post) or ((pretest or pre test) and (posttest or post test)) or quasiexperiment* or quasi experiment* or pseudo experiment* or pseudoexperiment* or evaluat* or time series or time point? or repeated measur*).ti,ab.
31	non‐randomized controlled trials as topic/
32	interrupted time series analysis/
33	controlled before‐after studies/
34	or/23‐33
35	exp animals/
36	humans/
37	35 not (35 and 36)
38	review.pt.
39	meta analysis.pt.
40	news.pt.
41	comment.pt.
42	editorial.pt.
43	cochrane database of systematic reviews.jn.
44	comment on.cm.
45	(systematic review or literature review).ti.
46	or/37‐45
47	34 not 46
48	22 and 47
49	Economics/
50	Value of life/
51	exp "Costs and Cost Analysis"/
52	exp Economics, Hospital/
53	exp Economics, Medical/
54	Economics, Nursing/
55	Economics, Pharmaceutical/
56	exp "Fees and Charges"/
57	exp Budgets/
58	budget*.ti,ab.
59	cost*.ti.
60	(economic* or pharmaco?economic*).ti.
61	(price* or pricing*).ti,ab.
62	(cost* adj2 (effective* or utilit* or benefit* or minimi* or unit* or estimat* or variable*)).ab.
63	(financ* or fee or fees).ti,ab.
64	(value adj2 (money or monetary)).ti,ab.
65	or/49‐64
66	47 or 65
67	22 and 66

Embase (OVID)

Embase 1974 to 2018 March 21

No.	Search terms
1	*advanced practice nurse/
2	*clinical nurse specialist/
3	*expert nurse/
4	*nurse consultant/
5	(nurs* adj1 (clinician? or specialist? or expert?)).ti,ab.
6	(advance? practice adj1 nurs*).ab.
7	((nurse or nurses or nursing) adj1 (assistant? or assistive personnel)).ti,ab.
8	((usual or conventional) adj4 nursing).ti,ab.
9	((nurse or nursing) adj1 (consultant? or advisor?)).ti,ab.
10	((nurse or nurses or nursing) adj2 (roster? or rostering)).ti,ab.
11	(fewer adj2 ("rn" or "rns" or nurses or nurse or registered nurse?)).ti,ab.
12	("nurse? patient? ratio?" or "patient? nurse? ratio?").ti,ab.
13	(nurs* and (mix or skillmix)).ti,ab.
14	*nursing staff/
15	*nursing/
16	or/14‐15
17	*"organization and management"/
18	16 and 17
19	(nurs* adj3 (staffing or delivery or model?)).ti,ab.
20	exp *nurse practitioner/
21	or/18‐20
22	hospital?.ti,ab,hw.
23	21 and 22
24	or/1‐13,23
25	randomized controlled trial/
26	controlled clinical trial/
27	quasi experimental study/
28	pretest posttest control group design/
29	time series analysis/
30	experimental design/
31	multicenter study/
32	(randomis* or randomiz* or randomly).ti,ab.
33	groups.ab.
34	(trial or multicentre or multicenter or multi centre or multi center).ti.
35	(intervention? or effect? or impact? or controlled or control group? or (before adj5 after) or (pre adj5 post) or ((pretest or pre test) and (posttest or post test)) or quasiexperiment* or quasi experiment* or pseudo experiment* or pseudoexperiment* or evaluat* or time series or time point? or repeated measur*).ti,ab.
36	or/25‐35
37	(systematic review or literature review).ti.
38	"cochrane database of systematic reviews".jn.
39	exp animals/ or exp invertebrate/ or animal experiment/ or animal model/ or animal tissue/ or animal cell/ or nonhuman/
40	human/ or normal human/ or human cell/
41	39 not (39 and 40)
42	37 or 38 or 41
43	36 not 42
44	health economics/
45	exp economic evaluation/
46	exp health care cost/
47	exp fee/
48	budget/
49	funding/
50	budget*.ti,ab.
51	cost*.ti.
52	(economic* or pharmaco?economic*).ti.
53	(price* or pricing*).ti,ab.
54	(cost* adj2 (effective* or utilit* or benefit* or minimi* or unit* or estimat* or variable*)).ab.
55	(financ* or fee or fees).ti,ab.
56	(value adj2 (money or monetary)).ti,ab.
57	or/44‐56
58	43 or 57
59	24 and 58

The Cochrane Library

No.	Search terms
#1	[mh "nurse clinicians"]
#2	[mh "advanced practice nursing"]
#3	(nurs* near/1 (clinician* or specialist* or expert*)):ti,ab
#4	(advance* practice near/1 nurs*):ti,ab
#5	((nurse or nurses or nursing) near/1 (assistant? or assistive personnel)):ti,ab
#6	((usual or conventional) near/4 nursing):ti,ab
#7	((nurse or nursing) near/1 (consultant* or advisor*)):ti,ab
#8	(nurs* near/2 roster*):ti,ab
#9	(fewer near/2 ("rn" or "rns" or nurses or nurse or registered nurse*)):ti,ab
#10	("nurse* patient* ratio*" or "patient* nurse* ratio*"):ti,ab
#11	(nurs* and (mix or skillmix)):ti,ab
#12	[mh "nursing service, hospital"/OG]
#13	[mh "nursing staff, hospital"/OG,SD]
#14	{or #1‐#13}
#15	[mh "nursing team"]
#16	[mh "nurse practitioners"]
#17	(nurs* near/3 (staffing or delivery or model?)):ti,ab
#18	{or #15‐#17}
#19	hospital?:ti,ab,kw
#20	#18 and #19
#21	#14 or #20

CINAHL (EBSCO)

No.	Search terms
S1	(MH "Advanced Practice Nurses") OR (MH "Clinical Nurse Specialists")
S2	TI (nurs* N1 (clinician* or specialist* or expert*))
S3	AB (nurs* N1 (clinician* or specialist* or expert*))
S4	TI (advance* practice N1 nurs*)
S5	AB (advance* practice N1 nurs*)
S6	TI ((nurse or nurses or nursing) N1 (assistant* or assistive personnel))
S7	AB ((nurse or nurses or nursing) N1 (assistant* or assistive personnel))
S8	TI ((usual or conventional) N4 nursing)
S9	AB ((usual or conventional) N4 nursing)
S10	TI ((nurse or nursing) N1 (consultant* or advisor*))
S11	AB ((nurse or nursing) N1 (consultant* or advisor*))
S12	TI ((nurse or nurses or nursing) N2 (roster* or rostering))
S13	AB ((nurse or nurses or nursing) N2 (roster* or rostering))
S14	TI (fewer N2 ("rn" or "rns" or nurses or nurse or registered nurse*))
S15	AB (fewer N2 ("rn" or "rns" or nurses or nurse or registered nurse*))
S16	TI ("nurse* patient* ratio*" or "patient* nurse* ratio*")
S17	AB ("nurse* patient* ratio*" or "patient* nurse* ratio*")
S18	TI (nurs* and (mix or skillmix))
S19	AB (nurs* and (mix or skillmix))
S20	(MM "Nursing Staff, Hospital/MA/OG")
S21	S1 OR S2 OR S3 OR S4 OR S5 OR S6 OR S7 OR S8 OR S9 OR S10 OR S11 OR S12 OR S13 OR S14 OR S15 OR S16 OR S17 OR S18 OR S19 OR S20
S22	(MH "Nurse Practitioners+")
S23	TI (nurs* N3 (staffing or delivery or model*))
S24	AB (nurs* N3 (staffing or delivery or model*))
S25	S22 OR S23 OR S24
S26	hospital*
S27	S25 AND S26
S28	S21 OR S27
S29	PT randomized controlled trial
S30	PT clinical trial
S31	PT research
S32	(MH "Randomized Controlled Trials")
S33	(MH "Clinical Trials")
S34	(MH "Intervention Trials")
S35	(MH "Nonrandomized Trials")
S36	(MH "Experimental Studies")
S37	(MH "Pretest‐Posttest Design+")
S38	(MH "Quasi‐Experimental Studies+")
S39	(MH "Multicenter Studies")
S40	(MH "Health Services Research")
S41	TI ( randomis* or randomiz* or randomly) OR AB ( randomis* or randomiz* or randomly)
S42	TI (trial or effect* or impact* or intervention* or before N5 after or pre N5 post or ((pretest or "pre test") and (posttest or "post test")) or quasiexperiment* or quasi W0 experiment* or pseudo experiment* or pseudoexperiment* or evaluat* or "time series" or time W0 point* or repeated W0 measur*) OR AB (trial or effect* or impact* or intervention* or before N5 after or pre N5 post or ((pretest or "pre test") and (posttest or "post test")) or quasiexperiment* or quasi W0 experiment* or pseudo experiment* or pseudoexperiment* or evaluat* or "time series" or time W0 point* or repeated W0 measur*)
S43	S29 OR S30 OR S31 OR S32 OR S33 OR S34 OR S35 OR S36 OR S37 OR S38 OR S39 OR S40 OR S41 OR S42
S44	S28 AND S43

ClinicalTrials.gov

Search terms

nurse staffing

WHO International Clinical Trials Registry Platform (ICTRP)

Search terms

nurse staffing

Appendix 2. Original cost data

ID

Currency

Costing Year

Study Period

Perspective

Direct cost

Indirect cost

N‐E

Mean‐E

SD

N‐C

Mean‐C

SD

MD

P value MD

Castro 2003

USD

1999

Not reported

50

5,726

5,679

46

12,188

19,352

6462

0.003

Ritz 2000

USD

Not reported (1996)

1995‐1997

Not reported

Not reported

Not reported

78

34,100

19,245

74

32,399

25,481

1701

0.128

Neidlinger 1993

USD

Not reported (1991)

January 1990 to June 1991

Not reported

Personnel cost

Not reported

27

Not reported

15

12 USD Mean‐E

Gardner 1991

USD

Not reported

Not reported

Not reported

Personnel cost based on DRGs

59.52

61.31

2.33

0.12

Shukla 1983

USD

Not reported

Jan‐Oct, but no year reported

Not reported

Personnel cost

Only reported as total cost

22.12 (RN‐M)

21.59 (M‐M)

20.19 (T‐M)

O'Connor 1992

USD

1989

1988‐1990

Not reported

Not reported

See Appendix 4 and Appendix 5

Abbreviations: N‐E: number of participants in experimental group; E: experimental; SD: standard deviation; N‐C: number of participants in control group; MD: mean difference; DRG: diagnosis‐related group; RN‐M: Registered Nurse model; M‐M: modular model; T‐M: team model; USD: USA dollars

Appendix 3. Cost data adjusted to 2016 USD

ID

Currency

Costing Year

Study Period

Perspective

Direct cost

Indirect cost

N‐E

Mean‐E

SD

N‐C

Mean‐C

SD

MD

P value MD

Castro 2003

USD

1999

Not reported

50

7,927.62

7,862.6

46

16,874.2

26,793

8946.61

0.003

Ritz 2000

USD

1996

1995‐1997

Not reported

Not reported

Not reported

78

49,283.8

27,814

74

46,825.4

36,827

2458.41

0.128

Neidlinger 1993

USD

1991

January 1990 to June 1991

Not reported

Personnel cost

Not reported

43.38

Not reported

24.1

19.28 USD Mean‐E

Gardner 1991

USD

1991

Not reported

Not reported

Personnel cost based on DRGs

95.63

98.5

3.74

0.12

Shukla 1983

USD

1983

Jan‐Oct, but no year reported

Not reported

Personnel cost

Only reported as total cost

45.78 (RN‐M)

44.68 (M‐M)

41.78 (T‐M)

Abbreviations: N‐E: number of participants experimental group; E: experimental; SD: standard deviation; N‐C: number of participants control group; MD: mean difference; DRG: diagnosis‐related group; RN‐M: Registered Nurse model; M‐M: modular model; T‐M: team model; USD: USA dollars

Appendix 4. Adjusted cost of nursing‐staff turnover in intervention group A (self‐staffing)

(O'Connor 1992)	Costing year	Currency	Turnover rate	Scenario 1
60 nursing staff	1989	USD	18%	USD 16
	1989	USD		921,600
Adjusted target price year	(IMF) 2016	USD	18%	1,586,537
Abbreviations: IMF: International Monetary Fund; USD: USA dollars

Appendix 5. Adjusted cost of nursing‐staff turnover in group B (self‐staffing)

(O'Connor 1992)	Costing Year	Currency	Turnover Rate
Turnover rate (varying turnover)			37%
Average RN @ USD 14.92 per hour	1989	USD
Plus lump sum cost factor to replace a professional nurse was applied
Lump sum = USD 30,000 per nurse	1989	USD	5,552,000
Adjusted target price year	(IMF) 2016	USD	9,557,783
Abbreviations: IMF: International Monetary Fund; RN: Registered Nurse; USD: USA dollars

Appendix 6. Full evidence profiles

Comparison: the introduction of advanced or specialist nurses to nursing workforce versus usual nurse staffing

Certainty assessment of evidence for each outcome

No of studies	Study design	Risk of bias	Inconsistency	Indirectness	Imprecision	Other considerations	Certainty (overall score)
Outcome: patient mortality
3	Randomised trials	Serious risk of bias (‐1)	No serious inconsistency (0)	No serious indirectness (0)	Serious imprecision (‐1)	None	⨁⨁◯◯ Low
Outcome: patient readmissions
7	5 randomised trials, 1 non‐randomised trial, 1 observational study	Very serious risk of bias (‐2)	Serious inconsistency (‐1)	No serious indirectness (0)	Serious imprecision (‐1)	None	⨁◯◯◯ Very low
Outcome: patient attendances at the ED
6	5 randomised trials, 1 non‐randomised trial	Very serious risk of bias (‐2)	Serious inconsistency (‐1)	No serious indirectness (0)	Serious imprecision (‐1)	None	⨁◯◯◯ Very low
Outcome: length of stay
3	Randomised trials	Very serious risk of bias (‐2)	No serious inconsistency (0)	No serious indirectness (0)	Serious imprecision (‐1)	None	⨁◯◯◯ Very low
Outcome: patients with pressure ulcers
1	CBA	Very serious risk of bias (‐2)	No serious inconsistency (0)	No serious indirectness (0)	No serious imprecision (0)	None	⨁◯◯◯ Very low
Outcome: costs
3	Randomised trials	Serious risk of bias (‐1)	Serious inconsistency (‐1)	Serious indirectness (‐1)	Serious imprecision (‐1)	None	⨁◯◯◯ Very low

Certainty score

Moderate: this research provides a good indication of the likely effect. The likelihood that the effect will be substantially different^a is moderate.

Low: this research provides some indication of the likely effect. However, the likelihood that it will be substantially different^a is high.
 Very low: this research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different^a is very high.

^aSubstantially different = a large enough difference that it might affect a decision

Comparison: the introduction of nursing assistive personnel to the hospital workforce versus usual staffing

Certainty assessment of evidence for each outcome

№ of studies	Study design	Risk of bias	Inconsistency	Indirectness	Imprecision	Other considerations	Certainty (overall score)
Costs
1	Controlled before‐after study	Very serious risk of bias (‐2)	No serious inconsistency (0)	No serious indirectness (0)	No serious imprecision (0)	None	⨁◯◯◯ Very low

Certainty score

Moderate: this research provides a good indication of the likely effect. The likelihood that the effect will be substantially different^a is moderate.

Low: this research provides some indication of the likely effect. However, the likelihood that it will be substantially different^a is high.

Very low: this research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different^a is very high.

^aSubstantially different = a large enough difference that it might affect a decision

Comparison: primary nursing compared to usual/team/functional nursing for staff‐related outcomes

Certainty assessment of evidence for each outcome

No of studies	Study design	Risk of bias	Inconsistency	Indirectness	Imprecision	Other considerations	Certainty (overall score)
Outcome: staff turnover
3	2 controlled before‐after studies, 1 non‐randomised trial	Very serious risk of bias (‐2)	No serious inconsistency (0)	No serious indirectness (0)	Serious imprecision (‐1)	None	⨁◯◯◯ Very low
Outcome: costs
2	1 controlled before‐after study, 1 non‐randomised trial	Very serious risk of bias (‐2)	No serious inconsistency (0)	No serious indirectness (0)	No serious imprecision (0)	None	⨁◯◯◯ Very low

Certainty score

Moderate: this research provides a good indication of the likely effect. The likelihood that the effect will be substantially different^a is moderate.

Low: this research provides some indication of the likely effect. However, the likelihood that it will be substantially different^a is high.

Very low: this research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different^a is very high.

^aSubstantially different = a large enough difference that it might affect a decision

Comparison self‐staffing versus usual staffing

Certainty assessment of evidence for each outcome

No of studies	Study design	Risk of bias	Inconsistency	Indirectness	Imprecision	Other considerations	Certainty (overall score)
Outcome: staff turnover
1	Observational study	Very serious risk of bias (‐2)	No serious inconsistency (0)	No serious indirectness (0)	No imprecision (0)	None	⨁◯◯◯ Very low

Certainty score

Moderate: this research provides a good indication of the likely effect. The likelihood that the effect will be substantially different^a is moderate.

Low: this research provides some indication of the likely effect. However, the likelihood that it will be substantially different^a is high.

Very low: this research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different^a is very high.

^aSubstantially different = a large enough difference that it might affect a decision

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Bakitas 2009.

Methods	Randomised trial
Participants	332 people newly diagnosed (within 8‐12 weeks) with advanced cancer of the gastrointestinal tract (unresectable stage III or IV), lung (stage IIIB or IV non–small cell or extensive small cell), genitourinary tract (stage IV), or breast (stage IV and visceral crisis, lung or liver metastasis, oestrogen‐receptor negative (ER−), human epidermal growth factor receptor 2‐positive (Her 2 neu)) cancer Patients identified at the Norris Cotton Cancer Center’s tumour boards with a life‐limiting cancer (prognosis of approximately 1 year)
Interventions	Intervention: a multicomponent, psychoeducational intervention (Project ENABLE (Educate, Nurture, Advise, Before Life Ends)) conducted by APNs consisting of 4 weekly educational sessions and monthly follow‐up sessions until death or study completion. Control: usual care in which patients were allowed to use all oncology and supportive services without restrictions including referral to the institutions’ interdisciplinary palliative care service.
Outcomes	Death ED visits Length of stay Days in intensive care unit Quality of life Symptom intensity Mood
Country/Setting	USA: 2 primary sites (Norris Cotton Cancer Center, New Hampshire; VA Medical Center, Vermont)
Notes	Department of Defense Clinical Nursing Researcher Award, American Cancer Society Doctoral Fellowship, NIH/National Institute of Nursing Research grant T32NR008346. National Cancer Institute grant R01 CA101704. Sponsors had no role in the research. No financial disclosures.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	A stratified randomisation scheme developed for each of the 2 primary sites (Norris Cotton Cancer Center, VA Medical Center). The schemes were stratified by disease and blocked within strata (block lengths of 2 and 4 varied randomly).
Allocation concealment (selection bias)	Unclear risk	Not discussed
Blinding of participants and personnel (performance bias) All outcomes	High risk	Participants notified of allocation ‐ mainly self‐report data (quality of life, Edmonton Symptom Assessment, mood)
Blinding of outcome assessment (detection bias) All outcomes	High risk	Apparently no blinding of assessors
Incomplete outcome data (attrition bias) All outcomes	Low risk	A number of participants died during the trial (analysed by intention‐to‐treat) ‐ not surprising given a palliative population. Slightly more deaths and withdrawals in control group; but sample size at last endpoint (13 months) was slightly greater in the control group. Did not look like any systematic bias in incomplete outcome data.
Selective reporting (reporting bias)	Low risk	None apparent
Baseline characteristics similar for intervention group and control?	Low risk	Characteristics reported and both groups were similar.
Other bias	Low risk	None apparent

Boumans 1999.

Methods	Controlled before‐after study
Participants	Nurses working on 5 units in a 850‐bed hospital in the Netherlands. 5 units were: 2 surgical units (units A and C), 2 internal medicine units (units B and D) and 1 orthopaedic unit (unit E). Units A and B made up the experimental group (group 1); units C, D and E the control group (group 2) (see Figure 1 in trial report). The units were selected on the basis of comparable size, staff structure, bed capacity and patient population. Before the implementation of Primary Nursing, all 5 units used a Functional Nursing system. The sample comprised 145 nurses at T1, 131 nurses at T2 and 119 nurses at T3. A total of 59 nurses (57 females and 2 males) participated at all 3 measuring moments; 23 in group 1 and 36 in group 2. These 59 nurses were included in the analyses.
Interventions	Intervention: Dutch version of primary nursing introduced to 2 units (1 surgical and 1 medical) in a Dutch hospital. This comprised the following: each unit was divided into 2 teams in each team 2 RNs were responsible for a specific group of about 6 patients this patient allocation lasted 8 hours a day (1 work shift) 5 days a week RNs used the nursing process as the basis for practice. Control: 3 units using a functional nursing system and selected on the basis of comparable size, staff structure, bed capacity and patient population to the intervention units
Outcomes	Absence frequency and duration Job satisfaction Experience of job significance Health complaints
Country/Setting	The Netherlands: 850‐bed hospital
Notes	Absence was the only outcome relevant to this review. Funding not reported. No interests disclosed.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Not done ‐ CBA design
Allocation concealment (selection bias)	High risk	Not done ‐ CBA design
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not possible in this type of design. Discussion referred to "contamination" and "Hawthorn effect"
Blinding of outcome assessment (detection bias) All outcomes	High risk	Not possible in this type of study. Outcomes were self‐reported (rather than recorded from the hospitals systems) and lack of blinding may have impacted on outcomes.
Incomplete outcome data (attrition bias) All outcomes	High risk	Response rates ranged from 63% to 100%, average response rate was 83%. However, only 59 nurses responded at all 3 time periods, therefore the actual response rate was much lower than that reported. This risk of bias (more motivated respondents) is mentioned in the Discussion.
Selective reporting (reporting bias)	Low risk	No evidence of selective outcome reporting.
Baseline characteristics similar for intervention group and control?	High risk	Only means were presented, no other descriptives provided, no testing of distributions was presented. Given that absence frequency (number of times absent) had a mean < 1, it could reasonably be expected to follow a Poisson distribution. This was not mentioned in the results ‐ just t‐tests used
Other bias	Unclear risk	Several sources of bias: survey response ‐ only participants in all 3 stages were included in the analysis (59 nurses out of a possible 145 at T1,131 at T2,119 at T3); response rate in control units = 100%; in intervention units = 63%; absence data were self‐reported rather than collected from hospital system. The authors cited research to suggest that this was a reliable source in healthcare workers; intervention introduced in control units after T2. Therefore control was only valid at T2.

Castro 2003.

Methods	Randomised trial
Participants	96 participants, all admitted with the primary admitting diagnosis of asthma between September 1996 and July 1999
Interventions	Intervention: provision of an asthma nurse specialist to provide a multifaceted approach to asthma care for 'high‐risk' inpatients. Control: usual care provided by private primary care physician
Outcomes	Hospital patient readmissions Costs Quality of life
Country/Setting	USA: Barnes‐Jewish Hospital, Washington
Notes	No funding source reported. Declared no conflict of interest
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Patients randomly assigned to intervention or usual care group using a prerandomised assignment.
Allocation concealment (selection bias)	Low risk	Blind concealment sequence allocation using a prerandomised assignment in a sealed envelope.
Blinding of participants and personnel (performance bias) All outcomes	High risk	The patients and healthcare team were not blinded to treatment assignment due to the nature of the intervention.
Blinding of outcome assessment (detection bias) All outcomes	High risk	Quote: "Three consecutive nurses provided the intervention and collected the data for the study."
Incomplete outcome data (attrition bias) All outcomes	Low risk	Primary outcomes reported.
Selective reporting (reporting bias)	Low risk	All primary outcome data reported.
Baseline characteristics similar for intervention group and control?	Low risk	Baseline characteristics of both groups reported comprehensively.
Other bias	Low risk	No other bias evident within paper.

Choi 1986.

Methods	Randomised trial
Participants	792 nurses (RNs and LPNs) regularly assigned to nursing stations on 18 medical and surgical units at the participating hospital
Interventions	Interventions: 3 different shift models were implemented in the experimental units: straight shifts; computer assisted scheduling (called "compflex"); unit designed its own schedule (called "select‐a‐plan").
Outcomes	Nurse satisfaction and retention
Country/Setting	USA: large (788 bed) tertiary‐care hospital
Notes	Supported in part by funding made available by RMH Health Services, Incorporated. No conflict of interest declared.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Stratified sampling, randomisation at level of station rather than individual. Not clear how randomisation was conducted.
Allocation concealment (selection bias)	High risk	Not discussed, probably not possible
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not done and probably not possible to blind, and outcome likely to be susceptible to lack of blinding
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Not done, probably not possible
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	98% response rate reported, but data reporting incomplete.
Selective reporting (reporting bias)	Unclear risk	Outcome reporting unclear
Baseline characteristics similar for intervention group and control?	Low risk	Quote: "before the intervention, there was a difference in only one scale ‐ experience in privacy of work ‐ judged to be inconsequential"
Other bias	Low risk	None apparent

Davies 2001.

Methods	Randomised trial
Participants	300 patients admitted to the medical and surgical wards at University Hospital of Wales, Cardiff with type 1 or 2 diabetes (n = 148 intervention group) (n = 152 control group) 14 participants missing from primary outcomes, 153 from questionnaire (focusing on patient knowledge, diabetes quality of life, post‐discharge events, subsequent attendances, contacts with primary and social care and time away from normal activities) sent 1 month post discharge.
Interventions	Intervention: care and advice from a Diabetes Specialist Nurse (DSN) in addition to standard care. DSN care was individual structured patient education appropriate to need, practical management advice including verbal and written case‐note feedback to ward‐based medical and nursing staff. DSN care began on randomisation and lasted until discharge. Control: standard care, defined as any management carried out by health care professionals (medical, general nursing, dietetic) other than the in‐patient DSN.
Outcomes	Length of stay Patient readmission Time to readmission Costs Quality of life Patient knowledge Patient satisfaction
Country/Setting	Wales, United Kingdom: University Hospital of Wales, Cardiff
Notes	Only length of stay, readmission and cost outcomes were relevant to this review. Funded by the Welsh Office for Research and Development for Health and Social Care. No interests disclosed.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	No sequence generation reported.
Allocation concealment (selection bias)	High risk	No concealment reported.
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not made explicit in paper
Blinding of outcome assessment (detection bias) All outcomes	High risk	Not done
Incomplete outcome data (attrition bias) All outcomes	Low risk	Data complete for primary outcomes.
Selective reporting (reporting bias)	Low risk	None apparent
Baseline characteristics similar for intervention group and control?	Low risk	It was reported that participant characteristics were similar in both groups but there were more participants with type 1 diabetes in the intervention group
Other bias	Low risk	None apparent

Einstadter 1996.

Methods	Non‐randomised trial
Participants	472 medical patients admitted to resident physicians of a particular firm at a tertiary referral centre in Ohio, over a 6‐month period. 243 were admitted to nurse case manager team and 229 to the control team.
Interventions	Intervention: a Master’s prepared NP and nurse case manager (also assigned part‐time to work in the medical clinic) was assigned to work with one team in the selected medical firm. Control: usual care
Outcomes	Appointment within 3 days One documented visit within 30 days Patient readmission within 30 days Patients attending the ED within 30 days of discharge
Country/Setting	USA: Metro‐Health Medical Centre, Cleveland, Ohio
Notes	Only readmission within 30 days and attendance at ED within 30 days were relevant to this study. No funding source reported. No interests declared.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Not done
Allocation concealment (selection bias)	High risk	Not done
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not done
Blinding of outcome assessment (detection bias) All outcomes	High risk	Not done
Incomplete outcome data (attrition bias) All outcomes	Low risk	All data available and no evidence of selective reporting.
Selective reporting (reporting bias)	Low risk	No evidence of selective reporting ‐ all outcomes listed were reported.
Baseline characteristics similar for intervention group and control?	Low risk	Baseline characteristics were reported and no significant differences apparent between groups.
Other bias	Low risk	Other than design limitations, no other biases were evident

Forbes 2006.

Methods	Controlled before‐after study
Participants	753 patients with MS (multiple sclerosis) attending 6 neurological services in 4 English regions. 616 participants (82%) completed follow‐up.
Interventions	Intervention: addition of MS Specialist Nurse to usual care for patients at 4 sites. Intervention not specifically described, but referred to 4 dimensions to role described by Forbes 2003 in Background section, as follows: psychological assessment and intervention, social assessment and intervention, physical assessment and intervention, co‐ordination and care management, specialist MS assessment and intervention, education and support, and research and audit. Control: 2 general neurology services sites acted as controls and did not have an MS Specialist Nurse.
Outcomes	Hospital admissions within 12 months Number of participants with pressure ulcers Experience and severity of MS‐related problems Health‐related quality of life
Country/Setting	UK: 6 neurological services in 4 English regions (1 in the South East, 1 in the South West, 2 in the North)
Notes	Only hospital admissions and pressure ulcers were relevant to this review. Funded by the MS Society of Great Britain. No interests declared.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Not done ‐ CBA
Allocation concealment (selection bias)	High risk	Not done ‐ CBA
Blinding of participants and personnel (performance bias) All outcomes	High risk	Blinding not possible and participants might well have been susceptible to lack of blinding.
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Outcome assessment managed by a researcher under supervision of another researcher. No blinding of outcome assessor or patient. However, questionnaire items were unlikely to be influenced by lack of blinding.
Incomplete outcome data (attrition bias) All outcomes	High risk	More non‐completers in the severe MS groups. Differences reported between sites and site data were not reported. Results section talked about significant 'group time effects observed for some of the SF‐36 items' but these data were not presented. There were poorer outcomes in the intervention sites compared to the control sites. Detailed data not provided for hospital admissions although the overall results were reported. Baseline data reported for SF‐36 (36‐item Short Form Survey) and MSIS_29 (Multiple Sclerosis Impact Scale) but no data reported for T1 or T2.
Selective reporting (reporting bias)	Low risk	No evidence of selective reporting.
Baseline characteristics similar for intervention group and control?	Low risk	Baseline characteristics were well reported. One potential issue dealt with ‐ conducted and reported in detail. Some differences identified ‐ control sites had a younger population with shorter duration of disease. These differences were factored into the analysis of outcomes.
Other bias	High risk	It was reported that baseline (pre‐intervention) differences were identified in hospital admissions between the intervention and control sites. It was reported that this should not prejudice the analysis as group‐time effects were used in the analysis, which were independent of the starting point. The report referred to differences between sites in the intervention and also several confounding factors ‐ e.g. an MS nurse was established already; contact with other professionals

Forster 2005.

Methods	Randomised trial
Participants	Patients admitted to 1 of 4 general medicine teams at the Ottawa Hospital (a public university teaching hospital) between January 21 and April 28 2002. 620 participants randomised, 361 discharged to community, 328 completed study, 290 completed satisfaction survey. Missing participants: 33 to completion, 71 to satisfaction survey
Interventions	Intervention: Addition of Clinical Nurse Specialist (CNS) to physician teams as a nurse team co‐ordinator. In addition to usual care, participants received care from a CNS added to 1 of 4 general medicine teams. CNS’s activities prioritised to: retrieving information collected by family physicians and consultants before admission; arranging in‐hospital imaging, procedures and consultations; facilitating patient education; and telephoning patients early after discharge from hospital (average 3 days) to answer questions and address early problems. Control: regular care
Outcomes	In‐hospital mortality Transfer home or transfer Time to discharge or patient transfer ED visit, participant readmission, or death Time to ED visit, readmission, or death Adverse events post‐discharge Patient satisfaction
Country/Setting	Canada: General and Civic campuses of the Ottawa Hospital, Ontario
Notes	Only in‐hospital mortality, ED visit, participant readmission, or death, and adverse events post‐discharge were relevant to this review. Funded by the Ottawa Internists Research Group. No interests declared.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Randomisation was stratified by study co‐ordinator in blocks of 4 with varying random order.
Allocation concealment (selection bias)	Low risk	Once baseline screening was conducted, nurse registered patients with study co‐ordinator who then randomised patients to study groups using sequentially numbered opaque envelopes.
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Participants were unaware of the group to which they were randomised. Not possible to blind care providers. Primary outcomes mortality, post‐discharge event, unlikely to be influenced by blinding.
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Data collected by researchers or physicians who were blinded to participant allocation.
Incomplete outcome data (attrition bias) All outcomes	Low risk	33 (9.1%) participants were lost to follow‐up with similar proportions per group (CNS 10.3%, control 8.1%; P = 0.46)
Selective reporting (reporting bias)	Unclear risk	Study protocol not available. Prespecified secondary in‐hospital outcomes included time‐to‐discharge or transfer. Prespecified secondary post‐discharge outcomes included various time‐to‐event outcomes.
Baseline characteristics similar for intervention group and control?	High risk	Significant baseline differences noted, but implication on analyses not discussed. Comparisons of differences in baseline status conducted, but results not reported.
Other bias	Low risk	None except for concern about sicker patients (not defined) in control group

Gardner 1991.

Methods	Controlled before‐after study
Participants	138 nurses working on medical units in a 526‐bed urban tertiary care teaching hospital. Medical patients on the study units for more than 2 days, understood English and fitted into one of 3 cardiac Diagnosis Related Groups (DRGs) (DRG 121, DRG 122, DRG 127), or were in the same room as patients with one of these DRGs.
Interventions	Intervention: primary nursing ‐ concepts operationalised using the Manthey 1980 definition of primary nursing ‐ used to train, educate and guide staff on a daily basis. Control: units using team nursing
Outcomes	Quality of nursing care Hospital stress rating Nurses' support Retention of RNs Mean DRG relative cost weights DRG cost by length‐of‐stay
Country/Setting	USA: Rochester General Hospital Rochester, New York
Notes	Retention of nurses was the only outcome relevant to this study. The study was supported by the Pew Charitable Trusts, ref: 86:0506HE. No interests declared.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Not done ‐ CBA
Allocation concealment (selection bias)	High risk	Not done ‐ CBA
Blinding of participants and personnel (performance bias) All outcomes	High risk	Blinding procedures not discussed ‐ probably not done.
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Blinding procedures not discussed ‐ probably not done.
Incomplete outcome data (attrition bias) All outcomes	Low risk	All outcomes were reported.
Selective reporting (reporting bias)	Unclear risk	Study protocol not available. Cost per patient day analysis separated old wing and new wing primary vs team nursing costs (excluded new wing primary group to find a statistically significant result) (P‐hacking?).
Baseline characteristics similar for intervention group and control?	Unclear risk	Limited information provided about the baseline characteristics. However, participants with similar DRGs were used across the units and it was reported that "in the pre intervention phase, all units had comparable staffing and patient mix and used a functional/team nursing model". Nursing stress scale, direct nursing care time baseline data not reported. Baseline mean and SD scores provided for qualpacs, Hospital Stress Rating Scale (HSRS) and Nursing Support Scale (NSS).
Other bias	Unclear risk	Change in study setting could have effects on outcomes. No report of sensitivity analyses having been conducted.

McPhail 1990.

Methods	Randomised cross‐over trial
Participants	21 nurses working on the unit: 10 nurses doing primary nursing, 11 nurses doing team nursing; 108 patients: 53 receiving primary nursing, 55 receiving team nursing; 16 clinicians
Interventions	Intervention: primary nursing Control: team nursing
Outcomes	Work environment scale; patient satisfaction, nurse absenteeism
Country/Setting	Canada: 35‐bed medical/surgical unit in a tertiary care teaching hospital
Notes	No source of funding reported. No conflict of interest reported.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Nurses were stratified for their days of the week and previous years' absenteeism and randomly assigned to Group A or B. Sequence generation not discussed.
Allocation concealment (selection bias)	High risk	Not discussed, probably not done.
Blinding of participants and personnel (performance bias) All outcomes	High risk	Blinding not possible and bias likely from lack of blinding.
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Not clear how nurse absenteeism data was obtained; patient data was self‐reported; nurses' work environment scale data was self‐reported.
Incomplete outcome data (attrition bias) All outcomes	High risk	9/20 nurses refused to complete questionnaire; only 40% of patients completed satisfaction questionnaire.
Selective reporting (reporting bias)	Low risk	All outcomes reported.
Baseline characteristics similar for intervention group and control?	Unclear risk	Not discussed, but cross‐over trial.
Other bias	High risk	Small sample; possible issues of contamination/cross‐over ‐ i.e. not clear what was the washout effect of crossing over, and whether there was any evaluation of the integrity of the primary nursing and team nursing models after crossing over.

Melchoir 1996.

Methods	Controlled before‐after study
Participants	492 nurses (psychiatric nurses, practical nurses, nurses' aides) providing direct care on 1 of 35 long‐stay psychiatric wards at 5 hospitals in the Netherlands that were randomly selected to participate. High attrition was reported over the 3 data collection times due to staff turnover: 366 (74.3%) participated at T1, 161 (32.7%) at T3
Interventions	Intervention: based on general principles of primary nursing: both psychiatric and practical nurses were assigned to participants as primary nurses based on the complexity of care needed. Nurse managers or quality care co‐ordinators provided the primary nurse with the feedback and support needed. They also gave advice on skills needed and promoted communication between the primary nurses and other healthcare providers. A special support meeting between primary nurses and other healthcare specialists was planned. Primary nurses followed a training programme that emphasised communication skills. The interventions were fully described in an intervention book. The process of implementing the intervention was supported by a group and was evaluated monthly. Control: the previous model of nurse staffing continued on the control units and was not described by the authors.
Outcomes	Nursing‐staff turnover Burnout
Country/Setting	The Netherlands: 5 psychiatric hospitals
Notes	Staff turnover was the only relevant outcome. No funding reported. No interests disclosed.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Not done ‐ CBA
Allocation concealment (selection bias)	High risk	Not done ‐ CBA
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not done ‐ not possible in this type of study
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Not done but not likely to impact on measurement of turnover.
Incomplete outcome data (attrition bias) All outcomes	High risk	Significant missing data ‐ only 161/361 nurses completed all times and therefore included (49.4%). No evidence of selective reporting.
Selective reporting (reporting bias)	Low risk	No evidence of selective outcome reporting.
Baseline characteristics similar for intervention group and control?	High risk	Data collected on baseline characteristics but not reported. Also no discussion of possible differences between groups. Overall biographical data gender, age, length in nursing and length on ward
Other bias	High risk	Contamination to control units reported "imitation" ‐ due to data leakage and rotation of student nurses; mean response rate in intervention group 83% (n = 60), 68% (n = 101) in control units; the source of the turnover data was not reported.

Neidlinger 1993.

Methods	Controlled before‐after study
Participants	6769 patients admitted to 1 of 4 units at a 560‐bed hospital in San Francisco between January and June 1990 (pre‐intervention) and January and June 1991 (post‐intervention).
Interventions	Interventions: incorporating Nursing Assistive Personnel (NAP) into nursing professional practice model. 2 intervention units: senior nurses and managers met to agree on the role of the NAP and to agree on the educational needs of staff and other resources required for the intervention. 3 NAPs were recruited to each unit and received a 2‐day didactic preparation and a 2‐week orientation programme. Each NAP assigned to work with 2 to 3 registered nurses, assisting in the care of 12 to 18 participants. Control: 2 units were selected on the basis of perceived similarities to the intervention units and continued with the pre‐existing nursing professional practice model.
Outcomes	Costs Care quality Patient satisfaction Staff satisfaction
Country/Setting	USA: 560 bed unionised university medical centre.
Notes	Only costs were relevant to this review. Supported in part by the Nursing Collaborative Clinical Research Initiative. No interests declared.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Not done ‐ CBA
Allocation concealment (selection bias)	High risk	Not done ‐ CBA
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not possible in this study design
Blinding of outcome assessment (detection bias) All outcomes	High risk	No discussion of blinding of outcome assessors, therefore assumed it did not take place.
Incomplete outcome data (attrition bias) All outcomes	High risk	Only mean values were reported, no information on SD or range. Data were rolled up for both intervention and control groups rather than reported separately. However, old study and not likely to make contact with authors. No evidence of selective reporting.
Selective reporting (reporting bias)	Low risk	All outcomes were reported.
Baseline characteristics similar for intervention group and control?	Unclear risk	Although it was reported that baseline data were collected, these do not appear to have related to the characteristics of the units and there was no evidence that the 4 units were similar.
Other bias	High risk	Control units quite different clinically from intervention units (borne out by cost data (table 1). Also, intervention appears to have changed during the study. If this is considered to be an CBA study then two time periods before and after are required ‐ but for cost data there was only one measure pre‐ and one measure post‐intervention.

O'Connor 1992.

Methods	Controlled before‐after study
Participants	647 nurses working on one of 21 units over study period
Interventions	Intervention: self‐staffing: in order to meet patient care demands, units would use only their own nursing staff. The central staffing office did not supply additional help, even if there were increased patient care demands, staff from other units could not be moved around to help. Therefore the unit took more responsibility for staffing and staff had input into policies and procedures concerning staffing on the units. Group A – self‐staffing introduced in Year 1 Group B – self‐staffing introduced in Year 2 Group C – self‐staffing introduced in Year 3 Control: 3 units that remained on the usual hospital staffing (Group D)
Outcomes	Nursing‐staff turnover rate
Country/Setting	USA: urban health centre in a Midwestern teaching tertiary centre of over 500 beds
Notes	PhD study, no funding reported, no interests disclosed.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Not done ‐ CBA
Allocation concealment (selection bias)	High risk	Not done ‐ CBA
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not done
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Not done but standard approach used to calculate staff turnover and verified across 2 sources. 100% agreement from both sources.
Incomplete outcome data (attrition bias) All outcomes	Low risk	No evidence of missing outcome data.
Selective reporting (reporting bias)	Low risk	None apparent
Baseline characteristics similar for intervention group and control?	High risk	Baseline characteristics not provided.
Other bias	Unclear risk	"Multiple probe design" (interventions introduced in units at different times) ‐ not clear what impact this might have had on results.

Plant 2015.

Methods	Randomised trial
Participants	500 patients with chronic illness presenting to the ED of Nepean Hospital, Sydney, New South Wales. High‐risk status for an unplanned admission was defined as: ≥ 3 unplanned hospital admissions in 12 months for patients aged 70, or at least 1 admission for cardiac or respiratory disease in patients aged 16–69 years; or judged by a CN nurse to be high risk and likely to benefit.
Interventions	Interventions: introduction of 3 nursing care navigation roles: Inbound: managing patients at presentation to ED, assessment, directing them to best method of care in hospital or community; Inflight: monitoring progress and minimising delays to discharge; Outbound: reviewing patient's hospital stay, making arrangements for out‐of‐hospital and on‐going care. Control: standard care
Outcomes	Representation at ED Patient readmissions Length of stay
Country/Setting	Australia: Nepean Hospital, Sydney, New South Wales
Notes	Funded by the National Health and Medical Research Council and NSW Health. Reported that Stephen Leeder was Editor‐in‐Chief of the Medical Journal of Australia when the manuscript was accepted for publication.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Quote: "The sequence of treatment allocation was determined by block design. A phone‐based randomisation service provided by the National Health and Medical Research Council Clinical Trials Centre was used to allocate treatment arms to participants after consent was given."
Allocation concealment (selection bias)	Low risk	Blinded
Blinding of participants and personnel (performance bias) All outcomes	High risk	Personnel were responsible for either delivering the service, or clinical staff referring to the service. They were not blinded. Participants were required to consent to participation and were probably aware of the research and the intervention.
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Quote: "Researchers who collected outcome data or performed statistical analyses were blinded to treatment allocation."
Incomplete outcome data (attrition bias) All outcomes	Low risk	Very similar numbers lost at allocation and lost to follow‐up at 12 months and at 24 months. All outcomes listed in the aims and methods were comprehensively reported in the results.
Selective reporting (reporting bias)	Low risk	All outcomes listed in the aims and methods were comprehensively reported in the results.
Baseline characteristics similar for intervention group and control?	Unclear risk	Although baseline characteristics were presented, no statistics conducted comparing intervention and control groups at baseline. May be a slight difference in sex ratio between the two groups (55% male in control group, vs 55% female in intervention group).
Other bias	Low risk	Slightly underpowered according to their power calculations (i.e. only about 300 participants completed EQ‐5D (EuroQol five‐dimension scale) outcomes) but otherwise not apparently subject to other sources of bias.

Pozen 1977.

Methods	Randomised trial
Participants	313 patients admitted to the critical care unit of Baltimore City Hospitals during a 16‐month period who had MI (myocardial infarction) (documented by history, serial enzymes and typical electrocardiogram changes) and were willing to participate in the study and follow‐up.
Interventions	Intervention: routine care plus access to a critical care unit‐based nurse rehabilitator. Objectives were to: optimise participants' long‐term work and rehabilitation through an aggressive programme of psychological support and education; to improve participants' knowledge and compliance to medical therapy by teaching them about MI, risk factors, basic physiology, rationale for therapy, and the appropriate convalescent programme; and reduce anxiety by assisting the participant to understand and cope with MI. Control: usual care provided by routine nurses and attending physicians
Outcomes	Anxiety Functional status Complications Knowledge Smoking and weight regimes Employment status at 6 months (previously employed)
Country/Setting	USA: Baltimore City Hospitals, Baltimore, Maryland
Notes	Employment status only outcome relevant to this study. Supported by funds from the Johns Hopkins Health Services Research and Development Grant #HS 000429, Robert Wood Johnson Clinical Scholars Program Grant #5 501 RRO 5556. No interests declared.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Reported that participants were assigned first to high‐ and low‐risk categories using specific criteria and then were randomly assigned in equal proportions to the study and control groups. There was no discussion of sequence generation, but this was an old study.
Allocation concealment (selection bias)	High risk	Not reported ‐ assumed not done
Blinding of participants and personnel (performance bias) All outcomes	High risk	Staff and participants not blinded.
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Not done, but questionnaires were self‐completion questionnaires.
Incomplete outcome data (attrition bias) All outcomes	Low risk	15% patients lost to follow‐up ‐ distributed across groups. No evidence of incomplete reporting.
Selective reporting (reporting bias)	Low risk	No evidence of selective reporting.
Baseline characteristics similar for intervention group and control?	Low risk	Baseline measures taken and it was reported that there were no differences between the groups and that the characteristics of the groups were typical of MI populations.
Other bias	Low risk	None evident

Ritz 2000.

Methods	Randomised trial
Participants	211 women ≥ 21 years of age diagnosed with breast cancer between 1995‐1997, able to read and write English and give informed consent. Also required physician referral, care within the system and consent within 2 weeks of diagnosis
Interventions	Intervention: standard medical care plus APN care APN contact within 2 weeks of diagnosis, written and verbal information about breast cancer, what to expect in consultations with physicians, decision‐making support, answering questions and presence for support. Subsequent contacts at scheduled clinic visits, by telephone, home visits or patient initiated visits. Contacts based on need as determined by patient, family and APNs. 1 of 2 APNs was on call 8 am to 8 pm Monday to Friday and 8 am to 12 noon on weekends. Control: standard medical care
Outcomes	Quality of life Costs
Country/Setting	USA: Integrated healthcare system in a large Midwestern metropolitan area
Notes	Only costs data relevant to this study. Supported by the US Army Research and Material Command Grant #DAMD17‐94‐J‐4449. No interests declared.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Reported that women were randomly assigned to 1 of 2 groups, but method of sequence generation not discussed.
Allocation concealment (selection bias)	High risk	Not discussed, assumed not done.
Blinding of participants and personnel (performance bias) All outcomes	High risk	Blinding not possible.
Blinding of outcome assessment (detection bias) All outcomes	High risk	Not done, but risk reduced as data were collected from hospital and clinic billing systems and through self‐completion questionnaires. APNs recorded time spent ‐ potential bias (not objective)
Incomplete outcome data (attrition bias) All outcomes	High risk	Data reported on all outcomes included in methods section. 2 issues noted: cost data did not include all provider fees (anaesthesiologists, ED physicians, radiation oncologists); missing cost data from 58 participants (28 in intervention and 30 in control) Focus in this paper was on cost and length of stay data. Graphs and charts used to report some data, rather than actual values (quality of life, Mishel Uncertainty in Illness Scale (MUIS), Profile of Mood States (POMS) so that it was not possible to extract these data ‐ but these were not considered to be primary outcomes in this review
Selective reporting (reporting bias)	Unclear risk	Focus in this paper was on cost and length of stay data. Graphs and charts used to report some data, rather than actual values (quality of life, MUIS, POMS) so that it was not possible to extract these data ‐ but these were not considered to be primary outcomes in this review.
Baseline characteristics similar for intervention group and control?	Low risk	Minor differences between the intervention and control groups ‐ women in intervention group significantly more likely to have lower histology and to receive adjuvant hormone therapy.
Other bias	Unclear risk	The following limitations were noted: sample primarily Caucasian (understood to be white participants), middle‐income women with high level of education; process improvements were implemented during the study that may have impacted on the outcomes.

Shukla 1983.

Methods	Non‐randomised trial
Participants	Patients admitted to 1 of 3 units (5 East = Primary Nursing; 3 East = Team Nursing; 5 West = Modular Nursing) during the study period.
Interventions	Interventions: Primary nursing: 100% RN; 1 nurse had direct and indirect responsibility for nursing care for a given number of hospital patients. Each nurse was assigned 4 to 6 patients for whom she had 24‐hour responsibility. RN delegated to associate nurses when off duty. Modular nursing: 50% RN, 50% LPN; a hybrid system under which 1 RN and 1 LPN provide cared for about 12 patients in one hospital area. The nurses were always assigned to the same modules to promote continuity of care. Control: Team nursing: 50% RN, 25% LPNs (licensed practical nurses), 25% aides; a group of RNs, licensed practical nurses, and nursing aides were led and directed by an RN, the team leader. Team usually consisted of 4 nursing staff, cared for 20 to 25 patients.
Outcomes	Quality of patient care Nurses' perception of quality Physicians' perception of quality Clinical care index Infection rate Costs: actual cost over study period per unit direct personnel costs total costs per patient day Nursing‐staff turnover
Country/Setting	US: Riverside Hospital, Virginia, major medical centre with 641 beds
Notes	Only objective outcomes were infection rate, costs and nursing‐staff turnover. Total number of participants or nurses was not reported. Partially supported by Grant 036501 from the National Center for Health Services Research, Medical College of Virginia, and Riverside Hospital, Newport News. No conflicts of interest reported.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Non randomised trial. Only 3 hospital wards included, 1 was experimental (3 East, Team Nursing), 1 was control (5 West, Primary Nursing), 1 was 'Modular' (hybrid) (5 East)
Allocation concealment (selection bias)	High risk	Allocation concealment was not possible with this design (no randomisation); furthermore, nurses chose to work in each of these settings (potentially biased towards the model of care).
Blinding of participants and personnel (performance bias) All outcomes	High risk	Blinding of personnel not possible. Possible that participants (patients) would not be aware of the intervention/control status, but they were a secondary consideration here.
Blinding of outcome assessment (detection bias) All outcomes	High risk	Nurses reported infection rates.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Restricted to 2 outcomes relevant to the review: infection rates and costs;.both apparently reported in full.
Selective reporting (reporting bias)	Low risk	All outcomes appeared to be reported.
Baseline characteristics similar for intervention group and control?	High risk	Authors report baseline measurement was not possible as the units were already established. However, they reported the units were very similar and measured control variables ‐ staffing, workload and average RN competency ‐ over the study period to monitor the implementation of each intervention. However these data were only collected for 3 months of the data collection period.
Other bias	High risk	This was a very low‐quality study, a non‐randomised trial. There were no 'pre' measures. There was only 1 ward in each arm of the study. Most of the outcomes were not objective. While substantial effort was made to 'control' for variables (e.g. staff competency), the risk of bias for the participant nurses (who worked on each unit by choice) was very great. Statistics were difficult to follow ‐ e.g. the mean infection rates were tested by paired t‐tests, but t‐tests are not considered to be appropriate when there are 3 groups. Turnover rates were mentioned in the text, but were not reported in tables (and were not mentioned in the Methods).

Sisk 2006.

Methods	Randomised trial
Participants	254 adults ≥ 18 years of age admitted to 4 hospitals in Harlem, New York Inclusion criteria: systolic dysfunction documented on a cardiac test (echocardiography, radionuclide ventriculography, myocardial stress sestamibi or thallium stress testing, or left‐heart catheterisation); English‐language or Spanish‐language speakers; community‐dwelling at enrolment; and current patient in a general medicine, geriatrics, or cardiology clinic or office at a participating site
Interventions	Intervention: nurse‐managed care: 1 of 3 trained registered nurses met once with each participant. Counselled the participant about the relationship among sodium intake; fluid build‐up; and symptoms, such as shortness of breath; mailed participants the reports from the food‐frequency questionnaire after each administration; served as a bridge between the participant and the clinician; contacted participants’ clinicians to discuss specific medications and arranged any prescription changes and examinations ordered. Control: usual care; participants received federal consumer guidelines for managing systolic dysfunction but no other intervention.
Outcomes	Death Hospitalisations: total hospitalisations ‐ all causes participants hospitalised ‐ all causes hospitalisations for heart failure ED visits: participants with any ED visit total ED visits Participant quality of life Medications in last 12 months
Country/Setting	US: 4 large, medium and small private and municipal hospitals in Harlem, New York
Notes	Only death, hospitalisations and ED visits were relevant for this review. Supported by the Agency for Healthcare Research and Quality, grant number: R01 HS 10402. No conflicts of interest reported.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Quote: "The project’s statistician used a computer‐generated, random‐number sequence without blocking or stratification to centrally determine randomization assignments"
Allocation concealment (selection bias)	Low risk	Quote: "The project's statistician ... concealed treatment group assignments in sealed, opaque envelopes"
Blinding of participants and personnel (performance bias) All outcomes	High risk	No discussion of blinding, but it appears that participants and personnel were not blinded and it would have been very difficult to do.
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Not done, but deaths, ED visits and hospitalisations taken from hospital and billing data.
Incomplete outcome data (attrition bias) All outcomes	Low risk	All outcomes fully reported,
Selective reporting (reporting bias)	Low risk	None apparent
Baseline characteristics similar for intervention group and control?	Unclear risk	Reported that the 127 participants in each group who were followed for 18 months were similar. Data were provided for both groups at 12 months and some differences appeared (e.g. living alone, pulmonary disease) but not clear if these were statistically significant.
Other bias	Low risk	None apparent

Talley 1990.

Methods	Randomised trial
Participants	107 patients (85 non‐suicidal and 22 suicidal) admitted to an adult medical, surgical, obstetrical or gynaecological unit in a large northeastern university hospital in the USA and assigned a sitter for at least 1 shift on 2 consecutive days between 4 January‐31 March 1988
Interventions	Intervention: consultation with a Psychiatric Liaison Nurse Specialist (PLNS) Seen by PLNS for the duration of the sitter order. Consultation initiated as soon as possible after the second sitter day by 1 of the hospital's 2 PNLSs. Consultation was based on modified version of PLNS consultation (Lewis 1982). Consultation was individualised to the particular participant situation and typically began with the reason for the sitter request, a review of the chart, and exploration of the staff nurse’s view of the participant's problem. The participant was then assessed and interventions were based on identified problems, with approaches targeted to nursing staff, participants and sitters. Participants received ongoing, direct PLNS interventions based on their potential for co‐operation and the nature of the problem that necessitated sitters. Control: no intervention
Outcomes	Length of stay Number of sitter shifts Number of charted observations of mood, behaviour and mental status Number of patient incident reports during the time with sitters Number of incidents of sitter refusal or walk‐offs
Country/Setting	USA: large northeastern University Hospital
Notes	Only length of stay was relevant to this review. Supported in part by funding from Sigma Theta Tau Delta Mu Chapter. No interests declared.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Participants were randomly assigned to either treatment or control group. No further details provided.
Allocation concealment (selection bias)	High risk	Not reported, assumed not done.
Blinding of participants and personnel (performance bias) All outcomes	High risk	Participants and staff could not be concealed from the presence of the PLNS for the treatment group.
Blinding of outcome assessment (detection bias) All outcomes	High risk	The research team collected the data.
Incomplete outcome data (attrition bias) All outcomes	Low risk	None evident
Selective reporting (reporting bias)	Low risk	None evident
Baseline characteristics similar for intervention group and control?	Low risk	Fully reported and were similar
Other bias	Low risk	None evident

Abbreviations

APN: Advanced Practice Nurse
 ED: emergency department/room
 LPN: Liaison Psychiatric Nurse
 RN: Registered Nurse
 SD: standard deviation

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Aiken 2008	CBA with less than 2 intervention and 2 control sites
Alvarez 2011	CBA with less than 2 intervention and 2 control sites
Armstrong 2004	ITS with insufficient data points pre‐ and post‐intervention
Arts 2000	ITS with insufficient data points pre‐ and post‐intervention
Bae 2014	CBA with less than 2 intervention and 2 control sites
Bender 2012	CBA with less than 2 intervention and 2 control sites
Biro 2000	Although previously included ‐ considered now to be a midwifery staffing ‐ not nurse staffing
Blegen 2011	CBA with less than 2 intervention and 2 control sites
Bowers 2012	CBA with less than 2 intervention and 2 control sites
Breckenridge Sproat 2012	CBA with less than 2 intervention and 2 control sites
Brett 1990	ITS with insufficient data points pre‐ and post‐intervention
Buresi 2014	CBA with less than 2 intervention and 2 control sites
Burnes Bolton 2007	ITS with insufficient data points pre‐ and post‐intervention
Carthon 2012	CBA with less than 2 intervention and 2 control sites
Cavan 2001	ITS with insufficient data points pre‐ and post‐intervention
Chaboyer 2007	CBA with less than 2 intervention and 2 control sites
Cook 2015	CBA with less than 2 intervention and 2 control sites
Courtenay 2007	CBA with less than 2 intervention and 2 control sites
Cox 1990	CBA with less than 2 intervention and 2 control sites
Davies 1994	CBA with less than 2 intervention and 2 control sites
Dawes 2007	Although included in original review, now considered to be an early discharge intervention rather than nurse staffing intervention
Donaldson 2005	ITS with insufficient data points pre‐ and post‐intervention
Duncan 2006	Although included in original review, now considered to be dietician staffing, not nurse staffing
Eck 1999	ITS with insufficient data points pre‐ and post‐intervention
Feddersen 1994	Although included in original review, now considered to be an educational intervention, facilitated by nurses
Forbes 2003	CBA with less than 2 intervention and 2 control sites
Grillo‐Peck 1995	ITS with insufficient data points pre‐ and post‐intervention
Hanneman 1993	CBA with less than 2 intervention and 2 control sites
Harr 2015	CBA with less than 2 intervention and 2 control sites
Hinshaw 1981	ITS with insufficient data points pre‐ and post‐intervention
Jansen 1994	ITS with insufficient data points pre‐ and post‐intervention
Lea 2003	ITS with insufficient data points pre‐ and post‐intervention
Lee 2005	ITS with insufficient data points pre‐ and post‐intervention
Lee 2011	CBA with less than 2 intervention and 2 control sites
Lengacher 1994	CBA with less than 2 intervention and 2 control sites
Lewis 1994	ITS with insufficient data points pre‐ and post‐intervention
Munnich 2014	CBA with less than 2 intervention and 2 control sites
O'Hare 2006	ITS with insufficient data points pre‐ and post‐intervention
Parasurum 2011	CBA with less than 2 intervention and 2 control sites
Pratt 1993	ITS with insufficient data points pre‐ and post‐intervention
Richardson 2009	ITS with insufficient data points pre‐ and post‐intervention
Rideout 2007	ITS with insufficient data points pre‐ and post‐intervention
Roche 2012	CBA with less than 2 intervention and 2 control sites
Ryan 2012	CBA with less than 2 intervention and 2 control sites
Sarkissan 1999	ITS with insufficient data points pre‐ and post‐intervention
Sheill 1993	ITS with insufficient data points pre‐ and post‐intervention
Sivendran 2014	CBA with less than 2 intervention and 2 control sites
Smith 2006	ITS with insufficient data points pre‐ and post‐intervention
Strayer 2008	ITS with insufficient data points pre‐ and post‐intervention
Thompson 2014	CBA with less than 2 intervention and 2 control sites
Tourangeau 1999	CBA with less than 2 intervention and 2 control sites
Twigg 2011	ITS with insufficient data points pre‐ and post‐intervention
Williams 2000	ITS with insufficient data points pre‐ and post‐intervention
Yong 2002	ITS with insufficient data points pre‐ and post‐intervention
Zidek 2003	ITS with insufficient data points pre‐ and post‐intervention

Abbreviations

CBA: controlled before‐after study
 ITS: interrupted‐time‐series study

Characteristics of studies awaiting assessment [ordered by study ID]

Benson 2008.

Methods	Unclear
Participants	Hospital staff and hospital patients
Interventions	Introduction of a rapid response team staffed by physician extenders (APNs)
Outcomes	Costs, in‐patient mortality, failure to rescue, staff satisfaction
Notes	Insufficient information to assess eligibility ‐ authors contacted December 2015 and unable to release further information prior to pending publication.

Campolo 1998.

Methods	Unclear
Participants	Hospital nurses
Interventions	Implementation of a 12‐hour shift for nurses
Outcomes	Staff retention, sick leave, work performance, inservice education
Notes	Insufficient information on study design to assess eligibility. Unable to contact author.

Counsell 1999.

Methods	Unclear
Participants	Hospital nurses and hospital patients
Interventions	Implementation of a Advanced Registered Nurse Practitioner Role in an acute care neurosurgical unit
Outcomes	Length of stay, costs, patient satisfaction
Notes	Insufficent information on study design to assess eligibility. Authors contacted 12 March 2009 ‐ no response.

Danello 2008.

Methods	Unclear
Participants	Hospital nurses
Interventions	Internet‐based open shift management system
Outcomes	Staff retention, staff satisfaction, costs
Notes	Insufficient information on study design to assess eligibility. Authors contacted December 2015

Davis 1997.

Methods	Unclear
Participants	Hospital nurses, patients and physicians
Interventions	Establishment of new nurse: patient ratios and work redesign in the ICU
Outcomes	Length of stay, complications, readmissions to the unit, clinical incidents, staff perception of quality of care, staff confidence in the health care delivery team, and staff use of problem solving, patient or family satisfaction, physician satisfaction, staff satisfaction
Notes	Insufficient information available about study design to assess eligibility. Unable to contact authors.

Eriksen 1992.

Methods	Unclear
Participants	Hospital nurses and physicians
Interventions	Introduction of the Licensed Vocational Nurse to the critical care unit
Outcomes	Nurse satisfaction, nurse turnover, illness absence
Notes	Insufficient information on study design to assess eligibility. Unable to contact authors.

Kenney 2001.

Methods	Unclear
Participants	Hospital nurses and patients
Interventions	Hiring Licensed Practical Nurses into available float pool positions in an acute care hospital
Outcomes	Medication and treatment errors, patient falls, patient satisfaction, staff satisfaction
Notes	Insufficient information on study design to assess eligibility. Unable to contact authors.

Ringerman 2000.

Methods	Unclear
Participants	Hospital nurses and patients
Interventions	Introduction of appropriately trained Licensed Vocational Nurses to critical care staffing
Outcomes	Patient falls, medication errors, nosocomial infection rates, decubiti incidents (pressure ulcers), mortality rates, costs, patient, physician and nurse satisfaction
Notes	Insufficient information on study design to assess eligibility. Unable to contact authors.

Characteristics of ongoing studies [ordered by study ID]

Drennan 2017.

Trial name or title	Programme of research into safe nurse staffing and skill‐mix
Methods	It is proposed that an interrupted‐time‐series analysis will be used to measure the impact of introducing nursing hours per patient day (NHPPD) as the approach to determining nurse‐staffing levels in medical and surgical settings.
Participants	Data will be collected at ward level from nursing staff, patients (both primary and secondary patient data) and organisational level data.
Interventions	The introduction of NHPPD as the approach to determining staffing levels in medical and surgical settings
Outcomes	Patient‐level outcomes (primary): patient experience of nursing care Patient‐level outcomes (secondary): central nervous system complications, wound infections, pulmonary failure, urinary tract infections, pressure ulcers, pneumonia, deep vein thromboses, upper gastro‐intestinal bleeds, sepsis, physiologic/metabolic derangement, shock/cardiac arrest, mortality, failure to rescue and length of stay Nurse level outcomes: care left undone; job satisfaction, intention to leave, burnout, nursing work Organisational‐level outcomes: agency use; sickness absence; supervisory time for clinical nurse manager; staff turnover; variance in NHPPD Economic outcomes: cost of staff uplift, economic impact on agency use; cost of staff turnover
Starting date	1 June 2017
Contact information	Professor Jonathan Drennan (email: Jonathan.Drennan@ucc.ie)
Notes

Driscoll 2017.

Trial name or title	A nurse practitioner program improves outcomes for patients diagnosed with heart failure
Methods	Non randomised trial
Participants	Patients with heart failure
Interventions	Care from an inpatient Heart Failure Nurse Practitioner (HF NP)
Outcomes	In‐hospital mortality, readmissions, quality of care
Starting date	Not clear
Contact information	andrea.driscoll@deakin.edu.au
Notes	Author contacted and reported that the final report had not yet been published

Differences between protocol and review

Minor changes were made to include economic outcomes and economic databases in the search strategy. There are four new authors of this updated review, and three previous authors (Rita Collins, Eileen Vilis and Al Mayhew) are no longer included.

Contributions of authors

All authors have contributed to this systematic review. MB led the writing of the protocol, all other authors provided comment and feedback. MB, PH, AS, TS, LK, JB and RK screened records for eligibility. TR and MB screened and extracted economic reviews. All other team members contributed to screening and extracting the remaining studies. MB, JD and TS conducted a second review of the risk of bias assessment. MB, JD and TR conducted the analysis and the interpretation of results. MB wrote the review with input from all authors. MB and TS edited the final draft.

Sources of support

Internal sources

• No sources of support supplied

External sources

• Health Research Board, Ireland.

  Provided support for the original review through HRB Cochrane Fellowship. All authors contributed to the update on a voluntary basis.

Declarations of interest

Jonathan Drennan has conducted research in the area of nursing skill mix but there is no conflict of interest in this review.

Jonathan Drennan and Michelle Butler are involved in a programme of research into safe nurse staffing and skill mix in Ireland. This study is on‐going and there is no conflict of interest with this review.

TS: none known

PH: none known

AS: none known

LK: none known

TR: none known

JB: none known

RK: none known

New search for studies and content updated (conclusions changed)