Evaluation of the impact of an augmented model of The Productive Ward: Releasing Time to Care on staff and patient outcomes: a naturalistic stepped-wedge trial

Abstract

Background

Improving the quality and efficiency of healthcare is an international priority. A range of complex ward based quality initiatives have been developed over recent years, perhaps the most influential programme has been Productive Ward: Releasing Time to Care. The programme aims to improve work processes and team efficiency with the aim of ‘releasing time’, which would be used to increase time with patients ultimately improving patient care, although this does not form a specific part of the programme. This study aimed to address this and evaluate the impact using recent methodological advances in complex intervention evaluation design.

Method

The objective of this study was to assess the impact of an augmented version of The Productive Ward: Releasing Time to Care on staff and patient outcomes. The design was a naturalistic stepped-wedge trial. The setting included fifteen wards in two acute hospitals in a Scottish health board region. The intervention was the Productive Ward: Releasing Time to Care augmented with practice development transformational change methods that focused on staff caring behaviours, teamwork and patient feedback. The primary outcomes included nurses’ shared philosophy of care, nurse emotional exhaustion, and patient experience of nurse communication. Secondary outcomes covered additional key dimensions of staff and patient experience and outcomes and frequency of emergency admissions for same diagnosis within 6 months of discharge.

Results

We recruited 691 patients, 177 nurses and 14 senior charge nurses. We found statistically significant improvements in two of the study’s three primary outcomes: patients’ experiences of nurse communication (Effect size=0.15, 95% CI; 0.05 to 0.24), and nurses’ shared philosophy of care (Effect size =0.42, 95% CI; 0.14 to 0.70). There were also significant improvements in secondary outcomes: patients’ overall rating of ward quality; nurses’ positive affect; and items relating to nursing team climate. We found no change in frequency of emergency admissions within six months of discharge.

Conclusions

We found evidence that the augmented version of The Productive Ward: Releasing Time to Care Intervention was successful in improving a number of dimensions of nurse experience and ward culture, in addition to improved patient experience and evaluations of the quality of care received. Despite these positive summary findings across all wards, intervention implementation appeared to vary between wards. By addressing the contextual factors, which may influence these variations, and tailoring some elements of the intervention, it is likely that greater improvements could be achieved.

Trial registration number

UKCRN 14195.

Introduction

Definitions of quality have been refined over the past three decades.^{1 2} While mortality and clinical outcomes have remained important throughout, other issues such as patient safety, experience, accessibility, equity, efficiency and involvement in decision-making have grown in standing.³ Furthermore, there has been a growing empirical basis to support the interconnectedness of these issues.⁴ Wards are, after all, highly complex care environments.⁵

Addressing problems within such complex environments highlights several considerations: complex problems tend to require complex solutions; the more complex a solution is, the more likely its feasibility and effectiveness will vary across sites⁶; and evaluation of the impact of complex solutions is methodologically challenging. Simultaneously maintaining the internal rigour of a study and its applicability to the ‘real world’ is notoriously difficult.⁷ The ward is an ever-changing environment, embedded within an organisation which itself tends to shift as staff, policies and initiatives change. To assess the impact of a complex quality initiative against such a backdrop, and be confident that any improvement in outcomes identified can be attributed to the intervention, is difficult. Yet, such interventions are costly in terms of finance and staff time and energy, therefore requiring a strong evidence base to justify their implementation.

A range of complex ward-based quality initiatives have been developed over recent years. Perhaps the most influential programme has been ‘The Productive Ward: Releasing Time to Care’ (TPW:RTC).⁸ Designed in conjunction with nurses, TPW:RTC draws on ‘Lean methodology’. It focuses on streamlining ward processes, improving the ward environment and thus increasing time for face-to-face patient contact. The programme was introduced in England in 2008, and spread rapidly. By 2009, 80% of English Trusts had signed up⁹; it then spread to a number of other countries.^8–10

A substantial number of evaluations of TPW:RTC have been published that fall into three broad categories. First, those that describe the programme along with its anticipated benefits.^8–11 Second, those that describe its adoption, implementation and level of engagement in a particular context or locality.^12–16 And finally, those that describe reported outcomes from the programme.^17–22 The majority report benefits of the programme at both staff and patient levels, while others indicate that data support an increase in patient contact time.^22–24 However, the majority of studies lack rigour with very few robust evaluations.²⁵ Recent studies have applied a range of sophisticated methodological approaches.^26–28 However, robust evidence based on comparator wards is lacking.

In this study we draw on recent methodological advances in complex intervention evaluation design and apply these in a large-scale experimental evaluation of an augmented version of TPW:RTC (TPW:RTC-Plus) which included practice development change methods related to leadership, teamwork and caring behaviours. The aim of the study was to assess the impact of TPW:RTC-Plus on patients and nursing staff, specifically, three primary outcomes: experience of nurse communication, nurse shared philosophy of care and emotional exhaustion.

In line with the Medical Research Council’s (MRC) guidance on the design and evaluation of complex interventions,²⁹ the research team developed a Theory of Change (ToC) that outlined the mechanisms and means by which TPW:RTC-Plus was anticipated to improve patient experience of care.

While the 2011 Rapid Impact Assessment of TPW:RTC³⁰ estimated an increase of 42% direct patient care time, how these data were collected at board level was ambiguous and could not be clearly attributed to the intervention. However, the assessment did predict that there would be improvements in patient experience and a range of safety measures. At the time of the study, there were national programmes being introduced in the participating Board, which focused on improving patient safety. Our ToC (figure 1) therefore focused additionally on the augmented aspects of the intervention and improving patient experience, to contribute to the evidence base.

Figure 1.

The Productive Ward: Releasing Time to Care (TPW:RTC)-Plus Theory of Change.

Methods

The aim of the study was to assess the impact of TPW:RTC-Plus on staff and patient primary and secondary outcomes (tables 1 and 2). The trial was necessarily highly pragmatic, thereby ensuring increased likelihood that it could point to real benefits for future roll-out, if they existed.

Table 1.

Primary outcomes

	Before Mean (SD) n	After Mean (SD) n	Effect size Adjusted mean difference (95% CI)
HCAHPS nurse communication35	0.70 (0.41) 292	0.80 (0.34) 368	0.15 (0.05 to 0.24)
AHCOP shared philosophy of care37	3.84 (0.79) 184	3.86 (0.81) 163	0.42 (0.14 to 0.70)
MBI emotional exhaustion39	21.9 (11.0) 183	21.2 (13.8) 156	−2.33 (−5.92 to 1.26)

Results in bold indicate statistically significant improvements.

AHCOP, Acute Hospital Care for Older People; HCAHPS, Hospital Consumer Assessment of Healthcare Providers and Systems; MBI, Maslach Burnout Inventory.

Table 2.

Secondary patient outcomes

		Before	After	Effect estimate (95% CI)*
Time point 1: discharge or ward transfer
HCAHPS35
Nursing services†	n, mean (SD)	95, 0.63 (0.42)	122, 0.66 (0.37)	0.07 (−0.08 to 0.23)
Doctor communication	n, mean (SD)	302, 0.77 (0.38)	367, 0.77 (0.38)	0.09 (0.00 to 0.19)
Physical environment	n, mean (SD)	276, 0.49 (0.31)	347, 0.49 (0.31)	0.06 (−0.03 to 0.16)
Pain control*†	n, mean (SD)	232, 0.69 (0.40)	244, 0.69 (0.40)	0.13 (0.00 to 0.26)
Communication about medicines†	n, mean (SD)	191, 0.54 (0.40)	215, 0.54 (0.40)	0.07 (−0.06 to 0.21)
Discharge information	n, mean (SD)	233, 0.65 (0.48)	249, 0.65 (0.48)	0.09 (−0.06 to 0.24)
Fatigue and nausea†	n, mean (SD)	91, 0.79 (0.35)	88, 0.79 (0.35)	−0.12 (−0.31 to 0.06)
Overall health	n, mean (SD)	280, 0.45 (0.26)	345, 0.45 (0.26)	−0.02 (−0.09 to 0.05)
PANAS48
Positive Affect Scale	n, mean (SD)	244, 26.7 (8.2)	263, 26.7 (8.2)	−0.18 (−2.86 to 2.51)
Negative Affect Scale	n, mean (SD)	251, 16.9 (6.6)	279, 16.9 (6.6)	−2.01 (−3.84 to −0.18)
HADS43
Anxiety	n, mean (SD)	258, 5.8 (4.2)	314, 5.8 (4.2)	−0.63 (−1.77 to 0.51)
Depression	n, mean (SD)	259, 5.0 (3.9)	321, 5.0 (3.9)	−0.49 (−1.54 to 0.55)
EORTC42
Pain	n, mean (SD)	281, 46.6 (33.1)	323, 46.6 (33.1)	−7.6 (−17.5 to 2.3)
Fatigue	n, mean (SD)	282, 59.2 (26.4)	324, 59.2 (26.4)	−5.6 (−13.4 to 2.2)
Nausea/vomiting	n, mean (SD)	279, 15.9 (24.1)	321, 15.9 (24.1)	−1.4 (−6.0 to 8.7)
Ward rating
Confidence in future treatment	n, mean (SD)	292, 8.49 (1.9)	357, 8.9 (1.6)	0.32 (−0.12 to 0.80)
Overall ward rating	n, mean (SD)	293, 8.5 (1.6)	359, 8.9 (1.5)	0.46 (0.01 to 0.92)
HCAHPS recommendation to family/friends35
Definitely yes	n/N (%)	181/294 (61.6)	255/360 (70.8)	0.71 (0.40 to 1.26)
Probably yes	n/N (%)	91/294 (31.0)	87/360 (24.2)
Probably no	n/N (%)	16/294 (5.4)	10/360 (2.8)
Defiantly no	n/N (%)	6/294 (2.0)	8/360 (2.2)
Time point 2: 4–6 weeks after discharge/ward transfer
HCAHPS35
Overall health	n, mean (SD)	130, 0.43 (0.28)	149, 0.40 (0.24)	−0.05 (−0.14 to 0.05)
EORTC42
Pain	n, mean (SD)	128, 34.9 (30.0)	150, 37.1 (34.9)	−0.5 (−12.6 to 11.7)
Fatigue	n, mean (SD)	129, 45.6 (29.5)	147, 50.3 (29.7)	5.9 (−5.4 to 17.3)
Nausea/vomiting	n, mean (SD)	127, 8.3 (16.5)	149, 9.7 (20.0)	−0.7 (−8.7 to 7.2)
HCAHPS recommendation to family/friends35
Definitely yes	n/N (%)	76/131 (58.0)	96/150 (64.0)	0.79 (0.31 to 2.09)
Probably yes	n/N (%)	40/131 (30.5)	46/150 (30.5)
Probably no	n/N (%)	12/131 (9.2)	6/150 (9.2)
Definitely no	n/N (%)	3/131 (2.3)	2/150 (2.3)
Patient Enablement Instrument score44	n, mean (SD)	99, 4.05 (3.86)	107, 4.31 (3.75)	0.03 (−1.63 to 1.68)
Ward rating
Overall ward rating	n, mean (SD)	132, 8.1 (1.6)	153, 8.6 (1.5)	0.68 (0.49 to 1.31)
Confidence in future treatment	n, mean (SD)	131, 8.4 (1.9)	151, 8.9 (1.7)	0.72 (−0.02 to 1.47)

Results in bold indicate statistically significant improvements.

*Effect estimate is the adjusted mean difference, except for recommendation to family/friends where it is an odds ratio.

†There were screening questions for nursing services, pain control, communication about medicines and fatigue/nausea, so these scrores were only calculated for applicable patients

EORTC, European Organization for the Research and Treatment of Cancer; HADS, Hospital Anxiety and Depression Scale; HCAHPS, Hospital Consumer Assessment of Healthcare Providers and Systems; PANAS, Positive and Negative Affect Schedule.

Setting

Recruitment took place in two acute hospitals in a single Scottish Health Board region serving mixed urban/rural catchment area. The Board was chosen as it was taking part in a national roll-out of TPW:RTC but had not begun implementation.

Study design

Initial consideration was given to a cluster randomised trial. However, after discussion with the Board it was clear that resources were not available to deliver the intervention to all wards in a fixed time period and there was organisational desire to provide TPW:RTC-Plus to all wards. These issues have been cited as legitimate reasons for the adoption of a stepped-wedge trial.^{31 32} In a stepped-wedge trial, clusters are usually allocated randomly to prespecified time periods (steps).

The host Board believed that randomisation to steps could severely limit implementation success where factors such as organisational instability influenced wards locally (eg, absence of a clinical leader). Such wards were excluded or deferred until later in the study to optimise their chance of effectively engaging with the intervention. Data from the study were analysed in accordance with a stepped-wedge design. This element of non-randomisation leads to our descriptor of the study as a quasi (non-randomised) or ‘naturalistic’ stepped-wedge trial. For patients, single measurements were taken from individual participants, but involved different participants at each step of the study. At the patient level, the study is therefore a ‘cross-sectional’ quasi stepped-wedge trial.³³ However, we followed nursing staff throughout the study period with new staff permitted to enter the study at any phase of data collection. At the nursing staff level, the study is therefore an ‘open cohort’ quasi stepped-wedge trial. Recruitment and data collection occurred in six steps, each covering a 4-month period (online supplementary appendix 1). Questionnaires were collected between February 2013 and December 2014.

Sample

Obstetrics, children’s and mental health wards, and wards that did not have at least 16 weeks of organisational stability to allow intervention implementation were excluded. This left 15 wards for inclusion. All nursing staff members, at all grades, employed for the length of the step on participating wards were eligible for inclusion. Patients were eligible if they stayed on a participating ward for ≥24 hours during any of the six recruitment periods. Patients transferred to another ward remained eligible. Inpatients <17 years and those with an anticipated ward stay of <24 hours were excluded. We sought clinical opinion before approaching patients to ensure cognitive competence and readiness of patients to participate. Patients who could not understand English sufficiently to answer the questionnaires were excluded.

Sample size

The sample size calculation was based on before and after comparisons. The planned sample size of 800 patients across 15 wards for the primary patient outcome (Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) nurse communication) anticipated a 90% power to detect a difference of 0.025 (or 2.5%) based on an expected mean of 0.86, SD of 0.08 and intraclass correlation coefficient (ICC) of 0.05.^{34 35} For the primary nurse outcomes, the study was designed to have 80% power to detect standardised effect sizes of 0.22 SD based on a sample size of 394 nurses and ICC of 0.05.^36–39

Recruitment

Senior charge nurses (SCN) were informed of the research by the Board Practice Development Team and a member of the research team explained the research to ward staff and provided a copy of the research protocol. Staff members were ensured of data confidentiality and given at least three days to consider participation. There were an average of 31 nursing team members per ward and 15 wards available for the TPW:RTC-Plus interventions. The study aimed to recruit 60% of eligible staff for each ward.

All eligible patients were given an information leaflet about the study by ward staff. Following a minimum of 24 hours for patients to consider, and in agreement with nursing staff, the research assistant (RA) approached the patient, provided further information regarding the study and sought their consent to participate. After phase 1 (with ethical approval), an additional step was introduced following the initial leaflet. Patients were able to complete a ‘permission to contact’ form and were contacted after discharge to discuss potential participation. Patients were reassured about the confidentiality of the data they provided.

Control

Prior to receiving the interventions (control steps), wards were operating under ‘normal care’ conditions and no other initiatives were introduced or delivered during the study period.

Intervention

TPW:RTC⁸ shares some principles with Transforming Care at the Bedside in the USA.⁴⁰ It uses methodologies from the business world, including Lean production,⁴¹ to improve work process and team efficiency with the aim of ‘releasing time’ which would be used to increase caring time spent directly with patients. The team worked through a series of modules facilitated by a member of the Practice Development Team.

While TPW:RTC had the ultimate aim of improving patient care, this aspect is not addressed. There is instead an assumption that nurses will use time gained to increase direct care time. To address this, the Practice Development Team developed the ‘Plus’ aspect to empower the team to systematically evaluate care standards and experiences and use that in collaborative ways to improve care and care cultures by incorporating into the intervention three measures of quality (staff caring behaviours, patient experience and teamwork). The results from these measures were shared with nursing teams in facilitated sessions as a focus for their improvement efforts. A full description of TPW:RTC-Plus is provided in online supplementary appendix 2.

We therefore hypothesised that TPW:RTC-Plus, with its focus on Lean and a team approach to designing changes based on patient-reported care experiences and peer observations, would result in an increased awareness of the gap between desired care standards and actual care delivered. Crucial to this approach was the facilitated support that was provided to the team to encourage a shared and mutual philosophy of care, improve better team working practices and spend more time with patients.

Blinding

The nature of the intervention meant that ward staff could not be blinded to what they were receiving. However, given that allocation to the intervention was not at the patient level and that receipt of the intervention was based on time step, it was possible to blind patients.

Outcome measures

Potential outcome measures were mapped against our ToC model (figure 1). In keeping with the MRC guidance on complex interventions,²⁹ several primary outcomes were selected, these covered patient and nurse-level outcomes. The three primary outcome measures chosen had documented validity and reliability. First, the HCAHPS inventory: nurse communication subscale,³⁵ completed by patients. Second, the Acute Hospital Care for Older People (AHCOP) toolkit; a measure of ‘shared philosophy of care’, completed by nursing staff.³⁷ Third, the Maslach Burnout Inventory (MBI) Emotional Exhaustion (EE) component scale for nursing staff.³⁹ The full range of secondary outcome measures is found in online supplementary appendix 3.

The distal patient primary outcome, nurse communication, derived from HCAHPS, was developed in the USA as a patient-reported questionnaire to benchmark patient experience of hospital care quality. It was important that the measure of care reflected behaviours experienced at ward level, therefore the measure was adapted. The nurse communication subscale consisted of four summed questions relating to: courtesy and respect; nurses listening carefully; nurses explaining things in a way which the patient could understand; and responses to call buttons. Possible responses were: Never, Sometimes, Usually or Always, scored 0–3 respectively. Other subscales were used as secondary outcomes (your care from your nurses; your care from your doctors; the ward environment; your experiences in this ward; overall rating of the ward; general health; care for pain; care for fatigue and nausea symptoms (adapted)) (online supplementary appendix 3).

Beyond the ToC, the study also examined whether the intervention might reduce symptoms. Secondary outcomes included patients’ experience of pain, fatigue and nausea symptoms using the European Organization for the Research and Treatment of Cancer Quality of Life Questionnaire (EORTC QLQ-C30) measures.⁴² They were also asked about anxiety and depression using the Hospital and Anxiety Depression Scale.⁴³ The research team was also interested whether the intervention had the potential to improve postdischarge self-care measured by the Patient Enablement Instrument⁴⁴ and follow-up symptom questions. The potential of the intervention to impact on clinical outcomes was assessed via an examination of hospital records to identify frequency of emergency admissions for same diagnosis with 6 months of discharge.

A staff primary outcome, shared philosophy of care, was derived from the AHCOP, a collection of patient relative/carer, SCN and nursing team questionnaires. The nursing team scale consists of a number of subscales. The subscale ‘shared philosophy of care’ was seen as a potential proxy measure for the stage 2 mechanisms of collective vision, purpose, value and identity in our ToC model. The subscale consists of five summed questions relating to: sharing an explicit philosophy of care; valuing psychological aspects of care; communicating the philosophy of care to new staff; involving patients and carers; and valuing relational rather than procedural work. Possible responses, on a 5-point Likert scale, ranged from strongly disagree to strongly agree, scored 1–5 respectively. Other subscales were used as secondary outcomes: having resources; supporting each other; feeling safe; improving practice; having a say; developing our skills; too much to do; multidisciplinary team working; leading by example; and support from the top (online supplementary appendix 3).

The third primary outcome was based on the premise that increased empowerment and efficiency could help reduce the stress experienced by nursing staff. To measure this, the MBI (EE subscale) (Human Services Survey Form) was used (22 summed questions).³⁹ This was rated on a 7-point frequency scale from never to every day.

Other secondary outcomes included questions at the SCN level. These included two subscales from the AHCOP: meeting patients’ needs; looking to improve. Scoring was similar to the nurse questions. Nurses and SCNs were also asked an adapted version of Swanson’s Caring Professional Scale.⁴⁵ This includes two subscales of attributes of caring behaviours: compassionate healer, competent practitioner. They were also asked an adapted global care question, as a correlate of the net promoter question asked of patients,³⁵ ‘If a relative or friend were to become a patient in this ward, how confident would you be that they would get the care they needed and deserved?’ (scored 0–100).

Data collection

Questionnaire data were collected during six pre-specified fortnightly periods, beginning with one at baseline (pre-intervention), when all wards were in the control arm of the study, then during four subsequent fortnights for each phase of the quasi stepped wedge and during a final fortnight in the follow-up post-intervention phase. We employed three RAs to ensure ward staff were not involved in the recruitment of patients or collecting data from patients. Questionnaires were precoded with a unique participant ID (hospital ward/identifier/check code).

Patients were asked to complete a paper Ward questionnaire at/after discharge/transfer (or at the end of the data collection phase, whichever came first). Patients discharged directly home were asked to complete a follow-up, Home questionnaire 4–6 weeks later. At both stages, RAs offered to provide assistance to patients to mitigate potential literacy or burden issues. The Home questionnaire could be completed by post, telephone or online.

Consenting SCNs completed a questionnaire for each participating ward. This was repeated at each time phase (six) of the study. All consenting nursing team staff members were asked to complete a questionnaire at each time phase (six). Contributors were entered into a prize draw with one winner per phase of collection winning a £50 voucher, and a £5 voucher for each member of their team who returned a questionnaire.

Data management

Data were digitised (optical character recognition), pseudonymised and stored by the Health Informatics Centre (HIC) at University of Dundee. The Community Health Index number was used to pseudonymise study data sets, with patient consent, to link hospital records. Identifiable data were retained on National Health Service (NHS) networks by HIC. Data released by HIC to the research team for analysis were anonymised and held securely on Glasgow Caledonian University computers.

Analysis

Analysis was conducted on an ‘intention to treat’ basis, operationalised within a stepped-wedge design as analysed according to the allocated cross-over time, irrespective of whether that was achieved.^{31 33} Analysis of the primary outcomes estimated the effect of the interventions from the mean differences (and 95% CIs) before and after the intervention using a generalised linear multilevel mixed model, by regressing the outcome variable on an indicator variable (ie, before or after intervention received) adjusting for baseline covariates and clustering at ward and hospital levels. Models were adjusted for age, gender, deprivation, religion, ethnicity, education, employment and household status (for patient outcomes), and age, gender, years of experience, contracted weekly hours and grade (for all nurse outcomes) with the addition of qualification for SCN outcomes. Recruitment phases were fitted as fixed effects. Significance was set at the 5% level. All secondary outcomes were analysed in a manner similar to the analysis of the primary outcome, but using generalised linear multilevel mixed models appropriate for the type of data including ordered logit regression for ordinal outcomes. Statistical analysis was conducted using Stata version 13 (StataCorp LP, TX, USA).⁴⁶

Missing data

Imputation of questionnaire responses was undertaken at item level according to the rules of the specific instrument. Where the instrument had no specific requirements relating to missing data, item-level data which were missing were replaced using person mean substitution (otherwise known as ipsative mean imputation or pro-rating) provided that no more than half of the items in the scale were missing for the individual respondent. Otherwise, the scale value remained missing.

Results

Overview

We successfully recruited 691 patients, 177 nurses and 14 SCNs. Consolidated Standards of Reporting Trials diagrams for both patients and nurses are shown in figure 2. Our sample was almost exclusively white, broadly reflecting the demographics of the catchment area. Comparison with national figures indicates that this sample was representative in terms of age and gender. They were slightly more advantaged relative to the national figure.⁴⁷ See online supplementary appendix 4 for patient demographics.

Figure 2.

Patient and nurse recruitment.

Primary outcomes

Statistically significant improvements were seen for patient experience measured by the HCAHPS subscale for nurse communication (mean difference=0.15, 95% CI 0.05 to 0.24).³⁵ In addition, the nurse reported AHCOP shared philosophy of care demonstrated a statistically significant improvement (mean difference=0.42, 95% CI 0.14 to 0.70).³⁷ No statistically significant improvement was seen for the MBI EE scores reported by nurses³⁹ (table 1).

Secondary outcomes: patients

Patients were more likely to rate the ward more highly following TPW:RTC-Plus intervention, both at discharge from the ward (Ward questionnaire mean difference; 0.46 (95% CI 0.01 to 0.92)) and at follow-up at home (Home questionnaire effect estimate; 0.68 (95% CI 0.49 to 1.31)).

Patient improvements in reports of HCAHPS doctor communication (effect estimate; 0.09 (95% CI 0.00 to 0.19)) and HCAHPS pain control (mean difference; 0.13 (95% CI 0.00 to 0.26)) following TPW:RTC-Plus approached statistical significance.³⁵ No other effects were found in the remainder of the HCAHPS measures, or in EORTC, Positive and Negative Affect Schedule (PANAS) or Hospital Anxiety and Depression Scale measures^{42 43 48} (table 2).

A simple bivariate analysis examining the effect of the intervention on the incidence of emergency admission for the same diagnosis within 6 months of discharge found no significant difference (Pearson χ²=1.68; p=0.194). After multivariate analysis (random effects model) accounting for sex, age, Scottish index of multiple deprivation (SIMD), qualifications, employment and household status, there was still no significant intervention effect (adjusted OR 1.18; 95% CI 0.65 to 2.13).

Secondary outcomes: nurses and SCNs

A series of statistically significant improvements in nursing team outcomes was apparent. AHCOP 'supporting each other' (mean difference=0.45, 95% CI 0.15 to 0.74), AHCOP ‘feeling safe’ (mean difference=0.43, 95% CI 0.05 to 0.81), AHCOP ‘improving practice’ (mean difference=0.37, 95% CI 0.05 to 0.69) and AHCOP ‘leading by example’ (mean difference=0.33, 95% CI 0.01 to 0.66) all showed positive increases following the intervention, indicating an improvement in nursing team climate.³⁷ TPW:RTC-Plus was also associated with a positive effect on Positive Affect (mean difference=2.83, 95% CI 0.66 to 5.00).⁴⁸ TPW:RTC-Plus had significant, though modest effects on nursing team culture and nursing resilience.³⁷

There were no significant changes in SCN reports relating to AHCOP ‘meeting the needs of patients’, AHCOP ‘looking to improve’³⁷ (table 3). SCN confidence that patients and families receive appropriate treatment was high at outset and improved marginally, but not significantly. The CIs for the effect sizes were wide, suggesting heterogeneity in intervention effect at the ward level. There was no effect of TPW:RTC-Plus on either subscale from the Caring Professional Scale.⁴⁵

Table 3.

Secondary nursing outcomes

Nursing staff outcomes		Before	After	Mean difference (95% CI)
MBI39
Depersonalisation	n, mean (SD)	183, 8.4 (4.5)	154, 9.4 (5.5)	−0.62 (−2.54 to 1.31)
Personal achievement	n, mean (SD)	178, 33.0 (7.0)	155, 32.1 (7.5)	0.82 (−1.65 to 3.29)
AHCOP37
Nursing team climate
Having resources	n, mean (SD)	187, 2.9 (1.0)	165, 3.0 (1.0)	0.24 (−0.10 to 0.59)
Supporting each other	n, mean (SD)	188, 3.3 (0.8)	164, 3.4 (0.9)	0.45 (0.15 to 0.74)
Feeling safe	n, mean (SD)	188, 3.1 (1.0)	165, 3.1 (1.0)	0.43 (0.05 to 0.81)
Improving practice	n, mean (SD)	183, 3.3 (0.9)	162, 3.3 (0.9)	0.37 (0.05 to 0.69)
Having a say	n, mean (SD)	186, 3.2 (0.9)	165, 3.0 (1.0)	0.09 (−0.25 to 0.44)
Developing our skills	n, mean (SD)	185, 2.8 (1.2)	162, 2.9 (1.1)	0.09 (−0.35 to 0.52)
Too much to do	n, mean (SD)	185, 3.3 (1.0)	162, 3.3 (1.1)	0.01 (−0.43 to 0.45)
Multidisciplinary team working	n, mean (SD)	184, 3.9 (0.9)	164, 4.0 (0.9)	0.13 (−0.17 to 0.43)
Leading by example	n, mean (SD)	185, 3.6 (1.0)	161, 3.6 (1.1)	0.33 (0.01 to 0.66)
Support from the top	n, mean (SD)	183, 2.9 (0.8)	160, 3.0 (0.8)	0.19 (−0.11 to 0.49)
Quality of person-centred care	n, mean (SD)	184, 4.2 (0.7)	164, 4.1 (0.8)	0.22 (−0.06 to 0.51)
PANAS48
Positive Affect Scale	n, mean (SD)	163, 37.5 (7.1)	149, 37.9 (7.5)	2.83 (0.66 to 5.00)
Negative Affect Scale	n, mean (SD)	168, 15.8 (5.3)	155, 17.1 (8.0)	0.48 (−1.55 to 2.50)
Confidence in necessary treatment	n, mean (SD)	184, 80.5 (17.9)	153, 77.8 (21.9)	1.31 (−4.92 to 7.54)
Caring Professional Scale45
Compassionate healer	n, mean (SD)	180, 32.0 (5.0)	157, 32.1 (5.5)	0.21 (−1.78 to 2.19)
Competent practitioner	n, mean (SD)	184, 28.2 (4.1)	159, 28.2 (4.7)	0.64 (−0.93 to 2.21)
Senior charge nurse outcomes
AHCOP37
Meeting needs of patients	n, mean (SD)	21, 3.9 (0.8)	14, 4.0 (0.6)	−0.10 (−0.51 to 0.31)
Looking to improve	n, mean (SD)	21, 3.5 (0.5)	14, 3.5 (0.4)	0.33 (−0.09 to 0.75)
Confidence in necessary treatment	n, mean (SD)	20, 91.7 (9.8)	13, 92.9 (7.1)	4.33 (−2.31 to 10.98)
Caring Professional Scale45
Compassionate healer	n, mean (SD)	28, 33.7 (4.0)	29, 34.1 (3.7)	0.94 (−1.05 to 2.93)
Competent practitioner	n, mean (SD)	28, 29.0 (2.8)	29, 29.4 (3.2)	(−0.68 to 2.53)

Results in bold indicate statistically significant improvements.

AHCOP, Acute Hospital Care for Older People; MBI, Maslach Burnout Inventory; PANAS, Positive and Negative Affect Schedule.

Discussion

TPW:RTC-Plus was successful in improving dimensions of patient and nurse experience and evaluations of the quality of care received. We found statistically significant improvements in two out of three primary outcomes: patients’ experiences of nurse communication and nurses’ shared philosophy of care. A lack of evidence of improvement in nurse-reported emotional exhaustion should be further explored, but is consistent with a lack of change in reported Negative Affect (PANAS); the ToC should be revisited. There were also significant improvements in secondary outcomes: patients’ overall rating of ward quality; nurses’ positive affect and team climate.

Strengths and limitations

Addressing criticism of other studies we employed an experimental design, with comparator wards located within the same health board, enabling wider hospital factors that may have contributed to changes to be accounted for. Validated tools were used to test the underlying theory embedded within the intervention. We included a key intended outcome, patient experience.⁴⁰ Patients were blind to the intervention being delivered and provided outcome data at both discharge and follow-up.

We acknowledge the relatively moderate rate of patient recruitment and our inability to blind staff to the intervention, which may produce a Hawthorne effect.⁴⁹ Additionally, our inability to randomise wards to particular steps means secular trends may have contributed to an effect.⁵⁰ However, we can confidently attribute outcomes to the intervention as a parallel stepped-wedge trial (reported elsewhere) for a different intervention, in the same locality, found no significant change.

It can be said that if time is released for direct care, the explicit outcome for TPW:RTC, it would be methodologically challenging to attribute how any time released was spent. For example, time released (at our point of measurement) may have partially been allocated to further improvement.

Comparison with previous literature

Other studies have suggested that TPW:RTC increased direct care time with patients,^22–24; however, we believe ours to be the first to show an impact on patient experience itself within a robust design. Furthermore, other elements were approaching statistical significance (eg, doctor communication, physical environment, communication about medicines and discharge). Significant complaints and determinants of patient dissatisfaction relate to poor communication and interpersonal skills,^{51 52} suggesting communication is central to improving patient ratings of quality of care. Our findings indicate that TPW:RTC-Plus has potential to reduce the growing complaint figures within the NHS.⁵³ Evaluating complaints as part of this study was not possible due to lack of ward-level attribution. The question as to whether the TPW:RTC can improve wider elements of patient experience or clinical outcomes remains unclear.

Our finding of a change in primary outcome: shared philosophy of care, and secondary outcomes: supporting each other; feeling safe; improving practice and staff positive aspect may overlap White and Waldron’s⁵⁴ review of 96 papers examining TPW: RTC. These suggestd potential impacts in nine areas: empowerment, leadership, engagement, stress and resilience, teamwork, morale, role enhancement, sociocultural impact and staff satisfaction.⁵⁴ This may go some way to explaining findings elsewhere that TPW:RTC has the capacity to improve ‘work engagement’,^{37 55} defined as ‘a positive fulfilling work related state of mind characterised by vigour, dedication and absorption’.⁵⁶

Implications for practice and future research

Providing robust evidence for the impact of complex quality initiatives being delivered in complex, ever-changing settings is not straightforward. Our evaluation focused on an augmented version of TPW:RTC and we cannot disentangle the impact of the additional element (the Plus) or how the time released is used to benefit patients. The intervention was nonetheless designed on evidence supporting practice development methodology, which empowers staff transformation care quality. We would suggest incorporating it within wider roll-outs.

Future research could focus on four key areas: improvements to TPW:RTC-Plus, effective implementation, sustainability and understanding which caring behaviours are impacting on patient experience. While TPW:RTC-Plus appears to produce a number of welcome impacts, the mechanism(s) by which these are achieved are not clear. Finally, the ‘Theseus Paradox’ applies to this context. A single implementation of TPW:RTC-Plus on a ward is unlikely to produce sustainable change. There is a question as to whether an object that has had all of its components replaced over time retains its original characteristics or essentially becomes a new object; this may apply to healthcare interventions that shift over time.⁵⁷ In this case, wards with frequent staff turnover may essentially become a different ward over time. Future work could develop indicators to suggest further ‘top-ups’ where appropriate.