Evaluation of the effect of a Nursing System Framework on Nurse Sensitive Indicators, mortality and readmission in an NHS Trust

Abstract

In 2018, an NHS Trust (UK) implemented an innovative Nursing System Framework (NSF). The NSF formalized a two‐year strategy, which provided teams with clear aims and measurable objectives to deliver care. Failures of coordination of nursing services are well‐recognized threats to the quality, safety and sustainability of care provision.

Aim

To evaluate the efficacy of introducing a NSF in an NHS Trust, using nursing sensitive indicators and pre‐selected mortality, data outcome measures.

Design

A before and after implementation, observational study.

Methods

105,437 admissions were extracted at an admission record level. Data was extracted from 1st September 2018 through to the 31st August 2019.

Results

Using SQUIRE guidelines to report the study, insufficient evidence was found to reject a null hypothesis with a chi‐squared test of association between in‐hospital death and the NSF intervention period, with a p‐value of .091. However, trends were seen in the data, which suggested a positive association.

Conclusion

The NSF is a complex intervention, which provides direction for improvements but requires further research to understand the benefits for nurses, Midwives, Health Visitors and patients.

Key impact statements.

• The NSF can provide the infrastructure to enable nurses, Midwives and Health Visitors to drive improvements.

• The NSF plan can be used by leaders to gain support from their organization to highlight the integrated service elements (Pillars) into a focussed easy to follow plan based on the elements of a good Nursing, Midwifery and Health Visitor services. It also enables leaders to articulate and plan for areas needing investment, improvement and provides timeframes for accountable feedback.

• Future research is required into nursing service frameworks to understand if they can be flexible, generalizable and cost effective.

1. INTRODUCTION

1.1. Background and problem description

In September 2018, The Royal Wolverhampton NHS Trust introduced an innovative quality improvement and assurance framework. This was named the Nursing System Framework (NSF). The concept was initially designed and piloted successfully in Qatar (Cannaby et al., 2017). Through local engagement from Nursing, Midwifery and Health Visiting teams and further review of contemporary literature, the framework originally created in Qatar was developed for local context and adoption. The NSF formalized a two‐year strategy, giving teams, clearer aims and measurable objectives.

A high performing, compassionate, nursing service should include good oversight through governance, leadership, teamwork; and requires a series of overlapping and inter‐weaved components (Aiken et al., 2008; Gkantaras et al., 2016; West, 2021; Zwillinger & Huster, 2017). It should also include an excellent evidence base, an appropriate education provision, a skilled workforce and an effective leadership structure (Jackson et al., 2021). Nursing work (that of nurses, midwives and health visitors) is complex and multi‐faceted, with numerous unrecognized aspects that are difficult to specify and quantify; and are not only responsive to patients' needs, but also manage demands of the service and the sustainability of healthcare systems (Jackson et al., 2021; Musy et al., 2021). Failures of coordination of nursing services are well‐recognized threats to the quality, safety and the sustainability of good quality healthcare provision (Francis, 2013; Horton et al., 2021; Jabbal, 2017; Kobewka et al., 2016; West, 2021).

The association between the quality of nursing care provided and its direct impact on patient outcomes has been reported in the literature. Examples include: Alghamdi et al. (2021), Cannaby et al. (2017), Estabrooks et al. (2005), Jabbal, 2017, Kelly et al. (2014), Lawton et al., 2014, McHugh et al. (2013) Zwillinger and Huster (2017), West (2021).

1.2. Available knowledge and rationale

A framework, therefore, acts as a resource; not only to benchmark service quality standards but also to help to deliver them, in an organized manner. Nursing work can be understood as a series of complex interventions, which includes physical, emotional, cognitive and organizational labour (Jackson et al., 2021). Furthermore, West et al. (2020) suggest that there are other components needed to enable workforce efficiency while also necessitating well‐being and motivation too. These include autonomy, control, belonging, contribution and competence. A renewed approach in any context also needs to be proactive and credible to the workforce according to them.

The only recognized model for organizing nursing services internationally is Magnet®. Magnet® is an American centric, thematic model, which emerged from nursing workforce recruitment and retention issues in the 1980's (Sullivan & Aiken, 1999). It is held in high esteem as an exemplar Nursing accreditation model. Five key components feature in the Magnet® model, which are as follows: Transformational leadership, Structural empowerment, Exemplary professional practice; New knowledge, Innovations and improvements and Empirical outcomes (ANCC, 2020). Barriers to hospitals adopting the Magnet® model include the complexity of the accreditation process, difference between countries in regulatory frameworks for both educational and service organization and its cost, which to most organizations is an unsurmountable expenditure (RCN, 2015).

Cannaby et al. (2017) developed and evaluated an alternative nursing systems framework (NSF). The NSF originally including, 34 complex nursing interventions, organized around six core themes See: Table 1. These included: Professionalization, Education, Structure, Quality of Nursing Care, Part of an Academic Health System and Communication. Cannaby et al. (2017) conducted a before and after evaluation of the implementation of their NSF on defined measurable variables, which suggested a reduction in mortality as consequence of embedding the framework in the healthcare system in Qatar. Additionally, a positive association with the implementation of an NSF on patient outcomes was also observed.

TABLE 1.

Descriptors of Qatar and UK versions of the Nursing System Framework (Adapted from Cannaby et al., 2017)

Qatar Version (NSF)
Pillar	Intervention/theme
Professionalization	A career Framework introduced, which provided a professional infrastructure. This included scopes of practice, job descriptions, career structure, new roles, which included Clinical Nurse Specialists CNSs). A code of ethical practice and behaviour for nurses was develop and introduced
Education	Educational interventions and opportunities for nurses (Pre‐licensure and Diploma RN to BSN Transition) and master's Courses on offer to staff. Remodelling of accreditation of In‐service educational activities and continuing professional development initiatives. This was done via the American Nursing Credentialing Centre (ANCC). Initiation of a Graduate internship programme and speciality foundation programmes. Further opportunities for nurse Educators provided to undertake Post Graduate certificates in Learning and Teaching. Recruitment of graduate nurses only
Structure	Nursing structures and committees were reviewed through a system wide approach across the Corporation. Efforts made to improve two‐way communication flow and transparency. Increased visibility and contribution at corporate level from nurses seen. Engagement with Corporate Agenda followed. Shared Governance introduced. Senior management posts recruited to throughout the healthcare infrastructure. Nursing ratios and patient care hours reviewed, established and business cases initiated where needed in the Corporation
Part of an academic health system	Appointment of Academic Nurse Professors with an agenda to integrate and influence the research agenda across the Corporation. Research workshops, education and supervision provided for nurses interested learning more about research. Support provided for nurses to write for publication (as authors/co‐authors) of papers. Establishment of an ‘Evidence Collaborative’ set up to support systematic reviews and rapid appraisals of evidence
Communication	A comprehensive communication exercise undertaken to launch and promote the NSF strategy. This included the introduction of a periodic Newsletter ‘The Nurse Advocate’. The publication was written by Nurses for Nurses. A dedicated website dedicated to the nursing service was also used to provide up‐to‐date feedback. Regular feedback to the Executive team at the Corporation was maintained highlighting progress, areas of success and areas for improvement required

UK‐Royal Wolverhampton NHS Trust Version (NSF)
Pillar	Intervention/theme
Right staff in the right place at the right time	Implementation of a workforce plan, which focuses on recruitment and retention. Further development of workforce metrics and activities to ensure effective management of the nursing resource. Develop opportunities for professional development, throughout all grades and career time‐points. Flexible working and inclusion of proactive succession planning
Team structure	Definition and signposting of career pathways to guide and support career development through clinical specialities, management, education and research roles. Clear guidelines and leadership about delivery of high level quality care measured against international standards and best evidence‐based practice. Equip nurses at all levels to engage in quality improvement methodology and participate in clinical developments. Initiate and develop shared governance throughout the organization to engage staff and enable them to influence patient care at executive level.
The education faculty	Increase student numbers and invest in the future workforce. Invest in Advanced Clinical Practice and Specialist Nursing provision to improve the service we provide for patients. Strengthen induction programmes for new starters to the organization. Ensure ongoing educational and development opportunities for all staff. Develop new ways of working and flexibility to help the profession flourish and help to retain nurses in the profession
Excellence in practice	Provide the best care possible, which is underpinned by the best and latest evidence. Set realistic targets and create interventions to Proactively impact; key performance indicators in the organization. Set targets to reduce in‐patient falls and Pressure Ulcers, through continuous, active responsive learning and improvement. Seek external reviews of services; to drive continual improvement in the organization
Research & Technology	Continue to evidence our excellence through national/ international recognition via publications, research and achieving national peer recognition awards. Develop further a culture where research becomes a normative part of clinical practice. Support and develop local research, which underpins safe, effective and high‐quality care. Develop research locally, which impacts on health care nationally and internationally. Use new technologies to deliver nursing care, which enhances care delivery and is patient focused
Communications plan	Develop blended communication methods in the organization (internal and external) Implement a new nursing newsletter ‘Care To Share’ and new blogging capabilities to ensure wider engagement and communication. Hold monthly forums on topical and clinical issues in the organization

The Royal Wolverhampton NHS Trust, Nursing System Framework (NSF) revised dimensions were adapted into a developed system, (UK) model. These included 30 defined work‐stream objectives organized around 6 core pillars (themes), which were to be delivered over a 2‐year period.

The NSF booklets set out strategic local plans and delivery timelines, which were distributed throughout the organization. It was paramount that the content was communicated effectively. The three‐monthly milestone metrics were compiled into a formal reporting system. The report/s were discussed and shared with leaders and disseminated with teams. The information articulated progress, achievements and summarized plans if work streams had not gone to plan.

See: Figure 1 and Table 1.

FIGURE 1.

NSF Booklet front cover and NSF 6 pillars. ©The Royal Wolverhampton NHS Trust, reproduced with permission.

1.3. Aim

To evaluate the efficacy of introducing a Nursing System Framework (NSF) in an NHS Trust (UK), using pre‐selected mortality, data outcome measures and nurse sensitive indicators.

1.4. Design

A before and after evaluation, (observation) study was designed to enable a comparison of the impact of the NSF over a three‐year period. The initial intention was to evaluate the framework over a four‐year period; 2 years pre‐ and postimplementation of the NSF. However, due to the unprecedented COVID‐19, (SARS‐CoV‐2) global pandemic the research team decided to limit bias and impact from COVID−19 on the mortality data and from the variables analysed. A three‐year data set was extracted from 1st September 2016 through to 30 September 2019. Data were retrospectively extracted, pseudo—anonymised by informatics specialists at the Trust, who have access to staff and patient data as part of their roles on a day‐to‐day basis. Testing of the process was carried out to ascertain that it was not possible to link the data sets received to any personal identifiers before being released to the research team for the analysis.

Data were extracted from the Trust Patient Administration System (PAS) and downloaded onto a secure Trust IT approved encrypted removable device, so it could be analysed by the research team. The data stored on the device was encrypted and password protected and only accessible by the research team members (data statisticians).

The patient hospital number was de‐identified (pseudonymized) by applying a formula (three step process) that was only known to the research team statisticians. The patient's age was also anonymised (i.e. patients age [in years] and the date of birth were omitted which is potentially identifiable).

This paper uses SQUIRE guidelines (Version 2.0) to provide structure to report the study. SQUIRE guidelines are intended for reports that describe system level work to improve the quality, safety and value of health care and used methods to establish that observed outcomes were due to the intervention(s) made (SQUIRE, 2020).

1.5. Setting and context

The Royal Wolverhampton NHS Trust is an integrated (Acute and Community care) Trust. Its services are located over three sites and accommodates over 960 in‐patient beds. The Trust also hosts regional specialities, which includes the Heart and Lung centre (138 beds). It employs over 10,000 staff and is one of the largest employers in the locality (RWT, 2021). In addition, there are more than 20 community sites delivering services for adults, children, walk in centres, therapy and rehabilitation services. Community services are delivered by 9, GP practices presently in the Trusts vertical integration service model, offering extended opening hours to patients who require healthcare services (RWT, 2021).

The Nursing Midwifery and Health Visitor, staff complement was 3,370, Full Time Equivalents (FTE) across all divisions in the Trust (May 2019). This equated to 2,257 registrant practitioners, the remaining 1,123 (FTE) were Healthcare Assistants and Ward Clerical staff. The Trust does not routinely use external agency staff/nurses. It has a system of bank nurses (Registered Nurses and non‐registered care and administrative staff) employed internally (This equated to 1,151 recorded bank staff [May 2019]). The Trust presently sets its staffing levels using safer care methodology, which is part of the safer staffing strategy, guided by the Shelford Group work (2013, 2020).

2. INTERVENTIONS AND MEASURES

2.1. Inclusion and exclusion criteria

All in‐patient admissions were included. Admissions were identified as being all patients who remained in hospital at midnight on the day of admission (from birth). Exclusion criteria were applied to the study. These included:

• Admissions for delivery or pregnancy related conditions

• Psychiatric patients (N.B. referrals are routinely made externally for specialist care)

• Accident and Emergency (who are not admitted to a hospital ward).

Variables such as outpatient and day‐case patients were also excluded as their data were considered not to be part of the main hospital patient administration system data‐set or relevant. No withdrawal criteria were considered for the study.

For a data extraction summary see: Table 2.

TABLE 2.

Data variable and description of assigned analysis level

Variable created	Variable type	Raw data fields used	Levels/description
Age	Categorical, independent variable	Age	Under 65, 65 and over
Gender	Categorical, independent variable	Gender	Male, female
Ethnicity	Categorical, independent variable	Ethnicity	White, other than white*
Admission method	Categorical, independent variable	Admission method	Elective, emergency (variable not used for the readmissions analysis as this is emergency only)
Deprivation	Categorical, independent variable	LSOA of residence	Bottom quintile, not bottom quintile**
Length of stay	Numeric, continuous, independent variable	Admission date, discharge date and care spell	Days between admission date and discharge date
Intervention period	Categorical, independent variable	Discharge date	Pre‐NSF period, NSF period
Died	Binary, dependent variable	Discharge reason	1 = Died 0 = Survived
Readmitted	Binary, dependent variable	Admission and discharge date	1 = readmitted in 30 days of discharge 0 = not readmitted in 30 days of discharge
Nurse Sensitive Indicators (NSI'S)	Sample of audits carried out at Trust	Aggregated data sets	Descriptive analysis

^*For comparisons with the White group (As a whole) and ethnic minority groups, all other ethnic groups are combined (Asian, Black, Mixed, Other) and are referred to as “other than white” as per guidance (ONS ‐ Writing about Ethnicity 2021).

^**Bottom quintile refers to residing within the most (relatively) deprived 20% of neighbourhoods (The English Indices of MultipleDeprivation 2019).

2.2. Methods

The study is an observational study as the researchers could not assign causality for any observed difference in mortality or readmissions to the intervention—so we assessed any perceived intermediate outcomes (process evaluation) to demonstrate plausibility that any association of mortality/readmission with the before and after period could potentially be a result of the NSF introduction. Data was extracted from 1st September 2018 through to 31st August 2019 as per your request.

2.3. Data source and extraction

Data were extracted in October 2020. Data sources and data extraction, checking and recoding were undertaken by members of the Trust Information team in collaboration with the research team, whom are all employees or associates of the Royal Wolverhampton NHS Trust.

2.3.1. The unexposed (before NSF implementation) control period

The data set extracted from 1st September 2016 through to 31st August 2018 aligned with the unexposed period (time—period prior to the NSF introduction). A further 30‐days of data (up to 30th September 2018) was extracted to allow for readmissions for individuals admitted in August 2018.

2.3.2. The exposed (after NSF implementation) intervention period

The data set extracted from 1st September 2018 through to 31st August 2019 aligned with the exposed period (time—period after the introduction of the NSF). A further 30‐days of data (up to 30th September 2019) was extracted to allow for readmissions for individuals admitted in August 2019.

For admissions that spanned over both unexposed and exposed periods and could not be exclusively assigned to the pre‐ or post NSF cohorts were excluded from the analysis (100 admissions).

2.4. Data variables and analysis

2.4.1. Internal Process Audits (Fidelity Checks)

In addition to the specified data extraction, internal Nursing, Midwifery and Health Visitor process audits (Fidelity Checks) were identified.

Specific nurse sensitive indicator audits (including peer review audits—those carried out by colleagues from differing teams) were developed, initiated and facilitated by the ‘Nursing Quality Team’ in the Trust, using an external data platform for the aggregation of data. These align closely to the six identified work‐stream pillars of the NSF and key lines of enquiry (KLOE'S) advocated from the Care Quality Commission (Safe, Effective Caring, Responsive and Well‐led) (CQC, 2017). The introduction of the audits ensured that the essential elements of the framework were implemented with a high degree of fidelity. Progress made about the specific work‐stream milestones were tracked via an NSF reporting tool template, which were facilitated by the pillar leads. Essentially this assisted the Nursing teams to evidence progress made, document accomplishments, identify issues and communicate enablers and barriers and document any actions made. It is not the authors intention to report the findings from all the specific audits or milestone objectives set in this paper. It is, however, important to highlight that they are essential elements of the implementation of the NSF from an operational context.

2.4.2. Nurse Sensitive Indicators

A sample of Nursing Sensitive Indicators (NSI's) were identified and included in the study analysis. Many indicators are well‐established and used to gather metrics in teams. There are some metrics, which are submitted and compared to national data sets as routine practice also. The samples were included; to provide; oversight, quality control and to ascertain impact from the complex interventions planned and delivered in the NSF. There were no specific inclusion and exclusion criteria selected and were chosen to illustrate outcome data. The NSI's considered in the study, include falls (all categories of fall), which occur on healthcare premises, hospital acquired infection rates, hospital acquired pressure sores and friends and family feedback. Other examples of NSI data included in the study analysis were nursing workforce turnover rates used to establish staff retention rates.

Data were checked and cleaned as follows:

• Data in the database for any outlying or anomalous values, for example a negative patients age (−55 years old), or an extreme length of stay (e.g. length of stay 10,999 days). We also checked the data set for any impossible numbers, for example patients with an age of 150 for example.

• Once data have been checked and recoded this represented our final data set that was locked and used for the study analysis.

• The data set was analysed using the following techniques: standard summary descriptive techniques (number, proportions, means, standard deviation, interquartile ranges), key demographics (age, gender, deprivation, etc). This included a univariate analysis to determine the association between implementation of the NSF and mortality. Multivariate analysis, therefore, determined the independent variables that are associated with mortality, adjusting for co‐variates.

2.5. Analysis

A standard data cleaning process was followed and was processed using a secure Trust Microsoft SQL server. The data set was pseudonymized so that individuals could not be identified. As this was a before and after implementation of the NSF cohort, evaluation design, data relates to admissions rather than individuals. After exclusions were considered, in total, 105,437 admissions were extracted at an admission record level. Fields extracted for each admission, contained information about the admission itself, that is date, type, discharge reason etc. Other fields pertained to demographics of the individual admitted, that is age, gender, ethnicity and lower super output area (LSOA) of residence. Once extracted, further processing was completed to code and format data to obtain the required variables for analysis. See Table 3.

TABLE 3.

Data dictionary/variables/source

Data item		Data source		Rationale
Care‐spell ID		In‐patient clinical data source		Included as a unique spell identifier
Pseudo ID		In‐patient clinical data source		Included as a unique patient identifier, derived from Hospital Number and not identifiable
Admission method		In‐patient clinical data source		To determine whether an admission was planned or unplanned—required for readmission calculation
Admission date		In‐patient clinical data source		Required for various calculations, for example length of stay (LoS), readmissions etc.
Discharge date		In‐patient clinical data source		Required for various calculations, for example LoS, readmissions etc.
Discharge method		In‐patient clinical data source		Required to review outcome
Age on admission		In‐patient clinical data source		Demographic variable
Gender		In‐patient clinical data source		Demographic variable
Ethnicity		In‐patient clinical data source		Demographic variable
LSOA		In‐patient CDS—Derived from postcode using look up		Demographic variable
Clinical speciality on admission		In‐patient clinical data source		Clinical variable
Clinical speciality on discharge		In‐patient clinical data source		Clinical variable
SHMI diagnosis group		Derived using detail from last episode in spell^		Clinical variable
Primary diagnosis		In‐patient clinical data source—last episode		Clinical variable
Co‐morbidity diagnosis 1 up to 13		In‐patient clinical data source—last episode		Clinical variable
Charlson Co‐Morbidity score		In‐patient clinical data source—last episode		Clinical variable
Nursing service‐trust audits		Audits‐		Nurse sensitive indicators

Initial data analysis was performed in Microsoft Excel 2010, creating a series of pivot tables for descriptive statistics including median, for length of stay and percentages in each cohort for all independent and dependant variables.

This preliminary univariate analysis was used to help inform the required multivariate analysis to carry out.

2.6. Statistical methods

R version 3.5.2 statistical software and ‘table one’ and ‘stats’ packages were used for the statistical analysis. A summary of all variables stratified by intervention period was conducted. This was to highlight any differences between the pre‐ and post‐intervention cohorts and to establish if there was any skew for the variable of length of stay, with non‐parametric tests applied where appropriate. A chi‐squared test of association was also part of this analysis, this is a well‐established test that does not require any prior assumptions of the distribution of the data. This test explored if there was an association between the dependent variables and the pre‐NSF and postNSF intervention periods. The null hypotheses were as follows:

1. There is no association between in‐hospital deaths and the NSF intervention.

2. There is no association between emergency readmissions in 30 days following a prior emergency admission and the NSF intervention.

This initial statistical analysis was used to identify if any adjustments to the independent variables were required and to ascertain if there was any early indication of association before considering further multivariate analysis.

Multivariate analysis used was a generalized logistic regression model with the dependent variables of in‐hospital deaths and readmissions in 30 days, (separately) with all other variables as independent variables. This was used to identify which variables contributed the most to the outcomes and their significance, with stepwise (both forwards and backwards) logistic regression performed where relevant. Usual diagnostics for assessing model fit were carried out. Finally, odd ratios with 95% confident intervals were calculated to identify the relative measure of effect of the independent variables for interpreting risk of in‐hospital death and readmissions.

3. RESULTS

3.1. Nurse Sensitive Indicators

Most of the data sets reviewed during the study period, both pre‐ and postNSF were somewhat unremarkable, see Figure 2, with slight improvement shown in 2019 for Falls, also See Figure 3, hospital acquired infection rates see Figure 2, and Hospital acquired pressure sores also compared to the occurrence average rates see Figure 4.

FIGURE 2.

Nurse Sensitive Indicators sample.

FIGURE 3.

Falls data set (all categories) September 2016–2019 compared against the average number of falls per month over the whole data set.

FIGURE 4.

Hospital acquired pressure ulcers (HAPU) in the control and exposed study period compared against the average number of HAPU per month over the whole data set September 2016–September 2019.

The Trust patient satisfaction feedback, however, see Figure 5, which included the results in percentages from 35 specific questions posed to patients and their family members and friends during a hospital in‐patient stay, showed greater improvement than the other years illustrated. Comparisons were made with national aggregated benchmarking data between 2016–2019. While the overall 2018 scores had shown some deterioration in performance when compared to 2017, the 2019 results have shown improvement. The Nursing workforce turnover rates used and assessed to illustrate staff retention at the Trust shows a decline (this indicating improvement as turnover decreased as did nursing vacancy rates) from September 2018 in the exposed NSF intervention period as opposed to the previous 2 years in the unexposed period See Figure 6.

FIGURE 5.

Trust patient satisfaction feedback compared with national benchmarking comparisons 2016–2019.

FIGURE 6.

Nursing workforce turnover rates used to illustrate as an indicator for staff retention at Trust—September 2018 NSF introduced year 1 and year 2 anniversaries highlighted.

3.2. In‐hospital deaths

For the outcome measure of in‐hospital deaths (the dependant variable), 97,516 admissions in total were used, 63,657 in the pre‐NSF cohort and 33,859 in the postNSF intervention cohort. The mean age at admission for both cohorts was 58 years. The initial summary indicated that the length of stay variable was skewed, and therefore, for this variable, a non‐parametric test was applied. Remaining characteristics of the admissions and a comparison between both cohorts are listed in Table 4. p‐values <.05, indicated that the distribution for the pre‐NSF and NSF admission cohorts were statistically significantly different for age, gender and deprivation. To expand, the pre‐NSF period included admissions relating to fewer males, slightly younger and more deprived populations. This highlighted further multivariate analysis with adjustments was required. A chi‐squared test of association between in‐hospital death and the NSF intervention period, with a p value of 0.091 suggested that there was insufficient evidence to reject the null hypothesis. Therefore, there was insufficient evidence to suggest there was an association between in‐hospital deaths and the implementation of the NSF. See: Table 5.

TABLE 4.

Hospital data comparisons pre‐ and postimplementation of the NSF

Variables	Pre‐NSF	Post‐NSF	p‐value
Patients (n)	53,838	29,828	–
Age = 65 and over (%)	27,092 (50.3)	15,267 (51.2)	.017
Gender = male (%)	26,727 (49.6)	14,973 (50.2)	.126
Ethnicity = other than white (%)	9,855 (18.3) *	5,563 (18.7)	.221
Deprivation = not bottom quintile (%) **	31,194 (57.9)	17,468 (58.6)	.082
Length of stay (median [IQR])	3.00 [1.00, 8.00]	3.00 [1.00, 8.00]	.252
Readmissions = 1 (%)	6,659 (12.4)	3,552 (11.9)	.053

^* For comparisons with the White group (As a whole) and ethnic minority groups, all other ethnic groups are combined (Asian, Black, Mixed, Other) and are referred to as “other than white” as per guidance (ONS ‐ Writing about Ethnicity 2021).

^** Bottom quintile refers to residing within the most (relatively) deprived 20% of neighbourhoods (The English Indices of MultipleDeprivation 2019).

TABLE 5.

In‐hospital Data comparisons pre‐ and postimplementation of the NSF

Variables	Pre‐NSF	Post‐NSF	p‐value
Patients (n)	63,657	33,859	–
Age = 65 and over (%)	31,995 (50.3)	17,398 (51.4)	.001
Gender = male (%)	31,225(49.1)	16,841 (49.7)	.042
Ethnicity = other than white (%) *	11,116 (17.5)	6021 (17.8)	.214
Deprivation = not bottom quintile (%) **	38,238 (60.1)	20,573 (60.8)	.036
Admission method = emergency (%)	51,680 (81.2)	27,556 (81.4)	.453
Length of stay (median [IQR])	3.00 [1.00, 7.00]	3.00 [1.00, 7.00]	.134
Died = 1 (%)	3177 (5.0)	1,606 (4.7)	.091

^* For comparisons with the White group (As a whole) and ethnic minority groups, all other ethnic groups are combined (Asian, Black, Mixed, Other) and are referred to as “other than white” as per guidance (ONS ‐ Writing about Ethnicity 2021).

^** Bottom quintile refers to residing within the most (relatively) deprived 20% of neighbourhoods (The English Indices of MultipleDeprivation 2019).

An adjusted generalized logistic regression model was performed with key output elements comprising of coefficients, standard errors and p‐values. All outputs relating to this analysis omitted here (can be found in the Supporting Information Outputs 1–4 on request).

The p‐values derived from the test statistics indicated the significance of the relationship between each variable and the outcome of in‐hospital death (p‐value <.05 is statistically significant). Therefore, this model highlighted that all independent variables were associated to the outcome of in‐hospital death and were statistically significantly different to 0. Low standard error values ≤0.1 provided confidence in the co‐efficient estimates, indicating variability in the admissions populations was low. Admissions for those aged 65 and over, males and emergency admissions were associated with an increased probability of in‐hospital death. Similarly, an increase in length of stay days was also associated with in‐hospital death. Admissions for those who do not reside in the bottom deprivation quintile, those from an ethnic background ‘Other than White’ and admissions during the NSF intervention period were associated with a lower risk of in‐hospital deaths.

To explore the magnitude of these affects, adjusted odds ratios (OR) and 95% confidence intervals (CI) were calculated. An OR of 1 or a CI that crosses 1, suggests that the outcome of interest is equally likely to occur in all levels of a categorical independent variable, or for a continuous variable there is no association between an increase or decrease in the variable and outcome of interest.

ORs indicated that emergency admissions and admissions for those aged 65 and over, has the highest risk of in‐hospital death occurring (emergency admissions, OR 12.868, CI 10.4467–16.048 and aged 65 and over, OR 6.270, CI 5.755–6.842). Although on a much smaller scale, males and a longer length of stay also had an increased likelihood of in‐hospital death (male OR 1.157, CI 1.090–1.229, length of stay OR 1.011, CI 1.010–1.013). The ethnic group ‘Other than White’ showed a lower risk of in‐hospital death (OR 0.761, CI 0.689–0.838). This was also the case for admissions for people not in the poorest deprivation quintile and those exposed to the NSF intervention; however, the OR CI crossed (or was very close to) 1. Therefore, it is not possible to infer that there was a difference in the outcome of in‐hospital death for these variables.

Analysis of deviance used to assess model fit and variable contributions to the model was performed. Key output elements are deviance figures, where large values describe higher model contribution, and p‐values, which assess the model and if the variable statistically significantly contributes to the model when explaining variability (p‐values <.05 are statistically significant). Figures supported that the independent variables; age, admission method and length of stay, contributed most to the model when explaining the outcome of in‐hospital death. Furthermore, the other variables, whilst statistically significant enough to keep in the model, had a much smaller impact when explaining the outcome of in‐hospital death (confirmed by a stepwise regression model where all variables were left in). The pseudo‐R‐squared value of 0.12 suggested the model was a reasonable fit (values between 0.2–0.4 provide the best fit).

3.3. Readmissions

Outcomes for an emergency admission in 30 days of a prior emergency admission (the dependant variable), 83,666 admissions in total were analysed. There were 53,838 in the pre‐NSF cohort and 29,828 in the postNSF intervention cohort. The mean age at admission for both cohorts was 57. The length of stay variable was skewed; again, for this variable a non‐parametric test was applied. Remaining characteristics of the admissions and a comparison between both cohorts are listed in Table 4. p‐values <.05, indicated that the distribution for the pre‐NSF and postNSF emergency admission cohorts were statistically significantly different for age only, where the pre‐NSF period included admissions relating to a slightly younger population. A chi‐squared test of association between emergency readmissions following a prior emergency admission and the NSF intervention period, with a p‐value of .053 suggested that there was insufficient evidence to reject the null hypothesis. Therefore, there was insufficient evidence to suggest an association between emergency readmissions and the implementation of the NSF. See: Table 4.

The key components and outputs for the logistic regression, analysis of deviance table and OR are as before. All outputs relating to this analysis omitted here can be found in the Supporting Information (Outputs 5–9).

The adjusted logistic regression model highlighted that most of the independent variables except for ethnicity and the NSF intervention period, were statistically significantly associated to the outcome of an emergency readmission following a prior emergency admission. It is worth noting that the NSF intervention period independent variable was statistically significant at the <.1 level but not the <.05 level with a p‐value of .05027. Admissions for those aged 65 and over, males and an increase in length of stay were associated with an increased probability of a readmission after 30 days. Admissions for those who do not reside in the bottom deprivation quintile, those from an ethnic background ‘Other than White’, and admissions during the NSF intervention period were associated with a lower risk of readmission. Analysis of deviance supported that the independent variables of age contributed most to the model when explaining the outcome of emergency readmissions. It also highlighted that the variable ethnicity was not adding anything extra to the model and the other variables also had a very small impact. When assessing model fit the pseudo‐R‐squared value at 0.008 suggested the model was a poorer fit.

A stepwise logistic regression model confirmed that the independent variable of ethnicity should be removed from the model, all other variables were retained; admissions for those patients aged 65 and over, males and an increase in length of stay were associated with an increased probability of an emergency readmission after 30 days of a prior emergency admission. Admissions for those who do not reside in the bottom deprivation quintile and admissions exposed to the NSF intervention were associated with a lower risk of readmission. However, the magnitude of effect for each variable, whilst statistically significantly associated with the outcome, was relatively small (<1). Despite applying the stepwise approach and removing the ethnicity independent variable, the fit was still relatively poor (pseudo‐R‐squared <0.008).

The OR for admissions for those aged 65 and over, had the highest risk of an emergency readmission occurring following a prior emergency admission (OR 1.517, CI 1.452, 1.584). OR for all other variables were very close to 1 or had confidence intervals that were also very close to 1. This supports previous statements around small magnitudes of effect; this makes it difficult to infer whether there is a difference in risk of emergency readmissions for all other independent variables.

4. DISCUSSION

The aim of this study was to evaluate the efficacy of introducing a Nursing System Framework (NSF) in an NHS Trust, using pre‐selected nurse sensitive indicators and mortality, data outcome measures. The summary statistics suggest that those exposed to the NSF intervention period have less in‐hospital deaths and emergency readmissions; although this is not statistically significant and for both outcomes there is insufficient evidence to suggest there is an association between the outcomes and the NSF intervention period, independent variables. The independent variables that contribute the most to in‐hospital deaths were age, type of admissions and LOS (Length of Stay) and have the highest risk. For readmissions variables, it was age, with all other variables displaying little effect. However, the absence of an association does not mean that it is not there. The researchers acknowledge that the study would have benefited from having more admissions data in the NSF intervention period. A positive association is, therefore, a more appropriate outcome description.

The nurse sensitive indicators observed also suggest a positive association with the introduction of the NSF at the Trust. This may have been stronger if the researcher's original intention of carrying out a two‐year, postimplementation of the NSF analysis was carried out. This was unfortunately not possible, except for the patient and family satisfaction and Nursing turnover data.

This work coupled with the Cannaby et al. (2017) study does validate the evidence further in demonstrating that the adoption and implementation of a comprehensive NSF as a resource is a favourable one and worthy of merit. In addition providing the appropriate architecture for the delivery of effective services, care, leadership and appropriate governance it has been useful in articulating the organizations shared goals. Introduction of the NSF may have also contributed to a greater sense of autonomy, control, belonging and contribution and competence in the workforce to improve patient outcomes, as described and advocated by West et al. (2020). This according to them is what an effective workforce needs to work optimally.

From a delivery perspective for the NSF to be fully integrated at all levels of the service, it does need to be visible, talked about and meaningful at an operational level and an executive level. Buy in and engagement of the concept is, therefore, paramount. These are fundamental effective team requirements according to Aiken et al. (2008), Cannaby et al. (2017), Gkantaras et al. (2016), Zwillinger and Huster (2017) West (2021).

5. LIMITATIONS

It is also possible that patients included in the study may have been admitted to another hospital during the time analysed and patients can appear in both cohorts (this is an admission‐based analysis). It is acknowledged that the research team were not able to determine if this happened or whether an individual had multiple admissions. It was, however, anticipated that (patients admitted to another hospital) this was likely to be a comparatively unusual occurrence because the Trust tends to have a defined population (i.e. few patients go out of area). It is acknowledged that sample size calculation is not required for this study because the inclusion of all exposed and unexposed patients is included in the sample. The Covid‐19 pandemic meant that less data were collected and analysed, but also difficult to repeat or measure going forward with quantitative analysis. No comorbidities variables were included in the data, and we were unable to exclude end of life patients. The NSF exposed and non‐exposed period only relates to a small period. The NSF preparation could have impacted the non‐exposed admissions and some admissions might not have fully implemented the NSF for a period after it was launched? The R‐squared value and model fit was also not ideal, but models rarely see values as high as 0.2–0.4.

Improvements in patients' outcomes may have been due to many variables such as: new drugs/ devices or modes of intervention for example, pressure relieving mattresses or new personnel. These variables were acknowledged by the research team during development of the protocol, however, measuring their impact generally was seen as being problematic and beyond the scope of the study.

Due to the Covid‐19 pandemic a decision was made to extract the data only for one year after exposure to the intervention. In the previous study (Cannaby et al., 2017), in which the interventions indicated statistically significant patient benefits, data extraction took place over a 2‐year period after the plan was implemented in the organization. Future research, therefore, into nursing frameworks and their impacts on patients and the delivery workforce may need to consider the length of organizational exposure prior to data collection. The onset and impact of a global pandemic was an unprecedented occurrence.

6. CONCLUSION

The NSF is a repository of complex interventions, which involves stakeholder engagement and provides a focus for Nurses, Midwives and Health Visitors to drive improvements. Whilst our analysis does not show any causative factors in relation to patient outcomes there are positive associations seen in elements of the framework such as recruitment, where having a focussed plan enables the healthcare organization to demonstrate tangible improvements.

The NSF is constructed using the identified pillars of what a good nursing service is viewed as requiring to be effective. These include Right staff in the right place at the right time, Team structure, Education, Excellence in care, Research and Technology and Communication.

It is recognized that further research is required to enable a greater understanding of implementing a complex plan, which incorporates the elements of what is required in a good nursing service. It is the intention of the research team to test the NSF model further by carrying out further explorative work. The need for a practical, accessible, affordable and flexible nursing framework, which can help improve patient outcomes and help with the retention of the workforce is fundamental to healthcare services has never been stronger.

CONFLICT OF INTEREST

The authors declared that there is no conflict of interest.

7. ETHICAL APPROVAL

Advice from the United Kingdom Policy framework for Health and Social Care Research (HRA, 2021) and local Research and Development experts confirmed the study was a service evaluation. Formal Research Ethics Committee approvals were, therefore, not required for this work, but the principles of good clinical practice were adhered to. Formal local approvals were still required as per local governance policy prior to data extraction.

Supporting information

Appendix S1

Cannaby, A‐M , Carter, V. , Warren, K. , Gwinnett, J. , Bailey, K. , Mahmud, S. , & Gray, R. (2023). Evaluation of the effect of a Nursing System Framework on Nurse Sensitive Indicators, mortality and readmission in an NHS Trust. Nursing Open, 10, 953–966. 10.1002/nop2.1362

DATA AVAILABILITY STATEMENT

Data available in article supplementary material. Data available on request from the authors. Have submitted for reviewing peers to see.