Mining routinely collected acute data to reveal non-linear relationships between nurse staffing levels and outcomes

Abstract

Objectives

Nursing is a safety critical activity but not easily quantified. This makes the building of predictive staffing models a challenge. The aim of this study was to determine if relationships between registered and non-registered nurse staffing levels and clinical outcomes could be discovered through the mining of routinely collected clinical data. The secondary aim was to examine the feasibility and develop the use of ‘big data’ techniques commonly used in industry for this area of healthcare and examine future uses.

Setting

The data were obtained from 1 large acute National Health Service hospital trust in England. Routinely collected physiological, signs and symptom data from a clinical database were extracted, imported and mined alongside a bespoke staffing and outcomes database using Mathmatica V.10. The physiological data consisted of 120 million patient entries over 6 years, the bespoke database consisted of 9 years of daily data on staffing levels and safety factors such as falls.

Primary and secondary outcomes

To discover patterns in these data or non-linear relationships that would contribute to modelling. To examine feasibility of this technique in this field.

Results

After mining, 40 correlations (p<0.00005) emerged between safety factors, physiological data (such as the presence or absence of nausea) and staffing factors. Several inter-related factors demonstrated step changes where registered nurse availability appeared to relate to physiological parameters or outcomes such as falls and the management of symptoms. Data extraction proved challenging as some commercial databases were not built for extraction of the massive data sets they contain.

Conclusions

The relationship between staffing and outcomes appears to exist. It appears to be non-linear but calculable and a data-driven model appears possible. These findings could be used to build an initial mathematical model for acute staffing which could be further tested.

Strengths and limitations of this study.

• This study has revealed patterns in acute hospital and staffing data.

• This study has described probable relationships between staffing and outcomes which could be further explored and used to model staffing in the acute sector.

• Data mining methodology is feasible if the data sets are available.

• This study is strengthened by the use of patient-level data.

• Limitations of this study include a limited data set and the data originate from one NHS Trust in England.

Introduction

Providers of nursing care such as registered nurses (RN) and unregistered healthcare support workers (HCSW) provide the majority of inpatient contact time in the UK and this workforce represents the largest proportion of workforce and subsequently expenditure.¹ Whole systems approaches to workforce modelling require more knowledge about nursing as the profession supplies a large proportion of care.

Nursing work is complex.^2–5 The assumption that nursing is a serial linear activity rather than a set of inter-related activities has led to nursing work being subjected to activity analysis—for example, time and motion studies, but such methods do not capture complex work well.^{6 7} Current staffing models make the assumption that nursing work is time filled with tasks. Nursing work contains other components that are essential in safety critical professions. Failure to match staffing with patient needs is associated with an increase in mortality.⁸

Safety is currently defined more by its absence than its presence,⁹ and registered nursing practices that ensure patient safety primarily due to vigilance¹⁰ and rescue from deterioration^{11 12} are seldom seen. It is only when this safety practice is absent that issues are recognised.¹³ RNs in the acute inpatient setting are often responsible for the day-to-day management of care using clinical judgement,¹⁴ including the application of vigilance^{10 15 16} in order to ensure patient safety. It is suggested when one or more of this combination of elements is missing then harm can happen¹⁷ and older adults are at a higher risk of harm in these situations.¹⁸

There are many reported variations in standards of care—in terms of patient experience and adverse events which also incur cost.^{19 20} Feeling safe as well as being safe is a contributory factor for patient experience and contributes to the sense of comfort patients receive from nurses.¹⁷

Many attempts have been made in the past to measure nursing activity and establish staffing levels such as those given in the recent National Institute for Care Excellence (NICE) guidance.²¹ Most of these methods rely on types of linear measurement or aggregating average scores. This has been successful to a point but does not offer much sensitivity. Recent work by Lord Carter,^19,20 for example, has suggested an aggregate average to reflect care needs which is unlikely to reflect complexity.

There have been studies which look at the relationship between RN numbers and outcomes^{22 23} and the non-linear relationships that exist between staffing and length of stay.²⁴ There is also an increasing amount of data collected about the care, experiences and outcomes of patients. These data are increasingly collected but rarely mined within the context of nursing. However, data mining has been used in nursing on previous occasions, for example, to explore variables associated with poor glycaemic control,²⁵ to reveal patterns in end-of-life nursing care,²⁶ and to examine the complexity of the work of specialist nurses in the UK.^{4 5 27 28} It is anticipated that such approaches would develop nursing knowledge through the identification of patterns and important links between data, nursing interventions and patient outcomes.²⁹

The advent of technology and in particular developments in the field of computational mathematics has allowed society to understand the patterns that lie within complex systems in a dynamic ‘real world’ state. By understanding these patterns, it is possible to reveal complex, non-linear relationships and even predict behaviour. This approach is now common in areas such as retail, banking and aviation.³⁰

There is assumed to be a relationship between safety (using a proxy measure such as falls, pressure ulcers (PUs) or other clinical incidents) and registered nurse presence, but this relationship is not fully understood. While it is clear that there is no simple relationship between either numbers or skill mix of nursing staff and either outcomes and cost,³¹ there have been numerous statistical studies which demonstrate correlations between registered nurse numbers and adverse outcomes,^{23 32} including failure to rescue from deterioration, suggesting a relationship with safety.

The complex activity of nurses is hard to capture or measure in a direct way. One approach to this problem is to use data mining and pattern recognition of existing data sets to build a representative model that reflects the activity in all its dimensions and then to use this knowledge to improve mathematical models that reflect the real-world situation.³⁰ By mining data and modelling the relationship that is assumed to exist, it may be possible to build more accurate data capture tools which provide increased insight into the relationship between complex nursing care and patient safety factors (and subsequent cost-effectiveness), which might allow predictive modelling in future. A challenge for previous researchers in examining the contribution of nursing activity has been the lack of specific nurse activity data. Nursing work, despite being the largest proportion of contact time, is rarely collected as routine. It is likely very high volumes of data collected in healthcare in the UK by some National Health Service (NHS) Trusts may yield new patterns—revealing new knowledge.

The aim of this study was to mine a number of large routinely collected data sets in one NHS acute trust to reveal any apparent relationships between registered and unregistered nurse staffing and outcome variables such as safety factors, to undertake a feasibility study in terms of accessing routine data and to build the first iteration of a model for nurse staffing in acute care.

Methods

Setting

The setting was an acute NHS hospital in the Midlands. The Trust has 1189 beds across 33 wards in 2 hospitals and a range of care settings such as acute medical and surgical as well as specialist wards. Routine data collection had taken place over 9 years and consisted of a number of unintegrated, warehoused data sets, four of which were examined for this study (∼200 million data points were extracted). This study took 10 weeks. A systematic approach to obtaining, mining and analysis of the data was taken. Data were extracted and anonymised internally within the Trust from three commercial applications (VitalPAC used for recording observations and other physiological parameters, Datix an incident reporting system and Allocate a staffing and e-roster application) and one bespoke in house application.

Like most NHS Trusts, several systems were in place for the collection of data with little integration. The amount of data held by each of these systems is large and each package uses a different architecture, thus downloading data from various commercial repositories proved challenging. Fortunately, a number of the manufacturers offered and provided assistance, in particular Datix and Allocate. Initial data files were extracted from the VitaPAC platform and supplied in common .csv formats. Extraction proved very challenging VitalPAC was not able to assist apart from communication via their call centre and so a local work around solution was deployed by the Trust IT specialist to extract the data. The staffing and key performance indicator (KPI) database was designed within the Trust in Excel and easily extracted. All data sets described were included in an initial review for volume and integrity. After initial review, the Allocate data set was excluded as it contained only 1 year of data and the Datix data set needed further work to be anonymised, thus was not used in this run due to time constraints. The Datix data set has been cleaned for future use.

Analysis

The Trusts own bespoke data set of inhouse design—consisting of staffing levels for RN and HCSW different types of leave, rates of slips, trips and falls (STFs) and occurrences of PUs. These are shown in table 1.

Table 1.

The items in the trust bespoke data set that were used

Data field	Units used (per ward)
Total N&M establishment	WTE
Funded registered establishment	WTE
Funded registered establishment-in post	WTE
Funded registered establishment-variance	WTE
Funded HCSW establishment	WTE
Funded HCSW-in post	WTE
Funded HCSW variance	WTE
Registered sickness	WTE
Unregistered sickness	WTE
Overall sickness	% of total hours
Study leave registered	hours
Study leave unregistered	hours
Maternity leave registered	hours
Maternity leave unregistered	hours
Slips, trips and falls	binary
Pressure ulcers	discrete

Data were imported into Mathematica V.10 and mined alongside the VitalPAC data set. The VitalPAC data set consisted of the physiological data such as observations. The observations were split into three key types based on the numerical form of the measurements (table 2). These were: continuous—for continuous observations recorded over a range of values, for example, heart rate; binary—for observations recorded as true/false states, for example, does the patient have an irregular pulse and discrete—for observations with a series of levels—for example, pain scores. For each of these observation families, a selection of metrics have been defined as single-point representations of the ward in that month. In all, over 120×10⁶ items were mined.

Table 2.

The VitalPac items used

Data field	Data type
Heart rate	Continuous
Temperature	Continuous
SATS (oxygen saturation)	Continuous
BP systolic	Continuous
BP diastolic	Continuous
Respiratory rate	Continuous
Oxygen concentration	Continuous
Vomit	Binary
Nausea	Binary
Pulse regularity	Binary
AVPU (level of consciousness)	Discrete
Bowels open	Discrete
Oxygen flow rate	Discrete

The analysis was carried out using the Wolfram Language—operating within the Mathematica 10.1 platform. Import code was constructed. All code structures were run on a single, local machine system with no attempts at cluster parallelisation. Despite this, the running time for each step of the analysis has been achieved in time intervals in the window of 10 s to 10 min.

Prior to full analysis of the correlation behaviours—an initial pass through the data sets was carried out to identify suitable entities within the VitalPAC data for inclusion in the investigation using a sample of data (6 months). Wards were excluded if they were not clinical areas within the hospital, for example, entities created by information technology such as test areas, and wards created or renamed during the time period of interest. Each of these exclusions reduces the sample size, and possibly introduces a systematic biasing of the data set. However, these rejected wards are evenly and randomly distributed throughout the medical functions of the observed system—curtailing any biasing.

With the individual observations aggregated on the ward, and month level, the data sets can be reduced to numerical representations of the ward. For this study, a series of such metrics have been selected depending on the nature of the variables (continuous, binary or discrete).

The metrics calculated were: continuous data sets: number of metrics evaluated per observation: 3 with metric types: mean (μ) SD (σ). Degree of normality—that is, % of data outside the window (μ±3 σ). Significant derivation from 0.5% to 1% suggests a non-normal distribution about μ. For the binary data sets, the number of metrics evaluated per observation was 1 with metric types as a %Occurrence. For discrete data sets, the number of metrics per observation: variable for observation: ‘AVPU’ with metric types as %Occurrence, observation ‘Bowels Open’ with metric types: mean, SD, %Occurrence. Observation: ‘Oxygen Flowrate Ext’ with metric types: %Negative %Null %Positive and for such physiological parameters.

Each of these metrics have been selected to be sensitive to different aspects of the data set—for example, variation in the mean represents a change in average location, and variation in degree of normality reflects the presence of outliers within the data set. These metrics are not exhaustive—but have been chosen as they are numerically simple, to calculate and interpret.

Metric pairs were generated between the VitalPAC and staffing data sets using the ward ID as a lookup key. Initially, paired metrics were checked for correlation using the Pearson, Spearman, Kendall and Hoeffding tests.

The data were broken into bivariate samples, comprising the 34 metrics drawn from the VitalPAC database, 26 of the staffing metrics and the 2 safety metrics (STFs and PUs). This resulted in a total of a parameter space consisting of ∼1000 variable pairings (34×26+34×2+26×2). Checking for a single significant pairing at the 5% level (ie, p<0.05) over 1000 tests implies a reduction in the significance level to p<0.00005.

A selection of these findings were visualised and are presented in the Results section.

Research governance

This was granted alongside the approval of the Research and Development department of the Trust for secondary data analysis.

Results

After extracting, importing and mining the data, a number of correlations started to appear between and within the data sets. Forty correlations were found at this pass (p<0.00005). Five KPI metrics demonstrated significance with staffing (p<0.00001). These are shown in table 3 and were: Total Nursing and Midwifery Establishment, Funded RN-in post, Funded HCSW establishment, Funded HCSW-in post and unregistered sickness-total hours. They are primarily the correlations between staffing and the incidence of STF. STFs are recorded in the NHS in England as a safety indicator and so correlations between the staffing levels and incidences of STFs have been explored.

Table 3.

Key performance indicator metrics that highly correlated with falls

Observation	Rho	p Value
Total nurses (WTE)	0.49	3.6×10−12
Funded HCSW (WTE)	0.65	1.7×10−24
HCSW-in post (WTE)	0.67	9.9×10−26
HCSW sickness (hours)	0.44	5.8×10−10
HCSW sickness (Spells)	0.45	3.7×10−10

The data revealed many possible relationships between staffing and STF. In addition to relationships between STF and staffing, the wellness of the patients was determined from the vital signs data. Examples of these correlations are shown here. Figure 1 shows how wards with a higher ratio of RN to HCSW appear to have less STF on average and figure 2 shows that wards with higher HCSW establishment (whole time equivalent, WTE) appear to have a higher rate of STF.

Figure 1.

Relationship between local-averaged RN% of total nursing and recorded slips, trips and falls (STF). Wards where the ratio of registered nurses (RNs) to healthcare support workers favours RNs recorded a lower average level of STF. (Data: • Spline fit –).

Figure 2.

Relationship between local-averaged HCSW staffing levels and slips, trips and falls (Data: • Spline fit –). Wards with increased healthcare support workers establishment reported a higher rate of slips, trips and falls. This does not factor size of ward; however, there was little variation in ward size (mode 24 beds) occupancy 99%.

The VitalPAC metrics also demonstrated interesting links to the KPI ‘Registered Establishment In Post’. For ‘vomit’ and ‘nausea’ %Occurrence, there is a distinct drop in events after ca. 30 WTE. Below this limit, there is a wide distribution in the %Occurrence of the two VitalPAC metrics, but, above the limit, the variation decreases dramatically giving a near constant value. The data demonstrated an apparent step function when visually inspected; however, the average behaviour appeared more linear in nature. To investigate, the data were analysed by binning the data with respect to the Registered Establishment in Post, and taking the mean and SD of the data. The data were then converted into three data sets, making the x-variable the mean for each set and the y-variable the mean of the %Occurrence of Vomiting plus either 0 SD, 1 SD or 2 SD. This is shown in figure 3. The occurrences to 3 SD are shown in table 4.

Figure 3.

Variation in %Occurrence of vomiting with Registered-in post staffing as a step change.

Table 4.

Summary of pair wise correlation statistic and significance between VitaPac metrics and “incidents of slips, trips and falls”. Occurrence to 3 SD.

	Metric 1		Metric 2		Metric 3
Observation	Rho	p Value	Rho	p Value	Rho	p Value
Heart rate	−0.03	6.5E–01	−0.01	9.3E–01	0.20	9.8E–03
Temperature	−0.34	3.2E–06	0.34	4.0E–06	0.20	7.7E–03
SATS	−0.40	2.8E–08	0.31	2.8E–05	0.19	1.1E–02
Oxygen flowrate	−0.07	3.7E–01	−0.04	6.0E–01	0.02	8.0E–01
BP systolic	0.31	2.5E–05	0.42	4.9E–09	0.11	1.6E–01
BP diastolic	−0.11	1.4E–01	0.30	6.0E–05	0.38	1.9E–07
Respiratory rate	0.34	3.6E–06	0.13	9.0E–02	0.03	7.3E–01
Oxygen concentration	−0.15	5.6E–02	0.12	1.2E–01	0.06	4.4E–01
Vomit	−0.41	8.0E–09
Nausea	−0.52	3.6E–14
Pulse regularity	0.34	2.8E–06
AVPU	0.30	5.8E–05
Bowels open	−0.39	9.9E–08	−0.38	1.9E–07	−0.37	2.7E–07
Oxygen flowrate	0.11	1.6E–01	−0.23	2.3E–03	−0.02	7.8E–01

Data were binned with respect to the x-variable in each scenario, splitting the data into ordered even sized units and then taking the mean in x and y to represent the location. Errors in x and y reflect the SE on the bin. The step function shape applied is of the form [1], where C represents the offset in y, A is the amplitude of the step, x₀ the offset in x and B a width factor

1

Assuming the data inside each bin are normally distributed, these data sets would then represent the distribution at the average, the limit of the 68% confidence window and the limit of the 95% confidence window, respectively. These resulting data sets were then fit by a step function [1].

The rate of STF is also of interest—these seem to peak then start to drop at a total nursing establishment of ∼30 WTE (figure 4). This might suggest either understaffing on wards with average size or that wards with highest establishment have the least mobile patients—limiting the likelihood of falling.

Figure 4.

The rate of slips, trips and falls and total nursing whole time equivalent.

A link between Total N&M Establishment and STFs may be causal. In both cases, further investigation will be necessary to identify the variable link and produce recommendations for optimised patient care. This peaked behaviour against STFs appears in multiple metrics pairs, including STFs against ‘Funded registered establishment-in post’ and STFs against ‘Unregistered sickness-total hours’, peaking at values of 25–30 WTE and 400 hours, respectively.

It was also possible to explore other relationships in this data set, for example, STF and wellness in terms of nausea and vomiting. As rates of nausea on the wards increased, the likelihood of falling decreased (figure 5) exploring relationships such as these will help with understanding of such relationships. Alongside these peaked correlation profiles, there are strong correlations between STFs and the Nausea and Vomit metrics. These behaviours demonstrate a clear negative relationship between the two parameters, with an increase in ‘illness’ metrics (%Occurrence Nausea) and (%Occurrence Vomit) linked to a decrease in STFs .

Figure 5.

Variation in well-being (nausea and vomiting) and slips, trips and falls.

These two types of correlations support the hypothesis that the majority of STFs occur from patient who appear to be healthiest according to the VitalPAC metrics. While this may seem clinically obvious, it offers some assurance that the data and methodology are reflective of the real world. The wards with the highest degree of STFs appear to be linked to patients who appear to be medically well from observations—possibly as a result of higher mobility. This correlation dominated the data set and may mean that this NHS Trust is currently well suited to minimising STFs of patients that are most vulnerable from a medical perspective. If these ‘healthy, but not healthy enough’ patients are still unwell and push themselves to be mobile earlier, they are more at risk of STFs than ill-patients who will remain bed bound for longer.

The VitalPAC metrics also demonstrate interesting links to the KPI ‘Registered Establishment In Post’. For ‘vomit’ and ‘nausea’ %Occurrence, there is a distinct drop in events after ca. 30 WTE. Below this limit, there is a wide distribution in the %Occurrence of the two VitalPAC metrics but, above the limit, the variation decreases dramatically giving a near-constant value.

The raw data set and the fits from the converted data are shown in figure 3. The 1 σ and 2σ data sets demonstrate a clear step function in the window of 25 to 30 WTE RN.

In all figures, the lines were generated as B-splines fit to local averages of the data.

Other metrics such as avoidable PUs did not display strong correlations, but the incidence of avoidable PUs in this Trust is low. The correlation matrix between PUs and VitalPAC metrics demonstrates one significant correlation at the 0.001 level—between PUs and the first ‘Respiratory Rate’ metric. A possible interpretation of an increase in PUs with a decrease in the mean respiratory rate would be to assume a lower average respiratory rate implies a lower degree of patient activity, arising from wards on which patients are more commonly immobile.

Discussion

This study has gained insight into the complexity and the non-linear nature of the relationship between staffing and outcomes. Additionally, it has tested a method commonly used in other industries to provide insight within large data sets. Understanding these relationships will help form a data ontology or knowledge structure for capture of these data in future. Multiple hypotheses need to be constructed from the significant correlations and then tested using local knowledge and in ward observation to extend this work in terms of risk or return in investment. However, there does appear, at least from these data, to be some benefit in vigilance and several clinical safety/staffing step changes which float but could be calculated in real time. A recent comprehensive spending review in England suggests the use of an aggregate average as a metric¹⁹ for staffing; however, using a fairly simplistic arithmetical approach to what appears a complex relationship is unlikely to reflect the real-world situation or form the basis of a robust model.

From this work and the work of others that describe non-linear relationships,^{24 33 34} it seems likely that a data-driven approach would be able to develop a specific optimum staffing model if good quality data were available. Insight generated in this way is used in areas such as retail, banking, aeronautics, aviation, petrochemical and other safety critical industries to maximise efficiency, quality and safety. The process is easily adapted to healthcare but requires good quality data in large volume and also robust concepxtual modelling to ensure any models reflect the real-world situation.

The primary limitation to this study is that the data set originates from one location. Although there was variety, it would be preferable to rerun this work on data from other organisations. In addition, the data and outcomes were predetermined/defined, that is, ‘outcomes’ and this trust had a relatively good RN to patient ratio (mean 1:6). Previous work has shown that RN to patient ratio is a significant factor which affects morbidity and mortality^{22 31} and other studies have shown improved outcomes associated with RN and specialist certification.^{22 33–35}

These techniques rely heavily on high volume, high quality data. The NHS Trust had a bespoke data set of high quality available for this work. This cannot be assured in other similar organisations. In addition, the commercial informatics packages in clinical use that are designed to collect these data rarely appear to consider the extraction of data and the research team relied on the manufacturers support and local expertise to ‘workaround’ issues. Technical issues such as extracting data need to be overcome to make this a practical solution.

Conclusion

The relationship between staffing and outcomes appears to exist. It appears to be non-linear but calculable and a data-driven model appears possible. These findings could be used to build an initial mathematical model for acute staffing which could be further tested. This is the first iteration of a highly iterative process and would require more research to produce a real-time solution. It could even lead to whole systems approaches to modelling staffing, including predictive modelling.