Modeling the Distribution of Nursing Effort Using Structured Labor and Delivery Documentation

Abstract

Our study objectives included the development and evaluation of models for representing the distribution of shared unit-wide nursing care resources among individual Labor and Delivery patients using quantified measurements of nursing care, referred to as Nursing Effort. The models were intended to enable discrimination between the amounts of care delivered to patient subsets defined by attributes such as patient acuity. For each of five proposed models, scores were generated using an analysis set of 686,402 computerized nurse-documented events associated with 1,093 patients at three hospitals during January and February 2006. Significant differences were detected in Nursing Effort scores according to patient acuity, care facility, and in scores generated during shift-change versus non shift-change hours. The development of nursing care quantification strategies proposed in this study supports outcomes analysis by establishing a foundation for measuring the effect of patient-level nursing care on individual patient outcomes.

Introduction

Computerized information systems that incorporate structured documentation provide tremendous opportunities for improving resource management and refining clinical and administrative processes. Documentation of nursing activities, including the timing of task completion and information about the patient for whom nursing activities were performed, enables modeling and analysis of nursing practice patterns. Accurate models of nursing care patterns that consider individualized patient needs and potential outcomes support the prediction of resource requirements enabling efficient distribution of available nursing resources. These models inform staffing decisions, facilitating adequate apportioning of nurses while maintaining quality care. In addition to impacting the efficiency of clinical and financial systems, development of nursing care performance indicators, such as measures of care provided to individual patients, supports quality improvement and accountability. [1] Measuring and reporting nursing quality indicators serve to quantify the influence of nursing care on patient safety and outcomes, allow benchmarking of best practices, support the identification of standards for staffing rates, and help to identify gaps in quality. [2]

Previous patient safety and quality of care initiatives have focused on providing empirical evidence to support the identification of relationships between quality nursing care and staffing. [3, 4] Though prior work has captured the amount of care delivered to the unit, specialty, or hospital, the current study used structured documentation as a surrogate for nursing care in representing nursing care quantities received by individual patients. Efforts in the study focused on enabling patient-level analyses of correlations between nursing care and patient outcomes. Modeling nursing practice within the L&D setting served as a prototype for modeling care in other settings. Though the resulting models are likely to vary as nursing practice varies from setting to setting, the presented methods are extensible to other clinical domains in which structured and computerized point-of-care documentation are available.

Background

Many investigators of nursing quality improvement efforts have published research where nursing resources in various care settings were represented using measured nurse-to-patient ratios. In California, nursing quality initiatives have focused on determining minimum nurse staffing rates that maintain quality care. [5] Preliminary results from the California studies have failed to identify a significant effect of mandated ratios on patient care and further studies have been recommended. [6, 7] Although other studies have found evidence that nursing structures and processes are correlated with patient outcomes, variation in data sources, reliance on administrative data, which have been identified as poor measurements of complications, and issues related to data extrapolation methods have yielded imprecise and inconclusive results. [8– 10] At least one other study established a correlation between nursing rates and the quality of patient care; however, the study did not explore the distribution of nursing care among patients of varying acuity, nor did the study capture the patterns of interaction between individual nurses and patients. [11]

Some researchers suggest that development of additional nursing quality and patient safety indicators would support the establishment of evidence correlating staffing levels with patient outcomes. [9] Scoring schemes developed for use in the Intensive Care Unit (ICU) measure nurse workload and identify resource requirements at the unit level. [12–19] Though these schemes provide some fundamental insight into workload demands, and may support decision makers in planning staffing assignments based upon workload expectations, they do not capture the actual amount of care delivered to individual patients. In a sense, these scoring schemes estimate overall expected work rather than measuring actual work performed, and none of the schemes characterize the distribution of work among individual nurses or patients. Although these schemes are beneficial for determining manageable workloads, they do not offer insight into the effect of altered workload on patient outcomes.

The Nursing Care Hours (NCH) indicator, developed by The National Database of Nursing Quality Indicators (NDNQI), is defined as, “the number of productive hours worked by nursing staff assigned to the unit who have direct patient care responsibilities for greater than 50% of their shift.” [20] In the current study, models were defined for measuring the hourly quantity of nursing care, referred to as Nursing Effort, received at the patient level. Nursing Effort models were patterned after the NCH indicator, but implemented a simpler calculation. Scores could be computed directly from nursing documentation, eliminating the need for nurse schedule logs or the comprehensive review of nurse activities to determine the percentage of a shift devoted to direct patient care. Similar to the automatic calculation of other nurse workload measurements, [21] the calculation of Nursing Effort from structured nursing documentation may be automated. Nursing Effort measurements provide quantifiable measures of nursing care extrapolated from documentation according to distinct instances of documented nursing activities. The measures, as represented by the documentation of specific nursing observations and interventions, provide more information than a log of the sheer number of nurses present at a given time. The consistent use of indicators such as Nursing Effort will potentially reduce variation in study results, providing stronger evidence that supports minimal staffing rates and the association between nursing care and patient outcomes.

Electronic, structured L&D documentation at Intermountain Healthcare, which includes details about each nurse-patient interaction, has supported a number of initiatives for improving clinical and administrative processes. Labor level categories, which represent patient acuity levels, are calculated by evaluating structured point-of-care patient data captured at the patient bedside. Previous work has automated the calculation of labor levels supporting the charge capture process. [22] Accurate charge summaries were generated by integrating embedded time and data-driven logic into the charge rules to account for variations in intensity and duration of nursing activities. Our previous work focused on extracting nursing practice patterns from structured L&D data. [23] Through identifying variations in patterns based on patient characteristics, the research demonstrated the feasibility of generating patient profiles and forecasting patient outcomes. Additionally, the study noted the potential of using practice pattern analysis to examine the effects of patient load on nursing documentation rates, suggesting that the inclusion of patient acuity data could improve nurse patient load analysis.

In the current study, our research objective was to analyze the distribution of documented nursing activities among individual patients sharing unit-wide nursing care resources. Five Nursing Effort models were implemented, compared, and evaluated with respect to their capacity to identify variations in nursing care according to patient acuity, facility where care was provided, and hour of the day (comparing shift change hours to non-shift change hours). The research was intended to establish a foundation for measuring the effect of nursing care quantity on patient outcomes.

Methods

Design

This descriptive study incorporated structured retrospective data into an analysis framework that involved defining models for Nursing Effort, data selection and preparation, the development of an object-oriented application to support score calculations, and comparison of results from the various models. The objective was to develop models that would facilitate the identification of variations in the amount of nursing care delivered to various patient subsets. For example, to be successful, the models needed to discriminate between the amounts of nursing care provided to patients of varying acuity. The amount of care measured for high acuity patients needed to represent the level of nursing intensity required by those patients.

In 2006, Intermountain Healthcare’s Storkbytes™ (Intermountain Healthcare, Salt Lake City, UT) clinical information system served approximately 29,000 patients at 15 facilities. Storkbytes™ combined electronic acquisition of fetal monitor measurements with computerized nurse charting. [24] The Storkbytes™ information system represented nursing concepts and patient outcomes using an internally developed controlled vocabulary. The objectives for development and refinement of the proprietary coding system have been in line with the motivation for developing standard nursing terminologies: namely, to identify, name, and classify the major concepts of the domain [25] “for recording and studying the patient care problems nurses address,” [26] and to support improved efficiency in nurse documentation. [27] Like standard nursing terminologies, Storkbytes™ codes were developed to represent concepts involving nurse-patient interactions as opposed to unit level resource needs. The efforts enabled implementation of a computer information system, [24] exchange and warehousing of data from various Intermountain Healthcare L&D units, support for reimbursement of nursing activities, [22] and the development of nursing knowledge discovery tools. [28] Limitations of existing nursing reference terminologies [29] – specifically in terms of coverage and granularity – motivated the development of Intermountain’s L&D coding system. Modeled after the best aspects of available nursing terminologies, with a focus on including codes for nurse-patient interactions absent from standard nursing terminologies (such as “patient repositioned for comfort”), [30] Storkbytes™ codes enable nurse documentation representative of actual L&D care processes delivered at the patient level.

Deployed in the 1980’s, Storkbytes™ has undergone iterative modifications in response to ongoing feedback from clinical users and developments in nursing terminology standards. The Storkbytes™ controlled vocabulary was mapped to Intermountain Healthcare’s internal controlled vocabulary, which in turn, has undergone continual mapping to evolving terminologies. The L&D Nursing Standards and Education work group of the Women and Newborn Clinical Program has provided ongoing validation and refinement of the nurse documentation elements to ensure that the system accurately represents nursing care processes. The application provided a menu-based interface allowing structured documentation of nursing interventions and observations (2,552 such concepts were supported). Metadata were recorded with each data point identifying the documenting nurse, the time of documentation, and the time that the nursing activity occurred. Each month, Storkbytes™ data were extracted to Intermountain Healthcare’s Enterprise Data Warehouse (EDW) along with demographic, financial, case mix, and other clinical data relating to each patient. [31]. The availability of large amounts of structured documentation representing high levels of nurse-patient interaction qualified Labor and Delivery as an appropriate setting for these analyses

Data Models

Five increasingly complex Nursing Effort scoring models were defined for measuring the hourly quantity of nursing care received by each patient in the study. Each hour, every documenting nurse was allocated one unit of distributable Nursing Effort to dispense among patients (non-clinical activities were not included in the measurement, thus the actual time spent with patients was one hour minus the time spent on other activities). The sum of Nursing Effort scores for all patients within a given facility or unit during a particular hour was equal to the number of documenting nurses present during that hour. Although the simplest model considered only the numbers of nurses and patients present during an hour, successive models considered the size of nurses’ patient loads, patient acuities, and the number of activities documented for each patient.

The Simple Nurse Patient Ratio, represented by Eq. (1), was intended as a benchmark for comparison to the other four models. Consistent with measures used in previous quality initiatives, [5] the model represented hourly nursing care with a straightforward ratio of the numbers of patients and nurses present at a particular facility. The numbers of patients and nurses were inferred from documentation occurring during that time period. For example, during a particular hour in a facility with five patients and six documenting nurses, the six available units of Nursing Effort were distributed equally among the five patients, allotting each patient a score of 1.2.

$${\mathit{\text{NursingEffort}}}_1=\frac{\mathit{\text{Total Number of Nurses}}}{\mathit{\text{Total Number of Patients}}}$$ (1)

In the Sum of Unadjusted Load Fractions model, represented by Eq. (2), each patient for whom a nurse entered documentation represented an equal fraction of that nurse’s total patient load. Nursing Effort became a patient-level, rather than a unit-level, attribute calculated by summing the appropriate fraction of each documenting nurse’s patient load. For example, if a patient received documentation from two nurses A and B during an hour and nurse A documented for three total patients during the hour while nurse B documented for two patients during the same hour, according to the model, the patient received 1/3 of a unit of Nursing Effort from nurse A and 1/2 of a unit of Nursing Effort from nurse B. The total Nursing Effort score for the patient, calculated by summing fractional Nursing Effort from each nurse, was 0.83. Like the Simple Nurse Patient Ratio, the key simplifying assumption of this model was that each patient consumed an equal portion of each documenting nurse’s total time.

$${\mathit{\text{NursingEffort}}}_2={\displaystyle \sum _{\underset{\mathit{\text{For This Patient}}}{\mathit{\text{Nurses Documenting}}}}\frac{1}{\mathit{\text{Patient Load}}}}$$ (2)

The remaining models relied on the assumption that higher acuity patients required a greater amount of nursing care than did lower acuity patients. Acuity was reflected by assignment to one of four labor levels (detailed below), with level one representing the least severe cases and level four indicating the most severe. Eq. (3) defined a labor level relationship derived from the billing rate ratio of hourly nursing care for patients of each labor level. The relationship was used to weigh each patient’s share of nurse load in the acuity dependent Nursing Effort models.

$$2.45{\mathit{\text{LL}}}_1=2.04{\mathit{\text{LL}}}_2=1.55{\mathit{\text{LL}}}_3={\mathit{\text{LL}}}_4$$ (3)

The Sum of Acuity Adjusted Fractions by Majority of Hour model, represented by Eq. (4), incorporated patient acuity into the Nursing Effort score, yet maintained a simplified calculation. Similar to the Sum of Unadjusted Load Fractions equation, each patient’s hourly score was calculated from the summed fractions of Nursing Effort assigned from each documenting nurse. In this model, however, the fractions of effort were weighted as a function of that patient’s labor level for the majority of the hour so that higher labor level patients received a larger portion of a documenting nurse’s available Nursing Effort. According to the labor level relationship, a level-one patient and level-four patient receiving care from a single nurse obtained unequal portions of the nurse’s one hour of available effort. The level one patient received a score of 1/3.45 or 0.29 while the level-four patient received a score of 2.45/3.45 or 0.71

$${\mathit{\text{NursingEffort}}}_3={\displaystyle \sum _{\underset{\mathit{\text{For This Patient}}}{\mathit{\text{Nurses Documenting}}}}\frac{f\mathit{(}\mathit{\text{This Patient's Majority of Hour Labour Level}}\mathit{)}}{{\displaystyle \sum _{\mathit{\text{Documented Patients}}}f\mathit{(}\mathit{\text{Patient Labour Level}}\mathit{)}}}}$$ (4)

The Sum of Acuity Adjusted Fractions by Minute model, represented by Eq. (5), more realistically reflected patient acuity. As in the majority of hour acuity model, the equation was calculated using the weighting relationship defined in Eq. (3). In this model, rather than distributing Nursing Effort among patients according to the majority of hour labor level, Nursing Effort was distributed based upon the relative number of minutes each patient qualified for each labor level.

$${\mathit{\text{NursingEffort}}}_4={\displaystyle \sum _{\underset{\mathit{\text{For This Patient}}}{\mathit{\text{Nurses Documenting}}}}\frac{f\mathit{(}\mathit{\text{This Patient's Minute}}-\mathit{\text{by}}-\mathit{\text{Minute Labour Level}}\mathit{)}}{{\displaystyle \sum _{\mathit{\text{Documented Patients}}}f\mathit{(}\mathit{\text{Patient Labour Level}}\mathit{)}}}}$$ (5)

The Acuity Adjusted Documented Activity Ratio, represented by Eq. (6), considered the actual number of tasks documented to capture where nurses spent effort. The equation was calculated by summing the ratio of tasks documented by each nurse for a specific patient and the total number of tasks documented by each nurse during the hour. The tasks were then weighted so that tasks documented for lower level patients were assigned less weight than tasks documented for higher level patients.

$${\mathit{\text{Nursing Effort}}}_5={\displaystyle \sum _{\underset{\mathit{\text{For This Patient}}}{\mathit{\text{Nurses Documenting}}}}\frac{f\mathit{(}\mathit{\text{This Patient's Number of Documented Activities}}\mathit{)}}{{\displaystyle \sum _{\mathit{\text{Documented Patients}}}f\mathit{(}\mathit{\text{Number of Documented Activities}}\mathit{)}}}}$$ (6)

Data Selection and Preparation

Data for 1,093 patients (no exclusions) admitted to one of three Intermountain Healthcare L&D units during the months of January and February 2006 were retrieved from the EDW for analysis. The selected facilities varied in size and in the number of supported births during 2006: the largest, a trauma one center, accounted for approximately 4,200 deliveries; the midsize facility had 2,300 deliveries; and the smallest, a community hospital, had 1,300 deliveries. All three facilities used a 12-hour shift rotation.

Administrative data and documentation events associated with each patient were retrieved to support analysis. Each documentation event consisted of a Storkbytes™ code identifying the particular type of intervention or observation documented, an identifier for the nurse who entered the data, and the time and date of the event. For the 1,093 patients, 686,402 documentation events were retrieved. Administrative data, including patient admission time and date as well as admission facility, were used to create patient objects with which the retrieved documentation elements could be grouped.

The selected data elements required additional preparation to support analysis. Three of the five models required patient acuity information to generate Nursing Effort scores. An algorithm, developed by Intermountain Healthcare, used nursing documentation events to assign patients to each of four labor levels. For each patient, events were chronologically processed to generate a timeline of patient labor level changes. Using the timelines, it was possible to determine each patient’s majority of hour or minute-by-minute labor level to calculate scores using the particular Nursing Effort model. To support comparison of shift change and non-shift change hour patterns, documentation events were assigned a Boolean identifier representing whether or not they occurred during the hour of a shift change.

Application Development

The Nursing Effort Analysis Tool (NEAT) software application was developed to manage data preparation and the calculation and evaluation of Nursing Effort scores. NEAT, developed using Visual C++ (Microsoft Corporation, Redmond, WA), interpreted delimited query results retrieved from the EDW and stored in a text file. Each data row contained information about a particular patient and a single coded documentation event. As the text file was parsed, NEAT instantiated instances of C++ objects to represent each unique patient and each documentation event. NEAT also generated nurse, hour, and facility specific objects for each hour during the analysis period to enable the calculation of Nursing Effort scores (see Table 1). The NEAT class infrastructure facilitated the calculation of hourly scores for every patient according to each of the five Nursing Effort models. Regardless of model, because the sum of distributed Nursing Effort scores during an hour was equal to the number of documenting nurses, the mean of all Nursing Effort scores during an hour was always equal to the number of documenting nurses divided by the number of patients. Thus, instead of calculating hourly facility-wide means, means were calculated for patient subsets based on majority of hour labor level. Once calculated, scores were exported to an external application for statistical evaluation.

Table 1.

Description of key NEAT objects supporting the calculation of Nursing Effort scores

Object	Description
Documentation Event	Contains a coded identifier of the type of intervention or observation represented by the event, event time and date, event measurement values or relevant details, nurse identifier, and a patient identifier.
Patient	Contains a de-identified index number, patient demographic information, all of the patients Documentation Event objects, and a timeline of labor level changes throughout labor.
Labor Level Conditions	Contains the status of relevant conditions throughout patient labor to facilitate the calculation of a patient labor level timeline.
Nurse Hour	Contains a nurse identifier, and indices of all the patients for whom the nurse entered documentation during a particular hour
Facility Hour	Contains all Nurse Hour objects for a specific hour and facility. Also contains Nursing Effort scores, stratified by majority of hour labor level, for all patients treated during the hour at the facility.

Model Evaluation

Following data preparation and processing, score calculations were exported from NEAT into the SAS™ (SAS Institute Inc., Cary, NC) application for further evaluation. Each exported data row contained information about a specific patient during one hour of the study period. Data included a de-identified patient index, a care facility identifier, a flag indicating shift change hour status, patient majority of the hour labor level, and scores generated by each of the five models. Summary data were produced to tabulate the proportion of hourly scores that fell into each category of labor level, facility, and shift change. The distributions of scores generated from each model were examined and analysis of variance tests were performed to detect differences among Nursing Effort scores generated by each model according to patient acuity, facility, and shift-change status. The Nursing Effort scores provided statistical power (1-β) of 90.0% for detection of an effect size of 0.15 with a two-tailed α of 0.05. Additional analysis was performed to determine statistical correlation among the various Nursing Effort scoring models from throughout the 2-month analysis period.

Results

Table 2 presents the composition of calculated hourly Nursing Effort scores. With respect to majority of hour labor level groups, labor level two scores were the most prevalent followed by level three, then level one. As expected, because level four represented the most severe, and uncommon, patients, there was a small percentage of level four hourly scores in the analysis (0.2%). Shift changes were represented in 8.1% of the hourly scores.

Table 2.

Composition of hourly Nursing Effort scores

	Number of Instances (N = 12,017)	Percentage of Total
Labor Level 1	2,849	23.7%
Labor Level 2	5,170	43.0%
Labor Level 3	3,980	33.1%
Labor Level 4	18	0.2%
Large Facility	6,937	57.7%
Medium Facility	3,248	27.0%
Small Facility	1,832	15.3%
Shift Change Hours	972	8.1%
Non-Shift Change Hours	11,045	91.9%

Comparison of each model’s minimum, maximum, mean, and standard deviation values (Table 3) demonstrated that although the mean score value for each model was consistent, the variation in scores increased from model one to model five. The means remained constant because each Nursing Effort distribution model represented data from the same set of nurses and patients. Though the allocation of Nursing Effort varied in each model, the amount of allocatable units remained constant. The values of scores produced ranged from 0.0 to 4.48. Within that span, using precision to the hundredths place, 449 possible values could be represented (0.0, 0.1, 0.2 … 4.47, 4.48). Model two used the fewest number of available values (6.0%) producing the least granular score distribution. Rather than utilizing the continuum of values, model two generated a large number of repeated scores. Model five made use of 63.1% of the possible values offering the finest granularity of score distribution.

Table 3.

Attributes for scores from each Nursing Effort model

	Model 1	Model 2	Model 3	Model 4	Model 5
Minimum Calculated Score	0.33	0.25	0.21	0.21	0.0
Maximum Calculated Score	4.00	4.00	4.14	4.16	4.48
Mean of Calculated Scores	1.13	1.13	1.13	1.13	1.13
Standard Deviation of Scores	0.31	0.51	0.51	0.51	0.54
Utilization of Available Score Values	9.6%	6.0%	34.0%	47.6%	63.1%

Table 4 lists mean hourly Nursing Effort scores according to each of the four labor levels. Analysis of variance tests revealed statistically significant differences among majority of hour labor level groups in each of models two (p = 0.0002), three (p < 0.0001), four (p < 0.0001), and five (p < 0.0001). Significant differences, however, were not detected among groups using model one scores (p = 0.09).

Table 4.

Comparison of mean hourly Nursing Effort scores stratified by patient majority of hour labor level

Majority of Hour Labor Level	Model 1	Model 2	Model 3	Model 4	Model 5
1	1.16	1.14	1.12	1.12	1.13
2	1.16	1.16	1.16	1.16	1.16
3	1.16	1.18	1.20	1.19	1.20
4	1.14	1.00	1.07	1.07	0.98
P-Value	0.09	0.0002	< 0.0001	< 0.0001	< 0.0001

Computing Pearson’s correlation coefficients (Table 5) allowed analysis of correlation between scores in each model. In spite of the more sophisticated algorithms used to generate Nursing Effort measures using models three, four, and five as compared to those generated by model two, very high correlation was found among scores generated by all four models. Additional analysis focused on detecting significant differences among facility subgroups and between scores generated during shift change versus non-shift change hours. Table 6 displays mean Nursing Effort scores for each facility stratified by majority of hour labor level. For each model, analysis of variance tests demonstrated a significant difference among the scores from each facility (p < 0.0001). Table 7 compares mean Nursing Effort scores occurring during shift change hours to scores generated during non-shift change hours and stratified by majority of hour labor level. Again, analysis of variance revealed a statistically significant difference between scores occurring during shift change and non-shift change hours in each model (p < 0.0001) indicating greater amounts of Nursing Effort received by patients during shift change hours.

Table 5.

Pearson correlation between hourly scores generated by each Nursing Effort model

	Model 1	Model 2	Model 3	Model 4	Model 5
Model 1	1.000	-	-	-	-
Model 2	0.611	1.000	-	-	-
Model 3	0.610	0.997	1.000	-	-
Model 4	0.610	0.998	1.000	1.000	-
Model 5	0.571	0.963	0.964	0.964	1.000

Table 6.

Comparison of mean hourly Nursing Effort scores at each facility stratified by patient majority of hour labor level

Model	Majority of Hour Labor Level	Small Facility	Medium Facility	Large Facility
1	1	1.26	1.13	1.13
	2	1.25	1.13	1.14
	3	1.26	1.13	1.15
	4	N/A	1.00	1.16
2	1	1.21	1.14	1.11
	2	1.28	1.13	1.13
	3	1.26	1.14	1.18
	4	N/A	1.00	1.00
3	1	1.19	1.13	1.08
	2	1.28	1.13	1.12
	3	1.28	1.15	1.20
	4	N/A	1.00	1.09
4	1	1.20	1.13	1.09
	2	1.28	1.13	1.13
	3	1.28	1.15	1.20
	4	N/A	1.00	1.09
5	1	1.19	1.14	1.10
	2	1.27	1.13	1.12
	3	1.30	1.15	1.20
	4	N/A	1.00	0.98

Table 7.

Comparison of mean hourly Nursing Effort scores during shift change and non-shift change hours stratified by patient majority of hour labor level

Model	Majority of Hour Labor Level	Shift Change Hours	Non-Shift Change Hours
1	1	1.54	1.12
	2	1.62	1.12
	3	1.56	1.13
	4	1.80	1.10
2	1	1.34	1.12
	2	1.75	1.11
	3	1.65	1.14
	4	2.00	0.94
3	1	1.33	1.10
	2	1.75	1.11
	3	1.66	1.16
	4	2.00	1.02
4	1	1.33	1.10
	2	1.75	1.11
	3	1.66	1.16
	4	2.00	1.02
5	1	1.33	1.11
	2	1.74	1.11
	3	1.67	1.16
	4	2.00	0.92

Discussion

Significance of Findings

As no account for labor level went into model one (Simple Nurse Patient Ratio) score calculations, we did not expect to detect a significant difference among the scores associated with each labor level using the model. The second model (Sum of Unadjusted Load Fractions) also did not consider labor levels in its calculations; nevertheless, a statistically significant difference was detected among model two scores associated with the various labor levels. This finding suggests that the second model implicitly represented acuity by capturing two dynamics of the nursing care process: higher acuity patients had more nurses documenting for them than did lower acuity patients; higher acuity patients shared documenting nurses with fewer other patients. As models three (Sum of Acuity Adjusted Fractions by Majority of Hour), four (Sum of Acuity Adjusted Fractions by Minute), and five (Acuity Adjusted Documented Activity Ratio) included explicit representations of patient acuity, the detection of significant differences in Nursing Effort scores among labor level groups was expected.

Though statistically significant, differences in Nursing Effort scores must be further evaluated to determine clinical importance. Key to illuminating the clinical significance of variations in Nursing Effort is an examination of those variations with respect to the outcomes experienced by individual patients. Within Labor and Delivery, small variations of measured Nursing Effort may not have a drastic impact on the likelihood of quantitative outcomes such as Cesarean delivery, the occurrence of fetal distress, or the occurrence of other labor complications; however, even small doses of additional nursing care may affect patient satisfaction or reduce the durations of various labor stages. Larger variations in the quantity of nursing care received by patients may have an effect on the occurrence of unfavorable outcomes in higher-risk patients as well as on the quality of uncomplicated labor in lower-risk patients. By examining the relationship between care and outcomes, future studies will focus on determining the amount of nursing care variation considered clinically “large” or “small.” Identifying the outcomes that are most sensitive to variations in nursing care processes is vital to improving care quality and outcomes in L&D as well as in other care domains. By recognizing the patients that are most likely to benefit from additional care, staff managers will be better able to focus available resources where they will have the greatest impact.

Model Comparison

As demonstrated by the Pearson’s analysis, there was high correlation (0.963 or higher) among scores generated by models two, three, four and five, indicating comparable representations of Nursing Effort by the four models. Given the added complexity of calculating scores for models three, four, and five, model two (Sum of Unadjusted Load Fractions) appeared preferable. Consistent with Occam’s Razor, the simpler model may be the best alternative for calculating informative Nursing Effort scores. As model two calculations did not necessitate the generation of patient acuity from nursing documentation, calculating Nursing Effort scores using the model simplified the scoring process. Additionally, as shown in Table 3, model two had the least granular scores, using only 6.0% of the available range of scores compared to the finer-grained distribution of model five scores with 63.1% utilization of the available score band. Scores generated by model two typically aligned with one of four clusters that could be reproduced by approximating scores generated by models three, four, and five. Fine score granularity, or high precision of nursing care quantification, may not be crucial to distinguishing between qualitative ratings of nursing care. The quality of care associated with a given Nursing Effort score may be easier to determine by identifying proximity of the score to a particular Nursing Effort cluster, or threshold value. It is possible that added granularity provided by models three, four, and five could be more informative in alternate care settings, especially if outcomes were highly sensitive to small variations in nursing care.

Model one was fundamentally different than the other four models, representing scores as simple unit-level nurse-to-patient ratios. Scores generated by the first model were not highly correlated with scores generated by the other models. Further research will identify whether the unit-level or patient-level approach at measuring nursing care is better suited for predicting likely patient outcomes. As model one used nurse-to-patient ratios to measure the availability of nursing resources, it is possible that high model one scores may be correlated with successful management of critical scenarios. On the other hand, model one scores may inadequately represent the amount of nursing care received by individual patients, limiting the evaluation of nursing care effects on individual outcomes. As models two, three, four, and five measured nursing interactions at the patient level, their scores may offer additional insight into the distribution of nursing care, and better indicate the quality of care provided to lower-risk patients. Of course, we would not rule out the potential for development of other models that also support analysis of nursing care patterns and enable improvements to patient care quality.

Documentation Requirements

Advocates of charting by exception argue that the capture of abnormal data more efficiently represents patient conditions and highlights possible complications. [32] To allow extraction of nursing care patterns, the implementation of any charting model must not obscure the details of nurse patient interactions, nor must it depend on human interpretation for determination of the abnormal. Without a record of timing and frequency of nurse-patient interactions, it would be difficult to accurately identify patterns in nursing behavior or to model the distribution of nursing care. The current research demonstrates the usefulness of a structured computerized documentation system that captures normal data elements along with observations and interventions deemed abnormal.

Limitations and Future Directions

The developed models relied on a number of simplifying assumptions. First, documentation was used as a surrogate for nursing care. The limiting assumption was that all performed tasks were not only documented, but that documentation was accurate, timely, and complete. Missing or erroneous instances of documentation would influence score calculations. Although not measured in the current study, analysis of charting compliance using other real-time clinical information systems suggests that training and education support the capture of accurate and complete documentation [33] as required for generating representative Nursing Effort calculations. The second model was the most robust to missing documentation. As long as the providing nurse documented a single event for each managed patient each hour, model two calculations were accurate. A second limitation was the assumption that that quality of care represented by a unit of Nursing Effort was equal among all providers. Over 90% of Storkbytes™ users were Registered Nurses (RNs); however, factoring for variation in training and competencies when measuring contributions from individual nurses would provide a qualitative weighting for quantified measurements. A third simplifying assumption gave all nursing activities equal weight in calculations by the fifth model. Though some tasks required more time to complete, we assumed that difficult and easy tasks would result in a mean per task time that was consistent among same-acuity patients. Acuity weightings in the model represented increased task complexity, and greater Nursing Effort received by higher acuity patients.

The limited number of level four patients was another study constraint. Only 0.2% of the generated scores came from these most severe patients. Furthermore, only two of the 1,093 patients in the study reached labor level four. In all five models, the level four patient scores were lower than those measured for the three lower acuity levels. This may be partly a result of the way documentation was recorded for level four patients. By standard, level four patients were constantly managed by at least two dedicated nurses. In this high intensity scenario, documentation, though accurate and complete, may have been exclusively performed by one of the managing nurses. In the study, accurate appraisal of available Nursing Effort depended on all nurses entering documentation for each managed patient every hour. An alternate mechanism may be appropriate for measuring Nursing Effort provided to level four patients, though the scarcity of the patient subset limits the impact of possible level four miscalculations on Nursing Effort scores generated for the level one, two, and three patients. Nevertheless, the size of the patient subset limits the conclusions that may be made from the available data.

In the next phase of research, we will examine individual patient scores over the duration of care. In addition to calculating per patient scores, as opposed to hourly calculations of Nursing Effort, we will include patient outcomes in the analysis to support identification of relationships between Nursing Effort and individual patient outcomes. Additional patient attributes will be included in the analysis to detect variations in effects across various patient subgroups. As discussed previously, the sensitivity of outcomes to changes in Nursing Effort scores will be explored both in the general case, and within patient profiles consisting of similar patient characteristics. Longer-term research aims will explore the effects of prenatal care throughout the duration of pregnancy on Labor and Delivery outcomes.

Conclusions

Comparison of Nursing Effort distribution patterns by facility, by time of day, or by any other sub-categorization enables a more fine-grained examination of practice variations. Higher fidelity descriptions of nursing care delivery (according to facility, unit, time of day, or season) facilitate more accurate specification of minimal staffing to ensure that adequate nursing resources are available to maintain quality patient care. By establishing measurements of Nursing Effort that implicitly or explicitly consider patient acuity, this research paves the way for more detailed studies of the relationship between nursing care delivery (including staffing) and patient outcomes.

Study findings also provide a foundation for additional research examining patient and provider issues in obstetric care. To study obstetric patients, similar analyses may be performed to identify statistical differences between patient subgroups defined by demographics, insurer information, or other salient patient characteristics. At the provider level, some literature suggests that a cascade of interventions exists in obstetric care, wherein one obstetric intervention has unintended consequences that lead to further interventions, unnecessarily increasing the intensity of nursing and cost of care. [34] The cascade of interventions could be studied using the patient-level Nursing Effort scores.

The availability of warehoused electronic nursing documentation was essential for the calculation of the new Nursing Effort scores proposed in this study. The calculation of patient-level Nursing Effort scores necessitated structured computerized documentation and a system that captured normal data elements along with observations and interventions deemed abnormal. As new charting paradigms emerge, precautions must be taken to ensure that collected data continue to allow extraction of care delivery patterns and patient outcomes. Development of the Nursing Effort measurement also highlights the importance of the continual evolution of standard nursing terminologies that represent nurse-patient interactions. Efforts to generalize the presented methods would benefit from standard representations of nursing concepts, which embody all aspects of nursing care processes, and are consistent in disparate computerized documentation systems and across care units, facilities, and institutions.

As nursing workflows and representations of patient acuity are dependent on the setting of care delivery, it is expected that Nursing Effort distribution also varies according to care setting. Though Nursing Effort analyses were performed in the Labor and Delivery setting in the current study, similar methods could be executed in other clinical environments where structured and computerized nursing documentation are available. Such efforts would offer insight into domain specific nursing patterns and enable study of the effects of nursing processes on patient outcomes relevant to the study setting.