Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Dec 28.
Published in final edited form as: Nurs Res. 2015 Sep-Oct;64(5):361–371. doi: 10.1097/NNR.0000000000000112

Using Electronic Health Record (EHR) “Big Data” to Examine the Influence of Nurse Continuity on a Hospital-Acquired Never Event

Janet STIFTER 1, Yingwei YAO 1, Muhammad Kamran LODHI 2, Karen Dunn LOPEZ 1, Ashfaq KHOKHAR 3, Diana J WILKIE 1,4, Gail M KEENAN 1,4
PMCID: PMC4692274  NIHMSID: NIHMS743280  PMID: 26325278

Abstract

Background

There is little research demonstrating the influence of nurse continuity on patient outcomes despite an intuitive belief that continuity of care makes a difference in care outcomes.

Objective

To examine the influence of nurse continuity (the number of consecutive care days by the same/consistent RN[s]) on the prevention of hospital-acquired pressure ulcers (HAPU).

Method

A secondary use of data from the Hands on Automated Nursing Data System (HANDS) was performed for this comparative study. The HANDS is a nursing plan of care (POC) “big data” database containing 42,403 episodes documented by 787 nurses, on 9 units, in four hospitals and includes nurse staffing and patient characteristics. Via data mining, we created an analytic dataset of 840 care episodes, 210 with and 630 without HAPUs, matched by nursing unit, patient age, and patient characteristics. Logistic regression analysis determined the influence of nurse continuity and additional nurse-staffing variables on the presence of HAPUs.

Results

Poor nurse continuity (Continuity Index=.21-.42 [1.0=optimal continuity]) was noted on all nine study units. Nutrition, mobility, perfusion, hydration, and skin problems on admission as well as patient age were associated with HAPUs (p<.001). Controlling for patient characteristics, nurse continuity and the interactions between nurse continuity and other nurse-staffing variables were not significantly associated with HAPU development.

Discussion

Patient characteristics including nutrition, mobility, and perfusion were associated with HAPUs, but nurse continuity was not. One study implication is that to reduce the incidence of HAPUs the most effective resource utilization might be in the continued development of best practices to address patient characteristics that lead to pressure ulcer vulnerability rather than a focus on nurse staffing.

Keywords: Health information technology, data mining, nurse continuity, pressure ulcers

Introduction

Patient safety and superior quality outcomes have become the mantra even as healthcare organizations struggle with declining resources. Healthcare costs have escalated, largely due to negative care associated outcomes including never events (e.g., pressure ulcers [PUs], falls); serious, largely preventable adverse outcomes that occur during hospitalization.1 Approximately 13.5% or 1 in 7 hospitalized Medicare beneficiaries were affected by an adverse event2 with estimated costs to hospitals of 21 billion dollars annually.3 One of nurses’ primary concerns is to protect patients from harm while under their care.4 Nurses can prevent patient harm from adverse events through their ongoing assessments and monitoring, by identifying risks, and selecting appropriate preventive interventions. However, hospital nursing services are increasingly being reorganized to reduce expenses. This reorganization (i.e., decreasing registered nurse [RN] care hours, use of more assistive and temporary nursing personnel) has fragmented staffing practices, affected care continuity, and may be related to poor patient outcomes.

Poor care continuity may be an influential nurse-staffing variable for patient outcomes. Current staffing practices, unfortunately, have nurses frequently assigned to new patients every shift. This practice leaves them with no basis of comparison from the day before and may affect their ability to identify and intervene on ominous changes in patient status. This problem is further magnified with novice or new graduate RNs who have little experiential base to compare their findings. In the current hospital environment nurses float between units, part-time nurses fill gaps in the schedule, and nurses care for a larger number of patients than in previous eras. These staffing practices also affect care continuity and may further impair the nurses’ ability to recognize and expediently intervene on negative changes in patient status. Researchers have examined other nurse-staffing variables integral to reorganization efforts such as RN education, experience, or patient-to-nurse ratios but failed to demonstrate convincing evidence to explain how changes in nurse staffing may influence negative care associated outcomes such as never events.5-6 We believe an examination of nurse continuity may provide the missing link to demonstrate that connection.

Chief Nursing Officers (CNOs) have historically not been able to challenge these current staffing trends as they have not had readily available data to do so. A CNO needs data representing staffing variables such as nurse continuity as well as linked data that can help demonstrate the connection between nurse staffing and patient outcomes. Without that data the reorganization of nursing services will continue, potentially exacerbating hospital-acquired adverse events. Discovering the hidden knowledge available within electronic health record (EHR) data and then leveraging this big data to improve nurse staffing and subsequent patient care outcomes may offer an important approach to provide evidence needed to reduce hospital costs and enhance care quality. The purpose of this study was to determine the association between nurse continuity and the presence of one never event, hospital-acquired pressure ulcers (HAPUs), using the Hands on Automated Nursing Data System (HANDS),7 an electronic nursing plan of care (POC) database, as the data source.

Background

Extensive research has demonstrated the influence of multiple individual nurse-staffing variables on patient outcomes. These studies were compiled in a recent state of the science review8 of 29 systematic or literature reviews. Significant associations between additional nursing care hours/higher proportion of RN care and patient outcomes including reductions in mortality,9-10 falls,9 and PUs9,11 have been documented. Kane et al.10 noted that reducing the number of patients cared for by each RN reduced the odds of nosocomial sepsis, cardiac arrest, and medical complications. Aiken et al.12 learned that use of an additional RN per patient day reduced the odds of dying within 30 days by more than one half. Additional associations included a richer RN skill mix (i.e., a higher RN-to-ancillary personnel ratio) and increased satisfaction13 and more RN education and experience, and improved mortality.14-16 Research studies describing the influence of shift length include some negative outcomes, with longer shift lengths being associated with increased errors17-18 and performance lags,19 with less nursing time spent in direct patient care as the shift lengthens.20 Finally, staff ‘churn’ (change in staffing with use of part-time, float, or agency staff) has been found to interfere with team functioning and care continuity21 with potential negative repercussions for patient outcomes.

There has been little research investigating the influence of nurse continuity on patient outcomes. Medical continuity has been examined in one study22 focused on patient outcomes associated with care continuity with primary care practitioners. Findings included improved patient satisfaction, and decreased hospitalizations and emergency room visits for patients with consistent primary care physicians.22 As profiled in the following sections, only three nursing studies examining nurse continuity were located, with only one considering adverse events.

Bostrom et al.23 noted increased satisfaction with aspects of patient care (e.g., patient and family involvement) when nurse continuity was studied in 116 medical-surgical patients discharged from one tertiary medical center. Nurse continuity was measured in terms of constancy and consistency, with constancy defined as the maximum number of consecutive days the patient was cared for by the same nurse.23 A consistency index for each shift (i.e., days, evenings, nights) was calculated using the number of shifts the patient was hospitalized divided by the number of care providers a patient had on a specific shift. Larger consistency index and constancy values, reflecting better nurse continuity, were associated with the increased satisfaction levels in this study.23

Russell et al.24 found reduced use of hospitals and emergent care, and improved function in the activities of daily living when studying home health care visits (N=59,854 patients) with consistent nurse care providers from one home care agency. Continuity of care was measured as consistency of nurse care providers across a series of home health care visits.24 Each patient was awarded a continuity score using a formula that calculated a score based on the number of care providers, the number of visits by each care provider, and the total number of visits for each patient with all of the care providers. Scores ranged from 0-1 with higher scores representing greater continuity in home health care services whereas lower scores indicated less continuity.24

Siow25 found in a secondary data analysis that greater levels of nurse continuity were associated with a longer length of stay and a greater number of ventilator days in one pediatric intensive care unit (N=332 patients). No significant associations between nurse continuity and adverse events or hospital-acquired infections were noted. Continuity was measured using an adapted version of Curley and Hickey’s Continuity of Care Index (CCI).26 Siow reversed the formula so that higher scores represented higher continuity (CCI=1–Total number of different nurses/Total number of shifts). The CCI scores ranged from a high of one representing strong continuity to zero for poor continuity.25

Despite nurse continuity once being a critical feature of the primary nursing model it has been infrequently studied, especially related to patient outcomes. This lack of study may be attributed to the inconsistent definition of continuity and the absence of reliable and valid evaluation measures that link continuity and patient outcomes. To help demonstrate the connection between nurse staffing and patient outcomes the EHR was used in this study as an untapped data source. The EHR can be a valuable big data source for data mining if it contains POC documentation coded in a way that allows easy access and is in a form amenable to analysis. In this study the HANDS database was used to examine the influence of nurse-staffing variables, including nurse continuity, and the prevention of one never event, HAPUs.

Recently a new conceptual model for studying the effect of nurse continuity and nurse-staffing variables on patient outcomes was proposed (Figure 1).27 The model depicts three relationships based on the hypothesis that nurse continuity is an integral nurse-staffing variable that can influence patient outcomes or moderate other nurse-staffing or unit environment variables leading to improved patient outcomes. It also shows a fourth relationship, specific to an examination of HAPUs, depicting the influence of patient characteristics on patient outcomes. PUs are a prevalent never event (5-10%)28 and a nurse-sensitive quality outcome. Multiple factors contribute to PUs including impairments of mobility, nutrition, and perfusion28 that are further affected by timely nursing assessment, monitoring, decision making, and interventions. Based on the model and controlling for patient characteristics, our study aim was to determine the influence of nurse continuity and nurse-staffing variables on the presence of HAPUs.

Figure 1.

Figure 1

Conceptual Model for Studying the Effect of Nurse Continuity on Patient Outcomes

Methods

Study Design

This comparative study was a secondary use of the HANDS database. This same database has been used to examine best practice findings related to pain29-30 and death anxiety31 outcomes for patients at the end of life and to document findings from usability studies that examine nurse preferences for the presentation of evidence as part of clinical decision support interfaces.32 This study was submitted to the Institutional Review Board (IRB) for review and was determined to not meet the definition of human subject research because the data are de-identified.

Setting

The HANDS is an electronic documentation tool with standardized nursing terminologies (SNTs) used by RNs to enter data each shift. The HANDS tracks a patient’s diagnoses (North American Nursing Diagnosis Association International [NANDA-I]),33 interventions (Nursing Interventions Classification [NIC]),34 and outcomes (Nursing Outcomes Classification [NOC]),35 patient demographics, and nurse characteristics.7 The HANDS is a valid and reliable database30,36 implemented as the nursing documentation system on nine units in four hospitals for 12 or 24 months. The hospitals included two large community, one university, and one small community settings with units comprised of medical-surgical and critical care patients.

Sample

The HANDS database contains 42,403 episodes of care (Table 1) defined as a continuous patient stay on a single hospital unit and consist of all care plans entered at admission, the end of each shift, and discharge.30 From this original HANDS dataset we extracted the 24,609 episodes that satisfied the following criteria: (1) no PUs present upon admission, (2) each episode associated with a unique patient (one HAPU episode was excluded in this process), (3) patients 18 years or older, and (4) episode lasted more than one day to allow a meaningful continuity measure. We also measured gaps in documentation, defined as missing care plans within a patient care episode, and found an 11% average gap. Given the low percentage of missing data, we used mean imputation in our analysis.

Table 1.

Number of Available Episodes with Pressure Ulcers by Unit (N = 42,403)

Hospital and Unit Number of
Nurses by
Unita
Total
Episodes
of Care by
Unit
Pressure
Ulcers
Present on
Admission
(APU)
Hospital-
Acquired
Pressure
Ulcers
(HAPUs)
LCH1b: General Medicine 113 5640 194 46
LCH1: Medical Intensive Care
   Unit (MICU)
54 1228 27 13
LCH1: Gerontology 80 9565 91 31
SCHc: Medicine 63 4881 1 1
LCH2d: Medicine 59 3432 16 6
LCH2: Gerontology 25 1673 41 13
UHe: Neuro Surgery 181 8216 2 3
UH: Cardiac Surgery 210 6112 59 59
UH: Intensive Care Unit
  (ICU)
66 1656 254 39
Total 787 42403 685 211
a

The sum of the number of nurses for all units is actually 851 due to nurses who worked on multiple units. 787 reflect the total # of unique nurses.

b

LCH1 = Large Community Hospital 1

c

SCH = Small Community Hospital

d

LCH2 = Large Community Hospital 2

e

UH = University Hospital

Measures

HAPUs

Structured query language (SQL) commands were developed to identify and extract all HAPU episodes in the HANDS dataset. The HAPU episodes were defined as all episodes in which the label NIC: Pressure Ulcer Care was placed on the patient’s POC at least 24 hours after admission to the unit. Episodes in which this NIC was placed on a patient’s POC within 24 hours of admission to the unit were considered as admission pressure ulcers (APU). This definition ensured that PUs present on admission to the unit that were identified and documented by the nurse within the first 24 hours of care were not included as HAPUs. A total of 896 pressure ulcers were located in the original database (N=42,403) with 685 classified as APU and 211 as HAPUs (Table 1).

Patient risk factors

Eighteen NANDA-I, 29 NOC, and 39 NIC labels (Table 2) were selected to identify patient characteristics that might predispose PUs. Two methods were used to elicit these labels. The first method examined common labels for the patient characteristics using the NANDA-I to NOC and NOC to NIC linkages available in the HANDS system and in the NNN literature.33-35 Next, a list of all NNN labels present on the POCs for the 896 PU episodes were reviewed to narrow down the most consistently used labels for these patient characteristics. The final list of 86 NNN labels (Table 2) was validated by a clinical nurse expert with extensive experience and research in the use of SNTs. These labels fall under seven distinct categories: Nutrition, Mobility, Hydration, Continence, Skin, Perfusion, and Cognition, each representing a factor suggesting PU vulnerability. For each episode, we examined the admission POC and extracted the relevant NOC ratings to indicate the patient condition in each of these categories.

Table 2.

NANDA-Ia, NOCb, and NICc (NNN)d Labels Used to Identify Patient Characteristics

Category NANDA-I NOC NIC
Continence
  • Impaired Urinary Elimination

  • Bowel Incontinence

  • Diarrhea

  • Bowel Elimination

  • Bowel Continence

  • Urinary Continence

  • Urinary Elimination

  • Urinary Incontinence Care

  • Self-Care Assistance: Toileting

  • Diarrhea Management

  • Urinary Elimination Management

  • Bowel Incontinence Care

  • Bowel Management

Mobility
  • Impaired Bed Mobility

  • Impaired Physical Mobility

  • Mobility

  • Body Positioning: Self-initiated

  • Neurological Status: Cranial Sensory/Motor Function

  • Immobility Consequences: Physiological

  • Positioning

  • Positioning: Wheelchair

  • Pressure Management

  • Bed Rest Care

  • Positioning: Neurologic

Nutrition
  • Imbalanced Nutrition: Less Than Body Requirements

  • Nutritional Status

  • Nutritional Status: Energy

  • Nutritional Status: Nutrient Intake

  • Nutritional Status: Food and Fluid Intake

  • Nutritional Status: Biochemical Measures

  • Nutritional Monitoring

  • Enteral Tube Feeding

  • Nutrition Management

  • Total Parenteral Nutrition Administration

  • Nutrition Therapy

Hydration
  • Deficit Fluid Volume

  • Risk for Deficit Fluid Volume

  • Risk for Imbalanced Fluid Volume

  • Hydration

  • Electrolyte & Acid/Base Balance

  • Fluid Balance

  • Electrolyte Management

  • Fluid Management

  • Intravenous (IV) Therapy

  • Hypovolemia Management

  • Electrolyte Management: Hypernatremia

  • Fluid and Electrolyte Management

  • Fluid Resuscitation

  • Electrolyte Monitoring

  • Fluid Monitoring

Perfusion
  • Ineffective Tissue Perfusion: Peripheral

  • Risk for Peripheral Neurovascular Dysfunction

  • Neurological Status: Cranial Sensory/Motor Function

  • Tissue Perfusion: Peripheral

  • Circulation Status

  • Cardiac Pump Effectiveness

  • Circulatory Care: Arterial Insufficiency

  • Lower Extremity Monitoring

  • Hemodynamic Regulation

  • Invasive Hemodynamic Monitoring

  • Circulatory Precautions

  • Circulatory Care: Venous Insufficiency

Cognition
  • Disturbed Sensory Perception: Tactile

  • Disturbed Thought Processes

  • Acute Confusion

  • Chronic Confusion

  • Neurological Status: Consciousness

  • Information Processing

  • Cognitive Orientation

  • Communication

  • Communication: Expressive

  • Distorted Thought Self Control

  • Tissue Perfusion: Cerebral

  • Cognition

  • Neurological Status

  • Communication Enhancement: Speech Deficit

  • Dementia Management

  • Reality Orientation

  • Delirium Management

  • Sedation Management

  • Delusion Management

Skin
  • Impaired Skin Integrity

  • Risk for Impaired Skin Integrity

  • Impaired Tissue Integrity

  • Skin Surveillance

  • Pressure Ulcer Prevention

a

NANDA-I = North American Nursing Diagnosis Association-International

b

NOC = Nursing Outcomes Classification

c

NIC = Nursing Interventions Classification

d

NNN = NANDA-I, NOC, NIC

Nurse staffing

Nurse-staffing variables were operationalized using the raw data also available in the HANDS dataset (Table 3). For the HAPU episodes nurse-staffing variables were examined on the shifts leading up to the ulcer occurrence. The variable nurse continuity was calculated using the total number of consecutive days worked by each RN with the patient and operationalized as the percent of consecutive care days by the same/single RNs in a patient episode. For our calculation (Figure 2) we computed the episode continuity index by dividing the total number of consecutive days (from all RNs) by the total number of possible consecutive days excluding the first day. Nurse experience was calculated using the number of years the RNs were in the nursing profession and operationalized as the percent of time a patient was cared for by RNs with at least two years of experience. The RN education variable was determined by the highest nursing degree reported and operationalized as the percent of time a patient was cared for by RNs with a BSN or greater. The shift length variable was calculated using the number of consecutive hours worked by each RN with the patient and operationalized as the percent of 8-hour RN care shifts in a patient episode. The RN work pattern variable (full, part, or very part-time status) was calculated using the actual hours worked as a fraction of full-time status (80 hours over two weeks) for each RN caring for the patient in an episode. Very part-time status was defined as any nurse who worked less than or equal to 24 hours (.3) per pay period (pay period=80 hours over two weeks) with part-time to full-time status classified as greater than .3–1.0. The RN work pattern variable was operationalized as the percent of very part-time RNs caring for a patient. To determine the average patient-to-nurse ratio across a patient episode, we computed for each four-hour window the patient-to-nurse ratio on the unit and averaged the ratios of all windows spanned by the patient episode. Finally, worked hours per patient days (whppd) were determined by calculating an average whppd over the course of a patient care episode. The whppd definition used in this study was the total number of RN hours on a unit in a 24-hour period divided by the number of patients on that unit at midnight.

Table 3.

Raw Data in HANDS and Operationalized Definitions for Patient Characteristics, Nurse Staffing, and Continuity Variables

Variable Raw Data Found in HANDS Operationalized Definition
Nurse Staffing Variables
Shift Length Number of consecutive hours
worked by each registered nurse
(RN) during a care episode.
% of 8-hour RN care shifts in a
patient episode.
RN Work Pattern (Shifts
of Care by Part Time
[PT]/Full Time [FT] vs.
Very Part Time [VPT]
Workers)
Fraction of time status (FT, PT, VPT)
for each RN who cared for the
patient during a care episode.
% of care shifts by very part
time status RNs (0.3 [24
hours] or less]) in a patient
episode.

Nurse Experience Years of experience as an RN. % of time cared for by RNs
with ≥ to 2 years of
experience in a patient
episode.
Nurse Education Diploma, ADN,a BSN,b BSN and some
additional coursework, Master’s
degree in nursing, or Doctoral
degree in nursing.
% of time cared for by RNs
with a BSN or greater in a
patient episode.
Patient-to-Nurse Ratio Actual number of patients cared for
by a single RN during a shift.
The average patient-to-nurse
ratio over the course of the
patient care episode
Worked hours per
patient day (whppd)
The total number of RN hours on a
unit in a 24-hour period divided by
the number of patients on that unit
at the midnight census.
The average whppd over the
course of the patient care
episode

Nurse Continuity
Variable
Number of consecutive
days cared for by the
same/single RNs
The total number of consecutive
care days worked by each RN with
the patient during a care episode.
% of consecutive care days by
the same/single RNs in a
patient episode.

Patient Characteristics
Nutrition 1 NANDA-I,c 5 NOC,d 5 NICe Nutrition
labels
NNNf Nutrition label
appearing on the admission
POCg in the patient’s episode
Continence 3 NANDA-I, 4 NOC, 6 NIC Continence
labels
NNN Continence label
appearing on at the
admission POC in the
patient’s episode
Hydration 3 NANDA- I, 3 NOC, 9 NIC Hydration
labels
NNN Hydration label
appearing on the admission
POC in the patient’s episode
Mobility 2 NANDA- I, 4 NOC, 5 NIC Mobility
labels
NNN Mobility label appearing
on the admission POC in the
patient’s episode
Perfusion 2 NANDA-I, 4 NOC, 5 NIC Perfusion
labels
NNN Perfusion label
appearing on the admission
POC in the patient’s episode
Cognition 4 NANDA-I, 9 NOC, 6 NIC Cognition
labels
NNN Cognition label
appearing on the admission
POC in the patient’s episode
Skin 3 NANDA-I, 2 NIC Skin Labels NNN Skin label appearing on
the admission POC in the
patient’s episode
Age Age in years Age in years
a

ADN = Associate Degree in Nursing

b

BSN = Bachelor of Science in Nursing

c

NANDA-I = North American Nursing Diagnosis Association – International

d

NOC = Nursing Outcomes Classification

e

NIC = Nursing Interventions Classification

f

NNN = NANDA-I NOC NIC, POC

g

POC = Plan of Care

Figure 2.

Figure 2

Continuity Index Calculation

Analyses

HANDS database structure

The HANDS POC documentation method was implemented and tested in four hospitals over 12–24 months from 2005-2008.36 Seven hundred and eighty seven unique RNs documented the care they provided using HANDS for either one (4 units) or two (5 units) years creating a dataset of 42,403 episodes of care. The HANDS dataset resides in a relational database consisting of 89 tables and 747 columns of data.7

HANDS data entry process

A training program with unit champions oriented all nurse users to the HANDS method and nurses were competency validated on creating admission care plans as well as updating care plans using NANDA-I, NOC, and NIC (NNN) terminologies.36 The HANDS documentation begins with the admission POC for an episode submitted at the first nursing handoff.36 This POC includes nursing diagnoses, outcomes, and interventions (delineated with NNN labels) identified during the initial shift. The subsequent care plans build on the initial plan and include any additions, deletions, or resolutions that occur during the time period since the last POC submission. The current status of all POC NOC outcomes are rated (or re-rated) at each handoff. When a NOC outcome label is first added to a POC an expected NOC rating (goal at discharge from a unit) is also identified. The entry of a current NOC status rating at each handoff and the expected NOC outcome rating make it possible to gauge progress toward meeting outcome goals during an episode. All NOC outcomes are scored on a rating scale ranging from 1-5, with a 5 representing the best outcome.36 The reliability and validity of the NOC outcomes ratings were established in prior studies.37-38

Data processing to identify a matched sample for the analytic dataset

Cluster mining was used to identify the matched control group for the analytic dataset. Clustering is a machine learning algorithm that groups observations into multiple clusters, with each cluster different from another, but members within each cluster consist of observations of similar characteristics. The objective was to identify matched controls for HAPU episodes in the database. Thus, clustering was an ideal tool to group episodes with similar PU risk factors together.

To utilize clustering to accomplish our goal, we defined a similarity measure based on variables predictive of PUs, including NOC ratings for seven patient characteristics (Perfusion, Skin, Nutrition, Mobility, Hydration, Cognition, Continence), patient age, and hospital unit. Variables were weighted using a Weight by Uncertainty operator, which assigns higher weights to variables that are more predictive (and thus reduce more uncertainty) of the HAPU outcome to ensure that the similarity measure closely reflects the similarity in HAPU risks between episodes. A range of cluster solutions was examined to identify a solution with no significant imbalance between the HAPU and control episodes. For each cluster solution, we randomly selected for each HAPU episode three matched control episodes from the same cluster but without PUs. Our goal was to locate a cluster solution with clusters that contained highly similar members based on the NOC ratings for the patient characteristics, patient age, and/or units that either led to HAPUs or did not lead to HAPUs (matched controls). As the number of clusters increased the episodes within each cluster should become more similar allowing us to achieve a better balance between the control and HAPU cases.

For example, in a 5-cluster solution the episodes were clustered primarily by unit and there were significant differences in the mean NOC outcome ratings for 5 out of 7 patient characteristics and age comparing HAPUs and controls. By 18 clusters, the differences in the mean NOC outcome ratings for some of the patient characteristics and age had decreased when comparing the HAPU episodes and matched controls (e.g., the p values for the perfusion characteristic t tests went from 0.00 in the 5-cluster solution to 0.43 in the 18-cluster solution); however, there remained significant differences in patient age as well as 3 out of 7 characteristics between the HAPU and control episodes. With a 200-cluster solution we achieved our goal of no statistically significant differences in HAPU versus matched control episodes noted with the seven patient characteristics and patient age as measured by the independent t test. There was also no difference in the distribution of HAPU and non-HAPU episodes across the nine clinical units (Fisher’s Exact Test, p=1.0). The 630 matched control cases selected from this cluster solution and the 210 HAPU cases comprised the analytic dataset for the regression analysis.

Aim Analyses

Analyses to address the study aim included descriptive statistics (frequencies, means, and standard deviations) to enhance understanding of the patient characteristics associated with HAPUs in the HANDS database. Frequencies were also calculated for the nurse-staffing variables by patient unit. A logistic regression using the STATA 12 statistical software package was performed regressing HAPU outcomes against nurse continuity and six nurse-staffing variables (whppd, patient-to-nurse ratio, RN experience, RN education, shift length, RN work pattern [number of shifts cared for by very part-time versus part/full-time staff]). A second logistic regression introduced interaction terms between nurse continuity and each nurse-staffing variable to determine whether nurse continuity had a moderating effect, enhancing the influence of other nurse-staffing variables on HAPU outcomes.

Results

Within the reduced dataset the average age of patients with HAPUs was 68.9 (15.3) years as compared to 65.1 (18.2) years for episodes without (Table 4). The mean admission NOC outcome ratings were higher for all seven patient characteristics in the non-HAPU versus HAPU episodes. Results were statistically significant at a p<.001 for all but the cognition and continence patient characteristics (Table 4). Study analyses were then conducted on a subset of HAPU episodes (n=210) with matched controls (n=630) derived from the reduced dataset as described in the measures and analysis sections. With our sample of 840 episodes (including 210 HAPU episodes) and a two-sided significance level of 0.05, we detected with 80% power the continuity effect size that was 0.22 or larger.

Table 4.

Patient Characteristics and HAPUa vs. Non-HAPU Episodes in the Reduced Dataset (N = 24,609)

Patient
Characteristics
HAPUs
Mean of NOC bon
Admission (S.D.)c
Non-HAPUs
Mean of NOC on
Admission (S.D.)
t test
P value
Nutrition 4.35 (1.27) 4.62 (0.94) 0.00
Mobility 4.10 (1.25) 4.49 (0.98) 0.00
Cognition 4.57 (1.02) 4.64 (0.85) 0.26
Perfusion 3.95 (1.29) 4.49 (0.92) 0.00
Continence 4.83 (0.66) 4.88 (0.54) 0.20
Hydration 4.12 (1.17) 4.60 (0.86) 0.00
Skin 4.10 (1.29) 4.47 (1.01) 0.00
Age 68.9 (15.3) 65.1 (18.2) 0.00
a

HAPU = Hospital-Acquired Pressure Ulcer

b

NOC = Nursing Outcomes Classification

c

S.D. = Standard Deviation

The overall continuity index for the reduced dataset was low, 0.32 (SD=0.35) (0 to 1.0 possible) with the lowest values on the ICU and neuro units (0.21, SD=0.21) and the highest (0.42, SD=0.24) on the gerontology unit in the large community hospital (Table 5). Comparing the HAPU and the matched control episodes, HAPU episodes had a lower percentage of BSN nurses and a lower patient-to-nurse ratio, but a higher percentage of eight-hour shifts, VPT nurses, experienced nurses, whppd, and continuity. None of these differences were statistically significant (Table 6). Over 50% of the RNs in the dataset had a BSN, more than 75% of the RNs had greater than 2 years of experience, and just over 40% of the nurses worked 8-hour shifts.

Table 5.

Continuity Index by Unit in the Reduced Dataset (N = 24,609)

Continuity
Mean (SD)a Median (IQR)b
Overall 0.32 (0.25) 0.33 (0.14-0.50)
LCH1c: Med 0.32 (0.23) 0.32 (0.17-0.50)
LCH1:ICU 0.31 (0.26) 0.29 (0.08-0.43)
LCH1:Gerontology 0.42 (0.24) 0.43 (0.29-0.54)
SCHd: Med 0.37 (0.29) 0.33 (0.13-0.50)
LCH2e:Med 0.32 (0.21) 0.33 (0.20-0.44)
LCH2:Gerontology 0.35 (0.21) 0.33 (0.22-0.50)
UHf:Neuro 0.21 (0.22) 0.18 (0.00-0.33)
UH:ICU 0.21 (0.21) 0.18 (0.00-0.33)
UH:Cardiac 0.28 (0.22) 0.25 (0.14-0.40)
a

SD = Standard Deviation

b

IQR = Interquartile Range

c

LCH1 = Large Community Hospital 1

d

SCH = Small Community Hospital

e

LCH2 = Large Community Hospital 2

f

UH = University Hospital

Table 6.

Nurse Staffing Characteristics: HAPUa vs. non-HAPU Episodes in the Analytic (N=840) Dataset

HAPU non HAPU p value

BSNb 53.3% (23.0%) 54.1% (22.3%) .66
Experienced 75.3% (21.5%) 75.1% (23.2%) .93
8-hour Shifts 44.9% (30.3%) 43.5% (32.3%) .57
Workload 3.63 (1.46) 3.67 (1.33) .72
Whppdc 8.32 (3.19) 8.25 (3.15) .79
%VPTd 3.9% (6.3%) 3.5% (6.5%) .37
Continuity 0.30 (0.22) 0.29 (0.22) .51
a

HAPU = Hospital-Acquired Pressure Ulcer

b

BSN = Bachelor of Science in Nursing

c

Whppd = Worked Hours per Patient Day

d

%VPT = Percent Very Part Time

The regression analysis revealed that continuity was not significantly associated with the HAPU outcome both when it was the only predictor (p=.50) and when we controlled for other nurse-staffing variables (p=.37) (Table 7). Similarly, none of the nurse-staffing variables entered in the logistic regression reached statistical significance with HAPU outcomes. In a second logistic regression that introduced six interaction terms between nurse continuity and each nurse-staffing variable no statistically significant relationships with HAPU outcomes were noted.

Table 7.

Regressing Nurse Continuity and Nurse Staffing Variables on Hospital-Acquired Pressure Ulcers (HAPUs) (N = 840)a

Outcome Predictor Coefficient SE z P > [z]
HAPUs Nurse
Continuity
.35 .39 0.90 0.37
Experience −.09 .40 −0.23 0.82
BSNb −.14 .37 −0.38 0.70
Load (patient-
to-nurse ratio)
−.06 .12 −0.47 0.64
Eight-hour shifts .22 .30 0.73 0.46
Very part-time 1.09 1.24 0.88 0.38
Whppdc −.00 .05 −0.02 0.98
a

Analytic dataset contains episodes with and without HAPUs (matched controls) using the patient characteristics, units, and patient age as the similarity measure

b

BSN = Bachelor of Science in Nursing

c

Whppd = Worked hours per patient day

Discussion

Nurse continuity is an understudied nurse-staffing variable in part due to difficulties in defining and measuring the concept. Consequently, research examining the influence of continuous nurse care providers on patient outcomes is very limited. In this study we used the HANDS, an EHR with linked data to operationalize and measure nurse continuity. Our continuity definition was based on consecutive nurse care days and examined the influence of nurse continuity and several other nurse-staffing variables on HAPUs. None of the nine units participating in our study had good continuity in practice with unit-wide average continuity ranging from very poor (0.21) to poor (0.42). We were able to validate that nutrition, mobility, perfusion, hydration, and skin problems on admission as well as patient age were associated with HAPU development (p<.001). However, we were not able to demonstrate statistically significant relationships between continuity and HAPUs or with nurse continuity as a moderator of other nurse-staffing variables and HAPU development. Combining these findings with our power analysis our study results indicate that the effect of nurse continuity and other nurse-staffing variables on HAPU outcomes over a patient episode are likely small. Patient characteristics such as nutrition, mobility, and perfusion on the other hand were found to be clearly associated with HAPUs. One study implication is that to reduce the potential for HAPU never events the preferred strategies may be ones that focus on prevention (good skin care, alternative bed surfaces) and aggressive intervention to address patient characteristics that contribute to skin breakdown (poor nutrition, impaired mobility) rather than adjustments to nurse staffing.

A second important finding was that we found a wide variation between good and poor continuity on each of the nine care units as noted by our standard deviation calculations. This finding might reflect individual nurse efforts to direct their assignments to achieve continuity with particular patients, a strategy which can lead to increased nurse satisfaction, retention and better patient outcomes.39 While this study did not reveal a significant relationship between nurse continuity and the development of HAPUs a future study using the HANDS database might allow us to examine the influence of continuous nurse staffing on other outcomes variables such as nurse turnover and nurse satisfaction or patient satisfaction. Another future study might consider drilling deeper into hospital-acquired adverse outcomes by looking at the characteristics of an individual nurse and the staffing on the actual shift in which a HAPU or a fall occurred. The availability of nurse-staffing variables, unit environment characteristics, and patient outcomes data in HANDS make these additional queries possible.

A third notable finding is that it took 200 clusters before we were able to identify a cluster solution with no significant imbalance between the HAPU and matched control episodes. Our goal was to locate a cluster solution with each individual cluster different from the next one but within each cluster members that were highly similar based on the NOC ratings for the patient characteristics, patient age, and/or units that either led or did not lead to HAPUs (matched controls). The fact that there were 200 different clusters in our solution, each made up of a particular combination of patient characteristics, patient age, and/or units, reflects the highly diverse profile of vulnerability that leads to a HAPU. The implication of this finding is that the one size fits all standardized care bundles now commonly used for many nurse-sensitive outcomes (i.e., ventilator associated pneumonia, catheter associated urinary tract infections) may not be the most effective strategy to address the complexity of this patient condition. Rather a case-by-case analysis of patient vulnerabilities with a customized intervention strategy may be the most optimal approach for addressing this unique and persistent hospital-acquired event.

This study was ground breaking for its use of an EHR big data set containing nursing POC documentation as a means to inform about a nurse-sensitive patient outcome. Historically use of nursing POC documentation in medical records for research inquiries would have been futile as documentation has been incomplete and of poor quality.40-41 In one pre- and post-EHR implementation study only 28% (N=20/71 records) demonstrated full documentation of a nursing problem, interventions, and outcomes using the electronic system.41 In a second study of 161 records from a health care center, nursing POCs were rarely found and if present none of the plans included complete documentation of a problem, interventions, and outcomes.40 However, for this study, the HANDS POC documentation system was uniquely available to examine nurse staffing and patient outcomes with a remarkable 78-92% compliance rate at shift hand-off for a 12 to 24 months of use period.36 This database allowed a specific examination of a HAPU outcome linking it back to the nurse staffing for patient care over the course of a hospital episode. Though the National Database for Nursing Quality Indicators (NDNQI) is available and frequently used to comparatively display nursing quality indicators across hospital units, at this juncture that database is unable to connect patient specific outcomes with individual nurse characteristics. The availability of the HANDS POC system and our use of cluster mining uniquely allowed us to identify patients with HAPUs in the dataset as well as to operationalize the characteristics of the nurse caregivers over the course of the episode. We were thus able to drill down into readily available nursing POC EHR data to examine such questions as whether the percentage of experienced nurses or the average patient load influenced PU development. This accomplishment represents an important advance for nursing science in the big data era.

Limitations

There were several study limitations that may have affected the significance of our findings. For example, we recognize that HAPUs physiologically develop over time but manifest on a single shift. Our analysis did not allow us to determine if the care was inadequate on one particular shift or whether insufficient nurse staffing over the course of several shifts contributed to the PU vulnerability. For our next analysis we will examine the shift in which a HAPU occurred and examine the characteristics of the individual nurse caregiver including his/her continuity with the patient on prior shifts or in previous days. A second limitation is that hospital never events, though an important nurse-sensitive indicator, occur infrequently, making it more challenging to demonstrate a significant relationship between nurse staffing and patient outcomes. We had sufficient power to detect a small effect (.22) of nurse continuity on HAPU development after controlling for the other factors influential in HAPU development. In this study, however, we lacked power to detect very small effects of nurse continuity, those smaller than .22. A further limitation included our secondary use of a dataset that restricted inquiries to the available data and did not include the entire EHR. For example, the HANDS dataset had only RN POC documentation omitting the contributions of assistive nursing personnel.

Two final study limitations may have been our continuity and pressure ulcer definitions. As noted in the literature review, nurse continuity was previously studied using consistency indices looking at the number of repeat days by the same nurses over the hospitalization.23, 25 It was noted early in our inquiry that this type of continuity measure was confounded by length of stay (LOS) and so we created a new definition that defined continuity as the number of consecutive care days by the same RN. If an RN missed a consecutive care day but was assigned to the patient again at a later point in the episode that day was not weighted in our definition. In retrospect this definition might not have been broad enough and in light of our current short hospital LOS may have increased our difficulty in discerning an influence from continuity.

This study also used a very conservative PU definition (prevalence = 0.7% versus 5-10%28 in published research) as we selected a single label, NIC: Pressure Ulcer Care, to increase our certainty that included cases reflected only HAPUs. Expanding our definition to include other labels in the HANDS database such as NIC: Pressure Management would have increased our case count by 1,000 (prevalence = 2.1%) but without further study it is unclear that this less specific term applies only to HAPUs. We acknowledge the possibility that PU cases may have existed and gone unrecognized in our matched controls due to our conservative definition.

Conclusions

In this study we demonstrated use of a large EHR database, the HANDS, as an innovative data mining source for practice-based research. We set out to measure and determine the influence of nurse continuity, an understudied nursing-staffing variable, on the prevention of a common never event, HAPUs. Our initial findings suggest that nurse continuity and other nurse-staffing variables may not be as influential in HAPU development as previously thought. However, despite the absence of significant findings in this first study, we are encouraged by our study method to continue examining patient outcomes influenced by nurse staffing, made possible by the standardized clinical practice data in the HANDS database. We believe that these studies are urgently needed so that hospital administrators can make informed decisions when reorganizing health care delivery systems, thus securing safe, high quality care for all hospitalized patients.

Acknowledgments

Funding: This research was made possible by an RO1 NR012949 grant from the National Institutes of Health, National Institute of Nursing Research and a 1R36HS023072-01 grant from the Agency for Healthcare Research and Quality (AHRQ). Its contents are solely the responsibility of the authors and do not necessarily reflect the official views of the National Institute for Nursing Research or the Agency for Healthcare Research and Quality.

Footnotes

Conflict of interest: The HANDS software, which includes the NANDA-I, NIC, and NOC standardized nursing terminologies, is owned and distributed by HealthTeam IQ, LLC. Dr. Gail Keenan is currently the President and CEO of this company and has a current conflict of interest statement of explanation and management plan in place with the University of Illinois at Chicago.

References

  • 1.U. S. Department of Health and Human Services. Agency for Healthcare Research and Quality Patient safety network patient safety primers: Never events. 2013 Jan 21; Retrieved from: http://www.psnet.ahrq.gov/primer.
  • 2.U. S. Department of Health and Human Services. Office of Inspector General Hospital incident reporting systems do not capture most patient harm. 2012 Retrieved from https://oig.hhs.gov.
  • 3.Shreve JM, Van Den Bos J, Gray T, Halford M, Rustagi K, Ziemkiewicz E. The economic measurement of medical errors. The Society of Actuaries Health Section and Milliman, Inc.; 2010. [Google Scholar]
  • 4.American Nurses Association (ANA) Code of ethics for nurses. ANA; Silver Spring, MD: Jul 7, 2014. Retrieved from http://www.nursingworld.org/ethics/code/protect.htm. [Google Scholar]
  • 5.Bolton LB, Aydin CE, Donaldson N, Brown DS, Sandhu M, Fridman M, Aronow HU. Mandated nurse staffing ratios in California: A comparison of staffing and nursing-sensitive outcomes pre- and post-regulation. Policy, Politics, & Nursing Practice. 2007;8(4):238–250. doi: 10.1177/1527154407312737. [DOI] [PubMed] [Google Scholar]
  • 6.Mark BA, Harless DW, McCue M, Xu Y. A longitudinal examination of hospital registered nurse staffing and quality of care. Health Services Research. 2004;39(2):279–300. doi: 10.1111/j.1475-6773.2004.00228.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Keenan GM, Tschannen D, Wesley ML. Standardized nursing terminologies can transform practice. Journal of Nursing Administration. 2008;38(3):103–106. doi: 10.1097/01.NNA.0000310728.50913.de. [DOI] [PubMed] [Google Scholar]
  • 8.Brennan CW, Daly BJ, Jones KR. State of the science: The relationship between nurse staffing and patient outcomes. The Western Journal of Nursing Research. 2013;35:760–794. doi: 10.1177/0193945913476577. [DOI] [PubMed] [Google Scholar]
  • 9.Blegen MA, Goode CJ, Spetz J, Vaughn T, Park SH. Nurse staffing effects on patient outcomes: Safety-net and non-safety-net hospitals. Medical Care. 2011;49:406–414. doi: 10.1097/MLR.0b013e318202e129. [DOI] [PubMed] [Google Scholar]
  • 10.Kane RL, Shamliyan TA, Mueller C, Duval S, Wilt TJ. The association of registered nurse staffing levels and patient outcomes: Systematic review and meta-analysis. Medical Care. 2007;45:1195–1204. doi: 10.1097/MLR.0b013e3181468ca3. Retrieved from http://www.jstor.org.proxy.cc.uic.edu. [DOI] [PubMed] [Google Scholar]
  • 11.Sovie MD, Jawad AF. Hospital restructuring and its impact on outcomes. Journal of Nursing Administration. 2001;31:588–600. doi: 10.1097/00005110-200112000-00010. Retrieved from http://ovidsp.txovid.com.proxy.cc.uic.edu. [DOI] [PubMed] [Google Scholar]
  • 12.Aiken LH, Sloane DM, Lake ET, Sochalski J, Weber AL. Organization and outcomes of inpatient AIDS care. Medical Care. 1999;37:771–787. doi: 10.1097/00005650-199908000-00006. Retrieved from http://www.jstor.org.proxy.cc.uic.edu. [DOI] [PubMed] [Google Scholar]
  • 13.Seago JA, Williamson A, Atwood C. Longitudinal analyses of nurse staffing and patient outcomes. The Journal of Nursing Administration. 2006;36(1):13–21. doi: 10.1097/00005110-200601000-00005. Retrieved from http://web.ebscohost.com.proxy.cc.uic.edu. [DOI] [PubMed] [Google Scholar]
  • 14.Aiken LH, Clarke SP, Cheung RB, Sloane DM, Silber JH. Educational levels of hospital nurses and surgical patient mortality. Journal of the American Medical Association. 2003;290(12):1617–1623. doi: 10.1001/jama.290.12.1617. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Estabrooks C, Midodzi WK, Cummings GG, Ricker K, Giovannetti P. The impact of hospital nursing characteristics on 30-day mortality. Nursing Research. 2005;54(2):74–84. doi: 10.1097/00006199-200503000-00002. Retrieved from http://ovidsp.tx.ovid.com.proxy.cc.uic.edu. [DOI] [PubMed] [Google Scholar]
  • 16.Tourangeau AE, Doran DM, McGillis Hall L, O’Brien Pallas L, Pringle D, Tu JV, Cranley LA. Impact of hospital nursing care on 30-day mortality for acute medical patients. Journal of Advanced Nursing. 2007;57(1):32–44. doi: 10.1111/j.1365-2648.2006.04084.x. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/17184372. [DOI] [PubMed] [Google Scholar]
  • 17.Rogers AE, Hwang WT, Scott LD, Aiken LH, Dinges DF. The working hours of hospital staff nurses and patient safety. Health Affairs. 2004;23:202–212. doi: 10.1377/hlthaff.23.4.202. Retrieved from http://search.proquest.com.proxy.cc.uic.edu. [DOI] [PubMed] [Google Scholar]
  • 18.Scott LD, Rogers AE, Shwant WT, Zhang Y. Effects of critical care nurses’ work hours on vigilance and patients’ safety. American Journal of Critical Care. 2006;15(1):30–37. Retrieved from http://web.ebscohost.com.proxy.cc.uic.edu. [PubMed] [Google Scholar]
  • 19.Fitzpatrick JM, While AE, Roberts JD. Shift work and its impact upon nurse performance: Current knowledge and research issues. Journal of Advanced Nursing. 1999;29(1):18–27. doi: 10.1046/j.1365-2648.1999.00861.x. Retrieved from http://web.ebscohost.com.proxy.cc.uic.edu. [DOI] [PubMed] [Google Scholar]
  • 20.Reid N, Robinson G, Todd C. The quantity of nursing care on wards working 8- and 12- hour shifts. International Journal of Nursing Studies. 1993;30:403–413. doi: 10.1016/0020-7489(93)90050-5. Retrieved from www.sciencedirect.com.proxy.cc.uic.edu. [DOI] [PubMed] [Google Scholar]
  • 21.Duffield C, Roche M, O’Brien-Pallas L, Catling-Paull C. The implications of staff ‘churn’ for nurse managers, staff, and patients. Nursing Economic$ 2009;27:103–110. Retrieved from http://web.ebscohost.com.proxy.cc.uic.edu. [PubMed] [Google Scholar]
  • 22.Van Walraven C, Oake N, Jennings A, Forster AJ. The association between continuity of care and outcomes: A systematic and critical review. Journal of Evaluation in Clinical Practice. 2010;16:947–956. doi: 10.1111/j.1365-2753.2009.01235.x. [DOI] [PubMed] [Google Scholar]
  • 23.Bostrom J, Tisnado J, Zimmerman J, Lazar N. The impact of continuity of nursing care personnel on patient satisfaction. Journal of Nursing Administration. 1994;24(10):64–68. doi: 10.1097/00005110-199410000-00012. [DOI] [PubMed] [Google Scholar]
  • 24.Russell D, Rosati RJ, Rosenfeld P, Marren JM. Continuity in home health care: Is consistency in nursing personnel associated with better patient outcomes? Journal for Healthcare Quality: Promoting Excellence in Healthcare. 2011;33(6):33–39. doi: 10.1111/j.1945-1474. [DOI] [PubMed] [Google Scholar]
  • 25.Siow KCE. Doctoral dissertation. University of Pennsylvania; 2012. Impact of continuity in nursing care on patient outcomes in the pediatric intensive care unit. Retrieved from http://repository.upenn.edu/dissertations/AAI3509486. [Google Scholar]
  • 26.Curley MAQ, Hickey PA. The Nightingale metrics. American Journal of Nursing. 2006;106(10):66–70. doi: 10.1097/00000446-200610000-00036. Retrieved from: http://ovidsp.tx.ovid.com.proxy.cc.uic.edu. [DOI] [PubMed] [Google Scholar]
  • 27.Stifter J, Yao Y, Dunn Lopez K, Khokhar A, Wilkie DJ, Keenan GM. Defining a new conceptual model and an innovative approach for measuring the influence of nurse continuity on patient outcomes. 2014. Manuscript submitted for publication.
  • 28.Anders J, Heinemann A, Leffmann C, Leutenegger M, Profener F, von Rentein-Kruse W. Decubitus ulcers – Pathophysiology and primary prevention. Deutsches Arzteblatt International. 2010;107:371–382. doi: 10.3238/arztebl.2010.0371. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Al-Masalha F, Xu D, Keenan GM, Khokhar A, Yao Y, Chen YC, Wilkie DJ. Data mining nursing care plans of end-of-life patients: A study to improve healthcare decision making. International Journal of Nursing Knowledge. 2013;24:15–24. doi: 10.1111/j.2047-3095.2012.01217.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Yao Y, Keenan GM, Al-Masalha F, Dunn-Lopez K, Khokhar A, Johnson A, Ansari R, Wilkie DJ. Current state of pain care for hospitalized patients at end of life. The American Journal of Hospice and Patient Care. 2013;30(2):128–136. doi: 10.1177/1049909112444458. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Lodhi MK, Cheema UI, Stifter J, Wilkie DJ, Keenan GM, Yao Y, Khokhar AA. Death anxiety in hospitalized end-of-life patients as captured from a structured electronic health record: Differences by patient and nurse characteristics. Research in Gerontological Nursing. 2014;7:224–234. doi: 10.3928/19404921-20140818-01. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Febretti A, Dunn Lopez K, Stifter J, Johnson AE, Keenan GM, Wilkie DJ. A component-based evaluation protocol for clinical decision support interfaces. In: Marcus A, editor. Design, user experience, and usability design philosophy, methods, and tools. Springer; Berlin Heidelberg: 2013. pp. 232–241. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.NANDA International . Nursing diagnoses: Definition and classification 2003-2004. NANDA International; Philadelphia, PA: 2003. [Google Scholar]
  • 34.McCloskey Dochterman JC, Bulechek GM. Nursing interventions classification (NIC) Mosby; St. Louis, MO: 2004. [PubMed] [Google Scholar]
  • 35.Moorhead S, Johnson M, Maas M. Iowa outcomes project. Nursing outcomes classification (NOC) Mosby; St. Louis, MO: 2004. [Google Scholar]
  • 36.Keenan GM, Yakel E, Yao Y, Xu D, Szalacha L, Tschannen D, Wilkie DJ. Maintaining a consistent big picture: Meaningful use of a web based POC EHR system. International Journal of Nursing Knowledge. 2012;23(3):119–123. doi: 10.1111/j.2047-3095.2012.01215.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Keenan G, Barkauskas V, Johnson M, Maas M, Moorhead S, Reed D. Establishing the validity, reliability, and sensitivity of NOC in adult care nurse practitioners clinics. Outcomes Management. 2003;7(7):74–83. Retrieved from http://ovidsp.txovid.com.proxy.cc.uic.edu. [PubMed] [Google Scholar]
  • 38.Keenan G, Stocker J, Barkauskas V, Johnson M, Maas M, Moorhead S, Reed D. Assessing the reliability, validity, and sensitivity of nursing outcomes classification in home care settings. Journal of Nursing Measurement. 2003;11(2):135–155. doi: 10.1891/jnum.11.2.135.57285. [DOI] [PubMed] [Google Scholar]
  • 39.Farrell D, Frank B, Brady C, McLaughlin M, Gray A. Focus on caregiving. A case for consistent assignment. Provider. 2006:47–51. Retrieved from www.providermagazine.com. [PubMed] [Google Scholar]
  • 40.Gunningberg L, Fogelberg-Dahm M, Ehrenberg A. Improved quality and comprehensiveness in nursing documentation of pressure ulcers after implementing an electronic health record in hospital care. Journal of Clinical Nursing. 2009;18:1557–1564. doi: 10.1111/j.1365-2702.2008.02647.x. [DOI] [PubMed] [Google Scholar]
  • 41.Ehrenberg A, Ehnfors M, Ekman I. Older patients with chronic heart failure within Swedish community health care: A record review of nursing assessments and interventions. Journal of Clinical Nursing. 2004;13:90–96. doi: 10.1046/j.1365-2702.2003.00837.x. [DOI] [PubMed] [Google Scholar]

RESOURCES