Abstract
Background
Composite indices are single measures that combine the strengths of two or more individual measures and provide broader, easy-to-use measures for evaluation of provider performance and comparisons across units and hospitals to support quality improvement.
Objective
The study objective was to develop a unit-level inpatient composite nursing care quality performance index – the Pressure Ulcer and Fall Rate Quality Composite Index.
Design
Two-phase measure development study.
Settings
5,144 patient care units in 857 United States hospitals participating in the National Database of Nursing Quality Indictors® during the year 2013.
Methods
The Pressure Ulcer and Fall Rate Quality Composite Index was developed in two phases. In Phase 1 the formula was generated using a utility function and generalized penalty analysis. Experts with experience in healthcare quality measurement provided the point of indicator equivalence. In Phase 2 initial validity evidence was gathered based on hypothesized relationships between the Pressure Ulcer and Fall Rate Quality Composite Index and other variables using two-level (unit, hospital) hierarchical linear mixed modeling.
Results
The Pressure Ulcer and Fall Rate Quality Composite Index = 100 − PUR − FR, where PUR is pressure ulcer rate and FR is total fall rate. Higher scores indicate better quality. Bland-Altman plots demonstrated agreement between pairs of experts and provided evidence for inter-rater reliability of the formula. The validation process demonstrated that higher registered nurse skill mix, higher percent of registered nurses with a baccalaureate in nursing or higher degree, higher percent of registered nurses with national specialty certification, and lower percent of hours supplied by agency staff were significantly associated with higher Pressure Ulcer and Fall Rate Quality Composite Index scores. Higher percentages of unit patients at risk for a hospital-acquired pressure ulcer and higher unit rates of physical restraint use were not associated with higher Pressure Ulcer and Fall Rate Quality Composite Index scores.
Conclusions
The Pressure Ulcer and Fall Rate Quality Composite Index is a step toward providing a more holistic perspective of unit level nursing quality than individual measures and may help nurses nursing administrators obtain a broader view of which patient care units are the higher and lower performers. Further study is needed to examine the usability of the Pressure Ulcer and Fall Rate Quality Composite Index.
Keywords: composite indicator, falls, pressure ulcers, quality of health care, quality improvement
1. Background
There is increasing demand for better quality indicators in order for healthcare leaders and providers to benchmark standards of care and implement quality improvement strategies (Simms et al., 2012). However, one criticism of individual quality measures is that they may not reflect the overall quality of care by providers (Majeed, Lester, & Bindman, 2007). Improvement in scores on one quality measure may have the unintended consequence of decreasing quality scores on another measure. For example, changes in clinical practice to improve quality scores for hospital-acquired pressure ulcer rate could decrease the quality scores for fall rate due to the focus on hospital-acquired pressure ulcer prevention including increased or earlier mobilization of patients.
Health care quality composite indices are single measures that combine the strengths of two or more individual quality measures and provide easy-to-use indices for performance evaluations and comparisons. Simms et al. (2012) argue that composite indices provide a better comparison of provider performance than single indicators. For example, Dimick, Staiger, Baser, and Birkmeyer (2009) combined two quality measures (observed mortality rate and hospital volume for each of six surgical operations) and found the composite to be a strong predictor of future performance. Our study objective was to develop an inpatient unit-level composite nursing care quality outcome performance indicator – the Pressure Ulcer and Fall Rate Quality Composite Index.
1.1. Nursing Quality of Care Measures
Nursing quality of care measures or nursing-sensitive measures are “processes and outcomes — and structural proxies for these processes and outcomes (e.g., skill mix, nurse staffing hours) — that are affected, provided, and/or influenced by nursing personnel, but for which nursing is not exclusively responsible” (NQF, 2004, p. 2). What has been measured as nursing sensitive generally has fallen under Donabedian’s (1988, 1992) Structure-Process-Outcome paradigm (Maben, Marrow, Ball, Robert, & Griffiths, 2012), in which the structure and processes of care influence the outcomes of care. Structure refers to setting attributes, including provider characteristics (e.g., skill mix, nurse staffing hours) in which patient care takes place. Processes are what is done, including interactions between patient and providers (e.g., risk assessment, prevention interventions, patient-provider communication). Patient outcomes are results of care and include patient safety measures (e.g., hospital-acquired pressure ulcers, falls, healthcare associated infections). Comprehensive lists of nursing quality measures are provided by Heslop and Lu (2014) and Maben et al. (2012).
In a recent concept analysis of nursing-sensitive indicators, Heslop and Lu (2014) found that pressure ulcers and falls were the two most frequently cited outcome attributes of nursing-sensitive indicators. Hospital-acquired pressure ulcer and total fall rates are widely recognized, are important measures of inpatient nursing care quality, and have been endorsed by the National Quality Forum as nursing-sensitive (Helsop & Lu; Maben et al., 2012; NQF). Further, both adverse events are foci of United States health care policy and patient safety initiatives. The National Database of Nursing Quality Indicators (NDNQI) collects data on hospital-acquired pressure ulcer and total fall rates from participating hospitals in order to provide standardized rate calculations across patient care units and hospitals – allowing for easy comparisons of like units in like hospitals for quality improvement initiatives. The two measures are the most reported by NDNQI participating hospitals. Therefore, hospital-acquired pressure ulcer rate and total fall rate were selected for development of an initial inpatient unit-level composite nursing care quality performance indicator, the Pressure Ulcer and Fall Rate Quality Composite Index.
1.2. Hospital-acquired pressure ulcers and inpatient falls
Hospital-acquired pressure ulcer rate is measured as the percentage of patients assessed who have a least one pressure ulcer that developed after hospital admission and total fall rate is measured as the total number of falls per 1000 patient days (NQF, 2004). The National Quality Forum endorsed hospital-acquired pressure ulcer rate and total fall rate as national consensus measures, meeting the National Quality Forum criteria of (a) importance to measure and report, (b) scientific acceptability, (c) feasibility, (d) usability and use, and (e) comparison to related or competing measures (NQF, 2015). Reliability evidence for the hospital-acquired pressure ulcer measure includes a Kappa agreement of .56 for wound identification, indicating moderate reliability, and .65 for pressure ulcer staging, indicating substantial reliability (Hart, Bergquist, Gajewski, & Dunton, 2006). Bergquist-Beringer, Gajewski, Dunton, & Klaus (2011) found substantial reliability for pressure ulcer staging among both certified and noncertified nurses, Kappa = .75 and .68 respectively. For the total falls measure, reliability evidence includes a sensitivity of .90 and specificity of .88 of fall classification (Simon, Klaus, Gajewski, & Dunton, 2013). Garrard, Boyle, Simon, Dunton, & Gajewski (2016) reported an overall intra-class correlation (ICC 1,1) of .85 for injury level assignment of falls, indicating substantial reliability.
Nursing care characteristics associated with lower hospital-acquired pressure ulcer rate are higher skill mix (percent of total nursing care hours supplied by registered nurses), higher registered nurse hours per patient day, and higher percentage of baccalaureate prepared registered nurses. The percent of nursing care hours supplied by agency nurses has not been shown to be associated with hospital-acquired pressure ulcers (Dunton Gajewski, Klaus, & Pierson, 2007; Xue, Aiken, Freund, & Noyes, 2012). The nursing care processes associated with lower hospital-acquired pressure ulcer rate are recent pressure ulcer risk assessment and having pressure ulcer prevention measures in place (Bergquist-Beringer, Dong, He, & Dunton, 2013; Blegen, Goode, Park, Vaughn, & Spetz, 2013; Dunton et al., 2007; Park, Boyle, Bergquist-Beringer, Staggs, & Dunton, 2014). A higher rate of physical restraint use has been associated with higher hospital-acquired pressure ulcer rate (Castle & Engberg, 2009; Rakhmatullina, Taub, & Jacob, 2013).
Nursing care characteristics linked to lower total fall rate are higher skill mix, higher total nursing hours per patient day, higher registered nurse hours per patient day, lower non-registered nurse hours per patient day, and higher percent of registered nurses with nursing specialty certification (Boyle, Cramer, Potter, & Staggs, 2015; Kendall-Gallagher & Blegen, 2009; Lake, Shang, Klaus, & Dunton, 2010; Titler, Shever, Kanak, Picone, & Qin, 2011). The percent of nursing care hours supplied by agency nurses has not been shown to be associated with fall rates (Dunton et al., 2007; Xue et al., 2012). Higher rates of physical restraint use have been associated with higher fall rate (Castle & Engberg, 2009).
1.3. Study objective
The study objective was to develop a composite indicator that combines hospital-acquired pressure ulcer rate and total fall rate in order to take a step toward having a broader view of unit nursing care quality.
We conducted the Pressure Ulcer and Fall Rate Quality Composite Index development in two phases (see Figure 1 for Flow Chart of Study Phases). In Phase 1 we developed the Pressure Ulcer and Fall Rate Quality Composite Index formula using a utility function – which assigns numerical values to different changes in the two quality measures of hospital-acquired pressure ulcer rate and total fall rate – and generalized penalty analysis. Higher scores on the Pressure Ulcer and Fall Rate Quality Composite Index indicate better outcomes.
Figure 1.
Flow Chart of Study Phases
In Phase 2 we obtained beginning evidence of validity for the Pressure Ulcer and Fall Rate Quality Composite Index by examining relationships of the Pressure Ulcer and Fall Rate Quality Composite Index scores to “other variables that the test is expected to correlate with or predict, as well as variables that the test is not expected to correlate with” (Goodwin, 2002, p.104). We hypothesized that
A higher percentage of unit registered nurses with a baccalaureate in nursing or higher degree, a higher unit percentage of registered nurses holding certification in a specialty area of nursing practice, higher unit total nursing hours per patient day, and higher unit registered nurse skill mix will be associated with higher unit Pressure Ulcer and Fall Rate Quality Composite Index scores;
The unit percent of nursing hours supplied by agency staff will not be associated with unit Pressure Ulcer and Fall Rate Quality Composite Index scores;
A higher percentage of unit patients at risk for a hospital-acquired pressure ulcer will be associated with lower unit Pressure Ulcer and Fall Rate Quality Composite Index scores
A higher percentage of unit patients assessed for pressure ulcer risk in the 24-hour period before the hospital-acquired pressure ulcer assessment and higher unit mean number of pressure ulcer prevention measures in place per patient will be associated with higher unit Pressure Ulcer and Fall Rate Quality Composite Index scores; and
Higher unit restraint rate will be associated with lower unit Pressure Ulcer and Fall Rate Quality Composite Index scores.
Prior to the study start, approval for both phases was obtained from the institutional review boards at the University of Wyoming and the University of Kansas Medical Center.
2. Development Phase 1
2.1. Method
Using a utility function, our goal was to develop a simple composite measure of nursing care quality from hospital-acquired pressure ulcer and total fall rates. Three experts were recruited to provide the point of indicator equivalence: 1) a doctoral-prepared registered nurse with expertise in pressure ulcers, 2) a doctoral-prepared statistician with expertise in falls, and 3) a doctoral-prepared sociologist with expertise in measurement of both hospital-acquired pressure ulcers and falls. All experts had extensive experience in healthcare quality measurement.
The experts were trained with scenarios in which each was asked to independently decide whether patient care Unit A was better, patient care Unit B was better, or both the units were comparable based on various simulated combinations of hospital-acquired pressure ulcer and total fall rates. Once the experts (raters) understood the task, they were given the following question and scenarios. Suppose patient care Unit B has a lower total fall rate than Unit A, but Unit A has a lower hospital-acquired pressure ulcer rate than Unit B. What would the hospital-acquired pressure ulcer rate and total fall rate have to be so that Unit A has the same quality as Unit B? For each of the two patient care units, a hospital-acquired pressure ulcer rate and total fall rate were provided for one unit, but there was a missing rate value for either hospital-acquired pressure ulcer rate or fall rate on the second unit. For example Unit A had a hospital-acquired pressure ulcer rate of 4% and a total fall rate of 3 per 1000 patient days and Unit B had a total fall rate of 4 per 1000 patient days but the hospital-acquired pressure ulcer rate was missing. Experts were asked to provide a number for the missing Unit B hospital-acquired pressure ulcer rate in such a way that the value assigned indicated the two units were equal in quality of care. The three experts completed ten different scenarios (see Supplemental Data File 1 for scenarios), with each deciding the missing value individually without consultation or input from the other experts. Table 1 provides the median of experts’ provided missing values. The first scenario was a training scenario and was eliminated from further analysis.
Table 1.
Median of Experts’ Values for Making Unit A and Unit B Equal in Quality of Care.
| Scenario | Unit A | Unit B | ||
|---|---|---|---|---|
| Fall Rate | Pressure Ulcer Rate | Fall Rate | Pressure Ulcer Rate | |
| 1 | 3.0 | 4.0% | 2.0 | (5.5%) |
| 2 | 6.0 | 2.0% | (4.75) | 2.5% |
| 3 | 3.0 | 4.0% | 5.0 | (2.0%) |
| 4 | 3.0 | 8.0% | (5.5) | 4.0% |
| 5 | 6.0 | 3.0% | (2.25) | 8.0% |
| 6 | 2.0 | 1.0% | (0.5) | 2.5% |
| 7 | 2.5 | 3.0% | 3.0 | (2.75%) |
| 8 | 1.0 | 15.0% | 10.0 | (3.5%) |
| 9 | 3.5 | 7.0% | 2.0 | (7.5%) |
| 10 | 7.0 | 1.0% | 1.0 | (4.5%) |
Note. Values in parentheses are the median of experts’ assignment of the respective missing value. Scenario 1 was a training scenario and eliminated from further analysis.
2.2. Results of Phase I
2.2.1. Reliability
Bland-Altman plots for agreement between any pair of two raters (Figure 2) illustrated that the raters were very consistent on all but two scenarios. For two patient care units to have the same composite score on quality of care in the scenarios, the difference between units on one measure (hospital-acquired pressure ulcer rate or total fall rate) must be offset by an equivalent difference in the opposite direction on the other measure. Rater median scores were plotted to demonstrate the inverse relationship (see Supplemental Data File 2 for the plots). Average slope of the lines in the plots was −1.12 indicating the experts had given almost equal weights to the two measures.
Figure 2.
Bland – Altman Plots for Experts (Raters).
2.2.2. Proposed Overall Composite Nursing Care Quality Index
We then used a generalized penalty analysis to determine the overall Pressure Ulcer and Fall Rate Quality Composite Index, which is the Pressure Ulcer and Fall Rate Quality Composite Index = 100 − PUR − cFR, where PUR and FR stand for unit hospital-acquired pressure ulcer rate and total fall rate respectively and c is the parameter. We used 100 to make the Pressure Ulcer and Fall Rate Quality Composite Index score intuitive because every unit starts with a Pressure Ulcer and Fall Rate Quality Composite Index score of 100 and then quality is reduced based on the unit hospital-acquired pressure ulcer rate and total fall rate. The c parameter is estimated to equal 1.12 based on the reliability study of experts’ ratings, however we use c equal to 1 for simplicity. Because a higher value of total fall rate represents a lower level of quality, c is a positive parameter and is the relative penalty for total fall rate. Therefore, the final formula is Pressure Ulcer and Fall Rate Quality Composite Index = 100 − PUR − FR. For every one percent hospital-acquired pressure ulcer rate, the Pressure Ulcer and Fall Rate Quality Composite Index is reduced by one point. Then, for every one fall per 1000 patient days, the Pressure Ulcer and Fall Rate Quality Composite Index also is reduced by one point. Higher Pressure Ulcer and Fall Rate Quality Composite Index values represent higher unit nursing quality. The highest possible Pressure Ulcer and Fall Rate Quality Composite Index score is 100. Scores could be negative in theory, but not likely in practice.
3. Development Phase 2
3.1. Methods
3.1.1. Data Source
Development Phase 2 was a secondary analysis using cross sectional data from the year 2013 National Database of Nursing Quality Indicators (NDNQI), at the time a program of the American Nurses Association. The NDNQI collects unit level nursing structure (e.g., nurse staffing characteristics), process (e.g., pressure ulcer risk assessment and prevention interventions), and outcome data (e.g., pressure ulcer and total fall rates) on a quarterly basis. The University of Kansas Medical Center Institutional Review Board provides ongoing database oversight and approval. Hospitals electively join the database. A site coordinator at each NDNQI hospital is responsible for collecting and submitting data or overseeing the process of data collection and submission according to NDNQI standardized definitions and protocols. The site coordinator and each person collecting or reporting data must undergo training about the NDNQI data collection guidelines and pass tests on critical elements with high accuracy. All data are entered directly into a secure, password protected NDNQI website. Data are subjected to extensive quality audits by NDNQI staff, such as checks for out-of-range or illogical data and Mahalanobis distance algorithms that detect changes over time.
The analytic file was extracted by NDNQI analysts and provided to the investigators. The file was checked for errors both by NDNQI analysts and investigators and corrected as needed. File checking included manual checks of randomly selected hospitals to ensure that the correct unit and hospital data were matched and investigation of all outliers (e.g., unusually high total fall rate) to determine if they were accurate or needed to be corrected (e.g., by NDNQI staff contacting the hospital).
In December 2013, 20293 units in 1970 hospitals participated in the quarterly data submission to NDNQI. NDNQI units are classified by patient population (e.g., adult, pediatric) and then by acuity level (e.g., critical care, step-down, medical). To be eligible for our study, units were limited to those that reported hospital-acquired pressure ulcer rate and patient total fall rate, as well as nursing characteristics and processes associated with lower hospital-acquired pressure ulcer rate and total fall rate. Unit types included adult critical care, step-down, medical, surgical, medical-surgical combined, rehabilitation, and critical access.
3.1.2. Study sample
Patient care units were the main unit of analysis. The 5,144 units meeting study criteria were nested in 857 NDNQI hospitals. The included unit types were included adult critical care (921 units), step-down (767 units), medical (1,126), surgical (762 units), medical-surgical combined (1,319 units), rehabilitation (237 units), and critical access (12 units).
Of the 857 sample hospitals, 24% had 300 staffed beds or more, 83% were not-for-profit hospitals, 34% were teaching hospitals, and 20% were Magnet designated hospitals. These hospital characteristics were of similar distribution to all NDNQI member hospitals in 2013. However, hospitals reporting to the 2011 American Hospital Association Annual Survey had lower proportions on each characteristic: 15% had 300 staffed beds or more, 26% were not-for-profit hospitals, and 6% were Magnet designated hospitals.
Table 2 displays unit-level nursing characteristics by unit type. Critical care units had higher mean total nursing hours per patient day (16.59), skill mix (84.5% registered nurses), percent of registered nurses with a baccalaureate in nursing degree or higher (61.8%), percent of registered nurses with national specialty certification (22.0%), and percent of nursing hours supplied by agency staff (1.4%) than other unit types. Critical access units had the second highest total nursing hours per patient day (12.01) and percent of nursing hours supplied by RNs (72.9%).
Table 2.
Unit-Level Characteristics by Unit Type
| Variable | Critical Care (n=921) |
Step-Down (n=767) |
Medical (n=1,126) |
Surgical (n=762) |
Med-Surg (n=1,319) |
Rehab (n=237) |
Critical Access (n=12) |
|---|---|---|---|---|---|---|---|
| Mean (SD) |
Mean (SD) |
Mean (SD) |
Mean (SD) |
Mean (SD) |
Mean (SD) |
Mean (SD) |
|
| TNHPPD | 16.59 (4.11) |
10.19 (2.10) |
8.72 (1.86) |
8.99 (1.77) |
8.66 (1.61) |
8.10 (1.58) |
12.01 (1.37) |
| %Skill mix | 84.49 (6.25) |
72.86 (8.29) |
67.11 (8.18) |
68.82 (8.15) |
67.16 (8.56) |
59.02 (10.04) |
72.89 (7.49) |
| % RNs with BSN or higher | 61.78 (16.95) |
50.63 (18.22) |
49.66 (20.11) |
49.34 (19.62) |
45.83 (18.36) |
44.11 (20.24) |
32.31 (15.40) |
| % RNs with national specialty certification | 21.99 (14.86) |
11.04 (11.89) |
14.41 (14.93) |
14.04 (13.40) |
12.68 (13.20) |
22.32 (17.53) |
5.75 (6.02) |
| % nursing hours supplied by agency staff | 1.43 (2.78) |
1.23 (2.81) |
1.16 (2.97) |
0.84 (2.04) |
1.01 (2.51) |
0.83 (2.54) |
0.52 (1.11) |
| % Patients at risk for HAPU | 77.80 (16.79) |
40.41 (19.32) |
38.80 (16.09) |
31.54 (18.57) |
35.82 (17.03) |
56.14 (23.85) |
40.96 (16.03) |
| % Patients assessed for risk of HAPU in last 24 hours | 80.05 (22.15) |
75.83 (24.05) |
71.96 (26.58) |
72.34 (25.73) |
72.93 (27.14) |
63.26 (34.48) |
83.82 (22.99) |
| Mean HAPU prevention measures per patient | 3.93 (1.25) |
1.87 (1.05) |
1.74 (0.88) |
1.29 (0.84) |
1.62 (0.88) |
2.61 (1.41) |
1.72 (0.61) |
| % Patients with restraints | 17.36 (12.82) |
1.49 (2.81) |
0.88 (1.75) |
0.47 (1.36) |
0.71 (1.77) |
2.82 (7.55) |
0.00 (0.00) |
| Total fall rate | 1.13 (0.75) |
3.03 (1.31) |
3.76 (1.61) |
2.74 (1.32) |
3.33 (1.45) |
6.09 (2.72) |
2.89 (0.96) |
| HAPU rate | 6.42 (5.44) |
2.96 (3.45) |
2.23 (2.62) |
1.87 (2.54) |
1.97 (2.62) |
2.34 (2.95) |
1.52 (2.87) |
Note. Med-Surg = Medical-Surgical Combined, Rehab = Rehabilitation, SD= Standard deviation, TNHPPD = Total nursing hours per patient day, RN= Registered nurse, Skill mix = % Nursing hours supplied by RNs, BSN = Bachelor of Science in Nursing, HAPU = Hospital-acquired pressure ulcer
For process and outcome variables (Table 2), critical care units had the highest percent of patients at risk for hospital-acquired pressure ulcers (77.8%), average number of hospital-acquired pressure ulcer prevention measures in place (3.93), percent of patients with restraints in place (17.4%), and hospital-acquired pressure ulcer rate (6.4%). Critical access units had the highest percent of patients assessed for risk of pressure ulcers in the last 24 hours (83.8%), along with the lowest restraint rate (0.00). Rehabilitation units had the highest fall rate (6.09/1000 patient days).
3.1.3. Measures
Select unit-level nursing characteristics and processes known to be associated with nursing care quality were chosen from the NDNQI database for the Pressure Ulcer and Fall Rate Quality Composite Index validation process. Hypotheses are listed at the end of the study objective section.
Nurse staffing characteristics. Percent of unit registered nurse with a baccalaureate or higher degree and percent of unit registered nurses holding certification in a specialty area of nursing practice granted by a nationally accredited nursing certification program (Miller & Boyle, 2008) were derived from NDNQI administrative data, which are submitted yearly. The NDNQI staffing variables of total nursing hours per patient day and skill mix are National Quality Forum endorsed. Reliability evidence of the nursing care hours measure includes inter-class correlations ranging from .76 to .99 for total nursing hours per patient day (Klaus, Dunton, Gajewski, & Potter, 2013).
Nursing Processes. Data on pressure ulcer risk and prevention were gathered during the same one-day assessment of hospital-acquired pressure ulcer rate (Bergquist-Beringer et al., 2013). The measure set includes skin and pressure ulcer risk assessment on admission (yes, no), a risk assessment 24 hours or less before the survey (yes, no), and risk status (at risk, not at risk). For those at risk, prevention measures include a skin assessment, pressure redistribution surface use, routine repositioning, nutritional support, and moisture management during the 24-hour period preceding the survey (yes, no). Waugh and Bergquist-Beringer (2016) reported evidence for the overall reliability of the pressure ulcer risk and prevention measures, including an overall prevalence-adjusted kappa (PAK) of .92 for the pressure ulcer risk measures indicating high agreement between study expert and non-expert raters on these measures. The overall prevalence-adjusted kappa value for the pressure ulcer prevention measures was .71 indicating substantial agreement. However, reliability results for nutrition support (PAK = .50), moisture management (PAK = .56) and routine repositioning (PAK = .58) showed only moderate agreement between raters. For our study, prevention measures in place was a count of their number (possible range 1–5). Unit processes for falls were not included because NDNQI collects process data only on patients who fall, including percent at risk of falling.
The NDNQI physical restraint data are collected quarterly using a one-day, point-in-time patient assessment. The restraint measure also is National Quality Forum endorsed. The rate is a percentage calculated as the number of unit patients with limb and/or vest restraints in place divided by the number of patients surveyed, multiplied by 100.
Control variables. We included select unit and hospital characteristics as controls to address differential risk of falls and hospital-acquired pressure ulcers. The main approach was to control for patient care unit type, which cluster patients by acuity. In addition, we controlled for hospital patient severity using the United States Centers for Medicare & Medicaid Services hospital Case-Mix Index (low = 0–1.42, medium = 1.43–1.75, high > 1.76).
Prior researchers have found that nursing characteristics, nurse staffing and patient outcomes can vary by hospital characteristics such as American Nurses Credentialing Center (ANCC) Magnet Recognition Program (ANCC, n.d.) status, ownership, and location (Bergquist-Beringer et al., 2013; Dunton et al., 2007; Lake et al., 2010; Park et al., 2014). Therefore, we included Magnet status (Magnet, not Magnet), hospital bed size (<100, 100–199, 200–299, 300–399, 400–499, >=500), ownership (not-for-profit; for-profit; government, federal; government, non-federal), and metropolitan location (> 50,000 population, < 50,000 and > 10,000 population, < 10,000 population) as control variables.
3.1.4. Data Analysis
The validity testing modeling was fitted with SPSS 22.0 (IBM) using 2-level (unit, hospital) hierarchical linear mixed modeling to determine associations of nursing characteristics, nursing processes, and physical restraints with the Pressure Ulcer and Fall Rate Quality Composite Index (See Supplemental Data File 3 for SPSS code). The hierarchical analysis was performed using two statistical model specifications varying by hospital level control variables. Unit type was included in both statistical models. The effects of interactions among variables within and between levels were not considered, as effects of interactions was not one of our hypotheses. We only included units in the analysis that had all variables non-missing and did not perform any imputation. The distributional assumptions in the model were not a concern as the number of units was so large that the central limit theorem provides estimates that are normally distributed. The statistical model with the better fit (i.e., lowest Bayesian Information Criteria, [BIC]) was selected for the final statistical inference.
A sensitivity analysis was performed and it was determined that units with less than 30 patients assessed for hospital-acquired pressure ulcers during any quarter should be excluded from all analyses. The model then was re-analyzed and the final are results presented in the results section below. When including units only with 30 or more patients we found two changes relative to the all-inclusive model. First, we found that total nursing hours per patient day became significantly associated with the Pressure Ulcer and Fall Rate Quality Composite Index. Second, mean hospital-acquired pressure ulcer prevention measures in place per patient became significantly associated with the Pressure Ulcer and Fall Rate Quality Composite Index. All the other explanatory variables resulted in the same conclusions.
3.2. Results of Phase 2
3.2.1. Pressure Ulcer and Fall Rate Quality Composite Index descriptive statistics distribution
Figure 3 displays the descriptive statistics and distribution for the Pressure Ulcer and Fall Rate Quality Composite Index by unit type. Critical access units had the highest mean Pressure Ulcer and Fall Rate Quality Composite Index (95.58), followed by surgical (95.39) and medical-surgical combined units (94.70). Critical care units had the largest range of scores (68.96 – 100). All distributions were skewed left.
Figure 3.
Pressure Ulcer and Fall Rate Quality Composite Index Distribution and Descriptive Statistics by Unit Type.
Pressure Ulcer and Fall Rate Quality Composite indicator
3.2.2. Validity testing results: Pressure Ulcer and Fall Rate Quality Composite Index
The better-fitting statistical model (lower BIC) included unit type, all hospital control variables, and hospital as the only random effect. The estimated intraclass correlation coefficient was 0.25, which is large enough to support the need for the random effects model. All correlations among the explanatory variables were no larger than 0.50, except for the correlation between mean hospital-acquired pressure ulcer prevention measures in place per patient and percent of patients at risk for hospital-acquired pressure ulcers; this correlation was 0.90. To assess for collinearity, we ran the model without the risk variable. Results remained substantively the same except that mean pressure ulcer prevention measures in place became significant. We report the final model below without the risk variable.
Two models were considered. The first had no hospital control variables and higher BIC (27174). The second and final model (BIC=26115) included hospital level variables and is shown in Table 3. Lower total nursing hours per patient day (B = −.101, p = .001), higher registered nurse skill mix (B = .018, p = .018), higher percent of registered nurses with a baccalaureate in nursing or higher degree (B = .009, p= .010), higher percent of registered nurses with national specialty certification (B = .010, p = .014), lower percent of nursing hours supplied by agency nurses (B = −.066, p = .003), and lower mean pressure ulcer prevention measures in place (B = −864, p = .000) were significantly associated with higher Pressure Ulcer and Fall Rate Quality Composite Index scores. The percent of patients assessed for pressure ulcer risk in the last 24 hours and the percent of patients with physical restraints were not significantly associated with the Pressure Ulcer and Fall Rate Quality Composite Index.
Table 3.
Estimates of Effects of Unit-Level Nursing Characteristics and Processes on the Pressure Ulcer and Fall Rate Quality Composite Index
| Unit Level Variables | B | Standard Error | t | p |
|---|---|---|---|---|
| TNHPPD | −0.101 | .031 | −3.270 | 0.001 |
| % Skill Mix | 0.018 | 0.007 | 2.374 | 0.018 |
| % RNs with BSN or higher | 0.009 | 0.004 | 2.565 | 0.010 |
| % RNs with national specialty certification | 0.010 | 0.004 | 2.447 | 0.014 |
| % Nursing hours supplied by agency staff | −0.066 | 0.023 | −2.926 | 0.003 |
| % Patients assessed for risk of HAPU in last 24 hours | 0.001 | 0.002 | 0.265 | 0.791 |
| Mean HAPU prevention measures in place per patient | −0.864 | 0.058 | −14.836 | 0.000 |
| Physical restraint rate | −1.435 | 0.848 | −1.693 | 0.091 |
Note. TNHPPD = Total nursing hours per patient day; BSN = Bachelor of Science in Nursing; HAPU = Hospital-acquired pressure ulcer.
4. Discussion
We developed a Pressure Ulcer and Fall Rate Quality Composite Index combining hospital-acquired pressure ulcer rate and fall rate and provided initial evidence of reliability and validity. The Pressure Ulcer and Fall Rate Quality Composite Index formula, Pressure Ulcer and Fall Rate Quality Composite Index = 100 − PUR − FR, is intuitive because every unit starts with a Pressure Ulcer and Fall Rate Quality Composite Index score of 100 and then quality is reduced based on the unit hospital-acquired pressure ulcer rate and total fall rate. High agreement among the diverse set of raters when deciding missing values in the simulated scenarios, indicated inter-rater reliability of the resulting formula.
The validity of the Pressure Ulcer and Fall Rate Quality Composite Index then was tested in 5,144 units with measures known to be associated with nursing quality. As hypothesized, we found that higher registered nurse skill mix, higher percent of registered nurses with a baccalaureate degree or higher, and higher percent of registered nurse with national specialty certification were associated with higher Pressure Ulcer and Fall Rate Quality Composite Index scores.
We were unable to examine hypothesis three that higher percentages of patients at risk for developing hospital-acquired pressure ulcers would be significantly associated with lower Pressure Ulcer and Fall Rate Quality Composite Index scores because the risk variable was deleted in the collinearity analysis. However, we unexpectedly found that higher mean number of pressure ulcer prevention measures in place per patient was associated with lower Pressure Ulcer and Fall Rate Quality Composite Index scores – contrary to hypothesis 4. The findings are likely due to the high correlation (.90) between the percentage of patients at pressure ulcer risk and the mean number of prevention measures in place. Hospitals report prevention measures in place only for patients at risk for a pressure ulcer, so units with a greater proportion of patients at risk tend to have a higher number of prevention measures in place per patient. In the absence of the risk variable, the prevention measures variable is something of a proxy for pressure ulcer risk. Those at risk are more likely to develop a pressure ulcer (Bergquist-Beringer et al., 2013). The finding supports hypothesis three if the mean number of prevention measures in place can be considered a proxy for risk.
The finding that lower total nursing hours per patient day was associated with higher Pressure Ulcer and Fall Rate Quality Composite Index scores – opposite the direction hypothesized – deserves some comment. Total nursing hours per patient day includes all nursing care hours of registered nurses, licensed practical/vocational nurses, and nurse aides without regard to the proportion of each subset of nursing staff. Lake et al. (2010) found statistically significant opposite effects of registered nurse hours as compared to licensed practical/vocational nurses, and nurse aide hours – higher registered nurse hours per patient day were associated with lower fall rates, whereas higher licensed practical/vocational nurse hours per patient day and nurse aide hours per patient day were associated with higher fall rates. Therefore, it may be more useful to consider our finding that higher skill mix (percent of nursing care hours supplied by registered nurses) was associated with higher Pressure Ulcer and Fall Rate Quality Composite Index scores.
Our hypothesis that there would be no association between the Pressure Ulcer and Fall Rate Quality Composite Index and percent of nursing hours supplied by agency nurses was not supported; instead, the higher the percent of hours supplied by agency staff, the lower the Pressure Ulcer and Fall Rate Quality Composite Index. Two previous studies found no relationship between agency staff and hospital-acquired pressure ulcers and fall rate, however, both included smaller sample sizes than our study. One of these studies was based on a sample of 19 units in one hospital (Xue et al., 2012). In the other study the number of units was 1,610 (Dunton et al., 2007). Therefore, an explanation for our finding may be the larger sample size. More research is needed to explore the relationship.
4.1 The Patient Safety Approach of the Pressure Ulcer and Fall Rate Quality Composite Index
Hospital-acquired pressure ulcer and total fall rates, as well as the Pressure Ulcer and Fall Rate Quality Composite Index, fit within a commonly used framework to assess patient safety – that of an injury-based approach, which has the goal of eliminating patient harm (Scanlon, Karsh, & Saran, 2008). Scanlon et al. argue that “although injury-based metrics might aid the prevention of harm, limitations include poor discrimination of preventability, resulting in misdirected interventions, missed opportunities, and disregard for the systems-based nature of unsafe health care” (p.1). They proposed a framework that is risk-based in which hazard identification and control are the focus. However, the purpose of our composite indicator is different.
We would argue that rather than disregarding system-based factors underlying adverse outcomes, the composite nature of our index allows us to measure systemic or structural aspects of a unit’s quality of care with more sensitivity than risk-based measures related to any single adverse outcome (e. g., count of hospital-acquired pressure ulcer risk-reduction methods implemented for a patient) would allow. In the absence of risk-based quality measures for every adverse outcome, measuring a unit’s overall quality with a set of indicators – ideally more than two, with the Pressure Ulcer and Fall Rate Quality Composite Index an initial example – is the only way to identify units with high and low overall quality and investigate what factors make them different. Further, there are not good risk-based safety metrics for falls, so that approach isn’t an option.
It also is worth noting that risk- and injury-based approaches to safety measurement are not mutually exclusive but complementary. The success of any risk-based prevention method (e.g., CLABSI checklist) can only be assessed by observing its effect on the rate of injury or harm. In terms of Donabedian’s framework, a focus on improving structures or processes of care to reduce risk must be accompanied by a measurable effect on outcomes—i.e., injury or harm.
4.2 Further Study of the Pressure Ulcer and Fall Rate Quality Composite Index
The Pressure Ulcer and Fall Rate Quality Composite Index provides a more holistic and quick view of unit level nursing quality than a measure based on a single outcome. The Pressure Ulcer and Fall Rate Quality Composite Index may be useful in dashboards that provide a higher level, more aggregate view of organizational quality for nursing administrators and other senior organizational leaders. The Pressure Ulcer and Fall Rate Quality Composite Index also may provide a broader view of which patient care units are higher and lower performers, so resources for quality improvement can be allocated accordingly. Nurse managers and staff nurses may find the individual measures more useful for focusing on improvements on a particular outcome. Mixed-methods studies therefore are needed among administrators first to examine the usability and interpretative ease of the Pressure Ulcer and Fall Rate Quality Composite Index, as well as whether the Pressure Ulcer and Fall Rate Quality Composite Index would allow them to direct resources better.
The Pressure Ulcer and Fall Rate Quality Composite Index could be enhanced to provide an even broader assessment of quality by incorporating additional quality measures (e.g., rates of various healthcare-acquired infections), although the benefit of additional measures should be weighed against any resulting reduction in the number of units for which data on all Pressure Ulcer and Fall Rate Quality Composite Index component measures are available.
The Pressure Ulcer and Fall Rate Quality Composite Index also may be improved by risk adjustment. For example, the percentage of patients on a unit at risk for a hospital-acquired pressure ulcer could be used to adjust the hospital-acquired pressure ulcer component of the Pressure Ulcer and Fall Rate Quality Composite Index score. Of course, the improvement requires that risk adjustment data are available for all unit patients and, in the case of falls, NDNQI does not collect such data. Nevertheless, risk adjustment is a possible topic for further research.
Examination of unit-level time trends is warranted to determine whether the Pressure Ulcer and Fall Rate Quality Composite Index changes over time and is sensitive to events such as the Centers for Medicare & Medicaid Services non-payment policy for reasonably preventable hospital-acquired complications that became effective on October 1, 2008. There is evidence that the time trend in hospital-acquired pressure ulcer rates decreased after the Centers for Medicare & Medicaid Services ruling (He, Staggs, Bergquist-Beringer, & Dunton, 2013), but researchers failed to find statistically significant changes in time trends for rates of injurious falls and rates of stage III/IV hospital-acquired pressure ulcers (Waters, Daniels, Bazzoli, Perencevich, Dunton, et al., 2015). The extent to which nursing care quality in general, as distinct from specific patient outcomes, improved due to the policy change is a topic for further research.
An identification and examination of the structure and process characteristics of units that consistently score at or near 100 on the Pressure Ulcer and Fall Rate Quality Composite Index over time could provide evidence about mechanisms to enhance patient safety in lower performing units. Similarly, along with the actual hospital-acquired pressure ulcer rate and total fall rate, very low-scoring units on the Pressure Ulcer and Fall Rate Quality Composite Index could be studied to identify reasons for poor performance. In addition to the nursing care characteristics included in our study, organizational-level factors such as leadership characteristics, culture, and climate should be included along with unit level factors such as nurse-physician communication, nurse-nurse communication, autonomy, and decision-making.
Finally, to complement the Pressure Ulcer and Fall Rate Quality Composite Index, a further composite index that takes into account the cost of falls and hospital-acquired pressure ulcers (and possibly other outcome measures) would be useful. The composite cost index would reflect cost rather than expert opinion trade-off, giving greater weight to the more costly outcome(s).
4.3. Limitations
Although our sample included a large number of hospitals and units, NDNQI hospitals electively join the database, are a select sample, and are not representative of the overall population of U.S. hospitals. NDNQI is under-representative of small hospitals. Although about half of U.S. hospitals are under 100 beds, only one quarter of NDNQI hospitals are of that size. We limited our attention to seven unit types. Further some NDNQI units do not collect both hospital-acquired pressure ulcer rate and total fall rate, so those units were not included in the study. Therefore, results may not be generalizable to other hospitals and units.
As a caution, smaller units should be cautious about applying the Pressure Ulcer and Fall Rate Quality Composite Index over a short period of time. For statistical stability there should be at least 30 patients surveyed for hospital-acquired pressure ulcers in a quarter before considering the Pressure Ulcer and Fall Rate Quality Composite Index. Due to the small number of critical access units in the study, the results regarding critical access units especially should be used with caution.
5. Conclusions
We developed a composite Pressure Ulcer and Fall Rate Quality Composite Index combining hospital-acquired pressure ulcer rate and total fall rate and provided initial evidence of reliability and validity. The Pressure Ulcer and Fall Rate Quality Composite Index is a step toward providing a more holistic and quick view of unit level nursing quality. The Pressure Ulcer and Fall Rate Quality Composite Index, along with individual hospital-acquired pressure ulcer rates and total fall rates, may help nursing administrators and nurses obtain a broader view of which patient care units are the higher and lower performers and direct resources for quality improvement accordingly.
Supplementary Material
What is already known about the topic?
The individual nursing sensitive quality indicators of hospital-acquired pressure ulcer rate and total fall rate are reliable and valid indicators that provide standardized rate calculations across patient care units and hospitals.
A criticism of individual quality measures is that they may not reflect the overall quality of care by providers.
Health care quality composite indices are single measures that combine the strengths of two or more individual quality measures and provide easy-to-use indices.
What this paper adds
We developed the Pressure Ulcer and Fall Rate Quality Composite Index and provide initial evidence of reliability and validity.
The Pressure Ulcer and Fall Rate Quality Composite Index is a step toward a broader view of unit level nursing quality that may be useful in providing a higher level, more aggregate view of organizational quality.
Acknowledgments
The authors thank Dr. Emily Cramer, Research Assistant Professor, University of Kansas School of Nursing, for her review and comments on an earlier version of the paper.
Source of Funding: Research reported in this publication was supported by the National Institutes of Health under Award Numbers NIH/NCRR 5P20RR 016474, NIH/NIGMS 8P20GM103432 and by the University of Wyoming Excellence in Higher Education Fund. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the University of Wyoming.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Conflicts of Interest: No conflicts of interest declared for all authors.
Contributor Information
Diane K. Boyle, Email: dboyle@uwyo.edu.
Ananda Jayawardhana, Email: ajayawardhana@pittstate.edu.
Mary E. Burman, Email: mburman@uwyo.edu.
Nancy E. Dunton, Email: ndunton@kumc.edu.
Vincent S. Staggs, Email: vstaggs@cmh.edu.
Sandra Bergquist-Beringer, Email: sbergquist-beringer@kumc.edu.
Byron J. Gajewski, Email: bgajewski@kumc.edu.
References
- American Nurses Credentialing Center (ANCC) Magnet Recognition Program. Magnet Recognition Program Overview. (n.d.) Retrieved from http://www.nursecredentialing.org/Magnet/ProgramOverview.
- Bergquist-Beringer S, Dong L, He J, Dunton N. Pressure ulcers and prevention among acute care hospitals in the United States. Joint Commission Journal on Quality and Patient Safety. 2013;39:404–414. doi: 10.1016/s1553-7250(13)39054-0. [DOI] [PubMed] [Google Scholar]
- Bergquist-Beringer S, Gajewski B, Dunton N, Klaus S. The reliability of the National Database of Nursing Quality Indicators pressure ulcer indicator: A triangulation approach. Journal of Nursing Care Quality. 2011;26:292–301. doi: 10.1097/NCQ.0b013e3182169452. [DOI] [PubMed] [Google Scholar]
- Blegen MA, Goode CJ, Park SH, Vaughn T, Spetz J. Baccalaureate education in nursing and patient outcomes. Journal of Nursing Administration. 2013;43(2):89–94. doi: 10.1097/NNA.0b013e31827f2028. [DOI] [PubMed] [Google Scholar]
- Boyle DK, Cramer E, Potter C, Staggs VS. The effect of longitudinal changes in RN specialty certification rates on total patient fall rates in acute care hospitals. Nursing Research. 2015;64:291–299. doi: 10.1097/NNR.0000000000000107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Castle NG, Engberg J. The health consequences of using physical restraints in nursing homes. Medical Care. 2009;47:1164–1173. doi: 10.1097/MLR.0b013e3181b58a69. [DOI] [PubMed] [Google Scholar]
- Dimick JB, Staiger DO, Baser O, Birkmeyer JD. Composite measures for predicting surgical mortality in the hospital. Health Affairs. 2009;28:1189–1198. doi: 10.1377/hlthaff.28.4.1189. [DOI] [PubMed] [Google Scholar]
- Donabedian A. The quality of care: How can it be assessed? JAMA. 1988;260:1743–1748. doi: 10.1001/jama.260.12.1743. [DOI] [PubMed] [Google Scholar]
- Donabedian A. The role of outcomes in quality assessment and assurance. Quality Review Bulletin. 1992;11:356–60. doi: 10.1016/s0097-5990(16)30560-7. [DOI] [PubMed] [Google Scholar]
- Dunton N, Gajewski B, Klaus S, Pierson B. The relationships of nursing workforce characteristics to patient outcomes. Online Journal of Issues in Nursing. 2007;12(3) doi: 10.3912/OJIN.Vol12No03Man03. manuscript 3. [DOI] [Google Scholar]
- Garrard L, Boyle DK, Simon M, Dunton N, Gajewski B. Reliability and validity of the NDNQI® injury falls measure. Western Journal of Nursing Research. 2016;38:111–128. doi: 10.1177/0193945914542851. doi:0193945914542851. [DOI] [PubMed] [Google Scholar]
- Goodwin LD. Changing conceptions of measurement validity: An update on the new standards. The Journal of Nursing Education. 2002;41(3):100–106. doi: 10.3928/0148-4834-20020301-05. [DOI] [PubMed] [Google Scholar]
- Hart S, Bergquist S, Gajewski B, Dunton N. Reliability testing of the National Database of Nursing Quality Indicators pressure ulcer indicator. Journal of Nursing Care Quality. 2006;21:256–265. doi: 10.1097/00001786-200607000-00011. [DOI] [PubMed] [Google Scholar]
- He J, Staggs VS, Bergquist-Beringer S, Dunton N. Unit-level time trends and seasonality in the rate of hospital-acquired pressure ulcers in US acute care hospitals. Research in Nursing & Health. 2013;36:171–180. doi: 10.1002/nur.21527. [DOI] [PubMed] [Google Scholar]
- Helsop L, Lu S. Nursing sensitive indicators: A concept analysis. Journal of Advanced Nursing. 2014;70:2469–2482. doi: 10.1111/jan.12503. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kendall-Gallagher D, Blegen MA. Competence and certification of registered nurses and safety of patients in intensive care units. American Journal of Critical Care. 2009;18:106–113. doi: 10.4037/ajcc2009487. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Klaus S, Dunton N, Gajewski B, Potter C. Reliability of the nursing care hour measure: A descriptive study. International Journal of Nursing Studies. 2013;50:924–932. doi: 10.1016/j.ijnurstu.2012.07.012. [DOI] [PubMed] [Google Scholar]
- Lake ET, Shang J, Klaus S, Dunton NE. Patient falls: Association with hospital Magnet status and nursing unit staffing. Research in Nursing and Health. 2010;33:413–425. doi: 10.1002/nur.20399. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Maben J, Morrow E, Ball J, Robert G, Griffiths P. High Quality Care Metrics for Nursing. National Nursing Research Unit; King’s College London: 2012. [Google Scholar]
- Majeed A, Lester H, Bindman A. Improving the quality of care with performance indicators. BMJ. 2007;335(7626):916–918. doi: 10.1002/nur.20399. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Miller PA, Boyle DK. Nursing specialty certification: A measure of expertise. Nursing Management. 2008;39(10):10–16. doi: 10.1097/01.NUMA.0000338302.02631.5b. [DOI] [PubMed] [Google Scholar]
- National Quality Forum (NQF) National voluntary consensus standards for nursing-sensitive care: An initial performance measure set—A consensus report. Washington DC: 2004. Retrieved from http://www.qualityforum.org/Publications/2004/10/National_Voluntary_Consensus_Standards_for_Nursing-Sensitive_Care__An_Initial_Performance_Measure_Set.aspx. [Google Scholar]
- National Quality Forum (NQF) Measure evaluation criteria and guidance for evaluating measures for endorsement. Washington DC: 2015. Retrieved from http://www.qualityforum.org/Measuring_Performance/Submitting_Standards.aspx. [Google Scholar]
- Park SH, Boyle DK, Bergquist-Beringer S, Staggs VS, Dunton NE. Concurrent and lagged effects of registered nurse turnover and staffing on unit-acquired pressure ulcers. Health Services Research. 2014;49:1205–1225. doi: 10.1111/1475-6773.12158. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rakhmatullina M, Taub A, Jacob T. Morbidity and mortality associated with utilization of restraints. The Psychiatric Quarterly. 2013;84:499–512. doi: 10.1007/s11126-013-9262-6. [DOI] [PubMed] [Google Scholar]
- Scanlon MC, Karsh B, Saran KA. Risk-based patient safety metrics. In: Henriksen K, Battles JB, Keyes MA, editors. Advances in Patient Safety: New Directions and Alternative Approaches. Vol. 1. Rockville MD: Agency for Healthcare Research and Quality; 2008. Assessment. Retrieved from: http://www.ncbi.nlm.nih.gov/books/NBK43628/ [PubMed] [Google Scholar]
- Simms AD, Baxter PD, Cattle BA, Batlin PD, Wilson JI, West RM, Gale CP. An assessment of complete measures of hospital performance and associated mortality for patients with acute myocardial infarction. Analysis of individual hospital performance and outcome for the National Institute for Cardiovascular Outcome Research (NICOR) European Heart Journal. Acute Cardiovascular Care. 2012;2(1):9–18. doi: 10.1177/2048872612469132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Simon M, Klaus S, Gajewski BJ, Dunton N. Agreement of fall classifications among staff in US hospitals. Nursing Research. 2013;62:74–81. doi: 10.1097/NNR.0b013e31827bf8c9. [DOI] [PubMed] [Google Scholar]
- Titler MG, Shever LL, Kanak MF, Picone DM, Qin R. Factors associated with falls during hospitalization in an older adult population. Research and Theory for Nursing Practice. 2011;25(2):127–152. doi: 10.1891/1541-6577.25.2.127. [DOI] [PubMed] [Google Scholar]
- Waters TM, Daniels MJ, Bazzoli GJ, Perencevich E, Dunton N, Staggs VS, Potter C, Fareed N, Liu M, Shorr RI. Effect of Medicare’s nonpayment for hospital-acquired conditions: Lessons for future policy. JAMA Internal Medicine. 2015;175(3):347–354. doi: 10.1001/jamainternmed.2014.5486. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Waugh SM, Bergquist-Beringer S. Inter-rater agreement of pressure ulcer risk and prevention measures in the National Database of Nursing Quality Indicators® (NDNQI) Research in Nursing & Health. 2016;39:164–174. doi: 10.1002/nur.21717. [DOI] [PubMed] [Google Scholar]
- Xue Y, Aiken LH, Freund DA, Noyes K. Quality outcomes of hospital supplemental nurse staffing. Journal of Nursing Administration. 2012;42:580–585. doi: 10.1097/NNA.0b013e318274b5bc. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.