Hospitals Known for Nursing Excellence Associated with Better Hospital Experience for Patients

Abstract

Objective

To examine the relationship between Magnet recognition, an indicator of nursing excellence, and patients' experience with their hospitalization reported in the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey.

Data Sources

This secondary analysis includes cross‐sectional data from the 2010 HCAHPS survey, the American Hospital Association, and the American Nurses Credentialing Center.

Study Design

We conducted a retrospective observational study.

Data Collection

Using common hospital identifiers, we created a matched set of 212 Magnet hospitals and 212 non‐Magnet hospitals.

Principal Findings

Patients in Magnet hospitals gave their hospitals higher overall ratings, were more likely to recommend their hospital, and reported more positive care experiences with nurse communication.

Conclusions

Magnet recognition is associated with better patient care experiences, which may positively enhance reimbursement for hospitals.

Value‐based purchasing (VBP) initiatives have been implemented in response to escalating health care costs and concerns about the quality and safety of health care in the United States. These initiatives are designed to promote transparency and improve quality in health care by rewarding or penalizing providers or systems based on their performance on a prescribed set of metrics (Millenson 2013).

Patients' experience with care has become a focal point for quality improvement as attention to patient‐centered care has increased. Comparable measures across hospitals of patient experience that are used to determine financial incentives under VBP are now easily accessible on the Centers for Medicare and Medicaid Services (CMS) Hospital Compare website (Department of Health and Human Services 2013). The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey is designed to evaluate patients' short‐term hospital care experience (Centers for Medicare & Medicaid Services 2013). Under the Affordable Care Act's VBP program, hospitals' reimbursement is tied to either achievement or improvement from their baseline performance score. The higher of the two, achievement or improvement, is used for each HCAHPS measure in calculating the total performance score.

Beginning with fiscal year (FY) 2013, hospitals' performance on the HCAHPS constitutes 30 percent of their total performance score for reimbursement by CMS, and beginning in FY2016, this will be changed to 25 percent of the total performance score. The remaining portion of the score is determined by efficiency, process of care, and outcome performance measures (Tourison 2014). The maximum reduction in reimbursement was 1.0 percent in FY2013, with incremental increases each year up to 2.0 percent in FY2017.

One straightforward strategy to potentially improve patients' experience and the performance scores derived from them involves registered nurses, who provide the most direct patient care in hospitals. Research has shown that patients' experience with hospital care is significantly related to whether hospitals are well‐resourced with respect to nursing, for example, nurse staffing, supportive work environments, and reasonable nurse work hours (Kutney‐Lee et al. 2009; Aiken et al. 2012; Stimpfel, Sloane, and Aiken 2012). With adequate numbers of well‐trained nurses in an environment that promotes professional practice, nurses are able to provide critical surveillance of their patients and intervene when appropriate, educate patients on medications, give discharge instructions, and perform other important bedside care.

Despite the evidence showing that patient outcomes are better in hospitals with good nurse work environments, it is not always clear for hospital management and policy makers how to translate this evidence into practice and reform work environments. One way to implement improvements in nurse work environments is through the Magnet recognition program, led by the American Nurses Credentialing Center. Magnet hospitals follow a blueprint, which holds a strong nurse practice environment at its core, to demonstrate nursing excellence through empirical outcomes. Achievement in five components is required to attain Magnet recognition, including (1) transformational leadership, (2) structural empowerment, (3) exemplary professional practice, (4) new knowledge, innovations and improvements, and (5) empirical outcomes (American Nurses Credentialing Center 2013).

Magnet hospitals have been associated with superior patient outcomes, including lower patient mortality (Aiken, Smith, and Lake 1994; McHugh et al. 2013) and higher nurse job satisfaction (Aiken et al. 1999; Gokenbach and Drenkard 2011; Kelly, McHugh, and Aiken 2011). However, very few studies have evaluated patients' experience in Magnet hospitals (Aiken et al. 1999; Gokenbach and Drenkard 2011), and research has only just begun to explore Magnet's relationship with the HCAHPS survey (Smith 2014).

This study evaluates the association between Magnet recognition and patients' experience with their hospitalization, using Donabedian's structure, process, outcomes model as a conceptual guide (Donabedian 1966). The classic model posits that structure influences processes, which ultimately affect outcomes of care. Both structures (i.e., relatively stable features where care is provided, including human and material resources) and processes (i.e., giving care) are transformed to become Magnet recognized.

Because implementing the Magnet blueprint for nurse work environment improvement takes strong institutional commitment, it is possible that better resourced hospitals would apply for and receive this credential. This makes it more complicated to compare patient experience in Magnet hospitals with patient experience in non‐Magnet hospitals because differences that are found between the two groups could be attributed to Magnet recognition or could be due to underlying differences in the hospitals' characteristics. In addition to employing standard regression techniques to estimate difference in patient experience between Magnet and non‐Magnet hospitals, we employed propensity score matching to reduce the differences in measurable hospital characteristics so that we could examine patient experience in comparable sets of Magnet and non‐Magnet hospitals.

Methods

Study Design

This study involves an analysis of observational, cross‐sectional data from a national sample of hospitals derived from three sources in 2010—the HCAHPS survey, American Hospital Association Annual Survey, and the American Nurses Credentialing Center annual report of Magnet hospitals. Data were linked using common hospital identifiers (Medicare Provider Number and hospital names and addresses). The HCAHPS survey evaluates adult patients' experience with their short‐term, acute care hospitalization. Patients with a range of medical, surgical, and obstetrical conditions are randomly surveyed between 2 days and 6 weeks following discharge. The results are collected quarterly and then aggregated annually to the hospital level to produce, for example, the percentage of patients that rate the hospital highly, the percentage that would recommend the hospital, and so on. Eighteen of the questions are used to create the ten publicly available scores found on the Hospital Compare website (Centers for Medicare & Medicaid Services 2013; Department of Health and Human Services 2013). The American Hospital Association surveys all hospitals annually and collects information on hospital structural characteristics, financial status, and other hospital features. The American Nurses Credentialing Center publishes an annual report, which indicates the Magnet recognized hospitals by year of designation (American Nurses Credentialing Center 2012).

Sample

We focused our analysis on nonfederal general acute care hospitals in the United States that had 300 or more patients respond to the HCAHPS survey per CMS's recommendation for reliability (Centers for Medicare & Medicaid Services 2007), had complete characteristics in the American Hospital Association survey, and could be linked to the Magnet dataset. We were able to match 212 Magnet hospitals with a set of 212 non‐Magnets from a pool of 1,886 non‐Magnet hospitals. HCAHPS patient data are available only at the hospital level, so the hospital is the unit of analysis.

Measures

Patient Experience

The HCAHPS survey is composed of 27 questions, and it is administered four ways—via mail, telephone, mail with telephone follow‐up, or interactive voice recognition. In aggregating responses to the hospital level, before release to the public, responses are adjusted for eight patient‐mix variables (linear self‐reported health status, linear education, service line, categorical age, emergency room [ER] admission source, response percentile, service by linear age interactions, and primary language other than English) and for survey mode (Mail Only, Telephone Only, Mail with Telephone follow‐up [also known as Mixed mode], and Active Interactive Voice Response) (HCAHPS 2007). Ten questions are publicly reported as a hospital‐level percentage. Two single questions assess overall experience of care, including “Would you recommend the hospital to family or friends?” and “What number would you rate this hospital during your stay?” on a scale 0–10, with higher numbers reflecting better ratings. The other eight composite items are related to communication with doctors and nurses, responsiveness of hospital staff to patient needs, pain management, education about medications, discharge preparation, and the hospital environment, that is, cleanliness and quietness (Centers for Medicare & Medicaid Services 2013). Before being publicly released, the HCAHPS data are top‐coded. The percentages published for each question are the percentages of patients providing the most positive response. Data from calendar year 2010 were used for this analysis.

Magnet Recognition

Hospitals voluntarily apply for Magnet recognition, going through a rigorous application process to be credentialed, resulting in changes to both structure and process of nursing care. First, the organization conducts a self‐assessment to evaluate each of the five Magnet components. This is followed by an analysis of gaps in any component and an action plan is created to increase performance in the identified areas. After the changes have been made to address the identified areas, the formal application is submitted and fees are paid on a sliding scale. An outside group of appraisers assesses the organization's application and conducts a site visit over several days. Hospitals that are successful are required to provide interim reporting and redesignation is possible every 4 years after a comprehensive evaluation (American Nurses Credentialing Center 2013). Currently less than 8 percent of all U.S. hospitals are Magnet recognized. We used a binary variable to distinguish hospitals that were Magnet with hospitals that were not Magnet.

Hospital Characteristics

Using data from the American Hospital Association annual survey, we measured hospital characteristics based on previous research suggesting an association of these characteristics with Magnet and patient outcomes (Aiken, Smith, and Lake 1994; Kutney‐Lee et al. 2013; McHugh et al. 2013), including structural, ownership, financial, and geographic characteristics. Structural characteristics included number of beds; high technology status, which contrasted hospitals that performed major organ transplants and/or open heart surgery or both; and teaching status, which contrasted major teaching, minor teaching, and nonteaching hospitals based on the number and extent of postgraduate medical residents. Ownership or affiliation characteristics included hospitals that were not‐for‐profit and percent Catholic‐affiliated. Financial characteristics included occupancy rate, payroll expenses per bed, percent of patients with Medicare as primary payer, and percent of patients with Medicaid as primary payer. Lastly, geographic characteristics included urban hospitals based on core‐based statistical areas and rural referral center status.

Analysis

Descriptive statistics were calculated to examine differences in the organizational features of Magnet and non‐Magnet hospitals. Next, we estimated bivariate and multivariate ordinary least squares regression models assessing the association between Magnet recognition and patient experience using the full sample of Magnet and non‐Magnet hospitals, before and after taking account of other hospital characteristics. In an ideal world, we would randomly assign which hospitals are Magnet hospitals and which ones are not, and then observe the differences in outcomes. This is not feasible, so by design we must account for selection differences in the probability of particular hospitals pursuing and attaining Magnet status. Otherwise, these selection differences could confound our results because any outcomes advantage we observe for Magnet compared with non‐Magnet hospitals could be a product of other differences. A traditional regression approach would include these characteristics as covariates, but by including hospitals in the analysis that are fundamentally different from Magnet hospitals, we could be unfairly comparing apples and oranges. The use of propensity score matching accounts for this by creating a comparison group that is similar to the hospitals that are Magnet except that they have no such recognition. This approach “balances” the two groups on measured characteristics, reduces the unmeasured differences between groups, and results in a more precise estimate of the effect of Magnet recognition.

To accomplish the matching, we first calculated a propensity score for each hospital in the sample using a logistic regression model. The model predicts the likelihood of a hospital being a Magnet hospital given a set of observed covariates (Rosenbaum and Rubin 1983). We used the 11 covariates described above (total bed size, occupancy rate, the percentage of patients with Medicare as their primary payer, the percentage of patients with Medicaid as their primary payer, teaching status, high technology status, ownership, Catholic affiliation, average payroll expenses per bed, core‐based statistical area, and rural referral center status) likely to be related to treatment selection, that is, to Magnet recognition. We then tested various matching algorithms and evaluating which one achieved the best balance in the observed hospital covariates using the MatchIt package in R (Ho et al. 2007). We used balance diagnostics such as graphical displays of quantile–quantile plots, histograms, and jitter plots as well as standardized differences in means of the covariates (Austin 2011). The plots and standardized differences in means indicated that greedy nearest neighbor matching achieved the best balance compared to other matching approaches (greedy matching with nearest neighbor with replacement, 1:k matching, and full matching).

Once the matching process was completed, we conducted postmatching regression analyses with our matched sample of hospitals. We first estimated the unadjusted differences in the various patient experience measures between the Magnet hospitals and matched non‐Magnet hospitals. Our final regression models reestimated the differences in those measures between the Magnet hospitals and matched non‐Magnet hospitals after adjusting for the same set of 11 covariates used in the matching process to account for any residual covariate imbalance. All matching and sensitivity analyses were conducted using R and SAS version 9.3 (SAS Institute, Cary, NC, USA). Statistical significance was set at the p < .05 level.

Results

Table 1 describes the characteristics of (1) all the hospitals in this study (n = 2,098), (2) the full set of non‐Magnet hospitals (n = 1,886), (3) the Magnet hospitals (n = 212), and (4) the subset of matched non‐Magnet hospitals (n = 212), determined by the propensity score matching process. Column 1 shows that most of the hospitals in the full sample were large, nonprofit, and located in urban locations. Less than half (40 percent) had high technology available and most (73 percent) were not teaching hospitals. The penultimate column in the table shows that before matching, Magnet hospitals' structural characteristics were significantly different than non‐Magnet hospitals. The final column in the table indicates that, after propensity score matching, the differences in the Magnet and non‐Magnet hospitals markedly decreased. The most dramatic improvements were seen in bed size, teaching status, technology, and urban location. Our use of propensity score matching allowed us to create a set of Magnet and non‐Magnet hospitals appearing almost identical on observable characteristics except for Magnet recognition.

Table 1.

Hospital Characteristics before and after Matching

	(1) All Hospitals (N = 2,098)	(2) Non‐Magnet Hospitals (n = 1,886)	(3) Magnet Hospitals (n = 212)	(4) Matched Non‐Magnet Hospitals (n = 212)	Differences
					(3) − (2)	(3) − (4)
Structural
Average (mean) no. of beds	401	219	425	394	206	31
Percent high technology (%)	40.2	36.4	74.1	75.9	37.6	−1.9
Teaching status (%)
Percent nonteaching	72.6	75.0	50.9	56.6	−24.1	−5.7
Percent minor teaching	18.9	17.9	27.4	25.0	9.4	2.4
Percent major teaching	8.5	7.1	21.7	18.4	14.6	3.3
Ownership/affiliation (%)
Percent not‐for‐profit	85.5	84.3	96.7	97.6	12.4	−0.9
Percent Catholic‐affiliated	15.4	15.2	17.5	18.4	2.2	−0.9
Financial
Average (mean) occupancy rate	129	129	127	127	−2	0
Average (mean) payroll expenses per bed in dollars	346,349	338,878	412,819	413,080	73,941	−261
% Medicare (mean)	52	51	49	49	−2.0	0.0
% Medicaid (mean)	19	19	16	16	−3.0	0.0
Geographic (%)
Percent urban	70.5	68.0	92.5	92.9	24.4	−0.4
Percent rural referral centers	11.7	12.0	9.0	11.8	−3.0	−2.8

The average propensity score is 0.2215 for Magnets and 0.2195 for matched non‐Magnets.

Source: Authors' analysis of data from the American Hospital Association Annual Survey and American Nurses Credentialing Center list of Magnet hospitals.

Figure 1 provides a visual representation of the results of the matching process. The histograms on the left of the figure show the distributions of the propensity scores in the Magnet and non‐Magnet hospitals, before and after matching. There is marked similarity in the distributions after matching. In the right side of the figure, the numbers on the left show the standardized differences in the covariates for the non‐Magnet and Magnet hospitals before propensity score matching, and the numbers on the right show the standardized differences in the covariates for the non‐Magnet and Magnet hospitals after matching. The differences between Magnets and non‐Magnets in the matched sample are much smaller, suggesting successful matching (Stuart 2010).

Figure 1.

Matching Diagnostics Following Propensity Scoring for Magnet Recognition

• Note. The histograms on the left show the distributions of propensity scores for the non‐Magnet and Magnet hospitals before (left) and after (right) matching. The figure on the right shows the absolute standardized difference in means of the covariates before and after matching.

• Source. Authors' analysis of the American Hospital Association Annual Survey and American Nurses Credentialing Center list of Magnet hospitals.

Table 2 shows the results from traditional ordinary least squares regression models before and after matching. In the analyses using all data from the full prematched sample, we found a significant relationship between Magnet recognition and patients' experience with hospital care before and after statistical adjustment for covariates. Specifically, patients who were cared for in Magnet hospitals had higher global assessments of care and more favorable perceptions of nurse communication, explanation of medications, pain control, and discharge instructions. After adjustment for potential hospital confounders, we found that Magnet recognition was associated with a 3 and 4 point increase in the percentage of patients reporting the quality of their hospital care as high (9 or 10) and that they would definitely recommend the hospital.

Table 2.

The Association between Magnet Recognition and Patient Experience of Care

Outcomes (Mean, SD)	Prematching		Postmatching
	Unadjusted Coefficients	Adjusted Coefficients	Unadjusted Coefficients	Adjusted Coefficients
Patients rated the hospital a 9 or 10 (69, 6)	3.5***	3.0***	1.78**	2.03***
Patients would definitely recommend the hospital (74, 7)	6.33***	4.1***	3.0***	3.22***
Nurses communicated well (76, 4)	0.42	1.33***	0.8	0.9*
Staff explained medications (60, 4)	0.02	1.15**	0.65	0.75*
Pain was controlled (69, 4)	0.25	1.09***	0.8*	0.9**
Patients received help as soon as they wanted (61, 6)	−1.59**	0.83	0.2	0.5
Staff gave patients discharge information (83, 4)	1.29***	1.59***	1.65***	1.8***
Room was clean (69, 6)	−1.97***	0.25	−1.4	−1.3
Doctors communicated well (79,4)	−0.9**	0.58	−0.1	0.02
Quiet at night (55, 8)	−2.9***	−0.48	−0.7	−0.72

N = 2,098 in prematching sample, N = 424 in postmatching sample (n = 212 Magnet hospitals, n = 212 non‐Magnet hospitals). Fully adjusted ordinary least squares linear regression models accounted for teaching status, technology available, bed size, ownership, core‐based statistical area (urban), Catholic affiliation, rural referral center, occupancy rate, payroll expenses per bed, percent Medicare, and percent Medicaid.

*p < .05; **p < .01; ***p < .001.

Source: Authors' analysis of data from the Hospital Consumer Assessment of Healthcare Providers and Systems Survey, American Hospital Association Annual Survey, and American Nurses Credentialing Center's list of Magnet hospitals.

After conducting our propensity score matching, we estimated similar regression models. We observed similar, albeit somewhat attenuated effects of Magnet status on patients' care experience in the postmatching regression models. Patients' high rating of the hospital and willingness to recommend the hospital exhibited the largest Magnet effects. As with the prematched sample, physician communication and the cleanliness and quietness of the hospital were not significantly associated with Magnet recognition. Results in the postmatching analyses were substantially the same before and after adjusting for the same covariates used in the matching, which attests to the quality of the matching process.

Discussion

Our study is one of the most comprehensive studies to date to demonstrate an association between Magnet recognition and patients' experience with their hospital care using the HCAHPS survey. We observed that patients who received care in Magnet recognized hospitals reported significantly better care experiences. Patients in Magnet hospitals rated their overall experiences significantly better as reflected by the two global measures of experience with care and more positive experiences with nurse communication.

Magnet recognition is now incorporated into highly visible, national quality assessments, such as U.S. News and World Reports “Best Hospitals” and the Leapfrog Group's annual survey. The Leapfrog Group, which represents public and private purchasers of health benefits, automatically awards Magnet hospitals full credit for the nursing workforce measure. This measure accounts for nurse staffing as well as the hospital's inclusion of nurses in leadership positions (Leapfrog Group 2011). The addition of Magnet status in this survey acknowledges the importance of high‐quality nursing care to overall hospital quality and safety.

The Magnet recognition process emphasizes innovation; evidence‐based, patient‐centered care; and a collaborative culture (American Nurses Credentialing Center 2013), making these likely contributors to the more favorable patient care experience demonstrated in our results. For example, when nurses work in an environment that is well resourced and management is supportive of nurses, nurses are able to spend time assessing and educating patients and families (Ausserhofer et al. 2014), which may explain the better communication, pain control, and medication explanation scores that we observed. Our findings are aligned with previous research showing that nurses in Magnet hospitals are less likely to miss care related to patient education, call‐light response, and other nursing care responsibilities that could directly affect patient experience (Kalisch and Lee 2012).

The observational, cross‐sectional nature of our study limits our ability to determine causality. Replication of this study with new data or a longitudinal design would enhance confidence in a causal relationship between Magnet and patient experience. Data were not available for all factors related to Magnet recognition, including additional nursing characteristics. The HCAHPS are adjusted for patient‐mix and mode of survey response, and recent work has suggested that the additional adjustments for clinical characteristics may not be warranted (Cleary et al. 2014); however, there are potential confounders that we have not accounted for in our regression models. We were not able to control for hospitals that may have applied for Magnet recognition and were not successful. Finally, propensity scoring offers improvement over traditional regression methods, and we were able to take advantage of national, readily available sets of data.

Conclusion

Our findings suggest a significant association between Magnet recognition and a better care experience for patients. With an estimated pool of over $1 billion for performance incentives in FY2014 (Rau 2013) and increases in subsequent years, there is good reason for hospital leaders to be motivated to improve their performance on the HCAHPS survey. Investment in Magnet recognition has been shown to have a number of positive advantages for hospitals, including higher net patient income, reductions of costly nursing turnover, and lower mortality (Aiken, Smith, and Lake 1994; McHugh et al. 2013; Jayawardhana, Welton, and Lindrooth 2014). The results of this paper add yet another advantage—the possibility of a better patient experience—which is good for quality of care and good for hospitals' fiscal health.

Ethical approval: Approval for this study was obtained by the authors' institutional review board.

Supporting information

Appendix SA1: Author Matrix.