Patient Satisfaction and Quality of Surgical Care in US Hospitals

Introduction

Delivering patient-centered care is an important component of a high-quality health care system.¹ The importance of patient experience as a key quality metric has been underscored by inclusion of Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) scores into the Centers for Medicare and Medicaid Services (CMS) Value-Based Purchasing Program (VBP).² In 2013, the VBP program put at risk 1% of total Medicare payments, a number that will rise to 2% by 2017; thus, poor performance on patient satisfaction metrics may represent a substantial financial risk for hospitals.

Because of the lack of evidence on the relationship between patient experience and the ultimate goal of improving patient outcomes after surgery, the use of patient experience as a measure of quality in the VBP has been controversial. Critics argue that patients respond to “concierge” services and not the quality of actual clinical care provided, while advocates counter that the measures focus on distinct features of clinical care that only the patient can report. This tension may be particularly salient for surgical care, where intraoperative technical skill of the surgeon—not captured by surveys of patient experience—can have a profound impact on morbidity and mortality after a major procedure.³ While the non-patient experience aspect of VBP provide incentives to hospitals based on adherence to process measures such as the Surgical Care Improvement Project (SCIP), process measures have been shown to have limited relationships with outcomes,⁴ further fueling the concern that the HCAHPS component of VBP may not only do little to reward quality, but may provide unintended consequences.⁵ If national efforts to promote patient experience are punishing the best hospitals, i.e. those with the lowest mortality and complication rates, because those institutions are not as focused on patient experience, the policy could have potentially detrimental effects on surgical care.

While this concern is widely discussed, we know little about the relationship between hospital performance on patient satisfaction with two important aspects of surgical care—efficiency and quality. One recent study showed no correlation between adherence to evidence-based surgical processes and patient experience, although the study focused on a small number of select hospitals.⁶ As national policy efforts, such as VBP and the Hospital Readmissions Reduction Program (HRRP) expand their focus towards surgical care, understanding the potential tradeoffs between patient satisfaction and measures of efficiency and surgical quality become even more important. Because federal policy on quality has been largely driven by data from medical conditions, there is a need to understand the policy impacts on surgical care given the different relationship that quality metrics such as readmissions have with surgical and medical conditions.⁷ It is unknown if hospitals that perform well on quality incentives designed for medical conditions would also perform well for surgical conditions. Empirical data on patient experience and surgical outcomes from a nationally representative group of U.S. hospitals that perform major surgeries would fill a critical gap in knowledge.

Therefore, in this study, we focused on U.S. hospitals that perform major surgical procedures and sought to answer three questions. First, what are the structural features of hospitals with high patient satisfaction and how do they compare to those with low satisfaction? Second, are hospitals with better scores on patient experience measures more efficient for surgical care, as measured by length of stay for major procedures? And finally, do hospitals that score the best on patient experience provide higher quality surgical care—adherence to surgical processes of care, mortality rates, and readmission rates?

Study Data and Methods

Data

We linked together the 2010 and 2011 Medicare 100% inpatient claims data with the American Hospital Association (AHA) annual survey on hospital characteristics and the Hospital Compare data which contains performance on surgical quality measures as well the HCAHPS survey from 2010 and 2011.

We then used ICD-9 procedure codes on our discharge-level inpatient data set to identify patients that underwent any of six major surgical procedures: coronary artery bypass grafting (CABG), pulmonary lobectomy, endovascular aortic aneurysm repair (EVAR), open abdominal aortic aneurysm repair (AAA), colectomy, and hip replacement. These procedures were selected because they are common, costly, and associated with substantial morbidity and mortality in the Medicare population. Additionally, these procedures reflect a range of surgical subspecialties—cardiac, thoracic, vascular, colorectal, and orthopedic—that would generalize our findings to the spectrum of inpatient surgical care.

Patients undergoing concurrent valve repairs were excluded from the CABG sample. Lobectomy, EVAR, AAA, and colectomy procedures were restricted by ICD-9 diagnosis codes for lung cancer, non-ruptured aneurysms, and colorectal cancer, respectively, to allow for more clinically relevant comparisons. Patients undergoing procedures performed during December of each year were also excluded because we lacked the data to capture readmissions that occurred in the following calendar year. Additionally, as is standard practice with these data, we excluded patients who were only enrolled in fee-for-service for part of the year, those discharged from a federal hospital, and those discharged from hospitals outside the 50 United States or the District of Columbia.

Variables

Our primary predictor was hospital performance on patient satisfaction as measured by the percentage of a hospital’s patients that would “definitely recommend” a hospital. Although the HCAHPS survey asks patients about their experiences with a hospital across eight domains of care, two summary measures of patient experience are reported by CMS: a global rating of the hospital from 0 to 10, and if a patient would recommend the hospital to family and friends, with the possible responses of “definitely yes,” “probably yes,” “probably no,” or “definitely no”.^{6, 8} The data are adjusted for patient-level demographics and the survey administration mode.^{9, 10} The two summary measures of patient experience (global rating of 9 or 10 and percentage of patients who would “definitely recommend” a hospital) were highly correlated (r=0.91). Therefore, we focused on the percent of patients that would “definitely recommend” a hospital as our primary predictor of patient satisfaction. Because we used data from 2010 as well as 2011, we used the average of the summary patient satisfaction score from the two years as our primary predictor.

From the AHA annual survey, we identified key structural characteristics of interest including hospital size, teaching status, region, and ownership (non-profit, for-profit, and public). Using a well-published methodology, we calculated the percent of discharges that are related to surgical care.^11–13 The hospital-level percent of surgical discharges were then included as a covariate in our multivariate regression models. Because the HCAHPS survey is a random sample of hospital discharges, adjusting for the percent surgical discharges in our model would approximate adjusting for the percent of HCAHPS responses that are from surgical patients.

Our measure of efficiency of care was risk-adjusted length of stay. Length of stay was derived from the discharge-level dataset for each procedure and risk-adjusted using the Elixhauser risk-adjustment approach, a well-validated and commonly used approach developed by the Agency for Healthcare Research and Quality (AHRQ).^{14, 15} The approach uses patient demographics as well as the presence or absence of up to 29 comorbidities. We utilized a multivariable Poisson regression accounting for the patient-level Elixhauser comorbidities to produce the expected length of stay. The risk-adjustment model was clustered at the level of the hospital to produce robust standard errors. We then calculated observed-to-expected ratios for each hospital which were multiplied to the national average length of stay to arrive at risk-adjusted length of stay for each hospital We used indirect standardization to create a composite risk-adjusted length of stay across the six procedures.

We examined three primary dependent variables representing well-established measures of surgical quality: the Surgical Care Improvement Project (SCIP) surgical process score, risk-adjusted 30-day readmission rate, and risk-adjusted perioperative mortality rate. We chose these three measures of quality for several reasons. First, they represent process, structural, and outcome domains of hospital quality in the Donabedian framework.¹⁶ Secondly, these quality measures directly represent components of key national policy initiatives, specifically the Value-Based Purchasing Program (VBP) and the Hospital Readmissions Reduction Program (HRRP).²

We calculated procedure-specific risk-adjusted 30-day readmission and mortality rates for each of the six procedures. Similar to our risk-adjustment model for length of stay, readmission rates were risk-adjusted using the Elixhauser risk-adjustment approach accounting for clustering at the hospital to arrive at observed-to-expected ratios for each measure. We then built a composite readmission rate across all six procedures using indirect standardization.⁷

Because our main analyses focused on a series of different metrics (SCIP, mortality, readmissions, and length of stay) that reflect different facets of quality, we also created a composite measure of quality by standardizing the individual surgical measures to the same scale using z-scores, which were weighted equally across all four measures. We inverted the z-scores for the SCIP measure in order to align the directionality of SCIP (where higher adherence is better) to the other measures (where lower readmissions, mortality, and length of stay are better).

Analysis

Our main unit of analysis was the hospital. We first plotted the distribution of patient satisfaction scores across US hospitals that performed any of our six procedures of interest to illustrate variations in patient satisfaction scores. We next compared the characteristics, including size, profit status, teaching status, region, nurse-ratio, and percentage of Medicare and Medicaid patients, of the hospitals above and below the median patient satisfaction scores to understand the key structural features of hospitals associated with patient satisfaction scores. Chi-squared and Wilcoxon tests were used to compare across the two groups for categorical and continuous variables, respectively.

We modeled the relationship between patient satisfaction and our outcome of risk-adjusted length of stay using a linear regression. This model included patient satisfaction as the main predictor while adjusting for bed size, profit status, teaching status, region, rural location, and percent of surgical discharges.

We then built three separate multivariable linear regression models for our three dependent variables of surgical process score, risk-adjusted surgical readmission rate, and risk-adjusted mortality rate. For the readmission and mortality models we further adjusted for the SCIP surgical process score as SCIP measures and patient experience represent the technical as well as inter-personal dimensions of process quality, and we were interested in assessing the independent impact of patient experience with the readmissions and mortality measures. These models again included hospital characteristics including the percent of discharges from each hospital that were surgical. To assess the relationship between patient experience and a composite measure of surgical quality, we used a multivariate linear regression model with the surgical quality z-score (a composite of SCIP performance, mortality rate, readmission rate, and length of stay) as our dependent variable.

All multivariable regression models were weighted by the six-procedure composite surgical volume to account for random fluctuations in performance on quality metrics for hospitals that serve a limited number of patients. For multi-level categorical predictors and covariates, p-values for trend are presented. All analyses were performed using Stata 12. This study was approved by the Office of Human Research Administration and the Institutional Review Board at the Harvard School of Public Health.

Sensitivity Analysis

Because the HCAHPS survey produces two main summary measures of patient satisfaction—the percentage of patients “definitely recommending” a hospital and the percentage of patients rating a hospital 9 or 10 out of 10—we performed an additional set of sensitivity analyses using the percentage of patients rating hospitals a 9 or 10 as the predictor of interest. We also performed procedure-specific analyses as indirect standardization could potentially overweight the contribution of one surgical procedure over another. We also used individual HCAHPS domains as a predictor to assess the relationship between domains of patient satisfaction and surgical quality. We were concerned that case-mix may be affect performance on patient experience, so in addition to employing the Elixhauser risk-adjustment approach for our measures of surgical quality, we further performed additional analyses that adjusted for the hospital-level case-mix index. To account for the possibility that hospitals that have favorable outcomes for Medicare patients may not necessarily have favorable outcomes for non-Medicare patients, we performed additional analyses that adjusted for the percent of Medicare discharges at each hospital.

Study Results

Hospital Characteristics and Performance on Patient Satisfaction

There were 2,953 hospitals that performed at least one of the six procedures on Medicare patients between 20010 and 2011. Patient satisfaction scores ranged from 33.5% to 98.5% among these hospitals and the median patient satisfaction score was 69.5% (Interquartile Range 63%–75.5%, Figure 1).

Figure 1.

Distribution of Patient Satisfaction Score Across US Hospitals Performing Major Surgery

Hospitals with the highest patient satisfaction scores were on average larger than those with the lowest patient satisfaction scores (204 beds vs. 140 beds, p<0.001, Table 1). Highest performing hospitals were more likely to be non-profit (75.3% vs. 54.0%, p<0.001), less likely located in a rural setting (3.3% vs. 11.7%, p<0.001), and more likely to be a major teaching hospital (13.5% vs. 4.9%, p<0.001). Hospitals with the highest patient satisfaction scores also had higher nurse-to-census ratios (8.3 vs. 6.0, p<0.001), lower percentages of Medicaid (13.7% vs. 20.1%, p<0.001), and lower percentages of Medicare patients (45.3% vs. 47.9%, p<0.001).

Table 1.

Characteristics of hospitals by performance on patient satisfaction

Hospital Characteristics	Lowest Quartile of Patient Satisfaction (n=750), [33.5%–63%]	2nd Quartile of Patient Satisfaction (n=768), [63.5%– 69.5%]	3rd Quartile of Patient Satisfaction (n=740), [70%– 75.5%]	Highest Quartile of Patient Satisfaction (n=695), [76%–98.5%]	P- Value
Hospital Bed Size (median, IQR)	140 [88, 227]	174 [90, 280]	205 [102, 342]	204 [88, 392]	<0.001
Ownership
For Profit	28.9%	21.0%	14.9%	13.2%	<0.001
Non-Profit	54.0%	61.1%	71.6%	75.3%
Public	17.1%	18.0%	13.5%	11.5%
Rural	11.7%	8.1%	5.8%	3.3%	<0.001
Major Teaching Hospital	4.9%	7.8%	10.5%	13.5%	<0.001
Region
Northeast	21.7%	15.3%	16.0%	11.9%	<0.001
Midwest	18.3%	24.6%	26.2%	26.9%
South	40.7%	42.6%	39.9%	39.0%
West	19.3%	17.5%	18.0%	22.2%
Intensive Care Unit	72.1%	78.7%	87.2%	83.5%	<0.001
Nurse-to-census Ratio (median, IQR)	6.0 (4.7, 7.5)	6.6 (5.3, 8.3)	7.2 (5.9, 9.0)	8.3 (6.8, 10.1)	<0.001
Percent Medicare (median, IQR)	47.9% (42.5%, 54.1%)	47.9% (32.8%, 53.8%)	47.8% (41.0%, 53.0%)	45.3% (39.3%, 51.8%)	<0.001
Percent Medicaid (median, IQR)	20.1% (15.9%, 25.5%)	18.2% (14.6%, 24.6%)	16.6% (12.0%, 21.5%)	13.7% (8.5%, 19.0%)	<0.001
Disproportionate Share Index (median, IQR)	29.6% (21.3%, 41.9%)	26.6% (18.9%, 35.8%)	22.7% (16.6%, 31.5%)	19.4% (10.7%, 27.5%)	<0.001

The Relationship between Patient Experience and Efficiency

In analyses that focused on length of stay as our outcome, adjusted for hospital characteristics and procedure volume, we found that higher patient satisfaction was related to shorter lengths of stay following major surgery. Hospitals in the highest quartile of performance on patient satisfaction had length of stay that was, on average, 0.6 days shorter than those with the lowest patient satisfaction (7.1 days versus 7.7 days, p<0.001, Figure 2).

Figure 2.

Multivariate Regression Analysis of Patient Satisfaction and Risk-Adjusted Length of Stay

Models adjust for bed size, profit status, teaching status, region, rural location, and percent surgical discharges. P-value for trend is <0.001.

The Relationship between Patient Experience and Surgical Process, Mortality, and Readmissions

We found that when we accounted for differences in hospital characteristics and volume, hospitals in the highest quartile of patient satisfaction scores had slightly higher SCIP process scores compared to hospitals in the lowest quartile of patient satisfaction scores (96.5 vs. 95.5, p<0.001, Table 2). Patient satisfaction was closely related to risk-adjusted 30-day readmission rates, with hospitals with the highest patient satisfaction scores having lower readmission rates compared to hospitals with the lowest patient satisfaction scores (12.3% vs. 13.6%, p<0.001). Finally, patient satisfaction was also related to risk-adjusted perioperative mortality rates, with hospitals with the highest patient satisfaction scores having the lowest 30-day mortality rates (3.1% vs. 3.6%, p<0.001).

Table 2.

Multivariate Analysis of Patient Satisfaction and Quality of Surgical Process and Outcomes

	Multivariate Adjusted Surgical Process Score	P-Value	Multivariate Adjusted Surgical Readmission Rate	P- Value	Multivariate Adjusted Surgical Mortality Rate	P- Value
Patient Satisfaction
Lowest Quartile	95.5	<0.001	13.6%	<0.001	3.6%	<0.001
2nd Quartile	95.8		12.8%		3.3%
3rd Quartile	96.1		12.6%		3.4%
Highest Quartile	96.5		12.3%		3.1%

Three separate multivariate regression models with Surgical Care Improvement Project (SCIP) Process Score, risk-adjusted surgical readmission rate, and risk-adjusted perioperative mortality rate as the dependent variables and patient satisfaction score as the predictor. Patient satisfaction corresponds to percentage of patients that would “definitely recommend” a hospital by the HCAHPS survey. Models adjust for bed size, profit status, teaching status, region, rural location, and percent surgical discharges. The readmissions and mortality models additional adjust for performance on the SCIP process score. P-value for trend is <0.001.

The Relationship between Patient Experience and a Composite Measure of Surgical Quality

Because performance on length of stay, process, mortality, and readmissions represent different dimensions of surgical quality, we built a composite z-score of surgical quality to assess if hospitals that performed well on patient experience would also perform well on all measures of surgical quality together. The z-score allowed for comparisons across different units of measurement, with a lower z-score representing better quality, as lower mortality and readmission rates and shorter length of stays are considered markers of higher quality hospitals. We found no tradeoffs between patient experience and surgical quality. Hospitals with the highest patient satisfaction scores had the highest composite surgical quality (−0.145 vs. −0.010, p <0.001, Table 3).

Table 3.

Multivariate Analysis of Patient Satisfaction and Standardized Composite Measure of Surgical Quality

	Bivariate Composite Standardized Surgical Quality Score	P-Value	Model I Multivariate Composite Standardized Surgical Quality Score	P-Value	Model 2 Multivariate Composite Standardized Surgical Quality Score	P-Value
Patient Satisfaction
Lowest Quartile	0.226	<0.001	0.113	<0.001	−0.010	<0.001
2nd Quartile	0.019		0.006		−0.079
3rd Quartile	−0.078		−0.040		−0.105
Highest Quartile	−0.182		−0.088		−0.145

The dependent variable reflects a composite standardized surgical quality with equal weights for z-scores from the SCIP surgical score, risk-adjusted readmissions, risk-adjusted mortality, and risk-adjusted length of stay measures. SCIP z-scores were inverted to match the directionality of the other measures; therefore, more negative z-scores represent higher surgical quality. All models adjust for bed size, profit status, teaching status, region, rural location, and percent surgical discharges. Model 1 represents unweighted results, while Model 2 represents results weighted by hospital composite surgical procedure volume.

Sensitivity Analyses

In our sensitivity analyses that used the percentage of patients giving the hospital a 9 or 10 as the predictor of interest, we found nearly identical results: better scores were associated with higher performance on process measures, lower readmission rates, and lower mortality (Table, Supplemental Digital Content 1). They were also correlated with shorter lengths of stay. Procedure-specific as well as domain-specific results were also similar in pattern to our main results (Table, Supplemental Digital Content 2 and Table, Supplemental Digital Content 3). We also examined the relationship between hospital-level case-mix and performance on patient satisfaction. Hospitals that performed better on HCAHPS had a higher case-mix index (Table, Supplemental Digital Content 4), therefore, additional adjustment through hospital-level case-mix strengthened the effect of the relationship between HCAHPS and surgical quality (Table, Supplemental Digital Content 5). Lastly, adjusting for the percent of Medicare patients did not substantively change the nature of the relationship between HCAHPS and surgical quality (Table, Supplemental Digital Content 6).

Discussion

Among U.S. hospitals that perform major surgeries, we found large variations in patient satisfaction, although the general level of satisfaction was reasonably high. Hospitals with high patient satisfaction provided more efficient care with shorter lengths of stay for surgical patients. These hospitals also had higher surgical process quality, lower surgical readmission rates, and lower surgical mortality rates.

This set of analyses is particularly salient in light of current federal policy efforts. CMS has two major mandatory changes to its payment program for all U.S. hospitals: VBP and HRRP. These two programs focus primarily on four measures: patient experience, process scores such as performance on SCIP, mortality rates, and readmission rates. Both of these federal payment programs are increasingly focused on incorporating surgical care. Further, the ACA makes substantial payment cuts to hospitals, putting pressure on hospitals to reduce costs, by potentially discharging patients early to reduce LOS. The combination of these efforts potentially creates tensions and choices for hospitals and surgeons.

As reimbursements become linked to both performance on patient experience and quality, there is a need to ascertain that federal incentive programs are not working at cross-purposes. This becomes especially salient for minority-serving or safety-net hospitals which may be disproportionately penalized.¹⁷ If performance on patient experience were in the opposite direction of surgical quality, then financial penalties based on patient experience may not only put further strain on safety-net hospitals, but may have the potential to actually worsen quality of care for vulnerable patient populations as administrators direct resources away from quality improvement to patient experience efforts. Our findings of a significant positive relationship between patient experience and a composite measure of surgical quality aggregating multiple domains of efficiency and quality suggest that incentives to improve surgical patient experience and surgical quality are all truly aligned. As national policy efforts move towards a greater focus on surgical care, our findings are reassuring – indeed, there need not be a tradeoff between good quality of care for surgical patients and ensuring a positive patient experience.

We are unaware of any study that has previously examined whether hospital-level patient experience is related to length of stay for medical or surgical conditions. Our finding that hospitals with higher patient satisfaction were more likely to also be more efficient has important implications as federal policymakers reduce payments to hospitals through VBP, HRRP, and other financing policies aimed at reducing the escalating cost of health care in the Medicare system. Despite what many fear, VBP incentives for improving patient experience should have no disincentives for hospitals that are trying to manage payment cuts through the provision of more efficient care.

The direct relationship between patient experience and surgical quality remain unclear. It is possible that patients are responding to the lower complication rates that these institutions have, although most patients do not suffer a major complication, even at the lowest quality hospitals.^{18, 19} Therefore, patients’ own personal experiences with adverse events are unlikely alone to explain the entirety of our findings. More likely, patients are responding to other signals that might be present in a high quality hospital—better nursing care, faster relief of pain – signals that are likely to be correlated with better surgical outcomes. High quality institutions may have better clinical workflows, better communications, and they may be more attuned to patient needs. The current policy efforts are attempting to nudge hospitals towards paying greater attention to these issues.

Examining the relationship between patient satisfaction and quality for surgical patients is important because surgical care is different from medical care. Previous studies on medical conditions have examined the relationship between patient experience and compliance with process measures^{8, 20, 21} as well mortality rates with similar findings.²² However, practice patterns, training of providers, and the relative impact of technical skills and hospital resources on outcomes can vary greatly between surgical and medical care. The data on the relationship between patient satisfaction and process quality for surgical care has been limited. One recent study showed no correlation between adherence to evidence-based surgical processes and patient experience, although it was limited by a focus on a select group of hospitals and may have been underpowered to find modest differences.⁶ Our data, based on a national sample of U.S. hospitals that perform major surgeries for Medicare beneficiaries, not only has more statistical power, but more importantly expands on prior work by examining other important measures of surgical performance such as mortality,²³ readmissions,^{24, 25} and length of stay.^{26, 27} As policymakers expand federal policy initiatives on quality towards surgical care, there is an important need for empirical data to guide both the implementation of policy as well as innovation of delivery by surgeons and hospital administrators. Our findings further bolster the hypothesis that there are important hospital-level differences in quality that likely affect surgical care and patient experience.

There are limitations to our study. Because we used administrative data, we were unable to adequately account for factors not captured by billing codes. However, we would expect that inadequate risk adjustment would likely diminish our ability to find an association. Another concern is that we focused on the Medicare population, and therefore cannot say whether our findings would extend to non-elderly Americans, although Medicare recipients do receive a large proportion of these procedures and suffer the highest rates of associated complications. Because this was an observational study, we could not identify the mechanisms underlying the relationships between patient satisfaction and quality of surgical care. We carried out a series of analyses to identify whether there are measurable confounders that explain the relationship, including performing a Hausman test, which did not identify any such confounders. That said, these efforts at accounting of confounding are imperfect and the relationships we identified are unlikely to be causal. Instead, they likely reflect the experience of being in a higher quality institution.

There are also limitations to the HCAHPS survey itself, as there may be selection bias in the patients that choose to respond to the survey. Significant variations in response rates across hospitals as well as racial differences in perceptions of care may also influence a hospital’s overall patient satisfaction rating.^{28, 29} Despite these limitations, we rely on the HCAHPS survey as our measure of patient satisfaction as it is a national survey which allows comparisons across hospitals and is also the most policy-relevant survey of patient satisfaction. The HCAHPS survey of patient experience is a random sample of all discharges from a hospital, and although we adjusted for the percent of surgical discharges for each hospital, our findings reflect overall patient satisfaction at a hospital instead of the specific experience of surgical patients at a hospital.

In conclusion, we found marked variation in patient satisfaction across U.S. hospitals performing major surgery. While our results do not imply causality between patient experience and quality and efficiency of care, patient satisfaction was associated with both the quality and efficiency of surgical care, with high patient satisfaction hospitals having higher process quality, lower readmission rates, lower mortality rates, and shorter length of stay. Incentives designed at improving patient satisfaction and at improving surgical quality would not force hospitals to choose between better care and more responsive care.