Abstract
Objective
To determine the relationship between postoperative morbidity and mortality and patients' perspectives of care.
Summary Background Data
Priorities in healthcare quality research are shifting to place greater emphasis on patient-centered outcomes. Whether patients' perspectives of care correlate with surgical outcomes remains unclear.
Methods
Using data from the Michigan Surgical Quality Collaborative clinical registry (2008–2012), we identified 41,833 patients undergoing major elective general or vascular surgery. Our exposure variables were the HCAHPS Total and Base Scores derived from the Hospital Value-Based Purchasing (VBP) Patient Experience of Care Domain. Using multilevel mixed-effects logistic regression models, we adjusted hospitals' rates of morbidity and mortality for patient comorbidities and case mix. We stratified reporting of outcomes by quintiles of hospitals' Total and Base Scores.
Results
Risk-adjusted morbidity (13.6% to 28.6%) varied widely across hospitals. There were no significant differences in risk-adjusted morbidity rates between hospitals with the lowest versus highest HCAHPS Total Score (24.5% vs. 20.2%, p=0.312). The HCAHPS Base Score, which quantifies sustained achievement or improvement in patients' perspectives of care, was not associated with a reduction in postoperative morbidity over the study period despite an overall decrease of 2.5% for all centers. We observed a similar relationship between HCAHPS Total and Base Scores and postoperative mortality.
Conclusions and Relevance
Patients' perspectives of care do not correlate with the incidence of morbidity and mortality following major surgery. Improving patients' perspectives and objective outcomes may require separate initiatives for surgeons in Michigan.
Introduction
In 2002, the Centers for Medicare & Medicaid Services (CMS) partnered with the Agency for Healthcare Research and Quality (AHRQ) to implement the first national, standardized survey of patients' perspectives of healthcare.1, 2 CMS now reports this information for many acute care hospitals through its Hospital Compare program, which is designed to help patients make decisions about where to obtain care and incentivize providers to improve quality.3, 4 These actions are reflective of shifting priorities in healthcare quality research emphasizing the greater importance of patient-centered outcomes.5 More recently, CMS began using this same data to augment payment to acute care hospitals through its value-based purchasing (VBP) program.6 Within these same hospitals, inpatient surgical care accounts for approximately 40% of annual hospital and physician spending– a figure due in large part to the management of postoperative complications.7, 8
Whether patients' perspectives of care correlate with surgical outcomes remains unclear. Contextually similar work in the surgical literature is limited to specific populations. There are numerous studies correlating objective outcome measures with patient-centered outcomes in cancer patients.9–11 This work suggests a negative correlation between higher patients' quality of life and the incidence of postoperative complications. Beyond the surgical literature, evidence from a large statewide demonstration suggests that patients' perspectives of care can incentivize providers to improve quality in the primary care setting.12, 13 It is unclear whether this information can service as a stimulus for quality improvement in the delivery of surgical care.
Using data from the Michigan Surgical Quality Collaborative (MSQC), we sought to characterize the relationship between hospitals' performance on the Hospital Care Quality Information from the Consumer Perspective (HCAHPS) survey and risk-adjusted outcomes following major surgery. A better understanding of how patients' perspectives of care relate to objective surgical outcomes may align priorities for quality improvement and promote greater attention to patient-centered care.
Methods
Data Source and Study Population
We used data from the Michigan Surgical Quality Collaborative (MSQC) clinical registry to identify patients undergoing major general or vascular surgery within participating hospitals between 2008 and 2012. We restricted our population to those patients who underwent elective operations requiring an inpatient hospitalization of at least 24 hours. The MSQC is a provider-led quality improvement organization funded by Blue Cross and Blue Shield of Michigan. Data for this project employed standard data definitions and collection protocols of the MSQC platform as previously described.14 All available variables were collected for this analysis including patient demographics, preoperative risk factors, laboratory values, perioperative factors, and 30-day postoperative morbidity and mortality. Additional hospital-level data (bed size, hospital occupancy, surgical volume, Council of Teaching Hospital status, pain management services, hospice services, and inpatient social work) was obtained from the American Hospital Association (AHA) Hospital and Health System Data Resources a priori based on plausible relationship with surgical care quality and patients' perspectives.
HCAHPS data are publicly available through the CMS Hospital Compare program and the first reporting occurred in March of 2008.4 The scoring of HCAHPS data is complicated. Briefly, HCAHPS surveys are disseminated to persons recently discharged from an acute care hospital for a medical, surgical, or maternity-related admission. There are ten measures related to patients' perspectives of care which are included in the HCAHPS survey. (Table 1) Each measure is evaluated by two or three items that vary in the level of affirmation for a given statement (e.g. Patients who reported that their doctors Always “communicated well.”). The most affirmative items for each measure, referred to as “top-box” items, are used to calculate the Patient Experience of Care Domain for the Hospital VBP program.15
Table 1.
Summary Measures |
Communication with nurses |
Communication with doctors |
Responsiveness of hospital staff |
Pain management |
Communication about medicines |
Providing discharge information |
Individual Items |
Cleanliness of hospital environment |
Quietness of hospital environment |
Global Items |
Overall rating of hospital |
Willingness to recommend hospital |
The Patient Experiences of Care Domain is comprised of two separate scores: a Base Score and a Consistency Score. The Base Score (80 points maximum) represents an achievement mark (performance relative to the national median) or an improvement mark, calculated by changes in HCAHPS scores between a baseline period (July 2009–March 2010) and a performance period (July 2011–March 2012). The Consistency Score (20 points maximum) is derived from hospitals' scores for their lowest performing HCAHPS measure. The sum of these scores forms the Patient Experiences of Care Domain with a maximum score of 100. We will subsequently reference this as the HCAHPS Total Score. This score augments 30% of the Hospital VBP Total Performance Score. The remaining 70% is derived from Clinical Processes of Care domain.
Outcomes
The primary outcomes for this study were the incidence of 30-day in-hospital morbidity and mortality. We determined various in-hospital postoperative complications such as surgical site infection (superficial, deep, and organ space defined separately), deep venous thrombosis, urinary tract infection, acute renal failure, postoperative bleeding requiring transfusion, stroke, unplanned intubation, fascial dehiscence, prolonged mechanical ventilation over 48 hours, myocardial infarction, pneumonia, pulmonary embolism, sepsis, vascular graft loss, and renal insufficiency.
Statistical Analysis
Pearson chi-squared (X2) test, Student's t-test, and Wilcoxon signed-rank test were used to compare patient characteristics and unadjusted rates of postoperative outcomes. We compared continuous variables using Pearson's correlation coefficient and Cohen's kappa statistic. Multilevel mixed-effects logistic regression models were used to adjust hospitals' morbidity and mortality rates. Final models included patient level risk factors such as age, race, BMI, comorbid conditions, American Society of Anesthesiologists (ASA) class, surgeon specialty, work Relative Value Units (RVU), and do-not-resuscitate (DNR) status as fixed-effects. We accounted for clustering of outcomes within centers using a hospital identifier. Next, we adjusted our estimates for reliability using empirical Bayes techniques as has been previously described.16
We conducted a primary analysis to determine the relationship between hospitals' HCAHPS Total Score and adjusted rates of postoperative morbidity and mortality. We grouped hospitals by quintiles based on their HCAHPS Total Score to improve generalizability and ensure that hospitals' could not be identified given that HCAHPS information is publicly available. For this analysis we evaluated hospitals' adjusted morbidity and mortality rates from March 2008 (the start of HCAHPS public reporting) through March 2012 (the completion of the HCAHPS performance period) in aggregate. We then conducted a sensitivity analysis using hospitals' scores for two separate “top-box” items for overall quality. Items included patients rating the hospital 9 or 10 on a scale from 0 to 10 and patients stating that they would definitely recommend the hospital.
We then conducted a second analysis to investigate whether changes in morbidity and mortality rates within hospitals correlated with the HCAHPS Base Score, which reflects sustained achievement or improvement in patients' perspectives of care between the baseline period and performance period. The baseline period concluded on March 31, 2010. We compared adjusted rates of morbidity and mortality within hospitals and for the total population before and after this date. We calculated absolute differences in outcome rates and reported results across quintiles of hospitals' HCAHPS Base Score.
All statistical analyses were performed using Stata statistical software version 12.1 (College Station, Texas). This study was approved by the University of Michigan Institutional Review Board.
Results
Patient Characteristics
We identified 41,833 patients undergoing major inpatient operations and meeting inclusion criteria with 32 participating hospitals. There were significant differences in patient demographics and comorbid disease burden between the hospitals stratified by quintiles of HCAHPS Total Score. (Table 2) In general, the overall profile of patient comorbidities was not markedly different across quintiles of HCAHPS Total Score. Patients with non-white race were twice as common in the lowest HCAHPS Total Score hospitals when compared to those with the highest HCAHPS Total Score (p<0.001). The distribution of procedure mix between major general versus vascular operations did not vary across quintiles (p=0.11). HCAHPS Total Score (range=55) and Base Score (range=48) were variable across hospitals. The most common general surgery procedure performed was colectomy (n=12,181; 29%), and the most common vascular procedure was abdominal aortic aneurysm repair (n=2,948; 7%).
Table 2.
Lowest (N=9,441) | Low (N=9,518) | Average (N=8,610) | High (N=6,620) | Highest (N=7,644) | p-value | |
---|---|---|---|---|---|---|
Demographics | ||||||
Median age, years (IQR) | 71 (20) | 72 (21) | 74 (21) | 72 (22) | 74 (20) | <0.01 |
Percent male sex | 51% | 51% | 52% | 51% | 53% | 0.08 |
Percent non-white race | 31% | 36% | 26% | 19% | 15% | <0.01 |
Mean body mass index (SD) | 29 (8) | 28 (7) | 29 (8) | 29 (7) | 28 (8) | 0.52 |
Comorbidities and Procedure Mix (%) | ||||||
Diabetes mellitus | 25% | 23% | 29% | 21% | 24% | 0.03 |
Smoking in past year | 31% | 26% | 31% | 27% | 28% | <0.01 |
Do not resuscitate status | 1% | 1% | 1% | 1% | 2% | <0.01 |
Chronic obstructive pulmonary disease | 11% | 10% | 13% | 8% | 11% | 0.88 |
Congestive heart failure | 3% | 1% | 3% | 2% | 1% | <0.01 |
Dialysis | 8% | 5% | 8% | 3% | 5% | <0.01 |
Peripheral vascular disease | 15% | 10% | 18% | 13% | 22% | <0.01 |
Transient ischemic attack | 5% | 4% | 5% | 4% | 7% | <0.01 |
Disseminated cancer | 5% | 4% | 2% | 4% | 3% | <0.01 |
Weight loss prior to surgery (>10%) | 6% | 4% | 3% | 3% | 4% | <0.01 |
Steroid use | 5% | 5% | 4% | 5% | 5% | 0.67 |
Albumin <3.5 g/dl | 31% | 23% | 35% | 24% | 33% | <0.01 |
Independent functional status | 16% | 11% | 12% | 10% | 14% | <0.01 |
Hypertension requiring medication | 66% | 61% | 69% | 60% | 63% | <0.01 |
ASA class ≤ 2 | 28% | 21% | 25% | 37% | 30% | <0.01 |
Major general operations | 64% | 70% | 70% | 72% | 63% | 0.11 |
Major vascular operations | 36% | 29% | 39% | 28% | 37% |
HCAHPS Total Score and Surgical Outcomes
Unadjusted morbidity and mortality rates for the entire patient population were 22.0% and 3.7%, respectively. Adjusted morbidity rates varied widely across hospitals ranging from 13.6% to 28.6%. Adjusted mortality rates across hospitals ranged from 2.7% to 5.2%. When stratifying hospitals in to quintiles by their HCAHPS Total Score, we observed no significant differences in risk adjusted morbidity. (Figure 1) There were no significant differences in risk-adjusted mortality rates between hospitals with the lowest versus highest HCAHPS Total Score (24.5% vs. 20.2%, p=0.312). Further, adjusted mortality rates for hospitals with the lowest HCAHPS Total Score (4.0%; 95%CI 3.2% to 5.1%) were not statistically different from those in hospitals with the highest HCAHPS Total Score (3.3%; 95%CI 2.7% to 4.2%). We did not observe a stepwise decrease or increase in adjusted outcome rates across quintiles of HCAHPS Total Score. When assessing the relationship between patients' perspectives of care and postoperative morbidity in a continuous fashion, we found no correlation (κ=0.03, p=0.155). In Figure 2, we observed a random association between hospitals ranks of HCAHPS Total Score and risk-adjusted morbidity rates. The four quadrants represent hospital performance as high or low for each domain – objective outcomes and patients' perspectives.
HCAHPS Base Score and Outcome Improvement
No statistically significant differences in adjusted mortality rates were observed within hospitals between the HCAHPS baseline and performance periods. Overall mortality for the entire population decreased from 3.8% to 3.5%, though this difference was not statistically significant (p=0.200). In contrast, overall adjusted morbidity rates for the entire population decreased from 23.3% to 20.8% (p<0.001) between the HCAHPS baseline and performance periods. We observed statistically significant improvement in 13 of 32 hospitals (41%) and these centers were equally distributed across quintiles of HCAHPS Base Score. The absolute difference in adjusted morbidity rates was not statistically different between hospitals with the lowest 20% (−2.0%) versus highest 20% of HCAHPS Base Score (−3.4%, p=0.192).
Hospital Characteristics, Patients' Perspectives, and Surgical Outcomes
Finally, we conducted a comparison of hospital characteristics between hospitals with high performance in both domains versus those with low performance described in Figure 2. We found no significant differences in overall size, inpatients surgical volume, average occupancy, and overall annual admissions. We also found that all hospitals in both groups provided resources associated with improved patients' perspectives of care such as inpatient social work, pain management teams, and palliative care/hospice. All hospitals also reported disseminating information to the patient community regarding the quality and cost of hospital care.
Discussion
In this study, we report adverse outcomes of patients undergoing major general or vascular inpatient surgery across Michigan hospitals stratified by their performance on the HCAHPS survey. With the incorporation of patients' perspectives of care into the Hospital VBP Program, CMS places a growing incentive on patient-centered outcomes. We failed to observe a significant relationship between patients' perspectives of care and the incidence of 30-day postoperative morbidity and mortality within a statewide surgical collaborative. When comparing hospitals' adverse event rates between the CMS baseline and performance periods, we also failed to show significant correlation between outcome improvement and sustained or improved patients' perspectives of care. These results indicate that metrics of objective and subjective surgical quality measure equally important, but not necessarily overlapping domains of the surgical experience.
Accompanying the Patient Protection and Affordable Care Act, the Patient Centered Outcomes Research Institute (PCORI) was founded to improve the availability and quality of information provided to healthcare consumers.17 With the development of PCORI, a new era of comparative effectiveness research brought increasing focus on patient-centered outcomes research. Despite this, the utility of publically available patient-centered hospital rankings remains uncertain.18 Previous studies have demonstrated that hospital choice is less influenced by comparative quality metrics, but more so by word of mouth and anecdotal evidence.19, 20 Additionally, the interpretation of quality metrics is highly variable with age, literacy and knowledge of healthcare having huge implications for choice.21 The importance of hospital choice for major operations obligates the surgical community to improve delivery and sharpen the focus of hospital quality metrics.
As a leader in outcome measurement and quality improvement, the field of surgery should continue to focus on improving objective outcomes in conjunction with patient-centered care. Though beyond the scope of this analysis, it is entirely possible that metrics for postoperative morbidity and patient-centered care will not correlate at the hospital level. Changes in care practice that influence postoperative morbidity and mortality may be exclusive from hospitals' cultural changes that improve patients' perspectives of care. Previous collaboratives like MSQC have demonstrated quality improvement through public reporting.22 When the Alliance Group of Madison, Wisconsin publically released hospital quality rankings for hip and knee surgery, cardiac care and obstetrics, significant improvements in quality were seen in those hospitals included in the report.23 Despite this, market share was unaffected by the move and there was no significant change in volume across hospitals. CMS indicates that the reporting of HCAHPS measures may improve outcomes.3 As surgeons navigate between their clinical responsibilities and hospital-wide pressures to improve patients' perspectives of care, it will be important for all parties involved to understand that improvements in one domain of patient may not augment another.
An important qualification of HCAHPS data as reported by Hospital Compare is that it is not unique to particular patient populations. This data can be available for particular demographics or wards of a hospital. However, that information is generally not publicly available is not used to augment reimbursement through the Hospital VBP Program. While one may argue that little can be inferred about surgical care from data that includes patients admitted to medical and obstetrical wards, there are several considerations that refute this assertion. First, the HCAHPS survey assesses many domains that are likely not unique to surgical care (e.g. hospital cleanliness and quietness). Second, hospitals are rewarded or penalized collectively for their HCAHPS scores. Surgical teams whose patients record high HCAHPS scores will see no benefit to this if the overall hospital scores in aggregate are low.15 Most important is the fact that the data used for this analysis is derived from CMS and serves to directly impact hospital reimbursement and inform patients' and providers as to the “quality” of a particular hospital. It will be important to recognize that quality takes many forms that may be exclusive and should be considered independently given the current metrics.
This study has several important limitations. This is a retrospective study and we are not able to assert causation from our findings. Given this study design, our risk-adjustment models are thereby subject to bias from unmeasured confounding, though we have no reason to believe this would be systematically different across different centers. It is plausible that HCAHPS Total and Base Scores may not truly reflect patients' perspectives of care during the study period. It is also possible that sample selection and survey-taker bias confound the interpretation of HCAHPS survey data. However, this information is still being used to augment hospital payments and is publicly available to inform patient hospital choice. The HCAHPS Base Score reflects sustained achievement or improvement, and we are unable to determine which through the available data. Finally, while this may provide an accurate portrayal of the State of Michigan, regional differences in healthcare opinions may limit the generalizability of our results.
In conclusion, the incidence of morbidity and mortality following major surgery does not correlate with patients' perspectives of care from the HCAHPS survey. These data call to question the use of HCAHPS scores to inform patient decision-making and incentivize quality improvement. Further work is needed to appropriately inform efforts to improve patients' perspectives of care and surgical outcomes – as these efforts may be uniquely different in overall approach.
Acknowledgments
Funding: MJE, NIH – NIDDK (K08 DK0827508) and the Blue Cross Blue Shield of Michigan Foundation.
Footnotes
The authors have no conflicts of interest to disclose.
References
- 1.Goldstein E, Fyock J. Reporting of CAHPS quality information to medicare beneficiaries. Health Serv Res. 2001;36:477–488. [PMC free article] [PubMed] [Google Scholar]
- 2.Hargraves JL, Hays RD, Cleary PD. Psychometric properties of the Consumer Assessment of Health Plans Study (CAHPS) 2.0 adult core survey. Health Serv Res. 2003;38:1509–1527. doi: 10.1111/j.1475-6773.2003.00190.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. [Accessed July 29, 2013];Care Quality Information from the Consumer Perspective Hospital Survey. Available at: http://www.hcahpsonline.org/home.aspx.
- 4. [Accessed July 29, 2013];Medicare Hospital Compare. Available at: http://www.medicare.gov/hospitalcompare/search.html.
- 5.Selby JV, Beal AC, Frank L. The Patient-Centered Outcomes Research Institute (PCORI) national priorities for research and initial research agenda. JAMA. 2012;307:1583–1584. doi: 10.1001/jama.2012.500. [DOI] [PubMed] [Google Scholar]
- 6.VanLare JM, Conway PH. Value-based purchasing--national programs to move from volume to value. N Engl J Med. 2012;367:292–295. doi: 10.1056/NEJMp1204939. [DOI] [PubMed] [Google Scholar]
- 7.Birkmeyer JD, Gust C, Baser O, et al. Medicare payments for common inpatient procedures: implications for episode-based payment bundling. Health Serv Res. 2010;45:1783–1795. doi: 10.1111/j.1475-6773.2010.01150.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Birkmeyer JD, Gust C, Dimick JB, et al. Hospital quality and the cost of inpatient surgery in the United States. Ann Surg. 2012;255:1–5. doi: 10.1097/SLA.0b013e3182402c17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Karanicolas PJ, Graham D, Gonen M, et al. Quality of life after gastrectomy for adenocarcinoma: a prospective cohort study. Ann Surg. 2013;257:1039–1046. doi: 10.1097/SLA.0b013e31828c4a19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Lee ES, Lee MK, Kim SH, et al. Health-related quality of life in survivors with breast cancer 1 year after diagnosis compared with the general population: a prospective cohort study. Ann Surg. 2011;253:101–108. doi: 10.1097/sla.0b013e3181f662ce. [DOI] [PubMed] [Google Scholar]
- 11.Andersson J, Angenete E, Gellerstedt M, et al. Health-related quality of life after laparoscopic and open surgery for rectal cancer in a randomized trial. Br J Surg. 2013;100:941–949. doi: 10.1002/bjs.9144. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Rodriguez HP, von Glahn T, Elliott MN, et al. The effect of performance-based financial incentives on improving patient care experiences: a statewide evaluation. J Gen Intern Med. 2009;24:1281–1288. doi: 10.1007/s11606-009-1122-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Rodriguez HP, von Glahn T, Rogers WH, et al. Organizational and market influences on physician performance on patient experience measures. Health Serv Res. 2009;44:880–901. doi: 10.1111/j.1475-6773.2009.00960.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Campbell DA. Accelerating the pace of surgical quality improvement: the power of hospital collaboration. Arch Surg (Chicago. 1960) 2010;145:985–991. doi: 10.1001/archsurg.2010.220. [DOI] [PubMed] [Google Scholar]
- 15.HCAHPS Fact Sheet. Centers for Medicare & Medicaid Services; 2012. [Google Scholar]
- 16.Dimick JB, Ghaferi AA, Osborne NH, et al. Reliability adjustment for reporting hospital outcomes with surgery. Ann Surg. 2012;255:703–707. doi: 10.1097/SLA.0b013e31824b46ff. [DOI] [PubMed] [Google Scholar]
- 17.Methodology Committee of the Patient-Centered Outcomes Research I Methodological standards and patient-centeredness in comparative effectiveness research: the PCORI perspective. JAMA. 2012;307:1636–40. doi: 10.1001/jama.2012.466. [DOI] [PubMed] [Google Scholar]
- 18.Marshall MN, Shekelle PG, Leatherman S, et al. The public release of performance data: what do we expect to gain? A review of the evidence. JAMA. 2000;283:1866–1874. doi: 10.1001/jama.283.14.1866. [DOI] [PubMed] [Google Scholar]
- 19.Schwartz LM, Woloshin S, Birkmeyer JD. How do elderly patients decide where to go for major surgery? Telephone interview survey. BMJ. 2005;331:821. doi: 10.1136/bmj.38614.449016.DE. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Huppertz JW, Carlson JP. Consumers' use of HCAHPS ratings and word-of-mouth in hospital choice. Health Serv Res. 2010;45:1602–1613. doi: 10.1111/j.1475-6773.2010.01153.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Zwijnenberg NC, Hendriks M, Damman OC, et al. Understanding and using comparative healthcare information; the effect of the amount of information and consumer characteristics and skills. BMC Med Inform Decis Mak. 2012;12:101. doi: 10.1186/1472-6947-12-101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Lamb GC, Smith MA, Weeks WB, et al. Publicly reported quality-of-care measures influenced Wisconsin physician groups to improve performance. Health Aff (Millwood) 2013;32:536–543. doi: 10.1377/hlthaff.2012.1275. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Hibbard JH, Stockard J, Tusler M. Hospital performance reports: impact on quality, market share, and reputation. Health Aff (Millwood) 2005;24:1150–1160. doi: 10.1377/hlthaff.24.4.1150. [DOI] [PubMed] [Google Scholar]