Patient-Reported Outcomes: Understanding Surgical Efficacy and Quality from the Patient’s Perspective

Abstract

In surgery, quality assessment encourages improved care delivery, better outcomes, and helps determine surgical efficacy. Quality is important from a patient, provider, payer, and policy maker standpoint. However, given the growth of outpatient procedures, expansion of surgical indications to enhance function, and the decline of perioperative morbidity and mortality, many traditional quality metrics, such as mortality, re-admissions, and complications, may not fully capture quality. As such, patient-reported outcomes can be used to complement the established clinical outcomes and describe surgical efficacy and quality from the patient’s point of view. Generic and disease-specific patient-reported outcome measures capture health-related quality of life, functional status, and pain. These measures permit a more holistic understanding of how surgery affects different aspects of a patient’s health, augment other clinical outcomes, and are commonly used to determine efficacy in clinical trials. Moreover, our national reimbursement structure is currently evolving to include patient-reported outcomes for certain surgical conditions in measures of quality and with direct linkage to payments. Yet, even so, there continues to be challenges in implementation of patient-reported outcome measures in everyday surgical practice with questions of optimal administration and how to integrate these measures into provider work flow. Despite these challenges, patient-reported outcomes provide vital information regarding surgical efficacy and quality and are critical in the delivery of patient-centered care.

Introduction

National healthcare reform aims to provide patients with high quality care. Attention to the importance of quality accelerated in 2001 when the Institute of Medicine released “Crossing the Quality Chasm: A New Health System for the 21^st Century,” offering a framework to improve the safety and quality of patient care. Six priorities were identified to enhance the delivery of healthcare and guide policymaking: safe, effective, timely, efficient, equitable, and patient-centered care.¹ These goals intend to provide patients with better and safer care with improved outcomes. With this in mind, defining quality is critical for patients, providers, payers, and policy makers. Accurate and transparent measures of quality can allow patients to make informed decisions regarding healthcare and promote shared decision-making with providers. For hospitals and providers, quality helps to identify opportunities to develop efficient and timely care delivery, enhance the well-being of patients, and provide patient-centered healthcare. By establishing quality benchmarks, accepted metrics for comparison among providers and hospitals can highlight areas of improvement.

Quality is also central for healthcare payment and physician reimbursement. For payers, robust quality metrics can evaluate effectiveness and return on investment of new technologies and treatments compared with current standards of care. The Centers of Medicare and Medicaid (CMS) have included quality as a criterion for payment, serving as a way to incentivize high performance for providers, hospitals, and health systems. For example, in 2015, the Medicare Access and CHIP Reauthorization Act (MACRA) created explicit links to connect high quality healthcare with reimbursement. Through the Quality Payment Program and Merit-based Incentive Payment System (MIPS), all providers, including those practicing in academic health systems and private practices, are financially rewarded for delivering care that achieves specific quality benchmarks with the goal of continuous improvement.² However, the gold-standard by which quality can be considered remains elusive. Quality can be assessed through multiple metrics, ranging from clinical outcomes such as morbidity and mortality to patient experiences, and varies by the nuances and context of clinical care. Though quality is important to key stakeholders including patients, providers, policy makers, and payers, how best to measure quality continues to be debated, and has profound implications on the delivery of care and the outcomes of patients (Figure 1).

Figure 1:

Priorities of Key Stakeholders in Quality Assessment

For surgical disciplines, measures of quality and clinical effectiveness have historically been determined by clinical outcomes, such as mortality, postoperative complications, and readmission rates, which are easily captured in clinical care and have high face validity.^3,4 Moreover, clinical outcomes are objective, easily quantified, and understandable to both patients and providers. However, surgery has become exceedingly safe, rendering some clinical outcomes like mortality obsolete for minimal risk surgery.⁵ Additionally, many surgical procedures are performed on an outpatient basis with few complications necessitating readmission or inpatient care. For this type of care, clinical outcomes may not reflect actual clinical efficacy nor quality.^6,7 Finally, a substantial number of surgical procedures are performed for improvement in symptomatology, including function and pain relief. Typical clinical quality metrics may not adequately capture outcomes that depend on self-report, such as quality of life, pain, satisfaction, or regret after surgical procedures.

Often, patient-reported symptoms, such as pain, function and ease of activities of daily living are the clinical criteria to pursue surgery. For example, the indications for performing a total joint arthroplasty include pain and quality of life when conservative measures fail, rather than radiographic wear of the joint which does not determine surgical intervention.⁸ Systematically measuring the quality and clinical efficacy of procedures performed for symptomatology may be challenging (Figure 2).⁷ For low risk procedures, postoperative functional health status reported by patients is the gold standard. Even for procedures with higher risk for clinical complications, quality of life, pain control, and functional status matters. For example, patients with rectal cancer who underwent low anterior resection or abdominoperineal resection had similar health-related quality of life as the general population in long-term follow-up. In this study, the authors used EORTC QLQ-C30, a generic cancer patient-reported outcome (PRO) measure, and QLQ-CR38, a colorectal cancer specific PRO measure. These PROs captured global health status, functional status, cognitive status, sexual function, body image, and other symptomatology, revealing that surgery does not have long-term consequences for functional status and quality of life for these patients.⁹ Thus, for many surgical procedures, commonly used clinical outcomes do not comprehensively capture quality, and quality may differ amongst providers (i.e. postoperative complications) and patients (i.e. quality of life). Therefore, measuring the gap between clinical outcomes and PROs can inform the value (outcomes relative to cost) of surgical care.

Figure 2:

Conceptual Model of Surgical Efficacy

Aligning of Surgical Efficacy and Patient-Centeredness: Patient-Reported Outcomes

Measurement of Patient-Reported Outcomes

PROs capture quality and clinical efficacy from the patient’s perspective.¹⁰ Methods to assess PROs translate these experiences into quantifiable elements that can be compared and evaluated. Instruments to assess PROs attempt to define latent traits, where there is no perfect measurement, but the essence of a domain can be captured by an increasing number of items or queries that correspond to that trait.¹¹ Methods to capture PROs can be qualitative or quantitative, with most quantitative instruments classified as either generic or condition specific. Generic PRO measures describe health-related dimensions that are ideally consistent across conditions and measure specific dimensions of overall health state. Examples of generic PROs include functional outcomes, pain, and overall quality of life. The SF-36 was the first set of instruments developed to reliably measure health and mental status of adults and to quantify the outcomes of health interventions from the patient’s perspective.^12–14 Since then, the SF-36 (including derivatives) has been implemented to measure the quality of life of older adults^15,16, patients with chronic diseases,^17,18 surgical patients,^19–22 among other conditions and has been translated and validated in multiple different languages.^23,24 More recently, the Patient Reported Outcomes Measurement System (PROMIS) was developed by the National Institutes of Health and leverages item response theory to address measurement and reliability problems and minimize responder burden.^25,26 PROMIS contains over 300 measures of health-related physical, mental, and social well-being (Table 1).²⁷ PROMIS domains are easily translatable and have been validated for healthy patients and patients with many health conditions and include pediatric domains. Generic PROMs offer the unique ability to compare conditions and treatments and may detect unexpected effects. ^28,29 However, they do not contain granular clinical information that may be valuable for specific disease states, and floor and ceiling effects are common. For example, children with cleft lip and palate match population norms in multiple domains of quality of life assessed using PROMIS, but marked differences emerge using more cleft- and orofacial specific instruments.³⁰ Accordingly, condition specific PRO measures capture disease-specific well-being, such as the European Organization for Research and Treatment of Cancer QLQ-C30 (EORTC QLQ-C30) which is a cancer-specific quality of life measure^31,32 or the Michigan Hand Outcomes Questionnaire which measures hand function.^33,34 These PRO measures quantify condition-specific outcomes of interest and are clinically relevant, but they cannot be compared to patients without the disease or across health conditions.³⁵ Finally, PROs often predict clinical outcomes. For example, cancer patients who have a superior quality of life have an increased overall survival and disease-free survival.^36,37 Moreover, when PROs are integrated into routine metastatic cancer care, patients live longer, which may be due to improvements in provider responses to symptomatology leading to chemotherapy modifications and prompt referrals.³⁸ Similarly, preoperative quality of life and functional status, such as the frailty index and SF-36, predict postoperative complications.³⁹ For other conditions, preoperative measures of anxiety, depression, and increased pain are associated with postoperative chronic opioid dependence, decreased mobility, and greater pain scores.^40–43 In the perioperative setting, PROs for quality of life can be an early indicator for increased risk of complications.⁴⁴ Therefore, measuring PROs may lead to improved clinical efficacy. Lastly, for prostate cancer, changes in prostate-specific PROs are able to predict disease progression, demonstrating an association between PROs and important clinical outcomes.⁴⁵ The measurement of PROs in the surgical episode can predict clinical outcomes and complications and has implications for improving established quality benchmarks.

Table 1:

PROMIS Measures

	Physical Health	Mental Health	Social Health
Global PROMIS Domains	Fatigue Pain Function Sleep	Anxiety Depression	Participation in Social Roles Social Activities
Specific PROMIS Domains	GI Symptoms Pain (quality, behavior) Sexual Function Shortness of Breath Sleep (Impairment)	Anger Cognitive Functioning Satisfaction with Life Psychosocial Illness Substance Use (Alcohol, Smoking, Other Substances) Self-efficacy	Social Support (Isolation) Companionship Satisfaction with Social Activities

PROMIS: Patient-Reported Outcomes Measurement Information System

To date, PROs are commonly used alongside traditional clinical outcomes in clinical trials to capture treatment effectiveness, and funding agencies expect PROs to reported. Moreover, the Food and Drug Administrations (FDA) established the FDA PRO guidance to ensure optimal incorporation of PROs into trial design and appropriate analysis of PRO data to encompass all aspects of drug therapy.^46,47 PROs often augment clinical outcomes,⁴⁸ and important health-related endpoints are associated with patient experiences. For example, among patients undergoing oncologic colorectal or breast surgery, the five-year survival rate is high, but quality of life after surgery may be variable. In a trial assessing three treatments for prostate cancer (watchful monitoring, surgery, or radiation therapy), prostate cancer-free survival at 10-years after treatment was no different among the groups.⁴⁹ Cancer progression was higher among the monitoring group with more metastatic disease at a median follow-up time of 10 years. However, patient-reported urinary incontinence, sexual function, bowel function, and quality of life revealed that patients undergoing surgery had poorer sexual and urinary function and quality of life than the other groups.⁵⁰ In this way, PRO measures revealed important consequences of surgery for the treatment of prostate cancer that are important to patients beyond disease survival. In addition, PROs augment the assessment of clinical effectiveness by capturing health status such as disability or pain. For example, in a randomized controlled trial of patients with lumbar spinal stenosis undergoing either (1) decompression alone or (2) decompression and spinal fusion, both clinical and patient-reported outcomes at five years postoperatively between the two groups were no different, rendering the addition of spinal fusion unnecessary.^36,51 In this case, PROs in this trial permitted a more comprehensive evaluation of surgical efficacy.

Patient-Reported Outcomes Versus Patient Experience

Patient-reported health status differs from measures of the patient experience during healthcare delivery in important ways. Measures of patient experience describe elements of a clinical encounter ranging from environment to accessibility of care, and can provide important data regarding delivery and processes of care.⁵² For example, the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) was established in 2002 by CMS and Agency for Healthcare Research and Quality to provide a standardized measurement of hospital quality through collection of patient-specific perspectives of hospital care.^53,54 This survey contains domains such as communication between doctors and patients, responsiveness to patient needs, and transitions of care. These measures are publicly available on the Hospital Compare Website and permit comparison of hospitals across the domains.⁵⁵ HCAHPS provides insights into areas of improvement to provide a better patient experience⁵⁶ and are associated with some objective surgical measures of quality including failure to rescue and minor complications.⁵⁷ However, patient satisfaction is not related to surgical mortality or major complications and does not capture the complete picture of clinical efficacy.⁵⁸ Though patient experiences and satisfaction have associations with clinical outcomes, they provide a different measure of quality than PROs, and augment a holistic approach to defining surgical quality.

Provider Feedback, Reimbursement, and PROs

With a growing interest in applying PROs toward treatment effectiveness, there is an increased application of PROs in the setting of surgical quality. For example, in 2009, the United Kingdom began routinely collecting PRO measures for elective surgery including groin hernia surgery, total hip arthroplasty, total knee arthroplasty, and varicose vein surgery to assess symptoms and health-related quality of life.^59–61 PRO measures used by the National Health Services include the generic measure, EuroQol 5 Dimension, and condition-specific instruments, such as Oxford Hip Score and Oxford Knee Score for total joint arthroplasty.⁶² Through the National PROM Programme in the United Kingdom, collection of PRO measures help provide comparison and continual quality of providers and hospitals.⁶³ The National Health Provider-level scores are published quarterly and have been directly associated with quality improvement initiatives and cost savings for specific hospitals.⁶⁴ This also allows patients to have full transparency when choosing providers or locations for treatment.

Moreover, the National PROM Programme has linked PRO measures, provider feedback, and reimbursement. Through the use of PROs by the National Health Services of the United Kingdom, some medical practices now reflect newly established best practices, and an environment of continual quality improvement has been fostered. However, PROs applies for this purpose have limitations, and may be influenced by non-response bias and missing data.^65–67 Finally, PROs may also be used as a measure of provider performance to increase provider accountability.⁶⁸ PROs vary across providers and hospitals and can provide vital feedback on treatment outcomes.^61,63 Through an iterative feedback process, PROs can be reported back to providers to deliver meaningful assessments on symptomatic control and patient-level quality of life. This can lead to continuous quality improvement for the individual provider or hospital.

Centers of Medicare and Medicaid: MACRA and MIPS

In the United States, CMS revised the reimbursement model, MACRA/MIPS, with a goal to transition healthcare reimbursement from a volume-based system to a value-based system.⁶⁹ These reimbursement reforms reward physicians for achieving quality benchmarks and connects quality improvement to payments. Two ways to implement MACRA exist through the Quality Payment Program: Alternative Payment Model (APM) or Merit-based Incentive Program (MIPS).

In the APM model, physicians are incentivized to provide high-value healthcare through added reimbursement for high quality and cost-efficient care. The APM model can be applied to a condition, episode of care, or specific patient population. CMS has developed care models that qualify for APM incentives that reward efficiency. In order to qualify for APM, practices must satisfy the following criteria: implementation of an electronic health record, payment for professional services to be based on quality (which is similar to the MIPS quality performance measures) and practices must be part of either a Medical Home Model or have patients that are at a substantial financial risk for using healthcare (i.e. patients with multiple chronic conditions). For example, the Oncology Care Model uses financial incentives to improve care for patients undergoing chemotherapy.⁷⁰ This model tests episode-based payments for the entire chemotherapeutic episode, which holds providers and practices responsible for care coordination, cost, and quality.⁷¹ Moreover, the Oncology Care Model provides each participating practice with an Aggregate Quality Score from 12 different quality measures, highlighting the importance of delivering high quality care.⁷⁰ Currently, preliminary data are not yet available on how this model performs and if this model can reduce costs and improve outcomes. Additionally, the Oncology Care Model does not incorporated PROs, which may provide valuable information regarding quality and clinical efficacy from the patient’s perspective.

For MIPS, physicians collect and report performance data regarding quality, cost, improvement activities, and interoperability to come up with a composite score. This generated score influences future physician reimbursement. In 2018, quality encompasses 50% of the MIPS score given to physicians (Figure 3). Current quality benchmarks in MIPS that relate to surgery include process measures such as choice of perioperative antibiotic or venous thromboembolism prophylaxis. Table 2 illustrates the outcome measures that MIPS captures for surgeons.⁷² Currently, only a few PROs exist to assess quality; however, development of multiple other PRO measures for functional and quality of life measures to be incorporated into MIPS is underway, which may be more applicable to providers across disciplines and across various types of surgical care being delivered. In the CMS Quality Measurement Development Plan, they identified the gaps in outcomes assessment, and the need to use PROs for certain surgical conditions. MIPS plans to include oncologic outcomes for medical, surgical, and radiation treatment with functional status measurements before and after these treatments. Measurements of quality of life and patient engagement will also be collected for patients undergoing cancer care.⁷³ Moreover, MIPS will begin to incorporate caregivers’ experiences, specifically patients’ families and support networks, in quality benchmarking, pivoting from the usual measured clinical outcomes such as mortality to capture the entire experience of the patient and their caregivers.

Figure 3:

Merit-Based Incentive Payment System Allocation by Year

Table 2:

Merit-based Incentive Program Outcome Quality Measures Specific for Surgeons

National Quality Strategy Domain	Measure Description
Person and Caregiver-Centered Experience and Outcomes	Function and pain assessment of patients with osteoarthritis
Communication and Coordination of Care	Pain assessment and follow up
Effective Clinical Care	Prolonged intubation after coronary artery bypass grafting
Effective Clinical Care	Sternal wound infection rate after coronary artery bypass grafting
Effective Clinical Care	Stroke after coronary artery bypass grafting
Effective Clinical Care	Postoperative renal failure after coronary artery bypass grafting
Effective Clinical Care	Surgical re-exploration after coronary artery bypass grafting
Communication and Care Coordination	Functional status change for patients with knee, hip, foot/ankle, lumbar, shoulder, elbow, wrist, or hand impairments
Patient Safety	Rate of open repair and endovascular repair of non-ruptured infrarenal abdominal aortic aneurysm without major complications, rate of carotid endarterectomy without complications
Effective Clinical Care	Rate of carotid artery stenting for asymptomatic patients without major complications
Effective Clinical Care	Rate of carotid artery stenting for asymptomatic patients who are discharged stroke-free or alive
Patient Safety	Rate of endovascular aneurysm repair or open repair for non-ruptured abdominal aortic aneurysm who are discharged alive
Patient safety	Anastomotic leak intervention
Patient safety	Unplanned reoperation within 30 days postoperatively
Effective Clinical Care	Unplanned hospitalization within 30 days of principal procedure
Effective Clinical Care	Surgical site infection
Patient Safety	Unplanned surgical bypass or amputation within 48 hours of lower extremity endovascular revascularization
Effective Clinical Care	Risk-adjusted operative morality after coronary artery bypass grafting
Patient Safety	Operative morality within 30 days after congenital heart surgery
Person and Caregiver-Centered Experience and Outcomes	Average change in leg pain after lumbar discectomy/laminectomy (preoperative to three months postoperative)
Effective Clinical Care	Use of patient reported outcome measure to assess improvement in disease specific measures after saphenous ablation
Patient Safety	Perioperative temperature assessment
Patient Safety	Bladder injury, bowel injury, or ureter injury at the time of pelvic organ prolapse surgery

Future Directions

For PROs to provide value, the instruments must be created using sound psychometric methodology including validity, reliability, and responsiveness.⁸ Accurate assessment of the measures is important in order to detect a clinically relevant change. Also, providers must understand what domains are important to the patient and choose the specific PRO measure that is most applicable to that patient population. Moreover, for many surgical interventions, a number of PRO measures exist that measure the same domain, thus complicating the choice of which measure to use. For example, the National Cancer Institute developed a core set of symptoms that should be included in PROs for oncologic clinical trials to promote consistent measurement of symptomology.⁷⁴ However, there are multiple PRO measures available to assess functional status, cognitive ability, and quality of life with minimal interoperability between measures. This limits the ability to compare between hospital PROs or between provider PROs because there is no agreed upon measure.⁷⁵ With the increase in use of PRO measures, no “best” measure exists, and multiple measures are commonly used to capture a comprehensive picture of health status.

With a push towards patient-centered healthcare, there has been a growing interest in using PROs in everyday surgical practice to measure clinical effectiveness and as a benchmark for quality. However, challenges with implementation of PRO measures must be acknowledged. The collection of PRO measures using paper forms is time consuming and may impede clinical workflow.^75,76 Therefore, understanding when and how to implement PROs into clinical care requires attention.⁷⁷ For postoperative PROs, the optimal timing of when to capture these PRO measures is unknown, and likely varies based on the surgical procedure. When are specific PRO measures considered stable? What are the exact intervals after a surgical intervention that PROs should be measured? Further research is needed to answer these questions. Lastly, for collection of PROs that are valuable to both the clinician and the patient, patient engagement in the domain decision-making process must be prioritized, thus creating an environment of shared partnership between patients and physicians.

Conclusion

Continual quality improvement in surgery is the goal for providing high-value care. The inclusion of PROs complements more established clinical outcomes by capturing efficacy from the patient’s perspective. For surgery, PROs will continue to become routine from both a quality assessment and reimbursement standpoint. Accurate and valid measurements of patient experiences through PRO measures in the preoperative and postoperative period can help surgeons provide better quality patient-centered care. Through incorporating a patient-centered approach, PROs will be critical in informing future health policies.

Synopsis.

Patient-reported outcomes capture surgical efficacy and quality from the patient’s point of view and compliment traditional quality metrics. Integration of patient-reported outcomes into practice can help surgeons provide better quality patient-centered care.