Current Status of Quality Measurement in Colon and Rectal Surgery

Abstract

There is increasing pressure from the public and insurance payors for transparency and accountability in ensuring the quality of health care. In response to this, hospitals are now mandated to participate in quality improvement initiatives, and to report on their performance. This article describes three programs related to the measurement of quality that impact colon and rectal surgery: the Surgical Care Improvement Project, the National Surgical Quality Improvement Program, and the Surgical Care and Outcomes Assessment Program.

CME Objectives: On completion of this article, the reader should be able to summarize the current status of quality measurement protocols and practices in colon and rectal surgery.

Since the publication of the Institute of Medicine report To Err is Human: Building a Safer Health System in 1999 and Crossing the Quality Chasm: A New Health System for the 21st Century in 2001, detailing the rate of medical errors and their human costs, there has been increasing scrutiny of the quality of medical care by the government, insurance payors, patients, and health care providers themselves. There is growing demand for transparency in the evaluation and reporting of health care quality in the public media. This quality movement has also led to efforts to incentivize providers and reward good quality care through “pay for performance” programs. Significant resources are being devoted to the measurement and improvement of quality of care. But what defines good quality care? How do we measure it? What should be measured? How is this information being used? This article will discuss several national programs that attempt to define and measure quality of care benchmarks to improve colorectal surgery outcomes for our patients. The process and outcomes measures being evaluated by these programs (the Surgical Care Improvement Project [SCIP], the National Surgical Quality Improvement Program [NSQIP], and the Surgical Care and Outcomes Assessment Program [SCOAP]) and their impact on patients, providers, and our current medical system will be reviewed.

Conceptual Model of Health Care Quality

The Donabedian model conceptualizes health care quality into three interrelated components of structure, process, and outcome.¹ “Structure” refers to the context in which care is provided, and includes metrics such as hospital and surgeon volume, nursing ratios, and availability of an electronic medical record. These measures are often indicators of the amount of resources within a system, and while they can impact the overall quality of care, they are often more static and nonmutable. Examples of structural measures specific to colon and rectal surgery include hospital and surgeon volume and surgeon board certification. “Process” refers to the activities of care provision. Process measures assess whether a specific intervention was performed for a defined patient population. “Outcomes” are the results of providing care. Examples include mortality and morbidity, surgical site infections (SSIs), disease-free survival, quality of life, and patient satisfaction. The underlying idea is that the structure of the system impacts the process of providing care, which then affects patient outcomes. Any one of these components can be modified to improve overall quality of care.

The Surgical Care Improvement Project

An increased awareness of the frequency and cost of medical adverse events led to the development of several national initiatives aimed at standardizing treatment protocols to improve patient outcomes in the United States. The first such initiative was aimed at reducing the incidence of SSIs and was called the Surgical Infection Prevention Project (SIP). SIP was initiated by the Centers for Medicare and Medicaid Services (CMS) and the Centers for Disease Control and Prevention (CDC) in 2002 and focused on the improvement of process measures aimed at decreasing the incidence of SSIs, the third most commonly reported nosocomial infection, and a major source of surgical morbidity and health care costs.² SSIs were targeted because they were believed to be largely preventable and the goal of SIP was to reduce SSI rates for major, high-volume procedures. These included five areas of care: elective colorectal resection, total hip/knee replacement, open heart surgery, peripheral vascular procedures, and abdominal/vaginal hysterectomy.

To reduce the incidence of SSIs, SIP focused on process measures designed to promote the proper use of antibiotics, which were being inappropriately administered in a large percentage of patients. The proposed interventions relied on well-established, evidence-based infection prevention measures that were shown to reduce SSIs.

In January 2003, the Medicare National Surgical Infection Prevention Project hosted the SIP guideline writer's workgroup meeting, which consisted of surgical experts as well as representatives from various surgical infection societies to review the published data and establish guidelines for surgical antimicrobial prophylaxis.³ Three process measures, which served as quality metrics, were identified. These included (1) antibiotic administration within a 60-minute window before skin incision, (2) proper choice of antibiotic, and (3) discontinuation of antibiotics within 24 hours of completion of the operation. All were based on strong medical evidence.

Compliance with these three measures was tested at the initiation of SIP using a national sample of Medicare patients who underwent one of these five high-volume surgical procedures in 2001. The study showed that compliance with these three process measures was low (55.7% received antibiotics within 60 minutes, 92.6% were given the appropriate antibiotic, and 59.3% received antibiotics beyond 24 hours) and confirmed that there was considerable room for improvement.⁴

To test the impact of these measures on the prevention of SSIs, a national collaborative was sponsored by CMS that included 56 hospitals and 43 Medicare Quality Improvement organizations. Following the implementation of SIP, hospitals that participated in this 1-year collaborative effort identified statistically significant reductions in postoperative infection rates, reporting a mean 27% reduction in SSI rates.⁵

Given its early success, in 2005 SIP was expanded to become the SCIP. SCIP differed from SIP by expanding on existing process measures. Similar to SIP, SCIP focused on perioperative adverse events that were common, potentially preventable, and associated with a high medical cost.

As with SIP, evidence-based practice guidelines were created to define best practices aimed at improving specific perioperative outcomes. These guidelines were based on the medical literature and are still reviewed and updated regularly. Process measures related to SSI reduction were expanded from three to six criteria and included (1) proper glucose control in cardiac surgery, (2) hair removal at the surgical site, and (3) normothermia in patients undergoing colorectal resection. Performance measures were expanded to also address cardiovascular (the use of perioperative β-blockers to reduce the risk of cardiac events), venous thromboembolic (deep venous thrombosis prophylaxis), and respiratory complications (reducing postoperative pneumonia). The goal of SCIP was to reduce preventable surgical morbidity and mortality by 25% by 2010.

CMS published SCIP in 2006, and it was subsequently adopted by multiple organizations including the Joint Commission on the Accreditation of Hospital Organizations, the Agency for Healthcare Research and Quality, the Centers for Disease Control and Prevention, the Veteran Affairs, the Institution for Healthcare Improvement, and the American College of Surgeons (ACS). SCIP performance measures are now collected by more than 3,700 hospitals that submit abstracted data.

The Joint Commission also recommended that hospitals voluntarily report compliance with these SCIP measures. In 2011, this became a mandatory component of Joint Commission accreditation. Compliance with SCIP is now linked to pay for performance measures designed to incentivize better quality and cost containment. Since 2011, SCIP compliance has affected Medicare and Medicaid reimbursement rates.

National Surgical Quality Improvement Program

NSQIP is a program that was formed in parallel with SCIP, and was originally developed out of the Department of Veterans Affairs (VA) health system. In response to public scrutiny over its operative mortality rates, and under mandate by Public Law 99–166 passed by the Congress in 1986, the VA conducted the National Veterans Affairs Surgical Risk Study (NVASRS) to measure and report risk-adjusted surgical outcomes, and to compare VA outcomes to national averages. NVASRS was a prospective cohort study conducted in 44 VA medical centers. The study was designed to collect reliable and valid data on patient risk factors and postoperative outcomes. Patients undergoing major noncardiac operations in eight surgical subspecialties under general, spinal, and epidural anesthesia between 1991 and 1993 were included. Dedicated, trained nurses were used to collect preoperative, intraoperative, and postoperative data. Outcomes measured included 30-day all-cause mortality and 21 major morbidities.⁶ Using these data, multivariable logistic regression risk adjustment models were then created to compare outcomes across VA facilities. The risk-adjusted outcomes were expressed as observed-to-expected (O:E) ratios.^{7 8} The “expected” outcome is based on the preoperative risk profile of the patient population. A ratio of less than 1 means that the institution is performing better than expected, and vice versa for a ratio greater than 1.

Subsequently in 1994, the Surgical Risk Study was then rolled out to 123 VA facilities as the VA NSQIP.⁹ The VA NSQIP program provided individual facilities with site-specific outcome reports, self-assessment tools, site visits, and best practices. Using the information derived from this program, individual facilities were then able to institute targeted quality improvement projects to yield measurable improvements in patient care.¹⁰ By 2002, the VA NSQIP program was able to demonstrate a 27% decrease in 30-day postoperative mortality, and 45% decrease in 30-day postoperative morbidity.¹¹ In 1999, a pilot study was then conducted in three academic institutions that demonstrated the feasibility of applying NSQIP methodology in non-VA hospitals.¹² In 2001, the ACS received an Agency for Healthcare Research and Quality grant to expand the NSQIP program to 18 private sector hospitals.

Currently, more than 400 hospitals in the United States participate in the ACS NSQIP. Each hospital is required to pay a participation fee to the program, and provide a surgeon champion and funding for a trained surgical clinical reviewer (SCR) for data collection. Validity and reliability of the data are ensured by standardized training of the SCRs by the ACS, site visits to ensure inter-rater reliability, and continued training for data collectors. In large volume institutions, there is random sampling of cases for data collection, and in smaller volume institutions all cases are included. Patients aged 18 or older receiving all types of anesthesia in both inpatient and outpatient settings are included. All data are reported to NSQIP via a web-based data entry program. The validity of the 30-day outcomes is increased by direct communication with surgical patients by phone or letter, and by public record death searches to complete the 30-day follow-up. The program collects more than 130 preoperative, intraoperative, and postoperative data variables, and reports on more than 20 risk-adjusted outcomes. Outcomes analyzed include 30-day mortality, cardiac arrest requiring cardiopulmonary resuscitation, myocardial infarction, deep vein thrombosis, pulmonary embolism, sepsis, SSI (superficial, deep, and organ space), wound disruption, unplanned reintubation, ventilator dependence > 48 hours, pneumonia, renal failure, and unplanned reoperation and urinary tract infection, among other morbidities. The outcomes are reported to individual participating hospitals as an “observed versus expected ratio (O:E ratio),” and surgeon-level outcomes are also available.

NSQIP provides participating institutions with semi-annual outcomes reports to compare risk-adjusted outcomes to other participating hospitals. There are site-specific reports for preoperative risk factors, patient statistics, 30-day mortality, and morbidity, and blinded surgeon-level outcomes are also reported. Participants also have access to best practices guidelines, risk calculators to help inform patients about operative risk, and a participant use data file for research and quality improvement programs. For example, there is a morbidity and mortality risk calculator specifically for colorectal surgery.¹³ The colorectal risk calculator uses data from the pooled NSQIP database to help inform patients' individual risk for various outcomes based on their age, gender, and other preoperative factors. The data provided by NSQIP can be used to aid in quality improvement activities by identifying areas in which an individual hospital is performing below what is based on the risk-adjusted outcomes.

NSQIP is a validated nationwide outcomes-based database that only examines 30-day postoperative patient outcomes. It is not intended to examine long-term outcomes, and does not necessarily collect data on all outcomes of interest to the surgeon. And although hospital-based processes can be targeted to improve outcomes in deficient areas, it may be limited when it comes to targeting individual-based processes, such as the technical aspects of surgical care.

Surgical Care and Outcomes Assessment Program

“SCOAP is a developing Washington State initiative that was designed and implemented by practicing surgeons, the leadership of the statewide ACS chapter, the Washington State Hospital Association, and [Quality Improvement] organizations across the state to track and reduce variability in abdominal surgical practice and outcomes.”¹⁴ It is a physician and hospital collaborative that includes more than 50 hospitals in Washington state. Although it has data variables in common with NSQIP and SCIP, unlike NSQIP and SCIP, which focus on outcomes and process measures, respectively, SCOAP includes both process and outcomes measures. Also unlike other quality measures such as SCIP or other Joint Commission Core Measures, which are mandated and publicly reported, SCOAP's nature as a quality improvement project allows the information gathered under SCOAP to be protected by Washington State Statute.

SCOAP is a quality improvement project that aims to improve quality by reducing variation in the provision of surgical care. The specific procedures that were initially included are colon and rectal resection, appendectomy, and bariatric surgery. Because of the surgeon driven and procedure-focused nature of the project, the data variables are more nuanced and procedure specific. For example, the data collection form for colon procedures includes information about prior colon or pelvic surgery, the specific type of anastomosis performed (e.g., the use of Seamguard or other sealing devices during stapled anastomosis), whether a leak test was performed and via what method (scope, methylene blue, air insufflation, palpation/inspection, etc.), use of bowel prep and oral antibiotics, information on neoadjuvant treatment, lymph node harvest, resection margins, whether a total mesorectal excision was performed, and many others.¹⁵

As SCOAP includes both process and outcome measures, it allows performance improvement investigators to query the impact of clinical processes on patient outcomes. For example, Kwon et al looked at the impact of routine leak testing of left-sided colorectal anastomoses on postoperative adverse events, and found a significant reduction in adverse events in hospitals where leak testing is routinely preformed.¹⁶ Based on this type of data, there is potential to reduce the variation in the use of routine leak testing to decrease anastomotic complications and improve patient outcomes.

What Has Been the Impact of These Programs?

SCIP, NSQIP, and SCOAP all provide infrastructure for quality improvement measures, along with real data to assess the actual impact of these measures on patient outcomes. These programs have the potential to improve the quality of medical care that is provided at a lower cost, which is beneficial to patients, hospitals, and providers of health care at all levels.

Despite their potential advantages, support of such programs requires considerable resources to collect, report, and analyze the data appropriately. The accuracy of the program relies on detailed and labor-intensive surveillance and reporting.

A distinction can be made between what we know to be high-quality care, what can be measured, what is currently routinely measured, and what should be measured to assess quality of care. For example, though each SCIP measure is based on the best available medical evidence, there has been a lack of consistent data supporting a strong correlation between SCIP compliance and improvement in the associated outcome measures,¹⁷ though the correlation is stronger in outlier institutions.¹⁸ In fact, from 2001 to 2006, post-op infection rates increased in colon surgery,¹⁹ as well as from 2006 to 2008.¹⁸ Because the focus of SIP and SCIP has been on perioperative processes that can be reliably gathered and analyzed, the selected process measures only partially impact the surgical outcome being assessed.

While the validity of these programs remains debatable, the value lies in their focus on quality improvement, self-reporting, and self-assessment. In an era of transparency and accountability, SCOAP, SCIP, and NSQIP are the first steps in a national effort to improve surgical outcomes and improve the quality of care.

Conclusion

To conclude, SCIP, NSQIP, and SCOAP have had a significant impact on surgical practices nationwide. Providers now must comply with evidence-based practices that standardize specific, evidence-based perioperative processes. Hospitals have adopted this quality improvement culture and have dedicated considerable resources to insure SCIP compliance, such as the hiring of SCIP nurses for data collection and review, and to track risk-adjusted patient outcomes. Ultimately, these efforts should improve the overall quality of medical care that we provide for our patients in the United States.