Surgical Care and Outcomes Assessment Program (SCOAP): A Nuanced, Flexible Platform for Colorectal Surgical Research

Abstract

The Surgical Care and Outcomes Assessment Program (SCOAP) is a surgeon-led quality improvement (QI) initiative developed in Washington State to track and reduce variability in surgical care. It has developed into a collaboration of over two-thirds of the hospitals in the state, who share data and receive regular benchmarking reports. Data are abstracted at each site by trained abstractors. While there has some overlap with other national QI databases, the data captured by SCOAP has clinical nuances that make it pragmatic for studying surgical care. We review the unique properties of SCOAP and offer some examples of its novel applications.

The Washington State SCOAP Collaborative

The Surgical Care and Outcomes Assessment Program (SCOAP) is a surgeon-led collaborative that was initiated in 2003 in Washington State to promote statewide quality improvement (QI) through data sharing. Within the collaborative, surgeons are engaged in the process of selecting evidence-based QI metrics to track. Over two-thirds of Washington State hospitals that provide surgical care participate in SCOAP. A few hospitals in neighboring states in the region have also joined ( Fig. 1 ). The nonprofit Foundation for Health Care Quality (FHCQ) serves as the administrative base for SCOAP and facilitates the inclusion of various nonhospital stakeholders. Member organizations (hospitals, providers, etc.) pay an annual administrative fee to participate, and provide the salary/benefits for the abstractor.

Fig. 1.

Surgical Care and Outcomes Assessment Program includes hospitals across Washington State, Oregon, and Montana, and has representation in rural and urban areas, large and small hospitals.

SCOAP is unlike National Surgical Quality Improvement Program (NSQIP) and Surgical Care Improvement Project (SCIP) in that it is not an accreditation mechanism or a data registry. Rather, it serves to build a community of surgeons who collect data on performance measures of their collective choosing.

SCOAP reports are published quarterly. The reports focus on benchmark performance indicators of the hospital in comparison to the other hospitals who participate in the collaborative. In a confidential and de-identified manner, hospitals are “rated” according to a signal—red, yellow, green—which indicate on-par performance or underperformance ( Fig. 2 ). These reports are intended to create opportunities for improvement via collaborative rather than punitive mechanisms. The program is also working to include individual physician performance via the National Provider Identifier (NPI). SCOAP has been tremendously successful in engaging participating surgeons by showing them how they compare with colleagues and also by giving them action items to improve the care they deliver.

Fig. 2.

Sample screen showing quarterly performance metrics of a Surgical Care and Outcomes Assessment Program hospital. Data are de-identified, but hospitals can see how their performance matches up to other participants. Green light, yellow light, and gray lights are used to indicate on-par (G), under-par (Y), and needs attention (A) rating scales, respectively. DVT, deep vein thrombosis.

Although the data captured by SCOAP have overlap with NSQIP, SCIP and other large surgical outcome datasets (especially in terms of how variables are collected, categorized, and coded), the surgeon-driven nature of the SCOAP project and the protection of the data under Washington State Statute allows for a unique clinical nuance that makes it practical to surgeons and allows SCOAP investigators to target novel dimensions of surgical care including individual processes and technical aspects of surgical care. ^{1 2}

The data are protected within a statewide statute for coordinated quality improvement programs (CQIP) that protects the data from discovery, much like how our M&M conferences work within our individual hospital systems. In a sense, the SCOAP collaborative is a steward for data that is fundamentally owned by the individual hospitals.

Data are entered directly into a web-based platform from the individual medical record by abstractors, who are funded by the individual institution. Data are stored in HIPAA-protected, electronic files. Reports are run quarterly in a de-identified format. Individual sites can run ad hoc reports on their own data, and are aware of the alias codes given to surgeons and patients. To protect the CQIP, SCOAP data must be used strictly to support QI activities and must not be used for public relations, contracting, or other non-QI uses. SCOAP staff conduct training and orientation regarding protection and use of SCOAP data as part of the orientation to the SCOAP QI initiative when hospitals enroll in SCOAP.

To gain access to these data for research purposes, a data use agreement is submitted and reviewed by a committee within the organization's leadership structure. Anyone can submit a data use agreement for review. Between 2012 and 2015, the committee approved 20 of 28 submitted data use agreements. Current and published SCOAP articles can be found on the group's Web site ( http://www.scoap.org/tools-resources/scoap-articles ).

SCOAP and Colorectal Surgery

A colorectal surgery workgroup has been housed within SCOAP to advise on which benchmarks, specific to our field, should be targeted. Topics that have arisen during advisory board meetings include anastomotic leak testing for low pelvic anastomoses, elective sigmoid resections in the setting of chronic diverticulitis, and the effect of specific medications on outcomes (nonsteroidal anti-inflammatory drugs [NSAIDs], opioid inhibitors, venous thromboembolism [VTE] chemoprophylaxis).

Anastomotic Leak Testing

As data emerged in the 1990s and 2000s about the potential benefit of anastomotic leak testing, ^{3 4 5} evaluating the impact of leak testing became particularly challenging because of practice variation among individual surgeons. For instance, some surgeons used leak testing as a screening tool, routinely testing all anastomoses. Others tested only when there is concern about the adequacy of “high-risk” anastomoses. Unfortunately, this latter, selective use of leak testing would be particularly difficult to interpret using observational data, because leaks would be expected to occur more frequently in these higher risk situations, confounding whether testing truly prevented leaks.

To address this question, the SCOAP authors collected data from 40 hospitals and nearly 3,500 patients. ⁶ “ Anastomotic leak testing was defined as the documented use of transrectal methylene blue, povidone-iodine, isotonic saline, or air with or without sigmoidoscope, as well as distention of an anastomosis generated by occluding/palpating the anastomosis.” ⁶ This highlights the depth of the SCOAP data collection, as the manner in which leak testing was performed varied across surgeons at the participating institutions. The authors performed a sensitivity analysis of the cases using palpation method and ultimately included the technique in their definition. While this report appropriately avoided the claim that leak testing prevented leaks and associated complications, the data demonstrated that the regular measurement of leak testing in the SCOAP network resulted in increased leak testing from 56 to 76% ( p  < 0.001) over the period of the study ( Fig. 3a ). Furthermore, hospitals where routine leak testing was occurring had decreased composite adverse events related to an anastomotic leak from 7.0 to 4.6% ( p  = 0.01; Fig. 3b ). The authors performed a sensitivity analysis excluding cases with fecal diversion, which did not alter their conclusions. These findings substantiate prior literature supporting routine leak testing, ⁵ which is now considered standard practice for left-sided anastomoses by the American Society of Colon and Rectal Surgeons. ⁷

Fig. 3.

( a ) Trend during quarters of participation in the Surgical Care and Outcomes Assessment Program (SCOAP) in the use of anastomotic leak testing after an ileorectal or colorectal anastomosis. (Reprinted with permission from Kwon et al. ⁶ ) ( b ) Trend during quarters of participation in the SCOAP in the incidence of composite adverse events after an ileorectal or colorectal anastomosis. (Reprinted with permission from Kwon et al. ⁶ )

Beyond this hospital-level process improvement, the SCOAP investigators recently demonstrated the application of the dataset to characterize surgeon-specific patterns of leak testing. ⁸ Across 44 hospitals and 282 surgeons, some surgeons demonstrated “recency bias” by increasing their leak testing after their patients had an anastomotic leak in an untested case. This study again highlights the depth of the SCOAP variable definitions—since leaks can present variably, the authors used a composite definition of anastomotic leak including “radiologically demonstrated anastomotic leak or enterocutaneous fistula, postoperative percutaneous drainage of abscess, or unplanned reoperative intervention requiring colostomy and/or ileostomy, abscess drainage, operative drain placement, or anastomotic revision.” ⁸ The authors identified different patterns of leak testing in high- and low-volume surgeons, and postulated that those most susceptible to “recency bias” may be newly appointed surgeons, who performed more cases for nonmalignant indications, had longer operative times, and more frequently began cases minimally invasively (all p  < 0.001). Beyond the clinical relevancy of the variables, this study showcases a novel way to utilize observational data to characterize clinician behavior in individual clinical situations.

Diverticulitis: Using Data to Drive Change

The unique variable collection and feedback mechanism employed by SCOAP has allowed the group to translate data into actionable change in their network. In 2010, SCOAP initiated a surveillance, benchmarking, and education initiative related to the surgeon-reported indications for elective colon resection for diverticulitis. ⁹ Surgeons at SCOAP hospitals were issued standard dictation templates and asked to document the indication for elective resection into the operative note using this format ( Fig. 4 ). Data from all sites, as well as existing guidelines and evidence, ^{10 11} were compiled and disseminated regularly in a series of comparative reports, in-person and streamed presentations, and informal newsletters. The authors reported a reduction in the proportion of cases that did not meet indications from 38.4 to 26.4% ( p  < 0.001) as well as a reduction in the proportion of patients with missing indication data from 38.1 to 21.6% ( p  < 0.001). ⁹

Fig. 4.

The SCOAP Diverticulitis Operative Report Guide is provided to all surgeons at participating hospitals, and the reported measures were agreed upon by the SCOAP Colorectal working group.

Because SCOAP developed from a peer-to-peer collaborative of community practice and academic surgeons, the data collection combines of experience, common sense, and developing evidence. ¹² Saliently, this means that investigators have the ability to discuss the implications and critique of their studies with their peers, and design prospective data collection that is proactive at addressing key concerns. For instance, cognizant that critics of delayed elective operations for diverticulitis often pointed to the potential for increased complications, emergency surgery, and lack of the ability to complete operations laparoscopically, SCOAP included these variables as part of the data collection tool. In the aforementioned study, the proportion of patients with complicated disease remained stable from 28 to 32% ( p  = 0.69) as did the yearly rate of emergency operations, 5.6 to 5.9/year, p  = 0.81. ⁸ The authors subsequently published a detailed report on laparoscopic conversion rates of operations for diverticulitis, ¹³ showing that increasing episodes of diverticulitis were not associated with higher conversion rates, even after adjustment for surgeon-specific case volume as a surrogate for experience ( p  = 0.75).

Medication Use and SCOAP Data Linkage

Beyond the clinician-led QI surveillance and feedback mechanism, the data collected through SCOAP can be expanded outside the index procedure and hospitalization by linkage to other existing data sources.

Chemoprophylaxis for VTE Prevention

For instance, it is increasingly recognized that the risk of VTE remains elevated for up to 12 weeks following surgery. ^{14 15 16} VTE that occurs during this time may be related to the management of patients while in the hospital; most QI databases, including SCOAP, do not capture VTE when it occurs after patients have left the hospital. To address this, SCOAP investigators linked their dataset to the Comprehensive Hospital Abstract Reporting System (CHARS), an administrative dataset of all hospital admissions in the state, to determine readmissions for VTE up to 90 days, and to the Washington State vital records registry, to determine postoperative mortality. ¹⁷ Indeed, the authors found that 39% (142 of 360) of VTE events occurred after the index hospitalization. Because the detailed SCOAP data collection captures VTE chemoprophylaxis before and after operations, the authors reported that VTE chemoprophylaxis increased in three relevant settings: perioperative, in-hospital postoperative, and post discharge, all p  < 0.001 ( Fig. 5 ). Despite this, the authors raise awareness that adjusted VTE rate at 90 days remains unchanged ( p  = 0.09). The authors speculated that the unchanged VTE rates could reflect increased detection of clinically silent VTE because of increased surveillance, appropriate prophylaxis of high-risk patients (so that an increase in VTE rates was not seen), or that perhaps some VTE is not preventable.

Fig. 5.

Trends in the use of venous thromboembolism prophylaxis by perioperative setting and annual incidence of 90-day VTE. Denominators vary for each group. VTE indicates venous thromboembolism. (Reprinted with permission from Nelson et al. ¹⁷ )

Opioid Inhibitors and Hospital Length of Stay

As another example of contemporary evidence integration, SCOAP began tracking administration of alvimopan (Entereg; Merck), a pharmaceutical that acts as an enteric mu -opioid receptor antagonist to reduce opioid-related ileus, in 2008, immediately after FDA approval. ¹⁸ While emerging data had suggested a benefit to alvimopan at increasing return of bowel function and reducing length of stay (LOS), ^{19 20 21 22} there remained considerable concern about the cost of the drug, estimated at $67.50 per pill or $937.50 for a full course of treatment. ²³ To address whether the reduction in LOS (measured in SCOAP) was financially acceptable from a payer's perspective across the multiple practice settings in Washington State, the authors once again linked the data to the CHARS dataset, where total charges for index hospitalization are captured. “After adjustment, LOS was 0.9 days shorter ( p  < 0.01), and hospital costs were $636 lower ( p  = 0.02) among those receiving alvimopan compared with those who did not.” ¹⁸ The authors performed several adjustments for potential bias in the study, ultimately concluding that the cost savings and LOS decrease were important, and highlighting the ability of SCOAP data to be applied to broad, effectiveness evaluations.

Nonsteroidal Anti-inflammatory Drugs and Anastomotic Leak

Similarly, with the expansion of NSAIDs in the postoperative setting because of emerging intravenous formulations, data emerged that NSAID use might increase postoperative complications, in particular anastomotic problems. ^{24 25} To examine whether their patients were at risk, the SCOAP group included NSAID use in their data collection. “Abstractors were trained to review the medical record for the administration of NSAIDs (including ibuprofen, naproxen sodium, ketorolac tromethamine, caldolor, celecoxib, and diclofenac) starting within 24 hours of surgery.” ²⁶ Across 47 hospitals and over 13,000 patients undergoing bariatric and colorectal surgery, the group evaluated anastomotic failure and other complications up to 90 days (linkage to CHARS). After risk adjustment, NSAID use was associated with increased anastomotic leak (odds ratio [OR], 1.24; 95% confidence interval [CI], 1.01–1.56]; p  = 0.04), but this was isolated to nonelective surgery. The increased mortality at 90 days appeared to be driven by cardiac events. ²⁶

Giving SCOAP Leaders Voice

The authors had the opportunity to garner comments from three key figures in the SCOAP collaborative's leadership ( Fig. 6 ): David Flum, MD, FACS, who is SCOAP's founder, Richard Thirlby, MD, FACS, who is SCOAP's medical director, and Joseph Frankhouse, MD, FACS, FASCRS, who is one of SCOAP's advisory board members.

Fig. 6.

SCOAP leaders Drs. David Flum ( a ), Richard Thirlby ( b ), and Joseph Frankhouse ( c ).

David Flum, MD: SCOAP Founder

In an era with increasing awareness about significant variability in surgical outcomes, SCOAP data allow for meaningful risk adjustment, process of care information, and granular clinical details needed to actually improve quality for colorectal surgeons in Washington State. In part, this is because the SCOAP data definitions and metrics are created and refined by the clinical community itself in response to new clinical evidence, emerging technology, or changes in practice. For example, within months of coming into the market, SCOAP developed the practice of adding innovations like opioid antagonists, new anticoagulants, and other medication and technology to the registry. In this way, SCOAP acts as a roving spotlight to identify QI opportunities for the colorectal surgical community and support initiatives to address them.

Richard Thirlby, MD: SCOAP Medical Director

What makes SCOAP a perfect tool for research on colorectal surgical outcomes lies in its inclusion of many procedure-specific data fields. SCOAP also tracks procedure-specific complications. For example, the practice of placing (and leaving) a nasogastric tube during colorectal resections is something we can track very specifically. Using a synoptic operative report from which our SCOAP participating intuitions and surgeons are trained to dictate, we can tell whether it was placed and whether the patient left the operating room with it in place. Another example is the work we have been able to do with the research on alvimopan. Essentially, I see the collaborative as a mobile ship that can alter its course. We can add and remove the variables we collect and we do this by consensus. Once agreed upon by consensus, these changes to the collection tool are made quickly and we can start gathering data within months.

Joseph Frankhouse, MD, FACS, FASCRS, Advisory Board Member, SCOAP

As the Medical Director of colon and rectal surgery at Legacy Health, based out of Portland, OR, I have seen SCOAP as the backbone for quality improvement in our hospital system. It has been said previously that “you cannot change, what you cannot measure.” Of course, every surgeon and hospital wants to improve quality, but the ability to collect and compare data is often missing. With SCOAP, the collection of hospitals and surgeons is smaller and more collaborative than national programs. Therefore, the end results are more easily obtained and disseminated.

Much of what we are achieving is trying to move away from practices of tradition and dogma to those which are more evidenced based. Efforts are made to improve value in surgical care, which is defined best when we combine quality and cost efficiency with appropriateness of surgery. For management of rectal cancer, diverticulitis, or small bowel obstruction, it is important for us all to share and develop the best methods of delivering care both surgically and non-surgically.

A great example of the way SCOAP can make improvements in colorectal surgery is the transparency that exists in the management of rectal cancer. We can provide the individual hospitals with outcomes on specific metrics in rectal cancer surgery such as the abdominoperineal resection rate based on the distance from the anal verge, proper pretreatment staging, and proper selection of neoadjuvant therapy modalities. These analyses will allow surgeons to evaluate their own practice and make the appropriate changes.

Another great example for gastrointestinal surgery is our work with enhanced recovery after surgery (ERAS) principles. ERAS is a concept initiated almost 30 years ago in Europe, that the United States has been relatively slow to adopt. However, data suggest that outcomes and cost are greatly improved upon when hospitals and surgeons utilize such principles. Length of stay, which many argue is the most important end point to measure with surgical patients is reduced consistently with such programs. We have the ability through SCOAP to encourage and educate our colleagues, and then provide the ability to use metrics to validate the approach and improve the value of our care.

Conclusion

The limitations of the SCOAP datasets reflect the challenges of the voluntary, clinician-led QI surveillance of project from which it arose. ¹² For instance, the data platform is dependent on several sources (medical records, images, and operative transcriptions) which vary across institutions and may lack critical details if they are not recorded. In the aforementioned examples, the results of an intraoperative leak test and the subsequent decision making (to re-do the anastomosis or divert), ^{6 8} the reason for conversion from laparoscopy (planned or because of a particular intraoperative event), ¹³ or missed doses of medicines ^{17 18} are often not captured. In addition, automating data flow, acquiring additional data for linkage, and creating the infrastructure for comparative effectiveness research from the data, as well as implementing the data across multiple clinical arenas are resource intensive. To that end, SCOAP has partnered with the Comparative Effectiveness Research Translation Network (CERTAIN), and would consider partnering with other such research groups for analytic and editorial pursuits. ^{27 28}

These challenges notwithstanding, SCOAP data remain uniquely flexible and actionable because of ongoing, peer-based collaboration. The value of this clinically pertinent and nuanced dataset is unique and critical at measuring the relevant and evolving aspects of improvement in the quality of surgical care.