Benchmarking Rectal Cancer Care: Institutional Compliance with a Longitudinal Checklist

Abstract

Background

In 2012, the American Society of Colon and Rectal Surgeons published the Rectal Cancer Surgery Checklist, a consensus document listing 25 essential elements of care for all patients undergoing radical surgery for rectal cancer. The authors herein examine checklist adherence among a mature, multi-surgeon specialty academic practice.

Materials and Methods

A retrospective medical record review of patients undergoing elective radical resection for rectal adenocarcinoma over a 23-month period was conducted. Checklists were completed post hoc for each patient, and these results were tabulated to determine levels of compliance. Subgroup analyses by compliance levels and experience level of the treating surgeon were performed.

Results

161 patients underwent resection, demonstrating a median completion rate of 84% per patient. Poor compliance was noted consistently in documenting baseline sexual function (0%), multidisciplinary discussion of treatment plans (16.8%), pelvic nerve identification (8.7%) and leak testing (52.9%), and radial margin status reporting (57.5%). Junior surgeons achieved higher rates of compliance and were more likely to restage following neoadjuvant therapy (67.9% vs 29.4%, p < 0.001), discuss patients at tumor board (31.3% vs 13.2%, p=0.014), and document leak testing (86.7% vs 47.2%, p=0.005) compared to senior surgeons.

Conclusions

Checklist compliance within a high-volume, specialty academic practice remains varied. Only surgeon experience level was significantly associated with high checklist compliance. Junior surgeons achieved greater compliance with certain items, particularly those that reinforce decision-making. Further efforts to standardize rectal cancer care should focus on checklist implementation, targeted surgeon outreach, and assessment of checklist compliance correlation to clinical outcomes.

Introduction

The treatment of rectal adenocarcinoma is complex and often requires multimodal therapy including specialty imaging, radiation, surgery, and chemotherapy.(1) Data have repeatedly demonstrated that close coordination and collaboration between radiologists, surgeons, and radiation and medical oncologists improve outcomes for patients undergoing surgical resection.(2–4) Yet in North America, patients have continued to experience highly variable treatment and inconsistent outcomes – including abnormally high rates of permanent ostomy creation, local recurrence, and mortality – over the past two decades.(5–7) In response, the American Society of Colon and Rectal Surgeons (ASCRS) published the Rectal Cancer Surgery Checklist in 2012.

The checklist, based on expert consensus and an iterative feedback process from ASCRS members, was intended to standardize care and guide clinicians caring for rectal cancer patients undergoing curative resection.(8) It contains 25 components of pre-, peri-, and post-operative care that should be performed for every patient (see Appendix). However, neither adherence rates nor clinical use of the checklist has been previously reported. It remains unclear whether full compliance with all checklist items is a feasible goal in a busy clinic practice.

We therefore designed this study to benchmark ASCRS Rectal Cancer Surgery Checklist adherence among a high-volume, academic specialty practice as well as identify factors associated with high compliance. We hypothesize that checklist compliance will be less than 100% and vary among surgeons. Overall checklist compliance among patients undergoing curative resection for rectal cancer is reported, along with subgroup analysis of junior and senior surgeon practice trends.

Methods

Data Source

All patients undergoing elective, curative resection for rectal adenocarcinoma at a single tertiary, academic specialty practice from November 2013 through December 2015 were selected from a prospectively maintained billing registry. Those aged under 18 years, undergoing urgent or endoscopic (transanal endoscopic microsurgery) resection, or diagnosed with bowel obstruction or inflammatory bowel disease were excluded from further review. A single reviewer (P.C.) then used a centralized medical record to retrospectively complete an ASCRS Rectal Cancer Surgery Checklist for each eligible patient. Additional demographic data including insurance source, Charlson Comorbidity Index(9), travel distance from treating medical center, location of neoadjuvant therapy administration, and operating surgeon were collected from the same medical record. Each patient underwent either Low Anterior Resection (LAR) or Abdominoperineal Resection (APR) by 7 surgical faculty with 1 to 22 years of post-fellowship experience. The Washington University School of Medicine Institutional Review Board approved this investigation and granted a waiver of informed consent.

Variables and Subgroups

The 25-item ASCRS Rectal Cancer Surgery Checklist was developed and published by the ASCRS Quality Assessment and Safety committee in 2012 following extensive literature review and iterative expert discussions (see Appendix).(8) Each item from the checklist served as a variable for initial analysis. Items completed across all surveyed patients represented “Complete Compliance.” Among all patients, the median number of completed checklist items was 21. For analysis of factors associated with checklist compliance, the dataset was divided into groups containing High Compliance checklists (having 21 or more completed items) or Low Compliance checklists (having fewer than 21 completed items). To analyze the impact of surgeon experience specifically, checklists were separately classified based on the experience level of the treating surgeon. Surgeons with ten or fewer years of post-fellowship experience were arbitrarily deemed Junior, with the remaining surgeons deemed Senior. Individual item compliance rates were then compared between groups.

Statistical Analysis

Continuous and categorical variables are reported as mean with standard deviation and proportions throughout this study. Continuous variables with grossly skewed distributions, however, are reported as median with an interquartile range. Bivariate analysis of continuous and categorical variables was performed with either the Student’s T-test or the Wilcoxon rank-sum test depending on distribution and Fishers exact test respectively. SAS statistical software (version 9.3, SAS Institutes Inc., Cary NC, USA) was used for all analyses. All tests were two sided with an alpha level of 0.05.

Results

A total of 161 patients met inclusion criteria for this study. The mean age was 58.8 years, and 65% of the population was male. (Table 1) The median number of checklist items completed per patient was 21 (IQR 20–23). Components reaching complete compliance included preoperative endoluminal colonic evaluation, clinical staging for metastatic disease, consideration of neoadjuvant therapy, en bloc resection of organs when indicated, documentation of anastomotic type, postoperative stoma teaching and medical oncology referral when indicated. Consistently poorer performance was found among documentation of preoperative sexual function (0%), post-neoadjuvant restaging (69.3%), documentation of intraoperative pelvic nerve assessment (8.7%), and documentation of radial and distal margin status in pathology reports (57.5%). (Table 2)

Table 1.

Patient characteristics

	Total (n=161)	Patients Treated by Senior Surgeons (n=129)	Patients Treated by Junior Surgeons (n=32)	p
Male	64.6%	62.8%	71.9%	0.34
Mean Age (Standard deviation)	58.8 (12.2)	59.5 (12.4)	56.0 (10.0)	0.09
Mean BMI (Standard deviation)	29.0 (6.4)	29.0 (6.4)	29.0 (6.7)	0.70
Race
Caucasian	80.7%	82.9%	71.9%	0.42
African-American	14.9%	13.2%	21.9%
Hispanic	0.6%	0.8%	0%
Other	3.7%	3.1%	6.3%
Insurance Status
Private	45.3%	47.3%	37.5%	0.16
Medicare	41.6%	42.6%	37.5%
Medicaid	9.3%	7.0%	18.8%
Uninsured	3.7%	3.1%	6.3%
Charlson Comorbidity Index (Standard deviation)	4.2 (2.0)	4.3 (2.0)	3.8 (1.8)	0.18
Procedure
LAR	66.5%	68.2%	59.4%	0.34
APR	33.5%	31.8%	40.6%

Table 2.

Checklist compliance rates by item

PREOPERATIVE	Compliance Rate
Formal pathology review	100.0%
Complete colonic evaluation	100.0%
Tumor location documentation	93.7%
Documentation of sexual function and continence	0.0%
Tumor staging (ERUS or MRI)	96.3%
Metastatic staging evaluation	100.0%
Preoperative CEA Measurement	93.8%
Documentation of neoadjuvant therapy consideration	100.0%
Any post-neoadjuvant restaging performed	69.3%
By physical exam only	31.4%
By repeat imaging	37.1%
Documentation of multi-disciplinary discussion of therapy (Tumor Board)	16.8%
Preoperative stoma siting	94.3%
INTRA-OPERATIVE
Exploration for extra-pelvic disease documented	89.4%
Total mesorectal excision performed	98.8%
Distal resection gross margin documented	92.5%
En bloc resection of involved organs	100.0%
Documentation of pelvic nerve integrity	8.7%
Documentation of resection status	88.8%
Rationale for intestinal continuity	91.9%
Documentation of anastomotic type (handsewn vs stapled)	100.0%
Rationale for anastomotic approach (pouch vs end-to-side)	80.0%
Documentation of anastomotic location	92.3%
Documentation of leak test	52.9%
Documentation of diversion consideration	96.0%
POSTOPERATIVE
Stoma care teaching provided	100.0%
Postoperative medical oncology referral for Stage II or III cancers	100.0%
Documentation of radial and distal margin status	57.5%

Analysis of factors related to high checklist compliance found only surgeon experience level significant; senior surgeons accounted for 73% of highly compliant checklists but 87% of low compliant ones (p=0.02). Other factors including age, distance traveled, insurance status, and location of neoadjuvant therapy administration were not significantly associated with high checklist compliance. (Table 3)

Table 3.

Factors associated with checklist compliance

Variable	Percent Total Patients by Variable (n= 161)	Percent High Compliance Patients By Variable (n=76)	Percent Low Compliance Patients by Variable (n=85)	p
Age > 65	30.40%	31.60%	29.40%	0.76
Distance (miles)				0.75
< 25	29.10%	31.50%	28.20%
25–100	37.30%	34.20%	40.00%
> 100	32.90%	34.20%	31.70%
Private insurance	54.00%	51.30%	56.50%	0.51
Neo-adjuvant Treatment Received at Study Hospital	55.70%	54.30%	57.10%	0.73
Surgeon Experience > 10 years	80.10%	72.40%	87.10%	0.02

Three junior surgeons treated 32 (19.9%) of the study’s patients, while the four senior surgeons treated 129 (80.1%). No significant variation in sex, race, insurance status, comorbidity index, or procedure type was noted between these patient groups. (Table 1) However, checklist compliance did significantly vary in several areas. For those patients receiving neoadjuvant therapy, junior surgeons were more likely to restage in general (85.7% versus 64.3%, p=0.029) and use radiographic testing in particular (67.9% versus 29.4%, p<0.001). Junior surgeons documented significantly more multidisciplinary discussions of treatment regimens (31.3% versus 13.2%; p=0.014) and intraoperative anastomotic leak tests (86.7% versus 47.2%; p=0.005). Though the majority did not reach statistical significance, junior surgeons achieved higher rates of compliance on 9 out of the 10 most varied checklist items. (Figure 1)

Figure 1.

Junior and Senior surgeon compliance rates among ten most variable checklist items

*Checklist items with significant difference (p<0.05) in compliance rates between junior and senior surgeons.

Discussion

This study is the first quantitative analysis of a surgical practice’s compliance with a comprehensive, longitudinal checklist for care of the rectal cancer patient. Median checklist item completion rate was 84% per patient. The only significant factor associated with high checklist compliance was surgeon experience less than 10 years. Despite prior association with decreased consistency in rectal cancer care, patient factors such as age, distance traveled to our tertiary center, insurance status, and neoadjuvant treatment location had no effect on checklist compliance. We demonstrate that opportunities for improved adherence to established rectal cancer therapeutic guidelines remain, that significant differences exist between junior and senior faculty documentation and practice patterns, and that further initiatives to guide rectal cancer treatment are needed to completely standardize practice at this high-volume, specialty center. These findings are particularly relevant to the ongoing debate regarding centralization of rectal cancer care.

Most importantly, this study shows that even among a single, specialized surgical practice, adherence to widely published rectal cancer treatment guidelines continues to vary greatly despite repeated national efforts to define and address this problem.(6, 7, 10, 11) This finding echoes recent analyses showing persistent variability in rectal cancer clinical outcomes as well.(12, 13) In fact, the high degree of variability we note among surgeons is precisely why the checklist was created – to provide a tool for rectal cancer care standardization.(8) Therefore, proponents of centralized rectal cancer centers must recognize that simply referring patients to specialty centers – such as the site of this study – will not alone improve treatment variability. Instead, comprehensive efforts including additional initiatives like physician education, mandatory outcomes reporting, and creation of validated point-of-care decision support tools must also be considered. While checklist creation was a critical first step, the current study demonstrates the clear need for a multifaceted approach. This sentiment is distinctly reflected by the actions of the American College of Surgeons, which recently finalized the National Accreditation Program for Rectal Cancer.(14)

Interestingly, the study also found a significant inverse relationship between surgeon experience level and checklist compliance. Experienced surgeons were more likely to omit care components in areas that reinforce treatment decision-making, particularly restaging and inviting multidisciplinary discussion of treatment regimens. This pattern may be attributable to several factors including confidence in their own treatment selection ability, greater familiarity with typical recommendations from partnering medical and radiation oncologists, or an established practice pattern that does not routinely involve these components. While prior work by Russ et al. found senior physicians’ resistance to surgical checklist implementation to be a major barrier, this does not appear to be a significant factor at the study institution since the checklist is not used at the point-of-care.(15) Since our analysis was not intended to assess clinical efficacy of the checklist, we did not examine differences in clinical outcomes due to experience level of treating surgeon. Clearly, however, efforts to improve experienced physician adherence to published guidelines are critical to standardizing care and documentation, especially since these surgeons treat the majority of rectal cancers (over 80%) at the study institution.

A weakness of this study is the retrospective nature of data collection. Since the majority of checklist variables were extracted from the medical record, it was often unclear if the functional aspect of patient care in question was omitted altogether or performed but left undocumented. For example, radial and distal margin status was specifically documented by pathologic report for only 57% of cases. Among the other 43%, the majority (95%) of reports included generic margin descriptions (“Surgical margins negative”); the remainder (5%) had at least one positive margin. Though these findings suggest that a lack of margin status documentation is likely clerical in nature, we could not determine whether complete margin assessment was performed by the reporting pathologist due the retrospective study design. This is an important distinction that should be further evaluated in subsequent prospective analyses.

The ASCRS Rectal Cancer Surgery Checklist has never been prospectively validated, nor has checklist adherence been correlated to clinical outcomes. This study was not intended to do either; instead, it highlights the varied compliance to a recognized national care standard at our specialized center and the correlation between surgeon experience and reduced reliance on checklist items. Since other surgical checklists have proven most effective when used at the point of care, efforts to improve adherence should focus on implementation of the checklist as a decision-support tool as well as analysis of potential associations between checklist completion and clinical outcomes.(16) Further study of checklist implementation must include prospectively collected data, as well as a pre-post assessment focused on factors impeding clinical adoption.

Conclusion

Rectal cancer care and documentation at a large, tertiary specialty practice remains varied despite prior standardization efforts including publication of the ASCRS Rectal Surgery Checklist. With baseline ASCRS checklist compliance now established at a high-volume specialty center, future efforts should focus on checklist implementation techniques, targeted outreach to experienced surgeons treating rectal cancer, and assessment of the correlation between checklist completion and clinical outcomes in this patient population. Proponents of centralized rectal cancer centers should consider additional interventions to improve standardization of rectal cancer treatment in the United States.