Assessing Surgeon Behavior Change after Anastomotic Leak in Colorectal Surgery

Abstract

Background

Recency effect suggests that people disproportionately value events from the immediate past when making decisions, but the extent of this impact on surgeons’ decisions is unknown. This study evaluates for recency effect in surgeons by examining use of preventative leak testing before and after colorectal operations with anastomotic leaks.

Materials and Methods

Prospective cohort of adult patients (≥18 years) undergoing elective colorectal operations at Washington State hospitals participating in the Surgical Care and Outcomes Assessment Program (2006–2013). The main outcome measure was surgeons’ change in leak testing from 6 months before to 6 months after an anastomotic leak occurred.

Results

Across 4,854 elective colorectal operations performed by 282 surgeons at 44 hospitals, there was a leak rate of 2.6% (n=124). The 40 leaks (32%) in which the anastomosis was not tested occurred across 25 surgeons. While the ability to detect an overall difference in use of leak testing was limited by small sample size, 9 (36%) of 25 surgeons increased their leak testing by 5 percent points or more after leaks in cases where the anastomosis was not tested. Surgeons who increased their leak testing more frequently performed operations for diverticulitis (45% vs 33%), more frequently began their cases laparoscopically (65% vs 37%), and had longer mean operative times (195±99 vs 148±87 minutes), all p<0.001.

Conclusions

Recency effect was demonstrated by only one-third of eligible surgeons. Understanding the extent to which clinical decisions may be influenced by recency effect may be important in crafting quality improvement initiatives that require clinician behavior change.

INTRODUCTION

Over the last few decades, behavioral economics has challenged traditional understanding of how people make decisions.^1–3 Psychologists, consumer scientists and economists have shown that people deviate from economically rational decisions in predictable ways, in part because of human limitations on computational power, willpower, and self-interest.^3,4 Many have proposed interventions – termed “nudges” or “choice architecture” – to take advantage of the heuristics and biases highlighted by behavioral economics in order to improve people’s decisions. However, whether behavioral economics can be used to improve the decisions made by clinicians has not been well established.⁵

For example, behavioral economics has highlighted that people disproportionately value events that occurred recently compared to those that occurred further in the past. This so-termed “recency bias” is well-recognized in financial domains to explain recent performance of stock markets and are used to guide sales and purchase behaviors.⁶ Other disciplines term this decision tendency “recency effect” or “availability heuristic,” noting a link between recent events and increased estimation of similar events in the future.^7,8 There is increasing evidence to suggest clinicians’ behaviors may also be influenced by availability or recency^9,10 and anecdotally, decisions based on recent personal experience appear pervasive in clinical practice.¹¹ Quality and surveillance databases may provide a unique opportunity to leverage observational data to evaluate how clinicians change their behavior after recent incidents in clinical settings.

The objective of this study was to evaluate for recency effect by examining surgeon use of anastomotic leak test before and after colorectal cases with anastomotic leaks. Intra-operative leak-testing can reduce the risk of anastomotic leaks, rare but potentially life-threating complications, after surgery by up to 50%.¹² However, not all surgeons routinely leak test, suggesting leak testing is not equally valued. Since perceived value may drive behavior, understanding how surgeons change their use of leak testing after anastomotic leaks may be an opportunity to evaluate recency effect in clinical practice. We hypothesized that surgeons might display recency effect by increasing the use of leak testing in operations subsequent to an anastomotic leak.

MATERIALS AND METHODS

This study was determined as not human subjects research by the University of Washington’s Institutional Review Board.

Data Source and Population

The primary cohort was defined by all consecutive adult (≥18 years) patients who underwent elective colon or rectal resection between January 1, 2006 and December 31, 2013 at 44 Washington State hospitals that participate in the Surgical Care and Outcomes Assessment Program (SCOAP). Cases without an anastomosis were excluded. Unique, hospital-specific codes were assigned to each surgeon, so data could be clustered at hospital and surgeon levels. Data from surgeons who performed 2 or fewer cases, or those who did not have cases within 6 months of each other to permit evaluation of 6-month rates, were excluded. In addition, to avoid left- and right-censoring bias, we excluded surgeons who only had cases in the first 6 months or last 6 months of their enrollment in SCOAP, respectively. The final cohort included 4,854 cases performed by 282 surgeons from 44 hospitals. For each case, sociodemographic, clinical, and operative details were extracted from inpatient medical records by trained chart abstractors at each clinical site. SCOAP metrics and data dictionary are available via a secure page at www.SCOAP.org. A modified Charlson comorbidity index for each patient was calculated.¹³

Definitions

Leak test

Only cases with a testable anastomosis were included (left colectomy, low-anterior resection including sigmoidectomy, and total abdominal colectomy with ileosigmoid/rectal anastomosis). Since anastomotic leak testing can be performed by using an endoscope, methylene blue dye, or air/saline injection, these methods were combined into the composite definition of “leak test.”

Leaks

Anastomotic leaks are rare and can present variably after an operation. Accordingly, we grouped post-operative leak events into a composite term “leak,” defined as radiologically-demonstrated anastomotic leak or enterocutaneous fistula, postoperative percutaneous drainage of abscess, or unplanned re-operative intervention requiring colostomy/ileostomy, abscess drainage, operative drain placement or anastomotic revision.

Outcomes

The main outcome was change in rate of leak-testing by individual surgeons. Since rates depend on number of cases performed during a time period, we limited our evaluation to rates of leak-testing 6 months before and 6 months following a leak. In addition, our prior evaluation of SCOAP data suggests that the median number of colorectal procedures was 5 per year per surgeon. Accordingly, we defined higher-volume surgeons as those who performed 5 or more elective colectomies per year.

Analysis

Longitudinal patterns of leak testing were constructed for each surgeon. Rates of leak testing (95% CI) before and after a leak were determined, stratified by surgeon’s case volume and whether a leak test was performed during the case with a leak. We compared change in surgeons’ leak testing after a leak to changes in leak testing in cases without a leak using a linear-regression based difference-in-difference non-parametric model, clustered at the surgeon level. Our model adjusted for surgeon-specific rates of protective stoma creation, which may serve as an alternative to leak testing. Cognizant that leak testing after a leak might be different after leaks that occurred despite performing a leak test, and that high-volume and low-volume surgeons might have difference responses to leaks, we defined 4 strata based on surgeon volume and whether a leak test was performed and applied the difference-in-difference model for each stratum (so that similar surgeons were being compared – e.g. leak-testing change for low-volume surgeon who had leaks compared to leak-testing change in low-volume surgeons who did not have leaks). While it is unclear what should be considered a “meaningful” increase in leak testing, we defined recency effect apriori as increase in leak testing rates by 5 or more percent points from the surgeon’s baseline in the prior 6 months. To explore potential differences in surgeons who had a leak in an untested case, we summarized patient characteristics, operative indications, and outcomes, stratified by surgeons who did and did not display recency effect. Data were summarized using frequency distributions for categorical variables and mean (±SD) for continuous variables. Categorical variables were compared using Pearson χ² statistic. Continuous variables were compared using the Wilcoxon rank-sum (Mann-Whitney) test. A p-value of less than 0.05 was considered statistically significant throughout. All analysis was performed using STATA version 13 (STATA Corp, College Station, Tex).

RESULTS

From 2006 to 2013 in Washington State, across 4,854 elective colorectal operations performed at 44 hospitals, leaks occurred in 124 cases (2.6%). Leak-testing was performed in 71.4% (n= 3,467) of cases, increasing from 60.0% in 2006 to 79.6% of cases in 2013 (p-trend<0.001), but this rise was not different between surgeons who did and did not have leaks (p=0.87). There were 40 cases (32%) with leaks in which the anastomosis was not tested, performed by 25 surgeons.

Longitudinal leak testing patterns were constructed for all surgeons, as in Figure 1. Surgeon A demonstrated recency effect by transitioning to essentially routine leak testing after a leak in an untested case. Surgeon B, however, does not appear to meaningfully change leak testing practice, despite several leaks in both tested and untested cases.

Figure 1.

Examples of individual surgeons’ leak testing patterns demonstrating recency effect(A), a shift to essentially-routine leak testing after anastomotic leak, and (B) no change in leak testing despite multiple leaks in both tested and untested cases.

In the 6 months after a leak, surgeons having leaks in cases that were leak tested had minimal change in rates of testing (Table 1). Lower-volume surgeons who did not leak test at the time of the case with a leak increased their leak testing from 41% to 64%, while higher-volume surgeons decreased their leak testing from 71% to 59%, though these changes did not reach statistical significance using our difference-in-difference analysis (all p>0.1).

Table 1.

Leak testing rates before and after anastomotic leak, stratified by surgeon volume and whether or not leak testing was performed during the case that leaked.

Surgeon volume	Leak Test	Clustered by Surgeon		P-value1
		6 months preceding leak Testing Rate, % (95% CI)	6 months following leak Testing Rate, % (95% CI)
< 5 cases per year	No	41.0% (16.7% – 65.3%)	63.6% (36.2% – 91.1%)	0.50
	Yes	64.2% (43.9% – 84.4%)	67.3% (43.0% – 91.5%)	0.73
5+ cases per year	No	70.7% (55.6% – 85.8%)	59.2% (44.3% – 74.1%)	0.19
	Yes	75.5% (69.2% – 81.8%)	75.3% (67.9% – 82.7%)	0.91

¹Difference-in-difference non-parametric model, clustered at the surgeon level, was used to compare change in surgeons’ leak testing after a leak to change in leak testing across cases without a leak. The model adjusted for surgeon-specific rate of protective stoma, as creating a protective stoma may be a reason to not leak test an anastomosis.

Four surgeons experienced multiple leaks in untested cases (three experienced 2 leaks, one experienced 3 leaks). Only the first leak case was evaluated for recency effect in these surgeons. Nine of 25 surgeons (36%) demonstrated recency effect by increasing their leak testing by 5 percent points, and 5 surgeons (20%) increased their leak testing by 10 or more points. The number of cases per year and mean leak testing rates between surgeons who did and did not demonstrate recency effect were similar (Table 2). Surgeons demonstrating recency effect more frequently performed operations for diverticulitis (45% vs 33%, p<0.001), more frequently began their cases minimally invasively (65% vs 37%, p<0.001), and had longer mean operative times (195±99 vs 148±87 minutes, p<0.001). Notably, the patients of surgeons who did not increase their leak testing received protective stomas twice as often (20% vs 9%, p<0.001).

Table 2.

Patient characteristics, operative details and outcomes for surgeons who did and did not display recency effect.

	No Recency Effect1		Recency Effect1		P-value2
SURGEONS3	N=16	64.0%	N=9	36.0%
Mean cases/year (±SD) 4	22.6	±16.6	14.4	±6.6	0.38
Mean Test Rate (±SD), %4	66.0%	±14.0%	61.3%	±22.7%	0.95
Mean Leak rate (±SD), %4	4.2%	±3.4%	4.1%	±2.7%	0.29
CASES5	N=890	69.6%	N=388	30.4%
DEMOGRAPHICS
Mean age (±SD), years4	59.0	±13.4	60.7	±13.6	0.03
Male	410	46.1%	197	50.8%	0.12
Private Insurance	640	72.8%	270	69.6%	0.24
Comorbidity Index: 3+					0.88
0	672	75.5%	292	75.3%
1	174	19.6%	75	19.3%
2	32	3.6%	17	4.4%
3+	12	1.3%	4	1.0%
BMI 30+	276	32.1%	105	27.4%	0.10
Indication					<0.001
Cancer6	319	35.8%	130	33.5%
Diverticulitis	294	33.0%	176	45.4%
Inflammatory7	34	3.8%	8	2.1%
Other	243	27.3%	74	19.1%
Current Cigarette Smoker	168	19.0%	66	17.1%	0.419
OPERATIVE
Leak-Tested	580	65.2%	240	61.9%	0.256
Minimally Invasive8	327	36.8%	251	64.7%	<0.001
Colon Operation					0.048
Left colectomy	180	20.2%	87	22.4%
Low Anterior Resection9	647	72.7%	287	74.0%
Total abdominal colectomy	63	7.1%	14	3.6%
Mean OR time (±SD), min4	148	±87	195	±99	<0.001
Protective Stoma10	179	20.2%	35	9.1%	<0.001
OUTCOMES
Leak	41	4.6%	18	4.6%	0.98
Mean LOS (±SD), days4	6.5	±5.1	5.8	±5.0	<0.001
Discharge Disposition					0.347
Home	832	93.5%	371	95.9%
Rehab	8	0.9%	3	0.8%
SNF	38	4.3%	12	3.1%
Acute Care	4	0.4%	0	0.0%
Death	8	0.9%	1	0.3%

¹Recency effect defined as increase in leak testing by 5 percent points or more in 6 months after leak compared to 6 months before

²Reported as Pearson’s Chi-squared unless otherwise stated.

³Total number of Surgeons who had a leak in untested case

⁴Using the Wilcoxon rank-sum (Mann-Whitney) test

⁵Cases performed by surgeons in 6 months before and after the untested anastomotic leak

⁶Includes colon cancer, rectal cancer, colon mass or polyps

⁷Includes Crohn’s disease and ulcerative colitis.

⁸Includes laparoscopic, laparoscopic converted to open, laparoscopic/hand-assisted, robotic or robotic converted to open

⁹Including sigmoidectomy

¹⁰Including colostomy or ileostomy.

DISCUSSION

Recency effect was associated with leak testing decisions for only one-third of surgeons. In the majority of surgeons, leak testing may have been resistant to recency effect because of practice preferences or external factors.

Behavioral economics and decision-making science more broadly are only recently being applied to healthcare questions.⁵ Despite the potential benefit of choice architecture demonstrated in non-healthcare decisions,^2,3 work remains to be done in terms of standardization and dissemination of concepts, methods, and implementation of behavioral economic approaches in healthcare.⁵ Since clinician decision making directly impacts safety, quality and effectiveness, understanding patterns of clinician decision making has the potential to impact many outcome domains. Furthermore, as clinicians appear to be influenced by availability and recency effect,^9,10 creating reminders, prompts, or other decision architecture that taps into past events may be an opportunity to improve decisions.³

In this study, however, we found that recency effect was associated with only a minority of surgeons. Still, it is unclear what proportion of a clinician population would be needed to exhibit recency effect to make it a relevant target for choice architecture – 36% of surgeons increased by 5 percent points, and 20% increased by 10 percent points or more. Leak testing may be resistant to recency effect because, at a rate of only 2%, leaks may occur too infrequently to be impactful on a surgeon’s decision to leak test. Alternatively, leak testing may be influenced by factors other than a surgeon’s experience with a recent leak. Contemporaneous to the collection of this data, a state-wide campaign at SCOAP hospitals to increase leak testing was in place, and many training programs now teach routine leak testing. In this environment, many surgeons may have increased leak testing rates for medico-legal reasons as evidence supporting leak testing also emerged in this time period.^12,14 To address the potential influence of temporal change on our results, we performed a sensitivity analysis adjusting for year of surgery, but found no change in leak-testing patterns from our original model. In addition, while surgeons routinely make decisions on behalf of their patients intra-operatively, it has been proposed that “agents” who make decisions for others may not demonstrate the same regret and loss aversion from recent events (leaks are “experienced” by the patient) as might have been predicted.^15,16 To that end, it remains to be determined how much change in leak testing should be considered to reflect recency from an anastomotic leak. We chose to define recency effect as 5 percent points within a 6-month period, but it may be that a higher or lower rate may be better suited for various clinical situations.

Within these circumstances, the characterization of surgeons who increased their leak testing may provide some insight. While overall rates of testing and leaks were not different, surgeons who increased their leak testing after anastomotic leaks had a higher proportion of patients with nonmalignant indications, longer operative times and more frequently started cases laparoscopically (Table 2). Importantly, with such few surgeons in each group, these patient characteristics may be disproportionately influenced by the higher-volume surgeons. Still, this constellation of findings raises the possibility that surgeons displaying recency effect may be newly appointed. While this de-identified dataset precludes us from confirming this hypothesis, the tendency of younger surgeons to change their behavior differently from more experienced surgeons is supported by prior studies.^17,18

This study has several limitations. First, important details about intra-operative decisions were not available through SCOAP, including leak test results or surgeon’s perceived quality of the anastomosis. To address this, we adjusted for surgeon-specific rates of protective stoma, but found no impact on the rates of leak-testing change. It is important to note, however, that surgeons who did not increase their leak testing more frequently performed diverting stomas, but whether these were planned apriori or as a result of intra-operative events cannot be determined. Additionally, since SCOAP data is used for quality and surveillance purposes, surgeon-level data are de-identified, such that specialty training or practice across multiple hospitals is not available. These factors may influence leak testing of individual surgeons and potentially confound the apparent lack of change in our analysis. Furthermore, SCOAP data is abstracted from operative reports and is subject to variability in surgeon’s documentation. For instance, leak testing may be so routine for a particular surgeon that they do not always document it, which would result in rates of leak-testing being under-reported. Alternatively, a surgeon may be more apt to document their leak-test in a high-risk case. This ‘selective reporting’ could make it seem that leak testing is a marker of an anastomotic leak, as has been previously described by our group.¹² Therefore, it is possible that instead of surgeons changing their practice after a leak, they simply became better at documenting the results of their leak tests in the operative report as a method of defensive medicine. Finally, because leaks are rare, our surgeon sample size was small. As an estimate, to detect a difference between 60% and 80% leak testing (alpha = 0.05, power =0.90) would require 120 surgeons that had leaks in untested cases. Cognizant of this limitation, we did not subdivide our composite leak definition into separate grades of leak. It is possible, however, that anastomotic leaks with more clinical significance (requiring reoperation or resulting in patient death, for example), may be more impactful in surgeon behavior change.

While only a minority of surgeons increased leak testing after recent leak, understanding which clinical decisions may be influenced by recency effect and the extent of that influence may be important in crafting quality improvement initiatives that require clinician behavior change. Acknowledging that not all clinicians exhibit recency effect, it may be that personalized choice architecture for certain clinicians is warranted. Behavioral economic theory offers new approaches to research clinician decision making and may yield tools to improve delivery of clinical care.