Surgeon perceived most important factors to achieve the best hospital performance on colorectal cancer surgery: a Dutch modified Delphi method

Abstract

Objectives

Hospital variation in risk-adjusted outcomes after colorectal cancer surgery has been shown. However, explanatory factors are not sufficiently clear. The objective of this study was to identify factors perceived by gastrointestinal surgeons as important to achieve excellent casemix-adjusted outcomes after colorectal cancer surgery.

Design

Based on literature and experts’ opinion, 86 factors associated with serious complications, failure to rescue and mortality were listed. These were presented to gastrointestinal surgeons through two web-based surveys and an expert meeting. Participants were asked to choose their top 10 of most important factors.

Participants

Dutch gastrointestinal surgeons (n=52) of different hospitals and different hospital types (general/teaching/academic).

Results

Of 31 invited experts for the first survey and meeting, 71% responded. Of 130 invited surgeons, 34 responded to the second survey. Factors deemed important were: procedural hospital volume (46% in top 10), specialised surgeons performing surgery, (elective 87%, emergency 60% and reoperations 62% in top 10), accessibility of, and daily ward rounds by specialised surgeons (41% and 38% in top 10), preoperative screening for malnutrition (57% in top 10), a protocol for recognition of anastomotic leakage and rapid reintervention (54% and 49% in top 10).

Conclusion

Procedural hospital volume, specialisation of surgeons, screening for malnutrition, early recognition of complications followed by rapid action were perceived as most important factors to achieve good outcomes by gastrointestinal surgeons.

Strengths and limitations of this study.

• This study was designed find consensus among gastrointestinal surgeons about important factors leading to the best performance in colorectal cancer surgery, an important first step in targeting quality improvement initiatives.

• Strengths of this study are the extensive questioning, in different rounds and in different manners, increasing the reliability of the identified most important factors.

• Extensive questioning also creates a risk of low response rates.

Introduction

Hospital variation in surgical outcomes is receiving increasing attention. Nowadays, in many countries, quality of surgical care is assured by concentration of care, minimum care standards, increasing specialisation of surgeons and clinical audit projects. Still, variation in casemix-adjusted outcomes between hospitals remains and mechanisms behind this are not completely understood.^1–3

Traditionally, in high-complex and low-volume procedures, variation in postoperative mortality was approached by centralisation of care; reducing complications by increasing hospital procedural volumes. This volume–outcome relationship proved to be less strong in more commonly performed colorectal cancer surgery.^{4 5} Literature shows associations between outcomes and some structural factors, for example, level of intensive care unit, nurse staffing or hospital procedural volume. However, the best-performing hospitals differ in their structural characteristics, and structural factors alone do not explain hospital variation in the outcomes.^6–9

Simultaneous to centralisation efforts, many countries initiated the clinical audits. Their aim was stimulating quality improvement projects in individual hospitals by benchmarking outcomes. In the Netherlands, the Dutch Surgical Colorectal Audit (DSCA) measures and feeds back processes and casemix-adjusted outcomes to all hospitals performing colorectal cancer surgery.¹⁰ Furthermore, an expert team visits hospitals with significantly more serious complications or mortality than average, aiming to initiate an improvement cycle. Over the last years, the Dutch serious complication, ‘failure to rescue’ (FTR) and mortality rates improved, although variation between hospitals remains.^{2 3}

In addition to structural factors, many procedural factors could explain the variation in outcomes, such as differences in postoperative monitoring and time to recognition.^{1 11–13} Furthermore, different healthcare professionals are involved in perioperative patient care: nurses, paramedical personnel and different medical specialists. Hence, teamwork, culture and communication between these professionals may be important.¹¹ Another important factor could be the increasing role of specialised gastrointestinal surgeons rather than the general surgeon, in surgery, but also in postoperative management and daily ward rounds.¹⁴

Which factors are the most important in relation to the observed variation in outcomes is not sufficiently clear. In unravelling this process of care in good performing hospitals, the first step is identifying factors related to differences in outcomes in a structured fashion.^15–17 This study aims to identify factors that could be related to better or worse outcomes in colorectal cancer surgery and to identify those factors that gastrointestinal surgeons perceive as most important to explain hospital differences in outcomes.

Methods

A modified Delphi method was conducted to collect opinions of Dutch surgeons participating in the DSCA (figure 1). The outcomes of interest were: serious complications, FTR and postoperative mortality. Serious complications were defined as complications leading to, prolonged in-hospital stay (>14 days), a reintervention or mortality. FTR was defined as the percentage of patients dying after a serious complication.¹⁸ Two web-based surveys¹⁹ and one expert meeting were held between August 2014 and June 2015. After a literature search for potential influencing factors, a comprehensive list comprised. The first web-based survey (round 1) and the expert meeting (round 2) were used to shorten the initial list of factors. The second web-based survey (round 3) was used to test perceived importance of selected factors among a larger group of gastrointestinal surgeons. The respondents were invited by email, one reminder sent in case of non-response. Both web-based surveys were open for 3 months. Informed consent from respondents was given at the start of the survey.

Figure 1.

Modified Delphi method. DSCA, Dutch Surgical Colorectal Audit.

Literature search

First, literature was searched for factors related to short-term outcomes after colorectal cancer surgery. The search was done in PubMed and Embase. Search terms used were: (morbidity OR mortality OR postoperative complications OR Failure to rescue) AND (Colorectal Surgery OR ((surgery OR surgical OR operative OR operation) AND (colon OR colonic OR rectal OR rectum OR colorectal)). Additional literature searches were done for factors derived from clinical experience. After this initial step, the list contained 86 factors, which were grouped into nine categories.

Round 1: web-based survey 1

The 86 factors were presented to a selected group (n=31) of (mandated) clinical experts in colorectal cancer care through a web-based survey. The respondents were first asked to select their top 10 of most important factors, and to divide 100 points between the factors mentioned in the top 10. Second, they were asked to rank the factors within nine categories on the extent to which factors would vary between hospitals and would be relevant for the outcomes: serious complications, FTR and postoperative mortality. The nine categories covered preoperative, intraoperative and postoperative processes, the process of diagnosis and treatment of complications, structural hospital factors and factors involving the healthcare providers, including their accessibility outside working hours, communication and level of education. One was defined as the highest rank; the lowest rank was dependent on the number of factors present in each category (varying from 8 to 10). Last, the respondents were asked to select their top 10 again, as seeing all factors in detail might have changed their assessment.

Round 2: expert meeting

Results of the first web-based survey were presented during the expert meeting. Of the 31 invited experts, 14 were present at this meeting. Every category was presented and factors were ordered based on a combination of the perceived extent of variation between hospitals and perceived importance for outcomes. Mean rankings within a category for each factor were shown, if this was higher than the median in that category, or if a factor was mentioned in a top 10, it was considered relevant for the next round. The experts had the possibility to switch factors from relevant to irrelevant or vice versa and the possibility to change the categories to which the factors were assigned. If new factors were mentioned during the meeting, consensus was found to decide if these were added to a suitable category.

After the expert meeting, the list consisted of 47 factors divided into 9 (new) categories: preoperative, intraoperative (elective and emergency setting) and postoperative processes, communication, the team of healthcare providers and their accessibility during, and outside working hours, and structural hospital factors.

Round 3: web-based survey 2

The second web-based survey was distributed to all surgeons (n=130) participating in the DSCA. This survey questioned the importance of the 47 factors explaining the hospital variation in outcomes. Respondents were specifically asked to select factors explaining variation between hospitals, so they would not choose factors that are embedded in all Dutch hospitals (eg, enhanced recovery protocols). Again, respondents were asked to select a top 10. Additionally, respondents had the option to divide 100 points among the factors in their top 10 and to rank the factors within their 9 categories.

Analyses

The percentage of respondents selecting a factor in their top 10 was calculated. The factors were ordered based on this percentage from high to low and we selected the ten factors with the highest percentage in the top 10 as most important factors. Potential differences assigned by surgeons working in general hospitals and in academic (affiliated) hospitals were tested using the X² test.

The additional questions were used to calculate a sum of points assigned to a factor and the mean rank (MR) within its category and to order the factors again based on these outcomes. Statistical analyses were performed in IBM SPSS Statistic V.22.

Patient and public involvement

Patients were not involved in this study.

Results

A total of 52 (40%) gastrointestinal surgeons participated in one or more rounds of the modified Delphi method (table 1). Of these respondents, 13 were working in academic, 31 in teaching and 11 in general hospitals. In the first two rounds, the response rate was 71% and in the last round, the response rate was 26% (table 1).

Table 1.

Number of respondents in all rounds

	Invited	Total response	Academic	Teaching	General
Composing list of items total	31	22 (71%)	9	13	0
Websurvey 1 only (round 1)		8
Expert meeting only (round 2)		2
Both round 1 and 2		12
Selecting most important items total	130	34 (26%)	5	18	11
Total participants	130	52 (40%)	13	28	11

Round 1 and 2: composing a list of factors

All new mentioned factors in the first web-based survey were included in our analyses. From the 86 factors initially selected by the project group and 15 new factors, 47 were selected for the next round based on consensus among the experts. Some factors were deleted and some were adapted or merged with another factor. The resulting list of 47 factors across 9 categories was used in the second web-based survey and is presented in table 2.

Table 2.

Percentage of respondents reporting a factor in their top 10, sum of points assigned and mean rank within category

Factors leading to good outcomes in colorectal cancer surgery	Percentage in top 10 n=34, %	Assigned points (minimum–maximum) n=31	Mean rank (SD) n=27–24
Preoperative			n=27
Preoperative screening of patients for malnutrition, followed by dietary measures*	57	150 (2–20)	3.07 (SD 1.639)
Preoperative screening of elderly by a geriatrician	35	61 (3–10)	4.7 (SD 2.072)
Preoperative visit of the patient to an anaesthesiologists	32	63 (3–15)	4.33 (SD 2.572)
Preoperative opportunity to discuss a complex patient in a preoperative discussion with an intensivist or anaesthesiologist	29	93 (5–20)	4.59 (SD 3.067)
Preoperative counselling patients to quit smoking	19	55 (5–20)	5.19 (SD 2.760)
Preoperative pulmonary training	19	53 (0–20)	4.37 (SD 1.757)
The surgeon visits the patient the day before the surgery, or has seen the patient at the preoperative consultation	8	20 (10–10)	5.63 (SD 2.871)
Preoperative visit of patients to a multidisciplinary outpatient clinic	3	7 (7–7)	6.52 (SD 2.190)
The anaesthesiologist that performs the anaesthesia visits the patient the day before the surgery, or sees the patient at the preoperative consultation	3	0	6.59 (SD 2.258)
Intraoperative, elective			n=26
Elective surgery is performed by surgeons with a specialisation in gastrointestinal oncology*	87	358 (5–50)	1.42 (SD 0.987)
A hypovolemic situation during the surgery is actively avoided	32	90 (5–20)	3.73 (SD 1.756)
The ratio between laparoscopy and laparotomy in an elective setting	16	37 (2–20)	4.69 (SD 2.035)
The percentage of patients with elective surgery that receive a definite ostomy	14	35 (5–15)	4.04 (SD 1.708)
The percentage of patients with elective surgery that receive a diverting ostomy in addition to the anastomosis	8	25 (5–20)	3.77 (SD 1.751)
The percentage of patients with laparoscopic surgery that has to be converted to laparotomy	5	0	4.85 (SD 1.515)
The percentage of patients that receive epidural anaesthetics	5	0	5.5 (SD 1.421)
Intraoperative, emergency/urgent			n=26
Emergency or urgent surgery is performed by surgeons with specialisation in gastrointestinal oncology*	60	235 (5–20)	1.31 (SD 0.549)
The percentage of patient that receive an anastomosis in emergency or urgent surgery	22	65 (5–20)	1.88 (SD 0.516)
De ratio between laparoscopy and laparotomy surgery in emergency or urgent setting	5	10 (5–5)	2.81 (SD 0.567)
Postoperative			n=24
Presence of a protocol for early recognition of anastomotic leakage*	54	175 (5–20)	2.96 (SD 1.628)
Accessibility of a surgeon specialised in gastrointestinal oncology to also review a patient, during business hours (beyond ward rounds)*	41	108 (5–15)	2.79 (SD 1.062)
Daily ward rounds by the surgeon that performed the surgery or another surgeon specialised in gastrointestinal oncology*	38	123 (5–20)	2.08 (SD 1.472)
Patients are postoperative admitted on a ward specialised on gastrointestinal and oncological surgery	32	90 (5–20)	3.71 (SD 1.517)
Presence of a protocol for testing CRP and consequences according to outcomes	22	80 (5–20)	4.21 (SD 1.414)
Presence of a case manager who contacts the patient after hospitalisation	3	0	5.25 (SD 1.152)
Complications/reinterventions			n=24
Reoperation is performed by surgeons with a specialisation in gastrointestinal oncology*	62	185 (2–20)	1.71 (SD 0.999)
Time elapsed between first symptoms of a complication and a re-intervention*	49	210 (5–25)	1.62 (SD 0.576)
No of reinterventions per patient with a serious complication	5	10 (10–10)	3.08 (SD 0.776)
The ratio between radiological and surgical reintervention	3	10 (10–10)	3.58 (SD 0.504)
Evening, night and weekend shifts			n=24
24/7 a surgeon specialised in gastrointestinal oncology is ‘on call’ (he/she does not have to be in the hospital, though is available for consultation)*	65	170 (5–15)	1.83 (SD 1.167)
A surgeon specialised in gastrointestinal oncology is present at ward rounds in weekends	14	18 (0–8)	2.96 (SD 1.628)
Surgeon ‘on call’ is present at the evening report	11	20 (10–10)	3.58 (SD 1.558)
The emergency room is 24/7 accessible	8	10 (10–10)	3.08 (SD 1.283)
Presence of a surgeon in the hospital 24/7	3	10 (10–10)	4.92 (SD 1.349)
Presence of a resident in the hospital 24/7	0	0	4.63 (SD 1.245)
Communication			n=24
Communication between nurses and interns, residents or surgeons	30	63 (3–12)	1.54 (SD 0.779)
Communication between surgeons	24	45 (5–10)	2.13 (SD 0.741)
At least monthly discussion of outcomes (including discussion of complications)	19	55 (5–15)	3.46 (SD 0.932)
Communication between surgeon and anaesthesiologist	8	15 (5–10)	2.88 (SD 0.992)
Healthcare providers			n=24
Average experience of nurses on the wards	30	70 (2–20)	1.75 (SD 0.737)
Average experience of interns and residents responsible for the wards	11	18 (8–10)	2.63 (SD 1.173)
The hospital is a referral centre for colorectal surgery	5	5 (5–5)	2.75 (SD 1.073)
No of nurses per patient (nurse/patient ratio)	5	5 (5–5)	2.88 (SD 1.154)
No of surgeons specialised in gastrointestinal oncology in a hospital	5	10 (10–10)	4.29 (SD 1.628)
Hospital structure			n=24
No of colorectal surgeries (both benign and malignant) performed in the hospital annually*	46	103 (2–20)	2.42 (SD 1.586)
Presence of emergency intervention team	24	53 (2–15)	2.54 (SD 1.668)
Accessibility of an intervention radiologist	19	35 (5–10)	4.17 (SD 1.633)
The ICU level of the hospital	11	30 (5–20)	4.08 (SD 1.381)
The operating team also performs other high-complex surgeries	8	17 (2–10)	3.5 (SD 1.445)

*Ten most important factors are marked with.

CRP, C reactive protein; ICU, intensive care unit.

Round 3: identifying most important factors

Forty-four surgeons started the second web-based survey, 34 surgeons selected their top 10 (figure 1). Table 2 shows the percentage of surgeons that selected the factor in the top 10, sorted from highest to lowest importance within their category (10 most important factors marked with *). Our consensus shows 1–3 important factors in seven of the nine categories. In the categories ‘communication’ and ‘healthcare providers’, no factor was selected.

Only one factor differed significantly between surgeons working in general hospitals and surgeons working in academic (affiliated) hospitals. The accessibility of a specialised surgeon to also examine a patient during business hours was mentioned in the top 10 by more surgeons in general hospitals (67%) compared with surgeons in academic (affiliated) hospitals (28%, p=0.025). Other factors did not differ significantly, indicating broad consensus on most important factors regardless of work setting.

Additionally, 31 surgeons assigned points to the factors in their top 10 (total 3100 points) and, 24 surgeons ranked the factors in all categories. The sum of assigned points and MRs (including SD) are shown in table 2. The main analyses and these subanalyses resulted in the same 10 most important factors (marked with * in table 2 and shown in box 1).

Box 1. Surgeons perceived 10 most important factors to achieve the best hospital performance on colorectal cancer surgery.

• Elective surgery is performed by surgeons with a specialisation in gastrointestinal oncology.

• 24/7 a surgeon specialised in gastrointestinal oncology is ‘on call’ (he/she does not have to be in the hospital, though is available for consultation).

• Reoperation is performed by surgeons with a specialisation in gastrointestinal oncology.

• Emergency or urgent surgery is performed by surgeons with specialisation in gastrointestinal oncology.

• Preoperative screening of patients for malnutrition followed by dietary measures.

• Presence of a protocol for early recognition of anastomotic leakage.

• Time elapsed between first symptoms of a complication and a reintervention.

• Number of colorectal surgeries (both benign and malignant) performed in the hospital annually.

• Accessibility of a surgeon specialised in gastrointestinal oncology to also review a patient, during business hours (beyond ward rounds).

• Daily ward rounds by the surgeon that performed the surgery or another surgeon specialised in gastrointestinal oncology.

Figure 2 shows the strong association between percentage of respondents selecting a factor in their top 10 and MR assigned to this factor. It shows all most important factors in the top right part of the figure, indicating a combination of high ranks (highest=1) and high percentages mentioning the factor in the top 10.

Figure 2.

MR within category by percentage of surgeons selecting this factor in their top 10. Most important factors are: (A) Preoperative screening of patients for malnutrition, followed by dietary measures; (B) Elective surgery is performed by surgeons with a specialisation in gastrointestinal oncology; (C) Emergency or urgent surgery is performed by surgeons with specialisation in gastrointestinal oncology; (D) Presence of a protocol for early recognition of anastomotic leakage; (E) Accessibility of a surgeon specialised in gastrointestinal oncology to also review a patient, during business hours (beyond ward rounds); (F) Daily ward rounds by the surgeon that performed the surgery or another surgeon specialised in gastrointestinal oncology; (G) Reoperation is performed by surgeons with a specialisation in gastrointestinal oncology; (H) Time elapsed between first symptoms of a complication and a reintervention; (I) 27/7 a surgeon specialised in gastrointestinal oncology is ‘on call’ (he/she does not have to be in the hospital, though is available for consultation) (J) Number of colorectal surgeries (both benign and malignant) performed in the hospital annually. MR, mean rank.

Discussion

This modified Delphi method identified 10 factors deemed as most important by Dutch gastrointestinal surgeons in explaining variation in casemix-adjusted outcomes after colorectal cancer surgery. Procedural hospital volume was selected as an important structural factor. In six of the most important factors, the specialised surgeon has an important role. In the preoperative phase, screening for malnutrition was considered important; in the postoperative phase, fast recognition of a complication followed by a rapid reintervention and a protocol for early recognition of anastomotic leakage. In total, 10 factors were selected as most important.

Comparing our results to the literature, it is striking that surgeons perceive procedural hospital volume as one of the most important factors. While procedural hospital volume has often been considered a proxy for high quality of care in high-complex and low-volume procedures, the evidence in colorectal surgery is less convincing. Several systematic reviews show a volume–outcome relationship in colorectal cancer surgery. However, the effect sizes are small, it is a non-linear effect and the definition of high hospital volume is heterogenic between different studies. This makes it hard to define a clear threshold for minimum annual number of colorectal cancer surgeries per hospital.^20–24 Also, a difference between US and non-US data suggests that hospital level variability was less of a problem outside the USA than in the USA.^{20 24} Furthermore, Dutch studies show that volume is no guarantee for optimal outcomes in colorectal cancer care in the Netherlands.^{25 26} More importantly, investigating structural hospital factors only will not reveal differences in patterns of care that lead to good outcomes.^{25 26}

In addition to procedural hospital volume, the effect of individual surgeon volume and specialisation of the surgeons seems more important.^{20 21} Respondents in our study strongly expressed that specialisation of the surgeons performing colorectal cancer resections is associated with better outcomes. This was in accordance with studies reporting better survival, but also a better adherence to guidelines and a better use of appropriate surgical techniques.^{27 28} The postoperative presence and accessibility of specialised surgeons and daily presence at ward rounds is considered important by our respondents, although there is little evidence substantiating this.¹⁴ The surgeons perceive that specialisation helps in installing preventive measures, and early recognition and better treatment of complications. Literature shows that for fast recognition and fast treatment, teamwork, communication and culture is important, and that the role of nurses and doctors of the ward is pivotal.^{11 29 30} Interestingly, in our study respondents did not choose the factors on ‘communication’ or ‘experience of healthcare providers’ as most important. However, respondents found adherence to postoperative protocols important, which could be another way to regulate preventive measures, early recognition and better treatment of complications. These postoperative protocols for recognition of anastomotic leakage as well as standardised care programmes are developed to improve patient safety and surgical outcomes.^{31 32} Adherence to these protocols is not well known and can very well differ between hospitals. Better adoption of programmes and protocols that have shown to improve postoperative outcomes could lead to an improvement of colorectal cancer care.^{33 34}

With an increasing population of elderly patients and a high prevalence of malnutrition in colorectal cancer patients, risk assessment and personalised preparation is increasingly important. Risk assessment is mostly done with the American Society of Anesthesiologist classification, but also specific scores for screening on malnutrition are used, for example, malnutrition universal screening tool or Short Nutritional Assessment Questionnaire. Our respondents perceive screening on malnutrition as highly important. Future data collection on these factors should show the extent to which this explains the observed variation in outcomes between hospitals.

There are limitations to this study. First, 40% of the invited surgeons participated in one or more rounds of this study, but 26% participated in the last round. Although these percentages are common in surveys among professionals, this could imply selection bias.^{35 36} However, the respondents were distributed across different types of hospitals (general and academic (affiliated) hospitals) in the same way this is distributed in the non-responders. This makes it plausible that our responders could reflect the beliefs of their (non-responding) colleagues. Second, we have no information on the experience of the surgeons. Still, because the invited surgeons were participating in the DSCA, interest is expected, and specialisation in colorectal cancer surgery is likely. In this study, surgeons were invited and no nurses or doctors of the wards, which might be reflected in the answers given. Perspectives of other healthcare providers and even patients could result in selection of other factors than the ones listed in this study. However, because surgeons make most perioperative decisions and are up to date on available evidence regarding colorectal surgery, they were thought to have the best expertise to judge which factors might explain the observed variation in outcomes between hospitals. Certainly, this study did not address all factors possibly influencing outcomes in colorectal cancer care (eg, discharge follow-up or innovative technologies being offered). However, the surgeons had the possibility to add new factors, which they thought to be important, in the expert meeting and those factors were included in the study. Furthermore, we investigated the opinion of Dutch gastrointestinal surgeons. Still, we feel that this study could be applicable to other countries with comparable processes already implemented nationwide. Last, we decided to select 10 most important factors. This cut-off point is arbitrary.

Conclusion

According to Dutch surgeons, important factors to achieve low morbidity and mortality, and to explain hospital variation in casemix-adjusted short-term postoperative outcomes after colorectal cancer surgery are: procedural hospital volume, specialisation of surgeons, screening for malnutrition and early recognition of complications followed by rapid action. More research is needed to determine to what extent the hospital variation in outcomes of colorectal cancer surgery in the Netherlands is explained by these factors. The consensus present on these factors could form the basis for further in-depth research on possible differences between hospitals with better and worse outcomes.