Effect of public reporting of surgeons’ outcomes on patient selection, “gaming,” and mortality in colorectal cancer surgery in England: population based cohort study

Abstract

Objective

To determine the effect of surgeon specific outcome reporting in colorectal cancer surgery on risk averse clinical practice, “gaming” of clinical data, and 90 day postoperative mortality.

Design

National cohort study.

Setting

English National Health Service hospital trusts.

Population

111 431 patients diagnosed as having colorectal cancer from 1 April 2011 to 31 March 2015 included in the National Bowel Cancer Audit.

Intervention

Public reporting of surgeon specific 90 day mortality in elective colorectal cancer surgery in England introduced in June 2013.

Main outcome measures

Proportion of patients with colorectal cancer who had an elective major resection, predicted 90 day mortality based on characteristics of patients and tumours, and observed 90 day mortality adjusted for differences in characteristics of patients and tumours, comparing patients who had surgery between April 2011 and June 2013 and between July 2013 and March 2015.

Results

The proportion of patients with colorectal cancer undergoing major resection did not change after the introduction of surgeon specific public outcome reporting (39 792/62 854 (63.3%) before versus 30 706/48 577 (63.2%) after; P=0.8). The proportion of these major resections categorised as elective or scheduled also did not change (33 638/39 792 (84.5%) before versus 25 905/30 706 (84.4%) after; P=0.5). The predicted 90 day mortality remained the same (2.7% v 2.7%; P=0.3), but the observed 90 day mortality fell (952/33 638 (2.8%) v 552/25 905 (2.1%)). Change point analysis showed that this reduction was over and above the existing downward trend in mortality before the introduction of public outcome reporting (P=0.03).

Conclusions

This study did not find evidence that the introduction of public reporting of surgeon specific 90 day postoperative mortality in elective colorectal cancer surgery has led to risk averse clinical practice behaviour or “gaming” of data. However, its introduction coincided with a significant reduction in 90 day mortality.

Introduction

In 2012 NHS England, the national body that coordinates the commissioning of health services in the English National Health Service (NHS), called for quality measures and mortality data to be made publicly available.¹ In response, public reporting of named surgeon specific outcomes was introduced in June 2013 across nine surgical specialties, with the data being provided by pre-existing national clinical audits. Public reporting has now been extended to most surgical and some interventional specialties.

Since the reporting of surgeon specific outcomes was introduced in 2013, case numbers and risk adjusted 90 day mortality rates in patients undergoing an “elective” or “scheduled” major colorectal cancer resection have been reported for individual consultant colorectal surgeons. These rates are publicly available on the websites of the Association of Coloproctology of Great Britain and Ireland (ACPGBI) and NHS Choices.² The results are based on analyses of data provided by the National Bowel Cancer Audit (NBOCA).³ This national audit has been publishing surgical mortality results at hospital trust level in its annual reports since 2010.⁴

The publication of surgeon specific outcomes has been controversial.^{5 6 7} Proponents believe that surgeon specific outcome reporting facilitates transparency, allows patients to make informed choices, and may drive quality improvement.⁸ Critics argue that the potential professional and financial implications of the public reporting of perceived negative outcomes encourages risk averse behaviour, whereby surgeons are less likely to offer surgery to patients at higher risk. Data may also be manipulated to increase patients’ predicted risk or to make patients ineligible for public reporting, which is often referred to as “gaming.” So far, the evidence that reporting of surgeon specific outcome leads to improvements in the quality of patient care is surprisingly weak. The effect of public reporting of surgeon specific outcomes has been studied only in cardiac surgery, and these studies were almost exclusively carried out in the United States [8].

In this study, we analysed NBOCA data to look for evidence of risk averse behaviour, manipulation of data, and change in surgical mortality in the time period immediately before and after the introduction of surgeon specific outcome reporting in colorectal cancer surgery.

This is the first time that the effect of public reporting of surgeon specific outcomes has been presented outside cardiac surgery. Uniquely, we studied its effect on all patients with colorectal cancer diagnosed during the study period, irrespective of whether they underwent elective major surgery. In this way, we overcame a major limitation in all published studies to date that included only patients who had surgery and therefore could not assess the effect of risk averse behaviour and changes in patient selection.

Methods

Data collection and reporting of surgeon specific outcomes

NHS hospital trusts in England have a mandatory requirement to submit data for all patients with a new diagnosis of colorectal cancer via a secure online platform to the NBOCA database. Information is collected that captures surgical urgency (categorised as elective, scheduled, urgent, or emergency according to the National Confidential Enquiry into Patient Outcome and Death (NCEPOD) classification of intervention),⁹ the American Society of Anesthesiologists (ASA) grade, pathological TNM staging, cancer site within the colon or rectum, and surgical procedure performed.

Data from NBOCA is linked to Hospital Episode Statistics (HES), the administrative database of all admissions to NHS hospitals in England. Information on admission type (elective or emergency), the formation of a stoma, and comorbidities is identified from these linked HES records. The Royal College of Surgeons’ Charlson score is used to identify comorbid conditions in the HES records in the preceding year.¹⁰ The date of death is obtained from linked data from the Office for National Statistics (ONS).¹¹

All elective or scheduled major colorectal cancer resections in patients aged 18 years and older at diagnosis, and listed under a consultant surgeon’s General Medical Council (GMC) code, are eligible for surgeon specific outcome reporting. This code is a unique identifier that is available for all clinicians registered with the GMC, the body that maintains the official register of medical practitioners in the UK.¹² In the UK, a consultant surgeon is a surgeon who has completed all his or her specialist training and has been placed on the GMC’s Specialist Register.

Reports of the eligible patients are sent to the NHS hospital trusts, which allows them to check the data before analysis. Hospital trusts are notified of observed and adjusted 90 day mortality rates at the level of both the surgeon and the trust before publication. Surgeons’ outcomes are not published individually for surgeons who have carried out fewer than 10 major cancer resections within the reporting period.

Study population and outcome measures

We included in this study patients submitted to the NBOCA dataset and diagnosed as having primary colorectal cancer from 1 April 2011 to 31 March 2015. The quality of NBOCA data improved considerably from 1 April 2011, which was why we chose this start date. Data collection for metastatic disease was changed from Dukes’ stage to M stage in 2013. This change corresponded with a drop in data completeness.

Risk aversion in bowel cancer surgery may manifest as surgeons becoming less willing to offer patients a surgical resection, especially patients deemed to be at higher risk. In addition, surgeons may be more likely to refer patients to hospital trusts deemed to be more specialised. Also, in treating rectal cancer, surgeons may be more inclined to offer patients a surgical procedure associated with lesser morbidity and mortality, such as performing a defunctioning stoma at the time of anterior resection or preferentially performing an abdominoperineal excision of rectum or Hartmann’s procedure over restoring intestinal continuity. Surgeons may also be more disposed to offer minimally invasive procedures, such as endoscopic mucosal resection or transanal endoscopic microsurgery (which are not included in public reporting).

To investigate risk averse behaviour, we therefore studied the overall rate of major resection, the characteristics of patients undergoing major resection and their tumours, the predicted 90 day mortality according to characteristics of patients and tumours, the proportion of patients undergoing surgery at a different hospital trust from the one at which their tumour was diagnosed, and the type of treatment in patients with rectal cancer. We compared all of these before and after the introduction of public reporting of outcomes.

The deliberate manipulation, or gaming, of data may manifest in data submitted to the audit that is more “subjective” in nature, including the classification of surgical urgency and ASA grade. Gaming of surgical urgency data may result in an increase in the number of patients who had elective surgery being reclassified as “urgent” or “emergency,” and therefore recorded as ineligible for surgeon specific outcome reporting. Of the nine variables included in the risk adjustment score for 90 day mortality,¹³ only ASA grade can be considered subjective. Consequently, a tendency may exist to report higher ASA grades in patients undergoing elective surgery, which would lead to higher predicted surgical 90 day mortality and, in turn, a lower overall adjusted mortality. To assess for evidence of gaming of data, we compared the proportion of patients undergoing major resection who were classified as urgent or emergency (and therefore ineligible for surgeon specific outcome reporting) and the ASA grade recorded in patients undergoing elective major resection, before and after the introduction of surgeon specific outcome reporting.

To determine the effect of surgeon specific outcome reporting on outcomes for patients eligible for inclusion in public outcome reporting, we compared the observed 90 day postoperative mortality without and with adjustment for characteristics of patients and tumours in patients undergoing elective or scheduled major resection. We included patients ineligible for public outcome reporting as comparison groups. For this purpose, we studied the observed and adjusted 90 day postoperative mortality for patients undergoing urgent or emergency major resection and the observed six month survival after diagnosis of patients not undergoing major resection. We chose a longer analysis period of six months after diagnosis for these patients to reflect the fact that most would not undergo surgery and would have received palliative management.

Statistical analysis

The risk adjustment model used by NBOCA was developed and validated previously.¹³ The risk factors included in this logistic regression model are age, sex, ASA grade, Charlson comorbidity score, mode of admission, site of tumour, and pathological T stage, N stage, and M stage, with an additional interaction between age and M stage. We modelled a random intercept for each hospital trust to reflect the possible clustering of results within trusts. Missing values for the risk factors were imputed with multiple imputation using chained equations creating 10 datasets and using Rubin’s rules to combine the model estimates across the datasets. An adjusted outcome was then produced by indirect standardisation.¹⁴ We compared proportions before and after the introduction of public reporting with a χ² test, median age with the Kruskal-Wallis test, and differences in predicted 90 day mortality with a t test.

We used a change point analysis to study the change over time in adjusted 90 day mortality after elective/scheduled major resection (eligible for surgeon specific outcome reporting) and after urgent/emergency major resection (ineligible for surgeon specific outcome reporting).¹⁵ The change point was fixed to the date of the introduction of surgeon specific outcome reporting (28 June 2013). We used a multivariable logistic regression model for 90 day mortality, with a slope for calendar time and an interaction between time pre-introduction versus post-introduction of surgeon specific outcome reporting, in addition to all of the risk adjustment variables. This modelled a change in the slope of mortality at the point that surgeon specific outcome reporting was introduced but no immediate change in mortality.

As a sensitivity analysis, we fitted two further change point models to the same data. The first modelled an immediate shift in mortality at the introduction of surgeon specific outcome reporting by including a term for before and after the introduction of public reporting. The second modelled both an immediate shift and a change in the slope. We used Stata version 14.1 for all analyses.

Patient involvement

No patients were involved in setting the research question or the outcome measures, nor were they involved in developing plans for recruitment, design, or implementation of the study. No patients were asked to advise on interpretation or writing up of results. The results of this research will be publicly disseminated via the National Bowel Cancer Audit website as well as through the charity and patient representatives who act as part of the audit’s Clinical Advisory Group.

Results

Risk averse behaviour

We identified 111 431 patients diagnosed as having colorectal cancer during the study period. Of the 62 854 patients who received a diagnosis before the introduction of public reporting, 39 792 (63.3%) had a major resection, and this was the case for 30 706 (63.2%) of the 48 577 patients who received a diagnosis thereafter (P=0.8) (fig 1).

Fig 1.

Proportion of patients undergoing major resection according to date of diagnosis. Dotted line represents date of introduction of surgeon specific public outcome reporting

The characteristics of the 33 638 patients who had an elective or scheduled major resection before the introduction of public reporting and the 25 905 patients who had one afterwards differed little (table 1). The proportion of patients with metastatic disease decreased from 9.6% before to 7.2% after the introduction (P<0.001). The mean predicted 90 day mortality based on characteristics of patients and tumours did not change (2.7% before and 2.7% after the introduction of public reporting; P=0.3) (fig 2). The proportion of patients undergoing surgical resection at a different hospital trust from where they were given a diagnosis slightly reduced; 1981/33 638 (5.9%) patients given a diagnosis before introduction and 1401/25 905 (5.4%) patients given a diagnosis afterwards were treated in a different hospital (P=0.01).

Table 1.

Clinico-pathological characteristics of patients undergoing elective or scheduled major resection from 1 April 2011 to 31 March 2015 according to year of surgery. Values are numbers (percentages) unless stated otherwise

Characteristics	Before public outcome reporting (n=33 638)	After public outcome reporting (n=25 905)	Total (n=59 543)	P value
Male sex	19 504 (58.0)	14 807 (57.2)	34 311 (57.6)	0.04
Median (IQR) age, years	70 (62-78)	70 (62-78)	70 (62-78)	0.6
ASA grade:
1	4410 (14.0)	3481 (14.1)	7891 (14.1)	0.08
2	18 263 (58.0)	14 524 (58.9)	32 787 (58.4)
3	8202 (26.1)	6247 (25.3)	14 449 (25.7)
4/5	597 (1.9)	426 (1.7)	1023 (1.8)
Missing	2166	1227	3393
Charlson comorbidity score:
0	23 053 (70.2)	16 748 (67.5)	39 801 (69.0)	<0.001
1	7397 (22.5)	5874 (23.7)	13 271 (23.0)
2	2397 (7.3)	2191 (8.8)	4588 (8.0)
Missing	791	1092	1883
T stage:
0	534 (1.6)	496 (1.9)	1030 (1.8)	0.03
1	2516 (7.7)	1879 (7.3)	4395 (7.5)
2	6046 (18.4)	4791 (18.7)	10 837 (18.5)
3	17 652 (53.8)	13 733 (53.6)	31 385 (53.7)
4	6075 (18.5)	4724 (18.4)	10 799 (18.5)
Missing	815	282	1097
N stage:
0	20 012 (61.0)	15 849 (61.9)	35 861 (61.4)	0.09
1	8190 (25.0)	6275 (24.5)	14 465 (24.8)
2	4620 (14.1)	3502 (13.7)	8122 (13.9)
Missing	816	279	1095
M stage:
0	29 398 (90.5)	21 498 (92.8)	50 896 (91.4)	<0.001
1	3103 (9.6)	1668 (7.2)	4771 (8.6)
Missing	1137	2739	3876
Referred to alternative trust for surgery	1981 (5.9)	1401 (5.4)	3382 (5.7)	0.01

ASA=American Society of Anesthesiologists; IQR=interquartile range.

Fig 2.

Predicted 90 day mortality for elective and scheduled major resection according to date of surgery. Dotted line represents date of introduction of surgeon specific public outcome reporting

We found no evidence of a change in the care received by patients with rectal cancer. Of the 18 700 patients diagnosed as having rectal cancer before the introduction of public reporting, 1274 (6.8%) had a local excision compared with 1012 (7.0%) of the 14 457 diagnosed after the introduction (P=0.6). Of the rectal cancer patients undergoing major resection, no increase occurred in the proportion of patients undergoing procedures with no primary anastomosis (Hartmann’s procedure or abdominoperineal excision of rectum) with the introduction of surgeon specific outcome reporting (3410/10 206 (33.4%) before and 2735/7871 (34.7%) after; P=0.06). The proportion of patients with rectal cancer undergoing anterior resection with the addition of a defunctioning stoma was 4748/6430 (73.8%) before introduction compared with 3466/4788 (72.4%) after introduction (P=0.09).

Gaming of data

We found no evidence of gaming of data (fig 3). The proportion of major resections categorised as urgent or emergency—and therefore ineligible for surgeon specific outcome reporting—did not change, as 6154 (15.5%) of 39 792 major resections were categorised as urgent or emergency before and 4801 (15.6%) of 30 706 major resections after the introduction of public reporting (P=0.5). The distribution of ASA grades did not change before and after introduction (P=0.08) (table 1).

Fig 3.

Proportion of major resections classified as urgent or emergency (therefore ineligible for surgeon specific outcome reporting) according to date of diagnosis. Dotted line represents date of introduction of surgeon specific public outcome reporting

Mortality in patients eligible for inclusion in surgeon specific outcome reporting

The 90 day mortality in patients undergoing an elective or scheduled major resection fell during the study period from 952/33 638 (2.8%) before the introduction of surgeon specific outcome reporting to 552/25 905 (2.1%) after (fig 4). Therefore, we carried out change point analysis which showed a steeper decline in 90 day mortality after the introduction of public reporting (P=0.03). The change point analysis also found a significant effect of public reporting when it was modelled as an immediate shift in 90 day mortality (P=0.01) and when it was modelled as both an immediate shift and a change in slope (P=0.04).

Fig 4.

Observed and adjusted mortality in patients undergoing elective or scheduled major resection according to date of surgery. Dotted line represents date of introduction of surgeon specific public outcome reporting

Mortality in patients not eligible for inclusion in surgeon specific outcome reporting

We saw no change in the slope of 90 day mortality in patients undergoing urgent or emergency surgery with the introduction of surgeon specific outcome reporting (941/6154 (15.3%) before and 749/4801 (15.6%) after; P=0.3) (fig 5). The six month mortality of patients not undergoing major resection remained static over the study period; 7514/23 062 (32.6%) had died six months after diagnosis before the introduction of surgeon specific outcome reporting and 5728/17 871 (32.1%) after, as shown in figure 6.

Fig 5.

Observed and adjusted mortality in patients undergoing urgent or emergency major resection according to date of surgery. Dotted line represents date of introduction of surgeon specific public outcome reporting

Fig 6.

Observed six month mortality (from diagnosis) in patients not undergoing major resection. Dotted line represents date of introduction of surgeon specific public outcome reporting

Discussion

This study, using data from a high profile, national service evaluation project, found no evidence that the introduction of surgeon specific public reporting of 90 day mortality after elective colorectal surgery has led to risk averse behaviour or gaming of data in the English NHS. The proportion of patients undergoing major resection, the proportion of these patients potentially eligible for outcome reporting, and the predicted risk in those undergoing major resection remained the same. However, the introduction of the public reporting of outcomes coincided with a reduction in the 90 day mortality, exceeding the existing downward trend, in patients who were included in the outcome reporting programme—namely, those patients who had an elective or scheduled major resection. No effect on mortality was seen in patients who were not included—colorectal cancer patients who had an urgent or emergency major resection or those who did not have a major resection—showing that outcomes in these groups were not negatively affected.

Strengths and weaknesses of study

This is the first study to evaluate the effect of the public reporting of surgeons’ outcomes in gastrointestinal surgery. Previous studies assessing the effect of public reporting of surgeon specific outcomes were limited to cardiac surgery.^{16 17 18 19} It is also the first study to compare the effect of surgeon specific outcome reporting on all patients with a specific diagnosis, rather than only those undergoing a specific procedure, so that the effect of risk aversive behaviour and selection of patients could also be studied.

We identified more than 111 000 patients who had colorectal cancer in a national database, almost 60 000 of whom had undergone an elective major resection. This group represents 92% of all colorectal cancer patients admitted to an English NHS hospital.³ This national coverage and the high “case ascertainment” reduces the risk of selection bias.

Clinicians had little opportunity to manipulate the outcome data. We obtained mortality data through record linkage with death records provided by the Office for National Statistics. Other clinical data were entered by audit staff from patients’ records. Clinical teams have a role in this process, but surgeons themselves do not have individual access rights to enter or edit the data themselves. Nevertheless, it is conceivable that a patient’s operative urgency could retrospectively be changed from “scheduled” to “urgent,” which would make the patient ineligible for outcome reporting, or that a patient’s ASA grade could be increased, which would decrease the 90 mortality adjusted for patients’ risk factors. However, we were able to examine whether the data entered for patients diagnosed as having colorectal cancer up until June 2013 had changed after the introduction of outcome reporting, and we found no difference in the distribution of reported ASA grades or the number of patients undergoing elective or scheduled major resection.

In studies using a “before-after” design, changes may occur over time in the quality of the data and in clinical factors that may have an effect on the study results. We tried to overcome this limitation by rigorously checking differences in the accuracy and completeness of the data as part of our assessment of possible gaming of data before and after the introduction of public reporting. Also, we aimed to overcome differences in patients’ characteristics through adjustment for possible differences in the risk profile of patients who had surgery before and after the introduction of public reporting, on the basis of a previously published validated prognostic model.¹³ Lastly, we used change point modelling to take into account the fact that outcomes of colorectal cancer surgery are gradually improving over time. We found that the introduction of public reporting had a significant effect on surgical mortality, irrespective of whether this effect was modelled as a change in the slope (representing a faster decrease in mortality after the introduction of public reporting), an immediate shift in mortality, or a combination of these effects.

Risk averse behaviour

The use of risk adjustment methods that adequately ensure that providers who treat patients at higher risk are not unfairly penalised is critical in public reporting of surgical outcomes for several reasons. Firstly, if surgeons do not fully accept the accuracy of these methods, they may be more inclined to avoid carrying out major surgical procedures in patients at higher risk.²⁰ A recent survey conducted across 293 English colorectal surgeons asking about their views on surgeon specific outcome reporting found that almost all had concerns about the possibility that it could lead to colleagues avoiding operating on patients at high risk.²¹ These concerns are not supported by the results of our study. For example, we found no change in the expected mortality risk before and after the introduction of public reporting of mortality outcomes.

Evidence of risk averse behaviour after the introduction of surgeon specific outcome reporting is mixed in cardiac surgery. Patients undergoing cardiac bypass procedures in states in America with public reporting of outcomes were found to have lower illness severity than patients in states that did not publicly release such data.²² In contrast, in the UK, the average predicted mortality of patients undergoing cardiac surgery seemed to increase rather than decrease in the years immediately after the introduction of surgeon specific outcome reporting.¹⁷

Although the overall risk of patients undergoing elective colorectal cancer resection was unchanged across the study period, the proportion of patients undergoing primary resection with metastatic disease was slightly reduced. Recent population based studies carried out in the US and in Europe found a similar decrease in the percentage of metastatic colorectal cancer patients undergoing primary tumour resection, which suggests that this decrease reflects an overall change in practice rather than risk averse behaviour.^{23 24}

Gaming of data

Manipulation of data has been reported in studies investigating the effect of public reporting of individual surgeons’ outcomes. A study carried out in the 1990s evaluating the effect of surgeon specific outcome reporting in New York State found that about 40% of the reduction in risk adjusted mortality from coronary artery bypass grafting could be explained by the gaming of data, either as a result of an increase in risk factor coding or a change in definitions.²⁵

To reduce the possible effect of deliberately increases in the ASA grade recorded for a patient, the Association of Coloproctology of Great Britain and Ireland has recommended that the ASA grade of each patient should be determined by the anaesthetist rather than by the surgeon and that this grade should be recorded before the resection of the cancer.²⁶ This may in part explain why we did not observe any change in the proportion of patients with higher ASA grades among patients undergoing an elective major resection after the introduction of public surgeon specific reporting.

Mortality

The introduction of surgeon specific outcome reporting coincided with a decrease in mortality after elective major colorectal cancer surgery over and above the pre-existing downward trend in mortality. This study did not show a similar reduction in mortality in patients undergoing emergency colorectal cancer resection after the introduction of surgeon specific public outcome reporting. This suggests that changes to preoperative preparation and planning of perioperative and postoperative care—for example, through the implementation of programmes aiming to enhance recovery after surgery²⁷—may have contributed to the improved mortality rate we have shown. Preoperative cardiopulmonary exercise testing, which can only be implemented in patients who undergo an elective or scheduled procedure, may have led to changes in the delivery of postoperative care, such as the increased use of level 2 and level 3 beds in patients at high risk.^{28 29} Rather than resulting in risk aversion, public reporting of outcomes may have focused attention on an individual patient’s risk and ways of minimising that risk.

Technical aspects of the surgical procedure are unlikely to have improved dramatically over the study period, but it is certainly conceivable that the focus on individual consultant surgeons’ practice may have indirectly led to greater involvement of surgeons in the patients’ actual preoperative and postoperative care, with more proactive management of complications. Surgical trainees have also highlighted that public reporting of outcomes has reduced training opportunities in patients eligible for public reporting.⁷

Effect of public reporting of surgeon specific outcome data

It has been argued that public reporting of outcomes for hospitals or clinicians can have many uses.³⁰ Firstly, it may facilitate patient choice in selecting a specific hospital or clinician that seems to have better outcomes. Evidence from several countries such as the UK, the US, and the Netherlands shows that younger and more affluent patients are more likely to have their treatment in other hospitals than the one closest to where they live.³¹ Secondly, outcome data may influence decisions of purchasers or commissioners of health services either with the aim of controlling costs or to obtain the best outcomes within a limited budget. Thirdly, regulators of healthcare services, such as the Care Quality Commission in England, can consider reported outcomes when they monitor the quality and safety of care delivered by individual providers. Finally, hospitals and clinicians can use the outcomes to assess the performance of their organisation or their individual staff members when implementing quality improvement initiatives.

The outcomes of surgeons performing fewer than 10 elective or scheduled major resection cases a year are not reported in the public domain, but the outcomes for these patients are included in this analysis. Although it is feasible that due to surgeon specific outcome reporting the number of surgeons performing the occasional resection has decreased, therefore resulting in improved outcomes for patients, we have not seen this reflected in NBOCA data—the proportion of elective or scheduled cases by surgeons performing fewer than 10 major resection a year has remained constant (5.0% pre-outcome reporting, 5.2% post-outcome reporting).

We found a drop in mortality of patients who had elective colorectal surgery immediately after the introduction of surgeon specific public outcome reporting. This immediacy of the effect suggests a direct response by the hospitals and clinicians themselves rather than a more indirect response prompted by choices that patients make or by actions from purchasers or regulators. Similarly, although other major improvements in care of patients with colorectal cancer have been made during the study period, such as advances in chemotherapy, these changes are more likely to affect overall survival rather than specifically 90 day postoperative mortality in patients having elective and scheduled surgery.

However, surgical outcomes are influenced by many organisational factors, and an individual surgeon’s results may therefore merely be a “surrogate reflection” of the hospital’s overall performance.³² Also, the number of bowel cancer procedures that a bowel cancer surgeon does each year is low, as is the power to detect a surgeon who is performing poorly. Previously, we have argued that a major risk of public reporting of individual surgeons’ outcomes is that it could lead to false complacency.³³

What general lessons can be learnt from these results? If we assume that the decrease in mortality can be causally linked to the reporting or individual surgeons’ outcomes, the process of reporting itself rather than a response to particular outcomes may have triggered a change in how patients who are scheduled to have major colorectal cancer surgery are managed, not only by the surgeon but by the entire team responsible for a patient’s care, which could be considered as a manifestation of the Hawthorne effect.³⁴ This team response could have been mediated through an individual surgeon’s “heightened responsibility,” which in turn may have galvanised the entire team involved in managing patients before, during, and after elective bowel cancer surgery.

This interpretation provides a more nuanced position in the debate about whether outcome results should be published for clinicians or for teams or hospitals. A further consideration is that the specific circumstances of elective surgery may also play a role, given the emerging evidence for the effectiveness of enhanced recovery programmes and cardiopulmonary exercise testing, which provide possible routes to improve outcomes of patients undergoing scheduled procedures.³⁵ In this context, an increase in the use of radiotherapy before surgery could also play a role for patients with rectal cancer. Unfortunately, we do not have evidence on increases in the use of radiotherapy over time.³⁶

Conclusions

In an era of ongoing “assessment and accountability,”³⁷ consultant colorectal cancer surgeons who practise in the English NHS have an obligation to engage with individual public outcome reporting. The potential positive and negative consequences of public reporting are widely speculated on but are largely unexplored.³⁰ This national study provides unique evidence that the introduction of public reporting of outcomes for individual colorectal cancer surgeons has not led to a decrease in the number of patients at high risk undergoing a major resection and has coincided with an improvement in 90 day mortality for eligible patients.

Public reporting of outcomes for individual clinicians seems to have triggered an improvement in outcomes after elective procedures that can be achieved only through the involvement of the entire clinical team. We did not find similar improvements in the outcome of emergency procedures. This points to improved preoperative preparation and planning of perioperative and postoperative care, only possible for elective procedures, as a potential possible explanation for the reduction in surgical mortality.

What is already known on this topic

• Public reporting of outcomes at hospital level leads to increased quality improvement activity

• The introduction of public reporting of individual surgeons’ outcomes has been controversial but has been studied only in cardiac surgery and almost exclusively in the US

• Public reporting of outcomes is often criticised for encouraging risk averse behaviour and manipulation of data

What this study adds

• No evidence exists of risk averse clinical practice or “gaming” of data after the introduction of surgeon specific outcome reporting in elective colorectal cancer surgery in England

• The introduction of surgeon specific outcome reporting coincided with a significant decrease in mortality, which exceeded the existing downward trend

• Outcomes in patients having emergency surgery for bowel cancer did not change, suggesting improvement in preoperative preparation and planning of perioperative and postoperative care