Reporting individual surgeon outcomes does not lead to risk aversion in abdominal aortic aneurysm surgery

Abstract

INTRODUCTION

Reporting surgeons’ outcomes has recently been introduced in the UK. This has the potential to result in surgeons becoming risk averse. The aim of this study was to investigate whether reporting outcomes for abdominal aortic aneurysm (AAA) surgery impacts on the number and risk profile (level of fitness) of patients offered elective treatment.

METHODS

Publically available National Vascular Registry data were used to compare the number of AAAs treated in those centres across the UK that reported outcomes for the periods 2008–2012, 2009–2013 and 2010–2014. Furthermore, the number and characteristics of patients referred for consideration of elective AAA repair at a single tertiary unit were analysed yearly between 2010 and 2014. Clinic, casualty and theatre event codes were searched to obtain all AAAs treated. The results of cardiopulmonary exercise testing (CPET) were assessed.

RESULTS

For the 85 centres that reported outcomes in all three five-year periods, the median number of AAAs treated per unit increased between the periods 2008–2012 and 2010–2014 from 192 to 214 per year (p=0.006). In the single centre cohort study, the proportion of patients offered elective AAA repair increased from 74% in 2009–2010 to 81% in 2013–2014, with a maximum of 84% in 2012–2013. The age, aneurysm size and CPET results (anaerobic threshold levels) for those eventually offered elective treatment did not differ significantly between 2010 and 2014.

CONCLUSIONS

The results do not support the assumption that reporting individual surgeon outcomes is associated with a risk averse strategy regarding patient selection in aneurysm surgery at present.

Abdominal aortic aneurysm (AAA) is an important health problem and a common cardiovascular cause of death. Rupture carries a mortality risk of 74–90%,^1,2 and approximately 5,000 individuals die annually in England and Wales from causes relating directly to an AAA.³ Even in the elective setting, AAA repair has been associated with significant rates of mortality in the UK (7% in 2008), which were higher than the European average.⁴

As a result, major changes have taken place during the last decade to the delivery of vascular services in the UK, including aneurysm surgery. The National Health Service AAA screening programme was introduced and fully rolled out nationwide in 2013. Vascular surgery became an independent specialty in 2012, ensuring trainees would complete a separate training pathway. The provision of vascular services was also centralised, based on the fact that smaller units with low volume caseloads had the highest mortality rates (some greater than 20%)⁵ while high volume centres had favourable outcomes (in both carotid and aneurysm surgery).^6,7 Endovascular aneurysm repair (EVAR) enjoyed a rapid uptake by vascular surgeons around the country, having been associated with favourable outcomes over the short and medium term.⁸ This has led to a reduction in mortality relating to elective AAA repair from around 7% to below 2%.⁴

While these interventions have been successful in the reduction of mortality, a further measure has been introduced: the compulsory reporting of surgeon specific mortality data (SSMD) for elective AAA and carotid endarterectomy repair.⁹ Although this improves transparency and accountability of surgeons, there is presently little evidence to suggest SSMD reporting will reduce mortality or morbidity following AAA surgery.¹⁰ There is a danger of SSMD leading to risk averse behaviour among vascular surgeons; this has already been reported for cardiac surgery SSMD.¹¹

Risk aversion due to SSMD can have disastrous consequences for patients. Those who would have benefited from surgical repair but who have co-morbidities that predispose them to complications may not be offered intervention.¹² These patients tend to have poorer outcomes as a result of being turned down and with an estimated two-year survival rate of 35% following rejection from intervention,¹³ this is no small consideration. It is therefore imperative to investigate the impact of SSMD reporting on elective AAA surgery in the UK.

Methods

In order to assess any effect of SSMD reporting on patient selection and outcomes after elective AAA repair, a two-step approach was employed, using data from the National Vascular Registry (NVR) relating to the number of procedures undertaken by each consultant and trust as well as data from our tertiary referral centre, in which all patients eligible for AAA repair undergo cardiopulmonary exercise testing (CPET).

National Vascular Registry data

Since 2013, surgeon specific mortality data (SSMD) on aneurysm surgery have been made publically available through the NVR.⁹ Each unit reports the number of procedures performed and related patient characteristics through a streamlined pathway. The data are published annually for the preceding five years. Datasets are therefore available for the periods 2008–2012, 2009–2013 and 2010–2014.

Given the ongoing reorganisation of vascular services in the country between 2008 and 2015, some smaller centres that reported data for 2008–2010 ceased to exist after 2010–2011. However, 85 units reported data for all three five-year periods and these were the only centres that were included in our analysis. From the available surgeon specific data, it was possible to pool the data for each centre in terms of overall number of elective AAA repairs, number of EVARs, number of open aneurysm repairs [OARs] and inpatient mortality. The average numbers of repairs performed per period were compared so as to assess whether the introduction of publically available SSMD on a mandatory basis from 2013 onwards had an impact on the number of AAA repairs performed (ie whether there was an increase in ‘turn-down’ rates).

Single centre cohort study

The publically available NVR data do not include information relating to patient fitness. In order to fully assess the impact of SSMD reporting on patient selection and possible risk averse surgeon behaviour, it is important to know the risk profiles (levels of fitness) of patients offered repair. To this end, all patients were reviewed who were referred for consideration of elective AAA repair between April 2009 and March 2014 at our centre, where CPET is offered to all patients as part of the decision making process. This strategy allowed quantification of the patients’ risk profile at baseline using a validated method.

The type of procedure (EVAR, fenestrated EVAR, OAR, no intervention) was recorded for each patient using electronic records and operation notes (where available), and this was cross-matched to the trust’s coding system. In addition to CPET results, details of patient demographics, baseline biochemistry and cardiovascular risk factors were obtained using clinic letters, vascular multidisciplinary meeting lists and discharge summaries. Finally, in order to ensure all patients referred for consideration of AAA repair were included, casualty and theatre event codes were searched retrospectively using the trust’s coding system.

The overall number of patients referred for consideration of AAA treatment during the study period was obtained using various strategies. These comprised review of the list of patients discussed at the vascular multidisciplinary meetings, review of patients with a diagnosis of AAA (identified via a hospital-wide electronic code search) and review of patients with a code for AAA who were seen in a vascular outpatient clinic during the study period.

Cardiopulmonary exercise testing

CPET is a validated reproducible assessment of cardiopulmonary reserve that is now widely used in aneurysm surgery.¹⁴ It provides a detailed evaluation of a patient’s functional status before surgery as it essentially simulates the requirements of major surgery.¹⁵

Anaerobic threshold (AT) assessed through CPET is one of the commonly used risk stratification markers prior to surgery. It is a marker of the combined efficiency of the lungs, heart and circulation. With increasing exercise, oxygen demand will begin to exceed supply. As a result, muscle cells will generate adenosine triphosphate through anaerobic metabolism, which produces lactic acid. The latter will be buffered by circulating bicarbonate, resulting in an increased production of CO₂. AT is defined as the volume of O₂ at the point at which this occurs. An AT of 11 ml/kg/min has traditionally been used as a cut-off for selecting candidates for major surgery.¹⁵ In AAA repair, a low AT has been associated with postoperative complications and 30-day mortality.^14,16 For this reason, AT was used as a marker of cardiopulmonary reserve in our population.

Statistical analysis

Statistical analysis was performed with SPSS^® version 21.0 (IBM, New York, US). Normality of distribution was assessed using skewness and kurtosis as well as the Kolmogorov–Smirnov test. Analysis of variance or a chi-squared test were employed for continuous and categorical data respectively. The Wilcoxon signed-rank test was used to compare non-parametric variables. A p-value of <0.05 was considered statistically significant.

Results

National Vascular Registry data

Table 1 shows the overall numbers of elective AAA repairs in the three separate reporting periods for all centres included in the NVR. For the 85 units that contributed to all three reporting periods, the median number of AAA repairs per unit increased from 192 in the first period to 214 in the third period (p=0.006) (Table 2). In terms of EVARs, the increase was statistically significant (from 111 to 139, p<0.001), but for OARs the increase (from 66 to 69) did not reach statistical significance (p=0.34).

Table 1.

The numbers of abdominal aortic aneurysm repairs in the three separate reporting periods for all centres included in the UK National Vascular Registry

Reporting period	All repairs	EVAR	OAR
2008–2012	19,452	11,982	7,470
2009–2013	20,355	12,923	7,432
2010–2014	20,120	13,203	6,917

EVAR = endovascular aneurysm repair; OAR = open aneurysm repair

Table 2.

Average number of aneurysm repairs performed in the 85 units reporting data for all of the three separate five-year periods

Reporting period	Number (range) per unit over reporting period				Number per unit per year
	All repairs*	EVAR*	OAR*	Mortality**	All repairs**	EVAR**	OAR**
2008–2012	192 (35–626)	111 (0–473)	66 (1–263)	2.2%	42.8	26.3	16.4
2009–2013	208 (6–623)	134 (0–465)	68 (4–277)	2.3%	44.7	28.4	16.3
2010–2014	214 (6–617)	139 (0–545)	69 (2–267)	1.8%	46.8	30.7	16.1

EVAR = endovascular aneurysm repair; OAR = open aneurysm repair

*median; **mean

Single centre cohort study

A total of 288 elective AAA repairs took place at our unit between April 2009 and March 2014, including both EVARs and OARs. Table 3 summarises the patient demographics and cardiovascular profiles for each year during the five-year study period. Despite the introduction of SSMD reporting, the overall number of patients offered treatment increased during this time (from 41 to 70 per year). Furthermore, the proportion of patients offered treatment did not change significantly (range: 74–84%, p=1.0). Among those who were offered surgical repair, there were no differences in terms of age, sex or history of prior ischaemic heart disease throughout the five-year period. There were also no differences with regard to mean AT levels (p=0.48). Finally, the number of EVARs performed increased steadily over the study period from 19 to 56.

Table 3.

Characteristics of patients offered repair in the single centre cohort study

	2009–2010	2010–2011	2011–2012	2012–2013	2013–2014	p-value
Mean age in years	72 (SD: 5)	72 (SD: 6)	73 (SD: 8)	73 (SD: 5)	75 (SD: 4)	0.46
Female	4%	2%	2%	3%	6%	0.58
Previous MI	7%	9%	6%	8%	11%	0.67
Hypertension	77%	79%	82%	81%	76%	0.78
Smoking	74%	77%	78%	82%	84%	0.87
Mean AAA size in cm	6.6 (SD: 0.9)	6.4 (SD: 0.7)	6.5 (SD: 0.6)	7.1 (SD: 1.2)	6.7 (SD: 1.1)	0.21
Mean AT level in ml/kg/min	14.1 (SD: 4.5)	14.5 (SD: 5.2)	14.7 (SD: 4.6)	14.2 (SD: 5.3)	13.7 (SD: 6.1)	0.48
Any AAA repair	41	53	54	70	70	–
EVAR	19	29	34	55	56	–
OAR	22	24	20	15	14	–
Referrals offered repair	74%	79%	82%	84%	81%	1.0

AAA = abdominal aortic aneurysm; AT = anaerobic threshold; EVAR = endovascular aneurysm repair; MI = myocardial infarction; OAR = open aneurysm repair; SD = standard deviation

Discussion

This study has attempted to assess whether reporting individual surgeon outcomes is associated with a risk averse strategy regarding patient selection in aneurysm surgery. Based on nationally collected data, there has been no reduction in the number of AAA repairs across the 85 units that have been reporting outcomes consistently since 2008 (prior to mandatory SSMD reporting). The risk profiles of patients offered elective AAA repair between 2009 and 2014 at our centre (where all patients are assessed for fitness using CPET) were also not different. These findings do not support the hypothesis that vascular surgeons will develop risk averse behaviour owing to SSMD reporting.

Vascular services in the UK have undergone major reconfiguration in recent years following reports that centralisation of services is associated with better outcomes in complex surgery, such as AAA repair.^5,7,17 Among the several changes introduced during the last decade, reporting of SSMD has become mandatory for all cases of elective AAA repair and carotid endarterectomy in England.⁹ This has been controversial and the impact of this policy remains largely unknown. The supporters of public reporting of SSMD advocate transparency and quality improvement as the major benefits of this strategy.¹⁸ On the other hand, potential pitfalls include risk averse surgeon behaviour and an increase in conservative management of patients as well as a reduction of training opportunities in the operating theatre for surgical trainees.¹¹

Local and wide scale implementation of well planned quality improvement initiatives and audit processes has led to better patient care and even reduced healthcare costs in several instances.^19–21 Nevertheless, there is no clear evidence to support the assumption that public SSMD reporting and attribution of poor outcomes to specific named surgeons can indeed improve patient care. Public disclosure of SSMD was introduced in cardiac surgery in the UK and elsewhere long before it became mandatory for vascular surgeons to report their own outcomes.¹¹ The two specialties share common ground in that they offer complex interventions to high risk patients. The prior experience of cardiac surgery with SSMD reporting may therefore be invaluable for vascular surgeons.

Interestingly, risk averse behaviour and a negative impact on surgical training have been reported in cardiac surgery since adoption of SSMD reporting.^11,22,23 More importantly, however, the major concern is that this strategy can lead to an overall greater population mortality because high risk patients are denied intervention in an attempt to reduce postoperative deaths. The prognosis of patients with an AAA who are denied intervention is poor (despite underreporting in the currently available literature), with a quoted survival rate of 35% over two years;¹³ the chance of rupture can be as high as 32.5% within one year for AAAs exceeding 7 cm in diameter.²⁴

Unfortunately, SSMD do not capture the number of patients refused treatment in each centre or by each individual surgeon and they do not record reasons for not operating on each individual. Consequently, it is impossible to assess the selection process that takes place in each unit and how this may impact on overall population mortality. Our study has shown that there has not been a reduction in the overall number of AAAs treated when comparing the three sets of publically available SSMD. On the other hand, it is not possible to determine the number of referrals that were made in the corresponding periods and so no comment can be made on turn-down rates or population mortality on a nationwide basis.

Furthermore, the NVR data do not provide any detail about the risk profiles of these patients and it is not possible to ascertain the precise number of repairs per centre per year as the data are only published for five-year periods. To this end, our study attempted to assess turn-down rates and patient risk profiles using data from a tertiary unit at which patients undergo CPET prior to subsequent discussion about suitability for AAA repair in the vascular multidisciplinary meeting. As a result, it was possible to capture the overall number of patients considered for treatment and assess their risk profiles using a validated quantitative measure of cardiopulmonary reserve.

There was no difference in the risk profiles of patients treated at our centre over the five years studied and the proportion of patients offered treatment did not change following the introduction of mandatory SSMD reporting. Again, this finding is not indicative of risk averse behaviour in the vascular community.

An interesting observation is the fact that more patients are currently offered EVAR than OAR. One may argue that this is consistent with risk aversion but in fact it is probably a sign of increasing confidence in treating more complex anatomies using endovascular means. Also, new devices and techniques (such as endovascular sealing) have allowed the treatment of shorter and more angulated proximal aneurysm necks, and so the rise in the number of EVARs is to be expected.

Limitations

There are several limitations of note. First, the NVR data are not broken down by year and as the NVR acknowledges, some of the AAA repairs may not have been listed (incomplete data). It is not possible to know how complete/incomplete data capture has been and there are no Hospital Episode Statistics (HES) data available on elective AAA repairs in past years to assess that. The NVR committee may be able to address this issue in the future by obtaining combined HES and NVR data. In addition, the study design does not allow any associations to be made with subsequent mortality and morbidity following the introduction of SSMD publication.

In the single centre cohort study, some inpatient referrals may have been missed when attempting to capture all patients assessed for AAA repair; however, all possible forms of patient identification and data capture were included in order to assess turn-down rates. Finally, as our data collection was carried out retrospectively, the data regarding AAA morphology for this series are not exhaustive. We do not have complete CPET data at hand for the individuals who did not undergo surgery and meaningful comparisons cannot therefore be performed.

Conclusions

These data do not support the assumption that SSMD reporting in aneurysm surgery leads to risk adverse behaviour. Nevertheless, SSMD reporting may have adverse effects on training and population mortality, and it is evident that the type of data reported currently provides no measure of turn-down rates or patient fitness. Further research is therefore necessary to fully assess the impact of SSMD reporting on patients, surgeons and healthcare providers.