Improving mortality of coronary surgery over first four years of independent practice: retrospective examination of prospectively collected data from 15 surgeons

Abstract

Objective To study the “learning curve” associated with independent practice in coronary artery surgery.

Design Retrospective analysis of prospectively collected data.

Setting All NHS centres in north west England that carry out cardiac surgery in adults.

Participants 18 913 patients undergoing coronary artery surgery for the first time between April 1997 and March 2003, 5678 of whom were operated on by 15 surgeons in the first four years after their consultant appointment.

Main outcome measures Observed and predicted mortality (EuroSCORE) for surgeons in their first, second, third, and fourth years after appointment as a consultant compared with figures for established surgeons.

Results Overall mortality decreased over the six years of study (P = 0.01). Of the patients operated on by established surgeons or newly appointed consultants, 265/13 235 (2.0%) and 109/5678 (1.9%), respectively, died (P = 0.71). There was a progressive decrease in observed mortality with time after appointment as a consultant from 2.2% in the first year to 1.2% in the fourth year (P = 0.049). This result remained significant after adjustment for time and case mix (P = 0.019).

Conclusions Mortality in patients operated on by newly appointed consultant surgeons is similar to mortality in patients operated on by established surgeons. There are significant decreases in crude and risk adjusted mortality in the four years after appointment. These findings should influence the nature of practice in newly appointed surgeons.

Introduction

In British medicine consultants are appointed to hospitals after a defined period of training, once they have satisfied national training bodies and been successful in open competition for an advertised post. NHS consultants are independent practitioners who function under clinical governance systems in the institutions in which they are employed. Increasingly, they are also regulated by national initiatives, such as the planned publication of surgeon specific mortality data for coronary artery surgery in the United Kingdom in 2004.¹

In a recent study we looked at the outcomes of coronary artery surgery for individual surgeons in north west England.² An incidental finding was a strong association between the volume of operations each surgeon had performed and mortality. This may be due to a “learning curve” effect, with higher mortality in patients operated on by newly appointed surgeons, starting late in the study period. We studied this in more detail by looking at changes in case mix and mortality after new consultant appointments in cardiac surgery in north west England.

Methods

The north west quality improvement programme in cardiac interventions is a regional consortium involving all four NHS centres (Blackpool Victoria Hospital; Cardiothoracic Centre, Liverpool; Manchester Royal Infirmary; and South Manchester University Hospital) that perform cardiac surgery and percutaneous coronary interventions in adults in north west England. The group aims to improve continuously the quality of care for patients receiving cardiac interventions using a regionally based systems approach.³

We prospectively collected data on 18 913 consecutive adult patients undergoing cardiac surgery between 1 April 1997 and 31 March 2003 in north west England. We included only those patients undergoing isolated coronary artery surgery (excluding repeat surgery). For each patient we collected data on preoperative and operative variables to calculate predicted mortality. Data were collected in each institution and returned to a central source for analysis. Validation of data was conducted in each centre. Mortality was defined as any death in hospital. Every patient record contained an identifier for each consultant surgeon.

Design of the project

We addressed four specific questions:

• Are there differences in observed and predicted mortality between patients operated on by newly appointed surgeons and their more experienced colleagues?

• Are there changes in observed and predicted mortality in patients operated on by newly appointed surgeons over the first four years of appointment?

• If there are improvements in risk adjusted mortality over the first few years of appointment, what is the time course of those changes?

• If there are decreases in mortality, are they seen in low or high risk patients?

The north west regional audit project started in 1997. Since that time there have been 13 new appointments. In addition, two surgeons practising in 1997 were in the first four years of their consultant appointment. For the analysis we pooled surgeons into four groups according to year of independent practice (first, second, third, and fourth years). We included all temporary appointments of less than a year as first year appointments. Associate specialists who had been in independent practice for more than four years by the time of their consultant appointment were not included as new appointments; if they had been in independent practice for less than four years they were included in the appropriate year as a new consultant. We also analysed data excluding locum appointments from the dataset.

Statistical analysis

Categorical data are shown as percentages with 95% confidence intervals, while continuous data are shown as means. We used χ² tests for trend and Wilcoxon rank sum tests as appropriate to test for significant differences between groups. Crude mortality was determined for each patient and a predicted mortality calculated by using additive EuroSCORE.⁴ If a factor necessary to calculate the EuroSCORE was missing in the record, that factor was assumed to be absent (this occurred in less than 2% of cases). While the mean EuroSCORE is often given as an indication of operative risk of a group, the score is not normally distributed and so we have given results as means and medians with 25th and 75th centiles. To investigate in which type of patients the observed changes occurred, we subdivided patients into low (EuroSCORE < 6) and high (EuroSCORE < 6) risk groups for further analysis.² To adjust for a potential time effect we constructed a logistic regression model on all cases (n = 18 913) that included the financial year of the operation (treated as 1997-8 = 1, 1998-9 = 2, etc) and the EuroSCORE.⁵ Using the logistic model, we then calculated risk adjusted mortality percentages for the first four years of independent practice, in line with the system developed in the state of New York.⁶ The results reflect performance adjusted for time and case mix.

Results

A total of 18 913 patients underwent surgery in the region during the study period, 5678 of whom were included in the learning curve analysis. Table 1 shows details of the new consultant appointments and number of patients operated on. The number of patients pooled into the first, second, third, and fourth years were 1967, 1534, 1283, and 894 respectively. The average number of patients per surgeon was 379. The learning curve dataset included 30% of the total number of patients undergoing surgery.

Table 1.

Number of cases dealt with by new consultants according to year of appointment

Consultant	Year of appointment	No of cases
1	1995	290
2	1995	380
3	1997	516
4	1997	735
5	1997	792
6	1998	449
7	1999	58
8	2000	357
9	2000	400
10	2000	408
11	2000	441
12	2001	295
13	2001	362
14	2002	73
15	2002	122

Overall, 374 patients (2.0%) undergoing isolated coronary artery surgery during the study period died. Mortality in patients operated on by surgeons in their first four years after appointment and that of established consultants was similar (109/5678 (1.9%, 95% confidence interval 1.6% to 2.3%) and 265/13 235 (2.0%, 1.8% to 2.2%), respectively, P = 0.71). The mean predicted mortality for patients operated on by consultants in the first four years after appointment (EuroSCORE mean 3.1, median 3, 25th and 75th centiles 1 and 4) was similar to that of those operated on by more established surgeons (mean 3, median 3, 25th and 75th centiles 1 and 4) (P = 0.79).

The ratio of observed to expected mortality is the best risk adjusted summary of performance. The ratios for the patients operated on by surgeons in the first four years of practice and more established surgeons were 0.61 and 0.66, respectively.

There was a progressive decrease in observed mortality during the first four years after appointment, from 2.2% (1.6% to 3.0%) to 1.2% (0.5% to 2.1%) in the first and fourth years, respectively (P = 0.05, table 2). The predicted mortality progressively increased over the four years from a mean EuroSCORE of 3.0 to 3.3 (P = 0.03). The ratio reduced by a factor of 50% from the first to the fourth year after appointment (P < 0.001). Exclusion of locum appointments from the dataset had no effect on the results. After we accounted for the effect of time and case mix by using the logistic model, the adjusted mortality decreased from 2.3% in the first year to 1.0% in the fourth year (P = 0.019, table 2).

Table 2.

Observed mortality, predicted mortality, observed to expected mortality ratios, and adjusted mortality in patients operated on by newly appointed surgeons according to year after appointment

Year	Observed mortality (%) (95% CI)	Mean EuroSCORE (median; 25th, 75th centiles)	O:E ratio*	Adjusted mortality (%)†
1	2.2 (1.6 to 3.0)	3 (3; 1, 4)	0.73	2.3
2	2.1 (1.5 to 3.0)	3.1 (3; 1, 4)	0.68	2.2
3	1.6 (0.9 to 2.3)	3.1 (3; 1, 5)	0.52	1.7
4	1.2 (0.5 to 2.1)	3.3 (3; 1, 5)	0.36	1.0
P value‡	0.049	0.03	<0.001	0.019

^*Ratio between observed and expected (predicted) mortality is indicator of overall quality of care adjusted for risk.

^†Adusted for time and case mix.

^‡Significance for trend over time.

Table 3 shows changes in mortality in the low and high risk groups. There was a significant reduction in mortality in the low risk group with increasing length of time since appointment that was not seen in high risk patients. Table 4 shows that for established consultants there was a significant reduction in mortality over the time of the study (P = 0.01).

Table 3.

Changes in observed mortality in low and high risk patients for newly appointed surgeons by year after appointment. Figures are percentage mortality

	Low risk group (deaths/patients)	High risk group (deaths/patients)
Year 1	1.7 (29/1700)	5.6 (15/267)
Year 2	1.6 (20/1280)	5.1 (13/254)
Year 3	0.8 (9/1099)	6.5 (12/184)
Year 4	0.3 (2/722)	5.2 (9/172)
P value*	0.011	0.93

^*Significance for trend over time.

Table 4.

Overall mortality for patients undergoing surgery by established consultants for each year of study period

Year	No (%) of deaths/total patients
1997-8	55/2431 (2.3)
1998-9	53/1991 (2.7)
1999-2000	40/2155 (1.9)
2000-1	43/1998 (2.2)
2001-2	35/2133 (1.6)
2002-3	39/2527 (1.5)
P value*	0.01

^*Significance for trend over time.

Discussion

Principal findings

We have shown significant improvements in crude and risk adjusted mortality associated with coronary artery surgery over the four years after surgeons are appointed to independent practice. The observed mortality for surgeons in their first and second years after appointment was slightly higher than for established consultants but lower in the third and fourth years. These improvements were due to better outcomes in patients undergoing low risk surgery.

Strengths and weaknesses of the study

Our study has been conducted on a dataset produced from all patients undergoing cardiac surgery in north west England since 1997. The data have been subjected to local validation and have the confidence of clinicians but have not been subjected to external validation, which is a weakness. We consider that the number of surgeons, the large population of patients, and the fact that these appointments were across four different hospitals add strength to our study. We were initially concerned that some of the learning curve effect detected may have been due to overall changes in mortality because of progressive improvements in outcome with time, which is an effect that has been clearly seen in previous work.⁷ The improvements we have shown over the first four years of practice were still significant after we adjusted for time and case mix by logistic regression. We have looked only at mortality and not at morbidity, which may be a more sensitive indicator of subtle improvements in the quality of care.

We observed a significant decrease in crude mortality as surgeons become more experienced. We used a well accepted model of risk prediction, the additive EuroSCORE,⁴ to look for changes in case mix to see if alterations in predicted mortality might explain the observed mortality. The EuroSCORE is generally accepted to be a good but not perfect predictor of mortality.²,⁸,⁹ Predicted mortality increased in the four years after appointment, suggesting that surgeons are operating on patients with increasing comorbidity, which confirms local clinical perceptions. Use of the ratio of observed to expected mortality as an overall indicator of risk adjusted quality of care highlights this effect of decreasing mortality despite increasing complexity of case mix. While there must be some caution exercised when using risk prediction models, it would seem to be appropriate in this setting, but it is conceivable that more experienced surgeons may become “smarter” at exploiting limitations within the risk prediction model, which would not affect our observations on crude mortality but may potentially confound the risk adjusted analysis.

The mortality in patients of surgeons in the first two years of appointment was slightly higher than that in patients of more established consultants performing surgery during the time period studied. The observation that mortality in patients of surgeons in their third and fourth years of appointment is below that seen in patients of more established surgeons (both crude mortality and risk adjusted outcomes), suggests that either the performance of these consultants is better than their peers, or that there are complex issues associated with case mix and higher risk surgery that may account for these differences but were not detected with our methods.

Comparison with other studies

There are few studies on learning curves in surgery. There have been reports of learning curve effects with new techniques or with new complex operations in both cardiac¹⁰,¹¹ and non-cardiac surgery.^12-17 There are thought to be three separate components of the learning curve for new procedures: the specialist community, the institution, and the individual surgeon.¹¹ We examined the effects of the learning curve in surgeons on a well established procedure for which they have been trained. Most trainees will have performed many hundreds of coronary artery bypass operations before appointment in hospitals with existing skills and infrastructure, so we are not looking at aspects of learning curves of the specialist community or institution. We saw significant improvements in risk adjusted mortality in the four years after appointment, which is an “individual surgeon” effect. We are not aware of previous reports on learning curve effects associated with moving from training grade to independent practice.

A recent study on learning curve effects in complex laparoscopic colorectal surgery¹⁴ suggests a steady state of technical competence is achieved after 70-80 procedures. The number of operations needed to achieve the “plateau phase” for technical issues in coronary artery surgery is probably similar. As all new appointees in this study would have performed well over 100 operations, while there may still be ongoing technical improvements after appointment, the improvements we have detected are probably related to issues other than primary technical ability.

Implications

If the improvements seen after appointment are not related to technical ability it implies that surgeons improve the non-technical aspects of surgery, including decisions on case selection and aspects of preoperative, intraoperative, and postoperative care and decision making. Our study suggests that patients and hospitals can be reassured that mortality is not higher in patients of newly appointed surgeons, but initiatives that may modify the learning curve will have advantages for patients, hospitals, and surgeons.

There are several possible ways to minimise the effects of learning curves: improve training, increase supervision of new appointments, and introduce other strategies to accelerate improvement. The effect seen here was detected in surgeons who have been trained under existing structures (the tiered registrar/senior registrar system or seamless Calman training) and will have spent in excess of five years as specialist trainees. Plans to shorten the surgical training period, along with a reduction in training hours due to implementation of the European working time directive, may have implications on the experience of new consultants in the future, which would act to increase possible learning curve effects unless other modifying influences are introduced.

It has been suggested that learning curve effects in medicine are similar to those in manufacturing¹⁷ and that by applying learning curve theory, higher quality health care may be achieved. Learning curve theories accepted in manufacturing are that outcomes improve with higher volume due to learning, increases in improvement become smaller with progressive experience, and increases in improvement follow a predictable pattern. In previous studies of learning curves in cardiac¹⁰,¹¹ and non-cardiac^12-17 surgery, effects have all been detected on a relatively small number of cases. We have observed a learning curve effect in surgeons who are performing high volumes of surgery. While we saw continuous improvements in each of the first four years of appointment, the most marked improvements were between the third and fourth years, which is at odds with data from manufacturing. The annual volumes of coronary artery surgery in units and individuals are high, which differs from the typical small number of cases undertaken in other surgical specialties.

Volume of experience, incentives, and managerial factors are potential modifying influences on the learning curve.¹⁸ It seems inappropriate to increase the volume of surgery to higher than current levels. Planned publication of surgeon specific mortality and use of performance data in clinical excellence awards may well influence learning curves in future but may also encourage newly appointed surgeons to turn down higher risk cases, unless robust risk adjusted measures are used for performance management. Managerial input—including the use of clinical pathways, robust clinical risk management strategies, effective supervision, and targeted continuing professional development—would seem to be the most effective way to accelerate learning curve effects. Experienced clinical support for new appointments, not just for technical issues but for all aspects of clinical practice, including case selection and perioperative management, seems sensible.

Unanswered questions and future research

Our study raises several questions that need to be addressed in future studies. We do not know whether causes of death change as surgeons become more experienced, which may help to guide plans to improve outcomes in newly appointed surgeons. While the effect for cardiac surgery seems to be marked, we do not know whether the learning curve is restricted to this specialty or whether it is seen in other surgical specialties or is a widespread phenomenon throughout medicine. There are several initiatives that are leading to a reduction in hours during the working week and limiting the number of years of training for surgeons, and we do not know if this will make the effects we have seen more marked. Finally, we do not know whether initiatives such as a structured supervision and targeted continuous professional development would accelerate the learning curve effect.

What is already known on this topic

Learning curves are associated with the introduction of new procedures in medicine

Although mortality for coronary artery surgery has decreased over time, it is not known whether mortality among patients of new consultants changes as the doctors become more experienced and whether such changes are independent of overall improvements of their peers

What this study adds

In the first four years after the appointment of UK consultant surgeons to independent practitioner status outcomes significantly improve

While initial mortality associated with newly appointed consultant surgeons is not significantly higher than that for more established consultants, there is a 50% reduction in risk adjusted mortality over the subsequent four years