Assessment of whether in-hospital mortality for lobectomy is a useful standard for the quality of lung cancer surgery: retrospective study

Abstract

Objectives To calculate in-hospital mortality after lobectomy for primary lung cancer in the United Kingdom; to explore the validity of using such data to assess the quality of UK thoracic surgeons; and to investigate the relation between in-hospital mortality and the number of procedures performed by surgeons.

Design Retrospective study.

Setting 36 departments dealing with thoracic surgery in UK hospitals.

Participants 4028 patients who had undergone lobectomy for primary lung cancer by one of 102 surgeons.

Main outcome measures In-hospital mortality in relation to individual surgeons, among all patients, and among each of five groups of patients defined by the number of operations performed by the surgeon.

Results 103 patients (2.6%, 95% confidence interval 2.1% to 3.1%) died after surgery during the same hospital admission. No significant difference was found for in-hospital mortality between the five groups.

Conclusions The number of procedures performed by a thoracic surgeon is not related to in-hospital mortality. Reporting data on in-hospital mortality after lobectomy for primary lung cancer is a poor tool for measuring a surgeon's performance.

Introduction

Since 1998 British cardiothoracic surgeons have had to report their mortality figures for selected “marker” operations. This was in response to publicity surrounding the high mortality from congenital heart surgery at the Bristol Royal Infirmary. Lobectomy for lung cancer was chosen as the marker operation for thoracic surgery because it is the only major thoracic operation performed often enough for meaningful statistical comparisons to be feasible. It is the only general thoracic operation for which data are specific to a surgeon, and data are now available covering two years.

Questions arise as to how such data should be presented and whether the reporting of such data provides a useful tool for assessing the quality of thoracic surgery.¹ We aimed to calculate the mortality from this procedure and in relation to individual surgeons. No other details were available on which to base a risk adjustment. We knew nothing about the patients, such as age or sex, only whether they had left hospital alive or died.

Studies of other surgical procedures have shown a link between how frequently a surgeon performs a procedure and survival of the patient to leave hospital and at five years.²,³ The possibility of such a volume effect is of importance in UK thoracic surgery as around half of lobectomies are performed by cardiothoracic surgeons, who perform lobectomy infrequently compared with surgeons who concentrate on lung surgery (table 1). We explored the relation between in-hospital mortality of patients undergoing lobectomy for primary lung cancer and the number of lobectomies performed by the surgeon. From the pooled data it was not possible to identify any individual surgeon, and this was not our purpose. We wanted to highlight some of the pitfalls of interpreting in-hospital mortality in relation to individual surgeons.

Table 1.

Number of lobectomies performed, by specialty of operating surgeon, 1999-2001

Surgeon's specialty	No (%) of surgeons (n=102)	No (%) of lobectomies (n=4028)	Median annual No of lobectomies per surgeon
Cardiothoracic surgery	68 (67)	2028 (50)	13 (6-21)*
Thoracic surgery	34 (33)	2000 (50)	25 (17-35)*

^*25th to 75th centile range.

Methods

Data were collected by a designated member of each thoracic surgical unit (surgeon or data manager) as part of the UK Society of Cardiothoracic Surgeons' annual return and collated centrally.⁴ These returns are not audited externally, but the accuracy of the computer entry is verified by each unit. Operations are categorised under 237 headings and pooled for the unit. Postoperative deaths during an admission for a procedure are recorded, but not the cause of death.

We present data on 4028 patients who had undergone lobectomy for primary lung cancer by one of 102 surgeons reporting in both of the financial years 1999-2000 and 2000-1. Firstly, we calculated the overall mortality from lobectomy with 95% confidence intervals, then we produced a scatter plot of mortality in relation to the number of procedures performed by the surgeon over the two year period. We also plotted absolute mortality over the two years in relation to the number of procedures performed by the surgeon during that time. As a guide, we added three lines to the plot: one corresponding to the overall average in-hospital mortality and two between which 95% of points would be expected to lie if the risk of in-hospital mortality were the same for all patients. We divided the data into fifths according to the number of operations performed by the surgeon over the two years. For each fifth, we calculated the mortality and exact 95% confidence intervals and used a χ² test to determine any significant differences in in-hospital mortality between the five groups.²

Results

Of the 4028 patients who had undergone a lobectomy over the two year period, 103 died (mortality 2.6%, exact 95% confidence interval 2.1% to 3.1%) during the same hospital admission. Figures 1 and 2 present the data for each of the 102 surgeons. Figure 1 gives an initial impression of a relation between surgical volume and risk. Most of the data in figure 2² lie within the 95% limits that would be expected if the risk of in-hospital mortality were the same for all patients, regardless of surgeon or other factors. Table 2 presents the mortality among the five groups of patients defined by the number of procedures performed by the surgeon over the two years.

Fig 1.

In-hospital mortality in relation to number of operations performed by surgeon during 1999-2001

Fig 2.

Number of patient deaths in relation to number of operations performed by surgeon during 1999-2001

Table 2.

Data on in-hospital mortality for five groups of patients defined by number of operations performed by surgeon

Group	No of cases	No of deaths	No of surgeons	No of cases a year	Mortality (95% CI)
First	806	22	49	1-15	2.7 (1.7 to 4.1)
Second	811	21	21	16-23	2.6 (1.6 to 3.9)
Third	825	24	15	24-32	2.9 (1.9 to 4.3)
		21*			2.5 (1.6 to 3.9)
Fourth	797	16	11	32-40	2.0 (1.2 to 3.2)
		19*			2.4 (1.4 to 3.7)
Fifth	789	20	6	47-96	2.5 (1.6 to 3.9)
Total	4028	103	102		2.6 (2.1 to 3.1)

^*Effect of switching two surgeons that both had 32 cases a year.

To give a roughly equal number of patients in each group, we allocated the patients of two surgeons that had each performed 64 operations to different groups. Three deaths were related to one of the surgeons and none to the other. At the first analysis the split was made according to the sequence in which the surgeons' cases were listed. We found no significant differences in in-hospital mortality between the five groups (χ²=1.477, df=4, P=0.83).

Discussion

In-hospital mortality after lobectomy for primary lung cancer is not related to the number of procedures performed by an individual surgeon. The relation between a surgeon's volume and outcome has attracted interest, and it seems reasonable to believe that there could be an effect. Data have been published on 2.5 million operations over five years in the Medicare system.² These data, which were analysed with the model we used, show powerful effects for pancreatic and oesophageal cancer, which also apply to pneumonectomy and lobectomy for lung cancer. Five year survival is also influenced by the number of lung resections for cancer performed by a surgeon.³ This may be a learning curve or practice effect.⁵ It may also reflect better decisions made by teams dealing with lung cancer rather than those whose major focus is cardiac.

The UK data, however, do not show a relation between volume and outcome. Patients in the first group, who were operated on by 49 surgeons with an annual volume of one to 15 lobectomies, compared well with patients in the fifth group who were operated on by just six surgeons doing 47 to 96 lobectomies a year. Across the five groups the mortality varied from 2.0% to 2.9%, with no evidence of relation to volume.

Most of the 102 surgeons we reviewed performed both heart and lung surgery.⁶,⁷ Lobectomy for lung cancer was thus a small proportion of their work. They typically performed around 150 to 200 cardiac operations a year, so any practice effect for lobectomy (an operation of intermediate technical difficulty in cardiothoracic surgery) is likely to be subsumed under the overall case mix. It is also likely that any volume effect according to setting is lost because the lower volume of lobectomies are most commonly performed within cardiothoracic units, which deal with a large volume of major chest surgery.² In the United Kingdom we need the contribution of “low volume” surgeons (surgeons performing less than 24 lobectomies a year provide 40% of the UK service for this procedure), so our results are reassuring.⁷

Figure 1 gives the impression of a relation between surgical volume and outcome. This can be explained by the eye being drawn to the single high mortality figure of 20%, representing one death out of five cases, and not to the 35 of 59 points along the x axis which represent surgeons performing fewer than 20 lobectomies a year. It is more than likely that at least one surgeon with a low surgical volume would display high mortality purely by chance—for example, if all the patients of all the surgeons faced the same risk of in-hospital death, there would be a 65% chance of at least one of the surgeons who performed five or fewer operations over the two years having a mortality of 20% or more. Similarly deceptive is the way in which the points line up in asymptotic curves. All points representing a single death describe a curve running from 20% to about 2%, as the denominator increases from five to over 50. The same effect is seen as the points for surgeons with two and three deaths line up. The major characteristics of figure 1 are therefore artefacts rather than useful information about the performance of individual surgeons. Figure 2 provides a graphical representation of the data that is less open to misinterpretation. Importantly, the low level of absolute mortality is more obvious than in figure 1.

What is already known on this topic

British cardiothoracic surgeons must report their mortality figures for “marker” operations

Lobectomy for primary lung cancer is the marker operation for thoracic surgery; half are performed by surgeons who perform the procedure infrequently

Studies of other surgical procedures have shown a link between surgical volume and survival to leave hospital

What this study adds

In-hospital mortality after lobectomy is not related to the number of procedures performed by a surgeon

Mortality figures derived from a small number of cases are unreliable and should not be used as the basis of important decisions made by patients

Mortality from lobectomy for primary lung cancer is a poor means of measuring surgical performance

So what purpose does the collection of surgeon specific data serve? Annual surgical volumes are too small to form the basis of any statistically valid comparison of one surgeon with another. It is also likely that anaesthesia and postoperative care are as important as surgical competence in determining survival after lobectomy. The more important determinants of survival are patient related and include age, pulmonary function, cardiac status, and other comorbidity. And yet the data collected on lobectomy are not adjusted for risk. The development of risk scoring within thoracic surgery is clearly needed.

Previous pooling of data by the Society of Cardiothoracic Surgeons was based on anonymity. The current system is open to dishonesty and the likelihood of surgeons refusing to operate on high risk cases. For these reasons we believe that the registration of lobectomy data is an inadequate means of measuring surgical competence. Lung cancer is a rapidly fatal disease, and an informed patient might well choose to face a substantial surgical risk for the chance of cure.⁸ A better indicator of quality would be cancer free survival at five years,³ and this would reflect better the performance of the cancer multidisciplinary team, who are jointly responsible for patient selection.