External validation of the Ferguson pulmonary risk score for predicting major pulmonary complications after oesophagectomy

Abstract

OBJECTIVES

Pulmonary complications remain a frequent cause of morbidity in patients undergoing oesophagectomy. Risk screening tools assist in patient stratification. Ferguson proposed a risk score system to predict major pulmonary complications after oesophagectomy. Our objective was to externally validate this risk score system.

METHODS

We analysed our institutional database for patients undergoing oesophagectomy for cancer from August 2009 to December 2012. We analysed patients who had complete documentation of variables used in the Ferguson risk score calculation: forced expiratory volume in the 1 s, diffusion capacity of the lung for carbon monoxide, performance status and age. One hundred and thirty-six patients qualified for analysis in the validation study. Outcome variables measured included major pulmonary complications, defined as need for reintubation for respiratory failure and pneumonia. The risk score was then calculated for each individual based on the model. Incidence of major pulmonary events was assessed in the five risk class groupings to assess the discriminative ability of the Ferguson score.

RESULTS

Major pulmonary complications occurred in 35% of patients (47/136). Overall mortality was 6% (8/136). Patients were grouped into five risk categories according to their Ferguson pulmonary risk score: 0–2, 8 patients (6%); 3–4, 24 patients (18%); 5–6, 49 patients (36%); 29 patients (21%); 9–14, 26 patients (19%). The incidence of major pulmonary complications in these categories was 0, 17, 20, 41 and 77%, respectively. The accuracy of the risk score system for predicting major pulmonary complications was 76% (P < 0.0001).

CONCLUSIONS

This pulmonary risk scoring system is a reliable instrument to be used during the preoperative phase to differentiate patients who may be at higher risk for pulmonary complications after oesophagectomy. These data can assist in patient selection, and in patient education/informed consent and can guide postoperative management.

INTRODUCTION

Oesophageal cancer is the sixth leading cause of cancer-related mortality worldwide, and has the fastest growing incidence of any cancer in the USA [1]. Surgical resection is the mainstay of therapy, offering a chance of long-term survival. Oesophagectomy is associated with risk, in both short-term complications and longer term loss of quality of life. Over half of the patients suffer at least one adverse postoperative event [2]. Surgeons have a need for methods to better predict risk for these patients. Risk stratification is a valuable tool to preoperatively identify which patients are at increased risk. An addition to risk stratification is the advent of risk scoring systems. These allow each patient to be stratified into a predictive group based on the system. This technique provides an estimate of an individual's actual risk.

Pulmonary complications remain a frequent postoperative event after oesophagectomy, significantly contributing to prolonged length of stay and postoperative mortality. Ferguson et al. [3] developed a risk scoring system to predict postoperative pulmonary complication following oesophagectomy. These data were collected from their institutional database over 30 years. Utilizing these data, they created a score system using four factors: age, performance status, forced expiratory volume in the first second expressed as percent predicted (FEV1%) and diffusion capacity of the lung for carbon monoxide expressed as percent predicted (DLCO%). Using weighted scores of these four variables, the scoring system predicted pulmonary complications with an accuracy of 70.8%. Our aim was to externally validate the risk score's accuracy using our institutional data.

MATERIALS AND METHODS

The study was reviewed, and approved by our Institutional Review Board. Specific patient consent for this study was waived. We performed a retrospective analysis of our prospectively collected patient database for patients undergoing oesophagectomy for cancer from August 2009 to December 2012.

We collected variables necessary to calculate the Ferguson pulmonary risk score: age, performance status, FEV1 and DLCO%. Pulmonary function tests utilized for this study were obtained prior to resection but after neoadjuvant therapy, if administered. We identified 136 patients that qualified for analysis in the validation study. Two-hundred and seven patients were excluded during the inclusion period secondary to insufficient variables to calculate the risk score. We then analysed the presence of outcome variables. Major outcome variables were major pulmonary complications, defined as need for reintubation for isolated respiratory insufficiency and/or pneumonia, documented by fever, elevated white blood cell count and pulmonary infiltrate requiring antibiotic therapy. Other outcomes evaluated were mortality, defined as 30-day operative or in-hospital mortality. Oesophagectomy was partial or total, and performed utilizing the following approaches: Ivor Lewis, transhiatal, McKeown modification of the Ivor Lewis approach, total or hybrid minimally invasive approaches or resection without reconstruction. Patients were managed according to our standard institution protocol with epidural catheters for pain management, early extubation and ambulation and early enteral nutrition via jejunostomy tube (Supplemental material).

The statistical analysis was conducted using SAS version 9.3 (SAS Institute, Inc., Cary, NC, USA). For univariate comparisons, chi-square tests were utilized to evaluate the association between categorical variables and occurrence of pulmonary complication; when expected counts were low, Fisher's exact test was used. The association between continuous variables and pulmonary complications was analysed using t-tests and Wilcoxon rank-sum tests.

The Ferguson pulmonary risk score is a 5-level risk score comprising four weighted factors (age, performance status, FEV1% and DLCO%), each having 5 different score categories from 0 to 4 (Table 1) [3]. According to the individual weighted scores assigned to each factor, an aggregate score was calculated for each patient. Patients were then grouped into the five different categories of risk according to their scores, and incidence of major pulmonary events was assessed in each class.

Table 1:

Ferguson weighted scoring system for pulmonary complications

Assigned score value	0	1	2	3	4
Age	<50	51–60	61–70	71–80	>80
Performance Status (Zubrod/ECOG)	0	1	2	3	4
FEV1%	≥100	90–99.9	80–89.9	70–79.9	<70
DLCO%	≥100	90–99.9	80–89.9	70–79.9	<70

ECOG: eastern cooperative oncology group; FEV1%: forced expiratory volume in the first second expressed as percent predicted; DLCO%: diffusion capacity of the lung for carbon monoxide expressed as percent predicted.

The risk score was calculated for each individual based on the model. The incidence of major pulmonary events was assessed in the five risk class groupings to assess the discriminative ability of the pulmonary risk score. Receiver operating characteristic curve analysis was utilized to evaluate the accuracy of the system to predict pulmonary complications. As a method of directly validating the scoring system, comparison between the expected and observed outcomes was performed using the Hosmer–Lemeshow goodness-of-fit test.

RESULTS

During the study period, 136 patients undergoing oesophagectomy had the necessary variables available for calculation of the Ferguson risk score. A risk score was not able to be calculated in another 207 patients during this same time period, due to missing pulmonary function tests. Neoadjuvant chemoradiotherapy was given to 81% (110/136) of patients. The majority of resections were for tumours of the lower third (66/136; 49%) or gastroesophageal junction (62/136; 46%) and the histology was adenocarcinoma in 87% (118/136). Approaches to oesophagectomy included the following: Ivor Lewis in 96 (71%), transhiatal in 21 (15%), minimally invasive in 9 (7%), McKeown in 8 (6%) and resection without reconstruction in 2 (2%).

Operative mortality occurred in 8 patients (6%). Major morbidity occurred in 42% (57/136). These included anastomotic leak in 13% of patients (18/136), pulmonary complications in 47 patients (35%), reoperation in 17% (23/136), empyema in 7% (9/136), chylothorax 6% (8/136), myocardial infarction 2% (2/136) and pulmonary embolus 2% (3/136). Major pulmonary events were pneumonia in 44 patients and isolated respiratory failure requiring intubation in 3 additional patients. Of the pneumonia patients, over half (23; 52%) required reintubation, and 48% (21/44) required bronchoscopy for secretion management.

Univariate analysis of the cohort for pulmonary complications is detailed in Table 2. The following were found to be significant predictors: Zubrod/Eastern Cooperative Oncology Group (ECOG) performance status 2 or 3, American Society of Anesthesiology (ASA) classification III, congestive heart failure, hypertension, peripheral vascular disease, smoking status, chronic obstructive pulmonary disease (COPD), insulin requiring diabetes (DM) and worsening FEV1 and DLCO.

Table 2:

Characteristics and univariate analysis of patients undergoing oesophagectomy

Characteristics	All patients	Pulmonary complication (no)	Pulmonary complication (yes)	P-value
Age	63.8 (±8.5)	63.1 (±8.7)	65.2 (± 8.1)	0.18
Gender				0.95
Male	119 (87%)	78 (65%)	41 (35%)
Female	17 (13%)	11 (65%)	6 (35%)
Performance status (Zubrod/ECOG)				<0.001
0–1	127	89 (70%)	38 (30%)
2–3	9	0 (0%)	9 (100%)
ASA classification				0.002
I/II	40	34 (85%)	6 (15%)
III	96	55 (57%)	41 (43%)
Weight loss 3 months (kg)	4.9 (±5.4)	4.5 (±4.9)	5.8 (±6)	0.16
Congestive heart failure				0.048
No	131	88 (67%)	43 (33%)
Yes	5	1 (20%)	4 (80%)
Coronary artery disease				0.36
No	110	74 (67%)	36 (33%)
Yes	26	15 (58%)	11 (42%)
Peripheral vascular disease				0.001
No	125	87 (70%)	38 (30%)
Yes	11	2 (18%)	9 (82%)
Hypertension				0.028
No	58	44 (76%)	14 (24%)
Yes	78	45 (58%)	33 (42%)
COPD				0.008
No	107	76 (71%)	31 (29%)
Yes	29	13 (45%)	16 (55%)
Preoperative chemoradiotherapy				0.36
No	26	15 (58%)	11 (42%)
Yes	110	74 (67%)	36 (33%)
Steroids				1.00
No	133	87 (65%)	46 (35%)
Yes	3	2 (67%)	1 (33%)
Prior cardiothoracic surgery				0.34
No	112	75 (67%)	37 (33%)
Yes	23	13 (56%)	10 (44%)
Diabetes mellitus				0.23
No	101	69 (68%)	32 (32%)
Yes	35	20 (57%)	15 (43)
Diabetic category				0.008
No DM	101	69 (68%)	32 (32%)
Non-insulin	26	18 (69%)	8 (31%)
Insulin	9	2 (22%)	7 (78%)
Cigarette smoking				0.043
Never	33	27 (82%)	6 (18%)
Past smoker	83	52 (63%)	31 (37%)
Current	20	10 (50%)	10 (50%)
FEV1% predicted				0.001
≥60%	123	86 (70%)	37 (30%)
<60%	13	3 (23%)	10 (77%)
DLCO % predicted				0.019
≥60%	122	84 (69%)	38 (31%)
<60%	14	5 (36%)	9 (64%)
Location of disease				0.68
Lower third	66	41 (62%)	25 (38%)
Gastro-oesophageal junction	62	43 (69%)	19 (31%)
Other	8	5 (63%)	3 (37%)

Patients suffering a major pulmonary complication had an increased rate of mortality at 15% (7/47) compared with those who did not suffer one (mortality 1%; 1/89) (P < 0.001). Seven of the 8 patients who died suffered major pulmonary complications, whereas the rate of major pulmonary complications in the rest of the cohort was 31% (40/128). Overall median length of stay for the entire cohort was 8 days [interquartile range (IQR): 7–13 days]. Median length of stay was significantly longer for patients with pulmonary complications at 12 days (IQR: 8–23 days) compared with those without pulmonary complications, 7 days (IQR: 6–10 days) (P < 0.001).

Figure 1 displays the distribution of patients according to their pulmonary risk score, and Fig. 2 graphically illustrates the incidence of pulmonary complications according to risk score. The patients were then subdivided into the five risk categories based on the score number: 0–2, 3–4, 5–6, 7–8and 9–13, as divided in the Ferguson pulmonary risk scoring system. The rate of major pulmonary complications in these patients according to the risk categories was then determined, and displayed in Table 3 (P < 0.001) and detailed graphically in Fig. 3.

Figure 1:

Distribution of risk scores for the study population.

Figure 2:

Incidence of pulmonary complications according to risk scores.

Table 3:

Incidence of pulmonary complications according to risk score quintiles

Ferguson risk score category, no.	Number of cases	Major pulmonary complications	P <0.001
1–2	8	0 (0%)
3–4	24	4 (17%)
4–6	49	10 (20%)
7–8	29	13 (45%)
9–14	26	20 (77%)

Figure 3:

Incidence of pulmonary complications according to risk score quintiles.

The receiver operating characteristic area under the curve of the Ferguson pulmonary risk score system in this validation cohort was 0.762 for predicting major pulmonary events (Fig. 4). Overall, this correlates to a discriminative ability of the score of 76% to predict these events. This indicates a moderate discriminative ability of this index. The Hosmer–Lemeshow goodness-of-fit test indicated no difference between the expected and observed results (P = 0.2394).

Figure 4:

Receiver operating characteristic curve for risk score and pulmonary complications (area under the curve = 0.762; 95% confidence interval: 0.719—0.805; P < 0.0001).

DISCUSSION

Risk stratification is a valuable tool for surgeons in determining which patients are at increased risk for postoperative events. Further, in the new paradigm of health care delivery, with emphasis on value and quality of outcomes, surgeons must have the best and most up-to-date methods to accurately predict patient outcomes. The Goldman index for predicting postoperative cardiac events is perhaps the best known surgical risk stratification method, more recently updated into the revised cardiac risk index (RCRI) [4, 5]. The thoracic revised cardiac risk index is a further derivation of the RCRI, specialty specific, which predicts risk for major cardiovascular events after pulmonary resection [6]. These systems illustrate the applicability of global as well as specialty-specific risk scoring systems.

Pulmonary complications continue to contribute to major morbidity and mortality after oesophagectomy. Ferguson et al. assessed their institution's database of over 30 years of oesophageal resections, and created a risk score to attempt to predict these complications. He further used his data to internally validate this model. We wished to assess this pulmonary risk score with an external cohort.

Traditionally, pulmonary complications have been attributed to a variety of a factors, including advanced age, pulmonary dysfunction, performance status, use of thoracotomy, poor nutrition preoperatively, continued smoking and use of induction therapy [7–12]. The pulmonary risk scoring system evaluated in this study simplifies these down to four weighted variables: age, performance status, as defined by the Zubrod system, FEV1% and DLCO%.

The incidence of pulmonary complications in our population was similar to the original score data set (35% compared with 38%). We also showed an association between pulmonary complications and prolonged length of stay and mortality. Overall, the morbidity rate was high, but is in line with current and historical large series of oesophageal resections [2, 13–16]. Our major pulmonary complication rate was slightly higher than other large series; however, we used a strict definition similar to that described by Ferguson et al. Many other studies only report pneumonia. Our cardiovascular morbidity was very low.

Substantial disagreement exists whether neoadjuvant therapy increases the risk of postoperative morbidity and mortality. Multiple studies found that neoadjuvant therapy does increase the risk of morbidity postoperatively [7, 11, 17]. However, other prominent studies showed the opposite [8, 18–20]. Over 80% of patients in our series underwent neoadjuvant therapy with no difference in the rate of pulmonary complications. Ferguson made this conclusion from this series as well from his series [3]. The practice at our centre is to obtain pulmonary function tests 4 weeks after completion of neoadjuvant treatment, usually the day prior to surgery, allowing for a recovery period after chemotherapy and radiotherapy.

Frailty often is described as a measure of physiological reserve, and has recently been used in attempts to predict postoperative morbidity and mortality [21–24]. Frailty is a factor increasingly recognized as contributing to adverse outcomes, and may enhance or supplant other risk scores in the future.

For a predictive model to be clinically useful, ideally it is simple with few variables and rapid to interpret. The current frailty indices are unwieldy for use in busy day-to-day clinical practice. Ferguson et al.'s model potentially could be a surrogate for these complex frailty indices. However, this composite score could be simplified into the Ferguson predictor since it includes an assessment of overall functional status (Zubrod/ECOG) and pulmonary function.

One inherent bias in this study is that the data are from a single, high-volume centre. Secondly, this study is retrospective; however, the patients were collected prospectively in our department database. Further, the overall cohort from the time frame studied had a large number of missing pulmonary function test variables. This prevented imputation techniques to include more patients during this time frame. These missing tests also potentially may have skewed our results towards more debilitated patients being included, as presumably younger or patients in better condition may not have had preoperative pulmonary function tests. Examining the remaining cohort of 207 patients without pulmonary function tests during the time period, the overall pulmonary complication rate was 27% (56/207), with pneumonia in 24% (49/207) and isolated respiratory failure in 3% (7/207). Overall operative mortality in this cohort was 1% (3/207). Thus, as presupposed, the patients who did have pulmonary functions tests were a higher risk group based on the fact that the test was performed. Nevertheless, an examination of the combined cohort of 343 patients did not find that the choice to obtain pulmonary function tests themselves was an independent risk factor for major morbidity or mortality.

In conclusion, we validated the Ferguson pulmonary risk score with an external independent series of oesophageal resections. Although the generalization of the present findings would benefit from a larger database validation, we believe that this validation supports the use of this score as a screening instrument during patient risk stratification prior to oesophagectomy. The Ferguson pulmonary risk scoring system is a reliable instrument to be used during the preoperative phase to differentiate patients who may be at higher risk for pulmonary complications after oesophagectomy. These data can assist in patient selection, patient education/informed consent and guide postoperative management.

SUPPLEMENTARY MATERIAL

Supplementary Material is available at EJCTS online.

Conflict of interest: none declared.

APPENDIX. CONFERENCE DISCUSSION

Scan to your mobile or go to http://www.oxfordjournals.org/page/6153/1 to search for the presentation on the EACTS library

Dr S. Rathinam (Leicester, UK): The risk complications of an oesophagectomy is a major factor and Dr Ferguson came up with a nice algorithm to identify that, and your group has validated it in a nice fashion. I have a few questions which might have a bearing on your results and outcomes.

You have excluded a sizable proportion of your patients owing to lack of the full pulmonary function test.

So what is your institution's preoperative work-up algorithm, and equally, how do these factors contribute to PPCs after oesophagectomy in terms of your mobilisation, physiotherapy contrast? So what is your postoperative algorithm?

My second question is in terms of timing of your pulmonary lung function tests, in patients having induction therapy, do you repeat them before surgery, or do you go by the base value before the induction therapy?

Dr Reinersman: Thank you. I will answer the second question first. Often, the patients have pulmonary function tests prior to neoadjuvant therapy, but we obtain repeat pulmonary function tests after neoadjuvant therapy prior to proceeding to resection.

As for the first question, all patients have a history and physical and appropriate staging which includes EGD, EUS and a PET/CT. From this staging information, we consider whether these patients proceed directly to surgery, or if they have T3 or N1 positive tumours, then those patients will progress to concurrent neoadjuvant radiotherapy and chemotherapy.

They come back four weeks afterwards, and are seen again, have a repeat history and physical exam, as well as a repeat PET/CT to look for distant metastases, and then we move forward with resection.

Otherwise, as for who gets PFTs, I think over the recent past, now that we have realized the importance of PFTs, we have been starting to get them more often and in a majority of our patients; as opposed to previously, they were not available as often in the earlier patients two or three years ago.

As for postoperative management, all of our patients are aggressively mobilized early. We work towards immediate extubation postoperatively so that we can have them up, walking around, with early pulmonary therapy, and pulmonary rehab. We have dedicated respiratory therapists on our ward, as well as a pulmonary care rehab centre that the patients are taken to on a daily basis to achieve these means.

Dr D. Wood (Seattle, WA, USA): I have a couple of questions for you.

One, back to the aspect of 207 patients not being included because they did not have PFTs, I am going to make a supposition that may be incorrect, but I am going to guess that those were better patients, patients that maybe looked really good, and therefore, did not have PFTs. So what we are seeing is a subset that you have analysed that is actually the patients with more co-morbidity.

Do you think that is correct, and does that impact at all how to interpret this? Maybe 2 or 3 patients do not even get onto the scoring system, because they are in good shape.

Dr Reinersman: Yes, we addressed this a bit in the manuscript. Due to the fact that our mortality, as well as other complications, was a bit higher than in our overall internal institutional database, these are likely sicker patients. The patients without PFTs upon initial assessment were likely healthier and were not deemed necessary to get pulmonary function tests ahead of time.

Dr Wood: And my second question, I am presuming that a majority of your resections were Ivor-Lewis oesophagectomies?

Dr Reinersman: Yes. Approximately 70% of them were Ivor-Lewis oesophagectomies.

Dr Wood: Do you think this influences your new desire to get PFTs on everyone, if you are going to be doing a transhiatal oesophagectomy rather than a transthoracic oesophagectomy?

Dr Reinersman: I think my inherent bias would be to say yes; however, in Dr Ferguson's original paper, he did show that there was no difference in risk in the patients who had a thoracotomy versus a transhiatal oesophagectomy.

Dr A. Lerut (Leuven, Belgium): Following up on this aspect of access, were they all done by open surgery, or were they done by minimally invasive thoracoscopic and laparoscopic surgery?

Because I guess that are interfering factors, and the other factor, which is not taken up in the Ferguson model. Or other surgical factors, like anastomotic leaks, where of course, you have a much higher chance for a pulmonary infection that therefore may interfere in this model. Were they all open surgeries?

Dr Reinersman: The majority of them were open. Approximately 7% of the patients were done with a minimally invasive oesophagectomy, with usually a combination of a laparoscopic approach in the abdomen, and then some version of a robotic or thoracoscopic approach in the chest.

Dr Lerut: And did you look at whether there was an influence from that minimally invasive access on your scoring model?

Dr Reinersman: We did not specifically look at that in this dataset whether it made a difference.

Dr Lerut: Any anastomotic leaks?

Dr Reinersman: We did not assess whether that was associated, but the leak rate was approximately 10%.

Dr G. Friedel (Stuttgart, Germany): A short question. I think 3 of your factors are not independent of a higher age. Maybe it has a higher ECOG, and maybe it has a decrease in FEV1?

Dr Reinersman: I agree.