Postoperative Mortality is an Inadequate Quality Indicator for Lung Cancer Resection

Abstract

Background

Postoperative mortality is the most commonly reported surgical quality measure. However, such metrics may be incapable of identifying performance outliers. The purpose of this study was to compare different measures of postoperative mortality following lung cancer resection using a large multi-institutional database.

Methods

Data were extracted for lung cancer resection patients from the linked SEER-Medicare registry (2006–2010) which provides detailed and longitudinal information about Medicare beneficiaries with cancer. Four definitions of postoperative mortality were evaluated: in-hospital, 30-day, perioperative and 90-day. Hierarchical regression models were used to estimate mortality risk at 30 and 90 days, and provider quality was assessed by comparing observed versus expected mortality.

Results

We identified 11,787 lung cancer resection patients from 686 hospitals. The median age was 74 years and 52% of patients were treated with open lobectomy. While 30-day, perioperative and in-hospital mortality rates were between 3–4%, 90-day mortality was almost double (6.89%). Clinical variables associated with 90-day mortality included gender, preexisting comorbidities, and procedure type. There were no statistically-significant differences in 30- or 90-day mortality among providers.

Conclusions

Currently-reported measures of in-hospital and 30-day postoperative mortality do not adequately represent a patient’s true mortality risk as mortality almost doubles by 90-days. Due to low occurrence rate and variable provider volumes, neither 30- nor 90-day mortality are suitable quality indicators for lung resection.

Introduction

Lung cancer remains the leading cause of cancer death in the United States (1). Over the past decade, the Society of Thoracic Surgeons (STS) General Thoracic Surgery Database (GTSD) has provided data reflecting predictors of morbidity and mortality following surgical interventions for lung and esophageal cancer (2–6). Within this database comprised primarily of board-certified thoracic surgeons at high-volume centers, 30-day postoperative mortality has been estimated at 2.2% (7), and thorascopic techniques have been associated with reduced postoperative morbidity (8, 9). While the STS-GTSD data is comprised primarily of general thoracic surgery specialists (6), the Surveillance Epidemiology and End Results (SEER)-Medicare database may better represent operative experiences nationwide (10).

The Center for Medicare and Medicaid Services (CMS) derives its surgical quality indictors from the Agency for Healthcare Research and Quality (AHRQ), with heavy focus on inhospital and 30-day mortality. These quality indicators possess inherent appeal: they are clinically relevant and accurately measured through claims records. However, as critical care capabilities improve and thoracic specialists pursue more aggressive resections, 30-day mortality may underestimate surgery-related mortality and morbidity (11–13). Although 90-day mortality is a clinically relevant outcome measure, this metric is rarely reported as most databases do not track patients after 30 days. Furthermore, there is little evidence that postoperative mortality is able to differentiate between good and poor performers given its low occurrence rate and variable provider volumes. The objectives of this study were to compare inhospital, 30-day, perioperative, and 90-day mortality measures following lung cancer resection and assess these mortality metrics’ capacities to differentiate between good and poor performers.

Methods

SEER-Medicare Database

The SEER registry is a population-based collection of incident cases, and includes cancer diagnostic, descriptive, and therapeutic information linked to survival data. The National Cancer Institute links the SEER registry to Medicare data for eligible patients to provide comprehensive information on survival, inpatient admissions, outpatient events, and other healthcare claims for 93% of patients 65 years old or older (14). Although there are differences between SEER registry patients and the Medicare population as a whole, the combined SEER-Medicare database encompasses approximately 26% of the population, and provides an opportunity for longitudinal studies broadly generalizable to the Medicare population.

Patient Selection

The 2006 to 2010 SEER-Medicare database was used to identify records for all patients age 66 or greater with non-small cell lung cancer (NSCLC) of any stage by American Joint Committee on Cancer criteria who received surgical resection (15). Exclusionary criteria included enrollment in a Medicare HMO, lung cancer diagnoses made at autopsy, prior lung cancer diagnosis within one year of index diagnosis, missing date of diagnosis, and wedge resection for stage IV disease as this was more likely to be a diagnostic procedure. To ensure that all patients had at least one year of pre-surgical records to identify comorbid diseases present at the time of surgery, we additionally excluded patients who did not meet insurance criteria during the three months prior to surgery, or who were diagnosed in 2006.

Demographic information included age, gender, race and treating facility. Clinical data included year of operation, final pathologic stage, procedure type and approach, and comorbidities. Comorbidities were identified using the Deyo modification of the Charlson index (16), and were collected using SAS search code provided by the National Cancer Institute based on inpatient files (MEDPAR), outpatient files (OUTSAF), and physician claims data (NCH) (17, 18). The primary objectives were to estimate patient risk and compare providers across four postoperative outcomes measures: in-hospital, 30-day, perioperative, and 90-day mortalities. In-hospital mortality was defined as death prior to discharge following surgery and perioperative mortality included any death occurring in-hospital or within 30 days of surgery. All mortality measures were based on Medicare death certificate records within the SEER-Medicare database.

Statistical Analyses

To compare 30- and 90-day mortality, we calculated the 95% confidence interval for proportion of deaths occurring in the second and third months postoperatively. Hierarchical generalized logistic regression models were used to estimate 30-day and 90-day mortality risk, with adjustments for data clustered by treatment provider. Model predictors were selected a priori based on literature review and frequency of occurrence within our dataset. Modeling was first performed with individual comorbidity variables, and then repeated using the composite Charlson-Deyo comorbidity index score. The statistical significance of each predictor of mortality included in the models was assessed using the F test statistic.

To test the utility of mortality rate as a quality measure, we removed the hospital clustering effect to calculate an expected mortality rate for each hospital based on patient characteristics, and then compared this to each provider’s observed mortality. A Bonferroni correction was used to adjust for multiple comparisons at the alpha = 0.05 level. All outcomes data were analyzed using SAS statistical software (version 9.3; SAS Institute, Inc, Cary, NC). Provider volume data is represented using R statistical software together with the ggplot package (19, 20). The University of Virginia Institutional Review Board for Health Sciences Research approved this study.

Results

Between 2007 and 2010, SEER-Medicare captured 11,787 patients who underwent surgical resection for NSCLC and met all inclusion criteria (Figure 1). The median age was 74 years at the time of surgery and most patients presented with stage I disease (70%, 8103/11787). Roughly half of patients were female (51%, 6012/11787) and the predominant race was white (89.6%, 10599/11787). The most common procedure performed was an open lobectomy (51.9%, 6119/11787), and thoracoscopic approaches accounted for 26.4% (3110/11787) of resections (Table 1). Patients included within the final study population were treated at 686 hospitals. Hospital case volume ranged between 1 and 383 cases, and one third of hospitals (32.9%, 226/686) treated two or fewer patients during the study period (Figure 2). Postoperative mortality rates are shown in Table 2. While 30-day, perioperative and in-hospital mortality rates were each between 3–4%, 90-day mortality was almost double (6.89%, 812/11787). Etiologies of deaths occurring between 30 and 90 days from surgery were assessed through review of ICD-10 codes within the SEER-Medicare database. The most frequently captured code was malignant lung neoplasm. Excluding these, the most common causes of death were COPD, infection, and coronary artery disease.

Figure 1.

Inclusion and exclusion criteria for lung cancer resection dataset, based on the 2006–20010 SEER-Medicare registry. Patients diagnosed in 2006 were excluded to ensure availability of 1 year of preoperative comorbidity records. NSCLC: non-small cell lung cancer. SEER: Surveillance Epidemiology and End Results

Table 1.

Preoperative Patient Demographics.

Demographics	N (%)
Age
65–69	2806 (23.8%)
70–74	3518 (29.9%)
75–79	3129 (26.6%)
80–84	1782 (15.1%)
85+	552 (4.7%)
Gender
Male	5775 (49.0%)
Female	6012 (51.0%)
Year of Diagnosis
2007	3989 (33.8%)
2008	3935 (33.4%)
2009	3863 (32.8%)
Race
White	10559 (89.6%)
Black	630 (5.3%)
Asian	270 (2.3%)
Hispanic	85 (0.7%)
Other	243 (2.1%)
Tumor Stage
I	8103 (70.0%)
II	1325 (11.5%)
III	1753 (15.1%)
IV	388 (3.4%)
NA	218 (1.9%)
Operation
Open lobectomy/bilobectomy	6119 (51.9%)
Open wedge resection	1271 (10.8%)
Open segmentectomy	823 (7.0%)
Open pneumonectomy	372 (3.2%)
Chest wall resection with lung	92 (0.8%)
VATS lobectomy	1661 (14.1%)
VATS wedge resection	950 (8.1%)
VATS segmentectomy	499 (4.2%)
Comorbidities
Myocardial Infarction	1241 (10.5%)
Heart Failure	1345 (11.4%)
Peripheral Vascular Disease	1812 (15.4%)
Cerebral Vascular Disease	1198 (10.2%)
Chronic Pulmonary Disease	7313 (62.0%)
Diabetes	3598 (30.5%)
Chronic Renal Failure	1064 (9.0%)

Figure 2.

Provider lung cancer resection volumes between 2007 and 2010. Case volumes ranged from 1 to 383. A total of 226 hospitals (33%) performed two or fewer resections during the study period. SEER: Surveillance Epidemiology and End Results

Table 2.

Postoperative Mortality Measures.

Mortality	N
In-hospital	355 (3.01%)
30-day	435 (3.69%)
Perioperative	482 (4.09%)
90-day	812 (6.89%)

Hierachchical generalized linear regression models were used to estimate the probability of 30- and 90-day mortality. Hospital provider was included in the models as a random effect, accounting for the clustering of procedures within hospitals. For the purpose of clinical applicability, we present the models using individual comorbid conditions as covariates. However, using the combined Charlson-Deyo comorbidity index rather than individual comorbidities yielded similar model fit. Areas under the ROC curve (C-statistic) for predictive accuracy were 0.746 and 0.722 for 30-day and 90-day models, respectively. Predictors of 30-day mortality with the highest F-statistic in descending order were gender, congestive heart failure, cerebrovascular disease, and procedure type (Table 3). By comparison, in the 90-day mortality model, congestive heart failure displaced gender as the most statistically significant predictor of mortality (Table 4). Of the procedure types evaluated, pneumonectomy and combined lung and chest wall resection had the highest odds ratio of mortality.

Table 3.

Hierarchical Linear Regression Model: 30-day Mortality

Variable	OR	CI	p-value	F test
Procedure Type				13.63
Pneumonectomy	7.25	4.16 – 12.63	<0.001
Chest wall resection with lung	5.35	2.31 –12.36	<0.001
Lobectomy/bilobectomy	1.92	1.19 – 3.08	0.007
Segmentectomy	1.80	0.99 – 3.26	0.052
Wedge resection	2.15	1.26 – 3.66	0.005
VATS lobectomy	1.25	0.73 – 2.15	0.42
VATS segmentectomy	0.58	0.23 – 1.46	0.247
VATS wedge resection	REF
Surgical Year				2.25
2007	0.62	0.39 – 0.98	0.043
2008	0.74	0.47 – 1.17	0.203
2009	0.82	0.52 – 1.30	0.401
2010	REF
Age				7.48
65–69	0.40	0.26 – 0.61	<0.001
70–74	0.43	0.29 – 0.65	<0.001
75–79	0.68	0.46 – 1.01	0.053
80–84	0.58	0.38 – 0.88	0.012
85+	REF
Gender				32.5
Female	0.55	0.44 – 0.67	<0.001
Male	REF
Race				0.79
Asian	1.47	0.83 – 2.59	0.186
Black	1.16	0.75 – 1.77	0.507
Other	1.19	0.68 – 2.07	0.551
White	REF
Induction Radiation	1.71	1.20 – 2.43	0.003	8.94
Acute MI	1.54	1.17 – 2.02	0.002	9.57
CHF	1.86	1.44 – 2.39	<0.001	22.88
PVD	0.98	0.76 – 1.28	0.907	0.01
Cerebrovascular Disease	1.87	1.44 – 2.43	<0.001	22.06
COPD	1.11	0.90 – 1.37	0.334	0.93
Diabetes	1.03	0.82 – 1.28	0.801	0.06
Renal Failure	0.99	0.72 – 1.35	0.933	0.01

REF = reference VATS = video-assisted thorascopic surgery MI = myocardial infarction CHF = congestive heart failure PVD = peripheral vascular disease COPD = chronic obstructive pulmonary disease

Table 4.

Hierarchical Linear Regression Model: 90-day Mortality

Variable	OR	CI	p-value	F test
Procedure Type				21.67
Pneumonectomy	5.97	3.99 – 8.94	<0.001
Chest wall resection with lung	5.65	3.08 – 10.36	<0.001
Lobectomy/bilobectomy	1.57	1.14 – 2.17	0.006
Segmentectomy	1.71	1.14 – 2.58	0.01
Wedge resection	1.60	1.10 – 2.34	0.14
VATS lobectomy	1.01	0.69 – 1.49	0.942
VATS segmentectomy	0.66	0.37 – 1.19	0.165
VATS wedge resection	REF
Surgical Year				3.88
2007	0.70	0.48 – 1.02	0.061
2008	0.80	0.56 – 1.15	0.23
2009	0.96	0.67 – 1.38	0.813
2010	REF
Age				10.24
65–69	0.39	0.28 – 0.54	<0.001
70–74	0.47	0.35 – 0.64	<0.001
75–79	0.61	0.45 – 0.82	0.001
80–84	0.63	0.46 – 0.87	0.0047
85+	REF
Gender				48.68
Female	0.58	0.50 – 0.67	<0.001
Male	REF
Race				0.16
Asian	0.86	0.50 – 1.45	0.565
Black	1.05	0.76 – 1.45	0.766
Other	0.94	0.60 – 1.48	0.799
White	REF
Induction Radiation	1.69	1.28 – 2.23	<0.001	13.98
Acute MI	1.25	1.00 – 1.55	0.045	4.01
CHF	2.13	1.76 – 2.57	<0.001	61.11
PVD	1.08	0.89 – 1.32	0.428	0.63
Cerebrovascular Disease	1.48	1.20 – 1.82	<0.001	13.09
COPD	1.24	1.06 – 1.46	0.009	6.89
Diabetes	1.03	0.87 – 1.21	0.745	0.11
Renal Failure	1.27	1.01 – 1.59	0.038	4.31

REF = reference VATS = video-assisted thorascopic surgery MI = myocardial infarction CHF = congestive heart failure PVD = peripheral vascular disease COPD = chronic obstructive pulmonary disease

Tests for provider covariance were significant for both 30- and 90-day models, suggesting that variations in provider quality do exist (p = 0.022 for both models). After removing the provider random effect and adjusting for patient-specific covariates, 90-day observed-to-expected mortality (O/E) ratio was significantly different from 1 (p < 0.05) for only 20 (3%) of the 686 hospitals evaluated. The O/E ratio was greater than 1 for 18 providers and less than 1 for 2 providers. Similarly, 30-day O/E ratio was significantly different from 1 (p < 0.05) for 20 providers, was greater than 1 for 19 providers, and less than 1 for 1 provider. However, because we tested O/E ratios for all 686 providers, some statistically significant results at the 0.05 level are expected due to multiple testing. After adjusting for multiple comparisons, no individual provider’s observed mortality rate was convincingly different from expected.

Discussion

The search for quality indicators following resection for lung cancer has historically been hampered by sampling and reporting biases. Although 30-day mortality is the most frequently quoted indicator for postoperative outcomes through AHRQ and CMS (21), our study indicates that 90-day mortality nearly doubles this. This finding emphasizes the importance of clinically-relevant discussions with patients regarding the expected outcomes following lung cancer resection. More broadly, postoperative mortality metrics of any definition are poor quality indicators for lung cancer resection as they fail to differentiate good from poor performers.

Our study using SEER-Medicare data encompasses a large, diverse group of surgical providers. Within this group, 30-day mortality following lung cancer resection is 3.69%. This proportion is higher than that reported by the STS-GTDB (2.2%) (7) and the American College of Surgeons Z0030 trial (2.0%) (22), but lower than the 5.2% mortality reported by the National Veterans Affairs Surgical Quality Improvement Program (23). Both the STS-GTDB and the ACOSOG Z0030 trial report outcomes predominantly from thoracic surgery specialists at tertiary referral centers which are likely not representative of the operative community nationwide. The Veterans Affairs Program represents a patient demographic that is predominantly male. The effect of this homogenous patient pool is emphasized by our finding that, after adjusting for the effects of other potential confounders, male gender is significantly associated with poorer 30-day outcomes, with a mortality odds ratio of 1.8.

The results of the present study question the utility of using postoperative mortality as a quality metric. No meaningful outliers were identified based on observed-to-expected mortality calculations at either 30 days or 90 days. The most probable explanation of this finding is that the volume of deaths contributed by any single provider during the study period was too low to draw meaningful statistical conclusions given the multiple O/E ratio tests performed. This hypothesis is supported by prior research indicating that coronary bypass surgery is the only operation for which there is sufficient operative volume and mortality rate to justify using postoperative mortality as a stand-alone quality metric (24). One potential adaptation of these methods would be to calculate O/E ratios only for providers with large case volumes. However, such a study would exclude the majority of hospitals performing lung cancer resections.

The hospital distribution of lung cancer resection case volumes within our 2007–2010 SEER-Medicare database indicates a predominance of low-volume providers, largely unchanged from the surgical landscape nearly 15 years ago (25). The effects of hospital volume and provider specialization on postoperative outcomes have come under extensive scrutiny over the last two decades. While Bach and colleagues originally categorized hospital volume and demonstrated its utility as a predictor of short- and long-term outcomes following lung cancer resection (26), recent evidence indicates that case volume may play a more minor role when interpreted as a continuous variable (27, 28). Our findings do not imply a relationship between surgical quality and volume, but rather that mortality is a poor quality indicator due to inadequate statistical power among the majority of centers.

The primary reason that postoperative mortality is a poor surgical quality indicator is its low rate of occurrence following lung cancer resection. Therefore, there is impetus to derive quality indicators from more frequently-occurring outcomes or from composite outcome metrics. The adult cardiac surgery community within STS has pioneered a multi-faceted model of quality based on structural, procedural, and outcome measures to create a composite scoring system (29, 30). This group has exemplified through its transparent methodology that it is imperative for interested parties to understand what measures are being included in a composite metric, how the different measures are weighted, and the statistical modeling procedures that are used. A lobectomy counterpart in the GTSD is in development, and takes into account risk-adjusted morbidity in addition to postoperative mortality (31).

We found that 30-day mortality is similar to perioperative and in-hospital mortality, but is clinically and statistically lower than 90-day mortality. Our results approximate findings by Rueth and colleagues, who found 30- and 90-day mortality rates of 4.01% and 6.3%, respectively, via a review of SEER-Medicare data from 2000–2005 limited to patients less than 80 years of age with stage I NSCLC (10). Within a thoracotomy population at a highly-specialized single institution, Cerfolio and colleagues determined that age, hypertension and history of coronary disease were significantly associated with 90-day mortality. (32) In the current study, patient comorbidities and type of resection were again strong predictors of increased mortality risk following lung cancer resection. Our model further identified gender, age, and induction radiation as predictors of increased mortality at 90 days, in agreement with findings from the STS database (7).

There are several limitations to this study. First, the SEER-Medicare database excludes most patients less than 65 years of age, a population that is presumably different in its comorbidity profile. However, the median age of patients included in this study was 74, which is only slightly older than the median age at diagnosis for NSCLC nationally (70 years) (33) and the age of lung cancer resection patients included in the STS database (68 years) (7). Second, SEER-Medicare cause of death reporting is based upon ICD-10 codes. Within this database, the etiology of roughly half of all deaths between 30 and 90 days postoperatively were coded as lung cancer. This artifact is likely due to unfamiliarity with ICD-10 coding resulting in the identification of lung cancer, rather than specific postoperative complications, as the root cause of death. However, other causes of death remain consistent in their relative frequencies, supporting the fact that the most common postoperative complications (cardiac, infectious, and pulmonary) remain highly relevant beyond 30 days from surgery.

Conclusions

Postoperative mortality following lung cancer resection is not a meaningful quality indicator for providers as it does not differentiate between good and poor performers. However, postoperative mortality rates are an important outcomes measure to help guide clinicians and their patients. Our findings suggest that 30-day mortality rate underestimates a patient’s true risk of death following lung cancer resection, and that the risk of death at 90-days is almost twice the 30-day rate. These findings are important for informing preoperative counseling and for promoting accurate patient expectations. The Society of Thoracic Surgeons Database should consider using 90-day mortality as an outcomes measure. Given the higher volume of procedures performed by its members, the 90-day mortality metric may improve the ability to differentiate hospital performance.