STS Lung Cancer Resection Risk Model: Higher Quality Data and Superior Outcomes

Abstract

Background

The Society of Thoracic Surgeons (STS) creates risk adjustment models for common cardiothoracic operations for quality improvement purposes. Our aim was to update the lung cancer resection risk model utilizing the STS General Thoracic Surgery Database (GTSD) with a larger and more contemporary cohort.

Methods

We queried the STS GTSD for all surgical resections of lung cancers from January 1, 2012 through December 31, 2014 Logistic regression was used to create three risk models for adverse events: operative mortality, major morbidity and composite mortality and major morbidity.

Results

27,844 lung cancer resections were performed at 231 centers. 62% (n=17,153) were performed via thoracoscopy. The mortality rate was 1.4% (n=401), major morbidity rate was 9.1% (n=2,545) and the composite rate was 9.5% (2,654). Predictors of mortality included age, male gender, FEV1, body mass index, cerebrovascular disease, steroids, coronary disease, peripheral vascular disease, renal dysfunction, Zubrod score, American Society of Anesthesiologists rating, thoracotomy approach, induction therapy, reoperation, tumor stage, and greater extent of resection. (all p<0.05). For major morbidity and the composite measure, cigarette smoking becomes a risk factor while stage, renal dysfunction, congestive heart failure, and cerebrovascular disease lose significance.

Conclusions

Operative mortality and complication rates are low for lung cancer resection among surgeons participating in the GTSD. Risk factors from the prior lung cancer resection model are refined and new risk factors such as prior thoracic surgery are identified. The GTSD risk models continue to evolve as more centers report and data is audited for quality assurance.

The Society of Thoracic Surgeons (STS) has utilized its clinical registry, the STS National Database, to create risk adjustment models to predict outcomes for common cardiothoracic operations based on patient characteristics.^1-6 These risk models inform clinical decision making and allow cardiothoracic surgeons to analyze their risk-adjusted outcomes for purposes of quality improvement. With regular feedback of results to participating sites, outcomes for major cardiothoracic procedures have continued to improve over time. In part due to these risk-adjustment models, the STS National Database is recognized as the premier clinical data registry in the United States.

In 2008, the first risk model for lung cancer resection was created utilizing the STS General Thoracic Surgery Database (GTSD).⁴ The primary clinical outcome measure in this model was postoperative length of hospital stay, as the database was relatively small and there were relatively few numbers of complications to allow for creation of a robust model. As the penetrance of the GTSD increased, the lung cancer resection model was updated in 2010.⁵ With the availability of more data, outcome measures at that time were changed to mortality, major morbidity, and composite mortality or major morbidity.

Since the publication of the last lung cancer resection risk model, several important changes have occurred in the general thoracic surgical community and with the GTSD. The use of minimally invasive techniques for pulmonary resections has become more widely disseminated in the thoracic surgical community, particularly among surgeons submitting to the STS GTSD. In addition, the penetrance of the GTSD has increased and, consequently, the number of centers submitting data and the number of patients included in the database has significantly increased. Further, in part due to education of data abstractors, the amount of missing data in the GTSD has notably decreased. This allows the use of more representative data in the generation of risk models. Finally, the GTSD has also undergone regular external audits since the creation of the last risk model, and high degree of accuracy in the data fields has been demonstrated.⁷ Our objective was to update the STS GTSD lung cancer resection risk model on a larger, contemporary cohort of patients.

Material and Methods

Society of Thoracic Surgeons Database

In 2002, The STS formally established the GTSD component of the STS National Database as a voluntary effort to support continued quality improvement efforts of thoracic surgeons and hospitals. The GTSD provides participating members with risk-adjusted national thoracic surgical benchmarks for lung and esophageal cancer resections. Risk adjusted short-term results are provided to participating institutions on a twice yearly basis. The STS GTSD has been externally audited since 2010.⁷ Audits have demonstrated high agreement rates with hospital records and validated the accuracy and completeness of the data. Each participating center exempted this investigation from formal Institutional Review Board approval as it represents an analysis of data collected for quality review and secondary research purposes with the absence of Health Insurance Portability and Accountability Act patient identifiers.

Patient Population

We queried the STS GTSD for patients treated with surgical resection for primary lung cancer from January 1, 2012 through December 31, 2014. Surgical procedures included were the following: wedge resection, segmentectomy, lobectomy, sleeve lobectomy, bilobectomy, and pneumonectomy. Patients were excluded if they had an extrapleural pneumonectomy, completion pneumonectomy, carinal pneumonectomy, occult carcinoma or benign disease on final pathology, or an urgent, emergent or palliative operation. Further, patients with missing age, gender, discharge mortality status, and predicted forced expiratory volume in one second (FEV1) were also excluded. Out of 28,473 patients eligible for analysis, 629 (2.2%) were excluded from analysis resulting in a final cohort of 27,844.

Outcome Measures

The primary outcome measures were operative mortality and major morbidity, as done in prior STS GTSD risk models.⁵ Postoperative events were defined according STS GTSD.⁸ A death during the index hospitalization for surgery or within 30 days of the procedure is classified as an operative mortality. Major morbidity was previously defined in the first GTSD lung cancer resection risk model through empiric selection of important adverse outcomes.⁵ These include: tracheostomy, reintubation, initial ventilatory support greater than 48 hours, adult respiratory distress syndrome, bronchopleural fistula, pulmonary embolus, pneumonia, unexpected return to the operating room (changed from bleeding requiring reoperation), and myocardial infarction. Three separate outcomes were examined: mortality, major morbidity, and composite mortality and major morbidity. Mortality is the most extreme complication but has a low event rate. Examining major morbidity and the composite measure allows for comparison of participants.

Covariate Selection

Covariates included for risk adjustment were selected a priori from the most recent version of the STS GTSD data collection instrument (v2.2). Diffusion capacity of the lung for carbon monoxide (DLCO) was excluded from analysis due to 14% (3,805) missingness. An imputation approach was not used for DLCO as the authors felt that the missing at random assumption was not met. Body mass index (BMI) was missing in only 1% of population and was imputed by a gender specific median. Table 1 depicts the baseline characteristics selected for analysis in our patient cohort.

Table 1.

Patient Baseline Characteristics

Variable	No. (% of all patients)	Mean ± Standard Deviation
Total	27,844 (100%)
Age, years		67.2 ± 10.1
Male gender (%)	12,647 (45.4%)
Race
White	24,099 (87.0%)
Black	2,369 (8.6%)
Other	1,217 (4.4%)
Body mass index (kg/m2)*		27.6 ± 6.2
Coronary artery disease	6,196 (22.3%)
Diabetes	5,158 (18.5%)
Renal dysfunction	504 (1.8%)
Induction chemotherapy and/or radiation	1,801 (6.5%)
Cigarette smoking
Never	3,895 (14.0%)
Past (stopped more than 1 month)	17,368 (62.4%)
Current	6,581 (23.6%)
Steroids	965 (3.5%)
Minimally invasive	17,153 (61.6%)
Thoracotomy	10,691 (38.4%)
Primary procedure
Wedge resection	3,815 (13.7%)
Segmentectomy	1,685 (6.1%)
Lobectomy	19,836 (71.2%)
Sleeve lobectomy	412 (1.5%)
Bilobectomy	980 (3.5%)
Pneumonectomy	1,116 (4.0%)

^*Missing values imputed to median by gender

Statistical Analysis

Three multivariable hierarchical logistic regression models were created to estimate the association of patient baseline characteristics with the primary outcome measures of mortality, morbidity and composite mortality and morbidity. A hierarchical model with participant specific random effects was utilized to account for potential dependence between patient outcomes within a participant. The composite outcome was defined as having either mortality or at least one major morbidity. All covariates were retained in the models. Model discrimination was assessed by examining the area under the receiver operator curve (C-statistic). Model calibration was assessed with the Hosmer-Lemeshow goodness-of-fit test.

Additionally, we examined variation in hospital performance for the composite outcome of mortality or major morbidity. The same hierarchical model as above was utilized but the Bayesian approach facilitated computation of a standardized incidence ratio (SIR) for each hospital (participant), along with 95% Bayesian credible intervals. SIRs summarize participant performance variation, as previously described.^4,6 The SIR is the ratio between the participant’s risk-adjusted rate and the risk-adjusted rate of a hypothetical average participant. A SIR greater than 1.0 is consistent with a higher risk-adjusted mortality or major morbidity in comparison with an average participant. Analyses were performed using SAS 9.4 statistical package utilizing the GLIMMIX and MCMC modules.

Results

A query of the STS-GTSD from January 1, 2012 through December 31, 2014 revealed 27,844 patients having undergone surgery for primary lung cancer from 231 centers. Baseline patient characteristics are depicted in Table 1. The majority of patients were Caucasian (87.0%) with a past or current history of smoking (86.0%), a Zubrod performance status of 0 or 1 (95.8%; 26,678/27,844), and an American Society of Anesthesiologists (ASA) rating of 3 (75.3%; 20,953/27,844). Over half of pulmonary resections for lung cancer were performed via thoracoscopy (61.6%).

Rates of individual major morbidities in lung cancer surgery patients are show in Table 2. In addition, the rates of other important postoperative complications are shown as well. The operative mortality rate was 1.4% (n=401), major morbidity rate was 9.1% (n=2,545) and the composite major morbidity or mortality rate was 9.5% (2,654). Table 3 demonstrates these rates stratified by procedure type.

Table 2.

Frequency of Complications

Variable	No. (% of all patients)
Tracheostomy	283 (1.0%)
Reintubation	899 (3.2%)
Initial Ventilatory support > 48 Hours	148 (0.5%)
Adult Respiratory Distress Syndrome	159 (0.6%)
Bronchopleural Fistula	149 (0.5%)
Pulmonary Embolus	131 (0.5%)
Pneumonia	1,116 (4.0)
Unexpected return to operating room	1050 (3.8%)
Myocardial Infarction	92 (0.3%)
DVT requiring treatment	148 (0.5%)
Atrial Arrhythmia requiring treatment	2,974 (10.7%)
Renal Failure (RIFLE criteria)	209 (0.8%)
Blood Transfusion
Intraoperative	696 (2.5%)
Postoperative	1438 (5.2%)
Sepsis	189 (0.7%)
Chylothorax
Requiring surgical ligation	49 (0.2%)
Medical treatment only	100 (0.4%)
Recurrent Laryngeal Nerve paralysis	139 (0.5%)

Table 3.

Mortality, Major Morbidity, and Composite Mortality or Major Morbidity Rates Stratified by Procedure Type

Procedure	Mortality	Major Morbidity	Composite Mortality or Major Morbidity
Wedge	0.8% (30/3,815)	5.3% (204/3,815)	5.6% (214/3,815)
Segmentectomy	0.8% (14/1685)	6.5% (109/1685)	7.0% (118/1685)
Lobectomy	1.3% (262/19,836)	9.3% (1,852/19,836)	9.7% (1,920/19,836)
Sleeve Lobectomy	1.7% (7/412)	12.1% (50/412)	12.9% (53/412)
Bilobectomy	3.4% (33/980)	15.3% (150/980)	15.7% (154/980)
Pneumonectomy	4.9% (55/1,116)	16.1% (180/1,116)	17.5% (195/1,116)

In Table 4, the three multivariable logistic regression models demonstrating the relation of patient baseline characteristics to the outcome measures of mortality, major morbidity, and composite mortality and major morbidity are shown. The C-statistics for the models are: 0.78 for mortality, 0.68 for major morbidity and 0.68 for composite mortality and major morbidity. Significant predictors of operative mortality included age, male gender, body mass index, steroids, congestive heart failure, coronary artery disease, peripheral vascular disease, reoperation, cerebrovascular disease, forced expiratory volume in one second, induction therapy, renal dysfunction, Zubrod score, ASA rating, thoracotomy approach, tumor stage, and greater extent of pulmonary resection. For operative mortality, predictors with a large effect included a Zubrod score of 2 or greater, an ASA rating of 4 or 5, a thoracotomy operative approach, stage IV cancer, and a bilobectomy or pneumonectomy. For major morbidity cigarette smoking increases the risk of complications by 60%, while stage, renal dysfunction, congestive heart failure, and cerebrovascular disease lose significance. A large effect is observed for lobectomy, sleeve lobectomy, bilobectomy and pneumonectomy, with the risk increasing with extent of resection. Finally, for the composite mortality and morbidity model, similar predictors are observed as seen in the morbidity model.

Table 4.

Predictors of Mortality, Major Morbidity and Composite Mortality or Major Morbidity^*

Variable	Mortality Model OR (95% CI)	p	Major Morbidity Model OR (95% CI)	p	Composite Model (Mortality or Major Morbidity) OR (95% CI)	p
Age (10 year increase)	1.64 (1.44, 1.87)	<0.001	1.13 (1.08, 1.19)	<0.001	1.14 (1.08, 1.90)	<0.001
Male gender	1.54 (1.23, 1.92)	<0.001	1.39 (1.28, 1.52)	<0.001	1.41 (1.29, 1.53)	<0.001
BMI (kg/m2)		0.006		<0.001		<0.001
≥18.5 to <25	1.00		1.00		1.00
≥6.0 to <18.5	1.44 (0.85, 2.44)		1.33 (1.07, 1.65)		1.35 (1.09, 1.66)
≥25.0 to <30.0	0.96 (0.75, 1.22)		0.83 (0.75, 0.91)		0.83 (0.75, 0.92)
≥30.0 to <35.0	0.61 (0.43, 0.85)		0.72 (0.64, 0.82)		0.72 (0.63, 0.82)
≥35.0 to ≤99.9	1.17 (0.82, 1.67)		0.81 (0.69, 0.96)		0.83 (0.71, 0.97)
Hypertension	0.93 (0.73, 1.17)	0.51	1.08 (0.98, 1.19)	0.12	1.06 (0.96, 1.16)	0.25
Steroids	1.72 (1.14, 2.60)	0.01	1.28 (1.05, 1.57)	0.017	1.33 (1.09, 1.62)	0.005
CHF	1.51 (1.01, 2.25)	0.046	1.17 (0.95, 1.44)	0.15	1.19 (0.97, 1.46)	0.10
CAD	1.32 (1.05, 1.67)	0.019	1.13 (1.02, 1.25)	0.022	1.14 (1.03, 1.26)	0.011
PVD	1.49 (1.13, 1.96)	0.005	1.43 (1.26, 1.62)	<0.001	1.43 (1.26, 1.63)	<0.001
Reoperation	1.38 (1.00, 1.94)	0.052	1.35 (1.16, 1.58)	<0.001	1.32 (1.13, 1.54)	<0.001
Cerebrovascular disease	1.42 (1.05, 1.90)	0.021	1.08 (0.94, 1.24)	0.29	1.11 (0.97, 1.28)	0.14
Diabetes	1.08 (0.85, 1.39)	0.53	1.01 (0.90, 1.12)	0.93	1.01 (0.91, 1.13)	0.84
% FEV1 (10% decrease)	1.07 (1.01, 1.12)	0.02	1.13 (1.10, 1.15)	<0.001	1.12 (1.10, 1.15)	<0.001
Induction therapy	1.51 (1.09, 2.10)	0.014	1.20 (1.02, 1.40)	0.024	1.20 (1.03, 1.39)	0.022
Renal dysfunction	1.74 (1.06, 2.86)	0.029	1.07 (0.81, 1.42)	0.64	1.11 (0.84, 1.46)	0.47
Cigarette smoking		0.14		<0.001		<0.001
Never	1.00		1.00		1.00
Past smoker	1.54 (1.00, 2.38)		1.20 (1.02, 1.41)		1.23 (1.05, 1.44)
Current smoker	1.54 (0.96, 2.49)		1.64 (1.38, 1.94)		1.64 (1.38, 1.94)
Zubrod score		<0.001		<0.001		<0.001
0	1.00		1.00		1.00
1	1.60 (1.25, 2.04)		1.14 (1.04, 1.25)		1.16 (1.06, 1.28)
2-5	2.21 (1.45, 3.37)		1.57 (1.29, 1.91)		1.60 (1.32, 1.95)
ASA		0.007		<0.001		<0.001
1 or 2	1.00		1.00		1.00
3	1.67 (1.05, 2.65)		1.25 (1.08, 1.45)		1.27 (1.09, 1.47)
4 or 5	2.26 (1.34, 3.80)		1.72 (1.42, 2.09)		1.76 (1.45, 2.13)
Approach		<0.001		<0.001		<0.001
MIS	1.00		1.00		1.00
Thoracotomy	1.87 (1.49, 2.36)		1.49 (1.35, 1.64)		1.51 (1.37, 1.66)
Pathologic stage		0.008		0.30		0.25
I	1.00		1.00		1.00
II	1.15 (0.89, 1.48)		1.07 (0.96, 1.19)		1.05 (0.95, 1.17)
III	1.46 (1.10, 1.96)		1.13 (0.99, 1.29)		1.14 (1.00, 1.30)
IV	2.23 (1.23, 4.02)		1.01 (0.73, 1.40)		1.04 (0.75, 1.42)
Procedure		<0.001		<0.001		<0.001
Wedge	1.00		1.00		1.00
Segmentectomy	0.98 (0.51, 1.88)		1.19 (0.93, 1.53)		1.24 (0.97, 1.57)
Lobectomy	1.69 (1.14, 2.53)		1.96 (1.67, 2.30)		1.93 (1.65, 2.26)
Sleeve	1.72 (0.72, 4.09)		1.93 (1.36, 2.75)		1.96 (1.39, 2.77)
Bilobectomy	3.57 (2.09, 6.12)		2.98 (2.34, 3.80)		2.91 (2.29, 3.70)
Pneumonectomy	4.80 (2.87, 8.02)		2.74 (2.15, 3.48)		2.83 (2.24, 3.58)
C statistic	0.78		0.68		0.68

ASA = American Society of Anesthesiologists; BMI = body mass index; CAD = coronary artery disease; CHF = congestive heart failure; CI = confidence interval; FEV₁ = force expiratory volume in first second of expiration; MIS = minimally invasive surgery; OR = odds ratio; PVD = peripheral vascular disease.

^*Intercept values for the models are -10.822 for Mortality, -5.651 for Major Morbidity, and -5.657 for Composite, respectively. Covariate specific coefficients can be obtained by taking natural logarithm of the odds ratios.

SIRs with 95% Bayesian credible intervals for the composite measure of mortality and major morbidity for all 231 hospitals are shown in Figure 1. There is no overlap in credible intervals between some of the best (3.5%=8/231 sites with upper limit below 1) and worst performing sites (6.9%=16/231 sites with lower limit above 1), indicating that this model provides meaningful discrimination between best and worst performers.

Figure 1.

Hospital performance variability. The standardized incidence ratios with 95% Bayesian credible intervals are shown for the composite measure of mortality and major morbidity following lung cancer resection among STS participating centers.

Comment

Surgeons submitting data to the STS GTSD perform surgical resection for lung cancer with low mortality and morbidity. Important predictors of mortality and major morbidity following lung cancer resection are identified with these models. Knowledge of such predictors informs clinical decision making by allowing physicians and patients to focus on individual patient characteristics and their impact on outcomes. These models replace prior versions of the lung cancer resection risk models.⁵ The STS will utilize these models to provide risk-adjusted outcomes for lung cancer resection with respect to: operative mortality, major morbidity, and composite mortality or morbidity. Centers will continue to receive feedback on their results, and centers of excellence and underperformers will be identified. Without knowledge of such outcomes, true quality improvement cannot occur.

Operative mortality in the GTSD has decreased from 2.2% in the years 2002-2008 to 1.4% in 2012-2014.⁵ These data represent the highest quality lung cancer surgery in the United States. Kozower and colleagues have previously demonstrated that compared with Nationwide Inpatient Sample (NIS) database, from 2002-2008 patients in the GTSD had lower unadjusted discharge mortality rates, median length of stay, and pulmonary complication rates for lobectomy.⁹ The major morbidity rate has increased from 8.6% to 9.1% during the same time. A potential explanation for this observation is more complete coding of complications by data abstractors as the result of education efforts from the STS, as well as inclusion of unexpected return to the operating room for any reason instead of only for bleeding.

We identified several predictors of operative mortality following lung cancer resection. The Zubrod performance score continues to be a strong predictor of mortality, and has demonstrated good predictive ability in other settings.¹⁰ ASA rating, a surrogate for patient comorbidities, also continues to predict mortality. Zubrod score and ASA also predict major morbidity and the composite mortality or major morbidity outcome. Coronary artery disease and peripheral vascular disease were predictors of all three outcome measures, whereas cerebrovascular disease and renal dysfunction were predictors of mortality. All are markers of systemic atherosclerosis and their importance is intuitive.

Operative approach had a significant effect on all three models, with a thoracotomy approach predicting worse outcomes. The utilization of thoracosocpy increased from 36.9% in the first risk model up to 61.6% in the current analysis. Such a finding is expected, as Paul and colleagues have demonstrated decreased morbidity with thoracoscopy compared to thoracotomy approaches to lobectomy in an analysis of GTSD data.¹¹ Further, as previously identified, the extent of pulmonary resection was found to be predictive of adverse events. In comparison to a non-anatomic wedge resection, lobectomy, sleeve lobectomy, bilobectomy and pneumonectomy were all significant predictors; sleeve resections, bilobectomies and pneumonectomies had particularly large effects. This is consistent with a GTSD review of 1,267 pneumonectomies from 2002-2007 by Shapiro and colleagues that determined perioperative major morbidity occurred in 30.4% and mortality in 5.6%.¹²

Older age and male gender continue to be predictors of all three outcome measures. Male gender has consistently been identified as a negative predictor in other lung cancer surgery models.^13,14 Increasing BMI is protective against major morbidity and composite mortality and major morbidity in this analysis, whereas a very low BMI indicates higher risk. Prior GTSD analysis have found an association with higher BMI and operative times for lobectomy but not complications.¹⁵

A reoperation is a predictor of adverse outcomes in our updated models, as opposed to the last iteration of the lung cancer resection models. This may be the result of the use of a different definition. Rather than any prior cardiothoracic operation, the variable used in the present model was any prior cardiothoracic surgery that affects the operative field. The administration of induction chemotherapy and/or radiation and the use of systemic steroids were also significant predictors.

A lower percent predicted FEV₁ increased the risk of all adverse outcomes, similar to what has been previously identified. Being a current smoker was also identified as increasing risk of major morbidity and the composite measure. Mason and colleagues have examined this association in the GTSD and found that current smoking status confers a greater risk of mortality and pulmonary complications.¹⁶

There are several limitations that must be considered when interpreting these lung cancer surgery risk-adjustment models. First, due to missing data in approximately 14% of patients, DLCO was not included as a covariate for analysis.¹⁷ Next, the performance of the separate risk models varies. The Morality model has a C statistic approaching 0.8, which would indicate strong performance. However, the C statistic of the Major Morbidity and Composite models approached 0.7, which represents fair performance. Further, results from these models may not be generalizable to surgeons not participating in the GTSD.⁹ Another limitation is that the GTSD has been limited to 30 day follow-up. Outcomes such as 90 day mortality, hospital readmissions and long-term survival are of critical importance to measure.^18-20 The GTSD is beginning to collect long-term survival data following lung cancer resections. In addition, linkage of GTSD data to administrative data from the Centers for Medicare and Medicaid Services (CMS) has been established and longitudinal follow-up for individuals 65 years and older will soon be available.²¹ Also, due to missing data, pathologic stage was used as a surrogate for clinical stage in this analysis. Finally, the selection of which complications constitute a major morbidity has been done empirically. These complications were selected by the General Thoracic Surgery Database Taskforce and are based on their frequency in the data and their clinical significance.

These analyses also have strengths. As demonstrated through external audit, the data in the STS-GTSD is of very high quality.⁷ Additionally, the use of robust clinical registry data provides greater granularity than other less detailed registries or administrative datasets. Data currently used for making treatment decisions in lung cancer surgery include: pulmonary function, performance status, smoking status, weight loss and other comorbid medical conditions. None of these variables are present in the Surveillance Epidemiology and End Results (SEER), National Cancer Database (NCDB), NIS, and CMS databases or any other existing large dataset.

In conclusion, thoracic surgeons contributing to the STS GTSD perform high quality lung cancer resections. Risk models generated from this database have identified several factors that are predictive of adverse events and can be used to measure hospital performance variation. With an ever increasing experience, these models continue to be refined to guide the quality improvement efforts of thoracic surgeons across the nation.