Evaluation of adherence to the Commission on Cancer lung cancer quality measures

Abstract

Objective

In this study we present historic data on adherence to and survival outcomes associated with recently introduced quality measures for the management of non-small-cell lung cancer.

Methods

The National Cancer Data Base was queried to identify all patients with non-small-cell lung cancer from 1998 to 2011. Adherence to guidelines was assessed for each of 3 Commission on Cancer-defined quality measures: (1) sampling 10 regional lymph nodes at surgery; (2a) surgery within 120 days of neoadjuvant chemotherapy or, (2b) 180 days of adjuvant chemotherapy; and (3) nonsurgical primary therapy in cN2 disease. The likelihood of measure adherence and the association of measure adherence with all-cause mortality were analyzed controlling for patient, hospital, and time period characteristics.

Results

Regional lymph node sampling was inadequate in 72.7% of cases. Only 28.7% began adjuvant chemotherapy within 180 days of surgery. However, 96.5% of patients who received neoadjuvant chemotherapy proceeded to surgery within 120 days and surgery was first-line treatment for cN2 disease in only 3.7% of patients. Uninsured or Medicaid status was an independent risk factor for a prolonged delay between neoadjuvant chemotherapy and surgery (odds ratio, 1.36; 95% confidence interval, 1.08–1.72) and surgery and adjuvant treatment (odds ratio, 1.92; 95% confidence interval, 1.69–2.19). Overall survival was significantly better in patients whose care conformed to quality standards for nodal sampling (measure 1), and timing of chemotherapy.

Conclusions

Adherence rates for nodal sampling at the time of surgery and receipt or adjuvant chemotherapy were low. these findings highlight opportunities for improvement efforts, but more measures are needed to more broadly assess the quality of lung cancer care.

Lung cancer is the leading cause of cancer-related mortality in the United States, accounting for 158,040 deaths in 2015. Detailed lung cancer treatment guidelines have been developed by a variety of professional societies and stakeholder groups. However, adherence to principles established in these guidelines has been shown to be highly variable.^1–3 To address this variation and provide performance feedback to hospitals, the American College of Surgeons Commission on Cancer (CoC)—a group of 56 member organizations who promote high-quality cancer care through the formal accreditation of cancer programs, the independent collection of high-quality data, and feedback of performance on audited quality metrics— developed and approved 3 lung cancer quality measures in the fall of 2014 and the spring of 2015 to encourage guideline adherence: (1) at least 10 regional lymph nodes are removed and pathologically examined for American Joint Committee on Cancer (AJCC) stage IA, IB, IIA, and IIB resected non-small-cell lung cancer (NSCLC); (2) systemic chemotherapy is administered within 4 months to the day of surgery preoperatively or from day of surgery to 6 months postoperatively, or it is recommended for surgically resected cases with pathologic, lymph node-positive (pN1) and (pN2) NSCLC; and (3) surgery is not the first course of treatment for cN2, M0 lung cases. These measures are currently being used by the CoC only to provide program feedback and facilitate quality improvement. However, these metrics are likely to become a part of the accreditation reporting structure in the future. Further, little is known regarding the adherence to the principles of care outlined in the CoC quality measures before their public release. We sought to formally evaluate baseline national performance on these 3 lung cancer quality measures and examine patient, tumor, hospital, and time period characteristics associated with nonadherence to these guidelines at the time of their introduction. We also analyze all-cause mortality outcomes associated with adherence to guidelines for the relevant patient populations.

METHODS

Data Source

The National Cancer Data Base captures data on the diagnosis, staging, and treatment of all patients treated for cancer at hospitals accredited by the joint American College of Surgeons/American Cancer Society CoC. All CoC hospitals are required to participate in the National Cancer Database (NCDB) as a condition of their accreditation and data are collected from direct review of the medical records by trained clinical data abstractors. This database is estimated to capture approximately 70% of all new cancer diagnoses, including 82% of lung cancer diagnoses and includes approximately 1430 accredited cancer programs in the United States.⁴ The current study of deidentified public data was ruled exempt by the Northwestern University institutional review board.

Patients

To identify patients treated for NSCLC, the NCDB was queried from January 1, 1998 to December 31, 2011. Patients with recurrent cancer, or possible metastatic disease from an extrathoracic primary tumor were excluded from the analysis because these groups were believed to represent a biologically distinct disease. Lung cancer stage was determined according to the AJCC Sixth Edition Cancer Staging Manual.⁵ Preference was given to the pathological stage when discrepancies were noted between clinical and pathologic stage determinations with the exception of measure 3, for which cN2 status was used preferentially for inclusion. The specific inclusion and exclusion criteria differed according to the measure being studied and were applied according to published CoC definitions for each measure.⁶ Patient demographic variables included age, sex, Zip Code median income and education quartiles, geographic region in which care was received, and patient travel distance to the treating center. Disease characteristics examined included tumor size, nodal involvement, and the presence of extrathoracic disease. The treating facility type, academic versus community hospital, was also assessed. Academic hospitals are defined by the CoC as having ≥500 newly diagnosed cancer cases per year and offering graduate medical education programs in >4 disciplines.

Measures

Three discrete quality measures in NSCLC have been endorsed by the CoC.⁷ The first 2 measures were released in the fall of 2014 with the third measure being included in a subsequent update in the spring of 2015. Measure 1 requires that “at least 10 regional lymph nodes are removed and pathologically examined for AJCC stage IA, IB, IIA, and IIB resected NSCLC” for compliance. Coordination of surgical and medical oncologic care is the focus of measure 2, which assesses whether “systemic chemotherapy is administered within 4 months to day preoperatively or day of surgery to 6 months postoperatively, or it is recommended for surgically resected cases with pathologic, lymph node-positive (pN1) and (pN2) NSCLC.” Measure 3 examines the appropriate use of surgery in locally advanced disease states, as an operation should “not [be] the first course of treatment for cN2, M0” clinical IIIA patients. Because the denominators are different for each measure, we performed the analysis of adherence in 4 separate groups, dividing measure 2 into 2 subgroups: (a) surgery after neoadjuvant chemotherapy; and (b) referral for adjuvant chemotherapy. Inclusion and exclusion criteria defined by the CoC were used for the analysis. Care was deemed to be adherent if all specifications of the measure were met.

Analysis

Adherence was calculated for each measure by determining the percentage of patients who received guideline-adherent care. Patients for whom all necessary lung cancer inclusion criteria were not documented were excluded from the analysis. Twenty-four percent of all NCDB lung tumor diagnoses were histologic types other than NSCLC (eg, carcinoid tumors, small-cell lung cancer, metastatic disease) and were excluded. Outcomes for all measures were considered to be binary (pass/fail) on the basis of the definition structure provided by the CoC. Baseline study population characteristics were described for each measure. Univariate analyses were performed to assess the association of patient, hospital, and time level factors on measure adherence with statistical significance determined using χ² tests. To assess the simultaneous association of patient (sociodemographic), tumor (clinical), hospital, and time period factors on measure adherence, a multivariable logistic regression model was estimated with standard errors adjusted for clustering of data within hospitals. Models were estimated on the basis of inclusion of all variables with a potential, a priori effect on adherence and survival including region, age group, sex, tumor size, cancer stage, insurance status (Medicaid or uninsured vs other insurance), patient Zip Code quartile of income and education, hospital type, and distance from the patient’s residence to the treating facility. Survival as a function of measure adherence was calculated for each measure subpopulation for patients diagnosed earlier than 2007, the last year for which NCDB survival data were available.

Cox proportional hazards analysis was used to test the association of measure adherence with survival time controlled for the same variables. Analyses were performed using STATA version 14 software (Stata Corp, College Station, Tex).

RESULTS

During the study period, a total of 1,743,534 patients diagnosed at 1245 hospitals were examined, of whom 1,323,095 were diagnosed with NSCLC. The number of patients eligible for evaluation under each measure ranged from 386,866 (lymph node sampling) to 32,098 (neoadjuvant chemotherapy) because of differences in the inclusion criteria for each measure (Figure 1).

FIGURE 1.

Consort flow diagram of patients included in the analysis of each measure. NCDB, National Cancer Database; NSCLC, non-small-cell lung cancer.

Measure 1: Surgical Lymph Node Staging

Adequate surgical lymph node staging as defined by the measure was performed in 27.3% of patients, whereas 72.7% had < 10 lymph nodes sampled at the time of lung resection (Figure 2). Those with a smaller tumor, earlier stage, and at the extremes of age were more likely to receive nonadherent treatment (Table 1). In multiple logistic regression analysis, Hispanic individuals (odds ratio [OR], 1.09; 95% confidence interval [CI], 1.00–1.20) and African American individuals (OR, 1.22; 95% CI, 1.07–1.20) were more likely to have inadequate nodal sampling than white individuals because those patients were from lower Zip Code education quartiles or had stage IV or unknown stage disease. Treatment in an academic versus community medical center was associated with adequate nodal sampling (OR, 0.56; 95% CI, 0.50–0.63). Measure adherence increased in every time period compared with the 1998 to 2000 baseline period, with a 45% lower odds of failure in 2010 to 2011. Survival was significantly worse for patients with measure failure (hazard ratio [HR], 1.18; 95% CI, 1.16–1.20) after controlling for patient, hospital, and time period characteristics.

FIGURE 2.

National adherence to each of 3 Commission on Cancer (CoC) measures of quality in lung cancer care. Because of differing inclusion criteria, measure 2 is depicted in 2 parts (a and b). Graph depicts percentage of patients found to meet criteria for each measure.

TABLE 1.

Multivariable logistic regression predictors of adherence to CoC Quality Measure 1 (sampling >10 regional lymph nodes) and influenceon all-cause mortality as predicted by Cox proportional hazards modeling

	>10 Lymph nodes sampled at resection (n = 386,866)
	Sample percentage	Failure percentage	Failure proportion P value	Odds ratio for failure (95% CI)	Hazard ratio for failure (95% CI)
Sex			<.0001
Male	52.76	71.94		0.96 (0.94–0.981	1.25 (1.24–1.27)
Female	47.24	73.61		Reference	Reference
Age, y			<.0001
<40	0.57	75.40		1.28 (1.14–1.43)	0.82 (0.75–0.89)
40–49	4.92	71.70		1.04 (1.01–1.08)	0.87 (0.85–0.90)
50–59	16.34	70.73		Reference	Reference
60–69	33.05	71.83		1.04 (1.02–1.07)	1.28 (1.1.25–1.30)
70–79	35.53	73.59		1.13 (1.10–1.16)	1.75 (1.71–1.78)
≥80	9.59	76.85		1.42 (1.36–1.47)	2.39 (2.33–2.44)
Race and ethnicity			<.0001
Non-Hispanic white	87.09	72.62		Reference	Reference
Non-Hispanic black	8.13	74.69		1.22 (1.15–1.28)	1.01 (0.99–1.04)
Hispanic	2.08	73.60		1.09 (1.00–1.20)	0.92 (0.88–0.95)
Asian	1.54	70.15		1.01 (0.92–1.12)	0.80 (0.77–0.84)
Other/unknown	1.17	69.86		0.95 (0.82–1.10)	0.93 (0.88–0.98)
Zip Code income quartile			<.0001
Lowest quartile	14.12	74.62		1.01 (0.90–1.14)	1.09 (1.07–1,12)
Second quartile	19.00	73.86		1.00 (0.91–1.09)	1.09 (1.06–1.11)
Third quartile	27.07	72.56		0.97 (0.91–1.03)	1.05 (1.03–1.07)
Highest quartile	34.69	71.50		Reference	Reference
Other/unknown	5.13	72.96		1.46 (0.59–3.61)	0.66 (0.42–1.03)
Zip Code education quartile			<.0001
Lowest quartile	16.50	74.31		1.12 (1.00–1.25)	1.09 (1.06–1.13)
Second quartile	23.91	73.54		1.09 (1.01–1.16)	1.08 (1,06–1.10)
Third quartile	23.53	72.39		1.03 (1.00–1.07)	1.06 (1.04–1.07)
Highest quartile	30.92	71.54		Reference	Reference
Other/unknown	5.14	72.94		0.71 (0.29–1.73	1.66 (1.06–2.61)
Insurance status			<.0001
Medicaid or not insured	94.42	72.79		1.03 (0.99–1.08)	1.19 (1.16–1.22)
Other types of insurance	5.58	71.70		Reference	Reference
Clinical stage			<.0001
0	0.20	77.78		0.96 (0.79–1.17)	0.99 8 (0.86–1.13)
I	61.21	76.04		Reference	Reference
II	15.09	63.47		0.64 (0.62–0.65)	1.55 (1.53–1.57)
III	14.18	65.48		0.67 (0.65–0.69)	2.10 (2.06–2.12)
IV	6.21	83.81		1.66 (1.57–1.76)	3.52 (3.42–3.62)
Other/unknown	3.12	79.79		1.22 (1.10–1.35)	1.43 (1.38–1.49)
Tumor size, cm			<.0001
≤1	4.58	89.24		Reference	Reference
2	6.11	81.46		0.79 (0.75–0.82)	1.03 (1.00–1.06)
3	28.24	78.20		0.62 (0.59–0.65)	1.13 (1.10–1.16)
5	24.62	73,93		0.48 (0.45–0.50)	1.25 (1.22–1.29)
7	23.67	68.11		0.40 (0.38–0.42)	1.41 (1.36–1.45)
>8	8.13	62.86		0.37 (0.35–0.39)	1.60 (1.53–1.67)
Other/unknown	4.66	59.93		1.67 (1.52–1.83)	1.53 (1.46–1.61)
Hospital location			<.0001
Large urban region	45.66	71.83		Reference	Reference
Medium urban region	19.03	72.95		1.04 (0.95–1.15)	1.03 (1.01–1.06)
Small urban region	27.37	74.09		1.10 (0.98–1.23)	1.03 (1.01–1.05)
Rural region	2.24	73.11		1.03 (0.88–1.20)	1.01 (0.97–1.06)
Other/unknown	5.70	72.81		1.09 (1.00–1.20)	0.96 (0.92–1.00)
Type of hospital			<.0001
Community	69.40	76.67		Reference	Reference
Academic	30.60	65.14		0.56 (0.50–0.63)	0.93 (0.90–0.96)
Travel distance			<.0001
>5 miles	29.55	75.16		0.91 (0.88–0.95)	0.99 (0.98–1.00)
<5 miles	70.45	71.76		Reference	Reference
Time period*			<.0001
1998–2000	21.39	78.08		Reference	Reference
2001–2003	21.39	75.99		0.87 (0.82–0.91)	0.97 (0.95–0.98)
2004–2006	21.78	72.35		0.70 (0.65–0.75)	0.90 (0.88–0.91)
2007–2009	21.83	69.39		0.59 (0.55–0.64)	Omitted*
2010–2011	13.62	66.78		0.55 (0.50–0.60)	Omitted*
>10 Lymph nodes sampled at resection					1.18 (1.16–1.20)

CI, Confidence interval.

^*NCDB survival data only available for patients diagnosed from 1998–2006. Survival analysis n = 240,680.

Measure 2a: Timing of Surgery After Neoadjuvant Chemotherapy

Patients who received preoperative chemotherapy were successfully brought to surgery within the suggested 120-day window defined in the measure 96.3% of the time. Patients from lower Zip Code education quartiles or who were uninsured or covered by Medicaid had a higher likelihood of failure (Table 2). Again, the likelihood of failure significantly decreased in more recent periods. Survival was significantly worse for patients with measure failure (HR, 1.17; 95% CI, 1.07–1.27) after controlling for all Cox model covariates.

TABLE 2.

Multivariable logistic regression predictors of adherence to CoC Quality Measure 2a (surgical resection within 120 days of neoadjuvant chemotherapy) and influence on all-cause mortality as predicted by Cox proportional hazards modeling

	Surgery within 120 days of neoadjuvant chemotherapy n = 32,098
	Mean sample percentage	Failure percentage	Failure proportion P value	Odds ratio for failure (95% CI)	Hazard ratio for failure (95% CI)
Sex			.35
Male	52.22	3.62		1.02 (0.90–1.16)	1.16 (1.11–1.20)
Female	47.78	3.42		Reference	Reference
Age, y			.004
<40	0.62	2.64		0.74 (0.34–1.57)	0.83 (0.68–1.02)
40–49	6.54	3.10		0.95 (0.73–1.24)	0.90 (0.83–0.98)
50–59	21.31	3.17		Reference	Reference
60–69	37.14	3.42		1.16 (0.99–1.36)	1.15 (1.09–1.21)
70–79	29.89	3.94		1.40(1.19–1.67)	1.44 (1.37–1.52)
≥80	4.51	5.14		1.98 (1.43–2.74)	1.83 (1.63–2.05)
Race and ethnicity			.02
Non-Hispanic white	84.70	3.47		Reference	Reference
Non-Hispanic black	10.88	4.29		1.16 (0.93–1.44)	1.03 (0.96–1.10)
Hispanic	1.68	2.71		0.70 (0.44–1.11)	0.94 (0.81–1.10)
Asian	2.30	4.39		1.25 (0.84–1.84)	0.89 (0.77–1.02)
Other/unknown	0.44	1.55		0.44 (0.18–1.07)	1.01 (0.82–1.24)
Zip Code income quartile			.45
Lowest quartile	14.50	4.02		1.07 (0.81–1.43)	1.08 (0.99–1.18)
Second quartile	18.13	3.47		0.94 (0.74–1.18)	1.05 (0.98–1.13)
Third quartile	26.26	3,36		0.90 (0.75–1.08)	1.04 (0.99–1.10)
Fourth quartile	35.90	3.49		Reference	Reference
Other/unknown	5.22	3.55		1.01 (0.77–1.34)	1.04 (0.90–1.20)
Zip Code education quartile			.10
Lowest quartile	16.18	3.77		1.23 (0.95–1.58)	1.07 (0.98–1.18)
Second quartile	24.14	3.65		1.27 (1.04–1.55)	1.05 (0.98–1.12)
Third quartile	25.82	3.79		1.32 (1.10–1.58)	1.02 (0.97–1.08)
Fourth quartile	28.65	3.12		Reference	Reference
Other/unknown	5.22	3.55		0.00 (0.00–0.00)	1.12 (0.62–3.40)
Insurance status			.02
Medicaid or uninsured	91.42	3.46		1.42 (1.12–1.79)	1.07 (0.99–1.16)
Other insurance	8.58	4.38		Reference	Reference
Clinical stage			.04
0	0.00	0.00		Omitted†	1.89 (0.71–5.08)
I	0.80	4.19		Reference	Reference
II	47.04	3.82		0.83 (0.42–1.65)	0.78 (0.57–1.07)
III	49.25	3.24		0.66 (0.33–1.30)	1.15 (0.84–1.56)
IV	1.33	3.68		0.85 (0.36–1.98)	1.77 (1.17–2.67)
Other/unknown	1.59	5.34		1.23 (0.54–2.80)	0.98 (0.69–1.40)
Tumor size, cm			.23
≤1	2.48	4.10		Reference	Reference
2	2.57	3.53		0.96 (0.65–1.42)	1.04 (0.91–1.18)
3	18.14	3.48		0.85 (0.58–1.25)	1.06 (0.94–1.21)
5	21.68	3.18		0.92 (0.63–1.35)	1.18 (1.03–1.35)
7	31.50	3.48		1.01 (0.60–1.51)	1.23 (1.05–1.37)
>8	13.89	3.77		1.17 (0.77–1.79)	1.59 (1.38–1.85)
Other/unknown	9.73	4.24		1.00 (0.57–1.73)	1.34 (1.11–1.61)
Type of hospital			.07
Large urban region	49.78	3.83		Reference	Reference
Medium urban region	17.77	3.24		0.84 (0.71–1.00)	1.04 (0.99–1.10)
Small urban region	25.38	3.35		0.86 (0.72–1.01)	1.03 (0.98–1.09)
Rural region	1.59	2.49		0.61 (0.37–1.00)	1.02 (0.89–1.16)
Other/unknown	5.48	3.29		0.76 (0.52–1.11)	1.00 (0.88–1.15 )
Type of hospital			.06
Community	64.63	3.39		Reference	Reference
Academic	35.37	3.81		1.12 (0.97–1.30)	0.88 (0.83–0.92)
Patient travel distance, miles			.44
>5	29.89	3.65		1.01 (0.88–1.16)	0.97 (0.93–1.01)
<5	70.11	3.47		Reference	Reference
Time period*			<.001
1998–2000	14.41	4.57		Reference	Reference
2001–2003	18.39	4.95		1.09 (0.87–1.35)	1.00 (0.94–1.05 )
2004–2006	24.49	3.27		0.66 (0.54–0.81)	0.81 (0.77–0.85)
2007–2009	24.93	3.01		0.60 (0.49–0.75 )	Omitted*
2010–2011	17.77	3.10		0.62 (0.50–0.79)	Omitted*
Surgery within 120 days of neoadjuvant chemotherapy					1.17 (1.07–1.27)

CI, Confidence interval.

^*NCDB survival data only available for patients diagnosed from 1998 to 2006. Survival analysis n = 16,237.

^†Stage 0 predicted failure perfectly,7 observations deleted.

Measure 2b: Referral for Adjuvant Treatment After Resection

Overall, 66.5% or patients who met inclusion criteria either were not referred or did not receive chemotherapy within 6 months of the date of their operation. Male and older patients, especially the oldest old, were significantly more likely to experience measure failure; patients age 70 to 79 years were 2.3 times more likely to experience measure failure than patients age 50 to 59 years (Table 3). Patients from lower Zip Code education quartiles and those with Medicaid or who were uninsured were also more likely to either not be referred, or have a delay in referral. A travel distance to the treating center >5 miles was also predictive of delay or nonreferral (OR, 1.07; 95% CI, 1.01–1.12). Similar to the pattern observed in previous measures, measure adherence significantly increased from the earliest study period and the number of patients categorized as “unknown whether a chemotherapeutic agent(s) was administered” decreased from 31.7% to 15.6%. Overall survival was lower for patients who did not received measure-adherent care (HR, 1.41; 95% CI, 1.37–1.44).

TABLE 3.

Multivariable logistic regression predictors of adherence to CoC Quality Measure 2b (referral for adjuvant chemotherapy with 180 days of resection for patients with pN1 or pN2 disease at the time of surgery) and influence on all-cause mortality as predicted by Cox proportional hazards modeling

	Adjuvant chemotherapy within 180 days of surgery n = 109,625
	Sample percentage	Failure percentage	Failure proportion P value	Odds ratio for failure (95% CI)	Hazard ratio for failure (95% CI)
Sex			<.001
Male	55.45	68.25		1.10 (1.07–1.141	1.19 (1.18–1.22)
Female	44.55	64.32		Reference	Reference
Age, y			<.001
<40	0.54	54.53		0.68 (0.57–0.81)	0.86 (0.78–0.97)
40–49	4.96	55.54		0.85 (0.80–0.91)	0.92 (0.88–0.97)
50–59	16.91	57.86		Reference	Reference
60–69	32.03	62.62		1.34 (1.29–1.39)	1.20 (1.17–1.23)
70–79	34.87	72.81		2.30 (2.20–2.40)	1.52 (1.48–1.56)
≥80	10.68	87.73		7.09 (6.55–7.68)	1.99 (1.92–2.07)
Race and ethnicity			.003
Non-Hispanic white	85.94	66.47		Reference	Reference
Non-Hispanic black	8.72	66.32		1.06 (0.99–1.14)	1.00 (0.96–1.03)
Hispanic	2.26	67.35		1.06 (0.94–1.18):	0.97 (0.91–1.03)
Asian	1.75	64.66		1.06 (0.91–1.24)	0.80 (0.76–0.85)
Other/unknown	1.33	70.87		1.34 (1.17–1.53)	0.93 (0.86–1.02)
Zip Code income quartile			<.001
Lowest quartile	14.90	70.29		1.09 (0.97–1.22)	1.06 (1.01–1.10)
Second quartile	19.24	67.81		1.03 (0.94–1.13)	1.02 (0.98–1.05)
Third quartile	27.08	66.30		1.02 (0.95–1.09)	1.04 (1.02–1.07)
Fourth quartile	34.00	64.74		Reference	Reference
Other/unknown	4.78	64.17		0.72 (0.16–3.34)	0.94 (0.37–2.76)
Zip Code education quartile			<.001
Lowest quartile	17.67	70.19		1.28 (1.16–1.41)	1.08 (1.04–1.12)
Second quartile	24.21	67.54		1.15 (1.07–1.23)	1.07 (1.04–1.11)
Third quartile	23.04	65.54		1.05 (1.00–1.10)	1.04 (1.01–1.07)
Fourth quartile	30.29	64.80		Reference	Reference
Other/unknown	4.79	64.19		1.47 (0.32–6.77)	1.10 (0.32–3.23)
Insurance status			.52
Medicaid or uninsured	93.29	66.52		1.37 (1.27–1.48)	1.13 (1.09–1.18)
Other insurance	6.71	66.16		Reference	Reference
Clinical stage			<.001
0	0.03	70.37		0.84 (0.39–1.79)	1.35 (0.71–2.54)
I	0.69	67.20		Reference	Reference
II	40.71	66.02		0.76 (0.65–0.88)	0.95 (0.84–1.07)
III	51.53	64.86		0.70 (0.60–0.82)	1.48 (1.31–1.67)
IV	4.92	88.64		4.35 (3.56–5.32)	3.25 (2.80–3.76)
Other/unknown	2.12	79.84		1.33 (1.08–1.65)	1.49 (1.30–1.70)
Tumor size, cm			<.001
≤1	10.89	87.84		Reference	Reference
2	2.16	62.44		0.90 (0.81–0.99)	1.00 (0.94–1.06)
3	13.50	60.32		0.96 (0.87–1.06)	1.06 (1.00–1.12)
5	20.50	63.61		1.01 (0.92–1.11)	1.17 (1.09–1.24)
7	30.51	65.54		1.09 (0.98–1.21)	1.32 (1.24–1.42)
>8	13.60	67.05		1.16 (1.04–1.29)	1.50 (1.40–1.61)
Other/unknown	8.83	67.61		3.55 (3.09–4.08)	1.69 (1.57–1.82)
Hospital location			<.001
Large urban region	45.60	66.18		Reference	Reference
Medium urban region	18.92	66.11		1.00 (0.91–1.11)	1.06 (1.02–1.09)
Small urban region	27.57	67.53		1.04 (0.95–1.14)	1.04 (1.01–1.08)
Rural region	2.45	69.04		1.07 (0.92–1.24)	1.03 (0.96–1.11)
Other/unknown	5.47	64.39		1.00 (0.89–1.11)	1.01 (0.95–1.08)
Type of hospital			.01
Community	65.48	66.24		Reference	Reference
Academic	34.52	67.00		1.10 (0.96–1.27)	0.90 (0.86–0.94)
Travel distance, miles			.04
>5	29.25	66.96		1.07 (1.01–1.12)	0.96 (0.94–0.98)
<5	70.75	66.31		Reference	Reference
Time period*			<.001
1998–2000	24.96	81.06		Reference	Reference
2001–2003	23.90	77.21		0.76 (0.72–0.81)	0.96 (0.93–0.99)
2004–2006	20.77	61.10		0.32 (0.30–0.35)	0.95 (0.93–0.97)
2007–2009	19.29	57.37		0.27 (0.25–0.29)	Omitted*
2010–2011	11.08	52.73		0.20 (0.18–0.22)	Omitted*
Adjuvant chemotherapy within 180 days of surgery					1.41 (1.37–1.44)

CI, Confidence interval.

^*NCDB survival data only available for patients diagnosed from 1998–2006. Survival analysis n = 58,479.

Measure 3: Nonsurgical Treatment of cN2 Disease

Performance in identifying and medically treating patients with clinical N2 disease was generally good, with only 2.51% of patients having an operation as their initial form of treatment. Older patients and those with smaller tumor size had a higher likelihood of measure failure, but only patients with unknown tumor size had significantly higher nonadherence in multivariable analysis (Table 4). Patients whose travel distance was >5 miles also had a higher likelihood of measure failure (OR, 1.23; 95% CI, 1.14–1.32). Unlike the other measures, survival was actually better for the small group of nonadherent patients (HR, 0.80; 95% CI, 0.77–0.82).

TABLE 4.

Multivariable logistic regression predictors of adherence to CoC Quality Measure 3 and influence on all-cause mortality as predicted by Cox proportional hazards modeling

	Suirgery not used a is first line therapy for N2 disease n = 153.355
	Sampley percentage	Failure percentage	Failure proportion P value	Odds ratio for failure (95% CI)	Hazard ratio for failure (95% CI)
Sex			.40
Male	57.79	2.49		1.04 (0.97–1.11)	1.13 (1.11–1.14)
Female	42.21	2.55		Reference	Reference
Age, y			<.001
<40	0.44	1.97		0.9 (0.55–1.46)	0.93 (0.90–0.97)
40–49	5.15	2.25		1.04 (0.88–1.24)	0.93 (0.90–0.96)
50–59	14.22	2.17		Reference	Reference
60–69	31.08	2.61		1.18 (1.07–1.31)	1.18 (1.16–1.21)
70–79	38.18	2.88		1.31 (1.18–1.46)	1.53 (1.50–1.56)
≥80	11.49	1.98		0.93 (0.80–1.07)	2.06 (2.00–2.11)
Race and ethnicity			<.001
Non-Hispanic white	85.53	2.59		Reference	Reference
Non-Hispanic black	8.72	1.87		0.71 (0.63–0.82)	0.97 (0.95–1.00)
Hispanic	2.38	2.99		1.20 (0.98–1.46)	0.94 (0.89–0.99)
Asian	1.51	2.67		1.01 (0.78–1.30)	0.84 (0.78–0.89);
Other/unknown	1.21	2.75		1.03 (0.76–1.40)	0.97 (0.91–1.03)
Zip Code income quartile			.01
Lowest quartile	16.47	2.46		1.06 (0.80–1.25)	1.05 (1.02–1.10)
Second quartile	18.68	2.36		0.97 (0.84–1.12)	1.03 (1.01–1.07)
Third quartile	26.64	2.42		0.99 (0.90–1.10)	1.02(1.00–1.04)
Highest quartile	33.57	2.72		Reference	Reference
Other/unknown	4.69	2.49		0.16(0.04–0.71)	1.20 (0.49–2.96)
Zip Code education quartile			.25
Lowest quartile	18.50	2.41		0.99 (0.84–1.16)	1.07 (1.04–1.10)
Second quartile	24.72	2.46		1.01 (0.89–1.48)	1.06 (1.03–1.08)
Third quartile	23.22	2.48		1.00 (0.89–1.11)	1.05 (1.03–1.07)
Highest quartile	27.48	2.66		Reference	Reference
Other/unknown	4.70	2.51		5.80 (1.30–24.04)	0.88 (0.35–2.18)
Insurance status			<001
Medicaid or not insured	93.15	2.56		0.88 (0.78–1.02)	1.17 (1.14–1.20)
Other types of insurance	6.85	2.01		Reference	Reference
Tumor size, cm			<.001
≤1	8.28	0.83		Reference	Reference
2	3.09	4.94		0.79 (0.75–0.82)	0.94 (0.89–1.00)
3	15.47	4.98		0.62 (0.59–0.65)	0.99 (0.94–1.05);
5	19.41	3.92		0.48 (0.45–0.50)	1,09 (1.03–1.15)
7	25.29	2.88		0.40 (0.38–0.42)	1.24 (1.18–1.31)
≥8	14.12	2.27		0.37 (0.35–0.39)	1.37 (1.30–1.45)
Other/unknown	12.10	2.22		1.67 (1.52–1.83)	1.43 (1,36–1.51)
Hospital location			.002
Large urban region	47.00	2.66		Reference	Reference
Medium urban region	17.54	2.25		0.85 (0.73–0.98)	1.04 (1.02–1.07)
Small urban region	27.68	2.43		0.93 (0.82–1.05)	1.02 (1.00–1.05)
Rural region	2.62	2.74		0.99 (0.78–1.291	1.00 (0.96–1.05)
Other/unknown	5.16	2.60		0.98 (0.73–1.08)	0.99 (0.95–1.03)
Type of Hospital			<.001
Community	63.42	2.19		Reference	Reference
Academic	36.58	3.39		1.56 (1.34–1.81)	0.90 (0.88–0.93)
Travel distance, miles			<.001
>5	28.20	2.07		1.26 (1.16–1.37)	0.94 (0.922–0.95)
<5	71.80	2.74		Reference	Reference
Time period*			<.001
1998–2000	18.83	2.51		Reference	Reference
2001–2003	22.53	2.45		0.93 (0.83–1.05)	0.95 (0.94–0.97)
2004–2006	21.59	2.11		0.75 (0.65–0.85)	0.89 (0.87–0.91)
2007–2009	23.77	2.70		0.93: (0.82–1.06)	Omitted*
2010–2011	15.59	2.94		0.97 (0.83–1.12)	Omitted*
Surgery not used as first-line therapy for N2 disease					0.80 (0.77–0.82)

CI, Confidence interval.

^*NCDB survival data only available for patients diagnosed from 1998–2006. Survival analysis n = 96,229.

DISCUSSION

This analysis of practice patterns from the NCDB shows a significant variability in lung cancer care across the United States. Of the variables examined, divergence from guidelines was most pronounced in the areas related to the perioperative care of lung cancer patients. Appropriate operative lymph node evaluation was not performed in 72.7% of patients and 66.5% did not receive adjuvant chemotherapy in a timely fashion after resection. Because these are both areas in which patient care is typically either completely, or in large part, directed by the surgeon, these findings represent a direct opportunity for surgical quality improvement. In contrast, national measure adherence was much better in the presurgical identification and treatment of patients with locally advanced disease. Only 3.5% of patients failed to proceed to surgery in a timely fashion after the completion of neoadjuvant treatment. Importantly, identification of patients with N2 disease before surgery was quite good, with only 3.7% of patients undergoing surgery as the initial therapy for their disease. Our analysis shows a survival advantage for patients who receive care that adheres to quality measures. Adherence to the standards outlined in these quality measures improved over time. For measures such as referral for adjuvant chemotherapy, this likely reflected an evolution in the paradigm of care over the study time frame. Similarly, improved dissemination of lung cancer guidelines, upon which the quality measures were based, might have resulted in improved adherence over time with other measures. However, despite these trends toward improvement. significant quality gaps remained in analysis of the most recent data, corresponding to the time of measure implementation.

Lymph Node Sampling

Inadequate intraoperative nodal evaluation has also been identified as a quality gap in previous studies, resulting in understaging of disease. In a landmark 2005 report that evaluated results of an American College of Surgeons CoC survey on lung cancer practice patterns. Little and colleagues reported significant gaps in staging and evaluation.⁸ Of the nearly 12,000 patients treated surgically, only 27.1% underwent mediastinoscopy and lymph node tissue was retrieved in only 47% of patients who actually underwent surgical mediastinal staging. Further, mediastinal nodes were sampled at the time of resection in only 58% of patients who underwent an operation. A similar pattern of inadequate nodal evaluation was seen in an analysis of the Society of Thoracic Surgeons database, in which only 65% of patients had mediastinal lymph node sampling during lung resection, with merely 54% of patients having complete Tumor, Node, Metastases staging at the completion of treatment.

The extent of lymph node dissection necessary to improve oncologic outcomes remains a topic of debate and might influence the extent of adherence to this CoC measure. However, sampling an increasing number of lymph nodes has been associated with improved survival in some series.^10,11 In a randomized trial of mediastinal lymphadenectomy at the time of lobectomy sponsored by the American College of Surgeons Oncology Group (Z0030), a median of 18 lymph nodes were sampled, with the number of nodes examined ranging from 1 to 72.¹² However, the performance of a complete lymphadenectomy did not in and of itself improve the primary end point of overall survival in the trial.¹³ The approach to resection has also been implicated as a factor in the extent of lymph node sampling. Merritt and colleagues argued that thoracoscopic nodal sampling was less complete than an open lymph node dissection.¹⁴ The implications of this potential difference remain unresolved, although it seems unlikely that the use of thoracoscopy explains the nodal undersampling seen in this analysis because the proportion of patients who met the threshold for adequate nodal sampling increased during the same time period when thoracoscopic resection also became more common.

The choice of a simple lymph node count as a quality measure might in and of itself influence compliance. Processing of lymph node specimens is not standardized in the perioperative environment. The assignment of stage in lung cancer is more dependent on the nodal station⁵ than the overall number of nodes.¹³ In this respect, it would be theoretically possible for a surgeon to sample all relevant mediastinal stations to achieve an accurate pathologic stage as has been recommended¹⁵ while still not achieving the requisite number of lymph nodes to be considered adherent with this measure. Further, lymph nodes are frequently excised as fragments, which might potentially each be categorized as independent nodes, thereby falsely elevating the nodal count while including no new information with respect to stage. However, the independent correlation between measure adherence and survival does argue that a metric for nodal sampling is indeed reasonable as a quality measure. Numerical assessment of lymph node sampling also offers a feasible first step in measurement because this poses little to no additional documentation burden and the data are readily abstracted from current sources. However, lymph node counts do not allow for the separation of N1 and N2 nodal levels, nor do they provide the ability to assess the sampling of individual stations. Addition of these disease-specific data elements would improve the ability to provide more meaningful information on the intraoperative process of nodal evaluation.

Chemotherapy Before and After Surgery

Measure compliance was exceptionally high in the transition to surgery after neoadjuvant chemotherapy, indicating that patients with N2 disease who underwent resection typically did so in a timely fashion. However, the measure only included patients who ultimately had an operation. This inclusion criteria potentially fails to analyze patients for whom surgery was initially intended to be a component multimodality treatment but had a deviation from the original treatment plan because of complications or other factors. The measures for adjuvant and neoadjuvant chemotherapy accept a relatively long time period between treatment modalities (120 days in the neoadjuvant setting and 180 days in the adjuvant setting). Thus it is possible that there are patients who indeed did have unintended delays in either surgery or chemotherapy who would still be characterized as compliant. Conversely, the long time frame for acceptable adjuvant therapy might have allowed for the inclusion of some patients with early recurrence who were receiving salvage chemotherapy rather than planned adjuvant treatment. Either of these scenarios would result in the perception of a more positive performance on the quality measures than what actually exists.

Referral patterns for adjuvant chemotherapy after lung cancer resection have been examined in previous studies as well. Our group recently analyzed patterns of adjuvant chemotherapy delivery for patients with stage IB through IIIA NSCLC after resection within the NCDB from 2002 to 2011¹⁶ and found rates ranging from 29.3% to 58.4% throughout the study period. Although there was improvement seen dining the study period, a significant gap remained between the care provided compared with published guidelines. A recent publication from the group at Washington University examined issues specific to referral for adjuvant treatment for N1 disease after surgical resection.¹⁷ They reported that only 53% of patients received adjuvant chemotherapy of any form despite a significant survival advantage associated with this treatment strategy.

The current study provides some information on the effectiveness of these care transitions between medical oncology and surgical teams in the treatment of patients with lung cancer. These handoffs of care are accomplished with a much greater degree of reliability in the preoperative setting, in which nearly 97% of patients proceeded to surgery within the 4-month window described in the guidelines. In contrast, only 33.5% of patients were appropriately referred for adjuvant treatment after resection. A number of potential explanations for this discrepancy might be entertained. Patients might deal with protracted recovery periods or become too ill after resection to proceed with additional therapy.¹⁷ Further, some patients might decline further treatment for any number of individual reasons.¹⁶ Financial and logistical concerns might play a key role, however, because patients with a lower level of income and education were less likely to be referred for adjuvant treatment. Data on referral for chemotherapy are collected by the NCDB even if the patient does not ultimately see an oncologist. However, documentation of the referral conversation itself might prove difficult to identify. For example, patients who are discussed in a tumor board setting who are and subsequently determined not to be candidates for adjuvant treatment might not have documentation indicating compliance with the measure despite the appropriate referral conversation having taken place. Access to care facilities is also likely to play a role because uninsured patients and those who were required to travel longer distances were more likely to fail the measure.

Multimodality Treatment of N2 Disease

The role of surgical treatment in IIIA disease remains an area of active research and controversy among the surgeons and medical oncologists who treat patients with lung cancer. However, there is general agreement that surgery should not be the initial approach to treatment in patients with clinically defined N2 disease, as supported by multiple studies. Several studies have shown the value of neoadjuvant treatment in reducing tumor size and facilitating resection.^18–23 In a single-center study, Rosell and colleagues²⁴ showed a significant survival advantage with neoadjuvant chemotherapy followed by surgery compared with surgery alone (26 months vs 8 months). Roth and colleagues²⁵ subsequently showed an even more pronounced survival advantage, reporting a median survival of 64 months in patients who received neoadjuvant chemotherapy compared with 8 months for stage IIIA patients treated with surgery alone.

Interestingly, stage IIIA (N2) patients who underwent primary surgical treatment actually had an improved HR for survival in this analysis. Exploratory analysis confirmed that this was not a function of clinical stage misclassification. Although this finding is unexpected, the data should be interpreted with extreme care because the cohort of patients treated with primary surgery was very highly selected. The overwhelming majority (97.5%) of patients were treated with some form of induction therapy before resection. Because of the small number of surgically treated patients relative to the remainder of the cohort, improved survival in even a small number of patients might drive this finding. Further, the 2.5% who had primary operative therapy are unlikely to be representative of most stage IIIA patients. It is possible that either tumor or patient factors that cannot be ascertained from the available NCDB data elements influenced patient survival in this group more greatly than measure adherence.

Limitation

Although this analysis of the NCDB data identified gaps in historical lung cancer care guideline adherence, we were not able to assess all factors that might drive poor performance through the available data. The intent of this study was to describe the baseline state of lung cancer care at the time of measure introduction. Although in the analysis we examined patient and hospital level factors that were associated with measure compliance, it is not possible to control for selection bias in treatment allocation, which might influence outcome measures such as survival. Further, the period of the data analysis extended from 1997 to 2012, encompassing the most current available data from the NCDB. Because the lung cancer quality measures were not released by the CoC until 2014, the data reported herein do not reflect the effect of the measures, but rather describe the prevailing care environment at the time of their introduction, thus representing a reasonable baseline assessment. We have incorporated an era variable into the analysis to adjust for the change in practice pattern over time and adherence to the quality measures increased in more recent years. Follow-up studies of measure compliance for patients treated subsequent to 2014 will be needed to assess the effect of measure publication on lung cancer care. However, publication of the CoC measures did not represent new knowledge, because the measures were themselves on the basis of well established clinical practice guidelines^26–29 that should have driven care at that time.

It is possible that the observed divergence from guidelines might be a byproduct of a knowledge deficit on the part of either patient or provider. In this circumstance, educational initiatives and the development of more expertise on the part of the provider might be necessary. Indeed, Farjah and colleagues showed that outcomes after lung cancer resection were improved when care was delivered by a general thoracic surgeon,³⁰ but their analysis did not specifically examine differences in either nodal sampling rates or referral for adjuvant treatment. Further, the delivery of guideline-adherent care, examined according to the CoC measures, often relies on the interaction of multiple providers, which might further dilute the effect of an individual knowledge gap. Timely delivery of chemotherapy proved to be a failure point in performance according to measure 2b, but the NCDB analysis does not allow for the attribution of care decisions to an individual practitioner. Hence, we cannot posit whether the difficulties referring patients for adjuvant chemotherapy are due to a lack of referral on the part of the surgeon, lack of availability by the medical oncologist, or a lack of willingness on the part of the patient. Indeed, elements of all of these factors might contribute to poor adherence. The areas in which measure adherence was poor might represent gaps in appropriate cancer care, which can be targets for improvement efforts.

CONCLUSIONS

To target interventions to improve adherence to quality measures in lung cancer care, and more broadly, to practice guidelines, further analysis and understanding of barriers to appropriate care at the hospital level coupled with the development of measures that track process of care will likely be necessary. Understanding areas of poor performance in cancer care delivery allows for targeted interventions that might improve patient outcomes. In this retrospective analysis, rates of appropriate nodal sampling at the time of surgery and referral for adjuvant chemotherapy fell below defined quality standards. Improvement efforts in these areas might improve the delivery of lung cancer care nationally. Development of robust and meaningful quality measures is needed to more accurately assess performance in lung cancer care delivery.

Central Message.

Significant historical variability is found in adherence to Commission on Cancer quality standards for lung cancer care, highlighting a need for improvement.

Perspective.

National performance on defined quality measures is highly variable, representing a meaningful opportunity to improve lung cancer care. Understanding performance in cancer care delivery allows for targeted interventions that might improve patient outcomes. Development of robust and clinically relevant quality measures is necessary to more accurately assess performance in lung cancer care delivery.

Abbreviations and Acronyms

AJCC

American Joint Committee on Cancer

CI

confidence interval

CoC

Commission on Cancer

HR

hazard ratio

NCDB

National Cancer Database

NSCLC

non-small-cell lung cancer

OR

odds ratio

Discussion

Craig Sussman. Our next paper is titled, “Concordance with Commission on Cancer Quality Measures Among Cancer Care; An Opportunity for Improvement,” presented by Dr David D. Odell from Northwestern University in Chicago.

Dr David D. Odell (Chicago, Ill). Good morning everyone. I’d like to thank the association for reaching back east a little bit and inviting me as someone who started my training on the west coast to come and join you all today. It’s been a wonderful meeting thus far and I really enjoy the opportunity to be here. We have no disclosures related to this work. I think the significance of lung cancer is a public health issue. It really does not need much introduction to this group. It’s responsible for 158,000 deaths annually and it’s the most common cause of cancer-related morbidity causing more deaths than colon, breast, and prostate cancer combined. Yet despite this, there’s a relative paucity of literature on quality in lung cancer and lung cancer care. This despite an increasing complexity of medical treatment options and a push toward centralization of care as medical systems consolidate and as payers push toward accountable care organizations, yet we don’t have much data on quality. We do have a significant body of literature to guide our care. This is from a PubMed search from just a couple of days ago. You can see over a quarter million articles have been written on the topic of lung cancer.

These have been coalesced into a series of guidelines so we have very good data to help us know what it is that we should be doing yet there is this gap between what we know, what we know we should be doing, and what we actually do. I would posit that as surgeons, it’s not only in our best interest, but it’s our responsibility to the public to take ownership of this gap and try to narrow it. Understanding our current performance really is critical in order to identify areas in which we can improve in the care of these patients. I wonder if that’s looking at this as the CoC and several folks have talked about this in prior presentations. It’s a joint program between the American College of Surgeons and the American Cancer Society and it kind of has a twofold mission. One is a development of surveillance metrics and quality standards and accreditation of cancer programs but it also is the group that oversees the NCDB. This is a database that actually is a clinical registry. It’s a data set that’s created through chart level abstraction by trained certified tumor registrars.

Overall, it encompasses approximately 70% of cancer care that’s delivered in the United States but it’s actually a little bit better in terms of its capture of lung cancer. Approximately 82% of lung cancer cases that are treated in the United States are actually included in the NCDB. In late 2014 and early 2015, the CoC released a series of lung cancer quality measures after basically a consensus panel process. There are 3 measures and I’ve sort of split them here into 4 separate categories. The first looks at regional nodal staging at the time of surgical resection, setting a threshold for quality at 10 regional lymph nodes for stage I and II lung cancer. The second measure deals with timing of chemotherapy relative to the timing of surgery within 4 months to the day of in the neoadjuvant setting and within 6 months for adjuvant therapy after surgical resection. The third measure deals with the identification of advanced stage disease stating that surgery should not be the first course of therapy for those patients with clinical N2 disease or 3A disease. The goals of the current study were twofold. First, we wanted to assess the baseline national performance on the CoC quality measures. How are we doing as a group of clinicians that treat lung cancer and second, we also wanted to assess the measures themselves in terms of reproducibility and the ability to actually discriminate quality care. We looked at the national cancer database from 1998 to 2012, which was the most recent available data set for all diagnoses of NSCLC.

Each CoC measure was defined independently using the methodology that the CoC set out as they were initially defining the measures. We assessed adherence to each of these measures and measured 2 subsets as I described before independently because the inclusion/exclusion criteria differ from measure to measure and conducted multivariant and invariant analyses of adherence. To investigate the effect of measured adherence on a hard outcome variable, we also assessed it in terms of its effect on overall survival with the multivariant model including stage and other risk factors: age, sex, treatment location, and distance of travel to the treating center to assess the specific effect of measure adherence on survival for these patients. When you look at the study cohort, it’s quite variable. There are about just over 1.7 million patients with lung cancer that were treated during the study period and the rates of inclusion for the measures varied from just under 400,000 patients who were operated on to 32,000 patients who underwent induction therapy in that group and then subsequently went on to surgery.

When we look at measure compliance overall and this is really kind of the take-home message from this talk, we saw that groups really separated into kind of 2 cohorts. One in which we had very high adherence to the measures in areas specifically targeting preoperative identification of patients. We did very well in terms of getting patients to the operating room after they received induction therapy and we did very well in terms of identifying patients with advanced stage disease or clinical N2 disease and not operating on them straight away. However, we didn’t do quite as well when it came to the referral of those patients on whom we had operated for adjuvant therapy, those who were found to have pathologic N1 or N2 disease, and also didn’t do quite as well in terms of regional staging for patients with only 27% being adequately staged at that 10 lymph node threshold that was set by the CoC. When we looked at the areas of more consistent quality, surgery after neoadjuvant treatment, we saw nonadherence as might have been suggested or assumed was a little bit worse in patients who had no insurance. The measure adherence was a significant predictor of survival.

When we investigated the nonsurgical treatment of N2 disease, initially this was somewhat surprising to us that patients who fell into the non-adherent measure group actually had a superior survival, however, in the multivariant analysis, this seemed to be driven purely by incorrect clinical staging so pathologic stage was the strongest predictor of survival in this group. We investigated areas in which major adherence was poor, so in lymph node evaluation, nonadherence was most common in male patients and those with, and this is a type of this, tumors that were smaller than 2 cm in size. Survival was significantly driven by nonadherence to the measure as well as by insurance status. When we investigated referral for adjuvant treatment, nonadherence was highest again in patients who had to travel the furthest for treatment and in those who lacked insurance, again, speaking to these problems with transitions and care and survival also was driven by nonadherence to the measure.

The study has several limitations. Some of the limitations of the NCDB have been discussed in previous reports but it’s a retrospective analysis so there is some limitation in the granularity of the data as well as the data fields themselves, many of which were adopted from the analyses of other cancers. We have no ability to attribute decision-making in the NCDB. In other words, we don’t know if failures in these transitions of care were related to surgeon, oncologist, or patient. The analytic cohort in question in our study also is one that was developed before release of the measures so this is basically a baseline performance measure, not how we’re doing in response to publication of vetted quality measures so we have no level, no institutional level, of understanding of how the response is and this is something that will require further study. We also have very limited discrimination in 2 measures in which we essentially had topped out. We were above 95% in terms of measured adherence. What underlies the compliance problem, it could be a knowledge gap. It could be an issue of clinicians either not being aware that there is a quality problem and not being able to focus efforts on it or even more, at a more foundational level, not being aware of the appropriate management. It could also be a knowledge gap on the part of the patients and the importance especially around the transitions of care. There could be technical issues in terms of coordination either within an individual center or the technical conduct of the operation in respect to nodal staging.

Also, one of the themes that came out was problems with access to care, the ability of the treating center to provide all of the necessary services, especially in handoffs between surgical and medical oncology groups, and the effect of patient travel. It’s also possible that this is not at all a compliance issue but a measurement issue. If we examine our first measure, that which deals with nodal sampling, the choice of 10 lymph nodes was on the basis of a report that Dr Miller’s group put out a little over 10 years ago, which showed that a 10-lymph node-sampling seemed to be an inflection point in terms of improvements in survival. However, the NCDB doesn’t have specific criteria as to what constitutes a node. What are the nodal processing standards? I know as we all know, stage is much more dependent on nodal station and nodal levels than it is on the number of lymph nodes sampled so perhaps the measure itself is not actually addressing the question that’s most clinically relevant. In terms of reasons for failed referral, we cannot differentiate process from outcome.

We don’t know if this is an issue of patient stability and a problem proximate to referral, if it’s an issue of patient willingness, especially to travel, or if it’s a problem with surgical referral or oncologist availability. It gives us very little ability to kind of dissect reasons for failure. In conclusion, I think it’s very important that we understand our performance in cancer care delivery because this is really what will allow us to target interventions and improvement in the future. Currently, when we evaluate the CoC measures, it appears that nodal sampling at the time of surgery and referral for adjuvant treatment are our areas of biggest problems and these are places in which we might be able to target improvement efforts in the future. However, I think it’s also important that we remember that it’s important that we develop measures even further than they are now so that they actually are giving us the data back that we need to drive these improvements and this is an area where our involvement in leadership as surgeons becomes critical. I’d like to thank the department of surgery at Northwestern for supporting this work as well as the American Association of Thoracic Surgeons grant foundation, and I thank the association again for the privilege of the podium today. I welcome your questions. Thank you.

Craig Sussman. Discussion will be opened by Dr Leah Backhus.

Dr Leah Backhus. First, I d like to thank the association as others before me have done for the opportunity to discuss this work and thank you to Dr Odell and his colleagues, who are to be commended for a very nice presentation and for providing me with an advanced copy of the manuscript for my review today. Just yesterday, we saw no fewer than 3 other NCDB analyses examining other cancer outcomes. This study specifically examined 3 CoC quality measures and the level of adherence over 15 years for nearly 400,000 patients. The overall approach is not entirely novel in that there have been numerous reports published by many researchers, myself included, that have merely reported on the rates of adherence to some performance metric or other but what’s most important in this analysis was not necessarily the level of adherence to specific metrics but rather the association between adherence and improved survival. The linkage is often missing from other well intentioned studies on adherence to national guidelines, which function as barriers to dissemination and widespread acceptance of the measures. The authors should be commended on their efforts to move the field forward, past the adherence and finger-pointing, toward linking these metrics to 2 quality outcomes that everyone should support. The other point I’d like to make is something that you actually underscored in one of your last slides, which was characterizing the level of baseline adherence that we have. Establishing a baseline is important as we move forward to see whether or not these metrics really translate into changes in practice. I have 3 questions for you. My first question is did you consider in your analyses, and I know this is a huge cohort that you’re dealing with, but did you consider some sort of composite measure of concordance or adherence, because one of the things that we fall prey to is the thinking that more is more. Although more is better theoretically, what you really want to look at it is the question of, “is more of something in one category and less in another category” really the ideal outcome measure. For instance, you might be a center that routinely does lymph node sampling alone but does a really good job with your lymph node sampling. You always get 11 stations or more every time you do surgery but you do a lot of surgery and you do a lot of surgery inappropriately so on another measure you performed poorly because you were actually inappropriately operating on patients who have N2 disease, for instance, as their upfront surgical therapy. Is there a way that you can actually analyze things in a composite manner to give a real meaningful measure of quality?

Dr Odell. Thanks very much for that question. I think it underscores a great point, that really when we’re looking at things and we’re looking at how we perform in the delivery of care, the real unit of analysis, the unit that’s important, is the patient but the unit that’s important in terms of being able to facilitate change is the facility. We’re actually in the process of aggregating these data to the facility level so there are 1245 independent institutions at which patients were treated during this time period for lung cancer. The CoC actually sets baseline quality thresholds per measure so they define kind of an adequately performing center metric as 85% adherence to the measure so 85% of the time, you are meeting an individual measure for that patient. When we look at overall measure adherence, at a center level, as might be expected, our rates go much, much lower than they are when we look at the patient level. Centers that are meeting, so nodal sampling, for example, only about 20% of centers in the country are meeting that 85% threshold. To aggregate the measures together themselves proves a little bit more difficult because the inclusion criteria are so different for them. It’s kind of difficult to … It’s not really an apples to apples comparison. How you’re doing with nodal sampling might not necessarily apply to your recognition of that N2 patient, and I think it’s been very difficult for us to kind of come up with a way that we would look at doing that because we have a lot of dropout of centers that might have, as you get to kind of a lower volume group, where they might have a good number of patients that you can analyze in measure 1 but they only have a couple of patients that you can analyze in measure 2. One thing that we’re looking at doing are just for the higher-volume centers, trying to create an aggregate measure but I think it’s something that has some methodological challenges but I think that you’re absolutely right. That’s the way that we need to move, especially as we start to break down quality measures as Varun Puri’s group at Wash U has done, by investigating just one disease state, so investigating just clinical stage I or pathologic stage I and how we do, and they’ve been able to show in some of their analyses from the NCDB that the number of measures that you would adhere to is actually the strongest predictor of survival. The CoC measures themselves don’t really lend to that quite as well but I think conceptually, that is exactly the way that we need to be thinking about how we assess quality as we move forward.

Dr Backhus. Great. My next question, how did you define those patients for whom surgery was intended as part of the 5-modality therapy? How was the intent for induction therapy established other than by the definition of receiving subsequent surgery? Could it have been that patients for whom surgery was intended after induction therapy, but they otherwise did poorly after their induction regimen, might have otherwise fallen off and never have been counted in that denominator?

Dr Odell. That’s a very astute point. We actually for this analysis, we followed exactly the analytic strategy that was defined when the measures were developed, which used surgery as the denominator inclusion point. Indeed, you’re right. This adherence measure and the way that it’s defined by the CoC misses that point of intention. We can’t attribute the decision-making again in this process to were you supposed to get to surgery and you failed that induction point. Those are patients that would be missed. That is a potential shortcoming in that preinduction measure. It’s not something that we can assess. Really, what the intent of the measure was is the timing. Are we getting patients ultimately to surgery? Are we applying surgery correctly in that window when induction therapy would be effective?

Dr Backhus. My last question is do you have data on how many patients underwent invasive mediastinal staging as a means to establish their nodal status? We know that many surgeons will consider surgical resection as initial therapy for patients either with incidentally found N2 disease or patients might not even be considered for mediastinoscopy for instance if they have very small tumors, which our data seem to bear out, that patients with very small tumors have less mediastinal staging. How is it that you were able to incorporate invasive mediastinal staging in your analysis?

Dr Odell. You’re talking specifically for the nodal analysis measure?

Dr Backhus. For the nodal measure. Measure 1.

Dr Odell. Yeah, so for that measure, the database doesn’t capture mediastinoscopy if it’s done at a separate time in terms of the nodal count. We know about 20%, a little over 20%, of patients underwent mediastinoscopy as a stand-alone procedure so that’s an area where we don’t necessarily have data as to how many nodes were taken there to be able to aggregate, so say someone had 4 node samples at the time of mediastinoscopy and 7 samples in the operating room. In this data rubric, that would be considered a failure.

Dr Backhus. But that’s significant.

Dr Odell. Because you wouldn’t necessarily be able to add. If the mediastinoscopy was done at the same time as the resection, so if they were done as kind of phase 1 and phase 2 of the procedure, all of those nodes are counted so, you know, again, it’s the way the measure was constructed in its potential flaw in the measure with …

Dr Backhus. Well, I think if 20% of your patients had a separate stand-alone mediastinoscopy then definitely that needs to be accounted for because obviously those patients will likely have fewer node samples at the time of resection.

Dr Odell. Yeah, I agree.

Dr Doug Wood (Seattle, Wash). Doug Wood, Seattle. I have 2, 3 questions. David, nicely presented. The first question is I want to make sure I understand the concordance on the lymph node rate, sampling rate because in the abstract in the book, it says 74% concordant but in your presentation, I think I saw the opposite or that, 26% concordance, so which is it?

Dr Odell. It’s the latter.

Dr Wood. Okay. Disappointing. The second question is actually more important, however, which is the definition of adjuvant chemotherapy because what I’m wondering is does that include all chemotherapy that is given at some point after any surgery? Meaning, it includes those that are appropriate for adjuvant therapy for example, a stage II or stage III patient, but also includes patients who a year later develop metastatic disease and appropriately then are reconsidered for chemotherapy. Is that unfortunately wrapped into adjuvant therapy in your analysis?

Dr Odell. It is. Iťs the way that the measure is defined by the CoC. It extends out to within that first year but if, say at the 6-month restaging scan, you had now progression of metastatic disease and went on to systemic therapy, you would be considered a failure. The way that the measure is intended, it’s the inclusion criteria are not just all patients who underwent surgery. Patients who underwent surgery and had pathologic N1 or N2 disease, and the measure can be met. You can be considered adherent if you either begin chemotherapy within that 6-month window or you’re been referred and have a visit with a medical oncologist, which is also tracked in the NCDB during that period of time. For example, if you had a patient who had N1 disease, you referred them to a medical oncologist, they declined chemotherapy, and then secondarily developed recurrent disease and went on systemic chemotherapy, as long as that patient was seen in that 6-month window by the oncologist, even if they didn’t initiate treatment, that would be considered a measure pass.

Dr Wood. So I’m actually more confused now than I was before the question. What I want to be clear about is, is your ability using the NCDB to accurately only analyze patients with stage II or III disease that would be appropriate for adjuvant chemotherapy, not subsequent chemotherapy because of relapse of disease later?

Dr Odell. Yes. Pathologic N1 or N2 disease, then the next marker is did they either begin chemotherapy or see a medical oncologist within 6 months.

Dr Wood. Perfect. Thank you.

Craig Sussman. Thank you, David.

Dr Dan Miller. Can I just ask one question? Dan Miller.

Moderator. Very quick, Dan.

Dr Miller. The one question I have, when we did the original study was the SIR database that we looked at and we could not go to stations. If I had to redo it, I would do stations and not lymph node count but the big thing is in your study, did you look at wedge versus segment versus lobe in regard to the number of lymph nodes that were in, because a lot of times, if it’s a segment, you’re not getting as many lymph nodes, most of them, are they in 2 nodes or not. In one, the pathologist isn’t going to dig that out.

Dr Odell. That’s a great point. We didn’t specifically do that analysis but that’s something that we’ll add in.

Dr Miller. Also, two, now we have a checklist in the operating room for our nodal evaluation. Just make sure you get that 2R, the 4R, the 7, 10, and 11. That’s very important and so I think because you’re going to come back and the pathologist is going to say you’ve only got 9 nodes. Well, why don’t you look again.

Dr Odell. Yeah.

Dr Miller. Thank you. Nice job.

Dr Odell. Thank you very much.