Measuring Institutional Quality in Head and Neck Surgery Using Hospital-Level Data

Key Points

Question

Can 2 surgical quality metrics that have been validated at the patient level in head and neck surgery (negative margins and lymph node yield from a neck dissection) be used to identify higher quality hospitals with improved outcomes?

Findings

Patients treated at hospitals that achieved the combined metric of 90% or higher negative margins and 80% or more of cases with lymph node yields of 18 or more experienced a significant reduction in mortality. Importantly, this survival benefit was independent of previously reported measures of institutional quality, such as hospital volume and teaching status.

Meaning

These 2 surgical quality metrics may be a surrogate measure of the quality of oncology care in head and neck surgery and should be considered when identifying hospitals for quality improvement initiatives.

Using data from the National Cancer Database, this study examines the association of negative margin rate and lymph node yield with survival in patients with head and neck squamous cell carcinomas at the hospital level.

Abstract

Importance

Negative margins and lymph node yields (LNY) of 18 or more from neck dissections in patients with head and neck squamous cell carcinomas (HNSCC) have been associated with improved patient survival. It is unclear whether these metrics can be used to identify hospitals with improved outcomes.

Objective

To determine whether 2 patient-level metrics would predict outcomes at the hospital level.

Design, Setting, and Participants

A retrospective review of records from the National Cancer Database (NCDB) was used to identify patients who underwent primary surgery and concurrent neck dissection for HNSCC between 2004 and 2013. The percentage of patients at each hospital with negative margins on primary resection and an LNY 18 or more from a neck dissection was quantified. Cox proportional hazard models were used to define the association between hospital performance on these metrics and overall survival.

Main Outcomes and Measures

Margin status and lymph node yield at hospital level. Overall survival (OS).

Results

We identified 1008 hospitals in the NCDB where 64 738 patients met inclusion criteria. Of the 64 738 participants, 45 170 (69.8%) were men and 19 568 (30.2%) were women. The mean SD age of included patients was 60.5 (12.0) years. Patients treated at hospitals attaining the combined metric of a 90% or higher negative margin rate and 80% or more of cases with LNYs of 18 or more experienced a significant reduction in mortality (hazard ratio [HR] 0.93; 95% CI, 0.89-0.98). This benefit in survival was independent of the patient-level improvement associated with negative margins (HR, 0.73; 95% CI, 0.71-0.76) and LNY of 18 or more (HR, 0.85; 95% CI, 0.83-0.88). Including these metrics in the model neutralized the association of traditional measures of hospital quality (volume and teaching status).

Conclusions and Relevance

Treatment at hospitals that attain a high rate of negative margins and LNY of 18 or more is associated with improved survival in patients undergoing surgery for HNSCC. These surgical outcome measures predicted outcomes independent of traditional, but generally nonmodifiable characteristics. Tracking of these metrics may help identify high-quality centers and provide guidance for institution-level quality improvement.

Introduction

Defining high-quality care may allow clinicians to have targets for quality improvement efforts and assist payers in developing value-based payment models. In head and neck surgery, there are currently few metrics for quality at the individual level and even fewer at the hospital level. Most contemporary studies on hospital quality have focused on the effect of hospital volume and teaching status on survival. These structural measures are difficult to improve on and literature has shown that patients are relatively unwilling to travel long distances to hospitals with better outcomes for care. Therefore it is critical to define modifiable quality measures associated with improvements in head and neck cancer survival.

At the individual level, there are at least 2 validated measures of high-quality surgical treatment for head and neck cancer. Compared with a positive resection margin, a negative margin around a primary tumor resection is a well-recognized indicator of a more favorable prognosis. In addition, multiple studies have reported an association between lymph node yield (LNY) from a neck dissection and overall survival in head and neck cancer for both node-negative and node-positive patients. It is unknown if these metrics can be used at the institutional level to help identify high-quality hospitals. Validated hospital-level quality metrics may provide measures of performance that can incorporate the complexity of care influencing a given treatment outcome and define objective practice goals on the scale that quality improvement initiatives are often implemented.

The purpose of this study was to evaluate 2 surgical outcomes measures, negative margin rate and lymph node yield, and their impact on survival at the hospital level. Our hypothesis was that hospitals achieving high rates of negative margins and LNY of 18 or more from neck dissections would be associated with better overall survival, even after adjusting for patient-level factors. A secondary aim was to define what percentage of procedures at a given institution would need to achieve these metrics for a hospital to show improved outcomes.

Methods

This study was granted an exemption by the university of Stanford institutional review board owing to the nature of the research and use of deidentified data. Records of patients treated for head and neck cancers were identified in the National Cancer Database (NCDB). The NCDB is a nationwide, facility-based, clinical surveillance resource oncology data set, jointly maintained by the American College of Surgeons and the American Cancer Society. The database contains cancer outcomes data for cancers treated in hospitals accredited by the American College of Surgeons Commission on Cancer (CoC). The CoC hospitals provide care for approximately 70% of patients with newly diagnosed cancers in the United States.

The data set includes individual-level demographic and clinical information, as well as characteristics of the hospitals where care was delivered. Patient demographics included sex, race, income quartile, and education level. Income and education level are measured at the ZIP code level. Clinical details included the Charlson Comorbidity index, site of primary tumor, tumor-node-metastasis (TNM) staging, and histologic analysis. Surgical and pathologic details included the number of lymph nodes examined, the number of positive lymph nodes, and margin status. The NCDB uses a hierarchical numerical code to capture margin status as recorded in the pathology report. Margins listed as both grossly and microscopically negative in primary pathology reports were coded as negative. Other codes for margin status that were considered positive included microscopic residual tumor, macroscopic residual tumor, or residual tumor not otherwise specified. Additional treatment details in the set include adjuvant radiotherapy and chemotherapy. Lymph node yield was defined as the number of lymph nodes removed and examined by a pathologist. Hospital characteristics include hospital type (community, academic, or integrated) and annual surgical volume (in quartiles).

We analyzed records of patients with head and neck cancers who were treated in CoC hospitals between 2004 and 2013. Using the International Classification of Diseases for Oncology, Third Edition (ICD-O-3), we identified patients with codes for tumors of the oropharynx (C-09.0-09.1, C- 09.8-09.9, C-10.0, C-10.2-10.4, C-10.8-10.9, C-02.4, and C-01.9), oral cavity (C-02.0- 02.3, C-02.8-03.1, C-03.9-04.1, C-04.8-05.2, C-05.8-06.2, C-06.8-06.9), larynx (C-32.0- 32.3, C-32.8-32.9), and hypopharynx (C-12.9-13.2, C-13.8-13.9). Patients with pathologic diagnosis of squamous cell carcinoma were included based on ICD-03 codes 8052, 8070-8076, 8083-8084, 8094, and 8560. We included all patients who underwent upfront surgery with regional nodal dissection. Patients without or with 1 lymph node recorded were excluded from the data set to avoid inclusion of simple lymph node biopsies rather than a standard neck dissection. Hospitals were excluded if they had fewer than 5 cases over 10 years.

For each of the quality measures, a hospital-level cutoff was selected based on a sensitivity analysis and review of the literature. The cutoff was the percentage of patients at each hospital that would have to achieve negative margins or LNY of 18 or more for the hospital to be categorized as having achieved the measure. Hospitals were then categorized in a binary fashion based on whether or not the cutoff was achieved. A sensitivity analysis was performed by doing sequential multivariable Cox proportional hazard regressions with cutoffs varying from 80% to 95% for negative margin rates and 70% to 90% for LNY of 18 or more at 5% intervals. The hazard ratios from this analysis were compared with standards set for a recent clinical trial (ECOG 33-11) and for other disease sites (colorectal cancer and gastric cancer). Based on this information, we used a cutoff of negative margins in 90% or more of cases, and LNY higher than 18 nodes in at least 80% of patients.

These cutoffs were combined to generate a hospital-level composite quality measure. Hospitals were given the presumptive designation as high-quality if the aggregate of patients treated at that institution were above both cutoffs. Univariate descriptions and χ² analyses of hospitals were performed. Multivariable Cox proportional hazard regression models estimated the HR of mortality by type of hospital (high quality vs other) after accounting for patient-level comorbidities, tumor, pathologic analysis, and treatment variables. In this model, the patient-level variables of negative margin status and LNY of 18 or more were included. Models including interaction terms (LNY≥18 x volume, LNY≥18 x hospital type, negative margins x volume, negative margins x hospital type) were run to differentiate the effect of the measures under study from the effect of more traditional measures of hospital quality. Proportional hazard assumption was tested for all models. All tests of significance were 2 tailed. All statistical analysis was performed using Statistical Analysis System (SAS, version 9.4, SAS Inc).

Results

Of the 1287 hospitals treating 312 972 patients with an upper aerodigestive tract malignant abnormality (involving the oral cavity, oropharynx, larynx, or hypopharynx), 1273 hospitals (98.9%) treated 132 847 patients (42.4%) with up-front surgery for a squamous cell carcinoma. We identified 65 097 total patients who met inclusion criteria (neck dissection with >1 lymph node documented on pathologic analysis) during the period under study. After excluding hospitals that treated 5 or fewer patients over 10 years, a total of 1008 hospitals and 64 738 patients were retained for the final analysis (eFigure 1 in the Supplement).

The characteristics of the 1008 hospitals are reported in eTable 1 in the Supplement. The hospitals that achieved higher rates of negative margins (≥90%) and higher rates of LNY of 18 or more (≥80%) had higher average patient volumes (97.5 patients per year) vs other hospitals (25.1 patient per year). These hospitals were also more frequently academic medical centers (25.6% vs 18.7%).

The negative margin rate at the hospital level is shown in Figure 1. While the majority of hospitals had high proportions of operations with negative margins (median, 76.1%; interquartile range [IQR], 66.7%-84.6%), only 105 hospitals (10.4%) achieved a negative margin in 90% or more of cases. Similarly, the percentage of cases with LNY of 18 or more at the hospital level is shown in Figure 2. Of the hospitals included in this study, there was a relatively wide variation in performance for this LNY metric (median, 64.3%; IQR, 51.6%-76.3%). There were 199 hospitals (19.7%) that achieved an 18 or more lymph node count neck dissection in 80% or more of patients.

Figure 1. Distribution of Hospitals by Rates of Negative Surgical Margins.

Head and neck cancers; data from the National Cancer Database, 2004 to 2013.

Figure 2. Distribution of Hospitals by Rates of Yielding 18 or more Lymph Nodes After Neck Dissection.

Head and neck cancers; data from the National Cancer Database, 2004 to 2013.

The sensitivity analysis to determine hospital-level cutoffs for negative margin rates and LNYs of 18 or more are shown in Tables 1 and 2. Treatment at hospitals achieving negative margins in at least 90% of patients was associated with decreased mortality (HR, 0.92; P < .001; 95% CI, 0.88-0.96) (Table 1). Below this threshold there was no difference in mortality. A 95% negative margin rate was also not associated with a significant difference in mortality (HR, 0.95; P = .39; 95% CI, 0.84-1.07); however, wide CIs observed in this stratum (>95%) suggest that the sample size prohibited a clear interpretation of the data at this level. Treatment at hospitals achieving LNY of 18 or more in at least 80% of patients was associated with a decrease in mortality (HR, 0.95; 95% CI, 0.92-0.98; P = .001). Higher cutoffs for LNY seemed to affect the hazard of mortality in a linear fashion beginning at 75%, with no appreciable plateau.

Table 1. Cox Regression Analysis Comparing Hospital-Level Hazard of Mortality by Percent of Cases With Negative Margins^a.

Negative Margin Threshold, %	Hospitals, No.	Patients, No.	HR (95% CI)	P Value
80	396	36 998	0.99 (0.96-1.02)	.37
85	243	22 486	0.99 (0.96-1.02)	.40
90	105	9003	0.92 (0.88-0.96)	<.001
95	43	831	0.95 (0.84-1.07)	.39

Abbreviation: HR, hazard ratio.

^aModel includes sex, age, race, comorbidities, site, cN stage, pT stage, number of positive nodes, receipt of adjuvant therapy, extracapsular extension, insurance, income, education level, hospital volume, and hospital type. Head and neck cancers; data from the National Cancer Database, 2004 to 2013.

Table 2. Cox regression Models Comparing Hazard of Mortality by Percent of Cases With Lymph Node Yields of 18 or More^a.

Lymph Node ≥18 Threshold, %	Hospitals, No.	Patients, No.	HR (95% CI)	P Value
70	395	38 830	0.98 (0.95-1.01)	.23
75	290	31 819	0.97 (0.94-1.00)	.03
80	199	23 336	0.951 (0.92-0.98)	.001
85	118	12 877	0.927 (0.89-0.96)	<.001
90	47	3101	0.91 (0.85-0.97)	.005

Abbreviation: HR, hazard ratio.

^aModel includes sex, age, race, comorbidities, site, cN stage, pT stage, number of positive nodes, receipt of adjuvant therapy, extracapsular extension, insurance, income, education level, hospital volume, and hospital type. Head and neck cancers; data from the National Cancer Database, 2004 to 2013.

To further estimate the hazard of mortality at the hospital level, we categorized hospitals as high-quality if they achieved negative margins in at least 90% of patients and lymph node counts of 18 or more in 80% or more of patients. Hospitals reaching both of these quality measures were compared against all others. Results of Cox models predicting the hazard of mortality at the hospital level, after accounting for patient-level covariates, are shown in Table 3.

Table 3. Cox Model Predicting the Hazard of Mortality by Hospital Process Quality^a.

Variables	Hazard Ratio (95% CI)	P Value
High-quality hospitalb	0.93 (0.89-0.98)	.003
Others	1 [Reference]
Lymph node ≥18 (patient level)	0.85 (0.83-0.88)	<.001
Negative margin (patient level)	0.73 (0.71-0.76)	<.001
Volume
Quartile 1 (lowest)	1 [Reference]
Quartile 2	1.08 (0.96-1.21)	.18
Quartile 3	1.05 (0.95-1.17)	.34
Quartile 4 (Highest)	1.00 (0.90-1.11)	.97
Hospital type
Community	1 [Reference]
Academic	1.04 (1.01-1.08)	.02
Integrated	1.01 (0.95-1.08)	.73

^aModel includes sex, age, race, comorbidities, site, cN stage, pT stage, number of positive nodes, receipt of adjuvant therapy, extracapsular extension, insurance, income, education level, hospital volume, and hospital type. Head and neck cancers; data from the National Cancer Database, 2004 to 2013.

^bHigh-quality hospitals were defined as hospitals where patients had at least 80% compliance with lymph node counts of 18 or more and at least 90% compliance with negative margins.

In this model, margin status (HR, 0.73; 95% CI, 0.71-0.76; P < .001) and LNY (HR,0.85; 95% CI, 0.83-0.88; P < .001) at the patient level were both independently associated with overall survival. After controlling for the impact of these patient-level factors, there still remained a significant benefit of being treated at a hospital compliant with the metrics. This independent hospital-level effect (HR, 0.93; 95% CI, 0.89-0.98; P = .003) of being classified as a high-quality institution showed a positive impact on overall survival. Notably, this effect was seen while controlling for both hospital volume and hospital type, 2 variables shown in different contexts to be predictive of quality care.

Two-level survival models and 1-level survival models were both tested. Using a random smaller sample, the results of a 1-level model were compared with the results of a 2-level model and there was no qualitative difference in our outcome. Therefore a 1-level model was used for this analysis. Interaction terms were tested in the model, but there was no qualitative change in the associations noted.

Discussion

The purpose of the current study was to determine the ability of 2 indicators of surgical quality in head and neck cancer treatment (negative surgical margins and adequate lymph node yield from a neck dissection) to predict outcome at the hospital level. The data presented here demonstrate that treatment at hospitals with at least a 90% negative margin rate and an LNY of at least 18 in 80% or more of cases is significantly associated with improved overall survival. The association at the hospital level persisted despite rigorous model adjustment for individual-level patient factors. We also found large variation in the achievement of high-quality performance, with only 20% of hospitals reporting at least 80% of cases with LNYs of 18 or more, and about 10% of hospitals with at least a 90% negative margin rate, highlighting an opportunity for potential quality improvement interventions.

These findings demonstrate that a 90% or more negative margin rate at the hospital level was associated with a significant improvement in survival. This negative margin rate is consistent with the cutoff used in a contemporary multiinstitutional clinical trial (ECOG 33-11), designed to evaluate the effectiveness of surgical therapy for oropharyngeal cancer. Surgeons became excluded from participating if their positive margin rate exceeded 10%. The acceptable rate of negative margins is not well studied, but based on the current literature, a 90% rate would likely be considered reasonable when including all T-stages.

Our data showing that 80% or more of cases with lymph node counts of 18 or more was associated with a significant survival advantage at the hospital level support the known correlation between lymph node counts and patient-level survival. A threshold of 80% accounts for the likelihood that achievement of an LNY of 18 or more may not be feasible in particular patient populations based on factors such as age, BMI, or gross extranodal extension. However, in confirming this association between adequate institutional lymph node yield rate and survival, we have provided support for the CoC recommendations for institution-level lymph node yields in gastric and colorectal cancer (80% and 85%, respectively). It is important to note that this metric is a combined effort of the surgeon and pathologist, and should not be attributed to surgical technique alone.

Despite these similarities to published literature and accreditation guidelines, our study is novel because we identified specific clinical measures in head and neck surgery that reflect outcome quality at the hospital level. These measures are relevant targets for quality improvement because they move toward capturing the complexity of oncologic treatment. Finally, we have developed clinically meaningful cutoffs for each of these processes at the hospital level.

In head and neck cancer surgery, the primary measures of hospital quality thus far have focused on structural and relatively nonmodifiable quality measures, such as hospital volume and hospital teaching status. In multiple studies, high-volume hospitals and academic medical centers have been associated with improved overall survival and superior treatment-related outcomes. Multiple studies in other disease sites have shown similar results. The mechanism by which higher hospital volume improves outcomes is unclear, but it is thought to signal a higher level of experience and expertise, which would then be applied to the treatment of individual patients. In contrast to the traditional emphasis placed on the effect of volume, our results demonstrate that accounting for negative margins and LNY neutralizes the effect of hospital volume on mortality.

Additional measures of surgical process, such as rates of appropriate referrals for adjuvant therapy and the frequency of perioperative antibiotic use, have also been advocated as quality measures in head and neck surgery. Despite use of these metrics, there is limited evidence that hospital compliance with many of these currently employed measures results in improved survival. Moreover, these process measures do not account for the critically important oncologic aspects of surgery and pathologic analysis. Our results address the limitations in prior work by identifying potentially modifiable measures, which reflect the importance of oncologic resections while also identifying and differentiating hospitals with improved survival. Based on the cut points we have defined, these measures may serve as useful goals for hospital-level quality improvement projects, and provide clinical targets for value-based payment programs.

Our study is important because it begins to shift the focus of measuring hospital quality away from traditional, nonactionable structural measures (such as hospital surgical volume) toward actionable, clinically based measures. Our work suggests that hospitals could improve outcomes for patients by intervening on clinical performance at the system level. The use of outcome measures, rather than structural measures, pinpoints a more tangible, patient-level metric for hospital administrators and providers. These measures may be used to identify higher quality institutions and promote sharing of best practices with hospitals working to improve quality in outcomes following treatment for head and neck cancer.

Limitations

Limitations of this study include the retrospective nature of a large database cohort study. We were only able to demonstrate an association between these measures and survival, and future work will be needed to demonstrate causation. Tumor factors, such as HPV status and perineural invasion are not contained in the database. Variation in what constitutes a positive margin and margin harvesting techniques may differ across surgeons, pathologists, and institutions. Each pathologist may use a different minimum distance from the tumor edge to declare a negative margin and this may vary based on tumor location. The NCDB does not capture tumor distance from the closest margin, making standardization across institutions impossible. Although prior work on lymph node yields has separately demonstrated a survival benefit in patients treated for clinically node-negative and clinically node-positive disease, these studies, and the current one, did not collect information on which levels of the neck dissection were performed and therefore we could not control for the reported extent of neck surgery. Finally, survival data are limited to only overall survival, therefore we could not assess cancer-specific mortality, disease-free survival, or morbidity. Achievement of negative margins may come at the expense of functional impairment and reduced quality of life, and therefore there continues to be a need to balance more aggressive surgery, which may potentially improve survival, with significant morbidity.

Conclusions

A composite measure of lymph node counts of at least 18 in 80% or more and negative margin rates of at least 90% identifies a subset of high-quality hospitals associated with a significant survival advantage when compared with other hospitals. This association is irrespective of patient clustering and hospital type. Performance on these quality measures may be useful in identifying hospitals that may benefit from quality improvement programs and, based on our findings, may also provide a basis for the design of value-based reimbursement schemes.

Supplement.

eTable. Descriptive characteristics of 1,008 hospitals in the final dataset

eFigure. CONSORT diagram illustrating patient/hospital inclusion and exclusion criteria