Abstract
IMPORTANCE
Quality metrics for patients with head and neck cancer are available, but it is unknown whether compliance with these metrics is associated with improved patient survival.
OBJECTIVE
To identify whether compliance with various process-related quality metrics is associated with improved survival in patients with oral cavity squamous cell carcinoma who receive definitive surgery with or without adjuvant therapy.
DESIGN, SETTING, AND PARTICIPANTS
A retrospective cohort study was conducted at a tertiary academic medical center among 192 patients with previously untreated oral cavity squamous cell carcinoma who underwent definitive surgery with or without adjuvant therapy between January 1, 2003, and December 31, 2010. Data analysis was performed from January 26 to August 7, 2015.
INTERVENTIONS
Surgery with or without adjuvant therapy.
MAIN OUTCOMES AND MEASURES
Compliance with a collection of process-related quality metrics possessing face validity that covered pretreatment evaluation, treatment, and posttreatment surveillance was evaluated. Association between compliance with these quality metrics and overall survival, disease-specific survival, and disease-free survival was calculated using univariable and multivariable Cox proportional hazards analysis.
RESULTS
Among 192 patients, compliance with the individual quality metrics ranged from 19.7% to 93.6% (median, 82.8%). No pretreatment or surveillance metrics were associated with improved survival. Compliance with the following treatment-related quality metrics was associated with improved survival: elective neck dissection with lymph node yield of 18 or more, no unplanned surgery within 14 days of the index surgery, no unplanned 30-day readmissions, and referral for adjuvant radiotherapy for pathologic stage III or IV disease. Increased compliance with a “clinical care signature” composed of these 4 metrics was associated with improved overall survival, disease-specific survival, and disease-free survival on univariable analysis (log-rank test; P < .05 for each). On multivariable analysis controlling for pT stage, pN stage, extracapsular spread, margin status, and comorbidity, increased compliance with these 4 metrics was associated with improved overall survival (100% vs ≤50% compliance: adjusted hazard ratio [aHR], 4.2; 95% CI, 2.1-8.5; 100% vs 51%-99% compliance: aHR, 1.7; 95% CI, 1.0-3.1), improved disease-specific survival (100% vs ≤50% compliance: aHR, 3.9; 95% CI, 1.7-9.0; 100% vs 51%-99%: aHR, 1.3; 95% CI, 0.6-2.9), and improved disease-free survival (100% vs ≤50% compliance: aHR, 3.0; 95% CI, 1.5-5.8; 100% vs 51%-99% compliance: aHR, 1.6; 95% CI, 0.9-2.7).
CONCLUSIONS AND RELEVANCE
Compliance with a core set of process-related quality metrics was associated with improved survival for patients with surgically managed oral cavity squamous cell carcinoma. Multi-institutional validation of these metrics is warranted.
The Agency for Healthcare Research and Quality has defined quality health care as “doing the right thing, at the right time, in the right way, for the right person, and having the best possible results.”1 Measures of quality are being used at the individual physician and hospital-wide levels to determine ratings, accreditation, and reimbursement. However, there remains disagreement about how to define quality in a practical and actionable manner,2,3 as well as how current metrics correlate with true quality care.4,5
According to the Donabedian6 model, the quality of health care can be assessed using 3 types of measures: structure (the environment of health care delivery), process (what a health care professional does for the patient), and outcomes (what happens to the patient). Process-related quality metrics are attractive because they can empower health care professionals to change behavior at the individual level.7
Process-related quality measures have been evaluated in general surgery with variable findings between compliance and the outcome of interest.5,8–10 Nationally endorsed and validated quality metrics by groups such as the National Quality Forum and the American College of Surgeons/National Cancer Database Commission on Cancer exist for patients with colorectal, lung, esophageal, breast, and gastric cancer. To our knowledge, no such measures exist for patients with head and neck cancer.
A call for the development of quality measures for patients with oral cavity squamous cell carcinoma (OCSCC) was approved by the executive council of the American Head and Neck Society in 2007.11 In the interim, several publications have begun to address the topic,12–16 and international efforts to codify quality care performance indicators for head and neck cancer have occurred.17,18 However, 9 years later, nationally endorsed quality measures for patients with head and neck cancer still do not exist in the United States,4 and it is unknown whether compliance with the metrics suggested in 2007 is associated with improved patient outcomes. Given the increasing emphasis placed on quality care,1,19 it is imperative to establish quality metrics with predictive validity for patients with head and neck cancer and then demonstrate that compliance with these measures correlates with improved outcomes.
In this study, using a population of patients with surgically treated OCSCC, we sought to determine compliance with a set of process-related quality metrics that possess face validity, measure the association between compliance and survival, identify the individual quality metrics associated with improved survival, describe a “clinical care signature” group of metrics, and determine if compliance with these metrics correlates with improved survival.
Methods
Patient Data
A retrospective review of the medical records of patients 18 years or older who under went primary surgical intervention for an oral cavity malignant neoplasm at our academic medical center from January 1, 2003, to December 31, 2010, was performed. Inclusion criteria for the study were previously untreated primary oral cavity cancer, no history of head and neck cancer, squamous cell carcinoma histologic findings, definitive surgical management, and a minimum 12-month follow-up (or death). Two hundred sixty-seven patients met inclusion criteria; 75 were excluded owing to insufficient documentation in the medical record to assess compliance with the quality metrics. The final cohort comprised 192 patients with previously untreated OCSCC undergoing surgery with or without adjuvant therapy and a minimum follow-up of 12 months or death. Seven surgeons (including B.N.), along with other members of the multidisciplinary head and neck cancer team, treated these patients. The medical records were reviewed for data related to demographics; clinical variables, including comorbidity as measured by the Adult Comorbidity Evaluation-27 index20; details of surgical and adjuvant treatment; pathologic variables; surveillance; and death.
This study was approved by the Washington University School of Medicine Institutional Review Board. Patient data were deidentified once all relevant data had been abstracted from the medical records.
Selection of Potential Quality Metrics
The quality metrics were developed in a multidisciplinary fashion with 10 members of the head and neck cancer team (surgeons, radiation oncologists, and medical oncologists)at our institution (Table 1). The metrics encompassed the longitudinal care of the patient with head and neck cancer (pretreatment evaluation, treatment, and post treatment surveillance) and were modeled on metrics suggested by the American Headand Neck Society,11 National Comprehensive Cancer Network guidelines for oral cavity cancer,21 or other publications.12,22,23 The metrics were chosen because they possessed the following characteristics: usability (the information is actionable and understandable), feasibility (the data are easily measurable and collectable), reproducibility, meaningfulness (the metrics are agreed on by stakeholders), promotion of quality improvement (compliance with the metrics can be tracked over time), and possession of face validity (a priori suspicion of association with improved outcomes).
Table 1.
Process-Related Quality Metrics
| Quality Metric | Eligibility | Approved by AHNSa | Recommended by NCCN (February 2014)a |
|---|---|---|---|
| Pretreatment evaluation | |||
| Time from referral to clinic visit ≤14 d | All patients | ||
| Examination of pharynx and larynx (IME, FFL, DL) before definitive operation | All patients | Yesb | |
| cTNM stage documented | All patients | Yes | |
| Tobacco cessation counseling | Tobacco use within past 6 mo | Yes | Yesb |
| Dental evaluation prior to treatment | Teeth and clinical AJCC stage III or IV disease | Yesb | |
| Internal review of pathologic findings | Pathologic findings from outside hospital | ||
| Tumor board presentation | All patients | ||
| Treatment related | |||
| Time from initial evaluation to surgery ≤21 d | All patients | ||
| Elective neck dissection with ≥18 lymph nodes | Undergoing elective neck dissection | ||
| pTNM stage documented | All patients | ||
| Margin status documented | All patients | ||
| No unplanned surgery ≤14 d | All patients | ||
| No unplanned readmission ≤30 d after discharge | All patients | ||
| Referral for adjuvant radiotherapy if stage III or IV | Pathologic AJCC stage III or IV tumors | Yes | Yes |
| Referral for adjuvant chemoradiotherapy if ECS or positive margins | Positive margins or ECS | Yes | Yes |
| Start of adjuvant therapy ≤6 weeks postoperatively | Receiving adjuvant therapy | Yes | |
| Posttreatment surveillance | |||
| Thyroid function testing ≤12 mo after radiotherapy | Received adjuvant therapy and alive 12 mo after its completion | Yes | Yes |
| Surveillance imaging ≤6 mo after therapy completion | No clinical suspicion of recurrence and alive 6 mo after completion | Yes | |
| Multidisciplinary follow-up | Received multimodality therapy |
Abbreviations: AHNS, American Head and Neck Society; AJCC, American Joint Committee on Cancer; cTNM, clinical tumor, node, metastasis; DL, direct laryngoscopy; ECS, extracapsular spread; FFL, flexible fiberoptic laryngoscopy; IME, indirect mirror examination; NCCN, National Comprehensive Cancer Network; pTNM, pathologic tumor, node, metastasis.
Empty cells indicate institutional metric with face validity.
Recommended by NCCN when indicated.
These metrics can apply to any subsite of the head and neck mucosa. The oral cavity was chosen to evaluate these metrics because the American Head and Neck Society previously suggested the oral cavity as the site to test quality metrics11 and because the primary treatment modality of OCSCC is most commonly surgical, so there is little variation in the general approach to therapy.
Definitions of Variables and Eligibility
Table 1 shows the eligibility criteria for the quality metrics. Dental evaluation consisted of documentation of a dental referral before, or dental extractions during, the index hospitalization. Elective neck dissection lymph node yield was calculated on the ipsilateral side. If bilateral elective neck dissections were performed, the average of both sides was used.22 Unplanned surgery within 14 days did not include staged neck dissections or dental extractions but did include re-resection for positive margins. We defined a 30-day unplanned readmission as an admission to any service in the hospital within 30 days of discharge that was not expected.24 For thyroid surveillance, if a patient was dead within 12 months of completing radiation treatment, the metric was not evaluable. For surveillance imaging, if imaging was prompted by clinical suspicion of recurrence or the patient was dead within 6 months of the completion of therapy, the metric was not evaluable.
Outcome Measures
The outcome measures included compliance rates with quality metrics and measures of survival (overall survival [OS], disease-specific survival [DSS], and disease-free survival [DFS]). For each patient, a compliance score for the group of metrics was calculated based on the opportunity model used by the Centers for Medicare & Medicaid Services.25 If a patient was eligible for all 19 processes and received 10, the compliance score was 53% (10 of 19). If a patient was eligible for only 10 processes and received all 10, the compliance score was 100%.
Overall survival was calculated from the date of surgery to the date of death from any cause. Disease-specific survival was calculated from the date of surgery to the date of death from oral cavity cancer or direct effects of treatment. Disease-free survival was calculated from the date of surgery to the date of either death or first recurrence. A local recurrence was defined as squamous cell carcinoma within 2 cm of the original tumor and within 5 years of the original diagnosis; otherwise it was considered a second primary tumor.26 Patients lost to follow-up were censored at the date last known to be alive.
Measuring the Association Between Compliance With Quality Metrics and Survival
To determine the association between compliance with the group of quality metrics and survival, compliance scores were calculated for each patient as described above. Compliance scores were divided into 11 groups (1, 0%-9%; 2, 10%-19%; 3, 20%-29%; etc). Estimates of survival were calculated for OS, DSS, and DFS using the Kaplan-Meier method. The association between compliance score for the group of metrics and OS, DSS, and DFS was assessed using the log-rank test. Unadjusted Cox proportional hazards analysis was used to calculate the association between the overall compliance score and OS, DSS, and DFS. Multivariable Cox proportional hazards analyses were then performed for overall compliance, controlling for pT stage, pN stage, extracapsular spread, final margin status, and comorbidity. The Adult Comorbidity Evaluation-27 index comorbidity score was dichotomized to no or mild comorbidities (score of 0 or 1) and moderate or severe comorbidities (score of 2 or 3), pT stage was dichotomized into T1 and T2 vs T3 and T4, and pN stage was dichotomized into N0 and N1 vs N2 and N3. The association between compliance with the overall group of metrics and OS, DSS, and DFS was expressed as an adjusted hazard ratio and 95% CI. The same method was used to determine the association between compliance with individual quality metrics and OS, DSS, and DFS.
Clinical Care Signature
The clinical care signature is analogous to a gene expression signature: just as the combined gene expression alteration of a tumor may be associated with the tumor’s overall behavior, a small group of process-related care practices may also be associated with the overall quality of care. The clinical care signature was composed of 4 metrics associated with improved survival on univariable or multivariable analysis. Compliance with the clinical care signature was divided into 3 groups (1, 0% 50%; 2, 51%-99%; and 3, 100%). The method described above was used to determine the association between compliance with the clinical care signature and OS, DSS, and DFS.
Statistical Analysis
P ≤ .05 (2-tailed) was considered statistically significant. Data analysis was performed from January 26 to August 7, 2015, using SPSS, version 20.0 (SPSS Inc).
Results
Oncologic Characteristics
The demographic, oncologic, and treatment details of the 192 patients are presented in the eTable in the Supplement. Ninety-six of the tumors (50.0%) were located in the oral tongue. Sixty-four patients (33.3%) had advanced pathologic stage disease (stage T3 or T4), and 40 (21.4%) had pathologic N2 or N3 disease. Forty patients (20.8%) received adjuvant radiotherapy and 44 (22.9%) received adjuvant chemoradiation treatment. For the cohort of 192 patients with a median follow-up of 49 months, there were 66 patients who died during the study period, 41 patients who died of disease, and 80 patients with a death or recurrence. The 5-year OS was 68.5%, 5-year DSS was 74.8%, and 5-year DFS was 57.9%.
Compliance With Quality Metrics
Patient eligibility and compliance with process-related quality metrics are shown in the eFigure in the Supplement. For the overall group of 19 metrics, the median compliance rate was 82.8%. Compliance for individual care processes ranged from 19.7% for counseling on tobacco cessation to 93.6% for internal review of outside hospital pathologic findings. Compliance was greater than 80% for 10 of the 19 metrics.
Association Between Compliance With All Quality Metrics and Survival
When compliance scores for the group of all 19 metrics with face validity were grouped into deciles, there was no association between compliance with the group of metrics and improved OS, DSS, or DFS. On multivariable analysis adjusting for pT stage, pN stage, extracapsular spread, margin status, and comorbidity, there was no statistically significant association between increased compliance and improved survival.
Individual Quality Metrics Associated With Improved Survival
The association between compliance with individual quality metrics and OS, DSS, and DFS on univariable analysis is demonstrated in Table 2. There were no individual pretreatment or surveillance metrics for which increased compliance was associated with improved survival. Three treatment-related quality metrics were associated with improved survival: lymph node yield of 18 or more during elective neck dissection, no 30-day unplanned readmission, and no unplanned surgery within 14 days of the index surgery.
Table 2.
Univariable Analysis of Individual Quality Metrics and Survival
| Quality Metric | Hazard Ratio (95% CI) | ||
|---|---|---|---|
| OS | DSS | DFS | |
| Pretreatment | |||
| Time from referral to clinic visit ≤14 d | 1.08 (0.33-3.52) | 1.09 (0.26-4.62) | 1.18 (0.42-3.28) |
| Examination of pharynx and larynx before definitive operation | 0.94 (0.67-1.32) | 1.15 (0.48-2.73) | 1.19 (0.64-2.19) |
| cTNM stage documented | 1.03 (0.67-1.56) | 0.80 (0.31-2.03) | 1.33 (0.58-3.05) |
| Tobacco cessation counseling | 0.56 (0.19-1.65) | 0.83 (0.24-2.95) | 0.49 (0.17-1.43) |
| Dental evaluation | 1.14 (0.61-2.13) | 0.94 (0.43-2.08) | 1.15 (0.63-2.07) |
| Internal review of pathologic findings | 0.80 (0.25-2.64) | 1.44 (0.19-10.62) | 1.15 (0.36-3.72) |
| Tumor board presentation | 1.07 (0.64-1.77) | 0.93 (0.50-1.74) | 0.97 (0.62-1.51) |
| Treatment related | |||
| Time from initial evaluation to surgery ≤21 d | 1.09 (0.66-1.79) | 1.23 (0.65-2.33) | 0.92 (0.59-1.44) |
| Elective neck dissection with ≥18 lymph nodes | 0.30 (0.12-0.74)a | 0.22 (0.48-0.98)a | 0.48 (0.21-1.09)a |
| pTNM stage documented | 1.08 (0.52-2.26) | 1.45 (0.52-4.07) | 0.85 (0.46-1.57) |
| Margin status documented | 1.85 (1.06-3.23) | 2.19 (1.05-4.60) | 1.65 (1.01-2.68) |
| No unplanned surgery ≤14 d | 0.39 (0.21-0.73)a | 0.24 (0.12-0.48)a | 0.39 (0.22-0.71)a |
| No unplanned readmission ≤30 d after discharge | 0.30 (0.17-0.52)a | 0.34 (0.15-0.73)a | 0.43 (0.25-0.73)a |
| Referral for adjuvant radiotherapy if stage III or IV | 0.69 (0.31-1.53) | 0.90 (0.32-2.55) | 0.58 (0.27-1.23) |
| Referral for adjuvant chemoradiotherapy if ECS or positive margins | 0.85 (0.35-2.07) | 0.94 (0.28-3.13) | 1.08 (0.45-2.62) |
| Start of adjuvant therapy <6 wk postoperatively | 0.65 (0.34-1.25) | 0.54 (0.24-1.23) | 0.75 (0.40-1.39) |
| Posttreatment surveillance | |||
| Thyroid function testing ≤12 mo after radiotherapy | 0.95 (0.43-2.10) | 0.99 (0.35-2.78) | 1.11 (0.52-2.35) |
| Surveillance imaging <6 mo after therapy completion | 2.03 (1.12-3.69) | 2.44 (1.09-5.48) | 1.86 (1.09-3.19) |
| Multidisciplinary follow-up | 1.62 (0.57-4.60) | 2.25 (0.53-9.53) | 1.93 (0.68-5.50) |
Abbreviations: CRT, chemoradiotherapy; cTNM, clinical tumor, node, metastasis; DFS, disease-free survival; DSS, disease-specific survival; ECS, extracapsular spread; OS, overall survival; pTNM, pathologic tumor, node, metastasis.
Indicates variables significantly associated with improved survival.
The association between compliance with individual quality metrics and OS, DSS, and DFS was assessed in a multivariable Cox proportional hazards model controlling for pT, pN, extracapsular spread, margin status, and comorbidity. There were no individual pretreatment or surveillance metrics for which increased compliance was associated with improved survival. Three treatment-related quality metrics were associated with improved survival on multivariable analysis (Table 3): lymph node yield of 18 or more during elective neck dissection, no 30-day unplanned readmission, and referral for adjuvant radiotherapy for pathologic stage III or IV disease.
Table 3.
Multivariable Analysis of Individual Quality Metrics and Survival
| Quality Metric | Adjusted Hazard Ratio (95% CI) | ||
|---|---|---|---|
| OS | DSS | DFS | |
| Treatment related | |||
| Elective neck dissection with >18 lymph nodes | 0.28 (0.11-0.74)a | 0.08 (0.01-0.77)a | 0.48 (0.21-1.12) |
| No unplanned surgery <14 d | 0.82 (0.42-1.62) | 0.61 (0.28-1.32) | 0.59 (0.31-1.11) |
| No unplanned readmission <30 d after discharge | 0.39 (0.22-0.72)a | 0.44 (0.20-1.00)a | 0.58 (0.33-1.04) |
| Referral for adjuvant radiotherapy if stage III or IV | 0.24 (0.09-0.64)a | 0.21 (0.06-0.83)a | 0.24 (0.09-0.61)a |
Abbreviations: DFS, disease-free survival; DSS, disease-specific survival; OS, overall survival.
Indicates variables significantly associated with improved survival after adjusting for pT stage, pN stage, extracapsular spread, final margin status, and Adult Comorbidity Evaluation-27 index comorbidity score.
Twenty-five patients (13.0%) had a 30-day unplanned readmission. Of those patients, the most common readmission diagnoses were surgical site infections (n = 6), wound dehiscence (n = 4), hemorrhage (n = 3), re-resection of positive margins (n = 3), and nutritional complications (n = 3). Nineteen patients (9.9%) had an unplanned return to the operating room within 14 days of their index surgery, most commonly for re-resection of positive margins (n = 7), hemorrhage (n = 5), and wound complications (n = 5).
Clinical Care Signature
The 4 process-related quality metrics associated with improved survival on univariable or multivariable analysis were used to construct a clinical care signature, a small group of process-related care practices associated with the overall quality of care. Compliance with the clinical care signature was associated with significantly improved OS, DSS, and DFS on univariable analysis (log-rank test, P < .05 for each) (Figure 1). A multivariable model was constructed controlling for pT stage, pN stage, extracapsular spread, margin status, and comorbidity. Compliance with the clinical care signature remained associated with improved OS, DSS, and DFS on multivariable analysis (Figure 2).
Figure 1. Association Between Compliance With the “Clinical Care Signature” and Improved Survival on Univariable Analysis.
A, Kaplan-Meier estimate of overall survival (P < .05; log-rank test). B, Kaplan-Meier estimate of disease-specific survival (P < .05; log-rank test). C, Kaplan-Meier estimate of disease-free survival (P < .05; log-rank test). The clinical care signature is a small group of process-related care practices associated with the overall quality of care. All estimates are stratified by compliance with the clinical care signature quality metrics (100% vs 51%-99%. vs 50%).
Figure 2. Association Between Compliance With the “Clinical Care Signature” and Improved Survival on Multivariable Analysis.
A, Cox multivariable survival analysis for overall survival (100% vs ≤50%: adjusted hazard ratio [aHR], 4.2; 95% CI, 2.1-8.5; 100% vs 51%-99%: aHR, 1.7; 95% CI, 1.0-3.1). B, Cox multivariable survival analysis for disease-specific survival (100% vs ≤50%: aHR, 3.9; 95% CI, 1.7-9.0; 100% vs 51%-99%: aHR, 1.3; 95% CI, 0.6-2.9). C, Cox multivariable survival analysis for disease-free survival (100% vs ≤50%: aHR, 3.0; 95% CI, 1.5-5.8; 100% vs 51%-99%: aHR, 1.6; 95% CI, 0.9-2.7). The clinical care signature is a small group of process-related care practices associated with the overall quality of care. All estimates are stratified by compliance with the clinical care signature quality metrics after adjusting for pT stage, pN stage, extracapsular spread, margin status, and comorbidity.
Discussion
In this study, we found that compliance with individual quality metrics for a large group of patients with newly diagnosed oral cavity cancer was variable and ranged from 19.7% to 93.6%. Compliance for many of the metrics was low (eg, counseling patients on tobacco cessation and timely initiation of adjuvant therapy), but compliance for 9 of the metrics was 85% or more. For the 4 metrics that make up the clinical care signature, 135 patients (70.3%) received 100% compliance. The high compliance rate for the metrics in the clinical care signature suggests that they are already achieved in the majority of cases. Other institutions have published compliance rates for similar care processes and found them to be highly achievable.12
To date, little has been published analyzing quality metrics in head and neck cancer.12–15 What has been published for oral cavity cancer assesses compliance with a variety of metrics but does not correlate compliance with an outcome.12 Published studies for larynx cancer have been limited to elderly patients, and most of the metrics were related to surveillance, management of recurrent disease, and end-of-life care.14,15 None of the above-referenced studies assessed how performance of specific quality metrics was associated with survival outcomes.
These compliance data can be incorporated into an audit and feedback system at the surgeon or program level. Data on compliance with this group of 19 quality metrics was easily gathered from the medical record and required approximately 15 minutes per patient to collect. Knowing that compliance with these core quality metrics is associated with improved survival should facilitate motivation for quality improvement.
Individual Quality Metrics Associated With Improved Survival
In this study, 4 treatment-related quality metrics were associated with improved survival: lymph node yield of 18 or more during elective neck dissection, no unplanned readmission within 30 days, no unplanned surgery within 14 days, and referral for adjuvant radiotherapy for pathologic stage III or IV disease.
Lymph node yield has been suggested as a quality metric for colon,27 lung,28 and gastric cancer29 and endorsed as a quality metric for those cancers at a national level.30 Single- and multi-institution retrospective studies have suggested that lymph node yield of 18 or more during elective neck dissection for OCSCC is associated with improved survival.16,22,23,31 Our data lend further support for lymph node yield as a process-related quality metric for these patients.
Compliance with no 30-day unplanned readmissions was associated with improved survival. Unplanned readmissions represent an interaction between patient factors, transitions of care, and postoperative complications.24 In this study, most of the unplanned readmissions were related to surgical site infections, wound dehiscence, postoperative hemorrhage, and re-resection of positive margins. Surgical site infections have been targeted as a quality metric by the Surgical Care Improvement Project, which is endorsed by the Joint Commission and the National Quality Forum.32 Postoperative hemorrhage and positive margins are the criteria used for stopping rules in the ongoing ECOG 3311 trial,33 and thus are tacitly acknowledged as markers of surgical quality.
Compliance with no unplanned surgery within 14 days of the index surgery was associated with improved survival. As with unplanned 30-day readmissions, this metric may not appear to be in the surgeon’s control and thus not be a fair marker of quality. However, the findings of this study may suggest otherwise, as the majority of cases of unplanned surgery within 14 days were related to re-resection of positive margins or postoperative hemorrhage. While both of these circumstances occur because of a complex interplay between surgeon, patient comorbidities, and tumor biology, it seems intuitive that surgical quality is a large factor in determining the frequency of these occurrences. Correlation of this metric and no 30-day unplanned readmission with improved survival are both surprising findings and warrant further investigation.
Clinical Care Signature
A major finding in this study was the identification of a group of core quality metrics for which increasing compliance was associated with improved survival. We termed this core group of metrics a clinical care signature to suggest an analogy with gene expression arrays. Just as the combined expression profile of a few key genes can be associated with predicting overall tumor behavior and survival, physician performance on a few key metrics can also be associated with survival outcomes. Prior studies have shown that in addition to the direct effect that compliance with performance metrics has on patient outcomes, it is also a marker of other unmeasured aspects of quality healthcare.34 We suspect that a similar phenomenon is being observed here.
Despite these promising findings, this study has several limitations. The retrospective nature of the study is limited by the accuracy of the medical record. It is a single-institution study, so the generalizability of the results is not known. It is also unknown whether these data apply to other subsites of the head and neck. Two variables (no unplanned surgery ≤14 days and no unplanned readmission ≤30 days) were dichotomized into yes or no. Future research is needed to determine acceptable risk-adjusted frequencies for these events. It is also possible that compliance with some of the quality metrics assessed in this study are in fact truly associated with improved survival outcomes, but these associations were not captured because the study was underpowered to detect them. Finally, this model requires additional testing in an independent validation cohort of patients.
Although not a limitation of this study itself, application of these data may have unintended consequences (eg, electively dissecting unindicated levels of the neck to increase lymph node yield). Unintended consequences stemming from well-intentioned quality metrics have been previously described, such as an increase in the incidence of Clostridium difficile infection after implementation of a quality metric for antibiotic therapy in community-acquired pneumonia.3
Conclusions
This study identified process-related quality metrics that correlated with improved survival in patients with surgically managed OCSCC. When these quality metrics are grouped together as a clinical care signature, increasing compliance with the bundle of metrics is associated with improved survival even after adjusting for tumor-related prognostic factors and comorbidity. The significant quality metrics in this study were all related to treatment factors rather than pretreatment evaluation or posttreatment surveillance, suggesting that additional research is needed in those areas. Validation of the current findings in an independent population and then in a multi-institutional fashion appears warranted.
Supplementary Material
Acknowledgments
Funding Support: This research was funded in part by the P30 Research Center for Auditory and Vestibular Studies and grant P30DC04665 from the National Institutes of Health NIDCD.
Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Footnotes
Author Contributions: Drs Graboyes and Nussenbaum had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Graboyes, Kallogjeri, Al-Gilani, Stadler, Nussenbaum.
Acquisition, analysis, or interpretation of data: Graboyes, Gross, Kallogjeri, Piccirillo, Nussenbaum. Drafting of the manuscript: Graboyes, Kallogjeri, Nussenbaum.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Graboyes, Kallogjeri, Piccirillo. Administrative, technical, or material support: Nussenbaum.
Study supervision: Nussenbaum.
Conflict of Interest Disclosures: None reported.
Disclaimer: Dr Piccirillo is the editor of JAMA Otolaryngology–Head & Neck Surgery but was not involved in the editorial review or the decision to accept the manuscript for publication.
Additional Contributions: Douglas Adkins, MD, Jason Diaz, MD, Hiram Gay, MD, Wade Thorstad, MD, and Tanya Wildes, MD, Washington University School of Medicine, are the other members of the multidisciplinary head and neck cancer team who helped develop the quality metrics. They were not compensated for their contributions.
Contributor Information
Evan M. Graboyes, Department of Otolaryngology–Head and Neck Surgery, Washington University School of Medicine, St Louis, Missouri.
Jennifer Gross, Department of Otolaryngology–Head and Neck Surgery, Washington University School of Medicine, St Louis, Missouri.
Dorina Kallogjeri, Department of Otolaryngology–Head and Neck Surgery, Washington University School of Medicine, St Louis, Missouri; Editor, JAMA Otolaryngology–Head & Neck Surgery.
Jay F. Piccirillo, Department of Otolaryngology–Head and Neck Surgery, Washington University School of Medicine, St Louis, Missouri.
Maha Al-Gilani, Department of Otolaryngology–Head and Neck Surgery, Washington University School of Medicine, St Louis, Missouri.
Michael E. Stadler, Department of Otolaryngology and Communication Sciences, Medical College of Wisconsin, Milwaukee.
Brian Nussenbaum, Department of Otolaryngology–Head and Neck Surgery, Washington University School of Medicine, St Louis, Missouri.
References
- 1.Agency for Healthcare Research and Quality, US Dept of Health and Human Services. Your guide to choosing quality health care. http://archive.ahrq.gov/consumer/qnt/. Accessed May 30, 2015.
- 2.McGlynn EA, Schneider EC, Kerr EA. Reimagining quality measurement. N Engl J Med. 2014;371(23):2150–2153. doi: 10.1056/NEJMp1407883. [DOI] [PubMed] [Google Scholar]
- 3.Cassel CK, Conway PH, Delbanco SF, Jha AK, Saunders RS, Lee TH. Getting more performance from performance measurement. N Engl J Med. 2014;371(23):2145–2147. doi: 10.1056/NEJMp1408345. [DOI] [PubMed] [Google Scholar]
- 4.Gourin CG, Couch ME. Defining quality in the era of health care reform. JAMA Otolaryngol Head Neck Surg. 2014;140(11):997–998. doi: 10.1001/jamaoto.2014.2086. [DOI] [PubMed] [Google Scholar]
- 5.Nicholas LH, Osborne NH, Birkmeyer JD, Dimick JB. Hospital process compliance and surgical outcomes in medicare beneficiaries. Arch Surg. 2010;145(10):999–1004. doi: 10.1001/archsurg.2010.191. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Donabedian A. The Definition of Quality and Approaches in Assessment. Ann Arbor, MI: Health Administration Press; 1980. [Google Scholar]
- 7.Birkmeyer JD, Dimick JB, Birkmeyer NJ. Measuring the quality of surgical care: structure, process, or outcomes? J Am Coll Surg. 2004;198(4):626–632. doi: 10.1016/j.jamcollsurg.2003.11.017. [DOI] [PubMed] [Google Scholar]
- 8.Shafi S, Parks J, Ahn C, et al. Centers for Medicare & Medicaid Services quality indicators do not correlate with risk-adjusted mortality at trauma centers. J Trauma. 2010;68(4):771–777. doi: 10.1097/TA.0b013e3181d03a20. [DOI] [PubMed] [Google Scholar]
- 9.Shafi S, Barnes SA, Rayan N, et al. Compliance with recommended care at trauma centers: association with patient outcomes. J Am Coll Surg. 2014;219(2):189–198. doi: 10.1016/j.jamcollsurg.2014.04.005. [DOI] [PubMed] [Google Scholar]
- 10.Stulberg JJ, Delaney CP, Neuhauser DV, Aron DC, Fu P, Koroukian SM. Adherence to surgical care improvement project measures and the association with postoperative infections. JAMA. 2010;303(24):2479–2485. doi: 10.1001/jama.2010.841. [DOI] [PubMed] [Google Scholar]
- 11.The development of quality of care measures for oral cavity cancer. Arch Otolaryngol Head Neck Surg. 2008;134(6):672. doi: 10.1001/archotol.134.6.672. [DOI] [PubMed] [Google Scholar]
- 12.Hessel AC, Moreno MA, Hanna EY, et al. Compliance with quality assurance measures in patients treated for early oral tongue cancer. Cancer. 2010;116(14):3408–3416. doi: 10.1002/cncr.25031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Weber RS, Lewis CM, Eastman SD, et al. Quality and performance indicators in an academic department of head and neck surgery. Arch Otolaryngol Head Neck Surg. 2010;136(12):1212–1218. doi: 10.1001/archoto.2010.215. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Gourin CG, Frick KD, Blackford AL, et al. Quality indicators of laryngeal cancer care in the elderly. Laryngoscope. 2014;124(9):2049–2056. doi: 10.1002/lary.24593. [DOI] [PubMed] [Google Scholar]
- 15.Gourin CG, Starmer HM, Herbert RJ, et al. Quality of care and short- and long-term outcomes of laryngeal cancer care in the elderly. Laryngoscope. 2015;125(10):2323–2329. doi: 10.1002/lary.25378. [DOI] [PubMed] [Google Scholar]
- 16.Jaber JJ, Zender CA, Mehta V, et al. Multi-institutional investigation of the prognostic value of lymph nodel yield in advanced-stage oral cavity squamous cell carcinoma. Head Neck. 2014;36(10):1446–1452. doi: 10.1002/hed.23475. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Gilbert R, Devries-Aboud M, Winquist E, Waldron J, McQuestion M. The management of head and neck cancer in Ontario. https://www.cancercare.on.ca/common/pages/UserFile.aspx?fileId=58592. Published December 5, 2009. Accessed July 26, 2015.
- 18.Healthcare Improvement Scotland. Cancer clinical quality performance indicators (QPIs) http://www.healthcareimprovementscotland.org/our_work/cancer_care_improvement/cancer_qpis/quality_performance_indicators.aspx. Published August 8, 2013. Accessed July 26, 2015.
- 19.Committee on Quality of Health Care in America, Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001. [Google Scholar]
- 20.Kallogjeri D, Piccirillo JF, Spitznagel EL, Jr, Steyerberg EW. Comparison of scoring methods for ACE-27: simpler is better. J Geriatr Oncol. 2012;3(3):238–245. doi: 10.1016/j.jgo.2012.01.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Pfister DG, Spencer S, Brizel DM, et al. National Comprehensive Cancer Network Head and neck cancers, version 2.2014: clinical practice guidelines in oncology. J Natl Compr Canc Netw. 2014;12(10):1454–1487. doi: 10.6004/jnccn.2014.0142. [DOI] [PubMed] [Google Scholar]
- 22.Ebrahimi A, Clark JR, Zhang WJ, et al. Lymph node ratio as an independent prognostic factor in oral squamous cell carcinoma. Head Neck. 2011;33(9):1245–1251. doi: 10.1002/hed.21600. [DOI] [PubMed] [Google Scholar]
- 23.Ebrahimi A, Clark JR, Amit M, et al. Minimum nodal yield in oral squamous cell carcinoma: defining the standard of care in a multicenter international pooled validation study. Ann Surg Oncol. 2014;21(9):3049–3055. doi: 10.1245/s10434-014-3702-x. [DOI] [PubMed] [Google Scholar]
- 24.Graboyes EM, Liou TN, Kallogjeri D, Nussenbaum B, Diaz JA. Risk factors for unplanned hospital readmission in otolaryngology patients. Otolaryngol Head Neck Surg. 2013;149(4):562–571. doi: 10.1177/0194599813500023. [DOI] [PubMed] [Google Scholar]
- 25.Center for Medicare & Medicaid Services. CMS HQI demonstration project: composite quality score methodology overview. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/downloads/HospitalCompositeQualityScoreMethodologyOverview.pdf. Published March 26, 2004. Accessed May 24, 2015.
- 26.Sinha P, Hackman T, Nussenbaum B, Wu N, Lewis JS, Jr, Haughey BH. Transoral laser microsurgery for oral squamous cell carcinoma: oncologic outcomes and prognostic factors. Head Neck. 2014;36(3):340–351. doi: 10.1002/hed.23293. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Chang GJ, Rodriguez-Bigas MA, Skibber JM, Moyer VA. Lymph node evaluation and survival after curative resection of colon cancer: systematic review. J Natl Cancer Inst. 2007;99(6):433–441. doi: 10.1093/jnci/djk092. [DOI] [PubMed] [Google Scholar]
- 28.Ludwig MS, Goodman M, Miller DL, Johnstone PA. Postoperative survival and the number of lymph nodes sampled during resection of node-negative non–small cell lung cancer. Chest. 2005;128(3):1545–1550. doi: 10.1378/chest.128.3.1545. [DOI] [PubMed] [Google Scholar]
- 29.Al-Refaie WB, Gay G, Virnig BA, et al. Variations in gastric cancer care: a trend beyond racial disparities. Cancer. 2010;116(2):465–475. doi: 10.1002/cncr.24772. [DOI] [PubMed] [Google Scholar]
- 30.American College of Surgeons. CoC quality of care measures. https://www.facs.org/quality-programs/cancer/ncdb/qualitymeasures. Published March 26, 2004. Accessed May 24, 2015.
- 31.Shah JL, Kaplan M, Divi V, Le Q, Hara W. Effect of the extent of lymph node dissection on overall survival in patients treated for oral cavity squamous cell carcinoma. J Clin Oncol. 2015;33(15 suppl):6075. [Google Scholar]
- 32.The Joint Commission. Surgical care improvement project. http://www.jointcommission.org/surgical_care_improvement_project/. Published October 16, 2014. Accessed June 6, 2015.
- 33.ClinicalTrials.gov, US National Institutes of Health. Transoral surgery followed by low-dose or standard-dose radiation therapy with or without chemotherapy in treating patients with HPV positive stage III-IVA oropharyngeal cancer. https://clinicaltrials.gov/ct2/show/NCT01898494. Updated November 19, 2015. Accessed June 21, 2015.
- 34.Werner RM, Bradlow ET, Asch DA. Does hospital performance on process measures directly measure high quality care or is it a marker of unmeasured care? Health Serv Res. 2008;43(5, pt 1):1464–1484. doi: 10.1111/j.1475-6773.2007.00817.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.