Variation in long-term oncologic outcomes by type of cancer center accreditation: An analysis of a SEER-Medicare population with pancreatic cancer

Abstract

Background:

Cancer center accreditation is designed to identify centers that provide high-quality cancer care. This also guides patients and referring physicians towards centers of excellence for specialized care. We sought to examine if cancer center accreditation was associated with improved long-term oncologic outcomes in patients with pancreatic adenocarcinoma.

Methods:

Using the SEER-Medicare database, we identified patients who underwent pancreatectomy for pancreatic adenocarcinoma from 1996 to 2013. Hospitals were categorized into three groups: National Cancer Institute-designated (NCI-designated) centers, Commission on Cancer (CoC)-accredited centers, and “non-accredited” (NA) centers. Multilevel mixed-effects models were used to calculate adjusted examined lymph nodes, disease-specific survival (DSS), and overall survival (OS).

Results:

We identified 5,118 patients who underwent pancreatectomy at 632 hospitals (41.0% NA, 49.6% CoC, 9.4% NCI). NCI-designated centers had a greater median number of lymph nodes examined compared with CoC-accredited or NA centers (14 vs. 10 vs. 11.0 nodes, respectively; p < 0.001). Patients treated at NCI centers had a higher 5-year DSS compared to those treated at CoC or NA centers (31.2% vs. 23.6% vs. 23.0%, respectively; p < 0.001). Finally, patients treated at NCI centers had a higher 5-year OS compared to those treated at CoC or NA centers (23.5% vs. 18.9% vs. 17.9%, respectively; p < 0.001). The associations held true when adjusted analyses were performed.

Conclusion:

Patients with resected pancreatic cancer treated at NCI-designated centers were associated with improved long-term oncologic outcomes. There was no difference between CoC-accredited centers compared with NA centers. Meticulous validation of accreditation is warranted globally prior to implementation.

Introduction

Accreditation programs are established to improve the quality of care delivered through the measurement and monitoring of care processes and metrics, with the aim of identifying shortcomings and implementing quality improvement initiatives. As such, hospital accreditation provides an endorsement of the quality of care provided, and guides patients and referring physicians towards centers of excellence for specialized care. Although the benefits of accreditation to hospitals may seem obvious, the process is expensive and time-consuming,¹ with little high-quality evidence supporting beneficial impact on patient care. Therefore, accreditation has not been sought by all hospitals, and adoption is arguably more likely to favor smaller institutions whose patients might benefit from provision of standardized care with rigorous external oversight.

In cancer care in the United States (US), the two main bodies of accreditation are the National Cancer Institute (NCI)² and the American College of Surgeons (ACS) Commission on Cancer (CoC).³ Presently, most studies have only evaluated the association of cancer center accreditation with short-term outcomes and care processes, including index hospitalization outcomes.^4–6 However, benchmarks for cancer center accreditation should more importantly be aimed at optimizing long-term cancer outcomes. It is still unknown if receiving care at NCI-designated or CoC-accredited centers results in improved oncologic and long-term survival outcomes for patients with cancer.

The aim of this analysis was to determine the association between cancer center accreditation and long-term oncologic outcomes. Specifically, we sought to examine the number of lymph nodes examined, disease-specific survival (DSS), and overall survival (OS), as core measures of long-term cancer outcomes in patients who had undergone pancreatectomy for pancreatic cancer, given that pancreatic cancer is a cancer with complex multidisciplinary care requirements that demands treatment coordination at specialized centers.

Methods

Data sources and study population

This study was approved by the Massachusetts General Hospital Institutional Review Board (Protocol ID 2017P001741). All patients who underwent pancreatectomy for pancreatic adenocarcinoma from 1996 to 2013 were identified from the linked Surveillance, Epidemiology, and End Results (SEER)-Medicare linked database. The SEER program covers approximately 28% of the population in the United States, and is the only comprehensive population-based registry that provides information on stage of cancer at the time of diagnosis and patient survival data through its linkage with the National Death Index. The SEER-Medicare database links SEER data to the Centers for Medicare and Medicaid Services (CMS) Medicare claims database to provide longitudinal information on health care services provided from the time of patients’ Medicare eligibility until death. The linkage is a collaborative effort of the NCI, the SEER registries, and the CMS.

Patients undergoing a pancreatectomy for pancreatic adenocarcinoma from 1996 to 2013 were included. The pancreas was first identified as the primary organ site with the site recodes 250–254, 257, 258 and 259. Adenocarcinoma was then identified as the primary histology as defined by the 3rd edition International Classification of Disease for Oncology (ICD-O-3) codes 8021, 8140, 8141, 8143, 8144, 8211, 8230, 8255, 8260–8263, 8323, 8440, 8441, 8450, 8453, 8470, 8471, 8480, 8481, 8500, 8503, 8521, 8550, 8560, 8570, and 8576.^7,8 Pancreatectomy was defined as patients with the 9th revision International Classification of Diseases (ICD-9) procedure codes of 52.5, 52.51, 52.52, 52.53, 52.59, 52.6, and 52.7.

Patients with more than one primary cancer or missing cancer staging information were excluded from the analysis. Additionally, patients who were younger than 65 years of age in the Medicare dataset were also excluded to exclude younger patients who were enrolled based on disability or end stage renal disease requiring dialysis or transplantation.

The exposure, cancer center accreditation, was designated based on the accreditation status for the hospital where the patient underwent pancreatectomy for that specific year. There were only few facilities that were both NCI-designated and CoC-accredited (5.7%), which were excluded in order to allow interpretation of the effect of each these programs separately. Hospital volume was defined in hospital years, which was determined by summing the total number of patients treated at each facility for each year. Patients’ comorbidities were captured using the Charlson’s Comorbidity Index (CCI),⁹ which was adapted for use with ICD-9 codes in administrative databases and validated in a Medicare cohort.¹⁰ Patient age was redefined into ordinal categories of 65–74, 75–84, 85–89, and ≥90 years old to avoid assumption of linearity. Tumor stages were defined using the SEER historic staging classification, with the following categories of localized (confined entirely to organ of origin), regional (extended beyond the limits of the organ of origin directly into surrounding organs or tissues, or regional lymph nodes) and distant tumors (spread to distant organs or lymphatic system remote from the primary tumor). The follow up time for DSS was defined as the minimum time between the date of surgery to the date of death attributed to cancer, or end of study period. The follow up time for OS was defined as the minimum time between the date of surgery to either the date of death (any cause), or end of study period.

Statistical analysis

The Chi-Squared test was used to compare categorical variables. The Shapiro Wilk test was used to assess for normality of the distribution for continuous variables. All results for continuous variables were expressed as mean ± standard deviation, and skewed variables were expressed as median and interquartile range. A Student t-test was used to compare continuous variables with a normal distribution, and Kruskal-Wallis for continuous variables with a non-normal distribution. A multilevel linear mixed-effects model was used for adjusted analyses of continuous outcomes, assigning fixed-effects to patient-level predictors and random-effects to individual hospitals to account for intra-class correlation for patients nested within the same hospital. Similarly, multilevel mixed-effects parametric survival-time models were used for the survival analyses. Covariates with p-values <0.1 and clinically meaningful variables were included in the models. All tests were two-sided and a significance level of p < 0.05 was considered to indicate statistical significance. All statistical analyses were performed using Intercooled Stata software, version 15.1 (StataCorp, College Station, TX).

Results

Patient demographics

We identified 107,480 patients who had a diagnosis of pancreatic cancer from 1996 to 2013. After excluding patients with more than one primary tumor, those with non-adenocarcinoma tumors, those who did not undergo surgical resection, and those younger than 65 years old, 5,118 patients were eventually included in the study (Fig. 1). Of these, 989 (19.3%) were treated at an NCI-designated center, 2,070 (40.5%) were treated at a CoC-accredited center, and 2,059 (40.2%) patients were treated at a NA center. NCI-designated centers had higher median annual volume (6 cases per year) when compared to CoC-accredited (2 cases per year) and NA centers (3 cases per year) (p < 0.001).

Fig. 1.

Flow diagram of the study population.

There were no differences in gender, age and CCI score between patients who were treated at NCI, CoC, and NA centers (Table 1). However, non-Hispanic white patients were more likely to be treated at NCI-designated centers (87.3%) compared with CoC-accredited centers (84.1%) and NA centers (83.0%) (p = 0.001), whereas black patients were more likely to be treated at NA centers (9.2%) compared with NCI-designated centers (4.4%) and CoC-accredited centers (7.4%) (p = 0.001).

Table 1.

Demographics of patients undergoing pancreatectomy for pancreatic adenocarcinoma at non-accredited, CoC-accredited and NCI-designated centers.

	Non-accredited	CoC-accredited	NCI-designated	p-value
Female	1,117 (54.3)	1,117 (54.0)	542 (54.8)	0.909
Age				0.231
≥65 & <75	1,179 (57.3)	1,135 (54.8)	525 (53.1)
≥75 & <85	795 (38.6)	860 (41.6)	423 (42.8)
≥85 & <90a	>74 (>3.6)	>64 (>3.1)	>30 (>3.0)
≥90a	<11 (<0.5)	<11 (<0.5)	<11 (<1.1)
Race
Non-Hispanic Whites	1,705 (83.0)	1,739 (84.1)	860 (87.3)	0.001
Blacks	189 (9.2)	153 (7.4)	43 (4.4)
Hispanics	28 (1.4)	32 (1.6)	13 (1.3)
Asians	79 (3.8)	72 (3.5)	36 (3.7)
Others	54 (2.6)	73 (3.5)	33 (3.4)
CCI				0.215
≤2	544 (26.4)	504 (24.4)	273 (27.6)
3	295 (14.3)	331 (16.0)	142 (14.4)
>3	1,220 (59.3)	1,235 (59.7)	574 (58.0)

^aN suppressed in accordance with SEER-Medicare guidelines to mask cells that may be < 11 and ensure patient confidentiality.

Patients treated at CoC-accredited centers had larger tumors (3.4 cm) when compared to those treated at NCI (3.1 cm) and NA centers (3.1 cm, p = 0.002). However, there were no differences in tumor location or the proportion of patients with positive lymph node disease (Table 2). There were no differences in the proportion of patients who received adjuvant therapy at NCI (21.0%), CoC (21.8%), and NA centers (20.8%) (p = 0.749).

Table 2.

Tumor characteristics of patients with pancreatic adenocarcinoma treated at non-accredited, CoC-accredited and NCI-designated centers.

	Non-accredited	CoC-accredited	NCI-designated	p-value
Tumor location				0.204
Head	1,495 (72.6)	1,514 (73.1)	751 (75.9)
Body/tail	325 (15.8)	325 (15.7)	150 (15.2)
Other	239 (11.6)	231 (11.2)	88 (8.9)
Positive lymph nodes	1,185 (61.8)	1,208 (62.2)	597 (65.2)	0.191
Tumor size	3.1 (2.5–4.2)	3.4 (2.5–4.5)	3.1 (2.5–4.0)	0.002
Historic stage				0.057
Localized	290 (14.1)	309 (14.9)	112 (11.3)
Regional	1,544 (75.0)	1,539 (74.3)	780 (78.9)
Distant	225 (10.9)	224 (10.8)	97 (9.8)
Receipt of adjuvant therapy	379 (20.8)	401 (21.8)	873 (21.0)	0.749

Number of lymph nodes examined

Patients treated at an NCI-designated center were associated with a greater number of lymph nodes examined (median 14 nodes, IQR 8–20 nodes) after pancreatectomy compared to patients treated at CoC-accredited (median 10 nodes, IQR 5–15 nodes) and NA centers (median 11 nodes, IQR 5–17 nodes, p < 0.001). This association persisted on a linear mixed-effects model when adjusting for patient demographics, tumor location, tumor size, and disease stage. Patients treated at NCI-designated centers had 2.3 more lymph nodes examined (95% C.I. 1.1–3.4, p < 0.001) compared to patients treated at NA centers (Table 3). The predictors of increased number of lymph nodes examined included hospital volume (0.4 more lymph nodes per case per year, 95% C.I. 0.3–0.5, p < 0.001), patients with pancreatic head tumors (2.1 more lymph nodes compared to body or tail tumors, 95% C.I. 1.4–2.9, p < 0.001), regional tumors (3.3 more lymph nodes compared to localized tumors, 95% C.I. 2.6–4.1, p < 0.001) and advanced tumors (2.6 more lymph nodes compared to localized tumors, 95% C.I. 1.6–3.7, p < 0.001).

Table 3.

Multivariate linear regression depicting factors independently associated with number of lymph nodes examined after pancreatectomy for pancreatic adenocarcinoma.

	No. of lymph nodes examined	95% C.I.	p-value
Facility
NA centers	Reference	Reference	Reference
CoC centers	+0.312	−0.866 – +0.929	0.946
NCI centers	+2.265	+1.107 – +3.424	<0.001
Hospital volume	+0.409	+0.316 – +0.502	<0.001
Female	+0.801	+0.302 – +1.299	0.002
Age (years)
≥65 & <75	Reference	Reference	Reference
≥75 & <85	−0.473	−0.984 – +0.384	0.070
≥85 & <90	−0.630	−2.042 – +0.782	0.382
≥90	−0.558	−4.864 – +3.748	0.800
Race
Non-Hispanic Whites	Reference	Reference	Reference
Blacks	+0.974	+0.0153 – +1.932	0.046
Hispanics	+0.143	−2.094 – +2.380	0.900
Asians	+0.898	−0.586 – +2.383	0.236
Indians/others	−0.200	−1.744 – +1.343	0.799
CCI score
≤2	Reference	Reference	Reference
3	−0.478	−1.277 – +0.320	0.241
≥3	+0.318	−0.285 – +0.921	0.301
Tumor location
Body/tail	Reference	Reference	Reference
Head	+2.111	+1.371 – +2.852	<0.001
Others	+2.207	+1.185 – +3.229	<0.001
Tumor size (cm)	+0.002	−0.011 – +0.014	0.759
Tumor stage
Localized	Reference	Reference	Reference
Regional	+3.339	+2.572 – +4.107	<0.001
Advanced	+2.646	+1.568 – +3.725	<0.001

Survival analysis

Treatment at NCI-designated centers was associated with a longer median DSS (24.1 months) compared with CoC-accredited centers (16.7 months) and NA centers (17.4 months) (p < 0.001) (Fig. 2). The 5-year DSS rates for patients treated at NCI, CoC, and NA centers were 31.2%, 23.6% and 23.0%, respectively (p < 0.001). After adjusting for patient demographics and tumor characteristics, this association persisted, with patients treated at NCI-designated centers demonstrating enhanced DSS (HR 0.843, 95% C.I. 0.719–0.988, p = 0.027) compared to those treated at NA centers. There were no differences in DSS between patients treated at CoC-accredited centers (HR 0.902, 95% C.I. 0.800–1.016, p = 0.090) when compared to patients treated at NA centers. Tumor size, number of lymph nodes examined, lymph node involvement, and tumor stage were all independently associated with DSS.

Fig. 2.

Kaplan Meier curves depicting the (A) disease-specific survival and (B) overall survival for patients with pancreatic adenocarcinoma treated at NCI, CoC and NA centers. Hazard ratio adjusted for gender, race, age, Charlson comorbidity index score, tumor size, number of lymph nodes examined, number of positive lymph nodes and tumor historic stage.

*, denotes statistical significance.

NA, non-accredited centers; CoC, Commission on Cancer-accredited centers; NCI, National Cancer Institute-designated centers; HR, hazard ratio; CI, confidence interval.

Treatment at NCI centers was also associated with a longer median OS (20.3 months) compared with CoC-accredited centers (14.8 months) and NA centers (15.4 months) (p < 0.001) (Fig. 2). The 5-year OS rates for patients treated at NCI, CoC, and NA centers were 23.5%, 18.9% and 17.9%, respectively (p < 0.001). This association persisted on a mixed-effects parametric survival-time model when adjusting for patient demographics and tumor characteristics. Indeed, patients treated at NCI centers demonstrated better OS (HR 0.863, 95% C.I. 0.753–0.989, p = 0.035) when compared to those treated at NA centers. There were no differences in OS between patients treated at CoC-accredited centers (HR 0.962, 95% C.I. 0.864–1.070, p = 0.472) compared to patients treated at NA centers. Age, CCI scores, tumor size, number of lymph nodes examined, lymph node involvement, and tumor stage were all also independently associated with OS.

Sensitivity analyses were performed to exclude patients who died during index hospitalization. There was no difference in the persistent association between receiving treatment at NCI-designated centers and improved DSS and OS.

Discussion

In cancer care in the US, the NCI and CoC accreditation bodies have served to implement and maintain set quality standards that distinguish hospitals that provide high quality cancer care. However, most benchmarks utilize processes of care and are structural in nature. Unfortunately, little is known about how these translate to long-term oncologic and survival outcomes.^2,3 In an analysis of Medicare patients who underwent pancreatectomy for pancreatic adenocarcinoma, we found that patients treated at NCI-designated centers had more lymph nodes examined, and demonstrated enhanced DSS and OS compared to patients treated at CoC or NA centers, even after adjusting for hospital volume. Importantly, there were no significant differences in the number of lymph nodes examined, DSS, or OS between patients treated at CoC and NA centers. Our study demonstrates that NCI designation was the only accreditation process associated with improved long-term oncologic outcomes in the US. The implications of this finding may be substantial, with global implications on accreditation of centers providing specialized care. The etiology of this finding is likely multifactorial.

In the last few years, a seismic shift was encountered in bariatric surgery in the US. In 2006, CMS restricted coverage of bariatric surgery to hospitals designated as centers of excellence by the ACS, with the aim of centralizing procedures to high-performing centers only.¹¹ In 2013, investigators critically evaluated the policy and demonstrated no association between center of excellence designation and patient outcomes following bariatric surgery.¹² In fact, the only notable result of centralization of care was the burden associated with additional travel as a result of coverage restriction,¹³ thus, leading to its eventual repeal.¹⁴ As such, critical evaluation of accreditation is imperative in evaluating process measures implemented prior to care centralization to fewer centers of “perceived” excellence.

The CoC accreditation body defines structural and process of care standards, which do not appear to translate to improved long-term survival. A potential explanation is that CoC accreditation standards are simply easier to achieve without validated processes in place, whereas NCI designation is more rigorous with superior processes and more stringent standards that need to be demonstrated before accreditation is awarded. Case in point, in 2011, there were only 24 NCI-designated comprehensive cancer centers designated, but 145 CoC-accredited centers. A survey of staff at CoC-accredited centers reported that 32% cited “marketing decisions” and 36% “wanted access to the National Cancer Data Base and tools” as reasons their facilities decided to seek or maintain accreditation respectively.¹⁵ While centers may have met minimal structural and process of care standards, the implementation of standards lacking meaningful effect may have accounted for the absence of association with improved survival observed in this analysis. On the other hand, the NCI designates accreditation based on a facility’s expertise in laboratory, clinical, behavioral, and population-based research, which appear to be more impactful, and yield a culture of scientific excellence, guideline compliance,¹⁶ multidisciplinary collaboration, and effective treatment coordination with access to cutting edge clinical trials, which may have collectively accounted for the association with the improved oncologic outcomes observed.

Interestingly, our findings are congruent with a separate study reported by David and colleagues, which demonstrated that CoC accreditation was not associated with improved survival in patients with non-small cell lung cancer.¹⁷ More recently, a report by Lam and colleagues demonstrated that hospitals accredited by the Joint Commission in the United States conferred no benefits in patient mortality and satisfactions scores when compared to NA hospitals.¹⁸ The issue of ineffective accreditation featured in a recent editorial scrutinizing the laborious process associated with accreditation.¹⁹ Its sole author proposed that ineffective processes were potentially due to misguided prioritization of accreditation metrics, and highlighted the need for further investigation into the effectiveness of the different types of accreditation. Based on these reports, future studies should establish the underlying reasons for the lack of efficacy of CoC accreditation, including investigation of the individual elements of the NCI accreditation process that appear to contribute to improved survival of patients receiving treatment at NCI-designated centers.

Our study also revealed an additional disparity. Non-Hispanic white patients were more likely to receive treatment at NCI-designated centers, whereas black patients were more likely to receive treatment at NA centers. This finding is in keeping with several recent reports from a multitude of National databases demonstrating similar concerning trends.²⁰ While this likely demonstrates under-representation of minorities in the participation in cancer clinical trials in the United States, further study into the mechanisms of diagnosis, referral, and access are clearly warranted.^20–22 Importantly, our study excludes payer status as a likely cause of this disparity,²³ since our study analyzes a Medicare cohort. This suggests that such disparities extend beyond just insurance coverage, and, therefore, require urgent attention.^24,25

This study should be interpreted in the context of its design. We designated the cancer center accreditation status of facilities based on the center at which the patients underwent their index pancreatectomy, which was arguably the most specialized portion of the multidisciplinary treatment plan. In addition, accredited centers offer additional oncology-related services such as screening programs, which although less pertinent to pancreas cancer, the impact of such programs are hard to evaluate acutely given the impact may occur at a population level over years to decades. In this study, we evaluated a Medicare cohort only. However, this was helpful in the context of this study since greater than two-thirds of all patients with pancreatic cancer are over 65 years of age. Neo-adjuvant therapy use was not assessed because utilization rate was low over the study period of 1996–2013.

In conclusion, patients with pancreatic cancer who received their pancreatectomy at NCI-designated centers were associated with improved long-term oncologic outcomes compared to patients receiving treatment at CoC-accredited and NA centers in the US. There were no differences in outcomes between CoC-accredited hospitals and NA hospitals. Future studies into individual NCI accreditation processes are warranted to identify elements of cancer center accreditation that contribute to better oncologic and survival outcomes, in order to enhance care for all patients undergoing cancer treatment in the US.