Abstract
Background
Examination of at least 12 lymph nodes in the staging of colon cancer (CC) was recommended by the National Comprehensive Cancer Network in 2000; however, rates of an adequate examination remain low. This study compares the impact of the hospital contextual variance against that of the operating surgeon on delivery of an adequate lymph node examination.
Study Design
Retrospective analysis of California Cancer Registry data for all CC operations (2001–2006). Hierarchical models predicted the adequacy of lymph node examination as a function of patient, surgeon, and hospital characteristics. Models were created using penalized quasi-likelihood approximation with 2nd order Taylor linearization as implemented in MLwiN 2.15.
Results
25,606 resections involving 3,376 surgeons operating in 346 hospitals were analyzed. Half of cases had an adequate exam. Hierarchical models showed the median odds of an adequate exam associated with the hospital context ((MORhosp 2.05; 95% CI 1.9–2.2) was much higher than that associated with the surgeon (MORsurg 1.34; 95% CI 1.2–1.4)). Hospital characteristics teaching and high volume predicted higher odds of an adequate examination. There was no association with hospital revenue.
Conclusions
Approximately half of patients undergoing surgery for CC received an adequate lymph node examination. Hospital contextual factors had a stronger association with receipt of an adequate exam than surgeon factors. Our results suggest that quality improvement initiatives and incentives should be targeted at the hospital level to achieve the highest impact. Furthermore, we have identified non-teaching and low volume settings as rational targets for these efforts.
INTRODUCTION
Accurate staging of colon cancer (CC) depends on an assessment of the presence of tumor cells in the lymph nodes. If the lymph nodes are involved with tumor (stage III disease by definition), chemotherapy is indicated. In 1999, a multidisciplinary panel including leading US surgeons and pathologists convened to make recommendations about the optimal number of lymph nodes that should be examined to maximize staging accuracy. The panel recommended that at least 12 lymph nodes (LN) should be examined for microscopic evidence of disease spread(1). Since then, there have been a number of studies examining the predictors of an adequate lymph node examination. Strong correlations have been found between white or Asian race, younger age and receipt of an adequate exam (2–6). A large retrospective study assessing outcomes related to adequacy of the lymph node examination showed that patients with earlier stage cancers are also less likely to receive an adequate exam(7) and most importantly that those early stage patients who received an adequate examination had better outcomes than those who did not (7, 8). Based on the positive correlation with survival, the measure was endorsed by the National Comprehensive Cancer Network (NCCN), a multidisciplinary cancer care guideline-making body, as an important factor delivering evidence-based, high quality colon cancer care(9).
Meeting the standard is, by definition a coordinated effort between providers who may have little or no direct interaction. As a matter of routine, the surgeon does not attempt to count the number of lymph nodes at the time of the resection. Examination and count of lymph nodes is conducted in the treating hospital’s pathology department. Visual and tactile examination and/or chemical fixatives are used to identify the nodes. Small and inconspicuous nodes (less than 0.5cm) can be elusive. Chemical fixatives improve visualization but their use requires additional time. The process involves at least one pathologist and a variable number of pathology assistants (PA). These human resources vary among hospitals. In academic centers, the number of pathologists, PAs, and student trainees (i.e. PA students, residents, and fellows) may be substantial, as compared to low resource or non-teaching hospital settings. Thus, hospital resources and infrastructure may significantly influence the quality of the lymph node examination.
Previous work from our group and others has demonstrated that overall rate of meeting the LN standard is low (2, 6). Several studies have attempted to analyze the question of accountability between the surgeon and other operator dependent and independent factors within the hospital. The results of these studies have been somewhat inconsistent and at times contradictory. Some studies have shown no statistically significant association between LN counts and the surgeon or pathology staff (10–12). In contrast, other studies have shown that both the surgeon’s experience and the pathologist’s performance are associated with the quality of the examination (13–18). These studies have been limited by small cohorts, single institutional experience or failure to use hierarchical modeling to differentiate the effect of the provider nested within the hospital context. One investigation used a hierarchical approach to evaluate variation in meeting the standard, however, the authors used SEER-linked Medicare data (5) and found that the hospital made a larger contribution to variation than the surgeon or pathologist. However, generalizability is somewhat limited by homogeneity of the cohort (patients 65 years or older; all with similar insurance). Thus, there are no studies using a representative population based sample with appropriate methods to determine accountability for delivery on this metric.
Because the 12 LN minimum standard was subsequently endorsed by the National Quality Forum (NQF)(19), the metric is eligible to become a Center for Medicare and Medicaid Services (CMS) pay-for-performance measure. With the impending implementation of the Affordable Care Act and expected increases in pay-for-performance programs, it is imperative that the appropriate target for financial incentives be clearly identified. Some have argued against the application of incentives at the hospital level (2), suggesting that hospital level compliance does not predict hospital level outcomes. This argument effectively leaves individual providers exposed as targets for financial penalties, despite the necessity of coordinated care to achieve compliance. The purpose of the current study is to compare the contribution to variance in delivering an adequate lymph node examination between the surgeon and the hospital in an all-age, all-payer, large and diverse state level cohort. By doing so, we will elucidate rational targets for incentive based quality improvement. We hypothesize that hospitals will explain more of the variance in the adequacy of lymph node examination than individual surgeons.
METHODS
Sources of Data
Data for all patients undergoing colon cancer resections in California, 2001–2006, inclusive were obtained by linking the California Cancer Registry (CCR) data to data from the California Office of Statewide Health Planning and Development Patient Discharge Database (OSHPD-PDD). Hospital characteristics were obtained from the 2001 OSHPD Hospital Financial Data File (HAFD).
The California Cancer Registry is a statewide cancer database containing specific demographic, staging and treatment details for patients treated in California. Reporting occurs by legislative mandate for all primary cancers (except non-melanoma skin cancers). The registry has a very low loss-to-follow up rate because reporting occurs regardless of whether the patient has received care at multiple facilities within the state. The registry is diverse and robust, with fewer than 3% missing race/ethnicity data and fewer than 2% of cases identified through death records. Demographic and clinical variables include age, gender, race/ethnicity, AJCC consolidated tumor stage, number of lymph nodes examined, number of positive lymph nodes identified. Physician level variables include a unique identifier for the operating surgeon (license number). Using this identifier, the number of hospitals where a surgeon operates, and surgeon level volume can be calculated. There are no data collected in CCR on recurrent tumors. The OSHPD-PDD contains records for every adult discharge from all of the general acute, non-federal hospitals in California, regardless of intended payer. Each record contains a unique hospital identifier where care was delivered and International Classification of Disease 9th Clinical Modification (ICD9-CM) diagnostic and procedure codes indicating the primary diagnosis for the treatments provided during the index admission. The diagnostic codes indicate tumor location. Hospital annual volume can be calculated using the number of discharges with a primary diagnosis of colon cancer. After obtaining permission from the California State and Stanford IRBs, respectively, the CCR was linked to the OSHPD-PDD by the CCR staff using a probabilistic linkage algorithm based on patient’s date and year of birth and social security number (SSN). These matching variables were stripped from the records prior to disclosure to the investigators. Data was then aggregated at the hospital level using the OSHPD hospital identification number. Financial data from the 2001 HAFD was subsequently linked by the investigative team using the hospital identifier. The HAFD contains structural and financial characteristics of the discharging facilities at the start of the years under study.
Study Cohort
The analysis includes all patients who underwent surgical treatment for a primary diagnosis of stage I–III colon cancer, between the years 2001 and 2006, inclusive. Patients with stage IV disease were excluded because the presence of distant metastases determines the need for chemotherapy regardless of the LN count. Records with missing operating surgeon and/or lymph node data (i.e. no numeric data recorded) were excluded. Records with zero nodes examined were included in the analysis.
Measured Outcome
The outcome of the study is an adequate lymph node examination (defined as 12 or more lymph nodes), and the attribution of variance between the operating surgeon and the hospital which serves as a proxy for all other factors (pathology performance and coordination of care) related to achieving the standard.
Predictor Variables
Patient demographic information such as age, sex, and comorbidity score, were also included in the analysis (20, 21). The comorbidity score was calculated using the Deyo-modified Charlson co-morbidity score for administrative data(22). Because published studies have shown variation in the number of lymph nodes examined by gender, age, race and stage, these factors are included in the models.
Using the unique identifiers for surgeon and hospital, two surgeon-level variables were created. One describes the number of hospitals in which a surgeon performed colon resections because operating in different hospitals with different (perhaps, lower quality) institutional cultures may decrease a surgeon’s effectiveness. A second surgeon-level variable was created to describe the number of colon cancer surgeries that the surgeon performed during the study period since surgeon volume is likely associated with skill at achieving an adequate exam.
Hospitals were categorized in volume quartiles according to the average number of colon cancer surgeries performed per year, in keeping with others who have studied the effect of hospital volume on performance and outcomes (23–25). Hospitals were divided into quartiles of annual revenue per bed. We included this characteristic as a direct measure of hospital resources because differences in colon cancer survival (26), and low levels of compliance with the LN standard (27) have been associated with low resource settings. Hospital teaching status was also included in the analysis.
Analysis
Logistic multilevel modeling procedures were used to take account of the hierarchical data structure which consisted of hospitals hosting multiple surgeons who operate on multiple patients. The binary response was a measure of adequacy of examination of lymph nodes (y). The probability of having 12 or more lymph nodes examined (πijk) for an individual patient (i) operated on by a surgeon (j) in a hospital (k) was analyzed as a Bernoulli distribution. Each random intercepts model was created using a logit-link function in the form: log [πijk/(1−πijk)] = β0 + βX + u0jk + v0k.
The fixed part of the equation consisted of the intercept (β0) and the conditional coefficients of the predictors (βX). The coefficients of the predictors represent the change in log odds of a patient having at least 12 lymph nodes examined given a change in X characteristic (such as a comparing one age group to another) provided that the patient is operated on by the same surgeon in the same hospital. To simplify presentation, exponents of these coefficients were presented as odds ratios and 95% confidence intervals. The random part of the equation included an intercept for surgeons (u0jk) which was assumed to have a normal distribution with mean 0 and variance σu2 and an intercept for hospitals (v0k) which was assumed to have a normal distribution with mean 0 and variance σv2. The random intercept for surgeons provides an estimate for the variation in adequate node examination between surgeons after accounting for all fixed effects. The random intercept for hospitals provides an estimate for the variation in adequate note examination between hospitals after accounting for fixed effects and variation between surgeons.
Three analytic models were created to understand the influence of each level of data. The first model includes patient characteristics as fixed effects; the second includes patient and surgeon characteristics as fixed effects; and the third includes patient, surgeon, and hospital characteristics as fixed effects. All models were created using Bayesian methods as implemented through MCMC modeling in MLwiN 2.26(28). Measures of variance at the surgeon and hospital levels were presented as raw numbers and as median odds ratios(29) and 95% credible intervals(30). The median odds ratio is a measure which describes the increased likelihood of an outcome if a person moved from a randomly selected environment to a randomly selected higher performance environment. The 95% credible interval is analogous to a traditional 95% confidence interval.
Sensitivity Analyses
Sensitivity analyses were performed to address the possibility of cross-classification of surgeons who operated in more than one hospital as this violates hierarchical assumptions. Two additional datasets were created for analysis. The first consisted of only surgeons who operated in exactly one hospital (n=13,554 patients; 2,536 surgeons; 309 hospitals). In the other, we randomly selected a single hospital for surgeons who operated in more than one hospital, forcing all surgeons to be represented in only one hospital (n=21,632 patients; 3,376 surgeons; 330 hospitals). Both datasets were analyzed as described above, except there was no marker indicating the number of surgeons’ hospitals.
In order to determine whether the influence of operating in multiple hospitals varied depending on which hospitals the surgeon operated in we performed a random slopes analysis. Using the full data set, the model included all the fixed effects, except the categorical variable indicating the number of hospitals associated with one surgeon was converted to a binary variable indicating whether the surgeon operated in one hospital or more. The coefficient for this binary variable was treated as a random effect to create a random slope coefficient. The model was fit using penalized quasi-likelihood approximation with 2nd order Taylor linearization in MLwiN 2.23 because the MCMC modeling procedure would not converge with this model.
RESULTS
Of the 33,493 individuals with stage I–III disease that underwent colon cancer surgery during the study period, records for 7,577 had no information about the operating surgeon (see Table, Supplemental Digital Content 1 showing characteristics of patients excluded from analysis for missing surgeon data), 310 records had no information about the number of nodes examined. The final study cohort included 25,606 records for patients undergoing surgery by 3,376 surgeons at 346 hospitals over a 6-year period (tables 1 and 2).
Table 1.
Demographic and clinical characteristics of patients undergoing colon resections (Colon cancer, California 1996–2006)
| Characteristic | Total N (%) | Adequate exam (>/=12 LN examined) N (%) | Inadequate exam (<12 LN examined) N (%) |
|---|---|---|---|
|
| |||
| Total | 25606 (100.0) | 12801 (50.0) | 12805 (50.0) |
|
| |||
| Gender | |||
| Male | 12100 (47.3) | 5879 (45.9) | 6221 (51.2) |
| Female | 13506 (52.7) | 6922 (54.1) | 6584 (48.8) |
|
| |||
| Age (years) | |||
| less than 45 | 922 (3.6) | 613 (66.5) | 309 (33.5) |
| 45–54 | 2354 (9.2) | 1351 (57.4) | 1003 (42.6) |
| 55–64 | 4073 (15.9) | 2181 (53.6) | 1892 (46.5) |
| 65–74 | 6569 (25.7) | 3175 (48.3) | 3394 (5.6) |
| 75–84 | 8121 (31.7) | 3832 (47.2) | 4289 (52.8) |
| 85 and older | 3567 (13.9) | 1649 (46.2) | 1918 (53.8) |
|
| |||
| Race/Ethnicity | |||
| White | |||
| Black | 18017 (70.4) | 9137 (50.7) | 8880 (49.3) |
| API | 1682 (6.6) | 839 (49.9) | 843 (50.1) |
| Hispanic | 3193 (12.5) | 1520 (47.6) | 1673 (52.4) |
| Other | 2576 (10.1) | 1231 (47.8) | 1345 (52.2) |
| Unknown | 138 (0.05) | 74 (53.6) | 64 (46.4) |
|
| |||
| Insurance Status | |||
| Private | 10446 (40.8) | 5386 (51.6) | 5060 (48.4) |
| Medicare | 12577 (49.1) | 6021 (47.9) | 6556 (52.1) |
| Medicaid | 1053 (4.1) | 575 (54.6) | 478 (45.4) |
| No Insurance | 319 (1.3) | 192 (60.2) | 127 (39.8) |
| Missing | 1211 (4.7) | 627 (51.8) | 584 (48.2) |
|
| |||
| Charlson co-morbidity score | |||
| 0 | 15430 (60.3) | 8074 (52.3) | 7356 (47.7) |
| 1 | 6328 (24.7) | 3036 (48.0) | 3292 (52.0) |
| 2 | 1985 (7.8) | 876 (44.1) | 1109 (55.9) |
| 3+ | 1863 (7.3) | 815 (43.8) | 1048 (56.3) |
|
| |||
| Cancer Stage | |||
| Stage I | 6433 (25.1) | 2290 (35.6) | 4143 (64.4) |
| Stage II | 10421 (40.7) | 5461 (52.4) | 4960 (47.6) |
| Stage III | 8752 (34.2) | 5050 (57.7) | 3702 (42.3) |
|
| |||
| Tumor location | |||
| Right | 15799 (61.7) | 3551 (39.7) | 5403 (60.3) |
| Left | 8954 (39.1) | 8787 (55.6) | 7012 (44.4) |
| Unknown | 853 (3.3) | 463 (54.3) | 390 (45.7) |
|
| |||
| Emergent operation | |||
| Yes | 9014 (35.2) | 4507 (50.0) | 4507 (50.0) |
| No | 16592 (64.8) | 8294 (50.0) | 8298 (50.0) |
Abbreviations: LN=lymph node; API=Asian/Pacific Islander
Table 2.
Descriptive statistics of surgeon- and hospital-level contextual characteristics (Colon cancer, California 2001–2006)
| Surgeon level characteristic | N | % |
|---|---|---|
| Surgeon’s number of hospital affiliations | ||
| 1 | 2536 | 75.1 |
| 2 | 607 | 18.0 |
| 3 | 164 | 4.9 |
| 4+ | 69 | 2.0 |
| Surgeon’s number of colon surgeries | ||
| 1–4 | 2169 | 64.3 |
| 5–9 | 427 | 12.6 |
| 10–19 | 388 | 11.5 |
| 20 or more | 392 | 11.6 |
| Total | 3376 | 100.0 |
| Hospital level characteristic | N | % |
|---|---|---|
| Annual colon cancer surgical volume (cases) | ||
| 0.00–9.99 | 195 | 56.4 |
| 10.00–19.99 | 76 | 22.0 |
| 20.00–29.99 | 33 | 9.5 |
| 30.00–82.00 | 42 | 12.1 |
| Hospital annual revenue per bed (US dollars) | ||
| 14,000–312,999 | 80 | 23.1 |
| 313,000–446,999 | 80 | 23.1 |
| 447,000–638,999 | 80 | 23.1 |
| 639,000–2,009,999 | 80 | 23.1 |
| Unknown | 26 | 7.5 |
| Teaching hospital | ||
| No | 308 | 89.0 |
| Yes | 26 | 7.5 |
| Unknown | 12 | 3.5 |
| Total | 346 | 100.0 |
Individual Characteristics
Of the patients undergoing colon resection during the study period, 50% received an adequate LN examination. Older patients, racial/ethnic minorities, patients with Medicare insurance and those with higher co-morbidity scores received an adequate exam in lower proportions (all <50%). An adequate examination was achieved more often in patients with stage II (52.4%) and III (57.7%) disease, and those with proximal (right sided) tumors (60.3%); (table 1).
Surgeon and Hospital Characteristics
The majority of surgeons had only one hospital affiliation (75.1%) and performed fewer than 5 colon surgeries over the course of the study period (64.3%). Most hospitals hosted fewer than 10 colon cancer surgeries per year on average (56.4%). The mean hospital revenue per bed was $500,512; and the median was $446,070 (US Dollars). The majority of hospitals were not teaching hospitals (89%); (table 2).
Adequate examination in hierarchical models
Table 3 depicts the correlations between individual characteristics and an adequate examination, with adjustment for contextual factors. Similar to univariate comparisons, younger age, low co-morbidity scores and stage II and III disease were significant predictors for an adequate exam. Tumors located on the right side of the colon were twice as likely to be associated with an adequate exam as compared to distal tumors (OR=2.32; 95% CI=2.17–2.47). Table 4 shows the effect of contextual factors. There was no association between the number of hospitals where a surgeon operates and an adequate exam. We noted an 18% increase in the odds of achieving the guidelines associated with highest volume surgeons. Select hospital characteristics (highest volume quartile (OR=1.67; 95%CI= 1.28–2.17); and teaching (OR=1.47; 95%CI= 1.06–2.03)), were independently associated with increased odds of an adequate exam.
Table 3.
Hierarchical (fixed effect model) predicting the odds of receiving an adequate lymph node exam (12 or more) by individual characteristics (Colon Cancer, California 2001–2006)
| Individual Characteristics | Odds of an adequate exam (12 or more LN examined) OR (95% CI) | P value |
|---|---|---|
|
| ||
| Sex | ||
| Male | 1.00 (ref) | |
| Female | 1.13 (1.07–1.19) | <0.001 |
|
| ||
| Age (years) | ||
| 85 and older | 1.00 (ref) | |
| 75–84 | 1.15 (1.05–1.26) | <0.001 |
| 65–74 | 1.31 (1.19–1.45) | <0.001 |
| 55–64 | 1.71 (1.52–1.92) | <0.001 |
| 45–54 | 2.10 (1.83–2.40) | <0.001 |
| 45 and older | 3.14 (2.61–3.78) | <0.001 |
|
| ||
| Race/Ethnicity | ||
| White | 1.0 (ref) | |
| Black | 0.90 (0.80–1.02) | 0.11 |
| Hispanic | 0.93 (0.85–1.02) | 0.15 |
| Asian/Pacific Islander | 0.97 (0.88–1.08) | 0.61 |
| Other | 1.01 (0.66–1.53) | 0.97 |
|
| ||
| Insurance Status | ||
| Private | 1.00 (ref) | |
| Medicare | 1.05 (0.98–1.12) | 0.21 |
| Medicaid | 1.25 (1.06–1.46) | 0.01 |
| No Insurance | 1.18 (0.89–1.57) | 0.24 |
| Missing | 1.00 (0.87–1.15) | 0.99 |
|
| ||
| Charlson comorbidity score | ||
| 3+ | 1.00 (ref) | |
| 2 | 1.09 (0.96–1.25) | 0.19 |
| 1 | 1.21 (1.08–1.35) | <0.001 |
| 0 | 1.36 (1.23–1.51) | <0.001 |
|
| ||
| Cancer Stage | ||
| Stage I | 1.0 (ref) | |
| Stage II | 2.30 (2.14–2.47) | <0.001 |
| Stage III | 2.75 (2.56–2.96) | <0.001 |
|
| ||
| Tumor location | ||
| Left | 1.00 (ref) | |
| Right | 2.32 (2.17–2.47) | <0.001 |
| Unknown | 2.08 (1.76–2.45) | <0.001 |
|
| ||
| Emergent operation | ||
| No | 1.00 (ref) | |
| Yes | 0.98 (0.92–1.04) | <0.53 |
This model adjusted for contextual factorsshown in Table 4. Abbreviations: LN=lymph node; p <0.05 considered statistically significant.
Table 4.
Hierarchical (fixed effect model) predicting the odds of receiving an adequate lymph node exam (12 or more) by contextual characteristics (Colon Cancer, California 2001–2006)
| Contextual Characteristics | Odds of an adequate exam (12 or more LN examined) OR (95% CI) | P value |
|---|---|---|
|
| ||
| Surgeon Characteristics | ||
|
| ||
| Number of hospital affiliations | ||
| 1 | 1.00 (ref) | |
| 2 | 1.02 (0.93–1.11) | 0.68 |
| 3 | 1.10 (0.97–1.24) | 0.13 |
| 4+ | 0.93 (0.79–1.08) | 0.32 |
|
| ||
| Surgeon CC volume | ||
| 1–4 | 1.0 (ref) | |
| 5–9 | 1.13 (0.99–1.28) | 0.06 |
| 10–19 | 1.07 (0.95–1.21) | 0.27 |
| 20 or more | 1.18 (1.05–1.33) | 0.01 |
|
| ||
| Hospital Characteristics | ||
|
| ||
| Hospital CC surgery volume | ||
| 0–9.99 | 1.00 (ref) | |
| 10–19.99 | 1.19 (0.92–1.53) | 0.19 |
| 20–29.9 | 1.09 (0.77–1.53) | 0.63 |
| 30 and above | 1.67 (1.28–2.17) | <0.001 |
|
| ||
| Hospital Revenue per bed (quartiles) | ||
| Q1 (low) | 1.00 (ref) | |
| Q2 | 0.78 (0.58–1.05) | 0.11 |
| Q3 | 1.05 (0.76–1.44) | 0.78 |
| Q4 (high) | 1.06 (0.76–1.48) | 0.73 |
| Unknown | 0.88 (0.60–1.29) | 0.51 |
|
| ||
| Hospital teaching status | ||
| Non-Teaching | 1.00 (ref) | |
| Teaching | 1.47 (1.06–2.03) | 0.02 |
| Unknown | 1.44 (0.74–2.79) | 0.29 |
This model is also adjusted for patient level factors shown in Table 3. Abbreviations: LN=lymph node; CC=colon cancer; P values <0.05 statistically significant
Attribution of variation in performance
Hierarchical models quantified the unexplained variance in meeting the standard by contextual characteristics. Table 5 (model 3) shows the unexplained variance at the hospital level (0.620; SE 0.068) was more than 5 times that of the unexplained variance at the surgeon level (0.103; SE 0.031). Adjustment for patient, surgeon and hospital characteristics had little effect on the variance attributed to the surgeon and the hospital. The median odds ratio at the hospital level in the final model (MOR = 2.05; 95% CrI = 1.90–2.22) indicates that on average, if a patient moved between two randomly selected hospitals, they would more than double the odds of receiving an adequate exam by switching to the better of the two settings. Surgeon level variance (MOR = 1.34; 95% CrI = 1.17–1.42) indicates that a similar switch between surgeons would only confer a 34% increase in the likelihood of receiving an adequate exam.
Table 5.
Random effects model comparing the attribution of variance between hospital and surgeon in receiving an adequate (12 or more) lymph node examination. (Colon Cancer, California 2001–2006)
| Model 1 | Model 2 | Model 3 | ||||
|---|---|---|---|---|---|---|
| Contextual level | Variance | SE | Variance | SE | Variance | SE |
| Hospital | 0.674 | 0.072 | 0.665 | 0.069 | 0.620 | 0.068 |
| Surgeon | 0.127 | 0.016 | 0.127 | 0.020 | 0.103 | 0.031 |
| Contextual level | MOR | 95% CrI | MOR | 95% CrI | MOR | 95% CrI |
| Hospital | 2.11 | 1.97–2.30 | 2.10 | 1.96–2.27 | 2.05 | 1.90–2.22 |
| Surgeon | 1.39 | 1.32–1.43 | 1.38 | 1.32–1.46 | 1.34 | 1.17–1.42 |
Model 1 includes patient level factors alone; Model 2 includes patient and surgeon level characteristics; Model 3 includes patient, surgeon and hospital level characteristics. Abbreviations: SE=standard error; MOR=Median Odds Ratio; 95%CrI=95% credible interval; statistical significance is defined when 95%CrI excludes 1.
Sensitivity analyses
The sensitivity analyses using the reduced datasets revealed few differences from the analysis using the full dataset. The hospital- and surgeon-level variance components changed from 0.620 and 0.103 in the main dataset to 0.596 and 0.158 in the analysis of surgeons operating in one hospital and 0.599 and 0.119, in the dataset using one randomly selected hospital for each surgeon. In the random slopes model, the main fixed effect for the binary measure of whether a surgeon operated at multiple hospitals was not statistically significant (χ2, 1 degree of freedom [DF] = 0.185; p = 0.67). In this model, the coefficients for the additional effect of operating in multiple hospitals varied by hospital from −0.192 to +0.281 which is analogous to a 17% reduction and a 32% increase in odds of having an adequate number of nodes examined, respectively. However, the random slope coefficient (χ2, 1 DF = 1.427; p = 0.23); the covariance coefficient (χ2, 1 DF = 3.087; p = 0.08), and the joint test of the random slope and the covariance coefficients (χ2, 2 DF = 3.784; p = 0.15) were not statistically significant. This indicates that the fixed effect of a surgeon working at more than one hospital (compared to working at only one hospital) does not change the adequacy of the LN exam according to the hospital where a surgeon works.
DISCUSSION
In the current investigation, we have used a large, diverse, all age and all payer data base to quantify the contribution of the surgeon and hospital context to the adequacy of lymph node examination in colon cancer. Although the highest surgeon volume was correlated with receipt of an adequate exam, there was no significant contribution of the number of hospitals where the surgeon operated. Variance in receipt of an adequate exam was better explained by hospital than surgeon characteristics. We found that high volume, and teaching institutions were much more likely to meet the standard than low volume and non-teaching institutions. This likely reflects the impact of overarching hospital structure and the requirement of a multidisciplinary effort to ensure an adequate lymph node examination.
Our findings are consistent with prior research that has shown the relevance of select patient factors (such as age, race/ethnicity, stage and co-morbidities) in meeting the quality standard. We did not evaluate operator independent factors such as specimen length or tumor size which has recently been associated with a higher likelihood of an adequate examination (31). We did evaluate the other operator independent factors elucidated in this recent study and found similar correlations with tumor location. Importantly, on the contextual level, our work supports the findings of others who have shown that more of the variation is due to an entity other than the surgeon, and in particular, the hospital setting (5, 16, 17).
Despite similarities, the current study is novel because we have used a more diverse data set than those who previously assessed multilevel determinants of lymph node examination (5). This is important because previous studies focusing on an aging population may skew the results since elderly patients are less likely to receive an adequate exam. Our results are quite robust because we have performed sensitivity analyses which support our initial suspicion that surgeons may operate in multiple settings, but generally attempt to perform the same oncologic resection each time. That is to say, the surgeon’s performance does not appear to be directly affected by the performance of the various hospitals in which they operate. Finally, by reporting median odds ratios, we have quantified the positive impact of improving adequacy of the exam at the hospital level, which could have a more significant effect than improving performance at the surgeon level. Our findings of the larger proportion contributed by the hospital context than the surgeon has face validity because in order to meet the standard, multiple agents (surgeon and pathologist) within the hospital must work in a coordinated fashion. This coordination of care may be best mediated at the hospital level. For example, when lymph node counts are low, hospital policy may mandate a call between the pathologist and surgeon in order to explore ways to increase counts.
Recent studies have argued that lymph node counts are not an appropriate quality measure to attend to, in part because individual hospital’s performance on this measure does not correlate with hospital level mortality (32). While many studies have found this to be the case (2, 7), they have also found strong correlations between patient level lymph node counts and improved patient level survival(2, 7, 33). Thus, with the impending implementation of the Affordable Care Act and attendant pay-for-performance programs, it is critical to define rational targets that will have the most significant and positive impact on performance. Improving quality at the hospital level increases the likelihood of meeting the standard by greater proportions than targeting individual surgeon performance.
LIMITATIONS
This study has some limitations. First, it is secondary analysis of cross sectional, observational data. Therefore, we cannot assume causation in the relationships we observe. The study is also limited in that there is no direct measure of the specific characteristics or practices within hospitals that drive the resulting variation. We did not perform any direct analysis of pathologist performance because that data is not available in this set. Nor was there any detail about the chain of specimen handling, which might affect nodal counts. Thus, further research will be needed to uncover direct explanation’s for the large variation in performance attributable to the hospital.
CONCLUSIONS
We have demonstrated that variance in the adequacy of lymph node examination in colorectal cancer surgery is associated more with hospital factors than with surgeon factors. Incentives to improve adequacy of the lymph node examination should be targeted at the hospital-level. These incentives should be designed to drive the creation of internal policies that encourage providers to work together with the ultimate goal of improving colon cancer survival.
Supplementary Material
Supplemental Digital Content 1. Table showing the characteristics of patients excluded from analysis for missing surgeon data. doc
Footnotes
Poster presentation at the American Society of Colon and Rectal Surgeons Annual Meeting, Vancouver, Canada. May 12-14, 2011.
References
- 1.Compton CC, Fielding LP, Burgart LJ, Conley B, Cooper HS, Hamilton SR, et al. Prognostic factors in colorectal cancer. College of American Pathologists Consensus Statement 1999. Arch Pathol Lab Med. 2000;124(7):979–94. doi: 10.5858/2000-124-0979-PFICC. [DOI] [PubMed] [Google Scholar]
- 2.Wong SL, Ji H, Hollenbeck BK, Morris AM, Baser O, Birkmeyer JD. Hospital lymph node examination rates and survival after resection for colon cancer. JAMA. 2007;298(18):2149–54. doi: 10.1001/jama.298.18.2149. [DOI] [PubMed] [Google Scholar]
- 3.Chang GJ, Rodriguez-Bigas MA, Skibber JM, Moyer VA. Lymph node evaluation and survival after curative resection of colon cancer: systematic review. J Natl Cancer Inst. 2007;99(6):433–41. doi: 10.1093/jnci/djk092. [DOI] [PubMed] [Google Scholar]
- 4.Baxter NN, Virnig DJ, Rothenberger DA, Morris AM, Jessurun J, Virnig BA. Lymph node evaluation in colorectal cancer patients: a population-based study. J Natl Cancer Inst. 2005;97(3):219–25. doi: 10.1093/jnci/dji020. [DOI] [PubMed] [Google Scholar]
- 5.Nathan H, Shore AD, Anders RA, Wick EC, Gearhart SL, Pawlik TM. Variation in lymph node assessment after colon cancer resection: patient, surgeon, pathologist, or hospital? J Gastrointest Surg. 2011;15(3):471–9. doi: 10.1007/s11605-010-1410-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Rhoads KF, Cullen J, Ngo JV, Wren SM. Racial and ethnic differences in lymph node examination after colon cancer resection do not completely explain disparities in mortality. Cancer. 2012;118(2):469–77. doi: 10.1002/cncr.26316. [DOI] [PubMed] [Google Scholar]
- 7.Parsons HM, Tuttle TM, Kuntz KM, Begun JW, McGovern PM, Virnig BA. Association between lymph node evaluation for colon cancer and node positivity over the past 20 years. JAMA. 2011;306(10):1089–97. doi: 10.1001/jama.2011.1285. [DOI] [PubMed] [Google Scholar]
- 8.Huebner M, Therneau T, Larson D. Estimating underreported N2 disease in rectal cancer patients with low lymph node counts. J Surg Oncol. 2012;106(3):248–53. doi: 10.1002/jso.22158. [DOI] [PubMed] [Google Scholar]
- 9.National Comprehensive Cancer Network. National Clinical Practice Guidelines in Oncology. 2011 [Google Scholar]
- 10.Hsu CW, Lin CH, Wang JH, Wang HT, Ou WC, King TM. Factors that influence 12 or more harvested lymph nodes in early-stage colorectal cancer. World J Surg. 2009;33(2):333–9. doi: 10.1007/s00268-008-9850-z. [DOI] [PubMed] [Google Scholar]
- 11.Jakub JW, Russell G, Tillman CL, Lariscy C. Colon cancer and low lymph node count: who is to blame? Arch Surg. 2009;144(12):1115–20. doi: 10.1001/archsurg.2009.210. [DOI] [PubMed] [Google Scholar]
- 12.Bilchik AJ, DiNome M, Saha S, Turner RR, Wiese D, McCarter M, et al. Prospective multicenter trial of staging adequacy in colon cancer: preliminary results. Arch Surg. 2006;141(6):527–33. doi: 10.1001/archsurg.141.6.527. discussion 533–4. [DOI] [PubMed] [Google Scholar]
- 13.Chou JF, Row D, Gonen M, Liu YH, Schrag D, Weiser MR. Clinical and pathologic factors that predict lymph node yield from surgical specimens in colorectal cancer: a population-based study. Cancer. 2010;116(11):2560–70. doi: 10.1002/cncr.25032. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Nicholl MB, Wright BE, Conway WC, Aarnes-Leong T, Sim MS, Faries MB. Does specialized surgical training increase lymph node yield in colon cancer? Am Surg. 2009;75(10):887–91. [PubMed] [Google Scholar]
- 15.Leung AM, Scharf AW, Vu HN. Factors Affecting Number of Lymph Nodes Harvested in Colorectal Cancer. J Surg Res. 2009 doi: 10.1016/j.jss.2009.09.001. [DOI] [PubMed] [Google Scholar]
- 16.Evans MD, Barton K, Rees A, Stamatakis JD, Karandikar SS. The impact of surgeon and pathologist on lymph node retrieval in colorectal cancer and its impact on survival for patients with Dukes’ stage B disease. Colorectal Dis. 2008;10(2):157–64. doi: 10.1111/j.1463-1318.2007.01225.x. [DOI] [PubMed] [Google Scholar]
- 17.Stocchi L, Fazio VW, Lavery I, Hammel J. Individual Surgeon, Pathologist, and Other Factors Affecting Lymph Node Harvest in Stage II Colon Carcinoma. Is a Minimum of 12 Examined Lymph Nodes Sufficient? Ann Surg Oncol. 2010 doi: 10.1245/s10434-010-1308-5. [DOI] [PubMed] [Google Scholar]
- 18.Bamboat ZM, Deperalta D, Dursun A, Berger DL, Bordeianou L. Factors affecting lymph node yield from patients undergoing colectomy for cancer. Int J Colorectal Dis. 2011;26(9):1163–8. doi: 10.1007/s00384-011-1240-6. [DOI] [PubMed] [Google Scholar]
- 19.Forum NQ. NQF measure #225; lymph node examination after colon cancer. 2011 [Google Scholar]
- 20.Yost K, Perkins C, Cohen R, Morris C, Wright W. Socioeconomic status and breast cancer incidence in California for different race/ethnic groups. Cancer Causes Control. 2001;12(8):703–11. doi: 10.1023/a:1011240019516. [DOI] [PubMed] [Google Scholar]
- 21.Yost KJ, Hahn EA, Zaslavsky AM, Ayanian JZ, West DW. Predictors of health-related quality of life in patients with colorectal cancer. Health Qual Life Outcomes. 2008;6:66. doi: 10.1186/1477-7525-6-66. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45(6):613–9. doi: 10.1016/0895-4356(92)90133-8. [DOI] [PubMed] [Google Scholar]
- 23.Birkmeyer JD, Dimick JB, Staiger DO. Operative mortality and procedure volume as predictors of subsequent hospital performance. Ann Surg. 2006;243(3):411–7. doi: 10.1097/01.sla.0000201800.45264.51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Birkmeyer JD, Sun Y, Wong SL, Stukel TA. Hospital volume and late survival after cancer surgery. Ann Surg. 2007;245(5):777–83. doi: 10.1097/01.sla.0000252402.33814.dd. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Dimick JB, Cowan JA, Jr, Upchurch GR, Jr, Colletti LM. Hospital volume and surgical outcomes for elderly patients with colorectal cancer in the United States. J Surg Res. 2003;114(1):50–6. doi: 10.1016/s0022-4804(03)00207-5. [DOI] [PubMed] [Google Scholar]
- 26.Rhoads KF, Ackerson LK, Jha AK, Dudley RA. Quality of colon cancer outcomes in hospitals with a high percentage of Medicaid patients. J Am Coll Surg. 2008;207(2):197–204. doi: 10.1016/j.jamcollsurg.2008.02.014. [DOI] [PubMed] [Google Scholar]
- 27.Rhoads KFNJ, Dudley RA. Medicaid hospitals have poor compliance with evidence based processes of colon cancer care. Journal of Healthcare for the Poor and Underserved. 2013 in press. [Google Scholar]
- 28.WJB . MCMC Estimation in MLwiN Version 2.26. Center for Multilevel Modeling, University of Bristol; 2012. [Google Scholar]
- 29.Larson KMJ. Appropriate assessment of neighborhood effects on indivdual health; integrating random and fixed effects in multilevel logistic regression. Am J Epidemiol. 2005;161:81–88. doi: 10.1093/aje/kwi017. [DOI] [PubMed] [Google Scholar]
- 30.Merlo JCB, Ohlsson H, et al. A brief conceptual tutorial of multilevel analysis in social epidemiology: using measures of clustering in multilevel logistic regression to investigate contextual phenomena. J Epidemiol and Comm Health. 2006;60:290–297. doi: 10.1136/jech.2004.029454. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Morikawa T, Tanaka N, Kuchiba A, Nosho K, Yamauchi M, Hornick JL, et al. Predictors of lymph node count in colorectal cancer resections: data from US nationwide prospective cohort studies. Arch Surg. 2012;147(8):715–23. doi: 10.1001/archsurg.2012.353. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Wong SL. Lymph node evaluation in colon cancer: assessing the link between quality indicators and quality. JAMA. 2011;306(10):1139–41. doi: 10.1001/jama.2011.1318. [DOI] [PubMed] [Google Scholar]
- 33.Wong SL. Lymph node counts and survival rates after resection for colon and rectal cancer. Gastrointest Cancer Res. 2009;3(2 Suppl):S33–5. [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental Digital Content 1. Table showing the characteristics of patients excluded from analysis for missing surgeon data. doc