A novel risk-adjusted nomogram for rectal cancer surgery outcomes

Abstract

Context

Circumferential resection margin is the primary determinant of local recurrence and a major factor in survival in rectal cancer. Neither chemotherapy nor chemoradiation compensates for a positive margin.

Objective

To identify treatment-related factors associated with hospital margin-positive resection and to develop a tool that could be used by individual hospitals to assess their outcomes based on their unique mix of patient and tumor characteristics.

Design

Retrospective review of the National Cancer Data Base, 1998–2007.

Settings

Community and academic/research hospitals.

Patients

Histologically confirmed localized rectal/rectosigmoid adenocarcinoma.

Outcome measures

Rate of margin positivity determined and adjusted for patient- and tumor-related factors to calculate expected margin positivity per hospital. An observed/expected ratio was calculated based on patient and tumor factors to identify treatment associated variation.

Results

Overall margin-positive resection rate was 5.2%. Patients with positive margins were more likely to be older, male, and African American; not have private insurance; and have their cancer diagnosed later in the study period. Associated tumor factors include rectal location, higher American Joint Committee on Cancer stage, signet/mucinous histology, and poor/undifferentiated grade. Among hospitals that were significantly low outliers, 47% were comprehensive community hospitals and 43.9% were academic/research hospitals; of those that were significantly high outliers, 52.3% were comprehensive community hospitals and 17.8% were academic/research hospitals. High-volume centers made up 80% of significantly low-outlier hospitals and 17% of significantly high outliers. Rates of chemotherapy and radiation were similar, but low-outlier hospitals gave more neoadjuvant radiation (26.3% vs 17%).

Conclusions

After adjustment for patient and tumor factors we identified both low and high outliers for margin positivity at resection as well as potentially modifiable risk factors. The nomogram created in this model allows evaluation of observed and expected event rates for individual hospitals, providing a hospital self-assessment tool for identifying targets for improvement.

INTRODUCTION

In 2012 there were an estimated 40,290 cases of rectal cancer diagnosed in the United States. Colorectal cancer is second leading cause of US cancer death with rectal cancer accounting for approximately one third of colorectal cancer deaths.¹ For cases without metastases, the primary treatment is radical resection with proctectomy and en bloc radical lymphadenectomy; for locally advanced cases, primary treatment is perioperative radiation and chemotherapy. The widespread adoption of sharp total mesorectal excision (TME) has improved local control and survival by improving radical lymph node clearance and decreasing the risk for a positive surgical margin. Accordingly, local recurrence rates have decreased to 5–10% from as high as 50%. ^2–4 Circumferential resection margin status has been identified as one of the most important determinants of local recurrence risk. Positive resection margins are associated with metastatic disease and decreased survival. ^5,6 Unfortunately, neither radiation nor chemotherapy can compensate for a positive margin in rectal cancer. ^7,8 Therefore, achievement of a negative resection margin is a primary goal of surgery for rectal cancer.

A number of tumor factors are associated with risk for margin positivity at rectal cancer resection. Tumor location in the distal third of the rectum, T4 status, advanced local nodal stage, need for abdominoperineal resection and anteriorly located tumors are associated with higher risk of margin positivity. Patient and treatment factors have also been identified, including American Joint Committee on Cancer (AJCC) stage and advanced age. ^7,9–14

Factors influencing the risk for margin positivity at resection can be categorized as patient, tumor, and treatment factors. To further investigate these relationships, we used a large hospital-based national database to examine risk factors for margin-positive resection of rectal cancer. We specifically sought to identify treatment-related factors associated with margin-positive resection and to develop a tool that could be used by individual hospitals to assess their outcomes based on their unique mix of patient and tumor characteristics.

MATERIALS AND METHODS

Data Source

Data from the National Cancer Data Base (NCDB) were used for this study. The NCDB is aa joint program of the Commission on Cancer of the American College of Surgeons and the American Cancer Society as a surveillance tool to assess patterns of care for cancer patients. ¹⁵ Over 1500 cancer programs in the United States are Commission-accredited; the NCDB captures approximately 76% of newly diagnosed cancer cases. Hospitals contributing to the NCDB are classified as academic/research hospital cancer programs, comprehensive cancer programs, or community hospital cancer programs. Data collected for each cancer case include patient characteristics, cancer staging, tumor characteristics, types of treatment administered, and outcomes.

Population

Rectal or rectosigmoid cancers (International Classification of Disease for Oncology codes C199 and C209) were identified within the NCDB between 1998 and 2007. (Figure 1) All patients with histologically confirmed adenocarcinoma who had undergone partial or complete proctectomy (excluding cases of local tumor excision/destruction), were 18 to90 years of age, and had no prior cancer diagnosis were included. Patients who died within 30 days of cancer diagnosis, had stage IV disease, or had an unknown margin status were excluded. Finally, hospitals reporting a margin-positive rate of zero were excluded as they may have been subject to either very low procedure volume or coding discrepancies that precluded analysis.

Figure 1.

Study Design

Statistical Analysis

We analyzed data on the basis of patient, tumor, and treatment/hospital variables. The primary outcome measure was individual hospital rate of margin-positive resection adjusted for patient- and tumor-related risk factors. Demographic data included patient age (categorized 18–49, 50–75, and 76 –90 years), sex, race, year of diagnosis, insurance status, median quartile income, percentage of residents in the patient’s neighborhood without a high school diploma, and area-based measures of residence as metropolitan, urban, or rural. Neighborhood education and income were assessed by matching patient zip codes to files derived from 2000 US Census data coded as quartiles. Tumor characteristics included anatomic location (rectosigmoid versus rectum), AJCC 6^th edition tumor stage, tumor size, histology, and grade. Treatment variables included facility type (community cancer program, comprehensive community cancer program, or academic/research program), yearly hospital surgical volumes categorized by quartiles, receipt of chemotherapy or radiotherapy and the sequence of treatment, and specific type of surgery (low anterior resection, coloanal anastamosis, or abdominoperineal resection).

We sought to ascertain the hospital rate of margin positivity after adjustment for patient- and tumor-related risk factors in order to identify potentially modifiable treatment-related factors. We first developed a patient-level multi-variable risk model for margin positivity using multiple logistic regression analysis incorporating important patient- and tumor-related clinical variables, leaving the treatment-related factors unadjusted in the model for post-hoc comparison.

This risk model was used to calculate the expected number of patients with positive margin for each hospital by summing all patients’ risk estimates by hospital. The observed number of margin-positive cases was then divided by the expected number of events (based on patient- and tumor-related factors) to create an observed/expected (O/E) ratio for each hospital. To estimate the significance of the O/E ratio, the probability (P) that a hospital had exactly the observed number of events (X=k) was determined based on the binomial function: $\mathrm{P}(X=k)=\left(\begin{array}{c}n\\ k\end{array}\right)p^k{(1-p)}^{n-k}$, where k denotes the actual number of events observed within the hospital, n denotes the hospital volume, and p denotes the model-derived expected probability of the event. ¹⁶ Finally, hospital- and treatment-related factors were compared on the basis of statistically significant outlier status.

Nomogram

A nomogram was constructed using multivariate logistic regression analysis and internally validated for model discrimination and calibration by bootstrapping with 200 resamples. Model discrimination was first quantified using the concordance index to measure the predictive accuracy of the model by analyzing all possible pairs of patients. After quantifying the model discrimination, the model calibration was graphically assessed using a calibration plot. ¹⁷

All statistical analyses were performed using Stata MP software version 11.0 (release 2010; College Station, TX), and the R software (http://www.r-project.org/) with the RMS package was used to construct the nomogram. Because the analysis utilized preexisting data with no personal identifiers, it was exempt from review by our institutional review board.

RESULTS

A total of 85,190 patients treated in 1064 hospitals met study criteria and were eligible for analysis. The identification of cases and a flowchart of the analysis are shown in Figure 1. Demographic and tumor data for all patients are summarized in Table 1. Overall rate of margin-positive resections for the entire cohort was 5.2%. Patients with a margin-positive resection were more likely to be older, male, and African-American; to have their disease diagnosed in the more recent years of the study period (2004–2007); and to not have private insurance. Other demographic factors, including median income quartile, education, and rural versus urban residence, were not associated with risk of margin positivity.

Table 1.

Margin positive rate and adjusted risk of margin positive for patients with surgically treated rectal and rectosigmoid adenocarcinoma 1998–2007 (n=85,190)

Characteristics	Total		Proportion margin positive	Adjusted odds ratio
	No.	Col %	Row %	OR	95% CI
Overall	85,190	100%	5.2	-	-
Age of diagnosis
18–49	11,932	14	5.9	1	Reference
50–75	54,416	63.9	4.9	0.96	0.88–1.05
76–90	18,842	22.1	5.7	1.18	1.07–1.32
Gender
Female	37,203	43.7	5.1	1	Reference
Male	47,987	56.3	5.3	1.07	1.01–1.15
Race
White	75,134	88.2	5.1	1	Reference
Black	5,951	7.0	6.6	1.21	1.08–1.36
Others	4,105	4.8	5.6	1.04	0.90–1.19
Year of diagnosis
1998–1999	15,683	18.4	4.7	1	Reference
2000–2001	16,504	19.4	4.3	0.92	0.83–1.03
2002–2003	17,234	20.2	5.0	1.06	0.95–1.17
2004–2005	18,120	21.3	5.8	1.23	1.11–1.36
2006–2007	17,649	20.7	6.1	1.31	1.19–1.45
Insurance status
Private insurance	13,138	15.4	4.7	1	Reference
Not insured	2,254	2.6	8.5	1.54	1.29–1.84
Government	67,733	79.5	5.2	1.10	1.00–1.21
Unknown	2,065	2.4	5.6	1.15	0.93–1.42
Median income quartile
< $30,000	11,239	13.2	5.6	1	Reference
$30,000 – $35,000	15,761	18.5	5.4	1.04	0.93–1.17
$35,000 – $45,999	24,247	28.5	5.3	1.05	0.94–1.18
$46,000 +	33,943	39.8	4.9	0.99	0.87–1.13
Proportion without high school degree by Zip code
< 14%	14,022	16.5	5.5	1	Reference
14% – 19.9%	20,493	24.1	5.5	1.04	0.94–1.15
20% – 28.9%	21,345	25.1	5.2	1.02	0.91–1.14
29% +	29,330	34.4	4.9	1.00	0.88–1.13
Population density of residence
Metro area	69,087	81.1	5.1	1	Reference
Urban area	12,879	15.1	5.5	1.02	0.93–1.11
Rural area	3,224	3.8	5.4	0.97	0.83–1.14
Tumor location
Rectosigmoid	32,294	37.9	4.9	1	Reference
Rectum	52,896	62.1	5.4	1.19	1.11–1.27
AJCC 6th tumor stage
I	26,978	31.7	1.3	1	Reference
IIA	22,748	26.7	3.9	2.90	2.55–3.30
IIB	1,988	2.3	23.6	21.80	18.74–25.36
IIIA	5,954	7	2.2	1.70	1.39–2.08
IIIB	14,747	17.3	7.7	5.93	5.24–6.71
IIIC	12,775	15	11.5	8.33	7.36–9.42
Tumor size
<10mm	2,874	3.4	2.6	1	Reference
11–20mm	8,178	9.6	2.9	0.92	0.70–1.20
21–998mm	62,055	72.8	5.6	1.09	0.86–1.38
Unknown	12,083	14.2	5.3	1.30	1.01–1.66
Histology type
Adenocarcinoma	78,580	92.2	4.7	1	Reference
Signet ring	586	0.7	22.4	2.49	2.02–3.07
Mucinous	6,024	7.1	9.6	1.51	1.38–1.67
Tumor grade
Well- and moderately differentiate	68,940	80.9	4.5	1	Reference
Poorly- and un- differentiate	11,902	14	9.4	1.44	1.33–1.56
Unknown	4,348	5.1	4.7	0.95	0.82–1.11

Baseline patient and tumor characteristics and margin status are summarized in Table 1. Tumors located in the rectum with higher AJCC stage (final pathology stage), signet or mucinous histology, or classified as poorly differentiated or undifferentiated were associated with higher risk for positive margin. Tumor size was not associated with margin status.

Adjusted Risk Model

An adjusted risk model for margin positivity was developed based on patient and tumor characteristics. Nearly 64% of patients of the patient sample were between 40 and 75 years old. The margin-positive rate was lowest in this group. On multivariate analysis, patient factors associated with a small to moderate increase risk for margin positivity (Odds Ratio [OR] 1.0–1.50) included age greater than 75 years, male sex, African American race, and more recent year of diagnosis (2004–2005 and 2006–2007). A stronger association was observed with lack of private insurance with a 63% relative increase in margin positivity (8.5%; OR 1.54, 95% CI 1.29–1.84).

Tumor-specific factors were evaluated on final pathologic specimen. Factors significantly associated with a moderate increased risk for margin positivity included poorly differentiated grade and tumor location in the rectum. Stronger associations were observed for signet ring cell or mucinous histologic type, and more advanced T or N categoryHighest rates of margin-positive resection were associated with AJCC stage IIB tumors (margin-positive rate 23.6%; OR 21.8, 95% CI 18.74–25.36 vs. stage I tumors) and signet ring histology (22.4%, OR 2.49, 95% CI 2.02–3.07 vs. non-signet ring, non-mucinous adenocarcinomas). (Table 1). Notably 24% of stage IIB patients received neoadjuvant radiation and, of these, 20.3% were margin positive at resection. As expected, the margin positivity rate for the adjuvant group was higher at 31.5%.

Observed to Expected Rates

The O/E ratio of margin-positive resection was determined for individual hospitals (Figure 2) and plotted logarithmically to compare the ratios of observed to expected margin-positive resection rates according to the model-derived expected probabilities. Hospitals with O/E of 1 had exactly the same number of observed margin positive cases as expected according to patient and tumor characteristics. Those with O/E>1 had a greater number of observed margin-positive tumors than expected; those with O/E <1 had fewer observed margin-positive tumors than expected. These were further designated as significantly high (referred to as “high outliers”) and significantly low outliers (referred to as “low outliers”) on the basis of the probability of H₀:O=E, defined by P <0.05. This allowed us to make comparisons between the hospitals on the basis of outlier status, taking into account hospital characteristics and treatment patterns.

Figure 2.

Ratio of observed to expected margin positive rates (logarithmic scale)

Of the total 1064 hospitals examined, 173 were significant low or high outliers. Of the low outliers (significantly fewer than expected margin-positive resections), 9.1% were community cancer programs, 47% were comprehensive community hospitals, and 43.9% were academic/research hospitals (Figure 3). Of the high-outlier hospitals (significantly greater than expected margin-positive resections), 29.9% were community cancer programs, 52.3% were comprehensive community hospitals, and 17.8% were academic/research institutions.

Figure 3.

Percentage of significantly low and high margin positive outliers by facility type, hospital volume and surgery/radiation sequence

Hospitals were stratified by yearly surgical volume into quartiles, and differences in positive-margin rates between significantly high- and low-volume hospitals were assessed (Figure 3). Among the low-outlier hospitals, > 80% were the highest quartile for volume (10.7–98.9 cases/year). No hospitals from the lowest quartile for volume (1–4.4 cases/year) were low outliers. However, high volume did not preclude high-outlier status, as 17.8% of the high margin-positive outliers were high-volume hospitals, while the remaining hospitals were fairly evenly distributed among mid–high-volume (32.7%), mid–low-volume (23.4%), and low-volume (26.2%) quartiles.

Treatment variables assessed included receipt of chemotherapy or radiation therapy and their sequence. We also examined type of surgery—low anterior resection, coloanal anastomosis, and abdominoperineal resection—to examine differences between the significant outlier groups. Rates of chemotherapy (54.1% vs 54%) or radiation therapy (46.5% vs 45.8%) between the low- and high-outlier hospitals did not differ meaningfully. However, there were notable differences in the sequence of surgery and radiation. Among low-outlier hospitals, 56.6% of the patients who received radiation received it prior to surgery, whereas among high-outlier hospitals, only 37.3% of patients received radiation prior to surgery. Finally, there was little difference between the low- and high-outlier groups with regard to specific operation. The majority underwent low anterior resection, whereas coloanal anastomosis was a more common procedure in the low-outlier hospitals (8.5%) than in the high-outlier hospitals (5.5%).

Nomogram

We used the logistic regression model (see Table 1 for details) to construct a nomogram for determining the expected rate of margin positivity for a given hospital (Figure 4). The nomogram has a bootstrap-corrected concordance index of 0.7474 (Appendix Figure 1), indicating good performance. The calibration plot demonstrates that the predicted probability derived from the nomogram corresponds well with the observed probability; and is slightly overestimated only when >0.3. This nomogram can thus be utilized to determine an individual hospital’s expected rate of margin-positive resection for comparison with its observed rate within the NCDB.

Figure 4.

Nomogram for predicting the probability of margin positive. To calculate the probability of positive margin, first obtain the value for each predictor by drawing a vertical line straight upward from that factor to the points axis, then sum the points achieved for each predictor and locate this sum on the total points axis of the nomogram where the probability of positive margin can be located by drawing a vertical line downward

DISCUSSION

Resection margin status is a primary determinant of long-term oncologic outcome among patients with rectal cancer. In this study, we utilized the NCDB to investigate the relationship between patient, tumor, and treatment variables and the risk for margin positivity during resection for rectal cancer. The analysis demonstrates a number of patient-related risk factors for margin-positive resection, including older age, male sex, African American race, and non-private insurance. Tumor-related risk factors include more advanced AJCC stage (especially T category), signet or mucinous histology, and poor/undifferentiated grade.

Using the adjusted risk model, we determined the ratio of observed to expected rates of margin positivity and identified 173 of 1064 hospitals that were significant high or low outliers. High-outlier status was associated with a lower rate of neoadjuvant radiotherapy, lower hospital volume and non academic/research hospital type, whereas low-outlier status was associated with a higher rate of neoadjuvant radiotherapy and high-volume or academic/research hospital type. However, it was highly notable that neither academic/research hospital type or high hospital volume precluded high- outlier status.

The historically high rates of local failure following resection of rectal cancer were dramatically improved with the widespread adoption of the sharp total mesorectal excision technique (TME).^4,7,18,19 TME improves clearance of regional lymph nodes and achievement of negative circumferential resection margins. Previous studies of risk factors for circumferential resection margin positivity have focused on tumor factors such as T and N categories, tumor size and location, vascular invasion, and the need for abdominoperineal resection.^14,20–22 Other studies have reported associations between higher provider and hospital volumes and improved outcomes for oncologic procedures. ^23–25 In the present analysis, we have identified additional patient and tumor factors associated with underlying risk for margin-positive resection. Additionally, we have identified the use of preoperative radiotherapy and treating facility characteristics as important treatment factors accounting for outlier status and significant variance from risk-adjusted expected rates of margin positivity. We then developed a nomogram that individual hospitals can use to determine their own risk-adjusted expected margin-positivity rates. With this nomogram, individual hospital based cancer programs can utilize their own previously abstracted data to determine their expected rate of margin positive resection and compare to the benchmark expected rate based on their individual patient and tumor mix. If significant variance it identified, it can alert the cancer program to consider quality improvement initiatives to decrease their rates of margin positivity.

Resection margins have a significant impact on both local recurrence and disease-free survival. ^5,6,26–28 Neither preoperative radiotherapy nor adjuvant chemotherapy can compensate for margin positivity; however, randomized trials have demonstrated that neoadjuvant radiotherapy can improve local control. ^29–31 Furthermore, preoperative combined modality chemotherapy and radiation has been associated with improved local control, improved margin negative resection, and potential for sphincter preservation and disease free survival than post-operative therapy. ^31–33 Our analysis indicates that patients who receive preoperative radiation therapy have a higher likelihood of having been treated in a low outlier hospital; whereas those receiving postoperative therapy are more likely to have been treated in a high outlier hospital, identifying a simple key potentially modifiable treatment factor that may help hospitals improve their negative resection margin rates.

In this analysis, we have focused on comparing the ratio of the observed to expected rates of margin positivity within the NCDB. Outlier status was not exclusively predicted by hospital volume, nor by academic/research or non-academic/research hospital type, i.e. no hospital type was exempt from high-outlier status, nor could be ensured low-outlier status Thus the analysis is applicable and relevant for all types of individual hospitals that want to compare their observed rate to their risk-adjusted expected rate. All facility types may have factors in their practice that may be targeted to improve rectal cancer outcomes. High outlier hospitals can look for modifiable treatment factors to alter their clinical practice to improve rates and outcomes. Utilizing the O/E ratio allows a hospital to evaluate itself while controlling for its mix of patient and tumor risk factors. We have created a simple-to-use nomogram to facilitate this evaluation.

This study has important limitations as well as strengths. Analysis of a large database is subject to data constraints, incorrect or missing data, and inability to account for pathologic or surgical variability. The Commission on Cancer does conduct abstraction training, as well as annual quality control audits, to minimize these errors in the database. We also noted that the rate of margin positivity was higher during the later years of the study suggesting a secular trend in reporting bias as it the importance of margin status has become increasingly recognized. However, the analysis remains valid for individual hospitals during the reporting period as their individual patient and tumor characteristics served as the basis for the model. Moreover, this study includes only data from Commission on Cancer hospitals, so hospital selection bias may exist. The percentage of minorities in this study is smaller than expected (7% African American in this study versus 12.6% in the 2010 U.S. Census) suggesting this potential for bias. Information regarding provider specialty or experience was not available and the NCDB did not collect data on comorbid conditions until 2003, therefore the impact of these variables is unknown. However, while less surgeon familiarity with rectal cancer surgery may have been associated with margin positivity, being an academic/research or higher volume program did not preclude high-outlier status. Finally, while the nomogram was not independently validated, the information is taken from the NCDB which collects data on approximately 76% of incident cancer cases in the US as thus represents the majority of the US population with crectal cancer. For internal validation, we did perform bootstrapping and identified good performance with a concordance index of 0.75. The calibration plot also demonstrates that the predicted probability derived from the nomogram corresponds well with the observed probability.

CONCLUSION

Rates of margin positivity have decreased since introduction of total mesorectal excision, but margin positivity remains a major determinant of local recurrence, metastases, and overall survival. This study identified both patient- and tumor-related risk factors for margin positivity as well as treatment factors such as use of preoperative radiation therapy and facility characteristics that are associated with rates of positive-margin resection. Although type of hospital and volume were associated with outcomes, no hospital was immune to being a high outlier for margin positivity. The nomogram for determining risk-adjusted expected rate of margin positivity that has thus been developed in this study to permit hospitals to evaluate their own performance and identify potential areas for process improvement.