Sensitivity of Medicare claims to identify cancer recurrence in elderly colorectal and breast cancer patients

Abstract

Background

Researchers are increasingly interested in using observational data to evaluate cancer outcomes following treatment, including cancer recurrence and disease-free survival. Because population-based cancer registries do not collect recurrence data, recurrence is often imputed from health claims, primarily by identifying later cancer treatments after initial treatment. The validity of this approach has not been established.

Research Design

We used the linked SEER- Medicare data to assess the sensitivity of Medicare claims for cancer recurrence in patients very likely to have had a recurrence. We selected newly diagnosed stage II/III colorectal (CRC n=6,910) and female breast cancer (n=3,826) patients during 1994–2003 who received initial cancer surgery, had a treatment break, and then died from cancer in 1994–2008. We reviewed all claims from the treatment break until death for indicators of recurrence. We focused on additional cancer treatment (surgery, chemotherapy, radiation therapy) as the primary indicator, and used multivariate logistic regression analysis to evaluate patient factors associated with additional treatment. We also assessed metastasis diagnoses and end-of-life care as recurrence indicators.

Results

Additional treatment was the first indicator of recurrence for 38.8% of CRC patients and 35.2% of breast cancer patients. Patients ages 70 and older were less likely to have additional treatment (p<0.05), in adjusted analyses. Over 20% of patients either had no recurrence indicator before death or had end-of-life care as their first indicator.

Conclusions

Identifying recurrence through additional cancer treatment in Medicare claims will miss a large percentage of patients with recurrences; particularly those who are older.

INTRODUCTION

Researchers are increasingly interested in assessing outcomes such as recurrence and disease-free survival (DFS) following cancer treatment. Population-based cancer registries collect information about all incident cancers reported within defined geographic areas and some registries also conduct follow-up of their patients to determine vital status. However, registries do not collect or report information on cancer recurrence. Registry collection of recurrence and timing of recurrence data would be logistically challenging and very costly. Currently information on recurrence has been limited to data reported from clinical trials, which include only small numbers of patients and are not representative of cancer patients overall.

To overcome the lack of recurrence information in the cancer registry data, researchers have used health claims to impute recurrence and DFS for cancer patients by identifying claims for cancer treatment or metastatic disease after the initial treatment period. Validations of imputed recurrence from claims against medical records have been limited by very small sample sizes and lack of generalizability.^1,2

Despite the lack of population-based validation, a number of studies have used health claims to impute cancer recurrence.^3–10 These studies have primarily defined recurrence through algorithms that have identified surgery, chemotherapy or radiation therapy (RT) administered beyond the time of initial treatment, as reported on Medicare claims. This approach will misclassify patients if they do not receive treatment for their recurrence, a particular concern for elderly patients who are not offered or may decline additional cancer treatment. To date, there has been no population-based assessment of the sensitivity of Medicare claims for identifying recurrences from treatment-based algorithms.

In this study, we used the linked SEER-Medicare data to assess the sensitivity of Medicare claims to capture recurrence for patients diagnosed with Stage II/III colorectal (CRC) or female breast cancers. We chose these cancers and stages because they are common and these patients are likely to undergo initial curative treatment, with a sizeable percentage having cancer recurrence following their initial care.

Although the SEER-Medicare data do not explicitly capture recurrence, we identified patients very likely to have had a recurrence, those with only a single cancer diagnosis who were surgically treated, had a cancer treatment-free interval, and later died from their cancer. Our assessment included reviewing Medicare claims for possible indicators of recurrence following initial care. Our primary focus was whether claims for additional cancer treatment (defined as surgery, chemotherapy and RT) were sensitive indicators of whether a patient had a recurrence. We assessed if there was variation in the characteristics of patients who receive additional treatment for recurrence. In addition, we assessed if other measures-- diagnosis codes for metastasis or end-of-life care-- were sensitive indicators of recurrence. For an indicator of recurrence to be useful for studies of disease-free survival, it must be reported accurately near the time when the recurrence was first diagnosed. Therefore, we also examined the timing of each of the indicators in relation to time of diagnosis and death.

METHODS

Data sources

The SEER-Medicare data used for this analysis result from the linkage of cancer records for persons in the National Cancer Institute’s (NCI) Surveillance, Epidemiology and End Results (SEER) registries to Medicare claims.¹¹ The SEER data are population-based and include clinical information about incident cancer for patients living within defined geographic areas, representing approximately 28% of the US population.¹² For each patient, the SEER data contain a unique case number; demographic information such as age, sex and race, the number of primary incident cancers; month and year of diagnosis; type of surgery performed; site and stage of disease at diagnosis; and vital status. For patients who died, information about the month and year of death and underlying cause of death was obtained from the National Death Index. Cause of death on death certificates is reportedly independently of diagnoses reported on claims. The Medicare data used in this analysis include all inpatient hospitalizations, outpatient hospital services, physician services, durable medical equipment claims, and hospice service for beneficiaries with fee-for-service coverage. All files include dates of service and codes for specific diagnoses and procedures using either ICD-9-CM codes, Health Care Procedure Codes (HCPCS), or National Drug Codes (NDC). For persons reported to SEER who were age 65 or older, 94% have been linked to the Medicare enrollment file. Using the linked SEER-Medicare data allowed us to track patients across time from cancer diagnosis until death. The NIH's Office of Human Subjects Research has determined that the SEER-Medicare data are exempt from human subjects regulations.

Sample selection

The cohorts included all beneficiaries ages 65 and older who were diagnosed between 1994 and 2003 with Stages II or III CRC (n=18,127) or female breast cancer (n=8,173) as their only primary cancer who also died of cancer between 1994 and 2008. Patients identified only from death certificates or autopsy were excluded (CRC n=30; breast n=11). To ensure that all claims were available, patients were required to be enrolled in Medicare Part A and Part B coverage and not in an HMO from the month of their diagnosis until death (excluded: CRC n=5,560; breast n=2,514). All patients were required to have received cancer-directed surgery as initial treatment within 4 months of their cancer diagnosis. Cancer-directed surgery was identified from Medicare claims. If no surgery could be identified from the Medicare data, then surgery was identified from the SEER data. Patients without cancer-directed surgery were excluded (CRC n=2,160; breast n=1,085).

Identification of Surveillance Period to Ascertain Recurrences

To identify the “surveillance period”, the time after initial treatment when recurrences could be identified, we reviewed health claims from cancer diagnosis until death. For each patient, we divided the claims after diagnosis into three periods: initial treatment, treatment-free interval, and surveillance (Figure 1). The initial treatment period started on the date of cancer-directed surgery as reported on the Medicare claim or, if cancer directed surgery was only identified from the SEER data (and no date of service was available), the last day of the month of diagnosis. Many patients received adjuvant therapy following their surgery, and as a result, the ending date for the initial treatment period for patients receiving adjuvant therapy was later than for those who only had surgery. To determine whether a patient received adjuvant therapy, we reviewed Medicare claims for 4 months following the date of surgery to identify bills for chemotherapy or RT. If adjuvant therapy was not identified, patients were defined as having surgery only and their initial treatment period was defined as 4 months after the date of surgery. For patients who received adjuvant therapy, we reviewed their claims for up to 12 months following the date of surgery to determine when their adjuvant treatment ended, defined as no claims for chemotherapy or RT for 90 consecutive days. The date from the last adjuvant treatment claim was the end date of their initial treatment period. Patients could have a break in adjuvant treatment of up to 60 days, potentially reflecting adverse events from chemotherapy or a short term delay. We excluded patients with a break in treatment of more than 60 days but less than 90 days because we could not determine if the resumption of treatment after an extended break reflected additional initial therapy or treatment for recurrent disease. We also excluded patients whose initial treatment extended beyond 12 months (CRC n=3,467; breast n=737).

Figure 1.

Periods of Observation Used to Identify Surveillance Period

The treatment-free interval was defined as 90 days following the end date of the initial treatment period. Following the last date of the treatment-free interval, patients entered the surveillance period. The surveillance period extended until the last day of their month and year of death. The surveillance period was the focus of our study for reviewing indicators of recurrence.

Indicators of Recurrence

During the surveillance period, we reviewed Medicare hospital, physician, outpatient facility and durable medical equipment claims for additional cancer therapy-chemotherapy, RT, or cancer-related surgical procedures. Our evaluation did not include hormone therapy for breast cancer patients as most elderly breast cancer patients initiate hormone therapy at the time of diagnosis and continue treatment for five years following diagnosis. We also reviewed claims for metastasis diagnosis and end-of-life care, including services such as hospice care and pain management. A complete list of diagnoses and procedures used for each indicator of recurrence can be found in the Appendix (Table).

Defining the Time of Recurrence

First Indicator of Recurrence

We reviewed claims to determine which indicator of recurrence was found as the first indicator. Because more than one indicator could occur simultaneously, we created a mutually exclusive hierarchy for defining the first indicator: additional therapy, diagnosis of metastatic disease, or end-of-life care. Some patients with a metastasis diagnosis as their first indicator may have been referred shortly thereafter for additional treatment. Therefore, we reviewed their claims to determine if they received additional therapy within 2 weeks of the metastasis diagnosis. If additional therapy was identified, we classified the first indicator as being “additional therapy” rather than a metastasis diagnosis. We also determined the number of patients with no indicator of recurrence.

Given increasing interest in measuring disease-free survival, we compared the median time from diagnosis until recurrence for each first indicator of recurrence. We also estimated for each first indicator the median time from recurrence until death. Median times are presented in forest plot graphs, showing variation in median time by reporting the lower and upper quartiles. We report median times instead of mean times, as the means were skewed by outliers.

Assessing Patient Factors Associated with Receipt of Additional Therapy

Because prior studies have used receipt of additional cancer treatment to impute recurrence,^3–10 we calculated the percent of patients who had additional treatment at at any time during the surveillance period., We used separate multivariate logistic regression models to assess the association between stage at diagnosis, patient age, race or sex and receipt of additional treatment for CRC and breast cancer patients. All analyses were performed using SAS software, version 9.2 (SAS Institute Inc, Cary, NC).

RESULTS

The final cohort included 6,910 CRC patients and 3,826 breast cancer patients. (Table 1). Sixty-four percent of CRC patients and 56.6% of breast cancer patients were age 75 and over. Most CRC patients presented with Stage III cancer (59.7%), whereas most breast cancer patients presented with stage II (74.2%). To evaluate potential biases introduced by our cohort selection, we compared the age at diagnosis for patients in our cohort with all elderly CRC and breast patients in the SEER-Medicare data who had the same clinical presentation and years of diagnosis but did not die from cancer. The age distributions for the two cohorts were similar, although those who died from cancer were slightly more likely to be age 75+ than those who did not die (CRC-64.1% vs. 59.9%; breast cancer-56.7% vs. 53.1%). Following diagnosis, mean survival for CRC patients was 34.3 months, with a mean of 25.9 months in the surveillance period. For breast cancer patients, mean survival was 45.6 months with 37.0 months in the surveillance period.

Table 1.

Descriptive characteristics of elderly patients diagnosed, 1994–2003, with Stage II/III colorectal or breast cancer who died of cancer between 1994–2008

			Colorectal Cancer						Breast Cancer
			Mean Survival		Mean Surveillance Period				Mean Survival		Mean Surveillance Period
	n=	%	Months	95% CIs	Months	95% CIs	n=	%	Months	95% CIs	Months	95% CIs
Total cases	6910	100	34.3	(33.7,34.8)	25.9	(25.4,26.4)	3826		45.6	(44.6,46.5)	37.0	(36.1,37.9)
Age Group
65–69	1064	15.4	39.4	(38.0,40.9)	30.4	(28.9,31.8)	778	20.3	51.9	(49.6,54.1)	42.4	(40.2,44.6)
70–74	1417	20.5	37.3	(36.0,38.5)	28.5	(27.3,29.8)	882	23.1	51.3	(49.3,53.2)	42.4	(40.4,44.3)
75–79	1607	23.3	35.5	(34.4,36.6)	27.1	(26.0,28.2)	867	22.7	46.5	(44.6,48.4)	37.9	(36.0,39.8)
80+	2822	40.8	30.1	(29.3,30.8)	22.2	(21.5,23.0)	1299	34.0	37.3	(35.9,38.7)	29.4	(28.0,30.8)
Race
White	6038	87.4	34.4	(33.8,35.0)	26.0	(25.5,26.6)	3363	87.9	45.9	(45.0,46.9)	37.4	(36.4,38.4)
Black	539	7.8	32.4	(30.5,34.3)	23.9	(22.1,25.8)	351	9.2	40.6	(37.9,43.3)	32.1	(29.4,34.8)
Other/Unknown	333	4.8	34.9	(32.5,37.3)	26.5	(24.2,28.9)	112	2.9	49.4	(43.7,55.0)	40.5	(34.9,46.1)
Sex
Male	2941	42.6	36.2	(35.4,37.1)	27.7	(26.9,28.5)	----	----	----	----	----	----
Female	3969	57.4	32.8	(32.1,33.5)	24.6	(23.9,25.3)	3826	100	5.6	(44.6,46.5)	37.0	(36.1,37.9)
Stage
2	2788	40.3	37.9	(37.0,38.8)	30.0	(29.1,30.9)	2837	74.2	47.9	(46.8,49.0)	39.4	(38.3,40.5)
3	4122	59.7	31.8	(31.2,32.5)	23.1	(22.5,23.8)	989	25.8	38.7	(37.1,40.3)	30.0	(28.4,31.6)

First indicator of recurrence

For both CRC and breast cancer patients, slightly more than one-third had additional therapy as their first indicator of recurrence; the type of additional therapy varied by cancer site and stage (Table 2). A diagnosis of metastatic disease (without additional therapy within 2 weeks) was also a common first indicator, reported in 36.0% of the CRC patients and 45.7% of the breast cancer patients. The remaining patients either had no indicator of recurrence or the first indicator was end-of-life care, 25.2% and 19.1% for CRC and breast cancer patients, respectively.

Table 2.

First indicators of recurrence in Medicare claims among elderly patients diagnosed, 1994–2003, with Stage II/III colorectal or breast cancer who died of cancer between 1994–2008

	First* Indicator of Recurrence (%)
	Colorectal			Breast
Indicator**	Total	Stage II	Stage III	Total	Stage II	Stage III
Additional Therapy:	38.8	39.5	38.4	35.2	34.1	38.4
Cancer-related Surgery	18.3	22.2	15.6	5.6	6	4.7
Chemotherapy	16.4	13.1	18.5	13.4	12.5	15.8
RT	4.2	4.2	4.2	16.3	15.7	18
Diagnosis of metastatic disease	36.0	34.9	36.8	45.7	46.6	43
End-of-life care	16.7	16.7	16.6	12.8	12.9	12.3
No indicator of recurrence	8.5	8.9	8.2	6.3	6.3	6.3

^*First indicator includes hierarchy of all events within 14 days of first event

^**Mutually exclusive, indicators in italics are subsets of Additional Therapy

RT=radiation therapy

Patient characteristics associated with additional cancer treatment

The percentage of patients ever having a claim for additional cancer therapy at any time during the surveillance period was similar for CRC and breast cancer patients, 60.0% and 62.2%, respectively (data not shown). Patient characteristics were associated with receipt of additional cancer treatment in adjusted analyses (Table 3). Older patients, particularly those aged 80 and older, were significantly less likely to receive additional treatment compared to patients ages 65–69 (CRC OR =0.23; 95% CI: 0.19–0.27 and breast OR = 0.16; 95% CI: 0.13–0.20, respectively). For CRC patients, women were significantly less likely to receive additional cancer therapy than were men (OR= 0.87; 95% CI: 0.79–0.97).

Table 3.

Percent of patients receiving additional therapy for recurrence and adjusted odds of receiving therapy by selected patient characteristics^*

	Colorectal Cancer			Female Breast Cancer
Age Group	%	aORs	(95% CI)	%	aORs	(95% CI)
65–69**	78.5	ref	----	80.6	ref	----
70–74	71.3	0.68	0.57–0.82	75.1	0.72	0.57–0.92
75–79	64.5	0.50	0.42–0.60	65.1	0.45	0.36–0.56
80+	44.9	0.23	0.19–0.27	40.6	0.16	0.13–0.20
Stage
2**	59.7	ref	----	62.1	ref	----
3	60.2	0.96	0.87–1.06	62.5	1.09	0.93–1.28
Race
White **	59.8	ref	----	62.0	ref	----
Black	58.6	0.83	0.69–1.00	60.7	0.82	0.64–1.04
Other/Unknown	65.8	1.12	0.88–1.42	75.0	1.63	1.03–2.58
Sex
Male**	64.2	ref	----	----	----	----
Female	56.9	0.87	0.79–0.97	62.2	----	----

^*Additional Therapy includes patients who ever had chemotherapy, RT, or cancer-related surgery

^**reference group

aORs=adjusted odds ratios; adjusted for the listed variables

Timing of recurrence

The median time from diagnosis until the first indicator of recurrence varied by the cancer site, stage at diagnosis, and indicator (Figure 2). For both CRC and breast cancer patients, those with additional therapy as their first indicator of recurrence had shorter median times to recurrence than for those with other first indicators. For CRC patients, both metastasis and end-of-life indicators occurred about 2 months later than additional therapy, while for breast cancer patients, metastasis indicators and end-of-life indicators were found about 5 and 7 months later than additional therapy indicators. A reverse pattern was observed for the median time from the first indicator of recurrence until death (Figure 3). The longest median survival following first indicator of recurrence was observed for patients whose first indicator was additional treatment, while the shortest median survival was for patients with end-of-life care as the first indicator, 7 months shorter for CRC patients and 10 months shorter for breast cancer patients.

Figure 2.

Median time in months from diagnosis to first indicator of recurrence, by type of indicator, for elderly patients diagnosed with Stage II/III colorectal or breast cancer who died of cancer

Figure 3.

Median time in months from first indicator of recurrence until death, by type of indicator, for elderly patients diagnosed with Stage II/III colorectal or breast cancer who died of cancer

DISCUSSION

In this study, we assessed the sensitivity of the Medicare claims to detect cancer recurrence in patients who received initial cancer directed treatment, had a treatment-free interval, and later died from their cancer. Our results demonstrate that algorithms that rely only on ascertainment of additional cancer treatment to identify recurrence will significantly underestimate its frequency. Less than 40% of CRC and breast cancer patients had additional treatment as their first indicator of recurrence and about a third never had additional treatment reported in the Medicare claims. Patients who do not receive additional cancer treatment were primarily persons ages 70 and older and for CRC, women. These findings suggest that prior studies that have relied on additional treatment codes in the SEER-Medicare data to impute recurrence are systematically misclassifying recurrences, particularly for elderly populations, and the findings should be interpreted cautiously. ^3–10

While this study was conducted in CRC and breast cancer patients, we do not believe that treatment-based algorithms would have better performance for patients with other malignancies. Indeed, we expect that treatment-based algorithms would have poorer performance for prostate cancer. Because a sizeable number of prostate cancer patients either do not receive any therapy at diagnosis, or receive hormone ablation therapy as primary or neoadjuvant therapy, it is difficult to identify whether claims for additional therapies such as surgery or RT represent primary therapy or secondary therapy at recurrence.

Metastasis codes were found to be a frequent first indicator of recurrence in our analysis. Prior studies have reported a relatively low positive predictive value (PPV) of metastasis codes on claims for identifying metastatic disease in the SEER-Medicare data, ranging from 58–66% for breast cancer and 64–69% for colorectal cancer.^13–15 Although the addition of metastasis codes will improve the sensitivity of treatment-based algorithms for identifying recurrence, about one-third of patients will be incorrectly classified as having metastatic disease when they do not. Thus, we conclude that metastasis codes on Medicare claims are not an accurate way to identify recurrence.

About 25% of CRC patients and almost 20% of breast cancer patients either had no indicator of recurrence before death or had end-of-life care, shortly before death, as their first indication of recurrence. For these patients, there is no way to infer from claims when recurrence was first diagnosed.

This study has a number of strengths. We developed a definition of initial treatment that considered the variation of patterns of initial care, accounting for time required for adjuvant therapy. This approach likely resulted in a more accurate identification for each patient of when initial treatment actually ended than if we had applied a fixed time for initial treatment to all patients. Our analysis included a large population-based cohort of Medicare enrollees diagnosed with non-metastatic cancer who later died from cancer. Use of Medicare claims allowed for longitudinal tracking of recurrence indicators. Reporting of surgery, chemotherapy and RT from the Medicare and SEER data has been found to be complete.^16–20 In addition, Medicare has a designated hospice benefit which allowed us to identify accurately when patients initiated hospice care.

There were also limitations with this study. The SEER-Medicare data do not report if and when a recurrence occurs, and as a result, we did not have a “gold standard”, such as a medical record, to evaluate claims for identifying recurrence. Instead, we identified cohorts of patients who very likely experienced a recurrence - those with non-metastatic disease who received initial treatment, had a treatment-free interval and who died from their cancer. Our approach has face validity to capture a cohort of patients who had a recurrence. However, this approach excluded patients with recurrence who did not die of cancer within the period of observation. Thus we were not able to assess the predictive value or specificity of the Medicare claims to identify recurrence. We attributed all hospice admissions to the patient having cancer, although some patients who died from their cancer may have had other conditions that resulted in the hospice admission. Some patients may have had evidence of recurrence not captured in the Medicare claims, such as patients receiving care from other insurers, e.g. the Veteran’s Administration. In our analysis, we restricted the cohort to patients with a single cancer and any cancer death, relying on the death certificate for cause of death. There have been questions about the accuracy of cause of death on death certificates for some conditions. However several studies that have focused exclusively on cancer have found high agreement between clinicians’ assessment of cause of death with what was reported from the death certificate. ^21–23 Finally, although the age distribution of the Medicare patients who died from cancer was relatively similar to those with a comparable clinical presentation who did not die from cancer, those who did not die from cancer may be more likely to have been treated or differed systematically from those who died from their cancer. These findings may not be generalizable to younger populations who may be more likely to receive cancer therapy following recurrence.

Findings from our analysis have practical implications for studies designed to use health claims for studies of comparative effectiveness or cancer outcomes. With an increasing number of anti-cancer therapies and as clinical trials become more expensive and challenging to conduct, there is mounting interest in leveraging health claims to evaluate the real world consequences of treatment interventions. For cancers with favorable prognosis, overall survival is an imperfect endpoint. Evaluation of disease-free and recurrence-free survival is of increasing importance. Unfortunately, our findings demonstrate that Medicare claims are not a viable resource for estimating recurrence. Ideally, as software for electronic health records (EHR) is being refined, development of standardized approaches for defining and recording recurrence will be a priority for the oncology community. Such information will enable improved cancer surveillance and better assessment of treatment effectiveness. Until information from the EHR is available, use of Medicare data to assess recurrence will miss many patients and those identified will not be representative of all patients with recurrence.