IDENTIFYING SPECIFIC CHEMOTHERAPEUTIC AGENTS IN MEDICARE DATA: A VALIDATION STUDY

Abstract

Background

Large healthcare databases are increasingly used to examine the dissemination and benefits and harms of chemotherapy treatment in routine practice, particularly among patients excluded from trials (e.g., the elderly). Misclassification of chemotherapy could bias estimates of frequency and association, warranting an updated assessment.

Methods

We evaluated the validity of Medicare claims to identify receipt of chemotherapy and specific agents delivered to elderly stage II/III colorectal (CRC), in situ/early stage breast, non-small cell lung, and ovarian cancer patients using the National Cancer Institute’s Patterns of Care studies (POC) as the gold standard. The POC collected data on chemotherapy treatment by re-abstracting hospital records, contacting physicians, and reviewing medical records. Patients’ POC data were linked and compared to their Medicare claims for 2–12 months post-diagnosis. Kappa, sensitivity (Se), specificity (Sp), positive and negative predictive values and 95% confidence intervals were calculated for the receipt of any chemotherapy and specific agents.

Results

Se and Sp of Medicare claims to identify any chemotherapy were high across all cancer sites. We found substantial variation in validity across agents, by site and administration modality. Capecitabine, an oral CRC treatment, was identified in claims with high specificity (98%) but low sensitivity (47%), whereas oxaliplatin, an intravenously administered CRC agent had higher sensitivity (75%) and similar specificity (97%).

Conclusions

Receipt of chemotherapy and specific intravenous agents can be identified using Medicare claims, showing improvement from prior reports; yet, variation exists. Future studies should assess newly-approved agents and the impact of coverage decisions for these agents under the Medicare Part D program.

Introduction

Chemotherapy represents an integral part of the treatment plan for many individuals diagnosed with cancer, as it decreases the risk of recurrence and mortality in many settings. Randomized controlled trials have documented the efficacy of chemotherapeutic agents used to treat a variety of cancers. To examine the translation of this evidence into the routine clinical setting, large healthcare databases, such as the Surveillance, Epidemiology, and End Results (SEER) program-Medicare linked database, are increasingly used to conduct non-experimental studies evaluating the uses, benefits, and harms of these treatments among individuals excluded from trials, including older adults, those with multiple co-morbidities, and those treated off-label.(1–26)

The validity of these studies relies upon a variety of issues, including the ability of claims data to accurately capture treatment(s) of interest, study endpoint(s), and other important design and clinical issues.(27) Measurement error in the assessment of chemotherapy could lead to biased study results. Prior research supports the validity of claims data to identify intravenously administered chemotherapy treatment for a variety of cancer sites,(28–32) but does not address more recently approved or orally administered agents, or changes in validity using multiple claims windows following diagnosis.

We conducted a validation study to assess the utility of Medicare claims for capturing the receipt of any chemotherapy and specific agents delivered to patients diagnosed at age ≥65 with stage II or III colorectal cancer (CRC), in situ or early stage breast, non-small cell lung cancer (NSCLC), or ovarian cancer. This assessment 1) evaluated the validity of selected single agent chemotherapies, including an orally-administered agent and 2) described the variation in measures of validity for any chemotherapy and specific treatments over multiple follow-up periods and across cancer sites.

Methods

Data sources

We used the National Cancer Institute (NCI)’s data from the Patterns of Care studies (POC) as the gold standard for identifying chemotherapy and the linked SEER-Medicare data as the test source for identifying chemotherapy. The SEER program of cancer registries collects demographic information, clinical and tumor characteristics, vital status, and cause of death for all incident cancers reported for individuals who reside in one of the registries’ defined geographic areas.(33)

NCI supplements the standard SEER registry abstraction to obtain detailed information about treatment for a subset of SEER cases. This effort, known as the POC, was developed by NCI to investigate the dissemination of state-of-the-art cancer treatment into community practices. These studies selected a stratified random sample of individuals (proportionate registry size) from the SEER program 10, 12, and 13 cancer registries which covered up to 14% of the United States population.(34) All individuals were aged ≥20 years with a histologically confirmed cancer for selected sites, stages, and years. A listing of all cancers and stages examined by the POC are detailed elsewhere.(35) Patients were excluded if the cancer diagnosis was determined at autopsy or on the death certificate; the diagnosis was a second malignancy other than to a non-melanoma skin cancer; or if the individual was simultaneously diagnosed with another cancer. Individuals were sampled by gender with oversampling of African-Americans and Hispanics in all years and Asian/Pacific Islanders and American Indians/Alaskan Natives in 2005 only.

In addition to the standard SEER abstraction, the POC studies supplemented information on initial course of treatment by asking physicians (via mailed questionnaire) to verify the treatments delivered to patients; reviewing a unified medical record (inpatient and outpatient); and in some cases SEER registrars visited doctors’ offices to abstract data. Requested information included whether radiation, chemotherapy or immunotherapy was received as part of the initial course of treatment, identifying the specific agents delivered and the dates of first administration (2005 studies only).

The SEER-Medicare data arise from a linkage of persons in the SEER data with their Medicare enrollment, Part A (Hospital insurance) and B (Medical insurance) claims data. These data include approximately 3.3 million elderly individuals (age ≥ 65 years) diagnosed with cancer in one of the SEER areas or regions.(36) Approximately 94% of all elderly individuals included in SEER have been matched to the Medicare enrollment file with an established matching algorithm. Virtually 100% of all beneficiaries are eligible for Part A and 93% will opt to enroll in Part B.(37)

For Medicare-eligible individuals with fee-for-service coverage, Medicare claims are organized into files including claims for inpatient hospitalizations, durable medical equipment (DME), outpatient hospital services, and physician and other provider services (32). These claims encompass a multitude of information on specific service dates, diagnoses, procedures, and agents delivered during medical encounters using various medical coding systems. Diagnoses and procedures on hospital claims are reported using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9 CM) codes. ICD-9 CM diagnosis and procedure codes can be used to identify chemotherapy administration, but not specific agents. DME claims contain National Drug Codes (NDCs) that can be used to identify specific oral chemotherapeutic agents that are equivalent to other Medicare-covered intravenously administered chemotherapy agents.(38) Physician and outpatient claims include ICD-9 CM diagnosis codes and Healthcare Common Procedure Coding System (HCPCS) codes. HCPCS can be used to identify chemotherapy and specific agents. Outpatient claims include revenue center codes which serve as another means of identifying chemotherapy administration. The codes used in our analysis are presented in the Appendix.

Study sample and eligibility criteria

The cancer sites, stages, and years of diagnoses were selected based on availability of the POC data and included in-situ or early stage breast cancer diagnosed in 2000 and 2005, stage II or III CRC in 2000 and 2005, NSCLC in 2005, and ovarian cancer in 2002. All POC patients were required to be age ≥65 at cancer diagnosis; and have POC treatment information verified through physician confirmation or a unified medical record review. Patients identified as being enrolled in a clinical trial were excluded because Medicare only covers routine costs associated with federally funded clinical trials (e.g, office visits and medical tests), and may not cover the cost of the agents themselves.(39)

This study included eligible patients in the POC data who were matched to the SEER-Medicare data. Using the Medicare files, we required that all individuals were continuously enrolled in Medicare Parts A and B for the 2-, 4-, 6-, 8-, 10-, or 12-month periods following diagnosis (the post-diagnosis periods); were never enrolled in a health maintenance organization (HMO) during the associated post-diagnosis periods; did not have a subsequent cancer diagnosis (as reported by SEER) in the year following the qualifying POC cancer diagnosis; and had at least one Medicare claim during the specified post-diagnosis period. These criteria ensured that detailed claims for all individuals in the study were reported to Medicare and were not attributable to the treatment of a subsequent cancer. Due to the time-varying nature of these criteria, the number of individuals eligible for analysis in each post-diagnosis period decreased over time. Details of the 6-month post-diagnosis cohort exclusions are listed in the Appendix.

Identification of receipt of chemotherapy and specific agents in POC and SEER-Medicare

For this analysis, the POC cohort was considered the gold standard measure for the receipt of any chemotherapy and for specific agents. Individuals were defined in POC as receiving any chemotherapy if a physician verified or a unified medical record identified that the individual was administered any chemotherapeutic agent. The receipt of specific agents was identified in POC through the same mechanism. For the POC studies conducted in 2005, the date of first administration was collected for each specific agent delivered. Therefore, the analysis defined the initial course of treatment as the diagnosis date (set to the first day of the month, as only month of diagnosis is reported by SEER) to 365 days following the diagnosis date. If treatment was received outside of the year following diagnosis, it was not considered part of the initial course of chemotherapy.

Identifying the receipt of any chemotherapy and specific agents in Medicare claims required an examination of multiple claims files and their associated diagnosis, procedure, and drug codes and service dates. If a claim for a general chemotherapy procedure code, a diagnosis code for chemotherapy administration, or HCPCS code or NDC for a specific agent was found, the individual was defined as having received chemotherapy during the specified post-diagnosis period. The receipt of specific chemotherapy agents were defined similarly by identifying at least one claim with a HCPCS code or NDC for the specific agent during the post-diagnosis period.

Comparison of chemotherapy reported in POC and SEER-Medicare

Reporting of the agreement between the two data sources and the validity of chemotherapy captured in Medicare claims was examined at interval periods using the 2, 4, 6, 8, 10 and 12-month post-diagnosis cohorts. Specifically, we estimated the Kappa and corresponding 95% CIs to assess concordance between the two data sources, as well as the sensitivity (Se), specificity (Sp), positive predictive value (PPV), and negative predictive value (NPV) and their corresponding 95% CIs of the Medicare claims definitions using the POC as the gold standard.

We selected the specific chemotherapeutic agents to be validated based on their frequency of use in the 6-month post-diagnosis period. Using sample size calculations, we maximized the accuracy of the Se and Sp estimates to have a minimal acceptable lower confidence limit that is less than 10% from the point estimate (40). Based upon this sample size calculation, we included only specific chemotherapeutic agents where the POC reported that there were 37 or more individuals receiving the treatment. Due to the small number of in situ and early stage breast cancer patients receiving chemotherapy, the 2000 and 2005 POC data were combined for analysis.

While the POC studies were considered the gold standard, they may be subject to measurement error in their reporting of initial chemotherapy treatment. Therefore, beyond reporting the Kappa to assess concordance between the two sources, we also conducted a sensitivity analysis to examine the impact of potential misclassification of the gold standard (i.e., the POC),(41) focusing on an example of oxaliplatin receipt among stage II or III CRC patients diagnosed in 2005.

All analyses were conducted in SAS 9.2 (SAS Institute, Cary, NC). This study was reviewed by the University of North Carolina at Chapel Hill Institutional Review Board (IRB) and was determined to be exempt from IRB approval.

Results

The final validation cohort included 1,187 individuals diagnosed with a primary cancer of the breast in 2000 (n=156) or 2005 (n=155), colon or rectum in 2000 (n=171) or 2005 (n=338), lung (non-small cell only) in 2005 (n=195), and ovary in 2002 (n=170) (Table 1). The percentage of patients receiving any chemotherapy in this cohort was 17% for in-situ/early stage breast cancer diagnosed in 2000 and 20% in 2005; 61% for stage II/III CRC diagnosed in 2000 and 52% in 2005; 78% for ovarian cancer diagnosed in 2002; and 49% for NSCLC diagnosed in 2005.

Table 1.

Characteristics of individuals aged 65 and older included in the Patterns of Care Studies^* who were not enrolled in a clinical trial and had Medicare fee-for-service coverage only in the 6-month period following cancer diagnosis

	Breast (2000)	Breast (2005)	Colorectal (2000)	Colorectal (2005)	Ovary (2002)	Non-Small Cell Lung (2005)
	(%) n=156	(%) n=155	(%) n=171	(%) n=338	(%) n=170	(%) n=197
Demographics
Gender
Male	0.0	0.0	46.8	43.5	0.0	50.8
Female	100	100	53.2	56.5	100	49.2
Age at diagnosis (mean, SD)	75 (7)	74 (7)	75 (7)	76 (8)	75 (7)	74 (6)
65 – 69	23.7	31.0	24.6	21.3	25.9	24.9
70 – 74	24.4	22.6	25.2	24.6	24.1	31.5
75 – 79	32.7	26.5	22.2	20.4	26.5	24.4
80 – 84	10.3	9.7	16.4	18.6	15.9	15.2
85+	9.0	10.3	11.7	15.1	7.7	4.1
Race
White Non-Hispanic	53.2	42.6	54.4	50.3	70.0	46.2
Black Non-Hispanic	21.2	24.5	14.6	17.8	16.5	23.4
Hispanic	13.5	18.1	13.5	14.8	4.1	12.7
Other	12.2	13.6	17.5	17.2	9.4	16.8
Unknown	0.0	1.3	0.0	0.0	0.0	1.0
Marital status
Married	43.6	43.2	53.8	54.7	54.1	53.3
Other	53.2	54.8	45.0	44.1	44.1	46.2
Unknown	3.2	1.9	1.2	1.2	1.8	0.5
Median household income†
≤ $30,000	26.3	21.9	16.96	26.0	26.5	17.3
$30,001 – $45,000	31.4	25.8	32.75	26.3	26.5	36.0
$45,001 – $60,000	25.0	28.4	24.56	21.3	22.9	21.8
≥ $60,001	17.3	23.9	25.73	26.3	24.1	24.9
High school education†
≤ 70%	17.3	22.58	22.22	24.3	16.5	21.3
71 – 80%	18.6	20.65	16.37	14.2	18.8	18.8
81 – 90%	40.4	29.68	32.75	32.3	34.1	37.6
> 90%	23.7	27.1	28.65	29.3	30.6	22.3
County of residence in metro areas size†
Over 1 million population	42.3	61.3	63.2	63.3	48.2	52.3
250,000 – 1 million population	25.0	23.9	20.5	15.4	23.5	28.4
All other counties	32.7	14.8	16.4	21.3	28.2	19.3
Tumor characteristics at diagnosis
Histologic grade
Well-differentiated	16.0	18.7	2.9	5.9	5.9	4.6
Moderately differentiated	36.5	43.9	67.3	66.9	12.9	23.4
Poorly/undifferentiated	32.1	27.1	28.3	26.2	48.8	32.5
Unknown	15.4	10.3	1.2	1.2	32.4	39.6
Tumor extent
Tis	23.1	23.9	0.0	0.0	0.0	0.0
T1	48.1	41.9	1.2	2.1	25.3	24.9
T2	24.4	29.0	5.3	5.0	18.8	32.0
T3	2.6	4.5	75.4	79.0	34.7	8.1
T4	0.0	0.0	18.1	13.9	0.0	24.4
Unknown	1.9	0.7	0.0	0.0	21.2	10.7
Metastasis
No	100.0	100	100	100	78.8	70.1
Yes	0.0	0.0	0.0	0.0	21.2	29.4
Unknown	0.0	0.0	0.0	0.0	0.0	0.5
Number of positive lymph nodes
None	50.0	43.9	40.4	52.4	25.3	27.4
1 – 3 nodes	10.3	23.2	36.3	30.8	8.2	5.6
≥ 4 nodes	8.3	9.1	15.8	11.2	0.6	2.6
Positive but number unknown	0.0	0.0	1.2	0.0	0.6	1.5
Unknown or nodes not examined	31.4	23.9	6.4	5.6	65.3	63.0

^*POC studies in 2000, 2002, and 2005 include the SEER 10, SEER 12, and SEER 13 registries, respectively.

^†Median household income, percentage of census tract with a high school education, and county of residence in metro area size are linked from 2000 Census data.

Figure 1 displays the sources of chemotherapy claims found in the Medicare files (hospital, physician, outpatient, DME, or multiple files) for all individuals included in the validation studies by cancer site and year of diagnosis. The large majority of individuals receiving chemotherapy only had claims reported in the physician file with very few individuals having claims identified in the hospital file only (< 3%). However, variation by cancer site and year of diagnosis was evident, reflecting different settings in which treatment was delivered by site and over time. For example, the approval of capecitabine in 2005 for CRC increased the percentage of individuals with claims identified using the DME file in 2005, as bills for orally administered agents appear primarily in the DME file. Chemotherapy claims for breast cancer were largely identified by physician claims in both 2000 and 2005.

Figure 1.

Sources of chemotherapy claims for the year following diagnosis reported by Medicare for all individuals aged ≥65 years in the POC studies, by selected cancer site and year of diagnosis.

The comparisons of any chemotherapy identified by the POC and Medicare claims for the post-diagnosis periods for each cancer site/year are reported in Table 2. Individuals receiving chemotherapy according to each data source is reported. Overall, the measures of agreement and validity for identifying the receipt of any chemotherapy were high for all cancer sites and post-diagnosis periods, except for the 2- and 4-month periods. Excluding those periods, Kappa estimates of concordance ranged from 77% – 87%; Se ranged from 84% – 97%, Sp ranged from 78% – 97%, PPVs ranged from 87% – 96%, and NPVs ranged from 81% – 96%. The Sp estimates for the receipt of any chemotherapy for women diagnosed with ovarian cancer in 2002 were low in the later post-diagnosis periods. Due to the small number of women not receiving chemotherapy in the later post-diagnosis periods, the Sp estimates are unstable. Although the confidence intervals are wide, these intervals include Sp ranges that are consistent with estimates across other cancer sites. Across all cancer sites and year, the Sp and Se estimates for the receipt of any chemotherapy did not vary by patient characteristics (data not shown).

Table 2.

Comparison of any chemotherapy identified by SEER POC data and Medicare claims during various post-diagnosis periods for selected cancer sites and year

	Source reporting receipt of chemotherapy
	POC=Yes, Med=Yes	POC=No, Med=No	POC=Yes, Med=No	POC=No, Med=Yes	Kappa (%) (95% CI)	Se (%) (95% CI)	Sp (%) (95% CI)†	PPV (%) (95% CI)	NPV (%) (95% CI)
Breast (2000 and 2005)
2 months	11	259	46	2	27 (7, 46)	19 (10, 32)	99 (97, 100)	85 (55, 98)	85 (80, 89)
4 months	45	252	13	6	79 (70, 88)	78 (65, 87)	98 (95, 99)	88 (76, 96)	95 (92, 97)
6 months	48	247	9	7	83 (74, 91)	84 (72, 93)	97 (94, 99)	87 (76, 95)	96 (93, 98)
8 months	48	245	7	7	84 (77, 92)	87 (76, 95)	97 (94, 99)	87 (76, 95)	97 (94, 99)
10 months	48	240	7	8	83 (75, 92)	87 (76, 95)	97 (94, 99)	86 (74, 94)	97 (94, 99)
12 months	49	240	6	8	85 (77, 93)	89 (78, 96)	97 (94, 99)	86 (74, 94)	98 (95, 99)
Colorectal (2000)
2 months	45	78	61	3	36 (23, 49)	42 (33, 52)	96 (90, 99)	94 (83, 99)	56 (47, 65)
4 months	90	66	15	6	76 (66, 86)	86 (78, 92)	92 (83, 97)	94 (87, 98)	81 (71, 89)
6 months	92	60	12	7	77 (67, 87)	88 (81, 94)	90 (80, 96)	93 (86, 97)	83 (73, 91)
8 months	93	53	8	8	79 (69, 89)	92 (85, 97)	87 (76, 94)	92 (85, 97)	87 (76, 94)
10 months	91	50	7	8	79 (69, 89)	93 (86, 97)	86 (75, 94)	92 (85, 96)	88 (76, 95)
12 months	88	48	7	10	76 (65, 87)	93 (85, 97)	83 (71, 91)	90 (82, 95)	87 (76, 95)
Colorectal (2005)
2 months	70	172	115	3	36 (26, 45)	38 (31, 45)	98 (95, 100)	96 (88, 99)	60 (54, 66)
4 months	145	157	34	8	76 (69, 83)	81 (74, 86)	95 (91, 98)	95 (90, 98)	82 (76, 87)
6 months	154	151	23	10	81 (74, 87)	87 (81, 92)	94 (89, 97)	94 (89, 97)	87 (81, 91)
8 months	153	145	19	10	82 (76, 88)	89 (83, 93)	94 (88, 97)	94 (89, 97)	88 (83, 93)
10 months	148	144	17	10	83 (77, 89)	90 (84, 94)	94 (88, 97)	94 (89, 97)	89 (84, 94)
12 months	147	140	15	9	85 (79, 90)	91 (85, 95)	94 (89, 97)	94 (89, 97)	90 (85, 94)
Non-Small Cell Lung (2005)
2 months	61	149	60	5	50 (39, 60)	50 (41, 60)	97 (93, 99)	92 (83, 97)	71 (65, 77)
4 months	95	111	17	6	80 (72, 88)	85 (77, 91)	95 (89, 98)	94 (88, 98)	87 (80, 92)
6 months	89	95	8	5	87 (80, 94)	92 (84, 96)	95 (89, 98)	95 (88, 98)	92 (85, 97)
8 months	77	87	8	5	85 (78, 93)	91 (82, 96)	95 (88, 98)	94 (86, 98)	92 (84, 96)
10 months	70	76	6	4	87 (79, 95)	92 (84, 97)	95 (88, 99)	95 (87, 99)	93 (85, 97)
12 months	64	72	5	6	85 (76, 94)	93 (84, 98)	92 (84, 97)	91 (82, 97)	94 (85, 98)
Ovary (2002)*
2 months	96	45	48	3	46 (33, 59)	67 (58, 74)	94 (83, 99)	97 (91, 99)	48 (38, 59)
4 months	129	36	9	6	77 (66, 88)	93 (88, 97)	86 (71, 95)	96 (91, 98)	80 (65, 90)
6 months	125	32	5	5	83 (72, 93)	96 (91, 99)	86 (71, 95)	96 (91, 99)	86 (71, 95)
8 months	119	26	6	6	76 (64, 89)	95 (90, 98)	81 (64, 93)	95 (90, 98)	81 (64, 93)
10 months	112	25	5	6	77 (64, 90)	96 (90, 99)	81 (63, 93)	95 (89, 98)	83 (65, 94)
12 months	109	21	3	6	78 (65, 92)	97 (92, 99)	78 (58, 91)	95 (89, 98)	88 (68, 97)

POC = Patterns of Care, Med=Medicare, Se = Sensitivity, Sp = Specificity, PPV = Positive predictive value, NPV = Negative predictive value

^*Three ovarian cancer patients did not report any chemotherapy treatment data in POC and were removed from analysis.

^†Exact binomial 95% confidence intervals are rounded to the nearest digit. Therefore, none of the upper limits is exactly 100%.

Table 3 describes the measures of agreement and validity for the Medicare claims definitions used to identify the receipt of specific chemotherapeutic agents during the 6-month post-diagnosis period. For all intravenous agents administered to patients diagnosed with CRC and NSCLC, the measures of concordance and validity were high: Kappa ranged from 71% – 95%; Se ranged from 75% – 95%; Sp ranged from 90% – 99%; PPV ranged from 85% – 99%; and NPV ranged from 81% – 97%. Consistently, these measures (Kappa, Se, and PPV) were lowest for oxaliplatin. The measures of agreement and validity for identifying capecitabine, an orally administered agent equivalent to the intravenously administered 5-fluorouracil (5-FU) for CRC, in Medicare claims was poor with Kappa and Se of only 55% and 47%, respectively.

Table 3.

Comparison of specific chemotherapeutic agents identified by SEER POC data and Medicare claims during the 6-month post-diagnosis period for selected cancer sites and years^*

	Source reporting receipt of specific agent
	POC=Yes, Med=Yes	POC=No, Med=No	POC=Yes, Med=No	POC=No, Med=Yes	Kappa (%) (95% CI)	Se (%) (95% CI)	Sp (%) (95% CI)†	PPV (%) (95% CI)†	NPV (%) (95% CI)
Breast (2000 and 2005)
Cyclophosphamide	39	249	13	4	83 (73, 92)	75 (61, 86)	98 (96, 100)	91 (78, 97)	95 (92, 97)
Doxorubicin	27	266	10	3	78 (67, 90)	73 (56, 86)	99 (97, 100)	90 (73, 98)	96 (93, 98)
Colorectal (2000)
5-Fluorouracil (5-FU)	87	62	15	5	76 (66, 86)	85 (77, 92)	93 (83, 98)	95 (88, 98)	81 (70, 89)
Colorectal (2005)
5-Fluorouracil (5-FU)	114	192	14	11	83 (77, 89)	89 (82, 94)	95 (91, 97)	91 (85, 96)	93 (89, 96)
Capecitabine	22	279	25	5	55 (39, 70)	47 (32, 62)	98 (96, 99)	81 (62, 94)	92 (88, 95)
Oxaliplatin	51	254	17	9	73 (63, 82)	75 (63, 85)	97 (94, 98)	85 (73, 93)	94 (90, 96)
Non-Small Cell Lung (2005)
Carboplatin	77	112	4	1	95 (90, 99)	95 (88, 99)	99 (95, 100)	99 (93, 100)	97 (91, 99)
Paclitaxel	61	123	7	2	90 (83, 96)	90 (80, 96)	98 (94, 100)	97 (89, 100)	95 (89, 98)
Ovary (2002)
Carboplatin	110	35	11	10	68 (56, 81)	91 (84, 95)	78 (63, 89)	92 (85, 96)	76 (61, 87)
Paclitaxel	100	39	13	14	62 (49, 75)	88 (81, 94)	74 (60, 85)	88 (80, 93)	75 (61, 86)

POC = Patterns of Care, Med=Medicare, Se = Sensitivity, Sp = Specificity, PPV = Positive predictive value, NPV = Negative predictive value

^*Individuals lacking treatment data for the specific agent of interest were excluded from analysis.

^†Exact binomial 95% confidence intervals are rounded to the nearest digit. Therefore, none of the upper limits is exactly 100%.

For breast cancer, the Se estimates for cyclophosphamide and doxorubicin were lower than other cancer site-agents at 75% and 73%, respectively; however, the 95% confidence intervals included values consistent with other sites. For ovarian cancer, the Sp estimates for carboplatin and paclitaxel were low at 78% and 74%, respectively. The Sp estimates for the specific ovarian cancer agents were lower than agents used to treat other cancer sites across all post-diagnosis periods (data not shown). Evidence of variation was seen when comparing the above measures for the same agents across different cancer sites. The Kappa, Se, and Sp for the receipt of paclitaxel and carboplatin were higher among patient treated for NSCLC as compared to those treated for ovarian cancer.

Figure 2 illustrates how the use of multiple post-diagnosis periods changes the Se and Sp estimates for specific chemotherapeutic agents used to treat individuals diagnosed with stage II and III CRC in 2005. Generally, the Se for specific treatments reach their maximum close to the 8-month post-diagnosis period, with the exception of oxaliplatin for which Se continues to climb up to the 12-month post-diagnosis period. The Se of capecitabine is approximately 50% lower than the Se for all other CRC agents and remains steady over time. The Sp of Medicare claims for identifying patients who did not receive specific CRC chemotherapy agents was > 93% for all post-diagnosis periods.

Figure 2.

Sensitivity and specificity of Medicare claims for identifying the receipt of specific agents by post-diagnosis period, Colorectal cancer, 2005.

We conducted a sensitivity analysis to assess the impact that potential misclassification of the gold standard (i.e., the POC studies) could have on our results, using the specific example of oxaliplatin treatment for CRC patients in 2005. We identified 10 individuals diagnosed with CRC in 2005 who had 2 or more claims for oxaliplatin during the 12-months post-diagnosis, but were not identified by POC as having received oxaliplatin as part of the initial course of treatment. Because physicians would not likely submit claims to Medicare for administering oxaliplatin (an expensive treatment) unless it was actually delivered, we assumed that these patients were misclassified by the POC studies. We varied the percentage of oxaliplatin-treated patients that were missed by the 2005 CRC POC study from 0% to 60% (or 0 to 6 individuals) and assessed the changes in Se, Sp, and PPV. Over the range of values, the PPV increased the most from 84% to 94%, while the Se and Sp remained nearly constant, increasing only from 89% to 90% and 96% to 98%, respectively (data not shown).

Discussion

We found that utilizing 6, 8, 10, or 12 months of Medicare claims following a primary diagnosis of in situ or early stage breast, stage II or III colorectal, non-small cell lung, or ovarian cancer can accurately identify whether an individual received any chemotherapy as part of their initial course of treatment. However, the ability of Medicare claims to identify the receipt of specific chemotherapeutic agents appeared to vary by the agent, cancer site, and mode of administration. Medicare claims used to identify intravenously administered agents for CRC and NSCLC generally had a high Se, Sp, PPV, and NPV; although the Se tended to increase using longer post-diagnosis periods for more recently approved agents (i.e., oxaliplatin). The Se and Sp estimates for identifying any chemotherapy treatment among individuals diagnosed with breast and ovarian cancers were generally lower than those for CRC and NSCLC. Across cancer sites, Medicare claims performed best when identifying specific agents used to treat NSCLC (i.e., carboplatin and paclitaxel) with all measures of agreement and validity exceeding 90%.

Our findings update a prior study by Warren et al(32) utilizing POC data (1991, 1995, and 1996) to assess the utility of Medicare claims data for identifying the receipt of chemotherapy among individuals diagnosed with in situ or early stage breast, stage II or III CRC, and ovarian cancer. We found remarkably similar Kappa and Se estimates for identifying the receipt of any chemotherapy across cancer sites, with all confidence intervals encompassing the prior study estimates. However, our Kappa and Se estimates of Medicare claims for identifying specific chemotherapeutic agents are higher than those reported by Warren and colleagues. For example, in our study the Se of claims to identify the receipt of cyclophosphamide for the treatment of ovarian cancer was 75% (Table 3) compared with only 47% in the earlier study. It is possible that coding and reporting behavior improved over time, especially with the rising cost of chemotherapy.(42) These updated measures further confirm the utility of Medicare claims to identify these agents and provide the relevant information that may be used to correct for misclassification.

Our study extended the Warren study by examining the chemotherapeutic agents that were not included in the original study, such as doxorubicin for breast cancer, oxaliplatin and capecitabine for CRC, and paclitaxel for breast and NSCLC. Another study examined the validity of Medicare claims for identifying specific agents in comparison to two different clinical trials among breast (1995–1997) and lung (1998–2000) cancer patients. The study reported the Se and Sp for doxorubicin as 91% (95% CI: 79%, 98%) and 100%, and for paclitaxel as 86% (79%, 92%) and 100%, consistent with our findings.(29)

This is the first study to examine the validity of Medicare claims to identify oxaliplatin for individuals diagnosed with stage II and III CRC. The Se of Medicare claims to identify oxaliplatin increases with the length of the claims window post-diagnosis. A temporary HCPCS code was available for oxaliplatin (C9205) in 2005, while starting January 1, 2006, a permanent HCPCS code (J9263) was established. It is possible that physician coding improved after the permanent code was available, leading to better capture of oxaliplatin in later post-diagnosis periods.

There have been no prior validation studies examining the reporting of capecitabine in the Medicare data. We observed consistently low Se estimates for capecitabine in the Medicare claims for all post-diagnosis periods. One possible explanation for its poor Se is that patients who cannot afford their copayments received the drug through pharmacy assistance programs sponsored by the pharmaceutical company. It may also be that patients had prescription drug insurance that covered oral medications and the patient or the provider did not submit a claim for capecitabine to Medicare. Capecitabine is covered under Medicare Part B, as it is an oral alternative to an intravenous medication (5-FU). Chemotherapeutic agents that are only in oral form would be covered under Medicare’s Part D prescription drug coverage, which was implemented in 2006. Using Part D data to identify use of oral chemotherapies is limited as only 52% of Medicare beneficiaries have Part D enrollment.(43) Our findings, taken together with limited Part D enrollment among Medicare beneficiaries, suggest that the reporting of oral chemotherapeutic agents in the Medicare data may be incomplete. However, additional validation of oral chemotherapeutic agents in the Medicare data is needed. Two possible approaches to further explore the frequency of capecitabine claims in the outpatient drug setting would be to link: 1) Medicare dually-eligible individuals to their Medicaid prescription drug claims or 2) poor, elderly individuals that meet state pharmacy assistance program thresholds to their outpatient drug claims. These two groups are particularly unique and therefore results from these analyses may not be generalizable to the larger Medicare population.

This study has a number of strengths. Through cooperation with the NCI and SEER registries, we linked verified treatment data obtained through physician confirmation or unified medical record review to Medicare claims for a large number of individuals aged ≥65 years and diagnosed with one of four different cancers. The detailed POC data collection allowed us to assess the validity of Medicare claims to identify specific agents that have not previously been validated. We examined and reported variation in measures of validity across different post-diagnosis periods, whereas prior studies primarily used one or two broad post-diagnosis time windows.(28, 31, 32)

Our study is not without limitations. There may be patients in the study who received treatment from another healthcare payer (e.g., the Veterans Health Administration). These claims would not be captured in this analysis. Therefore, our results may be viewed as minimum thresholds which could be improved by combining information from other payers. Furthermore, approximately 26% of individuals in the POC studies lacked physician confirmation or unified medical record review and were therefore excluded from analysis. We also excluded individuals who had any HMO enrollment during the post-diagnosis periods, as detailed claims data were not reported to Medicare for these individuals. These exclusions along with our focus on individuals 65+ years limit the overall generalizability of our findings. This analysis examined the receipt of chemotherapy as part of the initial course of treatment, but did not distinguish between adjuvant and neoadjuvant treatment; we would not expect results to differ based on the receipt of therapy before or after surgery, however. Similarly, we cannot be sure that claims appearing later in the post-diagnosis period still relate to the initial course of treatment, or whether they are actually linked to treatment of recurrent or progressive cancer.

In conclusion, we assessed the utility of Medicare claims to identify the receipt of any chemotherapy and specific agents. Generally, Medicare claims can accurately identify the receipt of any chemotherapy and most specific agents administered intravenously. Medicare claims in combination with clinical data from cancer registries may be a valuable resource for health services research focused on evaluating treatment-related issues. Additionally, these results may be useful to assess the potential impact of treatment misclassification in future studies.