Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2008 Oct 14.
Published in final edited form as: Med Care. 2006 Feb;44(2):124–131. doi: 10.1097/01.mlr.0000196978.34283.a6

External Validation of Medicare Claims for Breast Cancer Chemotherapy Compared With Medical Chart Reviews

Xianglin L Du *, Charles R Key †, Lois Dickie †, Ronald Darling †, Jane M Geraci ‡, Dong Zhang §
PMCID: PMC2567101  NIHMSID: NIHMS71348  PMID: 16434911

Abstract

Background

Although Medicare claims data have been increasingly used to examine the patterns and outcomes of cancer chemotherapy, their external validity has not been well studied.

Objectives

We sought to validate Medicare claims for chemotherapy compared with medical chart reviews.

Patients and Methods

We completed medical chart reviews for 1228 women who were diagnosed with breast cancer at age 65 and older between 1993 and 1999 in New Mexico that were linked with Medicare claims data, achieving an estimated sensitivity of more than 90% and a 0.05 level of precision.

Results

Of the 150 subjects identified by Medicare claims as receiving chemotherapy within 6 months of diagnosis, 75% were confirmed by medical records as having received chemotherapy. Of the remaining 25% of cases without chart verification, (1) 33 cases had 7 or more claims for chemotherapy and also had specific chemotherapy drugs indicated in Medicare data, representing 22% (33/150) of all cases that received chemotherapy according to Medicare claims and (2) 4 cases had 1 to 6 claims for chemotherapy, representing 3% (4/150) of all cases with claims for chemotherapy. Of those 1078 subjects who did not receive chemotherapy according to Medicare claims, more than 99% were confirmed by chart reviews. Observed agreement on chemotherapy between Medicare claims and chart reviews was 94% and overall reliability (kappa) was 0.69 (95% confidence interval = 0.63–0.76).

Conclusions

Of cases identified as receiving chemotherapy by Medicare claims, 97% had strong evidence and only 3% had weak evidence for receiving this therapy.

Keywords: breast cancer, chemotherapy, Medicare claims, medical chart review, validity

Medicare claims data have been increasingly used in studies of health care outcomes during the past decade.1–16 More recently, Medicare claims data13–33 or tumor registry data34 have been used to identify patients with cancer who received chemotherapy. Because of concerns about the completeness of information on chemotherapy, the National Cancer Institute's Surveillance, Epidemiology and End Results (SEER) program does not release this information in the public-use dataset.4,16,17,35–37 Warren and colleagues recently reported the findings of a study on a random sample of cases from selected SEER tumor registries, known as the Patterns of Care (POC) study, which was initiated by the National Cancer Institute.17 Warren compared the chemotherapy information in Medicare claims to the POC study data on chemotherapy administration that was verified by the hospital medical records or by treating physicians. There was 98% agreement between Medicare claims and the POC study about whether subjects received chemotherapy for breast cancer, with a kappa of 0.82. Although the validity varied by type of cancer, overall agreement on the use of chemotherapy between these 2 databases was good for colon, rectal, and ovarian cancers. However, there has been no other study that externally validated chemotherapy information in Medicare claims. We undertook a study to validate Medicare claims for chemotherapy by comparing them to an independent medical chart review among patients diagnosed with breast cancer at age 65 or older in the state of New Mexico. This report presents the primary findings of this external validation study.

Patients and Methods

The Institutional Review Boards of the University of Texas Medical Branch and the University of New Mexico, and the Committee for the Protection of Human Subjects at the University of Texas Health Science Center at Houston approved this study.

Study Population and Sample Size Required

The primary goal of this current study was to compare an independent medical chart review on chemotherapy administration with Medicare claims for chemotherapy. Sample size considerations were therefore driven by the precision desired for estimating sensitivity. Sample size also is determined by the prevalence of chemotherapy use. We expected that the sensitivity of Medicare claims for chemotherapy was at least 90%. For this estimate of sensitivity, our study requires a total of 134 cases with chemotherapy to achieve a 0.05 level of precision.38 According to our previous reports,18,19 approximately 11% of women ages 65 years or older with breast cancer received chemotherapy. On the basis of this estimate, a total of 1218 patients with breast cancer were required to obtain 134 cases receiving chemotherapy.

Eligible cases should have both Medicare claims and medical chart review data. We aimed to include subjects with the most recent dates of cancer diagnosis because the New Mexico Tumor Registry investigators found it more difficult to locate the medical charts of patients with distant diagnosing dates. We first identified 1733 women who were diagnosed with stage I-IV breast cancer at age 65 or older between 1993 and 1999 in New Mexico. These included 2 groups of subjects: 905 cases from 1993 to 1996 that were from the available SEER-Medicare linked databases and 828 cases from 1997 to 1999 that were from the New Mexico Tumor Registry and were not linked to Medicare at the time of study. For cases diagnosed in 1997–1999, we obtained Medicare claims from the Center for Medicare and Medicaid Services (CMS). The linkage between the registry data and Medicare enrollment files was conducted by CMS with assistance from the New Mexico Tumor Registry. After the 2 databases were linked through social security numbers and gender, the CMS created and sent the merged data files to the registry. We excluded women who did not have full coverage of both Medicare Part A and Part B, or who were members of Health Maintenance Organizations in the year of diagnosis, because claims from these organizations may not be included in Medicare databases.

Of the 1733 potential cases, 1241 subjects had medical charts reviewed. Of these, 13 had incomplete abstraction forms, leaving 1228 cases for the final analysis. We compared the 1228 cases that had charts reviewed to the 492 eligible cases that did not have charts reviewed, and found no significant difference in the distribution of age between the 2 groups. However, those with chart reviews were more likely to have been diagnosed in 1997–1999 than those without (58% versus 22%).

Medical Chart Reviews and Medicare Claims for Chemotherapy

Medical Chart Reviews for Chemotherapy

The data collection form was initially pilot-tested on 20 cases. After a few minor modifications, the final abstract form contained 3 pages. The first page, which was kept at all times at the New Mexico Tumor Registry for confidentiality purposes, included the case identification (ID) number, name, social security number, name and address of the diagnosing facility and, if different, the names and addresses of facilities which provided surgery, radiation therapy, or chemotherapy. The second page contained the ID number, date of birth, date of breast cancer diagnosis, age at diagnosis, chemotherapy treatment (yes or no), type of chemotherapy, and date of therapy. Finally, the abstractor recorded where the chemotherapy information was obtained from, covering one or more chart data sources such as hospitals, oncologists' offices, radiology departments, or other physicians' offices. A subject was defined as having received chemotherapy according to chart reviews if any of these 4 chart sources (hospital, oncologist's office, radiology department, and other physician's office) indicated a receipt of chemotherapy. The third page was for information on hormone therapy for breast cancer. The data collection was conducted from March 2001 through February 2003.

The data abstractor sorted the 1733 eligible cases according to the hospitals where a breast cancer diagnosis was made. Although these cases were not selected randomly for medical chart review, the data abstractor worked through the list of cases according to the order of the identification number, and aimed to review medical charts for as many cases as possible during her visit to the hospital or to the oncologist's office across the state of New Mexico. If the medical records were not available during the visit, the data abstractor attempted to complete the abstraction form at the next visit, and so on until the desired sample size was reached.

After chart reviews were completed for 1228 cases as needed per our power analysis, our analysis indicated that a low percentage (25%) of the cases' records was reviewed from oncologists' offices. This was concerning because chemotherapy is administered most often in a medical oncologist's office. To address this concern, we conducted a second medical chart review for 150 subjects between August and October 2004, with a focus on contacting oncologists' and surgeons' offices. These cases were randomly selected from 2 groups of patients: those with Medicare claims for chemotherapy that lacked the first medical chart verification and those without Medicare claims for chemotherapy. The second medical chart abstractor was blinded to these cases' chemotherapy status according to Medicare claims or the first medical chart review. Of the 150 cases, 3 medical chart reviews were incomplete, leaving 147 cases for the final analysis for the second chart review.

Medicare Claims for Chemotherapy

The Medicare Program, administered by the CMS, covers medical services for more than 97% of persons aged 65 or older. A subject was defined as having received chemotherapy according to Medicare data if there was a claim for chemotherapy administration in any of 3 Medicare claims files (inpatient, outpatient, or physician claims). Detailed descriptions of the methods for defining chemotherapy in Medicare data are reported elsewhere.18,19 In brief, claim codes for chemotherapy included the ICD-9-CM procedure code of 99.25,39 the Common Procedure Terminology codes of 96400–96549, J9000–J9999, or Q0083–Q0085,40,41 revenue center codes of 0331, 0332, or 0335,42 and the ICD-9-CM V codes39 of V58.1, V66.2, or V67.2.

Analyses

The kappa statistic was calculated to quantify the degree of agreement (overall reliability) on the receipt of chemotherapy between the 2 databases, adjusting for chance agreement. The concordance or observed agreement rate (ie, number of cases agreed on the receipt of chemotherapy over the total number of cases) was also calculated. These analyses were repeated using different time periods (within 6 months and after 6 months of diagnosis). The analyses were also stratified by the source of medical charts for documenting chemotherapy, tumor stage, hormone therapy, and year of diagnosis. The Breslow-Day test was used for testing homogeneity across the strata. All analyses were completed using the SAS system.43

Results

A total of 1228 women diagnosed with breast cancer at age 65 years or older in 1993–1999 were available for the final analysis. At least 1 of the following 4 chart sources was checked: hospital (where a diagnosis usually was made), oncologist's office, radiology department, and other physician's office. Medical charts were reviewed for 100% of patients in hospitals. Of the 1228 cases, 25% also were reviewed in oncologists' offices, 20% in other physicians' offices, and 17% in radiology departments, whereas the corresponding percentages were 40%, 19%, and 16%, respectively, for those 108 cases that received chemotherapy according to chart reviews. Among 1228 subjects, 32% had records from at least 2 different sites, 13% from 3 sites, and 1% from all 4 sites, whereas 32%, 19%, and 1%, respectively, were for 108 cases with chemotherapy.

Table 1 presents comparisons between medical chart reviews and Medicare claims on the receipt of chemotherapy. Because chemotherapy was initiated at different time periods after diagnosis, we used the same time window of capturing treatment information (6 months) to make the data compatible between the 2 datasets. Of the 150 subjects identified by Medicare claims as receiving chemotherapy within 6 months of diagnosis, 90 (60%) cases were confirmed by medical records as having received chemotherapy, and 60 (40%) cases were not identified by chart reviews. Of those 1078 subjects who did not receive chemotherapy according to Medicare claims, more than 99% were confirmed by chart reviews. Observed agreement was 94% and overall reliability (kappa) was 0.69 (95% confidence interval of 0.63–0.76). Of the 90 subjects who received chemotherapy per both data sources, 41% agreed on the same day of therapy, 59% within 5 days, 69% within 10 days, and 86% within 30 days of chemotherapy.

Table 1.

Comparison Between Medical Chart Review and Medicare Claims on Chemotherapy That Was Administered Within 6 Months and After 6 Months of Diagnosis

Chemotherapy Categories (row %) [column %] Medicare Claims Observed Agreement Rate and Kappa (95% CI)
No Chemotherapy Chemotherapy Received Total
Chemotherapy within 6 months of diagnosis
 Chart review
  No chemotherapy 1069 (94.7) [99.2] 60 (5.3) [40.0] 1129 94.4%
  Chemotherapy received 9 (9.1) [0.8] 90 (90.9) [60.0] 99 0.69 (0.63–0.76)
  Total 1078 150 1228
Chemotherapy after 6 mo of diagnosis
 Chart review
  No chemotherapy 1193 (97.9) [99.3] 26 (2.1) [96.3] 1219 97.2%
  Chemotherapy received 8 (88.9) [0.7] 1 (11.1) [3.7] 9 0.05 (−0.06−0.15)
  Total 1201 27 1228
Open in a new tab

When analyzing those 27 cases receiving chemotherapy after 6 months of diagnosis in Medicare claims, only 1 case was identified by chart review as receiving this therapy after 6 months of diagnosis. The observed agreement rate was 97%, and kappa was low (Table 1). The test for homogeneity between these 2 time intervals was statistically significant (P < 0.01).

Table 2 presents comparisons between Medicare claims and the first medical chart review on the receipt of chemotherapy, stratified by the source of the medical charts, tumor stage, hormone therapy, and the year of diagnosis. The kappa statistic was higher in cases that were confirmed with oncologist's office records compared with those without (0.78 vs. 0.66). The test for homogeneity between these 2 groups was not significant (P = 0.624). The test for homogeneity also was not significant in subjects who were checked in radiation departments compared with those without, in cases that received hormone therapy versus those that did not, in subjects with stages I-II versus stage III-IV cancer, and in cases diagnosed in 1993–1996 versus 1997–1999. The agreement was lower in those who were checked in other physicians' offices compared with those without (P = 0.02).

Table 2.

Comparison Between Medical Chart Reviews and Medicare Claims on Chemotherapy

Agreement Between Medical Chart Reviews and Medicare Claims for Chemotherapy
Characteristics No. Patients Kappa (95% Confidence Interval) Observed Agreement (%) P Value for Homogeneity*
All patients 1228 0.69 (0.63–0.76) 94.4 —
Source of medical charts for documenting chemotherapy
 Oncologist's office
 Yes 308 0.78 (0.67–0.89) 95.1 0.624
 No 920 0.66 (0.57–0.74) 94.1
 Radiation department
 Yes 212 0.86 (0.74–0.98) 97.6 0.135
 No 1016 0.66 (0.59–0.74) 93.7
 Other physician's office
 Yes 251 0.52 (0.31–0.73) 93.6 0.020
 No 977 0.72 (0.65–0.79) 94.6
Tumor stage
 Early stage (I + II) 961 0.66 (0.57–0.76) 95.6 0.058
 Late stage (III + IV) 176 0.72 (0.61–0.83) 88.1
 Unstaged (unknown) 91 0.47 (0.11–0.83) 93.4
Hormone therapy
 Yes 421 0.80 (0.70–0.90) 96.4 0.479
 No 807 0.64 (0.56–0.73) 93.3
Year of diagnosis
 1993–1996 513 0.66 (0.52–0.80) 96.3 0.178
 1997–1999 715 0.70 (0.62–0.78) 93.0
Open in a new tab
*

P value of the Breslow-Day test for homogeneity across the strata.

Of the 147 cases in the second medical chart review, 78% were checked in oncologists' (48%) or surgeons' (30%) offices, and 95% were checked with these offices or cancer specialist treatment centers. The remaining 5% were obtained from the neighboring state cancer registries. Twenty-three (38.7%) of the 60 cases that had chemotherapy according to Medicare claims but not identified by the first chart review were confirmed by the second chart review as receiving chemotherapy. Of the 31 cases that had chemotherapy according to the second chart review, 77.4% were not identified by the first chart review.

We further examined the Medicare chemotherapy claim patterns for those 60 cases that had claims for chemotherapy but lacked medical record verification in the first chart review, compared with those 90 cases that were confirmed by medical records as receiving chemotherapy (Table 3). We calculated the number of claims for chemotherapy that were filed in Medicare inpatient, outpatient, or physician service claims. Of the 90 cases, none had 1 claim, 3% had 2 to 6 claims, and 97% had 7 or more claims for chemotherapy, which was significantly different from those 60 cases that did not have medical record confirmation (P = 0.03), among whom 5% had 1 claim, 7% had 2 to 6 claims, and 88% had 7 or more claims for chemotherapy (Table 3). This difference was attributable to the 3 cases without chart verification that had 1 claim only, because the distribution of the number of claims was not significantly different (P = 0.125) between the 2 groups when these 3 cases were excluded.

Table 3.

Chemotherapy Claim Patterns for Patients With Versus Without Medical Record Verification

Number and Percentage of Patients With Chemotherapy Identified by Medicare Claims, Stratified by Whether It Was Verified by Medical Records
Yes, Medical Records Also Indicated Chemotherapy No, Medical Records Indicated No Chemotherapy
No. Claims for Chemotherapy n Column % Row % n Column % Row %
1 claim 0 0 0 3 5.0 100.0
2–6 claims 3 3.3 42.9 4 6.7 57.1
7–12 claims 7 7.8 41.2 10 16.7 58.8
13–24 claims 28 31.1 75.7 9 15.0 24.3
25–48 claims 37 41.1 59.7 25 41.7 40.3
49 or more claims 15 16.7 62.5 9 15.0 37.5
Total 90 100.0 60.0 60 100.0 40.0
Open in a new tab

We also examined the distribution of other factors between these 2 groups of patients. There was no significant difference in the location of medical charts reviewed for chemotherapy in subjects with medical chart verification versus those without. There were also no significant differences in the distribution of age, year of diagnosis, and vital status between the 2 groups except for tumor stage (66% of those who did not have chart confirmation had early-stage cancer compared with 51% of those who had chart verification).

Table 4 presents the summary of the findings on the validity of Medicare claims for chemotherapy within 6 months of diagnosis compared with both chart reviews. More than 75% (113/150) of cases that received chemotherapy according to Medicare claims were confirmed by one of the chart reviews. Of the remaining 25% (37/150) of cases (1) 33 cases had 7 or more claims for chemotherapy and also had specific chemotherapy drugs indicated in Medicare data, representing 22% (33/150) of all cases that received chemotherapy according to Medicare claims and (2) 4 cases had 1 to 6 claims for chemotherapy, representing 3% (4/150) of all cases that received chemotherapy according to Medicare claims.

TABLE 4.

Summary of the Validity of Medicare Claims for Chemotherapy

No. Cases With Chemotherapy Indicated in the First Chart Review % of All Cases With Claims for Chemotherapy
Yes No
No. cases with claims for chemotherapy that were confirmed in medical records as receiving chemotherapy 90 23 75.3%
No. cases with 7 or more claims for chemotherapy that were not identified by medical records as receiving chemotherapy — 33 22.0%
No. cases with 2 to 6 claims for chemotherapy that were not identified by medical records as receiving chemotherapy — 2 1.3%
No. cases with 1 claim only for chemotherapy that were not identified by medical records as receiving chemotherapy — 2 1.3%
Total 90 60 100.0
Open in a new tab

Discussion

The primary question addressed by this study was whether Medicare claims data provide valid information on chemotherapy for patients known to have breast cancer. We examined this question by comparing Medicare claims to medical chart reviews. Among those cases identified as receiving no chemotherapy by Medicare claims, more than 99% were confirmed by medical records. Major concern, however, is that a substantial number of patients with Medicare claims for chemotherapy were not identified in medical records as having actually received it. There are 2 potential explanations for this. First, it is possible that the patients who did not have medical chart evidence of having received chemotherapy truly did not receive it, ie, their Medicare claims were erroneous or fraudulent. Second, the chart review process may have lacked accuracy or completeness. We will discuss these explanations further below.

We believe in the accuracy and completeness of Medicare claims for chemotherapy for the following reasons. First, chemotherapy is one of the few drug treatments covered by the Medicare program. Because chemotherapy drugs are expensive, patients would be highly likely to file a claim with Medicare if they are eligible to do so. Second, Medicare is a nationwide insurance program, covering medical services regardless of where patients seek their care in the country. In contrast, if patients received chemotherapy outside the state of New Mexico, their medical records would not be accessible for review within the scope of this study. Third, the nature of chemotherapy administration typically requires 4 to 12 cycles of combination chemotherapy, lasting for 3 to 8 months. Within each cycle, some drugs may need to be repeated within 8 days of the first dose. For example, the combination chemotherapy regimen CMF (ie, cyclophosphamide, methotrexate, and fluorouracil) commonly is used for women with breast cancer. Methotrexate and fluorouracil are given on day 1 for each cycle and are repeated on day 8 to complete 1 of the 6 cycles in total. Therefore, even though an occasional claim may be lost or not filed, it is highly unlikely for all these treatment records to be missing from Medicare claims. Moreover, because chemotherapy is primarily administered intravenously at outpatient clinics under the supervision of specialists (usually medical oncologists, rarely by other clinicians) and also medical providers are expected to file claims for reimbursement for both drug costs and drug administration fees, the increasing number of claims is clinically realistic and strong evidence for the occurrence of chemotherapy administration. Although we cannot rule out the possibility of fraudulent Medicare claims, the criminal and civil penalties resulting from charging Medicare for nonexistent services would serve as a strong incentive against this.

The second potential explanation is a deficiency in the chart review process. In this study, our initial chart review effort started at the hospitals where the diagnosis and initial treatments were given. If the diagnosing hospitals noted that patients were referred to medical oncologists, medical records were checked in those oncologists' offices. Several hospital cancer centers had a single chart for inpatient and outpatient treatments, under which circumstance the abstractors recorded this information as being obtained from the hospital instead of from the oncologist's office. Although all hospitals, cancer treatment centers, and all 7 free-standing oncology groups in New Mexico were visited, it is evident that not all patients' records were reviewed in their medical oncologists' offices. It would have been more advantageous to base the review on the oncologists' offices regardless of whether the hospital notes indicated that patients were referred to them, even though checking more than 1200 cases in every oncologist's office across the state of New Mexico would have been a huge task.

There were some variations in the agreement of information on chemotherapy between the 2 data sources. For example, the agreement was higher in cases that were confirmed with oncologists' office records compared with those other chart sources reviewed (Table 2). This is likely attributable to the fact that chemotherapy is primarily administered by medical oncologists in their offices. To address the concern about a low percentage of patients' charts reviewed in oncologists' offices, we performed a second medical chart review for a random sample of 147 cases. Of the 147 cases, 78% were reviewed in physicians' offices (48% in oncologists' offices and 30% in surgeons' offices), and 95% were checked with these offices or cancer specialist treatment centers. Of the 31 patients who had chemotherapy on this review, 77% were not so identified by the first chart review. This improved ability to detect chemotherapy in the medical records is likely due to more comprehensive reviews of medical records from their oncologist's office or a cancer treatment center where the patients actually received chemotherapy, compared with the first chart review.

However, we did not have information on how many patients were truly referred to medical oncologists before or after surgery in New Mexico, and there was little information on the referral to medical oncologists.44–46 It is often assumed that all patients with breast cancer be referred to medical oncologists, but that is not what has been observed in breast cancer practice.44–48 Two studies recently reported that just 19% to 29% of women ages 65 or older with breast cancer in SEER areas were referred to medical oncologists before surgery.47,48 Two earlier studies showed that 50% of women ages 65 or older with breast cancer did not have consultations with medical oncologists after surgery.44,45 Finally, in a recent analysis among Medicare beneficiaries, only 45% of patients with breast cancer were referred to medical oncologists (J. Warren, personal communication, 2004). These studies demonstrate that older women were substantially less likely to be referred to medical oncologists.44–48

Many older patients may be treated by primary care physicians or surgeons for hormone therapy instead of chemotherapy, and some patients may refuse to see oncologists.46 Furthermore, medical records themselves may have lacked the chemotherapy information, particularly in patients who received this therapy at out-of-state facilities, in which case claims for chemotherapy could still have been filed in Medicare. Previous studies showed that medical records often do not completely or accurately represent the care rendered to patients or certain aspects of their diseases, and some specific treatments may not have been well documented, especially in outpatient records.49 Our assessment that the chart review process may have been problematic for some cases in our study is supported by the fact that 77% of cases identified by the second chart review as receiving chemotherapy were missed by the first chart review. Thus, we conclude that our findings with respect to those cases without medical chart verification but with a large number of claims, in particular, are more likely due to the failure of finding confirmatory information in medical records, and less likely due to fraudulent Medicare claims.

Our study has several other limitations. First, this analysis only used Medicare claims for women identified from SEER or the New Mexico Tumor Registry as having been diagnosed with breast cancer. The validity of chemotherapy for nonregistry cases is unknown, particularly in women younger than the age of 65. Second, this study included only those women with breast cancer whose tumor registry data were linked with Medicare claims data, those with both Medicare Part A and Part B coverage, and those who were not enrolled in health maintenance organizations. It is unknown how the 2 data sources would agree on chemotherapy in excluded cases. Third, there were only a very small number of cases receiving chemotherapy more than 6 months after diagnosis, and only 1 case with Medicare claim for chemotherapy was confirmed by the chart review. This resulted in a low kappa statistic because of a sparse distribution of cases with chemotherapy, although the observed agreement was excellent (97%). This problem is a common occurrence when using kappa in this situation, which was well described as ‘mal-distribution’ by Feinstein.50 Therefore, our findings for those cases receiving chemotherapy within 6 months of diagnosis may not be generalizable to those receiving chemotherapy after 6 months of diagnosis. Finally, the findings in New Mexico may not be generalizable to other SEER registries. However, our findings were consistent with the only other external validation of Medicare claims for cancer chemotherapy in selected SEER areas,17 and were indirectly supported by the good validity of Medicare claims for surgery and radiotherapy.10,13,14,51

In summary, we conclude that (1) more than 75% of cases with Medicare claims for chemotherapy within 6 months of diagnosis have been externally validated as having received chemotherapy; (2) an additional 22% of cases with Medicare claims for chemotherapy had strong evidence for the receipt of chemotherapy; and (3) only 3% of cases with Medicare claims for chemotherapy had weak evidence for receiving this therapy (Table 4). Future validation studies may consider a number of strategies based on our results. First, reviewing all potential records in oncologists' offices will help increase the rate of medical record confirmation of the receipt of chemotherapy. This review will be essential to validate the number of chemotherapy cycles in particular, although it may also be labor-intensive. We also recommend conducting exploratory analyses that examine the impact of including versus excluding those 3% of cases with only 1 to 6 Medicare claims for chemotherapy. Because Medicare covers all service claims regardless of where the medical care was provided, the claims database can potentially capture patients with cancer chemotherapy that may be missed by other data sources (such as tumor registry and chart review) and therefore may provide a more complete picture of those receiving this treatment. However, because the chart review identified a few cases that received chemotherapy despite having no Medicare claims for this therapy, we recommend that the combined data from multiple data sources for cancer chemotherapy be used whenever available.

Acknowledgments

We thank Anna Davidson, James Goodwin, and Dawn Hamilton for help in initiating this study in New Mexico; Joan Warren for her helpful comments on an earlier draft of the manuscript; Charles Wiggins for the support of the second medical chart reviews; and faculty and staff from the New Mexico Tumor Registry and numerous medical facilities across the state of New Mexico for their cooperation in this study. We also thank Eric Boerwinkle, Craig Hanis, and Ann Coker for their helpful input.

Supported by a grant from the National Cancer Institute (R01-CA090626).

References

  • 1.Whittle J, Steinberg EP, Anderson GF, et al. Accuracy of Medicare claims data for estimation of cancer incidence and resection rates among elderly Americans. Med Care. 1991;29:1226–1236. doi: 10.1097/00005650-199112000-00005. [DOI] [PubMed] [Google Scholar]
  • 2.Fleming C, Fisher ES, Chang CH, et al. Studying outcomes and hospital utilization in the elderly. The advantages of a merged data base for Medicare and Veterans Affairs hospitals. Med Care. 1992;30:377–391. doi: 10.1097/00005650-199205000-00001. [DOI] [PubMed] [Google Scholar]
  • 3.Nattinger AB, Gottlieb MS, Veum J, et al. Geographic variation in the use of breast-conserving treatment for breast cancer. N Engl J Med. 1992;326:1102–1107. doi: 10.1056/NEJM199204233261702. [DOI] [PubMed] [Google Scholar]
  • 4.Potosky AL, Riley GF, Lubitz JD, et al. Potential for cancer related health services research using a linked Medicare-tumor registry database. Med Care. 1993;31:732–748. [PubMed] [Google Scholar]
  • 5.Lauderdale DS, Furner SE, Miles TP, et al. Epidemiologic uses of Medicare data. Epidemiol Rev. 1993;15:319–327. doi: 10.1093/oxfordjournals.epirev.a036123. [DOI] [PubMed] [Google Scholar]
  • 6.Mitchell JB, Bubolz T, Paul JE, et al. Using Medicare claims for outcomes research. Med Care. 1994;32(Suppl):JS38–JS51. [PubMed] [Google Scholar]
  • 7.McBean AM, Warren JL, Babish JD. Measuring the incidence of cancer in elderly Americans using Medicare claims data. Cancer. 1994;73:2417–2425. doi: 10.1002/1097-0142(19940501)73:9<2417::aid-cncr2820730927>3.0.co;2-l. [DOI] [PubMed] [Google Scholar]
  • 8.Ballard-Barbash R, Potosky AL, Harlan LC, et al. Factors associated with surgical and radiation therapy for early stage breast cancer in older women. J Natl Cancer Inst. 1996;88:716–726. doi: 10.1093/jnci/88.11.716. [DOI] [PubMed] [Google Scholar]
  • 9.McClish DK, Penberthy L, Whittemore M, et al. Ability of Medicare claims data and cancer registries to identify cancer cases and treatment. Am J Epidemiol. 1997;145:227–233. doi: 10.1093/oxfordjournals.aje.a009095. [DOI] [PubMed] [Google Scholar]
  • 10.Du XL, Freeman JL, Goodwin JS. Information on radiation treatment in patients with breast cancer: the advantages of the linked Medicare and SEER data. J Clin Epidemiol. 1999;52:463–470. doi: 10.1016/s0895-4356(99)00011-6. [DOI] [PubMed] [Google Scholar]
  • 11.Warren JL, Riley GF, Potosky AL, et al. Trends and outcomes of outpatient mastectomy in elderly women. J Natl Cancer Inst. 1998;90:833–840. doi: 10.1093/jnci/90.11.833. [DOI] [PubMed] [Google Scholar]
  • 12.Iezzoni LI. Data sources and implications: administrative databases. In: Iezzoni LI, editor. Risk Adjustment for Measuring Healthcare Outcomes. Chicago: Health Administration Press; 1997. pp. 169–242. [Google Scholar]
  • 13.Virnig BA, Warren JL, Cooper GS, et al. Studying radiation therapy using SEER-Medicare-linked data. Med Care. 2002;40(8 Suppl):49–54. doi: 10.1097/00005650-200208001-00007. [DOI] [PubMed] [Google Scholar]
  • 14.Cooper GS, Yuan Z, Stange KC, et al. Agreement of Medicare claims and tumor registry data for assessment of cancer-related treatment. Med Care. 2000;38:411–421. doi: 10.1097/00005650-200004000-00008. [DOI] [PubMed] [Google Scholar]
  • 15.Potosky AL, Warren JL, Riedel ER, et al. Measuring complications of cancer treatment using the SEER-Medicare data. Med Care. 2002;40(8 Suppl):62–68. doi: 10.1097/00005650-200208001-00009. [DOI] [PubMed] [Google Scholar]
  • 16.Warren JL, Klabunde CN, Schrag D, et al. Overview of the SEER-Medicare data: content, research applications, and generalizability to the United States elderly population. Med Care. 2002;40(8 Suppl):3–18. doi: 10.1097/01.MLR.0000020942.47004.03. [DOI] [PubMed] [Google Scholar]
  • 17.Warren JL, Harlan LC, Fahey A, et al. Utility of the SEER-Medicare data to identify chemotherapy use. Med Care. 2002;40(Suppl):IV-55–IV-61. doi: 10.1097/01.MLR.0000020944.17670.D7. [DOI] [PubMed] [Google Scholar]
  • 18.Du XL, Goodwin JS. Patterns of use of chemotherapy for breast cancer in older women: findings from Medicare claims data. J Clin Oncol. 2001;19:1455–1461. doi: 10.1200/JCO.2001.19.5.1455. [DOI] [PubMed] [Google Scholar]
  • 19.Du XL, Goodwin JS. Increase of chemotherapy use in older women with breast carcinoma from 1991 to 1996. Cancer. 2001;92:730–737. doi: 10.1002/1097-0142(20010815)92:4<730::aid-cncr1376>3.0.co;2-p. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Earle CC, Venditti LN, Neumann PJ, et al. Who gets chemotherapy for metastatic lung cancer? Chest. 2000;117:1239–1246. doi: 10.1378/chest.117.5.1239. [DOI] [PubMed] [Google Scholar]
  • 21.Earle CC, Tsai JS, Gelber RD, et al. Effectiveness of chemotherapy for advanced lung cancer in the elderly: instrumental variable and propensity analysis. J Clin Oncol. 2001;19:1064–1070. doi: 10.1200/JCO.2001.19.4.1064. [DOI] [PubMed] [Google Scholar]
  • 22.Schrag D, Cramer LD, Bach PB, et al. Age and adjuvant chemotherapy use after surgery for stage III colon cancer. J Natl Cancer Inst. 2001;93:850–857. doi: 10.1093/jnci/93.11.850. [DOI] [PubMed] [Google Scholar]
  • 23.Schrag D, Gelfand SE, Bach PB, et al. Who gets adjuvant treatment for stage II and III rectal cancer? Insight from Surveillance, Epidemiology, and End Results–Medicare. J Clin Oncol. 2001;19:3712–3718. doi: 10.1200/JCO.2001.19.17.3712. [DOI] [PubMed] [Google Scholar]
  • 24.Sundararajan V, Grann VR, Jacobson JS, et al. Variations in the use of adjuvant chemotherapy for node-positive colon cancer in the elderly: a population-based study. Cancer J. 2001;7:213–218. [PubMed] [Google Scholar]
  • 25.Lamont EB, Lauderdale DS, Schilsky RL, et al. Construct validity of Medicare chemotherapy claims: the case of 5FU. Med Care. 2002;40:201–211. doi: 10.1097/00005650-200203000-00004. [DOI] [PubMed] [Google Scholar]
  • 26.Schrag D, Rifas-Shiman S, Saltz L, et al. Adjuvant chemotherapy use for Medicare beneficiaries with stage II colon cancer. J Clin Oncol. 2002;20:3999–4005. doi: 10.1200/JCO.2002.11.084. [DOI] [PubMed] [Google Scholar]
  • 27.Earle CC, Neumann PJ, Gelber RD, et al. Impact of referral patterns on the use of chemotherapy for lung cancer. J Clin Oncol. 2002;20:1786–1792. doi: 10.1200/JCO.2002.07.142. [DOI] [PubMed] [Google Scholar]
  • 28.Sundararajan V, Hershman D, Grann VR, et al. Variations in the use of chemotherapy for elderly patients with advanced ovarian cancer: a population-based study. J Clin Oncol. 2002;20:173–178. doi: 10.1200/JCO.2002.20.1.173. [DOI] [PubMed] [Google Scholar]
  • 29.Sundararajan V, Mitra N, Jacobson JS, et al. Survival associated with 5-fluorouracil-based adjuvant chemotherapy among elderly patients with node-positive colon cancer. Ann Intern Med. 2002;136:349–357. doi: 10.7326/0003-4819-136-5-200203050-00007. [DOI] [PubMed] [Google Scholar]
  • 30.Neugut AI, Fleischauer AT, Sundararajan V, et al. Use of adjuvant chemotherapy and radiation therapy for rectal cancer among the elderly: a population-based study. J Clin Oncol. 2002;20:2643–2650. doi: 10.1200/JCO.2002.08.062. [DOI] [PubMed] [Google Scholar]
  • 31.Iwashyna TJ, Lamont EB. Effectiveness of adjuvant fluorouracil in clinical practice: a population-based cohort study of elderly patients with stage III colon cancer. J Clin Oncol. 2002;20:3992–3998. doi: 10.1200/JCO.2002.03.083. [DOI] [PubMed] [Google Scholar]
  • 32.Delea TE, Vera-Llonch M, Edelsberg JS, et al. The incidence and cost of hospitalization for 5-FU toxicity among Medicare beneficiaries with metastatic colorectal cancer. Value Health. 2002;5:35–43. doi: 10.1046/j.1524-4733.2002.51083.x. [DOI] [PubMed] [Google Scholar]
  • 33.Du XL, Osborne C, Goodwin JS. Population-based assessment of hospitalizations for toxicity from chemotherapy in older women with breast cancer. J Clin Oncol. 2002;20:4636–4642. doi: 10.1200/JCO.2002.05.088. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Du XL, Key CR, Osborne C, et al. Discrepancy between consensus recommendations and actual community use of adjuvant chemotherapy in women with breast cancer. Ann Intern Med. 2003;138:90–97. doi: 10.7326/0003-4819-138-2-200301210-00009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Ries LAG, Eisner MP, Kosary CL, et al., editors. SEER Cancer Statistics Review, 1975–2000. Bethesda, MD: National Cancer Institute; 2003. [November 23, 2005]. Available at: http://seer.cancer.gov/csr/1975_2000. [Google Scholar]
  • 36.National Cancer Institute. The SEER Program Code Manual. Revised. Bethesda, MD: National Cancer Institute; 1994. NIH Publication No. 94–1999. [Google Scholar]
  • 37.The New Mexico Tumor Registry. About the New Mexico Tumor Registry. [November 23, 2005]; Available at: http://hsc.unm.edu/epiccpro/nmtr.html.
  • 38.Hintze J. PASS, 2000. Kaysville, Utah: NCSS; 2000. [Google Scholar]
  • 39.U.S. Public Health Services. International Classification of Diseases, 9th Revision, Clinical Modification. 5th. Los Angeles: PMIC; 1996. [Google Scholar]
  • 40.American Medical Association. Physicians' Current Procedural Terminology–CPT 2000. Chicago: American Medical Association; 2000. [Google Scholar]
  • 41.Health Care Financing Administration. HCFA Common Procedure Coding System (HCPCS): National Level II Medicare Codes. Los Angeles: Practice Management Information Corporation; 2000. [Google Scholar]
  • 42.Health Care Financing Administration. HCFA Data Dictionary: Revenue Center Codes. Jun 17, 1999. [Google Scholar]
  • 43.Stokes ME, Davis CS, Koch GG. Categorical Data Analysis Using the SAS System. Cary, NC: SAS Institute Inc.; 1997. [Google Scholar]
  • 44.Newcomb PA, Carbone PP. Cancer treatment and age: patient perspectives. J Natl Cancer Inst. 1993;85:1580–1584. doi: 10.1093/jnci/85.19.1580. [DOI] [PubMed] [Google Scholar]
  • 45.Hillner BE, McDonald MK, Penberthy L, et al. Measuring standards of care for early breast cancer in an insured population. J Clin Oncol. 1997;15:1401–1408. doi: 10.1200/JCO.1997.15.4.1401. [DOI] [PubMed] [Google Scholar]
  • 46.Siminoff LA, Zhang A, Saunders Sturm CM, et al. Referral of breast cancer patients to medical oncologists after initial surgical management. Med Care. 2000;38:696–704. doi: 10.1097/00005650-200007000-00002. [DOI] [PubMed] [Google Scholar]
  • 47.Keating NL, Landrum MB, Ayanian JZ, et al. Consultation with a medical oncologist before surgery and type of surgery among elderly women with early-stage breast cancer. J Clin Oncol. 2003;21:4532–4539. doi: 10.1200/JCO.2003.05.131. [DOI] [PubMed] [Google Scholar]
  • 48.Baldwin LM, Taplin SH, Friedman H, et al. Access to multidisciplinary cancer care: is it linked to the use of breast-conserving surgery with radiation for early-stage breast carcinoma? Cancer. 2004;100:701–709. doi: 10.1002/cncr.20030. [DOI] [PubMed] [Google Scholar]
  • 49.Iezzoni LI. Data Sources and Implications: Information from Medical Records and Patients. In: Iezzoni LI, editor. Risk Adjustment for Measuring Healthcare Outcomes. Chicago, IL: Health Administration Press; 1997. pp. 243–278. [Google Scholar]
  • 50.Feinstein AR. Principles of Medical Statistics. Boca Raton, FL: Chapman & Hall/CRC; 2002. pp. 415–418. [Google Scholar]
  • 51.Du XL, Freeman JL, Warren JL, et al. Accuracy and completeness of Medicare claims data for surgical treatment of breast cancer. Med Care. 2000;38:719–727. doi: 10.1097/00005650-200007000-00004. [DOI] [PubMed] [Google Scholar]

RESOURCES