Challenges and Opportunities in Measuring Cancer Recurrence in the United States

Abstract

Cancer recurrence and disease-free survival are key outcomes for measuring the burden of illness, assessing the quality of cancer care, and informing decisions about increasingly costly cancer therapies. Yet information about recurrence is not collected in cancer registries or other population-based data sources. To address the lack of population-based recurrence information, researchers are increasingly using algorithms applied to health claims to infer recurrence. However, the validity of these approaches has not been comprehensively evaluated. In this commentary, we review existing studies and discuss options for improving the availability of recurrence data. We found that the validity of claims-based approaches appears promising in small, single institution studies, but larger population-based studies have identified substantial limitations with using claims to identify recurrence. With the increasing availability of health data, there are potential options that can be implemented to enhance information about recurrence. These options include design of software for the electronic medical record that enables rapid and standardized reporting of recurrence, use of electronic pathology reports to facilitate streamlined collection of recurrence by cancer registries, and mandates by insurers to require reporting of recurrence on health claims submitted by physicians. All of these options will require that governmental agencies, health insurers, professional societies, and other groups recognize the importance of population-based recurrence data and determine that this information is a priority for assessing cancer outcomes and costs.

Historically cancer mortality rates and overall survival rates following a cancer diagnosis have served as key population-based outcomes to assess the nation’s progress in cancer control and reduction of the burden of cancer. However, with increased cancer survivorship in the United States, these metrics of the national cancer burden are no longer sufficient. Many cancer deaths occur in patients who previously completed definitive treatment and were determined to be disease-free and then later experienced a cancer recurrence. To estimate the number of persons living with recurrent cancer as well as the benefit of treatment, the United States needs accurate population-level measurements of cancer recurrence and disease-free survival (DFS). Currently such information does not exist. Recurrence and DFS are routinely collected and reported in clinical trials, however only 3% of US adult cancer patients participate in trials (1) and those patients who do tend to be younger and healthier than cancer patients who do not participate or are not eligible for trials. These differences limit the generalizability of findings from clinical trials to the community setting, where most cancer patients are treated.

The risk of recurrence and expected DFS among cancer patients treated in the community are essential measures for clinicians recommending treatment and patients making informed decisions about different treatment options. The need for this information is increasingly important, as the costs associated with cancer treatment have escalated (2,3). The average monthly cost in the United States for a branded oncology drug is now approximately $10 000, double what was reported a decade ago (4). Many newer drugs exceed $100 000 for a course of therapy (5), and with standard copayments of 20%, cancer patients face large financial burdens from high-cost therapies. The survival benefit for patients who receive high-cost treatment for recurrence or metastatic disease may be limited and disproportionate relative to the treatment costs (6,7). It is crucial that physicians and patients have accurate population-based information about the frequency and timing of recurrence to assess the potential benefit of treatment relative to the often appreciable costs. In addition, there is frequent debate about the appropriate type and timing of postdiagnostic surveillance in clinical practice. This has led to the creation of practice guidelines that are often derived from expert opinion. Population-based data on recurrence could provide objective data to inform the development of guidelines for postdiagnostic surveillance.

Can Health Claims Be Used to Identify Recurrence?

Cancer registries do not currently collect cancer recurrence data; therefore, researchers have, in recent years, used as an alternative data source longitudinal patient-level health claims to infer which patients may have had a recurrence. Researchers have used two common approaches: 1) identifying patients who had a claim for metastatic disease based on International Classification of Disease (ICD)–9 codes for secondary neoplasms to lymph nodes or distant organs after initial treatment is completed and/or 2) identifying patients who had claims for additional cancer therapy at a later time after completion of initial treatment and following a treatment-free interval. The first approach requires that all providers completely and accurately record on their bills the code for the presence of metastatic disease, although they are not required to do so in order to obtain reimbursement. The latter approach’s utility is dependent on patients receiving treatment in the event they have a recurrence. However, not all patients are offered treatment following a recurrence and some patients who are offered treatment may decline it. Lack of treatment following a recurrence may be more common in the elderly and in patients with other comorbid conditions. This is a substantial concern because cancer is a disease that occurs more frequently in the elderly, and population-based measures that employ this approach may underestimate recurrences and incorrectly classify DFS.

Several small studies have been conducted in selected populations to validate approaches for identifying recurrence from claims, comparing codes on billing records and claims to the contents of the medical record (8–12). As can be seen on Table 1, findings from these studies suggest that billing records and claims have high sensitivity, specificity, and positive predictive value to identify recurrence and metastasis. These results suggest that the claims can provide accurate and complete data about recurrence. However, these studies, which relied on costly medical record abstraction, are limited in that they are based on a very small numbers of cancer patients, with sample sizes ranging from 45 to 292 patients. Recurrences were relatively infrequent in these studies, ranging from 12 to 61, respectively. In addition, these studies were conducted using data from academic medical centers, primarily single institutions, where coding may be more complete and accurate than coding practices in community settings. Further, these small studies focused on identifying recurrences in single cancer sites. This raises concerns as to whether the findings from one cancer site can be generalized to another cancer site, which may vary substantively in risk of recurrence and treatment patterns following a recurrence.

Table 1.

Studies of health claims–based methods to identify metastasis and/or recurrence using small selected samples

Author	Study aim: validation of algorithm to identify recurrence in health claims by using:	Data sources/setting	Population	Study period	Sample size/# of patients with recurrence/metastasis	Method to validate metastasis or recurrence	Findings (best algorithm)
Despande et al. (8)	ICD-9 and HCPCS codes on hospital billing data to identify local and distant metastasis for CRC patients	Barnes-Jewish Hospital Oncology Data Services inpatient/outpatient hospital billing data linked to medical record data	Consecutive cohort of patients age 65+ y with stage I-III first CRC undergoing surgery with curative intent	Cancer diagnosis: 2005–2009 Follow-up for recurrence: 2005–2010	174/32	Recurrence defined as code for metastasis 3 mo after initial surgery or code for chemotherapy, radiation, or surgery at least 8 mo after initial treatment compared with medical record and tumor data*	Sensitivity-81% Specificity-99% PPV-96% NPV-96%
Dolan et al. (9)	ICD-9 codes on claims to identify prostate cancer metastases	Robert Wood Johnson Hospital Inpatient billing records linked to medical records	Consecutive cohort of men of all ages hospitalized with prostate cancer	Cancer diagnosis: 1986–2007 Follow-up for metastasis: 1986–2010	292/61	Bone, lung, liver, and brain metastasis defined from claims based on codes compared with the medical record*	Sensitivity-95% Specificity-100% PPV-100% NPV-99%
Earle et al. (10)	ICD-9, CPT-4, HCPCS, and DRG codes on bills to identify relapse of acute myelogenous leukemia	Brigham and Women’s Hospital cancer registry data linked to institutional billing and medical records	Consecutive cohort of treated AML patients of all ages	Cancer diagnosis: 1991–1994 Follow-up for relapse: 1991–1996	89/22	Relapse defined as claim for chemotherapy after a break in treatment of ≥4 mo compared with medical record*	Sensitivity-86% Specificity-99% PPV-95% NPV-96%
Eichler et al. (11)	ICD-9 diagnosis codes on bills for brain metastasis	Massachusetts General Hospital cancer registry data linked to institutional billing records and medical records	Consecutive cohort of NSCLC patients of all ages	Cancer diagnosis: 2004-2004 Follow-up for metastasis: 2004–2008	241/59	Brain metastasis defined as bills with ICD-9 code for brain metastasis compared with medical record*	Sensitivity-97% Specificity-98% PPV-98%
Lamont et al. (12)	ICD-9 diagnosis codes for secondary neoplasm on claims to identify disease-free survival in breast cancer patients	Cancer and Leukemia Group B clinical trial data linked to Medicare claims	Medicare breast cancer patients aged 65+ participating in CALGB trial	Cancer diagnosis: 1995–1997 Follow-up for metastasis: 1995–2000	45/12	DFS defined from claims compared with reporting of DFS from the CALGB data*	2-year DFS Sensitivity-83% Specificity-95% 5-year DFS Sensitivity-100% Specificity-97%

* Gold standard for metastasis or recurrence was standard medical record or clinical trial data. AML = acute myelogenous leukemia; CALGB = Cancer and Leukemia Group B; CRC = colorectal cancer; DFS = disease-free survival; DRG = diagnostic-related group; HCPCS = Healthcare Common Procedure Coding System; ICD = International Classification of Disease; NSCLC = non–small cell lung cancer; NPV = negative predictive value; PPV = positive predictive value.

A growing number of studies have used population-based data to assess the completeness and accuracy of using health claims to infer recurrence and metastasis in large numbers of cancer patients as shown in Table 2, (13–19). These studies primarily used health claims obtained from large health insurers that include a diversity of clinics and hospitals from various geographic locations. The sample sizes of these population-based studies are large, with up to 85 132 patients per cancer site and a substantial number of recurrent or metastatic events. Two of these studies (13,15) assessed the validity of the diagnosis codes for metastasis at the time of diagnosis compared with stage from registry data. Because diagnosis codes for metastasis are often used to impute disease progression following initial diagnosis, these two studies are included in the inventory of population-based validations. The results from these large population-based studies present a markedly different picture about the utility of health claims to infer cancer recurrence and metastasis than what was determined from the small validation studies. In all of the population-based studies, the sensitivity, specificity, and positive predictive value of algorithms to identify recurrence or metastasis never simultaneously exceeded 80%. The findings from the population-based studies clearly demonstrate that using claims alone to identify cancer recurrence or metastasis will misclassify a sizeable number of patients and lead to a biased assessment of outcomes.

Table 2.

Studies of health claims–based methods for identify metastasis and/or recurrence in population-based samples*

Author	Study aim: validation of algorithm to identify metastases or recurrence in health claims by using:	Data sources/setting	Population	Study period	Sample size/# of patients with recurrence/metastasis	Method to validate metastasis or recurrence	Findings (best algorithm)
Chawla et al. (13)	ICD-9 diagnosis codes to identify presence and location of metastatic disease at the time of diagnosis for patients with breast, colorectal, or lung cancer diagnosis	Medicare inpatient, hospital outpatient, and physician claims linked to SEER cancer registry data	Medicare patients age 65+ y	Registry cancer diagnosis: 2005–2007	# cases/# distant metastasis BC-27 143/1642 CRC-24 216/3790 LC-28 693/13 594	Metastasis identified from diagnosis codes and stage inferred as local, regional, or distant based on metastasis location (or absence of codes); compared with registry data, used as gold standard	Distant metastasisBC: Sensitivity-51%   Specificity-99%   PPV-66%   NPV-97% CRC: Sensitivity-73%    Specificity- 94%    PPV-69%    NPV-95% LC: Sensitivity-43%   Specificity-95%   PPV-88%   NPV-65%
Chubak et al. (14)	ICD-9 diagnosis and procedure codes, HCPCS codes, and pharmacy claims to identify second breast cancer (BC) events, including recurrence	Health and pharmacy claims from an integrated health care system linked to medical records and SEER cancer registry	Women age 18+ y with stage I and II BC who participated in prior cohort studies	Cancer diagnosis: 1993–2006 Evaluation of second events after surgery	# of breast cancer cases-3152 # recurrences-299 # new primaries-93	Algorithms with and without SEER variables to identify second events and recurrences; medical record used as gold standard; analysis produced several receiver operator curves to identify high sensitivity or specificity	ROC for recurrence: High sensitivity   Sensitivity-94%   Specificity-92%   PPV-58%   NPV-99% High specificity & PPV   Sensitivity-72%   Specificity-97%   PPV-75%   NPV-97%
Cooper et al. (15)	ICD-9 diagnosis codes to identify presence and location of metastatic disease at the time of diagnosis for patients with breast, colorectal, endometrial, lung, pancreatic, or prostate cancer diagnosis	Medicare inpatient and hospital outpatient claims linked to SEER cancer registry data	Medicare patients age 65+ y	Registry cancer diagnosis: 1984–1993	# cases/# distant metastasis BC-60 445/3360 CRC-75 576/13 165 EN-14 157/1364 LC-71 468/28 255 PC-13 859/5857 PR-85 132/10 153	Metastasis identified from diagnosis codes within 3 mo +/- of diagnosis and stage inferred as local, regional, or distant based on metastasis location (or absence of codes); compared with registry data, used as gold standard	Performance of claims to identify distant metastasis: BC: Sensitivity-60%   PPV-58% CRC: Sensitivity-79%    PPV-63% EN: Sensitivity-53%   PPV-65% LC: Sensitivity-58%   PPV-79% PC: Sensitivity-79%   PPV-74% PR: Sensitivity-58%   PPV-65%
Hassett et al. (16)	ICD-9 diagnosis and procedure codes, HCPCS codes, DRG, and revenue center codes for secondary malignant neoplasm and chemotherapy codes for patients with breast, lung, colorectal, and prostate cancer	Cancer Care Outcomes Research and Surveillance medical record data linked to Medicare claims and HMO/Cancer Research Network data	CanCORs: Medicare patients with stage I-III lung and colorectal cancer who had definitive local therapy for stage I-III lung and colorectal cancer HMO/CRN: Age 21 y and older with stage I-IIIA cancer treated with definitive therapy	CanCORS cases diagnosed 2003–2005 and followed for 14 mo HMO/CRN cases diagnosed 2000–2005 and followed for 60 mo	# cases/# recurrence CanCORs:   LC-309/59   CRC-620/56 HMO/CRN:   BC-2726/212   CRC-1088/191   LC-333/129   PR-1151/89	Algorithm using ICD-9 diagnosis codes for secondary malignancy and chemotherapy codes to identify recurrence at 14 and 60 mo from claims or encounter data when compared with medical record data, used as gold standard	CanCORs 14-mo data: LC: Sensitivity-77%   Specificity-70% CRC: Sensitivity-81%    Specificity- 83% CRN 60-mo data: BC: Sensitivity-78%   Specificity-79%   PPV-30% CRC: Sensitivity-83%    Specificity-79%    PPV-53% LC: Sensitivity-85%   Specificity-72%   PPV-72% PR: Sensitivity-19%   Specificity- 83%   PPV-11%
Nordstrom et al. (17)	ICD-9 diagnosis and treatment codes, HCPCS codes, and NDC codes to identify presence of metastatic disease at the time of diagnosis for patients with breast, colorectal, lung, and prostate cancer	Health and pharmacy claims from a national insurer linked to electronic medical record data from oncologists	Patients of all ages with commercial (non-Medicare) insurance	Cancer diagnosed 2004–2010	BC- 1385/175 CRC-727/215 LC- 1036/477 PR-267/176	Metastases identified from cancer diagnoses and procedure or pharmacy claims for cancer treatment compared with EMR, used as gold standard for incidence and stage	BC: Sensitivity-62%   Specificity-97%   PPV-75%   NPV-95% CRC: Sensitivity-67%    Specificity- 93%    PPV-80%    NPV-87% LC: Sensitivity-60%   Specificity-88%   PPV-81%   NPV-72% PR: Sensitivity-81%   Specificity-75%   PPV-86%   NPV-67%
Warren et al. (18)	ICD-9 diagnosis and procedure codes, HCPCS codes, NDC codes to assess the sensitivity of Medicare claims to identify cancer recurrence for patients with breast and colorectal cancer	Medicare inpatient, hospital outpatient, physician, durable medical equipment, and hospice claims linked to cancer incidence in SEER cancer registry data	Patients ages 65+ y diagnosed with stage II or III BC or CRC who received definitive treatment, had a treatment- free interval and later died from cancer	Cancer diagnoses 1994–2003 Evaluation of recurrence: 1994-until death (2008 at latest)	BC-3826 CRC-6910	All patients were assumed to have recurred because they died from cancer; following definitive treatment and 3-mo treatment-free interval, claims were reviewed until death for additional cancer therapy (surgery, chemotherapy, radiation) or hospice admission	Additional therapy as first indicator of recurrence: BC-39% CRC-35% No indicator of recurrence or hospice only indicator of recurrence BC-19% CRC-25%
Whyte et al. (19)	ICD-9 and HCPCS codes on health claims to identify patients with metastatic breast, colorectal, or lung cancer	Inpatient, outpatient, emergency room, physician and surgery center claims from a national insurer linked to cancer reported in clinical oncology data	Patients of all ages with commercial (non-Medicare) insurance	Claims and clinical oncology data from 2007–2010; claims were reviewed for up to 1 y prior to and 3 mo following a cancer diagnosis in the clinical oncology database	BC-4631/371 CRC- 2058/528 LC- 2449/1204	General and cancer site-specific algorithms were evaluated using ICD-9 diagnosis codes for secondary neoplasms; algorithms varied by frequency/timing of codes and specific metastatic sites; compared with clinical oncology data as gold standard	Best algorithm: BC: Sensitivity-66%   Specificity-97%   PPV-75%   NPV-96% CRC: Sensitivity-63%    Specificity-88%    PPV-71%    NPV-83% LC: Sensitivity-61%   Specificity-81%   PPV-77%   NPV-68%

* BC = breast cancer; CanCORs = Cancer Care Outcomes Research and Surveillance; CRC = colorectal cancer; DFS = disease-free survival; DRG = diagnostic-related group; EN = endometrial cancer; HCPCS = Healthcare Common Procedure Coding System; ICD = International Classification of Disease; LC = lung cancer; NDC = national drug code; NPV = negative predictive value; PC = pancreatic cancer; PR = prostate cancer; PPV = positive predictive value; ROC = receiver operating characteristic curve; SEER = Surveillance, Epidemiology, and End Results.

Several of the studies have developed algorithms for identifying recurrence that can be used to create receiver operating characteristic curves (ROCs), which have potential utility for analyses where the goal is to identify with certainty a group of patients who have had a recurrence (high positive predictive value [PPV]) without concern for what percent of true recurrences are captured (low sensitivity), or, conversely, identifying a cohort of all patients who may have had a recurrence (high sensitivity) while the cohort will include a sizeable number of false-positive recurrences (low PPV). Although ROCs may determine a threshold that gives increased certainty of correctly including a group of patients with recurrence for inclusion in a study, inevitably there will be some bias associated with the selection of that cohort. Further, they cannot overcome the limitation that none of the algorithms have resulted in a method that will completely and accurately identify cancer recurrences or metastases and their timing from claims for use in population-level measures of disease burden or as outcomes in studies of treatment effectiveness.

Methods to Enhance Health Claims to Identify Recurrence

Although health claims currently do not contain sufficient information to identify accurately which cancer patients have had a recurrence, they could be modified to include more information about recurrence. The Center for Medicare and Medicaid Services (CMS) has operated the Physician Quality Reporting System (PQRS) that includes claims-based reporting of data on individual quality measures. The program has given physicians increased compensation if they voluntarily report on at least three applicable measures. In 2014, several cancer-related measures require that the physician consider information related to stage, such as “Percentage of patients aged 18 through 80 years with AJCC Stage III colon cancer who are referred for adjuvant chemotherapy” and the “Percentage of patients, regardless of age, with a diagnosis of breast, colon, or rectal cancer who are seen in the ambulatory setting who have a baseline AJCC cancer stage or documentation that the cancer is metastatic in the medical record at least once within 12 months” (20). While voluntary reporting of these measures provides some aggregate information, it does not ensure comprehensive information of each cancer patient’s disease status. Collecting such information would require that insurers mandate such reporting from physician. There is precedent for such a requirement. In 2008, CMS required that all erythropoietin stimulating agents (ESA) claims for non-ESRD patients must include on the bill information about a patient’s hematocrit and hemoglobin levels (21). This action was taken because of the high cost of ESAs to the Medicare program. The rising costs of chemotherapy may provide similar impetus to all health insurers that they impose mandatory requirement that oncologists provide information about the disease status on each patient’s claims.

Other Potential Population-Based Data Sources to Identify Recurrence

Enhanced reporting on health claims is but one option to improve the reporting of recurrence. Electronic health records (EHRs) have potential as an alternate source of information about cancer recurrence. In 2013, 78% of office-based physicians reported that they had some form of electronic record in their office (22) and 59% of hospitals have at least a basic electronic medical record (23). Some EHR vendors have developed dedicated oncology modules (eg, EPIC BEACON). The growing use of electronic health records has resulted in the American Society of Clinical Oncology (ASCO) creating the CancerLinQ project to improve quality of cancer care through information obtained from the medical record (24). Information about a patient’s disease status will be crucial to assess quality. Although there has been substantive adoption of use of electronic records, there is considerable variability in the EHR software and the data fields for reporting information about disease status. Using information from the EHR also raises questions about how recurrence will be defined. With complete clinical information at hand, is a recurrence defined based on biopsy, laboratory results, imaging, or symptoms? In addition, EHRs do not include complete information for patients who receive care from multiple, unaffiliated providers. Providers from closed healthcare systems such as the Veterans Administration’s Compensation and Pension Record Interchange (CAPRI) program (25) and the Virtual Data Warehouse from HMOs that are part of the Cancer Research Network (26) have developed methods to consolidate and standardize EHR data across different providers within their system. However, until EHRs cover the span of treatment locations for most patients and the EHR software is formatted to allow for the consistent reporting of disease status when a patient receives health care, the medical record will have a limited role in assessing cancer recurrence.

Population-based cancer registries serve as the major data source for assessing trends in cancer incidence, stage at diagnosis, overall survival, and mortality in the United States. Cancer registries are mandated to collect data about all incident cancers occurring in defined geographic areas. Collection of such information requires in-depth data abstraction by cancer registrars, which often must obtain information from in-person review of the hospital record. Registries are not funded to undertake patient follow-up other than obtaining information about vital status by linking to existing administrative data. Active follow-up, including contacts with community physicians who are treating cancer patients, is very challenging logistically and such efforts would be extremely expensive. As a result, registries do not collect information about the frequency and timing of cancer recurrence. However, the growing use of electronic pathology reports offers the possibility that registries could leverage existing electronic data to identify patients with recurrent disease in a less labor-intensive manner. The North American Association of Central Cancer Registries has promoted the concept of using electronic pathology reporting to enhance ascertainment of incident cancers (27), with 39 state registries currently using electronic pathology reports (28). While electronic pathology reports do not contain the totality of information needed to identify recurrence, they could be used to focus the efforts of cancer registrars, thereby reducing the resources needed to collect information about those recurrences.

Benefit From Improving the Measurement of Recurrence

There are many professional, governmental, and research entities that would benefit from improved reporting of cancer recurrence. The National Quality Forum has endorsed as a quality measure that patients with painful bone metastases should receive palliative radiation therapy (29). The Patient Centered Outcomes Research Institute (PCORI) has identified as a priority the need for more research on the impact of the fear of recurrence on cancer survivors (30). The National Comprehensive Cancer Network (NCCN) guidelines for patients with colon cancer include RAS testing for patients with metastatic disease (31). The ability to assess the measures from each of these groups’ recommendations is hampered by the lack of a comprehensive population-based information about whether a recurrence has occurred and, if so, when. In addition, the types of measures that are available to assess cancer quality and outcomes could be expanded if there was available information about disease status. This could result in better quality care for cancer patients and improved ways to incentivize providers.

Accurate information about disease status is needed by insurers. To address the increasing costs of chemotherapies, insurers are implementing programs that reward physicians if they treat patients with chemotherapy regimens that are supported by evidence and recommended or preferred by the insurer (32,33). To assess the appropriateness of the treatments provided, insurers need to have an accurate assessment of the cancer patient’s disease status. Reimbursement for oncology care is moving towards bundled or episode-based payments. Accurate reporting about a patient’s recurrence is an important component in determining the initiation or end of an episode and appropriate reimbursement using these new payment models.

The United States is in the midst of a paradigm shift related to how cancer care is delivered and valued. With the escalating cost of cancer care, patients, providers, and insurers want to know that they are receiving benefit relative to the amount spent on health care. Benefit is also being measured by the expanding quality measures and by greater interest in patient outcomes. With a growing and aging population, medical costs associated with cancer are anticipated to exceed $158 billion in 2020, an increase of at least 27% over 2010 (34). If utilization of expensive new chemotherapy agents continues to increase, these costs will be substantially higher. It is increasingly necessary to be able determine the value of these expenditures for patients and providers making decisions about treatment and for health systems and policy makers more broadly. For cancer care, we need to start by being able to assess recurrence. The availability of population-based data on cancer recurrence will only happen if governmental agencies, large insurers, professional societies, and advocacy groups place greater priority on the need for high-quality information about disease status. Greater recognition of the clinical and economic benefit of information about recurrence should mobilize groups to promote the need for more complete data about outcomes for cancer patients.