Comparing Cancer Care, Outcomes, and Costs Across Health Systems: Charting the Course

This monograph highlights the multiple payoffs from comparing patterns of cancer care, costs, and outcomes across health systems, both within a single country or across countries, and at a point in time or over time. The focus of comparative studies can be on the relative performance of systems in delivering quality cancer care, in controlling the cost of cancer care, or in improving outcomes, such as reducing mortality rates and improving survival. The focus also can be on comparing the effectiveness, cost, or cost-effectiveness of competing cancer prevention and control interventions within a given system or across systems, while taking into account variations in patient characteristics, disease incidence and severity, resource availability, unit costs, and other factors influencing system performance.

Two recurring themes in this monograph are: 1) the opportunities for cross-system analysis, learning, and improvement are enormous and just beginning to be tapped; and 2) the empirical and methodological challenges in realizing this potential are likewise enormous, but real progress is being made. In this concluding article, we revisit and illustrate both themes, with the aim of suggesting a research agenda for enhancing capacity to conduct strong empirical cross-system analyses in cancer care delivery. To focus the inquiry, we limit consideration to those cancer care systems, whether within or across countries, sufficiently developed to have access to registries that not only can document cancer incidence and mortality but, through linkage to additional data sources, can serve as platforms for patterns-of-care, costing, or other in-depth studies. This necessarily puts the spotlight on developed nations; and among these, we concentrate on those in Europe and North America represented at the September 2010 workshop, “Combining Epidemiology and Economics for Measurement of Cancer Costs,” in Frascati, Italy (1).

We distinguish between population-level studies, designed to compare the performance of health systems across countries or within a single country along specified dimensions, and patient-level studies, designed to investigate the effectiveness, cost, or cost-effectiveness of specific interventions and programs for individual patients (or individuals at risk for cancer) either within a given health-care system or across systems. In population-level studies, the outcome of interest might be summary measures of cancer mortality, survival, or other prominent patient outcome–oriented indexes of performance that are feasible to measure across systems for defined populations. Patient-level studies will often investigate the determinants of variations in patterns of care, costs, or outcomes, or apply economic evaluation methods to examine whether specific interventions offer good value for money. Although most patient-level studies to date are within-country or within-system, we note important examples of cross-country or cross-system analyses.

In the next section, we highlight some examples of population- and patient-level studies. This sets the stage for the subsequent sections discussing a range of options, including some already in progress, for strengthening the data, methods, and organizational infrastructure to support policy-relevant comparative research on cancer outcomes and costs.

Comparisons Across Health Systems: Informative but DifficultPopulation-Level Studies

The methods for conducting empirically sound cross-national comparisons of cancer incidence, mortality, and survival are relatively well developed. In recent years, important and frequent collaborative contributions have been made by research teams organized by the International Agency for Research on Cancer (IARC) of the World Health Organization and the International Association of Cancer Registries (IACR) (2), as well as by the EUROCARE (European Cancer Registry–based Study on Survival and Care) study group (3,4). Growing out of EUROCARE-3 was the CONCORD study, which provided survival estimates for about 1.9 million adults diagnosed with female breast, colon, rectum, or prostate cancers during 1990–1994, and followed up to 1999 (5). Projects led by EUROCARE and EUROPREVAL have analyzed cancer prevalence within and across European countries (4).

Although these and other prominent studies (6) have compared disease incidence, prevalence, mortality, and survival (singly or jointly), there are evidently no recent cross-national studies on cancer cost, whether overall or by disease site. Although Organization for Economic Cooperation and Development (OECD) compiles and publishes country-specific data on health expenditures and its components, it does not produce cross-national cost estimates by disease class or specific cancer diagnoses (7).

There are noteworthy examples of within-country efforts to monitor health system performance on cancer metrics over time. In Canada, Cancer Care Ontario (CCO) supports the Ontario Cancer System Quality Index (8). In the United States, the Agency for Healthcare Research and Quality publishes each year the National Health Care Quality Report (9), and several US cancer agencies and organizations collaborate to produce an annual “report to the nation” on incidence, mortality, survival, and selected special topics (10).

Patient-Level Comparative Studies

The substantial diversity of health-care delivery systems across countries, and indeed within any country, creates significant opportunities for policy-relevant research comparing alternative approaches to care delivery along the cancer continuum: prevention, detection, treatment, survivorship, and end-of-life care (11,12). By observing how seemingly similar individuals either at risk for cancer or with the disease are treated in different systems, we have the opportunity in principle of benefitting from what amounts to quasi-natural experiments in care delivery (13). This could allow for benchmarking of “high quality” or “high value” services and identifying best (and less than best) practices.

One cross-national comparison is well illustrated in the study of colorectal cancer treatment patterns in Italy and the United States reported herein by Gigli and colleagues (14), who found clear between-country differences in use of adjuvant therapy, open abdominal surgery and endoscopic procedures, and hospitalization. Similarly, Warren and colleagues (15) compared end-of-life care for non–small cell lung cancer patients aged 65 and over in Ontario and the United States, finding significantly greater use of chemotherapy in the United States, but higher rates of hospitalization in the last 30 days of life in Ontario. Each study was feasible because the participating countries could link high-quality cancer registry data with administrative files to identify similar cancer patients and then track receipt of services over time.

In cross-national settings where insurance or other administrative data files are not available or accessible, alternative strategies for augmenting cancer registry data can be pursued. An instructive case in point is the “high resolution” analyses reported by Gatta and colleagues (16), examining the impact of guideline-recommended care on survival in samples of patients diagnosed with breast, colorectal, or prostate cancer across a number of European countries. Building on earlier EUROCARE studies (17–20), these analyses brought together cancer registry data enhanced with additional clinical detail from multiple participating registries and countries (eg, for breast cancer, data from 26 registries in 12 countries). Included as determinants of cross-country survival differences were such macro-level variables as total spending on health care and the relative availability of such inputs as computed tomography, magnetic resonance imaging, and radiotherapy equipment.

Several implications flow from these cross-system studies. For valid and reliable analyses of cancer care, outcomes, and costs across geographical boundaries, high-quality registry data (or its clinical equivalent) are necessary, but generally not sufficient. Such data must be augmented with either administrative files or additional clinical information to provide an accurate time profile of patient-level diagnoses, services and procedures received, and outcomes, as well as patient, provider, and health system variables. For any given health system comparison, all pertinent variables should be defined and measured in the same way, or at least measure the same construct.

We are far from achieving widespread international “interoperability” in measurement and reporting of cancer care use and costs. The resulting challenges in being able to draw valid cross-country inferences from existing studies are well illustrated in our review here of economic studies in colorectal cancer, as conducted primarily in countries with well-developed networks of cancer registries (21). In the main, studies from different countries yielded estimates of direct medical costs in ways that precluded a sound comparison across studies. Few studies estimated direct nonmedical costs (eg, patient or caregiver time) or the productivity costs associated with disease and treatments. Indeed, aggregate and patient-level cost estimates varied in so many ways across countries that meaningful comparisons now are almost impossible. A broadly similar conclusion emerges from the review of colorectal cancer patterns of care studies from across Europe, Australia, and New Zealand (22) and in comparisons between Canada and the United States (23).

That challenges in conducting micro-level analyses can arise across health-care systems within a country is underscored by Fishman and colleagues (24). They describe the data system hurdles in conducting comparative effectiveness research in samples of elderly US cancer patients when some are enrolled in Medicare for-for-service (FFS) plans and others in Medicare-managed care plans that include health maintenance organizations (HMOs). As one direct response to the issue of data comparability within Medicare, Rosetti and colleagues (25) developed a “Standardized Relative Resource Cost Algorithm” (SRRCA) to assign standardized (comparable) relative costs to cancer patients in HMOs and FFS plans.

Such innovative fixes as the SRRCA represent important, yet incremental, steps toward addressing a more fundamental issue in conducting sound comparative effectiveness research within the United States. With its strong cancer registry networks but vast array of administrative data systems and non-interoperable electronic health informatics systems, how does the country advance toward a “national cancer data system,” as advocated by the Institute of Medicine in 1999 (26) and echoed by multiple cancer policy makers since then? (27).

Building Capacity for Comparative Studies Across Health Systems

Enhancing the Empirical Base

High-quality sources of data to support scientifically sound population-based studies of cancer care, outcomes, and costs have emerged most often from partnerships involving some combination of government agencies, professional and provider organizations, and researchers. The empirical infrastructure required for comparative analyses will not simply emerge on its own, as the product somehow of “natural market forces” in the health-care arena. Little disagreement arises among payers, providers, and consumers of cancer care surrounding the contention that decision making about competing interventions should be informed by solid evidence on effectiveness and costs. But only rarely does any single or combination of these private stakeholders have the financial and organizational wherewithal, or indeed an adequate incentive, to take on the full task of building and sustaining a population-level database for cancer research. Now, if by some means the necessary empirical infrastructure does emerge, one would want to encourage its broad and rapid application, not only by the parties that paid for it but by qualified researchers everywhere, and assure that its use by one set of researchers does not diminish its availability or utility to others. In this sense, the data infrastructure needed to support population-level cancer research could well be characterized as a type of public good, with the implication that it will be underproduced in the absence of collective action organized and supported by public agencies.

This line of argument (or at least aspects of it) has been well recognized in both the North American and European arenas for population-level cancer research (28). As noted, the EUROCARE project, based in Milan and Rome, has developed the capacity to draw survival and other surveillance data from over 80 publicly supported cancer registries in 21 European nations covering about 36% of their combined populations (16). In Canada, the health services research program jointly sponsored by CCO and the Institute for Clinical Evaluative Sciences (ICES) has developed publicly available datasets linking clinical and administrative information on cancer care, outcomes, and resource utilization in the province of Ontario (29), and now most Canadian provinces have similar linked datasets. Most recently, Ontario and British Columbia researchers teamed up to examine pre- and post-diagnosis cancer-related costs for multiple tumor sites (30). In the United States, the SEER–Medicare linked database represents a partnership involving the National Cancer Institute (NCI), the Centers for Medicare and Medicaid Services (CMS), and the federally supported SEER registries covering roughly 28% of the US population (31,32). The Cancer Research Network has developed standardized tumor, clinical, utilization, and cost data for large HMOs in the United States, all of which have electronic medical record systems (33,34). The Centers for Disease Control and Prevention (CDC), in collaboration with seven state cancer registries and multiple university-based researchers, have supported the Breast and Prostate Cancer Data Quality and Patterns of Care Study, creating large population-based samples to study quality-of-care and survival outcomes (35).

Current collaborative efforts, however, fall short of providing cancer researchers and policy makers with the data platforms required for population-based studies encompassing all geographical regions, all population groups, and the full range of clinical, patient-reported, and cost-related outcomes that can inform decision making. Specific research initiatives such as the NCI-created Cancer Care Outcomes Research and Surveillance (CanCORS) Consortium (36) have rendered proof of concept that primary data collection and multiple datasets linked together can effectively support a range of important innovative studies (37,38). But such initiatives alone are not intended to address the larger matter of how to develop and sustain the empirical base for population-based cancer research over time. What are the prospects for building sustainable data platforms that are accessible and affordable to a broad swath of individual researchers and policy makers? A comprehensive pursuit of this mammoth topic would require its own monograph, but we highlight some notable examples.

European Partnership for Action Against Cancer and Other European Confederations.

The European Partnership for Action Against Cancer (EPAAC) is a confederation of over 30 public and private sector organizations that seeks to work closely with the European Union, the IARC, the European Network of Cancer Registries (ENCR), the EUROCARE project, the OECD, and others to advance an ambitious agenda for cancer prevention and control research (39). Among EPAAC’s objectives is a “European Cancer Information System” that would draw on multiple partnerships to develop harmonized population-based data on cancer incidence, survival, prevalence, mortality, and also high-resolution studies to examine the impact of medical resource availability, patient-level variables including lifestyle factors, and specific interventions on outcomes. In a complementary development, IARC and ENCR announced in 2012 the creation of a European Cancer Observatory to provide easier access to basic surveillance data from over 40 European countries (40). Although not disease-focused, the “EUnetHTA” is a network of government-appointed organizations, regional agencies, and nonprofit organizations established in 2008 to harmonize and improve the quality of health technology assessment across Europe (41). As such, its work could eventually inform the evaluation efforts in specific domains, including cancer.

CCO–ICES and Other Provincial Partnerships in Canada.

Potentially well positioned to create and sustain data platforms for cancer care, cost, and outcomes research is Canada, at least on a province-by-province basis, as the CCO–ICES health services research initiative in Ontario is beginning to demonstrate (29). A particularly strong feature of this system is the capability of linking cancer registry data with additional clinical information and service provision data from the province’s publicly funded universal health-care system. As a result, it is possible to track medical services rendered, the corresponding resources consumed, and survival outcomes over time on a population basis.

American College of Surgeons and American Society of Clinical Oncology.

In the United States, there are several parallel initiatives underway to strengthen the capability for monitoring and improving the quality of cancer care. These include the American College of Surgeons (ACoS) Commission on Cancer’s (CoC) Rapid Quality Reporting System (42), already adopted in over 20% of the CoC’s 1500 approved cancer programs, and the new “CancerLinQ” information system under development by the American Society of Clinical Oncology (ASCO) (43). Both of these far-reaching initiatives are aimed at providing near real-time feedback to care providers and eventually at strengthening the basis for comparative effectiveness research of cancer therapies. As currently configured, neither appears readily geared to support population-based cost or cost-effectiveness analyses of care across the cancer continuum.

SEER–Medicare: Building on the Concept.

A key to making further progress on the economic analysis front is pursuit of a strategy that is simple in concept but complex in execution: Expand the SEER–Medicare linked dataset “model” to cover virtually 100% of the US population—in partnership with the CDC’s National Program of Cancer Registries—and to include linkages with administrative data from Medicaid and as many major private insurance plans and managed care organizations as possible. If data elements were standardized and harmonized across payers, the result would be linked cancer registry–claims data yielding population-representative samples across all ages, geographical areas, and types of health plans. Clearly, a number of major organizational, financial, and perhaps even legal hurdles would have to be cleared for such an ambitious plan to take flight and become sustainable over time.

Extracting Maximal Value From the Empirical Base: The Essential Role of Modeling

At the core of any epidemiologically based analysis of health outcomes and cost is a model (44) and a number of associated tasks. The tasks can be viewed as falling under two headings: 1) using the available data to assign values (either point estimates or probability distributions) to all the variables deployed in the analysis and then investigating each of the hypothesized causal connections, for example, impact of intervention A on health outcome X, or the impact of Y on cost outcome C, or both, after adjusting for confounding; and 2) combining these estimated variables, and their inferred causal connections, into some form of decision model to investigate the impact of alternative intervention strategies on the outcomes of interest (eg, health outcomes, cost, or cost-effectiveness) for some selected target population. The decision model becomes the analytical platform for posing compelling “what if” questions. For example, how costs are expected to shift if intervention X′ is selected rather than X? At the same time, the decision model is the vehicle for evaluating policy options (X versus X′) to optimize some designated criterion, for example, cost per quality-adjusted life year. The pivotal point is that in studying the impact of X versus X′ in the selected target population, the analyst is not necessarily constrained by data availability or data quality limitations within that population. Rather, the aim is to make the decision model appropriate to the question at hand by bringing to bear the best available data from all feasible sources.

Statistical Inference and Prediction

Whatever the outcome being investigated, the within-country or cross-country context, or the strengths and limitations of the corresponding empirical base, paying close attention to strategies for both statistical inference and decision modeling is foundational. We briefly call attention to three problems of statistical inference (among many) that are especially pertinent: (a) appropriately characterizing the distributional features of the outcome of interest (a particular concern when cost is the dependent variable); (b) adjusting for patient-related and other selection effects that otherwise can lead to biased inferences about the impact of factors on outcomes, costs, or both; and (c) recognizing that cancer care interventions may be complex, multilevel, and delivered in geographical and clinical environments characterized by the statistical phenomenon of “clustering.”

Over the past two decades, considerable progress has been made in coping with (a), especially in the area of cost, where robust generalized modeling approaches have been developed (45–47). Regarding (b), the threat of selection bias in the estimation of outcomes, including cost, has long been recognized in the econometrics literature. In recent years, two basic approaches to bias reduction have been pursued, with applications in the health-care arena accelerating over the past decade: propensity score matching or weighting (48) and instrumental variable (IV) methods (49–54), which seek to identify and remove biasing effects arising from observable or unobservable influences on the dependent variable of interest. Likewise, developing cost estimation and prediction models that jointly handle problems (a), (b), and (c) by recognizing the frequently hierarchical nature of interventions is a prime area for further work (54–56).

Decision Modeling

Consider the following policy questions:

• What are the relative contributions of screening and adjuvant therapy to achieving reductions in mortality from breast cancer?

• What is the effect of rising chemotherapy costs on the possible cost savings from colorectal cancer screening?

• What is the cost-effectiveness of human papillomavirus vaccination and cervical cancer screening in women older than 30?

• How may one estimate the clinical benefits, harms, and cost implications of a particular cancer screening program prior to its widespread adoption so as to inform decision making about optimal screening policy?

These seemingly diverse inquiries in cancer prevention and control have certain important features in common. They are complex, involving many clinical and economic considerations. The time horizon over which clinical benefits, harms, and costs flow at the patient level will not be measured in months but years and, indeed, may span the remainder of the individual’s life, from the point of intervention going forward. It is highly unlikely that either experimental or observational data would be available for any one cohort in sufficient detail and duration to include direct observations on all the variables involved in the multiperiod investigation.

There is one more feature in common: Each of these four questions has already been investigated in impressive detail using some form of decision modeling (57–60), most typically a variant of micro-simulation. However strong or deficient the empirical base for population-based cancer research within a health system or across health systems, adopting a decision modeling strategy provides the additional flexibility to bring the best available data to bear (whatever the source) on the problem at hand.

Conclusions

The central challenge in conducting technically sound comparative analyses of cancer care patterns, outcomes, or costs across health-care systems is marshaling the skill, the will, and the fiscal and administrative resources to develop and sustain the necessary data infrastructure that can support strong (and frequently team-based) research. Whether for cross-national studies or within-country studies, the task is made all the more difficult because most of the component building blocks for national, regional, or state cancer data systems—including insurance and other administrative data sources, medical records systems, and even cancer registries—were not originally designed to support research.

Nonetheless, the empirical base needed for a given investigation can frequently be created through some combination of dataset cleaning and updating (eg, re-abstracted registry records); dataset linkages (eg, registry data with claims files, or registry data with medical records); and/or dataset creation (eg, surveys to collect individual-level data on cancer risk-increasing or risk-reducing behaviors, time costs, or patient-reported outcomes, in some cases using the cancer registry to establish the sampling frame). Indeed, some projects have linked both secondary and newly created sources to provide a rich longitudinal picture of the cancer patient experience over time, from diagnosis, through treatment, and into the survivorship period (36).

Population-based cancer registries, whether covering a city, state, province, region, or entire country, are the bedrocks not only of epidemiological investigations of disease trends but also trends in cancer patterns of care and economic cost. As a result of sustained work by tumor registries and their affiliated experts worldwide, a consensus is emerging about the international rules-of-the-road for cancer surveillance data definition, collection, and analysis (2) (pp. 67–71). Over time, disparate registry operations have developed operational definitions and criteria for appraising data completeness, accurate identification of true-positive cancer cases, and approaches to computing and reporting statistics on incidence, prevalence, mortality, and survival (61,62). This standardization supports current and future efforts to foster comparative analyses of cancer care, outcomes, and costs.

Yet to date and to our knowledge, no country-level comparative studies of the cost of cancer have been published, either in the aggregate or by disease site. What is lacking, to be sure, is not the methodological wherewithal, but the data on cancer care resource consumption and prices that have historically been well beyond the scope of registries. Without some systematic, technically feasible, affordable, and sustainable strategy for augmenting registry data on an ongoing basis with additional sources of information on cancer care delivery and resource use, it is difficult to see how country-level comparisons of cancer costs can be estimated directly, that is, from the ground up. As suggested earlier, a viable alternative strategy is to deploy epidemiologically grounded economic modeling, bringing to bear the most appropriate data for cost inferences from multiple information sources.

The policy significance of comparative investigations across health systems has recently been underscored in a report issued by the US National Research Council and the Institute of Medicine finding that US males and females at all ages (up to 75) have greater rates of disease and injury, and shorter life expectancies, than in 16 other wealthy nations (63). The report’s recommendations to improve the quality and consistency of data, as well as analytic methods and study designs, highlight a growing consensus about the importance of building capacity for sound comparative analyses. That such comparative analyses can highlight successes, as well as failures, in pursuit of the “triple aim” of better health, better health care, and lower cost is well illustrated in a recently published series of papers (64).

In sum, progress in producing scientifically strong, policy-relevant comparative analyses of cancer care, health outcomes, and costs within and across systems requires continuing investments on three fronts: database development, statistical inference and prediction, and decision modeling. They go hand in hand. What would be the payoffs for such an investment? What are some of the compelling questions and issues that could be more effectively addressed through stronger cancer data systems and research methods? The list is long, but would surely include:

• Assessing the effects on downstream outcomes and costs of specific cancer prevention and screening strategies.

• Investigating the impact of existing high-cost anticancer agents and emerging technologies and interventions (eg, genomics-guided targeted therapies) on outcomes and the costs faced by patients, health-care systems, and governments.

• Evaluating alternative patient management strategies after the initial therapy, including surveillance during the survivorship period and end-of-life care.

• Studying the cost and cost-effectiveness of interventions at any point along the cancer continuum and including the direct medical costs, as incurred typically within health-care systems, direct nonmedical costs (eg, capturing the value of patient and caregiver time), and the cost of disease-related lost productivity.

Funding

This work was supported by the National Cancer Institute (contract HHSN261201100370P and 5P30CA138292, the Cancer Center Support Grant to Winship Cancer Institute of Emory University).