Evaluation of New Technologies for Cancer Control Based on Population Trends in Disease Incidence and Mortality

Abstract

Cancer interventions often disseminate in the population before evidence of their effectiveness is available. Population disease trends provide a natural experiment for assessing the characteristics of the disease and the potential impact of the intervention. We review models for extracting information from population data for use in economic evaluations of cancer screening interventions. We focus particularly on prostate-specific antigen (PSA) screening for prostate cancer and describe approaches that can be used to project the likely costs and benefits of competing screening policies. Results indicate that the lifetime probability of biopsy-detectable prostate cancer is 33%, the chance of clinical diagnosis without screening is 13%, and the average time from onset to clinical diagnosis is 14 years. Less aggressive screening policies that screen less often and use more conservative criteria (e.g., higher PSA thresholds) for biopsy referral may dramatically reduce PSA screening costs with modest impact on benefit.

Cancer interventions often disseminate in the population prematurely, before conclusive evidence of their efficacy has been obtained. For example, prostate-specific antigen (PSA) screening for prostate cancer became widespread in the United States in the early 1990s (1), but clinical trials to evaluate screening efficacy were initiated in 1993 and published results only in 2009 (2,3). Based largely on these results, the US Preventive Services Task Force (USPSTF) recently recommended against routine PSA screening, a reversal which goes against what has become standard practice in this country (4). However, a great deal of uncertainty still remains about the harms and benefits of prostate cancer screening.

In this chapter, we examine the conundrum—and the opportunity—represented by the premature adoption of cancer interventions. By premature we mean the adoption and dissemination of an intervention before conclusive evidence of its efficacy is available from clinical trials. Premature adoption of an intervention may have a negative impact—if the harms of the intervention ultimately turn out to outweigh the benefits. The key characteristic of a premature intervention in the setting of this paper is simply that conclusive evidence about harm–benefit tradeoffs has not yet been obtained. Our primary example is the case of PSA screening in the United States. Although PSA screening began in the late 1980s and became popular in the early 1990s, large clinical trials first published results concerning PSA screening benefit only in 2009.

The conundrum is clear—if an intervention is adopted in the absence of clarity about its benefits, then not only could we end up squandering money and resources for little benefit, but revelation that benefit is not what was expected could indicate that a reversal of contemporary standard practice is warranted. However, the adoption by a population of a novel intervention presents an opportunity as well, namely to assess the effectiveness and costs of the intervention in the population setting as opposed to the artificial setting of a clinical trial.

Because the population represents the ultimate uncontrolled experiment, great caution has to be exercised in making inferences about the comparative effectiveness of novel interventions based solely on population data. Examples of such inferences are provided by studies conducted by the Cancer Intervention and Surveillance Modeling Network (CISNET) (www.cisnet.cancer.gov). For example, CISNET models have been used to quantify the respective contributions of mammography and adjuvant chemotherapy, two major fronts of progress in breast cancer control, to declines in breast cancer mortality (5), and the contribution of colorectal cancer screening, diet, and treatment to declines in colorectal cancer mortality (6).

In this chapter, we show how premature adoption of cancer interventions and their effects on population trends can be used to help inform economic evaluation and policy decisions. We review and synthesize a series of modeling studies specifically focused on extracting the necessary information from population data following the dissemination of the intervention. In some cases, the models we present have been used to make inferences about the contributions of specific inferences to declines in population mortality; in other cases, models have been used to estimate disease progression rates and characteristics of the intervention from population data. This information is then incorporated in a medical decision-making modeling framework that is designed to facilitate inferences about harm–benefit tradeoffs. We focus specifically on questions about the benefits, harms, and likely costs of PSA screening for prostate cancer, but we also discuss how our methods have been used to learn from trends in colorectal cancer, which are a complex product of changes in behaviors over time as well as changes in screening and treatment practices. We show how well-calibrated models can be of value in determining cost–benefit tradeoffs for policy development and demonstrate that there is an important role for modeling to play in determining sound cancer control polices.

PSA Screening Patterns and Prostate Cancer Trends in the United States

The PSA screening era in the United States began in 1986 when the test was approved for monitoring prostate cancer progression but disseminated rapidly for early detection purposes. Different areas of the United States adopted PSA screening at slightly different times (7), but the period of most significant dissemination was the early 1990s when prostate cancer incidence more than doubled relative to historic trends (8). The peak in incidence was followed by a rapid decline as screening use stabilized, and it was at this point that prostate cancer deaths began to fall. The drop in disease-specific mortality has been sustained and impressive; prostate cancer deaths have declined by 44% since their peak in 1991 (9). Among men aged 50–84, the primary group targeted by screening, the fall has been even more substantial, reaching 49% by 2009.

The harms and benefits of PSA screening have been hotly debated, with speculation that PSA explains the mortality declines counterbalanced by skepticism. Until 2009, when results of the two large screening trials were published (2,3), the population data represented the best available evidence about screening benefit. However, interpreting population mortality trends is complex because the population constitutes the ultimate uncontrolled experiment. In the case of PSA and prostate cancer, there have been multiple other changes in disease control and management that have occurred concurrently with the spread of PSA screening. These include changes in primary treatment, with historical treatment trends showing dramatic increase in radical prostatectomy rates during the 1980s (2,3) and similar increases in the use of adjuvant hormone therapy for localized disease during the mid to late 1990s (10). There have also been changes in the detection and treatment of recurrent disease, primarily due to PSA monitoring following primary treatment.

Can we use population prostate cancer trends to learn about the benefits and harms of PSA screening despite these challenges? This has been the mission of the CISNET prostate group, which has used modeling of prostate cancer in the population as its primary approach.

Surveillance Modeling: Learning About Disease Progression From Population Cancer Trends

Surveillance modeling is an approach designed to learn about the process of disease progression from trends in population incidence and mortality. The central idea is that although the events in disease progression are not all observable, they produce an observable process, namely disease incidence trends, that can be used to inform about the underlying natural history. Disease incidence trends that have been recorded before and after the advent of screening in a population are particularly informative, so long as information is available about screening and biopsy referral practice patterns. In the case of prostate cancer, PSA screening became adopted in the late 1980s, so we have used prostate cancer incidence trends, together with retrospectively ascertained screening patterns in the United States, to make inferences about rates of disease onset, metastasis, and clinical detection in the absence of screening (11).

A Model of Prostate Cancer Progression: Parameter Estimation Using Population Incidence Data

Figure 1 summarizes our model, which includes two main components. The first describes how PSA grows in healthy men and cancer cases, and how this growth varies across the population. The second links PSA with disease progression and describes how the risks of disease spread and generation of clinical symptoms change as PSA grows after disease onset. We assume that the risk of disease onset increases with age and that the risks of disease spread and symptoms are proportional to the level of PSA at any given time. This assumption is a mathematical representation of a mechanism that generates the known correlation between the level of PSA and stage of disease at diagnosis, and was found to be most consistent of several models (12,13) with observed data on PSA growth and disease stage from a retrospective series (14). The natural history parameters are, therefore, the PSA growth rates and risks of disease onset, metastasis, and clinical symptoms.

Figure 1.

A model of prostate cancer (PCA) natural history, diagnosis, and survival in the absence and presence of screening. Following disease onset, PSA is assumed to grow exponentially. The risks of metastasis and clinical diagnosis (dx) increase proportionally with the PSA level. Without screening, the cancer is diagnosed in distant stage, but with screening, detection occurs while disease is still localized. The figure shows how overdiagnosis depends on the date of other-cause (OC) death relative to the lead time, which is the time from screen diagnosis to clinical diagnosis.

Estimation of the natural history parameters proceeds as follows. PSA growth and its variation are based on serial PSA data from the Prostate Cancer Prevention Trial (PCPT), which screened 18 882 men for up to 7 years (15). Of these, 9459 were in the control group and were used for our analysis. We use the results to simulate a population of men aged 50–84 beginning in 1975 and ending in 2000, of whom a fixed percentage experienced disease onset at a rate proportional to their age. After onset, PSA growth is reset based on the PCPT results, and the events of disease metastasis and clinical diagnosis are set to occur at rates that grow proportionally with the PSA level. We superimpose screening, according to US screening patterns (1), on this simulated population and project the corresponding trends in age- and stage-specific incidence. We then vary the rates of onset, metastasis, and clinical diagnosis so that the projected trends best match the observed trends in incidence. We use a simulated likelihood-based framework (11) to quantify the extent of the mismatch and optimize the simulated likelihood to obtain the best-fitting natural history parameters conditional on the PCPT-based PSA growth curves. Details of our methods and results are provided elsewhere (11,16); we note here that the projected stage-specific incidence curves under the fitted natural history parameters capture both the dramatic peak in local-regional incidence observed in the early 1990s and the steady decline in distant-stage incidence observed after this time. The fitted model suggests that the lifetime probability of biopsy-detectable prostate cancer is 33%, whereas the chance of a clinical diagnosis in the absence of screening is 13% and the average time from onset to clinical diagnosis is 14 years on average (17).

Using the Model to Explain Prostate Cancer Mortality Trends

We used our model to investigate the likely role of PSA screening versus changes in prostate cancer treatment in explaining the dramatic and sustained decline in prostate cancer deaths in the United States through the year 2005. To do so, we first needed to project what mortality rates would have been in the absence of screening. We assumed that in the absence of screening or treatment, stage-specific incidence of prostate cancer would have remained constant at levels observed in 1987, just prior to the PSA era, and disease-specific survival would have been similar to survival among cases in the Surveillance, Epidemiology, and End Results (SEER) database diagnosed from 1983 to 1986 who did not receive curative primary therapy. We then used information on treatment trends for localized prostate cancer and results from studies comparing primary treatments with each other and with observation (18,19) to project how changes in treatment might have impacted the number of cases dying from prostate cancer. We found that treatment changes explained about one-third of the drop in prostate cancer mortality by 2005 (20). This left two-thirds to be explained by other factors, chief among them being PSA screening.

Adding PSA screening to the model and projecting disease- specific survival under the resulting model-projected stage distribution produced further declines in disease-specific deaths; screening and treatment together accounted for two-thirds of the drop in prostate cancer mortality by 2005 (Figure 2). We concluded that treatment alone could not explain prostate cancer mortality decline in the United States; screening has likely played an important role and could account for as many as 10 000 lives saved per year by 2005.

Figure 2.

Modeled impact of changes in primary treatment and changes in primary treatment combined with screening on age-adjusted prostate cancer mortality in the United States. The figure shows mortality among men diagnosed after 1975 as observed and then as modeled given changes in treatment and screening. For comparison, the figure also shows total mortality due to prostate cancer in the United States. By 2005, treatment changes account for about one-third of the drop in disease-specific mortality (20), whereas the combination of screening and treatment changes accounts for about two-thirds of the drop in mortality.

Estimating Harms of Prostate Cancer Screening

It has become clear that screening for cancer can confer harm as well as benefits. Imperfect diagnostic tests can lead to false positive results, generating anxiety along with unnecessary biopsies. Overdiagnosis, or detection by screening of cancers that would never have presented clinically during a patients’ lifetime, can lead to unnecessary treatment with all of its consequences. Screening itself is a costly endeavor because of the sheer number of tests that must be conducted to screen a healthy population.

Overdiagnosis is a particular concern in prostate cancer screening. Because prostate cancer is known to have high latent prevalence relative to its clinical incidence, particularly in older men, there is enormous potential for overdiagnosis and overtreatment. The likelihood of overdiagnosis is closely linked with the lead time, which is the time by which screening advances diagnosis. Lead time, in turn, can be estimated from patterns of disease incidence following the dissemination of a new screening test, so long as information is available on screening patterns in the population. In particular, the height and width of the peak in disease incidence after the introduction of a novel screening test are informative about lead time (21). This is because when a sensitive screening test is adopted in a previously unscreened population where latent disease is prevalent, many cases are identified by the test and their date of diagnosis is correspondingly advanced by the lead time. In later years, these cases are no longer present and there is a consequent drop in disease incidence. The lead time determines when the later incidence drop takes place relative to the initial incidence gain. When the lead time is longer, the incidence drop takes place later and the initial incidence gains are sustained, producing a more pronounced incidence peak.

The likelihood of overdiagnosis can be estimated once the distribution of lead time is known, because overdiagnosis occurs when other-cause death takes place after screen detection but before the end of the lead time. Thus, given lead time, the chance of overdiagnosis can be calculated from population life tables.

In the case of PSA screening, the premature dissemination and rapid uptake of the test during the late 1980s and early 1990s have provided an excellent opportunity to estimate the lead time and corresponding overdiagnosis frequency associated with PSA screening. Indeed, our simulated likelihood-based framework for estimating our model parameters produces a virtual population of men in which the times of screen detection and clinical diagnosis in the absence of screening are known. We can use these data to produce empirical estimates of lead time and, given dates of other-cause death, overdiagnosis. We have developed several other algorithms that use data on PSA testing patterns and prostate cancer incidence to estimate lead time and overdiagnosis (17,22–24). Our results consistently point to a frequency of overdiagnosis during the1990s that amounts to approximately one out of every four screen-detected cases in men over age 50. Our results are consistent with another model developed using US data, but are lower than estimates from a model developed partially using data from the European Randomized Study of Screening for Prostate Cancer (24).

Economic Evaluation of Prostate Cancer Screening

The economic implications of cancer screening tests are vast and rest on the drivers of costs that we have already mentioned: the tests themselves, false positive results, and overdiagnosis. Estimation of the costs of prostate cancer screening, therefore, requires an assessment of the costs of testing as well as the costs of prostate biopsies and treatments, including the harms associated with treatment like impotence and incontinence. Given these costs, differences between screening strategies will be determined by how the cost drivers vary across the strategies.

The calibrated model provides a representation for how disease progresses in the absence of screening and, in particular, yields a distribution of age and stage at disease diagnosis without PSA testing. Superimposing a specified screening protocol produces a change in the timing of diagnosis and, consequently, a change in age and stage of disease in the presence of screening. Using stage-specific curves for prostate cancer survival (8), we are able to project the consequences of this earlier detection for disease-specific deaths.

The universe of potential PSA screening strategies is enormous and includes strategies that vary in terms of their starting and stopping ages, interscreening intervals, and criteria for biopsy referral. Each of these screening strategy parameters has been the topic of a great deal of debate and controversy. In the case of criteria for biopsy referral, for example, there is disagreement about the threshold for declaring a test to be abnormal and about whether to base biopsy referral decisions on PSA velocity in addition to absolute PSA (25).

Using our calibrated model, we considered a range of potential strategies and projected a large set of relevant outcomes, including the aforementioned drivers of cost and several measures of benefit. Figure 3 illustrates the results of varying the ages to start and stop screening, the interscreening intervals, and the criteria for biopsy. The results show clearly that less intensive strategies can materially reduce key drivers of cost although only modestly impacting screening benefit.

Figure 3.

Three outcomes of harm (false positive and overdiagnosis) and benefit (years of life saved) corresponding to six candidate PSA screening policies, varying ages to start and stop screening, and interscreening intervals as well as the criterion or threshold for biopsy referral. Outcomes are numbers of false positives, overdiagnoses, and lives saved per 1 million men screened. The ages to start and stop screening are specified below the figure; upper and lower bounds are provided and the interscreening interval is given in parentheses. As an example, the policy 40, 45, 50, (2), 75 indicates that screens take place at ages 40, 45, 50, and thereafter every 2 years until stopping at age 75. The figure shows that less intensive screening strategies can yield dramatic reductions in screening harms with very modest differences in benefit.

Modification of Natural History Models for Other Settings and Health Systems

Some aspects of natural history models are dependent on local population practice patterns. An example is the risk of clinical detection in the absence of screening. This depends on the intensity of prostate cancer diagnosis due to other means, and this can differ greatly across population settings. When the same model was calibrated to prostate cancer incidence patterns in the Rotterdam section of the European Randomized Trial of Screening for Prostate Cancer (ERSPC) and then again to data on prostate cancer incidence in the US population after the advent of PSA screening, the clinical incidence hazard rate was higher in the model fit to the US data than in the model fit to the Rotterdam data (24). This example indicates that one important criterion to be applied when selecting data sources as inputs for population-based modeling is that the data should match the setting for which policy is eventually going to be developed. In developing policies for prostate cancer screening in the United States, it will not be appropriate to use models calibrated to ERSPC data.

Discussion

In this chapter, we have shown how dissemination of cancer interventions at the population level can be used to inform about harm and benefit, key inputs for the development of sound public health policies. We have also demonstrated how a well-calibrated model can be adapted and used for economic evaluation of candidate policies that go beyond historic population practices. Our results focused on specific drivers of cost rather than the economic costs themselves, because we were interested in differentiating between harms like false positive tests and overdiagnosis. Unlike costs, which vary across clinical and geographical settings, these measures of harm have consistent absolute interpretations. However, the translations of these measures into economic costs of care will be necessary for cost-effectiveness comparisons. Information on the costs of care is available from a wide variety of sources. For example, Ekwueme et al. (26) reviewed 28 studies (15 US and 13 international) of publicly available data on the resource costs of prostate cancer screening, diagnosing, and staging. They were able to quantify and pool both direct costs—resources used, physician costs, medical supplies, and facility costs—and indirect costs, such as loss of income from time off work, transportation costs, and travel time. Once the costs of different aspects of care have been quantified, they can be incorporated into the models as multipliers of the numbers of corresponding procedures (eg, for screening tests or biopsies) or cases (eg, for treatment costs).

We have focused on the example of PSA screening for prostate cancer, adopted in the United States even before the initiation of the US trial of prostate cancer screening, which began enrollment in 1993. There are many other cases where interventions have been adopted prematurely and, with the subsequent release of data indicating adverse impact, have been dropped on a wide scale. A classic example is that of female hormone replacement therapy, which was broadly adopted in the United States until publication of results from the Women’s Health Initiative in 2002 showed that it adversely impacted cardiovascular and breast cancer risks (27). Examples in cancer chemotherapy abound. In France, for instance, between 2004 and 2010, 31 new cancer drugs obtained market approval, the majority of which were targeted therapies (usually monoclonal antibodies). Although the actual medical benefit from targeted therapies was seldom challenged, the Transparency Commission expressed reservations regarding the survival advantage over existing treatments. In 2009 and 2010, eight targeted drugs were reviewed and received market approval with no improvement in actual benefit and only a few were rated as providing a minor improvement in actual benefit. In the United States, the US Food and Drug Administration actually revoked its accelerated approval of the drug Avastin for advanced breast cancer, noting that the drug “used for metastatic breast cancer has not been shown to provide a benefit, in terms of delay in the growth of tumors, that would justify its serious and potentially life-threatening risks.”

We have demonstrated how the surveillance modeling approach allows us to separate the contributions of PSA screening and changes in primary treatment to the declines in prostate cancer mortality. This approach has been similarly used in breast cancer, to separate the contributions of screening and changes in chemotherapy (5), and in colorectal cancer (28), where changes in disease-impacting behaviors over time must also be considered. The MISCAN-colon micro-simulation model used four waves of data from the National Health and Nutrition Examination Survey (NHANES) to estimate the prevalence over time of risk factors, such as physical activity; fruit and vegetable consumption; and use of folate, aspirin, and female hormone replacement therapy. Incorporating estimates of the effects of these risk factors on colorectal cancer incidence from the epidemiological case-control studies allowed the model to separately project the contributions of these factors and the contributions of screening and treatment to mortality (6). The MISCAN-colon model has also been harnessed to compare different potential screening policies, and their results have been used by the USPSTF in determining their most recent recommendations (29). This case of the use of modeling within the policy development process is still unfortunately the exception rather than the rule. The USPSTF has used modeling in defining policy for both breast (30) and colorectal cancer screening (29), but not for prostate cancer screening. And most professional societies do not use models to quantify harm–benefit tradeoffs, but rather rely on literature review and consensus decision making on the basis of observed results. These may not even reflect the likely long-term population costs and benefits of the policies that are being considered. Certainly, economic evaluation on the basis of disease modeling may produce results that are unpopular, particularly if they project that costs of new promising interventions are excessive relative to benefits. However, this type of analysis, on the basis of well-calibrated models, is likely to be a critically important weapon in our battle to manage health-care costs while advancing cancer control in the future.

Funding

This work was made possible by Award Numbers U01-CA-157224 and U01-CA-152959 from the National Cancer Institute .

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Cancer Institute, the National Institutes of Health, or the Centers for Disease Control and Prevention.