The analysis of case-control studies of the efficacy of screening for recurrence of cancer

Abstract

Occasionally, the efficacy of screening for cancer recurrence has been evaluated by means of case-control studies. However, these studies generally have not been designed and analyzed in a manner that will yield an unbiased result. The Commentary shows that by adhering to the principles developed for the analysis of studies of the efficacy of screening for the presence of a given cancer in persons with no history of this disease, a case-control study of screening for recurrence will improve its chances of obtaining valid findings.

Unfortunately, some patients whose cancer initially appears to have been cured later sustain a recurrence that can prove fatal. Treatment is available for the management of recurrences of many forms of cancer, and it is possible that such treatment could be relatively more effective if recurrences were diagnosed as early as possible.

Efforts to identify recurrences of cancer before they would be clinically evident – often termed “surveillance” for recurrence – can take a variety of forms, including: direct visualization (e.g. post-surgical endoscopy for colon cancer); radiographic examination (e.g. mammographic surveillance in women with breast cancer who have not undergone bilateral mastectomy); and the detection of circulating antigens that are over-expressed by tumors (e.g. prostate specific antigen, carcinoembryonic antigen (CEA)).

While there are different approaches to evaluating the impact of the use of surveillance modalities on the likelihood of survival from cancer (1), an important class of these entails a comparison of cancer mortality in patients apparently cancer free after initial treatment who do and do not subsequently undergo surveillance (or who undergo different levels of surveillance). Occasionally, patients are entered into a trial in which chance alone dictates whether they do or do not undergo a particular surveillance regimen. For example, among patients who had an apparently curative resection of colon cancer, a program of relatively more aggressive surveillance - including frequent CEA testing and hepatic imaging, in addition to endoscopy – has been compared to a less intensive program (2). However, it is unusual for a surveillance regimen to be evaluated by means of a randomized trial. So, for some forms of surveillance non-randomized evaluations are conducted, in particular case-control studies. For example, among patients enrolled in Medicare who were diagnosed with local or regional stage colorectal cancer during 1986-96 and who had survived 6 months, Ramsey et al (3) contrasted the 8,130 patients who had died of colorectal cancer and a sample of the other patients (controls) for receipt of surveillance endoscopy. Had the authors observed that a lower proportion of fatal cases than controls received surveillance endoscopy, this would have suggested that early identification of local recurrence could lead to the prevention of some deaths from colorectal cancer.

It has been shown that in case-control studies of the efficacy of screening for a given cancer in persons with no history of that cancer, the likelihood of a valid result depends heavily on the ability of the study to accurately measure receipt of screening and to analyze properly the data obtained (4). The purpose of the present commentary is to show that these same lessons apply to case-control studies of the efficacy of surveillance for cancer recurrence.

Patterns of Screening for Recurrence in Cases and Controls

Surveillance has the potential to be successful if the modality in question is administered to a patient during the period of time that an asymptomatic recurrence is present and detectable by means of that modality – the detectable pre-clinical phase (DPP) of the recurrence. For a form of surveillance that is highly sensitive, an exam performed when a recurrence is present will very likely be positive and generally will lead promptly to a diagnosis of the recurrence. Thus, if such a test had been performed during the DPP of the recurrence in a person who went on to die of the cancer, most likely it would have been close in time to the diagnosis of the recurrence. However, for controls – typically one or more patients diagnosed with the cancer in question at about the same time as the fatal case who had not succumbed to the cancer and had not sustained a recurrence as of the date of the case’s recurrence – surveillance exams could have been performed at any point during the period corresponding to the case’s DPP. Unlike the surveillance exams in cases, those in controls would not be concentrated close to the time of “their” case’s recurrence diagnosis.

The figure illustrates a hypothetical study of 20 persons who died from a given form of cancer – after being clinically free of the disease following their initial treatment – and 20 controls. The latter are persons diagnosed with the cancer around the date of their fatal case’s initial diagnosis who survived and who were recurrence-free as of the time of the fatal case’s recurrence. In this example it is assumed that there is no treatment that influences survival after recurrence, so an identical proportion (6/20) of fatal cases and controls would have been screened for recurrence during the 12 months that such a recurrence could have been screen-detected. The distribution of the surveillance exams during that 12-month interval would have differed between fatal cases and controls – being 100% sensitive, the test would have taken place close to the time of diagnosis of recurrence of the six fatal cases who underwent surveillance, whereas the six surveillance exams in controls could have occurred at any time within the year prior to their matched case’s recurrence date. Thus, an analysis of these data that does not consider screening for recurrence during the whole of the presumed DPP of 12 months would be expected to give a biased result: it would not find the identical proportion of fatal cases and controls – 6/20 – to have undergone surveillance.

Figure.

Surveillance histories of fatal cases and their controls, assuming a 100% sensitivity of testing, a 12-month detectable preclinical phase of the recurrence, and no efficacy of treatment of surveillance-detected recurrence. Each rectangle represents one person who underwent surveillance for recurrent cancer.

It is also true that if the DPP is long enough for a surveillance exam that is 100% sensitive to be performed more than once, only in controls could this happen more than once: If performed during the DPP of the recurrence sustained by the person who died of cancer, a surveillance exam would have been positive and would have led to the diagnosis of the recurrence, barring the possibility of a second surveillance exam.

Analytic Approaches that Can Give Rise to an Ambiguous Interpretation

These disparities between fatal and nonfatal cases of cancer, with regard to the timing and frequency of a sensitive test for the presence of recurrence, will be present whether or not the recurrence can be effectively treated. The following two examples illustrate the bias that can arise when an analysis of a case-control study of the efficacy of screening for recurrent cancer fails to account for the disparities.

Example 1

In the analysis of the study (described above) of the efficacy of endoscopic screening for recurrent colorectal cancer (3), the investigators were obliged to accommodate the lack of information in automated Medicare records on the reason for endoscopy, i.e. surveillance (screening) versus an investigation of a symptom or sign of recurrent cancer. They chose to delete from the analysis endoscopic exams performed during the 6 months prior to the diagnosis of the recurrence (and the corresponding date in controls), judging the excluded exams to be those most likely to have been done in response to symptoms or signs. Unfortunately, this analytic approach will tend to exclude proportionately more true surveillance exams in person who died from colorectal cancer than in controls, given that only the exams in the former group would be expected to cluster close to the date of the diagnosis of the recurrence. If surveillance endoscopy were a sensitive means of identifying recurrences, even if there were no effective treatment for recurrent colorectal cancer (in terms of mortality reduction) this analysis would be expected to find a deficit of surveillance among persons who died of colorectal cancer. Such a result would be incorrectly interpreted as indicating a mortality benefit associated with surveillance, whereas the correct interpretation would be simply that a negative surveillance endoscopic exam predicted a low risk of death.

In fact, the Medicare-based study observed a similar proportion of fatal cases and controls to have undergone endoscopy (prior to 6 months before the fatal case’s recurrence was diagnosed), evidence of the insensitivity of endoscopy in identifying recurrences of colorectal cancer that prove fatal.

Example 2

In a cohort of 1,846 women with stage I or II unilateral breast cancer, those who died of their disease within the first five years after diagnosis (n=178) were compared to 634 matched controls (who survived for at least as long after diagnosis as the woman who died) with regard to the receipt of mammographic surveillance for recurrent breast cancer (5). Ninety-six (54 percent) of the women who died as a result of breast cancer had received one or more exams, in contrast to 379 (60 percent) of controls. However, the relative case-control difference grew with increasing number of surveillance exams, such that the odds ratios associated with receipt of three exams (0.4) and four or more (0.1) were quite low. Though the authors interpreted the results as being (at least to some degree) in “support [of] the hypothesis that post-therapy surveillance reduces the rate of breast cancer mortality”, a large mortality reduction resulting from repeated surveillance mammography seems implausible, in that but a small minority of recurrences that proved fatal were the local ones that mammograms can detect. Indeed, the observed pattern of results from multiple exams plausibly would have emerged even had a positive surveillance mammogram failed to lead to any therapy at all, since only women who are found to be negative on a surveillance can go on to receive another round of surveillance some time later – a 100% sensitive surveillance modality performed during the DPP of a recurrence would invariably identify it, barring subsequent surveillance. (Though mammography by itself may well not be a sensitive means of identifying potentially-fatal recurrences, it may be accompanied by other screening modalities, such as bone scans and serum tumor markers, that are more sensitive.) By virtue of having been found to be free of breast cancer at the time of the first exam, such women will be at a lower risk of death from breast cancer than those who had not previously undergone surveillance. To overcome this form of “healthy screenee bias” (6), the analysis of case-control studies of the efficacy of screening for recurrence must do no more than compare fatal cases and controls for a history of one or more surveillance exams during the period prior to the first evidence of the recurrence that corresponds to the estimated DPP. (Given the uncertainty as to even the mean DPP for a particular test and form of recurrence, much less the variability of the DPP from case to case, it is judicious to do several analyses, each of which employs a different plausible estimate of the DPP.)

Conclusion

The goal of a case-control study of the efficacy of surveillance for cancer recurrence is to estimate the combined impact of the abilities of: a) the screening modalities to identify recurrences; and b) the interventions administered to successfully treat those recurrences that are found more effectively than if they had been found later. To accomplish this goal it is necessary to be cognizant of the difference in the temporal distribution of sensitive surveillance exams in fatal cases and controls that would exist even if no effective treatment existed, and to accommodate this difference in the analysis.