Observational methods to assess the effectiveness of screening colonoscopy in reducing right colon cancer mortality risk: SCOLAR

Abstract

Aims:

Screening colonoscopy's effectiveness in reducing risk of death from right colon cancers remains unclear. Methodological challenges of existing observational studies addressing this issue motivated the design of ‘Effectiveness of Screening for Colorectal Cancer in Average-Risk Adults (SCOLAR)’.

Methods:

SCOLAR is a nested case–control study based on two large integrated health systems. This affords access to a large, well-defined historical cohort linked to integrated data on cancer outcomes, patient eligibility, test indications and important confounders.

Results:

We found electronic data adequate for excluding ineligible patients (except family history), but not the detailed information needed for test indication assignment.

Conclusion:

The lessons of SCOLAR's design and implementation may be useful for future studies seeking to evaluate the effectiveness of screening tests in community settings.

Colonoscopy is the most commonly used screening test for colorectal cancer (CRC) in the USA [1]. Yet, unlike fecal occult blood testing (FOBT) [2,3] and flexible sigmoidoscopy [4,5], which are also recommended for screening [6], the effectiveness of colonoscopy in reducing CRC mortality risk for average-risk individuals is not currently supported by evidence from randomized controlled trials. Such trials of screening colonoscopy are underway, but their results will not be available until at least 2022–2025 [7,8]. Thus, in the foreseeable future, the evidence to support the use of screening colonoscopy for reducing the risk of CRC death must be derived from observational data.

An important advantage of colonoscopy over sigmoidoscopy is visualization of the right colon, from the cecum through splenic flexure. Right colon cancers now account for over 40% of CRC cases [9]. Although the current literature includes a number of observational studies assessing the association of colonoscopy use and risk of CRC diagnosis or death [10–19], few of those studies were focused specifically on the effectiveness of screening colonoscopy for prevention of death from right colon tumors (Table 1). The ongoing debate about colonoscopy as a means of decreasing right colon cancer incidence and mortality [20] reflects current uncertainty about its effectiveness and underscores the need for additional studies.

Table 1. . Summary of studies evaluating the association between screening colonoscopy and right colon cancer.

Study (year)	Population, location	Study design; comparison groups	Exposure definition(s)	Determined test indication	Outcome of interest; analysis	Measure of association, estimate (95% CI)	Ref.
Baxter et al. (2009)	Ontario health insurance plan beneficiaries, Canada	Record linkage-based case–control study; CRC deaths (n = 10,292) and CRC-free controls (n = 51,460) matched on age, sex, income and residence	Any colonoscopy performed >6 months prior to the ‘referent date’ (date of diagnosis)	No	Mortality, adjusted for matched variables and Charlson co-morbidity index	Odds ratio: 1.07 (0.94–1.21)	[12]
Singh et al. (2010)	Manitoba health insurance beneficiaries, Canada	Population-based retrospective cohort study using claims; persons who had a colonoscopy (n = 54,803) compared with the Manitoba population	Any colonoscopy - excluded patients who had CRC diagnosed <6 months after the colonoscopy date	No	Mortality, indirect rate standardization for age, sex, calendar year	Standardized mortality rate: 0.95 (0.77–1.71)	[17]
Baxter et al. (2012)	Medicare parts A and B beneficiaries, 70+ years of age, USA	SEER-Medicare linkage-based case–control study; CRC deaths (n = 9458) and CRC-free controls (n = 27,641) matched on age, sex, race and SEER registry	Any colonoscopy performed >6 months prior to the ‘referent date’	No	Mortality, adjusted for matched variables plus co-morbidities, zip code based SES, and rural-urban residence	Odds ratio: 0.58 (0.53–0.64)	[13]
Nishihara et al. (2013)	Female nurses and male health professionals, USA	Occupational prospective cohort study; no lower endoscopy (1,182,248 person years) vs screening colonoscopy (357,008 person years)	Self-reported screening colonoscopy assessed every 2 years	No	Mortality; adjusted for BMI, smoking, FHx of CRC, PE, intakes of red meat, alcohol, folate calcium and total calories, and use of aspirin, NSAIDs and statins	Hazard (or risk) ratio: 0.18 (0.10–0.31)	[11]
Brenner et al. (2013)	Residents of Rhine Neckar region, Germany	Population-based case–control study; incident CRC cases (n = 2516) and controls (n = 2284) selected from population registries	Self-reported screening colonoscopy	No	Incidence; adjusted for age, sex, county of residence, education, FHx of CRC, smoking, BMI, use of BMI and HRT, and history of general health screening	Odds ratio: 0.22 (0.14–0.33)	[19]
Doubeni et al. (2013)	Members of four US-managed care organizations, USA	Case–control study nested in a historic cohort of health plan enrollees; incident CRC cases (n = 471) and CRC-free controls (n = 509) matched on age, sex, health plan and enrollment duration	Screening colonoscopy documented in medical records	Yes	Incidence; adjusted for matched variables plus census block SES, number of preventive health visits, FHx of CRC, and co-morbidities at baseline	Odds ratio: 0.36 (0.16–0.70)	[18]

CRC: Colorectal cancer; FHx: Family history; HRT Hormone replacement therapy; PE: Physical activity; SEER: Surveillance epidemiology and end results; SES: Socioeconomic status.

The existing gaps in the literature motivated the design of an observational study, ‘Effectiveness of Screening for Colorectal Cancer in Average-Risk Adults (SCOLAR)’. The present paper describes the main elements of the SCOLAR study design, data collection and lessons learned during its implementation. Prior to discussing methodological issues in the context of SCOLAR's design and implementation, we review the practical issues facing observational studies of screening colonoscopy, in general, and various approaches to address them.

Methodological challenges facing studies of screening colonoscopy

Methodological difficulties that may affect observational studies assessing the effectiveness of screening colonoscopy fall into five categories: having a sample that is large enough to detect meaningful effect sizes; distinguishing between screening and other indications for colonoscopy; evaluating procedure quality; accurate outcome ascertainment and adequately considering causes of confounding. These challenges are of particular interest for CRC because of the extended period of observation (10 years or longer) and large sample of patients needed to ascertain the potential benefits of screening. CRC is a leading cause of cancer deaths, but is relatively uncommon in the population as a whole and has a long precancerous phase [21,22]. These considerations underscore the need for studies with detailed exposure-outcome information, which will be difficult to obtain from administrative health plan or survey data.

Colonoscopy is a key element of the diagnostic workup for suspected CRC, either following a positive screening test (e.g., FOBT or fecal immunochemical test), or prompted by gastrointestinal symptoms and signs. Diagnostic procedures are not easily discernible from those performed for screening in asymptomatic average-risk persons in administrative or claims data. In an attempt to overcome uncertainties about test indication, some studies [12–13,17] excluded all procedures performed within a relatively short (e.g., 6 month) interval prior to CRC diagnosis in hopes of eliminating patients who had diagnostic colonoscopy. This approach, however, presents its own set of challenges as the majority of screening procedures performed in case patients may be done close to the diagnosis date, whereas examinations in cancer-free patients may be more uniformly distributed during a typical 10-year observation window. Thus, an appreciable bias away from the null may result when tests performed close to diagnosis are systematically excluded [23,24]. This bias may be magnified in the extant literature because screening colonoscopy became relatively more common only in recent years. Another option is to ascertain screening colonoscopy from surveys or questionnaires, as was done in a recent analysis from the Health Professionals Follow-up Study and the Nurses’ Health Study [11]. However, responders may not accurately distinguish screening from diagnostic examinations, leading to potential misclassification, and patients may not accurately distinguish between sigmoidoscopy and colonoscopy [25].

The quality of colonoscopy, conventionally measured based on the completeness and thoroughness of the examination, [26,27], may modify the effectiveness of screening colonoscopy [28,29]. When details of a procedure are not available, some studies consider specialty of the colonoscopist [13,17], a presumed surrogate for quality. However, quality measures should reflect actual performance for identifying histologically confirmed neoplasms (or adenoma detection rate) [28,29] and not merely training or experience.

Although misclassification of exposure is the primary concern in studies of screening colonoscopy and CRC mortality, outcome ascertainment may also be biased [30]. This includes misclassification of the underlying cause of death, which may occur differentially in right colon and left colon/rectal cancers [31].

People who undergo screening in community settings may be either at lower CRC risk than unscreened individuals because of healthier lifestyles [32,33], or at higher risk because of a strong family history of CRC. Differences in the propensity to receive colonoscopy are difficult to disentangle in observational studies and may require careful adjustment for confounders, but data on health-seeking behaviors that are also related to CRC risk are usually not available. Most studies control only for basic demographic characteristics such as age and sex. For example, only four studies assessing the effectiveness of colonoscopy adjusted for family history of CRC [11,18–19], and only one [19] controlled for hormone replacement therapy, a CRC risk factor and a potential marker of healthcare utilization.

Methods of the SCOLAR study

Study goals, design & setting

SCOLAR was initiated in September 2009 with the primary goal of assessing whether the use of colonoscopy to screen asymptomatic persons for CRC is effective in preventing CRC deaths, particularly deaths from cancer in the right colon. The institutional review boards of the University of Pennsylvania (coordinating center) and the collaborating sites approved the study protocol.

SCOLAR is a case–control study nested in two historical cohorts at Kaiser Permanente Northern and Southern California. The two health plans participate in the HMO Cancer Research Network [34], have used electronic medical records systems since at least 2004, and have electronic healthcare utilization and administrative data dating back to 1995 or earlier. They share similarly structured databases within a virtual data warehouse, which has identical variable names, formats and specifications, allowing the use of centrally generated or distributed informatics tools to extract data at each site including covariate information. Linkages to tumor registries, vital status records and US Census Bureau data make it possible to identify the eligible population and determine outcomes such as cancer diagnosis and death. They also allow for construction of historical cohorts, and tracking of the enrollment and healthcare utilization histories of members over extended periods of time.

Study population & sample selection

We used a dynamic or open population approach to select study patients [35]. Our design required patients who were at average risk for CRC and had long-standing enrollment in the health plans. Thus, we required patients to have on the reference date (defined below): a minimum of 5 years of prior enrollment in the participating health plan; no previous history of CRC, other gastrointestinal cancers, or partial or total colectomy for any reason; no documented diagnosis of inflammatory bowel disease and no documented strong family history of CRC. We defined a strong family history as having a CRC-associated syndrome such as familial adenomatous polyposis, at least one first-degree relative diagnosed with CRC before the age of 50, two or more biological relatives diagnosed with CRC at any age [36,37]. As these risk factors are more likely to be ascertained in patients with CRC around the time of diagnosis than in disease-free (particularly unscreened) individuals, the exclusion criteria were not considered if they were only documented in the 30-day period prior to the reference date.

Cases are health plan members who died from invasive colorectal adenocarcinoma between 2006 and 2012; and were 55–90 years old on the date of death. We only considered adenocarcinomas, which represent approximately 90–95% of all CRCs, because these tumors are believed to follow the adenoma-carcinoma sequence and are potentially preventable through screening [38]. The diagnosis date of case patients is set as the reference date for determining study eligibility and for matching cases to controls.

Matching of cases & controls

The matching process identified eight randomly selected candidate controls for each case on the assumption that at least one quarter of those would be eligible for the study after examination of medical records. The two closest eligible matched controls for each case were then selected for detailed medical chart audit. The existence of a very large population in the two integrated systems makes it possible to match patients on several variables within a relatively narrow range.

We selected two eligible controls for each case individually matched on study site, sex, birth date (±1 calendar year), years of enrollment (±1 year) in the health plan and geographic region within plans. For instance, a 57-year-old case man with 7 years of health plan enrollment and receiving care from a particular medical center in Northern California is matched to male control patients who are between 56 and 58 years of age, have had between 6 and 8 years of enrollment and are also receiving care from that medical center. We use incidence density-based matching with replacement, which means that a control patient may be selected for more than one case and can also become a case patient if the individual dies from CRC later in the study [35].

Data collection & integration

A key feature of SCOLAR's design is the integration of information from multiple complementary sources including electronic medical records, administrative data, and tumor and vital status registries (Figure 1). The information types and sources are summarized in Table 2. These data are necessary for selection of study patients and for accurate measurement of exposure and outcomes. The main data collection tools, a custom-built electronic chart audit form, were tested previously in a similarly designed smaller study at other HMO Cancer Research Network sites [39]. The data collection protocol for the current study underwent a number of modifications based on the experience with that earlier study.

Figure 1. . Overview of data collection process.

^†Fecal occult blood tests include guaiac fecal occult blood tests and fecal immunochemical tests.

Table 2. . Study parameters and data sources.

Data elements	Data sources
Demographic characteristics: age, sex and race/ethnicity, region, enrollment duration	Tumor registry, administrative (e.g., enrollment) data, and both paper and electronic medical records
Risk factors: personal/family history of CRC, IBD, HNPCC, FAP, colectomy	Medical records
Health history (at reference date and within 2 years): height/weight, medications, family history, co-morbidities	Electronic data on care utilization (diagnoses, procedures, laboratory results), pharmacy files, medical records
Cancer diagnosis: stage, location, histology	SEER registry
Vital status: date and cause of death	Mortality files, NDI, SSDMF, death certificates
Fecal occult blood testing: number of fecal blood tests in documented in the records; date ordered, collected or performed; reason for test; result of test	Electronic data on care utilization (diagnoses, procedures, laboratory results), and medical records
Procedures: number of tests, types and dates of tests, reasons for tests (e.g., screening, positive FOBT, symptoms), complications of tests (e.g., perforations or major bleeding)	Electronic data on care utilization and medical records
Provider characteristics: specialty, training, rate of complete colonoscopies, rate of polyp and adenoma detection	Electronic data on care utilization and medical records
Quality of colonoscopy: completeness to the cecum, total duration of test and withdrawal time	Electronic data on care utilization and medical records
Polyps or lesions found: count and location, size and shape, pathologic features	Electronic data on care utilization and medical records

CRC: Colorectal cancer; FAP: Familial adenomatous polyposis; FOBT: Fecal occult blood test; HNPCC: Hereditary nonpolyposis colorectal cancer or Lynch syndrome; IBD: Inflammatory bowel disease; NDI: National Death Index; SEER: Surveillance epidemiology and end results; SSDMF: Social Security Death Master File.

A computer program automatically excludes those with documented prior history of a gastrointestinal cancer, colectomy or inflammatory bowel disease. Patients with a family history were not excluded by the program but flagged for comparison with chart audit data. For all selected cases and controls, the program extracts data elements reflecting healthcare utilization such as dates and types of clinical visits, evidence of high-risk conditions, dates and results of CRC-related laboratory tests including iron studies and FOBT, and the dates of colonoscopy, sigmoidoscopy, barium enema and computed tomographic colonography (Table 2). This information is used to prepopulate (or preload) the electronic chart audit form with patient demographic information, healthcare utilization history, and the dates and types of CRC tests to facilitate chart review (Figure 1) [39]. This tool guides searching the medical records, standardizes chart abstraction across sites and enhances the accuracy of data collection.

Distinguishing screening from diagnostic colonoscopies

We determined exposures to and reasons for colonoscopies that were performed during the 10-year period prior to the reference date. We expanded the look-back interval to 15 years when necessary to obtain information about the initiating test for subsequent surveillance examinations to reconstruct a full history of testing.

We obtained information on reasons for colonoscopy from three sources: progress notes, referral notes and procedure reports. Sometimes these sources agreed with each other and indication could be classified with a simple algorithm. However, stated reasons for colonoscopy occasionally differed among the three sources. Some recorded reasons for the examination that are common in the adult population (e.g., constipation or abdominal fullness) may be related to CRC only if they are severe and sudden in onset [40,41]. When a clear determination of the reason for colonoscopy could not be made directly from the medical records or coded data, we used a panel of experts to adjudicate indications as has been described previously [39].

Assessment of procedure quality

We collected information on indicators of colonoscopy quality (adequacy of bowel preparation, completeness of the examination to the cecum and time spent inspecting the colon), documented findings, follow-up recommendations and specialty of the endoscopist. We have access to information on provider-level quality measures from linked electronic data [28].

Outcome ascertainment

The primary outcome for the study is death from invasive colorectal adenocarcinoma. Data on the underlying cause of death on decedents are obtained primarily from local and state vital status files, and in some instances, the National Death Index, as well as from cancer registries. The lag for death data obtained from the local and state sources is 6 months to 1 year, but is longer for the national sources. Because of this time delay in the availability of reliable death data, we performed a pilot study to manually review death certificates assessing the need for and feasibility of using this approach for the full study sample. We selected decedents for cause of death reviews based on documented CRC diagnosis, but with unknown cause of death or with CRC listed as a contributing cause.

Data on confounders

A number of factors such as demographic characteristics, medical and family history, socioeconomic status, prescription and over-the-counter medications, and various health behaviors, potentially contribute to both CRC risk [42–44]; and receipt of screening [32,45]. Some of these factors were available and taken into account in our design and analysis, but some (such as physical activity, diet, alcohol consumption and over-the-counter medications) were not (Figure 2).

Figure 2. . Flow of data elements collected on the study.

^†The additional data extracted were from paper and electronic medical records, and from electronic databases.

^‡Fecal occult blood tests include guaiac fecal occult blood tests and fecal immunochemical tests.

SES: Socioeconomic status indicator.

Sample size & power considerations

There were important epidemiological and methodological considerations for planning a sufficiently powered study. According to data from the Surveillance Epidemiology and End Results Program of the National Cancer Institute, relatively few adults in the general population (about 42.4 per 100,000 per year) get CRC and even fewer (about 15.5 per 100,000 per year) die from the disease [22]. In addition, because of the often lengthy detectable preclinical phase of CRC and precursor adenomas, it is necessary to ascertain screening histories for a considerable period of time prior to the reference date. This requires the ability to observe patients and their healthcare utilization histories over an extended time period, which we considered to be at least 10 years, consistent with current guidelines.

We address the large sample size requirement by basing the study in two established, integrated health systems with a combined total population in any year of about 6 million members, approximately one of every 50 persons in the USA. Statistical power is also increased through the use of more than one control per case, although this also increased resource expenditures.

We estimated a priori that a sample of 3600 patients would provide statistical power of 80% or higher, sufficient to detect a 50% reduction in all CRC mortality with a two-sided alpha error of 0.05. Based on the 2002–2011 Surveillance Epidemiology and End Results data [22] right colon tumors constitute about 40% of all CRC cases, and are characterized by lower survival. If 40% of deaths (n = 480) are attributable to right colon cancers, the study will have an 80% power to detect statistically significant odds ratios in the 0.3–0.4 range. These estimates are based on the assumption that the proportion of the study population receiving a screening colonoscopy is between 5 and 10%

Preliminary results

We identified a total of 5567 patients with approximately 1:2 case–control matching (n = 1831 and 3736, respectively). Of those, 205 were ineligible because of a missing diagnosis date, death prior to age 55 years or other exclusion criteria. Among ineligible patients, 36 were excluded at medical records review because of documented history of colectomy, two for inflammatory bowel disease and 17 for prior CRC history. A total of 458 patients were identified electronically as having a family history, of whom, 102 were confirmed to have a strong family history of CRC on audit. We also found that some details of the family history such as the age at diagnosis were not documented consistently in the medical records. The paper records of 24 patients had been either destroyed or lost. We also concluded that the manual review of death certificates did not provide additional information on the causes of death.

Of the 5567 patients, 4387 (78.8%) had at least one test during the follow-up period, (1610 cases and 2734 controls). A total of 2193 colonoscopies, 2674 sigmoidoscopies and 5995 stool-based tests were observed. Fifty-seven colonoscopies identified in electronic data were not found at chart audit, or had a different procedure date and 878 additional colonoscopies found at audit were not identified from the electronic data. Among cases, colonoscopies completed within 1 month of diagnosis constituted about 84% (n = 1077) of all procedures. The corresponding percentage of colonoscopies performed within a month of the reference date among controls was only 3% (n = 25). The remaining colonoscopies among controls were relatively evenly distributed in the 10-year period.

Information on procedure indication was missing in 38 colonoscopies and 28 sigmoidoscopies. Indication for stool-based tests was documented as diagnostic in 70 tests and the indication for the remainder was screening or unknown. Confounder data, especially for BMI, were incomplete prior to 2004 because the electronic medical record was not in routine use.

Discussion

We described the design and implementation of a population-based observational study conducted to examine the effectiveness of screening colonoscopy as a means of reducing the risk of death from CRC, particularly for cancers in the right colon. The key methodological features of our study are the integration of healthcare data with other complementary sources, and the detailed collection of information from paper and electronic medical records to help distinguish diagnostic from screening colonoscopies, which cannot be done from administrative or billing codes alone.

The conduct of this study had several lessons. First, medical audits confirmed that administrative or billing codes adequately identify eligible study subjects except for family history. Second, some of the tests were not identified using electronic methods alone; some had alternate examination dates or were only present in the chart and not captured with coded data. Third, the vast majority of colonoscopies in cases (84%) in contrast to controls (3%) occurred within 1 month of the reference date. Thus, excluding tests performed within 6 months prior to the reference date, as has been done previously, may result in systematic error favoring screening [23–24,46]. Fourth, a small portion of paper charts were lost or missing during the audit period of this study, but with the move to electronic medical record systems, this concern may prove to be less relevant in the future. Fifth, even with a detailed chart review, the information about the reason for colonoscopy was sometimes missing, and the indication for some examinations remained unclear. In some instances, the indication for a particular procedure could only be inferred though expert committee adjudication, which has proven to be a critical additional step. In other instances, uncertainties about the indication remained even after careful review. Sixth, the reasons for stool-based tests were frequently not documented in the chart. The health systems in this study routinely used stool-based testing for screening. Therefore, a screening indication can be inferred in the absence of a documented diagnostic indication. Finally, manual review of death certificates did not yield additional information on the causes of death than was available from the electronic vital status files. Prior research shows that the information on death certificates, which is the source for our mortality files, is adequate for ascertaining the underlying cause of death [47].

Confounders such as age, sex, hormone therapy, race and family history of CRC should be considered because controlling for these factors may substantially affect the risk ratio estimates; these factors can be ascertained in the current study. Our previous work demonstrated that family history of CRC and hormone therapy (along with age, sex and socioeconomic status measures) act as important confounders of the association between screening colonoscopy and CRC mortality [48]. Many patients with family history identified in the electronic data or on chart review were found to be eligible or lacked sufficient information to exclude them from the study. However, our prior studies have found relatively little bias from including the small number of patients with confirmed strong family history in the analyses [39].

Complete data on over-the-counter medications and lifestyle (e.g., diet and physical activity) are not available in the medical records and cannot be ascertained without a separate patient survey; however, our studies have found that these factors are not significant confounders [48]. The current study's design and implementation demonstrates several factors that can inform other studies of cancer screening. First, even for relatively common cancers, such as colorectal cancer, adequate numbers of outcomes such as cancer deaths or variation by site, such as deaths due to right colon cancers, can be feasibly achieved using extremely large, well-defined populations with longitudinal follow-up. In practical terms, at least in the US, this can only be done by basing the study in very large cohorts that include high quality validated survey data or from large integrated health systems with millions of members and comprehensive electronic medical records. Such health systems also provide data uncommon in many administrative datasets, such as vital status, reports that permit more accurate assignment of indication, data on major confounders, prior screening history and detailed cancer diagnosis data. Second, our pilot testing of death certificate reviews demonstrated no additional benefit to manual death certificate review, beyond the electronic data. Third, the assignment of test indication, a crucial element of screening studies, requires expert adjudication in a moderate proportion of subjects.

Conclusion

There is currently limited evidence for the effectiveness of screening colonoscopy in reducing the risk of death from CRC, particularly for cancers in the right colon. Observational data are important for evaluating the effectiveness of CRC in community settings. Methodological challenges, particularly in distinguishing screening from diagnostic indication, affect the validity of current observational studies. SCOLAR presents a workable model for identifying and, to the extent possible, overcoming methodological issues affecting observational studies of screening colonoscopy and can serve as a model for other screening studies. Our experiences highlight the challenges of determining the indication for CRC testing in observational data, which is resource intensive. Thus, determining test indication from electronic data algorithms will greatly facilitate research of CRC screening effectiveness in community settings, but will need further research.

Executive summary.

Background

• Colonoscopy is the most commonly used colorectal cancer (CRC) screening test in the USA, but has no strong evidence on its effectiveness in reducing the risk of right colon cancer deaths.

• Observational research is critical for assessing screening colonoscopy effectiveness in community settings, but faces many methodological challenges including correctly identifying screening tests.

Methods

• Effectiveness of Screening for Colorectal Cancer in Average-Risk Adults (SCOLAR) is a case–control study nested in historical cohorts at Kaiser Permanente Northern and Southern California.

• Patients were members in either health system, with cases defined as 55–90-year-olds who died from CRC between 2006 and 2012.

• Detailed data on exposure, outcomes, and covariates were collected from complementary sources (medical records, electronic databases, and tumor and vital status registries).

Preliminary results

• In total, 5567 patients with approximately 1:2 case–control matching (n = 1831 and 3736, respectively) were studied, of whom 205 were found to be ineligible by medical records reviews.

• Colonoscopies received by case patients clustered around the date of diagnosis, but were more evenly distributed over a 10-year study period in controls.

• Medical audits confirmed that administrative codes adequately identify eligible study subjects except for family history, and some tests were not identified using codes alone.

Discussion & conclusion

• The integration of health care data with other complementary sources, and the detailed collection of information from electronic medical records is needed to distinguish diagnostic from screening colonoscopies, and is key for observational studies of CRC screening effectiveness.

• Even with a detailed chart review, the information about the reason for colonoscopy was sometimes missing, and the indication for some examinations remained unclear. The electronic vital status files from the health systems were adequate for selecting eligible cases.

• Determining test indication is necessary for valid observational studies of cancer screening effectiveness, but it is challenging and resource intensive.