Comparing the effectiveness of competing tests for reducing colorectal cancer mortality: a network meta-analysis

Abstract

Background

Comparative effectiveness data pertaining to competing colorectal cancer (CRC) screening tests do not exist but are necessary to guide clinical decision-making and policy.

Objective

To perform a comparative synthesis of clinical outcomes studies evaluating the effects of competing tests on CRC-related mortality.

Design

Traditional and network meta-analyses.

Interventions and outcome measurement

Two reviewers identified studies evaluating the effect of guaiac-based fecal occult blood testing (gFOBT), flexible sigmoidoscopy (FS), or colonoscopy on CRC-related mortality. Traditional meta-analysis was performed to produce pooled estimates of the effect of each modality on CRC mortality. Bayesian network meta-analysis (NMA) was performed to indirectly compare the effectiveness of screening modalities. Multiple sensitivity analyses were performed.

Results

Traditional meta-analysis revealed that, compared with no intervention, colonoscopy reduces CRC-related mortality by 57% (RR 0.43; 95% CI, 0.33-0.58) whereas FS reduces it by 40% (RR 0.60; 95% CI, 0.45-0.78) and gFOBT reduces it by 18% (RR 0.82; 95% CI, 0.76-0.88). NMA demonstrated non-significant trends favoring colonoscopy over FS (RR 0.71; 95% CI, 0.45-1.11) and FS over gFOBT (RR 0.74; 95% CI, 0.51-1.09) for reducing CRC deaths. NMA-based simulations, however, revealed that colonoscopy has a 94% probability of being the most effective test for reducing CRC mortality and a 99% probability of being most effective when the analysis is restricted to screening studies.

Limitations

Randomized trials and observational studies were combined within the same analysis.

Conclusions

Clinical outcomes studies demonstrate that gFOBT, FS, and colonoscopy are all effective in reducing CRC-related mortality. Network meta-analysis suggests that colonoscopy is the most effective test.

Manuscript

Colorectal cancer (CRC) is a leading worldwide cause of cancer-related deaths.^1,2 Although screening for CRC reduces the incidence and mortality of this malignancy,^3,4 clinical outcomes studies directly comparing the effectiveness of competing screening tests are not available to guide clinical decision-making or policy.

Two randomized trials comparing colonoscopy with fecal immunohistochemical testing (FIT) are ongoing, however results may not be available for another decade or longer.^5,6 There are currently no ongoing registered clinical outcomes trials comparing colonoscopy with flexible sigmoidoscopy (FS), computed tomography colonography (CTC), or stool DNA testing. As a result, the optimal test remains uncertain and national screening strategies vary. Fecal occult blood testing (FOBT) is used in most European countries,⁷ Canada,⁸ and Japan.⁹ In contrast, colonoscopy – the most invasive and costly modality – is preferred in Germany, Poland, and the United States^7,10,11 despite the absence of comparative effectiveness data demonstrating its superiority.

Colonoscopy may indeed be the most effective screening modality because it provides structural evaluation of the entire colon, detects both pre-cancerous lesions and early prevalent cancers, and allows real-time polyp removal (thereby eliminating the risk of missing the lesion at follow-up examination). However, evidence of the comparative advantage of colonoscopy is necessary to justify its continued growth in this era of increasing screening acceptance¹² but limited endoscopic capacity¹³ and rising healthcare expenditures.¹⁴

Because substantial clinical outcomes data are available for each test, and validated methodologies exist to indirectly compare the effectiveness of modalities, we used traditional and network meta-analysis to perform a comprehensive comparative appraisal of the effects of competing screening tests on CRC-related mortality. The results of this analysis may inform additional research in this field and supplement previously published decision analyses^15,16 in guiding clinical decision-making and screening policy.

Methods

Data sources and search

The study was conducted in accordance with the PRISMA and MOOSE statements.^17,18 A research librarian designed and conducted a computer-assisted search using the National Library of Medicine's interface to PubMed/MEDLINE and Embase to identify potentially relevant papers. A search of human studies in these databases from inception through 20 April 2014 was performed using controlled vocabulary descriptors (Medical Subject Headings and Emtree) and keywords to represent the concepts of colorectal cancer, colonic or rectal cancer, screening, and mortality. Results from this base search were combined with descriptors and keywords for various diagnostic procedures or screening methods including colonoscopy, colonography, sigmoidoscopy, endoscopy, fecal occult blood testing (FOBT), fecal immunohistochemical testing, and stool DNA testing.

The search was augmented by manual searches of reference lists from potentially relevant papers to identify any additional studies that may have been missed using the computer-assisted strategy. Additionally, all available guidelines, systematic reviews, and meta-analyses pertaining to CRC screening or individual screening modalities published after 2007 were identified through a manual search of the PubMed.gov database. These documents and their reference lists were also reviewed for additional potentially relevant studies. The search was not limited by language.

Study selection

Two investigators (BJE, AKW) independently reviewed the titles of all identified citations to generate a list of potentially relevant articles. Abstract and brief manuscript review of articles with potentially relevant titles was then independently performed to select studies that may be appropriate for our analysis. These studies were then reviewed in depth and the following eligibility criteria applied: (1) published manuscripts that examine the effect of colonoscopy, flexible sigmoidoscopy, stool-based CRC screening tests, CT colonography, or some combination thereof on the mortality of colorectal cancer; (2) studies that evaluate clinical outcomes in humans (not test performance characteristics); (3) studies in which data or patients are not duplicated in another manuscript – for randomized controlled trials (RCTs) and cohort studies that were longitudinally updated, only the most recent report was included; (4) studies with at least 5 years mean follow-up (for trials and cohort studies); and (5) studies in which the number of events and total number of subjects in each study group were reported. Papers reporting the effects of rigid sigmoidoscopy and barium enema were excluded because these are no longer accepted screening modalities.

Data Extraction

The following data were abstracted from each study in duplicate (BJE, AGS) and independent fashion: first author, year of publication, country in which the study was conducted, screening modality or modalities evaluated, study methodology (trial, cohort study, case-control study, prospective vs. retrospective), whether or not the study focused primarily on screening, follow-up duration, the number of events and total number of subjects in the intervention and control groups (using the intention-to-treat principle for trials), and the reported (adjusted) summary estimate (with confidence limits) of the intervention's effect on overall CRC mortality or deaths related to CRC in the proximal or distal colon. Discrepancies in data extraction were resolved by consensus.

Quality Assessment

Two investigators (AGD, DAS) critically appraised and quality-rated all eligible studies using 2 instruments. Randomized controlled trials were assessed by criteria set forth by the Evidence-Based Gastroenterology Steering Group (EBGSG).¹⁹ These criteria were: (1) concealed random allocation; (2) blinding of patients and caregivers; (3) equal use of co-interventions for the treatment and placebo groups; (4) complete follow-up of study patients; and (5) use of an intention-to-treat analysis. Observational studies were evaluated using the Ottawa-Newcastle scale (ONS).²⁰ This instrument rates observational studies on a nine-point scale based upon the appropriateness of the study sample, comparability of study groups, and adequacy of assessing the exposure or outcome. Discrepancies in quality assessment were also resolved by consensus.

Data synthesis and analysis

The outcome analyzed was mortality due to CRC. All primary analyses were performed using crude event rates reported in, or derived from each included study (ie; number of mortality events and total number of subjects in the intervention and control groups). Pairwise comparisons of each screening modality vs. no active intervention were performed using traditional random-effects meta-analysis techniques in Stata 12.0 (StataCorp, College Station, TX). The Cochran Q test and I² inconsistency statistic were used to assess for statistical heterogeneity between trials. When heterogeneity was present, meta-influence analysis and Galbraith plot assessment were performed to identify responsible outlier studies. Pooled relative risks (RRs) and their 95% confidence intervals (95% CIs) were estimated for gFOBT, FS, and colonoscopy.

To compare testing modalities, a hierarchical Bayesian network meta-analysis (NMA) was performed with the GeMTC GUI statistical package.²¹ This form of meta-analysis generates estimates of effect sizes for all possible pairwise comparisons whether or not they have been evaluated in head-to-head trials. In particular, NMA allows indirect comparisons of colonoscopy vs. FS, colonoscopy vs. gFOBT, and FS vs. gFOBT based on each test's effect relative to a common comparator (no intervention). These comparisons form the basis for a rank probability analysis of competing modalities, which uses simulations to determine the probability of any particular intervention being most effective. Posterior distributions were estimated using Markov chain Monte Carlo simulations. A non-informative uniform prior distribution of effect sizes and precision was used. For each analysis, a series of 50,000 burn-in simulations was performed to allow convergence and an additional 50,000 simulations were performed to produce the probability statements. Convergence of iterations was evaluated using Gelman-Rubin-Brooke statistic. Publication bias was assessed using funnel plot asymmetry testing.

Sensitivity analyses

To assess the robustness of the results, several separate predefined traditional meta-analyses were repeated after: (1) restricting to studies that concentrate primarily on CRC screening, (2) eliminating statistical heterogeneity by removing the minimum number of outlier studies for each screening modality, (3) restricting to studies that are prospective, (4) restricting to randomized controlled trials, (5) restricting to observational studies, (6) removing case-control studies, (7) restricting to studies with a low risk of bias based on an EBGSG score >2 (RCTs) or an ONS score ≥ 7 (observational studies), and (8) pooling the adjusted summary estimates and confidence limits reported in the studies (not the crude event rates used in the other analyses). In this last analysis, odds ratios were assumed to approximate relative risk for this low probability outcome.

In addition, network meta-analyses were repeated for screening studies only and after excluding the outlier studies responsible for statistical heterogeneity.

To estimate the magnitude of superiority of the best test, a threshold analysis was performed by varying the RR of the best test toward the null in order to define the point at which it no longer ranks #1 according to NMA-based rank probability analysis.

Studies reporting the effect of a screening test on the mortality of right or left-sided CRC were also pooled using traditional meta-analysis. Because crude event rates were not reported or calculable for these endpoints in many of the included studies, we used the reported summary estimates and confidence intervals for this analysis. Again, odds ratios were assumed to approximate relative risk for this low probability outcome.

Results

Literature search

A flow diagram depicting the search and selection process is provided in Figure 1. Initial searches of the Medline and Embase databases yielded 3704 citations. A manual search of the PubMed.gov database for pertinent systematic reviews, meta-analyses, and guidelines identified 19 summary documents (online supplement A), review of which yielded 112 additional citations. Title review of these 2 groups of citations yielded 257 unique potentially relevant articles. Abstract and/or brief manuscript review of these articles yielded 60 manuscripts appropriate for detailed evaluation. Thirty of these were included in the final analysis. The remaining 30 articles (online supplement B) were excluded because they did not meet eligibility criteria, detailed in the flow diagram. An article by Nishihara et al.²² was excluded from the primary analysis because the number of events and total subjects in each study group were not reported and could not be calculated from the original dataset. However the summary estimates reported in this study were included in a sensitivity analysis that pooled the adjusted summary estimates of included studies. There was 100% agreement between reviewers regarding final study selection.

Figure 1.

Flow diagram depicting the article search and selection process.

Characteristics of Included Studies

A summary of the clinical outcomes-based evidence supporting available CRC screening tests is presented in Table 1. The thirty studies meeting eligibility criteria are listed in online supplement C. Characteristics of the studies included in the meta-analysis are listed in Table 2. The component studies included a total of 2,957,945 subjects. No comparative effectiveness studies were identified; all component studies compared 1 or more screening test to no active intervention. Seventeen studies, comprising 1,230,485 subjects, evaluated guaiac-based fecal occult blood testing. Of these, 4 were RCTs (329,642 patients), 1 was a quasi-experimental study (91,199), 3 were cohort studies (803,245 patients), and 9 were case-control studies (6399 patients). Seven studies, comprising 436,916 subjects, evaluated FS. Of these, 4 were RCTs (414,966 subjects), 1 was a quasi-experimental study (799 patients), and 2 were case-control studies (21,151 patients). Eight studies, comprising 1,290,544 subjects, evaluated colonoscopy. Of these, 5 were cohort studies (1,170,804 patients) and 3 were case-control studies (119,740 subjects). No clinical outcomes studies evaluating CTC, FIT, or fecal DNA were identified. Twenty-six studies (87%) focused primarily on screening, whereas the remainder enrolled subjects with symptoms or known polyps, or did not specifically restrict the analysis to asymptomatic average-risk patients.

Table 1.

Summary of the clinical outcomes-based evidence for CRC screening tests.

	Observational studies			Randomized trials
	Number of studies	Participants	Magnitude of benefit	Number of trials	Participants	Magnitude of benefit
Colonoscopy	8	1,290,544	Highest	0	0	N/A
Sigmoidoscopy	3	21,950	High	4	414,966	High
Guaiac-FOBT	13	900,843	Moderate	4	329,642	Moderate
CTC	0	0	N/A	0	0	N/A
FIT	0	0	N/A	0	0	N/A
Stool DNA tests	0	0	N/A	0	0	N/A

FOBT - fecal occult blood testing; CTC – computed tomography colonography; DNA – deoxyribonucleic acid; N/A – not applicable.

Table 2.

Characteristics of included studies.

Study	Country	Modality	Design	Sample size	Follow-up (years)	Subject age	Screening (yes/no)	Quality score**
Jorgensen 2002	Denmark	FOBT	RCT	61,933	13	45-75	Yes	1/5
Lindholm 2008	Sweden	FOBT	RCT	68,308	9	60-64	Yes	3/5
Scholefield 2012	UK	FOBT	RCT	152,850	19.5	45-74	Yes	2/5
Shaukat 2013	USA	FOBT	RCT	46,551	30 years	50-80	Yes	2/5
Faivre 2004	France	FOBT	Quasi experiment	91,199	11	45-74	Yes	8/9
Lee 2007	S. Korea	FOBT	Cohort	42,150	13.1	40-59	Yes	7/9
Malila 2007	Finland	FOBT	Cohort	1785	19.9	50-63	Yes	5/9
Libby 2012	UK	FOBT	Cohort	759,310	3-10	50-69	Yes	7/9
Newcomb 1992*	USA	FOBT	Case-control	262	n/a	Not reported	Yes	7/9
Hiwatashi 1993	Japan	FOBT	Case-control	112	n/a	Not reported	Yes	8/9
Selby 1993	USA	FOBT	Case-control	1213	n/a	>50	Yes	8/9
Lazovich 1995	USA	FOBT	Case-control	693	n/a	40-84	Yes	9/9
Saito 1995	Japan	FOBT	Case-control	770	n/a	40-79	Yes	9/9
Zappa 1997	Italy	FOBT	Case-control	1236	n/a	>41	Yes	6/9
Bertario 1999	Italy	FOBT	Case-control	570	n/a	>40	Yes	7/9
Faivre 1999	France	FOBT	Case-control	890	n/a	45-80	Yes	7/9
Scheitel 1999	USA	FOBT	Case-control	653	n/a	>45	Yes	9/9
Hoff 2009	Norway	FS	RCT	55,736	6	55-64	Yes	3/5
Atkin 2010	UK	FS	RCT	170,038	11.2	55-64	Yes	4/5
Segnan 2011	Italy	FS	RCT	34,292	11.4	55-64	Yes	3/5
Schoen 2012	USA	FS	RCT	154,900	11.9	55-74	Yes	4/5
Thiis-Evensen 1999	Norway	FS	Quasi Experiment	799	13	50-99	Yes	5/9
Newcomb 1992*	USA	FS	Case-control	262	n/a	Not reported	Yes	7/9
Muller 1995*	USA	FS	Case-control	20,889	n/a	69 (mean)	No	8/9
Kahi 2009	USA	Colonoscopy	Cohort	715	8	50-86	Yes	8/9
Singh 2010	Canada	Colonoscopy	Cohort	54,803	5.7	50-80	Yes	7/9
Jacob 2012	Canada	Colonoscopy	Cohort	1,089,998	5 to 9	50-74	Yes	8/9
Manser 2012	Switzerland	Colonoscopy	Cohort	22,686	6	50-80	Yes	6/9
Zauber 2012	USA	Colonoscopy	Cohort	2602	15.8	62 (mean)	No	6/9
Muller 1995*	USA	Colonoscopy	Case-control	20,889	n/a	69 (mean)	No	8/9
Baxter 2009	Canada	Colonoscopy	Case-control	61,752	n/a	52-90	No	6/9
Baxter 2012	USA	Colonoscopy	Case-control	37,099	n/a	70-89	No	6/9

FOBT – fecal occult blood test; FS – flexible sigmoidoscopy

^*Studies evaluating more than 1 modality

^**RCTs rated on a 5-point scale; observational studies rated on a 9-point scale

Testing for heterogeneity between eligible studies

Pooled analysis of the effects of colonoscopy, FS, and gFOBT on the mortality of CRC demonstrated significant statistical heterogeneity among included studies for each modality (gFOBT I²=50.1%, p=0.009, τ²=0.010, FS I²=82.0%, p<0.001, τ²=0.091, and colonoscopy I²=94.8%, p<0.001, τ²=0.11). Meta-influence analysis and visual inspection of Galbraith plots revealed that the following 5 outlier studies were responsible for the statistical heterogeneity: Lee 2007 (gFOBT), Muller 1995 (FS), Baxter 2009 (colonoscopy), Singh 2010 (colonoscopy), and Baxter 2012 (colonoscopy). These studies were included in the base-case analysis but excluded in sensitivity analyses intended to eliminate statistical heterogeneity.

Meta-analysis results

Compared with no intervention, traditional meta-analysis revealed that gFOBT results in an 18% reduction in CRC mortality (RR 0.82; 95% CI, 0.76-0.88). FS reduces CRC mortality by 40% (RR 0.60; 95% CI, 0.45-0.78) and colonoscopy reduces CRC mortality by 57% (RR 0.43; 95% CI, 0.33-0.58) (Figure 2).

Figure 2.

Meta-analysis of the effect of screening tests compared with no intervention on CRC mortality.

Network meta-analysis revealed a non-statistically significant trend favoring colonoscopy over FS (RR 0.71; 95% CI, 0.45-1.11) and a non-statistically significant trend favoring FS over gFOBT (RR 0.74; 95% CI, 0.50-1.09) (Figure 3). Colonoscopy was approximately 50% more effective than gFOBT for reducing CRC mortality (RR 0.53; 95% CI, 0.36-0.76) (Figure 3). NMA-based rank probability analysis demonstrated that colonoscopy had a 94% probability of being the most effective test to reduce CRC mortality compared to FS (second rank probability of 87%) and gFOBT (third rank probability of 93%). In other words, a simulation-based analysis assimilating data from the evidence network revealed that colonoscopy was the most effective test in 94% of simulations (not the most effective test in only 6% of simulations), FS was the second most effective modality in 87% of simulations, and gFOBT was the third most effective test in 93% of simulations.

Figure 3.

Evidence network of the effect of screening options on CRC mortality. Traditional meta-analysis-generated direct comparisons are denoted by solid lines and NMA-generated indirect comparisons are denoted by dashed lines. Direction of the arrow denotes superiority.

Sensitivity analyses

Predefined sensitivity analyses using traditional meta-analysis did not significantly change the overall findings (Figure 4). Repeat NMA after restricting to screening studies revealed that colonoscopy may be 44% more effective than FS (RR 0.56; 95% CI, 0.32-0.94) and 51% more effective than gFOBT (RR 0.49; 95% CI, 0.30-0.76). Repeat NMA after removing outlier studies revealed that colonoscopy may be 68% more effective than FS (RR 0.32; 95% CrI, 0.23-0.45) and 72% more effective than gFOBT (RR 0.28; 95% CrI, 0.21-0.37). In these sensitivity analyses, rank probability testing demonstrated that colonoscopy had a 99% to 100% probability of being the most effective test for reducing CRC mortality.

Figure 4.

Sensitivity analyses for each screening modality compared with no intervention.

NMA-based threshold analysis revealed that colonoscopy would remain the best test (first rank probability 51%) even if its absolute effectiveness compared with no intervention were 16% lower than observed in the primary analysis.

Four gFOBT studies, 5 FS studies, and 5 colonoscopy studies specifically reported the impact of an intervention on left- and right-sided CRC mortality. Subgroup analysis of these studies revealed that: (1) gFOBT is equally effective in reducing left (RR 0.89; 95% CI, 0.77-1.01) and right-sided (RR 0.87; 95% CI, 0.75-0.99) CRC mortality, (2) FS is significantly effective in the left side of the colon (RR 0.41; 95% CI, 0.21-0.61) but does not appear to impact right-sided CRC mortality (RR 0.89; 95% CI, 0.69-1.09), and (3) colonoscopy significantly reduces CRC mortality in the left side of the colon (RR 0.27; 95% CI, 0.15-0.40) and demonstrates a strong trend toward benefit in the right side of the colon (RR 0.74; 95% CI, 0.47-1.02).

Publication bias

The funnel plot asymmetry test for publication bias was negative using both the Harbord test (p=0.615) and the Peters test (p=0.79).

Discussion

This pooled analysis of available clinical outcomes data demonstrates that, when compared with no intervention, colonoscopy reduces CRC mortality by 57% compared with a 40% reduction associated with FS and an 18% reduction associated with gFOBT. NMA-based indirect comparisons show that colonoscopy has a very high likelihood of being the best modality for reducing CRC-related deaths. These findings remained consistent in multiple sensitivity analyses.

To create the most comprehensive and robust evidence network, all available clinical outcomes studies were evaluated in this NMA, including 4 studies that did not focus primarily on screening. Thus, the results of our primary analysis must be extrapolated to a certain extent to the screening context. However, a sensitivity analysis restricted to the 26 screening studies did not substantively change the overall results and in fact increased the relative effectiveness of colonoscopy compared with the other modalities.

Current screening recommendations are based primarily on simulation-driven decision analyses which suggest that annual FOBT, FS every 5 years (with mid-interval FOBT), and colonoscopy every 10 years are equivalent in reducing CRC mortality.^15,16 These analyses however, which do not consider clinical outcomes data, assume 100% adherence to screening, follow-up testing, and surveillance of patients with adenomas – rates much higher than expected in usual clinical practice.^23,24 In contrast, the estimates of effectiveness for each screening modality produced by this meta-analysis are based on empiric investigational data and are more likely to reflect important real-world factors such as suboptimal participation. Because colonoscopy is less sensitive than FOBT and FS to variations in patient adherence¹⁵ and has better performance characteristics, the comparative superiority of colonoscopy demonstrated in this meta-analysis seems quite plausible.

Certainly other considerations such as cost, safety, acceptability, and availability are integral in determining the most appropriate screening strategy for any particular population. Although this NMA suggests that colonoscopy is the most effective test, it is also the most invasive of all screening modalities, requires a full bowel preparation and systemic sedation, and generally results in loss of wages or other productivity for the patient and escort on the day of the procedure. Further, some patients may favor other screening tests on the basis of psychosocial or cultural factors. Therefore, promoting colonoscopy as the preferred option in all contexts may be counterproductive, especially because offering patients a choice of CRC screening tests has been shown to increase adherence.²⁵

Pooled estimates from the limited number of included studies that evaluated the impact of screening interventions on CRC outcomes in the left and right sides of the colon are consistent with prior reports suggesting that colonoscopy is less effective in the proximal colon than it is distally.^26-28 This observation may be due to non-modifiable factors such as the biology of right-sided neoplasms (more aggressive progression),^29,30 however addressable deficiencies in colonoscopy performance, such as preparation quality in the proximal colon, the recognition of subtle right-sided polyps, and the completeness of polypectomy are probably relevant and merit continued attention^31,32. Additional research focused on increasing the detection of serrated lesions in the right side of the colon through education and training³³, advances in wide-field imaging,^34,35 or molecular targeting³⁶ may be of significant value.

The results of this study should be interpreted in the context of several important limitations. First, all colonoscopy studies included in this comparative analysis were observational in nature, potentially exaggerating the relative effectiveness of this modality. This may occur because of differential confounding in favor of colonoscopy, and because cohort and case control study results are based on 100% adherence in the intervention group, whereas intention-to-screen results of RCTs (representing the majority of included FS and gFOBT data) are compromised by non-adherence. Indeed, the analyses we performed that were restricted to RCTs of FOBT and FS demonstrated 3-12% lower risk reduction compared with overall meta-analysis results (which include observational studies). Therefore, colonoscopy RCTs may yield estimates of benefit that are more conservative than those observed in this study. However, the threshold analysis we performed revealed that colonoscopy could be up to 16% less effective than observed in this meta-analysis and still remain the most effective modality for reducing CRC-related mortality. Additionally, it is not clearly established that non-adherence to colonoscopy in clinical practice would be substantially higher than to FS (as evidenced by comparing the attendance rates in some published FS RCTs³⁷ and the first interim report of a large colonoscopy RCT⁵) or to stool-based testing which requires annual or biennial participation for greater than 2 decades and attendance of follow-up colonoscopies.

Along these lines, performing a fully comprehensive comparative appraisal of the CRC screening literature necessitated combining randomized trials and observational studies within the same meta-analysis. The validity of combining observational and experimental data in the same synthesis has been questioned, but cogent methodologic arguments in favor of this approach exist,³⁸ especially when RCT data alone are insufficient to answer the clinical question.^38,39 Furthermore, RCTs with long follow-up (such as CRC screening trials) demonstrate characteristics similar to observational studies because of post-randomization confounding,⁴⁰ providing support to the strategy of combining observational and experimental data in our analysis.

Second, the included observational studies, particularly the case-control studies, often adjusted for imbalances in patient characteristics that could affect the risk of CRC. This adjustment was lost in our primary analysis as we used crude event rates to calculate relative risks, potentially biasing in favor of one screening intervention over the others. However, a sensitivity analysis pooling the reported adjusted summary estimates and confidence limits (rather than crude event rates) did not differ from the primary analysis.

Finally, because clinical outcomes data exist for gFOBT only, this NMA did not include studies of the more sensitive immunohistochemical⁴¹ or DNA tests,⁴² potentially biasing the results against stool-based screening. Based on favorable performance characteristics, most formal screening programs have abandoned gFOBT in favor of FIT, however understanding the comparative effectiveness of gFOBT remains of clinical importance because this test is still used routinely or sporadically in some countries, including the UK, Canada, and the USA. Furthermore, gFOBT studies represent the foundation for the evidence-base in support of stool testing.

Although this comparative appraisal suggests that colonoscopy is the most effective test, it also highlights the phenomenon that colonoscopy has gained widespread acceptance in several parts of the world without the support of a single randomized trial. This intensifies the importance of ongoing comparative effectiveness trials of colonoscopy vs. FIT,^5,6,43 but also emphasizes the need for trials comparing colonoscopy to FS, which has developed a very strong evidence-base. Although awaiting RCT results, this network meta-analysis along with previously published cost-effectiveness analyses^44,45 may support certain countries’ commitment to colonoscopy as the primary CRC screening strategy. Other public health organizations may use the results of this NMA as one factor in determining the most appropriate screening test for their patient population.

In conclusion, FOBT, FS, and colonoscopy are all effective in preventing CRC deaths. Within the limitations of comparing the higher-quality evidence supporting FOBT and FS with the observational data on colonoscopy, NMA suggests that colonoscopy is the most effective test for preventing CRC deaths.

Acronyms

CRC

colorectal cancer

FIT

fecal immunohistochemical testing

FS

flexible sigmoidoscopy

CTC

computed tomography colonography

FOBT

fecal occult blood testing

NMA

network meta-analysis

RCTs

randomized controlled trials

EBGSG

Evidence-Based Gastroenterology Steering Group

ONS

Ottawa-Newcastle scale

gFOBT

guaiac-based fecal occult blood testing

RR

relative risk