A prognostic model for advanced colorectal neoplasia recurrence

Abstract

Purpose

Following colonoscopic polypectomy, US Multisociety Task Force (USMSTF) guidelines stratify patients based on risk of subsequent advanced neoplasia (AN) using number, size, and histology of resected polyps, but have only moderate sensitivity and specificity. We hypothesized that a state-of-the-art statistical prediction model might improve identification of patients at high risk of future AN and address these challenges.

Methods

Data were pooled from seven prospective studies which had follow-up ascertainment of metachronous AN within 3–5 years of baseline polypectomy (combined n = 8,228). Pooled data were randomly split into training (n = 5,483) and validation (n = 2,745) sets. A prognostic model was developed using best practices. Two risk cut-points were identified in the training data which achieved a 10 percentage point improvement in sensitivity and specificity, respectively, over current USMSTF guidelines. Clinical benefit of USMSTF versus model-based risk stratification was then estimated using validation data.

Results

The final model included polyp location, prior polyp history, patient age, and number, size and histology of resected polyps. The first risk cut-point improved sensitivity but with loss of specificity. The second risk cut-point improved specificity without loss of sensitivity (specificity 46.2 % model vs. 42.1 % guidelines, p < 0.001; sensitivity 75.8 % model vs. 74.0 % guidelines, p = 0.64). Estimated AUC was 65 % (95 % CI: 62–69 %).

Conclusion

This model-based approach allows flexibility in trading sensitivity and specificity, which can optimize colonoscopy over- versus underuse rates. Only modest improvements in prognostic power are possible using currently available clinical data. Research considering additional factors such as adenoma detection rate for risk prediction appears warranted.

Introduction

Colorectal cancer (CRC) is a leading cause of cancer death worldwide. For example, annually in the USA, 130,000 individuals develop CRC, and about 50,000 die of the disease annually [1]. CRC can be prevented by the identification and removal of colorectal polyps [2]. After polypectomy, recommendations are routinely provided for repeat surveillance colonoscopy, with the goal of reducing risk of CRC-associated morbidity and mortality based on practice guidelines [3]. For example in the USA, current surveillance intervals are largely based on US Multisociety Task Force (USMSTF) guidelines, which stratify individuals for risk of future advanced neoplasia (AN) based on the number, size, and histology of resected polyps.

Although this approach is the standard of care after colonoscopy, evidence suggests that current guidelines can be improved [4]. Indeed, the sensitivity and specificity of USMSTF guidelines for predicting metachronous AN are estimated to be 59–81 % and 43–58 %, respectively [5–9]. Thus, many patients who later develop AN are classified as low risk at baseline, and many who remain free of AN on follow-up are classified as high risk at baseline. In different patient groups this results in over- or underuse of surveillance colonoscopy, exposing many individuals to the risks and costs of unnecessary colonoscopy, while others miss an opportunity for early detection and prevention. Improved risk stratification strategies are needed.

Incorporation of additional risk factors beyond the number, size, and histology of resected polyps might improve risk stratification. For example, age, sex, body mass index (BMI), and polyp location have each been associated with risk of metachronous AN, but have not been formally incorporated into risk stratification guidelines [5, 7, 8, 10–12]. Our aim was to determine whether a statistical model incorporating additional clinical factors could improve post-polypectomy risk stratification compared with USMSTF guidelines, utilizing a large pooled dataset of over 8,000 individuals who underwent polypectomy and subsequent surveillance colonoscopy.

Materials and methods

Design and participants

Data were pooled from seven prospective studies [13–19] of patients with sporadic colorectal adenoma in North America that included 8,228 individuals with polypectomy and repeat surveillance colonoscopy within 3–5 years, as previously described [5]. These studies ascertained baseline data on patient and adenoma characteristics considered as predictors in the analyses and assessed the number, size, histopathology of resected polyps, adenomas, and CRCs detected at follow-up.

Primary outcome

The primary outcome was metachronous AN within 3–5 years of polypectomy, defined by any of the following: adenoma with size ≥1 cm, high-grade dysplasia, and/or tubulovillous or villous histology or adenocarcinoma. AN occurring within 6 months of qualifying colonoscopy was counted as part of baseline rather than follow-up findings.

USMSTF guidelines [3]

The USMSTF guideline low-risk group includes patients with 1 or 2 small (<1 cm), tubular adenoma(s). The USMSTF guideline high-risk group includes patients with 3–10 adenomas, or who have any adenoma with size ≥1 cm, >25 % villous features, or high-grade dysplasia. The USMSTF guideline also specifies a very high (highest)-risk group with more than 10 adenomas. Very few subjects (n = 11) in our data were in highest risk group; therefore, we combined high-risk and highest risk subjects into one group.

Baseline risk factors (predictors)

At qualifying colonoscopy, in addition to adenoma size, number, and histology, we considered adenoma location (distal colorectum, proximal only, proximal and distal), age, sex, race/ethnicity, family history of CRC, cigarette smoking (current, former, never), body mass index (BMI), and history of prior polyp. Additionally, unlike current USMSTF guidelines, we considered number and size of adenoma(s) as continuous rather than categorical variables. To address missing data, we created an unknown category for variables for which more than 4 % of patients were missing information including family history of CRC, polyp location, and history of previous polyp. Polyps with non-adenomatous histology were grouped into one category. Presence of high-grade dysplasia was not characterized by two studies; therefore, this variable was characterized as unknown for patients from these two studies [14, 16].

Statistical analysis

All analyses were performed using R [20]. Patients were randomly assigned 2:1 to training (n = 5,483) and validation (n = 2,745) datasets. Training data were used to develop our prognostic model and identify risk stratification cut-points. Validation data were used to compare sensitivity, specificity, and estimated clinical benefit between the model-based risk stratification (as developed in the training data) and existing USMSTF guidelines.

Development of the prognostic model

Risk factors that were significantly (p < 0.15) associated with the outcome in univariate analysis were considered as potential predictors in a multivariable logistic regression model with the outcome AN. We used two complementary methods for variable selection that are considered to be superior to traditional stepwise model selection approaches [21, 22]. First, an L1-regularized logistic regression model (LASSO) [23, 24] was used to assess the order of entry of variables into the model, using the glmpath package in R. To assess stability of the entry order, a sensitivity analysis using bootstrap LASSO based on 1,000 samples was conducted. Second, Bayesian model averaging (BMA) [25] was used for variable selection, implemented in the BMA package in R. The performance of the models selected by BMA was compared using generalized R² and Brier’s scores, discrimination was assessed using the area under the receiver operating characteristic curve (AUC), and calibration was assessed using the Hosmer–Lemeshow goodness-of-fit test [26]. Among the top performing BMA models, we prioritized the model which was most consistent with the variables selected by LASSO.

Identification of cut-points for risk stratification

We then used the predicted probability of AN from the selected best model to determine a cut-point above which a patient would be identified as at high risk of AN. We made an a priori plan to identify two risk stratification cut-points. Defining sensitivity as the proportion of individuals with metachronous AN at follow-up who were classified as high risk at baseline (Table 1), we targeted the first cut-point to achieve a 10 percentage point improvement in sensitivity compared with the current USMSTF guidelines. Defining specificity as the proportion of individuals without metachronous AN at follow-up who were classified as low risk at baseline (Table 1), we targeted the other cut-point to achieve a 10 percentage point improvement in specificity [27–29]. The population sensitivity and specificity of USMSTF guidelines were estimated on the entire set of pooled data. The cut-point determination included all subjects in the training data with defined values for all predictors in the final model (including those predictors assigned an unknown category for missing values) and used the OptimalCutpoints package in R.

Table 1.

Glossary of risk stratification terms [49]

Sensitivity (true positive rate) proportion of individuals with metachronous advanced neoplasia at follow-up who was classified as ‘high risk’ at baseline by classification strategy (i.e., predictive model or guidelines)

Specificity (true negative rate) proportion of individuals without metachronous advanced neoplasia at follow-up who was classified as ‘low risk’ at baseline by classification strategy

Positive predictive value (PPV) proportion of individuals with metachronous advanced neoplasia at follow-up among those who were classified by a strategy as ‘high risk’ at baseline

Negative predictive value (NPV) proportion of individuals without metachronous advanced neoplasia at follow-up among those who were classified by a strategy as ‘low risk’ at baseline

Surveillance colonoscopy overuse rate (1-PPV) proportion of individuals without metachronous advanced neoplasia at follow-up among those who were classified by a strategy as ‘high risk’ at baseline

Surveillance colonoscopy underuse rate (1-NPV) proportion of individuals with metachronous advanced neoplasia at follow-up among those who were classified by a strategy as ‘low risk’ at baseline

Model validation and comparison of estimated clinical benefit

Model validation was performed on all subjects that could be classified by both the model-based method and the USMSTF guidelines in the validation dataset. Model discrimination was assessed by the AUC. Model calibration was assessed by comparing the predicted risk and observed risk of AN for 10 deciles of risk groups. Potential clinical benefit (Table 1) was assessed in the validation data using the model coefficients and cut-points identified in the training data, by estimated sensitivity and specificity for metachronous AN and estimated rates of over- and underuse of colonoscopy. Overuse of surveillance colonoscopy was defined as the proportion of those classified as high risk at baseline who did not develop metachronous AN (i.e., 1-positive predictive value). Underuse of surveillance colonoscopy was defined as the proportion of those classified as low risk at baseline who developed AN (1-negative predictive value). Improvement in specificity and sensitivity using the predictive model on the validation data was assessed by McNemar’s test. Differences in overuse and underuse between the predictive model and current guidelines were assessed by 95 % confidence intervals (CI) based on 1,000 bootstrap samples. Clinical benefit of using the predictive model over current guidelines was also assessed using net reclassification improvement (NRI) [30].

Results

Analytic cohort selection

The final dataset included 8,228 patients, randomly split into 5,483 training and 2,745 validation subjects. There were no major differences in baseline patient characteristics between the training and validation datasets. Using current USMSTF guidelines, 30.1 % of the cohort was classified as low risk, 47.9 % as high risk, and 22 % could not be classified due to missing data on adenoma size, adenoma number, histology, or high-grade dysplasia, which are required by USMSTF guidelines for risk classification. As described in the Methods, the unclassified subjects were included in predictive model development in the training data, but were excluded from model assessment and the comparison of the clinical benefit between the USMSTF guidelines and the predictive model in the validation data (Table 2).

Table 2.

Patient characteristics for the study cohort, stratified by training versus validation dataset assignment

	Total (n = 8,228) n (%)	Training (n = 5,483) n (%)	Validation (n = 2,745) n (%)	p valuea
Demographics
Age (year)
Mean (SD)	62.1 (9.42)	62.1 (9.43)	62.0 (9.41)	0.42
BMI
Mean (SD)	27.5 (4.43)	27.5 (4.42)	27.6 (4.47)	0.60
Sex
Female	2,374 (28.9)	1,566 (28.6)	808 (29.4)	0.42
Male	5,845 (71.1)	3,917 (71.4)	1,937 (70.6)	0.42
Race/ethnicity
White	7,316 (88.9)	4,870 (88.8)	2,466 (89.1)	0.91
African-American	461 (5.6)	311 (5.7)	150 (5.5)
Others	451 (5.5)	302 (5.5)	149 (5.4)
Family history of colorectal cancer
No	5,818 (70.7)	3,840 (70.0)	1,978 (72.1)	0.04
Yes	1,879 (22.8)	1,265 (23.1)	614 (22.4)
Unknown	531 (6.5)	378 (6.9)	153 (5.6)
Cigarette smoker
Never	2,805 (34.3)	1,906 (34.9)	899 (32.9)	0.18
Former	4,081 (49.9)	2,695 (49.4)	1,386 (50.8)
Current	1,299 (15.9)	853 (15.6)	446 (16.3)
Personal history of previous polyp
No	4,447 (54)	2,967 (54.1)	1,480 (53.9)	0.18
Yes	2,057 (25)	1,342 (24.5)	715 (26)
Unknown	1,724 (21)	1,174 (21.4)	550 (20)
Adenoma characteristics
Number (count)
Mean (SD)	1.73 (1.28)	1.73 (1.26)	1.74 (1.33)	0.85
Size of largest (mm)
Mean (SD)	8.14 (6.20)	8.23 (6.30)	7.95 (6.00)	0.02
Locationb
Distal colorectum	3,867 (47)	2,578 (47)	1,289 (47)	0.31
Proximal only	2,482 (30.2)	1,657 (30.2)	825 (30.1)
Proximal and distal	1,553 (18.9)	1,017 (18.5)	536 (19.5)
Unknown	326 (4.0)	231 (4.2)	95 (3.5)
Histology
Tubular	5,576 (67.8)	3,729 (68)	1,847 (67.3)	0.75
Tubullovillous or villous	1,728 (21)	1,147 (20.9)	581 (21.2)
Unknown and others	924 (11.2)	607 (11.1)	317 (11.5)
High-grade dysplasia
No	5,630 (68.4)	3,747 (68.3)	1,883 (68.6)	0.91
Yes	599 (7.3)	404 (7.4)	195 (7.1)
Unknownc	1,999 (24.3)	1,332 (24.3)	667 (24.3)
Predicted risk of advanced adenoma on follow-upd				0.53
High risk	2,477 (30.1)	1,635 (29.8)	842 (30.7)
Low risk	3,939 (47.9)	2,649 (48.3)	1,290 (47.0)
Unknownc	1,812 (22.0)	1,199 (21.9)	613 (22.3)

^ap value is for comparing the distribution of variables between training and validation datasets

^bAdenoma location is defined as proximal (cecum, ascending colon, hepatic flexure, transverse colon, and splenic flexure) or distal (descending colon, sigmoid colon, and rectum)

^cPresence of high-grade dysplasia was not characterized by AFT and CPPS studies, this variable was characterized as unknown for patients from these two studies

^dRisk of advanced adenoma was classified using USMSTF guidelines, subjects were classified as unknown if missing adenoma size, adenoma number, histology, or high-grade dysplasia

Model development

In univariate analyses of the training dataset, age, sex, history of prior polyps, adenoma number, adenoma size, adenoma location, presence of tubulovillous or villous histology, and presence of high-grade dysplasia were significantly associated with risk for AN on follow-up (Table 3). Including these variables in multivariable logistic regression, LASSO variable selection and bootstrap samples showed that sex and presence of high-grade dysplasia were less important predictors, consistently entering the model last. Considering the top two best models selected by BMA, we found that both models did not include sex and high-grade dysplasia which was consistent with the LASSO result. We also found that both models included age, history of prior polyp, adenoma number, size, and presence of tubulovillous or villous histology, while one included adenoma location and the other did not. Since adenoma location was identified as an important predictor in LASSO, the model including adenoma location was selected as the final model. In the training dataset, the final model had AUC = 0.68 (95 % CI: 0.66–0.70), with good calibration (goodness-of-fit test p = 0.39).

Table 3.

Factors associated with metachronous advanced neoplasia, training data

	Unadjusteda		Adjustedb
	OR (95 % CI)	p value	OR (95 % CI)	p value
Demographics
Age (per year)	1.04 (1.03–1.05)	<0.001	1.03 (1.02–1.04)	<0.001
BMI	1.01 (0.99–1.03)	0.17
Sex
Female	Ref
Male	1.24 (1.03–1.50)	0.02
Race/ethnicity
White	Ref
African-American	1.00 (0.71–1.42)	0.99
Other	0.88 (0.61–1.28)	0.51
Family history of colorectal cancer
No	Ref
Yes	1.04 (0.86–1.26)	0.68
Unknown	1.07 (0.78–1.47)	0.68
Cigarette smoker
Never	Ref
Former	1.10 (0.92–1.31)	0.31
Current	0.97 (0.75–1.25)	0.80
Personal history of previous polyp
No	Ref		Ref
Yes	1.45 (1.21–1.75)	<0.001	1.50 (1.23–1.84)	0.001
Unknown	0.85 (0.68–1.07)	0.16	0.90 (0.71–1.14)	0.38
Adenoma characteristics
Number (per adenoma)	1.25 (1.19–1.32)	<0.001	1.13 (1.06–1.21)	<0.001
Size (per mm)	1.05 (1.04–1.06)	<0.001	1.03 (1.02–1.04)	<0.001
Location
Distal colorectum	Ref		Ref
Proximal only	1.44 (1.19–1.76)	<0.001	1.41 (1.14–1.74)	0.002
Proximal and distal	2.25 (1.83–2.76)	<0.001	1.51 (1.17–1.95)	0.002
Unknown	1.08 (0.69–1.70)	0.73	0.96 (0.58–1.61)	0.88
Histology
Tubular	Ref		Ref
Tubullovillous or villous	2.11 (1.76–2.54)	<0.001	1.63 (1.32–2.00)	<0.001
Unknown/other	1.31 (1.01–1.71)	0.04	1.09 (0.83–1.45)	0.53
High-grade dysplasia
No	Ref
Yes	1.81 (1.38–2.36)	<0.001
Unknown CPPS	1.23 (0.96–1.58)	0.11
Unknown AFT	0.85 (0.65–1.10)	0.22

^aUnadjusted OR from simple logistic regression model

^bAdjusted OR from the best selected multivariable logistic regression model. Note that the ORs are not available for risk factors that were not selected in the best model. Ref, reference group

Identification of cut-points for risk stratification

Based on the entire set of pooled data, the sensitivity and specificity of USMSTF guidelines were estimated to be 78 and 41 %, respectively. Using the selected model, two cut-points were identified to achieve the targeted sensitivity of 88 % and the targeted specificity of 51 %, respectively; that is, a 10 percentage point improvement in sensitivity and specificity over the guidelines. When targeting improvement in sensitivity, the cut-point of 0.075 for predicted probability of AN was selected, and the actual sensitivity achieved at this cut-point was 88.1 % in the training data. When targeting improvement in specificity, the cut-point of 0.101 was selected, and the actual specificity achieved at this cut-point in the training data was 51.2 %. These two cut-points were finalized prior to applying the final predictive model to the validation dataset.

Model validation

When applied to the validation data, the AUC of the model was similar to that observed in the training data (AUC = 0.65; 95 % CI: 0.62–0.69, Fig. 1). Model calibration was also similar, as assessed by goodness-of-fit test (p = 0.21), and ratios of predicted to observed risk of AN, which were generally close to 1.00 (range 0.68–1.17, see Table S1, published online). The sensitivity and specificity of USMSTF guidelines in the validation data were plotted on the same figure (Fig. 1, red dot) and were observed to lie below the model-based ROC curve, indicating that the novel model has potential to improve upon risk stratification compared to current guidelines across a range of cut-points. For comparison, the sensitivity and specificity corresponding to the two a priori identified cut-points are also plotted in Fig. 1, demonstrating that the first model cut-point (Fig. 1, blue square) had superior sensitivity compared to USMSTF guidelines and also demonstrating that the second cut-point (Fig. 1, green triangle) had superior specificity compared to USMSTF guidelines.

Fig. 1.

ROC curve for the predictive model using validation data. This graph demonstrates the potential advantage of using a continuous predictive model for risk stratification. Cut-points for probability of metachronous advanced neoplasia, which were estimated by the predictive model, can be identified to improve either sensitivity or specificity over the USMSTF guidelines. The red dot represents the sensitivity and specificity estimated by the USMSTF guidelines in the validation data. The blue square represents the estimated sensitivity and specificity in the validation data corresponding to the cut-point targeted for improved sensitivity over USMSTF guidelines. The green triangle represents the estimated sensitivity and specificity in the validation data corresponding to the cut-point identified in the test data targeted to improve specificity over the USMSTF guidelines. (Color figure online)

Comparison of model-based and USMSTF guidelines for risk stratification

We found that USMSTF guideline sensitivity and specificity in the validation data for prediction of AN were estimated to be 74 and 42.1 %, respectively, consistent with prior reports [5–8]. Further, we found that 83.2 % of individuals classified as high risk at baseline by USMSTF guidelines in the validation dataset had no AN on follow-up (consistent with surveillance colonoscopy overuse), and 8.9 % classified as low risk developed AN (consistent with surveillance colonoscopy underuse). Using the two cut-points identified in the training data, we next compared the sensitivity and specificity, underuse and overuse rate between the model and the USMSTF guidelines in the validation data (Table 4).

Table 4.

Net clinical benefit of the novel model versus USMSTF guidelines using model cut-points targeted to improve (a) sensitivity, and (b) specificity, applied to the validation dataset

Model cut-point strategy	Measure of clinical benefit	Validation dataset results
		USMSTF guidelines (%)	Predictive model (%)	Difference (95 % CI)a
Improve sensitivity compared to USMSTF guidelines
Specified sensitivity within training data: 88 %	Sensitivity	74.0	88.9	14.9 (9.5, 20.2)***
Resulting cut-point = 0.075	Specificity	42.1	27.7	−14.4 (−16.7, −11.9)***
	Overuse	83.2	83.7	0.5 (−0.6,1.6)
	Underuse	8.9	6.0	−2.9 (−5.0, −0.9)†
Improve specificity compared to USMSTF guidelines
Specified specificity within training data = 51 %	Sensitivity	74.0	75.8	1.8 (−4.2, 7.4)
Resulting cut-point = 0.101	Specificity	42.1	46.2	4.1 (2.0, 6.7)***
	Overuse	83.2	81.8	−1.4 (−2.7, −0.1)†
	Underuse	8.9	7.7	−1.2 (−3.0, 0.6)

^aDifference is the estimate using predictive model minus the estimate using US guidelines and 95 % CI is based on 1,000 bootstrapped samples

^*p value < 0.05;

^**p value < 0.01;

^***p value < 0.001;

^†Statistically significant at alpha = 0.05 based on 95 % bootstrap CI

When using the cut-point targeted to improve sensitivity (Fig. 1, blue square), we found estimated sensitivity for the model-based rule to be significantly higher than for the USMSTF guidelines (88.9 % model vs. 74.0 % guidelines, p < 0.001), correctly identifying an additional net 43 subjects as high risk (14.9 % of all patients with AN on follow-up). However, there was a significant reduction in specificity when comparing the model-based and the USMSTF guidelines (27.7 % model vs. 42.1 % guidelines, p < 0.001), incorrectly classifying an additional net 262 subjects as high risk (14.4 % of all patients without AN on follow-up). The model-based risk stratification was associated with a 2.9 percentage point reduction in estimated surveillance colonoscopy underuse, when compared to the USMSTF guidelines (6.0 % model vs. 8.9 % guidelines, p < 0.05), without substantially changing estimated surveillance colonoscopy overuse among those classified as high risk (83.7 % model vs. 83.2 % guidelines, p > 0.05).

We then investigated the model cut-point targeted to improve specificity (Fig. 1, green triangle). Estimated specificity for the model was significantly superior to USMSTF guidelines in the validation data; however, it did not reach the 10 percentage point improvement that we targeted in the training data (46.2 % model vs. 42.1 % guidelines, p < 0.001). At this cut-point, the model correctly identified an additional net 76 subjects as low risk (4.2 % of all patients without AN on follow-up). There was no reduction in sensitivity (75.8 % model vs. 74.0 % guidelines, p = 0.64), and the model correctly identified an additional net 5 patients as at high risk (1.7 % of all patients with AN on follow-up). Model performance at this cut-point was associated with a modest absolute reduction in estimated surveillance colonoscopy overuse (81.8 % model vs. 83.2 % guidelines, p < 0.05), along with a nonsignificant reduction in colonoscopy underuse (7.7 % model vs. 8.9 % guidelines, p > 0.05).

Discussion

We have demonstrated that a novel predictive model for metachronous AN, together with an a priori established risk cut-point, can modestly improve specificity (46.2 % model vs. 42.1 % guidelines, p < 0.001) with no deterioration in sensitivity (75.8 % model vs. 74.0 % guidelines, p = 0.64), when compared to current USMSTF guidelines for post-polypectomy risk stratification. Our predictive model was developed using best statistical practices for predictive modeling on one of the most comprehensive datasets available. These practices include a priori specification of the model and cut-point development in the training data using optimal prediction strategies, and prespecifying the testing plan in the validation data. Statistical theory thus supports that the model likely represents a near optimal prediction rule based on available data, which is an advantage of a model-based approach over more ad hoc approaches. The improved test characteristics we observed were likely due to selection of additional variables into the final model which are not used by USMSTF guidelines, and also by employing some variables as continuous rather than categorical predictors (e.g., considering the absolute number of adenomas in the predictive model rather than categorizing the number of adenomas as 3 or more, or less than 3, as in the USMSTF guidelines). Although the magnitude of the estimated benefit was rather modest, our work provides proof of concept that improved modeling techniques and additional variables, beyond adenoma number, size, and histology as used for USMSTF guideline-based management, have potential to improve risk stratification of individuals with colorectal polyps, and merit further investigation.

An additional advantage of model-based risk stratification is the ability to tune sensitivity and specificity to clinical needs more finely by choosing a clinically appropriate cut-point. Indeed, as seen from Fig. 1, a risk cut-point to maximize sensitivity or specificity could be identified. Thus, the model could be used to select strategies for surveillance that maximizes either sensitivity or specificity depending on the priorities of patients, physicians, or policy makers. For example, use of the cut-point targeted to improve sensitivity in practice would have resulted in detection of an additional 14.9 % of all individuals with metachronous AN on follow-up. In situations with strained colonoscopy resources, or for patients at increased risk for colonoscopy associated complications due to factors such as age, use of the cut-point targeted to improve specificity could have resulted in reduced colonoscopy use, without an increase in underuse for individuals who developed metachronous AN. These examples also illustrate how a model-based approach can be used to investigate the balance between estimated population level costs of surveillance colonoscopy overuse and underuse across a range of risk stratification cut-points.

A number of cohort and case–control studies have identified risk factors for metachronous AN after initial polypectomy [6, 8, 10, 11, 31–45]. These include patient characteristics (age, sex, BMI, diabetes, and family history of CRC), polyp characteristics (adenoma size, number, location), and histology (villous and/or high-grade dysplasia). We also found many of these characteristics to be associated with risk of metachronous AN in our analysis. However, unlike prior reports, we extended our identification of risk factors for metachronous AN to develop and validate a statistical model for predicting this outcome. To our knowledge, there have not been previous reports of comprehensive statistical models that take into account multiple characteristics, and compare performance to established practice guidelines. As such, our findings confirm and extend prior work seeking to improve risk stratification of individuals after baseline polypectomy. In addition, our model-based stratification rule was based on choosing appropriate cut-points for predicted probability of risk and the performance of the risk stratification was evaluated through sensitivity and specificity, a strategy not commonly reported though it has the advantage of statistical rigor and direct clinical relevance.

Notably, our targeted cut-points, which aimed to achieve substantial improvements in either sensitivity or specificity, could not simultaneously improve sensitivity and specificity. This illustrates the limits of the potential improvements supported by the current data and highlights the need for incorporation of data from additional risk factors, which might support further improvements in overall prognostic power. It is likely that additional variables will need to be identified in order to achieve the goal of substantially improving both specificity and sensitivity at the same time. Although we included key patient and polyp characteristics that were not taken into account by USMSTF guidelines in our predictive model, there are other variables that might be influential but were not considered. For example, the adenoma detection rate has been found to be significantly associated with interval CRC after colonoscopy [46, 47], but was not available for our study. Colonoscopy is an operator-dependent test with performance characteristics (such as adenoma detection rate) varying within endoscopist and other factors [48]. Thus, indicators of the quality of colonoscopy are natural further candidates that could be considered in risk classification [35]. Indeed, because of the observed close relationship between colonoscopist adenoma detection rate and risk for interval cancer after colonoscopy [46, 47], it is possible that this parameter might be a very powerful prognostic variable for metachronous neoplasia. Aspirin and dietary variables, such as red meat, processed meat, and fiber, have also been linked with risk for CRC and may also be considered as candidate variables for risk stratification for metachronous advanced neoplasia [5]. It is likely that additional variables (such as adenoma detection rate and aspirin exposure) would have a substantial impact on the ability of the model to correctly identify both high-risk and low-risk patients. For variables already considered in the predictive model, additional information (such as types of previous polyp, serrated or not) might also improve model performance.

Several limitations may be considered in interpreting this work. First, colonoscopies contributing data to this study were performed between 1984 and 1998. Changes in colonoscopy quality may have occurred over time, such that individuals encountered in clinical practice now might be at different risk of metachronous AN than the individuals included in this study. Second, six of the seven studies contributing to this analysis were from prevention trials, and in two of the trials (one of aspirin, the other of calcium supplementation) a modest degree of intervention efficacy was shown. Further, studies contributing data were not limited to individuals undergoing first time colonoscopy. We are unable to ascertain whether these issues might have resulted in inclusion of a study population that was at higher or lower risk of advanced metachronous neoplasia compared with the general population. Thus, patients from included trials may not be representative of the general population of individuals with adenomas. Third, our training and validation sets were a random split of a larger dataset, and we did not have additional independent validation datasets available. Fourth, analyses may have been affected by the detail of variable ascertainment. For example, we used history of prior polyp as a predictor variable, but do not have detail on type of prior polyp. It is possible that within the prior polyp category, it is only those with prior history of AN who have increased risk. Fifth, some predictors had a high percentage of missing data (24 % for high-grade dysplasia and 21 % for history of previous polyps). We included an ‘unknown’ category for these variables in order to retain more subjects in the analysis and improve the study power. Note that our focus in this study is risk prediction, and optimal methods to address missing data in this context are an area of active statistical research. Further, family history data were collected differently across studies, and were aggregated in this analysis as history of CRC in one or more parents, siblings, or children. Details such as prior polyp type and relationship and age of family members with CRC were not available for analysis, and may have affected the performance of these variables in our models. All of these limitations might be addressed in the future by conducting analyses using large clinical datasets collected as part of usual care, or by developing prospective registries of patients undergoing polypectomy, with careful data collection.

In conclusion, our results suggest that a predictive model including polyp and patient characteristics can be used to improve post-polypectomy risk stratification. Such a model has potential to reduce the unnecessary colonoscopy risks and costs for low-risk patients and identify high-risk individuals who might benefit from early surveillance colonoscopy. However, in order to substantially improve prognostic models for planning colonoscopy surveillance, additional clinical variables associated with colonoscopy outcomes appear to be needed. In ongoing work, we will incorporate quality factors and other additional patient and polyp factors to build on our current methodology for predicting metachronous AN after initial polypectomy. More research is needed to develop and validate prognostic models for planning colonoscopy surveillance, and this should eventually lead to prospective trials that test model-based strategies for surveillance.

Abbreviations

AN

Advanced neoplasia

AUC

Area under the receiver operating characteristic curve

BMI

Body mass index

CI

Confidence intervals

CRC

Colorectal cancer

LASSO

L1-regularized logistic regression model

NRI

Net reclassification improvement

ROC

Receiver operating characteristic

USMSTF

US Multisociety Task Force