Development of Clinical Prediction Models for Surgery and Complications in Crohn’s Disease

Abstract

Background and Aims

Crohn’s disease-related complications account for a substantial proportion of inflammatory bowel disease-associated health care expenditure. Identifying patients at risk for complications may allow for targeted use of early therapeutic interventions to offset this natural course. We aimed to develop risk prediction models for Crohn’s disease-related surgery and complications.

Methods

Using data from the Randomised Evaluation of an Algorithm for Crohn’s Disease cluster-randomised clinical Trial [REACT], which involved 1898 patients from 40 community practices, separate prediction models were derived and internally validated for predicting Crohn’s disease-related surgery and disease-related complications [defined as the first disease-related surgery, hospitalisation, or complication within 24 months]. Model performance was assessed in terms of discrimination and calibration, decision curves, and net benefit analyses.

Results

There were 130 [6.8%] disease-related surgeries and 504 [26.6%] complications during the 24-month follow-up period. Selected baseline predictors of surgery included age, gender, disease location, Harvey-Bradshaw Index [HBI] score, stool frequency, antimetabolite or 5-aminosalicylate use, and the presence of a fistula, abscess, or abdominal mass. Selected predictors of complications included those same factors for surgery, plus corticosteroid or anti-tumour necrosis factor use, but excluded 5-aminosalicylate use. Discrimination ability, as measured by validated c-statistics, was 0.70 and 0.62 for the surgery and complication models, respectively. Score charts and nomograms were developed to facilitate future risk score calculation.

Conclusions

Separate risk prediction models for Crohn’s disease-related surgery and complications were developed using clinical trial data involving community gastroenterology practices. These models could be used to guide Crohn’s disease management. External validation is warranted.

1. Introduction

Crohn’s disease [CD] is an idiopathic disorder of the gastrointestinal tract, characterised by chronic, segmental inflammation, which typically has a relapsing and remitting course. Despite the use of immunosuppressive maintenance therapies, subclinical transmural inflammation persists in many patients,¹ which predisposes them to such complications as strictures and fistulas.² Recent studies estimate the long-term risk of surgery to be approximately 60–80%,^2–5 with the greatest risk in the first few years following diagnosis.

In the past two decades, the advent of biologic therapies and refinement of treatment paradigms have revolutionised the medical management of CD. Specific advances include the introduction of tumour necrosis factor [TNF] antagonists^6–13 and integrin inhibitors,^14–18 the use of combination therapy,^19–22 therapeutic drug monitoring,^23,24 and earlier initiation of effective therapies in high-risk patients.^{12,19–21,25–29} However, one of the greatest challenges in implementing these strategies is determining which patients are most at risk for complicated disease and thus candidates for more intensive monitoring and treatment. This decision is usually based on clinical judgement and heavily weighted by the patient’s disease activity as assessed by symptoms. This approach is practical, but it does not routinely incorporate prognostic factors that drive disease-related complications. Accordingly, identification of patients at highest risk of complications and disease progression, who have the greatest chance of benefiting from early initiation of effective therapy, is an aspirational goal.

In this regard, the approach to therapy in CD has changed dramatically. Formerly, ‘step-care’, in which drugs are used sequentially to attain symptomatic remission, was the preferred paradigm. Whereas this approach is attractive because it avoids over-treating low-risk patients, step-care delays initiation of effective therapy in patients most at risk for adverse outcomes. Recently, attention has turned to early introduction of combination immunosuppression therapy in high-risk patients, to promote mucosal healing and minimise exposure to corticosteroids.^19,20 This ‘top-down’ approach requires accurate identification of high-risk patients, to minimise over-treatment in lower-risk patients, with the associated risk of treatment-related adverse events and costs. Conversely, misclassification of high-risk patients delays administration of effective therapies and potentially results in increased risk of complications. Therefore, the ability to accurately risk-stratify patients has garnered considerable interest.^30,31

Retrospective analyses of single-centre and population-based cohorts have identified several prognostic factors in CD, including younger age at diagnosis, ileal disease location, perianal disease, stricturing or penetrating phenotype, current smoking, treatment with corticosteroids at diagnosis or corticosteroid dependence, and extensive disease involvement.^{1,3–5,31–40}

Existing clinical prediction models have several limitations. Notably, these models were developed from single centres, referral centres, retrospective samples, small development samples, or cohorts assembled before the widespread use of TNF antagonists, and they often predict excessively long-term risk estimates of outcomes [5–10 years].^{32,33,41–44} However, to fully utilise this information, prediction models must be established.

Data arising from randomised trials offer a unique ability in developing clinical prediction models, because of their scope, size, multicentre participation and prospective follow-up, thus overcoming some of the aforementioned design limitations. We developed and internally validated clinical prediction models for CD using data from a large cluster randomised trial conducted throughout community practices, comparing two treatment algorithms for CD.²¹ Our specific aim was to develop separate models for predicting risks of two binary endpoints within 24 months of follow up: [1] occurrence of CD-related surgery; and [2] occurrence of a composite of CD-related surgery, complications or hospitalisation [this composite outcome is henceforth referred to as disease-related complications].

2. Methods

2.1. Data source

The study population was participants from the Randomised Evaluation of an Algorithm for Crohn’s Disease [REACT] trial [NCT01030809].²¹ Briefly, REACT was a large, cluster-randomised, controlled trial of two distinct algorithms for the management of CD. Forty Canadian and Belgian community-based gastroenterology clinics were randomised, in a 1:1 ratio, to either early combined immunosuppression or step-care. In each clinic, consecutive adult patients with CD were enrolled, regardless of disease activity or concurrent therapy, and followed up to 24 months. The dataset was used to develop separate models to predict the risk over 24 months of having CD-related surgery and disease-related complications.

The reporting of this study follows the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis [TRIPOD] statement [Supplementary Table 1, available at ECCO-JCC online].⁴⁵

2.2. Clinical outcomes and definitions

Binary outcomes were defined within 2 years of follow up. The first was the occurrence of CD-related surgery. The second was a CD-related complication. Surgery was considered separately since it is a readily measured event of clinical importance. All events in the REACT trial used in this study were evaluated by an adjudication committee who were blinded to treatment assignment. Disease-related surgeries included resective bowel surgery [ileal resection, ileocaecal resection, proctocolectomy, colectomy, enterectomy, ostomy formation and repair, anastomosis/reanastomosis], stricureplasty, and fistula repair [incision and drainage of abscess, seton placement, fistulotomy, fistulectomy]. Disease-related complications were defined as a composite of disease-related surgery [as defined above], complications [including development of penetrating or stricturing disease, worsening abdominal pain, increased stool frequency, extra-intestinal manifestations, severe perianal disease, fistula, or abscess²¹] or hospitalisation.

2.3. Selection of predictors

Candidate prognostic factors were selected from the demographic and disease-related variables, using standardised clinical definitions, collected at baseline and before treatment allocation. A search of the literature identified potential prognostic factors of interest that were augmented by clinical judgement. These variables were age at enrolment, age at diagnosis, gender, smoking status, disease location, perianal disease, previous surgical resection for CD, use of each medication at baseline [including 5-aminosalicylates, corticosteroids, immunosuppressives, and TNF antagonists], abdominal pain, abdominal mass, extra-intestinal manifestations, strictures, fistulas, and stool frequency. No laboratory parameters, biomarkers, or cross-sectional imaging items were included. Since the purpose of these models is to predict risks using baseline factors before treatment allocation, treatment assignment was not considered as a predictor in the models. Moreover, coefficients for treatment effect were relatively small compared with other prognostic factors.

2.4. Missing data

No participants had missing outcome data. However, 4.24% [n = 84] of patients had at least one missing variable at baseline, of which it was most common to be missing components of the HBI score. No patterns were observed in the missing values. Only participants with complete baseline data were used for model development.

2.5. Model development

Exploratory univariate data analysis was first conducted to assess adequate event frequency between each outcome and the candidate prognostic factors. Univariable associations between candidate predictors and the outcomes were assessed graphically and using logistic regression. Each model was then constructed using multivariable logistic regression analysis. Predictors with univariate associations p < 0.20 entered the multivariable model. From this full model, unnecessary variables were removed based on a difference in Akaike information criteria [AIC; < 2.0]. In terms of the relative performance of models as estimated from bootstrap replicates, the removal of these variables had a negligible impact on discrimination, the Brier score [< 1% difference in mean difference], and calibration. Although the data arose from a cluster-randomised trial, regression coefficient estimates are not much affected when the degree of clustering is low as in this trial [intraclass correlation coefficient < 0.02].⁴⁶ Since the focus in prediction studies is on the absolute risk estimate from the combined predictor effects, these analyses were conducted at the patient level without accounting for clustering.

2.6. Predictive performance and model validation

The model development and validation process adhered to recommended guidelines.^47,48 Model performance was characterised by the discrimination ability and calibration: the discrimination ability of a model to distinguish between patients with the outcome and patients without the outcome. In the present context, discrimination is measured using the c-statistic, which is identical to the area under the receiver operating characteristic curve for binary outcomes.⁴⁹ A value of 0.50 for the c-statistic represents the prognostic ability of a coin flip and will therefore correctly differentiate 50% of cases. There are no universal guidelines for describing the quality of discrimination ability, but generally discrimination values below 60% are unacceptable, 60% to 70% may be acceptable and values from 70% to > 90% range from good to excellent. The interpretation of these values is context-dependent. Calibration refers to the agreement between predicted and observed risks.⁴⁷ This can be assessed by plotting the predicted risks and the observed percentages of patients who had the outcome.

Since model development tends to over-fit the data, commonly referred to as optimism, it is desirable to reduce optimism using internal validation and/or external validation.⁵⁰ Internal validation of the models was conducted using the bootstrap method with 500 replicates.⁵¹ Performance characteristics were calculated for the original [index] and validated models following correction for optimism. Additional details regarding statistical methodology and performance measures are provided in the Supplementary Appendix, available at ECCO-JCC online.

2.7. Sample size

Formal sample size calculation was not conducted. Of the 1982 participants in the REACT trial, 504 had disease-related complications and 130 had surgeries over 2 years of follow-up. According to the guideline of 10 events per variable,⁵² this dataset is large enough to construct prediction models with at least ten predictors.

2.8. General statistical methods

Exploratory analysis was performed using Stata [version 14.1/IC; StataCorp, College Station, TX]. Model building and evaluation were conducted using package rms⁴⁷ in R [version 3.3.1; Linux; R Core Team⁵³] through RStudio [version 0.99; RStudio Team⁵⁴]. Summary statistics are presented as mean ± standard deviation [SD], median, or frequencies and proportions as appropriate.

3. Results

3.1. Patient characteristics and clinical outcomes

Baseline characteristics are outlined in Table 1. A total of 1097 [57.79%] patients were female. Median disease duration was 148.8 months, the mean HBI was 4.1, and almost one-third of patients were taking biologics at baseline. Over the 24-month follow-up period, 6.84% [n = 130] of participants underwent CD-related surgery, and 26.55% [n = 504] of participants experienced a CD-related complication. Univariate associations for each outcome are shown in Table 2.

Table 1.

Overall patient characteristics at baseline [N = 1898].

Baseline characteristic	Overall N = 1898	Surgery group N = 130 6.8%	No surgery group N = 1768 93.1%	Complication group N = 504 26.6%	No complication group N = 1394 73.5%
Demographics
Age, years [mean ± SD]	44.0 ± 14.6	42.7 ± 14.0	44.1 ± 14.6	42.1 ± 14.6	44.7 ± 14.5
Female	1097 [57.78%]	68 [52.3%]	1029 [58.2%]	311 [61.7%]	786 [56.4%]
Smoking status
Non-smoker	939 [49.5%]	58 [44.6%]	881 [49.9%]	241 [47.9%]	698 [50.1%]
Ex-smoker	550 [29.0%]	36 [27.7%]	514 [29.1%]	405 [29.1%]	145 [28.8%]
Current smoker	407 [21.44%]	36 [27.7%]	371 [21.0%]	290 [20.8%]	233 [11.7%]
Disease characteristics
Duration, months (mean [median])	148.8 [119]	144.8 [111.1]	149.1 [119.2]	147.0 [114.0]	149.5 120.9
HBI score [median; mean ± SD]	3; 4.1 ± 1.1	5; 6.0 ± 3.5	3; 4.0 ± 1.1	4; 5.0 ± 4.9	3; 3.8 ± 3.8
Steroid-free remission [HBI ≤ 4]	1,065 [56.1%]	51 [39.2%]	1014 [57.4%]	236 [46.8%]	829 [59.5%]
Extra-intestinal manifestations at baseline (n [%])	594 [31.3%]	55 [42.3%]	539 [30.5%]	185 [36.7%]	409 [29.3%]
Previous history of disease-related surgery	864 [45.5%]	56 [43.1%]	808 [45.7%]	244 [48.4%]	620 [44.5%]
Involved intestinal areas
Colon	417 [22.0%]	17 [13.1%]	400 [22.6%]	92 [18.3%]	325 [23.3%]
Small bowel	654 [34.5%]	52 [40.0%]	602 [34.1%]	169 [33.5%]	485 [34.8%]
Colon and small bowel	827 [43.6%]	61 [46.9%]	766 [43.3%]	243 [48.2%]	584 [41.9%]
Concurrent medications use
Aminosalicylates	539 [28.4%]	22 [16.9%]	517 [29.2%]	116 [23.0%]	423 [30.3%]
Corticosteroids	348 [18.3%]	31 [23.9%]	317 [17.9%]	113 [22.4%]	235 [16.9%]
Anti-metabolites	826 [43.5%]	48 [36.9%]	778 [44.0%]	204 [40.5%]	622 [44.6%]
TNF antagonists	622 [32.8%]	53 [40.8%]	569 [32.2%]	201 [39.9%]	421 [30.2%]

Figures may vary slightly from those published in Khanna et al. [2015], due to exclusion of 84 patients with at least one missing predictor variable.

SD, standard deviation; HBI, Harvey-Bradshaw Index; TNF, tumour necrosis factor.

Table 2.

Univariate associations for Crohn’s disease-related complications or CD-related surgery alone.

Baseline variable	CD-related surgery		CD-related complicationa
	OR [95% CI]	p-value	OR [95% CI]	p-value
Current age	0.99 [0.98, 1.00]	0.27	0.99 [0.98, 1.00]	< 0.0001
Gender [male vs. female]	1.27 [0.89, 1.81]	0.19	0.80 [0.65, 0.99]	0.04
HBI score [except stool frequency]	1.36 [1.25, 1.48]	< 0.0001	1.18 [1.12, 1.24]	< 0.0001
Stool frequency	1.08 [1.15, 1.02]	< 0.01	1.07 [1.03, 1.11]	< 0.0001
Location of disease involvement
Colon only [reference]	1	0.03	1	0.02
Small bowel and colon	1.87 [1.08, 3.25]		1.47 [1.12, 1.94]
Small bowel only	2.03 [1.16, 3.57]		1.23 [0.92, 1.65]
Presence of new fistula, abscess, or definite abdominal mass	10.29 [6.23, 16.99]	< 0.0001	3.94 [2.48, 6.26]	< 0.0001
Anti-metabolite use	0.75 [0.52, 1.08]	0.11	0.84 [0.69, 1.04]	0.10
5-aminosalicylate use	0.49 [0.31, 0.79]	< 0.01	0.69 [0.54, 0.87]	< 0.01
Corticosteroid use	1.43 [0.94, 2.18]	0.10	1.43 [1.11, 1.84]	< 0.01
TNF antagonist use	1.45 [1.01, 2.09]	0.05	1.53 [1.24, 1.89]	< 0.0001
Current smoking vs. ex-/non-smoker	1.44 [0.96, 2.15]	0.08	1.15 [0.90, 1.47]	0.26

All outcomes are defined within 24 months of follow-up.

CD, Crohn’s disease; OR, odds ratio, CI, confidence interval; HBI, Harvey-Bradshaw Index; TNF, tumour necrosis factor.

^aComplication refers to the first occurrence of a CD-related surgery, complication or hospitalisation.

3.2. Model performance and validation

A separate logistic regression model was estimated for each outcome. Table 3 shows the estimates of regression coefficients for the CD-related surgery and CD-related complication models. The baseline predictors included in the surgery model were age, gender, disease location, HBI score, stool frequency, immunosuppressive use, 5-aminosalicylate use, and the presence of a fistula, abscess, or abdominal mass. The baseline predictors for the CD-related complication model also uniquely included the use of corticosteroids and TNF antagonists, in addition to all the variables identified for surgery.

Table 3.

Coefficient estimates of logistic regression models for CD-related surgery and a CD-related complication.

Baseline predictor	CD-related surgery			CD-related complication a
	Regression coefficientb	SE	p-value	Regression coefficientc	SE	p-value
Intercept	-3.607	0.327		-1.6134	0.162
Age subtract 45 years	-0.0027	0.007	0.72	-0.0102	0.004	0.02
Male vs female	0.3946	0.194	0.06	-0.1848	0.110	0.13
HBI score [except stool frequency]	0.2425	0.065	< 0.001	0.1139	0.042	0.01
Stool frequencyd	0.0741	0.060	0.25	0.0752	0.034	0.04
Location of disease involvement
Colon only [reference]	0	–	–	0	–	–
Small bowel and colon	0.3614	0.293	0.26	0.2525	0.145	0.12
Small bowel only	0.5167	0.297	0.11	0.1739	0.153	0.31
Corticosteroid use	–	–	–	0.2465	0.138	0.11
Anti-metabolite use	-0.4519	0.200	0.04	-0.1598	0.110	0.19
5-aminosalicylate use	-0.6015	0.253	0.03	–	–	–
TNF antagonist use	–	–	–	0.3887	0.114	< 0.01
Presence of new fistula, abscess, or definite abdominal mass	1.7019	0.306	< 0.0001	1.0840	0.261	< 0.001
Interaction
HBI score × stool frequency	-0.0182	0.014	0.22	-0.0136	0.009	0.18

SE, standard error; CD, Crohn’s disease; HBI, Harvey-Bradshaw Index; TNF, tumour necrosis factor.

^aComplication refers to the first occurrence of a CD-related surgery, complication, or hospitalisation.

^bRegression coefficients are presented before shrinkage [shrinkage factor = 0.92].

^cRegression coefficients are presented before shrinkage [shrinkage factor = 0.90].

^dStool frequency has a maximum value of 12.

The validated discrimination ability for the CD-related surgery model was reasonably good with a c-statistic of 0.70 (95% confidence interval [CI]: 0.69 to 0.71), whereas the discrimination ability of disease-related complication model was relatively acceptable at 0.62 [95% CI: 0.61 to 0.64] [Supplementary Table 2, available at ECCO-JCC online]. Both models demonstrated low degrees of optimism [shrinkage factors were 0.92 and 0.90, respectively], and excellent calibration and goodness-of-fit statistics [see Supplementary Appendix].

3.3. Computing a risk estimate, score chart, and nomogram

The validated logistic regression model for CD-related surgery within 2 years is shown in Equation 1.

$$logit\left(Pr\left(Surgery=Y\right)\right)=\begin{array}{l}-3.511\hfill \\ + \left(-0.0025\right)\times \left(Age-45 years\right)\hfill \\ + (0.3634)\times (1 if gender=Male; 0 otherwise)\hfill \\ + (0.2234)\times (HBI score less stool frequency)\hfill \\ + (0.0683)\times (stool frequency)\hfill \\ + (-0.0167)\times (HBI score less stool frequency)\times (stool frequency)\hfill \\ + (0.3328)\times (1 if small bowel and colonic disease; 0 otherwise)\hfill \\ + (0.4759)\times (1 if small bowel only disease; 0 otherwise)\hfill \\ + (-0.4162)\times (1 if anti-metabolite use; 0 otherwise)\hfill \\ + (-0.5540)\times (1 if 5-ASA use; 0 otherwise)\hfill \\ + (1.5674)\times (1 if new fistula, abscess or abdominal mass)\hfill \end{array}$$ (1)

The validation logistic regression model for CD-related complications is shown in Equation 2.

$$logit\left(Pr\left(Complication=Y\right)\right)\begin{array}{l}=-1.552\hfill \\ + \left(-0.0092\right)\times \left(Age-45 years\right)\hfill \\ + (-0.1669)\times (1 if gender=Male; 0 otherwise)\hfill \\ + (0.1028)\times (HBI score less stool frequency)\hfill \\ + (0.0679)\times (stool frequency)\hfill \\ + (-0.0123)\times (HBI score less stool frequency)\times (stool frequency)\hfill \\ + (0.2280)\times (1 if small bowel and colonic disease; 0 otherwise)\hfill \\ + (0.1570)\times (1 if small bowel only disease; 0 otherwise)\hfill \\ + (0.2226)\times (1 if corticosteroid use; 0 otherwise)\hfill \\ + (-0.1443)\times (1 if antimetabolite use; 0 otherwise)\hfill \\ + (0.3510)\times (1 if TNF-antagonist use; 0 otherwise)\hfill \\ + (0.9789)\times (1 if new fistula, abscess or abdominal mass)\hfill \end{array}$$ (2)

To compute the risk score, multiply the value of each variable with its associated regression coefficient, and then sum all values, producing the linear predictor [lp]. The linear predictor on the log-odds scale may be converted to a risk probability [%] by the inverse log-odds formula: $p=100\%\times {\left[1+\exp (lp)\right]}^{-1}$. An example risk calculation is demonstrated in Table 4. A risk score may alternatively be calculated using score charts and converted to a risk estimate [Tables 5 and 6] or computed by the provided nomograms [Figure 1A and B].

Table 4.

Computing the predicted risk score.

As an example, suppose a 50-year old male patient presents to the clinic with disease confined to the small bowel. His total HBI score is 3 with a stool frequency of two per day, he is currently taking 5-ASA, and does not have a fistula, stricture, or definite abdominal mass. The clinician would now like to predict the risk of surgery within the next 2 years. Start by plugging these values into the equation for the linear predictor. lp = -3.51 [Intercept] + [50 – 45; age, years]*[- 2.5*10–3] + [1; male]*[0.36] + [1 point; HBI subtract stools/day]*[0.22] + [2 stools/day]*[6.8*10–2] + [1; small bowel only]*[0.48] + [0; antimetabolite use]*[-0.42] + [1; 5-ASA use]*[-0.55] + [0; fistula, abscess, abdominal mass]*[1.57] + [1 point; HBI subtract stools/day]*[2 stools/day]*[-1.7*10–2] = -2.8175 ≈ -2.82. Now convert the linear predictor into percent risk. Risk = 100% * 1 / [1 + exp[-lp]] = 5.6% [expected risk is approximately 5.6%].

HBI, Harvey-Bradshaw Index; 5-ASA, 5-aminosalicylate.

Table 5.

Score chart for risk of CD-related surgery.

Table 6.

Score chart for risk of CD-related complications.

Figure 1.

Nomogram for the computation of percentage risk of Crohn’s disease-related surgery [A] or complication [B].

For prediction of CD-related surgery, the score ranges from 0 to 233 [Table 5]. A score of 61, 119, or 171 respectively predict a 5%, 20%, or 50% chance of experiencing a disease-related surgery within 24 months of follow-up. For prediction of CD-related complications, the score ranges from 0 to 314 [Table 6]. A score of 88, 177, or 266 respectively predict a 25%, 50%, or 75% chance of experiencing a disease-related complication within 24 months of follow-up.

4. Discussion

We developed two clinical prediction models, one for predicting individual risk of experiencing CD-related surgery, and the other for predicting CD-related complications within 24 months, using data from one of the largest clinical trials conducted in CD to date. These models have been developed using patients from the REACT trial, and are thus intended for use in a broad cross-section of patients found in community gastroenterology clinics and not solely with regard to a specific disease duration or severity. Although both models demonstrated high specificity and negative predictive values, the surgery model had relatively greater predictive ability, likely because surgery is an easily defined and more objective clinical outcome, compared with disease-related complications. Additionally, the development of a score chart using basic clinical variables which are readily calculable facilitates implementation of this score in the outpatient clinic setting, potentially allowing for enhanced decision making between patients and physicians. If validated in an external dataset, the use of this model may help identify patients at higher risk of disease-related complications and especially surgery, who may benefit most from more intensive treatment, combination therapy, and/or closer monitoring for symptoms that require planned surgery [eg. obstructive symptoms in the setting of fibrostenotic disease]. Additionally, all identified baseline predictors in our models were prospectively defined and measured. We believe this is a distinct advantage to retrospective cohort studies in which a greater potential exists for bias during the data acquisition process.

Existing prediction models for CD-related outcomes [summarised in Supplementary Table 5, available at ECCO-JCC online] have been limited by selection or referral centre bias, small sample size, evolving drug treatments and management strategies, long-term time horizons, or broadly inclusive outcome definitions. Furthermore, the time frame in which these models were produced must be considered. Since the greatest risk of complications occurs within the first few years after diagnosis, long-term risk prediction models eventually become confounded or outdated with evolving therapies and treatment strategies. For example, while some models were developed in the pre-biologic era, others define the need for thiopurines as advanced disease; but currently thiopurine monotherapy represents a treatment strategy generally reserved for less severe disease rather than severe disease.^55,56 Nevertheless, it is notable that many of the items we identified as independent predictors have previously been shown to have prognostic value in population-based cohorts with longer-term follow up,^1,4,57–60 supporting the validity of our results. Prediction of CD-related complications is limited by the use of non-standard or broadly inclusive outcome definitions [Supplementary Table 5], thus preventing meaningful comparisons with our model. This may also explain the lower performance of the model for prediction of CD-related complications.

In the present study, the surgery model has shown promise to accurately predict an adverse outcome which may be useful to guide management strategies and patient care. Such a model could also reasonably be used for risk stratification of patients in clinical studies.

A major strength of the current study is use of data from a large randomised controlled trial [RCT] conducted across community centres, meaning that the endpoints were well defined, complete to 24 months of follow-up, and adjudicated by an independent panel. The pragmatic nature of the trial design meant that consecutive patients were enrolled, regardless of disease activity, duration, phenotype, or treatment, and an algorithm of care applied which is broadly representative of the patient population seen in clinical practice. Thus, we believe these models may be more generalisable to the routine clinical care of adult patients with CD, and can be applied within the community clinic, regardless of disease duration, in keeping with the design of the trial from which the model and data were derived.

It is interesting to point out that smoking is commonly regarded as a poor prognostic factor for Crohn’s disease and especially for postoperative recurrence. In the present study, we found that smoking had limited predictive ability for CD-related complications. Another important consideration with smoking is that there is a degree of subjective reporting, which can lead to inaccurate categorisation of current/recent/past smoking status as well as confounding with the quantity of smoking. Thus, we did not retain this in the final prediction models, which did not appreciably affect predictive performance.

Limitations of this study should be acknowledged. Perhaps the greatest limitation is that these models were validated in the cohort in which they were developed. Thus, independent validation in an external dataset is required before the widespread adoption of any prediction model can be considered. The results presented here encourage seeking such external validation. Participants in the REACT study had a longer average duration of disease, though approximately 15% and 28% were diagnosed within 2 and 5 years, respectively. However, the operating characteristics of these models in only newly diagnosed patients remain to be determined. Indeed, the REACT design was intended to include a broad cross-section of patients within community clinics, so that we may not be limited to specific subsets of disease or demographic characteristics, and applies more generally to a broader patient population.

The CD-related surgery model performed better than the complication model, which may be related to using a more well-defined and objective outcome definition. Although the models use routinely available and clearly defined clinical data collected by trained gastroenterologists, inter-observer variability may influence the validity of the model. Also, the REACT study did not include biomarkers or serological or genetic factors; therefore these factors could not be incorporated into the current models. These factors may provide additional prognostic value beyond routine clinical data.⁴¹

Related to this issue of validation is discrimination performance. Prediction of CD-related complications is relatively difficult, as reflected in its low discriminative ability. However, the surgery model performs nominally as well as the model by Siegel and colleagues,⁴³ using completely different modelling approaches and relying solely on readily available clinical variables. Nevertheless, we are cautiously optimistic that the surgery model may prove useful once externally validated.

The treatment allocation was not included in our models, which some may view as a limitation. However, this decision was made a priori for two reasons. First, our primary interest was to develop prediction models using only baseline predictive factors that are available in practice. Second, the effect of treatment to predict these outcomes was found to be relatively small as compared with the factors in our prediction models. This is consistent with prediction literature in other areas, such as cardiovascular disease.⁶¹

In summary, we have developed and validated clinical prediction models for CD-related surgery and complications using data from a large randomised clinical trial with good overall performance for predicting the outcome of disease-related surgery within 2 years. We have transformed these models into scoring tools, facilitating their use in the clinic; external validation in independent cohorts with longitudinal follow up is recommended.

Funding

This work was not specifically supported by any funding agency in the public, commercial, or not-for-profit sectors.

Conflict of Interest

RK: honoraria from AbbVie, Janssen, Pfizer, Shire, and Takeda Pharma. PD: research support funding from Takeda and Pfizer, and a training grant through the National Institute of Diabetes and Digestive and Kidney Diseases [NIDDK T32DK007202]. WJS: grant support from Exact Sciences, the American College of Gastroenterology, and the Broad Foundation; grant support and personal fees from Receptos, Amgen, Prometheus Laboratories, AbbVie, Boehringer Ingelheim, Takeda, Atlantic Pharmaceuticals, Janssen, Bristol-Myers Squibb, Genentech, Pfizer, and Nutrition Science Partners; and personal fees from Kyowa Hakko Kirin, Millennium Pharmaceuticals, Celgene Cellular Therapeutics, Santarus, Salix Pharmaceuticals, Catabasis Pharmaceuticals, Vertex Pharmaceuticals, Warner Chilcott, Gilead Sciences, Cosmo Pharmaceuticals, Ferring Pharmaceuticals, Sigmoid Biotechnologies, Tillotts Pharma, Am Pharma BV, Dr. August Wolff, Avaxia Biologics, Zyngenia, Ironwood Pharmaceuticals, Index Pharmaceuticals, Nestle, Lexicon Pharmaceuticals, UCB Pharma, Orexigen, Luitpold Pharmaceuticals, Baxter Health care, Ferring Research Institute, Amgen, Novo Nordisk, Mesoblast Inc., Shire, Ardelyx Inc., Actavis, Seattle Genetics, MedImmune [AstraZeneca], Actogenix NV, Lipid Therapeutics Gmbh, Eisai, Qu Biologics, Toray Industries Inc., Teva Pharmaceuticals, Eli Lilly, Chiasma, TiGenix, Adherion Therapeutics, Immune Pharmaceuticals, Celgene, Arena Pharmaceuticals, Ambrx Inc., Akros Pharma, Vascular Biogenics, Theradiag, Forward Pharma, Regeneron, Galapagos, Seres Health, Ritter Pharmaceuticals, Theravance, Palatin, Biogen, and the University of Western Ontario [owner of Robarts Clinical Trials Inc]. VJ: scientific advisory board fees from Abbvie, Sandoz, Ferring and Janssen; speaker’s fees from Takeda and Ferring; and travel support for conference attendance from Vifor pharmaceuticals. BGF: grant/research support from Millennium Pharmaceuticals, Merck, Tillotts Pharma AG, AbbVie, Novartis Pharmaceuticals, Centocor Inc., Elan/Biogen, UCB Pharma, Bristol-Myers Squibb, Genentech, ActoGenix, and Wyeth Pharmaceuticals Inc.; consulting fees from Millennium Pharmaceuticals, Merck, Centocor Inc., Elan/Biogen, Janssen-Ortho, Teva Pharmaceuticals, Bristol-Myers Squibb, Celgene, UCB Pharma, AbbVie, Astra Zeneca, Serono, Genentech, Tillotts Pharma AG, Unity Pharmaceuticals, Albireo Pharma, Given Imaging Inc., Salix Pharmaceuticals, Novonordisk, GSK, Actogenix, Prometheus Therapeutics and Diagnostics, Athersys, Axcan, Gilead, Pfizer, Shire, Wyeth, Zealand Pharma, Zyngenia, GiCare Pharma Inc., and Sigmoid Pharma; and speaker fees from UCB, AbbVie, and J&J/Janssen.

Author Contributions

LG: study concept and design; acquisition of data; analysis and interpretation of data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; statistical analysis. GZ: study concept and design; acquisition of data; analysis and interpretation of data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; statistical analysis; study supervision. RK: study concept and design; acquisition of data; analysis and interpretation of data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; study supervision. PD: critical revision of the manuscript for important intellectual content. WJS: critical revision of the manuscript for important intellectual content. VJ: study concept and design; acquisition of data; analysis and interpretation of data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; study supervision. BGF: study concept and design; acquisition of data; analysis and interpretation of data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; study supervision. All authors have approved the final version of this manuscript.

Supplementary Data

Supplementary data are available at ECCO-JCC online.