Predicting the Individual Risk of Acute Severe Colitis at Diagnosis

Abstract

Background and Aims:

Acute severe colitis [ASC] is associated with major morbidity. We aimed to develop and externally validate an index that predicted ASC within 3 years of diagnosis.

Methods:

The development cohort included patients aged 16–89 years, diagnosed with ulcerative colitis [UC] in Oxford and followed for 3 years. Primary outcome was hospitalization for ASC, excluding patients admitted within 1 month of diagnosis. Multivariable logistic regression examined the adjusted association of seven risk factors with ASC. Backwards elimination produced a parsimonious model that was simplified to create an easy-to-use index. External validation occurred in separate cohorts from Cambridge, UK, and Uppsala, Sweden.

Results:

The development cohort [Oxford] included 34/111 patients who developed ASC within a median 14 months [range 1–29]. The final model applied the sum of 1 point each for extensive disease, C-reactive protein [CRP] > 10mg/l, or haemoglobin < 12g/dl F or < 14g/dl M at diagnosis, to give a score from 0/3 to 3/3. This predicted a 70% risk of developing ASC within 3 years [score 3/3]. Validation cohorts included different proportions with ASC [Cambridge = 25/96; Uppsala = 18/298]. Of those scoring 3/3 at diagnosis, 18/18 [Cambridge] and 12/13 [Uppsala] subsequently developed ASC. Discriminant ability [c-index, where 1.0 = perfect discrimination] was 0.81 [Oxford], 0.95 [Cambridge], 0.97 [Uppsala]. Internal validation using bootstrapping showed good calibration, with similar predicted risk across all cohorts. A nomogram predicted individual risk.

Conclusions:

An index applied at diagnosis reliably predicts the risk of ASC within 3 years in different populations. Patients with a score 3/3 at diagnosis may merit early immunomodulator therapy.

1. Introduction

Acute severe colitis [ASC] has well-defined criteria for diagnosis which guide management, the timing of decision making and predicts outcome.^1,2 The commonest indication for colectomy is ASC unresponsive to medical therapy, and one or more hospital admissions with ASC are associated with an overall risk of colectomy approaching 40%, compared with just 3% in those never hospitalized.³ Yet in 2015, ASC remains a potentially life-threatening condition. In the UK national audit, the overall mortality of 2981 patients hospitalized with ASC was 1.2%, but reached 1.9% in patients aged 50–59, 3.5% in those aged 60–69 and over 10% in those aged > 80 years.⁴

There is, therefore, a need to identify factors that predict the risk of ASC. Predicting a severe or disabling course of Crohn’s disease has received substantial attention in the past decade, deploying clinical criteria, gene expression profiling of CD8⁺ T cells, or biochemical and endoscopic predictors.^5–7 In Crohn’s disease, predictors can potentially be identified at diagnosis, when they might influence management and outcome [for a review, see reference⁷]. The same applies to UC, with the advantage that there are simpler endpoints [hospital admission with ASC or colectomy] and a more homogeneous patient population. As a surrogate for a progressive or severe course of UC, the need for immunomodulators was studied in a cohort of 262 patients.⁹ A relatively complex six-item model was derived, which predicted the need for immunomodulators within 5 years in 60% of patients, including simple items such as extent, CRP and anaemia. Preliminary examination of genetic factors also led to a risk score for colectomy, based on 46 single nucleotide polymorphisms [SNPs] that accounted for 48% of the variance.^13,14

From a patient’s perspective, however, there are few data that might predict hospital admission, although admission identifies those with a 10-fold higher risk of colectomy.³ Thresholds for admission vary between countries, but the internationally accepted criteria for diagnosis of ASC and relationship to outcome should supersede national differences.^2,16 This is an area of unmet need, because it potentially identifies at an early stage those likely to pursue a poor disease course, as treatment options proliferate. The aim of this study, therefore, was to identify predictive factors for developing ASC over a 3-year period from the date of diagnosis of UC, to create a simple index that could be applied in clinical practice, followed by validation in two independent cohorts of patients.

2. Methods

All patients diagnosed with UC between 2007 and 2010 in Oxford, and followed until September 2013, were used to develop the prediction model. Patients were identified from a registry of patients and centrally located case notes. UC diagnosis was based on standard clinical, endoscopic and histopathological criteria.² The primary outcome was ASC needing hospitalization, defined by Truelove andand Witts’ criteria¹ and confirmed for each case from contemporary records. Two independent cohorts from Cambridge [UK] and Uppsala [Sweden] were used for external validation. The Cambridge cohort included all patients diagnosed with UC over a similar period, and the Uppsala cohort was part of a population-based study of inflammatory bowel disease [IBD] during 2005 and 2009, including all incident cases of UC. Exclusion criteria included age less than 16 years, or hospitalization with ASC within 1 month of diagnosis, since a predictive index should predict a future event, not an event that might be in the process of happening.

Generic ethical permission was granted [Oxford: 09/H1204/30; Uppsala: 2006/173], but as a retrospective service evaluation of diagnostic assessment, specific permission from individual patients was not sought [Cambridge].

2.1. Risk factors

Age, gender, extent of disease at first evaluation according to the Montréal classification,¹⁷ haemoglobin [g/dl]at diagnosis, CRP [mg/l] at diagnosis and endoscopic and histological severity at diagnosis before treatment, were selected as potential risk factors after literature review and considering clinical simplicity. Disease extent was defined at first colonoscopy if the proximal extent had not been identified at an initial flexible sigmoidoscopy; some patients may have been on treatment at the time of first colonoscopy. Endoscopic severity was documented but later discarded, because diagnosis occurred before the advent of a reproducible index such as the Ulcerative Colitis Endoscopic Index of Severity [UCEIS]¹⁸ and there was no consistency between the sites. Histological severity [mild, moderate, or severe in the most severely affected area, according to Truelove and Richards,¹⁹ was re-scored from retained sections [LMY]. Because of small numbers in the ‘Mild’ category [11 patients with ASC], ‘Mild or Moderate’ were combined into one group to enhance simplicity, possibly at the expense of sensitivity. Data were collected on white cell count, platelets and albumin, but later excluded from analysis because they substantially reflect the CRP. Also excluded were treatment at diagnosis [choice of therapy is contingent on many variables and subjective factors make it inappropriate for an objective index], smoking [unlikely to be a constant variable during follow-up and unrelated to disease progression,²⁰ presence of extraintestinal manifestations, family history of IBD, and appendicectomy [each because of small patient numbers].

2.2. Validation cohorts

The performance of a prediction model can be evaluated using internal and external validation. External validation is the strongest test of a model, as it entails using independent data to evaluate the performance of a model, which we present using data from Cambridge and Sweden. Internal validation, which is used to assess any overfitting, entails evaluating the model on the same population from which the model was developed, using either cross-validation or more efficient bootstrapping [below]. Criteria for inclusion in the two independent cohorts were the same as the development cohort: patients diagnosed and followed up for at least 3 years, excluding those hospitalized with ASC within a month of diagnosis. Patients were identified from contemporary disease registries, without strict adherence to Truelove and Witts’ criteria for hospitalization: all those hospitalized met the criteria, but some who met criteria were not admitted.

2.3. Statistical methods

Sample size considerations for developing multivariable prediction models are based on the ratio of number of outcome events to the number of risk factors examined, referred to as events per variable [EPV]. A minimum EPV of between 5 and 10 is recommended to avoid overfitting.^21,22 Consequently analysis was restricted to seven factors: 34 patients in the development cohort were diagnosed with ASC, leading to an EPV of 5. Multiple imputation was used to replace missing values to maximize available information and reduce potential bias introduced by deleting incomplete records [CRP: 24% and 7% missing; Hb 21% and 7% missing in Oxford and Cambridge cohorts, respectively]. There were no missing data in the final model in the Uppsala cohort, reflecting differences between paper and electronic records. Logistic regression was used to investigate contributions for each risk factor to the prediction of ASC. Inclusion of risk factors was pragmatic, rather than statistical significant [notably Hb], based on their likelihood of being recorded at diagnosis and robustness [i.e. least subjective and low likelihood of disagreement in records].

Performance of the prediction model was assessed by examining measures of discrimination and calibration. Discrimination [whether the relative ranking of individual predictions is in the correct order] was quantified by the concordance index [c-index], equivalent to the area under the receiver operating characteristic curve.²³ A c-index of 0.5 indicates no discrimination, whereas a c-index of 1.0 indicates perfect discrimination. Calibration is the accuracy of agreement of the predicted probability of the outcome provided by the model with the observed frequency of the outcome. Internal validation of the prediction model was assessed using bootstrapping, which provided validation through a bootstrap biased-corrected c-index.^24,25 The entire modelling process was repeated, including variable selection in 200 bootstrap samples with replacement from the original sample.

A simplified index was created by assigning integer points to predictors in the final model. A nomogram [which is simply a graphical representation of the prediction model] to allow individualized risk prediction was developed based on the final logistic regression model for use in clinical practice. All analyses were performed using the R package [version 3.0.1] with the RMS library. We followed TRIPOD guidelines for reporting prediction model studies.^26,27

3. Results

3.1. Patient demographics

Features at diagnosis are shown [Table 1]. Each cohort is separated into patients who were and were not hospitalized for ASC within 3 years from diagnosis. In the Development cohort, out of 1850 patients with UC followed up in Oxford, most had been diagnosed outside Oxford; 212 had been diagnosed in Oxford before 2007 and 96 after 2010; 119 patients met inclusion criteria of diagnosis between 2007 and 2010, but eight patients were hospitalized with ASC at or within 1 month of diagnosis and so were excluded. Out of 111 patients included in the study [median follow-up 46 months, range 36–60], 34/111 [31%] were admitted with ASC at least once within 3 years from diagnosis [median 14 months after diagnosis, range 1–29]. All patients with ASC received standard intensive therapy according to the Oxford regimen;³ 14/34 [41%] achieved remission, 14/34 patients received infliximab or ciclosporin and 4/14 underwent surgery for rescue therapy failure. Of the 34, 6 patients underwent surgery without rescue therapy, giving a colectomy rate of 10/34 [29%]. The overall colectomy rate in the cohort was 13/111 [12%], which included three patients who had a colectomy outside an admission with ASC [one patient previously admitted with ASC went to colectomy for relapse 1 year after achieving remission on ciclosporin; two patients were never admitted with ASC, but had surgery for medically refractory UC].

Table 1.

Clinical characteristics of patients in the development and validation cohorts.

Characteristic at diagnosis		Oxford development [n = 111]		Cambridge validation [n = 96]		Uppsala validation [n = 298]
		ASC within 3 years of diagnosis n = 34 [31%]	No ASC within 3 years of diagnosis n = 77 [69%]	ASC within 3 years of diagnosis n = 25 [26%]	No ASC within 3 years of diagnosis n = 71 [74%]	ASC within 3 years of diagnosis n = 18 [6%]	No ASC within 3 years of diagnosis n = 280 [94%]
Median age [IQR]	years	37.4 [23.5, 42.8]	38.9 [25.8, 50.8]	48 [28, 63]	34 [25, 52]	30.5 [22.5, 54.8]	36 [25, 54.3]
Gender	M	15 [44%]	35 [45%]	15 [60%]	44 [62%]	9 [50%]	123 [44%]
	F	19 [56%]	42 [55%]	10 [40%]	27 [38%]	9 [50%]	157 [56%]
Extent	E1	1 [3%]	18 [23%]	0	18 [25%]	0	111 [40%]
	E2	17 [50%]	42 [56%]	5 [20%]	44 [62%]	3 [17%]	153 [55%]
	E3	16 [47%]	16 [21%]	20 [80%]	9 [9%]	15 [83%]	16 [5%]
Median CRP [IQR]	mg/l	14 [9.3, 43.8]	3.5 [2, 11]	23 [17, 36]	3 [2, 6]	19 [11, 53]	7 [3, 10]
Median haemoglobin [IQR]	g/dl	12 [11.3, 13.9]	13.4 [12.5, 14.6]	11.2 [10, 12]	13.5 [13, 14]	10.4 [10,11]	13.9 [13, 15]
Oral steroid therapy at diagnosis	[Yes]	27/77 [35%]	31/34 [91%]	Data not collected, since this did not form part of the predictive model
Endoscopy appearance	Mild/moderate	19 [56%]	74 [96%]
	Severe	15 [44%]	3 [4%]
Histopathology	Mild/moderate	20 [59%]	60 [78%]
	Severe	14 [42%]	17 [22%]
ASC outcomes	Complete response to steroids		14/34 [41%]
	Overall colectomy rate		10/34 [29%]

ASC, acute severe colitis; M, male; F, female; IQR, interquartile range.

3.2. Univariate and multivariable logistic regression

Results of univariate and multivariable logistic regression analysis of risk factors in the Development cohort are summarized in Table 2. Univariate analyses showed no significant difference for age and the significance of histological severity was borderline. All other factors showed statistically significant differences at diagnosis between patients subsequently hospitalized with ASC within 3 years and those not hospitalized.

Table 2.

Results of univariate and multivariable logistic regression analysis.

Characteristic	Univariate odds ratio [95% CI]	Multivariable odds ratio [95% CI]
Age [years]	0.99 [0.97, 1.02]	-
Extent [E3 vs E1/E2]	3.33 [1.40, 8.07]	2.91 [1.01, 8.31]
CRP [mg/l]	1.03 [1.01, 1.05]	1.02 [1.00, 1.04]
Haemoglobin [g/dl]	0.66 [0.47, 0.88]	0.76 [0.54, 1.06]
Endoscopic appearance [mild/moderate vs severe]	0.09 [0.02, 0.31]	0.11 [0.0, 0.67]
Histopathology [mild/moderate vs severe]	0.42 [0.17, 1.00]	1.04 [0.27, 4]

CI, confidence interval.

Multivariable logistic regression confirmed that extent, CRP and haemoglobin were significant predictors of ASC.

3.3. Initial model

The initial multivariable model [which included endoscopy, later discarded, see Methods] showed good discrimination, with a c-index of 0.84. The calibration plot [Figure 1a] grouped patients with similar predicted risks into fifths, from lowest to highest risk.

Figure 1. a.

Calibration plot for the multivariable logistic regression in the Development cohort. b. Calibration plot for the final model. These are complex figures for non-specialists that indicate how accurate the model predictions are.^26,27 Outcomes were grouped by fifths of predicted risk. The plots show the agreement between predictions from the model and what was actually observed. The mean predicted risk for each group and the mean observed frequency were calculated, and are plotted as triangles. The lines above and below the triangles are 95% confidence intervals. Perfect predictions should lie on the dashed line [--]. The dotted line […] is a smoothed regression line that broadly lies around the dashed line of perfect fit, indicating good calibration. The dotted line shows this at an individual patient level. At the bottom of the graph, the vertical lines above and below the horizontal line labelled 1 and 0, represent a histogram of predicted risks by whether the patients have ASC [labelled 1: going up] or not [labelled 0: going down].

3.4. Final prediction model

The final model included three factors: extent of disease, CRP and haemoglobin at diagnosis. The model still showed good discrimination [c-index = 0.77]. Bootstrapping for internal validation showed good calibration [Figure 1b].

To create an index of practical value in outpatients, the final predictive index assigned one point for each component: 0ne point for E3 extent of disease, one point for CRP > 10mg/l and one point for haemoglobin < 14g/dl [men], or < 12g/dl [women]. Values were selected for simplicity in practice: the most accurate predictive model identified haemoglobin thresholds of 13.8g/dl for men and 12.1g/dl for women, but rounding the numbers enhanced simplicity and did not alter the predictive value. This arbitrary decision is consistent with the goal of creating an index that is simple, potentially memorable and robust. An increasing number of components at diagnosis increased the predicted risk of being hospitalized with ASC within 3 years [Table 3]: a score 3/3 at diagnosis identified a 69% risk (interquartile range [IQR] 61–82%) in the Development cohort.

Table 3.

Risk of ASC predicted by the index in the Oxford Development and external Validation cohorts.

Index score	0/3	1/3	2/3	3/3
Number of patients
Oxford [n = 111]	43	35	23	10
Cambridge [n = 96]	36	27	15	18
Uppsala [n = 298]	157	93	35	13
Number with ASC [%]
Oxford	4/43 [9%]	8/35 [23%]	15/23 [65%]	7/10 [70%]
Cambridge	0/36 [0%]	2/27 [7%]	5/15 [33%]	18/18[100%]
Uppsala	0/157 [0%]	1/93 [1%]	5/35 [14%]	12/13 [92%]
Median predicted risk
Oxford	12%	25%	48%	69%
Cambridge	12%	20%	47%	68%
Uppsala	11%	21%	36%	72%
Interquartile range
Oxford	10 to 16%	11 to 30%	36 to 56%	61 to 82%
Cambridge	9 to 17%	18 to 28%	36 to 53%	63 to 78%
Uppsala	9 to 16%	17 to 27%	24 to 46%	64 to 81%

ASC, acute severe colitis.

3.5. Validation cohorts

Patients in Cambridge had a similar rate of hospitalization [n = 25/96, 26%] to Oxford [31%] during the median 3 years’ follow-up after diagnosis, but hospitalization for patients in Uppsala [n = 18/296, 6%] was uncommon [Table 1]. The ability of the index to discriminate between those who would or would not be hospitalized with ASC [c-index, where 1.0 = perfect discrimination] was 0.81 [Oxford], 0.95 [Cambridge] and 0.97 [Uppsala]. The distribution of predicted risks [from score = 0/3 to 3/3] was similar across all three cohorts, indicating good calibration and the ability to identify those at low or high risk of developing ASC [Table 3 and Figure 2].

Figure 2.

The risk of developing acute severe colitis in the 3 years after diagnosis according to the predictive index. The predictive index (whereby one point is assigned at diagnosis for E3 extent of disease, one point for CRP > 10mg/l and 1 point for haemoglobin < 14g/dl [men], or < 12g/d [women], range 0–3) was evaluated in three cohorts: Oxford, Cambridge and Uppsala [Sweden]. The x/N on top of each bar denotes the number of events over the number of individuals with a particular score, so 4/43 means [in the Oxford cohort] that 43 scored 0 of whom four developed ASC.

3.6. Nomogram for individual risk

In order to calculate individual risk more precisely, a nomogram was constructed [Figure 3] to enable the results for a patient to correspond with predictive points, by reading off the points on the top axis. The points for each component are summed to calculate ‘total points’, estimating the probability of ASC for the individual patient over the next 3 years from diagnosis by drawing a line down from the ‘total points’ axis.

Figure 3.

Nomogram to predict the risk of developing ASC over the next 3 years from diagnosis of UC for individual patients. For example, consider a male patient with pancolitis at diagnosis [E3: 28 points], a CRP of 23mg/dl [15 points] and haemoglobin 11.4g/dl [53 points], giving a total of 93 points. Check off the total score on the total points line and the individual risk of ASC is ~ 60%. The exact formula for calculating an individual risk of developing ASC over the next 3 years is the logistic regression equation p = 1/[1+exp[-[2.87 + 1.01*[Extent = E3] + 0.02*CRP - 0.35 *Hb]]]. Using the equation, the patient in the example would have a predicted risk of 1/[1+exp[-2.87+1.01+0.02*23-0.35*11.4]] = 0.61 [i.e. risk of 61%].

4. Discussion

This study shows that a simple index of three components [extent of disease, CRP and haemoglobin] applied at diagnosis reliably predicts the risk of ASC within the following 3 years in different populations. It implies that extensive, severe disease at diagnosis is associated with a worse prognosis, consistent with clinical experience. It potentially allows objective identification of a group of patients who need intensive monitoring or early intervention with the most effective therapy.

The simplicity of the index, using data that are readily available and almost invariably recorded, is particularly appealing. To create the simplest model that accurately predicted the risk of ASC in 69% of patients after diagnosis in the Development cohort [72% in the external validation cohort in Uppsala], one point was applied to a threshold for each component [extent of E3, CRP > 10mg/l, and haemoglobin < 14g/dl for men or < 12g/dl for women]. This is readily remembered and can be applied in outpatients. The risk score from 0/3 to 3/3 achieved predictive ability and good calibration. It was also possible to develop a more complex nomogram for individual patients [Figure 1]. It is unusual for a tool to relate risk specifically to an individual and this may conceivably help convince individual patients of a need for more proactive management.

It is interesting that the risk of hospitalization differed widely in Sweden [6% of 298 patients], compared with Cambridge [26% of 96 patients] or Oxford [31% of 111 patients]. This suggests different management strategies or different degrees of disease severity. The predictive index performed equally well in all three centres: a score of 3/3 accurately predicted subsequent hospitalization in every patient [18/18] in Cambridge and 93% [12/13] patients in Uppsala. Data at diagnosis [Table 1] show that severity may indeed differ: the proportion with extensive [E3] disease was 32/111[29%] and 29/96 [30%] in Oxford and Cambridge, but 31/298 [10%] in Uppsala. A significantly higher proportion had proctitis in Uppsala [E1 extent 111/298] compared with Oxford [19/111] or Cambridge [18/96], which reflects the population-based cohort in Sweden. This is consistent with it being the biological severity of UC, rather than management strategy or admission threshold, that predicts hospitalization and may account for the low rate of colectomy in the Swedish ICURE cohort [2.5% in the first year after diagnosis],²⁸ compared with Oxford.³ Haemoglobin and CRP are biological markers of severity, which gives further reason to believe that the index will have predictive value in practice. Whether the same thresholds apply to patients with UC in populations with a lower haemoglobin [eg India or East Asia] than the West, remains to be established. A key strength was demonstrating good performance of the index in three independent cohorts with different patient characteristics and rates of hospitalization.

There are limitations. Most notably, the study was retrospective, so patients diagnosed in the centres may have been missed or lost to follow-up. Such patients are more likely to have less severe disease. Although retrospective, the criteria are completely objective. Furthermore, external validation of the index in two independent cohorts tested the model. Proportions in Oxford and Cambridge who developed ASC and outcomes with regard to rescue therapy or colectomy are consistent with published data.^3,4,29 A more complex model derived from weighting of individual items after multivariable analysis might have performed better, but predictive models are best kept simple if they are to be used. The index identified 70-100% of patients at diagnosis who were hospitalized for colitis within 3 years, which is sufficient for clinical practice. Data collection excluded details of therapy, because choice constitutes too many variables: not only the drug, dose and duration, but also the individual physician, patient and preference. Nevertheless, steroids at diagnosis in 91% of those who later developed ASC [compared with 35% who did not] in the Development cohort [Table 1] is consistent with the premise that biologically severe disease at diagnosis has a worse outcome. The timing to assess disease extent also varied. Extent was defined at first colonoscopy, but precise timing [dependent on severity and access to colonoscopy] was not captured.

The index lends itself to assessment in an interventional study, since the principal value of an index is not to prognosticate but to guide treatment. Patients who score 3/3 at diagnosis of UC are candidates for early intervention with immunomodulators to reduce the risk of ASC. Effectiveness and safety need testing in a multicentre, randomixed controlled trial. Validation in different populations, including India or East Asia, would be informative. Meanwhile this index can be used to identify patients at diagnosis of UC as at high risk, who may need more proactive management than others.

Funding

This work was supported by the Translational Gastroenterology Unit, Nuffield Department of Experimental Medicine, Oxford, UK. Dr Cesarini was supported by a grant from IG-IBD [Italian Group for Inflammatory Bowel Disease].

Conflict of Interest

None.

Author Contributions

ST, SK, MC and GC designed the study, MC, AR and AS collected data under the supervision of ST, MP and DS. LMY performed the histological assessment. GC performed statistical analyses. The manuscript was written by MC and ST; all authors contributed to revised versions and reviewed the final manuscript. The manuscript is not under consideration for publication elsewhere.