Modeling Endoscopic Improvement after Induction Treatment with Mesalamine in Patients with Mild-to-Moderate Ulcerative Colitis

Abstract

Background and Aims:

Endoscopic improvement is an important treatment target for mild-to-moderate ulcerative colitis (UC). However, early endoscopic evaluation is not always feasible. We aimed to develop a clinical decision support tool to discriminate patients who have achieved endoscopic improvement from those with more severe inflammation following mesalamine induction therapy.

Methods:

We performed a post-hoc analysis of data from a phase 3 non-inferiority trial of 726 adults with mild-to-moderate UC treated with mesalamine. Multivariable logistic regression modeling determined associations between candidate variables and endoscopic improvement (Mayo endoscopic subscore=0–1 according to blinded central reading) at Week 8. Internal model validation was performed using bootstrap resampling. A clinical decision support tool was developed to stratify patients into low, intermediate, and high probability groups for endoscopic improvement.

Results:

Variables associated with endoscopic improvement at Week 8 included 50% reduction in fecal calprotectin from baseline (odds ratio [OR] 2.64, 95% CI: 1.81, 3.85), reduction in rectal bleeding (OR 1.79 per point reduction, 95% CI: 1.35, 2.89), and improvement in physician global assessment (OR 2.32 per point improvement, 95% CI: 1.88, 2.85). The baseline Geboes score (OR 0.74 per grade, 95% CI: 0.65, 0.85) and prolonged disease duration (OR 0.95 per year, 95% CI: 0.92, 0.98) were negatively associated with endoscopic improvement. This model strongly discriminated endoscopic improvement in the development dataset (area under the curve [AUC] 0.84, 95% CI: 0.81, 0.87) and during validation (AUC 0.83).

Conclusions:

We developed and validated a clinical decision support tool that has good discriminative performance for induction of endoscopic improvement in patients with mild-to-moderate UC treated with mesalamine.

INTRODUCTION

Most patients with ulcerative colitis (UC) have a mild-to-moderate disease course, characterized by active symptoms at diagnosis followed by variable periods of remitting and relapsing inflammation.¹ While biologic and novel small molecule therapies have improved the prognosis of patients with moderate-to-severe UC, the mainstay of treatment in milder disease remains oral and topical aminosalicylates. Clinical practice guidelines recommend the use of mesalamine suppositories or enemas in patients with proctosigmoiditis or left-sided colitis, respectively, and oral mesalamine or azo-bonded 5-aminosalicylates for extensive colitis, both as induction and maintenance treatment for patients with mild-to-moderate UC.^2–4 These guidelines are reflected in data from large cohort studies indicating that 73% to 97% of UC patients are exposed to aminosalicylates within the first year of diagnosis.^{5, 6}

During induction treatment, an important goal is to rapidly achieve symptomatic remission, which is characterized by normal stool frequency and resolution of rectal bleeding.⁷ However, inducing symptomatic improvement alone is insufficient, and endoscopic improvement has been identified as a critical treatment target for patients with UC.^2–4 While there is consensus that mucosal assessment should be performed when making important management decisions such as initiating therapy, assessing induction response, and prior to changing treatment, there may be practical limitations to repeated endoscopy. In an analysis of the Truven Health MarketScan Commercial Database, only one quarter of UC patients underwent objective mucosal evaluation within six months of starting biologic therapy.⁸ Comparatively, patients with mild UC may be even less accepting of an invasive intervention such as endoscopy to evaluate treatment response when the symptoms are often less severe, notwithstanding resource implications of routine repeat endoscopic assessment. Finally, in many jurisdictions, the availability of endoscopy has been significantly reduced because of the COVID-19 pandemic.⁹

Hence, there is a clear need for an accurate, non-invasive method for determining the probability of endoscopic improvement after treatment with mesalamine in patients with mild-to-moderate UC. Here, we developed and validated a clinical decision support tool using data from a large, phase 3, randomized controlled non-inferiority trial.¹⁰

MATERIALS AND METHODS

Data and Participants

We used data from an active-comparator, multi-center, randomized, non-inferiority trial of mesalamine ( Clinicaltrials.gov NCT01903252). The study design and trial outcomes have been previously reported.¹⁰ Adults with mild-to-moderate UC, defined by a total Mayo Clinic Score¹¹ (MCS) ≥5, rectal bleeding subscore ≥1, and Mayo endoscopic subscore (MES) ≥2 at baseline were randomized to receive 3.2 g of oral mesalamine, administered as either two 1600 mg tablets once daily or four 400 mg tablets twice daily for eight weeks. The primary efficacy outcome was clinical and endoscopic remission at Week 8. Eligibility and endoscopic outcomes were evaluated by a single blinded central reader. A total of 726 patients completed the induction study and had endoscopic evaluation at Week 8 suitable for post-hoc analysis.

Covariables

Candidate items potentially associated with endoscopic improvement were selected a priori. Both items at baseline and changes in item scores between baseline and Week 8 were considered. We included: a) clinical parameters: disease extent/duration, smoking status, MCS and its component items (rectal bleeding, stool frequency, and physician global assessment [PGA]), and patient subjective global rating; and b) biological parameters: age, sex, weight, baseline histological Geboes score¹², and change in fecal calprotectin. The Geboes score assesses the severity of histological change as grade 0 (structural change only), 1 (chronic inflammatory infiltrate), 2 (lamina propria neutrophils or eosinophils), 3 (neutrophils in the epithelium), 4 (crypt destruction), and 5 (erosions/ulceration). A linear relationship was observed between the Geboes grade and endoscopic improvement; this assumption was carried forward in modeling. There was no statistical difference by the likelihood ratio test in models with the Geboes score treated categorically vs. linearly. We evaluated fecal calprotectin as a candidate item using multiple cut-offs at Week 8 (<250 μg/g, <150 μg/g, <50 μg/g) and using different thresholds for change (Δ −25%, Δ −50%, Δ −75%). In exploratory analyses, we evaluated simplified models consisting of absence of rectal bleeding, fecal calprotectin, and PGA alone.

Endoscopic improvement defined by MES=0/1 was chosen a priori as the dependent variable. This is currently the recommended treatment target in UC, was the pre-specified outcome in the clinical trial, and is in accordance with regulatory recommendations.¹³ Outcome definitions in UC have evolved over time¹⁴; we used the term endoscopic improvement to differentiate this from modern definitions of endoscopic remission (MES=0) and mucosal healing (endoscopic and histologic remission).

Statistical Analysis

Multivariable logistic regression modeling was performed to evaluate the association between candidate items and endoscopic improvement expressed as adjusted odds ratios (OR) with 95% confidence intervals (CI). Covariable selection was based on backward stepwise multivariable elimination, retaining only statistically significant (p<0.05) items. The accuracy of the model was assessed using: 1) discrimination, measured by area under the receiver operator curve (AUC); 2) prediction error, measured by Nagelkerke’s R-squared¹⁵ and the Brier score¹⁶; and 3) calibration, assessed by the Hosmer-Lemeshow goodness-of-fit test.¹⁷ Internal model validation was conducted using the optimism-bootstrap resampling technique with 300 replications to determine reproducibility. An optimism-corrected AUC was reported to account for apparent over-fitting and used to construct a receiver operator curve.¹⁸

The multivariable model was used to calculate the probability of endoscopic improvement in an individual patient after eight weeks of mesalamine. The scoring method for the clinical decision support tool was developed by converting coefficients from the multivariable model to a points system.¹⁹ Cut points for low, intermediate, and high probability groups were determined based on quartiles of predicted probability of endoscopic improvement: low probability was defined as ≤ 1^st quartile (Q1) vs. high probability was defined as ≥ 3^rd quartile (Q3). The sensitivity, specificity, positive and negative predictive value, and positive and negative likelihood ratio for the high and low probability groups was calculated.

Statistical analyses were performed in SAS (version 9.4) and R (version 3.3.1).

RESULTS

Patients

A total of 726 patients were included and 436 patients (60.1%) achieved endoscopic improvement at Week 8. Mean disease duration was 5.3 years (standard deviation ±6.5 years) and mean baseline fecal calprotectin was 1,108 μg/g (±1,546 μg/g). Mean rectal bleeding subscore was 1.5 (±0.5) and mean baseline Geboes score was 4.3 (±1.3).

Modeling Endoscopic Improvement

Univariable associations with endoscopic improvement are summarized in Supplemental Table 1. Multivariable logistic regression analyses are presented in Table 1. Patients with >50% reduction in fecal calprotectin from baseline (OR 2.64 [95% CI: 1.81–3.85], p<0.0001), reduction in rectal bleeding subscore (OR 1.79 per point reduction [95% CI: 1.35–2.39], p<0.0001), or reduction in PGA subscore (OR 2.32 per point reduction [95% CI: 1.88–2.85], p<0.001) had increased odds of achieving endoscopic improvement. Factors associated with a lower likelihood of endoscopic improvement were prolonged disease duration (OR 0.95 per year [95% CI: 0.92–0.98], p=0.001) and more active histologic disease (OR 0.74 per Geboes grade increase [95% CI: 0.65–0.85], p<0.0001). Sex, age, disease extent and change in stool frequency after treatment were not associated with endoscopic improvement in multivariable analysis.

Table 1.

Multivariable logistic regression diagnostic model for Week 8 endoscopic improvement

Variable	Adjusted Odds Ratio	95% Confidence Interval	P value
>50% reduction in fecal calprotectin from baseline	2.64	1.81 – 3.85	<0.0001
1-point reduction in rectal bleeding subscore	1.79	1.35 – 2.38	<0.0001
1-point reduction in physician global assessment	2.32	1.88 – 2.85	<0.0001
Baseline Geboes score (per 1-grade increase)	0.74	0.65 – 0.85	<0.0001
Disease duration (per year)	0.95	0.92 – 0.98	0.001

The multivariable model had good discriminative ability, with an AUC of 0.84 [95% CI: 0.81–0.87] in the development dataset. The optimism corrected AUC in internal validation using bootstrap validation was 0.83 (Figure 1). Model fit was adequate based on the Hosmer-Lemeshow goodness-of-fit test (p=0.894). Other model performance measures are summarized in Supplemental Table 2.

Figure 1.

Optimism-adjusted receiver operator curve for multivariable model prediction of endoscopic improvement

In comparison to using fecal calprotectin alone, the optimism corrected AUC of the multivariable model (0.83) was higher than for any of the prespecified week 8 thresholds (<250 μg/g [AUC 0.72], <150 μg/g [AUC 0.71], <50 μg/g [AUC 0.68]) and any of the prespecified change from baseline benchmarks (Δ −25% [AUC 0.66], Δ −50% [AUC 0.67], Δ −75% [AUC 0.67]) (Supplemental Table 3). The multivariable model also discriminated patients with endoscopic improvement better than using rectal bleeding and fecal calprotectin (AUC 0.79), or rectal bleeding and fecal calprotectin with PGA (AUC 0.80).

Clinical Decision Support Tool

A scoring chart for each item included in the multivariable logistic regression model is presented in Table 2 and the probability of endoscopic improvement using the clinical decision support tool is summarized in Figure 2. Patients with a score >95 had a >94% probability of achieving endoscopic improvement at Week 8. In contrast, patients with a score <40 had a <5% probability of endoscopic improvement. Scores of patients included in the clinical trial ranged from 17 (probability of endoscopic improvement 0.005%) to 97 (probability 95.7%). The model cut points for low, intermediate, and high probability groups based on quartiles were ≤50 points, 51–75 points, and >75 points. The diagnostic accuracy of the model cut points is described in Table 3. A score ≤50 points demonstrated high specificity (96%) for the identification of patients without endoscopic improvement. Conversely, a score >75 points demonstrated high specificity (92%) for endoscopic improvement. A total of 190 patients had a low probability score; 179/190 (94.2%) UC patients in this group did not achieve endoscopic improvement. A total of 155 patients had a high probability score (>75 points); 122/155 (78.7%) UC patients with a high probability score achieved endoscopic improvement.

Table 2.

Score chart for predicting endoscopic improvement at Week 8

Variable	Score Component Value
Fecal calprotectin
≤50% reduction from baseline	0
>50% reduction from baseline	+9
Disease duration (years)
0 – <5 years	+7
5 – <10 years	+5
10 – <15 years	+2
≥15 years	0
Change in rectal bleeding subscore
Increase 2 points	0
Increase 1 point	+6
No change	+11
Reduction 1 point	+17
Reduction 2 points	+22
Change in physician global assessment (PGA)
Increase in PGA 3 points	0
Increase in PGA 2 points	+8
Increase in PGA 1 point	+16
No change in PGA	+24
Reduction in PGA 1 point	+32
Reduction in PGA 2 points	+40
Reduction in PGA 3 points	+48
Baseline Geboes score
Grade 0 (architectural change)	+14
Grade 1 (chronic inflammatory infiltrate)	+11
Grade 2 (lamina propria neutrophils/eosinophils)	+8
Grade 3 (epithelial neutrophils)	+6
Grade 4 (crypt destruction)	+3
Grade 5 (erosion/ulceration)	0

Abbreviations: PGA, physician global assessment

Figure 2.

Probability of endoscopic improvement by clinical decision support tool score

Table 3.

Diagnostic accuracy of the clinical decision support tool

Cut Off Score	Sensitivity [95% CI]	Specificity [95% CI]	PPV [95% CI]	NPV [95% CI]	PLR [95% CI]	NLR [95% CI]
≤50	41 [6–46]	96 [93–98]	94 [90–97]	52 [48–56]	10.82 [6.00–19.53]	0.61 [0.56–0.66]
>75	42 [36–48]	92 [90–95]	79 [71–85]	71 [67–74]	5.56 [3.90–7.92]	0.63 [0.57–0.69]

Abbreviations: CI, confidence interval; NLR, negative likelihood ratio; NPV, negative predictive value; PLR, positive likelihood ratio; PPV, positive predictive value.

DISCUSSION

Although targeting endoscopic endpoints is associated with improved long-term outcomes in patients with UC, repeated lower endoscopy to evaluate disease activity is time-intensive, costly, and invasive for patients. Furthermore, many practices are facing restricted endoscopy availability due to COVID-19, leading to a situation where many patients with mild-to-moderate UC will not undergo repeat endoscopy to measure treatment response to mesalamine. While existing cross-sectional studies have examined the relationship between patient reported outcomes and endoscopy,²⁰ the discordance between these measures in mild-to-moderate UC highlights a need for accurate methods of non-invasively estimating the probability of endoscopic improvement for each individual patient.²¹ In this study, we developed and internally validated a multivariable model that discriminates patients who have achieved endoscopic improvement from those who have not using relevant clinical and biological parameters at baseline and change after 8 weeks of mesalamine induction therapy. This model was used to derive a clinical decision support tool that demonstrates high sensitivity for ruling out and high specificity for ruling in endoscopic improvement. Furthermore, this tool better discriminates endoscopic outcomes compared to rectal bleeding, fecal calprotectin, or PGA, either alone or in combination.

Reduction in fecal calprotectin by >50% compared to baseline was most strongly associated with endoscopic improvement in our model. We postulate that this is due to the correlation between fecal calprotectin and histologic inflammation in UC. Changes in fecal calprotectin have been previously demonstrated to be associated with endoscopic response, particularly after biologic treatment.²² In a post-hoc analysis of the phase 3 GEMINI 1 study, Reinisch et al. determined that a fecal calprotectin concentration ≤150 μg/g at Week 6 was associated with clinical (AUC 0.745) and endoscopic (AUC 0.746) remission.²³ Other studies have used variable calprotectin cut-offs, ranging from 50–250 μg/g, recognizing that there is a trade-off between sensitivity and specificity as this threshold changes.^{22, 24, 25} This may reflect differences in patient populations since baseline fecal calprotectin concentrations are lower in patients with mild-to-moderate UC compared to those with moderate-to-severe disease starting biologics. Therefore, for patients with a relatively low fecal calprotectin at baseline, small absolute reductions to <150–250 μg/g after induction may not represent a meaningful biological response. Importantly, using the clinical decision support tool better discriminated endoscopic improvement than using fecal calprotectin alone, or fecal calprotectin in combination with rectal bleeding.

In our model, reduction in rectal bleeding but not stool frequency was significantly associated with endoscopic improvement, reflecting the relatively high degree of correlation between rectal bleeding and mucosal inflammation. Using the same non-inferiority trial population, we have previously shown that among UC patients with an MES of 0 or 1 at Week 8, only 16% (50/310) had ongoing rectal bleeding whereas over half of patients had persistently abnormal stool frequency.²⁶ It may be surprising that change in PGA between baseline and Week 8 retained a significant association with endoscopic improvement in multivariable regression. The PGA has come under scrutiny from regulatory bodies as scoring is considered to be arbitrary, and specific criteria for mild, moderate, and severe disease are poorly defined, leading to substantial inter-rater disagreement.^27–29 In guidance from the United States Food and Drug Administration, the PGA as a single general item, accounting for all pertinent clinical findings, was felt to be inadequate for capturing treatment benefit.¹³ As a result, an adapted 9-point MCS excluding the PGA has been used to evaluate disease activity in contemporary clinical trials.³⁰ Given these limitations, it seems paradoxical that the PGA would be retained in our model to a significance level <0.05. However, we emphasize that the predictive value was derived from the change in PGA from baseline to Week 8 rather than the baseline or Week 8 PGA scores alone. This suggests that while a single physician evaluation of disease activity may not be strongly associated with endoscopic outcome, the change in global assessment with treatment may have more power.

There are several clinical situations where our clinical decision support tool may be informative for making treatment decisions. The first scenario is to identify patients who are very unlikely to have achieved endoscopic improvement and should be considered for empiric treatment escalation. Take for example a patient with a new diagnosis of UC (+7 points), baseline fecal calprotectin of 500 μg/g, and ulceration on initial biopsies (+0 points). If the fecal calprotectin remains elevated at 400 μg/g (+0 points) and the patient has only mild improvement in symptoms (1-point reduction in rectal bleeding, +17 points, no change in PGA +24 points) after 8 weeks of mesalamine, then the probability of endoscopic improvement is just 11% (48 points). In such scenarios, patients are likely to benefit from empirically adding topical mesalamine therapy or maximizing the mesalamine dose (≥4 g/day). Whether clinicians and patients would be comfortable empirically escalating to immunosuppressive, biologic, or small molecule therapy is less clear given the approximately 1 in 7 chance of misclassification. In contrast, if the same patient had substantial improvement in fecal calprotectin to 150 μg/g (+9 points), 2-point reduction in rectal bleeding (+22 points), and 3-point normalization of the PGA (+48 points) after 8 weeks of mesalamine, the probability of endoscopic improvement would be 88% (86 points). In this situation, we speculate that most clinicians and patients would be comfortable deferring endoscopic evaluation.

Our study has several strengths. We used a large, well-characterized clinical trial dataset for model development and validation. The model is generalizable to patients with mild-to-moderate disease who are undergoing standard induction treatment with the most prescribed medication class in UC. Covariables were uniformly collected according to the trial protocol and all histologic and endoscopic evaluations were performed by a blinded central reader. Our study also has some important limitations. First, we performed internal, rather than external, model validation. Although external validation in a separate patient cohort is considered more rigorous to assess generalizability of the model in different settings, a sufficiently large trial dataset including the patient population and intervention of interest was not available. We specifically considered other approaches to internal validation: Steyerberg et al. compared data splitting, bootstrap resampling, and repeated cross-validation.³¹ Split sample analyses tend to give overly pessimistic estimates with large variability, and only resampling techniques can validate the entire procedure of model selection and estimation. Therefore, we performed internal validation using bootstrap resampling, which provides the most stable estimates with the lowest risk of bias and reported optimism-corrected AUC to mitigate over-fitting. Second, histological assessment after treatment was not available. Although we recognize the importance of histopathology for defining mucosal healing, this covariable would require a follow-up endoscopy to collect and measure, thus defeating the purpose of developing a non-invasive clinical decision support tool. We also acknowledge that neither the Geboes score nor other validated histopathology instruments for measuring disease activity are routinely reported in clinical practice. This poses several challenges to clinicians as generic terms such as ‘chronic active colitis’ are non-specific and ambiguous. Given the emerging and important role of histopathology in UC disease assessment and prognostication, more precise reporting of histology findings using validated instruments should be encouraged. Third, the mesalamine dose used in this trial was 3.2 g/day, whereas many clinicians use up to 4.8 g/day. While the ASCEND II trial demonstrated that treatment with 4.8 g/day was superior to 2.4 g/day for achieving Week 6 outcomes³², an intermediate dose was not studied and Kruis et al. have shown a potential threshold effect above 3 g/day.³³ Nevertheless, we hypothesize that a higher mesalamine dose may have increased the absolute number of responders, although is unlikely to have affected the factors associated with endoscopic improvement. Third, the range of predicted probabilities of endoscopic improvement in our model is quite broad, and the value assigned to each level of the selected covariables was different based on strength of association. However, this allowed our tool to have a wide dynamic range.

In conclusion, we have developed and internally validated a model for estimating the probability of endoscopic status in patients with mild-to-moderate UC following induction treatment with mesalamine. This tool will be clinically informative when endoscopy is unavailable or cannot be performed to directly visualize the mucosal response to treatment. Applying this score can discriminate patients who have a low probability of endoscopic improvement and may benefit from early treatment optimization from patients with a high probability of endoscopic improvement who may safely defer sigmoidoscopy or colonoscopy. This will improve the efficiency of non-invasive disease evaluation after induction beyond the use of fecal calprotectin, rectal bleeding, or PGA alone, although further research is required to independently validate this model in an external dataset and determine associations with long-term (>52 week) sustained endoscopic remission.

Supplementary Material

Supplementary file

Supplemental Table 1. Univariable associations of baseline and change from baseline covariables with endoscopic improvement

Supplemental Table 2. Diagnostic model performance measures

Supplemental Table 3. Diagnostic performance of fecal calprotectin for week 8 endoscopic improvement

Acronyms and Abbreviations:

AUC

area under the curve

CI

confidence interval

MCS

Mayo Clinic Score

MES

Mayo endoscopic subscore

OR

odds ratio

PGA

physician’s global assessment

Q1

1^st quartile

Q3

3^rd quartile

STRIDE

Selecting Therapeutic Targets in Inflammatory Bowel Disease

TRIPOD

Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis

UC

ulcerative colitis