Abstract
Background
Patients with Crohn’s disease (CD) are at risk of complications. Performance characteristics of a decision support tool assessing the risk of CD complications were evaluated.
Methods
CDPATH (formerly called the Personalized Risk and Outcome Prediction Tool [PROSPECT]) was calibrated and validated in 2 cohorts. Tool prediction of disease characteristics was assessed using Cox regression and Harrell’s C-statistic.
Results
All associations of CD complications and CDPATH components were significant except perianal location. There was a significant association between individualized risk assessment scores and CD complications in both cohorts.
Conclusion
CDPATH is validated as a clinical decision support tool for assessing the risk of CD complications.
Keywords: clinical decision support tool, complications, Crohn’s disease, risk assessment, validation
Introduction
Crohn’s disease (CD) presents anywhere in the gastrointestinal tract and in a variety of phenotypes.1 The goals of treatment for this complex disease are remission and prevention of complications and surgery.1 Management that is tailored to patients’ disease characteristics and risks of complications is needed to achieve these goals.2, 3 CDPATH (formerly called the Personalized Risk and Outcome Prediction Tool [PROSPECT]) was developed to assess individual risk of CD complications by modeling clinical, genetic, and serologic factors to identify dependent variables for the time from diagnosis to occurrence of the first complication of CD (defined as a bowel stricture, internal penetrating disease, or nonperianal surgery [bowel resection or strictureplasty]).2 For this investigation, we evaluated the performance characteristics of the CDPATH risk assessment tool in the clinical laboratory setting.
Methods
The risk assessment score (risk of a CD complication at 3 years) of the CDPATH algorithm was developed with a Cox proportional hazards model and univariable and multivariable analyses to model the association of clinical, serologic, and genetic biomarkers with risk of CD complications.2 Performance characteristics of the CDPATH tool were assessed with available data from a calibration cohort (Cedars-Sinai Medical Center, Los Angeles, CA; n = 106) and a validation cohort (Mount Sinai Hospital, University of Toronto, Toronto, ON, Canada; n = 32). All patients included in the study population were without CD complications at the time of entry into the cohort. Protocols were approved by institutional review boards at the enrolling institutions, and all participants provided informed consent.2
Individual risk scores were calculated as previously described.2 A Cox model was used to determine the hazard ratio (HR) for any patient in combination with the Breslow estimate of the cumulative hazard, which yields a predicted time-to-event curve. The risk of an individual patient is calculated according to the HR of each variable. The overall unit hazard function is the risk of a complication at any given time and is shown in the equation below:
The calculated risk of a complication at any given time is the product of the baseline function and the expression derived from the calculated HR of each variable (HRi) for the value of the variable. Variables included disease location (small bowel, left colon, perianal), serologic markers (anti–Saccharomyces cerevisiae antibodies [ASCA] immunoglobulin [Ig]A/IgG isotypes, anti-CBir1 IgG, perinuclear antineutrophil cytoplasmic antibodies [pANCA]), and a genetic marker (the NOD2 1007 frameshift mutation [NOD2 fs]). Risk was categorized as low (0%–19.9%), medium (20%–59.9%), and high (60%–100%). These risk cutoffs were developed in qualitative focus groups and cognitive interviews with patients with CD.2 Using think-aloud protocols, patients were asked what low, medium, and high risk of a complication meant to them and then were asked to quantify the percentage risk for each category. In subsequent focus groups, patients who have shown the cutoffs for low (0%–19.9%), medium (20%–59.9%), and high (60%–100%) risk agreed that they represented clinically meaningful decision points. One-on-one cognitive interviews were performed with 20 patients that indicated agreement and comprehension by patients. Finally, gastroenterologists were consulted to confirm face validity of these risk groupings, with universal agreement from participants.
Performance characteristics were assessed using Cox regression and Harrell’s C-statistic, which test the goodness of fit. CDPATH outputs were built with system dynamics analysis for real-time prediction of each patient’s disease characteristics.
Results
For the calibration and validation cohorts, respectively, mean age (standard error of the mean) was 36 (Cohort 1) years and 29 (Cohort 2) years (Table 1). Associations of CD complications within 3 years and disease location or serologic or genetic components of CDPATH were examined. All associations were significant (C-statistic ≥ 0.533; P ≤ .03) except for perianal location (C-statistic = 0.501; P = .95; Table 2). The highest C-statistic was for the risk assessment composite score, combining all components. Within 3 years, there was a significant association between individualized risk assessment scores and CD complications in the calibration cohort (C-statistic = 0.73; P < .001) and the validation cohort (C-statistic = 0.70; P = .015), respectively. The C-statistic for the combined cohorts was 0.713 ± 0.037.
Table 1.
Performance characteristics of a clinical decision support tool for disease complications in Crohn’s disease.
| Cohort 1 (Calibration) | Cohort 2 (Validation) | |
|---|---|---|
| N | 106 | 32 |
| Age, years, mean ± SEM | 36 ± 1 | 29 ± 2 |
| Sex, female, n (%) | 40 (38) | a |
| Ethnicity, n (%) | a | |
| White | 107 (95) | |
| African American | 2 (2) | |
| Hispanic | 4 (3) | |
| Time from diagnosis to specimen collection, years, mean ± SEM | 2.4 ± 0.3 | a |
| Time from specimen collection to complication, years, mean ± SEM | 2.7 ± 0.5 | a |
| Time from diagnosis to complication, years, mean ± SEM | 3.9 ± 0.6 | 1.2 ± 0.4 |
| Disease location, n (%) | ||
| Small bowel | 82 (77) | 28 (88) |
| Left colon | 63 (59) | 26 (81) |
| Perianal | 12 (11) | 11 (34) |
| Laboratory markers | ||
| pANCA status positive, n (%) | 20 (19) | 8 (25) |
| ASCA IgA, EU/mL, mean ± SEM | 21.6 ± 2.9 | 26.6 ± 6.2 |
| ASCA IgG, EU/mL, mean ± SEM | 17.9 ± 2.5 | 23.2 ± 5.1 |
| Anti-CBir1, EU/mL, mean ± SEM | 26.8 ± 2.5 | 33.1 ± 4.7 |
| NOD2 SNP13 (1007fs), n (%) | 8 (7) | 3 (9) |
Abbreviations: Anti-CBir1, antibodies to CBir1 flagellin; ASCA, anti–Saccharomyces cerevisiae antibodies; EU, ELISA units; Ig, immunoglobulin; NOD2 SNP13 (1007fs), frameshift mutation in the NOD2 gene; pANCA, perinuclear antineutrophil cytoplasmic antibodies; SEM, standard error of the mean.
Data are not available, but not required for model validation.
Table 2.
Characteristics for the combined cohorts overall and by risk category.
| Combined cohorts | Low-risk score (0%–19.9%) | Medium-risk score (20%–59.9%) | High-risk score (60.0%–100%) | HR estimate (95% CI) | C-stat (P value) | |
|---|---|---|---|---|---|---|
| Disease location | ||||||
| Small bowel, % (n/N) | 80 (110/138) | 30.7 (8/26) | 80.4 (41/51) | 100 (61/61) | 6.78 (1.65–27.94) | 0.588 ± 0.023 (<.01) |
| Left colon, % (n/N) | 64 (89/138) | 92.3 (24/26) | 68.6 (35/51) | 49.1 (30/61) | 0.38 (0.22–0.65) | 0.619 ± 0.037(<.01) |
| Perianal, % (n/N) | 17 (23/138) | 0 (0/26) | 15.7 (8/51) | 24.5 (15/61) | 0.98 (0.50–1.91) | 0.501 ± 0.031 (.95) |
| Laboratory markers | ||||||
| pANCA status positive, % (n/N) | 20 (28/138) | 50 (13/26) | 27.4 (14/51) | 1.6 (1/61) | 0.32 (0.13–0.81) | 0.587 ± 0.023 (.02) |
| ASCA IgA/IgG, EU/mL, ln, median (IQR) | 2.40 (1.39–3.47) | 1.25 (0.1–1.73) | 2.02 (1.34–2.66) | 3.46 (2.82–4.26) | 1.49 (1.20–1.84) | 0.643 ± 0.043 (< .01) |
| Anti-CBir1, EU/mL, ln, median (IQR) | 2.88 (2.29–3.59) | 2.37 (1.85–2.81) | 2.66 (2.08–3.48) | 3.40 (2.71–4.12) | 1.91 (1.37–2.65) | 0.645 ± 0.039 (< .01) |
| NOD2 SNP13 (1007fs), % (n/N) | 8 (11/138) | 0 (0/26) | 0 (0/51) | 18.3 (11/61) | 2.60 (1.09–6.21) | 0.533 ± 0.021 (.03) |
Abbreviations: Anti-CBir1, antibodies to CBir1 flagellin; ASCA, anti-Saccharomyces cerevisiae antibodies; C-stat, C-statistic; EU, ELISA units; HR, hazard ratio; Ig, immunoglobulin; IQR, interquartile ratio; ln, natural log; NOD2 SNP13 (1007fs), frameshift mutation in the NOD2 gene; pANCA, perinuclear antineutrophil cytoplasmic antibodies.
Among the 106 patients in the calibration cohort, 21.7%, 37.7%, and 40.6%, respectively, were categorized as low, medium, and high risk. Among the 32 patients in the validation cohort, 9.4%, 34.4%, and 56.2% were low, medium, and high risk, respectively. The performance characteristics of risk assessment scores were plotted by risk category in both cohorts (Figure 1). In the calibration cohort, the incidence of CD complications was 0%, 28%, and 58% among patients receiving the low-, medium-, and high-risk scores, respectively. Similarly, the incidence of CD complications was 0%, 36%, and 61%, respectively, for patients with low, medium, and high risk within the validation cohort.
Figure 1.
Performance characteristics of risk assessment scores plotted for patients with low, medium, and high levels of risk (defined as 0%–19.9%, 20.0%–59.9%, and ≥60%, respectively).
Discussion
We have verified the analytical and clinical validity and established the performance characteristics of this decision support tool for CD complications in the reference clinical laboratory setting. Running the model using all components, including clinical disease location and serologic and genetic factors, provided greater predictive accuracy for complications within 3 years compared with using each component separately. A limitation of this study is the lack of ethnic diversity in the calibration and validation cohorts, which may be an important factor to consider in the global applicability of this decision support tool. Although a genetic marker (the NOD2 gene frameshift variant) was included as a variable, exploring the performance of this tool according to the ethnicity of patients will require further analyses in a future study, such as the COMPASS CD registry (NCT04809363). However, these results demonstrate consistent and reproducible performance as shown by the individual risk assessment scores across the cohorts. Risk assessment scores from the CDPATH tool may facilitate a shared understanding of optimal management of CD between patients and physicians.
Acknowledgments
The authors thank Dr. Mark S. Silverberg of the Division of Gastroenterology, Mount Sinai Hospital Inflammatory Bowel Disease Centre, University of Toronto, Toronto, ON, Canada, for his contributions to the study and critical reading of the manuscript. Medical writing assistance was provided by Katherine Stevens-Favorite, PhD and Jennie G. Jacobson, PhD, CMPP, on behalf of Cadent Medical Communications, LLC, a Syneos Health™ group company. This assistance was funded by Takeda Pharmaceuticals U.S.A., Inc.
Author Contributions
Conceptualization: C.A.S., L.S.S., and M.C.D.; data curation: C.A.S., L.S.S., M.C.D., S.T., C.L., J. Bilsborough, and D.P.B.M.; formal analysis: C.A.S., L.S.S., M.C.D., T.D., J. Braun, and F.P.; funding acquisition: K.L. and N.C.; investigation: C.A.S., L.S.S., M.C.D., T.D., J. Braun, F.P., K.L., and N.C.; methodology: C.A.S., L.S.S., M.C.D., T.D., J. Braun, F.P., K.L., and N.C.; project administration: K.L. and N.C.; resources: K.L., N.C., and T.D.; software: C.A.S., L.S.S., M.C.D., T.D., J. Braun, and F.P.; supervision: K.L. and N.C.; validation: C.A.S., L.S.S., M.C.D., T.D., J. Braun, and F.P.; visualization: K.L., N.C., and T.D.; writing—original draft: K.L., N.C., C.A.S., L.S.S., M.C.D., and T.D.; writing—review and editing: all authors.
Conflicts of Interest
C.A.S. has received consulting/advising fees from AbbVie, Amgen, Celgene, Eli Lilly and Company, Janssen, Pfizer, Prometheus Biosciences, Sandoz, Sebela, and Takeda; has been a speaker for AbbVie, Janssen, Pfizer, and Takeda; has received grant support from AbbVie, the Agency for Healthcare Research and Quality (1R01HS021747-01), the Crohn’s & Colitis Foundation, Janssen, Pfizer, and Takeda; and has intellectual property for and is a cofounder of MiTest Health and ColonaryConcepts. L.L.S. has intellectual property for and is a cofounder of MiTest Health. M.C.D. has received consulting/advising fees from AbbVie, Arena, Boehringer Ingelheim, Bristol Myers Squibb, Celgene, Eli Lilly and Company, Genentech, Janssen, Pfizer, Prometheus Biosciences, Takeda, Target RWE, and UCB; has received grant support from AbbVie, Janssen, Pfizer, and Takeda; has intellectual property for MiTest Health and Trellus Health; owns stock in Trellus Health; is a cofounder of MiTest Health; and is founder of Cornerstones Health. S.T. is a cofounder of, consultant for, and a stockholder in Prometheus Biosciences. J. Braun and C.L. have no relevant disclosures. J. Bilsborough is a faculty member and employee of Cedars-Sinai Medical Center and a paid consultant and shareholder of Prometheus Biosciences, Inc.; has financial interests in Prometheus Biosciences, Inc., which has access to the data and specimens in Cedars-Sinai’s MIRIAD Biobank (including the data and specimens used in this study) and seeks to develop commercial products. F.P. and T.D. are employees of Prometheus Biosciences, Inc., and have stock or stock options. K.L. and N.C. are employees of Takeda Pharmaceuticals U.S.A., Inc., and have stock or stock options. D.P.B.M. has received consulting fees from Gilead Sciences, Boehringer Ingelheim, Pfizer, Bridge Biotherapeutics, Qu Biologics, Prometheus Biosciences, and Takeda, and grant support from Janssen and is a cofounder of, consultant for, and stockholder in Prometheus Biosciences.
Data Availability
Data are not publicly available.
References
- 1. Feuerstein JD, Cheifetz AS.. Crohn disease: epidemiology, diagnosis, and management. Mayo Clin Proc. 2017;92(7):1088–1103. [DOI] [PubMed] [Google Scholar]
- 2. Siegel CA, Horton H, Siegel LS, et al. A validated web-based tool to display individualised Crohn’s disease predicted outcomes based on clinical, serologic and genetic variables. Aliment Pharmacol Ther. 2016;43(2):262–271. [DOI] [PubMed] [Google Scholar]
- 3. Tsui JJ, Huynh HQ.. Is top-down therapy a more effective alternative to conventional step-up therapy for Crohn’s disease? Ann Gastroenterol. 2018;31(4):413–424. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Data are not publicly available.