Machine Learning in a Complex Disease: PREsTo Improves the Prognostication of Primary Sclerosing Cholangitis

Despite continued efforts, effective therapy for primary sclerosing cholangitis (PSC) has not yet emerged. This is due to the challenges posed by a rare, heterogeneous disease that includes biliary fibro-inflammation alongside inflammatory bowel disease. Additional challenges include suboptimal risk assessment, particularly at the individual level.

In this issue, Eaton et al. have made a major contribution to address this challenge with a novel prognostic system developed through Machine Learning (ML) technology.⁽¹⁾ The Primary Sclerosing Cholangitis Risk Estimate Tool (PREsTo), developed at a referral center in the United States and validated internationally, significantly outperforms current prediction scoring systems with a C statistic of at least 0.90. The application of ML to the prediction of PSC outcomes represents a welcome addition to the field.

A rapidly expanding discipline, ML uses artificial intelligence computational tools to “learn” from data, such that performance in executing a specific task improves with experience.⁽²⁾ This is particularly relevant for problems using large, complex, and heterogeneous data sets, and for decisions requiring a personalized assessment. Within hepatology, emerging ML-generated tools are better than conventional tools in predicting outcomes for patients with severe alcoholic hepatitis⁽³⁾ and chronic hepatitis C infection.⁽⁴⁾

This study applies ML to a cholestatic disease, and its use in PSC is particularly well-suited given the need to integrate heterogeneous, often fluctuating, variables into a single score. The authors should be commended for selecting hepatic decompensation (ascites, variceal hemorrhage, and hepatic encephalopathy) as the clinical outcome predicted by this score and not liver transplantation, given the wide variation in criteria for listing and transplantation within the United States and around the world. PREsTo contains nine variables: bilirubin, albumin, serum alkaline phosphatase (SAP) times the upper limit of normal, platelets, aspartate aminotransferase, hemoglobin, serum sodium, patient age, and the number of years since PSC diagnosis. Given the innovative application of ML to PSC prognostication, it is useful to highlight several distinguishing features of PREsTo.

One result of the ML approach to PSC prognostication is a recalibrated regard for the weight of SAP. Most scoring systems and expert opinion⁽⁵⁾ regard SAP as a biologically relevant indicator of cholestatic damage in PSC, and therefore a central variable in prognostication and as a surrogate endpoint.⁽⁶⁾ Although SAP is still included in the new tool, ML-generated PREsTo clearly identified bilirubin as a more significant prognostic variable. Persistent bilirubin elevation is typically a finding of advanced disease and therefore less relevant for earlier stages, and SAP remains an important indicator of progression risk for patients with mild disease. However, the ranked list of prognostic variables generated through ML for PREsTo is a reminder that SAP, taken in isolation, likely has a modest role in predicting clinical outcomes such as hepatic decompensation.

Another relevant variable incorporated into PREsTo is the number of years since the diagnosis of PSC. Although it has less relative weight compared with bilirubin and SAP, the inclusion of this variable highlights the increasing dilemma of patients diagnosed at very early stages, often asymptomatic and with little or no liver fibrosis. Recent studies suggesting a higher than expected prevalence of PSC, based on strictures on magnetic resonance imaging in patients with inflammatory bowel disease and normal liver enzymes,⁽⁷⁾ point to the question of optimally determining the transition from preclinical to clinical PSC and the impact of this decision on prognostication. In addition, the geographic distance between patients and medical centers with PSC expertise may influence the timing of diagnosis, even in patients with clinical manifestations. The inclusion of time to diagnosis into PREsTo should prompt studies in a variety of geographic regions, and in diverse populations, to better evaluate its impact on risk stratification and clinical management.

It is important for clinicians and patients to recognize that PREsTo was not been tested in several relevant PSC subgroups, including pediatrics, small-duct disease, overlapping autoimmune hepatitis, or in advanced disease with manifestations of portal hypertension. Therefore, the prognostic value of PREsTo in these groups is currently unknown. The recent demonstration of serum gamma glutamyltransferase as a risk-stratification variable in pediatric PSC highlights the possibility that ML-based scoring systems for children could be developed.⁽⁸⁾

In addition to PREsTo, other scores have been recently proposed, including the Amsterdam-Oxford model (AOM) and the UK-PSC risk score.^(9,10) The AOM, which predicts PSC-related death or liver transplantation, was developed and internationally validated with recent work highlighting an incremental improvement in the model’s performance over time (such as at 5 years after diagnosis).⁽¹¹⁾ Despite this, the C statistic of 0.67 at the time of diagnosis was relatively modest, and the AOM had less overall predictive ability compared with the Mayo risk score. The AOM performed best in those with a lower risk of events, suggesting that it may have increased utility in certain subgroups, and notably the AOM distinguishes between small and large duct PSC. The UK-PSC risk score, which predicts liver transplantation and all-cause mortality, includes a short-term (2-year) and a long-term (10-year) risk score, in which the latter includes clinical events (variceal bleeding). The C statistics for short-term and long-term risk (0.81 and 0.85) outperformed the Mayo risk score and the aspartate aminotransferase–to-platelet ratio index. However, neither the UK-PSC and the AOM are designed to predict hepatic decompensation. Although PREsTo’s impressive C statistic, which crosses 0.9, is significantly higher than these scores, it will be highly informative to directly compare these tools in a large, contemporaneous cohort of patients with varying disease severity and disease duration.

PREsTo was developed to improve prognostication, for individual patient self-assessment through an online calculator, and potentially for future clinical decision making. The gradient boosting technique used to develop PREsTo is robust, and this methodology was used recently to develop a novel risk score for individual-level upper gastrointestinal bleeding risk assessment that outperforms existing prediction tools.⁽¹²⁾ It is relevant to note that regulatory frameworks recently developed by the Food and Drug Administration will increasingly regard clinical tool software as medical devices and therefore be subject to regulatory oversight.⁽¹³⁾ The PREsTo study demonstrates a strong experimental design with clear delineation between patients in the training data set and the cohort of patients in the validation data set. However, despite describing the flexibility of gradient tree boosting and the use of imputation in handling missing variables, the absence of data remains a challenge when applying ML-based tools to patients over time. Prospective testing of PREsTo in real-world settings, and in clinical sets with varying levels of missing data, will be helpful in further determining its role in individual-level prognostication and management.

The development of PREsTo represents a landmark improvement in the assessment and prognostication of patients with PSC beyond other available models, and powerfully exemplifies the increasing role of ML in complex diseases. It is hoped that the addition of PREsTo to the PSC armamentarium will add much-needed insight to the risk assessment of individual patients, and ultimately promote clarity in the development of effective therapies.