Mortality prediction in immune TTP remains challenging despite artificial intelligence advances

TO THE EDITOR:

Advances in the diagnosis and treatment of immune thrombotic thrombocytopenic purpura (iTTP) have significantly improved survival outcomes.^1,2 However, contemporary iTTP mortality rates remain ∼5% to 10%.^1,3, 4, 5, 6 Predicting iTTP mortality risk has been of interest, with the goal of identifying patients who may potentially benefit from intensified therapy and monitoring as a means to improve survival. Proposed approaches for those deemed at high risk include critical care–level monitoring, twice-daily therapeutic plasma exchange,6, 7, 8 earlier rituximab, or more aggressive immunosuppression, such as pulse cyclophosphamide.⁶ Caplacizumab is recommended by the International Society on Thrombosis and Haemostasis (ISTH) guidelines as upfront therapy for all iTTP^9,10 because it may confer a survival benefit.^2,11,12 However, because of its cost, questions have been raised as to whether it should be reserved for patients with “severe” disease.¹³ Given that there is currently no universally accepted definition of severe disease, a high mortality risk may potentially be considered to reflect severe disease.

In 2012, the French Thrombotic Microangiopathy (TMA) reference score was developed as a predictive model for acute TTP mortality and validated in the French TMA registry.¹⁴ In 2021, our group evaluated its performance in the United States Thrombotic Microangiopathy (USTMA) registry, a large US database with demographic and outcome data on 771 patients with iTTP from 15 high-volume TTP referral centers between 1985 and 2019. In an analysis of 419 patients, the French TMA reference score demonstrated poor discrimination, with an area under the receiver operating curve (AUC) of 0.63 (95% confidence interval [CI], 0.50-0.77).^15,16 We subsequently developed a mortality prediction model using the least absolute shrinkage and selection operator regression. However, that model also demonstrated poor performance after crossvalidation, with an AUC of 0.69 (95% CI, 0.58-0.78).¹⁶

Given the inadequate performance of both the French TMA reference score and our initial logistic regression model in predicting iTTP mortality in the USTMA data set, our group then explored a machine learning approach in 2024. For a rare, complex disease such as iTTP, machine learning was a promising approach because of its ability to incorporate a large number of predictors, model variables continuously, and capture complex, nonlinear interactions between them in a large data set such as ours.

Using gradient boosting machine and the USTMA registry as a training data set, our group developed a novel prediction tool in 2024: the USTMA Mortality Index.¹⁵ It incorporates seven variables and generates mortality predictions as a continuous percentage risk, which was stratified into different risk categories. In repeated 10-fold crossvalidation, it demonstrated good discrimination in predicting iTTP mortality, yielding an AUC of 0.77 (95% CI, 0.71-0.82), which represented a notable improvement in the previous two models. Although it lacked sensitivity, it demonstrated good specificity. It could not “rule out” death, but it was able to identify those at high risk of death who could be considered for more intensive approaches. Because the model had not yet been externally validated, we sought to evaluate its performance in an independent external validation cohort.

In this study, we externally validated the model using a cohort of patients with confirmed iTTP from 2010 to 2025 from the following five institutions: The University of Utah, Case Western Reserve University, Rochester Regional Health, University of Illinois, and Mayo Clinic (Rochester, MN). Patients in this validation cohort were not part of the training data set (the USTMA registry). Acute iTTP mortality was defined as death within 30 days of admission. Variables collected at initial presentation included the following (in order of highest to lowest relative influence): lactate dehydrogenase, serum creatinine, age, hemoglobin, platelet count, presence of stupor/coma, and presence of fever.

Model performance was assessed using AUC. We calculated sensitivity, specificity, positive predictive value, and negative predictive value based on the previously defined cutoffs of 4.27% and 9.08% mortality risk in the original model, which categorized patients into the following three categories: standard risk, high risk, and very high risk of predicted mortality.

The external validation cohort included 103 patients with confirmed iTTP. Six patients were missing one variable, for which the model is still capable of generating predictions.¹⁵ There were 11 deaths (10.7% mortality rate). Patient characteristics are listed in Table 1. The USTMA Mortality Index yielded an AUC of 0.69 (95% CI, 0.55-0.83) in the validation cohort, corresponding to poor discriminative performance. After excluding patients who received caplacizumab (a treatment that none of the patients in the training data set received), the AUC was 0.71 (95% CI, 0.57-0.85) among the remaining 88 patients, only slightly above the 0.70 threshold for acceptable performance. The sensitivity, specificity, positive predictive value, and negative predictive value metrics are presented in Table 2 at the two preselected cutoffs.

Table 1.

Patient characteristics stratified by survival status

USTMA Mortality Index variable	Total (N = 103)	Alive (n = 92)	Dead (n = 11)
Age, median (IQR), y	45 (34-60)	44 (34-58)	56 (44-63)
LDH, median (IQR), U/L	779 (529-1426)	779 (536-1256)	1433 (544-2525)
Creatinine, median (IQR), mg/dL	1.2 (0.90-1.6)	1.1 (0.84-1.5)	1.6 (1.4-2.0)
Hemoglobin, median (IQR), g/dL	8.7 (7.2-10)	8.8 (7.5-10)	8.1 (6.8-9.0)
Platelet count, median (IQR), × 109/L	15 (9.0-24)	15 (9.8-23)	18 (7.5-26)
Stupor or coma, n (%)	19 (18)	13 (14)	6 (55)
Fever, n (%)	14 (14)	10 (11)	4 (36)
Predicted mortality risk by model, median (IQR), %	3 (2-6)	3 (2-5)	5 (3-8)

IQR, interquartile range; LDH, lactate dehydrogenase.

Table 2.

Performance metrics of the USTMA Mortality Index per risk cutoff in the validation cohort (caplacizumab recipients excluded)

Predicted mortality risk cutoff	Sensitivity, %	Specificity, %	PPV, %	NPV, %
Very high risk (≥9.08%)	18	88	18	88
High risk (≥4.27%)	64	71	24	93

NPV, negative predictive value; PPV, positive predictive value.

In this first external validation, the USTMA Mortality Index did not demonstrate reliable discrimination in predicting iTTP mortality and failed to reproduce its previously favorable performance in the training data set. The wide CI of the AUC (0.55-0.83), ranging from failed to good discrimination, reflects substantial uncertainty in model performance. This imprecision likely resulted from the small number of deaths (11/103) in the validation cohort, a significant limitation in determining model validity.

Several reasons likely contributed to the model’s poor performance, which may provide valuable lessons for future development of machine learning models in rare hematologic diseases. Overfitting, a common challenge in artificial intelligence–based clinical prediction models, is particularly difficult to avoid in rare diseases with limited outcome events, where models may inadvertently learn cohort-specific patterns that fail to generalize to external populations. The low number of fatal outcomes in the original training data set (24 deaths [5.7%]) likely hindered the model’s robustness, increasing susceptibility to overfitting. Another significant weakness of the model is that it included patients from several decades, during which mortality had significantly decreased with time,¹ thus representing heterogeneous eras of clinical practice. The limited number of fatal outcomes necessitated the inclusion of patients from earlier decades, a limitation that may potentially only be avoided in future data sets by including much larger cohorts of contemporary patients. It is also possible that the data set may not have captured the most optimal predictive variables.

However, another likely explanation is that mortality due to iTTP may simply be too difficult to predict regardless of methodology. The outcome of death is affected by not only heterogeneous disease biology and physiologic factors but also therapeutic variables, such as time to treatment, which was not incorporated in the training data set. This explanation is further supported by the contrasting findings of our group’s ongoing work developing TTP-MD (formerly TTP-14), a separate machine learning model that can predict early diagnosis of iTTP. Unlike the USTMA Mortality Index, TTP-MD maintained excellent performance on external validation, generating accurate predictions even with missing variables.¹⁷ This difference suggests that diagnosis as a target outcome may be more feasible to predict than death, although differences in training data set design may have played a role. Considerations for future models that aim to predict iTTP mortality would likely require improved methodology and data set design, for which our aforementioned weaknesses may provide useful guidance toward refinement before reattempting machine learning mortality prediction. Nevertheless, our collective experience suggests that efforts at improving iTTP outcomes may be more successful if focused on strategies that enhance early recognition rather than mortality prediction itself.

In conclusion, given that neither the 2012 French TMA reference score, nor our group’s 2021 logistic regression model, nor the 2024 USTMA Mortality Index could reliably predict mortality on external validation in our cohorts, these findings collectively suggest that mortality in acute iTTP remains difficult to predict at this time, regardless of statistical methodology or current machine learning capabilities. We further conclude that clinical decision-making, including the use of therapies such as caplacizumab, should not currently be guided by mortality risk predictions. Instead, we support adherence to the standardized ISTH guidelines that recommend upfront caplacizumab use for all patients with iTTP, unless contraindicated because of bleeding. Rather than relying on predicted risk stratification, efforts to reduce mortality and improve outcomes should prioritize improving early recognition, diagnosis, and treatment of iTTP, providing optimal access to health care resources, and enhancing clinical expertise in iTTP management. All patients with iTTP should be considered at high risk of death.

This study was approved by the institutional review board and conducted in accordance with the Declaration of Helsinki.

Conflict-of-interest disclosure: M.Y.A.-I. has received honoraria from Sanofi, Takeda, and Genentech. M.Y.L. has received honoraria from Sanofi, BioMarin, Bayer, Hema Biologics, and Takeda. The remaining authors declare no competing financial interests.