Machine learning for prediction in secondary hemophagocytic lymphohistiocytosis: real progress, real limits, and the potential of synthetic data

Secondary hemophagocytic lymphohistiocytosis (sHLH) is a rare syndrome associated with high mortality, which is difficult to diagnose and prognosticate, and challenging to investigate due to its rarity and heterogeneity [1–4]. The predictive modeling presented by Ruzicka et al. in a recent issue of Intensive Care Medicine represents a serious and methodologically rigorous attempt to address the prognostic gap in this condition using machine learning (ML) [5]. The strengths and limitations of this work illuminate a broader problem faced by the critical care community: when the disease is rare enough, the data itself becomes the bottleneck.

What the study achieves

Ruzicka et al. assembled 167 adult sHLH patients, spanning 15 years, from 6 centers across three European countries in a valuable multicentric effort. The models predict both Initial Disease Severity (IDS), defined as intensive care unit (ICU) admission or early death, and time-point specific mortality at 30, 60, 90, 180, and 365 days, achieving strong discriminatory performance (area under the curve (AUC) 0.845 for IDS; mean AUC 0.882 for mortality), calibration, and interpretable feature importance. This could broaden the prognostic options available in the field of sHLH, complementing existing diagnostic (HLH-94/HLH-2024 [6–9]; HScore [10]) and single-time-point mortality prediction tools [11].

The authors’ methodological choices could prove useful beyond the walls of ICUs to improve shared decision-making. First, the composite IDS endpoint, encapsulating ICU admission and death < 90 days without ICU admission, addresses a clinically pertinent question at the bedside: is a patient on a trajectory of severe disease requiring immediate escalation of care, whether that escalation takes the form of ICU transfer, initiation of intensive immunosuppression, or early referral to a tertiary center. In this sense, IDS is presented as an early severity signal. Nevertheless, the authors emphasize that IDS is a “marker of an adverse clinical course rather than as a direct ICU triage tool” and that “model performance for the composite endpoint should not be extrapolated to the individual endpoint components”. In a disease where the time-to-treatment is paramount, such a tool is valuable only if it guides an action. Defining such actions is one important topic for future research. Second, the authors chose to use only objective laboratory variables, deliberately excluding clinician-interpreted variables. The omission of the underlying sHLH trigger, for example, could have led to a loss of valuable information, but is well justified by the frequent uncertainty regarding its identification at the time of diagnosis.

Importantly, sIL-2R emerged as a key predictor of both IDS and mortality but was available in only 40% of the sample. The prominence of sIL-2R in feature importance analyses should be interpreted alongside the sensitivity analyses, which show only a modest AUC reduction when sIL-2R is excluded. This apparent tension reflects a property of ensemble models trained on correlated clinical variables: a single biomarker may be the most distinctive contributor to the model’s decision process while remaining partially redundant with its covariates. Thus, the HLH-Risk-Calculator retains meaningful predictive performance in centers where sIL-2R is unavailable, while sIL-2R adds incremental discriminative value where it can be measured. The high missingness of sIL-2R in the training cohort means that both its feature importance and the magnitude of its marginal contribution to the model could be underestimated. There is, however, a risk of bias in the study in that sIL-2R availability might correlate with disease severity. The true predictive gain from systematically available, high-resolution sIL-2R measurements remains to be established in future prospective cohorts.

Where the bottleneck lies and a potential path forward

After excluding patients with excessive missingness, 152 patients remained. The holdout test set for mortality contained 43 patients, and for IDS, 32 patients. Model hyperparameters were explicitly constrained to prevent overfitting. In sHLH, as in most rare critical illness syndromes, the scarcity of high-quality patient data is the real bottleneck.

Generative synthetic data augmentation is a methodological paradigm gaining traction in rare disease research. Its principle is to train a deep generative model on large amounts of representative real patient data, learn the underlying multidimensional correlations between the characteristics of the clinical population, and generate additional synthetic patients that reproduce the distributional properties of the real cohort, including non-linear correlations, without duplicating individual patients [12–14]. Applied to a cohort like that of Ruzicka et al., such an approach could serve two distinct purposes. First, training set augmentation: generating several hundred synthetic patients to supplement the real training set, allowing random forest models to explore deeper feature interactions without the aggressive regularization constraints currently required. Second, class rebalancing: in a six-center dataset where contribution sizes are unequal, synthetic generation conditioned on smaller-center patient characteristics could homogenize center contributions, addressing the site-specific heterogeneity identified by the authors. Both fidelity (statistical similarity to real patients) and utility (performance of models trained on synthetic-augmented data, tested on real patients) of synthetic data are required, justifying rigorous evaluation based on statistical metrics and human expert judgment. A complementary approach would be to leverage new statistical ideas that enable valid statistical inference even when some of the data originates from a potentially inaccurate ML model [15].

sHLH is one instance of a broader category of rare critical illness syndromes with slow accumulation of clinical data despite multicentric efforts. Ruzicka et al. have possibly built the most comprehensive prognostic tool for sHLH to date. Biologically plausible synthetic data augmentation may represent the next methodological frontier for prognostic modeling in rare critical illness syndromes.

Clinical relevance for sHLH

The authors are to be congratulated on developing this HLH-Risk-Calculator. External validation in independent cohorts remains essential to assess the models’ real-world applicability, and the authors’ decision to make the HLH-Risk-Calculator publicly available for research purposes facilitates this important validation. Ultimately, if external validation supports the reported findings, clinicians will have a valuable tool to predict Initial Disease Severity (IDS) and time-point-specific mortality in sHLH, which may enable both better guidance of clinical decisions and prognosis-based stratification of patients for treatment trials.

Data availability statement

Not applicable.

Declarations

Conflicts of interest

The authors declare that they have no conflict of interest.